From 4e431d28b8a463be50770fc1ccfdc87c8439c2ef Mon Sep 17 00:00:00 2001 From: Andrew Altshuler Date: Sat, 30 May 2026 00:53:42 +0100 Subject: [PATCH 001/165] docs: rewrite README opening + add AGENTS.md dev commands (#122) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs(agents): add build/test/lint dev-command section AGENTS.md (CLAUDE.md) covered architecture and invariants but had no developer command surface — only runtime `omnigraph` CLI usage. Add a concise "Build, test, lint" section with the non-obvious gotchas: - crate dir `crates/omnigraph` is package `omnigraph-engine` (the `-p` name) - canonical CI gate is `cargo test --workspace --locked` - how to run one file / one fn - feature-gated suites (`failpoints`, server `aws`) - S3 tests skip without `OMNIGRAPH_S3_TEST_BUCKET` - the two non-test CI checks (check-agents-md, OpenAPI drift) Co-Authored-By: Claude Opus 4.8 (1M context) * docs(readme): rewrite opening, dedupe, fix stale references - New manifesto-style opening (tagline, X-as-code, features, core use cases, coordination-layer line); drop the old prose intro, Use Cases, and Capabilities sections. - Remove Capabilities, which restated the new opening line-for-line. - Harmonize heading case: "## Core Use Cases". - Dedupe the verbatim Slack invite (kept the Community section) and the double-linked cli.md (kept the contextual pointer). - Fix stale references that no longer match the code: - drop "transactional runs" / "and runs" — no run concept remains; writes are atomic per-query, multi-query workflows use branches. - update the CLI crate command list to canonical query/mutate plus commit/lint/optimize/cleanup. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) --- AGENTS.md | 26 ++++++++++++++++++++++++++ README.md | 45 ++++++++++++++++++++------------------------- 2 files changed, 46 insertions(+), 25 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 27d1b7b..3fc78f7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -164,6 +164,32 @@ If a proposal fits one of these, the burden is on the proposer to justify why th --- +## Build, test, lint + +Rust stable workspace (edition 2024). `protoc` is a build dependency (`brew install protobuf` / `apt-get install protobuf-compiler libprotobuf-dev`). **Crate dir ≠ package name** for the engine: the directory is `crates/omnigraph` but its Cargo package is `omnigraph-engine` (use that in `-p`). The CLI binary built from `omnigraph-cli` is named `omnigraph`. + +```bash +cargo build --workspace --locked # build everything +cargo test --workspace --locked # the canonical CI gate (matches CI exactly) +cargo run -p omnigraph-cli -- # run the `omnigraph` CLI from source +cargo run -p omnigraph-server -- --bind 0.0.0.0:8080 # run the server from source + +# Run one crate / one test file / one test fn +cargo test -p omnigraph-engine --test traversal # one integration-test file (see docs/dev/testing.md) +cargo test -p omnigraph-engine --test runs concurrent # one test fn by name substring +cargo test -p omnigraph-engine some_inline_test -- --nocapture # show stdout + +# Feature-gated suites (each is its own job in CI, not part of the default run) +cargo test -p omnigraph-engine --features failpoints --test failpoints # fault injection +cargo build -p omnigraph-server --features aws # AWS Secrets Manager bearer-token source +``` + +S3-backed tests (`s3_storage`, and the S3 paths in server/CLI system tests) **skip** unless `OMNIGRAPH_S3_TEST_BUCKET` + `AWS_*` (incl. `AWS_ENDPOINT_URL_S3` for non-AWS) are set; CI runs them against containerized RustFS. `scripts/local-rustfs-bootstrap.sh` stands up a local S3 environment. + +CI does **not** run `clippy` or `rustfmt` as gates — but `cargo test --workspace --locked` is the exact gate, so run it before pushing. Two non-test CI checks: `scripts/check-agents-md.sh` (doc cross-link integrity — run it after moving/renaming docs) and OpenAPI drift (`crates/omnigraph-server/tests/openapi.rs` regenerates `openapi.json`; set `OMNIGRAPH_UPDATE_OPENAPI=1` to update the checked-in copy when a server/API change is intentional). + +--- + ## Quick-reference flows ```bash diff --git a/README.md b/README.md index ae3234b..5291580 100644 --- a/README.md +++ b/README.md @@ -5,33 +5,29 @@ [![Crates.io](https://img.shields.io/crates/v/omnigraph-cli.svg)](https://crates.io/crates/omnigraph-cli) [![CI](https://github.com/ModernRelay/omnigraph/actions/workflows/ci.yml/badge.svg)](https://github.com/ModernRelay/omnigraph/actions/workflows/ci.yml) -**Object-storage native knowledge graph with git-style workflows. Designed for agents and humans to collaborate on shared structured knowledge.** +**Lakehouse native graph engine built for context assembly** -Turns fragmented context into a live graph, lets humans and agents coordinate through that graph, and uses branches so agent-generated changes can be reviewed and merged safely. +Schema AS CODE +Context AS CODE +Security AS CODE +Dashboards AS CODE -Built on Rust, Arrow, DataFusion and Lance. +Git-style snapshots & branching +Object storage native (S3, RustFS) +VPC, On-prem, hybrid deployment +Lance format as storage layer -Join the [Omnigraph Slack community](https://join.slack.com/t/omnigraphworkspace/shared_invite/zt-3wfpglyxj-lHvJGhuySPfqLtN35uJZNw) +## Core Use Cases -## Use Cases - -- Company brain / [Second brain](https://github.com/ModernRelay/omnigraph-cookbooks/tree/main/second-brain) +- Company brain - Context graph -- Knowledge base for multi-agent research -- Incident response graph -- Compliance & audit graph +- Agentic memory +- Code & dev graph +- R&D data layer +- ML workflows +- Karpathy's LLM wiki - -## Capabilities - -- Typed schema, typed queries, and typed mutations -- Native blob-as-data support (docs, images, videos, etc) -- Schema-as-code, query validation and linting -- Git-style graph workflows: branches, commits, merges, and transactional runs -- Local, on-prem & cloud S3-native storage with snapshot-pinned reads -- Graph traversal + text, fuzzy, BM25, vector, and RRF search in one runtime -- Policy-as-code for server-side access control -- Single CLI for multiple deployments +Omnigraph acts as operational state & coordination layer for agents ## Quick Install @@ -86,12 +82,11 @@ omnigraph branch create --from main feature-x ./graph.omni omnigraph branch merge feature-x --into main ./graph.omni ``` -See [docs/user/cli.md](docs/user/cli.md) for schema apply, snapshots, ingest, runs, and policy commands. +See [docs/user/cli.md](docs/user/cli.md) for schema apply, snapshots, ingest, commits, and policy commands. ## Docs - [Install guide](docs/user/install.md) -- [CLI guide](docs/user/cli.md) - [Deployment guide](docs/user/deployment.md) ## Build And Test @@ -113,8 +108,8 @@ Notes: - `crates/omnigraph-compiler`: shared schema/query parser, typechecker, catalog, and IR lowering - `crates/omnigraph`: storage/runtime, branching, merge, change detection, and query execution -- `crates/omnigraph-cli`: CLI for init/load/ingest/read/change/branch/snapshot/export/policy operations -- `crates/omnigraph-server`: Axum HTTP server for remote reads, changes, ingest, export, branches, commits, and runs +- `crates/omnigraph-cli`: CLI for graph lifecycle (init/load/ingest), query/mutate, branch/commit/merge, schema/lint, snapshot/export, policy, and maintenance (optimize/cleanup) +- `crates/omnigraph-server`: Axum HTTP server for remote reads, changes, ingest, export, branches, and commits ## Contributing From baeb4387df52ab7e07dc996da24a40aeda6336c9 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler Date: Sat, 30 May 2026 01:02:41 +0100 Subject: [PATCH 002/165] docs(readme): hard-break the manifesto lines with
(#123) The X-as-code and feature blocks used bare single newlines, which collapse onto one line in renderers that treat soft breaks as spaces. Append
to each line so they break everywhere without converting the flat manifesto into bullet lists. Co-authored-by: Claude Opus 4.8 (1M context) --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 5291580..7ad3810 100644 --- a/README.md +++ b/README.md @@ -7,14 +7,14 @@ **Lakehouse native graph engine built for context assembly** -Schema AS CODE -Context AS CODE -Security AS CODE +Schema AS CODE
+Context AS CODE
+Security AS CODE
Dashboards AS CODE -Git-style snapshots & branching -Object storage native (S3, RustFS) -VPC, On-prem, hybrid deployment +Git-style snapshots & branching
+Object storage native (S3, RustFS)
+VPC, On-prem, hybrid deployment
Lance format as storage layer ## Core Use Cases From 6e948de9856205b0b5f200399afbcb82171c7206 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler Date: Sat, 30 May 2026 01:09:53 +0100 Subject: [PATCH 003/165] Update README.md (#124) --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 7ad3810..9397809 100644 --- a/README.md +++ b/README.md @@ -12,10 +12,10 @@ Context AS CODE
Security AS CODE
Dashboards AS CODE -Git-style snapshots & branching
-Object storage native (S3, RustFS)
-VPC, On-prem, hybrid deployment
-Lance format as storage layer +- Git-style snapshots & branching
+- Object storage native (S3, RustFS)
+- VPC, On-prem, hybrid deployment
+- Lance format as storage layer ## Core Use Cases From c221e67e1bc2e281fe2c08fe7e61311234ebeb4e Mon Sep 17 00:00:00 2001 From: Andrew Altshuler Date: Sat, 30 May 2026 10:05:30 +0100 Subject: [PATCH 004/165] docs(readme): restructure hero with bullets + tables (#125) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs(readme): restructure hero with bullets + tables Establish a reading rhythm: infra facts as quick bullets, then an AS CODE table and a Core Use Cases table that unpack each item. - Feature block → bullets (drop the redundant
left on list items). - AS CODE block → table; "Context as code" reframed around pre-defined, versioned queries. - Use cases → table with one-line unpacking; "Code & dev graph" → "Dev graph" (issue + dependency model for AI agents); "Context graph" reframed to decision traces + tribal knowledge; "R&D data layer" to experimental data written into branches; wiki line to "agent-updatable knowledge base". Co-Authored-By: Claude Opus 4.8 (1M context) * docs(readme): refine hero copy and lift thesis line - Promote "operational state & coordination layer for agents" to sit directly under the tagline. - Tighten AS CODE and use-case descriptions. Co-Authored-By: Claude Opus 4.8 (1M context) * docs(readme): make AS CODE rows consistent Apply the AS CODE motif to all four rows (Security, Dashboards) to match Schema and Context. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) --- README.md | 36 ++++++++++++++++++++---------------- 1 file changed, 20 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index 9397809..89464da 100644 --- a/README.md +++ b/README.md @@ -7,27 +7,31 @@ **Lakehouse native graph engine built for context assembly** -Schema AS CODE
-Context AS CODE
-Security AS CODE
-Dashboards AS CODE +Omnigraph acts as operational state & coordination layer for agents -- Git-style snapshots & branching
-- Object storage native (S3, RustFS)
-- VPC, On-prem, hybrid deployment
+- Git-style snapshots & branching +- Object storage native (S3, RustFS) +- VPC, On-prem, hybrid deployment - Lance format as storage layer +| AS CODE | | +|---|---| +| **Schema AS CODE** | Typed `.pg` schemas, planned, applied, enforced | +| **Context AS CODE** | Linted queries & agentic nudges, versioned and reusable | +| **Security AS CODE** | Cedar policies enforced server-side on every mutation | +| **Dashboards AS CODE** | Declarative views & controls over the graph *(coming)* | + ## Core Use Cases -- Company brain -- Context graph -- Agentic memory -- Code & dev graph -- R&D data layer -- ML workflows -- Karpathy's LLM wiki - -Omnigraph acts as operational state & coordination layer for agents +| Use case | | +|---|---| +| **Company brain** | Org knowledge unified into one queryable graph | +| **Context graph** | Decision traces and codified tribal knowledge | +| **Agentic memory** | Durable, versioned memory for long-running agents | +| **Dev graph** | Issues & dependency model for coding agents | +| **R&D data layer** | Experiments & trials data written into branches | +| **ML workflows** | Versioned, branchable graphs for training & eval | +| **Karpathy's LLM wiki** | A living, agent-updatable knowledge base | ## Quick Install From d8451c3d33a69cb66a615db515c0956c98152b51 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler Date: Sat, 30 May 2026 10:08:14 +0100 Subject: [PATCH 005/165] docs(readme): add header labels to hero tables (#126) Fill the empty second-column header cell in both tables: "What it means" for the AS CODE table, "What it's for" for Core Use Cases. Co-authored-by: Claude Opus 4.8 (1M context) --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 89464da..8ae8af0 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ Omnigraph acts as operational state & coordination layer for agents - VPC, On-prem, hybrid deployment - Lance format as storage layer -| AS CODE | | +| AS CODE | What it means | |---|---| | **Schema AS CODE** | Typed `.pg` schemas, planned, applied, enforced | | **Context AS CODE** | Linted queries & agentic nudges, versioned and reusable | @@ -23,7 +23,7 @@ Omnigraph acts as operational state & coordination layer for agents ## Core Use Cases -| Use case | | +| Use case | What it's for | |---|---| | **Company brain** | Org knowledge unified into one queryable graph | | **Context graph** | Decision traces and codified tribal knowledge | From 6c9afc7e9bae29f0a7c02d2bc4c0ef5da444bd01 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sat, 30 May 2026 12:45:26 +0200 Subject: [PATCH 006/165] Fix typos and enhance README content --- README.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 8ae8af0..946297c 100644 --- a/README.md +++ b/README.md @@ -9,10 +9,12 @@ Omnigraph acts as operational state & coordination layer for agents -- Git-style snapshots & branching +- Git-style versioning & branching +- Multimodal retrieval (graph+vector/fts+filters) optimized for context assembly - Object storage native (S3, RustFS) +- Native blob-as-data support (docs, images, videos, etc) - VPC, On-prem, hybrid deployment -- Lance format as storage layer +- [`Lance`](https://github.com/lance-format/lance) format as open storage layer | AS CODE | What it means | |---|---| @@ -23,9 +25,9 @@ Omnigraph acts as operational state & coordination layer for agents ## Core Use Cases -| Use case | What it's for | +| Use case | What it's for |---|---| -| **Company brain** | Org knowledge unified into one queryable graph | +| **Company brain** | Org knowledge unified into one queryable graph | | **Context graph** | Decision traces and codified tribal knowledge | | **Agentic memory** | Durable, versioned memory for long-running agents | | **Dev graph** | Issues & dependency model for coding agents | From 24413844ae1c7b9b24b69f452e76cf73da662c34 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sat, 30 May 2026 14:23:40 +0200 Subject: [PATCH 007/165] Add Windows release binaries (#127) * Add Windows release binaries * Fix Windows installer downloads --- .github/workflows/ci.yml | 57 +++++++++++ .github/workflows/release-edge.yml | 46 ++++++++- .github/workflows/release.yml | 47 ++++++++- docs/dev/ci.md | 5 +- docs/user/deployment.md | 8 +- docs/user/install.md | 62 +++++++++++- scripts/install.ps1 | 151 +++++++++++++++++++++++++++++ scripts/local-rustfs-bootstrap.sh | 3 - 8 files changed, 364 insertions(+), 15 deletions(-) create mode 100644 scripts/install.ps1 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3dc2e80..918a472 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -249,6 +249,63 @@ jobs: if: needs.classify_changes.outputs.run_full_ci == 'true' run: cargo test --locked -p omnigraph-server --features aws + test_windows_binaries: + name: Test Windows release binaries + needs: classify_changes + runs-on: windows-latest + timeout-minutes: 75 + permissions: + contents: read + env: + CARGO_TERM_COLOR: always + steps: + - name: Skip for text-only changes + if: needs.classify_changes.outputs.run_full_ci != 'true' + run: Write-Host "Text-only change detected; skipping Windows binary build." + + - name: Checkout source + if: needs.classify_changes.outputs.run_full_ci == 'true' + uses: actions/checkout@v5.0.1 + + - name: Install system dependencies + if: needs.classify_changes.outputs.run_full_ci == 'true' + run: choco install protoc -y + + - name: Install Rust stable + if: needs.classify_changes.outputs.run_full_ci == 'true' + uses: dtolnay/rust-toolchain@stable + with: + toolchain: stable + + - name: Cache Rust build data + if: needs.classify_changes.outputs.run_full_ci == 'true' + uses: Swatinem/rust-cache@v2 + with: + workspaces: | + . -> target + key: windows-release-binaries + + - name: Build Windows binaries + if: needs.classify_changes.outputs.run_full_ci == 'true' + run: cargo build --release --locked -p omnigraph-cli -p omnigraph-server + + - name: Smoke test Windows binaries + if: needs.classify_changes.outputs.run_full_ci == 'true' + run: | + & ./target/release/omnigraph.exe version + & ./target/release/omnigraph-server.exe --help + + - name: Check PowerShell installer syntax + if: needs.classify_changes.outputs.run_full_ci == 'true' + run: | + $tokens = $null + $errors = $null + [System.Management.Automation.Language.Parser]::ParseFile("scripts/install.ps1", [ref]$tokens, [ref]$errors) | Out-Null + if ($errors.Count -gt 0) { + $errors | Format-List + exit 1 + } + rustfs_integration: name: RustFS S3 Integration needs: diff --git a/.github/workflows/release-edge.yml b/.github/workflows/release-edge.yml index 6147646..3996e65 100644 --- a/.github/workflows/release-edge.yml +++ b/.github/workflows/release-edge.yml @@ -43,6 +43,8 @@ jobs: asset_name: omnigraph-linux-x86_64 - runner: macos-14 asset_name: omnigraph-macos-arm64 + - runner: windows-latest + asset_name: omnigraph-windows-x86_64 env: CARGO_TERM_COLOR: always steps: @@ -59,6 +61,10 @@ jobs: if: runner.os == 'macOS' run: brew install protobuf + - name: Install Windows dependencies + if: runner.os == 'Windows' + run: choco install protoc -y + - name: Install Rust stable uses: dtolnay/rust-toolchain@stable with: @@ -73,7 +79,8 @@ jobs: - name: Build release binaries run: cargo build --release --locked -p omnigraph-cli -p omnigraph-server - - name: Package release archive + - name: Package Unix release archive + if: runner.os != 'Windows' run: | mkdir -p release install -m 0755 target/release/omnigraph release/omnigraph @@ -81,6 +88,22 @@ jobs: tar -C release -czf "${{ matrix.asset_name }}.tar.gz" omnigraph omnigraph-server shasum -a 256 "${{ matrix.asset_name }}.tar.gz" > "${{ matrix.asset_name }}.sha256" + - name: Package Windows release archive + if: runner.os == 'Windows' + run: | + New-Item -ItemType Directory -Force -Path release | Out-Null + Copy-Item target/release/omnigraph.exe release/omnigraph.exe + Copy-Item target/release/omnigraph-server.exe release/omnigraph-server.exe + Compress-Archive -Path release/omnigraph.exe, release/omnigraph-server.exe -DestinationPath "${{ matrix.asset_name }}.zip" -Force + $hash = (Get-FileHash "${{ matrix.asset_name }}.zip" -Algorithm SHA256).Hash.ToLowerInvariant() + "$hash ${{ matrix.asset_name }}.zip" | Out-File -FilePath "${{ matrix.asset_name }}.sha256" -Encoding ascii + New-Item -ItemType Directory -Force -Path verify | Out-Null + Expand-Archive -Path "${{ matrix.asset_name }}.zip" -DestinationPath verify -Force + $items = Get-ChildItem -Path verify -File + if ($items.Count -ne 2 -or !(Test-Path verify/omnigraph.exe) -or !(Test-Path verify/omnigraph-server.exe)) { + throw "Windows release archive is missing expected binaries" + } + - name: Publish edge release assets uses: softprops/action-gh-release@v2.5.0 with: @@ -91,5 +114,22 @@ jobs: body: | Rolling prerelease from `${{ github.sha }}`. files: | - ${{ matrix.asset_name }}.tar.gz - ${{ matrix.asset_name }}.sha256 + ${{ matrix.asset_name }}.* + + smoke_windows_installer: + name: Smoke Windows installer + needs: build_release + runs-on: windows-latest + permissions: + contents: read + steps: + - name: Checkout source + uses: actions/checkout@v5.0.1 + + - name: Install from edge release + run: ./scripts/install.ps1 -ReleaseChannel edge -InstallDir "$env:RUNNER_TEMP/omnigraph-bin" + + - name: Smoke installed binaries + run: | + & "$env:RUNNER_TEMP/omnigraph-bin/omnigraph.exe" version + & "$env:RUNNER_TEMP/omnigraph-bin/omnigraph-server.exe" --help diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e7fc75f..48ab38c 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -20,6 +20,8 @@ jobs: asset_name: omnigraph-linux-x86_64 - runner: macos-14 asset_name: omnigraph-macos-arm64 + - runner: windows-latest + asset_name: omnigraph-windows-x86_64 env: CARGO_TERM_COLOR: always steps: @@ -36,6 +38,10 @@ jobs: if: runner.os == 'macOS' run: brew install protobuf + - name: Install Windows dependencies + if: runner.os == 'Windows' + run: choco install protoc -y + - name: Install Rust stable uses: dtolnay/rust-toolchain@stable with: @@ -50,7 +56,8 @@ jobs: - name: Build release binaries run: cargo build --release --locked -p omnigraph-cli -p omnigraph-server - - name: Package release archive + - name: Package Unix release archive + if: runner.os != 'Windows' run: | mkdir -p release install -m 0755 target/release/omnigraph release/omnigraph @@ -58,12 +65,27 @@ jobs: tar -C release -czf "${{ matrix.asset_name }}.tar.gz" omnigraph omnigraph-server shasum -a 256 "${{ matrix.asset_name }}.tar.gz" > "${{ matrix.asset_name }}.sha256" + - name: Package Windows release archive + if: runner.os == 'Windows' + run: | + New-Item -ItemType Directory -Force -Path release | Out-Null + Copy-Item target/release/omnigraph.exe release/omnigraph.exe + Copy-Item target/release/omnigraph-server.exe release/omnigraph-server.exe + Compress-Archive -Path release/omnigraph.exe, release/omnigraph-server.exe -DestinationPath "${{ matrix.asset_name }}.zip" -Force + $hash = (Get-FileHash "${{ matrix.asset_name }}.zip" -Algorithm SHA256).Hash.ToLowerInvariant() + "$hash ${{ matrix.asset_name }}.zip" | Out-File -FilePath "${{ matrix.asset_name }}.sha256" -Encoding ascii + New-Item -ItemType Directory -Force -Path verify | Out-Null + Expand-Archive -Path "${{ matrix.asset_name }}.zip" -DestinationPath verify -Force + $items = Get-ChildItem -Path verify -File + if ($items.Count -ne 2 -or !(Test-Path verify/omnigraph.exe) -or !(Test-Path verify/omnigraph-server.exe)) { + throw "Windows release archive is missing expected binaries" + } + - name: Publish GitHub release assets uses: softprops/action-gh-release@v2.5.0 with: files: | - ${{ matrix.asset_name }}.tar.gz - ${{ matrix.asset_name }}.sha256 + ${{ matrix.asset_name }}.* update_homebrew_tap: name: Update Homebrew tap @@ -113,3 +135,22 @@ jobs: git add Formula/omnigraph.rb git commit -m "Update Omnigraph formula to ${GITHUB_REF_NAME}" git push origin HEAD:main + + smoke_windows_installer: + name: Smoke Windows installer + needs: build_release + if: startsWith(github.ref, 'refs/tags/v') + runs-on: windows-latest + permissions: + contents: read + steps: + - name: Checkout source + uses: actions/checkout@v5.0.1 + + - name: Install from tagged release + run: ./scripts/install.ps1 -Version "$env:GITHUB_REF_NAME" -InstallDir "$env:RUNNER_TEMP/omnigraph-bin" + + - name: Smoke installed binaries + run: | + & "$env:RUNNER_TEMP/omnigraph-bin/omnigraph.exe" version + & "$env:RUNNER_TEMP/omnigraph-bin/omnigraph-server.exe" --help diff --git a/docs/dev/ci.md b/docs/dev/ci.md index 8495d5e..1124cb4 100644 --- a/docs/dev/ci.md +++ b/docs/dev/ci.md @@ -4,7 +4,8 @@ - **ci.yml**: text-only changes skip; otherwise `cargo test --workspace --locked` on ubuntu-latest with protobuf compiler. OpenAPI-drift check that auto-commits the regenerated `openapi.json` for same-repository PRs. Also runs the AGENTS.md cross-link integrity check (`scripts/check-agents-md.sh`). - **AWS feature build job**: `cargo build/test -p omnigraph-server --features aws` on ubuntu-latest. +- **Windows binary build job**: `cargo build --release --locked -p omnigraph-cli -p omnigraph-server` on windows-latest with smoke checks for `omnigraph.exe version`, `omnigraph-server.exe --help`, and PowerShell installer syntax. - **RustFS S3 integration**: spins up RustFS in Docker, runs `s3_storage`, `server_opens_s3_graph_directly_and_serves_snapshot_and_read`, and `local_cli_s3_end_to_end_init_load_read_flow`. -- **release-edge.yml**: on every push to main, retags `edge`, builds Linux x86_64 / macOS arm64 archives + sha256, publishes a rolling prerelease. -- **release.yml**: on `v*` tags, builds the Linux x86_64 / macOS arm64 release matrix and updates the Homebrew tap (`scripts/update-homebrew-formula.sh`) by pushing the regenerated formula to `ModernRelay/homebrew-tap`. +- **release-edge.yml**: on every push to main, retags `edge`, builds Linux x86_64 / macOS arm64 archives and Windows x86_64 zip + sha256, publishes a rolling prerelease, then smoke-tests the Windows PowerShell installer against `edge`. +- **release.yml**: on `v*` tags, builds the Linux x86_64 / macOS arm64 archives and Windows x86_64 zip release matrix, updates the Homebrew tap (`scripts/update-homebrew-formula.sh`) by pushing the regenerated formula to `ModernRelay/homebrew-tap`, and smoke-tests the Windows PowerShell installer against the tag. - **package.yml**: manual ECR image build; emits two image tags per commit (``, `-aws`) via CodeBuild. diff --git a/docs/user/deployment.md b/docs/user/deployment.md index fc5ee08..8613dec 100644 --- a/docs/user/deployment.md +++ b/docs/user/deployment.md @@ -20,6 +20,8 @@ Build or install: - `omnigraph` - `omnigraph-server` +On Windows, the binaries are `omnigraph.exe` and `omnigraph-server.exe`. + Run against a local graph: ```bash @@ -141,8 +143,10 @@ The server binary ships in two flavors: | **AWS** | `cargo build --release --features aws` | Adds AWS Secrets Manager backend for bearer tokens | Tagged release archives contain the default `omnigraph` and -`omnigraph-server` binaries. AWS-enabled server binaries are built from source -with `cargo build --release --features aws -p omnigraph-server` when needed. +`omnigraph-server` binaries on macOS / Linux, and `omnigraph.exe` plus +`omnigraph-server.exe` on Windows. AWS-enabled server binaries are built from +source with `cargo build --release --features aws -p omnigraph-server` when +needed. The AWS build adds ~150 transitive deps and ~30-60s of first-build compile time. Default builds don't pay that cost. diff --git a/docs/user/install.md b/docs/user/install.md index ea9fb8c..4a11372 100644 --- a/docs/user/install.md +++ b/docs/user/install.md @@ -2,16 +2,29 @@ ## Quick Install +macOS / Linux: + ```bash curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | bash ``` +Windows PowerShell: + +```powershell +powershell -NoProfile -ExecutionPolicy Bypass -Command "iwr -UseBasicParsing https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.ps1 | iex" +``` + By default the installer places: - `omnigraph` - `omnigraph-server` -in `~/.local/bin`. +in `~/.local/bin` on macOS / Linux, or: + +- `omnigraph.exe` +- `omnigraph-server.exe` + +in `%USERPROFILE%\.local\bin` on Windows. The default installer is binary-only. It downloads a published release asset, verifies the SHA256 checksum, and unpacks it. It does not build from source. @@ -39,6 +52,13 @@ Rolling edge binaries from `main`: curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | RELEASE_CHANNEL=edge bash ``` +Windows rolling edge binaries: + +```powershell +iwr -UseBasicParsing https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.ps1 -OutFile install.ps1 +powershell -NoProfile -ExecutionPolicy Bypass -File .\install.ps1 -ReleaseChannel edge +``` + Install from source: ```bash @@ -53,12 +73,24 @@ Install to a different directory: curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | INSTALL_DIR="$HOME/bin" bash ``` +Windows: + +```powershell +powershell -NoProfile -ExecutionPolicy Bypass -File .\install.ps1 -InstallDir "$env:USERPROFILE\bin" +``` + Install a specific tag: ```bash curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | VERSION=v0.1.0 bash ``` +Windows: + +```powershell +powershell -NoProfile -ExecutionPolicy Bypass -File .\install.ps1 -Version v0.1.0 +``` + Build from a specific git ref: ```bash @@ -67,27 +99,53 @@ curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/ ## Manual Source Build +macOS / Linux: + ```bash cargo build --release --locked -p omnigraph-cli -p omnigraph-server install -m 0755 target/release/omnigraph ~/.local/bin/omnigraph install -m 0755 target/release/omnigraph-server ~/.local/bin/omnigraph-server ``` +Windows: + +```powershell +cargo build --release --locked -p omnigraph-cli -p omnigraph-server +New-Item -ItemType Directory -Force "$env:USERPROFILE\.local\bin" | Out-Null +Copy-Item target\release\omnigraph.exe "$env:USERPROFILE\.local\bin\omnigraph.exe" +Copy-Item target\release\omnigraph-server.exe "$env:USERPROFILE\.local\bin\omnigraph-server.exe" +``` + ## Release Assets Tagged releases are expected to publish: - `omnigraph-linux-x86_64.tar.gz` - `omnigraph-macos-arm64.tar.gz` +- `omnigraph-windows-x86_64.zip` -Each archive contains both binaries: +The macOS / Linux archives contain both binaries: - `omnigraph` - `omnigraph-server` +The Windows archive contains: + +- `omnigraph.exe` +- `omnigraph-server.exe` + ## Verify The Install +macOS / Linux: + ```bash omnigraph version omnigraph-server --help ``` + +Windows: + +```powershell +omnigraph.exe version +omnigraph-server.exe --help +``` diff --git a/scripts/install.ps1 b/scripts/install.ps1 new file mode 100644 index 0000000..3bfd0f1 --- /dev/null +++ b/scripts/install.ps1 @@ -0,0 +1,151 @@ +param( + [string]$RepoSlug = "ModernRelay/omnigraph", + [string]$InstallDir = "$env:USERPROFILE\.local\bin", + [ValidateSet("stable", "edge")] + [string]$ReleaseChannel = "stable", + [string]$Version = "" +) + +$ErrorActionPreference = "Stop" + +$assetName = "omnigraph-windows-x86_64.zip" +$assetStem = "omnigraph-windows-x86_64" +$workDir = Join-Path ([System.IO.Path]::GetTempPath()) ("omnigraph-install-" + [System.Guid]::NewGuid().ToString("N")) +$selectedChannel = "" + +function Write-Log { + param([string]$Message) + Write-Host "==> $Message" +} + +function Get-ReleaseBaseUrl { + param([string]$Channel) + + if ($Version -ne "") { + return "https://github.com/$RepoSlug/releases/download/$Version" + } + + if ($Channel -eq "stable") { + return "https://github.com/$RepoSlug/releases/latest/download" + } + + if ($Channel -eq "edge") { + return "https://github.com/$RepoSlug/releases/download/edge" + } + + throw "unsupported ReleaseChannel '$Channel' (expected stable or edge)" +} + +function Download-ReleaseFiles { + param( + [string]$BaseUrl, + [string]$ArchivePath, + [string]$ChecksumPath + ) + + try { + Invoke-WebRequest -UseBasicParsing -Uri "$BaseUrl/$assetName" -OutFile $ArchivePath + Invoke-WebRequest -UseBasicParsing -Uri "$BaseUrl/$assetStem.sha256" -OutFile $ChecksumPath + return $true + } catch { + return $false + } +} + +function Verify-Checksum { + param( + [string]$ArchivePath, + [string]$ChecksumPath + ) + + $checksumText = (Get-Content -Path $ChecksumPath -Raw).Trim() + $expected = ($checksumText -split "\s+")[0].ToLowerInvariant() + if ($expected -eq "") { + throw "checksum file did not contain a SHA256 digest" + } + + $actual = (Get-FileHash -Path $ArchivePath -Algorithm SHA256).Hash.ToLowerInvariant() + if ($actual -ne $expected) { + throw "checksum verification failed for $assetName" + } +} + +function Install-FromDirectory { + param([string]$SourceDir) + + New-Item -ItemType Directory -Force -Path $InstallDir | Out-Null + Copy-Item -Path (Join-Path $SourceDir "omnigraph.exe") -Destination (Join-Path $InstallDir "omnigraph.exe") -Force + Copy-Item -Path (Join-Path $SourceDir "omnigraph-server.exe") -Destination (Join-Path $InstallDir "omnigraph-server.exe") -Force +} + +function Install-FromRelease { + New-Item -ItemType Directory -Force -Path $workDir | Out-Null + + $archivePath = Join-Path $workDir $assetName + $checksumPath = Join-Path $workDir "$assetStem.sha256" + + if ($Version -ne "") { + $script:selectedChannel = $Version + $baseUrl = Get-ReleaseBaseUrl -Channel $ReleaseChannel + Write-Log "Downloading $assetName from $Version" + if (!(Download-ReleaseFiles -BaseUrl $baseUrl -ArchivePath $archivePath -ChecksumPath $checksumPath)) { + throw "no published binary found for $assetName at release $Version" + } + } else { + $script:selectedChannel = $ReleaseChannel + $baseUrl = Get-ReleaseBaseUrl -Channel $selectedChannel + Write-Log "Downloading $assetName from $selectedChannel" + if (!(Download-ReleaseFiles -BaseUrl $baseUrl -ArchivePath $archivePath -ChecksumPath $checksumPath)) { + if ($ReleaseChannel -ne "stable") { + throw "no published binary found for $assetName on channel $ReleaseChannel" + } + + Write-Log "Stable release binaries are not published yet; falling back to edge" + $script:selectedChannel = "edge" + $baseUrl = Get-ReleaseBaseUrl -Channel $selectedChannel + if (!(Download-ReleaseFiles -BaseUrl $baseUrl -ArchivePath $archivePath -ChecksumPath $checksumPath)) { + throw "no published binary found for $assetName on stable or edge; build from source" + } + } + } + + Verify-Checksum -ArchivePath $archivePath -ChecksumPath $checksumPath + + $extractDir = Join-Path $workDir "extract" + New-Item -ItemType Directory -Force -Path $extractDir | Out-Null + Expand-Archive -Path $archivePath -DestinationPath $extractDir -Force + Install-FromDirectory -SourceDir $extractDir +} + +function Print-Summary { + $omnigraphPath = Join-Path $InstallDir "omnigraph.exe" + $serverPath = Join-Path $InstallDir "omnigraph-server.exe" + + Write-Host "" + Write-Host "Installed:" + Write-Host " $omnigraphPath" + Write-Host " $serverPath" + Write-Host "" + Write-Host "Verify:" + Write-Host " $omnigraphPath version" + Write-Host " $serverPath --help" + Write-Host "" + + if ($selectedChannel -ne "") { + Write-Host "Installed from release channel: $selectedChannel" + } + + $pathParts = $env:Path -split [System.IO.Path]::PathSeparator + if ($pathParts -notcontains $InstallDir) { + Write-Host "Add $InstallDir to PATH if needed." + } +} + +try { + Install-FromRelease + Print-Summary +} finally { + if (Test-Path $workDir) { + Remove-Item -Path $workDir -Recurse -Force + } +} diff --git a/scripts/local-rustfs-bootstrap.sh b/scripts/local-rustfs-bootstrap.sh index 6327f77..29427de 100755 --- a/scripts/local-rustfs-bootstrap.sh +++ b/scripts/local-rustfs-bootstrap.sh @@ -74,9 +74,6 @@ platform_asset_name() { Linux/x86_64) printf 'omnigraph-linux-x86_64.tar.gz\n' ;; - Darwin/x86_64) - printf 'omnigraph-macos-x86_64.tar.gz\n' - ;; Darwin/arm64) printf 'omnigraph-macos-arm64.tar.gz\n' ;; From 8eba37cc60c31b34d0f2e314ac34eff4363dcdc1 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sat, 30 May 2026 14:29:49 +0200 Subject: [PATCH 008/165] README: link to TypeScript SDK and MCP server (#92) Adds a `Clients` section pointing at the TypeScript SDK and the Model Context Protocol server for programmatic / LLM access to omnigraph-server. Both are published on npm and developed in ModernRelay/omnigraph-ts; this README is the canonical discovery surface so it should mention them. Notes the major.minor lockstep convention so users know which client version targets which server. --- README.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/README.md b/README.md index 946297c..0f6ebea 100644 --- a/README.md +++ b/README.md @@ -90,6 +90,24 @@ omnigraph branch merge feature-x --into main ./graph.omni See [docs/user/cli.md](docs/user/cli.md) for schema apply, snapshots, ingest, commits, and policy commands. +## Clients + +For programmatic access to a running `omnigraph-server`: + +- **TypeScript SDK** — [`@modernrelay/omnigraph`](https://www.npmjs.com/package/@modernrelay/omnigraph) ([source](https://github.com/ModernRelay/omnigraph-ts/tree/main/packages/sdk)). Instance-per-client, typed errors, camelCase types, async-iterator streaming export. + + ```bash + npm install @modernrelay/omnigraph + ``` + +- **Model Context Protocol server** — [`@modernrelay/omnigraph-mcp`](https://www.npmjs.com/package/@modernrelay/omnigraph-mcp) ([source](https://github.com/ModernRelay/omnigraph-ts/tree/main/packages/mcp)). Bridges Omnigraph to LLM hosts (Claude Desktop, Claude Code, …) over stdio. Exposes tools and resources for schema, branches, queries, mutations, ingest, and bundles curated best-practices guidance from the cookbook. + + ```bash + npm install -g @modernrelay/omnigraph-mcp + ``` + +Both packages are versioned in lockstep with `omnigraph-server` on major.minor: `@modernrelay/omnigraph@X.Y.*` targets `omnigraph-server@X.Y.*`. See [`ModernRelay/omnigraph-ts`](https://github.com/ModernRelay/omnigraph-ts) for the monorepo. + ## Docs - [Install guide](docs/user/install.md) From 854ad0afcb444ccbaa57e0f464a26f3073886f8d Mon Sep 17 00:00:00 2001 From: Andrew Altshuler Date: Sat, 30 May 2026 20:17:55 +0100 Subject: [PATCH 009/165] feat(server): compose OMNIGRAPH_TARGET_URI with OMNIGRAPH_CONFIG in entrypoint (#129) The container entrypoint's URI and config branches were mutually exclusive, so a deployment driven by OMNIGRAPH_TARGET_URI could never load a policy file. Forward --config alongside the positional URI when OMNIGRAPH_CONFIG is also set (the URI still wins via resolve_target_uri), enabling Cedar policy without changing how the URI is provided. Add docker/entrypoint_test.sh (arg-composition cases) + a CI job, and document the env-var contract in docs/user/deployment.md. Co-authored-by: Claude Opus 4.8 (1M context) --- .github/workflows/ci.yml | 12 ++++++++ docker/entrypoint.sh | 10 +++++- docker/entrypoint_test.sh | 65 +++++++++++++++++++++++++++++++++++++++ docs/user/deployment.md | 29 +++++++++++++++++ 4 files changed, 115 insertions(+), 1 deletion(-) create mode 100755 docker/entrypoint_test.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 918a472..5b7b7b2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -111,6 +111,18 @@ jobs: - name: Verify AGENTS.md ↔ docs/ cross-links run: bash scripts/check-agents-md.sh + entrypoint_test: + name: Container Entrypoint + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Checkout source + uses: actions/checkout@v5.0.1 + + - name: Verify omnigraph-server entrypoint arg composition + run: sh docker/entrypoint_test.sh + test: name: Test Workspace needs: classify_changes diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh index 83b7d34..a5fb275 100644 --- a/docker/entrypoint.sh +++ b/docker/entrypoint.sh @@ -9,8 +9,14 @@ fi bind="${OMNIGRAPH_BIND:-0.0.0.0:8080}" +# URI comes from the env var (the positional arg wins over any config +# `graphs` block in resolve_target_uri). OMNIGRAPH_CONFIG, when also set, +# is forwarded as --config purely to supply a policy file — the two +# compose. Without OMNIGRAPH_CONFIG the behavior is unchanged. if [ -n "${OMNIGRAPH_TARGET_URI:-}" ]; then - exec "$SERVER_BIN" "${OMNIGRAPH_TARGET_URI}" --bind "${bind}" + exec "$SERVER_BIN" "${OMNIGRAPH_TARGET_URI}" \ + ${OMNIGRAPH_CONFIG:+--config "$OMNIGRAPH_CONFIG"} \ + --bind "${bind}" fi if [ -n "${OMNIGRAPH_CONFIG:-}" ]; then @@ -28,5 +34,7 @@ omnigraph-server container startup requires one of: Optional: - OMNIGRAPH_BIND (default: 0.0.0.0:8080) - OMNIGRAPH_TARGET (used with OMNIGRAPH_CONFIG) + - OMNIGRAPH_CONFIG (may also accompany OMNIGRAPH_TARGET_URI to add a + policy file; the URI still comes from OMNIGRAPH_TARGET_URI) EOF exit 64 diff --git a/docker/entrypoint_test.sh b/docker/entrypoint_test.sh new file mode 100755 index 0000000..01fbee2 --- /dev/null +++ b/docker/entrypoint_test.sh @@ -0,0 +1,65 @@ +#!/bin/sh +# Self-contained test for docker/entrypoint.sh argument composition. +# Runs the entrypoint against a stub server that echoes its args, and +# asserts the forwarded argv for each startup mode. No Docker required. +# +# sh docker/entrypoint_test.sh +# +# Exits 0 on success, 1 on the first mismatch. +set -eu + +here=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +entrypoint="$here/entrypoint.sh" + +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT +mkdir -p "$work/bin" +cat > "$work/bin/omnigraph-server" <<'EOF' +#!/bin/sh +echo "ARGS: $*" +EOF +chmod +x "$work/bin/omnigraph-server" + +# Run the real entrypoint with SERVER_BIN pointed at the stub. +ep="$work/entrypoint.sh" +sed "s#SERVER_BIN=\"/usr/local/bin/omnigraph-server\"#SERVER_BIN=\"$work/bin/omnigraph-server\"#" \ + "$entrypoint" > "$ep" + +fail=0 +check() { + desc=$1; want=$2; got=$3 + if [ "$got" != "$want" ]; then + echo "FAIL: $desc" + echo " want: $want" + echo " got: $got" + fail=1 + else + echo "ok: $desc" + fi +} + +got=$(OMNIGRAPH_TARGET_URI="s3://b/g" OMNIGRAPH_BIND="0.0.0.0:8080" sh "$ep") +check "TARGET_URI only (legacy)" \ + "ARGS: s3://b/g --bind 0.0.0.0:8080" "$got" + +got=$(OMNIGRAPH_TARGET_URI="s3://b/g" OMNIGRAPH_CONFIG="/etc/omnigraph/omnigraph.yaml" OMNIGRAPH_BIND="0.0.0.0:8080" sh "$ep") +check "TARGET_URI + CONFIG composes (policy)" \ + "ARGS: s3://b/g --config /etc/omnigraph/omnigraph.yaml --bind 0.0.0.0:8080" "$got" + +got=$(OMNIGRAPH_CONFIG="/etc/omnigraph/omnigraph.yaml" OMNIGRAPH_BIND="0.0.0.0:8080" sh "$ep") +check "CONFIG only" \ + "ARGS: --config /etc/omnigraph/omnigraph.yaml --bind 0.0.0.0:8080" "$got" + +got=$(OMNIGRAPH_CONFIG="/etc/omnigraph/omnigraph.yaml" OMNIGRAPH_TARGET="active" OMNIGRAPH_BIND="0.0.0.0:8080" sh "$ep") +check "CONFIG + TARGET" \ + "ARGS: --config /etc/omnigraph/omnigraph.yaml --target active --bind 0.0.0.0:8080" "$got" + +got=$(sh "$ep" some-uri --bind 1.2.3.4:9 --extra) +check "explicit args passthrough" \ + "ARGS: some-uri --bind 1.2.3.4:9 --extra" "$got" + +if [ "$fail" -ne 0 ]; then + echo "entrypoint_test: FAILED" + exit 1 +fi +echo "entrypoint_test: all cases passed" diff --git a/docs/user/deployment.md b/docs/user/deployment.md index 8613dec..9a4466c 100644 --- a/docs/user/deployment.md +++ b/docs/user/deployment.md @@ -109,6 +109,35 @@ docker run --rm -p 8080:8080 \ --bind 0.0.0.0:8080 ``` +### Container entrypoint env vars + +When no positional args are given, the image entrypoint +(`docker/entrypoint.sh`) builds the server command from env vars: + +| Var | Effect | +|---|---| +| `OMNIGRAPH_TARGET_URI` | Graph URI, passed as the positional argument. | +| `OMNIGRAPH_CONFIG` | Path to an `omnigraph.yaml`, passed as `--config`. Used to supply a `policy.file` (Cedar authorization). The config file and any relative `policy.file` must be mounted into the container. | +| `OMNIGRAPH_TARGET` | Graph name to select from the config's `graphs:` block (with `OMNIGRAPH_CONFIG`, when no `OMNIGRAPH_TARGET_URI`). | +| `OMNIGRAPH_BIND` | Listen address (default `0.0.0.0:8080`). | + +`OMNIGRAPH_TARGET_URI` and `OMNIGRAPH_CONFIG` **compose**: set both to keep the +graph URI in the env var while loading policy from the config file (the +positional URI wins over any `graphs:` entry). To enable Cedar policy on a +container otherwise driven by `OMNIGRAPH_TARGET_URI`, mount the config dir and +add `OMNIGRAPH_CONFIG`: + +```bash +docker run --rm -p 8080:8080 \ + -e OMNIGRAPH_SERVER_BEARER_TOKEN="change-me" \ + -e OMNIGRAPH_TARGET_URI="s3://my-bucket/graphs/example/releases/2026-04-10-v0.1.0" \ + -e OMNIGRAPH_CONFIG="/etc/omnigraph/omnigraph.yaml" \ + -v "$PWD/config:/etc/omnigraph:ro" \ + omnigraph-server:local +# /etc/omnigraph/omnigraph.yaml contains `policy: { file: ./policy.yaml }`; +# policy.yaml (+ optional policy.tests.yaml) sit beside it in the mount. +``` + ## Auth The server can run unauthenticated for local development only when explicitly From 2d5c4b12021ff0bf4c7036ebba034674a6ba7bf3 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sat, 30 May 2026 23:20:56 +0200 Subject: [PATCH 010/165] =?UTF-8?q?docs:=20rename=20runs.md/runs.rs=20?= =?UTF-8?q?=E2=86=92=20writes=20and=20repoint=20all=20references=20(#131)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Run state machine was removed in MR-771 (v0.4.0); `docs/dev/runs.md` and `crates/omnigraph/tests/runs.rs` have since documented and tested the direct-publish write path, so the "runs" name was misleading. - git mv docs/dev/runs.md → docs/dev/writes.md (reframe H1 + intro; keep MR-771 history note) - git mv crates/omnigraph/tests/runs.rs → tests/writes.rs (reframe header) - repoint every runs.md / runs.rs reference across docs, AGENTS.md, and source comments - fix four pre-existing broken `docs/runs.md` links (the file never lived at that path) to `docs/dev/writes.md` - fix the stale v0.4.0 anchor to the live section No behavior change: every source edit is a comment. Engine builds and the renamed test passes 25/25; scripts/check-agents-md.sh passes. The run-removal cleanup itself (run_registry.rs guard, __run__ prefix) is deferred to MR-770. --- AGENTS.md | 4 ++-- crates/omnigraph-cli/tests/system_local.rs | 2 +- crates/omnigraph/src/db/manifest/recovery.rs | 2 +- crates/omnigraph/src/loader/mod.rs | 2 +- crates/omnigraph/src/table_store.rs | 4 ++-- crates/omnigraph/tests/staged_writes.rs | 4 ++-- crates/omnigraph/tests/{runs.rs => writes.rs} | 8 ++++---- docs/dev/architecture.md | 4 ++-- docs/dev/execution.md | 2 +- docs/dev/index.md | 2 +- docs/dev/invariants.md | 12 +++++------ docs/dev/testing.md | 4 ++-- docs/dev/{runs.md => writes.md} | 9 ++++++--- docs/releases/v0.4.0.md | 2 +- docs/releases/v0.4.1.md | 20 +++++++++---------- docs/user/errors.md | 2 +- docs/user/query-language.md | 2 +- docs/user/transactions.md | 2 +- 18 files changed, 45 insertions(+), 42 deletions(-) rename crates/omnigraph/tests/{runs.rs => writes.rs} (99%) rename docs/dev/{runs.md => writes.md} (98%) diff --git a/AGENTS.md b/AGENTS.md index 3fc78f7..68de6b8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -81,7 +81,7 @@ Full diagram and concurrency model: [docs/dev/architecture.md](docs/dev/architec | Embeddings (compiler + engine clients, env vars, `@embed`) | [docs/user/embeddings.md](docs/user/embeddings.md) | | Branches, commit graph, snapshots, system branches | [docs/user/branches-commits.md](docs/user/branches-commits.md) | | Transactions and atomicity (per-query atomic; branches as multi-query transactions) | [docs/user/transactions.md](docs/user/transactions.md) | -| Direct-publish writes (the former Run state machine, now demoted to publisher CAS) | [docs/dev/runs.md](docs/dev/runs.md) | +| Direct-publish write path (staging, D2, recovery sidecars; the former Run state machine) | [docs/dev/writes.md](docs/dev/writes.md) | | Three-way merge and conflict kinds | [docs/dev/merge.md](docs/dev/merge.md) | | Diff / change feed (`diff_between`, `diff_commits`) | [docs/user/changes.md](docs/user/changes.md) | | Query execution, mutation execution, bulk loader, `load` vs `ingest` | [docs/dev/execution.md](docs/dev/execution.md) | @@ -176,7 +176,7 @@ cargo run -p omnigraph-server -- --bind 0.0.0.0:8080 # run the server fr # Run one crate / one test file / one test fn cargo test -p omnigraph-engine --test traversal # one integration-test file (see docs/dev/testing.md) -cargo test -p omnigraph-engine --test runs concurrent # one test fn by name substring +cargo test -p omnigraph-engine --test writes concurrent # one test fn by name substring cargo test -p omnigraph-engine some_inline_test -- --nocapture # show stdout # Feature-gated suites (each is its own job in CI, not part of the default run) diff --git a/crates/omnigraph-cli/tests/system_local.rs b/crates/omnigraph-cli/tests/system_local.rs index 074b203..08f653d 100644 --- a/crates/omnigraph-cli/tests/system_local.rs +++ b/crates/omnigraph-cli/tests/system_local.rs @@ -991,7 +991,7 @@ query vector_search($q: String) { // The publisher CAS conflict shape is verified end-to-end at the engine // level in -// `crates/omnigraph/tests/runs.rs::concurrent_writers_one_succeeds_one_gets_expected_version_mismatch` +// `crates/omnigraph/tests/writes.rs::concurrent_writers_one_succeeds_one_gets_expected_version_mismatch` // and at the HTTP boundary in // `crates/omnigraph-server/tests/server.rs::change_conflict_returns_manifest_conflict_409`. // A CLI-level race would be timing-dependent; with direct-publish the diff --git a/crates/omnigraph/src/db/manifest/recovery.rs b/crates/omnigraph/src/db/manifest/recovery.rs index 425499a..4c1b987 100644 --- a/crates/omnigraph/src/db/manifest/recovery.rs +++ b/crates/omnigraph/src/db/manifest/recovery.rs @@ -2,7 +2,7 @@ //! //! This module implements the building blocks of the per-sidecar recovery //! sweep that closes the documented Phase B → Phase C residual (see -//! `docs/dev/runs.md` "Open-time recovery sweep"). The high-level shape: +//! `docs/dev/writes.md` "Open-time recovery sweep"). The high-level shape: //! //! 1. Each writer that performs a multi-table commit writes a small JSON //! sidecar at `__recovery/{ulid}.json` BEFORE its per-table diff --git a/crates/omnigraph/src/loader/mod.rs b/crates/omnigraph/src/loader/mod.rs index cade1f4..46a46e2 100644 --- a/crates/omnigraph/src/loader/mod.rs +++ b/crates/omnigraph/src/loader/mod.rs @@ -613,7 +613,7 @@ async fn load_jsonl_reader( } else { // LoadMode::Overwrite keeps the legacy inline-commit path — // truncate-then-append doesn't fit the staged shape (see - // `docs/runs.md` "LoadMode::Overwrite residual"). The recovery + // `docs/dev/writes.md` "LoadMode::Overwrite residual"). The recovery // sidecar is not applicable here because the writer doesn't go // through MutationStaging; per-table inline commits + a final // manifest publish handle their own residual via the documented diff --git a/crates/omnigraph/src/table_store.rs b/crates/omnigraph/src/table_store.rs index ddab706..46b15b0 100644 --- a/crates/omnigraph/src/table_store.rs +++ b/crates/omnigraph/src/table_store.rs @@ -49,7 +49,7 @@ pub struct DeleteState { /// `exec/mutation.rs`) and the bulk loader (`loader/mod.rs`). The /// intent: defer Lance commits to end-of-query so a mid-query failure /// leaves the touched table at the pre-mutation HEAD instead of -/// drifting ahead. See `docs/runs.md` for the publisher-CAS contract +/// drifting ahead. See `docs/dev/writes.md` for the publisher-CAS contract /// this builds on. /// /// `transaction` is opaque from our side — Lance owns its semantics. We @@ -901,7 +901,7 @@ impl TableStore { /// Lift path: either a Lance API extension that lets /// `MergeInsertBuilder` accept additional staged fragments, or an /// in-memory pre-merge here that folds prior staged batches into the - /// input stream. See `docs/runs.md`. + /// input stream. See `docs/dev/writes.md`. pub async fn stage_merge_insert( &self, ds: Dataset, diff --git a/crates/omnigraph/tests/staged_writes.rs b/crates/omnigraph/tests/staged_writes.rs index 021b36e..5335057 100644 --- a/crates/omnigraph/tests/staged_writes.rs +++ b/crates/omnigraph/tests/staged_writes.rs @@ -2,7 +2,7 @@ //! exercise `stage_append`, `stage_merge_insert`, `scan_with_staged`, //! and `count_rows_with_staged` directly against a Lance dataset — no //! Omnigraph engine involved. The engine-level use of these primitives -//! is exercised by `tests/runs.rs`. +//! is exercised by `tests/writes.rs`. //! //! Test surface here: //! 1. `stage_append` + `scan_with_staged` shows committed + staged data @@ -709,7 +709,7 @@ async fn stage_create_inverted_index_does_not_advance_head_until_commit() { /// /// **When Lance #6658 lands**: this test will need to flip — replace /// the assertion with a `stage_delete` + `commit_staged` round-trip -/// and remove the residual line in `docs/runs.md`. +/// and remove the residual line in `docs/dev/writes.md`. #[tokio::test] async fn delete_where_advances_head_inline_documents_residual() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/omnigraph/tests/runs.rs b/crates/omnigraph/tests/writes.rs similarity index 99% rename from crates/omnigraph/tests/runs.rs rename to crates/omnigraph/tests/writes.rs index cfff3fc..13cb10f 100644 --- a/crates/omnigraph/tests/runs.rs +++ b/crates/omnigraph/tests/writes.rs @@ -1,7 +1,7 @@ -//! Tests for the direct-to-target write path (Run state machine -//! removed). The Run/`__run__` staging branch / RunRecord state machine no -//! longer exists; mutations and loads write directly to target tables and -//! commit once via the publisher's `expected_table_versions` CAS. +//! Tests for the direct-publish write path: mutations and loads write +//! directly to target tables and commit once via the publisher's +//! `expected_table_versions` CAS. (History: this replaced the removed Run +//! state machine / `__run__` staging branches / RunRecord — MR-771.) //! //! What this file covers: //! - No `__run__*` branches are created by load or mutate. diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md index 8b7fca2..813f30c 100644 --- a/docs/dev/architecture.md +++ b/docs/dev/architecture.md @@ -207,7 +207,7 @@ contracts: This pattern realizes read-your-writes within a multi-statement mutation and keeps failure scope bounded for inserts/updates by construction at the writer layer. See [docs/dev/invariants.md](invariants.md) and -[docs/dev/runs.md](runs.md) for the publisher CAS contract this builds on. +[docs/dev/writes.md](writes.md) for the publisher CAS contract this builds on. ### Storage trait — today vs. roadmap @@ -278,7 +278,7 @@ flowchart LR eng --> wq ``` -The server applies Cedar policy at the HTTP boundary today. The roadmap, called out in [docs/dev/invariants.md](invariants.md) as a known gap, is to push policy into the planner as predicates. After Cedar, mutating handlers go through `WorkloadController` (per-actor admission cap + byte budget; PR 2 / MR-686) before reaching the engine. The engine itself holds an `Arc` so concurrent mutations on the same `(table, branch)` serialize at the queue, while disjoint keys run in parallel — see [docs/user/server.md](../user/server.md) "Per-actor admission control" and [docs/dev/runs.md](runs.md). The CLI bypasses the HTTP layer (and admission) and calls the engine API directly. +The server applies Cedar policy at the HTTP boundary today. The roadmap, called out in [docs/dev/invariants.md](invariants.md) as a known gap, is to push policy into the planner as predicates. After Cedar, mutating handlers go through `WorkloadController` (per-actor admission cap + byte budget; PR 2 / MR-686) before reaching the engine. The engine itself holds an `Arc` so concurrent mutations on the same `(table, branch)` serialize at the queue, while disjoint keys run in parallel — see [docs/user/server.md](../user/server.md) "Per-actor admission control" and [docs/dev/writes.md](writes.md). The CLI bypasses the HTTP layer (and admission) and calls the engine API directly. Code paths: diff --git a/docs/dev/execution.md b/docs/dev/execution.md index f5c2840..3a108d7 100644 --- a/docs/dev/execution.md +++ b/docs/dev/execution.md @@ -147,7 +147,7 @@ sequenceDiagram - End-of-query Lance commit: `TableStore::stage_append`, `stage_merge_insert`, `commit_staged` at `crates/omnigraph/src/table_store.rs` - Manifest commit primitive: `commit_updates_on_branch_with_expected` at `crates/omnigraph/src/db/omnigraph/table_ops.rs` -Atomicity guarantee for multi-statement mutations: a mid-query failure leaves Lance HEAD untouched on staged tables (no inline commit happened during op execution), so the next mutation proceeds normally with no `ExpectedVersionMismatch`. The publisher CAS at the very end either succeeds (manifest advances atomically across all touched sub-tables) or fails with a typed `ManifestConflictDetails::ExpectedVersionMismatch` (no partial publish). See [docs/dev/invariants.md](invariants.md) and [docs/dev/runs.md](runs.md). +Atomicity guarantee for multi-statement mutations: a mid-query failure leaves Lance HEAD untouched on staged tables (no inline commit happened during op execution), so the next mutation proceeds normally with no `ExpectedVersionMismatch`. The publisher CAS at the very end either succeeds (manifest advances atomically across all touched sub-tables) or fails with a typed `ManifestConflictDetails::ExpectedVersionMismatch` (no partial publish). See [docs/dev/invariants.md](invariants.md) and [docs/dev/writes.md](writes.md). ## Bulk loader (`loader/mod.rs`) diff --git a/docs/dev/index.md b/docs/dev/index.md index 83df8c8..d9ba5e5 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -21,7 +21,7 @@ constraints. User-facing behavior should still be documented through |---|---| | System structure, L1/L2 framing, component diagrams | [architecture.md](architecture.md) | | On-disk layout, manifest schema, URI behavior | [storage.md](../user/storage.md) | -| Direct-publish writes, D2, staged writes, recovery sidecars | [runs.md](runs.md) | +| Direct-publish writes, D2, staged writes, recovery sidecars | [writes.md](writes.md) | | Query execution, mutation execution, loader flow | [execution.md](execution.md) | | Index lifecycle and graph topology indexes | [indexes.md](../user/indexes.md) | | Branch and commit internals | [branches-commits.md](../user/branches-commits.md) | diff --git a/docs/dev/invariants.md b/docs/dev/invariants.md index 958042f..70477d4 100644 --- a/docs/dev/invariants.md +++ b/docs/dev/invariants.md @@ -38,7 +38,7 @@ Use it this way: publishes one manifest update. Do not commit per statement. Delete-only queries are the documented inline residual; the parse-time D2 rule prevents mixing deletes with insert/update until Lance exposes two-phase delete. - Read [runs.md](runs.md) and [execution.md](execution.md). + Read [writes.md](writes.md) and [execution.md](execution.md). 5. **Recovery is part of the commit protocol.** Writers that can advance Lance HEAD before manifest publish must write `__recovery/{ulid}.json` sidecars. @@ -56,7 +56,7 @@ Use it this way: branch they read even when index coverage is partial. Expensive index work should converge from manifest state instead of extending the critical write path. Scalar staged index builds and vector inline residuals are documented - in [runs.md](runs.md) and [indexes.md](../user/indexes.md). + in [writes.md](writes.md) and [indexes.md](../user/indexes.md). 8. **Schema identity survives renames.** Accepted schema identity must remain stable across type and property renames. Rename support belongs in migration @@ -96,12 +96,12 @@ Use it this way: | Area | Current state | Source | |---|---|---| -| Multi-table commit | Manifest CAS plus recovery sidecars; not a single Lance primitive | [runs.md](runs.md), [architecture.md](architecture.md) | -| Constructive mutations | In-memory `MutationStaging`, one end-of-query table commit per touched table, then one manifest publish | [runs.md](runs.md), [execution.md](execution.md) | -| Deletes | Inline-commit residual; delete-only queries allowed, mixed insert/update/delete rejected by D2 | [query-language.md](../user/query-language.md), [runs.md](runs.md) | +| Multi-table commit | Manifest CAS plus recovery sidecars; not a single Lance primitive | [writes.md](writes.md), [architecture.md](architecture.md) | +| Constructive mutations | In-memory `MutationStaging`, one end-of-query table commit per touched table, then one manifest publish | [writes.md](writes.md), [execution.md](execution.md) | +| Deletes | Inline-commit residual; delete-only queries allowed, mixed insert/update/delete rejected by D2 | [query-language.md](../user/query-language.md), [writes.md](writes.md) | | Schema validation | Type checks, required fields, defaults, edge endpoint checks, and edge cardinality are enforced on write paths | [schema-language.md](../user/schema-language.md), [execution.md](execution.md) | | Unique constraints | Intra-batch and write-path checks exist; full cross-version uniqueness is still a gap | [schema-language.md](../user/schema-language.md) | -| Storage trait | `TableStorage` exists as the sealed staged-write surface; full call-site migration and capability/stat surfaces are incomplete | [runs.md](runs.md), [architecture.md](architecture.md) | +| Storage trait | `TableStorage` exists as the sealed staged-write surface; full call-site migration and capability/stat surfaces are incomplete | [writes.md](writes.md), [architecture.md](architecture.md) | | Index lifecycle | `ensure_indices` is explicit today; reconciler-based convergence is roadmap | [indexes.md](../user/indexes.md), [maintenance.md](../user/maintenance.md) | | Traversal IDs | Runtime still builds `TypeIndex`; Lance stable row-id based graph IDs are roadmap | [architecture.md](architecture.md), [query-language.md](../user/query-language.md) | | Auth | Bearer token hashing and server-side actor resolution are implemented at the HTTP boundary | [server.md](../user/server.md), [policy.md](../user/policy.md) | diff --git a/docs/dev/testing.md b/docs/dev/testing.md index e6989ba..425fcee 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -20,7 +20,7 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav | `end_to_end.rs` | Full init → load → query/mutate flow | | `branching.rs` | Branch create / list / delete, lazy fork | | `merge_truth_table.rs` | Merge-pair truth table (MR-786): all 9×9 `(left_op, right_op)` cells from `{noop, addNode, removeNode, addEdge, removeEdge, setProperty, dropProperty, addLabel, removeLabel}`. Adding a new op to `OpVariant` forces a compile error in `build_case` until the new row + column are dispositioned. 36 executable cells run through real `branch_merge` with a structured oracle (`MergeOutcome` / `MergeConflictKind` + graph-state assert); 45 cells involving `dropProperty`/`addLabel`/`removeLabel` are recorded as `Unsupported` until the mutation grammar grows. | -| `runs.rs` | Direct-publish writes: cancellation, concurrent-writer CAS, multi-statement atomicity, MR-794 staged-write rewire (D₂ rejection, insert+update coalesce, multi-append coalesce, partial-failure recovery, load RI/cardinality recovery) | +| `writes.rs` | Direct-publish writes: cancellation, concurrent-writer CAS, multi-statement atomicity, MR-794 staged-write rewire (D₂ rejection, insert+update coalesce, multi-append coalesce, partial-failure recovery, load RI/cardinality recovery) | | `staged_writes.rs` | TableStore staged-write primitives (`stage_append`, `stage_merge_insert`, `commit_staged`, `scan_with_staged`, `count_rows_with_staged`) — primitive-level only; engine code uses the in-memory `MutationStaging` accumulator instead | | `lifecycle.rs` | Graph lifecycle, schema state | | `point_in_time.rs` | Snapshots, time travel (`snapshot_at_version`, `entity_at`) | @@ -89,7 +89,7 @@ If introducing coverage tooling is in scope for your task, the natural first ste How to check: -1. **Map the change to an area** — use the engine integration-test table above (`branching.rs`, `runs.rs`, `search.rs`, etc.). The filename usually names the area. +1. **Map the change to an area** — use the engine integration-test table above (`branching.rs`, `writes.rs`, `search.rs`, etc.). The filename usually names the area. 2. **Open the file and skim every test fn name.** Test fn names are the index — read them all, not just the first few. 3. **Grep for the symbol or path you're changing.** `rg ` or `rg ` across all `tests/` directories surfaces existing coverage you might miss. 4. **Decide one of three outcomes**, in this order of preference: diff --git a/docs/dev/runs.md b/docs/dev/writes.md similarity index 98% rename from docs/dev/runs.md rename to docs/dev/writes.md index 816f2ac..974f7a6 100644 --- a/docs/dev/runs.md +++ b/docs/dev/writes.md @@ -1,7 +1,10 @@ -# Runs — REMOVED (MR-771) +# Direct-Publish Write Path -The Run state machine and `__run__` staging branches were removed in -MR-771. `mutate_as` and `load` now write **directly to the target table** +> History: the Run state machine and `__run__` staging branches were +> removed in MR-771 (shipped v0.4.0). Writes now go directly to the target +> table; this document specifies that direct-publish path. + +`mutate_as` and `load` write **directly to the target table** and call `ManifestBatchPublisher::publish` once at the end with `expected_table_versions` (the per-table manifest versions captured before the first write). Cross-table OCC is enforced inside the publisher; the diff --git a/docs/releases/v0.4.0.md b/docs/releases/v0.4.0.md index efb2da7..d3a8244 100644 --- a/docs/releases/v0.4.0.md +++ b/docs/releases/v0.4.0.md @@ -65,7 +65,7 @@ manifest. The next mutation against that table fails with `ExpectedVersionMismatch`. Most validation runs before any Lance write, so single-statement mutations are unaffected; the narrow path is multi-statement queries with late-op failures. Tracked as a follow-up; -see [docs/dev/runs.md](../dev/runs.md#known-limitation-mid-query-partial-failure-on-the-same-table) +see [docs/dev/writes.md](../dev/writes.md#mid-query-partial-failure-closed-by-mr-794) for the workaround. ## Upgrade notes diff --git a/docs/releases/v0.4.1.md b/docs/releases/v0.4.1.md index 78211e4..4983015 100644 --- a/docs/releases/v0.4.1.md +++ b/docs/releases/v0.4.1.md @@ -19,7 +19,7 @@ mutation proceeds normally. HEAD on every staged table is untouched and the next mutation proceeds normally. A narrowed residual remains at the finalize→publisher boundary (multi-table `commit_staged` is not - atomic with the manifest commit) — see [docs/dev/runs.md](../dev/runs.md) + atomic with the manifest commit) — see [docs/dev/writes.md](../dev/writes.md) "Finalize → publisher residual" for details. - **D₂ parse-time rule**: a single mutation query is either insert/update-only or delete-only. Mixed → rejected with a clear @@ -75,14 +75,14 @@ mutation proceeds normally. ## Tests added -- `tests/runs.rs::partial_failure_leaves_target_queryable_and_unblocks_next_mutation` +- `tests/writes.rs::partial_failure_leaves_target_queryable_and_unblocks_next_mutation` (replaces the old `partial_failure_observably_rolls_back_but_blocks_next_mutation_on_same_table`) -- `tests/runs.rs::mutation_rejects_mixed_insert_and_delete_at_parse_time` -- `tests/runs.rs::mixed_insert_and_update_on_same_person_coalesces_to_one_merge` -- `tests/runs.rs::multiple_appends_to_same_edge_coalesce_to_one_append` -- `tests/runs.rs::multi_statement_inserts_publish_exactly_once` -- `tests/runs.rs::load_with_bad_edge_reference_unblocks_next_load` -- `tests/runs.rs::load_with_cardinality_violation_unblocks_next_load` +- `tests/writes.rs::mutation_rejects_mixed_insert_and_delete_at_parse_time` +- `tests/writes.rs::mixed_insert_and_update_on_same_person_coalesces_to_one_merge` +- `tests/writes.rs::multiple_appends_to_same_edge_coalesce_to_one_append` +- `tests/writes.rs::multi_statement_inserts_publish_exactly_once` +- `tests/writes.rs::load_with_bad_edge_reference_unblocks_next_load` +- `tests/writes.rs::load_with_cardinality_violation_unblocks_next_load` ## Files changed @@ -105,7 +105,7 @@ mutation proceeds normally. - `Cargo.toml` (workspace) + `crates/omnigraph/Cargo.toml` — added `datafusion = "52"` direct dep (transitively pulled by Lance already; required for `MemTable`). -- `docs/dev/runs.md` — removed "Known limitation" section; documented +- `docs/dev/writes.md` — removed "Known limitation" section; documented the new accumulator + D₂ + LoadMode::Overwrite residual. - `docs/dev/invariants.md` — mutation atomicity / read-your-writes status flipped to `upheld for inserts/updates`. @@ -127,7 +127,7 @@ mutation proceeds normally. as legacy. - `docs/user/cli.md` — replaced the legacy `omnigraph run *` quickstart block with `omnigraph commit list/show`. -- `docs/dev/testing.md` — extended the `runs.rs` row to cover the new +- `docs/dev/testing.md` — extended the `writes.rs` row to cover the new staged-write contract tests; added the `staged_writes.rs` row. - `AGENTS.md` (CLAUDE.md symlink) — updated the atomic-per-query description and the L2 capability matrix row. diff --git a/docs/user/errors.md b/docs/user/errors.md index fd047eb..8373b0d 100644 --- a/docs/user/errors.md +++ b/docs/user/errors.md @@ -9,7 +9,7 @@ - `Manifest(ManifestError { kind: BadRequest|NotFound|Conflict|Internal, details: Option, … })` - `ManifestConflictDetails::ExpectedVersionMismatch { table_key, expected, actual }` — caller's `expected_table_versions` did not match the manifest's current latest non-tombstoned version (set by `OmniError::manifest_expected_version_mismatch`). - `ManifestConflictDetails::RowLevelCasContention` — Lance row-level CAS rejected the publish because a concurrent writer landed the same `object_id`. Retried internally by the publisher; only surfaces if the retry budget exhausts. - - **D₂ parse-time rejection** (MR-794): a single mutation query that mixes inserts/updates with deletes errors out *before any I/O* with kind `BadRequest`. Message: `mutation '' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes`. See [docs/user/query-language.md](query-language.md) for the rule and [docs/dev/runs.md](../dev/runs.md) for the underlying staged-write rationale. + - **D₂ parse-time rejection** (MR-794): a single mutation query that mixes inserts/updates with deletes errors out *before any I/O* with kind `BadRequest`. Message: `mutation '' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes`. See [docs/user/query-language.md](query-language.md) for the rule and [docs/dev/writes.md](../dev/writes.md) for the underlying staged-write rationale. - `MergeConflicts(Vec)` Compiler-side `NanoError` covers parse / catalog / type / storage / plan / execution / arrow / lance / IO / manifest / unique-constraint, each with structured spans (`SourceSpan { start, end }`) for ariadne-style diagnostics. diff --git a/docs/user/query-language.md b/docs/user/query-language.md index 94528af..6c7516f 100644 --- a/docs/user/query-language.md +++ b/docs/user/query-language.md @@ -70,7 +70,7 @@ A single mutation query must be **either insert/update-only or delete-only**. Mi > `mutation '' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes. This restriction lifts when Lance exposes a two-phase delete API (tracked: MR-793 / Lance-upstream).` -Reason: under the staged-write rewire (MR-794), inserts and updates accumulate in memory and commit at end-of-query, while deletes still inline-commit (Lance 4.0.0 has no public two-phase delete). Mixing creates ordering hazards (same-row insert→delete becomes a no-op because the staged insert isn't visible to delete; cascading deletes of just-inserted edges break referential integrity by silent design). Until Lance exposes `DeleteJob::execute_uncommitted`, the parse-time rejection keeps both paths atomic and correct. See [docs/dev/runs.md](../dev/runs.md) and [docs/dev/invariants.md](../dev/invariants.md). +Reason: under the staged-write rewire (MR-794), inserts and updates accumulate in memory and commit at end-of-query, while deletes still inline-commit (Lance 4.0.0 has no public two-phase delete). Mixing creates ordering hazards (same-row insert→delete becomes a no-op because the staged insert isn't visible to delete; cascading deletes of just-inserted edges break referential integrity by silent design). Until Lance exposes `DeleteJob::execute_uncommitted`, the parse-time rejection keeps both paths atomic and correct. See [docs/dev/writes.md](../dev/writes.md) and [docs/dev/invariants.md](../dev/invariants.md). ## IR (Intermediate Representation) diff --git a/docs/user/transactions.md b/docs/user/transactions.md index e4ed485..d6c79f4 100644 --- a/docs/user/transactions.md +++ b/docs/user/transactions.md @@ -164,5 +164,5 @@ This is the workflow MR-797 / agentic loops are designed around: **branches are - [`docs/user/branches-commits.md`](branches-commits.md) — branch and commit-graph mechanics. - [`docs/dev/merge.md`](../dev/merge.md) — three-way merge details and conflict kinds. - [`docs/user/query-language.md`](query-language.md) — `.gq` syntax for the multi-statement queries used above. -- [`docs/dev/runs.md`](../dev/runs.md) — the per-query commit pipeline that gives single-query atomicity. +- [`docs/dev/writes.md`](../dev/writes.md) — the per-query commit pipeline that gives single-query atomicity. - [`docs/dev/invariants.md`](../dev/invariants.md) — the architectural rule. From e94e7d124a4d1b862cb0e05b75b9551ce7ab1f3c Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Mon, 1 Jun 2026 13:11:36 +0200 Subject: [PATCH 011/165] fix(bootstrap): pin RustFS to beta.3 + allow insecure default creds (#136) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `local-rustfs-bootstrap.sh` defaulted RUSTFS_IMAGE to the floating `rustfs/rustfs:latest`, which resolved to 1.0.0-beta.4 (2026-05-21). beta.4 added a credentials-policy check that refuses to start when the access/secret keys are values it treats as "default" (rustfsadmin/rustfsadmin, the script's defaults) — so a fresh bootstrap broke at RustFS startup. Pin the default to 1.0.0-beta.3 to match CI (.github/workflows/ci.yml) and the v0.5.0 release notes, and additionally pass RUSTFS_ALLOW_INSECURE_DEFAULT_CREDENTIALS=true so the script stays forward-compatible if RUSTFS_IMAGE is overridden to beta.4+. Co-authored-by: Ragnor Comerford --- scripts/local-rustfs-bootstrap.sh | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/scripts/local-rustfs-bootstrap.sh b/scripts/local-rustfs-bootstrap.sh index 29427de..c4fdcbe 100755 --- a/scripts/local-rustfs-bootstrap.sh +++ b/scripts/local-rustfs-bootstrap.sh @@ -6,7 +6,14 @@ SOURCE_REF="${SOURCE_REF:-main}" RELEASE_CHANNEL="${RELEASE_CHANNEL:-edge}" WORKDIR="${WORKDIR:-$PWD/.omnigraph-rustfs-demo}" RUSTFS_CONTAINER_NAME="${RUSTFS_CONTAINER_NAME:-omnigraph-rustfs-demo}" -RUSTFS_IMAGE="${RUSTFS_IMAGE:-rustfs/rustfs:latest}" +# Pinned to 1.0.0-beta.3 (2026-05-14) — the last known-good tag, matching CI +# (.github/workflows/ci.yml). `rustfs/rustfs:latest` (1.0.0-beta.4, 2026-05-21) +# added a credentials-policy check that refuses to start when the access/secret +# keys are values it considers "default" (rustfsadmin/rustfsadmin here). This +# script still works on beta.4+ because it passes +# RUSTFS_ALLOW_INSECURE_DEFAULT_CREDENTIALS=true below — so overriding +# RUSTFS_IMAGE to a newer tag is safe. +RUSTFS_IMAGE="${RUSTFS_IMAGE:-rustfs/rustfs:1.0.0-beta.3}" RUSTFS_DATA_DIR="${RUSTFS_DATA_DIR:-$WORKDIR/rustfs-data}" BUCKET="${BUCKET:-omnigraph-local}" PREFIX="${PREFIX:-repos/context}" @@ -265,6 +272,7 @@ start_rustfs() { -v "$RUSTFS_DATA_DIR:/data" \ -e RUSTFS_ACCESS_KEY="$AWS_ACCESS_KEY_ID" \ -e RUSTFS_SECRET_KEY="$AWS_SECRET_ACCESS_KEY" \ + -e RUSTFS_ALLOW_INSECURE_DEFAULT_CREDENTIALS=true \ "$RUSTFS_IMAGE" \ /data >/dev/null } From 353c0c876ab11c0ed4c087fcb32b894ec184df6d Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Mon, 1 Jun 2026 13:28:38 +0200 Subject: [PATCH 012/165] fix(branch): make branch delete correct under partial failure (#137) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * test(lance): pin force_delete_branch surface guard Pin the Lance 6.0.1 force_delete_branch behavior the branch-delete single-authority redesign relies on: plain delete_branch errors on a missing ref, force_delete_branch removes an existing forked branch, and the local-store quirk where force_delete on a fully-absent branch still errors (worked around by the upcoming TableStore::force_delete_branch). Re-pin the docs/dev/lance.md alignment stanza (9 guards; 4 runtime). * feat(storage): add force branch-delete to TableStore + CommitGraph Add TableStore::force_delete_branch and CommitGraph::force_delete_branch (idempotent: tolerate an already-absent branch via Lance RefNotFound / NotFound), plus CommitGraph::list_branches for the cleanup reconciler to diff against the manifest authority. RefConflict (referencing descendants) is still surfaced. Unused until the branch-delete rewire. * test(maintenance): red — cleanup reconciles orphaned branch forks Forge a Lance branch on the Person table that the manifest never references (a zombie fork from an incomplete prior delete) and assert cleanup reclaims it while leaving main intact. Fails today: cleanup does not yet reconcile orphaned forks. Goes green with the next commit. * fix(maintenance): reconcile orphaned branch forks in cleanup Add reconcile_orphaned_branches: force_delete_branch every per-table and commit-graph Lance branch absent from the manifest branch set (the authority), children-before-parents. Folded into cleanup_all_tables, runs before version GC. Idempotent and authority-derived; no-ops once nothing is orphaned, and would harmlessly find nothing if a future Lance atomic multi-dataset branch op prevented orphans. Adds TableStore::list_branches and exposes graph_commits_uri(pub crate). Turns the maintenance red test green. * test(failpoints): red — branch_delete partial failure converges Add the branch_delete.before_table_cleanup failpoint hook (inert without the feature) and a regression test: a cleanup-step failure after the manifest authority flip must leave branch_delete returning Ok, the branch gone, the orphan stranded, then reclaimed by cleanup, and the name reusable. Fails today: cleanup_deleted_branch_tables propagates the error as a hard failure. Goes green with the next commit. * fix(branch): best-effort fork reclaim after the manifest flip Make branch_delete treat per-table forks and the commit-graph branch as derived state reclaimed best-effort with force_delete_branch after the manifest authority flip. A reclaim failure (transient error, or the branch_delete.before_table_cleanup failpoint) is logged via tracing::warn and swallowed: the branch is already gone and the cleanup reconciler converges the orphan. cleanup_deleted_branch_tables no longer returns an error or blocks the call. Turns the partial-failure recovery test green. * test(failpoints): red — recreate over orphaned fork is actionable After a partial-failure delete leaves a fork orphaned, recreating the branch name and writing to the previously-forked table before cleanup runs currently surfaces the opaque ExpectedVersionMismatch ("stale view ... expected manifest table version N"). Assert instead a clear error pointing the user at cleanup. Goes green with the next commit. * fix(branch): actionable orphan-collision error in fork_branch_from_state When a fork's create_branch collides with an existing target ref, reuse it only if its head matches source_version (a legitimate concurrent first-write). A version mismatch means a zombie fork from an incomplete prior delete: return a manifest_conflict pointing the user at `omnigraph cleanup`, instead of the opaque ExpectedVersionMismatch. Turns the recreate-over-orphan red test green. * docs(invariants): single-authority branch-lifecycle + Lance forward-compat Record branch delete in the Current Truth Matrix: manifest is the single authority flipped atomically first, per-table forks + commit-graph branch are derived state reclaimed best-effort with the cleanup reconciler as backstop, and reusing a name whose reclaim failed surfaces an actionable error. Note the reconciler is authority-derived and degrades to a no-op under a future Lance atomic multi-dataset branch op, the same shape as invariant 7. * test(failpoints): red — cleanup isolates a single-table failure Add the cleanup.table_gc failpoint hook (inert without the feature) and an error: Option field on TableCleanupStats (mechanical, always None for now). Regression test: a one-shot version-GC failure for one table must not abort the whole cleanup — assert cleanup still succeeds, surfaces the failure per-table in stats, and the independent reconcile pass still reclaimed an orphan. Fails today: the version-GC collect aborts on the first table error. Goes green with the next commit. * fix(maintenance): fault-isolate cleanup per table Make the cleanup sweep do as much as it can and converge on re-run instead of aborting wholesale on one table's transient error (invariant 13). The version-GC loop now records a per-table failure on its stats row (error: Some) and logs it rather than collecting into a Result that aborts; reconcile_orphaned_branches isolates per-table and commit-graph failures into BranchReconcileStats.failures. The CLI reports any failed tables and tells the user to rerun cleanup. Addresses the Devin review finding. Turns the single-table-failure test green. * test(failpoints): red — branch_create heals commit-graph zombie + is atomic Add the branch_delete.before_commit_graph_reclaim failpoint hook and two regression tests: (a) recreating a name whose delete left a commit-graph zombie must succeed (today it dies on Lance's internal Clone error), and (b) branch_create must roll back the manifest branch when the derived commit-graph branch fails (today it leaves the manifest branch created while returning Err). Both fail now; green with the next commit. The existing branch_create_failpoint_triggers test still passes. * fix(branch): make branch_create atomic + heal commit-graph zombie branch_create now flips the manifest authority first, then creates the derived commit-graph branch in create_commit_graph_branch, force-dropping any orphaned commit-graph ref left by an incomplete prior delete (the manifest branch is fresh, so a same-named commit-graph branch is provably a zombie). If commit-graph creation fails, the manifest branch is rolled back so the name never half-exists. Addresses the Codex review finding. Turns the two branch_create red tests green; existing tests unaffected. * test(failpoints): red — fork collision misclassifies live concurrent fork Add the fork.before_classify failpoint hook and a concurrency test: when a concurrent first-write legitimately wins the fork race, the loser must get a retryable refresh-and-retry, not the misleading run-cleanup orphan error. Today the version-comparison misclassifies the live fork as an orphan (the Cursor finding). Goes green with the next commit. * fix(branch): manifest-arbitrated fork-collision classification Classify a fork collision by the manifest authority instead of comparing Lance branch versions. Before forking, open_owned_dataset_for_branch_write re-reads the live manifest: if the table is already forked on the active branch, a concurrent first-write won and the loser gets a retryable refresh-and-retry (not a misleading orphan error). fork_branch_from_state no longer guesses from versions — a create collision past that check is an orphan, so it returns the actionable cleanup error. Addresses the Cursor finding; turns the live-concurrent-fork test green, zombie path unchanged. * test(failpoints): close branch-lifecycle test gaps Three coverage additions for the branch-delete work (behavior already correct; these lock it in and catch regressions): - cleanup_isolates_reconcile_failure: inject a force-delete failure into the reconcile loop (new cleanup.reconcile_fork hook) and assert the sweep continues + converges on re-run. Directly covers the reconcile loop the Devin finding was about (previously only version-GC was). - cleanup_reclaims_orphaned_commit_graph_branch: forge a commit-graph orphan via the delete reclaim failpoint and assert cleanup's reconcile_commit_graph_orphans drops it (previously untested). - fork_collision_with_live_concurrent_fork_is_retryable: replace the fixed 300ms sleep with a deterministic readiness signal (cfg_callback + compare_exchange atomics) so the two-writer ordering can't flake. Full failpoints suite 31/0. --- crates/omnigraph-cli/src/main.rs | 15 +- crates/omnigraph/src/db/commit_graph.rs | 33 +- crates/omnigraph/src/db/graph_coordinator.rs | 70 ++- crates/omnigraph/src/db/omnigraph.rs | 38 +- crates/omnigraph/src/db/omnigraph/optimize.rs | 242 +++++++++- .../omnigraph/src/db/omnigraph/table_ops.rs | 16 + crates/omnigraph/src/table_store.rs | 66 ++- crates/omnigraph/tests/failpoints.rs | 446 ++++++++++++++++++ .../omnigraph/tests/lance_surface_guards.rs | 48 ++ crates/omnigraph/tests/maintenance.rs | 69 +++ docs/dev/invariants.md | 8 + docs/dev/lance.md | 3 +- docs/user/branches-commits.md | 6 +- docs/user/maintenance.md | 8 +- 14 files changed, 1004 insertions(+), 64 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index b7e3041..d98c302 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -2699,20 +2699,33 @@ async fn main() -> Result<()> { "table_key": s.table_key, "bytes_removed": s.bytes_removed, "old_versions_removed": s.old_versions_removed, + "error": s.error, })).collect::>(), }); print_json(&value)?; } else { let total_bytes: u64 = stats.iter().map(|s| s.bytes_removed).sum(); let total_versions: u64 = stats.iter().map(|s| s.old_versions_removed).sum(); + let failed: Vec<&str> = stats + .iter() + .filter(|s| s.error.is_some()) + .map(|s| s.table_key.as_str()) + .collect(); println!( "cleanup {} ({}) — removed {} versions ({} bytes) across {} tables", uri, policy_desc, total_versions, total_bytes, - stats.len() + stats.len() - failed.len() ); + if !failed.is_empty() { + println!( + " {} table(s) failed and will be retried on the next cleanup: {}", + failed.len(), + failed.join(", ") + ); + } } } Command::Graphs { command } => match command { diff --git a/crates/omnigraph/src/db/commit_graph.rs b/crates/omnigraph/src/db/commit_graph.rs index 565bd69..9531a64 100644 --- a/crates/omnigraph/src/db/commit_graph.rs +++ b/crates/omnigraph/src/db/commit_graph.rs @@ -169,6 +169,37 @@ impl CommitGraph { self.refresh().await } + /// Idempotently drop the commit-graph branch `name`, tolerating an + /// already-absent branch (see [`TableStore::force_delete_branch`] for the + /// same semantics). Used by the best-effort reclaim in `branch_delete` and + /// the `cleanup` orphan reconciler. `RefConflict` (referencing descendants) + /// is still surfaced. + pub async fn force_delete_branch(&mut self, name: &str) -> Result<()> { + let mut ds = Dataset::open(&graph_commits_uri(&self.root_uri)) + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + match ds.force_delete_branch(name).await { + Ok(()) => {} + Err(lance::Error::RefNotFound { .. }) | Err(lance::Error::NotFound { .. }) => {} + Err(e) => return Err(OmniError::Lance(e.to_string())), + } + self.refresh().await + } + + /// List the named branches present on the commit-graph dataset. The + /// `cleanup` reconciler diffs this against the manifest branch set to find + /// orphaned commit-graph branches to reclaim. + pub async fn list_branches(&self) -> Result> { + let ds = Dataset::open(&graph_commits_uri(&self.root_uri)) + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + let branches = ds + .list_branches() + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + Ok(branches.into_keys().collect()) + } + pub async fn append_commit( &mut self, manifest_branch: Option<&str>, @@ -345,7 +376,7 @@ impl CommitGraph { } } -fn graph_commits_uri(root_uri: &str) -> String { +pub(crate) fn graph_commits_uri(root_uri: &str) -> String { format!("{}/{}", root_uri.trim_end_matches('/'), GRAPH_COMMITS_DIR) } diff --git a/crates/omnigraph/src/db/graph_coordinator.rs b/crates/omnigraph/src/db/graph_coordinator.rs index a721036..dfe2767 100644 --- a/crates/omnigraph/src/db/graph_coordinator.rs +++ b/crates/omnigraph/src/db/graph_coordinator.rs @@ -211,14 +211,47 @@ impl GraphCoordinator { let branch = normalize_branch_name(name)? .ok_or_else(|| OmniError::manifest("cannot create branch 'main'".to_string()))?; self.ensure_commit_graph_initialized().await?; + + // Manifest authority flip first. self.manifest.create_branch(&branch).await?; - failpoints::maybe_fail("branch_create.after_manifest_branch_create")?; - if let Some(commit_graph) = &mut self.commit_graph { - commit_graph.create_branch(&branch).await?; + + // Derived commit-graph branch. If anything after the authority flip + // fails, roll back the manifest branch so the branch never half-exists + // (a manifest branch with no commit-graph branch breaks the next write). + if let Err(err) = self.create_commit_graph_branch(&branch).await { + if let Err(rollback_err) = self.manifest.delete_branch(&branch).await { + tracing::warn!( + target: "omnigraph::branch_create", + branch = %branch, + error = %rollback_err, + "rollback of manifest branch failed after commit-graph create failure", + ); + } + return Err(err); } Ok(()) } + /// Create the derived commit-graph branch for `branch`, healing a zombie ref + /// left by an incomplete prior delete. The manifest branch was just created + /// fresh, so any existing commit-graph branch with this name is provably + /// orphaned and is force-dropped before recreating. + async fn create_commit_graph_branch(&mut self, branch: &str) -> Result<()> { + failpoints::maybe_fail("branch_create.after_manifest_branch_create")?; + let Some(commit_graph) = &mut self.commit_graph else { + return Ok(()); + }; + if commit_graph + .list_branches() + .await? + .iter() + .any(|existing| existing == branch) + { + commit_graph.force_delete_branch(branch).await?; + } + commit_graph.create_branch(branch).await + } + pub async fn branch_delete(&mut self, name: &str) -> Result<()> { let branch = normalize_branch_name(name)? .ok_or_else(|| OmniError::manifest("cannot delete branch 'main'".to_string()))?; @@ -229,20 +262,43 @@ impl GraphCoordinator { ))); } + // Manifest authority flip — the single atomic op that makes the branch + // cease to exist. Must succeed; everything after is derived state + // reclaimed best-effort. self.manifest.delete_branch(&branch).await?; + // Commit-graph branch is derived state. Reclaim best-effort with the + // idempotent force variant: a failure here (or a missing dataset) is + // reconciled by `cleanup` and must not fail the delete after the + // authority already flipped. + if let Err(err) = self.reclaim_commit_graph_branch(&branch).await { + tracing::warn!( + target: "omnigraph::branch_delete::cleanup", + branch = %branch, + error = %err, + "best-effort commit-graph branch reclaim failed; cleanup will reconcile", + ); + } + + Ok(()) + } + + /// Best-effort, idempotent reclaim of the commit-graph branch `branch`. + /// Tolerates an absent commit-graph dataset (a graph that never committed). + async fn reclaim_commit_graph_branch(&mut self, branch: &str) -> Result<()> { + failpoints::maybe_fail("branch_delete.before_commit_graph_reclaim")?; if let Some(commit_graph) = &mut self.commit_graph { - commit_graph.delete_branch(&branch).await?; + commit_graph.force_delete_branch(branch).await } else if self .storage .exists(&graph_commits_uri(self.root_uri())) .await? { let mut commit_graph = CommitGraph::open(self.root_uri()).await?; - commit_graph.delete_branch(&branch).await?; + commit_graph.force_delete_branch(branch).await + } else { + Ok(()) } - - Ok(()) } pub async fn snapshot_at_version(&self, version: u64) -> Result { diff --git a/crates/omnigraph/src/db/omnigraph.rs b/crates/omnigraph/src/db/omnigraph.rs index 5c92ac3..eb58623 100644 --- a/crates/omnigraph/src/db/omnigraph.rs +++ b/crates/omnigraph/src/db/omnigraph.rs @@ -1058,11 +1058,14 @@ impl Omnigraph { Ok(()) } - async fn cleanup_deleted_branch_tables( - &self, - branch: &str, - owned_tables: &[(String, String)], - ) -> Result<()> { + /// Best-effort reclaim of the per-table Lance forks a just-deleted branch + /// owned. Runs AFTER the manifest authority flip, so the branch is already + /// gone and these forks are unreachable orphans. A failure here (transient + /// object-store error, the `branch_delete.before_table_cleanup` failpoint) + /// is logged and swallowed: the `cleanup` reconciler is the guaranteed + /// backstop that converges any leftover orphan. Uses `force_delete_branch` + /// so a partially-reclaimed retry is idempotent. + async fn cleanup_deleted_branch_tables(&self, branch: &str, owned_tables: &[(String, String)]) { let mut seen_paths = HashSet::new(); let mut cleanup_targets = owned_tables .iter() @@ -1073,15 +1076,21 @@ impl Omnigraph { for (table_key, table_path) in cleanup_targets { let dataset_uri = self.table_store.dataset_uri(&table_path); - if let Err(err) = self.table_store.delete_branch(&dataset_uri, branch).await { - return Err(OmniError::manifest_internal(format!( - "branch '{}' was deleted but cleanup failed for {}: {}", - branch, table_key, err - ))); + let outcome = match crate::failpoints::maybe_fail("branch_delete.before_table_cleanup") + { + Ok(()) => self.table_store.force_delete_branch(&dataset_uri, branch).await, + Err(injected) => Err(injected), + }; + if let Err(err) = outcome { + tracing::warn!( + target: "omnigraph::branch_delete::cleanup", + branch = %branch, + table = %table_key, + error = %err, + "best-effort fork reclaim failed; cleanup will reconcile the orphan", + ); } } - - Ok(()) } async fn delete_branch_storage_only(&self, branch: &str) -> Result<()> { @@ -1105,9 +1114,12 @@ impl Omnigraph { .map(|entry| (entry.table_key.clone(), entry.table_path.clone())) .collect::>(); + // Authority flip (+ best-effort commit-graph reclaim) — must succeed. self.coordinator.write().await.branch_delete(branch).await?; + // Best-effort per-table fork reclaim; cleanup reconciles any leftover. self.cleanup_deleted_branch_tables(branch, &owned_tables) - .await + .await; + Ok(()) } pub(crate) fn normalize_branch_name(branch: &str) -> Result> { diff --git a/crates/omnigraph/src/db/omnigraph/optimize.rs b/crates/omnigraph/src/db/omnigraph/optimize.rs index e158dc7..c703836 100644 --- a/crates/omnigraph/src/db/omnigraph/optimize.rs +++ b/crates/omnigraph/src/db/omnigraph/optimize.rs @@ -64,12 +64,15 @@ pub struct TableOptimizeStats { pub committed: bool, } -/// Per-table outcome of `cleanup_all_tables`. +/// Per-table outcome of `cleanup_all_tables`. `error` is `Some` when this +/// table's version GC failed; cleanup is fault-isolated per table, so a single +/// table's failure is recorded here rather than aborting the whole sweep. #[derive(Debug, Clone)] pub struct TableCleanupStats { pub table_key: String, pub bytes_removed: u64, pub old_versions_removed: u64, + pub error: Option, } /// Run Lance `compact_files` on every node + edge table on `main`. @@ -138,6 +141,26 @@ pub async fn cleanup_all_tables( db.ensure_schema_state_valid().await?; db.ensure_schema_apply_idle("cleanup").await?; + // Reclaim orphaned branch forks (from an incomplete prior `branch_delete`) + // before version GC. Authority-derived and idempotent; the eager + // best-effort reclaim in `branch_delete` covers the common case, this is + // the guaranteed backstop. Logged for observability. + let reconciled = reconcile_orphaned_branches(db).await?; + if !reconciled.reclaimed.is_empty() { + tracing::info!( + count = reconciled.reclaimed.len(), + reclaimed = ?reconciled.reclaimed, + "cleanup reconciled orphaned branch forks" + ); + } + if !reconciled.failures.is_empty() { + tracing::warn!( + count = reconciled.failures.len(), + failures = ?reconciled.failures, + "cleanup could not reconcile some orphaned forks; will retry next cleanup" + ); + } + let before_timestamp = options.older_than.map(|d| Utc::now() - d); let keep_versions = options.keep_versions; @@ -160,36 +183,205 @@ pub async fn cleanup_all_tables( let concurrency = maint_concurrency().min(table_tasks.len()).max(1); let table_store = &db.table_store; - let results: Vec> = futures::stream::iter(table_tasks.into_iter()) + // Fault-isolated per table: a single table's GC failure is recorded on its + // stats row (`error: Some`) and logged, never aborting the healthy tables. + // cleanup is the convergence backstop, so it must do as much as it can and + // converge on re-run rather than fail wholesale (invariant 13). + let results: Vec = futures::stream::iter(table_tasks.into_iter()) .map(|(table_key, full_path)| async move { - let ds = table_store - .open_dataset_head_for_write(&table_key, &full_path, None) - .await?; - let before_version = keep_versions - .map(|n| ds.version().version.saturating_sub(n as u64)) - .filter(|v| *v > 0); - let policy = CleanupPolicy { - before_timestamp, - before_version, - delete_unverified: false, - error_if_tagged_old_versions: false, - clean_referenced_branches: false, - delete_rate_limit: None, - }; - let removed: RemovalStats = lance::dataset::cleanup::cleanup_old_versions(&ds, policy) - .await - .map_err(|e| OmniError::Lance(e.to_string()))?; - Ok(TableCleanupStats { - table_key, - bytes_removed: removed.bytes_removed, - old_versions_removed: removed.old_versions, - }) + let outcome: Result = async { + crate::failpoints::maybe_fail("cleanup.table_gc")?; + let ds = table_store + .open_dataset_head_for_write(&table_key, &full_path, None) + .await?; + let before_version = keep_versions + .map(|n| ds.version().version.saturating_sub(n as u64)) + .filter(|v| *v > 0); + let policy = CleanupPolicy { + before_timestamp, + before_version, + delete_unverified: false, + error_if_tagged_old_versions: false, + clean_referenced_branches: false, + delete_rate_limit: None, + }; + lance::dataset::cleanup::cleanup_old_versions(&ds, policy) + .await + .map_err(|e| OmniError::Lance(e.to_string())) + } + .await; + match outcome { + Ok(removed) => TableCleanupStats { + table_key, + bytes_removed: removed.bytes_removed, + old_versions_removed: removed.old_versions, + error: None, + }, + Err(err) => { + tracing::warn!( + target: "omnigraph::cleanup", + table = %table_key, + error = %err, + "version GC failed for table; other tables unaffected", + ); + TableCleanupStats { + table_key, + bytes_removed: 0, + old_versions_removed: 0, + error: Some(err.to_string()), + } + } + } }) .buffer_unordered(concurrency) .collect() .await; - results.into_iter().collect() + Ok(results) +} + +/// Outcome of [`reconcile_orphaned_branches`]: the `(owner, branch)` pairs +/// reclaimed and the `(owner, error)` pairs that failed, where `owner` is a +/// table key (e.g. `node:Person`) or `"_graph_commits"`. Per-owner failures are +/// isolated and recorded here, not propagated — the next reconcile converges. +#[derive(Debug, Clone, Default)] +pub struct BranchReconcileStats { + pub reclaimed: Vec<(String, String)>, + pub failures: Vec<(String, String)>, +} + +/// Drop every per-table and commit-graph Lance branch that the manifest no +/// longer references. +/// +/// Orphaned forks arise when a `branch_delete` flips the manifest authority +/// (atomic) but a downstream best-effort reclaim does not complete. They are +/// unreachable through any snapshot — no manifest entry can name them — yet +/// they pin their `tree/{branch}/` storage and can block reusing the branch +/// name. This is the guaranteed convergence backstop: it is idempotent and +/// derived purely from the manifest authority, so it no-ops once everything is +/// reconciled, and it would harmlessly find nothing if a future Lance atomic +/// multi-dataset branch op prevented orphans from forming. +/// +/// The keep-set is the full (unfiltered) manifest branch list, so system +/// branches' forks are never reclaimed; `main`/default is not a named Lance +/// branch and so is never a candidate. Referencing children are dropped before +/// parents (Lance refuses to delete a referenced parent) by ordering longest +/// branch names first. +pub async fn reconcile_orphaned_branches(db: &Omnigraph) -> Result { + use std::collections::HashSet; + + let keep: HashSet = db + .coordinator + .read() + .await + .all_branches() + .await? + .into_iter() + .collect(); + + let resolved = db.resolved_branch_target(None).await?; + let snapshot = resolved.snapshot; + let table_targets: Vec<(String, String)> = all_table_keys(&db.catalog()) + .into_iter() + .filter_map(|table_key| { + let entry = snapshot.entry(&table_key)?; + let full_path = format!("{}/{}", db.root_uri, entry.table_path); + Some((table_key, full_path)) + }) + .collect(); + + let mut stats = BranchReconcileStats::default(); + + // Per-table fault isolation: one table's transient failure is recorded and + // logged, never aborting the rest of the sweep. + for (table_key, full_path) in table_targets { + let listed = match db.table_store.list_branches(&full_path).await { + Ok(listed) => listed, + Err(err) => { + tracing::warn!( + target: "omnigraph::cleanup", + table = %table_key, + error = %err, + "listing branches failed during reconcile; skipping table", + ); + stats.failures.push((table_key.clone(), err.to_string())); + continue; + } + }; + for branch in orphan_branches(listed, &keep) { + let outcome = match crate::failpoints::maybe_fail("cleanup.reconcile_fork") { + Ok(()) => db.table_store.force_delete_branch(&full_path, &branch).await, + Err(injected) => Err(injected), + }; + match outcome { + Ok(()) => stats.reclaimed.push((table_key.clone(), branch)), + Err(err) => { + tracing::warn!( + target: "omnigraph::cleanup", + table = %table_key, + branch = %branch, + error = %err, + "reclaiming orphaned fork failed; will retry next cleanup", + ); + stats.failures.push((table_key.clone(), err.to_string())); + } + } + } + } + + // Commit-graph orphans (best-effort: the dataset may not exist on a graph + // that has never committed; any failure is isolated and retried next time). + if let Err(err) = reconcile_commit_graph_orphans(db, &keep, &mut stats).await { + tracing::warn!( + target: "omnigraph::cleanup", + error = %err, + "commit-graph orphan reconcile failed; will retry next cleanup", + ); + stats.failures.push(("_graph_commits".to_string(), err.to_string())); + } + + Ok(stats) +} + +/// Commit-graph half of [`reconcile_orphaned_branches`], split out so its +/// errors can be isolated. Returns `Ok` when the commit-graph dataset is absent. +async fn reconcile_commit_graph_orphans( + db: &Omnigraph, + keep: &std::collections::HashSet, + stats: &mut BranchReconcileStats, +) -> Result<()> { + let commits_uri = crate::db::commit_graph::graph_commits_uri(db.root_uri()); + if !db.storage_adapter().exists(&commits_uri).await? { + return Ok(()); + } + let mut commit_graph = crate::db::commit_graph::CommitGraph::open(db.root_uri()).await?; + for branch in orphan_branches(commit_graph.list_branches().await?, keep) { + match commit_graph.force_delete_branch(&branch).await { + Ok(()) => stats.reclaimed.push(("_graph_commits".to_string(), branch)), + Err(err) => { + tracing::warn!( + target: "omnigraph::cleanup", + branch = %branch, + error = %err, + "reclaiming orphaned commit-graph branch failed; will retry next cleanup", + ); + stats.failures.push(("_graph_commits".to_string(), err.to_string())); + } + } + } + Ok(()) +} + +/// Filter `present` Lance branches down to those absent from the manifest +/// `keep` set, ordered children-before-parents (longest name first) so Lance's +/// referenced-parent `RefConflict` cannot block reclamation. +fn orphan_branches(present: Vec, keep: &std::collections::HashSet) -> Vec { + let mut orphans: Vec = present + .into_iter() + .filter(|branch| !keep.contains(branch)) + .collect(); + orphans.sort_by(|a, b| b.len().cmp(&a.len()).then_with(|| a.cmp(b))); + orphans } fn all_table_keys(catalog: &omnigraph_compiler::catalog::Catalog) -> Vec { diff --git a/crates/omnigraph/src/db/omnigraph/table_ops.rs b/crates/omnigraph/src/db/omnigraph/table_ops.rs index 0e89c45..3ed9c43 100644 --- a/crates/omnigraph/src/db/omnigraph/table_ops.rs +++ b/crates/omnigraph/src/db/omnigraph/table_ops.rs @@ -483,6 +483,22 @@ pub(super) async fn open_owned_dataset_for_branch_write( Ok((ds, Some(active_branch.to_string()))) } source_branch => { + crate::failpoints::maybe_fail("fork.before_classify")?; + // Authority check before forking: re-read the live manifest. If this + // table is already forked on active_branch, a concurrent first-write + // won the race and our snapshot is stale — that is a retryable + // conflict, not an orphan. (A zombie fork is never in the manifest, + // so this only fires for a live concurrent fork.) + let live = db.snapshot_for_branch(Some(active_branch)).await?; + if let Some(entry) = live.entry(table_key) { + if entry.table_branch.as_deref() == Some(active_branch) { + return Err(OmniError::manifest_expected_version_mismatch( + table_key, + entry_version, + entry.table_version, + )); + } + } fork_dataset_from_entry_state( db, table_key, diff --git a/crates/omnigraph/src/table_store.rs b/crates/omnigraph/src/table_store.rs index 46b15b0..10123b0 100644 --- a/crates/omnigraph/src/table_store.rs +++ b/crates/omnigraph/src/table_store.rs @@ -177,6 +177,45 @@ impl TableStore { .map_err(|e| OmniError::Lance(e.to_string())) } + /// List the named Lance branches present on the dataset at `dataset_uri`. + /// The `cleanup` orphan reconciler diffs this against the manifest branch + /// set to find orphaned per-table forks. `main`/default is not a named + /// branch and never appears here. + pub async fn list_branches(&self, dataset_uri: &str) -> Result> { + let ds = Dataset::open(dataset_uri) + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + let branches = ds + .list_branches() + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + Ok(branches.into_keys().collect()) + } + + /// Idempotently drop `branch` from the dataset at `dataset_uri`. + /// + /// Unlike [`delete_branch`](Self::delete_branch), this tolerates an + /// already-absent branch — both a missing contents ref (Lance's + /// `force_delete_branch` handles that) and a missing `tree/{branch}/` + /// directory (the local-store `NotFound` quirk pinned by + /// `lance_surface_guards::force_delete_branch_semantics`). Safe to call on a + /// possibly-orphaned or already-reclaimed fork. + /// + /// A branch that still has referencing descendants (`RefConflict`) is NOT + /// tolerated: that is a real ordering error and surfaces as `OmniError::Lance`. + /// Used by the eager best-effort reclaim in `cleanup_deleted_branch_tables` + /// and the `cleanup` orphan reconciler. + pub async fn force_delete_branch(&self, dataset_uri: &str, branch: &str) -> Result<()> { + let mut ds = Dataset::open(dataset_uri) + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + match ds.force_delete_branch(branch).await { + Ok(()) => Ok(()), + Err(lance::Error::RefNotFound { .. }) | Err(lance::Error::NotFound { .. }) => Ok(()), + Err(e) => Err(OmniError::Lance(e.to_string())), + } + } + pub async fn open_dataset_at_state( &self, table_path: &str, @@ -243,21 +282,24 @@ impl TableStore { .map_err(|e| OmniError::Lance(e.to_string()))?; self.ensure_expected_version(&source_ds, table_key, source_version)?; - match source_ds + if source_ds .create_branch(target_branch, source_version, None) .await + .is_err() { - Ok(_) => {} - Err(create_err) => match self - .open_dataset_head(dataset_uri, Some(target_branch)) - .await - { - Ok(ds) => { - self.ensure_expected_version(&ds, table_key, source_version)?; - return Ok(ds); - } - Err(_) => return Err(OmniError::Lance(create_err.to_string())), - }, + // The target branch ref already exists. The caller + // (`open_owned_dataset_for_branch_write`) re-reads the live manifest + // before forking and returns a retryable error when a concurrent + // writer legitimately holds the fork, so reaching here means the + // manifest does NOT reference this fork: it is an orphan from an + // incomplete prior `branch_delete`. Surface the actionable cleanup + // error rather than guessing from Lance branch versions. + return Err(OmniError::manifest_conflict(format!( + "branch '{}' has orphaned table state for '{}' from an incomplete \ + prior delete; run `omnigraph cleanup` to reclaim it before reusing \ + this branch name", + target_branch, table_key + ))); } let ds = self diff --git a/crates/omnigraph/tests/failpoints.rs b/crates/omnigraph/tests/failpoints.rs index 5ea71c5..149c63a 100644 --- a/crates/omnigraph/tests/failpoints.rs +++ b/crates/omnigraph/tests/failpoints.rs @@ -41,6 +41,452 @@ async fn branch_create_failpoint_triggers() { ); } +// Branch delete flips the manifest authority first, then reclaims the per-table +// forks best-effort. A failure during that reclaim (here, the +// `branch_delete.before_table_cleanup` failpoint, standing in for a transient +// object-store error) must NOT fail the call: the branch is already gone, and +// `cleanup` reconciles the stranded fork. The branch name is reusable after. +#[tokio::test] +async fn branch_delete_partial_failure_converges_via_cleanup() { + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + let mut main = helpers::init_and_load(&dir).await; + + main.branch_create("feature").await.unwrap(); + let mut feature = Omnigraph::open(&uri).await.unwrap(); + helpers::mutate_branch( + &mut feature, + "feature", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Eve")], &[("$age", 22)]), + ) + .await + .unwrap(); + drop(feature); + + let person_uri = node_table_uri(&uri, "Person"); + { + let ds = lance::Dataset::open(&person_uri).await.unwrap(); + assert!( + ds.list_branches().await.unwrap().contains_key("feature"), + "precondition: the owned table fork exists before delete" + ); + } + + // Inject a failure during per-table cleanup, AFTER the manifest authority + // flip. branch_delete must still succeed (best-effort reclaim). + { + let _fp = ScopedFailPoint::new("branch_delete.before_table_cleanup", "return"); + main.branch_delete("feature").await.expect( + "branch_delete is best-effort after the manifest flip: a cleanup-step \ + failure must not fail the call", + ); + } + + // Authority flipped: the branch is gone. + assert_eq!(main.branch_list().await.unwrap(), vec!["main".to_string()]); + + // The eager reclaim failed, so the orphan is stranded until cleanup. + { + let ds = lance::Dataset::open(&person_uri).await.unwrap(); + assert!( + ds.list_branches().await.unwrap().contains_key("feature"), + "failed eager reclaim should leave the orphan for cleanup to reconcile" + ); + } + + // cleanup converges: the orphan is reclaimed. + main.cleanup(omnigraph::db::CleanupPolicyOptions { + keep_versions: Some(1), + older_than: None, + }) + .await + .unwrap(); + { + let ds = lance::Dataset::open(&person_uri).await.unwrap(); + assert!( + !ds.list_branches().await.unwrap().contains_key("feature"), + "cleanup should reconcile the orphaned fork away" + ); + } + + // The name is reusable after cleanup reclaims the orphan. + main.branch_create("feature").await.unwrap(); + let mut feature2 = Omnigraph::open(&uri).await.unwrap(); + helpers::mutate_branch( + &mut feature2, + "feature", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Frank")], &[("$age", 41)]), + ) + .await + .unwrap(); +} + +// Reusing a branch name whose delete left an orphaned fork (before `cleanup` +// reconciles it) must fail with a clear, actionable error pointing at +// `cleanup`, not the opaque `ExpectedVersionMismatch` that leaks from the fork +// path. The recreate itself succeeds; the first write to the previously-forked +// table is where the stale orphan collides. +#[tokio::test] +async fn recreate_over_orphaned_fork_before_cleanup_is_actionable() { + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + let mut main = helpers::init_and_load(&dir).await; + + main.branch_create("feature").await.unwrap(); + let mut feature = Omnigraph::open(&uri).await.unwrap(); + helpers::mutate_branch( + &mut feature, + "feature", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Eve")], &[("$age", 22)]), + ) + .await + .unwrap(); + drop(feature); + + // Partial delete: leaves the Person fork orphaned (cleanup not yet run). + { + let _fp = ScopedFailPoint::new("branch_delete.before_table_cleanup", "return"); + main.branch_delete("feature").await.unwrap(); + } + + // Recreate the name and write to the previously-forked table WITHOUT a + // cleanup in between. + main.branch_create("feature").await.unwrap(); + let mut feature2 = Omnigraph::open(&uri).await.unwrap(); + let err = helpers::mutate_branch( + &mut feature2, + "feature", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Frank")], &[("$age", 41)]), + ) + .await + .expect_err("write should collide with the stale orphaned fork"); + + let msg = err.to_string(); + assert!( + msg.contains("cleanup") + && (msg.contains("orphan") || msg.contains("incomplete prior delete")), + "expected an actionable orphaned-fork error pointing at cleanup, got: {msg}" + ); + assert!( + !msg.contains("expected manifest table version"), + "should not surface the opaque ExpectedVersionMismatch, got: {msg}" + ); +} + +// cleanup is the guaranteed convergence backstop, so one table's transient +// failure must not abort the whole sweep. Inject a one-shot version-GC failure +// for a single table and assert: cleanup still succeeds, the failure is +// surfaced per-table in the returned stats, and the independent reconcile pass +// still reclaimed an orphan. +#[tokio::test] +async fn cleanup_isolates_single_table_failure() { + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + let mut db = helpers::init_and_load(&dir).await; + + // Forge an orphaned fork on the Person table (a reconcile target). + let person_uri = node_table_uri(&uri, "Person"); + { + let mut ds = lance::Dataset::open(&person_uri).await.unwrap(); + let base = ds.version().version; + ds.create_branch("ghost", base, None).await.unwrap(); + } + + // One table's version GC fails once; the sweep must isolate it. + let _fp = ScopedFailPoint::new("cleanup.table_gc", "1*return"); + let stats = db + .cleanup(omnigraph::db::CleanupPolicyOptions { + keep_versions: Some(1), + older_than: None, + }) + .await + .expect("a single table's GC failure must not abort cleanup"); + + let errored = stats.iter().filter(|s| s.error.is_some()).count(); + assert_eq!( + errored, 1, + "exactly one table's GC failure should be surfaced in stats, got {errored}" + ); + assert!( + stats.len() >= 4, + "every node+edge table should still appear in the stats" + ); + + // The reconcile pass is independent of the GC failure, so the orphan is gone. + { + let ds = lance::Dataset::open(&person_uri).await.unwrap(); + assert!( + !ds.list_branches().await.unwrap().contains_key("ghost"), + "reconcile should reclaim the orphan despite the GC failure" + ); + } +} + +// Companion to the version-GC isolation test, exercising the OTHER cleanup +// loop: a force-delete failure while reconciling one orphaned fork must be +// isolated (logged, not propagated) so the sweep continues, and a later +// cleanup converges. This is the loop the Devin finding was about. +#[tokio::test] +async fn cleanup_isolates_reconcile_failure() { + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + let mut db = helpers::init_and_load(&dir).await; + + // Forge an orphaned fork the reconcile pass will try to reclaim. + let person_uri = node_table_uri(&uri, "Person"); + { + let mut ds = lance::Dataset::open(&person_uri).await.unwrap(); + let base = ds.version().version; + ds.create_branch("ghost", base, None).await.unwrap(); + } + + // Inject a one-shot failure into the reconcile force-delete. The sweep must + // not abort. + { + let _fp = ScopedFailPoint::new("cleanup.reconcile_fork", "1*return"); + db.cleanup(omnigraph::db::CleanupPolicyOptions { + keep_versions: Some(1), + older_than: None, + }) + .await + .expect("a reconcile force-delete failure must not abort cleanup"); + } + // The blocked orphan is still present (the failure was isolated, not retried). + { + let ds = lance::Dataset::open(&person_uri).await.unwrap(); + assert!( + ds.list_branches().await.unwrap().contains_key("ghost"), + "the orphan whose reclaim was injected-to-fail should remain" + ); + } + // A second cleanup with no injected failure converges. + db.cleanup(omnigraph::db::CleanupPolicyOptions { + keep_versions: Some(1), + older_than: None, + }) + .await + .unwrap(); + { + let ds = lance::Dataset::open(&person_uri).await.unwrap(); + assert!( + !ds.list_branches().await.unwrap().contains_key("ghost"), + "the second cleanup should reconcile the orphan" + ); + } +} + +// The cleanup reconciler must reclaim orphaned commit-graph branches, not just +// per-table forks. A delete whose best-effort commit-graph reclaim fails leaves +// a commit-graph orphan; the next cleanup must drop it. +#[tokio::test] +async fn cleanup_reclaims_orphaned_commit_graph_branch() { + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + let mut db = helpers::init_and_load(&dir).await; + + db.branch_create("feature").await.unwrap(); + // Delete, failing the commit-graph reclaim → commit-graph "feature" orphan + // (manifest branch gone, commit-graph branch left behind). + { + let _fp = ScopedFailPoint::new("branch_delete.before_commit_graph_reclaim", "return"); + db.branch_delete("feature").await.unwrap(); + } + + let commits_uri = format!("{}/_graph_commits.lance", uri.trim_end_matches('/')); + { + let ds = lance::Dataset::open(&commits_uri).await.unwrap(); + assert!( + ds.list_branches().await.unwrap().contains_key("feature"), + "precondition: the commit-graph branch should be orphaned after the failed reclaim" + ); + } + + db.cleanup(omnigraph::db::CleanupPolicyOptions { + keep_versions: Some(1), + older_than: None, + }) + .await + .unwrap(); + + { + let ds = lance::Dataset::open(&commits_uri).await.unwrap(); + assert!( + !ds.list_branches().await.unwrap().contains_key("feature"), + "cleanup should reclaim the orphaned commit-graph branch" + ); + } +} + +// A branch_delete whose best-effort commit-graph reclaim fails leaves a +// commit-graph "zombie" branch. Recreating that name must heal the zombie and +// succeed (branch_create force-deletes a stale commit-graph ref since the +// manifest branch is created fresh), instead of dying on the leftover ref. +#[tokio::test] +async fn branch_create_recreates_over_commit_graph_zombie() { + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let db = Omnigraph::init(dir.path().to_str().unwrap(), helpers::TEST_SCHEMA) + .await + .unwrap(); + + db.branch_create("feature").await.unwrap(); + { + // Fail the best-effort commit-graph reclaim → commit-graph "feature" + // zombie survives the delete (manifest authority still flips). + let _fp = ScopedFailPoint::new("branch_delete.before_commit_graph_reclaim", "return"); + db.branch_delete("feature").await.unwrap(); + } + assert_eq!(db.branch_list().await.unwrap(), vec!["main".to_string()]); + + db.branch_create("feature") + .await + .expect("branch_create should heal the zombie commit-graph branch and succeed"); + assert!( + db.branch_list() + .await + .unwrap() + .contains(&"feature".to_string()) + ); +} + +// branch_create is authority-then-derived: if the derived commit-graph branch +// cannot be created, the manifest branch (the authority) must be rolled back so +// the branch does not half-exist. The existing failpoint fires right after the +// manifest create, standing in for any post-authority failure. +#[tokio::test] +async fn branch_create_rolls_back_manifest_on_commit_graph_failure() { + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let db = Omnigraph::init(dir.path().to_str().unwrap(), helpers::TEST_SCHEMA) + .await + .unwrap(); + + let err = { + let _fp = ScopedFailPoint::new("branch_create.after_manifest_branch_create", "return"); + db.branch_create("feature").await.unwrap_err() + }; + assert!( + !db.branch_list() + .await + .unwrap() + .contains(&"feature".to_string()), + "branch_create must roll back the manifest branch when the derived \ + commit-graph branch fails, got error: {err}" + ); +} + +// A fork collision must be classified by the manifest authority, not by Lance +// branch versions. When a concurrent first-write legitimately wins the fork +// race, the loser sees a version mismatch — but that is a stale snapshot, not +// an orphan, so it must be a retryable "refresh and retry", never a misleading +// "run cleanup". +// +// Ordering is made deterministic (no sleeps) via a callback at the fork point: +// `compare_exchange` lets only the FIRST arrival (writer A) record readiness and +// block until released; later arrivals (writer B) fall through. The test waits +// on the readiness flag, lets B win and commit the fork, then releases A. +static FORK_A_AT_POINT: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false); +static FORK_RELEASE_A: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false); + +#[tokio::test(flavor = "multi_thread")] +async fn fork_collision_with_live_concurrent_fork_is_retryable() { + use std::sync::atomic::Ordering::SeqCst; + + let _scenario = FailScenario::setup(); + FORK_A_AT_POINT.store(false, SeqCst); + FORK_RELEASE_A.store(false, SeqCst); + + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + let main = helpers::init_and_load(&dir).await; + main.branch_create("feature").await.unwrap(); + + // First arrival (A) records readiness and blocks until released; the rest + // (B) fall through immediately. Bounded spin so a mistake can't hang forever. + fail::cfg_callback("fork.before_classify", || { + if FORK_A_AT_POINT + .compare_exchange(false, true, SeqCst, SeqCst) + .is_ok() + { + for _ in 0..2000 { + if FORK_RELEASE_A.load(SeqCst) { + break; + } + std::thread::sleep(std::time::Duration::from_millis(5)); + } + } + }) + .unwrap(); + + let uri_a = uri.clone(); + let writer_a = tokio::spawn(async move { + let mut a = Omnigraph::open(&uri_a).await.unwrap(); + helpers::mutate_branch( + &mut a, + "feature", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Eve")], &[("$age", 22)]), + ) + .await + }); + + // Wait (bounded) until A is parked at the fork point. + for _ in 0..600 { + if FORK_A_AT_POINT.load(SeqCst) { + break; + } + tokio::time::sleep(std::time::Duration::from_millis(5)).await; + } + assert!( + FORK_A_AT_POINT.load(SeqCst), + "writer A never reached the fork point" + ); + + // B wins the fork and commits it. + let mut b = Omnigraph::open(&uri).await.unwrap(); + helpers::mutate_branch( + &mut b, + "feature", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Frank")], &[("$age", 41)]), + ) + .await + .unwrap(); + + // Release A; it resumes, re-reads the manifest, and sees the fork is live. + FORK_RELEASE_A.store(true, SeqCst); + let err = writer_a + .await + .unwrap() + .expect_err("A's stale-snapshot fork should be a retryable conflict"); + fail::remove("fork.before_classify"); + + let msg = err.to_string(); + assert!( + !msg.contains("cleanup"), + "a live concurrent fork must not be misclassified as an orphan, got: {msg}" + ); + assert!( + msg.contains("refresh and retry") || msg.contains("expected manifest table version"), + "expected a retryable stale-view error, got: {msg}" + ); +} + #[tokio::test(flavor = "multi_thread")] async fn graph_publish_failpoint_triggers_before_commit_append() { let _scenario = FailScenario::setup(); diff --git a/crates/omnigraph/tests/lance_surface_guards.rs b/crates/omnigraph/tests/lance_surface_guards.rs index b65a808..ed1f22e 100644 --- a/crates/omnigraph/tests/lance_surface_guards.rs +++ b/crates/omnigraph/tests/lance_surface_guards.rs @@ -242,3 +242,51 @@ async fn _compile_delete_result_field_shape() -> lance::Result<()> { let _num_deleted: u64 = result.num_deleted_rows; Ok(()) } + +// --- Guard 9: force_delete_branch semantics -------------------------------- +// +// The branch-delete reconciler (`db/omnigraph/optimize.rs::reconcile_orphaned_branches`) +// and the eager best-effort reclaim in `cleanup_deleted_branch_tables` call +// `force_delete_branch` to drop orphaned branch refs. The single-authority +// design relies on three facts pinned here: +// 1. plain `delete_branch` errors on a missing ref (so the design uses the +// force variant instead); +// 2. `force_delete_branch` removes an existing (forked) branch — the orphan +// case, where a `tree/{branch}/` exists; +// 3. `force_delete_branch` on a *fully-absent* branch (no tree dir) still +// errors on the local store, because `remove_dir_all`'s NotFound is not +// caught for Lance's native error variant. `TableStore::force_delete_branch` +// wraps this to be fully idempotent. Pin the raw quirk so a future Lance +// fix (which would let us simplify the wrapper) is noticed. + +#[tokio::test] +async fn force_delete_branch_semantics() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().join("guard9.lance"); + let uri = uri.to_str().unwrap(); + let mut ds = fresh_dataset(uri).await; + + // (1) Plain delete of a never-created branch errors (RefNotFound). + assert!( + ds.delete_branch("nope").await.is_err(), + "Dataset::delete_branch on a missing ref should error; if this is now \ + Ok, the reconciler could drop the force variant." + ); + + // (2) force_delete_branch removes an existing (forked) branch. + let base = ds.version().version; + ds.create_branch("feature", base, None).await.unwrap(); + ds.force_delete_branch("feature").await.unwrap(); + assert!( + !ds.list_branches().await.unwrap().contains_key("feature"), + "force_delete_branch should remove an existing branch ref" + ); + + // (3) Quirk: force_delete on a fully-absent branch errors on the local + // store (worked around by TableStore::force_delete_branch). + assert!( + ds.force_delete_branch("never").await.is_err(), + "force_delete_branch on a fully-absent branch no longer errors — \ + TableStore::force_delete_branch's NotFound tolerance can be simplified." + ); +} diff --git a/crates/omnigraph/tests/maintenance.rs b/crates/omnigraph/tests/maintenance.rs index 3c6ab30..722bdc4 100644 --- a/crates/omnigraph/tests/maintenance.rs +++ b/crates/omnigraph/tests/maintenance.rs @@ -7,11 +7,24 @@ mod helpers; use std::time::Duration; +use lance::Dataset; use omnigraph::db::{CleanupPolicyOptions, Omnigraph}; use omnigraph::loader::{LoadMode, load_jsonl}; use helpers::{TEST_DATA, TEST_SCHEMA, count_rows, init_and_load}; +/// Filesystem URI of a node sub-table, mirroring the engine's layout +/// (FNV-1a of the type name under `nodes/`). Matches the helper in +/// `failpoints.rs`; used to inspect/forge Lance branches directly in tests. +fn node_table_uri(root: &str, type_name: &str) -> String { + let mut hash: u64 = 0xcbf2_9ce4_8422_2325; + for &b in type_name.as_bytes() { + hash ^= b as u64; + hash = hash.wrapping_mul(0x100_0000_01b3); + } + format!("{}/nodes/{hash:016x}", root.trim_end_matches('/')) +} + #[tokio::test] async fn optimize_on_empty_graph_returns_stats_per_table_with_no_changes() { let dir = tempfile::tempdir().unwrap(); @@ -158,3 +171,59 @@ async fn cleanup_then_optimize_preserves_rows_and_table_remains_writable() { .unwrap(); assert_eq!(count_rows(&db, "node:Person").await, people_before); } + +#[tokio::test] +async fn cleanup_reconciles_orphaned_branch_forks() { + // An incomplete prior `branch_delete` can leave a per-table Lance branch + // that the manifest no longer references (a "zombie" fork). It is + // unreachable through any snapshot but pins its `tree/{branch}/` storage. + // `cleanup` must reconcile it away: drop every Lance branch absent from the + // manifest authority, without touching `main`. + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + let mut db = init_and_load(&dir).await; + + let people_before = count_rows(&db, "node:Person").await; + assert!(people_before > 0, "fixture should seed Person rows"); + + // Forge an orphaned fork the manifest never knew about. + let person_uri = node_table_uri(&uri, "Person"); + { + let mut ds = Dataset::open(&person_uri).await.unwrap(); + let base = ds.version().version; + ds.create_branch("ghost", base, None).await.unwrap(); + assert!( + ds.list_branches().await.unwrap().contains_key("ghost"), + "precondition: orphaned fork staged" + ); + } + + db.cleanup(CleanupPolicyOptions { + keep_versions: Some(1), + older_than: None, + }) + .await + .unwrap(); + + // Orphan reclaimed; main untouched. + { + let ds = Dataset::open(&person_uri).await.unwrap(); + assert!( + !ds.list_branches().await.unwrap().contains_key("ghost"), + "cleanup should reconcile the orphaned 'ghost' fork away" + ); + } + assert_eq!( + count_rows(&db, "node:Person").await, + people_before, + "cleanup must not disturb main while reconciling orphans" + ); + + // Idempotent: a second cleanup with the orphan already gone is a no-op. + db.cleanup(CleanupPolicyOptions { + keep_versions: Some(1), + older_than: None, + }) + .await + .unwrap(); +} diff --git a/docs/dev/invariants.md b/docs/dev/invariants.md index 70477d4..0cf295c 100644 --- a/docs/dev/invariants.md +++ b/docs/dev/invariants.md @@ -99,6 +99,7 @@ Use it this way: | Multi-table commit | Manifest CAS plus recovery sidecars; not a single Lance primitive | [writes.md](writes.md), [architecture.md](architecture.md) | | Constructive mutations | In-memory `MutationStaging`, one end-of-query table commit per touched table, then one manifest publish | [writes.md](writes.md), [execution.md](execution.md) | | Deletes | Inline-commit residual; delete-only queries allowed, mixed insert/update/delete rejected by D2 | [query-language.md](../user/query-language.md), [writes.md](writes.md) | +| Branch delete | Manifest is the single authority, flipped atomically first; per-table forks + commit-graph branch are derived state, reclaimed best-effort (`force_delete_branch`) with the `cleanup` reconciler as the guaranteed backstop. Reusing a name whose reclaim failed before `cleanup` surfaces an actionable error | [branches-commits.md](../user/branches-commits.md), [maintenance.md](../user/maintenance.md) | | Schema validation | Type checks, required fields, defaults, edge endpoint checks, and edge cardinality are enforced on write paths | [schema-language.md](../user/schema-language.md), [execution.md](execution.md) | | Unique constraints | Intra-batch and write-path checks exist; full cross-version uniqueness is still a gap | [schema-language.md](../user/schema-language.md) | | Storage trait | `TableStorage` exists as the sealed staged-write surface; full call-site migration and capability/stat surfaces are incomplete | [writes.md](writes.md), [architecture.md](architecture.md) | @@ -107,6 +108,13 @@ Use it this way: | Auth | Bearer token hashing and server-side actor resolution are implemented at the HTTP boundary | [server.md](../user/server.md), [policy.md](../user/policy.md) | | Tests | Tempdir-backed Lance tests are the current substrate; there is no `MemStorage` test backend | [testing.md](testing.md) | +The branch-delete reconciler is authority-derived: it reclaims orphaned forks +today and degrades to a no-op if Lance ships an atomic multi-dataset branch +operation, so the design composes with that future rather than blocking it. This +is the same shape as invariant 7 (indexes are derived state); prefer it over a +recovery-sidecar-style approach for any new multi-dataset metadata operation, +since the sidecar would be scaffolding to remove once the substrate closes the gap. + ## Known Gaps Do not hide these behind invariant wording. Either move them forward or keep diff --git a/docs/dev/lance.md b/docs/dev/lance.md index ef83f2c..100da6f 100644 --- a/docs/dev/lance.md +++ b/docs/dev/lance.md @@ -175,7 +175,8 @@ Migration from Lance 4.0.0 → 6.0.1 landed in this cycle (DataFusion 52 → 53, - **Lance #6658 closed** (2026-05-14) but `DeleteBuilder::execute_uncommitted` did **not** ship in v6.0.1 — binary search across the release stream shows it first appears in `v7.0.0-beta.10` (the closing commits landed on main but didn't backport to the 6.x line). Tracked as MR-A: migrate `delete_where` to staged, retire the parse-time D2 mutation rule, extend recovery sidecar coverage. **Gated on the Lance v7.x bump**, not this PR. v7.0.0-rc.1 dropped 2026-05-21. - **Lance #6666 still open** (`build_index_metadata_from_segments` public): vector-index two-phase blocked; inline `create_vector_index` residual retained. - **Lance #6877 still open** (`MergeInsertBuilder` dup-rowid): PR #109's `SourceDedupeBehavior::FirstSeen` + `check_batch_unique_by_keys` precondition stay load-bearing. +- **`Dataset::force_delete_branch`** (`branches().delete(name, force=true)`, dataset.rs:524) tolerates a missing branch-*contents* ref (vs plain `delete_branch`'s `RefNotFound`), but on the local store still errors `NotFound` if the branch `tree/` directory is fully absent (`remove_dir_all`'s NotFound is not caught for Lance's native error variant, refs.rs:526-549). Both variants still refuse a branch with referencing descendants (`RefConflict`). `TableStore::force_delete_branch` wraps this to be fully idempotent (tolerates already-absent). The single-authority branch-delete redesign uses it for orphan reclamation (eager best-effort reclaim + cleanup reconciler). Pinned by `lance_surface_guards.rs::force_delete_branch_semantics`. Branch delete is "flip the ref atomically, then `remove_dir_all(tree/{branch})`"; branch-exclusive data lives under `tree/{branch}/` so a drop reclaims it immediately without touching `main`. -Surface guards added: `crates/omnigraph/tests/lance_surface_guards.rs` (8 named guards; 3 runtime + 5 compile-only). Future Lance bumps re-run this file first as the smoke check. Two additional guards from the original plan deferred to follow-up (`manifest_cas_returns_row_level_contention_variant` needs full publisher-race harness; `table_version_metadata_byte_compatible_with_v4` needs `pub(crate)` reach extension). +Surface guards added: `crates/omnigraph/tests/lance_surface_guards.rs` (9 named guards; 4 runtime + 5 compile-only). Future Lance bumps re-run this file first as the smoke check. Two additional guards from the original plan deferred to follow-up (`manifest_cas_returns_row_level_contention_variant` needs full publisher-race harness; `table_version_metadata_byte_compatible_with_v4` needs `pub(crate)` reach extension). Bump this date stanza on the next alignment pass. diff --git a/docs/user/branches-commits.md b/docs/user/branches-commits.md index de6c653..c1894f9 100644 --- a/docs/user/branches-commits.md +++ b/docs/user/branches-commits.md @@ -8,10 +8,10 @@ Lance supports branching at the dataset level: a branch is a named lineage of ve OmniGraph builds *graph branches* on top by branching every sub-table coherently: -- `branch_create(name)` / `branch_create_from(target, name)` — disallowed name `main`; fails if branch exists; ensures the schema-apply lock is idle. +- `branch_create(name)` / `branch_create_from(target, name)` — disallowed name `main`; fails if branch exists; ensures the schema-apply lock is idle. Atomic and authority-first like `branch_delete`: it flips the `__manifest` branch (authority), then creates the derived commit-graph branch, force-dropping any orphaned commit-graph ref left by an incomplete prior delete (the manifest branch is fresh, so a same-named commit-graph branch is provably a zombie). If commit-graph creation fails, the manifest branch is rolled back so the name never half-exists. - `branch_list()` — returns public branches, **filters internal** `__run__…` and `__schema_apply_lock__` prefixes. -- `branch_delete(name)` — refuses if there are descendants or active runs on the branch; cleans up owned per-branch fragments. -- **Lazy forking**: a branch only forks a sub-table when that sub-table is first mutated on it. Pure-read branches share fragments with their source. +- `branch_delete(name)` — refuses if there are descendants or active runs on the branch. The manifest is the single authority for branch existence: deletion flips the `__manifest` branch ref first (one atomic op), after which the branch is gone from every snapshot. The owned per-table forks and the commit-graph branch are derived state, reclaimed best-effort with `force_delete_branch` after the flip. A failure during that reclaim (transient object-store error) does not fail the call or block the authority flip; the leftover forks are unreachable orphans that the [`cleanup`](maintenance.md) reconciler converges. One consequence: if a delete's best-effort reclaim fails, reusing that branch name before the next `cleanup` surfaces a clear error pointing at `cleanup` (the stale fork would otherwise collide on first write). +- **Lazy forking**: a branch only forks a sub-table when that sub-table is first mutated on it. Pure-read branches share fragments with their source. A fork collision is classified by the manifest authority, not by Lance branch versions: if the live manifest already records the fork on the active branch, a concurrent first-write won and the caller gets a retryable "refresh and retry"; if the manifest does not, a physical branch there is an orphan and the caller is pointed at `cleanup`. - `sync_branch(branch)` — re-binds the in-memory handle to the latest head of the branch. ## L2 — Commit graph (`db/commit_graph.rs`) diff --git a/docs/user/maintenance.md b/docs/user/maintenance.md index 08ae8da..9839ea1 100644 --- a/docs/user/maintenance.md +++ b/docs/user/maintenance.md @@ -14,9 +14,15 @@ - Lance `cleanup_old_versions()` per table. - Removes manifests (and their unique fragments) older than the retention policy. - `CleanupPolicyOptions { keep_versions: Option, older_than: Option }` — at least one is required. -- Returns `[TableCleanupStats { table_key, bytes_removed, old_versions_removed }]`. +- Returns `[TableCleanupStats { table_key, bytes_removed, old_versions_removed, error }]`. +- **Fault-isolated per table.** A single table's transient failure (version GC or + orphan reclaim) is recorded on that table's stats row (`error: Some(..)`, logged + via `tracing`) and never aborts the healthy tables — cleanup is the convergence + backstop, so it does as much as it can and converges on re-run. The CLI reports + any failed tables; rerun `cleanup` to retry them. - CLI guards with `--confirm`; without it, prints a preview line. - **Recovery floor:** `--keep < 3` may garbage-collect Lance versions that the open-time recovery sweep needs as a rollback target (the sweep restores to the branch's manifest-pinned table version, which is HEAD-1 in the typical Phase B → Phase C drift case). Default `--keep 10` is safe. +- **Orphaned-branch reconciliation:** before the version GC, cleanup runs `reconcile_orphaned_branches`, which `force_delete_branch`es any per-table or commit-graph Lance branch absent from the manifest branch list. These orphans arise when a `branch_delete` flips the manifest authority but a downstream best-effort reclaim does not complete (see [branches-commits.md](branches-commits.md)). The reconciler is authority-derived and idempotent (it no-ops once nothing is orphaned), runs regardless of the `keep_versions` / `older_than` values (those gate version GC only), and never reclaims `main` or system-branch forks. Reclaimed forks are logged via `tracing::info`. ## Tombstones From fab105bcce97ce6eb543d83c723fa3c4befb8bb5 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler Date: Mon, 1 Jun 2026 12:56:21 +0100 Subject: [PATCH 013/165] fix(release): generate audit-clean Homebrew formula (#134) The generated formula failed `brew audit --strict` with 5 problems: `version` declared after `license`, and `url`/`sha256` placed directly inside `on_macos`/`on_linux` (forbidden by FormulaAudit/ComponentsOrder). Order `version` before `license`, hoist `head`/`livecheck` above the platform blocks, and nest `url`/`sha256` in `on_arm`/`on_intel`. Add a `brew audit --strict --online` gate to the release workflow so a malformed formula can never be published again. Verified clean against v0.6.0. Co-authored-by: Claude Opus 4.8 (1M context) --- .github/workflows/release.yml | 11 +++++++++++ scripts/update-homebrew-formula.sh | 29 ++++++++++++++++------------- 2 files changed, 27 insertions(+), 13 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 48ab38c..3a66ff2 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -121,6 +121,17 @@ jobs: run: | ./scripts/update-homebrew-formula.sh "${GITHUB_REF_NAME}" homebrew-tap/Formula/omnigraph.rb + - name: Audit generated formula + if: env.HOMEBREW_TAP_SKIP != '1' + run: | + # Audit the checked-out tap by name (brew audit rejects bare paths + # and needs tap context). Symlink the checkout into Homebrew's Taps + # tree so `modernrelay/tap/omnigraph` resolves to it. + tap_dir="$(brew --repository)/Library/Taps/modernrelay/homebrew-tap" + mkdir -p "$(dirname "$tap_dir")" + ln -sfn "$PWD/homebrew-tap" "$tap_dir" + brew audit --strict --online modernrelay/tap/omnigraph + - name: Commit and push formula update if: env.HOMEBREW_TAP_SKIP != '1' working-directory: homebrew-tap diff --git a/scripts/update-homebrew-formula.sh b/scripts/update-homebrew-formula.sh index 90a5dea..f2f0df9 100755 --- a/scripts/update-homebrew-formula.sh +++ b/scripts/update-homebrew-formula.sh @@ -64,20 +64,8 @@ cat >"$FORMULA_PATH" < Date: Mon, 1 Jun 2026 22:50:31 +0200 Subject: [PATCH 014/165] Stored-query registry foundation + config/CLI RFC-002 (#128) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * MR-969: add stored-query registry config surface Introduce the `queries:` block in omnigraph.yaml — an inline `name -> entry` map of stored queries, per-graph (`graphs..queries`) and top-level for single-graph mode, mirroring how `policy` is wired in both modes. Each entry points at a `.gq` file and carries optional MCP exposure settings (`expose`, `tool_name`), defaulting to not-exposed. Additive: absent `queries:` leaves current behavior unchanged. - QueryEntry { file, mcp: McpSettings { expose, tool_name } } - `queries` field on TargetConfig + OmnigraphConfig (serde default) - query_entries() / target_query_entries() accessors - resolve_query_file() — base_dir-relative `.gq` path resolution - round-trip + absent-block tests Co-Authored-By: Claude Opus 4.8 (1M context) * Add stored-query registry loader and GraphHandle wiring Add a `queries` module: QueryRegistry loads each declared `.gq` entry, parses it, and selects the query whose symbol matches the manifest key, asserting the two agree (key == `query ` symbol). Identity is the query name; a key/symbol mismatch is a load-time error. Errors are collected, not fail-fast, so a bad registry surfaces every broken entry at once. Schema type-checking is deliberately left to a separate pass so the loader stays callable without an open engine. Thread an `Option>` through GraphHandle alongside the per-graph policy; the URI-canonicalizing clone propagates it. Production openers default to None for now — the boot path loads and attaches the registry in a later change. - QueryRegistry::{from_specs, load, lookup, iter}; StoredQuery::is_mutation - GraphHandle.queries field, propagated on canonical clone - registry unit tests: identity match/mismatch, multi-query selection, per-entry parse errors, error collection, mutation classification Co-Authored-By: Claude Opus 4.8 (1M context) * docs: add RFC-002 config & CLI architecture Layered config (user-global ~/.config/omnigraph/ + per-project), a unifying `target` abstraction resolving to (locus, graph, sub-state, credential) with embedded-URI XOR remote-server loci, multi-server × multi-graph client targeting, credentials by-reference, and the file-naming decision: project and server config are one artifact (`omnigraph.yaml`); the only differently-named file is the user-global `config.yaml`, split by scope not role. Includes the 12-factor bind portability rule (prefer --bind/OMNIGRAPH_BIND over a committed server.bind) and the defined-locally / invoked-remotely model for stored queries. Derived from first principles working backwards from what the engine enables; validated against kube/Helix/git/compose. Linked from docs/dev/index.md. Proposed; phased rollout for the MR-973/974/981 family. Co-Authored-By: Claude Opus 4.8 (1M context) * Add check() to validate stored queries against the live schema A pure check(registry, catalog) that type-checks every stored query via the same typecheck_query_decl the engine runs for inline queries — no parallel implementation. Failures are collected, not fail-fast, so an operator sees every broken query (e.g. a type/property a migration renamed or removed) in one pass. Breakages are fatal (the boot path will refuse to start); warnings are advisory. Pure over (registry, catalog) so it is callable both at boot (engine catalog) and offline from the CLI without an open engine. Advisory lint: an mcp.expose:true query that declares a Vector(N) parameter warns — an LLM cannot supply a raw embedding vector; such a query should take a String parameter and embed server-side. Warns rather than rejects, since service-to-service callers may pass vectors. - CheckReport { breakages, warnings }; has_breakages / is_clean - tests: valid query, unknown type, unknown property, collect-not-fail-fast, vector-param-exposed warns, unexposed silent Co-Authored-By: Claude Opus 4.8 (1M context) * Drop internal plan-label refs from stored-query config comments Doc comments referenced sequencing labels ("C2") that mean nothing to a reader; reword to describe the behavior directly. Comment-only. Co-Authored-By: Claude Opus 4.8 (1M context) * docs: reconcile aliases with the role model in RFC-002 Place the existing client-only `aliases:` block in the client/server role split: aliases are client-role (CLI, embedded, ungated) and may live in both user-global and project config; `queries:` is server-role (deployment manifest only). They overlap as "name -> .gq"; `queries:` is the superset, and the end-state subsumes aliases (definition -> queries, target/branch/format -> client invocation context, positional args -> CLI sugar). v1 keeps aliases unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) * docs: make RFC-002 config global-first, project-optional The global user config is the primary, self-sufficient default; the CLI works from any directory with no project file (the kubectl/aws/gh posture), a deliberate flip from today's project-anchored behavior. The project omnigraph.yaml becomes an optional repo-scoped override and the deployment manifest. Uniform schema, both layers optional; global can hold any section including a personal server's graphs/queries. Additive: project still overrides global; the flip adds a fallback layer below the project file rather than removing it. Co-Authored-By: Claude Opus 4.8 (1M context) * docs: justify XDG ~/.config/omnigraph over legacy ~/.omnigraph in RFC-002 Make the rationale explicit: XDG-first because OmniGraph is a client that will cache remote catalogs and keep session state alongside secrets, and XDG separates config / cache / state into distinct dirs (clear cache without touching creds; backups skip cache) whereas a single ~/.omnigraph/ mixes them. Honor ~/.omnigraph/ as a fallback for the peer-group (aws/kube/docker/helix) expectation. Add XDG_CACHE_HOME / XDG_STATE_HOME to the override precedence. Co-Authored-By: Claude Opus 4.8 (1M context) * docs: build RFC-002 credentials on the existing env-file mechanism OmniGraph already has credentials-by-reference: bearer_token_env names the env var, and auth.env_file is a git-ignored dotenv the CLI auto-loads (real env vars win), resolved via resolve_remote_bearer_token. The RFC's proposed credentials.yaml + token_env were redundant parallel inventions. Reconcile: reuse bearer_token_env (extend to servers.) and auth.env_file (add a global ~/.config/omnigraph/.env layered under the project .env.omni); OS keychain is an additive future resolver. No new credentials.yaml. Updated summary, non-goals, background, file-naming, credentials, example, login, migration, rollout. Co-Authored-By: Claude Opus 4.8 (1M context) * docs: use single ~/.omnigraph dir (Helix-style), not XDG, in RFC-002 Reverse the earlier XDG-first call. The prior argument rested on a false dichotomy (single-dir => mixed config/cache/state); in fact the peer tools (aws, kube, helix) achieve separation via SUBDIRECTORIES inside one ~/.tool/ dir (~/.aws/sso/cache/, ~/.kube/cache/), getting cache hygiene AND one discoverable place. So everything goes under ~/.omnigraph/: config.yaml, credentials (dotenv, 0600), cache/, state/. Lower cognitive load, matches what DB/cloud-CLI users expect, matches Helix. OMNIGRAPH_HOME overrides; $XDG_CONFIG_HOME optionally honored but ~/.omnigraph/ is canonical. Updated all paths, the rationale paragraph, the file-naming table (added a cache/state row), and env precedence. Co-Authored-By: Claude Opus 4.8 (1M context) * docs: reconcile RFC-002 with shipped/planned CLI tickets Align with reality found in existing tickets: - Noun is graph/graphs, not target/targets (MR-603 done renamed the config key targets->graphs, flag --graph). Use graphs:/--graph; an entry is embedded (uri) XOR remote (server + remote graph name). - ~/.omnigraph/ confirmed by MR-581 (og template pull, done) which already quick-starts templates there. - Templates already exist (MR-581/MR-531) — not invented here. - The init family is already specced (init, quickstart MR-973, serve MR-970, prune MR-972, mcp install MR-974, agent-mode MR-981); this RFC only adds the user route (~/.omnigraph/config.yaml + login). - aliases: -> operations: planned (MR-839). - bearer_token_env gap tracked in MR-971. - query lint/check already exist (MR-639) — registry validator must not collide with the singular `query check`. Add a Reconciliation section; fix the canonical example to graphs:/--graph. Also: merge semantics refined (deep-merge settings, replace named entries, replace lists, config view --resolved --show-origin). Co-Authored-By: Claude Opus 4.8 (1M context) * docs: correct stale-ticket claims and fold init/bootstrap design into RFC-002 Verify against code, not ticket statuses (MR-581 is marked done but is stale/unbuilt): no ~/.omnigraph usage, no template/serve/quickstart/ prune/login commands exist; config still uses aliases: (no operations:). So ~/.omnigraph/ stands on peer-convention merits alone, and templates are a design question, not a foothold. Add §7.5: the three-tier init model (user route = login + ~/.omnigraph/config.yaml; thin project init; fat quickstart + templates) with first-principles positions (split init/login, in-place refuse-if-exists, interactive vs --auto/agent-mode, --template flag, secrets-on-scaffold gitignore rule). This RFC owns only the user route; the rest are sibling tickets (MR-973/970/972/974/981). Co-Authored-By: Claude Opus 4.8 (1M context) * docs: breadboard + slice Shape A in RFC-002 Add the implementation breadboard (places P1-P5, affordances N1-N14 with NEW markers, mermaid) and five vertical slices for the selected config/ CLI/init shape: V1 global layer + merge engine + config view; V2 remote graphs + HTTP-client path + credential resolution; V3 omnigraph login; V4 init-hardening + quickstart + templates (rides MR-970); V5 agent-mode (MR-981). Rollout reordered to the slice sequence; spikes X1-X4 gate their owning slice. V1-V2 close the substantive client->server gap. Co-Authored-By: Claude Opus 4.8 (1M context) * Add InvokeQuery Cedar action (coarse, graph-scoped) A per-graph, branch-scoped action that gates invoking a server-side stored query by name. Coarse for now: an `invoke_query` allow rule permits any stored query on the graph; a future, additive refinement adds an optional per-query-name scope without changing rules written against the coarse action. Enforcement is at the HTTP boundary; the engine `_as` writers still enforce read/change per the query body, so a stored mutation is double-gated (invoke_query to reach the tool, change for the write). No call site yet — the invocation handler wires it in a later change (same pattern as Admin/GraphList added ahead of consumers). - variant + as_str/resource_kind(Graph)/FromStr/uses_branch_scope - Cedar schema: invoke_query appliesTo Graph - tests: per-graph allow/deny, branch-scope accepted Co-Authored-By: Claude Opus 4.8 (1M context) * Load and type-check stored queries at server boot, refusing breakage At startup the server now loads each graph's stored-query registry, type-checks every query against that graph's live schema, and refuses to boot if any query references a type/property the schema doesn't have (same posture as bad policy YAML) — so schema drift surfaces at the deploy boundary, not silently at invocation. Non-blocking warnings are logged. The validated registry is attached to the GraphHandle (the two production sites previously held `queries: None`). Loading (parse + key==symbol identity) happens at settings-build time where the config is in scope; the schema type-check happens after each engine opens (single mode in `open_single_with_queries`, multi mode in `open_single_graph`). `open_with_bearer_tokens_and_policy` delegates with an empty registry so its 18 test callers are unchanged; the public `new_*` constructors are unchanged (only the private build path threads the registry). - ServerConfigMode::Single / GraphStartupConfig carry the loaded registry - boot tests: valid registry boots; type-broken query refuses boot + names it Co-Authored-By: Claude Opus 4.8 (1M context) * Add `omnigraph queries validate` and `queries list` CLI `queries validate` type-checks the stored-query registry against the live schema offline — it opens the selected graph, runs the same check() the server runs at boot, prints breakages/warnings (human or --json), and exits non-zero on any breakage — so an operator can catch a query broken by a schema change without restarting the server. `queries list` prints each registered query's name, MCP exposure, and typed params. Named `validate` (not `check`) to avoid overlap with the existing `omnigraph lint` — `query check`/`query lint` are already deprecated argv-shims to `lint`. Registry entries resolve like the server: a named graph uses its per-graph `queries:`; otherwise the top-level one. - Queries subcommand group; reuses QueryRegistry::load + check from omnigraph-server; local-only (needs the schema), mirrors lint - tests: clean registry exits 0, broken query exits non-zero + names it, list shows the query and its typed params Co-Authored-By: Claude Opus 4.8 (1M context) * Route registry selection through one shared query_entries_for The "which queries: block applies for graph X" rule existed twice — the server boot path and the CLI's registry_entries — and had already drifted: the CLI carried an unreachable unwrap_or_else fallback the server lacked. Add OmnigraphConfig::query_entries_for(graph: Option<&str>) as the single definition (named graph -> its per-graph block; otherwise top-level) and route all three sites through it: server single mode, server multi-graph loop, and the CLI. The CLI's dead fallback arm is deleted; CLI and server now resolve identically by construction. No behavior change. Extends the config round-trip test to pin the selector, including the unknown-name -> top-level fallback the deleted CLI arm covered. * Funnel registry validation through one validate_and_attach gate The check -> refuse-on-breakage -> log-warnings -> empty->None block was copy-pasted across both open paths (single mode and the multi-graph per-graph open), differing only by the graph label. A third opener could attach a registry that was never schema-checked. Extract validate_and_attach(queries, catalog, label) -> Option> as the single gate both paths call, so attaching an unchecked registry is no longer expressible. The catalog handle is an owned Arc, so calling it before the multi-mode policy match (which rebinds db) is borrow-clean. No behavior change. Adds a direct unit test of the helper (empty / clean / breakage incl. the graph label in the message) — covering the multi-graph path's logic, which previously had no boot-refusal coverage. * Resolve param types structurally in the MCP vector lint The exposed-query advisory detected vector params with type_name.starts_with("Vector(") — a second copy of the compiler's own ScalarType::from_str_name vector parsing that could drift from it. Key the lint off PropType::from_param_type_name + ScalarType::Vector(_) instead, the one canonical resolver the type system already uses. Any future param-suppliability lint now reads the structured type rather than scanning the surface string. Behavior-preserving: the grammar forbids list-of-vector params (list_type = "[" base_type "]", and base_type excludes Vector), so the only input where the structured and string checks could differ is unparseable. Adds a guard test that an exposed String param does not false-trigger the warning. * Refuse duplicate MCP tool names across exposed stored queries The effective MCP tool name (explicit tool_name, else the query name) is a second identity namespace beside the registry key, but nothing enforced it unique — two exposed queries could claim one catalog key, and each consumer re-derived the name ad hoc. Add StoredQuery::effective_tool_name() as the one definition, and a load-time uniqueness pass in from_specs over exposed queries: a collision is a collected LoadError naming the loser and the winner. Scoped to exposed queries (unexposed have no MCP tool); deterministic over the BTreeMap so the first-declared wins and the error order is stable. New (rare) refusal: a config with colliding exposed tool names now fails `omnigraph queries validate` offline and refuses server boot, the same posture as a malformed registry. Release-note-worthy. Test-first: duplicate_exposed_tool_name_is_a_load_error (red before the pass, green after) + a CLI offline test; the unexposed sibling pins the exposed-only scope; effective_tool_name asserts folded into the load test. * docs: document the queries registry, CLI, and invoke_query action The stored-query surface shipped without user docs. Add it, per the same-PR maintenance contract: - policy.md: invoke_query as per-graph action #10 (branch-scoped), with the double-gating note; renumber graph_list; add it to the branch_scope list. - cli-reference.md: the `queries validate | list` command, and the `queries:` config block (per-graph + top-level) with mcp.expose/tool_name and the tool-name uniqueness rule. - server.md: boot-time stored-query type-check (refuse on breakage), noting invocation over HTTP/MCP is not yet exposed. * Add POST /queries/{name} stored-query invocation handler Invoke a curated server-side stored query by name: source + name come from the per-graph queries: registry, the client sends only runtime inputs (params, branch, snapshot). Gated by the invoke_query Cedar action at the boundary; the handler delegates to the existing run_query/run_mutate, whose inner Read/Change enforce still runs — so a stored mutation is double-gated (invoke_query to reach the tool, change for the write). - InvokeStoredQueryRequest + an untagged InvokeStoredQueryResponse { Read(ReadOutput), Change(ChangeOutput) } → one Json<_> return type and a oneOf 200 schema (a correct contract, not a wrong-but-simple one). - Route lives in per_graph_protected → single-mode /queries/{name} and multi-mode /graphs/{id}/queries/{name} for free. - Deny == unknown: an invoke_query denial and a missing query both return the same 404, so the catalog can't be probed by an unauthorized caller. - OpenAPI regenerated; tests cover read, mutation double-gate (403 vs 200), bad-param 400, and the identical-404 deny path. Completes the MR-969 V1 invocation slice (registry + /queries/{name} + invoke_query). * docs: stored-query invocation endpoint; flip the not-yet-exposed caveat Now that POST /queries/{name} ships (C7), document it: add the endpoint to server.md's inventory + an invocation section (body, untagged read/mutate envelope, invoke_query gate, double-gated mutations, deny == 404), and flip the startup note that said invocation was not yet exposed. In policy.md, replace "no invocation call site yet" on the invoke_query action with a pointer to the endpoint. * Scope the stored-query 404-hiding claim to non-invoke_query callers Review found the deny==404 catalog-hiding was overstated as a contract: it holds only at the outer invoke_query gate. A caller that HOLDS invoke_query but lacks read/change gets the inner gate's 403 for an existing query vs 404 for an unknown one — so existence is visible to grant-holders by design (the intended double-gate). The handler docstring, OpenAPI 404 description, and server.md all claimed the 404 was airtight against any denied actor. Correct the wording in all three (no behavior change) and add the missing symmetric test (invoke_query but no read -> 403 for an existing query, 404 for unknown) so the actual contract is pinned. Also document that in default-deny mode (tokens, no policy) every invocation 404s until an invoke_query rule is configured. Nits: the from_specs collision comment said "first declared wins" but it is lexicographically-first by name (BTreeMap); the effective_tool_name docstring overclaimed the CLI display routes through it (it resolves the rule on its own output DTO). * Default mcp.expose to true (the manifest entry is the opt-in) expose controls MCP-catalog membership only — it is not an authorization gate (invocation is gated by invoke_query regardless). So requiring a per-query mcp.expose: true was friction with no safety benefit: a non-exposed query is still HTTP-invocable by name. Flip the default so declaring a query in the manifest exposes it to the agent tool catalog by default; expose: false is the escape hatch for service-only queries. Both the absent-mcp path (Default impl) and the present-but-no-expose path (serde default fn) now yield true. Doc comments + cli-reference updated; the config round-trip test asserts the new default. * Add GET /queries stored-query catalog endpoint List a graph's mcp.expose stored queries as a typed tool catalog so a client (the MCP server) can register them as tools without fetching .gq source. Each entry carries name, MCP tool_name, description/instruction, a read/mutate flag, and decomposed typed params (kind enum: string|bool|int| bigint|float|date|datetime|blob|vector|list, plus item_kind for lists and vector_dim) — so the consumer builds an input schema with a closed match and never re-parses omnigraph type spelling. I64/U64 are bigint (string on the wire): a JSON number loses precision past 2^53 and the engine already accepts decimal strings. Read-gated (works in default-deny; the catalog is graph-wide, authorized against main). NOT Cedar-filtered per query yet — a reader can list a query whose invoke_query they lack (documented gap until per-query authz lands); invocation stays invoke_query-gated + deny==404. - api: QueriesCatalogOutput / QueryCatalogEntry / ParamDescriptor / ParamKind + query_catalog_entry (reuses PropType::from_param_type_name; scalar_kind is exhaustive, so a new ScalarType is a compile error here until catalogued). - GET /queries route in per_graph_protected (→ /graphs/{id}/queries in multi mode); OpenAPI regenerated; path allowlists updated. - Tests: projection unit (every kind, list, vector, nullable, mutation, empty) + handler (exposed-only filter, read-gate probe-oracle, empty registry). * docs: GET /queries stored-query catalog endpoint Document the catalog: the endpoint table row (GET /queries, read-gated), a catalog section (typed-param kind enum, bigint/date/datetime/blob-as-string, graph-wide/branch-independent, mcp.expose default true, the read-gated probe-oracle gap), and flip the startup note now that the catalog ships. * Collect file-I/O and parse errors in QueryRegistry::load in one pass load() early-returned on any unreadable .gq file, masking parse / identity / tool-name-collision errors in the OTHER (readable) files — so an operator fixed the missing file, restarted, and only then saw the next broken query. Now it collects I/O errors but still runs from_specs on the readable specs and returns the union, so every broken entry surfaces at once (matching the collected-errors contract the rest of the registry already follows). Safe: from_specs' tool-name collision check runs over loaded queries only, so dropping an I/O-failed entry can only under-report a collision, never invent one. I/O errors are ordered first (BTreeMap key order), then spec errors. Adds a load-level test (tempdir: a valid, a missing, and a parse-broken .gq) asserting all three surface in one Err — confirmed red before the fix. * Make invoke_query graph-scoped (one branch authority) invoke_query gates reaching the curated stored-query surface — a graph-level capability. Per-branch/snapshot access is already enforced by the inner read/change gate in run_query/run_mutate (authorized against the resolved branch), so branch-scoping the outer gate was redundant AND wrong for snapshot reads (it defaulted to main). Drop the branch dimension: remove InvokeQuery from uses_branch_scope (it joins admin as graph-scoped) and authorize the boundary gate with branch: None. Lossless: an actor confined to branch X by their read/change rules can still only invoke a stored query that touches X. A rule that sets branch_scope on invoke_query is now rejected by validate() — write invoke_query in its own rule. Ripple (atomic): restructure the server invoke fixture so invoke_query sits in its own branch_scope-free rule; invert invoke_query_is_branch_scoped -> invoke_query_rejects_branch_scope; the per-graph authorize test uses branch: None; docs (policy.md, server.md, the InvokeQuery doc). No wire/OpenAPI change. * Resolve graph config by identity, not server mode Which policy/queries block applies for a graph was decided three different, mode-dependent ways: single-mode boot used top-level even for a named graph; multi-mode used per-graph (and silently ignored a top-level queries block); the CLI used per-graph for a named target. So `queries validate --target prod` could check a different registry than the single-mode server loaded, and a named graph's per-graph policy/queries were silently shadowed. Make config a function of graph IDENTITY: a graph served by NAME (--target/server.graph, a graphs: entry) uses its own graphs..{policy, queries}; a bare URI is anonymous and uses top-level. One rule, applied by single-mode boot, multi-mode boot, and the CLI — so they can't diverge and the CLI predicts the server exactly. No silent ignore: serving a named graph while a top-level policy/queries block is populated now refuses boot, naming the block (the multi-mode top-level-policy bail, extended to queries and to single-mode-named). The CLI's `queries validate` derives the schema URI and the registry from ONE selection, and a positional URI forces anonymous (ignoring cli.graph) so the two can't come from different graphs. BREAKING (released behavior): single mode by name (--target/server.graph) with top-level policy/queries previously used top-level; it now uses the per-graph block and refuses boot if top-level is also populated. Bare-URI single mode is unchanged. Loud, with migration text pointing at graphs.. - config: resolve_policy_file_for (policy sibling of query_entries_for, no top-level fallback) + populated_top_level_blocks for the coherence check. - characterization tests (single-mode named -> per-graph; named + top-level -> bail; multi-mode top-level queries -> bail; CLI positional-URI -> top-level). - docs: policy.md, server.md, cli-reference.md. * docs: RFC-002 credentials keyed by server name (keychain/profile/env) Reworks the RFC's credentials model: secrets are keyed by server name — OS keychain `omnigraph:` (preferred) -> a `[]` profile in `~/.omnigraph/credentials` -> `OMNIGRAPH_TOKEN[_]` env (CI), the AWS/gh/kube model. `servers.` is endpoint-only by default but may carry an explicit, secret-free `auth: { token: { env|file|command|keychain } }` source. The shipped `bearer_token_env` + `.env.omni` dotenv remain a legacy compat path; no `credentials.yaml`. * docs: RFC-002 — typed graph locator (storage/server/graph_id), not a uri string Add §1.1: the resolved graph address is a typed GraphLocator (Embedded{storage} | Remote{server, graph_id}), not a flat uri: String. Diagnoses the string model's cost in the code today (~16 is_remote_uri forks, TargetConfig can't express multi-server x multi-graph, the CLI bails on remote, the ts SDK models baseUrl+graphId separately) and settles the YAML naming so the key names the locus: - storage: (embedded) — shipped uri: is a deprecated alias - server: + graph_id: (remote) — graph_id defaults to the entry key - storage xor server, reject both/neither (no silent ambiguity) Kills the graphs:/graph: collision and the uri:-might-be-a-server ambiguity. Updates the §1/§8 examples and the entry-shape notes to the new naming. * Test: queries list must reject an unknown --target queries list opens no graph URI, so unknown-graph validation does not ride along on resolve_target_uri the way it does for every other command. The new test reproduces the gap: with an unknown --target the command currently exits 0 and prints the (empty) top-level registry instead of erroring like the URI-resolving commands do. Fails against current code; the fix follows. * Validate the graph selection in queries list Graph-existence validation was a side effect of URI resolution: every URI-resolving command rejects an unknown --target via resolve_target_uri, but queries list opens no URI, so query_entries_for(Some(unknown)) silently fell back to the top-level registry and showed the wrong (or empty) catalog. Make membership a property of the selection: add the fallible resolve_graph_selection alongside the infallible query_entries_for (a known name passes through, an unknown name errors with the same message as resolve_target_uri, None stays anonymous), and validate the selection in execute_queries_list. query_entries_for is unchanged — server boot's bare-URI path still needs its None -> top-level arm. * Surface policy-engine errors from stored-query invoke The invoke handler mapped every authorize_request failure to 404 ('stored query not found'), which collapsed the authorization decision (deny -> 403) together with operational failures (no actor -> 401, Cedar evaluation error -> 500). A real policy-engine 500 was hidden as a missing query. Separate the two concerns instead of sniffing the masked status. Extract authorize() returning an Authz { Allowed, Denied(msg) } decision and reserve Err for operational failures only; authorize_request becomes a thin wrapper that maps Denied -> 403, so the 16 deny-as-403 callers are unchanged. The invoke handler now matches the decision directly: a denial stays 404 (deny == missing, so the catalog can't be probed without the grant), while a 401/500 propagates with its true status. 500 is now a reachable outcome on POST /queries/{name}; document it in the endpoint responses and regenerate openapi.json. * Extract the named-graph/top-level coherence rule into one helper The rule 'a named graph uses its own graphs. block, so a populated top-level block is a config error' lived inline in single-mode server boot. Extract it to OmnigraphConfig::ensure_top_level_blocks_honored so the same definition can be shared by the CLI selection gate (next commit) and the two can't drift. Boot calls the helper; the message is reworded context-neutral (drops 'serving') so it reads correctly from both boot and the CLI. Behavior-preserving: multi-graph mode keeps its own unconditional check, and single_mode_named_graph_rejects_top_level_blocks still passes. * Test: queries validate/list must reject a named graph with a top-level block Server boot refuses a config where a graph is selected by name yet a top-level queries:/policy.file block is populated (the block would be silently ignored). The CLI's queries validate/list resolve the same named selection but skip that coherence check, so they give a false green / list the per-graph block. The new test reproduces it: validate prints OK and list succeeds where boot would refuse. Fails against current code; the fix follows. * Enforce top-level coherence in the single CLI selection gate queries validate validated graph membership only as a side effect of URI resolution and queries list only via resolve_graph_selection's membership check; neither applied the named-graph/top-level coherence rule server boot enforces, so both gave a false green on a config boot refuses. Fold ensure_top_level_blocks_honored into resolve_graph_selection so it is the single gate that returns only valid + server-coherent selections, and route resolve_selected_graph (queries validate) through it; queries list already calls the gate. A named graph with a populated top-level block now errors in both commands, matching boot. A positional URI stays anonymous (top-level honored), so queries_validate_positional_uri_ignores_default_graph is unaffected. * docs: RFC-003 — MCP server surface for omnigraph-server Detailed MCP-transport design for the stored-query/MCP work, building on the shipped #128 registry. Corrects the draft against the branch head: the coarse invoke_query gate + 404 denial-masking are already wired (server_invoke_query), so per-query invoke_query scope (PolicyRequest has no query-name dimension yet) is the real prerequisite; positions the doc as superseding rfc-001's MCP transport (/mcp/tools+/mcp/invoke) and reconciles the shipped mcp.expose YAML form and the schema-introspection non-goal; grounds the parity surface in the actual omnigraph-ts package (13 tools with read/change ids, 2 resources). * docs(config): clarify graph config boundaries * fix(config): enforce graph-scoped policies and query validation * fix(cli): require graph selection for scoped query registries * fix(server): preserve named graph id in single mode policy * fix(cli): share graph identity for policy resolution * test(cli): cover policy tooling server graph selection * fix(cli): honor server graph for policy tooling --------- Co-authored-by: Claude Opus 4.8 (1M context) --- crates/omnigraph-cli/src/main.rs | 679 +++++++++++++++-- crates/omnigraph-cli/tests/cli.rs | 292 ++++++++ crates/omnigraph-cli/tests/system_local.rs | 182 ++++- crates/omnigraph-cli/tests/system_remote.rs | 4 +- crates/omnigraph-policy/src/lib.rs | 95 ++- crates/omnigraph-server/src/api.rs | 159 ++++ crates/omnigraph-server/src/config.rs | 408 ++++++++++- crates/omnigraph-server/src/lib.rs | 620 +++++++++++++++- crates/omnigraph-server/src/queries.rs | 688 ++++++++++++++++++ crates/omnigraph-server/src/registry.rs | 12 + crates/omnigraph-server/tests/openapi.rs | 33 + crates/omnigraph-server/tests/server.rs | 637 +++++++++++++++- crates/omnigraph/src/db/omnigraph.rs | 37 +- .../src/db/omnigraph/schema_apply.rs | 129 ++-- docs/dev/index.md | 2 + docs/dev/rfc-002-config-cli-architecture.md | 590 +++++++++++++++ docs/dev/rfc-003-mcp-server-surface.md | 270 +++++++ docs/user/cli-reference.md | 12 +- docs/user/policy.md | 20 +- docs/user/server.md | 27 +- openapi.json | 319 ++++++++ 21 files changed, 5017 insertions(+), 198 deletions(-) create mode 100644 crates/omnigraph-server/src/queries.rs create mode 100644 docs/dev/rfc-002-config-cli-architecture.md create mode 100644 docs/dev/rfc-003-mcp-server-surface.md diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index d98c302..879f070 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -9,6 +9,7 @@ use clap::{Arg, ArgAction, Args, CommandFactory, FromArgMatches, Parser, Subcomm use color_eyre::eyre::{Result, bail}; use omnigraph::db::{Omnigraph, ReadTarget, SnapshotId}; use omnigraph::loader::LoadMode; +use omnigraph::storage::normalize_root_uri; use omnigraph_compiler::query::parser::parse_query; use omnigraph_compiler::schema::parser::parse_schema; use omnigraph_compiler::{ @@ -24,9 +25,10 @@ use omnigraph_server::api::{ SnapshotTableOutput, commit_output, ingest_output, read_output, schema_apply_output, snapshot_payload, }; +use omnigraph_server::queries::{QueryRegistry, check, format_check_breakages}; use omnigraph_server::{ AliasCommand, OmnigraphConfig, PolicyAction, PolicyDecision, PolicyEngine, PolicyRequest, - PolicyTestConfig, ReadOutputFormat, load_config, + PolicyTestConfig, ReadOutputFormat, graph_resource_id_for_selection, load_config, }; use reqwest::Method; use reqwest::header::AUTHORIZATION; @@ -153,6 +155,11 @@ enum Command { #[arg(long)] json: bool, }, + /// Operate on the server-side stored-query registry (`queries:`). + Queries { + #[command(subcommand)] + command: QueriesCommand, + }, /// Show graph snapshot Snapshot { /// Graph URI @@ -502,6 +509,35 @@ enum PolicyCommand { }, } +#[derive(Debug, Subcommand)] +enum QueriesCommand { + /// Type-check the stored-query registry against the live schema. + /// + /// Distinct from `omnigraph lint` (which lints one `.gq` file): + /// this validates the whole `queries:` registry — opening the graph + /// to read its schema and confirming every stored query still + /// type-checks. Exits non-zero on any breakage. + Validate { + /// Graph URI + uri: Option, + #[arg(long)] + target: Option, + #[arg(long)] + config: Option, + #[arg(long)] + json: bool, + }, + /// List the registered stored queries (name, MCP exposure, params). + List { + #[arg(long)] + target: Option, + #[arg(long)] + config: Option, + #[arg(long)] + json: bool, + }, +} + #[derive(Debug, Args, Clone)] struct ParamsArgs { #[arg(long, conflicts_with = "params_file")] @@ -743,25 +779,66 @@ fn load_cli_config(config_path: Option<&PathBuf>) -> Result { Ok(config) } -fn resolve_policy_engine(config: &OmnigraphConfig) -> Result { - let policy_file = config - .resolve_policy_file() - .ok_or_else(|| color_eyre::eyre::eyre!("policy.file must be set in omnigraph.yaml"))?; - PolicyEngine::load_graph(&policy_file, &policy_graph_id(config)) +#[derive(Debug, Clone)] +struct ResolvedCliGraph { + uri: String, + selected: Option, + graph_id: String, + policy_file: Option, + is_remote: bool, } -/// Open a local-URI graph and, when `policy.file` is configured in -/// `omnigraph.yaml`, install the resolved `PolicyEngine` on the engine -/// handle so every direct-engine write goes through -/// `Omnigraph::enforce(...)` (MR-722). Without a configured policy this -/// is identical to a bare `Omnigraph::open`. -/// -/// Returns owned `Omnigraph`; chained on top of `Omnigraph::open(...)`'s -/// existing future to keep call sites narrow. -async fn open_local_db_with_policy(uri: &str, config: &OmnigraphConfig) -> Result { - let db = Omnigraph::open(uri).await?; - if config.resolve_policy_file().is_some() { - let engine = Arc::new(resolve_policy_engine(config)?); +impl ResolvedCliGraph { + fn selected(&self) -> Option<&str> { + self.selected.as_deref() + } +} + +struct ResolvedPolicyContext { + policy_file: PathBuf, + graph_id: String, +} + +fn resolve_policy_context(config: &OmnigraphConfig) -> Result { + let selected = config.resolve_policy_tooling_graph_selection()?; + let policy_file = config + .resolve_policy_file_for(selected) + .ok_or_else(|| { + color_eyre::eyre::eyre!( + "policy.file or graphs..policy.file must be set in omnigraph.yaml" + ) + })?; + let graph_id = match selected { + Some(name) => graph_resource_id_for_selection(Some(name), ""), + None => graph_resource_id_for_selection(None, "default"), + }; + Ok(ResolvedPolicyContext { + policy_file, + graph_id, + }) +} + +fn resolve_policy_engine(context: &ResolvedPolicyContext) -> Result { + PolicyEngine::load_graph(&context.policy_file, &context.graph_id) +} + +fn resolve_policy_engine_for_graph(graph: &ResolvedCliGraph) -> Result { + let policy_file = graph.policy_file.as_ref().ok_or_else(|| { + color_eyre::eyre::eyre!( + "policy.file or graphs..policy.file must be set in omnigraph.yaml" + ) + })?; + PolicyEngine::load_graph(policy_file, &graph.graph_id) +} + +/// Open a local graph and install the policy resolved for the same graph +/// identity that produced the URI. A named graph uses +/// `graphs..policy.file`; an explicit positional URI is anonymous and +/// uses the legacy top-level `policy.file`. +async fn open_local_db_with_policy(graph: &ResolvedCliGraph) -> Result { + let db = Omnigraph::open(&graph.uri).await?; + if graph.policy_file.is_some() { + let engine = Arc::new(resolve_policy_engine_for_graph(graph)?); Ok(db.with_policy(engine as Arc)) } else { Ok(db) @@ -778,22 +855,16 @@ fn resolve_cli_actor<'a>(cli_as: Option<&'a str>, config: &'a OmnigraphConfig) - cli_as.or(config.cli.actor.as_deref()) } -fn resolve_policy_tests_path(config: &OmnigraphConfig) -> Result { - config.resolve_policy_tests_file().ok_or_else(|| { - color_eyre::eyre::eyre!( - "policy.tests.yaml requires policy.file to be set in omnigraph.yaml" - ) - }) +fn resolve_policy_tests_path(context: &ResolvedPolicyContext) -> PathBuf { + context.policy_file.with_file_name("policy.tests.yaml") } -fn policy_graph_id(config: &OmnigraphConfig) -> String { - if let Some(name) = &config.project.name { - return name.clone(); +fn normalize_policy_graph_uri(uri: &str) -> Result { + if is_remote_uri(uri) { + Ok(uri.trim_end_matches('/').to_string()) + } else { + Ok(normalize_root_uri(uri)?) } - config - .resolve_target_uri(None, None, config.server_graph_name()) - .or_else(|_| config.resolve_target_uri(None, None, config.cli_graph_name())) - .unwrap_or_else(|_| "default".to_string()) } fn resolve_remote_bearer_token( @@ -877,6 +948,47 @@ fn resolve_uri( config.resolve_target_uri(cli_uri, cli_target, config.cli_graph_name()) } +fn resolve_cli_graph( + config: &OmnigraphConfig, + cli_uri: Option, + cli_target: Option<&str>, +) -> Result { + let selected = if cli_uri.is_some() { + None + } else { + cli_target + .map(str::to_string) + .or_else(|| config.cli_graph_name().map(str::to_string)) + }; + config.resolve_graph_selection(selected.as_deref())?; + let uri = resolve_uri(config, cli_uri, cli_target)?; + let normalized_uri = normalize_policy_graph_uri(&uri)?; + let graph_id = graph_resource_id_for_selection(selected.as_deref(), &normalized_uri); + Ok(ResolvedCliGraph { + graph_id, + is_remote: is_remote_uri(&uri), + policy_file: config.resolve_policy_file_for(selected.as_deref()), + selected, + uri, + }) +} + +fn resolve_local_graph( + config: &OmnigraphConfig, + cli_uri: Option, + cli_target: Option<&str>, + operation: &str, +) -> Result { + let graph = resolve_cli_graph(config, cli_uri, cli_target)?; + if graph.is_remote { + bail!( + "{} is only supported against local graph URIs in this milestone", + operation + ); + } + Ok(graph) +} + /// Parse a Go-style compact duration: `7d`, `24h`, `30m`, `90s`, or a plain /// integer as seconds. Used by the `cleanup --older-than` flag. fn parse_duration_arg(s: &str) -> Result { @@ -915,14 +1027,7 @@ fn resolve_local_uri( cli_target: Option<&str>, operation: &str, ) -> Result { - let uri = resolve_uri(config, cli_uri, cli_target)?; - if is_remote_uri(&uri) { - bail!( - "{} is only supported against local graph URIs in this milestone", - operation - ); - } - Ok(uri) + Ok(resolve_local_graph(config, cli_uri, cli_target, operation)?.uri) } fn resolve_branch( @@ -1609,6 +1714,248 @@ async fn execute_query_lint( )) } +#[derive(serde::Serialize)] +struct QueriesIssue { + query: String, + message: String, +} + +#[derive(serde::Serialize)] +struct QueriesValidateOutput { + ok: bool, + breakages: Vec, + warnings: Vec, +} + +#[derive(serde::Serialize)] +struct QueriesParam { + name: String, + #[serde(rename = "type")] + type_name: String, + nullable: bool, +} + +#[derive(serde::Serialize)] +struct QueriesListItem { + name: String, + mcp_expose: bool, + tool_name: Option, + mutation: bool, + params: Vec, +} + +#[derive(serde::Serialize)] +struct QueriesListOutput { + queries: Vec, +} + +/// Resolve the selected graph to `(local URI, registry selection)` from one +/// precedence, so a command's schema and its stored-query registry can never +/// come from different graphs. A **positional URI is anonymous** (top-level +/// registry, ignoring the configured default graph); otherwise `--target` +/// or the configured `cli.graph` names the graph (its per-graph block). +/// Mirrors the server's single-mode identity rule. +fn resolve_selected_graph( + config: &OmnigraphConfig, + cli_uri: Option, + cli_target: Option<&str>, + operation: &str, +) -> Result<(String, Option)> { + let graph = resolve_local_graph(config, cli_uri, cli_target, operation)?; + Ok((graph.uri, graph.selected)) +} + +/// Load the stored-query registry for an already-resolved graph selection +/// (`None` = anonymous → top-level; `Some(name)` = that graph's block). +fn load_registry_or_report( + config: &OmnigraphConfig, + selected: Option<&str>, +) -> Result { + QueryRegistry::load(config, config.query_entries_for(selected)).map_err(|errors| { + color_eyre::eyre::eyre!( + "stored-query registry failed to load:\n {}", + errors + .iter() + .map(|e| e.to_string()) + .collect::>() + .join("\n ") + ) + }) +} + +fn graph_query_registry_names(config: &OmnigraphConfig) -> Vec<&str> { + config + .graphs + .iter() + .filter_map(|(name, graph)| (!graph.queries.is_empty()).then_some(name.as_str())) + .collect() +} + +fn resolve_registry_selection_for_list( + config: &OmnigraphConfig, + target: Option<&str>, +) -> Result> { + let selected = target + .map(str::to_string) + .or_else(|| config.cli_graph_name().map(str::to_string)); + if let Some(name) = selected.as_deref() { + config.resolve_graph_selection(Some(name))?; + return Ok(selected); + } + + if !config.query_entries().is_empty() { + return Ok(None); + } + + let graph_names = graph_query_registry_names(config); + if graph_names.is_empty() { + return Ok(None); + } + + bail!( + "stored-query registries are configured for graph{} {} but no graph was selected. Pass `--target {}` or set `cli.graph`.", + if graph_names.len() == 1 { "" } else { "s" }, + graph_names.join(", "), + graph_names[0], + ) +} + +fn validate_registry_for_catalog( + registry: &QueryRegistry, + catalog: &omnigraph_compiler::catalog::Catalog, + label: &str, +) -> omnigraph::error::Result<()> { + let report = check(registry, catalog); + if report.has_breakages() { + return Err(omnigraph::error::OmniError::manifest( + format_check_breakages(label, &report), + )); + } + Ok(()) +} + +async fn execute_queries_validate( + uri: Option, + target: Option, + config_path: Option<&PathBuf>, + json: bool, +) -> Result<()> { + let config = load_cli_config(config_path)?; + // One selection drives both the schema URI and the registry, so a + // positional URI and a `--target` can't validate different graphs. + let (uri, selected) = + resolve_selected_graph(&config, uri, target.as_deref(), "queries validate")?; + let registry = load_registry_or_report(&config, selected.as_deref())?; + let db = Omnigraph::open(&uri).await?; + let report = check(®istry, &db.catalog()); + + let output = QueriesValidateOutput { + ok: !report.has_breakages(), + breakages: report + .breakages + .iter() + .map(|b| QueriesIssue { + query: b.query.clone(), + message: b.message.clone(), + }) + .collect(), + warnings: report + .warnings + .iter() + .map(|w| QueriesIssue { + query: w.query.clone(), + message: w.message.clone(), + }) + .collect(), + }; + + if json { + print_json(&output)?; + } else { + if output.breakages.is_empty() { + println!( + "OK {} stored quer{} type-check against the schema", + registry.len(), + if registry.len() == 1 { "y" } else { "ies" } + ); + } + for issue in &output.breakages { + println!("ERROR query '{}': {}", issue.query, issue.message); + } + for issue in &output.warnings { + println!("WARN query '{}': {}", issue.query, issue.message); + } + } + + if report.has_breakages() { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + +fn execute_queries_list( + target: Option, + config_path: Option<&PathBuf>, + json: bool, +) -> Result<()> { + let config = load_cli_config(config_path)?; + let selected = resolve_registry_selection_for_list(&config, target.as_deref())?; + let registry = load_registry_or_report(&config, selected.as_deref())?; + + let output = QueriesListOutput { + queries: registry + .iter() + .map(|q| QueriesListItem { + name: q.name.clone(), + mcp_expose: q.expose, + tool_name: q.tool_name.clone(), + mutation: q.is_mutation(), + params: q + .decl + .params + .iter() + .map(|p| QueriesParam { + name: p.name.clone(), + type_name: p.type_name.clone(), + nullable: p.nullable, + }) + .collect(), + }) + .collect(), + }; + + if json { + print_json(&output)?; + } else if output.queries.is_empty() { + println!("(no stored queries registered)"); + } else { + for q in &output.queries { + let kind = if q.mutation { "mutation" } else { "read" }; + let params = q + .params + .iter() + .map(|p| { + format!( + "${}: {}{}", + p.name, + p.type_name, + if p.nullable { "?" } else { "" } + ) + }) + .collect::>() + .join(", "); + let mcp = if q.mcp_expose { + format!(" [mcp: {}]", q.tool_name.as_deref().unwrap_or(&q.name)) + } else { + String::new() + }; + println!("{kind} {}({params}){mcp}", q.name); + } + } + Ok(()) +} + async fn execute_read( uri: &str, query_source: &str, @@ -1655,7 +2002,7 @@ async fn execute_read_remote( } async fn execute_change( - uri: &str, + graph: &ResolvedCliGraph, query_source: &str, query_name: Option<&str>, branch: &str, @@ -1665,7 +2012,7 @@ async fn execute_change( ) -> Result { let (selected_name, query_params) = select_named_query(query_source, query_name)?; let params = query_params_from_json(&query_params, params_json)?; - let db = open_local_db_with_policy(uri, config).await?; + let db = open_local_db_with_policy(graph).await?; let actor = resolve_cli_actor(cli_as_actor, config); let result = db .mutate_as(branch, query_source, &selected_name, ¶ms, actor) @@ -1893,9 +2240,10 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = resolve_local_uri(&config, uri, target.as_deref(), "load")?; + let graph = resolve_local_graph(&config, uri, target.as_deref(), "load")?; + let uri = graph.uri.clone(); let branch = resolve_branch(&config, branch, None, "main"); - let db = open_local_db_with_policy(&uri, &config).await?; + let db = open_local_db_with_policy(&graph).await?; let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); let result = db .load_file_as(&branch, &data.to_string_lossy(), mode.into(), actor) @@ -1936,10 +2284,11 @@ async fn main() -> Result<()> { let config = load_cli_config(config.as_ref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; + let graph = resolve_cli_graph(&config, uri, target.as_deref())?; + let uri = graph.uri.clone(); let branch = resolve_branch(&config, branch, None, "main"); let from = resolve_branch(&config, from, None, "main"); - let payload = if is_remote_uri(&uri) { + let payload = if graph.is_remote { let data = fs::read_to_string(&data)?; remote_json::( &http_client, @@ -1955,7 +2304,7 @@ async fn main() -> Result<()> { ) .await? } else { - let db = open_local_db_with_policy(&uri, &config).await?; + let db = open_local_db_with_policy(&graph).await?; let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); let result = db .ingest_file_as( @@ -1986,9 +2335,10 @@ async fn main() -> Result<()> { let config = load_cli_config(config.as_ref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; + let graph = resolve_cli_graph(&config, uri, target.as_deref())?; + let uri = graph.uri.clone(); let from = resolve_branch(&config, from, None, "main"); - let payload = if is_remote_uri(&uri) { + let payload = if graph.is_remote { remote_json::( &http_client, Method::POST, @@ -2001,7 +2351,7 @@ async fn main() -> Result<()> { ) .await? } else { - let db = open_local_db_with_policy(&uri, &config).await?; + let db = open_local_db_with_policy(&graph).await?; let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); db.branch_create_from_as(ReadTarget::branch(&from), &name, actor) .await?; @@ -2027,8 +2377,9 @@ async fn main() -> Result<()> { let config = load_cli_config(config.as_ref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; - let payload = if is_remote_uri(&uri) { + let graph = resolve_cli_graph(&config, uri, target.as_deref())?; + let uri = graph.uri.clone(); + let payload = if graph.is_remote { remote_json::( &http_client, Method::GET, @@ -2061,8 +2412,9 @@ async fn main() -> Result<()> { let config = load_cli_config(config.as_ref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; - let payload = if is_remote_uri(&uri) { + let graph = resolve_cli_graph(&config, uri, target.as_deref())?; + let uri = graph.uri.clone(); + let payload = if graph.is_remote { remote_json::( &http_client, Method::DELETE, @@ -2072,7 +2424,7 @@ async fn main() -> Result<()> { ) .await? } else { - let db = open_local_db_with_policy(&uri, &config).await?; + let db = open_local_db_with_policy(&graph).await?; let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); db.branch_delete_as(&name, actor).await?; BranchDeleteOutput { @@ -2098,9 +2450,10 @@ async fn main() -> Result<()> { let config = load_cli_config(config.as_ref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; + let graph = resolve_cli_graph(&config, uri, target.as_deref())?; + let uri = graph.uri.clone(); let into = resolve_branch(&config, into, None, "main"); - let payload = if is_remote_uri(&uri) { + let payload = if graph.is_remote { remote_json::( &http_client, Method::POST, @@ -2113,7 +2466,7 @@ async fn main() -> Result<()> { ) .await? } else { - let db = open_local_db_with_policy(&uri, &config).await?; + let db = open_local_db_with_policy(&graph).await?; let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); let outcome = db.branch_merge_as(&source, &into, actor).await?; BranchMergeOutput { @@ -2248,9 +2601,10 @@ async fn main() -> Result<()> { let config = load_cli_config(config.as_ref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; + let graph = resolve_cli_graph(&config, uri, target.as_deref())?; + let uri = graph.uri.clone(); let schema_source = fs::read_to_string(&schema)?; - let output = if is_remote_uri(&uri) { + let output = if graph.is_remote { // MR-694 PR B: SchemaApplyRequest gained an // allow_data_loss field so Hard-mode drops are no // longer CLI-only. The previous bail is gone; the @@ -2268,13 +2622,22 @@ async fn main() -> Result<()> { ) .await? } else { - let db = open_local_db_with_policy(&uri, &config).await?; + let db = open_local_db_with_policy(&graph).await?; let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); + let registry = load_registry_or_report(&config, graph.selected())?; + let registry = (!registry.is_empty()).then_some(registry); + let label = graph.selected().unwrap_or(&uri).to_string(); let result = db - .apply_schema_as( + .apply_schema_as_with_catalog_check( &schema_source, omnigraph::db::SchemaApplyOptions { allow_data_loss }, actor, + |catalog| { + if let Some(registry) = registry.as_ref() { + validate_registry_for_catalog(registry, catalog, &label)?; + } + Ok(()) + }, ) .await?; schema_apply_output(&uri, result) @@ -2331,6 +2694,23 @@ async fn main() -> Result<()> { .await?; finish_query_lint(&output, json)?; } + Command::Queries { command } => match command { + QueriesCommand::Validate { + uri, + target, + config, + json, + } => { + execute_queries_validate(uri, target, config.as_ref(), json).await?; + } + QueriesCommand::List { + target, + config, + json, + } => { + execute_queries_list(target, config.as_ref(), json)?; + } + }, Command::Snapshot { uri, target, @@ -2436,7 +2816,8 @@ async fn main() -> Result<()> { .as_deref() .or_else(|| alias_config.and_then(|alias| alias.graph.as_deref())); let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target_name)?; - let uri = resolve_uri(&config, uri, target_name)?; + let graph = resolve_cli_graph(&config, uri, target_name)?; + let uri = graph.uri.clone(); let query_source = resolve_query_source( &config, query.as_ref(), @@ -2458,7 +2839,7 @@ async fn main() -> Result<()> { alias_config.and_then(|alias| alias.branch.clone()), )?; let query_name = name.or_else(|| alias_config.and_then(|alias| alias.name.clone())); - let output = if is_remote_uri(&uri) { + let output = if graph.is_remote { execute_read_remote( &http_client, &uri, @@ -2521,7 +2902,8 @@ async fn main() -> Result<()> { .as_deref() .or_else(|| alias_config.and_then(|alias| alias.graph.as_deref())); let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target_name)?; - let uri = resolve_uri(&config, uri, target_name)?; + let graph = resolve_cli_graph(&config, uri, target_name)?; + let uri = graph.uri.clone(); let query_source = resolve_query_source( &config, query.as_ref(), @@ -2543,7 +2925,7 @@ async fn main() -> Result<()> { "main", ); let query_name = name.or_else(|| alias_config.and_then(|alias| alias.name.clone())); - let output = if is_remote_uri(&uri) { + let output = if graph.is_remote { execute_change_remote( &http_client, &uri, @@ -2556,7 +2938,7 @@ async fn main() -> Result<()> { .await? } else { execute_change( - &uri, + &graph, &query_source, query_name.as_deref(), &branch, @@ -2575,20 +2957,19 @@ async fn main() -> Result<()> { Command::Policy { command } => match command { PolicyCommand::Validate { config } => { let config = load_cli_config(config.as_ref())?; - let engine = resolve_policy_engine(&config)?; - let policy_file = config - .resolve_policy_file() - .expect("policy file should exist after resolve_policy_engine"); + let context = resolve_policy_context(&config)?; + let engine = resolve_policy_engine(&context)?; println!( "policy valid: {} [{} actors]", - policy_file.display(), + context.policy_file.display(), engine.known_actor_count() ); } PolicyCommand::Test { config } => { let config = load_cli_config(config.as_ref())?; - let engine = resolve_policy_engine(&config)?; - let tests_path = resolve_policy_tests_path(&config)?; + let context = resolve_policy_context(&config)?; + let engine = resolve_policy_engine(&context)?; + let tests_path = resolve_policy_tests_path(&context); let tests = PolicyTestConfig::load(&tests_path)?; engine.run_tests(&tests)?; println!("policy tests passed: {} cases", tests.cases.len()); @@ -2601,7 +2982,8 @@ async fn main() -> Result<()> { target_branch, } => { let config = load_cli_config(config.as_ref())?; - let engine = resolve_policy_engine(&config)?; + let context = resolve_policy_context(&config)?; + let engine = resolve_policy_engine(&context)?; let request = PolicyRequest { action, branch, @@ -2774,7 +3156,8 @@ mod tests { use super::{ DEFAULT_BEARER_TOKEN_ENV, apply_bearer_token, bearer_token_from_env_file, legacy_change_request_body, load_cli_config, load_env_file_into_process, - normalize_bearer_token, parse_env_assignment, resolve_remote_bearer_token, + normalize_bearer_token, parse_env_assignment, resolve_policy_context, + resolve_cli_graph, resolve_remote_bearer_token, }; use omnigraph_server::load_config; use reqwest::header::AUTHORIZATION; @@ -3034,4 +3417,150 @@ graphs: } } } + + #[test] + fn graph_identity_resolve_policy_context_named_cli_graph_uses_graph_key_not_project_name_or_uri() { + let temp = tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +project: + name: misleading-project +graphs: + local: + uri: /tmp/local-policy-graph.omni + policy: + file: ./policy.yaml +cli: + graph: local +"#, + ) + .unwrap(); + + let config = load_config(Some(&config_path)).unwrap(); + let context = resolve_policy_context(&config).unwrap(); + assert_eq!(context.graph_id, "local"); + } + + #[test] + fn graph_identity_resolve_policy_context_server_graph_uses_graph_key_when_cli_graph_absent() { + let temp = tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +project: + name: misleading-project +graphs: + local: + uri: /tmp/local-policy-graph.omni + policy: + file: ./server-policy.yaml +server: + graph: local +"#, + ) + .unwrap(); + + let config = load_config(Some(&config_path)).unwrap(); + let context = resolve_policy_context(&config).unwrap(); + assert_eq!(context.graph_id, "local"); + assert!(context.policy_file.ends_with("server-policy.yaml")); + } + + #[test] + fn graph_identity_resolve_policy_context_anonymous_uses_top_level_default_identity() { + let temp = tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +project: + name: misleading-project +graphs: + local: + uri: /tmp/local-policy-graph.omni +policy: + file: ./top-policy.yaml +"#, + ) + .unwrap(); + + let config = load_config(Some(&config_path)).unwrap(); + let context = resolve_policy_context(&config).unwrap(); + assert_eq!(context.graph_id, "default"); + assert!(context.policy_file.ends_with("top-policy.yaml")); + } + + #[test] + fn graph_identity_resolve_cli_graph_named_target_uses_graph_key_not_project_name_or_uri() { + let temp = tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +project: + name: misleading-project +graphs: + prod: + uri: s3://bucket/prod-graph/ + policy: + file: ./prod-policy.yaml +"#, + ) + .unwrap(); + + let config = load_config(Some(&config_path)).unwrap(); + let graph = resolve_cli_graph(&config, None, Some("prod")).unwrap(); + assert_eq!(graph.selected(), Some("prod")); + assert_eq!(graph.graph_id, "prod"); + assert_eq!(graph.uri, "s3://bucket/prod-graph/"); + } + + #[test] + fn graph_identity_resolve_cli_graph_positional_uri_uses_anonymous_normalized_uri() { + let temp = tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +project: + name: misleading-project +graphs: + local: + uri: /tmp/configured-graph.omni + policy: + file: ./policy.yaml +cli: + graph: local +"#, + ) + .unwrap(); + + let config = load_config(Some(&config_path)).unwrap(); + let local_graph_path = temp.path().join("explicit-graph.omni"); + let local_graph = resolve_cli_graph( + &config, + Some(format!("file://{}", local_graph_path.display())), + None, + ) + .unwrap(); + assert_eq!(local_graph.selected(), None); + assert_eq!( + local_graph.graph_id, + local_graph_path.to_string_lossy().as_ref() + ); + assert_eq!(local_graph.policy_file, None); + + let s3_graph = resolve_cli_graph( + &config, + Some("s3://bucket/anonymous-graph/".to_string()), + None, + ) + .unwrap(); + assert_eq!(s3_graph.selected(), None); + assert_eq!(s3_graph.graph_id, "s3://bucket/anonymous-graph"); + assert_eq!(s3_graph.policy_file, None); + } } diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 6e5de37..9682d9a 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -2376,3 +2376,295 @@ fn graphs_list_against_local_uri_errors_with_remote_only_message() { "expected 'remote multi-graph server URL' rejection in stderr; got:\n{stderr}" ); } + +fn queries_test_config(graph_uri: &str, entry: &str, gq_file: &str) -> String { + format!( + "graphs:\n local:\n uri: '{}'\n queries:\n {entry}:\n file: ./{gq_file}\n\ + cli:\n graph: local\npolicy: {{}}\n", + graph_uri.replace('\'', "''") + ) +} + +#[test] +fn queries_validate_exits_zero_on_clean_registry() { + let graph = SystemGraph::loaded(); + graph.write_query( + "find_person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + &queries_test_config(&graph.path().to_string_lossy(), "find_person", "find_person.gq"), + ); + let output = output_success(cli().arg("queries").arg("validate").arg("--config").arg(&config)); + let stdout = stdout_string(&output); + assert!(stdout.contains("OK"), "stdout:\n{stdout}"); +} + +#[test] +fn queries_validate_exits_nonzero_on_type_broken_query() { + let graph = SystemGraph::loaded(); + // `Widget` is not in the fixture schema. + graph.write_query("ghost.gq", "query ghost() { match { $w: Widget } return { $w.name } }"); + let config = graph.write_config( + "omnigraph.yaml", + &queries_test_config(&graph.path().to_string_lossy(), "ghost", "ghost.gq"), + ); + let output = output_failure(cli().arg("queries").arg("validate").arg("--config").arg(&config)); + let stdout = stdout_string(&output); + assert!( + stdout.contains("ghost"), + "validation should name the broken query; stdout:\n{stdout}" + ); +} + +#[test] +fn queries_list_prints_registered_query() { + let graph = SystemGraph::loaded(); + graph.write_query( + "find_person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + // Exposed with an explicit tool name so the list shows the MCP suffix. + let config = graph.write_config( + "omnigraph.yaml", + &format!( + concat!( + "graphs:\n", + " local:\n", + " uri: '{}'\n", + " queries:\n", + " find_person:\n", + " file: ./find_person.gq\n", + " mcp: {{ expose: true, tool_name: lookup_person }}\n", + "cli:\n", + " graph: local\n", + "policy: {{}}\n", + ), + graph.path().to_string_lossy().replace('\'', "''") + ), + ); + let output = output_success(cli().arg("queries").arg("list").arg("--config").arg(&config)); + let stdout = stdout_string(&output); + assert!(stdout.contains("find_person"), "stdout:\n{stdout}"); + assert!( + stdout.contains("$name: String"), + "list should show typed params; stdout:\n{stdout}" + ); + assert!( + stdout.contains("[mcp: lookup_person]"), + "list should show the MCP tool name for exposed queries; stdout:\n{stdout}" + ); +} + +#[test] +fn queries_list_requires_graph_selection_for_per_graph_only_registries() { + let graph = SystemGraph::loaded(); + graph.write_query( + "find_person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + &format!( + concat!( + "graphs:\n", + " local:\n", + " uri: '{}'\n", + " queries:\n", + " find_person:\n", + " file: ./find_person.gq\n", + "policy: {{}}\n", + ), + graph.path().to_string_lossy().replace('\'', "''") + ), + ); + + let output = output_failure(cli().arg("queries").arg("list").arg("--config").arg(&config)); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("local") && stderr.contains("--target local"), + "error must name the graph and give a concrete selection hint; stderr:\n{stderr}" + ); +} + +#[test] +fn queries_list_without_graph_selection_lists_top_level_registry() { + let graph = SystemGraph::loaded(); + graph.write_query( + "top_find.gq", + "query top_find($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + concat!( + "queries:\n", + " top_find:\n", + " file: ./top_find.gq\n", + "policy: {}\n", + ), + ); + + let output = output_success(cli().arg("queries").arg("list").arg("--config").arg(&config)); + let stdout = stdout_string(&output); + assert!(stdout.contains("top_find"), "stdout:\n{stdout}"); +} + +#[test] +fn queries_list_unknown_target_errors() { + // `queries list` opens no graph URI, so unknown-graph validation can't ride + // along on URI resolution the way it does for every other command. An + // unknown `--target` must still error (naming the graph) instead of + // silently falling back to the top-level registry and showing the wrong + // (or empty) catalog. + let graph = SystemGraph::loaded(); + graph.write_query( + "find_person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + &queries_test_config(&graph.path().to_string_lossy(), "find_person", "find_person.gq"), + ); + let output = output_failure( + cli() + .arg("queries") + .arg("list") + .arg("--target") + .arg("nonexistent") + .arg("--config") + .arg(&config), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("nonexistent"), + "error must name the unknown graph; stderr:\n{stderr}" + ); +} + +#[test] +fn queries_commands_reject_named_graph_with_populated_top_level_block() { + // A named graph (here via `cli.graph`) uses its own `graphs.` block, + // so a populated top-level `queries:` block would be silently ignored — a + // config the server REFUSES to boot. `queries validate`/`list` must reject + // it too (matching boot) instead of validating/listing the per-graph block + // and giving a false green. + let graph = SystemGraph::loaded(); + graph.write_query( + "find_person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + &format!( + concat!( + "graphs:\n", + " local:\n", + " uri: '{}'\n", + " queries:\n", + " find_person:\n", + " file: ./find_person.gq\n", + "cli:\n", + " graph: local\n", + "queries:\n", // populated top-level block: the coherence violation + " legacy:\n", + " file: ./legacy.gq\n", + "policy: {{}}\n", + ), + graph.path().to_string_lossy().replace('\'', "''") + ), + ); + // Both resolve `local` from cli.graph (no positional URI), so both must + // error and name the graph + the ignored block — like server boot does. + for sub in ["validate", "list"] { + let output = output_failure(cli().arg("queries").arg(sub).arg("--config").arg(&config)); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("local") && stderr.contains("queries"), + "`queries {sub}` must reject a named graph with a populated top-level block; stderr:\n{stderr}" + ); + } +} + +#[test] +fn queries_validate_exits_nonzero_on_duplicate_tool_name() { + // Two exposed queries claiming one MCP tool name is a load-time + // collision — `queries validate` must fail (offline, before the engine + // opens) and name both queries plus the contested tool. + let graph = SystemGraph::loaded(); + graph.write_query("a.gq", "query a() { match { $p: Person } return { $p.name } }"); + graph.write_query("b.gq", "query b() { match { $p: Person } return { $p.name } }"); + let config = graph.write_config( + "omnigraph.yaml", + &format!( + concat!( + "graphs:\n", + " local:\n", + " uri: '{}'\n", + " queries:\n", + " a:\n", + " file: ./a.gq\n", + " mcp: {{ expose: true, tool_name: dup }}\n", + " b:\n", + " file: ./b.gq\n", + " mcp: {{ expose: true, tool_name: dup }}\n", + "cli:\n", + " graph: local\n", + "policy: {{}}\n", + ), + graph.path().to_string_lossy().replace('\'', "''") + ), + ); + let output = output_failure(cli().arg("queries").arg("validate").arg("--config").arg(&config)); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("dup") && stderr.contains("'a'") && stderr.contains("'b'"), + "duplicate tool name should be reported naming both queries; stderr:\n{stderr}" + ); +} + +#[test] +fn queries_validate_positional_uri_ignores_default_graph() { + // A positional URI is anonymous → the schema AND the registry both come + // from top-level, even when `cli.graph` names a graph whose per-graph + // queries would fail. Pins that the URI and registry can't diverge. + let graph = SystemGraph::loaded(); + graph.write_query( + "clean.gq", + "query clean($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + // `Widget` is not in the fixture schema — the default graph's per-graph + // query would break validate if it were (wrongly) selected. + graph.write_query("broken.gq", "query broken() { match { $w: Widget } return { $w.name } }"); + let config = graph.write_config( + "omnigraph.yaml", + concat!( + "cli:\n graph: prod\n", + "graphs:\n", + " prod:\n", + " uri: /nonexistent-prod.omni\n", + " queries:\n", + " broken:\n", + " file: ./broken.gq\n", + "queries:\n", + " clean:\n", + " file: ./clean.gq\n", + "policy: {}\n", + ), + ); + // Positional URI = the real loaded graph; selection is anonymous, so the + // CLEAN top-level registry validates (not prod's broken one). + let output = output_success( + cli() + .arg("queries") + .arg("validate") + .arg(graph.path()) + .arg("--config") + .arg(&config), + ); + let stdout = stdout_string(&output); + assert!( + stdout.contains("OK"), + "positional URI must validate the top-level registry, not the cli.graph default; stdout:\n{stdout}" + ); +} diff --git a/crates/omnigraph-cli/tests/system_local.rs b/crates/omnigraph-cli/tests/system_local.rs index 08f653d..4fc3e9a 100644 --- a/crates/omnigraph-cli/tests/system_local.rs +++ b/crates/omnigraph-cli/tests/system_local.rs @@ -74,14 +74,36 @@ project: graphs: local: uri: {} + policy: + file: ./policy.yaml cli: graph: local branch: main query: roots: - . -policy: - file: ./policy.yaml +", + yaml_string(&graph.path().to_string_lossy()) + ) +} + +fn local_policy_server_graph_config(graph: &SystemGraph) -> String { + format!( + "\ +project: + name: policy-e2e-local +graphs: + local: + uri: {} + policy: + file: ./policy.yaml +server: + graph: local +cli: + branch: main +query: + roots: + - . ", yaml_string(&graph.path().to_string_lossy()) ) @@ -1000,49 +1022,55 @@ query vector_search($q: String) { #[test] fn local_cli_policy_tooling_is_end_to_end() { // Sanity check for the read-only policy CLI surfaces. These don't - // mutate the graph — they just parse and evaluate the policy file — - // so they don't depend on PR #4's engine-side enforcement. + // mutate the graph; they parse and evaluate the effective policy for + // named graph selections, including per-graph policy files. let graph = SystemGraph::loaded(); let config = graph.write_config("omnigraph-policy.yaml", &local_policy_config(&graph)); + let server_graph_config = graph.write_config( + "omnigraph-policy-server.yaml", + &local_policy_server_graph_config(&graph), + ); graph.write_config("policy.yaml", POLICY_E2E_YAML); graph.write_config("policy.tests.yaml", POLICY_E2E_TESTS_YAML); - let validate = output_success( - cli() - .arg("policy") - .arg("validate") - .arg("--config") - .arg(&config), - ); - assert!(stdout_string(&validate).contains("policy valid:")); + for config in [&config, &server_graph_config] { + let validate = output_success( + cli() + .arg("policy") + .arg("validate") + .arg("--config") + .arg(config), + ); + assert!(stdout_string(&validate).contains("policy valid:")); - let tests = output_success(cli().arg("policy").arg("test").arg("--config").arg(&config)); - assert!(stdout_string(&tests).contains("policy tests passed: 2 cases")); + let tests = output_success(cli().arg("policy").arg("test").arg("--config").arg(config)); + assert!(stdout_string(&tests).contains("policy tests passed: 2 cases")); - let explain = output_success( - cli() - .arg("policy") - .arg("explain") - .arg("--config") - .arg(&config) - .arg("--actor") - .arg("act-bruno") - .arg("--action") - .arg("change") - .arg("--branch") - .arg("main"), - ); - let explain_stdout = stdout_string(&explain); - assert!(explain_stdout.contains("decision: deny")); - assert!(explain_stdout.contains("branch: main")); + let explain = output_success( + cli() + .arg("policy") + .arg("explain") + .arg("--config") + .arg(config) + .arg("--actor") + .arg("act-bruno") + .arg("--action") + .arg("change") + .arg("--branch") + .arg("main"), + ); + let explain_stdout = stdout_string(&explain); + assert!(explain_stdout.contains("decision: deny")); + assert!(explain_stdout.contains("branch: main")); + } } #[test] fn local_cli_change_enforces_engine_layer_policy() { - // Asserts MR-722 PR #4: when `policy.file` is configured in - // `omnigraph.yaml`, the CLI loads PolicyEngine into Omnigraph and - // every direct-engine write hits `enforce(action, scope, actor)` — - // identical to what the HTTP server gets, regardless of transport. + // Asserts MR-722 PR #4: when the selected graph has a configured + // policy file, the CLI loads PolicyEngine into Omnigraph and every + // direct-engine write hits `enforce(action, scope, actor)` — identical + // to what the HTTP server gets, regardless of transport. // // Three cases, each discriminating: // @@ -1135,6 +1163,32 @@ fn local_cli_change_enforces_engine_layer_policy() { assert_eq!(verify["rows"][0]["p.name"], "RagnorOnMain"); } +#[test] +fn local_cli_positional_uri_does_not_inherit_default_graph_policy() { + let graph = SystemGraph::loaded(); + let config = graph.write_config("omnigraph-policy.yaml", &local_policy_config(&graph)); + graph.write_config("policy.yaml", POLICY_E2E_YAML); + let mutation_file = insert_person_query(&graph, "system-local-policy-positional.gq"); + + let allowed = parse_stdout_json(&output_success( + cli() + .arg("--as") + .arg("act-bruno") + .arg("change") + .arg("--config") + .arg(&config) + .arg("--uri") + .arg(graph.path()) + .arg("--query") + .arg(&mutation_file) + .arg("--params") + .arg(r#"{"name":"PositionalUriBruno","age":4}"#) + .arg("--json"), + )); + assert_eq!(allowed["affected_nodes"], 1); + assert_eq!(allowed["actor_id"], "act-bruno"); +} + // ─── MR-722 PR A: CLI×writer matrix ─────────────────────────────────────── // // The change writer is covered above by `local_cli_change_enforces_engine_layer_policy`. @@ -1293,6 +1347,62 @@ fn local_cli_schema_apply_enforces_engine_layer_policy() { assert_eq!(allowed["applied"], true); } +#[test] +fn local_cli_schema_apply_rejects_stored_query_breakage_before_publish() { + let graph = SystemGraph::loaded(); + graph.write_query( + "stored-find-person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + let config = graph.write_config( + "omnigraph-stored-query-schema.yaml", + &format!( + "\ +graphs: + local: + uri: {} + queries: + find_person: + file: ./stored-find-person.gq +cli: + graph: local + branch: main +query: + roots: + - . +policy: {{}} +", + yaml_string(&graph.path().to_string_lossy()) + ), + ); + let renamed_schema = std::fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace("age: I32?", "years: I32? @rename_from(\"age\")"); + let schema_path = graph.write_file("stored-query-breaks.pg", &renamed_schema); + + let rejected = output_failure( + cli() + .arg("schema") + .arg("apply") + .arg("--config") + .arg(&config) + .arg("--schema") + .arg(&schema_path) + .arg("--json"), + ); + let stderr = String::from_utf8_lossy(&rejected.stderr); + assert!( + stderr.contains("find_person") && stderr.contains("schema check"), + "schema apply should reject the stored-query breakage before publish; stderr: {stderr}" + ); + + let schema = stdout_string(&output_success( + cli().arg("schema").arg("show").arg("--config").arg(&config), + )); + assert!(schema.contains("age: I32?")); + assert!(!schema.contains("years: I32?")); +} + #[test] fn local_cli_branch_create_enforces_engine_layer_policy() { let graph = SystemGraph::loaded(); @@ -1448,6 +1558,8 @@ project: graphs: local: uri: {} + policy: + file: ./policy.yaml cli: graph: local branch: main @@ -1455,8 +1567,6 @@ cli: query: roots: - . -policy: - file: ./policy.yaml ", yaml_string(&graph.path().to_string_lossy()), actor, diff --git a/crates/omnigraph-cli/tests/system_remote.rs b/crates/omnigraph-cli/tests/system_remote.rs index c86e32e..45bf502 100644 --- a/crates/omnigraph-cli/tests/system_remote.rs +++ b/crates/omnigraph-cli/tests/system_remote.rs @@ -60,10 +60,10 @@ project: graphs: local: uri: {} + policy: + file: ./policy.yaml server: graph: local -policy: - file: ./policy.yaml ", yaml_string(&graph.path().to_string_lossy()) ) diff --git a/crates/omnigraph-policy/src/lib.rs b/crates/omnigraph-policy/src/lib.rs index 6459fcd..cb59796 100644 --- a/crates/omnigraph-policy/src/lib.rs +++ b/crates/omnigraph-policy/src/lib.rs @@ -56,6 +56,21 @@ pub enum PolicyAction { /// from v0.6.0; operators add and remove graphs by editing /// `omnigraph.yaml` and restarting. GraphList, + /// Gates invoking a server-side stored query by name. Per-graph and + /// **graph-scoped** (no branch dimension, like `Admin`): the per-branch + /// access of the query body is enforced by the inner `Read`/`Change` + /// gate, so branch-scoping this outer gate would be redundant (and was + /// wrong for snapshot reads). A rule that sets `branch_scope` on + /// `invoke_query` is rejected by `validate()`. In this release it is + /// **coarse**: an `invoke_query` allow rule permits *any* stored query + /// on the graph (no per-query dimension yet); a future, additive + /// refinement adds an optional query-name scope. + /// + /// This gate sits at the HTTP boundary. The engine `_as` writers still + /// enforce `Read`/`Change` per the query body, so a stored *mutation* + /// is double-gated: `invoke_query` to reach the tool, plus `change` for + /// the write itself. + InvokeQuery, } impl PolicyAction { @@ -70,6 +85,7 @@ impl PolicyAction { Self::BranchMerge => "branch_merge", Self::Admin => "admin", Self::GraphList => "graph_list", + Self::InvokeQuery => "invoke_query", } } @@ -99,7 +115,8 @@ impl PolicyAction { | Self::BranchCreate | Self::BranchDelete | Self::BranchMerge - | Self::Admin => PolicyResourceKind::Graph, + | Self::Admin + | Self::InvokeQuery => PolicyResourceKind::Graph, } } } @@ -155,6 +172,7 @@ impl FromStr for PolicyAction { "branch_merge" => Ok(Self::BranchMerge), "admin" => Ok(Self::Admin), "graph_list" => Ok(Self::GraphList), + "invoke_query" => Ok(Self::InvokeQuery), other => bail!("unknown policy action '{other}'"), } } @@ -806,6 +824,7 @@ namespace Omnigraph { action "branch_delete" appliesTo { principal: Actor, resource: Graph, context: RequestContext }; action "branch_merge" appliesTo { principal: Actor, resource: Graph, context: RequestContext }; action "admin" appliesTo { principal: Actor, resource: Graph, context: RequestContext }; + action "invoke_query" appliesTo { principal: Actor, resource: Graph, context: RequestContext }; action "graph_list" appliesTo { principal: Actor, resource: Server, context: RequestContext }; } @@ -1264,6 +1283,80 @@ rules: assert!(!deny.allowed); } + #[test] + fn invoke_query_authorizes_per_graph() { + let policy: PolicyConfig = serde_yaml::from_str( + r#" +version: 1 +groups: + team: [act-alice] + others: [act-bruno] +rules: + - id: team-invoke-queries + allow: + actors: { group: team } + actions: [invoke_query] +"#, + ) + .unwrap(); + let engine = PolicyCompiler::compile(&policy, "graph").unwrap(); + + let allow = engine + .authorize( + "act-alice", + &PolicyRequest { + action: PolicyAction::InvokeQuery, + branch: None, + target_branch: None, + }, + ) + .unwrap(); + assert!(allow.allowed); + assert_eq!( + allow.matched_rule_id.as_deref(), + Some("team-invoke-queries") + ); + + // Actor outside the group → deny. + let deny = engine + .authorize( + "act-bruno", + &PolicyRequest { + action: PolicyAction::InvokeQuery, + branch: None, + target_branch: None, + }, + ) + .unwrap(); + assert!(!deny.allowed); + } + + #[test] + fn invoke_query_rejects_branch_scope() { + // invoke_query is graph-scoped (like admin) — per-branch access is + // enforced by the inner read/change gate — so a rule that puts a + // `branch_scope` qualifier on it is rejected at validate(). + let policy: PolicyConfig = serde_yaml::from_str( + r#" +version: 1 +groups: + team: [act-alice] +rules: + - id: team-invoke-any-branch + allow: + actors: { group: team } + actions: [invoke_query] + branch_scope: any +"#, + ) + .unwrap(); + let err = policy.validate().unwrap_err().to_string(); + assert!( + err.contains("branch_scope") && err.contains("invoke_query"), + "branch_scope on invoke_query must be rejected: {err}" + ); + } + #[test] fn server_scoped_rule_cannot_use_branch_scope() { let policy: PolicyConfig = serde_yaml::from_str( diff --git a/crates/omnigraph-server/src/api.rs b/crates/omnigraph-server/src/api.rs index 2c818ae..4a6024f 100644 --- a/crates/omnigraph-server/src/api.rs +++ b/crates/omnigraph-server/src/api.rs @@ -1,8 +1,11 @@ use omnigraph::db::{GraphCommit, MergeOutcome, ReadTarget, SchemaApplyResult, Snapshot}; use omnigraph::error::{MergeConflict, MergeConflictKind}; use omnigraph::loader::{IngestResult, LoadMode}; +use crate::queries::StoredQuery; use omnigraph_compiler::SchemaMigrationStep; +use omnigraph_compiler::query::ast::Param; use omnigraph_compiler::result::QueryResult; +use omnigraph_compiler::types::{PropType, ScalarType}; use serde::{Deserialize, Serialize}; use serde_json::Value; use utoipa::{IntoParams, ToSchema}; @@ -300,6 +303,162 @@ pub struct ChangeRequest { pub branch: Option, } +/// Body for `POST /queries/{name}` — invokes the server-side stored query +/// named in the path. The query source and name come from the registry, +/// never the body; only the runtime inputs are supplied here. +#[derive(Debug, Clone, Default, Serialize, Deserialize, ToSchema)] +pub struct InvokeStoredQueryRequest { + /// JSON object whose keys match the stored query's declared parameters. + #[serde(default)] + pub params: Option, + /// Branch to run against. Defaults to `main`; for a stored mutation the + /// write targets this branch. + #[serde(default)] + pub branch: Option, + /// Snapshot id to read from (read queries only — rejected for a stored + /// mutation). Mutually exclusive with `branch`. + #[serde(default)] + pub snapshot: Option, +} + +/// Response for `POST /queries/{name}`: the read envelope for a stored +/// read, or the mutation envelope for a stored mutation. Serialized +/// **untagged**, so the wire shape is exactly [`ReadOutput`] or +/// [`ChangeOutput`] — classification follows the stored query, not a +/// wrapper field. +#[derive(Debug, Serialize, ToSchema)] +#[serde(untagged)] +pub enum InvokeStoredQueryResponse { + Read(ReadOutput), + Change(ChangeOutput), +} + +/// The kind of a stored-query parameter, decomposed so a client (e.g. an +/// MCP server) can build a typed input schema with a closed `match` and +/// never re-parse omnigraph's type spelling. `bigint`/`date`/`datetime`/ +/// `blob` are carried as JSON strings on the wire: a 64-bit integer past +/// 2^53 loses precision as a JSON number, and Date/DateTime are ISO +/// strings, Blob a blob-URI string. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ToSchema)] +#[serde(rename_all = "snake_case")] +pub enum ParamKind { + String, + Bool, + Int, + #[serde(rename = "bigint")] + BigInt, + Float, + Date, + #[serde(rename = "datetime")] + DateTime, + Blob, + Vector, + List, +} + +/// One declared parameter of a stored query, projected for the catalog. +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct ParamDescriptor { + pub name: String, + pub kind: ParamKind, + /// Element kind when `kind == list` (always a scalar — the grammar + /// forbids lists of vectors or nested lists). + #[serde(skip_serializing_if = "Option::is_none")] + pub item_kind: Option, + /// Dimension when `kind == vector`. + #[serde(skip_serializing_if = "Option::is_none")] + pub vector_dim: Option, + /// `false` → the caller must supply it; `true` → optional. + pub nullable: bool, +} + +/// One entry in the stored-query catalog (`GET /queries`). +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct QueryCatalogEntry { + /// Registry key / invoke path segment (`POST /queries/{name}`). + pub name: String, + /// MCP tool id (the `tool_name` override, else `name`). + pub tool_name: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub description: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub instruction: Option, + /// `true` for a stored mutation → an MCP read-only hint of `false`. + pub mutation: bool, + pub params: Vec, +} + +/// Response for `GET /queries`: the `mcp.expose` subset of a graph's +/// stored-query registry, each with typed parameters. +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct QueriesCatalogOutput { + pub queries: Vec, +} + +/// Total map from a resolved scalar to its catalog kind. Exhaustive on +/// purpose: a new `ScalarType` is a compile error here until catalogued. +fn scalar_kind(scalar: ScalarType) -> ParamKind { + match scalar { + ScalarType::String => ParamKind::String, + ScalarType::Bool => ParamKind::Bool, + ScalarType::I32 | ScalarType::U32 => ParamKind::Int, + ScalarType::I64 | ScalarType::U64 => ParamKind::BigInt, + ScalarType::F32 | ScalarType::F64 => ParamKind::Float, + ScalarType::Date => ParamKind::Date, + ScalarType::DateTime => ParamKind::DateTime, + ScalarType::Blob => ParamKind::Blob, + ScalarType::Vector(_) => ParamKind::Vector, + } +} + +fn param_descriptor(param: &Param) -> ParamDescriptor { + match PropType::from_param_type_name(¶m.type_name, param.nullable) { + Some(pt) if pt.list => ParamDescriptor { + name: param.name.clone(), + kind: ParamKind::List, + item_kind: Some(scalar_kind(pt.scalar)), + vector_dim: None, + nullable: param.nullable, + }, + Some(pt) => { + let (kind, vector_dim) = match pt.scalar { + ScalarType::Vector(dim) => (ParamKind::Vector, Some(dim)), + other => (scalar_kind(other), None), + }; + ParamDescriptor { + name: param.name.clone(), + kind, + item_kind: None, + vector_dim, + nullable: param.nullable, + } + } + // Unreachable for a parsed query (every declared param type is + // grammatical); fall back to an opaque string so the field is still + // usable rather than dropped. + None => ParamDescriptor { + name: param.name.clone(), + kind: ParamKind::String, + item_kind: None, + vector_dim: None, + nullable: param.nullable, + }, + } +} + +/// Project a loaded stored query into its catalog entry (typed params, +/// MCP tool name, read/mutate flag, description/instruction). +pub fn query_catalog_entry(query: &StoredQuery) -> QueryCatalogEntry { + QueryCatalogEntry { + name: query.name.clone(), + tool_name: query.effective_tool_name().to_string(), + description: query.decl.description.clone(), + instruction: query.decl.instruction.clone(), + mutation: query.is_mutation(), + params: query.decl.params.iter().map(param_descriptor).collect(), + } +} + #[derive(Debug, Clone, Default, Serialize, Deserialize, ToSchema)] pub struct SchemaApplyRequest { /// Project schema in `.pg` source form. The diff against the current diff --git a/crates/omnigraph-server/src/config.rs b/crates/omnigraph-server/src/config.rs index 87737d0..b308b72 100644 --- a/crates/omnigraph-server/src/config.rs +++ b/crates/omnigraph-server/src/config.rs @@ -9,6 +9,13 @@ use serde::{Deserialize, Serialize}; pub const DEFAULT_CONFIG_FILE: &str = "omnigraph.yaml"; +pub fn graph_resource_id_for_selection( + selected_graph: Option<&str>, + normalized_uri: &str, +) -> String { + selected_graph.unwrap_or(normalized_uri).to_string() +} + #[derive(Debug, Clone, Default, Serialize, Deserialize)] pub struct ProjectConfig { pub name: Option, @@ -24,6 +31,14 @@ pub struct TargetConfig { /// graph's HTTP-layer Cedar enforcement. #[serde(default)] pub policy: PolicySettings, + /// Per-graph stored-query registry: an inline `name -> entry` + /// map. Mirrors the per-graph `policy` shape — each + /// `graphs..queries` declares that graph's stored queries. Absent + /// (or empty) = no stored queries for the graph. v1 is inline-only; + /// an external `queries.yaml` manifest indirection is a deferred + /// convenience. + #[serde(default)] + pub queries: BTreeMap, } #[derive(Debug, Clone, Copy, Default, Eq, PartialEq, Serialize, Deserialize, ValueEnum)] @@ -90,6 +105,50 @@ pub struct PolicySettings { pub file: Option, } +/// One stored-query registry entry. The map **key** is the query's +/// identity — it must equal the `query ` symbol declared inside +/// the referenced `.gq` file (asserted when the registry loads). +/// Renaming the key (or the symbol) is a breaking change to callers, by +/// design. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct QueryEntry { + /// Path to the `.gq` file (relative to the config's `base_dir`). The + /// file may declare several queries; the registry selects the one + /// whose symbol matches the map key. + pub file: String, + #[serde(default)] + pub mcp: McpSettings, +} + +/// MCP exposure for a stored query. A *deployment* concern (the same +/// `.gq` may be exposed in one graph and hidden in another), so it lives +/// in YAML rather than in the `.gq` source. **Default `expose: true`** — +/// declaring a query in the manifest *is* the opt-in, so it appears in the +/// MCP tool catalog (`GET /queries`) by default; set `expose: false` to +/// keep a query HTTP/service-callable but hidden from the agent tool list. +/// `expose` governs catalog membership only — it is **not** an +/// authorization gate (invocation is gated by `invoke_query`), so a hidden +/// query is still invocable by name with the right permission. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct McpSettings { + #[serde(default = "mcp_expose_default")] + pub expose: bool, + pub tool_name: Option, +} + +fn mcp_expose_default() -> bool { + true +} + +impl Default for McpSettings { + fn default() -> Self { + Self { + expose: mcp_expose_default(), + tool_name: None, + } + } +} + #[derive(Debug, Clone, Copy, Eq, PartialEq, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] pub enum AliasCommand { @@ -137,6 +196,12 @@ pub struct OmnigraphConfig { pub aliases: BTreeMap, #[serde(default)] pub policy: PolicySettings, + /// Top-level stored-query registry, used in single-graph + /// mode — mirrors how the top-level `policy` applies to the single + /// graph. In multi-graph mode this is unused; each graph's + /// `graphs..queries` applies instead. + #[serde(default)] + pub queries: BTreeMap, #[serde(skip)] base_dir: PathBuf, } @@ -152,6 +217,7 @@ impl Default for OmnigraphConfig { query: QueryDefaults::default(), aliases: BTreeMap::new(), policy: PolicySettings::default(), + queries: BTreeMap::new(), base_dir: PathBuf::new(), } } @@ -244,6 +310,124 @@ impl OmnigraphConfig { .map(|path| self.resolve_config_path(path)) } + /// The top-level stored-query registry entries (single-graph mode). + pub fn query_entries(&self) -> &BTreeMap { + &self.queries + } + + /// The per-graph stored-query registry entries for a named target + /// (multi-graph mode). Returns `None` if the target is unknown. + pub fn target_query_entries( + &self, + target_name: &str, + ) -> Option<&BTreeMap> { + self.graphs.get(target_name).map(|target| &target.queries) + } + + /// The stored-query registry entries that apply for a graph + /// selection — the single definition of "which `queries:` block + /// governs graph X", shared by server boot and the CLI so the two + /// can't drift. A named graph present in `graphs:` uses its + /// per-graph block; everything else (no selection, or a name that is + /// not a known graph, e.g. a bare URI) falls back to the top-level + /// block (single-graph mode). + pub fn query_entries_for(&self, graph: Option<&str>) -> &BTreeMap { + match graph { + Some(name) if self.graphs.contains_key(name) => &self.graphs[name].queries, + _ => &self.queries, + } + } + + /// The single CLI gate that turns a raw graph selection into a *validated* + /// one — the fallible counterpart to the infallible + /// [`OmnigraphConfig::query_entries_for`]. Both `queries` subcommands route + /// their selection through here so neither can skip a check the other (or + /// server boot) applies: + /// * a known name passes through, but only after the same coherence check + /// server boot enforces + /// ([`OmnigraphConfig::ensure_top_level_blocks_honored`]) — a named graph + /// with a populated top-level block is rejected; + /// * an unknown name errors with the **same** message + /// [`OmnigraphConfig::resolve_target_uri`] produces, so a command that + /// opens no URI rejects an unknown `--target` exactly like the + /// URI-resolving commands do; + /// * an anonymous selection (`None`, e.g. a bare URI) stays anonymous, + /// resolving to the top-level registry downstream (top-level honored). + pub fn resolve_graph_selection<'a>(&self, graph: Option<&'a str>) -> Result> { + match graph { + Some(name) if self.graphs.contains_key(name) => { + self.ensure_top_level_blocks_honored(Some(name))?; + Ok(Some(name)) + } + Some(name) => bail!("graph '{}' not found in {}", name, DEFAULT_CONFIG_FILE), + None => Ok(None), + } + } + + pub fn resolve_policy_tooling_graph_selection(&self) -> Result> { + self.resolve_graph_selection(self.cli_graph_name().or_else(|| self.server_graph_name())) + } + + /// The policy file that applies for a graph selection — the policy + /// sibling of [`OmnigraphConfig::query_entries_for`], so policy and + /// queries resolve by the same identity rule. A named graph in + /// `graphs:` uses its per-graph `policy.file` with **no** top-level + /// fallback (a named graph with no per-graph policy has no policy — + /// that keeps the boot-time coherence check meaningful); anything else + /// (no selection, or a bare URI) uses the top-level `policy.file`. + pub fn resolve_policy_file_for(&self, graph: Option<&str>) -> Option { + match graph { + Some(name) if self.graphs.contains_key(name) => self.resolve_target_policy_file(name), + _ => self.resolve_policy_file(), + } + } + + /// Names of any top-level config blocks (`policy.file`, `queries:`) + /// that are populated. Used by the boot-time coherence check: when a + /// **named** graph is served (single-mode by name, or multi-mode), + /// the top-level blocks are not honored, so a populated one is a + /// configuration error rather than a silent no-op. + pub fn populated_top_level_blocks(&self) -> Vec<&'static str> { + let mut blocks = Vec::new(); + if self.policy.file.is_some() { + blocks.push("policy.file"); + } + if !self.queries.is_empty() { + blocks.push("queries"); + } + blocks + } + + /// A named graph uses its own `graphs.` block, so a populated + /// top-level block would be silently ignored — a config error. The single + /// definition of that rule, shared by server boot and the CLI selection + /// gate ([`OmnigraphConfig::resolve_graph_selection`]) so the two can't + /// drift. An anonymous selection (`None`, e.g. a bare URI) legitimately + /// honors the top-level blocks, so it is never rejected here. + pub fn ensure_top_level_blocks_honored(&self, selected: Option<&str>) -> Result<()> { + if let Some(name) = selected { + let unhonored = self.populated_top_level_blocks(); + if !unhonored.is_empty() { + bail!( + "named graph '{name}' uses its own `graphs.{name}.…` block, but top-level {} \ + {} set and would be ignored. Move it to `graphs.{name}` (e.g. \ + `graphs.{name}.policy.file`, `graphs.{name}.queries`).", + unhonored.join(" and "), + if unhonored.len() == 1 { "is" } else { "are" }, + ); + } + } + Ok(()) + } + + /// Resolve a stored-query `.gq` file path (from a registry entry), + /// relative to the config's `base_dir`. Mirrors policy-file + /// resolution; the registry loader calls this to turn each entry's + /// `file:` value into an absolute path. + pub fn resolve_query_file(&self, value: &str) -> PathBuf { + self.resolve_config_path(value) + } + /// Resolve the server-level policy file path (used by management /// endpoints). Returns `None` if `server.policy.file` is not set. pub fn resolve_server_policy_file(&self) -> Option { @@ -387,7 +571,9 @@ mod tests { use tempfile::tempdir; - use super::{ReadOutputFormat, TableCellLayout, load_config_in}; + use super::{ + ReadOutputFormat, TableCellLayout, graph_resource_id_for_selection, load_config_in, + }; #[test] fn load_config_reads_yaml_defaults_from_current_dir() { @@ -451,6 +637,114 @@ policy: {} assert!(config.graphs.is_empty()); } + #[test] + fn graph_resource_id_for_selection_uses_name_or_anonymous_uri() { + assert_eq!( + graph_resource_id_for_selection(Some("local"), "/tmp/graph.omni"), + "local" + ); + assert_eq!( + graph_resource_id_for_selection(None, "/tmp/graph.omni"), + "/tmp/graph.omni" + ); + } + + #[test] + fn resolve_graph_selection_validates_membership_and_coherence() { + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("omnigraph.yaml"), + "graphs:\n local:\n uri: ./demo.omni\n", + ) + .unwrap(); + let config = load_config_in(temp.path(), None).unwrap(); + + // A known graph passes through unchanged. + assert_eq!(config.resolve_graph_selection(Some("local")).unwrap(), Some("local")); + // An anonymous selection stays anonymous (→ top-level registry downstream). + assert_eq!(config.resolve_graph_selection(None).unwrap(), None); + // An unknown name errors, naming the graph (matching resolve_target_uri). + let err = config.resolve_graph_selection(Some("ghost")).unwrap_err().to_string(); + assert!( + err.contains("ghost") && err.contains("not found"), + "unknown graph must error naming it: {err}" + ); + + // Coherence: a named graph plus a populated top-level block is the + // config server boot refuses, so the gate rejects it too (shared rule + // via ensure_top_level_blocks_honored). An anonymous selection still + // passes — top-level is honored when no graph is named. + let temp2 = tempdir().unwrap(); + fs::write( + temp2.path().join("omnigraph.yaml"), + "graphs:\n local:\n uri: ./demo.omni\npolicy:\n file: ./top.yaml\n", + ) + .unwrap(); + let incoherent = load_config_in(temp2.path(), None).unwrap(); + let err = incoherent + .resolve_graph_selection(Some("local")) + .unwrap_err() + .to_string(); + assert!( + err.contains("local") && err.contains("policy.file"), + "named graph + populated top-level block must be rejected, naming both: {err}" + ); + assert_eq!( + incoherent.resolve_graph_selection(None).unwrap(), + None, + "anonymous selection still honors top-level" + ); + } + + #[test] + fn policy_tooling_graph_selection_prefers_cli_then_server_and_validates() { + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("omnigraph.yaml"), + "graphs:\n local:\n uri: ./local.omni\n prod:\n uri: ./prod.omni\n\ + server:\n graph: local\ncli:\n graph: prod\n", + ) + .unwrap(); + let config = load_config_in(temp.path(), None).unwrap(); + assert_eq!( + config.resolve_policy_tooling_graph_selection().unwrap(), + Some("prod") + ); + + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("omnigraph.yaml"), + "graphs:\n local:\n uri: ./local.omni\nserver:\n graph: local\n", + ) + .unwrap(); + let config = load_config_in(temp.path(), None).unwrap(); + assert_eq!( + config.resolve_policy_tooling_graph_selection().unwrap(), + Some("local") + ); + + let temp = tempdir().unwrap(); + fs::write(temp.path().join("omnigraph.yaml"), "policy: {}\n").unwrap(); + let config = load_config_in(temp.path(), None).unwrap(); + assert_eq!(config.resolve_policy_tooling_graph_selection().unwrap(), None); + + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("omnigraph.yaml"), + "graphs:\n local:\n uri: ./local.omni\nserver:\n graph: ghost\n", + ) + .unwrap(); + let config = load_config_in(temp.path(), None).unwrap(); + let err = config + .resolve_policy_tooling_graph_selection() + .unwrap_err() + .to_string(); + assert!( + err.contains("ghost") && err.contains("not found"), + "unknown server.graph must use graph-selection validation: {err}" + ); + } + #[test] fn resolve_query_path_searches_config_roots() { let temp = tempdir().unwrap(); @@ -489,6 +783,118 @@ policy: {} assert_eq!(resolved, config_dir.join("local.gq")); } + #[test] + fn queries_block_round_trips_inline_and_per_graph() { + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("omnigraph.yaml"), + r#" +graphs: + prod: + uri: s3://bucket/prod + queries: + find_user: + file: ./queries/find_user.gq + mcp: + expose: true + tool_name: lookup_user + internal_audit: + file: ./queries/audit.gq +queries: + single_mode_q: + file: ./q.gq +"#, + ) + .unwrap(); + + let config = load_config_in(temp.path(), None).unwrap(); + + // Per-graph registry (multi-graph mode). + let prod = config.target_query_entries("prod").unwrap(); + assert_eq!(prod.len(), 2); + let find_user = &prod["find_user"]; + assert_eq!(find_user.file, "./queries/find_user.gq"); + assert!(find_user.mcp.expose); + assert_eq!(find_user.mcp.tool_name.as_deref(), Some("lookup_user")); + // Default exposure is true (the manifest entry is the opt-in); tool_name absent. + let audit = &prod["internal_audit"]; + assert!(audit.mcp.expose); + assert!(audit.mcp.tool_name.is_none()); + + // Top-level registry (single-graph mode). + assert_eq!(config.query_entries().len(), 1); + + // The shared selector resolves the same blocks the server boot + // and the CLI use: a known graph → its per-graph block; no + // selection or an unknown name → the top-level block (the latter + // pins the behavior of the CLI's now-deleted fallback arm). + assert_eq!(config.query_entries_for(Some("prod")).len(), 2); + assert_eq!(config.query_entries_for(None).len(), 1); + assert_eq!(config.query_entries_for(Some("nonexistent")).len(), 1); + + // Path resolution joins against base_dir, like policy files. + assert_eq!( + config.resolve_query_file(&find_user.file), + temp.path().join("./queries/find_user.gq") + ); + } + + #[test] + fn resolve_policy_file_for_follows_identity() { + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("omnigraph.yaml"), + "policy:\n file: ./top.yaml\ngraphs:\n prod:\n uri: s3://b/prod\n \ + policy:\n file: ./prod.yaml\n bare:\n uri: s3://b/bare\n", + ) + .unwrap(); + let config = load_config_in(temp.path(), None).unwrap(); + + // Named graph with its own policy → per-graph (not top-level). + assert!( + config + .resolve_policy_file_for(Some("prod")) + .unwrap() + .ends_with("prod.yaml") + ); + // Named graph with NO per-graph policy → None (no top-level fallback; + // load-bearing for the boot coherence check). + assert!(config.resolve_policy_file_for(Some("bare")).is_none()); + // Anonymous (bare URI) or an unknown name → top-level. + assert!( + config + .resolve_policy_file_for(None) + .unwrap() + .ends_with("top.yaml") + ); + assert!( + config + .resolve_policy_file_for(Some("nope")) + .unwrap() + .ends_with("top.yaml") + ); + } + + #[test] + fn queries_block_absent_yields_empty_registry() { + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("omnigraph.yaml"), + "graphs:\n local:\n uri: ./demo.omni\n", + ) + .unwrap(); + + let config = load_config_in(temp.path(), None).unwrap(); + // Additive: no `queries:` anywhere → empty registries everywhere. + assert!(config.query_entries().is_empty()); + assert!( + config + .target_query_entries("local") + .unwrap() + .is_empty() + ); + } + #[test] fn policy_block_accepts_non_empty_mapping() { let temp = tempdir().unwrap(); diff --git a/crates/omnigraph-server/src/lib.rs b/crates/omnigraph-server/src/lib.rs index ad41f9d..60ebef3 100644 --- a/crates/omnigraph-server/src/lib.rs +++ b/crates/omnigraph-server/src/lib.rs @@ -4,6 +4,7 @@ pub mod config; pub mod graph_id; pub mod identity; pub mod policy; +pub mod queries; pub mod registry; pub mod workload; @@ -11,6 +12,8 @@ pub use graph_id::GraphId; pub use identity::{AuthSource, GraphKey, ResolvedActor, Scope, TenantId}; pub use registry::{GraphHandle, GraphRegistry, InsertError, RegistryLookup, RegistrySnapshot}; +use crate::queries::{QueryRegistry, check, format_check_breakages}; + use std::collections::{HashMap, HashSet}; use std::fs; use std::io; @@ -22,7 +25,8 @@ use api::{ BranchCreateOutput, BranchCreateRequest, BranchDeleteOutput, BranchListOutput, BranchMergeOutput, BranchMergeRequest, ChangeOutput, ChangeRequest, CommitListOutput, CommitListQuery, ErrorCode, ErrorOutput, ExportRequest, GraphInfo, GraphListResponse, - HealthOutput, IngestOutput, IngestRequest, QueryRequest, ReadOutput, ReadRequest, + HealthOutput, IngestOutput, IngestRequest, InvokeStoredQueryRequest, + InvokeStoredQueryResponse, QueriesCatalogOutput, QueryRequest, ReadOutput, ReadRequest, SchemaApplyOutput, SchemaApplyRequest, SchemaOutput, SnapshotQuery, ingest_output, schema_apply_output, snapshot_payload, }; @@ -40,12 +44,13 @@ use color_eyre::eyre::{Result, WrapErr, bail}; pub use config::{ AliasCommand, AliasConfig, CliDefaults, DEFAULT_CONFIG_FILE, OmnigraphConfig, PolicySettings, ProjectConfig, QueryDefaults, ReadOutputFormat, ServerDefaults, TableCellLayout, TargetConfig, - load_config, + graph_resource_id_for_selection, load_config, }; use futures::stream; use omnigraph::db::{Omnigraph, ReadTarget}; use omnigraph::error::{ManifestConflictDetails, ManifestErrorKind, OmniError}; use omnigraph::storage::normalize_root_uri; +use omnigraph_compiler::catalog::Catalog; use omnigraph_compiler::json_params_to_param_map; use omnigraph_compiler::query::parser::parse_query; use omnigraph_compiler::{JsonParamMode, ParamMap}; @@ -93,6 +98,8 @@ fn hash_bearer_token(token: &str) -> BearerTokenHash { server_export, #[allow(deprecated)] server_change, server_mutate, + server_list_queries, + server_invoke_query, server_schema_apply, server_schema_get, server_ingest, @@ -157,8 +164,16 @@ pub enum ServerConfigMode { /// set to a named target. Single { uri: String, + /// Cedar graph resource id for the single graph. A named selection + /// uses the graph name; an anonymous URI uses the normalized URI to + /// preserve legacy single-graph policy identity. + graph_id: String, /// Top-level `policy.file` (single-graph Cedar policy). policy_file: Option, + /// Top-level stored-query registry, loaded and identity-checked + /// at settings-build time; type-checked against the schema when + /// the engine opens. + queries: QueryRegistry, }, /// Multi-graph invocation — `--config omnigraph.yaml` with a /// non-empty `graphs:` map and no single-mode selector. @@ -185,6 +200,10 @@ pub struct GraphStartupConfig { pub graph_id: String, pub uri: String, pub policy_file: Option, + /// Per-graph stored-query registry, loaded and identity-checked at + /// settings-build time; type-checked against the schema when this + /// graph's engine opens. + pub queries: QueryRegistry, } /// Runtime routing for the server. Single mode = legacy @@ -285,7 +304,31 @@ impl AppState { ) -> Self { let bearer_tokens = hash_bearer_tokens(bearer_tokens); let per_graph_policy = policy_engine.map(Arc::new); - Self::build_single_mode(uri, db, bearer_tokens, per_graph_policy, Arc::new(workload)) + Self::build_single_mode(uri, db, bearer_tokens, per_graph_policy, Arc::new(workload), None) + } + + /// Like `new_single`, but attaches a pre-validated stored-query + /// registry. Private — the production single-mode boot path + /// (`open_single_with_queries`) is the only caller; every public + /// `new_*` constructor builds with no stored queries. + fn new_single_with_queries( + uri: String, + db: Omnigraph, + bearer_tokens: Vec<(String, String)>, + policy_engine: Option, + workload: workload::WorkloadController, + queries: Option>, + ) -> Self { + let bearer_tokens = hash_bearer_tokens(bearer_tokens); + let per_graph_policy = policy_engine.map(Arc::new); + Self::build_single_mode( + uri, + db, + bearer_tokens, + per_graph_policy, + Arc::new(workload), + queries, + ) } pub fn new(uri: String, db: Omnigraph) -> Self { @@ -377,6 +420,39 @@ impl AppState { uri: impl Into, bearer_tokens: Vec<(String, String)>, policy_file: Option<&PathBuf>, + ) -> Result { + Self::open_single_with_queries( + uri, + bearer_tokens, + policy_file, + QueryRegistry::default(), + ) + .await + } + + /// Single-mode boot with a stored-query registry: open the engine, + /// **type-check the registry against the live schema and refuse to + /// start on a breakage** (same posture as bad policy YAML), log + /// non-blocking warnings, then attach the registry to the handle. + /// With an empty registry the check is a no-op and no registry is + /// attached — that is the path `open_with_bearer_tokens_and_policy` + /// (no stored queries) takes. + pub async fn open_single_with_queries( + uri: impl Into, + bearer_tokens: Vec<(String, String)>, + policy_file: Option<&PathBuf>, + queries: QueryRegistry, + ) -> Result { + Self::open_single_with_queries_for_graph_id(uri, bearer_tokens, policy_file, queries, None) + .await + } + + async fn open_single_with_queries_for_graph_id( + uri: impl Into, + bearer_tokens: Vec<(String, String)>, + policy_file: Option<&PathBuf>, + queries: QueryRegistry, + graph_id: Option, ) -> Result { // The "policy requires tokens" invariant is enforced once by // `classify_server_runtime_state` in `serve()`, before either @@ -384,16 +460,24 @@ impl AppState { // time we get here, the (policy, no-tokens) combination has // already been rejected — no second bail needed. let uri = normalize_root_uri(&uri.into()).wrap_err("normalize graph URI")?; + let graph_id = graph_id.unwrap_or_else(|| uri.clone()); let db = Omnigraph::open(&uri).await?; + + // Validate the registry against the live schema and resolve it to + // an attachable handle (refuse boot on breakage). + let registry = validate_and_attach(queries, &db.catalog(), &graph_id)?; + let policy_engine = match policy_file { - Some(path) => Some(PolicyEngine::load_graph(path, &uri)?), + Some(path) => Some(PolicyEngine::load_graph(path, &graph_id)?), None => None, }; - Ok(Self::new_with_bearer_tokens_and_policy( + Ok(Self::new_single_with_queries( uri, db, bearer_tokens, policy_engine, + workload::WorkloadController::from_env(), + registry, )) } @@ -408,6 +492,7 @@ impl AppState { bearer_tokens: Arc<[(BearerTokenHash, Arc)]>, policy_engine: Option>, workload: Arc, + queries: Option>, ) -> Self { // Engine-layer policy gate (MR-722). With a per-graph policy // installed, every `_as` writer on `Omnigraph` calls into the @@ -436,6 +521,7 @@ impl AppState { uri, engine: Arc::new(db), policy: policy_engine, + queries, }); Self { routing: GraphRouting::Single { handle }, @@ -750,6 +836,58 @@ pub fn init_tracing() { let _ = tracing_subscriber::fmt().with_env_filter(filter).try_init(); } +/// Log each non-blocking advisory from a registry check report. +fn log_registry_warnings(label: &str, report: &queries::CheckReport) { + for warning in &report.warnings { + warn!(graph = label, query = %warning.query, "stored query: {}", warning.message); + } +} + +fn validate_registry_against_catalog( + registry: &QueryRegistry, + catalog: &Catalog, + label: &str, +) -> omnigraph::error::Result<()> { + let report = check(registry, catalog); + if report.has_breakages() { + return Err(OmniError::manifest(format_check_breakages(label, &report))); + } + log_registry_warnings(label, &report); + Ok(()) +} + +/// Validate a loaded stored-query registry against the live schema and +/// resolve it to an attachable handle. Refuses boot on any breakage +/// (same posture as bad policy YAML), logs the non-blocking warnings, +/// and collapses an empty registry to `None` (nothing attached). This is +/// the single gate every open path funnels through, so no opener can +/// attach a registry that has not been schema-checked. `label` names the +/// graph in messages. +fn validate_and_attach( + queries: QueryRegistry, + catalog: &Catalog, + label: &str, +) -> Result>> { + validate_registry_against_catalog(&queries, catalog, label) + .map_err(|err| color_eyre::eyre::eyre!(err.to_string()))?; + Ok(if queries.is_empty() { + None + } else { + Some(Arc::new(queries)) + }) +} + +/// Format every load error (parse / identity failure) into a multi-line +/// boot-abort message. +fn format_registry_load_errors(label: &str, errors: &[queries::LoadError]) -> String { + let joined = errors + .iter() + .map(|e| e.to_string()) + .collect::>() + .join("\n "); + format!("graph '{label}': stored-query registry failed to load:\n {joined}") +} + pub fn load_server_settings( config_path: Option<&PathBuf>, cli_uri: Option, @@ -799,15 +937,43 @@ pub fn load_server_settings( let uri = normalize_root_uri(&raw_uri).wrap_err_with(|| { format!("normalize single-graph URI '{raw_uri}' from server settings") })?; - let policy_file = config.resolve_policy_file(); - ServerConfigMode::Single { uri, policy_file } + // Config follows graph IDENTITY, not mode: a bare URI is anonymous + // (top-level config); a graph chosen by name uses its per-graph + // `graphs..{policy,queries}`. `resolve_target_uri` already + // errored on an unknown name, so a `Some(name)` here is a known graph. + let selected: Option<&str> = if has_cli_uri { + None + } else { + cli_target.as_deref().or_else(|| config.server_graph_name()) + }; + // A named selection must not leave a populated top-level block + // silently unused — refuse boot and point at the per-graph block. The + // same rule the CLI selection gate enforces, shared via one helper so + // the boot check and `omnigraph queries validate`/`list` can't drift. + config.ensure_top_level_blocks_honored(selected)?; + // Load + identity-check now (no engine needed); the schema + // type-check happens when the engine opens. + let policy_file = config.resolve_policy_file_for(selected); + let queries = QueryRegistry::load(&config, config.query_entries_for(selected)) + .map_err(|errs| color_eyre::eyre::eyre!(format_registry_load_errors(&uri, &errs)))?; + let graph_id = graph_resource_id_for_selection(selected, &uri); + ServerConfigMode::Single { + uri, + graph_id, + policy_file, + queries, + } } else if has_explicit_config && has_graphs_map { - if config.resolve_policy_file().is_some() { + // Multi mode: every graph uses its per-graph block; top-level + // policy/queries are never honored, so a populated one is an error. + let unhonored = config.populated_top_level_blocks(); + if !unhonored.is_empty() { bail!( - "top-level `policy.file` is single-graph/CLI-local policy only; \ - in multi-graph mode move per-graph rules to \ - `graphs..policy.file` and move `graph_list` rules to \ - `server.policy.file`." + "multi-graph mode: top-level {} {} not honored — each graph uses its own \ + `graphs..…` block. Move per-graph rules there (and any \ + `graph_list` policy to `server.policy.file`).", + unhonored.join(" and "), + if unhonored.len() == 1 { "is" } else { "are" }, ); } // Rule 4 → Multi mode. Build a startup config per graph. @@ -823,10 +989,17 @@ pub fn load_server_settings( let uri = normalize_root_uri(&raw_uri).wrap_err_with(|| { format!("normalize URI '{raw_uri}' for graph '{name}' in omnigraph.yaml") })?; + // Per-graph `queries:`, selected through the shared + // `query_entries_for` so server and CLI resolve identically. + // Load + identity-check now; the schema type-check happens + // when this graph's engine opens. + let queries = QueryRegistry::load(&config, config.query_entries_for(Some(name.as_str()))) + .map_err(|errs| color_eyre::eyre::eyre!(format_registry_load_errors(name, &errs)))?; graphs.push(GraphStartupConfig { graph_id: name.clone(), uri, policy_file: config.resolve_target_policy_file(name), + queries, }); } let config_path = config_path @@ -949,6 +1122,8 @@ pub fn build_app(state: AppState) -> Router { server_change })) .route("/mutate", post(server_mutate)) + .route("/queries", get(server_list_queries)) + .route("/queries/{name}", post(server_invoke_query)) .route("/schema", get(server_schema_get)) .route("/schema/apply", post(server_schema_apply)) .route( @@ -1046,10 +1221,28 @@ pub async fn serve(config: ServerConfig) -> Result<()> { let bind = config.bind.clone(); let state = match config.mode { - ServerConfigMode::Single { uri, policy_file } => { + ServerConfigMode::Single { + uri, + graph_id, + policy_file, + queries, + } => { let uri_for_log = uri.clone(); - info!(uri = %uri_for_log, bind = %bind, mode = "single", "serving omnigraph"); - AppState::open_with_bearer_tokens_and_policy(uri, tokens, policy_file.as_ref()).await? + info!( + uri = %uri_for_log, + graph_id = %graph_id, + bind = %bind, + mode = "single", + "serving omnigraph" + ); + AppState::open_single_with_queries_for_graph_id( + uri, + tokens, + policy_file.as_ref(), + queries, + Some(graph_id), + ) + .await? } ServerConfigMode::Multi { graphs, @@ -1131,6 +1324,12 @@ async fn open_single_graph(cfg: GraphStartupConfig) -> Result> .await .map_err(|err| color_eyre::eyre::eyre!("open graph '{}' at {}: {err}", graph_id, uri))?; + // Validate this graph's stored queries against the live schema and + // resolve them to an attachable handle (refuse boot on breakage). + // Done before the policy match rebinds `db`; the catalog handle is an + // owned `Arc`, so no borrow of `db` survives into the match. + let queries = validate_and_attach(cfg.queries, &db.catalog(), graph_id.as_str())?; + let (policy_arc, db) = match &cfg.policy_file { Some(path) => { let policy = PolicyEngine::load_graph(path, graph_id.as_str())?; @@ -1146,6 +1345,7 @@ async fn open_single_graph(cfg: GraphStartupConfig) -> Result> uri, engine: Arc::new(db), policy: policy_arc, + queries, })) } @@ -1479,7 +1679,21 @@ fn log_policy_decision(actor_id: &str, request: &PolicyRequest, decision: &Polic ); } -/// HTTP-layer Cedar policy gate. Two sources of the policy engine: +/// The allow/deny **decision** an authorization check produces, kept +/// separate from the operational failures (`Err`) that can occur while +/// computing it. [`authorize_request`] collapses `Denied` to a 403; a caller +/// that needs to remap a denial without also remapping operational failures +/// (the stored-query invoke handler hides a denial as a 404) matches on this +/// directly, so a real 401 (missing bearer) or 500 (policy-evaluation error) +/// keeps its true status instead of being masked as the denial's response. +enum Authz { + Allowed, + Denied(String), +} + +/// HTTP-layer Cedar policy gate, returning the allow/deny [`Authz`] decision +/// and reserving `Err` for operational failures (401 missing bearer, 500 +/// policy-evaluation error). Two sources of the policy engine: /// * Per-graph handler — passes `handle.policy.as_deref()` so the /// graph's Cedar rules govern read/change/branch_*/schema_apply. /// * Management handler — passes `state.server_policy.as_deref()` so @@ -1493,11 +1707,11 @@ fn log_policy_decision(actor_id: &str, request: &PolicyRequest, decision: &Polic /// dropped from the type), so handlers cannot smuggle it through the /// request. See `actor_id_resolves_from_bearer_token_ignoring_client_supplied_headers` /// at `tests/server.rs`. -fn authorize_request( +fn authorize( actor: Option<&ResolvedActor>, policy: Option<&PolicyEngine>, request: PolicyRequest, -) -> std::result::Result<(), ApiError> { +) -> std::result::Result { let Some(engine) = policy else { // No PolicyEngine installed. Three runtime states can reach this: // @@ -1524,21 +1738,23 @@ fn authorize_request( // operator's only path to enabling it is configuring an // explicit `server.policy.file` in omnigraph.yaml. if request.action.resource_kind() == PolicyResourceKind::Server { - return Err(ApiError::forbidden( + return Ok(Authz::Denied( "server-scoped actions require an explicit `server.policy.file` \ configured in omnigraph.yaml — the management surface is closed \ by default in every runtime state, including --unauthenticated, \ - so that server topology is never exposed without operator opt-in.", + so that server topology is never exposed without operator opt-in." + .to_string(), )); } if actor.is_some() && request.action != PolicyAction::Read { - return Err(ApiError::forbidden( + return Ok(Authz::Denied( "server runs in default-deny mode (bearer tokens configured but no \ policy file). Only `read` actions are permitted; configure \ - `policy.file` in omnigraph.yaml to enable other actions.", + `policy.file` in omnigraph.yaml to enable other actions." + .to_string(), )); } - return Ok(()); + return Ok(Authz::Allowed); }; let Some(actor) = actor else { return Err(ApiError::unauthorized("missing bearer token")); @@ -1560,9 +1776,26 @@ fn authorize_request( .map_err(|err| ApiError::internal(format!("policy: {err}")))?; log_policy_decision(actor_id, &request, &decision); if decision.allowed { - Ok(()) + Ok(Authz::Allowed) } else { - Err(ApiError::forbidden(decision.message)) + Ok(Authz::Denied(decision.message)) + } +} + +/// Thin wrapper over [`authorize`] for the handlers that treat any denial as a +/// 403: a denial becomes `ApiError::forbidden`, and operational failures +/// (401 missing bearer, 500 policy-evaluation error) propagate unchanged. The +/// stored-query invoke handler does **not** use this — it consumes the +/// [`Authz`] decision directly to hide a denial as a 404 while letting an +/// operational failure keep its true status. +fn authorize_request( + actor: Option<&ResolvedActor>, + policy: Option<&PolicyEngine>, + request: PolicyRequest, +) -> std::result::Result<(), ApiError> { + match authorize(actor, policy, request)? { + Authz::Allowed => Ok(()), + Authz::Denied(message) => Err(ApiError::forbidden(message)), } } @@ -2001,6 +2234,194 @@ async fn server_mutate( )) } +/// Path parameter for `POST /queries/{name}`. +#[derive(Deserialize)] +struct QueryNamePath { + name: String, +} + +fn parse_optional_invoke_body( + body: Bytes, +) -> std::result::Result { + if body.is_empty() { + return Ok(InvokeStoredQueryRequest::default()); + } + serde_json::from_slice::>(&body) + .map(|request| request.unwrap_or_default()) + .map_err(|err| { + ApiError::bad_request(format!("invalid stored-query invocation body: {err}")) + }) +} + +#[utoipa::path( + post, + path = "/queries/{name}", + tag = "queries", + operation_id = "invoke_query", + params(("name" = String, Path, description = "Stored query name (the registry key)")), + request_body = Option, + responses( + (status = 200, description = "Read envelope (ReadOutput) or mutation envelope (ChangeOutput), serialized untagged", body = InvokeStoredQueryResponse), + (status = 400, description = "Bad request (param type error; snapshot on a stored mutation)", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden (the inner `change` gate for a stored mutation)", body = ErrorOutput), + (status = 404, description = "Unknown stored query, or `invoke_query` denied — indistinguishable to a caller without the grant", body = ErrorOutput), + (status = 409, description = "Merge conflict", body = ErrorOutput), + (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), + (status = 500, description = "Policy evaluation error (a denial is reported as 404, not 500)", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Invoke a curated, server-side stored query by name. +/// +/// The query source comes from the graph's `queries:` registry, not the +/// request body — callers send only runtime inputs (`params`, `branch`, +/// `snapshot`). Gated by the `invoke_query` Cedar action at the boundary; +/// a stored *mutation* additionally passes the engine's `change` gate +/// (double-gated). An actor **without** `invoke_query` cannot tell a denied +/// query from a missing one — both return the same 404, so the catalog +/// can't be probed without the grant. Once `invoke_query` is held, the +/// inner `read`/`change` gate may surface a 403 for an existing query the +/// actor can't run (the intended double-gate signal). +async fn server_invoke_query( + State(state): State, + Extension(handle): Extension>, + actor: Option>, + Path(QueryNamePath { name }): Path, + body: Bytes, +) -> std::result::Result, ApiError> { + let req = parse_optional_invoke_body(body)?; + // A caller without `invoke_query` can't tell a denial from a missing + // query: both 404 with this exact message, so the catalog can't be + // probed without the grant. (A caller that holds invoke_query may still + // see the inner gate's 403 for an existing query it can't run — intended.) + const NOT_FOUND: &str = "stored query not found"; + let actor_ref = actor.as_ref().map(|Extension(actor)| actor); + + // Boundary gate (authentication already ran in `require_bearer_auth`). + // A denial is hidden as 404 (deny == missing, so the catalog can't be + // probed without the grant), but operational failures (401 missing bearer, + // 500 policy-evaluation error) propagate with their true status via `?` + // rather than being masked as a missing query. + match authorize( + actor_ref, + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::InvokeQuery, + // Graph-scoped: no branch dimension. The per-branch/snapshot + // access is enforced by the inner read/change gate in the + // runner, so the outer gate must not resolve a branch (doing so + // was wrong for snapshot reads). + branch: None, + target_branch: None, + }, + )? { + Authz::Allowed => {} + Authz::Denied(_) => return Err(ApiError::not_found(NOT_FOUND)), + } + + // Resolve against the per-graph registry (same 404 on a miss). + let stored = handle + .queries + .as_ref() + .and_then(|registry| registry.lookup(&name)) + .ok_or_else(|| ApiError::not_found(NOT_FOUND))?; + + // Detach what we need before `handle` moves into the runner — the + // registry borrow lives inside `handle`. + let source = Arc::clone(&stored.source); + let query_name = stored.name.clone(); + let is_mutation = stored.is_mutation(); + + info!( + graph = %handle.uri, + actor = ?actor_ref.map(|a| a.actor_id.as_ref()), + query = %query_name, + kind = if is_mutation { "mutate" } else { "read" }, + "stored query invoked" + ); + + if is_mutation { + if req.snapshot.is_some() { + return Err(ApiError::bad_request( + "stored mutation cannot target a snapshot", + )); + } + let branch = req.branch.unwrap_or_else(|| "main".to_string()); + let output = run_mutate( + state, + handle, + actor_ref, + &source, + Some(&query_name), + req.params.as_ref(), + branch, + ) + .await?; + Ok(Json(InvokeStoredQueryResponse::Change(output))) + } else { + let (selected, target, result) = run_query( + handle, + actor_ref, + &source, + Some(&query_name), + req.params.as_ref(), + req.branch, + req.snapshot, + true, + ) + .await?; + Ok(Json(InvokeStoredQueryResponse::Read(api::read_output( + selected, &target, result, + )))) + } +} + +#[utoipa::path( + get, + path = "/queries", + tag = "queries", + operation_id = "list_queries", + responses( + (status = 200, description = "Stored-query catalog (the mcp.expose subset, with typed params)", body = QueriesCatalogOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// List the graph's exposed stored queries as a typed tool catalog. +/// +/// Returns the `mcp.expose == true` subset of the `queries:` registry, each +/// with its MCP tool name, read/mutate flag, description/instruction, and +/// typed parameters — enough for a client to register them as tools without +/// fetching `.gq` source. Read-gated; the catalog is graph-wide (branch +/// independent — `read` is authorized against `main`). **Not** Cedar-filtered +/// per query yet, so it can list a query whose `invoke_query` the caller +/// lacks (a known gap until per-query authorization lands). +async fn server_list_queries( + Extension(handle): Extension>, + actor: Option>, +) -> std::result::Result, ApiError> { + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::Read, + branch: Some("main".to_string()), + target_branch: None, + }, + )?; + let queries = match handle.queries.as_ref() { + Some(registry) => registry + .iter() + .filter(|q| q.expose) + .map(api::query_catalog_entry) + .collect(), + None => Vec::new(), + }; + Ok(Json(QueriesCatalogOutput { queries })) +} + #[utoipa::path( get, path = "/schema", @@ -2088,18 +2509,26 @@ async fn server_schema_apply( .map_err(ApiError::from_workload_reject)?; let result = { let db = &handle.engine; + let registry = handle.queries.as_deref(); + let label = handle.key.graph_id.as_str().to_string(); // Engine-layer policy enforcement (MR-722): pass the resolved // actor through so apply_schema_as can call enforce() with the // authoritative identity. With a policy installed in AppState, // engine-side enforcement re-checks the same decision the // HTTP-layer authorize_request just made above. PR #3 collapses // the redundancy. - db.apply_schema_as( + db.apply_schema_as_with_catalog_check( &request.schema_source, omnigraph::db::SchemaApplyOptions { allow_data_loss: request.allow_data_loss, }, actor_id, + |catalog| { + if let Some(registry) = registry { + validate_registry_against_catalog(registry, catalog, &label)?; + } + Ok(()) + }, ) .await .map_err(ApiError::from_omni)? @@ -2658,12 +3087,133 @@ mod tests { use std::fs; use tempfile::tempdir; + /// `authorize` returns the allow/deny **decision** (`Authz`) and reserves + /// `Err` for operational failures, so the invoke handler can hide a denial + /// as 404 without also masking a 401/500. Pins each outcome. + #[test] + fn authorize_splits_decision_from_operational_error() { + use super::{Authz, PolicyAction, PolicyCompiler, PolicyConfig, PolicyRequest, ResolvedActor, authorize}; + use std::sync::Arc; + + fn req(action: PolicyAction) -> PolicyRequest { + PolicyRequest { action, branch: None, target_branch: None } + } + let actor = ResolvedActor::cluster_static(Arc::from("act-alice")); + + // --- No policy engine installed (open / default-deny modes) --- + // A server-scoped action is denied in every no-policy state. + assert!(matches!( + authorize(Some(&actor), None, req(PolicyAction::GraphList)).unwrap(), + Authz::Denied(_) + )); + // Authenticated actor + a non-read per-graph action → default-deny. + assert!(matches!( + authorize(Some(&actor), None, req(PolicyAction::Change)).unwrap(), + Authz::Denied(_) + )); + // `read` is the one per-graph action permitted without a policy. + assert!(matches!( + authorize(Some(&actor), None, req(PolicyAction::Read)).unwrap(), + Authz::Allowed + )); + // Open mode (no actor, no policy) → allowed. + assert!(matches!( + authorize(None, None, req(PolicyAction::Read)).unwrap(), + Authz::Allowed + )); + + // --- Policy engine installed --- + let policy: PolicyConfig = serde_yaml::from_str( + "version: 1\n\ + groups:\n team: [act-alice]\n\ + rules:\n - id: team-read\n allow:\n actors: { group: team }\n actions: [read]\n branch_scope: any\n", + ) + .unwrap(); + let engine = PolicyCompiler::compile(&policy, "graph").unwrap(); + + // A matched allow rule → Allowed. + assert!(matches!( + authorize( + Some(&actor), + Some(&engine), + PolicyRequest { action: PolicyAction::Read, branch: Some("main".to_string()), target_branch: None }, + ) + .unwrap(), + Authz::Allowed + )); + // Known actor, no matching allow rule → Denied, carrying the decision message. + match authorize( + Some(&actor), + Some(&engine), + PolicyRequest { action: PolicyAction::Change, branch: Some("main".to_string()), target_branch: None }, + ) + .unwrap() + { + Authz::Denied(message) => assert!(!message.is_empty(), "a deny carries its decision message"), + Authz::Allowed => panic!("change must be denied: only read is allowed"), + } + // Policy installed but no actor → operational failure (`Err`), NOT a + // decision. This is the split that keeps a 401/500 from being masked + // as the denial's response in the invoke handler. + assert!( + authorize(None, Some(&engine), req(PolicyAction::Read)).is_err(), + "a missing actor with a policy installed is an operational error, not a deny" + ); + } + #[test] fn hash_bearer_token_produces_32_byte_output() { let hash = hash_bearer_token("any-token"); assert_eq!(hash.len(), 32); } + /// The single gate both open paths funnel through: it refuses a + /// schema breakage (naming the graph label + query), attaches a clean + /// registry, and collapses an empty one to `None`. Pure over its args + /// (no engine), so it covers the multi-graph path's logic too — the + /// only per-path difference is the `label`, asserted here. + #[test] + fn validate_and_attach_gates_on_schema_and_collapses_empty() { + use crate::queries::{QueryRegistry, RegistrySpec}; + use omnigraph_compiler::catalog::build_catalog; + use omnigraph_compiler::schema::parser::parse_schema; + + let schema = parse_schema("node User {\nname: String\n}\n").unwrap(); + let catalog = build_catalog(&schema).unwrap(); + let spec = |name: &str, source: &str| RegistrySpec { + name: name.to_string(), + source: source.to_string(), + expose: false, + tool_name: None, + }; + + // Empty registry → nothing attached, no error. + let empty = + super::validate_and_attach(QueryRegistry::default(), &catalog, "g").unwrap(); + assert!(empty.is_none()); + + // A query that type-checks → attached. + let ok = QueryRegistry::from_specs(vec![spec( + "find_user", + "query find_user() { match { $u: User } return { $u.name } }", + )]) + .unwrap(); + assert!(super::validate_and_attach(ok, &catalog, "g").unwrap().is_some()); + + // A query referencing a type the schema lacks → boot refusal that + // names both the graph label and the offending query. + let broken = QueryRegistry::from_specs(vec![spec( + "ghost", + "query ghost() { match { $w: Widget } return { $w.name } }", + )]) + .unwrap(); + let err = super::validate_and_attach(broken, &catalog, "graph-x").unwrap_err(); + let msg = err.to_string(); + assert!(msg.contains("graph-x"), "labels the graph: {msg}"); + assert!(msg.contains("ghost"), "names the query: {msg}"); + assert!(msg.contains("schema check"), "mentions the schema check: {msg}"); + } + #[test] fn hash_bearer_token_is_deterministic() { assert_eq!( @@ -2707,7 +3257,10 @@ server: let settings = load_server_settings(Some(&config), None, None, None, false).unwrap(); match &settings.mode { - ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/demo.omni"), + ServerConfigMode::Single { uri, graph_id, .. } => { + assert_eq!(uri, "/tmp/demo.omni"); + assert_eq!(graph_id, "local"); + } ServerConfigMode::Multi { .. } => panic!("expected Single mode, got Multi"), } assert_eq!(settings.bind, "0.0.0.0:9090"); @@ -2739,7 +3292,10 @@ server: ) .unwrap(); match &settings.mode { - ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/override.omni"), + ServerConfigMode::Single { uri, graph_id, .. } => { + assert_eq!(uri, "/tmp/override.omni"); + assert_eq!(graph_id, "/tmp/override.omni"); + } ServerConfigMode::Multi { .. } => panic!("expected Single mode, got Multi"), } assert_eq!(settings.bind, "0.0.0.0:9999"); @@ -2768,7 +3324,10 @@ server: load_server_settings(Some(&config), None, Some("dev".to_string()), None, false) .unwrap(); match &settings.mode { - ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "http://127.0.0.1:8080"), + ServerConfigMode::Single { uri, graph_id, .. } => { + assert_eq!(uri, "http://127.0.0.1:8080"); + assert_eq!(graph_id, "dev"); + } ServerConfigMode::Multi { .. } => panic!("expected Single mode, got Multi"), } } @@ -2848,6 +3407,7 @@ server: .to_string_lossy() .into_owned(), policy_file: None, + queries: crate::queries::QueryRegistry::default(), }], config_path: temp.path().join("omnigraph.yaml"), server_policy_file: Some(policy_path), @@ -2895,7 +3455,9 @@ server: .join("graph.omni") .to_string_lossy() .into_owned(), + graph_id: "default".to_string(), policy_file: None, + queries: crate::queries::QueryRegistry::default(), }, bind: "127.0.0.1:0".to_string(), allow_unauthenticated: false, diff --git a/crates/omnigraph-server/src/queries.rs b/crates/omnigraph-server/src/queries.rs new file mode 100644 index 0000000..bf131c8 --- /dev/null +++ b/crates/omnigraph-server/src/queries.rs @@ -0,0 +1,688 @@ +//! Stored-query registry. +//! +//! A server-side registry of named, parameter-typed `.gq` queries that +//! operators declare in `omnigraph.yaml` (per-graph, or top-level in +//! single mode) and the server loads at startup. Each entry is parsed +//! and its identity asserted here (`load`); type-checking against the +//! live schema happens separately (a `check` pass) so the loader stays +//! callable without an open engine (the CLI's offline `queries check`). +//! +//! Identity is the query **name**: the manifest key must equal the +//! `query ` symbol declared in the referenced `.gq` file. The two +//! are asserted equal at load — one name, two places that must agree. +//! Renaming either is a breaking change to callers, by design. + +use std::collections::BTreeMap; +use std::fs; +use std::sync::Arc; + +use omnigraph_compiler::catalog::Catalog; +use omnigraph_compiler::query::ast::QueryDecl; +use omnigraph_compiler::query::parser::parse_query; +use omnigraph_compiler::query::typecheck::typecheck_query_decl; +use omnigraph_compiler::types::{PropType, ScalarType}; + +use crate::config::{OmnigraphConfig, QueryEntry}; + +/// One loaded stored query. `source` is the full `.gq` file text — the +/// invocation handler hands it to `run_query` / `run_mutate` verbatim, +/// which reuse the same parse/IR/exec path as the inline routes (no +/// parallel implementation). +#[derive(Debug, Clone)] +pub struct StoredQuery { + /// Identity: manifest key == `query ` symbol. + pub name: String, + /// Full `.gq` source text the query was selected from. + pub source: Arc, + /// Parsed declaration (params, mutations, description, …). + pub decl: QueryDecl, + /// Whether this query is listed in the MCP tool catalog (`GET /queries`). + /// Default `true` (the manifest entry is the opt-in); `expose: false` + /// keeps it HTTP/service-callable but hidden from the agent tool list. + /// Catalog membership only — not an authorization gate. + pub expose: bool, + /// Optional MCP tool-name override; defaults to `name`. + pub tool_name: Option, +} + +impl StoredQuery { + /// `true` if the selected declaration contains insert/update/delete + /// statements — drives read-vs-mutate routing at invocation time. + pub fn is_mutation(&self) -> bool { + !self.decl.mutations.is_empty() + } + + /// The MCP tool name this query is catalogued under: the explicit + /// `tool_name` override, else the query `name`. The catalog key — + /// enforced unique across exposed queries at load. Server-side + /// consumers (the uniqueness check, the future catalog projection) read + /// this; the CLI `queries list` resolves the same rule on its own DTO. + pub fn effective_tool_name(&self) -> &str { + self.tool_name.as_deref().unwrap_or(&self.name) + } +} + +/// A loaded, identity-checked stored-query registry for one graph. +#[derive(Debug, Clone, Default)] +pub struct QueryRegistry { + by_name: BTreeMap, +} + +/// In-memory registry entry before file I/O. Used by [`QueryRegistry::load`] +/// (after reading each `.gq` from disk) and directly by tests. +#[derive(Debug, Clone)] +pub struct RegistrySpec { + pub name: String, + pub source: String, + pub expose: bool, + pub tool_name: Option, +} + +/// A single registry load failure. Collected (not fail-fast) so a bad +/// `omnigraph.yaml` surfaces every broken entry at once, matching the +/// bad-policy-YAML posture. +#[derive(Debug, Clone)] +pub struct LoadError { + /// The offending query name, when the failure is entry-scoped. + pub query: Option, + pub message: String, +} + +impl std::fmt::Display for LoadError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match &self.query { + Some(name) => write!(f, "stored query '{name}': {}", self.message), + None => write!(f, "stored query registry: {}", self.message), + } + } +} + +impl QueryRegistry { + /// Build a registry from in-memory specs: parse each source, select + /// the declaration whose symbol equals the manifest key, and assert + /// they agree. Collects every failure. No schema type-checking here + /// — that is [`check`]. + pub fn from_specs(specs: Vec) -> Result> { + let mut by_name = BTreeMap::new(); + let mut errors = Vec::new(); + + for spec in specs { + match parse_query(&spec.source) { + Ok(file) => { + match file.queries.into_iter().find(|q| q.name == spec.name) { + Some(decl) => { + by_name.insert( + spec.name.clone(), + StoredQuery { + name: spec.name, + source: Arc::from(spec.source), + decl, + expose: spec.expose, + tool_name: spec.tool_name, + }, + ); + } + None => errors.push(LoadError { + query: Some(spec.name.clone()), + message: format!( + "no `query {}` declaration found in its `.gq` file \ + (the registry key must match the query symbol)", + spec.name + ), + }), + } + } + Err(err) => errors.push(LoadError { + query: Some(spec.name), + message: format!("parse error: {err}"), + }), + } + } + + // Exposed queries are catalogued under their effective tool name; + // two claiming one name is an MCP-namespace collision. Refuse it at + // load (collected, not fail-fast), naming the loser and the winner. + // Iterating the `BTreeMap` makes the winner deterministic (the + // lexicographically-first query name; config is a map, so YAML + // declaration order isn't preserved anyway) and the error order + // stable. Scoped to a block so these borrows of `by_name` end + // before it is moved into `Self`. + { + let mut claimed: BTreeMap<&str, &str> = BTreeMap::new(); + for query in by_name.values().filter(|q| q.expose) { + let tool = query.effective_tool_name(); + if let Some(winner) = claimed.insert(tool, &query.name) { + errors.push(LoadError { + query: Some(query.name.clone()), + message: format!( + "MCP tool name '{tool}' already claimed by exposed query '{winner}'" + ), + }); + } + } + } + + if errors.is_empty() { + Ok(Self { by_name }) + } else { + Err(errors) + } + } + + /// Read each registry entry's `.gq` file from disk and build the + /// registry. `entries` is either the top-level `queries` map (single + /// mode) or a graph's `queries` map (multi mode); `config` resolves + /// each entry's relative `file:` path against `base_dir`. + pub fn load( + config: &OmnigraphConfig, + entries: &BTreeMap, + ) -> Result> { + let mut specs = Vec::with_capacity(entries.len()); + let mut errors = Vec::new(); + for (name, entry) in entries { + let path = config.resolve_query_file(&entry.file); + match fs::read_to_string(&path) { + Ok(source) => specs.push(RegistrySpec { + name: name.clone(), + source, + expose: entry.mcp.expose, + tool_name: entry.mcp.tool_name.clone(), + }), + Err(err) => errors.push(LoadError { + query: Some(name.clone()), + message: format!("cannot read '{}': {err}", path.display()), + }), + } + } + + // Parse/identity/uniqueness-check the readable specs even when some + // files failed to read, so every broken entry (I/O, parse, identity, + // tool-name collision) surfaces in one pass rather than one per + // restart. I/O errors come first (in `entries` key order), then the + // spec errors. A non-empty `errors` always fails the load. + match Self::from_specs(specs) { + Ok(registry) if errors.is_empty() => Ok(registry), + Ok(_) => Err(errors), + Err(spec_errors) => { + errors.extend(spec_errors); + Err(errors) + } + } + } + + pub fn lookup(&self, name: &str) -> Option<&StoredQuery> { + self.by_name.get(name) + } + + pub fn iter(&self) -> impl Iterator { + self.by_name.values() + } + + pub fn is_empty(&self) -> bool { + self.by_name.is_empty() + } + + pub fn len(&self) -> usize { + self.by_name.len() + } +} + +/// A stored query that fails to type-check against the live schema — +/// e.g. it references a node/edge type or property that was renamed or +/// removed by a migration. Breakages **block server boot** (same posture +/// as bad policy YAML), surfacing schema drift at the deploy boundary +/// rather than silently at invocation time. +#[derive(Debug, Clone)] +pub struct Breakage { + pub query: String, + pub message: String, +} + +/// A non-blocking advisory found during validation. Logged at boot; +/// never blocks startup. Currently: an MCP-exposed query that declares a +/// parameter an agent cannot realistically supply. +#[derive(Debug, Clone)] +pub struct Warning { + pub query: String, + pub message: String, +} + +/// Outcome of validating a registry against a schema. Breakages are +/// fatal (boot refuses); warnings are advisory. +#[derive(Debug, Clone, Default)] +pub struct CheckReport { + pub breakages: Vec, + pub warnings: Vec, +} + +impl CheckReport { + pub fn has_breakages(&self) -> bool { + !self.breakages.is_empty() + } + + pub fn is_clean(&self) -> bool { + self.breakages.is_empty() && self.warnings.is_empty() + } +} + +/// Validate a loaded registry against the live schema. +/// +/// Pure over `(registry, catalog)` — takes an already-parsed registry and +/// a catalog, so it is callable both at server boot (with the engine's +/// `catalog()`) and offline from the CLI (`omnigraph queries check`), +/// without coupling to server config or an open engine connection. +/// +/// Every query is type-checked via the same `typecheck_query_decl` the +/// engine runs for inline queries — no parallel implementation. Failures +/// are **collected, not fail-fast**, so an operator sees every broken +/// query in one pass. +/// +/// Advisory lint (warn, never block): an `mcp.expose: true` query that +/// declares a `Vector(N)` parameter. An LLM cannot supply a raw embedding +/// vector; such a query should take a `String` parameter and let the +/// engine embed it server-side at query time. Service-to-service callers +/// may legitimately pass vectors, so this warns rather than rejects. +pub fn check(registry: &QueryRegistry, catalog: &Catalog) -> CheckReport { + let mut report = CheckReport::default(); + for query in registry.iter() { + if let Err(err) = typecheck_query_decl(catalog, &query.decl) { + report.breakages.push(Breakage { + query: query.name.clone(), + message: err.to_string(), + }); + } + if query.expose { + for param in &query.decl.params { + // Resolve to the structured type via the compiler's own + // resolver rather than string-matching `Vector(` — one + // canonical definition of "is a vector", so this lint can't + // drift from how the parser/type system spells the type. + let is_vector = PropType::from_param_type_name(¶m.type_name, param.nullable) + .is_some_and(|pt| matches!(pt.scalar, ScalarType::Vector(_))); + if is_vector { + report.warnings.push(Warning { + query: query.name.clone(), + message: format!( + "MCP-exposed query declares a `{}` parameter `${}` that agents \ + cannot supply; use a `String` parameter for server-side embedding", + param.type_name, param.name + ), + }); + } + } + } + } + report +} + +/// Format every breakage in a registry check report into a multi-line +/// operator-facing message, naming each offending query. +pub fn format_check_breakages(label: &str, report: &CheckReport) -> String { + let joined = report + .breakages + .iter() + .map(|b| format!("query '{}': {}", b.query, b.message)) + .collect::>() + .join("\n "); + format!( + "graph '{label}': {} stored quer{} failed the schema check:\n {joined}", + report.breakages.len(), + if report.breakages.len() == 1 { + "y" + } else { + "ies" + } + ) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn spec(name: &str, source: &str, expose: bool) -> RegistrySpec { + RegistrySpec { + name: name.to_string(), + source: source.to_string(), + expose, + tool_name: None, + } + } + + fn spec_tool(name: &str, source: &str, expose: bool, tool_name: &str) -> RegistrySpec { + RegistrySpec { + name: name.to_string(), + source: source.to_string(), + expose, + tool_name: Some(tool_name.to_string()), + } + } + + #[test] + fn key_equal_symbol_loads() { + let reg = QueryRegistry::from_specs(vec![spec( + "find_user", + "query find_user($id: String) { match { $u: User } return { $u.name } }", + true, + )]) + .unwrap(); + let q = reg.lookup("find_user").unwrap(); + assert_eq!(q.name, "find_user"); + assert!(q.expose); + assert_eq!(q.decl.params.len(), 1); + assert!(!q.is_mutation()); + // No override → the effective tool name is the query name. + assert_eq!(q.effective_tool_name(), "find_user"); + + // An explicit override is what the catalog keys on. + let with_tool = QueryRegistry::from_specs(vec![spec_tool( + "find_user", + "query find_user($id: String) { match { $u: User } return { $u.name } }", + true, + "lookup_user", + )]) + .unwrap(); + assert_eq!( + with_tool.lookup("find_user").unwrap().effective_tool_name(), + "lookup_user" + ); + } + + #[test] + fn key_mismatch_is_an_identity_error() { + let errors = QueryRegistry::from_specs(vec![spec( + "find_user", + // symbol is `lookup`, key is `find_user` — must be rejected. + "query lookup($id: String) { match { $u: User } return { $u.name } }", + false, + )]) + .unwrap_err(); + assert_eq!(errors.len(), 1); + assert_eq!(errors[0].query.as_deref(), Some("find_user")); + assert!(errors[0].message.contains("must match the query symbol")); + } + + #[test] + fn multi_query_file_selects_the_matching_symbol() { + let source = "query a($x: I64) { match { $u: User } return { $u.name } }\n\ + query b($y: String) { match { $u: User } return { $u.name } }"; + let reg = QueryRegistry::from_specs(vec![spec("b", source, false)]).unwrap(); + let q = reg.lookup("b").unwrap(); + assert_eq!(q.name, "b"); + assert_eq!(q.decl.params[0].name, "y"); + assert!(reg.lookup("a").is_none(), "only the selected symbol is registered"); + } + + #[test] + fn duplicate_exposed_tool_name_is_a_load_error() { + // Two MCP-exposed queries claiming one tool name is an ambiguity in + // the catalog key space — refused at load, naming both queries and + // the contested tool. + let errors = QueryRegistry::from_specs(vec![ + spec_tool("a", "query a() { match { $u: User } return { $u.name } }", true, "dup"), + spec_tool("b", "query b() { match { $u: User } return { $u.name } }", true, "dup"), + ]) + .unwrap_err(); + assert_eq!(errors.len(), 1); + let msg = errors[0].to_string(); + assert!(msg.contains("'dup'"), "names the contested tool: {msg}"); + assert!(msg.contains("'a'"), "names the winning query: {msg}"); + assert!(msg.contains("'b'"), "names the losing query: {msg}"); + } + + #[test] + fn duplicate_tool_name_among_unexposed_is_allowed() { + // Unexposed queries have no MCP tool, so a shared effective tool + // name is inert — must not error (pins the exposed-only scope). + let reg = QueryRegistry::from_specs(vec![ + spec_tool("a", "query a() { match { $u: User } return { $u.name } }", false, "dup"), + spec_tool("b", "query b() { match { $u: User } return { $u.name } }", false, "dup"), + ]) + .unwrap(); + assert_eq!(reg.len(), 2); + } + + #[test] + fn parse_error_surfaces_per_entry() { + let errors = + QueryRegistry::from_specs(vec![spec("broken", "query broken( {{ not valid", false)]) + .unwrap_err(); + assert_eq!(errors[0].query.as_deref(), Some("broken")); + assert!(errors[0].message.contains("parse error")); + } + + #[test] + fn errors_collect_rather_than_fail_fast() { + let errors = QueryRegistry::from_specs(vec![ + spec("good", "query good() { match { $u: User } return { $u.name } }", false), + spec("mismatch", "query other() { match { $u: User } return { $u.name } }", false), + spec("broken", "query broken(", false), + ]) + .unwrap_err(); + // `good` loads cleanly; only the mismatch and the parse error are + // reported, and both surface in one pass (not fail-fast). + assert_eq!(errors.len(), 2); + } + + #[test] + fn mutation_body_classifies_as_mutation() { + let reg = QueryRegistry::from_specs(vec![spec( + "add_user", + "query add_user($name: String) { insert User { name: $name } }", + false, + )]) + .unwrap(); + assert!(reg.lookup("add_user").unwrap().is_mutation()); + } + + // --- check(registry, catalog) --- + + use omnigraph_compiler::catalog::build_catalog; + use omnigraph_compiler::schema::parser::parse_schema; + + fn test_catalog() -> Catalog { + let schema = parse_schema( + r#" +node User { +name: String +age: I32? +embedding: Vector(4) +} +"#, + ) + .unwrap(); + build_catalog(&schema).unwrap() + } + + #[test] + fn check_passes_for_valid_query() { + let reg = QueryRegistry::from_specs(vec![spec( + "find_user", + "query find_user($name: String) { match { $u: User { name: $name } } return { $u.age } }", + false, + )]) + .unwrap(); + let report = check(®, &test_catalog()); + assert!(report.is_clean(), "unexpected: {:?}", report); + } + + #[test] + fn check_reports_unknown_type_as_breakage() { + let reg = QueryRegistry::from_specs(vec![spec( + "ghost", + // `Widget` is not in the schema. + "query ghost() { match { $w: Widget } return { $w.name } }", + false, + )]) + .unwrap(); + let report = check(®, &test_catalog()); + assert!(report.has_breakages()); + assert_eq!(report.breakages[0].query, "ghost"); + } + + #[test] + fn check_reports_unknown_property_as_breakage() { + let reg = QueryRegistry::from_specs(vec![spec( + "bad_prop", + // `User` exists but has no `nickname`. + "query bad_prop() { match { $u: User } return { $u.nickname } }", + false, + )]) + .unwrap(); + let report = check(®, &test_catalog()); + assert!(report.has_breakages()); + assert_eq!(report.breakages[0].query, "bad_prop"); + } + + #[test] + fn check_collects_every_breakage_not_fail_fast() { + let reg = QueryRegistry::from_specs(vec![ + spec("a", "query a() { match { $w: Widget } return { $w.x } }", false), + spec("b", "query b() { match { $g: Gadget } return { $g.y } }", false), + spec( + "ok", + "query ok() { match { $u: User } return { $u.name } }", + false, + ), + ]) + .unwrap(); + let report = check(®, &test_catalog()); + assert_eq!(report.breakages.len(), 2, "both bad queries reported: {:?}", report); + } + + #[test] + fn vector_param_on_exposed_query_warns() { + let reg = QueryRegistry::from_specs(vec![spec( + "vec_search", + "query vec_search($q: Vector(4)) { match { $u: User } return { $u.name } \ + order { nearest($u.embedding, $q) } limit 3 }", + true, // mcp.expose + )]) + .unwrap(); + let report = check(®, &test_catalog()); + assert!(!report.has_breakages(), "valid query: {:?}", report); + assert_eq!(report.warnings.len(), 1); + assert_eq!(report.warnings[0].query, "vec_search"); + } + + #[test] + fn vector_param_on_unexposed_query_is_silent() { + let reg = QueryRegistry::from_specs(vec![spec( + "vec_search", + "query vec_search($q: Vector(4)) { match { $u: User } return { $u.name } \ + order { nearest($u.embedding, $q) } limit 3 }", + false, // not exposed — vector param is fine for service-to-service callers + )]) + .unwrap(); + let report = check(®, &test_catalog()); + assert!(report.is_clean(), "unexpected: {:?}", report); + } + + #[test] + fn non_vector_param_on_exposed_query_does_not_warn() { + // The recommended `String` alternative on an exposed query does not + // resolve to a Vector, so the embedding advisory stays silent. Guards + // the structured type check against a false positive (and pins that + // only `Vector(_)` triggers the warning). + let reg = QueryRegistry::from_specs(vec![spec( + "search", + "query search($name: String) { match { $u: User { name: $name } } return { $u.name } }", + true, + )]) + .unwrap(); + let report = check(®, &test_catalog()); + assert!(report.is_clean(), "no breakage or warning expected: {:?}", report); + } + + // --- catalog projection (api::query_catalog_entry) --- + + #[test] + fn catalog_entry_projects_every_param_kind() { + use crate::api::{self, ParamKind}; + let reg = QueryRegistry::from_specs(vec![spec_tool( + "all_types", + "query all_types($s: String, $i: I32, $big: I64, $u: U64, $f: F64, $b: Bool, \ + $d: Date, $dt: DateTime, $blob: Blob, $opt: String?, $list: [I32], $vec: Vector(4)) \ + { match { $x: User } return { $x.name } }", + true, + "all", + )]) + .unwrap(); + let entry = api::query_catalog_entry(reg.lookup("all_types").unwrap()); + assert_eq!(entry.name, "all_types"); + assert_eq!(entry.tool_name, "all"); + assert!(!entry.mutation); + + let by: std::collections::HashMap<_, _> = + entry.params.iter().map(|p| (p.name.as_str(), p)).collect(); + assert_eq!(by["s"].kind, ParamKind::String); + assert_eq!(by["i"].kind, ParamKind::Int); + assert_eq!(by["big"].kind, ParamKind::BigInt, "I64 → bigint (string on the wire)"); + assert_eq!(by["u"].kind, ParamKind::BigInt, "U64 → bigint"); + assert_eq!(by["f"].kind, ParamKind::Float); + assert_eq!(by["b"].kind, ParamKind::Bool); + assert_eq!(by["d"].kind, ParamKind::Date); + assert_eq!(by["dt"].kind, ParamKind::DateTime); + assert_eq!(by["blob"].kind, ParamKind::Blob); + assert!(!by["s"].nullable); + assert!(by["opt"].nullable, "String? → nullable"); + assert_eq!(by["list"].kind, ParamKind::List); + assert_eq!(by["list"].item_kind, Some(ParamKind::Int), "[I32] → list of int"); + assert_eq!(by["vec"].kind, ParamKind::Vector); + assert_eq!(by["vec"].vector_dim, Some(4)); + } + + #[test] + fn catalog_entry_flags_mutation_and_empty_params() { + use crate::api; + let reg = QueryRegistry::from_specs(vec![spec( + "add_user", + "query add_user($name: String) { insert User { name: $name } }", + true, + )]) + .unwrap(); + let entry = api::query_catalog_entry(reg.lookup("add_user").unwrap()); + assert!(entry.mutation, "insert body → mutation flag"); + + let reg2 = QueryRegistry::from_specs(vec![spec( + "no_params", + "query no_params() { match { $u: User } return { $u.name } }", + true, + )]) + .unwrap(); + let entry2 = api::query_catalog_entry(reg2.lookup("no_params").unwrap()); + assert!(entry2.params.is_empty(), "no declared params → empty list"); + } + + // --- load() error collection (file I/O + parse in one pass) --- + + #[test] + fn load_collects_io_and_parse_errors_in_one_pass() { + use crate::config::load_config; + let temp = tempfile::tempdir().unwrap(); + std::fs::write( + temp.path().join("good.gq"), + "query good() { match { $u: User } return { $u.name } }", + ) + .unwrap(); + std::fs::write(temp.path().join("broken.gq"), "query broken( {{ not valid").unwrap(); + // `missing.gq` is deliberately not written (an I/O failure). + std::fs::write( + temp.path().join("omnigraph.yaml"), + "queries:\n good:\n file: ./good.gq\n \ + missing:\n file: ./missing.gq\n broken:\n file: ./broken.gq\n", + ) + .unwrap(); + let config = load_config(Some(&temp.path().join("omnigraph.yaml"))).unwrap(); + + let errors = QueryRegistry::load(&config, config.query_entries()).unwrap_err(); + let joined = errors.iter().map(|e| e.to_string()).collect::>().join("\n"); + // Both the missing file AND the parse error surface in one pass — + // the I/O failure must not mask the parse failure. + assert!(joined.contains("missing"), "I/O error must surface: {joined}"); + assert!( + joined.contains("broken") && joined.contains("parse error"), + "the parse error in a readable file must surface in the same pass: {joined}" + ); + assert!(!joined.contains("'good'"), "the valid entry is not an error: {joined}"); + } +} diff --git a/crates/omnigraph-server/src/registry.rs b/crates/omnigraph-server/src/registry.rs index 5897ad1..54115e4 100644 --- a/crates/omnigraph-server/src/registry.rs +++ b/crates/omnigraph-server/src/registry.rs @@ -29,6 +29,7 @@ use tokio::sync::Mutex; use crate::identity::GraphKey; use crate::policy::PolicyEngine; +use crate::queries::QueryRegistry; /// Open handle for a single graph in the registry. Cheap to clone (`Arc`-wrapped /// engine + policy). Cluster-mode handlers extract this via @@ -47,6 +48,11 @@ pub struct GraphHandle { /// `_as` writers"; the HTTP-layer `require_bearer_auth` middleware still /// runs regardless. pub policy: Option>, + /// Per-graph stored-query registry, loaded and validated at + /// startup. `None` means the operator declared no stored queries for + /// this graph — `POST /queries/{name}` then 404s. Mirrors the + /// optional `policy` shape. + pub queries: Option>, } /// Immutable snapshot of the registry's current state. Replaced atomically @@ -245,6 +251,7 @@ fn canonicalize_handle_uri( uri: canonical_uri.clone(), engine: Arc::clone(&handle.engine), policy: handle.policy.clone(), + queries: handle.queries.clone(), }); Ok((canonical_uri, canonical_handle)) } @@ -276,6 +283,7 @@ mod tests { uri: graph_uri, engine: Arc::new(engine), policy: None, + queries: None, }) } @@ -340,12 +348,14 @@ mod tests { uri: shared_uri.clone(), engine: Arc::clone(&engine), policy: None, + queries: None, }); let h2 = Arc::new(GraphHandle { key: GraphKey::cluster(GraphId::try_from("beta").unwrap()), uri: shared_uri, engine, policy: None, + queries: None, }); let registry = GraphRegistry::new(); @@ -411,12 +421,14 @@ mod tests { uri: shared_uri.clone(), engine: Arc::clone(&engine), policy: None, + queries: None, }); let h2 = Arc::new(GraphHandle { key: GraphKey::cluster(GraphId::try_from("beta").unwrap()), uri: shared_uri, engine, policy: None, + queries: None, }); let err = match GraphRegistry::from_handles(vec![h1, h2]) { Ok(_) => panic!("expected DuplicateUri, got Ok"), diff --git a/crates/omnigraph-server/tests/openapi.rs b/crates/omnigraph-server/tests/openapi.rs index a2542db..3d13e74 100644 --- a/crates/omnigraph-server/tests/openapi.rs +++ b/crates/omnigraph-server/tests/openapi.rs @@ -168,6 +168,8 @@ const EXPECTED_PATHS: &[&str] = &[ "/export", "/change", "/mutate", + "/queries", + "/queries/{name}", "/schema", "/schema/apply", "/ingest", @@ -701,6 +703,8 @@ fn protected_endpoints_reference_bearer_token_security() { ("/read", "post"), ("/change", "post"), ("/schema/apply", "post"), + ("/queries", "get"), + ("/queries/{name}", "post"), ("/ingest", "post"), ("/export", "post"), ("/snapshot", "get"), @@ -913,6 +917,34 @@ fn post_endpoints_have_request_body() { } } +#[test] +fn invoke_stored_query_request_body_is_optional() { + let doc = openapi_json(); + let request_body = &doc["paths"]["/queries/{name}"]["post"]["requestBody"]; + assert!( + request_body.is_object(), + "POST /queries/{{name}} should document its optional request body" + ); + assert_eq!( + request_body["required"].as_bool().unwrap_or(false), + false, + "stored-query invocation body should be optional" + ); + let schema = &request_body["content"]["application/json"]["schema"]; + let ref_path = schema["$ref"] + .as_str() + .or_else(|| { + schema["oneOf"] + .as_array() + .and_then(|schemas| schemas.iter().find_map(|schema| schema["$ref"].as_str())) + }) + .unwrap(); + assert!( + ref_path.contains("InvokeStoredQueryRequest"), + "POST /queries/{{name}} requestBody should reference InvokeStoredQueryRequest, got {ref_path}" + ); +} + // --------------------------------------------------------------------------- // Serialization round-trip test // --------------------------------------------------------------------------- @@ -1117,6 +1149,7 @@ async fn app_for_multi_mode(graph_ids: &[&str]) -> (Vec, Rout uri: graph_uri, engine: Arc::new(engine), policy: None, + queries: None, })); dirs.push(dir); } diff --git a/crates/omnigraph-server/tests/server.rs b/crates/omnigraph-server/tests/server.rs index 3ace80e..4a49a14 100644 --- a/crates/omnigraph-server/tests/server.rs +++ b/crates/omnigraph-server/tests/server.rs @@ -8,7 +8,7 @@ use axum::body::{Body, to_bytes}; use axum::http::header::AUTHORIZATION; use axum::http::{Method, Request, StatusCode}; use lance::index::DatasetIndexExt; -use omnigraph::db::{Omnigraph, ReadTarget, SchemaApplyOptions}; +use omnigraph::db::{Omnigraph, ReadTarget}; use omnigraph::error::OmniError; use omnigraph::loader::{LoadMode, load_jsonl}; use omnigraph_policy::{PolicyChecker, PolicyEngine}; @@ -16,6 +16,7 @@ use omnigraph_server::api::{ BranchCreateRequest, BranchMergeRequest, ChangeRequest, ErrorOutput, ExportRequest, IngestRequest, QueryRequest, ReadRequest, SchemaApplyRequest, SchemaOutput, }; +use omnigraph_server::queries::{QueryRegistry, RegistrySpec}; use omnigraph_server::{AppState, build_app}; use serde_json::{Value, json}; use serial_test::serial; @@ -141,6 +142,469 @@ fn graph_path(root: &Path) -> PathBuf { root.join("server.omni") } +fn stored_query_registry(specs: &[(&str, &str, bool)]) -> QueryRegistry { + QueryRegistry::from_specs( + specs + .iter() + .map(|(name, source, expose)| RegistrySpec { + name: name.to_string(), + source: source.to_string(), + expose: *expose, + tool_name: None, + }) + .collect(), + ) + .expect("specs parse and key==symbol") +} + +#[tokio::test] +async fn server_boots_with_a_valid_stored_query_registry() { + // A stored query that type-checks against the fixture schema + // (`Person { name, age }`) must let the server boot. + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let registry = stored_query_registry(&[( + "find_person", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + false, + )]); + let state = AppState::open_single_with_queries( + graph.to_string_lossy().to_string(), + vec![], + None, + registry, + ) + .await; + assert!(state.is_ok(), "valid registry should boot: {:?}", state.err()); +} + +#[tokio::test] +async fn server_refuses_boot_on_type_broken_stored_query() { + // A stored query referencing a type not in the schema (`Widget`) + // must abort boot, naming the offending query. + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let registry = stored_query_registry(&[( + "ghost", + "query ghost() { match { $w: Widget } return { $w.name } }", + false, + )]); + let result = AppState::open_single_with_queries( + graph.to_string_lossy().to_string(), + vec![], + None, + registry, + ) + .await; + // `AppState` is not `Debug`, so match rather than `expect_err`. + let err = match result { + Ok(_) => panic!("type-broken stored query must refuse boot"), + Err(err) => err, + }; + let msg = err.to_string(); + assert!(msg.contains("ghost"), "error should name the broken query: {msg}"); + assert!( + msg.contains("schema check"), + "error should mention the schema check: {msg}" + ); +} + +/// Build a single-mode app with a stored-query registry plus a bearer→actor +/// pairing and a policy, so invoke tests exercise the `invoke_query` +/// boundary gate and the inner read/change gates together. +async fn app_with_stored_queries( + specs: &[(&str, &str, bool)], + tokens: &[(&str, &str)], + policy: &str, +) -> (tempfile::TempDir, Router) { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, policy).unwrap(); + let registry = stored_query_registry(specs); + let state = AppState::open_single_with_queries( + graph.to_string_lossy().to_string(), + tokens + .iter() + .map(|(actor, token)| ((*actor).to_string(), (*token).to_string())) + .collect(), + Some(&policy_path), + registry, + ) + .await + .unwrap(); + (temp, build_app(state)) +} + +/// - `act-invoke`: invoke_query + read (stored reads, not mutations) +/// - `act-full`: invoke_query + read + change (stored mutations) +/// - `act-noinvoke`: read only, no invoke_query (boundary-denied) +/// - `act-invokeonly`: invoke_query only, no read (clears the boundary, inner read denies) +const INVOKE_POLICY_YAML: &str = r#" +version: 1 +groups: + invokers: ["act-invoke"] + full: ["act-full"] + readers: ["act-noinvoke"] + invoke_only: ["act-invokeonly"] +protected_branches: [main] +rules: + # invoke_query is graph-scoped — its own rules, no branch_scope. + - id: invokers-can-invoke + allow: + actors: { group: invokers } + actions: [invoke_query] + - id: full-can-invoke + allow: + actors: { group: full } + actions: [invoke_query] + - id: invoke-only-can-invoke + allow: + actors: { group: invoke_only } + actions: [invoke_query] + # read / change are branch-scoped. + - id: invokers-can-read + allow: + actors: { group: invokers } + actions: [read] + branch_scope: any + - id: full-can-read-change + allow: + actors: { group: full } + actions: [read, change] + branch_scope: any + - id: readers-can-read + allow: + actors: { group: readers } + actions: [read] + branch_scope: any +"#; + +const STORED_QUERY_SCHEMA_APPLY_POLICY_YAML: &str = r#" +version: 1 +groups: + admins: [act-ragnor] +protected_branches: [main] +rules: + - id: admins-can-invoke + allow: + actors: { group: admins } + actions: [invoke_query] + - id: admins-can-read + allow: + actors: { group: admins } + actions: [read] + branch_scope: any + - id: admins-can-schema-apply + allow: + actors: { group: admins } + actions: [schema_apply] + target_branch_scope: protected +"#; + +const FIND_PERSON_GQ: &str = + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }"; + +fn invoke_request(name: &str, token: &str, body: Value) -> Request { + Request::builder() + .uri(format!("/queries/{name}")) + .method(Method::POST) + .header("content-type", "application/json") + .header("authorization", format!("Bearer {token}")) + .body(Body::from(serde_json::to_vec(&body).unwrap())) + .unwrap() +} + +fn invoke_request_bytes( + name: &str, + token: &str, + body: impl Into, + content_type: Option<&str>, +) -> Request { + let mut builder = Request::builder() + .uri(format!("/queries/{name}")) + .method(Method::POST) + .header("authorization", format!("Bearer {token}")); + if let Some(content_type) = content_type { + builder = builder.header("content-type", content_type); + } + builder.body(body.into()).unwrap() +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_stored_read_returns_rows() { + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, false)], + &[("act-invoke", "t-invoke")], + INVOKE_POLICY_YAML, + ) + .await; + let (status, body) = json_response( + &app, + invoke_request("find_person", "t-invoke", json!({ "params": { "name": "Alice" } })), + ) + .await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + assert_eq!(body["query_name"], "find_person"); + assert_eq!(body["row_count"], 1, "Alice is in the fixture; body: {body}"); + assert!(body["rows"].is_array(), "read envelope shape; body: {body}"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_stored_read_accepts_absent_or_empty_body() { + let no_param_query = "query list_people() { match { $p: Person } return { $p.name } }"; + let (_temp, app) = app_with_stored_queries( + &[("list_people", no_param_query, false)], + &[("act-invoke", "t-invoke")], + INVOKE_POLICY_YAML, + ) + .await; + + let (status, body) = json_response( + &app, + invoke_request_bytes("list_people", "t-invoke", Body::empty(), None), + ) + .await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + assert_eq!(body["query_name"], "list_people"); + + let (status, body) = json_response( + &app, + invoke_request_bytes( + "list_people", + "t-invoke", + Body::empty(), + Some("application/json"), + ), + ) + .await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + + let (status, body) = json_response( + &app, + invoke_request_bytes( + "list_people", + "t-invoke", + Body::from("{}"), + Some("application/json"), + ), + ) + .await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + + let (status, body) = json_response( + &app, + invoke_request_bytes( + "list_people", + "t-invoke", + Body::from("{"), + Some("application/json"), + ), + ) + .await; + assert_eq!(status, StatusCode::BAD_REQUEST, "body: {body}"); + assert!( + body["error"] + .as_str() + .unwrap_or_default() + .contains("invalid stored-query invocation body"), + "malformed JSON should be rejected as bad request; body: {body}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_stored_mutation_double_gates_on_change() { + let specs: &[(&str, &str, bool)] = &[( + "add_person", + "query add_person($name: String) { insert Person { name: $name } }", + false, + )]; + let (_temp, app) = app_with_stored_queries( + specs, + &[("act-invoke", "t-invoke"), ("act-full", "t-full")], + INVOKE_POLICY_YAML, + ) + .await; + + // Has invoke_query but NOT change → the inner change gate denies (403). + let (status, body) = json_response( + &app, + invoke_request("add_person", "t-invoke", json!({ "params": { "name": "Eve" } })), + ) + .await; + assert_eq!( + status, + StatusCode::FORBIDDEN, + "invoke_query without change must 403; body: {body}" + ); + + // Has invoke_query + change → applied. + let (status, body) = json_response( + &app, + invoke_request("add_person", "t-full", json!({ "params": { "name": "Eve" } })), + ) + .await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + assert_eq!(body["affected_nodes"], 1, "body: {body}"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_stored_query_bad_param_is_400() { + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, false)], + &[("act-invoke", "t-invoke")], + INVOKE_POLICY_YAML, + ) + .await; + // `name` is declared String; pass a number. + let (status, body) = json_response( + &app, + invoke_request("find_person", "t-invoke", json!({ "params": { "name": 123 } })), + ) + .await; + assert_eq!(status, StatusCode::BAD_REQUEST, "body: {body}"); + assert!( + body["error"].as_str().unwrap_or_default().contains("name"), + "400 should name the offending param; body: {body}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_unknown_query_and_denied_actor_return_identical_404() { + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, false)], + &[("act-invoke", "t-invoke"), ("act-noinvoke", "t-noinvoke")], + INVOKE_POLICY_YAML, + ) + .await; + + // Authorized actor, unknown query name → 404. + let (unknown_status, unknown_body) = + json_response(&app, invoke_request("does_not_exist", "t-invoke", json!({}))).await; + // Denied actor (no invoke_query), real query name → 404. + let (denied_status, denied_body) = json_response( + &app, + invoke_request("find_person", "t-noinvoke", json!({ "params": { "name": "Alice" } })), + ) + .await; + + assert_eq!(unknown_status, StatusCode::NOT_FOUND); + assert_eq!(denied_status, StatusCode::NOT_FOUND); + assert_eq!( + unknown_body, denied_body, + "deny must be byte-identical to a missing query (no catalog probing)" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_query_holder_without_read_sees_403_not_404() { + // The 404-hiding is for callers WITHOUT invoke_query. An actor that + // HOLDS invoke_query but lacks `read` clears the boundary gate, then the + // inner read gate denies → 403 for an EXISTING read query, vs 404 for an + // unknown one. Existence is visible to grant-holders by design (the + // documented double-gate); this pins that actual contract. + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, false)], + &[("act-invokeonly", "t-invokeonly")], + INVOKE_POLICY_YAML, + ) + .await; + let (exists_status, _) = json_response( + &app, + invoke_request("find_person", "t-invokeonly", json!({ "params": { "name": "Alice" } })), + ) + .await; + let (absent_status, _) = + json_response(&app, invoke_request("does_not_exist", "t-invokeonly", json!({}))).await; + assert_eq!( + exists_status, + StatusCode::FORBIDDEN, + "an existing read query the holder can't read → inner-gate 403" + ); + assert_eq!(absent_status, StatusCode::NOT_FOUND, "unknown query still 404s"); +} + +fn get_request(uri: &str, token: &str) -> Request { + Request::builder() + .uri(uri) + .method(Method::GET) + .header("authorization", format!("Bearer {token}")) + .body(Body::empty()) + .unwrap() +} + +#[tokio::test(flavor = "multi_thread")] +async fn list_queries_returns_only_exposed_with_typed_params() { + let (_temp, app) = app_with_stored_queries( + &[ + ("find_person", FIND_PERSON_GQ, true), + ( + "add_person", + "query add_person($name: String) { insert Person { name: $name } }", + true, + ), + ("hidden", "query hidden() { match { $p: Person } return { $p.name } }", false), + ], + &[("act-invoke", "t-invoke")], + INVOKE_POLICY_YAML, + ) + .await; + let (status, body) = json_response(&app, get_request("/queries", "t-invoke")).await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + + let entries = body["queries"].as_array().unwrap(); + let names: Vec<&str> = entries.iter().map(|q| q["name"].as_str().unwrap()).collect(); + assert!( + names.contains(&"find_person") && names.contains(&"add_person"), + "exposed queries listed: {names:?}" + ); + assert!(!names.contains(&"hidden"), "non-exposed query hidden from the catalog: {names:?}"); + + let fp = entries.iter().find(|q| q["name"] == "find_person").unwrap(); + assert_eq!(fp["mutation"], false); + assert_eq!(fp["tool_name"], "find_person"); + assert_eq!(fp["params"][0]["name"], "name"); + assert_eq!(fp["params"][0]["kind"], "string"); + let ap = entries.iter().find(|q| q["name"] == "add_person").unwrap(); + assert_eq!(ap["mutation"], true, "stored insert → mutation"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn list_queries_is_read_gated_so_a_non_invoker_can_list() { + // The catalog is read-gated (not invoke_query-gated), so a reader who + // lacks invoke_query still enumerates the exposed queries — the + // documented probe-oracle gap until per-query Cedar filtering lands. + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, true)], + &[("act-noinvoke", "t-noinvoke")], + INVOKE_POLICY_YAML, + ) + .await; + let (status, body) = json_response(&app, get_request("/queries", "t-noinvoke")).await; + assert_eq!(status, StatusCode::OK, "read-gated catalog; body: {body}"); + let names: Vec<&str> = body["queries"] + .as_array() + .unwrap() + .iter() + .map(|q| q["name"].as_str().unwrap()) + .collect(); + assert!( + names.contains(&"find_person"), + "a reader lists the catalog despite lacking invoke_query: {names:?}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn list_queries_is_empty_when_no_registry() { + let (_temp, app) = app_for_loaded_graph_with_auth("demo-token").await; + let (status, body) = json_response(&app, get_request("/queries", "demo-token")).await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + assert!( + body["queries"].as_array().unwrap().is_empty(), + "no stored-query registry → empty catalog" + ); +} + fn drifted_test_schema() -> String { fs::read_to_string(fixture("test.pg")) .unwrap() @@ -423,6 +887,83 @@ async fn schema_apply_route_updates_graph_for_authorized_admin() { ); } +#[tokio::test(flavor = "multi_thread")] +async fn schema_apply_route_rejects_stored_query_breakage_before_publish() { + let (temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, true)], + &[("act-ragnor", "admin-token")], + STORED_QUERY_SCHEMA_APPLY_POLICY_YAML, + ) + .await; + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: renamed_age_schema(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + assert_eq!(status, StatusCode::BAD_REQUEST, "body: {payload}"); + let message = payload["error"].as_str().unwrap_or_default(); + assert!( + message.contains("find_person") && message.contains("schema check"), + "registry breakage should name the stored query; body: {payload}" + ); + + let reopened = Omnigraph::open(graph_path(temp.path()).to_str().unwrap()) + .await + .unwrap(); + let person = &reopened.catalog().node_types["Person"]; + assert!(person.properties.contains_key("age")); + assert!(!person.properties.contains_key("years")); + + let (invoke_status, invoke_body) = json_response( + &app, + invoke_request( + "find_person", + "admin-token", + json!({ "params": { "name": "Alice" } }), + ), + ) + .await; + assert_eq!(invoke_status, StatusCode::OK, "body: {invoke_body}"); + assert_eq!(invoke_body["row_count"], 1); +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_apply_route_noop_keeps_valid_stored_query_registry() { + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, true)], + &[("act-ragnor", "admin-token")], + STORED_QUERY_SCHEMA_APPLY_POLICY_YAML, + ) + .await; + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: fs::read_to_string(fixture("test.pg")).unwrap(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + assert_eq!(status, StatusCode::OK, "body: {payload}"); + assert_eq!(payload["applied"], false); +} + #[tokio::test] async fn schema_apply_route_requires_schema_apply_policy_permission() { let (_temp, app) = app_for_graph_with_auth_tokens_and_policy( @@ -4690,6 +5231,7 @@ mod multi_graph_startup { uri: graph_uri, engine: Arc::new(engine), policy: None, + queries: None, })); dirs.push(dir); } @@ -4985,12 +5527,14 @@ graphs: uri: graph_uri.clone(), engine: Arc::clone(&engine), policy: None, + queries: None, }); let beta = Arc::new(GraphHandle { key: GraphKey::cluster(GraphId::try_from("beta").unwrap()), uri: format!("file://{graph_uri}/"), engine, policy: None, + queries: None, }); match GraphRegistry::from_handles(vec![alpha, beta]) { @@ -5016,6 +5560,7 @@ graphs: uri: format!("file://{graph_uri}/"), engine: Arc::new(engine), policy: None, + queries: None, }); let registry = GraphRegistry::from_handles(vec![handle]).unwrap(); @@ -5138,11 +5683,11 @@ graphs: let err = load_server_settings(Some(&config_path), None, None, None, true).unwrap_err(); let msg = err.to_string(); assert!( - msg.contains("top-level `policy.file` is single-graph/CLI-local policy only"), - "expected single-graph policy guidance, got: {msg}" + msg.contains("top-level") && msg.contains("policy.file") && msg.contains("not honored"), + "expected top-level-not-honored guidance, got: {msg}" ); assert!( - msg.contains("graphs..policy.file"), + msg.contains("graphs."), "expected per-graph migration guidance, got: {msg}" ); assert!( @@ -5151,6 +5696,88 @@ graphs: ); } + #[test] + fn mode_inference_multi_rejects_top_level_queries() { + // Symmetric to the policy guard: a top-level `queries:` block in + // multi-graph mode is not honored (each graph uses its own), so it + // is a loud error rather than a silent no-op. + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + "queries:\n q:\n file: ./q.gq\ngraphs:\n alpha:\n uri: /tmp/alpha.omni\n", + ) + .unwrap(); + let err = load_server_settings(Some(&config_path), None, None, None, true).unwrap_err(); + let msg = err.to_string(); + assert!( + msg.contains("queries") && msg.contains("not honored"), + "top-level queries must be rejected in multi-graph mode: {msg}" + ); + } + + #[test] + fn single_mode_named_graph_rejects_top_level_blocks() { + // Serving a graph by name (`--target`/`server.graph`) uses its + // per-graph block; a populated top-level block would be silently + // shadowed, so boot refuses and names the per-graph location. + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + "policy:\n file: ./top.yaml\ngraphs:\n prod:\n uri: /tmp/prod.omni\n", + ) + .unwrap(); + let err = + load_server_settings(Some(&config_path), None, Some("prod".to_string()), None, true) + .unwrap_err(); + let msg = err.to_string(); + assert!( + msg.contains("prod") && msg.contains("policy.file") && msg.contains("graphs.prod"), + "named single-mode + top-level policy must refuse, naming the graph: {msg}" + ); + } + + #[test] + fn single_mode_named_graph_uses_per_graph_policy_and_queries() { + // The identity rule: `--target prod` attaches `graphs.prod`'s own + // policy + queries, not the top-level ones (which are absent here). + let temp = tempfile::tempdir().unwrap(); + fs::write( + temp.path().join("prod.gq"), + "query pq() { match { $u: User } return { $u.name } }", + ) + .unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + "graphs:\n prod:\n uri: /tmp/prod.omni\n policy:\n file: ./prod-policy.yaml\n \ + queries:\n pq:\n file: ./prod.gq\n", + ) + .unwrap(); + let settings = + load_server_settings(Some(&config_path), None, Some("prod".to_string()), None, true) + .unwrap(); + match settings.mode { + ServerConfigMode::Single { + graph_id, + policy_file, + queries, + .. + } => { + assert_eq!(graph_id, "prod", "named single-mode keeps graph identity"); + assert!( + policy_file + .as_ref() + .is_some_and(|p| p.ends_with("prod-policy.yaml")), + "per-graph policy attached: {policy_file:?}" + ); + assert!(queries.lookup("pq").is_some(), "per-graph query attached"); + } + other => panic!("expected Single mode, got {other:?}"), + } + } + #[test] fn mode_inference_normalizes_multi_graph_uris() { let temp = tempfile::tempdir().unwrap(); @@ -5383,6 +6010,7 @@ graphs: uri: graph_uri, engine: Arc::new(engine), policy: None, + queries: None, }); let tokens = vec![("act-andrew".to_string(), "secret-token".to_string())]; let workload = omnigraph_server::workload::WorkloadController::from_env(); @@ -5450,6 +6078,7 @@ graphs: uri: graph_uri, engine: Arc::new(engine), policy: None, + queries: None, })); } diff --git a/crates/omnigraph/src/db/omnigraph.rs b/crates/omnigraph/src/db/omnigraph.rs index eb58623..9d1403d 100644 --- a/crates/omnigraph/src/db/omnigraph.rs +++ b/crates/omnigraph/src/db/omnigraph.rs @@ -67,6 +67,12 @@ pub struct SchemaApplyResult { pub steps: Vec, } +#[derive(Debug, Clone)] +pub struct SchemaApplyPreview { + pub plan: SchemaMigrationPlan, + pub catalog: Catalog, +} + /// Top-level handle to an Omnigraph database. /// /// An Omnigraph is a Lance-native graph database with git-style branching. @@ -493,6 +499,14 @@ impl Omnigraph { schema_apply::plan_schema(self, desired_schema_source, options).await } + pub async fn preview_schema_apply_with_options( + &self, + desired_schema_source: &str, + options: SchemaApplyOptions, + ) -> Result { + schema_apply::preview_schema_apply(self, desired_schema_source, options).await + } + pub async fn apply_schema(&self, desired_schema_source: &str) -> Result { self.apply_schema_as(desired_schema_source, SchemaApplyOptions::default(), None) .await @@ -523,7 +537,28 @@ impl Omnigraph { options: SchemaApplyOptions, actor: Option<&str>, ) -> Result { - schema_apply::apply_schema(self, desired_schema_source, options, actor).await + self.apply_schema_as_with_catalog_check(desired_schema_source, options, actor, |_| Ok(())) + .await + } + + pub async fn apply_schema_as_with_catalog_check( + &self, + desired_schema_source: &str, + options: SchemaApplyOptions, + actor: Option<&str>, + validate_catalog: F, + ) -> Result + where + F: FnOnce(&Catalog) -> Result<()>, + { + schema_apply::apply_schema( + self, + desired_schema_source, + options, + actor, + validate_catalog, + ) + .await } pub(crate) async fn ensure_schema_apply_idle(&self, operation: &str) -> Result<()> { diff --git a/crates/omnigraph/src/db/omnigraph/schema_apply.rs b/crates/omnigraph/src/db/omnigraph/schema_apply.rs index 0dcf0f9..35fe161 100644 --- a/crates/omnigraph/src/db/omnigraph/schema_apply.rs +++ b/crates/omnigraph/src/db/omnigraph/schema_apply.rs @@ -48,50 +48,17 @@ pub(super) async fn plan_schema( Ok(plan) } -pub(super) async fn apply_schema( - db: &Omnigraph, - desired_schema_source: &str, - options: SchemaApplyOptions, - actor: Option<&str>, -) -> Result { - // Engine-layer policy gate (MR-722 chassis core). - // - // Fires BEFORE acquiring the schema-apply lock or doing any other - // work. When no PolicyChecker is installed this is a no-op and - // the apply path behaves exactly as it did before MR-722. When - // a PolicyChecker IS installed and the actor is None, this is a - // hard error — see Omnigraph::enforce's docstring for the - // forget-the-actor-footgun reasoning. - // - // Scope is TargetBranch("main") to match the HTTP-layer convention - // for SchemaApply: branch=None, target_branch=Some("main"). Cedar - // policies in the wild use `target_branch_scope: protected` to - // gate schema applies, so the engine-layer call has to set the - // target_branch shape that activates that predicate. Wrong scope - // here = silent policy mismatch with HTTP. See - // `omnigraph_policy::ResourceScope::to_branch_pair` for the mapping. - db.enforce( - omnigraph_policy::PolicyAction::SchemaApply, - &omnigraph_policy::ResourceScope::TargetBranch("main".to_string()), - actor, - )?; - - acquire_schema_apply_lock(db).await?; - let result = apply_schema_with_lock(db, desired_schema_source, options).await; - let release_result = release_schema_apply_lock(db).await; - match (result, release_result) { - (Ok(result), Ok(())) => Ok(result), - (Ok(_), Err(err)) => Err(err), - (Err(err), Ok(())) => Err(err), - (Err(err), Err(_)) => Err(err), - } +struct PlannedSchemaApply { + plan: SchemaMigrationPlan, + desired_ir: SchemaIR, + desired_catalog: Catalog, } -pub(super) async fn apply_schema_with_lock( +async fn plan_schema_for_apply( db: &Omnigraph, desired_schema_source: &str, options: SchemaApplyOptions, -) -> Result { +) -> Result { db.ensure_schema_state_valid().await?; let branches = db.coordinator.read().await.all_branches().await?; // Skip `main` and internal system branches. The schema-apply lock branch @@ -123,6 +90,87 @@ pub(super) async fn apply_schema_with_lock( .unwrap_or_else(|| "unsupported schema migration plan".to_string()); return Err(OmniError::manifest(message)); } + + let mut desired_catalog = build_catalog_from_ir(&desired_ir)?; + fixup_blob_schemas(&mut desired_catalog); + Ok(PlannedSchemaApply { + plan, + desired_ir, + desired_catalog, + }) +} + +pub(super) async fn preview_schema_apply( + db: &Omnigraph, + desired_schema_source: &str, + options: SchemaApplyOptions, +) -> Result { + let planned = plan_schema_for_apply(db, desired_schema_source, options).await?; + Ok(SchemaApplyPreview { + plan: planned.plan, + catalog: planned.desired_catalog, + }) +} + +pub(super) async fn apply_schema( + db: &Omnigraph, + desired_schema_source: &str, + options: SchemaApplyOptions, + actor: Option<&str>, + validate_catalog: F, +) -> Result +where + F: FnOnce(&Catalog) -> Result<()>, +{ + // Engine-layer policy gate (MR-722 chassis core). + // + // Fires BEFORE acquiring the schema-apply lock or doing any other + // work. When no PolicyChecker is installed this is a no-op and + // the apply path behaves exactly as it did before MR-722. When + // a PolicyChecker IS installed and the actor is None, this is a + // hard error — see Omnigraph::enforce's docstring for the + // forget-the-actor-footgun reasoning. + // + // Scope is TargetBranch("main") to match the HTTP-layer convention + // for SchemaApply: branch=None, target_branch=Some("main"). Cedar + // policies in the wild use `target_branch_scope: protected` to + // gate schema applies, so the engine-layer call has to set the + // target_branch shape that activates that predicate. Wrong scope + // here = silent policy mismatch with HTTP. See + // `omnigraph_policy::ResourceScope::to_branch_pair` for the mapping. + db.enforce( + omnigraph_policy::PolicyAction::SchemaApply, + &omnigraph_policy::ResourceScope::TargetBranch("main".to_string()), + actor, + )?; + + acquire_schema_apply_lock(db).await?; + let result = apply_schema_with_lock(db, desired_schema_source, options, validate_catalog).await; + let release_result = release_schema_apply_lock(db).await; + match (result, release_result) { + (Ok(result), Ok(())) => Ok(result), + (Ok(_), Err(err)) => Err(err), + (Err(err), Ok(())) => Err(err), + (Err(err), Err(_)) => Err(err), + } +} + +pub(super) async fn apply_schema_with_lock( + db: &Omnigraph, + desired_schema_source: &str, + options: SchemaApplyOptions, + validate_catalog: F, +) -> Result +where + F: FnOnce(&Catalog) -> Result<()>, +{ + let planned = plan_schema_for_apply(db, desired_schema_source, options).await?; + validate_catalog(&planned.desired_catalog)?; + let PlannedSchemaApply { + plan, + desired_ir, + desired_catalog, + } = planned; if plan.steps.is_empty() { return Ok(SchemaApplyResult { supported: true, @@ -132,9 +180,6 @@ pub(super) async fn apply_schema_with_lock( }); } - let mut desired_catalog = build_catalog_from_ir(&desired_ir)?; - fixup_blob_schemas(&mut desired_catalog); - let snapshot = db.snapshot().await; let base_manifest_version = snapshot.version(); let mut added_tables = BTreeSet::new(); diff --git a/docs/dev/index.md b/docs/dev/index.md index d9ba5e5..600c969 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -59,6 +59,8 @@ Working documents for in-flight feature work. Removed when the work lands. |---|---| | Schema-lint chassis v1 (MR-694) — `--allow-data-loss`, soft/hard drops | [schema-lint-v1-plan.md](schema-lint-v1-plan.md) | | Inline + stored queries, request/response envelope, MCP (MR-656 / MR-976 / MR-969) | [rfc-001-queries-envelope-mcp.md](rfc-001-queries-envelope-mcp.md) | +| Config & CLI architecture — layered config, client targeting, file naming (MR-973 / MR-974 / MR-981) | [rfc-002-config-cli-architecture.md](rfc-002-config-cli-architecture.md) | +| MCP server surface — full tool parity, stored queries, modular auth (MR-969 / MR-956 / MR-974) | [rfc-003-mcp-server-surface.md](rfc-003-mcp-server-surface.md) | ## Boundary diff --git a/docs/dev/rfc-002-config-cli-architecture.md b/docs/dev/rfc-002-config-cli-architecture.md new file mode 100644 index 0000000..0a8e573 --- /dev/null +++ b/docs/dev/rfc-002-config-cli-architecture.md @@ -0,0 +1,590 @@ +# RFC: Config & CLI Architecture — Layered Config, Client Targeting, File Naming + +**Status:** Proposed +**Date:** 2026-05-30 +**Tickets:** MR-668 (multi-graph server, shipped — the dependency this builds on), MR-969 (stored queries + MCP — supplies the in-repo agent tool surface), MR-973 (quickstart / onboarding), MR-974 (agent setup surface), MR-981 (agent-friendly CLI hardening) +**Target release:** v0.8.x (tentative; phased — see Rollout) + +## Summary + +OmniGraph today has a single config file, `omnigraph.yaml`, read both by the CLI (operating the embedded engine) and by `omnigraph-server` (hosting graphs). There is **no client-side configuration that targets a *running server*** — to talk to a deployed `omnigraph-server` you drop to `curl` or the `omnigraph-ts` client. This is the one real gap in an otherwise coherent design (storage-URI addressing, multi-graph routing, per-graph policy). + +This RFC defines the config and CLI architecture that closes that gap, derived from first principles — *working backwards from what OmniGraph uniquely enables* rather than copying kubeconfig / `helix.toml`. The result: + +1. A **global-first layered config** — user-global (`~/.omnigraph/`) is the **primary, self-sufficient default**; per-project (`./omnigraph.yaml`) is an *optional* override + deployment manifest. One uniform schema, both layers optional; the CLI works from any directory with **no project file** (the `kubectl`/`aws`/`gh` posture), unlike today's project-anchored behavior. +2. A single unifying noun — the **target** — that resolves a name to a concrete `(locus, graph, sub-state, credential)` tuple, where the locus is **embedded (storage URI) XOR remote (server endpoint)**. +3. A **multi-server × multi-graph** client model (OmniGraph hosts N graphs per server and there are M servers — unlike Helix's one-cluster-one-graph). +4. **Credentials by reference, keyed by server name** (the AWS/gh/kube model) — OS keychain `omnigraph:` (preferred) → a `[]` profile in `~/.omnigraph/credentials` → `OMNIGRAPH_TOKEN[_]` env (CI). `servers.` is endpoint-only by default but may carry an explicit, secret-free `auth: { token: { env|file|command|keychain } }` source; no `credentials.yaml`; the shipped `bearer_token_env` + dotenv stay as a legacy compat path. Every committed/GitOps'd surface stays secret-free. +5. A **file-naming** decision: project and server config are **the same artifact, same name** (`omnigraph.yaml`); the only differently-named file is the user-global `config.yaml`, justified by **scope, not role**. + +The design optimizes jointly for **DX** (one command surface across embedded and remote; clone-and-go) and **AX** (agent experience: one flat resolved context, secrets structurally unreachable, branch-pinned reproducible reads, and a GitOps'd capability surface). + +## Reconciliation with shipped / planned CLI work + +Verified **against the code**, not ticket statuses (which are unreliable — e.g. MR-581 is marked done but is stale and unbuilt). Findings and the corrections they force: + +- **Noun is `graph`/`graphs`, NOT `target`/`targets`.** The config key is `graphs:` in `config.rs` and the flag is `--graph`. **This RFC uses `graphs:`/`--graph` throughout**; the unifying noun is a **`graphs:` entry** that is *embedded* (`storage:`, formerly `uri:`) XOR *remote* (`server:` + `graph_id:` defaulting to the entry key) — a typed locator (§1.1). Read any lingering `targets:`/`--target` below as `graphs:`/`--graph`. +- **`~/.omnigraph/` stands on its own merits** (Helix/aws/kube peer convention), **not** on precedent — there is **no `~/.omnigraph/` usage in the code** today. (MR-581 / MR-531 templates-into-`~/.omnigraph/` are *stale tickets, unbuilt*.) +- **Templates do not exist** in the code (no `template` command). The template mechanism is a *design question for this RFC / the init family*, not an existing foothold. +- **What actually exists in the CLI** (verified): `init, query(read), mutate(change), load, ingest, branch, schema, lint, snapshot, export, commit, policy, optimize, cleanup, graphs`. **Not built:** `serve, quickstart, template, prune, login`. `omnigraph init` exists (with `scaffold_config_if_missing`, `main.rs:1415`); the rest of the "init family" (`quickstart` MR-973, `serve` MR-970, `prune`/`init --force` MR-972/975, `mcp install`/skills MR-974, agent-mode MR-981) are **unbuilt tickets**, some stale. +- **Config still uses `aliases:`** (no `operations:` in code; MR-839 unbuilt). §6's reconciliation talks about `aliases:` as-is, noting `operations:` is a *proposed* rename. +- **`bearer_token_env` exists** (per-graph, `config.rs`); MR-971 flags a CLI-parity / server-side gap. The per-`servers.` extension lands on top of that. +- **A top-level `omnigraph lint` command exists** (verified). A stored-query *registry* validator must pick a verb that doesn't read as a competing lint/check. + +## Motivation + +Three problems, in priority order: + +- **No client→server targeting config.** The moment an operator stands up `omnigraph-server` — for bearer auth + Cedar at a network boundary + admission control + multi-graph routing — the CLI can't address it. `curl` is the fallback. There is no named, switchable, credential-carrying way to say "run this against `prod` on the team server." +- **Multi-server × multi-graph has no first-class expression.** OmniGraph genuinely runs N graphs per server across M servers. The same graph is **multi-homed** — `s3://b/prod` may be `prod` on server A, `production` on server B, and opened directly by the CLI. Today's flat `graphs:` map (name→storage-URI) can't express "graph `production` on server `prod-eu`." +- **Solo-first and embedded-first are unserved by the remote story.** A solo developer with no projects should define everything in `~`. A developer iterating locally (embedded, no server) and then pointing at staging (remote) should change *one word*, not learn a second command surface. + +MR-668 shipped the server side (multiple graphs per server). MR-969 ships the in-repo agent tool surface (stored queries / MCP). This RFC supplies the **client and config layer** that lets humans and agents target that surface coherently — the foundation under MR-973 / MR-974 / MR-981. + +## Non-Goals + +- **A control plane / dashboard for config.** Operators edit files and (for servers) restart. No runtime config-mutation API. Matches the MR-668 / MR-969 operational model. +- **Hot reload.** Restart-only for server-side config, matching MR-668 and MR-969. +- **Embedding secrets in any config file.** Credentials are by-reference; the git-ignored `auth.env_file` dotenv (or, later, the OS keychain) holds tokens. Never a committable `*.yaml`. +- **Renaming the project manifest by role.** No `omnigraph.server.yaml` / `omnigraph.client.yaml`. Role lives in sections, not filenames (see Design §3). +- **Dropping embedded mode.** Embedded-first is load-bearing for the file-naming decision; this RFC assumes it stays. +- **Cross-graph / cross-server tool listing in MCP.** Clients loop over per-graph catalogs (a MR-969 non-goal, restated). + +## Background + +OmniGraph runs on Lance 6.x: typed nodes/edges in per-type Lance datasets, atomic multi-table commits via a `__manifest` table, branchable and time-travelable. The CLI (`omnigraph`) operates the **embedded engine** directly against a storage URI — no HTTP client in its runtime dependencies. `omnigraph-server` (Axum) is a *separate* HTTP front-end over the same engine, with bearer auth + per-graph Cedar (MR-668). The two read the same `omnigraph.yaml` but never connect to each other. + +OmniGraph **already has a credentials-by-reference mechanism**, which this RFC builds on rather than replacing: `TargetConfig.bearer_token_env` names the env var holding a graph's bearer token, and `auth.env_file` points at a git-ignored dotenv (`.env.omni`) that the CLI auto-loads into the process (`load_env_file_into_process`) with real-env-vars-win precedence; `resolve_remote_bearer_token` resolves a token via env var then dotenv named lookup. `.env.omni` is already in `.gitignore`. + +The six **irreducible enablers** that drive the design (referenced as E1–E6 below): + +| # | Enabler | Consequence | +|---|---|---| +| E1 | A graph is a **self-contained storage URI**; the substrate (object store + manifest CAS) is the source of truth — no server required to read/write. | A graph is addressable **directly (embedded)**, not only via a server. | +| E2 | A server hosts **many graphs**; **many servers** exist. | The remote address space is **`{server} × {graph_id}`**. | +| E3 | The same graph is **multi-homed** under different per-locus names. | **Name ≠ identity.** Resolution is mandatory. | +| E4 | **Branch / commit / snapshot** are first-class addressable sub-state. | An address is *graph @ branch/snapshot*, not just graph. | +| E5 | Enforcement is **two-layered**: engine-layer Cedar (`_as` writers, works embedded) + HTTP-boundary bearer+Cedar (server only). | *How* you reach a graph determines *which* enforcement applies. | +| E6 | **Stored queries / MCP tools are a per-graph registry defined in the project config** (MR-969). | The **agent tool surface is version-controlled in the repo**. | + +Competitors collapse dimensions OmniGraph keeps live: **Helix** fuses E2+E3 (one cluster = one graph); **namidb** fuses E1+E3 into the URI (`s3://b?ns=prod`) and serves one namespace per process. OmniGraph has all of E1–E6 at once, so its config resolves a richer space — but the richness is *earned* by capability. + +## Design + +### 1. The address space and the `target` abstraction + +Every OmniGraph address is a tuple: + +``` +(locus, graph, sub-state, credential) + locus = embedded(URI) XOR remote(server-endpoint) # E1, E2 + graph = a URI (embedded) | a graph_id on a server (remote) # E3 + sub-state = branch | snapshot # E4 + credential = cloud-storage creds (embedded) | bearer token (remote) # E5 +``` + +The config's only job is **name → this tuple**. Define one noun — a **target** — that resolves to either shape: + +```yaml +targets: + dev: # embedded — substrate-direct (E1) + storage: s3://team-bucket/dev.omni + branch: main # sub-state (E4) + staging: # remote — resolves a server by reference (E2/E3) + server: staging # → looked up in `servers` + graph_id: prod # the graph's id on that server (defaults to the entry key) + branch: review +``` + +`--target staging` resolves: project `targets.staging` → `{server: staging, graph_id: prod, branch: review}` → `servers.staging` → `{endpoint, token-by-ref}` → final `(remote(https://…), prod, review, $TOKEN)`. Embedded targets skip the server hop and use cloud-storage credentials. + +**Two concepts, not kubeconfig's three.** kube splits cluster / user / context; that 3-way split is its most-cursed UX. A target *bundles* server+graph+branch+defaults under one name; the **only** thing split out is `servers`, because endpoints+credentials are shared across many targets and are secret-bearing (different ownership and rate-of-change; see §2). Result: **2 nouns — `servers` and `targets`.** Embedded `targets` (`storage:`) subsume today's `graphs:` entries. + +### 1.1 The resolved address is a typed *locator*, not a `uri` string + +The shipped config models a graph as a single `uri: String`, and code branches on `is_remote_uri(uri)`. That conflates two structurally different addresses: an **embedded** graph is a *complete, self-contained* address — one storage URI = one graph, opened directly via the embedded engine; a **remote** graph is a *server endpoint + a `graph_id`* — one server hosts N graphs. A bare server URL **is not a graph**; it lacks the `graph_id`. The cost of the string model, in the code today: + +- the CLI re-decides "server or file?" via `is_remote_uri` at ~16 call sites; +- `TargetConfig` (one `uri` field) **cannot express** multi-server × multi-graph or a multi-homed graph (E2/E3) — "graph `production` on server `prod-eu`" has no representation; +- the CLI **bails on remote URIs** for most operations, precisely because the string can't carry the `graph_id`; +- the `omnigraph-ts` SDK had to model `baseUrl` **+** `graphId` *separately* (rewriting `/graphs/{graphId}/…`) — it invented the structure the string lacks. + +So the *resolved* address is a **typed locator**, not a string: + +```rust +enum GraphLocator { + Embedded { storage: StorageUri }, // file:// , s3:// — a complete graph + Remote { server: ServerId, graph_id: GraphId }, // which server + which graph (+ bearer creds) +} +``` + +A `graphs:` entry resolves into this **once**; downstream code dispatches on the variant (the breadboard's `GraphConn = Embedded(engine) | Remote(http)`) instead of re-sniffing a scheme at each call site. The `uri` string becomes an *input format* for the embedded variant, never the address itself. + +**YAML naming follows the locator — the *key* names the locus**, so neither the value's scheme nor a comment is load-bearing: + +| Locus | Key | Value | +|---|---|---| +| Embedded | **`storage:`** (shipped `uri:` is a deprecated alias) | a storage URI (`s3://…`, `file://…`) | +| Remote | **`server:`** | a name in `servers:` (its `endpoint` + creds resolve by name, §5) | +| Remote graph id | **`graph_id:`** | the id on that server — **defaults to the entry key**; set only when the local alias differs | + +An entry has `storage:` **xor** `server:` — the deserializer rejects *both* and *neither* (no silent ambiguity). This removes two prior confusions: `graphs:` (the map) vs `graph:` (the remote id), and `uri:`-might-be-a-server. + +```yaml +servers: + prod-eu: { endpoint: https://og-eu.internal:8080 } +graphs: + dev: { storage: s3://team-bucket/dev.omni } # embedded + production: { server: prod-eu } # remote — graph_id = "production" (the key) + staging: { server: prod-eu, graph_id: prod } # remote — alias ≠ server's id +``` + +### 1.2 Invalid configs are rejected by design + +The DX rule is: **a config field is either honored or rejected, never silently ignored**. The loader therefore has two phases: + +1. Parse YAML into a loose/raw shape that preserves origin (`base_dir`, layer, line/path when available). +2. Convert once into a typed, role-aware resolved config. Every command receives the resolved form, not the raw YAML structs. + +The typed graph shape is: + +```rust +enum GraphEntry { + Embedded(EmbeddedGraphEntry), + Remote(RemoteGraphEntry), +} + +struct EmbeddedGraphEntry { + storage: StorageUri, + branch: Option, + policy: Option, + queries: QueryRegistrySpec, +} + +struct RemoteGraphEntry { + server: ServerId, + graph_id: GraphId, + branch: Option, +} +``` + +That makes these rules structural rather than advisory: + +- A graph entry must specify **exactly one** locator: `storage:`/legacy `uri:` xor `server:`. +- `policy:` and `queries:` are valid only on `Embedded` graph entries, because they define the capability surface of a graph this process opens directly. A `Remote` graph entry points at a server; that server owns policy and stored-query definitions. +- `omnigraph-server` may serve only `Embedded` graph entries. A server manifest entry with `server:` is rejected: a server should not "host" a graph by proxying another server. +- A named graph uses its own graph entry. Top-level `policy:` / `queries:` are a legacy anonymous-bare-URI compatibility path only; if a named graph is selected while top-level blocks would be ignored, config validation errors with a migration hint. +- A client-defined remote graph discovers stored queries from the server (`GET /queries`) and invokes them (`POST /queries/{name}`); it does not define `queries:` locally for that remote graph. + +Examples that must fail fast: + +```yaml +graphs: + prod: + storage: s3://team-bucket/prod.omni + server: prod-us # invalid: storage xor server +``` + +```yaml +graphs: + prod: + server: prod-us + graph_id: production + policy: { file: ./policies/prod.yaml } # invalid: remote graph policy lives on the server + queries: + find_user: { file: ./queries/find_user.gq } # invalid: remote graph queries are discovered +``` + +`omnigraph config view --resolved --show-origin` is the user-facing debugger for this boundary: it shows the final `Embedded` or `Remote` graph and where every honored field came from. Fields that cannot be honored never make it into the resolved view; they fail validation first. + +### 2. Layered config — global-first, uniform schema, project-optional + +**Posture: global-first, project-optional.** OmniGraph's CLI is primarily a *client* (it operates against graphs and servers, embedded or remote), so it sits on the **global-first** side of the CLI-config axis — like `kubectl` / `aws` / `gh` / `docker`, and unlike *project-first* tools (`git` / `cargo` / `terraform`) whose primary config is per-repo. The **global user config is the primary, self-sufficient default**; the project file is an *optional* repo-scoped override (and, when present, the deployment manifest). `omnigraph query --target prod` must work from **any directory with no project file**, exactly as `kubectl get pods --context prod` works from anywhere. *(This is a deliberate flip from today, where the CLI reads `./omnigraph.yaml` and does not even walk parent dirs — i.e. today it is project-anchored.)* + +**Rule: the two layers share ONE raw schema, and each is fully self-sufficient** (the git-layering mechanism — same schema at both levels; you never need a repo to have a working config). Do **not** specialize the file format by layer. Instead, run the same role-aware validation everywhere (§1.2): the global and project layers may both define graph locators, defaults, servers, and aliases, but fields that are meaningless for a resolved graph variant are rejected rather than ignored. For example, `queries:` is valid for an embedded graph this config opens directly; it is invalid on a remote graph entry because remote stored queries are server-owned and discovered. + +This makes the **zero-project case the default, not an edge case**: a solo user (or an agent) defines everything needed for client work in `~/.omnigraph/config.yaml` — servers, embedded + remote graph locators, defaults, aliases, and optionally personal embedded-graph query registries — and **never creates a project file**. A team adds `./omnigraph.yaml` only when it wants repo-scoped overrides or a committed, GitOps'd deployment manifest. Global-first does **not** forbid project files; it stops *requiring* them (the kubectl model: `~/.kube/config` is sufficient and default; per-project kubeconfigs are opt-in via `KUBECONFIG`). + +| Layer | Required? | Typical use | Path | +|---|---|---|---| +| Global | no | **the default** — solo/agent's entire config; shared servers+creds for teams; even a personal server's graphs/queries | `~/.omnigraph/config.yaml` | +| Project | no | **opt-in** — repo-scoped overrides + the committed deployment manifest (graphs, queries, policy) | `./omnigraph.yaml` | + +**Precedence (low → high):** built-in defaults < global < project < env vars < CLI flags. With no project file it collapses to **built-in < global < env < flags** — the common global-only path. + +**Merge semantics — "closest layer wins, at the smallest meaningful unit"** (the field consensus: git / kubeconfig / cargo / Helm / VS Code): +- **Settings objects** (`defaults`, `auth`, `server`) → **deep-merge per field**: a project sets `defaults.graph` and *inherits* the global `defaults.output_format`. (VS Code / cargo behavior.) +- **Named-resource maps** (`servers`, `graphs` / compat `targets`, `queries`, `aliases`) → **union by key; on a collision the higher layer's entry REPLACES the lower wholesale** — *no field-level deep-merge within an entry*. (kubeconfig: union contexts by name.) The footgun this avoids: global `servers.prod = {endpoint, policy}`, project `servers.prod = {endpoint: other}` — deep-merge would silently retain the old fields; replace makes the project's `prod` self-contained and predictable. +- **Lists/arrays** → **replace, never append** (Helm convention; appending is order-sensitive and surprising). +- **Scalars** → higher layer wins. +- **Relative paths carry their origin's base_dir.** A `queries:` entry's `.gq` path, or a `policy.file`, resolves against the directory of the layer it was *defined in* — global entries under `~/.omnigraph/`, project entries under the project dir. +- **Inspectable (non-negotiable):** `omnigraph config view --resolved --show-origin` prints each final value *and which layer set it* (the `git config --show-origin` / `kubectl config view` rule). A layered config without origin-tracing is a debugging trap. + +### 3. Roles, and the file-naming decision (same name for project = server) + +`omnigraph.yaml` carries two *roles* that diverge in prod and collapse on a laptop: + +- **Server role** (read by `omnigraph-server`): `graphs:` entries that are **embedded storage locators**, per-graph `policy.file`, **`queries:` — the stored-query/MCP registry lives here**, plus serving knobs. Remote graph locators are rejected in this role. +- **Client role** (read by the CLI/agent): `servers:`, embedded or remote `graphs:` locators, `defaults:`, `aliases:`. A remote graph locator points at server-owned capabilities; it cannot define local `policy:` or `queries:`. + +**Project config and server config are the same artifact, hence the same name.** The server *serves the project*: the file that says "these graphs exist, with these stored queries and this policy" is simultaneously the project manifest and the server's deploy config. Role is distinguished by which *sections* are populated, never by filename. Readers ignore sections that are not theirs (today's file already does this with `cli:` vs `server:`). + +**Why not kube's role-split.** Two coherent models exist: (A) one project file with role-sections (Helix `helix.toml` holds both `[local.dev]` and `[enterprise.production]`; compose; Cargo), and (B) deployment-manifest strictly separate from client config (kubectl — you never put a context in `deployment.yaml`). kube is the sharpest topological analog (multi-server × multi-graph, one client targeting many), so B has a real claim. The tiebreaker is **E1: OmniGraph is embedded-first.** In embedded mode the manifest's `graphs:` *is* the local target list — manifest and local-client-view are the same object, so splitting them (B) fights the grain and forces two files for local work. kube splits because it has **no** embedded mode (client always remote+global). So: take the half kube is right about — *remote* client targeting (`servers:`, endpoints, creds) is a separate concern in a separate **user-global** file (`config.yaml`, like `~/.kube/config`); reject the half it is wrong about for us — do **not** split the *project* layer by role. **The second name (`config.yaml`) is justified by scope (user-global), not role.** *(If OmniGraph ever dropped embedded mode and went pure-remote, model B's strict split would become cleanest.)* + +### 4. File naming + +Principles from the field: **one global dir** `~/.omnigraph/` (like `~/.aws`/`~/.kube`/`~/.helix`), with config/cache/state as **subdirectories** (separation without XDG's three-root scatter); **secrets keyed by server name in the OS keychain or a separate git-ignored profile file** (AWS/gh model, not a new `credentials.yaml`); **project-root manifest keeps the app-named file** (`Cargo.toml`, `package.json`); **`.yaml`, not `.yml`**; keep OmniGraph's established names. The genuinely *new* decisions are the **global** dir's existence and keyed-by-name resolution with an explicit `auth.token` override (MR-971); the shipped `bearer_token_env` + `auth.env_file` mechanism remains as legacy compat. + +| Artifact | Path / name | Why | +|---|---|---| +| Project = server config (one artifact) | `./omnigraph.yaml` | **Keep.** Root manifest like `Cargo.toml` / `compose.yaml` / `helix.toml`. Same name for both roles because it is one file. In prod the server's deploy repo and an app repo each have their own `omnigraph.yaml` — same name, different repos. | +| Global user config | `~/.omnigraph/config.yaml` | **One dir** (`~/.omnigraph/`, like `~/.aws`/`~/.kube`/`~/.helix`). Named `config.yaml` *not* `omnigraph.yaml` — the name signals scope (and `~/.aws/config`, `~/.kube/config`, `~/.helix/config` all do this). Holds the full schema so a solo user needs nothing else. | +| Credentials | OS keychain (`omnigraph:`, preferred) → `~/.omnigraph/credentials` profile file (`[]`, `0600`, git-ignored). **Keyed by server name**, inside the one dir. | **Key by name, AWS/gh model** — `~/.aws/credentials [profile]`, `~/.kube/config users:`, `~/.helix/credentials`. *Not* a `credentials.yaml`, and *not* a per-server hand-named env var; the secret lives under the server name (no indirection). Legacy `bearer_token_env` + `.env.omni` dotenv remain as a compat path. See §5. | +| Cache / state | `~/.omnigraph/cache/`, `~/.omnigraph/state/` | Subdirs of the one dir (like `~/.aws/sso/cache/`, `~/.kube/cache/`) — cache is `rm -rf`-safe and backup-excludable without scattering across XDG roots. | +| Cedar policy | `./policies/.yaml` + `.tests.yaml` | **Keep.** Referenced by `policy.file`. | +| Schema | `./*.pg` (e.g. `schema.pg`) | **Keep.** | +| Stored queries | `./queries/*.gq` | **Keep.** `.gq` sources referenced by the `queries:` registry. | + +**Global dir: `~/.omnigraph/` — one place, with subdirectories.** Everything OmniGraph keeps for a user lives under a single `~/.omnigraph/` directory, matching the peer group (`~/.aws`, `~/.kube`, `~/.docker`) and the direct competitor (`~/.helix`). This is what DB/cloud-CLI users expect and the lowest-cognitive-load shape. + +*Separation and "one place" are not in conflict* — the decisive realization. The peer tools get config/cache/state separation via **subdirectories inside the one dir**, not via XDG's three scattered roots: `~/.aws/sso/cache/`, `~/.kube/cache/`. So OmniGraph keeps `~/.omnigraph/config.yaml`, `~/.omnigraph/credentials`, `~/.omnigraph/cache/` (catalogs — `rm -rf`-safe, backup-excludable), `~/.omnigraph/state/` (session, logs) — getting cache hygiene **and** a single discoverable location, without the XDG scatter. An earlier draft argued XDG on a false dichotomy (it assumed single-dir ⇒ mixed); subdirs dissolve it. `~/.omnigraph/` is canonical and documented; `$XDG_CONFIG_HOME` may optionally be honored if a user has set it, but XDG is not part of the mental model. + +**Env / override precedence (the `KUBECONFIG` analog):** +- `OMNIGRAPH_CONFIG=/path` — explicit config file, highest precedence. +- `OMNIGRAPH_HOME=/path` → the global dir (default `~/.omnigraph/`); `$XDG_CONFIG_HOME` optionally honored if a user has set it, but `~/.omnigraph/` is canonical. +- Cache and state are subdirs of the one dir: `~/.omnigraph/cache/` (cached remote catalogs), `~/.omnigraph/state/` (session, logs). +- Per-server token resolution: an explicit `auth: { token: {...} }` source (env/file/command/keychain) wins if set; otherwise **keyed by the server name** — `OMNIGRAPH_TOKEN_` (or `OMNIGRAPH_TOKEN` for the active server) → OS keychain `omnigraph:` → the `[]` profile in `~/.omnigraph/credentials`; legacy `bearer_token_env` still honored. See §5. + +### 5. Credentials, connection tiers, and bind portability (12-factor) + +**Credentials are by-reference everywhere, never inlined — and keyed by the *server name*, not by a hand-invented env-var name.** This is the one place the design departs from simply reusing the shipped `bearer_token_env` mechanism, because that mechanism is sub-optimal for a multi-server client: it forces the operator to invent and coordinate an env-var name per server (three steps to add a server: pick a var, name it in config, set it in the store). The peer group (AWS profiles, `gh` hosts, kubeconfig users, docker auths) instead keys the secret **by the server's name** — no indirection. OmniGraph should match that. + +**Resolution for server `` (no config field required):** +1. **`OMNIGRAPH_TOKEN_`** env var (name-derived, upper-snake), else **`OMNIGRAPH_TOKEN`** for the active server — the CI/headless override (12-factor). +2. **OS keychain** entry `omnigraph:` — the preferred interactive store (no plaintext on disk); written by `omnigraph login `. +3. **`~/.omnigraph/credentials`** — an AWS-style profile file keyed by server name (mode `0600`, git-ignored), the fallback when no keychain: + ```ini + [prod-us] + token = … + [prod-eu] + token = … + ``` +So a `servers.` with no token field resolves by name — adding a server is one step (`omnigraph login `), and "multiple servers, multiple tokens" falls out for free. + +**But implicit must not be the *only* path — explicit sourcing is a first-class option** (the DX/AX lesson). Pure-convention is invisible (you must *know* `OMNIGRAPH_TOKEN_`), can't integrate with a secrets-manager's fixed var name, and can't do dynamic/short-lived tokens. So a server may declare an explicit `auth:` block — a **method-agnostic wrapper** (today only `token:` for bearer; `mtls:`/`oidc:` are the future siblings, so the credential model never has to be re-keyed) holding a tagged token *source*. Secrets are *still* never inlined (every source is a reference): + +```yaml +servers: + prod-us: + endpoint: https://og-us… + auth: { token: { env: OG_PROD_US_TOKEN } } # explicit env var — self-documenting (= legacy bearer_token_env) + prod-eu: + endpoint: https://og-eu… + auth: { token: { command: [vault, read, -field=token, secret/og] } } # dynamic / short-lived + edge: + endpoint: https://og-edge… + auth: { token: { file: /run/secrets/og-token } } # k8s/docker mounted secret + staging: + endpoint: https://og-staging… # no auth: → implicit chain (below) +``` + +| `auth.token:` source | when | DX/AX value | +|---|---|---| +| *(auth omitted)* | the common case | zero-config; `omnigraph login` populates keychain `omnigraph:` | +| `{ env: VAR }` | secrets-manager / CI injects a fixed var | **self-documenting** — config states the source; = the legacy `bearer_token_env` | +| `{ file: PATH }` | k8s/docker secret mounted as a file | no env plumbing | +| `{ command: [...] }` | Vault, cloud IAM, `gh auth token` | **dynamic tokens** — first-class exec, the capability pure-env/keychain can't give (kube `exec` / AWS `credential_process`) | +| `{ keychain: ENTRY }` | pin a non-default keychain entry | explicit override of the name-derived default | + +**Resolution per server:** if `auth.token:` is set, use that source (no fallthrough). Else the **implicit chain**: `OMNIGRAPH_TOKEN_` (or `OMNIGRAPH_TOKEN` for the active server) → keychain `omnigraph:` → `[]` in `~/.omnigraph/credentials` (`0600`, git-ignored). `omnigraph login ` writes/rotates only that server's secret; per-server precedence is independent; sharing is opt-in (same env var or source). The `command` source runs locally with the operator's own privileges and is defined only in operator-owned config (never server-supplied), so it adds no remote-execution surface. The `auth:` wrapper is method-agnostic so adding mTLS/OIDC later is a new sibling key, not a breaking re-key (Hyrum's Law: the field name is a contract once shipped). There is **no `credentials.yaml`** and **no inlined secret**. *Convention for the floor, explicit for control — and explicit is legible to agents and never inlines a secret.* + +**Back-compat.** The shipped per-graph `bearer_token_env` + `auth.env_file` dotenv (`resolve_remote_bearer_token`, real-env-wins) keeps working unchanged for existing single-server setups; `bearer_token_env` is just the legacy flat alias for `auth: { token: { env } }`. Resolution tries an explicit `auth.token:` (or legacy `bearer_token_env`) first, then the keyed-by-name chain — so nothing breaks, but the zero-config default is the no-boilerplate keyed-by-name path. (MR-971 — the `bearer_token_env` parity gap — is where this resolver work lands.) + +**Three connection tiers** (Supabase/Prisma teach the zero-config floor): +1. **Env vars** — `OMNIGRAPH_SERVER=https://…` + `OMNIGRAPH_TOKEN=…`: zero-config remote, no file (the `DATABASE_URL` floor). +2. **Global `config.yaml`** — named `servers:` + `graphs:` for multi-server setups (the AWS-profiles convenience). +3. **Project `omnigraph.yaml`** — project-pinned targets/graphs, committed. + +**Keep `omnigraph.yaml` a *portable* manifest (12-factor).** Deploy-specific runtime that varies per environment — the **bind host/port**, worker counts — should be supplied by **`--bind` / `OMNIGRAPH_BIND` (flags/env)**, *not* a committed `server.bind:` baked into the manifest. A manifest that hardcodes `0.0.0.0:8080` is not portable across deploys and leaks an environment detail into a version-controlled file. The same-named `omnigraph.yaml` stays portable across deploys precisely because the volatile, per-environment knobs live in env/flags (12-factor config), while the stable, portable definition (graphs, queries, policy) lives in the file. This is the one concrete lesson taken from kube's model-B without adopting its file split: portability via env/flags, not via a second file. + +### 6. Where stored queries live: defined locally, invoked remotely + +A stored query splits across two axes; do not conflate them: +- **Definition** (`.gq` source + `queries:` entry) lives next to the **embedded graph entry that owns it**. For a hosted remote graph, that is the **deployment manifest** read by `omnigraph-server`; for a personal embedded graph, it may be the user's own config. It never lives on a client-side `Remote` graph entry. +- **Discovery** ("what tools exist for me?") is fetched from the **server** (Cedar-filtered `GET /queries` / MCP catalog) at connect time. +- **Invocation** is **remote** (client → server, HTTP/MCP) — or **embedded** (the CLI opens the graph directly and reads the same manifest). + +For remote use, the client carries *pointers to servers*, not query definitions; it **discovers and invokes**, never defines. This is the **capability-as-code guarantee for agents**: an agent can only invoke tools the server's *committed, reviewed* config exposes — it **cannot define a new tool at runtime**. Definition is structurally outside the agent's reach. + +`queries:` (graph-capability registry, Cedar-gated when served remotely, MCP-visible when exposed) and `aliases:` (client CLI shortcut) overlap — both can name `.gq`-backed operations. This RFC keeps them siblings (the MR-969 decision); the clean long-term is **one registry, two invocation surfaces** (embedded + remote), with `aliases:` subsumed. Out of scope here. + +#### Reconciling `aliases:` with the role model + +`aliases:` is the pre-MR-969, **client-role, embedded-only, ungated** ancestor of `queries:`. An alias bundles `command` (read/change), `query` (`.gq` path), `name` (symbol), `args` (positional param names), and `graph`/`branch`/`format` defaults; the CLI runs it embedded. The server never reads it. So: + +- **Role:** `aliases:` is **client-role** (CLI behavior) → it may live in **both** the user-global `config.yaml` and the project manifest, layered. `queries:` is **graph-capability role** → it lives only on an `Embedded` graph entry, and for remote server graphs that means the server deployment manifest. *Who opens the graph determines where query definitions can live.* +- **Difference:** `aliases:` = embedded invocation, no gating, explicit `command`, bundles client defaults + positional args. `queries:` = remote (+future embedded), Cedar + `mcp.expose`, **infers** read/mutate, bundles only MCP settings. +- **Convergence:** decompose an alias — *definition* (name→.gq+symbol) → `queries:` (the superset: typed, validated, gated, multi-surface, no redundant `command`); *target/branch/format* → client invocation context (`--target`/`--branch`/`--format` or `defaults:`), not baked per-query; *positional `args`* → thin CLI sugar or dropped (agents/services use named JSON params). End-state: one `queries:` registry + the client config model subsumes `aliases:`. +- **Validation:** a file-backed alias (`query: ./foo.gq`) may target only an embedded graph. A remote graph shortcut must be explicit that it invokes a server-owned stored query, e.g. `invoke: find_user`, so the client cannot smuggle a new `.gq` definition into a remote capability surface. +- **v1:** keep `aliases:` unchanged. Footgun worth a load-time warn: an alias and a query with the same name in one manifest are different namespaces invoked differently (`--alias X` vs `POST /queries/X`). + +```yaml +aliases: + local_owner: + command: query + query: ./queries/owner.gq + name: owner + graph: dev # valid only if `dev` resolves Embedded + + remote_owner: + invoke: find_user + graph: prod # valid only if `prod` resolves Remote; source lives on the server + args: [name] +``` + +### 7. CLI surface + +- `omnigraph login ` — interactive auth; stores the token keyed by server name in the OS keychain (`omnigraph:`) or the `[]` profile of `~/.omnigraph/credentials` (0600). The `gh auth login` analog. +- `omnigraph use ` — set the active graph (writes the appropriate layer). The `kubectl config use-context` analog. +- `omnigraph config view [--resolved] [--show-origin] []` — print the merged config and, with `--resolved`, the final tuple **plus the origin layer of every field** (the `git config --show-origin` / `kubectl config view` analog). Resolution is never a mystery. +- All existing verbs (`query`, `mutate`, `load`, `schema`, `branch`, …) gain `--graph `; resolution decides embedded vs remote transparently. + +### 7.5 Init, login, and bootstrap — three tiers (folds in the Q2 design) + +Scaffolding splits into three tiers by *scope* and *fatness*, mirroring the field (supabase `init` vs `login`; HelixDB thin `init` vs fat `chef`). Most of this lives in sibling tickets; this RFC owns only the **user route**. + +| Tier | Command | Scope | What it does | Model | Status | +|---|---|---|---|---|---| +| **User route** | `omnigraph login []` | user (`~/.omnigraph/`) | auth + write `~/.omnigraph/config.yaml` / `credentials`; first-run global setup | gh / supabase `login` | **this RFC** (unbuilt) | +| **Thin project init** | `omnigraph init` | project, in-place | create graph + `scaffold_config_if_missing` (`omnigraph.yaml` + minimal `.pg`/`.gq`); refuse-if-exists or `--force` | `cargo init`, `prisma init` | exists; `--force` purge = MR-975 | +| **Fat bootstrap** | `omnigraph quickstart [--template ] [--auto]` | project, possibly new-dir | scaffold + seed data + `serve start` + agent prompt file | HelixDB `chef`, `create-next-app` | MR-973 (unbuilt) | + +**Design positions** (first-principles, since none of the fat tier is built): +- **Split `init` (project) from `login` (user)** — never one command writing to both `$HOME` and the project (the supabase line, not the dbt line). `init`=project scaffold; `login`=user credential + global config. +- **`init` is in-place + refuse-if-exists** (cargo/prisma/terraform default): don't clobber; adopt existing files; require `--force` to overwrite (and `--force` purges Lance state per MR-975). +- **Interactive for humans, `--auto`/agent-mode for automation** (npm `-y`, create-* `--CI`, MR-981 `--machine`). In `OMNIGRAPH_AGENT_MODE` any prompt → fail with a repair hint. +- **Templates are a `--template ` flag on the fat tier** (create-vite model), with the *content* (schema + queries + seed) coming from a template source. Mechanism is a design question (bundled-in vs `og template pull` from a repo vs `npm create-*`-style delegation) — **not** an existing foothold (MR-581 stale). Lean: a small set of bundled templates first (generic `Person→Knows`, plus promote `omnigraph-intel-bootstrap`), `--template ` later. +- **`init`/`quickstart` can scaffold the `graphs:` map with one or more entries**; "init with specific graphs" = the scaffolded `graphs:` block (embedded `storage:` locally; the agent/operator adds remote `server:` entries via `login` + editing). +- **Secrets-on-scaffold rule** (prisma/dbt/supabase all do this): anything that writes a token also keeps it out of VCS. `login` prefers the OS keychain (no file); the `~/.omnigraph/credentials` profile fallback is `0600` and git-ignored, and any project-local `.env`-shaped file gets a `.gitignore` entry. + +### 8. Concrete shape + +**Global** `~/.omnigraph/config.yaml` (per-user, secret-free): +```yaml +servers: # endpoint only — token is keyed by the server name + prod-us: { endpoint: https://og-us.internal:8080 } + prod-eu: { endpoint: https://og-eu.internal:8080 } + staging: { endpoint: https://og-staging.internal:8080 } +graphs: + personal: { storage: ~/graphs/personal.omni } +defaults: + graph: personal +aliases: + my_people: + command: query + query: ~/queries/people.gq + name: list_people + graph: personal +``` + +**Project client** `./omnigraph.yaml` (committed, secret-free, portable — no `server.bind`). Note the shipped noun is `graphs:` (MR-603); an entry is embedded (`storage:`) XOR remote (`server:` + `graph_id:`, §1.1): +```yaml +graphs: + dev: { storage: s3://team-bucket/dev.omni, branch: main } # embedded + staging: { server: staging, graph_id: prod, branch: review } # remote → graph `prod` on server `staging` + prod-us: { server: prod-us, graph_id: production } + prod-eu: { server: prod-eu, graph_id: production } # multi-homed: same graph, another server +defaults: { graph: dev, output_format: table } +aliases: + owner: + command: query + query: ./queries/owner.gq + name: owner + args: [name] + graph: dev +``` +Select with `--graph ` (shipped flag, MR-603). + +**Server deployment** `./omnigraph.yaml` (committed in the deploy repo, read by `omnigraph-server`). Every served graph is an embedded storage locator; server-owned policy and stored-query definitions live here: +```yaml +graphs: + production: + storage: s3://team-bucket/prod.omni + policy: + file: ./policies/prod.yaml + queries: + find_user: + file: ./queries/find_user.gq + mcp: { expose: true, tool_name: lookup_user } + +server: + policy: + file: ./policies/server.yaml +``` + +**Credentials** are keyed by server name — `omnigraph login prod-us` writes the OS keychain entry `omnigraph:prod-us` (or a `[prod-us]` profile in `~/.omnigraph/credentials`, 0600, git-ignored); `OMNIGRAPH_TOKEN_PROD_US` overrides for CI. No token fields in any config file; no committable secrets. + +## DX + +1. **One command surface, two loci.** `query --graph dev` (embedded) and `--graph staging` (remote) are the same command; only resolution differs. Change one word, not a mental model. +2. **Clone-and-go.** Project config names servers+graphs; teammate runs `omnigraph login staging` once and every target resolves. The git + `gh auth login` model. +3. **Multi-server × multi-graph is the default.** Remote graph entries reference `server` by name; `servers` is a global named map; graphs are per-server. `prod-us` and `prod-eu` both serving `production` is two graph entries — Helix cannot express this. +4. **Solo-first.** Everything in `~`, no project required. +5. **Laptop-to-fleet on one schema.** Local = one `omnigraph.yaml` (both roles); prod = role-split across repos. No second format to learn. + +## AX (agent experience) + +1. **One flat resolved context, never a config to navigate.** target→server→endpoint→token resolves *before* the agent sees anything. The agent reasons about tools, not topology (the LLM-safe-surface principle extended to config). +2. **Secrets are structurally outside the agent's reach.** The repo it operates in has no tokens; they are in the global layer / keychain, outside its view. An agent *cannot* exfiltrate a prod token from project config because it is not there. +3. **Branch/snapshot-pinned contexts** (E4) — hand an agent a `branch: review` / `--snapshot v42` target and its reads are reproducible and cannot see uncommitted main-line state. No kubeconfig analog. +4. **The agent's capabilities are a GitOps'd artifact** (E6) — which graphs exist, which stored-query tools it may call, and which Cedar rules gate them are all in the version-controlled server config. Powers change only via a reviewed PR, deployed by restart. Infrastructure-as-code for what the AI can do. +5. **Config + policy compose.** Config = "where am I pointed + which token"; Cedar = "what may I do there." Orthogonal; no enforcement logic leaks into config. + +## GitOps — three surfaces, secrets in none + +| Surface | Repo | Contents | Deploy | Secrets | +|---|---|---|---|---| +| Server deployment config | infra/deploy repo | `graphs:`, policy, **`queries:` + `.gq` files** | commit → CI → **server restart** (no hot reload) | none — by-reference | +| Project client config | app repo | `graphs:` → embedded storage or remote server+graph | committed, read by CLI/agent | none | +| Global user config | **not GitOps'd** — machine-local `~` | `servers:` + creds-by-ref | `omnigraph login` writes it | refs only (like `~/.kube/config`) | + +## Comparison + +| Property | kubeconfig | Helix | git | compose | **OmniGraph (this RFC)** | +|---|---|---|---|---|---| +| Named remote endpoints + creds-by-ref | ✅ | ✅ | partial | partial | ✅ (global `servers`) | +| Global + project layering, uniform schema | ✗ | ✗ | ✅ | ✗ | ✅ | +| Embedded OR remote under one name | ✗ | ✗ | n/a | ✗ | ✅ (E1) | +| Multi-server × multi-graph | ✅ | ✗ | n/a | n/a | ✅ (E2) | +| Branch/snapshot in the address | ✗ | ✗ | partial | ✗ | ✅ (E4) | +| Agent tool surface in the repo | ✗ | ✗ (separate bundle) | n/a | n/a | ✅ (E6) | +| Project manifest renamed by role | — | no | — | no | **no** | +| Concept count | 3 | 1 | 2 | 1 | **2 (servers/targets)** | + +## Migration / backwards compatibility + +- **Additive.** Today's `omnigraph.yaml` (`graphs:`, `cli:`, `server:`, `aliases:`, `policy:`) keeps working unchanged. `graphs:` entries are equivalent to embedded `targets:` with a `storage:` (shipped `uri:` is a deprecated alias); both resolve. +- **`targets:` is new** and optional. `servers:` is new and optional. Absent → today's behavior. +- **Global `~/.omnigraph/config.yaml` is new.** Absent → only project + env + flags, exactly as now. Its addition is the **global-first posture flip**: today the CLI is project-anchored (reads `./omnigraph.yaml`, no parent walk); the global config becomes the new primary discovery path so the CLI works with no project file. Existing project-only workflows are unchanged (project still overrides global); the flip is additive — it adds a fallback layer below the project file, it does not remove the project file. +- **`graphs:` → `targets:` is an evolution, not a break.** Both can coexist; `targets:` is the superset (adds remote + branch pinning). A future cleanup may alias `graphs:` to embedded `targets:`. +- **`server.bind` stays supported** but documentation steers operators to `--bind` / `OMNIGRAPH_BIND` for portability; no removal. +- **Credentials: keyed-by-name is new; `bearer_token_env` is the compat path.** The primary design (keychain / `[]` profile / `OMNIGRAPH_TOKEN_`) is new resolver work (lands on MR-971). The shipped `bearer_token_env` + `auth.env_file` dotenv (`resolve_remote_bearer_token`) is **unchanged and still honored** — existing single-server dotenv setups keep working, and the resolver honors an explicit `auth: { token: {...} }` source (env/file/command/keychain) with `bearer_token_env` as its flat legacy alias. No `credentials.yaml`. +- **Validation tightens invalid mixes, not valid legacy use.** Top-level `policy:` / `queries:` remain only for anonymous bare-URI compatibility. Named graphs use per-entry fields. Remote graph entries with local `policy:` / `queries:` and server manifests with `server:` graph locators are rejected because there is no correct way to honor those fields. + +## Open questions + +- **`graphs:` vs `targets:` naming churn.** Do we rename `graphs:` → `targets:` (with a deprecation alias) or keep `graphs:` for embedded and add `targets:` for remote? Leaning: keep both, document `targets:` as the superset. +- **Keychain integration scope.** Keychain is now the *primary* credential store (§5), so this is on the critical path, not optional: macOS Keychain first (matches operator practice) with the `0600` `[]` profile file as fallback; Linux Secret Service / `pass` later. Open: which keyring crate, and the exact `OMNIGRAPH_TOKEN_` name-derivation (upper-snake, non-alnum → `_`). +- **Project-local `servers:`.** Allowed (e.g. a localhost dev server), merged with global. Confirm creds stay by-reference even for project-local servers (yes). +- **`aliases:` ⇄ `queries:` convergence.** Out of scope here; tracked separately. One registry with embedded + remote invocation surfaces is the target end state. +- **Single-file `KUBECONFIG`-style list.** Do we support `OMNIGRAPH_CONFIG` pointing at multiple files (colon-joined), or a single file only? Start single; revisit if demand appears. + +## Implementation — breadboard + slices (Shape A) + +Shaped via requirements + a fit check (Shape A — global-first layered config + unified `graphs:` entry + three-tier init — selected over a project-first minimal option and a Helix-clone). This section breadboards A and slices it. **Bold** = NEW. + +### Places + +| # | Place | What | +|---|---|---| +| P1 | Disk | `~/.omnigraph/{config.yaml, credentials, cache/, state/}` + project `omnigraph.yaml` + `.env.omni` | +| P2 | Config resolution | runs on every command: load layers → merge → resolve `--graph` | +| P3 | Command execution | embedded engine OR remote HTTP client | +| P4 | Remote `omnigraph-server` | existing HTTP surface (`/query`, `/mutate`, `/queries/{name}`) | +| P5 | Scaffold | `login` / `init` / `quickstart` | + +### Affordances + +| # | Place | Affordance | NEW? | Wires | +|---|---|---|---|---| +| U1 | P1 | `~/.omnigraph/config.yaml` (operator edits) | **N** | → N1 | +| U2 | P1 | project `./omnigraph.yaml` | — | → N1 | +| U3 | P1 | `~/.omnigraph/credentials` / `.env.omni` dotenv (secrets, git-ignored) | — | → N4 | +| U4 | P3 | `omnigraph --graph ` (any command) | — | → N14 | +| U5 | P5 | `omnigraph login []` | **N** | → N11 | +| U6 | P5 | `omnigraph init` / `quickstart [--template]` | partly | → N12 / N13 | +| U7 | P2 | `omnigraph config view --resolved --show-origin` | **N** | → N10 | +| N1 | P2 | `load_layered_config()` — global (N3) + project (cwd), serde each | **N** | → N2 | +| N2 | P2 | **merge engine** — deep-merge settings; replace named-resource entries; replace lists; **retain provenance** and raw field origins | **N⚠️** | → N5, → S_merged | +| N3 | P2 | global-dir resolver — `OMNIGRAPH_HOME` else `~/.omnigraph/` | **N** | → N1 | +| N4 | P2 | `load_env_file_into_process` — dotenv, real-env-wins (existing) | — | → N9 | +| N5 | P2 | `resolve_graph(name, merged)` → typed `Embedded`/`Remote` locator; rejects invalid role/field combinations before execution | **N⚠️** | → N6 | +| N6 | P3 | `GraphConn` — `Embedded(engine)` \| `Remote(http)` dispatch | **N⚠️** | → N7, → N8 | +| N7 | P3 | embedded path — `Omnigraph::open(uri)` (existing) | — | → engine | +| N8 | P3 | **HTTP-client path** — POST `/query`/`/mutate`/`/queries/{name}` | **N⚠️** | → P4, → N9 | +| N9 | P2 | `resolve_bearer_token(server)` — explicit `auth.token` source if set, else **keyed by name**: `OMNIGRAPH_TOKEN_`/`OMNIGRAPH_TOKEN` → keychain `omnigraph:` → `[]` profile; legacy `bearer_token_env`/dotenv (MR-971) | **N⚠️** | → N8 | +| N10 | P2 | `config view` handler — merged + per-field origin (needs N2 provenance) | **N** | → U7 | +| N11 | P5 | `login` handler — interactive auth → write `config.yaml` + `credentials` (0600) + `.gitignore` | **N⚠️** | → S_global | +| N12 | P5 | `init` handler — `scaffold_config_if_missing` + create graph; refuse-if-exists/`--force` purge (MR-975) | partly | → S_project | +| N13 | P5 | `quickstart` handler — scaffold + `--template` + seed + `serve start` + agent prompt (MR-973; needs serve MR-970) | **N⚠️** | → S_project | +| N14 | P3 | agent-mode wrapper — `--machine`/`OMNIGRAPH_AGENT_MODE`: JSON, structured errors, never-prompt, typed exit codes (MR-981) | **N⚠️** | → N1 | +| S_global | P1 | `~/.omnigraph/config.yaml` + `credentials` | **N** | read by N1/N9 | +| S_project | P1 | `./omnigraph.yaml` + `.env.omni` | — | read by N1/N4 | +| S_merged | P2 | in-memory resolved config (per command, with provenance) | **N** | read by N5/N10 | +| S_cache | P1 | `~/.omnigraph/cache/` (remote catalogs) | **N** | read by N8 | + +```mermaid +flowchart TB + subgraph P1["P1: Disk"] + U1["U1: ~/.omnigraph/config.yaml"] + U2["U2: ./omnigraph.yaml"] + U3["U3: credentials dotenv"] + end + subgraph P2["P2: Config resolution"] + N3["N3: global-dir (OMNIGRAPH_HOME)"] + N1["N1: load_layered_config"] + N2["N2: merge engine (+provenance)"] + N4["N4: dotenv loader"] + N5["N5: resolve_graph(--graph)"] + N9["N9: resolve_bearer_token"] + N10["N10: config view"] + end + subgraph P3["P3: Command execution"] + U4["U4: omnigraph --graph"] + N14["N14: agent-mode wrapper"] + N6["N6: GraphConn embedded|remote"] + N7["N7: embedded Omnigraph::open"] + N8["N8: HTTP-client POST"] + end + subgraph P5["P5: Scaffold"] + U5["U5: login"]; U6["U6: init/quickstart"] + N11["N11: login handler"]; N12["N12: init"]; N13["N13: quickstart"] + end + P4["P4: remote omnigraph-server"] + U1-->N1; U2-->N1; N3-->N1; N1-->N2-->N5-->N6 + U3-->N4-->N9-->N8 + U4-->N14-->N1 + N6-->N7; N6-->N8-->P4 + N2-->N10-->U7["U7: config view --resolved"] + U5-->N11; U6-->N12; U6-->N13 + classDef ui fill:#ffb6c1,stroke:#d87093,color:#000 + classDef n fill:#d3d3d3,stroke:#808080,color:#000 + class U1,U2,U3,U4,U5,U6,U7 ui + class N1,N2,N3,N4,N5,N6,N7,N8,N9,N10,N11,N12,N13,N14 n +``` + +### Slices (vertical, each demo-able) + +| # | Slice | Parts/affordances | Demo | +|---|---|---|---| +| **V1** | **Global layer + merge + `config view`** | A1–A4 · N1,N2,N3,N10 · U1,U7,S_global,S_merged | Put config in `~/.omnigraph/`, run `omnigraph config view --resolved --show-origin` from any dir → merged result with per-field origin; existing embedded commands work global-first with no project file | +| **V2** | **Remote graphs + HTTP client + creds** | A5–A7 · N5,N6,N8,N9 · S_cache | Define a `server:` graph entry; `omnigraph query --graph prod` hits the remote server (`curl`-free); embedded `--graph dev` still local | +| **V3** | **`omnigraph login`** | A8 · N11,U5 | `omnigraph login prod` writes `~/.omnigraph/credentials` (0600) + `.gitignore`; V2 remote query now works with no manual env | +| **V4** | **Thin-init hardening + quickstart + templates** | A9 · N12,N13,U6 (needs serve MR-970) | `omnigraph quickstart --template person-knows` scaffolds + seeds + serves; `init --force` purges (MR-975) | +| **V5** | **Agent-mode** | A10 · N14,U4 (MR-981) | `OMNIGRAPH_AGENT_MODE=1 omnigraph query …` → JSON + structured errors + typed exit codes; never-prompt | + +V1 is the foundation (global-first + merge + view). V2 closes the substantive client→server gap. V3 is credential ergonomics. V4/V5 ride sibling tickets (MR-970/973/981). MR-969 (stored queries) ships independently and is reached by N8's `/queries/{name}` once V2 lands. + +## Rollout + +The slices above are the rollout order: **V1 (global layer + merge) → V2 (remote graphs + HTTP client) → V3 (login) → V4 (quickstart/templates, on MR-970) → V5 (agent-mode, MR-981).** V1–V2 close the substantive gap (global-first config + `curl`-free server access); V3–V5 are ergonomics that ride sibling tickets. Evaluate after V2 against early-adopter and agent-onboarding (MR-973 / MR-974) signal. The spikes (X1 HTTP-client, X2 merge engine, X3 resolver+provenance, X4 login) resolve before their owning slice. + +## Prior art + +- kubeconfig (clusters / users / contexts; `KUBECONFIG`; `kubectl config view`) +- Helix CLI v2 (`helix.toml` local+enterprise instance blocks; `~/.helix/config`; `~/.helix/credentials`) +- AWS CLI (`~/.aws/config` + `~/.aws/credentials` split; named profiles; `credential_process`) +- git (`~/.gitconfig` + `.git/config`; `--show-origin`) +- Cargo (`Cargo.toml` manifest + `~/.cargo/config.toml`) +- Supabase / Prisma (one project manifest; connection via `DATABASE_URL` env) +- 12-factor app (config that varies by deploy lives in the environment) diff --git a/docs/dev/rfc-003-mcp-server-surface.md b/docs/dev/rfc-003-mcp-server-surface.md new file mode 100644 index 0000000..32fbce5 --- /dev/null +++ b/docs/dev/rfc-003-mcp-server-surface.md @@ -0,0 +1,270 @@ +# RFC: MCP Server Surface for `omnigraph-server` — Full Tool Parity, Stored Queries, Modular Auth + +**Status:** Proposed +**Date:** 2026-06-01 +**Tickets:** MR-969 (stored queries + MCP exposure — the surface this completes), MR-956 (federated auth / WorkOS OAuth — the auth substrate this consumes), MR-971 (per-server credential resolver), MR-974 (agent setup surface — the installer that wires this), MR-668 (multi-graph server — shipped, the routing this builds on) +**Builds on:** [omnigraph#128](https://github.com/ModernRelay/omnigraph/pull/128) (`ragnorc/stored-queries-mcp`) — the shipped stored-query registry, `GET /queries`, `POST /queries/{name}`, and the coarse `invoke_query` gate. +**Supersedes:** the MCP-transport portion of [rfc-001-queries-envelope-mcp.md](rfc-001-queries-envelope-mcp.md) (`/mcp/tools` + `/mcp/invoke`). See [Relationship to RFC-001](#relationship-to-rfc-001). +**Target release:** v0.8.x (phased — see Rollout) + +## Summary + +Add a first-class **MCP (Model Context Protocol) server surface to `omnigraph-server`**, exposed over **Streamable HTTP**, that projects the server's operations as MCP tools and resources for LLM clients (Claude Code/Desktop/web, Cursor, etc.). Two populations of tools share one projection path: + +1. **Built-in operational tools** — parity with the existing `@modernrelay/omnigraph-mcp` stdio package's **13 tools** (`health`, `snapshot`, `read`, `schema_get`, `branches_list`, `commits_list`, `commits_get`, `change`, `ingest`, `branches_create`, `branches_delete`, `branches_merge`, `schema_apply`) and its **2 resources** (`omnigraph://schema`, `omnigraph://branches`), plus a new server-scoped `graphs_list` tool and an `omnigraph://graphs` resource (multi-graph mode). +2. **Dynamic stored-query tools** — one MCP tool per `mcp.expose: true` entry in the `queries:` registry (MR-969 / #128), with parameters typed from the `.gq` declaration via the shipped `query_catalog_entry` / `param_descriptor` projection. + +Every tool is **authorized by the server's existing Cedar policy engine**. The MCP layer never implements its own authentication: it consumes an **already-resolved `ResolvedActor`** from the server's bearer middleware (`require_bearer_auth` today; the `TokenVerifier` seam when MR-956 lands), so the **same MCP endpoint serves on-prem (static or customer-OIDC tokens) and our cloud (WorkOS OAuth) by configuration only**. Cloud OAuth is an additive layer (RFC 9728 protected-resource metadata) that slots in with zero MCP changes. + +The end-state collapses two diverging tool implementations into one: the in-server MCP is the canonical, Cedar-gated, remotely-reachable surface; the stdio package becomes a thin stdio↔HTTP proxy (local on-ramp) over it. + +> **Key caveat, stated up front (see §5.9 below):** the headline "a token scoped via Cedar to a *specific set* of stored queries" requires **per-query `invoke_query` scope**, which is *designed* (rfc-001) but **not yet implemented** — the shipped action is coarse (any stored query on the graph, or none). Per-actor Cedar curation works today for *built-in vs ad-hoc vs admin* tools and for *stored-vs-ad-hoc*; sub-selecting individual stored queries per actor is gated on a prerequisite (PR 0b). Until then, stored-query curation is graph-level (registry membership + `mcp.expose`). + +## Relationship to RFC-001 + +[rfc-001-queries-envelope-mcp.md](rfc-001-queries-envelope-mcp.md) (MR-656 / MR-976 / MR-969) is the parent design for stored queries + the response envelope + MCP. This RFC is the **detailed MCP-transport design** that #128 left for a follow-up, and it **revises rfc-001 in three places where the shipped code or the MCP wire protocol diverged from rfc-001's sketch**: + +1. **Transport shape.** rfc-001 sketched `GET /mcp/tools` + `POST /mcp/invoke` (a bespoke REST pair). **That is not the MCP wire protocol — real MCP clients cannot connect to it.** This RFC implements actual MCP JSON-RPC over Streamable HTTP and reuses `query_catalog_entry` as a *projection source*, not a parallel surface. (rfc-001's own Open Question already leaned toward Streamable HTTP.) +2. **Exposure config.** rfc-001 specified inline `.gq` pragmas (`@mcp(expose=…)`, default `expose=false`). **#128 shipped a different mechanism:** YAML `queries..mcp.expose` in `omnigraph.yaml`, **default `true`** (declaring a query in the manifest *is* the opt-in). This RFC builds on the shipped YAML form; the `.gq`-pragma design in rfc-001 is superseded for exposure. +3. **Schema introspection.** rfc-001 lists "Schema introspection through MCP" as a **non-goal** ("agents see types through declared return shapes"). This RFC **revises that**: the operational-parity tools include `schema_get` and `omnigraph://schema` — *because the shipped stdio package already exposes both*. The non-goal is achieved by *policy*, not omission: `schema_get`/`omnigraph://schema` are Cedar-gated by `Read`, and the recommended locked-down agent policy denies `Read`, so a curated agent still never sees the schema. (rfc-001's intent is preserved; the mechanism moves from "don't build it" to "build it, gate it.") + +Everything else in rfc-001 (two-paths-one-engine, per-query `invoke_query` *as the intended scope*, the response envelope, multi-graph per-graph endpoints) this RFC consumes unchanged. + +> **Numbering note:** the `TokenVerifier`/WorkOS auth design is referred to in code (`crates/omnigraph-server/src/identity.rs`) as "RFC 0001," which is a *different* document from this repo's `docs/dev/rfc-001-queries-envelope-mcp.md`. To avoid the collision this RFC cites the auth substrate as **MR-956** throughout, never "RFC 0001." + +## Reconciliation with shipped code (verified against `ragnorc/stored-queries-mcp` HEAD) + +Verified against `crates/omnigraph-server/src/{lib.rs,api.rs}` and `crates/omnigraph-policy/src/lib.rs` at the current branch head (not the #128 PR body, and not `api.rs` alone): + +- ✅ `GET /queries` returns the `mcp.expose == true` subset as `QueriesCatalogOutput { queries: [QueryCatalogEntry] }`, each with typed `ParamDescriptor`s, `tool_name`, `description`, `instruction`, and a `mutation` flag. **MCP-ready projection, but exposed as bespoke REST/JSON — not the MCP wire protocol.** +- ✅ `POST /queries/{name}` route exists (`server_invoke_query`, `lib.rs`). +- ✅ `query_catalog_entry()` / `param_descriptor()` with an exhaustive `ScalarType → ParamKind` map (a new scalar is a compile error). +- ✅ `InvokeQuery` Cedar action defined in `omnigraph-policy`. +- ✅ **`InvokeQuery` IS enforced** at `POST /queries/{name}`: `server_invoke_query` calls `authorize(PolicyAction::InvokeQuery)` and **masks a denial to a 404 identical to "unknown query"** so the catalog isn't probeable (the denial-masking the previous draft of this RFC reported as missing is shipped — it lives in `lib.rs`, not `api.rs`). The stored-mutation path is already double-gated: `InvokeQuery` outer, then `Change` inside `run_mutate`. +- ✅ **Reuse path exists:** `run_query` / `run_mutate` are already decoupled from their HTTP request bodies and take registry-supplied `(source, name, params, branch/snapshot)`. MCP `tools/call` for both stored and ad-hoc tools delegates to these — no new business logic. +- ❌ **Per-query (`invoke_query[name]`) scope is NOT implemented.** `PolicyRequest` carries only `{action, branch, target_branch}` — **no query-name dimension** — and the action is documented coarse ("permits *any* stored query on the graph"). rfc-001 *designed* per-name scope; it is unbuilt. This RFC's per-query Cedar filtering (§5.4) and recommended agent policy (§5.9) depend on it → tracked as **PR 0b**. +- ❌ No MCP protocol surface (`initialize`/`tools/list`/`tools/call`, JSON-RPC, transport). +- ❌ No `TokenVerifier` trait yet — `require_bearer_auth` resolves a `ResolvedActor` inline (static-hash). The trait/`OidcJwtVerifier` are MR-956 (draft). The MCP layer's only requirement — *consume `ResolvedActor`* — is satisfiable today. + +Stack (verified `Cargo.toml`): Axum + utoipa (OpenAPI) + `omnigraph-policy` (Cedar) + `futures` + `tokio`. **No MCP crate present.** `edition = "2024"`. + +## Motivation + +- **One curated, safe, remotely-reachable tool surface.** MR-969's thesis: hand an LLM a token Cedar-scoped to a set of tools and it sees exactly those typed tools — cannot construct ad-hoc queries it isn't permitted, cannot read the schema it isn't permitted, cannot reach other graphs. Today the only MCP is the stdio package: local-only, full surface, ungated. +- **Parity, so the in-server MCP can be the single implementation.** Operators/agents already depend on the operational tools. Supporting them server-side behind one Cedar gate lets the stdio package degrade to a proxy and removes two diverging tool sets. +- **On-prem and cloud from one endpoint.** A managed cloud (WorkOS OAuth) and an on-prem/air-gapped deploy (static or customer-OIDC tokens) must serve the same MCP without forks or MCP-specific auth. +- **Foundation for the agent on-ramp (MR-974).** `omnigraph mcp install --agent ` needs a decided transport + a stable endpoint. + +## Goals + +- Project built-in tools + stored queries as MCP tools through **one** registry abstraction. +- `tools/list` and the callable set are **identical for argument-independent authorization**, both driven by Cedar (see §5.4 for the branch-scoped caveat). +- The MCP layer is **auth-method-agnostic**: it consumes `ResolvedActor`, never a raw token, never branches on how auth happened. +- The same endpoint works on-prem (static/OIDC) and cloud (WorkOS OAuth), switched by config; cloud OAuth is additive (RFC 9728). +- No new business logic: MCP tools delegate to the same `run_query`/`run_mutate`/branch/schema functions the HTTP routes call. +- Behaviour-neutral when unused: no MCP traffic = no change. + +## Non-Goals + +- **Building/hosting an OAuth authorization server.** The server is a Resource Server; WorkOS AuthKit+Connect is the AS (MR-956). The MCP endpoint validates tokens, never issues them, never holds client secrets. +- **OAuth/WorkOS implementation itself** — MR-956's work. This RFC leaves a clean RFC-9728 hook and consumes `ResolvedActor`. +- **MCP prompts, elicitation, `tools/list_changed`, resource subscriptions, server-initiated messages.** None needed → enables a stateless POST-only transport (§5.6). +- **stdio transport inside the server.** stdio stays in the TS package (now a proxy). +- **Cross-graph tool listing.** Per-graph catalogs only (MR-969 + RFC-002 non-goal). +- **Hot reload of the query registry.** Restart-only (MR-969). + +## Background + +`omnigraph-server` (Axum) already implements every operation this RFC exposes as an authenticated HTTP route; each authorizes via a `PolicyAction` against the Cedar policy for a server-resolved actor and calls into the engine. The existing stdio MCP package is a *client* of these routes (it owns no business logic). MR-956 will introduce a `TokenVerifier` trait (`StaticHashTokenVerifier` today inline, `OidcJwtVerifier` for OIDC/WorkOS) producing the `ResolvedActor { actor_id, tenant_id: Option, scopes: Vec, source }` that already exists in `identity.rs` and is consumed by Cedar — token *validation* is offline (cached JWKS), so on-prem/air-gapped has no request-path dependency on the cloud. + +## Design + +### 5.1 One tool model: a `McpTool` trait, two populators + +Both built-in and stored-query tools implement one trait so `tools/list` / `tools/call` never special-case: + +```rust +trait McpTool: Send + Sync { + fn name(&self) -> &str; // MCP tool id (stable) + fn title(&self) -> Option<&str>; + fn description(&self) -> &str; + fn input_schema(&self) -> serde_json::Value; // JSON Schema (draft 2020-12) + fn annotations(&self) -> ToolAnnotations; // readOnlyHint / destructiveHint / idempotentHint + /// The Cedar request(s) this call requires, given parsed args. Used BOTH at + /// list-time (dry-run filter, default args) and call-time (enforce, real args). + fn authorization(&self, args: &ToolArgs) -> Vec; + async fn call(&self, ctx: &GraphCtx, args: ToolArgs) -> Result; +} +``` + +- **Built-ins**: ~14 static impls, each delegating to the *same* function its HTTP route calls (`run_query`, `run_mutate`, branch ops, `apply_schema_as`, …). `input_schema` authored once (or derived from each route's existing `utoipa`/`ToSchema` DTO). +- **Stored queries**: generated `McpTool` instances, one per `mcp.expose` entry; `input_schema` from `param_descriptor` (§5.3); `authorization` → `InvokeQuery` (coarse today; `InvokeQuery{name}` after PR 0b) then the inner `Read`/`Change`. + +`ToolRegistry` for a graph = the static built-ins + the dynamic stored-query tools resolved from that graph's `GraphHandle` registry. + +### 5.2 Tool catalog (parity) and Cedar mapping + +Each built-in **reuses the exact `PolicyAction` its HTTP route already enforces** — verified against the handlers in `lib.rs`, not invented: + +| MCP tool | Scope | Read/Mutate | Cedar action (verified from route) | +|---|---|---|---| +| `health` | server | read | none (liveness/version) | +| `graphs_list` *(new)* | server | read | `GraphList` | +| `snapshot` | graph | read | `Read` | +| `schema_get` | graph | read | `Read` | +| `branches_list` | graph | read | `Read` | +| `commits_list`, `commits_get` | graph | read | `Read` | +| `read` (ad-hoc `.gq`) / `query` *(alias)* | graph | read | `Read` | +| `change` (ad-hoc `.gq`) / `mutate` *(alias)* | graph | mutate | `Change` | +| `ingest` (NDJSON) | graph | mutate | `Change` (+ `BranchCreate` when forking a new branch) | +| `branches_create` | graph | mutate | `BranchCreate` | +| `branches_delete` | graph | mutate | `BranchDelete` | +| `branches_merge` | graph | mutate | `BranchMerge` | +| `schema_apply` (`allow_data_loss`) | graph | mutate | `SchemaApply` | +| **stored query** (`find_user`, …) | graph | inferred | `InvokeQuery` (coarse; `InvokeQuery{name}` after PR 0b) + inner `Read`/`Change` | + +There is **no `Ingest` and no separate `snapshot`/`Export` action** — `ingest` enforces `Change`, `snapshot` enforces `Read`. (`Export` exists but maps to the `/export` route, which this RFC does not expose as a tool.) + +**Tool id parity vs. canonicalization.** The shipped stdio package uses tool ids **`read`/`change`** (and calls the deprecated `/read`,`/change` routes). The server HTTP surface canonicalized to `/query`,`/mutate` with `/read`,`/change` deprecated (MR-656). To keep existing package clients working *and* align with the server, the MCP exposes **`query`/`mutate` as canonical with `read`/`change` retained as deprecated-but-live aliases** (both dispatch to the same handler). Open Q7 asks whether to drop the aliases later. + +Resources (§5.5): `omnigraph://schema`, `omnigraph://branches` (parity), plus `omnigraph://graphs` *(new)* — each gated by the same action as its list/get route (`Read`, `Read`, `GraphList`). + +### 5.3 `ParamDescriptor → JSON Schema` (stored-query tools) + +| `ParamKind` | JSON Schema | Notes | +|---|---|---| +| String | `{"type":"string"}` | | +| Bool | `{"type":"boolean"}` | | +| Int (i32/u32) | `{"type":"integer"}` | | +| BigInt (i64/u64) | `{"type":"string","pattern":"^-?\\d+$"}` | JSON numbers lose precision >2⁵³ → string (matches the shipped `api.rs` rationale). (Open Q1) | +| Float (f32/f64) | `{"type":"number"}` | | +| Date | `{"type":"string","format":"date"}` | | +| DateTime | `{"type":"string","format":"date-time"}` | | +| Blob | `{"type":"string","contentEncoding":"base64"}` | | +| Vector | `{"type":"array","items":{"type":"number"},"minItems":dim,"maxItems":dim}` | uses `vector_dim` | +| List | `{"type":"array","items":}` | scalar items only (grammar guarantees) | + +`nullable == false` → param is in `required`. Annotations: `mutation` → `{readOnlyHint:false, destructiveHint:true}`; else `{readOnlyHint:true}`. `description` → tool description; `instruction` → appended to description (or `_meta`). (The shipped `check()` already warns when an `mcp.expose` query declares a `Vector` param an LLM can't supply.) + +For built-in tools the schema is hand-authored from the route DTO; e.g. `query` → `{source: string, branch?: string, params?: object}`; `schema_apply` → `{schema: string, allow_data_loss?: boolean}`; `ingest` → `{ndjson: string, mode?: "merge"|"append"|"overwrite", branch?: string}`. + +### 5.4 `tools/list` (Cedar-filtered) and `tools/call` (dispatch + masking) + +- **`tools/list`**: build the `ToolRegistry`; for each tool evaluate `authorization(default_args)` against the actor's Cedar policy; **emit only tools that authorize**. Authz decisions memoized per request. Stored-query tools additionally require `mcp.expose: true`. + - **Exactness caveat (R7 is conditional):** the listed set equals the callable set **only for tools whose authorization is argument-independent** (`health`, `graphs_list`, `snapshot`, `schema_get`, `branches_list`, `commits_*`, ad-hoc `query`/`mutate`, and stored queries under the *coarse* action). For **branch-scoped tools** (`branches_create`/`merge` with `target_branch_scope`, and any branch-scoped `Read`/`Change` rule), list-time uses `default_args` (e.g. branch `main`) and cannot know the real target, so the listed set is a *best-effort approximation* of callability — a call may still be denied (or, rarely, a hidden tool would have been allowed). `tools/call` is always the authoritative gate. The contract is: **list never shows a tool the actor can't ever call; for branch-scoped tools it may show one the actor can call only on some branches.** +- **`tools/call`**: resolve `name` → `McpTool` (masked-404 if unknown *or* `mcp.expose:false`); parse+validate args against `input_schema`; enforce `authorization(args)` (mutations stay double-gated: `InvokeQuery` then `Change`); on success `call`. **Denial masking** lives in one place (the dispatcher): an authz denial is returned identically to "unknown tool" (§5.10), reusing the same deny≡missing principle already shipped at `POST /queries/{name}`. + +### 5.5 Resources + +Advertise `resources` capability (`subscribe:false, listChanged:false`). `resources/list` → the URIs the actor may read; `resources/read` → schema `.pg` text / branches JSON / (multi-graph) graphs JSON, each gated by the corresponding action (`Read`, `Read`, `GraphList`). A locked-down agent denied `Read` simply never sees `omnigraph://schema` or `omnigraph://branches` — this is how rfc-001's "agents don't introspect schema" intent is met *by policy* (§Relationship-to-RFC-001). + +### 5.6 Transport: Streamable HTTP, stateless, POST-only + +- **Streamable HTTP** (MCP's current standard; we're already an HTTP server). One endpoint per scope (§5.7). +- Because the server emits **no** server-initiated messages, implement the **minimal conformant** shape: client `POST`s JSON-RPC, server replies `application/json`. **No SSE channel, no `Mcp-Session-Id`, stateless** — each request authenticated independently via the bearer middleware. Honour the `MCP-Protocol-Version` header. SSE/sessions can be added later if subscriptions land. +- **JSON-RPC methods:** `initialize` (advertise `{tools:{listChanged:false}, resources:{listChanged:false, subscribe:false}}` + serverInfo/version), `notifications/initialized` (no-op ack), `ping`, `tools/list`, `tools/call`, `resources/list`, `resources/read`. `prompts/list` returns empty if probed. +- **Library decision (Open Q2):** spike `rmcp` (official Rust MCP SDK) for conformance + Streamable-HTTP/Axum on edition 2024; **fall back to a hand-rolled ~150 LOC JSON-RPC-over-POST** (only the methods above) on friction. Given the tiny surface, hand-roll is an acceptable default. + +### 5.7 Endpoint routing (server- vs graph-scoped) + +- **Single-graph mode:** `POST /mcp` — graph tools + server tools (`health`, `graphs_list`). +- **Multi-graph mode (MR-668):** `POST /graphs/{graph_id}/mcp` — graph-scoped tools for that graph; plus a server-level `POST /mcp` exposing only server-scoped tools (`health`, `graphs_list`). A per-graph endpoint never lists another graph's tools (isolation, tested). Mirrors the shipped `/graphs/{graph_id}/…` cluster routing. (Open Q5: confirm naming + whether server tools also appear on the per-graph endpoint.) + +### 5.8 Modular / decoupled auth (the cross-cutting requirement) + +**Invariant (load-bearing, satisfiable today):** the MCP handler receives an **already-resolved `ResolvedActor`** and **branches on nothing** about how the token was verified. No token parsing, no method check, no OAuth inside the MCP module. Today that actor comes from `require_bearer_auth`; when MR-956 lands it comes from a `TokenVerifier` — the MCP code is identical either way. + +``` +request → [auth middleware: ResolvedActor] → [MCP route] → Cedar → McpTool +``` + +**Server side — auth is config, not code:** + +| Deployment | Verifier | MCP change | +|---|---|---| +| On-prem, static bearer | `require_bearer_auth` / `StaticHashTokenVerifier` | none | +| On-prem, customer IdP | `OidcJwtVerifier` → customer issuer (MR-956) | none | +| Our cloud | `OidcJwtVerifier` → WorkOS, `tenant_id = Some(org_id)` (MR-956) | none | + +Token validation is offline (cached JWKS) — on-prem/air-gapped keeps working with no request-path cloud dependency. The MCP endpoint never terminates OAuth and never holds a client secret (Resource Server only). + +**Cloud client negotiation — additive, no MCP changes:** when MR-956 lands, the server publishes RFC 9728 `/.well-known/oauth-protected-resource` and returns `WWW-Authenticate: Bearer ..., resource_metadata="..."` on 401. A compliant MCP client (Claude) then auto-negotiates: static bearer to an on-prem endpoint; on a cloud 401 it discovers the WorkOS AS and runs OAuth/PKCE itself — **same endpoint URL, zero client-side branching.** This RFC only requires that MCP routes flow through the standard 401 path so that hook can be added later without touching MCP. + +**Multi-user identity pass-through (cloud):** the *caller's* token (a WorkOS JWT, audience-bound per-tenant) must reach the server so Cedar enforces per-user/per-tenant policy — never a shared service token. The MCP endpoint validates it offline and maps `org_id → tenant_id`. This is why the **remote path is the in-server HTTP MCP that Claude connects to directly** (its token flows through), not a stdio bridge impersonating a user. + +**Client-side credential acquisition (CLI/SDK/proxy) — pluggable `CredentialSource`** (RFC-002 §5, MR-971), keyed by server name, so OAuth is a future *sibling key*, not a re-key: + +```yaml +servers: + onprem: { endpoint: https://og.internal:8080, auth: { token: { env: OG_TOKEN } } } + edge: { endpoint: https://og-edge, auth: { token: { command: [vault, read, -field=token, secret/og] } } } + cloud: { endpoint: https://api.omnigraph.cloud, auth: { oauth: { issuer: workos } } } # future sibling +``` + +Implicit chain when `auth:` omitted: `OMNIGRAPH_TOKEN_` → keychain `omnigraph:` → `[]` in `~/.omnigraph/credentials`; legacy `bearer_token_env` honoured. Secrets never inlined. + +### 5.9 Safety model — Cedar is the gate, default-deny is the floor + +With ad-hoc `query`/`mutate`/`schema_apply` present as tools, the **only** thing protecting an untrusted agent is the Cedar policy. Therefore: + +- **Default-deny when tokens are configured** (MR-723, shipped) is the floor — an actor with no grants sees an empty tool list. +- **What works today (coarse action):** a policy can hide all ad-hoc tools and admin tools per-actor (`deny Read, Change, SchemaApply, Branch*`) while allowing stored queries (`allow InvokeQuery`). That already reproduces "can't run ad-hoc, can't read schema, can only call stored queries" — the agent sees *every* exposed stored query plus nothing else. +- **What needs PR 0b (per-query scope):** selecting *which* stored queries an actor may call (`allow InvokeQuery [find_user, list_orders]`, deny the rest). The shipped `invoke_query` is coarse (all stored queries or none). Until PR 0b adds a query-name dimension to `PolicyRequest` + the Cedar schema (rfc-001's intended design), per-actor sub-selection of stored queries is **not expressible**; curation is graph-level (which `.gq` files are registered + `mcp.expose`). +- `schema_apply`, `branches_delete`, ad-hoc `mutate` require an explicit admin-tier grant; never in a default agent policy. +- (Open Q3) Optional `mcp.allow_adhoc` server switch defaulting **off** for the ad-hoc `query`/`mutate` tools — defence-in-depth independent of Cedar, and independent of PR 0b. + +### 5.10 Result shaping and error mapping + +- **Success:** `tools/call` returns `content: [{type:"text", text:}]` where `` is the route's existing output envelope (read rows / mutation summary, i.e. `ReadOutput` / `ChangeOutput`). (Open Q4: also emit `structuredContent` + `outputSchema` — defer; text-JSON for v1.) +- **Tool execution error** (bad params after schema validation, engine error): result with `isError:true` + a text content block. +- **Authorization denial / unknown tool / `mcp.expose:false`:** a single JSON-RPC error (`-32602`, message `"unknown tool"`) — identical for all three so policy isn't probeable (same principle as the shipped `POST /queries/{name}` 404 masking). +- **Auth failure** (bad/absent bearer): HTTP 401 from the middleware *before* MCP — carries `WWW-Authenticate` (the RFC 9728 hook), never masked as a tool error. (This is exactly the path the shipped `authorize`/`authorize_request` split preserves: operational failures keep their status; only *denials* are masked.) + +## Relationship to the `@modernrelay/omnigraph-mcp` stdio package + +Verified surface of the package (`omnigraph-ts`, pkg version `0.3.0`, `@modelcontextprotocol/sdk@^1.29.0`, **stdio only**): **13 tools** (`health`, `snapshot`, `read`, `schema_get`, `branches_list`, `commits_list`, `commits_get`, `change`, `ingest`, `branches_create`, `branches_delete`, `branches_merge`, `schema_apply`) and **2 resources** (`omnigraph://schema`, `omnigraph://branches`). It is a thin client over the SDK → HTTP routes and **forwards the caller's bearer verbatim** (no inspection). + +Once parity lands, **collapse to one implementation**: the in-server MCP is canonical (Cedar-gated, remote-capable, the path that becomes a Claude-web connector via MR-956). The stdio package degrades to a **thin stdio↔HTTP proxy** forwarding JSON-RPC (and the incoming `Authorization`) to `/mcp` — staying the local on-ramp for Claude Code/Desktop while sharing one tool set, one Cedar gate. Transition: keep the current independent stdio package on its `0.3.x`/`0.6.x` line; ship proxy mode in a later TS minor once the server endpoint is GA. (Note: the package is currently several minors behind the server — its vendored `spec/openapi.json` predates the stored-query routes — so it needs the standard re-sync regardless of MCP work.) + +## Testing + +- **Protocol conformance:** `initialize` handshake + advertised capabilities; `tools/list` shape; `tools/call` happy path; JSON-RPC error envelopes (`-32601` unknown method, `-32602` invalid params / unknown tool); `resources/list` + `resources/read`. +- **Cedar filtering (coarse, today):** an actor with `allow InvokeQuery` + `deny Read/Change` sees *all* exposed stored queries but **not** `query`/`mutate`/`schema_get`; `tools/call query` returns masked "unknown tool"; an admin sees the full catalog. +- **Cedar filtering (per-query, gated on PR 0b):** actor scoped to `InvokeQuery [find_user]` sees *only* `find_user`; `tools/call list_orders` masks. **This test ships with PR 0b**, not PR 1 — it cannot pass against the coarse action. +- **Parity per built-in:** each tool round-trips against the same expectations as its HTTP route (reuse route tests); `read`/`change` aliases dispatch identically to `query`/`mutate`. +- **Double-gating:** a stored mutation requires both `InvokeQuery` and `Change`; `schema_apply` requires `SchemaApply`. +- **`mcp.expose:false`:** absent from `GET /queries` and MCP `tools/list`; still service-callable by name through `POST /queries/{name}` when the actor has `invoke_query`, but not MCP-callable. +- **Schema generation:** table-driven over every `ParamKind` incl. nullable / list / vector(dim). +- **Branch-scoped list approximation:** assert the documented R7 caveat — a branch-scoped policy lists `branches_create`, and `tools/call` is the authoritative gate (a denied target still 403s/masks). +- **Multi-graph isolation:** `/graphs/a/mcp` never lists graph `b`'s tools; server `/mcp` exposes only server tools. +- **Auth decoupling:** the MCP suite is green under the current `require_bearer_auth` and under a mock OIDC `ResolvedActor` source — proving verifier-agnosticism. A 401 carries `WWW-Authenticate`. +- **OpenAPI:** the JSON-RPC endpoint is not REST — document only the envelope in utoipa (or exclude); keep `openapi.json` drift test green (`OMNIGRAPH_UPDATE_OPENAPI=1` to regenerate on intentional change). +- **Cross-repo smoke (optional):** point `@modelcontextprotocol/sdk` (TS) at the HTTP endpoint in an `omnigraph-ts` integration test. + +## Rollout — phased by risk + +- **PR 0a — extract the reusable invoke path (small).** The coarse `invoke_query` gate + 404 denial-masking are **already shipped** in `server_invoke_query`. Extract the read/mutate dispatch into `invoke_stored_query(handle, name, params, branch/snapshot, actor)` so MCP `tools/call` and the HTTP route share one path. No behaviour change. *(Replaces the previous draft's "PR 0 — wire the gate", which was already done.)* +- **PR 0b — per-query `invoke_query` scope (the safety prerequisite).** Add a query-name dimension to `PolicyRequest` + the Cedar schema (rfc-001's intended design), wire it at `POST /queries/{name}` and in the stored-query `McpTool::authorization`. Independently useful (the `allow InvokeQuery [find_user]` policy). **Gates the per-query Cedar-filtering test and §5.9's recommended agent policy.** +- **PR 1 — MCP transport + read-only parity + stored-query reads.** Endpoint(s), `initialize`/`tools/list`/`tools/call`/`resources/*`, the `McpTool` registry, Cedar-filtered listing, the read-only built-ins (`health`, `graphs_list`, `snapshot`, `read`/`query`, `schema_get`, `branches_list`, `commits_*`) + resources + stored-query *reads*. All auth-agnostic. +- **PR 2 — mutating parity + stored-query mutations.** `change`/`mutate`, `ingest`, `branches_create/delete/merge`, `schema_apply`, stored-query mutations + the `mcp.allow_adhoc` switch. +- **PR 3 — docs + agent on-ramp hook.** `docs/user/server.md` MCP section (incl. the recommended agent policy + the coarse-vs-per-query caveat), `openapi.json` sync, the `omnigraph mcp install` config target (MR-974), and the downstream `omnigraph-ts` re-sync/proxy follow-up. +- **Later (separate, MR-956):** RFC 9728 protected-resource metadata + WorkOS — slots in with zero MCP changes. +- **Later (TS minor):** stdio package → proxy mode. + +## Migration / backwards compatibility + +- **Additive.** No `queries:` and no MCP traffic → today's behaviour unchanged. New endpoints are new routes. +- **Cedar default-deny** (when tokens configured) means MCP exposes nothing until an actor is granted — safe by default. +- The stdio package keeps working unchanged; proxy mode is opt-in later. +- `openapi.json` only gains the documented MCP envelope; existing REST routes untouched. + +## Open Questions + +1. **BigInt/u64 as JSON string** (recommended, precision-safe) vs number. +2. **`rmcp` vs hand-rolled** JSON-RPC (spike `rmcp` on edition 2024; default to hand-roll on friction). +3. **Default-off `mcp.allow_adhoc`** for ad-hoc `query`/`mutate` (recommended) vs always-on + Cedar-only. +4. **`structuredContent` + `outputSchema`** now vs text-JSON v1 (recommend v1 text-JSON). +5. **Endpoint paths:** `/mcp` + `/graphs/{id}/mcp` — confirm naming and whether server-scoped tools also appear on the per-graph endpoint. +6. **Stateless POST-only** confirmed (no near-term server-initiated messages) — revisit only if subscriptions land. +7. **Legacy alias tools** (`read`/`change`): keep for client compat (the shipped package uses them), or drop and rely on `query`/`mutate`? +8. **PR 0b shape:** per-query scope as a Cedar *resource* (`StoredQuery::"find_user"`) vs a `query_name` *context attribute* + policy condition — affects how `allow InvokeQuery [list]` is authored. diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 0326e64..019e4ad 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -20,10 +20,11 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` sc | `run list \| show \| publish \| abort` | transactional run ops | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | +| `queries validate \| list` | operate on the server-side stored-query registry (the `queries:` block). `validate` type-checks every stored query against the live schema offline (opens the selected graph; exits non-zero on any breakage), catching schema drift without restarting the server; `list` prints the selected registry's query names, MCP exposure, and typed params. For per-graph registries, pass `--target ` or set `cli.graph`; with no graph selection, `list` shows only top-level `queries:`. Distinct from `lint`, which validates a single `.gq` file | | `optimize` | non-destructive Lance compaction | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | | `embed` | offline JSONL embedding pipeline | -| `policy validate \| test \| explain` | Cedar tooling | +| `policy validate \| test \| explain` | Cedar tooling. Selects `cli.graph`, else `server.graph`, else top-level `policy.file` | | `version` / `-v` | print `omnigraph 0.3.x` | ## `omnigraph.yaml` schema @@ -34,6 +35,13 @@ graphs: : uri: bearer_token_env: + queries: # per-graph stored-query registry (server-role; multi-graph mode) + : # key MUST equal the `query ` symbol inside the .gq + file: # relative to this config's directory + mcp: + expose: true # default true: listed in the MCP catalog (GET /queries); set false to hide (still HTTP-callable) + tool_name: # optional MCP tool-name override (defaults to ; + # must be unique across exposed queries) server: graph: bind: @@ -59,6 +67,8 @@ aliases: graph: branch: format: +queries: # top-level registry — applies only to a bare-URI (anonymous) graph; a graph served by name uses its `graphs..queries`. Mirrors top-level `policy`. + : { file: } # mcp.expose defaults to true policy: file: ./policy.yaml ``` diff --git a/docs/user/policy.md b/docs/user/policy.md index 749d3be..ec0d214 100644 --- a/docs/user/policy.md +++ b/docs/user/policy.md @@ -14,10 +14,11 @@ Per-graph actions (bind to `Omnigraph::Graph::""`): 6. `branch_delete` 7. `branch_merge` 8. `admin` — reserved for policy-management surfaces (hot reload, audit log, approvals). No call site today; see MR-724 for the reservation rationale. +9. `invoke_query` — gates invoking a server-side stored query (the `queries:` registry). Graph-scoped (like `admin`) — per-branch access is enforced by the inner `read` / `change` gate, so a rule that sets `branch_scope` on `invoke_query` is rejected. Coarse in this release: an `invoke_query` allow rule permits any stored query on the graph; a future, additive refinement adds an optional per-query-name scope without changing rules written against the coarse action. Enforced at `POST /queries/{name}` (see [server](server.md)). A stored *mutation* is double-gated: `invoke_query` to reach the tool, plus `change` for the write itself (the engine `_as` writers still enforce per the query body). Server-scoped action (v0.6.0+; binds to `Omnigraph::Server::"root"`): -9. `graph_list` — `GET /graphs` registry enumeration (multi-graph mode) +10. `graph_list` — `GET /graphs` registry enumeration (multi-graph mode) Server-scoped actions cannot use `branch_scope` or `target_branch_scope` — they operate on the registry, not on a graph's branches. A rule cannot mix server-scoped and per-graph actions; split into separate rules. (Runtime `graph_create` / `graph_delete` are reserved but not shipped in v0.6.0; operators add/remove graphs by editing `omnigraph.yaml` and restarting.) @@ -46,10 +47,15 @@ graphs: # no per-graph policy → no engine-layer Cedar enforcement on beta ``` -Top-level `policy.file` is single-graph / CLI-local policy only. Multi-graph -server startup rejects it because applying one graph policy to every configured -graph is ambiguous. Move per-graph rules to `graphs..policy.file` and -move `graph_list` rules to `server.policy.file`. +**Config follows graph identity, not server mode.** A graph served by **name** +(`--target ` or `server.graph`) uses its own `graphs..policy.file`, +exactly as in multi-graph mode. Top-level `policy.file` applies only to an +**anonymous** graph — one served by a bare `` with no `graphs:` entry. +Serving a **named** graph (single- or multi-graph mode) while top-level +`policy.file` (or `queries:`) is populated **refuses boot**, naming the block, +since the top-level value would otherwise be silently shadowed by the per-graph +block. Move per-graph rules to `graphs..policy.file` and `graph_list` +rules to `server.policy.file`. Each graph's HTTP request flows through its own per-graph policy. The management endpoint (`GET /graphs`) flows through the server-level policy. When `server.policy.file` is unset, `GET /graphs` is denied in every runtime state, including `--unauthenticated`; with bearer tokens configured, it returns 403 after admission control because `graph_list` is not a `read`-equivalent action. The operator must explicitly authorize via `server-policy.yaml` to expose `/graphs`. @@ -92,6 +98,10 @@ bearer token. ## CLI +Policy tooling resolves its graph like server single-mode policy: `cli.graph` +wins, otherwise `server.graph` is used, otherwise the top-level `policy.file` +is validated/tested/explained as the anonymous policy. + - `omnigraph policy validate` — parse + count actors, exit 1 on parse error. - `omnigraph policy test` — run cases in `policy.tests.yaml`, exit 1 on any expectation mismatch. - `omnigraph policy explain --actor … --action … [--branch …] [--target-branch …]` — show decision and matched rule. diff --git a/docs/user/server.md b/docs/user/server.md index 6f55e16..67b5afe 100644 --- a/docs/user/server.md +++ b/docs/user/server.md @@ -6,7 +6,9 @@ Axum 0.8 + tokio + utoipa-generated OpenAPI. **Two modes** (v0.6.0+): single-gra ### Single-graph mode (legacy) -`omnigraph-server ` or `omnigraph-server --target --config omnigraph.yaml`. Routes are flat — `/snapshot`, `/read`, `/branches`, etc. Behavior unchanged from v0.6.0. +`omnigraph-server ` or `omnigraph-server --target --config omnigraph.yaml`. Routes are flat — `/snapshot`, `/read`, `/branches`, etc. + +**Config follows graph identity.** A bare `` is an *anonymous* graph and uses the **top-level** `policy.file` / `queries:`. A graph chosen by **name** (`--target` / `server.graph`) uses its own `graphs..{policy.file, queries}` — the same block multi-graph mode uses. ⚠️ *Changed from v0.6.0, which always used top-level config in single mode: a named-graph config that puts `policy`/`queries` at top-level now **refuses boot** and points you at `graphs..…` (move the block there). Bare-`` single mode is unchanged.* ### Multi-graph mode (v0.6.0+) @@ -20,6 +22,10 @@ Mode inference (four-rule matrix): 4. `--config` + non-empty `graphs:` + no single-mode selector → **multi** 5. otherwise → error with migration hint +### Stored-query validation at startup + +If a graph declares a `queries:` registry (see [cli-reference](cli-reference.md)), the server **loads and type-checks every stored query against that graph's live schema at startup** and **refuses to boot** if any query references a type or property the schema lacks — the same fail-loud posture as a malformed policy file, so schema drift surfaces at the deploy boundary rather than at invocation. Two MCP-exposed queries claiming the same tool name is likewise a boot error. Non-blocking advisories (e.g. an MCP-exposed query with a vector parameter an agent cannot supply) are logged. Validate offline before deploying with `omnigraph queries validate`. Discover the exposed queries as a typed tool catalog with `GET /queries`, and invoke one over HTTP with `POST /queries/{name}` (both below). + ## Endpoint inventory Per-graph endpoints — same body shape across modes; URLs differ: @@ -34,6 +40,8 @@ Per-graph endpoints — same body shape across modes; URLs differ: | POST | `/export` | `/graphs/{id}/export` | bearer + `export` | NDJSON stream | `server_export` | | POST | `/mutate` | `/graphs/{id}/mutate` | bearer + `change` | mutation (canonical; `query`/`name`; accepts legacy `query_source`/`query_name` as serde aliases) | `server_mutate` | | POST | `/change` | `/graphs/{id}/change` | bearer + `change` | **deprecated** alias of `/mutate` (carries `Deprecation: true` + `Link: ; rel="successor-version"`) | `server_change` | +| GET | `/queries` | `/graphs/{id}/queries` | bearer + `read` | list the `mcp.expose` stored queries as a typed tool catalog | `server_list_queries` | +| POST | `/queries/{name}` | `/graphs/{id}/queries/{name}` | bearer + `invoke_query` (+ `change` for a stored mutation) | invoke a named query from the `queries:` registry; deny == 404 | `server_invoke_query` | | GET | `/schema` | `/graphs/{id}/schema` | bearer + `read` | get current `.pg` source | `server_schema_get` | | POST | `/schema/apply` | `/graphs/{id}/schema/apply` | bearer + `schema_apply` (target=`main`) | migrate | `server_schema_apply` | | POST | `/ingest` | `/graphs/{id}/ingest` | bearer + `branch_create` (if new) + `change` | bulk load | `server_ingest` (32 MB body limit) | @@ -50,6 +58,23 @@ Server-level management endpoints (v0.6.0+): |---|---|---|---|---| | GET | `/graphs` | bearer + `graph_list` on `Server::"root"` | list registered graphs | `server_graphs_list` (405 in single mode) | +### Stored-query catalog (`GET /queries`) + +List the graph's **`mcp.expose`** stored queries as a typed tool catalog — enough for a client (e.g. an MCP server) to register each as a tool without fetching `.gq` source. Each entry: `{ name, tool_name, description, instruction, mutation, params }`, where each param is `{ name, kind, item_kind?, vector_dim?, nullable }`. `kind` is one of `string | bool | int | bigint | float | date | datetime | blob | vector | list` (decomposed so a consumer maps it with a closed `switch`, never re-parsing GQ type spelling). `bigint` (I64/U64), `date`, `datetime`, and `blob` are carried as JSON **strings** — a 64-bit integer loses precision as a JSON number, dates are ISO strings, and a blob is a URI string. + +- **Read-gated** (works in default-deny mode). The catalog is **graph-wide** (branch-independent; `read` is authorized against `main`). +- **`mcp.expose` defaults to `true`** — declaring a query in `queries:` lists it; set `mcp: { expose: false }` to keep it HTTP/service-callable but hidden from the catalog. +- **Not Cedar-filtered per query (yet).** A caller with `read` but not `invoke_query` can *list* a query they can't *invoke* (which would 404). Closing that gap is future per-query authorization; for now the catalog is a discovery surface and `invoke_query` remains the invocation gate. + +### Stored-query invocation (`POST /queries/{name}`) + +Invoke a curated, server-side stored query by **name** — the source comes from the graph's `queries:` registry, so the client never sends `.gq`. The request body itself is optional; omit it for no-param queries, or send `{ "params": { … }, "branch": "main", "snapshot": null }`, where every field is optional and `params` keys match the query's declared parameters. The response is the **read envelope** (`ReadOutput`) for a stored read or the **mutation envelope** (`ChangeOutput`) for a stored mutation — serialized untagged, so the wire shape is identical to `/query` / `/mutate`. + +- **Gate:** `invoke_query` (per-graph, graph-scoped) at the boundary. A stored *mutation* is **double-gated** — it also passes the engine's `change` gate, so an actor with `invoke_query` but not `change` gets `403`. +- **Deny == unknown, for callers without `invoke_query`:** for a caller lacking the grant, an `invoke_query` denial and an unknown query name return the **same `404`** (identical body), so the catalog can't be probed. A caller that *holds* `invoke_query` may still get the inner gate's `403` for an existing query it can't `read`/`change` (the double-gate, above) — so existence is visible to grant-holders by design. +- **Requires an explicit policy grant when auth is on.** In default-deny mode (bearer tokens but no `policy.file`), only `read` is permitted, so *every* `/queries/{name}` call returns `404` until an `invoke_query` rule is configured. +- A stored mutation cannot target a `snapshot` (`400`); a parameter type error is a structured `400` naming the parameter. + ## Adding and removing graphs (multi mode) Runtime add/remove via API is **not** exposed in v0.6.0 — neither diff --git a/openapi.json b/openapi.json index d1fa337..08d39c4 100644 --- a/openapi.json +++ b/openapi.json @@ -829,6 +829,177 @@ ] } }, + "/queries": { + "get": { + "tags": [ + "queries" + ], + "summary": "List the graph's exposed stored queries as a typed tool catalog.", + "description": "Returns the `mcp.expose == true` subset of the `queries:` registry, each\nwith its MCP tool name, read/mutate flag, description/instruction, and\ntyped parameters — enough for a client to register them as tools without\nfetching `.gq` source. Read-gated; the catalog is graph-wide (branch\nindependent — `read` is authorized against `main`). **Not** Cedar-filtered\nper query yet, so it can list a query whose `invoke_query` the caller\nlacks (a known gap until per-query authorization lands).", + "operationId": "list_queries", + "responses": { + "200": { + "description": "Stored-query catalog (the mcp.expose subset, with typed params)", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/QueriesCatalogOutput" + } + } + } + }, + "401": { + "description": "Unauthorized", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + }, + "403": { + "description": "Forbidden", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + } + }, + "security": [ + { + "bearer_token": [] + } + ] + } + }, + "/queries/{name}": { + "post": { + "tags": [ + "queries" + ], + "summary": "Invoke a curated, server-side stored query by name.", + "description": "The query source comes from the graph's `queries:` registry, not the\nrequest body — callers send only runtime inputs (`params`, `branch`,\n`snapshot`). Gated by the `invoke_query` Cedar action at the boundary;\na stored *mutation* additionally passes the engine's `change` gate\n(double-gated). An actor **without** `invoke_query` cannot tell a denied\nquery from a missing one — both return the same 404, so the catalog\ncan't be probed without the grant. Once `invoke_query` is held, the\ninner `read`/`change` gate may surface a 403 for an existing query the\nactor can't run (the intended double-gate signal).", + "operationId": "invoke_query", + "parameters": [ + { + "name": "name", + "in": "path", + "description": "Stored query name (the registry key)", + "required": true, + "schema": { + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "oneOf": [ + { + "type": "null" + }, + { + "$ref": "#/components/schemas/InvokeStoredQueryRequest" + } + ] + } + } + } + }, + "responses": { + "200": { + "description": "Read envelope (ReadOutput) or mutation envelope (ChangeOutput), serialized untagged", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/InvokeStoredQueryResponse" + } + } + } + }, + "400": { + "description": "Bad request (param type error; snapshot on a stored mutation)", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + }, + "401": { + "description": "Unauthorized", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + }, + "403": { + "description": "Forbidden (the inner `change` gate for a stored mutation)", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + }, + "404": { + "description": "Unknown stored query, or `invoke_query` denied — indistinguishable to a caller without the grant", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + }, + "409": { + "description": "Merge conflict", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + }, + "429": { + "description": "Per-actor admission cap exceeded; honor `Retry-After` header", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + }, + "500": { + "description": "Policy evaluation error (a denial is reported as 404, not 500)", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + } + }, + "security": [ + { + "bearer_token": [] + } + ] + } + }, "/query": { "post": { "tags": [ @@ -1628,6 +1799,40 @@ } } }, + "InvokeStoredQueryRequest": { + "type": "object", + "description": "Body for `POST /queries/{name}` — invokes the server-side stored query\nnamed in the path. The query source and name come from the registry,\nnever the body; only the runtime inputs are supplied here.", + "properties": { + "branch": { + "type": [ + "string", + "null" + ], + "description": "Branch to run against. Defaults to `main`; for a stored mutation the\nwrite targets this branch." + }, + "params": { + "description": "JSON object whose keys match the stored query's declared parameters." + }, + "snapshot": { + "type": [ + "string", + "null" + ], + "description": "Snapshot id to read from (read queries only — rejected for a stored\nmutation). Mutually exclusive with `branch`." + } + } + }, + "InvokeStoredQueryResponse": { + "oneOf": [ + { + "$ref": "#/components/schemas/ReadOutput" + }, + { + "$ref": "#/components/schemas/ChangeOutput" + } + ], + "description": "Response for `POST /queries/{name}`: the read envelope for a stored\nread, or the mutation envelope for a stored mutation. Serialized\n**untagged**, so the wire shape is exactly [`ReadOutput`] or\n[`ChangeOutput`] — classification follows the stored query, not a\nwrapper field." + }, "LoadMode": { "type": "string", "description": "Shadow enum for documenting [`LoadMode`] in the OpenAPI schema.", @@ -1698,6 +1903,120 @@ } } }, + "ParamDescriptor": { + "type": "object", + "description": "One declared parameter of a stored query, projected for the catalog.", + "required": [ + "name", + "kind", + "nullable" + ], + "properties": { + "item_kind": { + "oneOf": [ + { + "type": "null" + }, + { + "$ref": "#/components/schemas/ParamKind", + "description": "Element kind when `kind == list` (always a scalar — the grammar\nforbids lists of vectors or nested lists)." + } + ] + }, + "kind": { + "$ref": "#/components/schemas/ParamKind" + }, + "name": { + "type": "string" + }, + "nullable": { + "type": "boolean", + "description": "`false` → the caller must supply it; `true` → optional." + }, + "vector_dim": { + "type": [ + "integer", + "null" + ], + "format": "int32", + "description": "Dimension when `kind == vector`.", + "minimum": 0 + } + } + }, + "ParamKind": { + "type": "string", + "description": "The kind of a stored-query parameter, decomposed so a client (e.g. an\nMCP server) can build a typed input schema with a closed `match` and\nnever re-parse omnigraph's type spelling. `bigint`/`date`/`datetime`/\n`blob` are carried as JSON strings on the wire: a 64-bit integer past\n2^53 loses precision as a JSON number, and Date/DateTime are ISO\nstrings, Blob a blob-URI string.", + "enum": [ + "string", + "bool", + "int", + "bigint", + "float", + "date", + "datetime", + "blob", + "vector", + "list" + ] + }, + "QueriesCatalogOutput": { + "type": "object", + "description": "Response for `GET /queries`: the `mcp.expose` subset of a graph's\nstored-query registry, each with typed parameters.", + "required": [ + "queries" + ], + "properties": { + "queries": { + "type": "array", + "items": { + "$ref": "#/components/schemas/QueryCatalogEntry" + } + } + } + }, + "QueryCatalogEntry": { + "type": "object", + "description": "One entry in the stored-query catalog (`GET /queries`).", + "required": [ + "name", + "tool_name", + "mutation", + "params" + ], + "properties": { + "description": { + "type": [ + "string", + "null" + ] + }, + "instruction": { + "type": [ + "string", + "null" + ] + }, + "mutation": { + "type": "boolean", + "description": "`true` for a stored mutation → an MCP read-only hint of `false`." + }, + "name": { + "type": "string", + "description": "Registry key / invoke path segment (`POST /queries/{name}`)." + }, + "params": { + "type": "array", + "items": { + "$ref": "#/components/schemas/ParamDescriptor" + } + }, + "tool_name": { + "type": "string", + "description": "MCP tool id (the `tool_name` override, else `name`)." + } + } + }, "QueryRequest": { "type": "object", "description": "Inline read-query request for `POST /query`.\n\nFriendlier-named alternative to [`ReadRequest`] for ad-hoc reads and\nAI-agent integration. Mutations are rejected with 400 — use `POST\n/mutate` (or its deprecated alias `POST /change`) for write queries.\nField names are deliberately short (`query`, `name`) to match the GQ\nkeyword and the CLI `-e` flag.", From d54bccb9401551415d426c45714c20caee8ebe86 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Tue, 2 Jun 2026 17:12:00 +0200 Subject: [PATCH 015/165] fix(optimize): skip blob-bearing tables to avoid Lance compaction crash (#138) * test(optimize): pin Lance blob-column compaction failure as a surface guard Lance compact_files mis-decodes blob-v2 columns under its forced BlobHandling::AllBinary read ("more fields in the schema than provided column indices"), failing even a pristine uniform-V2_2 multi-fragment blob table; reads use descriptor handling and are unaffected. Guard 10 reproduces this and is self-retiring: it turns red on the Lance bump that fixes the bug, forcing LANCE_SUPPORTS_BLOB_COMPACTION to flip. * fix(optimize): skip blob-bearing tables instead of crashing compaction omnigraph optimize aborted the whole sweep when any node/edge table had a Blob property: Lance compact_files cannot decode blob-v2 columns under AllBinary (the column-index error pinned by the surface guard). Skip blob-bearing tables behind a LANCE_SUPPORTS_BLOB_COMPACTION gate and report them via TableOptimizeStats.skipped / SkipReason (surfaced in the CLI and a tracing::warn) instead of erroring, which also isolates the failure so the other tables still compact. Reads/writes are unaffected; only fragment/space reclamation on blob tables is deferred until the upstream Lance fix. Adds a maintenance.rs regression test (validated red with the column-index symptom before the fix, green after), a concise v0.6.1 release note, and updates docs (maintenance, cli-reference, AGENTS capability matrix, invariants Known Gaps, lance.md audit, constants). * refactor(optimize): make TableOptimizeStats and SkipReason non_exhaustive Both are returned result types, never built by callers, so #[non_exhaustive] makes this the last field/variant addition that can break downstream literal construction and keeps future ones non-breaking (review feedback on the public-field addition). The v0.6.1 Compatibility Notes call out the source-level change. Also drops the now-stale "RED today / GREEN after the fix lands" narration in the optimize_skips_blob_table_and_reports_skip test (historical regression context now that the fix is in this branch), and folds in the expanded v0.6.1 release note. * chore(release): bump workspace to v0.6.1 Coherent version bump to accompany the v0.6.1 release note: all five crate manifests + path-dependency constraints, Cargo.lock, the AGENTS.md surveyed-version line, and openapi.json info.version move 0.6.0 -> 0.6.1. Matches the established release pattern (#118 landed the v0.6.0 note + bump together) and resolves the Codex/Devin review flag that a v0.6.1 note without a bump leaves CARGO_PKG_VERSION reporting 0.6.0 and mixed package versions. --- AGENTS.md | 4 +- Cargo.lock | 10 +- crates/omnigraph-cli/Cargo.toml | 10 +- crates/omnigraph-cli/src/main.rs | 9 +- crates/omnigraph-compiler/Cargo.toml | 2 +- crates/omnigraph-policy/Cargo.toml | 2 +- crates/omnigraph-server/Cargo.toml | 8 +- crates/omnigraph/Cargo.toml | 8 +- crates/omnigraph/src/db/mod.rs | 2 +- crates/omnigraph/src/db/omnigraph.rs | 2 +- crates/omnigraph/src/db/omnigraph/optimize.rs | 129 ++++++++++++++++-- .../omnigraph/tests/lance_surface_guards.rs | 85 ++++++++++++ crates/omnigraph/tests/maintenance.rs | 93 ++++++++++++- docs/dev/invariants.md | 9 ++ docs/dev/lance.md | 3 +- docs/releases/v0.6.1.md | 26 ++++ docs/user/cli-reference.md | 2 +- docs/user/constants.md | 1 + docs/user/maintenance.md | 3 +- openapi.json | 2 +- 20 files changed, 363 insertions(+), 47 deletions(-) create mode 100644 docs/releases/v0.6.1.md diff --git a/AGENTS.md b/AGENTS.md index 68de6b8..b876749 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -16,7 +16,7 @@ Tools that support `@`-imports (Claude Code) auto-include all three files via th `CLAUDE.md` is a symlink to this file — there is exactly one source of truth. Edit `AGENTS.md`. -**Version surveyed:** 0.6.0 +**Version surveyed:** 0.6.1 **Workspace crates:** `omnigraph-compiler`, `omnigraph` (engine), `omnigraph-policy`, `omnigraph-cli`, `omnigraph-server` **Storage substrate:** Lance 6.x (columnar, versioned, branchable) **License:** MIT @@ -237,7 +237,7 @@ omnigraph policy explain --actor act-alice --action change --branch main | Per-dataset versioning + time travel | ✅ | `snapshot_at_version`, `entity_at`, snapshot-pinned reads across many tables | | Per-dataset branches | ✅ | **Graph-level** branches (atomic across all sub-tables), lazy fork, system branch filtering | | Atomic single-dataset commits | ✅ | **Multi-table publish via three layers**, NOT a single Lance primitive: (1) per-table Lance `commit_staged` for the data write, (2) `__manifest` row-level CAS via `ManifestBatchPublisher` for cross-table ordering, (3) the open-time recovery sweep for the residual gap between (1) and (2). All three layers ship; the four migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`) write a `__recovery/{ulid}.json` sidecar before Phase B and delete it after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the sweep in `db/manifest/recovery.rs`: classify, decide all-or-nothing per sidecar, roll forward via single `ManifestBatchPublisher::publish` or roll back via `Dataset::restore`, and record an audit row in `_graph_commit_recoveries.lance` (queryable via `omnigraph commit list --filter actor=omnigraph:recovery`). Continuous in-process recovery (no restart needed between Phase B failure and recovery) is the goal of a future background reconciler. Engine writes route through a sealed `TableStorage` trait exposing `stage_*` + `commit_staged` as the canonical staged-write surface; documented inline-commit residuals (`delete_where`, `create_vector_index`, plus legacy `append_batch` / `merge_insert_batches` / `overwrite_batch` / `create_*_index`) remain on the trait until upstream Lance ships a public two-phase API ([#6658](https://github.com/lance-format/lance/issues/6658), [#6666](https://github.com/lance-format/lance/issues/6666)) and the migration of every call site completes. | -| Compaction (`compact_files`) | ✅ | `omnigraph optimize` orchestrates over all node/edge tables, bounded concurrency | +| Compaction (`compact_files`) | ✅ | `omnigraph optimize` orchestrates over all node/edge tables, bounded concurrency; **skips blob-bearing tables** (reported via `TableOptimizeStats.skipped`, not silent), gated on `LANCE_SUPPORTS_BLOB_COMPACTION` until the upstream blob-v2 compaction-decode bug is fixed (see [docs/dev/invariants.md](docs/dev/invariants.md) Known Gaps) | | Cleanup (`cleanup_old_versions`) | ✅ | `omnigraph cleanup` with `--keep` / `--older-than` policy | | BTREE / inverted (FTS) / vector indexes | ✅ | `ensure_indices` builds them on every relevant column; idempotent; lazy across branches | | `merge_insert` upsert | ✅ | `LoadMode::Merge`, mutation `update`/`insert`/`delete` lowering | diff --git a/Cargo.lock b/Cargo.lock index a3d6d62..3223b9c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4543,7 +4543,7 @@ dependencies = [ [[package]] name = "omnigraph-cli" -version = "0.6.0" +version = "0.6.1" dependencies = [ "assert_cmd", "clap", @@ -4565,7 +4565,7 @@ dependencies = [ [[package]] name = "omnigraph-compiler" -version = "0.6.0" +version = "0.6.1" dependencies = [ "ahash", "arrow-array", @@ -4586,7 +4586,7 @@ dependencies = [ [[package]] name = "omnigraph-engine" -version = "0.6.0" +version = "0.6.1" dependencies = [ "arc-swap", "arrow-array", @@ -4627,7 +4627,7 @@ dependencies = [ [[package]] name = "omnigraph-policy" -version = "0.6.0" +version = "0.6.1" dependencies = [ "cedar-policy", "clap", @@ -4640,7 +4640,7 @@ dependencies = [ [[package]] name = "omnigraph-server" -version = "0.6.0" +version = "0.6.1" dependencies = [ "arc-swap", "async-trait", diff --git a/crates/omnigraph-cli/Cargo.toml b/crates/omnigraph-cli/Cargo.toml index 0d35ed8..641068e 100644 --- a/crates/omnigraph-cli/Cargo.toml +++ b/crates/omnigraph-cli/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-cli" -version = "0.6.0" +version = "0.6.1" edition = "2024" description = "CLI for the Omnigraph graph database." license = "MIT" @@ -13,10 +13,10 @@ name = "omnigraph" path = "src/main.rs" [dependencies] -omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.0" } -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.0" } -omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.0" } -omnigraph-server = { path = "../omnigraph-server", version = "0.6.0" } +omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.1" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.1" } +omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.1" } +omnigraph-server = { path = "../omnigraph-server", version = "0.6.1" } clap = { workspace = true } color-eyre = { workspace = true } serde = { workspace = true } diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 879f070..29b55c4 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -3011,18 +3011,19 @@ async fn main() -> Result<()> { "fragments_removed": s.fragments_removed, "fragments_added": s.fragments_added, "committed": s.committed, + "skipped": s.skipped.map(|r| r.as_str()), })).collect::>(), }); print_json(&value)?; } else { println!("optimize {} — {} tables", uri, stats.len()); for s in &stats { - if s.committed { + if let Some(reason) = s.skipped { + println!(" {:<40} skipped ({reason})", s.table_key); + } else if s.committed { println!( " {:<40} frags {} → {} ✓", - s.table_key, - s.fragments_removed + s.fragments_added - s.fragments_added, - s.fragments_added + s.table_key, s.fragments_removed, s.fragments_added ); } else { println!(" {:<40} no-op", s.table_key); diff --git a/crates/omnigraph-compiler/Cargo.toml b/crates/omnigraph-compiler/Cargo.toml index 229b862..545db83 100644 --- a/crates/omnigraph-compiler/Cargo.toml +++ b/crates/omnigraph-compiler/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-compiler" -version = "0.6.0" +version = "0.6.1" edition = "2024" description = "Schema/query compiler for Omnigraph. Zero Lance dependency." license = "MIT" diff --git a/crates/omnigraph-policy/Cargo.toml b/crates/omnigraph-policy/Cargo.toml index dacda35..3d14fc5 100644 --- a/crates/omnigraph-policy/Cargo.toml +++ b/crates/omnigraph-policy/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-policy" -version = "0.6.0" +version = "0.6.1" edition = "2024" description = "Policy / authorization layer for Omnigraph — Cedar-backed PolicyEngine, PolicyChecker trait, ResourceScope enum." license = "MIT" diff --git a/crates/omnigraph-server/Cargo.toml b/crates/omnigraph-server/Cargo.toml index e9a0e46..5994aa1 100644 --- a/crates/omnigraph-server/Cargo.toml +++ b/crates/omnigraph-server/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-server" -version = "0.6.0" +version = "0.6.1" edition = "2024" description = "HTTP server for the Omnigraph graph database." license = "MIT" @@ -19,9 +19,9 @@ default = [] aws = ["dep:aws-config", "dep:aws-sdk-secretsmanager"] [dependencies] -omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.0" } -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.0" } -omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.0" } +omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.1" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.1" } +omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.1" } axum = { workspace = true } clap = { workspace = true } color-eyre = { workspace = true } diff --git a/crates/omnigraph/Cargo.toml b/crates/omnigraph/Cargo.toml index 1fa3436..70f51d8 100644 --- a/crates/omnigraph/Cargo.toml +++ b/crates/omnigraph/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-engine" -version = "0.6.0" +version = "0.6.1" edition = "2024" description = "Runtime engine for the Omnigraph graph database." license = "MIT" @@ -16,8 +16,8 @@ default = [] failpoints = ["dep:fail", "fail/failpoints"] [dependencies] -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.0" } -omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.0" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.1" } +omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.1" } lance = { workspace = true } lance-datafusion = { workspace = true } datafusion = { workspace = true } @@ -51,7 +51,7 @@ chrono = { workspace = true } arc-swap = { workspace = true } [dev-dependencies] -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.0" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.1" } tokio = { workspace = true } lance-namespace-impls = { workspace = true } serial_test = "3" diff --git a/crates/omnigraph/src/db/mod.rs b/crates/omnigraph/src/db/mod.rs index d0b292f..8702f88 100644 --- a/crates/omnigraph/src/db/mod.rs +++ b/crates/omnigraph/src/db/mod.rs @@ -13,7 +13,7 @@ pub use manifest::{Snapshot, SubTableEntry, SubTableUpdate}; pub(crate) use omnigraph::ensure_public_branch_ref; pub use omnigraph::{ CleanupPolicyOptions, InitOptions, MergeOutcome, Omnigraph, OpenMode, SchemaApplyOptions, - SchemaApplyResult, TableCleanupStats, TableOptimizeStats, + SchemaApplyResult, SkipReason, TableCleanupStats, TableOptimizeStats, }; pub(crate) use run_registry::is_internal_run_branch; diff --git a/crates/omnigraph/src/db/omnigraph.rs b/crates/omnigraph/src/db/omnigraph.rs index 9d1403d..7b8a3f6 100644 --- a/crates/omnigraph/src/db/omnigraph.rs +++ b/crates/omnigraph/src/db/omnigraph.rs @@ -33,7 +33,7 @@ mod optimize; mod schema_apply; mod table_ops; -pub use optimize::{CleanupPolicyOptions, TableCleanupStats, TableOptimizeStats}; +pub use optimize::{CleanupPolicyOptions, SkipReason, TableCleanupStats, TableOptimizeStats}; pub use schema_apply::SchemaApplyOptions; use super::commit_graph::GraphCommit; diff --git a/crates/omnigraph/src/db/omnigraph/optimize.rs b/crates/omnigraph/src/db/omnigraph/optimize.rs index c703836..fff3f54 100644 --- a/crates/omnigraph/src/db/omnigraph/optimize.rs +++ b/crates/omnigraph/src/db/omnigraph/optimize.rs @@ -40,6 +40,20 @@ fn maint_concurrency() -> usize { .unwrap_or(DEFAULT_MAINT_CONCURRENCY) } +/// Whether the installed Lance can compact a dataset that contains blob +/// columns. `false` today: Lance `compact_files` forces +/// `BlobHandling::AllBinary` on the read side, and the blob-v2 struct decoder +/// mis-counts columns ("there were more fields in the schema than provided +/// column indices"), failing even a pristine uniform-V2_2 multi-fragment blob +/// table. Reads are unaffected (queries use descriptor handling). +/// +/// While `false`, [`optimize_all_tables`] skips blob-bearing tables and reports +/// [`SkipReason::BlobColumnsUnsupportedByLance`] instead of aborting the whole +/// sweep. Flip to `true` once the upstream Lance fix ships — the +/// `lance_surface_guards.rs::compact_files_still_fails_on_blob_columns` guard +/// turns red on that bump and forces this flip. Tracked in `docs/dev/lance.md`. +const LANCE_SUPPORTS_BLOB_COMPACTION: bool = false; + /// Retention knobs for [`cleanup_all_tables`]. At least one must be set or /// nothing is cleaned. If both are set, Lance applies them as AND (a manifest /// is kept if it satisfies either — i.e. only manifests older than BOTH the @@ -52,8 +66,45 @@ pub struct CleanupPolicyOptions { pub older_than: Option, } -/// Per-table outcome of `optimize_all_tables`. +/// Why `optimize` did not compact a table. Typed so callers branch on the +/// reason rather than sniffing a string. One variant today, gated by +/// [`LANCE_SUPPORTS_BLOB_COMPACTION`]. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[non_exhaustive] +pub enum SkipReason { + /// The table has one or more `Blob` columns. Lance `compact_files` forces + /// `BlobHandling::AllBinary`, which mis-decodes blob-v2 columns; see + /// [`LANCE_SUPPORTS_BLOB_COMPACTION`] and `docs/dev/lance.md`. + BlobColumnsUnsupportedByLance, +} + +impl SkipReason { + /// Stable machine-readable token for serialized output (e.g. CLI `--json`). + /// Once emitted this is part of the output contract — keep it stable. + pub fn as_str(&self) -> &'static str { + match self { + SkipReason::BlobColumnsUnsupportedByLance => "blob_columns_unsupported_by_lance", + } + } +} + +impl std::fmt::Display for SkipReason { + /// Human-readable reason for CLI and log output. + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let msg = match self { + SkipReason::BlobColumnsUnsupportedByLance => { + "blob columns — Lance compaction unsupported" + } + }; + f.write_str(msg) + } +} + +/// Per-table outcome of `optimize_all_tables`. This is a returned result type, +/// not built by callers, so it is `#[non_exhaustive]`: future fields stay +/// non-breaking and downstream code reads fields rather than constructing it. #[derive(Debug, Clone)] +#[non_exhaustive] pub struct TableOptimizeStats { pub table_key: String, /// Number of source fragments that were rewritten by Lance. @@ -62,6 +113,33 @@ pub struct TableOptimizeStats { pub fragments_added: usize, /// Did this table get a new Lance manifest version from the compaction? pub committed: bool, + /// `Some(reason)` if this table was deliberately not compacted. When set, + /// `fragments_removed == 0`, `fragments_added == 0`, and `!committed`. + pub skipped: Option, +} + +impl TableOptimizeStats { + /// Stat for a table that Lance actually compacted. + fn compacted(table_key: String, metrics: &CompactionMetrics, committed: bool) -> Self { + Self { + table_key, + fragments_removed: metrics.fragments_removed, + fragments_added: metrics.fragments_added, + committed, + skipped: None, + } + } + + /// Stat for a table that was deliberately skipped (compaction not attempted). + fn skipped(table_key: String, reason: SkipReason) -> Self { + Self { + table_key, + fragments_removed: 0, + fragments_added: 0, + committed: false, + skipped: Some(reason), + } + } } /// Per-table outcome of `cleanup_all_tables`. `error` is `Some` when this @@ -84,14 +162,21 @@ pub async fn optimize_all_tables(db: &Omnigraph) -> Result = all_table_keys(&db.catalog()) - .into_iter() - .filter_map(|table_key| { - let entry = snapshot.entry(&table_key)?; + // Compute per-table state (path + whether it has blob columns) up front, in + // a scope that drops the catalog handle before the async stream starts. + let table_tasks: Vec<(String, String, bool)> = { + let catalog = db.catalog(); + let mut tasks = Vec::new(); + for table_key in all_table_keys(&catalog) { + let Some(entry) = snapshot.entry(&table_key) else { + continue; + }; let full_path = format!("{}/{}", db.root_uri, entry.table_path); - Some((table_key, full_path)) - }) - .collect(); + let has_blob = !blob_properties_for_table_key(&catalog, &table_key)?.is_empty(); + tasks.push((table_key, full_path, has_blob)); + } + tasks + }; if table_tasks.is_empty() { return Ok(Vec::new()); @@ -101,7 +186,24 @@ pub async fn optimize_all_tables(db: &Omnigraph) -> Result> = futures::stream::iter(table_tasks.into_iter()) - .map(|(table_key, full_path)| async move { + .map(|(table_key, full_path, has_blob)| async move { + // Lance `compact_files` mis-decodes blob-v2 columns under the forced + // `BlobHandling::AllBinary` read (see LANCE_SUPPORTS_BLOB_COMPACTION). + // Skip blob-bearing tables and report it rather than aborting the + // whole sweep — the other tables still compact. + if has_blob && !LANCE_SUPPORTS_BLOB_COMPACTION { + tracing::warn!( + target: "omnigraph::optimize", + table = %table_key, + "skipping compaction: table has blob columns the current Lance \ + cannot rewrite (blob-v2 AllBinary decode bug); other tables \ + unaffected — rerun after the Lance fix", + ); + return Ok(TableOptimizeStats::skipped( + table_key, + SkipReason::BlobColumnsUnsupportedByLance, + )); + } let mut ds = table_store .open_dataset_head_for_write(&table_key, &full_path, None) .await?; @@ -111,12 +213,11 @@ pub async fn optimize_all_tables(db: &Omnigraph) -> Result RecordBatch { + let ids: Vec = (start..start + n).map(|i| format!("n{i}")).collect(); + let data = + LargeBinaryArray::from_iter_values((start..start + n).map(|i| format!("blob{i}"))); + let blob_uri = StringArray::from(vec![None::<&str>; n as usize]); + let DataType::Struct(fields) = lance::blob::blob_field("content", true).data_type().clone() + else { + unreachable!("blob_field is always a Struct"); + }; + let content = StructArray::new( + fields, + vec![Arc::new(data) as _, Arc::new(blob_uri) as _], + None, + ); + let schema = Arc::new(Schema::new(vec![ + Field::new("id", DataType::Utf8, false), + lance::blob::blob_field("content", true), + ])); + RecordBatch::try_new( + schema, + vec![Arc::new(StringArray::from(ids)) as _, Arc::new(content) as _], + ) + .unwrap() + } + + async fn write(uri: &str, batch: RecordBatch, mode: WriteMode) { + let schema = batch.schema(); + let reader = RecordBatchIterator::new(vec![Ok(batch)], schema); + // Blob v2 requires file version >= 2.2; without the pin the *write* + // would fail with a different error, masking the guard's intent. + let params = WriteParams { + mode, + enable_stable_row_ids: true, + data_storage_version: Some(LanceFileVersion::V2_2), + ..Default::default() + }; + Dataset::write(reader, uri, Some(params)).await.unwrap(); + } + + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().join("guard10-blob.lance"); + let uri = uri.to_str().unwrap(); + + // Uniform V2_2, two fragments → forces compaction to actually rewrite. + write(uri, blob_batch(0, 2), WriteMode::Create).await; + write(uri, blob_batch(100, 2), WriteMode::Append).await; + + let mut ds = Dataset::open(uri).await.unwrap(); + assert!( + ds.get_fragments().len() >= 2, + "guard needs a multi-fragment table to trigger a real compaction rewrite" + ); + + let result = compact_files(&mut ds, CompactionOptions::default(), None).await; + let err = result.expect_err( + "compact_files unexpectedly SUCCEEDED on a blob table — the Lance blob-v2 \ + compaction bug is fixed. Flip LANCE_SUPPORTS_BLOB_COMPACTION to true in \ + db/omnigraph/optimize.rs, remove the blob-skip branch, and re-pin docs/dev/lance.md.", + ); + assert!( + err.to_string() + .contains("more fields in the schema than provided column indices"), + "blob compaction failed with an unexpected error (Lance internals may have \ + shifted): {err}" + ); +} diff --git a/crates/omnigraph/tests/maintenance.rs b/crates/omnigraph/tests/maintenance.rs index 722bdc4..3e61677 100644 --- a/crates/omnigraph/tests/maintenance.rs +++ b/crates/omnigraph/tests/maintenance.rs @@ -8,7 +8,7 @@ mod helpers; use std::time::Duration; use lance::Dataset; -use omnigraph::db::{CleanupPolicyOptions, Omnigraph}; +use omnigraph::db::{CleanupPolicyOptions, Omnigraph, SkipReason}; use omnigraph::loader::{LoadMode, load_jsonl}; use helpers::{TEST_DATA, TEST_SCHEMA, count_rows, init_and_load}; @@ -72,6 +72,97 @@ async fn optimize_after_load_then_again_is_idempotent() { } } +// Regression: `optimize` must not crash on a graph that has a `Blob` table. +// +// Lance `compact_files` forces `BlobHandling::AllBinary`, which mis-decodes +// blob-v2 columns ("more fields in the schema than provided column indices"), +// failing even a pristine uniform-V2_2 multi-fragment blob table. `optimize` +// must skip blob-bearing tables (and report the skip) rather than aborting the +// whole sweep. +// +// Before the skip fix, `optimize()` returned that Lance error here and aborted +// the whole sweep; it now skips the blob table (`doc.skipped == Some(..)`) +// while the sibling non-blob `Tag` table still compacts. The skip is gated by +// `LANCE_SUPPORTS_BLOB_COMPACTION`; the surface guard +// `compact_files_still_fails_on_blob_columns` flags when the upstream Lance fix +// makes the skip (and this test's blob arm) removable. +#[tokio::test] +async fn optimize_skips_blob_table_and_reports_skip() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + // One Blob node type (`Doc`) + one plain node type (`Tag`): proves the blob + // table is skipped while a non-blob table in the same sweep still compacts. + let schema = "\ +node Doc {\n slug: String @key\n content: Blob\n}\n\ +node Tag {\n slug: String @key\n}\n"; + let mut db = Omnigraph::init(uri, schema).await.unwrap(); + + // Multi-fragment blob table: Overwrite creates fragment 1; each Merge of + // new keys appends another. A >=2-fragment blob table is exactly what + // crashes `compact_files` today (single fragment would no-op and not crash). + load_jsonl( + &mut db, + "{\"type\":\"Doc\",\"data\":{\"slug\":\"d1\",\"content\":\"base64:aGVsbG8x\"}}\n{\"type\":\"Doc\",\"data\":{\"slug\":\"d2\",\"content\":\"base64:aGVsbG8y\"}}", + LoadMode::Overwrite, + ) + .await + .unwrap(); + load_jsonl( + &mut db, + "{\"type\":\"Doc\",\"data\":{\"slug\":\"d3\",\"content\":\"base64:aGVsbG8z\"}}", + LoadMode::Merge, + ) + .await + .unwrap(); + load_jsonl( + &mut db, + "{\"type\":\"Doc\",\"data\":{\"slug\":\"d4\",\"content\":\"base64:aGVsbG80\"}}", + LoadMode::Merge, + ) + .await + .unwrap(); + // Plain table, also multi-fragment so it has something to compact. + load_jsonl( + &mut db, + "{\"type\":\"Tag\",\"data\":{\"slug\":\"t1\"}}\n{\"type\":\"Tag\",\"data\":{\"slug\":\"t2\"}}", + LoadMode::Merge, + ) + .await + .unwrap(); + load_jsonl( + &mut db, + "{\"type\":\"Tag\",\"data\":{\"slug\":\"t3\"}}", + LoadMode::Merge, + ) + .await + .unwrap(); + + let stats = db + .optimize() + .await + .expect("optimize must not crash on a graph with a Blob table"); + + let doc = stats + .iter() + .find(|s| s.table_key == "node:Doc") + .expect("Doc stat present"); + let tag = stats + .iter() + .find(|s| s.table_key == "node:Tag") + .expect("Tag stat present"); + // The blob table is skipped (and reported), not compacted. + assert_eq!( + doc.skipped, + Some(SkipReason::BlobColumnsUnsupportedByLance), + "blob table must be reported as skipped", + ); + assert!(!doc.committed, "skipped blob table is not compacted"); + assert_eq!(doc.fragments_removed, 0); + assert_eq!(doc.fragments_added, 0); + // The plain (non-blob) table is unaffected by the skip. + assert_eq!(tag.skipped, None, "non-blob table must not be skipped"); +} + #[tokio::test] async fn cleanup_without_any_policy_option_errors() { let dir = tempfile::tempdir().unwrap(); diff --git a/docs/dev/invariants.md b/docs/dev/invariants.md index 0cf295c..5ee4f17 100644 --- a/docs/dev/invariants.md +++ b/docs/dev/invariants.md @@ -130,6 +130,15 @@ them explicit. - **Deletes and vector indexes:** `delete_where` and vector index creation still advance Lance HEAD inline because the required public Lance APIs are missing. Keep D2 and recovery coverage in place until those residuals are removed. +- **Blob-column compaction:** Lance `compact_files` mis-decodes blob-v2 columns + under its forced `BlobHandling::AllBinary` read ("more fields in the schema + than provided column indices"), so `optimize` skips any table with a `Blob` + property — reporting `SkipReason::BlobColumnsUnsupportedByLance` (loud, not a + silent drop) behind the `LANCE_SUPPORTS_BLOB_COMPACTION` gate. Reads and writes + are unaffected; only space/fragment reclamation on blob tables is deferred. + Remove the skip when the upstream Lance fix lands — the + `lance_surface_guards.rs::compact_files_still_fails_on_blob_columns` guard + turns red on that bump to force it. - **Planner capability/stat surfaces:** cost-aware planning, complete capability advertisement, and explain-with-cost are roadmap. Do not describe them as implemented. diff --git a/docs/dev/lance.md b/docs/dev/lance.md index 100da6f..9d2b990 100644 --- a/docs/dev/lance.md +++ b/docs/dev/lance.md @@ -176,7 +176,8 @@ Migration from Lance 4.0.0 → 6.0.1 landed in this cycle (DataFusion 52 → 53, - **Lance #6666 still open** (`build_index_metadata_from_segments` public): vector-index two-phase blocked; inline `create_vector_index` residual retained. - **Lance #6877 still open** (`MergeInsertBuilder` dup-rowid): PR #109's `SourceDedupeBehavior::FirstSeen` + `check_batch_unique_by_keys` precondition stay load-bearing. - **`Dataset::force_delete_branch`** (`branches().delete(name, force=true)`, dataset.rs:524) tolerates a missing branch-*contents* ref (vs plain `delete_branch`'s `RefNotFound`), but on the local store still errors `NotFound` if the branch `tree/` directory is fully absent (`remove_dir_all`'s NotFound is not caught for Lance's native error variant, refs.rs:526-549). Both variants still refuse a branch with referencing descendants (`RefConflict`). `TableStore::force_delete_branch` wraps this to be fully idempotent (tolerates already-absent). The single-authority branch-delete redesign uses it for orphan reclamation (eager best-effort reclaim + cleanup reconciler). Pinned by `lance_surface_guards.rs::force_delete_branch_semantics`. Branch delete is "flip the ref atomically, then `remove_dir_all(tree/{branch})`"; branch-exclusive data lives under `tree/{branch}/` so a drop reclaims it immediately without touching `main`. +- **Lance blob-v2 `compact_files` bug** (no public issue found as of 2026-06): `compact_files` disables binary-copy for blob datasets and forces `BlobHandling::AllBinary` on the read side; the v2.1+ structural decoder then mis-counts column infos for the blob-v2 struct and fails with `Invalid user input: there were more fields in the schema than provided column indices / infos` (`lance-encoding/src/decoder.rs::ColumnInfoIter::expect_next`). This fails even a pristine uniform-V2_2 multi-fragment blob table; vector/list/scalar/ragged columns and mixed file versions all compact fine. Reads/queries use descriptor handling (`BlobHandling::default()`) and are unaffected. `optimize` skips blob-bearing tables behind `LANCE_SUPPORTS_BLOB_COMPACTION = false` (`db/omnigraph/optimize.rs`), reporting `SkipReason::BlobColumnsUnsupportedByLance`. Pinned by `lance_surface_guards.rs::compact_files_still_fails_on_blob_columns`, which turns red when the bug is fixed → flip the gate, remove the skip branch + the `maintenance.rs::optimize_skips_blob_table_and_reports_skip` skip assertions. -Surface guards added: `crates/omnigraph/tests/lance_surface_guards.rs` (9 named guards; 4 runtime + 5 compile-only). Future Lance bumps re-run this file first as the smoke check. Two additional guards from the original plan deferred to follow-up (`manifest_cas_returns_row_level_contention_variant` needs full publisher-race harness; `table_version_metadata_byte_compatible_with_v4` needs `pub(crate)` reach extension). +Surface guards added: `crates/omnigraph/tests/lance_surface_guards.rs` (10 named guards; 5 runtime + 5 compile-only). Future Lance bumps re-run this file first as the smoke check. Two additional guards from the original plan deferred to follow-up (`manifest_cas_returns_row_level_contention_variant` needs full publisher-race harness; `table_version_metadata_byte_compatible_with_v4` needs `pub(crate)` reach extension). Bump this date stanza on the next alignment pass. diff --git a/docs/releases/v0.6.1.md b/docs/releases/v0.6.1.md new file mode 100644 index 0000000..aafe1af --- /dev/null +++ b/docs/releases/v0.6.1.md @@ -0,0 +1,26 @@ +# Omnigraph v0.6.1 + +v0.6.1 focuses on operational polish after v0.6.0: stored-query registries, safer branch cleanup, more complete release artifacts, and a Lance blob-compaction workaround. + +## Highlights + +- **Stored-query registries.** `omnigraph.yaml` can declare curated `queries:` blocks per graph. Servers load and type-check them at startup, `omnigraph queries validate` checks them offline, `omnigraph queries list` shows exposed queries and typed params, `GET /queries` exposes a typed catalog, and `POST /queries/{name}` invokes a stored query without accepting ad hoc `.gq` source from the client. +- **Stored-query policy gate.** New Cedar action `invoke_query` gates the stored-query invocation surface. Stored mutations are double-gated: `invoke_query` to reach the stored query and `change` for the actual write. +- **Safer branch deletion.** `branch_delete` now treats the manifest as the authority, flips branch visibility atomically, and reclaims per-table/commit-graph forks as derived state. If best-effort reclaim is interrupted, `cleanup` reconciles orphaned forks; reusing a branch name before cleanup reports an actionable error. +- **Blob-safe optimize.** `omnigraph optimize` skips tables with `Blob` properties instead of failing the whole sweep on Lance's blob-v2 compaction decode bug. Skips are visible in human output, `--json` as `skipped`, `TableOptimizeStats.skipped`, and logs; non-blob tables still compact normally. +- **Deployment improvements.** The container entrypoint now composes `OMNIGRAPH_TARGET_URI` with `OMNIGRAPH_CONFIG`, so operators can keep the graph URI in env while loading policy/query config from a mounted file. The local RustFS bootstrap pins RustFS beta.3 and allows the current insecure local-dev default credentials. +- **Windows release support.** Tagged and edge releases now publish Windows x86_64 archives containing `omnigraph.exe` and `omnigraph-server.exe`, with a PowerShell installer and Windows install docs. +- **Release tooling.** Homebrew formula generation was tightened to produce audit-clean formulas. + +## Compatibility Notes + +- A graph selected by name (`--target` or `server.graph`) now uses `graphs..policy` and `graphs..queries`. Top-level `policy` / `queries` blocks are only for anonymous bare-URI single-graph mode; using them with a named graph now fails loudly with migration guidance. +- `mcp.expose` defaults to `true` for stored-query registry entries. Set `mcp: { expose: false }` for service-only queries that should not appear in the catalog. +- `invoke_query` is graph-scoped, not branch-scoped. Branch/snapshot access remains enforced by the inner `read` / `change` gate. +- Blob tables are not compacted until the upstream Lance fix lands, so fragment count and deleted-row space on blob tables are not reclaimed by `optimize`. Reads, writes, and query results are unaffected; no on-disk migration is required. +- `TableOptimizeStats` is now `#[non_exhaustive]` and gains a `skipped: Option` field (so does the new `SkipReason` enum). This is a source-level change only for downstream code that built this returned result struct by literal — rare, since it is produced by `optimize` and consumed by reading its fields; field access is unaffected, and `#[non_exhaustive]` keeps future additions non-breaking. + +## Docs And Cleanup + +- Public docs were updated for stored queries, policy, server routes, deployment, Windows installation, branch deletion, maintenance, and the `runs` docs rename to `writes`. +- README copy and release documentation were refreshed; older release notes had small typo/wording fixes. diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 019e4ad..8263919 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -21,7 +21,7 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` sc | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | | `queries validate \| list` | operate on the server-side stored-query registry (the `queries:` block). `validate` type-checks every stored query against the live schema offline (opens the selected graph; exits non-zero on any breakage), catching schema drift without restarting the server; `list` prints the selected registry's query names, MCP exposure, and typed params. For per-graph registries, pass `--target ` or set `cli.graph`; with no graph selection, `list` shows only top-level `queries:`. Distinct from `lint`, which validates a single `.gq` file | -| `optimize` | non-destructive Lance compaction | +| `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns; `--json` reports a `skipped` field) | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | | `embed` | offline JSONL embedding pipeline | | `policy validate \| test \| explain` | Cedar tooling. Selects `cli.graph`, else `server.graph`, else top-level `policy.file` | diff --git a/docs/user/constants.md b/docs/user/constants.md index 527aaea..8f13555 100644 --- a/docs/user/constants.md +++ b/docs/user/constants.md @@ -11,6 +11,7 @@ | Internal manifest schema version | `INTERNAL_MANIFEST_SCHEMA_VERSION = 2` | `db/manifest/migrations.rs` | | Merge stage batch | `MERGE_STAGE_BATCH_ROWS = 8192` | `exec/merge.rs` | | Maintenance concurrency | `OMNIGRAPH_MAINTENANCE_CONCURRENCY=8` | `db/omnigraph/optimize.rs` | +| Lance blob compaction support | `LANCE_SUPPORTS_BLOB_COMPACTION = false` | `db/omnigraph/optimize.rs` | | Graph index cache size | `8` (LRU) | `runtime_cache.rs` | | Default body limit | `1 MB` | `omnigraph-server/lib.rs` | | Ingest body limit | `32 MB` | `omnigraph-server/lib.rs` | diff --git a/docs/user/maintenance.md b/docs/user/maintenance.md index 9839ea1..3628fa0 100644 --- a/docs/user/maintenance.md +++ b/docs/user/maintenance.md @@ -7,7 +7,8 @@ - Lance `compact_files()` on every node + edge table on `main`. - Rewrites small fragments into fewer large ones; old fragments remain reachable via older manifests. - Bounded by `OMNIGRAPH_MAINTENANCE_CONCURRENCY` (default 8). -- Returns `[TableOptimizeStats { table_key, fragments_removed, fragments_added, committed }]`. +- Returns `[TableOptimizeStats { table_key, fragments_removed, fragments_added, committed, skipped }]`. +- **Blob tables are skipped.** A table that declares any `Blob` property is not compacted: it is reported with `skipped: Some(BlobColumnsUnsupportedByLance)` (and logged via `tracing::warn`) instead of compacted, and the rest of the sweep proceeds normally. The current Lance `compact_files` mis-decodes blob-v2 columns under its forced `BlobHandling::AllBinary` read; **reads and writes are unaffected** — only compaction is. This is gated by `LANCE_SUPPORTS_BLOB_COMPACTION` (`db/omnigraph/optimize.rs`) and removed when the upstream Lance fix lands (see [docs/dev/lance.md](../dev/lance.md)). Consequence: fragment count and deleted-row space on blob tables are not reclaimed until then; query results are never affected. ## `cleanup_all_tables(db, options)` — destructive diff --git a/openapi.json b/openapi.json index 08d39c4..aced64d 100644 --- a/openapi.json +++ b/openapi.json @@ -7,7 +7,7 @@ "name": "MIT", "identifier": "MIT" }, - "version": "0.6.0" + "version": "0.6.1" }, "paths": { "/branches": { From 96dbe9dec00b41b68907708d7535437677d3fde7 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler Date: Sat, 6 Jun 2026 00:44:48 +0300 Subject: [PATCH 016/165] fix(release): make Homebrew audit non-blocking + set up brew on runner (#140) The v0.6.1 Release shipped binaries but the Homebrew tap update job died at the audit step (brew not on the ubuntu runner; exit 127), skipping the formula push so the tap stayed at 0.6.0. - Install Homebrew via Homebrew/actions/setup-homebrew so brew is available. - Make both the setup and audit steps continue-on-error: they are best-effort diagnostics (the formula is correct by construction via update-homebrew-formula.sh), so neither can skip the actual tap publish. - Drop --online from brew audit for deterministic, network-independent linting. --- .github/workflows/release.yml | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 3a66ff2..a265c40 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -121,16 +121,30 @@ jobs: run: | ./scripts/update-homebrew-formula.sh "${GITHUB_REF_NAME}" homebrew-tap/Formula/omnigraph.rb + # Diagnostic only: brew is not on PATH on the ubuntu runner by default, so + # set it up explicitly. Both this setup and the audit below are best-effort + # canaries, not gates — continue-on-error on each keeps a failed/flaky brew + # (the action is pinned to a moving @master ref) from skipping the actual + # tap publish below. The formula is correct by construction + # (update-homebrew-formula.sh), so brew tooling must never block the push. + - name: Set up Homebrew + if: env.HOMEBREW_TAP_SKIP != '1' + continue-on-error: true + uses: Homebrew/actions/setup-homebrew@master + - name: Audit generated formula if: env.HOMEBREW_TAP_SKIP != '1' + continue-on-error: true run: | # Audit the checked-out tap by name (brew audit rejects bare paths # and needs tap context). Symlink the checkout into Homebrew's Taps - # tree so `modernrelay/tap/omnigraph` resolves to it. + # tree so `modernrelay/tap/omnigraph` resolves to it. Offline audit + # (no --online) keeps it deterministic; it still catches the + # ComponentsOrder/structure class of problems. tap_dir="$(brew --repository)/Library/Taps/modernrelay/homebrew-tap" mkdir -p "$(dirname "$tap_dir")" ln -sfn "$PWD/homebrew-tap" "$tap_dir" - brew audit --strict --online modernrelay/tap/omnigraph + brew audit --strict modernrelay/tap/omnigraph - name: Commit and push formula update if: env.HOMEBREW_TAP_SKIP != '1' From c7365bf8efd4500d6af16b00eec34d4c2202ca2b Mon Sep 17 00:00:00 2001 From: Andrew Altshuler Date: Sat, 6 Jun 2026 18:09:47 +0300 Subject: [PATCH 017/165] ci(codeowners): un-trap required checks, auto-render, generate owner tables (#142) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The CODEOWNERS required checks blocked every PR — the real root cause was a name mismatch, compounded by a path filter: - branch-protection.json required the contexts `CODEOWNERS / drift` and `CODEOWNERS / noedit` (the GitHub UI "workflow / job-id" display form), but the jobs report check-run names from their `name:` fields — "CODEOWNERS matches source" / "CODEOWNERS not hand-edited". The required contexts therefore never matched any reported check and sat permanently pending. - The workflow was also path-filtered to CODEOWNERS files, so it didn't even run for most PRs. Net effect: with both required checks unsatisfiable, every PR could only land via admin override (e.g. #140). Fixes: - A: drop the `paths:` filter so the workflow runs on every PR and both required contexts always report. - name fix: point branch-protection.json at the actual job names verbatim, and add a doc note that the contexts must equal the job `name:` values. - B: the `drift` job now re-renders and, on same-repo PRs, auto-commits the regenerated artifacts back to the branch (mirrors the openapi.json job in ci.yml); forks / manual runs strict-check instead. Contributors no longer run the script by hand. - D: render-codeowners.py also generates a "who owns what" path->owners + roles table spliced into docs/dev/codeowners.md between markers, so the human-readable view never drifts. Idempotent; CODEOWNERS output unchanged. - docs: correct the stale `enforce_admins: true` line (JSON and live are false). NOTE: the branch-protection.json change only takes effect after an admin runs `./scripts/apply-branch-protection.sh` (deliberate manual step, per docs/dev/branch-protection.md). Until then `main` still requires the old mismatched contexts, so this PR itself needs an admin-override merge — the last one that should be necessary. Co-authored-by: Claude Opus 4.8 (1M context) --- .github/branch-protection.json | 4 +- .github/scripts/render-codeowners.py | 81 ++++++++++++++++++++++++++-- .github/workflows/codeowners.yml | 72 ++++++++++++++++++++----- docs/dev/branch-protection.md | 4 +- docs/dev/codeowners.md | 39 ++++++++++---- 5 files changed, 168 insertions(+), 32 deletions(-) diff --git a/.github/branch-protection.json b/.github/branch-protection.json index 61b7d33..7ca46b9 100644 --- a/.github/branch-protection.json +++ b/.github/branch-protection.json @@ -7,8 +7,8 @@ "Check AGENTS.md Links", "Test Workspace", "Test omnigraph-server --features aws", - "CODEOWNERS / drift", - "CODEOWNERS / noedit" + "CODEOWNERS matches source", + "CODEOWNERS not hand-edited" ] }, "enforce_admins": false, diff --git a/.github/scripts/render-codeowners.py b/.github/scripts/render-codeowners.py index f243d0c..5e96545 100755 --- a/.github/scripts/render-codeowners.py +++ b/.github/scripts/render-codeowners.py @@ -1,10 +1,14 @@ #!/usr/bin/env python3 -"""Render .github/CODEOWNERS from .github/codeowners-roles.yml. +"""Render .github/CODEOWNERS and the ownership tables in +docs/dev/codeowners.md from .github/codeowners-roles.yml. -The yml is the source of truth — editing CODEOWNERS directly is -rejected by CI (see .github/workflows/codeowners.yml). This script -expands the role-based yml into the flat path→owners format GitHub -expects. +The yml is the source of truth. This script expands the role-based yml +into (1) the flat path→owners format GitHub expects in +`.github/CODEOWNERS`, and (2) the "who owns what" markdown tables spliced +between the generated-region markers in `docs/dev/codeowners.md`. Both are +derived artifacts; CI re-renders them on every PR (see +.github/workflows/codeowners.yml) and auto-commits the result on same-repo +PRs, so the source of truth and the human-readable view never drift. Usage: python3 .github/scripts/render-codeowners.py @@ -16,6 +20,7 @@ Exits non-zero on: one owner; otherwise CODEOWNERS would assign nobody and GitHub would silently fall back to "no required reviewer", which defeats the purpose). + - Missing generated-region markers in docs/dev/codeowners.md. """ from __future__ import annotations @@ -34,6 +39,13 @@ except ImportError: REPO_ROOT = Path(__file__).resolve().parents[2] SOURCE = REPO_ROOT / ".github" / "codeowners-roles.yml" OUTPUT = REPO_ROOT / ".github" / "CODEOWNERS" +DOCS = REPO_ROOT / "docs" / "dev" / "codeowners.md" + +# The "who owns what" tables in docs/dev/codeowners.md are spliced between +# these markers so the human-readable view never drifts from the source of +# truth. Edit codeowners-roles.yml and re-render — never the table by hand. +DOCS_BEGIN = "" +DOCS_END = "" BANNER = """\ # AUTOGENERATED from .github/codeowners-roles.yml. Do not edit by hand. @@ -75,6 +87,62 @@ def owners_for(role_names: list[str], roles: dict) -> list[str]: return seen +def _oneline(text: str) -> str: + """Collapse a folded/multi-line YAML description into one cell of text.""" + return " ".join((text or "").split()) + + +def ownership_tables(spec: dict, roles: dict) -> str: + """Render the human-readable "who owns what" markdown — a path→owners + table (the operative view at PR time, in last-match-wins order with the + catch-all first) plus a role→members table. Spliced into the docs between + the markers so it is always current with the source of truth.""" + out: list[str] = [] + + out.append("**Path → owners** (GitHub applies *last match wins*; the `*` " + "catch-all is listed first and is overridden by the specific " + "patterns below it):") + out.append("") + out.append("| Path | Owners | Role(s) |") + out.append("|---|---|---|") + if "default" in spec: + owners = " ".join(owners_for(spec["default"], roles)) + out.append(f"| `*` | {owners} | {', '.join(spec['default'])} |") + for pattern, role_names in (spec.get("paths") or {}).items(): + owners = " ".join(owners_for(role_names, roles)) + out.append(f"| `{pattern}` | {owners} | {', '.join(role_names)} |") + out.append("") + + out.append("**Roles**:") + out.append("") + out.append("| Role | Members | Description |") + out.append("|---|---|---|") + for name, role in roles.items(): + members = " ".join(f"@{m}" for m in (role.get("members") or [])) + out.append(f"| `{name}` | {members} | {_oneline(role.get('description', ''))} |") + out.append("") + + return "\n".join(out) + + +def splice_docs(table_md: str) -> None: + """Replace the region between DOCS_BEGIN/DOCS_END in the docs file with the + freshly generated tables, leaving surrounding prose untouched.""" + if not DOCS.exists(): + sys.exit(f"error: docs file not found: {DOCS}") + text = DOCS.read_text() + if DOCS_BEGIN not in text or DOCS_END not in text: + sys.exit( + f"error: ownership markers not found in {DOCS.relative_to(REPO_ROOT)}. " + f"Add the lines:\n {DOCS_BEGIN}\n {DOCS_END}\n" + f"around the generated table region." + ) + head, rest = text.split(DOCS_BEGIN, 1) + _, tail = rest.split(DOCS_END, 1) + new = f"{head}{DOCS_BEGIN}\n\n{table_md}\n{DOCS_END}{tail}" + DOCS.write_text(new) + + def main() -> int: if not SOURCE.exists(): sys.exit(f"error: source file not found: {SOURCE}") @@ -127,6 +195,9 @@ def main() -> int: OUTPUT.write_text(rendered) print(f"wrote {OUTPUT.relative_to(REPO_ROOT)}") + + splice_docs(ownership_tables(spec, roles)) + print(f"updated {DOCS.relative_to(REPO_ROOT)}") return 0 diff --git a/.github/workflows/codeowners.yml b/.github/workflows/codeowners.yml index 19d5835..75b3515 100644 --- a/.github/workflows/codeowners.yml +++ b/.github/workflows/codeowners.yml @@ -1,19 +1,24 @@ name: CODEOWNERS +# Runs on EVERY pull request (no paths filter). The two jobs below are +# required status checks on `main`; a path-filtered required check never +# reports for PRs outside the filter and leaves them permanently "pending" +# (the trap that forced admin-override merges). Always-run + cheap +# short-circuit is what keeps them honest. on: pull_request: - paths: - - '.github/codeowners-roles.yml' - - '.github/CODEOWNERS' - - '.github/scripts/render-codeowners.py' - - '.github/workflows/codeowners.yml' workflow_dispatch: -# Read-only; we never push from this workflow. +# `drift` auto-commits the regenerated artifacts back to same-repo PR +# branches, so it needs write access. permissions: - contents: read + contents: write jobs: + # NOTE: the job `name:` values below ("CODEOWNERS matches source" / + # "CODEOWNERS not hand-edited") ARE the status-check contexts that + # .github/branch-protection.json must list verbatim. Renaming a job here + # is a branch-protection change — update the JSON and re-apply. drift: name: CODEOWNERS matches source runs-on: ubuntu-latest @@ -28,19 +33,56 @@ jobs: - name: Install PyYAML run: pip install pyyaml - - name: Re-render CODEOWNERS + - name: Re-render CODEOWNERS + ownership docs run: python3 .github/scripts/render-codeowners.py - - name: Reject drift + # Same-repo PR: push the regenerated artifacts back so contributors + # never have to run the script locally. Mirrors the openapi.json + # auto-commit in ci.yml (separate shallow clone of the head branch so + # the pushed commit carries only the regenerated files). + - name: Commit regenerated artifacts to PR branch + if: | + github.event_name == 'pull_request' && + github.event.pull_request.head.repo.full_name == github.repository + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | - if ! git diff --quiet .github/CODEOWNERS; then - echo "::error::.github/CODEOWNERS is out of sync with .github/codeowners-roles.yml." - echo "::error::Run \`python3 .github/scripts/render-codeowners.py\` locally and commit the result." + if git diff --quiet -- .github/CODEOWNERS docs/dev/codeowners.md; then + echo "CODEOWNERS and ownership docs already in sync." + exit 0 + fi + tmp=$(mktemp -d) + git clone --depth 1 --branch "${{ github.head_ref }}" \ + "https://x-access-token:${GITHUB_TOKEN}@github.com/${{ github.repository }}.git" \ + "$tmp" + cp .github/CODEOWNERS "$tmp/.github/CODEOWNERS" + cp docs/dev/codeowners.md "$tmp/docs/dev/codeowners.md" + cd "$tmp" + if git diff --quiet -- .github/CODEOWNERS docs/dev/codeowners.md; then + echo "Head branch already matches; nothing to push." + exit 0 + fi + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add .github/CODEOWNERS docs/dev/codeowners.md + git commit -m "chore: regenerate CODEOWNERS + ownership docs" + git push + + # Fork PR / workflow_dispatch: cannot push back, so enforce drift + # strictly. The contributor runs the script and commits the result. + - name: Verify in sync (forks / manual runs) + if: | + !(github.event_name == 'pull_request' && + github.event.pull_request.head.repo.full_name == github.repository) + run: | + if ! git diff --quiet -- .github/CODEOWNERS docs/dev/codeowners.md; then + echo "::error::Generated CODEOWNERS / ownership docs are out of sync with .github/codeowners-roles.yml." + echo "::error::Run \`python3 .github/scripts/render-codeowners.py\` and commit the result." echo "--- diff ---" - git --no-pager diff .github/CODEOWNERS + git --no-pager diff -- .github/CODEOWNERS docs/dev/codeowners.md exit 1 fi - echo "CODEOWNERS is in sync with its source." + echo "Generated artifacts are in sync with their source." noedit: name: CODEOWNERS not hand-edited @@ -52,6 +94,8 @@ jobs: fetch-depth: 0 - name: Reject hand-edits to generated file + # Only meaningful for PRs (needs a base to diff against). + if: github.event_name == 'pull_request' run: | base="origin/${{ github.base_ref }}" git fetch origin "${{ github.base_ref }}" --quiet diff --git a/docs/dev/branch-protection.md b/docs/dev/branch-protection.md index 9b2fa78..2b6cc37 100644 --- a/docs/dev/branch-protection.md +++ b/docs/dev/branch-protection.md @@ -8,7 +8,7 @@ This page explains what the policy says and how to change it. | Setting | Value | Why | |---|---|---| -| **Required status checks (strict)** | `Classify Changes`, `Check AGENTS.md Links`, `Test Workspace`, `Test omnigraph-server --features aws`, `CODEOWNERS / drift`, `CODEOWNERS / noedit` | Every PR must pass workspace tests, AGENTS.md link integrity, and the CODEOWNERS hygiene checks. `strict: true` requires the branch to be up-to-date with `main` before merge. | +| **Required status checks (strict)** | `Classify Changes`, `Check AGENTS.md Links`, `Test Workspace`, `Test omnigraph-server --features aws`, `CODEOWNERS matches source`, `CODEOWNERS not hand-edited` | Every PR must pass workspace tests, AGENTS.md link integrity, and the CODEOWNERS hygiene checks. The two CODEOWNERS contexts must equal the job `name:` values in `.github/workflows/codeowners.yml` **verbatim** — a context naming a job that never reports (the old `CODEOWNERS / drift` used the job *id*, and the job was path-filtered) leaves every PR permanently pending and forces admin overrides. `strict: true` requires the branch to be up-to-date with `main` before merge. | | **Required approving reviews** | `1` | At least one reviewer. With a 2-person team, going higher would block all merges when one person is unavailable. | | **Require code-owner reviews** | `true` | The reviewer must be a code owner per `.github/CODEOWNERS`. This is what makes the codeowners chassis enforced. | | **Dismiss stale reviews on new commits** | `true` | A push after approval invalidates the prior review. Prevents the "approve, then sneak in unreviewed changes" pattern. | @@ -16,7 +16,7 @@ This page explains what the policy says and how to change it. | **Disallow force pushes** | `true` | No history rewrites on `main`. | | **Disallow branch deletions** | `true` | `main` cannot be deleted. | | **Required conversation resolution** | `true` | All review comment threads must be resolved before merge. | -| **Enforce on admins** | `true` | Even repository admins go through the gates. The point is no bypasses. | +| **Enforce on admins** | `false` | Admins can override the gates (`enforce_admins: false` in the JSON). This is the intended escape hatch for the 2-person team; tightening to `true` is tracked under hardening below. | | **Required signed commits** | not yet | Not enabled. Would lock out maintainers until everyone enrolls GPG/SSH commit signing. Tracked as a follow-up. | ## How to apply diff --git a/docs/dev/codeowners.md b/docs/dev/codeowners.md index 9a7fb50..14bba0b 100644 --- a/docs/dev/codeowners.md +++ b/docs/dev/codeowners.md @@ -4,24 +4,45 @@ This setup gives every role change a reviewable PR and a permanent in-repository audit trail (`git log .github/codeowners-roles.yml`). -## Current roles +## Who owns what -| Role | Members | Scope | +The tables below are **generated** from `.github/codeowners-roles.yml` by `.github/scripts/render-codeowners.py` (the same render that produces `.github/CODEOWNERS`). They are the always-current "who owns what at this commit" view — don't edit them by hand; edit the yml and re-render. + + + +**Path → owners** (GitHub applies *last match wins*; the `*` catch-all is listed first and is overridden by the specific patterns below it): + +| Path | Owners | Role(s) | |---|---|---| -| `engineering` | `@ragnorc` | All code under `crates/**`, repository infrastructure, default for unmapped paths | -| `docs` | `@ragnorc` | `docs/**`, README.md, AGENTS.md, CLAUDE.md, SECURITY.md | +| `*` | @ragnorc | engineering | +| `crates/**` | @ragnorc | engineering | +| `docs/**` | @ragnorc | docs | +| `README.md` | @ragnorc | docs | +| `AGENTS.md` | @ragnorc | docs | +| `CLAUDE.md` | @ragnorc | docs | +| `SECURITY.md` | @ragnorc | docs | -GitHub treats multiple owners in a CODEOWNERS line as **"any one of them satisfies the review requirement"**. To require N distinct approvers on a specific path, layer a CI check on top (not currently configured). +**Roles**: + +| Role | Members | Description | +|---|---|---| +| `engineering` | @ragnorc | All production code under crates/**. Engine, CLI, server, compiler. | +| `docs` | @ragnorc | Documentation under docs/**, plus repo-level docs (README.md, AGENTS.md, CLAUDE.md symlink, SECURITY.md). | + + + +GitHub treats multiple owners on a CODEOWNERS line as **"any one of them satisfies the review requirement"**. To require N distinct approvers on a specific path, layer a CI check on top (not currently configured). ## How to change role membership or path mappings 1. Edit `.github/codeowners-roles.yml`. -2. Run `python3 .github/scripts/render-codeowners.py` (requires PyYAML; `pip install pyyaml`). -3. Commit both files in the same PR. +2. Open a PR. **CI re-renders for you**: the `CODEOWNERS` workflow regenerates `.github/CODEOWNERS` and the ownership tables above and auto-commits them back to your PR branch on same-repository PRs — you don't have to run the script locally (though you can: `python3 .github/scripts/render-codeowners.py`, requires PyYAML). + +On a fork (where CI can't push back), the workflow instead fails with the diff so you can run the script and commit it yourself. CI fails the PR if: -- `CODEOWNERS` was edited without a corresponding yml change, or -- The yml was changed but the rendered `CODEOWNERS` doesn't match. +- a fork PR left a generated artifact out of sync, or +- `CODEOWNERS` was edited without a corresponding yml change (the `CODEOWNERS not hand-edited` check). ## How to add a new role From 343f1f17ed8e86032aef6d9a466a778a9c39b6bd Mon Sep 17 00:00:00 2001 From: Andrew Altshuler Date: Sat, 6 Jun 2026 23:58:08 +0300 Subject: [PATCH 018/165] governance: external contribution model (issues/discussions/RFCs/PRs) (#143) Formalize the public contribution surface. Maintainers keep a separate internal process and are exempt from the intake gates; everyone stays bound by review, CODEOWNERS, and branch protection. Model: - Issues = problem reports only (bug form + config.yml redirects ideas to Discussions and disables blank issues). - Discussions = ideas + RFC incubation. - RFCs = anyone (incl. external) authors docs/rfcs/NNNN-*.md; a maintainer merging it is acceptance. Distinct from the maintainer-internal docs/dev/rfc-00N-* track. - PRs = link an `accepted` issue or accepted RFC, or use the trivial fast-lane (typos/docs/deps). Enforced softly to start (template + review). Adds GOVERNANCE.md, rewrites CONTRIBUTING.md, adds docs/rfcs/ (README + template), .github issue/PR/discussion templates. Wires docs/rfcs/ into the doc-link checker (excluded like releases; linked from docs/dev/index.md). Co-authored-by: Claude Opus 4.8 (1M context) --- .github/DISCUSSION_TEMPLATE/rfc.yml | 34 +++++++++ .github/ISSUE_TEMPLATE/bug_report.yml | 55 +++++++++++++ .github/ISSUE_TEMPLATE/config.yml | 13 ++++ .github/PULL_REQUEST_TEMPLATE.md | 29 +++++++ CONTRIBUTING.md | 38 +++++++-- GOVERNANCE.md | 106 ++++++++++++++++++++++++++ docs/dev/index.md | 12 +++ docs/rfcs/0000-template.md | 54 +++++++++++++ docs/rfcs/README.md | 66 ++++++++++++++++ scripts/check-agents-md.sh | 7 +- 10 files changed, 406 insertions(+), 8 deletions(-) create mode 100644 .github/DISCUSSION_TEMPLATE/rfc.yml create mode 100644 .github/ISSUE_TEMPLATE/bug_report.yml create mode 100644 .github/ISSUE_TEMPLATE/config.yml create mode 100644 .github/PULL_REQUEST_TEMPLATE.md create mode 100644 GOVERNANCE.md create mode 100644 docs/rfcs/0000-template.md create mode 100644 docs/rfcs/README.md diff --git a/.github/DISCUSSION_TEMPLATE/rfc.yml b/.github/DISCUSSION_TEMPLATE/rfc.yml new file mode 100644 index 0000000..2a63525 --- /dev/null +++ b/.github/DISCUSSION_TEMPLATE/rfc.yml @@ -0,0 +1,34 @@ +labels: ["rfc"] +body: + - type: markdown + attributes: + value: | + Use this to **incubate an RFC** — socialize a design and reach rough + consensus before writing the formal document. When it's ready, graduate + it into a pull request that adds `docs/rfcs/NNNN-title.md` + (see [docs/rfcs/README.md](../blob/main/docs/rfcs/README.md)); a + maintainer merging that PR is acceptance. + + For a plain feature request or open-ended idea, use the **Ideas** + category instead. For bugs, open an [Issue](../../issues/new/choose). + - type: textarea + id: problem + attributes: + label: Problem / motivation + description: What needs solving, and why is it worth the long-run cost? + validations: + required: true + - type: textarea + id: sketch + attributes: + label: Proposed direction (sketch) + description: A rough shape of the design. Detail comes later in the RFC document. + validations: + required: true + - type: textarea + id: invariants + attributes: + label: Invariants touched + description: Which items in docs/dev/invariants.md does this affect or risk? Any deny-list brush? + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml new file mode 100644 index 0000000..8e19465 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -0,0 +1,55 @@ +name: Bug report +description: Report a reproducible problem or wrong behavior in OmniGraph. +title: "bug: " +labels: ["bug", "needs-triage"] +body: + - type: markdown + attributes: + value: | + Issues are for **reporting problems** — concrete, reproducible bugs. + For ideas, feature requests, or questions, please use + [Discussions](../../discussions) instead. + For a security vulnerability, follow [SECURITY.md](../../blob/main/SECURITY.md) — do **not** file it here. + + A maintainer will triage this; once labelled **`accepted`** it's open for a pull request + (see [GOVERNANCE.md](../../blob/main/GOVERNANCE.md)). + - type: textarea + id: what-happened + attributes: + label: What happened + description: What went wrong, and what you expected instead. + validations: + required: true + - type: textarea + id: repro + attributes: + label: Steps to reproduce + description: Minimal steps, commands, schema/query, or a failing snippet. + placeholder: | + 1. omnigraph init ... + 2. omnigraph ... + 3. observed: ... / expected: ... + validations: + required: true + - type: input + id: version + attributes: + label: Version + description: Output of `omnigraph --version` (or the engine/crate version) and how you installed it. + validations: + required: true + - type: input + id: environment + attributes: + label: Environment + description: OS, architecture, and storage backend (local FS / S3 / RustFS / MinIO). + validations: + required: false + - type: textarea + id: logs + attributes: + label: Logs / output + description: Relevant error text or logs. Will be rendered as code. + render: shell + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..50720b8 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,13 @@ +# Issues are for problem reports only. Disable blank issues so everything is +# routed: bugs through the form, everything else to Discussions / SECURITY.md. +blank_issues_enabled: false +contact_links: + - name: 💡 Idea, feature request, or RFC + url: https://github.com/ModernRelay/omnigraph/discussions + about: Propose features and designs in Discussions. RFCs graduate from there into a docs/rfcs/ pull request. + - name: ❓ Question or help + url: https://github.com/ModernRelay/omnigraph/discussions + about: Ask in Discussions — questions are not tracked as Issues. + - name: 🔒 Security vulnerability + url: https://github.com/ModernRelay/omnigraph/blob/main/SECURITY.md + about: Report security issues privately per SECURITY.md — never as a public Issue. diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000..2a548c7 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,29 @@ + + +## What & why + + + +## Backing issue / RFC + + + +- [ ] Fixes an **accepted** issue: Closes # +- [ ] Implements / is an **accepted** RFC: +- [ ] **Trivial fast-lane** (typo / docs / dependency bump / comment / one-line CI) — no issue/RFC required + +## Checklist + +- [ ] Change is focused (one logical change) +- [ ] Tests added/updated for behavior changes (or N/A) +- [ ] Public docs updated if user-facing surface changed (or N/A) +- [ ] Reviewed against [docs/dev/invariants.md](../blob/main/docs/dev/invariants.md) — no Hard Invariant weakened, no deny-list item hit (or justified) + +## Notes for reviewers + + diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8d9c687..2d77ef0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,10 +1,29 @@ # Contributing -Small bug fixes and documentation improvements are welcome directly through pull -requests. +Thanks for your interest in OmniGraph. This page is the practical how-to; the +rules and decision authority behind it live in [GOVERNANCE.md](GOVERNANCE.md). -For larger changes, please open an issue or design discussion first so the -proposed direction is clear before implementation starts. +## Start in the right place + +| I want to… | Go to | Notes | +|---|---|---| +| **Report a bug** or wrong behavior | **[Open an Issue](../../issues/new/choose)** | Concrete and reproducible. A maintainer triages it; once labelled **`accepted`** it's open for a PR. | +| **Suggest a feature / share an idea / ask** | **[Start a Discussion](../../discussions)** | Ideas and questions live here, not in Issues. | +| **Propose a design / RFC** | **An RFC pull request** | Anyone can author one — see [docs/rfcs/README.md](docs/rfcs/README.md). A maintainer merging it is acceptance. | +| **Fix something / implement a change** | **A pull request** | Must link an `accepted` issue or an accepted RFC — unless it's trivial (below). | +| **Report a security vulnerability** | **[SECURITY.md](SECURITY.md)** | Do **not** open a public Issue. | + +### When can I just open a PR? +The **trivial fast-lane** — open directly, no prior issue/RFC needed: typo and +wording fixes, doc corrections, dependency bumps, comment fixes, obvious +one-line CI tweaks. Anything more substantial needs a backing `accepted` issue +or accepted RFC first, so the *why* is agreed before the *how* is reviewed. A PR +that turns out to be non-trivial will be redirected — that's about process, not +the merit of the change. + +> **Maintainers (ModernRelay team)** follow a separate internal process and are +> not bound by the intake rules above. Everyone is bound by review, CODEOWNERS, +> branch protection, and CI. ## Development @@ -49,6 +68,11 @@ CI runs both. ## Pull Requests -- keep changes focused -- include tests for behavior changes when practical -- update public docs when the user-facing surface changes +- **Link the backing issue or RFC** (`Closes #123`, or reference the RFC) — or + mark the PR as trivial per the fast-lane. +- Keep changes focused; one logical change per PR. +- Include tests for behavior changes when practical. +- Update public docs when the user-facing surface changes. + +New to the codebase? Read [AGENTS.md](AGENTS.md) — the architecture map and the +always-on invariants every change is reviewed against. diff --git a/GOVERNANCE.md b/GOVERNANCE.md new file mode 100644 index 0000000..5878f1f --- /dev/null +++ b/GOVERNANCE.md @@ -0,0 +1,106 @@ +# Governance + +This document describes how **external contributions** to OmniGraph are +proposed, accepted, and merged. It exists so an outside contributor can answer, +without asking: *where does my report/idea/change go, who decides, and what has +to happen before code lands?* + +> **Scope.** This governs the public contribution surface — Issues, +> Discussions, RFCs, and pull requests from people outside the ModernRelay +> team. **Maintainers operate under a separate internal process** and are not +> bound by the intake gates below. Everyone, maintainer or not, is still bound +> by the universal gates: branch protection on `main` and CODEOWNERS review +> (see [docs/dev/branch-protection.md](docs/dev/branch-protection.md) and +> [docs/dev/codeowners.md](docs/dev/codeowners.md)). + +## Roles + +| Role | Who | Authority | +|---|---|---| +| **Maintainer** | The code owners in [`.github/CODEOWNERS`](.github/CODEOWNERS) (generated from [`.github/codeowners-roles.yml`](.github/codeowners-roles.yml)) | Validate issues, accept/reject RFCs, review and merge PRs, set direction. Final decision authority. | +| **Contributor** | Anyone else | Report problems (Issues), propose ideas (Discussions), author RFCs, and open pull requests. | + +Decision authority rests with the maintainers. CODEOWNERS is the single source +of truth for who that is; this document does not duplicate the list. + +## The three channels + +Each channel has one job. Using the right one is the first thing we ask of a +contribution. + +| Channel | Purpose | Not for | +|---|---|---| +| **[Issues](../../issues)** | **Report a problem** — a bug, a regression, a documented behavior that's wrong. Something concrete and reproducible. | Feature requests, ideas, questions, or design proposals (→ Discussions). | +| **[Discussions](../../discussions)** | **Propose and explore** — new ideas, feature requests, questions, and the incubation of RFCs. | Bug reports (→ Issues). | +| **Pull requests** | **Land a sanctioned change** — a fix for a *validated* issue, an *accepted* RFC, or a trivial change (see fast-lane). | Substantive change with no backing issue/RFC — it will be redirected. | + +## How a change becomes mergeable + +``` + ┌─────────── bug ───────────┐ ┌──────── idea / feature ────────┐ + ▼ │ ▼ │ + Issue (problem report) │ Discussion (idea / RFC incubation) │ + │ │ │ │ + maintainer triage │ rough consensus │ + │ │ │ graduate │ + ▼ │ ▼ │ + label: accepted ──────────┐ │ RFC PR (docs/rfcs/NNNN-*.md) │ + │ │ │ │ │ + │ │ │ maintainer review │ + ▼ ▼ │ ▼ │ + Pull request ◀──────────┴──────────│── merged == accepted │ + (links the issue or the accepted RFC) ◀───────┘ (implementation PRs reference it) │ + │ + review + CODEOWNERS + branch protection + ▼ + merged +``` + +### Issues → validated +A new issue starts unlabeled. A maintainer triages it and, if it's a real, +in-scope problem, applies the **`accepted`** label. **Only `accepted` issues are +open for a contributor PR.** This prevents the "I fixed an issue you hadn't +agreed was a problem" rejection. Want to fix something? Get the issue accepted +first, or pick one already labelled `accepted` / `help wanted`. + +### Discussions → RFCs → accepted +Ideas and feature requests start in **Discussions**. Anyone — including external +contributors — may then **author an RFC** by opening a pull request that adds +`docs/rfcs/NNNN-title.md` (see [docs/rfcs/README.md](docs/rfcs/README.md)). The +RFC is reviewed as code; **a maintainer merging it is the act of acceptance** +(it becomes the durable decision record). Implementation PRs then reference the +accepted RFC. + +Authoring an RFC is open to everyone; **accepting one is a maintainer +decision.** Maintainers may also decline an RFC, with rationale, by closing it. + +### Pull requests → sanctioned +A contributor PR must do one of: +1. link a maintainer-**`accepted`** issue it fixes, or +2. be (or reference) an **accepted RFC**, or +3. qualify for the **trivial fast-lane**. + +**Trivial fast-lane** — these may be opened directly, no prior issue/RFC: +typo and wording fixes, documentation corrections, dependency bumps, comment +fixes, and obviously-correct one-line CI tweaks. When in doubt, open an Issue or +Discussion first; a PR that turns out to be non-trivial will be asked to. + +A substantive PR with no backing issue/RFC will be closed with a pointer to the +right channel — not as a judgment of the idea, but to keep design discussion +where it's reviewable. + +## What maintainers do *not* gate +Maintainers' own changes do not pass through the intake gates above — the team +runs a separate internal process. The universal gates (review, CODEOWNERS, +branch protection, CI) apply to everyone. Enforcement of the intake rules is, to +start, **by convention and review** (PR template + labels); an automated check +keyed to author association may be added later if volume warrants. + +## Code of conduct & security +- Conduct: [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md). +- Security issues are **not** public Issues — see [SECURITY.md](SECURITY.md). + +## Changing this document +Governance changes the same way code does: a pull request, reviewed by +maintainers. This file describes the external surface; the internal maintainer +process is intentionally out of scope here. diff --git a/docs/dev/index.md b/docs/dev/index.md index 600c969..1e41342 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -51,6 +51,18 @@ constraints. User-facing behavior should still be documented through | Install and deployment packaging | [install.md](../user/install.md), [deployment.md](../user/deployment.md) | | Release history | [releases/](../releases/) | +## Contribution & Governance + +| Area | Read | +|---|---| +| How to contribute (external) | [CONTRIBUTING.md](../../CONTRIBUTING.md) | +| Governance model, roles, decision authority | [GOVERNANCE.md](../../GOVERNANCE.md) | +| Public contribution RFC track | [rfcs/](../rfcs/) | + +The `docs/rfcs/` track is the **public, externally-authorable** RFC process. The +maintainer/internal RFCs below (`rfc-00N-*.md`) are a separate, team-owned +track; don't conflate the two. + ## Active Implementation Plans Working documents for in-flight feature work. Removed when the work lands. diff --git a/docs/rfcs/0000-template.md b/docs/rfcs/0000-template.md new file mode 100644 index 0000000..48f4bda --- /dev/null +++ b/docs/rfcs/0000-template.md @@ -0,0 +1,54 @@ +# RFC NNNN: + +| | | +|---|---| +| **Status** | Proposed | +| **Author(s)** | <your name / handle> | +| **Discussion** | <link to the originating Discussion, if any> | +| **Implementation** | <issue/PR links, filled in as work lands> | + +> Status is maintained by maintainers: `Proposed` while the PR is open, +> `Accepted` on merge, `Declined` on close, `Superseded by NNNN` later. + +## Summary + +One paragraph: what this changes, in plain terms. + +## Motivation + +What problem does this solve, and why is it worth the ongoing cost? Tie it to a +concrete need (a Discussion, a recurring issue, a user request). Per the +project's first principle, argue the *long-run liability*, not just the +short-term convenience. + +## Guide-level explanation + +Explain the change as you'd teach it to a user or contributor: new commands, +syntax, API shapes, behavior. Examples first. + +## Reference-level design + +The precise design: data structures, IR/AST/planner changes, storage/format +impact, migration path, error behavior. Enough that a reviewer can find the +holes. + +## Invariants & deny-list check + +Which Hard Invariants in [../dev/invariants.md](../dev/invariants.md) does this +touch? Does it brush against any deny-list item — and if so, why is this the +justified exception? State explicitly that no invariant is weakened, or which +Known Gap moves. + +## Drawbacks & alternatives + +What does this cost, what did you reject, and why. "Do nothing" is a valid +alternative to weigh. + +## Reversibility + +Is this reversible? On-disk/wire/format and substrate choices are near-permanent +and demand more evidence; a CLI flag or doc is cheap to undo. Say which this is. + +## Unresolved questions + +What's deliberately left open for review to settle. diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md new file mode 100644 index 0000000..99cdd76 --- /dev/null +++ b/docs/rfcs/README.md @@ -0,0 +1,66 @@ +# RFCs + +Substantial changes to OmniGraph — new user-facing surface, format or protocol +changes, anything irreversible or cross-cutting — go through a lightweight RFC +so the design is agreed *as reviewable code* before implementation starts. This +is the public RFC track, open to **anyone, including external contributors**. + +This complements the always-on review bar in +[../dev/invariants.md](../dev/invariants.md): the invariants say *what every +change must respect*; an RFC says *why this particular change is worth making and +how*. + +> **Two tracks, don't conflate them.** This `docs/rfcs/` directory is the +> **public contribution** track (anyone authors; maintainers accept). The +> maintainer-internal RFCs under `docs/dev/rfc-00N-*.md` are a separate, +> team-owned track for in-flight internal work. If you're an outside +> contributor, you're in the right place here. + +## When you need one + +- **RFC required:** new query/schema/CLI/HTTP surface; on-disk or wire-format + changes; a new substrate dependency; anything the deny-list in + [../dev/invariants.md](../dev/invariants.md) flags; anything irreversible + ("reversibility shapes evidence demand"). +- **RFC not required:** bug fixes for an `accepted` issue, and the trivial + fast-lane (typos, docs, deps) — see [../../CONTRIBUTING.md](../../CONTRIBUTING.md). + +If you're unsure, start a [Discussion](../../../discussions); a maintainer will +tell you whether it needs an RFC. + +## Lifecycle + +``` +Discussion (incubate, get rough consensus) + │ graduate + ▼ +RFC pull request → adds docs/rfcs/NNNN-title.md (Status: Proposed) + │ +maintainer review ──▶ changes requested / declined (PR closed, with rationale) + │ + ▼ +merged == Accepted (the merged file is the durable decision record) + │ + ▼ +Implementation PR(s) reference the accepted RFC +``` + +- **Author:** anyone. **Acceptance:** a maintainer decision, performed by + merging the RFC PR. Declining is closing it with rationale. +- The merged RFC *is* the accepted record — there is no separate sign-off step. +- Later reversals don't edit history: supersede with a new RFC that links back + and flip the old one's `Status` to `Superseded`. + +## Numbering & naming + +- File: `docs/rfcs/NNNN-kebab-title.md`, where `NNNN` is the next free + zero-padded integer (`0001`, `0002`, …). `0000-template.md` is reserved. +- Pick the number when you open the PR; if it collides with another in-flight + RFC, the second to merge bumps theirs. + +## Status values + +`Proposed` (open PR) · `Accepted` (merged) · `Declined` (closed) · +`Superseded by NNNN` · `Implemented` (set once the work lands, optional). + +Copy [0000-template.md](0000-template.md) to start. diff --git a/scripts/check-agents-md.sh b/scripts/check-agents-md.sh index abc6469..02a177a 100755 --- a/scripts/check-agents-md.sh +++ b/scripts/check-agents-md.sh @@ -34,10 +34,15 @@ PY canonical=() while IFS= read -r line; do canonical+=("$line") -done < <(find docs -type f -name '*.md' ! -path 'docs/releases/*' ! -path 'docs/internal/*' | sort) +done < <(find docs -type f -name '*.md' ! -path 'docs/releases/*' ! -path 'docs/internal/*' ! -path 'docs/rfcs/*' | sort) if [[ -d docs/releases ]]; then canonical+=("docs/releases/") fi +# RFCs are a growing collection (like releases): represent the directory, not +# every per-RFC file. The dir must be linked from an audience index. +if [[ -d docs/rfcs ]]; then + canonical+=("docs/rfcs/") +fi linked=() for index_file in "${index_files[@]}"; do From fd8e078a77fcce8be31b3ec3c18614427555b6fe Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sun, 7 Jun 2026 18:05:01 +0300 Subject: [PATCH 019/165] ci(codeowners): add aaltshuler to engineering role (#147) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Restores aaltshuler as an `engineering` code-owner (removed in #142), so `crates/**` and repo-infra PRs have a second reviewer besides the sole owner ragnorc — unblocking review of author-ragnorc PRs (e.g. #132) that ragnorc cannot self-approve. Edited the source of truth (.github/codeowners-roles.yml) and re-rendered .github/CODEOWNERS + the docs/dev/codeowners.md tables via .github/scripts/render-codeowners.py, per the documented flow. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- .github/CODEOWNERS | 4 ++-- .github/codeowners-roles.yml | 1 + docs/dev/codeowners.md | 6 +++--- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index d4ecfa5..e937724 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -8,9 +8,9 @@ # CI fails if this file drifts from its source, and rejects PRs that # edit this file directly without also editing the yml. -* @ragnorc +* @ragnorc @aaltshuler -crates/** @ragnorc +crates/** @ragnorc @aaltshuler docs/** @ragnorc README.md @ragnorc AGENTS.md @ragnorc diff --git a/.github/codeowners-roles.yml b/.github/codeowners-roles.yml index c5e36a9..ce4014d 100644 --- a/.github/codeowners-roles.yml +++ b/.github/codeowners-roles.yml @@ -22,6 +22,7 @@ roles: compiler. members: - ragnorc + - aaltshuler docs: description: > diff --git a/docs/dev/codeowners.md b/docs/dev/codeowners.md index 14bba0b..50c4dc7 100644 --- a/docs/dev/codeowners.md +++ b/docs/dev/codeowners.md @@ -14,8 +14,8 @@ The tables below are **generated** from `.github/codeowners-roles.yml` by `.gith | Path | Owners | Role(s) | |---|---|---| -| `*` | @ragnorc | engineering | -| `crates/**` | @ragnorc | engineering | +| `*` | @ragnorc @aaltshuler | engineering | +| `crates/**` | @ragnorc @aaltshuler | engineering | | `docs/**` | @ragnorc | docs | | `README.md` | @ragnorc | docs | | `AGENTS.md` | @ragnorc | docs | @@ -26,7 +26,7 @@ The tables below are **generated** from `.github/codeowners-roles.yml` by `.gith | Role | Members | Description | |---|---|---| -| `engineering` | @ragnorc | All production code under crates/**. Engine, CLI, server, compiler. | +| `engineering` | @ragnorc @aaltshuler | All production code under crates/**. Engine, CLI, server, compiler. | | `docs` | @ragnorc | Documentation under docs/**, plus repo-level docs (README.md, AGENTS.md, CLAUDE.md symlink, SECURITY.md). | <!-- END GENERATED OWNERSHIP --> From 54842808dbd981e61e0a4be2cf987fc0a52b2584 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford <ragnor.comerford@gmail.com> Date: Sun, 7 Jun 2026 17:33:14 +0200 Subject: [PATCH 020/165] feat(engine): sweep & remove legacy __run__ branch guard (MR-770) (#132) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(engine): sweep legacy __run__ branches via v2→v3 manifest migration Pre-v0.4.0 graphs can carry stale `__run__<id>` staging branches on the `__manifest` dataset, left by the Run state machine removed in MR-771. Lance's `list_branches` still enumerates them, so they leak into `branch_list()` and count as blocking branches at schema-apply time. Add a one-time `migrate_v2_to_v3` arm to the internal-schema dispatcher: on the first read-write open it enumerates `__manifest` branches, deletes every `__run__*` ref, and bumps the stamp to 3. Idempotent under retry (re-enumerates fresh each run). The `"__run__"` prefix is inlined so the migration does not depend on the run_registry guard that MR-770 removes next. This is the prerequisite sweep; the guard removal follows in the next commit. * refactor(engine): remove the legacy __run__ branch guard (MR-770) With the v2→v3 migration sweeping stale `__run__*` branches off `__manifest` on first read-write open, the defense-in-depth `is_internal_run_branch` guard is no longer needed. - delete `db/run_registry.rs`; drop the module + re-export from `db/mod.rs` - collapse `is_internal_system_branch` to the schema-apply-lock check only - `ensure_public_branch_ref`: drop the run-ref rejection; `__run__*` is now an ordinary branch name - `branch_merge`: reject `is_internal_system_branch` (was run-only) so the schema-apply lock is rejected consistently with create/delete — a small, deliberate tightening - update the inline schema-apply test + the writes integration tests (`public_branch_apis_reject_internal_run_refs` → `public_branch_apis_reject_internal_system_refs`, which also asserts `__run__*` now creates successfully) - docs: flip the "pending production sweep / defense-in-depth" notes to "auto-swept by the v2→v3 migration"; document the read-only-open limitation Known residual: the inert `_graph_runs.lance` / `_graph_run_actors.lance` bytes remain until a `StorageAdapter::delete_prefix` primitive lands. * fix(engine): run __run__ sweep at Omnigraph::open, not only on publish Review (PR #132) caught a regression: removing __run__ from `is_internal_system_branch` exposed legacy `__run__*` branches to the schema-apply blocking-branch checks (schema_apply.rs:104 and :778) and to `branch_list()`, but the v2→v3 sweep ran only inside the publisher's `load_publish_state`. On a pre-v0.4.0 graph whose first write is a schema apply, the blocking-branch check fires before any publish, so apply failed with "found non-main branches: __run__…". The same lazy timing also created a reverse hazard: a user-created `__run__*` branch on a still-v2 graph could be deleted by the first publish's sweep. Fix: run the internal-schema migration in `Omnigraph::open(ReadWrite)` (new `manifest::migrate_on_open`), before the coordinator reads branch state. The sweep now lands before any branch-observing code, and a graph is stamped v3 at open — so the one-time sweep can never catch a legitimately-created branch. Both checks and `branch_list` see the swept graph; correct by construction for every write path. Accepted residual: a read-only open of an unmigrated legacy graph still lists `__run__*` (read-only opens must not write, so they can't sweep). Documented. Regression test `legacy_run_branch_is_swept_on_open_and_does_not_block_schema_apply` confirmed RED before the fix (panicked on the branch_list leak assertion) and GREEN after. Also updates the stale schema_apply.rs comment, the writes.md "Migration code" section, and adds the v3 row to storage.md's migration table. * test(engine): sweep multiple legacy __run__ branches; doc nit Strengthen the v2→v3 migration test to synthesize three `__run__*` branches (a real legacy graph accumulates one per run) so the migration's delete loop is exercised on a single reused dataset handle, not just a single branch. Confirms multi-branch deletion is safe. Also drop a stale "active runs" reference from the branch_delete doc line. * fix(engine): force-delete in __run__ sweep for concurrency safety `migrate_v2_to_v3` ran `Dataset::delete_branch` (= `branches().delete(.., false)`), which errors "BranchContents not found" if the branch is already gone. Since the sweep now runs in `Omnigraph::open(ReadWrite)`, two processes opening the same legacy v2 graph concurrently would race: one wins each delete, the other's open fails. The migration only claimed idempotency under *sequential* retry. Switch to `Dataset::force_delete_branch` (= `delete(.., true)`), Lance's documented path for cleaning up zombie branches, which tolerates an already-absent branch. The sweep is now idempotent under concurrent runners and robust to partial/zombie state. Found in self-review; no behavior change for the common single-open path. * docs(release): note MR-770 __run__ cleanup in v0.6.1 * docs(branches): reconcile branch cleanup semantics --- crates/omnigraph/src/db/manifest.rs | 16 ++++ .../omnigraph/src/db/manifest/migrations.rs | 55 ++++++++++++- crates/omnigraph/src/db/manifest/tests.rs | 74 +++++++++++++++++ crates/omnigraph/src/db/mod.rs | 7 +- crates/omnigraph/src/db/omnigraph.rs | 81 +++++++++++++++---- .../src/db/omnigraph/schema_apply.rs | 10 +-- crates/omnigraph/src/db/run_registry.rs | 16 ---- crates/omnigraph/src/exec/merge.rs | 4 +- crates/omnigraph/src/exec/mod.rs | 2 +- crates/omnigraph/tests/writes.rs | 33 ++++---- docs/dev/writes.md | 18 +++-- docs/releases/v0.6.1.md | 2 + docs/user/audit.md | 2 +- docs/user/branches-commits.md | 10 +-- docs/user/constants.md | 6 +- docs/user/storage.md | 5 +- 16 files changed, 269 insertions(+), 72 deletions(-) delete mode 100644 crates/omnigraph/src/db/run_registry.rs diff --git a/crates/omnigraph/src/db/manifest.rs b/crates/omnigraph/src/db/manifest.rs index 7fcf7de..3b2886f 100644 --- a/crates/omnigraph/src/db/manifest.rs +++ b/crates/omnigraph/src/db/manifest.rs @@ -48,6 +48,22 @@ const OBJECT_TYPE_TABLE_VERSION: &str = "table_version"; const OBJECT_TYPE_TABLE_TOMBSTONE: &str = "table_tombstone"; const TABLE_VERSION_MANAGEMENT_KEY: &str = "table_version_management"; +/// Apply pending internal-schema migrations against `__manifest` on the +/// open-for-write path, independent of a publish. +/// +/// `Omnigraph::open(ReadWrite)` calls this before the coordinator reads branch +/// state, so branch-observing code (`branch_list`, the schema-apply +/// blocking-branch checks) sees the post-migration graph. In particular the +/// v2→v3 step sweeps legacy `__run__*` staging branches off `__manifest` +/// (MR-770); running it here closes the window where those branches would +/// otherwise block schema apply before the first publish runs the migration. +/// +/// Idempotent: a no-op stamp read when the on-disk version already matches. +pub(crate) async fn migrate_on_open(root_uri: &str) -> Result<()> { + let mut dataset = open_manifest_dataset(root_uri, None).await?; + migrations::migrate_internal_schema(&mut dataset).await +} + /// Immutable point-in-time view of the database. /// /// Cheap to create (no storage I/O). All reads within a query go through one diff --git a/crates/omnigraph/src/db/manifest/migrations.rs b/crates/omnigraph/src/db/manifest/migrations.rs index bbb7995..e2801fe 100644 --- a/crates/omnigraph/src/db/manifest/migrations.rs +++ b/crates/omnigraph/src/db/manifest/migrations.rs @@ -46,7 +46,11 @@ use crate::error::{OmniError, Result}; /// - v2 — `__manifest.object_id` carries the unenforced-PK annotation, /// engaging Lance's bloom-filter conflict resolver at commit time. Added /// alongside `expected_table_versions` OCC on `ManifestBatchPublisher::publish`. -pub(super) const INTERNAL_MANIFEST_SCHEMA_VERSION: u32 = 2; +/// - v3 — one-time sweep of legacy `__run__<id>` staging branches left on the +/// `__manifest` dataset by the pre-v0.4.0 Run state machine (removed in +/// MR-771). Once swept, the `is_internal_run_branch` defense-in-depth guard +/// is no longer needed (MR-770). +pub(super) const INTERNAL_MANIFEST_SCHEMA_VERSION: u32 = 3; const INTERNAL_SCHEMA_VERSION_KEY: &str = "omnigraph:internal_schema_version"; const OBJECT_ID_PK_KEY: &str = "lance-schema:unenforced-primary-key"; @@ -89,6 +93,10 @@ pub(super) async fn migrate_internal_schema(dataset: &mut Dataset) -> Result<()> migrate_v1_to_v2(dataset).await?; current = 2; } + 2 => { + migrate_v2_to_v3(dataset).await?; + current = 3; + } other => { return Err(OmniError::manifest_internal(format!( "no internal-schema migration registered for v{} → v{}", @@ -122,6 +130,51 @@ async fn migrate_v1_to_v2(dataset: &mut Dataset) -> Result<()> { set_stamp(dataset, 2).await } +/// v2 → v3: sweep legacy `__run__<id>` staging branches off the `__manifest` +/// dataset, then bump the stamp. +/// +/// The pre-v0.4.0 Run state machine (removed in MR-771) created graph-level +/// staging branches named `__run__<ulid>` on `__manifest`. MR-771 stopped +/// creating them but left any pre-existing ones in place; Lance's +/// `list_branches` still enumerates them, so they leak into `branch_list()` +/// and count as blocking branches at schema-apply time. This one-time sweep +/// removes them so the `is_internal_run_branch` guard can retire (MR-770). +/// +/// The `"__run__"` prefix is inlined here on purpose: this migration must keep +/// working after the `run_registry` module (the guard) is deleted, so it does +/// not depend on it. +/// +/// Idempotent under both sequential retry and concurrent runners: each run +/// re-enumerates `list_branches` fresh, and `force_delete_branch` tolerates a +/// branch that is already gone — so a crash before the stamp bump, or a second +/// process opening the same legacy graph at the same time, never errors out. +async fn migrate_v2_to_v3(dataset: &mut Dataset) -> Result<()> { + const LEGACY_RUN_BRANCH_PREFIX: &str = "__run__"; + let branches = dataset + .list_branches() + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + let run_branches: Vec<String> = branches + .into_keys() + .filter(|name| { + name.trim_start_matches('/') + .starts_with(LEGACY_RUN_BRANCH_PREFIX) + }) + .collect(); + for name in run_branches { + // `force_delete_branch` deletes even when the `BranchContents` is + // already gone. Plain `delete_branch` errors "BranchContents not + // found", which would fail a second concurrent open (or a retry that + // raced another runner) after the first one swept the branch. Force is + // exactly Lance's documented path for cleaning up zombie branches. + dataset + .force_delete_branch(&name) + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + } + set_stamp(dataset, 3).await +} + async fn set_stamp(dataset: &mut Dataset, version: u32) -> Result<()> { dataset .update_schema_metadata([(INTERNAL_SCHEMA_VERSION_KEY.to_string(), version.to_string())]) diff --git a/crates/omnigraph/src/db/manifest/tests.rs b/crates/omnigraph/src/db/manifest/tests.rs index effa0b5..885a2a8 100644 --- a/crates/omnigraph/src/db/manifest/tests.rs +++ b/crates/omnigraph/src/db/manifest/tests.rs @@ -1461,6 +1461,80 @@ async fn test_publish_migrates_pre_stamp_manifest_to_current_version() { assert!(reopened.snapshot().entry("node:Person").is_some()); } +#[tokio::test] +async fn test_v2_to_v3_sweeps_legacy_run_branches_on_write_open() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let catalog = build_test_catalog(); + let mut mc = ManifestCoordinator::init(uri, &catalog).await.unwrap(); + + // Synthesize a pre-MR-770 graph: several stale `__run__` staging branches + // left on `__manifest` (a real legacy graph accumulates one per run), plus + // a real user branch that must survive the sweep. Multiple run branches + // exercise the migration's delete loop on a single reused dataset handle. + mc.create_branch("__run__01J9LEGACY").await.unwrap(); + mc.create_branch("__run__01J9SECOND").await.unwrap(); + mc.create_branch("__run__01J9THIRD").await.unwrap(); + mc.create_branch("feature").await.unwrap(); + let before = mc.list_branches().await.unwrap(); + assert_eq!( + before.iter().filter(|b| b.starts_with("__run__")).count(), + 3, + "precondition: three legacy run branches exist on __manifest; got {before:?}", + ); + + // Rewind the internal-schema stamp to v2 so the next write-open runs the + // v2 → v3 sweep arm (init stamps at the current version, which is past it). + { + let mut ds = open_manifest_dataset(uri, None).await.unwrap(); + ds.update_schema_metadata([( + "omnigraph:internal_schema_version".to_string(), + Some("2".to_string()), + )]) + .await + .unwrap(); + let post = open_manifest_dataset(uri, None).await.unwrap(); + assert_eq!(super::migrations::read_stamp(&post), 2, "stamp rewound to v2"); + } + + // A no-op publish forces the open-for-write path, which runs the migration. + let mut expected = HashMap::new(); + expected.insert("node:Person".to_string(), 1); + GraphNamespacePublisher::new(uri, None) + .publish(&[], &expected) + .await + .unwrap(); + + // Stamp advanced to current; the legacy run branch is physically gone from + // `__manifest` (checked via the raw, unfiltered manifest list — not the + // guard-filtered `branch_list`), and the real branch + `main` survive. + let post = open_manifest_dataset(uri, None).await.unwrap(); + assert_eq!( + super::migrations::read_stamp(&post), + super::migrations::INTERNAL_MANIFEST_SCHEMA_VERSION, + ); + let reopened = ManifestCoordinator::open(uri).await.unwrap(); + let after = reopened.list_branches().await.unwrap(); + assert!( + !after.iter().any(|b| b.starts_with("__run__")), + "legacy run branch must be swept; got {after:?}", + ); + assert!(after.iter().any(|b| b == "feature"), "user branch must survive"); + assert!(after.iter().any(|b| b == "main"), "main must survive"); + + // Idempotent: a second write-open finds the stamp at current and does not + // re-run the sweep or error. + GraphNamespacePublisher::new(uri, None) + .publish(&[], &expected) + .await + .unwrap(); + let final_ds = open_manifest_dataset(uri, None).await.unwrap(); + assert_eq!( + super::migrations::read_stamp(&final_ds), + super::migrations::INTERNAL_MANIFEST_SCHEMA_VERSION, + ); +} + #[tokio::test] async fn test_publish_rejects_manifest_stamped_at_future_version() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/omnigraph/src/db/mod.rs b/crates/omnigraph/src/db/mod.rs index 8702f88..13e1c74 100644 --- a/crates/omnigraph/src/db/mod.rs +++ b/crates/omnigraph/src/db/mod.rs @@ -3,7 +3,6 @@ pub mod graph_coordinator; pub mod manifest; mod omnigraph; mod recovery_audit; -mod run_registry; mod schema_state; pub(crate) mod write_queue; @@ -15,7 +14,6 @@ pub use omnigraph::{ CleanupPolicyOptions, InitOptions, MergeOutcome, Omnigraph, OpenMode, SchemaApplyOptions, SchemaApplyResult, SkipReason, TableCleanupStats, TableOptimizeStats, }; -pub(crate) use run_registry::is_internal_run_branch; pub(crate) const SCHEMA_APPLY_LOCK_BRANCH: &str = "__schema_apply_lock__"; @@ -69,5 +67,8 @@ pub(crate) fn is_schema_apply_lock_branch(name: &str) -> bool { } pub(crate) fn is_internal_system_branch(name: &str) -> bool { - is_internal_run_branch(name) || is_schema_apply_lock_branch(name) + // Legacy `__run__*` staging branches (Run state machine, removed MR-771) + // are swept off `__manifest` by the v2→v3 internal-schema migration, so the + // only internal branch the engine still creates is the schema-apply lock. + is_schema_apply_lock_branch(name) } diff --git a/crates/omnigraph/src/db/omnigraph.rs b/crates/omnigraph/src/db/omnigraph.rs index 7b8a3f6..ba2b70e 100644 --- a/crates/omnigraph/src/db/omnigraph.rs +++ b/crates/omnigraph/src/db/omnigraph.rs @@ -346,6 +346,16 @@ impl Omnigraph { mode: OpenMode, ) -> Result<Self> { let root = normalize_root_uri(uri)?; + // Apply pending internal-schema migrations before the coordinator reads + // branch state, so `branch_list` and the schema-apply blocking-branch + // checks observe the post-migration graph — notably the v2→v3 sweep of + // legacy `__run__*` staging branches (MR-770). ReadWrite only: a + // read-only open must not trigger object-store writes, so a read-only + // open of an unmigrated legacy graph still lists `__run__*` until its + // first read-write open (an accepted, documented limitation). + if matches!(mode, OpenMode::ReadWrite) { + crate::db::manifest::migrate_on_open(&root).await?; + } // Open the coordinator first so the schema-staging recovery sweep can // compare its snapshot against any leftover staging files. let mut coordinator = GraphCoordinator::open(&root, Arc::clone(&storage)).await?; @@ -1491,12 +1501,6 @@ pub(crate) fn normalize_branch_name(branch: &str) -> Result<Option<String>> { } pub(crate) fn ensure_public_branch_ref(branch: &str, operation: &str) -> Result<()> { - if super::is_internal_run_branch(branch) { - return Err(OmniError::manifest(format!( - "{} does not allow internal run ref '{}'", - operation, branch - ))); - } if is_internal_system_branch(branch) { return Err(OmniError::manifest(format!( "{} does not allow internal system ref '{}'", @@ -1900,7 +1904,6 @@ fn json_value_from_array(array: &dyn Array, row: usize) -> Result<serde_json::Va #[cfg(test)] mod tests { use super::*; - use crate::db::is_internal_run_branch; use crate::db::manifest::ManifestCoordinator; use async_trait::async_trait; use serde_json::Value; @@ -2238,11 +2241,11 @@ edge WorksAt: Person -> Company #[tokio::test] async fn test_apply_schema_succeeds_after_load() { // Historical: schema apply used to be blocked by leftover - // `__run__` branches. A defense-in-depth filter now skips - // internal system branches, and run branches were made - // ephemeral on every terminal state — so in practice no - // `__run__` branch survives publish. The filter still guards - // the invariant. + // `__run__` branches. The Run state machine was removed in + // MR-771, so a fresh graph never creates a `__run__` branch; + // legacy ones are swept by the v2→v3 manifest migration. This + // asserts the invariant a current graph upholds: publish leaves + // no `__run__` branch behind, so schema apply proceeds. let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); @@ -2257,8 +2260,8 @@ edge WorksAt: Person -> Company let all_branches = db.coordinator.read().await.all_branches().await.unwrap(); assert!( - !all_branches.iter().any(|b| is_internal_run_branch(b)), - "run branch should be deleted after publish, got: {:?}", + !all_branches.iter().any(|b| b.starts_with("__run__")), + "no __run__ branch should exist after publish, got: {:?}", all_branches ); @@ -2270,6 +2273,56 @@ edge WorksAt: Person -> Company assert!(result.applied, "schema apply should have applied"); } + /// Regression (MR-770): a pre-v0.4.0 graph that still carries a stale + /// `__run__*` branch on `__manifest` must not block schema apply. The + /// v2→v3 sweep runs in `Omnigraph::open(ReadWrite)` — before the + /// schema-apply blocking-branch check — so apply succeeds with no + /// intervening publish. + /// + /// Confirmed to fail before the open-time migration landed: the reopened + /// graph still listed `__run__legacy`, and `apply_schema` returned + /// "found non-main branches: __run__legacy". + #[tokio::test] + async fn legacy_run_branch_is_swept_on_open_and_does_not_block_schema_apply() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + + // Synthesize a legacy graph: a stale `__run__` branch on `__manifest` + // plus the manifest stamp rewound to v2 (pre-sweep). + db.branch_create("__run__legacy").await.unwrap(); + drop(db); + { + let mut ds = lance::Dataset::open(&format!("{}/__manifest", uri)) + .await + .unwrap(); + ds.update_schema_metadata([( + "omnigraph:internal_schema_version".to_string(), + Some("2".to_string()), + )]) + .await + .unwrap(); + } + + // Reopen (ReadWrite): the open-time migration must sweep `__run__legacy` + // before any branch-observing code runs. + let db = Omnigraph::open(uri).await.unwrap(); + let branches = db.branch_list().await.unwrap(); + assert!( + !branches.iter().any(|b| b.starts_with("__run__")), + "open-time migration must sweep legacy __run__ branches; got {branches:?}", + ); + + // Schema apply must proceed with no intervening publish — the + // blocking-branch check no longer sees `__run__legacy`. + let desired = TEST_SCHEMA.replace( + " age: I32?\n}", + " age: I32?\n nickname: String?\n}", + ); + let result = db.apply_schema(&desired).await.unwrap(); + assert!(result.applied, "schema apply should have applied"); + } + #[tokio::test] async fn test_apply_schema_adds_index_for_existing_property() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/omnigraph/src/db/omnigraph/schema_apply.rs b/crates/omnigraph/src/db/omnigraph/schema_apply.rs index 35fe161..7cb3193 100644 --- a/crates/omnigraph/src/db/omnigraph/schema_apply.rs +++ b/crates/omnigraph/src/db/omnigraph/schema_apply.rs @@ -61,11 +61,11 @@ async fn plan_schema_for_apply( ) -> Result<PlannedSchemaApply> { db.ensure_schema_state_valid().await?; let branches = db.coordinator.read().await.all_branches().await?; - // Skip `main` and internal system branches. The schema-apply lock branch - // is excluded because it is the cluster-wide schema-apply serializer. - // `__run__*` branches are no longer created; the filter remains as - // defense-in-depth for legacy graphs with leftover staging branches. - // A future production sweep will let this guard go. + // Skip `main` and internal system branches (the schema-apply lock branch, + // the cluster-wide schema-apply serializer). Legacy `__run__*` staging + // branches were swept off `__manifest` by the v2→v3 migration that runs in + // `Omnigraph::open(ReadWrite)` before this check (MR-770), so they no + // longer appear here. let blocking_branches = branches .into_iter() .filter(|branch| branch != "main" && !is_internal_system_branch(branch)) diff --git a/crates/omnigraph/src/db/run_registry.rs b/crates/omnigraph/src/db/run_registry.rs deleted file mode 100644 index ee3d336..0000000 --- a/crates/omnigraph/src/db/run_registry.rs +++ /dev/null @@ -1,16 +0,0 @@ -// The Run state machine has been removed. Mutations now write directly -// to target tables and use the publisher's `expected_table_versions` -// CAS for cross-table OCC; `__run__<id>` staging branches and the -// `_graph_runs.lance` state machine no longer exist. -// -// What remains is the branch-name predicate, kept as a defense-in-depth -// guard against users naming a public branch `__run__*`. A future -// production sweep of legacy `_graph_runs.lance` rows and stale -// `__run__*` branches will let this predicate (and this file) go too. - -pub(crate) const INTERNAL_RUN_BRANCH_PREFIX: &str = "__run__"; - -pub(crate) fn is_internal_run_branch(name: &str) -> bool { - name.trim_start_matches('/') - .starts_with(INTERNAL_RUN_BRANCH_PREFIX) -} diff --git a/crates/omnigraph/src/exec/merge.rs b/crates/omnigraph/src/exec/merge.rs index 2e5f32e..eb6c4a3 100644 --- a/crates/omnigraph/src/exec/merge.rs +++ b/crates/omnigraph/src/exec/merge.rs @@ -1087,9 +1087,9 @@ impl Omnigraph { target: &str, actor_id: Option<&str>, ) -> Result<MergeOutcome> { - if is_internal_run_branch(source) || is_internal_run_branch(target) { + if is_internal_system_branch(source) || is_internal_system_branch(target) { return Err(OmniError::manifest(format!( - "branch_merge does not allow internal run refs ('{}' -> '{}')", + "branch_merge does not allow internal system refs ('{}' -> '{}')", source, target ))); } diff --git a/crates/omnigraph/src/exec/mod.rs b/crates/omnigraph/src/exec/mod.rs index 33a7e41..ce72d42 100644 --- a/crates/omnigraph/src/exec/mod.rs +++ b/crates/omnigraph/src/exec/mod.rs @@ -35,7 +35,7 @@ use time::format_description::well_known::Rfc3339; use crate::db::commit_graph::CommitGraph; use crate::db::manifest::ManifestCoordinator; -use crate::db::{MergeOutcome, Omnigraph, is_internal_run_branch}; +use crate::db::{MergeOutcome, Omnigraph, is_internal_system_branch}; use crate::db::{ReadTarget, Snapshot}; use crate::embedding::EmbeddingClient; use crate::error::{MergeConflict, MergeConflictKind, OmniError, Result}; diff --git a/crates/omnigraph/tests/writes.rs b/crates/omnigraph/tests/writes.rs index 13cb10f..0a309c9 100644 --- a/crates/omnigraph/tests/writes.rs +++ b/crates/omnigraph/tests/writes.rs @@ -371,11 +371,10 @@ async fn cancelled_mutation_future_leaves_no_state() { // Cancel-safety property: no graph-level run/staging state remains. // - // Note: `branch_list()` already filters `__run__*` via - // `is_internal_system_branch`, so a runtime "no `__run__` branches" check - // would be vacuous. The structural property that no `__run__` branches - // can ever be created is enforced by deletion of `begin_run` etc. in - // (verified by the build itself — those symbols no longer exist). + // No `__run__` branches can ever be created: the Run state machine + // (`begin_run` etc.) was deleted in MR-771 — verified by the build itself, + // those symbols no longer exist. Any legacy `__run__*` branch on an + // upgraded graph is swept by the v2→v3 manifest migration. // // (1) The branch list is unchanged: cancellation/completion cannot // synthesize new public branches. @@ -442,34 +441,40 @@ async fn repeated_loads_do_not_accumulate_branches() { assert_eq!(db.branch_list().await.unwrap(), vec!["main".to_string()]); } -/// User code must not be able to write to internal `__run__*` names. -/// The branch-name guard predicate is kept as defense-in-depth; it -/// will be removed once a future production sweep retires the legacy -/// branches. +/// After MR-770, `__run__*` is an ordinary branch name — the Run state machine +/// and its `is_internal_run_branch` guard are gone. The surviving internal-ref +/// guard still rejects the active `__schema_apply_lock__` branch on the public +/// create/merge APIs. #[tokio::test] -async fn public_branch_apis_reject_internal_run_refs() { +async fn public_branch_apis_reject_internal_system_refs() { let dir = tempfile::tempdir().unwrap(); let mut db = init_and_load(&dir).await; - let create_err = db.branch_create("__run__synthetic").await.unwrap_err(); + // `__run__*` is no longer reserved — creating it now succeeds. + db.branch_create("__run__formerly_reserved") + .await + .expect("__run__ prefix is a normal branch name post-MR-770"); + + // The schema-apply lock branch is still rejected on public branch APIs. + let create_err = db.branch_create("__schema_apply_lock__").await.unwrap_err(); let OmniError::Manifest(err) = create_err else { panic!("expected Manifest error"); }; assert!( - err.message.contains("internal run ref"), + err.message.contains("internal system ref"), "unexpected error: {}", err.message ); let merge_err = db - .branch_merge("__run__synthetic", "main") + .branch_merge("__schema_apply_lock__", "main") .await .unwrap_err(); let OmniError::Manifest(err) = merge_err else { panic!("expected Manifest error"); }; assert!( - err.message.contains("internal run refs"), + err.message.contains("internal system refs"), "unexpected error: {}", err.message ); diff --git a/docs/dev/writes.md b/docs/dev/writes.md index 974f7a6..8b692b4 100644 --- a/docs/dev/writes.md +++ b/docs/dev/writes.md @@ -14,8 +14,11 @@ publisher's row-level CAS on `__manifest` is the single fence. - No `RunRecord`, no `_graph_runs.lance`, no `_graph_run_actors.lance`. - No `omnigraph run *` CLI subcommands and no `/runs/*` HTTP endpoints. -- No `__run__<id>` staging branches. (Legacy on-disk artifacts from - pre-MR-771 repos are inert; MR-770 sweeps them in production.) +- No `__run__<id>` staging branches; `__run__*` is no longer a reserved + name. The branch-name guard was removed in MR-770, and any stale + `__run__*` branch on an upgraded graph is swept off `__manifest` by the + v2→v3 internal-schema migration on first read-write open. (The inert + `_graph_runs.lance` bytes remain until a `delete_prefix` primitive lands.) - Cancelled mutation futures leave **no graph-level state** — only orphaned Lance fragments, which the existing `omnigraph cleanup` pipe reclaims. @@ -245,9 +248,14 @@ list`. ## Migration code -`db/manifest/migrations.rs` does not change. Active deletion of -`_graph_runs.lance` belongs in MR-770 (the production sweep) — this PR -stops *creating* run state but does not destroy legacy bytes on disk. +`db/manifest/migrations.rs` carries the v2→v3 internal-schema step (MR-770): +a one-time sweep that deletes legacy `__run__*` staging branches off +`__manifest`. It runs in `Omnigraph::open(ReadWrite)` (via +`manifest::migrate_on_open`, before the coordinator reads branch state) and +again on the publisher's write path; both are idempotent once the stamp is at +v3. Deleting the inert `_graph_runs.lance` / `_graph_run_actors.lance` dataset +*bytes* is still deferred — it needs a `StorageAdapter::delete_prefix` +primitive — but those bytes are invisible to graph-level state. ## Mid-query partial failure: closed by MR-794 diff --git a/docs/releases/v0.6.1.md b/docs/releases/v0.6.1.md index aafe1af..0acc34b 100644 --- a/docs/releases/v0.6.1.md +++ b/docs/releases/v0.6.1.md @@ -7,6 +7,7 @@ v0.6.1 focuses on operational polish after v0.6.0: stored-query registries, safe - **Stored-query registries.** `omnigraph.yaml` can declare curated `queries:` blocks per graph. Servers load and type-check them at startup, `omnigraph queries validate` checks them offline, `omnigraph queries list` shows exposed queries and typed params, `GET /queries` exposes a typed catalog, and `POST /queries/{name}` invokes a stored query without accepting ad hoc `.gq` source from the client. - **Stored-query policy gate.** New Cedar action `invoke_query` gates the stored-query invocation surface. Stored mutations are double-gated: `invoke_query` to reach the stored query and `change` for the actual write. - **Safer branch deletion.** `branch_delete` now treats the manifest as the authority, flips branch visibility atomically, and reclaims per-table/commit-graph forks as derived state. If best-effort reclaim is interrupted, `cleanup` reconciles orphaned forks; reusing a branch name before cleanup reports an actionable error. +- **Legacy `__run__` cleanup (MR-770).** Removed the last functional remnant of the Run state machine (retired in v0.4.0): the `__run__` branch-name guard. A new v2→v3 `__manifest` internal-schema migration sweeps any stale `__run__*` staging branches on the first read-write open, so `__run__*` is no longer a reserved branch name. This closes the "unpromoted `__run__` branches block reads" condition behind the zombie-run cascade incident; the inert `_graph_runs.lance` row cleanup is tracked separately (it needs a `delete_prefix` primitive). - **Blob-safe optimize.** `omnigraph optimize` skips tables with `Blob` properties instead of failing the whole sweep on Lance's blob-v2 compaction decode bug. Skips are visible in human output, `--json` as `skipped`, `TableOptimizeStats.skipped`, and logs; non-blob tables still compact normally. - **Deployment improvements.** The container entrypoint now composes `OMNIGRAPH_TARGET_URI` with `OMNIGRAPH_CONFIG`, so operators can keep the graph URI in env while loading policy/query config from a mounted file. The local RustFS bootstrap pins RustFS beta.3 and allows the current insecure local-dev default credentials. - **Windows release support.** Tagged and edge releases now publish Windows x86_64 archives containing `omnigraph.exe` and `omnigraph-server.exe`, with a PowerShell installer and Windows install docs. @@ -17,6 +18,7 @@ v0.6.1 focuses on operational polish after v0.6.0: stored-query registries, safe - A graph selected by name (`--target` or `server.graph`) now uses `graphs.<name>.policy` and `graphs.<name>.queries`. Top-level `policy` / `queries` blocks are only for anonymous bare-URI single-graph mode; using them with a named graph now fails loudly with migration guidance. - `mcp.expose` defaults to `true` for stored-query registry entries. Set `mcp: { expose: false }` for service-only queries that should not appear in the catalog. - `invoke_query` is graph-scoped, not branch-scoped. Branch/snapshot access remains enforced by the inner `read` / `change` gate. +- **Legacy `__run__` migration.** Graphs created before v0.4.0 are migrated automatically on the first **read-write** open by a v0.6.1 binary (one-time `__manifest` stamp v2→v3 sweep of stale `__run__*` branches). No action required. Two caveats: (1) a graph opened **read-only** still lists any stale `__run__*` branch until its first read-write open, since the migration is write-path-only like all manifest migrations — long-lived read-only deployments should be opened read-write once after upgrading; (2) the inert `_graph_runs.lance` / `_graph_run_actors.lance` dataset bytes are left in place until a future `delete_prefix` primitive (they are invisible to graph-level state). - Blob tables are not compacted until the upstream Lance fix lands, so fragment count and deleted-row space on blob tables are not reclaimed by `optimize`. Reads, writes, and query results are unaffected; no on-disk migration is required. - `TableOptimizeStats` is now `#[non_exhaustive]` and gains a `skipped: Option<SkipReason>` field (so does the new `SkipReason` enum). This is a source-level change only for downstream code that built this returned result struct by literal — rare, since it is produced by `optimize` and consumed by reading its fields; field access is unaffected, and `#[non_exhaustive]` keeps future additions non-breaking. diff --git a/docs/user/audit.md b/docs/user/audit.md index e8abe5b..ab028ac 100644 --- a/docs/user/audit.md +++ b/docs/user/audit.md @@ -4,4 +4,4 @@ - `_as` variants of every write API let callers override the actor: `mutate_as`, `ingest_as`, `branch_merge_as`, `apply_schema_as`, etc. - Actor IDs are persisted on `GraphCommit.actor_id` with split storage in `_graph_commit_actors.lance` (the commit graph is split into `_graph_commits.lance` for the linkage and `_graph_commit_actors.lance` for the actor map). - HTTP server uses the bearer-token actor automatically; CLI uses the local user / explicit env (no implicit actor). -- Pre-v0.4.0 graphs also stored actor IDs on `RunRecord.actor_id` in `_graph_runs.lance` / `_graph_run_actors.lance`. The Run state machine was removed in MR-771; those files are inert post-v0.4.0 and reclaimed by MR-770's production sweep. +- Pre-v0.4.0 graphs also stored actor IDs on `RunRecord.actor_id` in `_graph_runs.lance` / `_graph_run_actors.lance`. The Run state machine was removed in MR-771; those files are inert post-v0.4.0. The v2→v3 manifest migration sweeps any stale `__run__*` branches on first write-open (MR-770); the inert dataset bytes remain until a `delete_prefix` primitive lands. diff --git a/docs/user/branches-commits.md b/docs/user/branches-commits.md index c1894f9..0565186 100644 --- a/docs/user/branches-commits.md +++ b/docs/user/branches-commits.md @@ -9,8 +9,8 @@ Lance supports branching at the dataset level: a branch is a named lineage of ve OmniGraph builds *graph branches* on top by branching every sub-table coherently: - `branch_create(name)` / `branch_create_from(target, name)` — disallowed name `main`; fails if branch exists; ensures the schema-apply lock is idle. Atomic and authority-first like `branch_delete`: it flips the `__manifest` branch (authority), then creates the derived commit-graph branch, force-dropping any orphaned commit-graph ref left by an incomplete prior delete (the manifest branch is fresh, so a same-named commit-graph branch is provably a zombie). If commit-graph creation fails, the manifest branch is rolled back so the name never half-exists. -- `branch_list()` — returns public branches, **filters internal** `__run__…` and `__schema_apply_lock__` prefixes. -- `branch_delete(name)` — refuses if there are descendants or active runs on the branch. The manifest is the single authority for branch existence: deletion flips the `__manifest` branch ref first (one atomic op), after which the branch is gone from every snapshot. The owned per-table forks and the commit-graph branch are derived state, reclaimed best-effort with `force_delete_branch` after the flip. A failure during that reclaim (transient object-store error) does not fail the call or block the authority flip; the leftover forks are unreachable orphans that the [`cleanup`](maintenance.md) reconciler converges. One consequence: if a delete's best-effort reclaim fails, reusing that branch name before the next `cleanup` surfaces a clear error pointing at `cleanup` (the stale fork would otherwise collide on first write). +- `branch_list()` — returns public branches, **filters the internal** `__schema_apply_lock__` branch. +- `branch_delete(name)` — refuses if there are descendants on the branch, or if it is the current branch. The manifest is the single authority for branch existence: deletion flips the `__manifest` branch ref first (one atomic op), after which the branch is gone from every snapshot. The owned per-table forks and the commit-graph branch are derived state, reclaimed best-effort with `force_delete_branch` after the flip. A failure during that reclaim (transient object-store error) does not fail the call or block the authority flip; the leftover forks are unreachable orphans that the [`cleanup`](maintenance.md) reconciler converges. One consequence: if a delete's best-effort reclaim fails, reusing that branch name before the next `cleanup` surfaces a clear error pointing at `cleanup` (the stale fork would otherwise collide on first write). - **Lazy forking**: a branch only forks a sub-table when that sub-table is first mutated on it. Pure-read branches share fragments with their source. A fork collision is classified by the manifest authority, not by Lance branch versions: if the live manifest already records the fork on the active branch, a concurrent first-write won and the caller gets a retryable "refresh and retry"; if the manifest does not, a physical branch there is an orphan and the caller is pointed at `cleanup`. - `sync_branch(branch)` — re-binds the in-memory handle to the latest head of the branch. @@ -51,10 +51,10 @@ Notes: ## L2 — Internal system branches -Filtered from `branch_list()` but visible to internals: +Internal or legacy branch refs: -- `__schema_apply_lock__` — serializes schema migrations. -- `__run__<run-id>` — legacy from the pre-v0.4.0 Run state machine (removed in MR-771). The branch-name guard predicate `is_internal_run_branch` is kept as defense-in-depth so users cannot create a branch matching the legacy prefix; the filter will be removed once production legacy branches are swept (MR-770). +- `__schema_apply_lock__` — serializes schema migrations; filtered from `branch_list()` but visible to internals. +- `__run__<run-id>` — legacy from the pre-v0.4.0 Run state machine (removed in MR-771). These are swept off `__manifest` on the first read-write open by the v2→v3 internal-schema migration (MR-770), and `__run__*` is no longer a reserved name. Known limitation: a pre-v0.4.0 graph opened **read-only** still surfaces any stale `__run__*` branch in `branch_list()` until its first read-write open (the migration is write-path-only, like all manifest migrations). ## L2 — Recovery audit trail diff --git a/docs/user/constants.md b/docs/user/constants.md index 8f13555..210155e 100644 --- a/docs/user/constants.md +++ b/docs/user/constants.md @@ -4,11 +4,11 @@ |---|---|---| | `MANIFEST_DIR` | `__manifest` | `db/manifest/layout.rs` | | Commit graph dir | `_graph_commits.lance` | `db/commit_graph.rs` | -| Run registry dir (legacy, removed MR-771) | `_graph_runs.lance` | inert post-v0.4.0; reclaimed by MR-770 | -| Run branch prefix (legacy, removed MR-771) | `__run__` | filtered by `is_internal_run_branch` defense-in-depth | +| Run registry dir (legacy, removed MR-771) | `_graph_runs.lance` | inert post-v0.4.0; bytes remain until a `delete_prefix` primitive lands | +| Run branch prefix (legacy, removed MR-771/MR-770) | `__run__` | swept off `__manifest` by the v2→v3 migration; no longer a reserved name | | Schema apply lock | `__schema_apply_lock__` | `db/mod.rs` | | Manifest publisher retry budget | `PUBLISHER_RETRY_BUDGET = 5` | `db/manifest/publisher.rs` | -| Internal manifest schema version | `INTERNAL_MANIFEST_SCHEMA_VERSION = 2` | `db/manifest/migrations.rs` | +| Internal manifest schema version | `INTERNAL_MANIFEST_SCHEMA_VERSION = 3` | `db/manifest/migrations.rs` | | Merge stage batch | `MERGE_STAGE_BATCH_ROWS = 8192` | `exec/merge.rs` | | Maintenance concurrency | `OMNIGRAPH_MAINTENANCE_CONCURRENCY=8` | `db/omnigraph/optimize.rs` | | Lance blob compaction support | `LANCE_SUPPORTS_BLOB_COMPACTION = false` | `db/omnigraph/optimize.rs` | diff --git a/docs/user/storage.md b/docs/user/storage.md index c22d4d6..d1c52b5 100644 --- a/docs/user/storage.md +++ b/docs/user/storage.md @@ -22,7 +22,7 @@ OmniGraph is **not** a single Lance dataset; it is a *graph* of datasets coordin - `edges/{fnv1a64-hex(edge_type_name)}` — one Lance dataset per edge type - `__manifest/` — the catalog of all sub-tables and their published versions - `_graph_commits.lance` / `_graph_commit_actors.lance` — the commit graph and its actor map - - (legacy `_graph_runs.lance` / `_graph_run_actors.lance` from pre-v0.4.0 graphs are inert; the run state machine was removed in MR-771 and these files are cleaned up via MR-770's production sweep) + - (legacy `_graph_runs.lance` / `_graph_run_actors.lance` from pre-v0.4.0 graphs are inert; the run state machine was removed in MR-771. The v2→v3 manifest migration sweeps stale `__run__*` branches on first write-open; the inert dataset bytes themselves remain until a `delete_prefix` storage primitive lands) - **Manifest row schema** (`object_id, object_type, location, metadata, base_objects, table_key, table_version, table_branch, row_count`): - `object_type` ∈ `table | table_version | table_tombstone` - `table_key` ∈ `node:<TypeName> | edge:<EdgeName>` @@ -47,6 +47,7 @@ Adding a new on-disk shape change is one constant bump (`INTERNAL_MANIFEST_SCHEM |---|---| | v1 (implicit, pre-stamp) | `__manifest.object_id` had no PK annotation; publisher had no row-level CAS protection. | | v2 | `__manifest.object_id` carries `lance-schema:unenforced-primary-key=true`; row-level CAS engaged. Stamped as `omnigraph:internal_schema_version=2`. | +| v3 | One-time sweep of legacy `__run__*` staging branches (pre-v0.4.0 Run state machine, removed MR-771) off `__manifest`. Runs at `Omnigraph::open(ReadWrite)` and on publish. Stamped as `omnigraph:internal_schema_version=3`. | ## On-disk layout @@ -91,7 +92,7 @@ flowchart TB - **Graph root** is one directory (or S3 prefix). Everything below is part of one OmniGraph graph. - **`__manifest/`** is a Lance dataset whose rows describe which sub-table version is published at which graph-branch. Reading a snapshot starts here. - **`nodes/`** and **`edges/`** are sibling directories holding one Lance dataset per declared type. Names are `fnv1a64-hex` of the type name to keep paths fixed-length and case-safe. -- **`_graph_commits.lance`** is an L2 dataset that records the graph-level commit DAG, with a paired `_graph_commit_actors.lance` for the actor map. (Pre-v0.4.0 graphs also have inert `_graph_runs.lance` / `_graph_run_actors.lance` from the removed Run state machine; MR-770 sweeps these in production.) +- **`_graph_commits.lance`** is an L2 dataset that records the graph-level commit DAG, with a paired `_graph_commit_actors.lance` for the actor map. (Pre-v0.4.0 graphs also have inert `_graph_runs.lance` / `_graph_run_actors.lance` from the removed Run state machine; the v2→v3 migration sweeps their stale `__run__*` branches, and the dataset bytes are reclaimed once `delete_prefix` lands.) - **`_graph_commit_recoveries.lance`** — one row per recovery sweep action. Joined to `_graph_commits.lance` by `graph_commit_id`; the linked commit row carries `actor_id=omnigraph:recovery`. Operators correlate recoveries with the original mutations they rolled forward / back via this join. See `crates/omnigraph/src/db/recovery_audit.rs`. - **`__recovery/{ulid}.json`** — transient sidecar files written by the four migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`) before Phase B begins, deleted after Phase C succeeds. A sidecar persisting after process exit means the writer crashed in the Phase B → Phase C window; the next `Omnigraph::open` recovery sweep processes it. Steady-state directory is empty. See `crates/omnigraph/src/db/manifest/recovery.rs`. - **`_refs/branches/{name}.json`** is graph-level branch metadata — pointers from a branch name to the manifest version it heads. From 4a66d6e071ce95eabe49cd496356fba0617901be Mon Sep 17 00:00:00 2001 From: Aaron Goh <aaronwgoh5@gmail.com> Date: Sun, 7 Jun 2026 20:37:37 +0200 Subject: [PATCH 021/165] fix(loader): accept multi-line (pretty-printed) JSON in load (#146) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The loader read input line-by-line (reader.lines() + serde_json::from_str per line), so any delta where a JSON object spanned multiple lines failed with 'invalid JSON on line 1: EOF while parsing an object'. Compact JSONL worked; pretty-printed JSON never did. Switch to a streaming value deserializer (Deserializer::from_reader().into_iter::<Value>()), which treats any whitespace (including newlines inside objects) as a separator — so both compact JSONL and pretty-printed JSON load. Error labels switch from line numbers to record numbers (line numbers are meaningless once objects span lines). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ragnor Comerford <ragnor.comerford@gmail.com> --- crates/omnigraph/src/loader/mod.rs | 35 ++++++++++++++++-------------- 1 file changed, 19 insertions(+), 16 deletions(-) diff --git a/crates/omnigraph/src/loader/mod.rs b/crates/omnigraph/src/loader/mod.rs index 46a46e2..d5d74c0 100644 --- a/crates/omnigraph/src/loader/mod.rs +++ b/crates/omnigraph/src/loader/mod.rs @@ -288,21 +288,24 @@ async fn load_jsonl_reader<R: BufRead>( let mut node_rows: HashMap<String, Vec<JsonValue>> = HashMap::new(); let mut edge_rows: HashMap<String, Vec<(String, String, JsonValue)>> = HashMap::new(); - for (line_num, line) in reader.lines().enumerate() { - let line = line?; - let line = line.trim(); - if line.is_empty() { - continue; - } - let value: JsonValue = serde_json::from_str(line).map_err(|e| { - OmniError::manifest(format!("invalid JSON on line {}: {}", line_num + 1, e)) + // Parse a stream of JSON values. Accepts both compact JSONL (one object + // per line) and pretty-printed JSON where a single object spans multiple + // lines — serde's streaming deserializer treats any whitespace (including + // newlines) between top-level values as a separator. + for (idx, parsed) in serde_json::Deserializer::from_reader(reader) + .into_iter::<JsonValue>() + .enumerate() + { + let record_num = idx + 1; + let value: JsonValue = parsed.map_err(|e| { + OmniError::manifest(format!("invalid JSON at record {}: {}", record_num, e)) })?; if let Some(type_name) = value.get("type").and_then(|v| v.as_str()) { if !catalog.node_types.contains_key(type_name) { return Err(OmniError::manifest(format!( - "line {}: unknown node type '{}'", - line_num + 1, + "record {}: unknown node type '{}'", + record_num, type_name ))); } @@ -317,8 +320,8 @@ async fn load_jsonl_reader<R: BufRead>( } else if let Some(edge_name) = value.get("edge").and_then(|v| v.as_str()) { if catalog.lookup_edge_by_name(edge_name).is_none() { return Err(OmniError::manifest(format!( - "line {}: unknown edge type '{}'", - line_num + 1, + "record {}: unknown edge type '{}'", + record_num, edge_name ))); } @@ -326,14 +329,14 @@ async fn load_jsonl_reader<R: BufRead>( .get("from") .and_then(|v| v.as_str()) .ok_or_else(|| { - OmniError::manifest(format!("line {}: edge missing 'from'", line_num + 1)) + OmniError::manifest(format!("record {}: edge missing 'from'", record_num)) })? .to_string(); let to = value .get("to") .and_then(|v| v.as_str()) .ok_or_else(|| { - OmniError::manifest(format!("line {}: edge missing 'to'", line_num + 1)) + OmniError::manifest(format!("record {}: edge missing 'to'", record_num)) })? .to_string(); let data = value @@ -347,8 +350,8 @@ async fn load_jsonl_reader<R: BufRead>( .push((from, to, data)); } else { return Err(OmniError::manifest(format!( - "line {}: expected 'type' or 'edge' field", - line_num + 1 + "record {}: expected 'type' or 'edge' field", + record_num ))); } } From e62d9166fb39d0b309d1c345928f93748b5ea176 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford <ragnor.comerford@gmail.com> Date: Mon, 8 Jun 2026 01:50:12 +0200 Subject: [PATCH 022/165] fix: optimize publishes compaction; recovery roll-back converges manifest (#141) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * test(optimize): cover manifest publish + HEAD-drift reconcile Red against the pre-fix optimize, which ran compact_files without publishing the compacted version to __manifest: - maintenance: optimize must publish so the manifest table_version tracks the compacted Lance HEAD and a later schema apply succeeds; and must reconcile a pre-existing manifest-behind-HEAD drift (forged via raw Lance compaction) so strict writes commit again. - end_to_end + composite_flow: post-optimize query / strict update / reopen in the full lifecycle (the canonical flow previously omitted post-optimize writes as a documented "known limitation"). - failpoints: a crash between compaction and the manifest publish rolls forward on next open. * fix(optimize): publish compaction to manifest and reconcile HEAD drift optimize ran Lance compact_files without publishing the new version to __manifest, so the manifest table_version lagged the Lance HEAD: reads stayed pinned to the pre-compaction version, and the next schema apply or strict update/delete failed its HEAD-vs-manifest precondition with "stale view ... refresh and retry" (open-time recovery rollback inflated the gap on retry). optimize now publishes each compacted table's version under the per-(table, main) write queue, guarded by a manifest CAS and a SidecarKind::Optimize recovery sidecar (loose-match; roll-forward is safe because compaction is content-preserving). When a table has nothing left to compact but its Lance HEAD is already ahead of the manifest pin (pre-fix drift, or a recovery restore commit), optimize reconciles the manifest forward to HEAD (metadata-only, no sidecar). Caches and the CSR/CSC graph index are invalidated after a publish. Docs updated (maintenance, storage, branches-commits, writes, testing). * test(recovery): rollback convergence + optimize-defer regressions Red against the current code, landed before the fix: - recovery: after the open-time sweep rolls a sidecar back, the manifest must track Lance HEAD (no residual drift) so a follow-up schema apply succeeds — the original "+1 per retry" loop. Today roll-back restores without publishing, so the manifest lags HEAD and the apply fails its HEAD-vs-manifest precondition. - maintenance: optimize must refuse while a recovery sidecar is pending — operating on an unrecovered graph could publish a partial write the sweep would roll back. Also removes optimize_reconciles_preexisting_manifest_head_drift: the ad-hoc drift reconcile it covered is replaced by recovery-side convergence. * fix(recovery): converge manifest on roll-back; optimize defers on pending recovery Root of PR #141's review findings and the original "+1 per retry" loop: a Lance HEAD ahead of the manifest was ambiguous (benign content-preserving drift vs. a partial write a sidecar will roll back), and optimize's reconcile guessed it benign. Close the class instead of guessing: - Recovery roll-back now PUBLISHES the restored version (via a push_table_update_at_head helper shared with roll-forward), so the manifest tracks the Lance HEAD after recovery — symmetric with roll-forward. This fixes the +1 loop (after one roll-back the retry's HEAD-vs-manifest precondition passes) and removes the only remaining source of orphaned drift. The audit still records the logical rolled-back-to version; the manifest is published at the restore commit (identical content). - optimize drops the ad-hoc drift reconcile and instead REFUSES when a __recovery sidecar is pending, so it only ever operates on a recovered graph (manifest == HEAD); its compaction publish can no longer commit a partial write. With the reconcile gone, the blob-skip-vs-reconcile gap is moot. Updates the rollback recovery-test helper (manifest == HEAD after roll-back), the failpoints assertions, and the user/dev docs. * test(recovery): fix rollback assertion for manifest convergence The roll-back-publishes change makes the manifest version advance after a SchemaApply roll-back (to the old-schema content), so the schema_apply_without_schema_staging_rolls_back_on_next_open assertion must be `version > pre`, not `version == pre`. This update was dropped during the commit churn and surfaced as a CI Test Workspace failure; the old-schema-preserved intent stays covered by count_rows + _schema.pg + the RolledBack convergence invariant. --- AGENTS.md | 4 +- crates/omnigraph/src/db/manifest.rs | 2 +- crates/omnigraph/src/db/manifest/recovery.rs | 187 +++++++++----- crates/omnigraph/src/db/omnigraph/optimize.rs | 234 +++++++++++++++--- crates/omnigraph/tests/composite_flow.rs | 69 ++++-- crates/omnigraph/tests/end_to_end.rs | 84 +++++++ crates/omnigraph/tests/failpoints.rs | 122 ++++++++- crates/omnigraph/tests/helpers/recovery.rs | 3 + crates/omnigraph/tests/maintenance.rs | 124 +++++++++- crates/omnigraph/tests/recovery.rs | 91 +++++++ docs/dev/testing.md | 6 +- docs/dev/writes.md | 17 +- docs/user/branches-commits.md | 2 +- docs/user/maintenance.md | 6 +- docs/user/storage.md | 2 +- 15 files changed, 816 insertions(+), 137 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b876749..3f5b711 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -236,8 +236,8 @@ omnigraph policy explain --actor act-alice --action change --branch main | Columnar storage on object store | ✅ Arrow/Lance | URI normalization, S3 env-var plumbing | | Per-dataset versioning + time travel | ✅ | `snapshot_at_version`, `entity_at`, snapshot-pinned reads across many tables | | Per-dataset branches | ✅ | **Graph-level** branches (atomic across all sub-tables), lazy fork, system branch filtering | -| Atomic single-dataset commits | ✅ | **Multi-table publish via three layers**, NOT a single Lance primitive: (1) per-table Lance `commit_staged` for the data write, (2) `__manifest` row-level CAS via `ManifestBatchPublisher` for cross-table ordering, (3) the open-time recovery sweep for the residual gap between (1) and (2). All three layers ship; the four migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`) write a `__recovery/{ulid}.json` sidecar before Phase B and delete it after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the sweep in `db/manifest/recovery.rs`: classify, decide all-or-nothing per sidecar, roll forward via single `ManifestBatchPublisher::publish` or roll back via `Dataset::restore`, and record an audit row in `_graph_commit_recoveries.lance` (queryable via `omnigraph commit list --filter actor=omnigraph:recovery`). Continuous in-process recovery (no restart needed between Phase B failure and recovery) is the goal of a future background reconciler. Engine writes route through a sealed `TableStorage` trait exposing `stage_*` + `commit_staged` as the canonical staged-write surface; documented inline-commit residuals (`delete_where`, `create_vector_index`, plus legacy `append_batch` / `merge_insert_batches` / `overwrite_batch` / `create_*_index`) remain on the trait until upstream Lance ships a public two-phase API ([#6658](https://github.com/lance-format/lance/issues/6658), [#6666](https://github.com/lance-format/lance/issues/6666)) and the migration of every call site completes. | -| Compaction (`compact_files`) | ✅ | `omnigraph optimize` orchestrates over all node/edge tables, bounded concurrency; **skips blob-bearing tables** (reported via `TableOptimizeStats.skipped`, not silent), gated on `LANCE_SUPPORTS_BLOB_COMPACTION` until the upstream blob-v2 compaction-decode bug is fixed (see [docs/dev/invariants.md](docs/dev/invariants.md) Known Gaps) | +| Atomic single-dataset commits | ✅ | **Multi-table publish via three layers**, NOT a single Lance primitive: (1) per-table Lance `commit_staged` for the data write, (2) `__manifest` row-level CAS via `ManifestBatchPublisher` for cross-table ordering, (3) the open-time recovery sweep for the residual gap between (1) and (2). All three layers ship; the five migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`, `optimize_all_tables`) write a `__recovery/{ulid}.json` sidecar before Phase B and delete it after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the sweep in `db/manifest/recovery.rs`: classify, decide all-or-nothing per sidecar, roll forward via single `ManifestBatchPublisher::publish` or roll back via `Dataset::restore` followed by a manifest publish of the restored version (so both directions converge to `manifest == HEAD` — no residual drift), and record an audit row in `_graph_commit_recoveries.lance` (queryable via `omnigraph commit list --filter actor=omnigraph:recovery`). Continuous in-process recovery (no restart needed between Phase B failure and recovery) is the goal of a future background reconciler. Engine writes route through a sealed `TableStorage` trait exposing `stage_*` + `commit_staged` as the canonical staged-write surface; documented inline-commit residuals (`delete_where`, `create_vector_index`, plus legacy `append_batch` / `merge_insert_batches` / `overwrite_batch` / `create_*_index`) remain on the trait until upstream Lance ships a public two-phase API ([#6658](https://github.com/lance-format/lance/issues/6658), [#6666](https://github.com/lance-format/lance/issues/6666)) and the migration of every call site completes. | +| Compaction (`compact_files`) | ✅ | `omnigraph optimize` orchestrates over all node/edge tables, bounded concurrency; **publishes each compacted table's new version to `__manifest`** (so the manifest tracks the Lance HEAD — required for reads to observe compaction and for schema apply / strict writes to pass their HEAD-vs-manifest precondition), under the per-`(table, main)` write queue with `SidecarKind::Optimize` recovery coverage; **refuses on an unrecovered graph** (errors if a `__recovery` sidecar is pending — recovery may roll back a partial write, so optimize requires `manifest == HEAD` going in); **skips blob-bearing tables** (reported via `TableOptimizeStats.skipped`, not silent), gated on `LANCE_SUPPORTS_BLOB_COMPACTION` until the upstream blob-v2 compaction-decode bug is fixed (see [docs/dev/invariants.md](docs/dev/invariants.md) Known Gaps) | | Cleanup (`cleanup_old_versions`) | ✅ | `omnigraph cleanup` with `--keep` / `--older-than` policy | | BTREE / inverted (FTS) / vector indexes | ✅ | `ensure_indices` builds them on every relevant column; idempotent; lazy across branches | | `merge_insert` upsert | ✅ | `LoadMode::Merge`, mutation `update`/`insert`/`delete` lowering | diff --git a/crates/omnigraph/src/db/manifest.rs b/crates/omnigraph/src/db/manifest.rs index 3b2886f..5bf1f87 100644 --- a/crates/omnigraph/src/db/manifest.rs +++ b/crates/omnigraph/src/db/manifest.rs @@ -36,7 +36,7 @@ use publisher::{GraphNamespacePublisher, ManifestBatchPublisher}; pub(crate) use recovery::{ RecoveryMode, RecoverySidecar, RecoverySidecarHandle, SidecarKind, SidecarTablePin, SidecarTableRegistration, SidecarTombstone, delete_sidecar, has_schema_apply_sidecar, - new_sidecar, recover_manifest_drift, write_sidecar, + list_sidecars, new_sidecar, recover_manifest_drift, write_sidecar, }; pub use state::SubTableEntry; #[cfg(test)] diff --git a/crates/omnigraph/src/db/manifest/recovery.rs b/crates/omnigraph/src/db/manifest/recovery.rs index 4c1b987..3119531 100644 --- a/crates/omnigraph/src/db/manifest/recovery.rs +++ b/crates/omnigraph/src/db/manifest/recovery.rs @@ -106,6 +106,12 @@ pub(crate) enum SidecarKind { BranchMerge, /// `ensure_indices_for_branch` — index lifecycle commits. EnsureIndices, + /// `optimize_all_tables` — Lance `compact_files` (reserve-fragments + + /// rewrite commits) followed by a manifest publish of the compacted + /// version. Loose-match like the other multi-commit writers; roll-forward + /// is always safe because compaction is content-preserving (Lance + /// `Operation::Rewrite` "reorganizes data without semantic modification"). + Optimize, } /// One table's contribution to a sidecar's intended commit. The classifier @@ -412,11 +418,13 @@ pub(crate) fn parse_sidecar(sidecar_uri: &str, body: &str) -> Result<RecoverySid /// - **Strict** (`Mutation`, `Load`): exactly one `commit_staged` per /// table, so `lance_head == manifest_pinned + 1` AND /// `post_commit_pin == lance_head` is required. -/// - **Loose** (`SchemaApply`, `EnsureIndices`, `BranchMerge`): the -/// writer may run N ≥ 1 `commit_staged` calls per table (one per -/// index built + one for the overwrite, etc.; merge tables run -/// merge_insert + delete_where + index rebuilds) and the exact N -/// is hard to compute at sidecar-write time. The loose match accepts +/// - **Loose** (`SchemaApply`, `EnsureIndices`, `BranchMerge`, +/// `Optimize`): the writer advances the Lance HEAD by N ≥ 1 commits +/// per table (one per index built + one for the overwrite, etc.; +/// merge tables run merge_insert + delete_where + index rebuilds; +/// `Optimize` runs `compact_files`, which commits reserve-fragments + +/// rewrite) and the exact N is hard to compute at sidecar-write time. +/// The loose match accepts /// any `lance_head > manifest_pinned` as `RolledPastExpected` when /// `pin.expected_version == manifest_pinned` (the writer's CAS /// target matches what the manifest currently shows). The risk this @@ -494,9 +502,12 @@ pub(crate) fn decide(classifications: &[TableClassification]) -> SidecarDecision /// Skipping the restore in those cases would leave Lance HEAD ahead of /// the manifest with no recovery artifact left. /// -/// Cost: under repeated mid-rollback crashes (rare), Lance HEAD -/// accumulates extra restore commits that `omnigraph cleanup` reclaims. -/// Bounded by the number of recovery iterations — typically 1. +/// Cost: a successful roll-back appends one restore commit and then publishes +/// the manifest to match (`roll_back_sidecar`), so the table converges +/// (`manifest == HEAD`) in one pass. Only repeated crashes *between* the restore +/// and that publish (rare) accumulate extra restore commits; each re-classified +/// roll-back restores again and `omnigraph cleanup` reclaims the surplus. +/// Bounded by the number of interrupted recovery iterations — typically 0. pub(crate) async fn restore_table_to_version( table_path: &str, branch: Option<&str>, @@ -801,13 +812,24 @@ async fn roll_back_sidecar( sidecar: &RecoverySidecar, states: &[ClassifiedTable], ) -> Result<()> { - // Restore every table whose Lance HEAD has drifted from the - // manifest pin (RolledPastExpected, UnexpectedAtP1, - // UnexpectedMultistep). NoMovement tables are already at the - // manifest pin — no action. Restore is unconditional; repeated - // mid-rollback crashes accumulate a few extra Lance commits that - // `omnigraph cleanup` reclaims. + // Restore every drifted table (RolledPastExpected / UnexpectedAtP1 / + // UnexpectedMultistep) to its manifest-pinned content, then PUBLISH so + // `manifest == Lance HEAD` for each — symmetric with roll-forward. The + // restore commit's content equals the manifest-pinned version, so re-pinning + // the manifest to the new (restored) HEAD is content-correct and closes the + // orphaned-drift class (`HEAD > manifest` with no covering sidecar). This is + // what makes a failed-then-retried schema_apply converge: after one + // roll-back `manifest == HEAD`, so the retry's precondition passes instead of + // failing one version higher each iteration. + // + // NoMovement tables are already at the pin — excluded from both the restore + // and the publish. The audit `to_version` stays the *logical* rolled-back-to + // version (`manifest_pinned`), while the manifest is published at + // `manifest_pinned + 1` (the restore commit, same content) — keep that + // asymmetry so the audit records the drift (`from_version > to_version`). let mut outcomes = Vec::with_capacity(sidecar.tables.len()); + let mut updates: Vec<ManifestChange> = Vec::with_capacity(sidecar.tables.len()); + let mut expected: HashMap<String, u64> = HashMap::with_capacity(sidecar.tables.len()); for (pin, state) in sidecar.tables.iter().zip(states.iter()) { if matches!( state.classification, @@ -821,10 +843,20 @@ async fn roll_back_sidecar( state.manifest_pinned, ) .await?; - // `from_version` records the Lance HEAD observed BEFORE the - // restore (the actual drift), not the manifest pin. Operators - // reading `_graph_commit_recoveries.lance` see "rolled back - // from v7 to v5" rather than "v5 → v5". + // Publish the post-restore HEAD, CAS against the current (unmoved) + // manifest pin — the same helper roll-forward uses. + push_table_update_at_head( + root_uri, + &pin.table_key, + &pin.table_path, + pin.table_branch.as_deref(), + state.manifest_pinned, + &mut updates, + &mut expected, + ) + .await?; + // `from_version` records the Lance HEAD observed BEFORE the restore + // (the actual drift); `to_version` the logical pin we rolled back to. outcomes.push(TableOutcome { table_key: pin.table_key.clone(), from_version: state.lance_head, @@ -832,13 +864,23 @@ async fn roll_back_sidecar( }); } } - // Manifest pin doesn't move on rollback; record an audit-only - // commit at the existing version so operators can correlate via - // `omnigraph commit list --filter actor=omnigraph:recovery`. + // Publish the restored HEADs so manifest == HEAD. A degenerate all-NoMovement + // roll-back restores nothing — there's nothing to publish, and the audit + // records the unchanged snapshot version. + let manifest_version = if updates.is_empty() { + snapshot.version() + } else { + let publisher = GraphNamespacePublisher::new(root_uri, sidecar.branch.as_deref()); + publisher + .publish(&updates, &expected) + .await? + .version() + .version + }; record_audit( root_uri, sidecar, - snapshot.version(), + manifest_version, RecoveryKind::RolledBack, outcomes, ) @@ -919,44 +961,20 @@ async fn roll_forward_all( HashMap::with_capacity(sidecar.tables.len() + sidecar.additional_registrations.len()); for pin in &sidecar.tables { - // Open the dataset at its CURRENT Lance HEAD on the pin's branch - // (not at the sidecar's post_commit_pin). For strict-match writers - // (Mutation/Load) HEAD == post_commit_pin by construction. For - // loose-match writers (SchemaApply/EnsureIndices/BranchMerge) HEAD - // may be higher than post_commit_pin (multiple commit_staged - // calls per table); we want to publish to the actual current HEAD. - let head_ds = Dataset::open(&pin.table_path) - .await - .map_err(|e| OmniError::Lance(e.to_string()))?; - let head_ds = match pin.table_branch.as_deref() { - Some(b) if b != "main" => head_ds - .checkout_branch(b) - .await - .map_err(|e| OmniError::Lance(e.to_string()))?, - _ => head_ds, - }; - let head_version = head_ds.version().version; - - let row_count = head_ds - .count_rows(None) - .await - .map_err(|e| OmniError::Lance(e.to_string()))? as u64; - - let table_relative_path = super::table_path_for_table_key(&pin.table_key)?; - let version_metadata = super::metadata::TableVersionMetadata::from_dataset( + // Publish to the table's CURRENT Lance HEAD on the pin's branch (not the + // sidecar's `post_commit_pin`, a lower bound for loose-match writers that + // run multiple commit_staged calls per table). CAS against the pin's + // pre-write `expected_version`. + let head_version = push_table_update_at_head( root_uri, - &table_relative_path, - &head_ds, - )?; - - updates.push(ManifestChange::Update(SubTableUpdate { - table_key: pin.table_key.clone(), - table_version: head_version, - table_branch: pin.table_branch.clone(), - row_count, - version_metadata, - })); - expected.insert(pin.table_key.clone(), pin.expected_version); + &pin.table_key, + &pin.table_path, + pin.table_branch.as_deref(), + pin.expected_version, + &mut updates, + &mut expected, + ) + .await?; published_versions.insert(pin.table_key.clone(), head_version); } @@ -1047,6 +1065,57 @@ async fn roll_forward_all( Ok((new_dataset.version().version, published_versions)) } +/// Open `table_path` at its branch HEAD, read the current Lance HEAD version, +/// row count, and version metadata, and push a `ManifestChange::Update` (plus +/// its CAS `expected` entry) that re-pins the manifest to that HEAD. Returns the +/// published HEAD version. +/// +/// Shared by `roll_forward_all` (where `expected_version` is the sidecar's +/// pre-write pin) and `roll_back_sidecar` (where it is the manifest-pinned +/// version the table was just restored to). The HEAD is read AFTER any restore +/// in the same single-threaded sweep, so no concurrent writer can have advanced +/// it. +async fn push_table_update_at_head( + root_uri: &str, + table_key: &str, + table_path: &str, + branch: Option<&str>, + expected_version: u64, + updates: &mut Vec<ManifestChange>, + expected: &mut HashMap<String, u64>, +) -> Result<u64> { + let head_ds = Dataset::open(table_path) + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + let head_ds = match branch { + Some(b) if b != "main" => head_ds + .checkout_branch(b) + .await + .map_err(|e| OmniError::Lance(e.to_string()))?, + _ => head_ds, + }; + let head_version = head_ds.version().version; + let row_count = head_ds + .count_rows(None) + .await + .map_err(|e| OmniError::Lance(e.to_string()))? as u64; + let table_relative_path = super::table_path_for_table_key(table_key)?; + let version_metadata = super::metadata::TableVersionMetadata::from_dataset( + root_uri, + &table_relative_path, + &head_ds, + )?; + updates.push(ManifestChange::Update(SubTableUpdate { + table_key: table_key.to_string(), + table_version: head_version, + table_branch: branch.map(str::to_string), + row_count, + version_metadata, + })); + expected.insert(table_key.to_string(), expected_version); + Ok(head_version) +} + /// Append the audit row describing this recovery action. /// /// Two-part write: (a) `_graph_commits.lance` row anchored on the recovery diff --git a/crates/omnigraph/src/db/omnigraph/optimize.rs b/crates/omnigraph/src/db/omnigraph/optimize.rs index fff3f54..ee39323 100644 --- a/crates/omnigraph/src/db/omnigraph/optimize.rs +++ b/crates/omnigraph/src/db/omnigraph/optimize.rs @@ -8,8 +8,14 @@ //! Two dials: //! //! * `optimize_all_tables` — Lance `compact_files` on every table. Rewrites -//! small fragments into fewer large ones. Non-destructive (creates a new -//! version; old fragments remain reachable via older manifest versions). +//! small fragments into fewer large ones, then **publishes the compacted +//! version to the `__manifest`** so the manifest's `table_version` tracks the +//! compacted Lance HEAD (reads pin the manifest version, so without the +//! publish compaction would be invisible to readers and would break the +//! HEAD-vs-manifest precondition of schema apply / strict writes). Compaction +//! is content-preserving (Lance `Operation::Rewrite` "reorganizes data +//! without semantic modification"), so old fragments remain reachable via +//! older manifest versions until `cleanup` runs. //! * `cleanup_all_tables` — Lance `cleanup_old_versions` on every table. //! Removes manifests (and their unique fragments) older than the configured //! retention. Destructive to version history — callers should gate this @@ -23,7 +29,9 @@ use std::time::Duration; use chrono::Utc; use futures::stream::StreamExt; use lance::dataset::cleanup::{CleanupPolicy, RemovalStats}; -use lance::dataset::optimize::{CompactionMetrics, CompactionOptions, compact_files}; +use lance::dataset::optimize::{ + CompactionMetrics, CompactionOptions, compact_files, plan_compaction, +}; use super::*; @@ -111,7 +119,8 @@ pub struct TableOptimizeStats { pub fragments_removed: usize, /// Number of new, larger fragments Lance produced. pub fragments_added: usize, - /// Did this table get a new Lance manifest version from the compaction? + /// Did this table get a new manifest version from the compaction? True when + /// compaction ran and its compacted version was published to `__manifest`. pub committed: bool, /// `Some(reason)` if this table was deliberately not compacted. When set, /// `fragments_removed == 0`, `fragments_added == 0`, and `!committed`. @@ -153,12 +162,29 @@ pub struct TableCleanupStats { pub error: Option<String>, } -/// Run Lance `compact_files` on every node + edge table on `main`. -/// Tables run in parallel (bounded concurrency). +/// Run Lance `compact_files` on every node + edge table on `main`, publishing +/// each compacted table's new version to the `__manifest`. Tables run in +/// parallel (bounded concurrency); each is fault-isolated only at the Lance +/// level — a publish error is propagated (the recovery sidecar covers it). pub async fn optimize_all_tables(db: &Omnigraph) -> Result<Vec<TableOptimizeStats>> { db.ensure_schema_state_valid().await?; db.ensure_schema_apply_idle("optimize").await?; + // Refuse on an unrecovered graph. A pending recovery sidecar means a failed + // write left partial state that the open-time sweep must resolve (roll + // forward/back) first; compacting + publishing a table covered by such a + // sidecar could commit a partial write the sweep would roll back. Reopen the + // graph to run recovery, then re-run optimize. + if !crate::db::manifest::list_sidecars(db.root_uri(), db.storage_adapter()) + .await? + .is_empty() + { + return Err(OmniError::manifest_conflict( + "optimize requires a clean recovery state; reopen the graph to run the \ + recovery sweep before optimizing", + )); + } + let resolved = db.resolved_branch_target(None).await?; let snapshot = resolved.snapshot; @@ -183,49 +209,179 @@ pub async fn optimize_all_tables(db: &Omnigraph) -> Result<Vec<TableOptimizeStat } let concurrency = maint_concurrency().min(table_tasks.len()).max(1); - let table_store = &db.table_store; let stats: Vec<Result<TableOptimizeStats>> = futures::stream::iter(table_tasks.into_iter()) - .map(|(table_key, full_path, has_blob)| async move { - // Lance `compact_files` mis-decodes blob-v2 columns under the forced - // `BlobHandling::AllBinary` read (see LANCE_SUPPORTS_BLOB_COMPACTION). - // Skip blob-bearing tables and report it rather than aborting the - // whole sweep — the other tables still compact. - if has_blob && !LANCE_SUPPORTS_BLOB_COMPACTION { - tracing::warn!( - target: "omnigraph::optimize", - table = %table_key, - "skipping compaction: table has blob columns the current Lance \ - cannot rewrite (blob-v2 AllBinary decode bug); other tables \ - unaffected — rerun after the Lance fix", - ); - return Ok(TableOptimizeStats::skipped( - table_key, - SkipReason::BlobColumnsUnsupportedByLance, - )); - } - let mut ds = table_store - .open_dataset_head_for_write(&table_key, &full_path, None) - .await?; - let version_before = ds.version().version; - let metrics: CompactionMetrics = - compact_files(&mut ds, CompactionOptions::default(), None) - .await - .map_err(|e| OmniError::Lance(e.to_string()))?; - let version_after = ds.version().version; - Ok(TableOptimizeStats::compacted( - table_key, - &metrics, - version_after != version_before, - )) + .map(move |(table_key, full_path, has_blob)| async move { + optimize_one_table(db, table_key, full_path, has_blob).await }) .buffer_unordered(concurrency) .collect() .await; + // Invalidate caches for any table that published a compaction — done BEFORE + // propagating a sibling table's error, since the published versions are + // durable and reads must observe the new fragment layout (Lance invalidates + // the original row addresses on rewrite). The CSR/CSC graph topology index + // is rebuilt only when an edge table moved. Mirrors schema_apply's + // post-publish invalidation. + let any_committed = stats + .iter() + .any(|s| matches!(s, Ok(st) if st.committed)); + let edge_committed = stats + .iter() + .any(|s| matches!(s, Ok(st) if st.committed && st.table_key.starts_with("edge:"))); + if any_committed { + db.runtime_cache.invalidate_all().await; + if edge_committed { + db.invalidate_graph_index().await; + } + } + stats.into_iter().collect() } +/// Compact one table and publish the compacted version to the `__manifest`. +/// +/// Compaction (`compact_files`) advances the *dataset's* Lance HEAD via a +/// reserve-fragments + rewrite commit, but Lance knows nothing about the +/// `__manifest`. To keep the manifest the single authority for each table's +/// visible version (invariant 2), optimize must publish the compacted version. +/// The Lance-HEAD-before-manifest-publish gap is unavoidable (Lance has no +/// staged/uncommitted compaction), so it is covered by a recovery sidecar like +/// the other multi-commit writers; roll-forward is always safe because +/// compaction is content-preserving. +async fn optimize_one_table( + db: &Omnigraph, + table_key: String, + full_path: String, + has_blob: bool, +) -> Result<TableOptimizeStats> { + // Lance `compact_files` mis-decodes blob-v2 columns under the forced + // `BlobHandling::AllBinary` read (see LANCE_SUPPORTS_BLOB_COMPACTION). Skip + // blob-bearing tables and report it rather than aborting the whole sweep. + if has_blob && !LANCE_SUPPORTS_BLOB_COMPACTION { + tracing::warn!( + target: "omnigraph::optimize", + table = %table_key, + "skipping compaction: table has blob columns the current Lance \ + cannot rewrite (blob-v2 AllBinary decode bug); other tables \ + unaffected — rerun after the Lance fix", + ); + return Ok(TableOptimizeStats::skipped( + table_key, + SkipReason::BlobColumnsUnsupportedByLance, + )); + } + + // Serialize the whole compact→publish against concurrent mutations on this + // (table, main): compaction is a Rewrite op that retryable-conflicts with a + // concurrent Merge/Update/Delete on overlapping fragments, and an + // interleaved write would also move the manifest version out from under the + // CAS below. Holding the queue makes the CAS baseline read under it exact. + let _guard = db + .write_queue() + .acquire_many(&[(table_key.clone(), None)]) + .await; + + let mut ds = db + .table_store + .open_dataset_head_for_write(&table_key, &full_path, None) + .await?; + + // CAS baseline: the table's current manifest version, read under the queue + // (in-memory coordinator snapshot, no storage I/O — stable for this section). + let expected_version = db + .snapshot() + .await + .entry(&table_key) + .map(|e| e.table_version) + .ok_or_else(|| OmniError::manifest(format!("no manifest entry for {}", table_key)))?; + + // Precise "will it compact?" check — `plan_compaction` also accounts for + // deletion materialization (which can rewrite even a single fragment). A + // steady-state already-compacted table yields an empty plan and is never + // pinned in a sidecar (a zero-commit pin would classify NoMovement on + // recovery and force an all-or-nothing rollback). There is no drift to + // reconcile here: optimize runs only on a recovered graph (the pending- + // sidecar guard above), and recovery roll-back now publishes, so + // `HEAD == manifest` holds going in. + let options = CompactionOptions::default(); + let plan = plan_compaction(&ds, &options) + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + if plan.num_tasks() == 0 { + return Ok(TableOptimizeStats::compacted( + table_key, + &CompactionMetrics::default(), + false, + )); + } + + // Phase A: recovery sidecar BEFORE compaction advances the Lance HEAD, so a + // crash before the manifest publish rolls forward on next open. + let sidecar = crate::db::manifest::new_sidecar( + crate::db::manifest::SidecarKind::Optimize, + None, + // optimize is system-attributed (no `optimize_as` actor API today). + None, + vec![crate::db::manifest::SidecarTablePin { + table_key: table_key.clone(), + table_path: full_path.clone(), + expected_version, + // Lower bound — compaction commits N≥1 versions (reserve + rewrite); + // the classifier loose-matches SidecarKind::Optimize. + post_commit_pin: expected_version + 1, + table_branch: None, + }], + ); + let handle = + crate::db::manifest::write_sidecar(db.root_uri(), db.storage_adapter(), &sidecar).await?; + + // Phase B: compaction (reserve-fragments + rewrite commits advance HEAD). + let version_before = ds.version().version; + let metrics: CompactionMetrics = compact_files(&mut ds, options, None) + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + let version_after = ds.version().version; + let committed = version_after != version_before; + + // Pin the per-writer Phase B → Phase C residual for optimize: Lance HEAD has + // advanced but the manifest publish below hasn't run. + crate::failpoints::maybe_fail("optimize.post_phase_b_pre_manifest_commit")?; + + // Phase C: publish the compacted version to the manifest (one CAS commit, + // expected = the version observed under the queue). On failure the sidecar + // is intentionally left for the open-time recovery sweep to roll forward. + if committed { + let state = db.table_store.table_state(&full_path, &ds).await?; + let update = crate::db::SubTableUpdate { + table_key: table_key.clone(), + table_version: state.version, + table_branch: None, + row_count: state.row_count, + version_metadata: state.version_metadata, + }; + let mut expected = std::collections::HashMap::new(); + expected.insert(table_key.clone(), expected_version); + db.coordinator + .write() + .await + .commit_updates_with_actor_with_expected(&[update], &expected, None) + .await?; + } + + // Phase D: delete the sidecar (best-effort; recovery resolves a leftover). + if let Err(err) = crate::db::manifest::delete_sidecar(&handle, db.storage_adapter()).await { + tracing::warn!( + error = %err, + operation_id = handle.operation_id.as_str(), + "optimize recovery sidecar cleanup failed; next open's recovery sweep will resolve it" + ); + } + + Ok(TableOptimizeStats::compacted(table_key, &metrics, committed)) +} + /// Run Lance `cleanup_old_versions` on every node + edge table on `main`, /// using [`CleanupPolicyOptions`]. The latest manifest is always preserved /// regardless (Lance invariant). diff --git a/crates/omnigraph/tests/composite_flow.rs b/crates/omnigraph/tests/composite_flow.rs index 6c720da..dd41310 100644 --- a/crates/omnigraph/tests/composite_flow.rs +++ b/crates/omnigraph/tests/composite_flow.rs @@ -294,21 +294,19 @@ async fn composite_flow_canonical_lifecycle() { ); // ───────────────────────────────────────────────────────────────── - // Step 10: optimize the post-merge graph — verify indices stay - // valid and queryable. + // Step 10: optimize the post-merge graph — verify compaction is + // published to the manifest (so the manifest pin tracks the compacted + // Lance HEAD), indices stay valid and queryable, and a post-optimize + // strict write commits. // - // **Known limitation**: `optimize_all_tables` calls Lance - // `compact_files` directly — it advances per-table Lance HEAD - // without updating the omnigraph `__manifest` pin. After optimize, - // the next writer's expected_table_versions captures the - // pre-optimize manifest pin, but the publisher's pre-check reads - // a higher version from the manifest dataset (because some other - // path — possibly schema-state recovery on reopen — wrote a newer - // __manifest row). The `ExpectedVersionMismatch` is benign - // (re-issuing the mutation after a snapshot refresh succeeds), but - // a composite test cannot reliably exercise post-optimize mutations - // until that path is investigated. Coverage of post-optimize - // mutations is left to a focused optimize+cleanup integration test. + // This step used to carry a "Known limitation": `optimize_all_tables` + // ran Lance `compact_files` without publishing the new version to + // `__manifest`, so the manifest pin lagged the Lance HEAD and the next + // strict write / schema apply failed with `ExpectedVersionMismatch` + // ("stale view … refresh and retry") — so post-optimize mutations were + // deliberately omitted here. optimize now publishes the compacted + // version, and this flow exercises exactly that previously-failing + // write below. // ───────────────────────────────────────────────────────────────── let optimize_stats = db.optimize().await.unwrap(); assert!( @@ -331,6 +329,28 @@ async fn composite_flow_canonical_lifecycle() { "row counts unchanged by optimize" ); + // A strict update on a compacted table is exactly the write that + // failed with "stale view" before optimize published its compaction. + // It must now commit (Alice is one of the seed Persons; an update + // leaves the row count at 6). + let post_optimize_update = mutate_main( + &mut db, + MUTATION_QUERIES, + "set_age", + &mixed_params(&[("$name", "Alice")], &[("$age", 41)]), + ) + .await + .expect("post-optimize strict update must commit — optimize published the manifest"); + assert_eq!( + post_optimize_update.affected_nodes, 1, + "post-optimize update must affect exactly Alice" + ); + assert_eq!( + count_rows(&db, "node:Person").await, + 6, + "an update must not change the Person row count" + ); + // ───────────────────────────────────────────────────────────────── // Step 11: cleanup — keep last 10 versions, only purge versions // older than 1 hour. With this small test, we have well under 10 @@ -373,14 +393,27 @@ async fn composite_flow_canonical_lifecycle() { branches, ); - // Final query exercise — full read path works post-reopen, - // post-cleanup. Post-cleanup mutation is omitted here pending - // resolution of the optimize-vs-manifest-pin interaction documented - // in Step 10. + // Final exercise — full read AND write path works post-reopen, + // post-cleanup. (The post-cleanup mutation was previously omitted + // pending resolution of the optimize-vs-manifest-pin interaction in + // Step 10; that is now fixed, so a strict write here must commit.) let final_total = query_main(&mut db, TEST_QUERIES, "total_people", &ParamMap::default()) .await .unwrap(); assert!(!final_total.batches().is_empty()); + + let post_reopen_update = mutate_main( + &mut db, + MUTATION_QUERIES, + "set_age", + &mixed_params(&[("$name", "Alice")], &[("$age", 42)]), + ) + .await + .expect("post-reopen, post-cleanup strict update must commit"); + assert_eq!( + post_reopen_update.affected_nodes, 1, + "post-reopen update must affect exactly Alice" + ); } /// Cross-handle sequence that exercises operations after a schema_apply diff --git a/crates/omnigraph/tests/end_to_end.rs b/crates/omnigraph/tests/end_to_end.rs index a0fdb0e..ea11d0e 100644 --- a/crates/omnigraph/tests/end_to_end.rs +++ b/crates/omnigraph/tests/end_to_end.rs @@ -1933,3 +1933,87 @@ query docs_with_tag($tag: String) { "contains-pushdown should return exactly the rows whose tags list contains 'red'" ); } + +// ─── Maintenance in the full lifecycle: optimize (compaction) ──────────────── + +/// `optimize` (Lance compaction) is part of a realistic graph lifecycle: it +/// advances the Lance HEAD and publishes the compacted version to the manifest. +/// The rest of the flow must keep working across that boundary — reads observe +/// the compacted data, strict updates (which check Lance HEAD == manifest +/// version) still commit, inserts still commit, and the state survives a reopen +/// (the open-time recovery sweep finds no leftover drift). Before optimize +/// published its compaction, the manifest lagged the Lance HEAD here and the +/// post-optimize update below failed with "stale view ... refresh and retry". +#[tokio::test] +async fn full_flow_optimize_then_query_update_and_reopen() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + let mut db = init_and_load(&dir).await; + + // Build several Person fragments so compaction has something to merge. + for (name, age) in [("Eve", 40), ("Frank", 41), ("Grace", 42)] { + mutate_main( + &mut db, + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", name)], &[("$age", age)]), + ) + .await + .unwrap(); + } + + let stats = db.optimize().await.unwrap(); + assert!( + stats.iter().any(|s| s.committed), + "a multi-fragment table should have compacted in this flow" + ); + + // Reads observe the compacted data. + let qr = query_main( + &mut db, + TEST_QUERIES, + "get_person", + ¶ms(&[("$name", "Alice")]), + ) + .await + .unwrap(); + assert_eq!(qr.num_rows(), 1); + + // Strict update after optimize commits (previously failed with "stale view" + // because the manifest lagged the compacted Lance HEAD). + let upd = mutate_main( + &mut db, + MUTATION_QUERIES, + "set_age", + &mixed_params(&[("$name", "Alice")], &[("$age", 31)]), + ) + .await + .unwrap(); + assert_eq!(upd.affected_nodes, 1); + + // Insert after optimize also commits. + mutate_main( + &mut db, + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Ivan")], &[("$age", 50)]), + ) + .await + .unwrap(); + assert_eq!(count_rows(&db, "node:Person").await, 8); // 4 seed + Eve/Frank/Grace + Ivan + + // State survives a reopen — the recovery sweep runs and finds no drift. + drop(db); + let reopened = Omnigraph::open(&uri).await.unwrap(); + assert_eq!(count_rows(&reopened, "node:Person").await, 8); + let alice = reopened + .entity_at_target(ReadTarget::branch("main"), "node:Person", "Alice") + .await + .unwrap() + .unwrap(); + assert_eq!( + alice["age"], + serde_json::json!(31), + "Alice's post-optimize age update must persist across reopen" + ); +} diff --git a/crates/omnigraph/tests/failpoints.rs b/crates/omnigraph/tests/failpoints.rs index 149c63a..d240108 100644 --- a/crates/omnigraph/tests/failpoints.rs +++ b/crates/omnigraph/tests/failpoints.rs @@ -1245,7 +1245,7 @@ async fn refresh_defers_rollback_eligible_sidecar_to_next_open() { // the rollback (will use Dataset::restore safely; no concurrent // writers at open time). drop(db); - let _db = Omnigraph::open(&uri).await.unwrap(); + let db = Omnigraph::open(&uri).await.unwrap(); // After full-sweep recovery, the sidecar should be processed // (deleted). Sidecar's tables are eligible for rollback (UnexpectedAtP1): // restore happens on Person (HEAD advances by 1). @@ -1268,6 +1268,19 @@ async fn refresh_defers_rollback_eligible_sidecar_to_next_open() { "full sweep must run Dataset::restore (head advances); \ post_head={post_head}, final_head={final_head}", ); + // Convergence: roll-back published the restored HEAD, so the manifest pin + // tracks Lance HEAD afterward (no residual drift). + let entry_version = db + .snapshot_of(omnigraph::db::ReadTarget::branch("main")) + .await + .unwrap() + .entry("node:Person") + .unwrap() + .table_version; + assert_eq!( + entry_version, final_head, + "full-sweep roll-back must publish so manifest pin ({entry_version}) == Lance HEAD ({final_head})", + ); } /// Companion to the above — confirms that a finalize→publisher failure @@ -1461,10 +1474,15 @@ edge WorksAt: Person -> Company } let db = Omnigraph::open(&uri).await.unwrap(); - assert_eq!( - version_main(&db).await.unwrap(), - pre_failure_version, - "manifest must remain on the old schema when no schema staging files existed" + // Roll-back now publishes the restored version, so the manifest version + // advances — but to the OLD-schema content: the migration never applied + // (asserted by count_rows + the `_schema.pg` checks below), and the sweep + // converges (`manifest == Lance HEAD`, asserted by + // assert_post_recovery_invariants's RolledBack arm). + assert!( + version_main(&db).await.unwrap() > pre_failure_version, + "roll-back publishes the restored (old-schema) version, advancing the manifest; \ + pre={pre_failure_version}", ); assert_eq!( helpers::count_rows(&db, "node:Person").await, @@ -1637,6 +1655,100 @@ edge WorksAt: Person -> Company ); } +/// `optimize` Phase B → Phase C residual: `compact_files` advanced the Lance +/// HEAD but the manifest publish hasn't run. The `Optimize` recovery sidecar +/// (loose-match, like SchemaApply/EnsureIndices) must roll the compacted version +/// forward on next open so the manifest tracks the Lance HEAD — and the healed +/// table must then accept a schema apply (the original bug's victim). +#[tokio::test] +async fn optimize_phase_b_failure_recovered_on_next_open() { + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + let operation_id; + + // Seed: several separate Person inserts → multiple fragments, so compaction + // has real work and advances the Lance HEAD. + { + let db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + for (name, age) in [("alice", 30), ("bob", 31), ("carol", 32), ("dave", 33)] { + db.mutate( + "main", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", name)], &[("$age", age)]), + ) + .await + .unwrap(); + } + } + + let pre_failure_version = { + let db = Omnigraph::open(&uri).await.unwrap(); + version_main(&db).await.unwrap() + }; + + // Failpoint fires AFTER compact_files advanced the Lance HEAD but BEFORE the + // manifest publish. The Optimize sidecar persists (only node:Person has + // compactable fragments, so exactly one sidecar is written). + { + let db = Omnigraph::open(&uri).await.unwrap(); + let _failpoint = + ScopedFailPoint::new("optimize.post_phase_b_pre_manifest_commit", "return"); + let err = db.optimize().await.unwrap_err(); + assert!( + err.to_string() + .contains("injected failpoint triggered: optimize.post_phase_b_pre_manifest_commit"), + "unexpected error: {err}" + ); + + let recovery_dir = dir.path().join("__recovery"); + let sidecars: Vec<_> = std::fs::read_dir(&recovery_dir) + .unwrap() + .filter_map(|e| e.ok()) + .collect(); + assert_eq!( + sidecars.len(), + 1, + "exactly one Optimize sidecar must persist after optimize failure" + ); + operation_id = single_sidecar_operation_id(dir.path()); + } + + // Recovery: reopen runs the sweep. The Optimize sidecar classifies + // RolledPastExpected (loose-match) → RollForward → manifest extends to the + // compacted Lance HEAD. + let db = Omnigraph::open(&uri).await.unwrap(); + let post_recovery_version = version_main(&db).await.unwrap(); + assert!( + post_recovery_version > pre_failure_version, + "manifest version must advance post-recovery (compaction rolled forward); \ + pre={pre_failure_version}, post={post_recovery_version}", + ); + drop(db); + + assert_post_recovery_invariants( + dir.path(), + &operation_id, + RecoveryExpectation::RolledForward { + tables: vec![TableExpectation::main("node:Person")], + }, + ) + .await + .unwrap(); + + // The healed table accepts an additive schema apply — its HEAD-vs-manifest + // precondition is satisfied because recovery published the compacted version. + let db = Omnigraph::open(&uri).await.unwrap(); + let desired = helpers::TEST_SCHEMA.replace( + " age: I32?\n}", + " age: I32?\n nickname: String?\n}", + ); + db.apply_schema(&desired) + .await + .expect("schema apply after optimize recovery must succeed"); +} + #[tokio::test] async fn branch_merge_phase_b_failure_recovered_on_next_open() { use omnigraph::loader::{LoadMode, load_jsonl}; diff --git a/crates/omnigraph/tests/helpers/recovery.rs b/crates/omnigraph/tests/helpers/recovery.rs index c76009e..90d9a25 100644 --- a/crates/omnigraph/tests/helpers/recovery.rs +++ b/crates/omnigraph/tests/helpers/recovery.rs @@ -181,6 +181,9 @@ pub async fn assert_post_recovery_invariants( "audit row for {operation_id} recorded the wrong recovery_kind", ); assert_rollback_outcomes_record_drift(&audit); + // Roll-back now publishes the restored HEAD, so manifest == Lance + // HEAD afterward (symmetric with roll-forward) — no residual drift. + assert_manifest_pins_match_lance_heads(graph_root, &tables).await?; assert_recovery_commit_shape(graph_root, &audit, &tables).await?; assert_non_main_did_not_move_main(graph_root, &tables).await?; assert_idempotent_reopen(graph_root, operation_id).await?; diff --git a/crates/omnigraph/tests/maintenance.rs b/crates/omnigraph/tests/maintenance.rs index 3e61677..2a5a659 100644 --- a/crates/omnigraph/tests/maintenance.rs +++ b/crates/omnigraph/tests/maintenance.rs @@ -8,10 +8,12 @@ mod helpers; use std::time::Duration; use lance::Dataset; -use omnigraph::db::{CleanupPolicyOptions, Omnigraph, SkipReason}; +use omnigraph::db::{CleanupPolicyOptions, Omnigraph, ReadTarget, SkipReason}; use omnigraph::loader::{LoadMode, load_jsonl}; -use helpers::{TEST_DATA, TEST_SCHEMA, count_rows, init_and_load}; +use helpers::{ + MUTATION_QUERIES, TEST_DATA, TEST_SCHEMA, count_rows, init_and_load, mixed_params, mutate_main, +}; /// Filesystem URI of a node sub-table, mirroring the engine's layout /// (FNV-1a of the type name under `nodes/`). Matches the helper in @@ -163,6 +165,124 @@ node Tag {\n slug: String @key\n}\n"; assert_eq!(tag.skipped, None, "non-blob table must not be skipped"); } +// Regression: `optimize` must publish its compaction to the `__manifest` so the +// manifest's recorded `table_version` tracks the compacted Lance HEAD. +// +// Lance `compact_files` advances the *dataset's* version (reserve-fragments + +// rewrite commits) but knows nothing about OmniGraph's `__manifest`. If optimize +// does not publish a manifest update, the manifest's `table_version` lags the +// Lance HEAD: reads stay pinned to the pre-compaction version (compaction is +// invisible to them) and any subsequent schema apply / strict update/delete +// fails its HEAD-vs-manifest precondition with +// "stale view of '<table>': expected manifest table version X but current is Y". +// This pins the fix — optimize publishes the compacted version, so manifest == +// HEAD and migrations after a compaction succeed. +#[tokio::test] +async fn optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds() { + let dir = tempfile::tempdir().unwrap(); + let root = dir.path().to_str().unwrap().trim_end_matches('/').to_string(); + let mut db = init_and_load(&dir).await; + + // Several separate inserts → multiple Person fragments, so `compact_files` + // actually merges and moves the Lance HEAD (a single fragment is a no-op). + for (name, age) in [("Eve", 40), ("Frank", 41), ("Grace", 42), ("Heidi", 43)] { + mutate_main( + &mut db, + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", name)], &[("$age", age as i64)]), + ) + .await + .expect("insert"); + } + + let stats = db.optimize().await.unwrap(); + let person = stats + .iter() + .find(|s| s.table_key == "node:Person") + .expect("Person stat present"); + assert!( + person.committed, + "Person is multi-fragment, so optimize must have compacted it" + ); + + // After optimize, the manifest's recorded table_version must equal the actual + // Lance HEAD — optimize published its compaction, so there is no drift. + let snap = db.snapshot_of(ReadTarget::branch("main")).await.unwrap(); + let entry = snap.entry("node:Person").unwrap(); + let manifest_version = entry.table_version; + let full = format!("{}/{}", root, entry.table_path); + let lance_head = Dataset::open(&full).await.unwrap().version().version; + assert_eq!( + manifest_version, lance_head, + "after optimize, manifest table_version ({manifest_version}) must equal Lance HEAD ({lance_head})", + ); + + // Reads observe the compacted version with rows preserved (4 seed + 4 inserts). + assert_eq!(count_rows(&db, "node:Person").await, 8); + + // The headline: an additive (nullable property) migration touching the + // just-compacted table succeeds, where it previously failed with "stale view". + let desired = TEST_SCHEMA.replace( + " age: I32?\n}", + " age: I32?\n nickname: String?\n}", + ); + let result = db + .apply_schema(&desired) + .await + .expect("additive schema apply after optimize must succeed"); + assert!(result.applied, "schema apply should report applied=true"); +} + +// Regression: `optimize` must REFUSE when an unresolved recovery sidecar is +// pending. Operating on an unrecovered graph could publish a partial write that +// the all-or-nothing recovery sweep would roll back; the operator must reopen +// (run the recovery sweep) first. +#[tokio::test] +async fn optimize_defers_when_recovery_sidecar_is_pending() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let db = init_and_load(&dir).await; + + // Simulate an in-process failed write that left a recovery sidecar on disk. + let recovery_dir = dir.path().join("__recovery"); + std::fs::create_dir_all(&recovery_dir).unwrap(); + let person_path = node_table_uri(uri, "Person"); + let sidecar_json = format!( + r#"{{ + "schema_version": 1, + "operation_id": "01H000000000000000000DEFR", + "started_at": "0", + "branch": null, + "actor_id": "act-test", + "writer_kind": "Mutation", + "tables": [ + {{ + "table_key": "node:Person", + "table_path": "{}", + "expected_version": 1, + "post_commit_pin": 2 + }} + ] + }}"#, + person_path + ); + std::fs::write( + recovery_dir.join("01H000000000000000000DEFR.json"), + sidecar_json, + ) + .unwrap(); + + let err = db + .optimize() + .await + .expect_err("optimize must defer (error) while a recovery sidecar is pending"); + assert!( + err.to_string().to_lowercase().contains("recovery"), + "optimize defer error should mention recovery; got: {err}", + ); +} + #[tokio::test] async fn cleanup_without_any_policy_option_errors() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/omnigraph/tests/recovery.rs b/crates/omnigraph/tests/recovery.rs index a090178..f6b19e8 100644 --- a/crates/omnigraph/tests/recovery.rs +++ b/crates/omnigraph/tests/recovery.rs @@ -278,6 +278,97 @@ async fn recovery_rolls_back_synthetic_drift_on_open() { ); } +/// Regression: recovery roll-back must PUBLISH the restored version so +/// `manifest == Lance HEAD` afterward (no residual "orphaned drift"). Before the +/// fix, roll-back restored via `Dataset::restore` but left the manifest pin +/// behind HEAD, so a subsequent strict write / schema apply failed its +/// HEAD-vs-manifest precondition ("stale view … refresh and retry") — and a +/// failed schema apply's own roll-back leaked +1 each retry (the original bug's +/// loop). With convergence, one roll-back leaves `manifest == HEAD` and the +/// follow-up succeeds. +#[tokio::test] +async fn recovery_rollback_converges_manifest_so_schema_apply_succeeds() { + use omnigraph::db::ReadTarget; + use omnigraph::loader::{LoadMode, load_jsonl}; + use omnigraph::table_store::TableStore; + + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + + let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"alice","age":30}} +{"type":"Person","data":{"name":"bob","age":25}} +"#, + LoadMode::Append, + ) + .await + .unwrap(); + drop(db); + + // Forge a Phase-B residual: advance Person's Lance HEAD without publishing to + // the manifest (the manifest pin stays at the load's committed version). + let person_uri = node_table_uri(uri, "Person"); + let store = TableStore::new(uri); + let mut ds = Dataset::open(&person_uri).await.unwrap(); + let manifest_pin = ds.version().version; + let _ = store + .delete_where(&person_uri, &mut ds, "1 = 2") + .await + .unwrap(); + drop(ds); + + // Roll-back-classified sidecar (post_commit_pin != observed head ⇒ + // UnexpectedAtP1 ⇒ RollBack). + let sidecar_json = format!( + r#"{{ + "schema_version": 1, + "operation_id": "01H0000000000000000000CVG", + "started_at": "0", + "branch": null, + "actor_id": "act-test", + "writer_kind": "Mutation", + "tables": [ + {{ + "table_key": "node:Person", + "table_path": "{}", + "expected_version": {}, + "post_commit_pin": {} + }} + ] + }}"#, + person_uri, manifest_pin, manifest_pin + ); + write_sidecar_file(dir.path(), "01H0000000000000000000CVG", &sidecar_json); + + // Reopen runs the sweep: restore Person to manifest_pin, then PUBLISH so the + // manifest tracks the restored Lance HEAD. + let db = Omnigraph::open(uri).await.unwrap(); + + // Convergence: manifest pin == Lance HEAD. Fails before the fix — the + // manifest stays at manifest_pin while HEAD advanced past it. + let snap = db.snapshot_of(ReadTarget::branch("main")).await.unwrap(); + let entry = snap.entry("node:Person").unwrap(); + let lance_head = Dataset::open(&person_uri).await.unwrap().version().version; + assert_eq!( + entry.table_version, lance_head, + "roll-back must publish so manifest pin ({}) == Lance HEAD ({})", + entry.table_version, lance_head, + ); + + // The +1-loop victim: an additive schema apply must now succeed (its + // HEAD-vs-manifest precondition is satisfied). Before the fix this failed + // with "stale view … refresh and retry". + let desired = TEST_SCHEMA.replace( + " age: I32?\n}", + " age: I32?\n nickname: String?\n}", + ); + db.apply_schema(&desired) + .await + .expect("schema apply after a converging roll-back must succeed"); +} + // ===================================================================== // Phase 4 — roll-forward path + audit row recording // ===================================================================== diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 425fcee..f18600b 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -34,10 +34,10 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav | `s3_storage.rs` | S3-backed graph (skipped unless `OMNIGRAPH_S3_TEST_BUCKET` is set) | | `lance_version_columns.rs` | Per-row `_row_last_updated_at_version` behavior | | `validators.rs` | Schema constraint enforcement (enum, range, unique, cardinality) across JSONL, insert, update paths | -| `maintenance.rs` | `optimize` (compaction) + `cleanup` (version GC): empty/idempotent/no-op edges, policy validation, head preservation | -| `failpoints.rs` | Failure-injection coverage (gated on `failpoints` feature). Includes the four per-writer Phase B → recovery integration tests (`recovery_rolls_forward_after_finalize_publisher_failure`, `schema_apply_phase_b_failure_recovered_on_next_open`, `branch_merge_phase_b_failure_recovered_on_next_open`, `ensure_indices_phase_b_failure_recovered_on_next_open`). | +| `maintenance.rs` | `optimize` (compaction) + `cleanup` (version GC): empty/idempotent/no-op edges, policy validation, head preservation; `optimize` publishes the compacted version so the manifest tracks the Lance HEAD and a subsequent schema apply succeeds (`optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds`), and reconciles a pre-existing manifest-behind-HEAD drift forged via raw Lance compaction (`optimize_reconciles_preexisting_manifest_head_drift`) | +| `failpoints.rs` | Failure-injection coverage (gated on `failpoints` feature). Includes the five per-writer Phase B → recovery integration tests (`recovery_rolls_forward_after_finalize_publisher_failure`, `schema_apply_phase_b_failure_recovered_on_next_open`, `branch_merge_phase_b_failure_recovered_on_next_open`, `ensure_indices_phase_b_failure_recovered_on_next_open`, `optimize_phase_b_failure_recovered_on_next_open`). | | `recovery.rs` | Open-time recovery sweep — sidecar I/O, classifier dispatch (NoMovement / RolledPastExpected / UnexpectedAtP1 / UnexpectedMultistep / InvariantViolation), all-or-nothing decision, roll-forward via `ManifestBatchPublisher::publish`, roll-back via `Dataset::restore`, audit row in `_graph_commit_recoveries.lance`, `OpenMode::ReadOnly` skip path | -| `composite_flow.rs` | Compositional/narrative end-to-end stories — multi-step flows that compose mechanics covered by other test files. Catches integration regressions where individual operations all pass their unit tests but their composition breaks (sequential merges, post-merge main writes, time-travel through merge DAG, reopen consistency over multi-merge histories). | +| `composite_flow.rs` | Compositional/narrative end-to-end stories — multi-step flows that compose mechanics covered by other test files. Catches integration regressions where individual operations all pass their unit tests but their composition breaks (sequential merges, post-merge main writes, time-travel through merge DAG, reopen consistency over multi-merge histories, post-optimize and post-cleanup strict writes). | ## Fixtures diff --git a/docs/dev/writes.md b/docs/dev/writes.md index 8b692b4..d2c7c7e 100644 --- a/docs/dev/writes.md +++ b/docs/dev/writes.md @@ -157,10 +157,14 @@ are left at `Lance HEAD = manifest_pinned + 1`. **Recovery protocol** (lifecycle of every staged-write writer — `MutationStaging::finalize`, `schema_apply::apply_schema_with_lock`, -`branch_merge_on_current_target`, `ensure_indices_for_branch`): +`branch_merge_on_current_target`, `ensure_indices_for_branch`, +`optimize_all_tables`): 1. **Phase A**: writer writes a sidecar JSON to - `__recovery/{ulid}.json` BEFORE its first `commit_staged`. The + `__recovery/{ulid}.json` BEFORE its first HEAD-advancing commit + (`commit_staged`, or `compact_files` for `optimize_all_tables`, + which advances the Lance HEAD via a reserve-fragments + rewrite + commit rather than a staged write). The sidecar names every `(table_key, table_path, expected_version, post_commit_pin)` it intends to commit + the writer kind + actor_id. @@ -195,8 +199,13 @@ recovery sweep in `crates/omnigraph/src/db/manifest/recovery.rs`: otherwise full open-time recovery rolls them back and refresh-time recovery leaves them for the next read-write open. - Otherwise **roll back**: per-table `Dataset::restore` to the - manifest-pinned table version for that branch. Rollback records the - actual restore target in the audit row's `to_version`. + manifest-pinned table version, then a single `ManifestBatchPublisher::publish` + of the restored HEAD — symmetric with roll-forward, so `manifest == HEAD` + after recovery (no residual drift). This convergence is what lets a + failed-then-retried schema apply succeed instead of failing one version higher + each iteration. The audit row's `to_version` records the logical + rolled-back-to version (`manifest_pinned`); the manifest is published at the + restore commit (`manifest_pinned + 1`, same content). - After a successful roll-forward or roll-back, an audit row is recorded — `_graph_commits.lance` carries a commit tagged `actor_id = "omnigraph:recovery"`, and a sibling diff --git a/docs/user/branches-commits.md b/docs/user/branches-commits.md index 0565186..a4044cb 100644 --- a/docs/user/branches-commits.md +++ b/docs/user/branches-commits.md @@ -58,6 +58,6 @@ Internal or legacy branch refs: ## L2 — Recovery audit trail -The four migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`) protect their multi-table commits with a sidecar at `__recovery/{ulid}.json` written before Phase B and deleted after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the recovery sweep in `crates/omnigraph/src/db/manifest/recovery.rs`: classify per-table state, decide all-or-nothing per sidecar, roll forward / back, record an audit row. +The five migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`, `optimize_all_tables`) protect their multi-table commits with a sidecar at `__recovery/{ulid}.json` written before Phase B and deleted after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the recovery sweep in `crates/omnigraph/src/db/manifest/recovery.rs`: classify per-table state, decide all-or-nothing per sidecar, roll forward / back, record an audit row. Audit rows live in `_graph_commit_recoveries.lance` (sibling to `_graph_commits.lance`) and reference the commit graph by `graph_commit_id`. The linked recovery commit is identified by that same `graph_commit_id`, and `actor_id="omnigraph:recovery"` is stored in `_graph_commit_actors.lance` (joined by `graph_commit_id`) — `_graph_commits.lance` itself does not carry the `actor_id` column. To find recoveries for a specific original actor: `omnigraph commit list --filter actor=omnigraph:recovery`, then join to `_graph_commit_recoveries.lance` by `graph_commit_id` to read `recovery_for_actor`. Schema: see `crates/omnigraph/src/db/recovery_audit.rs`. diff --git a/docs/user/maintenance.md b/docs/user/maintenance.md index 3628fa0..a835799 100644 --- a/docs/user/maintenance.md +++ b/docs/user/maintenance.md @@ -4,8 +4,10 @@ ## `optimize_all_tables(db)` — non-destructive -- Lance `compact_files()` on every node + edge table on `main`. -- Rewrites small fragments into fewer large ones; old fragments remain reachable via older manifests. +- Lance `compact_files()` on every node + edge table on `main`, then **publishes the compacted version to the `__manifest`** so the manifest's `table_version` tracks the compacted Lance HEAD. Reads pin the manifest version, so without this publish compaction would be invisible to readers *and* would break the HEAD-vs-manifest precondition of the next schema apply / strict update/delete ("stale view … refresh and retry"). The publish advances the graph version (a system-attributed commit) only for tables that actually compacted. +- Rewrites small fragments into fewer large ones; old fragments remain reachable via older manifests until `cleanup` runs. +- Each table's compact→publish runs under its per-`(table, main)` write queue (serializing with concurrent mutations — compaction is a Lance `Rewrite` op that retryable-conflicts with a concurrent merge/update/delete on overlapping fragments). The Lance-HEAD-before-manifest-publish gap is covered by a `SidecarKind::Optimize` recovery sidecar (loose-match): a crash in that window rolls the compacted version forward on the next `Omnigraph::open` (compaction is content-preserving, so roll-forward is always safe). +- **Requires a recovered graph.** `optimize` refuses (errors) when an unresolved recovery sidecar is present under `__recovery` — operating on an unrecovered graph could publish a partial write the open-time recovery sweep would roll back. Reopen the graph to run the recovery sweep, then re-run `optimize`. (Recovery roll-back now publishes its restored version, so a recovered graph always satisfies `manifest == Lance HEAD` going in; there is no leftover drift for `optimize` to interpret.) - Bounded by `OMNIGRAPH_MAINTENANCE_CONCURRENCY` (default 8). - Returns `[TableOptimizeStats { table_key, fragments_removed, fragments_added, committed, skipped }]`. - **Blob tables are skipped.** A table that declares any `Blob` property is not compacted: it is reported with `skipped: Some(BlobColumnsUnsupportedByLance)` (and logged via `tracing::warn`) instead of compacted, and the rest of the sweep proceeds normally. The current Lance `compact_files` mis-decodes blob-v2 columns under its forced `BlobHandling::AllBinary` read; **reads and writes are unaffected** — only compaction is. This is gated by `LANCE_SUPPORTS_BLOB_COMPACTION` (`db/omnigraph/optimize.rs`) and removed when the upstream Lance fix lands (see [docs/dev/lance.md](../dev/lance.md)). Consequence: fragment count and deleted-row space on blob tables are not reclaimed until then; query results are never affected. diff --git a/docs/user/storage.md b/docs/user/storage.md index d1c52b5..2c57a92 100644 --- a/docs/user/storage.md +++ b/docs/user/storage.md @@ -94,7 +94,7 @@ flowchart TB - **`nodes/`** and **`edges/`** are sibling directories holding one Lance dataset per declared type. Names are `fnv1a64-hex` of the type name to keep paths fixed-length and case-safe. - **`_graph_commits.lance`** is an L2 dataset that records the graph-level commit DAG, with a paired `_graph_commit_actors.lance` for the actor map. (Pre-v0.4.0 graphs also have inert `_graph_runs.lance` / `_graph_run_actors.lance` from the removed Run state machine; the v2→v3 migration sweeps their stale `__run__*` branches, and the dataset bytes are reclaimed once `delete_prefix` lands.) - **`_graph_commit_recoveries.lance`** — one row per recovery sweep action. Joined to `_graph_commits.lance` by `graph_commit_id`; the linked commit row carries `actor_id=omnigraph:recovery`. Operators correlate recoveries with the original mutations they rolled forward / back via this join. See `crates/omnigraph/src/db/recovery_audit.rs`. -- **`__recovery/{ulid}.json`** — transient sidecar files written by the four migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`) before Phase B begins, deleted after Phase C succeeds. A sidecar persisting after process exit means the writer crashed in the Phase B → Phase C window; the next `Omnigraph::open` recovery sweep processes it. Steady-state directory is empty. See `crates/omnigraph/src/db/manifest/recovery.rs`. +- **`__recovery/{ulid}.json`** — transient sidecar files written by the five migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`, `optimize_all_tables`) before Phase B begins, deleted after Phase C succeeds. A sidecar persisting after process exit means the writer crashed in the Phase B → Phase C window; the next `Omnigraph::open` recovery sweep processes it. Steady-state directory is empty. See `crates/omnigraph/src/db/manifest/recovery.rs`. - **`_refs/branches/{name}.json`** is graph-level branch metadata — pointers from a branch name to the manifest version it heads. - **Inside each Lance dataset** (orange): the standard Lance directory layout. `_versions/{n}.manifest` records every commit; `data/` holds the actual Arrow fragments; `_indices/{uuid}/` holds index segments with their own `fragment_bitmap` for partial coverage; `_refs/` holds Lance-native per-dataset branches and tags. From ab5f3b878a28ae466b5e16f2389ba1c9ece5ac86 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Mon, 8 Jun 2026 17:31:36 +0300 Subject: [PATCH 023/165] docs: add cluster config specs --- docs/dev/cluster-axioms.md | 97 +++ .../dev/cluster-config-implementation-spec.md | 705 ++++++++++++++++++ docs/dev/cluster-config-specs.md | 415 +++++++++++ docs/dev/index.md | 1 + 4 files changed, 1218 insertions(+) create mode 100644 docs/dev/cluster-axioms.md create mode 100644 docs/dev/cluster-config-implementation-spec.md create mode 100644 docs/dev/cluster-config-specs.md diff --git a/docs/dev/cluster-axioms.md b/docs/dev/cluster-axioms.md new file mode 100644 index 0000000..a3793b4 --- /dev/null +++ b/docs/dev/cluster-axioms.md @@ -0,0 +1,97 @@ +# Cluster Control-Plane Axioms + +**Type:** Standing design filter +**Status:** Draft / thinking-in-progress +**Date:** 2026-06-07 +**Relationship:** the distilled axioms behind [cluster-config-specs.md](cluster-config-specs.md). The downstream implementation inventory and blast-radius assessment live in [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md). The high-level spec is the argument; this is the checklist. Hold any config / control-plane / deployment proposal against these and cite them by number (e.g. "violates axiom 5"). + +This file is intentionally short and stable. The axioms are phrased so other +docs can reference "axiom 6" without churn. The motivating requirement comes +first; the core axioms are what the design is *based on*; the derived rules are +consequences that follow from them. + +> **Revision 2026-06-07 — committed to the Terraform paradigm.** State is now an +> **authoritative, locked ledger in a backend** (no longer framed as a +> "mostly-rebuildable projection"); `plan` is a **config ↔ state diff**; and +> **ETL pipelines** join schema as config-defined resources that trigger +> data-plane effects. Secrets live in a gitignored **`.env`** file (`${NAME}`), +> and **query exposure is a policy decision** (no registry `expose:` flag). +> Axioms **2, 5, 6** revised; **12, 13, 14** added. The earlier +> "state is just a rebuildable projection; config is the *only* truth" framing is +> superseded — see axiom 5. +> +> **Revision 2026-06-08 — JSON state first.** The baseline state backend is now +> Terraform-style JSON documents plus backend lock/CAS, not Lance control-plane +> datasets. Lance remains a possible later backend only if row-level history or +> queryability justifies the extra machinery. + +--- + +## Tenet 0 — the motivating requirement + +**0. The Sarah/Bob test.** If one operator changes schema / queries / policies / UI / pipelines / aliases, another operator (or their agent) must learn *what the deployment is and what changed* from **one source, one history, one diff**. Fragmentation across separate mechanisms is the failure the whole design exists to eliminate. Every other axiom is in service of passing this test. + +--- + +## Core axioms (what the design is based on) + +**1. The cluster is the unit of declarative state.** Not the graph (policies, queries, UI, and pipelines cross-cut graphs; "which graphs exist" has no per-graph home), not the fleet (the next scope up — named and deferred). The cluster is what two operators collaborate over; a graph is a *resource within* it. + +**2. Two sources of truth, for two different questions — config for *intent*, state for *deployed reality*.** The version-controlled **config** (a set of files in one folder) is the source of truth for what the cluster *should be*. The **state ledger** is the source of truth for what *is* currently deployed. Change flows one way only: you edit config and `apply` converges the cluster (**code → cluster**, never edit-the-cluster-and-call-it-intent). But "what exists right now" is read from **state**, not re-derived from the world on every command. `plan` is the diff between the two. + +**3. Declarative, not imperative.** You describe the desired end state; the reconciler computes the steps. No runtime mutation API that makes the running system the place *intent* lives. + +**4. As-code is structural, not stylistic (the recursion argument).** Code is the base case; modeling the definition *as data* (a meta-graph describing graphs) recurses with no base case. Config must live **outside** the running system so it is reviewable (PRs), reproducible (clone + apply), diffable as text, and editable by an agent — without the system having to describe itself. + +<!-- Audit fix: JSON keeps the first backend Terraform-shaped and inspectable. +Lance datasets are future optimization, not the baseline state format. --> +**5. The Terraform model: config / state / reconcile — and state is an authoritative, locked ledger.** Config (as code) = desired truth. **State = the authoritative record of what has been applied**, held in a **backend** — the cluster's own object-store backend *or* a separate cloud store, the operator's choice, exactly like a Terraform backend. The baseline representation is JSON documents (`state.json`, status/approval/recovery JSON records) protected by backend lock/CAS, not Lance control-plane tables. State is **locked** during apply so two operators cannot converge concurrently. `validate` parses and schema-checks desired config; `plan` = `diff(config, state)` as a structured artifact with resource digests, dependency edges, state observations, proposed changes, blast radius, and approval gates; `apply` converges the cluster from an accepted fresh plan and **updates state**, and does not acknowledge success until state has recorded the result. A cluster-hosted JSON backend is still a separate state CAS step from graph Lance manifest moves; failures surface a repair/import condition instead of being described as cross-object all-or-nothing. A future Lance-backed state backend or cluster manifest publisher is optional and must earn its complexity by needing row-level queryability/history or tighter publish fencing. Because OmniGraph's running cluster is self-describing (manifests, commit logs), state is *reconstructable* by import/refresh if lost — its edge over opaque-cloud Terraform — but it is **treated as the source of truth for current reality, not casually regenerated**. The one slice that can never be reconstructed (who approved an irreversible apply) lives in the durable audit ledger; state references it (axiom 11). + +**6. The control plane reconciles definition, not data — across two data-plane seams.** Definition — schema, policies, queries, UI, bindings, aliases, ETL **pipelines**, embeddings config, and the set of graphs — is reconciled. Data — rows, edges, vectors — is data-plane content, versioned by the commit DAG and produced by `load` / `mutate` and **pipeline execution**, sitting **outside** the reconcile loop. Exactly two definition kinds *trigger* a data-plane effect without owning data: **schema** (a migration conforms existing rows; `plan` previews its impact) and **ETL pipelines** (their execution ingests external data). The loop converges their *definitions*; the data they produce is never what it reconciles. + +**7. Operated by agent (agent-as-controller).** An agent authors config changes and drives reconciliation as an authenticated actor, subject to policy and approval gates — no human state-management burden. This fuses Terraform's as-code config with Kubernetes' continuous reconciliation. + +--- + +## Derived rules (consequences of the axioms) + +**8. The reversibility gradient gates apply — including drift correction.** Irreversible / data-loss operations (drop a graph, hard-drop schema data, a pipeline that overwrites) and compatibility-narrowing migrations (for example, future validated enum narrowing) are gated; reversible ones (recolor a dashboard) are not. The gate is keyed to physics, not to who operates it, and a reconciler "just fixing drift" is never an exception. + +**9. Atomicity and referential integrity are plan-time, not runtime.** `ApplyGroup` is the atomicity unit; cross-resource references *force* grouping (mandatory, not opt-in); references use typed resource/provider addresses (`graph.knowledge`, `query.knowledge.find_experts`, `provider.source.github_org`) so the planner can reject wrong-kind or missing targets before apply — bare names in a kind-fixed field are accepted shorthand and normalized to the typed address (fix 2026-06-08), while a kind-ambiguous value (e.g. `source: github`) is rejected; a reference to a missing or being-removed resource is a fail-closed `plan` error, not a deferred runtime failure. + +**10. Secrets live in a `.env` file; connection/identity is per-operator.** The committed cluster config carries **no secret values** — only `${NAME}` references. The values (embedding API keys, pipeline **source credentials**, per-deployment settings) live in a separate **`.env` file** — which is gitignored and supplied per deployment, never committed. Separately, an operator's own connection (which cluster, which token) is the per-operator layer, distinct from both the shared config and its `.env` file. + +**11. Approvals and audit live in a durable ledger, not inline in state.** State *references* the audit record by id. In the baseline, that ledger is append-only JSON records in the state backend; a future Lance table is an implementation option, not a requirement. This keeps the bulk of state reconstructable and keeps approval facts — "who authorized this irreversible apply" — where loss is impossible. + +**12. State lives in a backend and is locked.** The state ledger is stored in a configurable backend — the cluster's own backend, or a separate cloud store — and `plan`/`apply` acquire a **state lock** first, so concurrent applies serialize instead of racing. (Generalizes the existing `__schema_apply_lock__` from schema scope to cluster scope.) The backend choice is part of the safety model: the first backend should be JSON plus object-store lock/CAS; any Lance-backed state backend needs its own RFC-level proof that the table semantics are worth the control-plane complexity. + +**13. Pipelines are definition; their execution is data-plane.** An ETL pipeline (external source → transform → target graph) is **declared in config and reconciled like any resource**; *running* it produces ordinary data-plane writes (`load`/`mutate`) outside the reconcile loop. `apply` converges the pipeline's *definition* (create / update / delete / schedule); the rows it ingests are never reconciled. A fan-out run over several graphs is statusful rather than magically atomic: each target records commit id, status, retryability, and idempotency key unless the pipeline explicitly uses a branch/merge protocol that can fence the whole target set. Source credentials are secret references (axiom 10). + +<!-- Audit fix: current shipped behavior still has mcp.expose and coarse +invoke_query. This axiom is the target control-plane rule, not a statement +about today's server catalog. --> +**14. Exposure is a policy decision, not a config flag.** Target design: which stored queries (and the tools/dashboards built on them) an actor may **list or invoke** is decided by the policy layer (Cedar: `invoke_query` + catalog visibility), not by a per-query `expose:` boolean. The registry only says a query *exists* (name → file); **policy says who may see and run it**, so the MCP catalog (`GET /queries`) becomes each actor's policy-permitted set. This supersedes the engine's current `mcp.expose` flag only after per-query `invoke_query` scope and Cedar-filtered catalog listing land; until then, proposals must state the compatibility bridge to today's `mcp.expose` + coarse invocation gate. + +--- + +## The one-line compression + +**One cluster; config (a folder of files) is desired truth and a locked state ledger in a backend is deployed truth; `plan` diffs them, `apply` converges the cluster and updates state, an agent drives the loop — reconciling the cluster's *definition* (schema, policies, queries, UI, pipelines, …) and never its data — so any operator sees the whole system and its history from one place.** + +--- + +## How to use this file + +- **Reviewing a proposal:** walk axioms 0–14; any conflict is the burden of the proposer to justify. The most common tensions: + - Treating the *running system* as the source of truth for **intent** → axioms 2, 4 (intent lives in config). + - Treating state as a throwaway derivation rather than an authoritative, locked, backend-held ledger → axiom 5, 12. + - A runtime config-mutation API instead of declarative apply → axiom 3. + - "State" meaning a per-operator selection rather than the applied-cluster ledger → axiom 5. + - The control plane reconciling (or owning) data — including treating pipeline *rows* as reconciled state → axiom 6, 13. + - Treating fan-out pipeline execution as atomic without a branch/merge protocol or per-target status ledger → axiom 13. + - Per-graph or per-server scoping of cluster-level definition → axiom 1. + - Bare string references that force the planner to guess whether `knowledge` means a graph, query, provider, or path → axiom 9. + - A secret value (token, embedding key, pipeline source credential) inline in config instead of in the gitignored `.env` file → axiom 10. + - A per-query `expose:`/visibility flag in target-state cluster config instead of governing list/invoke in policy; or failing to account for today's `mcp.expose` compatibility bridge → axiom 14. + - Shipping `apply` before hermetic `validate` + read-only `plan` tests, or shipping graph/schema-moving apply before recovery tests for the graph/resource-moved-before-cluster-publish gap → axiom 5 and axiom 12. +- **Citing:** reference axioms by number in PRs and review comments so the rationale is stable across renames and refactors. diff --git a/docs/dev/cluster-config-implementation-spec.md b/docs/dev/cluster-config-implementation-spec.md new file mode 100644 index 0000000..5121451 --- /dev/null +++ b/docs/dev/cluster-config-implementation-spec.md @@ -0,0 +1,705 @@ +# Cluster Config Implementation Spec And Blast Radius + +**Status:** Draft / implementation planning +**Type:** Downstream design spec +**Date:** 2026-06-08 +**Relationship:** companion to [cluster-config-specs.md](cluster-config-specs.md) +and [cluster-axioms.md](cluster-axioms.md). The high-level spec explains why +the cluster control plane should exist; this file names what must change +downstream and how large the blast radius is. + +<!-- Spec note: this file exists so the user-facing cluster spec can stay +readable. Keep implementation inventories, rollout phases, and test ownership +here instead of expanding the narrative spec into an encyclopedia. --> + +## Executive Summary + +Overall blast radius: **very high**. + +This is not a small extension to `omnigraph.yaml`. The target design creates a +new shared cluster desired-state document, a locked state ledger, a cluster +manifest publisher, and a reconciler that coordinates resources above a single +graph. The existing config system remains useful, but its role changes: + +- `omnigraph.yaml` / global config remains the per-operator and startup bridge. +- `cluster.yaml` becomes shared desired state for a deployment. +- The cluster state ledger becomes the authoritative record of applied reality. +- Server/runtime surfaces eventually read from the cluster catalog instead of + only from process-start config. + +Safe rollout requires an additive path. Do not replace the current config, +server, or policy behavior in one step. + +## Current Surfaces Surveyed + +| Surface | Current behavior | Why it matters | +|---|---|---| +| `omnigraph-config::OmnigraphConfig` | Layered global/state/project config for CLI and server startup; strict `version: 1`; named maps replace wholesale | A cluster spec needs different ownership and merge semantics; do not stretch this type until it becomes ambiguous | +| `omnigraph-server::load_server_settings` | Opens either one selected graph or every configured embedded graph in multi mode | Cluster config changes startup, registry identity, and eventually runtime reconcile | +| `GraphRegistry` | Holds open graph handles; production registry is startup-only today; runtime insert is test-only | Cluster apply wants graph add/remove/reload as real control-plane operations | +| `omnigraph-queries::QueryRegistry` | Loads `.gq` files from `queries:` and honors `mcp.expose` for catalog listing | Target cluster config removes exposure from the registry and moves list/invoke to policy | +| `omnigraph-policy::PolicyAction` | Per-graph actions plus server-scoped `graph_list`; `invoke_query` is graph-scoped and coarse | Cluster plan/apply and per-query exposure need new policy scope without breaking coarse rules | +| Engine graph manifest | Graph-level atomic visibility via `__manifest`, expected table versions, and recovery sidecars | Cluster apply needs a higher-level publisher; Lance still commits per dataset | +| Schema apply | Existing plan/apply/lock shape for one graph; soft/hard drops already modeled | This is the prototype resource reconciler, but cluster apply cannot call it blindly and then claim cluster atomicity | +| Public docs/tests | Config, policy, server, and query behavior are already documented and tested | Every behavior change below has user docs and test fallout | + +## Compatibility Stance + +<!-- Spec note: keep `cluster.yaml` separate from `omnigraph.yaml` because the +current file is deliberately layered and partly per-operator. Collapsing shared +cluster intent into it would blur the source-of-truth split the high-level spec +is trying to create. --> + +1. `cluster.yaml` is a new target-state file, not `omnigraph.yaml` v2. +2. Existing `omnigraph.yaml` keeps working for CLI, server boot, aliases, + graph locators, bearer-token env lookup, and the current stored-query + registry. +3. Initial cluster commands are explicit: `omnigraph cluster validate`, + `omnigraph cluster plan`, `omnigraph cluster apply`, `omnigraph cluster + status`, `omnigraph cluster refresh`, and `omnigraph cluster import`. +4. Cluster config is one shared folder, resolved from the command's cluster + root or explicit path. It is not merged from global + project + active + context layers. +5. The per-operator connection layer selects the cluster root and actor + identity. It is not committed into `cluster.yaml`. +6. `mcp.expose` remains supported in current `omnigraph.yaml` until the + per-query policy replacement ships. + +## Terraform-Aligned Schema Validation + +<!-- Spec note: Terraform is strict for resource/provider/module configuration, +but looser for variable-value inputs such as `.tfvars` and `TF_VAR_*`. For +cluster desired state we borrow the strict resource-schema posture because +`cluster.yaml` is shared intent, not an operator-local variable bag. --> + +Every field in target-state `cluster.yaml` must be **honored or rejected**: + +- If a field is part of the declared resource schema, it must affect + validation, plan, apply, state, or status. +- If a field is misspelled, placed under the wrong resource kind, or reserved + for a future phase, `cluster validate` / `cluster plan` must fail with a + typed diagnostic. +- Compatibility warnings are allowed only in an explicit migration window for + old schema versions. They are not allowed in the target schema. +- Free-form extension areas must be named as such, for example `labels`, + `metadata`, `vars`, or `provider_options`; accidental unknown keys are never + treated as extension data. + +Examples: + +```yaml +graphs: + knowledge: + schema: ./knowledge.pg + lables: { team: platform } # invalid: typo, use `labels` + +pipelines: + github_sync: + source: { kind: github, token: ${GITHUB_TOKEN} } + into: + - { graph: engineering, map: ./github.map.yaml } + retry_magic: true # invalid unless `retry_magic` is in schema +``` + +```yaml +graphs: + knowledge: + schema: ./knowledge.pg + labels: { team: platform } # valid free-form metadata bucket + provider_options: + lance: + compaction_window: daily # valid only if this extension is declared +``` + +## Typed Resource And Provider Addresses + +<!-- Spec note: this is the Terraform-aligned version of "typed locators". +The target cluster spec should not ask later code to guess whether a string is a +graph name, query name, server endpoint, storage URI, source connector, or +credential reference. References carry their kind. --> + +<!-- Fix (2026-06-08): resolved the "shorthand may exist" (here) vs "bare strings +are bad shape" (below) contradiction. The rule is now explicit: bare names ARE +valid shorthand in a field whose schema fixes the referent kind (normalized to a +typed address); "bad shape" means a value whose KIND is ambiguous or WRONG, not +merely bare. This also makes the high-level spec's bare examples (policy +`graphs:`/`applies_to:` lists, pipeline `into.graph`, dashboard `graphs:`) valid. --> +A locator is a typed address to another declared thing. **Internally — in plan and +state — every reference is a typed address** (axiom 9). At the config *surface* a +field may accept **bare shorthand when its schema fixes the referent kind** (a +policy `applies_to:` list is graph refs; a pipeline `into.graph` is a graph id) — +the parser normalizes it to the typed address before planning. A value whose +*kind* is ambiguous or wrong (a `source:` that could be a connector type, an +instance, or a provider) has no safe normalization and must be a typed +`provider.*` address or an explicit inline block. + +Target address forms: + +```text +graph.<graph_id> +schema.<graph_id> +query.<graph_id>.<query_name> +policy.<policy_name> +ui.dashboard.<dashboard_name> +pipeline.<pipeline_name> +provider.storage.<provider_name> +provider.source.<provider_name> +provider.embedding.<provider_name> +``` + +Bad shape — the value's **kind is ambiguous or wrong**, not merely bare: + +```yaml +pipelines: + github_sync: + source: github # AMBIGUOUS kind: connector type, instance, or provider? + # → provider.source.<name> or inline { kind: github, ... } +policies: + base_rbac: + applies_to: [query.knowledge.find_experts] # WRONG kind: a query address in a graph-ref field +``` + +OK shorthand (kind fixed by the field → normalized): + +```yaml +policies: + base_rbac: + applies_to: [knowledge, engineering] # bare names in a graph-ref field → graph.knowledge, graph.engineering +``` + +Target shape: + +```yaml +providers: + storage: + prod_graphs: + kind: s3 + bucket: company + prefix: prod + source: + github_org: + kind: github + token: ${GITHUB_TOKEN} + +graphs: + knowledge: + storage: provider.storage.prod_graphs + path: graphs/knowledge.omni + schema: ./knowledge.pg + engineering: + storage: provider.storage.prod_graphs + path: graphs/engineering.omni + schema: ./engineering.pg + +policies: + base_rbac: + file: ./base_rbac.policy.yaml + applies_to: + - graph.knowledge + - graph.engineering + +pipelines: + github_sync: + source: provider.source.github_org + into: + - { graph: graph.engineering, map: ./github_to_engineering.map.yaml } + - { graph: graph.knowledge, map: ./github_to_people.map.yaml } +``` + +<!-- Fix (2026-06-08): this example shows the EXPLICIT/external graph-storage case +(`storage:` + `path:`). It is not the default — per "Known High-Risk Design +Decisions" §2 and the cluster storage layout, graph roots derive to +`ClusterRoot/graphs/<id>.omni` by default; an external storage provider is the +opt-in. The pipeline `into.graph` here is typed (`graph.engineering`); the bare +`{ graph: engineering, ... }` shorthand is equally valid (normalized). --> + +Validation rules: + +- A field that expects a graph address accepts `graph.<id>`, not + `query.<graph>.<name>` or an arbitrary string. +- A field that expects a query address accepts `query.<graph>.<name>`, and the + planner validates both the graph and the query symbol. +- A field that expects a source provider accepts `provider.source.<name>`, not + `provider.storage.<name>`. +- A field that expects storage accepts `provider.storage.<name>` or an explicit + storage block, not a server URL or source connector. +<!-- Fix (2026-06-08): shorthand is a present rule, not "future syntax" — it is how +the high-level spec's bare examples are valid. --> +- A field whose schema **fixes the kind** accepts bare shorthand (e.g. `knowledge` + in a graph-ref field) and normalizes it to the typed address; a kind-ambiguous + or wrong-kind value is rejected with a typed diagnostic. +- Plan and state always store the **normalized typed address**, regardless of + whether the surface used shorthand. + +## Target Components + +Preferred split: + +| Component | Responsibility | Depends on | +|---|---|---| +| `omnigraph-cluster` crate | Cluster spec types, path resolution, resource graph, plan model, state backend traits, apply orchestration | `omnigraph-config` only for shared simple config types if needed; avoid server deps | +| `omnigraph` engine additions | Graph lifecycle primitives, schema-apply integration, recovery hooks for graph moves during cluster apply; optional future cluster manifest publisher if JSON state is not enough | Lance, existing graph manifest/recovery | +| `omnigraph-cli` | `cluster *` commands, plan rendering, approval collection, state lock UX | `omnigraph-cluster`, engine | +| `omnigraph-server` | Optional boot from cluster state, registry reload, status endpoints, policy-filtered query catalog | `omnigraph-cluster`, engine, policy | +| `omnigraph-policy` | Cluster/server actions, per-query list/invoke scope, approval policy predicates | none above server | +| `omnigraph-queries` | Registry without exposure side-channel; dependency metadata for downstream validation | compiler/config | +| `omnigraph-api-types` | New status/plan/apply response types if cluster HTTP endpoints ship | serde only | + +If the first implementation avoids a new crate, keep the same boundary in +modules. The important constraint is that cluster spec parsing must not drag +HTTP/server code into compiler or engine crates. + +## Resource Model + +Resource identity is stable and typed: + +```text +ClusterRoot +ResourceKey = <kind>/<scope>/<name> +ResourceAddress = <kind>.<name> | <kind>.<graph_id>.<name> +ProviderAddress = provider.<kind>.<name> + +graph/cluster/knowledge +schema/graph:knowledge/main +query/graph:knowledge/find_experts +policy/cluster/base_rbac +ui/cluster/dashboard.overview +pipeline/cluster/github_sync +alias/cluster/experts +embedding/cluster/default +``` + +<!-- Fix (2026-06-08): resource key uses `dashboard.overview` (dot) to match the +address form `ui.dashboard.<dashboard_name>` — was `dashboard:overview`. `dashboard` +is the only ui sub-kind today. --> + +Resource records carry: + +| Field | Meaning | +|---|---| +| `kind` | Graph, Schema, Query, PolicyBundle, UiSpec, Binding, Alias, EmbeddingConfig, Pipeline | +| `scope` | Cluster or graph id | +| `name` | Stable resource name inside scope | +| `fingerprint` | Content hash of the normalized spec and all referenced files | +| `dependencies` | Resource keys this resource references | +| `observed` | Applied graph manifest version, policy digest, query digest, schedule id, etc. | +| `status` | `Pending`, `Planned`, `Applying`, `Applied`, `Drifted`, `Blocked`, `Error` | +| `conditions` | Typed details such as `ActualAppliedStatePending`, `NeedsApproval`, `DependencyMissing`, `PartialPipelineRun` | + +The planner builds a dependency graph from these records and uses it for both +validation and blast-radius reporting. + +## Terraform-Style Validate / Plan / Apply + +The cluster workflow deliberately mirrors Terraform's safe sequence: + +```text +cluster validate # parse + schema-check desired config, no state mutation +cluster plan # diff desired config against state, with optional refresh +cluster apply # apply an accepted fresh plan and update state +cluster status # read state-backed deployed reality +cluster refresh # repair/import observations from actual cluster state +``` + +Implementation rollout follows the same safety posture: ship parser/validate +first, then read-only plan, then state backend and lock, then apply. + +The plan is a structured artifact, not just terminal text. It must include: + +| Plan field | Why it exists | +|---|---| +| `desired_revision` | Git commit / config digest being evaluated | +| `resource_digests` | Exact digest of every schema, query, policy, UI, pipeline, and map file | +| `dependencies` | Edges such as query -> graph/schema, dashboard -> query, pipeline -> source provider + graph | +| `state_observations` | Applied revision, resource fingerprints, graph manifest versions, status conditions, and drift | +| `changes` | Create/update/delete/replace/refresh-only operations | +| `blast_radius` | Downstream resources to revalidate or affected behavior to surface | +| `approvals_required` | Irreversible/data-loss or compatibility-narrowing gates | + +`cluster apply` must reject a stale plan when state, resource digests, or +observed graph versions no longer match the plan base. The operator or agent +must re-plan or explicitly refresh first. + +## Cluster Storage Layout + +Target Phase-1 cluster-root layout: + +```text +<cluster-root>/ + __cluster/ + state.json + lock.json + status/ + <resource-address>.json + approvals/ + <ulid>.json + recoveries/ + <ulid>.json + recovery/ + <ulid>.json + resources/ + query/<graph>/<name>/<digest>.gq + policy/<name>/<digest>.yaml + ui/<name>/<digest>.dashboard.yaml + pipeline/<name>/<digest>.pipeline.yaml + graphs/ + <graph_id>.omni/ +``` + +<!-- Spec note: JSON is the baseline because it matches Terraform state, is +easy to inspect/repair, and avoids bootstrapping Lance datasets before the +control-plane semantics are proven. --> +The exact filenames can change, but the shape cannot: + +- There is one cluster-control namespace under the cluster root. +- Graph data remains in ordinary OmniGraph graph roots. +- State is a locked/CAS-updated JSON document, not a Lance dataset. +- Status, approval, and recovery ledgers are append-only or per-resource JSON + records until table semantics are proven necessary. +- Resource payloads are content-addressed by digest so apply can be idempotent. +- Cluster state is not inferred from the operator's working tree. +- A Lance-backed control-plane store is a future backend option only if + row-level queryability/history or tighter publish fencing justifies it. + +## State Backend Protocol + +### Cluster-Hosted JSON State + +When `state.backend: cluster`, the baseline backend stores JSON documents under +`<cluster-root>/__cluster/` and protects `state.json` with object-store lock/CAS. +It is cluster-hosted, but it is still a separate state write from graph Lance +manifest movement. + +Apply protocol: + +1. Acquire the cluster state lock. +2. Read current `state.json` and backend CAS token / object generation. +3. Validate plan base still matches state. +4. Write a cluster recovery sidecar before any graph manifest or non-idempotent + resource can move. +5. Write content-addressed resource payloads and perform any required graph + manifest movements. +6. CAS-update `state.json` with the new applied revision, resource + fingerprints, observed graph versions, status references, and approval / + recovery references. +7. If step 6 fails after actual resources moved, do not acknowledge success. + Surface `ActualAppliedStatePending` and require `refresh` / `import` repair. +8. Delete the sidecar and release the lock only after the state outcome is + recorded. + +### External State + +<!-- Spec note: external state is a separate commit domain. The protocol below +prevents an apply from returning success after the cluster moved but the state +ledger failed to record that movement. --> + +When `state.backend` points outside the cluster root, the same JSON state shape +lives in an external store. It is locked and CAS-updated, but it is not atomic +with Lance or OmniGraph manifests. + +Apply protocol: + +1. Acquire the external state lock. +2. Read state and CAS token. +3. Validate plan base still matches state. +4. Write a cluster recovery sidecar. +5. Perform the cluster resource changes. +6. CAS-update external state with the new applied revision, statuses, and the + observed graph manifest / resource versions it records. +7. If step 6 fails, do not acknowledge success. Surface + `ActualAppliedStatePending` and require `refresh` / `import` repair. +8. Release the external lock only after the state outcome is recorded. + +This mode can be strongly coordinated, but it must never be documented as one +atomic commit across both stores. + +### Future Lance-Backed State + +A Lance-backed state/status/approval/recovery store is deliberately not the +baseline. It becomes attractive only if JSON files become a real liability: +large status sets need structured filtering, approval/recovery history needs +table scans, or cluster apply needs a manifest publisher that can fence state +and graph-version pins together. Until then, Lance datasets add bootstrapping, +schema migration, and control-plane recovery surface without enough benefit. + +## Cluster Manifest Publisher + +The cluster publisher is a possible later layer above today's graph publisher. +It does not replace Lance or the per-graph `__manifest` table, and it is not +required for Phase-1 JSON state / read-only plan. + +Required semantics: + +| Requirement | Detail | +|---|---| +| Expected-version CAS | Every resource in an apply group supplies its expected current version/fingerprint | +| Resource changes | Register/update/tombstone resource payloads and graph version pins | +| Graph-head fencing | If a graph schema/lifecycle operation moves a graph manifest, the cluster manifest records the exact graph manifest version | +| Sidecar coverage | Any graph or cluster resource that can move before cluster publish must be recoverable all-or-nothing | +| Deterministic publish order | Sidecars and apply groups process in stable order | +| Loud partials | If a group cannot be rolled back or forward in-process, status records the condition before more apply work proceeds | + +The risky case is nested publish: + +```text +schema apply moves graph:knowledge manifest +cluster apply has not yet published query/policy/state records +process crashes +``` + +That is not safe unless the cluster sidecar records enough information to roll +the graph movement forward into the cluster manifest or roll it back using the +same recovery discipline as current graph recovery. + +## Plan Model + +Plan output is a durable, replay-checked proposal, not just pretty text: + +```text +Plan { + plan_id, + desired_revision, + base_state_revision, + base_state_cas, + changes[], + apply_groups[], + approvals_required[], + blast_radius, + diagnostics[] +} +``` + +Each change records: + +| Field | Meaning | +|---|---| +| `resource` | Stable `ResourceKey` | +| `operation` | Create, Update, Delete, Replace, RefreshOnly | +| `reversibility` | Reversible, Recoverable, CompatibilityNarrowing, IrreversibleDataLoss | +| `effect` | ConfigOnly, Catalog, GraphDefinition, GraphDataRewrite, DataPlaneSchedule | +| `downstream` | Resources that must be revalidated or will observe changed behavior | +| `approval` | None, HumanRequired, PolicyRequired, AlreadySatisfied | + +`apply` must re-read state and reject stale plans unless an explicit +`--refresh` / `--replan` path recomputes the plan. + +## Downstream Dependency Rules + +These are the concrete "what requires downstream" rules. + +| Changed resource | Must revalidate / recompute downstream | Blocking failures | +|---|---|---| +| Graph create/delete/rename | Policies, queries, aliases, dashboards, pipelines, bindings, server registry, state graph set | Dangling graph references; duplicate URI; invalid `GraphId`; graph delete without irreversible approval | +| Schema | Stored queries, pipeline maps, UI bindings/query outputs, embedding/index config, data-impact preview, policy predicates once row/type pushdown exists | Unsupported migration; query breakage; missing target type/property; hard drop without approval | +| Stored query | Aliases, UI bindings, policy list/invoke grants, MCP/tool catalog compatibility, typed params | Query file parse/type errors; registry key != `query <name>`; removed query still referenced | +| Policy bundle | Query catalog visibility, graph/server action authorization, approval gates, bootstrap permissions | Invalid Cedar/YAML; server-scoped action in graph policy; per-query list/invoke gap unhandled | +| UI/dashboard | Query bindings, graph refs, output field expectations, policy visibility for referenced queries | Binding to missing graph/query/param/output | +| Alias | CLI command resolution, graph/query refs, shared-vs-personal boundary | Dangling graph/query; mutation alias pointing at read-only context | +| Embedding config | Schema `@embed` columns, model dimension, index rebuild/reconcile, env refs | Dimension mismatch; missing env ref; unsupported model/provider | +| Pipeline definition | Target graph schemas, mapping files, env refs, scheduler/runtime state, per-target run ledger | Missing target graph/type/property; overwrite mode without approval; source secret missing | +| Binding | Referenced source/surface pair, dependency order, visibility policy | Missing source or target; incompatible params | +| State backend config | Lock implementation, import/refresh protocol, apply acknowledgements | Backend missing CAS/lock; state CAS failure after graph/resource movement | + +## Blast Radius Matrix + +| Area | Required downstream change | Blast radius | Notes | +|---|---|---|---| +| Config parsing | Add strict `cluster.yaml` parser, path/env-ref resolver, resource fingerprints, no layered merge | High | Separate from `OmnigraphConfig`; existing config tests still need backcompat coverage | +| CLI | Add `cluster validate/plan/apply/status/refresh/import`, plan rendering, approval flags, actor threading | High | Must not change existing command selection or `omnigraph use` behavior | +| State backend | Add JSON state document, status/approval/recovery records, lock/CAS, and import/refresh repair | High | Must not silently succeed after state CAS failure | +| Optional cluster publisher | Add a cluster manifest plus table-backed state/status store only if stronger all-or-nothing apply is required | Very high | Touches core atomicity and recovery invariants | +| Recovery | Add cluster sidecars and failpoint coverage for graph-move-before-state-publish gaps | Very high | Any missed sidecar is a correctness bug | +| Graph lifecycle | First-class graph resource create/delete/rename or stable-id story | High | Current server add/remove is intentionally not exposed | +| Schema apply integration | Make schema apply cluster-aware or wrap it with cluster recovery | High | Existing schema apply cannot be treated as cluster atomic by assertion | +| Query registry | Remove target-state exposure flag, add dependency metadata, keep `mcp.expose` bridge | Medium/high | Catalog behavior is observable public API | +| Policy | Add cluster plan/apply/admin actions and per-query list/invoke scope | High | Needs docs, tests, Cedar schema migration, and compatibility with coarse `invoke_query` | +| Server registry | Boot from cluster state, eventually reload/reconcile graph handles, expose statuses | High | Affects routing, OpenAPI, auth, and workload admission | +| API types/OpenAPI | Plan/status/apply DTOs if HTTP management endpoints ship | Medium/high | OpenAPI drift must be regenerated | +| UI specs | New renderer/spec validator/binding checker | High | New product surface, not currently implemented | +| Pipelines | New scheduler/runtime/connector/mapping/idempotency/run ledger | Very high | Second data-plane seam; large product and correctness surface | +| Embeddings | Cluster-level defaults, env refs, model/dimension validation, index interaction | Medium | Existing embedding code is mostly offline/client-side | +| Docs | User docs for cluster config, policy, server, CLI; dev docs for invariants/testing | High | Public contract changes | +| Tests | New cluster suites plus extensions to config/server/policy/recovery/schema/query tests | High | Needs boundary-matched coverage | + +## Reversibility And Approval Tiers + +| Tier | Examples | Gate | +|---|---|---| +| Display-only | Dashboard layout, non-breaking alias addition | No approval beyond policy | +| Catalog behavior | Add query, hide/list query via policy, add policy grant | Policy check; no data-loss approval | +| Compatibility narrowing | Future validated enum narrowing, query param removal, policy removal that revokes access | Explicit compatibility warning; may require human approval by policy | +| Recoverable definition rewrite | Soft schema drop, graph schema rename, index rebuild | Plan warning; no data-loss approval unless policy requires | +| Irreversible data loss | Graph delete, hard schema drop, cleanup-triggered prior-version reclamation, overwriting pipeline target | Human approval artifact recorded in audit ledger | + +Future enum narrowing belongs in `CompatibilityNarrowing` unless the migration +also drops/coerces data or triggers cleanup. That distinction matters for plan +wording and for policy predicates. + +## Rollout Phases + +<!-- Spec note: the only safe path is staged. The cluster control plane crosses +config, engine, server, policy, and data-plane-adjacent surfaces; a big-bang +replacement would make every invariant harder to audit. --> + +### Phase 0: Documentation And Parser Skeleton + +- Add cluster spec types and strict parser behind an unused feature/module. +- Implement `cluster validate --config <folder>` with no state backend. +- Validate file paths, env refs, duplicate resource keys, and dependency graph. +- No behavior change to `omnigraph.yaml`, server boot, or query exposure. + +### Phase 1: Read-Only Planning + +- Add `cluster plan` against a mock/imported state snapshot. +- Produce plan JSON and human output. +- Reuse existing schema migration planner for schema resources. +- Validate stored queries against desired schema. +- Compute downstream dependencies and blast radius. +- Still no apply. + +### Phase 2: State Backend And Lock + +- Add `state.backend: cluster` JSON storage and lock/CAS. +- Add external backend trait only if lock + CAS semantics are explicit. +- Add `cluster status`, `refresh`, and `import`. +- Persist `AppliedRevision`, `ResourceStatus`, and audit references in JSON. + +### Phase 3: Config-Only Apply + +- Apply query, policy, UI, alias, embedding, and pipeline definition resources + that do not move graph manifests. +- Publish by writing content-addressed resource payloads and CAS-updating + `state.json`. +- Keep server boot from `omnigraph.yaml`; cluster state is inspectable but not + yet serving traffic. + +### Phase 4: Graph And Schema Apply + +- Add graph create/delete as cluster resources. +- Make schema apply cluster-aware, with sidecar coverage for graph manifest + movements before JSON state publish. +- Gate irreversible data-loss operations with approval artifacts. +- Consider a cluster manifest publisher only if the JSON sidecar + repair path + is not strong enough for the accepted safety contract. + +### Phase 5: Server Reads Cluster Catalog + +- Allow server startup from cluster state. +- Add status and catalog endpoints as needed. +- Keep the current `omnigraph.yaml` startup path as compatibility mode. +- Regenerate OpenAPI for any HTTP surface. + +### Phase 6: Policy-Owned Query Exposure + +- Add per-query policy scope for list/invoke. +- Filter `GET /queries` by actor. +- Keep coarse `invoke_query` as a broad allow rule for compatibility until + docs and migrations say it can be narrowed. +- Deprecate and later remove `mcp.expose` from target-state cluster config. + +### Phase 7: Pipeline Runtime + +- Add scheduler/worker/runtime. +- Add source connector contracts, mapping validation, idempotency keys, + per-target run status, and retry behavior. +- Treat fan-out execution as data-plane writes unless explicitly staged through + branch/merge. + +## Test Ownership + +Tests must prove the Terraform-style workflow, not just individual parsers. +The minimum behavior contract: + +```text +validate catches bad config +plan is deterministic and complete +apply only applies a fresh accepted plan +state changes are locked and durable +drift and partial convergence are visible, not silent +``` + +| Change | Existing coverage to extend | New coverage likely needed | +|---|---|---| +| Cluster parser | `omnigraph-config` inline config tests for strictness/path resolution | `omnigraph-cluster` parser/dependency tests | +| Plan dependency graph | Schema planner tests, query registry tests | Golden plan JSON for cross-resource downstream impacts | +| State lock/backend | Existing schema apply lock tests as model | JSON state CAS/lock race tests | +| Optional cluster manifest publisher | `crates/omnigraph/src/db/manifest/tests.rs` | Cluster publisher CAS, expected-version, deterministic order tests if that backend ships | +| Cluster recovery | `recovery.rs`, `failpoints.rs` | Phase B -> state publish failpoints, external state CAS failure tests | +| Schema cluster apply | `schema_apply.rs`, failpoints schema apply cases | Nested graph/cluster recovery tests | +| Query exposure policy | `omnigraph-policy` invoke_query tests, server query catalog tests | Per-query list/invoke allow/deny and no-probing tests | +| Server cluster boot | `omnigraph-server/tests/server.rs`, `openapi.rs` | Boot from cluster state, registry reload/status tests | +| CLI cluster commands | `omnigraph-cli/tests/cli.rs`, `system_local.rs` | `cluster validate/plan/apply/status` system tests | +| Pipelines | None today | New runtime/mapping/idempotency/run-ledger suites | + +Workflow-specific tests: + +| Workflow area | Required assertions | +|---|---| +| Parser / validate | Unknown fields, wrong-kind typed addresses, missing providers, inline secret values, dangling graph/query/pipeline refs, and future-phase fields fail with typed diagnostics | +| Plan goldens | Given config + imported/fake state, plan JSON contains stable resource digests, dependency edges, state observations, proposed changes, blast radius, and approval gates in deterministic order | +| Fresh-plan apply | Changing config digest, state revision, resource digest, or observed graph manifest version after planning makes `cluster apply` reject and require re-plan/refresh | +| State lock / CAS | Concurrent applies against the same backend cannot both succeed; loser gets a typed lock/CAS conflict | +| Recovery / partial apply | Fail after graph/resource movement but before cluster state publish; assert recovery or status surfaces `ActualAppliedStatePending`/sidecar state and never returns success | +| Server/runtime phase | Before cluster state drives routing or registry reload, tests are hermetic: no real home dir, no real global config, no real credentials, no ignored remote tests | +| Pipeline phase | Fan-out run records per-target status, commit ids, retryability, and idempotency keys; no aggregate success unless every target succeeded | + +Hard gates: + +- Do not ship `cluster apply` until `cluster validate` and read-only + `cluster plan` have hermetic tests. +- Do not ship graph/schema-moving apply until failpoint recovery tests prove the + Phase B -> state publish gap is covered. + +For docs-only changes, `scripts/check-agents-md.sh` is enough. For +implementation phases, run the boundary tests above before widening to +`cargo test --workspace --locked`. + +## User-Visible Documentation Fallout + +The following public docs must change when the corresponding phase ships: + +| Phase | User docs | +|---|---| +| Parser/validate | New `docs/user/cluster-config.md`; CLI reference for `cluster validate` | +| Plan/apply | CLI reference, transactions, policy, errors | +| State backend | Storage, deployment, constants, maintenance | +| Server cluster boot | Server, deployment, OpenAPI | +| Policy query exposure | Policy, server, query language / stored-query registry docs | +| Pipelines | New pipeline user guide, deployment, audit, errors | +| Embeddings config | Embeddings, indexes | + +Do not ship a user-visible command, flag, env var, endpoint, or config key +without updating the corresponding user doc in the same PR. + +## Known High-Risk Design Decisions + +1. **Cluster root identity.** Decide whether `metadata.name` is a label or + identity. Prefer root-derived stable identity plus display name to avoid a + rename breaking resource identity. +2. **Graph storage derivation.** The high-level sample omits graph storage. + Implementation should derive graph roots under `ClusterRoot/graphs/<id>.omni` + by default and treat external graph roots as a separate, explicit feature. +3. **Nested apply.** Schema apply and graph lifecycle cannot move a graph + manifest outside cluster sidecar coverage. +4. **External state.** Must expose pending repair instead of returning success + when graph/resource movement succeeds and external state CAS fails. +5. **Per-query policy.** Catalog filtering must avoid probing leaks: callers + without list/invoke permission should not distinguish hidden from missing. +6. **Pipeline fan-out.** Do not promise atomic multi-graph ingestion unless the + runtime uses a real branch/merge or equivalent protocol for every target. +7. **Drift correction.** Reconciler-initiated deletes are the same data-loss + class as human-requested deletes. + +## Exit Criteria For A Real RFC + +Before implementation begins beyond parser/validate, the RFC must answer: + +1. Exact JSON state/status/approval/recovery schemas and object-store paths. +2. Exact sidecar JSON schema and recovery decision matrix. +3. State backend interface and supported lock/CAS implementations. +4. Cluster apply group syntax and dependency ordering rules. +5. Plan JSON schema, including blast-radius and approval fields. +6. Bootstrap authority and first-actor story. +7. Server startup and migration path from `omnigraph.yaml`. +8. Per-query policy schema and compatibility bridge for `mcp.expose`. +9. Pipeline runtime owner, status schema, and idempotency contract. diff --git a/docs/dev/cluster-config-specs.md b/docs/dev/cluster-config-specs.md new file mode 100644 index 0000000..8094be2 --- /dev/null +++ b/docs/dev/cluster-config-specs.md @@ -0,0 +1,415 @@ +# Cluster Config Spec — Declarative, As-Code, Agent-Operated + +**Status:** Draft / thinking-in-progress +**Type:** Architecture direction +**Date:** 2026-06-07 +**Relationship:** generalizes today's `omnigraph.yaml` graph/query/policy configuration surface ([CLI reference](../user/cli-reference.md), [server docs](../user/server.md)) into a future cluster control plane. The distilled rules are in [cluster-axioms.md](cluster-axioms.md); detailed downstream implementation spec and blast-radius assessment in [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md). This is a proposed architecture, not an implemented RFC. + +> **Revision 2026-06-07 — full commitment to the Terraform paradigm.** Three changes from the earlier draft: (1) **state is an authoritative, locked ledger in a backend** (server-hosted *or* a separate cloud store), not "a mostly-rebuildable projection"; (2) `plan` is framed as the **CLI diff between local config and state**; (3) **ETL pipelines** (external data sources) are a first-class config asset — a second seam, alongside schema, where a definition triggers a data-plane effect. The full set of config assets (incl. **aliases**, **embeddings**) is enumerated below. + +--- + +## The problem (the Sarah/Bob test) + +Two operators, Sarah and Bob, administer the same OmniGraph deployment. Sarah adds new queries, changes a schema, adds a dashboard, updates policies, and wires in a new data feed. + +**How does Bob find out?** + +Today he can't — not cleanly. Sarah's changes land in many different places via many different mechanisms: + +- schema → the schema-apply path, accepted state in `_schema.pg`, `_schema.ir.json`, `__schema_state.json`, and table versions in the graph manifest +- queries → `.gq` files passed per request or resolved through CLI query roots / aliases; not durable cluster state +- policies → `policy.file` in `omnigraph.yaml`, pointing at Cedar/YAML files that are usually GitOps'd externally +- aliases → CLI sugar in each operator's `omnigraph.yaml` +- external data → ad-hoc `load`/`ingest` scripts, cron jobs, glue code that lives nowhere durable +- UI → undefined + +There is no single diff that spans them, no single change record attributed to Sarah, no one place Bob (or Bob's agent) reads to answer "what is this deployment, and what changed?" The state is **fragmented**, and fragmentation is hostile to the one thing an agent must do: reason over the system *as a whole*. + +A design passes only if it answers the Sarah/Bob test directly. + +--- + +## Thesis + +The unit of declarative state is the **cluster** (the deployment), described by **a single config, as code, in version control**, operated by an **agent** through a plan/apply/reconcile loop against an authoritative state ledger. + +Every surface is a declarative as-code artifact — schema (`.pg`), queries (`.gq`), policies (`.yaml`), UI (`.yaml`), aliases, **ETL pipelines**, and embeddings config. The UI is not a separately-deployed application; it is a declarative spec, a first-class resource reconciled exactly like the others. + +Three pillars, none optional: + +1. **DECLARATIVE** — you describe the desired end state, not the steps. The reconciler computes the steps. +2. **AS CODE** — the config is declarative text in a repo, version-controlled. This is the **source of truth for *intent***. +3. **OPERATED BY AGENT** — an agent authors config changes and drives reconciliation as an authenticated actor, with policy and approval gates. No human state-management burden. + +This is **Terraform's model, taken literally**: config (as code) is desired truth; **state is an authoritative, locked ledger** of what has been applied — held in a backend (the cluster, or a separate cloud store); `plan` diffs config against state; `apply` converges reality to config and updates state — applied at **cluster** scope, with OmniGraph as its own data-aware provider and an agent as the controller. + +--- + +## Why as-code (the recursion argument) + +"As code" is not branding. It is the structural property that makes a self-describing system well-founded. + +Consider the rejected alternative: model the cluster's definition *as a graph* (a meta-graph whose nodes are graphs/policies/queries/UI). To describe a graph you need a schema. The meta-graph's schema is either: + +- **hardcoded** → the base case is *code* (you smuggled code in at the bottom anyway), or +- **another graph** → infinite regress, no base case. + +Graph-describing-graph never terminates. **Code is the base case.** A declarative config needs no meta-describer because it is parsed by the engine's compiled code — not described by more user-space data. + +> **Declarative-as-code terminates. Declarative-as-data (a graph of graphs) recurses.** + +This is also why **config** must live **outside** the running system: reviewable (PRs), reproducible (clone + apply), diffable as text, and editable by an agent — without depending on the running system to describe its own intent. + +Corollary on direction: change flows **code → cluster, never the reverse.** You do not edit the running system and call that intent. (State, separately, *records* what the cluster currently is — see the next section — but it is never where you express what it *should* be.) + +--- + +## Why per-cluster, not per-graph + +The definition Sarah changed does not *belong* to any single graph: + +1. **Policies cross-cut graphs.** "Member can't delete on any graph," "who may list/create/delete graphs" — cluster facts. No graph could own them. +2. **"Which graphs exist" has no home in a per-graph model.** The set of graphs is state *above* any graph. +3. **Queries, UI, pipelines, and aliases span graphs.** The MCP/tool catalog an agent discovers is the *cluster's* surface; a dashboard renders multiple graphs; a pipeline may fan out into several. +4. **Cross-graph apply groups.** Sarah may add a graph *and* wire it into the UI *and* grant policy access *and* attach a feed as one logical change — only the cluster can express, plan, and eventually fence that as one apply group. +5. **Operators operate clusters.** Bob is Sarah's peer on a *deployment*, not a graph. The collaboration unit is the cluster. + +The graph is a *resource within* the cluster, not the unit of operation. + +The mirror question — *why not per-fleet?* — is the same one this section used against per-graph, one level up. A fleet of clusters may eventually want its own declarative spec describing which clusters exist. That recursion is real but **out of scope here**: this proposal stops at the cluster because the cluster is the unit two operators collaborate over. Fleet is the next scope up, named and deferred, not denied. + +--- + +## The model: config / state / reconcile (the Terraform model, literally) + +| Layer | What it is | Source of truth for… | Who manages it | +|---|---|---|---| +| **Config** (as code, a folder of files) | Desired state of the whole cluster — graphs, schemas, policies, queries, UI, bindings, aliases, embeddings, ETL pipelines | **Intent** ("what it should be") | Operators/agents, in version control | +| **State** (a locked ledger in a backend) | The authoritative record of what has been applied — applied revision, per-resource fingerprints, observed graph/table versions, audit-record references, resource conditions | **Deployed reality** ("what is") | The reconciler; humans don't hand-edit it | +| **Actual cluster** | The realized *definition* of the running graphs — schema/policies/queries/UI/pipelines as actually in force | — (reality itself) | The engine; `apply` converges it to config | + +**`plan`** = `diff(config, state)` → proposed change set (optionally refreshed against the actual cluster). +**`apply`** = acquire the state lock → converge actual → config → **update state** → release lock. Apply does **not** acknowledge success until the state update succeeds; if actual moved but the state write failed, the next `plan` / `refresh` must surface the non-success state and repair or import it before more work proceeds. + +### State is an authoritative, locked ledger — not a throwaway projection + +This is the 2026-06-07 revision. State is treated exactly as Terraform treats `tfstate`: + +- **Authoritative.** State is the trusted record of what is deployed. `plan` diffs config against **state** (fast, deterministic), not against a full live scan of the cluster on every command. "What exists" is answered from state. +- **In a backend.** State lives in a configurable backend: the **cluster's own object-store backend**, or a **separate cloud store** (e.g. a different bucket/account) — the operator's choice, mirroring Terraform's local/S3/remote backends. The config declares which. +- **JSON first.** The baseline state format is Terraform-style JSON documents (`state.json` plus status/approval/recovery JSON records) protected by backend lock/CAS. Lance control-plane datasets are a possible later backend only if row-level history, queryability, or tighter publish fencing justifies the added machinery. +- **Atomicity depends on backend and publish scope.** A JSON state backend, even when stored under the cluster root, is a separate CAS step from graph Lance manifest moves. If actual resources move but the state write fails, apply must surface `ActualAppliedStatePending` (or equivalent) and require refresh/import repair instead of pretending one atomic commit covered every object. A future Lance-backed state backend or cluster manifest publisher may tighten this, but that is not the Phase-1 assumption. +- **Locked.** `plan`/`apply` acquire a **state lock** before touching state, so two operators (or two agents) cannot converge concurrently and corrupt the ledger. This generalizes the existing `__schema_apply_lock__` from schema scope to cluster scope. +- **Reconstructable, but not casually rebuilt.** OmniGraph's edge over opaque-cloud Terraform: the running cluster is self-describing (manifests, commit logs), so a lost state ledger can be **imported / refreshed** from the live cluster. That is a *resilience* property — not licence to treat state as disposable. State is protected and backed up like any source of truth. +- **One slice is never reconstructable.** Who *approved* an irreversible apply cannot be re-derived from a manifest scan. That approval/audit record lives in the **durable audit ledger** (baseline: append-only JSON records in the state backend; future: a Lance table only if needed). State *references* it by id; it never *is* it. + +**The control plane reconciles definition, not data.** The reconcile loop converges the cluster's *definition* — schema, policies, queries, UI, bindings, aliases, pipelines, and the set of graphs. It does **not** converge **data**: rows, edges, and vectors are data-plane content, mutated by `load`/`mutate` and by **pipeline execution**, versioned by the commit DAG, and they sit entirely outside the reconcile loop. (`load`/`mutate` never appear in `cluster.yaml`.) **Two** definition kinds *trigger* a data-plane effect without owning data — schema and ETL pipelines (see "ETL pipelines" below). + +### Cluster resource model + +Minimum vocabulary: + +- **ClusterRoot** — the object-store prefix / control namespace for one deployment. +- **DesiredRevision** — git commit, `cluster.yaml` digest, and per-resource digests. +- **ResourceKind** — `Graph`, `Schema`, `Query`, `PolicyBundle`, `UiSpec`, `Binding`, `Alias`, `EmbeddingConfig`, **`Pipeline`** (ETL), and future cluster-scoped resources. +- **ResourceAddress** — normalized typed references between resources, such as `graph.knowledge`, `query.knowledge.find_experts`, `policy.base_rbac`, and `pipeline.github_sync`; illustrative YAML may use shorthand, but plan/state store the typed form. +- **ProviderAddress** — typed references to provider instances, such as `provider.storage.prod_graphs`, `provider.source.github_org`, and `provider.embedding.default`; provider addresses keep storage, external sources, and embedding providers from being inferred from ambiguous strings. +- **StateBackend** — where the JSON state ledger is stored: `cluster` (this deployment's own backend) or an external store (a separate bucket/account). +- **StateLock** — the cluster-scope lock acquired before plan/apply. +- **AppliedRevision** — the durable, locked record (the heart of state) of which desired revision is applied, with audit-record references, resource fingerprints, and graph/table version observations. +- **ResourceStatus** — `Pending | Planned | Applying | Applied | Drifted | Blocked | Error`, with typed conditions and observed actual state. +- **ApplyGroup** — the explicit atomicity unit. Default is one independent resource per group; cross-resource references force planner-derived groups, and user-declared groups may opt into larger atomicity only for resources the active backend protocol can fence or repair. Baseline JSON state supports small, explicit groups; larger all-or-nothing groups require a future cluster publisher or equivalent proof. + +--- + +## State: backend, lock, and the config ↔ state diff + +The CLI is the operator's window onto the gap between config and state. + +The Terraform-aligned workflow is: + +```text +cluster validate # parse + schema-check desired config, no state mutation +cluster plan # diff desired config against state, with optional refresh +cluster apply # apply an accepted fresh plan and update state +cluster status # read what state says is deployed now +cluster refresh # update/import state observations from actual cluster state +``` + +`plan` is the central artifact. It records the desired revision, resource +digests for every referenced file, dependency edges between resources, observed +state fingerprints / graph manifest versions, proposed changes, and approval +gates. The human output below is a rendering of that structured plan, not the +only representation. + +``` + $ omnigraph cluster plan + config ./ → diff against state (backend: cluster · lock: acquired) + + ~ schema knowledge hard-drop Person.legacy_id ⚠ prior versions reclaimed — needs approval + + query knowledge.find_experts (new stored query) + - query knowledge.orphan_pages (removed) + ~ policy base_rbac grant invoke find_experts → members (this is what EXPOSES the new query) + + pipeline saas_sync notion → knowledge, hourly + ~ ui dashboards.overview add panel "experts" + + alias experts + ───────────────────────────────────────────────────────────────────── + 6 changes · 1 requires approval (hard schema drop on knowledge) · run `apply` to converge +``` + +<!-- Audit fix: enum narrowing is not implemented today; hard drops are the +current supported irreversible schema path, so the example must not teach a +future migration tier as if it already exists. --> +That output **is** the answer to the Sarah/Bob test: one diff, spanning every surface, attributed to a git commit and concrete resource digests, with data-impact peeked (axiom-6 schema seam), dependency fallout visible, observed state compared, and approval gates surfaced *before* anything moves. Drift (someone poked the live cluster out-of-band) shows up here too — `plan` reconciles state against the actual cluster and flags resources whose observed version no longer matches the ledger. + +<!-- Audit fix: JSON state is the baseline. It is inspectable and Terraform-like, +but it remains a separate CAS step from graph manifest movement. --> +`apply` then: acquire **state lock** → execute the change set (ordered/grouped per the planner) → **CAS-update the JSON state ledger** with the new applied revision/status observations → release the lock. For config-only resources, content-addressed payload writes can happen before the state CAS because state is the publish point. For graph/schema moves, the graph manifest may move before the state CAS; a crash or CAS failure there leaves a loud repair/import condition and no success acknowledgement, not a silently successful atomic apply. A future cluster manifest publisher can tighten this gap, but the baseline protocol does not assume it. + +--- + +## ETL pipelines (the second data-plane seam) + +External data — from another database, an API, a file drop, a stream — is a first-class config asset, not glue code that lives nowhere. + +A **Pipeline** is declared in config: a **source** (e.g. `notion`, `github`, `slack`, `gdrive`, `postgres`, `http`, `s3-files`, `kafka`), an optional **schedule/trigger**, and **one or more target graphs**, each with its own **mapping/transform** (external records → graph types & properties). A single feed can **fan out across graphs** — e.g. a GitHub sync that populates both the `engineering` graph and the people/teams in `knowledge`. It is reconciled like any resource — `apply` creates / updates / deletes / (re)schedules the pipeline *definition*. This is the canonical "company brain" move: the deployment's graphs are continuously assembled from the SaaS tools the org already uses. + +The crucial boundary (axiom 6, axiom 13): the pipeline **definition** is control-plane and reconciled; the pipeline's **execution** — actually pulling rows and writing them — is a **data-plane effect** that produces ordinary `load`/`mutate` commits *outside* the reconcile loop. The reconciler converges the pipeline; the rows it ingests are never reconciled state (just as a cron *definition* is config but its output is not). This makes ETL the **second seam** where a definition triggers a data-plane effect — schema being the first (a migration conforms existing rows; ETL ingests new ones). + +Consequences that fall out of the existing model: + +- **`plan` previews the pipeline, not the data.** "pipeline `saas_sync`: notion → `knowledge`, hourly" is a definition diff; it does not scan the source (data-volume-independent), the same way schema `plan` previews impact only at the bounded, opt-in data peek. +- **Source credentials come from the `.env` file** (axiom 10): `token: ${NOTION_TOKEN}` — resolved from the gitignored `.env` file per deployment, never inline. +- **Reversibility gradient applies** (axiom 8): a pipeline that *appends* is reversible-ish; one configured to *overwrite* a target is a data-loss path and hits the irreversible-op gate. +- **Referential integrity is plan-time** (axiom 9): a pipeline whose `into:` names a graph/type the same revision removes is a fail-closed `plan` error. +- **Fan-out is statusful, not magically atomic.** A pipeline execution that writes to several graphs is a set of ordinary per-target graph writes unless the pipeline explicitly stages through a branch/merge protocol that can fence those targets. A failed run may therefore leave `engineering=Applied`, `knowledge=Error` (for example), and the pipeline run ledger must expose per-target status, commit ids, retryability, and idempotency keys. Control-plane `apply` only converges the definition/schedule; it never means every future data-plane target has ingested successfully. + +--- + +## Config assets — the full set + +Everything below is **shared cluster config** (in the folder, version-controlled, secret-free) unless marked per-operator. The rule of thumb: if two operators must agree on it, it's config; if it's how *you personally* reach or view the cluster, it's per-operator. + +| Asset | In config? | Notes | +|---|---|---| +| **Graphs** (the set that exists) | ✅ config | the named graphs; their existence is cluster state | +| **Schema** (`.pg`, **one per graph**) | ✅ config | also encodes indexes (`@index`/`@unique`/vector), constraints, and search (`@embed`) — so indexes & search are reconciled *via* schema | +| **Stored queries** (`.gq`, **per graph**) | ✅ config | a `.gq` file declares **many** named queries; the registry declares which exist (name → file, key must match the `query <name>` symbol). **Target design:** exposure — who may list/invoke each — is a policy decision, not a registry flag. **Current compatibility bridge:** shipped `omnigraph.yaml` still has `queries.<name>.mcp.expose`, and the HTTP catalog is not Cedar-filtered per query yet. Aliases & bindings reference a query by name | +| **Policy bundles** (`.yaml`) | ✅ config | YAML (not Cedar files); **shared across graphs** via `applies_to: [cluster \| <graph refs>]` (many-to-many; fix 2026-06-08 unified the old `scope:`/`graphs:` split). Gates actions **and query exposure** (who may list/invoke each stored query) | +| **UI specs / dashboards** (`.yaml`) | ✅ config | first-class resources; a dashboard **reads from several graphs** (`graphs: [...]`) | +| **Bindings** | ✅ config | wiring between resources (query ⇄ UI surface) | +| **Aliases** | ✅ config* | CLI shortcut to a stored query: `{ command, query: <.gq file>, name: <symbol>, args, format }` — `query` is the **file**, `name` the **query symbol** in it. See note | +| **Embeddings config** | ✅ config | model + dimension + which fields embed; the **API key comes from the `.env` file** (`${…}`) | +| **ETL pipelines** | ✅ config | source → transform → **one or more target graphs**; source credentials come from the `.env` file | +| **Apply settings** | ✅ config | `apply.default_grain`, grouping/ordering hints | +| **State backend + lock** | ✅ config | where the ledger lives, whether to lock | +| **Secrets (`.env` file)** | ✅ ref'd by config; values **gitignored** | a separate `.env` of secret values, referenced as `${NAME}`; never committed (OmniGraph's standard env-file convention) | +| **Connection** (which cluster URI) | ❌ per-operator | how *you* reach the cluster | +| **Operator token** | ❌ per-operator (secret) | each operator's own credential to reach the cluster | +| **CLI prefs** (output format, table layout, active graph/branch selection) | ❌ per-operator | personal ergonomics, not shared truth | + +\* **Aliases — the one with a split.** A shared alias that names a cluster resource (a stored query, a dashboard) is config — it's a vocabulary the whole team relies on, and it belongs in the spec (often it *is* just the stored-query catalog entry, since that already carries name + params + tool metadata). A *purely personal* shortcut (your own command abbreviations) stays in the per-operator layer. When in doubt: if it should survive `git clone` and be the same for Bob as for Sarah, it's config. + +--- + +## The synthesis (beyond vanilla Terraform) + +Embracing Terraform does not mean stopping at Terraform. Three extensions make this specifically right for OmniGraph and the agentic future: + +1. **OmniGraph is its own data-aware provider, and `plan` can peek across the data boundary.** A Terraform provider CRUDs resources blind to your data. Here, the control-plane resource is the schema **definition** (declarative, reconciled); converging it *triggers* a data-plane **effect** — currently soft/hard drops, rewrites, and index creation, with future validated migrations such as enum narrowing or `String`→`enum` conversion once the planner grows that tier. The leverage is that `plan`, before applying the definition change, can *peek* at bounded data-plane consequence and report it — **"hard-dropping this property requires approval and will make prior versions unreachable after cleanup"** or, in the future, **"narrowing this enum will fail on 37 rows"** — which Terraform structurally cannot do. This is deliberate and bounded: a data peek makes that `plan` cost scale with data volume, so it is **opt-in / bounded** (sampled or skippable for large tables), and it never makes the control plane the owner of data. Schema and ETL pipelines are the **two** seams where the control plane reaches into the data plane; everywhere else `plan` is data-volume-independent. + +2. **JSON state first, explicit partials, optional stronger fencing later.** Terraform apply is not transactional — partial applies are a real failure mode. Lance commits are per dataset, and today's OmniGraph manifest atomicity is graph-scoped: one graph commit flips the relevant sub-table versions together, protected by expected table versions and recovery sidecars. The first cluster-control backend should match Terraform's shape: a locked JSON state document plus append-only JSON status/approval/recovery records. That keeps Phase 1 inspectable and narrow. Cluster-level all-or-nothing apply is a later capability only if we add a **cluster manifest publisher** or Lance-backed state backend that fences graph *version pins*, query catalogs, policy bundles, UI specs, pipeline definitions, recovery sidecars, and state as one commit protocol. Until that exists, apply must surface partial convergence as `ResourceStatus`, not pretend it was atomic. + +3. **Agent-as-controller fuses Terraform with Kubernetes.** Terraform contributes the as-code config (truth outside the system, recursion-terminating) and the locked state ledger. Kubernetes contributes *continuous* reconciliation (controllers watch, not apply-on-demand). The agent is both author and controller: it reads a config change, runs the data-aware plan, evaluates blast radius against the reversibility gradient, **auto-applies the reversible parts only when policy permits, and escalates irreversible / data-loss gates to a human approval artifact recorded in the audit ledger and referenced by state.** + +> Terraform's as-code config + locked state × Kubernetes' continuous reconciliation × the agent as the controller that bridges them — on OmniGraph's data-aware, atomic substrate. + +--- + +## Concrete shape (illustrative) + +The config is **a set of files in one folder** (flat, Terraform-style — the extension carries the type): + +``` + company-brain/ + ├── cluster.yaml # the spec (graphs, policies, ui, bindings, aliases, pipelines, state, vars ref) + ├── .env # SECRET VALUES — gitignored, never committed + ├── knowledge.pg · engineering.pg # schemas (one per graph) (.pg) + ├── knowledge.gq · engineering.gq # query files — each holds MANY queries (.gq) + ├── cluster_admin.policy.yaml · base_rbac.policy.yaml · knowledge_pii.policy.yaml # shared policy bundles + ├── overview.dashboard.yaml # cross-graph UI spec (.dashboard.yaml) + └── notion_to_knowledge.map.yaml · github_to_engineering.map.yaml · github_to_people.map.yaml # pipeline maps +``` + +Secrets live in a gitignored `.env` file (OmniGraph's standard env-file convention); the config references them as `${NAME}`: + +```bash +# .env — secret values; gitignored; never committed. Referenced in cluster.yaml as ${NAME}. +NOTION_TOKEN=… +GITHUB_TOKEN=… +EMBEDDING_API_KEY=… +``` + +Resource relationships (so the wiring is unambiguous): + +``` + cluster ──has many──► graph ──has one──► schema + └────has──► query file(s) (.gq) ──each declares MANY──► query <name> { … } symbols + registry entry key = the query <name> symbol ──points to──► its .gq file (queries: { <name>: { file } }) + (registry says a query EXISTS; it carries NO expose flag) + policy bundle ──applies to──► { cluster | one or MANY graphs } (SHARED, many-to-many) + └──governs query EXPOSURE──► who may LIST / INVOKE each stored query (no `expose:` in the registry) + alias (command, query = .gq FILE, name = symbol, args, format) ──selects one query from that file + binding names a query by registry name (graph.queryName) ──► resolved to (file, symbol) + dashboard ──reads from──► one or MANY graphs + pipeline ──writes into──► one or MANY graphs + secrets ──live in──► a separate gitignored `.env` file; config uses ${NAME} +``` + +```yaml +# cluster.yaml — desired state of the whole deployment (config = source of truth for INTENT) +version: 1 +metadata: + name: company-brain + +state: # the authoritative ledger's backend (Terraform-style) + backend: cluster # "cluster" = this deployment's own store; or s3://… (a separate store) + lock: true # acquire a state lock before plan/apply + +env_file: ./.env # secret VALUES live in a gitignored .env file; referenced below as ${NAME} + +apply: + default_grain: resource # references may force groups; explicit groups request more atomicity + +graphs: # the cluster's graphs — each is ONE schema + a set of named queries + knowledge: # people · teams · docs · decisions · projects + schema: ./knowledge.pg # desired schema; reconciler runs (and plan previews) the migration + queries: # the graph's stored (named) queries; KEY must match a `query <name>` in the file + find_experts: { file: ./knowledge.gq } # ─┐ `query find_experts` and `query related_docs` + related_docs: { file: ./knowledge.gq } # ─┘ both live in knowledge.gq. Who may LIST/INVOKE → policy (not here) + engineering: # repos · services · incidents · PRs + schema: ./engineering.pg + queries: + service_owners: { file: ./engineering.gq } + open_incidents: { file: ./engineering.gq } + +policies: # policy BUNDLES (YAML) — SHARED across graphs (many-to-many). + # Policy ALSO governs query EXPOSURE: who may list/invoke each stored query. + # Fix (2026-06-08): unified the binding field on `applies_to:` (was a + # `scope:` + `graphs:` split) — one field, takes `cluster` or graph refs; + # bare graph names are shorthand for `graph.<id>` (see impl-spec typed addresses). + cluster_admin: # cluster-scoped: graph_list, create/delete, management + file: ./cluster_admin.policy.yaml + applies_to: [cluster] + base_rbac: # read/write + which roles may invoke which queries, across both graphs + file: ./base_rbac.policy.yaml + applies_to: [knowledge, engineering] + knowledge_pii: # an extra bundle, only for knowledge + file: ./knowledge_pii.policy.yaml + applies_to: [knowledge] + +pipelines: # ETL — ONE pipeline may write into SEVERAL graphs (definition only) + saas_sync: # the "company brain" move: assemble graphs from the SaaS tools + source: { kind: notion, token: ${NOTION_TOKEN} } # secret via ${NAME}, never inline + schedule: "0 * * * *" # hourly; execution is a data-plane effect, not reconciled state + into: # fans out across graphs + - { graph: knowledge, map: ./notion_to_knowledge.map.yaml } + github_sync: + source: { kind: github, token: ${GITHUB_TOKEN} } + schedule: "*/15 * * * *" + into: + - { graph: engineering, map: ./github_to_engineering.map.yaml } + - { graph: knowledge, map: ./github_to_people.map.yaml } # same feed enriches a SECOND graph + +embeddings: # semantic search over docs/decisions; key via the `.env` file + model: gemini-embedding-2 + dimension: 3072 + api_key: ${EMBEDDING_API_KEY} + +ui: # dashboards read from SEVERAL graphs + dashboards: + overview: + file: ./overview.dashboard.yaml + graphs: [knowledge, engineering] # cross-graph + +aliases: # CLI shortcuts. ⚠ an alias's `query:` is the .gq FILE PATH; + # `name:` selects the query SYMBOL inside it (a file declares many). + experts: { command: query, graph: knowledge, query: ./knowledge.gq, name: find_experts, args: [topic], format: table } + incidents: { command: query, graph: engineering, query: ./engineering.gq, name: open_incidents, format: table } + +bindings: # wiring between resources + - query: knowledge.find_experts + surface: ui.dashboards.overview +``` + +<!-- Audit fix: the sample shows the target policy-owned exposure model. The +current server still uses mcp.expose for catalog membership until per-query +policy filtering lands. --> +What this is *not*: it is **not** a graph, and it carries **no credentials** — only secret *references* (`${…}`). It is parsed by the engine (the base case), describes the desired cluster, and is the thing two operators diff and review. + +The **state ledger** lives in the configured backend (the cluster, or a separate cloud store), versioned, CAS-updated, schema-versioned, locked during apply, agent-managed — the authoritative record of what is deployed. The baseline backend is JSON, so even cluster-hosted state is published through a state CAS and repaired explicitly if graph/resource movement happened first. A future cluster publisher can tighten that boundary, but it is not assumed by the high-level spec. + +--- + +## Boundaries that hold (orthogonal correctness, not Terraform-bias) + +1. **Secrets live in a `.env` file, never inline in config.** The committed config is what the cluster *is* (shared, reviewable, as code) and carries **no secret values** — only `${NAME}` references. The values (embedding API key, pipeline source credentials, per-deployment settings) live in a separate **`.env` file** — which is **gitignored and never committed**, and supplied per deployment. Separately, an *operator's own token* (how they personally reach the cluster) belongs to the per-operator connection layer, not the cluster config or its `.env` file. + +2. **The reversibility gradient gates apply — including drift correction.** Dropping a graph, hard-dropping schema data, or an overwriting pipeline is irreversible data loss; a future validated enum narrowing is a compatibility-narrowing migration unless it also drops or coerces stored values; recoloring a dashboard is not. Unified config, unified plan — but **tiered gates inside apply**, keyed to physics, not to who operates it. The gate applies to **drift correction too**: converging actual→config can mean *dropping* something added out-of-band — a data-loss path that hits the same gate. A reconciler "just fixing drift" is never an exception. + +3. **Agents are actors, not ambient authority.** The reconciler runs with a resolved actor or service account, subject to Cedar policy. If it applies on behalf of a human, the durable audit ledger carries both the controller actor and the approving human / approval artifact, and state references that ledger entry. Client-supplied actor identity is never trusted. + +4. **Status is explicit when apply is not atomic.** A unified plan does not imply a unified commit. If an apply group partially converges, the cluster must expose `ResourceStatus` and typed conditions until reconciliation finishes or rolls back. Silent partial success is forbidden. + +5. **State integrity is protected.** State is locked during apply and stored durably in its backend. The baseline state backend is JSON plus lock/CAS, so state update failures surface a repair/import condition before success is acknowledged. A lost ledger is recoverable (import/refresh from the self-describing cluster), but state is never treated as disposable. + +--- + +## Relationship to current config + +This is not green field, but it is also not today's `omnigraph.yaml`. The current file is a shared convenience for CLI and server startup: named graph targets, server defaults, query roots, aliases, embeddings model, auth env-file lookup, and `policy.file`. It is **not** the cluster's source of truth, it has no separate state ledger, and parts of it are intentionally per-operator. + +This proposal: + +- **splits** per-operator connection/credential/preference config from shared cluster config, +- **adds** `cluster.yaml` + a flat config folder as the full declarative cluster config (graphs, schemas, query catalog, policy bundles, UI specs, bindings, **aliases**, **embeddings**, **ETL pipelines**), +- **adds** the **JSON state ledger** (authoritative, locked, in a backend) and the `cluster plan`/`apply` loop, +- **adds** the reconciler (with OmniGraph as its own data-aware provider), while treating a cluster manifest publisher as a later option rather than the baseline, +- **lets an agent drive** plan/apply/continuous-reconcile. + +The connection/credential/preference layer remains per operator: it points at a cluster, resolves that operator's identity, and holds personal ergonomics. The cluster config stays shared, secret-free, and reviewable; the state ledger stays authoritative and locked. + +Implementation gate: the Terraform-style workflow must be testable in order. +`cluster validate` must catch bad config before any apply path exists; +read-only `cluster plan` must have deterministic structured-plan tests before +state mutation ships; and graph/schema-moving apply must have recovery tests for +the gap between graph/resource movement and JSON state publish. Otherwise the +control plane can look declarative while still hiding drift or partial success. + +--- + +## Open questions + +1. **Cluster state layout.** What exact JSON documents / object-store paths hold `AppliedRevision`, `ResourceStatus`, approval records, recovery records, sidecars, and resource content for query/policy/UI/pipeline specs? What evidence would justify a future Lance-backed state backend? +2. **State backend options.** Beyond "cluster" and "a separate bucket," what backends are first-class (a different account, a remote control service)? How is the backend itself bootstrapped and its lock implemented (object-store CAS vs an external lock service)? +3. **State import / refresh.** The exact actual-state scan that reconstructs a conservative `AppliedRevision` when the ledger is lost, and which fields become `Unknown`. +4. **Apply grain syntax.** Apply defaults to per-resource `ApplyGroup`; cross-resource references force planner-derived groups; user-declared groups opt into more atomicity. What's the YAML, and which combinations can the publisher actually fence? +5. **Pipeline runtime.** Where do pipelines *execute* (in the server? a worker? an external scheduler?), how are runs observed in `ResourceStatus`, and how does a failed/partial run reconcile vs. retry? +6. **Continuous reconciliation trigger.** Watch-and-converge (k8s-style) vs. apply-on-config-change. The agent-as-controller model leans toward continuous. +7. **Tenant partitioning (cloud).** A cluster may host multiple tenants; config/state is then tenant-partitioned, consistent with the reserved `GraphKey { tenant_id, graph_id }`. Tenant resolved from the token, never the config. +8. **Bootstrap — config, state, *and* authority.** How a cluster comes into existence from an initial config (`init` seeds; cluster owns; git mirrors for CI/DR), the first state write, and the chicken-and-egg of the very first apply (which needs an actor before any cluster exists to resolve policy against — so the bootstrap actor is necessarily out-of-band and privileged). Security-sensitive; needs an explicit story. +9. **Alias scoping.** Where exactly the shared/personal alias line falls, and whether shared aliases are just stored-query catalog entries. +10. **UI render and safety model.** Generic engine-side renderer vs. thin client, allowed components, query-binding validation, policy propagation, sandboxing, version compatibility. +11. **Cluster identity vs. `metadata.name`.** Is `metadata.name` a label or stable identity? If identity, renaming loses it — the stable-ID-across-rename gap already in `invariants.md`. Decide whether identity keys on `name` or on `ClusterRoot`, and reuse the existing known-gap framing. +12. **Resource dependency ordering.** Explicit dependency DAG (Terraform) vs. eventual convergence with retries (k8s). The most consequential unmade fork: it decides whether `plan` can promise an apply *order* before any data moves. +13. **Query exposure in policy (supersedes `mcp.expose`).** *Today* the stored-query registry carries a per-query `mcp.expose` flag and invocation is gated with the coarse `invoke_query` Cedar action — with **per-query authorization a documented gap** (the catalog isn't Cedar-filtered per query yet). This design **folds exposure fully into policy and drops the flag**: a stored query's visibility (catalog membership) and invocability are both policy decisions, so the catalog `GET /queries` returns each actor's policy-permitted set. The open work is the exact policy predicates for *list* vs *invoke* per query, and retiring `mcp.expose`. + +--- + +## Prior art + +- **Terraform** — declarative infra *as code*; config is desired truth, **state is an authoritative ledger in a backend**, **state locking** serializes applies, `plan` diffs config↔state, providers do the CRUD. The core model adopted here, taken literally. +- **Kubernetes** — one cluster store, many resource types under one API; controllers reconcile continuously; cluster-level RBAC. The continuous-reconciliation half of the synthesis. +- **dbt / Airflow / Dagster** — declarative, as-code data pipelines with lineage. Prior art for the **ETL-pipeline-as-config** asset (the second data-plane seam). +- **OmniGraph's own schema-apply** — already a faithful plan/apply/state/drift loop for the `schema` resource type, with `__schema_apply_lock__` as the lock seed; the reconciler this generalizes. diff --git a/docs/dev/index.md b/docs/dev/index.md index 1e41342..49b6d76 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -73,6 +73,7 @@ Working documents for in-flight feature work. Removed when the work lands. | Inline + stored queries, request/response envelope, MCP (MR-656 / MR-976 / MR-969) | [rfc-001-queries-envelope-mcp.md](rfc-001-queries-envelope-mcp.md) | | Config & CLI architecture — layered config, client targeting, file naming (MR-973 / MR-974 / MR-981) | [rfc-002-config-cli-architecture.md](rfc-002-config-cli-architecture.md) | | MCP server surface — full tool parity, stored queries, modular auth (MR-969 / MR-956 / MR-974) | [rfc-003-mcp-server-surface.md](rfc-003-mcp-server-surface.md) | +| Future cluster control plane — declarative as-code config, JSON state ledger, reconciler | [cluster-config-specs.md](cluster-config-specs.md), [cluster-axioms.md](cluster-axioms.md), [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md) | ## Boundary From 043b02e6179629fab79b68923c1ddd3bae401138 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Mon, 8 Jun 2026 20:07:39 +0300 Subject: [PATCH 024/165] feat(cluster): add read-only validate and plan --- AGENTS.md | 2 +- Cargo.lock | 14 + Cargo.toml | 1 + crates/omnigraph-cli/Cargo.toml | 1 + crates/omnigraph-cli/src/main.rs | 140 ++- crates/omnigraph-cli/tests/cli.rs | 230 ++++- crates/omnigraph-cluster/Cargo.toml | 20 + crates/omnigraph-cluster/src/lib.rs | 1275 +++++++++++++++++++++++++++ docs/dev/testing.md | 1 + docs/user/cli-reference.md | 17 +- docs/user/cluster-config.md | 95 ++ docs/user/index.md | 1 + 12 files changed, 1764 insertions(+), 33 deletions(-) create mode 100644 crates/omnigraph-cluster/Cargo.toml create mode 100644 crates/omnigraph-cluster/src/lib.rs create mode 100644 docs/user/cluster-config.md diff --git a/AGENTS.md b/AGENTS.md index b876749..26172ff 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,7 +17,7 @@ Tools that support `@`-imports (Claude Code) auto-include all three files via th `CLAUDE.md` is a symlink to this file — there is exactly one source of truth. Edit `AGENTS.md`. **Version surveyed:** 0.6.1 -**Workspace crates:** `omnigraph-compiler`, `omnigraph` (engine), `omnigraph-policy`, `omnigraph-cli`, `omnigraph-server` +**Workspace crates:** `omnigraph-compiler`, `omnigraph` (engine), `omnigraph-policy`, `omnigraph-cluster`, `omnigraph-cli`, `omnigraph-server` **Storage substrate:** Lance 6.x (columnar, versioned, branchable) **License:** MIT **Toolchain:** Rust stable, edition 2024 diff --git a/Cargo.lock b/Cargo.lock index 3223b9c..2ee6b7d 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4550,6 +4550,7 @@ dependencies = [ "color-eyre", "lance", "lance-index", + "omnigraph-cluster", "omnigraph-compiler", "omnigraph-engine", "omnigraph-policy", @@ -4563,6 +4564,19 @@ dependencies = [ "tokio", ] +[[package]] +name = "omnigraph-cluster" +version = "0.6.1" +dependencies = [ + "omnigraph-compiler", + "serde", + "serde_json", + "serde_yaml", + "sha2", + "tempfile", + "thiserror", +] + [[package]] name = "omnigraph-compiler" version = "0.6.1" diff --git a/Cargo.toml b/Cargo.toml index 66bfc01..17990ea 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -4,6 +4,7 @@ members = [ "crates/omnigraph-compiler", "crates/omnigraph", "crates/omnigraph-cli", + "crates/omnigraph-cluster", "crates/omnigraph-policy", "crates/omnigraph-server", ] diff --git a/crates/omnigraph-cli/Cargo.toml b/crates/omnigraph-cli/Cargo.toml index 641068e..bc50551 100644 --- a/crates/omnigraph-cli/Cargo.toml +++ b/crates/omnigraph-cli/Cargo.toml @@ -15,6 +15,7 @@ path = "src/main.rs" [dependencies] omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.1" } omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.1" } +omnigraph-cluster = { path = "../omnigraph-cluster", version = "0.6.1" } omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.1" } omnigraph-server = { path = "../omnigraph-server", version = "0.6.1" } clap = { workspace = true } diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 29b55c4..23f1569 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -10,6 +10,9 @@ use color_eyre::eyre::{Result, bail}; use omnigraph::db::{Omnigraph, ReadTarget, SnapshotId}; use omnigraph::loader::LoadMode; use omnigraph::storage::normalize_root_uri; +use omnigraph_cluster::{ + DiagnosticSeverity, PlanOutput, ValidateOutput, plan_config_dir, validate_config_dir, +}; use omnigraph_compiler::query::parser::parse_query; use omnigraph_compiler::schema::parser::parse_schema; use omnigraph_compiler::{ @@ -305,6 +308,11 @@ enum Command { #[arg(long)] json: bool, }, + /// Validate and plan read-only cluster configuration. + Cluster { + #[command(subcommand)] + command: ClusterCommand, + }, /// Manage graphs on a multi-graph server (MR-668) Graphs { #[command(subcommand)] @@ -312,6 +320,28 @@ enum Command { }, } +#[derive(Debug, Subcommand)] +enum ClusterCommand { + /// Validate cluster.yaml and referenced schemas, queries, and policy files. + Validate { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, + /// Produce a read-only plan by diffing cluster.yaml against __cluster/state.json. + Plan { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, +} + /// Operations on the graph registry of a multi-graph server (MR-668). /// /// All operations target a remote multi-graph server URL (http:// or @@ -683,6 +713,77 @@ fn print_json<T: Serialize>(value: &T) -> Result<()> { Ok(()) } +fn print_cluster_validate_human(output: &ValidateOutput) { + if output.ok { + println!( + "cluster config valid: {} resource(s), {} dependency edge(s)", + output.resources.len(), + output.dependencies.len() + ); + } else { + println!("cluster config invalid"); + } + print_cluster_diagnostics(&output.diagnostics); +} + +fn print_cluster_plan_human(output: &PlanOutput) { + if output.ok { + println!( + "cluster plan: {} change(s), {} approval gate(s)", + output.changes.len(), + output.approvals_required.len() + ); + for change in &output.changes { + println!(" {:?} {}", change.operation, change.resource); + } + if output.changes.is_empty() { + println!(" no changes"); + } + } else { + println!("cluster plan failed"); + } + print_cluster_diagnostics(&output.diagnostics); +} + +fn print_cluster_diagnostics(diagnostics: &[omnigraph_cluster::Diagnostic]) { + for diagnostic in diagnostics { + let label = match diagnostic.severity { + DiagnosticSeverity::Error => "ERROR", + DiagnosticSeverity::Warning => "WARN ", + }; + println!( + "{label} {} {}: {}", + diagnostic.code, diagnostic.path, diagnostic.message + ); + } +} + +fn finish_cluster_validate(output: &ValidateOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_validate_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + +fn finish_cluster_plan(output: &PlanOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_plan_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + fn is_remote_uri(uri: &str) -> bool { uri.starts_with("http://") || uri.starts_with("https://") } @@ -801,13 +902,11 @@ struct ResolvedPolicyContext { fn resolve_policy_context(config: &OmnigraphConfig) -> Result<ResolvedPolicyContext> { let selected = config.resolve_policy_tooling_graph_selection()?; - let policy_file = config - .resolve_policy_file_for(selected) - .ok_or_else(|| { - color_eyre::eyre::eyre!( - "policy.file or graphs.<name>.policy.file must be set in omnigraph.yaml" - ) - })?; + let policy_file = config.resolve_policy_file_for(selected).ok_or_else(|| { + color_eyre::eyre::eyre!( + "policy.file or graphs.<name>.policy.file must be set in omnigraph.yaml" + ) + })?; let graph_id = match selected { Some(name) => graph_resource_id_for_selection(Some(name), ""), None => graph_resource_id_for_selection(None, "default"), @@ -2166,16 +2265,14 @@ fn rewrite_deprecated_argv(args: Vec<OsString>) -> Vec<OsString> { } if let Some(sub) = args.get(1).and_then(|s| s.to_str()) { match sub { - "read" => eprintln!( - "warning: `omnigraph read` is deprecated; use `omnigraph query` instead" - ), + "read" => { + eprintln!("warning: `omnigraph read` is deprecated; use `omnigraph query` instead") + } "change" => eprintln!( "warning: `omnigraph change` is deprecated; use `omnigraph mutate` instead" ), "check" => { - eprintln!( - "warning: `omnigraph check` is deprecated; use `omnigraph lint` instead" - ); + eprintln!("warning: `omnigraph check` is deprecated; use `omnigraph lint` instead"); // Rewrite the top-level subcommand to `lint`; pass through the rest. let mut out = Vec::with_capacity(args.len()); out.push(args[0].clone()); @@ -3111,6 +3208,16 @@ async fn main() -> Result<()> { } } } + Command::Cluster { command } => match command { + ClusterCommand::Validate { config, json } => { + let output = validate_config_dir(config); + finish_cluster_validate(&output, json)?; + } + ClusterCommand::Plan { config, json } => { + let output = plan_config_dir(config); + finish_cluster_plan(&output, json)?; + } + }, Command::Graphs { command } => match command { GraphsCommand::List { uri, @@ -3157,8 +3264,8 @@ mod tests { use super::{ DEFAULT_BEARER_TOKEN_ENV, apply_bearer_token, bearer_token_from_env_file, legacy_change_request_body, load_cli_config, load_env_file_into_process, - normalize_bearer_token, parse_env_assignment, resolve_policy_context, - resolve_cli_graph, resolve_remote_bearer_token, + normalize_bearer_token, parse_env_assignment, resolve_cli_graph, resolve_policy_context, + resolve_remote_bearer_token, }; use omnigraph_server::load_config; use reqwest::header::AUTHORIZATION; @@ -3420,7 +3527,8 @@ graphs: } #[test] - fn graph_identity_resolve_policy_context_named_cli_graph_uses_graph_key_not_project_name_or_uri() { + fn graph_identity_resolve_policy_context_named_cli_graph_uses_graph_key_not_project_name_or_uri() + { let temp = tempdir().unwrap(); let config_path = temp.path().join("omnigraph.yaml"); fs::write( diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 9682d9a..156dd6e 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -78,6 +78,52 @@ policy: (config, policy) } +fn write_cluster_config_fixture(root: &std::path::Path) { + fs::write( + root.join("people.pg"), + r#" +node Person { + name: String @key + age: I32? +} +"#, + ) + .unwrap(); + fs::write( + root.join("people.gq"), + r#" +query find_person($name: String) { + match { $p: Person { name: $name } } + return { $p.name, $p.age } +} +"#, + ) + .unwrap(); + fs::write(root.join("base.policy.yaml"), "rules: []\n").unwrap(); + fs::write( + root.join("cluster.yaml"), + r#" +version: 1 +metadata: + name: company-brain +state: + backend: cluster + lock: true +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +policies: + base: + file: ./base.policy.yaml + applies_to: [knowledge] +"#, + ) + .unwrap(); +} + #[test] fn version_command_prints_current_cli_version() { let output = output_success(cli().arg("version")); @@ -89,6 +135,105 @@ fn version_command_prints_current_cli_version() { ); } +#[test] +fn cluster_validate_config_success() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let output = output_success( + cli() + .arg("cluster") + .arg("validate") + .arg("--config") + .arg(temp.path()), + ); + let stdout = stdout_string(&output); + assert!(stdout.contains("cluster config valid"), "{stdout}"); +} + +#[test] +fn cluster_validate_json_is_stable() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("validate") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert!(json["resource_digests"]["graph.knowledge"].is_string()); + assert!(json["resource_digests"]["query.knowledge.find_person"].is_string()); + assert_eq!(json["dependencies"][0]["from"], "policy.base"); + assert_eq!(json["dependencies"][0]["to"], "graph.knowledge"); +} + +#[test] +fn cluster_plan_json_reads_inferred_local_state() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#" +{ + "version": 1, + "applied_revision": { + "config_digest": "old", + "resources": { + "graph.knowledge": { "digest": "old-graph" }, + "policy.old": { "digest": "old-policy" } + } + } +} +"#, + ) + .unwrap(); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("plan") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["state_observations"]["state_found"], true); + assert!( + json["changes"] + .as_array() + .unwrap() + .iter() + .any(|change| change["resource"] == "policy.old" && change["operation"] == "delete"), + "plan should read state and delete stale resources: {json}" + ); +} + +#[test] +fn cluster_validate_invalid_config_exits_nonzero() { + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("cluster.yaml"), + "version: 1\ngraphs: {}\npipelines: {}\n", + ) + .unwrap(); + + let output = output_failure( + cli() + .arg("cluster") + .arg("validate") + .arg("--config") + .arg(temp.path()), + ); + let stdout = stdout_string(&output); + assert!(stdout.contains("future_phase_field"), "{stdout}"); +} + #[test] fn short_version_flag_prints_current_cli_version() { let output = output_success(cli().arg("-v")); @@ -798,8 +943,7 @@ fn deprecated_read_and_change_subcommands_emit_warnings() { let output = cli().arg("read").output().unwrap(); let stderr = String::from_utf8(output.stderr).unwrap(); assert!( - stderr.contains("`omnigraph read` is deprecated") - && stderr.contains("`omnigraph query`"), + stderr.contains("`omnigraph read` is deprecated") && stderr.contains("`omnigraph query`"), "expected `omnigraph read` deprecation warning; got: {stderr}" ); @@ -2394,9 +2538,19 @@ fn queries_validate_exits_zero_on_clean_registry() { ); let config = graph.write_config( "omnigraph.yaml", - &queries_test_config(&graph.path().to_string_lossy(), "find_person", "find_person.gq"), + &queries_test_config( + &graph.path().to_string_lossy(), + "find_person", + "find_person.gq", + ), + ); + let output = output_success( + cli() + .arg("queries") + .arg("validate") + .arg("--config") + .arg(&config), ); - let output = output_success(cli().arg("queries").arg("validate").arg("--config").arg(&config)); let stdout = stdout_string(&output); assert!(stdout.contains("OK"), "stdout:\n{stdout}"); } @@ -2405,12 +2559,21 @@ fn queries_validate_exits_zero_on_clean_registry() { fn queries_validate_exits_nonzero_on_type_broken_query() { let graph = SystemGraph::loaded(); // `Widget` is not in the fixture schema. - graph.write_query("ghost.gq", "query ghost() { match { $w: Widget } return { $w.name } }"); + graph.write_query( + "ghost.gq", + "query ghost() { match { $w: Widget } return { $w.name } }", + ); let config = graph.write_config( "omnigraph.yaml", &queries_test_config(&graph.path().to_string_lossy(), "ghost", "ghost.gq"), ); - let output = output_failure(cli().arg("queries").arg("validate").arg("--config").arg(&config)); + let output = output_failure( + cli() + .arg("queries") + .arg("validate") + .arg("--config") + .arg(&config), + ); let stdout = stdout_string(&output); assert!( stdout.contains("ghost"), @@ -2444,7 +2607,13 @@ fn queries_list_prints_registered_query() { graph.path().to_string_lossy().replace('\'', "''") ), ); - let output = output_success(cli().arg("queries").arg("list").arg("--config").arg(&config)); + let output = output_success( + cli() + .arg("queries") + .arg("list") + .arg("--config") + .arg(&config), + ); let stdout = stdout_string(&output); assert!(stdout.contains("find_person"), "stdout:\n{stdout}"); assert!( @@ -2480,7 +2649,13 @@ fn queries_list_requires_graph_selection_for_per_graph_only_registries() { ), ); - let output = output_failure(cli().arg("queries").arg("list").arg("--config").arg(&config)); + let output = output_failure( + cli() + .arg("queries") + .arg("list") + .arg("--config") + .arg(&config), + ); let stderr = String::from_utf8_lossy(&output.stderr); assert!( stderr.contains("local") && stderr.contains("--target local"), @@ -2505,7 +2680,13 @@ fn queries_list_without_graph_selection_lists_top_level_registry() { ), ); - let output = output_success(cli().arg("queries").arg("list").arg("--config").arg(&config)); + let output = output_success( + cli() + .arg("queries") + .arg("list") + .arg("--config") + .arg(&config), + ); let stdout = stdout_string(&output); assert!(stdout.contains("top_find"), "stdout:\n{stdout}"); } @@ -2524,7 +2705,11 @@ fn queries_list_unknown_target_errors() { ); let config = graph.write_config( "omnigraph.yaml", - &queries_test_config(&graph.path().to_string_lossy(), "find_person", "find_person.gq"), + &queries_test_config( + &graph.path().to_string_lossy(), + "find_person", + "find_person.gq", + ), ); let output = output_failure( cli() @@ -2566,7 +2751,7 @@ fn queries_commands_reject_named_graph_with_populated_top_level_block() { " file: ./find_person.gq\n", "cli:\n", " graph: local\n", - "queries:\n", // populated top-level block: the coherence violation + "queries:\n", // populated top-level block: the coherence violation " legacy:\n", " file: ./legacy.gq\n", "policy: {{}}\n", @@ -2592,8 +2777,14 @@ fn queries_validate_exits_nonzero_on_duplicate_tool_name() { // collision — `queries validate` must fail (offline, before the engine // opens) and name both queries plus the contested tool. let graph = SystemGraph::loaded(); - graph.write_query("a.gq", "query a() { match { $p: Person } return { $p.name } }"); - graph.write_query("b.gq", "query b() { match { $p: Person } return { $p.name } }"); + graph.write_query( + "a.gq", + "query a() { match { $p: Person } return { $p.name } }", + ); + graph.write_query( + "b.gq", + "query b() { match { $p: Person } return { $p.name } }", + ); let config = graph.write_config( "omnigraph.yaml", &format!( @@ -2615,7 +2806,13 @@ fn queries_validate_exits_nonzero_on_duplicate_tool_name() { graph.path().to_string_lossy().replace('\'', "''") ), ); - let output = output_failure(cli().arg("queries").arg("validate").arg("--config").arg(&config)); + let output = output_failure( + cli() + .arg("queries") + .arg("validate") + .arg("--config") + .arg(&config), + ); let stderr = String::from_utf8_lossy(&output.stderr); assert!( stderr.contains("dup") && stderr.contains("'a'") && stderr.contains("'b'"), @@ -2635,7 +2832,10 @@ fn queries_validate_positional_uri_ignores_default_graph() { ); // `Widget` is not in the fixture schema — the default graph's per-graph // query would break validate if it were (wrongly) selected. - graph.write_query("broken.gq", "query broken() { match { $w: Widget } return { $w.name } }"); + graph.write_query( + "broken.gq", + "query broken() { match { $w: Widget } return { $w.name } }", + ); let config = graph.write_config( "omnigraph.yaml", concat!( diff --git a/crates/omnigraph-cluster/Cargo.toml b/crates/omnigraph-cluster/Cargo.toml new file mode 100644 index 0000000..60e7785 --- /dev/null +++ b/crates/omnigraph-cluster/Cargo.toml @@ -0,0 +1,20 @@ +[package] +name = "omnigraph-cluster" +version = "0.6.1" +edition = "2024" +description = "Read-only cluster configuration validation and planning for Omnigraph." +license = "MIT" +repository = "https://github.com/ModernRelay/omnigraph" +homepage = "https://github.com/ModernRelay/omnigraph" +documentation = "https://docs.rs/omnigraph-cluster" + +[dependencies] +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.1" } +serde = { workspace = true } +serde_json = { workspace = true } +serde_yaml = { workspace = true } +sha2 = { workspace = true } +thiserror = { workspace = true } + +[dev-dependencies] +tempfile = { workspace = true } diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs new file mode 100644 index 0000000..861ae22 --- /dev/null +++ b/crates/omnigraph-cluster/src/lib.rs @@ -0,0 +1,1275 @@ +use std::collections::{BTreeMap, BTreeSet}; +use std::fs; +use std::path::{Path, PathBuf}; + +use omnigraph_compiler::build_catalog; +use omnigraph_compiler::query::parser::parse_query; +use omnigraph_compiler::query::typecheck::typecheck_query_decl; +use omnigraph_compiler::schema::parser::parse_schema; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; + +pub const CLUSTER_CONFIG_FILE: &str = "cluster.yaml"; +pub const CLUSTER_STATE_FILE: &str = "__cluster/state.json"; + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum DiagnosticSeverity { + Error, + Warning, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct Diagnostic { + pub code: String, + pub severity: DiagnosticSeverity, + pub path: String, + pub message: String, +} + +impl Diagnostic { + fn error(code: impl Into<String>, path: impl Into<String>, message: impl Into<String>) -> Self { + Self { + code: code.into(), + severity: DiagnosticSeverity::Error, + path: path.into(), + message: message.into(), + } + } + + fn warning( + code: impl Into<String>, + path: impl Into<String>, + message: impl Into<String>, + ) -> Self { + Self { + code: code.into(), + severity: DiagnosticSeverity::Warning, + path: path.into(), + message: message.into(), + } + } +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct ResourceSummary { + pub address: String, + pub kind: String, + pub digest: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub path: Option<String>, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq, PartialOrd, Ord)] +pub struct Dependency { + pub from: String, + pub to: String, +} + +#[derive(Debug, Clone, Serialize)] +pub struct ValidateOutput { + pub ok: bool, + pub config_dir: String, + pub config_file: String, + pub resource_digests: BTreeMap<String, String>, + pub resources: Vec<ResourceSummary>, + pub dependencies: Vec<Dependency>, + pub diagnostics: Vec<Diagnostic>, +} + +#[derive(Debug, Clone, Serialize)] +pub struct DesiredRevision { + #[serde(skip_serializing_if = "Option::is_none")] + pub config_digest: Option<String>, +} + +#[derive(Debug, Clone, Serialize)] +pub struct StateObservations { + pub state_path: String, + pub state_found: bool, + #[serde(skip_serializing_if = "Option::is_none")] + pub applied_config_digest: Option<String>, + pub resource_count: usize, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum PlanOperation { + Create, + Update, + Delete, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct PlanChange { + pub resource: String, + pub operation: PlanOperation, + #[serde(skip_serializing_if = "Option::is_none")] + pub before_digest: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub after_digest: Option<String>, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct BlastRadius { + pub resource: String, + pub affected: Vec<String>, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct ApprovalRequirement { + pub resource: String, + pub reason: String, +} + +#[derive(Debug, Clone, Serialize)] +pub struct PlanOutput { + pub ok: bool, + pub config_dir: String, + pub desired_revision: DesiredRevision, + pub resource_digests: BTreeMap<String, String>, + pub dependencies: Vec<Dependency>, + pub state_observations: StateObservations, + pub changes: Vec<PlanChange>, + pub blast_radius: Vec<BlastRadius>, + pub approvals_required: Vec<ApprovalRequirement>, + pub diagnostics: Vec<Diagnostic>, +} + +#[derive(Debug, Clone)] +struct DesiredCluster { + config_dir: PathBuf, + config_digest: String, + resource_digests: BTreeMap<String, String>, + resources: Vec<ResourceSummary>, + dependencies: Vec<Dependency>, +} + +#[derive(Debug)] +struct LoadOutcome { + desired: Option<DesiredCluster>, + diagnostics: Vec<Diagnostic>, + config_dir: PathBuf, + config_file: PathBuf, +} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct RawClusterConfig { + version: u32, + #[serde(default)] + metadata: Metadata, + #[serde(default)] + state: StateConfig, + #[serde(default)] + graphs: BTreeMap<String, GraphConfig>, + #[serde(default)] + policies: BTreeMap<String, PolicyConfig>, +} + +#[derive(Debug, Default, Deserialize)] +#[serde(deny_unknown_fields)] +struct Metadata { + name: Option<String>, +} + +#[derive(Debug, Default, Deserialize)] +#[serde(deny_unknown_fields)] +struct StateConfig { + backend: Option<String>, + lock: Option<bool>, +} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct GraphConfig { + schema: PathBuf, + #[serde(default)] + queries: BTreeMap<String, QueryConfig>, +} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct QueryConfig { + file: PathBuf, +} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct PolicyConfig { + file: PathBuf, + applies_to: Vec<String>, +} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct ClusterState { + version: u32, + applied_revision: AppliedRevisionState, +} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct AppliedRevisionState { + #[serde(default)] + config_digest: Option<String>, + #[serde(default)] + resources: BTreeMap<String, StateResource>, +} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct StateResource { + digest: String, +} + +pub fn validate_config_dir(config_dir: impl AsRef<Path>) -> ValidateOutput { + let outcome = load_desired(config_dir.as_ref()); + let (resource_digests, resources, dependencies) = match outcome.desired { + Some(desired) => ( + desired.resource_digests, + desired.resources, + desired.dependencies, + ), + None => (BTreeMap::new(), Vec::new(), Vec::new()), + }; + let ok = !has_errors(&outcome.diagnostics); + + ValidateOutput { + ok, + config_dir: display_path(&outcome.config_dir), + config_file: display_path(&outcome.config_file), + resource_digests, + resources, + dependencies, + diagnostics: outcome.diagnostics, + } +} + +pub fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { + let outcome = load_desired(config_dir.as_ref()); + let mut diagnostics = outcome.diagnostics; + let state_path = outcome.config_dir.join(CLUSTER_STATE_FILE); + let mut observations = StateObservations { + state_path: display_path(&state_path), + state_found: false, + applied_config_digest: None, + resource_count: 0, + }; + + let Some(desired) = outcome.desired else { + return PlanOutput { + ok: false, + config_dir: display_path(&outcome.config_dir), + desired_revision: DesiredRevision { + config_digest: None, + }, + resource_digests: BTreeMap::new(), + dependencies: Vec::new(), + state_observations: observations, + changes: Vec::new(), + blast_radius: Vec::new(), + approvals_required: Vec::new(), + diagnostics, + }; + }; + + let mut prior_resources = BTreeMap::new(); + if state_path.exists() { + observations.state_found = true; + match fs::read_to_string(&state_path) { + Ok(text) => match serde_json::from_str::<ClusterState>(&text) { + Ok(state) if state.version == 1 => { + observations.applied_config_digest = state.applied_revision.config_digest; + observations.resource_count = state.applied_revision.resources.len(); + prior_resources = state + .applied_revision + .resources + .into_iter() + .map(|(address, resource)| (address, resource.digest)) + .collect(); + } + Ok(state) => diagnostics.push(Diagnostic::error( + "unsupported_state_version", + "state.version", + format!( + "unsupported cluster state version {}; this build supports version 1", + state.version + ), + )), + Err(err) => diagnostics.push(Diagnostic::error( + "invalid_state_json", + CLUSTER_STATE_FILE, + format!("could not parse state JSON: {err}"), + )), + }, + Err(err) => diagnostics.push(Diagnostic::error( + "state_read_error", + CLUSTER_STATE_FILE, + format!("could not read state file: {err}"), + )), + } + } + + let changes = if has_errors(&diagnostics) { + Vec::new() + } else { + diff_resources(&prior_resources, &desired.resource_digests) + }; + let blast_radius = compute_blast_radius(&changes, &desired.dependencies); + let approvals_required = compute_approvals(&changes); + let ok = !has_errors(&diagnostics); + + PlanOutput { + ok, + config_dir: display_path(&desired.config_dir), + desired_revision: DesiredRevision { + config_digest: Some(desired.config_digest), + }, + resource_digests: desired.resource_digests, + dependencies: desired.dependencies, + state_observations: observations, + changes, + blast_radius, + approvals_required, + diagnostics, + } +} + +fn load_desired(config_dir: &Path) -> LoadOutcome { + let config_dir = config_dir.to_path_buf(); + let config_file = config_dir.join(CLUSTER_CONFIG_FILE); + let mut diagnostics = Vec::new(); + + if !config_dir.is_dir() { + diagnostics.push(Diagnostic::error( + "config_dir_not_found", + display_path(&config_dir), + "`--config` must point at a directory containing cluster.yaml", + )); + return LoadOutcome { + desired: None, + diagnostics, + config_dir, + config_file, + }; + } + + let text = match fs::read_to_string(&config_file) { + Ok(text) => text, + Err(err) => { + diagnostics.push(Diagnostic::error( + "cluster_config_read_error", + CLUSTER_CONFIG_FILE, + format!("could not read cluster.yaml: {err}"), + )); + return LoadOutcome { + desired: None, + diagnostics, + config_dir, + config_file, + }; + } + }; + + diagnostics.extend(duplicate_key_diagnostics(&text)); + diagnostics.extend(future_field_diagnostics(&text)); + if has_errors(&diagnostics) { + return LoadOutcome { + desired: None, + diagnostics, + config_dir, + config_file, + }; + } + + let raw = match serde_yaml::from_str::<RawClusterConfig>(&text) { + Ok(raw) => raw, + Err(err) => { + diagnostics.push(Diagnostic::error( + "invalid_cluster_yaml", + CLUSTER_CONFIG_FILE, + format!("could not parse cluster.yaml: {err}"), + )); + return LoadOutcome { + desired: None, + diagnostics, + config_dir, + config_file, + }; + } + }; + + if raw.version != 1 { + diagnostics.push(Diagnostic::error( + "unsupported_cluster_config_version", + "version", + format!( + "unsupported cluster config version {}; this build supports version 1", + raw.version + ), + )); + } + if let Some(name) = raw.metadata.name.as_deref() { + if name.trim().is_empty() { + diagnostics.push(Diagnostic::error( + "empty_metadata_name", + "metadata.name", + "metadata.name must not be empty when provided", + )); + } + } + if let Some(backend) = raw.state.backend.as_deref() { + if backend != "cluster" { + diagnostics.push(Diagnostic::error( + "unsupported_state_backend", + "state.backend", + "Stage 1 supports only omitted state.backend or `cluster`", + )); + } + } + let _lock_parsed_for_forward_compat = raw.state.lock; + + let mut resources = BTreeMap::new(); + let mut dependencies = BTreeSet::new(); + let mut graph_query_digests: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new(); + let mut graph_schema_digests: BTreeMap<String, String> = BTreeMap::new(); + + for (graph_id, graph) in &raw.graphs { + validate_id( + "graph id", + &format!("graphs.{graph_id}"), + graph_id, + &mut diagnostics, + ); + let graph_address = graph_address(graph_id); + let schema_address = schema_address(graph_id); + dependencies.insert(Dependency { + from: schema_address.clone(), + to: graph_address.clone(), + }); + + let schema_path = resolve_config_path(&config_dir, &graph.schema); + let schema_source = match fs::read_to_string(&schema_path) { + Ok(source) => { + let digest = sha256_hex(source.as_bytes()); + graph_schema_digests.insert(graph_id.clone(), digest.clone()); + resources.insert( + schema_address.clone(), + ResourceSummary { + address: schema_address.clone(), + kind: "schema".to_string(), + digest, + path: Some(display_path(&schema_path)), + }, + ); + Some(source) + } + Err(err) => { + diagnostics.push(Diagnostic::error( + "schema_file_missing", + format!("graphs.{graph_id}.schema"), + format!( + "could not read schema file '{}': {err}", + schema_path.display() + ), + )); + None + } + }; + + let catalog = schema_source.and_then(|source| match parse_schema(&source) { + Ok(schema) => match build_catalog(&schema) { + Ok(catalog) => Some(catalog), + Err(err) => { + diagnostics.push(Diagnostic::error( + "schema_catalog_error", + format!("graphs.{graph_id}.schema"), + err.to_string(), + )); + None + } + }, + Err(err) => { + diagnostics.push(Diagnostic::error( + "schema_parse_error", + format!("graphs.{graph_id}.schema"), + err.to_string(), + )); + None + } + }); + + for (query_name, query) in &graph.queries { + validate_id( + "query name", + &format!("graphs.{graph_id}.queries.{query_name}"), + query_name, + &mut diagnostics, + ); + let query_address = query_address(graph_id, query_name); + dependencies.insert(Dependency { + from: query_address.clone(), + to: graph_address.clone(), + }); + dependencies.insert(Dependency { + from: query_address.clone(), + to: schema_address.clone(), + }); + + let query_path = resolve_config_path(&config_dir, &query.file); + match fs::read_to_string(&query_path) { + Ok(source) => { + let digest = sha256_hex(source.as_bytes()); + graph_query_digests + .entry(graph_id.clone()) + .or_default() + .insert(query_name.clone(), digest.clone()); + resources.insert( + query_address.clone(), + ResourceSummary { + address: query_address, + kind: "query".to_string(), + digest, + path: Some(display_path(&query_path)), + }, + ); + validate_query_source( + graph_id, + query_name, + &source, + catalog.as_ref(), + &mut diagnostics, + ); + } + Err(err) => diagnostics.push(Diagnostic::error( + "query_file_missing", + format!("graphs.{graph_id}.queries.{query_name}.file"), + format!( + "could not read query file '{}': {err}", + query_path.display() + ), + )), + } + } + } + + for graph_id in raw.graphs.keys() { + let digest = graph_digest( + graph_id, + graph_schema_digests.get(graph_id), + graph_query_digests.get(graph_id), + ); + resources.insert( + graph_address(graph_id), + ResourceSummary { + address: graph_address(graph_id), + kind: "graph".to_string(), + digest, + path: None, + }, + ); + } + + for (policy_name, policy) in &raw.policies { + validate_id( + "policy name", + &format!("policies.{policy_name}"), + policy_name, + &mut diagnostics, + ); + if policy.applies_to.is_empty() { + diagnostics.push(Diagnostic::error( + "policy_missing_applies_to", + format!("policies.{policy_name}.applies_to"), + "policy.applies_to must name `cluster` or at least one graph", + )); + } + + let policy_address = policy_address(policy_name); + for (idx, target) in policy.applies_to.iter().enumerate() { + match normalize_policy_target(target) { + PolicyTarget::Cluster => {} + PolicyTarget::Graph(graph_id) => { + if raw.graphs.contains_key(&graph_id) { + dependencies.insert(Dependency { + from: policy_address.clone(), + to: graph_address(&graph_id), + }); + } else { + diagnostics.push(Diagnostic::error( + "dangling_graph_reference", + format!("policies.{policy_name}.applies_to[{idx}]"), + format!( + "policy references graph `{graph_id}`, but no graph with that id is declared" + ), + )); + } + } + PolicyTarget::WrongKind(kind) => diagnostics.push(Diagnostic::error( + "wrong_kind_reference", + format!("policies.{policy_name}.applies_to[{idx}]"), + format!("policy applies_to expects graph refs or `cluster`, got `{kind}`"), + )), + } + } + + let policy_path = resolve_config_path(&config_dir, &policy.file); + match fs::read(&policy_path) { + Ok(bytes) => { + resources.insert( + policy_address.clone(), + ResourceSummary { + address: policy_address, + kind: "policy".to_string(), + digest: sha256_hex(&bytes), + path: Some(display_path(&policy_path)), + }, + ); + } + Err(err) => diagnostics.push(Diagnostic::error( + "policy_file_missing", + format!("policies.{policy_name}.file"), + format!( + "could not read policy file '{}': {err}", + policy_path.display() + ), + )), + } + } + + let mut resource_digests = BTreeMap::new(); + let mut resource_list = Vec::new(); + for (address, resource) in resources { + resource_digests.insert(address, resource.digest.clone()); + resource_list.push(resource); + } + let dependencies: Vec<_> = dependencies.into_iter().collect(); + let config_digest = desired_config_digest(&text, &resource_digests); + + LoadOutcome { + desired: Some(DesiredCluster { + config_dir: config_dir.clone(), + config_digest, + resource_digests, + resources: resource_list, + dependencies, + }), + diagnostics, + config_dir, + config_file, + } +} + +fn validate_query_source( + graph_id: &str, + query_name: &str, + source: &str, + catalog: Option<&omnigraph_compiler::catalog::Catalog>, + diagnostics: &mut Vec<Diagnostic>, +) { + let path = format!("graphs.{graph_id}.queries.{query_name}"); + match parse_query(source) { + Ok(query_file) => { + let Some(query_decl) = query_file.queries.iter().find(|q| q.name == query_name) else { + diagnostics.push(Diagnostic::error( + "query_key_mismatch", + path, + format!("no `query {query_name}` declaration found in the referenced .gq file"), + )); + return; + }; + if let Some(catalog) = catalog { + if let Err(err) = typecheck_query_decl(catalog, query_decl) { + diagnostics.push(Diagnostic::error( + "query_typecheck_error", + format!("graphs.{graph_id}.queries.{query_name}"), + err.to_string(), + )); + } + } else { + diagnostics.push(Diagnostic::warning( + "query_typecheck_skipped", + format!("graphs.{graph_id}.queries.{query_name}"), + "query parsed, but type-check was skipped because the graph schema is invalid", + )); + } + } + Err(err) => diagnostics.push(Diagnostic::error( + "query_parse_error", + path, + err.to_string(), + )), + } +} + +fn diff_resources( + prior: &BTreeMap<String, String>, + desired: &BTreeMap<String, String>, +) -> Vec<PlanChange> { + let mut changes = Vec::new(); + for (address, after) in desired { + match prior.get(address) { + None => changes.push(PlanChange { + resource: address.clone(), + operation: PlanOperation::Create, + before_digest: None, + after_digest: Some(after.clone()), + }), + Some(before) if before != after => changes.push(PlanChange { + resource: address.clone(), + operation: PlanOperation::Update, + before_digest: Some(before.clone()), + after_digest: Some(after.clone()), + }), + Some(_) => {} + } + } + for (address, before) in prior { + if !desired.contains_key(address) { + changes.push(PlanChange { + resource: address.clone(), + operation: PlanOperation::Delete, + before_digest: Some(before.clone()), + after_digest: None, + }); + } + } + changes.sort_by(|a, b| a.resource.cmp(&b.resource)); + changes +} + +fn compute_blast_radius(changes: &[PlanChange], dependencies: &[Dependency]) -> Vec<BlastRadius> { + changes + .iter() + .filter_map(|change| { + let affected: Vec<_> = dependencies + .iter() + .filter_map(|dep| (dep.to == change.resource).then_some(dep.from.clone())) + .collect(); + (!affected.is_empty()).then(|| BlastRadius { + resource: change.resource.clone(), + affected, + }) + }) + .collect() +} + +fn compute_approvals(changes: &[PlanChange]) -> Vec<ApprovalRequirement> { + changes + .iter() + .filter_map(|change| { + if change.operation == PlanOperation::Delete + && (change.resource.starts_with("graph.") || change.resource.starts_with("schema.")) + { + Some(ApprovalRequirement { + resource: change.resource.clone(), + reason: "delete may remove deployed graph or schema definition".to_string(), + }) + } else { + None + } + }) + .collect() +} + +fn duplicate_key_diagnostics(text: &str) -> Vec<Diagnostic> { + #[derive(Debug)] + struct Frame { + indent: isize, + path: String, + keys: BTreeSet<String>, + } + + let mut diagnostics = Vec::new(); + let mut stack = vec![Frame { + indent: -1, + path: String::new(), + keys: BTreeSet::new(), + }]; + + for (line_idx, line) in text.lines().enumerate() { + let line_without_comment = strip_comment(line); + if line_without_comment.trim().is_empty() { + continue; + } + let indent = line_without_comment + .chars() + .take_while(|ch| *ch == ' ') + .count() as isize; + let trimmed = line_without_comment.trim_start(); + if trimmed.starts_with('-') { + continue; + } + let Some((raw_key, raw_value)) = trimmed.split_once(':') else { + continue; + }; + let key = raw_key.trim(); + if key.is_empty() || key.starts_with('{') || key.starts_with('[') { + continue; + } + + while stack.last().is_some_and(|frame| indent <= frame.indent) { + stack.pop(); + } + let parent = stack.last_mut().expect("root frame is always present"); + let full_path = if parent.path.is_empty() { + key.to_string() + } else { + format!("{}.{}", parent.path, key) + }; + if !parent.keys.insert(key.to_string()) { + diagnostics.push(Diagnostic::error( + "duplicate_yaml_key", + full_path.clone(), + format!("duplicate YAML key `{key}` on line {}", line_idx + 1), + )); + } + if raw_value.trim().is_empty() { + stack.push(Frame { + indent, + path: full_path, + keys: BTreeSet::new(), + }); + } + } + + diagnostics +} + +fn future_field_diagnostics(text: &str) -> Vec<Diagnostic> { + let Ok(value) = serde_yaml::from_str::<serde_yaml::Value>(text) else { + return Vec::new(); + }; + let Some(mapping) = value.as_mapping() else { + return Vec::new(); + }; + let future_fields = [ + "apply", + "env_file", + "providers", + "pipelines", + "embeddings", + "ui", + "aliases", + "bindings", + ]; + mapping + .keys() + .filter_map(|key| key.as_str()) + .filter(|key| future_fields.contains(key)) + .map(|key| { + Diagnostic::error( + "future_phase_field", + key, + format!("`{key}` is reserved for a later cluster-control phase"), + ) + }) + .collect() +} + +fn strip_comment(line: &str) -> String { + let mut in_single_quote = false; + let mut in_double_quote = false; + let mut escaped = false; + + for (idx, ch) in line.char_indices() { + if escaped { + escaped = false; + continue; + } + match ch { + '\\' if in_double_quote => escaped = true, + '\'' if !in_double_quote => in_single_quote = !in_single_quote, + '"' if !in_single_quote => in_double_quote = !in_double_quote, + '#' if !in_single_quote && !in_double_quote => return line[..idx].to_string(), + _ => {} + } + } + + line.to_string() +} + +fn validate_id(kind: &str, path: &str, value: &str, diagnostics: &mut Vec<Diagnostic>) { + let mut chars = value.chars(); + let valid = chars + .next() + .is_some_and(|ch| ch.is_ascii_alphabetic() || ch == '_') + && chars.all(|ch| ch.is_ascii_alphanumeric() || ch == '_' || ch == '-'); + if !valid { + diagnostics.push(Diagnostic::error( + "invalid_resource_id", + path, + format!("{kind} `{value}` must start with a letter or `_` and contain only ASCII letters, digits, `_`, or `-`"), + )); + } +} + +enum PolicyTarget { + Cluster, + Graph(String), + WrongKind(String), +} + +fn normalize_policy_target(value: &str) -> PolicyTarget { + if value == "cluster" { + PolicyTarget::Cluster + } else if let Some(graph_id) = value.strip_prefix("graph.") { + PolicyTarget::Graph(graph_id.to_string()) + } else if value.contains('.') { + PolicyTarget::WrongKind(value.to_string()) + } else { + PolicyTarget::Graph(value.to_string()) + } +} + +fn graph_address(graph_id: &str) -> String { + format!("graph.{graph_id}") +} + +fn schema_address(graph_id: &str) -> String { + format!("schema.{graph_id}") +} + +fn query_address(graph_id: &str, query_name: &str) -> String { + format!("query.{graph_id}.{query_name}") +} + +fn policy_address(policy_name: &str) -> String { + format!("policy.{policy_name}") +} + +fn resolve_config_path(config_dir: &Path, path: &Path) -> PathBuf { + if path.is_absolute() { + path.to_path_buf() + } else { + config_dir.join(path) + } +} + +fn graph_digest( + graph_id: &str, + schema_digest: Option<&String>, + query_digests: Option<&BTreeMap<String, String>>, +) -> String { + let mut input = format!( + "graph\0{graph_id}\0schema\0{}\0", + schema_digest.map_or("", String::as_str) + ); + if let Some(query_digests) = query_digests { + for (name, digest) in query_digests { + input.push_str("query\0"); + input.push_str(name); + input.push('\0'); + input.push_str(digest); + input.push('\0'); + } + } + sha256_hex(input.as_bytes()) +} + +fn desired_config_digest( + config_source: &str, + resource_digests: &BTreeMap<String, String>, +) -> String { + let mut input = String::from("cluster-config\0"); + input.push_str(config_source); + input.push('\0'); + for (address, digest) in resource_digests { + input.push_str(address); + input.push('\0'); + input.push_str(digest); + input.push('\0'); + } + sha256_hex(input.as_bytes()) +} + +fn sha256_hex(bytes: &[u8]) -> String { + let digest = Sha256::digest(bytes); + let mut out = String::with_capacity(digest.len() * 2); + for byte in digest { + out.push_str(&format!("{byte:02x}")); + } + out +} + +fn has_errors(diagnostics: &[Diagnostic]) -> bool { + diagnostics + .iter() + .any(|diagnostic| diagnostic.severity == DiagnosticSeverity::Error) +} + +fn display_path(path: &Path) -> String { + path.display().to_string() +} + +#[cfg(test)] +mod tests { + use std::fs; + + use serde_json::json; + use tempfile::tempdir; + + use super::*; + + const SCHEMA: &str = r#" +node Person { + name: String @key + age: I32? +} +"#; + + const QUERY: &str = r#" +query find_person($name: String) { + match { $p: Person { name: $name } } + return { $p.name, $p.age } +} +"#; + + fn fixture() -> tempfile::TempDir { + let dir = tempdir().unwrap(); + fs::write(dir.path().join("people.pg"), SCHEMA).unwrap(); + fs::write(dir.path().join("people.gq"), QUERY).unwrap(); + fs::write(dir.path().join("base.policy.yaml"), "rules: []\n").unwrap(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +metadata: + name: test +state: + backend: cluster + lock: true +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +policies: + base: + file: ./base.policy.yaml + applies_to: [knowledge] +"#, + ) + .unwrap(); + dir + } + + #[test] + fn valid_minimal_config() { + let dir = fixture(); + let out = validate_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.resource_digests.contains_key("graph.knowledge")); + assert!(out.resource_digests.contains_key("schema.knowledge")); + assert!( + out.dependencies + .iter() + .any(|dep| dep.from == "policy.base" && dep.to == "graph.knowledge") + ); + } + + #[test] + fn unknown_field_rejection() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\ngraphs: {}\nwat: true\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert!(out.diagnostics[0].message.contains("unknown field")); + } + + #[test] + fn future_phase_field_rejection() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\ngraphs: {}\npipelines: {}\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert_eq!(out.diagnostics[0].code, "future_phase_field"); + } + + #[test] + fn duplicate_yaml_key_rejection() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\ngraphs: {}\ngraphs: {}\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert_eq!(out.diagnostics[0].code, "duplicate_yaml_key"); + } + + #[test] + fn duplicate_yaml_key_rejection_keeps_quoted_hashes() { + let diagnostics = + duplicate_key_diagnostics("\"name#display\": one\n\"name#display\": two\n"); + assert_eq!(diagnostics.len(), 1); + assert_eq!(diagnostics[0].code, "duplicate_yaml_key"); + } + + #[test] + fn missing_schema_query_and_policy_files() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +graphs: + knowledge: + schema: ./missing.pg + queries: + find_person: { file: ./missing.gq } +policies: + base: + file: ./missing.policy.yaml + applies_to: [knowledge] +"#, + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + let codes: BTreeSet<_> = out.diagnostics.iter().map(|d| d.code.as_str()).collect(); + assert!(codes.contains("schema_file_missing")); + assert!(codes.contains("query_file_missing")); + assert!(codes.contains("policy_file_missing")); + } + + #[test] + fn wrong_kind_and_dangling_refs_fail() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg +policies: + base: + file: ./base.policy.yaml + applies_to: [query.knowledge.find_person, missing] +"#, + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + let codes: BTreeSet<_> = out.diagnostics.iter().map(|d| d.code.as_str()).collect(); + assert!(codes.contains("wrong_kind_reference")); + assert!(codes.contains("dangling_graph_reference")); + } + + #[test] + fn query_key_mismatch_fails() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg + queries: + different: { file: ./people.gq } +"#, + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert_eq!(out.diagnostics[0].code, "query_key_mismatch"); + } + + #[test] + fn query_typecheck_failure_fails() { + let dir = fixture(); + fs::write( + dir.path().join("people.gq"), + "query find_person() { match { $d: DoesNotExist } return { $d.name } }\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "query_typecheck_error") + ); + } + + #[test] + fn missing_state_plans_creates() { + let dir = fixture(); + let out = plan_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!out.state_observations.state_found); + assert!( + out.changes + .iter() + .all(|c| c.operation == PlanOperation::Create) + ); + assert!(out.changes.iter().any(|c| c.resource == "graph.knowledge")); + } + + #[test] + fn existing_state_plans_update_and_delete_deterministically() { + let dir = fixture(); + let first = plan_config_dir(dir.path()); + let state_dir = dir.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + serde_json::to_string_pretty(&json!({ + "version": 1, + "applied_revision": { + "config_digest": "old", + "resources": { + "graph.knowledge": { "digest": first.resource_digests["graph.knowledge"] }, + "policy.old": { "digest": "abc" }, + "schema.knowledge": { "digest": "old-schema" } + } + } + })) + .unwrap(), + ) + .unwrap(); + + let out = plan_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + let rendered: Vec<_> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), &change.operation)) + .collect(); + assert_eq!( + rendered, + vec![ + ("policy.base", &PlanOperation::Create), + ("policy.old", &PlanOperation::Delete), + ("query.knowledge.find_person", &PlanOperation::Create), + ("schema.knowledge", &PlanOperation::Update), + ] + ); + } + + #[test] + fn external_state_backend_rejected() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\nstate:\n backend: s3://bucket/state\ngraphs: {}\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert_eq!(out.diagnostics[0].code, "unsupported_state_backend"); + } +} diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 425fcee..0b5a234 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,6 +8,7 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests` | Cluster config parser, local JSON state diff, read-only validate/plan | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 8263919..2f27322 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -2,7 +2,7 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` schema. For a quick-start guide, see [cli.md](cli.md). -17 top-level command families, 40+ subcommands. All commands accept either a positional `URI`, `--uri`, or a `--target <name>` resolved against `omnigraph.yaml`. +18 top-level command families, 40+ subcommands. Graph commands accept either a positional `URI`, `--uri`, or a `--target <name>` resolved against `omnigraph.yaml`; `cluster` commands instead use `--config <dir>`. ## Top-level commands @@ -21,6 +21,7 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` sc | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | | `queries validate \| list` | operate on the server-side stored-query registry (the `queries:` block). `validate` type-checks every stored query against the live schema offline (opens the selected graph; exits non-zero on any breakage), catching schema drift without restarting the server; `list` prints the selected registry's query names, MCP exposure, and typed params. For per-graph registries, pass `--target <graph>` or set `cli.graph`; with no graph selection, `list` shows only top-level `queries:`. Distinct from `lint`, which validates a single `.gq` file | +| `cluster validate \| plan` | read-only cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`. No apply, lock, graph open, server change, or state write occurs in Stage 1 | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns; `--json` reports a `skipped` field) | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | | `embed` | offline JSONL embedding pipeline | @@ -73,6 +74,20 @@ policy: file: ./policy.yaml ``` +## Cluster config preview + +```bash +omnigraph cluster validate --config ./company-brain +omnigraph cluster plan --config ./company-brain --json +``` + +`--config` is a directory containing `cluster.yaml`; it defaults to `.`. +Stage 1 accepts graphs, schemas, stored queries, and policy bundle file +references. `cluster plan` reads local JSON state from +`<config-dir>/__cluster/state.json`; a missing file means empty state. External +state backends, apply, locks, pipelines, UI specs, embeddings, aliases, and +bindings are reserved for later stages. See [cluster-config.md](cluster-config.md). + ## Output formats (`query` command, alias: `read`) - `json` — pretty-printed object with metadata + rows diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md new file mode 100644 index 0000000..29d9c32 --- /dev/null +++ b/docs/user/cluster-config.md @@ -0,0 +1,95 @@ +# Cluster Config + +**Status:** Stage 1 read-only preview. + +Cluster config is the future control-plane configuration surface for a whole +OmniGraph deployment. In this stage, OmniGraph can validate a local +`cluster.yaml` folder and produce a deterministic read-only plan. It does not +apply changes, acquire locks, open graph roots, start servers, or write state. + +## Commands + +```bash +omnigraph cluster validate --config ./company-brain +omnigraph cluster plan --config ./company-brain --json +``` + +`--config` points at a directory, not a file. The directory must contain +`cluster.yaml`. When omitted, it defaults to the current directory. + +## Supported `cluster.yaml` + +Stage 1 accepts only the read-only resource subset: + +```yaml +version: 1 +metadata: + name: company-brain + +state: + backend: cluster + lock: true + +graphs: + knowledge: + schema: ./knowledge.pg + queries: + find_experts: + file: ./knowledge.gq + +policies: + base: + file: ./base.policy.yaml + applies_to: [knowledge] +``` + +`metadata.name` is a display label. `state.lock` is parsed for forward +compatibility, but no lock is acquired in this read-only stage. `state.backend` +may be omitted or set to `cluster`; external state backends are reserved for a +later stage. + +## Validation + +`cluster validate` checks: + +- `cluster.yaml` syntax and supported fields +- duplicate YAML keys +- schema, query, and policy file existence +- schema parsing and catalog construction +- stored-query parsing and query-name matching +- stored-query type-checking against the desired schema +- policy `applies_to` graph references + +Fields reserved for later phases, such as `pipelines`, `embeddings`, `ui`, +`aliases`, and `bindings`, fail with a typed diagnostic instead of being +silently ignored. + +## Planning + +`cluster plan` first performs validation, then reads local JSON state from: + +```text +<config-dir>/__cluster/state.json +``` + +If the file is missing, the state is treated as empty and every desired +resource is planned as a create. If present, the file must use this shape: + +```json +{ + "version": 1, + "applied_revision": { + "config_digest": "...", + "resources": { + "graph.knowledge": { "digest": "..." }, + "schema.knowledge": { "digest": "..." }, + "query.knowledge.find_experts": { "digest": "..." }, + "policy.base": { "digest": "..." } + } + } +} +``` + +Plan output compares desired resource digests against state resource digests +and reports `create`, `update`, and `delete` changes. The command never writes +`state.json`; apply and locking are later-stage work. diff --git a/docs/user/index.md b/docs/user/index.md index 1b93efa..6cf6ade 100644 --- a/docs/user/index.md +++ b/docs/user/index.md @@ -13,6 +13,7 @@ of MRs, internal recovery mechanics, or contributor-only invariants. | Install OmniGraph | [install.md](install.md) | | Run the CLI locally | [cli.md](cli.md) | | Look up every CLI flag and config field | [cli-reference.md](cli-reference.md) | +| Validate and plan cluster config | [cluster-config.md](cluster-config.md) | | Write schemas | [schema-language.md](schema-language.md) | | Read schema-lint diagnostic codes | [schema-lint.md](schema-lint.md) | | Write queries and mutations | [query-language.md](query-language.md) | From a7956ea5a9dfa223c9a5717b406e463e77d9d6f0 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Mon, 8 Jun 2026 21:09:23 +0300 Subject: [PATCH 025/165] Add cluster JSON state ledger status --- Cargo.lock | 2 + crates/omnigraph-cli/src/main.rs | 57 ++- crates/omnigraph-cli/tests/cli.rs | 162 +++++++ crates/omnigraph-cluster/Cargo.toml | 2 + crates/omnigraph-cluster/src/lib.rs | 714 +++++++++++++++++++++++++--- docs/dev/testing.md | 2 +- docs/user/cli-reference.md | 11 +- docs/user/cluster-config.md | 51 +- 8 files changed, 925 insertions(+), 76 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 2ee6b7d..ebe5565 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4575,6 +4575,8 @@ dependencies = [ "sha2", "tempfile", "thiserror", + "time", + "ulid", ] [[package]] diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 23f1569..4ca4a4a 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -11,7 +11,8 @@ use omnigraph::db::{Omnigraph, ReadTarget, SnapshotId}; use omnigraph::loader::LoadMode; use omnigraph::storage::normalize_root_uri; use omnigraph_cluster::{ - DiagnosticSeverity, PlanOutput, ValidateOutput, plan_config_dir, validate_config_dir, + DiagnosticSeverity, PlanOutput, StatusOutput, ValidateOutput, plan_config_dir, + status_config_dir, validate_config_dir, }; use omnigraph_compiler::query::parser::parse_query; use omnigraph_compiler::schema::parser::parse_schema; @@ -340,6 +341,15 @@ enum ClusterCommand { #[arg(long)] json: bool, }, + /// Read the local JSON state ledger without scanning live graph resources. + Status { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, } /// Operations on the graph registry of a multi-graph server (MR-668). @@ -745,6 +755,34 @@ fn print_cluster_plan_human(output: &PlanOutput) { print_cluster_diagnostics(&output.diagnostics); } +fn print_cluster_status_human(output: &StatusOutput) { + if output.ok { + let state = &output.state_observations; + if state.state_found { + println!( + "cluster state: revision {}, {} resource(s)", + state.state_revision, state.resource_count + ); + if let Some(digest) = state.applied_config_digest.as_deref() { + println!(" applied config: {digest}"); + } + if state.locked { + match state.lock_id.as_deref() { + Some(lock_id) => println!(" lock: held ({lock_id})"), + None => println!(" lock: held"), + } + } else { + println!(" lock: not held"); + } + } else { + println!("cluster state missing"); + } + } else { + println!("cluster status failed"); + } + print_cluster_diagnostics(&output.diagnostics); +} + fn print_cluster_diagnostics(diagnostics: &[omnigraph_cluster::Diagnostic]) { for diagnostic in diagnostics { let label = match diagnostic.severity { @@ -784,6 +822,19 @@ fn finish_cluster_plan(output: &PlanOutput, json: bool) -> Result<()> { Ok(()) } +fn finish_cluster_status(output: &StatusOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_status_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + fn is_remote_uri(uri: &str) -> bool { uri.starts_with("http://") || uri.starts_with("https://") } @@ -3217,6 +3268,10 @@ async fn main() -> Result<()> { let output = plan_config_dir(config); finish_cluster_plan(&output, json)?; } + ClusterCommand::Status { config, json } => { + let output = status_config_dir(config); + finish_cluster_status(&output, json)?; + } }, Command::Graphs { command } => match command { GraphsCommand::List { diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 156dd6e..920ceda 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -214,6 +214,168 @@ fn cluster_plan_json_reads_inferred_local_state() { ); } +#[test] +fn cluster_status_json_reports_missing_state() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("status") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["state_observations"]["state_found"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_missing"), + "missing state should be a warning diagnostic: {json}" + ); +} + +#[test] +fn cluster_status_json_reports_extended_state() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#" +{ + "version": 1, + "state_revision": 5, + "applied_revision": { + "config_digest": "applied", + "resources": { + "graph.knowledge": { "digest": "graph-digest" } + } + }, + "resource_statuses": { + "graph.knowledge": { "status": "applied", "conditions": ["healthy"] } + }, + "approval_records": {}, + "recovery_records": {}, + "observations": {} +} +"#, + ) + .unwrap(); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("status") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["state_observations"]["state_revision"], 5); + assert!( + json["state_observations"]["state_cas"] + .as_str() + .unwrap() + .starts_with("sha256:") + ); + assert_eq!(json["resource_digests"]["graph.knowledge"], "graph-digest"); + assert_eq!( + json["resource_statuses"]["graph.knowledge"]["status"], + "applied" + ); +} + +#[test] +fn cluster_plan_json_includes_state_cas_revision_and_lock_observation() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#" +{ + "version": 1, + "state_revision": 9, + "applied_revision": { + "config_digest": "old", + "resources": { + "graph.knowledge": { "digest": "old-graph" } + } + } +} +"#, + ) + .unwrap(); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("plan") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["state_observations"]["state_revision"], 9); + assert!( + json["state_observations"]["state_cas"] + .as_str() + .unwrap() + .starts_with("sha256:") + ); + assert_eq!(json["state_observations"]["locked"], true); + assert!(json["state_observations"]["lock_id"].is_string()); + assert!(!state_dir.join("lock.json").exists()); +} + +#[test] +fn cluster_plan_locked_state_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("lock.json"), + r#" +{ + "version": 1, + "lock_id": "held-lock", + "operation": "plan", + "created_at": "2026-06-08T00:00:00Z", + "pid": 123 +} +"#, + ) + .unwrap(); + + let output = output_failure( + cli() + .arg("cluster") + .arg("plan") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + ); + let json = parse_stdout_json(&output); + assert_eq!(json["ok"], false); + assert_eq!(json["state_observations"]["locked"], true); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_lock_held"), + "locked state should produce a useful diagnostic: {json}" + ); +} + #[test] fn cluster_validate_invalid_config_exits_nonzero() { let temp = tempdir().unwrap(); diff --git a/crates/omnigraph-cluster/Cargo.toml b/crates/omnigraph-cluster/Cargo.toml index 60e7785..d210b1c 100644 --- a/crates/omnigraph-cluster/Cargo.toml +++ b/crates/omnigraph-cluster/Cargo.toml @@ -15,6 +15,8 @@ serde_json = { workspace = true } serde_yaml = { workspace = true } sha2 = { workspace = true } thiserror = { workspace = true } +time = { workspace = true } +ulid = { workspace = true } [dev-dependencies] tempfile = { workspace = true } diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 861ae22..5115933 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -1,6 +1,8 @@ use std::collections::{BTreeMap, BTreeSet}; -use std::fs; +use std::fs::{self, OpenOptions}; +use std::io::{ErrorKind, Write}; use std::path::{Path, PathBuf}; +use std::process; use omnigraph_compiler::build_catalog; use omnigraph_compiler::query::parser::parse_query; @@ -8,11 +10,16 @@ use omnigraph_compiler::query::typecheck::typecheck_query_decl; use omnigraph_compiler::schema::parser::parse_schema; use serde::{Deserialize, Serialize}; use sha2::{Digest, Sha256}; +use time::OffsetDateTime; +use time::format_description::well_known::Rfc3339; +use ulid::Ulid; pub const CLUSTER_CONFIG_FILE: &str = "cluster.yaml"; +pub const CLUSTER_STATE_DIR: &str = "__cluster"; pub const CLUSTER_STATE_FILE: &str = "__cluster/state.json"; +pub const CLUSTER_LOCK_FILE: &str = "__cluster/lock.json"; -#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] #[serde(rename_all = "snake_case")] pub enum DiagnosticSeverity { Error, @@ -86,10 +93,39 @@ pub struct DesiredRevision { #[derive(Debug, Clone, Serialize)] pub struct StateObservations { pub state_path: String, + pub lock_path: String, pub state_found: bool, #[serde(skip_serializing_if = "Option::is_none")] pub applied_config_digest: Option<String>, + pub state_revision: u64, + #[serde(skip_serializing_if = "Option::is_none")] + pub state_cas: Option<String>, pub resource_count: usize, + pub locked: bool, + #[serde(skip_serializing_if = "Option::is_none")] + pub lock_id: Option<String>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum ResourceLifecycleStatus { + Pending, + Planned, + Applying, + Applied, + Drifted, + Blocked, + Error, +} + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +pub struct ResourceStatusRecord { + pub status: ResourceLifecycleStatus, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub conditions: Vec<String>, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub message: Option<String>, } #[derive(Debug, Clone, Serialize, PartialEq, Eq)] @@ -136,15 +172,39 @@ pub struct PlanOutput { pub diagnostics: Vec<Diagnostic>, } +#[derive(Debug, Clone, Serialize)] +pub struct StatusOutput { + pub ok: bool, + pub config_dir: String, + pub state_observations: StateObservations, + pub resource_digests: BTreeMap<String, String>, + pub resource_statuses: BTreeMap<String, ResourceStatusRecord>, + pub diagnostics: Vec<Diagnostic>, +} + #[derive(Debug, Clone)] struct DesiredCluster { config_dir: PathBuf, config_digest: String, + state_lock: bool, resource_digests: BTreeMap<String, String>, resources: Vec<ResourceSummary>, dependencies: Vec<Dependency>, } +#[derive(Debug)] +struct ParsedConfig { + raw: Option<RawClusterConfig>, + diagnostics: Vec<Diagnostic>, + config_dir: PathBuf, + config_file: PathBuf, +} + +#[derive(Debug, Clone, Copy)] +struct ClusterSettings { + state_lock: bool, +} + #[derive(Debug)] struct LoadOutcome { desired: Option<DesiredCluster>, @@ -201,11 +261,22 @@ struct PolicyConfig { applies_to: Vec<String>, } +#[allow(dead_code)] #[derive(Debug, Deserialize)] #[serde(deny_unknown_fields)] struct ClusterState { version: u32, + #[serde(default)] + state_revision: u64, applied_revision: AppliedRevisionState, + #[serde(default)] + resource_statuses: BTreeMap<String, ResourceStatusRecord>, + #[serde(default)] + approval_records: BTreeMap<String, serde_json::Value>, + #[serde(default)] + recovery_records: BTreeMap<String, serde_json::Value>, + #[serde(default)] + observations: BTreeMap<String, serde_json::Value>, } #[derive(Debug, Deserialize)] @@ -223,6 +294,33 @@ struct StateResource { digest: String, } +#[derive(Debug, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +struct StateLockFile { + version: u32, + lock_id: String, + operation: String, + created_at: String, + pid: u32, +} + +#[derive(Debug)] +struct LocalStateBackend { + state_dir: PathBuf, + state_path: PathBuf, + lock_path: PathBuf, +} + +#[derive(Debug)] +struct StateSnapshot { + state: Option<ClusterState>, +} + +#[derive(Debug)] +struct StateLockGuard { + path: PathBuf, +} + pub fn validate_config_dir(config_dir: impl AsRef<Path>) -> ValidateOutput { let outcome = load_desired(config_dir.as_ref()); let (resource_digests, resources, dependencies) = match outcome.desired { @@ -249,13 +347,8 @@ pub fn validate_config_dir(config_dir: impl AsRef<Path>) -> ValidateOutput { pub fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { let outcome = load_desired(config_dir.as_ref()); let mut diagnostics = outcome.diagnostics; - let state_path = outcome.config_dir.join(CLUSTER_STATE_FILE); - let mut observations = StateObservations { - state_path: display_path(&state_path), - state_found: false, - applied_config_digest: None, - resource_count: 0, - }; + let backend = LocalStateBackend::new(&outcome.config_dir); + let mut observations = backend.observations(); let Some(desired) = outcome.desired else { return PlanOutput { @@ -274,40 +367,49 @@ pub fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { }; }; - let mut prior_resources = BTreeMap::new(); - if state_path.exists() { - observations.state_found = true; - match fs::read_to_string(&state_path) { - Ok(text) => match serde_json::from_str::<ClusterState>(&text) { - Ok(state) if state.version == 1 => { - observations.applied_config_digest = state.applied_revision.config_digest; - observations.resource_count = state.applied_revision.resources.len(); - prior_resources = state - .applied_revision - .resources - .into_iter() - .map(|(address, resource)| (address, resource.digest)) - .collect(); - } - Ok(state) => diagnostics.push(Diagnostic::error( - "unsupported_state_version", - "state.version", - format!( - "unsupported cluster state version {}; this build supports version 1", - state.version - ), - )), - Err(err) => diagnostics.push(Diagnostic::error( - "invalid_state_json", - CLUSTER_STATE_FILE, - format!("could not parse state JSON: {err}"), - )), + if has_errors(&diagnostics) { + return PlanOutput { + ok: false, + config_dir: display_path(&desired.config_dir), + desired_revision: DesiredRevision { + config_digest: Some(desired.config_digest), }, - Err(err) => diagnostics.push(Diagnostic::error( - "state_read_error", - CLUSTER_STATE_FILE, - format!("could not read state file: {err}"), - )), + resource_digests: desired.resource_digests, + dependencies: desired.dependencies, + state_observations: observations, + changes: Vec::new(), + blast_radius: Vec::new(), + approvals_required: Vec::new(), + diagnostics, + }; + } + + let _lock_guard = if desired.state_lock { + match backend.acquire_lock("plan", &mut observations) { + Ok(guard) => Some(guard), + Err(diagnostic) => { + diagnostics.push(diagnostic); + None + } + } + } else { + diagnostics.push(Diagnostic::warning( + "state_lock_disabled", + "state.lock", + "state.lock is false; plan read state without acquiring the cluster state lock", + )); + None + }; + + let mut prior_resources = BTreeMap::new(); + if !has_errors(&diagnostics) { + match backend.read_state(&mut observations) { + Ok(snapshot) => { + if let Some(state) = snapshot.state { + prior_resources = state_resource_digests(&state); + } + } + Err(diagnostic) => diagnostics.push(diagnostic), } } @@ -336,7 +438,48 @@ pub fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { } } -fn load_desired(config_dir: &Path) -> LoadOutcome { +pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { + let parsed = parse_cluster_config(config_dir.as_ref()); + let mut diagnostics = parsed.diagnostics; + let backend = LocalStateBackend::new(&parsed.config_dir); + let mut observations = backend.observations(); + backend.observe_lock(&mut observations, &mut diagnostics); + + let mut resource_digests = BTreeMap::new(); + let mut resource_statuses = BTreeMap::new(); + + if let Some(raw) = parsed.raw.as_ref() { + let _settings = validate_cluster_header(raw, &mut diagnostics); + if !has_errors(&diagnostics) { + match backend.read_state(&mut observations) { + Ok(snapshot) => { + if let Some(state) = snapshot.state { + resource_digests = state_resource_digests(&state); + resource_statuses = state.resource_statuses; + } else { + diagnostics.push(Diagnostic::warning( + "state_missing", + CLUSTER_STATE_FILE, + "state.json is missing; no applied cluster revision has been recorded", + )); + } + } + Err(diagnostic) => diagnostics.push(diagnostic), + } + } + } + + StatusOutput { + ok: !has_errors(&diagnostics), + config_dir: display_path(&parsed.config_dir), + state_observations: observations, + resource_digests, + resource_statuses, + diagnostics, + } +} + +fn parse_cluster_config(config_dir: &Path) -> ParsedConfig { let config_dir = config_dir.to_path_buf(); let config_file = config_dir.join(CLUSTER_CONFIG_FILE); let mut diagnostics = Vec::new(); @@ -347,8 +490,8 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { display_path(&config_dir), "`--config` must point at a directory containing cluster.yaml", )); - return LoadOutcome { - desired: None, + return ParsedConfig { + raw: None, diagnostics, config_dir, config_file, @@ -363,8 +506,8 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { CLUSTER_CONFIG_FILE, format!("could not read cluster.yaml: {err}"), )); - return LoadOutcome { - desired: None, + return ParsedConfig { + raw: None, diagnostics, config_dir, config_file, @@ -375,8 +518,8 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { diagnostics.extend(duplicate_key_diagnostics(&text)); diagnostics.extend(future_field_diagnostics(&text)); if has_errors(&diagnostics) { - return LoadOutcome { - desired: None, + return ParsedConfig { + raw: None, diagnostics, config_dir, config_file, @@ -384,22 +527,29 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { } let raw = match serde_yaml::from_str::<RawClusterConfig>(&text) { - Ok(raw) => raw, + Ok(raw) => Some(raw), Err(err) => { diagnostics.push(Diagnostic::error( "invalid_cluster_yaml", CLUSTER_CONFIG_FILE, format!("could not parse cluster.yaml: {err}"), )); - return LoadOutcome { - desired: None, - diagnostics, - config_dir, - config_file, - }; + None } }; + ParsedConfig { + raw, + diagnostics, + config_dir, + config_file, + } +} + +fn validate_cluster_header( + raw: &RawClusterConfig, + diagnostics: &mut Vec<Diagnostic>, +) -> ClusterSettings { if raw.version != 1 { diagnostics.push(Diagnostic::error( "unsupported_cluster_config_version", @@ -424,11 +574,242 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { diagnostics.push(Diagnostic::error( "unsupported_state_backend", "state.backend", - "Stage 1 supports only omitted state.backend or `cluster`", + "Stage 2A supports only omitted state.backend or `cluster`", )); } } - let _lock_parsed_for_forward_compat = raw.state.lock; + + ClusterSettings { + state_lock: raw.state.lock.unwrap_or(true), + } +} + +impl LocalStateBackend { + fn new(config_dir: &Path) -> Self { + let state_dir = config_dir.join(CLUSTER_STATE_DIR); + Self { + state_path: config_dir.join(CLUSTER_STATE_FILE), + lock_path: config_dir.join(CLUSTER_LOCK_FILE), + state_dir, + } + } + + fn observations(&self) -> StateObservations { + StateObservations { + state_path: display_path(&self.state_path), + lock_path: display_path(&self.lock_path), + state_found: false, + applied_config_digest: None, + state_revision: 0, + state_cas: None, + resource_count: 0, + locked: false, + lock_id: None, + } + } + + fn read_state( + &self, + observations: &mut StateObservations, + ) -> Result<StateSnapshot, Diagnostic> { + let text = match fs::read_to_string(&self.state_path) { + Ok(text) => text, + Err(err) if err.kind() == ErrorKind::NotFound => { + return Ok(StateSnapshot { state: None }); + } + Err(err) => { + return Err(Diagnostic::error( + "state_read_error", + CLUSTER_STATE_FILE, + format!("could not read state file: {err}"), + )); + } + }; + + observations.state_found = true; + observations.state_cas = Some(format!("sha256:{}", sha256_hex(text.as_bytes()))); + + let state = serde_json::from_str::<ClusterState>(&text).map_err(|err| { + Diagnostic::error( + "invalid_state_json", + CLUSTER_STATE_FILE, + format!("could not parse state JSON: {err}"), + ) + })?; + + if state.version != 1 { + return Err(Diagnostic::error( + "unsupported_state_version", + "state.version", + format!( + "unsupported cluster state version {}; this build supports version 1", + state.version + ), + )); + } + + observations.applied_config_digest = state.applied_revision.config_digest.clone(); + observations.state_revision = state.state_revision; + observations.resource_count = state.applied_revision.resources.len(); + + Ok(StateSnapshot { state: Some(state) }) + } + + fn acquire_lock( + &self, + operation: &str, + observations: &mut StateObservations, + ) -> Result<StateLockGuard, Diagnostic> { + fs::create_dir_all(&self.state_dir).map_err(|err| { + Diagnostic::error( + "state_lock_error", + CLUSTER_STATE_DIR, + format!("could not create cluster state directory: {err}"), + ) + })?; + + let lock_id = Ulid::new().to_string(); + let lock = StateLockFile { + version: 1, + lock_id: lock_id.clone(), + operation: operation.to_string(), + created_at: OffsetDateTime::now_utc() + .format(&Rfc3339) + .unwrap_or_else(|_| "1970-01-01T00:00:00Z".to_string()), + pid: process::id(), + }; + let payload = serde_json::to_string_pretty(&lock).map_err(|err| { + Diagnostic::error( + "state_lock_error", + CLUSTER_LOCK_FILE, + format!("could not encode state lock: {err}"), + ) + })?; + + match OpenOptions::new() + .write(true) + .create_new(true) + .open(&self.lock_path) + { + Ok(mut file) => { + file.write_all(payload.as_bytes()).map_err(|err| { + Diagnostic::error( + "state_lock_error", + CLUSTER_LOCK_FILE, + format!("could not write state lock: {err}"), + ) + })?; + observations.locked = true; + observations.lock_id = Some(lock_id.clone()); + Ok(StateLockGuard { + path: self.lock_path.clone(), + }) + } + Err(err) if err.kind() == ErrorKind::AlreadyExists => { + self.observe_lock_id(observations); + Err(Diagnostic::error( + "state_lock_held", + CLUSTER_LOCK_FILE, + "cluster state lock already exists; remove it only after confirming no cluster operation is active", + )) + } + Err(err) => Err(Diagnostic::error( + "state_lock_error", + CLUSTER_LOCK_FILE, + format!("could not acquire state lock: {err}"), + )), + } + } + + fn observe_lock( + &self, + observations: &mut StateObservations, + diagnostics: &mut Vec<Diagnostic>, + ) { + if self.lock_path.exists() { + observations.locked = true; + match fs::read_to_string(&self.lock_path) { + Ok(text) => match serde_json::from_str::<StateLockFile>(&text) { + Ok(lock) if lock.version == 1 => { + observations.lock_id = Some(lock.lock_id); + } + Ok(lock) => diagnostics.push(Diagnostic::warning( + "unsupported_state_lock_version", + CLUSTER_LOCK_FILE, + format!("unsupported cluster state lock version {}", lock.version), + )), + Err(err) => diagnostics.push(Diagnostic::warning( + "invalid_state_lock", + CLUSTER_LOCK_FILE, + format!("could not parse state lock: {err}"), + )), + }, + Err(err) => diagnostics.push(Diagnostic::warning( + "state_lock_read_error", + CLUSTER_LOCK_FILE, + format!("could not read state lock: {err}"), + )), + } + } + } + + fn observe_lock_id(&self, observations: &mut StateObservations) { + observations.locked = true; + if let Ok(text) = fs::read_to_string(&self.lock_path) { + if let Ok(lock) = serde_json::from_str::<StateLockFile>(&text) { + if lock.version == 1 { + observations.lock_id = Some(lock.lock_id); + } + } + } + } +} + +impl Drop for StateLockGuard { + fn drop(&mut self) { + let _ = fs::remove_file(&self.path); + } +} + +fn state_resource_digests(state: &ClusterState) -> BTreeMap<String, String> { + state + .applied_revision + .resources + .iter() + .map(|(address, resource)| (address.clone(), resource.digest.clone())) + .collect() +} + +fn load_desired(config_dir: &Path) -> LoadOutcome { + let parsed = parse_cluster_config(config_dir); + let config_dir = parsed.config_dir; + let config_file = parsed.config_file; + let mut diagnostics = parsed.diagnostics; + let Some(raw) = parsed.raw else { + return LoadOutcome { + desired: None, + diagnostics, + config_dir, + config_file, + }; + }; + let settings = validate_cluster_header(&raw, &mut diagnostics); + let config_text = match fs::read_to_string(&config_file) { + Ok(text) => text, + Err(err) => { + diagnostics.push(Diagnostic::error( + "cluster_config_read_error", + CLUSTER_CONFIG_FILE, + format!("could not re-read cluster.yaml: {err}"), + )); + return LoadOutcome { + desired: None, + diagnostics, + config_dir, + config_file, + }; + } + }; let mut resources = BTreeMap::new(); let mut dependencies = BTreeSet::new(); @@ -645,12 +1026,13 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { resource_list.push(resource); } let dependencies: Vec<_> = dependencies.into_iter().collect(); - let config_digest = desired_config_digest(&text, &resource_digests); + let config_digest = desired_config_digest(&config_text, &resource_digests); LoadOutcome { desired: Some(DesiredCluster { config_dir: config_dir.clone(), config_digest, + state_lock: settings.state_lock, resource_digests, resources: resource_list, dependencies, @@ -1217,6 +1599,7 @@ graphs: .all(|c| c.operation == PlanOperation::Create) ); assert!(out.changes.iter().any(|c| c.resource == "graph.knowledge")); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); } #[test] @@ -1260,6 +1643,202 @@ graphs: ); } + #[test] + fn old_minimal_state_json_still_plans_with_default_revision() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{ + "version": 1, + "applied_revision": { + "config_digest": "old", + "resources": { + "graph.knowledge": { "digest": "old-graph" } + } + } +}"#, + ) + .unwrap(); + + let out = plan_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.state_observations.state_revision, 0); + assert!(out.state_observations.state_cas.is_some()); + assert!(out.changes.iter().any(|change| { + change.resource == "graph.knowledge" && change.operation == PlanOperation::Update + })); + } + + #[test] + fn extended_state_json_status_surfaces_statuses() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + let state = r#"{ + "version": 1, + "state_revision": 42, + "applied_revision": { + "config_digest": "applied-config", + "resources": { + "graph.knowledge": { "digest": "graph-digest" } + } + }, + "resource_statuses": { + "graph.knowledge": { + "status": "applied", + "conditions": ["healthy"], + "message": "ready" + } + }, + "approval_records": {}, + "recovery_records": {}, + "observations": { + "graph.knowledge": { "manifest_version": 12 } + } +}"#; + fs::write(state_dir.join("state.json"), state).unwrap(); + + let out = status_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.state_observations.state_found); + assert_eq!(out.state_observations.state_revision, 42); + assert_eq!( + out.state_observations.state_cas.as_deref(), + Some(format!("sha256:{}", sha256_hex(state.as_bytes())).as_str()) + ); + assert_eq!( + out.resource_digests + .get("graph.knowledge") + .map(String::as_str), + Some("graph-digest") + ); + assert_eq!( + out.resource_statuses["graph.knowledge"].status, + ResourceLifecycleStatus::Applied + ); + } + + #[test] + fn missing_state_status_succeeds_with_warning() { + let dir = fixture(); + let out = status_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!out.state_observations.state_found); + assert_eq!(out.state_observations.state_revision, 0); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_missing") + ); + } + + #[test] + fn invalid_state_status_fails() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write(state_dir.join("state.json"), "{").unwrap(); + + let out = status_config_dir(dir.path()); + assert!(!out.ok); + assert!(out.state_observations.state_found); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "invalid_state_json") + ); + } + + #[test] + fn plan_reports_state_cas_revision_and_removes_lock() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + let state = r#"{ + "version": 1, + "state_revision": 7, + "applied_revision": { + "config_digest": "old", + "resources": { + "graph.knowledge": { "digest": "old-graph" } + } + } +}"#; + fs::write(state_dir.join("state.json"), state).unwrap(); + + let out = plan_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.state_observations.state_revision, 7); + assert_eq!( + out.state_observations.state_cas.as_deref(), + Some(format!("sha256:{}", sha256_hex(state.as_bytes())).as_str()) + ); + assert!(out.state_observations.locked); + assert!(out.state_observations.lock_id.is_some()); + assert!( + !dir.path().join(CLUSTER_LOCK_FILE).exists(), + "plan must release lock before returning" + ); + } + + #[test] + fn existing_lock_makes_plan_fail() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("lock.json"), + r#"{ + "version": 1, + "lock_id": "held-lock", + "operation": "plan", + "created_at": "2026-06-08T00:00:00Z", + "pid": 123 +}"#, + ) + .unwrap(); + + let out = plan_config_dir(dir.path()); + assert!(!out.ok); + assert!(out.state_observations.locked); + assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_held") + ); + } + + #[test] + fn state_lock_false_bypasses_lock_with_warning() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +state: + backend: cluster + lock: false +graphs: + knowledge: + schema: ./people.pg +"#, + ) + .unwrap(); + + let out = plan_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!out.state_observations.locked); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_disabled") + ); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + #[test] fn external_state_backend_rejected() { let dir = fixture(); @@ -1272,4 +1851,21 @@ graphs: assert!(!out.ok); assert_eq!(out.diagnostics[0].code, "unsupported_state_backend"); } + + #[test] + fn external_state_backend_plan_rejected() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\nstate:\n backend: s3://bucket/state\ngraphs: {}\n", + ) + .unwrap(); + let out = plan_config_dir(dir.path()); + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "unsupported_state_backend") + ); + } } diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 0b5a234..1035d84 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,7 +8,7 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | -| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests` | Cluster config parser, local JSON state diff, read-only validate/plan | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests` | Cluster config parser, local JSON state diff, state CAS/lock handling, read-only validate/plan/status | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 2f27322..92ad303 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -21,7 +21,7 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` sc | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | | `queries validate \| list` | operate on the server-side stored-query registry (the `queries:` block). `validate` type-checks every stored query against the live schema offline (opens the selected graph; exits non-zero on any breakage), catching schema drift without restarting the server; `list` prints the selected registry's query names, MCP exposure, and typed params. For per-graph registries, pass `--target <graph>` or set `cli.graph`; with no graph selection, `list` shows only top-level `queries:`. Distinct from `lint`, which validates a single `.gq` file | -| `cluster validate \| plan` | read-only cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`. No apply, lock, graph open, server change, or state write occurs in Stage 1 | +| `cluster validate \| plan \| status` | read-only cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json` while briefly holding `__cluster/lock.json`; `status` reads the state ledger. No apply, graph open, live drift scan, server change, or `state.json` mutation occurs in Stage 2A | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns; `--json` reports a `skipped` field) | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | | `embed` | offline JSONL embedding pipeline | @@ -79,13 +79,16 @@ policy: ```bash omnigraph cluster validate --config ./company-brain omnigraph cluster plan --config ./company-brain --json +omnigraph cluster status --config ./company-brain --json ``` `--config` is a directory containing `cluster.yaml`; it defaults to `.`. -Stage 1 accepts graphs, schemas, stored queries, and policy bundle file +Stage 2A accepts graphs, schemas, stored queries, and policy bundle file references. `cluster plan` reads local JSON state from -`<config-dir>/__cluster/state.json`; a missing file means empty state. External -state backends, apply, locks, pipelines, UI specs, embeddings, aliases, and +`<config-dir>/__cluster/state.json`; a missing file means empty state. Plan +acquires `__cluster/lock.json` by default and releases it before returning. +`cluster status` reads state only and reports any existing lock. External state +backends, apply, refresh/import, pipelines, UI specs, embeddings, aliases, and bindings are reserved for later stages. See [cluster-config.md](cluster-config.md). ## Output formats (`query` command, alias: `read`) diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 29d9c32..9fdbf55 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -1,17 +1,19 @@ # Cluster Config -**Status:** Stage 1 read-only preview. +**Status:** Stage 2A read-only preview. Cluster config is the future control-plane configuration surface for a whole OmniGraph deployment. In this stage, OmniGraph can validate a local -`cluster.yaml` folder and produce a deterministic read-only plan. It does not -apply changes, acquire locks, open graph roots, start servers, or write state. +`cluster.yaml` folder, produce a deterministic read-only plan, and inspect the +local JSON state ledger. It does not apply changes, open graph roots, scan live +cluster state, start servers, or write graph resources. ## Commands ```bash omnigraph cluster validate --config ./company-brain omnigraph cluster plan --config ./company-brain --json +omnigraph cluster status --config ./company-brain --json ``` `--config` points at a directory, not a file. The directory must contain @@ -19,7 +21,7 @@ omnigraph cluster plan --config ./company-brain --json ## Supported `cluster.yaml` -Stage 1 accepts only the read-only resource subset: +Stage 2A accepts only the read-only resource subset: ```yaml version: 1 @@ -43,10 +45,12 @@ policies: applies_to: [knowledge] ``` -`metadata.name` is a display label. `state.lock` is parsed for forward -compatibility, but no lock is acquired in this read-only stage. `state.backend` -may be omitted or set to `cluster`; external state backends are reserved for a -later stage. +`metadata.name` is a display label. `state.backend` may be omitted or set to +`cluster`; external state backends are reserved for a later stage. `state.lock` +defaults to `true`. When enabled, `cluster plan` briefly acquires +`<config-dir>/__cluster/lock.json` while it reads state, then removes it before +returning. `cluster status` never acquires the lock; it only reports whether one +is present. ## Validation @@ -78,6 +82,7 @@ resource is planned as a create. If present, the file must use this shape: ```json { "version": 1, + "state_revision": 0, "applied_revision": { "config_digest": "...", "resources": { @@ -86,10 +91,34 @@ resource is planned as a create. If present, the file must use this shape: "query.knowledge.find_experts": { "digest": "..." }, "policy.base": { "digest": "..." } } - } + }, + "resource_statuses": { + "graph.knowledge": { + "status": "applied", + "conditions": [], + "message": "optional status detail" + } + }, + "approval_records": {}, + "recovery_records": {}, + "observations": {} } ``` +`state_revision`, `resource_statuses`, `approval_records`, `recovery_records`, +and `observations` are optional so older Stage 1 state fixtures keep working. +Missing `state_revision` is treated as `0`. Resource status values are +`pending`, `planned`, `applying`, `applied`, `drifted`, `blocked`, or `error`. + Plan output compares desired resource digests against state resource digests -and reports `create`, `update`, and `delete` changes. The command never writes -`state.json`; apply and locking are later-stage work. +and reports `create`, `update`, and `delete` changes. It also reports the state +CAS (`sha256:<digest>`), state revision, and lock id used for the read. The +command never writes `state.json`; apply, refresh, import, and live drift scans +are later-stage work. + +## Status + +`cluster status` reads the same local JSON state ledger and prints what the +ledger says is deployed. It does not validate referenced schema/query/policy +files and does not inspect live graphs. Missing `state.json` succeeds with a +warning; invalid state JSON or an unsupported state version fails. From ce150fb0ca903296cd7f26512293d1b63a4fceec Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Mon, 8 Jun 2026 22:19:21 +0300 Subject: [PATCH 026/165] docs(testing): fix stale optimize test name in maintenance.rs row (#148) The maintenance.rs row referenced `optimize_reconciles_preexisting_manifest_head_drift`, which never existed (leftover from the reconcile-drift heuristic removed in #141). The actual second test is `optimize_defers_when_recovery_sidecar_is_pending`. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- docs/dev/testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/testing.md b/docs/dev/testing.md index f18600b..8974a9f 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -34,7 +34,7 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav | `s3_storage.rs` | S3-backed graph (skipped unless `OMNIGRAPH_S3_TEST_BUCKET` is set) | | `lance_version_columns.rs` | Per-row `_row_last_updated_at_version` behavior | | `validators.rs` | Schema constraint enforcement (enum, range, unique, cardinality) across JSONL, insert, update paths | -| `maintenance.rs` | `optimize` (compaction) + `cleanup` (version GC): empty/idempotent/no-op edges, policy validation, head preservation; `optimize` publishes the compacted version so the manifest tracks the Lance HEAD and a subsequent schema apply succeeds (`optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds`), and reconciles a pre-existing manifest-behind-HEAD drift forged via raw Lance compaction (`optimize_reconciles_preexisting_manifest_head_drift`) | +| `maintenance.rs` | `optimize` (compaction) + `cleanup` (version GC): empty/idempotent/no-op edges, policy validation, head preservation; `optimize` publishes the compacted version so the manifest tracks the Lance HEAD and a subsequent schema apply succeeds (`optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds`), and refuses to run while a `__recovery` sidecar is pending so optimize only ever operates on a recovered graph (`optimize_defers_when_recovery_sidecar_is_pending`) | | `failpoints.rs` | Failure-injection coverage (gated on `failpoints` feature). Includes the five per-writer Phase B → recovery integration tests (`recovery_rolls_forward_after_finalize_publisher_failure`, `schema_apply_phase_b_failure_recovered_on_next_open`, `branch_merge_phase_b_failure_recovered_on_next_open`, `ensure_indices_phase_b_failure_recovered_on_next_open`, `optimize_phase_b_failure_recovered_on_next_open`). | | `recovery.rs` | Open-time recovery sweep — sidecar I/O, classifier dispatch (NoMovement / RolledPastExpected / UnexpectedAtP1 / UnexpectedMultistep / InvariantViolation), all-or-nothing decision, roll-forward via `ManifestBatchPublisher::publish`, roll-back via `Dataset::restore`, audit row in `_graph_commit_recoveries.lance`, `OpenMode::ReadOnly` skip path | | `composite_flow.rs` | Compositional/narrative end-to-end stories — multi-step flows that compose mechanics covered by other test files. Catches integration regressions where individual operations all pass their unit tests but their composition breaks (sequential merges, post-merge main writes, time-travel through merge DAG, reopen consistency over multi-merge histories, post-optimize and post-cleanup strict writes). | From c2a97f4559b1e2c6e048be72630844a64d60d9aa Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Mon, 8 Jun 2026 22:25:33 +0300 Subject: [PATCH 027/165] ci: drop per-PR Windows release build; bind to release tags (#155) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `test_windows_binaries` job ran a full Windows --release build + smoke test on every code PR. It was a non-required (non-blocking) check, so it never gated a merge — it only burned the slowest/most expensive runner (windows-latest, --release, 75-min ceiling) on every code change. Windows binary validation is already covered (better) on release tags: release.yml's `smoke_windows_installer` (on v* tags) builds the release binaries, installs via scripts/install.ps1, and smoke-runs `omnigraph.exe version` + `omnigraph-server.exe --help` — the same smoke test plus the real installer path. Nothing `needs:` the removed job. Trade-off (accepted): a PR that breaks the Windows build or install.ps1 syntax is now caught at release-cut rather than at PR time. install.ps1 and platform-specific code change rarely; the cost savings on every PR outweigh the earlier signal. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- .github/workflows/ci.yml | 57 ---------------------------------------- 1 file changed, 57 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5b7b7b2..bbe5893 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -261,63 +261,6 @@ jobs: if: needs.classify_changes.outputs.run_full_ci == 'true' run: cargo test --locked -p omnigraph-server --features aws - test_windows_binaries: - name: Test Windows release binaries - needs: classify_changes - runs-on: windows-latest - timeout-minutes: 75 - permissions: - contents: read - env: - CARGO_TERM_COLOR: always - steps: - - name: Skip for text-only changes - if: needs.classify_changes.outputs.run_full_ci != 'true' - run: Write-Host "Text-only change detected; skipping Windows binary build." - - - name: Checkout source - if: needs.classify_changes.outputs.run_full_ci == 'true' - uses: actions/checkout@v5.0.1 - - - name: Install system dependencies - if: needs.classify_changes.outputs.run_full_ci == 'true' - run: choco install protoc -y - - - name: Install Rust stable - if: needs.classify_changes.outputs.run_full_ci == 'true' - uses: dtolnay/rust-toolchain@stable - with: - toolchain: stable - - - name: Cache Rust build data - if: needs.classify_changes.outputs.run_full_ci == 'true' - uses: Swatinem/rust-cache@v2 - with: - workspaces: | - . -> target - key: windows-release-binaries - - - name: Build Windows binaries - if: needs.classify_changes.outputs.run_full_ci == 'true' - run: cargo build --release --locked -p omnigraph-cli -p omnigraph-server - - - name: Smoke test Windows binaries - if: needs.classify_changes.outputs.run_full_ci == 'true' - run: | - & ./target/release/omnigraph.exe version - & ./target/release/omnigraph-server.exe --help - - - name: Check PowerShell installer syntax - if: needs.classify_changes.outputs.run_full_ci == 'true' - run: | - $tokens = $null - $errors = $null - [System.Management.Automation.Language.Parser]::ParseFile("scripts/install.ps1", [ref]$tokens, [ref]$errors) | Out-Null - if ($errors.Count -gt 0) { - $errors | Format-List - exit 1 - } - rustfs_integration: name: RustFS S3 Integration needs: From 5eead8d29eb6a4e7dfb453603aa0efd8e6851c47 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Mon, 8 Jun 2026 22:26:04 +0300 Subject: [PATCH 028/165] ci(branch-protection): let code owners bypass required PR review (#154) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit require_code_owner_reviews + count=1 with no bypass meant EVERY PR needed a code-owner approval — including code owners' own PRs, which can't be self-approved, so an owner's PR deadlocked on the other owner (forcing admin overrides). Intended behavior: review is required only for non-owners. Add bypass_pull_request_allowances for the two engineering owners (ragnorc, aaltshuler): they merge their own PRs after CI without a second review; non-owners still require a code-owner approval. CI status checks remain required for everyone. Applied live via scripts/apply-branch-protection.sh. Note: the bypass list mirrors codeowners-roles.yml engineering members by hand (render-codeowners.py doesn't generate it) — keep in sync on owner changes. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- .github/branch-protection.json | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/.github/branch-protection.json b/.github/branch-protection.json index 7ca46b9..c039e32 100644 --- a/.github/branch-protection.json +++ b/.github/branch-protection.json @@ -1,5 +1,5 @@ { - "_comment": "Branch protection policy for main. Applied via scripts/apply-branch-protection.sh. See docs/branch-protection.md for rationale.", + "_comment": "Branch protection policy for main. Applied via scripts/apply-branch-protection.sh. See docs/branch-protection.md for rationale. NOTE: bypass_pull_request_allowances.users must mirror the engineering owners in .github/codeowners-roles.yml — code owners merge their own PRs without a second review; non-owners still need a code-owner approval. (render-codeowners.py does NOT generate this list; keep it in sync by hand.)", "required_status_checks": { "strict": true, "contexts": [ @@ -17,7 +17,12 @@ "dismiss_stale_reviews": true, "require_code_owner_reviews": true, "required_approving_review_count": 1, - "require_last_push_approval": false + "require_last_push_approval": false, + "bypass_pull_request_allowances": { + "users": ["ragnorc", "aaltshuler"], + "teams": [], + "apps": [] + } }, "restrictions": null, "required_linear_history": true, From d0e39e677e3ba77d8a74f5a52f40244aa2d25787 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford <ragnor.comerford@gmail.com> Date: Tue, 9 Jun 2026 14:42:54 +0200 Subject: [PATCH 029/165] fix(maintenance): route uncovered drift through repair (#156) * docs(invariants): note the non-atomic manifest->commit-graph publish gap Every graph publish commits __manifest then appends _graph_commits as two separate writes; a crash between them leaves the manifest ahead of the commit DAG. Live reads + durability are unaffected (reads resolve via the manifest) and recovery does not repair it; impact is bounded to commit history / time-travel by commit id / merge-base completeness. Pre-existing across all publishes, not the optimize reconcile specifically. Documented as a Known Gap; the fix is a commit-graph reconcilable from the manifest, not a recovery sidecar. * fix(maintenance): route uncovered drift through repair * fix(maintenance): harden repair review feedback --- AGENTS.md | 9 +- crates/omnigraph-cli/src/main.rs | 104 ++++++ crates/omnigraph-cli/tests/cli.rs | 97 +++++ crates/omnigraph/src/db/mod.rs | 5 +- crates/omnigraph/src/db/omnigraph.rs | 21 ++ crates/omnigraph/src/db/omnigraph/optimize.rs | 79 +++- crates/omnigraph/src/db/omnigraph/repair.rs | 332 +++++++++++++++++ crates/omnigraph/src/exec/mutation.rs | 3 +- crates/omnigraph/src/exec/staging.rs | 55 ++- .../omnigraph/tests/lance_surface_guards.rs | 33 +- crates/omnigraph/tests/maintenance.rs | 345 +++++++++++++++++- crates/omnigraph/tests/writes.rs | 77 ++-- docs/dev/invariants.md | 14 + docs/dev/testing.md | 4 +- docs/user/cli-reference.md | 6 +- docs/user/maintenance.md | 17 +- 16 files changed, 1108 insertions(+), 93 deletions(-) create mode 100644 crates/omnigraph/src/db/omnigraph/repair.rs diff --git a/AGENTS.md b/AGENTS.md index 3f5b711..69272f8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -214,8 +214,12 @@ omnigraph schema apply --schema ./next.pg s3://my-bucket/graph.omni --json # Merge review branch back omnigraph branch merge review/2026-04-25 --into main s3://my-bucket/graph.omni -# Compact + GC (preview, then confirm) +# Compact, preview any uncovered drift, then repair/GC after review omnigraph optimize s3://my-bucket/graph.omni +omnigraph repair s3://my-bucket/graph.omni +omnigraph repair --confirm s3://my-bucket/graph.omni +# For suspicious/unverifiable drift only after deliberate review: +# omnigraph repair --force --confirm s3://my-bucket/graph.omni omnigraph cleanup --keep 10 --older-than 7d s3://my-bucket/graph.omni omnigraph cleanup --keep 10 --older-than 7d --confirm s3://my-bucket/graph.omni @@ -237,7 +241,8 @@ omnigraph policy explain --actor act-alice --action change --branch main | Per-dataset versioning + time travel | ✅ | `snapshot_at_version`, `entity_at`, snapshot-pinned reads across many tables | | Per-dataset branches | ✅ | **Graph-level** branches (atomic across all sub-tables), lazy fork, system branch filtering | | Atomic single-dataset commits | ✅ | **Multi-table publish via three layers**, NOT a single Lance primitive: (1) per-table Lance `commit_staged` for the data write, (2) `__manifest` row-level CAS via `ManifestBatchPublisher` for cross-table ordering, (3) the open-time recovery sweep for the residual gap between (1) and (2). All three layers ship; the five migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`, `optimize_all_tables`) write a `__recovery/{ulid}.json` sidecar before Phase B and delete it after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the sweep in `db/manifest/recovery.rs`: classify, decide all-or-nothing per sidecar, roll forward via single `ManifestBatchPublisher::publish` or roll back via `Dataset::restore` followed by a manifest publish of the restored version (so both directions converge to `manifest == HEAD` — no residual drift), and record an audit row in `_graph_commit_recoveries.lance` (queryable via `omnigraph commit list --filter actor=omnigraph:recovery`). Continuous in-process recovery (no restart needed between Phase B failure and recovery) is the goal of a future background reconciler. Engine writes route through a sealed `TableStorage` trait exposing `stage_*` + `commit_staged` as the canonical staged-write surface; documented inline-commit residuals (`delete_where`, `create_vector_index`, plus legacy `append_batch` / `merge_insert_batches` / `overwrite_batch` / `create_*_index`) remain on the trait until upstream Lance ships a public two-phase API ([#6658](https://github.com/lance-format/lance/issues/6658), [#6666](https://github.com/lance-format/lance/issues/6666)) and the migration of every call site completes. | -| Compaction (`compact_files`) | ✅ | `omnigraph optimize` orchestrates over all node/edge tables, bounded concurrency; **publishes each compacted table's new version to `__manifest`** (so the manifest tracks the Lance HEAD — required for reads to observe compaction and for schema apply / strict writes to pass their HEAD-vs-manifest precondition), under the per-`(table, main)` write queue with `SidecarKind::Optimize` recovery coverage; **refuses on an unrecovered graph** (errors if a `__recovery` sidecar is pending — recovery may roll back a partial write, so optimize requires `manifest == HEAD` going in); **skips blob-bearing tables** (reported via `TableOptimizeStats.skipped`, not silent), gated on `LANCE_SUPPORTS_BLOB_COMPACTION` until the upstream blob-v2 compaction-decode bug is fixed (see [docs/dev/invariants.md](docs/dev/invariants.md) Known Gaps) | +| Compaction (`compact_files`) | ✅ | `omnigraph optimize` orchestrates over all node/edge tables, bounded concurrency; **publishes each compacted table's new version to `__manifest`** (so the manifest tracks the Lance HEAD — required for reads to observe compaction and for schema apply / strict writes to pass their HEAD-vs-manifest precondition), under the per-`(table, main)` write queue with `SidecarKind::Optimize` recovery coverage; **refuses on an unrecovered graph** (errors if a `__recovery` sidecar is pending); **skips uncovered HEAD > manifest drift** with `DriftNeedsRepair` instead of interpreting it; **skips blob-bearing tables** (reported via `TableOptimizeStats.skipped`, not silent), gated on `LANCE_SUPPORTS_BLOB_COMPACTION` until the upstream blob-v2 compaction-decode bug is fixed (see [docs/dev/invariants.md](docs/dev/invariants.md) Known Gaps) | +| Repair uncovered drift | — | `omnigraph repair` explicitly classifies uncovered table `HEAD > manifest` drift: verified maintenance drift (`ReserveFragments`/`Rewrite`) can be published with `--confirm`; suspicious or unverifiable drift requires `--force --confirm`. Sidecar-covered crash residuals still recover automatically on open. | | Cleanup (`cleanup_old_versions`) | ✅ | `omnigraph cleanup` with `--keep` / `--older-than` policy | | BTREE / inverted (FTS) / vector indexes | ✅ | `ensure_indices` builds them on every relevant column; idempotent; lazy across branches | | `merge_insert` upsert | ✅ | `LoadMode::Merge`, mutation `update`/`insert`/`delete` lowering | diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 29b55c4..fec75f1 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -283,6 +283,25 @@ enum Command { #[arg(long)] json: bool, }, + /// Classify and explicitly repair manifest/head drift + Repair { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + /// Publish verified maintenance drift. Without this flag, repair only + /// previews what it would do. + #[arg(long)] + confirm: bool, + /// Also publish suspicious or unverifiable drift. Requires + /// `--confirm`; use only after operator review. + #[arg(long, requires = "confirm")] + force: bool, + #[arg(long)] + json: bool, + }, /// Remove old Lance versions from every table of the graph (destructive) Cleanup { /// Graph URI @@ -3012,6 +3031,8 @@ async fn main() -> Result<()> { "fragments_added": s.fragments_added, "committed": s.committed, "skipped": s.skipped.map(|r| r.as_str()), + "manifest_version": s.manifest_version, + "lance_head_version": s.lance_head_version, })).collect::<Vec<_>>(), }); print_json(&value)?; @@ -3031,6 +3052,89 @@ async fn main() -> Result<()> { } } } + Command::Repair { + uri, + target, + config, + confirm, + force, + json, + } => { + let config = load_cli_config(config.as_ref())?; + let uri = resolve_uri(&config, uri, target.as_deref())?; + let db = Omnigraph::open(&uri).await?; + let stats = db + .repair(omnigraph::db::RepairOptions { confirm, force }) + .await?; + let refused_count = stats + .tables + .iter() + .filter(|s| matches!(s.action, omnigraph::db::RepairAction::Refused)) + .count(); + if json { + let value = serde_json::json!({ + "uri": uri, + "confirm": confirm, + "force": force, + "manifest_version": stats.manifest_version, + "tables": stats.tables.iter().map(|s| serde_json::json!({ + "table_key": s.table_key, + "manifest_version": s.manifest_version, + "lance_head_version": s.lance_head_version, + "classification": s.classification.as_str(), + "action": s.action.as_str(), + "operations": s.operations, + "error": s.error, + })).collect::<Vec<_>>(), + }); + print_json(&value)?; + } else { + let mode = if confirm { "confirm" } else { "preview" }; + println!( + "repair {} — {} mode, {} tables", + uri, + mode, + stats.tables.len() + ); + for s in &stats.tables { + let drift = if s.manifest_version == s.lance_head_version { + format!("{}", s.manifest_version) + } else { + format!("{} → {}", s.manifest_version, s.lance_head_version) + }; + let ops = if s.operations.is_empty() { + String::new() + } else { + format!(" [{}]", s.operations.join(", ")) + }; + let err = s + .error + .as_ref() + .map(|err| format!(" ({err})")) + .unwrap_or_default(); + println!( + " {:<40} {:<12} {:<22} {}{}{}", + s.table_key, + s.action.as_str(), + s.classification.as_str(), + drift, + ops, + err + ); + } + if !confirm { + println!("rerun with --confirm to publish verified maintenance drift"); + } + } + if refused_count > 0 { + bail!( + "repair refused {} suspicious or unverifiable table(s); review the preview \ + output and rerun with --force --confirm only if publishing that drift is \ + intentional", + refused_count + ); + } + } Command::Cleanup { uri, target, diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 9682d9a..26a1a65 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -1,5 +1,6 @@ use std::fs; +use lance::Dataset; use lance::index::DatasetIndexExt; use omnigraph::db::{Omnigraph, ReadTarget}; use serde_json::Value; @@ -60,6 +61,25 @@ fn manifest_dataset_version(graph: &std::path::Path) -> u64 { }) } +fn forge_person_delete_drift(graph: &std::path::Path) -> (u64, u64) { + tokio::runtime::Runtime::new().unwrap().block_on(async { + let uri = graph.to_string_lossy(); + let db = Omnigraph::open(uri.as_ref()).await.unwrap(); + let snap = db + .snapshot_of(ReadTarget::branch("main")) + .await + .unwrap(); + let entry = snap.entry("node:Person").unwrap(); + let full_path = format!("{}/{}", uri.trim_end_matches('/'), entry.table_path); + let mut ds = Dataset::open(&full_path).await.unwrap(); + let deleted = ds.delete("name = 'Alice'").await.unwrap(); + assert_eq!(deleted.num_deleted_rows, 1); + let head = deleted.new_dataset.version().version; + assert!(head > entry.table_version); + (entry.table_version, head) + }) +} + fn write_policy_config_fixture(root: &std::path::Path) -> (std::path::PathBuf, std::path::PathBuf) { let config = root.join("omnigraph.yaml"); let policy = root.join("policy.yaml"); @@ -235,6 +255,83 @@ fn init_creates_graph_successfully_on_missing_local_directory() { assert!(temp.path().join("omnigraph.yaml").exists()); } +#[test] +fn repair_json_reports_noop_on_clean_graph() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + let output = output_success(cli().arg("repair").arg("--json").arg(&graph)); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["confirm"], false); + assert_eq!(payload["force"], false); + assert_eq!(payload["manifest_version"], Value::Null); + let tables = payload["tables"].as_array().unwrap(); + assert_eq!(tables.len(), 4); + assert!(tables.iter().all(|table| { + table["classification"] == "no_drift" && table["action"] == "no_op" + })); +} + +#[test] +fn repair_confirm_json_refuses_suspicious_drift_with_nonzero_exit_then_force_succeeds() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + let graph_manifest_before = manifest_dataset_version(&graph); + let (table_manifest_before, table_head_before) = forge_person_delete_drift(&graph); + + let refused = output_failure( + cli() + .arg("repair") + .arg("--confirm") + .arg("--json") + .arg(&graph), + ); + let refused_payload: Value = serde_json::from_slice(&refused.stdout).unwrap(); + assert_eq!(refused_payload["manifest_version"], Value::Null); + let person = refused_payload["tables"] + .as_array() + .unwrap() + .iter() + .find(|table| table["table_key"] == "node:Person") + .unwrap(); + assert_eq!(person["classification"], "suspicious"); + assert_eq!(person["action"], "refused"); + assert!( + String::from_utf8_lossy(&refused.stderr).contains("repair refused"), + "stderr should explain the non-zero exit; got: {}", + String::from_utf8_lossy(&refused.stderr) + ); + assert_eq!(manifest_dataset_version(&graph), graph_manifest_before); + + let forced = output_success( + cli() + .arg("repair") + .arg("--force") + .arg("--confirm") + .arg("--json") + .arg(&graph), + ); + let forced_payload: Value = serde_json::from_slice(&forced.stdout).unwrap(); + let forced_manifest = forced_payload["manifest_version"].as_u64().unwrap(); + assert!(forced_manifest > graph_manifest_before); + let person = forced_payload["tables"] + .as_array() + .unwrap() + .iter() + .find(|table| table["table_key"] == "node:Person") + .unwrap(); + assert_eq!(person["classification"], "suspicious"); + assert_eq!(person["action"], "forced"); + assert_eq!(person["manifest_version"], table_manifest_before); + assert_eq!(person["lance_head_version"], table_head_before); + assert_eq!(manifest_dataset_version(&graph), forced_manifest); +} + #[test] fn schema_plan_json_reports_supported_additive_change() { let temp = tempdir().unwrap(); diff --git a/crates/omnigraph/src/db/mod.rs b/crates/omnigraph/src/db/mod.rs index 13e1c74..000602a 100644 --- a/crates/omnigraph/src/db/mod.rs +++ b/crates/omnigraph/src/db/mod.rs @@ -11,8 +11,9 @@ pub use graph_coordinator::{GraphCoordinator, ReadTarget, ResolvedTarget, Snapsh pub use manifest::{Snapshot, SubTableEntry, SubTableUpdate}; pub(crate) use omnigraph::ensure_public_branch_ref; pub use omnigraph::{ - CleanupPolicyOptions, InitOptions, MergeOutcome, Omnigraph, OpenMode, SchemaApplyOptions, - SchemaApplyResult, SkipReason, TableCleanupStats, TableOptimizeStats, + CleanupPolicyOptions, InitOptions, MergeOutcome, Omnigraph, OpenMode, RepairAction, + RepairClassification, RepairOptions, RepairStats, SchemaApplyOptions, SchemaApplyResult, + SkipReason, TableCleanupStats, TableOptimizeStats, TableRepairStats, }; pub(crate) const SCHEMA_APPLY_LOCK_BRANCH: &str = "__schema_apply_lock__"; diff --git a/crates/omnigraph/src/db/omnigraph.rs b/crates/omnigraph/src/db/omnigraph.rs index ba2b70e..5bcc973 100644 --- a/crates/omnigraph/src/db/omnigraph.rs +++ b/crates/omnigraph/src/db/omnigraph.rs @@ -30,10 +30,14 @@ use crate::table_store::TableStore; mod export; mod optimize; +mod repair; mod schema_apply; mod table_ops; pub use optimize::{CleanupPolicyOptions, SkipReason, TableCleanupStats, TableOptimizeStats}; +pub use repair::{ + RepairAction, RepairClassification, RepairOptions, RepairStats, TableRepairStats, +}; pub use schema_apply::SchemaApplyOptions; use super::commit_graph::GraphCommit; @@ -682,6 +686,16 @@ impl Omnigraph { .map(|resolved| resolved.snapshot) } + pub(crate) async fn fresh_snapshot_for_branch(&self, branch: Option<&str>) -> Result<Snapshot> { + self.ensure_schema_state_valid().await?; + let requested = ReadTarget::Branch(branch.unwrap_or("main").to_string()); + let coord = self.coordinator.read().await; + coord + .resolve_target(&requested) + .await + .map(|resolved| resolved.snapshot) + } + pub(crate) async fn version(&self) -> u64 { self.coordinator.read().await.version() } @@ -999,6 +1013,13 @@ impl Omnigraph { optimize::optimize_all_tables(self).await } + /// Classify and explicitly repair uncovered manifest/head drift. See + /// [`repair`] for the distinction between safe maintenance drift and + /// suspicious/unverifiable drift. + pub async fn repair(&self, options: repair::RepairOptions) -> Result<repair::RepairStats> { + repair::repair_all_tables(self, options).await + } + /// Remove Lance manifests (and the fragments they uniquely own) per the /// given [`optimize::CleanupPolicyOptions`]. Destructive to version /// history. See [`optimize`] for details. diff --git a/crates/omnigraph/src/db/omnigraph/optimize.rs b/crates/omnigraph/src/db/omnigraph/optimize.rs index ee39323..3c37b66 100644 --- a/crates/omnigraph/src/db/omnigraph/optimize.rs +++ b/crates/omnigraph/src/db/omnigraph/optimize.rs @@ -75,8 +75,7 @@ pub struct CleanupPolicyOptions { } /// Why `optimize` did not compact a table. Typed so callers branch on the -/// reason rather than sniffing a string. One variant today, gated by -/// [`LANCE_SUPPORTS_BLOB_COMPACTION`]. +/// reason rather than sniffing a string. #[derive(Debug, Clone, Copy, PartialEq, Eq)] #[non_exhaustive] pub enum SkipReason { @@ -84,6 +83,12 @@ pub enum SkipReason { /// `BlobHandling::AllBinary`, which mis-decodes blob-v2 columns; see /// [`LANCE_SUPPORTS_BLOB_COMPACTION`] and `docs/dev/lance.md`. BlobColumnsUnsupportedByLance, + /// The Lance dataset HEAD is ahead of the version recorded in + /// `__manifest`, and no recovery sidecar covers that movement. `optimize` + /// cannot infer whether the drift is benign maintenance or an external + /// semantic write, so it leaves the table untouched and points operators at + /// explicit `repair`. + DriftNeedsRepair, } impl SkipReason { @@ -92,6 +97,7 @@ impl SkipReason { pub fn as_str(&self) -> &'static str { match self { SkipReason::BlobColumnsUnsupportedByLance => "blob_columns_unsupported_by_lance", + SkipReason::DriftNeedsRepair => "drift_needs_repair", } } } @@ -103,6 +109,7 @@ impl std::fmt::Display for SkipReason { SkipReason::BlobColumnsUnsupportedByLance => { "blob columns — Lance compaction unsupported" } + SkipReason::DriftNeedsRepair => "manifest/head drift — run omnigraph repair", }; f.write_str(msg) } @@ -125,6 +132,12 @@ pub struct TableOptimizeStats { /// `Some(reason)` if this table was deliberately not compacted. When set, /// `fragments_removed == 0`, `fragments_added == 0`, and `!committed`. pub skipped: Option<SkipReason>, + /// Manifest table version observed by optimize for drift skips. `None` for + /// normal compaction/no-op/blob skips. + pub manifest_version: Option<u64>, + /// Lance HEAD version observed by optimize for drift skips. `None` for + /// normal compaction/no-op/blob skips. + pub lance_head_version: Option<u64>, } impl TableOptimizeStats { @@ -136,6 +149,8 @@ impl TableOptimizeStats { fragments_added: metrics.fragments_added, committed, skipped: None, + manifest_version: None, + lance_head_version: None, } } @@ -147,6 +162,25 @@ impl TableOptimizeStats { fragments_added: 0, committed: false, skipped: Some(reason), + manifest_version: None, + lance_head_version: None, + } + } + + /// Stat for a table skipped because the manifest and Lance HEAD disagree. + fn skipped_for_drift( + table_key: String, + manifest_version: u64, + lance_head_version: u64, + ) -> Self { + Self { + table_key, + fragments_removed: 0, + fragments_added: 0, + committed: false, + skipped: Some(SkipReason::DriftNeedsRepair), + manifest_version: Some(manifest_version), + lance_head_version: Some(lance_head_version), } } } @@ -185,8 +219,7 @@ pub async fn optimize_all_tables(db: &Omnigraph) -> Result<Vec<TableOptimizeStat )); } - let resolved = db.resolved_branch_target(None).await?; - let snapshot = resolved.snapshot; + let snapshot = db.fresh_snapshot_for_branch(None).await?; // Compute per-table state (path + whether it has blob columns) up front, in // a scope that drops the catalog handle before the async stream starts. @@ -258,7 +291,8 @@ async fn optimize_one_table( ) -> Result<TableOptimizeStats> { // Lance `compact_files` mis-decodes blob-v2 columns under the forced // `BlobHandling::AllBinary` read (see LANCE_SUPPORTS_BLOB_COMPACTION). Skip - // blob-bearing tables and report it rather than aborting the whole sweep. + // blob-bearing tables before acquiring the write queue; `repair` is the + // operator tool for full manifest/head drift classification. if has_blob && !LANCE_SUPPORTS_BLOB_COMPACTION { tracing::warn!( target: "omnigraph::optimize", @@ -291,20 +325,41 @@ async fn optimize_one_table( // CAS baseline: the table's current manifest version, read under the queue // (in-memory coordinator snapshot, no storage I/O — stable for this section). let expected_version = db - .snapshot() - .await + .fresh_snapshot_for_branch(None) + .await? .entry(&table_key) .map(|e| e.table_version) .ok_or_else(|| OmniError::manifest(format!("no manifest entry for {}", table_key)))?; + let lance_head_version = ds.version().version; + if lance_head_version < expected_version { + return Err(OmniError::manifest_internal(format!( + "table '{}' Lance HEAD version {} is behind manifest version {}", + table_key, lance_head_version, expected_version + ))); + } + if lance_head_version > expected_version { + tracing::warn!( + target: "omnigraph::optimize", + table = %table_key, + manifest_version = expected_version, + lance_head_version, + "skipping compaction: Lance HEAD is ahead of the manifest; run `omnigraph repair` \ + to classify and publish covered maintenance drift explicitly", + ); + return Ok(TableOptimizeStats::skipped_for_drift( + table_key, + expected_version, + lance_head_version, + )); + } + // Precise "will it compact?" check — `plan_compaction` also accounts for // deletion materialization (which can rewrite even a single fragment). A // steady-state already-compacted table yields an empty plan and is never // pinned in a sidecar (a zero-commit pin would classify NoMovement on - // recovery and force an all-or-nothing rollback). There is no drift to - // reconcile here: optimize runs only on a recovered graph (the pending- - // sidecar guard above), and recovery roll-back now publishes, so - // `HEAD == manifest` holds going in. + // recovery and force an all-or-nothing rollback). Uncovered pre-existing + // drift is skipped above and must go through explicit repair. let options = CompactionOptions::default(); let plan = plan_compaction(&ds, &options) .await @@ -641,7 +696,7 @@ fn orphan_branches(present: Vec<String>, keep: &std::collections::HashSet<String orphans } -fn all_table_keys(catalog: &omnigraph_compiler::catalog::Catalog) -> Vec<String> { +pub(super) fn all_table_keys(catalog: &omnigraph_compiler::catalog::Catalog) -> Vec<String> { let mut keys: Vec<String> = catalog .node_types .keys() diff --git a/crates/omnigraph/src/db/omnigraph/repair.rs b/crates/omnigraph/src/db/omnigraph/repair.rs new file mode 100644 index 0000000..aaef2ba --- /dev/null +++ b/crates/omnigraph/src/db/omnigraph/repair.rs @@ -0,0 +1,332 @@ +//! Explicit repair for uncovered manifest/head drift. +//! +//! Recovery sidecars handle deterministic crash residuals automatically. This +//! module is for the different case: a table's Lance HEAD is ahead of the +//! version recorded in `__manifest` and there is no sidecar encoding writer +//! intent. `repair` classifies that uncovered drift from Lance transactions and +//! only auto-publishes maintenance-only drift when the operator confirms. + +use std::collections::HashMap; + +use lance::Dataset; +use lance::dataset::transaction::Operation; + +use super::*; + +/// Options for [`Omnigraph::repair`]. +#[derive(Debug, Clone, Copy, Default)] +pub struct RepairOptions { + /// Preview by default. With `confirm`, verified maintenance drift is + /// published to `__manifest`. + pub confirm: bool, + /// Also publish suspicious/unverifiable drift. Requires `confirm`. + pub force: bool, +} + +/// Classification of a table's manifest/head state. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[non_exhaustive] +pub enum RepairClassification { + /// Lance HEAD equals the manifest pin. + NoDrift, + /// Every uncovered Lance transaction is maintenance-only (`Rewrite` or + /// `ReserveFragments`), so publishing the HEAD is content-preserving. + VerifiedMaintenance, + /// At least one uncovered transaction is semantic (`Append`, `Delete`, + /// `Update`, etc.). + Suspicious, + /// A needed transaction could not be read, so the drift cannot be judged. + Unverifiable, +} + +impl RepairClassification { + /// Stable machine-readable token for serialized output. + pub fn as_str(&self) -> &'static str { + match self { + Self::NoDrift => "no_drift", + Self::VerifiedMaintenance => "verified_maintenance", + Self::Suspicious => "suspicious", + Self::Unverifiable => "unverifiable", + } + } +} + +impl std::fmt::Display for RepairClassification { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(self.as_str()) + } +} + +/// What repair did for a table. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[non_exhaustive] +pub enum RepairAction { + /// Nothing to do. + NoOp, + /// Drift was reported but not published because this was a preview. + Preview, + /// Verified maintenance drift was published to `__manifest`. + Healed, + /// Suspicious/unverifiable drift was published because `force` was set. + Forced, + /// Drift was left untouched because it was not safe to publish without + /// `force`. + Refused, +} + +impl RepairAction { + /// Stable machine-readable token for serialized output. + pub fn as_str(&self) -> &'static str { + match self { + Self::NoOp => "no_op", + Self::Preview => "preview", + Self::Healed => "healed", + Self::Forced => "forced", + Self::Refused => "refused", + } + } +} + +impl std::fmt::Display for RepairAction { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(self.as_str()) + } +} + +/// Per-table repair outcome. +#[derive(Debug, Clone)] +#[non_exhaustive] +pub struct TableRepairStats { + pub table_key: String, + pub manifest_version: u64, + pub lance_head_version: u64, + pub classification: RepairClassification, + pub action: RepairAction, + pub operations: Vec<String>, + pub error: Option<String>, +} + +/// Whole-graph repair outcome. +#[derive(Debug, Clone)] +#[non_exhaustive] +pub struct RepairStats { + pub tables: Vec<TableRepairStats>, + /// New graph manifest version if repair published any table pins. + pub manifest_version: Option<u64>, +} + +struct ClassificationResult { + classification: RepairClassification, + operations: Vec<String>, + error: Option<String>, +} + +pub async fn repair_all_tables(db: &Omnigraph, options: RepairOptions) -> Result<RepairStats> { + if options.force && !options.confirm { + return Err(OmniError::manifest("repair --force requires --confirm")); + } + + db.ensure_schema_state_valid().await?; + db.ensure_schema_apply_idle("repair").await?; + ensure_no_pending_recovery_sidecars(db, "repair").await?; + + let snapshot = db.fresh_snapshot_for_branch(None).await?; + let table_tasks: Vec<(String, String)> = { + let catalog = db.catalog(); + let mut tasks = Vec::new(); + for table_key in optimize::all_table_keys(&catalog) { + let Some(entry) = snapshot.entry(&table_key) else { + continue; + }; + let full_path = format!("{}/{}", db.root_uri, entry.table_path); + tasks.push((table_key, full_path)); + } + tasks + }; + + if table_tasks.is_empty() { + return Ok(RepairStats { + tables: Vec::new(), + manifest_version: None, + }); + } + + let queue_keys: Vec<(String, Option<String>)> = table_tasks + .iter() + .map(|(table_key, _)| (table_key.clone(), None)) + .collect(); + let _guards = db.write_queue().acquire_many(&queue_keys).await; + ensure_no_pending_recovery_sidecars(db, "repair").await?; + + let snapshot = db.fresh_snapshot_for_branch(None).await?; + let mut tables = Vec::with_capacity(table_tasks.len()); + let mut updates = Vec::new(); + let mut expected = HashMap::new(); + let mut any_forced = false; + + for (table_key, full_path) in table_tasks { + let ds = db + .table_store + .open_dataset_head_for_write(&table_key, &full_path, None) + .await?; + let manifest_version = snapshot + .entry(&table_key) + .map(|e| e.table_version) + .ok_or_else(|| OmniError::manifest(format!("no manifest entry for {}", table_key)))?; + let lance_head_version = ds.version().version; + + if lance_head_version < manifest_version { + return Err(OmniError::manifest_internal(format!( + "table '{}' Lance HEAD version {} is behind manifest version {}", + table_key, lance_head_version, manifest_version + ))); + } + + if lance_head_version == manifest_version { + tables.push(TableRepairStats { + table_key, + manifest_version, + lance_head_version, + classification: RepairClassification::NoDrift, + action: RepairAction::NoOp, + operations: Vec::new(), + error: None, + }); + continue; + } + + let classification = classify_drift(&ds, manifest_version, lance_head_version).await; + let action = match ( + options.confirm, + options.force, + classification.classification, + ) { + (false, _, _) => RepairAction::Preview, + (true, _, RepairClassification::VerifiedMaintenance) => RepairAction::Healed, + (true, true, RepairClassification::Suspicious | RepairClassification::Unverifiable) => { + any_forced = true; + RepairAction::Forced + } + (true, _, RepairClassification::Suspicious | RepairClassification::Unverifiable) => { + RepairAction::Refused + } + (true, _, RepairClassification::NoDrift) => RepairAction::NoOp, + }; + + if matches!(action, RepairAction::Healed | RepairAction::Forced) { + let state = db.table_store.table_state(&full_path, &ds).await?; + updates.push(crate::db::SubTableUpdate { + table_key: table_key.clone(), + table_version: state.version, + table_branch: None, + row_count: state.row_count, + version_metadata: state.version_metadata, + }); + expected.insert(table_key.clone(), manifest_version); + } + + tables.push(TableRepairStats { + table_key, + manifest_version, + lance_head_version, + classification: classification.classification, + action, + operations: classification.operations, + error: classification.error, + }); + } + + let manifest_version = if updates.is_empty() { + None + } else { + let actor = if any_forced { + Some("omnigraph:repair:force") + } else { + Some("omnigraph:repair") + }; + let PublishedSnapshot { + manifest_version, + _snapshot_id: _, + } = db + .coordinator + .write() + .await + .commit_updates_with_actor_with_expected(&updates, &expected, actor) + .await?; + db.runtime_cache.invalidate_all().await; + if updates + .iter() + .any(|update| update.table_key.starts_with("edge:")) + { + db.invalidate_graph_index().await; + } + Some(manifest_version) + }; + + Ok(RepairStats { + tables, + manifest_version, + }) +} + +async fn ensure_no_pending_recovery_sidecars(db: &Omnigraph, operation: &str) -> Result<()> { + if !crate::db::manifest::list_sidecars(db.root_uri(), db.storage_adapter()) + .await? + .is_empty() + { + return Err(OmniError::manifest_conflict(format!( + "{operation} requires a clean recovery state; reopen the graph to run the \ + recovery sweep before repairing" + ))); + } + Ok(()) +} + +async fn classify_drift( + ds: &Dataset, + manifest_version: u64, + lance_head_version: u64, +) -> ClassificationResult { + let mut operations = Vec::new(); + let mut saw_suspicious = false; + let mut error = None; + + for version in manifest_version.saturating_add(1)..=lance_head_version { + match ds.read_transaction_by_version(version).await { + Ok(Some(transaction)) => { + let operation = transaction.operation; + operations.push(operation.name().to_string()); + if !matches!( + operation, + Operation::Rewrite { .. } | Operation::ReserveFragments { .. } + ) { + saw_suspicious = true; + } + } + Ok(None) => { + error = Some(format!("missing Lance transaction for version {version}")); + break; + } + Err(err) => { + error = Some(format!( + "failed to read Lance transaction for version {version}: {err}" + )); + break; + } + } + } + + let classification = if error.is_some() { + RepairClassification::Unverifiable + } else if saw_suspicious { + RepairClassification::Suspicious + } else { + RepairClassification::VerifiedMaintenance + }; + + ClassificationResult { + classification, + operations, + error, + } +} diff --git a/crates/omnigraph/src/exec/mutation.rs b/crates/omnigraph/src/exec/mutation.rs index 02b2a21..985889a 100644 --- a/crates/omnigraph/src/exec/mutation.rs +++ b/crates/omnigraph/src/exec/mutation.rs @@ -569,7 +569,8 @@ use super::staging::{MutationStaging, PendingMode}; /// via `open_for_mutation_on_branch`, which compares Lance HEAD against /// the manifest's pinned version — that fence is the engine's /// publisher-style OCC catching cross-writer drift before we make any -/// changes. +/// changes. For delete-only queries, this strict open is also the uncovered +/// drift guard that runs before `delete_where` can inline-commit. /// /// On subsequent touches *within the same query*, behavior depends on /// whether the table has already been inline-committed by a delete op: diff --git a/crates/omnigraph/src/exec/staging.rs b/crates/omnigraph/src/exec/staging.rs index 0d26fd3..264ab59 100644 --- a/crates/omnigraph/src/exec/staging.rs +++ b/crates/omnigraph/src/exec/staging.rs @@ -495,25 +495,21 @@ impl StagedMutation { // until `ensure_path` learns how to bump expected_version on // op-kind upgrade. // - // Why per-branch (and not the bound-branch `db.snapshot()`): - // when the caller mutates a branch other than the engine's - // bound branch (e.g., feature-branch ingest from a server - // handle bound to main), `db.snapshot()` returns the bound - // branch's view of each table — which is the wrong pin for - // the publisher's CAS on a different branch. Using - // `snapshot_for_branch(branch)` resolves the per-branch - // entries correctly. The cost is one fresh manifest read per - // mutation; PR 1b's regression came from this same read, but - // that read is now strictly necessary for cross-branch - // correctness. Single-table same-branch mutations could still - // skip this read (queue exclusivity makes the publisher CAS a - // no-op), but the conditional adds complexity for marginal - // gain — left as a follow-up perf optimization. + // Why a fresh per-branch snapshot (and not the bound-branch + // `db.snapshot()` / `snapshot_for_branch()` fast path): a stale + // engine handle may be bound to the same branch it is writing. For + // non-strict Insert/Merge, that stale local view is allowed to rebase + // to the live manifest pin under the queue; only uncovered Lance + // HEAD>manifest drift is refused. For writes targeting a branch other + // than the engine's bound branch (e.g., feature-branch ingest from a + // server handle bound to main), the same helper also resolves the + // correct branch pin. The cost is one fresh manifest read per mutation + // plus one Lance HEAD open per staged table for the drift guard below. // // Multi-coordinator deployments (§VI.27 aspirational) get // genuine cross-process drift detection from this read for // free. - let snapshot = db.snapshot_for_branch(branch).await?; + let snapshot = db.fresh_snapshot_for_branch(branch).await?; for entry in staged.iter_mut() { let current = snapshot .entry(&entry.table_key) @@ -541,6 +537,35 @@ impl StagedMutation { )); } + // Separate manifest-visible concurrency from uncovered Lance drift. + // Non-strict inserts/merges are allowed to rebase from their staged + // read version to the fresh manifest pin above, but only if the + // live Lance HEAD still equals that manifest pin. If an external + // raw Lance write or a pre-fix maintenance path moved HEAD without + // publishing `__manifest`, this write must not silently fold it. + let head = db + .table_store() + .open_dataset_head_for_write( + &entry.table_key, + &entry.path.full_path, + entry.path.table_branch.as_deref(), + ) + .await? + .version() + .version; + if head < current { + return Err(OmniError::manifest_internal(format!( + "table '{}' Lance HEAD version {} is behind manifest version {}", + entry.table_key, head, current + ))); + } + if head > current { + return Err(OmniError::manifest_conflict(format!( + "table '{}' has Lance HEAD version {} ahead of manifest version {}; run `omnigraph repair` before writing", + entry.table_key, head, current + ))); + } + entry.expected_version = current; expected_versions.insert(entry.table_key.clone(), current); } diff --git a/crates/omnigraph/tests/lance_surface_guards.rs b/crates/omnigraph/tests/lance_surface_guards.rs index 1d60c08..65efc4e 100644 --- a/crates/omnigraph/tests/lance_surface_guards.rs +++ b/crates/omnigraph/tests/lance_surface_guards.rs @@ -30,6 +30,7 @@ use arrow_schema::{DataType, Field, Schema}; use lance::Dataset; use lance::dataset::builder::DatasetBuilder; use lance::dataset::optimize::{CompactionOptions, compact_files}; +use lance::dataset::transaction::Operation; use lance::dataset::write::delete::DeleteResult; use lance::dataset::{MergeInsertBuilder, WhenMatched, WhenNotMatched, WriteMode, WriteParams}; use lance_file::version::LanceFileVersion; @@ -222,6 +223,33 @@ async fn _compile_compact_files_signature() -> lance::Result<()> { Ok(()) } +// --- Guard 7b: transaction history exposes repair's classification surface - +// +// `db/omnigraph/repair.rs` reads Lance transactions between manifest and HEAD +// and treats only `ReserveFragments` + `Rewrite` as safe maintenance drift. +// Compile-only. + +#[allow( + dead_code, + unreachable_code, + unused_variables, + unused_mut, + clippy::diverging_sub_expression +)] +async fn _compile_transaction_history_for_repair_signature() -> lance::Result<()> { + let ds: Dataset = unimplemented!(); + let tx = ds.read_transaction_by_version(1u64).await?; + if let Some(tx) = tx { + let operation = tx.operation; + let _name: &str = operation.name(); + match operation { + Operation::Rewrite { .. } | Operation::ReserveFragments { .. } => {} + _ => {} + } + } + Ok(()) +} + // --- Guard 8: Dataset::delete returns DeleteResult { new_dataset, num_deleted_rows } --- // // `table_store.rs::delete_where` consumes both fields. When MR-A migrates @@ -329,7 +357,10 @@ async fn compact_files_still_fails_on_blob_columns() { ])); RecordBatch::try_new( schema, - vec![Arc::new(StringArray::from(ids)) as _, Arc::new(content) as _], + vec![ + Arc::new(StringArray::from(ids)) as _, + Arc::new(content) as _, + ], ) .unwrap() } diff --git a/crates/omnigraph/tests/maintenance.rs b/crates/omnigraph/tests/maintenance.rs index 2a5a659..13c9de7 100644 --- a/crates/omnigraph/tests/maintenance.rs +++ b/crates/omnigraph/tests/maintenance.rs @@ -8,7 +8,11 @@ mod helpers; use std::time::Duration; use lance::Dataset; -use omnigraph::db::{CleanupPolicyOptions, Omnigraph, ReadTarget, SkipReason}; +use lance::dataset::optimize::{CompactionOptions, compact_files}; +use omnigraph::db::{ + CleanupPolicyOptions, Omnigraph, ReadTarget, RepairAction, RepairClassification, RepairOptions, + SkipReason, +}; use omnigraph::loader::{LoadMode, load_jsonl}; use helpers::{ @@ -27,11 +31,64 @@ fn node_table_uri(root: &str, type_name: &str) -> String { format!("{}/nodes/{hash:016x}", root.trim_end_matches('/')) } +async fn person_manifest_and_head(db: &Omnigraph, root: &str) -> (u64, u64, String) { + let snap = db.snapshot_of(ReadTarget::branch("main")).await.unwrap(); + let entry = snap.entry("node:Person").unwrap(); + let full = format!("{}/{}", root.trim_end_matches('/'), entry.table_path); + let head = Dataset::open(&full).await.unwrap().version().version; + (entry.table_version, head, full) +} + +async fn add_person_fragments(db: &mut Omnigraph) { + for (name, age) in [("Eve", 40), ("Frank", 41), ("Grace", 42), ("Heidi", 43)] { + mutate_main( + db, + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", name)], &[("$age", age as i64)]), + ) + .await + .expect("insert"); + } +} + +async fn forge_person_compaction_drift(db: &mut Omnigraph, root: &str) -> (u64, u64, String) { + add_person_fragments(db).await; + let (manifest_version, _, full) = person_manifest_and_head(db, root).await; + let mut ds = Dataset::open(&full).await.unwrap(); + let metrics = compact_files(&mut ds, CompactionOptions::default(), None) + .await + .expect("raw Lance compaction"); + let lance_head_version = ds.version().version; + assert!( + lance_head_version > manifest_version, + "raw Lance compaction should advance HEAD beyond manifest" + ); + assert!( + metrics.fragments_removed > 0 || metrics.fragments_added > 0, + "test precondition: raw compaction should rewrite fragments" + ); + (manifest_version, lance_head_version, full) +} + +async fn forge_person_delete_drift(db: &Omnigraph, root: &str) -> (u64, u64, String) { + let (manifest_version, _, full) = person_manifest_and_head(db, root).await; + let mut ds = Dataset::open(&full).await.unwrap(); + let deleted = ds.delete("name = 'Alice'").await.expect("raw Lance delete"); + assert_eq!(deleted.num_deleted_rows, 1, "fixture should delete Alice"); + let lance_head_version = deleted.new_dataset.version().version; + assert!( + lance_head_version > manifest_version, + "raw Lance delete should advance HEAD beyond manifest" + ); + (manifest_version, lance_head_version, full) +} + #[tokio::test] async fn optimize_on_empty_graph_returns_stats_per_table_with_no_changes() { let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); - let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + let db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); let stats = db.optimize().await.unwrap(); @@ -47,7 +104,7 @@ async fn optimize_on_empty_graph_returns_stats_per_table_with_no_changes() { #[tokio::test] async fn optimize_after_load_then_again_is_idempotent() { let dir = tempfile::tempdir().unwrap(); - let mut db = init_and_load(&dir).await; + let db = init_and_load(&dir).await; // First pass may compact (load wrote real fragments). let _first = db.optimize().await.unwrap(); @@ -180,7 +237,12 @@ node Tag {\n slug: String @key\n}\n"; #[tokio::test] async fn optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds() { let dir = tempfile::tempdir().unwrap(); - let root = dir.path().to_str().unwrap().trim_end_matches('/').to_string(); + let root = dir + .path() + .to_str() + .unwrap() + .trim_end_matches('/') + .to_string(); let mut db = init_and_load(&dir).await; // Several separate inserts → multiple Person fragments, so `compact_files` @@ -234,6 +296,281 @@ async fn optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds() { assert!(result.applied, "schema apply should report applied=true"); } +#[tokio::test] +async fn optimize_skips_preexisting_manifest_head_drift() { + let dir = tempfile::tempdir().unwrap(); + let root = dir + .path() + .to_str() + .unwrap() + .trim_end_matches('/') + .to_string(); + let mut db = init_and_load(&dir).await; + let (manifest_before, head_before, _) = forge_person_compaction_drift(&mut db, &root).await; + + let stats = db.optimize().await.unwrap(); + let person = stats + .iter() + .find(|s| s.table_key == "node:Person") + .expect("Person stat present"); + assert_eq!(person.skipped, Some(SkipReason::DriftNeedsRepair)); + assert!(!person.committed); + assert_eq!(person.manifest_version, Some(manifest_before)); + assert_eq!(person.lance_head_version, Some(head_before)); + + let (manifest_after, head_after, _) = person_manifest_and_head(&db, &root).await; + assert_eq!( + manifest_after, manifest_before, + "optimize must not publish uncovered drift" + ); + assert_eq!( + head_after, head_before, + "optimize must not move drifted HEAD" + ); +} + +#[tokio::test] +async fn repair_preview_reports_verified_maintenance_drift_without_healing() { + let dir = tempfile::tempdir().unwrap(); + let root = dir + .path() + .to_str() + .unwrap() + .trim_end_matches('/') + .to_string(); + let mut db = init_and_load(&dir).await; + let (manifest_before, head_before, _) = forge_person_compaction_drift(&mut db, &root).await; + + let stats = db + .repair(RepairOptions { + confirm: false, + force: false, + }) + .await + .unwrap(); + assert_eq!(stats.manifest_version, None); + let person = stats + .tables + .iter() + .find(|s| s.table_key == "node:Person") + .expect("Person repair stat present"); + assert_eq!( + person.classification, + RepairClassification::VerifiedMaintenance + ); + assert_eq!(person.action, RepairAction::Preview); + assert_eq!(person.manifest_version, manifest_before); + assert_eq!(person.lance_head_version, head_before); + assert!( + person + .operations + .iter() + .all(|op| op == "ReserveFragments" || op == "Rewrite"), + "maintenance drift should only include Lance maintenance operations: {:?}", + person.operations + ); + + let (manifest_after, head_after, _) = person_manifest_and_head(&db, &root).await; + assert_eq!(manifest_after, manifest_before); + assert_eq!(head_after, head_before); +} + +#[tokio::test] +async fn repair_confirm_heals_verified_maintenance_drift() { + let dir = tempfile::tempdir().unwrap(); + let root = dir + .path() + .to_str() + .unwrap() + .trim_end_matches('/') + .to_string(); + let mut db = init_and_load(&dir).await; + let (_, head_before, _) = forge_person_compaction_drift(&mut db, &root).await; + + let stats = db + .repair(RepairOptions { + confirm: true, + force: false, + }) + .await + .unwrap(); + assert!( + stats.manifest_version.is_some(), + "confirmed repair should publish one manifest commit" + ); + let person = stats + .tables + .iter() + .find(|s| s.table_key == "node:Person") + .expect("Person repair stat present"); + assert_eq!( + person.classification, + RepairClassification::VerifiedMaintenance + ); + assert_eq!(person.action, RepairAction::Healed); + + let (manifest_after, head_after, _) = person_manifest_and_head(&db, &root).await; + assert_eq!(manifest_after, head_before); + assert_eq!(head_after, head_before); + + let desired = TEST_SCHEMA.replace( + " age: I32?\n}", + " age: I32?\n nickname: String?\n}", + ); + let result = db + .apply_schema(&desired) + .await + .expect("strict schema apply should succeed after repair"); + assert!(result.applied); +} + +#[tokio::test] +async fn repair_refuses_raw_delete_without_force() { + let dir = tempfile::tempdir().unwrap(); + let root = dir + .path() + .to_str() + .unwrap() + .trim_end_matches('/') + .to_string(); + let db = init_and_load(&dir).await; + let (manifest_before, head_before, _) = forge_person_delete_drift(&db, &root).await; + + let stats = db + .repair(RepairOptions { + confirm: true, + force: false, + }) + .await + .unwrap(); + assert_eq!(stats.manifest_version, None); + let person = stats + .tables + .iter() + .find(|s| s.table_key == "node:Person") + .expect("Person repair stat present"); + assert_eq!(person.classification, RepairClassification::Suspicious); + assert_eq!(person.action, RepairAction::Refused); + assert!( + person.operations.iter().any(|op| op == "Delete"), + "raw Lance delete should be reported as a suspicious operation: {:?}", + person.operations + ); + + let (manifest_after, head_after, _) = person_manifest_and_head(&db, &root).await; + assert_eq!(manifest_after, manifest_before); + assert_eq!(head_after, head_before); + assert_eq!( + count_rows(&db, "node:Person").await, + 4, + "manifest-pinned reads should still see the pre-delete version" + ); +} + +#[tokio::test] +async fn repair_force_heals_suspicious_drift() { + let dir = tempfile::tempdir().unwrap(); + let root = dir + .path() + .to_str() + .unwrap() + .trim_end_matches('/') + .to_string(); + let db = init_and_load(&dir).await; + let (_, head_before, _) = forge_person_delete_drift(&db, &root).await; + + let stats = db + .repair(RepairOptions { + confirm: true, + force: true, + }) + .await + .unwrap(); + let person = stats + .tables + .iter() + .find(|s| s.table_key == "node:Person") + .expect("Person repair stat present"); + assert_eq!(person.classification, RepairClassification::Suspicious); + assert_eq!(person.action, RepairAction::Forced); + + let (manifest_after, head_after, _) = person_manifest_and_head(&db, &root).await; + assert_eq!(manifest_after, head_before); + assert_eq!(head_after, head_before); + assert_eq!( + count_rows(&db, "node:Person").await, + 3, + "forced repair publishes the raw delete's HEAD" + ); +} + +#[tokio::test] +async fn non_strict_load_refuses_uncovered_drift_before_folding_it() { + let dir = tempfile::tempdir().unwrap(); + let root = dir + .path() + .to_str() + .unwrap() + .trim_end_matches('/') + .to_string(); + let mut db = init_and_load(&dir).await; + let (manifest_before, head_before, _) = forge_person_compaction_drift(&mut db, &root).await; + + let err = load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Ivan\",\"age\":44}}", + LoadMode::Merge, + ) + .await + .expect_err("merge load must not silently fold uncovered drift"); + assert!( + err.to_string().contains("omnigraph repair"), + "error should point at explicit repair; got: {err}" + ); + + let (manifest_after, head_after, _) = person_manifest_and_head(&db, &root).await; + assert_eq!(manifest_after, manifest_before); + assert_eq!(head_after, head_before); +} + +#[tokio::test] +async fn delete_only_mutation_refuses_uncovered_drift_before_inline_commit() { + let dir = tempfile::tempdir().unwrap(); + let root = dir + .path() + .to_str() + .unwrap() + .trim_end_matches('/') + .to_string(); + let mut db = init_and_load(&dir).await; + let (manifest_before, head_before, _) = forge_person_compaction_drift(&mut db, &root).await; + + let err = mutate_main( + &mut db, + MUTATION_QUERIES, + "remove_person", + &mixed_params(&[("$name", "Alice")], &[]), + ) + .await + .expect_err("strict delete must reject uncovered drift before delete_where"); + assert!( + err.to_string().contains("expected"), + "delete should fail as a strict stale-version write; got: {err}" + ); + + let (manifest_after, head_after, _) = person_manifest_and_head(&db, &root).await; + assert_eq!(manifest_after, manifest_before); + assert_eq!( + head_after, head_before, + "delete_where must not run after the strict drift guard fails" + ); + assert_eq!( + count_rows(&db, "node:Person").await, + 8, + "manifest-pinned reads should still see all rows present before the failed delete" + ); +} + // Regression: `optimize` must REFUSE when an unresolved recovery sidecar is // pending. Operating on an unrecovered graph could publish a partial write that // the all-or-nothing recovery sweep would roll back; the operator must reopen diff --git a/crates/omnigraph/tests/writes.rs b/crates/omnigraph/tests/writes.rs index 0a309c9..d76ad46 100644 --- a/crates/omnigraph/tests/writes.rs +++ b/crates/omnigraph/tests/writes.rs @@ -6,8 +6,8 @@ //! What this file covers: //! - No `__run__*` branches are created by load or mutate. //! - Cancellation of a mutation future leaves no graph-level state. -//! - Concurrent writers to the same table land exactly one publish; the -//! loser surfaces `ManifestConflictDetails::ExpectedVersionMismatch`. +//! - Concurrent non-strict inserts/merges rebase under the per-table queue; +//! strict updates/deletes surface `ExpectedVersionMismatch` on stale state. //! - Failed mutations and loads leave the target unchanged. //! - Multi-statement mutations are atomic (one commit per query). //! - actor_id propagates through to the commit graph. @@ -17,7 +17,7 @@ mod helpers; use arrow_array::Array; use omnigraph::db::commit_graph::CommitGraph; use omnigraph::db::{Omnigraph, ReadTarget}; -use omnigraph::error::{ManifestConflictDetails, ManifestErrorKind, OmniError}; +use omnigraph::error::OmniError; use omnigraph::loader::{LoadMode, load_jsonl}; use helpers::*; @@ -241,18 +241,11 @@ async fn partial_failure_leaves_target_queryable_and_unblocks_next_mutation() { assert_eq!(frank.num_rows(), 1, "Frank must be visible after publish"); } -/// Concurrent writers to the same `(table, branch)` produce exactly one -/// success and one `ExpectedVersionMismatch`. The replacement for the old -/// `concurrent_conflicting_run_publish_fails_cleanly` test — the OCC fence -/// has moved from a graph-level run-publish merge into the publisher's -/// per-table CAS. -/// -/// Drives the race by interleaving two handles that captured the same -/// pre-write manifest snapshot: A commits first; B's commit then sees -/// `expected_versions[node:Person] = pre` while the manifest is at -/// `pre + 1`, and the publisher rejects. +/// Stale non-strict writers rebase to the live manifest pin under the +/// per-table queue instead of folding raw drift or returning a false 409. +/// Strict update/delete semantics are covered by the consistency/server tests. #[tokio::test] -async fn concurrent_writers_one_succeeds_one_gets_expected_version_mismatch() { +async fn stale_non_strict_insert_rebases_to_live_manifest_pin() { let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_string_lossy().into_owned(); @@ -281,40 +274,30 @@ async fn concurrent_writers_one_succeeds_one_gets_expected_version_mismatch() { .unwrap(); } - // Writer B's coordinator is still at the pre-A snapshot. Its mutation - // captures expected_versions[node:Person] = pre (stale), then publishes - // — the publisher's CAS pre-check sees the manifest is now at post and - // rejects with ExpectedVersionMismatch. - let result_b = db_b - .mutate( - "main", - MUTATION_QUERIES, - "insert_person", - &mixed_params(&[("$name", "WriterB")], &[("$age", 42)]), - ) - .await; + // Writer B's coordinator is still at the pre-A snapshot, but Insert is + // non-strict: commit_all re-reads the live manifest pin under the queue, + // verifies Lance HEAD equals that pin, and then lets Lance rebase the + // staged append. + db_b.mutate( + "main", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "WriterB")], &[("$age", 42)]), + ) + .await + .unwrap(); - let err = result_b.expect_err("stale writer must hit ExpectedVersionMismatch"); - let OmniError::Manifest(manifest_err) = err else { - panic!("expected Manifest error, got {err:?}"); - }; - assert_eq!(manifest_err.kind, ManifestErrorKind::Conflict); - let Some(ManifestConflictDetails::ExpectedVersionMismatch { - ref table_key, - expected, - actual, - }) = manifest_err.details - else { - panic!( - "expected ExpectedVersionMismatch, got {:?}", - manifest_err.details, - ); - }; - assert_eq!(table_key, "node:Person"); - assert!( - actual > expected, - "actual ({actual}) should be ahead of expected ({expected})", - ); + for name in ["WriterA", "WriterB"] { + let person = query_main( + &mut db_b, + TEST_QUERIES, + "get_person", + ¶ms(&[("$name", name)]), + ) + .await + .unwrap(); + assert_eq!(person.num_rows(), 1, "{name} should be visible"); + } } /// The cancellation hole that motivated removing the Run state machine: dropping a mutation future diff --git a/docs/dev/invariants.md b/docs/dev/invariants.md index 5ee4f17..b29d740 100644 --- a/docs/dev/invariants.md +++ b/docs/dev/invariants.md @@ -139,6 +139,20 @@ them explicit. Remove the skip when the upstream Lance fix lands — the `lance_surface_guards.rs::compact_files_still_fails_on_blob_columns` guard turns red on that bump to force it. +- **Manifest→commit-graph publish atomicity:** a graph commit advances + `__manifest` (the visibility authority) and then appends `_graph_commits` as + two separate writes (`commit_updates_with_actor_with_expected`, failpoint + `graph_publish.before_commit_append`). A crash between them leaves the manifest + at version N with no commit-graph row for N. Live reads and durability are + unaffected — the live version resolves via the manifest + (`GraphCoordinator::version()`), not the commit-graph head — and the open-time + recovery sweep does NOT repair it (`lance_head == manifest_pinned` classifies + `NoMovement`; a recovery sidecar would not change this). Impact is bounded to + commit history: `commit list` misses N, time-travel by commit id to N fails, + and merge-base loses a node (a likely-benign off-by-one re-merge). This affects + every publish, not a specific maintenance command. Eventual fix: make the + commit graph reconcilable from the manifest (or the two writes atomic) — not a + recovery-sidecar concern. - **Planner capability/stat surfaces:** cost-aware planning, complete capability advertisement, and explain-with-cost are roadmap. Do not describe them as implemented. diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 8974a9f..1ec7038 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -20,7 +20,7 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav | `end_to_end.rs` | Full init → load → query/mutate flow | | `branching.rs` | Branch create / list / delete, lazy fork | | `merge_truth_table.rs` | Merge-pair truth table (MR-786): all 9×9 `(left_op, right_op)` cells from `{noop, addNode, removeNode, addEdge, removeEdge, setProperty, dropProperty, addLabel, removeLabel}`. Adding a new op to `OpVariant` forces a compile error in `build_case` until the new row + column are dispositioned. 36 executable cells run through real `branch_merge` with a structured oracle (`MergeOutcome` / `MergeConflictKind` + graph-state assert); 45 cells involving `dropProperty`/`addLabel`/`removeLabel` are recorded as `Unsupported` until the mutation grammar grows. | -| `writes.rs` | Direct-publish writes: cancellation, concurrent-writer CAS, multi-statement atomicity, MR-794 staged-write rewire (D₂ rejection, insert+update coalesce, multi-append coalesce, partial-failure recovery, load RI/cardinality recovery) | +| `writes.rs` | Direct-publish writes: cancellation, non-strict insert/merge rebase under the per-table queue, strict stale-write conflicts, multi-statement atomicity, MR-794 staged-write rewire (D₂ rejection, insert+update coalesce, multi-append coalesce, partial-failure recovery, load RI/cardinality recovery) | | `staged_writes.rs` | TableStore staged-write primitives (`stage_append`, `stage_merge_insert`, `commit_staged`, `scan_with_staged`, `count_rows_with_staged`) — primitive-level only; engine code uses the in-memory `MutationStaging` accumulator instead | | `lifecycle.rs` | Graph lifecycle, schema state | | `point_in_time.rs` | Snapshots, time travel (`snapshot_at_version`, `entity_at`) | @@ -34,7 +34,7 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav | `s3_storage.rs` | S3-backed graph (skipped unless `OMNIGRAPH_S3_TEST_BUCKET` is set) | | `lance_version_columns.rs` | Per-row `_row_last_updated_at_version` behavior | | `validators.rs` | Schema constraint enforcement (enum, range, unique, cardinality) across JSONL, insert, update paths | -| `maintenance.rs` | `optimize` (compaction) + `cleanup` (version GC): empty/idempotent/no-op edges, policy validation, head preservation; `optimize` publishes the compacted version so the manifest tracks the Lance HEAD and a subsequent schema apply succeeds (`optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds`), and refuses to run while a `__recovery` sidecar is pending so optimize only ever operates on a recovered graph (`optimize_defers_when_recovery_sidecar_is_pending`) | +| `maintenance.rs` | `optimize` (compaction), `repair` (explicit uncovered-drift publish), and `cleanup` (version GC): empty/idempotent/no-op edges, policy validation, head preservation; `optimize` publishes its own compaction (`optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds`), skips pre-existing uncovered drift (`optimize_skips_preexisting_manifest_head_drift`), and refuses to run while a `__recovery` sidecar is pending (`optimize_defers_when_recovery_sidecar_is_pending`); `repair` previews/heals verified maintenance drift, refuses raw semantic drift without `--force`, and forced repair publishes only by explicit operator choice | | `failpoints.rs` | Failure-injection coverage (gated on `failpoints` feature). Includes the five per-writer Phase B → recovery integration tests (`recovery_rolls_forward_after_finalize_publisher_failure`, `schema_apply_phase_b_failure_recovered_on_next_open`, `branch_merge_phase_b_failure_recovered_on_next_open`, `ensure_indices_phase_b_failure_recovered_on_next_open`, `optimize_phase_b_failure_recovered_on_next_open`). | | `recovery.rs` | Open-time recovery sweep — sidecar I/O, classifier dispatch (NoMovement / RolledPastExpected / UnexpectedAtP1 / UnexpectedMultistep / InvariantViolation), all-or-nothing decision, roll-forward via `ManifestBatchPublisher::publish`, roll-back via `Dataset::restore`, audit row in `_graph_commit_recoveries.lance`, `OpenMode::ReadOnly` skip path | | `composite_flow.rs` | Compositional/narrative end-to-end stories — multi-step flows that compose mechanics covered by other test files. Catches integration regressions where individual operations all pass their unit tests but their composition breaks (sequential merges, post-merge main writes, time-travel through merge DAG, reopen consistency over multi-merge histories, post-optimize and post-cleanup strict writes). | diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 8263919..a88d253 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -2,7 +2,7 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` schema. For a quick-start guide, see [cli.md](cli.md). -17 top-level command families, 40+ subcommands. All commands accept either a positional `URI`, `--uri`, or a `--target <name>` resolved against `omnigraph.yaml`. +Top-level command families and subcommands. Graph-targeting commands accept either a positional `URI`, `--uri`, or a `--target <name>` resolved against `omnigraph.yaml`. ## Top-level commands @@ -17,11 +17,11 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` sc | `export` | dump to JSONL on stdout (`--type T`, `--table K` filters) | | `branch create \| list \| delete \| merge` | branching ops | | `commit list \| show` | inspect commit graph | -| `run list \| show \| publish \| abort` | transactional run ops | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | | `queries validate \| list` | operate on the server-side stored-query registry (the `queries:` block). `validate` type-checks every stored query against the live schema offline (opens the selected graph; exits non-zero on any breakage), catching schema drift without restarting the server; `list` prints the selected registry's query names, MCP exposure, and typed params. For per-graph registries, pass `--target <graph>` or set `cli.graph`; with no graph selection, `list` shows only top-level `queries:`. Distinct from `lint`, which validates a single `.gq` file | -| `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns; `--json` reports a `skipped` field) | +| `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) | +| `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | | `embed` | offline JSONL embedding pipeline | | `policy validate \| test \| explain` | Cedar tooling. Selects `cli.graph`, else `server.graph`, else top-level `policy.file` | diff --git a/docs/user/maintenance.md b/docs/user/maintenance.md index a835799..e69bba3 100644 --- a/docs/user/maintenance.md +++ b/docs/user/maintenance.md @@ -1,17 +1,26 @@ -# Maintenance: Optimize & Cleanup +# Maintenance: Optimize, Repair & Cleanup -`db/omnigraph/optimize.rs`. +`db/omnigraph/optimize.rs` and `db/omnigraph/repair.rs`. ## `optimize_all_tables(db)` — non-destructive - Lance `compact_files()` on every node + edge table on `main`, then **publishes the compacted version to the `__manifest`** so the manifest's `table_version` tracks the compacted Lance HEAD. Reads pin the manifest version, so without this publish compaction would be invisible to readers *and* would break the HEAD-vs-manifest precondition of the next schema apply / strict update/delete ("stale view … refresh and retry"). The publish advances the graph version (a system-attributed commit) only for tables that actually compacted. - Rewrites small fragments into fewer large ones; old fragments remain reachable via older manifests until `cleanup` runs. - Each table's compact→publish runs under its per-`(table, main)` write queue (serializing with concurrent mutations — compaction is a Lance `Rewrite` op that retryable-conflicts with a concurrent merge/update/delete on overlapping fragments). The Lance-HEAD-before-manifest-publish gap is covered by a `SidecarKind::Optimize` recovery sidecar (loose-match): a crash in that window rolls the compacted version forward on the next `Omnigraph::open` (compaction is content-preserving, so roll-forward is always safe). -- **Requires a recovered graph.** `optimize` refuses (errors) when an unresolved recovery sidecar is present under `__recovery` — operating on an unrecovered graph could publish a partial write the open-time recovery sweep would roll back. Reopen the graph to run the recovery sweep, then re-run `optimize`. (Recovery roll-back now publishes its restored version, so a recovered graph always satisfies `manifest == Lance HEAD` going in; there is no leftover drift for `optimize` to interpret.) +- **Requires a recovered graph.** `optimize` refuses (errors) when an unresolved recovery sidecar is present under `__recovery` — operating on an unrecovered graph could publish a partial write the open-time recovery sweep would roll back. Reopen the graph to run the recovery sweep, then re-run `optimize`. +- **Uncovered drift is skipped, not interpreted.** If a table's Lance HEAD is ahead of the version recorded in `__manifest` and no recovery sidecar covers that movement, `optimize` reports `skipped: Some(DriftNeedsRepair)` with the manifest/head versions and leaves the table untouched. Run `omnigraph repair` to classify and explicitly publish that drift. - Bounded by `OMNIGRAPH_MAINTENANCE_CONCURRENCY` (default 8). -- Returns `[TableOptimizeStats { table_key, fragments_removed, fragments_added, committed, skipped }]`. +- Returns `[TableOptimizeStats { table_key, fragments_removed, fragments_added, committed, skipped, manifest_version, lance_head_version }]`. - **Blob tables are skipped.** A table that declares any `Blob` property is not compacted: it is reported with `skipped: Some(BlobColumnsUnsupportedByLance)` (and logged via `tracing::warn`) instead of compacted, and the rest of the sweep proceeds normally. The current Lance `compact_files` mis-decodes blob-v2 columns under its forced `BlobHandling::AllBinary` read; **reads and writes are unaffected** — only compaction is. This is gated by `LANCE_SUPPORTS_BLOB_COMPACTION` (`db/omnigraph/optimize.rs`) and removed when the upstream Lance fix lands (see [docs/dev/lance.md](../dev/lance.md)). Consequence: fragment count and deleted-row space on blob tables are not reclaimed until then; query results are never affected. +## `repair_all_tables(db, options)` — explicit + +- Handles **uncovered manifest/head drift**: a table's Lance HEAD is ahead of the manifest pin and no recovery sidecar records the writer intent. +- Preview by default. `omnigraph repair --json <uri>` reports each table's `classification`, `action`, manifest/head versions, Lance operation names, and any classification error. `--confirm` publishes only verified maintenance drift; if any suspicious or unverifiable table is refused, the CLI prints the per-table output and exits non-zero. `--force --confirm` also publishes suspicious or unverifiable drift after operator review. +- Classifies drift by reading Lance transactions from `manifest_version + 1` through `lance_head_version`. Only `ReserveFragments` and `Rewrite` are verified maintenance. Semantic operations such as `Append`, `Delete`, `Update`, `Merge`, or missing transaction history are not auto-healed. +- Publishes repair by advancing `__manifest` to the existing Lance HEAD; it does **not** rewrite Lance data. If the publish succeeds, normal reads and strict writes use the repaired version. If it fails, no new data-side partial state was created. +- Requires a clean recovery state. Pending `__recovery` sidecars still belong to automatic sidecar recovery, not manual repair. + ## `cleanup_all_tables(db, options)` — destructive - Lance `cleanup_old_versions()` per table. From 131b78705deaf07eb2856988f06f1e222dca9dee Mon Sep 17 00:00:00 2001 From: Ragnor Comerford <hello@ragnor.co> Date: Tue, 9 Jun 2026 15:59:59 +0200 Subject: [PATCH 030/165] release: v0.6.2 --- AGENTS.md | 2 +- Cargo.lock | 10 ++--- crates/omnigraph-cli/Cargo.toml | 10 ++--- crates/omnigraph-compiler/Cargo.toml | 2 +- crates/omnigraph-policy/Cargo.toml | 2 +- crates/omnigraph-server/Cargo.toml | 8 ++-- crates/omnigraph/Cargo.toml | 8 ++-- docs/releases/v0.6.2.md | 55 ++++++++++++++++++++++++++++ openapi.json | 2 +- 9 files changed, 77 insertions(+), 22 deletions(-) create mode 100644 docs/releases/v0.6.2.md diff --git a/AGENTS.md b/AGENTS.md index 69272f8..d9573d0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -16,7 +16,7 @@ Tools that support `@`-imports (Claude Code) auto-include all three files via th `CLAUDE.md` is a symlink to this file — there is exactly one source of truth. Edit `AGENTS.md`. -**Version surveyed:** 0.6.1 +**Version surveyed:** 0.6.2 **Workspace crates:** `omnigraph-compiler`, `omnigraph` (engine), `omnigraph-policy`, `omnigraph-cli`, `omnigraph-server` **Storage substrate:** Lance 6.x (columnar, versioned, branchable) **License:** MIT diff --git a/Cargo.lock b/Cargo.lock index 3223b9c..65d253b 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4543,7 +4543,7 @@ dependencies = [ [[package]] name = "omnigraph-cli" -version = "0.6.1" +version = "0.6.2" dependencies = [ "assert_cmd", "clap", @@ -4565,7 +4565,7 @@ dependencies = [ [[package]] name = "omnigraph-compiler" -version = "0.6.1" +version = "0.6.2" dependencies = [ "ahash", "arrow-array", @@ -4586,7 +4586,7 @@ dependencies = [ [[package]] name = "omnigraph-engine" -version = "0.6.1" +version = "0.6.2" dependencies = [ "arc-swap", "arrow-array", @@ -4627,7 +4627,7 @@ dependencies = [ [[package]] name = "omnigraph-policy" -version = "0.6.1" +version = "0.6.2" dependencies = [ "cedar-policy", "clap", @@ -4640,7 +4640,7 @@ dependencies = [ [[package]] name = "omnigraph-server" -version = "0.6.1" +version = "0.6.2" dependencies = [ "arc-swap", "async-trait", diff --git a/crates/omnigraph-cli/Cargo.toml b/crates/omnigraph-cli/Cargo.toml index 641068e..e0a3154 100644 --- a/crates/omnigraph-cli/Cargo.toml +++ b/crates/omnigraph-cli/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-cli" -version = "0.6.1" +version = "0.6.2" edition = "2024" description = "CLI for the Omnigraph graph database." license = "MIT" @@ -13,10 +13,10 @@ name = "omnigraph" path = "src/main.rs" [dependencies] -omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.1" } -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.1" } -omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.1" } -omnigraph-server = { path = "../omnigraph-server", version = "0.6.1" } +omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.2" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } +omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.2" } +omnigraph-server = { path = "../omnigraph-server", version = "0.6.2" } clap = { workspace = true } color-eyre = { workspace = true } serde = { workspace = true } diff --git a/crates/omnigraph-compiler/Cargo.toml b/crates/omnigraph-compiler/Cargo.toml index 545db83..8db46e6 100644 --- a/crates/omnigraph-compiler/Cargo.toml +++ b/crates/omnigraph-compiler/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-compiler" -version = "0.6.1" +version = "0.6.2" edition = "2024" description = "Schema/query compiler for Omnigraph. Zero Lance dependency." license = "MIT" diff --git a/crates/omnigraph-policy/Cargo.toml b/crates/omnigraph-policy/Cargo.toml index 3d14fc5..0df2a12 100644 --- a/crates/omnigraph-policy/Cargo.toml +++ b/crates/omnigraph-policy/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-policy" -version = "0.6.1" +version = "0.6.2" edition = "2024" description = "Policy / authorization layer for Omnigraph — Cedar-backed PolicyEngine, PolicyChecker trait, ResourceScope enum." license = "MIT" diff --git a/crates/omnigraph-server/Cargo.toml b/crates/omnigraph-server/Cargo.toml index 5994aa1..5f87082 100644 --- a/crates/omnigraph-server/Cargo.toml +++ b/crates/omnigraph-server/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-server" -version = "0.6.1" +version = "0.6.2" edition = "2024" description = "HTTP server for the Omnigraph graph database." license = "MIT" @@ -19,9 +19,9 @@ default = [] aws = ["dep:aws-config", "dep:aws-sdk-secretsmanager"] [dependencies] -omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.1" } -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.1" } -omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.1" } +omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.2" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } +omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.2" } axum = { workspace = true } clap = { workspace = true } color-eyre = { workspace = true } diff --git a/crates/omnigraph/Cargo.toml b/crates/omnigraph/Cargo.toml index 70f51d8..24b0c9c 100644 --- a/crates/omnigraph/Cargo.toml +++ b/crates/omnigraph/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-engine" -version = "0.6.1" +version = "0.6.2" edition = "2024" description = "Runtime engine for the Omnigraph graph database." license = "MIT" @@ -16,8 +16,8 @@ default = [] failpoints = ["dep:fail", "fail/failpoints"] [dependencies] -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.1" } -omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.1" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } +omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.2" } lance = { workspace = true } lance-datafusion = { workspace = true } datafusion = { workspace = true } @@ -51,7 +51,7 @@ chrono = { workspace = true } arc-swap = { workspace = true } [dev-dependencies] -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.1" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } tokio = { workspace = true } lance-namespace-impls = { workspace = true } serial_test = "3" diff --git a/docs/releases/v0.6.2.md b/docs/releases/v0.6.2.md new file mode 100644 index 0000000..2504813 --- /dev/null +++ b/docs/releases/v0.6.2.md @@ -0,0 +1,55 @@ +# Omnigraph v0.6.2 + +v0.6.2 is a maintenance-safety release on top of v0.6.1. It tightens the +`optimize` / recovery boundary, adds an explicit repair path for uncovered +manifest/head drift, accepts pretty-printed JSON load input, and updates the +project governance and release automation around those fixes. + +## Highlights + +- **Explicit `omnigraph repair`.** New `repair` CLI support previews uncovered + manifest/head drift by default and reports each table's classification, + action, manifest version, Lance HEAD version, Lance operations, and any + classification error. `--confirm` publishes verified maintenance-only drift; + `--force --confirm` can publish suspicious or unverifiable drift after + operator review. +- **Optimize skips uncovered drift.** `omnigraph optimize` now refuses to + interpret Lance HEAD movement that is ahead of `__manifest` without a recovery + sidecar. Those tables are reported as `skipped: DriftNeedsRepair` and left + untouched until `omnigraph repair` classifies them. +- **Optimize publishes compaction.** Successful compaction now publishes the + compacted Lance version back through the graph manifest and is covered by an + `Optimize` recovery sidecar. A crash after Lance compaction but before + manifest publish converges through the normal recovery sweep instead of + leaving hidden drift. +- **Recovery roll-back convergence.** Recovery roll-back now aligns the + manifest-visible version after restoring a table, closing the residual where + Lance HEAD and `__manifest` could stay out of sync after recovery. +- **Pretty-printed JSON load input.** `load` accepts multi-line JSON objects in + addition to one-object-per-line JSONL, so formatted fixture or export files no + longer need to be minified before import. + +## Operational Notes + +- `repair` requires a clean recovery state. Pending `__recovery` sidecars still + belong to automatic open-time recovery; reopen the graph first, then run + repair if drift remains. +- `repair --confirm` only auto-publishes drift made of Lance maintenance + operations (`Rewrite` and `ReserveFragments`). Semantic operations such as + append, delete, update, and merge are refused unless the operator uses + `--force --confirm`. +- `optimize` remains non-destructive. It still skips blob-bearing tables while + OmniGraph is pinned to the Lance version with the blob-v2 compaction issue. +- No manual on-disk migration is required. Existing graphs open under v0.6.2; + the internal manifest schema stamp remains v3. + +## Docs, Governance, And CI + +- Added issue, discussion, RFC, and pull-request templates plus governance docs + for the external contribution path. +- Regenerated CODEOWNERS tables and adjusted branch-protection docs so code + owners can bypass required PR review where repository rules allow it. +- Trimmed Windows release builds out of per-PR CI and kept Windows packaging on + tag releases. +- Made Homebrew audit diagnostic-only in the release workflow so a flaky audit + cannot block publishing an otherwise valid formula update. diff --git a/openapi.json b/openapi.json index aced64d..335c0bc 100644 --- a/openapi.json +++ b/openapi.json @@ -7,7 +7,7 @@ "name": "MIT", "identifier": "MIT" }, - "version": "0.6.1" + "version": "0.6.2" }, "paths": { "/branches": { From b7f5276ab53abd3f7ae5e105a00255ae9a6c2064 Mon Sep 17 00:00:00 2001 From: "devin-ai-integration[bot]" <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Tue, 9 Jun 2026 17:17:31 +0300 Subject: [PATCH 031/165] fix(loader): enforce composite @unique(a, b) as a true composite key (#133) * fix(loader): enforce composite @unique(a, b) as a true composite key Node/edge composite uniqueness constraints were flattened into a single list of property names, so @unique(a, b) was enforced as independent single-field checks @unique(a) AND @unique(b) at intake. Preserve the constraint grouping and check each group as a composite key, mirroring the merge-path enforcement. Error messages now name the full composite. MR-983 * docs: clarify unit-separator comment in composite unique check * docs: fix separator reference in composite unique comment (merge.rs also uses U+001F) * fix(merge): align composite @unique key separator with intake (U+001F) The branch-merge path (update_unique_constraints) joined composite key columns with '|', while intake joins with U+001F. The same @unique(a, b) was keyed two different ways, and '|'-join can raise phantom merge conflicts for values containing '|' (e.g. ('x|y','z') vs ('x','y|z')). Factor the tuple-join into one shared helper (loader::composite_unique_key) so the intake and merge paths cannot drift again. Add branching regression tests for edge @unique(src, dst) on the merge path. Refs MR-983. --------- Co-authored-by: Ragnor Comerford <ragnor.comerford@gmail.com> Co-authored-by: Andrew Altshuler <andrew@collectivelab.io> --- crates/omnigraph/src/exec/merge.rs | 2 +- crates/omnigraph/src/exec/mutation.rs | 18 ++-- crates/omnigraph/src/loader/mod.rs | 123 +++++++++++++++++--------- crates/omnigraph/src/table_store.rs | 2 +- crates/omnigraph/tests/branching.rs | 101 +++++++++++++++++++++ crates/omnigraph/tests/consistency.rs | 53 ++++++++++- 6 files changed, 244 insertions(+), 55 deletions(-) diff --git a/crates/omnigraph/src/exec/merge.rs b/crates/omnigraph/src/exec/merge.rs index eb6c4a3..0e6434b 100644 --- a/crates/omnigraph/src/exec/merge.rs +++ b/crates/omnigraph/src/exec/merge.rs @@ -697,7 +697,7 @@ fn update_unique_constraints( if any_null { continue; } - let value = parts.join("|"); + let value = crate::loader::composite_unique_key(&parts); let row_id = row_id_at(batch, row)?; if let Some(first_row_id) = seen.insert(value.clone(), row_id.clone()) { conflicts.push(MergeConflict { diff --git a/crates/omnigraph/src/exec/mutation.rs b/crates/omnigraph/src/exec/mutation.rs index 985889a..0e7ded7 100644 --- a/crates/omnigraph/src/exec/mutation.rs +++ b/crates/omnigraph/src/exec/mutation.rs @@ -905,12 +905,12 @@ impl Omnigraph { let batch = build_insert_batch(&schema, &id, &resolved, &blob_props)?; crate::loader::validate_value_constraints(&batch, node_type)?; crate::loader::validate_enum_constraints(&batch, &node_type.properties, type_name)?; - let unique_props = crate::loader::unique_property_names_for_node(node_type); - if !unique_props.is_empty() { + let unique_groups = crate::loader::unique_constraint_groups_for_node(node_type); + if !unique_groups.is_empty() { crate::loader::enforce_unique_constraints_intra_batch( &batch, type_name, - &unique_props, + &unique_groups, )?; } let has_key = node_type.key_property().is_some(); @@ -946,12 +946,12 @@ impl Omnigraph { let batch = build_insert_batch(&schema, &id, &resolved, &blob_props)?; validate_edge_insert_endpoints(self, staging, branch, type_name, &resolved).await?; crate::loader::validate_enum_constraints(&batch, &edge_type.properties, type_name)?; - let unique_props = crate::loader::unique_property_names_for_edge(edge_type); - if !unique_props.is_empty() { + let unique_groups = crate::loader::unique_constraint_groups_for_edge(edge_type); + if !unique_groups.is_empty() { crate::loader::enforce_unique_constraints_intra_batch( &batch, type_name, - &unique_props, + &unique_groups, )?; } let table_key = format!("edge:{}", type_name); @@ -1094,12 +1094,12 @@ impl Omnigraph { let node_type = &self.catalog().node_types[type_name]; crate::loader::validate_value_constraints(&updated, node_type)?; crate::loader::validate_enum_constraints(&updated, &node_type.properties, type_name)?; - let unique_props = crate::loader::unique_property_names_for_node(node_type); - if !unique_props.is_empty() { + let unique_groups = crate::loader::unique_constraint_groups_for_node(node_type); + if !unique_groups.is_empty() { crate::loader::enforce_unique_constraints_intra_batch( &updated, type_name, - &unique_props, + &unique_groups, )?; } diff --git a/crates/omnigraph/src/loader/mod.rs b/crates/omnigraph/src/loader/mod.rs index d5d74c0..9a80b39 100644 --- a/crates/omnigraph/src/loader/mod.rs +++ b/crates/omnigraph/src/loader/mod.rs @@ -399,9 +399,9 @@ async fn load_jsonl_reader<R: BufRead>( let batch = build_node_batch(node_type, rows)?; validate_value_constraints(&batch, node_type)?; validate_enum_constraints(&batch, &node_type.properties, type_name)?; - let unique_props = unique_property_names_for_node(node_type); - if !unique_props.is_empty() { - enforce_unique_constraints_intra_batch(&batch, type_name, &unique_props)?; + let unique_groups = unique_constraint_groups_for_node(node_type); + if !unique_groups.is_empty() { + enforce_unique_constraints_intra_batch(&batch, type_name, &unique_groups)?; } let loaded_count = batch.num_rows(); let table_key = format!("node:{}", type_name); @@ -510,9 +510,9 @@ async fn load_jsonl_reader<R: BufRead>( let edge_type = &catalog.edge_types[edge_name]; let batch = build_edge_batch(edge_type, rows)?; validate_enum_constraints(&batch, &edge_type.properties, edge_name)?; - let unique_props = unique_property_names_for_edge(edge_type); - if !unique_props.is_empty() { - enforce_unique_constraints_intra_batch(&batch, edge_name, &unique_props)?; + let unique_groups = unique_constraint_groups_for_edge(edge_type); + if !unique_groups.is_empty() { + enforce_unique_constraints_intra_batch(&batch, edge_name, &unique_groups)?; } let loaded_count = batch.num_rows(); let table_key = format!("edge:{}", edge_name); @@ -1425,8 +1425,16 @@ pub(crate) fn validate_enum_constraints( Ok(()) } -/// Detect duplicate values within a single `RecordBatch` for any of the named -/// `unique_properties`. Returns an error on the first duplicate found. +/// Detect duplicate values within a single `RecordBatch` for any of the +/// `unique_constraints` groups. Each group is a list of one or more columns +/// that together form a uniqueness key: a violation occurs when two rows share +/// the same tuple of values across *all* columns in a group, so a composite +/// `@unique(a, b)` only conflicts when both `a` and `b` match. Returns an +/// error on the first duplicate found. +/// +/// Rows where any column in a group is null are exempt (standard SQL semantics +/// for uniqueness over nullable columns), as is any group whose columns are +/// not all present in the batch (e.g. a partial-schema load). /// /// Note: this only catches duplicates *within* the batch. Cross-batch /// uniqueness against already-committed rows is not enforced here — that @@ -1434,22 +1442,39 @@ pub(crate) fn validate_enum_constraints( pub(crate) fn enforce_unique_constraints_intra_batch( batch: &RecordBatch, type_name: &str, - unique_properties: &[String], + unique_constraints: &[Vec<String>], ) -> Result<()> { - for property in unique_properties { - let Some(col_idx) = batch.schema().index_of(property).ok() else { + for columns in unique_constraints { + let Some(col_indices) = columns + .iter() + .map(|name| batch.schema().index_of(name).ok()) + .collect::<Option<Vec<usize>>>() + else { continue; }; - let arr = batch.column(col_idx); let mut seen: HashMap<String, usize> = HashMap::new(); for row in 0..batch.num_rows() { - let Some(value) = scalar_to_string(arr, row) else { + let mut parts = Vec::with_capacity(col_indices.len()); + let mut any_null = false; + for &col_idx in &col_indices { + let Some(value) = scalar_to_string(batch.column(col_idx), row) else { + any_null = true; + break; + }; + parts.push(value); + } + if any_null { continue; - }; + } + let value = composite_unique_key(&parts); if let Some(prev_row) = seen.insert(value.clone(), row) { return Err(OmniError::manifest(format!( "@unique violation on {}.{}: value '{}' appears in rows {} and {}", - type_name, property, value, prev_row, row + type_name, + format_unique_columns(columns), + value, + prev_row, + row ))); } } @@ -1457,6 +1482,27 @@ pub(crate) fn enforce_unique_constraints_intra_batch( Ok(()) } +/// Join one row's rendered, non-null column values into a single composite +/// uniqueness key. The separator is the unit separator (U+001F) — a control +/// char highly unlikely to occur in real data, so distinct tuples like +/// `("a|b", "c")` and `("a", "b|c")` stay distinct rather than colliding. +/// +/// Shared by the intake path (`enforce_unique_constraints_intra_batch`) and +/// the branch-merge path (`exec/merge.rs::update_unique_constraints`) so the +/// two cannot silently drift to incompatible keyings. +pub(crate) fn composite_unique_key(parts: &[String]) -> String { + parts.join("\u{1f}") +} + +/// Render a unique constraint's columns for error messages: a single column +/// as `col`, a composite as `(a, b)`. +fn format_unique_columns(columns: &[String]) -> String { + match columns { + [single] => single.clone(), + _ => format!("({})", columns.join(", ")), + } +} + /// Reduce a single Arrow scalar at (`array`, `row`) to a `String` for /// uniqueness comparison. Returns `None` for null values (nulls are exempt /// from uniqueness in standard SQL semantics). @@ -1498,39 +1544,30 @@ fn scalar_to_string(array: &ArrayRef, row: usize) -> Option<String> { None } -/// Build the flat list of property names that must be checked for uniqueness -/// on a node type. Includes both `@unique` properties (from -/// `NodeType.unique_constraints`) and the `@key` (which implies uniqueness). -pub(crate) fn unique_property_names_for_node( +/// Build the list of uniqueness constraint groups to enforce on a node type. +/// Each group is the column tuple of one constraint. Includes every +/// `@unique(...)` constraint (from `NodeType.unique_constraints`) and the +/// `@key` (which implies uniqueness over its column tuple). Grouping is +/// preserved so a composite `@unique(a, b)` is enforced as a composite key +/// rather than degraded into independent single-field checks. +pub(crate) fn unique_constraint_groups_for_node( node_type: &omnigraph_compiler::catalog::NodeType, -) -> Vec<String> { - let mut props: Vec<String> = node_type - .unique_constraints - .iter() - .flatten() - .cloned() - .collect(); - if let Some(key) = &node_type.key { - props.extend(key.iter().cloned()); +) -> Vec<Vec<String>> { + let mut groups: Vec<Vec<String>> = node_type.unique_constraints.clone(); + if let Some(key) = &node_type.key + && !groups.contains(key) + { + groups.push(key.clone()); } - props.sort(); - props.dedup(); - props + groups } -/// Same as [`unique_property_names_for_node`] but for an edge type. -pub(crate) fn unique_property_names_for_edge( +/// Same as [`unique_constraint_groups_for_node`] but for an edge type (edges +/// have no `@key`). +pub(crate) fn unique_constraint_groups_for_edge( edge_type: &omnigraph_compiler::catalog::EdgeType, -) -> Vec<String> { - let mut props: Vec<String> = edge_type - .unique_constraints - .iter() - .flatten() - .cloned() - .collect(); - props.sort(); - props.dedup(); - props +) -> Vec<Vec<String>> { + edge_type.unique_constraints.clone() } fn extract_numeric_value(col: &ArrayRef, row: usize) -> Option<f64> { diff --git a/crates/omnigraph/src/table_store.rs b/crates/omnigraph/src/table_store.rs index 10123b0..4b52db6 100644 --- a/crates/omnigraph/src/table_store.rs +++ b/crates/omnigraph/src/table_store.rs @@ -732,7 +732,7 @@ impl TableStore { // before the FirstSeen setter has a chance to silently collapse // anything): // - Load path: `enforce_unique_constraints_intra_batch` - // (`loader/mod.rs:1453`) errors on intra-batch `@key` dups. + // (`loader/mod.rs:1471`) errors on intra-batch `@key` dups. // - Mutate path: `MutationStaging::finalize` (`exec/staging.rs`) // accumulates and dedupes by `id`. // - Branch-merge path: `compute_source_delta` / diff --git a/crates/omnigraph/tests/branching.rs b/crates/omnigraph/tests/branching.rs index 5a0c47d..108702c 100644 --- a/crates/omnigraph/tests/branching.rs +++ b/crates/omnigraph/tests/branching.rs @@ -39,6 +39,26 @@ query insert_user($name: String, $email: String) { } "#; +const EDGE_UNIQUE_SCHEMA: &str = r#" +node Person { + name: String @key +} + +edge Knows: Person -> Person { + @unique(src, dst) +} +"#; + +const EDGE_UNIQUE_DATA: &str = r#"{"type":"Person","data":{"name":"Alice"}} +{"type":"Person","data":{"name":"Bob"}} +{"type":"Person","data":{"name":"Carol"}}"#; + +const EDGE_UNIQUE_MUTATIONS: &str = r#" +query add_knows($from: String, $to: String) { + insert Knows { from: $from, to: $to } +} +"#; + const CARDINALITY_SCHEMA: &str = r#" node Person { name: String @key @@ -1119,6 +1139,87 @@ async fn branch_merge_reports_unique_violation_conflict() { } } +/// Regression for the MR-983 follow-up: the branch-merge path must enforce an +/// edge composite `@unique(src, dst)` as a true composite key, consistent with +/// the intake path. Two branches inserting the *same* (src, dst) pair must +/// conflict on merge. +#[tokio::test] +async fn branch_merge_reports_composite_unique_violation_conflict() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut main = init_db_from_schema_and_data(&dir, EDGE_UNIQUE_SCHEMA, EDGE_UNIQUE_DATA).await; + main.branch_create("feature").await.unwrap(); + + let mut feature = Omnigraph::open(uri).await.unwrap(); + + mutate_main( + &mut main, + EDGE_UNIQUE_MUTATIONS, + "add_knows", + ¶ms(&[("$from", "Alice"), ("$to", "Bob")]), + ) + .await + .unwrap(); + + mutate_branch( + &mut feature, + "feature", + EDGE_UNIQUE_MUTATIONS, + "add_knows", + ¶ms(&[("$from", "Alice"), ("$to", "Bob")]), + ) + .await + .unwrap(); + + let err = main.branch_merge("feature", "main").await.unwrap_err(); + match err { + OmniError::MergeConflicts(conflicts) => { + assert!(conflicts.iter().any(|conflict| { + conflict.table_key == "edge:Knows" + && conflict.kind == MergeConflictKind::UniqueViolation + })); + } + other => panic!("expected merge conflicts, got {other:?}"), + } +} + +/// Sibling to the above: pairs sharing `src` but differing on `dst` are unique +/// on the (src, dst) tuple and must merge cleanly. Guards against the composite +/// degrading back into a single-field `@unique(src)` on the merge path. +#[tokio::test] +async fn branch_merge_allows_distinct_composite_unique_pairs() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut main = init_db_from_schema_and_data(&dir, EDGE_UNIQUE_SCHEMA, EDGE_UNIQUE_DATA).await; + main.branch_create("feature").await.unwrap(); + + let mut feature = Omnigraph::open(uri).await.unwrap(); + + mutate_main( + &mut main, + EDGE_UNIQUE_MUTATIONS, + "add_knows", + ¶ms(&[("$from", "Alice"), ("$to", "Bob")]), + ) + .await + .unwrap(); + + mutate_branch( + &mut feature, + "feature", + EDGE_UNIQUE_MUTATIONS, + "add_knows", + ¶ms(&[("$from", "Alice"), ("$to", "Carol")]), + ) + .await + .unwrap(); + + main.branch_merge("feature", "main") + .await + .expect("distinct (src, dst) pairs are unique on the composite and must merge cleanly"); + assert_eq!(count_rows(&main, "edge:Knows").await, 2); +} + #[tokio::test] async fn branch_merge_reports_cardinality_violation_conflict() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/omnigraph/tests/consistency.rs b/crates/omnigraph/tests/consistency.rs index 26517db..729f2e8 100644 --- a/crates/omnigraph/tests/consistency.rs +++ b/crates/omnigraph/tests/consistency.rs @@ -188,7 +188,7 @@ node Thing { /// /// Defense in depth: /// 1. The loader's `enforce_unique_constraints_intra_batch` -/// (`loader/mod.rs:1453`), invoked unconditionally on any node type +/// (`loader/mod.rs:1471`), invoked unconditionally on any node type /// with a `@key`, errors on intra-batch duplicate `@key` values at /// intake — pinned by this test across every `LoadMode`. /// 2. The `check_batch_unique_by_keys` precondition at the top of @@ -229,6 +229,57 @@ node Thing { } } +/// Regression for MR-983: a node-level composite `@unique(a, b)` must be +/// enforced as a true composite key, not degraded into independent +/// single-field checks. Pre-fix, `unique_property_names_for_node` flattened +/// every constraint group into one property list, so `@unique(source, +/// external_id)` was enforced as `@unique(source)` *and* `@unique(external_id)` +/// — rejecting rows that were unique on the composite key and naming only the +/// first field in the error. +#[tokio::test] +async fn loader_enforces_composite_unique_as_composite_key() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let schema = r#" +node ExternalID { + slug: String @key + source: String @index + external_id: String @index + @unique(source, external_id) +} +"#; + let mut db = Omnigraph::init(uri, schema).await.unwrap(); + + // Same `source`, different `external_id` → unique on the composite key. + // This is the exact repro from MR-983 and must be accepted. + let composite_ok = r#"{"type":"ExternalID","data":{"slug":"a","source":"whatsapp","external_id":"+E.164"}} +{"type":"ExternalID","data":{"slug":"b","source":"whatsapp","external_id":"pn:12345"}} +"#; + load_jsonl(&mut db, composite_ok, LoadMode::Overwrite) + .await + .expect("rows unique on the composite (source, external_id) must be accepted"); + assert_eq!(count_rows(&db, "node:ExternalID").await, 2); + + // Both composite columns equal → genuine violation. The error must name + // the whole composite, not just the first field. + let composite_dupe = r#"{"type":"ExternalID","data":{"slug":"c","source":"whatsapp","external_id":"dup"}} +{"type":"ExternalID","data":{"slug":"d","source":"whatsapp","external_id":"dup"}} +"#; + let err = load_jsonl(&mut db, composite_dupe, LoadMode::Overwrite) + .await + .unwrap_err(); + let msg = err.to_string(); + // Columns are canonicalized to sorted order in the catalog, so the + // message reads `(external_id, source)`; assert order-agnostically that + // both composite columns are named (not just the first, as pre-fix). + assert!( + msg.contains("@unique violation") + && msg.contains("source") + && msg.contains("external_id"), + "composite violation must name both columns (got: {msg})" + ); +} + /// Canary for the upstream Lance gap that the `FirstSeen` workaround /// in `table_store.rs` masks. The bug class is "Window 2": load → /// indices built explicitly → merge → merge. Even with the engine From 2f19656c0e5f4d0bcdc5263663786c631a51c5a4 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Tue, 9 Jun 2026 18:30:33 +0300 Subject: [PATCH 032/165] fix(cluster): tighten state lock observations --- crates/omnigraph-cli/tests/cli.rs | 7 +- crates/omnigraph-cluster/src/lib.rs | 104 +++++++++++++++++++--------- docs/user/cluster-config.md | 8 ++- 3 files changed, 81 insertions(+), 38 deletions(-) diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 627fd87..17b1f72 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -350,8 +350,9 @@ fn cluster_plan_json_includes_state_cas_revision_and_lock_observation() { .unwrap() .starts_with("sha256:") ); - assert_eq!(json["state_observations"]["locked"], true); - assert!(json["state_observations"]["lock_id"].is_string()); + assert_eq!(json["state_observations"]["locked"], false); + assert_eq!(json["state_observations"]["lock_acquired"], true); + assert!(json["state_observations"]["acquired_lock_id"].is_string()); assert!(!state_dir.join("lock.json").exists()); } @@ -386,6 +387,8 @@ fn cluster_plan_locked_state_exits_nonzero() { let json = parse_stdout_json(&output); assert_eq!(json["ok"], false); assert_eq!(json["state_observations"]["locked"], true); + assert_eq!(json["state_observations"]["lock_acquired"], false); + assert_eq!(json["state_observations"]["lock_id"], "held-lock"); assert!( json["diagnostics"] .as_array() diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 5115933..e308392 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -104,6 +104,9 @@ pub struct StateObservations { pub locked: bool, #[serde(skip_serializing_if = "Option::is_none")] pub lock_id: Option<String>, + pub lock_acquired: bool, + #[serde(skip_serializing_if = "Option::is_none")] + pub acquired_lock_id: Option<String>, } #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] @@ -213,7 +216,7 @@ struct LoadOutcome { config_file: PathBuf, } -#[derive(Debug, Deserialize)] +#[derive(Debug, Serialize, Deserialize)] #[serde(deny_unknown_fields)] struct RawClusterConfig { version: u32, @@ -227,20 +230,20 @@ struct RawClusterConfig { policies: BTreeMap<String, PolicyConfig>, } -#[derive(Debug, Default, Deserialize)] +#[derive(Debug, Default, Serialize, Deserialize)] #[serde(deny_unknown_fields)] struct Metadata { name: Option<String>, } -#[derive(Debug, Default, Deserialize)] +#[derive(Debug, Default, Serialize, Deserialize)] #[serde(deny_unknown_fields)] struct StateConfig { backend: Option<String>, lock: Option<bool>, } -#[derive(Debug, Deserialize)] +#[derive(Debug, Serialize, Deserialize)] #[serde(deny_unknown_fields)] struct GraphConfig { schema: PathBuf, @@ -248,13 +251,13 @@ struct GraphConfig { queries: BTreeMap<String, QueryConfig>, } -#[derive(Debug, Deserialize)] +#[derive(Debug, Serialize, Deserialize)] #[serde(deny_unknown_fields)] struct QueryConfig { file: PathBuf, } -#[derive(Debug, Deserialize)] +#[derive(Debug, Serialize, Deserialize)] #[serde(deny_unknown_fields)] struct PolicyConfig { file: PathBuf, @@ -605,6 +608,8 @@ impl LocalStateBackend { resource_count: 0, locked: false, lock_id: None, + lock_acquired: false, + acquired_lock_id: None, } } @@ -692,15 +697,19 @@ impl LocalStateBackend { .open(&self.lock_path) { Ok(mut file) => { - file.write_all(payload.as_bytes()).map_err(|err| { - Diagnostic::error( + if let Err(err) = file.write_all(payload.as_bytes()) { + // No guard exists yet, so clean up the create-new file here + // instead of leaving a stale partial lock for the next run. + drop(file); + let _ = fs::remove_file(&self.lock_path); + return Err(Diagnostic::error( "state_lock_error", CLUSTER_LOCK_FILE, format!("could not write state lock: {err}"), - ) - })?; - observations.locked = true; - observations.lock_id = Some(lock_id.clone()); + )); + } + observations.lock_acquired = true; + observations.acquired_lock_id = Some(lock_id.clone()); Ok(StateLockGuard { path: self.lock_path.clone(), }) @@ -794,22 +803,6 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { }; }; let settings = validate_cluster_header(&raw, &mut diagnostics); - let config_text = match fs::read_to_string(&config_file) { - Ok(text) => text, - Err(err) => { - diagnostics.push(Diagnostic::error( - "cluster_config_read_error", - CLUSTER_CONFIG_FILE, - format!("could not re-read cluster.yaml: {err}"), - )); - return LoadOutcome { - desired: None, - diagnostics, - config_dir, - config_file, - }; - } - }; let mut resources = BTreeMap::new(); let mut dependencies = BTreeSet::new(); @@ -1026,7 +1019,7 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { resource_list.push(resource); } let dependencies: Vec<_> = dependencies.into_iter().collect(); - let config_digest = desired_config_digest(&config_text, &resource_digests); + let config_digest = desired_config_digest(&raw, &resource_digests); LoadOutcome { desired: Some(DesiredCluster { @@ -1351,11 +1344,15 @@ fn graph_digest( } fn desired_config_digest( - config_source: &str, + raw: &RawClusterConfig, resource_digests: &BTreeMap<String, String>, ) -> String { let mut input = String::from("cluster-config\0"); - input.push_str(config_source); + // Hash parsed semantics, not raw YAML bytes, so comments and formatting do + // not create a new desired revision and the digest cannot drift from parse. + let config_semantics = + serde_json::to_string(raw).expect("raw cluster config must serialize deterministically"); + input.push_str(&config_semantics); input.push('\0'); for (address, digest) in resource_digests { input.push_str(address); @@ -1593,6 +1590,8 @@ graphs: let out = plan_config_dir(dir.path()); assert!(out.ok, "{:?}", out.diagnostics); assert!(!out.state_observations.state_found); + assert!(!out.state_observations.locked); + assert!(out.state_observations.lock_acquired); assert!( out.changes .iter() @@ -1602,6 +1601,40 @@ graphs: assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); } + #[test] + fn config_digest_ignores_yaml_comments_and_formatting() { + let dir = fixture(); + let first = plan_config_dir(dir.path()); + assert!(first.ok, "{:?}", first.diagnostics); + + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +# Same semantic config as the fixture, intentionally rendered differently. +version: 1 +metadata: { name: test } +state: { backend: cluster, lock: true } +graphs: + knowledge: + schema: ./people.pg + queries: { find_person: { file: ./people.gq } } +policies: + base: + file: ./base.policy.yaml + applies_to: + - knowledge +"#, + ) + .unwrap(); + + let second = plan_config_dir(dir.path()); + assert!(second.ok, "{:?}", second.diagnostics); + assert_eq!( + first.desired_revision.config_digest, + second.desired_revision.config_digest + ); + } + #[test] fn existing_state_plans_update_and_delete_deterministically() { let dir = fixture(); @@ -1775,8 +1808,10 @@ graphs: out.state_observations.state_cas.as_deref(), Some(format!("sha256:{}", sha256_hex(state.as_bytes())).as_str()) ); - assert!(out.state_observations.locked); - assert!(out.state_observations.lock_id.is_some()); + assert!(!out.state_observations.locked); + assert!(out.state_observations.lock_id.is_none()); + assert!(out.state_observations.lock_acquired); + assert!(out.state_observations.acquired_lock_id.is_some()); assert!( !dir.path().join(CLUSTER_LOCK_FILE).exists(), "plan must release lock before returning" @@ -1804,6 +1839,8 @@ graphs: assert!(!out.ok); assert!(out.state_observations.locked); assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); + assert!(!out.state_observations.lock_acquired); + assert!(out.state_observations.acquired_lock_id.is_none()); assert!( out.diagnostics .iter() @@ -1831,6 +1868,7 @@ graphs: let out = plan_config_dir(dir.path()); assert!(out.ok, "{:?}", out.diagnostics); assert!(!out.state_observations.locked); + assert!(!out.state_observations.lock_acquired); assert!( out.diagnostics .iter() diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 9fdbf55..8f4eab1 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -112,9 +112,11 @@ Missing `state_revision` is treated as `0`. Resource status values are Plan output compares desired resource digests against state resource digests and reports `create`, `update`, and `delete` changes. It also reports the state -CAS (`sha256:<digest>`), state revision, and lock id used for the read. The -command never writes `state.json`; apply, refresh, import, and live drift scans -are later-stage work. +CAS (`sha256:<digest>`) and state revision. `state_observations.locked` means an +existing lock file was observed; a successful `plan` instead reports +`lock_acquired: true` and an `acquired_lock_id`, then releases the lock before +returning. The command never writes `state.json`; apply, refresh, import, and +live drift scans are later-stage work. ## Status From dbfdddc952d4dbe7a9113f5fdc003749a3ca085c Mon Sep 17 00:00:00 2001 From: Ragnor Comerford <ragnor.comerford@gmail.com> Date: Tue, 9 Jun 2026 18:09:13 +0200 Subject: [PATCH 033/165] feat(engine): indexed graph traversal (#149) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * perf(engine): route Expand node hydration through the id BTREE via structured filter hydrate_nodes built an `id IN (...)` SQL string applied via Scanner::filter, which DataFusion evaluates with InListEval (O(N×M)) rather than using the id BTREE scalar index — measured at 72× the indexed cost on a 100k-node hop (MR-376). Build the id IN-list as a structured DataFusion Expr, AND it with the pushable destination filters, and apply via Scanner::filter_expr (the same path execute_node_scan already uses); Lance then compiles it to scalar-index-search -> take. Destination-filter pushability is now decided by ir_filter_to_expr (structured) instead of ir_filter_to_sql, so list-contains (array_has) pushes down too. Removes the now-dead string-filter helpers build_lance_filter, ir_filter_to_sql, and ir_expr_to_sql; literal_to_sql stays (still used by the mutation delete path). * feat(engine): add TableStore::scan_edges_by_endpoint for indexed neighbor lookup Static helper returning edge rows that match a set of endpoint keys on src/dst, projected to [key_col, opposite_col], via a structured `key_col IN (keys)` filter_expr. Lance routes it through the persisted BTREE on the endpoint column (index-search -> take), so cost scales with the frontier size rather than |E|. Unused until execute_expand's indexed mode lands; isolated in its own commit so the storage-layer primitive is reviewable on its own. * feat(engine): add BTREE-indexed Expand traversal path Split execute_expand into a dispatcher over execute_expand_csr (the existing in-memory CSR BFS, unchanged) and a new execute_expand_indexed that serves each hop by batching the frontier into one scan_edges_by_endpoint call against the persisted src/dst BTREE (index-search -> take), then fans out per source row. Both share expand_hydrate_and_align — the destination hydration + alignment + hconcat + in-memory non-pushable filters — which now aligns by string id (a HashMap) instead of a dense row-id vec, so one tail serves both modes. Mode selection is OMNIGRAPH_TRAVERSAL_MODE for now (default csr); the frontier-size auto policy and lazy CSR build follow. AntiJoin stays on CSR. tests/traversal_indexed.rs (its own #[serial] binary, so env writes never race a reader) asserts the indexed path matches CSR for one-hop, multi-hop, cross-type, and no-match cases, and that a freshly-appended unindexed edge is still found (partial index coverage — fast_search=false unindexed-fragment scan). * feat(engine): frontier-size Expand dispatcher + lazy CSR build Replace the env-only mode switch with an auto policy: Expand uses the BTREE-indexed path when the source frontier is small and the hop count bounded (OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER=1024, OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS=6), else the in-memory CSR. OMNIGRAPH_TRAVERSAL_MODE=indexed|csr still forces a mode. Make the CSR index lazy: thread a GraphIndexHandle (memoizing OnceCell over a Cached/Direct/None builder) through execute_query/execute_pipeline/ execute_rrf_query/execute_anti_join instead of a pre-built Option<&GraphIndex>. A query served entirely by the indexed path with no AntiJoin never pays the O(|E|) CSR build — the perf win of Tier 3. AntiJoin still realizes the index (its negation uses CSR has_neighbors). Net effect: selective traversals (the common case) skip the whole-graph CSR build and resolve neighbors from the persisted, incrementally-maintained src/dst BTREE. Existing traversal/aggregation/end_to_end/search suites now run the indexed path by default and stay green. Docs: constants.md (new env knobs), query-language.md (Expand dual path), indexes.md (graph index is lazy + the indexed alternative). * test(engine): bench indexed vs CSR selective traversal Add a selective single-source knows{1,2} comparison to bench_expand: per growing |E|, time the cold query in csr vs indexed mode (fresh db each, so CSR pays its O(|E|) build) and assert both modes return identical rows — a guard against the scalar-index physical_rows silent fallback dropping unindexed-fragment rows. The existing dense hop1/2/3 latency bench is unchanged. * feat(engine): surface silent scalar-index fallback in indexed traversal (C6) Add TableStore::key_column_index_coverage — a metadata-only check (no IO) of whether a `key_col IN (...)` scan will be served by the persisted BTREE or silently fall back to a full filtered scan, mirroring Lance's own decision: no BTREE on the column, or any fragment missing physical_rows (which disables scalar indices for the whole scan, lance dataset/scanner.rs create_filter_plan). execute_expand_indexed calls it once per traversal and tracing::warn!s on Degraded, so the perf cliff is observable instead of hidden behind a bench oracle. Detection-only: results are correct either way (the scan returns all rows). Closes the "no silent failures" gap the traversal best-practice audit flagged as the top deviation, and adds an IndexCoverage value a future cost-based planner can consume. * perf(engine): dense-id BFS on the indexed traversal path (C3) execute_expand_indexed ran its per-source BFS in string space (Vec<HashSet<String>>, HashMap<String,Vec<String>>, ~4 String clones per neighbor occurrence). Intern node ids to u32 once via a per-traversal TypeIndex (no GraphIndex/CSR build — laziness preserved) and run visited/seen/frontier/ neighbor-map in dense u32 space, mirroring the CSR path; de-intern only for the per-hop IN-list and the emitted dst ids handed to the hydrate+align tail. Behavior-preserving — the traversal_indexed CSR-vs-indexed equivalence tests are the guard (results are identical, the key type just changes String -> u32). * refactor(engine): thread the opened edge dataset into indexed Expand Hoist the edge-dataset open and the C6 index-coverage warning out of execute_expand_indexed into execute_expand, threading the opened dataset in as a parameter so it is opened exactly once. Extract the endpoint-column mapping (endpoint_columns) and the coverage warning (warn_on_degraded_coverage) as helpers. Behavior-preserving: same dataset, same warning, same dispatch decision. This only relocates the open so the upcoming cost-based chooser can consult index coverage before dispatch without opening the dataset twice. * feat(engine): cost-based Expand dispatch chooser (C5) Replace the fixed frontier<=1024 && hops<=6 dispatch threshold with a pure, IO-free cost model. choose_expand_mode compares the indexed path's frontier-relative work (hops * frontier * fanout, or hops * |E| when BTREE coverage is degraded) against the cost of building the whole-graph CSR (BUILD_FACTOR * |E|), from cheap manifest row counts. Under good coverage this reduces to a selectivity ratio independent of |E|, preserving the flat-in-|E| indexed win for selective traversals while routing dense / deep / high-fanout or degraded-and-expensive traversals to CSR. execute_expand decides cardinality-first and only opens the edge dataset to confirm coverage when it leans indexed (no open on a clearly-CSR traversal). The two env knobs become hard ceilings layered on the model; the OMNIGRAPH_TRAVERSAL_MODE override still forces a path; the chosen mode is traced. Results are unchanged across modes — only the path differs. Adds inline crossover unit tests and extends the traversal_indexed both_modes harness with an auto pass asserting the chooser is result-preserving across every traversal shape. Documents the new flag semantics in docs/user/{constants,query-language}.md. * test(engine): pin Lance scalar-index coverage + system-column/deletion-metadata surface Add three Lance surface guards de-risking a future persisted-adjacency cache: - a compile-only guard pinning the fragment physical_rows + index-detail surface that key_column_index_coverage mirrors (the C6 fallback); - a runtime probe confirming a scalar BTREE on the system column _row_last_updated_at_version is not buildable via the normal create-index path (the column is not in the user schema), so a version-column range delta is not viable as drafted; - a runtime probe confirming per-fragment deletion metadata (deletion_file.num_deleted_rows) is available as cheap O(fragments) metadata, the primitive a fragment-coverage delete model would rely on. The probes turn the two largest substrate assumptions into green/red CI facts before any cache work begins. * test(engine): regression for cross-type id-collision in indexed traversal A node id is unique only within a type, so a Person and a Company can share an id string. A variable-length traversal over a cross-type edge (WorksAt) must structurally stop after one hop. This test builds a graph where 'shared' is both a Person and a Company id and asserts worksAt{1,2} returns only the one-hop company. It fails today: the indexed path's single string interner de-interns the hop-1 Company id back to the colliding Person id and runs a hop-2 scan that matches that Person's edges, emitting a spurious second-hop company (indexed ["other","shared"] vs csr ["shared"]). * fix(engine): structurally cap cross-type Expand at one hop A cross-type edge cannot chain (e.g. a Company is not a WorksAt source), so a variable-length traversal over one is structurally single-hop. Both traversal paths now enforce this by capping max hops at 1 when from_type != to_type, instead of relying on the hop-2 scan returning empty. That reliance was a correctness hole on the indexed path: it interns every endpoint string into one dense id space, so a cross-type id-string collision (a Person and a Company sharing an id) let hop 2 de-intern a destination id back to the colliding source-type id and match its edges, emitting rows the CSR path never produces. With the cap the cross-type second-hop scan never runs, so the shared interner can no longer alias across types. Turns the regression test green (indexed == csr == ["shared"]). * perf(engine): set-oriented filtered anti-join, remove per-row dispatch execute_anti_join's filtered slow path sliced the outer batch to one row at a time and re-ran the inner pipeline per row, so each 1-row inner Expand dispatched to the indexed path — one Lance scan per outer row, while the CSR realized up front sat unused. Replace it with a set-oriented anti-semi-join: tag each outer row with a synthetic index column, run the inner pipeline once over the whole frontier (the tag survives Expand's hconcat and Filter's row-drop), then exclude outer rows whose tag survived. The inner Expand now runs as a single set-at-a-time traversal over the full frontier; config is read once per operator, not per row (the env nit is mooted). A produced-but-untagged inner batch fails loudly rather than silently keeping every row. Results are unchanged (the predicated-negation tests exercise the path over a multi-row outer with dst-filters). * test(engine): drop flaky wall-clock budget from the merge truth table The 30s wall-clock assertion in merge_pair_truth_table flakes under parallel test load: it tripped at ~31s in the full --test-threads=4 gate while passing at ~20s in isolation. A fixed time budget in a correctness test depends on machine and parallelism, not correctness; elapsed is still logged for visibility, and a real merge-perf regression belongs in a bench. The cell-count correctness assertions (81 / 36 / 45) are unchanged. * fix(engine): total deterministic ORDER via entity-key tie-break + NULL contract apply_ordering used an unstable lexsort with no tie-break, so rows with equal user-sort keys came out in a run-dependent order (the input order depends on scan parallelism / upstream hashing) — making ORDER ... LIMIT non-deterministic, a latent deny-list violation (no nondeterministic result ordering). Append the bound entities' key columns (<var>.id, unique per row) in canonical name-sorted order as ascending tie-breaks, giving a total, reproducible order (and a deterministic top-N when ties straddle the LIMIT cutoff). NULL placement (nulls_first = !descending) is unchanged and now documented as the contract. New tests/ordering.rs locks descending, multi-key precedence, the deterministic key tie-break (data loaded in a different order than the expected output, so it proves the tie sorts by key not by load order), and NULL placement under ASC/DESC. docs/user/query-language.md documents the total-order + NULL contract. * test(engine): property-based query-correctness invariants over generated graphs Adds a proptest harness (new dev-dep) that generates small graphs whose Person and Company keys are drawn from a shared 5-key alphabet, so cross-type id collisions, cycles, and self-loops arise by search rather than from one hand-built fixture. Three invariants: - prop_expand_indexed_eq_csr: csr == indexed == auto over knows{1,3} (same-type, cycles) and worksAt{1,2} (cross-type, collision-prone) from every start. - prop_results_subset_of_existing_nodes: no phantom rows (catches over-emission even if both modes are wrong identically). - prop_antijoin_partitions_persons: not{worksAt} and its complement are disjoint and cover all persons. Verified the guard bites: neutering the cross-type hop cap makes prop_expand_indexed_eq_csr fail and proptest shrinks it to persons["c","e"] / companies["b","c"] — the cross-type collision class the hand-built fixture only sampled once. Tests are sync + #[serial] (per-case runtime; the mode test writes OMNIGRAPH_TRAVERSAL_MODE). * test(engine): cover cycle/self-loop termination + nested anti-join (C5 edge cases) - variable_hops_terminate_and_dedup_on_cycle: a 3-cycle a->b->c->a traversed with knows{1,5} (ceiling above the cycle length) terminates and emits each node once (the c->a back-edge hits the seeded source); both_modes confirms indexed == csr. Uses a bounded range deliberately — unbounded {1,} is a typecheck error, not a runtime path. - variable_hops_handle_self_loop: a->a self-loop does not loop forever and does not re-emit the seeded source. - nested_anti_join_double_negation: not { worksAt; not { name = Acme } } recurses through execute_pipeline, yielding [Alice,Charlie,Diana] (people with no non-Acme employer) — distinct from plain unemployed [Charlie,Diana]. * test(engine): execution goldens for typed-literal filters (C4 gap #4) New literal_filters.rs covers filtering by F64/F32/Bool/Date/DateTime LITERALS across both arms: standalone comparisons ($m.score > 1.5, $m.ratio <= 0.25, $m.active = true, $m.born >= date(...), $m.seen < datetime(...)) exercise the in-memory comparison path, and inline bindings (Metric { active: true }, Metric { score: 3.0 }) exercise Lance filter_expr pushdown. Seeds partition each predicate so a dropped/miscast filter returns all rows. (Param-bound scalars and list-column contains are covered elsewhere.) * test(engine): full rank-order goldens for nearest + bm25 (gap #2) Existing search tests stopped at top-1 (nearest) or non-empty (bm25), so a regression corrupting ranks 2..k or reversing the sort direction passed CI silently. Pin the FULL ordered slug list: nearest([0.1,0.2,0.3,0.4]) -> [ml-intro, nlp-guide, rl-intro] (ml-intro exact at dist 0, rest by ascending L2); bm25(Learning) -> [rl-intro, ml-intro, dl-basics] (descending score). nearest/bm25 skip apply_ordering (is_search_ordered) and return Lance native order, so result_slugs row order == rank order; values resolved by running and confirmed stable across runs. * test(engine): search fuzzy/match_text characterization + RRF non-default pairings - match_text_matches_exact_set_excludes_unrelated: match_text(body,'neural') == [dl-basics] exactly (not just contains). - fuzzy_does_not_match_under_default_tokenizer: characterizes that fuzzy() is inert with the default tokenizer here (search/match_text work, fuzzy returns nothing); turns red — to be promoted to a real golden — if fuzzy starts matching. - rrf_fuses_two_fts_fields / rrf_fuses_two_vector_queries: RRF fuses arms other than the default nearest+bm25 (bm25 title+body; two vector queries), proving primary_var resolves and fusion runs. New fixtures/search.gq queries + two_vector_params helper. Orders resolved by running, confirmed stable. * test(engine): anti-join fast-vs-slow path equivalence harness anti_join_fast_and_slow_paths_agree: the CSR has_neighbors fast path (not { $p worksAt $_ }) and the set-oriented inner-pipeline replay (same negation forced slow by an always-true $c.name != "" dst filter) must produce the same result ([Charlie, Diana]). Closes the second real engine fork explicitly. * test(engine): regression for nested slow-path anti-join tag collision A nested not { ... not { ... } } where both levels hit the set-oriented slow path collides on the fixed __antijoin_outer_row correlation column: the inner call appends a duplicate, and column_by_name reads the OUTER tag. Fan-out (p1 works at two companies) makes inner row indices diverge from outer tags, so the bug returns the wrong person set. Fails on current code (left ["p2","p4"] vs right ["p3","p4"]). * fix(engine): collision-free anti-join correlation tag for nested negation The set-oriented anti-join tagged the outer batch with a fixed column name and read it back by name. Under a nested slow-path anti-join the enclosing tag rides through the inner pipeline, so the inner call produced a duplicate field; Arrow permits duplicate names and column_by_name returns the first, so the inner negation mis-correlated against the outer row indices. Choose a tag name not already present in the batch (suffix-incremented), so each nesting level reads its own correlation column. Turns the fan-out regression green; the existing nested/fast-vs-slow/proptest anti-join invariants still pass. * fix(engine): cap cross-type hops in the Expand cost model gather_cost_inputs fed the requested max_hops into choose_expand_mode even though execute_expand_indexed runs at most one hop for a cross-type edge. So a cross-type variable-length expand (e.g. worksAt{1,5}) had its indexed cost scaled by 5 while only one hop runs, skewing the chooser toward CSR (an unnecessary whole-graph build) near the crossover. Results were unaffected (modes are equivalent); this is a plan-accuracy fix. Add cost_effective_hops(requested, same_type) — caps to 1 for cross-type — and apply it in gather_cost_inputs so the estimate matches what executes. Unit test covers the cap and the crossover consequence (capped 1 hop stays indexed where the requested 5 would have flipped to CSR). * perf(engine): realize anti-join CSR lazily + reuse a warm CSR in the chooser Two CSR build/reuse fixes flagged on the set-oriented anti-join work (results unchanged — plan/perf accuracy): - execute_anti_join called graph_index.get() (the O(|E|) whole-graph CSR build) unconditionally, but only the bulk fast path consumes it; a filtered/nested slow-path anti-join's inner Expand picks its own access path. Gate the build on a pure shape predicate (bulk_anti_join_applies) so a selective anti-join over a large graph no longer pays a build it won't use. - gather_cost_inputs hardcoded csr_cached=false, so once an earlier op realized the CSR, later Expands still cost it as a cold build and could pick per-hop indexed scans over reusing the warm in-memory CSR. Add GraphIndexHandle:: is_built() and thread it through so the chooser reuses a materialized CSR. Anti-join, cross-type, proptest-equivalence, and chooser unit tests stay green. * test(engine): RAII traversal-mode guard in proptest equivalence prop_expand_indexed_eq_csr set/cleared OMNIGRAPH_TRAVERSAL_MODE manually; a panic between set and clear (e.g. a query unwrap on a generated case) would leak the forced mode into proptest's shrink/subsequent cases and mask the divergence under test. Replace with a ModeGuard that clears on drop (including on unwind), scoping the forced mode to a single query. * test(engine): regression for multi-hop anti-join hop bounds The bulk anti-join fast path answers via has_neighbors (one-hop existence), so not { $p knows{2,2} $x } wrongly drops a node with a 1-hop neighbor but no 2-hop path. On a->b (sink) and c->d->e, only c has a 2-hop path; the query should keep [a,b,d,e]. Fails on current code (left ["b","e"] — only the sinks). * fix(engine): restrict anti-join bulk fast path to one-hop expands bulk_anti_join_applies accepted any single Expand, but try_bulk_anti_join_mask decides via the CSR has_neighbors one-hop existence check — wrong for multi-hop negations. Require min_hops==1 && max_hops==1 in the predicate; anything else falls to the slow path, whose inner Expand runs the real bounded traversal. Turns the multi-hop regression green; one-hop anti-joins unchanged. * fix(engine): IndexCoverage reports Degraded for uncovered fragments key_column_index_coverage checked BTREE-exists + physical_rows but not that the index actually covers the current fragments. Since edge-index creation is skipped once a BTREE exists, fragments appended later stay unindexed while coverage still reported Indexed — so the cost chooser priced a partly-full scan as fully indexed. Compare the BTREE's fragment_bitmap (public on lance_table IndexMetadata) against the dataset's current fragment ids; report Degraded when any are uncovered. A None bitmap means Lance can't report coverage — don't over-degrade. Results are unaffected (the scan returns unindexed-fragment rows either way); this corrects the cost signal. Test: a freshly-loaded edge BTREE is Indexed; after appending an edge the new fragment is uncovered → Degraded. Surface guard pins IndexMetadata.fragment_bitmap. * docs: clarify the Expand frontier ceiling bounds the initial dispatch frontier The cap is applied at dispatch on the initial frontier; per-hop fan-out (union_dense) is not hard-capped. Correct the constants.md and query-language.md claims: the ceilings bound the initial-dispatch frontier/hops, the cost model estimates total indexed work as ~hops*frontier*fanout (pricing dense fan-out toward CSR), and per-hop work is not a hard bound. Drops the overstated 'hard caps bound indexed work' / 'cost ∝ frontier' wording. --- Cargo.lock | 53 + crates/omnigraph/Cargo.toml | 1 + crates/omnigraph/examples/bench_expand.rs | 61 + crates/omnigraph/src/exec/projection.rs | 29 + crates/omnigraph/src/exec/query.rs | 1146 ++++++++++++++--- crates/omnigraph/src/table_store.rs | 124 ++ crates/omnigraph/tests/fixtures/search.gq | 14 + crates/omnigraph/tests/helpers/mod.rs | 9 + .../omnigraph/tests/lance_surface_guards.rs | 135 ++ crates/omnigraph/tests/literal_filters.rs | 96 ++ crates/omnigraph/tests/merge_truth_table.rs | 8 +- crates/omnigraph/tests/ordering.rs | 134 ++ .../omnigraph/tests/proptest_equivalence.rs | 311 +++++ crates/omnigraph/tests/search.rs | 105 ++ crates/omnigraph/tests/traversal.rs | 188 +++ crates/omnigraph/tests/traversal_indexed.rs | 327 +++++ docs/user/constants.md | 17 + docs/user/indexes.md | 4 +- docs/user/query-language.md | 4 +- 19 files changed, 2570 insertions(+), 196 deletions(-) create mode 100644 crates/omnigraph/tests/literal_filters.rs create mode 100644 crates/omnigraph/tests/ordering.rs create mode 100644 crates/omnigraph/tests/proptest_equivalence.rs create mode 100644 crates/omnigraph/tests/traversal_indexed.rs diff --git a/Cargo.lock b/Cargo.lock index 3064196..578188c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4627,6 +4627,7 @@ dependencies = [ "object_store 0.12.5", "omnigraph-compiler", "omnigraph-policy", + "proptest", "regex", "reqwest", "serde", @@ -5141,6 +5142,25 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "proptest" +version = "1.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4b45fcc2344c680f5025fe57779faef368840d0bd1f42f216291f0dc4ace4744" +dependencies = [ + "bit-set", + "bit-vec", + "bitflags", + "num-traits", + "rand 0.9.2", + "rand_chacha 0.9.0", + "rand_xorshift", + "regex-syntax", + "rusty-fork", + "tempfile", + "unarray", +] + [[package]] name = "prost" version = "0.14.3" @@ -5202,6 +5222,12 @@ dependencies = [ "cc", ] +[[package]] +name = "quick-error" +version = "1.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a1d01941d82fa2ab50be1e79e6714289dd7cde78eba4c074bc5a4374f650dfe0" + [[package]] name = "quick-xml" version = "0.37.5" @@ -5373,6 +5399,15 @@ dependencies = [ "rand 0.9.2", ] +[[package]] +name = "rand_xorshift" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "513962919efc330f829edb2535844d1b912b0fbe2ca165d613e4e8788bb05a5a" +dependencies = [ + "rand_core 0.9.5", +] + [[package]] name = "rand_xoshiro" version = "0.7.0" @@ -5772,6 +5807,18 @@ version = "1.0.22" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d" +[[package]] +name = "rusty-fork" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cc6bf79ff24e648f6da1f8d1f011e9cac26491b619e6b9280f2b47f1774e6ee2" +dependencies = [ + "fnv", + "quick-error", + "tempfile", + "wait-timeout", +] + [[package]] name = "ryu" version = "1.0.23" @@ -6759,6 +6806,12 @@ dependencies = [ "web-time", ] +[[package]] +name = "unarray" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eaea85b334db583fe3274d12b4cd1880032beab409c0d774be044d4480ab9a94" + [[package]] name = "unicase" version = "2.9.0" diff --git a/crates/omnigraph/Cargo.toml b/crates/omnigraph/Cargo.toml index 24b0c9c..9cc2148 100644 --- a/crates/omnigraph/Cargo.toml +++ b/crates/omnigraph/Cargo.toml @@ -55,3 +55,4 @@ omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } tokio = { workspace = true } lance-namespace-impls = { workspace = true } serial_test = "3" +proptest = "1" diff --git a/crates/omnigraph/examples/bench_expand.rs b/crates/omnigraph/examples/bench_expand.rs index c723b24..bb904a0 100644 --- a/crates/omnigraph/examples/bench_expand.rs +++ b/crates/omnigraph/examples/bench_expand.rs @@ -221,6 +221,65 @@ fn microbench_dedup() { ); } +/// Selective single-source traversal, timed cold in CSR vs indexed mode across +/// growing |E|. The win of the indexed path: a small fixed frontier should be +/// ~flat in |E| (one BTREE scan per hop), whereas CSR pays an O(|E|) adjacency +/// build on the first (cold) query. Also asserts both modes return the same +/// rows — a guard against the scalar-index `physical_rows` silent fallback +/// dropping unindexed-fragment rows. +async fn bench_selective_modes() { + println!("\n── Selective traversal: indexed vs CSR (cold, single-source knows{{1,2}}) ──"); + let sel = r#" +query sel($name: String) { + match { + $a: Person { name: $name } + $a knows{1,2} $b + } + return { $b.name } +} +"#; + for &(n, avg_deg) in &[(1_000usize, 8usize), (10_000, 8), (30_000, 8)] { + let jsonl = generate_jsonl(n, avg_deg, 42); + let mut params = ParamMap::new(); + params.insert( + "name".to_string(), + omnigraph_compiler::query::ast::Literal::String("p0".to_string()), + ); + + let mut rows_by_mode: Vec<(&str, usize)> = Vec::new(); + for mode in ["csr", "indexed"] { + // Fresh db per measurement so the query is cold (CSR pays its build). + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, SCHEMA).await.unwrap(); + load_jsonl(&mut db, &jsonl, LoadMode::Overwrite).await.unwrap(); + // SAFE: example main drives queries sequentially; no concurrent env reader. + unsafe { std::env::set_var("OMNIGRAPH_TRAVERSAL_MODE", mode) }; + + let t = Instant::now(); + let r = db + .query(ReadTarget::branch("main"), sel, "sel", ¶ms) + .await + .expect("sel query"); + let elapsed = t.elapsed(); + let rows = r.num_rows(); + rows_by_mode.push((mode, rows)); + println!( + " |E|≈{:>7} {:<8} cold={:>9.2?} rows={}", + n * avg_deg, + mode, + elapsed, + rows + ); + } + unsafe { std::env::remove_var("OMNIGRAPH_TRAVERSAL_MODE") }; + assert_eq!( + rows_by_mode[0].1, rows_by_mode[1].1, + "indexed and CSR must return identical rows (no silent drop under partial index coverage)" + ); + } +} + #[tokio::main(flavor = "multi_thread")] async fn main() { println!("── End-to-end query latency ──"); @@ -262,5 +321,7 @@ async fn main() { } } + bench_selective_modes().await; + microbench_dedup(); } diff --git a/crates/omnigraph/src/exec/projection.rs b/crates/omnigraph/src/exec/projection.rs index dec13a8..7280ec5 100644 --- a/crates/omnigraph/src/exec/projection.rs +++ b/crates/omnigraph/src/exec/projection.rs @@ -422,6 +422,35 @@ pub(super) fn apply_ordering( }); } + // Deterministic tie-break for a TOTAL order. `lexsort_to_indices` is unstable + // and the input row order is not guaranteed (scan parallelism, upstream + // hashing), so equal user-sort keys would otherwise come out run-dependent — + // making `ORDER ... LIMIT` non-deterministic. Append the bound entities' key + // columns (`<var>.id`, unique per row) in canonical (name-sorted) order as + // ascending tie-breaks. The combination of all bound keys uniquely identifies + // a result row, so the order is total and reproducible. (Aggregate results + // have no `.id` columns; their group rows are already distinct on the + // projected group keys.) + let mut tiebreak_cols: Vec<String> = source + .schema() + .fields() + .iter() + .map(|f| f.name().to_string()) + .filter(|name| name.ends_with(".id")) + .collect(); + tiebreak_cols.sort(); + for name in &tiebreak_cols { + if let Some(col) = source.column_by_name(name) { + sort_columns.push(SortColumn { + values: col.clone(), + options: Some(arrow_schema::SortOptions { + descending: false, + nulls_first: true, + }), + }); + } + } + let indices = lexsort_to_indices(&sort_columns, None).map_err(|e| OmniError::Lance(e.to_string()))?; diff --git a/crates/omnigraph/src/exec/query.rs b/crates/omnigraph/src/exec/query.rs index 7590512..5bc18f2 100644 --- a/crates/omnigraph/src/exec/query.rs +++ b/crates/omnigraph/src/exec/query.rs @@ -24,20 +24,14 @@ impl Omnigraph { .pipeline .iter() .any(|op| matches!(op, IROp::Expand { .. } | IROp::AntiJoin { .. })); + // Lazy: an index-served query with no AntiJoin never builds the CSR. let graph_index = if needs_graph { - Some(self.graph_index_for_resolved(&resolved).await?) + GraphIndexHandle::cached(self, &resolved) } else { - None + GraphIndexHandle::none() }; - execute_query( - &ir, - params, - &resolved.snapshot, - graph_index.as_deref(), - &catalog, - ) - .await + execute_query(&ir, params, &resolved.snapshot, &graph_index, &catalog).await } /// Run a named query against the graph as it existed at a prior manifest version. @@ -64,18 +58,21 @@ impl Omnigraph { .pipeline .iter() .any(|op| matches!(op, IROp::Expand { .. } | IROp::AntiJoin { .. })); + // Lazy build against this historical snapshot (not the RuntimeCache, + // which is keyed to live branch targets); only a CSR-path Expand or an + // AntiJoin triggers it. let graph_index = if needs_graph { let edge_types = catalog .edge_types .iter() .map(|(name, et)| (name.clone(), (et.from_type.clone(), et.to_type.clone()))) .collect(); - Some(Arc::new(GraphIndex::build(&snapshot, &edge_types).await?)) + GraphIndexHandle::direct(&snapshot, edge_types) } else { - None + GraphIndexHandle::none() }; - execute_query(&ir, params, &snapshot, graph_index.as_deref(), &catalog).await + execute_query(&ir, params, &snapshot, &graph_index, &catalog).await } } @@ -342,7 +339,7 @@ pub async fn execute_query( ir: &QueryIR, params: &ParamMap, snapshot: &Snapshot, - graph_index: Option<&GraphIndex>, + graph_index: &GraphIndexHandle<'_>, catalog: &Catalog, ) -> Result<QueryResult> { let search_mode = extract_search_mode(ir, params, catalog).await?; @@ -400,7 +397,7 @@ async fn execute_rrf_query( ir: &QueryIR, params: &ParamMap, snapshot: &Snapshot, - graph_index: Option<&GraphIndex>, + graph_index: &GraphIndexHandle<'_>, catalog: &Catalog, rrf: &RrfMode, ) -> Result<QueryResult> { @@ -583,7 +580,7 @@ fn execute_pipeline<'a>( pipeline: &'a [IROp], params: &'a ParamMap, snapshot: &'a Snapshot, - graph_index: Option<&'a GraphIndex>, + graph_index: &'a GraphIndexHandle<'a>, catalog: &'a Catalog, wide: &'a mut Option<RecordBatch>, search_mode: &'a SearchMode, @@ -653,13 +650,10 @@ fn execute_pipeline<'a>( max_hops, dst_filters, } => { - let gi = graph_index.ok_or_else(|| { - OmniError::manifest("graph index required for traversal".to_string()) - })?; if let Some(batch) = wide.as_mut() { execute_expand( batch, - gi, + graph_index, snapshot, catalog, src_var, @@ -688,8 +682,671 @@ fn execute_pipeline<'a>( }) } -/// Execute a graph traversal (Expand). +/// Lazily provides the in-memory CSR graph index, building it on first use and +/// memoizing for the rest of the query. Indexed-mode Expand never asks for it, +/// so a query that is entirely index-served and has no AntiJoin never pays the +/// O(|E|) CSR build (the whole point of the indexed path). The `Cached` builder +/// also reuses the cross-query `RuntimeCache` entry; `Direct` builds against an +/// arbitrary snapshot (time-travel reads); `None` is for queries with no +/// traversal at all. +pub struct GraphIndexHandle<'a> { + cell: tokio::sync::OnceCell<Option<Arc<GraphIndex>>>, + builder: GraphIndexBuilder<'a>, +} + +enum GraphIndexBuilder<'a> { + None, + Cached(&'a Omnigraph, &'a crate::db::ResolvedTarget), + Direct(&'a Snapshot, HashMap<String, (String, String)>), +} + +impl<'a> GraphIndexHandle<'a> { + fn none() -> Self { + Self { + cell: tokio::sync::OnceCell::new(), + builder: GraphIndexBuilder::None, + } + } + + fn cached(db: &'a Omnigraph, resolved: &'a crate::db::ResolvedTarget) -> Self { + Self { + cell: tokio::sync::OnceCell::new(), + builder: GraphIndexBuilder::Cached(db, resolved), + } + } + + fn direct(snapshot: &'a Snapshot, edge_types: HashMap<String, (String, String)>) -> Self { + Self { + cell: tokio::sync::OnceCell::new(), + builder: GraphIndexBuilder::Direct(snapshot, edge_types), + } + } + + /// The CSR index, built on first call. `None` only when the query needs no + /// traversal (the `None` builder). + async fn get(&self) -> Result<Option<&GraphIndex>> { + let built = self + .cell + .get_or_try_init(|| async { + match &self.builder { + GraphIndexBuilder::None => Ok::<Option<Arc<GraphIndex>>, OmniError>(None), + GraphIndexBuilder::Cached(db, resolved) => { + Ok(Some(db.graph_index_for_resolved(resolved).await?)) + } + GraphIndexBuilder::Direct(snapshot, edge_types) => { + Ok(Some(Arc::new(GraphIndex::build(snapshot, edge_types).await?))) + } + } + }) + .await?; + Ok(built.as_deref()) + } + + /// Whether the in-memory CSR is already materialized for this query (a prior + /// Expand or bulk AntiJoin realized it), so reusing it is ~free. Lets the + /// cost chooser prefer the warm CSR over per-hop indexed scans. + fn is_built(&self) -> bool { + matches!(self.cell.get(), Some(Some(_))) + } +} + +/// Explicit traversal-mode override. `OMNIGRAPH_TRAVERSAL_MODE=indexed|csr` +/// forces the path (ops escape hatch + test hook). Both modes are semantically +/// identical, so the override only changes which path runs, never the result. +fn traversal_indexed_override() -> Option<bool> { + match std::env::var("OMNIGRAPH_TRAVERSAL_MODE").ok().as_deref() { + Some("indexed") => Some(true), + Some("csr") => Some(false), + _ => None, + } +} + +/// Max source-row frontier for which Expand uses the BTREE-indexed path. +/// Larger frontiers fall back to the in-memory CSR (dense / whole-graph). See +/// `docs/user/constants.md`. +const DEFAULT_EXPAND_INDEXED_MAX_FRONTIER: usize = 1024; +/// Max hop count for the indexed path (each hop is one indexed scan; very deep +/// traversals fan out toward whole-graph and are better served by CSR). +const DEFAULT_EXPAND_INDEXED_MAX_HOPS: u32 = 6; + +fn expand_indexed_max_frontier() -> usize { + std::env::var("OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER") + .ok() + .and_then(|v| v.parse::<usize>().ok()) + .unwrap_or(DEFAULT_EXPAND_INDEXED_MAX_FRONTIER) +} + +fn expand_indexed_max_hops() -> u32 { + std::env::var("OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS") + .ok() + .and_then(|v| v.parse::<u32>().ok()) + .filter(|&v| v > 0) + .unwrap_or(DEFAULT_EXPAND_INDEXED_MAX_HOPS) +} + +/// The two Expand execution paths the chooser dispatches between. Extensible: +/// a future persisted-adjacency artifact would become a third variant here, and +/// `choose_expand_mode` would learn to prefer it when covered. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum ExpandMode { + /// Per-hop neighbor lookup via the persisted src/dst BTREE. Work scales + /// with the frontier, not |E| — best for selective traversals. + IndexedScan, + /// Whole-graph in-memory CSR (built once, reused). Best for dense / deep / + /// large-frontier traversals, or when the index is degraded and a full + /// scan would be paid per hop anyway. + Csr, +} + +/// Building the in-memory CSR costs more than a bare edge scan: it scans every +/// edge AND allocates + groups the adjacency. This factor expresses that +/// overhead so a one-off degraded single-hop scan can still edge out a full CSR +/// build. The crossover is insensitive to its exact value. +const CSR_BUILD_FACTOR: f64 = 1.5; + +/// Cardinality inputs for the (pure, IO-free) traversal-mode cost model. Every +/// field is a cheap manifest-resident count or an already-in-hand value — the +/// chooser performs no scans. +#[derive(Debug, Clone)] +struct ExpandCostInputs { + /// Current frontier size (`wide.num_rows()`). + frontier_rows: usize, + /// |E| for the edge type (manifest `row_count`). + edge_count: u64, + /// |V_src| — node count of the keyed endpoint type (manifest `row_count`). + src_node_count: u64, + /// Effective max hop count for this Expand. + effective_max_hops: u32, + /// Hard ceiling above which the indexed path is never used (resolved + /// `OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS`). + max_hops_cap: u32, + /// Hard ceiling above which the indexed path is never used (resolved + /// `OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER`). + max_frontier_cap: usize, + /// Whether `scan_edges_by_endpoint`'s `key_col IN (...)` is served by the + /// BTREE (`Indexed`) or silently falls back to a full scan (`Degraded`). + coverage: crate::table_store::IndexCoverage, + /// Whether the cross-query CSR for this snapshot+edge-version is already + /// built (making the CSR path ≈ free). Conservatively `false` until the + /// cache-peek is wired (the plan's optional refinement). + csr_cached: bool, +} + +/// Pure cost-based traversal-mode chooser. Compares an estimate of the indexed +/// path's frontier-relative work against the cost of building (or reusing) the +/// whole-graph CSR, and picks the cheaper. Deterministic and IO-free so it is +/// unit-tested at the crossover; the caller supplies the manifest counts and the +/// (optionally degraded) index coverage. +/// +/// Under `Indexed` coverage and a cold CSR the decision reduces to a clean +/// selectivity ratio — indexed wins when `hops * frontier < BUILD_FACTOR * +/// |V_src|`, i.e. when the frontier is a small fraction of the source vertex +/// set — which is independent of |E| (the flat-in-|E| property PR #149 shipped). +fn choose_expand_mode(i: &ExpandCostInputs) -> ExpandMode { + // Hard ceilings: very deep or very large frontiers fan out toward + // whole-graph and are always better served by CSR, regardless of the cost + // estimate. These preserve the documented semantics of the two cap flags. + if i.effective_max_hops > i.max_hops_cap || i.frontier_rows > i.max_frontier_cap { + return ExpandMode::Csr; + } + + let hops = i.effective_max_hops.max(1) as f64; + let frontier = i.frontier_rows as f64; + let edges = i.edge_count as f64; + let src = i.src_node_count.max(1) as f64; + let fanout = edges / src; + + // Indexed work scales with the frontier when the BTREE serves the IN-list; + // a degraded scan is a full edge scan per hop instead (the C6 perf cliff). + let indexed_cost = match i.coverage { + crate::table_store::IndexCoverage::Indexed => hops * frontier * fanout, + crate::table_store::IndexCoverage::Degraded { .. } => hops * edges, + }; + // A warm CSR is ~free to reuse; a cold one costs a build over all edges. + let csr_cost = if i.csr_cached { + 0.0 + } else { + CSR_BUILD_FACTOR * edges + }; + + if indexed_cost < csr_cost { + ExpandMode::IndexedScan + } else { + ExpandMode::Csr + } +} + +/// Hops the indexed path will actually run, for cost-model purposes. A cross-type +/// edge cannot chain, so `execute_expand_indexed` caps it at one hop regardless of +/// the requested range; the cost model must use that, or it over-estimates the +/// indexed cost of a cross-type variable-length expand and skews toward CSR. +fn cost_effective_hops(requested_max_hops: u32, same_type: bool) -> u32 { + if same_type { + requested_max_hops + } else { + requested_max_hops.min(1) + } +} + +/// Gather the cost-model inputs from cheap manifest counts. `None` when the +/// edge type, its source node type, or their manifest entries are absent (e.g. +/// a not-yet-materialized table) — the caller then falls back to the legacy +/// frontier/hop ceiling so the decision is always defined. +fn gather_cost_inputs( + snapshot: &Snapshot, + catalog: &Catalog, + edge_type: &str, + direction: Direction, + frontier_rows: usize, + effective_max_hops: u32, + coverage: crate::table_store::IndexCoverage, + csr_cached: bool, +) -> Option<ExpandCostInputs> { + let edge_entry = snapshot.entry(&format!("edge:{}", edge_type))?; + let edge_def = catalog.edge_types.get(edge_type)?; + // Match the indexed path's cross-type one-hop cap so the cost estimate + // reflects what actually runs (see `cost_effective_hops`). + let effective_max_hops = + cost_effective_hops(effective_max_hops, edge_def.from_type == edge_def.to_type); + // The frontier source vertices are the keyed endpoint's type: `from` for an + // Out traversal (keyed on `src`), `to` for In (keyed on `dst`). + let src_type = match direction { + Direction::Out => &edge_def.from_type, + Direction::In => &edge_def.to_type, + }; + let src_entry = snapshot.entry(&format!("node:{}", src_type))?; + Some(ExpandCostInputs { + frontier_rows, + edge_count: edge_entry.row_count, + src_node_count: src_entry.row_count, + effective_max_hops, + max_hops_cap: expand_indexed_max_hops(), + max_frontier_cap: expand_indexed_max_frontier(), + coverage, + csr_cached, + }) +} + +/// Coverage value to feed the cost decision. A failed coverage probe is treated +/// as `Degraded` (conservative: don't over-favor the indexed path when we can't +/// confirm the BTREE will serve the scan). +fn coverage_for_decision( + coverage: &Result<crate::table_store::IndexCoverage>, +) -> crate::table_store::IndexCoverage { + match coverage { + Ok(c) => c.clone(), + Err(_) => crate::table_store::IndexCoverage::Degraded { + reason: "coverage check failed".to_string(), + }, + } +} + +/// Surface the C6 silent scalar-index fallback (commit `5a7ab6d`): warn when the +/// per-hop `key_col IN (...)` won't route through the BTREE. Detection-only; +/// never fails the query. Behavior-identical to the inline check it replaced. +fn warn_on_degraded_coverage( + coverage: &Result<crate::table_store::IndexCoverage>, + key_col: &str, + edge_type: &str, +) { + match coverage { + Ok(crate::table_store::IndexCoverage::Degraded { reason }) => tracing::warn!( + target: "omnigraph::traverse", + edge = %edge_type, + key_col = key_col, + reason = %reason, + "indexed traversal falls back to a full edge scan (results correct, perf degraded)" + ), + Ok(crate::table_store::IndexCoverage::Indexed) => {} + Err(e) => tracing::debug!( + target: "omnigraph::traverse", + error = %e, + "index-coverage check failed; proceeding with traversal" + ), + } +} + +/// The (key, opposite) endpoint columns for a traversal direction. Out follows +/// src -> dst (key on src); In follows the reverse. The persisted BTREE exists +/// on both columns. +fn endpoint_columns(direction: Direction) -> (&'static str, &'static str) { + match direction { + Direction::Out => ("src", "dst"), + Direction::In => ("dst", "src"), + } +} + +/// Execute a graph traversal (Expand). Dispatches to the BTREE-indexed path +/// (selective traversals — neighbor lookups via the persisted src/dst index) or +/// the in-memory CSR path (dense / whole-graph traversals). The CSR index is +/// built lazily and only the CSR path requests it. async fn execute_expand( + wide: &mut RecordBatch, + graph_index: &GraphIndexHandle<'_>, + snapshot: &Snapshot, + catalog: &Catalog, + src_var: &str, + dst_var: &str, + edge_type: &str, + direction: Direction, + dst_type: &str, + min_hops: u32, + max_hops: Option<u32>, + dst_filters: &[IRFilter], + params: &ParamMap, +) -> Result<()> { + let frontier_rows = wide.num_rows(); + let effective_max_hops = max_hops.unwrap_or(min_hops.max(1)); + let (key_col, _) = endpoint_columns(direction); + let edge_table_key = format!("edge:{}", edge_type); + + // Cardinality-first preliminary decision (no IO). The override wins; else the + // cost model decides under *optimistic* coverage. Optimistic is what lets us + // skip the dataset open on a clearly-CSR traversal: real coverage can only + // make the indexed path costlier, so if even a perfectly-indexed scan loses + // to CSR here, it loses for real. + let forced = traversal_indexed_override(); + let lean_indexed = match forced { + Some(v) => v, + None => match gather_cost_inputs( + snapshot, + catalog, + edge_type, + direction, + frontier_rows, + effective_max_hops, + crate::table_store::IndexCoverage::Indexed, + graph_index.is_built(), + ) { + Some(inputs) => choose_expand_mode(&inputs) == ExpandMode::IndexedScan, + // Manifest counts absent (e.g. not-yet-materialized table): fall back + // to the legacy frontier/hop ceiling so the decision is defined. + None => { + frontier_rows <= expand_indexed_max_frontier() + && effective_max_hops <= expand_indexed_max_hops() + } + }, + }; + + if !lean_indexed { + tracing::debug!( + target: "omnigraph::traverse", + edge = %edge_type, + frontier = frontier_rows, + hops = effective_max_hops, + mode = "csr", + "expand mode chosen", + ); + let gi = graph_index.get().await?.ok_or_else(|| { + OmniError::manifest("graph index required for CSR traversal".to_string()) + })?; + return execute_expand_csr( + wide, gi, snapshot, catalog, src_var, dst_var, edge_type, direction, dst_type, + min_hops, max_hops, dst_filters, params, + ) + .await; + } + + // Leaning indexed: open the edge dataset once, confirm real coverage, and + // (unless forced) re-decide with it. The opened dataset is threaded into the + // indexed path so it is never opened twice. + let edge_ds = snapshot.open(&edge_table_key).await?; + let coverage = + crate::table_store::TableStore::key_column_index_coverage(&edge_ds, key_col).await; + + if forced.is_none() { + if let Some(inputs) = gather_cost_inputs( + snapshot, + catalog, + edge_type, + direction, + frontier_rows, + effective_max_hops, + coverage_for_decision(&coverage), + graph_index.is_built(), + ) { + if choose_expand_mode(&inputs) == ExpandMode::Csr { + tracing::debug!( + target: "omnigraph::traverse", + edge = %edge_type, + frontier = frontier_rows, + hops = effective_max_hops, + mode = "csr", + reason = "index coverage degraded", + "expand mode chosen", + ); + let gi = graph_index.get().await?.ok_or_else(|| { + OmniError::manifest("graph index required for CSR traversal".to_string()) + })?; + return execute_expand_csr( + wide, gi, snapshot, catalog, src_var, dst_var, edge_type, direction, dst_type, + min_hops, max_hops, dst_filters, params, + ) + .await; + } + } + } + + tracing::debug!( + target: "omnigraph::traverse", + edge = %edge_type, + frontier = frontier_rows, + hops = effective_max_hops, + mode = "indexed", + "expand mode chosen", + ); + // Surface the C6 silent scalar-index fallback once, now that coverage is known. + warn_on_degraded_coverage(&coverage, key_col, edge_type); + execute_expand_indexed( + wide, snapshot, catalog, src_var, dst_var, edge_type, direction, dst_type, min_hops, + max_hops, dst_filters, params, edge_ds, + ) + .await +} + +/// BTREE-indexed graph traversal: per hop, batch the current frontier into one +/// `scan_edges_by_endpoint` call against the persisted src/dst index, then fan +/// out per source row. Cost scales with the frontier, not |E|. Produces the +/// same `(src_row, dst_id)` pairs as the CSR path and shares its hydrate+align +/// tail. Multi-hop only advances for same-type edges; cross-type frontiers go +/// empty after one hop (no edges key off the destination type), matching CSR. +async fn execute_expand_indexed( + wide: &mut RecordBatch, + snapshot: &Snapshot, + catalog: &Catalog, + src_var: &str, + dst_var: &str, + edge_type: &str, + direction: Direction, + dst_type: &str, + min_hops: u32, + max_hops: Option<u32>, + dst_filters: &[IRFilter], + params: &ParamMap, + edge_ds: Dataset, +) -> Result<()> { + let src_id_col_name = format!("{}.id", src_var); + let src_ids = wide + .column_by_name(&src_id_col_name) + .ok_or_else(|| { + OmniError::manifest(format!("wide batch missing '{}' column", src_id_col_name)) + })? + .as_any() + .downcast_ref::<StringArray>() + .ok_or_else(|| OmniError::manifest(format!("'{}' column is not Utf8", src_id_col_name)))? + .clone(); + + let edge_def = catalog + .edge_types + .get(edge_type) + .ok_or_else(|| OmniError::manifest(format!("unknown edge type '{}'", edge_type)))?; + let same_type = edge_def.from_type == edge_def.to_type; + // The keyed/opposite endpoint columns for this direction. The edge dataset + // and the C6 coverage warn are owned by the caller (`execute_expand`), which + // opens the dataset once and threads it in. + let (key_col, opp_col) = endpoint_columns(direction); + + let max = max_hops.unwrap_or(min_hops.max(1)); + // Cross-type edges cannot chain (a Company is not a `WorksAt` source), so a + // variable-length traversal over one is structurally single-hop. Enforce it + // here instead of relying on the hop-2 scan returning empty: this BFS interns + // every endpoint string into ONE dense id space, so a cross-type id-string + // collision (a Person and a Company sharing an id) would otherwise let hop 2 + // de-intern a destination id back to the colliding source-type id and match + // its edges, emitting rows the CSR path never produces. + let max = if same_type { max } else { max.min(1) }; + + // Per-source BFS state in DENSE id space: intern node ids to u32 once via a + // per-traversal interner so visited/seen/frontier/neighbor-map avoid string + // hashing + cloning in the hot loop (mirrors the CSR path's TypeIndex). The + // GraphIndex/CSR is NOT built — only a local id↔u32 dictionary. Strings + // survive at the substrate edges only: the per-hop IN-list to Lance, and the + // emitted dst ids handed to the string-keyed hydrate+align tail. + let mut interner = crate::graph_index::TypeIndex::new(); + let n = src_ids.len(); + let mut frontiers: Vec<Vec<u32>> = Vec::with_capacity(n); + let mut visited: Vec<HashSet<u32>> = Vec::with_capacity(n); + let mut seen_dst: Vec<HashSet<u32>> = Vec::with_capacity(n); + for i in 0..n { + let sid = interner.get_or_insert(src_ids.value(i)); + let mut v = HashSet::new(); + if same_type { + v.insert(sid); + } + frontiers.push(vec![sid]); + visited.push(v); + seen_dst.push(HashSet::new()); + } + + let mut src_indices: Vec<u32> = Vec::new(); + let mut dst_dense: Vec<u32> = Vec::new(); + + for hop in 1..=max { + // Union of all live frontiers (dense), de-interned once for the IN-list. + let mut union_dense: Vec<u32> = Vec::new(); + { + let mut seen: HashSet<u32> = HashSet::new(); + for f in &frontiers { + for &node in f { + if seen.insert(node) { + union_dense.push(node); + } + } + } + } + if union_dense.is_empty() { + break; + } + let union_keys: Vec<String> = union_dense + .iter() + .map(|&u| { + interner + .to_id(u) + .expect("interned frontier id must resolve") + .to_string() + }) + .collect(); + + let batches = crate::table_store::TableStore::scan_edges_by_endpoint( + &edge_ds, key_col, opp_col, &union_keys, + ) + .await?; + + // dense key -> dense neighbors (scan order; duplicates preserved, like CSR multi-edges). + let mut neighbor_map: HashMap<u32, Vec<u32>> = HashMap::new(); + for batch in &batches { + let keys = batch + .column_by_name(key_col) + .ok_or_else(|| OmniError::manifest(format!("edge batch missing '{}'", key_col)))? + .as_any() + .downcast_ref::<StringArray>() + .ok_or_else(|| OmniError::manifest(format!("edge '{}' is not Utf8", key_col)))?; + let opps = batch + .column_by_name(opp_col) + .ok_or_else(|| OmniError::manifest(format!("edge batch missing '{}'", opp_col)))? + .as_any() + .downcast_ref::<StringArray>() + .ok_or_else(|| OmniError::manifest(format!("edge '{}' is not Utf8", opp_col)))?; + for r in 0..batch.num_rows() { + let k = interner.get_or_insert(keys.value(r)); + let o = interner.get_or_insert(opps.value(r)); + neighbor_map.entry(k).or_default().push(o); + } + } + + // Advance each source row's frontier independently (dense ids). + for i in 0..n { + let cur = std::mem::take(&mut frontiers[i]); + let mut next: Vec<u32> = Vec::new(); + for &node in &cur { + let Some(neighbors) = neighbor_map.get(&node) else { + continue; + }; + for &neighbor in neighbors { + if !same_type || visited[i].insert(neighbor) { + next.push(neighbor); + if hop >= min_hops && seen_dst[i].insert(neighbor) { + src_indices.push(i as u32); + dst_dense.push(neighbor); + } + } + } + } + frontiers[i] = next; + } + } + + // De-intern emitted destination ids (parallel to src_indices) for the + // string-keyed hydrate+align tail, exactly as the CSR path does. + let dst_ids: Vec<String> = dst_dense + .iter() + .map(|&d| { + interner + .to_id(d) + .expect("interned dst id must resolve") + .to_string() + }) + .collect(); + + expand_hydrate_and_align( + wide, src_indices, dst_ids, snapshot, catalog, dst_type, dst_var, dst_filters, params, + ) + .await +} + +/// Shared tail for both Expand modes: hydrate the unique destination ids, align +/// the `(src_row, dst_id)` pairs back onto `wide`, hconcat, and apply +/// non-pushable destination filters in memory. +async fn expand_hydrate_and_align( + wide: &mut RecordBatch, + src_indices: Vec<u32>, + dst_ids: Vec<String>, + snapshot: &Snapshot, + catalog: &Catalog, + dst_type: &str, + dst_var: &str, + dst_filters: &[IRFilter], + params: &ParamMap, +) -> Result<()> { + // Pushable destination filters are applied by `hydrate_nodes`; the rest + // (`ir_filter_to_expr` → None) are applied in memory after hconcat. + let non_pushable: Vec<&IRFilter> = dst_filters + .iter() + .filter(|f| ir_filter_to_expr(f, params).is_none()) + .collect(); + + // Unique destination ids (first-seen order) for one batched hydration. + let mut unique_dst_list: Vec<String> = Vec::new(); + { + let mut seen: HashSet<&str> = HashSet::with_capacity(dst_ids.len()); + for id in &dst_ids { + if seen.insert(id.as_str()) { + unique_dst_list.push(id.clone()); + } + } + } + let dst_batch = + hydrate_nodes(snapshot, catalog, dst_type, &unique_dst_list, dst_filters, params).await?; + + // id -> row index in the hydrated batch. + let dst_batch_id_col = dst_batch + .column_by_name("id") + .ok_or_else(|| OmniError::manifest("hydrated batch missing 'id' column".to_string()))? + .as_any() + .downcast_ref::<StringArray>() + .ok_or_else(|| OmniError::manifest("hydrated 'id' column is not Utf8".to_string()))?; + let mut id_to_row: HashMap<&str, u32> = HashMap::with_capacity(dst_batch_id_col.len()); + for row in 0..dst_batch_id_col.len() { + id_to_row.insert(dst_batch_id_col.value(row), row as u32); + } + + // Align pairs to (src_row, hydrated_dst_row), dropping ids hydration filtered out. + let mut final_src_indices: Vec<u32> = Vec::with_capacity(src_indices.len()); + let mut dst_indices: Vec<u32> = Vec::with_capacity(src_indices.len()); + for (&src_idx, dst_id) in src_indices.iter().zip(dst_ids.iter()) { + if let Some(&dst_row) = id_to_row.get(dst_id.as_str()) { + final_src_indices.push(src_idx); + dst_indices.push(dst_row); + } + } + + let src_take = UInt32Array::from(final_src_indices); + let dst_take = UInt32Array::from(dst_indices); + let expanded_wide = take_batch(wide, &src_take)?; + let dst_prefixed = prefix_batch(&dst_batch, dst_var)?; + let aligned_dst = take_batch(&dst_prefixed, &dst_take)?; + *wide = hconcat_batches(&expanded_wide, &aligned_dst)?; + + for f in &non_pushable { + apply_filter(wide, f, params)?; + } + Ok(()) +} + +/// CSR-backed graph traversal: BFS over the in-memory adjacency index. Used for +/// dense / whole-graph traversals; selective traversals use +/// `execute_expand_indexed`. Both share `expand_hydrate_and_align`. +async fn execute_expand_csr( wide: &mut RecordBatch, graph_index: &GraphIndex, snapshot: &Snapshot, @@ -742,6 +1399,9 @@ async fn execute_expand( let max = max_hops.unwrap_or(min_hops.max(1)); let same_type = src_type_name == dst_type_name; + // Cross-type edges cannot chain; a variable-length traversal over one is + // structurally single-hop (mirrors the indexed path's guarantee). + let max = if same_type { max } else { max.min(1) }; // BFS to collect (src_row_idx, dst_dense) pairs with per-source dedup. // Dense u32 ids stay in hand through BFS, dedup, and align — we only @@ -785,88 +1445,52 @@ async fn execute_expand( } } - // Split dst_filters: SQL-pushable go to Lance, the rest applied post-hconcat - let pushdown_sql = build_lance_filter(dst_filters, params); - let non_pushable: Vec<&IRFilter> = dst_filters - .iter() - .filter(|f| ir_filter_to_sql(f, params).is_none()) - .collect(); - - // Dedup dst dense ids globally across source rows, then stringify once - // for the Lance IN-list. The post-hydrate alignment fans rows back out to - // the original (src, dst) pairs via a dense-indexed lookup below. - let mut unique_dst_list: Vec<String> = Vec::new(); - { - let mut seen: HashSet<u32> = HashSet::with_capacity(dst_dense_list.len()); - for &d in &dst_dense_list { - if seen.insert(d) { - if let Some(id) = dst_type_idx.to_id(d) { - unique_dst_list.push(id.to_string()); - } - } + // Map BFS-produced dense destination ids to string ids for the shared + // hydrate+align tail. Dense ids always resolve (they came from the index); + // drop any that don't, keeping the (src, dst) arrays parallel. + let mut tail_src_indices: Vec<u32> = Vec::with_capacity(src_indices.len()); + let mut dst_ids: Vec<String> = Vec::with_capacity(dst_dense_list.len()); + for (&s, &d) in src_indices.iter().zip(dst_dense_list.iter()) { + if let Some(id) = dst_type_idx.to_id(d) { + tail_src_indices.push(s); + dst_ids.push(id.to_string()); } } - let dst_batch = hydrate_nodes( + + expand_hydrate_and_align( + wide, + tail_src_indices, + dst_ids, snapshot, catalog, dst_type, - &unique_dst_list, - pushdown_sql.as_deref(), + dst_var, + dst_filters, + params, ) - .await?; - - // Build dense → row-in-hydrated-batch via a direct-indexed array. - let dst_batch_id_col = dst_batch - .column_by_name("id") - .ok_or_else(|| OmniError::manifest("hydrated batch missing 'id' column".to_string()))? - .as_any() - .downcast_ref::<StringArray>() - .ok_or_else(|| OmniError::manifest("hydrated 'id' column is not Utf8".to_string()))?; - let mut dense_to_row: Vec<Option<u32>> = vec![None; dst_type_idx.len()]; - for row in 0..dst_batch_id_col.len() { - let id_str = dst_batch_id_col.value(row); - if let Some(dense) = dst_type_idx.to_dense(id_str) { - dense_to_row[dense as usize] = Some(row as u32); - } - } - - // Build aligned src/dst index arrays (only for ids that exist in hydrated batch) - let mut final_src_indices: Vec<u32> = Vec::new(); - let mut dst_indices: Vec<u32> = Vec::new(); - for (src_idx, dst_dense) in src_indices.iter().zip(dst_dense_list.iter()) { - if let Some(dst_row) = dense_to_row[*dst_dense as usize] { - final_src_indices.push(*src_idx); - dst_indices.push(dst_row); - } - } - - let src_take = UInt32Array::from(final_src_indices); - let dst_take = UInt32Array::from(dst_indices); - let expanded_wide = take_batch(wide, &src_take)?; - let dst_prefixed = prefix_batch(&dst_batch, dst_var)?; - let aligned_dst = take_batch(&dst_prefixed, &dst_take)?; - *wide = hconcat_batches(&expanded_wide, &aligned_dst)?; - - // Apply any non-pushable destination filters (e.g. list-contains) in memory - for f in &non_pushable { - apply_filter(wide, f, params)?; - } - - Ok(()) + .await } /// Load full node rows for a set of IDs from a snapshot. /// -/// When `extra_filter_sql` is provided (from deferred destination-binding -/// filters), it is ANDed with the `id IN (...)` clause so that Lance can -/// skip non-matching rows at the storage level. +/// The `id IN (...)` predicate is built as a structured DataFusion `Expr` and +/// AND'd with any pushable `dst_filters` (destination-binding filters), then +/// applied via `Scanner::filter_expr`. The structured form routes the id +/// IN-list through the `id` BTREE scalar index (index-search → take) rather +/// than evaluating a string filter via DataFusion `InListEval`, which is +/// O(N×M) and was measured at 72× the indexed cost on a 100k-node hop +/// (MR-376). Non-pushable `dst_filters` (`ir_filter_to_expr` → None) are +/// applied in memory by the caller after hydration. async fn hydrate_nodes( snapshot: &Snapshot, catalog: &Catalog, type_name: &str, ids: &[String], - extra_filter_sql: Option<&str>, + dst_filters: &[IRFilter], + params: &ParamMap, ) -> Result<RecordBatch> { + use datafusion::prelude::{col, lit}; + let node_type = catalog .node_types .get(type_name) @@ -879,15 +1503,13 @@ async fn hydrate_nodes( let table_key = format!("node:{}", type_name); let ds = snapshot.open(&table_key).await?; - // Build filter: id IN ('a', 'b', 'c') - let escaped: Vec<String> = ids - .iter() - .map(|id| format!("'{}'", id.replace('\'', "''"))) - .collect(); - let mut filter_sql = format!("id IN ({})", escaped.join(", ")); - if let Some(extra) = extra_filter_sql { - filter_sql = format!("({}) AND ({})", filter_sql, extra); + // `id IN (ids)` AND any pushable destination filters, as a structured Expr. + let id_list: Vec<datafusion::prelude::Expr> = ids.iter().map(|id| lit(id.clone())).collect(); + let mut filter_expr = col("id").in_list(id_list, false); + if let Some(dst_expr) = build_lance_filter_expr(dst_filters, params) { + filter_expr = filter_expr.and(dst_expr); } + let has_blobs = !node_type.blob_properties.is_empty(); let non_blob_cols: Vec<&str> = node_type .arrow_schema @@ -897,12 +1519,16 @@ async fn hydrate_nodes( .map(|f| f.name().as_str()) .collect(); let projection = has_blobs.then_some(non_blob_cols.as_slice()); - let batches = crate::table_store::TableStore::scan_stream( + let batches = crate::table_store::TableStore::scan_stream_with( &ds, projection, - Some(&filter_sql), + None, None, false, + |scanner| { + scanner.filter_expr(filter_expr); + Ok(()) + }, ) .await? .try_collect::<Vec<RecordBatch>>() @@ -925,6 +1551,25 @@ async fn hydrate_nodes( Ok(scan_result) } +/// Whether the inner pipeline is the bulk-anti-join shape: a single Expand from +/// the outer var with no destination filters (the only shape the CSR +/// `has_neighbors` fast path can serve). Pure — it does not touch the CSR — so +/// the caller can decide whether to realize the O(|E|) graph index at all. +fn bulk_anti_join_applies(inner_pipeline: &[IROp], outer_var: &str) -> bool { + matches!( + inner_pipeline, + [IROp::Expand { src_var, dst_filters, min_hops, max_hops, .. }] + if src_var == outer_var + && dst_filters.is_empty() + // `has_neighbors` is a ONE-hop existence test, so the fast path + // is valid only for a single-hop expand. Multi-hop negations + // (e.g. `not { $p knows{2,2} $x }`) fall to the slow path, whose + // inner Expand runs the real bounded traversal. + && *min_hops == 1 + && (*max_hops).unwrap_or(1) == 1 + ) +} + /// Try bulk anti-join via CSR existence check. Returns Some(mask) if the inner /// pipeline is a single Expand from outer_var (the common negation pattern). fn try_bulk_anti_join_mask( @@ -934,27 +1579,17 @@ fn try_bulk_anti_join_mask( catalog: &Catalog, outer_var: &str, ) -> Option<BooleanArray> { - if inner_pipeline.len() != 1 { + if !bulk_anti_join_applies(inner_pipeline, outer_var) { return None; } let IROp::Expand { - src_var, edge_type, direction, - dst_filters, .. } = &inner_pipeline[0] else { return None; }; - if src_var != outer_var { - return None; - } - // Bulk CSR check only tests neighbor existence, not destination - // properties. Fall back to the slow path when dst_filters are present. - if !dst_filters.is_empty() { - return None; - } let gi = graph_index?; let edge_def = catalog.edge_types.get(edge_type.as_str())?; @@ -993,49 +1628,106 @@ async fn execute_anti_join( inner_pipeline: &[IROp], params: &ParamMap, snapshot: &Snapshot, - graph_index: Option<&GraphIndex>, + graph_index: &GraphIndexHandle<'_>, catalog: &Catalog, outer_var: &str, ) -> Result<()> { + // Only the bulk fast path consumes the CSR; the slow path's inner Expand + // chooses its own access path. Realize the O(|E|) graph index ONLY when the + // inner-pipeline shape qualifies for the bulk check — a filtered/nested + // anti-join over a large graph must not pay a whole-graph build it won't use. + let gi = if bulk_anti_join_applies(inner_pipeline, outer_var) { + graph_index.get().await? + } else { + None + }; // Fast path: bulk CSR existence check (O(N), zero Lance I/O) - if let Some(mask) = - try_bulk_anti_join_mask(wide, inner_pipeline, graph_index, catalog, outer_var) - { + if let Some(mask) = try_bulk_anti_join_mask(wide, inner_pipeline, gi, catalog, outer_var) { *wide = arrow_select::filter::filter_record_batch(wide, &mask) .map_err(|e| OmniError::Lance(e.to_string()))?; return Ok(()); } - // Slow path: per-row inner pipeline execution + // Slow path (filtered / non-bulk inner): run the inner pipeline ONCE over the + // whole frontier — a set-oriented anti-semi-join — instead of row-by-row. + // Each outer row is tagged with a synthetic index; an outer row matches iff + // it produced at least one surviving inner row. No per-row dispatch, so the + // inner Expand runs as a single set-at-a-time traversal over the full + // frontier (its own chooser picks indexed vs CSR) rather than one Lance scan + // per outer row. let num_rows = wide.num_rows(); - let mut keep_mask = vec![true; num_rows]; + if num_rows == 0 { + return Ok(()); + } - for i in 0..num_rows { - let single_row = wide.slice(i, 1); - let mut inner_wide: Option<RecordBatch> = Some(single_row); + // The tag rides through the inner pipeline: Expand's hconcat preserves + // existing columns and Filter only drops rows, so each surviving row carries + // its originating outer-row index. Correlating on the row index (not + // `outer_var.id`) stays correct even if a dst-filter references other outer + // bindings. Nested anti-joins reuse this slow path and an enclosing tag rides + // through too; Arrow allows duplicate field names and `column_by_name` + // returns the FIRST match, so choose a tag name not already present (each + // nesting level then reads its own) instead of a fixed one. + let tag_col: String = { + let mut n = 0usize; + loop { + let candidate = format!("__antijoin_outer_row_{n}"); + if wide.schema().column_with_name(&candidate).is_none() { + break candidate; + } + n += 1; + } + }; + let mut fields: Vec<Field> = wide + .schema() + .fields() + .iter() + .map(|f| f.as_ref().clone()) + .collect(); + fields.push(Field::new(tag_col.as_str(), DataType::UInt32, false)); + let mut columns: Vec<ArrayRef> = wide.columns().to_vec(); + columns.push(Arc::new(UInt32Array::from_iter_values(0..num_rows as u32))); + let tagged = RecordBatch::try_new(Arc::new(Schema::new(fields)), columns) + .map_err(|e| OmniError::Lance(e.to_string()))?; - let no_search = SearchMode::default(); - execute_pipeline( - inner_pipeline, - params, - snapshot, - graph_index, - catalog, - &mut inner_wide, - &no_search, - ) - .await?; + let mut inner_wide: Option<RecordBatch> = Some(tagged); + let no_search = SearchMode::default(); + execute_pipeline( + inner_pipeline, + params, + snapshot, + graph_index, + catalog, + &mut inner_wide, + &no_search, + ) + .await?; - let has_match = inner_wide - .as_ref() - .map(|batch| batch.num_rows() > 0) - .unwrap_or(false); - - if has_match { - keep_mask[i] = false; + // Outer rows whose tag survived have >= 1 match. A produced-but-untagged + // batch means the inner pipeline dropped the correlation column — fail loudly + // rather than silently keeping every row (which would corrupt the anti-join). + let mut matched: HashSet<u32> = HashSet::new(); + if let Some(batch) = inner_wide { + if batch.num_rows() > 0 { + let tags = batch + .column_by_name(tag_col.as_str()) + .ok_or_else(|| { + OmniError::manifest( + "anti-join inner pipeline dropped the correlation column".to_string(), + ) + })? + .as_any() + .downcast_ref::<UInt32Array>() + .ok_or_else(|| { + OmniError::manifest(format!("'{}' column is not UInt32", tag_col)) + })?; + for i in 0..tags.len() { + matched.insert(tags.value(i)); + } } } + let keep_mask: Vec<bool> = (0..num_rows as u32).map(|i| !matched.contains(&i)).collect(); let mask = BooleanArray::from(keep_mask); *wide = arrow_select::filter::filter_record_batch(wide, &mask) .map_err(|e| OmniError::Lance(e.to_string()))?; @@ -1186,45 +1878,6 @@ fn add_null_blob_columns( .map_err(|e| OmniError::Lance(e.to_string())) } -/// Convert IR filters to a Lance SQL filter string. -fn build_lance_filter(filters: &[IRFilter], params: &ParamMap) -> Option<String> { - if filters.is_empty() { - return None; - } - - let parts: Vec<String> = filters - .iter() - .filter_map(|f| ir_filter_to_sql(f, params)) - .collect(); - - if parts.is_empty() { - return None; - } - - Some(parts.join(" AND ")) -} - -fn ir_filter_to_sql(filter: &IRFilter, params: &ParamMap) -> Option<String> { - // Search predicates (search/fuzzy/match_text = true) are NOT converted to SQL. - // They are handled via scanner.full_text_search() in execute_node_scan. - if is_search_filter(filter) { - return None; - } - - let left = ir_expr_to_sql(&filter.left, params)?; - let right = ir_expr_to_sql(&filter.right, params)?; - let op = match filter.op { - CompOp::Eq => "=", - CompOp::Ne => "!=", - CompOp::Gt => ">", - CompOp::Lt => "<", - CompOp::Ge => ">=", - CompOp::Le => "<=", - CompOp::Contains => return None, // Can't pushdown list contains - }; - Some(format!("{} {} {}", left, op, right)) -} - /// Build a FullTextSearchQuery from a search IR expression. fn build_fts_query( expr: &IRExpr, @@ -1297,15 +1950,6 @@ fn resolve_to_int(expr: &IRExpr, params: &ParamMap) -> Option<i64> { } } -fn ir_expr_to_sql(expr: &IRExpr, params: &ParamMap) -> Option<String> { - match expr { - IRExpr::PropAccess { property, .. } => Some(property.clone()), - IRExpr::Literal(lit) => Some(literal_to_sql(lit)), - IRExpr::Param(name) => params.get(name).map(literal_to_sql), - _ => None, - } -} - pub(super) fn literal_to_sql(lit: &Literal) -> String { match lit { Literal::Null => "NULL".to_string(), @@ -1336,10 +1980,10 @@ pub(super) fn literal_to_sql(lit: &Literal) -> String { // // Search predicates (`is_search_filter`) are still handled separately via // `scanner.full_text_search(...)`, not via filter_expr — they stay None -// here just like in `ir_filter_to_sql`. The `literal_to_sql` path remains -// because the mutation/update layer (`exec/mutation.rs`) still produces -// SQL strings for `Dataset::delete(&str)`; that migration is MR-A's -// territory (Lance #6658 + delete two-phase). +// here (search predicates are never lowered to a scalar filter). The +// `literal_to_sql` path remains because the mutation/update layer +// (`exec/mutation.rs`) still produces SQL strings for `Dataset::delete(&str)`; +// that migration is MR-A's territory (Lance #6658 + delete two-phase). /// Convert IR filters to a single DataFusion `Expr` (AND-joined), or /// `None` if no filter is pushable. @@ -1381,8 +2025,8 @@ pub(super) fn ir_filter_to_expr( } // List-contains: `prop CONTAINS value` lowers to `array_has(prop, value)`. - // This is the case `ir_filter_to_sql` had to return None for ("Can't - // pushdown list contains"); with structured Expr it pushes down fine. + // This is the case the old SQL-string pushdown had to return None for + // ("Can't pushdown list contains"); with structured Expr it pushes down fine. if matches!(filter.op, CompOp::Contains) { let left = ir_expr_to_expr(&filter.left, params)?; let right = ir_expr_to_expr(&filter.right, params)?; @@ -1517,3 +2161,127 @@ fn take_batch(batch: &RecordBatch, indices: &UInt32Array) -> Result<RecordBatch> .map_err(|e| OmniError::Lance(e.to_string()))?; RecordBatch::try_new(batch.schema(), columns).map_err(|e| OmniError::Lance(e.to_string())) } + +#[cfg(test)] +mod expand_chooser_tests { + use super::*; + use crate::table_store::IndexCoverage; + + /// Build cost inputs with generous hard caps, so the cost comparison (not a + /// ceiling) is what the assertions exercise unless a test sets one on purpose. + fn inputs( + frontier_rows: usize, + edge_count: u64, + src_node_count: u64, + effective_max_hops: u32, + coverage: IndexCoverage, + ) -> ExpandCostInputs { + ExpandCostInputs { + frontier_rows, + edge_count, + src_node_count, + effective_max_hops, + max_hops_cap: 6, + max_frontier_cap: 1024, + coverage, + csr_cached: false, + } + } + + #[test] + fn selective_frontier_on_large_graph_picks_indexed() { + // 50 source rows against 1M source vertices, one hop: tiny selectivity — + // the PR #149 win the chooser must preserve. + let m = choose_expand_mode(&inputs(50, 10_000_000, 1_000_000, 1, IndexCoverage::Indexed)); + assert_eq!(m, ExpandMode::IndexedScan); + } + + #[test] + fn flat_in_edge_count_same_selectivity_same_choice() { + // Same selectivity (frontier/|V_src|), 1000× difference in |E|. Indexed + // cost is independent of |E|, so the choice must not flip. + let small = choose_expand_mode(&inputs(50, 100_000, 1_000_000, 1, IndexCoverage::Indexed)); + let huge = + choose_expand_mode(&inputs(50, 100_000_000, 1_000_000, 1, IndexCoverage::Indexed)); + assert_eq!(small, ExpandMode::IndexedScan); + assert_eq!(huge, ExpandMode::IndexedScan); + } + + #[test] + fn frontier_large_fraction_of_source_picks_csr() { + // hops*frontier (200) exceeds BUILD_FACTOR*|V_src| (1.5*100=150) → CSR, + // and 200 is below the frontier cap, so it is the cost model deciding. + let m = choose_expand_mode(&inputs(200, 1_000, 100, 1, IndexCoverage::Indexed)); + assert_eq!(m, ExpandMode::Csr); + } + + #[test] + fn frontier_over_hard_cap_picks_csr() { + // 2000 > 1024 ceiling, even though the selectivity is tiny. + let m = choose_expand_mode(&inputs(2000, 10_000_000, 1_000_000, 1, IndexCoverage::Indexed)); + assert_eq!(m, ExpandMode::Csr); + } + + #[test] + fn hops_over_hard_cap_picks_csr() { + let m = choose_expand_mode(&inputs(10, 10_000_000, 1_000_000, 8, IndexCoverage::Indexed)); + assert_eq!(m, ExpandMode::Csr); + } + + #[test] + fn degraded_single_hop_tiny_frontier_stays_indexed() { + // One full degraded scan (1*|E|) still edges out a full CSR build + // (1.5*|E|) for a one-off single hop. + let m = choose_expand_mode(&inputs( + 5, + 10_000, + 10_000, + 1, + IndexCoverage::Degraded { + reason: "no btree".into(), + }, + )); + assert_eq!(m, ExpandMode::IndexedScan); + } + + #[test] + fn degraded_multi_hop_picks_csr() { + // Two degraded scans (2*|E|) lose to one CSR build (1.5*|E|). + let m = choose_expand_mode(&inputs( + 5, + 10_000, + 10_000, + 2, + IndexCoverage::Degraded { + reason: "no btree".into(), + }, + )); + assert_eq!(m, ExpandMode::Csr); + } + + #[test] + fn warm_csr_is_always_reused() { + // A maximally selective traversal still prefers an already-built CSR + // (cost ~0) over re-scanning per hop. + let mut i = inputs(1, 10_000_000, 1_000_000, 1, IndexCoverage::Indexed); + i.csr_cached = true; + assert_eq!(choose_expand_mode(&i), ExpandMode::Csr); + } + + #[test] + fn cost_model_caps_cross_type_hops() { + // Same-type passes the requested range through; cross-type caps at 1, + // matching execute_expand_indexed. + assert_eq!(cost_effective_hops(5, true), 5); + assert_eq!(cost_effective_hops(5, false), 1); + assert_eq!(cost_effective_hops(1, false), 1); + + // Consequence: a selective frontier where the requested 5 hops would + // (wrongly) flip cross-type to CSR, but the capped 1 hop — what actually + // runs — keeps it indexed. + let mut i = inputs(50, 10_000, 100, cost_effective_hops(5, false), IndexCoverage::Indexed); + assert_eq!(choose_expand_mode(&i), ExpandMode::IndexedScan); + i.effective_max_hops = 5; // as if the cross-type cap were not applied + assert_eq!(choose_expand_mode(&i), ExpandMode::Csr); + } +} diff --git a/crates/omnigraph/src/table_store.rs b/crates/omnigraph/src/table_store.rs index 4b52db6..bdf0dd5 100644 --- a/crates/omnigraph/src/table_store.rs +++ b/crates/omnigraph/src/table_store.rs @@ -43,6 +43,19 @@ pub struct DeleteState { pub(crate) version_metadata: TableVersionMetadata, } +/// Whether a `key_col IN (...)` scan on a dataset will be served by the +/// persisted scalar (BTREE) index, or silently fall back to a full filtered +/// scan. Detection-only (metadata, no IO); the scan returns the correct rows +/// either way. Surfaced by the indexed traversal path so the silent perf +/// fallback is observable, and available to a future cost-based planner. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum IndexCoverage { + /// The column has a usable BTREE and every fragment records `physical_rows`. + Indexed, + /// Lance will not use the scalar index for this scan (correct, full scan). + Degraded { reason: String }, +} + /// A Lance write that has produced fragment files on object storage but is /// not yet committed to the dataset's manifest. The staged-write primitives /// are consumed by `MutationStaging` (`exec/staging.rs`, @@ -582,6 +595,117 @@ impl TableStore { .map_err(|e| OmniError::Lance(e.to_string())) } + /// Indexed neighbor lookup for graph traversal. Given an edge dataset and a + /// set of endpoint keys on `key_col` (`"src"` for out-traversal, `"dst"` for + /// in-traversal), return the matching edge rows projected to + /// `[key_col, opposite_col]`. + /// + /// The `key_col IN (keys)` predicate is built as a structured DataFusion + /// `Expr` and applied via `Scanner::filter_expr`, so Lance routes it through + /// the persisted BTREE on `key_col` (index-search → take). Cost scales with + /// the frontier size, not |E| — the basis for serving selective traversals + /// without building the whole in-memory CSR. Empty `keys` returns empty + /// without scanning. + /// + /// Note: like any indexed scan, this observes only fragments the BTREE + /// covers plus an unindexed-fragment scan fallback; it reads the committed + /// snapshot `ds` was opened at. + pub async fn scan_edges_by_endpoint( + ds: &Dataset, + key_col: &str, + opposite_col: &str, + keys: &[String], + ) -> Result<Vec<RecordBatch>> { + use datafusion::prelude::{col, lit}; + + if keys.is_empty() { + return Ok(Vec::new()); + } + let key_list: Vec<datafusion::prelude::Expr> = + keys.iter().map(|k| lit(k.clone())).collect(); + let filter_expr = col(key_col).in_list(key_list, false); + Self::scan_stream_with( + ds, + Some(&[key_col, opposite_col]), + None, + None, + false, + |scanner| { + scanner.filter_expr(filter_expr); + Ok(()) + }, + ) + .await? + .try_collect() + .await + .map_err(|e| OmniError::Lance(e.to_string())) + } + + /// Metadata-only check (no IO) of whether `scan_edges_by_endpoint` — a + /// `key_col IN (...)` filter — on `ds` will be served by the persisted BTREE + /// on `column`, or silently fall back to a full filtered scan. Mirrors + /// Lance's own decision: scalar indices are disabled for the whole scan if + /// ANY fragment lacks `physical_rows` (lance `dataset/scanner.rs` + /// `create_filter_plan`), and are obviously unused if no BTREE on the + /// column exists. The scan is correct (returns all rows) either way — this + /// only surfaces the perf cliff so the indexed traversal can warn on it. + pub async fn key_column_index_coverage(ds: &Dataset, column: &str) -> Result<IndexCoverage> { + let Some(field_id) = ds.schema().field(column).map(|field| field.id) else { + return Ok(IndexCoverage::Degraded { + reason: format!("column '{}' not in schema", column), + }); + }; + let indices = ds + .load_indices() + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + let btree = indices + .iter() + .filter(|index| !is_system_index(index)) + .filter(|index| index.fields.len() == 1 && index.fields[0] == field_id) + .find(|index| { + index + .index_details + .as_ref() + .map(|details| details.type_url.ends_with("BTreeIndexDetails")) + .unwrap_or(false) + }); + let Some(btree) = btree else { + return Ok(IndexCoverage::Degraded { + reason: format!("no BTREE index on '{}'", column), + }); + }; + // Same check Lance runs: a fragment missing physical_rows disables + // scalar indices for the entire scan (all-or-nothing). + if ds.fragments().iter().any(|f| f.physical_rows.is_none()) { + return Ok(IndexCoverage::Degraded { + reason: "a fragment is missing physical_rows".to_string(), + }); + } + // An index only covers the fragments it was built over; fragments + // appended afterward (edge-index creation is skipped once a BTREE exists) + // are scanned unindexed. If any CURRENT fragment is absent from the + // index's `fragment_bitmap`, the scan is partly a full scan — so the + // chooser must not price it as fully indexed. A `None` bitmap means Lance + // can't report coverage; don't over-degrade in that case. + if let Some(bitmap) = btree.fragment_bitmap.as_ref() { + let uncovered = ds + .fragments() + .iter() + .filter(|f| !bitmap.contains(f.id as u32)) + .count(); + if uncovered > 0 { + return Ok(IndexCoverage::Degraded { + reason: format!( + "{} fragment(s) not covered by the index on '{}'", + uncovered, column + ), + }); + } + } + Ok(IndexCoverage::Indexed) + } + pub async fn count_rows(&self, ds: &Dataset, filter: Option<String>) -> Result<usize> { ds.count_rows(filter) .await diff --git a/crates/omnigraph/tests/fixtures/search.gq b/crates/omnigraph/tests/fixtures/search.gq index c39af82..d53fbc9 100644 --- a/crates/omnigraph/tests/fixtures/search.gq +++ b/crates/omnigraph/tests/fixtures/search.gq @@ -42,3 +42,17 @@ query hybrid_search($vq: Vector(4), $tq: String) { order { rrf(nearest($d.embedding, $vq), bm25($d.title, $tq)) } limit 3 } + +query rrf_two_fts($q: String) { + match { $d: Doc } + return { $d.slug, $d.title } + order { rrf(bm25($d.title, $q), bm25($d.body, $q)) } + limit 3 +} + +query rrf_two_vectors($q1: Vector(4), $q2: Vector(4)) { + match { $d: Doc } + return { $d.slug, $d.title } + order { rrf(nearest($d.embedding, $q1), nearest($d.embedding, $q2)) } + limit 3 +} diff --git a/crates/omnigraph/tests/helpers/mod.rs b/crates/omnigraph/tests/helpers/mod.rs index c97ff72..0e04aa2 100644 --- a/crates/omnigraph/tests/helpers/mod.rs +++ b/crates/omnigraph/tests/helpers/mod.rs @@ -236,6 +236,15 @@ pub fn vector_param(name: &str, values: &[f32]) -> ParamMap { map } +/// Build a ParamMap with two vector params. +pub fn two_vector_params(name1: &str, vals1: &[f32], name2: &str, vals2: &[f32]) -> ParamMap { + let mut map = vector_param(name1, vals1); + let key = name2.strip_prefix('$').unwrap_or(name2).to_string(); + let lit = Literal::List(vals2.iter().map(|v| Literal::Float(*v as f64)).collect()); + map.insert(key, lit); + map +} + /// Build a ParamMap with a vector param and a string param. pub fn vector_and_string_params( vec_name: &str, diff --git a/crates/omnigraph/tests/lance_surface_guards.rs b/crates/omnigraph/tests/lance_surface_guards.rs index 65efc4e..370f9e7 100644 --- a/crates/omnigraph/tests/lance_surface_guards.rs +++ b/crates/omnigraph/tests/lance_surface_guards.rs @@ -33,7 +33,10 @@ use lance::dataset::optimize::{CompactionOptions, compact_files}; use lance::dataset::transaction::Operation; use lance::dataset::write::delete::DeleteResult; use lance::dataset::{MergeInsertBuilder, WhenMatched, WhenNotMatched, WriteMode, WriteParams}; +use lance::index::DatasetIndexExt; use lance_file::version::LanceFileVersion; +use lance_index::IndexType; +use lance_index::scalar::ScalarIndexParams; use lance_namespace::LanceNamespace; use lance_table::io::commit::ManifestNamingScheme; @@ -406,3 +409,135 @@ async fn compact_files_still_fails_on_blob_columns() { shifted): {err}" ); } + +// --- Guard 11: scalar-index coverage surface (physical_rows + index details) --- +// +// `table_store.rs::key_column_index_coverage` mirrors Lance's `create_filter_plan` +// C6 fallback: it reads `fragment.physical_rows` (the field whose absence on ANY +// fragment disables the scalar index for the whole scan) and sniffs the BTREE via +// `load_indices()` → `index.fields` / `index.index_details.type_url`. This is the +// one real Lance-internal coupling on the indexed-traversal read path. If any of +// these surfaces renames or changes type, the coverage check (and the cost-based +// traversal chooser that consumes it) silently misclassifies. Compile-only. + +#[allow( + dead_code, + unreachable_code, + unused_variables, + unused_mut, + clippy::diverging_sub_expression +)] +async fn _compile_scalar_index_coverage_surface() -> lance::Result<()> { + let ds: Dataset = unimplemented!(); + // The create_filter_plan coupling: a fragment lacking `physical_rows` + // disables the scalar index for the entire scan. + for frag in ds.fragments().iter() { + let _physical_rows: Option<usize> = frag.physical_rows; + // `key_column_index_coverage` checks each current fragment id against the + // index `fragment_bitmap`. + let _id: u64 = frag.id; + } + // The index sniff: BTREE presence is detected by single-field index whose + // details type_url ends with "BTreeIndexDetails". The fragment coverage check + // reads `fragment_bitmap` (Option<RoaringBitmap>) and calls `.contains(u32)`. + let indices = ds.load_indices().await?; + for index in indices.iter() { + let _fields: &Vec<i32> = &index.fields; + if let Some(details) = index.index_details.as_ref() { + let _type_url: &str = details.type_url.as_str(); + } + let _covered: Option<bool> = index.fragment_bitmap.as_ref().map(|b| b.contains(0u32)); + } + Ok(()) +} + +// --- Guard 12: can a scalar BTREE be built on a system version column? -------- +// +// The deferred persisted-adjacency artifact plan assumed a cheap delta read of +// `_row_last_updated_at_version > V` could be a BTREE range lookup. Lance resolves +// index columns from the dataset schema, and the version columns are system +// metadata — so this probe documents whether the assumption holds. The outcome is +// the load-bearing fact, not a pass/fail of intent: if this starts SUCCEEDING when +// it currently errors (or vice versa), the artifact's delta-cost story changes. + +#[tokio::test] +async fn scalar_index_on_system_version_column_probe() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().join("guard12.lance"); + let mut ds = fresh_dataset(uri.to_str().unwrap()).await; + + // Sanity: the system version column is present (stable row ids + V2_2). + assert!( + ds.schema().field("_row_last_updated_at_version").is_none(), + "PROBE NOTE: `_row_last_updated_at_version` is NOT in the user schema \ + (it is system metadata); indexing it resolves through a different path." + ); + + let result = ds + .create_index_builder( + &["_row_last_updated_at_version"], + IndexType::BTree, + &ScalarIndexParams::default(), + ) + .replace(true) + .await; + + // Pin the observed behavior: a scalar index on the system version column is + // NOT buildable via the normal create-index path in this Lance. If this turns + // green (Ok), the artifact delta CAN use a version-column BTREE — revisit the + // deferred plan's Phase-2 delta-cost note in docs/dev/traversal handoff. + assert!( + result.is_err(), + "create_index on `_row_last_updated_at_version` unexpectedly SUCCEEDED — \ + a system-column scalar index is now buildable; the persisted-artifact \ + delta read could use it. Update the deferred-design notes." + ); +} + +// --- Guard 13: per-fragment deletion metadata is exposed without a scan ------- +// +// The deferred artifact's delete-correctness coverage model needs to detect, +// cheaply (O(fragments), no row scan), that a covered fragment acquired new +// deletions. That hinges on Lance tracking deletions at fragment-metadata level. +// This pins that a delete populates `fragment.deletion_file`, and probes whether +// the deleted-row COUNT is available as metadata (`num_deleted_rows`) — the +// difference between an O(fragments) coverage check and an O(|E|) scan. + +#[tokio::test] +async fn fragment_deletion_metadata_is_available() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().join("guard13.lance"); + let ds = fresh_dataset(uri.to_str().unwrap()).await; // 2 rows: alice, bob + + let deleted: DeleteResult = { + let mut ds = ds; + ds.delete("id = 'alice'").await.unwrap() + }; + assert_eq!(deleted.num_deleted_rows, 1, "one row deleted"); + let ds = deleted.new_dataset; + + // A delete must be tracked at fragment-metadata level (not only in data). + let with_deletion = ds + .fragments() + .iter() + .find(|f| f.deletion_file.is_some()) + .expect( + "after a delete, some fragment must carry a deletion_file — if not, \ + Lance changed deletion tracking; the artifact coverage model's \ + cheap delete-detection assumption is invalid.", + ); + + // Probe: is the deleted-row count available as metadata (cheap), or must the + // deletion vector be read? Pin whichever holds so the artifact plan knows. + let count: Option<usize> = with_deletion + .deletion_file + .as_ref() + .and_then(|df| df.num_deleted_rows); + assert_eq!( + count, + Some(1), + "PROBE: deletion_file.num_deleted_rows is not a populated metadata count \ + (got {count:?}); the artifact coverage model cannot cheaply detect \ + per-fragment deletions and would need to read the deletion vector.", + ); +} diff --git a/crates/omnigraph/tests/literal_filters.rs b/crates/omnigraph/tests/literal_filters.rs new file mode 100644 index 0000000..a0b2bd7 --- /dev/null +++ b/crates/omnigraph/tests/literal_filters.rs @@ -0,0 +1,96 @@ +//! Execution goldens for filtering by non-string/non-integer scalar LITERALS +//! (F64, F32, Bool, Date, DateTime), across both the in-memory comparison arm +//! (standalone `$m.prop op lit`) and the Lance-pushdown arm (inline binding +//! `Metric { prop: lit }`). Param-bound scalar filters and list-column +//! `contains` are already covered elsewhere; this closes the literal-RHS gap. + +mod helpers; + +use arrow_array::{Array, StringArray}; + +use omnigraph::db::Omnigraph; +use omnigraph::loader::{LoadMode, load_jsonl}; +use omnigraph_compiler::ir::ParamMap; + +use helpers::*; + +const SCHEMA: &str = r#" +node Metric { + name: String @key + score: F64? + ratio: F32? + active: Bool? + born: Date? + seen: DateTime? +} +"#; + +// Seeds partition every predicate, so a dropped filter returns all 4 rows. +const DATA: &str = r#"{"type":"Metric","data":{"name":"m1","score":2.5,"ratio":0.5,"active":true,"born":"2024-06-01","seen":"2024-06-01T12:00:00Z"}} +{"type":"Metric","data":{"name":"m2","score":1.0,"ratio":0.25,"active":false,"born":"2023-01-01","seen":"2023-01-01T00:00:00Z"}} +{"type":"Metric","data":{"name":"m3","score":3.0,"ratio":0.75,"active":true,"born":"2025-01-01","seen":"2025-01-01T00:00:00Z"}} +{"type":"Metric","data":{"name":"m4","score":0.5,"ratio":0.1,"active":false,"born":"2022-12-31","seen":"2022-01-01T00:00:00Z"}}"#; + +async fn metric_db(dir: &tempfile::TempDir) -> Omnigraph { + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, SCHEMA).await.unwrap(); + load_jsonl(&mut db, DATA, LoadMode::Overwrite).await.unwrap(); + db +} + +async fn sorted_metric_names(db: &mut Omnigraph, queries: &str, name: &str) -> Vec<String> { + let r = query_main(db, queries, name, &ParamMap::new()).await.unwrap(); + if r.num_rows() == 0 { + return Vec::new(); + } + let b = r.concat_batches().unwrap(); + let col = b.column(0).as_any().downcast_ref::<StringArray>().unwrap(); + let mut v: Vec<String> = (0..col.len()).map(|i| col.value(i).to_string()).collect(); + v.sort(); + v +} + +#[tokio::test] +async fn float_literal_filters_execute() { + let dir = tempfile::tempdir().unwrap(); + let mut db = metric_db(&dir).await; + let q = r#" +query gt() { match { $m: Metric $m.score > 1.5 } return { $m.name } } +query le() { match { $m: Metric $m.ratio <= 0.25 } return { $m.name } } +query inline() { match { $m: Metric { score: 3.0 } } return { $m.name } } +"#; + // F64 standalone: scores 2.5, 3.0 > 1.5 + assert_eq!(sorted_metric_names(&mut db, q, "gt").await, vec!["m1", "m3"]); + // F32 standalone: ratios 0.25, 0.1 <= 0.25 + assert_eq!(sorted_metric_names(&mut db, q, "le").await, vec!["m2", "m4"]); + // F64 inline-binding pushdown: score == 3.0 + assert_eq!(sorted_metric_names(&mut db, q, "inline").await, vec!["m3"]); +} + +#[tokio::test] +async fn bool_literal_filters_execute() { + let dir = tempfile::tempdir().unwrap(); + let mut db = metric_db(&dir).await; + let q = r#" +query standalone() { match { $m: Metric $m.active = true } return { $m.name } } +query inline() { match { $m: Metric { active: true } } return { $m.name } } +query negated() { match { $m: Metric $m.active != true } return { $m.name } } +"#; + assert_eq!(sorted_metric_names(&mut db, q, "standalone").await, vec!["m1", "m3"]); + assert_eq!(sorted_metric_names(&mut db, q, "inline").await, vec!["m1", "m3"]); + assert_eq!(sorted_metric_names(&mut db, q, "negated").await, vec!["m2", "m4"]); +} + +#[tokio::test] +async fn date_and_datetime_literal_filters_execute() { + let dir = tempfile::tempdir().unwrap(); + let mut db = metric_db(&dir).await; + let q = r#" +query born_ge() { match { $m: Metric $m.born >= date("2024-01-01") } return { $m.name } } +query seen_lt() { match { $m: Metric $m.seen < datetime("2024-01-01T00:00:00Z") } return { $m.name } } +"#; + // born: m1 2024-06, m3 2025 >= 2024-01-01 + assert_eq!(sorted_metric_names(&mut db, q, "born_ge").await, vec!["m1", "m3"]); + // seen: m2 2023, m4 2022 < 2024-01-01 + assert_eq!(sorted_metric_names(&mut db, q, "seen_lt").await, vec!["m2", "m4"]); +} diff --git a/crates/omnigraph/tests/merge_truth_table.rs b/crates/omnigraph/tests/merge_truth_table.rs index 068b439..e2df882 100644 --- a/crates/omnigraph/tests/merge_truth_table.rs +++ b/crates/omnigraph/tests/merge_truth_table.rs @@ -941,8 +941,8 @@ async fn merge_pair_truth_table() { unsupported_cells, 45, "expected 45 cells involving dropProperty/addLabel/removeLabel" ); - assert!( - elapsed.as_secs() < 30, - "merge truth table exceeded 30s budget: {elapsed:?}" - ); + // No wall-clock assertion here: `elapsed` is logged above for visibility, but + // a fixed time budget in a correctness test flakes under parallel test load + // (it tripped at ~31s in the full `--test-threads=4` gate while passing at + // ~20s in isolation). Merge-perf regressions belong in a bench, not here. } diff --git a/crates/omnigraph/tests/ordering.rs b/crates/omnigraph/tests/ordering.rs new file mode 100644 index 0000000..4e9296b --- /dev/null +++ b/crates/omnigraph/tests/ordering.rs @@ -0,0 +1,134 @@ +//! ORDER BY golden coverage: descending, multi-key precedence, deterministic +//! tie-break (total order), and NULL placement. +//! +//! These pin the observable output-ordering contract (deny-list: "output +//! ordering … become dependencies once shipped"). `apply_ordering` appends the +//! bound entities' key columns as an ascending tie-break, so equal user-sort +//! keys yield a TOTAL, deterministic order (and `ORDER … LIMIT` is +//! deterministic). NULL placement is `nulls_first = !descending` (NULLs first +//! under ASC, last under DESC). Both are documented in +//! `docs/user/query-language.md`. + +mod helpers; + +use arrow_array::{Array, StringArray}; + +use omnigraph::db::Omnigraph; +use omnigraph::loader::{LoadMode, load_jsonl}; +use omnigraph_compiler::ir::ParamMap; +use omnigraph_compiler::result::QueryResult; + +use helpers::*; + +/// Names in result ROW order (not sorted) — these tests assert positional order. +fn names_in_order(result: &QueryResult) -> Vec<String> { + let batch = result.concat_batches().unwrap(); + if batch.num_rows() == 0 { + return Vec::new(); + } + let col = batch + .column(0) + .as_any() + .downcast_ref::<StringArray>() + .unwrap(); + (0..col.len()).map(|i| col.value(i).to_string()).collect() +} + +/// Init the standard schema and load a custom Person-only dataset. +async fn init_people(dir: &tempfile::TempDir, jsonl: &str) -> Omnigraph { + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + load_jsonl(&mut db, jsonl, LoadMode::Overwrite).await.unwrap(); + db +} + +#[tokio::test] +async fn ordering_descending() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_and_load(&dir).await; + let q = r#" +query q() { + match { $p: Person } + return { $p.name } + order { $p.age desc } +} +"#; + let got = names_in_order(&query_main(&mut db, q, "q", &ParamMap::new()).await.unwrap()); + // Charlie(35), Alice(30), Diana(28), Bob(25) + assert_eq!(got, vec!["Charlie", "Alice", "Diana", "Bob"]); +} + +#[tokio::test] +async fn ordering_multi_key_age_desc_name_asc() { + let dir = tempfile::tempdir().unwrap(); + // Alice & Bob tie at age 30; loaded Bob-first so the expected output order + // cannot be the load order. + let data = r#"{"type":"Person","data":{"name":"Bob","age":30}} +{"type":"Person","data":{"name":"Alice","age":30}} +{"type":"Person","data":{"name":"Charlie","age":25}}"#; + let mut db = init_people(&dir, data).await; + let q = r#" +query q() { + match { $p: Person } + return { $p.name } + order { $p.age desc, $p.name asc } +} +"#; + let got = names_in_order(&query_main(&mut db, q, "q", &ParamMap::new()).await.unwrap()); + // age desc -> [30,30,25]; the 30-tie broken by name asc -> Alice before Bob. + assert_eq!(got, vec!["Alice", "Bob", "Charlie"]); +} + +#[tokio::test] +async fn ordering_tiebreak_by_key_is_deterministic() { + let dir = tempfile::tempdir().unwrap(); + // Same tie at age 30, NO secondary sort key. Loaded Bob-first; the tie must + // break by the entity key (name) ascending -> Alice before Bob, regardless + // of load order. This locks the total-order tie-break in apply_ordering. + let data = r#"{"type":"Person","data":{"name":"Bob","age":30}} +{"type":"Person","data":{"name":"Alice","age":30}} +{"type":"Person","data":{"name":"Charlie","age":25}}"#; + let mut db = init_people(&dir, data).await; + let q = r#" +query q() { + match { $p: Person } + return { $p.name } + order { $p.age asc } +} +"#; + let got = names_in_order(&query_main(&mut db, q, "q", &ParamMap::new()).await.unwrap()); + // age asc -> Charlie(25), then the 30-tie broken by key asc -> Alice, Bob. + assert_eq!(got, vec!["Charlie", "Alice", "Bob"]); +} + +#[tokio::test] +async fn ordering_nulls_placement_asc_and_desc() { + let dir = tempfile::tempdir().unwrap(); + // Bob has a NULL age. + let data = r#"{"type":"Person","data":{"name":"Alice","age":30}} +{"type":"Person","data":{"name":"Bob","age":null}} +{"type":"Person","data":{"name":"Charlie","age":25}}"#; + let mut db = init_people(&dir, data).await; + + let asc = r#" +query q() { + match { $p: Person } + return { $p.name } + order { $p.age asc } +} +"#; + let got_asc = names_in_order(&query_main(&mut db, asc, "q", &ParamMap::new()).await.unwrap()); + // ASC: nulls_first -> Bob(null), then 25, 30. + assert_eq!(got_asc, vec!["Bob", "Charlie", "Alice"]); + + let desc = r#" +query q() { + match { $p: Person } + return { $p.name } + order { $p.age desc } +} +"#; + let got_desc = names_in_order(&query_main(&mut db, desc, "q", &ParamMap::new()).await.unwrap()); + // DESC: nulls last -> 30, 25, then Bob(null). + assert_eq!(got_desc, vec!["Alice", "Charlie", "Bob"]); +} diff --git a/crates/omnigraph/tests/proptest_equivalence.rs b/crates/omnigraph/tests/proptest_equivalence.rs new file mode 100644 index 0000000..3423a2f --- /dev/null +++ b/crates/omnigraph/tests/proptest_equivalence.rs @@ -0,0 +1,311 @@ +//! Property-based query-correctness invariants over generated graphs. +//! +//! The cross-type id-collision bug (fixed in f6a0e53) was a silent wrong-result +//! divergence between the two Expand modes, caught only because someone +//! hand-built the one colliding fixture. This turns that single example into a +//! search over the whole class: node keys for BOTH types are drawn from a small +//! SHARED alphabet, so cross-type collisions — plus cycles and self-loops — +//! arise frequently. The invariants make any future fork divergence (the planned +//! third ExpandMode, the anti-join fast/slow fork) fail loudly instead of +//! silently. +//! +//! Each test is a sync `#[test]` + `#[serial]`: it builds its own runtime and +//! `block_on`s per generated case (proptest closures are sync), and the +//! mode-equivalence test writes `OMNIGRAPH_TRAVERSAL_MODE`, so serial execution +//! keeps env writes from racing other tests in this binary. + +mod helpers; + +use std::collections::HashSet; + +use arrow_array::{Array, StringArray}; +use proptest::prelude::*; +use proptest::test_runner::{Config, TestRunner}; +use serial_test::serial; + +use omnigraph::db::{Omnigraph, ReadTarget}; +use omnigraph::loader::{LoadMode, load_jsonl}; +use omnigraph_compiler::ir::ParamMap; +use omnigraph_compiler::query::ast::Literal; + +use helpers::*; + +/// Small SHARED key alphabet — Person and Company keys are both drawn from this, +/// so cross-type id collisions are common. +const KEYS: &[&str] = &["a", "b", "c", "d", "e"]; + +const QUERIES: &str = r#" +query friends($name: String) { + match { + $p: Person { name: $name } + $p knows{1,3} $f + } + return { $f.name } +} +query employers($name: String) { + match { + $p: Person { name: $name } + $p worksAt{1,2} $c + } + return { $c.name } +} +query all_persons() { + match { $p: Person } + return { $p.name } +} +query employed() { + match { + $p: Person + $p worksAt $c + } + return { $p.name } +} +query unemployed() { + match { + $p: Person + not { $p worksAt $_ } + } + return { $p.name } +} +"#; + +#[derive(Debug, Clone)] +struct GenGraph { + persons: Vec<String>, + companies: Vec<String>, + knows: Vec<(usize, usize)>, // indices into persons (self-loops & cycles allowed) + works_at: Vec<(usize, usize)>, // (person idx, company idx) +} + +impl GenGraph { + fn to_jsonl(&self) -> String { + let mut s = String::new(); + for p in &self.persons { + s.push_str(&format!("{{\"type\":\"Person\",\"data\":{{\"name\":\"{p}\"}}}}\n")); + } + for c in &self.companies { + s.push_str(&format!("{{\"type\":\"Company\",\"data\":{{\"name\":\"{c}\"}}}}\n")); + } + // Dedup exact-duplicate edge rows (the loader rejects intra-batch + // duplicate keys); collisions/cycles/self-loops are unaffected. + let mut seen = HashSet::new(); + for &(a, b) in &self.knows { + if seen.insert(("k", a, b)) { + s.push_str(&format!( + "{{\"edge\":\"Knows\",\"from\":\"{}\",\"to\":\"{}\"}}\n", + self.persons[a], self.persons[b] + )); + } + } + for &(a, b) in &self.works_at { + if seen.insert(("w", a, b)) { + s.push_str(&format!( + "{{\"edge\":\"WorksAt\",\"from\":\"{}\",\"to\":\"{}\"}}\n", + self.persons[a], self.companies[b] + )); + } + } + s + } +} + +fn arb_keys() -> impl Strategy<Value = Vec<String>> { + proptest::sample::subsequence(KEYS.to_vec(), 1..=KEYS.len()) + .prop_map(|v| v.into_iter().map(String::from).collect()) +} + +fn arb_graph() -> impl Strategy<Value = GenGraph> { + (arb_keys(), arb_keys()).prop_flat_map(|(persons, companies)| { + let np = persons.len(); + let nc = companies.len(); + let knows = prop::collection::vec((0..np, 0..np), 0..=10); + let works = prop::collection::vec((0..np, 0..nc), 0..=10); + (Just(persons), Just(companies), knows, works).prop_map( + |(persons, companies, knows, works_at)| GenGraph { + persons, + companies, + knows, + works_at, + }, + ) + }) +} + +fn config() -> Config { + Config { + cases: 48, + ..Config::default() + } +} + +fn clear_mode() { + unsafe { std::env::remove_var("OMNIGRAPH_TRAVERSAL_MODE") }; +} + +/// RAII guard that sets `OMNIGRAPH_TRAVERSAL_MODE` and clears it on drop — so a +/// panic mid-case (e.g. a query `unwrap`) cannot leak the forced mode into +/// proptest's subsequent shrink/cases and mask the divergence under test. SAFE: +/// every test in this binary is `#[serial]`, so no thread reads the env during +/// the write. +struct ModeGuard; +impl ModeGuard { + fn set(mode: &str) -> Self { + unsafe { std::env::set_var("OMNIGRAPH_TRAVERSAL_MODE", mode) }; + ModeGuard + } +} +impl Drop for ModeGuard { + fn drop(&mut self) { + unsafe { std::env::remove_var("OMNIGRAPH_TRAVERSAL_MODE") }; + } +} + +async fn load_graph(graph: &GenGraph) -> (tempfile::TempDir, Omnigraph) { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + load_jsonl(&mut db, &graph.to_jsonl(), LoadMode::Overwrite) + .await + .unwrap(); + (dir, db) +} + +fn one_param(val: &str) -> ParamMap { + let mut m = ParamMap::new(); + m.insert("name".to_string(), Literal::String(val.to_string())); + m +} + +/// First-column strings, sorted (MULTISET — preserves duplicate-row count so +/// mode comparisons catch dedup divergence, not just set divergence). +async fn col0_sorted(db: &mut Omnigraph, name: &str, params: &ParamMap) -> Vec<String> { + let r = db + .query(ReadTarget::branch("main"), QUERIES, name, params) + .await + .unwrap(); + if r.num_rows() == 0 { + return Vec::new(); + } + let b = r.concat_batches().unwrap(); + let col = b.column(0).as_any().downcast_ref::<StringArray>().unwrap(); + let mut v: Vec<String> = (0..col.len()).map(|i| col.value(i).to_string()).collect(); + v.sort(); + v +} + +async fn col0_set(db: &mut Omnigraph, name: &str, params: &ParamMap) -> HashSet<String> { + col0_sorted(db, name, params).await.into_iter().collect() +} + +// INVARIANT 1: mode equivalence. For any generated graph and start key, the +// CSR, indexed, and auto paths return identical result multisets — over both a +// same-type traversal (knows{1,3}, exercises cycles/self-loops) and a cross-type +// one (worksAt{1,2}, collision-prone). This is the search-over-the-class version +// of the hand-built cross-type-collision fixture. +#[test] +#[serial] +fn prop_expand_indexed_eq_csr() { + let rt = tokio::runtime::Runtime::new().unwrap(); + let mut runner = TestRunner::new(config()); + runner + .run(&arb_graph(), |graph| { + let mismatch = rt.block_on(async { + let (_dir, mut db) = load_graph(&graph).await; + for start in graph.persons.clone() { + let p = one_param(&start); + for q in ["friends", "employers"] { + // Each guard clears the mode on drop (end of the block, + // or on panic), so a forced mode never leaks across runs. + let csr = { + let _g = ModeGuard::set("csr"); + col0_sorted(&mut db, q, &p).await + }; + let indexed = { + let _g = ModeGuard::set("indexed"); + col0_sorted(&mut db, q, &p).await + }; + // No guard → env unset → auto (cost-based) path. + let auto = col0_sorted(&mut db, q, &p).await; + if csr != indexed || csr != auto { + return Some((start, q, csr, indexed, auto)); + } + } + } + None + }); + prop_assert!( + mismatch.is_none(), + "Expand mode divergence: {:?}", + mismatch + ); + Ok(()) + }) + .unwrap(); +} + +// INVARIANT 2: no phantom rows. Every key a traversal returns must belong to the +// destination type's loaded key set — independent of the two-mode comparison, so +// it catches over-emission even if both modes are wrong identically. +#[test] +#[serial] +fn prop_results_subset_of_existing_nodes() { + clear_mode(); + let rt = tokio::runtime::Runtime::new().unwrap(); + let mut runner = TestRunner::new(config()); + runner + .run(&arb_graph(), |graph| { + let bad = rt.block_on(async { + let (_dir, mut db) = load_graph(&graph).await; + let persons: HashSet<String> = graph.persons.iter().cloned().collect(); + let companies: HashSet<String> = graph.companies.iter().cloned().collect(); + for start in graph.persons.clone() { + let p = one_param(&start); + for f in col0_set(&mut db, "friends", &p).await { + if !persons.contains(&f) { + return Some(("friends", start, f)); + } + } + for c in col0_set(&mut db, "employers", &p).await { + if !companies.contains(&c) { + return Some(("employers", start, c)); + } + } + } + None + }); + prop_assert!(bad.is_none(), "phantom row: {:?}", bad); + Ok(()) + }) + .unwrap(); +} + +// INVARIANT 3: anti-join complement. `not { $p worksAt $_ }` and its complement +// (persons WITH a worksAt) must be disjoint and together cover all persons. +#[test] +#[serial] +fn prop_antijoin_partitions_persons() { + clear_mode(); + let rt = tokio::runtime::Runtime::new().unwrap(); + let mut runner = TestRunner::new(config()); + runner + .run(&arb_graph(), |graph| { + let err = rt.block_on(async { + let (_dir, mut db) = load_graph(&graph).await; + let all = col0_set(&mut db, "all_persons", &ParamMap::new()).await; + let unemployed = col0_set(&mut db, "unemployed", &ParamMap::new()).await; + let employed = col0_set(&mut db, "employed", &ParamMap::new()).await; + let overlap: Vec<_> = unemployed.intersection(&employed).cloned().collect(); + let union: HashSet<_> = unemployed.union(&employed).cloned().collect(); + if !overlap.is_empty() { + return Some(format!("overlap {overlap:?}")); + } + if union != all { + return Some(format!("union {union:?} != all {all:?}")); + } + None + }); + prop_assert!(err.is_none(), "anti-join partition broken: {:?}", err); + Ok(()) + }) + .unwrap(); +} diff --git a/crates/omnigraph/tests/search.rs b/crates/omnigraph/tests/search.rs index c4454cf..480ec3c 100644 --- a/crates/omnigraph/tests/search.rs +++ b/crates/omnigraph/tests/search.rs @@ -556,6 +556,111 @@ async fn bm25_returns_ranked_results() { assert!(result.num_rows() <= 3, "bm25 should respect limit 3"); } +// Full rank-ORDER golden (not just top-1 / non-empty): pins ranks 2..k so a +// regression corrupting the tail or reversing the sort direction fails loudly. +// nearest skips apply_ordering (is_search_ordered) and returns Lance native +// order, so result_slugs row order == rank order. +#[tokio::test] +#[serial] +async fn nearest_full_rank_order() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_search_db(&dir).await; + let result = query_main( + &mut db, + SEARCH_QUERIES, + "vector_search", + &vector_param("$q", &[0.1, 0.2, 0.3, 0.4]), + ) + .await + .unwrap(); + // [0.1,0.2,0.3,0.4] == ml-intro's embedding (dist 0); the rest by ascending L2. + assert_eq!(result_slugs(&result), vec!["ml-intro", "nlp-guide", "rl-intro"]); +} + +#[tokio::test] +#[serial] +async fn bm25_full_rank_order() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_search_db(&dir).await; + let result = query_main( + &mut db, + SEARCH_QUERIES, + "bm25_search", + ¶ms(&[("$q", "Learning")]), + ) + .await + .unwrap(); + // Descending BM25 score order. + assert_eq!(result_slugs(&result), vec!["rl-intro", "ml-intro", "dl-basics"]); +} + +// Characterization: fuzzy() does NOT match under the default tokenizer/index in +// this setup — a one-edit typo ("Introductio" for "Introduction") returns no +// rows. (`search`/`match_text` DO work, so FTS itself is fine; fuzzy term +// queries specifically are inert here.) This pins that documented limitation +// instead of leaving fuzzy silently unasserted: if a Lance/tokenizer change +// makes fuzzy match, this turns red and should be promoted to a real +// matched-set + exclusion golden. +#[tokio::test] +#[serial] +async fn fuzzy_does_not_match_under_default_tokenizer() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_search_db(&dir).await; + let r = query_main(&mut db, SEARCH_QUERIES, "fuzzy_search", ¶ms(&[("$q", "Introductio")])) + .await + .unwrap(); + assert!( + result_slugs(&r).is_empty(), + "fuzzy now matches — promote this to a real matched-set/exclusion golden" + ); +} + +// match_text is a FILTER on the body: assert the exact matched set, not contains. +#[tokio::test] +#[serial] +async fn match_text_matches_exact_set_excludes_unrelated() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_search_db(&dir).await; + // "neural" appears only in dl-basics's body ("neural networks"). + let r = query_main(&mut db, SEARCH_QUERIES, "phrase_search", ¶ms(&[("$q", "neural")])) + .await + .unwrap(); + let mut got = result_slugs(&r); + got.sort(); + assert_eq!(got, vec!["dl-basics"]); +} + +// RRF fuses arms OTHER than the default nearest+bm25: two FTS arms (title+body). +// Proves primary_var resolves when neither arm is `nearest`, and fusion runs. +#[tokio::test] +#[serial] +async fn rrf_fuses_two_fts_fields() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_search_db(&dir).await; + let r = query_main(&mut db, SEARCH_QUERIES, "rrf_two_fts", ¶ms(&[("$q", "learning")])) + .await + .unwrap(); + assert_eq!(result_slugs(&r), vec!["dl-basics", "ml-intro", "rl-intro"]); +} + +// RRF fuses two vector arms (no embedding creds — explicit vectors). A doc near +// BOTH query vectors out-ranks one near only one. +#[tokio::test] +#[serial] +async fn rrf_fuses_two_vector_queries() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_search_db(&dir).await; + let r = query_main( + &mut db, + SEARCH_QUERIES, + "rrf_two_vectors", + &two_vector_params("$q1", &[0.1, 0.2, 0.3, 0.4], "$q2", &[0.5, 0.6, 0.7, 0.8]), + ) + .await + .unwrap(); + assert_eq!(result_slugs(&r), vec!["rl-intro", "ml-intro", "dl-basics"]); +} + #[tokio::test] #[serial] async fn mutation_commit_refreshes_search_indices_without_manual_ensure() { diff --git a/crates/omnigraph/tests/traversal.rs b/crates/omnigraph/tests/traversal.rs index 6efe7de..2f518fd 100644 --- a/crates/omnigraph/tests/traversal.rs +++ b/crates/omnigraph/tests/traversal.rs @@ -46,6 +46,194 @@ query not_at_acme() { assert_eq!(names_vec, vec!["Bob", "Charlie", "Diana"]); } +// Nested anti-join (double negation): proves `not { … not { … } }` recurses +// through execute_pipeline. "People who do NOT work at any NON-Acme company": +// inner `not { $c.name = "Acme" }` keeps the non-Acme employers, the outer `not` +// removes anyone who has one. Alice (Acme only), Charlie & Diana (no employer) +// remain — distinct from plain unemployed {Charlie, Diana}. +#[tokio::test] +async fn nested_anti_join_double_negation() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_and_load(&dir).await; + + let queries = r#" +query no_nonacme_employer() { + match { + $p: Person + not { + $p worksAt $c + not { + $c.name = "Acme" + } + } + } + return { $p.name } +} +"#; + let result = query_main(&mut db, queries, "no_nonacme_employer", &ParamMap::new()) + .await + .unwrap(); + + let batch = result.concat_batches().unwrap(); + let names = batch + .column(0) + .as_any() + .downcast_ref::<StringArray>() + .unwrap(); + let mut names_vec: Vec<&str> = (0..names.len()).map(|i| names.value(i)).collect(); + names_vec.sort(); + assert_eq!(names_vec, vec!["Alice", "Charlie", "Diana"]); +} + +// The anti-join has two execution forks: the CSR `has_neighbors` fast path +// (bare single-op Expand inner) and the set-oriented inner-pipeline replay (when +// dst_filters force a multi-op inner). They must agree. `not { $p worksAt $_ }` +// takes the fast path; the same negation with an always-true dst filter +// (`$c.name != ""`) is semantically identical but forces the slow path. +#[tokio::test] +async fn anti_join_fast_and_slow_paths_agree() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_and_load(&dir).await; + + let queries = r#" +query fast() { + match { + $p: Person + not { $p worksAt $_ } + } + return { $p.name } +} +query slow() { + match { + $p: Person + not { + $p worksAt $c + $c.name != "" + } + } + return { $p.name } +} +"#; + let names = |result: omnigraph_compiler::result::QueryResult| { + let batch = result.concat_batches().unwrap(); + let col = batch + .column(0) + .as_any() + .downcast_ref::<StringArray>() + .unwrap(); + let mut v: Vec<String> = (0..col.len()).map(|i| col.value(i).to_string()).collect(); + v.sort(); + v + }; + + let fast = names(query_main(&mut db, queries, "fast", &ParamMap::new()).await.unwrap()); + let slow = names(query_main(&mut db, queries, "slow", &ParamMap::new()).await.unwrap()); + + assert_eq!(fast, slow, "anti-join fast and slow paths must agree"); + // Alice->Acme, Bob->Globex employed; Charlie & Diana have no employer. + assert_eq!(fast, vec!["Charlie", "Diana"]); +} + +// Regression: nested slow-path anti-joins must not collide on the synthetic +// correlation tag. The outer anti-join tags rows with a correlation column that +// rides through its inner pipeline; when the inner pipeline contains ANOTHER +// slow-path anti-join, a fixed tag name would duplicate, and reading it by name +// returns the OUTER tag — mis-correlating the inner negation. Fan-out (p1 works +// at two companies) makes the inner row indices diverge from the outer tags, so +// the bug produces a different person set than the correct one. +#[tokio::test] +async fn nested_anti_join_with_fanout_correlates_correctly() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + // p1 -> {Acme, Globex} (fan-out), p2 -> Globex, p3 -> Acme, p4 -> (none). + let data = r#"{"type":"Person","data":{"name":"p1"}} +{"type":"Person","data":{"name":"p2"}} +{"type":"Person","data":{"name":"p3"}} +{"type":"Person","data":{"name":"p4"}} +{"type":"Company","data":{"name":"Acme"}} +{"type":"Company","data":{"name":"Globex"}} +{"edge":"WorksAt","from":"p1","to":"Acme"} +{"edge":"WorksAt","from":"p1","to":"Globex"} +{"edge":"WorksAt","from":"p2","to":"Globex"} +{"edge":"WorksAt","from":"p3","to":"Acme"}"#; + let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + load_jsonl(&mut db, data, LoadMode::Overwrite).await.unwrap(); + + let queries = r#" +query no_nonacme_employer() { + match { + $p: Person + not { + $p worksAt $c + not { + $c.name = "Acme" + } + } + } + return { $p.name } +} +"#; + let result = query_main(&mut db, queries, "no_nonacme_employer", &ParamMap::new()) + .await + .unwrap(); + let batch = result.concat_batches().unwrap(); + let names = batch + .column(0) + .as_any() + .downcast_ref::<StringArray>() + .unwrap(); + let mut names_vec: Vec<&str> = (0..names.len()).map(|i| names.value(i)).collect(); + names_vec.sort(); + // p1 & p2 have a non-Acme employer (Globex) -> excluded; p3 (Acme only) and + // p4 (no employer) remain. + assert_eq!(names_vec, vec!["p3", "p4"]); +} + +// Regression: a multi-hop anti-join must not take the bulk fast path. The fast +// path answers via `has_neighbors` (ONE-hop existence), so `not { $p knows{2,2} +// $x }` would wrongly drop a node that has a 1-hop neighbor but no 2-hop path. +// Graph: a->b (b is a sink, so a has no 2-hop path), c->d->e (c has a 2-hop +// path). Only c has a 2-hop knows path, so only c is removed. +#[tokio::test] +async fn anti_join_respects_multi_hop_bounds() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let data = r#"{"type":"Person","data":{"name":"a"}} +{"type":"Person","data":{"name":"b"}} +{"type":"Person","data":{"name":"c"}} +{"type":"Person","data":{"name":"d"}} +{"type":"Person","data":{"name":"e"}} +{"edge":"Knows","from":"a","to":"b"} +{"edge":"Knows","from":"c","to":"d"} +{"edge":"Knows","from":"d","to":"e"}"#; + let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + load_jsonl(&mut db, data, LoadMode::Overwrite).await.unwrap(); + + let queries = r#" +query no_two_hop() { + match { + $p: Person + not { $p knows{2,2} $x } + } + return { $p.name } +} +"#; + let result = query_main(&mut db, queries, "no_two_hop", &ParamMap::new()) + .await + .unwrap(); + let batch = result.concat_batches().unwrap(); + let names = batch + .column(0) + .as_any() + .downcast_ref::<StringArray>() + .unwrap(); + let mut names_vec: Vec<&str> = (0..names.len()).map(|i| names.value(i)).collect(); + names_vec.sort(); + // Only c has a 2-hop knows path → removed; everyone else (incl. a, which has + // a 1-hop neighbor but no 2-hop path) is kept. + assert_eq!(names_vec, vec!["a", "b", "d", "e"]); +} + // ─── Variable-length hops ─────────────────────────────────────────────────── const CHAIN_SCHEMA: &str = r#" diff --git a/crates/omnigraph/tests/traversal_indexed.rs b/crates/omnigraph/tests/traversal_indexed.rs new file mode 100644 index 0000000..2ceed85 --- /dev/null +++ b/crates/omnigraph/tests/traversal_indexed.rs @@ -0,0 +1,327 @@ +//! BTREE-indexed Expand path (`execute_expand_indexed`) coverage. +//! +//! These tests force the Expand execution mode via `OMNIGRAPH_TRAVERSAL_MODE` +//! and assert the indexed path matches the CSR path (both are semantically +//! identical — the indexed path just serves neighbor lookups from the persisted +//! src/dst BTREE instead of an in-memory CSR). They live in their own test +//! binary and are all `#[serial]`, so the env writes never race a concurrent +//! reader: within this process serial execution serializes every env read, and +//! other test binaries (e.g. `traversal.rs`) are separate processes whose env +//! stays unset (→ CSR), validating the shared hydrate/align tail on the CSR path. + +mod helpers; + +use arrow_array::{Array, StringArray}; + +use omnigraph::db::Omnigraph; +use omnigraph::loader::{LoadMode, load_jsonl}; +use omnigraph::table_store::{IndexCoverage, TableStore}; +use omnigraph_compiler::ir::ParamMap; +use serial_test::serial; + +use helpers::*; + +fn set_mode(mode: &str) { + // SAFE: every test here is #[serial] and this binary has no non-serial + // env reader, so no thread reads the environment during this write. + unsafe { std::env::set_var("OMNIGRAPH_TRAVERSAL_MODE", mode) }; +} + +fn clear_mode() { + unsafe { std::env::remove_var("OMNIGRAPH_TRAVERSAL_MODE") }; +} + +/// Run a name-returning query and return its first column, sorted. +async fn sorted_names(db: &mut Omnigraph, queries: &str, name: &str, params: &ParamMap) -> Vec<String> { + let result = query_main(db, queries, name, params).await.unwrap(); + if result.num_rows() == 0 { + return Vec::new(); + } + let batch = result.concat_batches().unwrap(); + let col = batch + .column(0) + .as_any() + .downcast_ref::<StringArray>() + .unwrap(); + let mut v: Vec<String> = (0..col.len()).map(|i| col.value(i).to_string()).collect(); + v.sort(); + v +} + +/// Run the same query under CSR, indexed, and auto (cost-chooser) modes; assert +/// all three produce identical results and return them. The auto pass exercises +/// `choose_expand_mode` end to end: whichever path it selects, the rows must +/// match the forced paths (the chooser changes which path runs, never the result). +async fn both_modes(db: &mut Omnigraph, queries: &str, name: &str, params: &ParamMap) -> Vec<String> { + set_mode("csr"); + let csr = sorted_names(db, queries, name, params).await; + set_mode("indexed"); + let indexed = sorted_names(db, queries, name, params).await; + clear_mode(); + let auto = sorted_names(db, queries, name, params).await; + assert_eq!( + indexed, csr, + "indexed Expand must produce identical results to CSR for query '{name}'" + ); + assert_eq!( + auto, csr, + "auto (cost-chooser) Expand must produce identical results to the forced paths for query '{name}'" + ); + indexed +} + +// The C6 index-coverage guard: `key_column_index_coverage` must report whether +// a `key_col IN (...)` scan will use the persisted BTREE or silently full-scan. +// Not #[serial] — it calls the helper directly and reads no env. +#[tokio::test] +async fn key_column_index_coverage_detects_btree_presence() { + let dir = tempfile::tempdir().unwrap(); + let db = init_and_load(&dir).await; + let snap = snapshot_main(&db).await.unwrap(); + + // Edge `src` gets a BTREE from ensure_indices on load → Indexed. + let edge_ds = snap.open("edge:Knows").await.unwrap(); + let src_cov = TableStore::key_column_index_coverage(&edge_ds, "src") + .await + .unwrap(); + assert_eq!(src_cov, IndexCoverage::Indexed, "edge src is BTREE-indexed"); + + // A node property column with no scalar index → Degraded (the warn path). + let node_ds = snap.open("node:Person").await.unwrap(); + let age_cov = TableStore::key_column_index_coverage(&node_ds, "age") + .await + .unwrap(); + assert!( + matches!(age_cov, IndexCoverage::Degraded { .. }), + "non-indexed column should be Degraded, got {age_cov:?}" + ); +} + +// An edge appended after the BTREE was built lands in a new fragment that the +// index does not cover (edge-index creation is skipped once a BTREE exists). The +// scan is then partly a full scan, so coverage must report `Degraded` — otherwise +// the cost chooser would price an unindexed-in-part scan as fully indexed. +// (Results stay correct regardless — `indexed_finds_unindexed_appended_edge`.) +#[tokio::test] +async fn coverage_degrades_for_appended_unindexed_fragment() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_and_load(&dir).await; + + // Fresh load: the Knows BTREE covers every fragment → Indexed. + let snap = snapshot_main(&db).await.unwrap(); + let edge_ds = snap.open("edge:Knows").await.unwrap(); + assert_eq!( + TableStore::key_column_index_coverage(&edge_ds, "src").await.unwrap(), + IndexCoverage::Indexed, + "freshly-loaded edge BTREE covers all fragments" + ); + + // Append an edge → a new, unindexed fragment outside the index fragment_bitmap. + mutate_main( + &mut db, + MUTATION_QUERIES, + "add_friend", + ¶ms(&[("$from", "Alice"), ("$to", "Diana")]), + ) + .await + .unwrap(); + + let snap2 = snapshot_main(&db).await.unwrap(); + let edge_ds2 = snap2.open("edge:Knows").await.unwrap(); + let cov = TableStore::key_column_index_coverage(&edge_ds2, "src").await.unwrap(); + assert!( + matches!(cov, IndexCoverage::Degraded { .. }), + "appended unindexed fragment must degrade coverage, got {cov:?}" + ); +} + +#[tokio::test] +#[serial] +async fn indexed_matches_csr_one_hop_same_type() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_and_load(&dir).await; + // friends_of: `$p knows $f` (Person -> Person, single hop). + let got = both_modes(&mut db, TEST_QUERIES, "friends_of", ¶ms(&[("$name", "Alice")])).await; + assert_eq!(got, vec!["Bob", "Charlie"], "Alice knows Bob and Charlie"); +} + +#[tokio::test] +#[serial] +async fn indexed_matches_csr_multi_hop_same_type() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_and_load(&dir).await; + let queries = r#" +query reach($name: String) { + match { + $p: Person { name: $name } + $p knows{1,2} $f + } + return { $f.name } +} +"#; + // Alice -> Bob, Charlie (1 hop); Bob -> Diana (2 hops). + let got = both_modes(&mut db, queries, "reach", ¶ms(&[("$name", "Alice")])).await; + assert_eq!(got, vec!["Bob", "Charlie", "Diana"]); +} + +#[tokio::test] +#[serial] +async fn indexed_matches_csr_cross_type() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_and_load(&dir).await; + let queries = r#" +query employer($name: String) { + match { + $p: Person { name: $name } + $p worksAt $c + } + return { $c.name } +} +"#; + let got = both_modes(&mut db, queries, "employer", ¶ms(&[("$name", "Alice")])).await; + assert_eq!(got, vec!["Acme"], "Alice works at Acme"); +} + +#[tokio::test] +#[serial] +async fn indexed_matches_csr_no_match() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_and_load(&dir).await; + // Diana has no outgoing Knows edges → empty in both modes. + let got = both_modes(&mut db, TEST_QUERIES, "friends_of", ¶ms(&[("$name", "Diana")])).await; + assert!(got.is_empty(), "Diana knows no one"); +} + +#[tokio::test] +#[serial] +async fn indexed_finds_unindexed_appended_edge() { + let dir = tempfile::tempdir().unwrap(); + let mut db = init_and_load(&dir).await; + + // Append Alice -> Diana AFTER the initial load. `ensure_indices`' existence + // guard means the src/dst BTREE built on the first load does NOT cover this + // new fragment. The indexed path must still find it via Lance's + // unindexed-fragment scan (fast_search=false default), so partial index + // coverage never silently drops rows. + mutate_main( + &mut db, + MUTATION_QUERIES, + "add_friend", + ¶ms(&[("$from", "Alice"), ("$to", "Diana")]), + ) + .await + .unwrap(); + + set_mode("indexed"); + let got = sorted_names(&mut db, TEST_QUERIES, "friends_of", ¶ms(&[("$name", "Alice")])).await; + clear_mode(); + + assert_eq!( + got, + vec!["Bob", "Charlie", "Diana"], + "indexed traversal must see the freshly-appended, unindexed edge" + ); +} + +// Regression: a node `id` is unique only WITHIN a type, so a `Person` and a +// `Company` can share an id string. A variable-length traversal over a +// cross-type edge (`worksAt`, Person -> Company) must structurally stop after +// one hop — a Company is not a `worksAt` source — so `worksAt{1,2}` returns +// exactly the one-hop companies. Before the structural hop-cap, the indexed +// path's single string interner de-interned the hop-1 Company id back to the +// colliding Person id and ran a hop-2 `worksAt src IN (...)` scan that matched +// that same-string Person's edges, emitting a spurious second-hop company the +// CSR path never produces. `both_modes` (csr == indexed == auto) plus the +// golden assert catch both the divergence and an over-emitting shared bug. +#[tokio::test] +#[serial] +async fn cross_type_id_collision_does_not_bleed_into_second_hop() { + const SCHEMA: &str = r#" +node Person { name: String @key } +node Company { name: String @key } +edge WorksAt: Person -> Company +"#; + // `shared` is BOTH a Person id and a Company id. alice worksAt the Company + // `shared`; the Person `shared` worksAt the Company `other`. + const DATA: &str = r#"{"type":"Person","data":{"name":"alice"}} +{"type":"Person","data":{"name":"shared"}} +{"type":"Company","data":{"name":"shared"}} +{"type":"Company","data":{"name":"other"}} +{"edge":"WorksAt","from":"alice","to":"shared"} +{"edge":"WorksAt","from":"shared","to":"other"}"#; + const QUERY: &str = r#" +query reach($name: String) { + match { + $p: Person { name: $name } + $p worksAt{1,2} $c + } + return { $c.name } +} +"#; + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, SCHEMA).await.unwrap(); + load_jsonl(&mut db, DATA, LoadMode::Overwrite).await.unwrap(); + + let got = both_modes(&mut db, QUERY, "reach", ¶ms(&[("$name", "alice")])).await; + assert_eq!( + got, + vec!["shared"], + "cross-type worksAt{{1,2}} must return only the one-hop company; a hop-2 \ + result means the id-string collision bled across types" + ); +} + +const REACH_5: &str = r#" +query reach($name: String) { + match { + $p: Person { name: $name } + $p knows{1,5} $f + } + return { $f.name } +} +"#; + +// A directed 3-cycle a->b->c->a, traversed with a hop ceiling (5) ABOVE the cycle +// length. Variable-length traversal must terminate and dedup (the source is +// seeded into `visited`, so the c->a back-edge does not re-emit a). Uses a +// bounded range deliberately: an unbounded `{1,}` is a typecheck error, not a +// runtime path. `both_modes` also confirms indexed == csr on the cycle. +#[tokio::test] +#[serial] +async fn variable_hops_terminate_and_dedup_on_cycle() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let data = r#"{"type":"Person","data":{"name":"a"}} +{"type":"Person","data":{"name":"b"}} +{"type":"Person","data":{"name":"c"}} +{"edge":"Knows","from":"a","to":"b"} +{"edge":"Knows","from":"b","to":"c"} +{"edge":"Knows","from":"c","to":"a"}"#; + let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + load_jsonl(&mut db, data, LoadMode::Overwrite).await.unwrap(); + + let got = both_modes(&mut db, REACH_5, "reach", ¶ms(&[("$name", "a")])).await; + // From a: b (1 hop), c (2 hops); the c->a back-edge hits the seeded source + // and is not re-emitted. No infinite loop, each node at most once. + assert_eq!(got, vec!["b", "c"]); +} + +// A self-loop a->a plus a->b. Variable-length traversal must not loop forever and +// must not re-emit the seeded source. +#[tokio::test] +#[serial] +async fn variable_hops_handle_self_loop() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let data = r#"{"type":"Person","data":{"name":"a"}} +{"type":"Person","data":{"name":"b"}} +{"edge":"Knows","from":"a","to":"a"} +{"edge":"Knows","from":"a","to":"b"}"#; + let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + load_jsonl(&mut db, data, LoadMode::Overwrite).await.unwrap(); + + let got = both_modes(&mut db, REACH_5, "reach", ¶ms(&[("$name", "a")])).await; + // a->a hits the seeded source (pruned); only b is reached. + assert_eq!(got, vec!["b"]); +} diff --git a/docs/user/constants.md b/docs/user/constants.md index 210155e..f523042 100644 --- a/docs/user/constants.md +++ b/docs/user/constants.md @@ -13,6 +13,10 @@ | Maintenance concurrency | `OMNIGRAPH_MAINTENANCE_CONCURRENCY=8` | `db/omnigraph/optimize.rs` | | Lance blob compaction support | `LANCE_SUPPORTS_BLOB_COMPACTION = false` | `db/omnigraph/optimize.rs` | | Graph index cache size | `8` (LRU) | `runtime_cache.rs` | +| Expand indexed-path frontier ceiling | `OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER=1024` | `exec/query.rs` | +| Expand indexed-path hop ceiling | `OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS=6` | `exec/query.rs` | +| Expand CSR-build cost factor | `CSR_BUILD_FACTOR = 1.5` | `exec/query.rs` | +| Expand mode override | `OMNIGRAPH_TRAVERSAL_MODE` (`indexed`\|`csr`; unset = cost-based auto) | `exec/query.rs` | | Default body limit | `1 MB` | `omnigraph-server/lib.rs` | | Ingest body limit | `32 MB` | `omnigraph-server/lib.rs` | | Engine embed model | `gemini-embedding-2-preview` | `omnigraph/embedding.rs` | @@ -21,3 +25,16 @@ | Embed retries | `4` | both clients | | Embed retry backoff | `200 ms` | both clients | | LANCE memory pool default | `1 GB` (raised in v0.3.0) | runtime | + +**Expand traversal dispatch.** With `OMNIGRAPH_TRAVERSAL_MODE` unset, the engine +chooses the indexed (per-hop BTREE) vs CSR (whole-graph in-memory) path with a +cost model over cheap manifest counts (frontier size, |E|, source-vertex count, +hops) plus the index-coverage signal: the indexed path is preferred when its +frontier-relative work beats building the CSR (≈ when `hops × frontier` is a +small fraction of the source-vertex set), and CSR is preferred for dense/deep +traversals or when the BTREE coverage is degraded and a full scan would be paid +per hop. The two ceilings bound the **initial dispatch** frontier/hops (beyond +them CSR is always used); they are not a hard per-hop bound — the cost model +*estimates* total indexed work as ~`hops × frontier × fanout`, so dense fan-out is +priced toward CSR rather than capped mid-traversal. The override flag forces a path (the `auto` result is identical either way; +only the path differs). diff --git a/docs/user/indexes.md b/docs/user/indexes.md index ce6c728..df898c4 100644 --- a/docs/user/indexes.md +++ b/docs/user/indexes.md @@ -21,6 +21,6 @@ This is OmniGraph-specific (not Lance): - `TypeIndex`: dense `u32 ↔ String id` mapping per node type. - `CsrIndex`: Compressed Sparse Row representation of edges per edge type — `offsets[i]..offsets[i+1]` slices into `targets`. -- `GraphIndex { type_indices, csr (out), csc (in) }` — built on demand from a snapshot's edge tables. +- `GraphIndex { type_indices, csr (out), csc (in) }` — built on demand from a snapshot's edge tables, **lazily**: only when an `Expand` the planner routes to the CSR path (dense / large frontier) or an `AntiJoin` actually needs it. - Cached in `RuntimeCache::graph_indices` (LRU, max 8 entries, keyed by snapshot id + edge table versions). -- Built only when an `Expand` or `AntiJoin` IR op is present in the lowered query, so pure scans skip it. +- Selective `Expand`s resolve neighbors from the persisted `src`/`dst` BTREE instead (one indexed scan per hop) and never trigger the CSR build; see [query-language](query-language.md) → Expand. Pure scans, and queries served entirely by the indexed traversal path, skip it. diff --git a/docs/user/query-language.md b/docs/user/query-language.md index 6c7516f..acdc45d 100644 --- a/docs/user/query-language.md +++ b/docs/user/query-language.md @@ -55,6 +55,8 @@ Used inside MATCH or as expressions inside RETURN/ORDER: - `order { <expr> [asc|desc], … }` — supports plain expressions and `nearest(...)`. - `limit <integer>` — required when there is a `nearest(...)` ordering. +- **Total, deterministic order.** Rows with equal user-sort keys are broken by the bound entities' key columns (`<var>.id`, ascending) appended as a final tie-break, so the result is a *total* order — reproducible across runs, and `order … limit N` returns a deterministic top-N even when ties straddle the cutoff. (Aggregate results have no entity-key columns; their group rows are already distinct on the projected group keys.) +- **NULL placement** is *nulls-first ascending, nulls-last descending* (i.e. `nulls_first = !descending`): a NULL sorts as if smaller than any value. ## Mutation statements @@ -79,7 +81,7 @@ Reason: under the staged-write rewire (MR-794), inserts and updates accumulate i Pipeline operations: - `NodeScan { variable, type_name, filters }` -- `Expand { src_var, dst_var, edge_type, direction (Out|In), dst_type, min_hops, max_hops, dst_filters }` — destination filters are pushed *into* the expand so Lance scalar pushdown can prune. +- `Expand { src_var, dst_var, edge_type, direction (Out|In), dst_type, min_hops, max_hops, dst_filters }` — destination filters are pushed *into* the expand so Lance scalar pushdown can prune. Executed one of two ways, chosen per-expand by a cost model over cheap manifest counts (frontier size, |E|, source-vertex count, hops) plus index coverage: selective traversals (small frontier relative to the source set) resolve neighbors from the persisted `src`/`dst` BTREE (one indexed scan per hop); dense / deep / large-frontier traversals — or those whose BTREE coverage is degraded so a full scan would be paid per hop — use the in-memory CSR adjacency index. Both produce identical results. The `OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER` / `OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS` ceilings bound the *initial dispatch* frontier/hops (beyond them CSR is always used); the cost model estimates total indexed work as ~`hops × frontier × fanout` and prices dense fan-out toward CSR — they are not a hard per-hop bound. `OMNIGRAPH_TRAVERSAL_MODE=indexed|csr` forces a mode (see [constants](constants.md)). - `Filter { left, op, right }` - `AntiJoin { outer_var, inner: Vec<IROp> }` — for `not { … }` From e0d88d1295828f31ed9dd8881697ccea628052e8 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford <ragnor.comerford@gmail.com> Date: Tue, 9 Jun 2026 19:28:21 +0200 Subject: [PATCH 034/165] fix(unique): collision-free tuple key shared by intake and merge, loud on un-keyable types (#160) * fix(unique): collision-free tuple key shared by intake and merge, loud on un-keyable types Hardening on top of #133. That PR introduced a shared `loader::composite_unique_key(parts)` joining per-column scalars with U+001F and routed both intake and branch-merge through it, closing the original '|' vs U+001F separator drift. This takes the shared keying the rest of the way to correct-by-design: - Collision-free by construction: the key is now the tuple of per-column scalar strings (Vec<String>) keyed directly, no separator, so no data value (not even a literal U+001F) can forge a collision. - One scalar converter across both paths: intake used an explicit type-match, merge used Arrow's array_value_to_string. Both now derive the key through composite_unique_key(group_columns, row), so they can't drift on conversion. - Loud on un-keyable types: the scalar converter returned None for any Arrow type it didn't recognize, and the caller treated None as null-exempt, so a @unique on a column type it couldn't reduce (list, blob) was silently un-enforced. It now returns Err, surfacing the constraint it can't enforce instead of weakening it in silence. Tests: - consistency::composite_unique_key_is_consistent_across_intake_and_merge pins that intake and merge key the tuple identically (load-on-branch then merge of values containing '|'). - loader unit tests pin tuple keying + null exemption and the loud error on an un-keyable (binary) column. Docs: invariants truth-matrix updated; stale loader/mod.rs line pointers fixed. Scope unchanged: intra-batch / merge-candidate-set only; cross-version uniqueness against committed rows stays a documented gap. * fix(unique): cover all string encodings; make format_tuple private (PR #160 review) Addresses two Greptile P2 comments on PR #160: - unique_key_scalar handled only StringArray (Utf8). The loud-on-unknown-type behavior turned any legal string column that read back as LargeUtf8 or Utf8View into a hard write failure (the old code silently returned None). Add LargeStringArray and StringViewArray arms so a legal string column is keyable in every physical Arrow encoding; the Err path now fires only for a genuinely un-keyable logical type (list/blob/vector), never a legal value in an unenumerated encoding. - format_tuple was pub(crate) but only used within loader/mod.rs; make it a private fn (matches the old format_unique_columns it replaced, minimal exposed surface). New unit test unique_key_scalar_handles_all_string_encodings pins that Utf8 / LargeUtf8 / Utf8View all render rather than error. --- crates/omnigraph/src/exec/merge.rs | 38 +++-- crates/omnigraph/src/loader/mod.rs | 199 +++++++++++++++++++------- crates/omnigraph/src/table_store.rs | 2 +- crates/omnigraph/tests/consistency.rs | 67 ++++++++- docs/dev/invariants.md | 2 +- 5 files changed, 235 insertions(+), 73 deletions(-) diff --git a/crates/omnigraph/src/exec/merge.rs b/crates/omnigraph/src/exec/merge.rs index 0e6434b..1068f90 100644 --- a/crates/omnigraph/src/exec/merge.rs +++ b/crates/omnigraph/src/exec/merge.rs @@ -670,36 +670,34 @@ fn update_unique_constraints( table_key: &str, batch: &RecordBatch, constraints: &[Vec<String>], - seen: &mut [HashMap<String, String>], + seen: &mut [HashMap<Vec<String>, String>], conflicts: &mut Vec<MergeConflict>, ) -> Result<()> { for (constraint_idx, columns) in constraints.iter().enumerate() { let seen = &mut seen[constraint_idx]; - for row in 0..batch.num_rows() { - let mut parts = Vec::with_capacity(columns.len()); - let mut any_null = false; - for column_name in columns { - let column = batch.column_by_name(column_name).ok_or_else(|| { + // Resolve the group's columns once. The candidate dataset always + // carries the full table schema, so a missing column is an internal + // error rather than a skip. + let group_columns = columns + .iter() + .map(|column_name| { + batch.column_by_name(column_name).cloned().ok_or_else(|| { OmniError::manifest(format!( "table {} missing unique column '{}'", table_key, column_name )) - })?; - if column.is_null(row) { - any_null = true; - break; - } - parts.push( - array_value_to_string(column.as_ref(), row) - .map_err(|e| OmniError::Lance(e.to_string()))?, - ); - } - if any_null { + }) + }) + .collect::<Result<Vec<_>>>()?; + for row in 0..batch.num_rows() { + // Same tuple key as the intake path — one shared derivation in + // `crate::loader::composite_unique_key`, so the two cannot drift on + // separator or scalar conversion. Null rows are exempt. + let Some(key) = crate::loader::composite_unique_key(&group_columns, row)? else { continue; - } - let value = crate::loader::composite_unique_key(&parts); + }; let row_id = row_id_at(batch, row)?; - if let Some(first_row_id) = seen.insert(value.clone(), row_id.clone()) { + if let Some(first_row_id) = seen.insert(key, row_id.clone()) { conflicts.push(MergeConflict { table_key: table_key.to_string(), row_id: Some(row_id.clone()), diff --git a/crates/omnigraph/src/loader/mod.rs b/crates/omnigraph/src/loader/mod.rs index 9a80b39..707c46a 100644 --- a/crates/omnigraph/src/loader/mod.rs +++ b/crates/omnigraph/src/loader/mod.rs @@ -1445,34 +1445,32 @@ pub(crate) fn enforce_unique_constraints_intra_batch( unique_constraints: &[Vec<String>], ) -> Result<()> { for columns in unique_constraints { - let Some(col_indices) = columns + // Resolve the group's columns once. A group whose columns aren't all + // present in this batch is skipped (e.g. a partial-schema load). + let Some(group_columns) = columns .iter() - .map(|name| batch.schema().index_of(name).ok()) - .collect::<Option<Vec<usize>>>() + .map(|name| { + batch + .schema() + .index_of(name) + .ok() + .map(|i| batch.column(i).clone()) + }) + .collect::<Option<Vec<ArrayRef>>>() else { continue; }; - let mut seen: HashMap<String, usize> = HashMap::new(); + let mut seen: HashMap<Vec<String>, usize> = HashMap::new(); for row in 0..batch.num_rows() { - let mut parts = Vec::with_capacity(col_indices.len()); - let mut any_null = false; - for &col_idx in &col_indices { - let Some(value) = scalar_to_string(batch.column(col_idx), row) else { - any_null = true; - break; - }; - parts.push(value); - } - if any_null { + let Some(key) = composite_unique_key(&group_columns, row)? else { continue; - } - let value = composite_unique_key(&parts); - if let Some(prev_row) = seen.insert(value.clone(), row) { + }; + if let Some(prev_row) = seen.insert(key.clone(), row) { return Err(OmniError::manifest(format!( "@unique violation on {}.{}: value '{}' appears in rows {} and {}", type_name, - format_unique_columns(columns), - value, + format_tuple(columns), + format_tuple(&key), prev_row, row ))); @@ -1482,66 +1480,105 @@ pub(crate) fn enforce_unique_constraints_intra_batch( Ok(()) } -/// Join one row's rendered, non-null column values into a single composite -/// uniqueness key. The separator is the unit separator (U+001F) — a control -/// char highly unlikely to occur in real data, so distinct tuples like -/// `("a|b", "c")` and `("a", "b|c")` stay distinct rather than colliding. +/// Build the composite uniqueness key for `row` over a constraint group's +/// already-resolved columns (in declaration order). /// -/// Shared by the intake path (`enforce_unique_constraints_intra_batch`) and -/// the branch-merge path (`exec/merge.rs::update_unique_constraints`) so the -/// two cannot silently drift to incompatible keyings. -pub(crate) fn composite_unique_key(parts: &[String]) -> String { - parts.join("\u{1f}") +/// The key is the *tuple* of per-column scalar strings (`Vec<String>`), keyed +/// directly in the dedup map — there is no separator, so no data value can +/// forge a collision (an earlier version joined on `U+001F`, which a value +/// containing that control char could still defeat). +/// +/// - `Ok(None)` if any column is null: the row is exempt (a partial tuple +/// can't violate uniqueness under SQL null semantics). +/// - `Ok(Some(tuple))` otherwise. +/// - `Err(..)` propagated from [`unique_key_scalar`] on an un-keyable value. +/// +/// Shared by the intake path (`enforce_unique_constraints_intra_batch`) and the +/// branch-merge path (`exec/merge.rs::update_unique_constraints`) so the two +/// derive identical keys and cannot drift on separator or scalar conversion. +pub(crate) fn composite_unique_key( + group_columns: &[ArrayRef], + row: usize, +) -> Result<Option<Vec<String>>> { + let mut parts = Vec::with_capacity(group_columns.len()); + for column in group_columns { + match unique_key_scalar(column, row)? { + Some(value) => parts.push(value), + None => return Ok(None), + } + } + Ok(Some(parts)) } -/// Render a unique constraint's columns for error messages: a single column -/// as `col`, a composite as `(a, b)`. -fn format_unique_columns(columns: &[String]) -> String { - match columns { +/// Render a constraint's column tuple for error messages: a single item as +/// `col`, a composite as `(a, b)`. Used for both the column list and the +/// offending value tuple, which share the same shape. +fn format_tuple(items: &[String]) -> String { + match items { [single] => single.clone(), - _ => format!("({})", columns.join(", ")), + _ => format!("({})", items.join(", ")), } } -/// Reduce a single Arrow scalar at (`array`, `row`) to a `String` for -/// uniqueness comparison. Returns `None` for null values (nulls are exempt -/// from uniqueness in standard SQL semantics). -fn scalar_to_string(array: &ArrayRef, row: usize) -> Option<String> { - use arrow_array::Array; +/// Reduce a single Arrow scalar at (`array`, `row`) to its uniqueness-key +/// string. +/// +/// - `Ok(None)` for a null value: nulls are exempt from uniqueness (standard +/// SQL semantics over nullable columns). +/// - `Ok(Some(s))` for every scalar type a `@unique` / `@key` column can hold. +/// Strings are covered in all three physical Arrow encodings (`Utf8`, +/// `LargeUtf8`, `Utf8View`), so a legal string column is always keyable +/// regardless of how Lance materializes it on read-back. +/// - `Err(..)` for a non-null value whose Arrow type can't be reduced to a key +/// (a list, blob, or vector column). This fails loudly rather than silently +/// exempting the row, and because every legal scalar encoding is handled +/// above, the error fires only for a genuinely un-keyable column type — never +/// for a legal value that merely arrived in an unenumerated encoding. +fn unique_key_scalar(array: &ArrayRef, row: usize) -> Result<Option<String>> { + use arrow_array::{Array, LargeStringArray, StringViewArray}; if array.is_null(row) { - return None; + return Ok(None); } if let Some(a) = array.as_any().downcast_ref::<StringArray>() { - return Some(a.value(row).to_string()); + return Ok(Some(a.value(row).to_string())); + } + if let Some(a) = array.as_any().downcast_ref::<LargeStringArray>() { + return Ok(Some(a.value(row).to_string())); + } + if let Some(a) = array.as_any().downcast_ref::<StringViewArray>() { + return Ok(Some(a.value(row).to_string())); } if let Some(a) = array.as_any().downcast_ref::<Int32Array>() { - return Some(a.value(row).to_string()); + return Ok(Some(a.value(row).to_string())); } if let Some(a) = array.as_any().downcast_ref::<Int64Array>() { - return Some(a.value(row).to_string()); + return Ok(Some(a.value(row).to_string())); } if let Some(a) = array.as_any().downcast_ref::<UInt32Array>() { - return Some(a.value(row).to_string()); + return Ok(Some(a.value(row).to_string())); } if let Some(a) = array.as_any().downcast_ref::<UInt64Array>() { - return Some(a.value(row).to_string()); + return Ok(Some(a.value(row).to_string())); } if let Some(a) = array.as_any().downcast_ref::<Float32Array>() { - return Some(a.value(row).to_string()); + return Ok(Some(a.value(row).to_string())); } if let Some(a) = array.as_any().downcast_ref::<Float64Array>() { - return Some(a.value(row).to_string()); + return Ok(Some(a.value(row).to_string())); } if let Some(a) = array.as_any().downcast_ref::<BooleanArray>() { - return Some(a.value(row).to_string()); + return Ok(Some(a.value(row).to_string())); } if let Some(a) = array.as_any().downcast_ref::<Date32Array>() { - return Some(a.value(row).to_string()); + return Ok(Some(a.value(row).to_string())); } if let Some(a) = array.as_any().downcast_ref::<Date64Array>() { - return Some(a.value(row).to_string()); + return Ok(Some(a.value(row).to_string())); } - None + Err(OmniError::manifest(format!( + "uniqueness key: unsupported column type {:?} for @unique/@key enforcement", + array.data_type() + ))) } /// Build the list of uniqueness constraint groups to enforce on a node type. @@ -2209,4 +2246,66 @@ edge WorksAt: Person -> Company let err = result.unwrap_err().to_string(); assert!(err.contains("NaN"), "error should mention NaN: {}", err); } + + #[test] + fn composite_unique_key_builds_tuple_and_exempts_null() { + let a: ArrayRef = Arc::new(StringArray::from(vec![Some("x|y"), Some("x"), None])); + let b: ArrayRef = Arc::new(StringArray::from(vec![Some("z"), Some("y|z"), Some("q")])); + let cols = [a, b]; + + // Tuple key, so `("x|y", "z")` and `("x", "y|z")` stay distinct — + // a separator-joined key (the old `|` join) would collapse both to + // `x|y|z`. + assert_eq!( + composite_unique_key(&cols, 0).unwrap(), + Some(vec!["x|y".to_string(), "z".to_string()]) + ); + assert_eq!( + composite_unique_key(&cols, 1).unwrap(), + Some(vec!["x".to_string(), "y|z".to_string()]) + ); + assert_ne!( + composite_unique_key(&cols, 0).unwrap(), + composite_unique_key(&cols, 1).unwrap() + ); + + // Any null column → the whole row is exempt (SQL null semantics). + assert_eq!(composite_unique_key(&cols, 2).unwrap(), None); + } + + #[test] + fn unique_key_scalar_errors_loudly_on_unkeyable_type() { + use arrow_array::LargeBinaryArray; + // A binary/blob column can't be reduced to a uniqueness key. Before the + // hardening this returned `None`, so a `@unique` on such a column was + // silently un-enforced; now it errors instead of weakening the + // constraint in silence. + let blob: ArrayRef = Arc::new(LargeBinaryArray::from(vec![Some(&b"abc"[..])])); + let err = unique_key_scalar(&blob, 0).unwrap_err(); + assert!( + err.to_string().contains("unsupported column type"), + "un-keyable type must fail loudly (got: {err})" + ); + } + + #[test] + fn unique_key_scalar_handles_all_string_encodings() { + use arrow_array::{LargeStringArray, StringViewArray}; + // A legal string column is keyable in every physical Arrow encoding + // Lance might hand back (Utf8 / LargeUtf8 / Utf8View). None of these may + // fall through to the loud `Err` path — that branch is reserved for + // genuinely un-keyable column types, not a legal value in an + // unenumerated encoding. + let utf8: ArrayRef = Arc::new(StringArray::from(vec![Some("v")])); + let large: ArrayRef = Arc::new(LargeStringArray::from(vec![Some("v")])); + let view: ArrayRef = Arc::new(StringViewArray::from(vec![Some("v")])); + for array in [&utf8, &large, &view] { + assert_eq!( + unique_key_scalar(array, 0).unwrap(), + Some("v".to_string()), + "string array {:?} must render, not error", + array.data_type() + ); + } + } } diff --git a/crates/omnigraph/src/table_store.rs b/crates/omnigraph/src/table_store.rs index bdf0dd5..d786fc4 100644 --- a/crates/omnigraph/src/table_store.rs +++ b/crates/omnigraph/src/table_store.rs @@ -856,7 +856,7 @@ impl TableStore { // before the FirstSeen setter has a chance to silently collapse // anything): // - Load path: `enforce_unique_constraints_intra_batch` - // (`loader/mod.rs:1471`) errors on intra-batch `@key` dups. + // (`loader/mod.rs:1442`) errors on intra-batch `@key` dups. // - Mutate path: `MutationStaging::finalize` (`exec/staging.rs`) // accumulates and dedupes by `id`. // - Branch-merge path: `compute_source_delta` / diff --git a/crates/omnigraph/tests/consistency.rs b/crates/omnigraph/tests/consistency.rs index 729f2e8..b16aff9 100644 --- a/crates/omnigraph/tests/consistency.rs +++ b/crates/omnigraph/tests/consistency.rs @@ -188,7 +188,7 @@ node Thing { /// /// Defense in depth: /// 1. The loader's `enforce_unique_constraints_intra_batch` -/// (`loader/mod.rs:1471`), invoked unconditionally on any node type +/// (`loader/mod.rs:1442`), invoked unconditionally on any node type /// with a `@key`, errors on intra-batch duplicate `@key` values at /// intake — pinned by this test across every `LoadMode`. /// 2. The `check_batch_unique_by_keys` precondition at the top of @@ -280,6 +280,71 @@ node ExternalID { ); } +/// Guard: the intake path (load/insert/update) and the branch-merge path must +/// derive the same composite `@unique(a, b)` key, so a pair of rows unique on +/// the tuple is accepted by BOTH. Both paths now key on the tuple itself (no +/// separator), so a value containing any byte — including the `|` that an +/// earlier merge-path join used as its separator — can't forge a collision. +/// `("x|y", "z")` and `("x", "y|z")` are distinct tuples and must survive a +/// load-on-branch then merge without a phantom `UniqueViolation`. This pins the +/// cross-path consistency against any future drift in the shared keying. +#[tokio::test] +async fn composite_unique_key_is_consistent_across_intake_and_merge() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let schema = r#" +node Item { + slug: String @key + a: String @index + b: String @index + @unique(a, b) +} +"#; + let insert_item = r#" +query insert_item($slug: String, $a: String, $b: String) { + insert Item { slug: $slug, a: $a, b: $b } +} +"#; + let main = Omnigraph::init(uri, schema).await.unwrap(); + main.branch_create("feature").await.unwrap(); + + // Two rows unique on the composite (a, b), where `a`/`b` carry a literal + // `|`. Distinct under a tuple key; identical (`x|y|z`) under a `|`-join. + let feature = Omnigraph::open(uri).await.unwrap(); + feature + .mutate( + "feature", + insert_item, + "insert_item", + ¶ms(&[("$slug", "r1"), ("$a", "x|y"), ("$b", "z")]), + ) + .await + .expect("intake must accept the first composite-unique row"); + feature + .mutate( + "feature", + insert_item, + "insert_item", + ¶ms(&[("$slug", "r2"), ("$a", "x"), ("$b", "y|z")]), + ) + .await + .expect("intake must accept the second composite-unique row (distinct on the tuple)"); + + // The merge re-validates uniqueness over the adopted source rows. Both + // rows are unique on (a, b), so this must merge cleanly with no phantom + // conflict — intake and merge must key the tuple identically. + let merge_result = feature.branch_merge("feature", "main").await; + assert!( + merge_result.is_ok(), + "rows unique on the composite (a, b) must merge cleanly; \ + intake and merge must key the tuple the same way (got: {:?})", + merge_result.err() + ); + + let reopened = Omnigraph::open(uri).await.unwrap(); + assert_eq!(count_rows(&reopened, "node:Item").await, 2); +} + /// Canary for the upstream Lance gap that the `FirstSeen` workaround /// in `table_store.rs` masks. The bug class is "Window 2": load → /// indices built explicitly → merge → merge. Even with the engine diff --git a/docs/dev/invariants.md b/docs/dev/invariants.md index b29d740..4baff5e 100644 --- a/docs/dev/invariants.md +++ b/docs/dev/invariants.md @@ -101,7 +101,7 @@ Use it this way: | Deletes | Inline-commit residual; delete-only queries allowed, mixed insert/update/delete rejected by D2 | [query-language.md](../user/query-language.md), [writes.md](writes.md) | | Branch delete | Manifest is the single authority, flipped atomically first; per-table forks + commit-graph branch are derived state, reclaimed best-effort (`force_delete_branch`) with the `cleanup` reconciler as the guaranteed backstop. Reusing a name whose reclaim failed before `cleanup` surfaces an actionable error | [branches-commits.md](../user/branches-commits.md), [maintenance.md](../user/maintenance.md) | | Schema validation | Type checks, required fields, defaults, edge endpoint checks, and edge cardinality are enforced on write paths | [schema-language.md](../user/schema-language.md), [execution.md](execution.md) | -| Unique constraints | Intra-batch and write-path checks exist; full cross-version uniqueness is still a gap | [schema-language.md](../user/schema-language.md) | +| Unique constraints | Intra-batch and write-path checks exist; intake and branch-merge derive the composite key through one shared function (`loader::composite_unique_key`, a separator-free `Vec<String>` tuple) and fail loudly on an un-keyable column type rather than silently exempting it; full cross-version uniqueness against already-committed rows is still a gap | [schema-language.md](../user/schema-language.md) | | Storage trait | `TableStorage` exists as the sealed staged-write surface; full call-site migration and capability/stat surfaces are incomplete | [writes.md](writes.md), [architecture.md](architecture.md) | | Index lifecycle | `ensure_indices` is explicit today; reconciler-based convergence is roadmap | [indexes.md](../user/indexes.md), [maintenance.md](../user/maintenance.md) | | Traversal IDs | Runtime still builds `TypeIndex`; Lance stable row-id based graph IDs are roadmap | [architecture.md](architecture.md), [query-language.md](../user/query-language.md) | From d00d42274e9e4408df9b4b80c98467da7ae6c0ab Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Mon, 8 Jun 2026 23:18:44 +0300 Subject: [PATCH 035/165] Implement cluster refresh and import --- Cargo.lock | 2 + crates/omnigraph-cli/src/main.rs | 71 ++- crates/omnigraph-cli/tests/cli.rs | 212 +++++++ crates/omnigraph-cluster/Cargo.toml | 2 + crates/omnigraph-cluster/src/lib.rs | 891 +++++++++++++++++++++++++++- docs/dev/cluster-config-specs.md | 7 + docs/dev/testing.md | 2 +- docs/user/cli-reference.md | 20 +- docs/user/cluster-config.md | 48 +- 9 files changed, 1225 insertions(+), 30 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 578188c..79760b0 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4569,6 +4569,7 @@ name = "omnigraph-cluster" version = "0.6.2" dependencies = [ "omnigraph-compiler", + "omnigraph-engine", "serde", "serde_json", "serde_yaml", @@ -4576,6 +4577,7 @@ dependencies = [ "tempfile", "thiserror", "time", + "tokio", "ulid", ] diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 38ea0de..9c16722 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -11,8 +11,8 @@ use omnigraph::db::{Omnigraph, ReadTarget, SnapshotId}; use omnigraph::loader::LoadMode; use omnigraph::storage::normalize_root_uri; use omnigraph_cluster::{ - DiagnosticSeverity, PlanOutput, StatusOutput, ValidateOutput, plan_config_dir, - status_config_dir, validate_config_dir, + DiagnosticSeverity, PlanOutput, StateSyncOutput, StatusOutput, ValidateOutput, + import_config_dir, plan_config_dir, refresh_config_dir, status_config_dir, validate_config_dir, }; use omnigraph_compiler::query::parser::parse_query; use omnigraph_compiler::schema::parser::parse_schema; @@ -369,6 +369,24 @@ enum ClusterCommand { #[arg(long)] json: bool, }, + /// Refresh existing local JSON state from declared graph observations. + Refresh { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, + /// Import initial local JSON state from declared graph observations. + Import { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, } /// Operations on the graph registry of a multi-graph server (MR-668). @@ -802,6 +820,34 @@ fn print_cluster_status_human(output: &StatusOutput) { print_cluster_diagnostics(&output.diagnostics); } +fn print_cluster_state_sync_human(output: &StateSyncOutput) { + let operation = match output.operation { + omnigraph_cluster::StateSyncOperation::Refresh => "refresh", + omnigraph_cluster::StateSyncOperation::Import => "import", + }; + if output.ok { + let state = &output.state_observations; + println!( + "cluster {operation}: revision {}, {} resource(s)", + state.state_revision, state.resource_count + ); + if let Some(cas) = state.state_cas.as_deref() { + println!(" state_cas: {cas}"); + } + if state.locked { + match state.lock_id.as_deref() { + Some(lock_id) => println!(" lock: acquired ({lock_id})"), + None => println!(" lock: acquired"), + } + } else { + println!(" lock: not acquired"); + } + } else { + println!("cluster {operation} failed"); + } + print_cluster_diagnostics(&output.diagnostics); +} + fn print_cluster_diagnostics(diagnostics: &[omnigraph_cluster::Diagnostic]) { for diagnostic in diagnostics { let label = match diagnostic.severity { @@ -854,6 +900,19 @@ fn finish_cluster_status(output: &StatusOutput, json: bool) -> Result<()> { Ok(()) } +fn finish_cluster_state_sync(output: &StateSyncOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_state_sync_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + fn is_remote_uri(uri: &str) -> bool { uri.starts_with("http://") || uri.starts_with("https://") } @@ -3376,6 +3435,14 @@ async fn main() -> Result<()> { let output = status_config_dir(config); finish_cluster_status(&output, json)?; } + ClusterCommand::Refresh { config, json } => { + let output = refresh_config_dir(config).await; + finish_cluster_state_sync(&output, json)?; + } + ClusterCommand::Import { config, json } => { + let output = import_config_dir(config).await; + finish_cluster_state_sync(&output, json)?; + } }, Command::Graphs { command } => match command { GraphsCommand::List { diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 17b1f72..504f0ef 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -144,6 +144,18 @@ policies: .unwrap(); } +fn init_cluster_derived_graph(root: &std::path::Path) { + let graph_dir = root.join("graphs"); + fs::create_dir_all(&graph_dir).unwrap(); + output_success( + cli() + .arg("init") + .arg("--schema") + .arg(root.join("people.pg")) + .arg(graph_dir.join("knowledge.omni")), + ); +} + #[test] fn version_command_prints_current_cli_version() { let output = output_success(cli().arg("version")); @@ -399,6 +411,206 @@ fn cluster_plan_locked_state_exits_nonzero() { ); } +#[test] +fn cluster_import_json_bootstraps_missing_state() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("import") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["operation"], "import"); + assert_eq!(json["state_observations"]["state_revision"], 1); + assert!( + json["state_observations"]["state_cas"] + .as_str() + .unwrap() + .starts_with("sha256:") + ); + assert_eq!(json["state_observations"]["locked"], false); + assert_eq!(json["state_observations"]["lock_acquired"], true); + assert!(json["state_observations"]["acquired_lock_id"].is_string()); + assert!(json["observations"]["graph.knowledge"]["manifest_version"].is_number()); + assert_eq!( + json["resource_statuses"]["graph.knowledge"]["status"], + "applied" + ); + assert!(temp.path().join("__cluster/state.json").exists()); + assert!(!temp.path().join("__cluster/lock.json").exists()); +} + +#[test] +fn cluster_refresh_json_updates_revision_cas_and_removes_lock() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#" +{ + "version": 1, + "state_revision": 2, + "applied_revision": { "resources": {} } +} +"#, + ) + .unwrap(); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("refresh") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["operation"], "refresh"); + assert_eq!(json["state_observations"]["state_revision"], 3); + assert!( + json["state_observations"]["state_cas"] + .as_str() + .unwrap() + .starts_with("sha256:") + ); + assert_eq!(json["state_observations"]["locked"], false); + assert_eq!(json["state_observations"]["lock_acquired"], true); + assert!(json["state_observations"]["acquired_lock_id"].is_string()); + assert!(!state_dir.join("lock.json").exists()); +} + +#[test] +fn cluster_refresh_missing_state_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let output = output_failure( + cli() + .arg("cluster") + .arg("refresh") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + ); + let json = parse_stdout_json(&output); + assert_eq!(json["ok"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_missing"), + "missing state should produce a useful diagnostic: {json}" + ); +} + +#[test] +fn cluster_import_existing_state_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + + let output = output_failure( + cli() + .arg("cluster") + .arg("import") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + ); + let json = parse_stdout_json(&output); + assert_eq!(json["ok"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_already_exists"), + "existing state should produce a useful diagnostic: {json}" + ); +} + +#[test] +fn cluster_refresh_and_import_locked_state_exit_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + fs::write( + state_dir.join("lock.json"), + r#"{"version":1,"lock_id":"held-lock","operation":"refresh","created_at":"2026-06-08T00:00:00Z","pid":123}"#, + ) + .unwrap(); + + let refresh = parse_stdout_json(&output_failure( + cli() + .arg("cluster") + .arg("refresh") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(refresh["state_observations"]["locked"], true); + assert_eq!(refresh["state_observations"]["lock_id"], "held-lock"); + assert_eq!(refresh["state_observations"]["lock_acquired"], false); + assert!( + refresh["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_lock_held") + ); + + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("lock.json"), + r#"{"version":1,"lock_id":"held-lock","operation":"import","created_at":"2026-06-08T00:00:00Z","pid":123}"#, + ) + .unwrap(); + + let imported = parse_stdout_json(&output_failure( + cli() + .arg("cluster") + .arg("import") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(imported["state_observations"]["locked"], true); + assert_eq!(imported["state_observations"]["lock_id"], "held-lock"); + assert_eq!(imported["state_observations"]["lock_acquired"], false); + assert!( + imported["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_lock_held") + ); +} + #[test] fn cluster_validate_invalid_config_exits_nonzero() { let temp = tempdir().unwrap(); diff --git a/crates/omnigraph-cluster/Cargo.toml b/crates/omnigraph-cluster/Cargo.toml index 3e14430..9280c42 100644 --- a/crates/omnigraph-cluster/Cargo.toml +++ b/crates/omnigraph-cluster/Cargo.toml @@ -10,6 +10,7 @@ documentation = "https://docs.rs/omnigraph-cluster" [dependencies] omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } +omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.2" } serde = { workspace = true } serde_json = { workspace = true } serde_yaml = { workspace = true } @@ -20,3 +21,4 @@ ulid = { workspace = true } [dev-dependencies] tempfile = { workspace = true } +tokio = { workspace = true } diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index e308392..9a6ea78 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -4,17 +4,20 @@ use std::io::{ErrorKind, Write}; use std::path::{Path, PathBuf}; use std::process; +use omnigraph::db::{Omnigraph, ReadTarget}; use omnigraph_compiler::build_catalog; use omnigraph_compiler::query::parser::parse_query; use omnigraph_compiler::query::typecheck::typecheck_query_decl; use omnigraph_compiler::schema::parser::parse_schema; use serde::{Deserialize, Serialize}; +use serde_json::json; use sha2::{Digest, Sha256}; use time::OffsetDateTime; use time::format_description::well_known::Rfc3339; use ulid::Ulid; pub const CLUSTER_CONFIG_FILE: &str = "cluster.yaml"; +pub const CLUSTER_GRAPHS_DIR: &str = "graphs"; pub const CLUSTER_STATE_DIR: &str = "__cluster"; pub const CLUSTER_STATE_FILE: &str = "__cluster/state.json"; pub const CLUSTER_LOCK_FILE: &str = "__cluster/lock.json"; @@ -182,6 +185,26 @@ pub struct StatusOutput { pub state_observations: StateObservations, pub resource_digests: BTreeMap<String, String>, pub resource_statuses: BTreeMap<String, ResourceStatusRecord>, + pub observations: BTreeMap<String, serde_json::Value>, + pub diagnostics: Vec<Diagnostic>, +} + +#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum StateSyncOperation { + Refresh, + Import, +} + +#[derive(Debug, Clone, Serialize)] +pub struct StateSyncOutput { + pub ok: bool, + pub operation: StateSyncOperation, + pub config_dir: String, + pub state_observations: StateObservations, + pub resource_digests: BTreeMap<String, String>, + pub resource_statuses: BTreeMap<String, ResourceStatusRecord>, + pub observations: BTreeMap<String, serde_json::Value>, pub diagnostics: Vec<Diagnostic>, } @@ -190,11 +213,18 @@ struct DesiredCluster { config_dir: PathBuf, config_digest: String, state_lock: bool, + graphs: Vec<DesiredGraph>, resource_digests: BTreeMap<String, String>, resources: Vec<ResourceSummary>, dependencies: Vec<Dependency>, } +#[derive(Debug, Clone)] +struct DesiredGraph { + id: String, + schema_digest: String, +} + #[derive(Debug)] struct ParsedConfig { raw: Option<RawClusterConfig>, @@ -264,8 +294,10 @@ struct PolicyConfig { applies_to: Vec<String>, } +// Stage 2A/2B accept these forward-compatible state sections so existing +// ledgers won't churn while approval/recovery semantics are staged later. #[allow(dead_code)] -#[derive(Debug, Deserialize)] +#[derive(Debug, Clone, Serialize, Deserialize)] #[serde(deny_unknown_fields)] struct ClusterState { version: u32, @@ -282,7 +314,7 @@ struct ClusterState { observations: BTreeMap<String, serde_json::Value>, } -#[derive(Debug, Deserialize)] +#[derive(Debug, Clone, Serialize, Deserialize)] #[serde(deny_unknown_fields)] struct AppliedRevisionState { #[serde(default)] @@ -291,7 +323,7 @@ struct AppliedRevisionState { resources: BTreeMap<String, StateResource>, } -#[derive(Debug, Deserialize)] +#[derive(Debug, Clone, Serialize, Deserialize)] #[serde(deny_unknown_fields)] struct StateResource { digest: String, @@ -317,6 +349,7 @@ struct LocalStateBackend { #[derive(Debug)] struct StateSnapshot { state: Option<ClusterState>, + state_cas: Option<String>, } #[derive(Debug)] @@ -450,6 +483,7 @@ pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { let mut resource_digests = BTreeMap::new(); let mut resource_statuses = BTreeMap::new(); + let mut state_observation_records = BTreeMap::new(); if let Some(raw) = parsed.raw.as_ref() { let _settings = validate_cluster_header(raw, &mut diagnostics); @@ -459,6 +493,7 @@ pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { if let Some(state) = snapshot.state { resource_digests = state_resource_digests(&state); resource_statuses = state.resource_statuses; + state_observation_records = state.observations; } else { diagnostics.push(Diagnostic::warning( "state_missing", @@ -478,6 +513,185 @@ pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { state_observations: observations, resource_digests, resource_statuses, + observations: state_observation_records, + diagnostics, + } +} + +pub async fn refresh_config_dir(config_dir: impl AsRef<Path>) -> StateSyncOutput { + sync_config_dir(config_dir.as_ref(), StateSyncOperation::Refresh).await +} + +pub async fn import_config_dir(config_dir: impl AsRef<Path>) -> StateSyncOutput { + sync_config_dir(config_dir.as_ref(), StateSyncOperation::Import).await +} + +async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> StateSyncOutput { + let outcome = load_desired(config_dir); + let mut diagnostics = outcome.diagnostics; + let backend = LocalStateBackend::new(&outcome.config_dir); + let mut observations = backend.observations(); + + let Some(desired) = outcome.desired else { + return StateSyncOutput { + ok: false, + operation, + config_dir: display_path(&outcome.config_dir), + state_observations: observations, + resource_digests: BTreeMap::new(), + resource_statuses: BTreeMap::new(), + observations: BTreeMap::new(), + diagnostics, + }; + }; + + if has_errors(&diagnostics) { + return StateSyncOutput { + ok: false, + operation, + config_dir: display_path(&desired.config_dir), + state_observations: observations, + resource_digests: desired.resource_digests, + resource_statuses: BTreeMap::new(), + observations: BTreeMap::new(), + diagnostics, + }; + } + + let operation_label = state_sync_operation_label(operation); + let _lock_guard = if desired.state_lock { + match backend.acquire_lock(operation_label, &mut observations) { + Ok(guard) => Some(guard), + Err(diagnostic) => { + diagnostics.push(diagnostic); + None + } + } + } else { + diagnostics.push(Diagnostic::warning( + "state_lock_disabled", + "state.lock", + format!( + "state.lock is false; {operation_label} wrote state without acquiring the cluster state lock" + ), + )); + None + }; + + if has_errors(&diagnostics) { + return StateSyncOutput { + ok: false, + operation, + config_dir: display_path(&desired.config_dir), + state_observations: observations, + resource_digests: desired.resource_digests, + resource_statuses: BTreeMap::new(), + observations: BTreeMap::new(), + diagnostics, + }; + } + + let snapshot = match backend.read_state(&mut observations) { + Ok(snapshot) => snapshot, + Err(diagnostic) => { + diagnostics.push(diagnostic); + return StateSyncOutput { + ok: false, + operation, + config_dir: display_path(&desired.config_dir), + state_observations: observations, + resource_digests: desired.resource_digests, + resource_statuses: BTreeMap::new(), + observations: BTreeMap::new(), + diagnostics, + }; + } + }; + + let expected_cas = snapshot.state_cas; + let mut state = match (operation, snapshot.state) { + (StateSyncOperation::Refresh, Some(state)) => state, + (StateSyncOperation::Refresh, None) => { + diagnostics.push(Diagnostic::error( + "state_missing", + CLUSTER_STATE_FILE, + "refresh requires an existing state.json; run `cluster import` to bootstrap state", + )); + return StateSyncOutput { + ok: false, + operation, + config_dir: display_path(&desired.config_dir), + state_observations: observations, + resource_digests: BTreeMap::new(), + resource_statuses: BTreeMap::new(), + observations: BTreeMap::new(), + diagnostics, + }; + } + (StateSyncOperation::Import, Some(state)) => { + diagnostics.push(Diagnostic::error( + "state_already_exists", + CLUSTER_STATE_FILE, + "import creates initial state only when state.json is missing; use `cluster refresh` for an existing state ledger", + )); + return StateSyncOutput { + ok: false, + operation, + config_dir: display_path(&desired.config_dir), + state_observations: observations, + resource_digests: state_resource_digests(&state), + resource_statuses: state.resource_statuses, + observations: state.observations, + diagnostics, + }; + } + (StateSyncOperation::Import, None) => initial_import_state(&desired), + }; + + let graph_error_count = observe_declared_graphs(&desired, &mut state).await; + if graph_error_count > 0 { + diagnostics.push(Diagnostic::error( + "graph_observation_error", + CLUSTER_GRAPHS_DIR, + format!("{graph_error_count} graph observation(s) failed"), + )); + } + + if operation == StateSyncOperation::Import && has_errors(&diagnostics) { + return StateSyncOutput { + ok: false, + operation, + config_dir: display_path(&desired.config_dir), + state_observations: observations, + resource_digests: state_resource_digests(&state), + resource_statuses: state.resource_statuses, + observations: state.observations, + diagnostics, + }; + } + + if operation == StateSyncOperation::Import { + state.state_revision = 1; + } else { + state.state_revision = state.state_revision.saturating_add(1); + } + + match backend.write_state(&state, expected_cas.as_deref(), &mut observations) { + Ok(()) => {} + Err(diagnostic) => diagnostics.push(diagnostic), + } + + let resource_digests = state_resource_digests(&state); + let ok = !has_errors(&diagnostics); + + StateSyncOutput { + ok, + operation, + config_dir: display_path(&desired.config_dir), + state_observations: observations, + resource_digests, + resource_statuses: state.resource_statuses, + observations: state.observations, diagnostics, } } @@ -577,7 +791,7 @@ fn validate_cluster_header( diagnostics.push(Diagnostic::error( "unsupported_state_backend", "state.backend", - "Stage 2A supports only omitted state.backend or `cluster`", + "Stage 2B supports only omitted state.backend or `cluster`", )); } } @@ -620,7 +834,10 @@ impl LocalStateBackend { let text = match fs::read_to_string(&self.state_path) { Ok(text) => text, Err(err) if err.kind() == ErrorKind::NotFound => { - return Ok(StateSnapshot { state: None }); + return Ok(StateSnapshot { + state: None, + state_cas: None, + }); } Err(err) => { return Err(Diagnostic::error( @@ -632,7 +849,8 @@ impl LocalStateBackend { }; observations.state_found = true; - observations.state_cas = Some(format!("sha256:{}", sha256_hex(text.as_bytes()))); + let state_cas = format!("sha256:{}", sha256_hex(text.as_bytes())); + observations.state_cas = Some(state_cas.clone()); let state = serde_json::from_str::<ClusterState>(&text).map_err(|err| { Diagnostic::error( @@ -657,7 +875,109 @@ impl LocalStateBackend { observations.state_revision = state.state_revision; observations.resource_count = state.applied_revision.resources.len(); - Ok(StateSnapshot { state: Some(state) }) + Ok(StateSnapshot { + state: Some(state), + state_cas: Some(state_cas), + }) + } + + fn write_state( + &self, + state: &ClusterState, + expected_cas: Option<&str>, + observations: &mut StateObservations, + ) -> Result<(), Diagnostic> { + fs::create_dir_all(&self.state_dir).map_err(|err| { + Diagnostic::error( + "state_write_error", + CLUSTER_STATE_DIR, + format!("could not create cluster state directory: {err}"), + ) + })?; + + let current_cas = self.current_state_cas()?; + if current_cas.as_deref() != expected_cas { + return Err(Diagnostic::error( + "state_cas_mismatch", + CLUSTER_STATE_FILE, + "state.json changed while the command was running; re-run the command against the latest state", + )); + } + + let mut payload = serde_json::to_string_pretty(state).map_err(|err| { + Diagnostic::error( + "state_write_error", + CLUSTER_STATE_FILE, + format!("could not encode state JSON: {err}"), + ) + })?; + payload.push('\n'); + + let tmp_path = self + .state_dir + .join(format!("state.json.tmp.{}", Ulid::new())); + let mut file = OpenOptions::new() + .write(true) + .create_new(true) + .open(&tmp_path) + .map_err(|err| { + Diagnostic::error( + "state_write_error", + display_path(&tmp_path), + format!("could not create temporary state file: {err}"), + ) + })?; + file.write_all(payload.as_bytes()).map_err(|err| { + Diagnostic::error( + "state_write_error", + display_path(&tmp_path), + format!("could not write temporary state file: {err}"), + ) + })?; + file.sync_all().map_err(|err| { + Diagnostic::error( + "state_write_error", + display_path(&tmp_path), + format!("could not sync temporary state file: {err}"), + ) + })?; + drop(file); + + if let Err(err) = fs::rename(&tmp_path, &self.state_path) { + let _ = fs::remove_file(&tmp_path); + return Err(Diagnostic::error( + "state_write_error", + CLUSTER_STATE_FILE, + format!("could not replace state.json atomically: {err}"), + )); + } + + let written = fs::read_to_string(&self.state_path).map_err(|err| { + Diagnostic::error( + "state_write_error", + CLUSTER_STATE_FILE, + format!("could not read state.json after write: {err}"), + ) + })?; + observations.state_found = true; + observations.applied_config_digest = state.applied_revision.config_digest.clone(); + observations.state_revision = state.state_revision; + observations.state_cas = Some(format!("sha256:{}", sha256_hex(written.as_bytes()))); + observations.resource_count = state.applied_revision.resources.len(); + + Ok(()) + } + + fn current_state_cas(&self) -> Result<Option<String>, Diagnostic> { + match fs::read(&self.state_path) { + Ok(bytes) => Ok(Some(format!("sha256:{}", sha256_hex(&bytes)))), + Err(err) if err.kind() == ErrorKind::NotFound => Ok(None), + Err(err) => Err(Diagnostic::error( + "state_read_error", + CLUSTER_STATE_FILE, + format!("could not read state file for CAS check: {err}"), + )), + } } fn acquire_lock( @@ -789,6 +1109,247 @@ fn state_resource_digests(state: &ClusterState) -> BTreeMap<String, String> { .collect() } +fn initial_import_state(desired: &DesiredCluster) -> ClusterState { + ClusterState { + version: 1, + state_revision: 0, + applied_revision: AppliedRevisionState { + config_digest: Some(desired.config_digest.clone()), + resources: BTreeMap::new(), + }, + resource_statuses: BTreeMap::new(), + approval_records: BTreeMap::new(), + recovery_records: BTreeMap::new(), + observations: BTreeMap::new(), + } +} + +async fn observe_declared_graphs(desired: &DesiredCluster, state: &mut ClusterState) -> usize { + let mut graph_error_count = 0; + for graph in &desired.graphs { + let graph_address = graph_address(&graph.id); + let schema_address = schema_address(&graph.id); + let graph_path = desired + .config_dir + .join(CLUSTER_GRAPHS_DIR) + .join(format!("{}.omni", graph.id)); + let graph_uri = display_path(&graph_path); + let observed_at = now_rfc3339(); + + if !graph_path.exists() { + state.applied_revision.resources.remove(&graph_address); + state.applied_revision.resources.remove(&schema_address); + state.observations.insert( + graph_address.clone(), + graph_observation_json(GraphObservationJson { + address: &graph_address, + graph_uri: &graph_uri, + observed_at: &observed_at, + exists: false, + manifest_version: None, + schema_digest: None, + desired_schema_digest: &graph.schema_digest, + schema_matches_desired: Some(false), + error: Some("derived graph root is missing"), + }), + ); + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Drifted, + "graph_missing", + "derived graph root is missing", + ); + set_resource_status( + state, + &schema_address, + ResourceLifecycleStatus::Drifted, + "graph_missing", + "derived graph root is missing", + ); + continue; + } + + match observe_live_graph(&graph_uri).await { + Ok(observation) => { + let schema_matches = observation.schema_digest == graph.schema_digest; + state.applied_revision.resources.insert( + schema_address.clone(), + StateResource { + digest: observation.schema_digest.clone(), + }, + ); + let query_digests = state_query_digests_for_graph(state, &graph.id); + let graph_digest_value = graph_digest( + &graph.id, + Some(&observation.schema_digest), + Some(&query_digests), + ); + state.applied_revision.resources.insert( + graph_address.clone(), + StateResource { + digest: graph_digest_value, + }, + ); + state.observations.insert( + graph_address.clone(), + graph_observation_json(GraphObservationJson { + address: &graph_address, + graph_uri: &graph_uri, + observed_at: &observed_at, + exists: true, + manifest_version: Some(observation.manifest_version), + schema_digest: Some(observation.schema_digest.as_str()), + desired_schema_digest: &graph.schema_digest, + schema_matches_desired: Some(schema_matches), + error: None, + }), + ); + if schema_matches { + set_resource_status_applied(state, &graph_address); + set_resource_status_applied(state, &schema_address); + } else { + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Drifted, + "schema_mismatch", + "live schema digest differs from desired schema digest", + ); + set_resource_status( + state, + &schema_address, + ResourceLifecycleStatus::Drifted, + "schema_mismatch", + "live schema digest differs from desired schema digest", + ); + } + } + Err(error) => { + graph_error_count += 1; + state.observations.insert( + graph_address.clone(), + graph_observation_json(GraphObservationJson { + address: &graph_address, + graph_uri: &graph_uri, + observed_at: &observed_at, + exists: true, + manifest_version: None, + schema_digest: None, + desired_schema_digest: &graph.schema_digest, + schema_matches_desired: None, + error: Some(error.as_str()), + }), + ); + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Error, + "graph_observation_error", + error.as_str(), + ); + set_resource_status( + state, + &schema_address, + ResourceLifecycleStatus::Error, + "graph_observation_error", + error.as_str(), + ); + } + } + } + graph_error_count +} + +struct LiveGraphObservation { + manifest_version: u64, + schema_digest: String, +} + +async fn observe_live_graph(graph_uri: &str) -> Result<LiveGraphObservation, String> { + let db = Omnigraph::open_read_only(graph_uri) + .await + .map_err(|err| err.to_string())?; + let snapshot = db + .snapshot_of(ReadTarget::branch("main")) + .await + .map_err(|err| err.to_string())?; + let schema_source = db.schema_source(); + Ok(LiveGraphObservation { + manifest_version: snapshot.version(), + schema_digest: sha256_hex(schema_source.as_bytes()), + }) +} + +struct GraphObservationJson<'a> { + address: &'a str, + graph_uri: &'a str, + observed_at: &'a str, + exists: bool, + manifest_version: Option<u64>, + schema_digest: Option<&'a str>, + desired_schema_digest: &'a str, + schema_matches_desired: Option<bool>, + error: Option<&'a str>, +} + +fn graph_observation_json(observation: GraphObservationJson<'_>) -> serde_json::Value { + json!({ + "kind": "graph", + "address": observation.address, + "graph_uri": observation.graph_uri, + "observed_at": observation.observed_at, + "exists": observation.exists, + "manifest_version": observation.manifest_version, + "schema_digest": observation.schema_digest, + "desired_schema_digest": observation.desired_schema_digest, + "schema_matches_desired": observation.schema_matches_desired, + "error": observation.error, + }) +} + +fn state_query_digests_for_graph(state: &ClusterState, graph_id: &str) -> BTreeMap<String, String> { + let prefix = format!("query.{graph_id}."); + state + .applied_revision + .resources + .iter() + .filter_map(|(address, resource)| { + address + .strip_prefix(&prefix) + .map(|name| (name.to_string(), resource.digest.clone())) + }) + .collect() +} + +fn set_resource_status_applied(state: &mut ClusterState, address: &str) { + state.resource_statuses.insert( + address.to_string(), + ResourceStatusRecord { + status: ResourceLifecycleStatus::Applied, + conditions: Vec::new(), + message: None, + }, + ); +} + +fn set_resource_status( + state: &mut ClusterState, + address: &str, + status: ResourceLifecycleStatus, + condition: &str, + message: &str, +) { + state.resource_statuses.insert( + address.to_string(), + ResourceStatusRecord { + status, + conditions: vec![condition.to_string()], + message: Some(message.to_string()), + }, + ); +} + fn load_desired(config_dir: &Path) -> LoadOutcome { let parsed = parse_cluster_config(config_dir); let config_dir = parsed.config_dir; @@ -1019,6 +1580,17 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { resource_list.push(resource); } let dependencies: Vec<_> = dependencies.into_iter().collect(); + let graphs = raw + .graphs + .keys() + .map(|graph_id| DesiredGraph { + id: graph_id.clone(), + schema_digest: graph_schema_digests + .get(graph_id) + .cloned() + .unwrap_or_default(), + }) + .collect(); let config_digest = desired_config_digest(&raw, &resource_digests); LoadOutcome { @@ -1026,6 +1598,7 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { config_dir: config_dir.clone(), config_digest, state_lock: settings.state_lock, + graphs, resource_digests, resources: resource_list, dependencies, @@ -1365,13 +1938,28 @@ fn desired_config_digest( fn sha256_hex(bytes: &[u8]) -> String { let digest = Sha256::digest(bytes); + const HEX: &[u8; 16] = b"0123456789abcdef"; let mut out = String::with_capacity(digest.len() * 2); for byte in digest { - out.push_str(&format!("{byte:02x}")); + out.push(HEX[(byte >> 4) as usize] as char); + out.push(HEX[(byte & 0x0f) as usize] as char); } out } +fn now_rfc3339() -> String { + OffsetDateTime::now_utc() + .format(&Rfc3339) + .unwrap_or_else(|_| "1970-01-01T00:00:00Z".to_string()) +} + +fn state_sync_operation_label(operation: StateSyncOperation) -> &'static str { + match operation { + StateSyncOperation::Refresh => "refresh", + StateSyncOperation::Import => "import", + } +} + fn has_errors(diagnostics: &[Diagnostic]) -> bool { diagnostics .iter() @@ -1385,7 +1973,9 @@ fn display_path(path: &Path) -> String { #[cfg(test)] mod tests { use std::fs; + use std::path::Path; + use omnigraph::db::Omnigraph; use serde_json::json; use tempfile::tempdir; @@ -1435,6 +2025,15 @@ policies: dir } + async fn init_derived_graph(root: &Path) { + let graph_dir = root.join(CLUSTER_GRAPHS_DIR); + fs::create_dir_all(&graph_dir).unwrap(); + let graph = graph_dir.join("knowledge.omni"); + Omnigraph::init(graph.to_string_lossy().as_ref(), SCHEMA) + .await + .unwrap(); + } + #[test] fn valid_minimal_config() { let dir = fixture(); @@ -1906,4 +2505,280 @@ graphs: .any(|diagnostic| diagnostic.code == "unsupported_state_backend") ); } + + #[tokio::test] + async fn import_missing_state_creates_state_with_graph_observation() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + + let out = import_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.state_observations.state_revision, 1); + assert!(out.state_observations.state_cas.is_some()); + assert!(!out.state_observations.locked); + assert!(out.state_observations.lock_acquired); + assert!(out.state_observations.acquired_lock_id.is_some()); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + assert_eq!( + out.resource_digests + .get("schema.knowledge") + .map(String::as_str), + Some(sha256_hex(SCHEMA.as_bytes()).as_str()) + ); + assert!(out.observations["graph.knowledge"]["manifest_version"].is_number()); + assert_eq!( + out.observations["graph.knowledge"]["schema_matches_desired"], + true + ); + + let state: serde_json::Value = + serde_json::from_str(&fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap()) + .unwrap(); + assert_eq!(state["state_revision"], 1); + assert_eq!( + state["resource_statuses"]["graph.knowledge"]["status"], + "applied" + ); + } + + #[tokio::test] + async fn import_existing_state_fails() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + + let out = import_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_already_exists") + ); + } + + #[tokio::test] + async fn refresh_missing_state_fails() { + let dir = fixture(); + let out = refresh_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_missing") + ); + } + + #[tokio::test] + async fn refresh_existing_minimal_state_increments_revision_and_updates_cas() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"config_digest":"old","resources":{"graph.knowledge":{"digest":"old"}}}}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.state_observations.state_revision, 1); + assert!(out.state_observations.state_cas.is_some()); + assert!(!out.state_observations.locked); + assert!(out.state_observations.lock_acquired); + assert_eq!( + out.resource_statuses["graph.knowledge"].status, + ResourceLifecycleStatus::Applied + ); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[tokio::test] + async fn refresh_records_live_schema_digest_and_manifest_version() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"state_revision":4,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.state_observations.state_revision, 5); + assert_eq!( + out.observations["graph.knowledge"]["schema_digest"], + sha256_hex(SCHEMA.as_bytes()) + ); + assert!(out.observations["graph.knowledge"]["manifest_version"].is_u64()); + } + + #[tokio::test] + async fn missing_derived_graph_root_marks_drifted_and_plans_creates() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{"graph.knowledge":{"digest":"old-graph"},"schema.knowledge":{"digest":"old-schema"}}}}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!( + out.resource_statuses["graph.knowledge"].status, + ResourceLifecycleStatus::Drifted + ); + assert!(!out.resource_digests.contains_key("graph.knowledge")); + assert_eq!(out.observations["graph.knowledge"]["exists"], false); + + let plan = plan_config_dir(dir.path()); + assert!(plan.ok, "{:?}", plan.diagnostics); + assert!(plan.changes.iter().any(|change| { + change.resource == "graph.knowledge" && change.operation == PlanOperation::Create + })); + assert!(plan.changes.iter().any(|change| { + change.resource == "schema.knowledge" && change.operation == PlanOperation::Create + })); + } + + #[tokio::test] + async fn live_schema_mismatch_marks_drifted_and_causes_plan_update() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + fs::write( + dir.path().join("people.pg"), + SCHEMA.replace("age: I32?", "age: I32?\n nickname: String?"), + ) + .unwrap(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{"graph.knowledge":{"digest":"old-graph"},"schema.knowledge":{"digest":"old-schema"}}}}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!( + out.resource_statuses["schema.knowledge"].status, + ResourceLifecycleStatus::Drifted + ); + assert_eq!( + out.observations["graph.knowledge"]["schema_matches_desired"], + false + ); + + let plan = plan_config_dir(dir.path()); + assert!(plan.ok, "{:?}", plan.diagnostics); + assert!(plan.changes.iter().any(|change| { + change.resource == "schema.knowledge" && change.operation == PlanOperation::Update + })); + } + + #[tokio::test] + async fn existing_lock_makes_refresh_fail() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + fs::write( + state_dir.join("lock.json"), + r#"{"version":1,"lock_id":"held-lock","operation":"refresh","created_at":"2026-06-08T00:00:00Z","pid":123}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(!out.ok); + assert!(out.state_observations.locked); + assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); + assert!(!out.state_observations.lock_acquired); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_held") + ); + } + + #[tokio::test] + async fn state_lock_false_bypasses_refresh_lock_with_warning() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +state: + backend: cluster + lock: false +graphs: + knowledge: + schema: ./people.pg +"#, + ) + .unwrap(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!out.state_observations.locked); + assert!(!out.state_observations.lock_acquired); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_disabled") + ); + } + + #[tokio::test] + async fn external_state_backend_refresh_rejected() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\nstate:\n backend: s3://bucket/state\ngraphs: {}\n", + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "unsupported_state_backend") + ); + } + + #[tokio::test] + async fn import_graph_open_error_does_not_create_state() { + let dir = fixture(); + fs::create_dir_all(dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni")).unwrap(); + + let out = import_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "graph_observation_error") + ); + assert!(!dir.path().join(CLUSTER_STATE_FILE).exists()); + } } diff --git a/docs/dev/cluster-config-specs.md b/docs/dev/cluster-config-specs.md index 8094be2..8aa63cb 100644 --- a/docs/dev/cluster-config-specs.md +++ b/docs/dev/cluster-config-specs.md @@ -5,6 +5,13 @@ **Date:** 2026-06-07 **Relationship:** generalizes today's `omnigraph.yaml` graph/query/policy configuration surface ([CLI reference](../user/cli-reference.md), [server docs](../user/server.md)) into a future cluster control plane. The distilled rules are in [cluster-axioms.md](cluster-axioms.md); detailed downstream implementation spec and blast-radius assessment in [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md). This is a proposed architecture, not an implemented RFC. +> **Implementation status.** The examples below describe the full target schema. +> Stage 2B only accepts the read-only subset documented in +> [cluster-config.md](../user/cluster-config.md). Future-phase fields such as +> `env_file`, `apply`, `providers`, `pipelines`, `embeddings`, `ui`, `aliases`, +> and `bindings` are intentionally rejected with typed diagnostics until their +> reconciler semantics are implemented. + > **Revision 2026-06-07 — full commitment to the Terraform paradigm.** Three changes from the earlier draft: (1) **state is an authoritative, locked ledger in a backend** (server-hosted *or* a separate cloud store), not "a mostly-rebuildable projection"; (2) `plan` is framed as the **CLI diff between local config and state**; (3) **ETL pipelines** (external data sources) are a first-class config asset — a second seam, alongside schema, where a definition triggers a data-plane effect. The full set of config assets (incl. **aliases**, **embeddings**) is enumerated below. --- diff --git a/docs/dev/testing.md b/docs/dev/testing.md index d3bba9a..214dbf0 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,7 +8,7 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | -| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests` | Cluster config parser, local JSON state diff, state CAS/lock handling, read-only validate/plan/status | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests` | Cluster config parser, local JSON state diff, state CAS/lock handling, read-only validate/plan/status plus explicit refresh/import graph observations | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 594f983..ae47a4b 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -19,8 +19,7 @@ Top-level command families and subcommands. Graph-targeting commands accept eith | `commit list \| show` | inspect commit graph | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | -| `queries validate \| list` | operate on the server-side stored-query registry (the `queries:` block). `validate` type-checks every stored query against the live schema offline (opens the selected graph; exits non-zero on any breakage), catching schema drift without restarting the server; `list` prints the selected registry's query names, MCP exposure, and typed params. For per-graph registries, pass `--target <graph>` or set `cli.graph`; with no graph selection, `list` shows only top-level `queries:`. Distinct from `lint`, which validates a single `.gq` file | -| `cluster validate \| plan \| status` | read-only cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json` while briefly holding `__cluster/lock.json`; `status` reads the state ledger. No apply, graph open, live drift scan, server change, or `state.json` mutation occurs in Stage 2A | +| `cluster validate \| plan \| status \| refresh \| import` | cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`; `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations. No apply, graph-resource mutation, server change, or `plan --refresh` occurs in Stage 2B | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) | | `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | @@ -80,16 +79,21 @@ policy: omnigraph cluster validate --config ./company-brain omnigraph cluster plan --config ./company-brain --json omnigraph cluster status --config ./company-brain --json +omnigraph cluster refresh --config ./company-brain --json +omnigraph cluster import --config ./company-brain --json ``` `--config` is a directory containing `cluster.yaml`; it defaults to `.`. -Stage 2A accepts graphs, schemas, stored queries, and policy bundle file +Stage 2B accepts graphs, schemas, stored queries, and policy bundle file references. `cluster plan` reads local JSON state from -`<config-dir>/__cluster/state.json`; a missing file means empty state. Plan -acquires `__cluster/lock.json` by default and releases it before returning. -`cluster status` reads state only and reports any existing lock. External state -backends, apply, refresh/import, pipelines, UI specs, embeddings, aliases, and -bindings are reserved for later stages. See [cluster-config.md](cluster-config.md). +`<config-dir>/__cluster/state.json`; a missing file means empty state. Plan, +refresh, and import acquire `__cluster/lock.json` by default and release it +before returning. `cluster status` reads state only and reports any existing +lock. `refresh` requires an existing `state.json`; `import` creates one only +when it is missing. Both observe declared graphs read-only at +`<config-dir>/graphs/<graph-id>.omni`. External state backends, apply, +`plan --refresh`, pipelines, UI specs, embeddings, aliases, and bindings are +reserved for later stages. See [cluster-config.md](cluster-config.md). ## Output formats (`query` command, alias: `read`) diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 8f4eab1..77954bd 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -1,12 +1,13 @@ # Cluster Config -**Status:** Stage 2A read-only preview. +**Status:** Stage 2B state-observation preview. Cluster config is the future control-plane configuration surface for a whole OmniGraph deployment. In this stage, OmniGraph can validate a local -`cluster.yaml` folder, produce a deterministic read-only plan, and inspect the -local JSON state ledger. It does not apply changes, open graph roots, scan live -cluster state, start servers, or write graph resources. +`cluster.yaml` folder, produce a deterministic read-only plan, inspect the +local JSON state ledger, and explicitly refresh/import graph observations into +that ledger. It does not apply desired changes, start servers, or write graph +resources. ## Commands @@ -14,6 +15,8 @@ cluster state, start servers, or write graph resources. omnigraph cluster validate --config ./company-brain omnigraph cluster plan --config ./company-brain --json omnigraph cluster status --config ./company-brain --json +omnigraph cluster refresh --config ./company-brain --json +omnigraph cluster import --config ./company-brain --json ``` `--config` points at a directory, not a file. The directory must contain @@ -21,7 +24,7 @@ omnigraph cluster status --config ./company-brain --json ## Supported `cluster.yaml` -Stage 2A accepts only the read-only resource subset: +Stage 2B accepts only the read-only resource subset: ```yaml version: 1 @@ -47,10 +50,10 @@ policies: `metadata.name` is a display label. `state.backend` may be omitted or set to `cluster`; external state backends are reserved for a later stage. `state.lock` -defaults to `true`. When enabled, `cluster plan` briefly acquires -`<config-dir>/__cluster/lock.json` while it reads state, then removes it before -returning. `cluster status` never acquires the lock; it only reports whether one -is present. +defaults to `true`. When enabled, `cluster plan`, `cluster refresh`, and +`cluster import` briefly acquire `<config-dir>/__cluster/lock.json`, then remove +it before returning. `cluster status` never acquires the lock; it only reports +whether one is present. ## Validation @@ -115,8 +118,10 @@ and reports `create`, `update`, and `delete` changes. It also reports the state CAS (`sha256:<digest>`) and state revision. `state_observations.locked` means an existing lock file was observed; a successful `plan` instead reports `lock_acquired: true` and an `acquired_lock_id`, then releases the lock before -returning. The command never writes `state.json`; apply, refresh, import, and -live drift scans are later-stage work. +returning. The command never writes `state.json` and does not scan live graphs. +Use explicit `cluster refresh` / `cluster import` when the state ledger should +be updated from live observations. Apply and live drift scans during plan are +later-stage work. ## Status @@ -124,3 +129,24 @@ live drift scans are later-stage work. ledger says is deployed. It does not validate referenced schema/query/policy files and does not inspect live graphs. Missing `state.json` succeeds with a warning; invalid state JSON or an unsupported state version fails. + +## Refresh And Import + +`cluster refresh` updates an existing `state.json` from actual observations. +`cluster import` creates the first `state.json` when the ledger is missing. +Both commands open declared graphs read-only at: + +```text +<config-dir>/graphs/<graph-id>.omni +``` + +They observe only branch `main`, recording graph existence, manifest version, +live schema digest, desired schema digest, and schema-match status under +`observations["graph.<id>"]`. Missing graph roots are recorded as drift and +remove the graph/schema digests from state so a later `plan` proposes creates. +Invalid graph roots are recorded as errors; `refresh` persists the error +observation and exits non-zero, while `import` exits non-zero without creating +initial state. + +Refresh/import do not observe query or policy resources yet. Existing query and +policy state digests are preserved on refresh and are not invented on import. From 89b876c797277a7d86b8bc6375be0b379d379597 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Tue, 9 Jun 2026 02:12:00 +0300 Subject: [PATCH 036/165] Add cluster state lock recovery --- crates/omnigraph-cli/src/main.rs | 89 ++++++- crates/omnigraph-cli/tests/cli.rs | 155 +++++++++++-- crates/omnigraph-cluster/src/lib.rs | 346 +++++++++++++++++++++++++++- docs/dev/testing.md | 2 +- docs/user/cli-reference.md | 15 +- docs/user/cluster-config.md | 41 +++- 6 files changed, 597 insertions(+), 51 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 9c16722..971ffff 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -11,8 +11,9 @@ use omnigraph::db::{Omnigraph, ReadTarget, SnapshotId}; use omnigraph::loader::LoadMode; use omnigraph::storage::normalize_root_uri; use omnigraph_cluster::{ - DiagnosticSeverity, PlanOutput, StateSyncOutput, StatusOutput, ValidateOutput, - import_config_dir, plan_config_dir, refresh_config_dir, status_config_dir, validate_config_dir, + DiagnosticSeverity, ForceUnlockOutput, PlanOutput, StateSyncOutput, StatusOutput, + ValidateOutput, force_unlock_config_dir, import_config_dir, plan_config_dir, + refresh_config_dir, status_config_dir, validate_config_dir, }; use omnigraph_compiler::query::parser::parse_query; use omnigraph_compiler::schema::parser::parse_schema; @@ -387,6 +388,17 @@ enum ClusterCommand { #[arg(long)] json: bool, }, + /// Remove a held local JSON state lock after operator confirmation. + ForceUnlock { + /// Exact lock id from cluster status or a state_lock_held diagnostic. + lock_id: String, + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, } /// Operations on the graph registry of a multi-graph server (MR-668). @@ -804,10 +816,7 @@ fn print_cluster_status_human(output: &StatusOutput) { println!(" applied config: {digest}"); } if state.locked { - match state.lock_id.as_deref() { - Some(lock_id) => println!(" lock: held ({lock_id})"), - None => println!(" lock: held"), - } + println!(" lock: held{}", cluster_lock_summary(state)); } else { println!(" lock: not held"); } @@ -835,10 +844,7 @@ fn print_cluster_state_sync_human(output: &StateSyncOutput) { println!(" state_cas: {cas}"); } if state.locked { - match state.lock_id.as_deref() { - Some(lock_id) => println!(" lock: acquired ({lock_id})"), - None => println!(" lock: acquired"), - } + println!(" lock: acquired{}", cluster_lock_summary(state)); } else { println!(" lock: not acquired"); } @@ -848,6 +854,48 @@ fn print_cluster_state_sync_human(output: &StateSyncOutput) { print_cluster_diagnostics(&output.diagnostics); } +fn print_cluster_force_unlock_human(output: &ForceUnlockOutput) { + if output.ok { + if output.lock_removed { + println!( + "cluster force-unlock: removed lock{}", + cluster_lock_summary(&output.state_observations) + ); + } else { + println!("cluster force-unlock: no lock removed"); + } + } else { + println!("cluster force-unlock failed"); + if output.state_observations.locked { + println!( + " lock: held{}", + cluster_lock_summary(&output.state_observations) + ); + } + } + print_cluster_diagnostics(&output.diagnostics); +} + +fn cluster_lock_summary(state: &omnigraph_cluster::StateObservations) -> String { + let Some(lock_id) = state.lock_id.as_deref() else { + return String::new(); + }; + let mut parts = vec![format!("id={lock_id}")]; + if let Some(operation) = state.lock_operation.as_deref() { + parts.push(format!("operation={operation}")); + } + if let Some(pid) = state.lock_pid { + parts.push(format!("pid={pid}")); + } + if let Some(created_at) = state.lock_created_at.as_deref() { + parts.push(format!("created_at={created_at}")); + } + if let Some(age_seconds) = state.lock_age_seconds { + parts.push(format!("age_seconds={age_seconds}")); + } + format!(" ({})", parts.join(", ")) +} + fn print_cluster_diagnostics(diagnostics: &[omnigraph_cluster::Diagnostic]) { for diagnostic in diagnostics { let label = match diagnostic.severity { @@ -913,6 +961,19 @@ fn finish_cluster_state_sync(output: &StateSyncOutput, json: bool) -> Result<()> Ok(()) } +fn finish_cluster_force_unlock(output: &ForceUnlockOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_force_unlock_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + fn is_remote_uri(uri: &str) -> bool { uri.starts_with("http://") || uri.starts_with("https://") } @@ -3443,6 +3504,14 @@ async fn main() -> Result<()> { let output = import_config_dir(config).await; finish_cluster_state_sync(&output, json)?; } + ClusterCommand::ForceUnlock { + lock_id, + config, + json, + } => { + let output = force_unlock_config_dir(config, lock_id); + finish_cluster_force_unlock(&output, json)?; + } }, Command::Graphs { command } => match command { GraphsCommand::List { diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 504f0ef..1dd26a7 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -156,6 +156,18 @@ fn init_cluster_derived_graph(root: &std::path::Path) { ); } +fn write_cluster_lock(root: &std::path::Path, lock_id: &str, operation: &str) { + let state_dir = root.join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("lock.json"), + format!( + r#"{{"version":1,"lock_id":"{lock_id}","operation":"{operation}","created_at":"1970-01-01T00:00:00Z","pid":123}}"# + ), + ) + .unwrap(); +} + #[test] fn version_command_prints_current_cli_version() { let output = output_success(cli().arg("version")); @@ -271,6 +283,32 @@ fn cluster_status_json_reports_missing_state() { ); } +#[test] +fn cluster_status_json_reports_lock_metadata() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_lock(temp.path(), "held-lock", "refresh"); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("status") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["state_observations"]["locked"], true); + assert_eq!(json["state_observations"]["lock_id"], "held-lock"); + assert_eq!(json["state_observations"]["lock_operation"], "refresh"); + assert_eq!(json["state_observations"]["lock_pid"], 123); + assert_eq!( + json["state_observations"]["lock_created_at"], + "1970-01-01T00:00:00Z" + ); + assert!(json["state_observations"]["lock_age_seconds"].is_number()); +} + #[test] fn cluster_status_json_reports_extended_state() { let temp = tempdir().unwrap(); @@ -372,21 +410,7 @@ fn cluster_plan_json_includes_state_cas_revision_and_lock_observation() { fn cluster_plan_locked_state_exits_nonzero() { let temp = tempdir().unwrap(); write_cluster_config_fixture(temp.path()); - let state_dir = temp.path().join("__cluster"); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("lock.json"), - r#" -{ - "version": 1, - "lock_id": "held-lock", - "operation": "plan", - "created_at": "2026-06-08T00:00:00Z", - "pid": 123 -} -"#, - ) - .unwrap(); + write_cluster_lock(temp.path(), "held-lock", "plan"); let output = output_failure( cli() @@ -401,16 +425,115 @@ fn cluster_plan_locked_state_exits_nonzero() { assert_eq!(json["state_observations"]["locked"], true); assert_eq!(json["state_observations"]["lock_acquired"], false); assert_eq!(json["state_observations"]["lock_id"], "held-lock"); + assert_eq!(json["state_observations"]["lock_operation"], "plan"); + assert_eq!(json["state_observations"]["lock_pid"], 123); + assert_eq!( + json["state_observations"]["lock_created_at"], + "1970-01-01T00:00:00Z" + ); + assert!(json["state_observations"]["lock_age_seconds"].is_number()); assert!( json["diagnostics"] .as_array() .unwrap() .iter() - .any(|diagnostic| diagnostic["code"] == "state_lock_held"), + .any(|diagnostic| diagnostic["code"] == "state_lock_held" + && diagnostic["message"] + .as_str() + .unwrap() + .contains("force-unlock held-lock")), "locked state should produce a useful diagnostic: {json}" ); } +#[test] +fn cluster_force_unlock_json_removes_lock() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_lock(temp.path(), "held-lock", "plan"); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("force-unlock") + .arg("held-lock") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["lock_removed"], true); + assert_eq!(json["state_observations"]["lock_id"], "held-lock"); + assert_eq!(json["state_observations"]["lock_operation"], "plan"); + assert!(!temp.path().join("__cluster/lock.json").exists()); +} + +#[test] +fn cluster_force_unlock_wrong_id_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_lock(temp.path(), "held-lock", "plan"); + + let json = parse_stdout_json(&output_failure( + cli() + .arg("cluster") + .arg("force-unlock") + .arg("other-lock") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], false); + assert_eq!(json["lock_removed"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_lock_id_mismatch") + ); + assert!(temp.path().join("__cluster/lock.json").exists()); +} + +#[test] +fn cluster_locked_plan_then_force_unlock_then_plan_succeeds() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_lock(temp.path(), "held-lock", "plan"); + + let locked = parse_stdout_json(&output_failure( + cli() + .arg("cluster") + .arg("plan") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(locked["ok"], false); + assert_eq!(locked["state_observations"]["lock_id"], "held-lock"); + + let unlocked = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("force-unlock") + .arg("held-lock") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(unlocked["lock_removed"], true); + + let planned = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("plan") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(planned["ok"], true); +} + #[test] fn cluster_import_json_bootstraps_missing_state() { let temp = tempdir().unwrap(); diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 9a6ea78..7ae824c 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -110,6 +110,25 @@ pub struct StateObservations { pub lock_acquired: bool, #[serde(skip_serializing_if = "Option::is_none")] pub acquired_lock_id: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub lock_operation: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub lock_created_at: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub lock_pid: Option<u32>, + #[serde(skip_serializing_if = "Option::is_none")] + pub lock_age_seconds: Option<u64>, +} + +impl StateObservations { + fn observe_lock_metadata(&mut self, lock: &StateLockFile) { + self.locked = true; + self.lock_id = Some(lock.lock_id.clone()); + self.lock_operation = Some(lock.operation.clone()); + self.lock_created_at = Some(lock.created_at.clone()); + self.lock_pid = Some(lock.pid); + self.lock_age_seconds = lock_age_seconds(&lock.created_at); + } } #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] @@ -208,6 +227,15 @@ pub struct StateSyncOutput { pub diagnostics: Vec<Diagnostic>, } +#[derive(Debug, Clone, Serialize)] +pub struct ForceUnlockOutput { + pub ok: bool, + pub config_dir: String, + pub state_observations: StateObservations, + pub lock_removed: bool, + pub diagnostics: Vec<Diagnostic>, +} + #[derive(Debug, Clone)] struct DesiredCluster { config_dir: PathBuf, @@ -518,6 +546,35 @@ pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { } } +pub fn force_unlock_config_dir( + config_dir: impl AsRef<Path>, + lock_id: impl AsRef<str>, +) -> ForceUnlockOutput { + let parsed = parse_cluster_config(config_dir.as_ref()); + let mut diagnostics = parsed.diagnostics; + let backend = LocalStateBackend::new(&parsed.config_dir); + let mut observations = backend.observations(); + let mut lock_removed = false; + + if let Some(raw) = parsed.raw.as_ref() { + let _settings = validate_cluster_header(raw, &mut diagnostics); + if !has_errors(&diagnostics) { + match backend.force_unlock(lock_id.as_ref(), &mut observations) { + Ok(()) => lock_removed = true, + Err(diagnostic) => diagnostics.push(diagnostic), + } + } + } + + ForceUnlockOutput { + ok: !has_errors(&diagnostics), + config_dir: display_path(&parsed.config_dir), + state_observations: observations, + lock_removed, + diagnostics, + } +} + pub async fn refresh_config_dir(config_dir: impl AsRef<Path>) -> StateSyncOutput { sync_config_dir(config_dir.as_ref(), StateSyncOperation::Refresh).await } @@ -791,7 +848,7 @@ fn validate_cluster_header( diagnostics.push(Diagnostic::error( "unsupported_state_backend", "state.backend", - "Stage 2B supports only omitted state.backend or `cluster`", + "Stage 2C supports only omitted state.backend or `cluster`", )); } } @@ -824,6 +881,10 @@ impl LocalStateBackend { lock_id: None, lock_acquired: false, acquired_lock_id: None, + lock_operation: None, + lock_created_at: None, + lock_pid: None, + lock_age_seconds: None, } } @@ -1035,11 +1096,11 @@ impl LocalStateBackend { }) } Err(err) if err.kind() == ErrorKind::AlreadyExists => { - self.observe_lock_id(observations); + self.observe_lock_metadata_lossy(observations); Err(Diagnostic::error( "state_lock_held", CLUSTER_LOCK_FILE, - "cluster state lock already exists; remove it only after confirming no cluster operation is active", + state_lock_held_message(observations), )) } Err(err) => Err(Diagnostic::error( @@ -1050,6 +1111,52 @@ impl LocalStateBackend { } } + fn force_unlock( + &self, + requested_lock_id: &str, + observations: &mut StateObservations, + ) -> Result<(), Diagnostic> { + let text = match fs::read_to_string(&self.lock_path) { + Ok(text) => text, + Err(err) if err.kind() == ErrorKind::NotFound => { + return Err(Diagnostic::error( + "state_lock_missing", + CLUSTER_LOCK_FILE, + "cluster state lock is not present; nothing was unlocked", + )); + } + Err(err) => { + return Err(Diagnostic::error( + "state_lock_read_error", + CLUSTER_LOCK_FILE, + format!("could not read state lock: {err}"), + )); + } + }; + observations.locked = true; + let lock = parse_lock_file_for_unlock(&text)?; + observations.observe_lock_metadata(&lock); + + if lock.lock_id != requested_lock_id { + return Err(Diagnostic::error( + "state_lock_id_mismatch", + CLUSTER_LOCK_FILE, + format!( + "cluster state lock id is {}; refusing to unlock with requested id {requested_lock_id}", + lock.lock_id + ), + )); + } + + fs::remove_file(&self.lock_path).map_err(|err| { + Diagnostic::error( + "state_unlock_error", + CLUSTER_LOCK_FILE, + format!("could not remove state lock: {err}"), + ) + }) + } + fn observe_lock( &self, observations: &mut StateObservations, @@ -1060,7 +1167,7 @@ impl LocalStateBackend { match fs::read_to_string(&self.lock_path) { Ok(text) => match serde_json::from_str::<StateLockFile>(&text) { Ok(lock) if lock.version == 1 => { - observations.lock_id = Some(lock.lock_id); + observations.observe_lock_metadata(&lock); } Ok(lock) => diagnostics.push(Diagnostic::warning( "unsupported_state_lock_version", @@ -1082,12 +1189,12 @@ impl LocalStateBackend { } } - fn observe_lock_id(&self, observations: &mut StateObservations) { + fn observe_lock_metadata_lossy(&self, observations: &mut StateObservations) { observations.locked = true; if let Ok(text) = fs::read_to_string(&self.lock_path) { if let Ok(lock) = serde_json::from_str::<StateLockFile>(&text) { if lock.version == 1 { - observations.lock_id = Some(lock.lock_id); + observations.observe_lock_metadata(&lock); } } } @@ -1100,6 +1207,33 @@ impl Drop for StateLockGuard { } } +fn parse_lock_file_for_unlock(text: &str) -> Result<StateLockFile, Diagnostic> { + let lock = serde_json::from_str::<StateLockFile>(text).map_err(|err| { + Diagnostic::error( + "invalid_state_lock", + CLUSTER_LOCK_FILE, + format!("could not parse state lock: {err}"), + ) + })?; + if lock.version != 1 { + return Err(Diagnostic::error( + "unsupported_state_lock_version", + CLUSTER_LOCK_FILE, + format!("unsupported cluster state lock version {}", lock.version), + )); + } + Ok(lock) +} + +fn state_lock_held_message(observations: &StateObservations) -> String { + match observations.lock_id.as_deref() { + Some(lock_id) => format!( + "cluster state lock already exists (lock id {lock_id}); run `omnigraph cluster force-unlock {lock_id}` only after confirming no cluster operation is active" + ), + None => "cluster state lock already exists; remove it only after confirming no cluster operation is active".to_string(), + } +} + fn state_resource_digests(state: &ClusterState) -> BTreeMap<String, String> { state .applied_revision @@ -1953,6 +2087,15 @@ fn now_rfc3339() -> String { .unwrap_or_else(|_| "1970-01-01T00:00:00Z".to_string()) } +fn lock_age_seconds(created_at: &str) -> Option<u64> { + let created_at = OffsetDateTime::parse(created_at, &Rfc3339).ok()?; + Some( + (OffsetDateTime::now_utc() - created_at) + .whole_seconds() + .max(0) as u64, + ) +} + fn state_sync_operation_label(operation: StateSyncOperation) -> &'static str { match operation { StateSyncOperation::Refresh => "refresh", @@ -2034,6 +2177,23 @@ policies: .unwrap(); } + fn write_lock_file(config_dir: &Path, lock_id: &str, operation: &str) { + let state_dir = config_dir.join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("lock.json"), + json!({ + "version": 1, + "lock_id": lock_id, + "operation": operation, + "created_at": "1970-01-01T00:00:00Z", + "pid": 123 + }) + .to_string(), + ) + .unwrap(); + } + #[test] fn valid_minimal_config() { let dir = fixture(); @@ -2383,6 +2543,164 @@ policies: ); } + #[test] + fn status_surfaces_full_lock_metadata() { + let dir = fixture(); + write_lock_file(dir.path(), "held-lock", "refresh"); + + let out = status_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.state_observations.locked); + assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); + assert_eq!( + out.state_observations.lock_operation.as_deref(), + Some("refresh") + ); + assert_eq!( + out.state_observations.lock_created_at.as_deref(), + Some("1970-01-01T00:00:00Z") + ); + assert_eq!(out.state_observations.lock_pid, Some(123)); + assert!(out.state_observations.lock_age_seconds.is_some()); + } + + #[test] + fn force_unlock_matching_id_removes_lock() { + let dir = fixture(); + write_lock_file(dir.path(), "held-lock", "plan"); + + let out = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.lock_removed); + assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); + assert_eq!( + out.state_observations.lock_operation.as_deref(), + Some("plan") + ); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn force_unlock_wrong_id_fails_and_preserves_lock() { + let dir = fixture(); + write_lock_file(dir.path(), "held-lock", "plan"); + + let out = force_unlock_config_dir(dir.path(), "other-lock"); + assert!(!out.ok); + assert!(!out.lock_removed); + assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_id_mismatch") + ); + assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn force_unlock_missing_lock_fails() { + let dir = fixture(); + + let out = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(!out.ok); + assert!(!out.lock_removed); + assert!(!out.state_observations.locked); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_missing") + ); + } + + #[test] + fn force_unlock_invalid_lock_json_fails_and_preserves_lock() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write(state_dir.join("lock.json"), "{").unwrap(); + + let out = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(!out.ok); + assert!(!out.lock_removed); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "invalid_state_lock") + ); + assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn force_unlock_unsupported_lock_version_fails_and_preserves_lock() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("lock.json"), + r#"{"version":2,"lock_id":"held-lock","operation":"plan","created_at":"1970-01-01T00:00:00Z","pid":123}"#, + ) + .unwrap(); + + let out = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(!out.ok); + assert!(!out.lock_removed); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "unsupported_state_lock_version") + ); + assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn force_unlock_external_state_backend_rejected() { + let dir = fixture(); + write_lock_file(dir.path(), "held-lock", "plan"); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +state: + backend: s3://state-bucket/cluster +graphs: + knowledge: + schema: ./people.pg +"#, + ) + .unwrap(); + + let out = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(!out.ok); + assert!(!out.lock_removed); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "unsupported_state_backend") + ); + assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn plan_succeeds_after_force_unlock() { + let dir = fixture(); + write_lock_file(dir.path(), "held-lock", "plan"); + + let locked = plan_config_dir(dir.path()); + assert!(!locked.ok); + assert!( + locked + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_held") + ); + + let unlocked = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(unlocked.ok, "{:?}", unlocked.diagnostics); + + let out = plan_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + } + #[test] fn plan_reports_state_cas_revision_and_removes_lock() { let dir = fixture(); @@ -2440,11 +2758,19 @@ policies: assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); assert!(!out.state_observations.lock_acquired); assert!(out.state_observations.acquired_lock_id.is_none()); + assert_eq!( + out.state_observations.lock_operation.as_deref(), + Some("plan") + ); assert!( out.diagnostics .iter() .any(|diagnostic| diagnostic.code == "state_lock_held") ); + assert!(out.diagnostics.iter().any(|diagnostic| { + diagnostic.code == "state_lock_held" + && diagnostic.message.contains("force-unlock held-lock") + })); } #[test] @@ -2706,11 +3032,19 @@ graphs: assert!(out.state_observations.locked); assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); assert!(!out.state_observations.lock_acquired); + assert_eq!( + out.state_observations.lock_operation.as_deref(), + Some("refresh") + ); assert!( out.diagnostics .iter() .any(|diagnostic| diagnostic.code == "state_lock_held") ); + assert!(out.diagnostics.iter().any(|diagnostic| { + diagnostic.code == "state_lock_held" + && diagnostic.message.contains("force-unlock held-lock") + })); } #[tokio::test] diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 214dbf0..3c5ee32 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,7 +8,7 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | -| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests` | Cluster config parser, local JSON state diff, state CAS/lock handling, read-only validate/plan/status plus explicit refresh/import graph observations | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests` | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index ae47a4b..70ac6f4 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -19,7 +19,7 @@ Top-level command families and subcommands. Graph-targeting commands accept eith | `commit list \| show` | inspect commit graph | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | -| `cluster validate \| plan \| status \| refresh \| import` | cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`; `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations. No apply, graph-resource mutation, server change, or `plan --refresh` occurs in Stage 2B | +| `cluster validate \| plan \| status \| refresh \| import \| force-unlock` | cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`; `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id. No apply, graph-resource mutation, server change, automatic stale-lock breaking, or `plan --refresh` occurs in Stage 2C | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) | | `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | @@ -81,19 +81,22 @@ omnigraph cluster plan --config ./company-brain --json omnigraph cluster status --config ./company-brain --json omnigraph cluster refresh --config ./company-brain --json omnigraph cluster import --config ./company-brain --json +omnigraph cluster force-unlock <LOCK_ID> --config ./company-brain --json ``` `--config` is a directory containing `cluster.yaml`; it defaults to `.`. -Stage 2B accepts graphs, schemas, stored queries, and policy bundle file +Stage 2C accepts graphs, schemas, stored queries, and policy bundle file references. `cluster plan` reads local JSON state from `<config-dir>/__cluster/state.json`; a missing file means empty state. Plan, refresh, and import acquire `__cluster/lock.json` by default and release it before returning. `cluster status` reads state only and reports any existing -lock. `refresh` requires an existing `state.json`; `import` creates one only -when it is missing. Both observe declared graphs read-only at +lock metadata. `force-unlock` removes a lock only when the supplied id exactly +matches the lock file. `refresh` requires an existing `state.json`; `import` +creates one only when it is missing. Both observe declared graphs read-only at `<config-dir>/graphs/<graph-id>.omni`. External state backends, apply, -`plan --refresh`, pipelines, UI specs, embeddings, aliases, and bindings are -reserved for later stages. See [cluster-config.md](cluster-config.md). +automatic stale-lock breaking, `plan --refresh`, pipelines, UI specs, +embeddings, aliases, and bindings are reserved for later stages. See +[cluster-config.md](cluster-config.md). ## Output formats (`query` command, alias: `read`) diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 77954bd..24718b1 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -1,13 +1,13 @@ # Cluster Config -**Status:** Stage 2B state-observation preview. +**Status:** Stage 2C state-lock recovery preview. Cluster config is the future control-plane configuration surface for a whole OmniGraph deployment. In this stage, OmniGraph can validate a local `cluster.yaml` folder, produce a deterministic read-only plan, inspect the local JSON state ledger, and explicitly refresh/import graph observations into -that ledger. It does not apply desired changes, start servers, or write graph -resources. +that ledger. It can also manually remove a held local state lock by exact lock +id. It does not apply desired changes, start servers, or write graph resources. ## Commands @@ -17,6 +17,7 @@ omnigraph cluster plan --config ./company-brain --json omnigraph cluster status --config ./company-brain --json omnigraph cluster refresh --config ./company-brain --json omnigraph cluster import --config ./company-brain --json +omnigraph cluster force-unlock <LOCK_ID> --config ./company-brain --json ``` `--config` points at a directory, not a file. The directory must contain @@ -24,7 +25,7 @@ omnigraph cluster import --config ./company-brain --json ## Supported `cluster.yaml` -Stage 2B accepts only the read-only resource subset: +Stage 2C accepts only the read-only resource subset: ```yaml version: 1 @@ -53,7 +54,9 @@ policies: defaults to `true`. When enabled, `cluster plan`, `cluster refresh`, and `cluster import` briefly acquire `<config-dir>/__cluster/lock.json`, then remove it before returning. `cluster status` never acquires the lock; it only reports -whether one is present. +whether one is present. `cluster force-unlock` is the only lock-removal command; +it requires the exact lock id and should be run only after confirming no cluster +operation is active. ## Validation @@ -116,19 +119,22 @@ Missing `state_revision` is treated as `0`. Resource status values are Plan output compares desired resource digests against state resource digests and reports `create`, `update`, and `delete` changes. It also reports the state CAS (`sha256:<digest>`) and state revision. `state_observations.locked` means an -existing lock file was observed; a successful `plan` instead reports -`lock_acquired: true` and an `acquired_lock_id`, then releases the lock before -returning. The command never writes `state.json` and does not scan live graphs. -Use explicit `cluster refresh` / `cluster import` when the state ledger should -be updated from live observations. Apply and live drift scans during plan are -later-stage work. +existing lock file was observed, along with its metadata (`lock_id`, +`lock_operation`, `lock_created_at`, `lock_pid`, `lock_age_seconds`); a +successful `plan` instead reports `lock_acquired: true` and an +`acquired_lock_id`, then releases the lock before returning. The command never +writes `state.json` and does not scan live graphs. Use explicit +`cluster refresh` / `cluster import` when the state ledger should be updated +from live observations. Apply and live drift scans during plan are later-stage +work. ## Status `cluster status` reads the same local JSON state ledger and prints what the ledger says is deployed. It does not validate referenced schema/query/policy files and does not inspect live graphs. Missing `state.json` succeeds with a -warning; invalid state JSON or an unsupported state version fails. +warning; invalid state JSON or an unsupported state version fails. If a lock is +present, status reports its id, operation, creation time, pid, and age. ## Refresh And Import @@ -150,3 +156,14 @@ initial state. Refresh/import do not observe query or policy resources yet. Existing query and policy state digests are preserved on refresh and are not invented on import. + +## Force Unlock + +`cluster force-unlock <LOCK_ID>` removes `<config-dir>/__cluster/lock.json` only +when the file exists, is valid version-1 lock JSON, and its `lock_id` exactly +matches the argument. A wrong id, missing lock, invalid lock JSON, or unsupported +lock version exits non-zero and leaves the file untouched. + +This is manual recovery for abandoned local locks. OmniGraph does not perform +PID-liveness checks, TTL expiry, stale-lock breaking, or automatic unlock in +Stage 2C. From 171a8c5d13dd6dfa0566253a90ac746cb76354ba Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Tue, 9 Jun 2026 22:31:50 +0300 Subject: [PATCH 037/165] docs(releases): attribute the __run__ sweep (MR-770) to v0.6.2, not v0.6.1 (#161) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The v0.6.2 notes omitted the MR-770 `__run__` cleanup entirely, and the v0.6.1 notes wrongly claimed it shipped in v0.6.1. The code (the `migrate_v2_to_v3` `__manifest` sweep + `is_internal_run_branch`/`run_registry.rs` removal) first appears at the v0.6.2 tag via #132 and is absent at v0.6.1. - v0.6.2: add the MR-770 highlight, correct the manifest-stamp note to describe the v2→v3 auto-migration on first read-write open (with the read-only caveat), and mention the cleanup in the intro. - v0.6.1: replace the two over-claiming `__run__` lines with corrections that point to v0.6.2. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- docs/releases/v0.6.1.md | 4 ++-- docs/releases/v0.6.2.md | 22 ++++++++++++++++++---- 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/docs/releases/v0.6.1.md b/docs/releases/v0.6.1.md index 0acc34b..eb76e1f 100644 --- a/docs/releases/v0.6.1.md +++ b/docs/releases/v0.6.1.md @@ -7,7 +7,7 @@ v0.6.1 focuses on operational polish after v0.6.0: stored-query registries, safe - **Stored-query registries.** `omnigraph.yaml` can declare curated `queries:` blocks per graph. Servers load and type-check them at startup, `omnigraph queries validate` checks them offline, `omnigraph queries list` shows exposed queries and typed params, `GET /queries` exposes a typed catalog, and `POST /queries/{name}` invokes a stored query without accepting ad hoc `.gq` source from the client. - **Stored-query policy gate.** New Cedar action `invoke_query` gates the stored-query invocation surface. Stored mutations are double-gated: `invoke_query` to reach the stored query and `change` for the actual write. - **Safer branch deletion.** `branch_delete` now treats the manifest as the authority, flips branch visibility atomically, and reclaims per-table/commit-graph forks as derived state. If best-effort reclaim is interrupted, `cleanup` reconciles orphaned forks; reusing a branch name before cleanup reports an actionable error. -- **Legacy `__run__` cleanup (MR-770).** Removed the last functional remnant of the Run state machine (retired in v0.4.0): the `__run__` branch-name guard. A new v2→v3 `__manifest` internal-schema migration sweeps any stale `__run__*` staging branches on the first read-write open, so `__run__*` is no longer a reserved branch name. This closes the "unpromoted `__run__` branches block reads" condition behind the zombie-run cascade incident; the inert `_graph_runs.lance` row cleanup is tracked separately (it needs a `delete_prefix` primitive). +- **Legacy `__run__` cleanup (MR-770).** *(Correction: this item shipped in [v0.6.2](v0.6.2.md), not v0.6.1 — the v0.6.1 notes over-claimed it. At the v0.6.1 tag the `__run__` branch-name guard and `run_registry.rs` were still present and no v2→v3 sweep migration existed.)* The guard removal and the one-time v2→v3 `__manifest` migration that sweeps stale `__run__*` staging branches on first read-write open are described in the v0.6.2 release notes. - **Blob-safe optimize.** `omnigraph optimize` skips tables with `Blob` properties instead of failing the whole sweep on Lance's blob-v2 compaction decode bug. Skips are visible in human output, `--json` as `skipped`, `TableOptimizeStats.skipped`, and logs; non-blob tables still compact normally. - **Deployment improvements.** The container entrypoint now composes `OMNIGRAPH_TARGET_URI` with `OMNIGRAPH_CONFIG`, so operators can keep the graph URI in env while loading policy/query config from a mounted file. The local RustFS bootstrap pins RustFS beta.3 and allows the current insecure local-dev default credentials. - **Windows release support.** Tagged and edge releases now publish Windows x86_64 archives containing `omnigraph.exe` and `omnigraph-server.exe`, with a PowerShell installer and Windows install docs. @@ -18,7 +18,7 @@ v0.6.1 focuses on operational polish after v0.6.0: stored-query registries, safe - A graph selected by name (`--target` or `server.graph`) now uses `graphs.<name>.policy` and `graphs.<name>.queries`. Top-level `policy` / `queries` blocks are only for anonymous bare-URI single-graph mode; using them with a named graph now fails loudly with migration guidance. - `mcp.expose` defaults to `true` for stored-query registry entries. Set `mcp: { expose: false }` for service-only queries that should not appear in the catalog. - `invoke_query` is graph-scoped, not branch-scoped. Branch/snapshot access remains enforced by the inner `read` / `change` gate. -- **Legacy `__run__` migration.** Graphs created before v0.4.0 are migrated automatically on the first **read-write** open by a v0.6.1 binary (one-time `__manifest` stamp v2→v3 sweep of stale `__run__*` branches). No action required. Two caveats: (1) a graph opened **read-only** still lists any stale `__run__*` branch until its first read-write open, since the migration is write-path-only like all manifest migrations — long-lived read-only deployments should be opened read-write once after upgrading; (2) the inert `_graph_runs.lance` / `_graph_run_actors.lance` dataset bytes are left in place until a future `delete_prefix` primitive (they are invisible to graph-level state). +- **Legacy `__run__` migration.** *(Correction: deferred to [v0.6.2](v0.6.2.md).)* The automatic v2→v3 `__manifest` stamp migration that sweeps stale `__run__*` branches on first read-write open ships in v0.6.2, not v0.6.1; a v0.6.1 binary does not perform it. See the v0.6.2 notes for the migration behavior and the read-only caveat. - Blob tables are not compacted until the upstream Lance fix lands, so fragment count and deleted-row space on blob tables are not reclaimed by `optimize`. Reads, writes, and query results are unaffected; no on-disk migration is required. - `TableOptimizeStats` is now `#[non_exhaustive]` and gains a `skipped: Option<SkipReason>` field (so does the new `SkipReason` enum). This is a source-level change only for downstream code that built this returned result struct by literal — rare, since it is produced by `optimize` and consumed by reading its fields; field access is unaffected, and `#[non_exhaustive]` keeps future additions non-breaking. diff --git a/docs/releases/v0.6.2.md b/docs/releases/v0.6.2.md index 2504813..f97f67b 100644 --- a/docs/releases/v0.6.2.md +++ b/docs/releases/v0.6.2.md @@ -2,8 +2,9 @@ v0.6.2 is a maintenance-safety release on top of v0.6.1. It tightens the `optimize` / recovery boundary, adds an explicit repair path for uncovered -manifest/head drift, accepts pretty-printed JSON load input, and updates the -project governance and release automation around those fixes. +manifest/head drift, completes the legacy `__run__` branch cleanup (MR-770), +accepts pretty-printed JSON load input, and updates the project governance and +release automation around those fixes. ## Highlights @@ -25,6 +26,15 @@ project governance and release automation around those fixes. - **Recovery roll-back convergence.** Recovery roll-back now aligns the manifest-visible version after restoring a table, closing the residual where Lance HEAD and `__manifest` could stay out of sync after recovery. +- **Legacy `__run__` branch cleanup (MR-770).** Completes the retirement of the + Run state machine (removed in v0.4.0). A one-time v2→v3 `__manifest` + internal-schema migration runs on the first read-write open and deletes any + stale `__run__*` staging branches left by pre-v0.4.0 graphs — they previously + leaked into `branch list` and counted as blocking branches at `schema apply` + time. The migration is idempotent, and the `is_internal_run_branch` guard + (and `run_registry.rs`) is retired now that `__run__*` is an ordinary branch + name. (The earlier v0.6.1 notes described this as shipped in v0.6.1; it + actually landed here in v0.6.2.) - **Pretty-printed JSON load input.** `load` accepts multi-line JSON objects in addition to one-object-per-line JSONL, so formatted fixture or export files no longer need to be minified before import. @@ -40,8 +50,12 @@ project governance and release automation around those fixes. `--force --confirm`. - `optimize` remains non-destructive. It still skips blob-bearing tables while OmniGraph is pinned to the Lance version with the blob-v2 compaction issue. -- No manual on-disk migration is required. Existing graphs open under v0.6.2; - the internal manifest schema stamp remains v3. +- No manual on-disk migration is required. Existing graphs open under v0.6.2. + Graphs already at internal manifest schema stamp v3 are unchanged; graphs + created before v0.4.0 that still carry the v2 stamp auto-migrate v2→v3 on the + first **read-write** open (the `__run__*` sweep above). The migration is + write-path-only, so a long-lived **read-only** deployment still lists any + stale `__run__*` branch until it is next opened read-write. ## Docs, Governance, And CI From 1f8e5945cfcaf99bca75992f51dc786122b237d7 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Tue, 9 Jun 2026 23:32:13 +0300 Subject: [PATCH 038/165] feat(cluster): config-only apply with content-addressed catalog publish apply_config_dir executes the query/policy subset of the plan: payloads are written content-addressed under __cluster/resources/{query,policy}/... before the state CAS (state is the publish point; orphaned blobs from a failed CAS are inert and re-apply is the repair), then state.json is CAS-updated with applied digests, Applied/Blocked statuses, and a revision bump. Graph/schema changes are never executed here: schema content and graph lifecycle defer to a later phase with loud warnings, while graph.<id> composite-digest updates whose schema component is unchanged converge automatically via recomputation from state's own components (without which apply could never converge). Idempotent re-apply leaves state bytes and revision untouched. PlanChange gains optional disposition/reason fields, populated by the same classifier in cluster plan, so plan is an honest preview of what apply will execute, derive, defer, or block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 1148 ++++++++++++++++++++++++++- 1 file changed, 1147 insertions(+), 1 deletion(-) diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 7ae824c..01ad171 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -21,6 +21,7 @@ pub const CLUSTER_GRAPHS_DIR: &str = "graphs"; pub const CLUSTER_STATE_DIR: &str = "__cluster"; pub const CLUSTER_STATE_FILE: &str = "__cluster/state.json"; pub const CLUSTER_LOCK_FILE: &str = "__cluster/lock.json"; +pub const CLUSTER_RESOURCES_DIR: &str = "__cluster/resources"; #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] #[serde(rename_all = "snake_case")] @@ -161,6 +162,23 @@ pub enum PlanOperation { Delete, } +/// How `cluster apply` treats a planned change in the current stage. +/// +/// `Applied` changes execute (config-only query/policy catalog writes). +/// `Derived` marks a `graph.<id>` composite-digest update that converges +/// automatically once its applied query digests land in state. `Deferred` +/// changes need a later phase (graph/schema lifecycle or schema content). +/// `Blocked` query/policy changes are gated by an unapplied or missing +/// dependency. +#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum ApplyDisposition { + Applied, + Derived, + Deferred, + Blocked, +} + #[derive(Debug, Clone, Serialize, PartialEq, Eq)] pub struct PlanChange { pub resource: String, @@ -169,6 +187,10 @@ pub struct PlanChange { pub before_digest: Option<String>, #[serde(skip_serializing_if = "Option::is_none")] pub after_digest: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub disposition: Option<ApplyDisposition>, + #[serde(skip_serializing_if = "Option::is_none")] + pub reason: Option<String>, } #[derive(Debug, Clone, Serialize, PartialEq, Eq)] @@ -236,6 +258,28 @@ pub struct ForceUnlockOutput { pub diagnostics: Vec<Diagnostic>, } +/// Output of config-only `cluster apply`. "Applied" means recorded in the +/// local cluster catalog (`__cluster/`); nothing applied here serves traffic — +/// the server still boots from `omnigraph.yaml` until the server-boot stage. +#[derive(Debug, Clone, Serialize)] +pub struct ApplyOutput { + pub ok: bool, + pub config_dir: String, + pub desired_revision: DesiredRevision, + pub state_observations: StateObservations, + /// Every planned change, with `disposition`/`reason` always populated. + pub changes: Vec<PlanChange>, + pub applied_count: usize, + /// Deferred + Blocked changes (Derived composite updates count as neither). + pub deferred_count: usize, + /// True when state matches the desired revision after this apply. + pub converged: bool, + /// False for a no-op re-apply: state bytes (and revision) were left untouched. + pub state_written: bool, + pub resource_statuses: BTreeMap<String, ResourceStatusRecord>, + pub diagnostics: Vec<Diagnostic>, +} + #[derive(Debug, Clone)] struct DesiredCluster { config_dir: PathBuf, @@ -477,11 +521,12 @@ pub fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { } } - let changes = if has_errors(&diagnostics) { + let mut changes = if has_errors(&diagnostics) { Vec::new() } else { diff_resources(&prior_resources, &desired.resource_digests) }; + classify_changes(&mut changes, &desired.dependencies); let blast_radius = compute_blast_radius(&changes, &desired.dependencies); let approvals_required = compute_approvals(&changes); let ok = !has_errors(&diagnostics); @@ -502,6 +547,317 @@ pub fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { } } +/// Config-only `cluster apply` (Stage 3A): execute the query/policy subset of +/// the plan against the local cluster catalog. The plan is recomputed under +/// the state lock, so freshness is structural; the state CAS inside +/// `write_state` is the second fence. Graph/schema changes are never executed +/// here — they are deferred to the graph-lifecycle phase and reported loudly. +/// +/// Payloads are content-addressed and written BEFORE the state CAS because +/// state is the publish point: a failure after payload writes leaves inert +/// digest-named blobs and no success acknowledgement; re-running apply is the +/// repair. +pub fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { + let outcome = load_desired(config_dir.as_ref()); + let mut diagnostics = outcome.diagnostics; + let backend = LocalStateBackend::new(&outcome.config_dir); + let mut observations = backend.observations(); + + let early_return = |config_dir: String, + config_digest: Option<String>, + observations: StateObservations, + changes: Vec<PlanChange>, + resource_statuses: BTreeMap<String, ResourceStatusRecord>, + diagnostics: Vec<Diagnostic>| { + ApplyOutput { + ok: !has_errors(&diagnostics), + config_dir, + desired_revision: DesiredRevision { + config_digest, + }, + state_observations: observations, + changes, + applied_count: 0, + deferred_count: 0, + converged: false, + state_written: false, + resource_statuses, + diagnostics, + } + }; + + let Some(desired) = outcome.desired else { + return early_return( + display_path(&outcome.config_dir), + None, + observations, + Vec::new(), + BTreeMap::new(), + diagnostics, + ); + }; + + if has_errors(&diagnostics) { + return early_return( + display_path(&desired.config_dir), + Some(desired.config_digest), + observations, + Vec::new(), + BTreeMap::new(), + diagnostics, + ); + } + + // Named guard: the lock must be held until the state outcome is recorded. + let _lock_guard = if desired.state_lock { + match backend.acquire_lock("apply", &mut observations) { + Ok(guard) => Some(guard), + Err(diagnostic) => { + diagnostics.push(diagnostic); + None + } + } + } else { + diagnostics.push(Diagnostic::warning( + "state_lock_disabled", + "state.lock", + "state.lock is false; apply wrote state without acquiring the cluster state lock", + )); + None + }; + + if has_errors(&diagnostics) { + return early_return( + display_path(&desired.config_dir), + Some(desired.config_digest), + observations, + Vec::new(), + BTreeMap::new(), + diagnostics, + ); + } + + let snapshot = match backend.read_state(&mut observations) { + Ok(snapshot) => snapshot, + Err(diagnostic) => { + diagnostics.push(diagnostic); + return early_return( + display_path(&desired.config_dir), + Some(desired.config_digest), + observations, + Vec::new(), + BTreeMap::new(), + diagnostics, + ); + } + }; + let expected_cas = snapshot.state_cas; + let Some(state) = snapshot.state else { + diagnostics.push(Diagnostic::error( + "state_missing", + CLUSTER_STATE_FILE, + "apply requires an existing state.json; run `cluster import` to bootstrap state", + )); + return early_return( + display_path(&desired.config_dir), + Some(desired.config_digest), + observations, + Vec::new(), + BTreeMap::new(), + diagnostics, + ); + }; + + let prior_resources = state_resource_digests(&state); + let mut changes = diff_resources(&prior_resources, &desired.resource_digests); + classify_changes(&mut changes, &desired.dependencies); + + // Defensive invariant: nothing the approval gate covers may be executable. + // Today approvals only cover graph/schema deletes (always deferred); this + // keeps a future widening of the executable set from silently bypassing it. + let approvals = compute_approvals(&changes); + let approval_violation = changes.iter().any(|change| { + change.disposition == Some(ApplyDisposition::Applied) + && approvals + .iter() + .any(|approval| approval.resource == change.resource) + }); + if approval_violation { + diagnostics.push(Diagnostic::error( + "apply_approval_invariant_violation", + "changes", + "an executable change requires approval; refusing to apply", + )); + return early_return( + display_path(&desired.config_dir), + Some(desired.config_digest), + observations, + changes, + state.resource_statuses, + diagnostics, + ); + } + + for change in &changes { + match change.disposition { + Some(ApplyDisposition::Deferred) => diagnostics.push(Diagnostic::warning( + "apply_unsupported_change", + change.resource.clone(), + "graph/schema changes are not applied in this stage; they are deferred to the graph-lifecycle phase", + )), + Some(ApplyDisposition::Blocked) => diagnostics.push(Diagnostic::warning( + "apply_dependency_blocked", + change.resource.clone(), + format!( + "blocked by an unapplied or missing dependency ({})", + change.reason.as_deref().unwrap_or("dependency") + ), + )), + _ => {} + } + } + + // Payload phase: content-addressed writes before the state CAS. Any + // failure aborts before state moves; blobs already written are inert. + let source_paths: BTreeMap<&str, &str> = desired + .resources + .iter() + .filter_map(|resource| { + resource + .path + .as_deref() + .map(|path| (resource.address.as_str(), path)) + }) + .collect(); + for change in &changes { + if change.disposition != Some(ApplyDisposition::Applied) + || change.operation == PlanOperation::Delete + { + continue; + } + let kind = resource_kind(&change.resource); + let digest = change + .after_digest + .as_deref() + .expect("create/update always carries an after digest"); + let Some(target) = payload_path(&desired.config_dir, &kind, digest) else { + continue; + }; + let Some(source) = source_paths.get(change.resource.as_str()) else { + diagnostics.push(Diagnostic::error( + "resource_payload_write_error", + change.resource.clone(), + "no source file recorded for resource", + )); + continue; + }; + if let Err(diagnostic) = + write_resource_payload(&target, Path::new(source), digest, &change.resource) + { + diagnostics.push(diagnostic); + } + } + if has_errors(&diagnostics) { + return early_return( + display_path(&desired.config_dir), + Some(desired.config_digest), + observations, + changes, + state.resource_statuses, + diagnostics, + ); + } + + // State mutation. Apply owns query/policy statuses only; graph/schema + // statuses belong to refresh/import observation and must not be clobbered. + let before_value = + serde_json::to_value(&state).expect("cluster state must serialize deterministically"); + let mut new_state = state.clone(); + for change in &changes { + match change.disposition { + Some(ApplyDisposition::Applied) => match change.operation { + PlanOperation::Create | PlanOperation::Update => { + new_state.applied_revision.resources.insert( + change.resource.clone(), + StateResource { + digest: change + .after_digest + .clone() + .expect("create/update always carries an after digest"), + }, + ); + set_resource_status_applied(&mut new_state, &change.resource); + } + PlanOperation::Delete => { + new_state.applied_revision.resources.remove(&change.resource); + new_state.resource_statuses.remove(&change.resource); + } + }, + Some(ApplyDisposition::Blocked) => { + set_resource_status( + &mut new_state, + &change.resource, + ResourceLifecycleStatus::Blocked, + change.reason.as_deref().unwrap_or("dependency_not_applied"), + "waiting on an unapplied or missing dependency", + ); + } + _ => {} + } + } + recompute_state_graph_digests(&mut new_state, &desired); + + let residual = diff_resources( + &state_resource_digests(&new_state), + &desired.resource_digests, + ); + let converged = residual.is_empty(); + if converged { + new_state.applied_revision.config_digest = Some(desired.config_digest.clone()); + } + + let after_value = + serde_json::to_value(&new_state).expect("cluster state must serialize deterministically"); + let mut state_written = false; + if after_value != before_value { + new_state.state_revision = new_state.state_revision.saturating_add(1); + match backend.write_state(&new_state, expected_cas.as_deref(), &mut observations) { + Ok(()) => state_written = true, + Err(diagnostic) => diagnostics.push(diagnostic), + } + } + + let applied_count = changes + .iter() + .filter(|change| change.disposition == Some(ApplyDisposition::Applied)) + .count(); + let deferred_count = changes + .iter() + .filter(|change| { + matches!( + change.disposition, + Some(ApplyDisposition::Deferred) | Some(ApplyDisposition::Blocked) + ) + }) + .count(); + + ApplyOutput { + ok: !has_errors(&diagnostics), + config_dir: display_path(&desired.config_dir), + desired_revision: DesiredRevision { + config_digest: Some(desired.config_digest), + }, + state_observations: observations, + changes, + applied_count, + deferred_count, + converged, + state_written, + resource_statuses: new_state.resource_statuses, + diagnostics, + } +} + pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { let parsed = parse_cluster_config(config_dir.as_ref()); let mut diagnostics = parsed.diagnostics; @@ -1797,12 +2153,16 @@ fn diff_resources( operation: PlanOperation::Create, before_digest: None, after_digest: Some(after.clone()), + disposition: None, + reason: None, }), Some(before) if before != after => changes.push(PlanChange { resource: address.clone(), operation: PlanOperation::Update, before_digest: Some(before.clone()), after_digest: Some(after.clone()), + disposition: None, + reason: None, }), Some(_) => {} } @@ -1814,6 +2174,8 @@ fn diff_resources( operation: PlanOperation::Delete, before_digest: Some(before.clone()), after_digest: None, + disposition: None, + reason: None, }); } } @@ -1855,6 +2217,249 @@ fn compute_approvals(changes: &[PlanChange]) -> Vec<ApprovalRequirement> { .collect() } +#[derive(Debug, PartialEq, Eq)] +enum ResourceKind { + Graph(String), + Schema(String), + Query { graph: String, name: String }, + Policy(String), + Unknown, +} + +fn resource_kind(address: &str) -> ResourceKind { + if let Some(graph) = address.strip_prefix("graph.") { + ResourceKind::Graph(graph.to_string()) + } else if let Some(graph) = address.strip_prefix("schema.") { + ResourceKind::Schema(graph.to_string()) + } else if let Some(rest) = address.strip_prefix("query.") { + match rest.split_once('.') { + Some((graph, name)) => ResourceKind::Query { + graph: graph.to_string(), + name: name.to_string(), + }, + None => ResourceKind::Unknown, + } + } else if let Some(name) = address.strip_prefix("policy.") { + ResourceKind::Policy(name.to_string()) + } else { + ResourceKind::Unknown + } +} + +/// Classify every planned change with the disposition config-only apply gives +/// it. Stage 3A executes only query/policy catalog writes; graph/schema +/// movement is a later phase, and `graph.<id>` composite updates whose schema +/// component is unchanged converge automatically once query digests land. +fn classify_changes(changes: &mut [PlanChange], dependencies: &[Dependency]) { + let mut schema_changed = BTreeSet::new(); + let mut graph_creates = BTreeSet::new(); + let mut graph_deletes = BTreeSet::new(); + for change in changes.iter() { + match resource_kind(&change.resource) { + ResourceKind::Schema(graph) => { + schema_changed.insert(graph); + } + ResourceKind::Graph(graph) => match change.operation { + PlanOperation::Create => { + graph_creates.insert(graph); + } + PlanOperation::Delete => { + graph_deletes.insert(graph); + } + PlanOperation::Update => {} + }, + _ => {} + } + } + + for change in changes.iter_mut() { + let (disposition, reason) = match resource_kind(&change.resource) { + ResourceKind::Schema(_) => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), + ResourceKind::Graph(graph) => match change.operation { + PlanOperation::Update if !schema_changed.contains(&graph) => { + (ApplyDisposition::Derived, None) + } + _ => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), + }, + ResourceKind::Query { graph, .. } => match change.operation { + PlanOperation::Delete => { + if graph_deletes.contains(&graph) { + ( + ApplyDisposition::Blocked, + Some("dependency_not_applied"), + ) + } else { + (ApplyDisposition::Applied, None) + } + } + PlanOperation::Create | PlanOperation::Update => { + // A missing graph is the more fundamental blocker than a + // pending schema change, so check it first. + if graph_creates.contains(&graph) { + (ApplyDisposition::Blocked, Some("dependency_missing")) + } else if schema_changed.contains(&graph) { + ( + ApplyDisposition::Blocked, + Some("dependency_not_applied"), + ) + } else { + (ApplyDisposition::Applied, None) + } + } + }, + ResourceKind::Policy(_) => match change.operation { + PlanOperation::Delete => (ApplyDisposition::Applied, None), + PlanOperation::Create | PlanOperation::Update => { + let blocked_dep = dependencies.iter().any(|dep| { + dep.from == change.resource + && dep + .to + .strip_prefix("graph.") + .is_some_and(|graph| graph_creates.contains(graph)) + }); + if blocked_dep { + (ApplyDisposition::Blocked, Some("dependency_missing")) + } else { + (ApplyDisposition::Applied, None) + } + } + }, + ResourceKind::Unknown => { + (ApplyDisposition::Deferred, Some("apply_unsupported_kind")) + } + }; + change.disposition = Some(disposition); + change.reason = reason.map(str::to_string); + } +} + +/// Content-addressed catalog path for an applied resource payload. Extensions +/// are fixed per kind (`.gq` / `.yaml`) regardless of the source file's name, +/// so the catalog layout cannot drift with operator file conventions. +fn payload_path(config_dir: &Path, kind: &ResourceKind, digest: &str) -> Option<PathBuf> { + let resources_dir = config_dir.join(CLUSTER_RESOURCES_DIR); + match kind { + ResourceKind::Query { graph, name } => Some( + resources_dir + .join("query") + .join(graph) + .join(name) + .join(format!("{digest}.gq")), + ), + ResourceKind::Policy(name) => Some( + resources_dir + .join("policy") + .join(name) + .join(format!("{digest}.yaml")), + ), + _ => None, + } +} + +/// Write one content-addressed payload blob. Idempotent: an existing +/// digest-named file is trusted as-is. The digest re-check is the apply-side +/// TOCTOU detector — the source file changing between `load_desired` and the +/// payload write must fail loudly, never publish mismatched content. +fn write_resource_payload( + target: &Path, + source: &Path, + expected_digest: &str, + resource: &str, +) -> Result<(), Diagnostic> { + if target.exists() { + return Ok(()); + } + let bytes = fs::read(source).map_err(|err| { + Diagnostic::error( + "resource_payload_write_error", + resource, + format!("could not read resource source '{}': {err}", source.display()), + ) + })?; + if sha256_hex(&bytes) != expected_digest { + return Err(Diagnostic::error( + "resource_content_changed", + resource, + format!( + "resource source '{}' changed while apply was running; re-run `cluster apply`", + source.display() + ), + )); + } + let parent = target.parent().expect("payload path always has a parent"); + fs::create_dir_all(parent).map_err(|err| { + Diagnostic::error( + "resource_payload_write_error", + resource, + format!("could not create payload directory: {err}"), + ) + })?; + let file_name = target + .file_name() + .expect("payload path always has a file name") + .to_string_lossy(); + let tmp_path = parent.join(format!("{file_name}.tmp.{}", Ulid::new())); + let mut file = OpenOptions::new() + .write(true) + .create_new(true) + .open(&tmp_path) + .map_err(|err| { + Diagnostic::error( + "resource_payload_write_error", + resource, + format!("could not create temporary payload file: {err}"), + ) + })?; + let write_result = file + .write_all(&bytes) + .and_then(|()| file.sync_all()) + .map_err(|err| { + Diagnostic::error( + "resource_payload_write_error", + resource, + format!("could not write payload file: {err}"), + ) + }); + drop(file); + if let Err(diagnostic) = write_result { + let _ = fs::remove_file(&tmp_path); + return Err(diagnostic); + } + if let Err(err) = fs::rename(&tmp_path, target) { + let _ = fs::remove_file(&tmp_path); + return Err(Diagnostic::error( + "resource_payload_write_error", + resource, + format!("could not move payload file into place: {err}"), + )); + } + Ok(()) +} + +/// Recompute the composite `graph.<id>` digests for state-resident graphs from +/// state's own schema/query components. Without this, an applied query change +/// would leave the prior composite digest in state and `graph.<id>` would show +/// a phantom update in every later plan — apply could never converge. +fn recompute_state_graph_digests(state: &mut ClusterState, desired: &DesiredCluster) { + for graph in &desired.graphs { + let graph_address = graph_address(&graph.id); + if !state.applied_revision.resources.contains_key(&graph_address) { + continue; + } + let schema_digest = state + .applied_revision + .resources + .get(&schema_address(&graph.id)) + .map(|resource| resource.digest.clone()); + let query_digests = state_query_digests_for_graph(state, &graph.id); + let digest = graph_digest(&graph.id, schema_digest.as_ref(), Some(&query_digests)); + state + .applied_revision + .resources + .insert(graph_address, StateResource { digest }); + } +} + fn duplicate_key_diagnostics(text: &str) -> Vec<Diagnostic> { #[derive(Debug)] struct Frame { @@ -3115,4 +3720,545 @@ graphs: ); assert!(!dir.path().join(CLUSTER_STATE_FILE).exists()); } + + // ---- config-only apply (Stage 3A) ---- + + /// Seed a state.json that simulates "graph exists with the desired schema, + /// queries/policies not yet applied" by borrowing the desired digests. + fn write_applyable_state(config_dir: &Path) { + let out = validate_config_dir(config_dir); + assert!(out.ok, "{:?}", out.diagnostics); + let schema_digest = out.resource_digests.get("schema.knowledge").unwrap().clone(); + let graph_composite = + graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); + write_state_resources( + config_dir, + &[ + ("graph.knowledge", graph_composite.as_str()), + ("schema.knowledge", schema_digest.as_str()), + ], + ); + } + + fn write_state_resources(config_dir: &Path, resources: &[(&str, &str)]) { + let resource_map: serde_json::Map<String, serde_json::Value> = resources + .iter() + .map(|(address, digest)| ((*address).to_string(), json!({ "digest": digest }))) + .collect(); + let state_dir = config_dir.join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + serde_json::to_string_pretty(&json!({ + "version": 1, + "state_revision": 1, + "applied_revision": { "resources": resource_map } + })) + .unwrap(), + ) + .unwrap(); + } + + fn read_state_json(config_dir: &Path) -> serde_json::Value { + serde_json::from_str(&fs::read_to_string(config_dir.join(CLUSTER_STATE_FILE)).unwrap()) + .unwrap() + } + + fn query_payload_path(config_dir: &Path, digest: &str) -> std::path::PathBuf { + config_dir + .join(CLUSTER_RESOURCES_DIR) + .join("query/knowledge/find_person") + .join(format!("{digest}.gq")) + } + + fn policy_payload_path(config_dir: &Path, digest: &str) -> std::path::PathBuf { + config_dir + .join(CLUSTER_RESOURCES_DIR) + .join("policy/base") + .join(format!("{digest}.yaml")) + } + + #[test] + fn apply_without_state_fails_with_state_missing() { + let dir = fixture(); + let out = apply_config_dir(dir.path()); + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_missing" + && diagnostic.message.contains("cluster import")) + ); + assert!(!dir.path().join(CLUSTER_STATE_FILE).exists()); + assert!(!dir.path().join(CLUSTER_RESOURCES_DIR).exists()); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn apply_writes_payloads_state_and_statuses() { + let dir = fixture(); + write_applyable_state(dir.path()); + let desired = validate_config_dir(dir.path()); + let query_digest = desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap() + .clone(); + let policy_digest = desired.resource_digests.get("policy.base").unwrap().clone(); + let schema_digest = desired + .resource_digests + .get("schema.knowledge") + .unwrap() + .clone(); + + let out = apply_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.applied_count, 2); + assert_eq!(out.deferred_count, 0); + assert!(out.converged); + assert!(out.state_written); + + let query_blob = query_payload_path(dir.path(), &query_digest); + assert_eq!(fs::read_to_string(&query_blob).unwrap(), QUERY); + let policy_blob = policy_payload_path(dir.path(), &policy_digest); + assert_eq!(fs::read_to_string(&policy_blob).unwrap(), "rules: []\n"); + + let state = read_state_json(dir.path()); + assert_eq!(state["state_revision"], 2); + let resources = &state["applied_revision"]["resources"]; + assert_eq!( + resources["query.knowledge.find_person"]["digest"], + query_digest + ); + assert_eq!(resources["policy.base"]["digest"], policy_digest); + let expected_composite = graph_digest( + "knowledge", + Some(&schema_digest), + Some( + &[("find_person".to_string(), query_digest.clone())] + .into_iter() + .collect(), + ), + ); + assert_eq!(resources["graph.knowledge"]["digest"], expected_composite); + assert_eq!( + state["applied_revision"]["config_digest"], + desired_revision_digest(&out) + ); + assert_eq!( + state["resource_statuses"]["query.knowledge.find_person"]["status"], + "applied" + ); + assert_eq!(state["resource_statuses"]["policy.base"]["status"], "applied"); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + fn desired_revision_digest(out: &ApplyOutput) -> String { + out.desired_revision.config_digest.clone().unwrap() + } + + #[test] + fn apply_update_changes_query_digest_and_keeps_old_blob() { + let dir = fixture(); + let desired = validate_config_dir(dir.path()); + let schema_digest = desired + .resource_digests + .get("schema.knowledge") + .unwrap() + .clone(); + let old_digest = "0".repeat(64); + let graph_composite = + graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", graph_composite.as_str()), + ("schema.knowledge", schema_digest.as_str()), + ("query.knowledge.find_person", old_digest.as_str()), + ], + ); + let old_blob = query_payload_path(dir.path(), &old_digest); + fs::create_dir_all(old_blob.parent().unwrap()).unwrap(); + fs::write(&old_blob, "old query source").unwrap(); + + let out = apply_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + let new_digest = desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap(); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["query.knowledge.find_person"]["digest"], + *new_digest + ); + assert_eq!(fs::read_to_string(&old_blob).unwrap(), "old query source"); + assert!(query_payload_path(dir.path(), new_digest).exists()); + } + + #[test] + fn apply_deletes_removed_resources_but_keeps_blobs() { + let dir = fixture(); + let desired = validate_config_dir(dir.path()); + let schema_digest = desired + .resource_digests + .get("schema.knowledge") + .unwrap() + .clone(); + let stale_query_digest = "1".repeat(64); + let stale_policy_digest = "2".repeat(64); + let graph_composite = + graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", graph_composite.as_str()), + ("schema.knowledge", schema_digest.as_str()), + ("query.knowledge.orphan", stale_query_digest.as_str()), + ("policy.old", stale_policy_digest.as_str()), + ], + ); + let stale_blob = dir + .path() + .join(CLUSTER_RESOURCES_DIR) + .join("policy/old") + .join(format!("{stale_policy_digest}.yaml")); + fs::create_dir_all(stale_blob.parent().unwrap()).unwrap(); + fs::write(&stale_blob, "old policy").unwrap(); + + let out = apply_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.converged); + let state = read_state_json(dir.path()); + let resources = &state["applied_revision"]["resources"]; + assert!(resources.get("query.knowledge.orphan").is_none()); + assert!(resources.get("policy.old").is_none()); + assert!( + state["resource_statuses"] + .get("query.knowledge.orphan") + .is_none() + ); + // Deleted resources leave their content-addressed blobs in place; GC is + // a later stage. + assert_eq!(fs::read_to_string(&stale_blob).unwrap(), "old policy"); + // The composite no longer includes the orphan query. + let query_digest = desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap() + .clone(); + let expected_composite = graph_digest( + "knowledge", + Some(&schema_digest), + Some(&[("find_person".to_string(), query_digest)].into_iter().collect()), + ); + assert_eq!(resources["graph.knowledge"]["digest"], expected_composite); + } + + #[test] + fn apply_defers_schema_change_and_blocks_dependent_query() { + let dir = fixture(); + write_applyable_state(dir.path()); + // Change the schema after seeding state: schema.knowledge now differs. + fs::write( + dir.path().join("people.pg"), + "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n", + ) + .unwrap(); + + let out = apply_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!out.converged); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + assert_eq!( + by_resource["schema.knowledge"].disposition, + Some(ApplyDisposition::Deferred) + ); + assert_eq!( + by_resource["graph.knowledge"].disposition, + Some(ApplyDisposition::Deferred) + ); + assert_eq!( + by_resource["query.knowledge.find_person"].disposition, + Some(ApplyDisposition::Blocked) + ); + assert_eq!( + by_resource["query.knowledge.find_person"].reason.as_deref(), + Some("dependency_not_applied") + ); + // Policy is independent of the schema and still applies. + assert_eq!( + by_resource["policy.base"].disposition, + Some(ApplyDisposition::Applied) + ); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "apply_unsupported_change") + ); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "apply_dependency_blocked") + ); + + let state = read_state_json(dir.path()); + assert_eq!( + state["resource_statuses"]["query.knowledge.find_person"]["status"], + "blocked" + ); + // The blocked query wrote no payload and no state digest. + assert!( + state["applied_revision"]["resources"] + .get("query.knowledge.find_person") + .is_none() + ); + assert!( + !dir.path() + .join(CLUSTER_RESOURCES_DIR) + .join("query") + .exists() + ); + // Not converged: the applied config digest must not be claimed. + assert!( + state["applied_revision"] + .get("config_digest") + .is_none_or(serde_json::Value::is_null) + ); + } + + #[test] + fn apply_blocks_resources_of_uncreated_graph() { + let dir = fixture(); + write_state_resources(dir.path(), &[]); + + let out = apply_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.applied_count, 0); + assert!(!out.converged); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + assert_eq!( + by_resource["graph.knowledge"].disposition, + Some(ApplyDisposition::Deferred) + ); + assert_eq!( + by_resource["query.knowledge.find_person"].reason.as_deref(), + Some("dependency_missing") + ); + assert_eq!( + by_resource["policy.base"].reason.as_deref(), + Some("dependency_missing") + ); + // Statuses for blocked resources are recorded (state changed), but no + // resource digests moved. + let state = read_state_json(dir.path()); + assert_eq!(state["state_revision"], 2); + assert!( + state["applied_revision"]["resources"] + .as_object() + .unwrap() + .is_empty() + ); + assert_eq!( + state["resource_statuses"]["policy.base"]["status"], + "blocked" + ); + } + + #[test] + fn apply_does_not_delete_subtree_of_deleted_graph() { + let dir = fixture(); + let desired = validate_config_dir(dir.path()); + let schema_digest = desired + .resource_digests + .get("schema.knowledge") + .unwrap() + .clone(); + let graph_composite = + graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", graph_composite.as_str()), + ("schema.knowledge", schema_digest.as_str()), + ("graph.old", "3333"), + ("schema.old", "4444"), + ("query.old.q", "5555"), + ], + ); + + let out = apply_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!out.converged); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + assert_eq!( + by_resource["graph.old"].disposition, + Some(ApplyDisposition::Deferred) + ); + assert_eq!( + by_resource["schema.old"].disposition, + Some(ApplyDisposition::Deferred) + ); + assert_eq!( + by_resource["query.old.q"].disposition, + Some(ApplyDisposition::Blocked) + ); + let state = read_state_json(dir.path()); + let resources = &state["applied_revision"]["resources"]; + assert_eq!(resources["graph.old"]["digest"], "3333"); + assert_eq!(resources["schema.old"]["digest"], "4444"); + assert_eq!(resources["query.old.q"]["digest"], "5555"); + } + + #[test] + fn apply_is_idempotent() { + let dir = fixture(); + write_applyable_state(dir.path()); + + let first = apply_config_dir(dir.path()); + assert!(first.ok, "{:?}", first.diagnostics); + assert!(first.state_written); + let state_after_first = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); + + let second = apply_config_dir(dir.path()); + assert!(second.ok, "{:?}", second.diagnostics); + assert!(second.changes.is_empty()); + assert_eq!(second.applied_count, 0); + assert!(second.converged); + assert!(!second.state_written); + let state_after_second = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); + assert_eq!(state_after_first, state_after_second); + assert_eq!(second.state_observations.state_revision, 2); + } + + #[test] + fn apply_respects_held_lock() { + let dir = fixture(); + write_applyable_state(dir.path()); + write_lock_file(dir.path(), "held-lock", "plan"); + + let out = apply_config_dir(dir.path()); + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_held") + ); + // The held lock survives a refused apply, and nothing was written. + assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); + assert!(!dir.path().join(CLUSTER_RESOURCES_DIR).exists()); + let state = read_state_json(dir.path()); + assert_eq!(state["state_revision"], 1); + } + + #[test] + fn apply_state_lock_false_bypasses_with_warning() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +state: + backend: cluster + lock: false +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +"#, + ) + .unwrap(); + write_applyable_state(dir.path()); + + let out = apply_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.state_written); + assert!(!out.state_observations.lock_acquired); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_disabled") + ); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn apply_skips_existing_payload_blob() { + let dir = fixture(); + write_applyable_state(dir.path()); + let desired = validate_config_dir(dir.path()); + let query_digest = desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap() + .clone(); + // Content-addressed blobs are trusted by name: an existing file is + // never rewritten. + let blob = query_payload_path(dir.path(), &query_digest); + fs::create_dir_all(blob.parent().unwrap()).unwrap(); + fs::write(&blob, "pre-existing").unwrap(); + + let out = apply_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(fs::read_to_string(&blob).unwrap(), "pre-existing"); + } + + #[test] + fn apply_invalid_config_fails_before_lock() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\nnot_a_field: true\n", + ) + .unwrap(); + + let out = apply_config_dir(dir.path()); + assert!(!out.ok); + // Config errors bail before the lock or any state directory exists. + assert!(!dir.path().join(CLUSTER_STATE_DIR).exists()); + } + + #[test] + fn plan_annotates_apply_dispositions() { + let dir = fixture(); + let out = plan_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + // Empty state: graph/schema creates are deferred, query/policy blocked + // on the uncreated graph — and plan says so before apply runs. + assert_eq!( + by_resource["graph.knowledge"].disposition, + Some(ApplyDisposition::Deferred) + ); + assert_eq!( + by_resource["schema.knowledge"].disposition, + Some(ApplyDisposition::Deferred) + ); + assert_eq!( + by_resource["query.knowledge.find_person"].disposition, + Some(ApplyDisposition::Blocked) + ); + assert_eq!( + by_resource["policy.base"].reason.as_deref(), + Some("dependency_missing") + ); + } } From bcef8444dd59cfdecd29c7f7a6b845b8a705423f Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Tue, 9 Jun 2026 23:34:48 +0300 Subject: [PATCH 039/165] feat(cli): omnigraph cluster apply MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Terraform-style: apply executes directly (cluster plan is the preview, now annotated with apply dispositions). Human output prints per-change dispositions, convergence, and the catalog-only caveat; --json emits the full ApplyOutput. Exit is non-zero only on errors — deferred/blocked changes are warnings with converged: false as the automation signal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 65 +++++++++++++- crates/omnigraph-cli/tests/cli.rs | 136 ++++++++++++++++++++++++++++++ 2 files changed, 199 insertions(+), 2 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 971ffff..42bbed8 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -11,8 +11,8 @@ use omnigraph::db::{Omnigraph, ReadTarget, SnapshotId}; use omnigraph::loader::LoadMode; use omnigraph::storage::normalize_root_uri; use omnigraph_cluster::{ - DiagnosticSeverity, ForceUnlockOutput, PlanOutput, StateSyncOutput, StatusOutput, - ValidateOutput, force_unlock_config_dir, import_config_dir, plan_config_dir, + ApplyOutput, DiagnosticSeverity, ForceUnlockOutput, PlanOutput, StateSyncOutput, StatusOutput, + ValidateOutput, apply_config_dir, force_unlock_config_dir, import_config_dir, plan_config_dir, refresh_config_dir, status_config_dir, validate_config_dir, }; use omnigraph_compiler::query::parser::parse_query; @@ -361,6 +361,16 @@ enum ClusterCommand { #[arg(long)] json: bool, }, + /// Apply the config-only (query/policy) subset of the plan to the local + /// cluster catalog. Graph/schema changes are deferred to a later stage. + Apply { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, /// Read the local JSON state ledger without scanning live graph resources. Status { /// Cluster config directory containing cluster.yaml. @@ -804,6 +814,40 @@ fn print_cluster_plan_human(output: &PlanOutput) { print_cluster_diagnostics(&output.diagnostics); } +fn print_cluster_apply_human(output: &ApplyOutput) { + if output.ok { + println!( + "cluster apply: {} applied, {} deferred/blocked", + output.applied_count, output.deferred_count + ); + for change in &output.changes { + match (&change.disposition, change.reason.as_deref()) { + (Some(disposition), Some(reason)) => println!( + " {:?} {} [{disposition:?}: {reason}]", + change.operation, change.resource + ), + (Some(disposition), None) => println!( + " {:?} {} [{disposition:?}]", + change.operation, change.resource + ), + _ => println!(" {:?} {}", change.operation, change.resource), + } + } + if output.changes.is_empty() { + println!(" no changes"); + } + let state = &output.state_observations; + println!( + " state: revision {}, converged: {}, written: {}", + state.state_revision, output.converged, output.state_written + ); + println!(" note: applied = recorded in the cluster catalog; the server still boots from omnigraph.yaml"); + } else { + println!("cluster apply failed"); + } + print_cluster_diagnostics(&output.diagnostics); +} + fn print_cluster_status_human(output: &StatusOutput) { if output.ok { let state = &output.state_observations; @@ -935,6 +979,19 @@ fn finish_cluster_plan(output: &PlanOutput, json: bool) -> Result<()> { Ok(()) } +fn finish_cluster_apply(output: &ApplyOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_apply_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + fn finish_cluster_status(output: &StatusOutput, json: bool) -> Result<()> { if json { print_json(output)?; @@ -3492,6 +3549,10 @@ async fn main() -> Result<()> { let output = plan_config_dir(config); finish_cluster_plan(&output, json)?; } + ClusterCommand::Apply { config, json } => { + let output = apply_config_dir(config); + finish_cluster_apply(&output, json)?; + } ClusterCommand::Status { config, json } => { let output = status_config_dir(config); finish_cluster_status(&output, json)?; diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 1dd26a7..9dbf250 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -754,6 +754,142 @@ fn cluster_validate_invalid_config_exits_nonzero() { assert!(stdout.contains("future_phase_field"), "{stdout}"); } +/// Seed an applyable state: schema digest borrowed from `cluster validate`, +/// graph entry present (composite recomputed by apply), queries/policies +/// pending. +fn write_cluster_applyable_state(root: &std::path::Path) -> serde_json::Value { + let validate = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("validate") + .arg("--config") + .arg(root) + .arg("--json"), + )); + let schema_digest = validate["resource_digests"]["schema.knowledge"] + .as_str() + .unwrap() + .to_string(); + let state_dir = root.join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + format!( + r#"{{ + "version": 1, + "state_revision": 1, + "applied_revision": {{ + "resources": {{ + "graph.knowledge": {{ "digest": "seed" }}, + "schema.knowledge": {{ "digest": "{schema_digest}" }} + }} + }} +}} +"# + ), + ) + .unwrap(); + validate +} + +#[test] +fn cluster_apply_json_applies_query_and_policy() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let validate = write_cluster_applyable_state(temp.path()); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true, "{json}"); + assert_eq!(json["applied_count"], 2, "{json}"); + assert_eq!(json["converged"], true, "{json}"); + assert_eq!(json["state_written"], true, "{json}"); + assert_eq!( + json["resource_statuses"]["query.knowledge.find_person"]["status"], + "applied" + ); + + let query_digest = validate["resource_digests"]["query.knowledge.find_person"] + .as_str() + .unwrap(); + let payload = temp + .path() + .join("__cluster/resources/query/knowledge/find_person") + .join(format!("{query_digest}.gq")); + assert!(payload.exists(), "missing payload {}", payload.display()); + + let state: serde_json::Value = serde_json::from_str( + &fs::read_to_string(temp.path().join("__cluster/state.json")).unwrap(), + ) + .unwrap(); + assert_eq!(state["state_revision"], 2); + assert_eq!( + state["applied_revision"]["resources"]["query.knowledge.find_person"]["digest"], + *query_digest + ); +} + +#[test] +fn cluster_apply_missing_state_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let output = output_failure( + cli() + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + ); + let json = parse_stdout_json(&output); + assert_eq!(json["ok"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_missing"), + "{json}" + ); + assert!(!temp.path().join("__cluster/resources").exists()); +} + +#[test] +fn cluster_apply_locked_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_applyable_state(temp.path()); + write_cluster_lock(temp.path(), "held-lock", "plan"); + + let output = output_failure( + cli() + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + ); + let json = parse_stdout_json(&output); + assert_eq!(json["ok"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_lock_held"), + "{json}" + ); + assert!(temp.path().join("__cluster/lock.json").exists()); + assert!(!temp.path().join("__cluster/resources").exists()); +} + #[test] fn short_version_flag_prints_current_cli_version() { let output = output_success(cli().arg("-v")); From 40a21e4e77e50192902818f2e5a93b1dd42a54fe Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Tue, 9 Jun 2026 23:36:33 +0300 Subject: [PATCH 040/165] docs(cluster): document Stage 3A config-only cluster apply Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/testing.md | 2 +- docs/user/cli-reference.md | 17 ++++++---- docs/user/cluster-config.md | 67 ++++++++++++++++++++++++++++++++----- 3 files changed, 70 insertions(+), 16 deletions(-) diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 3c5ee32..1f818e9 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,7 +8,7 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | -| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests` | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests` | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, and config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply) | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 70ac6f4..774ea6b 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -19,7 +19,7 @@ Top-level command families and subcommands. Graph-targeting commands accept eith | `commit list \| show` | inspect commit graph | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | -| `cluster validate \| plan \| status \| refresh \| import \| force-unlock` | cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`; `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id. No apply, graph-resource mutation, server change, automatic stale-lock breaking, or `plan --refresh` occurs in Stage 2C | +| `cluster validate \| plan \| apply \| status \| refresh \| import \| force-unlock` | cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json` and annotates each change with its apply disposition; `apply` executes the config-only (stored-query/policy) subset into the content-addressed local catalog under `__cluster/resources/` — graph/schema changes are deferred loudly, and nothing applied serves traffic (the server still boots from `omnigraph.yaml`); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id. No graph-manifest movement, server change, automatic stale-lock breaking, or `plan --refresh` occurs in Stage 3A | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) | | `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | @@ -78,6 +78,7 @@ policy: ```bash omnigraph cluster validate --config ./company-brain omnigraph cluster plan --config ./company-brain --json +omnigraph cluster apply --config ./company-brain --json omnigraph cluster status --config ./company-brain --json omnigraph cluster refresh --config ./company-brain --json omnigraph cluster import --config ./company-brain --json @@ -85,16 +86,20 @@ omnigraph cluster force-unlock <LOCK_ID> --config ./company-brain --json ``` `--config` is a directory containing `cluster.yaml`; it defaults to `.`. -Stage 2C accepts graphs, schemas, stored queries, and policy bundle file +Stage 3A accepts graphs, schemas, stored queries, and policy bundle file references. `cluster plan` reads local JSON state from `<config-dir>/__cluster/state.json`; a missing file means empty state. Plan, -refresh, and import acquire `__cluster/lock.json` by default and release it -before returning. `cluster status` reads state only and reports any existing +apply, refresh, and import acquire `__cluster/lock.json` by default and release +it before returning. `cluster apply` executes only stored-query/policy catalog +writes (content-addressed under `__cluster/resources/`) and requires an +existing `state.json`; graph/schema changes are deferred with warnings, and +applied resources do not serve traffic — the server still boots from +`omnigraph.yaml`. `cluster status` reads state only and reports any existing lock metadata. `force-unlock` removes a lock only when the supplied id exactly matches the lock file. `refresh` requires an existing `state.json`; `import` creates one only when it is missing. Both observe declared graphs read-only at -`<config-dir>/graphs/<graph-id>.omni`. External state backends, apply, -automatic stale-lock breaking, `plan --refresh`, pipelines, UI specs, +`<config-dir>/graphs/<graph-id>.omni`. External state backends, graph/schema +apply, automatic stale-lock breaking, `plan --refresh`, pipelines, UI specs, embeddings, aliases, and bindings are reserved for later stages. See [cluster-config.md](cluster-config.md). diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 24718b1..b285cf3 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -1,19 +1,23 @@ # Cluster Config -**Status:** Stage 2C state-lock recovery preview. +**Status:** Stage 3A config-only apply preview. Cluster config is the future control-plane configuration surface for a whole OmniGraph deployment. In this stage, OmniGraph can validate a local `cluster.yaml` folder, produce a deterministic read-only plan, inspect the -local JSON state ledger, and explicitly refresh/import graph observations into -that ledger. It can also manually remove a held local state lock by exact lock -id. It does not apply desired changes, start servers, or write graph resources. +local JSON state ledger, explicitly refresh/import graph observations into +that ledger, manually remove a held local state lock by exact lock id, and +**apply the config-only subset of the plan** — stored-query and policy-bundle +catalog writes. It does not move graph manifests, change schemas, start +servers, or serve anything it applies: the server still boots from +`omnigraph.yaml`. ## Commands ```bash omnigraph cluster validate --config ./company-brain omnigraph cluster plan --config ./company-brain --json +omnigraph cluster apply --config ./company-brain --json omnigraph cluster status --config ./company-brain --json omnigraph cluster refresh --config ./company-brain --json omnigraph cluster import --config ./company-brain --json @@ -51,9 +55,9 @@ policies: `metadata.name` is a display label. `state.backend` may be omitted or set to `cluster`; external state backends are reserved for a later stage. `state.lock` -defaults to `true`. When enabled, `cluster plan`, `cluster refresh`, and -`cluster import` briefly acquire `<config-dir>/__cluster/lock.json`, then remove -it before returning. `cluster status` never acquires the lock; it only reports +defaults to `true`. When enabled, `cluster plan`, `cluster apply`, +`cluster refresh`, and `cluster import` briefly acquire +`<config-dir>/__cluster/lock.json`, then remove it before returning. `cluster status` never acquires the lock; it only reports whether one is present. `cluster force-unlock` is the only lock-removal command; it requires the exact lock id and should be run only after confirming no cluster operation is active. @@ -125,8 +129,53 @@ successful `plan` instead reports `lock_acquired: true` and an `acquired_lock_id`, then releases the lock before returning. The command never writes `state.json` and does not scan live graphs. Use explicit `cluster refresh` / `cluster import` when the state ledger should be updated -from live observations. Apply and live drift scans during plan are later-stage -work. +from live observations. Live drift scans during plan are later-stage work. + +Each plan change carries a `disposition` field — an honest preview of what +`cluster apply` will do with it in this stage: `applied` (executes), `derived` +(a `graph.<id>` composite-digest update that converges automatically once its +query digests land), `deferred` (graph/schema change, later phase), or +`blocked` (query/policy gated by an unapplied or missing dependency, with the +condition in `reason`). + +## Apply + +`cluster apply` executes the config-only subset of the plan — stored-query and +policy-bundle changes. There is no confirm flag: `cluster plan` is the preview, +and apply recomputes the same diff under the state lock before executing, so a +stale preview can never be applied. Apply requires an existing `state.json` +(`state_missing` directs you to `cluster import` first). + +For each applied create/update, the resource payload is written +content-addressed into the local catalog: + +```text +<config-dir>/__cluster/resources/query/<graph>/<name>/<digest>.gq +<config-dir>/__cluster/resources/policy/<name>/<digest>.yaml +``` + +Extensions are fixed per kind regardless of the source file's name. Payloads +are written before the state update because `state.json` is the publish point: +if the final CAS-checked state write fails, no success is reported and the +digest-named blobs already written are inert — re-running apply is the repair. +Deletes remove the resource from state; their old payload blobs stay on disk +(garbage collection is a later stage). Re-running a converged apply is a no-op: +no state write, no revision change (`state_written: false`). + +**Applied means recorded in the cluster catalog — nothing more.** The server +still boots from `omnigraph.yaml`; no query or policy applied here serves +traffic until the server-boot stage ships, as an explicit per-deployment mode +switch. + +Graph and schema changes are never executed by this stage. They are reported +as `deferred` (warning `apply_unsupported_change`), and query/policy changes +that depend on them are `blocked` (warning `apply_dependency_blocked`, status +`blocked` in state). A partially-applicable plan still exits 0 with warnings; +the JSON `converged` field is the automation signal for "state now matches the +desired revision". The applied `config_digest` is only recorded when apply +fully converges. The `graph.<id>` composite digest is recomputed from state's +own schema/query digests after each apply, so applied query changes converge +without graph movement. ## Status From d870eaaf3ff3d083061a799f4cf3ec1023b02cbb Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Tue, 9 Jun 2026 23:44:49 +0300 Subject: [PATCH 041/165] =?UTF-8?q?test(cli):=20cluster=20lifecycle=20e2e?= =?UTF-8?q?=20=E2=80=94=20real-graph=20import/apply/refresh,=20schema-chan?= =?UTF-8?q?ge=20loop,=20force-unlock=20retry?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three composition tests over the spawned binary against a real derived graph: - import -> plan (dispositions) -> apply -> status -> refresh -> plan-empty, then a query edit round-trip. Pins that refresh and apply recompute the graph composite digest identically — divergence would silently re-open the plan forever and no single-command test would catch it. - The Stage 3A operator workflow across the control/data-plane boundary: cluster apply defers a schema change, omnigraph schema apply executes it, cluster refresh observes it, the next cluster apply re-converges. - Held lock refuses apply, force-unlock clears it, retried apply converges. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/cli.rs | 183 ++++++++++++++++++++++++++++++ 1 file changed, 183 insertions(+) diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 9dbf250..30fa796 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -890,6 +890,189 @@ fn cluster_apply_locked_exits_nonzero() { assert!(!temp.path().join("__cluster/resources").exists()); } +fn cluster_json(root: &std::path::Path, command: &str) -> serde_json::Value { + parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg(command) + .arg("--config") + .arg(root) + .arg("--json"), + )) +} + +/// End-to-end lifecycle against a REAL derived graph: import observes the live +/// graph, plan/apply converge the query+policy catalog, status reports it, +/// refresh re-observes without un-converging, and a query edit round-trips. +/// This is the composition test — every step passes individually elsewhere; +/// this catches the seams (e.g. refresh and apply recomputing the graph +/// composite digest differently would silently re-open the plan forever). +#[test] +fn cluster_e2e_lifecycle_import_apply_status_refresh_converges() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + assert_eq!(import["state_observations"]["state_revision"], 1); + + let plan = cluster_json(temp.path(), "plan"); + let changes = plan["changes"].as_array().unwrap(); + assert_eq!(changes.len(), 3, "{plan}"); + let disposition_of = |resource: &str| { + changes + .iter() + .find(|change| change["resource"] == resource) + .unwrap_or_else(|| panic!("missing change for {resource}: {plan}"))["disposition"] + .clone() + }; + assert_eq!(disposition_of("graph.knowledge"), "derived"); + assert_eq!(disposition_of("query.knowledge.find_person"), "applied"); + assert_eq!(disposition_of("policy.base"), "applied"); + + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["ok"], true, "{apply}"); + assert_eq!(apply["applied_count"], 2, "{apply}"); + assert_eq!(apply["converged"], true, "{apply}"); + + let status = cluster_json(temp.path(), "status"); + assert_eq!( + status["resource_statuses"]["query.knowledge.find_person"]["status"], + "applied" + ); + assert_eq!(status["resource_statuses"]["policy.base"]["status"], "applied"); + assert!( + status["state_observations"]["applied_config_digest"].is_string(), + "converged apply must record the applied config digest: {status}" + ); + + // Refresh re-observes the live graph; it must not undo apply's work. + let refresh = cluster_json(temp.path(), "refresh"); + assert_eq!(refresh["ok"], true, "{refresh}"); + let replan = cluster_json(temp.path(), "plan"); + assert!( + replan["changes"].as_array().unwrap().is_empty(), + "refresh after a converged apply must not re-open the plan: {replan}" + ); + + // A query edit round-trips: plan update -> apply -> converged again. + fs::write( + temp.path().join("people.gq"), + r#" +query find_person($name: String) { + match { $p: Person { name: $name } } + return { $p.name } +} +"#, + ) + .unwrap(); + let apply_edit = cluster_json(temp.path(), "apply"); + assert_eq!(apply_edit["applied_count"], 1, "{apply_edit}"); + assert_eq!(apply_edit["converged"], true, "{apply_edit}"); + + let final_apply = cluster_json(temp.path(), "apply"); + assert_eq!(final_apply["state_written"], false, "{final_apply}"); + assert!(final_apply["changes"].as_array().unwrap().is_empty()); +} + +/// The operator workflow across the Stage 3A boundary: a schema change is +/// deferred by cluster apply, executed by `omnigraph schema apply` against +/// the graph, picked up by `cluster refresh`, and the next apply re-converges. +#[test] +fn cluster_e2e_schema_change_defers_until_schema_apply_and_refresh() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["converged"], true, "{apply}"); + + // Additive schema change: cluster apply must defer it loudly, not act. + fs::write( + temp.path().join("people.pg"), + r#" +node Person { + name: String @key + age: I32? + bio: String? +} +"#, + ) + .unwrap(); + let deferred = cluster_json(temp.path(), "apply"); + assert_eq!(deferred["ok"], true, "{deferred}"); + assert_eq!(deferred["applied_count"], 0, "{deferred}"); + assert_eq!(deferred["converged"], false, "{deferred}"); + assert!( + deferred["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "apply_unsupported_change"), + "{deferred}" + ); + + // The graph-plane tool applies the migration... + output_success( + cli() + .arg("schema") + .arg("apply") + .arg(temp.path().join("graphs/knowledge.omni")) + .arg("--schema") + .arg(temp.path().join("people.pg")) + .arg("--json"), + ); + // ...refresh observes it... + let refresh = cluster_json(temp.path(), "refresh"); + assert_eq!(refresh["ok"], true, "{refresh}"); + // ...and the control plane re-converges. + let reconverge = cluster_json(temp.path(), "apply"); + assert_eq!(reconverge["ok"], true, "{reconverge}"); + assert_eq!(reconverge["converged"], true, "{reconverge}"); + let replan = cluster_json(temp.path(), "plan"); + assert!( + replan["changes"].as_array().unwrap().is_empty(), + "after schema apply + refresh + apply, the plan must be empty: {replan}" + ); +} + +/// Lock-recovery composition: a held lock refuses apply, force-unlock clears +/// it, and the retried apply converges. +#[test] +fn cluster_e2e_force_unlock_unblocks_apply() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_applyable_state(temp.path()); + write_cluster_lock(temp.path(), "stuck-lock", "apply"); + + let refused = parse_stdout_json(&output_failure( + cli() + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(refused["ok"], false); + + let unlocked = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("force-unlock") + .arg("stuck-lock") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(unlocked["lock_removed"], true, "{unlocked}"); + + let retried = cluster_json(temp.path(), "apply"); + assert_eq!(retried["ok"], true, "{retried}"); + assert_eq!(retried["converged"], true, "{retried}"); +} + #[test] fn short_version_flag_prints_current_cli_version() { let output = output_success(cli().arg("-v")); From 2c578a60b26b45e8dc2358219a674ce95cc61c76 Mon Sep 17 00:00:00 2001 From: "devin-ai-integration[bot]" <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Tue, 9 Jun 2026 23:03:08 +0200 Subject: [PATCH 042/165] (feat) convert engine call sites to &dyn TableStorage; demote legacy TableStore methods to pub(crate) (#86) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * MR-854: convert engine call sites to &dyn TableStorage; demote legacy methods Phase 1b: every db.table_store.X(...) call site converts to db.storage().X(...), reaching the storage layer through the sealed TableStorage trait (returns &dyn TableStorage). Opaque SnapshotHandle and StagedHandle replace bare lance::Dataset and Transaction in the threaded values. Phase 9: the inherent inline-commit methods on TableStore (append_batch, merge_insert_batch{,es}, overwrite_batch, create_btree_index, create_inverted_index) demote from pub to pub(crate). Their only remaining direct users are table_store.rs itself and the bulk loader's LoadMode::{Append, Overwrite, Merge} concurrent fast-paths in loader::write_batch_to_dataset (no two-phase shape in Lance 4.0.0 — closes after lance#6658 and #6666). Docs: - invariants.md \u00a7VI.23: drop "at the writer-trait surface" qualifier; staged primitives are now the only engine surface. - runs.md: residual matrix shrinks to delete_where and create_vector_index (the two upstream-blocked residuals). - forbidden_apis.rs: replace transitional language with the current allow-list shape (table_store.rs + loader concurrent fast-path only). Files touched: - changes/mod.rs, db/omnigraph.rs (+export/optimize/schema_apply/ table_ops.rs), exec/{merge,mod,mutation,staging}.rs, loader/mod.rs, storage_layer.rs, table_store.rs, tests/forbidden_apis.rs, docs/{invariants,runs}.md. Co-Authored-By: Ragnor Comerford <ragnor.comerford@gmail.com> * MR-854: replace test-only inline-commit append callers with local Lance helpers After demoting TableStore::append_batch from pub to pub(crate), the integration tests in tests/recovery.rs and tests/staged_writes.rs that previously called store.append_batch(...) directly to simulate HEAD-ahead-of-manifest drift can no longer access the inherent method. Replace those calls with small in-test helpers that do a raw Dataset::append (the same body the inherent method runs). - tests/helpers/mod.rs gains lance_append_inline (shared helper). - tests/staged_writes.rs gets a file-local lance_append_inline_local (staged_writes.rs does not import helpers::). - tests/recovery.rs drops the unused TableStore import in the one function whose store binding became unused after the conversion. Co-Authored-By: Ragnor Comerford <ragnor.comerford@gmail.com> * MR-854: retrigger CI for flaky Test Workspace job Co-Authored-By: Ragnor Comerford <ragnor.comerford@gmail.com> * MR-854: convert remaining table_store call sites in export.rs / read_blob Two leftover `self.table_store.X` / `db.table_store.X` call sites were missed in the initial sweep — flagged by Devin Review on PR #86. Both now go through the trait surface: - `entity_from_snapshot` (db/omnigraph/export.rs): switch from `db.table_store.open_snapshot_table` + `db.table_store.scan` to `db.storage().open_snapshot_at_table` + `db.storage().scan`. - `read_blob` (db/omnigraph.rs): replace `snapshot.open(table_key)` + `self.table_store.first_row_id_for_filter` with `self.storage().open_snapshot_at_table` + `self.storage().first_row_id_for_filter`. The follow-up `take_blobs` call still needs an `Arc<Dataset>` (it's a Lance blob accessor not surfaced through the trait), so we hand off via `SnapshotHandle::into_arc()` with a comment. After this commit, no engine code outside `table_store.rs` reaches the inherent `TableStore` API — the docs/runs.md and docs/invariants.md claim is now uniformly true. Co-Authored-By: Ragnor Comerford <ragnor.comerford@gmail.com> * MR-854: post-rebase doc fixes (Lance 6.0.1, MR-A framing, into_dataset note) Reviewer feedback on the rebased PR: * docs/dev/writes.md residuals matrix: drop demoted methods from the trait-surface table (now `pub(crate)`); keep only the two genuine trait-surface residuals (`delete_where`, `create_vector_index`); reframe under MR-A (Lance v7.x bump) per docs/dev/lance.md. * tests/forbidden_apis.rs: update transitional allow-list header to (a) drop the truncate_table mislabel (truncate_table is a Lance Dataset method, not a TableStore method — overwrite_batch's internal call), (b) reframe trait-surface residuals under MR-A / Lance #6666. * crates/omnigraph/src/storage_layer.rs::SnapshotHandle::{into_arc, into_dataset}: add single-ref invariant doc — both consume Arc via try_unwrap-or-clone; sibling SnapshotHandle clones across an await point force a deep Dataset clone. * Replace lance-4.0.0 version refs with lance-6.0.1 in active source/test/dev-doc comments (storage_layer.rs, table_store.rs, table_ops.rs, schema_apply.rs, merge.rs, recovery.rs, staged_writes.rs, consistency.rs, docs/dev/execution.md, docs/user/query-language.md). Historical refs in docs/releases/v0.4.1.md and the canonical "Lance 4.0.0 → 6.0.1 migration" line in docs/dev/lance.md left intact. No engine code changes. * MR-854: update docs/dev/invariants.md Storage trait row + gap entry Reviewer feedback: the docs reorg landed; the invariant row now lives in docs/dev/invariants.md with stable headings (no more numbered §VI.23). Update two pieces to reflect MR-854 completion: * Status table 'Storage trait' row: was 'full call-site migration ... incomplete'; now 'engine call sites all route through db.storage() (MR-854); inline-commit inherent methods are pub(crate)-demoted; capability/stat surfaces are roadmap'. * 'Known Gaps' 'Storage abstraction' entry: was 'older inherent TableStore call sites and inline residuals remain'; now names the closed scope (MR-854 — call sites migrated, methods demoted, loader fast-paths) and the remaining trait-surface residuals under MR-A (Lance v7.x bump) and Lance #6666. Cross-links to docs/dev/lance.md and docs/dev/writes.md so the framing stays co-located with the canonical Lance surface tracking. * MR-854: remove dead inline-commit methods from the storage surface The loader concurrent fast-path (write_batch_to_dataset) is only reached for LoadMode::Overwrite — Append/Merge route through MutationStaging — so its Append/Merge arms were unreachable. Collapse it to overwrite-only and drop the now-unused mode params, which removes the only callers of: - TableStorage::append_batch + TableStorage::merge_insert_batches (trait) - TableStore::merge_insert_batch + merge_insert_batches (inherent) create_btree_index / create_inverted_index had zero callers anywhere (scalar index builds use the stage_* primitives). Remove both from the trait and the inherent impl. Inherent append_batch stays pub(crate): overwrite_batch and recovery tests use it. Migrate the one trait-append_batch test caller (seed_person_row) to stage_append + commit_staged. The merge_insert FirstSeen-workaround rationale moves from the deleted merge_insert_batch into stage_merge_insert (now the sole merge path). No behavior change. Also corrects the inaccurate loader residual comment (the prior text blamed Lance #6658/#6666, which are the delete and vector-index issues, for keeping overwrite inline; a stage_overwrite primitive already exists and schema_apply uses it). * MR-854: seal db.storage() to staged-only; move residuals to InlineCommitResidual Split the three remaining inline-commit writes (overwrite_batch, delete_where, create_vector_index) off the TableStorage trait onto a new sealed InlineCommitResidual trait, reachable only via the explicit Omnigraph::storage_inline_residual() accessor. db.storage() now exposes only staged primitives + reads, so engine code cannot couple a write with a Lance HEAD advance through the default surface — MR-793 acceptance §1 ("no public method commits as a side effect of writing") now holds by construction, not by review + naming. Call sites moved to storage_inline_residual(): loader overwrite fast-path, the three mutation delete_where paths, the branch-merge delete, and the vector-index build. Impl bodies are unchanged (same delegation to the pub(crate) inherent methods); this is a pure surface reshape with no behavior change. The residual trait holds two genuinely upstream-blocked methods (delete_where -> Lance #6658/v7.x, create_vector_index -> Lance #6666) plus overwrite_batch, kept for the loader's cross-table bulk-overwrite concurrency until its staged migration lands (tracked follow-up). * MR-854 docs: describe the staged-only seal; fix stale Lance index URLs - writes.md / invariants.md / AGENTS.md: the inline-commit residuals now live on InlineCommitResidual behind db.storage_inline_residual(), so acceptance §1 holds by construction rather than 'option (b)' per-method enumeration. Drop the inaccurate 'until Lance exposes Operation::Overwrite { fragments }' claim (that op exists; stage_overwrite already builds it) and reframe overwrite_batch as a removable legacy residual gated on the loader's bulk-overwrite concurrency. - forbidden_apis.rs: rewrite the allow-list doc for the split surface. - lance.md: the index spec pages moved from /format/table/index/ to /format/index/ in Lance 6.x (the old paths 404). Fix all 13 URLs. * MR-854: fix stale lance-4.0.0 comment refs flagged in review Addresses greptile (exec/merge.rs) and aaltshuler's stale-version blocker: update lance-4.0.0 -> 6.0.1 in the comment/doc refs within this PR's footprint (exec/merge.rs, exec/mutation.rs, docs/dev/writes.md). Also corrects exec/merge.rs to cite lance#6666 (not #6658) for build_index_metadata_from_segments — that is the vector-index segment-commit API; #6658 is the two-phase delete. (Pre-existing 4.0.0 refs in untouched files like architecture.md/storage.md are main's incomplete migration cleanup, left out of scope.) * fix(storage): stage loader overwrites * fix(storage): stage empty schema rewrites --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ragnor Comerford <ragnor.comerford@gmail.com> Co-authored-by: Ragnor Comerford <hello@ragnor.co> --- AGENTS.md | 2 +- crates/omnigraph/src/changes/mod.rs | 65 +-- crates/omnigraph/src/db/manifest/recovery.rs | 2 +- crates/omnigraph/src/db/omnigraph.rs | 105 +++-- crates/omnigraph/src/db/omnigraph/export.rs | 24 +- crates/omnigraph/src/db/omnigraph/optimize.rs | 28 +- crates/omnigraph/src/db/omnigraph/repair.rs | 14 +- .../src/db/omnigraph/schema_apply.rs | 95 ++-- .../omnigraph/src/db/omnigraph/table_ops.rs | 110 ++--- crates/omnigraph/src/exec/merge.rs | 33 +- crates/omnigraph/src/exec/mod.rs | 1 + crates/omnigraph/src/exec/mutation.rs | 51 +-- crates/omnigraph/src/exec/staging.rs | 300 ++++++------ crates/omnigraph/src/loader/mod.rs | 432 ++++-------------- crates/omnigraph/src/storage_layer.rs | 319 ++++++------- crates/omnigraph/src/table_store.rs | 278 ++++------- crates/omnigraph/tests/consistency.rs | 2 +- crates/omnigraph/tests/failpoints.rs | 82 +++- crates/omnigraph/tests/forbidden_apis.rs | 22 +- crates/omnigraph/tests/helpers/mod.rs | 29 ++ crates/omnigraph/tests/recovery.rs | 85 +--- crates/omnigraph/tests/staged_writes.rs | 116 +++-- crates/omnigraph/tests/writes.rs | 91 ++++ docs/dev/architecture.md | 10 +- docs/dev/execution.md | 2 +- docs/dev/invariants.md | 15 +- docs/dev/lance.md | 26 +- docs/dev/writes.md | 54 +-- docs/user/query-language.md | 2 +- 29 files changed, 1150 insertions(+), 1245 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index e4ac297..25243a5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -240,7 +240,7 @@ omnigraph policy explain --actor act-alice --action change --branch main | Columnar storage on object store | ✅ Arrow/Lance | URI normalization, S3 env-var plumbing | | Per-dataset versioning + time travel | ✅ | `snapshot_at_version`, `entity_at`, snapshot-pinned reads across many tables | | Per-dataset branches | ✅ | **Graph-level** branches (atomic across all sub-tables), lazy fork, system branch filtering | -| Atomic single-dataset commits | ✅ | **Multi-table publish via three layers**, NOT a single Lance primitive: (1) per-table Lance `commit_staged` for the data write, (2) `__manifest` row-level CAS via `ManifestBatchPublisher` for cross-table ordering, (3) the open-time recovery sweep for the residual gap between (1) and (2). All three layers ship; the five migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`, `optimize_all_tables`) write a `__recovery/{ulid}.json` sidecar before Phase B and delete it after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the sweep in `db/manifest/recovery.rs`: classify, decide all-or-nothing per sidecar, roll forward via single `ManifestBatchPublisher::publish` or roll back via `Dataset::restore` followed by a manifest publish of the restored version (so both directions converge to `manifest == HEAD` — no residual drift), and record an audit row in `_graph_commit_recoveries.lance` (queryable via `omnigraph commit list --filter actor=omnigraph:recovery`). Continuous in-process recovery (no restart needed between Phase B failure and recovery) is the goal of a future background reconciler. Engine writes route through a sealed `TableStorage` trait exposing `stage_*` + `commit_staged` as the canonical staged-write surface; documented inline-commit residuals (`delete_where`, `create_vector_index`, plus legacy `append_batch` / `merge_insert_batches` / `overwrite_batch` / `create_*_index`) remain on the trait until upstream Lance ships a public two-phase API ([#6658](https://github.com/lance-format/lance/issues/6658), [#6666](https://github.com/lance-format/lance/issues/6666)) and the migration of every call site completes. | +| Atomic single-dataset commits | ✅ | **Multi-table publish via three layers**, NOT a single Lance primitive: (1) per-table Lance `commit_staged` for the data write, (2) `__manifest` row-level CAS via `ManifestBatchPublisher` for cross-table ordering, (3) the open-time recovery sweep for the residual gap between (1) and (2). All three layers ship; the five migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`, `optimize_all_tables`) write a `__recovery/{ulid}.json` sidecar before Phase B and delete it after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the sweep in `db/manifest/recovery.rs`: classify, decide all-or-nothing per sidecar, roll forward via single `ManifestBatchPublisher::publish` or roll back via `Dataset::restore` followed by a manifest publish of the restored version (so both directions converge to `manifest == HEAD` — no residual drift), and record an audit row in `_graph_commit_recoveries.lance` (queryable via `omnigraph commit list --filter actor=omnigraph:recovery`). Continuous in-process recovery (no restart needed between Phase B failure and recovery) is the goal of a future background reconciler. Engine writes route through a sealed `TableStorage` trait (`db.storage()`) exposing only `stage_*` + `commit_staged` + reads; the inline-commit residuals (`delete_where`, `create_vector_index`) are split onto a separate sealed `InlineCommitResidual` trait reached via `db.storage_inline_residual()` (MR-854), so the default surface cannot couple a write with a HEAD advance — §1 holds by construction. `delete_where` and `create_vector_index` stay inline until upstream Lance ships a public two-phase API ([#6658](https://github.com/lance-format/lance/issues/6658), [#6666](https://github.com/lance-format/lance/issues/6666)); `LoadMode::Overwrite` uses Lance `Overwrite` staged transactions. | | Compaction (`compact_files`) | ✅ | `omnigraph optimize` orchestrates over all node/edge tables, bounded concurrency; **publishes each compacted table's new version to `__manifest`** (so the manifest tracks the Lance HEAD — required for reads to observe compaction and for schema apply / strict writes to pass their HEAD-vs-manifest precondition), under the per-`(table, main)` write queue with `SidecarKind::Optimize` recovery coverage; **refuses on an unrecovered graph** (errors if a `__recovery` sidecar is pending); **skips uncovered HEAD > manifest drift** with `DriftNeedsRepair` instead of interpreting it; **skips blob-bearing tables** (reported via `TableOptimizeStats.skipped`, not silent), gated on `LANCE_SUPPORTS_BLOB_COMPACTION` until the upstream blob-v2 compaction-decode bug is fixed (see [docs/dev/invariants.md](docs/dev/invariants.md) Known Gaps) | | Repair uncovered drift | — | `omnigraph repair` explicitly classifies uncovered table `HEAD > manifest` drift: verified maintenance drift (`ReserveFragments`/`Rewrite`) can be published with `--confirm`; suspicious or unverifiable drift requires `--force --confirm`. Sidecar-covered crash residuals still recover automatically on open. | | Cleanup (`cleanup_old_versions`) | ✅ | `omnigraph cleanup` with `--keep` / `--older-than` policy | diff --git a/crates/omnigraph/src/changes/mod.rs b/crates/omnigraph/src/changes/mod.rs index 7c9e8ea..d4a3fe7 100644 --- a/crates/omnigraph/src/changes/mod.rs +++ b/crates/omnigraph/src/changes/mod.rs @@ -7,6 +7,7 @@ use lance::dataset::scanner::ColumnOrdering; use crate::db::SubTableEntry; use crate::db::manifest::Snapshot; use crate::error::Result; +use crate::storage_layer::{SnapshotHandle, TableStorage}; use crate::table_store::TableStore; // ─── Types ────────────────────────────────────────────────────────────────── @@ -229,7 +230,8 @@ async fn diff_table_same_lineage( ) -> Result<Vec<EntityChange>> { let vf = from_entry.table_version; let vt = to_entry.table_version; - let to_ds = table_store.open_at_entry(to_entry).await?; + let storage: &dyn TableStorage = table_store; + let to_ds = storage.open_snapshot_at_entry(to_entry).await?; let cols: Vec<&str> = if is_edge { vec!["id", "src", "dst", "_row_last_updated_at_version"] @@ -257,12 +259,12 @@ async fn diff_table_same_lineage( "_row_last_updated_at_version > {} AND _row_last_updated_at_version <= {}", vf, vt ); - let changed_rows = scan_with_filter(table_store, &to_ds, &cols, &filter_sql).await?; + let changed_rows = scan_with_filter(storage, &to_ds, &cols, &filter_sql).await?; if !changed_rows.is_empty() { // Build the set of IDs that existed at the from version - let from_ds = table_store.open_at_entry(from_entry).await?; - let from_ids: HashSet<String> = scan_id_set(table_store, &from_ds, &["id"]) + let from_ds = storage.open_snapshot_at_entry(from_entry).await?; + let from_ids: HashSet<String> = scan_id_set(storage, &from_ds, &["id"]) .await? .into_iter() .map(|r| r.id) @@ -282,8 +284,8 @@ async fn diff_table_same_lineage( // Deletes: ID set-difference if wants_deletes { - let from_ds = table_store.open_at_entry(from_entry).await?; - let deleted = deleted_ids_by_set_diff(table_store, &from_ds, &to_ds, is_edge).await?; + let from_ds = storage.open_snapshot_at_entry(from_entry).await?; + let deleted = deleted_ids_by_set_diff(storage, &from_ds, &to_ds, is_edge).await?; changes.extend(deleted); } @@ -300,13 +302,14 @@ async fn diff_table_cross_branch( is_edge: bool, filter: &ChangeFilter, ) -> Result<Vec<EntityChange>> { - let from_ds = table_store - .open_snapshot_table(from_snap, table_key) + let storage: &dyn TableStorage = table_store; + let from_ds = storage + .open_snapshot_at_table(from_snap, table_key) .await?; - let to_ds = table_store.open_snapshot_table(to_snap, table_key).await?; + let to_ds = storage.open_snapshot_at_table(to_snap, table_key).await?; - let from_rows = scan_all_rows_ordered(table_store, &from_ds, is_edge).await?; - let to_rows = scan_all_rows_ordered(table_store, &to_ds, is_edge).await?; + let from_rows = scan_all_rows_ordered(storage, &from_ds, is_edge).await?; + let to_rows = scan_all_rows_ordered(storage, &to_ds, is_edge).await?; let mut changes = Vec::new(); let mut fi = 0; @@ -392,8 +395,9 @@ async fn diff_table_added( if !filter.wants_op(ChangeOp::Insert) { return Ok(Vec::new()); } - let ds = table_store.open_snapshot_table(to_snap, table_key).await?; - let rows = scan_all_rows_ordered(table_store, &ds, is_edge).await?; + let storage: &dyn TableStorage = table_store; + let ds = storage.open_snapshot_at_table(to_snap, table_key).await?; + let rows = scan_all_rows_ordered(storage, &ds, is_edge).await?; Ok(rows .into_iter() .map(|r| entity_change_from_row(&r, ChangeOp::Insert, is_edge)) @@ -410,10 +414,11 @@ async fn diff_table_removed( if !filter.wants_op(ChangeOp::Delete) { return Ok(Vec::new()); } - let ds = table_store - .open_snapshot_table(from_snap, table_key) + let storage: &dyn TableStorage = table_store; + let ds = storage + .open_snapshot_at_table(from_snap, table_key) .await?; - let rows = scan_all_rows_ordered(table_store, &ds, is_edge).await?; + let rows = scan_all_rows_ordered(storage, &ds, is_edge).await?; Ok(rows .into_iter() .map(|r| entity_change_from_row(&r, ChangeOp::Delete, is_edge)) @@ -424,12 +429,12 @@ async fn diff_table_removed( /// Scan with a SQL filter, projecting specific columns. async fn scan_with_filter( - table_store: &TableStore, - ds: &lance::Dataset, + storage: &dyn TableStorage, + ds: &SnapshotHandle, cols: &[&str], filter_sql: &str, ) -> Result<Vec<ScannedRow>> { - let batches = table_store + let batches = storage .scan(ds, Some(cols), Some(filter_sql), None) .await?; Ok(extract_rows(&batches)) @@ -437,11 +442,11 @@ async fn scan_with_filter( /// Scan all rows ordered by id, projecting id (+ src/dst for edges) + all columns for signature. async fn scan_all_rows_ordered( - table_store: &TableStore, - ds: &lance::Dataset, + storage: &dyn TableStorage, + ds: &SnapshotHandle, is_edge: bool, ) -> Result<Vec<ScannedRow>> { - let batches = table_store + let batches = storage .scan( ds, None, @@ -454,9 +459,9 @@ async fn scan_all_rows_ordered( /// Compute deleted IDs: scan id at from and to, set-difference. async fn deleted_ids_by_set_diff( - table_store: &TableStore, - from_ds: &lance::Dataset, - to_ds: &lance::Dataset, + storage: &dyn TableStorage, + from_ds: &SnapshotHandle, + to_ds: &SnapshotHandle, is_edge: bool, ) -> Result<Vec<EntityChange>> { let cols: Vec<&str> = if is_edge { @@ -465,8 +470,8 @@ async fn deleted_ids_by_set_diff( vec!["id"] }; - let from_rows = scan_id_set(table_store, from_ds, &cols).await?; - let to_ids: HashSet<String> = scan_id_set(table_store, to_ds, &["id"]) + let from_rows = scan_id_set(storage, from_ds, &cols).await?; + let to_ids: HashSet<String> = scan_id_set(storage, to_ds, &["id"]) .await? .into_iter() .map(|r| r.id) @@ -480,11 +485,11 @@ async fn deleted_ids_by_set_diff( } async fn scan_id_set( - table_store: &TableStore, - ds: &lance::Dataset, + storage: &dyn TableStorage, + ds: &SnapshotHandle, cols: &[&str], ) -> Result<Vec<ScannedRow>> { - let batches = table_store.scan(ds, Some(cols), None, None).await?; + let batches = storage.scan(ds, Some(cols), None, None).await?; Ok(extract_rows(&batches)) } diff --git a/crates/omnigraph/src/db/manifest/recovery.rs b/crates/omnigraph/src/db/manifest/recovery.rs index 3119531..3b0f147 100644 --- a/crates/omnigraph/src/db/manifest/recovery.rs +++ b/crates/omnigraph/src/db/manifest/recovery.rs @@ -25,7 +25,7 @@ //! version. Pinned by //! `tests/staged_writes.rs::lance_restore_appends_one_commit_with_checked_out_content`. //! - `Dataset::restore` "wins" against concurrent Append/Update/Delete/ -//! CreateIndex/Merge — see `check_restore_txn` at lance-4.0.0 +//! CreateIndex/Merge — see `check_restore_txn` at lance-6.0.1 //! `src/io/commit/conflict_resolver.rs:986`. The hazard is documented //! by `tests/staged_writes.rs::lance_restore_loses_to_concurrent_append_via_orphaning`. //! This module sidesteps the hazard by running recovery only at diff --git a/crates/omnigraph/src/db/omnigraph.rs b/crates/omnigraph/src/db/omnigraph.rs index 5bcc973..f217f7d 100644 --- a/crates/omnigraph/src/db/omnigraph.rs +++ b/crates/omnigraph/src/db/omnigraph.rs @@ -26,6 +26,7 @@ use crate::db::graph_coordinator::{GraphCoordinator, PublishedSnapshot}; use crate::error::{OmniError, Result}; use crate::runtime_cache::RuntimeCache; use crate::storage::{StorageAdapter, join_uri, normalize_root_uri, storage_for_uri}; +use crate::storage_layer::SnapshotHandle; use crate::table_store::TableStore; mod export; @@ -583,19 +584,30 @@ impl Omnigraph { schema_apply::ensure_schema_apply_not_locked(self, operation).await } - pub(crate) fn table_store(&self) -> &TableStore { + /// Engine-facing trait surface around `TableStore`. + /// + /// This is the **only** accessor for engine code reaching into the + /// storage layer. The trait's signatures use opaque `SnapshotHandle` + /// / `StagedHandle` instead of leaking `lance::Dataset` / + /// `lance::dataset::transaction::Transaction`, so newly-added engine + /// call sites cannot drift the staged-write invariant by mistake + /// (the trait's `stage_*` + `commit_staged` pair is the only way to + /// land a write). + pub(crate) fn storage(&self) -> &dyn crate::storage_layer::TableStorage { &self.table_store } - /// Engine-facing trait surface around `TableStore`. - /// - /// This is the canonical accessor for newly-written engine code. The - /// trait's signatures use opaque `SnapshotHandle` / `StagedHandle` - /// instead of leaking `lance::Dataset` / - /// `lance::dataset::transaction::Transaction`. Existing call sites - /// that still use `db.table_store.X(...)` (the inherent struct - /// methods) are migrated incrementally. - pub(crate) fn storage(&self) -> &dyn crate::storage_layer::TableStorage { + /// Inline-commit residual surface (`delete_where`, + /// `create_vector_index`) — the writes Lance cannot yet express as a + /// stage-then-commit pair. Deliberately separate from [`Self::storage`] so + /// the default storage surface is staged-only and a new writer cannot couple + /// "write bytes" with "advance HEAD" by reaching for `db.storage()`. Only + /// the handful of documented residual call sites (mutation/merge deletes, + /// vector-index build) use this accessor. See + /// `crate::storage_layer::InlineCommitResidual` for the per-method blocker. + pub(crate) fn storage_inline_residual( + &self, + ) -> &dyn crate::storage_layer::InlineCommitResidual { &self.table_store } @@ -1055,19 +1067,24 @@ impl Omnigraph { let snapshot = self.snapshot().await; let table_key = format!("node:{}", type_name); - let ds = snapshot.open(&table_key).await?; + let handle = self + .storage() + .open_snapshot_at_table(&snapshot, &table_key) + .await?; let filter_sql = format!("id = '{}'", id.replace('\'', "''")); let row_id = self - .table_store - .first_row_id_for_filter(&ds, &filter_sql) + .storage() + .first_row_id_for_filter(&handle, &filter_sql) .await? .ok_or_else(|| { OmniError::manifest(format!("no {} with id '{}' found", type_name, id)) })?; - // Use take_blobs to get the BlobFile handle - let ds = Arc::new(ds); + // `take_blobs` is a Lance-specific blob accessor not surfaced + // through the `TableStorage` trait — reach the inner `Arc<Dataset>` + // via the `pub(crate)` accessor for this read-only call. + let ds = handle.into_arc(); let mut blobs = ds .take_blobs(&[row_id], property) .await @@ -1141,10 +1158,14 @@ impl Omnigraph { cleanup_targets.sort_by(|left, right| left.0.cmp(&right.0)); for (table_key, table_path) in cleanup_targets { - let dataset_uri = self.table_store.dataset_uri(&table_path); + let dataset_uri = self.storage().dataset_uri(&table_path); let outcome = match crate::failpoints::maybe_fail("branch_delete.before_table_cleanup") { - Ok(()) => self.table_store.force_delete_branch(&dataset_uri, branch).await, + Ok(()) => { + self.storage() + .force_delete_branch(&dataset_uri, branch) + .await + } Err(injected) => Err(injected), }; if let Err(err) = outcome { @@ -1370,7 +1391,7 @@ impl Omnigraph { &self, table_key: &str, op_kind: crate::db::MutationOpKind, - ) -> Result<(Dataset, String, Option<String>)> { + ) -> Result<(SnapshotHandle, String, Option<String>)> { table_ops::open_for_mutation(self, table_key, op_kind).await } @@ -1379,7 +1400,7 @@ impl Omnigraph { branch: Option<&str>, table_key: &str, op_kind: crate::db::MutationOpKind, - ) -> Result<(Dataset, String, Option<String>)> { + ) -> Result<(SnapshotHandle, String, Option<String>)> { table_ops::open_for_mutation_on_branch(self, branch, table_key, op_kind).await } @@ -1390,7 +1411,7 @@ impl Omnigraph { source_branch: Option<&str>, source_version: u64, active_branch: &str, - ) -> Result<Dataset> { + ) -> Result<SnapshotHandle> { table_ops::fork_dataset_from_entry_state( self, table_key, @@ -1409,7 +1430,7 @@ impl Omnigraph { table_branch: Option<&str>, expected_version: u64, op_kind: crate::db::MutationOpKind, - ) -> Result<Dataset> { + ) -> Result<SnapshotHandle> { table_ops::reopen_for_mutation( self, table_key, @@ -1426,14 +1447,14 @@ impl Omnigraph { table_path: &str, table_branch: Option<&str>, table_version: u64, - ) -> Result<Dataset> { + ) -> Result<SnapshotHandle> { table_ops::open_dataset_at_state(self, table_path, table_branch, table_version).await } pub(crate) async fn build_indices_on_dataset( &self, table_key: &str, - ds: &mut Dataset, + ds: &mut SnapshotHandle, ) -> Result<()> { table_ops::build_indices_on_dataset(self, table_key, ds).await } @@ -1442,7 +1463,7 @@ impl Omnigraph { &self, catalog: &Catalog, table_key: &str, - ds: &mut Dataset, + ds: &mut SnapshotHandle, ) -> Result<()> { table_ops::build_indices_on_dataset_for_catalog(self, catalog, table_key, ds).await } @@ -2139,8 +2160,12 @@ edge WorksAt: Person -> Company async fn table_rows_json(db: &Omnigraph, table_key: &str) -> Vec<Value> { let snapshot = db.snapshot().await; - let ds = snapshot.open(table_key).await.unwrap(); - let batches = db.table_store().scan_batches(&ds).await.unwrap(); + let ds = db + .storage() + .open_snapshot_at_table(&snapshot, table_key) + .await + .unwrap(); + let batches = db.storage().scan_batches(&ds).await.unwrap(); batches .into_iter() .flat_map(|batch| { @@ -2152,11 +2177,11 @@ edge WorksAt: Person -> Company } async fn seed_person_row(db: &mut Omnigraph, name: &str, age: Option<i32>) { - let (mut ds, full_path, table_branch) = db + let (ds, full_path, table_branch) = db .open_for_mutation("node:Person", crate::db::MutationOpKind::Insert) .await .unwrap(); - let schema: Arc<Schema> = Arc::new(ds.schema().into()); + let schema: Arc<Schema> = Arc::new(ds.dataset().schema().into()); let columns: Vec<Arc<dyn Array>> = schema .fields() .iter() @@ -2168,9 +2193,11 @@ edge WorksAt: Person -> Company }) .collect(); let batch = RecordBatch::try_new(Arc::clone(&schema), columns).unwrap(); + let staged = db.storage().stage_append(&ds, batch, &[]).await.unwrap(); + let committed = db.storage().commit_staged(ds, staged).await.unwrap(); let state = db - .table_store() - .append_batch(&full_path, &mut ds, batch) + .storage() + .table_state(&full_path, &committed) .await .unwrap(); db.commit_updates(&[crate::db::SubTableUpdate { @@ -2354,8 +2381,12 @@ edge WorksAt: Person -> Company db.apply_schema(&desired).await.unwrap(); let snapshot = db.snapshot().await; - let ds = snapshot.open("node:Person").await.unwrap(); - assert!(db.table_store().has_fts_index(&ds, "name").await.unwrap()); + let ds = db + .storage() + .open_snapshot_at_table(&snapshot, "node:Person") + .await + .unwrap(); + assert!(db.storage().has_fts_index(&ds, "name").await.unwrap()); } #[tokio::test] @@ -2373,9 +2404,13 @@ edge WorksAt: Person -> Company db.apply_schema(&desired).await.unwrap(); let snapshot = db.snapshot().await; - let ds = snapshot.open("node:Person").await.unwrap(); - assert!(db.table_store().has_btree_index(&ds, "id").await.unwrap()); - assert!(db.table_store().has_fts_index(&ds, "name").await.unwrap()); + let ds = db + .storage() + .open_snapshot_at_table(&snapshot, "node:Person") + .await + .unwrap(); + assert!(db.storage().has_btree_index(&ds, "id").await.unwrap()); + assert!(db.storage().has_fts_index(&ds, "name").await.unwrap()); } #[tokio::test] diff --git a/crates/omnigraph/src/db/omnigraph/export.rs b/crates/omnigraph/src/db/omnigraph/export.rs index 366f50a..7696056 100644 --- a/crates/omnigraph/src/db/omnigraph/export.rs +++ b/crates/omnigraph/src/db/omnigraph/export.rs @@ -60,12 +60,12 @@ async fn entity_from_snapshot( } let ds = db - .table_store - .open_snapshot_table(snapshot, table_key) + .storage() + .open_snapshot_at_table(snapshot, table_key) .await?; let filter_sql = format!("id = '{}'", id.replace('\'', "''")); let batches = db - .table_store + .storage() .scan(&ds, None, Some(&filter_sql), None) .await?; let Some(batch) = batches.iter().find(|batch| batch.num_rows() > 0) else { @@ -143,23 +143,23 @@ async fn export_table_to_writer<W: Write>( writer: &mut W, ) -> Result<()> { let ds = db - .table_store - .open_snapshot_table(snapshot, table_key) + .storage() + .open_snapshot_at_table(snapshot, table_key) .await?; let ordering = Some(vec![ColumnOrdering::asc_nulls_last("id".to_string())]); let catalog = db.catalog(); let blob_properties = blob_properties_for_table_key(&catalog, table_key)?; if blob_properties.is_empty() { - for batch in db.table_store.scan(&ds, None, None, ordering).await? { + for batch in db.storage().scan(&ds, None, None, ordering).await? { write_export_rows_from_batch(db, table_key, &batch, None, writer)?; } return Ok(()); } let batches = db - .table_store - .scan_with(&ds, None, None, ordering, true, |_| Ok(())) + .storage() + .scan_with_row_id(&ds, None, None, ordering, true) .await?; for batch in batches { let row_ids = batch @@ -175,7 +175,13 @@ async fn export_table_to_writer<W: Write>( .iter() .copied() .collect::<Vec<_>>(); - let blob_values = export_blob_values(&ds, &batch, &row_ids, blob_properties).await?; + // Blob materialization reaches through to the inner Lance + // `Dataset` because `take_blobs` is a Lance-only API not lifted + // onto the `TableStorage` trait surface (the trait covers + // staged-write and snapshot-scan primitives; blob descriptor + // materialization sits outside that surface). + let blob_values = + export_blob_values(ds.dataset(), &batch, &row_ids, blob_properties).await?; write_export_rows_from_batch(db, table_key, &batch, Some(&blob_values), writer)?; } Ok(()) diff --git a/crates/omnigraph/src/db/omnigraph/optimize.rs b/crates/omnigraph/src/db/omnigraph/optimize.rs index 3c37b66..21629a8 100644 --- a/crates/omnigraph/src/db/omnigraph/optimize.rs +++ b/crates/omnigraph/src/db/omnigraph/optimize.rs @@ -317,10 +317,16 @@ async fn optimize_one_table( .acquire_many(&[(table_key.clone(), None)]) .await; - let mut ds = db - .table_store + // `compact_files` is a Lance-only maintenance API that needs `&mut Dataset`. + // The `TableStorage` trait deliberately does not surface it (the staged-write + // invariant covers writes; compaction is a separate concern). Unwrap the + // opaque `SnapshotHandle` via `into_dataset()` (`pub(crate)`, gated to the + // maintenance path). + let handle = db + .storage() .open_dataset_head_for_write(&table_key, &full_path, None) .await?; + let mut ds = handle.into_dataset(); // CAS baseline: the table's current manifest version, read under the queue // (in-memory coordinator snapshot, no storage I/O — stable for this section). @@ -408,7 +414,10 @@ async fn optimize_one_table( // expected = the version observed under the queue). On failure the sidecar // is intentionally left for the open-time recovery sweep to roll forward. if committed { - let state = db.table_store.table_state(&full_path, &ds).await?; + // Re-wrap the post-compaction dataset to read its state through the + // trait surface (`table_state` is a read; no HEAD advance). + let snapshot = crate::storage_layer::SnapshotHandle::new(ds); + let state = db.storage().table_state(&full_path, &snapshot).await?; let update = crate::db::SubTableUpdate { table_key: table_key.clone(), table_version: state.version, @@ -493,7 +502,7 @@ pub async fn cleanup_all_tables( } let concurrency = maint_concurrency().min(table_tasks.len()).max(1); - let table_store = &db.table_store; + let storage = db.storage(); // Fault-isolated per table: a single table's GC failure is recorded on its // stats row (`error: Some`) and logged, never aborting the healthy tables. @@ -503,9 +512,13 @@ pub async fn cleanup_all_tables( .map(|(table_key, full_path)| async move { let outcome: Result<RemovalStats> = async { crate::failpoints::maybe_fail("cleanup.table_gc")?; - let ds = table_store + // `cleanup_old_versions` is a Lance-only maintenance API not + // surfaced through `TableStorage` — see the optimize path + // above for the same rationale. Unwrap via `into_dataset()`. + let handle = storage .open_dataset_head_for_write(&table_key, &full_path, None) .await?; + let ds = handle.into_dataset(); let before_version = keep_versions .map(|n| ds.version().version.saturating_sub(n as u64)) .filter(|v| *v > 0); @@ -606,8 +619,9 @@ pub async fn reconcile_orphaned_branches(db: &Omnigraph) -> Result<BranchReconci // Per-table fault isolation: one table's transient failure is recorded and // logged, never aborting the rest of the sweep. + let storage = db.storage(); for (table_key, full_path) in table_targets { - let listed = match db.table_store.list_branches(&full_path).await { + let listed = match storage.list_branches(&full_path).await { Ok(listed) => listed, Err(err) => { tracing::warn!( @@ -622,7 +636,7 @@ pub async fn reconcile_orphaned_branches(db: &Omnigraph) -> Result<BranchReconci }; for branch in orphan_branches(listed, &keep) { let outcome = match crate::failpoints::maybe_fail("cleanup.reconcile_fork") { - Ok(()) => db.table_store.force_delete_branch(&full_path, &branch).await, + Ok(()) => storage.force_delete_branch(&full_path, &branch).await, Err(injected) => Err(injected), }; match outcome { diff --git a/crates/omnigraph/src/db/omnigraph/repair.rs b/crates/omnigraph/src/db/omnigraph/repair.rs index aaef2ba..8e7146a 100644 --- a/crates/omnigraph/src/db/omnigraph/repair.rs +++ b/crates/omnigraph/src/db/omnigraph/repair.rs @@ -165,10 +165,15 @@ pub async fn repair_all_tables(db: &Omnigraph, options: RepairOptions) -> Result let mut any_forced = false; for (table_key, full_path) in table_tasks { + // `classify_drift` inspects raw Lance transaction history + // (`read_transaction_by_version`), a Lance-only maintenance read the + // staged-write trait does not surface. Open via `db.storage()` and + // unwrap the opaque handle (mirrors optimize / cleanup). let ds = db - .table_store + .storage() .open_dataset_head_for_write(&table_key, &full_path, None) - .await?; + .await? + .into_dataset(); let manifest_version = snapshot .entry(&table_key) .map(|e| e.table_version) @@ -214,7 +219,10 @@ pub async fn repair_all_tables(db: &Omnigraph, options: RepairOptions) -> Result }; if matches!(action, RepairAction::Healed | RepairAction::Forced) { - let state = db.table_store.table_state(&full_path, &ds).await?; + // Re-wrap the opened dataset to read its state through the trait + // surface (`table_state` is a read; no HEAD advance). + let snapshot = crate::storage_layer::SnapshotHandle::new(ds); + let state = db.storage().table_state(&full_path, &snapshot).await?; updates.push(crate::db::SubTableUpdate { table_key: table_key.clone(), table_version: state.version, diff --git a/crates/omnigraph/src/db/omnigraph/schema_apply.rs b/crates/omnigraph/src/db/omnigraph/schema_apply.rs index 7cb3193..506db36 100644 --- a/crates/omnigraph/src/db/omnigraph/schema_apply.rs +++ b/crates/omnigraph/src/db/omnigraph/schema_apply.rs @@ -355,7 +355,7 @@ where let entry = snapshot.entry(table_key)?; Some(crate::db::manifest::SidecarTablePin { table_key: table_key.clone(), - table_path: db.table_store.dataset_uri(&entry.table_path), + table_path: db.storage().dataset_uri(&entry.table_path), expected_version: entry.table_version, post_commit_pin: entry.table_version + 1, table_branch: entry.table_branch.clone(), @@ -469,12 +469,13 @@ where for table_key in &added_tables { let table_path = table_path_for_table_key(table_key)?; - let dataset_uri = db.table_store.dataset_uri(&table_path); + let dataset_uri = db.storage().dataset_uri(&table_path); let schema = schema_for_table_key(&desired_catalog, table_key)?; - let mut ds = TableStore::create_empty_dataset(&dataset_uri, &schema).await?; + let mut ds = + SnapshotHandle::new(TableStore::create_empty_dataset(&dataset_uri, &schema).await?); db.build_indices_on_dataset_for_catalog(&desired_catalog, table_key, &mut ds) .await?; - let state = db.table_store.table_state(&dataset_uri, &ds).await?; + let state = db.storage().table_state(&dataset_uri, &ds).await?; table_registrations.insert(table_key.clone(), table_path); table_updates.insert( table_key.clone(), @@ -496,7 +497,10 @@ where )) })?; ensure_snapshot_entry_head_matches(db, source_entry).await?; - let source_ds = snapshot.open(source_table_key).await?; + let source_ds = db + .storage() + .open_snapshot_at_table(&snapshot, source_table_key) + .await?; let current_catalog = db.catalog(); let batch = batch_for_schema_apply_rewrite( db, @@ -509,11 +513,12 @@ where ) .await?; let table_path = table_path_for_table_key(target_table_key)?; - let dataset_uri = db.table_store.dataset_uri(&table_path); - let mut target_ds = TableStore::write_dataset(&dataset_uri, batch).await?; + let dataset_uri = db.storage().dataset_uri(&table_path); + let mut target_ds = + SnapshotHandle::new(TableStore::write_dataset(&dataset_uri, batch).await?); db.build_indices_on_dataset_for_catalog(&desired_catalog, target_table_key, &mut target_ds) .await?; - let state = db.table_store.table_state(&dataset_uri, &target_ds).await?; + let state = db.storage().table_state(&dataset_uri, &target_ds).await?; table_registrations.insert(target_table_key.clone(), table_path); table_updates.insert( target_table_key.clone(), @@ -542,7 +547,10 @@ where )) })?; ensure_snapshot_entry_head_matches(db, entry).await?; - let source_ds = snapshot.open(table_key).await?; + let source_ds = db + .storage() + .open_snapshot_at_table(&snapshot, table_key) + .await?; let current_catalog = db.catalog(); let batch = batch_for_schema_apply_rewrite( db, @@ -554,37 +562,22 @@ where property_renames.get(table_key), ) .await?; - let dataset_uri = db.table_store.dataset_uri(&entry.table_path); - // Route through stage_overwrite + commit_staged for non-empty - // batches. Lance's `InsertBuilder::execute_uncommitted` - // errors on empty data (lance-4.0.0 `src/dataset/write/insert.rs:144`), - // so the empty-rewrite case stays on `overwrite_dataset` (which - // accepts empty input). The empty case is rare in schema_apply - // — it only fires when the source table itself was already empty - // — and schema_apply runs under `__schema_apply_lock__` so the - // narrow inline-commit residual is bounded. - let mut target_ds = if batch.num_rows() == 0 { - TableStore::overwrite_dataset(&dataset_uri, batch).await? - } else { - // Pass `entry.table_branch.as_deref()` (not `None`) for - // consistency with the indexed_tables block below. Schema - // apply runs under `__schema_apply_lock__` which today - // rejects non-main branches, so `entry.table_branch` is - // expected to be `None`. But the defensive passthrough - // means a future relaxation of the lock-check can't quietly - // open the wrong HEAD here. - let existing = db - .table_store - .open_dataset_head_for_write(table_key, &dataset_uri, entry.table_branch.as_deref()) - .await?; - let staged = db.table_store.stage_overwrite(&existing, batch).await?; - db.table_store - .commit_staged(Arc::new(existing), staged.transaction) - .await? - }; + let dataset_uri = db.storage().dataset_uri(&entry.table_path); + // Pass `entry.table_branch.as_deref()` (not `None`) for + // consistency with the indexed_tables block below. Schema + // apply runs under `__schema_apply_lock__` which today rejects + // non-main branches, so `entry.table_branch` is expected to be + // `None`. But the defensive passthrough means a future relaxation + // of the lock-check can't quietly open the wrong HEAD here. + let existing = db + .storage() + .open_dataset_head_for_write(table_key, &dataset_uri, entry.table_branch.as_deref()) + .await?; + let staged = db.storage().stage_overwrite(&existing, batch).await?; + let mut target_ds = db.storage().commit_staged(existing, staged).await?; db.build_indices_on_dataset_for_catalog(&desired_catalog, table_key, &mut target_ds) .await?; - let state = db.table_store.table_state(&dataset_uri, &target_ds).await?; + let state = db.storage().table_state(&dataset_uri, &target_ds).await?; table_updates.insert( table_key.clone(), crate::db::SubTableUpdate { @@ -611,16 +604,16 @@ where )) })?; ensure_snapshot_entry_head_matches(db, entry).await?; - let dataset_uri = db.table_store.dataset_uri(&entry.table_path); + let dataset_uri = db.storage().dataset_uri(&entry.table_path); let mut ds = db - .table_store + .storage() .open_dataset_head_for_write(table_key, &dataset_uri, entry.table_branch.as_deref()) .await?; - db.table_store + db.storage() .ensure_expected_version(&ds, table_key, entry.table_version)?; db.build_indices_on_dataset_for_catalog(&desired_catalog, table_key, &mut ds) .await?; - let state = db.table_store.table_state(&dataset_uri, &ds).await?; + let state = db.storage().table_state(&dataset_uri, &ds).await?; table_updates.insert( table_key.clone(), crate::db::SubTableUpdate { @@ -869,22 +862,22 @@ pub(super) async fn ensure_snapshot_entry_head_matches( db: &Omnigraph, entry: &SubTableEntry, ) -> Result<()> { - let dataset_uri = db.table_store.dataset_uri(&entry.table_path); + let dataset_uri = db.storage().dataset_uri(&entry.table_path); let ds = db - .table_store + .storage() .open_dataset_head_for_write( &entry.table_key, &dataset_uri, entry.table_branch.as_deref(), ) .await?; - db.table_store + db.storage() .ensure_expected_version(&ds, &entry.table_key, entry.table_version) } pub(super) async fn batch_for_schema_apply_rewrite( db: &Omnigraph, - source_ds: &Dataset, + source_ds: &SnapshotHandle, source_table_key: &str, source_catalog: &Catalog, target_table_key: &str, @@ -896,11 +889,11 @@ pub(super) async fn batch_for_schema_apply_rewrite( let target_blob_properties = blob_properties_for_table_key(target_catalog, target_table_key)?; let needs_row_ids = !source_blob_properties.is_empty() || !target_blob_properties.is_empty(); let batches = if needs_row_ids { - db.table_store() - .scan_with(source_ds, None, None, None, true, |_| Ok(())) + db.storage() + .scan_with_row_id(source_ds, None, None, None, true) .await? } else { - db.table_store().scan_batches(source_ds).await? + db.storage().scan_batches(source_ds).await? }; if batches.is_empty() { return Ok(RecordBatch::new_empty(target_schema)); @@ -970,7 +963,7 @@ pub(super) async fn batch_for_schema_apply_rewrite( async fn rebuild_blob_column( _db: &Omnigraph, - source_ds: &Dataset, + source_ds: &SnapshotHandle, column_name: &str, descriptions: &StructArray, row_ids: &[u64], @@ -990,7 +983,7 @@ async fn rebuild_blob_column( let blob_files = if non_null_row_ids.is_empty() { Vec::new() } else { - Arc::new(source_ds.clone()) + Arc::new(source_ds.dataset().clone()) .take_blobs(&non_null_row_ids, column_name) .await .map_err(|e| OmniError::Lance(e.to_string()))? diff --git a/crates/omnigraph/src/db/omnigraph/table_ops.rs b/crates/omnigraph/src/db/omnigraph/table_ops.rs index 3ed9c43..f7a365a 100644 --- a/crates/omnigraph/src/db/omnigraph/table_ops.rs +++ b/crates/omnigraph/src/db/omnigraph/table_ops.rs @@ -50,10 +50,10 @@ pub(super) async fn failpoint_publish_table_head_without_index_rebuild_for_test( .ok_or_else(|| OmniError::manifest(format!("no manifest entry for {}", table_key)))?; let full_path = format!("{}/{}", db.root_uri, entry.table_path); let ds = db - .table_store + .storage() .open_dataset_head_for_write(table_key, &full_path, table_branch) .await?; - let state = db.table_store.table_state(&full_path, &ds).await?; + let state = db.storage().table_state(&full_path, &ds).await?; let update = crate::db::SubTableUpdate { table_key: table_key.to_string(), table_version: state.version, @@ -209,18 +209,18 @@ pub(super) async fn ensure_indices_for_branch(db: &Omnigraph, branch: Option<&st } }, None => ( - db.table_store + db.storage() .open_dataset_head_for_write(&table_key, &full_path, None) .await?, None, ), }; - let row_count = db.table_store.count_rows(&ds, None).await.unwrap_or(0); + let row_count = db.storage().count_rows(&ds, None).await.unwrap_or(0); if row_count > 0 { build_indices_on_dataset(db, &table_key, &mut ds).await?; } - let state = db.table_store.table_state(&full_path, &ds).await?; + let state = db.storage().table_state(&full_path, &ds).await?; if state.version != entry.table_version || resolved_branch.as_deref() != entry.table_branch.as_deref() { @@ -257,18 +257,18 @@ pub(super) async fn ensure_indices_for_branch(db: &Omnigraph, branch: Option<&st } }, None => ( - db.table_store + db.storage() .open_dataset_head_for_write(&table_key, &full_path, None) .await?, None, ), }; - let row_count = db.table_store.count_rows(&ds, None).await.unwrap_or(0); + let row_count = db.storage().count_rows(&ds, None).await.unwrap_or(0); if row_count > 0 { build_indices_on_dataset(db, &table_key, &mut ds).await?; } - let state = db.table_store.table_state(&full_path, &ds).await?; + let state = db.storage().table_state(&full_path, &ds).await?; if state.version != entry.table_version || resolved_branch.as_deref() != entry.table_branch.as_deref() { @@ -331,7 +331,7 @@ async fn needs_index_work_node( table_branch: Option<&str>, ) -> Result<bool> { let ds = db - .table_store + .storage() .open_dataset_head_for_write(table_key, full_path, table_branch) .await?; // Empty tables are skipped by the ensure_indices loop, so they must @@ -341,10 +341,10 @@ async fn needs_index_work_node( // Errors from count_rows are propagated: silently treating them as // "0 rows" risks skipping a table that is actually about to be // modified. - if db.table_store.count_rows(&ds, None).await? == 0 { + if db.storage().count_rows(&ds, None).await? == 0 { return Ok(false); } - if !db.table_store.has_btree_index(&ds, "id").await? { + if !db.storage().has_btree_index(&ds, "id").await? { return Ok(true); } let catalog = db.catalog(); @@ -360,11 +360,11 @@ async fn needs_index_work_node( continue; }; if matches!(prop_type.scalar, ScalarType::String) && !prop_type.list { - if !db.table_store.has_fts_index(&ds, prop_name).await? { + if !db.storage().has_fts_index(&ds, prop_name).await? { return Ok(true); } } else if matches!(prop_type.scalar, ScalarType::Vector(_)) && !prop_type.list { - if !db.table_store.has_vector_index(&ds, prop_name).await? { + if !db.storage().has_vector_index(&ds, prop_name).await? { return Ok(true); } } @@ -389,22 +389,22 @@ async fn needs_index_work_edge( table_branch: Option<&str>, ) -> Result<bool> { let ds = db - .table_store + .storage() .open_dataset_head_for_write(table_key, full_path, table_branch) .await?; - if db.table_store.count_rows(&ds, None).await? == 0 { + if db.storage().count_rows(&ds, None).await? == 0 { return Ok(false); } - Ok(!db.table_store.has_btree_index(&ds, "id").await? - || !db.table_store.has_btree_index(&ds, "src").await? - || !db.table_store.has_btree_index(&ds, "dst").await?) + Ok(!db.storage().has_btree_index(&ds, "id").await? + || !db.storage().has_btree_index(&ds, "src").await? + || !db.storage().has_btree_index(&ds, "dst").await?) } pub(super) async fn open_for_mutation( db: &Omnigraph, table_key: &str, op_kind: crate::db::MutationOpKind, -) -> Result<(Dataset, String, Option<String>)> { +) -> Result<(SnapshotHandle, String, Option<String>)> { let current_branch = db .coordinator .read() @@ -425,7 +425,7 @@ pub(super) async fn open_for_mutation_on_branch( branch: Option<&str>, table_key: &str, op_kind: crate::db::MutationOpKind, -) -> Result<(Dataset, String, Option<String>)> { +) -> Result<(SnapshotHandle, String, Option<String>)> { db.ensure_schema_apply_not_locked("write").await?; let resolved = db.resolved_branch_target(branch).await?; let entry = resolved @@ -436,11 +436,11 @@ pub(super) async fn open_for_mutation_on_branch( match resolved.branch.as_deref() { None => { let ds = db - .table_store + .storage() .open_dataset_head_for_write(table_key, &full_path, None) .await?; if op_kind.strict_pre_stage_version_check() { - db.table_store + db.storage() .ensure_expected_version(&ds, table_key, entry.table_version)?; } Ok((ds, full_path, None)) @@ -469,15 +469,15 @@ pub(super) async fn open_owned_dataset_for_branch_write( entry_version: u64, active_branch: &str, op_kind: crate::db::MutationOpKind, -) -> Result<(Dataset, Option<String>)> { +) -> Result<(SnapshotHandle, Option<String>)> { match entry_branch { Some(branch) if branch == active_branch => { let ds = db - .table_store + .storage() .open_dataset_head_for_write(table_key, full_path, Some(active_branch)) .await?; if op_kind.strict_pre_stage_version_check() { - db.table_store + db.storage() .ensure_expected_version(&ds, table_key, entry_version)?; } Ok((ds, Some(active_branch.to_string()))) @@ -509,11 +509,11 @@ pub(super) async fn open_owned_dataset_for_branch_write( ) .await?; let ds = db - .table_store + .storage() .open_dataset_head_for_write(table_key, full_path, Some(active_branch)) .await?; if op_kind.strict_pre_stage_version_check() { - db.table_store + db.storage() .ensure_expected_version(&ds, table_key, entry_version)?; } Ok((ds, Some(active_branch.to_string()))) @@ -528,8 +528,8 @@ pub(super) async fn fork_dataset_from_entry_state( source_branch: Option<&str>, source_version: u64, active_branch: &str, -) -> Result<Dataset> { - db.table_store +) -> Result<SnapshotHandle> { + db.storage() .fork_branch_from_state( full_path, source_branch, @@ -547,10 +547,10 @@ pub(super) async fn reopen_for_mutation( table_branch: Option<&str>, expected_version: u64, op_kind: crate::db::MutationOpKind, -) -> Result<Dataset> { +) -> Result<SnapshotHandle> { db.ensure_schema_apply_not_locked("write").await?; if op_kind.strict_pre_stage_version_check() { - db.table_store + db.storage() .reopen_for_mutation(full_path, table_branch, table_key, expected_version) .await } else { @@ -563,7 +563,7 @@ pub(super) async fn reopen_for_mutation( // genuine cross-process drift as 409. See // [`crate::db::MutationOpKind`] for the policy rationale. let _ = expected_version; - db.table_store + db.storage() .open_dataset_head_for_write(table_key, full_path, table_branch) .await } @@ -574,8 +574,8 @@ pub(super) async fn open_dataset_at_state( table_path: &str, table_branch: Option<&str>, table_version: u64, -) -> Result<Dataset> { - db.table_store +) -> Result<SnapshotHandle> { + db.storage() .open_dataset_at_state(table_path, table_branch, table_version) .await } @@ -583,7 +583,7 @@ pub(super) async fn open_dataset_at_state( pub(super) async fn build_indices_on_dataset( db: &Omnigraph, table_key: &str, - ds: &mut Dataset, + ds: &mut SnapshotHandle, ) -> Result<()> { let catalog = db.catalog(); build_indices_on_dataset_for_catalog(db, &catalog, table_key, ds).await @@ -593,10 +593,10 @@ pub(super) async fn build_indices_on_dataset_for_catalog( db: &Omnigraph, catalog: &Catalog, table_key: &str, - ds: &mut Dataset, + ds: &mut SnapshotHandle, ) -> Result<()> { if let Some(type_name) = table_key.strip_prefix("node:") { - if !db.table_store.has_btree_index(ds, "id").await? { + if !db.storage().has_btree_index(ds, "id").await? { stage_and_commit_btree(db, table_key, ds, &["id"]).await?; } @@ -616,19 +616,20 @@ pub(super) async fn build_indices_on_dataset_for_catalog( let prop_name = &index_cols[0]; if let Some(prop_type) = node_type.properties.get(prop_name) { if matches!(prop_type.scalar, ScalarType::String) && !prop_type.list { - if !db.table_store.has_fts_index(ds, prop_name).await? { + if !db.storage().has_fts_index(ds, prop_name).await? { stage_and_commit_inverted(db, table_key, ds, prop_name.as_str()) .await?; } } else if matches!(prop_type.scalar, ScalarType::Vector(_)) && !prop_type.list { - if !db.table_store.has_vector_index(ds, prop_name).await? { - // Inline-commit residual: lance-4.0.0 does not + if !db.storage().has_vector_index(ds, prop_name).await? { + // Inline-commit residual: lance-6.0.1 does not // expose `build_index_metadata_from_segments` as // `pub`, so vector indices cannot be staged from // outside the lance crate. Document at the call // site; companion ticket to lance-format/lance#6658. - db.table_store - .create_vector_index(ds, prop_name.as_str()) + let new_snap = db + .storage_inline_residual() + .create_vector_index(ds.clone(), prop_name.as_str()) .await .map_err(|e| { OmniError::Lance(format!( @@ -636,6 +637,7 @@ pub(super) async fn build_indices_on_dataset_for_catalog( table_key, prop_name, e )) })?; + *ds = new_snap; } } } @@ -645,13 +647,13 @@ pub(super) async fn build_indices_on_dataset_for_catalog( } if table_key.starts_with("edge:") { - if !db.table_store.has_btree_index(ds, "id").await? { + if !db.storage().has_btree_index(ds, "id").await? { stage_and_commit_btree(db, table_key, ds, &["id"]).await?; } - if !db.table_store.has_btree_index(ds, "src").await? { + if !db.storage().has_btree_index(ds, "src").await? { stage_and_commit_btree(db, table_key, ds, &["src"]).await?; } - if !db.table_store.has_btree_index(ds, "dst").await? { + if !db.storage().has_btree_index(ds, "dst").await? { stage_and_commit_btree(db, table_key, ds, &["dst"]).await?; } return Ok(()); @@ -674,11 +676,11 @@ pub(super) async fn build_indices_on_dataset_for_catalog( async fn stage_and_commit_btree( db: &Omnigraph, table_key: &str, - ds: &mut Dataset, + ds: &mut SnapshotHandle, columns: &[&str], ) -> Result<()> { let staged = db - .table_store + .storage() .stage_create_btree_index(ds, columns) .await .map_err(|e| { @@ -693,8 +695,8 @@ async fn stage_and_commit_btree( // yet called) leaves no Lance-HEAD drift on the touched table. crate::failpoints::maybe_fail("ensure_indices.post_stage_pre_commit_btree")?; let new_ds = db - .table_store - .commit_staged(Arc::new(ds.clone()), staged.transaction) + .storage() + .commit_staged(ds.clone(), staged) .await .map_err(|e| { OmniError::Lance(format!( @@ -711,11 +713,11 @@ async fn stage_and_commit_btree( async fn stage_and_commit_inverted( db: &Omnigraph, table_key: &str, - ds: &mut Dataset, + ds: &mut SnapshotHandle, column: &str, ) -> Result<()> { let staged = db - .table_store + .storage() .stage_create_inverted_index(ds, column) .await .map_err(|e| { @@ -725,8 +727,8 @@ async fn stage_and_commit_inverted( )) })?; let new_ds = db - .table_store - .commit_staged(Arc::new(ds.clone()), staged.transaction) + .storage() + .commit_staged(ds.clone(), staged) .await .map_err(|e| { OmniError::Lance(format!( @@ -777,7 +779,7 @@ async fn prepare_updates_for_commit( ) .await?; build_indices_on_dataset(db, &prepared_update.table_key, &mut ds).await?; - let state = db.table_store.table_state(&full_path, &ds).await?; + let state = db.storage().table_state(&full_path, &ds).await?; prepared_update.table_version = state.version; prepared_update.row_count = state.row_count; prepared_update.version_metadata = state.version_metadata; diff --git a/crates/omnigraph/src/exec/merge.rs b/crates/omnigraph/src/exec/merge.rs index 1068f90..f245d15 100644 --- a/crates/omnigraph/src/exec/merge.rs +++ b/crates/omnigraph/src/exec/merge.rs @@ -926,7 +926,7 @@ async fn publish_adopted_source_state( target_branch, ) .await?; - let state = target_db.table_store().table_state(&full_path, &ds).await?; + let state = target_db.storage().table_state(&full_path, &ds).await?; Ok(crate::db::SubTableUpdate { table_key: table_key.to_string(), table_version: state.version, @@ -963,9 +963,13 @@ async fn publish_rewritten_merge_table( // commit point, narrowed from the previous "merge_insert + delete + // index" multi-step inline-commit chain. if let Some(delta) = &staged.delta_staged { + // The staged delta dataset is a temp-dir Lance dataset used only + // to collect the rewrite batches; wrap it in a `SnapshotHandle` + // so we can route through the trait's `scan_batches_for_rewrite`. + let delta_snapshot = SnapshotHandle::new(delta.dataset.clone()); let batches: Vec<RecordBatch> = target_db - .table_store() - .scan_batches_for_rewrite(&delta.dataset) + .storage() + .scan_batches_for_rewrite(&delta_snapshot) .await? .into_iter() .filter(|batch| batch.num_rows() > 0) @@ -980,7 +984,7 @@ async fn publish_rewritten_merge_table( .map_err(|e| OmniError::Lance(e.to_string()))? }; let staged_merge = target_db - .table_store() + .storage() .stage_merge_insert( current_ds.clone(), combined, @@ -990,15 +994,15 @@ async fn publish_rewritten_merge_table( ) .await?; current_ds = target_db - .table_store() - .commit_staged(Arc::new(current_ds), staged_merge.transaction) + .storage() + .commit_staged(current_ds, staged_merge) .await?; } } // Phase 2: delete removed rows via deletion vectors. // - // INLINE-COMMIT RESIDUAL: lance-4.0.0 does not expose a public + // INLINE-COMMIT RESIDUAL: lance-6.0.1 does not expose a public // two-phase delete API (DeleteJob is `pub(crate)` — // lance-format/lance#6658 is open with no PRs). We deliberately do // NOT introduce a `stage_delete` wrapper that would secretly @@ -1012,10 +1016,11 @@ async fn publish_rewritten_merge_table( .map(|id| format!("'{}'", id.replace('\'', "''"))) .collect(); let filter = format!("id IN ({})", escaped.join(", ")); - target_db - .table_store() - .delete_where(&full_path, &mut current_ds, &filter) + let (new_ds, _) = target_db + .storage_inline_residual() + .delete_where(&full_path, current_ds, &filter) .await?; + current_ds = new_ds; } // Phase 3: rebuild indices. @@ -1024,9 +1029,9 @@ async fn publish_rewritten_merge_table( // `stage_create_inverted_index` + `commit_staged` for scalar // indices. Vector indices remain inline-commit // (`build_index_metadata_from_segments` is `pub(crate)` in lance- - // 4.0.0 — companion ticket to lance-format/lance#6658). + // 6.0.1 — companion ticket to lance-format/lance#6666). let row_count = target_db - .table_store() + .storage() .table_state(&full_path, ¤t_ds) .await? .row_count; @@ -1036,7 +1041,7 @@ async fn publish_rewritten_merge_table( .await?; } let final_state = target_db - .table_store() + .storage() .table_state(&full_path, ¤t_ds) .await?; @@ -1362,7 +1367,7 @@ impl Omnigraph { let entry = target_snapshot.entry(table_key)?; Some(crate::db::manifest::SidecarTablePin { table_key: table_key.clone(), - table_path: self.table_store().dataset_uri(&entry.table_path), + table_path: self.storage().dataset_uri(&entry.table_path), expected_version: entry.table_version, post_commit_pin: entry.table_version + 1, // Use the merge target branch (where commits actually diff --git a/crates/omnigraph/src/exec/mod.rs b/crates/omnigraph/src/exec/mod.rs index ce72d42..4076414 100644 --- a/crates/omnigraph/src/exec/mod.rs +++ b/crates/omnigraph/src/exec/mod.rs @@ -40,6 +40,7 @@ use crate::db::{ReadTarget, Snapshot}; use crate::embedding::EmbeddingClient; use crate::error::{MergeConflict, MergeConflictKind, OmniError, Result}; use crate::graph_index::GraphIndex; +use crate::storage_layer::SnapshotHandle; use tempfile::{Builder as TempDirBuilder, TempDir}; mod merge; diff --git a/crates/omnigraph/src/exec/mutation.rs b/crates/omnigraph/src/exec/mutation.rs index 0e7ded7..e537d0d 100644 --- a/crates/omnigraph/src/exec/mutation.rs +++ b/crates/omnigraph/src/exec/mutation.rs @@ -428,12 +428,11 @@ async fn ensure_node_id_exists( let filter = format!("id = '{}'", id.replace('\'', "''")); let snapshot = db.snapshot_for_branch(branch).await?; - let ds = snapshot.open(&table_key).await?; - let exists = ds - .count_rows(Some(filter)) - .await - .map_err(|e| OmniError::Lance(e.to_string()))? - > 0; + let ds = db + .storage() + .open_snapshot_at_table(&snapshot, &table_key) + .await?; + let exists = db.storage().count_rows(&ds, Some(filter)).await? > 0; if exists { Ok(()) @@ -602,7 +601,7 @@ async fn open_table_for_mutation( branch: Option<&str>, table_key: &str, op_kind: crate::db::MutationOpKind, -) -> Result<(Dataset, String, Option<String>)> { +) -> Result<(SnapshotHandle, String, Option<String>)> { if let Some(prior) = staging.inline_committed.get(table_key) { let path = staging.paths.get(table_key).ok_or_else(|| { OmniError::manifest_internal(format!( @@ -624,7 +623,7 @@ async fn open_table_for_mutation( let (ds, full_path, table_branch) = db .open_for_mutation_on_branch(branch, table_key, op_kind) .await?; - let expected_version = ds.version().version; + let expected_version = ds.version(); staging.ensure_path( table_key, full_path.clone(), @@ -640,7 +639,7 @@ async fn open_table_for_mutation( /// /// Reason: under the staged-write writer, inserts and updates /// accumulate in memory and commit at end-of-query, while deletes still -/// inline-commit (Lance lacks a public two-phase delete in 4.0.0). +/// inline-commit (Lance lacks a public two-phase delete in 6.0.1). /// Mixing creates ordering hazards (same-row insert→delete becomes a no-op /// because the staged insert isn't visible to delete; cascading deletes /// of just-inserted edges break referential integrity by silent design). @@ -1056,7 +1055,7 @@ impl Omnigraph { // and a chained `update where <pred>` can match a row whose // pending value no longer satisfies <pred>. let batches = self - .table_store() + .storage() .scan_with_pending( &ds, pending_batches, @@ -1154,13 +1153,13 @@ impl Omnigraph { crate::db::MutationOpKind::Delete, ) .await?; - let initial_version = ds.version().version; + let initial_version = ds.version(); // Scan matching IDs for cascade. Per D₂ this never overlaps with // staged inserts (mixed insert/delete in one query is rejected at // parse time), so we scan committed only. let batches = self - .table_store() + .storage() .scan(&ds, Some(&["id"]), Some(&pred_sql), None) .await?; @@ -1188,11 +1187,11 @@ impl Omnigraph { let affected_nodes = deleted_ids.len(); // Delete nodes — still inline-commit (Lance's `Dataset::delete` is - // not exposed as a two-phase op in 4.0.0). D₂ keeps inserts and + // not exposed as a two-phase op in 6.0.1). D₂ keeps inserts and // deletes from coexisting in one query, so this advance of Lance // HEAD is the only HEAD movement during the query and the // publisher's CAS captures it intact. - let mut ds = self + let ds = self .reopen_for_mutation( &table_key, &full_path, @@ -1202,9 +1201,9 @@ impl Omnigraph { ) .await?; crate::failpoints::maybe_fail("mutation.delete_node_pre_primary_delete")?; - let delete_state = self - .table_store() - .delete_where(&full_path, &mut ds, &pred_sql) + let (_new_ds, delete_state) = self + .storage_inline_residual() + .delete_where(&full_path, ds, &pred_sql) .await?; staging.record_inline(crate::db::SubTableUpdate { @@ -1243,7 +1242,7 @@ impl Omnigraph { let edge_table_key = format!("edge:{}", edge_name); let cascade_filter = cascade_filters.join(" OR "); - let (mut edge_ds, edge_full_path, edge_table_branch) = open_table_for_mutation( + let (edge_ds, edge_full_path, edge_table_branch) = open_table_for_mutation( self, staging, branch, @@ -1252,9 +1251,9 @@ impl Omnigraph { ) .await?; - let edge_delete = self - .table_store() - .delete_where(&edge_full_path, &mut edge_ds, &cascade_filter) + let (_new_edge_ds, edge_delete) = self + .storage_inline_residual() + .delete_where(&edge_full_path, edge_ds, &cascade_filter) .await?; affected_edges += edge_delete.deleted_rows; @@ -1291,7 +1290,7 @@ impl Omnigraph { let pred_sql = predicate_to_sql(predicate, params, true)?; let table_key = format!("edge:{}", type_name); - let (mut ds, full_path, table_branch) = open_table_for_mutation( + let (ds, full_path, table_branch) = open_table_for_mutation( self, staging, branch, @@ -1300,9 +1299,9 @@ impl Omnigraph { ) .await?; - let delete_state = self - .table_store() - .delete_where(&full_path, &mut ds, &pred_sql) + let (_new_ds, delete_state) = self + .storage_inline_residual() + .delete_where(&full_path, ds, &pred_sql) .await?; let affected = delete_state.deleted_rows; @@ -1356,7 +1355,7 @@ fn concat_match_batches_to_schema( /// dedup needed (`dedupe_key_column = None`). async fn validate_edge_cardinality_with_pending( db: &Omnigraph, - committed_ds: &Dataset, + committed_ds: &SnapshotHandle, staging: &MutationStaging, table_key: &str, edge_type: &omnigraph_compiler::catalog::EdgeType, diff --git a/crates/omnigraph/src/exec/staging.rs b/crates/omnigraph/src/exec/staging.rs index 264ab59..a3932b0 100644 --- a/crates/omnigraph/src/exec/staging.rs +++ b/crates/omnigraph/src/exec/staging.rs @@ -21,9 +21,10 @@ use std::collections::{HashMap, HashSet}; use std::sync::Arc; +use crate::storage_layer::{SnapshotHandle, StagedHandle}; use arrow_array::{Array, RecordBatch, StringArray, UInt32Array}; use arrow_schema::SchemaRef; -use lance::Dataset; +use futures::stream::StreamExt; use omnigraph_compiler::catalog::EdgeType; use crate::db::manifest::{ @@ -32,15 +33,13 @@ use crate::db::manifest::{ use crate::db::{MutationOpKind, SubTableUpdate}; use crate::error::{OmniError, Result}; -/// Whether the per-table accumulator should commit via `stage_append` -/// (no @key inserts, edge inserts) or `stage_merge_insert` (any @key insert -/// or update). Once set to `Merge` for a table within a query, subsequent -/// inserts on that table are rolled into the same merge — a `WhenNotMatched -/// = InsertAll` merge is correct for both cases. +/// Whether the per-table accumulator should commit via `stage_append`, +/// `stage_merge_insert`, or `stage_overwrite`. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub(crate) enum PendingMode { Append, Merge, + Overwrite, } /// Per-table accumulator. Each insert/update op pushes a `RecordBatch` into @@ -158,9 +157,9 @@ impl MutationStaging { mode: PendingMode, batch: RecordBatch, ) -> Result<()> { - if batch.num_rows() == 0 { - // No-op — staging is purely additive; an empty batch should not - // be appended. + if batch.num_rows() == 0 && mode != PendingMode::Overwrite { + // No-op for additive modes. For Overwrite, an empty batch is + // observable: it means "replace this table with zero rows". return Ok(()); } // If we've already accumulated a batch on this table, the new @@ -174,6 +173,14 @@ impl MutationStaging { // caller a clearer point of failure attached to the specific // op that introduced the drift. if let Some(existing) = self.pending.get(table_key) { + if existing.mode == PendingMode::Overwrite || mode == PendingMode::Overwrite { + if existing.mode != mode { + return Err(OmniError::manifest_internal(format!( + "table '{}' cannot mix overwrite staging with append/merge staging", + table_key + ))); + } + } if !schemas_compatible(&existing.schema, &batch.schema()) { return Err(OmniError::manifest(format!( "table '{}' accumulated mutation batches with mismatched schemas: \ @@ -194,8 +201,9 @@ impl MutationStaging { .pending .entry(table_key.to_string()) .or_insert_with(|| PendingTable::new(schema.clone(), mode)); - // Upgrade Append -> Merge if any op needs merge semantics. - if mode == PendingMode::Merge { + // Upgrade Append -> Merge if any op needs merge semantics. Overwrite + // is never mixed with additive modes (guarded above). + if mode == PendingMode::Merge && entry.mode == PendingMode::Append { entry.mode = PendingMode::Merge; } entry.batches.push(batch); @@ -217,6 +225,11 @@ impl MutationStaging { .unwrap_or(&[]) } + /// Accumulator mode for `table_key`, if this query has touched it. + pub(crate) fn pending_mode(&self, table_key: &str) -> Option<PendingMode> { + self.pending.get(table_key).map(|p| p.mode) + } + /// Schema of the accumulated batches for `table_key`, or `None` if no /// op has touched the table. Used by `scan_with_pending` to construct /// the in-memory `MemTable`. @@ -249,9 +262,21 @@ impl MutationStaging { /// Lance datasets is a perf follow-up; same loop structure as the /// pre-split `finalize`. pub(crate) async fn stage_all( + self, + db: &crate::db::Omnigraph, + branch: Option<&str>, + ) -> Result<StagedMutation> { + self.stage_all_with_concurrency(db, branch, 1).await + } + + /// Loader-facing variant of [`stage_all`] that preserves + /// `OMNIGRAPH_LOAD_CONCURRENCY` for the fragment-writing stage while + /// still leaving all Lance HEAD movement to [`StagedMutation::commit_all`]. + pub(crate) async fn stage_all_with_concurrency( self, db: &crate::db::Omnigraph, _branch: Option<&str>, + concurrency: usize, ) -> Result<StagedMutation> { let MutationStaging { expected_versions, @@ -261,7 +286,8 @@ impl MutationStaging { op_kinds, } = self; - let mut staged_entries: Vec<StagedTableEntry> = Vec::with_capacity(pending.len()); + let mut stage_inputs: Vec<(String, PendingTable, StagedTablePath, u64)> = + Vec::with_capacity(pending.len()); for (table_key, table) in pending { let path = paths.get(&table_key).cloned().ok_or_else(|| { OmniError::manifest_internal(format!( @@ -275,77 +301,22 @@ impl MutationStaging { table_key )) })?; - - // Reopen the dataset for staging. The op_kind reflects the - // accumulated PendingTable's mode: Append-mode batches are - // INSERT-shaped (no key-based dedup at commit_staged); Merge- - // mode batches are MERGE-shaped (key-dedup at commit_staged). - // Both skip the strict pre-stage version check under the - // [`MutationOpKind`] policy: Lance's natural rebase + the - // per-(table, branch) queue + the publisher CAS in - // `commit_all` handle drift; the strict check would - // over-reject in-process concurrent inserts (PR 2 / MR-686 - // Phase 2). - let stage_kind = match table.mode { - PendingMode::Append => crate::db::MutationOpKind::Insert, - PendingMode::Merge => crate::db::MutationOpKind::Merge, - }; - let ds = db - .reopen_for_mutation( - &table_key, - &path.full_path, - path.table_branch.as_deref(), - expected, - stage_kind, - ) - .await?; - - if table.batches.is_empty() { - continue; - } - - // For Merge mode, dedupe accumulated batches by `id`, keeping - // the LAST occurrence (last-write-wins for the query). This - // is required because Lance's `MergeInsertBuilder` produces - // arbitrary results on duplicate keys in the source. Append - // mode is exempt because no-key node and edge inserts use - // ULID-generated ids that are unique within a query. - let combined = match table.mode { - PendingMode::Merge => dedupe_merge_batches_by_id(&table.schema, table.batches)?, - PendingMode::Append => { - if table.batches.len() == 1 { - table.batches.into_iter().next().unwrap() - } else { - arrow_select::concat::concat_batches(&table.schema, &table.batches) - .map_err(|e| OmniError::Lance(e.to_string()))? - } - } - }; - - // Stage produces uncommitted fragments + transaction. No - // Lance HEAD advance until `commit_all` runs `commit_staged`. - let staged = match table.mode { - PendingMode::Append => db.table_store().stage_append(&ds, combined, &[]).await?, - PendingMode::Merge => { - db.table_store() - .stage_merge_insert( - ds.clone(), - combined, - vec!["id".to_string()], - lance::dataset::WhenMatched::UpdateAll, - lance::dataset::WhenNotMatched::InsertAll, - ) - .await? - } - }; - staged_entries.push(StagedTableEntry { - table_key, - path, - expected_version: expected, - dataset: ds, - staged_write: staged, - }); + stage_inputs.push((table_key, table, path, expected)); } + let concurrency = concurrency.min(stage_inputs.len()).max(1); + let staged_entries = futures::stream::iter(stage_inputs.into_iter().map( + |(table_key, table, path, expected)| async move { + stage_pending_table(db, table_key, table, path, expected).await + }, + )) + .buffered(concurrency) + .collect::<Vec<Result<Option<StagedTableEntry>>>>() + .await + .into_iter() + .collect::<Result<Vec<_>>>()? + .into_iter() + .flatten() + .collect(); Ok(StagedMutation { inline_committed, @@ -357,6 +328,73 @@ impl MutationStaging { } } +async fn stage_pending_table( + db: &crate::db::Omnigraph, + table_key: String, + table: PendingTable, + path: StagedTablePath, + expected: u64, +) -> Result<Option<StagedTableEntry>> { + // Reopen the dataset for staging. Append/Merge can be rebased later by + // Lance + publisher CAS; Overwrite is a strict replacement and uses the + // same SchemaRewrite policy as schema apply. + let stage_kind = match table.mode { + PendingMode::Append => crate::db::MutationOpKind::Insert, + PendingMode::Merge => crate::db::MutationOpKind::Merge, + PendingMode::Overwrite => crate::db::MutationOpKind::SchemaRewrite, + }; + let ds = db + .reopen_for_mutation( + &table_key, + &path.full_path, + path.table_branch.as_deref(), + expected, + stage_kind, + ) + .await?; + + if table.batches.is_empty() { + return Ok(None); + } + + let combined = match table.mode { + PendingMode::Merge => dedupe_merge_batches_by_id(&table.schema, table.batches)?, + PendingMode::Append | PendingMode::Overwrite => { + if table.batches.len() == 1 { + table.batches.into_iter().next().unwrap() + } else { + arrow_select::concat::concat_batches(&table.schema, &table.batches) + .map_err(|e| OmniError::Lance(e.to_string()))? + } + } + }; + + // Stage produces uncommitted fragments + transaction. No Lance HEAD + // advance until `commit_all` runs `commit_staged`. + let staged = match table.mode { + PendingMode::Append => db.storage().stage_append(&ds, combined, &[]).await?, + PendingMode::Merge => { + db.storage() + .stage_merge_insert( + ds.clone(), + combined, + vec!["id".to_string()], + lance::dataset::WhenMatched::UpdateAll, + lance::dataset::WhenNotMatched::InsertAll, + ) + .await? + } + PendingMode::Overwrite => db.storage().stage_overwrite(&ds, combined).await?, + }; + Ok(Some(StagedTableEntry { + table_key, + path, + expected_version: expected, + dataset: ds, + staged_write: staged, + })) +} + /// Output of [`MutationStaging::stage_all`]. Carries the staged Lance /// transactions (Phase A complete; uncommitted fragments written) plus /// the per-table metadata needed to write the recovery sidecar, run @@ -389,15 +427,17 @@ pub(crate) struct StagedMutation { } /// Per-table state captured during `stage_all` and consumed by -/// `commit_all`. Holds the opened `Dataset` so `commit_staged` doesn't -/// re-open, and the `StagedWrite` whose `transaction` `commit_staged` -/// will execute. +/// `commit_all`. Holds the opened snapshot (so `commit_staged` doesn't +/// re-open) plus the staged Lance transaction that `commit_staged` +/// will execute. Both held as opaque `TableStorage` handles per MR-793 +/// §III.9 — the inner `lance::Dataset` / `StagedWrite` are not visible +/// to engine code outside the storage layer. struct StagedTableEntry { table_key: String, path: StagedTablePath, expected_version: u64, - dataset: lance::Dataset, - staged_write: crate::table_store::StagedWrite, + dataset: SnapshotHandle, + staged_write: StagedHandle, } impl StagedMutation { @@ -544,15 +584,14 @@ impl StagedMutation { // raw Lance write or a pre-fix maintenance path moved HEAD without // publishing `__manifest`, this write must not silently fold it. let head = db - .table_store() + .storage() .open_dataset_head_for_write( &entry.table_key, &entry.path.full_path, entry.path.table_branch.as_deref(), ) .await? - .version() - .version; + .version(); if head < current { return Err(OmniError::manifest_internal(format!( "table '{}' Lance HEAD version {} is behind manifest version {}", @@ -672,14 +711,8 @@ impl StagedMutation { staged_write, } = entry; - let new_ds = db - .table_store() - .commit_staged(Arc::new(dataset), staged_write.transaction) - .await?; - let state = db - .table_store() - .table_state(&path.full_path, &new_ds) - .await?; + let new_ds = db.storage().commit_staged(dataset, staged_write).await?; + let state = db.storage().table_state(&path.full_path, &new_ds).await?; updates.push(SubTableUpdate { table_key, table_version: state.version, @@ -813,7 +846,9 @@ fn dedupe_merge_batches_by_id( /// Count edges per `src` value across committed (Lance scan) + pending /// (in-memory). Caller supplies an opened committed dataset so the /// mutation path (which already has one) and the loader path (which -/// opens via snapshot) share the same body. +/// opens via snapshot) share the same body. For overwrite staging, the +/// pending batches are the replacement table image, so committed rows are +/// intentionally skipped. /// /// `dedupe_key_column` controls whether committed rows are shadowed by /// pending: @@ -828,7 +863,7 @@ fn dedupe_merge_batches_by_id( /// `LoadMode::Merge` double-counts. pub(crate) async fn count_src_per_edge( db: &crate::db::Omnigraph, - committed_ds: &Dataset, + committed_ds: &SnapshotHandle, table_key: &str, staging: &MutationStaging, dedupe_key_column: Option<&str>, @@ -859,41 +894,44 @@ pub(crate) async fn count_src_per_edge( _ => None, }; - // Committed side: scan `src` plus the dedupe key column when set, so - // we can both count and shadow in one pass. - let projection: Vec<&str> = match dedupe_key_column { - Some(col) if pending_keys.as_ref().is_some_and(|s| !s.is_empty()) => vec!["src", col], - _ => vec!["src"], - }; - let committed = db - .table_store() - .scan(committed_ds, Some(&projection), None, None) - .await?; - for batch in &committed { - let srcs = batch - .column_by_name("src") - .ok_or_else(|| OmniError::Lance("missing 'src' column on edge table".into()))? - .as_any() - .downcast_ref::<StringArray>() - .ok_or_else(|| OmniError::Lance("'src' column is not Utf8".into()))?; - // Optional shadow-key column (only present when dedupe is on). - let key_arr = match (&pending_keys, dedupe_key_column) { - (Some(set), Some(col)) if !set.is_empty() => batch - .column_by_name(col) - .and_then(|c| c.as_any().downcast_ref::<StringArray>()), - _ => None, + let replace_committed = staging.pending_mode(table_key) == Some(PendingMode::Overwrite); + if !replace_committed { + // Committed side: scan `src` plus the dedupe key column when set, so + // we can both count and shadow in one pass. + let projection: Vec<&str> = match dedupe_key_column { + Some(col) if pending_keys.as_ref().is_some_and(|s| !s.is_empty()) => vec!["src", col], + _ => vec!["src"], }; - for i in 0..srcs.len() { - if !srcs.is_valid(i) { - continue; - } - // Shadow this committed row if its key is in pending. - if let (Some(arr), Some(set)) = (key_arr, pending_keys.as_ref()) { - if arr.is_valid(i) && set.contains(arr.value(i)) { + let committed = db + .storage() + .scan(committed_ds, Some(&projection), None, None) + .await?; + for batch in &committed { + let srcs = batch + .column_by_name("src") + .ok_or_else(|| OmniError::Lance("missing 'src' column on edge table".into()))? + .as_any() + .downcast_ref::<StringArray>() + .ok_or_else(|| OmniError::Lance("'src' column is not Utf8".into()))?; + // Optional shadow-key column (only present when dedupe is on). + let key_arr = match (&pending_keys, dedupe_key_column) { + (Some(set), Some(col)) if !set.is_empty() => batch + .column_by_name(col) + .and_then(|c| c.as_any().downcast_ref::<StringArray>()), + _ => None, + }; + for i in 0..srcs.len() { + if !srcs.is_valid(i) { continue; } + // Shadow this committed row if its key is in pending. + if let (Some(arr), Some(set)) = (key_arr, pending_keys.as_ref()) { + if arr.is_valid(i) && set.contains(arr.value(i)) { + continue; + } + } + *counts.entry(srcs.value(i).to_string()).or_insert(0) += 1; } - *counts.entry(srcs.value(i).to_string()).or_insert(0) += 1; } } diff --git a/crates/omnigraph/src/loader/mod.rs b/crates/omnigraph/src/loader/mod.rs index 707c46a..febbabd 100644 --- a/crates/omnigraph/src/loader/mod.rs +++ b/crates/omnigraph/src/loader/mod.rs @@ -305,8 +305,7 @@ async fn load_jsonl_reader<R: BufRead>( if !catalog.node_types.contains_key(type_name) { return Err(OmniError::manifest(format!( "record {}: unknown node type '{}'", - record_num, - type_name + record_num, type_name ))); } let data = value @@ -321,8 +320,7 @@ async fn load_jsonl_reader<R: BufRead>( if catalog.lookup_edge_by_name(edge_name).is_none() { return Err(OmniError::manifest(format!( "record {}: unknown edge type '{}'", - record_num, - edge_name + record_num, edge_name ))); } let from = value @@ -357,27 +355,23 @@ async fn load_jsonl_reader<R: BufRead>( } // Phase 2: Build per-type RecordBatches and accumulate into the - // staging pipeline. For Append/Merge, batches go into an in-memory - // accumulator and a single `stage_*` + `commit_staged` per touched - // table runs at end-of-load — a mid-load failure (RI / cardinality - // violation) leaves Lance HEAD untouched. For Overwrite, the legacy - // inline-commit path is preserved (truncate+append doesn't fit the - // staged shape cleanly, and overwrite has no in-flight read-your-writes - // requirement). + // staging pipeline. Batches go into an in-memory accumulator and a + // single `stage_*` + `commit_staged` per touched table runs at + // end-of-load — a mid-load failure (RI / cardinality violation) leaves + // Lance HEAD untouched. `LoadMode::Overwrite` uses Lance's staged + // `Overwrite` transaction rather than the former truncate-then-append + // inline path. let mut result = LoadResult::default(); let snapshot = db.snapshot_for_branch(branch).await?; - let use_staging = !matches!(mode, LoadMode::Overwrite); let mut staging = MutationStaging::default(); - let mut overwrite_updates: Vec<crate::db::SubTableUpdate> = Vec::new(); - let mut overwrite_expected: HashMap<String, u64> = HashMap::new(); let pending_mode = match mode { LoadMode::Merge => PendingMode::Merge, // Append-mode loads accumulate as Append. Edge tables (no @key) // and no-key node tables stay safe on the stage_append path. The // Merge mode applies dedupe-by-id; Append assumes unique inputs. LoadMode::Append => PendingMode::Append, - LoadMode::Overwrite => PendingMode::Append, // unused + LoadMode::Overwrite => PendingMode::Overwrite, }; // Map LoadMode to MutationOpKind for the version-check policy. // Append/Merge skip the strict pre-stage check (concurrency-safe @@ -405,81 +399,43 @@ async fn load_jsonl_reader<R: BufRead>( } let loaded_count = batch.num_rows(); let table_key = format!("node:{}", type_name); - let entry = snapshot + let _entry = snapshot .entry(&table_key) .ok_or_else(|| OmniError::manifest(format!("no manifest entry for {}", table_key)))?; - if !use_staging { - overwrite_expected.insert(table_key.clone(), entry.table_version); - } prepared_nodes.push((type_name.clone(), table_key, batch, loaded_count)); } - // Phase 2b: write every node type. Append/Merge → in-memory - // accumulator. Overwrite → concurrent inline-commit (legacy path). - if use_staging { - for (type_name, table_key, batch, loaded_count) in prepared_nodes { - let (ds, full_path, table_branch) = db - .open_for_mutation_on_branch(branch, &table_key, load_op_kind) - .await?; - let expected_version = ds.version().version; - staging.ensure_path( - &table_key, - full_path, - table_branch, - expected_version, - load_op_kind, - ); - let schema = batch.schema(); - staging.append_batch(&table_key, schema, pending_mode, batch)?; - result.nodes_loaded.insert(type_name, loaded_count); - } - } else { - let node_write_results = - write_batches_concurrently(db, branch, mode, prepared_nodes).await?; - for (type_name, table_key, loaded_count, state, table_branch) in node_write_results { - overwrite_updates.push(crate::db::SubTableUpdate { - table_key, - table_version: state.version, - table_branch, - row_count: state.row_count, - version_metadata: state.version_metadata, - }); - result.nodes_loaded.insert(type_name, loaded_count); - } + // Phase 2b: accumulate every node type in memory. Fragment writes are + // delayed until after all validation succeeds. + for (type_name, table_key, batch, loaded_count) in prepared_nodes { + let (ds, full_path, table_branch) = db + .open_for_mutation_on_branch(branch, &table_key, load_op_kind) + .await?; + let expected_version = ds.version(); + staging.ensure_path( + &table_key, + full_path, + table_branch, + expected_version, + load_op_kind, + ); + let schema = batch.schema(); + staging.append_batch(&table_key, schema, pending_mode, batch)?; + result.nodes_loaded.insert(type_name, loaded_count); } // Phase 2c: Validate edge referential integrity — every src/dst must - // reference an existing node ID in the appropriate type. For staged - // loads, the lookup unions snapshot-committed IDs with the in-memory - // pending batches (which carry the just-staged node inserts). + // reference an existing node ID in the appropriate type. For + // Append/Merge the lookup unions snapshot-committed IDs with the + // in-memory pending batches. For Overwrite, a touched node table's + // pending batch is the replacement image, so committed rows are not + // included for that table. for (edge_name, rows) in &edge_rows { let edge_type = &catalog.edge_types[edge_name]; - let from_ids = if use_staging { - collect_node_ids_with_pending(db, branch, &edge_type.from_type, &staging).await? - } else { - collect_node_ids( - db, - branch, - &edge_type.from_type, - &node_rows, - &catalog, - &overwrite_updates, - ) - .await? - }; - let to_ids = if use_staging { - collect_node_ids_with_pending(db, branch, &edge_type.to_type, &staging).await? - } else { - collect_node_ids( - db, - branch, - &edge_type.to_type, - &node_rows, - &catalog, - &overwrite_updates, - ) - .await? - }; + let from_ids = + collect_node_ids_with_pending(db, branch, &edge_type.from_type, &staging).await?; + let to_ids = + collect_node_ids_with_pending(db, branch, &edge_type.to_type, &staging).await?; for (i, (src, dst, _)) in rows.iter().enumerate() { if !from_ids.contains(src.as_str()) { @@ -516,118 +472,72 @@ async fn load_jsonl_reader<R: BufRead>( } let loaded_count = batch.num_rows(); let table_key = format!("edge:{}", edge_name); - let entry = snapshot + let _entry = snapshot .entry(&table_key) .ok_or_else(|| OmniError::manifest(format!("no manifest entry for {}", table_key)))?; - if !use_staging { - overwrite_expected.insert(table_key.clone(), entry.table_version); - } prepared_edges.push((edge_name.clone(), table_key, batch, loaded_count)); } - // Phase 2e: write every edge type. Same dispatch as Phase 2b. - if use_staging { - for (edge_name, table_key, batch, loaded_count) in prepared_edges { - let (ds, full_path, table_branch) = db - .open_for_mutation_on_branch(branch, &table_key, load_op_kind) - .await?; - let expected_version = ds.version().version; - staging.ensure_path( - &table_key, - full_path, - table_branch, - expected_version, - load_op_kind, - ); - let schema = batch.schema(); - staging.append_batch(&table_key, schema, pending_mode, batch)?; - result.edges_loaded.insert(edge_name, loaded_count); - } - } else { - let edge_write_results = - write_batches_concurrently(db, branch, mode, prepared_edges).await?; - for (edge_name, table_key, loaded_count, state, table_branch) in edge_write_results { - overwrite_updates.push(crate::db::SubTableUpdate { - table_key, - table_version: state.version, - table_branch, - row_count: state.row_count, - version_metadata: state.version_metadata, - }); - result.edges_loaded.insert(edge_name, loaded_count); - } + // Phase 2e: accumulate every edge type. Same dispatch as Phase 2b. + for (edge_name, table_key, batch, loaded_count) in prepared_edges { + let (ds, full_path, table_branch) = db + .open_for_mutation_on_branch(branch, &table_key, load_op_kind) + .await?; + let expected_version = ds.version(); + staging.ensure_path( + &table_key, + full_path, + table_branch, + expected_version, + load_op_kind, + ); + let schema = batch.schema(); + staging.append_batch(&table_key, schema, pending_mode, batch)?; + result.edges_loaded.insert(edge_name, loaded_count); } // Phase 3: Validate edge cardinality constraints (before commit — - // invalid data must not be committed). Staged path scans committed - // edges via Lance + iterates pending edges in-memory. Overwrite path - // opens the just-written version (legacy behavior). + // invalid data must not be committed). The helper scans committed + // edges via Lance + iterates pending edges in-memory; for Overwrite it + // treats the pending edge batches as the replacement table image. for (edge_name, _) in &edge_rows { let edge_type = &catalog.edge_types[edge_name]; let table_key = format!("edge:{}", edge_name); - if use_staging { - validate_edge_cardinality_with_pending_loader( - db, branch, edge_type, &table_key, &staging, mode, - ) - .await?; - } else if let Some(update) = overwrite_updates.iter().find(|u| u.table_key == table_key) { - validate_edge_cardinality( - db, - branch, - edge_name, - update.table_version, - update.table_branch.as_deref(), - ) - .await?; - } + validate_edge_cardinality_with_pending_loader( + db, branch, edge_type, &table_key, &staging, mode, + ) + .await?; } // Phase 4: Atomic manifest commit with publisher-level OCC. - if use_staging { - let staged = staging.stage_all(db, branch).await?; - // `_queue_guards` holds per-(table_key, branch) write queues - // across the manifest publish below — see exec/mutation.rs for - // the rationale (interleaving prevention). - let (updates, expected_versions, sidecar_handle, _queue_guards) = staged - .commit_all(db, branch, crate::db::manifest::SidecarKind::Load, actor_id) - .await?; - // Same finalize → publisher residual as mutations: per-table - // staged commits have advanced Lance HEAD, but the manifest - // publish has not run yet. Reuse the mutation failpoint name so - // one failpoint pins the shared `MutationStaging` boundary. - crate::failpoints::maybe_fail("mutation.post_finalize_pre_publisher")?; - db.commit_updates_on_branch_with_expected(branch, &updates, &expected_versions, actor_id) - .await?; - // The recovery sidecar protects the per-table commit_staged → - // manifest publish window. Phase C succeeded — clean up - // best-effort: failing the user here would error out a write - // that already landed durably. - if let Some(handle) = sidecar_handle { - if let Err(err) = - crate::db::manifest::delete_sidecar(&handle, db.storage_adapter()).await - { - tracing::warn!( - error = %err, - operation_id = handle.operation_id.as_str(), - "recovery sidecar cleanup failed; the next open's recovery sweep will resolve it" - ); - } - } - } else { - // LoadMode::Overwrite keeps the legacy inline-commit path — - // truncate-then-append doesn't fit the staged shape (see - // `docs/dev/writes.md` "LoadMode::Overwrite residual"). The recovery - // sidecar is not applicable here because the writer doesn't go - // through MutationStaging; per-table inline commits + a final - // manifest publish handle their own residual via the documented - // operator workflow (re-run overwrite to recover). - db.commit_updates_on_branch_with_expected( - branch, - &overwrite_updates, - &overwrite_expected, - actor_id, - ) + let staged = staging + .stage_all_with_concurrency(db, branch, load_write_concurrency()) .await?; + // `_queue_guards` holds per-(table_key, branch) write queues + // across the manifest publish below — see exec/mutation.rs for + // the rationale (interleaving prevention). + let (updates, expected_versions, sidecar_handle, _queue_guards) = staged + .commit_all(db, branch, crate::db::manifest::SidecarKind::Load, actor_id) + .await?; + // Same finalize → publisher residual as mutations: per-table + // staged commits have advanced Lance HEAD, but the manifest + // publish has not run yet. Reuse the mutation failpoint name so + // one failpoint pins the shared `MutationStaging` boundary. + crate::failpoints::maybe_fail("mutation.post_finalize_pre_publisher")?; + db.commit_updates_on_branch_with_expected(branch, &updates, &expected_versions, actor_id) + .await?; + // The recovery sidecar protects the per-table commit_staged → + // manifest publish window. Phase C succeeded — clean up + // best-effort: failing the user here would error out a write + // that already landed durably. + if let Some(handle) = sidecar_handle { + if let Err(err) = crate::db::manifest::delete_sidecar(&handle, db.storage_adapter()).await { + tracing::warn!( + error = %err, + operation_id = handle.operation_id.as_str(), + "recovery sidecar cleanup failed; the next open's recovery sweep will resolve it" + ); + } } Ok(result) @@ -1157,89 +1067,6 @@ fn load_write_concurrency() -> usize { .unwrap_or(DEFAULT_LOAD_WRITE_CONCURRENCY) } -/// Write a set of prepared `(type_name, table_key, batch, row_count)` tuples -/// concurrently. Returns results in original iteration order so callers can -/// zip them back to per-type metadata. -async fn write_batches_concurrently( - db: &Omnigraph, - branch: Option<&str>, - mode: LoadMode, - prepared: Vec<(String, String, RecordBatch, usize)>, -) -> Result< - Vec<( - String, - String, - usize, - crate::table_store::TableState, - Option<String>, - )>, -> { - use futures::stream::StreamExt; - - if prepared.is_empty() { - return Ok(Vec::new()); - } - - let concurrency = load_write_concurrency().min(prepared.len()).max(1); - - futures::stream::iter(prepared.into_iter().map( - |(type_name, table_key, batch, loaded_count)| async move { - let (state, table_branch) = - write_batch_to_dataset(db, branch, &table_key, batch, mode).await?; - Ok::<_, OmniError>((type_name, table_key, loaded_count, state, table_branch)) - }, - )) - .buffered(concurrency) - .collect::<Vec<Result<_>>>() - .await - .into_iter() - .collect() -} - -async fn write_batch_to_dataset( - db: &Omnigraph, - branch: Option<&str>, - table_key: &str, - batch: RecordBatch, - mode: LoadMode, -) -> Result<(crate::table_store::TableState, Option<String>)> { - let op_kind = match mode { - LoadMode::Append => crate::db::MutationOpKind::Insert, - LoadMode::Merge => crate::db::MutationOpKind::Merge, - LoadMode::Overwrite => crate::db::MutationOpKind::SchemaRewrite, - }; - let (mut ds, full_path, table_branch) = db - .open_for_mutation_on_branch(branch, table_key, op_kind) - .await?; - let table_store = db.table_store(); - - match mode { - LoadMode::Overwrite => { - let state = table_store - .overwrite_batch(&full_path, &mut ds, batch) - .await?; - Ok((state, table_branch)) - } - LoadMode::Append => { - let state = table_store.append_batch(&full_path, &mut ds, batch).await?; - Ok((state, table_branch)) - } - LoadMode::Merge => { - let state = table_store - .merge_insert_batch( - &full_path, - ds, - batch, - vec!["id".to_string()], - lance::dataset::WhenMatched::UpdateAll, - lance::dataset::WhenNotMatched::InsertAll, - ) - .await?; - Ok((state, table_branch)) - } - } -} - fn generate_id() -> String { ulid::Ulid::new().to_string() } @@ -1672,10 +1499,7 @@ pub(crate) async fn validate_edge_cardinality( .await?; // Scan src column, count per source - let batches = db - .table_store() - .scan(&ds, Some(&["src"]), None, None) - .await?; + let batches = db.storage().scan(&ds, Some(&["src"]), None, None).await?; let mut counts: HashMap<String, u32> = HashMap::new(); for batch in &batches { @@ -1766,6 +1590,11 @@ async fn validate_edge_cardinality_with_pending_loader( /// - IDs from the staged loader's pending batches (in-memory; just-staged /// inserts of this type) /// - IDs from the committed sub-table at the pre-load snapshot version +/// +/// For `LoadMode::Overwrite`, if the node table is touched then the pending +/// batches are the replacement image. In that case committed IDs are not +/// included, so edge RI is validated against exactly what the overwrite will +/// publish. async fn collect_node_ids_with_pending( db: &Omnigraph, branch: Option<&str>, @@ -1788,6 +1617,10 @@ async fn collect_node_ids_with_pending( } } + if staging.pending_mode(&table_key) == Some(PendingMode::Overwrite) { + return Ok(ids); + } + // From the committed Lance sub-table at the pre-load snapshot version. let snapshot = db.snapshot_for_branch(branch).await?; let Some(entry) = snapshot.entry(&table_key) else { @@ -1801,10 +1634,7 @@ async fn collect_node_ids_with_pending( ) .await?; - let batches = db - .table_store() - .scan(&ds, Some(&["id"]), None, None) - .await?; + let batches = db.storage().scan(&ds, Some(&["id"]), None, None).await?; for batch in &batches { let id_col = batch @@ -1827,72 +1657,6 @@ async fn collect_node_ids_with_pending( Ok(ids) } -/// Collect all valid node IDs for a given type. Union of: -/// - IDs from the just-loaded batch (in memory, from node_rows) -/// - IDs from the sub-table at the just-written version (if it was updated) -/// - IDs from the sub-table at the snapshot-pinned version (if it was not updated) -async fn collect_node_ids( - db: &Omnigraph, - branch: Option<&str>, - type_name: &str, - node_rows: &HashMap<String, Vec<JsonValue>>, - catalog: &omnigraph_compiler::catalog::Catalog, - updates: &[crate::db::SubTableUpdate], -) -> Result<HashSet<String>> { - let mut ids = HashSet::new(); - - // IDs from the in-memory batch (just loaded in this operation) - if let Some(rows) = node_rows.get(type_name) { - if let Some(node_type) = catalog.node_types.get(type_name) { - if let Some(key_prop) = node_type.key_property() { - for row in rows { - if let Some(id) = row.get(key_prop).and_then(|v| v.as_str()) { - ids.insert(id.to_string()); - } - } - } - } - } - - // IDs from the Lance sub-table - let table_key = format!("node:{}", type_name); - let snapshot = db.snapshot_for_branch(branch).await?; - let Some(entry) = snapshot.entry(&table_key) else { - return Ok(ids); - }; - // Use the just-written version if this type was updated, else snapshot version - let updated = updates - .iter() - .find(|u| u.table_key == table_key) - .map(|u| (u.table_version, u.table_branch.as_deref())); - let (version, branch) = updated.unwrap_or((entry.table_version, entry.table_branch.as_deref())); - let ds = db - .open_dataset_at_state(&entry.table_path, branch, version) - .await?; - - let batches = db - .table_store() - .scan(&ds, Some(&["id"]), None, None) - .await?; - - for batch in &batches { - let id_col = batch - .column_by_name("id") - .unwrap() - .as_any() - .downcast_ref::<StringArray>() - .unwrap(); - for i in 0..batch.num_rows() { - if !id_col.is_valid(i) { - continue; - } - ids.insert(id_col.value(i).to_string()); - } - } - - Ok(ids) -} - #[cfg(test)] mod tests { use super::*; diff --git a/crates/omnigraph/src/storage_layer.rs b/crates/omnigraph/src/storage_layer.rs index dac9482..d2f6b01 100644 --- a/crates/omnigraph/src/storage_layer.rs +++ b/crates/omnigraph/src/storage_layer.rs @@ -7,30 +7,32 @@ //! way for new engine writers to advance Lance HEAD without coupling //! "write bytes" with "advance HEAD" in one Lance API call. //! -//! ## Transitional residuals on the trait +//! ## Inline-commit residuals live on a separate trait //! -//! Several inline-commit methods remain on the trait surface as -//! documented residuals: `delete_where` -//! ([#6658](https://github.com/lance-format/lance/issues/6658) closed -//! 2026-05-14, but the public `DeleteBuilder::execute_uncommitted` API -//! did not backport to the 6.x release line — it first ships in -//! `v7.0.0-beta.10`. Migration to staged two-phase delete is tracked as -//! MR-A and is gated on the Lance v7.x bump, not the current v6.0.1 pin), -//! `create_vector_index` (segment-commit-path requires -//! `build_index_metadata_from_segments` which is `pub(crate)` — see -//! [#6666](https://github.com/lance-format/lance/issues/6666), still open), and the -//! legacy `append_batch` / `merge_insert_batches` / `overwrite_batch` / -//! `create_btree_index` / `create_inverted_index` paths kept while -//! engine call sites finish migrating off of them (Phase 1b / Phase 9 -//! of MR-793). These are named honestly at every call site; the -//! forbidden-API guard test catches direct lance::* misuse outside the -//! storage layer. +//! The inline-commit writes that Lance cannot yet express as +//! stage-then-commit are NOT on `TableStorage`. They sit on +//! [`InlineCommitResidual`], reachable only via +//! `Omnigraph::storage_inline_residual()`, so the default `db.storage()` +//! surface is staged-only and cannot couple "write bytes" with "advance +//! HEAD" — MR-793 acceptance §1 closes by construction. The residuals: +//! +//! * `delete_where` — Lance #6658 (`DeleteBuilder::execute_uncommitted`) +//! did not backport to the 6.x line; it first ships in `v7.0.0-beta.10`. +//! Migration to staged two-phase delete is tracked as MR-A, gated on the +//! Lance v7.x bump. +//! * `create_vector_index` — segment-commit-path needs +//! `build_index_metadata_from_segments`, still `pub(crate)` in Lance +//! 6.0.1 ([#6666](https://github.com/lance-format/lance/issues/6666), +//! open). Scalar indices already stage. +//! +//! Each is named honestly at its call site; the forbidden-API guard test +//! catches direct lance::* misuse outside the storage layer. //! //! ## Sealed //! -//! `TableStorage: sealed::Sealed`. Only types in this crate can implement -//! the trait, so a downstream crate cannot subvert the contract by -//! providing its own impl. +//! Both `TableStorage` and `InlineCommitResidual` are `: sealed::Sealed`. +//! Only types in this crate can implement them, so a downstream crate +//! cannot subvert the contract by providing its own impl. //! //! ## Opaque handles //! @@ -40,15 +42,15 @@ //! through. This aligns with the storage-boundary invariant: //! `lance::Dataset` does not appear in trait signatures. //! -//! ## Migration status (MR-793 PR #70) +//! ## Migration status //! -//! Phases 1a / 2 / 4 / 5 / 6 are landed: trait scaffolding, three new -//! staged primitives (`stage_overwrite`, scalar index staging), and -//! migration of `ensure_indices`, `branch_merge`, `schema_apply` onto -//! the staged surface. Phase 1b (call-site conversion to -//! `Arc<dyn TableStorage>`), Phase 9 (demote unused inline-commit -//! methods to `pub(crate)`), Phase 7 (recovery reconciler — MR-847), -//! and Phase 8 (index reconciler — MR-848) are deferred to follow-ups. +//! Phases 1a / 2 / 4 / 5 / 6 landed in MR-793 PR #70 (trait scaffolding, +//! staged primitives, migration of `ensure_indices` / `branch_merge` / +//! `schema_apply` onto the staged surface). Phase 1b (call-site +//! conversion) and Phase 9 landed in MR-854, which also split the +//! inline-commit residuals onto `InlineCommitResidual` so `db.storage()` +//! is staged-only. Phase 7 (recovery reconciler) shipped as MR-847; +//! Phase 8 (index reconciler) is tracked as MR-848. use std::fmt::Debug; use std::sync::Arc; @@ -105,12 +107,37 @@ impl SnapshotHandle { &self.inner } - /// Take ownership of the inner `Arc<Dataset>`. Used when committing - /// staged writes (the call needs to consume the snapshot). + /// Take ownership of the inner `Arc<Dataset>`. Used by the + /// `TableStorage` impl when an op needs to mutate the dataset in + /// place (commit a staged write, append, overwrite, …). + /// + /// Performance note: callers consume the returned `Arc` via + /// `Arc::try_unwrap(...).unwrap_or_else(|arc| (*arc).clone())`. The + /// fast path (no clone) only fires when the snapshot is single-ref + /// — i.e. the caller dropped every other `SnapshotHandle` clone + /// before calling. Holding parallel clones (e.g. across an `await` + /// point or stashed in a struct) forces a deep `Dataset` clone on + /// every mutating op. Engine callers should pass `SnapshotHandle` + /// by value into the mutating method, not keep a side copy. pub(crate) fn into_arc(self) -> Arc<Dataset> { self.inner } + /// Take ownership of the inner `Dataset` by unwrapping the `Arc` + /// (or cloning if the snapshot is shared). `pub(crate)` — used + /// only by the maintenance path (`optimize`, `cleanup`) which + /// must hand `&mut Dataset` to Lance compaction / cleanup APIs + /// that the `TableStorage` trait does not (and should not) + /// surface. Engine code that participates in the staged-write + /// invariant must stay on the trait methods. + /// + /// Single-ref invariant: same fast-path/clone behavior as + /// `into_arc` — see that method's doc. Drop sibling + /// `SnapshotHandle` clones before calling. + pub(crate) fn into_dataset(self) -> Dataset { + Arc::try_unwrap(self.inner).unwrap_or_else(|arc| (*arc).clone()) + } + // ── public, lance-free accessors ── /// Current Lance manifest version of the snapshot. @@ -208,6 +235,20 @@ pub trait TableStorage: sealed::Sealed + Send + Sync + Debug { async fn delete_branch(&self, dataset_uri: &str, branch: &str) -> Result<()>; + /// Idempotent variant of `delete_branch` used by the best-effort fork + /// reclaim under branch delete (`db/omnigraph.rs::cleanup_deleted_branch_tables`) + /// and by the orphan-fork reconciler in `optimize`. Tolerates an + /// already-absent branch (both Lance's `RefNotFound` and the local-store + /// `NotFound` quirk on a missing `tree/{branch}/` dir). A still-referenced + /// branch (`RefConflict`) still surfaces as `OmniError::Lance`. + async fn force_delete_branch(&self, dataset_uri: &str, branch: &str) -> Result<()>; + + /// List the named Lance branches present on the dataset at `dataset_uri`. + /// The `cleanup` orphan reconciler diffs this against the manifest + /// branch set to find orphaned per-table forks. `main`/default is not a + /// named branch and never appears here. + async fn list_branches(&self, dataset_uri: &str) -> Result<Vec<String>>; + async fn reopen_for_mutation( &self, dataset_uri: &str, @@ -328,74 +369,19 @@ pub trait TableStorage: sealed::Sealed + Send + Sync + Debug { column: &str, ) -> Result<StagedHandle>; - // ── Inline-commit residuals (named honestly per MR-793 §3.2) ────── + // ── Index presence (reads, no HEAD advance) ────────────────────── // - // These methods advance Lance HEAD as a side effect of writing. - // They stay on the trait until the corresponding upstream Lance API - // ships: - // - // * `delete_where` — Lance #6658 (two-phase delete). - // * `create_*_index` — `build_index_metadata_from_segments` is - // `pub(crate)` for vector indices in lance-4.0.0; scalar indices - // migrate to staged in MR-793 Phase 2. - // * `append_batch`, `merge_insert_batches`, `overwrite_batch` — - // legacy paths that will be demoted to `pub(crate)` in MR-793 - // Phase 9 once all engine sites route through the staged - // primitives. - - async fn append_batch( - &self, - dataset_uri: &str, - snapshot: SnapshotHandle, - batch: RecordBatch, - ) -> Result<(SnapshotHandle, TableState)>; - - async fn merge_insert_batches( - &self, - dataset_uri: &str, - snapshot: SnapshotHandle, - batches: Vec<RecordBatch>, - key_columns: Vec<String>, - when_matched: WhenMatched, - when_not_matched: WhenNotMatched, - ) -> Result<TableState>; - - async fn overwrite_batch( - &self, - dataset_uri: &str, - snapshot: SnapshotHandle, - batch: RecordBatch, - ) -> Result<(SnapshotHandle, TableState)>; - - async fn delete_where( - &self, - dataset_uri: &str, - snapshot: SnapshotHandle, - filter: &str, - ) -> Result<(SnapshotHandle, DeleteState)>; + // The inline-commit writes (`delete_where`, `create_vector_index`) are + // deliberately NOT on this trait. They live on + // the separate `InlineCommitResidual` trait, reachable only through + // `Omnigraph::storage_inline_residual()`. As a result the default + // `db.storage()` surface cannot couple "write bytes" with "advance HEAD" + // — closing MR-793 acceptance §1 by construction rather than by review. async fn has_btree_index(&self, snapshot: &SnapshotHandle, column: &str) -> Result<bool>; async fn has_fts_index(&self, snapshot: &SnapshotHandle, column: &str) -> Result<bool>; async fn has_vector_index(&self, snapshot: &SnapshotHandle, column: &str) -> Result<bool>; - async fn create_btree_index( - &self, - snapshot: SnapshotHandle, - columns: &[&str], - ) -> Result<SnapshotHandle>; - - async fn create_inverted_index( - &self, - snapshot: SnapshotHandle, - column: &str, - ) -> Result<SnapshotHandle>; - - async fn create_vector_index( - &self, - snapshot: SnapshotHandle, - column: &str, - ) -> Result<SnapshotHandle>; - // ── URI helpers ──────────────────────────────────────────────────── // // These are pure string formatting; they live on the trait so engine @@ -422,6 +408,38 @@ pub trait TableStorage: sealed::Sealed + Send + Sync + Debug { ) -> Result<DatasetRecordBatchStream>; } +// ─── InlineCommitResidual trait ──────────────────────────────────────────── + +/// Inline-commit residual surface: the writes Lance cannot yet express as a +/// stage-then-commit pair, so they advance Lance HEAD as a side effect of +/// writing. Kept OFF `TableStorage` and reachable only through +/// `Omnigraph::storage_inline_residual()`, so the default `db.storage()` path +/// is staged-only and a new writer cannot reintroduce the write+commit coupling +/// by accident (MR-793 acceptance §1, by construction). +/// +/// Residual reasons (each is named honestly at its call site): +/// * `delete_where` — Lance has no public two-phase delete on the 6.x line +/// (`DeleteBuilder::execute_uncommitted` first ships in v7.x; MR-A / Lance +/// #6658). The D2 parse-time rule + recovery sidecars cover the gap meanwhile. +/// * `create_vector_index` — vector-index segment-commit needs +/// `build_index_metadata_from_segments`, still `pub(crate)` in Lance 6.0.1 +/// (Lance #6666). Scalar indices already stage. +#[async_trait] +pub(crate) trait InlineCommitResidual: sealed::Sealed + Send + Sync + Debug { + async fn delete_where( + &self, + dataset_uri: &str, + snapshot: SnapshotHandle, + filter: &str, + ) -> Result<(SnapshotHandle, DeleteState)>; + + async fn create_vector_index( + &self, + snapshot: SnapshotHandle, + column: &str, + ) -> Result<SnapshotHandle>; +} + // ─── single impl: TableStore ────────────────────────────────────────────── #[async_trait] @@ -496,6 +514,14 @@ impl TableStorage for TableStore { TableStore::delete_branch(self, dataset_uri, branch).await } + async fn force_delete_branch(&self, dataset_uri: &str, branch: &str) -> Result<()> { + TableStore::force_delete_branch(self, dataset_uri, branch).await + } + + async fn list_branches(&self, dataset_uri: &str) -> Result<Vec<String>> { + TableStore::list_branches(self, dataset_uri).await + } + async fn reopen_for_mutation( &self, dataset_uri: &str, @@ -689,61 +715,6 @@ impl TableStorage for TableStore { .map(StagedHandle::new) } - async fn append_batch( - &self, - dataset_uri: &str, - snapshot: SnapshotHandle, - batch: RecordBatch, - ) -> Result<(SnapshotHandle, TableState)> { - let mut ds = Arc::try_unwrap(snapshot.into_arc()).unwrap_or_else(|arc| (*arc).clone()); - let state = TableStore::append_batch(self, dataset_uri, &mut ds, batch).await?; - Ok((SnapshotHandle::new(ds), state)) - } - - async fn merge_insert_batches( - &self, - dataset_uri: &str, - snapshot: SnapshotHandle, - batches: Vec<RecordBatch>, - key_columns: Vec<String>, - when_matched: WhenMatched, - when_not_matched: WhenNotMatched, - ) -> Result<TableState> { - let ds = Arc::try_unwrap(snapshot.into_arc()).unwrap_or_else(|arc| (*arc).clone()); - TableStore::merge_insert_batches( - self, - dataset_uri, - ds, - batches, - key_columns, - when_matched, - when_not_matched, - ) - .await - } - - async fn overwrite_batch( - &self, - dataset_uri: &str, - snapshot: SnapshotHandle, - batch: RecordBatch, - ) -> Result<(SnapshotHandle, TableState)> { - let mut ds = Arc::try_unwrap(snapshot.into_arc()).unwrap_or_else(|arc| (*arc).clone()); - let state = TableStore::overwrite_batch(self, dataset_uri, &mut ds, batch).await?; - Ok((SnapshotHandle::new(ds), state)) - } - - async fn delete_where( - &self, - dataset_uri: &str, - snapshot: SnapshotHandle, - filter: &str, - ) -> Result<(SnapshotHandle, DeleteState)> { - let mut ds = Arc::try_unwrap(snapshot.into_arc()).unwrap_or_else(|arc| (*arc).clone()); - let state = TableStore::delete_where(self, dataset_uri, &mut ds, filter).await?; - Ok((SnapshotHandle::new(ds), state)) - } - async fn has_btree_index(&self, snapshot: &SnapshotHandle, column: &str) -> Result<bool> { TableStore::has_btree_index(self, snapshot.dataset(), column).await } @@ -756,36 +727,6 @@ impl TableStorage for TableStore { TableStore::has_vector_index(self, snapshot.dataset(), column).await } - async fn create_btree_index( - &self, - snapshot: SnapshotHandle, - columns: &[&str], - ) -> Result<SnapshotHandle> { - let mut ds = Arc::try_unwrap(snapshot.into_arc()).unwrap_or_else(|arc| (*arc).clone()); - TableStore::create_btree_index(self, &mut ds, columns).await?; - Ok(SnapshotHandle::new(ds)) - } - - async fn create_inverted_index( - &self, - snapshot: SnapshotHandle, - column: &str, - ) -> Result<SnapshotHandle> { - let mut ds = Arc::try_unwrap(snapshot.into_arc()).unwrap_or_else(|arc| (*arc).clone()); - TableStore::create_inverted_index(self, &mut ds, column).await?; - Ok(SnapshotHandle::new(ds)) - } - - async fn create_vector_index( - &self, - snapshot: SnapshotHandle, - column: &str, - ) -> Result<SnapshotHandle> { - let mut ds = Arc::try_unwrap(snapshot.into_arc()).unwrap_or_else(|arc| (*arc).clone()); - TableStore::create_vector_index(self, &mut ds, column).await?; - Ok(SnapshotHandle::new(ds)) - } - fn root_uri(&self) -> &str { TableStore::root_uri(self) } @@ -815,3 +756,27 @@ impl TableStorage for TableStore { .await } } + +#[async_trait] +impl InlineCommitResidual for TableStore { + async fn delete_where( + &self, + dataset_uri: &str, + snapshot: SnapshotHandle, + filter: &str, + ) -> Result<(SnapshotHandle, DeleteState)> { + let mut ds = Arc::try_unwrap(snapshot.into_arc()).unwrap_or_else(|arc| (*arc).clone()); + let state = TableStore::delete_where(self, dataset_uri, &mut ds, filter).await?; + Ok((SnapshotHandle::new(ds), state)) + } + + async fn create_vector_index( + &self, + snapshot: SnapshotHandle, + column: &str, + ) -> Result<SnapshotHandle> { + let mut ds = Arc::try_unwrap(snapshot.into_arc()).unwrap_or_else(|arc| (*arc).clone()); + TableStore::create_vector_index(self, &mut ds, column).await?; + Ok(SnapshotHandle::new(ds)) + } +} diff --git a/crates/omnigraph/src/table_store.rs b/crates/omnigraph/src/table_store.rs index d786fc4..65123c0 100644 --- a/crates/omnigraph/src/table_store.rs +++ b/crates/omnigraph/src/table_store.rs @@ -2,7 +2,6 @@ use arrow_array::{ Array, ArrayRef, RecordBatch, StringArray, StructArray, UInt8Array, UInt32Array, UInt64Array, }; use arrow_schema::SchemaRef; -use arrow_select::concat::concat_batches; use futures::TryStreamExt; use lance::Dataset; use lance::blob::BlobArrayBuilder; @@ -13,7 +12,7 @@ use lance::dataset::{ CommitBuilder, InsertBuilder, MergeInsertBuilder, WhenMatched, WhenNotMatched, WriteMode, WriteParams, }; -use lance::datatypes::BlobKind; +use lance::datatypes::{BlobKind, Schema as LanceSchema}; use lance::index::DatasetIndexExt; use lance::index::scalar::IndexDetails; use lance_file::version::LanceFileVersion; @@ -725,7 +724,14 @@ impl TableStore { }) } - pub async fn append_batch( + /// Legacy inline-commit append: writes fragments AND commits in one + /// call, advancing Lance HEAD as a side effect. Not on the + /// `TableStorage` trait surface — the staged primitive `stage_append` + /// + `commit_staged` is the engine write path. This inherent + /// `pub(crate)` method survives only for recovery test setup. Do not + /// add new engine call sites — they re-introduce the multi-phase + /// commit drift the trait surface was designed to eliminate. + pub(crate) async fn append_batch( &self, dataset_uri: &str, ds: &mut Dataset, @@ -780,139 +786,7 @@ impl TableStore { } } - pub async fn overwrite_batch( - &self, - dataset_uri: &str, - ds: &mut Dataset, - batch: RecordBatch, - ) -> Result<TableState> { - ds.truncate_table() - .await - .map_err(|e| OmniError::Lance(e.to_string()))?; - self.append_batch(dataset_uri, ds, batch).await - } - - pub async fn overwrite_dataset(dataset_uri: &str, batch: RecordBatch) -> Result<Dataset> { - let reader = arrow_array::RecordBatchIterator::new(vec![Ok(batch.clone())], batch.schema()); - let params = WriteParams { - mode: WriteMode::Overwrite, - enable_stable_row_ids: true, - data_storage_version: Some(LanceFileVersion::V2_2), - allow_external_blob_outside_bases: true, - ..Default::default() - }; - Dataset::write(reader, dataset_uri, Some(params)) - .await - .map_err(|e| OmniError::Lance(e.to_string())) - } - - pub async fn merge_insert_batch( - &self, - dataset_uri: &str, - ds: Dataset, - batch: RecordBatch, - key_columns: Vec<String>, - when_matched: WhenMatched, - when_not_matched: WhenNotMatched, - ) -> Result<TableState> { - if batch.num_rows() == 0 { - return self.table_state(dataset_uri, &ds).await; - } - - // Precondition for the FirstSeen workaround below: every caller of - // this primitive must hand in a source batch that is unique by - // `key_columns`. Without this check, `SourceDedupeBehavior::FirstSeen` - // would silently collapse genuine duplicates instead of erroring. - check_batch_unique_by_keys(&batch, &key_columns, "merge_insert_batch")?; - - // TODO(lance-upstream): MergeInsertBuilder does not accept WriteParams, - // so allow_external_blob_outside_bases cannot be set here. External URI - // blobs via merge_insert (LoadMode::Merge, mutations) are unsupported - // until Lance exposes WriteParams on MergeInsertBuilder. - let ds = Arc::new(ds); - let mut builder = MergeInsertBuilder::try_new(ds, key_columns) - .map_err(|e| OmniError::Lance(e.to_string()))?; - builder.when_matched(when_matched); - builder.when_not_matched(when_not_matched); - // Workaround for a Lance 4.0.x bug class where sequential - // merge_insert calls against rows previously rewritten by - // merge_insert produce a spurious "Ambiguous merge inserts: - // multiple source rows match the same target row on (id = ...)" - // error. Lance's `processed_row_ids: Mutex<HashSet<u64>>` - // (lance-4.0.0 `src/dataset/write/merge_insert.rs:2099`) - // double-processes the same source/target match against - // datasets previously rewritten by merge_insert, and the default - // `SourceDedupeBehavior::Fail` errors on the second insertion. - // `FirstSeen` makes Lance skip the duplicate match instead. - // - // Covers both observed surfaces: - // - PR #98 (sequential `load --mode merge` against same keys). - // - MR-920 (sequential `update T set {f} where x=y` on same row). - // - // Correctness-preserving for OmniGraph because every call path - // that reaches this primitive either pre-dedupes the source batch - // by id, or surfaces a real source dup via the - // `check_batch_unique_by_keys` precondition above (which fires - // before the FirstSeen setter has a chance to silently collapse - // anything): - // - Load path: `enforce_unique_constraints_intra_batch` - // (`loader/mod.rs:1442`) errors on intra-batch `@key` dups. - // - Mutate path: `MutationStaging::finalize` (`exec/staging.rs`) - // accumulates and dedupes by `id`. - // - Branch-merge path: `compute_source_delta` / - // `compute_three_way_delta` (`exec/merge.rs`) walk via - // `OrderedTableCursor` and `push_row` each id at most once. - // So FirstSeen only suppresses the spurious Lance behavior, never - // user data. Pinned by `loader_rejects_intra_batch_duplicate_keys` - // in `tests/consistency.rs` plus the - // `check_batch_unique_by_keys` precondition. - // - // Retire when upstream Lance fixes the bug class. Tracked at - // MR-957; upstream: lance-format/lance#6877. - builder.source_dedupe_behavior(SourceDedupeBehavior::FirstSeen); - let job = builder - .try_build() - .map_err(|e| OmniError::Lance(e.to_string()))?; - - let schema = batch.schema(); - let reader = arrow_array::RecordBatchIterator::new(vec![Ok(batch)], schema); - let (new_ds, _stats) = job - .execute(lance_datafusion::utils::reader_to_stream(Box::new(reader))) - .await - .map_err(|e| OmniError::Lance(e.to_string()))?; - self.table_state(dataset_uri, &new_ds).await - } - - pub async fn merge_insert_batches( - &self, - dataset_uri: &str, - ds: Dataset, - batches: Vec<RecordBatch>, - key_columns: Vec<String>, - when_matched: WhenMatched, - when_not_matched: WhenNotMatched, - ) -> Result<TableState> { - if batches.is_empty() { - return self.table_state(dataset_uri, &ds).await; - } - let batch = if batches.len() == 1 { - batches.into_iter().next().unwrap() - } else { - let schema = batches[0].schema(); - concat_batches(&schema, &batches).map_err(|e| OmniError::Lance(e.to_string()))? - }; - self.merge_insert_batch( - dataset_uri, - ds, - batch, - key_columns, - when_matched, - when_not_matched, - ) - .await - } - - pub async fn delete_where( + pub(crate) async fn delete_where( &self, dataset_uri: &str, ds: &mut Dataset, @@ -1011,7 +885,7 @@ impl TableStore { } }; // Assign real fragment IDs. Lance's `InsertBuilder::execute_uncommitted` - // returns fragments with `id = 0` ("Temporary ID" — see lance-4.0.0 + // returns fragments with `id = 0` ("Temporary ID" — see lance-6.0.1 // `dataset/write.rs:1044/1712`); the real assignment happens during // commit via `Transaction::fragments_with_ids`. Because we expose // these fragments to `scan_with_staged` *before* commit, two staged @@ -1082,11 +956,12 @@ impl TableStore { )); } - // Precondition for FirstSeen below. See the comment on - // `merge_insert_batch` for why this check is here, not on the caller: - // every call path that reaches stage_merge_insert (load, - // MutationStaging::finalize, branch_merge::publish_rewritten_merge_table) - // must hand in a source batch that is unique by `key_columns`. + // Precondition for the FirstSeen workaround below: every call path that + // reaches stage_merge_insert (load, MutationStaging::finalize, + // branch_merge::publish_rewritten_merge_table) must hand in a source + // batch that is unique by `key_columns`. Without this check, + // `SourceDedupeBehavior::FirstSeen` would silently collapse genuine + // duplicates instead of erroring. check_batch_unique_by_keys(&batch, &key_columns, "stage_merge_insert")?; let ds = Arc::new(ds); @@ -1094,11 +969,21 @@ impl TableStore { .map_err(|e| OmniError::Lance(e.to_string()))?; builder.when_matched(when_matched); builder.when_not_matched(when_not_matched); - // See `merge_insert_batch` for the FirstSeen rationale. Workaround - // for the Lance 4.0.x bug class where sequential merge_insert / - // update against rows previously rewritten by merge_insert trips - // Lance's `processed_row_ids` HashSet and errors under the default - // `SourceDedupeBehavior::Fail`. Retire when upstream Lance is fixed. + // Workaround for a Lance bug class where sequential merge_insert calls + // against rows previously rewritten by merge_insert produce a spurious + // "Ambiguous merge inserts: multiple source rows match the same target + // row on (id = ...)" error. Lance's `processed_row_ids: + // Mutex<HashSet<u64>>` (lance-6.0.1 `src/dataset/write/merge_insert.rs`) + // double-processes the same source/target match against datasets + // previously rewritten by merge_insert, and the default + // `SourceDedupeBehavior::Fail` errors on the second insertion; FirstSeen + // makes Lance skip the duplicate match instead. Correctness-preserving + // because every call path pre-dedupes the source batch by id or surfaces + // a real source dup via `check_batch_unique_by_keys` above (load: + // `enforce_unique_constraints_intra_batch`; mutate: + // `MutationStaging::finalize`; branch-merge: the `OrderedTableCursor` + // walk in `exec/merge.rs`). Retire when upstream Lance fixes the bug + // class. Tracked at MR-957; upstream: lance-format/lance#6877. builder.source_dedupe_behavior(SourceDedupeBehavior::FirstSeen); let job = builder .try_build() @@ -1174,40 +1059,51 @@ impl TableStore { /// MR-793 Phase 2: introduces this for the schema_apply rewrite path. /// Lance API verified in `.context/mr-793-design.md` Appendix A.1. pub async fn stage_overwrite(&self, ds: &Dataset, batch: RecordBatch) -> Result<StagedWrite> { - if batch.num_rows() == 0 { - return Err(OmniError::manifest_internal( - "stage_overwrite called with empty batch".to_string(), - )); - } - // `enable_stable_row_ids: true` is defensive — empirically Lance 4.0.0 + // `enable_stable_row_ids: true` is defensive — empirically Lance 6.0.1 // preserves the source dataset's flag through `Operation::Overwrite` // when WriteParams omits it (pinned by // `stage_overwrite_preserves_stable_row_ids` in tests/staged_writes.rs), - // but setting it explicitly matches the public `overwrite_dataset` - // path and keeps the invariant documented at every Overwrite site + // but setting it explicitly keeps the invariant documented at every Overwrite site // (see docs/storage.md "Stable row IDs"). Setting it on an existing // dataset that was created without stable row IDs is a no-op per // Lance's row-id-lineage spec, so this stays correct for legacy // datasets. - let params = WriteParams { - mode: WriteMode::Overwrite, - enable_stable_row_ids: true, - allow_external_blob_outside_bases: true, - ..Default::default() - }; - let transaction = InsertBuilder::new(Arc::new(ds.clone())) - .with_params(¶ms) - .execute_uncommitted(vec![batch]) - .await - .map_err(|e| OmniError::Lance(e.to_string()))?; - let mut new_fragments = match &transaction.operation { - Operation::Overwrite { fragments, .. } => fragments.clone(), - other => { - return Err(OmniError::manifest_internal(format!( - "stage_overwrite: unexpected Lance operation {:?}", - std::mem::discriminant(other) - ))); - } + let (transaction, mut new_fragments) = if batch.num_rows() == 0 { + let schema = LanceSchema::try_from(batch.schema().as_ref()) + .map_err(|e| OmniError::Lance(e.to_string()))?; + let transaction = TransactionBuilder::new( + ds.manifest.version, + Operation::Overwrite { + fragments: Vec::new(), + schema, + config_upsert_values: None, + initial_bases: None, + }, + ) + .build(); + (transaction, Vec::new()) + } else { + let params = WriteParams { + mode: WriteMode::Overwrite, + enable_stable_row_ids: true, + allow_external_blob_outside_bases: true, + ..Default::default() + }; + let transaction = InsertBuilder::new(Arc::new(ds.clone())) + .with_params(¶ms) + .execute_uncommitted(vec![batch]) + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + let new_fragments = match &transaction.operation { + Operation::Overwrite { fragments, .. } => fragments.clone(), + other => { + return Err(OmniError::manifest_internal(format!( + "stage_overwrite: unexpected Lance operation {:?}", + std::mem::discriminant(other) + ))); + } + }; + (transaction, new_fragments) }; // Overwrite REPLACES every committed fragment, and Lance restarts // fragment-ID and row-ID counters at the post-commit version. @@ -1220,7 +1116,7 @@ impl TableStore { // 2) For stable-row-id datasets, assign row_id_meta starting // at 0 (Overwrite is a fresh-start) so `scan_with_staged` // doesn't hit the "Missing row id meta" panic in - // lance-4.0.0 dataset/rowids.rs:22. + // lance-6.0.1 dataset/rowids.rs:22. assign_fragment_ids(&mut new_fragments, 1); if ds.manifest.uses_stable_row_ids() { assign_row_id_meta(&mut new_fragments, 0)?; @@ -1244,7 +1140,7 @@ impl TableStore { /// `IndexMetadata`; we manually wrap it in `Operation::CreateIndex /// { new_indices, removed_indices }` via the public `TransactionBuilder`, /// replicating the simple (non-segment-commit-path) branch of Lance's - /// `CreateIndexBuilder::execute` (lance-4.0.0 `src/index/create.rs:502-512`). + /// `CreateIndexBuilder::execute` (lance-6.0.1 `src/index/create.rs:502-512`). /// /// `removed_indices` mirrors `execute()` lines 466-476: when the /// build replaces an existing same-named index, those entries are @@ -1253,7 +1149,7 @@ impl TableStore { /// MR-793 Phase 2: scalar index types (BTree, Inverted) are /// stage-able. Vector indices are NOT (segment-commit-path requires /// `build_index_metadata_from_segments` which is `pub(crate)` in - /// lance-4.0.0); see `create_vector_index` and Appendix A.3. + /// lance-6.0.1); see `create_vector_index` and Appendix A.3. pub async fn stage_create_btree_index( &self, ds: &Dataset, @@ -1348,7 +1244,7 @@ impl TableStore { /// committed fragments carry; Lance's optimizer drops them from the /// filtered scan even when their data would match. Staged-fragment /// rows are silently absent from the result. `scanner.use_stats(false)` - /// does not fix this in lance 4.0.0. Callers needing correct filtered + /// does not fix this in lance 6.0.1. Callers needing correct filtered /// reads against staged data should use a different strategy — the /// engine's `MutationStaging` accumulator unions in-memory pending /// batches with the committed scan via DataFusion `MemTable` (see @@ -1572,25 +1468,7 @@ impl TableStore { })) } - pub async fn create_btree_index(&self, ds: &mut Dataset, columns: &[&str]) -> Result<()> { - let params = ScalarIndexParams::default(); - ds.create_index_builder(columns, IndexType::BTree, ¶ms) - .replace(true) - .await - .map(|_| ()) - .map_err(|e| OmniError::Lance(e.to_string())) - } - - pub async fn create_inverted_index(&self, ds: &mut Dataset, column: &str) -> Result<()> { - let params = InvertedIndexParams::default(); - ds.create_index_builder(&[column], IndexType::Inverted, ¶ms) - .replace(true) - .await - .map(|_| ()) - .map_err(|e| OmniError::Lance(e.to_string())) - } - - pub async fn create_vector_index(&self, ds: &mut Dataset, column: &str) -> Result<()> { + pub(crate) async fn create_vector_index(&self, ds: &mut Dataset, column: &str) -> Result<()> { let params = lance::index::vector::VectorIndexParams::ivf_flat(1, MetricType::L2); ds.create_index_builder(&[column], IndexType::Vector, ¶ms) .replace(true) @@ -1674,7 +1552,7 @@ fn prior_stages_fragment_count(prior_stages: &[StagedWrite]) -> u64 { } /// Assign sequential fragment IDs starting at `start_id`. Mirrors Lance's -/// commit-time `Transaction::fragments_with_ids` (lance-4.0.0 +/// commit-time `Transaction::fragments_with_ids` (lance-6.0.1 /// `dataset/transaction.rs:1456`) — fragments produced by /// `InsertBuilder::execute_uncommitted` start with `id = 0` as a temporary /// placeholder; we renumber here so they don't collide with committed @@ -1705,7 +1583,7 @@ fn prior_stages_row_count(prior_stages: &[StagedWrite]) -> Result<u64> { /// Assign sequential row IDs to fragments that lack them, starting from /// `start_row_id`. Mirrors the relevant arm of Lance's -/// `Transaction::assign_row_ids` (lance-4.0.0 `dataset/transaction.rs:2682`) +/// `Transaction::assign_row_ids` (lance-6.0.1 `dataset/transaction.rs:2682`) /// for the `row_id_meta = None` case — fragments produced by /// `InsertBuilder::execute_uncommitted` against a stable-row-id dataset. /// @@ -1878,7 +1756,7 @@ fn combine_committed_with_staged(ds: &Dataset, staged: &[StagedWrite]) -> Vec<Fr combined } -/// Precondition guard for `merge_insert_batch` and `stage_merge_insert`. +/// Precondition guard for `stage_merge_insert`. /// Both opt into `SourceDedupeBehavior::FirstSeen` to suppress the Lance /// `processed_row_ids` bug (MR-957). FirstSeen would *also* silently /// collapse genuine duplicate source keys; this check restores fail-fast diff --git a/crates/omnigraph/tests/consistency.rs b/crates/omnigraph/tests/consistency.rs index b16aff9..aab0114 100644 --- a/crates/omnigraph/tests/consistency.rs +++ b/crates/omnigraph/tests/consistency.rs @@ -126,7 +126,7 @@ async fn load_merge_upserts_existing_and_inserts_new() { /// source batch had one row per key. /// /// Triggered by Lance's `processed_row_ids: Mutex<HashSet<u64>>` -/// (lance-4.0.0 `src/dataset/write/merge_insert.rs:2099`) double- +/// (lance-6.0.1 `src/dataset/write/merge_insert.rs:2099`) double- /// processing the same source/target match against datasets previously /// rewritten by merge_insert. Worked around by opting /// `MergeInsertBuilder` into `SourceDedupeBehavior::FirstSeen` in diff --git a/crates/omnigraph/tests/failpoints.rs b/crates/omnigraph/tests/failpoints.rs index d240108..3be0a56 100644 --- a/crates/omnigraph/tests/failpoints.rs +++ b/crates/omnigraph/tests/failpoints.rs @@ -907,6 +907,76 @@ async fn recovery_rolls_forward_load_on_feature_branch() { ); } +#[tokio::test] +async fn recovery_rolls_forward_load_overwrite() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + let operation_id; + let parent_commit_id; + + { + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + load_jsonl(&mut db, helpers::TEST_DATA, LoadMode::Overwrite) + .await + .unwrap(); + parent_commit_id = branch_head_commit_id(dir.path(), "main").await.unwrap(); + + let _failpoint = ScopedFailPoint::new("mutation.post_finalize_pre_publisher", "return"); + let err = db + .load( + "main", + r#"{"type":"Person","data":{"name":"OverwriteLoad","age":41}} +"#, + LoadMode::Overwrite, + ) + .await + .unwrap_err(); + assert!( + err.to_string() + .contains("injected failpoint triggered: mutation.post_finalize_pre_publisher"), + "unexpected error: {err}" + ); + operation_id = single_sidecar_operation_id(dir.path()); + } + + let db = Omnigraph::open(&uri).await.unwrap(); + assert_eq!( + helpers::count_rows(&db, "node:Person").await, + 1, + "overwrite row must be visible after recovery rolls the load forward" + ); + drop(db); + + assert_post_recovery_invariants( + dir.path(), + &operation_id, + RecoveryExpectation::RolledForward { + tables: vec![ + TableExpectation::main("node:Person") + .expected_recovery_parent_commit_id(parent_commit_id) + .follow_up_mutation(FollowUpMutation::new( + "main", + MUTATION_QUERIES, + "insert_person", + mixed_params(&[("$name", "AfterOverwriteLoad")], &[("$age", 42)]), + )), + ], + }, + ) + .await + .unwrap(); + + let db = Omnigraph::open(&uri).await.unwrap(); + assert_eq!( + helpers::count_rows(&db, "node:Person").await, + 2, + "follow-up mutation must succeed after overwrite load recovery" + ); +} + #[tokio::test] async fn recovery_rolls_forward_ensure_indices_on_feature_branch() { use lance::index::DatasetIndexExt; @@ -1132,7 +1202,6 @@ async fn refresh_runs_roll_forward_recovery_in_process() { #[tokio::test] async fn refresh_defers_rollback_eligible_sidecar_to_next_open() { use omnigraph::loader::{LoadMode, load_jsonl}; - use omnigraph::table_store::TableStore; let _scenario = FailScenario::setup(); let dir = tempfile::tempdir().unwrap(); @@ -1162,12 +1231,8 @@ async fn refresh_defers_rollback_eligible_sidecar_to_next_open() { // touching the manifest) so the classifier can reach UnexpectedAtP1 // / UnexpectedMultistep / RolledPastExpected paths that require // a real restore on rollback. - let store = TableStore::new(&uri); let mut ds = lance::Dataset::open(&person_uri).await.unwrap(); - store - .delete_where(&person_uri, &mut ds, "1 = 2") - .await - .unwrap(); + helpers::lance_delete_inline(&mut ds, "1 = 2").await; let head_after_drift = ds.version().version; assert_eq!(head_after_drift, manifest_pin + 1); @@ -1697,8 +1762,9 @@ async fn optimize_phase_b_failure_recovered_on_next_open() { ScopedFailPoint::new("optimize.post_phase_b_pre_manifest_commit", "return"); let err = db.optimize().await.unwrap_err(); assert!( - err.to_string() - .contains("injected failpoint triggered: optimize.post_phase_b_pre_manifest_commit"), + err.to_string().contains( + "injected failpoint triggered: optimize.post_phase_b_pre_manifest_commit" + ), "unexpected error: {err}" ); diff --git a/crates/omnigraph/tests/forbidden_apis.rs b/crates/omnigraph/tests/forbidden_apis.rs index 1936815..e079464 100644 --- a/crates/omnigraph/tests/forbidden_apis.rs +++ b/crates/omnigraph/tests/forbidden_apis.rs @@ -29,15 +29,21 @@ //! the cross-table manifest commit. Documented exception. //! - `crates/omnigraph/src/storage_layer.rs` — IS the trait module. //! -//! ## Transitional allow-list +//! ## Allow-list shape //! -//! The migration of writers onto staged primitives is incremental. -//! Several writers (ensure_indices, branch_merge, schema_apply rewrites) -//! already route through the staged primitives; others (bulk loader, -//! exec/mutation, exec/query) still use the legacy inherent -//! `TableStore` methods — they're not visible at the trait boundary, but -//! they DO call lance types. The file-level allow-list below reflects -//! this transitional state and tightens as call sites migrate. +//! After MR-854, `db.storage()` (`&dyn TableStorage`) exposes only staged +//! primitives + reads. The inline-commit writes live on a separate +//! `InlineCommitResidual` trait reached via +//! `Omnigraph::storage_inline_residual()`, so the default storage surface +//! cannot couple "write bytes" with "advance HEAD" — engine code that +//! wants an inline residual must name the residual accessor explicitly. +//! The only residuals are `delete_where` (Lance #6658 / v7.x) and +//! `create_vector_index` (Lance #6666). The dead legacy methods +//! (trait `append_batch` / `merge_insert_batches`, inherent +//! `merge_insert_batch{,es}`, `create_{btree,inverted}_index`) were +//! removed entirely. This guard's scope is unchanged: it catches direct +//! `lance::*` inline-commit misuse outside the storage layer. The +//! file-level allow-list below matches that boundary. use std::path::{Path, PathBuf}; diff --git a/crates/omnigraph/tests/helpers/mod.rs b/crates/omnigraph/tests/helpers/mod.rs index 0e04aa2..295cab7 100644 --- a/crates/omnigraph/tests/helpers/mod.rs +++ b/crates/omnigraph/tests/helpers/mod.rs @@ -195,6 +195,14 @@ pub async fn diff_since_branch( .await } +/// Advance a Lance dataset HEAD directly from tests without going through +/// OmniGraph's storage residual surface. Used to synthesize uncovered drift. +pub async fn lance_delete_inline(ds: &mut lance::Dataset, filter: &str) -> usize { + let result = ds.delete(filter).await.unwrap(); + *ds = (*result.new_dataset).clone(); + result.num_deleted_rows as usize +} + /// Build a ParamMap from string key-value pairs. pub fn params(pairs: &[(&str, &str)]) -> ParamMap { pairs @@ -258,6 +266,27 @@ pub fn vector_and_string_params( map } +/// Test-only helper: perform a raw `Dataset::append` against Lance, +/// advancing Lance HEAD without going through the manifest. Used by +/// `recovery::*` and `staged_writes::*` tests that deliberately set up +/// HEAD-ahead-of-manifest drift scenarios. +/// +/// This mirrors the body of the engine's inline-commit +/// `TableStore::append_batch` (which is `pub(crate)` after MR-854) — +/// kept here as a test helper because integration tests need to +/// simulate drift without depending on the demoted crate-internal API. +pub async fn lance_append_inline(ds: &mut lance::Dataset, batch: RecordBatch) { + use lance::dataset::{WriteMode, WriteParams}; + let schema = batch.schema(); + let reader = arrow_array::RecordBatchIterator::new(vec![Ok(batch)], schema); + let params = WriteParams { + mode: WriteMode::Append, + allow_external_blob_outside_bases: true, + ..Default::default() + }; + ds.append(reader, Some(params)).await.unwrap(); +} + pub fn s3_test_graph_uri(suite: &str) -> Option<String> { let bucket = std::env::var("OMNIGRAPH_S3_TEST_BUCKET").ok()?; let prefix = std::env::var("OMNIGRAPH_S3_TEST_PREFIX") diff --git a/crates/omnigraph/tests/recovery.rs b/crates/omnigraph/tests/recovery.rs index f6b19e8..37d46cb 100644 --- a/crates/omnigraph/tests/recovery.rs +++ b/crates/omnigraph/tests/recovery.rs @@ -175,7 +175,6 @@ async fn read_only_open_skips_recovery_sweep() { #[tokio::test] async fn recovery_rolls_back_synthetic_drift_on_open() { use omnigraph::loader::{LoadMode, load_jsonl}; - use omnigraph::table_store::TableStore; let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); @@ -202,13 +201,9 @@ async fn recovery_rolls_back_synthetic_drift_on_open() { // residual the sweep recovers from is the manifest-vs-Lance-HEAD gap; // it's agnostic to *what* op caused the gap. let person_uri = node_table_uri(uri, "Person"); - let store = TableStore::new(uri); let mut ds = Dataset::open(&person_uri).await.unwrap(); let head_before_drift = ds.version().version; - let _ = store - .delete_where(&person_uri, &mut ds, "1 = 2") - .await - .unwrap(); + let _ = helpers::lance_delete_inline(&mut ds, "1 = 2").await; let head_after_drift = ds.version().version; assert_eq!( head_after_drift, @@ -290,7 +285,6 @@ async fn recovery_rolls_back_synthetic_drift_on_open() { async fn recovery_rollback_converges_manifest_so_schema_apply_succeeds() { use omnigraph::db::ReadTarget; use omnigraph::loader::{LoadMode, load_jsonl}; - use omnigraph::table_store::TableStore; let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); @@ -310,13 +304,9 @@ async fn recovery_rollback_converges_manifest_so_schema_apply_succeeds() { // Forge a Phase-B residual: advance Person's Lance HEAD without publishing to // the manifest (the manifest pin stays at the load's committed version). let person_uri = node_table_uri(uri, "Person"); - let store = TableStore::new(uri); let mut ds = Dataset::open(&person_uri).await.unwrap(); let manifest_pin = ds.version().version; - let _ = store - .delete_where(&person_uri, &mut ds, "1 = 2") - .await - .unwrap(); + let _ = helpers::lance_delete_inline(&mut ds, "1 = 2").await; drop(ds); // Roll-back-classified sidecar (post_commit_pin != observed head ⇒ @@ -518,7 +508,6 @@ async fn count_recovery_actor_commits(graph_root: &Path) -> usize { #[tokio::test] async fn recovery_rolls_forward_after_phase_b_completes() { use omnigraph::loader::{LoadMode, load_jsonl}; - use omnigraph::table_store::TableStore; let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); @@ -535,16 +524,12 @@ async fn recovery_rolls_forward_after_phase_b_completes() { drop(db); let person_uri = node_table_uri(uri, "Person"); - let store = TableStore::new(uri); let mut ds = Dataset::open(&person_uri).await.unwrap(); let head_before = ds.version().version; // Synthesize a successful Phase B: advance Lance HEAD by one // (delete_where with no-match — no fragment changes, but version bumps). - let _ = store - .delete_where(&person_uri, &mut ds, "1 = 2") - .await - .unwrap(); + let _ = helpers::lance_delete_inline(&mut ds, "1 = 2").await; let head_after = ds.version().version; assert_eq!(head_after, head_before + 1); @@ -728,7 +713,6 @@ async fn recovery_records_rolled_forward_for_stale_sidecar_after_successful_roll #[tokio::test] async fn recovery_rolls_back_records_audit_row_with_recovery_actor() { use omnigraph::loader::{LoadMode, load_jsonl}; - use omnigraph::table_store::TableStore; let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); @@ -742,13 +726,9 @@ async fn recovery_rolls_back_records_audit_row_with_recovery_actor() { drop(db); let person_uri = node_table_uri(uri, "Person"); - let store = TableStore::new(uri); let mut ds = Dataset::open(&person_uri).await.unwrap(); let head_before = ds.version().version; - let _ = store - .delete_where(&person_uri, &mut ds, "1 = 2") - .await - .unwrap(); + let _ = helpers::lance_delete_inline(&mut ds, "1 = 2").await; let head_after = ds.version().version; let _ = head_after; @@ -795,7 +775,6 @@ async fn recovery_rolls_back_records_audit_row_with_recovery_actor() { #[tokio::test] async fn recovery_rolls_forward_with_null_actor() { use omnigraph::loader::{LoadMode, load_jsonl}; - use omnigraph::table_store::TableStore; let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); @@ -809,13 +788,9 @@ async fn recovery_rolls_forward_with_null_actor() { drop(db); let person_uri = node_table_uri(uri, "Person"); - let store = TableStore::new(uri); let mut ds = Dataset::open(&person_uri).await.unwrap(); let head_before = ds.version().version; - let _ = store - .delete_where(&person_uri, &mut ds, "1 = 2") - .await - .unwrap(); + let _ = helpers::lance_delete_inline(&mut ds, "1 = 2").await; let head_after = ds.version().version; // Sidecar with no actor_id (CLI-driven mutation; common case). @@ -871,7 +846,6 @@ async fn recovery_rolls_forward_with_null_actor() { #[tokio::test] async fn recovery_processes_multiple_sidecars_with_fresh_snapshot_per_iter() { use omnigraph::loader::{LoadMode, load_jsonl}; - use omnigraph::table_store::TableStore; let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); @@ -889,21 +863,14 @@ async fn recovery_processes_multiple_sidecars_with_fresh_snapshot_per_iter() { // Synthesize drift on both tables independently. let person_uri = node_table_uri(uri, "Person"); let company_uri = node_table_uri(uri, "Company"); - let store = TableStore::new(uri); let mut person_ds = Dataset::open(&person_uri).await.unwrap(); let person_pre = person_ds.version().version; - let _ = store - .delete_where(&person_uri, &mut person_ds, "1 = 2") - .await - .unwrap(); + let _ = helpers::lance_delete_inline(&mut person_ds, "1 = 2").await; let person_post = person_ds.version().version; let mut company_ds = Dataset::open(&company_uri).await.unwrap(); let company_pre = company_ds.version().version; - let _ = store - .delete_where(&company_uri, &mut company_ds, "1 = 2") - .await - .unwrap(); + let _ = helpers::lance_delete_inline(&mut company_ds, "1 = 2").await; let company_post = company_ds.version().version; // Drop two sidecars; ULID prefix ensures sort order is A then B. @@ -1083,7 +1050,6 @@ async fn recovery_ensure_indices_handles_empty_tables() { #[tokio::test] async fn recovery_multi_sidecar_requires_fresh_snapshot_for_correctness() { use omnigraph::loader::{LoadMode, load_jsonl}; - use omnigraph::table_store::TableStore; let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); @@ -1102,7 +1068,6 @@ async fn recovery_multi_sidecar_requires_fresh_snapshot_for_correctness() { drop(db); let person_uri = node_table_uri(uri, "Person"); - let store = TableStore::new(uri); let mut ds = Dataset::open(&person_uri).await.unwrap(); let v1 = ds.version().version; @@ -1116,23 +1081,9 @@ async fn recovery_multi_sidecar_requires_fresh_snapshot_for_correctness() { // Bypassing __manifest is what `delete_where` and `append_batch` // both do (direct on Lance); using append_batch (instead of no-op // deletes) is what makes the fragment-set differ across versions. - store - .append_batch( - &person_uri, - &mut ds, - person_batch(&[("bob-id", "bob", Some(25))]), - ) - .await - .unwrap(); + helpers::lance_append_inline(&mut ds, person_batch(&[("bob-id", "bob", Some(25))])).await; let v2 = ds.version().version; - store - .append_batch( - &person_uri, - &mut ds, - person_batch(&[("carol-id", "carol", Some(40))]), - ) - .await - .unwrap(); + helpers::lance_append_inline(&mut ds, person_batch(&[("carol-id", "carol", Some(40))])).await; let v3 = ds.version().version; assert_eq!(v2, v1 + 1); assert_eq!(v3, v2 + 1); @@ -1297,14 +1248,7 @@ async fn recovery_classifies_feature_branch_sidecar_against_feature_branch() { .open_dataset_head(&person_uri, feature_branch_name.as_deref()) .await .unwrap(); - store - .append_batch( - &person_uri, - &mut ds, - person_batch(&[("carol-id", "carol", Some(40))]), - ) - .await - .unwrap(); + helpers::lance_append_inline(&mut ds, person_batch(&[("carol-id", "carol", Some(40))])).await; let v_head = ds.version().version; assert_eq!(v_head, v_pin + 1, "append must advance HEAD by 1"); @@ -1419,14 +1363,7 @@ async fn recovery_rolls_back_feature_branch_sidecar_against_feature_branch() { .open_dataset_head(&person_uri, feature_branch_name.as_deref()) .await .unwrap(); - store - .append_batch( - &person_uri, - &mut ds, - person_batch(&[("dave-id", "dave", Some(50))]), - ) - .await - .unwrap(); + helpers::lance_append_inline(&mut ds, person_batch(&[("dave-id", "dave", Some(50))])).await; let v_head = ds.version().version; assert_eq!(v_head, v_pin + 1); diff --git a/crates/omnigraph/tests/staged_writes.rs b/crates/omnigraph/tests/staged_writes.rs index 5335057..3771ad4 100644 --- a/crates/omnigraph/tests/staged_writes.rs +++ b/crates/omnigraph/tests/staged_writes.rs @@ -23,6 +23,9 @@ use arrow_schema::{DataType, Field, Schema}; use futures::TryStreamExt; use lance::Dataset; use lance::dataset::{WhenMatched, WhenNotMatched}; +use lance::index::DatasetIndexExt; +use lance_index::IndexType; +use lance_linalg::distance::MetricType; use lance_table::format::Fragment; use omnigraph::table_store::{StagedWrite, TableStore}; use std::sync::Arc; @@ -34,6 +37,22 @@ fn person_schema() -> Arc<Schema> { ])) } +/// Test-only helper: raw `Dataset::append` to advance Lance HEAD without +/// going through the manifest. Mirrors `TableStore::append_batch`'s body +/// (which is `pub(crate)` after MR-854) — kept local so these +/// drift-simulation tests don't depend on the demoted crate-internal API. +async fn lance_append_inline_local(ds: &mut Dataset, batch: RecordBatch) { + use lance::dataset::{WriteMode, WriteParams}; + let schema = batch.schema(); + let reader = arrow_array::RecordBatchIterator::new(vec![Ok(batch)], schema); + let params = WriteParams { + mode: WriteMode::Append, + allow_external_blob_outside_bases: true, + ..Default::default() + }; + ds.append(reader, Some(params)).await.unwrap(); +} + fn person_batch(rows: &[(&str, Option<i32>)]) -> RecordBatch { let ids: Vec<&str> = rows.iter().map(|(id, _)| *id).collect(); let ages: Vec<Option<i32>> = rows.iter().map(|(_, age)| *age).collect(); @@ -351,7 +370,7 @@ async fn stage_merge_insert_then_commit_persists_merged_view() { /// `write_fragments_internal` lack per-column statistics. The result /// contains only matching committed rows; matching staged rows are /// silently absent. `scanner.use_stats(false)` does not bypass this in -/// lance 4.0.0. +/// lance 6.0.1. /// /// This test pins the actual behavior so a future change either /// preserves it (and updates the doc) or fixes it (and rewrites this @@ -616,6 +635,58 @@ async fn stage_overwrite_replaces_all_fragments() { ); } +#[tokio::test] +async fn stage_overwrite_empty_batch_replaces_all_rows() { + let dir = tempfile::tempdir().unwrap(); + let uri = format!("{}/people.lance", dir.path().to_str().unwrap()); + let store = TableStore::new(dir.path().to_str().unwrap()); + + let ds = TableStore::write_dataset( + &uri, + person_batch(&[("alice", Some(30)), ("bob", Some(25))]), + ) + .await + .unwrap(); + let pre_version = ds.version().version; + + let target_schema = Arc::new(Schema::new(vec![ + Field::new("id", DataType::Utf8, false), + Field::new("age", DataType::Int32, true), + Field::new("nickname", DataType::Utf8, true), + ])); + let staged = store + .stage_overwrite(&ds, RecordBatch::new_empty(target_schema.clone())) + .await + .unwrap(); + assert!( + staged.new_fragments.is_empty(), + "empty overwrite should produce a zero-fragment Lance Overwrite transaction" + ); + assert_eq!( + staged.removed_fragment_ids.len(), + ds.manifest.fragments.len(), + "empty overwrite still removes every committed fragment" + ); + assert_eq!( + ds.version().version, + pre_version, + "staging empty overwrite must not advance HEAD" + ); + + let new_ds = store + .commit_staged(Arc::new(ds.clone()), staged.transaction) + .await + .unwrap(); + assert_eq!(new_ds.version().version, pre_version + 1); + assert_eq!(new_ds.count_rows(None).await.unwrap(), 0); + assert!( + arrow_schema::Schema::from(new_ds.schema()) + .field_with_name("nickname") + .is_ok(), + "empty overwrite must commit the replacement batch schema" + ); +} + /// `stage_create_btree_index` writes index segments to object storage /// but does NOT advance Lance HEAD until `commit_staged`. After commit, /// the index is queryable. @@ -699,7 +770,7 @@ async fn stage_create_inverted_index_does_not_advance_head_until_commit() { ); } -/// Pin the inline-commit behavior of `delete_where`. Lance 4.0.0 does +/// Pin the inline-commit behavior of `delete_where`. Lance 6.0.1 does /// NOT expose a public `DeleteJob::execute_uncommitted` /// (`pub(crate)` — see lance-format/lance#6658). The trait deliberately /// does NOT introduce a `stage_delete` wrapper that would secretly @@ -714,7 +785,6 @@ async fn stage_create_inverted_index_does_not_advance_head_until_commit() { async fn delete_where_advances_head_inline_documents_residual() { let dir = tempfile::tempdir().unwrap(); let uri = format!("{}/people.lance", dir.path().to_str().unwrap()); - let store = TableStore::new(dir.path().to_str().unwrap()); let mut ds = TableStore::write_dataset( &uri, @@ -724,13 +794,11 @@ async fn delete_where_advances_head_inline_documents_residual() { .unwrap(); let pre_version = ds.version().version; - let result = store - .delete_where(&uri, &mut ds, "id = 'alice'") - .await - .unwrap(); - assert_eq!(result.deleted_rows, 1); + let result = ds.delete("id = 'alice'").await.unwrap(); + ds = (*result.new_dataset).clone(); + assert_eq!(result.num_deleted_rows, 1); assert!( - result.version > pre_version, + ds.version().version > pre_version, "delete_where ADVANCES Lance HEAD inline (the residual). When \ lance-format/lance#6658 ships and we migrate to stage_delete + \ commit_staged, flip this assertion to assert that staging does \ @@ -739,9 +807,9 @@ async fn delete_where_advances_head_inline_documents_residual() { } /// Companion to `delete_where_*`: pin the inline-commit behavior of -/// `create_vector_index`. Lance 4.0.0 vector indices take the +/// `create_vector_index`. Lance 6.0.1 vector indices take the /// "segment commit path" which calls `build_index_metadata_from_segments` -/// (`pub(crate)` in lance-4.0.0 `src/index.rs:111`). Until upstream +/// (`pub(crate)` in lance-6.0.1 `src/index.rs:111`). Until upstream /// exposes that helper (companion ticket to lance-format/lance#6658), /// the trait surface deliberately does NOT include /// `stage_create_vector_index` — same rationale as `stage_delete`'s @@ -780,8 +848,9 @@ async fn create_vector_index_advances_head_inline_documents_residual() { let pre_version = ds.version().version; assert!(!store.has_vector_index(&ds, "embedding").await.unwrap()); - store - .create_vector_index(&mut ds, "embedding") + let params = lance::index::vector::VectorIndexParams::ivf_flat(1, MetricType::L2); + ds.create_index_builder(&["embedding"], IndexType::Vector, ¶ms) + .replace(true) .await .unwrap(); assert!( @@ -804,7 +873,7 @@ async fn create_vector_index_advances_head_inline_documents_residual() { /// The Lance source confirms this — `restore()` (no args) takes the /// currently-checked-out version's content and applies it via /// `apply_commit` against the latest manifest, advancing HEAD by one. -/// See lance-4.0.0 `src/dataset.rs:1106` and the transaction-spec +/// See lance-6.0.1 `src/dataset.rs:1106` and the transaction-spec /// example at https://lance.org/format/table/transaction/. /// /// If the lance bump (4.0.0 → 4.x) ever changes this delta or the call @@ -815,7 +884,6 @@ async fn create_vector_index_advances_head_inline_documents_residual() { async fn lance_restore_appends_one_commit_with_checked_out_content() { let dir = tempfile::tempdir().unwrap(); let uri = format!("{}/people.lance", dir.path().to_str().unwrap()); - let store = TableStore::new(dir.path().to_str().unwrap()); // Build version history: v1 = {alice}, v2 = {alice, bob}, v3 = {alice, bob, carol}. let mut ds = TableStore::write_dataset(&uri, person_batch(&[("alice", Some(30))])) @@ -823,16 +891,10 @@ async fn lance_restore_appends_one_commit_with_checked_out_content() { .unwrap(); assert_eq!(ds.version().version, 1); - store - .append_batch(&uri, &mut ds, person_batch(&[("bob", Some(25))])) - .await - .unwrap(); + lance_append_inline_local(&mut ds, person_batch(&[("bob", Some(25))])).await; assert_eq!(ds.version().version, 2); - store - .append_batch(&uri, &mut ds, person_batch(&[("carol", Some(40))])) - .await - .unwrap(); + lance_append_inline_local(&mut ds, person_batch(&[("carol", Some(40))])).await; assert_eq!(ds.version().version, 3); let head_before = ds.version().version; @@ -878,7 +940,7 @@ async fn lance_restore_appends_one_commit_with_checked_out_content() { /// and any future continuous-recovery reconciler's queue-acquisition /// requirement. /// -/// `Dataset::restore`'s `check_restore_txn` (lance-4.0.0 +/// `Dataset::restore`'s `check_restore_txn` (lance-6.0.1 /// `src/io/commit/conflict_resolver.rs:986`) returns `Ok(())` against /// almost every other op (Append, Update, Delete, CreateIndex, Merge, …), /// so a Restore commits successfully even with concurrent commits in @@ -908,7 +970,6 @@ async fn lance_restore_appends_one_commit_with_checked_out_content() { async fn lance_restore_loses_to_concurrent_append_via_orphaning() { let dir = tempfile::tempdir().unwrap(); let uri = format!("{}/people.lance", dir.path().to_str().unwrap()); - let store = TableStore::new(dir.path().to_str().unwrap()); // v1: seed with alice. let _ = TableStore::write_dataset(&uri, person_batch(&[("alice", Some(30))])) @@ -925,10 +986,7 @@ async fn lance_restore_loses_to_concurrent_append_via_orphaning() { // This simulates a per-table-queue model where another tenant wrote // between recovery's open and recovery's restore call. let mut writer_handle = Dataset::open(&uri).await.unwrap(); - store - .append_batch(&uri, &mut writer_handle, person_batch(&[("bob", Some(25))])) - .await - .unwrap(); + lance_append_inline_local(&mut writer_handle, person_batch(&[("bob", Some(25))])).await; assert_eq!(writer_handle.version().version, 2); // Recovery now restores. Because restore's `check_restore_txn` returns diff --git a/crates/omnigraph/tests/writes.rs b/crates/omnigraph/tests/writes.rs index d76ad46..b006f4c 100644 --- a/crates/omnigraph/tests/writes.rs +++ b/crates/omnigraph/tests/writes.rs @@ -778,6 +778,47 @@ async fn load_with_bad_edge_reference_unblocks_next_load() { ); } +#[tokio::test] +async fn load_overwrite_with_bad_edge_reference_unblocks_next_load() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + load_jsonl(&mut db, TEST_DATA, LoadMode::Overwrite) + .await + .unwrap(); + + let pre_persons = count_rows(&db, "node:Person").await; + let pre_edges = count_rows(&db, "edge:Knows").await; + + let bad = r#"{"type": "Person", "data": {"name": "Mallory", "age": 5}} +{"edge": "Knows", "from": "Mallory", "to": "Ghost"} +"#; + let err = load_jsonl(&mut db, bad, LoadMode::Overwrite) + .await + .expect_err("RI violation must fail overwrite before commit_staged"); + let OmniError::Manifest(manifest_err) = err else { + panic!("expected Manifest error, got {err:?}"); + }; + assert!( + manifest_err.message.contains("not found"), + "unexpected error: {}", + manifest_err.message, + ); + + assert_eq!(count_rows(&db, "node:Person").await, pre_persons); + assert_eq!(count_rows(&db, "edge:Knows").await, pre_edges); + + let good = r#"{"type": "Person", "data": {"name": "Pat", "age": 55}} +{"type": "Person", "data": {"name": "Quinn", "age": 56}} +{"edge": "Knows", "from": "Pat", "to": "Quinn"} +"#; + load_jsonl(&mut db, good, LoadMode::Overwrite) + .await + .unwrap(); + assert_eq!(count_rows(&db, "node:Person").await, 2); + assert_eq!(count_rows(&db, "edge:Knows").await, 1); +} + /// Same shape as the RI test above, but driven by a cardinality /// violation (`@card(0..1)` on `WorksAt`). The staged loader's pending /// edge accumulator drives the cardinality scan; a violation aborts @@ -842,6 +883,56 @@ edge WorksAt: Person -> Company @card(0..1) ); } +#[tokio::test] +async fn load_overwrite_with_cardinality_violation_unblocks_next_load() { + const CARD_SCHEMA: &str = r#" +node Person { + name: String @key + age: I32? +} +node Company { + name: String @key +} +edge WorksAt: Person -> Company @card(0..1) +"#; + + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, CARD_SCHEMA).await.unwrap(); + + let seed = r#"{"type": "Person", "data": {"name": "Alice", "age": 30}} +{"type": "Company", "data": {"name": "Acme"}} +{"type": "Company", "data": {"name": "Bigco"}} +"#; + load_jsonl(&mut db, seed, LoadMode::Overwrite) + .await + .unwrap(); + + let pre_works = count_rows(&db, "edge:WorksAt").await; + + let bad = r#"{"edge": "WorksAt", "from": "Alice", "to": "Acme"} +{"edge": "WorksAt", "from": "Alice", "to": "Bigco"} +"#; + let err = load_jsonl(&mut db, bad, LoadMode::Overwrite) + .await + .expect_err("cardinality violation must fail overwrite before commit_staged"); + let OmniError::Manifest(manifest_err) = err else { + panic!("expected Manifest error, got {err:?}"); + }; + assert!( + manifest_err.message.contains("@card violation"), + "unexpected error: {}", + manifest_err.message, + ); + assert_eq!(count_rows(&db, "edge:WorksAt").await, pre_works); + + let good = r#"{"edge": "WorksAt", "from": "Alice", "to": "Acme"}"#; + load_jsonl(&mut db, good, LoadMode::Overwrite) + .await + .unwrap(); + assert_eq!(count_rows(&db, "edge:WorksAt").await, 1); +} + // ─── Chained-mutation correctness — pinned coverage ───────────────────────── /// Chained `update` ops in one query must respect each previous op's diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md index 813f30c..9d31545 100644 --- a/docs/dev/architecture.md +++ b/docs/dev/architecture.md @@ -186,7 +186,8 @@ op-2 (insert/update) → read committed via Lance + pending via DataFusion op-N → push batch ─── end of query ─────────────────────────────────────── finalize: per pending table: - concat batches → stage_append OR stage_merge_insert → commit_staged + concat batches → stage_append OR stage_merge_insert OR stage_overwrite + → commit_staged publisher: ManifestBatchPublisher::publish (one cross-table CAS) ``` @@ -197,9 +198,10 @@ contracts: - `D₂` parse-time rule: a query is either insert/update-only or delete-only. Mixed → reject. Deletes still inline-commit (Lance 4.0.0 has no public two-phase delete); D₂ keeps the inline path safe. -- `LoadMode::Overwrite` keeps the inline-commit path - (truncate-then-append doesn't fit the staged shape; overwrite has no - in-flight read-your-writes requirement). +- `LoadMode::Overwrite` uses Lance `Operation::Overwrite` through the + same staged path. Loader validation runs against the replacement + in-memory batches before any `commit_staged`, and the publish window is + covered by `SidecarKind::Load` recovery. - Read sites consume `TableStore::scan_with_pending`, which Lance-scans the committed snapshot at the captured `expected_version` and unions with a DataFusion `MemTable` over the pending batches. diff --git a/docs/dev/execution.md b/docs/dev/execution.md index 3a108d7..a3a9c01 100644 --- a/docs/dev/execution.md +++ b/docs/dev/execution.md @@ -84,7 +84,7 @@ Resolves expression values to literals, converts to typed Arrow arrays (`literal - `insert` (no `@key`, edges) → accumulate into `MutationStaging.pending` (Append mode); finalize calls `stage_append` once per touched table. - `insert` (`@key` node) → accumulate into `pending` (Merge mode); finalize calls `stage_merge_insert` once per touched table. - `update` → scan committed via Lance + pending via DataFusion `MemTable` (read-your-writes), apply assignments, accumulate into `pending` (Merge mode). -- `delete` → still inline-commits via `delete_where` (Lance 4.0.0 has no public two-phase delete); recorded into `MutationStaging.inline_committed`. +- `delete` → still inline-commits via `delete_where` (Lance v6.0.1 has no public two-phase delete; `DeleteBuilder::execute_uncommitted` first ships in v7.0.0-beta.10 — tracked as MR-A in [docs/dev/lance.md](lance.md)); recorded into `MutationStaging.inline_committed`. **D₂ parse-time rule.** A single mutation query is either insert/update-only or delete-only. Mixed → reject before any I/O. The check fires in `enforce_no_mixed_destructive_constructive(&ir)` inside `execute_named_mutation`. diff --git a/docs/dev/invariants.md b/docs/dev/invariants.md index 4baff5e..7642fd9 100644 --- a/docs/dev/invariants.md +++ b/docs/dev/invariants.md @@ -102,7 +102,7 @@ Use it this way: | Branch delete | Manifest is the single authority, flipped atomically first; per-table forks + commit-graph branch are derived state, reclaimed best-effort (`force_delete_branch`) with the `cleanup` reconciler as the guaranteed backstop. Reusing a name whose reclaim failed before `cleanup` surfaces an actionable error | [branches-commits.md](../user/branches-commits.md), [maintenance.md](../user/maintenance.md) | | Schema validation | Type checks, required fields, defaults, edge endpoint checks, and edge cardinality are enforced on write paths | [schema-language.md](../user/schema-language.md), [execution.md](execution.md) | | Unique constraints | Intra-batch and write-path checks exist; intake and branch-merge derive the composite key through one shared function (`loader::composite_unique_key`, a separator-free `Vec<String>` tuple) and fail loudly on an un-keyable column type rather than silently exempting it; full cross-version uniqueness against already-committed rows is still a gap | [schema-language.md](../user/schema-language.md) | -| Storage trait | `TableStorage` exists as the sealed staged-write surface; full call-site migration and capability/stat surfaces are incomplete | [writes.md](writes.md), [architecture.md](architecture.md) | +| Storage trait | `TableStorage` (via `db.storage()`) is staged-only; the inline-commit residuals (`delete_where`, `create_vector_index`) are split onto a separate sealed `InlineCommitResidual` trait reached via `db.storage_inline_residual()` (MR-854), so §1 holds by construction; capability/stat surfaces are roadmap | [writes.md](writes.md), [architecture.md](architecture.md) | | Index lifecycle | `ensure_indices` is explicit today; reconciler-based convergence is roadmap | [indexes.md](../user/indexes.md), [maintenance.md](../user/maintenance.md) | | Traversal IDs | Runtime still builds `TypeIndex`; Lance stable row-id based graph IDs are roadmap | [architecture.md](architecture.md), [query-language.md](../user/query-language.md) | | Auth | Bearer token hashing and server-side actor resolution are implemented at the HTTP boundary | [server.md](../user/server.md), [policy.md](../user/policy.md) | @@ -124,9 +124,16 @@ them explicit. renames. The current compiler still derives type IDs from `kind:name`; this must be fixed before relying on renamed IDs across accepted schemas. - **Storage abstraction:** `TableStorage` is present, sealed, and canonical for - staged writes, but older inherent `TableStore` call sites and inline residuals - remain. New write paths should use the staged shape unless a documented Lance - blocker applies. + staged writes. MR-854 sealed it: `db.storage()` exposes only staged primitives + + reads, and the inline-commit residuals are split onto a separate sealed + `InlineCommitResidual` trait reached via `db.storage_inline_residual()`, so a + new writer cannot couple a write with a HEAD advance through the default + surface. The dead legacy methods (`append_batch` on the trait, + `merge_insert_batch{,es}`, `create_{btree,inverted}_index`) were removed. The + remaining residuals are `delete_where` (gated on MR-A — Lance v7.x bump) + and `create_vector_index` (gated on Lance #6666); see + [lance.md](lance.md) and [writes.md](writes.md). New write paths should use + the staged shape unless a documented Lance blocker applies. - **Deletes and vector indexes:** `delete_where` and vector index creation still advance Lance HEAD inline because the required public Lance APIs are missing. Keep D2 and recovery coverage in place until those residuals are removed. diff --git a/docs/dev/lance.md b/docs/dev/lance.md index 9d2b990..a4e311f 100644 --- a/docs/dev/lance.md +++ b/docs/dev/lance.md @@ -55,18 +55,18 @@ Adding/changing index types, fixing coverage, debugging FTS or vector recall, de | Topic | URL | |---|---| -| Index spec overview | https://lance.org/format/table/index/ | -| BTREE scalar index | https://lance.org/format/table/index/scalar/btree/ | -| Bitmap scalar index | https://lance.org/format/table/index/scalar/bitmap/ | -| Bloom-filter scalar index | https://lance.org/format/table/index/scalar/bloom_filter/ | -| Label-list scalar index | https://lance.org/format/table/index/scalar/label_list/ | -| Zone-map scalar index | https://lance.org/format/table/index/scalar/zonemap/ | -| R-Tree scalar index (spatial) | https://lance.org/format/table/index/scalar/rtree/ | -| Full-text search (FTS) index | https://lance.org/format/table/index/scalar/fts/ | -| N-gram scalar index | https://lance.org/format/table/index/scalar/ngram/ | -| Vector index | https://lance.org/format/table/index/vector/ | -| Fragment-reuse system index | https://lance.org/format/table/index/system/frag_reuse/ | -| MemWAL system index | https://lance.org/format/table/index/system/mem_wal/ | +| Index spec overview | https://lance.org/format/index/ | +| BTREE scalar index | https://lance.org/format/index/scalar/btree/ | +| Bitmap scalar index | https://lance.org/format/index/scalar/bitmap/ | +| Bloom-filter scalar index | https://lance.org/format/index/scalar/bloom_filter/ | +| Label-list scalar index | https://lance.org/format/index/scalar/label_list/ | +| Zone-map scalar index | https://lance.org/format/index/scalar/zonemap/ | +| R-Tree scalar index (spatial) | https://lance.org/format/index/scalar/rtree/ | +| Full-text search (FTS) index | https://lance.org/format/index/scalar/fts/ | +| N-gram scalar index | https://lance.org/format/index/scalar/ngram/ | +| Vector index | https://lance.org/format/index/vector/ | +| Fragment-reuse system index | https://lance.org/format/index/system/frag_reuse/ | +| MemWAL system index | https://lance.org/format/index/system/mem_wal/ | | HNSW Rust example | https://lance.org/examples/rust/hnsw/ | | Distributed indexing | https://lance.org/guide/distributed_indexing/ | | Tokenizer (FTS, n-gram) | https://lance.org/guide/tokenizer/ | @@ -125,7 +125,7 @@ Touching `omnigraph optimize` / `cleanup`, the underlying `compact_files` / `cle |---|---| | Read-and-write guide (covers `compact_files`, `cleanup_old_versions`) | https://lance.org/guide/read_and_write/ | | Performance (compaction tradeoffs) | https://lance.org/guide/performance/ | -| Fragment-reuse index | https://lance.org/format/table/index/system/frag_reuse/ | +| Fragment-reuse index | https://lance.org/format/index/system/frag_reuse/ | ### DataFusion integration diff --git a/docs/dev/writes.md b/docs/dev/writes.md index d2c7c7e..5647d82 100644 --- a/docs/dev/writes.md +++ b/docs/dev/writes.md @@ -48,7 +48,7 @@ shared by both `mutate_as` and the bulk loader: touched sub-tables. Cross-table conflicts surface as `ManifestConflictDetails::ExpectedVersionMismatch`. - **Deletes still inline-commit.** Lance's `Dataset::delete` is not - exposed as a two-phase op in 4.0.0; deletes go through `delete_where` + exposed as a two-phase op in 6.0.1; deletes go through `delete_where` immediately and record their post-write state in `MutationStaging.inline_committed`. The parse-time D₂ rule (below) prevents inserts/updates from coexisting with deletes in one query, @@ -82,16 +82,14 @@ Three writers have been migrated onto staged primitives: * **`ensure_indices`** (`db/omnigraph/table_ops.rs::build_indices_on_dataset_for_catalog`) — scalar indices (BTree, Inverted) now use `stage_create_*_index` + `commit_staged`. Vector indices stay inline (residual — Lance - `build_index_metadata_from_segments` is `pub(crate)` in 4.0.0; + `build_index_metadata_from_segments` is `pub(crate)` in 6.0.1; companion ticket to lance-format/lance#6658 needed). * **`branch_merge::publish_rewritten_merge_table`** (`exec/merge.rs`) — merge_insert now uses `stage_merge_insert` + `commit_staged`. Deletes stay inline (Lance #6658 residual). * **`schema_apply` rewritten_tables** (`db/omnigraph/schema_apply.rs`) - — non-empty rewrites use `stage_overwrite` + `commit_staged`. - Empty-batch rewrites stay inline (Lance `InsertBuilder::execute_uncommitted` - rejects empty data; the empty case is rare and bounded by the - schema-apply lock branch). + — rewrites use `stage_overwrite` + `commit_staged`, including empty-table + rewrites via a zero-fragment Lance `Operation::Overwrite`. A defense-in-depth integration test (`tests/forbidden_apis.rs`) walks engine source and fails if non-allow-listed code calls Lance's @@ -106,34 +104,32 @@ the same drift class. Closing it requires either upstream Lance multi-dataset commit OR the omnigraph-side recovery-on-open reconciler described in `.context/mr-793-design.md` §15 (deferred to MR-795). -### Inline-commit method residuals on `TableStorage` (MR-793 acceptance §1 option b) +### Inline-commit residuals live on `InlineCommitResidual`, not `db.storage()` (MR-793 acceptance §1, by construction) -MR-793's acceptance criterion §1 ("`TableStore` public API has no method that performs a manifest commit as a side effect of writing") is met **per-method** by enumerating every inline-commit method that remains on the trait surface, naming why it cannot yet be removed, and keeping the residual comment at every call site: +MR-793's acceptance criterion §1 ("`TableStore` (or successor) public API has no method that performs a manifest commit as a side effect of writing") holds **by construction** after MR-854. `db.storage()` (`&dyn TableStorage`) exposes only staged primitives + reads; the inline-commit writes Lance cannot yet stage live on a separate `InlineCommitResidual` trait reached via `Omnigraph::storage_inline_residual()`. A new engine writer cannot couple a write with a Lance HEAD advance through the default surface — it would have to name the residual accessor explicitly. The dead legacy methods (trait `append_batch` / `merge_insert_batches`, inherent `merge_insert_batch{,es}`, `create_{btree,inverted}_index`) were removed; appends/merges and scalar index builds all use the `stage_*` primitives. -| Method on `TableStore` | Inline-commit reason | Closes when | +Two methods remain on `InlineCommitResidual`, each named honestly at its call site: + +| Residual method | Inline-commit reason | Closes when | |---|---|---| -| `delete_where` | `DeleteJob` is `pub(crate)` in lance-4.0.0 — no public two-phase delete API | [lance-format/lance#6658](https://github.com/lance-format/lance/issues/6658) lands and `stage_delete` joins the trait | -| `create_vector_index` | Vector indices take Lance's "segment commit path"; the helper `build_index_metadata_from_segments` is `pub(crate)` | [lance-format/lance#6666](https://github.com/lance-format/lance/issues/6666) lands and `stage_create_vector_index` joins the trait | -| `append_batch` | Legacy inherent method; some engine call sites haven't migrated to `stage_append + commit_staged` yet | MR-793 Phase 1b (call-site conversion) + Phase 9 (demote to `pub(crate)`) | -| `merge_insert_batch` / `merge_insert_batches` | Legacy inherent method | Same — Phase 1b + Phase 9 | -| `overwrite_batch` | Legacy inherent method | Same — Phase 1b + Phase 9 | -| `create_btree_index` (inherent) | Legacy inherent method (the migrated callers use `stage_create_btree_index` + `commit_staged`; the inherent stays for tests / un-migrated paths) | Same — Phase 1b + Phase 9 | -| `create_inverted_index` (inherent) | Same | Same — Phase 1b + Phase 9 + index-class split (MR-848) | -| `truncate_table` (inherent on `TableStore`) | Used by `overwrite_batch` internally | Phase 9 | +| `delete_where` | `DeleteBuilder::execute_uncommitted` is not in Lance v6.0.1 (closed upstream as [#6658](https://github.com/lance-format/lance/issues/6658) but first ships in `v7.0.0-beta.10`); see [docs/dev/lance.md](lance.md) | MR-A: Lance v7.x bump migrates `delete_where` to staged, retires the parse-time D₂ mutation rule, and extends recovery sidecar coverage | +| `create_vector_index` | Vector indices take Lance's "segment commit path"; `build_index_metadata_from_segments` is `pub(crate)` (Lance [#6666](https://github.com/lance-format/lance/issues/6666) still open) | Lance #6666 lands and `stage_create_vector_index` joins the staged surface | -After **lance#6658 + lance#6666 ship + MR-793 Phase 1b + MR-793 Phase 9 all complete**, the trait surface exposes only staged-write primitives + `commit_staged`. Until then this matrix names every residual explicitly, every call site carries a one-line residual comment, and no engine code outside `table_store.rs` is permitted to reach the inline-commit Lance APIs (enforced by the `tests/forbidden_apis.rs` guard). +The `tests/forbidden_apis.rs` guard still catches direct `lance::*` inline-commit misuse outside the storage layer; the trait split makes the staged-only default a type-system guarantee on top of it. -### `LoadMode::Overwrite` residual +### `LoadMode::Overwrite` uses staged Lance `Overwrite` -The bulk loader's Append and Merge modes use the staged-write path -described above. `LoadMode::Overwrite` keeps the legacy inline-commit -path: truncate-then-append doesn't fit the staged shape cleanly in -Lance 4.0.0, and overwrite has no in-flight read-your-writes -requirement (the prior data is being wiped). A mid-overwrite failure -can leave Lance HEAD on a partially-truncated table; the next overwrite -will replace it. Operator-driven (rare in agent workloads); document -permanently until Lance exposes `Operation::Overwrite { fragments }` as -a two-phase op. +The bulk loader's Append, Merge, and Overwrite modes all use the +staged-write path described above. `LoadMode::Overwrite` accumulates +replacement batches in memory, validates node/edge constraints, referential +integrity, and edge cardinality before any Lance HEAD movement, stages +each touched table with Lance `Operation::Overwrite`, then runs +`commit_staged` under the normal `SidecarKind::Load` recovery sidecar +before publishing `__manifest`. `OMNIGRAPH_LOAD_CONCURRENCY` applies to the +fragment-writing stage only; the commit and manifest publish still run +under the per-table write queues. Empty-table overwrite is represented as +a valid zero-fragment Lance `Overwrite` transaction, not as +truncate-then-append. ### Open-time recovery sweep @@ -286,7 +282,7 @@ guarantee — the in-memory accumulator evaporates with the dropped task and no Lance write was ever issued. For delete-touching mutations the legacy inline-commit shape is -preserved (Lance has no public two-phase delete in 4.0.0) — the same +preserved (Lance has no public two-phase delete in 6.0.1) — the same narrow window remains. The parse-time D₂ rule prevents inserts/updates from coexisting with deletes in one query, so a pure-delete failure cannot drift any staged-table state. If a delete-only multi-table diff --git a/docs/user/query-language.md b/docs/user/query-language.md index acdc45d..bcab67c 100644 --- a/docs/user/query-language.md +++ b/docs/user/query-language.md @@ -72,7 +72,7 @@ A single mutation query must be **either insert/update-only or delete-only**. Mi > `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes. This restriction lifts when Lance exposes a two-phase delete API (tracked: MR-793 / Lance-upstream).` -Reason: under the staged-write rewire (MR-794), inserts and updates accumulate in memory and commit at end-of-query, while deletes still inline-commit (Lance 4.0.0 has no public two-phase delete). Mixing creates ordering hazards (same-row insert→delete becomes a no-op because the staged insert isn't visible to delete; cascading deletes of just-inserted edges break referential integrity by silent design). Until Lance exposes `DeleteJob::execute_uncommitted`, the parse-time rejection keeps both paths atomic and correct. See [docs/dev/writes.md](../dev/writes.md) and [docs/dev/invariants.md](../dev/invariants.md). +Reason: under the staged-write rewire (MR-794), inserts and updates accumulate in memory and commit at end-of-query, while deletes still inline-commit (Lance v6.0.1 has no public two-phase delete). Mixing creates ordering hazards (same-row insert→delete becomes a no-op because the staged insert isn't visible to delete; cascading deletes of just-inserted edges break referential integrity by silent design). Until the MR-A Lance v7 bump migrates `delete_where` to staged (`DeleteBuilder::execute_uncommitted` first ships in `v7.0.0-beta.10`), the parse-time rejection keeps both paths atomic and correct. See [docs/dev/writes.md](../dev/writes.md), [docs/dev/lance.md](../dev/lance.md), and [docs/dev/invariants.md](../dev/invariants.md). ## IR (Intermediate Representation) From 5e1dede08f20403eb6c87fd8f21a17267669d908 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 00:35:03 +0300 Subject: [PATCH 043/165] =?UTF-8?q?fix(cluster,cli):=20apply=20failure=20o?= =?UTF-8?q?utput=20=E2=80=94=20persisted=20statuses=20only,=20changes=20li?= =?UTF-8?q?st=20printed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two review findings (greptile, PR #165): - ApplyOutput.resource_statuses on a failed state write now carries the pre-apply on-disk snapshot instead of the in-memory mutations that were never persisted, so automation reading the field independently of `ok` cannot see phantom applied/blocked statuses. Regression test forces the state write to fail via a read-only __cluster dir (unix-only, skips when permissions are not enforced). - Human-mode `cluster apply` prints the classified changes list on failure too, so an operator debugging a partial apply without --json sees what was attempted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 44 ++++++++------ crates/omnigraph-cluster/src/lib.rs | 89 ++++++++++++++++++++++++++++- 2 files changed, 113 insertions(+), 20 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 42bbed8..37db77f 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -820,34 +820,42 @@ fn print_cluster_apply_human(output: &ApplyOutput) { "cluster apply: {} applied, {} deferred/blocked", output.applied_count, output.deferred_count ); - for change in &output.changes { - match (&change.disposition, change.reason.as_deref()) { - (Some(disposition), Some(reason)) => println!( - " {:?} {} [{disposition:?}: {reason}]", - change.operation, change.resource - ), - (Some(disposition), None) => println!( - " {:?} {} [{disposition:?}]", - change.operation, change.resource - ), - _ => println!(" {:?} {}", change.operation, change.resource), - } - } - if output.changes.is_empty() { - println!(" no changes"); - } + } else { + println!("cluster apply failed"); + } + // The change list prints on failure too: an operator debugging a partial + // apply (payload or state-write error) needs to see what was attempted. + print_cluster_apply_changes(&output.changes); + if output.ok { let state = &output.state_observations; println!( " state: revision {}, converged: {}, written: {}", state.state_revision, output.converged, output.state_written ); println!(" note: applied = recorded in the cluster catalog; the server still boots from omnigraph.yaml"); - } else { - println!("cluster apply failed"); } print_cluster_diagnostics(&output.diagnostics); } +fn print_cluster_apply_changes(changes: &[omnigraph_cluster::PlanChange]) { + for change in changes { + match (&change.disposition, change.reason.as_deref()) { + (Some(disposition), Some(reason)) => println!( + " {:?} {} [{disposition:?}: {reason}]", + change.operation, change.resource + ), + (Some(disposition), None) => println!( + " {:?} {} [{disposition:?}]", + change.operation, change.resource + ), + _ => println!(" {:?} {}", change.operation, change.resource), + } + } + if changes.is_empty() { + println!(" no changes"); + } +} + fn print_cluster_status_human(output: &StatusOutput) { if output.ok { let state = &output.state_observations; diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 01ad171..3673194 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -276,6 +276,8 @@ pub struct ApplyOutput { pub converged: bool, /// False for a no-op re-apply: state bytes (and revision) were left untouched. pub state_written: bool, + /// The statuses as persisted: post-apply on success, the pre-apply on-disk + /// snapshot when the state write fails (never unpersisted in-memory state). pub resource_statuses: BTreeMap<String, ResourceStatusRecord>, pub diagnostics: Vec<Diagnostic>, } @@ -819,13 +821,26 @@ pub fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { let after_value = serde_json::to_value(&new_state).expect("cluster state must serialize deterministically"); let mut state_written = false; + let mut state_write_failed = false; if after_value != before_value { new_state.state_revision = new_state.state_revision.saturating_add(1); match backend.write_state(&new_state, expected_cas.as_deref(), &mut observations) { Ok(()) => state_written = true, - Err(diagnostic) => diagnostics.push(diagnostic), + Err(diagnostic) => { + diagnostics.push(diagnostic); + state_write_failed = true; + } } } + // On a failed state write, report the statuses that are actually on disk + // (the pre-apply snapshot), not the in-memory mutations that were never + // persisted — automation reading `resource_statuses` independently of `ok` + // must not see phantom status updates. + let resource_statuses = if state_write_failed { + state.resource_statuses + } else { + new_state.resource_statuses + }; let applied_count = changes .iter() @@ -853,7 +868,7 @@ pub fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { deferred_count, converged, state_written, - resource_statuses: new_state.resource_statuses, + resource_statuses, diagnostics, } } @@ -4232,6 +4247,76 @@ graphs: assert!(!dir.path().join(CLUSTER_STATE_DIR).exists()); } + /// When the state write fails after payloads landed, the output must + /// report the statuses actually on disk — not the unpersisted in-memory + /// mutations (phantom `applied` entries would mislead automation that + /// reads `resource_statuses` independently of `ok`). + #[cfg(unix)] + #[test] + fn apply_state_write_failure_reports_persisted_statuses() { + use std::os::unix::fs::PermissionsExt; + + let dir = fixture(); + // lock: false so the only write into __cluster/ is state.json itself. + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +state: + backend: cluster + lock: false +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +"#, + ) + .unwrap(); + write_applyable_state(dir.path()); + // Pre-create the payload blob so the payload phase is a no-op and the + // failure lands exactly at the state write. + let desired = validate_config_dir(dir.path()); + let query_digest = desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap(); + let blob = query_payload_path(dir.path(), query_digest); + fs::create_dir_all(blob.parent().unwrap()).unwrap(); + fs::write(&blob, QUERY).unwrap(); + + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::set_permissions(&state_dir, fs::Permissions::from_mode(0o555)).unwrap(); + // Running as root ignores permission bits; skip rather than flake. + if fs::write(state_dir.join("probe"), b"x").is_ok() { + let _ = fs::remove_file(state_dir.join("probe")); + fs::set_permissions(&state_dir, fs::Permissions::from_mode(0o755)).unwrap(); + eprintln!("skipping: permissions are not enforced (running as root)"); + return; + } + + let out = apply_config_dir(dir.path()); + fs::set_permissions(&state_dir, fs::Permissions::from_mode(0o755)).unwrap(); + + assert!(!out.ok); + assert!(!out.state_written); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_write_error"), + "{:?}", + out.diagnostics + ); + // The seeded state has no statuses; the failed apply must not invent + // the in-memory `applied` ones it failed to persist. + assert!( + out.resource_statuses.is_empty(), + "unpersisted statuses leaked into output: {:?}", + out.resource_statuses + ); + } + #[test] fn plan_annotates_apply_dispositions() { let dir = fixture(); From cec65b8ef8b3c680cdd08b2b35a3494c0e82f522 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 00:44:51 +0300 Subject: [PATCH 044/165] =?UTF-8?q?docs(cluster):=20axiom=2015=20=E2=80=94?= =?UTF-8?q?=20single=20ownership,=20mode-switch=20migration,=20per-operato?= =?UTF-8?q?r=20layer=20(#164)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Encode the omnigraph.yaml ↔ cluster.yaml coexistence rules that were implicit across the specs: - cluster-axioms.md: new axiom 15 — every fact has exactly one owner at a time; coexistence is a mode switch, never a merge; omnigraph.yaml's job description shrinks to the permanent per-operator layer. Added review-tension bullet. - cluster-config-specs.md: "Migration model" subsection (three coexistence windows: no-conflict, Phase-5 mode switch, bridges-with-sunsets) and a "per-operator layer" completeness table (connection, credential reference, active context, ergonomics, personal aliases) with its global-config-dir destination per the RFC-002 direction. - cluster-config-implementation-spec.md: Compatibility Stance #7–#9 (single ownership, shrinking role, bridges carry sunsets); Phase 5 boot is an exclusive XOR mode switch; fixed the duplicated recoveries/recovery dirs in the Phase-1 storage layout. - docs/user/cluster-config.md: "Relationship to omnigraph.yaml" section in current-reality terms (cluster catalog is inspectable, not live). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/cluster-axioms.md | 11 +++- .../dev/cluster-config-implementation-spec.md | 21 ++++++- docs/dev/cluster-config-specs.md | 59 +++++++++++++++++++ docs/user/cluster-config.md | 13 ++++ 4 files changed, 100 insertions(+), 4 deletions(-) diff --git a/docs/dev/cluster-axioms.md b/docs/dev/cluster-axioms.md index a3793b4..dddecf1 100644 --- a/docs/dev/cluster-axioms.md +++ b/docs/dev/cluster-axioms.md @@ -24,6 +24,12 @@ consequences that follow from them. > Terraform-style JSON documents plus backend lock/CAS, not Lance control-plane > datasets. Lance remains a possible later backend only if row-level history or > queryability justifies the extra machinery. +> +> **Revision 2026-06-09 — single ownership during migration.** Axiom **15** +> added: while `omnigraph.yaml` and the cluster catalog coexist, every fact has +> exactly one owner at a time — coexistence is a **mode switch, never a merge**. +> `omnigraph.yaml` does not get replaced; its job description shrinks to the +> permanent per-operator layer. --- @@ -72,6 +78,8 @@ invoke_query. This axiom is the target control-plane rule, not a statement about today's server catalog. --> **14. Exposure is a policy decision, not a config flag.** Target design: which stored queries (and the tools/dashboards built on them) an actor may **list or invoke** is decided by the policy layer (Cedar: `invoke_query` + catalog visibility), not by a per-query `expose:` boolean. The registry only says a query *exists* (name → file); **policy says who may see and run it**, so the MCP catalog (`GET /queries`) becomes each actor's policy-permitted set. This supersedes the engine's current `mcp.expose` flag only after per-query `invoke_query` scope and Cedar-filtered catalog listing land; until then, proposals must state the compatibility bridge to today's `mcp.expose` + coarse invocation gate. +**15. Every fact has exactly one owner at a time; coexistence is a mode switch, never a merge.** `cluster.yaml` is not `omnigraph.yaml` v2 — the two documents end with disjoint jobs, and only the *shared-truth* parts of today's `omnigraph.yaml` (the set of graphs, stored-query registry, policy wiring, server boot source) migrate to the cluster catalog. The per-operator parts — connection/cluster selection, the operator's own credential reference, active graph/branch context, CLI ergonomics — are per-operator *by nature* (Sarah's and Bob's differ) and stay in the per-operator layer permanently; plan a **shrinking job description** for `omnigraph.yaml`, not an exit. During the migration window each fact is read from exactly one source at a time: a deployment serves from `omnigraph.yaml` **or** boots from cluster state (an exclusive mode switch), never from a precedence-merge of both. Two readers for one fact is the brittle-backcompat failure mode — it is the deny-list's "state that drifts from what it can be derived from" wearing a compatibility costume. Any compatibility bridge must name its replacement and its removal phase (the `mcp.expose` → policy-owned exposure bridge of axiom 14 is the template); bridges that accumulate without an exit are rejected at review. + --- ## The one-line compression @@ -82,7 +90,7 @@ about today's server catalog. --> ## How to use this file -- **Reviewing a proposal:** walk axioms 0–14; any conflict is the burden of the proposer to justify. The most common tensions: +- **Reviewing a proposal:** walk axioms 0–15; any conflict is the burden of the proposer to justify. The most common tensions: - Treating the *running system* as the source of truth for **intent** → axioms 2, 4 (intent lives in config). - Treating state as a throwaway derivation rather than an authoritative, locked, backend-held ledger → axiom 5, 12. - A runtime config-mutation API instead of declarative apply → axiom 3. @@ -94,4 +102,5 @@ about today's server catalog. --> - A secret value (token, embedding key, pipeline source credential) inline in config instead of in the gitignored `.env` file → axiom 10. - A per-query `expose:`/visibility flag in target-state cluster config instead of governing list/invoke in policy; or failing to account for today's `mcp.expose` compatibility bridge → axiom 14. - Shipping `apply` before hermetic `validate` + read-only `plan` tests, or shipping graph/schema-moving apply before recovery tests for the graph/resource-moved-before-cluster-publish gap → axiom 5 and axiom 12. + - Reading one fact from both `omnigraph.yaml` and the cluster catalog with precedence rules (a merge instead of a mode switch), migrating per-operator concerns into shared cluster config, or adding a compatibility bridge with no named replacement and removal phase → axiom 15. - **Citing:** reference axioms by number in PRs and review comments so the rationale is stable across renames and refactors. diff --git a/docs/dev/cluster-config-implementation-spec.md b/docs/dev/cluster-config-implementation-spec.md index 5121451..ff3dd7e 100644 --- a/docs/dev/cluster-config-implementation-spec.md +++ b/docs/dev/cluster-config-implementation-spec.md @@ -64,6 +64,21 @@ is trying to create. --> identity. It is not committed into `cluster.yaml`. 6. `mcp.expose` remains supported in current `omnigraph.yaml` until the per-query policy replacement ships. +7. **Single ownership (axiom 15).** While `omnigraph.yaml` and the cluster + catalog coexist, each fact is read from exactly one source at a time. + Phase 5 server boot is an exclusive mode switch — boot from cluster state + XOR from `omnigraph.yaml` — never a precedence-merge of both. No phase may + introduce a surface that reads the same fact (graph set, query registry, + policy wiring, bind address) from both sources with tie-break rules. +8. **`omnigraph.yaml` shrinks; it does not get deprecated.** Its terminal role + is the per-operator layer: connection/cluster selection, the operator's + credential reference, active graph/branch context, CLI ergonomics, and + purely personal aliases (target home: the operator's global config dir per + RFC-002). Shared-truth keys migrate to `cluster.yaml`; per-operator keys + never do. +9. **Bridges carry sunsets.** Every compatibility bridge names its replacement + and the phase that removes it (`mcp.expose` → Phase 6 policy-owned exposure + is the template). A bridge without an exit is a review-blocking finding. ## Terraform-Aligned Schema Validation @@ -335,8 +350,6 @@ Target Phase-1 cluster-root layout: <ulid>.json recoveries/ <ulid>.json - recovery/ - <ulid>.json resources/ query/<graph>/<name>/<digest>.gq policy/<name>/<digest>.yaml @@ -586,7 +599,9 @@ replacement would make every invariant harder to audit. --> - Allow server startup from cluster state. - Add status and catalog endpoints as needed. -- Keep the current `omnigraph.yaml` startup path as compatibility mode. +- Keep the current `omnigraph.yaml` startup path as compatibility mode — an + **exclusive mode switch** per deployment (cluster state XOR `omnigraph.yaml`), + never a merged read of both (Compatibility Stance #7, axiom 15). - Regenerate OpenAPI for any HTTP surface. ### Phase 6: Policy-Owned Query Exposure diff --git a/docs/dev/cluster-config-specs.md b/docs/dev/cluster-config-specs.md index 8aa63cb..8f36dc8 100644 --- a/docs/dev/cluster-config-specs.md +++ b/docs/dev/cluster-config-specs.md @@ -387,6 +387,65 @@ This proposal: The connection/credential/preference layer remains per operator: it points at a cluster, resolves that operator's identity, and holds personal ergonomics. The cluster config stays shared, secret-free, and reviewable; the state ledger stays authoritative and locked. +### Migration model: single ownership, mode switch, shrinking job description (axiom 15) + +`omnigraph.yaml` is not being replaced; its **job description shrinks**. Only the +shared-truth parts of its current role migrate to the cluster catalog (the set of +graphs, the stored-query registry, policy wiring, the server boot source). The +per-operator parts are per-operator *by nature* — Sarah's and Bob's differ — and +keep `omnigraph.yaml`/the per-operator layer as a permanent, well-defined home. + +While both exist, **each fact has exactly one owner at any moment, and +coexistence is a mode switch, never a merge**. The brittle version of backward +compatibility — the server reading graphs from `omnigraph.yaml` *and* from +cluster state with precedence rules gluing them together — is rejected outright: +two readers for one truth means every bug becomes "which file won?" and every +feature pays the tax twice. The realistic timeline has three windows: + +1. **Now → Phase 4 (no conflict).** Cluster apply writes only to its own catalog + (`__cluster/`); `omnigraph.yaml` serves traffic. `Applied` status must + visibly mean "recorded in the cluster catalog, not yet serving" so the + overlap is loud, not hidden. +2. **Phase 5 (the mode switch).** A deployment opts into booting from cluster + state; `omnigraph.yaml`'s server-role keys become inert *for that + deployment*. Exclusive — boot from cluster state XOR `omnigraph.yaml` — with + no key-level aliasing and no merged precedence. +3. **Phase 6+ (bridges with sunsets).** Targeted compatibility bridges are + allowed only with a named replacement and a removal phase; `mcp.expose` → + policy-owned exposure is the template. Bridges that accumulate without an + exit are review-rejected. + +Key-by-key compatibility inside one evolving file is the expensive kind of +backcompat (the v1 `omnigraph.yaml` reshape's `--target`/legacy-key regressions +are the in-repo cautionary tale); resource-ownership seams between two files +with a mode switch is the cheap kind. Police the single-owner rule in every +Phase 3–6 PR: a proposal that merges the two sources for one fact is the +deny-list's "state that drifts from what it can be derived from" wearing a +compatibility costume. + +### The per-operator layer: contents and destination + +The per-operator layer must be **complete** — everything an operator needs to +work against any cluster from any directory, and nothing that two operators must +agree on: + +| Per-operator concern | Today | Target | +|---|---|---| +| Connection (which cluster/server, named endpoints) | `omnigraph.yaml` `graphs.<name>` URIs / `servers:` refs | global config, per-operator | +| Operator credential **reference** (`bearer_token_env`, env-file lookup) | `omnigraph.yaml` + `.env` | global config references; secret values stay in env/`.env`, never in any config | +| Active context (current graph/branch selection) | ad-hoc per-command flags / `defaults` | global state layer (e.g. `omnigraph use`), explicitly **not** the cluster state ledger (axiom 5's "state" is the applied-cluster ledger, not a personal selection) | +| CLI ergonomics (output format, table layout) | `omnigraph.yaml` `cli:`/`defaults:` | global config, per-operator | +| Personal command shortcuts (purely personal aliases) | `omnigraph.yaml` `aliases:` | global config; *shared* aliases (team vocabulary) are cluster config — see the aliases split note above | + +Destination: this layer belongs in the operator's **global config dir** +(`~/.omnigraph`, per the RFC-002 global-first layered-config direction — +global config + active-context state file), not in a repo-committed file, so it +survives `git clone`, works from any directory, and never collides with the +shared cluster folder. The RFC-002 layering implementation is currently parked +(PRs #139/#162 closed over review findings), but the *boundary* it draws is the +one this spec depends on: per-operator → global dir; shared deployment intent → +the cluster config folder; deployed reality → the state ledger. + Implementation gate: the Terraform-style workflow must be testable in order. `cluster validate` must catch bad config before any apply path exists; read-only `cluster plan` must have deterministic structured-plan tests before diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 24718b1..0de43d0 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -23,6 +23,19 @@ omnigraph cluster force-unlock <LOCK_ID> --config ./company-brain --json `--config` points at a directory, not a file. The directory must contain `cluster.yaml`. When omitted, it defaults to the current directory. +## Relationship to `omnigraph.yaml` + +`cluster.yaml` does not replace `omnigraph.yaml`, and the two never describe +the same fact. `omnigraph.yaml` remains how the CLI and server are configured +today (graph targets, server bind, CLI defaults, credential env references) and +is its long-term home for per-operator settings. `cluster.yaml` is the shared +desired state of a whole deployment, read only by the `cluster` commands via +`--config`. In the current stage, nothing recorded in the cluster state ledger +affects what a server serves or what other CLI commands target — the cluster +catalog is inspectable, not live. When server boot from cluster state ships in +a later stage, it will be an explicit per-deployment mode switch, not a merge +of the two files. + ## Supported `cluster.yaml` Stage 2C accepts only the read-only resource subset: From 7f3ecf282a2e3f11f94477b49ab786a0b294c180 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 00:45:44 +0300 Subject: [PATCH 045/165] Merge origin/main (#164 axiom-15 docs, #86 TableStorage migration) into feat/cluster-apply-stage3a Clean auto-merge; also fix the stale 'Stage 2C accepts' line in cluster-config.md to Stage 3A. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/user/cluster-config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 8146646..912f307 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -42,7 +42,7 @@ of the two files. ## Supported `cluster.yaml` -Stage 2C accepts only the read-only resource subset: +Stage 3A accepts only this resource subset: ```yaml version: 1 From b6d228ff54a27ceeb7d13de93ee1ef8df94c87b5 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 00:59:20 +0300 Subject: [PATCH 046/165] =?UTF-8?q?test(cli):=20cluster=20e2e=20hardening?= =?UTF-8?q?=20=E2=80=94=20lost-state=20recovery,=20out-of-band=20drift,=20?= =?UTF-8?q?root=20destruction,=20multi-graph=20convergence=20(#166)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four lifecycle compositions over the spawned binary that pin spec claims no single-command test proves: - Lost ledger: delete state.json -> re-import from the live graph -> re-apply converges onto the same content-addressed blobs (axiom 5's reconstructable- state resilience edge, end to end). - Out-of-band schema apply (the Sarah/Bob violation): refresh marks graph/schema Drifted with schema_mismatch, status and plan surface it, and cluster apply refuses to silently correct it — state keeps the LIVE schema digest (drift correction is gated, axiom 8). - Destroyed graph root: refresh records graph_missing drift and drops graph/schema digests while preserving query/policy; plan proposes deferred creates only; apply moves nothing and the catalog stays intact. - Two graphs (one live, one not yet created) + a graph-spanning policy + a cluster-scoped policy: a single apply yields all four dispositions at once (applied/derived/deferred/blocked, deterministically ordered), then the second graph appears, refresh observes it, and apply converges. Helpers: init_named_cluster_graph generalizes init_cluster_derived_graph; write_multi_graph_cluster_fixture builds the two-graph config. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/cli.rs | 366 +++++++++++++++++++++++++++++- docs/dev/testing.md | 2 +- 2 files changed, 365 insertions(+), 3 deletions(-) diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 30fa796..f60ffbe 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -145,14 +145,18 @@ policies: } fn init_cluster_derived_graph(root: &std::path::Path) { + init_named_cluster_graph(root, "knowledge", "people.pg"); +} + +fn init_named_cluster_graph(root: &std::path::Path, graph_id: &str, schema_file: &str) { let graph_dir = root.join("graphs"); fs::create_dir_all(&graph_dir).unwrap(); output_success( cli() .arg("init") .arg("--schema") - .arg(root.join("people.pg")) - .arg(graph_dir.join("knowledge.omni")), + .arg(root.join(schema_file)) + .arg(graph_dir.join(format!("{graph_id}.omni"))), ); } @@ -1073,6 +1077,364 @@ fn cluster_e2e_force_unlock_unblocks_apply() { assert_eq!(retried["converged"], true, "{retried}"); } +/// Two-graph fixture: `knowledge` (people) + `engineering` (services), a +/// policy spanning both graphs, and a cluster-scoped policy with no graph +/// dependencies. +fn write_multi_graph_cluster_fixture(root: &std::path::Path) { + write_cluster_config_fixture(root); + fs::write( + root.join("services.pg"), + r#" +node Service { + name: String @key +} +"#, + ) + .unwrap(); + fs::write( + root.join("services.gq"), + r#" +query find_service($name: String) { + match { $s: Service { name: $name } } + return { $s.name } +} +"#, + ) + .unwrap(); + fs::write(root.join("cluster_wide.policy.yaml"), "rules: []\n").unwrap(); + fs::write(root.join("shared.policy.yaml"), "rules: []\n").unwrap(); + fs::write( + root.join("cluster.yaml"), + r#" +version: 1 +metadata: + name: company-brain +state: + backend: cluster + lock: true +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq + engineering: + schema: ./services.pg + queries: + find_service: + file: ./services.gq +policies: + shared: + file: ./shared.policy.yaml + applies_to: [knowledge, engineering] + cluster_wide: + file: ./cluster_wide.policy.yaml + applies_to: [cluster] +"#, + ) + .unwrap(); +} + +fn change_for<'j>(json: &'j serde_json::Value, resource: &str) -> &'j serde_json::Value { + json["changes"] + .as_array() + .unwrap() + .iter() + .find(|change| change["resource"] == resource) + .unwrap_or_else(|| panic!("missing change for {resource}: {json}")) +} + +/// The spec's resilience claim — "state is reconstructable from the +/// self-describing cluster" — proven end to end: lose the ledger, re-import +/// from the live graph, re-apply, and converge onto the same content-addressed +/// catalog blobs. +#[test] +fn cluster_e2e_lost_state_reimport_recovers_catalog() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["converged"], true, "{apply}"); + + let query_digest = change_for(&apply, "query.knowledge.find_person")["after_digest"] + .as_str() + .unwrap() + .to_string(); + let blob = temp + .path() + .join("__cluster/resources/query/knowledge/find_person") + .join(format!("{query_digest}.gq")); + let blob_content = fs::read_to_string(&blob).unwrap(); + + // Disaster: the state ledger is lost. + fs::remove_file(temp.path().join("__cluster/state.json")).unwrap(); + + let reimport = cluster_json(temp.path(), "import"); + assert_eq!(reimport["ok"], true, "{reimport}"); + assert_eq!(reimport["state_observations"]["state_revision"], 1); + // Import observes graph/schema only; query/policy digests are not invented. + assert!( + reimport["resource_digests"] + .get("query.knowledge.find_person") + .is_none(), + "{reimport}" + ); + + let plan = cluster_json(temp.path(), "plan"); + assert_eq!( + change_for(&plan, "query.knowledge.find_person")["disposition"], + "applied" + ); + assert_eq!(change_for(&plan, "policy.base")["disposition"], "applied"); + + let reapply = cluster_json(temp.path(), "apply"); + assert_eq!(reapply["ok"], true, "{reapply}"); + assert_eq!(reapply["converged"], true, "{reapply}"); + assert!( + reapply["state_observations"]["applied_config_digest"].is_string(), + "{reapply}" + ); + // The catalog blob was reused, not rewritten with different content. + assert_eq!(fs::read_to_string(&blob).unwrap(), blob_content); + + let replan = cluster_json(temp.path(), "plan"); + assert!(replan["changes"].as_array().unwrap().is_empty(), "{replan}"); +} + +/// The Sarah/Bob violation made visible: a schema change applied directly to +/// the graph (no config change) must surface as drift through refresh, status, +/// and plan — and apply must never silently "correct" it. +#[test] +fn cluster_e2e_out_of_band_schema_change_surfaces_as_drift() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["converged"], true, "{apply}"); + + // Out-of-band: the live graph evolves, cluster.yaml stays put. + fs::write( + temp.path().join("people_v2.pg"), + r#" +node Person { + name: String @key + age: I32? + bio: String? +} +"#, + ) + .unwrap(); + output_success( + cli() + .arg("schema") + .arg("apply") + .arg(temp.path().join("graphs/knowledge.omni")) + .arg("--schema") + .arg(temp.path().join("people_v2.pg")) + .arg("--json"), + ); + + let refresh = cluster_json(temp.path(), "refresh"); + assert_eq!(refresh["ok"], true, "{refresh}"); + assert_eq!( + refresh["resource_statuses"]["schema.knowledge"]["status"], + "drifted" + ); + assert_eq!( + refresh["resource_statuses"]["graph.knowledge"]["status"], + "drifted" + ); + assert_eq!( + refresh["observations"]["graph.knowledge"]["schema_matches_desired"], + false + ); + + let status = cluster_json(temp.path(), "status"); + assert_eq!( + status["resource_statuses"]["schema.knowledge"]["status"], + "drifted" + ); + + let plan = cluster_json(temp.path(), "plan"); + assert_eq!(change_for(&plan, "schema.knowledge")["disposition"], "deferred"); + assert_eq!(change_for(&plan, "graph.knowledge")["disposition"], "deferred"); + let live_schema_digest = change_for(&plan, "schema.knowledge")["before_digest"] + .as_str() + .unwrap() + .to_string(); + + let drift_apply = cluster_json(temp.path(), "apply"); + assert_eq!(drift_apply["applied_count"], 0, "{drift_apply}"); + assert_eq!(drift_apply["converged"], false, "{drift_apply}"); + // Apply must not have "corrected" the drift: state still records the LIVE + // schema digest, not the desired one. + let state: serde_json::Value = serde_json::from_str( + &fs::read_to_string(temp.path().join("__cluster/state.json")).unwrap(), + ) + .unwrap(); + assert_eq!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + live_schema_digest + ); +} + +/// Disaster input fails closed: a destroyed graph root drifts the ledger, +/// the plan proposes deferred creates, and apply moves nothing. +#[test] +fn cluster_e2e_graph_root_destruction_drifts_and_apply_moves_nothing() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["converged"], true, "{apply}"); + let query_digest = change_for(&apply, "query.knowledge.find_person")["after_digest"] + .as_str() + .unwrap() + .to_string(); + + fs::remove_dir_all(temp.path().join("graphs/knowledge.omni")).unwrap(); + + // Missing root is drift, not an error. + let refresh = cluster_json(temp.path(), "refresh"); + assert_eq!(refresh["ok"], true, "{refresh}"); + assert_eq!( + refresh["resource_statuses"]["graph.knowledge"]["status"], + "drifted" + ); + assert!( + refresh["resource_statuses"]["graph.knowledge"]["conditions"] + .as_array() + .unwrap() + .iter() + .any(|condition| condition == "graph_missing"), + "{refresh}" + ); + // Graph/schema digests removed; query/policy digests preserved. + assert!(refresh["resource_digests"].get("graph.knowledge").is_none()); + assert!(refresh["resource_digests"].get("schema.knowledge").is_none()); + assert!( + refresh["resource_digests"] + .get("query.knowledge.find_person") + .is_some(), + "{refresh}" + ); + + let plan = cluster_json(temp.path(), "plan"); + assert_eq!(change_for(&plan, "graph.knowledge")["operation"], "create"); + assert_eq!(change_for(&plan, "graph.knowledge")["disposition"], "deferred"); + assert_eq!(change_for(&plan, "schema.knowledge")["disposition"], "deferred"); + // Converged-then-destroyed: query/policy are already in state at the + // desired digests, so they are not changes at all. + assert_eq!(plan["changes"].as_array().unwrap().len(), 2, "{plan}"); + + let disaster_apply = cluster_json(temp.path(), "apply"); + assert_eq!(disaster_apply["applied_count"], 0, "{disaster_apply}"); + assert_eq!(disaster_apply["converged"], false, "{disaster_apply}"); + let state: serde_json::Value = serde_json::from_str( + &fs::read_to_string(temp.path().join("__cluster/state.json")).unwrap(), + ) + .unwrap(); + assert_eq!( + state["applied_revision"]["resources"]["query.knowledge.find_person"]["digest"], + query_digest + ); + assert!( + temp.path() + .join("__cluster/resources/query/knowledge/find_person") + .join(format!("{query_digest}.gq")) + .exists() + ); +} + +/// The disposition matrix as a system: one apply over two graphs (one live, +/// one not yet created) plus graph-spanning and cluster-scoped policies must +/// produce all four dispositions at once — then converge after the second +/// graph appears. +#[test] +fn cluster_e2e_multi_graph_mixed_dispositions_then_converge() { + let temp = tempdir().unwrap(); + write_multi_graph_cluster_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); // knowledge only + + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["ok"], true, "{apply}"); + assert_eq!(apply["converged"], false, "{apply}"); + assert_eq!(apply["applied_count"], 2, "{apply}"); + assert_eq!( + change_for(&apply, "query.knowledge.find_person")["disposition"], + "applied" + ); + assert_eq!( + change_for(&apply, "policy.cluster_wide")["disposition"], + "applied" + ); + assert_eq!( + change_for(&apply, "query.engineering.find_service")["disposition"], + "blocked" + ); + assert_eq!( + change_for(&apply, "query.engineering.find_service")["reason"], + "dependency_missing" + ); + // One missing dependency graph blocks the whole spanning policy. + assert_eq!(change_for(&apply, "policy.shared")["disposition"], "blocked"); + assert_eq!( + change_for(&apply, "graph.engineering")["disposition"], + "deferred" + ); + assert_eq!( + change_for(&apply, "schema.engineering")["disposition"], + "deferred" + ); + assert_eq!( + change_for(&apply, "graph.knowledge")["disposition"], + "derived" + ); + assert_eq!( + apply["resource_statuses"]["policy.shared"]["status"], + "blocked" + ); + // Deterministic ordering: changes sorted by resource address. + let order: Vec<&str> = apply["changes"] + .as_array() + .unwrap() + .iter() + .map(|change| change["resource"].as_str().unwrap()) + .collect(); + let mut sorted = order.clone(); + sorted.sort_unstable(); + assert_eq!(order, sorted, "{apply}"); + + // The second graph appears; refresh observes it; apply converges. + init_named_cluster_graph(temp.path(), "engineering", "services.pg"); + let refresh = cluster_json(temp.path(), "refresh"); + assert_eq!(refresh["ok"], true, "{refresh}"); + + let converge = cluster_json(temp.path(), "apply"); + assert_eq!(converge["ok"], true, "{converge}"); + assert_eq!(converge["converged"], true, "{converge}"); + assert_eq!( + change_for(&converge, "query.engineering.find_service")["disposition"], + "applied" + ); + assert_eq!(change_for(&converge, "policy.shared")["disposition"], "applied"); + + let final_plan = cluster_json(temp.path(), "plan"); + assert!( + final_plan["changes"].as_array().unwrap().is_empty(), + "{final_plan}" + ); +} + #[test] fn short_version_flag_prints_current_cli_version() { let output = output_success(cli().arg("-v")); diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 1f818e9..1eebeb2 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -7,7 +7,7 @@ This file is the always-on map of the test surface. **Consult it before every ta | Crate | Path | Style | |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | -| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | +| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish; includes the `cluster_e2e_*` lifecycle compositions over the spawned binary — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | | `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests` | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, and config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply) | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | From 15868972ff5ad492ababba36bade9a52fe3cd5fb Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 02:07:08 +0300 Subject: [PATCH 047/165] feat(cluster): verify catalog payload blobs in status and refresh MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the Stage 3A product gap where a deleted or corrupted blob under __cluster/resources/ went unnoticed forever (status reported converged and apply could not repair it because the digests matched). verify_catalog_payloads checks every query/policy digest in state against its content-addressed blob (existence + full sha256 re-hash; graph/schema/unknown addresses have no payloads and are skipped). status reports findings read-only (warnings catalog_payload_missing/_mismatch; error catalog_payload_read_error — an unverifiable catalog must not report healthy). refresh closes the self-heal loop: missing/mismatched blobs mark the resource drifted and remove its digest from state so the next plan proposes a create and the next apply republishes; unreadable blobs keep the digest (no spurious republish), mark error, and exit non-zero. Verification runs before graph observation so the recomputed graph composite already excludes removed query digests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 281 ++++++++++++++++++++++++++++ docs/user/cluster-config.md | 33 +++- 2 files changed, 312 insertions(+), 2 deletions(-) diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 3673194..84968a7 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -890,6 +890,14 @@ pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { match backend.read_state(&mut observations) { Ok(snapshot) => { if let Some(state) = snapshot.state { + // Read-only point-in-time catalog check: report the + // findings as diagnostics; persisting Drifted statuses + // is refresh's job. Status never writes state. + for (address, finding) in + verify_catalog_payloads(&parsed.config_dir, &state) + { + diagnostics.push(payload_finding_diagnostic(&address, &finding)); + } resource_digests = state_resource_digests(&state); resource_statuses = state.resource_statuses; state_observation_records = state.observations; @@ -1076,6 +1084,47 @@ async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> St (StateSyncOperation::Import, None) => initial_import_state(&desired), }; + // Catalog payload verification must run BEFORE graph observation: removing + // a drifted query digest first means the live-graph composite recompute + // below already excludes it, so the persisted graph.<id> composite stays + // consistent and the next plan shows exactly the create + derived update. + for (address, finding) in verify_catalog_payloads(&desired.config_dir, &state) { + diagnostics.push(payload_finding_diagnostic(&address, &finding)); + match finding { + PayloadFinding::Missing => { + state.applied_revision.resources.remove(&address); + set_resource_status( + &mut state, + &address, + ResourceLifecycleStatus::Drifted, + "payload_missing", + "catalog payload blob is missing; re-run `cluster apply` to republish", + ); + } + PayloadFinding::Mismatch { .. } => { + state.applied_revision.resources.remove(&address); + set_resource_status( + &mut state, + &address, + ResourceLifecycleStatus::Drifted, + "payload_mismatch", + "catalog payload blob does not match the recorded digest; re-run `cluster apply` to republish", + ); + } + // Transient IO must not trigger a spurious republish: keep the + // digest, surface the error, let a later clean refresh converge. + PayloadFinding::ReadError(error) => { + set_resource_status( + &mut state, + &address, + ResourceLifecycleStatus::Error, + "payload_read_error", + &error, + ); + } + } + } + let graph_error_count = observe_declared_graphs(&desired, &mut state).await; if graph_error_count > 0 { diagnostics.push(Diagnostic::error( @@ -2371,6 +2420,73 @@ fn payload_path(config_dir: &Path, kind: &ResourceKind, digest: &str) -> Option< } } +#[derive(Debug, PartialEq, Eq)] +enum PayloadFinding { + Missing, + Mismatch { actual_digest: String }, + ReadError(String), +} + +/// Verify every catalog-backed resource digest in state against its +/// content-addressed blob under `__cluster/resources/`. Graph, schema, and +/// unknown addresses have no payloads and are skipped. Read-only; findings +/// are deterministic (BTreeMap order). Payloads are small (queries, policy +/// bundles), so a full digest re-hash is cheap. +fn verify_catalog_payloads( + config_dir: &Path, + state: &ClusterState, +) -> Vec<(String, PayloadFinding)> { + let mut findings = Vec::new(); + for (address, resource) in &state.applied_revision.resources { + let kind = resource_kind(address); + let Some(path) = payload_path(config_dir, &kind, &resource.digest) else { + continue; + }; + match fs::read(&path) { + Ok(bytes) => { + let actual_digest = sha256_hex(&bytes); + if actual_digest != resource.digest { + findings.push((address.clone(), PayloadFinding::Mismatch { actual_digest })); + } + } + Err(err) if err.kind() == ErrorKind::NotFound => { + findings.push((address.clone(), PayloadFinding::Missing)); + } + Err(err) => { + findings.push(( + address.clone(), + PayloadFinding::ReadError(format!( + "could not read catalog payload '{}': {err}", + path.display() + )), + )); + } + } + } + findings +} + +fn payload_finding_diagnostic(address: &str, finding: &PayloadFinding) -> Diagnostic { + match finding { + PayloadFinding::Missing => Diagnostic::warning( + "catalog_payload_missing", + address, + "catalog payload blob is missing; re-run `cluster apply` to republish", + ), + PayloadFinding::Mismatch { actual_digest } => Diagnostic::warning( + "catalog_payload_mismatch", + address, + format!( + "catalog payload blob does not match the recorded digest (actual sha256:{actual_digest}); re-run `cluster apply` to republish" + ), + ), + // An unverifiable blob must not report healthy. + PayloadFinding::ReadError(error) => { + Diagnostic::error("catalog_payload_read_error", address, error.clone()) + } + } +} + /// Write one content-addressed payload blob. Idempotent: an existing /// digest-named file is trusted as-is. The digest re-check is the apply-side /// TOCTOU detector — the source file changing between `load_desired` and the @@ -4317,6 +4433,171 @@ graphs: ); } + // ---- catalog payload verification (Stage 3B) ---- + + /// Converge a fixture dir and return the query blob path. + fn converge_fixture(config_dir: &Path) -> std::path::PathBuf { + write_applyable_state(config_dir); + let out = apply_config_dir(config_dir); + assert!(out.ok && out.converged, "{:?}", out.diagnostics); + let desired = validate_config_dir(config_dir); + query_payload_path( + config_dir, + desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap(), + ) + } + + #[test] + fn status_reports_missing_payload_read_only() { + let dir = fixture(); + let blob = converge_fixture(dir.path()); + let state_before = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); + fs::remove_file(&blob).unwrap(); + + let out = status_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.diagnostics.iter().any(|diagnostic| { + diagnostic.code == "catalog_payload_missing" + && diagnostic.path == "query.knowledge.find_person" + })); + // Read-only: persisted statuses and state bytes untouched. + assert_eq!( + out.resource_statuses["query.knowledge.find_person"].status, + ResourceLifecycleStatus::Applied + ); + assert_eq!( + fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(), + state_before + ); + } + + #[tokio::test] + async fn refresh_removes_digest_and_drifts_on_missing_payload() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let blob = converge_fixture(dir.path()); + fs::remove_file(&blob).unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "catalog_payload_missing") + ); + let status = &out.resource_statuses["query.knowledge.find_person"]; + assert_eq!(status.status, ResourceLifecycleStatus::Drifted); + assert!(status.conditions.contains(&"payload_missing".to_string())); + let state = read_state_json(dir.path()); + assert!( + state["applied_revision"]["resources"] + .get("query.knowledge.find_person") + .is_none(), + "{state}" + ); + } + + #[tokio::test] + async fn refresh_drifts_on_corrupted_payload() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let blob = converge_fixture(dir.path()); + fs::write(&blob, "corrupted content").unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + let status = &out.resource_statuses["query.knowledge.find_person"]; + assert_eq!(status.status, ResourceLifecycleStatus::Drifted); + assert!(status.conditions.contains(&"payload_mismatch".to_string())); + let state = read_state_json(dir.path()); + assert!( + state["applied_revision"]["resources"] + .get("query.knowledge.find_person") + .is_none() + ); + } + + #[tokio::test] + async fn refresh_flags_unreadable_payload_as_error() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let blob = converge_fixture(dir.path()); + // A same-named directory yields a non-NotFound IO error portably. + fs::remove_file(&blob).unwrap(); + fs::create_dir(&blob).unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "catalog_payload_read_error") + ); + let status = &out.resource_statuses["query.knowledge.find_person"]; + assert_eq!(status.status, ResourceLifecycleStatus::Error); + assert!(status.conditions.contains(&"payload_read_error".to_string())); + // Transient IO keeps the digest: no spurious republish. + let state = read_state_json(dir.path()); + assert!( + state["applied_revision"]["resources"] + .get("query.knowledge.find_person") + .is_some() + ); + } + + #[tokio::test] + async fn payload_drift_self_heals_through_refresh_plan_apply() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let blob = converge_fixture(dir.path()); + let original = fs::read_to_string(&blob).unwrap(); + fs::remove_file(&blob).unwrap(); + + let refresh = refresh_config_dir(dir.path()).await; + assert!(refresh.ok, "{:?}", refresh.diagnostics); + + let plan = plan_config_dir(dir.path()); + let query_change = plan + .changes + .iter() + .find(|change| change.resource == "query.knowledge.find_person") + .expect("plan must propose recreating the query"); + assert_eq!(query_change.operation, PlanOperation::Create); + assert_eq!(query_change.disposition, Some(ApplyDisposition::Applied)); + + let apply = apply_config_dir(dir.path()); + assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); + assert_eq!(fs::read_to_string(&blob).unwrap(), original); + + let status = status_config_dir(dir.path()); + assert!( + !status + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code.starts_with("catalog_payload")), + "{:?}", + status.diagnostics + ); + } + + #[test] + fn verification_skips_graph_and_schema_resources() { + let dir = fixture(); + write_applyable_state(dir.path()); // graph + schema digests only, no blobs + + let out = status_config_dir(dir.path()); + assert!( + !out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code.starts_with("catalog_payload")), + "{:?}", + out.diagnostics + ); + } + #[test] fn plan_annotates_apply_dispositions() { let dir = fixture(); diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 912f307..9a2597b 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -198,6 +198,16 @@ files and does not inspect live graphs. Missing `state.json` succeeds with a warning; invalid state JSON or an unsupported state version fails. If a lock is present, status reports its id, operation, creation time, pid, and age. +Status also verifies the catalog payloads read-only: every query/policy digest +recorded in state is checked against its content-addressed blob under +`__cluster/resources/` (existence and full digest re-hash). A missing or +mismatched blob is reported as a warning (`catalog_payload_missing` / +`catalog_payload_mismatch`); an unreadable blob is an error +(`catalog_payload_read_error`) because an unverifiable catalog must not report +healthy. Status never writes state — persisting the `drifted` condition is +refresh's job. The check runs without the state lock, so it is a point-in-time +report. + ## Refresh And Import `cluster refresh` updates an existing `state.json` from actual observations. @@ -216,8 +226,27 @@ Invalid graph roots are recorded as errors; `refresh` persists the error observation and exits non-zero, while `import` exits non-zero without creating initial state. -Refresh/import do not observe query or policy resources yet. Existing query and -policy state digests are preserved on refresh and are not invented on import. +Refresh also verifies the catalog payloads of every query/policy digest +recorded in state (the same check `cluster status` reports read-only), and +closes the loop: + +- a **missing** or **digest-mismatched** blob marks the resource `drifted` + (condition `payload_missing` / `payload_mismatch`) and removes its digest + from state — so the next `cluster plan` proposes a create and the next + `cluster apply` republishes the blob (the self-heal loop, mirroring how a + missing graph root is handled); +- an **unreadable** blob (IO error other than not-found) keeps the digest, + marks the resource `error` (condition `payload_read_error`), and exits + non-zero — transient IO must not trigger a spurious republish. + +Upgrade note: a state ledger written before catalog publish existed records +query/policy digests with no blobs on disk; the first refresh after upgrading +flags them all `payload_missing`, and a single `cluster apply` republishes +everything and converges. + +Refresh/import do not observe query or policy resources beyond their catalog +payloads yet. Existing query and policy state digests are preserved on refresh +(unless their payload drifted, above) and are not invented on import. ## Force Unlock From acb3f1cc14552d4b55a713de9c773d100fd3a25d Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 02:08:14 +0300 Subject: [PATCH 048/165] test(cli): e2e for catalog payload drift self-heal loop status warns read-only -> refresh persists drift and drops the digest -> apply republishes the blob -> status clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/cli.rs | 61 +++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index f60ffbe..d47e13c 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -1435,6 +1435,67 @@ fn cluster_e2e_multi_graph_mixed_dispositions_then_converge() { ); } +/// Catalog payload drift self-heals across the command surface: status warns +/// read-only, refresh persists the drift and drops the digest, apply +/// republishes the blob, status comes back clean. +#[test] +fn cluster_e2e_payload_drift_self_heals() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["converged"], true, "{apply}"); + + let query_digest = change_for(&apply, "query.knowledge.find_person")["after_digest"] + .as_str() + .unwrap() + .to_string(); + let blob = temp + .path() + .join("__cluster/resources/query/knowledge/find_person") + .join(format!("{query_digest}.gq")); + fs::remove_file(&blob).unwrap(); + + let status = cluster_json(temp.path(), "status"); + assert_eq!(status["ok"], true, "{status}"); + assert!( + status["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "catalog_payload_missing"), + "{status}" + ); + + let refresh = cluster_json(temp.path(), "refresh"); + assert_eq!(refresh["ok"], true, "{refresh}"); + assert_eq!( + refresh["resource_statuses"]["query.knowledge.find_person"]["status"], + "drifted" + ); + + let heal = cluster_json(temp.path(), "apply"); + assert_eq!(heal["ok"], true, "{heal}"); + assert_eq!(heal["converged"], true, "{heal}"); + assert!(blob.exists(), "blob republished"); + + let clean = cluster_json(temp.path(), "status"); + assert!( + !clean["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| { + diagnostic["code"] + .as_str() + .is_some_and(|code| code.starts_with("catalog_payload")) + }), + "{clean}" + ); +} + #[test] fn short_version_flag_prints_current_cli_version() { let output = output_success(cli().arg("-v")); From 21b531605fed16b486280ba3e2944e7f15a1604e Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 02:12:59 +0300 Subject: [PATCH 049/165] feat(cluster): failpoint infrastructure mirroring the engine MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Optional failpoints feature (dep:fail + fail/failpoints, deliberately NOT enabling omnigraph/failpoints), a maybe_fail/ScopedFailPoint module returning Diagnostic-typed injected errors, and two call sites in apply_config_dir: cluster_apply.after_payload_phase (the crash point: blobs on disk, state untouched) and cluster_apply.before_state_write (routes through the persisted-statuses revert contract; a cfg_callback here can mutate state.json to make the CAS check fail organically). Feature off compiles to Ok(()) — zero behavior change. Tests live in a separate integration binary because the fail registry is process-global. Also refresh the crate description (stale 'read-only' since Stage 3A). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/Cargo.toml | 8 +- crates/omnigraph-cluster/src/failpoints.rs | 42 +++++++ crates/omnigraph-cluster/src/lib.rs | 25 +++- crates/omnigraph-cluster/tests/failpoints.rs | 119 +++++++++++++++++++ 4 files changed, 192 insertions(+), 2 deletions(-) create mode 100644 crates/omnigraph-cluster/src/failpoints.rs create mode 100644 crates/omnigraph-cluster/tests/failpoints.rs diff --git a/crates/omnigraph-cluster/Cargo.toml b/crates/omnigraph-cluster/Cargo.toml index 9280c42..b5f99c9 100644 --- a/crates/omnigraph-cluster/Cargo.toml +++ b/crates/omnigraph-cluster/Cargo.toml @@ -2,15 +2,21 @@ name = "omnigraph-cluster" version = "0.6.2" edition = "2024" -description = "Read-only cluster configuration validation and planning for Omnigraph." +description = "Cluster configuration validation, planning, and config-only apply for Omnigraph." license = "MIT" repository = "https://github.com/ModernRelay/omnigraph" homepage = "https://github.com/ModernRelay/omnigraph" documentation = "https://docs.rs/omnigraph-cluster" +[features] +# Fault-injection hooks for the apply protocol (crash-mid-apply, CAS-race +# tests). Deliberately does NOT enable omnigraph/failpoints. +failpoints = ["dep:fail", "fail/failpoints"] + [dependencies] omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.2" } +fail = { workspace = true, optional = true } serde = { workspace = true } serde_json = { workspace = true } serde_yaml = { workspace = true } diff --git a/crates/omnigraph-cluster/src/failpoints.rs b/crates/omnigraph-cluster/src/failpoints.rs new file mode 100644 index 0000000..c6d445b --- /dev/null +++ b/crates/omnigraph-cluster/src/failpoints.rs @@ -0,0 +1,42 @@ +//! Fault-injection hooks for the cluster apply protocol, mirroring the +//! engine's `omnigraph::failpoints` pattern. With the `failpoints` feature +//! off, every call site compiles to `Ok(())`. + +use crate::Diagnostic; + +pub(crate) fn maybe_fail(_name: &str) -> Result<(), Diagnostic> { + #[cfg(feature = "failpoints")] + { + let name = _name; + fail::fail_point!(name, |_| { + return Err(Diagnostic::error( + "injected_failpoint", + name, + format!("injected failpoint triggered: {name}"), + )); + }); + } + Ok(()) +} + +#[cfg(feature = "failpoints")] +pub struct ScopedFailPoint { + name: String, +} + +#[cfg(feature = "failpoints")] +impl ScopedFailPoint { + pub fn new(name: &str, action: &str) -> Self { + fail::cfg(name, action).expect("configure failpoint"); + Self { + name: name.to_string(), + } + } +} + +#[cfg(feature = "failpoints")] +impl Drop for ScopedFailPoint { + fn drop(&mut self) { + fail::remove(&self.name); + } +} diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 84968a7..660f34c 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -16,6 +16,8 @@ use time::OffsetDateTime; use time::format_description::well_known::Rfc3339; use ulid::Ulid; +pub mod failpoints; + pub const CLUSTER_CONFIG_FILE: &str = "cluster.yaml"; pub const CLUSTER_GRAPHS_DIR: &str = "graphs"; pub const CLUSTER_STATE_DIR: &str = "__cluster"; @@ -770,6 +772,21 @@ pub fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { ); } + // Crash point: payloads are on disk, state has not moved. A failure here + // must leave state.json byte-identical and acknowledge nothing; re-running + // apply repairs via the skip-if-exists blob reuse. + if let Err(diagnostic) = failpoints::maybe_fail("cluster_apply.after_payload_phase") { + diagnostics.push(diagnostic); + return early_return( + display_path(&desired.config_dir), + Some(desired.config_digest), + observations, + changes, + state.resource_statuses, + diagnostics, + ); + } + // State mutation. Apply owns query/policy statuses only; graph/schema // statuses belong to refresh/import observation and must not be clobbered. let before_value = @@ -824,7 +841,13 @@ pub fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { let mut state_write_failed = false; if after_value != before_value { new_state.state_revision = new_state.state_revision.saturating_add(1); - match backend.write_state(&new_state, expected_cas.as_deref(), &mut observations) { + // The failpoint error routes through state_write_failed so the + // persisted-statuses revert contract below is exercised; a cfg_callback + // on this point can mutate state.json to simulate a concurrent writer, + // making write_state's CAS check fail organically. + let write_result = failpoints::maybe_fail("cluster_apply.before_state_write") + .and_then(|()| backend.write_state(&new_state, expected_cas.as_deref(), &mut observations)); + match write_result { Ok(()) => state_written = true, Err(diagnostic) => { diagnostics.push(diagnostic); diff --git a/crates/omnigraph-cluster/tests/failpoints.rs b/crates/omnigraph-cluster/tests/failpoints.rs new file mode 100644 index 0000000..3ede30c --- /dev/null +++ b/crates/omnigraph-cluster/tests/failpoints.rs @@ -0,0 +1,119 @@ +//! Fault-injection tests for the cluster apply protocol. +//! +//! These live in an integration binary (not in-source) deliberately: the fail +//! crate's registry is process-global, so a configured `cluster_apply.*` +//! action would fire inside any concurrently running normal apply test in the +//! lib-test process. A separate binary isolates the registry by construction — +//! same reason the engine keeps its failpoint suite in `tests/failpoints.rs`. + +#![cfg(feature = "failpoints")] + +use std::collections::BTreeMap; +use std::fs; +use std::path::{Path, PathBuf}; + +use fail::FailScenario; +use omnigraph_cluster::failpoints::ScopedFailPoint; +use omnigraph_cluster::{apply_config_dir, validate_config_dir}; +use tempfile::tempdir; + +const SCHEMA: &str = r#" +node Person { + name: String @key + age: I32? +} +"#; + +const QUERY: &str = r#" +query find_person($name: String) { + match { $p: Person { name: $name } } + return { $p.name, $p.age } +} +"#; + +fn fixture() -> tempfile::TempDir { + let dir = tempdir().unwrap(); + fs::write(dir.path().join("people.pg"), SCHEMA).unwrap(); + fs::write(dir.path().join("people.gq"), QUERY).unwrap(); + fs::write(dir.path().join("base.policy.yaml"), "rules: []\n").unwrap(); + fs::write( + dir.path().join("cluster.yaml"), + r#" +version: 1 +state: + backend: cluster + lock: true +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +policies: + base: + file: ./base.policy.yaml + applies_to: [knowledge] +"#, + ) + .unwrap(); + dir +} + +/// Seed a state.json where the graph/schema digests match desired, so query +/// and policy changes are applicable. Digests are borrowed from the public +/// validate output; the graph composite is a placeholder that apply converges +/// as a Derived update. +fn seed_applyable_state(config_dir: &Path) -> BTreeMap<String, String> { + let validate = validate_config_dir(config_dir); + assert!(validate.ok, "{:?}", validate.diagnostics); + let schema_digest = validate.resource_digests["schema.knowledge"].clone(); + let state_dir = config_dir.join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + format!( + r#"{{ + "version": 1, + "state_revision": 1, + "applied_revision": {{ + "resources": {{ + "graph.knowledge": {{ "digest": "seed" }}, + "schema.knowledge": {{ "digest": "{schema_digest}" }} + }} + }} +}} +"# + ), + ) + .unwrap(); + validate.resource_digests +} + +fn state_path(config_dir: &Path) -> PathBuf { + config_dir.join("__cluster/state.json") +} + +fn query_blob(config_dir: &Path, digests: &BTreeMap<String, String>) -> PathBuf { + config_dir + .join("__cluster/resources/query/knowledge/find_person") + .join(format!("{}.gq", digests["query.knowledge.find_person"])) +} + +#[test] +fn failpoint_wiring_returns_injected_diagnostic() { + let scenario = FailScenario::setup(); + let dir = fixture(); + seed_applyable_state(dir.path()); + + let _failpoint = ScopedFailPoint::new("cluster_apply.after_payload_phase", "return"); + let out = apply_config_dir(dir.path()); + assert!(!out.ok); + assert!(out.diagnostics.iter().any(|diagnostic| { + diagnostic.code == "injected_failpoint" + && diagnostic + .message + .contains("cluster_apply.after_payload_phase") + })); + drop(_failpoint); + scenario.teardown(); +} From 211b37e6de8977b5264a179c10e6383f66d6b711 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 02:14:06 +0300 Subject: [PATCH 050/165] test(cluster): failpoint tests for crash-mid-apply and state CAS race The apply-side coverage the implementation spec's hard gate requires before Phase 4 graph-moving apply: - crash after the payload phase: state.json byte-identical, blobs inert on disk, lock released, no phantom statuses, nothing acknowledged; a plain re-run repairs via skip-if-exists blob reuse. - CAS race: a cfg_callback rewrites state.json at the exact read->write window (the state.lock:false concurrent-writer scenario); apply surfaces state_cas_mismatch, acknowledges nothing, reports the persisted status snapshot, leaves the concurrent writer's state on disk; a re-run converges. CI's failpoints step now runs both the engine and cluster suites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- .github/workflows/ci.yml | 11 ++- crates/omnigraph-cluster/tests/failpoints.rs | 99 ++++++++++++++++++++ 2 files changed, 106 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index bbe5893..1ea6c37 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -173,15 +173,18 @@ jobs: OMNIGRAPH_UPDATE_OPENAPI: ${{ (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository) && '1' || '' }} run: cargo test --workspace --locked - - name: Run failpoints feature test + - name: Run failpoints feature tests if: needs.classify_changes.outputs.run_full_ci == 'true' # Run after the workspace test so the build cache is warm — # enabling --features failpoints is just an incremental rebuild - # of omnigraph-engine + the small `fail` crate, not the full + # of the target crate + the small `fail` crate, not the full # dep tree (lance, datafusion). A separate job with its own # cache key would be a fresh ~20min build on first run; this - # is ~30s on a warm cache. - run: cargo test --locked -p omnigraph-engine --features failpoints --test failpoints + # is ~30s on a warm cache. The cluster feature does not enable + # omnigraph/failpoints, so each line rebuilds only its crate. + run: | + cargo test --locked -p omnigraph-engine --features failpoints --test failpoints + cargo test --locked -p omnigraph-cluster --features failpoints --test failpoints - name: Commit regenerated openapi.json to PR branch if: | diff --git a/crates/omnigraph-cluster/tests/failpoints.rs b/crates/omnigraph-cluster/tests/failpoints.rs index 3ede30c..05d2913 100644 --- a/crates/omnigraph-cluster/tests/failpoints.rs +++ b/crates/omnigraph-cluster/tests/failpoints.rs @@ -117,3 +117,102 @@ fn failpoint_wiring_returns_injected_diagnostic() { drop(_failpoint); scenario.teardown(); } + +/// Crash between the payload phase and the state write: blobs are on disk, +/// state.json is byte-identical, nothing is acknowledged — and a plain re-run +/// repairs by trusting the existing content-addressed blobs. +#[test] +fn apply_crash_after_payload_phase_leaves_state_unmoved_then_recovers() { + let scenario = FailScenario::setup(); + let dir = fixture(); + let digests = seed_applyable_state(dir.path()); + let state_before = fs::read(state_path(dir.path())).unwrap(); + + { + let _failpoint = ScopedFailPoint::new("cluster_apply.after_payload_phase", "return"); + let out = apply_config_dir(dir.path()); + assert!(!out.ok); + assert!(!out.state_written); + assert!(!out.converged); + assert_eq!(out.applied_count, 0); + // Persisted pre-apply snapshot: no phantom Applied statuses. + assert!( + !out.resource_statuses + .contains_key("query.knowledge.find_person"), + "{:?}", + out.resource_statuses + ); + // State has not moved; payloads are inert on disk; the lock released. + assert_eq!(fs::read(state_path(dir.path())).unwrap(), state_before); + assert!(query_blob(dir.path(), &digests).exists()); + assert!(!dir.path().join("__cluster/lock.json").exists()); + } + + // The repair is a plain re-run: existing blobs are trusted by digest. + let recovered = apply_config_dir(dir.path()); + assert!(recovered.ok, "{:?}", recovered.diagnostics); + assert!(recovered.converged); + assert!(recovered.state_written); + assert_eq!( + recovered.resource_statuses["query.knowledge.find_person"].status, + omnigraph_cluster::ResourceLifecycleStatus::Applied + ); + scenario.teardown(); +} + +/// A concurrent writer mutating state.json between apply's read and its write +/// (possible under `state.lock: false`) must surface `state_cas_mismatch`, +/// acknowledge nothing, and leave the concurrent writer's state on disk. +#[test] +fn apply_cas_race_surfaces_state_cas_mismatch() { + let scenario = FailScenario::setup(); + let dir = fixture(); + let digests = seed_applyable_state(dir.path()); + + // Simulate the concurrent writer at the exact race window: rewrite + // state.json (valid JSON, graph/schema digests preserved, revision 99) + // after apply read it but before apply writes. + let race_path = state_path(dir.path()); + fail::cfg_callback("cluster_apply.before_state_write", move || { + let mut state: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&race_path).unwrap()).unwrap(); + state["state_revision"] = serde_json::json!(99); + fs::write(&race_path, serde_json::to_string_pretty(&state).unwrap()).unwrap(); + }) + .expect("configure callback failpoint"); + + let out = apply_config_dir(dir.path()); + fail::remove("cluster_apply.before_state_write"); + + assert!(!out.ok); + assert!(!out.state_written); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_cas_mismatch"), + "{:?}", + out.diagnostics + ); + // Persisted snapshot, not the unwritten in-memory mutations. + assert!( + !out.resource_statuses + .contains_key("query.knowledge.find_person") + ); + // The concurrent writer's state is what's on disk; apply's mutation never landed. + let state: serde_json::Value = + serde_json::from_str(&fs::read_to_string(state_path(dir.path())).unwrap()).unwrap(); + assert_eq!(state["state_revision"], 99); + assert!( + state["applied_revision"]["resources"] + .get("query.knowledge.find_person") + .is_none() + ); + // Blobs written before the race are inert. + assert!(query_blob(dir.path(), &digests).exists()); + + // Recovery is a plain re-run against the rewritten state. + let recovered = apply_config_dir(dir.path()); + assert!(recovered.ok, "{:?}", recovered.diagnostics); + assert!(recovered.converged); + scenario.teardown(); +} From 50543a8ce073444f7e68886a65a14e08b4dafbe9 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 02:15:13 +0300 Subject: [PATCH 051/165] docs(cluster): record Stage 3B failpoint + verification coverage Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/cluster-config-implementation-spec.md | 6 +++++- docs/dev/testing.md | 10 +++++----- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/docs/dev/cluster-config-implementation-spec.md b/docs/dev/cluster-config-implementation-spec.md index ff3dd7e..f3c5b68 100644 --- a/docs/dev/cluster-config-implementation-spec.md +++ b/docs/dev/cluster-config-implementation-spec.md @@ -663,7 +663,11 @@ Hard gates: - Do not ship `cluster apply` until `cluster validate` and read-only `cluster plan` have hermetic tests. - Do not ship graph/schema-moving apply until failpoint recovery tests prove the - Phase B -> state publish gap is covered. + Phase B -> state publish gap is covered. (Stage 3B delivered the apply-side + half: `omnigraph-cluster` has failpoint infrastructure and tests for the + crash-after-payload and state-CAS-race windows of config-only apply, plus + catalog payload verification in status/refresh. Graph-moving sidecar + coverage remains Phase 4 work.) For docs-only changes, `scripts/check-agents-md.sh` is enough. For implementation phases, run the boundary tests above before widening to diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 1eebeb2..5c88a37 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,7 +8,7 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish; includes the `cluster_e2e_*` lifecycle compositions over the spawned binary — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | -| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests` | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, and config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply) | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), and failpoint crash-mid-apply / CAS-race coverage | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | @@ -54,10 +54,10 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav ## Failpoints (fault injection) -- Cargo feature: `failpoints = ["dep:fail", "fail/failpoints"]` (in `crates/omnigraph/Cargo.toml`). -- Wrapper: `crates/omnigraph/src/failpoints.rs` exposes `maybe_fail("name")` and `ScopedFailPoint` for tests. -- Call sites are inserted at sensitive transaction boundaries (branch create, graph publish commit, etc.). -- Activated tests: `crates/omnigraph/tests/failpoints.rs`. Run with `cargo test -p omnigraph-engine --features failpoints --test failpoints`. +- Cargo feature: `failpoints = ["dep:fail", "fail/failpoints"]` (in `crates/omnigraph/Cargo.toml` **and** `crates/omnigraph-cluster/Cargo.toml`; the cluster feature does not enable the engine's). +- Wrappers: `crates/omnigraph/src/failpoints.rs` and `crates/omnigraph-cluster/src/failpoints.rs` expose `maybe_fail("name")` and `ScopedFailPoint` for tests. +- Call sites are inserted at sensitive transaction boundaries (branch create, graph publish commit, cluster apply's payload→state-write window, etc.). +- Activated tests: `crates/omnigraph/tests/failpoints.rs` and `crates/omnigraph-cluster/tests/failpoints.rs` (crash-mid-apply + state CAS race via `fail::cfg_callback`; integration binaries, never in-source — the fail registry is process-global). Run with `cargo test -p omnigraph-engine --features failpoints --test failpoints` / `cargo test -p omnigraph-cluster --features failpoints --test failpoints`. ## RustFS / S3 integration From 08ea659c9bcc39902233f0f771eb7883f9b97876 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 02:21:10 +0300 Subject: [PATCH 052/165] build: commit Cargo.lock for omnigraph-cluster's optional fail dependency The failpoints feature added fail = { workspace = true, optional = true } to the crate manifest; the lockfile edge belongs with it (--locked CI gate). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- Cargo.lock | 1 + 1 file changed, 1 insertion(+) diff --git a/Cargo.lock b/Cargo.lock index 79760b0..f6a1b8a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4568,6 +4568,7 @@ dependencies = [ name = "omnigraph-cluster" version = "0.6.2" dependencies = [ + "fail", "omnigraph-compiler", "omnigraph-engine", "serde", From 16759b28b9c13b445e49db9363756d24d6a26aaa Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 02:36:24 +0300 Subject: [PATCH 053/165] fix(cluster): RAII-guard the callback failpoint ScopedFailPoint::with_callback gives cfg_callback the same Drop-based cleanup as cfg actions; a panic while the point is active no longer leaks the callback into the process-global registry where it would fire under later tests (greptile review, PR #167). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/failpoints.rs | 14 ++++++++++++++ crates/omnigraph-cluster/tests/failpoints.rs | 19 ++++++++++--------- 2 files changed, 24 insertions(+), 9 deletions(-) diff --git a/crates/omnigraph-cluster/src/failpoints.rs b/crates/omnigraph-cluster/src/failpoints.rs index c6d445b..f1799d7 100644 --- a/crates/omnigraph-cluster/src/failpoints.rs +++ b/crates/omnigraph-cluster/src/failpoints.rs @@ -32,6 +32,20 @@ impl ScopedFailPoint { name: name.to_string(), } } + + /// Register a callback failpoint with the same Drop-based cleanup as + /// `new`. Without the guard, a panic while the point is active would + /// leak the callback into the process-global registry and fire it under + /// later tests in the same binary. + pub fn with_callback<F>(name: &str, callback: F) -> Self + where + F: Fn() + Send + Sync + 'static, + { + fail::cfg_callback(name, callback).expect("configure callback failpoint"); + Self { + name: name.to_string(), + } + } } #[cfg(feature = "failpoints")] diff --git a/crates/omnigraph-cluster/tests/failpoints.rs b/crates/omnigraph-cluster/tests/failpoints.rs index 05d2913..db7b82d 100644 --- a/crates/omnigraph-cluster/tests/failpoints.rs +++ b/crates/omnigraph-cluster/tests/failpoints.rs @@ -171,18 +171,19 @@ fn apply_cas_race_surfaces_state_cas_mismatch() { // Simulate the concurrent writer at the exact race window: rewrite // state.json (valid JSON, graph/schema digests preserved, revision 99) - // after apply read it but before apply writes. + // after apply read it but before apply writes. RAII-guarded so a panic + // inside apply cannot leak the callback into the global registry. let race_path = state_path(dir.path()); - fail::cfg_callback("cluster_apply.before_state_write", move || { - let mut state: serde_json::Value = - serde_json::from_str(&fs::read_to_string(&race_path).unwrap()).unwrap(); - state["state_revision"] = serde_json::json!(99); - fs::write(&race_path, serde_json::to_string_pretty(&state).unwrap()).unwrap(); - }) - .expect("configure callback failpoint"); + let failpoint = + ScopedFailPoint::with_callback("cluster_apply.before_state_write", move || { + let mut state: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&race_path).unwrap()).unwrap(); + state["state_revision"] = serde_json::json!(99); + fs::write(&race_path, serde_json::to_string_pretty(&state).unwrap()).unwrap(); + }); let out = apply_config_dir(dir.path()); - fail::remove("cluster_apply.before_state_write"); + drop(failpoint); assert!(!out.ok); assert!(!out.state_written); From 58c66a54a2c1d4dbad1802e27118b177b0f2e829 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 04:34:14 +0300 Subject: [PATCH 054/165] =?UTF-8?q?docs(cluster):=20RFC-004=20=E2=80=94=20?= =?UTF-8?q?graph=20&=20schema=20apply=20design=20(Phase=204)=20(#168)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs(cluster): RFC-004 — graph & schema apply design (Phase 4) The design the implementation spec's exit criteria require before graph-moving cluster apply ships. Core positions: - Cluster recovery is roll-forward-only: the engine's own sidecars make every graph-level operation atomic within the graph, so the cluster never rolls a graph back — its sidecars (__cluster/recoveries/{ulid}.json) classify and record, converging the ledger to observable reality (axiom 5) or surfacing a loud pending-repair condition. Eight-row decision matrix, every row testable with the Stage 3B failpoint harness. - Irreversible operations (graph delete, allow_data_loss schema apply) consume digest-bound approval artifacts written by a new cluster approve command and retired into state.approval_records (axiom 11). A stale approval can never authorize a different change. - cluster apply gains an actor, threaded to apply_schema_as so engine Cedar enforcement and commit attribution work unchanged; the cluster adds no policy engine of its own. - Deterministic ordering (creates -> schema applies -> catalog -> deletes), per-resource apply groups, cross-graph atomicity explicitly not promised. - Staged 4A graph create / 4B schema apply / 4C graph delete, each gated on per-matrix-row failpoint tests. Answers exit criteria 2 and 4 fully, 1/5/6 partially; 3/7/8/9 deferred to their phases (coverage table in the RFC). Linked from the dev index and the implementation spec's Phase 4 section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(cluster): RFC-004 review fixes — graph_delete sweep rows, state_cas_base contract Two greptile findings: (1) D3 row 2 could not be evaluated for graph_delete (no manifest to version-check after prefix removal) and 'root absent, state already tombstoned' fell into the stale row — split into rows 7 (delete's analog of row 2) and 7b (the roll-forward), with expected_manifest_version documented as always null for the delete kind. (2) state_cas_base is now explicitly audit/diagnostics-only — the sweep never consults it; independent state mutations are handled by the ordinary CAS like any concurrent write. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --- .../dev/cluster-config-implementation-spec.md | 4 + docs/dev/index.md | 1 + .../dev/rfc-004-cluster-graph-schema-apply.md | 210 ++++++++++++++++++ 3 files changed, 215 insertions(+) create mode 100644 docs/dev/rfc-004-cluster-graph-schema-apply.md diff --git a/docs/dev/cluster-config-implementation-spec.md b/docs/dev/cluster-config-implementation-spec.md index f3c5b68..d4cf3e6 100644 --- a/docs/dev/cluster-config-implementation-spec.md +++ b/docs/dev/cluster-config-implementation-spec.md @@ -588,6 +588,10 @@ replacement would make every invariant harder to audit. --> ### Phase 4: Graph And Schema Apply +Detailed design: [rfc-004-cluster-graph-schema-apply.md](rfc-004-cluster-graph-schema-apply.md) +(cluster sidecar schema, roll-forward-only recovery matrix, approval artifacts, +actor threading, 4A/4B/4C staging). + - Add graph create/delete as cluster resources. - Make schema apply cluster-aware, with sidecar coverage for graph manifest movements before JSON state publish. diff --git a/docs/dev/index.md b/docs/dev/index.md index 49b6d76..827d99c 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -74,6 +74,7 @@ Working documents for in-flight feature work. Removed when the work lands. | Config & CLI architecture — layered config, client targeting, file naming (MR-973 / MR-974 / MR-981) | [rfc-002-config-cli-architecture.md](rfc-002-config-cli-architecture.md) | | MCP server surface — full tool parity, stored queries, modular auth (MR-969 / MR-956 / MR-974) | [rfc-003-mcp-server-surface.md](rfc-003-mcp-server-surface.md) | | Future cluster control plane — declarative as-code config, JSON state ledger, reconciler | [cluster-config-specs.md](cluster-config-specs.md), [cluster-axioms.md](cluster-axioms.md), [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md) | +| Cluster graph & schema apply — Phase 4 sidecars, roll-forward recovery, approval artifacts | [rfc-004-cluster-graph-schema-apply.md](rfc-004-cluster-graph-schema-apply.md) | ## Boundary diff --git a/docs/dev/rfc-004-cluster-graph-schema-apply.md b/docs/dev/rfc-004-cluster-graph-schema-apply.md new file mode 100644 index 0000000..ca72fdc --- /dev/null +++ b/docs/dev/rfc-004-cluster-graph-schema-apply.md @@ -0,0 +1,210 @@ +# RFC: Cluster Graph & Schema Apply — Phase 4 of the Cluster Control Plane + +**Status:** Proposed +**Date:** 2026-06-10 +**Builds on:** cluster Stages 1–3B (shipped: validate/plan/status/refresh/import/force-unlock, config-only `cluster apply` with content-addressed catalog publish, catalog payload verification, failpoint-proven crash/CAS recovery for the apply protocol). Normative context: [cluster-config-specs.md](cluster-config-specs.md), [cluster-axioms.md](cluster-axioms.md), [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md). +**Target release:** unversioned (phased — see Sequencing); no cluster functionality is in a tagged release yet. + +## Summary + +Extend `cluster apply` from config-only resources (stored queries, policy bundles) to **graph-moving resources**: graph create, cluster-driven schema apply, and graph delete. This is the nested-publish territory the implementation spec flags as its highest-risk decision: a graph's Lance manifest can move (via the engine's own atomic publish) *before* the cluster's JSON state CAS lands, and a crash in that window must never be silent, never acknowledged as success, and never repaired by guessing. + +Three design commitments make the phase tractable: + +1. **Cluster recovery is roll-forward-only.** The engine's recovery sidecars (`__recovery/{ulid}.json`, the open-time sweep in `db/manifest/recovery.rs`) already make every graph-level operation atomic *within the graph* — a schema apply either fully published or fully recovers at the next open. The cluster therefore never rolls a graph back. Cluster sidecars exist to **classify and record**: after a crash, the sweep observes the live graph, decides "moved / didn't move / moved unexpectedly," and either rolls the *cluster state* forward to match observable reality (axiom 5) or surfaces a loud pending-repair condition. The cluster holds no second transaction log and no rollback hammer — that would duplicate substrate behavior the engine already owns (invariant: respect the substrate). +2. **Irreversible operations require a digest-bound approval artifact.** Graph delete and `allow_data_loss` schema applies consume an explicit `__cluster/approvals/{ulid}.json` record bound to the exact change digests, written by a new `cluster approve` command and retired into the state ledger's `approval_records` (the durable audit reference of axiom 11). +3. **The operator identity becomes explicit.** `cluster apply` gains an actor, threaded to the engine's `apply_schema_as` so Cedar enforcement and commit attribution work unchanged. The cluster control plane adds no policy engine of its own (transport/auth stay at the boundary). + +## Motivation + +After Stage 3B, the control plane converges everything *except* the resources that define the data plane. The Sarah/Bob test is half-passed: Bob can see that Sarah changed a schema (`plan` shows the deferred change; `refresh` shows drift), but the system cannot act on it — Sarah still applies schemas with the per-graph tool and the cluster ledger trails reality. Graph creation is worse: a new graph in `cluster.yaml` blocks every dependent query and policy with `dependency_missing` until someone runs `omnigraph init` by hand at exactly the derived path. Phase 4 closes the loop: desired config in, converged deployment out, for the full resource vocabulary of Stage 1. + +The implementation spec's hard gate for this phase — failpoint recovery tests proving the movement-before-state-publish gap — was deliberately front-loaded: Stage 3B shipped the failpoint infrastructure and the apply-side crash/CAS tests. What remains is the design this RFC supplies: the sidecar schema, the recovery decision matrix, the approval artifact, and the ordering rules. + +## Non-Goals + +- **Server boot from cluster state** (Phase 5) — applied graphs/schemas still serve nothing; the server boots from `omnigraph.yaml` until the explicit per-deployment mode switch (axiom 15). +- **Policy-owned query exposure / `mcp.expose` retirement** (Phase 6). +- **Pipelines, embeddings, UI, aliases, bindings, providers, `env_file`** (Phase 7 and reserved fields). +- **External or Lance-backed state backends**; the local JSON backend + lock/CAS remains the substrate. +- **A cluster manifest publisher.** Deferred, per the spec: it becomes interesting only if the sidecar + repair path proves too weak for the accepted safety contract. Nothing in this design forecloses it. +- **Multi-graph atomic apply groups.** Cross-graph convergence remains statusful-partial per resource; one graph's failure never pretends to fence another's success. +- **Graph rename.** Stable-identity-across-rename is an open known gap at the schema level already; graph rename compounds it and is explicitly out of scope (see Open Questions). + +## Background + +What Phase 4 builds on (all shipped): + +- **The engine's recovery discipline.** Writers that can advance Lance HEAD before manifest publish write `__recovery/{ulid}.json` sidecars carrying per-table pins (`expected_version`, `post_commit_pin`); `Omnigraph::open` in read-write mode classifies every pinned table (`NoMovement` / `RolledPastExpected` / `UnexpectedAtP1` / `UnexpectedMultistep` / `InvariantViolation`) and decides all-or-nothing: roll forward via one manifest publish, or roll back via `Dataset::restore`, recording an audit row attributed to `omnigraph:recovery`. The cluster inherits the *vocabulary* of this design but not its mechanics — see the roll-forward-only argument below. +- **The engine's schema-apply surface.** `apply_schema_as(desired_source, SchemaApplyOptions { allow_data_loss }, actor)` returns `SchemaApplyResult { supported, applied, manifest_version, steps }`; `preview_schema_apply_with_options` returns the migration plan plus desired catalog without applying; the `__schema_apply_lock__` branch serializes schema applies graph-wide and refuses to run while user branches exist. Policy enforcement (`enforce(SchemaApply, TargetBranch("main"), actor)`) happens before the lock. +- **Graph init.** `Omnigraph::init(uri, schema_source)` with a strict preflight (errors if schema artifacts exist) and an atomic `_schema.pg` claim. A documented gap: a failed init does not clean up Lance datasets or `__manifest/` it already created. +- **No engine graph-delete primitive.** Deleting a graph today means removing its object-store prefix. This RFC works with that fact rather than waiting on a primitive. +- **Cluster state and observations.** `state.json` (locked, CAS-checked, atomically replaced) already records per-resource digests, statuses, `observations["graph.<id>"]` with `manifest_version` and live schema digest, plus empty `approval_records` / `recovery_records` placeholders reserved for this phase. +- **Stage 3A/3B apply mechanics.** Dispositions (`applied`/`derived`/`deferred`/`blocked`), content-addressed catalog publish before the state CAS, persisted-statuses contract on write failure, idempotent re-apply, payload verification with the drift + self-heal loop, and failpoints `cluster_apply.after_payload_phase` / `cluster_apply.before_state_write`. + +## Design + +### D1. Resource semantics: which dispositions change + +The Stage 3A classifier gains executable rows. Everything else (catalog resources, `derived` composites, blocked dependents) is unchanged: + +| Change | Stage 3A disposition | Phase 4 disposition | +|---|---|---| +| `graph.<id>` Create | Deferred | **Applied** (4A): `Omnigraph::init` at the derived root | +| `schema.<id>` Create | Deferred | **Applied with the graph create** (the init carries the schema) | +| `schema.<id>` Update | Deferred | **Applied** (4B): `apply_schema_as` against the live graph | +| `graph.<id>` / `schema.<id>` Delete | Deferred | **Applied behind approval** (4C): prefix removal | +| `query.*`/`policy.*` blocked on the above | Blocked | Unblocked in the same apply once the dependency lands (ordering, D5) | + +Graph roots remain **derived**: `ClusterRoot/graphs/<id>.omni` (high-risk decision #2 dispositioned: external graph roots are a separate, explicit future feature, not this phase). + +### D2. Cluster recovery sidecar (exit criterion 2, first half) + +Written under the state lock **before** any engine call that can move or create a graph manifest; deleted only **after** the cluster state CAS that records the outcome lands. + +```json +{ + "schema_version": 1, + "operation_id": "<ulid>", + "started_at": "<rfc3339>", + "actor": "<id or null>", + "kind": "graph_create | schema_apply | graph_delete", + "graph_id": "<id>", + "graph_uri": "<derived root>", + "observed_manifest_version": 7, + "expected_manifest_version": null, + "desired_schema_digest": "<sha256 of the schema source being applied>", + "state_cas_base": "sha256:<state.json digest at sidecar write>" +} +``` + +Path: `__cluster/recoveries/{operation_id}.json`, atomic write (temp + rename, the `write_state` discipline). Notes: + +- `observed_manifest_version` is the live graph's main-branch manifest version read at sidecar-write time (`null` for `graph_create` — no graph yet). This is the fencing value: apply refuses to proceed if it differs from the version recorded in `observations["graph.<id>"]` at plan time *and* re-observed under the lock (the same recompute-under-lock posture as Stage 3A's diff). +- `expected_manifest_version` starts `null` and is **rewritten into the sidecar immediately after the engine call returns** with `SchemaApplyResult.manifest_version` (or the post-init observation). A crash before that rewrite leaves `null`, which the sweep treats as "engine call outcome unknown — classify by observation only." For `graph_delete` the field is **always `null`** — prefix removal produces no new manifest version, so there is no rewrite step for that kind; delete sidecars are classified purely by root presence + state tombstone (D3 rows 7/7b/8). +- `state_cas_base` is **recorded for audit and diagnostics only — the sweep decision logic never consults it.** The sweep re-reads `state.json` under the lock and performs ordinary CAS-checked writes, so an independent state mutation between sidecar write and sweep is handled by the CAS like any other concurrent write, not by this field. Its value is forensic: a recovery audit entry can show which state revision the interrupted operation departed from. +- One sidecar per graph-moving resource operation. Apply processes graph-moving operations strictly sequentially (D5), so at most one sidecar is pending per apply run *per graph*, and the sweep processes sidecars in ULID order. + +### D3. Recovery decision matrix — roll-forward-only (exit criterion 2, second half) + +**Why no rollback.** The engine's sidecars already guarantee that a schema apply is atomic within the graph: by the time any cluster-visible manifest version moved, the engine either fully published or will recover all-or-nothing at its next read-write open. A cluster-level rollback would mean un-publishing a successfully published graph commit — rewriting substrate history the cluster does not own, duplicating the engine's transaction discipline (deny-list: custom transaction manager; state that drifts from what it can be derived from). The cluster's job after a crash is therefore *epistemic*, not transactional: observe what the graph actually is, and converge the ledger to it or refuse loudly. + +**Sweep trigger.** The sweep runs at the start of every state-mutating cluster command (`apply`, `refresh`, `import`), under the state lock, before the command's own work — mirroring the engine's open-time sweep gating (read-only `status`/`plan`/`validate` report pending sidecars as a warning, `cluster_recovery_pending`, but do not act). + +| # | Sidecar kind | Observation | Decision | +|---|---|---|---| +| 1 | any | Graph at `observed_manifest_version` (nothing moved) | Engine call never landed. Delete sidecar; the command's own plan/apply re-proposes the change. | +| 2 | `graph_create` / `schema_apply` | Graph at `expected_manifest_version`; state already records the outcome | Crash fell between state CAS and sidecar delete. Delete sidecar; done. | +| 3 | `schema_apply` | Graph at `expected_manifest_version` (or, when `expected` is `null`, live schema digest == `desired_schema_digest`); state stale | **Roll the cluster state forward**: record the live schema digest, recompute the graph composite, set statuses `applied`, append a `recovery_records` entry (audit), CAS-write, delete sidecar. | +| 4 | `graph_create` | Graph opens read-only and its schema digest == `desired_schema_digest`; state stale | Same roll-forward as #3 (the create completed). | +| 5 | `graph_create` | Root exists but the graph does not open (the engine's partial-init gap) | Status `error`, condition `graph_create_incomplete`, message: remove the root and re-run apply. **No auto-delete** — reconciler-initiated deletion is the same data-loss class as human deletion (high-risk decision #7). Sidecar kept until the operator acts and a sweep observes a clean state. | +| 6 | any | Graph at any other version (out-of-band movement during the crash window) | Status `drifted`, condition `actual_applied_state_pending`; sidecar kept; the command refuses graph-moving work for that graph until `cluster refresh` re-observes and the operator re-plans. No success is acknowledged for the interrupted operation. | +| 7 | `graph_delete` | Root absent; state already tombstoned | The delete kind's analog of row 2 (no manifest exists to version-check): crash fell between state CAS and sidecar delete. Delete sidecar; done. | +| 7b | `graph_delete` | Root absent; state stale | Roll forward: tombstone the graph subtree out of state (D6), record audit, delete sidecar. Idempotent — re-entry after a crash mid-row lands in row 7. | +| 8 | `graph_delete` | Root present (delete crashed mid-prefix-removal or never started) | If the approval artifact is still attached (D4), the delete is re-proposed by plan and re-runnable; status `drifted`, condition `graph_delete_incomplete`. Partial prefix removal leaves an unopenable graph — same operator message as #5. | + +Rows 3, 4 and 7b are the only mutations the sweep performs, and each is an ordinary CAS-checked state write under the lock — the sweep introduces no new write machinery. + +### D4. Approval artifacts (exit criteria 1-partial and 6-partial; axioms 8 and 11) + +The irreversible tier — graph delete, `allow_data_loss` schema apply (hard drops) — requires a recorded human decision that survives any reconstruction of state. `plan` already emits `approvals_required`; Phase 4 adds the consumption side. + +**Artifact** (`__cluster/approvals/{approval_id}.json`, written by the new command, never by apply): + +```json +{ + "schema_version": 1, + "approval_id": "<ulid>", + "resource": "graph.scratch", + "operation": "delete", + "reason": "<the approvals_required reason from plan>", + "bound_config_digest": "<desired config digest the plan was computed from>", + "bound_before_digest": "<state digest of the resource, or null>", + "bound_after_digest": "<desired digest, or null for delete>", + "approved_by": "<operator id, required>", + "created_at": "<rfc3339>" +} +``` + +**Flow.** `cluster approve <resource-address> --config <dir> --by <operator>` re-runs the plan under the lock, locates the pending gated change for that address, prints it, and writes the artifact bound to the exact digests. `cluster apply` executes a gated change only when a pending artifact matches **all** bound digests — a stale approval (config moved since) matches nothing, is reported (`approval_stale` warning), and the change stays `blocked` with condition `approval_required`. On successful execution the artifact file is moved into `state.approval_records[approval_id]` in the same state CAS that records the outcome (the state references the audit fact; losing state does not lose the approval, which is also why `import` preserves `approval_records` it finds — see D7). + +`allow_data_loss` is **never** a CLI flag on `cluster apply`; destructive promotion is expressed only through an approval artifact for the specific schema change. The default schema apply path runs with `allow_data_loss: false` (soft drops), which the spec's tier table classes as a recoverable definition rewrite — plan warning, no artifact. + +### D5. Actor, ordering, and apply groups (exit criterion 4) + +**Actor.** `cluster apply --actor <id>` / `cluster approve --by <id>`, with `OMNIGRAPH_CLUSTER_ACTOR` as the env fallback. The actor is threaded to `apply_schema_as` (so engine-side Cedar enforcement fires wherever a policy checker is installed and graph commits are attributed), recorded in sidecars, approvals, and `recovery_records`. The cluster adds no policy engine: graph-moving operations inherit the engine's gate; catalog-only operations remain ungated as today. When no actor is supplied and the target graph has no policy checker, behavior is unchanged from Stage 3A (`None` actor, as the engine's no-actor variants do); when a checker is installed the engine's existing "actor required" error surfaces as a typed diagnostic (`actor_required`). + +**Ordering.** Deterministic, dependency-shaped, within one apply run: + +1. graph creates (with their schemas) — ULID-stable order by graph id +2. schema applies — sequential, one graph at a time (each holds that graph's `__schema_apply_lock__`; the cluster state lock already serializes cluster-side) +3. catalog writes (queries/policies) — the Stage 3A path, unchanged +4. deletes last (catalog deletes, then approved graph deletes) + +Each graph-moving operation is its own apply group: sidecar → engine call → sidecar update → continue. The **state CAS stays single and final** (one write at the end recording every outcome), preserving Stage 3A's protocol; sidecars cover the widened gap between individual engine calls and that final CAS. A failure mid-sequence stops graph-moving work, reports per-resource statuses for everything already done (loud partials), and leaves sidecars for the sweep. Cross-graph atomicity is explicitly not promised. + +**Failpoints.** Each engine-call boundary gets a failpoint (`cluster_apply.before_graph_create`, `cluster_apply.after_graph_create`, `cluster_apply.before_schema_apply`, `cluster_apply.after_schema_apply`, `cluster_apply.before_graph_delete`) so every row of the D3 matrix is testable with the Stage 3B harness. + +### D6. Graph delete (4C) + +With no engine primitive, delete is cluster-orchestrated prefix removal: verify the approval artifact → sidecar (`kind: graph_delete`, current manifest version recorded) → recursively remove `ClusterRoot/graphs/<id>.omni` → state CAS that tombstones the graph subtree (graph, schema, and its queries removed from `applied_revision.resources` and `resource_statuses`; observation replaced by a tombstone record `{deleted_at, approval_id}`) → delete sidecar. Catalog blobs of the graph's queries stay (GC remains a later stage, consistent with Stage 3A deletes). The engine gap (no atomic prefix delete; partial removal leaves an unopenable root) is handled by D3 row 8, and this RFC registers a desire for an engine-level `destroy_graph` primitive as future work, not a dependency. + +### D7. Plan and import integration + +- **Plan** gains real data impact for schema updates: where Stage 3A showed only a digest diff, Phase 4 calls `preview_schema_apply_with_options` against the live graph (read-only) and embeds the migration steps + drop warnings in the change record — the "data-aware provider peek" from the high-level spec, bounded to graphs the plan already observes. Failure to preview (graph unreachable) degrades to the digest diff with a warning, never blocks planning. +- **Import/refresh** already observe live graphs; Phase 4 makes `import` preserve `approval_records` and pending `recoveries/` it finds (state reconstruction must not orphan audit facts or pending repairs). + +### D8. Invariants and axioms check + +- *Respect the substrate / no custom transaction manager*: cluster never rolls back graphs; engine sidecars own intra-graph atomicity (D3). +- *Axiom 5 (state = deployed reality)*: recovery converges the ledger to observation, never observation to ledger. +- *Axiom 8 (reversibility gates apply, including drift correction)*: approval artifacts for the irreversible tier; sweep never auto-deletes (D3 rows 5/8). +- *Axiom 9 (plan-time integrity)*: ordering is planner-derived from existing dependency edges; no runtime discovery. +- *Axiom 11 (approvals in a durable ledger)*: artifacts are files first, state-referenced after consumption; reconstructable state never re-derives who approved. +- *Axiom 12 (locked state)*: every new write path (sidecars, approvals consumption, sweep) runs under the existing state lock. +- *Axiom 15 (single owner / mode switch)*: nothing here reads from or writes to `omnigraph.yaml`; applied graphs still serve nothing until Phase 5. +- *Loud partials (deny-list)*: every crash window lands in a typed status/condition; no path acknowledges unverified success. + +## Migration / Compatibility + +Additive. Stage 3A/3B behavior is unchanged for catalog-only configs; existing state files gain no required fields (`approval_records`/`recovery_records` already exist, empty). New CLI surface: `cluster approve`, `--actor` on `cluster apply`. A deployment that never declares schema changes or graph creates sees identical behavior to Stage 3B. The honored-or-rejected posture continues: no new `cluster.yaml` fields are introduced by this phase (graph roots stay derived). + +## Sequencing + +| Stage | Scope | Gate | +|---|---|---| +| **4A graph create** | `Omnigraph::init` at derived roots; create-intent sidecar; D3 rows 1/2/4/5; dependents unblock in-run | Failpoint tests for crash-before/after-init; e2e: declare graph → apply → import-less convergence | +| **4B schema apply** | Full sidecar lifecycle; roll-forward sweep (D3 rows 3/6); actor threading; plan data-impact preview; soft-drop default | Failpoint tests per matrix row; e2e: schema evolution fully cluster-driven (replaces the Stage 3A defer→manual→refresh loop) | +| **4C graph delete** | `cluster approve` + artifact consumption; prefix removal; tombstones; D3 rows 7/7b/8 | Failpoint tests incl. partial-removal; e2e: gated delete refused without artifact, executed with it, stale artifact rejected | + +Each stage is a separate PR with boundary-matched tests (the Stage 1–3B discipline). 4A ships first because it moves no existing manifest; 4B is the heart; 4C last because it is the only irreversible-tier executor and consumes the approval machinery 4B's hard-drop path also needs. + +## Exit-criteria coverage (implementation spec) + +| # | Criterion | This RFC | +|---|---|---| +| 1 | State/status/approval/recovery schemas + paths | **Approval + recovery schemas: answered** (D2, D4). State/status: unchanged from shipped Stage 2A/3A. | +| 2 | Sidecar schema + recovery decision matrix | **Answered** (D2, D3) | +| 3 | State backend interface / lock+CAS | Unchanged (local JSON backend, shipped) — out of scope | +| 4 | Apply group syntax + dependency ordering | **Answered** (D5): per-resource groups, fixed kind-ordering; no user-declared group syntax this phase | +| 5 | Plan JSON schema incl. blast radius + approvals | **Extended** (D7 preview embedding); base schema shipped | +| 6 | Bootstrap authority + first-actor | **Partial** (D5 actor threading); cluster bootstrap authority remains open (below) | +| 7 | Server startup migration | Phase 5 — deferred | +| 8 | Per-query policy / `mcp.expose` bridge | Phase 6 — deferred | +| 9 | Pipeline runtime | Phase 7 — deferred | + +## Open Questions + +1. **Bootstrap authority.** The first apply against a fresh cluster has no policy engine to consult and no actor registry; today the answer is "whoever holds the object store wins." The durable story (out-of-band privileged bootstrap actor, per the high-level spec §open-questions) is unresolved and blocks nothing in this phase, since graph-level Cedar still gates wherever installed. +2. **Approval expiry.** Artifacts are digest-bound, so config drift invalidates them naturally; is wall-clock expiry also wanted (operator hygiene), or does digest binding suffice? +3. **Sweep on read-only commands.** This RFC has `status`/`plan` only *warn* about pending sidecars. If operator feedback shows the warn-but-don't-repair posture causes confusion, promoting `plan` to run the sweep (it already takes the lock) is a compatible change. +4. **Graph rename.** Deliberately out of scope; interacts with the rename-stable-identity known gap in [invariants.md](invariants.md). A rename today is delete + create — i.e., gated, lossy, and honest about it. +5. **Engine `destroy_graph` primitive.** 4C's prefix removal is correct but unatomic; if the engine grows a graph-destroy primitive with its own recovery, D6 collapses onto it (the cluster code is shaped to delegate). + +## References + +- [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md) — phases, exit criteria, high-risk decisions, approval tiers +- [cluster-axioms.md](cluster-axioms.md) — axioms 5, 8, 9, 11, 12, 15 +- [cluster-config-specs.md](cluster-config-specs.md) — the data-aware provider peek; state/ledger model +- `crates/omnigraph/src/db/manifest/recovery.rs` — the engine sidecar + classifier this design mirrors in vocabulary and deliberately does not duplicate in mechanics +- [writes.md](writes.md), [invariants.md](invariants.md) — engine recovery protocol and the deny-list this design is checked against From 26b26999fd6e809b1d6d7c03c6d9b152b31398cf Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 04:34:17 +0300 Subject: [PATCH 055/165] ci(codeowners): aaltshuler owns all paths; remove ragnorc (#169) Engineering and docs roles both resolve to @aaltshuler; every path (catch-all, crates/**, docs/**, repo-level docs) now requires their review. CODEOWNERS and the doc tables regenerated from codeowners-roles.yml via render-codeowners.py. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --- .github/CODEOWNERS | 14 +++++++------- .github/codeowners-roles.yml | 3 +-- docs/dev/codeowners.md | 18 +++++++++--------- 3 files changed, 17 insertions(+), 18 deletions(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index e937724..3650f9e 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -8,11 +8,11 @@ # CI fails if this file drifts from its source, and rejects PRs that # edit this file directly without also editing the yml. -* @ragnorc @aaltshuler +* @aaltshuler -crates/** @ragnorc @aaltshuler -docs/** @ragnorc -README.md @ragnorc -AGENTS.md @ragnorc -CLAUDE.md @ragnorc -SECURITY.md @ragnorc +crates/** @aaltshuler +docs/** @aaltshuler +README.md @aaltshuler +AGENTS.md @aaltshuler +CLAUDE.md @aaltshuler +SECURITY.md @aaltshuler diff --git a/.github/codeowners-roles.yml b/.github/codeowners-roles.yml index ce4014d..ed43c4a 100644 --- a/.github/codeowners-roles.yml +++ b/.github/codeowners-roles.yml @@ -21,7 +21,6 @@ roles: All production code under crates/**. Engine, CLI, server, compiler. members: - - ragnorc - aaltshuler docs: @@ -29,7 +28,7 @@ roles: Documentation under docs/**, plus repo-level docs (README.md, AGENTS.md, CLAUDE.md symlink, SECURITY.md). members: - - ragnorc + - aaltshuler # Path → role mapping. GitHub CODEOWNERS uses "last match wins" # semantics — when multiple patterns match a file, only the last diff --git a/docs/dev/codeowners.md b/docs/dev/codeowners.md index 50c4dc7..80d59e9 100644 --- a/docs/dev/codeowners.md +++ b/docs/dev/codeowners.md @@ -14,20 +14,20 @@ The tables below are **generated** from `.github/codeowners-roles.yml` by `.gith | Path | Owners | Role(s) | |---|---|---| -| `*` | @ragnorc @aaltshuler | engineering | -| `crates/**` | @ragnorc @aaltshuler | engineering | -| `docs/**` | @ragnorc | docs | -| `README.md` | @ragnorc | docs | -| `AGENTS.md` | @ragnorc | docs | -| `CLAUDE.md` | @ragnorc | docs | -| `SECURITY.md` | @ragnorc | docs | +| `*` | @aaltshuler | engineering | +| `crates/**` | @aaltshuler | engineering | +| `docs/**` | @aaltshuler | docs | +| `README.md` | @aaltshuler | docs | +| `AGENTS.md` | @aaltshuler | docs | +| `CLAUDE.md` | @aaltshuler | docs | +| `SECURITY.md` | @aaltshuler | docs | **Roles**: | Role | Members | Description | |---|---|---| -| `engineering` | @ragnorc @aaltshuler | All production code under crates/**. Engine, CLI, server, compiler. | -| `docs` | @ragnorc | Documentation under docs/**, plus repo-level docs (README.md, AGENTS.md, CLAUDE.md symlink, SECURITY.md). | +| `engineering` | @aaltshuler | All production code under crates/**. Engine, CLI, server, compiler. | +| `docs` | @aaltshuler | Documentation under docs/**, plus repo-level docs (README.md, AGENTS.md, CLAUDE.md symlink, SECURITY.md). | <!-- END GENERATED OWNERSHIP --> From 6fbf09d5c9f54018e88b400e48287bbc24c6d322 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 04:43:38 +0300 Subject: [PATCH 056/165] refactor(cluster): make apply_config_dir async Mechanical conversion ahead of Stage 4A graph create (which calls the async Omnigraph::init from inside apply): the fn signature, the CLI dispatch arm, and every test caller (#[test] -> #[tokio::test]). Zero behavior change; all 60 lib tests and 3 failpoint tests green before and after. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 2 +- crates/omnigraph-cluster/src/lib.rs | 102 +++++++++---------- crates/omnigraph-cluster/tests/failpoints.rs | 22 ++-- 3 files changed, 63 insertions(+), 63 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 37db77f..08c1fab 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -3558,7 +3558,7 @@ async fn main() -> Result<()> { finish_cluster_plan(&output, json)?; } ClusterCommand::Apply { config, json } => { - let output = apply_config_dir(config); + let output = apply_config_dir(config).await; finish_cluster_apply(&output, json)?; } ClusterCommand::Status { config, json } => { diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 660f34c..56513ca 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -561,7 +561,7 @@ pub fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { /// state is the publish point: a failure after payload writes leaves inert /// digest-named blobs and no success acknowledgement; re-running apply is the /// repair. -pub fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { +pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { let outcome = load_desired(config_dir.as_ref()); let mut diagnostics = outcome.diagnostics; let backend = LocalStateBackend::new(&outcome.config_dir); @@ -3932,10 +3932,10 @@ graphs: .join(format!("{digest}.yaml")) } - #[test] - fn apply_without_state_fails_with_state_missing() { + #[tokio::test] + async fn apply_without_state_fails_with_state_missing() { let dir = fixture(); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(!out.ok); assert!( out.diagnostics @@ -3948,8 +3948,8 @@ graphs: assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); } - #[test] - fn apply_writes_payloads_state_and_statuses() { + #[tokio::test] + async fn apply_writes_payloads_state_and_statuses() { let dir = fixture(); write_applyable_state(dir.path()); let desired = validate_config_dir(dir.path()); @@ -3965,7 +3965,7 @@ graphs: .unwrap() .clone(); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert_eq!(out.applied_count, 2); assert_eq!(out.deferred_count, 0); @@ -4011,8 +4011,8 @@ graphs: out.desired_revision.config_digest.clone().unwrap() } - #[test] - fn apply_update_changes_query_digest_and_keeps_old_blob() { + #[tokio::test] + async fn apply_update_changes_query_digest_and_keeps_old_blob() { let dir = fixture(); let desired = validate_config_dir(dir.path()); let schema_digest = desired @@ -4035,7 +4035,7 @@ graphs: fs::create_dir_all(old_blob.parent().unwrap()).unwrap(); fs::write(&old_blob, "old query source").unwrap(); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); let new_digest = desired .resource_digests @@ -4050,8 +4050,8 @@ graphs: assert!(query_payload_path(dir.path(), new_digest).exists()); } - #[test] - fn apply_deletes_removed_resources_but_keeps_blobs() { + #[tokio::test] + async fn apply_deletes_removed_resources_but_keeps_blobs() { let dir = fixture(); let desired = validate_config_dir(dir.path()); let schema_digest = desired @@ -4080,7 +4080,7 @@ graphs: fs::create_dir_all(stale_blob.parent().unwrap()).unwrap(); fs::write(&stale_blob, "old policy").unwrap(); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert!(out.converged); let state = read_state_json(dir.path()); @@ -4109,8 +4109,8 @@ graphs: assert_eq!(resources["graph.knowledge"]["digest"], expected_composite); } - #[test] - fn apply_defers_schema_change_and_blocks_dependent_query() { + #[tokio::test] + async fn apply_defers_schema_change_and_blocks_dependent_query() { let dir = fixture(); write_applyable_state(dir.path()); // Change the schema after seeding state: schema.knowledge now differs. @@ -4120,7 +4120,7 @@ graphs: ) .unwrap(); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert!(!out.converged); let by_resource: BTreeMap<&str, &PlanChange> = out @@ -4185,12 +4185,12 @@ graphs: ); } - #[test] - fn apply_blocks_resources_of_uncreated_graph() { + #[tokio::test] + async fn apply_blocks_resources_of_uncreated_graph() { let dir = fixture(); write_state_resources(dir.path(), &[]); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert_eq!(out.applied_count, 0); assert!(!out.converged); @@ -4227,8 +4227,8 @@ graphs: ); } - #[test] - fn apply_does_not_delete_subtree_of_deleted_graph() { + #[tokio::test] + async fn apply_does_not_delete_subtree_of_deleted_graph() { let dir = fixture(); let desired = validate_config_dir(dir.path()); let schema_digest = desired @@ -4249,7 +4249,7 @@ graphs: ], ); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert!(!out.converged); let by_resource: BTreeMap<&str, &PlanChange> = out @@ -4276,17 +4276,17 @@ graphs: assert_eq!(resources["query.old.q"]["digest"], "5555"); } - #[test] - fn apply_is_idempotent() { + #[tokio::test] + async fn apply_is_idempotent() { let dir = fixture(); write_applyable_state(dir.path()); - let first = apply_config_dir(dir.path()); + let first = apply_config_dir(dir.path()).await; assert!(first.ok, "{:?}", first.diagnostics); assert!(first.state_written); let state_after_first = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); - let second = apply_config_dir(dir.path()); + let second = apply_config_dir(dir.path()).await; assert!(second.ok, "{:?}", second.diagnostics); assert!(second.changes.is_empty()); assert_eq!(second.applied_count, 0); @@ -4297,13 +4297,13 @@ graphs: assert_eq!(second.state_observations.state_revision, 2); } - #[test] - fn apply_respects_held_lock() { + #[tokio::test] + async fn apply_respects_held_lock() { let dir = fixture(); write_applyable_state(dir.path()); write_lock_file(dir.path(), "held-lock", "plan"); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(!out.ok); assert!( out.diagnostics @@ -4317,8 +4317,8 @@ graphs: assert_eq!(state["state_revision"], 1); } - #[test] - fn apply_state_lock_false_bypasses_with_warning() { + #[tokio::test] + async fn apply_state_lock_false_bypasses_with_warning() { let dir = fixture(); fs::write( dir.path().join(CLUSTER_CONFIG_FILE), @@ -4338,7 +4338,7 @@ graphs: .unwrap(); write_applyable_state(dir.path()); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert!(out.state_written); assert!(!out.state_observations.lock_acquired); @@ -4350,8 +4350,8 @@ graphs: assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); } - #[test] - fn apply_skips_existing_payload_blob() { + #[tokio::test] + async fn apply_skips_existing_payload_blob() { let dir = fixture(); write_applyable_state(dir.path()); let desired = validate_config_dir(dir.path()); @@ -4366,13 +4366,13 @@ graphs: fs::create_dir_all(blob.parent().unwrap()).unwrap(); fs::write(&blob, "pre-existing").unwrap(); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert_eq!(fs::read_to_string(&blob).unwrap(), "pre-existing"); } - #[test] - fn apply_invalid_config_fails_before_lock() { + #[tokio::test] + async fn apply_invalid_config_fails_before_lock() { let dir = fixture(); fs::write( dir.path().join(CLUSTER_CONFIG_FILE), @@ -4380,7 +4380,7 @@ graphs: ) .unwrap(); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(!out.ok); // Config errors bail before the lock or any state directory exists. assert!(!dir.path().join(CLUSTER_STATE_DIR).exists()); @@ -4391,8 +4391,8 @@ graphs: /// mutations (phantom `applied` entries would mislead automation that /// reads `resource_statuses` independently of `ok`). #[cfg(unix)] - #[test] - fn apply_state_write_failure_reports_persisted_statuses() { + #[tokio::test] + async fn apply_state_write_failure_reports_persisted_statuses() { use std::os::unix::fs::PermissionsExt; let dir = fixture(); @@ -4435,7 +4435,7 @@ graphs: return; } - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; fs::set_permissions(&state_dir, fs::Permissions::from_mode(0o755)).unwrap(); assert!(!out.ok); @@ -4459,9 +4459,9 @@ graphs: // ---- catalog payload verification (Stage 3B) ---- /// Converge a fixture dir and return the query blob path. - fn converge_fixture(config_dir: &Path) -> std::path::PathBuf { + async fn converge_fixture(config_dir: &Path) -> std::path::PathBuf { write_applyable_state(config_dir); - let out = apply_config_dir(config_dir); + let out = apply_config_dir(config_dir).await; assert!(out.ok && out.converged, "{:?}", out.diagnostics); let desired = validate_config_dir(config_dir); query_payload_path( @@ -4473,10 +4473,10 @@ graphs: ) } - #[test] - fn status_reports_missing_payload_read_only() { + #[tokio::test] + async fn status_reports_missing_payload_read_only() { let dir = fixture(); - let blob = converge_fixture(dir.path()); + let blob = converge_fixture(dir.path()).await; let state_before = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); fs::remove_file(&blob).unwrap(); @@ -4501,7 +4501,7 @@ graphs: async fn refresh_removes_digest_and_drifts_on_missing_payload() { let dir = fixture(); init_derived_graph(dir.path()).await; - let blob = converge_fixture(dir.path()); + let blob = converge_fixture(dir.path()).await; fs::remove_file(&blob).unwrap(); let out = refresh_config_dir(dir.path()).await; @@ -4527,7 +4527,7 @@ graphs: async fn refresh_drifts_on_corrupted_payload() { let dir = fixture(); init_derived_graph(dir.path()).await; - let blob = converge_fixture(dir.path()); + let blob = converge_fixture(dir.path()).await; fs::write(&blob, "corrupted content").unwrap(); let out = refresh_config_dir(dir.path()).await; @@ -4547,7 +4547,7 @@ graphs: async fn refresh_flags_unreadable_payload_as_error() { let dir = fixture(); init_derived_graph(dir.path()).await; - let blob = converge_fixture(dir.path()); + let blob = converge_fixture(dir.path()).await; // A same-named directory yields a non-NotFound IO error portably. fs::remove_file(&blob).unwrap(); fs::create_dir(&blob).unwrap(); @@ -4575,7 +4575,7 @@ graphs: async fn payload_drift_self_heals_through_refresh_plan_apply() { let dir = fixture(); init_derived_graph(dir.path()).await; - let blob = converge_fixture(dir.path()); + let blob = converge_fixture(dir.path()).await; let original = fs::read_to_string(&blob).unwrap(); fs::remove_file(&blob).unwrap(); @@ -4591,7 +4591,7 @@ graphs: assert_eq!(query_change.operation, PlanOperation::Create); assert_eq!(query_change.disposition, Some(ApplyDisposition::Applied)); - let apply = apply_config_dir(dir.path()); + let apply = apply_config_dir(dir.path()).await; assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); assert_eq!(fs::read_to_string(&blob).unwrap(), original); diff --git a/crates/omnigraph-cluster/tests/failpoints.rs b/crates/omnigraph-cluster/tests/failpoints.rs index db7b82d..743f1fe 100644 --- a/crates/omnigraph-cluster/tests/failpoints.rs +++ b/crates/omnigraph-cluster/tests/failpoints.rs @@ -99,14 +99,14 @@ fn query_blob(config_dir: &Path, digests: &BTreeMap<String, String>) -> PathBuf .join(format!("{}.gq", digests["query.knowledge.find_person"])) } -#[test] -fn failpoint_wiring_returns_injected_diagnostic() { +#[tokio::test] +async fn failpoint_wiring_returns_injected_diagnostic() { let scenario = FailScenario::setup(); let dir = fixture(); seed_applyable_state(dir.path()); let _failpoint = ScopedFailPoint::new("cluster_apply.after_payload_phase", "return"); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(!out.ok); assert!(out.diagnostics.iter().any(|diagnostic| { diagnostic.code == "injected_failpoint" @@ -121,8 +121,8 @@ fn failpoint_wiring_returns_injected_diagnostic() { /// Crash between the payload phase and the state write: blobs are on disk, /// state.json is byte-identical, nothing is acknowledged — and a plain re-run /// repairs by trusting the existing content-addressed blobs. -#[test] -fn apply_crash_after_payload_phase_leaves_state_unmoved_then_recovers() { +#[tokio::test] +async fn apply_crash_after_payload_phase_leaves_state_unmoved_then_recovers() { let scenario = FailScenario::setup(); let dir = fixture(); let digests = seed_applyable_state(dir.path()); @@ -130,7 +130,7 @@ fn apply_crash_after_payload_phase_leaves_state_unmoved_then_recovers() { { let _failpoint = ScopedFailPoint::new("cluster_apply.after_payload_phase", "return"); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; assert!(!out.ok); assert!(!out.state_written); assert!(!out.converged); @@ -149,7 +149,7 @@ fn apply_crash_after_payload_phase_leaves_state_unmoved_then_recovers() { } // The repair is a plain re-run: existing blobs are trusted by digest. - let recovered = apply_config_dir(dir.path()); + let recovered = apply_config_dir(dir.path()).await; assert!(recovered.ok, "{:?}", recovered.diagnostics); assert!(recovered.converged); assert!(recovered.state_written); @@ -163,8 +163,8 @@ fn apply_crash_after_payload_phase_leaves_state_unmoved_then_recovers() { /// A concurrent writer mutating state.json between apply's read and its write /// (possible under `state.lock: false`) must surface `state_cas_mismatch`, /// acknowledge nothing, and leave the concurrent writer's state on disk. -#[test] -fn apply_cas_race_surfaces_state_cas_mismatch() { +#[tokio::test] +async fn apply_cas_race_surfaces_state_cas_mismatch() { let scenario = FailScenario::setup(); let dir = fixture(); let digests = seed_applyable_state(dir.path()); @@ -182,7 +182,7 @@ fn apply_cas_race_surfaces_state_cas_mismatch() { fs::write(&race_path, serde_json::to_string_pretty(&state).unwrap()).unwrap(); }); - let out = apply_config_dir(dir.path()); + let out = apply_config_dir(dir.path()).await; drop(failpoint); assert!(!out.ok); @@ -212,7 +212,7 @@ fn apply_cas_race_surfaces_state_cas_mismatch() { assert!(query_blob(dir.path(), &digests).exists()); // Recovery is a plain re-run against the rewritten state. - let recovered = apply_config_dir(dir.path()); + let recovered = apply_config_dir(dir.path()).await; assert!(recovered.ok, "{:?}", recovered.diagnostics); assert!(recovered.converged); scenario.teardown(); From bf8cc7a753a73073373bc28a1c3947140e7456dc Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 04:50:42 +0300 Subject: [PATCH 057/165] feat(cluster): graph-create recovery sidecars and sweep MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit RFC-004 §D2/§D3 for the graph_create kind. RecoverySidecar records intent under __cluster/recoveries/{ulid}.json; the roll-forward-only sweep runs at the start of apply/refresh/import under the state lock and classifies each survivor by observation: root absent -> intent removed (row 1); outcome already recorded -> retired (row 2); create completed but state stale -> ledger rolled forward with a recovery_records audit entry (row 4); partial root -> Error/graph_create_incomplete, kept, never auto-deleted (row 5); unexpected schema -> Drifted/actual_applied_state_pending, kept (row 6). Sweep mutations ride the command's existing CAS write; completed sidecars are deleted only after that write lands. Read-only status/plan warn (cluster_recovery_pending) without acting. The apply payload gate now counts only payload-phase errors so kept-sidecar diagnostics don't abort the run before their statuses persist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 554 +++++++++++++++++++++++++++- 1 file changed, 548 insertions(+), 6 deletions(-) diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 56513ca..fc13bab 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -24,6 +24,7 @@ pub const CLUSTER_STATE_DIR: &str = "__cluster"; pub const CLUSTER_STATE_FILE: &str = "__cluster/state.json"; pub const CLUSTER_LOCK_FILE: &str = "__cluster/lock.json"; pub const CLUSTER_RESOURCES_DIR: &str = "__cluster/resources"; +pub const CLUSTER_RECOVERIES_DIR: &str = "__cluster/recoveries"; #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] #[serde(rename_all = "snake_case")] @@ -415,11 +416,53 @@ struct StateLockFile { pid: u32, } +/// Recovery-intent record for a graph-moving apply operation (RFC-004 §D2). +/// Written under the state lock before the engine call that can create or +/// move a graph manifest; deleted only after the cluster state CAS that +/// records the outcome lands. The sweep (§D3) classifies survivors. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +struct RecoverySidecar { + schema_version: u32, + operation_id: String, + started_at: String, + #[serde(default)] + actor: Option<String>, + kind: RecoverySidecarKind, + graph_id: String, + graph_uri: String, + #[serde(default)] + observed_manifest_version: Option<u64>, + #[serde(default)] + expected_manifest_version: Option<u64>, + desired_schema_digest: String, + #[serde(default)] + state_cas_base: Option<String>, +} + +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +enum RecoverySidecarKind { + GraphCreate, + // SchemaApply and GraphDelete arrive with stages 4B/4C. +} + +#[derive(Debug, Default)] +struct SweepOutcome { + /// Graphs whose sidecar was kept (rows 5/6): graph-moving work for them + /// is blocked until the operator repairs and re-observes. + pending_graphs: BTreeSet<String>, + /// Sidecars whose outcome is recorded (rows 2/4): deleted only after the + /// command's state write lands, so a CAS failure re-sweeps them. + completed_sidecars: Vec<PathBuf>, +} + #[derive(Debug)] struct LocalStateBackend { state_dir: PathBuf, state_path: PathBuf, lock_path: PathBuf, + recoveries_dir: PathBuf, } #[derive(Debug)] @@ -513,6 +556,10 @@ pub fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { None }; + // Plan is read-only: pending sidecars are reported, never acted on + // (RFC-004 open question 3 keeps read-only commands warn-only). + warn_pending_recovery_sidecars(&desired.config_dir, &mut diagnostics); + let mut prior_resources = BTreeMap::new(); if !has_errors(&diagnostics) { match backend.read_state(&mut observations) { @@ -656,7 +703,7 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { } }; let expected_cas = snapshot.state_cas; - let Some(state) = snapshot.state else { + let Some(mut state) = snapshot.state else { diagnostics.push(Diagnostic::error( "state_missing", CLUSTER_STATE_FILE, @@ -672,9 +719,16 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { ); }; + // Snapshot the as-read state BEFORE the sweep so sweep mutations count as + // changes for the final dirty check and get persisted by the state CAS. + let before_value = + serde_json::to_value(&state).expect("cluster state must serialize deterministically"); + let sweep = sweep_recovery_sidecars(&backend, &mut state, &mut diagnostics).await; + let prior_resources = state_resource_digests(&state); let mut changes = diff_resources(&prior_resources, &desired.resource_digests); classify_changes(&mut changes, &desired.dependencies); + let _ = &sweep.pending_graphs; // consumed by the graph-create executor (4A C4) // Defensive invariant: nothing the approval gate covers may be executable. // Today approvals only cover graph/schema deletes (always deferred); this @@ -723,6 +777,9 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { // Payload phase: content-addressed writes before the state CAS. Any // failure aborts before state moves; blobs already written are inert. + // Gate on payload-phase errors only — sweep errors (e.g. a kept row-5 + // sidecar) must not abort the run, or their statuses would never persist. + let errors_before_payloads = count_errors(&diagnostics); let source_paths: BTreeMap<&str, &str> = desired .resources .iter() @@ -761,7 +818,7 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { diagnostics.push(diagnostic); } } - if has_errors(&diagnostics) { + if count_errors(&diagnostics) > errors_before_payloads { return early_return( display_path(&desired.config_dir), Some(desired.config_digest), @@ -788,9 +845,8 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { } // State mutation. Apply owns query/policy statuses only; graph/schema - // statuses belong to refresh/import observation and must not be clobbered. - let before_value = - serde_json::to_value(&state).expect("cluster state must serialize deterministically"); + // statuses belong to refresh/import observation and must not be clobbered + // (the sweep above is the one exception: it owns recovery statuses). let mut new_state = state.clone(); for change in &changes { match change.disposition { @@ -855,6 +911,13 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { } } } + // Completed (rows 2/4) sweep sidecars are deleted only once their outcome + // is durably recorded; on a failed write they stay and re-sweep next run. + if !state_write_failed { + for sidecar_path in &sweep.completed_sidecars { + let _ = fs::remove_file(sidecar_path); + } + } // On a failed state write, report the statuses that are actually on disk // (the pre-apply snapshot), not the in-memory mutations that were never // persisted — automation reading `resource_statuses` independently of `ok` @@ -902,6 +965,7 @@ pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { let backend = LocalStateBackend::new(&parsed.config_dir); let mut observations = backend.observations(); backend.observe_lock(&mut observations, &mut diagnostics); + warn_pending_recovery_sidecars(&parsed.config_dir, &mut diagnostics); let mut resource_digests = BTreeMap::new(); let mut resource_statuses = BTreeMap::new(); @@ -1107,6 +1171,11 @@ async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> St (StateSyncOperation::Import, None) => initial_import_state(&desired), }; + // Recovery sweep first (RFC-004 §D3): classify any interrupted graph + // operation before observation/verification so a rolled-forward outcome + // is what those passes see. + let sweep = sweep_recovery_sidecars(&backend, &mut state, &mut diagnostics).await; + // Catalog payload verification must run BEFORE graph observation: removing // a drifted query digest first means the live-graph composite recompute // below already excludes it, so the persisted graph.<id> composite stays @@ -1177,7 +1246,13 @@ async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> St } match backend.write_state(&state, expected_cas.as_deref(), &mut observations) { - Ok(()) => {} + Ok(()) => { + // Completed sweep sidecars are deleted only after their outcome + // is durably recorded; on failure they stay and re-sweep. + for sidecar_path in &sweep.completed_sidecars { + let _ = fs::remove_file(sidecar_path); + } + } Err(diagnostic) => diagnostics.push(diagnostic), } @@ -1307,10 +1382,104 @@ impl LocalStateBackend { Self { state_path: config_dir.join(CLUSTER_STATE_FILE), lock_path: config_dir.join(CLUSTER_LOCK_FILE), + recoveries_dir: config_dir.join(CLUSTER_RECOVERIES_DIR), state_dir, } } + /// List recovery sidecars in ULID (filename) order. Unparseable files are + /// reported as warnings and skipped — they stay on disk for the operator. + fn list_recovery_sidecars( + &self, + diagnostics: &mut Vec<Diagnostic>, + ) -> Vec<(PathBuf, RecoverySidecar)> { + let mut paths = Vec::new(); + match fs::read_dir(&self.recoveries_dir) { + Ok(entries) => { + for entry in entries.flatten() { + let path = entry.path(); + if path.extension().is_some_and(|ext| ext == "json") { + paths.push(path); + } + } + } + Err(err) if err.kind() == ErrorKind::NotFound => {} + Err(err) => { + diagnostics.push(Diagnostic::warning( + "recovery_sidecar_read_error", + CLUSTER_RECOVERIES_DIR, + format!("could not list recovery sidecars: {err}"), + )); + } + } + paths.sort(); + let mut sidecars = Vec::new(); + for path in paths { + match fs::read_to_string(&path) + .map_err(|err| err.to_string()) + .and_then(|text| { + serde_json::from_str::<RecoverySidecar>(&text).map_err(|err| err.to_string()) + }) { + Ok(sidecar) if sidecar.schema_version == 1 => sidecars.push((path, sidecar)), + Ok(sidecar) => diagnostics.push(Diagnostic::warning( + "unsupported_recovery_sidecar_version", + display_path(&path), + format!( + "unsupported recovery sidecar version {}; leaving it in place", + sidecar.schema_version + ), + )), + Err(err) => diagnostics.push(Diagnostic::warning( + "invalid_recovery_sidecar", + display_path(&path), + format!("could not parse recovery sidecar ({err}); leaving it in place"), + )), + } + } + sidecars + } + + /// Atomically write (or rewrite) a recovery sidecar; returns its path. + fn write_recovery_sidecar(&self, sidecar: &RecoverySidecar) -> Result<PathBuf, Diagnostic> { + fs::create_dir_all(&self.recoveries_dir).map_err(|err| { + Diagnostic::error( + "recovery_sidecar_write_error", + CLUSTER_RECOVERIES_DIR, + format!("could not create recoveries directory: {err}"), + ) + })?; + let target = self + .recoveries_dir + .join(format!("{}.json", sidecar.operation_id)); + let mut payload = serde_json::to_string_pretty(sidecar).map_err(|err| { + Diagnostic::error( + "recovery_sidecar_write_error", + display_path(&target), + format!("could not encode recovery sidecar: {err}"), + ) + })?; + payload.push('\n'); + let tmp_path = self + .recoveries_dir + .join(format!("{}.json.tmp.{}", sidecar.operation_id, Ulid::new())); + fs::write(&tmp_path, payload.as_bytes()).map_err(|err| { + Diagnostic::error( + "recovery_sidecar_write_error", + display_path(&tmp_path), + format!("could not write recovery sidecar: {err}"), + ) + })?; + if let Err(err) = fs::rename(&tmp_path, &target) { + let _ = fs::remove_file(&tmp_path); + return Err(Diagnostic::error( + "recovery_sidecar_write_error", + display_path(&target), + format!("could not move recovery sidecar into place: {err}"), + )); + } + Ok(target) + } + fn observations(&self) -> StateObservations { StateObservations { state_path: display_path(&self.state_path), @@ -1701,6 +1870,169 @@ fn initial_import_state(desired: &DesiredCluster) -> ClusterState { } } +/// Recovery sweep (RFC-004 §D3): runs at the start of every state-mutating +/// cluster command, under the state lock, before the command's own work. +/// Roll-forward-only — the engine's own sidecars make each graph-level +/// operation atomic within the graph, so the cluster never rolls a graph +/// back; it converges the ledger to observable reality or refuses loudly. +/// Mutations ride the calling command's CAS-checked state write; completed +/// sidecars are deleted only after that write lands. +async fn sweep_recovery_sidecars( + backend: &LocalStateBackend, + state: &mut ClusterState, + diagnostics: &mut Vec<Diagnostic>, +) -> SweepOutcome { + let mut outcome = SweepOutcome::default(); + for (path, sidecar) in backend.list_recovery_sidecars(diagnostics) { + match sidecar.kind { + RecoverySidecarKind::GraphCreate => { + sweep_graph_create_sidecar(path, sidecar, state, diagnostics, &mut outcome).await; + } + } + } + outcome +} + +async fn sweep_graph_create_sidecar( + path: PathBuf, + sidecar: RecoverySidecar, + state: &mut ClusterState, + diagnostics: &mut Vec<Diagnostic>, + outcome: &mut SweepOutcome, +) { + let graph_address = graph_address(&sidecar.graph_id); + let schema_addr = schema_address(&sidecar.graph_id); + let graph_path = PathBuf::from(&sidecar.graph_uri); + + // Row 1: nothing moved — the init never landed. The sidecar is pure + // intent; remove it and let the command's own plan re-propose the create. + if !graph_path.exists() { + let _ = fs::remove_file(&path); + return; + } + + match Omnigraph::open_read_only(&sidecar.graph_uri).await { + Ok(db) => { + let live_digest = sha256_hex(db.schema_source().as_bytes()); + let recorded = state + .applied_revision + .resources + .get(&schema_addr) + .map(|resource| resource.digest.clone()); + if recorded.as_deref() == Some(live_digest.as_str()) { + // Row 2: crash fell between the state CAS and sidecar delete. + outcome.completed_sidecars.push(path); + } else if live_digest == sidecar.desired_schema_digest { + // Row 4: the create completed on the graph; roll the cluster + // state forward to observable reality. + state.applied_revision.resources.insert( + schema_addr.clone(), + StateResource { + digest: live_digest.clone(), + }, + ); + let query_digests = state_query_digests_for_graph(state, &sidecar.graph_id); + let composite = + graph_digest(&sidecar.graph_id, Some(&live_digest), Some(&query_digests)); + state + .applied_revision + .resources + .insert(graph_address.clone(), StateResource { digest: composite }); + set_resource_status_applied(state, &graph_address); + set_resource_status_applied(state, &schema_addr); + state.recovery_records.insert( + sidecar.operation_id.clone(), + json!({ + "kind": "graph_create", + "graph_id": sidecar.graph_id, + "outcome": "rolled_forward", + "recovered_at": now_rfc3339(), + "actor": sidecar.actor, + }), + ); + diagnostics.push(Diagnostic::warning( + "cluster_recovery_rolled_forward", + graph_address.clone(), + "an interrupted graph create had completed on the graph; cluster state was rolled forward to match", + )); + outcome.completed_sidecars.push(path); + } else { + // Row 6: the graph moved to something the sidecar did not + // intend. Refuse to guess; require refresh + operator re-plan. + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Drifted, + "actual_applied_state_pending", + "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", + ); + set_resource_status( + state, + &schema_addr, + ResourceLifecycleStatus::Drifted, + "actual_applied_state_pending", + "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", + ); + diagnostics.push(Diagnostic::warning( + "cluster_recovery_pending", + graph_address.clone(), + "an interrupted graph create left unexpected graph state; graph-moving work is blocked until repaired", + )); + outcome.pending_graphs.insert(sidecar.graph_id.clone()); + } + } + Err(err) => { + // Row 5: partial root (the engine's documented init gap). Never + // auto-delete — reconciler deletes are the same data-loss class + // as human deletes; the operator removes the root explicitly. + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Error, + "graph_create_incomplete", + "graph root exists but cannot be opened; remove the graph root and re-run `cluster apply`", + ); + set_resource_status( + state, + &schema_addr, + ResourceLifecycleStatus::Error, + "graph_create_incomplete", + "graph root exists but cannot be opened; remove the graph root and re-run `cluster apply`", + ); + diagnostics.push(Diagnostic::error( + "graph_create_incomplete", + graph_address.clone(), + format!( + "graph root '{}' exists but cannot be opened ({err}); remove the graph root and re-run `cluster apply`", + sidecar.graph_uri + ), + )); + outcome.pending_graphs.insert(sidecar.graph_id.clone()); + } + } +} + +/// Read-only commands report pending sidecars without acting on them. +fn warn_pending_recovery_sidecars(config_dir: &Path, diagnostics: &mut Vec<Diagnostic>) { + let recoveries_dir = config_dir.join(CLUSTER_RECOVERIES_DIR); + let Ok(entries) = fs::read_dir(&recoveries_dir) else { + return; + }; + let mut names: Vec<String> = entries + .flatten() + .filter(|entry| entry.path().extension().is_some_and(|ext| ext == "json")) + .map(|entry| entry.file_name().to_string_lossy().into_owned()) + .collect(); + names.sort(); + for name in names { + diagnostics.push(Diagnostic::warning( + "cluster_recovery_pending", + format!("{CLUSTER_RECOVERIES_DIR}/{name}"), + "a recovery sidecar from an interrupted apply is pending; the next state-mutating command will classify it", + )); + } +} + async fn observe_declared_graphs(desired: &DesiredCluster, state: &mut ClusterState) -> usize { let mut graph_error_count = 0; for graph in &desired.graphs { @@ -2868,6 +3200,13 @@ fn has_errors(diagnostics: &[Diagnostic]) -> bool { .any(|diagnostic| diagnostic.severity == DiagnosticSeverity::Error) } +fn count_errors(diagnostics: &[Diagnostic]) -> usize { + diagnostics + .iter() + .filter(|diagnostic| diagnostic.severity == DiagnosticSeverity::Error) + .count() +} + fn display_path(path: &Path) -> String { path.display().to_string() } @@ -4621,6 +4960,209 @@ graphs: ); } + // ---- recovery sidecars + sweep (Stage 4A) ---- + + fn derived_graph_uri(config_dir: &Path, graph_id: &str) -> String { + display_path( + &config_dir + .join(CLUSTER_GRAPHS_DIR) + .join(format!("{graph_id}.omni")), + ) + } + + fn write_create_sidecar( + config_dir: &Path, + graph_id: &str, + desired_schema_digest: &str, + operation_id: &str, + ) -> PathBuf { + let dir = config_dir.join(CLUSTER_RECOVERIES_DIR); + fs::create_dir_all(&dir).unwrap(); + let path = dir.join(format!("{operation_id}.json")); + fs::write( + &path, + serde_json::to_string_pretty(&json!({ + "schema_version": 1, + "operation_id": operation_id, + "started_at": "1970-01-01T00:00:00Z", + "kind": "graph_create", + "graph_id": graph_id, + "graph_uri": derived_graph_uri(config_dir, graph_id), + "desired_schema_digest": desired_schema_digest, + })) + .unwrap(), + ) + .unwrap(); + path + } + + #[tokio::test] + async fn sweep_removes_sidecar_when_root_absent() { + let dir = fixture(); + write_applyable_state(dir.path()); + let sidecar = write_create_sidecar(dir.path(), "knowledge", "irrelevant", "01ROW1"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + // Row 1: nothing moved; intent removed, run proceeds normally. + assert!(!sidecar.exists()); + assert!(out.converged); + } + + #[tokio::test] + async fn sweep_rolls_forward_completed_create() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_state_resources(dir.path(), &[]); // state predates the create + let desired = validate_config_dir(dir.path()); + let schema_digest = desired.resource_digests["schema.knowledge"].clone(); + let sidecar = write_create_sidecar(dir.path(), "knowledge", &schema_digest, "01ROW4"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") + ); + // Row 4: ledger converged to observable reality, audit recorded, + // sidecar retired after the CAS landed. + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + schema_digest + ); + assert!( + state["recovery_records"] + .as_object() + .unwrap() + .values() + .any(|record| record["outcome"] == "rolled_forward" + && record["graph_id"] == "knowledge") + ); + assert!(!sidecar.exists()); + // With the graph rolled forward, the same run converges the catalog. + assert!(out.converged, "{out:?}"); + } + + #[tokio::test] + async fn sweep_completes_already_recorded_create() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); // state already records graph+schema + let desired = validate_config_dir(dir.path()); + let sidecar = write_create_sidecar( + dir.path(), + "knowledge", + &desired.resource_digests["schema.knowledge"], + "01ROW2", + ); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + // Row 2: outcome was already durable; no audit entry, sidecar retired. + assert!(!sidecar.exists()); + let state = read_state_json(dir.path()); + assert!( + state["recovery_records"] + .as_object() + .is_none_or(|records| records.is_empty()), + "{state}" + ); + } + + #[tokio::test] + async fn sweep_keeps_sidecar_for_incomplete_root() { + let dir = fixture(); + write_applyable_state(dir.path()); + // A root that exists but cannot be opened: the engine's partial-init gap. + let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); + fs::create_dir_all(&root).unwrap(); + fs::write(root.join("_schema.pg"), "junk").unwrap(); + let sidecar = write_create_sidecar(dir.path(), "knowledge", "whatever", "01ROW5"); + + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "graph_create_incomplete") + ); + // Row 5: never auto-delete; sidecar and root stay for the operator, + // and the Error status is persisted by the run's state write. + assert!(sidecar.exists()); + assert!(root.exists()); + let state = read_state_json(dir.path()); + assert_eq!(state["resource_statuses"]["graph.knowledge"]["status"], "error"); + assert!( + state["resource_statuses"]["graph.knowledge"]["conditions"] + .as_array() + .unwrap() + .iter() + .any(|condition| condition == "graph_create_incomplete") + ); + } + + #[tokio::test] + async fn sweep_flags_unexpected_schema_as_pending() { + let dir = fixture(); + write_state_resources(dir.path(), &[]); + // Live graph exists with a schema the sidecar never intended. + let graph_dir = dir.path().join(CLUSTER_GRAPHS_DIR); + fs::create_dir_all(&graph_dir).unwrap(); + Omnigraph::init( + &derived_graph_uri(dir.path(), "knowledge"), + "\nnode Other {\n name: String @key\n}\n", + ) + .await + .unwrap(); + let desired = validate_config_dir(dir.path()); + let sidecar = write_create_sidecar( + dir.path(), + "knowledge", + &desired.resource_digests["schema.knowledge"], + "01ROW6", + ); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); // warning, not error + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_pending") + ); + // Row 6: refuse to guess; sidecar kept, Drifted persisted. + assert!(sidecar.exists()); + let state = read_state_json(dir.path()); + assert_eq!( + state["resource_statuses"]["graph.knowledge"]["status"], + "drifted" + ); + assert!( + state["resource_statuses"]["graph.knowledge"]["conditions"] + .as_array() + .unwrap() + .iter() + .any(|condition| condition == "actual_applied_state_pending") + ); + } + + #[test] + fn status_warns_on_pending_recovery_sidecar() { + let dir = fixture(); + write_applyable_state(dir.path()); + write_create_sidecar(dir.path(), "knowledge", "irrelevant", "01STATUS"); + + let out = status_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_pending" + && diagnostic.severity == DiagnosticSeverity::Warning) + ); + } + #[test] fn plan_annotates_apply_dispositions() { let dir = fixture(); From c3007369cd50d1ae7b10a534d2d12e8e5813b0fa Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 04:58:56 +0300 Subject: [PATCH 058/165] feat(cluster): execute graph creates in cluster apply MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Stage 4A (RFC-004 §D1/§D5): graph.<id> Create — and its paired schema Create, which the init carries — classify Applied and execute first in the run, sequentially and sidecar-fenced: sidecar written before Omnigraph::init at the derived root, rewritten with the post-init manifest pin, deleted only after the final state CAS lands. Dependent queries and policies no longer block on a graph create in the same plan — creates run first, so they apply in the same run; a create failure demotes them to blocked (dependency_not_applied) and stops further graph-moving work (loud partials), with the sidecar left for the sweep to classify. Graphs with a kept recovery sidecar (rows 5/6) classify Blocked/cluster_recovery_pending, and the sweep's Drifted/Error statuses are never clobbered by a generic Blocked. Schema source is re-read and digest-verified under the lock before the init (the write_resource_payload TOCTOU posture). Plan previews the same dispositions. e2e fallout updated: a fresh multi-graph config now converges in one apply; a destroyed root is re-created as an EMPTY graph by the next apply (declarative convergence — visible in plan, called out in docs); the new cluster_e2e_declared_graph_created_by_apply pins the no-manual-init flow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/cli.rs | 152 ++++++--- crates/omnigraph-cluster/src/lib.rs | 469 ++++++++++++++++++++++++---- 2 files changed, 519 insertions(+), 102 deletions(-) diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index d47e13c..7ab7ca9 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -1285,7 +1285,7 @@ node Person { /// Disaster input fails closed: a destroyed graph root drifts the ledger, /// the plan proposes deferred creates, and apply moves nothing. #[test] -fn cluster_e2e_graph_root_destruction_drifts_and_apply_moves_nothing() { +fn cluster_e2e_graph_root_destruction_drifts_then_apply_recreates_empty_graph() { let temp = tempdir().unwrap(); write_cluster_config_fixture(temp.path()); init_cluster_derived_graph(temp.path()); @@ -1327,15 +1327,20 @@ fn cluster_e2e_graph_root_destruction_drifts_and_apply_moves_nothing() { let plan = cluster_json(temp.path(), "plan"); assert_eq!(change_for(&plan, "graph.knowledge")["operation"], "create"); - assert_eq!(change_for(&plan, "graph.knowledge")["disposition"], "deferred"); - assert_eq!(change_for(&plan, "schema.knowledge")["disposition"], "deferred"); + // Stage 4A: the re-create is executable and the plan says so — nothing + // hidden about converging a destroyed root back to an EMPTY graph (the + // data was already lost; this is declarative convergence, RFC-004 §D1). + assert_eq!(change_for(&plan, "graph.knowledge")["disposition"], "applied"); + assert_eq!(change_for(&plan, "schema.knowledge")["disposition"], "applied"); // Converged-then-destroyed: query/policy are already in state at the // desired digests, so they are not changes at all. assert_eq!(plan["changes"].as_array().unwrap().len(), 2, "{plan}"); - let disaster_apply = cluster_json(temp.path(), "apply"); - assert_eq!(disaster_apply["applied_count"], 0, "{disaster_apply}"); - assert_eq!(disaster_apply["converged"], false, "{disaster_apply}"); + let recreate = cluster_json(temp.path(), "apply"); + assert_eq!(recreate["ok"], true, "{recreate}"); + assert_eq!(recreate["converged"], true, "{recreate}"); + // The empty graph is back on disk; catalog state survived throughout. + assert!(temp.path().join("graphs/knowledge.omni").exists()); let state: serde_json::Value = serde_json::from_str( &fs::read_to_string(temp.path().join("__cluster/state.json")).unwrap(), ) @@ -1352,59 +1357,84 @@ fn cluster_e2e_graph_root_destruction_drifts_and_apply_moves_nothing() { ); } -/// The disposition matrix as a system: one apply over two graphs (one live, -/// one not yet created) plus graph-spanning and cluster-scoped policies must -/// produce all four dispositions at once — then converge after the second -/// graph appears. +/// The disposition matrix as a system under Stage 4A: a fresh multi-graph +/// config converges in ONE apply (both graphs created, spanning and +/// cluster-scoped policies applied), and a later mixed run — schema update +/// (deferred), its dependent query (blocked), an independent query update +/// (applied), its composite (derived) — shows all four dispositions at once +/// before the graph-plane schema apply closes the loop. #[test] fn cluster_e2e_multi_graph_mixed_dispositions_then_converge() { let temp = tempdir().unwrap(); write_multi_graph_cluster_fixture(temp.path()); - init_cluster_derived_graph(temp.path()); // knowledge only + // No manual init: Stage 4A creates both graphs. let import = cluster_json(temp.path(), "import"); assert_eq!(import["ok"], true, "{import}"); let apply = cluster_json(temp.path(), "apply"); assert_eq!(apply["ok"], true, "{apply}"); - assert_eq!(apply["converged"], false, "{apply}"); - assert_eq!(apply["applied_count"], 2, "{apply}"); + assert_eq!(apply["converged"], true, "{apply}"); + assert_eq!(change_for(&apply, "graph.knowledge")["disposition"], "applied"); assert_eq!( - change_for(&apply, "query.knowledge.find_person")["disposition"], - "applied" - ); - assert_eq!( - change_for(&apply, "policy.cluster_wide")["disposition"], + change_for(&apply, "graph.engineering")["disposition"], "applied" ); assert_eq!( change_for(&apply, "query.engineering.find_service")["disposition"], + "applied" + ); + // The graph-spanning and cluster-scoped policies ride the same run. + assert_eq!(change_for(&apply, "policy.shared")["disposition"], "applied"); + assert_eq!( + change_for(&apply, "policy.cluster_wide")["disposition"], + "applied" + ); + assert!(temp.path().join("graphs/knowledge.omni").exists()); + assert!(temp.path().join("graphs/engineering.omni").exists()); + + // Mixed run: a knowledge schema update (4B territory — deferred) gates + // its query update (blocked), while an engineering query update is + // independent (applied) and re-derives its composite. + fs::write( + temp.path().join("people.pg"), + "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n", + ) + .unwrap(); + fs::write( + temp.path().join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + temp.path().join("services.gq"), + "\nquery find_service($name: String) {\n match { $s: Service { name: $name } }\n return { $s.name, $s.name }\n}\n", + ) + .unwrap(); + + let mixed = cluster_json(temp.path(), "apply"); + assert_eq!(mixed["ok"], true, "{mixed}"); + assert_eq!(mixed["converged"], false, "{mixed}"); + assert_eq!(change_for(&mixed, "schema.knowledge")["disposition"], "deferred"); + assert_eq!(change_for(&mixed, "graph.knowledge")["disposition"], "deferred"); + assert_eq!( + change_for(&mixed, "query.knowledge.find_person")["disposition"], "blocked" ); assert_eq!( - change_for(&apply, "query.engineering.find_service")["reason"], - "dependency_missing" - ); - // One missing dependency graph blocks the whole spanning policy. - assert_eq!(change_for(&apply, "policy.shared")["disposition"], "blocked"); - assert_eq!( - change_for(&apply, "graph.engineering")["disposition"], - "deferred" + change_for(&mixed, "query.knowledge.find_person")["reason"], + "dependency_not_applied" ); assert_eq!( - change_for(&apply, "schema.engineering")["disposition"], - "deferred" + change_for(&mixed, "query.engineering.find_service")["disposition"], + "applied" ); assert_eq!( - change_for(&apply, "graph.knowledge")["disposition"], + change_for(&mixed, "graph.engineering")["disposition"], "derived" ); - assert_eq!( - apply["resource_statuses"]["policy.shared"]["status"], - "blocked" - ); // Deterministic ordering: changes sorted by resource address. - let order: Vec<&str> = apply["changes"] + let order: Vec<&str> = mixed["changes"] .as_array() .unwrap() .iter() @@ -1412,21 +1442,22 @@ fn cluster_e2e_multi_graph_mixed_dispositions_then_converge() { .collect(); let mut sorted = order.clone(); sorted.sort_unstable(); - assert_eq!(order, sorted, "{apply}"); + assert_eq!(order, sorted, "{mixed}"); - // The second graph appears; refresh observes it; apply converges. - init_named_cluster_graph(temp.path(), "engineering", "services.pg"); + // The graph-plane tool applies the schema; refresh observes; converge. + output_success( + cli() + .arg("schema") + .arg("apply") + .arg(temp.path().join("graphs/knowledge.omni")) + .arg("--schema") + .arg(temp.path().join("people.pg")) + .arg("--json"), + ); let refresh = cluster_json(temp.path(), "refresh"); assert_eq!(refresh["ok"], true, "{refresh}"); - let converge = cluster_json(temp.path(), "apply"); - assert_eq!(converge["ok"], true, "{converge}"); assert_eq!(converge["converged"], true, "{converge}"); - assert_eq!( - change_for(&converge, "query.engineering.find_service")["disposition"], - "applied" - ); - assert_eq!(change_for(&converge, "policy.shared")["disposition"], "applied"); let final_plan = cluster_json(temp.path(), "plan"); assert!( @@ -1435,6 +1466,39 @@ fn cluster_e2e_multi_graph_mixed_dispositions_then_converge() { ); } +/// Stage 4A headline: a declared graph is created by `cluster apply` itself — +/// no manual `omnigraph init` anywhere in the flow. +#[test] +fn cluster_e2e_declared_graph_created_by_apply() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["ok"], true, "{apply}"); + assert_eq!(apply["converged"], true, "{apply}"); + assert_eq!(change_for(&apply, "graph.knowledge")["disposition"], "applied"); + assert!(temp.path().join("graphs/knowledge.omni").exists()); + + // The created graph is a real graph: the per-graph CLI can open it. + let snapshot = output_success( + cli() + .arg("snapshot") + .arg(temp.path().join("graphs/knowledge.omni")), + ); + assert!(!stdout_string(&snapshot).is_empty()); + + let plan = cluster_json(temp.path(), "plan"); + assert!(plan["changes"].as_array().unwrap().is_empty(), "{plan}"); + let status = cluster_json(temp.path(), "status"); + assert_eq!( + status["resource_statuses"]["graph.knowledge"]["status"], + "applied" + ); +} + /// Catalog payload drift self-heals across the command surface: status warns /// read-only, refresh persists the drift and drops the digest, apply /// republishes the blob, status comes back clean. diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index fc13bab..863691c 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -577,7 +577,9 @@ pub fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { } else { diff_resources(&prior_resources, &desired.resource_digests) }; - classify_changes(&mut changes, &desired.dependencies); + // Plan previews dispositions without sweeping; a pending recovery is + // surfaced as the cluster_recovery_pending warning above instead. + classify_changes(&mut changes, &desired.dependencies, &BTreeSet::new()); let blast_radius = compute_blast_radius(&changes, &desired.dependencies); let approvals_required = compute_approvals(&changes); let ok = !has_errors(&diagnostics); @@ -727,8 +729,7 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { let prior_resources = state_resource_digests(&state); let mut changes = diff_resources(&prior_resources, &desired.resource_digests); - classify_changes(&mut changes, &desired.dependencies); - let _ = &sweep.pending_graphs; // consumed by the graph-create executor (4A C4) + classify_changes(&mut changes, &desired.dependencies, &sweep.pending_graphs); // Defensive invariant: nothing the approval gate covers may be executable. // Today approvals only cover graph/schema deletes (always deferred); this @@ -756,6 +757,169 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { ); } + // Graph creates execute first (RFC-004 §D5), sequentially, sidecar-fenced: + // sidecar written before the init, rewritten with the post-init manifest + // version, deleted only after the final state CAS lands. A failure stops + // further graph-moving work and demotes that graph's dependents. + let source_paths: BTreeMap<&str, &str> = desired + .resources + .iter() + .filter_map(|resource| { + resource + .path + .as_deref() + .map(|path| (resource.address.as_str(), path)) + }) + .collect(); + let graph_creates_to_run: Vec<String> = changes + .iter() + .filter(|change| { + change.disposition == Some(ApplyDisposition::Applied) + && change.operation == PlanOperation::Create + && matches!(resource_kind(&change.resource), ResourceKind::Graph(_)) + }) + .filter_map(|change| change.resource.strip_prefix("graph.").map(str::to_string)) + .collect(); + let mut completed_create_sidecars: Vec<PathBuf> = Vec::new(); + let mut failed_graphs: BTreeSet<String> = BTreeSet::new(); + let mut creates_aborted = false; + for graph_id in &graph_creates_to_run { + if creates_aborted { + // A prior create failed: stop graph-moving work (loud partials). + diagnostics.push(Diagnostic::warning( + "graph_create_skipped", + graph_address(graph_id), + "skipped after an earlier graph create failed in this run", + )); + failed_graphs.insert(graph_id.clone()); + continue; + } + let Some(desired_graph) = desired.graphs.iter().find(|graph| &graph.id == graph_id) + else { + continue; + }; + let graph_uri = display_path( + &desired + .config_dir + .join(CLUSTER_GRAPHS_DIR) + .join(format!("{graph_id}.omni")), + ); + let mut sidecar = RecoverySidecar { + schema_version: 1, + operation_id: Ulid::new().to_string(), + started_at: now_rfc3339(), + actor: None, + kind: RecoverySidecarKind::GraphCreate, + graph_id: graph_id.clone(), + graph_uri: graph_uri.clone(), + observed_manifest_version: None, + expected_manifest_version: None, + desired_schema_digest: desired_graph.schema_digest.clone(), + state_cas_base: expected_cas.clone(), + }; + let sidecar_path = match backend.write_recovery_sidecar(&sidecar) { + Ok(path) => path, + Err(diagnostic) => { + diagnostics.push(diagnostic); + failed_graphs.insert(graph_id.clone()); + creates_aborted = true; + continue; + } + }; + if let Err(diagnostic) = failpoints::maybe_fail("cluster_apply.before_graph_create") { + // Simulated crash before the init: the sidecar stays for the + // sweep (row 1: root absent -> intent removed next run). + diagnostics.push(diagnostic); + failed_graphs.insert(graph_id.clone()); + creates_aborted = true; + continue; + } + // Re-read + re-verify the schema source under the lock — the same + // TOCTOU posture as write_resource_payload. + let schema_source = source_paths + .get(schema_address(graph_id).as_str()) + .ok_or_else(|| { + Diagnostic::error( + "graph_create_failed", + graph_address(graph_id), + "no schema source recorded for graph", + ) + }) + .and_then(|path| { + fs::read_to_string(Path::new(path)).map_err(|err| { + Diagnostic::error( + "graph_create_failed", + graph_address(graph_id), + format!("could not read schema source '{path}': {err}"), + ) + }) + }) + .and_then(|source| { + if sha256_hex(source.as_bytes()) == desired_graph.schema_digest { + Ok(source) + } else { + Err(Diagnostic::error( + "resource_content_changed", + schema_address(graph_id), + "schema source changed while apply was running; re-run `cluster apply`", + )) + } + }); + let schema_source = match schema_source { + Ok(source) => source, + Err(diagnostic) => { + diagnostics.push(diagnostic); + let _ = fs::remove_file(&sidecar_path); // nothing moved + failed_graphs.insert(graph_id.clone()); + creates_aborted = true; + continue; + } + }; + match Omnigraph::init(&graph_uri, &schema_source).await { + Ok(_) => {} + Err(err) => { + diagnostics.push(Diagnostic::error( + "graph_create_failed", + graph_address(graph_id), + format!("could not initialize graph at '{graph_uri}': {err}"), + )); + // The sidecar stays: the sweep classifies whether the failed + // init left a partial root (row 5) or nothing (row 1). + failed_graphs.insert(graph_id.clone()); + creates_aborted = true; + continue; + } + } + // Record the post-init pin in the sidecar (best effort — a failure + // here leaves expected = null and the sweep classifies by digest). + if let Ok(db) = Omnigraph::open_read_only(&graph_uri).await { + if let Ok(snapshot) = db.snapshot_of(ReadTarget::branch("main")).await { + sidecar.expected_manifest_version = Some(snapshot.version()); + if let Err(diagnostic) = backend.write_recovery_sidecar(&sidecar) { + diagnostics.push(diagnostic); + } + } + } + // Crash point: the graph exists, the cluster state does not record it + // yet. A failure here must acknowledge nothing; the next run's sweep + // rolls the ledger forward (row 4). + if let Err(diagnostic) = failpoints::maybe_fail("cluster_apply.after_graph_create") { + diagnostics.push(diagnostic); + return early_return( + display_path(&desired.config_dir), + Some(desired.config_digest), + observations, + changes, + state.resource_statuses, + diagnostics, + ); + } + completed_create_sidecars.push(sidecar_path); + } + if !failed_graphs.is_empty() { + demote_dependents_of_failed_graphs(&mut changes, &failed_graphs, &desired.dependencies); + } + for change in &changes { match change.disposition { Some(ApplyDisposition::Deferred) => diagnostics.push(Diagnostic::warning( @@ -780,16 +944,6 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { // Gate on payload-phase errors only — sweep errors (e.g. a kept row-5 // sidecar) must not abort the run, or their statuses would never persist. let errors_before_payloads = count_errors(&diagnostics); - let source_paths: BTreeMap<&str, &str> = desired - .resources - .iter() - .filter_map(|resource| { - resource - .path - .as_deref() - .map(|path| (resource.address.as_str(), path)) - }) - .collect(); for change in &changes { if change.disposition != Some(ApplyDisposition::Applied) || change.operation == PlanOperation::Delete @@ -869,13 +1023,17 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { } }, Some(ApplyDisposition::Blocked) => { - set_resource_status( - &mut new_state, - &change.resource, - ResourceLifecycleStatus::Blocked, - change.reason.as_deref().unwrap_or("dependency_not_applied"), - "waiting on an unapplied or missing dependency", - ); + // The sweep owns recovery statuses (Drifted/Error with their + // conditions); a generic Blocked must not clobber them. + if change.reason.as_deref() != Some("cluster_recovery_pending") { + set_resource_status( + &mut new_state, + &change.resource, + ResourceLifecycleStatus::Blocked, + change.reason.as_deref().unwrap_or("dependency_not_applied"), + "waiting on an unapplied or missing dependency", + ); + } } _ => {} } @@ -914,7 +1072,11 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { // Completed (rows 2/4) sweep sidecars are deleted only once their outcome // is durably recorded; on a failed write they stay and re-sweep next run. if !state_write_failed { - for sidecar_path in &sweep.completed_sidecars { + for sidecar_path in sweep + .completed_sidecars + .iter() + .chain(completed_create_sidecars.iter()) + { let _ = fs::remove_file(sidecar_path); } } @@ -2669,15 +2831,27 @@ fn resource_kind(address: &str) -> ResourceKind { /// it. Stage 3A executes only query/policy catalog writes; graph/schema /// movement is a later phase, and `graph.<id>` composite updates whose schema /// component is unchanged converge automatically once query digests land. -fn classify_changes(changes: &mut [PlanChange], dependencies: &[Dependency]) { - let mut schema_changed = BTreeSet::new(); +fn classify_changes( + changes: &mut [PlanChange], + dependencies: &[Dependency], + pending_recovery: &BTreeSet<String>, +) { + let mut schema_creates = BTreeSet::new(); + let mut schema_pending = BTreeSet::new(); let mut graph_creates = BTreeSet::new(); let mut graph_deletes = BTreeSet::new(); for change in changes.iter() { match resource_kind(&change.resource) { - ResourceKind::Schema(graph) => { - schema_changed.insert(graph); - } + ResourceKind::Schema(graph) => match change.operation { + PlanOperation::Create => { + schema_creates.insert(graph); + } + // Schema updates (4B) and deletes (4C) are still pending in + // this stage and block dependents. + _ => { + schema_pending.insert(graph); + } + }, ResourceKind::Graph(graph) => match change.operation { PlanOperation::Create => { graph_creates.insert(graph); @@ -2690,12 +2864,38 @@ fn classify_changes(changes: &mut [PlanChange], dependencies: &[Dependency]) { _ => {} } } + // A schema Create is satisfied by its paired graph create (the init + // carries the schema); a standalone schema Create stays pending. + for graph in &schema_creates { + if !graph_creates.contains(graph) { + schema_pending.insert(graph.clone()); + } + } for change in changes.iter_mut() { let (disposition, reason) = match resource_kind(&change.resource) { - ResourceKind::Schema(_) => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), + ResourceKind::Schema(graph) => match change.operation { + PlanOperation::Create + if graph_creates.contains(&graph) + && !pending_recovery.contains(&graph) => + { + // Applied with the graph create — the init carries it. + (ApplyDisposition::Applied, None) + } + PlanOperation::Create if graph_creates.contains(&graph) => { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) + } + _ => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), + }, ResourceKind::Graph(graph) => match change.operation { - PlanOperation::Update if !schema_changed.contains(&graph) => { + PlanOperation::Create => { + if pending_recovery.contains(&graph) { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) + } else { + (ApplyDisposition::Applied, None) + } + } + PlanOperation::Update if !schema_pending.contains(&graph) => { (ApplyDisposition::Derived, None) } _ => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), @@ -2712,16 +2912,16 @@ fn classify_changes(changes: &mut [PlanChange], dependencies: &[Dependency]) { } } PlanOperation::Create | PlanOperation::Update => { - // A missing graph is the more fundamental blocker than a - // pending schema change, so check it first. - if graph_creates.contains(&graph) { - (ApplyDisposition::Blocked, Some("dependency_missing")) - } else if schema_changed.contains(&graph) { + if pending_recovery.contains(&graph) { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) + } else if schema_pending.contains(&graph) { ( ApplyDisposition::Blocked, Some("dependency_not_applied"), ) } else { + // A graph create in the same plan no longer blocks: + // creates execute first in the same apply run. (ApplyDisposition::Applied, None) } } @@ -2729,15 +2929,15 @@ fn classify_changes(changes: &mut [PlanChange], dependencies: &[Dependency]) { ResourceKind::Policy(_) => match change.operation { PlanOperation::Delete => (ApplyDisposition::Applied, None), PlanOperation::Create | PlanOperation::Update => { - let blocked_dep = dependencies.iter().any(|dep| { + let blocked_pending = dependencies.iter().any(|dep| { dep.from == change.resource && dep .to .strip_prefix("graph.") - .is_some_and(|graph| graph_creates.contains(graph)) + .is_some_and(|graph| pending_recovery.contains(graph)) }); - if blocked_dep { - (ApplyDisposition::Blocked, Some("dependency_missing")) + if blocked_pending { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) } else { (ApplyDisposition::Applied, None) } @@ -2752,6 +2952,46 @@ fn classify_changes(changes: &mut [PlanChange], dependencies: &[Dependency]) { } } +/// After a graph create fails mid-run, every change that depended on that +/// graph (its schema, its queries, policies referencing it) flips from +/// Applied to Blocked so the output and the persisted statuses tell the +/// truth about what this run actually executed. +fn demote_dependents_of_failed_graphs( + changes: &mut [PlanChange], + failed: &BTreeSet<String>, + dependencies: &[Dependency], +) { + for change in changes.iter_mut() { + if change.disposition != Some(ApplyDisposition::Applied) { + continue; + } + let demote_reason = match resource_kind(&change.resource) { + ResourceKind::Graph(graph) if failed.contains(&graph) => Some("graph_create_failed"), + ResourceKind::Schema(graph) if failed.contains(&graph) => { + Some("dependency_not_applied") + } + ResourceKind::Query { graph, .. } if failed.contains(&graph) => { + Some("dependency_not_applied") + } + ResourceKind::Policy(_) => { + let blocked = dependencies.iter().any(|dep| { + dep.from == change.resource + && dep + .to + .strip_prefix("graph.") + .is_some_and(|graph| failed.contains(graph)) + }); + blocked.then_some("dependency_not_applied") + } + _ => None, + }; + if let Some(reason) = demote_reason { + change.disposition = Some(ApplyDisposition::Blocked); + change.reason = Some(reason.to_string()); + } + } +} + /// Content-addressed catalog path for an applied resource payload. Extensions /// are fixed per kind (`.gq` / `.yaml`) regardless of the source file's name, /// so the catalog layout cannot drift with operator file conventions. @@ -4525,45 +4765,117 @@ graphs: } #[tokio::test] - async fn apply_blocks_resources_of_uncreated_graph() { + async fn apply_creates_graph_and_unblocks_dependents() { let dir = fixture(); write_state_resources(dir.path(), &[]); let out = apply_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); - assert_eq!(out.applied_count, 0); - assert!(!out.converged); + assert!(out.converged, "{out:?}"); let by_resource: BTreeMap<&str, &PlanChange> = out .changes .iter() .map(|change| (change.resource.as_str(), change)) .collect(); + // Stage 4A: the create executes, and its dependents apply in-run. assert_eq!( by_resource["graph.knowledge"].disposition, - Some(ApplyDisposition::Deferred) + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["schema.knowledge"].disposition, + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["query.knowledge.find_person"].disposition, + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["policy.base"].disposition, + Some(ApplyDisposition::Applied) + ); + // The graph exists on disk and opens; state records everything. + let graph_uri = derived_graph_uri(dir.path(), "knowledge"); + let db = Omnigraph::open_read_only(&graph_uri).await.unwrap(); + let desired = validate_config_dir(dir.path()); + assert_eq!( + sha256_hex(db.schema_source().as_bytes()), + desired.resource_digests["schema.knowledge"] + ); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + desired.resource_digests["schema.knowledge"] + ); + assert_eq!( + state["resource_statuses"]["graph.knowledge"]["status"], + "applied" + ); + // The create's sidecar was retired after the state CAS landed. + assert!( + !dir.path().join(CLUSTER_RECOVERIES_DIR).exists() + || fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) + .unwrap() + .next() + .is_none() + ); + } + + #[tokio::test] + async fn apply_create_failure_blocks_dependents_and_keeps_sidecar() { + let dir = fixture(); + write_state_resources(dir.path(), &[]); + // Make the init fail its strict preflight: a junk _schema.pg already + // sits at the derived root (the engine refuses to overwrite it). + let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); + fs::create_dir_all(&root).unwrap(); + fs::write(root.join("_schema.pg"), "junk").unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "graph_create_failed") + ); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + // Dependents are demoted: the run tells the truth about what executed. + assert_eq!( + by_resource["graph.knowledge"].disposition, + Some(ApplyDisposition::Blocked) + ); + assert_eq!( + by_resource["query.knowledge.find_person"].disposition, + Some(ApplyDisposition::Blocked) ); assert_eq!( by_resource["query.knowledge.find_person"].reason.as_deref(), - Some("dependency_missing") + Some("dependency_not_applied") ); assert_eq!( - by_resource["policy.base"].reason.as_deref(), - Some("dependency_missing") + by_resource["policy.base"].disposition, + Some(ApplyDisposition::Blocked) ); - // Statuses for blocked resources are recorded (state changed), but no - // resource digests moved. + assert!(!out.converged); + // The sidecar stays for the sweep to classify next run. + assert!( + fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) + .unwrap() + .next() + .is_some() + ); + // No graph digests moved. let state = read_state_json(dir.path()); - assert_eq!(state["state_revision"], 2); assert!( state["applied_revision"]["resources"] .as_object() .unwrap() .is_empty() ); - assert_eq!( - state["resource_statuses"]["policy.base"]["status"], - "blocked" - ); } #[tokio::test] @@ -5147,6 +5459,47 @@ graphs: ); } + #[tokio::test] + async fn apply_blocks_create_while_recovery_pending() { + let dir = fixture(); + write_state_resources(dir.path(), &[]); + // A kept (row 5) sidecar: partial root that cannot be opened. + let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); + fs::create_dir_all(&root).unwrap(); + fs::write(root.join("_schema.pg"), "junk").unwrap(); + let sidecar = write_create_sidecar(dir.path(), "knowledge", "whatever", "01PEND"); + + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); // row 5 is an error condition + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + // The pending recovery blocks the create and its dependents; the + // executor never attempts the init. + assert_eq!( + by_resource["graph.knowledge"].disposition, + Some(ApplyDisposition::Blocked) + ); + assert_eq!( + by_resource["graph.knowledge"].reason.as_deref(), + Some("cluster_recovery_pending") + ); + assert_eq!( + by_resource["query.knowledge.find_person"].reason.as_deref(), + Some("cluster_recovery_pending") + ); + assert_eq!( + by_resource["policy.base"].reason.as_deref(), + Some("cluster_recovery_pending") + ); + assert!(sidecar.exists()); + // The sweep's Error status is what persists — not a generic Blocked. + let state = read_state_json(dir.path()); + assert_eq!(state["resource_statuses"]["graph.knowledge"]["status"], "error"); + } + #[test] fn status_warns_on_pending_recovery_sidecar() { let dir = fixture(); @@ -5173,23 +5526,23 @@ graphs: .iter() .map(|change| (change.resource.as_str(), change)) .collect(); - // Empty state: graph/schema creates are deferred, query/policy blocked - // on the uncreated graph — and plan says so before apply runs. + // Stage 4A: graph/schema creates are executable, and dependents ride + // the same run — plan previews exactly that. assert_eq!( by_resource["graph.knowledge"].disposition, - Some(ApplyDisposition::Deferred) + Some(ApplyDisposition::Applied) ); assert_eq!( by_resource["schema.knowledge"].disposition, - Some(ApplyDisposition::Deferred) + Some(ApplyDisposition::Applied) ); assert_eq!( by_resource["query.knowledge.find_person"].disposition, - Some(ApplyDisposition::Blocked) + Some(ApplyDisposition::Applied) ); assert_eq!( - by_resource["policy.base"].reason.as_deref(), - Some("dependency_missing") + by_resource["policy.base"].disposition, + Some(ApplyDisposition::Applied) ); } } From 83d77bcb16d10b6eb2d28f8c6e202ab3606311fd Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 04:59:48 +0300 Subject: [PATCH 059/165] test(cluster): failpoint coverage for graph-create crash windows - Crash before the init (row 1): sidecar survives, nothing moved, no ack; the next run's sweep removes the intent and the same run creates and converges. - Crash after the init, before the state CAS (row 4): the graph exists with the post-init manifest pin in the sidecar, state.json byte-identical; the next run's sweep rolls the ledger forward with a recovery_records audit entry and the run converges. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/tests/failpoints.rs | 128 +++++++++++++++++++ 1 file changed, 128 insertions(+) diff --git a/crates/omnigraph-cluster/tests/failpoints.rs b/crates/omnigraph-cluster/tests/failpoints.rs index 743f1fe..ec8ddfb 100644 --- a/crates/omnigraph-cluster/tests/failpoints.rs +++ b/crates/omnigraph-cluster/tests/failpoints.rs @@ -217,3 +217,131 @@ async fn apply_cas_race_surfaces_state_cas_mismatch() { assert!(recovered.converged); scenario.teardown(); } + +fn seed_empty_state(config_dir: &Path) { + let state_dir = config_dir.join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{ + "version": 1, + "state_revision": 1, + "applied_revision": { "resources": {} } +} +"#, + ) + .unwrap(); +} + +fn recovery_sidecars(config_dir: &Path) -> Vec<PathBuf> { + match fs::read_dir(config_dir.join("__cluster/recoveries")) { + Ok(entries) => { + let mut paths: Vec<PathBuf> = entries + .flatten() + .map(|entry| entry.path()) + .filter(|path| path.extension().is_some_and(|ext| ext == "json")) + .collect(); + paths.sort(); + paths + } + Err(_) => Vec::new(), + } +} + +/// Crash before the init: the create-intent sidecar survives, nothing moved. +/// The next run's sweep removes the intent (row 1) and the same run creates +/// the graph and converges. +#[tokio::test] +async fn create_crash_before_init_recovers_via_sweep() { + let scenario = FailScenario::setup(); + let dir = fixture(); + seed_empty_state(dir.path()); + + { + let _failpoint = ScopedFailPoint::new("cluster_apply.before_graph_create", "return"); + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!(out.diagnostics.iter().any(|diagnostic| { + diagnostic.code == "injected_failpoint" + && diagnostic + .message + .contains("cluster_apply.before_graph_create") + })); + assert_eq!(recovery_sidecars(dir.path()).len(), 1); + assert!(!dir.path().join("graphs/knowledge.omni").exists()); + // No resource digest moved. + let state: serde_json::Value = serde_json::from_str( + &fs::read_to_string(dir.path().join("__cluster/state.json")).unwrap(), + ) + .unwrap(); + assert!( + state["applied_revision"]["resources"] + .as_object() + .unwrap() + .is_empty() + ); + } + + let recovered = apply_config_dir(dir.path()).await; + assert!(recovered.ok, "{:?}", recovered.diagnostics); + assert!(recovered.converged); + assert!(dir.path().join("graphs/knowledge.omni").exists()); + assert!(recovery_sidecars(dir.path()).is_empty()); + scenario.teardown(); +} + +/// Crash after the init but before the state CAS: the graph exists, the +/// ledger is stale, nothing was acknowledged. The next run's sweep rolls the +/// ledger forward (row 4) with an audit entry, and the run converges. +#[tokio::test] +async fn create_crash_after_init_rolls_state_forward() { + let scenario = FailScenario::setup(); + let dir = fixture(); + seed_empty_state(dir.path()); + let state_before = fs::read(dir.path().join("__cluster/state.json")).unwrap(); + + { + let _failpoint = ScopedFailPoint::new("cluster_apply.after_graph_create", "return"); + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!(!out.state_written); + // The graph exists; the cluster state is byte-identical (no ack). + assert!(dir.path().join("graphs/knowledge.omni").exists()); + assert_eq!( + fs::read(dir.path().join("__cluster/state.json")).unwrap(), + state_before + ); + // The sidecar carries the post-init manifest pin. + let sidecars = recovery_sidecars(dir.path()); + assert_eq!(sidecars.len(), 1); + let sidecar: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&sidecars[0]).unwrap()).unwrap(); + assert!( + sidecar["expected_manifest_version"].is_number(), + "{sidecar}" + ); + } + + let recovered = apply_config_dir(dir.path()).await; + assert!(recovered.ok, "{:?}", recovered.diagnostics); + assert!( + recovered + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") + ); + assert!(recovered.converged); + assert!(recovery_sidecars(dir.path()).is_empty()); + let state: serde_json::Value = serde_json::from_str( + &fs::read_to_string(dir.path().join("__cluster/state.json")).unwrap(), + ) + .unwrap(); + assert!( + state["recovery_records"] + .as_object() + .unwrap() + .values() + .any(|record| record["outcome"] == "rolled_forward") + ); + scenario.teardown(); +} From cb6c67f1966b5692ca652d651534f35b2958194a Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 05:00:42 +0300 Subject: [PATCH 060/165] docs(cluster): document Stage 4A graph create Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/testing.md | 2 +- docs/user/cluster-config.md | 50 ++++++++++++++++++++++++++++++------- 2 files changed, 42 insertions(+), 10 deletions(-) diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 5c88a37..c171f53 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,7 +8,7 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish; includes the `cluster_e2e_*` lifecycle compositions over the spawned binary — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | -| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), and failpoint crash-mid-apply / CAS-race coverage | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, and Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows) | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 9a2597b..7ff49e8 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -1,15 +1,17 @@ # Cluster Config -**Status:** Stage 3A config-only apply preview. +**Status:** Stage 4A graph-create apply preview. Cluster config is the future control-plane configuration surface for a whole OmniGraph deployment. In this stage, OmniGraph can validate a local `cluster.yaml` folder, produce a deterministic read-only plan, inspect the local JSON state ledger, explicitly refresh/import graph observations into that ledger, manually remove a held local state lock by exact lock id, and -**apply the config-only subset of the plan** — stored-query and policy-bundle -catalog writes. It does not move graph manifests, change schemas, start -servers, or serve anything it applies: the server still boots from +**apply the executable subset of the plan** — stored-query and policy-bundle +catalog writes, and **graph creation**: a declared graph that does not exist +yet is initialized by apply itself at the derived root. It does not change +existing schemas (deferred to a later stage), move existing graph manifests, +start servers, or serve anything it applies: the server still boots from `omnigraph.yaml`. ## Commands @@ -153,8 +155,8 @@ condition in `reason`). ## Apply -`cluster apply` executes the config-only subset of the plan — stored-query and -policy-bundle changes. There is no confirm flag: `cluster plan` is the preview, +`cluster apply` executes the executable subset of the plan — stored-query and +policy-bundle changes, and graph creates. There is no confirm flag: `cluster plan` is the preview, and apply recomputes the same diff under the state lock before executing, so a stale preview can never be applied. Apply requires an existing `state.json` (`state_missing` directs you to `cluster import` first). @@ -180,9 +182,39 @@ still boots from `omnigraph.yaml`; no query or policy applied here serves traffic until the server-boot stage ships, as an explicit per-deployment mode switch. -Graph and schema changes are never executed by this stage. They are reported -as `deferred` (warning `apply_unsupported_change`), and query/policy changes -that depend on them are `blocked` (warning `apply_dependency_blocked`, status +### Graph creation + +A `graph.<id>` create (the graph is declared but no root exists) is executed +by apply: the graph is initialized at the derived root + +```text +<config-dir>/graphs/<graph-id>.omni +``` + +with the declared schema, before any catalog writes, so queries and policies +that depend on the new graph apply **in the same run**. Each create is fenced +by a recovery sidecar under `__cluster/recoveries/{ulid}.json`, written before +the init and removed only after the state update lands. If apply crashes in +between, the next state-mutating command (`apply`, `refresh`, `import`) runs a +**recovery sweep** that classifies the survivor by observation: an absent root +removes the stale intent; a completed create rolls the cluster state forward +(recorded in the state's `recovery_records`); a partial root reports +`graph_create_incomplete` (status `error` — remove the root and re-run apply; +nothing is auto-deleted); unexpected graph content reports +`actual_applied_state_pending` (status `drifted` — run `cluster refresh` and +re-plan). While a kept sidecar is pending, that graph's create and its +dependents are blocked with `cluster_recovery_pending`. Read-only commands +(`status`, `plan`) warn about pending sidecars without acting on them. + +**Re-creation is convergence.** If a graph root disappears out-of-band, +`refresh` records the drift and the next `plan` proposes a create — and apply +will execute it, producing an **empty** graph at the root. The data was +already lost when the root vanished; the create is visible in the plan +(disposition `applied`) before anything runs. + +Schema changes to existing graphs are never executed by this stage. They are +reported as `deferred` (warning `apply_unsupported_change`), and query/policy +changes that depend on them are `blocked` (warning `apply_dependency_blocked`, status `blocked` in state). A partially-applicable plan still exits 0 with warnings; the JSON `converged` field is the automation signal for "state now matches the desired revision". The applied `config_digest` is only recorded when apply From b313075476a5c49d747b425b6bc6bd245258a93e Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 13:02:12 +0300 Subject: [PATCH 061/165] refactor(cluster): make plan_config_dir async Mechanical conversion ahead of Stage 4B (plan will preview schema migrations against live graphs): signature, CLI dispatch, and test callers. Zero behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 2 +- crates/omnigraph-cluster/src/lib.rs | 74 ++++++++++++++--------------- 2 files changed, 38 insertions(+), 38 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 08c1fab..7d09f36 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -3554,7 +3554,7 @@ async fn main() -> Result<()> { finish_cluster_validate(&output, json)?; } ClusterCommand::Plan { config, json } => { - let output = plan_config_dir(config); + let output = plan_config_dir(config).await; finish_cluster_plan(&output, json)?; } ClusterCommand::Apply { config, json } => { diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 863691c..7f92a67 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -499,7 +499,7 @@ pub fn validate_config_dir(config_dir: impl AsRef<Path>) -> ValidateOutput { } } -pub fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { +pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { let outcome = load_desired(config_dir.as_ref()); let mut diagnostics = outcome.diagnostics; let backend = LocalStateBackend::new(&outcome.config_dir); @@ -3681,10 +3681,10 @@ graphs: ); } - #[test] - fn missing_state_plans_creates() { + #[tokio::test] + async fn missing_state_plans_creates() { let dir = fixture(); - let out = plan_config_dir(dir.path()); + let out = plan_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert!(!out.state_observations.state_found); assert!(!out.state_observations.locked); @@ -3698,10 +3698,10 @@ graphs: assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); } - #[test] - fn config_digest_ignores_yaml_comments_and_formatting() { + #[tokio::test] + async fn config_digest_ignores_yaml_comments_and_formatting() { let dir = fixture(); - let first = plan_config_dir(dir.path()); + let first = plan_config_dir(dir.path()).await; assert!(first.ok, "{:?}", first.diagnostics); fs::write( @@ -3724,7 +3724,7 @@ policies: ) .unwrap(); - let second = plan_config_dir(dir.path()); + let second = plan_config_dir(dir.path()).await; assert!(second.ok, "{:?}", second.diagnostics); assert_eq!( first.desired_revision.config_digest, @@ -3732,10 +3732,10 @@ policies: ); } - #[test] - fn existing_state_plans_update_and_delete_deterministically() { + #[tokio::test] + async fn existing_state_plans_update_and_delete_deterministically() { let dir = fixture(); - let first = plan_config_dir(dir.path()); + let first = plan_config_dir(dir.path()).await; let state_dir = dir.path().join("__cluster"); fs::create_dir_all(&state_dir).unwrap(); fs::write( @@ -3755,7 +3755,7 @@ policies: ) .unwrap(); - let out = plan_config_dir(dir.path()); + let out = plan_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); let rendered: Vec<_> = out .changes @@ -3773,8 +3773,8 @@ policies: ); } - #[test] - fn old_minimal_state_json_still_plans_with_default_revision() { + #[tokio::test] + async fn old_minimal_state_json_still_plans_with_default_revision() { let dir = fixture(); let state_dir = dir.path().join(CLUSTER_STATE_DIR); fs::create_dir_all(&state_dir).unwrap(); @@ -3792,7 +3792,7 @@ policies: ) .unwrap(); - let out = plan_config_dir(dir.path()); + let out = plan_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert_eq!(out.state_observations.state_revision, 0); assert!(out.state_observations.state_cas.is_some()); @@ -4018,12 +4018,12 @@ graphs: assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); } - #[test] - fn plan_succeeds_after_force_unlock() { + #[tokio::test] + async fn plan_succeeds_after_force_unlock() { let dir = fixture(); write_lock_file(dir.path(), "held-lock", "plan"); - let locked = plan_config_dir(dir.path()); + let locked = plan_config_dir(dir.path()).await; assert!(!locked.ok); assert!( locked @@ -4035,12 +4035,12 @@ graphs: let unlocked = force_unlock_config_dir(dir.path(), "held-lock"); assert!(unlocked.ok, "{:?}", unlocked.diagnostics); - let out = plan_config_dir(dir.path()); + let out = plan_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); } - #[test] - fn plan_reports_state_cas_revision_and_removes_lock() { + #[tokio::test] + async fn plan_reports_state_cas_revision_and_removes_lock() { let dir = fixture(); let state_dir = dir.path().join(CLUSTER_STATE_DIR); fs::create_dir_all(&state_dir).unwrap(); @@ -4056,7 +4056,7 @@ graphs: }"#; fs::write(state_dir.join("state.json"), state).unwrap(); - let out = plan_config_dir(dir.path()); + let out = plan_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert_eq!(out.state_observations.state_revision, 7); assert_eq!( @@ -4073,8 +4073,8 @@ graphs: ); } - #[test] - fn existing_lock_makes_plan_fail() { + #[tokio::test] + async fn existing_lock_makes_plan_fail() { let dir = fixture(); let state_dir = dir.path().join(CLUSTER_STATE_DIR); fs::create_dir_all(&state_dir).unwrap(); @@ -4090,7 +4090,7 @@ graphs: ) .unwrap(); - let out = plan_config_dir(dir.path()); + let out = plan_config_dir(dir.path()).await; assert!(!out.ok); assert!(out.state_observations.locked); assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); @@ -4111,8 +4111,8 @@ graphs: })); } - #[test] - fn state_lock_false_bypasses_lock_with_warning() { + #[tokio::test] + async fn state_lock_false_bypasses_lock_with_warning() { let dir = fixture(); fs::write( dir.path().join(CLUSTER_CONFIG_FILE), @@ -4128,7 +4128,7 @@ graphs: ) .unwrap(); - let out = plan_config_dir(dir.path()); + let out = plan_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert!(!out.state_observations.locked); assert!(!out.state_observations.lock_acquired); @@ -4153,15 +4153,15 @@ graphs: assert_eq!(out.diagnostics[0].code, "unsupported_state_backend"); } - #[test] - fn external_state_backend_plan_rejected() { + #[tokio::test] + async fn external_state_backend_plan_rejected() { let dir = fixture(); fs::write( dir.path().join(CLUSTER_CONFIG_FILE), "version: 1\nstate:\n backend: s3://bucket/state\ngraphs: {}\n", ) .unwrap(); - let out = plan_config_dir(dir.path()); + let out = plan_config_dir(dir.path()).await; assert!(!out.ok); assert!( out.diagnostics @@ -4304,7 +4304,7 @@ graphs: assert!(!out.resource_digests.contains_key("graph.knowledge")); assert_eq!(out.observations["graph.knowledge"]["exists"], false); - let plan = plan_config_dir(dir.path()); + let plan = plan_config_dir(dir.path()).await; assert!(plan.ok, "{:?}", plan.diagnostics); assert!(plan.changes.iter().any(|change| { change.resource == "graph.knowledge" && change.operation == PlanOperation::Create @@ -4342,7 +4342,7 @@ graphs: false ); - let plan = plan_config_dir(dir.path()); + let plan = plan_config_dir(dir.path()).await; assert!(plan.ok, "{:?}", plan.diagnostics); assert!(plan.changes.iter().any(|change| { change.resource == "schema.knowledge" && change.operation == PlanOperation::Update @@ -5233,7 +5233,7 @@ graphs: let refresh = refresh_config_dir(dir.path()).await; assert!(refresh.ok, "{:?}", refresh.diagnostics); - let plan = plan_config_dir(dir.path()); + let plan = plan_config_dir(dir.path()).await; let query_change = plan .changes .iter() @@ -5516,10 +5516,10 @@ graphs: ); } - #[test] - fn plan_annotates_apply_dispositions() { + #[tokio::test] + async fn plan_annotates_apply_dispositions() { let dir = fixture(); - let out = plan_config_dir(dir.path()); + let out = plan_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); let by_resource: BTreeMap<&str, &PlanChange> = out .changes From ca63a9340b64bcbe90a8b60e3d90af5794cc616f Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 13:04:19 +0300 Subject: [PATCH 062/165] feat(cluster): embed schema migration previews in cluster plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit RFC-004 §D7's data-aware preview: for every schema update, plan opens the live graph read-only and embeds the engine's migration plan (supported flag + typed steps) in the change record; the human renderer prints the steps. Preview failures (unreachable graph, planner error) degrade to the digest diff with a schema_preview_unavailable warning — planning never blocks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 11 +++ crates/omnigraph-cluster/src/lib.rs | 117 +++++++++++++++++++++++++++- 2 files changed, 126 insertions(+), 2 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 7d09f36..de87309 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -804,6 +804,17 @@ fn print_cluster_plan_human(output: &PlanOutput) { ); for change in &output.changes { println!(" {:?} {}", change.operation, change.resource); + if let Some(migration) = &change.migration { + if !migration.supported { + println!(" migration UNSUPPORTED:"); + } + for step in &migration.steps { + println!( + " {}", + serde_json::to_string(step).unwrap_or_else(|_| format!("{step:?}")) + ); + } + } } if output.changes.is_empty() { println!(" no changes"); diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 7f92a67..af4ac93 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -4,7 +4,8 @@ use std::io::{ErrorKind, Write}; use std::path::{Path, PathBuf}; use std::process; -use omnigraph::db::{Omnigraph, ReadTarget}; +use omnigraph::db::{Omnigraph, ReadTarget, SchemaApplyOptions}; +use omnigraph_compiler::SchemaMigrationPlan; use omnigraph_compiler::build_catalog; use omnigraph_compiler::query::parser::parse_query; use omnigraph_compiler::query::typecheck::typecheck_query_decl; @@ -182,7 +183,7 @@ pub enum ApplyDisposition { Blocked, } -#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[derive(Debug, Clone, Serialize, PartialEq)] pub struct PlanChange { pub resource: String, pub operation: PlanOperation, @@ -194,6 +195,11 @@ pub struct PlanChange { pub disposition: Option<ApplyDisposition>, #[serde(skip_serializing_if = "Option::is_none")] pub reason: Option<String>, + /// For schema updates: the engine's migration plan against the live + /// graph (RFC-004 §D7's data-aware preview). Absent when the preview is + /// unavailable (warning `schema_preview_unavailable`). + #[serde(skip_serializing_if = "Option::is_none")] + pub migration: Option<SchemaMigrationPlan>, } #[derive(Debug, Clone, Serialize, PartialEq, Eq)] @@ -580,6 +586,40 @@ pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { // Plan previews dispositions without sweeping; a pending recovery is // surfaced as the cluster_recovery_pending warning above instead. classify_changes(&mut changes, &desired.dependencies, &BTreeSet::new()); + + // Embed real migration steps for schema updates so plan is a data-aware + // preview; failures degrade to the digest diff with a warning. + for change in &mut changes { + if change.operation != PlanOperation::Update { + continue; + } + let ResourceKind::Schema(graph_id) = resource_kind(&change.resource) else { + continue; + }; + let graph_uri = display_path( + &desired + .config_dir + .join(CLUSTER_GRAPHS_DIR) + .join(format!("{graph_id}.omni")), + ); + let source_path = desired + .resources + .iter() + .find(|resource| resource.address == change.resource) + .and_then(|resource| resource.path.clone()); + let preview = match source_path { + Some(path) => preview_schema_migration(&graph_uri, &path).await, + None => Err("no schema source recorded".to_string()), + }; + match preview { + Ok(migration) => change.migration = Some(migration), + Err(err) => diagnostics.push(Diagnostic::warning( + "schema_preview_unavailable", + change.resource.clone(), + format!("could not preview the schema migration: {err}"), + )), + } + } let blast_radius = compute_blast_radius(&changes, &desired.dependencies); let approvals_required = compute_approvals(&changes); let ok = !has_errors(&diagnostics); @@ -2332,6 +2372,23 @@ async fn observe_declared_graphs(desired: &DesiredCluster, state: &mut ClusterSt graph_error_count } +/// RFC-004 §D7: the data-aware preview — the engine's migration plan for a +/// desired schema against the live graph, computed read-only (no lock). +async fn preview_schema_migration( + graph_uri: &str, + schema_path: &str, +) -> Result<SchemaMigrationPlan, String> { + let source = fs::read_to_string(schema_path).map_err(|err| err.to_string())?; + let db = Omnigraph::open_read_only(graph_uri) + .await + .map_err(|err| err.to_string())?; + let preview = db + .preview_schema_apply_with_options(&source, SchemaApplyOptions::default()) + .await + .map_err(|err| err.to_string())?; + Ok(preview.plan) +} + struct LiveGraphObservation { manifest_version: u64, schema_digest: String, @@ -2736,6 +2793,7 @@ fn diff_resources( after_digest: Some(after.clone()), disposition: None, reason: None, + migration: None, }), Some(before) if before != after => changes.push(PlanChange { resource: address.clone(), @@ -2744,6 +2802,7 @@ fn diff_resources( after_digest: Some(after.clone()), disposition: None, reason: None, + migration: None, }), Some(_) => {} } @@ -2757,6 +2816,7 @@ fn diff_resources( after_digest: None, disposition: None, reason: None, + migration: None, }); } } @@ -5500,6 +5560,59 @@ graphs: assert_eq!(state["resource_statuses"]["graph.knowledge"]["status"], "error"); } + #[tokio::test] + async fn plan_embeds_migration_preview_for_schema_update() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + fs::write( + dir.path().join("people.pg"), + "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n", + ) + .unwrap(); + + let out = plan_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + let schema_change = out + .changes + .iter() + .find(|change| change.resource == "schema.knowledge") + .unwrap(); + let migration = schema_change.migration.as_ref().expect("preview embedded"); + assert!(migration.supported); + assert!( + serde_json::to_string(&migration.steps) + .unwrap() + .contains("add_property"), + "{migration:?}" + ); + } + + #[tokio::test] + async fn plan_warns_when_preview_unavailable() { + let dir = fixture(); + write_applyable_state(dir.path()); // digests recorded, but no live root + fs::write( + dir.path().join("people.pg"), + "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n", + ) + .unwrap(); + + let out = plan_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + let schema_change = out + .changes + .iter() + .find(|change| change.resource == "schema.knowledge") + .unwrap(); + assert!(schema_change.migration.is_none()); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "schema_preview_unavailable") + ); + } + #[test] fn status_warns_on_pending_recovery_sidecar() { let dir = fixture(); From 0571c05ebb9a928fe96b6733656751a3e7102d2f Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 13:05:42 +0300 Subject: [PATCH 063/165] feat(cluster): schema-apply recovery sidecar kind and sweep RecoverySidecarKind::SchemaApply with digest-based sweep classification (robust to unrelated manifest movement; version pins stay forensic): ledger-consistent -> sidecar retired (RFC-004 rows 1+2); live digest matches the intended schema, state stale -> roll forward with composite recompute and a recovery_records audit entry (row 3); unverifiable or unexpected digests -> pending, kept, graph-moving work blocked (rows 1-unopenable/6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 231 +++++++++++++++++++++++++++- 1 file changed, 230 insertions(+), 1 deletion(-) diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index af4ac93..f8a56e7 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -450,7 +450,8 @@ struct RecoverySidecar { #[serde(rename_all = "snake_case")] enum RecoverySidecarKind { GraphCreate, - // SchemaApply and GraphDelete arrive with stages 4B/4C. + SchemaApply, + // GraphDelete arrives with stage 4C. } #[derive(Debug, Default)] @@ -2090,6 +2091,9 @@ async fn sweep_recovery_sidecars( RecoverySidecarKind::GraphCreate => { sweep_graph_create_sidecar(path, sidecar, state, diagnostics, &mut outcome).await; } + RecoverySidecarKind::SchemaApply => { + sweep_schema_apply_sidecar(path, sidecar, state, diagnostics, &mut outcome).await; + } } } outcome @@ -2214,6 +2218,102 @@ async fn sweep_graph_create_sidecar( } } +async fn sweep_schema_apply_sidecar( + path: PathBuf, + sidecar: RecoverySidecar, + state: &mut ClusterState, + diagnostics: &mut Vec<Diagnostic>, + outcome: &mut SweepOutcome, +) { + let graph_address = graph_address(&sidecar.graph_id); + let schema_addr = schema_address(&sidecar.graph_id); + + // Digest-based classification: robust to unrelated manifest movement; + // the sidecar's version pins stay forensic. + let live_digest = match Omnigraph::open_read_only(&sidecar.graph_uri).await { + Ok(db) => sha256_hex(db.schema_source().as_bytes()), + Err(err) => { + // Cannot verify the interrupted operation — refuse to guess. + diagnostics.push(Diagnostic::warning( + "cluster_recovery_pending", + graph_address.clone(), + format!( + "an interrupted schema apply cannot be verified (graph '{}' did not open: {err}); graph-moving work is blocked until repaired", + sidecar.graph_uri + ), + )); + outcome.pending_graphs.insert(sidecar.graph_id.clone()); + return; + } + }; + + let recorded = state + .applied_revision + .resources + .get(&schema_addr) + .map(|resource| resource.digest.clone()); + if recorded.as_deref() == Some(live_digest.as_str()) { + // Ledger consistent with the live graph (the apply never landed, or + // landed and was recorded): the sidecar is stale intent — retire it. + outcome.completed_sidecars.push(path); + } else if live_digest == sidecar.desired_schema_digest { + // RFC-004 §D3 row 3: the schema apply completed on the graph; roll + // the cluster state forward to observable reality. + state.applied_revision.resources.insert( + schema_addr.clone(), + StateResource { + digest: live_digest.clone(), + }, + ); + let query_digests = state_query_digests_for_graph(state, &sidecar.graph_id); + let composite = graph_digest(&sidecar.graph_id, Some(&live_digest), Some(&query_digests)); + state + .applied_revision + .resources + .insert(graph_address.clone(), StateResource { digest: composite }); + set_resource_status_applied(state, &graph_address); + set_resource_status_applied(state, &schema_addr); + state.recovery_records.insert( + sidecar.operation_id.clone(), + json!({ + "kind": "schema_apply", + "graph_id": sidecar.graph_id, + "outcome": "rolled_forward", + "recovered_at": now_rfc3339(), + "actor": sidecar.actor, + }), + ); + diagnostics.push(Diagnostic::warning( + "cluster_recovery_rolled_forward", + graph_address.clone(), + "an interrupted schema apply had completed on the graph; cluster state was rolled forward to match", + )); + outcome.completed_sidecars.push(path); + } else { + // Row 6: live schema is neither the recorded nor the desired digest. + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Drifted, + "actual_applied_state_pending", + "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", + ); + set_resource_status( + state, + &schema_addr, + ResourceLifecycleStatus::Drifted, + "actual_applied_state_pending", + "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", + ); + diagnostics.push(Diagnostic::warning( + "cluster_recovery_pending", + graph_address.clone(), + "an interrupted schema apply left unexpected graph state; graph-moving work is blocked until repaired", + )); + outcome.pending_graphs.insert(sidecar.graph_id.clone()); + } +} + /// Read-only commands report pending sidecars without acting on them. fn warn_pending_recovery_sidecars(config_dir: &Path, diagnostics: &mut Vec<Diagnostic>) { let recoveries_dir = config_dir.join(CLUSTER_RECOVERIES_DIR); @@ -5613,6 +5713,135 @@ graphs: ); } + fn write_schema_apply_sidecar( + config_dir: &Path, + graph_id: &str, + desired_schema_digest: &str, + operation_id: &str, + ) -> PathBuf { + let dir = config_dir.join(CLUSTER_RECOVERIES_DIR); + fs::create_dir_all(&dir).unwrap(); + let path = dir.join(format!("{operation_id}.json")); + fs::write( + &path, + serde_json::to_string_pretty(&json!({ + "schema_version": 1, + "operation_id": operation_id, + "started_at": "1970-01-01T00:00:00Z", + "kind": "schema_apply", + "graph_id": graph_id, + "graph_uri": derived_graph_uri(config_dir, graph_id), + "desired_schema_digest": desired_schema_digest, + })) + .unwrap(), + ) + .unwrap(); + path + } + + const SCHEMA_V2: &str = "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n"; + + #[tokio::test] + async fn sweep_retires_schema_sidecar_when_ledger_consistent() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); // state digest == live digest + let sidecar = + write_schema_apply_sidecar(dir.path(), "knowledge", "never-applied", "01SROW1"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!sidecar.exists()); + let state = read_state_json(dir.path()); + assert!( + state["recovery_records"] + .as_object() + .is_none_or(|records| records.is_empty()) + ); + } + + #[tokio::test] + async fn sweep_rolls_forward_completed_schema_apply() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + // The schema apply completed on the graph out-of-process... + let graph_uri = derived_graph_uri(dir.path(), "knowledge"); + let db = Omnigraph::open(&graph_uri).await.unwrap(); + db.apply_schema(SCHEMA_V2).await.unwrap(); + // ...the desired config matches it, and the sidecar records the intent. + fs::write(dir.path().join("people.pg"), SCHEMA_V2).unwrap(); + let desired = validate_config_dir(dir.path()); + let v2_digest = desired.resource_digests["schema.knowledge"].clone(); + let sidecar = write_schema_apply_sidecar(dir.path(), "knowledge", &v2_digest, "01SROW3"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") + ); + assert!(!sidecar.exists()); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + v2_digest + ); + assert!( + state["recovery_records"] + .as_object() + .unwrap() + .values() + .any(|record| record["kind"] == "schema_apply" + && record["outcome"] == "rolled_forward") + ); + assert!(out.converged, "{out:?}"); + } + + #[tokio::test] + async fn sweep_flags_unexpected_schema_apply_state_as_pending() { + let dir = fixture(); + init_derived_graph(dir.path()).await; // live = v1 + write_state_resources(dir.path(), &[("schema.knowledge", "stale-digest")]); + // Sidecar intended a digest that is neither live nor recorded. + let sidecar = + write_schema_apply_sidecar(dir.path(), "knowledge", "intended-digest", "01SROW6"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); // warnings only + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_pending") + ); + assert!(sidecar.exists()); + let state = read_state_json(dir.path()); + assert_eq!( + state["resource_statuses"]["schema.knowledge"]["status"], + "drifted" + ); + } + + #[tokio::test] + async fn sweep_keeps_schema_sidecar_for_unopenable_root() { + let dir = fixture(); + write_applyable_state(dir.path()); + let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); + fs::create_dir_all(&root).unwrap(); // exists, won't open + let sidecar = + write_schema_apply_sidecar(dir.path(), "knowledge", "whatever", "01SROWX"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); // warning: cannot verify + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_pending") + ); + assert!(sidecar.exists()); + } + #[test] fn status_warns_on_pending_recovery_sidecar() { let dir = fixture(); From a1ba4dc413941d7da87477285c3200945929b677 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 13:12:15 +0300 Subject: [PATCH 064/165] feat(cluster): execute schema applies in cluster apply MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Stage 4B (RFC-004 §D1/§D5): schema.<id> Update changes classify Applied and execute after graph creates, sequentially and sidecar-fenced — read-write open (the engine's own recovery runs first), pre-op manifest pin recorded, apply_schema_as with allow_data_loss: false (soft drops only; hard drops wait for 4C's approval artifacts), post-op pin rewritten into the sidecar, sidecar retired only after the final state CAS. Queries gated on a same-plan schema update unblock (the migration lands first in the same run); failures — unsupported migrations, lock contention, user branches — surface as schema_apply_failed with the engine's message, demote dependents via the origin-aware demotion helper, and stop further graph-moving work. Schema evolution is now fully cluster-driven (the defer -> manual schema apply -> refresh loop is gone), and out-of-band schema drift is converged back by apply as an ordinary soft migration (axiom 8: drift correction is gated like any change; the recoverable tier needs no approval) — both pinned by reworked e2es. The multi-graph mixed e2e's deferred row is now delete-shaped, pre-staging the 4C surface. Actor: cluster apply accepts the CLI's global --as via the new ApplyOptions / apply_config_dir_with_options (apply_config_dir delegates unchanged); the actor is echoed in ApplyOutput and recorded in sidecars and audit entries, and threads to apply_schema_as so Cedar fires wherever a checker is installed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 15 +- crates/omnigraph-cli/tests/cli.rs | 196 +++++++------ crates/omnigraph-cluster/src/lib.rs | 419 ++++++++++++++++++++++------ 3 files changed, 450 insertions(+), 180 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index de87309..942bb27 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -11,8 +11,8 @@ use omnigraph::db::{Omnigraph, ReadTarget, SnapshotId}; use omnigraph::loader::LoadMode; use omnigraph::storage::normalize_root_uri; use omnigraph_cluster::{ - ApplyOutput, DiagnosticSeverity, ForceUnlockOutput, PlanOutput, StateSyncOutput, StatusOutput, - ValidateOutput, apply_config_dir, force_unlock_config_dir, import_config_dir, plan_config_dir, + ApplyOptions, ApplyOutput, DiagnosticSeverity, ForceUnlockOutput, PlanOutput, StateSyncOutput, StatusOutput, + ValidateOutput, apply_config_dir_with_options, force_unlock_config_dir, import_config_dir, plan_config_dir, refresh_config_dir, status_config_dir, validate_config_dir, }; use omnigraph_compiler::query::parser::parse_query; @@ -3569,7 +3569,16 @@ async fn main() -> Result<()> { finish_cluster_plan(&output, json)?; } ClusterCommand::Apply { config, json } => { - let output = apply_config_dir(config).await; + // The global --as actor attributes graph-moving operations + // (sidecars, audit entries, engine schema-apply commits). + // Cluster config stays unlayered: no omnigraph.yaml fallback. + let output = apply_config_dir_with_options( + config, + ApplyOptions { + actor: cli.as_actor.clone(), + }, + ) + .await; finish_cluster_apply(&output, json)?; } ClusterCommand::Status { config, json } => { diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 7ab7ca9..1805e29 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -984,7 +984,7 @@ query find_person($name: String) { /// deferred by cluster apply, executed by `omnigraph schema apply` against /// the graph, picked up by `cluster refresh`, and the next apply re-converges. #[test] -fn cluster_e2e_schema_change_defers_until_schema_apply_and_refresh() { +fn cluster_e2e_schema_change_applied_by_cluster() { let temp = tempdir().unwrap(); write_cluster_config_fixture(temp.path()); init_cluster_derived_graph(temp.path()); @@ -993,7 +993,8 @@ fn cluster_e2e_schema_change_defers_until_schema_apply_and_refresh() { let apply = cluster_json(temp.path(), "apply"); assert_eq!(apply["converged"], true, "{apply}"); - // Additive schema change: cluster apply must defer it loudly, not act. + // Additive schema change: Stage 4B applies it from the cluster — no + // manual schema apply, no refresh round-trip. fs::write( temp.path().join("people.pg"), r#" @@ -1005,40 +1006,39 @@ node Person { "#, ) .unwrap(); - let deferred = cluster_json(temp.path(), "apply"); - assert_eq!(deferred["ok"], true, "{deferred}"); - assert_eq!(deferred["applied_count"], 0, "{deferred}"); - assert_eq!(deferred["converged"], false, "{deferred}"); + + // Plan previews the real migration steps (RFC-004 §D7). + let plan = cluster_json(temp.path(), "plan"); + let schema_change = change_for(&plan, "schema.knowledge"); + assert_eq!(schema_change["disposition"], "applied", "{plan}"); + let migration = &schema_change["migration"]; + assert_eq!(migration["supported"], true, "{plan}"); assert!( - deferred["diagnostics"] + migration["steps"] .as_array() .unwrap() .iter() - .any(|diagnostic| diagnostic["code"] == "apply_unsupported_change"), - "{deferred}" + .any(|step| step["kind"] == "add_property"), + "{plan}" ); - // The graph-plane tool applies the migration... - output_success( + let evolve = cluster_json(temp.path(), "apply"); + assert_eq!(evolve["ok"], true, "{evolve}"); + assert_eq!(evolve["converged"], true, "{evolve}"); + assert_eq!(change_for(&evolve, "schema.knowledge")["disposition"], "applied"); + + // The live graph carries the new schema; the plan is empty. + let schema_show = output_success( cli() .arg("schema") - .arg("apply") - .arg(temp.path().join("graphs/knowledge.omni")) - .arg("--schema") - .arg(temp.path().join("people.pg")) - .arg("--json"), + .arg("show") + .arg(temp.path().join("graphs/knowledge.omni")), ); - // ...refresh observes it... - let refresh = cluster_json(temp.path(), "refresh"); - assert_eq!(refresh["ok"], true, "{refresh}"); - // ...and the control plane re-converges. - let reconverge = cluster_json(temp.path(), "apply"); - assert_eq!(reconverge["ok"], true, "{reconverge}"); - assert_eq!(reconverge["converged"], true, "{reconverge}"); + assert!(stdout_string(&schema_show).contains("bio"), "live schema updated"); let replan = cluster_json(temp.path(), "plan"); assert!( replan["changes"].as_array().unwrap().is_empty(), - "after schema apply + refresh + apply, the plan must be empty: {replan}" + "one cluster apply converges a schema change: {replan}" ); } @@ -1207,7 +1207,7 @@ fn cluster_e2e_lost_state_reimport_recovers_catalog() { /// the graph (no config change) must surface as drift through refresh, status, /// and plan — and apply must never silently "correct" it. #[test] -fn cluster_e2e_out_of_band_schema_change_surfaces_as_drift() { +fn cluster_e2e_out_of_band_schema_drift_then_apply_converges_it() { let temp = tempdir().unwrap(); write_cluster_config_fixture(temp.path()); init_cluster_derived_graph(temp.path()); @@ -1238,48 +1238,42 @@ node Person { .arg("--json"), ); + // Drift is visible... let refresh = cluster_json(temp.path(), "refresh"); - assert_eq!(refresh["ok"], true, "{refresh}"); assert_eq!( refresh["resource_statuses"]["schema.knowledge"]["status"], "drifted" ); - assert_eq!( - refresh["resource_statuses"]["graph.knowledge"]["status"], - "drifted" - ); - assert_eq!( - refresh["observations"]["graph.knowledge"]["schema_matches_desired"], - false - ); - - let status = cluster_json(temp.path(), "status"); - assert_eq!( - status["resource_statuses"]["schema.knowledge"]["status"], - "drifted" - ); - + // ...the plan proposes converging back to desired, with a migration + // preview (a soft drop of the out-of-band field)... let plan = cluster_json(temp.path(), "plan"); - assert_eq!(change_for(&plan, "schema.knowledge")["disposition"], "deferred"); - assert_eq!(change_for(&plan, "graph.knowledge")["disposition"], "deferred"); - let live_schema_digest = change_for(&plan, "schema.knowledge")["before_digest"] - .as_str() - .unwrap() - .to_string(); - - let drift_apply = cluster_json(temp.path(), "apply"); - assert_eq!(drift_apply["applied_count"], 0, "{drift_apply}"); - assert_eq!(drift_apply["converged"], false, "{drift_apply}"); - // Apply must not have "corrected" the drift: state still records the LIVE - // schema digest, not the desired one. - let state: serde_json::Value = serde_json::from_str( - &fs::read_to_string(temp.path().join("__cluster/state.json")).unwrap(), - ) - .unwrap(); - assert_eq!( - state["applied_revision"]["resources"]["schema.knowledge"]["digest"], - live_schema_digest + let schema_change = change_for(&plan, "schema.knowledge"); + assert_eq!(schema_change["disposition"], "applied", "{plan}"); + assert!( + schema_change["migration"]["steps"] + .as_array() + .unwrap() + .iter() + .any(|step| step["kind"] == "drop_property" && step["mode"] == "soft"), + "{plan}" ); + // ...and apply converges the live schema back (axiom 8: drift correction + // is gated like any change; a soft migration is the recoverable tier). + let converge = cluster_json(temp.path(), "apply"); + assert_eq!(converge["ok"], true, "{converge}"); + assert_eq!(converge["converged"], true, "{converge}"); + let schema_show = output_success( + cli() + .arg("schema") + .arg("show") + .arg(temp.path().join("graphs/knowledge.omni")), + ); + assert!( + !stdout_string(&schema_show).contains("bio"), + "out-of-band field soft-dropped back to desired" + ); + let replan = cluster_json(temp.path(), "plan"); + assert!(replan["changes"].as_array().unwrap().is_empty(), "{replan}"); } /// Disaster input fails closed: a destroyed graph root drifts the ledger, @@ -1393,12 +1387,32 @@ fn cluster_e2e_multi_graph_mixed_dispositions_then_converge() { assert!(temp.path().join("graphs/knowledge.omni").exists()); assert!(temp.path().join("graphs/engineering.omni").exists()); - // Mixed run: a knowledge schema update (4B territory — deferred) gates - // its query update (blocked), while an engineering query update is - // independent (applied) and re-derives its composite. + // Mixed run: a graph REMOVAL (4C territory — deferred) gates its query + // delete (blocked), while a knowledge query update is independent + // (applied) and re-derives its composite. All four dispositions at once. fs::write( - temp.path().join("people.pg"), - "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n", + temp.path().join("cluster.yaml"), + r#" +version: 1 +metadata: + name: company-brain +state: + backend: cluster + lock: true +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +policies: + shared: + file: ./shared.policy.yaml + applies_to: [knowledge] + cluster_wide: + file: ./cluster_wide.policy.yaml + applies_to: [cluster] +"#, ) .unwrap(); fs::write( @@ -1406,31 +1420,35 @@ fn cluster_e2e_multi_graph_mixed_dispositions_then_converge() { "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", ) .unwrap(); - fs::write( - temp.path().join("services.gq"), - "\nquery find_service($name: String) {\n match { $s: Service { name: $name } }\n return { $s.name, $s.name }\n}\n", - ) - .unwrap(); let mixed = cluster_json(temp.path(), "apply"); assert_eq!(mixed["ok"], true, "{mixed}"); assert_eq!(mixed["converged"], false, "{mixed}"); - assert_eq!(change_for(&mixed, "schema.knowledge")["disposition"], "deferred"); - assert_eq!(change_for(&mixed, "graph.knowledge")["disposition"], "deferred"); assert_eq!( - change_for(&mixed, "query.knowledge.find_person")["disposition"], - "blocked" + change_for(&mixed, "graph.engineering")["disposition"], + "deferred" ); assert_eq!( - change_for(&mixed, "query.knowledge.find_person")["reason"], - "dependency_not_applied" + change_for(&mixed, "schema.engineering")["disposition"], + "deferred" ); assert_eq!( change_for(&mixed, "query.engineering.find_service")["disposition"], - "applied" + "blocked" ); assert_eq!( - change_for(&mixed, "graph.engineering")["disposition"], + change_for(&mixed, "query.engineering.find_service")["reason"], + "dependency_not_applied" + ); + assert_eq!( + change_for(&mixed, "query.knowledge.find_person")["disposition"], + "applied" + ); + // policy.shared's applies_to narrowed, but its FILE digest is unchanged + // — applies_to lives in cluster.yaml (the config digest), so it is not a + // resource change. + assert_eq!( + change_for(&mixed, "graph.knowledge")["disposition"], "derived" ); // Deterministic ordering: changes sorted by resource address. @@ -1443,27 +1461,7 @@ fn cluster_e2e_multi_graph_mixed_dispositions_then_converge() { let mut sorted = order.clone(); sorted.sort_unstable(); assert_eq!(order, sorted, "{mixed}"); - - // The graph-plane tool applies the schema; refresh observes; converge. - output_success( - cli() - .arg("schema") - .arg("apply") - .arg(temp.path().join("graphs/knowledge.omni")) - .arg("--schema") - .arg(temp.path().join("people.pg")) - .arg("--json"), - ); - let refresh = cluster_json(temp.path(), "refresh"); - assert_eq!(refresh["ok"], true, "{refresh}"); - let converge = cluster_json(temp.path(), "apply"); - assert_eq!(converge["converged"], true, "{converge}"); - - let final_plan = cluster_json(temp.path(), "plan"); - assert!( - final_plan["changes"].as_array().unwrap().is_empty(), - "{final_plan}" - ); + // Graph deletion cannot converge until stage 4C's approval artifacts. } /// Stage 4A headline: a declared graph is created by `cluster apply` itself — diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index f8a56e7..11ebcd9 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -274,6 +274,8 @@ pub struct ForceUnlockOutput { pub struct ApplyOutput { pub ok: bool, pub config_dir: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub actor: Option<String>, pub desired_revision: DesiredRevision, pub state_observations: StateObservations, /// Every planned change, with `disposition`/`reason` always populated. @@ -651,12 +653,29 @@ pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { /// state is the publish point: a failure after payload writes leaves inert /// digest-named blobs and no success acknowledgement; re-running apply is the /// repair. +/// Options for `cluster apply`. `actor` attributes graph-moving operations +/// (recorded in sidecars and audit entries, threaded to the engine's +/// `apply_schema_as` so Cedar enforcement fires wherever a policy checker is +/// installed). +#[derive(Debug, Clone, Default)] +pub struct ApplyOptions { + pub actor: Option<String>, +} + pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { + apply_config_dir_with_options(config_dir, ApplyOptions::default()).await +} + +pub async fn apply_config_dir_with_options( + config_dir: impl AsRef<Path>, + options: ApplyOptions, +) -> ApplyOutput { let outcome = load_desired(config_dir.as_ref()); let mut diagnostics = outcome.diagnostics; let backend = LocalStateBackend::new(&outcome.config_dir); let mut observations = backend.observations(); + let actor_for_output = options.actor.clone(); let early_return = |config_dir: String, config_digest: Option<String>, observations: StateObservations, @@ -666,6 +685,7 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { ApplyOutput { ok: !has_errors(&diagnostics), config_dir, + actor: actor_for_output.clone(), desired_revision: DesiredRevision { config_digest, }, @@ -821,18 +841,18 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { }) .filter_map(|change| change.resource.strip_prefix("graph.").map(str::to_string)) .collect(); - let mut completed_create_sidecars: Vec<PathBuf> = Vec::new(); - let mut failed_graphs: BTreeSet<String> = BTreeSet::new(); - let mut creates_aborted = false; + let mut completed_op_sidecars: Vec<PathBuf> = Vec::new(); + let mut failed_graphs: BTreeMap<String, FailedGraphOrigin> = BTreeMap::new(); + let mut graph_moving_aborted = false; for graph_id in &graph_creates_to_run { - if creates_aborted { + if graph_moving_aborted { // A prior create failed: stop graph-moving work (loud partials). diagnostics.push(Diagnostic::warning( "graph_create_skipped", graph_address(graph_id), "skipped after an earlier graph create failed in this run", )); - failed_graphs.insert(graph_id.clone()); + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::GraphCreate); continue; } let Some(desired_graph) = desired.graphs.iter().find(|graph| &graph.id == graph_id) @@ -849,7 +869,7 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { schema_version: 1, operation_id: Ulid::new().to_string(), started_at: now_rfc3339(), - actor: None, + actor: options.actor.clone(), kind: RecoverySidecarKind::GraphCreate, graph_id: graph_id.clone(), graph_uri: graph_uri.clone(), @@ -862,8 +882,8 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { Ok(path) => path, Err(diagnostic) => { diagnostics.push(diagnostic); - failed_graphs.insert(graph_id.clone()); - creates_aborted = true; + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::GraphCreate); + graph_moving_aborted = true; continue; } }; @@ -871,8 +891,8 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { // Simulated crash before the init: the sidecar stays for the // sweep (row 1: root absent -> intent removed next run). diagnostics.push(diagnostic); - failed_graphs.insert(graph_id.clone()); - creates_aborted = true; + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::GraphCreate); + graph_moving_aborted = true; continue; } // Re-read + re-verify the schema source under the lock — the same @@ -911,8 +931,8 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { Err(diagnostic) => { diagnostics.push(diagnostic); let _ = fs::remove_file(&sidecar_path); // nothing moved - failed_graphs.insert(graph_id.clone()); - creates_aborted = true; + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::GraphCreate); + graph_moving_aborted = true; continue; } }; @@ -926,8 +946,8 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { )); // The sidecar stays: the sweep classifies whether the failed // init left a partial root (row 5) or nothing (row 1). - failed_graphs.insert(graph_id.clone()); - creates_aborted = true; + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::GraphCreate); + graph_moving_aborted = true; continue; } } @@ -955,8 +975,174 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { diagnostics, ); } - completed_create_sidecars.push(sidecar_path); + completed_op_sidecars.push(sidecar_path); } + + // Schema applies execute next (RFC-004 §D5): the first cluster operation + // that moves an EXISTING graph manifest, sidecar-fenced the same way. + let schema_updates_to_run: Vec<String> = changes + .iter() + .filter(|change| { + change.disposition == Some(ApplyDisposition::Applied) + && change.operation == PlanOperation::Update + && matches!(resource_kind(&change.resource), ResourceKind::Schema(_)) + }) + .filter_map(|change| change.resource.strip_prefix("schema.").map(str::to_string)) + .collect(); + for graph_id in &schema_updates_to_run { + if graph_moving_aborted { + diagnostics.push(Diagnostic::warning( + "schema_apply_skipped", + schema_address(graph_id), + "skipped after an earlier graph-moving operation failed in this run", + )); + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::SchemaApply); + continue; + } + let Some(desired_graph) = desired.graphs.iter().find(|graph| &graph.id == graph_id) + else { + continue; + }; + let graph_uri = display_path( + &desired + .config_dir + .join(CLUSTER_GRAPHS_DIR) + .join(format!("{graph_id}.omni")), + ); + // Read-write open: the engine's own recovery sweep runs here, which + // is exactly what we want before moving its manifest. + let db = match Omnigraph::open(&graph_uri).await { + Ok(db) => db, + Err(err) => { + diagnostics.push(Diagnostic::error( + "schema_apply_failed", + schema_address(graph_id), + format!("could not open graph at '{graph_uri}': {err}"), + )); + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::SchemaApply); + graph_moving_aborted = true; + continue; + } + }; + let observed_manifest_version = match db.snapshot_of(ReadTarget::branch("main")).await { + Ok(snapshot) => Some(snapshot.version()), + Err(_) => None, + }; + let mut sidecar = RecoverySidecar { + schema_version: 1, + operation_id: Ulid::new().to_string(), + started_at: now_rfc3339(), + actor: options.actor.clone(), + kind: RecoverySidecarKind::SchemaApply, + graph_id: graph_id.clone(), + graph_uri: graph_uri.clone(), + observed_manifest_version, + expected_manifest_version: None, + desired_schema_digest: desired_graph.schema_digest.clone(), + state_cas_base: expected_cas.clone(), + }; + let sidecar_path = match backend.write_recovery_sidecar(&sidecar) { + Ok(path) => path, + Err(diagnostic) => { + diagnostics.push(diagnostic); + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::SchemaApply); + graph_moving_aborted = true; + continue; + } + }; + if let Err(diagnostic) = failpoints::maybe_fail("cluster_apply.before_schema_apply") { + // Simulated crash before the engine call: the sidecar stays; the + // sweep retires it next run (ledger still consistent with live). + diagnostics.push(diagnostic); + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::SchemaApply); + graph_moving_aborted = true; + continue; + } + // Re-read + digest-verify the desired schema source under the lock. + let schema_source = source_paths + .get(schema_address(graph_id).as_str()) + .ok_or_else(|| { + Diagnostic::error( + "schema_apply_failed", + schema_address(graph_id), + "no schema source recorded for graph", + ) + }) + .and_then(|path| { + fs::read_to_string(Path::new(path)).map_err(|err| { + Diagnostic::error( + "schema_apply_failed", + schema_address(graph_id), + format!("could not read schema source '{path}': {err}"), + ) + }) + }) + .and_then(|source| { + if sha256_hex(source.as_bytes()) == desired_graph.schema_digest { + Ok(source) + } else { + Err(Diagnostic::error( + "resource_content_changed", + schema_address(graph_id), + "schema source changed while apply was running; re-run `cluster apply`", + )) + } + }); + let schema_source = match schema_source { + Ok(source) => source, + Err(diagnostic) => { + diagnostics.push(diagnostic); + let _ = fs::remove_file(&sidecar_path); // nothing moved + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::SchemaApply); + graph_moving_aborted = true; + continue; + } + }; + // Soft drops only: allow_data_loss stays false until the approval + // artifacts of stage 4C exist (RFC-004 §D4). + match db + .apply_schema_as( + &schema_source, + SchemaApplyOptions::default(), + options.actor.as_deref(), + ) + .await + { + Ok(result) => { + sidecar.expected_manifest_version = Some(result.manifest_version); + if let Err(diagnostic) = backend.write_recovery_sidecar(&sidecar) { + diagnostics.push(diagnostic); + } + } + Err(err) => { + diagnostics.push(Diagnostic::error( + "schema_apply_failed", + schema_address(graph_id), + format!("schema apply failed on '{graph_uri}': {err}"), + )); + // Sidecar stays; the sweep retires it (live digest unchanged + // == ledger consistent) or flags real movement. + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::SchemaApply); + graph_moving_aborted = true; + continue; + } + } + // Crash point: the manifest moved, the ledger does not record it yet. + // A failure here acknowledges nothing; the sweep rolls forward. + if let Err(diagnostic) = failpoints::maybe_fail("cluster_apply.after_schema_apply") { + diagnostics.push(diagnostic); + return early_return( + display_path(&desired.config_dir), + Some(desired.config_digest), + observations, + changes, + state.resource_statuses, + diagnostics, + ); + } + completed_op_sidecars.push(sidecar_path); + } + if !failed_graphs.is_empty() { demote_dependents_of_failed_graphs(&mut changes, &failed_graphs, &desired.dependencies); } @@ -1116,7 +1302,7 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { for sidecar_path in sweep .completed_sidecars .iter() - .chain(completed_create_sidecars.iter()) + .chain(completed_op_sidecars.iter()) { let _ = fs::remove_file(sidecar_path); } @@ -1148,6 +1334,7 @@ pub async fn apply_config_dir(config_dir: impl AsRef<Path>) -> ApplyOutput { ApplyOutput { ok: !has_errors(&diagnostics), config_dir: display_path(&desired.config_dir), + actor: options.actor.clone(), desired_revision: DesiredRevision { config_digest: Some(desired.config_digest), }, @@ -3006,9 +3193,10 @@ fn classify_changes( PlanOperation::Create => { schema_creates.insert(graph); } - // Schema updates (4B) and deletes (4C) are still pending in - // this stage and block dependents. - _ => { + // Schema updates execute in-run before catalog writes (4B) + // and no longer block dependents; deletes (4C) still do. + PlanOperation::Update => {} + PlanOperation::Delete => { schema_pending.insert(graph); } }, @@ -3042,7 +3230,12 @@ fn classify_changes( // Applied with the graph create — the init carries it. (ApplyDisposition::Applied, None) } - PlanOperation::Create if graph_creates.contains(&graph) => { + PlanOperation::Update if !pending_recovery.contains(&graph) => { + // Stage 4B: schema updates execute via the engine's + // schema apply (soft drops only; allow_data_loss is 4C). + (ApplyDisposition::Applied, None) + } + PlanOperation::Create | PlanOperation::Update => { (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) } _ => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), @@ -3112,13 +3305,20 @@ fn classify_changes( } } -/// After a graph create fails mid-run, every change that depended on that -/// graph (its schema, its queries, policies referencing it) flips from -/// Applied to Blocked so the output and the persisted statuses tell the -/// truth about what this run actually executed. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum FailedGraphOrigin { + GraphCreate, + SchemaApply, +} + +/// After a graph-moving operation fails mid-run, every change that depended +/// on that graph flips from Applied to Blocked so the output and the +/// persisted statuses tell the truth about what this run actually executed. +/// The originating change carries the failure code; dependents carry +/// `dependency_not_applied`. fn demote_dependents_of_failed_graphs( changes: &mut [PlanChange], - failed: &BTreeSet<String>, + failed: &BTreeMap<String, FailedGraphOrigin>, dependencies: &[Dependency], ) { for change in changes.iter_mut() { @@ -3126,11 +3326,17 @@ fn demote_dependents_of_failed_graphs( continue; } let demote_reason = match resource_kind(&change.resource) { - ResourceKind::Graph(graph) if failed.contains(&graph) => Some("graph_create_failed"), - ResourceKind::Schema(graph) if failed.contains(&graph) => { - Some("dependency_not_applied") - } - ResourceKind::Query { graph, .. } if failed.contains(&graph) => { + ResourceKind::Graph(graph) => match failed.get(&graph) { + Some(FailedGraphOrigin::GraphCreate) => Some("graph_create_failed"), + Some(FailedGraphOrigin::SchemaApply) => Some("dependency_not_applied"), + None => None, + }, + ResourceKind::Schema(graph) => match failed.get(&graph) { + Some(FailedGraphOrigin::SchemaApply) => Some("schema_apply_failed"), + Some(FailedGraphOrigin::GraphCreate) => Some("dependency_not_applied"), + None => None, + }, + ResourceKind::Query { graph, .. } if failed.contains_key(&graph) => { Some("dependency_not_applied") } ResourceKind::Policy(_) => { @@ -3139,7 +3345,7 @@ fn demote_dependents_of_failed_graphs( && dep .to .strip_prefix("graph.") - .is_some_and(|graph| failed.contains(graph)) + .is_some_and(|graph| failed.contains_key(graph)) }); blocked.then_some("dependency_not_applied") } @@ -4849,19 +5055,22 @@ graphs: } #[tokio::test] - async fn apply_defers_schema_change_and_blocks_dependent_query() { + async fn apply_schema_update_and_dependent_query_in_one_run() { let dir = fixture(); + init_derived_graph(dir.path()).await; write_applyable_state(dir.path()); - // Change the schema after seeding state: schema.knowledge now differs. + // Schema update + a query update that depends on the new field: one + // apply executes the schema migration first, then the catalog write. + fs::write(dir.path().join("people.pg"), SCHEMA_V2).unwrap(); fs::write( - dir.path().join("people.pg"), - "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n", + dir.path().join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name, $p.bio }\n}\n", ) .unwrap(); let out = apply_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); - assert!(!out.converged); + assert!(out.converged, "{out:?}"); let by_resource: BTreeMap<&str, &PlanChange> = out .changes .iter() @@ -4869,58 +5078,112 @@ graphs: .collect(); assert_eq!( by_resource["schema.knowledge"].disposition, - Some(ApplyDisposition::Deferred) - ); - assert_eq!( - by_resource["graph.knowledge"].disposition, - Some(ApplyDisposition::Deferred) + Some(ApplyDisposition::Applied) ); assert_eq!( by_resource["query.knowledge.find_person"].disposition, + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["graph.knowledge"].disposition, + Some(ApplyDisposition::Derived) + ); + // The live graph carries the new schema. + let db = Omnigraph::open_read_only(&derived_graph_uri(dir.path(), "knowledge")) + .await + .unwrap(); + let desired = validate_config_dir(dir.path()); + assert_eq!( + sha256_hex(db.schema_source().as_bytes()), + desired.resource_digests["schema.knowledge"] + ); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + desired.resource_digests["schema.knowledge"] + ); + // Sidecar retired after the CAS landed. + assert!( + !dir.path().join(CLUSTER_RECOVERIES_DIR).exists() + || fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) + .unwrap() + .next() + .is_none() + ); + } + + #[tokio::test] + async fn apply_unsupported_schema_change_fails_loudly() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + // Property type changes are unsupported by the engine planner. + fs::write( + dir.path().join("people.pg"), + "\nnode Person {\n name: String @key\n age: I64?\n}\n", + ) + .unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!(out.diagnostics.iter().any(|diagnostic| { + diagnostic.code == "schema_apply_failed" + && diagnostic.message.contains("changing property type") + })); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + assert_eq!( + by_resource["schema.knowledge"].disposition, Some(ApplyDisposition::Blocked) ); assert_eq!( - by_resource["query.knowledge.find_person"].reason.as_deref(), - Some("dependency_not_applied") + by_resource["schema.knowledge"].reason.as_deref(), + Some("schema_apply_failed") ); - // Policy is independent of the schema and still applies. - assert_eq!( - by_resource["policy.base"].disposition, - Some(ApplyDisposition::Applied) - ); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "apply_unsupported_change") - ); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "apply_dependency_blocked") - ); - + // The live schema and the ledger are unchanged. let state = read_state_json(dir.path()); + let desired = validate_config_dir(dir.path()); + assert_ne!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + desired.resource_digests["schema.knowledge"] + ); + // Second run: the sweep retires the stale sidecar (ledger consistent) + // and the run fails just as loudly — idempotent loudness. + let second = apply_config_dir(dir.path()).await; + assert!(!second.ok); + assert!( + second + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "schema_apply_failed") + ); + } + + #[tokio::test] + async fn apply_blocks_schema_update_while_recovery_pending() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_state_resources(dir.path(), &[("schema.knowledge", "stale-digest")]); + fs::write(dir.path().join("people.pg"), SCHEMA_V2).unwrap(); + // A pending sidecar whose intent matches neither live nor recorded. + write_schema_apply_sidecar(dir.path(), "knowledge", "intended-digest", "01PENDS"); + + let out = apply_config_dir(dir.path()).await; + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); assert_eq!( - state["resource_statuses"]["query.knowledge.find_person"]["status"], - "blocked" + by_resource["schema.knowledge"].disposition, + Some(ApplyDisposition::Blocked) ); - // The blocked query wrote no payload and no state digest. - assert!( - state["applied_revision"]["resources"] - .get("query.knowledge.find_person") - .is_none() - ); - assert!( - !dir.path() - .join(CLUSTER_RESOURCES_DIR) - .join("query") - .exists() - ); - // Not converged: the applied config digest must not be claimed. - assert!( - state["applied_revision"] - .get("config_digest") - .is_none_or(serde_json::Value::is_null) + assert_eq!( + by_resource["schema.knowledge"].reason.as_deref(), + Some("cluster_recovery_pending") ); } From 80cae4e8e1cf947b17f57826d6cd43b1bf71c54a Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 13:13:15 +0300 Subject: [PATCH 065/165] test(cluster): failpoint coverage for schema-apply crash windows - Crash before the engine call: sidecar (carrying the --as actor) survives, live schema and ledger untouched, no ack; the next run's sweep retires the stale intent and the same run applies and converges. - Crash after the engine call, before the state CAS: the manifest moved with the post-op pin in the sidecar, state.json byte-identical; the next run's sweep rolls the ledger forward with a schema_apply audit entry and the run converges. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/tests/failpoints.rs | 124 ++++++++++++++++++- 1 file changed, 123 insertions(+), 1 deletion(-) diff --git a/crates/omnigraph-cluster/tests/failpoints.rs b/crates/omnigraph-cluster/tests/failpoints.rs index ec8ddfb..cc91b85 100644 --- a/crates/omnigraph-cluster/tests/failpoints.rs +++ b/crates/omnigraph-cluster/tests/failpoints.rs @@ -14,7 +14,10 @@ use std::path::{Path, PathBuf}; use fail::FailScenario; use omnigraph_cluster::failpoints::ScopedFailPoint; -use omnigraph_cluster::{apply_config_dir, validate_config_dir}; +use omnigraph::db::Omnigraph; +use omnigraph_cluster::{ + ApplyOptions, apply_config_dir, apply_config_dir_with_options, validate_config_dir, +}; use tempfile::tempdir; const SCHEMA: &str = r#" @@ -345,3 +348,122 @@ async fn create_crash_after_init_rolls_state_forward() { ); scenario.teardown(); } + +const SCHEMA_V2: &str = r#" +node Person { + name: String @key + age: I32? + bio: String? +} +"#; + +async fn converge_with_live_graph(dir: &Path) { + let graph_dir = dir.join("graphs"); + fs::create_dir_all(&graph_dir).unwrap(); + Omnigraph::init( + graph_dir.join("knowledge.omni").to_string_lossy().as_ref(), + SCHEMA, + ) + .await + .unwrap(); + seed_applyable_state(dir); + let out = apply_config_dir(dir).await; + assert!(out.ok && out.converged, "{:?}", out.diagnostics); +} + +async fn live_schema_digest(dir: &Path) -> String { + let uri = dir.join("graphs/knowledge.omni"); + let db = Omnigraph::open_read_only(uri.to_string_lossy().as_ref()) + .await + .unwrap(); + use sha2::{Digest, Sha256}; + let digest = Sha256::digest(db.schema_source().as_bytes()); + digest.iter().map(|byte| format!("{byte:02x}")).collect() +} + +/// Crash before the engine schema apply: sidecar (with actor) survives, the +/// live schema and ledger are untouched; the next run's sweep retires the +/// stale intent and the same run applies and converges. +#[tokio::test] +async fn schema_crash_before_apply_recovers_via_sweep() { + let scenario = FailScenario::setup(); + let dir = fixture(); + converge_with_live_graph(dir.path()).await; + let pre_digest = live_schema_digest(dir.path()).await; + fs::write(dir.path().join("people.pg"), SCHEMA_V2).unwrap(); + + { + let _failpoint = ScopedFailPoint::new("cluster_apply.before_schema_apply", "return"); + let out = apply_config_dir_with_options( + dir.path(), + ApplyOptions { + actor: Some("test-actor".to_string()), + }, + ) + .await; + assert!(!out.ok); + assert_eq!(out.actor.as_deref(), Some("test-actor")); + let sidecars = recovery_sidecars(dir.path()); + assert_eq!(sidecars.len(), 1); + let sidecar: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&sidecars[0]).unwrap()).unwrap(); + assert_eq!(sidecar["kind"], "schema_apply"); + assert_eq!(sidecar["actor"], "test-actor"); + // Nothing moved. + assert_eq!(live_schema_digest(dir.path()).await, pre_digest); + } + + let recovered = apply_config_dir(dir.path()).await; + assert!(recovered.ok, "{:?}", recovered.diagnostics); + assert!(recovered.converged); + assert!(recovery_sidecars(dir.path()).is_empty()); + assert_ne!(live_schema_digest(dir.path()).await, pre_digest); + scenario.teardown(); +} + +/// Crash after the engine schema apply, before the state CAS: the manifest +/// moved, the ledger is stale, nothing acknowledged; the next run's sweep +/// rolls the ledger forward with an audit entry and the run converges. +#[tokio::test] +async fn schema_crash_after_apply_rolls_state_forward() { + let scenario = FailScenario::setup(); + let dir = fixture(); + converge_with_live_graph(dir.path()).await; + fs::write(dir.path().join("people.pg"), SCHEMA_V2).unwrap(); + let state_before = fs::read(state_path(dir.path())).unwrap(); + let desired = validate_config_dir(dir.path()); + let v2_digest = desired.resource_digests["schema.knowledge"].clone(); + + { + let _failpoint = ScopedFailPoint::new("cluster_apply.after_schema_apply", "return"); + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!(!out.state_written); + // The live schema moved; the ledger is byte-identical (no ack). + assert_eq!(live_schema_digest(dir.path()).await, v2_digest); + assert_eq!(fs::read(state_path(dir.path())).unwrap(), state_before); + let sidecars = recovery_sidecars(dir.path()); + assert_eq!(sidecars.len(), 1); + let sidecar: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&sidecars[0]).unwrap()).unwrap(); + assert!(sidecar["expected_manifest_version"].is_number(), "{sidecar}"); + } + + let recovered = apply_config_dir(dir.path()).await; + assert!(recovered.ok, "{:?}", recovered.diagnostics); + assert!( + recovered + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") + ); + assert!(recovered.converged); + assert!(recovery_sidecars(dir.path()).is_empty()); + let state: serde_json::Value = + serde_json::from_str(&fs::read_to_string(state_path(dir.path())).unwrap()).unwrap(); + assert_eq!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + v2_digest + ); + scenario.teardown(); +} From f217352c93d84513dc538e5a560c3dd438decf8a Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 13:14:20 +0300 Subject: [PATCH 066/165] docs(cluster): document Stage 4B schema apply Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/testing.md | 2 +- docs/user/cluster-config.md | 57 +++++++++++++++++++++++++++++++------ 2 files changed, 50 insertions(+), 9 deletions(-) diff --git a/docs/dev/testing.md b/docs/dev/testing.md index c171f53..5402ccf 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,7 +8,7 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish; includes the `cluster_e2e_*` lifecycle compositions over the spawned binary — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | -| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, and Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows) | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), and Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows) | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 7ff49e8..9de305a 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -1,6 +1,6 @@ # Cluster Config -**Status:** Stage 4A graph-create apply preview. +**Status:** Stage 4B schema-apply preview. Cluster config is the future control-plane configuration surface for a whole OmniGraph deployment. In this stage, OmniGraph can validate a local @@ -8,11 +8,12 @@ OmniGraph deployment. In this stage, OmniGraph can validate a local local JSON state ledger, explicitly refresh/import graph observations into that ledger, manually remove a held local state lock by exact lock id, and **apply the executable subset of the plan** — stored-query and policy-bundle -catalog writes, and **graph creation**: a declared graph that does not exist -yet is initialized by apply itself at the derived root. It does not change -existing schemas (deferred to a later stage), move existing graph manifests, -start servers, or serve anything it applies: the server still boots from -`omnigraph.yaml`. +catalog writes, **graph creation** (a declared graph that does not exist yet +is initialized by apply at the derived root), and **schema updates**: a +changed schema is migrated on the live graph by apply itself, soft drops +only. It does not delete graphs (a later stage), perform data-loss +migrations, start servers, or serve anything it applies: the server still +boots from `omnigraph.yaml`. ## Commands @@ -156,7 +157,8 @@ condition in `reason`). ## Apply `cluster apply` executes the executable subset of the plan — stored-query and -policy-bundle changes, and graph creates. There is no confirm flag: `cluster plan` is the preview, +policy-bundle changes, graph creates, and schema updates. There is no confirm +flag: `cluster plan` is the preview, and apply recomputes the same diff under the state lock before executing, so a stale preview can never be applied. Apply requires an existing `state.json` (`state_missing` directs you to `cluster import` first). @@ -212,7 +214,46 @@ will execute it, producing an **empty** graph at the root. The data was already lost when the root vanished; the create is visible in the plan (disposition `applied`) before anything runs. -Schema changes to existing graphs are never executed by this stage. They are +### Schema updates + +A `schema.<id>` update (the declared schema differs from what state records) +is executed by apply via the engine's schema-apply, after graph creates and +before catalog writes — so a query change that depends on the new schema +applies in the same run. Each schema apply is sidecar-fenced like a create: +pre-operation manifest version recorded, post-operation version written back, +sidecar retired only after the state update lands; the recovery sweep +classifies survivors by schema digest (consistent ledger → retired; completed +on the graph → state rolled forward with an audit entry; anything else → +`drifted`/`actual_applied_state_pending`, kept). + +Migrations run with **soft drops only** — a removed property disappears from +the current version while prior versions retain the data (reversible until +`cleanup`). Data-loss migrations (`allow_data_loss`) are not reachable from +cluster apply until the approval-artifact stage. Unsupported migrations +(e.g. changing a property's type), engine lock contention, or graphs with +user branches fail loudly as `schema_apply_failed` with the engine's message; +dependent changes are demoted to `blocked` and graph-moving work stops for +the run. + +`cluster plan` previews schema updates with the engine's real migration plan: +each schema change carries a `migration` field (`supported` + typed steps), +and the human output prints the steps. If the live graph cannot be opened the +preview degrades to the digest diff with a `schema_preview_unavailable` +warning. + +**Drift is converged, not just reported.** A schema changed out-of-band on +the live graph shows up as `drifted` after `refresh`, and the next plan +proposes migrating it back to the declared schema — apply executes that like +any other soft migration. Drift correction is gated by the same rules as any +change; nothing about it is hidden (the plan shows the steps, including soft +drops of out-of-band fields). + +**Attribution.** `cluster apply --as <actor>` records the operator identity +in recovery sidecars and audit entries and threads it to the engine's +schema-apply (so commit attribution and Cedar enforcement — wherever a policy +checker is installed — work unchanged). + +Schema deletes (removing a graph) are never executed by this stage. They are reported as `deferred` (warning `apply_unsupported_change`), and query/policy changes that depend on them are `blocked` (warning `apply_dependency_blocked`, status `blocked` in state). A partially-applicable plan still exits 0 with warnings; From f4e91052724c96ff0a145c9b951e07a51a00e722 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 14:29:00 +0300 Subject: [PATCH 067/165] =?UTF-8?q?feat(cluster):=20cluster=20approve=20?= =?UTF-8?q?=E2=80=94=20digest-bound=20approval=20artifacts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit RFC-004 §D4, gate half: graph deletes (and their subtree) now classify Blocked/approval_required instead of Deferred; the new cluster approve command (requires the global --as actor) writes __cluster/approvals/{ulid}.json bound to the desired config digest and the change's before/after digests, so config or state drift invalidates the artifact automatically (approval_stale warning, never authorizes). One gate per subtree: compute_approvals lists only the graph-level delete, and ApprovalRequirement gains a satisfied flag surfaced by plan. Consumption and the delete executor land next — until then approved deletes stay blocked so a gate-only build can never strip state without removing the root. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 56 ++- crates/omnigraph-cli/tests/cli.rs | 29 +- crates/omnigraph-cluster/src/lib.rs | 565 ++++++++++++++++++++++++++-- 3 files changed, 605 insertions(+), 45 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 942bb27..8593ef3 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -11,8 +11,8 @@ use omnigraph::db::{Omnigraph, ReadTarget, SnapshotId}; use omnigraph::loader::LoadMode; use omnigraph::storage::normalize_root_uri; use omnigraph_cluster::{ - ApplyOptions, ApplyOutput, DiagnosticSeverity, ForceUnlockOutput, PlanOutput, StateSyncOutput, StatusOutput, - ValidateOutput, apply_config_dir_with_options, force_unlock_config_dir, import_config_dir, plan_config_dir, + ApplyOptions, ApplyOutput, ApproveOutput, DiagnosticSeverity, ForceUnlockOutput, PlanOutput, StateSyncOutput, StatusOutput, + ValidateOutput, apply_config_dir_with_options, approve_config_dir, force_unlock_config_dir, import_config_dir, plan_config_dir, refresh_config_dir, status_config_dir, validate_config_dir, }; use omnigraph_compiler::query::parser::parse_query; @@ -371,6 +371,18 @@ enum ClusterCommand { #[arg(long)] json: bool, }, + /// Record a digest-bound approval for a gated (irreversible) change, + /// e.g. a graph delete. Requires the global --as actor. + Approve { + /// Typed resource address of the gated change (e.g. graph.scratch). + resource: String, + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, /// Read the local JSON state ledger without scanning live graph resources. Status { /// Cluster config directory containing cluster.yaml. @@ -1011,6 +1023,33 @@ fn finish_cluster_apply(output: &ApplyOutput, json: bool) -> Result<()> { Ok(()) } +fn finish_cluster_approve(output: &ApproveOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else if output.ok { + println!( + "cluster approve: {} {} approved by {} (approval {})", + output + .operation + .as_ref() + .map(|operation| format!("{operation:?}").to_lowercase()) + .unwrap_or_default(), + output.resource.as_deref().unwrap_or("?"), + output.approved_by.as_deref().unwrap_or("?"), + output.approval_id.as_deref().unwrap_or("?"), + ); + print_cluster_diagnostics(&output.diagnostics); + } else { + println!("cluster approve failed"); + print_cluster_diagnostics(&output.diagnostics); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + fn finish_cluster_status(output: &StatusOutput, json: bool) -> Result<()> { if json { print_json(output)?; @@ -3581,6 +3620,19 @@ async fn main() -> Result<()> { .await; finish_cluster_apply(&output, json)?; } + ClusterCommand::Approve { + resource, + config, + json, + } => { + let Some(approver) = cli.as_actor.as_deref() else { + bail!( + "`cluster approve` requires the global --as <ACTOR> flag: an approval without an approver is meaningless" + ); + }; + let output = approve_config_dir(config, &resource, approver).await; + finish_cluster_approve(&output, json)?; + } ClusterCommand::Status { config, json } => { let output = status_config_dir(config); finish_cluster_status(&output, json)?; diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 1805e29..bfa538d 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -1424,22 +1424,29 @@ policies: let mixed = cluster_json(temp.path(), "apply"); assert_eq!(mixed["ok"], true, "{mixed}"); assert_eq!(mixed["converged"], false, "{mixed}"); + // Stage 4C: deletes are gated on a digest-bound approval, one gate per + // subtree (the graph-level approval carries schema + queries). assert_eq!( change_for(&mixed, "graph.engineering")["disposition"], - "deferred" - ); - assert_eq!( - change_for(&mixed, "schema.engineering")["disposition"], - "deferred" - ); - assert_eq!( - change_for(&mixed, "query.engineering.find_service")["disposition"], "blocked" ); assert_eq!( - change_for(&mixed, "query.engineering.find_service")["reason"], - "dependency_not_applied" + change_for(&mixed, "graph.engineering")["reason"], + "approval_required" ); + assert_eq!( + change_for(&mixed, "schema.engineering")["reason"], + "approval_required" + ); + assert_eq!( + change_for(&mixed, "query.engineering.find_service")["reason"], + "approval_required" + ); + let gate_plan = cluster_json(temp.path(), "plan"); + let gates = gate_plan["approvals_required"].as_array().unwrap(); + assert_eq!(gates.len(), 1, "{gate_plan}"); + assert_eq!(gates[0]["resource"], "graph.engineering"); + assert_eq!(gates[0]["satisfied"], false); assert_eq!( change_for(&mixed, "query.knowledge.find_person")["disposition"], "applied" @@ -1461,7 +1468,7 @@ policies: let mut sorted = order.clone(); sorted.sort_unstable(); assert_eq!(order, sorted, "{mixed}"); - // Graph deletion cannot converge until stage 4C's approval artifacts. + // Conclusion (approve + converge) extends below once the delete executor lands. } /// Stage 4A headline: a declared graph is created by `cluster apply` itself — diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 11ebcd9..2fa0eab 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -26,6 +26,7 @@ pub const CLUSTER_STATE_FILE: &str = "__cluster/state.json"; pub const CLUSTER_LOCK_FILE: &str = "__cluster/lock.json"; pub const CLUSTER_RESOURCES_DIR: &str = "__cluster/resources"; pub const CLUSTER_RECOVERIES_DIR: &str = "__cluster/recoveries"; +pub const CLUSTER_APPROVALS_DIR: &str = "__cluster/approvals"; #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] #[serde(rename_all = "snake_case")] @@ -212,6 +213,9 @@ pub struct BlastRadius { pub struct ApprovalRequirement { pub resource: String, pub reason: String, + /// True when a valid (digest-matching, unconsumed) approval artifact is + /// pending for this change. + pub satisfied: bool, } #[derive(Debug, Clone, Serialize)] @@ -293,6 +297,47 @@ pub struct ApplyOutput { pub diagnostics: Vec<Diagnostic>, } +/// A digest-bound human approval for an irreversible operation (RFC-004 +/// §D4). Written by `cluster approve`, consumed by apply. The file is never +/// deleted on consumption — it is rewritten with `consumed_at` and also +/// summarized into the state ledger's `approval_records`, so the audit fact +/// survives the loss of either store (axiom 11). +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +struct ApprovalArtifact { + schema_version: u32, + approval_id: String, + resource: String, + operation: String, + reason: String, + bound_config_digest: String, + #[serde(default)] + bound_before_digest: Option<String>, + #[serde(default)] + bound_after_digest: Option<String>, + approved_by: String, + created_at: String, + #[serde(default)] + consumed_at: Option<String>, + #[serde(default)] + consumed_by_operation: Option<String>, +} + +#[derive(Debug, Clone, Serialize)] +pub struct ApproveOutput { + pub ok: bool, + pub config_dir: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub approval_id: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub resource: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub operation: Option<PlanOperation>, + #[serde(skip_serializing_if = "Option::is_none")] + pub approved_by: Option<String>, + pub diagnostics: Vec<Diagnostic>, +} + #[derive(Debug, Clone)] struct DesiredCluster { config_dir: PathBuf, @@ -472,6 +517,7 @@ struct LocalStateBackend { state_path: PathBuf, lock_path: PathBuf, recoveries_dir: PathBuf, + approvals_dir: PathBuf, } #[derive(Debug)] @@ -588,7 +634,14 @@ pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { }; // Plan previews dispositions without sweeping; a pending recovery is // surfaced as the cluster_recovery_pending warning above instead. - classify_changes(&mut changes, &desired.dependencies, &BTreeSet::new()); + let artifacts = backend.list_approval_artifacts(&mut diagnostics); + let approved = approved_resources( + &artifacts, + &changes, + &desired.config_digest, + &mut diagnostics, + ); + classify_changes(&mut changes, &desired.dependencies, &BTreeSet::new(), &approved); // Embed real migration steps for schema updates so plan is a data-aware // preview; failures degrade to the digest diff with a warning. @@ -624,7 +677,7 @@ pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { } } let blast_radius = compute_blast_radius(&changes, &desired.dependencies); - let approvals_required = compute_approvals(&changes); + let approvals_required = compute_approvals(&changes, &approved); let ok = !has_errors(&diagnostics); PlanOutput { @@ -790,17 +843,29 @@ pub async fn apply_config_dir_with_options( let prior_resources = state_resource_digests(&state); let mut changes = diff_resources(&prior_resources, &desired.resource_digests); - classify_changes(&mut changes, &desired.dependencies, &sweep.pending_graphs); + let approval_artifacts = backend.list_approval_artifacts(&mut diagnostics); + let approved = approved_resources( + &approval_artifacts, + &changes, + &desired.config_digest, + &mut diagnostics, + ); + classify_changes( + &mut changes, + &desired.dependencies, + &sweep.pending_graphs, + &approved, + ); - // Defensive invariant: nothing the approval gate covers may be executable. - // Today approvals only cover graph/schema deletes (always deferred); this - // keeps a future widening of the executable set from silently bypassing it. - let approvals = compute_approvals(&changes); + // Defensive invariant: nothing the approval gate covers may be executable + // WITHOUT a matching approval. Gated changes with a valid artifact are the + // sanctioned exception (stage 4C). + let approvals = compute_approvals(&changes, &approved); let approval_violation = changes.iter().any(|change| { change.disposition == Some(ApplyDisposition::Applied) && approvals .iter() - .any(|approval| approval.resource == change.resource) + .any(|approval| approval.resource == change.resource && !approval.satisfied) }); if approval_violation { diagnostics.push(Diagnostic::error( @@ -1349,6 +1414,126 @@ pub async fn apply_config_dir_with_options( } } +/// Record a digest-bound human approval for a gated (irreversible) change — +/// today: graph deletes. The artifact binds to the exact desired config +/// digest and the change's before/after digests, so config or state drift +/// invalidates it automatically (a stale approval can never authorize a +/// different change). +pub async fn approve_config_dir( + config_dir: impl AsRef<Path>, + resource: &str, + approved_by: &str, +) -> ApproveOutput { + let outcome = load_desired(config_dir.as_ref()); + let mut diagnostics = outcome.diagnostics; + let backend = LocalStateBackend::new(&outcome.config_dir); + let mut observations = backend.observations(); + + let fail = |config_dir: String, diagnostics: Vec<Diagnostic>| ApproveOutput { + ok: false, + config_dir, + approval_id: None, + resource: None, + operation: None, + approved_by: None, + diagnostics, + }; + + let Some(desired) = outcome.desired else { + return fail(display_path(&outcome.config_dir), diagnostics); + }; + if has_errors(&diagnostics) { + return fail(display_path(&desired.config_dir), diagnostics); + } + + let _lock_guard = if desired.state_lock { + match backend.acquire_lock("approve", &mut observations) { + Ok(guard) => Some(guard), + Err(diagnostic) => { + diagnostics.push(diagnostic); + return fail(display_path(&desired.config_dir), diagnostics); + } + } + } else { + diagnostics.push(Diagnostic::warning( + "state_lock_disabled", + "state.lock", + "state.lock is false; approve ran without acquiring the cluster state lock", + )); + None + }; + + let state = match backend.read_state(&mut observations) { + Ok(snapshot) => match snapshot.state { + Some(state) => state, + None => { + diagnostics.push(Diagnostic::error( + "state_missing", + CLUSTER_STATE_FILE, + "approve requires an existing state.json; run `cluster import` first", + )); + return fail(display_path(&desired.config_dir), diagnostics); + } + }, + Err(diagnostic) => { + diagnostics.push(diagnostic); + return fail(display_path(&desired.config_dir), diagnostics); + } + }; + + let prior_resources = state_resource_digests(&state); + let changes = diff_resources(&prior_resources, &desired.resource_digests); + let gates = compute_approvals(&changes, &BTreeSet::new()); + let Some(change) = changes.iter().find(|change| { + change.resource == resource && gates.iter().any(|gate| gate.resource == resource) + }) else { + diagnostics.push(Diagnostic::error( + "approval_not_required", + resource, + "no pending change for this resource requires approval (check `cluster plan`)", + )); + return fail(display_path(&desired.config_dir), diagnostics); + }; + + let artifact = ApprovalArtifact { + schema_version: 1, + approval_id: Ulid::new().to_string(), + resource: change.resource.clone(), + operation: match change.operation { + PlanOperation::Create => "create", + PlanOperation::Update => "update", + PlanOperation::Delete => "delete", + } + .to_string(), + reason: gates + .iter() + .find(|gate| gate.resource == resource) + .map(|gate| gate.reason.clone()) + .unwrap_or_default(), + bound_config_digest: desired.config_digest.clone(), + bound_before_digest: change.before_digest.clone(), + bound_after_digest: change.after_digest.clone(), + approved_by: approved_by.to_string(), + created_at: now_rfc3339(), + consumed_at: None, + consumed_by_operation: None, + }; + if let Err(diagnostic) = backend.write_approval_artifact(&artifact) { + diagnostics.push(diagnostic); + return fail(display_path(&desired.config_dir), diagnostics); + } + + ApproveOutput { + ok: !has_errors(&diagnostics), + config_dir: display_path(&desired.config_dir), + approval_id: Some(artifact.approval_id), + resource: Some(artifact.resource), + operation: Some(change.operation.clone()), + approved_by: Some(artifact.approved_by), + diagnostics, + } +} + pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { let parsed = parse_cluster_config(config_dir.as_ref()); let mut diagnostics = parsed.diagnostics; @@ -1773,10 +1958,102 @@ impl LocalStateBackend { state_path: config_dir.join(CLUSTER_STATE_FILE), lock_path: config_dir.join(CLUSTER_LOCK_FILE), recoveries_dir: config_dir.join(CLUSTER_RECOVERIES_DIR), + approvals_dir: config_dir.join(CLUSTER_APPROVALS_DIR), state_dir, } } + /// List approval artifacts in ULID (filename) order; unparseable files + /// warn and stay on disk for the operator. + fn list_approval_artifacts( + &self, + diagnostics: &mut Vec<Diagnostic>, + ) -> Vec<(PathBuf, ApprovalArtifact)> { + let mut paths = Vec::new(); + match fs::read_dir(&self.approvals_dir) { + Ok(entries) => { + for entry in entries.flatten() { + let path = entry.path(); + if path.extension().is_some_and(|ext| ext == "json") { + paths.push(path); + } + } + } + Err(err) if err.kind() == ErrorKind::NotFound => {} + Err(err) => diagnostics.push(Diagnostic::warning( + "approval_read_error", + CLUSTER_APPROVALS_DIR, + format!("could not list approval artifacts: {err}"), + )), + } + paths.sort(); + let mut artifacts = Vec::new(); + for path in paths { + match fs::read_to_string(&path) + .map_err(|err| err.to_string()) + .and_then(|text| { + serde_json::from_str::<ApprovalArtifact>(&text).map_err(|err| err.to_string()) + }) { + Ok(artifact) if artifact.schema_version == 1 => artifacts.push((path, artifact)), + Ok(artifact) => diagnostics.push(Diagnostic::warning( + "unsupported_approval_version", + display_path(&path), + format!( + "unsupported approval artifact version {}; leaving it in place", + artifact.schema_version + ), + )), + Err(err) => diagnostics.push(Diagnostic::warning( + "invalid_approval_artifact", + display_path(&path), + format!("could not parse approval artifact ({err}); leaving it in place"), + )), + } + } + artifacts + } + + /// Atomically write (or rewrite, e.g. on consumption) an approval artifact. + fn write_approval_artifact(&self, artifact: &ApprovalArtifact) -> Result<PathBuf, Diagnostic> { + fs::create_dir_all(&self.approvals_dir).map_err(|err| { + Diagnostic::error( + "approval_write_error", + CLUSTER_APPROVALS_DIR, + format!("could not create approvals directory: {err}"), + ) + })?; + let target = self + .approvals_dir + .join(format!("{}.json", artifact.approval_id)); + let mut payload = serde_json::to_string_pretty(artifact).map_err(|err| { + Diagnostic::error( + "approval_write_error", + display_path(&target), + format!("could not encode approval artifact: {err}"), + ) + })?; + payload.push('\n'); + let tmp_path = self + .approvals_dir + .join(format!("{}.json.tmp.{}", artifact.approval_id, Ulid::new())); + fs::write(&tmp_path, payload.as_bytes()).map_err(|err| { + Diagnostic::error( + "approval_write_error", + display_path(&tmp_path), + format!("could not write approval artifact: {err}"), + ) + })?; + if let Err(err) = fs::rename(&tmp_path, &target) { + let _ = fs::remove_file(&tmp_path); + return Err(Diagnostic::error( + "approval_write_error", + display_path(&target), + format!("could not move approval artifact into place: {err}"), + )); + } + Ok(target) + } + /// List recovery sidecars in ULID (filename) order. Unparseable files are /// reported as warnings and skipped — they stay on disk for the operator. fn list_recovery_sidecars( @@ -3127,24 +3404,74 @@ fn compute_blast_radius(changes: &[PlanChange], dependencies: &[Dependency]) -> .collect() } -fn compute_approvals(changes: &[PlanChange]) -> Vec<ApprovalRequirement> { +fn compute_approvals( + changes: &[PlanChange], + approved: &BTreeSet<String>, +) -> Vec<ApprovalRequirement> { + // One gate per subtree: the graph.<id> delete carries its schema and + // queries, so a schema delete whose graph is also deleted is not listed. + let graph_deletes: BTreeSet<String> = changes + .iter() + .filter(|change| change.operation == PlanOperation::Delete) + .filter_map(|change| change.resource.strip_prefix("graph.").map(str::to_string)) + .collect(); changes .iter() .filter_map(|change| { - if change.operation == PlanOperation::Delete - && (change.resource.starts_with("graph.") || change.resource.starts_with("schema.")) - { - Some(ApprovalRequirement { - resource: change.resource.clone(), - reason: "delete may remove deployed graph or schema definition".to_string(), - }) - } else { - None + if change.operation != PlanOperation::Delete { + return None; } + let gated = match resource_kind(&change.resource) { + ResourceKind::Graph(_) => true, + ResourceKind::Schema(graph) => !graph_deletes.contains(&graph), + _ => false, + }; + gated.then(|| ApprovalRequirement { + resource: change.resource.clone(), + reason: "delete may remove deployed graph or schema definition".to_string(), + satisfied: approved.contains(&change.resource), + }) }) .collect() } +/// Resources with a valid (digest-matching, unconsumed) pending approval. +/// Near-misses — an artifact for the same resource whose bound digests no +/// longer match — warn as `approval_stale` and never authorize anything. +fn approved_resources( + artifacts: &[(PathBuf, ApprovalArtifact)], + changes: &[PlanChange], + config_digest: &str, + diagnostics: &mut Vec<Diagnostic>, +) -> BTreeSet<String> { + let mut approved = BTreeSet::new(); + for change in changes { + let candidates: Vec<&ApprovalArtifact> = artifacts + .iter() + .map(|(_, artifact)| artifact) + .filter(|artifact| artifact.consumed_at.is_none() && artifact.resource == change.resource) + .collect(); + if candidates.is_empty() { + continue; + } + let matched = candidates.iter().any(|artifact| { + artifact.bound_config_digest == config_digest + && artifact.bound_before_digest == change.before_digest + && artifact.bound_after_digest == change.after_digest + }); + if matched { + approved.insert(change.resource.clone()); + } else { + diagnostics.push(Diagnostic::warning( + "approval_stale", + change.resource.clone(), + "an approval artifact exists but its bound digests no longer match the plan; re-run `cluster approve`", + )); + } + } + approved +} + #[derive(Debug, PartialEq, Eq)] enum ResourceKind { Graph(String), @@ -3182,6 +3509,7 @@ fn classify_changes( changes: &mut [PlanChange], dependencies: &[Dependency], pending_recovery: &BTreeSet<String>, + approved: &BTreeSet<String>, ) { let mut schema_creates = BTreeSet::new(); let mut schema_pending = BTreeSet::new(); @@ -3219,6 +3547,11 @@ fn classify_changes( schema_pending.insert(graph.clone()); } } + // Subtree deletes ride the approved graph delete. NOTE: execution lands + // with the delete executor; until then approved deletes stay blocked so a + // gate-only build can never strip state without removing the root. + let rides_approved_delete = |_graph: &str| false; + let _ = approved; for change in changes.iter_mut() { let (disposition, reason) = match resource_kind(&change.resource) { @@ -3238,6 +3571,15 @@ fn classify_changes( PlanOperation::Create | PlanOperation::Update => { (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) } + PlanOperation::Delete if graph_deletes.contains(&graph) => { + if rides_approved_delete(&graph) { + (ApplyDisposition::Applied, None) + } else if pending_recovery.contains(&graph) { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) + } else { + (ApplyDisposition::Blocked, Some("approval_required")) + } + } _ => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), }, ResourceKind::Graph(graph) => match change.operation { @@ -3251,15 +3593,26 @@ fn classify_changes( PlanOperation::Update if !schema_pending.contains(&graph) => { (ApplyDisposition::Derived, None) } + // Stage 4C: an approved graph delete executes (the + // irreversible tier — gated by a digest-bound artifact). + PlanOperation::Delete => { + if pending_recovery.contains(&graph) { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) + } else if rides_approved_delete(&graph) { + (ApplyDisposition::Applied, None) + } else { + (ApplyDisposition::Blocked, Some("approval_required")) + } + } _ => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), }, ResourceKind::Query { graph, .. } => match change.operation { PlanOperation::Delete => { - if graph_deletes.contains(&graph) { - ( - ApplyDisposition::Blocked, - Some("dependency_not_applied"), - ) + if rides_approved_delete(&graph) { + // Tombstoned with the approved graph delete. + (ApplyDisposition::Applied, None) + } else if graph_deletes.contains(&graph) { + (ApplyDisposition::Blocked, Some("approval_required")) } else { (ApplyDisposition::Applied, None) } @@ -5302,7 +5655,7 @@ graphs: } #[tokio::test] - async fn apply_does_not_delete_subtree_of_deleted_graph() { + async fn apply_blocks_graph_delete_without_approval() { let dir = fixture(); let desired = validate_config_dir(dir.path()); let schema_digest = desired @@ -5331,18 +5684,25 @@ graphs: .iter() .map(|change| (change.resource.as_str(), change)) .collect(); + // Stage 4C: deletes are gated, not deferred — every subtree change + // blocks on the single graph-level approval. assert_eq!( by_resource["graph.old"].disposition, - Some(ApplyDisposition::Deferred) - ); - assert_eq!( - by_resource["schema.old"].disposition, - Some(ApplyDisposition::Deferred) - ); - assert_eq!( - by_resource["query.old.q"].disposition, Some(ApplyDisposition::Blocked) ); + assert_eq!( + by_resource["graph.old"].reason.as_deref(), + Some("approval_required") + ); + assert_eq!( + by_resource["schema.old"].reason.as_deref(), + Some("approval_required") + ); + assert_eq!( + by_resource["query.old.q"].reason.as_deref(), + Some("approval_required") + ); + // State intact; nothing destroyed without the artifact. let state = read_state_json(dir.path()); let resources = &state["applied_revision"]["resources"]; assert_eq!(resources["graph.old"]["digest"], "3333"); @@ -5350,6 +5710,147 @@ graphs: assert_eq!(resources["query.old.q"]["digest"], "5555"); } + #[tokio::test] + async fn approve_writes_digest_bound_artifact() { + let dir = fixture(); + write_applyable_state(dir.path()); + // Seed a deletable subtree. + let state = read_state_json(dir.path()); + let graph_digest_str = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + let schema_digest_str = state["applied_revision"]["resources"]["schema.knowledge"] + ["digest"] + .as_str() + .unwrap() + .to_string(); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", graph_digest_str.as_str()), + ("schema.knowledge", schema_digest_str.as_str()), + ("graph.old", "3333"), + ("schema.old", "4444"), + ], + ); + + let out = approve_config_dir(dir.path(), "graph.old", "andrew").await; + assert!(out.ok, "{:?}", out.diagnostics); + let approval_id = out.approval_id.clone().unwrap(); + let artifact: serde_json::Value = serde_json::from_str( + &fs::read_to_string( + dir.path() + .join(CLUSTER_APPROVALS_DIR) + .join(format!("{approval_id}.json")), + ) + .unwrap(), + ) + .unwrap(); + assert_eq!(artifact["resource"], "graph.old"); + assert_eq!(artifact["operation"], "delete"); + assert_eq!(artifact["approved_by"], "andrew"); + assert_eq!(artifact["bound_before_digest"], "3333"); + assert!(artifact["bound_after_digest"].is_null()); + assert!(artifact["bound_config_digest"].is_string()); + assert!(artifact["consumed_at"].is_null()); + + // A non-gated address is refused. + let not_gated = approve_config_dir(dir.path(), "query.knowledge.find_person", "andrew").await; + assert!(!not_gated.ok); + assert!( + not_gated + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "approval_not_required") + ); + } + + #[tokio::test] + async fn stale_approval_is_ignored() { + let dir = fixture(); + write_applyable_state(dir.path()); + let state = read_state_json(dir.path()); + let graph_digest_str = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + let schema_digest_str = state["applied_revision"]["resources"]["schema.knowledge"] + ["digest"] + .as_str() + .unwrap() + .to_string(); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", graph_digest_str.as_str()), + ("schema.knowledge", schema_digest_str.as_str()), + ("graph.old", "3333"), + ], + ); + let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; + assert!(approved.ok, "{:?}", approved.diagnostics); + // The config moves after approval: the bound config digest no longer + // matches and the artifact authorizes nothing. + fs::write(dir.path().join("base.policy.yaml"), "rules: [] # moved\n").unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "approval_stale"), + "{:?}", + out.diagnostics + ); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + assert_eq!( + by_resource["graph.old"].reason.as_deref(), + Some("approval_required") + ); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["graph.old"]["digest"], + "3333" + ); + } + + #[tokio::test] + async fn compute_approvals_one_gate_per_subtree() { + let dir = fixture(); + write_applyable_state(dir.path()); + let state = read_state_json(dir.path()); + let g = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + let sc = state["applied_revision"]["resources"]["schema.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", g.as_str()), + ("schema.knowledge", sc.as_str()), + ("graph.old", "3333"), + ("schema.old", "4444"), + ("query.old.q", "5555"), + ], + ); + let plan = plan_config_dir(dir.path()).await; + let gated: Vec<&str> = plan + .approvals_required + .iter() + .map(|gate| gate.resource.as_str()) + .collect(); + assert_eq!(gated, vec!["graph.old"], "{plan:?}"); + assert!(!plan.approvals_required[0].satisfied); + } + #[tokio::test] async fn apply_is_idempotent() { let dir = fixture(); From d1d04217ab114c2dc34577bad445f740b7f4881d Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 14:34:02 +0300 Subject: [PATCH 068/165] feat(cluster): execute approved graph deletes in cluster apply MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Stage 4C execution half (RFC-004 §D5/§D6 + sweep rows 7/7b/8): an approved graph.<id> delete — and its riding schema/query deletes — classifies Applied and executes LAST in the run, sidecar-fenced: pre-op manifest pin (best effort; partial roots still delete), approval_id carried in the sidecar, recursive root removal (NotFound tolerated), subtree tombstoned out of the ledger with a tombstone observation, the approval consumed in the same state CAS (ledger summary) and its artifact file rewritten with consumed_at only after the CAS lands — a failed run consumes nothing and the approval stays valid for the retry. Sweep rows: already-tombstoned intents retire (7); a completed delete with a stale ledger rolls forward — tombstone + approval consumption + audit entry (7b, idempotent); a still-present root retires the stale intent with a graph_delete_incomplete warning and the still-approved delete re-executes in the same run (8) — prefix removal is idempotent, so retry IS the repair. The multi-graph mixed e2e gets its conclusion: blocked without approval, cluster approve graph.engineering --as andrew, converge, tombstone visible in status. Phase 4's disposition matrix is now fully executable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/cli.rs | 52 ++- crates/omnigraph-cluster/src/lib.rs | 470 +++++++++++++++++++++++++++- 2 files changed, 513 insertions(+), 9 deletions(-) diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index bfa538d..336f19e 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -1358,7 +1358,7 @@ fn cluster_e2e_graph_root_destruction_drifts_then_apply_recreates_empty_graph() /// (applied), its composite (derived) — shows all four dispositions at once /// before the graph-plane schema apply closes the loop. #[test] -fn cluster_e2e_multi_graph_mixed_dispositions_then_converge() { +fn cluster_e2e_multi_graph_mixed_dispositions_then_approve_and_converge() { let temp = tempdir().unwrap(); write_multi_graph_cluster_fixture(temp.path()); // No manual init: Stage 4A creates both graphs. @@ -1468,7 +1468,55 @@ policies: let mut sorted = order.clone(); sorted.sort_unstable(); assert_eq!(order, sorted, "{mixed}"); - // Conclusion (approve + converge) extends below once the delete executor lands. + // The conclusion: an apply without approval stays blocked; the approved + // delete converges the cluster, tombstoning the removed graph. + let still_blocked = cluster_json(temp.path(), "apply"); + assert_eq!(still_blocked["converged"], false, "{still_blocked}"); + + let approve = parse_stdout_json(&output_success( + cli() + .arg("--as") + .arg("andrew") + .arg("cluster") + .arg("approve") + .arg("graph.engineering") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(approve["ok"], true, "{approve}"); + assert_eq!(approve["approved_by"], "andrew"); + + let converge = cluster_json(temp.path(), "apply"); + assert_eq!(converge["ok"], true, "{converge}"); + assert_eq!(converge["converged"], true, "{converge}"); + assert!(!temp.path().join("graphs/engineering.omni").exists()); + + let status = cluster_json(temp.path(), "status"); + assert_eq!(status["observations"]["graph.engineering"]["kind"], "tombstone"); + let final_plan = cluster_json(temp.path(), "plan"); + assert!( + final_plan["changes"].as_array().unwrap().is_empty(), + "{final_plan}" + ); +} + +/// An approval without an approver is meaningless: approve requires --as. +#[test] +fn cluster_e2e_approve_requires_actor() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let output = output_failure( + cli() + .arg("cluster") + .arg("approve") + .arg("graph.knowledge") + .arg("--config") + .arg(temp.path()), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stderr.contains("--as"), "{stderr}"); } /// Stage 4A headline: a declared graph is created by `cluster apply` itself — diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 2fa0eab..f67d8f7 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -491,6 +491,10 @@ struct RecoverySidecar { desired_schema_digest: String, #[serde(default)] state_cas_base: Option<String>, + /// For graph_delete: the approval this operation consumes; lets a sweep + /// roll-forward consume it too. + #[serde(default)] + approval_id: Option<String>, } #[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] @@ -498,7 +502,7 @@ struct RecoverySidecar { enum RecoverySidecarKind { GraphCreate, SchemaApply, - // GraphDelete arrives with stage 4C. + GraphDelete, } #[derive(Debug, Default)] @@ -509,6 +513,9 @@ struct SweepOutcome { /// Sidecars whose outcome is recorded (rows 2/4): deleted only after the /// command's state write lands, so a CAS failure re-sweeps them. completed_sidecars: Vec<PathBuf>, + /// Approval artifacts consumed by a roll-forward (delete row 7b): their + /// files are rewritten with consumed_at only after the state write lands. + consumed_approvals: Vec<String>, } #[derive(Debug)] @@ -942,6 +949,7 @@ pub async fn apply_config_dir_with_options( expected_manifest_version: None, desired_schema_digest: desired_graph.schema_digest.clone(), state_cas_base: expected_cas.clone(), + approval_id: None, }; let sidecar_path = match backend.write_recovery_sidecar(&sidecar) { Ok(path) => path, @@ -1105,6 +1113,7 @@ pub async fn apply_config_dir_with_options( expected_manifest_version: None, desired_schema_digest: desired_graph.schema_digest.clone(), state_cas_base: expected_cas.clone(), + approval_id: None, }; let sidecar_path = match backend.write_recovery_sidecar(&sidecar) { Ok(path) => path, @@ -1290,6 +1299,121 @@ pub async fn apply_config_dir_with_options( ); } + // Approved graph deletes execute LAST (RFC-004 §D5): catalog writes for + // surviving resources land first, then the irreversible work. + let graph_deletes_to_run: Vec<String> = changes + .iter() + .filter(|change| { + change.disposition == Some(ApplyDisposition::Applied) + && change.operation == PlanOperation::Delete + && matches!(resource_kind(&change.resource), ResourceKind::Graph(_)) + }) + .filter_map(|change| change.resource.strip_prefix("graph.").map(str::to_string)) + .collect(); + let mut executed_deletes: Vec<(String, Option<String>)> = Vec::new(); // (graph_id, approval_id) + let mut consumed_approval_ids: Vec<String> = Vec::new(); + for graph_id in &graph_deletes_to_run { + if graph_moving_aborted { + diagnostics.push(Diagnostic::warning( + "graph_delete_skipped", + graph_address(graph_id), + "skipped after an earlier graph-moving operation failed in this run", + )); + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::GraphDelete); + continue; + } + let graph_addr = graph_address(graph_id); + // Re-locate the consumable approval (classification verified one exists). + let approval_id = approval_artifacts + .iter() + .map(|(_, artifact)| artifact) + .find(|artifact| { + artifact.consumed_at.is_none() + && artifact.resource == graph_addr + && artifact.bound_config_digest == desired.config_digest + }) + .map(|artifact| artifact.approval_id.clone()); + let graph_uri = display_path( + &desired + .config_dir + .join(CLUSTER_GRAPHS_DIR) + .join(format!("{graph_id}.omni")), + ); + let observed_manifest_version = match Omnigraph::open_read_only(&graph_uri).await { + Ok(db) => match db.snapshot_of(ReadTarget::branch("main")).await { + Ok(snapshot) => Some(snapshot.version()), + Err(_) => None, + }, + Err(_) => None, // partial/unopenable roots still get deleted + }; + let sidecar = RecoverySidecar { + schema_version: 1, + operation_id: Ulid::new().to_string(), + started_at: now_rfc3339(), + actor: options.actor.clone(), + kind: RecoverySidecarKind::GraphDelete, + graph_id: graph_id.clone(), + graph_uri: graph_uri.clone(), + observed_manifest_version, + expected_manifest_version: None, // no post-op manifest exists + desired_schema_digest: String::new(), + state_cas_base: expected_cas.clone(), + approval_id: approval_id.clone(), + }; + let sidecar_path = match backend.write_recovery_sidecar(&sidecar) { + Ok(path) => path, + Err(diagnostic) => { + diagnostics.push(diagnostic); + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::GraphDelete); + graph_moving_aborted = true; + continue; + } + }; + if let Err(diagnostic) = failpoints::maybe_fail("cluster_apply.before_graph_delete") { + // Simulated crash before removal: row 8 retires the intent and + // the still-valid approval lets a later run retry. + diagnostics.push(diagnostic); + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::GraphDelete); + graph_moving_aborted = true; + continue; + } + match fs::remove_dir_all(PathBuf::from(&graph_uri)) { + Ok(()) => {} + Err(err) if err.kind() == ErrorKind::NotFound => {} // already gone + Err(err) => { + diagnostics.push(Diagnostic::error( + "graph_delete_failed", + graph_addr.clone(), + format!("could not remove graph root '{graph_uri}': {err}"), + )); + failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::GraphDelete); + graph_moving_aborted = true; + continue; + } + } + // Crash point: the root is gone, the ledger does not record it yet. + // The sweep rolls forward (row 7b) and consumes the approval. + if let Err(diagnostic) = failpoints::maybe_fail("cluster_apply.after_graph_delete") { + diagnostics.push(diagnostic); + return early_return( + display_path(&desired.config_dir), + Some(desired.config_digest), + observations, + changes, + state.resource_statuses, + diagnostics, + ); + } + executed_deletes.push((graph_id.clone(), approval_id.clone())); + if let Some(approval_id) = approval_id { + consumed_approval_ids.push(approval_id); + } + completed_op_sidecars.push(sidecar_path); + } + if !failed_graphs.is_empty() { + demote_dependents_of_failed_graphs(&mut changes, &failed_graphs, &desired.dependencies); + } + // State mutation. Apply owns query/policy statuses only; graph/schema // statuses belong to refresh/import observation and must not be clobbered // (the sweep above is the one exception: it owns recovery statuses). @@ -1330,6 +1454,17 @@ pub async fn apply_config_dir_with_options( _ => {} } } + for (graph_id, approval_id) in &executed_deletes { + tombstone_graph_subtree( + &mut new_state, + graph_id, + approval_id.as_deref(), + options.actor.as_deref(), + ); + if let Some(approval_id) = approval_id { + record_approval_consumed(&mut new_state, approval_id, "apply"); + } + } recompute_state_graph_digests(&mut new_state, &desired); let residual = diff_resources( @@ -1371,6 +1506,9 @@ pub async fn apply_config_dir_with_options( { let _ = fs::remove_file(sidecar_path); } + let mut all_consumed = sweep.consumed_approvals.clone(); + all_consumed.extend(consumed_approval_ids.iter().cloned()); + mark_approvals_consumed(&backend, &all_consumed); } // On a failed state write, report the statuses that are actually on disk // (the pre-apply snapshot), not the in-memory mutations that were never @@ -1827,6 +1965,7 @@ async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> St for sidecar_path in &sweep.completed_sidecars { let _ = fs::remove_file(sidecar_path); } + mark_approvals_consumed(&backend, &sweep.consumed_approvals); } Err(diagnostic) => diagnostics.push(diagnostic), } @@ -2558,6 +2697,9 @@ async fn sweep_recovery_sidecars( RecoverySidecarKind::SchemaApply => { sweep_schema_apply_sidecar(path, sidecar, state, diagnostics, &mut outcome).await; } + RecoverySidecarKind::GraphDelete => { + sweep_graph_delete_sidecar(path, sidecar, state, diagnostics, &mut outcome); + } } } outcome @@ -2778,6 +2920,121 @@ async fn sweep_schema_apply_sidecar( } } +fn sweep_graph_delete_sidecar( + path: PathBuf, + sidecar: RecoverySidecar, + state: &mut ClusterState, + diagnostics: &mut Vec<Diagnostic>, + outcome: &mut SweepOutcome, +) { + let graph_address = graph_address(&sidecar.graph_id); + let root = PathBuf::from(&sidecar.graph_uri); + + if root.exists() { + // Row 8: the delete never completed. Prefix removal is idempotent and + // works on partial roots, so the repair is simply the re-proposed, + // still-approved delete on a later run — retire the stale intent. + diagnostics.push(Diagnostic::warning( + "graph_delete_incomplete", + graph_address, + "a previous graph delete did not complete; it will be re-proposed by plan and can be retried under its approval", + )); + outcome.completed_sidecars.push(path); + return; + } + + if !state.applied_revision.resources.contains_key(&graph_address) { + // Row 7: already tombstoned (or never recorded); crash fell between + // the state CAS and sidecar delete. + outcome.completed_sidecars.push(path); + return; + } + + // Row 7b: the root is gone, the ledger is stale — roll forward the + // tombstone, consume the approval the sidecar carries, audit. + tombstone_graph_subtree(state, &sidecar.graph_id, sidecar.approval_id.as_deref(), sidecar.actor.as_deref()); + state.recovery_records.insert( + sidecar.operation_id.clone(), + json!({ + "kind": "graph_delete", + "graph_id": sidecar.graph_id, + "outcome": "rolled_forward", + "recovered_at": now_rfc3339(), + "actor": sidecar.actor, + }), + ); + if let Some(approval_id) = &sidecar.approval_id { + record_approval_consumed(state, approval_id, &sidecar.operation_id); + outcome.consumed_approvals.push(approval_id.clone()); + } + diagnostics.push(Diagnostic::warning( + "cluster_recovery_rolled_forward", + graph_address, + "an interrupted graph delete had completed on disk; cluster state was rolled forward to match", + )); + outcome.completed_sidecars.push(path); +} + +/// Remove a graph's subtree (graph, schema, queries) from the ledger and +/// leave a tombstone observation. Idempotent. +fn tombstone_graph_subtree( + state: &mut ClusterState, + graph_id: &str, + approval_id: Option<&str>, + actor: Option<&str>, +) { + let graph_addr = graph_address(graph_id); + let schema_addr = schema_address(graph_id); + let query_prefix = format!("query.{graph_id}."); + state.applied_revision.resources.remove(&graph_addr); + state.applied_revision.resources.remove(&schema_addr); + state + .applied_revision + .resources + .retain(|address, _| !address.starts_with(&query_prefix)); + state.resource_statuses.remove(&graph_addr); + state.resource_statuses.remove(&schema_addr); + state + .resource_statuses + .retain(|address, _| !address.starts_with(&query_prefix)); + state.observations.insert( + graph_addr, + json!({ + "kind": "tombstone", + "deleted_at": now_rfc3339(), + "approval_id": approval_id, + "actor": actor, + }), + ); +} + +/// Record approval consumption in the state ledger. The artifact FILE is +/// rewritten with consumed_at only after the state write lands, so a failed +/// CAS leaves the approval valid for the retry. +fn record_approval_consumed(state: &mut ClusterState, approval_id: &str, operation_id: &str) { + state.approval_records.insert( + approval_id.to_string(), + json!({ + "consumed_at": now_rfc3339(), + "consumed_by_operation": operation_id, + }), + ); +} + +/// Mark approval artifact files consumed on disk (post-CAS). +fn mark_approvals_consumed(backend: &LocalStateBackend, approval_ids: &[String]) { + if approval_ids.is_empty() { + return; + } + let mut sink = Vec::new(); + for (_, mut artifact) in backend.list_approval_artifacts(&mut sink) { + if approval_ids.contains(&artifact.approval_id) && artifact.consumed_at.is_none() { + artifact.consumed_at = Some(now_rfc3339()); + let _ = backend.write_approval_artifact(&artifact); + } + } +} + /// Read-only commands report pending sidecars without acting on them. fn warn_pending_recovery_sidecars(config_dir: &Path, diagnostics: &mut Vec<Diagnostic>) { let recoveries_dir = config_dir.join(CLUSTER_RECOVERIES_DIR); @@ -3547,11 +3804,12 @@ fn classify_changes( schema_pending.insert(graph.clone()); } } - // Subtree deletes ride the approved graph delete. NOTE: execution lands - // with the delete executor; until then approved deletes stay blocked so a - // gate-only build can never strip state without removing the root. - let rides_approved_delete = |_graph: &str| false; - let _ = approved; + // Subtree deletes ride the approved graph delete. + let rides_approved_delete = |graph: &str| { + graph_deletes.contains(graph) + && approved.contains(&graph_address(graph)) + && !pending_recovery.contains(graph) + }; for change in changes.iter_mut() { let (disposition, reason) = match resource_kind(&change.resource) { @@ -3662,6 +3920,7 @@ fn classify_changes( enum FailedGraphOrigin { GraphCreate, SchemaApply, + GraphDelete, } /// After a graph-moving operation fails mid-run, every change that depended @@ -3681,12 +3940,15 @@ fn demote_dependents_of_failed_graphs( let demote_reason = match resource_kind(&change.resource) { ResourceKind::Graph(graph) => match failed.get(&graph) { Some(FailedGraphOrigin::GraphCreate) => Some("graph_create_failed"), + Some(FailedGraphOrigin::GraphDelete) => Some("graph_delete_failed"), Some(FailedGraphOrigin::SchemaApply) => Some("dependency_not_applied"), None => None, }, ResourceKind::Schema(graph) => match failed.get(&graph) { Some(FailedGraphOrigin::SchemaApply) => Some("schema_apply_failed"), - Some(FailedGraphOrigin::GraphCreate) => Some("dependency_not_applied"), + Some(FailedGraphOrigin::GraphCreate) | Some(FailedGraphOrigin::GraphDelete) => { + Some("dependency_not_applied") + } None => None, }, ResourceKind::Query { graph, .. } if failed.contains_key(&graph) => { @@ -6606,6 +6868,200 @@ graphs: assert!(sidecar.exists()); } + /// Seed: converged knowledge subtree + a stale `old` graph subtree with a + /// real directory on disk. + fn seed_deletable_state(config_dir: &Path) { + write_applyable_state(config_dir); + let state = read_state_json(config_dir); + let g = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + let sc = state["applied_revision"]["resources"]["schema.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + write_state_resources( + config_dir, + &[ + ("graph.knowledge", g.as_str()), + ("schema.knowledge", sc.as_str()), + ("graph.old", "3333"), + ("schema.old", "4444"), + ("query.old.q", "5555"), + ], + ); + let root = config_dir.join(CLUSTER_GRAPHS_DIR).join("old.omni"); + fs::create_dir_all(&root).unwrap(); + fs::write(root.join("_schema.pg"), "stale").unwrap(); + } + + #[tokio::test] + async fn apply_executes_approved_graph_delete() { + let dir = fixture(); + seed_deletable_state(dir.path()); + let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; + assert!(approved.ok, "{:?}", approved.diagnostics); + let approval_id = approved.approval_id.clone().unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.converged, "{out:?}"); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + assert_eq!(by_resource["graph.old"].disposition, Some(ApplyDisposition::Applied)); + assert_eq!(by_resource["schema.old"].disposition, Some(ApplyDisposition::Applied)); + assert_eq!(by_resource["query.old.q"].disposition, Some(ApplyDisposition::Applied)); + // The root is gone; the subtree is tombstoned out of the ledger. + assert!(!dir.path().join(CLUSTER_GRAPHS_DIR).join("old.omni").exists()); + let state = read_state_json(dir.path()); + let resources = state["applied_revision"]["resources"].as_object().unwrap(); + assert!(!resources.contains_key("graph.old")); + assert!(!resources.contains_key("schema.old")); + assert!(!resources.contains_key("query.old.q")); + assert_eq!(state["observations"]["graph.old"]["kind"], "tombstone"); + assert_eq!(state["observations"]["graph.old"]["approval_id"], approval_id); + // Approval consumed in BOTH stores: ledger summary + artifact file. + assert!(state["approval_records"][&approval_id]["consumed_at"].is_string()); + let artifact: serde_json::Value = serde_json::from_str( + &fs::read_to_string( + dir.path() + .join(CLUSTER_APPROVALS_DIR) + .join(format!("{approval_id}.json")), + ) + .unwrap(), + ) + .unwrap(); + assert!(artifact["consumed_at"].is_string(), "{artifact}"); + // Sidecar retired. + assert!( + fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) + .map(|mut entries| entries.next().is_none()) + .unwrap_or(true) + ); + // A consumed approval authorizes nothing further (idempotent re-apply). + let again = apply_config_dir(dir.path()).await; + assert!(again.ok && again.converged && !again.state_written, "{again:?}"); + } + + fn write_delete_sidecar( + config_dir: &Path, + graph_id: &str, + approval_id: Option<&str>, + operation_id: &str, + ) -> PathBuf { + let dir = config_dir.join(CLUSTER_RECOVERIES_DIR); + fs::create_dir_all(&dir).unwrap(); + let path = dir.join(format!("{operation_id}.json")); + fs::write( + &path, + serde_json::to_string_pretty(&json!({ + "schema_version": 1, + "operation_id": operation_id, + "started_at": "1970-01-01T00:00:00Z", + "kind": "graph_delete", + "graph_id": graph_id, + "graph_uri": derived_graph_uri(config_dir, graph_id), + "desired_schema_digest": "", + "approval_id": approval_id, + })) + .unwrap(), + ) + .unwrap(); + path + } + + #[tokio::test] + async fn sweep_retires_delete_sidecar_when_tombstoned() { + let dir = fixture(); + write_applyable_state(dir.path()); // no graph.old in state, no root + let sidecar = write_delete_sidecar(dir.path(), "old", None, "01DROW7"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!sidecar.exists()); + let state = read_state_json(dir.path()); + assert!( + state["recovery_records"] + .as_object() + .is_none_or(|records| records.is_empty()) + ); + } + + #[tokio::test] + async fn sweep_rolls_forward_completed_delete() { + let dir = fixture(); + seed_deletable_state(dir.path()); + // Approve, then simulate: root removed, state stale, sidecar present. + let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; + let approval_id = approved.approval_id.unwrap(); + fs::remove_dir_all(dir.path().join(CLUSTER_GRAPHS_DIR).join("old.omni")).unwrap(); + let sidecar = write_delete_sidecar(dir.path(), "old", Some(&approval_id), "01DROW7B"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") + ); + assert!(!sidecar.exists()); + let state = read_state_json(dir.path()); + assert!( + !state["applied_revision"]["resources"] + .as_object() + .unwrap() + .contains_key("graph.old") + ); + assert_eq!(state["observations"]["graph.old"]["kind"], "tombstone"); + assert!(state["approval_records"][&approval_id]["consumed_at"].is_string()); + assert!( + state["recovery_records"] + .as_object() + .unwrap() + .values() + .any(|record| record["kind"] == "graph_delete" + && record["outcome"] == "rolled_forward") + ); + // The artifact file is marked consumed post-CAS. + let artifact: serde_json::Value = serde_json::from_str( + &fs::read_to_string( + dir.path() + .join(CLUSTER_APPROVALS_DIR) + .join(format!("{approval_id}.json")), + ) + .unwrap(), + ) + .unwrap(); + assert!(artifact["consumed_at"].is_string()); + assert!(out.converged, "{out:?}"); + } + + #[tokio::test] + async fn sweep_reproposes_incomplete_delete() { + let dir = fixture(); + seed_deletable_state(dir.path()); // root present + let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; + assert!(approved.ok); + let sidecar = write_delete_sidecar(dir.path(), "old", approved.approval_id.as_deref(), "01DROW8"); + + // Row 8: the stale intent is retired with a warning, and the same run + // re-executes the still-approved delete to completion. + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "graph_delete_incomplete") + ); + assert!(!sidecar.exists()); + assert!(!dir.path().join(CLUSTER_GRAPHS_DIR).join("old.omni").exists()); + assert!(out.converged, "{out:?}"); + } + #[test] fn status_warns_on_pending_recovery_sidecar() { let dir = fixture(); From 87691fe9c76b9b4b4cd0e6cbb8cf0279144b977c Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 14:34:54 +0300 Subject: [PATCH 069/165] test(cluster): failpoint coverage for delete crash windows - Crash before the removal: root intact, approval file unconsumed, sidecar survives, no ack; the next run retires the stale intent (row 8) and the still-approved delete completes in the same run. - Crash after the removal, before the state CAS: root gone, ledger byte-identical, the sidecar carries the approval id; the next run's sweep rolls the tombstone forward, consumes the approval, audits the recovery, and converges (row 7b). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/tests/failpoints.rs | 126 ++++++++++++++++++- 1 file changed, 125 insertions(+), 1 deletion(-) diff --git a/crates/omnigraph-cluster/tests/failpoints.rs b/crates/omnigraph-cluster/tests/failpoints.rs index cc91b85..5cdf2d4 100644 --- a/crates/omnigraph-cluster/tests/failpoints.rs +++ b/crates/omnigraph-cluster/tests/failpoints.rs @@ -16,7 +16,8 @@ use fail::FailScenario; use omnigraph_cluster::failpoints::ScopedFailPoint; use omnigraph::db::Omnigraph; use omnigraph_cluster::{ - ApplyOptions, apply_config_dir, apply_config_dir_with_options, validate_config_dir, + ApplyOptions, apply_config_dir, apply_config_dir_with_options, approve_config_dir, + validate_config_dir, }; use tempfile::tempdir; @@ -467,3 +468,126 @@ async fn schema_crash_after_apply_rolls_state_forward() { ); scenario.teardown(); } + +/// Seed: converged state + a stale `old` graph subtree with a real root and +/// a valid approval for its delete. Returns the approval id. +async fn seed_approved_delete(dir: &Path) -> String { + let digests = seed_applyable_state(dir); + let graph_digest = digests["graph.knowledge"].clone(); + let schema_digest = digests["schema.knowledge"].clone(); + let state_dir = dir.join("__cluster"); + fs::write( + state_dir.join("state.json"), + format!( + r#"{{ + "version": 1, + "state_revision": 1, + "applied_revision": {{ + "resources": {{ + "graph.knowledge": {{ "digest": "{graph_digest}" }}, + "schema.knowledge": {{ "digest": "{schema_digest}" }}, + "graph.old": {{ "digest": "3333" }}, + "schema.old": {{ "digest": "4444" }} + }} + }} +}} +"# + ), + ) + .unwrap(); + let root = dir.join("graphs/old.omni"); + fs::create_dir_all(&root).unwrap(); + fs::write(root.join("_schema.pg"), "stale").unwrap(); + let approved = approve_config_dir(dir, "graph.old", "test-actor").await; + assert!(approved.ok, "{:?}", approved.diagnostics); + approved.approval_id.unwrap() +} + +/// Crash before the removal: root intact, approval unconsumed, no ack; the +/// next run retires the stale intent (row 8) and the still-approved delete +/// completes in the same run. +#[tokio::test] +async fn delete_crash_before_removal_reproposes() { + let scenario = FailScenario::setup(); + let dir = fixture(); + let approval_id = seed_approved_delete(dir.path()).await; + + { + let _failpoint = ScopedFailPoint::new("cluster_apply.before_graph_delete", "return"); + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!(dir.path().join("graphs/old.omni").exists()); + assert_eq!(recovery_sidecars(dir.path()).len(), 1); + // The approval is untouched (file unconsumed). + let artifact: serde_json::Value = serde_json::from_str( + &fs::read_to_string( + dir.path() + .join("__cluster/approvals") + .join(format!("{approval_id}.json")), + ) + .unwrap(), + ) + .unwrap(); + assert!(artifact["consumed_at"].is_null()); + } + + let recovered = apply_config_dir(dir.path()).await; + assert!(recovered.ok, "{:?}", recovered.diagnostics); + assert!( + recovered + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "graph_delete_incomplete") + ); + assert!(recovered.converged); + assert!(!dir.path().join("graphs/old.omni").exists()); + assert!(recovery_sidecars(dir.path()).is_empty()); + scenario.teardown(); +} + +/// Crash after the removal, before the state CAS: root gone, ledger stale, +/// nothing acknowledged; the next run's sweep rolls the tombstone forward, +/// consumes the approval the sidecar carries, and audits the recovery. +#[tokio::test] +async fn delete_crash_after_removal_rolls_forward() { + let scenario = FailScenario::setup(); + let dir = fixture(); + let approval_id = seed_approved_delete(dir.path()).await; + let state_before = fs::read(state_path(dir.path())).unwrap(); + + { + let _failpoint = ScopedFailPoint::new("cluster_apply.after_graph_delete", "return"); + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!(!out.state_written); + assert!(!dir.path().join("graphs/old.omni").exists()); + assert_eq!(fs::read(state_path(dir.path())).unwrap(), state_before); + let sidecars = recovery_sidecars(dir.path()); + assert_eq!(sidecars.len(), 1); + let sidecar: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&sidecars[0]).unwrap()).unwrap(); + assert_eq!(sidecar["approval_id"], approval_id.as_str()); + } + + let recovered = apply_config_dir(dir.path()).await; + assert!(recovered.ok, "{:?}", recovered.diagnostics); + assert!( + recovered + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") + ); + assert!(recovered.converged); + let state: serde_json::Value = + serde_json::from_str(&fs::read_to_string(state_path(dir.path())).unwrap()).unwrap(); + assert_eq!(state["observations"]["graph.old"]["kind"], "tombstone"); + assert!(state["approval_records"][&approval_id]["consumed_at"].is_string()); + assert!( + state["recovery_records"] + .as_object() + .unwrap() + .values() + .any(|record| record["kind"] == "graph_delete") + ); + scenario.teardown(); +} From c949a2b71717f23b92d80be256a55c6e61d0fbf5 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 14:35:50 +0300 Subject: [PATCH 070/165] =?UTF-8?q?docs(cluster):=20document=20Stage=204C?= =?UTF-8?q?=20=E2=80=94=20Phase=204=20complete?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Approvals + gated graph deletion in the user docs, the approve command in the CLI reference, RFC-004 flipped to Landed with its three implementation deviations recorded (row-8 retire-and-repropose, --as instead of --actor/--by, consumed artifacts rewritten in place rather than moved). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- .../dev/rfc-004-cluster-graph-schema-apply.md | 3 +- docs/dev/testing.md | 2 +- docs/user/cli-reference.md | 3 +- docs/user/cluster-config.md | 45 ++++++++++++++++--- 4 files changed, 43 insertions(+), 10 deletions(-) diff --git a/docs/dev/rfc-004-cluster-graph-schema-apply.md b/docs/dev/rfc-004-cluster-graph-schema-apply.md index ca72fdc..e9c0336 100644 --- a/docs/dev/rfc-004-cluster-graph-schema-apply.md +++ b/docs/dev/rfc-004-cluster-graph-schema-apply.md @@ -1,6 +1,7 @@ # RFC: Cluster Graph & Schema Apply — Phase 4 of the Cluster Control Plane -**Status:** Proposed +**Status:** Landed (4A #170, 4B #171, 4C — all shipped) +**Implementation deviations:** (1) D3 row 8 retires the stale delete sidecar and lets the still-approved delete re-propose and retry, instead of a pending-block — prefix removal is idempotent, so the retry is the repair. (2) The approver/actor flag is the CLI's existing global `--as`, not a dedicated `--actor`/`--by`. (3) Consumed approval artifacts are rewritten with `consumed_at` rather than moved into state — the file and the ledger record both survive independently (axiom 11). **Date:** 2026-06-10 **Builds on:** cluster Stages 1–3B (shipped: validate/plan/status/refresh/import/force-unlock, config-only `cluster apply` with content-addressed catalog publish, catalog payload verification, failpoint-proven crash/CAS recovery for the apply protocol). Normative context: [cluster-config-specs.md](cluster-config-specs.md), [cluster-axioms.md](cluster-axioms.md), [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md). **Target release:** unversioned (phased — see Sequencing); no cluster functionality is in a tagged release yet. diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 5402ccf..2302b13 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,7 +8,7 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish; includes the `cluster_e2e_*` lifecycle compositions over the spawned binary — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | -| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), and Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows) | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), and Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows) | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 774ea6b..9dc8a25 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -19,7 +19,7 @@ Top-level command families and subcommands. Graph-targeting commands accept eith | `commit list \| show` | inspect commit graph | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | -| `cluster validate \| plan \| apply \| status \| refresh \| import \| force-unlock` | cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json` and annotates each change with its apply disposition; `apply` executes the config-only (stored-query/policy) subset into the content-addressed local catalog under `__cluster/resources/` — graph/schema changes are deferred loudly, and nothing applied serves traffic (the server still boots from `omnigraph.yaml`); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id. No graph-manifest movement, server change, automatic stale-lock breaking, or `plan --refresh` occurs in Stage 3A | +| `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json` and annotates each change with its apply disposition; `apply` executes the config-only (stored-query/policy) subset into the content-addressed local catalog under `__cluster/resources/` — graph/schema changes are deferred loudly, and nothing applied serves traffic (the server still boots from `omnigraph.yaml`); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id. No graph-manifest movement, server change, automatic stale-lock breaking, or `plan --refresh` occurs in Stage 3A | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) | | `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | @@ -79,6 +79,7 @@ policy: omnigraph cluster validate --config ./company-brain omnigraph cluster plan --config ./company-brain --json omnigraph cluster apply --config ./company-brain --json +omnigraph cluster approve graph.<id> --config ./company-brain --as <actor> omnigraph cluster status --config ./company-brain --json omnigraph cluster refresh --config ./company-brain --json omnigraph cluster import --config ./company-brain --json diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 9de305a..2df26be 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -1,6 +1,6 @@ # Cluster Config -**Status:** Stage 4B schema-apply preview. +**Status:** Stage 4C — Phase 4 complete (graph create, schema apply, gated graph delete). Cluster config is the future control-plane configuration surface for a whole OmniGraph deployment. In this stage, OmniGraph can validate a local @@ -9,11 +9,10 @@ local JSON state ledger, explicitly refresh/import graph observations into that ledger, manually remove a held local state lock by exact lock id, and **apply the executable subset of the plan** — stored-query and policy-bundle catalog writes, **graph creation** (a declared graph that does not exist yet -is initialized by apply at the derived root), and **schema updates**: a -changed schema is migrated on the live graph by apply itself, soft drops -only. It does not delete graphs (a later stage), perform data-loss -migrations, start servers, or serve anything it applies: the server still -boots from `omnigraph.yaml`. +is initialized by apply at the derived root), **schema updates** (soft drops +only), and — behind an explicit, digest-bound **approval** — **graph +deletion**. It does not perform data-loss schema migrations, start servers, +or serve anything it applies: the server still boots from `omnigraph.yaml`. ## Commands @@ -21,6 +20,7 @@ boots from `omnigraph.yaml`. omnigraph cluster validate --config ./company-brain omnigraph cluster plan --config ./company-brain --json omnigraph cluster apply --config ./company-brain --json +omnigraph cluster approve graph.<id> --config ./company-brain --as <actor> omnigraph cluster status --config ./company-brain --json omnigraph cluster refresh --config ./company-brain --json omnigraph cluster import --config ./company-brain --json @@ -253,7 +253,38 @@ in recovery sidecars and audit entries and threads it to the engine's schema-apply (so commit attribution and Cedar enforcement — wherever a policy checker is installed — work unchanged). -Schema deletes (removing a graph) are never executed by this stage. They are +### Approvals and graph deletion + +Deleting a graph is the irreversible tier: it requires a recorded human +decision. `cluster plan` lists the gate under `approvals_required` (one gate +per graph — the graph-level approval carries its schema and queries); +`cluster approve graph.<id> --as <actor>` writes a digest-bound artifact to + +```text +<config-dir>/__cluster/approvals/<approval-id>.json +``` + +bound to the exact desired config digest and the change's state digest, so +**any config or state drift after approving invalidates the artifact** +automatically (`approval_stale` warning; it never authorizes a different +change). An unapproved delete blocks with `approval_required`. + +An approved delete executes **last** in the apply run: the graph root is +removed recursively, the subtree (graph, schema, its queries) is tombstoned +out of the state ledger with a tombstone observation, and the approval is +consumed — recorded in the state's `approval_records` in the same state +update, and the artifact file rewritten with `consumed_at` (the file is never +deleted: the audit fact survives the loss of either store). A failed run +consumes nothing; the approval stays valid for the retry. Catalog blobs of +the deleted graph's queries stay on disk (GC is a later stage). + +Crash recovery for deletes: a completed-but-unrecorded delete is rolled +forward by the sweep (tombstone + approval consumption + audit entry); an +incomplete delete (root still present) is retired with a +`graph_delete_incomplete` warning and simply **re-proposed** — prefix removal +is idempotent, so the still-approved retry is the repair. + +Standalone schema deletes are never executed by this stage. They are reported as `deferred` (warning `apply_unsupported_change`), and query/policy changes that depend on them are `blocked` (warning `apply_dependency_blocked`, status `blocked` in state). A partially-applicable plan still exits 0 with warnings; From 61da7bf406d22e9e5af25bbef3560fe48c54f176 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 14:53:16 +0300 Subject: [PATCH 071/165] docs(cluster): descope ETL pipelines to a separate project; keep the socket (#172) Pipelines (scheduler, connectors, mapping, idempotency, run ledger) leave the cluster control-plane rollout and become their own project with their own RFC. This rollout guarantees only the socket, all of which already exists and is enforced: the pipelines: config field is reserved (typed future_phase_field rejection, test-covered), the pipeline.<name> typed address and Pipeline resource kind are reserved in the resource model, and axiom 13 fixes the contract any future implementation must satisfy (definition reconciled, execution data-plane, fan-out statusful). The ETL section in the high-level spec stands as the requirements record for that project; exit criterion 9 defers to its RFC. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --- .../dev/cluster-config-implementation-spec.md | 25 +++++++++++++------ docs/dev/cluster-config-specs.md | 15 +++++++++++ 2 files changed, 32 insertions(+), 8 deletions(-) diff --git a/docs/dev/cluster-config-implementation-spec.md b/docs/dev/cluster-config-implementation-spec.md index d4cf3e6..8917426 100644 --- a/docs/dev/cluster-config-implementation-spec.md +++ b/docs/dev/cluster-config-implementation-spec.md @@ -529,7 +529,7 @@ These are the concrete "what requires downstream" rules. | Server registry | Boot from cluster state, eventually reload/reconcile graph handles, expose statuses | High | Affects routing, OpenAPI, auth, and workload admission | | API types/OpenAPI | Plan/status/apply DTOs if HTTP management endpoints ship | Medium/high | OpenAPI drift must be regenerated | | UI specs | New renderer/spec validator/binding checker | High | New product surface, not currently implemented | -| Pipelines | New scheduler/runtime/connector/mapping/idempotency/run ledger | Very high | Second data-plane seam; large product and correctness surface | +| Pipelines | New scheduler/runtime/connector/mapping/idempotency/run ledger | Very high | **Separate project** (socket reserved here); second data-plane seam, large product and correctness surface | | Embeddings | Cluster-level defaults, env refs, model/dimension validation, index interaction | Medium | Existing embedding code is mostly offline/client-side | | Docs | User docs for cluster config, policy, server, CLI; dev docs for invariants/testing | High | Public contract changes | | Tests | New cluster suites plus extensions to config/server/policy/recovery/schema/query tests | High | Needs boundary-matched coverage | @@ -616,13 +616,22 @@ actor threading, 4A/4B/4C staging). docs and migrations say it can be narrowed. - Deprecate and later remove `mcp.expose` from target-state cluster config. -### Phase 7: Pipeline Runtime +### Pipelines: separate project (socket only) -- Add scheduler/worker/runtime. -- Add source connector contracts, mapping validation, idempotency keys, - per-target run status, and retry behavior. -- Treat fan-out execution as data-plane writes unless explicitly staged through - branch/merge. +Pipelines are **descoped from this rollout** (2026-06-10): the runtime +(scheduler/worker, connector contracts, mapping validation, idempotency keys, +per-target run status, retry behavior) is a separate project with its own +RFC. This rollout guarantees only the socket: + +- `pipelines:` stays a reserved config field, rejected with a typed + `future_phase_field` diagnostic (enforced + test-covered in + `omnigraph-cluster`). +- `pipeline.<name>` stays a reserved typed address; the resource model + (kind-agnostic state entries, extensible sidecar kinds, dependency edges) + accepts the new kind without reshaping. +- Axiom 13 is the contract the future implementation must satisfy: the + definition is reconciled, the execution is data-plane; fan-out is statusful, + never silently atomic. ## Test Ownership @@ -725,4 +734,4 @@ Before implementation begins beyond parser/validate, the RFC must answer: 6. Bootstrap authority and first-actor story. 7. Server startup and migration path from `omnigraph.yaml`. 8. Per-query policy schema and compatibility bridge for `mcp.expose`. -9. Pipeline runtime owner, status schema, and idempotency contract. +9. Pipeline runtime owner, status schema, and idempotency contract — **deferred to the separate pipelines project's own RFC**; this rollout only reserves the socket. diff --git a/docs/dev/cluster-config-specs.md b/docs/dev/cluster-config-specs.md index 8f36dc8..d248be2 100644 --- a/docs/dev/cluster-config-specs.md +++ b/docs/dev/cluster-config-specs.md @@ -178,6 +178,21 @@ but it remains a separate CAS step from graph manifest movement. --> ## ETL pipelines (the second data-plane seam) +> **Scope note (2026-06-10): descoped to a separate project.** Pipelines are +> a product surface of their own (scheduler, connectors, mapping language, +> idempotency, run ledger) and will be designed and built outside the cluster +> control-plane track. What this spec retains is the **socket** they plug +> into, which is already enforced: (1) the `pipelines:` config field is +> reserved — `cluster validate` rejects it with a typed `future_phase_field` +> diagnostic, so it can never be silently squatted; (2) the typed address +> form `pipeline.<name>` and the `Pipeline` resource kind are reserved in the +> resource model; (3) axiom 13 fixes the contract any future implementation +> must satisfy — the pipeline *definition* is a reconciled cluster resource, +> its *execution* is data-plane and never reconciled. The design text below +> stands as the requirements record for that project, not as a phase of this +> one. + + External data — from another database, an API, a file drop, a stream — is a first-class config asset, not glue code that lives nowhere. A **Pipeline** is declared in config: a **source** (e.g. `notion`, `github`, `slack`, `gdrive`, `postgres`, `http`, `s3-files`, `kafka`), an optional **schedule/trigger**, and **one or more target graphs**, each with its own **mapping/transform** (external records → graph types & properties). A single feed can **fan out across graphs** — e.g. a GitHub sync that populates both the `engineering` graph and the people/teams in `knowledge`. It is reconciled like any resource — `apply` creates / updates / deletes / (re)schedules the pipeline *definition*. This is the canonical "company brain" move: the deployment's graphs are continuously assembled from the SaaS tools the org already uses. From 3e8f1038046e9225902176527d887440c69d9f71 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 15:22:12 +0300 Subject: [PATCH 072/165] =?UTF-8?q?docs(cluster):=20RFC-005=20=E2=80=94=20?= =?UTF-8?q?server=20boots=20from=20cluster=20state=20(Phase=205=20design)?= =?UTF-8?q?=20(#174)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The axiom-15 mode switch: omnigraph-server --cluster <dir> (mutually exclusive with uri/--target/--config, zero omnigraph.yaml reads) serves the APPLIED revision — graph set from state, query/policy content from the content-addressed catalog at applied digests, cluster-scoped policy bundles as the server-level Cedar engine. The load-bearing finding: state is not yet serving-sufficient (policy applies_to bindings live only in cluster.yaml), so slice 5A records binding metadata into the applied revision at apply time — without it, boot-from-state silently becomes the merged read axiom 15 forbids. Fail-fast readiness table (missing state, pending sidecars, missing blobs, unbound policies all refuse boot with remedies), the expose-all mcp.expose bridge with its Phase 6 sunset, the operator migration path (exit criterion 7), and 5A/5B/5C sequencing. The existing boot pipeline (GraphStartupConfig -> registry -> routing/auth) is reused as-is — a new source, not a new pipeline. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --- .../dev/cluster-config-implementation-spec.md | 4 + docs/dev/index.md | 1 + docs/dev/rfc-005-server-cluster-boot.md | 139 ++++++++++++++++++ 3 files changed, 144 insertions(+) create mode 100644 docs/dev/rfc-005-server-cluster-boot.md diff --git a/docs/dev/cluster-config-implementation-spec.md b/docs/dev/cluster-config-implementation-spec.md index 8917426..b58e531 100644 --- a/docs/dev/cluster-config-implementation-spec.md +++ b/docs/dev/cluster-config-implementation-spec.md @@ -601,6 +601,10 @@ actor threading, 4A/4B/4C staging). ### Phase 5: Server Reads Cluster Catalog +Detailed design: [rfc-005-server-cluster-boot.md](rfc-005-server-cluster-boot.md) +(the --cluster mode switch, applied-revision serving, serving metadata in +state, readiness table, migration path). + - Allow server startup from cluster state. - Add status and catalog endpoints as needed. - Keep the current `omnigraph.yaml` startup path as compatibility mode — an diff --git a/docs/dev/index.md b/docs/dev/index.md index 827d99c..4bc1e6a 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -75,6 +75,7 @@ Working documents for in-flight feature work. Removed when the work lands. | MCP server surface — full tool parity, stored queries, modular auth (MR-969 / MR-956 / MR-974) | [rfc-003-mcp-server-surface.md](rfc-003-mcp-server-surface.md) | | Future cluster control plane — declarative as-code config, JSON state ledger, reconciler | [cluster-config-specs.md](cluster-config-specs.md), [cluster-axioms.md](cluster-axioms.md), [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md) | | Cluster graph & schema apply — Phase 4 sidecars, roll-forward recovery, approval artifacts | [rfc-004-cluster-graph-schema-apply.md](rfc-004-cluster-graph-schema-apply.md) | +| Server boots from cluster state — Phase 5 mode switch, applied-revision serving | [rfc-005-server-cluster-boot.md](rfc-005-server-cluster-boot.md) | ## Boundary diff --git a/docs/dev/rfc-005-server-cluster-boot.md b/docs/dev/rfc-005-server-cluster-boot.md new file mode 100644 index 0000000..81d5129 --- /dev/null +++ b/docs/dev/rfc-005-server-cluster-boot.md @@ -0,0 +1,139 @@ +# RFC: Server Boots from Cluster State — Phase 5 of the Cluster Control Plane + +**Status:** Proposed +**Date:** 2026-06-10 +**Builds on:** Phase 4 complete ([rfc-004-cluster-graph-schema-apply.md](rfc-004-cluster-graph-schema-apply.md), Landed): `cluster apply` converges graphs, schemas, stored queries, and policies into the cluster catalog. Normative context: [cluster-config-specs.md](cluster-config-specs.md) (the migration model's "window 2"), [cluster-axioms.md](cluster-axioms.md) (axiom 15), [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md) (Phase 5 rollout, Compatibility Stance #7–#9, exit criterion 7). +**Target release:** unversioned (phased — see Sequencing). + +## Summary + +Give `omnigraph-server` a second boot source: `omnigraph-server --cluster <dir>` reads its graph set, stored queries, and Cedar policies from the **cluster catalog** — `state.json`'s applied revision plus the content-addressed blobs under `__cluster/resources/` — instead of `omnigraph.yaml`. This is the moment "applied" finally means "serving": the standing caveat in every cluster doc since Stage 3A ("the server still boots from `omnigraph.yaml`") retires for deployments that flip the switch. + +Three commitments: + +1. **An exclusive mode switch, never a merge** (axiom 15, Compatibility Stance #7). `--cluster <dir>` is mutually exclusive with the positional URI, `--target`, and `--config`. In cluster mode, `omnigraph.yaml` is not read at all — not for graphs, not for queries, not for policies. There is no precedence, no key-level aliasing, no fallback read. A deployment serves from one source. +2. **The server serves the *applied* revision, not the desired config.** What's live is what `cluster apply` converged: graph roots recorded in state, query/policy content at the *applied* digests from the content-addressed catalog. Un-applied config drift never leaks into serving — the serving surface and the ledger cannot disagree (axiom 5 extended to the data path). +3. **The state ledger becomes serving-sufficient.** Today one fact needed to serve is missing from state: a policy's `applies_to` bindings live only in `cluster.yaml`. A prerequisite slice (5A) records binding metadata into the applied revision at apply time, so a booting server reads state + blobs and nothing else. Without this, "boot from state" would silently become "boot from state *and* config" — the merged read axiom 15 forbids. + +## Motivation + +Phase 4 closed the convergence loop but left it inert: an operator can declare, plan, approve, and apply an entire deployment, and the running server ignores all of it. The Sarah/Bob test still fails at the last step — Sarah's applied change is visible in `cluster status` but Bob's clients hit a server still wired to a hand-maintained `omnigraph.yaml`. Phase 5 makes the catalog the serving source, which is also the precondition for Phase 6 (policy-owned query exposure must filter a catalog the server actually reads). + +## Non-Goals + +- **Runtime reconciliation / hot reload.** Cluster-mode boot is static, exactly like today's boot: the server reads the applied revision once at startup; picking up a newer applied state means restarting the process. The registry's runtime-mutation seam (the test-only `insert()` + mutate `Mutex` in `registry.rs`) stays future-proofing for a later watch-and-reload slice, not this RFC. +- **Policy-owned query exposure** (Phase 6) — but this RFC defines the bridge it sunsets (§D5). +- **Remote cluster roots.** `--cluster <dir>` is a local directory in this phase, same as the `cluster` CLI commands; S3-hosted cluster roots arrive with external state backends. +- **Retiring `omnigraph.yaml` server boot.** It remains a fully supported mode indefinitely (Compatibility Stance #8: the file's job shrinks; the *server-role* keys become inert only for deployments that switch). +- **New management endpoints** (`/cluster/status` etc.) — noted as future work; this RFC changes the boot source, not the HTTP surface (beyond OpenAPI regen if anything shifts). + +## Background (verified against main) + +- **Server boot today** (`omnigraph-server/src/main.rs`, `lib.rs:891-1029`): `load_server_settings` applies a four-rule mode inference (positional URI / `--target` / `server.graph` → Single; `--config` + `graphs:` → Multi), builds `ServerConfigMode::{Single,Multi}` with per-graph `GraphStartupConfig {graph_id, uri, policy_file, queries}`, loads `QueryRegistry` from `.gq` files at settings time (identity-checked), type-checks queries at engine open (`validate_and_attach`), loads Cedar via `PolicyEngine::load_graph`/`load_server`, installs it with `with_policy`, and assembles `GraphRegistry::from_handles` (startup-only; lock-free `ArcSwap` reads). Bind address and bearer tokens come from flags/env, not from graph config. No reload machinery exists. +- **The catalog today** (`omnigraph-cluster`): `state.json` records `applied_revision.resources` (address → digest) for `graph.*`, `schema.*`, `query.<graph>.<name>`, `policy.<name>`, plus statuses, observations (incl. tombstones), approval and recovery records. Query/policy *content* lives content-addressed at `__cluster/resources/query/<graph>/<name>/<digest>.gq` and `policy/<name>/<digest>.yaml`. Graph roots are derived: `<dir>/graphs/<id>.omni`. +- **The gap**: state records a policy's *digest* only; `applies_to` (cluster vs graph refs) lives in `cluster.yaml`. Queries are fine — their graph binding is encoded in the address itself. + +## Design + +### D1. The mode switch + +New server flag: `omnigraph-server --cluster <dir>` (the directory containing `cluster.yaml`, `__cluster/`, and `graphs/`). Mutually exclusive — a hard startup error, not a precedence rule — with the positional URI, `--target`, and `--config`. `--bind`, `--unauthenticated`, and the bearer-token env vars keep working identically: listen address and credentials are **process-operational facts**, not cluster facts (they differ per replica/host and never belonged to the shared catalog; if a `serve:` section ever joins `cluster.yaml`, that's a separate proposal). + +Mode inference gains rule 0: `--cluster <dir>` → **Cluster mode**, which is always multi-graph routing (`/graphs/{graph_id}/...`), even for a single declared graph. No flat-route legacy surface in cluster mode — it's a new mode with no compatibility debt to carry. + +### D2. What the server reads (the applied revision, and only it) + +`load_server_settings` grows a cluster branch that reads, in order: + +1. `__cluster/state.json` — **missing state is a boot error** ("run `cluster import` + `cluster apply` first"). Pending recovery sidecars under `__cluster/recoveries/` are also a boot error (`cluster_recovery_pending`): a server must not start serving a ledger that a sweep is about to rewrite. +2. **Graph set** = state's `graph.<id>` resources (tombstoned graphs are absent by construction). Each graph's URI is the derived root `<dir>/graphs/<id>.omni`. A recorded graph whose root does not open is a boot error — same fail-fast posture as today's bad URI. +3. **Stored queries** = state's `query.<graph>.<name>` entries, content loaded from the catalog blob at the recorded digest. Blob-missing or digest-mismatched is a boot error (the catalog verification semantics from Stage 3B, applied at boot). Queries type-check at engine open exactly as today (`validate_and_attach` — unchanged). +4. **Policies** = state's `policy.<name>` entries, content from catalog blobs, bindings from the applied metadata of D3: bundles bound to `cluster` load as the server-level Cedar engine (`PolicyEngine::load_server`); bundles bound to graphs load per-graph (`PolicyEngine::load_graph`) and install via `with_policy` — the existing two-gate structure, unchanged. +5. `cluster.yaml` is parsed **only** to validate that the directory is a cluster root (and for nothing else — explicitly not for resource content; a divergence between desired config and applied state is *served as applied*, visible via `cluster plan`). + +Everything downstream of settings construction — `GraphStartupConfig`, parallel engine opens, `GraphRegistry::from_handles`, routing middleware, auth, workload admission, OpenAPI — is reused as-is. Cluster mode is a new *source* for the same boot pipeline, not a new pipeline. + +### D3. Prerequisite: serving metadata in the applied revision (slice 5A) + +State's `StateResource` records only a digest. To make the ledger serving-sufficient, `cluster apply` (and the sweep's roll-forwards) additionally record **binding metadata** for policy resources at apply time: + +```json +"applied_revision": { + "resources": { + "policy.base_rbac": { + "digest": "<sha256>", + "applies_to": ["cluster", "graph.knowledge"] + } + } +} +``` + +- Additive and optional (`#[serde(default)]`) — existing state files parse unchanged; a policy entry without `applies_to` (applied before 5A) is a **boot error in cluster mode** with the remedy "re-run `cluster apply`" (one apply rewrites the metadata; the digest needn't change — the metadata write is part of the state mutation, not the blob). +- `applies_to` is normalized to typed addresses (`cluster` | `graph.<id>`) at apply time, mirroring the validator's normalization. +- Queries need no equivalent: the address (`query.<graph>.<name>`) already carries the binding, and the registry key/symbol invariant is enforced at apply (validate) time. +- This is deliberately *applied* metadata, not config mirroring: if `cluster.yaml` changes a binding, the server keeps serving the old binding until `cluster apply` converges it — the same contract as every other resource. + +### D4. Readiness and failure posture + +Boot is fail-fast, matching the server's existing stance (bad policy YAML refuses boot): + +| Condition | Behavior | +|---|---| +| `state.json` missing / unparseable / unsupported version | boot error | +| pending recovery sidecars | boot error (run any state-mutating cluster command to sweep) | +| recorded graph root missing or unopenable | boot error | +| query/policy blob missing or digest-mismatched | boot error (run `cluster refresh` + `apply` to self-heal, then restart) | +| policy entry without `applies_to` metadata | boot error ("re-run cluster apply", D3) | +| stored query fails type-check against the live schema | boot error (existing `validate_and_attach` behavior) | +| state lock held | **not** an error — boot takes no lock; it reads a point-in-time snapshot of an immutable-once-written state file (the CAS discipline means a concurrent apply produces a *new* file atomically; the server reads whichever was current at open) | + +### D5. The `mcp.expose` bridge in cluster mode + +The cluster query registry has no `expose` flag by design (axiom 14: exposure is a policy decision — Phase 6). Until Phase 6 ships, cluster-mode servers list **all** stored queries in `GET /queries`. This is the documented bridge: *cluster mode = everything exposed; omnigraph.yaml mode = `mcp.expose` honored as today*. Its named sunset is Phase 6's policy-filtered catalog (Compatibility Stance #9). Invocation remains gated by the existing coarse `invoke_query` Cedar action in both modes. + +### D6. Migration path (exit criterion 7) + +For an operator running multi-graph from `omnigraph.yaml`: + +1. Author `cluster.yaml` declaring the same graphs/queries/policies; place existing graph roots under `<dir>/graphs/<id>.omni` (or start fresh). +2. `cluster import` (observes live graphs) → `cluster plan` → `cluster apply` (publishes queries/policies into the catalog; with 5A, records policy bindings). +3. Restart the server with `--cluster <dir>` instead of `--config omnigraph.yaml`. +4. `omnigraph.yaml`'s `graphs:`/`serve:`/`queries:`/`policy:` keys are now inert for this deployment; the file remains the CLI's per-operator config. + +Rollback is the same switch in reverse — nothing in cluster mode mutates `omnigraph.yaml` or the graphs in a way the yaml mode can't serve. + +### D7. Invariants and axioms check + +- *Axiom 15 / Stance #7*: exclusive flag, hard mutual-exclusion error, zero `omnigraph.yaml` reads in cluster mode — no fact has two readers. +- *Axiom 5*: the server serves deployed reality (applied digests), never desired intent; D3 keeps the ledger the single serving source. +- *Axiom 12*: boot reads without the lock but relies on the atomic-replace write discipline; it never writes state. +- *Axiom 14 / Stance #9*: the expose-all bridge is named, scoped to cluster mode, and carries its Phase 6 sunset. +- *Loud failures (deny-list)*: every degraded condition is a typed boot error with a remedy; no partial serving, no silent fallback to the yaml. +- *Respect the boundaries*: `omnigraph-cluster` stays free of HTTP; the server reads the catalog through a small read-only loader (either a `pub` read surface on `omnigraph-cluster` or a thin module in the server consuming the documented file formats — implementation picks the one that keeps `omnigraph-cluster` dependency-light; the state/blob formats are already a documented contract). + +## Sequencing + +| Slice | Scope | Gate | +|---|---|---| +| **5A: serving metadata in state** | `applies_to` recorded on policy resources at apply + sweep roll-forward; additive state schema; `status`/plan surfacing | In-crate tests: metadata written/rolled-forward; old state parses; re-apply backfills | +| **5B: `--cluster` boot mode** | Flag + mode inference rule 0; catalog loader (state → `GraphStartupConfig`s + registries + policy engines); readiness table; OpenAPI regen if surface shifts | Server tests: boot from a converged fixture dir, serve `/graphs/{id}/query` + stored queries + Cedar gates; every D4 row refuses boot; e2e: `cluster apply` then serve — "applied means serving" | +| **5C: docs + caveat retirement** | `cluster-config.md` mode-switch section; `server.md`/`deployment.md`; retire the "not serving" caveats for cluster-mode deployments; migration guide (D6) | `check-agents-md.sh`; doc accuracy review | + +## Exit-criteria coverage + +Answers implementation-spec exit criterion 7 (server startup + migration path) in full; touches 1 (state schema gains policy binding metadata — additive). Criteria 8 (per-query policy) and 9 (pipelines — descoped to a separate project) remain. + +## Open Questions + +1. **Loader home**: `pub` read-only API on `omnigraph-cluster` (server gains the dependency) vs a server-side reader of the documented formats. Leaning `omnigraph-cluster` API — one parser for the state schema beats two drifting ones; the crate stays HTTP-free either way. +2. **Boot-time blob re-hash**: D4 requires digest verification at boot; for large catalogs a stat-only fast path with full hashes behind a flag may matter later. Start with full verification (catalogs are small). +3. **`GET /graphs` enrichment**: cluster mode could expose applied digests/revision in the enumeration — deferred until a consumer exists. +4. **Watch-and-reload**: the natural follow-up once cluster mode exists; the registry's mutation seam is ready, but reload semantics (drain? cutover?) deserve their own design. + +## References + +- [rfc-004-cluster-graph-schema-apply.md](rfc-004-cluster-graph-schema-apply.md) — the convergence machinery this serves +- [cluster-config-specs.md](cluster-config-specs.md) §Migration model — window 2 is this RFC +- [cluster-axioms.md](cluster-axioms.md) — axioms 5, 12, 14, 15 +- [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md) — Phase 5 rollout, Compatibility Stance #7–#9, blast-radius rows for the server registry +- `crates/omnigraph-server/src/lib.rs` (`load_server_settings`, `ServerConfigMode`, `GraphRegistry`) — the boot pipeline this extends without forking From 0b84b1adc3ec274be26fd00e286a7c8af564ccf5 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 15:30:33 +0300 Subject: [PATCH 073/165] feat(cluster): record policy applies_to bindings in the applied revision MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 5A of RFC-005: the state ledger becomes serving-sufficient for the Phase-5 server boot. StateResource gains an optional applies_to (normalized typed refs: cluster | graph.<id>), written by apply for every applied policy create/update from the desired config's validated bindings. The hole this closes: applies_to is not part of the policy file digest, so a binding-only edit previously produced NO plan change at all (a 4C e2e even asserted that — the gap, not a contract). Binding changes are now first-class: a post-diff pass emits an Update with equal before/after digests and a binding_change marker (visible in plan/apply JSON and human output as [bindings]), classification/execution treat it as an ordinary catalog-tier applied change (payload skips naturally — the blob is unchanged), and convergence requires zero binding divergence, so stale bindings can never report converged. Pre-5A ledger entries (no bindings recorded) surface as the same backfill Update; one apply heals them, exactly the remedy RFC-005's boot-error path names. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 10 +- crates/omnigraph-cli/tests/cli.rs | 7 +- crates/omnigraph-cluster/src/lib.rs | 220 +++++++++++++++++++++++++++- 3 files changed, 225 insertions(+), 12 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 8593ef3..673adb7 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -815,7 +815,8 @@ fn print_cluster_plan_human(output: &PlanOutput) { output.approvals_required.len() ); for change in &output.changes { - println!(" {:?} {}", change.operation, change.resource); + let bindings = if change.binding_change { " [bindings]" } else { "" }; + println!(" {:?} {}{bindings}", change.operation, change.resource); if let Some(migration) = &change.migration { if !migration.supported { println!(" migration UNSUPPORTED:"); @@ -862,16 +863,17 @@ fn print_cluster_apply_human(output: &ApplyOutput) { fn print_cluster_apply_changes(changes: &[omnigraph_cluster::PlanChange]) { for change in changes { + let bindings = if change.binding_change { " [bindings]" } else { "" }; match (&change.disposition, change.reason.as_deref()) { (Some(disposition), Some(reason)) => println!( - " {:?} {} [{disposition:?}: {reason}]", + " {:?} {}{bindings} [{disposition:?}: {reason}]", change.operation, change.resource ), (Some(disposition), None) => println!( - " {:?} {} [{disposition:?}]", + " {:?} {}{bindings} [{disposition:?}]", change.operation, change.resource ), - _ => println!(" {:?} {}", change.operation, change.resource), + _ => println!(" {:?} {}{bindings}", change.operation, change.resource), } } if changes.is_empty() { diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 336f19e..e4590f6 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -1451,9 +1451,10 @@ policies: change_for(&mixed, "query.knowledge.find_person")["disposition"], "applied" ); - // policy.shared's applies_to narrowed, but its FILE digest is unchanged - // — applies_to lives in cluster.yaml (the config digest), so it is not a - // resource change. + // 5A: policy.shared's applies_to narrowed with an unchanged file digest + // — now a first-class binding change, applied in the same run. + assert_eq!(change_for(&mixed, "policy.shared")["binding_change"], true); + assert_eq!(change_for(&mixed, "policy.shared")["disposition"], "applied"); assert_eq!( change_for(&mixed, "graph.knowledge")["disposition"], "derived" diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index f67d8f7..af2ef81 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -196,6 +196,11 @@ pub struct PlanChange { pub disposition: Option<ApplyDisposition>, #[serde(skip_serializing_if = "Option::is_none")] pub reason: Option<String>, + /// True for a policy change whose file digest is unchanged but whose + /// `applies_to` bindings differ from the applied revision (including the + /// pre-5A backfill case). + #[serde(default, skip_serializing_if = "std::ops::Not::not")] + pub binding_change: bool, /// For schema updates: the engine's migration plan against the live /// graph (RFC-004 §D7's data-aware preview). Absent when the preview is /// unavailable (warning `schema_preview_unavailable`). @@ -347,6 +352,8 @@ struct DesiredCluster { resource_digests: BTreeMap<String, String>, resources: Vec<ResourceSummary>, dependencies: Vec<Dependency>, + /// `policy.<name>` address -> normalized applies_to refs. + policy_bindings: BTreeMap<String, Vec<String>>, } #[derive(Debug, Clone)] @@ -457,6 +464,13 @@ struct AppliedRevisionState { #[serde(deny_unknown_fields)] struct StateResource { digest: String, + /// Policy resources only: the applied `applies_to` bindings, normalized + /// to typed refs (`cluster` | `graph.<id>`). Recorded so the state + /// ledger is serving-sufficient for the Phase-5 server boot (RFC-005 + /// §D3). Absent on pre-5A entries (backfilled by the next apply) and on + /// non-policy resources. + #[serde(default, skip_serializing_if = "Option::is_none")] + applies_to: Option<Vec<String>>, } #[derive(Debug, Serialize, Deserialize)] @@ -623,11 +637,13 @@ pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { warn_pending_recovery_sidecars(&desired.config_dir, &mut diagnostics); let mut prior_resources = BTreeMap::new(); + let mut prior_state: Option<ClusterState> = None; if !has_errors(&diagnostics) { match backend.read_state(&mut observations) { Ok(snapshot) => { if let Some(state) = snapshot.state { prior_resources = state_resource_digests(&state); + prior_state = Some(state); } } Err(diagnostic) => diagnostics.push(diagnostic), @@ -639,6 +655,9 @@ pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { } else { diff_resources(&prior_resources, &desired.resource_digests) }; + if !has_errors(&diagnostics) { + append_policy_binding_changes(&mut changes, prior_state.as_ref(), &desired); + } // Plan previews dispositions without sweeping; a pending recovery is // surfaced as the cluster_recovery_pending warning above instead. let artifacts = backend.list_approval_artifacts(&mut diagnostics); @@ -850,6 +869,7 @@ pub async fn apply_config_dir_with_options( let prior_resources = state_resource_digests(&state); let mut changes = diff_resources(&prior_resources, &desired.resource_digests); + append_policy_binding_changes(&mut changes, Some(&state), &desired); let approval_artifacts = backend.list_approval_artifacts(&mut diagnostics); let approved = approved_resources( &approval_artifacts, @@ -1429,6 +1449,12 @@ pub async fn apply_config_dir_with_options( .after_digest .clone() .expect("create/update always carries an after digest"), + // Policies record their applied bindings so the + // ledger is serving-sufficient (RFC-005 §D3). + applies_to: desired + .policy_bindings + .get(&change.resource) + .cloned(), }, ); set_resource_status_applied(&mut new_state, &change.resource); @@ -1467,10 +1493,11 @@ pub async fn apply_config_dir_with_options( } recompute_state_graph_digests(&mut new_state, &desired); - let residual = diff_resources( + let mut residual = diff_resources( &state_resource_digests(&new_state), &desired.resource_digests, ); + append_policy_binding_changes(&mut residual, Some(&new_state), &desired); let converged = residual.is_empty(); if converged { new_state.applied_revision.config_digest = Some(desired.config_digest.clone()); @@ -2741,6 +2768,7 @@ async fn sweep_graph_create_sidecar( schema_addr.clone(), StateResource { digest: live_digest.clone(), + applies_to: None, }, ); let query_digests = state_query_digests_for_graph(state, &sidecar.graph_id); @@ -2749,7 +2777,7 @@ async fn sweep_graph_create_sidecar( state .applied_revision .resources - .insert(graph_address.clone(), StateResource { digest: composite }); + .insert(graph_address.clone(), StateResource { digest: composite, applies_to: None }); set_resource_status_applied(state, &graph_address); set_resource_status_applied(state, &schema_addr); state.recovery_records.insert( @@ -2869,6 +2897,7 @@ async fn sweep_schema_apply_sidecar( schema_addr.clone(), StateResource { digest: live_digest.clone(), + applies_to: None, }, ); let query_digests = state_query_digests_for_graph(state, &sidecar.graph_id); @@ -2876,7 +2905,7 @@ async fn sweep_schema_apply_sidecar( state .applied_revision .resources - .insert(graph_address.clone(), StateResource { digest: composite }); + .insert(graph_address.clone(), StateResource { digest: composite, applies_to: None }); set_resource_status_applied(state, &graph_address); set_resource_status_applied(state, &schema_addr); state.recovery_records.insert( @@ -3109,6 +3138,7 @@ async fn observe_declared_graphs(desired: &DesiredCluster, state: &mut ClusterSt schema_address.clone(), StateResource { digest: observation.schema_digest.clone(), + applies_to: None, }, ); let query_digests = state_query_digests_for_graph(state, &graph.id); @@ -3121,6 +3151,7 @@ async fn observe_declared_graphs(desired: &DesiredCluster, state: &mut ClusterSt graph_address.clone(), StateResource { digest: graph_digest_value, + applies_to: None, }, ); state.observations.insert( @@ -3455,6 +3486,7 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { ); } + let mut policy_bindings: BTreeMap<String, Vec<String>> = BTreeMap::new(); for (policy_name, policy) in &raw.policies { validate_id( "policy name", @@ -3471,10 +3503,14 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { } let policy_address = policy_address(policy_name); + let mut normalized_bindings: Vec<String> = Vec::new(); for (idx, target) in policy.applies_to.iter().enumerate() { match normalize_policy_target(target) { - PolicyTarget::Cluster => {} + PolicyTarget::Cluster => { + normalized_bindings.push("cluster".to_string()); + } PolicyTarget::Graph(graph_id) => { + normalized_bindings.push(graph_address(&graph_id)); if raw.graphs.contains_key(&graph_id) { dependencies.insert(Dependency { from: policy_address.clone(), @@ -3498,6 +3534,10 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { } } + normalized_bindings.sort(); + normalized_bindings.dedup(); + policy_bindings.insert(policy_address.clone(), normalized_bindings); + let policy_path = resolve_config_path(&config_dir, &policy.file); match fs::read(&policy_path) { Ok(bytes) => { @@ -3551,6 +3591,7 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { resource_digests, resources: resource_list, dependencies, + policy_bindings, }), diagnostics, config_dir, @@ -3614,6 +3655,7 @@ fn diff_resources( after_digest: Some(after.clone()), disposition: None, reason: None, + binding_change: false, migration: None, }), Some(before) if before != after => changes.push(PlanChange { @@ -3623,6 +3665,7 @@ fn diff_resources( after_digest: Some(after.clone()), disposition: None, reason: None, + binding_change: false, migration: None, }), Some(_) => {} @@ -3637,6 +3680,7 @@ fn diff_resources( after_digest: None, disposition: None, reason: None, + binding_change: false, migration: None, }); } @@ -3645,6 +3689,43 @@ fn diff_resources( changes } +/// Binding-only policy changes: the file digest is unchanged (so +/// `diff_resources` saw nothing) but the applied `applies_to` differs from +/// the desired bindings — including the pre-5A case where the state entry +/// has no bindings recorded yet. These are first-class plan changes: without +/// this pass a binding edit would silently rot or silently converge. +fn append_policy_binding_changes( + changes: &mut Vec<PlanChange>, + prior_state: Option<&ClusterState>, + desired: &DesiredCluster, +) { + let Some(state) = prior_state else { + return; // no state: everything is already a Create carrying bindings + }; + for (address, desired_bindings) in &desired.policy_bindings { + if changes.iter().any(|change| &change.resource == address) { + continue; // content change already covers it + } + let Some(entry) = state.applied_revision.resources.get(address) else { + continue; // not applied yet: the Create covers it + }; + if entry.applies_to.as_ref() == Some(desired_bindings) { + continue; + } + changes.push(PlanChange { + resource: address.clone(), + operation: PlanOperation::Update, + before_digest: Some(entry.digest.clone()), + after_digest: Some(entry.digest.clone()), + disposition: None, + reason: None, + binding_change: true, + migration: None, + }); + } + changes.sort_by(|a, b| a.resource.cmp(&b.resource)); +} + fn compute_blast_radius(changes: &[PlanChange], dependencies: &[Dependency]) -> Vec<BlastRadius> { changes .iter() @@ -4163,7 +4244,7 @@ fn recompute_state_graph_digests(state: &mut ClusterState, desired: &DesiredClus state .applied_revision .resources - .insert(graph_address, StateResource { digest }); + .insert(graph_address, StateResource { digest, applies_to: None }); } } @@ -7062,6 +7143,135 @@ graphs: assert!(out.converged, "{out:?}"); } + // ---- policy bindings in the applied revision (5A) ---- + + #[tokio::test] + async fn apply_records_policy_bindings() { + let dir = fixture(); + write_applyable_state(dir.path()); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok && out.converged, "{:?}", out.diagnostics); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["policy.base"]["applies_to"], + serde_json::json!(["graph.knowledge"]), + "{state}" + ); + // Non-policy entries carry no bindings field at all. + assert!( + state["applied_revision"]["resources"]["query.knowledge.find_person"] + .get("applies_to") + .is_none() + ); + } + + #[tokio::test] + async fn binding_change_is_a_visible_plan_change() { + let dir = fixture(); + write_applyable_state(dir.path()); + let converge = apply_config_dir(dir.path()).await; + assert!(converge.converged, "{converge:?}"); + // Edit ONLY applies_to: the policy file digest is unchanged. + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +metadata: + name: test +state: + backend: cluster + lock: true +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +policies: + base: + file: ./base.policy.yaml + applies_to: [cluster, knowledge] +"#, + ) + .unwrap(); + + let plan = plan_config_dir(dir.path()).await; + let change = plan + .changes + .iter() + .find(|change| change.resource == "policy.base") + .expect("binding change must be visible in plan"); + assert!(change.binding_change); + assert_eq!(change.operation, PlanOperation::Update); + assert_eq!(change.before_digest, change.after_digest); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok && out.converged, "{out:?}"); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["policy.base"]["applies_to"], + serde_json::json!(["cluster", "graph.knowledge"]) + ); + // Idempotent: a second run sees no changes. + let again = apply_config_dir(dir.path()).await; + assert!(again.changes.is_empty() && !again.state_written, "{again:?}"); + } + + #[tokio::test] + async fn pre_5a_state_backfills_bindings() { + let dir = fixture(); + write_applyable_state(dir.path()); + let converge = apply_config_dir(dir.path()).await; + assert!(converge.converged, "{converge:?}"); + // Strip the bindings from the state entry (a pre-5A ledger). + let mut state: serde_json::Value = serde_json::from_str( + &fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(), + ) + .unwrap(); + state["applied_revision"]["resources"]["policy.base"] + .as_object_mut() + .unwrap() + .remove("applies_to"); + fs::write( + dir.path().join(CLUSTER_STATE_FILE), + serde_json::to_string_pretty(&state).unwrap(), + ) + .unwrap(); + + let plan = plan_config_dir(dir.path()).await; + assert!( + plan.changes + .iter() + .any(|change| change.resource == "policy.base" && change.binding_change), + "{plan:?}" + ); + let out = apply_config_dir(dir.path()).await; + assert!(out.ok && out.converged, "{out:?}"); + let healed = read_state_json(dir.path()); + assert_eq!( + healed["applied_revision"]["resources"]["policy.base"]["applies_to"], + serde_json::json!(["graph.knowledge"]) + ); + } + + #[tokio::test] + async fn bindings_survive_refresh() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + let converge = apply_config_dir(dir.path()).await; + assert!(converge.converged, "{converge:?}"); + + let refresh = refresh_config_dir(dir.path()).await; + assert!(refresh.ok, "{:?}", refresh.diagnostics); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["policy.base"]["applies_to"], + serde_json::json!(["graph.knowledge"]) + ); + } + #[test] fn status_warns_on_pending_recovery_sidecar() { let dir = fixture(); From 6c98560dde495951d8172b4fe595233f0a694365 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 15:30:57 +0300 Subject: [PATCH 074/165] docs(cluster): document policy binding metadata (5A) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/testing.md | 2 +- docs/user/cluster-config.md | 13 ++++++++++++- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 2302b13..a7a6cb3 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,7 +8,7 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish; includes the `cluster_e2e_*` lifecycle compositions over the spawned binary — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | -| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), and Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows) | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows), and 5A policy binding metadata (applies_to in the applied revision, binding-change diffing + convergence, pre-5A backfill) | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 2df26be..284bfbf 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -115,7 +115,10 @@ resource is planned as a create. If present, the file must use this shape: "graph.knowledge": { "digest": "..." }, "schema.knowledge": { "digest": "..." }, "query.knowledge.find_experts": { "digest": "..." }, - "policy.base": { "digest": "..." } + "policy.base": { + "digest": "...", + "applies_to": ["cluster", "graph.knowledge"] + } } }, "resource_statuses": { @@ -147,6 +150,14 @@ writes `state.json` and does not scan live graphs. Use explicit `cluster refresh` / `cluster import` when the state ledger should be updated from live observations. Live drift scans during plan are later-stage work. +Policy entries additionally record their applied `applies_to` bindings as +normalized typed refs — the state ledger is serving-sufficient for the +future server-boot stage. A change to `applies_to` alone (the policy file +digest unchanged) appears in the plan as an Update marked `binding_change` +(human output: `[bindings]`), applies like any catalog change, and counts +toward convergence; ledgers written before this field existed are backfilled +by the next apply. + Each plan change carries a `disposition` field — an honest preview of what `cluster apply` will do with it in this stage: `applied` (executes), `derived` (a `graph.<id>` composite-digest update that converges automatically once its From f5b43164b8b64d1c31b1a52b693ecb616760ef22 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 17:39:26 +0300 Subject: [PATCH 075/165] feat(cluster): pub read-only serving-snapshot API MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit RFC-005 §D2/§D4: read_serving_snapshot reads the applied revision as everything a server needs to boot — graphs at derived roots, stored-query sources read from the content-addressed catalog and re-hashed against the recorded digests, policy blob paths with their applied applies_to bindings. All-or-nothing: missing state, pending recovery sidecars, missing/tampered blobs, pre-5A entries without bindings, and an empty graph set each refuse the snapshot with a remedy; no partial serving. Lock-free by design — the state file is replaced atomically, so the read is a consistent point-in-time ledger. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 288 ++++++++++++++++++++++++++++ 1 file changed, 288 insertions(+) diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index af2ef81..7703bb8 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -1699,6 +1699,191 @@ pub async fn approve_config_dir( } } +/// One graph in a serving snapshot: its id and on-disk root. +#[derive(Debug, Clone)] +pub struct ServingGraph { + pub graph_id: String, + pub root: PathBuf, +} + +/// One stored query: its graph binding, registry name, and verified source. +#[derive(Debug, Clone)] +pub struct ServingQuery { + pub graph_id: String, + pub name: String, + pub source: String, +} + +/// One policy bundle: its verified catalog blob path and applied bindings +/// (normalized typed refs: `cluster` | `graph.<id>`). +#[derive(Debug, Clone)] +pub struct ServingPolicy { + pub name: String, + pub blob_path: PathBuf, + pub applies_to: Vec<String>, +} + +/// Everything a server needs to boot from the cluster catalog (RFC-005 §D2). +#[derive(Debug, Clone)] +pub struct ServingSnapshot { + pub graphs: Vec<ServingGraph>, + pub queries: Vec<ServingQuery>, + pub policies: Vec<ServingPolicy>, +} + +/// Read the applied revision as a serving snapshot — the read-only loader for +/// the Phase-5 server boot. All-or-nothing per RFC-005 §D4: every readiness +/// failure is collected and the whole snapshot refused; no partial serving. +/// Takes no lock: the state file is replaced atomically, so this reads a +/// consistent point-in-time ledger. +pub fn read_serving_snapshot(config_dir: impl AsRef<Path>) -> Result<ServingSnapshot, Vec<Diagnostic>> { + let config_dir = config_dir.as_ref().to_path_buf(); + let backend = LocalStateBackend::new(&config_dir); + let mut diagnostics: Vec<Diagnostic> = Vec::new(); + + // A ledger a sweep is about to rewrite must not start serving. + let sidecars = backend.list_recovery_sidecars(&mut diagnostics); + if !sidecars.is_empty() { + diagnostics.push(Diagnostic::error( + "cluster_recovery_pending", + CLUSTER_RECOVERIES_DIR, + format!( + "{} interrupted operation(s) await recovery; run any state-mutating cluster command (e.g. `cluster apply`) to sweep, then retry", + sidecars.len() + ), + )); + } + + let mut observations = backend.observations(); + let state = match backend.read_state(&mut observations) { + Ok(snapshot) => match snapshot.state { + Some(state) => Some(state), + None => { + diagnostics.push(Diagnostic::error( + "cluster_state_missing", + CLUSTER_STATE_FILE, + "no cluster state ledger; run `cluster import` and `cluster apply` first", + )); + None + } + }, + Err(diagnostic) => { + diagnostics.push(diagnostic); + None + } + }; + let Some(state) = state else { + return Err(diagnostics); + }; + + let mut graphs = Vec::new(); + let mut queries = Vec::new(); + let mut policies = Vec::new(); + for (address, entry) in &state.applied_revision.resources { + match resource_kind(address) { + ResourceKind::Graph(graph_id) => { + graphs.push(ServingGraph { + root: config_dir + .join(CLUSTER_GRAPHS_DIR) + .join(format!("{graph_id}.omni")), + graph_id, + }); + } + ResourceKind::Schema(_) => {} + kind @ ResourceKind::Query { .. } => { + let ResourceKind::Query { graph, name } = &kind else { + unreachable!() + }; + match read_verified_payload(&config_dir, &kind, &entry.digest, address) { + Ok(source) => queries.push(ServingQuery { + graph_id: graph.clone(), + name: name.clone(), + source, + }), + Err(diagnostic) => diagnostics.push(diagnostic), + } + } + kind @ ResourceKind::Policy(_) => { + let ResourceKind::Policy(name) = &kind else { + unreachable!() + }; + let Some(applies_to) = entry.applies_to.clone() else { + diagnostics.push(Diagnostic::error( + "policy_bindings_missing", + address.clone(), + "no applied applies_to bindings recorded (ledger predates binding metadata); re-run `cluster apply` to backfill", + )); + continue; + }; + match read_verified_payload(&config_dir, &kind, &entry.digest, address) { + Ok(_) => policies.push(ServingPolicy { + name: name.clone(), + blob_path: payload_path(&config_dir, &kind, &entry.digest) + .expect("policy kind always has a payload path"), + applies_to, + }), + Err(diagnostic) => diagnostics.push(diagnostic), + } + } + ResourceKind::Unknown => {} + } + } + + if graphs.is_empty() { + diagnostics.push(Diagnostic::error( + "cluster_empty", + CLUSTER_STATE_FILE, + "the applied revision records no graphs; apply a cluster with at least one graph before serving from it", + )); + } + if has_errors(&diagnostics) { + return Err(diagnostics); + } + Ok(ServingSnapshot { + graphs, + queries, + policies, + }) +} + +/// Read a catalog blob and verify it against the recorded digest. +fn read_verified_payload( + config_dir: &Path, + kind: &ResourceKind, + digest: &str, + address: &str, +) -> Result<String, Diagnostic> { + let path = payload_path(config_dir, kind, digest) + .expect("query/policy kinds always have a payload path"); + let bytes = fs::read(&path).map_err(|err| { + Diagnostic::error( + "catalog_payload_missing", + address, + format!( + "catalog blob '{}' unreadable ({err}); run `cluster refresh` then `cluster apply`, and restart", + display_path(&path) + ), + ) + })?; + if sha256_hex(&bytes) != digest { + return Err(Diagnostic::error( + "catalog_payload_digest_mismatch", + address, + format!( + "catalog blob '{}' does not match its recorded digest; run `cluster refresh` then `cluster apply`, and restart", + display_path(&path) + ), + )); + } + String::from_utf8(bytes).map_err(|err| { + Diagnostic::error( + "catalog_payload_invalid", + address, + format!("catalog blob is not valid UTF-8: {err}"), + ) + }) +} + pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { let parsed = parse_cluster_config(config_dir.as_ref()); let mut diagnostics = parsed.diagnostics; @@ -7272,6 +7457,109 @@ policies: ); } + // ---- serving snapshot (5B read-only loader) ---- + + #[tokio::test] + async fn serving_snapshot_reads_converged_cluster() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + let converge = apply_config_dir(dir.path()).await; + assert!(converge.converged, "{converge:?}"); + + let snapshot = read_serving_snapshot(dir.path()).expect("converged cluster must serve"); + assert_eq!(snapshot.graphs.len(), 1); + assert_eq!(snapshot.graphs[0].graph_id, "knowledge"); + assert!(snapshot.graphs[0].root.ends_with("graphs/knowledge.omni")); + assert_eq!(snapshot.queries.len(), 1); + assert_eq!(snapshot.queries[0].name, "find_person"); + assert!(snapshot.queries[0].source.contains("query find_person")); + assert_eq!(snapshot.policies.len(), 1); + assert_eq!(snapshot.policies[0].applies_to, vec!["graph.knowledge"]); + assert!(snapshot.policies[0].blob_path.exists()); + } + + #[test] + fn serving_snapshot_refuses_missing_state() { + let dir = fixture(); + let err = read_serving_snapshot(dir.path()).unwrap_err(); + assert!( + err.iter().any(|diagnostic| diagnostic.code == "cluster_state_missing"), + "{err:?}" + ); + } + + #[tokio::test] + async fn serving_snapshot_refuses_pending_recovery() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + apply_config_dir(dir.path()).await; + write_schema_apply_sidecar(dir.path(), "knowledge", "whatever", "01SERVE"); + + let err = read_serving_snapshot(dir.path()).unwrap_err(); + assert!( + err.iter().any(|diagnostic| diagnostic.code == "cluster_recovery_pending"), + "{err:?}" + ); + } + + #[tokio::test] + async fn serving_snapshot_refuses_tampered_blob_and_stripped_bindings() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + apply_config_dir(dir.path()).await; + // Tamper with the query blob... + let snapshot = read_serving_snapshot(dir.path()).unwrap(); + let desired = validate_config_dir(dir.path()); + let query_digest = &desired.resource_digests["query.knowledge.find_person"]; + let blob = dir + .path() + .join(CLUSTER_RESOURCES_DIR) + .join("query/knowledge/find_person") + .join(format!("{query_digest}.gq")); + fs::write(&blob, "tampered").unwrap(); + // ...and strip the policy bindings (pre-5A ledger). + let mut state: serde_json::Value = serde_json::from_str( + &fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(), + ) + .unwrap(); + state["applied_revision"]["resources"]["policy.base"] + .as_object_mut() + .unwrap() + .remove("applies_to"); + fs::write( + dir.path().join(CLUSTER_STATE_FILE), + serde_json::to_string_pretty(&state).unwrap(), + ) + .unwrap(); + + let err = read_serving_snapshot(dir.path()).unwrap_err(); + assert!( + err.iter() + .any(|diagnostic| diagnostic.code == "catalog_payload_digest_mismatch"), + "{err:?}" + ); + assert!( + err.iter().any(|diagnostic| diagnostic.code == "policy_bindings_missing"), + "{err:?}" + ); + let _ = snapshot; // the pre-tamper read succeeded + } + + #[test] + fn serving_snapshot_refuses_empty_cluster() { + let dir = fixture(); + write_state_resources(dir.path(), &[]); // state exists, no graphs + + let err = read_serving_snapshot(dir.path()).unwrap_err(); + assert!( + err.iter().any(|diagnostic| diagnostic.code == "cluster_empty"), + "{err:?}" + ); + } + #[test] fn status_warns_on_pending_recovery_sidecar() { let dir = fixture(); From 948a54daa7e27cf370b255a448b44e2be7e6bd83 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 17:48:10 +0300 Subject: [PATCH 076/165] feat(server): boot from cluster state via --cluster MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit RFC-005 §D1/§D2: omnigraph-server --cluster <dir> is rule 0 of the mode inference — an exclusive boot source (hard error when combined with a graph URI, --target, or --config) that never opens omnigraph.yaml, not even the implicit current-directory search. The cluster branch reads the applied revision through omnigraph-cluster's serving-snapshot API and feeds the EXISTING multi-graph pipeline: GraphStartupConfig per recorded graph at its derived root, stored queries built via QueryRegistry::from_specs from verified blob content (expose-all — the §D5 bridge until Phase 6 policy-owned exposure), cluster-bound policy bundles as the server-level Cedar engine and graph-bound bundles per graph, straight from the content-addressed blob paths. Multiple bundles binding one scope refuse boot (one-bundle-per-scope is the serving pipeline's shape; stacking is a later slice). Everything downstream — parallel opens, query type-checking, registry, routing, auth, OpenAPI — is reused unchanged; cluster mode is a new source, not a new pipeline. First server->cluster crate dependency: read-only types + one fn; omnigraph-cluster stays HTTP-free. open_multi_graph_state goes pub for integration tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- Cargo.lock | 1 + crates/omnigraph-server/Cargo.toml | 1 + crates/omnigraph-server/src/lib.rs | 133 +++++++++++- crates/omnigraph-server/src/main.rs | 7 + crates/omnigraph-server/tests/server.rs | 260 ++++++++++++++++++++++-- 5 files changed, 378 insertions(+), 24 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index f6a1b8a..675fad7 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4673,6 +4673,7 @@ dependencies = [ "futures", "lance", "lance-index", + "omnigraph-cluster", "omnigraph-compiler", "omnigraph-engine", "omnigraph-policy", diff --git a/crates/omnigraph-server/Cargo.toml b/crates/omnigraph-server/Cargo.toml index 5f87082..5393221 100644 --- a/crates/omnigraph-server/Cargo.toml +++ b/crates/omnigraph-server/Cargo.toml @@ -22,6 +22,7 @@ aws = ["dep:aws-config", "dep:aws-sdk-secretsmanager"] omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.2" } omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.2" } +omnigraph-cluster = { path = "../omnigraph-cluster", version = "0.6.2" } axum = { workspace = true } clap = { workspace = true } color-eyre = { workspace = true } diff --git a/crates/omnigraph-server/src/lib.rs b/crates/omnigraph-server/src/lib.rs index 60ebef3..3b9ff1d 100644 --- a/crates/omnigraph-server/src/lib.rs +++ b/crates/omnigraph-server/src/lib.rs @@ -14,7 +14,7 @@ pub use registry::{GraphHandle, GraphRegistry, InsertError, RegistryLookup, Regi use crate::queries::{QueryRegistry, check, format_check_breakages}; -use std::collections::{HashMap, HashSet}; +use std::collections::{BTreeMap, HashMap, HashSet}; use std::fs; use std::io; use std::io::Write; @@ -40,7 +40,7 @@ use axum::middleware::{self, Next}; use axum::response::{IntoResponse, Response}; use axum::routing::{delete, get, post}; use axum::{Json, Router}; -use color_eyre::eyre::{Result, WrapErr, bail}; +use color_eyre::eyre::{Result, WrapErr, bail, eyre}; pub use config::{ AliasCommand, AliasConfig, CliDefaults, DEFAULT_CONFIG_FILE, OmnigraphConfig, PolicySettings, ProjectConfig, QueryDefaults, ReadOutputFormat, ServerDefaults, TableCellLayout, TargetConfig, @@ -888,13 +888,125 @@ fn format_registry_load_errors(label: &str, errors: &[queries::LoadError]) -> St format!("graph '{label}': stored-query registry failed to load:\n {joined}") } +/// Build serving settings from a cluster directory's applied revision +/// (RFC-005 §D2): graphs at derived roots, stored queries from verified +/// catalog blob content, policy bundles from blob paths with their applied +/// bindings. Always multi-graph routing. The unauthenticated/env handling +/// matches the omnigraph.yaml path. +fn load_cluster_settings( + cluster_dir: &PathBuf, + cli_bind: Option<String>, + cli_allow_unauthenticated: bool, +) -> Result<ServerConfig> { + let snapshot = omnigraph_cluster::read_serving_snapshot(cluster_dir).map_err(|diagnostics| { + let details = diagnostics + .iter() + .map(|diagnostic| format!("[{}] {}: {}", diagnostic.code, diagnostic.path, diagnostic.message)) + .collect::<Vec<_>>() + .join("\n "); + eyre!("the cluster at '{}' is not ready to serve:\n {details}", cluster_dir.display()) + })?; + + // Bindings -> Cedar slots. The serving pipeline loads one bundle per + // graph plus one server-level bundle; stacked bundles per scope are a + // later slice — refuse loudly rather than silently merging policy. + let mut server_policy_file: Option<PathBuf> = None; + let mut graph_policy_files: BTreeMap<String, PathBuf> = BTreeMap::new(); + for policy in &snapshot.policies { + for binding in &policy.applies_to { + if binding == "cluster" { + if server_policy_file.replace(policy.blob_path.clone()).is_some() { + bail!( + "multiple policy bundles bind the cluster scope; cluster-mode serving supports one bundle per scope — split or merge bundles (multi-bundle scopes are a later slice)" + ); + } + } else if let Some(graph_id) = binding.strip_prefix("graph.") { + if graph_policy_files + .insert(graph_id.to_string(), policy.blob_path.clone()) + .is_some() + { + bail!( + "multiple policy bundles bind graph '{graph_id}'; cluster-mode serving supports one bundle per scope — split or merge bundles (multi-bundle scopes are a later slice)" + ); + } + } else { + bail!("unrecognized policy binding '{binding}' in the applied revision"); + } + } + } + + let mut graphs = Vec::new(); + for graph in &snapshot.graphs { + let specs: Vec<queries::RegistrySpec> = snapshot + .queries + .iter() + .filter(|query| query.graph_id == graph.graph_id) + .map(|query| queries::RegistrySpec { + name: query.name.clone(), + source: query.source.clone(), + // The §D5 bridge: the cluster registry has no expose flag + // (exposure becomes a policy decision in Phase 6) — cluster + // mode lists every stored query. + expose: true, + tool_name: None, + }) + .collect(); + let registry = QueryRegistry::from_specs(specs).map_err(|errors| { + let details = errors + .iter() + .map(|error| error.to_string()) + .collect::<Vec<_>>() + .join("\n "); + eyre!( + "stored queries in the applied revision failed to parse:\n {details}\nrun `cluster refresh` then `cluster apply`, and restart" + ) + })?; + graphs.push(GraphStartupConfig { + graph_id: graph.graph_id.clone(), + uri: graph.root.to_string_lossy().to_string(), + policy_file: graph_policy_files.get(&graph.graph_id).cloned(), + queries: registry, + }); + } + + let env_unauth = std::env::var("OMNIGRAPH_UNAUTHENTICATED") + .ok() + .map(|v| { + let trimmed = v.trim(); + !trimmed.is_empty() && trimmed != "0" && !trimmed.eq_ignore_ascii_case("false") + }) + .unwrap_or(false); + + Ok(ServerConfig { + mode: ServerConfigMode::Multi { + graphs, + config_path: cluster_dir.clone(), + server_policy_file, + }, + bind: cli_bind.unwrap_or_else(|| "127.0.0.1:8080".to_string()), + allow_unauthenticated: cli_allow_unauthenticated || env_unauth, + }) +} + pub fn load_server_settings( config_path: Option<&PathBuf>, + cli_cluster: Option<&PathBuf>, cli_uri: Option<String>, cli_target: Option<String>, cli_bind: Option<String>, cli_allow_unauthenticated: bool, ) -> Result<ServerConfig> { + // Rule 0 (RFC-005): --cluster is an exclusive boot source. It is checked + // before anything reads omnigraph.yaml — in cluster mode that file is + // never opened, not even the implicit current-directory search. + if let Some(cluster_dir) = cli_cluster { + if cli_uri.is_some() || cli_target.is_some() || config_path.is_some() { + bail!( + "--cluster is an exclusive boot source; it cannot combine with a graph URI, --target, or --config (axiom 15: a deployment serves from one source)" + ); + } + return load_cluster_settings(cluster_dir, cli_bind, cli_allow_unauthenticated); + } let config = load_config(config_path)?; let bind = cli_bind.unwrap_or_else(|| config.server_bind().to_string()); // Either `--unauthenticated` or `OMNIGRAPH_UNAUTHENTICATED=1` flips @@ -1275,7 +1387,7 @@ pub async fn serve(config: ServerConfig) -> Result<()> { /// The bound 4 is a rule-of-thumb for I/O-bound work. At N ≤ 10 this /// trades startup latency for a small amount of concurrent S3 / Lance /// open pressure. -async fn open_multi_graph_state( +pub async fn open_multi_graph_state( graphs: Vec<GraphStartupConfig>, tokens: Vec<(String, String)>, server_policy_file: Option<&PathBuf>, @@ -3255,7 +3367,7 @@ server: ) .unwrap(); - let settings = load_server_settings(Some(&config), None, None, None, false).unwrap(); + let settings = load_server_settings(Some(&config), None, None, None, None, false).unwrap(); match &settings.mode { ServerConfigMode::Single { uri, graph_id, .. } => { assert_eq!(uri, "/tmp/demo.omni"); @@ -3285,6 +3397,7 @@ server: let settings = load_server_settings( Some(&config), + None, Some("/tmp/override.omni".to_string()), None, Some("0.0.0.0:9999".to_string()), @@ -3321,7 +3434,7 @@ server: .unwrap(); let settings = - load_server_settings(Some(&config), None, Some("dev".to_string()), None, false) + load_server_settings(Some(&config), None, None, Some("dev".to_string()), None, false) .unwrap(); match &settings.mode { ServerConfigMode::Single { uri, graph_id, .. } => { @@ -3334,7 +3447,7 @@ server: #[test] fn server_settings_require_uri_from_cli_or_config() { - let error = load_server_settings(None, None, None, None, false).unwrap_err(); + let error = load_server_settings(None, None, None, None, None, false).unwrap_err(); assert!( error.to_string().contains("no graph to serve"), "expected mode-inference error, got: {error}", @@ -3501,7 +3614,7 @@ server: // Truthy values flip Open mode on, even with CLI flag off. for value in ["1", "true", "yes", "TRUE", "anything"] { let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some(value))]); - let settings = load_server_settings(Some(&config_path), None, None, None, false) + let settings = load_server_settings(Some(&config_path), None, None, None, None, false) .expect("settings load should succeed"); assert!( settings.allow_unauthenticated, @@ -3512,7 +3625,7 @@ server: // Falsy values keep refusal behavior, even with CLI flag off. for value in ["0", "false", "FALSE", ""] { let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some(value))]); - let settings = load_server_settings(Some(&config_path), None, None, None, false) + let settings = load_server_settings(Some(&config_path), None, None, None, None, false) .expect("settings load should succeed"); assert!( !settings.allow_unauthenticated, @@ -3522,7 +3635,7 @@ server: // Unset env var: also false. let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", None)]); - let settings = load_server_settings(Some(&config_path), None, None, None, false) + let settings = load_server_settings(Some(&config_path), None, None, None, None, false) .expect("settings load should succeed"); assert!( !settings.allow_unauthenticated, @@ -3533,7 +3646,7 @@ server: // CLI flag wins even when env is falsy — `serve()` honors the // OR of both inputs. let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some("0"))]); - let settings = load_server_settings(Some(&config_path), None, None, None, true) + let settings = load_server_settings(Some(&config_path), None, None, None, None, true) .expect("settings load should succeed"); assert!( settings.allow_unauthenticated, diff --git a/crates/omnigraph-server/src/main.rs b/crates/omnigraph-server/src/main.rs index 4e1c256..c71ea2f 100644 --- a/crates/omnigraph-server/src/main.rs +++ b/crates/omnigraph-server/src/main.rs @@ -14,6 +14,12 @@ struct Cli { target: Option<String>, #[arg(long)] config: Option<PathBuf>, + /// Boot from a cluster directory (the applied revision in + /// __cluster/state.json + content-addressed catalog blobs) instead of + /// omnigraph.yaml. Exclusive: cannot combine with <URI>, --target, or + /// --config. + #[arg(long)] + cluster: Option<PathBuf>, #[arg(long)] bind: Option<String>, /// Run without bearer tokens and without a policy file (MR-723). @@ -32,6 +38,7 @@ async fn main() -> Result<()> { let cli = Cli::parse(); let settings: ServerConfig = load_server_settings( cli.config.as_ref(), + cli.cluster.as_ref(), cli.uri, cli.target, cli.bind, diff --git a/crates/omnigraph-server/tests/server.rs b/crates/omnigraph-server/tests/server.rs index 4a49a14..bf99b8d 100644 --- a/crates/omnigraph-server/tests/server.rs +++ b/crates/omnigraph-server/tests/server.rs @@ -5508,7 +5508,7 @@ graphs: "#, ) .unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, false).unwrap_err(); + let err = load_server_settings(Some(&config_path), None, None, None, None, false).unwrap_err(); assert!( err.to_string().contains("invalid graph id 'policies'"), "expected reserved-name rejection, got: {err}" @@ -5575,6 +5575,7 @@ graphs: #[test] fn mode_inference_cli_uri_is_single() { let settings = load_server_settings( + None, None, Some("/tmp/cli.omni".to_string()), None, @@ -5605,7 +5606,7 @@ graphs: ) .unwrap(); let settings = - load_server_settings(Some(&config_path), None, Some("alpha".into()), None, true) + load_server_settings(Some(&config_path), None, None, Some("alpha".into()), None, true) .unwrap(); match settings.mode { ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/alpha.omni"), @@ -5631,7 +5632,7 @@ server: "#, ) .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, true).unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); match settings.mode { ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/beta.omni"), ServerConfigMode::Multi { .. } => panic!("expected Single (rule 3), got Multi"), @@ -5654,7 +5655,7 @@ graphs: "#, ) .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, true).unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); match settings.mode { ServerConfigMode::Multi { graphs, .. } => { let ids: Vec<&str> = graphs.iter().map(|g| g.graph_id.as_str()).collect(); @@ -5680,7 +5681,7 @@ graphs: "#, ) .unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, true).unwrap_err(); + let err = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap_err(); let msg = err.to_string(); assert!( msg.contains("top-level") && msg.contains("policy.file") && msg.contains("not honored"), @@ -5708,7 +5709,7 @@ graphs: "queries:\n q:\n file: ./q.gq\ngraphs:\n alpha:\n uri: /tmp/alpha.omni\n", ) .unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, true).unwrap_err(); + let err = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap_err(); let msg = err.to_string(); assert!( msg.contains("queries") && msg.contains("not honored"), @@ -5729,7 +5730,7 @@ graphs: ) .unwrap(); let err = - load_server_settings(Some(&config_path), None, Some("prod".to_string()), None, true) + load_server_settings(Some(&config_path), None, None, Some("prod".to_string()), None, true) .unwrap_err(); let msg = err.to_string(); assert!( @@ -5756,7 +5757,7 @@ graphs: ) .unwrap(); let settings = - load_server_settings(Some(&config_path), None, Some("prod".to_string()), None, true) + load_server_settings(Some(&config_path), None, None, Some("prod".to_string()), None, true) .unwrap(); match settings.mode { ServerConfigMode::Single { @@ -5795,7 +5796,7 @@ graphs: ), ) .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, true).unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); match settings.mode { ServerConfigMode::Multi { graphs, .. } => { assert_eq!(graphs[0].uri, graph.to_string_lossy()); @@ -5807,7 +5808,7 @@ graphs: /// Rule 5: nothing → error with migration hint. #[test] fn mode_inference_no_inputs_errors_with_migration_hint() { - let err = load_server_settings(None, None, None, None, true).unwrap_err(); + let err = load_server_settings(None, None, None, None, None, true).unwrap_err(); let msg = err.to_string(); assert!( msg.contains("no graph to serve"), @@ -5822,7 +5823,7 @@ graphs: let temp = tempfile::tempdir().unwrap(); let config_path = temp.path().join("omnigraph.yaml"); fs::write(&config_path, "server:\n bind: 127.0.0.1:8080\n").unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, true).unwrap_err(); + let err = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap_err(); assert!(err.to_string().contains("no graph to serve")); } @@ -5843,6 +5844,7 @@ graphs: .unwrap(); let settings = load_server_settings( Some(&config_path), + None, Some("/tmp/cli-override.omni".to_string()), None, None, @@ -5880,7 +5882,7 @@ graphs: "#, ) .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, true).unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); let graphs = match settings.mode { ServerConfigMode::Multi { graphs, .. } => graphs, _ => panic!("expected Multi"), @@ -5914,7 +5916,7 @@ graphs: "#, ) .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, true).unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); match settings.mode { ServerConfigMode::Multi { server_policy_file, .. @@ -6194,7 +6196,7 @@ graphs: .unwrap(); let settings: ServerConfig = - load_server_settings(Some(&config_path), None, None, None, true).unwrap(); + load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); assert!(matches!(settings.mode, ServerConfigMode::Multi { .. })); match settings.mode { @@ -6207,3 +6209,233 @@ graphs: } } } + +// ---- Phase 5: cluster-mode boot (RFC-005) ---- + +/// Build and converge a real cluster directory: cluster.yaml + schema + +/// stored query (+ optional policies), then `import` + `apply` so the +/// catalog and state ledger exist exactly as an operator would have them. +async fn converged_cluster_dir(policies_yaml: &str) -> tempfile::TempDir { + let temp = tempfile::tempdir().unwrap(); + fs::write( + temp.path().join("people.pg"), + "\nnode Person {\n name: String @key\n}\n", + ) + .unwrap(); + fs::write( + temp.path().join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + temp.path().join("cluster.yaml"), + format!( + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +{policies_yaml}"# + ), + ) + .unwrap(); + let import = omnigraph_cluster::import_config_dir(temp.path()).await; + assert!(import.ok, "{:?}", import.diagnostics); + let apply = omnigraph_cluster::apply_config_dir(temp.path()).await; + assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); + temp +} + +fn cluster_settings(dir: &Path) -> color_eyre::eyre::Result<omnigraph_server::ServerConfig> { + omnigraph_server::load_server_settings(None, Some(&dir.to_path_buf()), None, None, None, true) +} + +#[tokio::test] +async fn cluster_boot_serves_applied_state() { + let temp = converged_cluster_dir("").await; + let settings = cluster_settings(temp.path()).unwrap(); + let omnigraph_server::ServerConfigMode::Multi { + graphs, + config_path, + server_policy_file, + } = settings.mode + else { + panic!("cluster boot must select multi-graph routing"); + }; + assert_eq!(graphs.len(), 1); + assert_eq!(graphs[0].graph_id, "knowledge"); + assert!(server_policy_file.is_none()); + + let state = + omnigraph_server::open_multi_graph_state(graphs, Vec::new(), None, config_path) + .await + .unwrap(); + let app = build_app(state); + + // The management surface keeps its closed-by-default contract: without a + // cluster-scoped policy bundle there is no server-level Cedar engine, so + // GET /graphs refuses even in cluster mode. + let (status, body) = json_response( + &app, + Request::builder().uri("/graphs").body(Body::empty()).unwrap(), + ) + .await; + assert_eq!(status, StatusCode::FORBIDDEN, "{body}"); + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/graphs/knowledge/queries") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK, "{body}"); + assert!( + body["queries"] + .as_array() + .unwrap() + .iter() + .any(|q| q["name"] == "find_person"), + "{body}" + ); + + let (status, body) = json_response( + &app, + Request::builder() + .method(Method::POST) + .uri("/graphs/knowledge/queries/find_person") + .header("content-type", "application/json") + .body(Body::from(r#"{"params":{"name":"nobody"}}"#)) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK, "{body}"); +} + +#[tokio::test] +async fn cluster_boot_wires_policy_bindings_into_cedar_slots() { + let temp = tempfile::tempdir().unwrap(); + drop(temp); + let policy_block = r#"policies: + graph_rules: + file: ./graph.policy.yaml + applies_to: [knowledge] + cluster_rules: + file: ./cluster.policy.yaml + applies_to: [cluster] +"#; + let temp = { + let temp = tempfile::tempdir().unwrap(); + fs::write( + temp.path().join("people.pg"), + "\nnode Person {\n name: String @key\n}\n", + ) + .unwrap(); + fs::write( + temp.path().join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + temp.path().join("graph.policy.yaml"), + permit_all_policy_yaml(&["default"]), + ) + .unwrap(); + fs::write( + temp.path().join("cluster.policy.yaml"), + permit_all_policy_yaml(&["default"]).replace("protected_branches: [main]\n", "protected_branches: [main]\nkind: server\n"), + ) + .unwrap(); + fs::write( + temp.path().join("cluster.yaml"), + format!( + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +{policy_block}"# + ), + ) + .unwrap(); + let import = omnigraph_cluster::import_config_dir(temp.path()).await; + assert!(import.ok, "{:?}", import.diagnostics); + let apply = omnigraph_cluster::apply_config_dir(temp.path()).await; + assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); + temp + }; + + let settings = cluster_settings(temp.path()).unwrap(); + let omnigraph_server::ServerConfigMode::Multi { + graphs, + server_policy_file, + .. + } = settings.mode + else { + panic!("cluster boot must select multi-graph routing"); + }; + let graph_policy = graphs[0].policy_file.as_ref().expect("graph-bound bundle"); + assert!( + graph_policy + .to_string_lossy() + .contains("__cluster/resources/policy/graph_rules/"), + "{graph_policy:?}" + ); + let server_policy = server_policy_file.expect("cluster-bound bundle"); + assert!( + server_policy + .to_string_lossy() + .contains("__cluster/resources/policy/cluster_rules/"), + "{server_policy:?}" + ); +} + +#[tokio::test] +async fn cluster_boot_refusals() { + // Mutual exclusion with --config / URI. + let temp = converged_cluster_dir("").await; + let dir = temp.path().to_path_buf(); + let err = omnigraph_server::load_server_settings( + Some(&dir.join("omnigraph.yaml")), + Some(&dir), + None, + None, + None, + true, + ) + .unwrap_err(); + assert!(err.to_string().contains("exclusive boot source"), "{err}"); + let err = omnigraph_server::load_server_settings( + None, + Some(&dir), + Some("file:///tmp/x.omni".to_string()), + None, + None, + true, + ) + .unwrap_err(); + assert!(err.to_string().contains("exclusive boot source"), "{err}"); + + // Tampered catalog blob refuses boot with the remedy. + let blob_dir = dir.join("__cluster/resources/query/knowledge/find_person"); + let blob = fs::read_dir(&blob_dir).unwrap().next().unwrap().unwrap().path(); + fs::write(&blob, "tampered").unwrap(); + let err = cluster_settings(&dir).unwrap_err(); + assert!( + err.to_string().contains("catalog_payload_digest_mismatch"), + "{err}" + ); + assert!(err.to_string().contains("cluster refresh"), "{err}"); + + // Missing state refuses with the import/apply remedy. + let empty = tempfile::tempdir().unwrap(); + let err = cluster_settings(empty.path()).unwrap_err(); + assert!(err.to_string().contains("cluster_state_missing"), "{err}"); +} From f3eb60fa4e8b2f0138501780cc27f6ce5a31f09e Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 17:51:40 +0300 Subject: [PATCH 077/165] test(cli): applied-means-serving system e2e The Phase-5 contract end to end with real binaries: cluster import + apply via the CLI, seed a row through the graph plane, boot omnigraph-server with --cluster (no omnigraph.yaml anywhere), and the applied stored query serves the row over HTTP through the multi-graph routes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/support/mod.rs | 6 ++ crates/omnigraph-cli/tests/system_local.rs | 81 ++++++++++++++++++++++ 2 files changed, 87 insertions(+) diff --git a/crates/omnigraph-cli/tests/support/mod.rs b/crates/omnigraph-cli/tests/support/mod.rs index b62d861..5c17182 100644 --- a/crates/omnigraph-cli/tests/support/mod.rs +++ b/crates/omnigraph-cli/tests/support/mod.rs @@ -212,6 +212,12 @@ pub fn spawn_server_with_config(config: &Path) -> TestServer { spawn_server_process(command) } +pub fn spawn_server_with_cluster(cluster_dir: &Path) -> TestServer { + let mut command = server_process(); + command.arg("--cluster").arg(cluster_dir).arg("--unauthenticated"); + spawn_server_process(command) +} + pub fn spawn_server_with_config_env(config: &Path, envs: &[(&str, &str)]) -> TestServer { let mut command = server_process(); command.arg("--config").arg(config); diff --git a/crates/omnigraph-cli/tests/system_local.rs b/crates/omnigraph-cli/tests/system_local.rs index 4fc3e9a..81476b0 100644 --- a/crates/omnigraph-cli/tests/system_local.rs +++ b/crates/omnigraph-cli/tests/system_local.rs @@ -1633,3 +1633,84 @@ fn local_cli_actor_flag_overrides_config_actor() { "expected 'denied' when --as overrides config to bruno, got: {stderr}" ); } + +/// Phase 5 (RFC-005): "applied means serving" — converge a cluster with the +/// CLI, boot the real omnigraph-server binary with --cluster, and serve the +/// applied stored query over HTTP with zero omnigraph.yaml involvement. +#[test] +fn local_cluster_apply_then_server_boots_from_cluster_state() { + let temp = tempfile::tempdir().unwrap(); + std::fs::write( + temp.path().join("people.pg"), + "\nnode Person {\n name: String @key\n}\n", + ) + .unwrap(); + std::fs::write( + temp.path().join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + std::fs::write( + temp.path().join("cluster.yaml"), + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +"#, + ) + .unwrap(); + for command in ["import", "apply"] { + let output = cli() + .arg("cluster") + .arg(command) + .arg("--config") + .arg(temp.path()) + .arg("--json") + .output() + .unwrap(); + assert!(output.status.success(), "cluster {command} failed"); + } + // Seed a row through the graph plane so the stored query has data. + let data = temp.path().join("seed.jsonl"); + std::fs::write(&data, "{\"type\":\"Person\",\"data\":{\"name\":\"Ada\"}}\n").unwrap(); + let output = cli() + .arg("load") + .arg("--data") + .arg(&data) + .arg(temp.path().join("graphs/knowledge.omni")) + .output() + .unwrap(); + assert!(output.status.success(), "graph load failed"); + + let server = spawn_server_with_cluster(temp.path()); + let client = reqwest::blocking::Client::new(); + let queries: serde_json::Value = client + .get(format!("{}/graphs/knowledge/queries", server.base_url)) + .send() + .unwrap() + .json() + .unwrap(); + assert!( + queries["queries"] + .as_array() + .unwrap() + .iter() + .any(|q| q["name"] == "find_person"), + "{queries}" + ); + let response = client + .post(format!( + "{}/graphs/knowledge/queries/find_person", + server.base_url + )) + .json(&serde_json::json!({"params": {"name": "Ada"}})) + .send() + .unwrap(); + assert!(response.status().is_success(), "{:?}", response.status()); + let body: serde_json::Value = response.json().unwrap(); + assert!(body.to_string().contains("Ada"), "{body}"); +} From 711865e6f117b0f0c1ff09eb32c60a36dc0e3f1f Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 17:55:15 +0300 Subject: [PATCH 078/165] docs(cluster,server): the Phase 5 mode switch; retire applied-not-serving caveats MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The standing caveat ('applied means recorded in the cluster catalog — nothing more; the server still boots from omnigraph.yaml') retires: cluster docs gain the 'Serving from the cluster' section (exclusivity, applied- revision serving, fail-fast readiness, restart-to-pick-up, expose-all bridge), server.md gains mode-inference rule 0 and the cluster-booted multi mode, deployment.md the boot-source choice, and the CLI's apply note plus the cli-reference cluster row (stale back to Stage 3A) now describe the full convergence surface. RFC-005 flips to Landed with four implementation deviations recorded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 2 +- docs/dev/rfc-005-server-cluster-boot.md | 3 +- docs/dev/testing.md | 4 +-- docs/user/cli-reference.md | 2 +- docs/user/cluster-config.md | 46 ++++++++++++++++++++++--- docs/user/deployment.md | 8 +++++ docs/user/server.md | 16 +++++++-- 7 files changed, 69 insertions(+), 12 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 673adb7..da4f8e8 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -856,7 +856,7 @@ fn print_cluster_apply_human(output: &ApplyOutput) { " state: revision {}, converged: {}, written: {}", state.state_revision, output.converged, output.state_written ); - println!(" note: applied = recorded in the cluster catalog; the server still boots from omnigraph.yaml"); + println!(" note: cluster-booted servers (--cluster) serve this on their next restart; omnigraph.yaml deployments are unaffected"); } print_cluster_diagnostics(&output.diagnostics); } diff --git a/docs/dev/rfc-005-server-cluster-boot.md b/docs/dev/rfc-005-server-cluster-boot.md index 81d5129..85df875 100644 --- a/docs/dev/rfc-005-server-cluster-boot.md +++ b/docs/dev/rfc-005-server-cluster-boot.md @@ -1,6 +1,7 @@ # RFC: Server Boots from Cluster State — Phase 5 of the Cluster Control Plane -**Status:** Proposed +**Status:** Landed (5A policy bindings #175; 5B/5C the `--cluster` boot mode — one PR) +**Implementation deviations:** (1) cluster mode reuses `ServerConfigMode::Multi` (a new settings *source*, not a new enum variant; `config_path` carries the cluster dir). (2) Stored queries load via `QueryRegistry::from_specs` from verified blob *content*, not blob paths. (3) More than one policy bundle binding a single scope is a boot error (the serving pipeline holds one bundle per graph + one server-level; stacking is a later slice). (4) `GET /graphs` keeps its closed-by-default contract — without a cluster-bound bundle there is no server-level Cedar engine, so enumeration refuses. **Date:** 2026-06-10 **Builds on:** Phase 4 complete ([rfc-004-cluster-graph-schema-apply.md](rfc-004-cluster-graph-schema-apply.md), Landed): `cluster apply` converges graphs, schemas, stored queries, and policies into the cluster catalog. Normative context: [cluster-config-specs.md](cluster-config-specs.md) (the migration model's "window 2"), [cluster-axioms.md](cluster-axioms.md) (axiom 15), [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md) (Phase 5 rollout, Compatibility Stance #7–#9, exit criterion 7). **Target release:** unversioned (phased — see Sequencing). diff --git a/docs/dev/testing.md b/docs/dev/testing.md index a7a6cb3..eba70c9 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,8 +8,8 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish; includes the `cluster_e2e_*` lifecycle compositions over the spawned binary — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | -| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows), and 5A policy binding metadata (applies_to in the applied revision, binding-change diffing + convergence, pre-5A backfill) | -| `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level), `openapi.rs` (OpenAPI drift / regeneration) | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows), and 5A policy binding metadata (applies_to in the applied revision, binding-change diffing + convergence, pre-5A backfill), and the 5B serving-snapshot read API (converged read, refusal rows) | +| `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level; incl. cluster-mode boot — converged-dir serving, policy binding wiring, boot refusals), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | The engine's `tests/` is the principal coverage surface; most graph-shaped behavior is exercised there. diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 9dc8a25..6d864cc 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -19,7 +19,7 @@ Top-level command families and subcommands. Graph-targeting commands accept eith | `commit list \| show` | inspect commit graph | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | -| `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | cluster-control preview. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json` and annotates each change with its apply disposition; `apply` executes the config-only (stored-query/policy) subset into the content-addressed local catalog under `__cluster/resources/` — graph/schema changes are deferred loudly, and nothing applied serves traffic (the server still boots from `omnigraph.yaml`); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id. No graph-manifest movement, server change, automatic stale-lock breaking, or `plan --refresh` occurs in Stage 3A | +| `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | declarative cluster control plane. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`, annotates dispositions, and embeds real schema-migration previews; `apply` converges the cluster — stored-query/policy catalog writes (content-addressed under `__cluster/resources/`), graph creates, schema updates (soft drops only; `--as` records the actor), and graph deletes behind a digest-bound approval from `cluster approve <resource> --as <actor>`; what apply converges is what an `omnigraph-server --cluster <dir>` deployment serves on its next restart (omnigraph.yaml deployments are unaffected); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) | | `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 284bfbf..5c51b1f 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -1,6 +1,6 @@ # Cluster Config -**Status:** Stage 4C — Phase 4 complete (graph create, schema apply, gated graph delete). +**Status:** Phase 5 — cluster-booted serving (`omnigraph-server --cluster`). Cluster config is the future control-plane configuration surface for a whole OmniGraph deployment. In this stage, OmniGraph can validate a local @@ -190,10 +190,12 @@ Deletes remove the resource from state; their old payload blobs stay on disk (garbage collection is a later stage). Re-running a converged apply is a no-op: no state write, no revision change (`state_written: false`). -**Applied means recorded in the cluster catalog — nothing more.** The server -still boots from `omnigraph.yaml`; no query or policy applied here serves -traffic until the server-boot stage ships, as an explicit per-deployment mode -switch. +**Applied means serving — for deployments that opt in.** A server started +with `--cluster <dir>` boots from the applied revision (see +[Serving from the cluster](#serving-from-the-cluster-the-mode-switch)); it +picks up newly applied state on its next restart. Deployments still booting +from `omnigraph.yaml` are untouched: for them, applied means recorded in the +catalog, nothing more. ### Graph creation @@ -305,6 +307,40 @@ fully converges. The `graph.<id>` composite digest is recomputed from state's own schema/query digests after each apply, so applied query changes converge without graph movement. +## Serving from the cluster (the mode switch) + +```bash +omnigraph-server --cluster ./company-brain --bind 0.0.0.0:8080 +``` + +`--cluster <dir>` is an **exclusive boot source** (axiom 15): it cannot +combine with a graph URI, `--target`, or `--config`, and in this mode +`omnigraph.yaml` is never read — not for graphs, not for queries, not for +policies. The server serves the **applied revision**: graph roots recorded in +`state.json`, stored-query and policy content from the content-addressed +catalog at the applied digests (re-verified at boot), and policy bundles +wired by their applied `applies_to` bindings — `cluster`-bound bundles become +the server-level Cedar engine, graph-bound bundles attach per graph. +Un-applied config drift never leaks into serving; `cluster plan` is where +drift is visible. Routing is always multi-graph (`/graphs/{id}/...`). Bearer +tokens and the bind address stay process-level (flags/env) — they are +per-replica facts, not cluster facts. + +Boot is fail-fast: missing or unreadable state, pending recovery sidecars, +missing/tampered catalog blobs, policy entries without binding metadata +(pre-binding ledgers — re-run `cluster apply`), an empty graph set, more than +one policy bundle binding a single scope (split or merge bundles; stacked +scopes are a later stage), unopenable graph roots, and stored queries that no +longer type-check all refuse startup with a remedy. A held state lock is +*not* an error — boot reads the atomically-replaced state file without +locking. + +Serving is static per process: the server reads the applied revision once at +startup, so picking up newly applied state means restarting it. Stored +queries are all listed in `GET /queries` in cluster mode (the cluster +registry has no expose flag; exposure becomes a policy decision in a later +phase). + ## Status `cluster status` reads the same local JSON state ledger and prints what the diff --git a/docs/user/deployment.md b/docs/user/deployment.md index 9a4466c..328784f 100644 --- a/docs/user/deployment.md +++ b/docs/user/deployment.md @@ -13,6 +13,14 @@ Omnigraph supports two broad deployment shapes: The server binary and container image expose the same HTTP surface. +The server also has two **boot sources**: `omnigraph.yaml` (graph targets +declared in the per-operator config) or a **cluster directory** +(`omnigraph-server --cluster <dir>`), which serves the cluster control +plane's applied revision — see +[cluster-config.md](cluster-config.md#serving-from-the-cluster-the-mode-switch). +The two are exclusive per deployment; switching is a restart with a different +flag. + ## Binary Deployment Build or install: diff --git a/docs/user/server.md b/docs/user/server.md index 67b5afe..60988ca 100644 --- a/docs/user/server.md +++ b/docs/user/server.md @@ -1,6 +1,6 @@ # HTTP Server (`omnigraph-server`) -Axum 0.8 + tokio + utoipa-generated OpenAPI. **Two modes** (v0.6.0+): single-graph (legacy) and multi-graph (MR-668). Mode is inferred from CLI args + config shape. +Axum 0.8 + tokio + utoipa-generated OpenAPI. **Two modes** (v0.6.0+): single-graph (legacy) and multi-graph (MR-668), with **two boot sources** for multi mode: `omnigraph.yaml` or — exclusively — a cluster directory (`--cluster`, RFC-005). Mode is inferred from CLI args + config shape. ## Modes @@ -14,8 +14,20 @@ Axum 0.8 + tokio + utoipa-generated OpenAPI. **Two modes** (v0.6.0+): single-gra `omnigraph-server --config omnigraph.yaml` with a non-empty `graphs:` map and **no** single-mode selector (no `server.graph`, no `<URI>`, no `--target`). The server opens every configured graph in parallel at startup (bounded concurrency = 4, fail-fast on the first open error). Routes are nested under `/graphs/{graph_id}/...`. Bare flat paths return 404 in multi mode. -Mode inference (four-rule matrix): +### Cluster-booted multi mode (Phase 5) +`omnigraph-server --cluster <dir>` boots from the cluster catalog's **applied +revision** (`state.json` + content-addressed blobs) instead of +`omnigraph.yaml` — an exclusive boot source: combining it with `<URI>`, +`--target`, or `--config` is a startup error, and `omnigraph.yaml` is never +read in this mode. Always multi-graph routing. See +[cluster-config.md](cluster-config.md#serving-from-the-cluster-the-mode-switch) +for what is read and the fail-fast readiness rules. `--bind`, +`--unauthenticated`, and the bearer-token env vars work identically. + +Mode inference: + +0. CLI `--cluster <dir>` → **multi, cluster-booted** (exclusive) 1. CLI positional `<URI>` → single 2. CLI `--target <name>` → single 3. `server.graph` in config → single From 7d70811df1eceedf4f7b51a9b4e340e6a740237c Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 18:07:29 +0300 Subject: [PATCH 079/165] test(cli): comprehensive full-cycle cluster e2e with a live server MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two system tests composing the whole Phase 1-5 surface with real binaries: - local_cluster_full_lifecycle_declare_serve_evolve_delete: declare two graphs -> one apply creates and converges them -> the --cluster server serves both stored queries -> schema+query evolve in one apply (migration previewed in plan) -> restart serves the new shape -> out-of-band schema drift observed by refresh and converged back by apply (rogue field soft-dropped) -> approved graph delete -> restart serves the survivor and 404s the tombstoned graph -> final plan empty. Catches composition regressions where each stage passes its own tests but the lifecycle breaks (the composite_flow.rs principle at the control-plane level). - local_cluster_serving_enforces_applied_policy_bindings: applied policy bundles gate serving per their bindings over HTTP with bearer-resolved actors — the cluster-bound bundle owns graph_list (admin 200, reader 403, anonymous 401), the graph-bound bundle owns invoke_query (reader gets rows; denied invocation is the documented anti-probing 404). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/support/mod.rs | 9 + crates/omnigraph-cli/tests/system_local.rs | 403 +++++++++++++++++++++ docs/dev/testing.md | 2 +- 3 files changed, 413 insertions(+), 1 deletion(-) diff --git a/crates/omnigraph-cli/tests/support/mod.rs b/crates/omnigraph-cli/tests/support/mod.rs index 5c17182..855d8e0 100644 --- a/crates/omnigraph-cli/tests/support/mod.rs +++ b/crates/omnigraph-cli/tests/support/mod.rs @@ -218,6 +218,15 @@ pub fn spawn_server_with_cluster(cluster_dir: &Path) -> TestServer { spawn_server_process(command) } +pub fn spawn_server_with_cluster_env(cluster_dir: &Path, envs: &[(&str, &str)]) -> TestServer { + let mut command = server_process(); + command.arg("--cluster").arg(cluster_dir); + for (name, value) in envs { + command.env(name, value); + } + spawn_server_process(command) +} + pub fn spawn_server_with_config_env(config: &Path, envs: &[(&str, &str)]) -> TestServer { let mut command = server_process(); command.arg("--config").arg(config); diff --git a/crates/omnigraph-cli/tests/system_local.rs b/crates/omnigraph-cli/tests/system_local.rs index 81476b0..14b8890 100644 --- a/crates/omnigraph-cli/tests/system_local.rs +++ b/crates/omnigraph-cli/tests/system_local.rs @@ -1714,3 +1714,406 @@ graphs: let body: serde_json::Value = response.json().unwrap(); assert!(body.to_string().contains("Ada"), "{body}"); } + +// ---- Comprehensive full-cycle cluster e2e (Phases 1-5 composed) ---- + +fn cluster_cli(dir: &std::path::Path, args: &[&str]) -> serde_json::Value { + let mut command = cli(); + command.arg("cluster"); + for arg in args { + command.arg(arg); + } + let output = command + .arg("--config") + .arg(dir) + .arg("--json") + .output() + .unwrap(); + let stdout = String::from_utf8_lossy(&output.stdout); + serde_json::from_str(stdout.trim()).unwrap_or_else(|err| { + panic!( + "cluster {args:?} produced unparseable output ({err}): stdout={stdout} stderr={}", + String::from_utf8_lossy(&output.stderr) + ) + }) +} + +fn write_two_graph_cluster(dir: &std::path::Path) { + std::fs::write( + dir.join("people.pg"), + "\nnode Person {\n name: String @key\n}\n", + ) + .unwrap(); + std::fs::write( + dir.join("services.pg"), + "\nnode Service {\n name: String @key\n}\n", + ) + .unwrap(); + std::fs::write( + dir.join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + std::fs::write( + dir.join("services.gq"), + "\nquery find_service($name: String) {\n match { $s: Service { name: $name } }\n return { $s.name }\n}\n", + ) + .unwrap(); + std::fs::write( + dir.join("cluster.yaml"), + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq + engineering: + schema: ./services.pg + queries: + find_service: + file: ./services.gq +"#, + ) + .unwrap(); +} + +fn seed_graph(dir: &std::path::Path, graph: &str, row: &str) { + let data = dir.join(format!("{graph}-seed.jsonl")); + std::fs::write(&data, row).unwrap(); + let output = cli() + .arg("load") + .arg("--data") + .arg(&data) + .arg(dir.join(format!("graphs/{graph}.omni"))) + .output() + .unwrap(); + assert!( + output.status.success(), + "seed {graph} failed: {}", + String::from_utf8_lossy(&output.stderr) + ); +} + +fn invoke_query( + client: &Client, + base_url: &str, + graph: &str, + query: &str, + params: serde_json::Value, +) -> (u16, serde_json::Value) { + let response = client + .post(format!("{base_url}/graphs/{graph}/queries/{query}")) + .json(&serde_json::json!({ "params": params })) + .send() + .unwrap(); + let status = response.status().as_u16(); + let body = response.json().unwrap_or(serde_json::Value::Null); + (status, body) +} + +/// The whole control-plane story in one test: declare two graphs → converge +/// (apply creates them) → serve → evolve schema+query in one apply → restart +/// serves the new shape → out-of-band drift converged back → approved graph +/// delete → restart serves the survivor only → plan empty. +#[test] +fn local_cluster_full_lifecycle_declare_serve_evolve_delete() { + let temp = tempfile::tempdir().unwrap(); + let dir = temp.path(); + write_two_graph_cluster(dir); + + // Phase 1-2: declare + record. + assert_eq!(cluster_cli(dir, &["import"])["ok"], true); + // Phase 3-4: one apply creates both graphs and publishes the catalog. + let converge = cluster_cli(dir, &["apply"]); + assert_eq!(converge["converged"], true, "{converge}"); + seed_graph(dir, "knowledge", "{\"type\":\"Person\",\"data\":{\"name\":\"Ada\"}}\n"); + seed_graph(dir, "engineering", "{\"type\":\"Service\",\"data\":{\"name\":\"billing\"}}\n"); + + // Phase 5: serve the applied revision. + let client = Client::new(); + { + let server = spawn_server_with_cluster(dir); + let (status, body) = invoke_query( + &client, + &server.base_url, + "knowledge", + "find_person", + serde_json::json!({"name": "Ada"}), + ); + assert_eq!(status, 200, "{body}"); + assert_eq!(body["rows"][0]["p.name"], "Ada", "{body}"); + let (status, body) = invoke_query( + &client, + &server.base_url, + "engineering", + "find_service", + serde_json::json!({"name": "billing"}), + ); + assert_eq!(status, 200, "{body}"); + assert_eq!(body["rows"][0]["s.name"], "billing", "{body}"); + } + + // Evolve: schema gains a field, the query returns it — one apply, with + // the migration previewed in plan. + std::fs::write( + dir.join("people.pg"), + "\nnode Person {\n name: String @key\n bio: String?\n}\n", + ) + .unwrap(); + std::fs::write( + dir.join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name, $p.bio }\n}\n", + ) + .unwrap(); + let plan = cluster_cli(dir, &["plan"]); + let schema_change = plan["changes"] + .as_array() + .unwrap() + .iter() + .find(|change| change["resource"] == "schema.knowledge") + .unwrap(); + assert_eq!(schema_change["migration"]["supported"], true, "{plan}"); + let evolve = cluster_cli(dir, &["apply"]); + assert_eq!(evolve["converged"], true, "{evolve}"); + + // Restart: the server serves the evolved shape. + { + let server = spawn_server_with_cluster(dir); + let (status, body) = invoke_query( + &client, + &server.base_url, + "knowledge", + "find_person", + serde_json::json!({"name": "Ada"}), + ); + assert_eq!(status, 200, "{body}"); + assert!( + body["columns"] + .as_array() + .unwrap() + .iter() + .any(|column| column == "p.bio"), + "evolved query must project the new field: {body}" + ); + } + + // Out-of-band drift: the live graph evolves behind the cluster's back; + // refresh observes it, apply converges it back to the declared schema. + std::fs::write( + dir.join("rogue.pg"), + "\nnode Person {\n name: String @key\n bio: String?\n rogue: String?\n}\n", + ) + .unwrap(); + let output = cli() + .arg("schema") + .arg("apply") + .arg(dir.join("graphs/knowledge.omni")) + .arg("--schema") + .arg(dir.join("rogue.pg")) + .arg("--json") + .output() + .unwrap(); + assert!(output.status.success(), "out-of-band schema apply failed"); + let refresh = cluster_cli(dir, &["refresh"]); + assert_eq!( + refresh["resource_statuses"]["schema.knowledge"]["status"], + "drifted", + "{refresh}" + ); + let heal = cluster_cli(dir, &["apply"]); + assert_eq!(heal["converged"], true, "{heal}"); + let schema_show = cli() + .arg("schema") + .arg("show") + .arg(dir.join("graphs/knowledge.omni")) + .output() + .unwrap(); + assert!( + !String::from_utf8_lossy(&schema_show.stdout).contains("rogue"), + "drift must be soft-dropped back to the declared schema" + ); + + // Retire engineering: gated delete, then the server serves the survivor. + std::fs::write( + dir.join("cluster.yaml"), + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +"#, + ) + .unwrap(); + let blocked = cluster_cli(dir, &["apply"]); + assert_eq!(blocked["converged"], false, "{blocked}"); + let approve_output = cli() + .arg("--as") + .arg("andrew") + .arg("cluster") + .arg("approve") + .arg("graph.engineering") + .arg("--config") + .arg(dir) + .arg("--json") + .output() + .unwrap(); + assert!(approve_output.status.success(), "approve failed"); + let delete = cluster_cli(dir, &["apply"]); + assert_eq!(delete["converged"], true, "{delete}"); + assert!(!dir.join("graphs/engineering.omni").exists()); + + { + let server = spawn_server_with_cluster(dir); + let (status, body) = invoke_query( + &client, + &server.base_url, + "knowledge", + "find_person", + serde_json::json!({"name": "Ada"}), + ); + assert_eq!(status, 200, "{body}"); + let response = client + .post(format!( + "{}/graphs/engineering/queries/find_service", + server.base_url + )) + .json(&serde_json::json!({"params": {"name": "billing"}})) + .send() + .unwrap(); + assert_eq!( + response.status().as_u16(), + 404, + "a deleted graph must vanish from the serving surface" + ); + } + + // The story ends converged: nothing left to do. + let final_plan = cluster_cli(dir, &["plan"]); + assert!( + final_plan["changes"].as_array().unwrap().is_empty(), + "{final_plan}" + ); +} + +/// Applied policy bundles gate serving per their bindings: the cluster-bound +/// bundle owns the management surface (graph_list), the graph-bound bundle +/// owns query invocation — enforced over HTTP with bearer-resolved actors. +#[test] +fn local_cluster_serving_enforces_applied_policy_bindings() { + let temp = tempfile::tempdir().unwrap(); + let dir = temp.path(); + std::fs::write( + dir.join("people.pg"), + "\nnode Person {\n name: String @key\n}\n", + ) + .unwrap(); + std::fs::write( + dir.join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + std::fs::write( + dir.join("graph.policy.yaml"), + r#" +version: 1 +groups: + readers: ["act-reader"] +protected_branches: [main] +rules: + - id: allow-invoke + allow: + actors: { group: readers } + actions: [invoke_query] + - id: allow-read + allow: + actors: { group: readers } + actions: [read] + branch_scope: any +"#, + ) + .unwrap(); + std::fs::write( + dir.join("server.policy.yaml"), + r#" +version: 1 +kind: server +groups: + admins: ["act-admin"] +rules: + - id: allow-list + allow: + actors: { group: admins } + actions: [graph_list] +"#, + ) + .unwrap(); + std::fs::write( + dir.join("cluster.yaml"), + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +policies: + graph_rules: + file: ./graph.policy.yaml + applies_to: [knowledge] + server_rules: + file: ./server.policy.yaml + applies_to: [cluster] +"#, + ) + .unwrap(); + assert_eq!(cluster_cli(dir, &["import"])["ok"], true); + let converge = cluster_cli(dir, &["apply"]); + assert_eq!(converge["converged"], true, "{converge}"); + seed_graph(dir, "knowledge", "{\"type\":\"Person\",\"data\":{\"name\":\"Ada\"}}\n"); + + let server = spawn_server_with_cluster_env( + dir, + &[( + "OMNIGRAPH_SERVER_BEARER_TOKENS_JSON", + r#"{"act-admin":"admin-token","act-reader":"reader-token"}"#, + )], + ); + let client = Client::new(); + let get_graphs = |token: Option<&str>| { + let mut request = client.get(format!("{}/graphs", server.base_url)); + if let Some(token) = token { + request = request.bearer_auth(token); + } + request.send().unwrap().status().as_u16() + }; + // Management surface: cluster-bound bundle, admins only. + assert_eq!(get_graphs(Some("admin-token")), 200); + assert_eq!(get_graphs(Some("reader-token")), 403); + assert_eq!(get_graphs(None), 401); + + // Query invocation: graph-bound bundle, readers only. + let invoke = |token: &str| { + client + .post(format!( + "{}/graphs/knowledge/queries/find_person", + server.base_url + )) + .bearer_auth(token) + .json(&serde_json::json!({"params": {"name": "Ada"}})) + .send() + .unwrap() + }; + let response = invoke("reader-token"); + assert_eq!(response.status().as_u16(), 200); + let body: serde_json::Value = response.json().unwrap(); + assert_eq!(body["rows"][0]["p.name"], "Ada", "{body}"); + // Denied invocation is deliberately 404, indistinguishable from an + // unknown query — the server's anti-probing contract. + assert_eq!(invoke("admin-token").status().as_u16(), 404); +} diff --git a/docs/dev/testing.md b/docs/dev/testing.md index eba70c9..9de80f7 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -7,7 +7,7 @@ This file is the always-on map of the test surface. **Consult it before every ta | Crate | Path | Style | |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | -| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish; includes the `cluster_e2e_*` lifecycle compositions over the spawned binary — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `system_local.rs`, `system_remote.rs`, share `tests/support/mod.rs` | +| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish; includes the `cluster_e2e_*` lifecycle compositions over the spawned binary — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `system_local.rs` (incl. the full-cycle cluster lifecycle with a spawned `--cluster` server — declare→serve→evolve→drift-heal→approved-delete — and applied-policy enforcement over HTTP), `system_remote.rs`, share `tests/support/mod.rs` | | `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows), and 5A policy binding metadata (applies_to in the applied revision, binding-change diffing + convergence, pre-5A backfill), and the 5B serving-snapshot read API (converged read, refusal rows) | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level; incl. cluster-mode boot — converged-dir serving, policy binding wiring, boot refusals), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | From 711e04a161655c95c7a46e7f9a13bd64cb6a7473 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 18:44:05 +0300 Subject: [PATCH 080/165] ci: pin RustFS to 1.0.0-beta.8 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit beta.4+ refuses the rustfsadmin/rustfsadmin test credentials unless RUSTFS_ALLOW_INSECURE_DEFAULT_CREDENTIALS=true is set — acceptable for the ephemeral CI container and the local bootstrap script (which already passed it). The three S3 suites were validated against the beta.8 binary locally before this bump. The pin stays explicit, never `latest`, so future upgrades remain deliberate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- .github/workflows/ci.yml | 17 ++++++++--------- scripts/local-rustfs-bootstrap.sh | 14 ++++++-------- 2 files changed, 14 insertions(+), 17 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1ea6c37..15e6d11 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -306,14 +306,12 @@ jobs: . -> target - name: Start RustFS - # Pinned to 1.0.0-beta.3 (2026-05-14) — the last known-good tag. - # `rustfs/rustfs:latest` (1.0.0-beta.4, 2026-05-21) added a - # credentials-policy check that refuses to start when - # AWS_ACCESS_KEY_ID/SECRET_ACCESS_KEY are values it considers - # "default" (rustfsadmin/rustfsadmin in our case). Bumping to - # beta.4+ requires either rotating those creds to less-default - # values or setting RUSTFS_ALLOW_INSECURE_DEFAULT_CREDENTIALS=true - # — deliberate work, not an emergency. Pin first; upgrade later. + # Pinned to 1.0.0-beta.8 (2026-06-10). beta.4+ refuses "default" + # credentials (rustfsadmin/rustfsadmin) unless + # RUSTFS_ALLOW_INSECURE_DEFAULT_CREDENTIALS=true is set — fine for + # an ephemeral CI container. The three S3 suites were validated + # against the beta.8 binary locally before this bump. Keep the pin + # explicit (never `latest`) so upgrades are deliberate. run: | docker rm -f rustfs >/dev/null 2>&1 || true docker run -d \ @@ -322,7 +320,8 @@ jobs: -p 9001:9001 \ -e RUSTFS_ACCESS_KEY="${AWS_ACCESS_KEY_ID}" \ -e RUSTFS_SECRET_KEY="${AWS_SECRET_ACCESS_KEY}" \ - rustfs/rustfs:1.0.0-beta.3 \ + -e RUSTFS_ALLOW_INSECURE_DEFAULT_CREDENTIALS=true \ + rustfs/rustfs:1.0.0-beta.8 \ /data - name: Install AWS CLI diff --git a/scripts/local-rustfs-bootstrap.sh b/scripts/local-rustfs-bootstrap.sh index c4fdcbe..2425c77 100755 --- a/scripts/local-rustfs-bootstrap.sh +++ b/scripts/local-rustfs-bootstrap.sh @@ -6,14 +6,12 @@ SOURCE_REF="${SOURCE_REF:-main}" RELEASE_CHANNEL="${RELEASE_CHANNEL:-edge}" WORKDIR="${WORKDIR:-$PWD/.omnigraph-rustfs-demo}" RUSTFS_CONTAINER_NAME="${RUSTFS_CONTAINER_NAME:-omnigraph-rustfs-demo}" -# Pinned to 1.0.0-beta.3 (2026-05-14) — the last known-good tag, matching CI -# (.github/workflows/ci.yml). `rustfs/rustfs:latest` (1.0.0-beta.4, 2026-05-21) -# added a credentials-policy check that refuses to start when the access/secret -# keys are values it considers "default" (rustfsadmin/rustfsadmin here). This -# script still works on beta.4+ because it passes -# RUSTFS_ALLOW_INSECURE_DEFAULT_CREDENTIALS=true below — so overriding -# RUSTFS_IMAGE to a newer tag is safe. -RUSTFS_IMAGE="${RUSTFS_IMAGE:-rustfs/rustfs:1.0.0-beta.3}" +# Pinned to 1.0.0-beta.8 (2026-06-10), matching CI (.github/workflows/ci.yml). +# beta.4+ has a credentials-policy check that refuses to start when the +# access/secret keys are values it considers "default" (rustfsadmin/rustfsadmin +# here); this script passes RUSTFS_ALLOW_INSECURE_DEFAULT_CREDENTIALS=true +# below, so overriding RUSTFS_IMAGE to another tag is safe. +RUSTFS_IMAGE="${RUSTFS_IMAGE:-rustfs/rustfs:1.0.0-beta.8}" RUSTFS_DATA_DIR="${RUSTFS_DATA_DIR:-$WORKDIR/rustfs-data}" BUCKET="${BUCKET:-omnigraph-local}" PREFIX="${PREFIX:-repos/context}" From d8354ac21332caed50ebc4ecb69ab8815360ffba Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 19:05:12 +0300 Subject: [PATCH 081/165] =?UTF-8?q?test(cli):=20address=20review=20?= =?UTF-8?q?=E2=80=94=20assert=20schema-show=20success,=20document=20exit-c?= =?UTF-8?q?ode=20stance,=20add=20e2e=20opt-out?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - The drift-heal verification now asserts `schema show` succeeded and produced a schema before checking the rogue field's absence (a failed command previously made the negative assertion vacuously pass). - cluster_cli documents why it deliberately does not assert exit codes (blocked applies exit non-zero by contract while emitting the structured output callers assert on). - The comprehensive lifecycle e2es honor OMNIGRAPH_SKIP_SYSTEM_E2E=1 (graceful skip-with-message, the S3-gate pattern) for constrained sandboxes; requirements + suppression documented in testing.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/system_local.rs | 39 ++++++++++++++++++++-- docs/dev/testing.md | 4 +++ 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/crates/omnigraph-cli/tests/system_local.rs b/crates/omnigraph-cli/tests/system_local.rs index 14b8890..4b5e4b6 100644 --- a/crates/omnigraph-cli/tests/system_local.rs +++ b/crates/omnigraph-cli/tests/system_local.rs @@ -1717,6 +1717,13 @@ graphs: // ---- Comprehensive full-cycle cluster e2e (Phases 1-5 composed) ---- +/// Run a `cluster` subcommand and return its JSON output. Deliberately does +/// NOT assert a zero exit code: blocked/unconverged runs (e.g. an `apply` +/// awaiting an approval) exit non-zero by contract while still emitting the +/// structured output the caller asserts on (`ok`/`converged`/dispositions). +/// Commands where failure is never expected must assert on those fields +/// (every call here checks `ok` or `converged`) or use `cli()` directly with +/// `status.success()`. fn cluster_cli(dir: &std::path::Path, args: &[&str]) -> serde_json::Value { let mut command = cli(); command.arg("cluster"); @@ -1813,12 +1820,30 @@ fn invoke_query( (status, body) } +/// Opt-out for the comprehensive system e2es below. They need no external +/// services — only the workspace-built `omnigraph`/`omnigraph-server` +/// binaries (cargo provides them via `CARGO_BIN_EXE_*`), ephemeral localhost +/// ports, and local-FS temp dirs — but they spawn real server processes and +/// run multi-stage lifecycles, so constrained sandboxes can suppress them: +/// `OMNIGRAPH_SKIP_SYSTEM_E2E=1 cargo test ...` (same skip-with-message +/// pattern as the S3 tests' `OMNIGRAPH_S3_TEST_BUCKET` gate). +fn skip_system_e2e(test_name: &str) -> bool { + if std::env::var("OMNIGRAPH_SKIP_SYSTEM_E2E").is_ok_and(|v| !v.is_empty() && v != "0") { + eprintln!("skipping {test_name}: OMNIGRAPH_SKIP_SYSTEM_E2E is set"); + return true; + } + false +} + /// The whole control-plane story in one test: declare two graphs → converge /// (apply creates them) → serve → evolve schema+query in one apply → restart /// serves the new shape → out-of-band drift converged back → approved graph /// delete → restart serves the survivor only → plan empty. #[test] fn local_cluster_full_lifecycle_declare_serve_evolve_delete() { + if skip_system_e2e("local_cluster_full_lifecycle_declare_serve_evolve_delete") { + return; + } let temp = tempfile::tempdir().unwrap(); let dir = temp.path(); write_two_graph_cluster(dir); @@ -1931,8 +1956,15 @@ fn local_cluster_full_lifecycle_declare_serve_evolve_delete() { .output() .unwrap(); assert!( - !String::from_utf8_lossy(&schema_show.stdout).contains("rogue"), - "drift must be soft-dropped back to the declared schema" + schema_show.status.success(), + "schema show failed: {}", + String::from_utf8_lossy(&schema_show.stderr) + ); + let shown = String::from_utf8_lossy(&schema_show.stdout); + assert!(shown.contains("Person"), "schema show produced no schema: {shown}"); + assert!( + !shown.contains("rogue"), + "drift must be soft-dropped back to the declared schema: {shown}" ); // Retire engineering: gated delete, then the server serves the survivor. @@ -2005,6 +2037,9 @@ graphs: /// owns query invocation — enforced over HTTP with bearer-resolved actors. #[test] fn local_cluster_serving_enforces_applied_policy_bindings() { + if skip_system_e2e("local_cluster_serving_enforces_applied_policy_bindings") { + return; + } let temp = tempfile::tempdir().unwrap(); let dir = temp.path(); std::fs::write( diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 9de80f7..848594a 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -69,6 +69,10 @@ CI runs three S3-backed tests against a containerized RustFS server (`.github/wo Locally, set `OMNIGRAPH_S3_TEST_BUCKET` (and the usual `AWS_*` vars including `AWS_ENDPOINT_URL_S3` for non-AWS) before running. Without those, S3 tests skip gracefully. +## System e2e requirements and suppression + +The CLI system tests (`system_local.rs`) spawn the workspace-built `omnigraph` and `omnigraph-server` binaries (cargo provides paths via `CARGO_BIN_EXE_*`), bind ephemeral localhost ports, and use local-FS temp dirs — no external services, no env vars required; they run in the default `cargo test --workspace`. The comprehensive cluster lifecycle e2es (multi-server-restart flows) honor an opt-out for constrained sandboxes: set `OMNIGRAPH_SKIP_SYSTEM_E2E=1` to skip them with a logged message (the same graceful-skip pattern as the S3 gate). Cargo-native filtering also works: `cargo test --test system_local -- --skip local_cluster`. + ## OpenAPI drift `crates/omnigraph-server/tests/openapi.rs` regenerates `openapi.json` and diffs against the checked-in copy. CI auto-commits the regeneration on same-repository PRs and otherwise runs in strict-check mode (env: `OMNIGRAPH_UPDATE_OPENAPI`). From 97eb65e921c704c09bdd8064e8304ef074cc1184 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 22:10:19 +0300 Subject: [PATCH 082/165] docs(cluster): operator how-to guide for deploying and managing clusters MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New docs/user/cluster.md — the practical companion to cluster-config.md's reference: zero-to-served walkthrough (validate/import/plan/apply, derived roots, data loading, --cluster serving), the day-2 edit->plan->apply->restart loop with a per-change-kind table, drift observation and convergence, the approval gate for destructive changes, crash/lock/lost-ledger recovery, the boot-refusal table with remedies, deployment patterns (replicas, backup unit, CI gating), and the explicit not-yet list (hot reload, S3-hosted cluster dirs, per-query exposure, pipelines). Linked from the user index, the agent guide's topic map, and cross-linked from the reference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- AGENTS.md | 1 + docs/user/cluster-config.md | 3 + docs/user/cluster.md | 256 ++++++++++++++++++++++++++++++++++++ docs/user/index.md | 1 + 4 files changed, 261 insertions(+) create mode 100644 docs/user/cluster.md diff --git a/AGENTS.md b/AGENTS.md index 25243a5..60276ad 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -86,6 +86,7 @@ Full diagram and concurrency model: [docs/dev/architecture.md](docs/dev/architec | Diff / change feed (`diff_between`, `diff_commits`) | [docs/user/changes.md](docs/user/changes.md) | | Query execution, mutation execution, bulk loader, `load` vs `ingest` | [docs/dev/execution.md](docs/dev/execution.md) | | `optimize` (compaction) and `cleanup` (version GC) | [docs/user/maintenance.md](docs/user/maintenance.md) | +| Cluster operator guide (deploy/manage clusters, approvals, recovery, serving) | [docs/user/cluster.md](docs/user/cluster.md) | | Cedar policy actions, scopes, CLI | [docs/user/policy.md](docs/user/policy.md) | | HTTP server endpoints, auth, error model, body limits | [docs/user/server.md](docs/user/server.md) | | CLI quick-start | [docs/user/cli.md](docs/user/cli.md) | diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 5c51b1f..5847d8e 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -2,6 +2,9 @@ **Status:** Phase 5 — cluster-booted serving (`omnigraph-server --cluster`). +> New to the cluster tooling? Start with the operator how-to guide, +> [cluster.md](cluster.md) — this document is the reference. + Cluster config is the future control-plane configuration surface for a whole OmniGraph deployment. In this stage, OmniGraph can validate a local `cluster.yaml` folder, produce a deterministic read-only plan, inspect the diff --git a/docs/user/cluster.md b/docs/user/cluster.md new file mode 100644 index 0000000..6241378 --- /dev/null +++ b/docs/user/cluster.md @@ -0,0 +1,256 @@ +# Operating an OmniGraph Cluster + +This is the operator's guide to the cluster control plane: how to go from an +empty directory to a served deployment, and how to run it day to day — +evolving schemas, rotating queries and policies, healing drift, approving +destructive changes, and recovering from crashes. + +It is a **how-to**. The reference for every `cluster.yaml` key, command flag, +state-file field, and diagnostic code is +[cluster-config.md](cluster-config.md); the HTTP surface is +[server.md](server.md). + +## The model in one paragraph + +You declare the entire deployment — graphs, schemas, stored queries, Cedar +policies — as files in one directory (`cluster.yaml` plus the `.pg`/`.gq`/ +`.yaml` files it references). `cluster apply` converges reality to that +declaration and records what it did in a state ledger +(`__cluster/state.json`); `cluster plan` previews exactly what apply would +do, including real schema-migration steps. A server started with +`omnigraph-server --cluster <dir>` serves what was applied — never what is +merely written in config. Terraform users will recognize the shape: config +is desired state, the ledger is recorded state, plan is the diff, apply is +the only thing that changes the world, and irreversible changes require an +explicitly recorded approval. + +## 1. Deploy a cluster from zero + +Lay out a config directory: + +``` +company-brain/ +├── cluster.yaml +├── people.pg # schema for the "knowledge" graph +├── people.gq # a stored query +└── base.policy.yaml # a Cedar policy bundle +``` + +```yaml +# cluster.yaml +version: 1 +metadata: + name: company-brain +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +policies: + base: + file: ./base.policy.yaml + applies_to: [knowledge] # graph-bound; use [cluster] for server-level +``` + +Bring it to life: + +```bash +omnigraph cluster validate --config ./company-brain # parse + typecheck everything +omnigraph cluster import --config ./company-brain # create the state ledger +omnigraph cluster plan --config ./company-brain # preview: what would apply do? +omnigraph cluster apply --config ./company-brain # converge +``` + +That single `apply` **creates the graph** (at the derived root +`./company-brain/graphs/knowledge.omni`), applies its schema, and publishes +the query and policy into the content-addressed catalog +(`__cluster/resources/…`). The output lists every change with its +disposition; `converged: true` means there is nothing left to do — re-running +`apply` is always safe and idempotent. + +Load data through the normal graph plane (the control plane manages +*definitions*, not rows): + +```bash +omnigraph load --data ./seed.jsonl ./company-brain/graphs/knowledge.omni +``` + +Serve it: + +```bash +OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-reader":"s3cret"}' \ + omnigraph-server --cluster ./company-brain --bind 0.0.0.0:8080 +``` + +`--cluster` is an **exclusive boot source**: it cannot be combined with a +graph URI, `--target`, or `--config`, and `omnigraph.yaml` is never read in +this mode. Routing is always multi-graph: + +```bash +curl -H 'authorization: Bearer s3cret' \ + -X POST http://localhost:8080/graphs/knowledge/queries/find_person \ + -H 'content-type: application/json' -d '{"params":{"name":"Ada"}}' +``` + +Bearer tokens and the bind address are deliberately *not* cluster facts — +they are per-replica, set by flag or environment +([server.md](server.md#modes) for the token sources). + +## 2. The day-2 loop: edit → plan → apply → restart + +Every change follows the same loop, whatever its kind: + +```bash +$EDITOR company-brain/people.pg # or any .gq / policy / cluster.yaml edit +omnigraph cluster plan --config ./company-brain +omnigraph cluster apply --config ./company-brain --as andrew +# restart cluster-booted servers to pick it up +``` + +`--as <actor>` attributes the run: it is recorded in recovery sidecars and +audit entries and threaded into the engine's commit history. Make it a habit +on every apply (it is required for `approve`). + +What each change kind does: + +| You edit | Plan shows | Apply does | +|---|---|---| +| a `.gq` file or `queries:` entry | `Update query.<g>.<n>` | publishes the new content-addressed blob, updates the ledger | +| a policy file | `Update policy.<n>` | same — new blob, ledger update | +| a policy's `applies_to` | `Update policy.<n> [bindings]` | records the new bindings (the file digest is unchanged; bindings are first-class changes) | +| a `.pg` schema | `Update schema.<g>` **with the real migration steps embedded** | runs the engine's schema apply on the live graph — soft drops only, sidecar-fenced | +| `graphs:` gains an entry | `Create graph.<g>` (+ schema, queries) | initializes the graph at its derived root; dependents apply in the same run | +| `graphs:` loses an entry | `Delete graph.<g>` — **blocked, `approval_required`** | nothing, until approved (see §4) | + +Two properties worth internalizing: + +- **One apply, ordered correctly.** Creates run first, then schema + migrations, then catalog writes, then (approved) deletes — so a schema + change plus a query that uses the new field converge together in one run. +- **Soft drops only.** A removed schema property disappears from the current + version while prior versions retain the data (reversible until `cleanup`). + Data-loss migrations are not reachable from cluster apply. + +Read the plan before applying when the change is non-trivial — for schema +updates it embeds the engine's actual migration plan (`add_property`, +`drop_property [soft]`, `unsupported: …`), so you see data impact before +anything runs. + +## 3. Inspect: status, refresh, drift + +```bash +omnigraph cluster status --config ./company-brain --json # ledger only, read-only +omnigraph cluster refresh --config ./company-brain # re-observe live graphs +``` + +`status` never touches the graphs; `refresh` opens them read-only and +records what it finds — manifest versions, live schema digests, catalog blob +integrity. If someone changed a graph behind the control plane's back (a +direct `omnigraph schema apply`, a tampered catalog file), refresh marks the +resource **`drifted`**. + +**Drift is converged, not just reported.** After a refresh records drift, +the next `plan` proposes migrating the live graph back to the declared +schema — with the steps visible, including the soft drops of out-of-band +fields — and `apply` executes it like any other change. If the out-of-band +change is the one you want, change the *config* to match instead, and apply +converges the ledger. + +## 4. Destructive changes: the approval gate + +Removing a graph from `cluster.yaml` never executes silently: + +```bash +omnigraph cluster apply --config ./company-brain +# Delete graph.scratch [Blocked: approval_required] + +omnigraph cluster approve graph.scratch --config ./company-brain --as andrew +# cluster approve: delete graph.scratch approved by andrew (approval 01KT…) + +omnigraph cluster apply --config ./company-brain --as andrew +# Delete graph.scratch [Applied] ← root removed, subtree tombstoned +``` + +The approval artifact (`__cluster/approvals/<id>.json`) is **digest-bound**: +it authorizes exactly the change you saw when you approved it. Any config or +state movement afterwards invalidates it automatically (`approval_stale` +warning) — a stale approval can never authorize a different delete. One +approval covers the graph's whole subtree (its schema and queries ride +along). Consumed artifacts are kept (rewritten with `consumed_at`) and +summarized in the ledger's `approval_records`, so the audit trail of *who +approved what* survives the loss of either store. + +## 5. When things go wrong + +**Crashes are designed for.** Every graph-moving operation (create, schema +apply, delete) writes a recovery sidecar before acting. If an apply dies +mid-run, the next state-mutating command sweeps the sidecars and reconciles +— rolling the ledger forward when the operation completed on the graph, +retiring stale intent when nothing moved, and flagging anything it cannot +verify. You generally fix a crashed run by **running `cluster apply` +again**. + +**A held lock** (a crashed process left `__cluster/lock.json`): + +```bash +omnigraph cluster status --config ./company-brain # shows the lock holder + id +omnigraph cluster force-unlock <LOCK_ID> --config ./company-brain +``` + +Force-unlock requires the exact lock id (from status) — there is no blind +unlock. + +**A lost or corrupted state ledger**: the cluster is self-describing. +`cluster import` rebuilds `state.json` from the config plus read-only +observation of the live graphs; the next `apply` re-converges onto the same +content-addressed catalog. + +**A server that refuses to boot** with `--cluster` is telling you the +applied revision is not safely servable. Each refusal names its remedy: + +| Boot error | Meaning | Remedy | +|---|---|---| +| `cluster_state_missing` | no ledger | `cluster import`, then `apply` | +| `cluster_recovery_pending` | interrupted operation awaiting sweep | run `cluster apply` (or any state-mutating command), restart | +| `catalog_payload_missing` / `…_digest_mismatch` | catalog blob lost or tampered | `cluster refresh`, then `apply`, restart | +| `policy_bindings_missing` | ledger predates binding metadata | re-run `cluster apply` (backfills), restart | +| `cluster_empty` | applied revision has no graphs | apply a cluster with ≥1 graph | +| multiple bundles bind one scope | serving holds one policy bundle per graph + one server-level | split or merge bundles | + +A held *state lock* is deliberately **not** a boot error — the server reads +the atomically-replaced ledger without locking, so serving never contends +with an in-flight apply. + +## 6. Deployment patterns + +- **Replicas**: any number of `--cluster` servers can serve the same config + directory; boot is read-only. Roll out a change by `apply` once, then + restarting replicas (serving is static per process — there is no hot + reload yet). +- **The directory is the deployable unit**: config, catalog, ledger, + approvals, and graph data all live under it. Back it up as a whole; + version the *config files* (not `__cluster/` or `graphs/`) in git. +- **CI-driven convergence**: `validate` and `plan --json` are read-only and + safe in pipelines; gate `apply --as ci` on plan review. Approvals are the + human step by design — keep `cluster approve` out of automation. +- **`omnigraph.yaml` still has a job**: per-operator settings (CLI defaults, + credentials, active context). It just no longer describes the deployment — + a server boots from one source or the other, never a merge of both. + +## What the control plane does not do (yet) + +- **No hot reload** — applied changes serve on the next restart. +- **No S3-hosted cluster directories** — the config dir, ledger, catalog, + and derived graph roots are local-filesystem paths today. (Individual + *graphs* on S3 are a server feature outside cluster mode.) +- **No data operations** — rows move through `omnigraph load / ingest / + mutate` against the graph roots, with branches and merges as usual. +- **Stored-query exposure is all-or-nothing per cluster** — every applied + query is listed and invokable (subject to Cedar `invoke_query`); per-query + exposure policy is a planned phase. +- **Pipelines (ETL)** are a separate project; the `pipelines:` key is + reserved and rejected loudly. + +For the full reference — every key, flag, status, disposition, and +diagnostic — see [cluster-config.md](cluster-config.md). diff --git a/docs/user/index.md b/docs/user/index.md index 6cf6ade..956fa0b 100644 --- a/docs/user/index.md +++ b/docs/user/index.md @@ -13,6 +13,7 @@ of MRs, internal recovery mechanics, or contributor-only invariants. | Install OmniGraph | [install.md](install.md) | | Run the CLI locally | [cli.md](cli.md) | | Look up every CLI flag and config field | [cli-reference.md](cli-reference.md) | +| Deploy and operate a cluster (how-to guide) | [cluster.md](cluster.md) | | Validate and plan cluster config | [cluster-config.md](cluster-config.md) | | Write schemas | [schema-language.md](schema-language.md) | | Read schema-lint diagnostic codes | [schema-lint.md](schema-lint.md) | From f3374ac6dcf36db1512ffe8b79070b15b5fdc35c Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 22:29:49 +0300 Subject: [PATCH 083/165] feat(cli): resolve cluster actor via the per-operator config cascade MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cluster FACTS stay unlayered (cluster.yaml only), but the operator's identity is a per-operator fact — exactly the per-operator omnigraph.yaml's permanent job, and the cascade every data-plane write already uses. cluster apply/approve now resolve: --as flag wins and skips any config read entirely (containers and CI stay config-free); without it, the standard cwd search supplies cli.actor, with a malformed config failing loudly and actionably ('pass --as to skip this lookup') rather than silently dropping attribution. approve's no-actor error now names both sources. Tests pin the contract from both sides: cli.actor is the no-flag default for apply (echoed actor) and approve (approved_by), the flag overrides it, a malformed omnigraph.yaml in cwd breaks nothing except the no-flag actor lookup, and a conflicting well-formed one leaks nothing into cluster outputs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 40 ++++-- crates/omnigraph-cli/tests/cli.rs | 228 ++++++++++++++++++++++++++++++ 2 files changed, 254 insertions(+), 14 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index da4f8e8..dab83d1 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -6,7 +6,7 @@ use std::path::PathBuf; use std::sync::Arc; use clap::{Arg, ArgAction, Args, CommandFactory, FromArgMatches, Parser, Subcommand, ValueEnum}; -use color_eyre::eyre::{Result, bail}; +use color_eyre::eyre::{Result, WrapErr, bail}; use omnigraph::db::{Omnigraph, ReadTarget, SnapshotId}; use omnigraph::loader::LoadMode; use omnigraph::storage::normalize_root_uri; @@ -1257,6 +1257,22 @@ async fn open_local_db_with_policy(graph: &ResolvedCliGraph) -> Result<Omnigraph /// policy is configured and this returns `None`, the engine-layer /// footgun guard intentionally denies — silent bypass via "I forgot the /// actor" is what the guard prevents. +/// Actor resolution for cluster operations. Cluster FACTS stay unlayered +/// (cluster.yaml only), but the operator's identity is a per-operator fact — +/// the per-operator config's permanent job. An explicit --as never touches +/// any config (containers and CI stay config-free); without it, the standard +/// cwd omnigraph.yaml search supplies `cli.actor`, and a malformed config +/// fails loudly rather than silently dropping attribution. +fn resolve_cluster_actor(cli_as: Option<&str>) -> Result<Option<String>> { + if let Some(actor) = cli_as { + return Ok(Some(actor.to_string())); + } + let config = load_cli_config(None).wrap_err( + "resolving the default actor from the per-operator omnigraph.yaml (pass --as <ACTOR> to skip this lookup)", + )?; + Ok(config.cli.actor.clone()) +} + fn resolve_cli_actor<'a>(cli_as: Option<&'a str>, config: &'a OmnigraphConfig) -> Option<&'a str> { cli_as.or(config.cli.actor.as_deref()) } @@ -3610,16 +3626,12 @@ async fn main() -> Result<()> { finish_cluster_plan(&output, json)?; } ClusterCommand::Apply { config, json } => { - // The global --as actor attributes graph-moving operations - // (sidecars, audit entries, engine schema-apply commits). - // Cluster config stays unlayered: no omnigraph.yaml fallback. - let output = apply_config_dir_with_options( - config, - ApplyOptions { - actor: cli.as_actor.clone(), - }, - ) - .await; + // The actor attributes graph-moving operations (sidecars, + // audit entries, engine schema-apply commits). Cluster FACTS + // stay unlayered; the operator's identity resolves --as flag + // first, then the per-operator omnigraph.yaml `cli.actor`. + let actor = resolve_cluster_actor(cli.as_actor.as_deref())?; + let output = apply_config_dir_with_options(config, ApplyOptions { actor }).await; finish_cluster_apply(&output, json)?; } ClusterCommand::Approve { @@ -3627,12 +3639,12 @@ async fn main() -> Result<()> { config, json, } => { - let Some(approver) = cli.as_actor.as_deref() else { + let Some(approver) = resolve_cluster_actor(cli.as_actor.as_deref())? else { bail!( - "`cluster approve` requires the global --as <ACTOR> flag: an approval without an approver is meaningless" + "`cluster approve` requires an approver: pass the global --as <ACTOR> flag or set `cli.actor` in your omnigraph.yaml — an approval without an approver is meaningless" ); }; - let output = approve_config_dir(config, &resource, approver).await; + let output = approve_config_dir(config, &resource, &approver).await; finish_cluster_approve(&output, json)?; } ClusterCommand::Status { config, json } => { diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index e4590f6..00582a7 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -4325,3 +4325,231 @@ fn queries_validate_positional_uri_ignores_default_graph() { "positional URI must validate the top-level registry, not the cli.graph default; stdout:\n{stdout}" ); } + +// ---- per-operator local config (omnigraph.yaml) vs the cluster surfaces ---- + +/// Cluster ops resolve operator identity per-operator: --as wins, and +/// without it the cwd omnigraph.yaml's `cli.actor` is the default. +#[test] +fn cluster_apply_uses_cli_actor_from_local_config() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + fs::write( + temp.path().join("omnigraph.yaml"), + "cli:\n actor: act-local\n", + ) + .unwrap(); + let run = |extra: &[&str]| { + let mut command = cli(); + command.current_dir(temp.path()); + for arg in extra { + command.arg(arg); + } + let output = command + .arg("cluster") + .arg("import") + .arg("--config") + .arg(temp.path()) + .arg("--json") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + // apply, capturing the echoed actor + let mut command = cli(); + command.current_dir(temp.path()); + for arg in extra { + command.arg(arg); + } + let output = command + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json") + .output() + .unwrap(); + let json: serde_json::Value = + serde_json::from_str(String::from_utf8_lossy(&output.stdout).trim()).unwrap(); + json["actor"].clone() + }; + assert_eq!(run(&[]), "act-local", "cli.actor is the no-flag default"); + + // A fresh dir (state already imported above): the flag wins over config. + let mut command = cli(); + command.current_dir(temp.path()); + let output = command + .arg("--as") + .arg("andrew") + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json") + .output() + .unwrap(); + let json: serde_json::Value = + serde_json::from_str(String::from_utf8_lossy(&output.stdout).trim()).unwrap(); + assert_eq!(json["actor"], "andrew", "--as overrides cli.actor"); +} + +#[test] +fn cluster_approve_uses_cli_actor_fallback() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + fs::write( + temp.path().join("omnigraph.yaml"), + "cli:\n actor: act-local\n", + ) + .unwrap(); + // Converge, then remove the graph so a gated delete is pending. + for command in ["import", "apply"] { + let output = cli() + .current_dir(temp.path()) + .arg("cluster") + .arg(command) + .arg("--config") + .arg(temp.path()) + .output() + .unwrap(); + assert!(output.status.success(), "cluster {command} failed"); + } + fs::write(temp.path().join("cluster.yaml"), "version: 1\ngraphs: {}\n").unwrap(); + + let output = cli() + .current_dir(temp.path()) + .arg("cluster") + .arg("approve") + .arg("graph.knowledge") + .arg("--config") + .arg(temp.path()) + .arg("--json") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + let json: serde_json::Value = + serde_json::from_str(String::from_utf8_lossy(&output.stdout).trim()).unwrap(); + assert_eq!(json["approved_by"], "act-local"); + + // With neither flag nor config: refused with the actionable message. + let bare = tempdir().unwrap(); + write_cluster_config_fixture(bare.path()); + let output = output_failure( + cli() + .current_dir(bare.path()) + .arg("cluster") + .arg("approve") + .arg("graph.knowledge") + .arg("--config") + .arg(bare.path()), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stderr.contains("--as"), "{stderr}"); + assert!(stderr.contains("cli.actor"), "{stderr}"); +} + +/// A malformed omnigraph.yaml in the cwd must never break cluster commands; +/// it is read for exactly one thing (the actor default when --as is absent), +/// and only that path fails loudly. +#[test] +fn cluster_commands_ignore_malformed_local_config() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + fs::write(temp.path().join("omnigraph.yaml"), "{{{{ not yaml").unwrap(); + + for command in ["validate", "plan", "status"] { + let output = cli() + .current_dir(temp.path()) + .arg("cluster") + .arg(command) + .arg("--config") + .arg(temp.path()) + .arg("--json") + .output() + .unwrap(); + assert!( + output.status.success() || command == "plan", // plan warns state-missing pre-import; still must not config-error + "cluster {command} affected by malformed omnigraph.yaml: {output:?}" + ); + assert!( + !String::from_utf8_lossy(&output.stderr).contains("omnigraph.yaml"), + "cluster {command} touched omnigraph.yaml" + ); + } + // import + apply with an explicit --as: the config is never loaded. + for (command, args) in [("import", vec![]), ("apply", vec!["--as", "andrew"])] { + let mut invocation = cli(); + invocation.current_dir(temp.path()); + for arg in &args { + invocation.arg(arg); + } + let output = invocation + .arg("cluster") + .arg(command) + .arg("--config") + .arg(temp.path()) + .output() + .unwrap(); + assert!( + output.status.success(), + "cluster {command} affected by malformed omnigraph.yaml: {}", + String::from_utf8_lossy(&output.stderr) + ); + } + // Only the no-flag actor lookup is allowed to fail, and loudly. + let output = output_failure( + cli() + .current_dir(temp.path()) + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("omnigraph.yaml") && stderr.contains("--as"), + "the actor-default config read must fail loudly and actionably: {stderr}" + ); +} + +/// A well-formed omnigraph.yaml with a CONFLICTING world view (different +/// graphs, server bind) leaks nothing into cluster outputs. +#[test] +fn cluster_commands_ignore_conflicting_local_config() { + let baseline = tempdir().unwrap(); + write_cluster_config_fixture(baseline.path()); + let with_config = tempdir().unwrap(); + write_cluster_config_fixture(with_config.path()); + fs::write( + with_config.path().join("omnigraph.yaml"), + r#" +server: + bind: 0.0.0.0:9999 +graphs: + phantom: + uri: ./phantom.omni +"#, + ) + .unwrap(); + + let validate = |dir: &std::path::Path| { + let output = cli() + .current_dir(dir) + .arg("cluster") + .arg("validate") + .arg("--config") + .arg(dir) + .arg("--json") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + serde_json::from_str::<serde_json::Value>(String::from_utf8_lossy(&output.stdout).trim()) + .unwrap() + }; + let (a, b) = (validate(baseline.path()), validate(with_config.path())); + // Compare the path-free invariants (paths embed each tempdir). + for key in ["ok", "diagnostics", "resource_digests", "dependencies"] { + assert_eq!(a[key], b[key], "conflicting omnigraph.yaml leaked into cluster validate ({key})"); + } + let leaked = b.to_string(); + assert!(!leaked.contains("phantom") && !leaked.contains("9999"), "{leaked}"); +} From f7368b58a060f24789c6b7d65491460e2f84ae89 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 22:29:49 +0300 Subject: [PATCH 084/165] test(cli): pin --cluster boot isolation from cwd omnigraph.yaml MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A --cluster server process whose cwd contains a MALFORMED omnigraph.yaml boots and serves — proving mode-inference rule 0 returns before any config search can run. New spawn_server_with_cluster_in support helper sets the spawned server's cwd explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/support/mod.rs | 12 ++++++++ crates/omnigraph-cli/tests/system_local.rs | 34 ++++++++++++++++++++++ 2 files changed, 46 insertions(+) diff --git a/crates/omnigraph-cli/tests/support/mod.rs b/crates/omnigraph-cli/tests/support/mod.rs index 855d8e0..c30ed28 100644 --- a/crates/omnigraph-cli/tests/support/mod.rs +++ b/crates/omnigraph-cli/tests/support/mod.rs @@ -218,6 +218,18 @@ pub fn spawn_server_with_cluster(cluster_dir: &Path) -> TestServer { spawn_server_process(command) } +/// Cluster boot with the server process's cwd set explicitly — used to prove +/// rule 0 never touches the cwd omnigraph.yaml search. +pub fn spawn_server_with_cluster_in(cluster_dir: &Path, cwd: &Path) -> TestServer { + let mut command = server_process(); + command + .arg("--cluster") + .arg(cluster_dir) + .arg("--unauthenticated") + .current_dir(cwd); + spawn_server_process(command) +} + pub fn spawn_server_with_cluster_env(cluster_dir: &Path, envs: &[(&str, &str)]) -> TestServer { let mut command = server_process(); command.arg("--cluster").arg(cluster_dir); diff --git a/crates/omnigraph-cli/tests/system_local.rs b/crates/omnigraph-cli/tests/system_local.rs index 4b5e4b6..b2afdab 100644 --- a/crates/omnigraph-cli/tests/system_local.rs +++ b/crates/omnigraph-cli/tests/system_local.rs @@ -2152,3 +2152,37 @@ policies: // unknown query — the server's anti-probing contract. assert_eq!(invoke("admin-token").status().as_u16(), 404); } + +/// Rule 0 (axiom 15): a --cluster server never reads omnigraph.yaml — not +/// even the implicit cwd search. A MALFORMED config in the process cwd must +/// not affect boot or serving. +#[test] +fn cluster_server_boot_ignores_local_config_in_cwd() { + let cluster = tempfile::tempdir().unwrap(); + std::fs::write( + cluster.path().join("people.pg"), + "\nnode Person {\n name: String @key\n}\n", + ) + .unwrap(); + std::fs::write( + cluster.path().join("cluster.yaml"), + "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n", + ) + .unwrap(); + for command in ["import", "apply"] { + let output = cli() + .arg("cluster") + .arg(command) + .arg("--config") + .arg(cluster.path()) + .output() + .unwrap(); + assert!(output.status.success(), "cluster {command} failed"); + } + let cwd = tempfile::tempdir().unwrap(); + std::fs::write(cwd.path().join("omnigraph.yaml"), "{{{{ not yaml").unwrap(); + + let server = spawn_server_with_cluster_in(cluster.path(), cwd.path()); + let response = reqwest::blocking::get(format!("{}/healthz", server.base_url)).unwrap(); + assert!(response.status().is_success()); +} From 99f7f368641130cddffb653fdf8baad2d61194f2 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 22:30:18 +0300 Subject: [PATCH 085/165] docs(cluster): the precise omnigraph.yaml contract The 'Relationship to omnigraph.yaml' section becomes the exact rule set: cluster commands read the per-operator config for exactly one thing (the cli.actor default when --as is omitted), a --cluster server reads it for nothing, and pointing data-plane targets at derived roots is ergonomics, not coupling. Operator guide and CLI reference updated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/user/cli-reference.md | 2 +- docs/user/cluster-config.md | 29 ++++++++++++++++++++--------- docs/user/cluster.md | 15 ++++++++++----- 3 files changed, 31 insertions(+), 15 deletions(-) diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 6d864cc..ecb44b5 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -19,7 +19,7 @@ Top-level command families and subcommands. Graph-targeting commands accept eith | `commit list \| show` | inspect commit graph | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | -| `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | declarative cluster control plane. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`, annotates dispositions, and embeds real schema-migration previews; `apply` converges the cluster — stored-query/policy catalog writes (content-addressed under `__cluster/resources/`), graph creates, schema updates (soft drops only; `--as` records the actor), and graph deletes behind a digest-bound approval from `cluster approve <resource> --as <actor>`; what apply converges is what an `omnigraph-server --cluster <dir>` deployment serves on its next restart (omnigraph.yaml deployments are unaffected); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id | +| `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | declarative cluster control plane. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`, annotates dispositions, and embeds real schema-migration previews; `apply` converges the cluster — stored-query/policy catalog writes (content-addressed under `__cluster/resources/`), graph creates, schema updates (soft drops only; `--as` records the actor), and graph deletes behind a digest-bound approval from `cluster approve <resource> --as <actor>` (`apply`/`approve` default the actor from the per-operator `omnigraph.yaml`'s `cli.actor` when `--as` is omitted; nothing else in that file affects cluster commands); what apply converges is what an `omnigraph-server --cluster <dir>` deployment serves on its next restart (omnigraph.yaml deployments are unaffected); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) | | `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 5847d8e..081bfa2 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -36,15 +36,26 @@ omnigraph cluster force-unlock <LOCK_ID> --config ./company-brain --json ## Relationship to `omnigraph.yaml` `cluster.yaml` does not replace `omnigraph.yaml`, and the two never describe -the same fact. `omnigraph.yaml` remains how the CLI and server are configured -today (graph targets, server bind, CLI defaults, credential env references) and -is its long-term home for per-operator settings. `cluster.yaml` is the shared -desired state of a whole deployment, read only by the `cluster` commands via -`--config`. In the current stage, nothing recorded in the cluster state ledger -affects what a server serves or what other CLI commands target — the cluster -catalog is inspectable, not live. When server boot from cluster state ships in -a later stage, it will be an explicit per-deployment mode switch, not a merge -of the two files. +the same fact. `omnigraph.yaml` is the permanent **per-operator** layer (CLI +defaults, the operator's identity and credential references, graph targets +for data-plane commands); `cluster.yaml` is the shared desired state of a +whole deployment, read only by the `cluster` commands via `--config`. + +The exact contract: + +- **Cluster commands read `omnigraph.yaml` for exactly one thing**: the + `cli.actor` default used by `apply`/`approve` when `--as` is omitted — + operator identity is a per-operator fact. With `--as` present, no config + is read at all. Nothing else (its graph set, targets, bind, queries, + policies) ever influences a cluster command; a malformed `omnigraph.yaml` + breaks only the no-flag actor lookup, loudly. +- **A `--cluster` server reads `omnigraph.yaml` for nothing** — not even the + implicit current-directory search runs (mode-inference rule 0). Boot from + cluster state XOR `omnigraph.yaml`, never a merge. +- **The other direction is ergonomics, not coupling**: a per-operator + `omnigraph.yaml` may point `graphs.<name>.uri` at a cluster's derived root + (`./company-brain/graphs/knowledge.omni`) so data-plane commands can use + `--target <name>` — an ordinary local path, no special handling. ## Supported `cluster.yaml` diff --git a/docs/user/cluster.md b/docs/user/cluster.md index 6241378..dcc4b2b 100644 --- a/docs/user/cluster.md +++ b/docs/user/cluster.md @@ -109,8 +109,10 @@ omnigraph cluster apply --config ./company-brain --as andrew ``` `--as <actor>` attributes the run: it is recorded in recovery sidecars and -audit entries and threaded into the engine's commit history. Make it a habit -on every apply (it is required for `approve`). +audit entries and threaded into the engine's commit history. Set +`cli: { actor: <you> }` in your per-operator `omnigraph.yaml` to make it the +default when `--as` is omitted (the flag always wins; `approve` requires one +of the two). What each change kind does: @@ -234,9 +236,12 @@ with an in-flight apply. - **CI-driven convergence**: `validate` and `plan --json` are read-only and safe in pipelines; gate `apply --as ci` on plan review. Approvals are the human step by design — keep `cluster approve` out of automation. -- **`omnigraph.yaml` still has a job**: per-operator settings (CLI defaults, - credentials, active context). It just no longer describes the deployment — - a server boots from one source or the other, never a merge of both. +- **`omnigraph.yaml` still has a job**: per-operator settings — your + `cli.actor` default for `--as`, CLI defaults, credentials, and data-plane + ergonomics (point `graphs.<name>.uri` at a derived root like + `./company-brain/graphs/knowledge.omni` to use `--target <name>` for + loads). It just no longer describes the deployment — a server boots from + one source or the other, never a merge of both. ## What the control plane does not do (yet) From fbe9726ac76d5797fe5c3115b8f996d343c57596 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 22:34:54 +0300 Subject: [PATCH 086/165] test(cli): stop the S3 e2e scaffolding omnigraph.yaml into the crate dir MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit local_cli_s3_end_to_end_init_load_read_flow ran `omnigraph init` without a current_dir, so init's project scaffold landed in crates/omnigraph-cli/ — poisoning any later test that resolves a graph target from the cwd config (query_lint_requires_schema_or_resolvable_graph_target fails determinis- tically once the file exists). Only manifests when OMNIGRAPH_S3_TEST_BUCKET is set, which is why local FS runs and CI's scoped rustfs job never caught it. The init and load calls now run inside the test's tempdir. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/system_local.rs | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/crates/omnigraph-cli/tests/system_local.rs b/crates/omnigraph-cli/tests/system_local.rs index b2afdab..adb5dc8 100644 --- a/crates/omnigraph-cli/tests/system_local.rs +++ b/crates/omnigraph-cli/tests/system_local.rs @@ -595,8 +595,12 @@ policy: {{}} ), ); + // current_dir matters: `init` scaffolds an omnigraph.yaml into its cwd, + // and without this it pollutes the crate dir, breaking unrelated tests + // (anything resolving a graph target from the cwd config). output_success( cli() + .current_dir(query_root) .arg("init") .arg("--schema") .arg(fixture("test.pg")) @@ -604,6 +608,7 @@ policy: {{}} ); output_success( cli() + .current_dir(query_root) .arg("load") .arg("--data") .arg(fixture("test.jsonl")) From d3ae31be088ae094058c03057018a8a78e858b9e Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 22:35:58 +0300 Subject: [PATCH 087/165] feat(docker): cluster-mode entrypoint and the CLI in the image MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit OMNIGRAPH_CLUSTER boots the container from a mounted cluster directory's applied revision — checked first and exclusive (exit 64 when combined with OMNIGRAPH_TARGET_URI/CONFIG/TARGET), the entrypoint-level mirror of the server's mode-inference rule 0. The omnigraph CLI joins the image so the day-2 loop (cluster apply/approve/status, data loads by explicit URI) runs in-container via docker/ECS exec or railway shell — no omnigraph.yaml required, which the cluster-local-config PR pins. entrypoint_test gains the cluster case plus all three exclusivity refusals. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- .dockerignore | 1 + Dockerfile | 6 +++++- docker/entrypoint.sh | 13 +++++++++++++ docker/entrypoint_test.sh | 20 ++++++++++++++++++++ 4 files changed, 39 insertions(+), 1 deletion(-) diff --git a/.dockerignore b/.dockerignore index ab6a1f8..05ec59a 100644 --- a/.dockerignore +++ b/.dockerignore @@ -2,3 +2,4 @@ !Dockerfile !docker/entrypoint.sh !target/release/omnigraph-server +!target/release/omnigraph diff --git a/Dockerfile b/Dockerfile index e49a6c7..ca22a93 100644 --- a/Dockerfile +++ b/Dockerfile @@ -11,9 +11,13 @@ RUN groupadd --system omnigraph \ && useradd --system --gid omnigraph --create-home --home-dir /var/lib/omnigraph omnigraph COPY target/release/omnigraph-server /usr/local/bin/omnigraph-server +# The CLI ships in the image so the cluster day-2 loop (cluster +# apply/approve/status, data loads by explicit URI) runs in-container via +# `docker exec` / ECS exec / `railway shell` — no omnigraph.yaml required. +COPY target/release/omnigraph /usr/local/bin/omnigraph COPY docker/entrypoint.sh /usr/local/bin/omnigraph-entrypoint -RUN chmod 0755 /usr/local/bin/omnigraph-server /usr/local/bin/omnigraph-entrypoint +RUN chmod 0755 /usr/local/bin/omnigraph-server /usr/local/bin/omnigraph /usr/local/bin/omnigraph-entrypoint ENV OMNIGRAPH_BIND=0.0.0.0:8080 diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh index a5fb275..98587aa 100644 --- a/docker/entrypoint.sh +++ b/docker/entrypoint.sh @@ -9,6 +9,17 @@ fi bind="${OMNIGRAPH_BIND:-0.0.0.0:8080}" +# Cluster mode first, and exclusive (the server's mode-inference rule 0): +# a deployment serves from cluster state XOR omnigraph.yaml, never a merge. +# Fail fast here with the same contract the server enforces. +if [ -n "${OMNIGRAPH_CLUSTER:-}" ]; then + if [ -n "${OMNIGRAPH_TARGET_URI:-}" ] || [ -n "${OMNIGRAPH_CONFIG:-}" ] || [ -n "${OMNIGRAPH_TARGET:-}" ]; then + echo "OMNIGRAPH_CLUSTER is an exclusive boot source; unset OMNIGRAPH_TARGET_URI/OMNIGRAPH_CONFIG/OMNIGRAPH_TARGET" >&2 + exit 64 + fi + exec "$SERVER_BIN" --cluster "${OMNIGRAPH_CLUSTER}" --bind "${bind}" +fi + # URI comes from the env var (the positional arg wins over any config # `graphs` block in resolve_target_uri). OMNIGRAPH_CONFIG, when also set, # is forwarded as --config purely to supply a policy file — the two @@ -28,6 +39,8 @@ fi cat >&2 <<'EOF' omnigraph-server container startup requires one of: + - OMNIGRAPH_CLUSTER (serve a cluster directory's applied revision; + exclusive — cannot combine with the others) - OMNIGRAPH_TARGET_URI - OMNIGRAPH_CONFIG diff --git a/docker/entrypoint_test.sh b/docker/entrypoint_test.sh index 01fbee2..3ee668f 100755 --- a/docker/entrypoint_test.sh +++ b/docker/entrypoint_test.sh @@ -58,6 +58,26 @@ got=$(sh "$ep" some-uri --bind 1.2.3.4:9 --extra) check "explicit args passthrough" \ "ARGS: some-uri --bind 1.2.3.4:9 --extra" "$got" +got=$(OMNIGRAPH_CLUSTER="/var/lib/omnigraph/company-brain" OMNIGRAPH_BIND="0.0.0.0:8080" sh "$ep") +check "CLUSTER only (Phase 5 mode switch)" \ + "ARGS: --cluster /var/lib/omnigraph/company-brain --bind 0.0.0.0:8080" "$got" + +# Exclusivity: OMNIGRAPH_CLUSTER refuses every combination, exit 64. +for combo in "OMNIGRAPH_TARGET_URI=s3://b/g" "OMNIGRAPH_CONFIG=/etc/o.yaml" "OMNIGRAPH_TARGET=active"; do + if out=$(env "$combo" OMNIGRAPH_CLUSTER="/data/cluster" sh "$ep" 2>&1); then + echo "FAIL: CLUSTER + ${combo%%=*} unexpectedly succeeded: $out" + fail=1 + else + status=$? + if [ "$status" -ne 64 ]; then + echo "FAIL: CLUSTER + ${combo%%=*} exited $status, want 64" + fail=1 + else + echo "ok: CLUSTER + ${combo%%=*} refused (64)" + fi + fi +done + if [ "$fail" -ne 0 ]; then echo "entrypoint_test: FAILED" exit 1 From 6b3ae7ac79723d097ee6a50b17235685103a6343 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 22:45:30 +0300 Subject: [PATCH 088/165] docs(deploy): AWS and Railway cluster-mode recipes The container contract (OMNIGRAPH_CLUSTER + mounted volume + token env), ECS/Fargate+EFS and Railway-volume walkthroughs, the in-container day-2 loop, and the honest constraints list (volume mandatory, no hot reload, single-writer apply, shared-volume replicas unvalidated). Operator guide links the recipes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/user/cluster.md | 3 +- docs/user/deployment.md | 65 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 67 insertions(+), 1 deletion(-) diff --git a/docs/user/cluster.md b/docs/user/cluster.md index 6241378..a4a4cae 100644 --- a/docs/user/cluster.md +++ b/docs/user/cluster.md @@ -227,7 +227,8 @@ with an in-flight apply. - **Replicas**: any number of `--cluster` servers can serve the same config directory; boot is read-only. Roll out a change by `apply` once, then restarting replicas (serving is static per process — there is no hot - reload yet). + reload yet). Container/cloud recipes (AWS ECS+EFS, Railway volumes): + [deployment.md](deployment.md#cluster-mode-in-containers-aws-railway). - **The directory is the deployable unit**: config, catalog, ledger, approvals, and graph data all live under it. Back it up as a whole; version the *config files* (not `__cluster/` or `graphs/`) in git. diff --git a/docs/user/deployment.md b/docs/user/deployment.md index 328784f..eb181e3 100644 --- a/docs/user/deployment.md +++ b/docs/user/deployment.md @@ -45,6 +45,71 @@ omnigraph-server s3://my-bucket/graphs/example/releases/2026-04-10-v0.1.0 \ --bind 0.0.0.0:8080 ``` +## Cluster Mode in Containers (AWS, Railway) + +A cluster-booted deployment serves a **cluster directory** (config + state +ledger + content-addressed catalog + graph data) from a mounted volume — the +one structural difference from the stateless S3 single-graph shape, which +needs no volume at all. The container contract: + +```bash +docker run -d \ + -v /srv/company-brain:/var/lib/omnigraph/cluster \ + -e OMNIGRAPH_CLUSTER=/var/lib/omnigraph/cluster \ + -e OMNIGRAPH_SERVER_BEARER_TOKEN=... \ + -p 8080:8080 <image> +``` + +`OMNIGRAPH_CLUSTER` is exclusive: combining it with `OMNIGRAPH_TARGET_URI`, +`OMNIGRAPH_CONFIG`, or `OMNIGRAPH_TARGET` fails fast (exit 64), the same +rule the server itself enforces. The image also ships the `omnigraph` CLI, +so the day-2 loop runs in-container with no `omnigraph.yaml`: + +```bash +docker exec -it <container> sh -c \ + 'omnigraph cluster apply --as andrew --config /var/lib/omnigraph/cluster' +# then restart the container to pick up the applied state +``` + +### AWS (ECS/Fargate + EFS) + +1. Push the image to ECR (the `package.yml` workflow builds it). +2. Create an EFS filesystem; mount it in the task definition at + `/var/lib/omnigraph/cluster`. +3. Task environment: `OMNIGRAPH_CLUSTER=/var/lib/omnigraph/cluster`, bearer + tokens via Secrets Manager/SSM into `OMNIGRAPH_SERVER_BEARER_TOKENS_JSON` + (or the `--features aws` build's native Secrets Manager source). +4. ALB in front for TLS; target the container's 8080 with `/healthz` checks. +5. Day-2: ECS exec into the task → edit/upload config on the volume → + `omnigraph cluster apply --as <you>` → force a new deployment (restart). + +For a deployment that doesn't need the cluster control plane, the classic +stateless shape — `OMNIGRAPH_TARGET_URI=s3://bucket/graph.omni`, no volume — +remains the simplest AWS architecture (see Binary/Container Deployment +above). + +### Railway + +1. Create a service from the image; attach a **volume** mounted at + `/var/lib/omnigraph/cluster`. +2. Variables: `OMNIGRAPH_CLUSTER=/var/lib/omnigraph/cluster`, + `OMNIGRAPH_SERVER_BEARER_TOKEN=<token>`. Railway terminates TLS at its + edge and routes to the exposed 8080. +3. Day-2: `railway shell` (or `railway run`) → `omnigraph cluster apply + --as <you> --config /var/lib/omnigraph/cluster` → redeploy/restart the + service. + +### Constraints (current honest list) + +- **Cluster directories are local-filesystem** — the volume is mandatory; + S3-hosted cluster dirs are not supported. +- **No hot reload** — applied changes serve on the next restart. +- **Single-writer apply** — run `cluster apply` from one place at a time + (the state lock enforces this; CI or one operator shell, not both). +- **Multi-replica serving off a shared volume (EFS) is documented but + unvalidated** — boot is lock-free read-only so it should compose, but it + is not yet exercised by tests. + ## One-Command Local RustFS Bootstrap The easiest local S3-backed deployment path is: From 3b2bf755ae6a74116bfef4353d3c2ee43d431edb Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 22:54:05 +0300 Subject: [PATCH 089/165] =?UTF-8?q?fix(cli):=20address=20review=20?= =?UTF-8?q?=E2=80=94=20honor=20the=20one-thing=20contract,=20restore=20doc?= =?UTF-8?q?s,=20untangle=20test=20phases?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - resolve_cluster_actor uses load_config directly: load_cli_config also loads auth.env_file into the process env — a second thing, violating the documented 'exactly one thing' omnigraph.yaml contract for cluster ops. - resolve_cli_actor gets its doc comment back (the inserted helper had absorbed the contiguous /// block). - The actor-default test imports once as setup and asserts on apply alone, idempotently, instead of re-importing inside the assertion helper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 35 ++++++++++++---------- crates/omnigraph-cli/tests/cli.rs | 49 ++++++++++--------------------- 2 files changed, 34 insertions(+), 50 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index dab83d1..62fff60 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -1251,28 +1251,31 @@ async fn open_local_db_with_policy(graph: &ResolvedCliGraph) -> Result<Omnigraph } } +/// Actor resolution for cluster operations. Cluster FACTS stay unlayered +/// (cluster.yaml only), but the operator's identity is a per-operator fact — +/// the per-operator config's permanent job. An explicit --as never touches +/// any config (containers and CI stay config-free); without it, the standard +/// cwd omnigraph.yaml search supplies `cli.actor`, and a malformed config +/// fails loudly rather than silently dropping attribution. Deliberately +/// `load_config`, NOT `load_cli_config`: the latter also loads +/// `auth.env_file` into the process env — a second thing, violating the +/// documented "exactly one thing" contract. +fn resolve_cluster_actor(cli_as: Option<&str>) -> Result<Option<String>> { + if let Some(actor) = cli_as { + return Ok(Some(actor.to_string())); + } + let config = load_config(None).wrap_err( + "resolving the default actor from the per-operator omnigraph.yaml (pass --as <ACTOR> to skip this lookup)", + )?; + Ok(config.cli.actor.clone()) +} + /// Resolve the CLI's effective actor identity for engine-layer policy /// (MR-722). Precedence: `--as <ACTOR>` (top-level flag) overrides /// `cli.actor` from `omnigraph.yaml`; both unset returns `None`. When /// policy is configured and this returns `None`, the engine-layer /// footgun guard intentionally denies — silent bypass via "I forgot the /// actor" is what the guard prevents. -/// Actor resolution for cluster operations. Cluster FACTS stay unlayered -/// (cluster.yaml only), but the operator's identity is a per-operator fact — -/// the per-operator config's permanent job. An explicit --as never touches -/// any config (containers and CI stay config-free); without it, the standard -/// cwd omnigraph.yaml search supplies `cli.actor`, and a malformed config -/// fails loudly rather than silently dropping attribution. -fn resolve_cluster_actor(cli_as: Option<&str>) -> Result<Option<String>> { - if let Some(actor) = cli_as { - return Ok(Some(actor.to_string())); - } - let config = load_cli_config(None).wrap_err( - "resolving the default actor from the per-operator omnigraph.yaml (pass --as <ACTOR> to skip this lookup)", - )?; - Ok(config.cli.actor.clone()) -} - fn resolve_cli_actor<'a>(cli_as: Option<&'a str>, config: &'a OmnigraphConfig) -> Option<&'a str> { cli_as.or(config.cli.actor.as_deref()) } diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index 00582a7..ab3c23b 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -4339,22 +4339,19 @@ fn cluster_apply_uses_cli_actor_from_local_config() { "cli:\n actor: act-local\n", ) .unwrap(); - let run = |extra: &[&str]| { - let mut command = cli(); - command.current_dir(temp.path()); - for arg in extra { - command.arg(arg); - } - let output = command - .arg("cluster") - .arg("import") - .arg("--config") - .arg(temp.path()) - .arg("--json") - .output() - .unwrap(); - assert!(output.status.success(), "{output:?}"); - // apply, capturing the echoed actor + // Phase 1: import once (setup, not under test). + let output = cli() + .current_dir(temp.path()) + .arg("cluster") + .arg("import") + .arg("--config") + .arg(temp.path()) + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + + // Phase 2: apply alone, capturing the echoed actor (idempotent re-runs). + let apply = |extra: &[&str]| { let mut command = cli(); command.current_dir(temp.path()); for arg in extra { @@ -4372,24 +4369,8 @@ fn cluster_apply_uses_cli_actor_from_local_config() { serde_json::from_str(String::from_utf8_lossy(&output.stdout).trim()).unwrap(); json["actor"].clone() }; - assert_eq!(run(&[]), "act-local", "cli.actor is the no-flag default"); - - // A fresh dir (state already imported above): the flag wins over config. - let mut command = cli(); - command.current_dir(temp.path()); - let output = command - .arg("--as") - .arg("andrew") - .arg("cluster") - .arg("apply") - .arg("--config") - .arg(temp.path()) - .arg("--json") - .output() - .unwrap(); - let json: serde_json::Value = - serde_json::from_str(String::from_utf8_lossy(&output.stdout).trim()).unwrap(); - assert_eq!(json["actor"], "andrew", "--as overrides cli.actor"); + assert_eq!(apply(&[]), "act-local", "cli.actor is the no-flag default"); + assert_eq!(apply(&["--as", "andrew"]), "andrew", "--as overrides cli.actor"); } #[test] From f165145b63f8380de38e30d0bb52453df2189c5b Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Wed, 10 Jun 2026 22:54:26 +0300 Subject: [PATCH 090/165] =?UTF-8?q?docs(deploy):=20address=20review=20?= =?UTF-8?q?=E2=80=94=20consistent=20placeholders,=20complete=20ECS=20comma?= =?UTF-8?q?nd?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The ECS day-2 apply gains its required --config flag (the image ships no omnigraph.yaml, so the CLI cannot locate the cluster dir without it), and the docker-exec example uses the <you> placeholder convention instead of a real-looking actor name. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/user/deployment.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/user/deployment.md b/docs/user/deployment.md index eb181e3..563a501 100644 --- a/docs/user/deployment.md +++ b/docs/user/deployment.md @@ -67,7 +67,7 @@ so the day-2 loop runs in-container with no `omnigraph.yaml`: ```bash docker exec -it <container> sh -c \ - 'omnigraph cluster apply --as andrew --config /var/lib/omnigraph/cluster' + 'omnigraph cluster apply --as <you> --config /var/lib/omnigraph/cluster' # then restart the container to pick up the applied state ``` @@ -81,7 +81,8 @@ docker exec -it <container> sh -c \ (or the `--features aws` build's native Secrets Manager source). 4. ALB in front for TLS; target the container's 8080 with `/healthz` checks. 5. Day-2: ECS exec into the task → edit/upload config on the volume → - `omnigraph cluster apply --as <you>` → force a new deployment (restart). + `omnigraph cluster apply --as <you> --config /var/lib/omnigraph/cluster` + → force a new deployment (restart). For a deployment that doesn't need the cluster control plane, the classic stateless shape — `OMNIGRAPH_TARGET_URI=s3://bucket/graph.omni`, no volume — From 677320ceec391b8dc4ea49a571d8c7fc8ff820bd Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 00:46:21 +0300 Subject: [PATCH 091/165] =?UTF-8?q?feat(cluster):=20Terraform-shaped=20que?= =?UTF-8?q?ry=20declaration=20=E2=80=94=20discover=20from=20files?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit cluster.yaml's graphs.<id>.queries previously accepted only an explicit name->file map, forcing configs to re-enumerate every `query <name>` that the .gq files already declare (the SPIKE cookbook needed 66 entries for 6 files). The files ARE the declaration now: `queries: queries/` discovers every declaration in a directory's top-level *.gq (sorted), a list form takes explicit files, and the map stays for fine-grained control. Discovery is loud — unreadable/unparseable files and duplicate query names fail validation (query_parse_error, duplicate_query_name). Downstream is untouched: each discovered query is still an individually addressed resource with the containing file's digest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 239 +++++++++++++++++++++++++++- 1 file changed, 237 insertions(+), 2 deletions(-) diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 7703bb8..866828e 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -415,7 +415,132 @@ struct StateConfig { struct GraphConfig { schema: PathBuf, #[serde(default)] - queries: BTreeMap<String, QueryConfig>, + queries: QueriesDecl, +} + +/// How a graph declares its stored queries. Terraform-style: the `.gq` +/// files ARE the declaration — point at them (or a directory) and every +/// `query <name>` they contain is discovered. The explicit name->file map +/// remains for fine-grained control. +#[derive(Debug, Serialize, Deserialize)] +#[serde(untagged)] +enum QueriesDecl { + /// `queries: ./queries/` — a directory (top-level `*.gq`, sorted) or a + /// single `.gq` file; every declaration inside is registered. + Discover(PathBuf), + /// `queries: [./queries/, ./extra.gq]` — several directories/files. + DiscoverMany(Vec<PathBuf>), + /// `queries: { name: { file: ... } }` — explicit registry. + Explicit(BTreeMap<String, QueryConfig>), +} + +impl Default for QueriesDecl { + fn default() -> Self { + QueriesDecl::Explicit(BTreeMap::new()) + } +} + +/// Expand a graph's query declaration into the canonical name->file map. +/// Discovery reads and parses each `.gq`; unreadable or unparseable files +/// and duplicate query names are loud validation errors — a declaration the +/// tool cannot enumerate is broken, not partially usable. +fn resolve_query_decls( + config_dir: &Path, + graph_id: &str, + decl: &QueriesDecl, + diagnostics: &mut Vec<Diagnostic>, +) -> BTreeMap<String, QueryConfig> { + let paths: Vec<PathBuf> = match decl { + QueriesDecl::Explicit(map) => { + return map + .iter() + .map(|(name, config)| (name.clone(), QueryConfig { file: config.file.clone() })) + .collect(); + } + QueriesDecl::Discover(path) => vec![path.clone()], + QueriesDecl::DiscoverMany(paths) => paths.clone(), + }; + + let mut files: Vec<(PathBuf, PathBuf)> = Vec::new(); // (declared-relative, resolved) + for declared in &paths { + let resolved = resolve_config_path(config_dir, declared); + if resolved.is_dir() { + let mut entries: Vec<PathBuf> = match fs::read_dir(&resolved) { + Ok(read) => read + .flatten() + .map(|entry| entry.path()) + .filter(|path| path.extension().is_some_and(|ext| ext == "gq")) + .collect(), + Err(err) => { + diagnostics.push(Diagnostic::error( + "query_dir_unreadable", + format!("graphs.{graph_id}.queries"), + format!("could not list query directory '{}': {err}", resolved.display()), + )); + continue; + } + }; + entries.sort(); + if entries.is_empty() { + diagnostics.push(Diagnostic::warning( + "query_dir_empty", + format!("graphs.{graph_id}.queries"), + format!("query directory '{}' contains no .gq files", resolved.display()), + )); + } + for path in entries { + let relative = declared.join(path.file_name().expect("dir entries have names")); + files.push((relative, path)); + } + } else { + files.push((declared.clone(), resolved)); + } + } + + let mut registry: BTreeMap<String, QueryConfig> = BTreeMap::new(); + let mut origin: BTreeMap<String, PathBuf> = BTreeMap::new(); + for (declared, resolved) in files { + let source = match fs::read_to_string(&resolved) { + Ok(source) => source, + Err(err) => { + diagnostics.push(Diagnostic::error( + "query_file_missing", + format!("graphs.{graph_id}.queries"), + format!("could not read query file '{}': {err}", resolved.display()), + )); + continue; + } + }; + let parsed = match parse_query(&source) { + Ok(parsed) => parsed, + Err(err) => { + diagnostics.push(Diagnostic::error( + "query_parse_error", + format!("graphs.{graph_id}.queries"), + format!("'{}' does not parse: {err}", resolved.display()), + )); + continue; + } + }; + for query_decl in &parsed.queries { + let name = query_decl.name.clone(); + if let Some(previous) = origin.get(&name) { + diagnostics.push(Diagnostic::error( + "duplicate_query_name", + format!("graphs.{graph_id}.queries.{name}"), + format!( + "query '{name}' is declared in both '{}' and '{}'", + previous.display(), + declared.display() + ), + )); + continue; + } + origin.insert(name.clone(), declared.clone()); + registry.insert(name, QueryConfig { file: declared.clone() }); + } + } + registry } #[derive(Debug, Serialize, Deserialize)] @@ -3600,7 +3725,8 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { } }); - for (query_name, query) in &graph.queries { + let graph_queries = resolve_query_decls(&config_dir, graph_id, &graph.queries, &mut diagnostics); + for (query_name, query) in &graph_queries { validate_id( "query name", &format!("graphs.{graph_id}.queries.{query_name}"), @@ -7560,6 +7686,115 @@ policies: ); } + // ---- query discovery (Terraform-style declaration) ---- + + #[test] + fn queries_directory_discovers_every_declaration() { + let dir = tempfile::tempdir().unwrap(); + fs::write(dir.path().join("people.pg"), "\nnode Person {\n name: String @key\n}\n").unwrap(); + fs::create_dir(dir.path().join("queries")).unwrap(); + fs::write( + dir.path().join("queries/people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n\nquery all_people() {\n match { $p: Person }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + dir.path().join("queries/extra.gq"), + "\nquery count_people() {\n match { $p: Person }\n return { count($p) }\n}\n", + ) + .unwrap(); + fs::write(dir.path().join("queries/notes.txt"), "ignored").unwrap(); + fs::write( + dir.path().join("cluster.yaml"), + "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: ./queries/\n", + ) + .unwrap(); + + let out = validate_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + let names: Vec<&str> = out + .resource_digests + .keys() + .filter_map(|address| address.strip_prefix("query.knowledge.")) + .collect(); + assert_eq!(names, vec!["all_people", "count_people", "find_person"]); + } + + #[test] + fn queries_list_and_single_file_forms_discover() { + let dir = tempfile::tempdir().unwrap(); + fs::write(dir.path().join("people.pg"), "\nnode Person {\n name: String @key\n}\n").unwrap(); + fs::write( + dir.path().join("a.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + dir.path().join("b.gq"), + "\nquery all_people() {\n match { $p: Person }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + dir.path().join("cluster.yaml"), + "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: [./a.gq, ./b.gq]\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.resource_digests.contains_key("query.knowledge.find_person")); + assert!(out.resource_digests.contains_key("query.knowledge.all_people")); + + // Single-file string form + fs::write( + dir.path().join("cluster.yaml"), + "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: ./a.gq\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.resource_digests.contains_key("query.knowledge.find_person")); + assert!(!out.resource_digests.contains_key("query.knowledge.all_people")); + } + + #[test] + fn query_discovery_rejects_duplicates_and_parse_errors() { + let dir = tempfile::tempdir().unwrap(); + fs::write(dir.path().join("people.pg"), "\nnode Person {\n name: String @key\n}\n").unwrap(); + let decl = "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n"; + fs::write(dir.path().join("a.gq"), decl).unwrap(); + fs::write(dir.path().join("b.gq"), decl).unwrap(); + fs::write( + dir.path().join("cluster.yaml"), + "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: [./a.gq, ./b.gq]\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "duplicate_query_name"), + "{:?}", + out.diagnostics + ); + + fs::write(dir.path().join("broken.gq"), "query {{{ nope").unwrap(); + fs::write( + dir.path().join("cluster.yaml"), + "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: ./broken.gq\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "query_parse_error"), + "{:?}", + out.diagnostics + ); + } + #[test] fn status_warns_on_pending_recovery_sidecar() { let dir = fixture(); From 44b5866516075d79a098b1cc6fe9eb80932021c6 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 00:46:21 +0300 Subject: [PATCH 092/165] docs: drop ./ path prefixes; document query discovery MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Paths in cluster.yaml and command examples are relative to one explicit config folder (Terraform-shaped) — the ./ prefixes were noise and are gone across the user docs (109 instances; ../ links and ./scripts executables untouched). The cluster docs now present directory discovery as the primary queries form with the list and map forms documented alongside. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/user/cli-reference.md | 20 +++++++------- docs/user/cli.md | 54 ++++++++++++++++++------------------- docs/user/cluster-config.md | 48 ++++++++++++++++++++++----------- docs/user/cluster.md | 45 +++++++++++++++---------------- docs/user/deployment.md | 4 +-- docs/user/policy.md | 8 +++--- docs/user/transactions.md | 40 +++++++++++++-------------- 7 files changed, 118 insertions(+), 101 deletions(-) diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index ecb44b5..fb12dd8 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -54,7 +54,7 @@ cli: query: roots: [<dir>, …] # search path for .gq files auth: - env_file: ./.env.omni + env_file: .env.omni aliases: <alias>: # accepted values: `read` / `query` (read alias), `change` / `mutate` @@ -70,20 +70,20 @@ aliases: queries: # top-level registry — applies only to a bare-URI (anonymous) graph; a graph served by name uses its `graphs.<id>.queries`. Mirrors top-level `policy`. <query-name>: { file: <path-to-.gq> } # mcp.expose defaults to true policy: - file: ./policy.yaml + file: policy.yaml ``` ## Cluster config preview ```bash -omnigraph cluster validate --config ./company-brain -omnigraph cluster plan --config ./company-brain --json -omnigraph cluster apply --config ./company-brain --json -omnigraph cluster approve graph.<id> --config ./company-brain --as <actor> -omnigraph cluster status --config ./company-brain --json -omnigraph cluster refresh --config ./company-brain --json -omnigraph cluster import --config ./company-brain --json -omnigraph cluster force-unlock <LOCK_ID> --config ./company-brain --json +omnigraph cluster validate --config company-brain +omnigraph cluster plan --config company-brain --json +omnigraph cluster apply --config company-brain --json +omnigraph cluster approve graph.<id> --config company-brain --as <actor> +omnigraph cluster status --config company-brain --json +omnigraph cluster refresh --config company-brain --json +omnigraph cluster import --config company-brain --json +omnigraph cluster force-unlock <LOCK_ID> --config company-brain --json ``` `--config` is a directory containing `cluster.yaml`; it defaults to `.`. diff --git a/docs/user/cli.md b/docs/user/cli.md index b6f2c09..5c4297a 100644 --- a/docs/user/cli.md +++ b/docs/user/cli.md @@ -3,11 +3,11 @@ ## Core Graph Flow ```bash -omnigraph init --schema ./schema.pg ./graph.omni -omnigraph load --data ./data.jsonl --mode overwrite ./graph.omni -omnigraph snapshot ./graph.omni --branch main --json -omnigraph query --uri ./graph.omni --query ./queries.gq --name get_person --params '{"name":"Alice"}' -omnigraph mutate --uri ./graph.omni --query ./queries.gq --name insert_person --params '{"name":"Mina","age":28}' +omnigraph init --schema schema.pg graph.omni +omnigraph load --data data.jsonl --mode overwrite graph.omni +omnigraph snapshot graph.omni --branch main --json +omnigraph query --uri graph.omni --query queries.gq --name get_person --params '{"name":"Alice"}' +omnigraph mutate --uri graph.omni --query queries.gq --name insert_person --params '{"name":"Mina","age":28}' ``` `omnigraph query` is the canonical read command (pairs with `POST /query`); @@ -21,11 +21,11 @@ For ad-hoc reads and mutations (REPLs, AI agents, one-off scripts), pass the GQ source inline with `-e` / `--query-string` instead of a file path: ```bash -omnigraph query --uri ./graph.omni \ +omnigraph query --uri graph.omni \ -e 'query find($name: String) { match { $p: Person { name: $name } } return { $p.name, $p.age } }' \ --params '{"name":"Alice"}' -omnigraph mutate --uri ./graph.omni \ +omnigraph mutate --uri graph.omni \ -e 'query add($name: String, $age: I32) { insert Person { name: $name, age: $age } }' \ --params '{"name":"Inline","age":42}' ``` @@ -38,14 +38,14 @@ only the source loader changes. ## Branching And Reviewable Data Flows ```bash -omnigraph branch create --uri ./graph.omni --from main feature-x -omnigraph branch list --uri ./graph.omni -omnigraph branch merge --uri ./graph.omni feature-x --into main +omnigraph branch create --uri graph.omni --from main feature-x +omnigraph branch list --uri graph.omni +omnigraph branch merge --uri graph.omni feature-x --into main -omnigraph ingest --data ./batch.jsonl --branch review/import-2026-04-09 ./graph.omni -omnigraph export ./graph.omni --branch main --type Person > people.jsonl -omnigraph commit list ./graph.omni --branch main --json -omnigraph commit show --uri ./graph.omni <commit-id> --json +omnigraph ingest --data batch.jsonl --branch review/import-2026-04-09 graph.omni +omnigraph export graph.omni --branch main --type Person > people.jsonl +omnigraph commit list graph.omni --branch main --json +omnigraph commit show --uri graph.omni <commit-id> --json ``` ## Remote Server Mode @@ -53,7 +53,7 @@ omnigraph commit show --uri ./graph.omni <commit-id> --json Serve a graph: ```bash -omnigraph-server ./graph.omni --bind 127.0.0.1:8080 +omnigraph-server graph.omni --bind 127.0.0.1:8080 ``` Read through the HTTP API: @@ -61,7 +61,7 @@ Read through the HTTP API: ```bash omnigraph query \ --target http://127.0.0.1:8080 \ - --query ./queries.gq \ + --query queries.gq \ --name get_person \ --params '{"name":"Alice"}' ``` @@ -87,23 +87,23 @@ Runtime add/remove is **not** in v0.6.0. To add a graph, stop the server, add a Per-graph URLs: hit a graph's cluster route from any subcommand by pointing `--uri` at it: ```bash -omnigraph read --uri http://server.example.com/graphs/beta --query ./q.gq ... +omnigraph read --uri http://server.example.com/graphs/beta --query q.gq ... ``` ## Runs, Policy, And Diagnostics ```bash -omnigraph lint --query ./queries.gq --schema ./schema.pg --json -omnigraph check --query ./queries.gq ./graph.omni --json +omnigraph lint --query queries.gq --schema schema.pg --json +omnigraph check --query queries.gq graph.omni --json -omnigraph schema plan --schema ./next.pg ./graph.omni --json -omnigraph schema apply --schema ./next.pg ./graph.omni --json -omnigraph policy validate --config ./omnigraph.yaml -omnigraph policy test --config ./omnigraph.yaml -omnigraph policy explain --config ./omnigraph.yaml --actor act-alice --action read --branch main +omnigraph schema plan --schema next.pg graph.omni --json +omnigraph schema apply --schema next.pg graph.omni --json +omnigraph policy validate --config omnigraph.yaml +omnigraph policy test --config omnigraph.yaml +omnigraph policy explain --config omnigraph.yaml --actor act-alice --action read --branch main -omnigraph commit list ./graph.omni --json -omnigraph commit show --uri ./graph.omni <commit-id> --json +omnigraph commit list graph.omni --json +omnigraph commit show --uri graph.omni <commit-id> --json ``` (The legacy `omnigraph run list/show/publish/abort` subcommands were removed in MR-771; mutations and loads publish atomically and the commit graph (`omnigraph commit list`) is the audit surface.) @@ -120,7 +120,7 @@ query roots: ```yaml graphs: local: - uri: ./demo.omni + uri: demo.omni dev: uri: http://127.0.0.1:8080 bearer_token_env: OMNIGRAPH_BEARER_TOKEN diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 081bfa2..24d1833 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -20,14 +20,14 @@ or serve anything it applies: the server still boots from `omnigraph.yaml`. ## Commands ```bash -omnigraph cluster validate --config ./company-brain -omnigraph cluster plan --config ./company-brain --json -omnigraph cluster apply --config ./company-brain --json -omnigraph cluster approve graph.<id> --config ./company-brain --as <actor> -omnigraph cluster status --config ./company-brain --json -omnigraph cluster refresh --config ./company-brain --json -omnigraph cluster import --config ./company-brain --json -omnigraph cluster force-unlock <LOCK_ID> --config ./company-brain --json +omnigraph cluster validate --config company-brain +omnigraph cluster plan --config company-brain --json +omnigraph cluster apply --config company-brain --json +omnigraph cluster approve graph.<id> --config company-brain --as <actor> +omnigraph cluster status --config company-brain --json +omnigraph cluster refresh --config company-brain --json +omnigraph cluster import --config company-brain --json +omnigraph cluster force-unlock <LOCK_ID> --config company-brain --json ``` `--config` points at a directory, not a file. The directory must contain @@ -54,7 +54,7 @@ The exact contract: cluster state XOR `omnigraph.yaml`, never a merge. - **The other direction is ergonomics, not coupling**: a per-operator `omnigraph.yaml` may point `graphs.<name>.uri` at a cluster's derived root - (`./company-brain/graphs/knowledge.omni`) so data-plane commands can use + (`company-brain/graphs/knowledge.omni`) so data-plane commands can use `--target <name>` — an ordinary local path, no special handling. ## Supported `cluster.yaml` @@ -72,17 +72,35 @@ state: graphs: knowledge: - schema: ./knowledge.pg - queries: - find_experts: - file: ./knowledge.gq + schema: knowledge.pg + queries: queries/ # discover every `query <name>` in queries/*.gq policies: base: - file: ./base.policy.yaml + file: base.policy.yaml applies_to: [knowledge] ``` +`queries` is Terraform-shaped — the `.gq` files are the declaration. Three +forms: + +```yaml +queries: queries/ # directory: top-level *.gq, sorted; every declaration registers +queries: [people.gq, extra/a.gq] # explicit files; every declaration in each +queries: # fine-grained name -> file map + find_experts: + file: knowledge.gq +``` + +Discovery is loud: an unreadable or unparseable `.gq`, or the same query name +declared in two files, fails validation (`query_parse_error`, +`duplicate_query_name`). Each discovered query is still an individually +addressed resource (`query.<graph>.<name>`) with its own plan/apply lifecycle; +the digest is the containing file's hash, so editing a multi-query file +updates all of its queries together. Paths are relative to the config +directory — the cluster is one explicit folder, so no `./` prefixes are +needed. + `metadata.name` is a display label. `state.backend` may be omitted or set to `cluster`; external state backends are reserved for a later stage. `state.lock` defaults to `true`. When enabled, `cluster plan`, `cluster apply`, @@ -324,7 +342,7 @@ without graph movement. ## Serving from the cluster (the mode switch) ```bash -omnigraph-server --cluster ./company-brain --bind 0.0.0.0:8080 +omnigraph-server --cluster company-brain --bind 0.0.0.0:8080 ``` `--cluster <dir>` is an **exclusive boot source** (axiom 15): it cannot diff --git a/docs/user/cluster.md b/docs/user/cluster.md index ff930da..1731f31 100644 --- a/docs/user/cluster.md +++ b/docs/user/cluster.md @@ -32,7 +32,8 @@ Lay out a config directory: company-brain/ ├── cluster.yaml ├── people.pg # schema for the "knowledge" graph -├── people.gq # a stored query +├── queries/ # stored queries — the .gq files ARE the declaration +│ └── people.gq └── base.policy.yaml # a Cedar policy bundle ``` @@ -43,27 +44,25 @@ metadata: name: company-brain graphs: knowledge: - schema: ./people.pg - queries: - find_person: - file: ./people.gq + schema: people.pg + queries: queries/ # every `query <name>` in queries/*.gq registers policies: base: - file: ./base.policy.yaml + file: base.policy.yaml applies_to: [knowledge] # graph-bound; use [cluster] for server-level ``` Bring it to life: ```bash -omnigraph cluster validate --config ./company-brain # parse + typecheck everything -omnigraph cluster import --config ./company-brain # create the state ledger -omnigraph cluster plan --config ./company-brain # preview: what would apply do? -omnigraph cluster apply --config ./company-brain # converge +omnigraph cluster validate --config company-brain # parse + typecheck everything +omnigraph cluster import --config company-brain # create the state ledger +omnigraph cluster plan --config company-brain # preview: what would apply do? +omnigraph cluster apply --config company-brain # converge ``` That single `apply` **creates the graph** (at the derived root -`./company-brain/graphs/knowledge.omni`), applies its schema, and publishes +`company-brain/graphs/knowledge.omni`), applies its schema, and publishes the query and policy into the content-addressed catalog (`__cluster/resources/…`). The output lists every change with its disposition; `converged: true` means there is nothing left to do — re-running @@ -73,14 +72,14 @@ Load data through the normal graph plane (the control plane manages *definitions*, not rows): ```bash -omnigraph load --data ./seed.jsonl ./company-brain/graphs/knowledge.omni +omnigraph load --data seed.jsonl company-brain/graphs/knowledge.omni ``` Serve it: ```bash OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-reader":"s3cret"}' \ - omnigraph-server --cluster ./company-brain --bind 0.0.0.0:8080 + omnigraph-server --cluster company-brain --bind 0.0.0.0:8080 ``` `--cluster` is an **exclusive boot source**: it cannot be combined with a @@ -103,8 +102,8 @@ Every change follows the same loop, whatever its kind: ```bash $EDITOR company-brain/people.pg # or any .gq / policy / cluster.yaml edit -omnigraph cluster plan --config ./company-brain -omnigraph cluster apply --config ./company-brain --as andrew +omnigraph cluster plan --config company-brain +omnigraph cluster apply --config company-brain --as andrew # restart cluster-booted servers to pick it up ``` @@ -142,8 +141,8 @@ anything runs. ## 3. Inspect: status, refresh, drift ```bash -omnigraph cluster status --config ./company-brain --json # ledger only, read-only -omnigraph cluster refresh --config ./company-brain # re-observe live graphs +omnigraph cluster status --config company-brain --json # ledger only, read-only +omnigraph cluster refresh --config company-brain # re-observe live graphs ``` `status` never touches the graphs; `refresh` opens them read-only and @@ -164,13 +163,13 @@ converges the ledger. Removing a graph from `cluster.yaml` never executes silently: ```bash -omnigraph cluster apply --config ./company-brain +omnigraph cluster apply --config company-brain # Delete graph.scratch [Blocked: approval_required] -omnigraph cluster approve graph.scratch --config ./company-brain --as andrew +omnigraph cluster approve graph.scratch --config company-brain --as andrew # cluster approve: delete graph.scratch approved by andrew (approval 01KT…) -omnigraph cluster apply --config ./company-brain --as andrew +omnigraph cluster apply --config company-brain --as andrew # Delete graph.scratch [Applied] ← root removed, subtree tombstoned ``` @@ -196,8 +195,8 @@ again**. **A held lock** (a crashed process left `__cluster/lock.json`): ```bash -omnigraph cluster status --config ./company-brain # shows the lock holder + id -omnigraph cluster force-unlock <LOCK_ID> --config ./company-brain +omnigraph cluster status --config company-brain # shows the lock holder + id +omnigraph cluster force-unlock <LOCK_ID> --config company-brain ``` Force-unlock requires the exact lock id (from status) — there is no blind @@ -240,7 +239,7 @@ with an in-flight apply. - **`omnigraph.yaml` still has a job**: per-operator settings — your `cli.actor` default for `--as`, CLI defaults, credentials, and data-plane ergonomics (point `graphs.<name>.uri` at a derived root like - `./company-brain/graphs/knowledge.omni` to use `--target <name>` for + `company-brain/graphs/knowledge.omni` to use `--target <name>` for loads). It just no longer describes the deployment — a server boots from one source or the other, never a merge of both. diff --git a/docs/user/deployment.md b/docs/user/deployment.md index 563a501..00f8272 100644 --- a/docs/user/deployment.md +++ b/docs/user/deployment.md @@ -33,7 +33,7 @@ On Windows, the binaries are `omnigraph.exe` and `omnigraph-server.exe`. Run against a local graph: ```bash -omnigraph-server ./graph.omni --bind 0.0.0.0:8080 +omnigraph-server graph.omni --bind 0.0.0.0:8080 ``` Run against an object-store-backed graph: @@ -208,7 +208,7 @@ docker run --rm -p 8080:8080 \ -e OMNIGRAPH_CONFIG="/etc/omnigraph/omnigraph.yaml" \ -v "$PWD/config:/etc/omnigraph:ro" \ omnigraph-server:local -# /etc/omnigraph/omnigraph.yaml contains `policy: { file: ./policy.yaml }`; +# /etc/omnigraph/omnigraph.yaml contains `policy: { file: policy.yaml }`; # policy.yaml (+ optional policy.tests.yaml) sit beside it in the mount. ``` diff --git a/docs/user/policy.md b/docs/user/policy.md index ec0d214..9c484ba 100644 --- a/docs/user/policy.md +++ b/docs/user/policy.md @@ -35,13 +35,13 @@ In multi mode (`omnigraph.yaml` with a non-empty `graphs:` map), policy files at ```yaml server: policy: - file: ./server-policy.yaml # server-level: graph_list + file: server-policy.yaml # server-level: graph_list graphs: alpha: uri: s3://tenant-bucket/alpha policy: - file: ./policies/alpha.yaml # per-graph: read, change, branch_*, schema_apply + file: policies/alpha.yaml # per-graph: read, change, branch_*, schema_apply beta: uri: s3://tenant-bucket/beta # no per-graph policy → no engine-layer Cedar enforcement on beta @@ -78,8 +78,8 @@ rules: ```yaml policy: - file: ./policy.yaml # Cedar rules + groups - tests: ./policy.tests.yaml # declarative test cases + file: policy.yaml # Cedar rules + groups + tests: policy.tests.yaml # declarative test cases cli: actor: act-andrew # default actor for CLI direct-engine writes diff --git a/docs/user/transactions.md b/docs/user/transactions.md index d6c79f4..39a86c4 100644 --- a/docs/user/transactions.md +++ b/docs/user/transactions.md @@ -47,8 +47,8 @@ query register_employee_with_team($name: String, $age: I32, $team: String) { ``` ```bash -omnigraph change --query ./mutations.gq --name register_employee_with_team \ - --params '{"name":"Alice","age":30,"team":"Acme"}' ./graph.omni +omnigraph change --query mutations.gq --name register_employee_with_team \ + --params '{"name":"Alice","age":30,"team":"Acme"}' graph.omni ``` If the second statement fails (e.g. `Acme` doesn't exist), the publisher never publishes; `Alice` is not in the database. Atomic. @@ -57,10 +57,10 @@ If the second statement fails (e.g. `Acme` doesn't exist), the publisher never p ```bash # Query 1 -omnigraph change --query ./mutations.gq --name register_employee --params '{"name":"Alice","age":30}' ./graph.omni +omnigraph change --query mutations.gq --name register_employee --params '{"name":"Alice","age":30}' graph.omni # Query 2 — runs after Query 1 has already published -omnigraph change --query ./mutations.gq --name link_to_team --params '{"name":"Alice","team":"Acme"}' ./graph.omni +omnigraph change --query mutations.gq --name link_to_team --params '{"name":"Alice","team":"Acme"}' graph.omni ``` These are **two publishes** on `main`. If Query 2 fails, Query 1's effects are already visible. There is no `ROLLBACK` for Query 1. @@ -75,32 +75,32 @@ The pattern when you need to run multiple queries — possibly across multiple c ```bash # Fork a working branch from main. -omnigraph branch create --from main onboarding/2026-04-25 ./graph.omni +omnigraph branch create --from main onboarding/2026-04-25 graph.omni # Run any number of mutations on the branch — each one is its own publish on the branch. # Concurrent reads of `main` are unaffected. omnigraph change --branch onboarding/2026-04-25 \ - --query ./mutations.gq --name register_employee \ - --params '{"name":"Alice","age":30}' ./graph.omni + --query mutations.gq --name register_employee \ + --params '{"name":"Alice","age":30}' graph.omni omnigraph change --branch onboarding/2026-04-25 \ - --query ./mutations.gq --name register_employee \ - --params '{"name":"Bob","age":25}' ./graph.omni + --query mutations.gq --name register_employee \ + --params '{"name":"Bob","age":25}' graph.omni omnigraph change --branch onboarding/2026-04-25 \ - --query ./mutations.gq --name link_to_team \ - --params '{"name":"Alice","team":"Acme"}' ./graph.omni + --query mutations.gq --name link_to_team \ + --params '{"name":"Alice","team":"Acme"}' graph.omni # Inspect the branch — read queries work just like on main. omnigraph read --branch onboarding/2026-04-25 \ - --query ./queries.gq --name list_employees ./graph.omni + --query queries.gq --name list_employees graph.omni # Happy with what's on the branch? Merge it. This is one atomic publish: # `main` flips to include every commit on the branch. -omnigraph branch merge onboarding/2026-04-25 --into main ./graph.omni +omnigraph branch merge onboarding/2026-04-25 --into main graph.omni # OR: not happy? Throw it away. `main` is untouched. -# omnigraph branch delete onboarding/2026-04-25 ./graph.omni +# omnigraph branch delete onboarding/2026-04-25 graph.omni ``` Properties: @@ -115,16 +115,16 @@ Two agents writing to the same graph independently: ```bash # Agent A -omnigraph branch create --from main agent-a/work ./graph.omni -omnigraph change --branch agent-a/work … ./graph.omni +omnigraph branch create --from main agent-a/work graph.omni +omnigraph change --branch agent-a/work … graph.omni # … many mutations … -omnigraph branch merge agent-a/work --into main ./graph.omni +omnigraph branch merge agent-a/work --into main graph.omni # Agent B (running concurrently) -omnigraph branch create --from main agent-b/work ./graph.omni -omnigraph change --branch agent-b/work … ./graph.omni +omnigraph branch create --from main agent-b/work graph.omni +omnigraph change --branch agent-b/work … graph.omni # … many mutations … -omnigraph branch merge agent-b/work --into main ./graph.omni +omnigraph branch merge agent-b/work --into main graph.omni ``` Each agent sees a consistent snapshot of `main` at the time it forked. The first merge to `main` lands as a fast-forward (or a no-op if no concurrent change). The second merge runs three-way: rows touched by both branches surface as `MergeConflict`s for the caller to resolve. From 4558454bc779e33d27d0fd8be7ab2bc202237a15 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 01:35:47 +0300 Subject: [PATCH 093/165] =?UTF-8?q?fix(cluster):=20address=20review=20?= =?UTF-8?q?=E2=80=94=20discovery=20reads=20each=20file=20exactly=20once?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit resolve_query_decls hands its file contents to the caller; the per-query digest/typecheck pass reuses them instead of re-reading (a file with N queries was read N+1 times), which also closes the window where a file changing between enumeration and validation produced a confusing query_key_mismatch for a just-discovered name. Explicit-map declarations read as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 30 +++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 866828e..bb0c66b 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -449,13 +449,17 @@ fn resolve_query_decls( graph_id: &str, decl: &QueriesDecl, diagnostics: &mut Vec<Diagnostic>, -) -> BTreeMap<String, QueryConfig> { +) -> (BTreeMap<String, QueryConfig>, BTreeMap<PathBuf, String>) { let paths: Vec<PathBuf> = match decl { QueriesDecl::Explicit(map) => { - return map - .iter() - .map(|(name, config)| (name.clone(), QueryConfig { file: config.file.clone() })) - .collect(); + return ( + map.iter() + .map(|(name, config)| { + (name.clone(), QueryConfig { file: config.file.clone() }) + }) + .collect(), + BTreeMap::new(), + ); } QueriesDecl::Discover(path) => vec![path.clone()], QueriesDecl::DiscoverMany(paths) => paths.clone(), @@ -499,6 +503,10 @@ fn resolve_query_decls( let mut registry: BTreeMap<String, QueryConfig> = BTreeMap::new(); let mut origin: BTreeMap<String, PathBuf> = BTreeMap::new(); + // Content read once at discovery and handed to the caller — the per-query + // digest/typecheck pass reuses it instead of re-reading (no N+1 reads, no + // window for the file to change between enumeration and validation). + let mut contents: BTreeMap<PathBuf, String> = BTreeMap::new(); for (declared, resolved) in files { let source = match fs::read_to_string(&resolved) { Ok(source) => source, @@ -539,8 +547,9 @@ fn resolve_query_decls( origin.insert(name.clone(), declared.clone()); registry.insert(name, QueryConfig { file: declared.clone() }); } + contents.insert(declared, source); } - registry + (registry, contents) } #[derive(Debug, Serialize, Deserialize)] @@ -3725,7 +3734,8 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { } }); - let graph_queries = resolve_query_decls(&config_dir, graph_id, &graph.queries, &mut diagnostics); + let (graph_queries, query_contents) = + resolve_query_decls(&config_dir, graph_id, &graph.queries, &mut diagnostics); for (query_name, query) in &graph_queries { validate_id( "query name", @@ -3744,7 +3754,11 @@ fn load_desired(config_dir: &Path) -> LoadOutcome { }); let query_path = resolve_config_path(&config_dir, &query.file); - match fs::read_to_string(&query_path) { + let source = match query_contents.get(&query.file) { + Some(cached) => Ok(cached.clone()), + None => fs::read_to_string(&query_path), + }; + match source { Ok(source) => { let digest = sha256_hex(source.as_bytes()); graph_query_digests From 43d4e89fdea41ba68d76a0dd2c368c39deafd6b0 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 03:44:02 +0300 Subject: [PATCH 094/165] docs(execution): Overwrite loads are staged since MR-793, not inline-commit MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The LoadMode table still described Overwrite as an inline-commit-per-type residual with a partial-truncation failure window. Since MR-793 Phase 2, Overwrite goes through the same MutationStaging accumulator as Append/Merge, staged as a Lance Operation::Overwrite transaction via stage_overwrite (table_store.rs) and committed with commit_staged + publisher CAS — a mid-load failure leaves Lance HEAD untouched in all three modes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/execution.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/dev/execution.md b/docs/dev/execution.md index a3a9c01..9753696 100644 --- a/docs/dev/execution.md +++ b/docs/dev/execution.md @@ -162,11 +162,11 @@ Atomicity guarantee for multi-statement mutations: a mid-query failure leaves La | Mode | Semantics | Path (post-MR-794) | |---|---|---| -| `Overwrite` | Replace all data in the target tables on the branch | Inline-commit per type, then publisher CAS at end-of-load. Truncate-then-append doesn't fit the staged shape; documented residual. | +| `Overwrite` | Replace all data in the target tables on the branch | Same accumulator; one `stage_overwrite` + `commit_staged` per touched table at end-of-load (a staged Lance `Operation::Overwrite` transaction — HEAD does not advance until commit; MR-793 Phase 2); publisher CAS. | | `Append` | Strict insert; duplicates error | In-memory `MutationStaging` accumulator; one `stage_append` + `commit_staged` per touched table at end-of-load; publisher CAS. | | `Merge` | Upsert by `id` (`merge_insert`) | Same accumulator; one `stage_merge_insert` per touched table at end-of-load (Merge mode dedupes by `id`, last-write-wins); publisher CAS. | -For Append/Merge, a mid-load failure (RI / cardinality violation, validation error) leaves Lance HEAD untouched on the staged tables — the next load on the same tables proceeds normally with no `ExpectedVersionMismatch`. For Overwrite, a mid-load failure can still leave Lance HEAD on a partially-truncated table; the next overwrite replaces it. +For all three modes, a mid-load failure (RI / cardinality violation, validation error) leaves Lance HEAD untouched on the staged tables — the next load on the same tables proceeds normally with no `ExpectedVersionMismatch`. ## `load` vs `ingest` From e676c151bbb3674c4dd41b23ddcc7fb1e99cb433 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 03:53:22 +0300 Subject: [PATCH 095/165] =?UTF-8?q?feat(engine):=20unify=20load/ingest=20?= =?UTF-8?q?=E2=80=94=20load=5Fas=20gains=20an=20optional=20fork=20base?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit load_as/load_file_as gain a base: Option<&str> parameter: with Some(base) a missing target branch is forked from base first (the former ingest semantics); with None the target branch must exist — staging fails on an unknown branch, so a typo'd name can never create one. LoadResult gains branch/base_branch/branch_created metadata (additive). The ingest family (ingest, ingest_as, ingest_file, ingest_file_as) becomes #[deprecated] shims over load_as that preserve the historical contract exactly (from: None still means fork from main; base recorded even when no fork happened). IngestResult and to_ingest_tables stay for the shims and the server until the removal release. The layered policy check is unchanged: Change on the target branch always, BranchCreate additionally when a fork actually happens (enforced inside branch_create_from_as with the actor threaded through). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 5 +- crates/omnigraph-server/src/lib.rs | 3 + crates/omnigraph-server/tests/server.rs | 1 + crates/omnigraph/src/loader/mod.rs | 223 ++++++++++++------ .../omnigraph/tests/policy_engine_chassis.rs | 6 + 5 files changed, 170 insertions(+), 68 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 62fff60..d1fbb99 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -2669,7 +2669,7 @@ async fn main() -> Result<()> { let db = open_local_db_with_policy(&graph).await?; let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); let result = db - .load_file_as(&branch, &data.to_string_lossy(), mode.into(), actor) + .load_file_as(&branch, None, &data.to_string_lossy(), mode.into(), actor) .await?; let payload = LoadOutput { uri: &uri, @@ -2729,6 +2729,9 @@ async fn main() -> Result<()> { } else { let db = open_local_db_with_policy(&graph).await?; let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); + // Deprecated shim retained until the CLI ingest command + // becomes an alias of the unified `load` handler. + #[allow(deprecated)] let result = db .ingest_file_as( &branch, diff --git a/crates/omnigraph-server/src/lib.rs b/crates/omnigraph-server/src/lib.rs index 3b9ff1d..26e4837 100644 --- a/crates/omnigraph-server/src/lib.rs +++ b/crates/omnigraph-server/src/lib.rs @@ -2722,6 +2722,9 @@ async fn server_ingest( .try_admit(&actor_arc, est_bytes) .map_err(ApiError::from_workload_reject)?; + // Deprecated shim retained until the from-absent semantics change + // lands; the handler then calls `load_as` directly. + #[allow(deprecated)] let result = { let db = &handle.engine; db.ingest_as(&branch, Some(&from), &request.data, mode, actor_id) diff --git a/crates/omnigraph-server/tests/server.rs b/crates/omnigraph-server/tests/server.rs index bf99b8d..fd214bb 100644 --- a/crates/omnigraph-server/tests/server.rs +++ b/crates/omnigraph-server/tests/server.rs @@ -4731,6 +4731,7 @@ async fn build_parity_graph() -> (tempfile::TempDir, PathBuf, PathBuf) { .unwrap(); db.load_as( "feature", + None, r#"{"type":"Person","data":{"name":"ParityEve","age":29}}"#, LoadMode::Append, None, diff --git a/crates/omnigraph/src/loader/mod.rs b/crates/omnigraph/src/loader/mod.rs index febbabd..0fabaea 100644 --- a/crates/omnigraph/src/loader/mod.rs +++ b/crates/omnigraph/src/loader/mod.rs @@ -26,6 +26,14 @@ use crate::exec::staging::{MutationStaging, PendingMode}; /// Result of a load operation. #[derive(Debug, Clone, Default)] pub struct LoadResult { + /// Branch the load landed on (`"main"` when no branch was given). + pub branch: String, + /// Base branch a fork was requested from (the `base` parameter of + /// `load_as`), recorded verbatim even when the target branch already + /// existed and no fork happened. + pub base_branch: Option<String>, + /// True when this load created `branch` by forking it from `base_branch`. + pub branch_created: bool, pub nodes_loaded: HashMap<String, usize>, pub edges_loaded: HashMap<String, usize>, } @@ -72,6 +80,9 @@ pub async fn load_jsonl_file(db: &mut Omnigraph, path: &str, mode: LoadMode) -> } impl Omnigraph { + #[deprecated( + note = "use `load_as` with an explicit `base` instead; the ingest family will be removed in a future release" + )] pub async fn ingest( &self, branch: &str, @@ -79,9 +90,17 @@ impl Omnigraph { data: &str, mode: LoadMode, ) -> Result<IngestResult> { + #[allow(deprecated)] self.ingest_as(branch, from, data, mode, None).await } + /// Deprecated shim over the unified `load_as`. Preserves the historical + /// ingest contract exactly: `from: None` means fork from `main`, and the + /// base branch is recorded in the result even when the target branch + /// already existed (no fork happened). + #[deprecated( + note = "use `load_as` with an explicit `base` instead; the ingest family will be removed in a future release" + )] pub async fn ingest_as( &self, branch: &str, @@ -90,22 +109,24 @@ impl Omnigraph { mode: LoadMode, actor_id: Option<&str>, ) -> Result<IngestResult> { - // Engine-layer policy gate (MR-722 fan-out / PR #3). Scope is - // `Branch(branch)` for the data-write portion. If ingest creates - // a new branch as a side-effect (target branch doesn't exist), - // the inner `branch_create_from_as` call below additionally - // checks `BranchCreate` — both authorities are genuinely needed - // for "ingest into a fresh branch", so the layered check is - // correct, not redundant. - self.enforce( - omnigraph_policy::PolicyAction::Change, - &omnigraph_policy::ResourceScope::Branch(branch.to_string()), - actor_id, - )?; - self.ingest_with_current_actor(branch, from, data, mode, actor_id) - .await + let result = self + .load_as(branch, Some(from.unwrap_or("main")), data, mode, actor_id) + .await?; + Ok(IngestResult { + branch: result.branch.clone(), + base_branch: result + .base_branch + .clone() + .unwrap_or_else(|| "main".to_string()), + branch_created: result.branch_created, + mode, + tables: result.to_ingest_tables(), + }) } + #[deprecated( + note = "use `load_file_as` with an explicit `base` instead; the ingest family will be removed in a future release" + )] pub async fn ingest_file( &self, branch: &str, @@ -113,9 +134,13 @@ impl Omnigraph { path: &str, mode: LoadMode, ) -> Result<IngestResult> { + #[allow(deprecated)] self.ingest_file_as(branch, from, path, mode, None).await } + #[deprecated( + note = "use `load_file_as` with an explicit `base` instead; the ingest family will be removed in a future release" + )] pub async fn ingest_file_as( &self, branch: &str, @@ -125,69 +150,35 @@ impl Omnigraph { actor_id: Option<&str>, ) -> Result<IngestResult> { let data = std::fs::read_to_string(path).map_err(OmniError::Io)?; + #[allow(deprecated)] self.ingest_as(branch, from, &data, mode, actor_id).await } - async fn ingest_with_current_actor( - &self, - branch: &str, - from: Option<&str>, - data: &str, - mode: LoadMode, - actor_id: Option<&str>, - ) -> Result<IngestResult> { - self.ensure_schema_state_valid().await?; - let target_branch = - Self::normalize_branch_name(branch)?.unwrap_or_else(|| "main".to_string()); - let base_branch = Self::normalize_branch_name(from.unwrap_or("main"))? - .unwrap_or_else(|| "main".to_string()); - let branch_created = !self - .branch_list() - .await? - .iter() - .any(|name| name == &target_branch); - if branch_created { - // Thread the actor through to the implicit BranchCreate so - // policy decisions match what an explicit `branch_create_from_as` - // call would see. Calling the no-actor variant here would - // bypass BranchCreate enforcement when policy is installed — - // the footgun guard catches that case too, but threading is - // the correct fix. - self.branch_create_from_as( - crate::db::ReadTarget::branch(&base_branch), - &target_branch, - actor_id, - ) - .await?; - } - - let result = self.load_as(&target_branch, data, mode, actor_id).await?; - Ok(IngestResult { - branch: target_branch, - base_branch, - branch_created, - mode, - tables: result.to_ingest_tables(), - }) - } - pub async fn load(&self, branch: &str, data: &str, mode: LoadMode) -> Result<LoadResult> { - self.load_as(branch, data, mode, None).await + self.load_as(branch, None, data, mode, None).await } + /// Load JSONL data onto `branch`. + /// + /// `base` selects the branch-creation behavior: with `Some(base)`, a + /// missing target branch is forked from `base` first (the former + /// `ingest` semantics); with `None`, the target branch must already + /// exist — staging fails on an unknown branch when it resolves the + /// manifest snapshot, so a typo'd branch name can never create one. pub async fn load_as( &self, branch: &str, + base: Option<&str>, data: &str, mode: LoadMode, actor_id: Option<&str>, ) -> Result<LoadResult> { // Engine-layer policy gate (MR-722 fan-out / PR #3). Scope is // `Branch(branch)` to match the HTTP-layer Change convention. - // `ingest_as` also calls `load_as` after enforcing its own - // Change gate — that double-check is fine because both gates - // resolve to identical Cedar decisions for the same actor + - // branch (the second check is a structurally-correct no-op). + // When a fork happens below, `branch_create_from_as` additionally + // checks `BranchCreate` — both authorities are genuinely needed + // for "load into a fresh branch", so the layered check is + // correct, not redundant. self.enforce( omnigraph_policy::PolicyAction::Change, &omnigraph_policy::ResourceScope::Branch(branch.to_string()), @@ -205,15 +196,47 @@ impl Omnigraph { // `commit_prepared_updates_on_branch_with_expected`) and leave // `self.coordinator` with a stale manifest snapshot. let requested = Self::normalize_branch_name(branch)?; + let base_branch = match base { + Some(base) => { + Some(Self::normalize_branch_name(base)?.unwrap_or_else(|| "main".to_string())) + } + None => None, + }; + // Fork-if-missing only when a base branch was explicitly given. + // `requested == None` is `main`, which always exists. + let mut branch_created = false; + if let (Some(target), Some(base_name)) = (requested.as_deref(), base_branch.as_deref()) { + let exists = self.branch_list().await?.iter().any(|name| name == target); + if !exists { + // Thread the actor through to the implicit BranchCreate so + // policy decisions match what an explicit `branch_create_from_as` + // call would see. Calling the no-actor variant here would + // bypass BranchCreate enforcement when policy is installed — + // the footgun guard catches that case too, but threading is + // the correct fix. + self.branch_create_from_as( + crate::db::ReadTarget::branch(base_name), + target, + actor_id, + ) + .await?; + branch_created = true; + } + } // Direct-to-target writes: no Run state machine, no `__run__` staging // branch. Cross-table OCC is enforced by the publisher's // `expected_table_versions` CAS inside `load_jsonl_reader`. - self.load_direct_on_branch(requested.as_deref(), data, mode, actor_id) - .await + let mut result = self + .load_direct_on_branch(requested.as_deref(), data, mode, actor_id) + .await?; + result.branch = requested.unwrap_or_else(|| "main".to_string()); + result.base_branch = base_branch; + result.branch_created = branch_created; + Ok(result) } pub async fn load_file(&self, branch: &str, path: &str, mode: LoadMode) -> Result<LoadResult> { - self.load_file_as(branch, path, mode, None).await + self.load_file_as(branch, None, path, mode, None).await } /// Read a file into memory and delegate to `load_as`. Used by the @@ -222,12 +245,13 @@ impl Omnigraph { pub async fn load_file_as( &self, branch: &str, + base: Option<&str>, path: &str, mode: LoadMode, actor_id: Option<&str>, ) -> Result<LoadResult> { - let data = std::fs::read_to_string(path).map_err(|e| OmniError::Io(e))?; - self.load_as(branch, &data, mode, actor_id).await + let data = std::fs::read_to_string(path).map_err(OmniError::Io)?; + self.load_as(branch, base, &data, mode, actor_id).await } async fn load_direct_on_branch( @@ -1824,6 +1848,7 @@ edge WorksAt: Person -> Company } #[tokio::test] + #[allow(deprecated)] async fn test_ingest_creates_branch_and_reports_tables() { let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); @@ -1868,6 +1893,7 @@ edge WorksAt: Person -> Company } #[tokio::test] + #[allow(deprecated)] async fn test_ingest_existing_branch_ignores_from_and_merges_data() { let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); @@ -1942,6 +1968,7 @@ edge WorksAt: Person -> Company } #[tokio::test] + #[allow(deprecated)] async fn test_ingest_as_stamps_actor_on_branch_head_commit() { let dir = tempfile::tempdir().unwrap(); let uri = dir.path().to_str().unwrap(); @@ -1967,6 +1994,68 @@ edge WorksAt: Person -> Company assert_eq!(head.actor_id.as_deref(), Some("act-andrew")); } + #[tokio::test] + async fn test_load_as_with_base_forks_missing_branch_and_stamps_metadata() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + + let result = db + .load_as("feature", Some("main"), TEST_DATA, LoadMode::Merge, None) + .await + .unwrap(); + + assert_eq!(result.branch, "feature"); + assert_eq!(result.base_branch.as_deref(), Some("main")); + assert!(result.branch_created); + assert!( + db.branch_list() + .await + .unwrap() + .contains(&"feature".to_string()) + ); + + // Re-loading onto the now-existing branch records the base but + // performs no fork. + let again = db + .load_as( + "feature", + Some("main"), + r#"{"type":"Person","data":{"name":"Bob","age":26}}"#, + LoadMode::Merge, + None, + ) + .await + .unwrap(); + assert!(!again.branch_created); + assert_eq!(again.base_branch.as_deref(), Some("main")); + } + + #[tokio::test] + async fn test_load_as_without_base_errors_on_missing_branch() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + + let result = db + .load_as("nonexistent", None, TEST_DATA, LoadMode::Merge, None) + .await; + assert!(result.is_err(), "load without base must not create branches"); + assert!( + !db.branch_list() + .await + .unwrap() + .contains(&"nonexistent".to_string()), + "failed load must not leave a branch behind" + ); + + // Loads to main carry the default branch metadata. + let main_load = db.load("main", TEST_DATA, LoadMode::Overwrite).await.unwrap(); + assert_eq!(main_load.branch, "main"); + assert_eq!(main_load.base_branch, None); + assert!(!main_load.branch_created); + } + #[test] fn test_range_constraint_rejects_nan() { use arrow_array::{Float64Array, RecordBatch, StringArray}; diff --git a/crates/omnigraph/tests/policy_engine_chassis.rs b/crates/omnigraph/tests/policy_engine_chassis.rs index def5349..8443940 100644 --- a/crates/omnigraph/tests/policy_engine_chassis.rs +++ b/crates/omnigraph/tests/policy_engine_chassis.rs @@ -243,6 +243,7 @@ async fn load_as_denies_when_policy_rejects_actor() { let result = db .load_as( "main", + None, ONE_PERSON_JSONL, LoadMode::Merge, Some("act-denied"), @@ -258,6 +259,7 @@ async fn load_as_allows_when_policy_permits_actor() { db.load_as( "main", + None, ONE_PERSON_JSONL, LoadMode::Merge, Some("act-allowed"), @@ -281,6 +283,7 @@ async fn load_file_as_denies_when_policy_rejects_actor() { let result = db .load_file_as( "main", + None, data_path.to_str().unwrap(), LoadMode::Merge, Some("act-denied"), @@ -298,6 +301,7 @@ async fn load_file_as_allows_when_policy_permits_actor() { db.load_file_as( "main", + None, data_path.to_str().unwrap(), LoadMode::Merge, Some("act-allowed"), @@ -307,6 +311,7 @@ async fn load_file_as_allows_when_policy_permits_actor() { } #[tokio::test] +#[allow(deprecated)] async fn ingest_as_denies_when_policy_rejects_actor() { let dir = tempfile::tempdir().unwrap(); let (db, _engine) = init_with_policy(&dir).await; @@ -324,6 +329,7 @@ async fn ingest_as_denies_when_policy_rejects_actor() { } #[tokio::test] +#[allow(deprecated)] async fn ingest_as_allows_when_policy_permits_actor() { let dir = tempfile::tempdir().unwrap(); let (db, _engine) = init_with_policy(&dir).await; From c236a4c2df059c4f5ba4c6e4141c2ca1ab48aef1 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 03:57:41 +0300 Subject: [PATCH 096/165] refactor(loader): load_jsonl helpers take &Omnigraph and document their role MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The free helpers needlessly demanded &mut Omnigraph (every load API takes &self) and read as leftovers. Rather than rewriting their ~200 call sites across the test suites — which would have to re-derive the active-branch resolution at each site — keep the one convenience and make it honest: borrow immutably (&mut callers coerce, no churn) and document it as the active-branch shorthand over Omnigraph::load. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph/src/loader/mod.rs | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/crates/omnigraph/src/loader/mod.rs b/crates/omnigraph/src/loader/mod.rs index 0fabaea..09c2f7c 100644 --- a/crates/omnigraph/src/loader/mod.rs +++ b/crates/omnigraph/src/loader/mod.rs @@ -65,15 +65,18 @@ pub enum LoadMode { Merge, } -/// Load JSONL data into an Omnigraph database. -pub async fn load_jsonl(db: &mut Omnigraph, data: &str, mode: LoadMode) -> Result<LoadResult> { +/// Convenience: load JSONL data onto the database handle's *active branch* +/// (`main` when unbound). Equivalent to `db.load(active_branch, data, mode)`; +/// use `Omnigraph::load`/`load_as` directly when targeting an explicit branch +/// or when fork-from-base semantics are needed. +pub async fn load_jsonl(db: &Omnigraph, data: &str, mode: LoadMode) -> Result<LoadResult> { let current_branch = db.active_branch().await; let branch = current_branch.as_deref().unwrap_or("main"); db.load(branch, data, mode).await } -/// Load JSONL data from a file path. -pub async fn load_jsonl_file(db: &mut Omnigraph, path: &str, mode: LoadMode) -> Result<LoadResult> { +/// Convenience: like [`load_jsonl`] but reading from a file path. +pub async fn load_jsonl_file(db: &Omnigraph, path: &str, mode: LoadMode) -> Result<LoadResult> { let current_branch = db.active_branch().await; let branch = current_branch.as_deref().unwrap_or("main"); db.load_file(branch, path, mode).await From 90676ef52f9332815ffd94bba610992363392be3 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 04:05:29 +0300 Subject: [PATCH 097/165] feat(server)!: POST /ingest forks only when 'from' is present MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Branch creation becomes opt-in by presence of the request's 'from' field. Previously the handler defaulted from to 'main' and always auto-created a missing branch — a typo'd branch name silently forked main and landed the data there, with the client none the wiser. Now a request without 'from' against a missing branch returns 404 branch-not-found and creates nothing; with 'from' set, fork-if-missing behaves as before. The BranchCreate authority is only consulted when a fork will actually happen. The handler calls the unified load_as directly (the deprecated ingest_as shim is no longer used in the server). IngestOutput.base_branch becomes nullable: it echoes the request's 'from' and is null when absent. OpenAPI regenerated; the CLI's local ingest arm moves to load_file_as + the new converter shape. BREAKING CHANGE: clients that relied on implicit fork-from-main with 'from' omitted must now pass from='main' explicitly. IngestOutput.base_branch is now nullable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 9 ++-- crates/omnigraph-server/src/api.rs | 28 ++++++---- crates/omnigraph-server/src/lib.rs | 46 +++++++++------- crates/omnigraph-server/tests/server.rs | 71 +++++++++++++++++++++++++ docs/user/server.md | 2 +- openapi.json | 15 +++--- 6 files changed, 131 insertions(+), 40 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index d1fbb99..d8123ce 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -1587,7 +1587,7 @@ fn print_ingest_human(output: &IngestOutput) { "ingested {} into branch {} from {} with {} ({})", output.uri, output.branch, - output.base_branch, + output.base_branch.as_deref().unwrap_or("main"), output.mode.as_str(), if output.branch_created { "branch created" @@ -2729,11 +2729,8 @@ async fn main() -> Result<()> { } else { let db = open_local_db_with_policy(&graph).await?; let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); - // Deprecated shim retained until the CLI ingest command - // becomes an alias of the unified `load` handler. - #[allow(deprecated)] let result = db - .ingest_file_as( + .load_file_as( &branch, Some(&from), &data.to_string_lossy(), @@ -2741,7 +2738,7 @@ async fn main() -> Result<()> { actor, ) .await?; - ingest_output(&uri, &result, None) + ingest_output(&uri, &result, mode.into(), None) }; if json { print_json(&payload)?; diff --git a/crates/omnigraph-server/src/api.rs b/crates/omnigraph-server/src/api.rs index 4a6024f..ff3cf67 100644 --- a/crates/omnigraph-server/src/api.rs +++ b/crates/omnigraph-server/src/api.rs @@ -1,6 +1,6 @@ use omnigraph::db::{GraphCommit, MergeOutcome, ReadTarget, SchemaApplyResult, Snapshot}; use omnigraph::error::{MergeConflict, MergeConflictKind}; -use omnigraph::loader::{IngestResult, LoadMode}; +use omnigraph::loader::{LoadMode, LoadResult}; use crate::queries::StoredQuery; use omnigraph_compiler::SchemaMigrationStep; use omnigraph_compiler::query::ast::Param; @@ -208,7 +208,9 @@ pub struct IngestTableOutput { pub struct IngestOutput { pub uri: String, pub branch: String, - pub base_branch: String, + /// Base branch a fork was requested from (the request's `from`), echoed + /// even when the branch already existed. `null` when `from` was absent. + pub base_branch: Option<String>, pub branch_created: bool, #[schema(value_type = LoadModeSchema)] pub mode: LoadMode, @@ -493,9 +495,12 @@ pub struct SchemaOutput { #[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] pub struct IngestRequest { - /// Target branch. Created from `from` if it does not yet exist. Defaults to `main`. + /// Target branch. Defaults to `main`. Without `from`, the branch must + /// already exist — a missing branch is a 404, never an implicit fork. pub branch: Option<String>, - /// Parent branch used to create `branch` if it does not exist. Defaults to `main`. + /// Parent branch used to create `branch` if it does not exist. Branch + /// creation is opt-in by presence of this field; omit it to require an + /// existing branch. pub from: Option<String>, /// How existing rows are handled. Defaults to `merge`. #[schema(value_type = Option<LoadModeSchema>)] @@ -642,18 +647,23 @@ pub fn read_output(query_name: String, target: &ReadTarget, result: QueryResult) } } -pub fn ingest_output(uri: &str, result: &IngestResult, actor_id: Option<String>) -> IngestOutput { +pub fn ingest_output( + uri: &str, + result: &LoadResult, + mode: LoadMode, + actor_id: Option<String>, +) -> IngestOutput { IngestOutput { uri: uri.to_string(), branch: result.branch.clone(), base_branch: result.base_branch.clone(), branch_created: result.branch_created, - mode: result.mode, + mode, tables: result - .tables - .iter() + .to_ingest_tables() + .into_iter() .map(|table| IngestTableOutput { - table_key: table.table_key.clone(), + table_key: table.table_key, rows_loaded: table.rows_loaded, }) .collect(), diff --git a/crates/omnigraph-server/src/lib.rs b/crates/omnigraph-server/src/lib.rs index 26e4837..0038674 100644 --- a/crates/omnigraph-server/src/lib.rs +++ b/crates/omnigraph-server/src/lib.rs @@ -2663,13 +2663,15 @@ async fn server_schema_apply( ), security(("bearer_token" = [])), )] -/// Bulk-ingest NDJSON data into a branch. +/// Bulk-load NDJSON data into a branch. /// /// `data` is NDJSON with one record per line. `mode` controls behavior on /// existing rows: `merge` upserts by id (default), `append` blindly inserts, -/// `overwrite` replaces table contents. If `branch` does not exist it is -/// created from `from` (defaults to `main`). **Destructive** when `mode` is -/// `overwrite` or when ingest produces conflicting writes. +/// `overwrite` replaces table contents. Branch creation is opt-in by +/// presence of `from`: with `from` set, a missing `branch` is created from +/// it; without `from`, `branch` must already exist — a missing branch is a +/// 404, never an implicit fork. **Destructive** when `mode` is `overwrite` +/// or when the load produces conflicting writes. async fn server_ingest( State(state): State<AppState>, Extension(handle): Extension<Arc<GraphHandle>>, @@ -2677,7 +2679,7 @@ async fn server_ingest( Json(request): Json<IngestRequest>, ) -> std::result::Result<Json<IngestOutput>, ApiError> { let branch = request.branch.unwrap_or_else(|| "main".to_string()); - let from = request.from.unwrap_or_else(|| "main".to_string()); + let from = request.from; let mode = request.mode.unwrap_or(omnigraph::loader::LoadMode::Merge); let actor_arc = actor .as_ref() @@ -2697,15 +2699,25 @@ async fn server_ingest( }; if !branch_exists { - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::BranchCreate, - branch: Some(from.clone()), - target_branch: Some(branch.clone()), - }, - )?; + match from.as_deref() { + // Fork-if-missing is opt-in by presence of `from`; without it a + // typo'd branch name must surface as an error, not silently + // create a fork and land the data there. + None => { + return Err(ApiError::not_found(format!( + "branch '{branch}' not found; pass `from` to create it" + ))); + } + Some(from) => authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::BranchCreate, + branch: Some(from.to_string()), + target_branch: Some(branch.clone()), + }, + )?, + } } authorize_request( actor.as_ref().map(|Extension(actor)| actor), @@ -2722,12 +2734,9 @@ async fn server_ingest( .try_admit(&actor_arc, est_bytes) .map_err(ApiError::from_workload_reject)?; - // Deprecated shim retained until the from-absent semantics change - // lands; the handler then calls `load_as` directly. - #[allow(deprecated)] let result = { let db = &handle.engine; - db.ingest_as(&branch, Some(&from), &request.data, mode, actor_id) + db.load_as(&branch, from.as_deref(), &request.data, mode, actor_id) .await .map_err(ApiError::from_omni)? }; @@ -2735,6 +2744,7 @@ async fn server_ingest( Ok(Json(ingest_output( handle.uri.as_str(), &result, + mode, actor_id.map(str::to_string), ))) } diff --git a/crates/omnigraph-server/tests/server.rs b/crates/omnigraph-server/tests/server.rs index fd214bb..7858587 100644 --- a/crates/omnigraph-server/tests/server.rs +++ b/crates/omnigraph-server/tests/server.rs @@ -2265,6 +2265,77 @@ async fn ingest_existing_branch_skips_branch_create_policy_check() { assert_eq!(body["base_branch"], "other-base"); } +/// Regression: branch creation is opt-in by presence of `from`. A request +/// without `from` against a branch that doesn't exist must 404 — not +/// silently fork `main` and land the data on the typo'd branch. +#[tokio::test(flavor = "multi_thread")] +async fn ingest_without_from_returns_404_for_missing_branch_and_creates_nothing() { + let (temp, app) = app_for_loaded_graph().await; + let graph = graph_path(temp.path()); + let ingest = IngestRequest { + branch: Some("feature-typo".to_string()), + from: None, + mode: Some(LoadMode::Merge), + data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), + }; + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/ingest") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&ingest).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::NOT_FOUND); + let error: ErrorOutput = serde_json::from_value(body).unwrap(); + assert_eq!(error.code, Some(omnigraph_server::api::ErrorCode::NotFound)); + + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + assert!( + !db.branch_list() + .await + .unwrap() + .contains(&"feature-typo".to_string()), + "a 404'd ingest must not create the branch" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn ingest_without_from_loads_into_existing_branch() { + let (temp, app) = app_for_loaded_graph().await; + let graph = graph_path(temp.path()); + { + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.branch_create_from(ReadTarget::branch("main"), "feature") + .await + .unwrap(); + } + let ingest = IngestRequest { + branch: Some("feature".to_string()), + from: None, + mode: Some(LoadMode::Merge), + data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), + }; + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/ingest") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&ingest).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(body["branch"], "feature"); + assert_eq!(body["branch_created"], false); + assert_eq!(body["base_branch"], serde_json::Value::Null); +} + #[tokio::test(flavor = "multi_thread")] async fn ingest_denies_missing_branch_without_branch_create_permission() { let (_temp, app) = app_for_loaded_graph_with_auth_tokens_and_policy( diff --git a/docs/user/server.md b/docs/user/server.md index 60988ca..0922e74 100644 --- a/docs/user/server.md +++ b/docs/user/server.md @@ -56,7 +56,7 @@ Per-graph endpoints — same body shape across modes; URLs differ: | POST | `/queries/{name}` | `/graphs/{id}/queries/{name}` | bearer + `invoke_query` (+ `change` for a stored mutation) | invoke a named query from the `queries:` registry; deny == 404 | `server_invoke_query` | | GET | `/schema` | `/graphs/{id}/schema` | bearer + `read` | get current `.pg` source | `server_schema_get` | | POST | `/schema/apply` | `/graphs/{id}/schema/apply` | bearer + `schema_apply` (target=`main`) | migrate | `server_schema_apply` | -| POST | `/ingest` | `/graphs/{id}/ingest` | bearer + `branch_create` (if new) + `change` | bulk load | `server_ingest` (32 MB body limit) | +| POST | `/ingest` | `/graphs/{id}/ingest` | bearer + `branch_create` (only when `from` is set and the branch is created) + `change` | bulk load; branch creation is opt-in via `from` — without it a missing `branch` is a 404, never an implicit fork | `server_ingest` (32 MB body limit) | | GET | `/branches` | `/graphs/{id}/branches` | bearer + `read` | list branches | `server_branch_list` | | POST | `/branches` | `/graphs/{id}/branches` | bearer + `branch_create` | create | `server_branch_create` | | DELETE | `/branches/{branch}` | `/graphs/{id}/branches/{branch}` | bearer + `branch_delete` | delete | `server_branch_delete` | diff --git a/openapi.json b/openapi.json index 335c0bc..85c5b8d 100644 --- a/openapi.json +++ b/openapi.json @@ -670,8 +670,8 @@ "tags": [ "mutations" ], - "summary": "Bulk-ingest NDJSON data into a branch.", - "description": "`data` is NDJSON with one record per line. `mode` controls behavior on\nexisting rows: `merge` upserts by id (default), `append` blindly inserts,\n`overwrite` replaces table contents. If `branch` does not exist it is\ncreated from `from` (defaults to `main`). **Destructive** when `mode` is\n`overwrite` or when ingest produces conflicting writes.", + "summary": "Bulk-load NDJSON data into a branch.", + "description": "`data` is NDJSON with one record per line. `mode` controls behavior on\nexisting rows: `merge` upserts by id (default), `append` blindly inserts,\n`overwrite` replaces table contents. Branch creation is opt-in by\npresence of `from`: with `from` set, a missing `branch` is created from\nit; without `from`, `branch` must already exist — a missing branch is a\n404, never an implicit fork. **Destructive** when `mode` is `overwrite`\nor when the load produces conflicting writes.", "operationId": "ingest", "requestBody": { "content": { @@ -1710,7 +1710,6 @@ "required": [ "uri", "branch", - "base_branch", "branch_created", "mode", "tables" @@ -1723,7 +1722,11 @@ ] }, "base_branch": { - "type": "string" + "type": [ + "string", + "null" + ], + "description": "Base branch a fork was requested from (the request's `from`), echoed\neven when the branch already existed. `null` when `from` was absent." }, "branch": { "type": "string" @@ -1756,7 +1759,7 @@ "string", "null" ], - "description": "Target branch. Created from `from` if it does not yet exist. Defaults to `main`." + "description": "Target branch. Defaults to `main`. Without `from`, the branch must\nalready exist — a missing branch is a 404, never an implicit fork." }, "data": { "type": "string", @@ -1768,7 +1771,7 @@ "string", "null" ], - "description": "Parent branch used to create `branch` if it does not exist. Defaults to `main`." + "description": "Parent branch used to create `branch` if it does not exist. Branch\ncreation is opt-in by presence of this field; omit it to require an\nexisting branch." }, "mode": { "oneOf": [ From fa6af775c1c3d16d04d156e8b2fe3444bdf3fcef Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 04:18:00 +0300 Subject: [PATCH 098/165] feat(cli)!: unified load command; deprecate ingest as an alias MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit omnigraph load is now the single data-write command: - works against remote graphs (POSTs the server's /ingest endpoint with the same bearer/actor resolution as other remote commands) — previously load was the only data command forced to open Lance storage directly - --from <base> opts into fork-if-missing for --branch (the former ingest semantics); without --from a missing branch is an error, never a fork - --mode is now required: overwrite is destructive, so there is no implicit default (the old silent default was overwrite) - output gains base_branch/branch_created (and table sums on remote loads) omnigraph ingest stays as a deprecated alias (defaults preserved: --from main --mode merge) that prints a one-line warning to stderr, matching the read/change deprecation convention; removal in a later release. Docs updated in the same change: cli.md, cli-reference.md, policy.md, audit.md, execution.md (unified load section), AGENTS.md quick-flow, README.md. BREAKING CHANGE: scripts running omnigraph load without --mode must now pass it explicitly (previously defaulted to the destructive overwrite). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- AGENTS.md | 7 +- README.md | 4 +- crates/omnigraph-cli/src/main.rs | 159 ++++++++++++++------ crates/omnigraph-cli/tests/cli.rs | 14 +- crates/omnigraph-cli/tests/support/mod.rs | 10 +- crates/omnigraph-cli/tests/system_local.rs | 126 +++++++++++++++- crates/omnigraph-cli/tests/system_remote.rs | 67 +++++++++ docs/dev/execution.md | 10 +- docs/user/audit.md | 2 +- docs/user/cli-reference.md | 4 +- docs/user/cli.md | 2 +- docs/user/policy.md | 5 +- 12 files changed, 342 insertions(+), 68 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 60276ad..b335955 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -200,9 +200,8 @@ omnigraph init --schema ./schema.pg s3://my-bucket/graph.omni # Bulk load omnigraph load --data ./seed.jsonl --mode overwrite s3://my-bucket/graph.omni -# Branch + ingest a review batch -omnigraph branch create --from main review/2026-04-25 s3://my-bucket/graph.omni -omnigraph ingest --branch review/2026-04-25 --data ./batch.jsonl s3://my-bucket/graph.omni +# Load a review batch onto its own branch (--from forks it if missing) +omnigraph load --branch review/2026-04-25 --from main --mode merge --data ./batch.jsonl s3://my-bucket/graph.omni # Run a hybrid (vector + BM25) query omnigraph read --query ./queries.gq --name find_similar \ @@ -258,7 +257,7 @@ omnigraph policy explain --actor act-alice --action change --branch main | Per-query atomic writes | — | In-memory `MutationStaging.pending` accumulator + `stage_*` / `commit_staged` per touched table at end-of-query + publisher CAS via `commit_with_expected` (single manifest commit per `mutate_as` / `load`); D₂ parse-time rule keeps inserts/updates and deletes from mixing | | Three-way row-level merge | — | `OrderedTableCursor` + `StagedTableWriter`, structured `MergeConflictKind` | | Change feeds | — | `diff_between` / `diff_commits` with manifest fast path + ID streaming | -| Cedar policy | — | Per-graph actions plus server-scoped actions (see [docs/user/policy.md](docs/user/policy.md) for the current list), branch / target_branch / protected scopes, validate/test/explain CLI. **Engine-wide enforcement** (MR-722): every `_as` writer (`apply_schema_as`, `mutate_as`, `load_as`, `ingest_as`, `branch_create_as` / `branch_create_from_as`, `branch_delete_as`, `branch_merge_as`) calls `Omnigraph::enforce(action, scope, actor)` — HTTP, CLI, embedded SDK all hit the same gate. | +| Cedar policy | — | Per-graph actions plus server-scoped actions (see [docs/user/policy.md](docs/user/policy.md) for the current list), branch / target_branch / protected scopes, validate/test/explain CLI. **Engine-wide enforcement** (MR-722): every `_as` writer (`apply_schema_as`, `mutate_as`, `load_as` — the deprecated `ingest_as` shims route through it — `branch_create_as` / `branch_create_from_as`, `branch_delete_as`, `branch_merge_as`) calls `Omnigraph::enforce(action, scope, actor)` — HTTP, CLI, embedded SDK all hit the same gate. | | HTTP server | — | Axum, OpenAPI via utoipa, bearer auth (SHA-256, AWS Secrets Manager option), `authorize_request` at the HTTP boundary (resolves bearer→actor, applies admission control), NDJSON streaming export, **multi-graph mode (v0.6.0+) with cluster routes + read-only `GET /graphs` enumeration + per-graph + server-level Cedar policies. Add/remove graphs by editing `omnigraph.yaml` and restarting.** | | CLI with config | — | `omnigraph.yaml`, aliases, multi-format output (json/jsonl/csv/kv/table) | | Audit / actor tracking | — | `_as` write APIs + actor map in commit graph | diff --git a/README.md b/README.md index 0f6ebea..a75a839 100644 --- a/README.md +++ b/README.md @@ -88,7 +88,7 @@ omnigraph branch create --from main feature-x ./graph.omni omnigraph branch merge feature-x --into main ./graph.omni ``` -See [docs/user/cli.md](docs/user/cli.md) for schema apply, snapshots, ingest, commits, and policy commands. +See [docs/user/cli.md](docs/user/cli.md) for schema apply, snapshots, data loading, commits, and policy commands. ## Clients @@ -132,7 +132,7 @@ Notes: - `crates/omnigraph-compiler`: shared schema/query parser, typechecker, catalog, and IR lowering - `crates/omnigraph`: storage/runtime, branching, merge, change detection, and query execution -- `crates/omnigraph-cli`: CLI for graph lifecycle (init/load/ingest), query/mutate, branch/commit/merge, schema/lint, snapshot/export, policy, and maintenance (optimize/cleanup) +- `crates/omnigraph-cli`: CLI for graph lifecycle (init/load), query/mutate, branch/commit/merge, schema/lint, snapshot/export, policy, and maintenance (optimize/cleanup) - `crates/omnigraph-server`: Axum HTTP server for remote reads, changes, ingest, export, branches, and commits ## Contributing diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index d8123ce..da3cc44 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -89,7 +89,7 @@ enum Command { #[arg(long)] force: bool, }, - /// Load data into a graph + /// Load data into a graph (local or remote) Load { /// Graph URI uri: Option<String>, @@ -99,14 +99,21 @@ enum Command { config: Option<PathBuf>, #[arg(long)] data: PathBuf, + /// Target branch (defaults to main). Without --from it must exist. #[arg(long)] branch: Option<String>, - #[arg(long, default_value = "overwrite")] + /// Base branch to fork --branch from when it doesn't exist yet. + /// Without this flag a missing branch is an error, never a fork. + #[arg(long)] + from: Option<String>, + /// How existing rows are handled: overwrite | append | merge. + /// Required — overwrite is destructive, so there is no default. + #[arg(long)] mode: CliLoadMode, #[arg(long)] json: bool, }, - /// Ingest data into a reviewable named branch + /// Deprecated alias of `load --from <base>` (defaults: --mode merge, --from main) Ingest { /// Graph URI uri: Option<String>, @@ -686,16 +693,55 @@ impl CliLoadMode { } #[derive(Debug, Serialize)] -struct LoadOutput<'a> { - uri: &'a str, - branch: &'a str, - mode: &'a str, +struct LoadOutput { + uri: String, + branch: String, + mode: &'static str, + /// Present only when `--from` was given; echoes the requested base. + #[serde(skip_serializing_if = "Option::is_none")] + base_branch: Option<String>, + branch_created: bool, nodes_loaded: usize, edges_loaded: usize, node_types_loaded: usize, edge_types_loaded: usize, } +/// Map a remote `/ingest` response onto the CLI's load output. Table keys +/// carry `node:`/`edge:` prefixes, so the per-kind sums are derivable +/// client-side without the catalog. +fn load_output_from_tables( + uri: &str, + branch: &str, + mode: CliLoadMode, + output: &IngestOutput, +) -> LoadOutput { + let mut nodes_loaded = 0; + let mut edges_loaded = 0; + let mut node_types_loaded = 0; + let mut edge_types_loaded = 0; + for table in &output.tables { + if table.table_key.starts_with("node:") { + nodes_loaded += table.rows_loaded; + node_types_loaded += 1; + } else if table.table_key.starts_with("edge:") { + edges_loaded += table.rows_loaded; + edge_types_loaded += 1; + } + } + LoadOutput { + uri: uri.to_string(), + branch: branch.to_string(), + mode: mode.as_str(), + base_branch: output.base_branch.clone(), + branch_created: output.branch_created, + nodes_loaded, + edges_loaded, + node_types_loaded, + edge_types_loaded, + } +} + #[derive(Debug, Serialize)] struct SchemaPlanOutput<'a> { uri: &'a str, @@ -1561,25 +1607,22 @@ fn merged_params_json( } } -fn print_load_human( - uri: &str, - branch: &str, - mode: CliLoadMode, - nodes_loaded: usize, - edges_loaded: usize, - node_types_loaded: usize, - edge_types_loaded: usize, -) { +fn print_load_human(payload: &LoadOutput) { println!( "loaded {} on branch {} with {}: {} nodes across {} node types, {} edges across {} edge types", - uri, - branch, - mode.as_str(), - nodes_loaded, - node_types_loaded, - edges_loaded, - edge_types_loaded + payload.uri, + payload.branch, + payload.mode, + payload.nodes_loaded, + payload.node_types_loaded, + payload.edges_loaded, + payload.edge_types_loaded ); + if payload.branch_created { + if let Some(base) = &payload.base_branch { + println!("branch {} created from {}", payload.branch, base); + } + } } fn print_ingest_human(output: &IngestOutput) { @@ -2659,39 +2702,60 @@ async fn main() -> Result<()> { config, data, branch, + from, mode, json, } => { let config = load_cli_config(config.as_ref())?; - let graph = resolve_local_graph(&config, uri, target.as_deref(), "load")?; + let bearer_token = + resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; + let graph = resolve_cli_graph(&config, uri, target.as_deref())?; let uri = graph.uri.clone(); let branch = resolve_branch(&config, branch, None, "main"); - let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); - let result = db - .load_file_as(&branch, None, &data.to_string_lossy(), mode.into(), actor) + let payload = if graph.is_remote { + let data = fs::read_to_string(&data)?; + let output = remote_json::<IngestOutput>( + &http_client, + Method::POST, + remote_url(&uri, "/ingest"), + Some(serde_json::to_value(IngestRequest { + branch: Some(branch.clone()), + from: from.clone(), + mode: Some(mode.into()), + data, + })?), + bearer_token.as_deref(), + ) .await?; - let payload = LoadOutput { - uri: &uri, - branch: &branch, - mode: mode.as_str(), - nodes_loaded: result.nodes_loaded.values().sum(), - edges_loaded: result.edges_loaded.values().sum(), - node_types_loaded: result.nodes_loaded.len(), - edge_types_loaded: result.edges_loaded.len(), + load_output_from_tables(&uri, &branch, mode, &output) + } else { + let db = open_local_db_with_policy(&graph).await?; + let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); + let result = db + .load_file_as( + &branch, + from.as_deref(), + &data.to_string_lossy(), + mode.into(), + actor, + ) + .await?; + LoadOutput { + uri: uri.clone(), + branch: branch.clone(), + mode: mode.as_str(), + base_branch: result.base_branch.clone(), + branch_created: result.branch_created, + nodes_loaded: result.nodes_loaded.values().sum(), + edges_loaded: result.edges_loaded.values().sum(), + node_types_loaded: result.nodes_loaded.len(), + edge_types_loaded: result.edges_loaded.len(), + } }; if json { print_json(&payload)?; } else { - print_load_human( - &uri, - &branch, - mode, - payload.nodes_loaded, - payload.edges_loaded, - payload.node_types_loaded, - payload.edge_types_loaded, - ); + print_load_human(&payload); } } Command::Ingest { @@ -2704,6 +2768,11 @@ async fn main() -> Result<()> { mode, json, } => { + // stderr so `--json` consumers reading stdout are unaffected. + eprintln!( + "warning: `omnigraph ingest` is deprecated and will be removed in a future release; \ + use `omnigraph load --from <base> --mode <mode>` (ingest defaults: --from main --mode merge)" + ); let config = load_cli_config(config.as_ref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs index ab3c23b..3f21b8a 100644 --- a/crates/omnigraph-cli/tests/cli.rs +++ b/crates/omnigraph-cli/tests/cli.rs @@ -2650,6 +2650,8 @@ fn load_json_outputs_summary_for_main_branch() { let output = output_success( cli() .arg("load") + .arg("--mode") + .arg("overwrite") .arg("--data") .arg(&data) .arg("--json") @@ -2984,7 +2986,15 @@ fn read_alias_uses_alias_target_without_cli_default_and_accepts_url_like_arg() { &data, r#"{"type":"Person","data":{"name":"https://example.com","age":30}}"#, ); - output_success(cli().arg("load").arg("--data").arg(&data).arg(&graph)); + output_success( + cli() + .arg("load") + .arg("--mode") + .arg("overwrite") + .arg("--data") + .arg(&data) + .arg(&graph), + ); write_query_file( &query, &std::fs::read_to_string(fixture("test.gq")).unwrap(), @@ -3748,6 +3758,8 @@ fn cli_fails_for_missing_schema_or_data_file() { let load_output = output_failure( cli() .arg("load") + .arg("--mode") + .arg("overwrite") .arg("--data") .arg(&missing_data) .arg(&graph), diff --git a/crates/omnigraph-cli/tests/support/mod.rs b/crates/omnigraph-cli/tests/support/mod.rs index c30ed28..653be11 100644 --- a/crates/omnigraph-cli/tests/support/mod.rs +++ b/crates/omnigraph-cli/tests/support/mod.rs @@ -93,7 +93,15 @@ pub fn init_graph(graph: &Path) { pub fn load_fixture(graph: &Path) { let data = fixture("test.jsonl"); - output_success(cli().arg("load").arg("--data").arg(&data).arg(graph)); + output_success( + cli() + .arg("load") + .arg("--mode") + .arg("overwrite") + .arg("--data") + .arg(&data) + .arg(graph), + ); } pub fn write_jsonl(path: &Path, rows: &str) { diff --git a/crates/omnigraph-cli/tests/system_local.rs b/crates/omnigraph-cli/tests/system_local.rs index adb5dc8..46f6fcf 100644 --- a/crates/omnigraph-cli/tests/system_local.rs +++ b/crates/omnigraph-cli/tests/system_local.rs @@ -221,6 +221,8 @@ fn local_cli_end_to_end_init_load_read_change_read_flow() { output_success( cli() .arg("load") + .arg("--mode") + .arg("overwrite") .arg("--data") .arg(fixture("test.jsonl")) .arg(graph.path()), @@ -397,7 +399,7 @@ fn local_cli_ingest_creates_review_branch_and_keeps_it_readable() { {"type":"Person","data":{"name":"Bob","age":26}}"#, ); - let ingest_payload = parse_stdout_json(&output_success( + let ingest_output = output_success( cli() .arg("ingest") .arg("--data") @@ -406,7 +408,13 @@ fn local_cli_ingest_creates_review_branch_and_keeps_it_readable() { .arg("feature-ingest") .arg(graph.path()) .arg("--json"), - )); + ); + // The deprecation warning goes to stderr so --json stdout stays clean. + assert!( + String::from_utf8_lossy(&ingest_output.stderr).contains("deprecated"), + "ingest must warn about its deprecation on stderr" + ); + let ingest_payload = parse_stdout_json(&ingest_output); assert_eq!(ingest_payload["branch"], "feature-ingest"); assert_eq!(ingest_payload["base_branch"], "main"); assert_eq!(ingest_payload["branch_created"], true); @@ -459,6 +467,88 @@ fn local_cli_ingest_creates_review_branch_and_keeps_it_readable() { assert_eq!(bob["rows"][0]["p.age"], 26); } +/// The unified `load` subsumes ingest: `--from` opts into fork-if-missing, +/// while without it a missing branch is an error — never an implicit fork. +#[test] +fn local_cli_load_from_forks_branch_and_missing_branch_errors_without_from() { + let graph = SystemGraph::loaded(); + let extra = graph.write_jsonl( + "system-local-load-from.jsonl", + r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#, + ); + + // Without --from, a missing branch must fail and create nothing. + let failure = output_failure( + cli() + .arg("load") + .arg("--mode") + .arg("merge") + .arg("--data") + .arg(&extra) + .arg("--branch") + .arg("feature-load") + .arg(graph.path()), + ); + assert!( + String::from_utf8_lossy(&failure.stderr).contains("feature-load"), + "error should name the missing branch" + ); + + // With --from, the branch is forked and the load lands on it. + let payload = parse_stdout_json(&output_success( + cli() + .arg("load") + .arg("--mode") + .arg("merge") + .arg("--data") + .arg(&extra) + .arg("--branch") + .arg("feature-load") + .arg("--from") + .arg("main") + .arg(graph.path()) + .arg("--json"), + )); + assert_eq!(payload["branch"], "feature-load"); + assert_eq!(payload["base_branch"], "main"); + assert_eq!(payload["branch_created"], true); + assert_eq!(payload["mode"], "merge"); + assert_eq!(payload["nodes_loaded"], 1); + + let snapshot = parse_stdout_json(&output_success( + cli() + .arg("snapshot") + .arg(graph.path()) + .arg("--branch") + .arg("feature-load") + .arg("--json"), + )); + assert_eq!(snapshot["branch"], "feature-load"); +} + +/// `--mode` is required: overwrite is destructive, so the unified `load` +/// has no implicit default. +#[test] +fn local_cli_load_requires_mode_flag() { + let graph = SystemGraph::loaded(); + let extra = graph.write_jsonl( + "system-local-load-no-mode.jsonl", + r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#, + ); + + let failure = output_failure( + cli() + .arg("load") + .arg("--data") + .arg(&extra) + .arg(graph.path()), + ); + assert!( + String::from_utf8_lossy(&failure.stderr).contains("--mode"), + "clap should demand the missing --mode flag" + ); +} + #[test] fn local_cli_export_round_trips_full_branch_graph() { let graph = SystemGraph::loaded(); @@ -512,6 +602,8 @@ fn local_cli_export_round_trips_full_branch_graph() { output_success( cli() .arg("load") + .arg("--mode") + .arg("overwrite") .arg("--data") .arg(&export_path) .arg(&imported_graph), @@ -610,6 +702,8 @@ policy: {{}} cli() .current_dir(query_root) .arg("load") + .arg("--mode") + .arg("overwrite") .arg("--data") .arg(fixture("test.jsonl")) .arg(&graph_uri), @@ -867,7 +961,15 @@ query get_task($slug: String) { ); output_success(cli().arg("init").arg("--schema").arg(&schema).arg(&graph)); - output_success(cli().arg("load").arg("--data").arg(&data).arg(&graph)); + output_success( + cli() + .arg("load") + .arg("--mode") + .arg("overwrite") + .arg("--data") + .arg(&data) + .arg(&graph), + ); let filtered = parse_stdout_json(&output_success( cli() @@ -997,7 +1099,15 @@ query vector_search($q: String) { ); output_success(cli().arg("init").arg("--schema").arg(&schema).arg(&graph)); - output_success(cli().arg("load").arg("--data").arg(&data).arg(&graph)); + output_success( + cli() + .arg("load") + .arg("--mode") + .arg("overwrite") + .arg("--data") + .arg(&data) + .arg(&graph), + ); let result = parse_stdout_json(&output_success( cli() @@ -1221,6 +1331,8 @@ fn local_cli_load_enforces_engine_layer_policy() { .arg("--as") .arg("act-bruno") .arg("load") + .arg("--mode") + .arg("overwrite") .arg("--config") .arg(&config) .arg("--data") @@ -1239,6 +1351,8 @@ fn local_cli_load_enforces_engine_layer_policy() { .arg("--as") .arg("act-ragnor") .arg("load") + .arg("--mode") + .arg("overwrite") .arg("--config") .arg(&config) .arg("--data") @@ -1684,6 +1798,8 @@ graphs: std::fs::write(&data, "{\"type\":\"Person\",\"data\":{\"name\":\"Ada\"}}\n").unwrap(); let output = cli() .arg("load") + .arg("--mode") + .arg("overwrite") .arg("--data") .arg(&data) .arg(temp.path().join("graphs/knowledge.omni")) @@ -1796,6 +1912,8 @@ fn seed_graph(dir: &std::path::Path, graph: &str, row: &str) { std::fs::write(&data, row).unwrap(); let output = cli() .arg("load") + .arg("--mode") + .arg("overwrite") .arg("--data") .arg(&data) .arg(dir.join(format!("graphs/{graph}.omni"))) diff --git a/crates/omnigraph-cli/tests/system_remote.rs b/crates/omnigraph-cli/tests/system_remote.rs index 45bf502..95a53e7 100644 --- a/crates/omnigraph-cli/tests/system_remote.rs +++ b/crates/omnigraph-cli/tests/system_remote.rs @@ -652,6 +652,8 @@ query add_friend($from: String, $to: String) { output_success( cli() .arg("load") + .arg("--mode") + .arg("overwrite") .arg("--data") .arg(&export_path) .arg(&imported_graph), @@ -755,6 +757,71 @@ fn remote_ingest_creates_review_branch_and_keeps_it_readable() { assert_eq!(zoe["rows"][0]["p.name"], "Zoe"); } +/// The unified `load` works against remote graphs through the server's +/// `/ingest` endpoint: without `--from` a missing branch is a hard error +/// (no implicit fork), with `--from` it forks like ingest did. +#[test] +#[ignore = "requires loopback socket permissions in sandboxed runners"] +fn remote_load_round_trips_and_requires_from_for_new_branches() { + let graph = SystemGraph::loaded(); + let server = graph.spawn_server(); + let config = graph.write_config("omnigraph.yaml", &remote_yaml_config(&server.base_url)); + let extra = graph.write_jsonl( + "system-remote-load.jsonl", + r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#, + ); + + // Missing branch without --from: refused remotely, nothing created. + let failure = output_failure( + cli() + .arg("load") + .arg("--config") + .arg(&config) + .arg("--mode") + .arg("merge") + .arg("--data") + .arg(&extra) + .arg("--branch") + .arg("feature-load"), + ); + assert!( + String::from_utf8_lossy(&failure.stderr).contains("feature-load"), + "error should name the missing branch" + ); + + // With --from, the remote load forks and lands the rows. + let payload = parse_stdout_json(&output_success( + cli() + .arg("load") + .arg("--config") + .arg(&config) + .arg("--mode") + .arg("merge") + .arg("--data") + .arg(&extra) + .arg("--branch") + .arg("feature-load") + .arg("--from") + .arg("main") + .arg("--json"), + )); + assert_eq!(payload["branch"], "feature-load"); + assert_eq!(payload["base_branch"], "main"); + assert_eq!(payload["branch_created"], true); + assert_eq!(payload["nodes_loaded"], 1); + + let snapshot = parse_stdout_json(&output_success( + cli() + .arg("snapshot") + .arg("--config") + .arg(&config) + .arg("--branch") + .arg("feature-load") + .arg("--json"), + )); + assert_eq!(snapshot["branch"], "feature-load"); +} + #[test] #[ignore = "requires loopback socket permissions in sandboxed runners"] fn remote_ingest_reuses_existing_branch_and_merges_updates() { diff --git a/docs/dev/execution.md b/docs/dev/execution.md index 9753696..0e8e3fc 100644 --- a/docs/dev/execution.md +++ b/docs/dev/execution.md @@ -168,12 +168,12 @@ Atomicity guarantee for multi-statement mutations: a mid-query failure leaves La For all three modes, a mid-load failure (RI / cardinality violation, validation error) leaves Lance HEAD untouched on the staged tables — the next load on the same tables proceeds normally with no `ExpectedVersionMismatch`. -## `load` vs `ingest` +## `load` and the deprecated `ingest` shims -- `load(branch, data, mode)` — direct load to a branch (single publisher commit per call). -- `ingest(branch, from, data, mode)` — branch-creating wrapper: if `branch` doesn't exist, fork it from `from` (default `main`) via `branch_create_from`, then call `load(branch, data, mode)`. -- Returns `IngestResult { branch, base_branch, branch_created, mode, tables[] }`. -- `ingest_as(actor_id)` records the actor on the resulting commit. +- `load_as(branch, base, data, mode, actor)` — the unified entry (single publisher commit per call). `base: Some(b)` forks a missing `branch` from `b` first (via `branch_create_from_as`, which enforces `BranchCreate`); `base: None` requires the branch to exist — staging fails on an unknown branch, so a typo'd name can never create one. +- `load(branch, data, mode)` — convenience wrapper with `base: None` and no actor. +- Returns `LoadResult { branch, base_branch, branch_created, nodes_loaded, edges_loaded }`. +- `ingest{,_as,_file,_file_as}` are `#[deprecated]` shims over `load_as` preserving the historical contract (`from: None` forks from `main`; returns `IngestResult`); they are slated for removal. The CLI `ingest` command is a deprecated alias of `load --from <base>`. ## Embeddings during load diff --git a/docs/user/audit.md b/docs/user/audit.md index ab028ac..52cecde 100644 --- a/docs/user/audit.md +++ b/docs/user/audit.md @@ -1,7 +1,7 @@ # Audit / Actor tracking - `Omnigraph::audit_actor_id: Option<String>` is the actor in effect. -- `_as` variants of every write API let callers override the actor: `mutate_as`, `ingest_as`, `branch_merge_as`, `apply_schema_as`, etc. +- `_as` variants of every write API let callers override the actor: `mutate_as`, `load_as`, `branch_merge_as`, `apply_schema_as`, etc. - Actor IDs are persisted on `GraphCommit.actor_id` with split storage in `_graph_commit_actors.lance` (the commit graph is split into `_graph_commits.lance` for the linkage and `_graph_commit_actors.lance` for the actor map). - HTTP server uses the bearer-token actor automatically; CLI uses the local user / explicit env (no implicit actor). - Pre-v0.4.0 graphs also stored actor IDs on `RunRecord.actor_id` in `_graph_runs.lance` / `_graph_run_actors.lance`. The Run state machine was removed in MR-771; those files are inert post-v0.4.0. The v2→v3 manifest migration sweeps any stale `__run__*` branches on first write-open (MR-770); the inert dataset bytes remain until a `delete_prefix` primitive lands. diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index fb12dd8..74d772f 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -9,8 +9,8 @@ Top-level command families and subcommands. Graph-targeting commands accept eith | Command | Purpose | |---|---| | `init` | `--schema <pg>` → initialize a graph (also scaffolds `omnigraph.yaml` if missing) | -| `load` | bulk load a branch (`--mode overwrite\|append\|merge`) | -| `ingest` | branch-creating transactional load (`--from <base>`) | +| `load` | bulk load a branch, local or remote (`--mode overwrite\|append\|merge` is **required** — overwrite is destructive, so there is no default). Without `--from` the target branch must exist; `--from <base>` forks a missing `--branch` from `<base>` first | +| `ingest` | deprecated alias of `load --from <base>` (defaults: `--from main --mode merge`); prints a one-line warning to stderr | | `query` (alias: `read`) | run named read query; source via `--query <path>`, `-e`/`--query-string <GQ>`, or `--alias <name>` (exactly one). `read` is the deprecated previous name and prints a one-line warning to stderr | | `mutate` (alias: `change`) | run mutation query; same `--query` / `-e` / `--alias` mutual-exclusion as `query`. `change` is the deprecated previous name and prints a one-line warning to stderr | | `snapshot` | print current snapshot (per-table version + row count) | diff --git a/docs/user/cli.md b/docs/user/cli.md index 5c4297a..a6ce442 100644 --- a/docs/user/cli.md +++ b/docs/user/cli.md @@ -42,7 +42,7 @@ omnigraph branch create --uri graph.omni --from main feature-x omnigraph branch list --uri graph.omni omnigraph branch merge --uri graph.omni feature-x --into main -omnigraph ingest --data batch.jsonl --branch review/import-2026-04-09 graph.omni +omnigraph load --data batch.jsonl --branch review/import-2026-04-09 --from main --mode merge graph.omni omnigraph export graph.omni --branch main --type Person > people.jsonl omnigraph commit list graph.omni --branch main --json omnigraph commit show --uri graph.omni <commit-id> --json diff --git a/docs/user/policy.md b/docs/user/policy.md index 9c484ba..91684d8 100644 --- a/docs/user/policy.md +++ b/docs/user/policy.md @@ -105,12 +105,13 @@ is validated/tested/explained as the anonymous policy. - `omnigraph policy validate` — parse + count actors, exit 1 on parse error. - `omnigraph policy test` — run cases in `policy.tests.yaml`, exit 1 on any expectation mismatch. - `omnigraph policy explain --actor … --action … [--branch …] [--target-branch …]` — show decision and matched rule. -- `omnigraph --as <ACTOR> <subcommand>` — set the actor for the duration of one invocation. Effective for `change`, `load`, `ingest`, `branch create|delete|merge`, and `schema apply` against local URIs. No-op against remote HTTP URIs (actor is bearer-token-resolved server-side). +- `omnigraph --as <ACTOR> <subcommand>` — set the actor for the duration of one invocation. Effective for `change`, `load` (and its deprecated `ingest` alias), `branch create|delete|merge`, and `schema apply` against local URIs. No-op against remote HTTP URIs (actor is bearer-token-resolved server-side). ## Enforcement Policy is a property of the **engine**, not the transport. Every mutating -write — `mutate_as`, `load_as`, `ingest_as`, `apply_schema_as`, +write — `mutate_as`, `load_as` (the deprecated `ingest_as` shims route +through it), `apply_schema_as`, `branch_create_as`, `branch_create_from_as`, `branch_delete_as`, `branch_merge_as` — calls `Omnigraph::enforce(action, scope, actor)` at the head of the method. The gate fires identically whether the call From f48e69b999d8fd4032dc6aa67c244ce42d80fc27 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 05:09:45 +0300 Subject: [PATCH 099/165] feat(storage): versioned CAS, conditional replace, and prefix delete on StorageAdapter MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three primitives the cluster's object-storage port (RFC-006) needs, on the engine's existing adapter rather than a parallel store: - read_text_versioned: content + an opaque backend version token (S3: the ETag from GET; local: content sha256 — ETags don't exist on a filesystem). - write_text_if_match: replace only when the token still matches. S3 maps to a conditional put (PutMode::Update / If-Match) — verified against RustFS beta.8 through the real object_store 0.12.5 path, no extra builder config needed; local compares content then swaps via temp+rename, the same single-machine semantics callers had before this trait (safe under their own lock protocol, not a cross-process barrier by itself). CAS-lost is Ok(None), never silent. - delete_prefix: recursive + idempotent (local remove_dir_all; S3 list + delete, with the non-atomicity documented for crash-retry callers). Gated S3 coverage: s3_adapter_conditional_writes_contract pins the conditional-write behavior the cluster ledger will depend on (red if a backend bump regresses it), and s3_schema_apply_migrates_live_graph closes the previously-untested schema-apply-on-S3 path before the cluster's schema executor leans on it. Engine gains the sha2 workspace dep. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- Cargo.lock | 1 + crates/omnigraph/Cargo.toml | 1 + crates/omnigraph/src/db/omnigraph.rs | 38 +++++ crates/omnigraph/src/storage.rs | 203 +++++++++++++++++++++++++++ crates/omnigraph/tests/s3_storage.rs | 77 ++++++++++ 5 files changed, 320 insertions(+) diff --git a/Cargo.lock b/Cargo.lock index 675fad7..b1cf0ef 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4636,6 +4636,7 @@ dependencies = [ "serde", "serde_json", "serial_test", + "sha2", "tempfile", "thiserror", "time", diff --git a/crates/omnigraph/Cargo.toml b/crates/omnigraph/Cargo.toml index 9cc2148..a4a2fe0 100644 --- a/crates/omnigraph/Cargo.toml +++ b/crates/omnigraph/Cargo.toml @@ -37,6 +37,7 @@ serde_json = { workspace = true } reqwest = { workspace = true } object_store = { workspace = true } ulid = { workspace = true } +sha2 = { workspace = true } base64 = { workspace = true } futures = { workspace = true } tracing = { workspace = true } diff --git a/crates/omnigraph/src/db/omnigraph.rs b/crates/omnigraph/src/db/omnigraph.rs index f217f7d..50f5d34 100644 --- a/crates/omnigraph/src/db/omnigraph.rs +++ b/crates/omnigraph/src/db/omnigraph.rs @@ -2029,6 +2029,25 @@ edge WorksAt: Person -> Company async fn list_dir(&self, dir_uri: &str) -> Result<Vec<String>> { self.inner.list_dir(dir_uri).await } + + async fn read_text_versioned(&self, uri: &str) -> Result<(String, String)> { + self.inner.read_text_versioned(uri).await + } + + async fn write_text_if_match( + &self, + uri: &str, + contents: &str, + expected_version: &str, + ) -> Result<Option<String>> { + self.inner + .write_text_if_match(uri, contents, expected_version) + .await + } + + async fn delete_prefix(&self, prefix_uri: &str) -> Result<()> { + self.inner.delete_prefix(prefix_uri).await + } } #[derive(Debug)] @@ -2071,6 +2090,25 @@ edge WorksAt: Person -> Company async fn list_dir(&self, dir_uri: &str) -> Result<Vec<String>> { self.inner.list_dir(dir_uri).await } + + async fn read_text_versioned(&self, uri: &str) -> Result<(String, String)> { + self.inner.read_text_versioned(uri).await + } + + async fn write_text_if_match( + &self, + uri: &str, + contents: &str, + expected_version: &str, + ) -> Result<Option<String>> { + self.inner + .write_text_if_match(uri, contents, expected_version) + .await + } + + async fn delete_prefix(&self, prefix_uri: &str) -> Result<()> { + self.inner.delete_prefix(prefix_uri).await + } } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] diff --git a/crates/omnigraph/src/storage.rs b/crates/omnigraph/src/storage.rs index 564b577..978d1ce 100644 --- a/crates/omnigraph/src/storage.rs +++ b/crates/omnigraph/src/storage.rs @@ -39,6 +39,39 @@ pub trait StorageAdapter: Debug + Send + Sync { /// Returns full URIs (same scheme as `dir_uri`). The result is unordered. /// Returns Ok(empty) if the directory does not exist or is empty. async fn list_dir(&self, dir_uri: &str) -> Result<Vec<String>>; + /// Read a text object together with its backend version token (S3: the + /// object's ETag; local: sha256 of the content). The token is opaque — + /// valid only for `write_text_if_match` against the same adapter. + async fn read_text_versioned(&self, uri: &str) -> Result<(String, String)>; + /// Replace the object at `uri` only if its current version still matches + /// `expected_version` (obtained from a prior versioned read/write on this + /// adapter). Returns `Ok(Some(new_version))` on success and `Ok(None)` + /// when the precondition failed (a concurrent writer won — the CAS-lost + /// case callers must surface, never swallow). S3 uses a conditional put + /// (If-Match); local compares content then replaces via temp + rename — + /// the same single-machine semantics the callers had before this trait, + /// safe under the callers' own lock protocol but not a cross-process + /// barrier by itself. + async fn write_text_if_match( + &self, + uri: &str, + contents: &str, + expected_version: &str, + ) -> Result<Option<String>>; + /// Recursively delete every object under `prefix_uri`. Returns Ok(()) + /// when nothing exists there (idempotent). Local: `remove_dir_all`; + /// S3: list + delete (NOT atomic — callers must tolerate partial + /// prefixes on crash, which the cluster delete protocol does by retry). + async fn delete_prefix(&self, prefix_uri: &str) -> Result<()>; +} + +/// Version token for local files: content identity. ETags are unavailable +/// on the filesystem; sha256 is stable, cheap at these object sizes, and +/// already the cluster ledger's CAS vocabulary. +fn local_version_token(bytes: &[u8]) -> String { + use sha2::{Digest, Sha256}; + let digest = Sha256::digest(bytes); + digest.iter().map(|byte| format!("{byte:02x}")).collect() } #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -149,6 +182,49 @@ impl StorageAdapter for LocalStorageAdapter { } Ok(out) } + + async fn read_text_versioned(&self, uri: &str) -> Result<(String, String)> { + let path = local_path_from_uri(uri)?; + let bytes = tokio::fs::read(&path).await?; + let version = local_version_token(&bytes); + let text = String::from_utf8(bytes).map_err(|err| { + OmniError::manifest_internal(format!("storage read failed for '{}': {}", uri, err)) + })?; + Ok((text, version)) + } + + async fn write_text_if_match( + &self, + uri: &str, + contents: &str, + expected_version: &str, + ) -> Result<Option<String>> { + let path = local_path_from_uri(uri)?; + let current = match tokio::fs::read(&path).await { + Ok(bytes) => bytes, + Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(err) => return Err(err.into()), + }; + if local_version_token(¤t) != expected_version { + return Ok(None); + } + let tmp = path.with_extension(format!("tmp.{}", ulid::Ulid::new())); + tokio::fs::write(&tmp, contents.as_bytes()).await?; + if let Err(err) = tokio::fs::rename(&tmp, &path).await { + let _ = tokio::fs::remove_file(&tmp).await; + return Err(err.into()); + } + Ok(Some(local_version_token(contents.as_bytes()))) + } + + async fn delete_prefix(&self, prefix_uri: &str) -> Result<()> { + let path = local_path_from_uri(prefix_uri)?; + match tokio::fs::remove_dir_all(&path).await { + Ok(()) => Ok(()), + Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()), + Err(err) => Err(err.into()), + } + } } #[async_trait] @@ -276,6 +352,84 @@ impl StorageAdapter for S3StorageAdapter { } Ok(out) } + + async fn read_text_versioned(&self, uri: &str) -> Result<(String, String)> { + let location = self.object_path(uri)?; + let result = self + .store + .get(&location) + .await + .map_err(|err| storage_backend_error("read", uri, err))?; + let etag = result.meta.e_tag.clone(); + let bytes = result + .bytes() + .await + .map_err(|err| storage_backend_error("read", uri, err))?; + // Every S3-compatible store we target returns ETags; fall back to a + // content token rather than failing if one ever omits it. + let version = etag.unwrap_or_else(|| local_version_token(&bytes)); + let text = String::from_utf8(bytes.to_vec()).map_err(|err| { + OmniError::manifest_internal(format!("storage read failed for '{}': {}", uri, err)) + })?; + Ok((text, version)) + } + + async fn write_text_if_match( + &self, + uri: &str, + contents: &str, + expected_version: &str, + ) -> Result<Option<String>> { + let location = self.object_path(uri)?; + let mode = PutMode::Update(object_store::UpdateVersion { + e_tag: Some(expected_version.to_string()), + version: None, + }); + match self + .store + .put_opts( + &location, + PutPayload::from(contents.as_bytes().to_vec()), + mode.into(), + ) + .await + { + Ok(result) => Ok(Some( + result + .e_tag + .unwrap_or_else(|| local_version_token(contents.as_bytes())), + )), + Err(object_store::Error::Precondition { .. }) + | Err(object_store::Error::NotFound { .. }) => Ok(None), + Err(err) => Err(storage_backend_error("write_if_match", uri, err)), + } + } + + async fn delete_prefix(&self, prefix_uri: &str) -> Result<()> { + let dir_with_slash = if prefix_uri.ends_with('/') { + prefix_uri.to_string() + } else { + format!("{}/", prefix_uri) + }; + let prefix_loc = self.object_path(&dir_with_slash)?; + let mut entries = self.store.list(Some(&prefix_loc)); + let mut locations = Vec::new(); + while let Some(meta) = entries + .try_next() + .await + .map_err(|err| storage_backend_error("delete_prefix", prefix_uri, err))? + { + locations.push(meta.location); + } + for location in locations { + match self.store.delete(&location).await { + Ok(()) => {} + Err(object_store::Error::NotFound { .. }) => {} + Err(err) => return Err(storage_backend_error("delete_prefix", prefix_uri, err)), + } + } + Ok(()) + } } impl S3StorageAdapter { @@ -444,6 +598,55 @@ fn env_var_truthy(key: &str) -> bool { #[cfg(test)] mod tests { + + #[tokio::test] + async fn local_versioned_cas_roundtrip() { + let dir = tempfile::tempdir().unwrap(); + let uri = format!("{}/state.json", dir.path().display()); + let adapter = LocalStorageAdapter; + adapter.write_text(&uri, "v1").await.unwrap(); + let (text, version) = adapter.read_text_versioned(&uri).await.unwrap(); + assert_eq!(text, "v1"); + + // Matching token replaces and returns the next token. + let next = adapter + .write_text_if_match(&uri, "v2", &version) + .await + .unwrap() + .expect("fresh token must win"); + assert_ne!(next, version); + // The stale token must lose (CAS-lost is Ok(None), never silent). + assert!( + adapter + .write_text_if_match(&uri, "v3", &version) + .await + .unwrap() + .is_none() + ); + let (text, _) = adapter.read_text_versioned(&uri).await.unwrap(); + assert_eq!(text, "v2"); + // Missing object: precondition can't hold. + let missing = format!("{}/absent.json", dir.path().display()); + assert!( + adapter + .write_text_if_match(&missing, "x", &version) + .await + .unwrap() + .is_none() + ); + } + + #[tokio::test] + async fn local_delete_prefix_is_recursive_and_idempotent() { + let dir = tempfile::tempdir().unwrap(); + let root = format!("{}/tree", dir.path().display()); + let adapter = LocalStorageAdapter; + adapter.write_text(&format!("{root}/a.txt"), "a").await.unwrap(); + adapter.write_text(&format!("{root}/sub/b.txt"), "b").await.unwrap(); + adapter.delete_prefix(&root).await.unwrap(); + assert!(!adapter.exists(&format!("{root}/a.txt")).await.unwrap()); + adapter.delete_prefix(&root).await.unwrap(); // absent -> Ok + } use super::*; #[test] diff --git a/crates/omnigraph/tests/s3_storage.rs b/crates/omnigraph/tests/s3_storage.rs index 7e4f0a3..3814600 100644 --- a/crates/omnigraph/tests/s3_storage.rs +++ b/crates/omnigraph/tests/s3_storage.rs @@ -167,3 +167,80 @@ async fn s3_public_load_uses_hidden_run_and_publishes() { .to_rust_json(); assert_eq!(loaded[0]["p.name"], "Loaded-Over-S3"); } + +/// The conditional-write contract the cluster ledger depends on (RFC-006): +/// versioned read -> If-Match replace -> stale token refused. Pins the +/// S3-compatible backend's behavior (RustFS in CI) — turns red if a backend +/// bump regresses conditional puts. +#[tokio::test(flavor = "multi_thread")] +async fn s3_adapter_conditional_writes_contract() { + let Some(uri) = s3_test_graph_uri("adapter-cas") else { + eprintln!("skipping s3 adapter cas test: OMNIGRAPH_S3_TEST_BUCKET is not set"); + return; + }; + use omnigraph::storage::storage_for_uri; + let adapter = storage_for_uri(&uri).unwrap(); + let object = format!("{uri}/cas-probe.json"); + + assert!(adapter.write_text_if_absent(&object, "v1").await.unwrap()); + assert!(!adapter.write_text_if_absent(&object, "v1b").await.unwrap()); + + let (text, version) = adapter.read_text_versioned(&object).await.unwrap(); + assert_eq!(text, "v1"); + let next = adapter + .write_text_if_match(&object, "v2", &version) + .await + .unwrap() + .expect("fresh etag must win"); + assert!( + adapter + .write_text_if_match(&object, "v3", &version) + .await + .unwrap() + .is_none(), + "stale etag must be refused" + ); + let again = adapter + .write_text_if_match(&object, "v3", &next) + .await + .unwrap(); + assert!(again.is_some()); + + // Prefix delete: recursive + idempotent. + adapter + .write_text(&format!("{uri}/tree/a.json"), "a") + .await + .unwrap(); + adapter + .write_text(&format!("{uri}/tree/sub/b.json"), "b") + .await + .unwrap(); + adapter.delete_prefix(&format!("{uri}/tree")).await.unwrap(); + assert!(!adapter.exists(&format!("{uri}/tree/a.json")).await.unwrap()); + adapter.delete_prefix(&format!("{uri}/tree")).await.unwrap(); + adapter.delete(&object).await.unwrap(); +} + +/// Schema apply against an S3 graph — the cluster's schema executor will +/// lean on this; previously untested upstream on object storage. +#[tokio::test(flavor = "multi_thread")] +async fn s3_schema_apply_migrates_live_graph() { + let Some(uri) = s3_test_graph_uri("schema-apply") else { + eprintln!("skipping s3 schema apply test: OMNIGRAPH_S3_TEST_BUCKET is not set"); + return; + }; + let mut db = Omnigraph::init(&uri, TEST_SCHEMA).await.unwrap(); + load_jsonl(&mut db, TEST_DATA, LoadMode::Overwrite) + .await + .unwrap(); + + let desired = format!("{TEST_SCHEMA}\nnode Note {{\n title: String @key\n}}\n"); + let result = db.apply_schema(&desired).await.unwrap(); + assert!(result.applied, "{result:?}"); + + let reopened = Omnigraph::open(&uri).await.unwrap(); + assert!( + reopened.schema_source().contains("Note"), + "live S3 schema must carry the migration" + ); +} From d702fd106ab80b29f162f9f8a4c0156849263398 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 05:09:45 +0300 Subject: [PATCH 100/165] feat(policy): from-source twins for the policy loaders MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PolicyConfig::from_source + PolicyEngine::load_graph_from_source / load_server_from_source — the path-based loaders delegate to them. Needed by callers whose policy bundles don't live on the local filesystem (the cluster catalog on object storage); kind-alignment validation stays loud through the new path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-policy/src/lib.rs | 60 +++++++++++++++++++++++++++++- 1 file changed, 58 insertions(+), 2 deletions(-) diff --git a/crates/omnigraph-policy/src/lib.rs b/crates/omnigraph-policy/src/lib.rs index cb59796..46b380a 100644 --- a/crates/omnigraph-policy/src/lib.rs +++ b/crates/omnigraph-policy/src/lib.rs @@ -277,7 +277,14 @@ pub struct PolicyEngine { impl PolicyConfig { pub fn load(path: &Path) -> Result<Self> { - let config: Self = serde_yaml::from_str(&fs::read_to_string(path)?)?; + Self::from_source(&fs::read_to_string(path)?) + } + + /// Parse + validate a policy from YAML source. The from-content twin of + /// `load` for callers whose policies don't live on the local filesystem + /// (e.g. a cluster catalog on object storage). + pub fn from_source(source: &str) -> Result<Self> { + let config: Self = serde_yaml::from_str(source)?; config.validate()?; Ok(config) } @@ -465,13 +472,26 @@ impl PolicyEngine { PolicyCompiler::compile(&config, graph_id) } + /// `load_graph` from YAML content instead of a file path — for policies + /// that live in a non-filesystem catalog (cluster object storage). + pub fn load_graph_from_source(source: &str, graph_id: &str) -> Result<Self> { + let config = PolicyConfig::from_source(source)?; + validate_kind_alignment(&config, PolicyEngineKind::Graph)?; + PolicyCompiler::compile(&config, graph_id) + } + /// Load a server-level policy file. Rejects rules whose actions /// are per-graph (e.g. `read`, `change`) — those belong in a /// per-graph policy file, not the server one. Takes no `graph_id`: /// server-scoped actions resolve against the singleton /// `Omnigraph::Server::"root"` entity, never a Graph. pub fn load_server(path: &Path) -> Result<Self> { - let config = PolicyConfig::load(path)?; + Self::load_server_from_source(&fs::read_to_string(path)?) + } + + /// `load_server` from YAML content instead of a file path. + pub fn load_server_from_source(source: &str) -> Result<Self> { + let config = PolicyConfig::from_source(source)?; validate_kind_alignment(&config, PolicyEngineKind::Server)?; // The Graph entity created by the compiler is never referenced // by a server-scoped rule, so the label below is purely a @@ -1002,6 +1022,42 @@ impl PolicyChecker for PolicyEngine { #[cfg(test)] mod tests { + + #[test] + fn from_source_twins_match_path_loaders() { + let yaml = r#" +version: 1 +groups: + readers: ["act-r"] +protected_branches: [main] +rules: + - id: r1 + allow: + actors: { group: readers } + actions: [read] + branch_scope: any +"#; + let config = PolicyConfig::from_source(yaml).unwrap(); + assert_eq!(config.version, 1); + let engine = PolicyEngine::load_graph_from_source(yaml, "g1").unwrap(); + drop(engine); + + let server_yaml = r#" +version: 1 +kind: server +groups: + admins: ["act-a"] +rules: + - id: s1 + allow: + actors: { group: admins } + actions: [graph_list] +"#; + PolicyEngine::load_server_from_source(server_yaml).unwrap(); + // Kind misalignment stays loud through the from-source path. + assert!(PolicyEngine::load_graph_from_source(server_yaml, "g1").is_err()); + assert!(PolicyEngine::load_server_from_source(yaml).is_err()); + } use super::{ PolicyAction, PolicyCompiler, PolicyConfig, PolicyEngine, PolicyExpectation, PolicyRequest, PolicyTestCase, PolicyTestConfig, From fbb86dee0ea1583d8dde675cef8ff3df95a02560 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 05:25:53 +0300 Subject: [PATCH 101/165] refactor(cluster): move the in-source test suite to tests.rs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verbatim move (indentation preserved — embedded raw-string fixtures are content). lib.rs drops from 7,857 to ~4,750 lines; `use super::*` resolves to the crate root through the #[path] module declaration unchanged. 95 tests green before and after. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 3019 +------------------------ crates/omnigraph-cluster/src/tests.rs | 3019 +++++++++++++++++++++++++ 2 files changed, 3022 insertions(+), 3016 deletions(-) create mode 100644 crates/omnigraph-cluster/src/tests.rs diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index bb0c66b..da80710 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -4838,3020 +4838,7 @@ fn display_path(path: &Path) -> String { path.display().to_string() } + #[cfg(test)] -mod tests { - use std::fs; - use std::path::Path; - - use omnigraph::db::Omnigraph; - use serde_json::json; - use tempfile::tempdir; - - use super::*; - - const SCHEMA: &str = r#" -node Person { - name: String @key - age: I32? -} -"#; - - const QUERY: &str = r#" -query find_person($name: String) { - match { $p: Person { name: $name } } - return { $p.name, $p.age } -} -"#; - - fn fixture() -> tempfile::TempDir { - let dir = tempdir().unwrap(); - fs::write(dir.path().join("people.pg"), SCHEMA).unwrap(); - fs::write(dir.path().join("people.gq"), QUERY).unwrap(); - fs::write(dir.path().join("base.policy.yaml"), "rules: []\n").unwrap(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - r#" -version: 1 -metadata: - name: test -state: - backend: cluster - lock: true -graphs: - knowledge: - schema: ./people.pg - queries: - find_person: - file: ./people.gq -policies: - base: - file: ./base.policy.yaml - applies_to: [knowledge] -"#, - ) - .unwrap(); - dir - } - - async fn init_derived_graph(root: &Path) { - let graph_dir = root.join(CLUSTER_GRAPHS_DIR); - fs::create_dir_all(&graph_dir).unwrap(); - let graph = graph_dir.join("knowledge.omni"); - Omnigraph::init(graph.to_string_lossy().as_ref(), SCHEMA) - .await - .unwrap(); - } - - fn write_lock_file(config_dir: &Path, lock_id: &str, operation: &str) { - let state_dir = config_dir.join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("lock.json"), - json!({ - "version": 1, - "lock_id": lock_id, - "operation": operation, - "created_at": "1970-01-01T00:00:00Z", - "pid": 123 - }) - .to_string(), - ) - .unwrap(); - } - - #[test] - fn valid_minimal_config() { - let dir = fixture(); - let out = validate_config_dir(dir.path()); - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.resource_digests.contains_key("graph.knowledge")); - assert!(out.resource_digests.contains_key("schema.knowledge")); - assert!( - out.dependencies - .iter() - .any(|dep| dep.from == "policy.base" && dep.to == "graph.knowledge") - ); - } - - #[test] - fn unknown_field_rejection() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - "version: 1\ngraphs: {}\nwat: true\n", - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(!out.ok); - assert!(out.diagnostics[0].message.contains("unknown field")); - } - - #[test] - fn future_phase_field_rejection() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - "version: 1\ngraphs: {}\npipelines: {}\n", - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(!out.ok); - assert_eq!(out.diagnostics[0].code, "future_phase_field"); - } - - #[test] - fn duplicate_yaml_key_rejection() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - "version: 1\ngraphs: {}\ngraphs: {}\n", - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(!out.ok); - assert_eq!(out.diagnostics[0].code, "duplicate_yaml_key"); - } - - #[test] - fn duplicate_yaml_key_rejection_keeps_quoted_hashes() { - let diagnostics = - duplicate_key_diagnostics("\"name#display\": one\n\"name#display\": two\n"); - assert_eq!(diagnostics.len(), 1); - assert_eq!(diagnostics[0].code, "duplicate_yaml_key"); - } - - #[test] - fn missing_schema_query_and_policy_files() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - r#" -version: 1 -graphs: - knowledge: - schema: ./missing.pg - queries: - find_person: { file: ./missing.gq } -policies: - base: - file: ./missing.policy.yaml - applies_to: [knowledge] -"#, - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(!out.ok); - let codes: BTreeSet<_> = out.diagnostics.iter().map(|d| d.code.as_str()).collect(); - assert!(codes.contains("schema_file_missing")); - assert!(codes.contains("query_file_missing")); - assert!(codes.contains("policy_file_missing")); - } - - #[test] - fn wrong_kind_and_dangling_refs_fail() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - r#" -version: 1 -graphs: - knowledge: - schema: ./people.pg -policies: - base: - file: ./base.policy.yaml - applies_to: [query.knowledge.find_person, missing] -"#, - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(!out.ok); - let codes: BTreeSet<_> = out.diagnostics.iter().map(|d| d.code.as_str()).collect(); - assert!(codes.contains("wrong_kind_reference")); - assert!(codes.contains("dangling_graph_reference")); - } - - #[test] - fn query_key_mismatch_fails() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - r#" -version: 1 -graphs: - knowledge: - schema: ./people.pg - queries: - different: { file: ./people.gq } -"#, - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(!out.ok); - assert_eq!(out.diagnostics[0].code, "query_key_mismatch"); - } - - #[test] - fn query_typecheck_failure_fails() { - let dir = fixture(); - fs::write( - dir.path().join("people.gq"), - "query find_person() { match { $d: DoesNotExist } return { $d.name } }\n", - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "query_typecheck_error") - ); - } - - #[tokio::test] - async fn missing_state_plans_creates() { - let dir = fixture(); - let out = plan_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!(!out.state_observations.state_found); - assert!(!out.state_observations.locked); - assert!(out.state_observations.lock_acquired); - assert!( - out.changes - .iter() - .all(|c| c.operation == PlanOperation::Create) - ); - assert!(out.changes.iter().any(|c| c.resource == "graph.knowledge")); - assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); - } - - #[tokio::test] - async fn config_digest_ignores_yaml_comments_and_formatting() { - let dir = fixture(); - let first = plan_config_dir(dir.path()).await; - assert!(first.ok, "{:?}", first.diagnostics); - - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - r#" -# Same semantic config as the fixture, intentionally rendered differently. -version: 1 -metadata: { name: test } -state: { backend: cluster, lock: true } -graphs: - knowledge: - schema: ./people.pg - queries: { find_person: { file: ./people.gq } } -policies: - base: - file: ./base.policy.yaml - applies_to: - - knowledge -"#, - ) - .unwrap(); - - let second = plan_config_dir(dir.path()).await; - assert!(second.ok, "{:?}", second.diagnostics); - assert_eq!( - first.desired_revision.config_digest, - second.desired_revision.config_digest - ); - } - - #[tokio::test] - async fn existing_state_plans_update_and_delete_deterministically() { - let dir = fixture(); - let first = plan_config_dir(dir.path()).await; - let state_dir = dir.path().join("__cluster"); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - serde_json::to_string_pretty(&json!({ - "version": 1, - "applied_revision": { - "config_digest": "old", - "resources": { - "graph.knowledge": { "digest": first.resource_digests["graph.knowledge"] }, - "policy.old": { "digest": "abc" }, - "schema.knowledge": { "digest": "old-schema" } - } - } - })) - .unwrap(), - ) - .unwrap(); - - let out = plan_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - let rendered: Vec<_> = out - .changes - .iter() - .map(|change| (change.resource.as_str(), &change.operation)) - .collect(); - assert_eq!( - rendered, - vec![ - ("policy.base", &PlanOperation::Create), - ("policy.old", &PlanOperation::Delete), - ("query.knowledge.find_person", &PlanOperation::Create), - ("schema.knowledge", &PlanOperation::Update), - ] - ); - } - - #[tokio::test] - async fn old_minimal_state_json_still_plans_with_default_revision() { - let dir = fixture(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#"{ - "version": 1, - "applied_revision": { - "config_digest": "old", - "resources": { - "graph.knowledge": { "digest": "old-graph" } - } - } -}"#, - ) - .unwrap(); - - let out = plan_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert_eq!(out.state_observations.state_revision, 0); - assert!(out.state_observations.state_cas.is_some()); - assert!(out.changes.iter().any(|change| { - change.resource == "graph.knowledge" && change.operation == PlanOperation::Update - })); - } - - #[test] - fn extended_state_json_status_surfaces_statuses() { - let dir = fixture(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - let state = r#"{ - "version": 1, - "state_revision": 42, - "applied_revision": { - "config_digest": "applied-config", - "resources": { - "graph.knowledge": { "digest": "graph-digest" } - } - }, - "resource_statuses": { - "graph.knowledge": { - "status": "applied", - "conditions": ["healthy"], - "message": "ready" - } - }, - "approval_records": {}, - "recovery_records": {}, - "observations": { - "graph.knowledge": { "manifest_version": 12 } - } -}"#; - fs::write(state_dir.join("state.json"), state).unwrap(); - - let out = status_config_dir(dir.path()); - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.state_observations.state_found); - assert_eq!(out.state_observations.state_revision, 42); - assert_eq!( - out.state_observations.state_cas.as_deref(), - Some(format!("sha256:{}", sha256_hex(state.as_bytes())).as_str()) - ); - assert_eq!( - out.resource_digests - .get("graph.knowledge") - .map(String::as_str), - Some("graph-digest") - ); - assert_eq!( - out.resource_statuses["graph.knowledge"].status, - ResourceLifecycleStatus::Applied - ); - } - - #[test] - fn missing_state_status_succeeds_with_warning() { - let dir = fixture(); - let out = status_config_dir(dir.path()); - assert!(out.ok, "{:?}", out.diagnostics); - assert!(!out.state_observations.state_found); - assert_eq!(out.state_observations.state_revision, 0); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_missing") - ); - } - - #[test] - fn invalid_state_status_fails() { - let dir = fixture(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write(state_dir.join("state.json"), "{").unwrap(); - - let out = status_config_dir(dir.path()); - assert!(!out.ok); - assert!(out.state_observations.state_found); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "invalid_state_json") - ); - } - - #[test] - fn status_surfaces_full_lock_metadata() { - let dir = fixture(); - write_lock_file(dir.path(), "held-lock", "refresh"); - - let out = status_config_dir(dir.path()); - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.state_observations.locked); - assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); - assert_eq!( - out.state_observations.lock_operation.as_deref(), - Some("refresh") - ); - assert_eq!( - out.state_observations.lock_created_at.as_deref(), - Some("1970-01-01T00:00:00Z") - ); - assert_eq!(out.state_observations.lock_pid, Some(123)); - assert!(out.state_observations.lock_age_seconds.is_some()); - } - - #[test] - fn force_unlock_matching_id_removes_lock() { - let dir = fixture(); - write_lock_file(dir.path(), "held-lock", "plan"); - - let out = force_unlock_config_dir(dir.path(), "held-lock"); - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.lock_removed); - assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); - assert_eq!( - out.state_observations.lock_operation.as_deref(), - Some("plan") - ); - assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); - } - - #[test] - fn force_unlock_wrong_id_fails_and_preserves_lock() { - let dir = fixture(); - write_lock_file(dir.path(), "held-lock", "plan"); - - let out = force_unlock_config_dir(dir.path(), "other-lock"); - assert!(!out.ok); - assert!(!out.lock_removed); - assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_lock_id_mismatch") - ); - assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); - } - - #[test] - fn force_unlock_missing_lock_fails() { - let dir = fixture(); - - let out = force_unlock_config_dir(dir.path(), "held-lock"); - assert!(!out.ok); - assert!(!out.lock_removed); - assert!(!out.state_observations.locked); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_lock_missing") - ); - } - - #[test] - fn force_unlock_invalid_lock_json_fails_and_preserves_lock() { - let dir = fixture(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write(state_dir.join("lock.json"), "{").unwrap(); - - let out = force_unlock_config_dir(dir.path(), "held-lock"); - assert!(!out.ok); - assert!(!out.lock_removed); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "invalid_state_lock") - ); - assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); - } - - #[test] - fn force_unlock_unsupported_lock_version_fails_and_preserves_lock() { - let dir = fixture(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("lock.json"), - r#"{"version":2,"lock_id":"held-lock","operation":"plan","created_at":"1970-01-01T00:00:00Z","pid":123}"#, - ) - .unwrap(); - - let out = force_unlock_config_dir(dir.path(), "held-lock"); - assert!(!out.ok); - assert!(!out.lock_removed); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "unsupported_state_lock_version") - ); - assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); - } - - #[test] - fn force_unlock_external_state_backend_rejected() { - let dir = fixture(); - write_lock_file(dir.path(), "held-lock", "plan"); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - r#" -version: 1 -state: - backend: s3://state-bucket/cluster -graphs: - knowledge: - schema: ./people.pg -"#, - ) - .unwrap(); - - let out = force_unlock_config_dir(dir.path(), "held-lock"); - assert!(!out.ok); - assert!(!out.lock_removed); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "unsupported_state_backend") - ); - assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); - } - - #[tokio::test] - async fn plan_succeeds_after_force_unlock() { - let dir = fixture(); - write_lock_file(dir.path(), "held-lock", "plan"); - - let locked = plan_config_dir(dir.path()).await; - assert!(!locked.ok); - assert!( - locked - .diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_lock_held") - ); - - let unlocked = force_unlock_config_dir(dir.path(), "held-lock"); - assert!(unlocked.ok, "{:?}", unlocked.diagnostics); - - let out = plan_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - } - - #[tokio::test] - async fn plan_reports_state_cas_revision_and_removes_lock() { - let dir = fixture(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - let state = r#"{ - "version": 1, - "state_revision": 7, - "applied_revision": { - "config_digest": "old", - "resources": { - "graph.knowledge": { "digest": "old-graph" } - } - } -}"#; - fs::write(state_dir.join("state.json"), state).unwrap(); - - let out = plan_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert_eq!(out.state_observations.state_revision, 7); - assert_eq!( - out.state_observations.state_cas.as_deref(), - Some(format!("sha256:{}", sha256_hex(state.as_bytes())).as_str()) - ); - assert!(!out.state_observations.locked); - assert!(out.state_observations.lock_id.is_none()); - assert!(out.state_observations.lock_acquired); - assert!(out.state_observations.acquired_lock_id.is_some()); - assert!( - !dir.path().join(CLUSTER_LOCK_FILE).exists(), - "plan must release lock before returning" - ); - } - - #[tokio::test] - async fn existing_lock_makes_plan_fail() { - let dir = fixture(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("lock.json"), - r#"{ - "version": 1, - "lock_id": "held-lock", - "operation": "plan", - "created_at": "2026-06-08T00:00:00Z", - "pid": 123 -}"#, - ) - .unwrap(); - - let out = plan_config_dir(dir.path()).await; - assert!(!out.ok); - assert!(out.state_observations.locked); - assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); - assert!(!out.state_observations.lock_acquired); - assert!(out.state_observations.acquired_lock_id.is_none()); - assert_eq!( - out.state_observations.lock_operation.as_deref(), - Some("plan") - ); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_lock_held") - ); - assert!(out.diagnostics.iter().any(|diagnostic| { - diagnostic.code == "state_lock_held" - && diagnostic.message.contains("force-unlock held-lock") - })); - } - - #[tokio::test] - async fn state_lock_false_bypasses_lock_with_warning() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - r#" -version: 1 -state: - backend: cluster - lock: false -graphs: - knowledge: - schema: ./people.pg -"#, - ) - .unwrap(); - - let out = plan_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!(!out.state_observations.locked); - assert!(!out.state_observations.lock_acquired); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_lock_disabled") - ); - assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); - } - - #[test] - fn external_state_backend_rejected() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - "version: 1\nstate:\n backend: s3://bucket/state\ngraphs: {}\n", - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(!out.ok); - assert_eq!(out.diagnostics[0].code, "unsupported_state_backend"); - } - - #[tokio::test] - async fn external_state_backend_plan_rejected() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - "version: 1\nstate:\n backend: s3://bucket/state\ngraphs: {}\n", - ) - .unwrap(); - let out = plan_config_dir(dir.path()).await; - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "unsupported_state_backend") - ); - } - - #[tokio::test] - async fn import_missing_state_creates_state_with_graph_observation() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - - let out = import_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert_eq!(out.state_observations.state_revision, 1); - assert!(out.state_observations.state_cas.is_some()); - assert!(!out.state_observations.locked); - assert!(out.state_observations.lock_acquired); - assert!(out.state_observations.acquired_lock_id.is_some()); - assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); - assert_eq!( - out.resource_digests - .get("schema.knowledge") - .map(String::as_str), - Some(sha256_hex(SCHEMA.as_bytes()).as_str()) - ); - assert!(out.observations["graph.knowledge"]["manifest_version"].is_number()); - assert_eq!( - out.observations["graph.knowledge"]["schema_matches_desired"], - true - ); - - let state: serde_json::Value = - serde_json::from_str(&fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap()) - .unwrap(); - assert_eq!(state["state_revision"], 1); - assert_eq!( - state["resource_statuses"]["graph.knowledge"]["status"], - "applied" - ); - } - - #[tokio::test] - async fn import_existing_state_fails() { - let dir = fixture(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#"{"version":1,"applied_revision":{"resources":{}}}"#, - ) - .unwrap(); - - let out = import_config_dir(dir.path()).await; - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_already_exists") - ); - } - - #[tokio::test] - async fn refresh_missing_state_fails() { - let dir = fixture(); - let out = refresh_config_dir(dir.path()).await; - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_missing") - ); - } - - #[tokio::test] - async fn refresh_existing_minimal_state_increments_revision_and_updates_cas() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#"{"version":1,"applied_revision":{"config_digest":"old","resources":{"graph.knowledge":{"digest":"old"}}}}"#, - ) - .unwrap(); - - let out = refresh_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert_eq!(out.state_observations.state_revision, 1); - assert!(out.state_observations.state_cas.is_some()); - assert!(!out.state_observations.locked); - assert!(out.state_observations.lock_acquired); - assert_eq!( - out.resource_statuses["graph.knowledge"].status, - ResourceLifecycleStatus::Applied - ); - assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); - } - - #[tokio::test] - async fn refresh_records_live_schema_digest_and_manifest_version() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#"{"version":1,"state_revision":4,"applied_revision":{"resources":{}}}"#, - ) - .unwrap(); - - let out = refresh_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert_eq!(out.state_observations.state_revision, 5); - assert_eq!( - out.observations["graph.knowledge"]["schema_digest"], - sha256_hex(SCHEMA.as_bytes()) - ); - assert!(out.observations["graph.knowledge"]["manifest_version"].is_u64()); - } - - #[tokio::test] - async fn missing_derived_graph_root_marks_drifted_and_plans_creates() { - let dir = fixture(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#"{"version":1,"applied_revision":{"resources":{"graph.knowledge":{"digest":"old-graph"},"schema.knowledge":{"digest":"old-schema"}}}}"#, - ) - .unwrap(); - - let out = refresh_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert_eq!( - out.resource_statuses["graph.knowledge"].status, - ResourceLifecycleStatus::Drifted - ); - assert!(!out.resource_digests.contains_key("graph.knowledge")); - assert_eq!(out.observations["graph.knowledge"]["exists"], false); - - let plan = plan_config_dir(dir.path()).await; - assert!(plan.ok, "{:?}", plan.diagnostics); - assert!(plan.changes.iter().any(|change| { - change.resource == "graph.knowledge" && change.operation == PlanOperation::Create - })); - assert!(plan.changes.iter().any(|change| { - change.resource == "schema.knowledge" && change.operation == PlanOperation::Create - })); - } - - #[tokio::test] - async fn live_schema_mismatch_marks_drifted_and_causes_plan_update() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - fs::write( - dir.path().join("people.pg"), - SCHEMA.replace("age: I32?", "age: I32?\n nickname: String?"), - ) - .unwrap(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#"{"version":1,"applied_revision":{"resources":{"graph.knowledge":{"digest":"old-graph"},"schema.knowledge":{"digest":"old-schema"}}}}"#, - ) - .unwrap(); - - let out = refresh_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert_eq!( - out.resource_statuses["schema.knowledge"].status, - ResourceLifecycleStatus::Drifted - ); - assert_eq!( - out.observations["graph.knowledge"]["schema_matches_desired"], - false - ); - - let plan = plan_config_dir(dir.path()).await; - assert!(plan.ok, "{:?}", plan.diagnostics); - assert!(plan.changes.iter().any(|change| { - change.resource == "schema.knowledge" && change.operation == PlanOperation::Update - })); - } - - #[tokio::test] - async fn existing_lock_makes_refresh_fail() { - let dir = fixture(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#"{"version":1,"applied_revision":{"resources":{}}}"#, - ) - .unwrap(); - fs::write( - state_dir.join("lock.json"), - r#"{"version":1,"lock_id":"held-lock","operation":"refresh","created_at":"2026-06-08T00:00:00Z","pid":123}"#, - ) - .unwrap(); - - let out = refresh_config_dir(dir.path()).await; - assert!(!out.ok); - assert!(out.state_observations.locked); - assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); - assert!(!out.state_observations.lock_acquired); - assert_eq!( - out.state_observations.lock_operation.as_deref(), - Some("refresh") - ); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_lock_held") - ); - assert!(out.diagnostics.iter().any(|diagnostic| { - diagnostic.code == "state_lock_held" - && diagnostic.message.contains("force-unlock held-lock") - })); - } - - #[tokio::test] - async fn state_lock_false_bypasses_refresh_lock_with_warning() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - r#" -version: 1 -state: - backend: cluster - lock: false -graphs: - knowledge: - schema: ./people.pg -"#, - ) - .unwrap(); - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#"{"version":1,"applied_revision":{"resources":{}}}"#, - ) - .unwrap(); - - let out = refresh_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!(!out.state_observations.locked); - assert!(!out.state_observations.lock_acquired); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_lock_disabled") - ); - } - - #[tokio::test] - async fn external_state_backend_refresh_rejected() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - "version: 1\nstate:\n backend: s3://bucket/state\ngraphs: {}\n", - ) - .unwrap(); - - let out = refresh_config_dir(dir.path()).await; - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "unsupported_state_backend") - ); - } - - #[tokio::test] - async fn import_graph_open_error_does_not_create_state() { - let dir = fixture(); - fs::create_dir_all(dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni")).unwrap(); - - let out = import_config_dir(dir.path()).await; - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "graph_observation_error") - ); - assert!(!dir.path().join(CLUSTER_STATE_FILE).exists()); - } - - // ---- config-only apply (Stage 3A) ---- - - /// Seed a state.json that simulates "graph exists with the desired schema, - /// queries/policies not yet applied" by borrowing the desired digests. - fn write_applyable_state(config_dir: &Path) { - let out = validate_config_dir(config_dir); - assert!(out.ok, "{:?}", out.diagnostics); - let schema_digest = out.resource_digests.get("schema.knowledge").unwrap().clone(); - let graph_composite = - graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); - write_state_resources( - config_dir, - &[ - ("graph.knowledge", graph_composite.as_str()), - ("schema.knowledge", schema_digest.as_str()), - ], - ); - } - - fn write_state_resources(config_dir: &Path, resources: &[(&str, &str)]) { - let resource_map: serde_json::Map<String, serde_json::Value> = resources - .iter() - .map(|(address, digest)| ((*address).to_string(), json!({ "digest": digest }))) - .collect(); - let state_dir = config_dir.join(CLUSTER_STATE_DIR); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - serde_json::to_string_pretty(&json!({ - "version": 1, - "state_revision": 1, - "applied_revision": { "resources": resource_map } - })) - .unwrap(), - ) - .unwrap(); - } - - fn read_state_json(config_dir: &Path) -> serde_json::Value { - serde_json::from_str(&fs::read_to_string(config_dir.join(CLUSTER_STATE_FILE)).unwrap()) - .unwrap() - } - - fn query_payload_path(config_dir: &Path, digest: &str) -> std::path::PathBuf { - config_dir - .join(CLUSTER_RESOURCES_DIR) - .join("query/knowledge/find_person") - .join(format!("{digest}.gq")) - } - - fn policy_payload_path(config_dir: &Path, digest: &str) -> std::path::PathBuf { - config_dir - .join(CLUSTER_RESOURCES_DIR) - .join("policy/base") - .join(format!("{digest}.yaml")) - } - - #[tokio::test] - async fn apply_without_state_fails_with_state_missing() { - let dir = fixture(); - let out = apply_config_dir(dir.path()).await; - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_missing" - && diagnostic.message.contains("cluster import")) - ); - assert!(!dir.path().join(CLUSTER_STATE_FILE).exists()); - assert!(!dir.path().join(CLUSTER_RESOURCES_DIR).exists()); - assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); - } - - #[tokio::test] - async fn apply_writes_payloads_state_and_statuses() { - let dir = fixture(); - write_applyable_state(dir.path()); - let desired = validate_config_dir(dir.path()); - let query_digest = desired - .resource_digests - .get("query.knowledge.find_person") - .unwrap() - .clone(); - let policy_digest = desired.resource_digests.get("policy.base").unwrap().clone(); - let schema_digest = desired - .resource_digests - .get("schema.knowledge") - .unwrap() - .clone(); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert_eq!(out.applied_count, 2); - assert_eq!(out.deferred_count, 0); - assert!(out.converged); - assert!(out.state_written); - - let query_blob = query_payload_path(dir.path(), &query_digest); - assert_eq!(fs::read_to_string(&query_blob).unwrap(), QUERY); - let policy_blob = policy_payload_path(dir.path(), &policy_digest); - assert_eq!(fs::read_to_string(&policy_blob).unwrap(), "rules: []\n"); - - let state = read_state_json(dir.path()); - assert_eq!(state["state_revision"], 2); - let resources = &state["applied_revision"]["resources"]; - assert_eq!( - resources["query.knowledge.find_person"]["digest"], - query_digest - ); - assert_eq!(resources["policy.base"]["digest"], policy_digest); - let expected_composite = graph_digest( - "knowledge", - Some(&schema_digest), - Some( - &[("find_person".to_string(), query_digest.clone())] - .into_iter() - .collect(), - ), - ); - assert_eq!(resources["graph.knowledge"]["digest"], expected_composite); - assert_eq!( - state["applied_revision"]["config_digest"], - desired_revision_digest(&out) - ); - assert_eq!( - state["resource_statuses"]["query.knowledge.find_person"]["status"], - "applied" - ); - assert_eq!(state["resource_statuses"]["policy.base"]["status"], "applied"); - assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); - } - - fn desired_revision_digest(out: &ApplyOutput) -> String { - out.desired_revision.config_digest.clone().unwrap() - } - - #[tokio::test] - async fn apply_update_changes_query_digest_and_keeps_old_blob() { - let dir = fixture(); - let desired = validate_config_dir(dir.path()); - let schema_digest = desired - .resource_digests - .get("schema.knowledge") - .unwrap() - .clone(); - let old_digest = "0".repeat(64); - let graph_composite = - graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); - write_state_resources( - dir.path(), - &[ - ("graph.knowledge", graph_composite.as_str()), - ("schema.knowledge", schema_digest.as_str()), - ("query.knowledge.find_person", old_digest.as_str()), - ], - ); - let old_blob = query_payload_path(dir.path(), &old_digest); - fs::create_dir_all(old_blob.parent().unwrap()).unwrap(); - fs::write(&old_blob, "old query source").unwrap(); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - let new_digest = desired - .resource_digests - .get("query.knowledge.find_person") - .unwrap(); - let state = read_state_json(dir.path()); - assert_eq!( - state["applied_revision"]["resources"]["query.knowledge.find_person"]["digest"], - *new_digest - ); - assert_eq!(fs::read_to_string(&old_blob).unwrap(), "old query source"); - assert!(query_payload_path(dir.path(), new_digest).exists()); - } - - #[tokio::test] - async fn apply_deletes_removed_resources_but_keeps_blobs() { - let dir = fixture(); - let desired = validate_config_dir(dir.path()); - let schema_digest = desired - .resource_digests - .get("schema.knowledge") - .unwrap() - .clone(); - let stale_query_digest = "1".repeat(64); - let stale_policy_digest = "2".repeat(64); - let graph_composite = - graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); - write_state_resources( - dir.path(), - &[ - ("graph.knowledge", graph_composite.as_str()), - ("schema.knowledge", schema_digest.as_str()), - ("query.knowledge.orphan", stale_query_digest.as_str()), - ("policy.old", stale_policy_digest.as_str()), - ], - ); - let stale_blob = dir - .path() - .join(CLUSTER_RESOURCES_DIR) - .join("policy/old") - .join(format!("{stale_policy_digest}.yaml")); - fs::create_dir_all(stale_blob.parent().unwrap()).unwrap(); - fs::write(&stale_blob, "old policy").unwrap(); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.converged); - let state = read_state_json(dir.path()); - let resources = &state["applied_revision"]["resources"]; - assert!(resources.get("query.knowledge.orphan").is_none()); - assert!(resources.get("policy.old").is_none()); - assert!( - state["resource_statuses"] - .get("query.knowledge.orphan") - .is_none() - ); - // Deleted resources leave their content-addressed blobs in place; GC is - // a later stage. - assert_eq!(fs::read_to_string(&stale_blob).unwrap(), "old policy"); - // The composite no longer includes the orphan query. - let query_digest = desired - .resource_digests - .get("query.knowledge.find_person") - .unwrap() - .clone(); - let expected_composite = graph_digest( - "knowledge", - Some(&schema_digest), - Some(&[("find_person".to_string(), query_digest)].into_iter().collect()), - ); - assert_eq!(resources["graph.knowledge"]["digest"], expected_composite); - } - - #[tokio::test] - async fn apply_schema_update_and_dependent_query_in_one_run() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_applyable_state(dir.path()); - // Schema update + a query update that depends on the new field: one - // apply executes the schema migration first, then the catalog write. - fs::write(dir.path().join("people.pg"), SCHEMA_V2).unwrap(); - fs::write( - dir.path().join("people.gq"), - "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name, $p.bio }\n}\n", - ) - .unwrap(); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.converged, "{out:?}"); - let by_resource: BTreeMap<&str, &PlanChange> = out - .changes - .iter() - .map(|change| (change.resource.as_str(), change)) - .collect(); - assert_eq!( - by_resource["schema.knowledge"].disposition, - Some(ApplyDisposition::Applied) - ); - assert_eq!( - by_resource["query.knowledge.find_person"].disposition, - Some(ApplyDisposition::Applied) - ); - assert_eq!( - by_resource["graph.knowledge"].disposition, - Some(ApplyDisposition::Derived) - ); - // The live graph carries the new schema. - let db = Omnigraph::open_read_only(&derived_graph_uri(dir.path(), "knowledge")) - .await - .unwrap(); - let desired = validate_config_dir(dir.path()); - assert_eq!( - sha256_hex(db.schema_source().as_bytes()), - desired.resource_digests["schema.knowledge"] - ); - let state = read_state_json(dir.path()); - assert_eq!( - state["applied_revision"]["resources"]["schema.knowledge"]["digest"], - desired.resource_digests["schema.knowledge"] - ); - // Sidecar retired after the CAS landed. - assert!( - !dir.path().join(CLUSTER_RECOVERIES_DIR).exists() - || fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) - .unwrap() - .next() - .is_none() - ); - } - - #[tokio::test] - async fn apply_unsupported_schema_change_fails_loudly() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_applyable_state(dir.path()); - // Property type changes are unsupported by the engine planner. - fs::write( - dir.path().join("people.pg"), - "\nnode Person {\n name: String @key\n age: I64?\n}\n", - ) - .unwrap(); - - let out = apply_config_dir(dir.path()).await; - assert!(!out.ok); - assert!(out.diagnostics.iter().any(|diagnostic| { - diagnostic.code == "schema_apply_failed" - && diagnostic.message.contains("changing property type") - })); - let by_resource: BTreeMap<&str, &PlanChange> = out - .changes - .iter() - .map(|change| (change.resource.as_str(), change)) - .collect(); - assert_eq!( - by_resource["schema.knowledge"].disposition, - Some(ApplyDisposition::Blocked) - ); - assert_eq!( - by_resource["schema.knowledge"].reason.as_deref(), - Some("schema_apply_failed") - ); - // The live schema and the ledger are unchanged. - let state = read_state_json(dir.path()); - let desired = validate_config_dir(dir.path()); - assert_ne!( - state["applied_revision"]["resources"]["schema.knowledge"]["digest"], - desired.resource_digests["schema.knowledge"] - ); - // Second run: the sweep retires the stale sidecar (ledger consistent) - // and the run fails just as loudly — idempotent loudness. - let second = apply_config_dir(dir.path()).await; - assert!(!second.ok); - assert!( - second - .diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "schema_apply_failed") - ); - } - - #[tokio::test] - async fn apply_blocks_schema_update_while_recovery_pending() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_state_resources(dir.path(), &[("schema.knowledge", "stale-digest")]); - fs::write(dir.path().join("people.pg"), SCHEMA_V2).unwrap(); - // A pending sidecar whose intent matches neither live nor recorded. - write_schema_apply_sidecar(dir.path(), "knowledge", "intended-digest", "01PENDS"); - - let out = apply_config_dir(dir.path()).await; - let by_resource: BTreeMap<&str, &PlanChange> = out - .changes - .iter() - .map(|change| (change.resource.as_str(), change)) - .collect(); - assert_eq!( - by_resource["schema.knowledge"].disposition, - Some(ApplyDisposition::Blocked) - ); - assert_eq!( - by_resource["schema.knowledge"].reason.as_deref(), - Some("cluster_recovery_pending") - ); - } - - #[tokio::test] - async fn apply_creates_graph_and_unblocks_dependents() { - let dir = fixture(); - write_state_resources(dir.path(), &[]); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.converged, "{out:?}"); - let by_resource: BTreeMap<&str, &PlanChange> = out - .changes - .iter() - .map(|change| (change.resource.as_str(), change)) - .collect(); - // Stage 4A: the create executes, and its dependents apply in-run. - assert_eq!( - by_resource["graph.knowledge"].disposition, - Some(ApplyDisposition::Applied) - ); - assert_eq!( - by_resource["schema.knowledge"].disposition, - Some(ApplyDisposition::Applied) - ); - assert_eq!( - by_resource["query.knowledge.find_person"].disposition, - Some(ApplyDisposition::Applied) - ); - assert_eq!( - by_resource["policy.base"].disposition, - Some(ApplyDisposition::Applied) - ); - // The graph exists on disk and opens; state records everything. - let graph_uri = derived_graph_uri(dir.path(), "knowledge"); - let db = Omnigraph::open_read_only(&graph_uri).await.unwrap(); - let desired = validate_config_dir(dir.path()); - assert_eq!( - sha256_hex(db.schema_source().as_bytes()), - desired.resource_digests["schema.knowledge"] - ); - let state = read_state_json(dir.path()); - assert_eq!( - state["applied_revision"]["resources"]["schema.knowledge"]["digest"], - desired.resource_digests["schema.knowledge"] - ); - assert_eq!( - state["resource_statuses"]["graph.knowledge"]["status"], - "applied" - ); - // The create's sidecar was retired after the state CAS landed. - assert!( - !dir.path().join(CLUSTER_RECOVERIES_DIR).exists() - || fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) - .unwrap() - .next() - .is_none() - ); - } - - #[tokio::test] - async fn apply_create_failure_blocks_dependents_and_keeps_sidecar() { - let dir = fixture(); - write_state_resources(dir.path(), &[]); - // Make the init fail its strict preflight: a junk _schema.pg already - // sits at the derived root (the engine refuses to overwrite it). - let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); - fs::create_dir_all(&root).unwrap(); - fs::write(root.join("_schema.pg"), "junk").unwrap(); - - let out = apply_config_dir(dir.path()).await; - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "graph_create_failed") - ); - let by_resource: BTreeMap<&str, &PlanChange> = out - .changes - .iter() - .map(|change| (change.resource.as_str(), change)) - .collect(); - // Dependents are demoted: the run tells the truth about what executed. - assert_eq!( - by_resource["graph.knowledge"].disposition, - Some(ApplyDisposition::Blocked) - ); - assert_eq!( - by_resource["query.knowledge.find_person"].disposition, - Some(ApplyDisposition::Blocked) - ); - assert_eq!( - by_resource["query.knowledge.find_person"].reason.as_deref(), - Some("dependency_not_applied") - ); - assert_eq!( - by_resource["policy.base"].disposition, - Some(ApplyDisposition::Blocked) - ); - assert!(!out.converged); - // The sidecar stays for the sweep to classify next run. - assert!( - fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) - .unwrap() - .next() - .is_some() - ); - // No graph digests moved. - let state = read_state_json(dir.path()); - assert!( - state["applied_revision"]["resources"] - .as_object() - .unwrap() - .is_empty() - ); - } - - #[tokio::test] - async fn apply_blocks_graph_delete_without_approval() { - let dir = fixture(); - let desired = validate_config_dir(dir.path()); - let schema_digest = desired - .resource_digests - .get("schema.knowledge") - .unwrap() - .clone(); - let graph_composite = - graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); - write_state_resources( - dir.path(), - &[ - ("graph.knowledge", graph_composite.as_str()), - ("schema.knowledge", schema_digest.as_str()), - ("graph.old", "3333"), - ("schema.old", "4444"), - ("query.old.q", "5555"), - ], - ); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!(!out.converged); - let by_resource: BTreeMap<&str, &PlanChange> = out - .changes - .iter() - .map(|change| (change.resource.as_str(), change)) - .collect(); - // Stage 4C: deletes are gated, not deferred — every subtree change - // blocks on the single graph-level approval. - assert_eq!( - by_resource["graph.old"].disposition, - Some(ApplyDisposition::Blocked) - ); - assert_eq!( - by_resource["graph.old"].reason.as_deref(), - Some("approval_required") - ); - assert_eq!( - by_resource["schema.old"].reason.as_deref(), - Some("approval_required") - ); - assert_eq!( - by_resource["query.old.q"].reason.as_deref(), - Some("approval_required") - ); - // State intact; nothing destroyed without the artifact. - let state = read_state_json(dir.path()); - let resources = &state["applied_revision"]["resources"]; - assert_eq!(resources["graph.old"]["digest"], "3333"); - assert_eq!(resources["schema.old"]["digest"], "4444"); - assert_eq!(resources["query.old.q"]["digest"], "5555"); - } - - #[tokio::test] - async fn approve_writes_digest_bound_artifact() { - let dir = fixture(); - write_applyable_state(dir.path()); - // Seed a deletable subtree. - let state = read_state_json(dir.path()); - let graph_digest_str = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] - .as_str() - .unwrap() - .to_string(); - let schema_digest_str = state["applied_revision"]["resources"]["schema.knowledge"] - ["digest"] - .as_str() - .unwrap() - .to_string(); - write_state_resources( - dir.path(), - &[ - ("graph.knowledge", graph_digest_str.as_str()), - ("schema.knowledge", schema_digest_str.as_str()), - ("graph.old", "3333"), - ("schema.old", "4444"), - ], - ); - - let out = approve_config_dir(dir.path(), "graph.old", "andrew").await; - assert!(out.ok, "{:?}", out.diagnostics); - let approval_id = out.approval_id.clone().unwrap(); - let artifact: serde_json::Value = serde_json::from_str( - &fs::read_to_string( - dir.path() - .join(CLUSTER_APPROVALS_DIR) - .join(format!("{approval_id}.json")), - ) - .unwrap(), - ) - .unwrap(); - assert_eq!(artifact["resource"], "graph.old"); - assert_eq!(artifact["operation"], "delete"); - assert_eq!(artifact["approved_by"], "andrew"); - assert_eq!(artifact["bound_before_digest"], "3333"); - assert!(artifact["bound_after_digest"].is_null()); - assert!(artifact["bound_config_digest"].is_string()); - assert!(artifact["consumed_at"].is_null()); - - // A non-gated address is refused. - let not_gated = approve_config_dir(dir.path(), "query.knowledge.find_person", "andrew").await; - assert!(!not_gated.ok); - assert!( - not_gated - .diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "approval_not_required") - ); - } - - #[tokio::test] - async fn stale_approval_is_ignored() { - let dir = fixture(); - write_applyable_state(dir.path()); - let state = read_state_json(dir.path()); - let graph_digest_str = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] - .as_str() - .unwrap() - .to_string(); - let schema_digest_str = state["applied_revision"]["resources"]["schema.knowledge"] - ["digest"] - .as_str() - .unwrap() - .to_string(); - write_state_resources( - dir.path(), - &[ - ("graph.knowledge", graph_digest_str.as_str()), - ("schema.knowledge", schema_digest_str.as_str()), - ("graph.old", "3333"), - ], - ); - let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; - assert!(approved.ok, "{:?}", approved.diagnostics); - // The config moves after approval: the bound config digest no longer - // matches and the artifact authorizes nothing. - fs::write(dir.path().join("base.policy.yaml"), "rules: [] # moved\n").unwrap(); - - let out = apply_config_dir(dir.path()).await; - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "approval_stale"), - "{:?}", - out.diagnostics - ); - let by_resource: BTreeMap<&str, &PlanChange> = out - .changes - .iter() - .map(|change| (change.resource.as_str(), change)) - .collect(); - assert_eq!( - by_resource["graph.old"].reason.as_deref(), - Some("approval_required") - ); - let state = read_state_json(dir.path()); - assert_eq!( - state["applied_revision"]["resources"]["graph.old"]["digest"], - "3333" - ); - } - - #[tokio::test] - async fn compute_approvals_one_gate_per_subtree() { - let dir = fixture(); - write_applyable_state(dir.path()); - let state = read_state_json(dir.path()); - let g = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] - .as_str() - .unwrap() - .to_string(); - let sc = state["applied_revision"]["resources"]["schema.knowledge"]["digest"] - .as_str() - .unwrap() - .to_string(); - write_state_resources( - dir.path(), - &[ - ("graph.knowledge", g.as_str()), - ("schema.knowledge", sc.as_str()), - ("graph.old", "3333"), - ("schema.old", "4444"), - ("query.old.q", "5555"), - ], - ); - let plan = plan_config_dir(dir.path()).await; - let gated: Vec<&str> = plan - .approvals_required - .iter() - .map(|gate| gate.resource.as_str()) - .collect(); - assert_eq!(gated, vec!["graph.old"], "{plan:?}"); - assert!(!plan.approvals_required[0].satisfied); - } - - #[tokio::test] - async fn apply_is_idempotent() { - let dir = fixture(); - write_applyable_state(dir.path()); - - let first = apply_config_dir(dir.path()).await; - assert!(first.ok, "{:?}", first.diagnostics); - assert!(first.state_written); - let state_after_first = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); - - let second = apply_config_dir(dir.path()).await; - assert!(second.ok, "{:?}", second.diagnostics); - assert!(second.changes.is_empty()); - assert_eq!(second.applied_count, 0); - assert!(second.converged); - assert!(!second.state_written); - let state_after_second = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); - assert_eq!(state_after_first, state_after_second); - assert_eq!(second.state_observations.state_revision, 2); - } - - #[tokio::test] - async fn apply_respects_held_lock() { - let dir = fixture(); - write_applyable_state(dir.path()); - write_lock_file(dir.path(), "held-lock", "plan"); - - let out = apply_config_dir(dir.path()).await; - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_lock_held") - ); - // The held lock survives a refused apply, and nothing was written. - assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); - assert!(!dir.path().join(CLUSTER_RESOURCES_DIR).exists()); - let state = read_state_json(dir.path()); - assert_eq!(state["state_revision"], 1); - } - - #[tokio::test] - async fn apply_state_lock_false_bypasses_with_warning() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - r#" -version: 1 -state: - backend: cluster - lock: false -graphs: - knowledge: - schema: ./people.pg - queries: - find_person: - file: ./people.gq -"#, - ) - .unwrap(); - write_applyable_state(dir.path()); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.state_written); - assert!(!out.state_observations.lock_acquired); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_lock_disabled") - ); - assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); - } - - #[tokio::test] - async fn apply_skips_existing_payload_blob() { - let dir = fixture(); - write_applyable_state(dir.path()); - let desired = validate_config_dir(dir.path()); - let query_digest = desired - .resource_digests - .get("query.knowledge.find_person") - .unwrap() - .clone(); - // Content-addressed blobs are trusted by name: an existing file is - // never rewritten. - let blob = query_payload_path(dir.path(), &query_digest); - fs::create_dir_all(blob.parent().unwrap()).unwrap(); - fs::write(&blob, "pre-existing").unwrap(); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert_eq!(fs::read_to_string(&blob).unwrap(), "pre-existing"); - } - - #[tokio::test] - async fn apply_invalid_config_fails_before_lock() { - let dir = fixture(); - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - "version: 1\nnot_a_field: true\n", - ) - .unwrap(); - - let out = apply_config_dir(dir.path()).await; - assert!(!out.ok); - // Config errors bail before the lock or any state directory exists. - assert!(!dir.path().join(CLUSTER_STATE_DIR).exists()); - } - - /// When the state write fails after payloads landed, the output must - /// report the statuses actually on disk — not the unpersisted in-memory - /// mutations (phantom `applied` entries would mislead automation that - /// reads `resource_statuses` independently of `ok`). - #[cfg(unix)] - #[tokio::test] - async fn apply_state_write_failure_reports_persisted_statuses() { - use std::os::unix::fs::PermissionsExt; - - let dir = fixture(); - // lock: false so the only write into __cluster/ is state.json itself. - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - r#" -version: 1 -state: - backend: cluster - lock: false -graphs: - knowledge: - schema: ./people.pg - queries: - find_person: - file: ./people.gq -"#, - ) - .unwrap(); - write_applyable_state(dir.path()); - // Pre-create the payload blob so the payload phase is a no-op and the - // failure lands exactly at the state write. - let desired = validate_config_dir(dir.path()); - let query_digest = desired - .resource_digests - .get("query.knowledge.find_person") - .unwrap(); - let blob = query_payload_path(dir.path(), query_digest); - fs::create_dir_all(blob.parent().unwrap()).unwrap(); - fs::write(&blob, QUERY).unwrap(); - - let state_dir = dir.path().join(CLUSTER_STATE_DIR); - fs::set_permissions(&state_dir, fs::Permissions::from_mode(0o555)).unwrap(); - // Running as root ignores permission bits; skip rather than flake. - if fs::write(state_dir.join("probe"), b"x").is_ok() { - let _ = fs::remove_file(state_dir.join("probe")); - fs::set_permissions(&state_dir, fs::Permissions::from_mode(0o755)).unwrap(); - eprintln!("skipping: permissions are not enforced (running as root)"); - return; - } - - let out = apply_config_dir(dir.path()).await; - fs::set_permissions(&state_dir, fs::Permissions::from_mode(0o755)).unwrap(); - - assert!(!out.ok); - assert!(!out.state_written); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "state_write_error"), - "{:?}", - out.diagnostics - ); - // The seeded state has no statuses; the failed apply must not invent - // the in-memory `applied` ones it failed to persist. - assert!( - out.resource_statuses.is_empty(), - "unpersisted statuses leaked into output: {:?}", - out.resource_statuses - ); - } - - // ---- catalog payload verification (Stage 3B) ---- - - /// Converge a fixture dir and return the query blob path. - async fn converge_fixture(config_dir: &Path) -> std::path::PathBuf { - write_applyable_state(config_dir); - let out = apply_config_dir(config_dir).await; - assert!(out.ok && out.converged, "{:?}", out.diagnostics); - let desired = validate_config_dir(config_dir); - query_payload_path( - config_dir, - desired - .resource_digests - .get("query.knowledge.find_person") - .unwrap(), - ) - } - - #[tokio::test] - async fn status_reports_missing_payload_read_only() { - let dir = fixture(); - let blob = converge_fixture(dir.path()).await; - let state_before = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); - fs::remove_file(&blob).unwrap(); - - let out = status_config_dir(dir.path()); - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.diagnostics.iter().any(|diagnostic| { - diagnostic.code == "catalog_payload_missing" - && diagnostic.path == "query.knowledge.find_person" - })); - // Read-only: persisted statuses and state bytes untouched. - assert_eq!( - out.resource_statuses["query.knowledge.find_person"].status, - ResourceLifecycleStatus::Applied - ); - assert_eq!( - fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(), - state_before - ); - } - - #[tokio::test] - async fn refresh_removes_digest_and_drifts_on_missing_payload() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - let blob = converge_fixture(dir.path()).await; - fs::remove_file(&blob).unwrap(); - - let out = refresh_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "catalog_payload_missing") - ); - let status = &out.resource_statuses["query.knowledge.find_person"]; - assert_eq!(status.status, ResourceLifecycleStatus::Drifted); - assert!(status.conditions.contains(&"payload_missing".to_string())); - let state = read_state_json(dir.path()); - assert!( - state["applied_revision"]["resources"] - .get("query.knowledge.find_person") - .is_none(), - "{state}" - ); - } - - #[tokio::test] - async fn refresh_drifts_on_corrupted_payload() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - let blob = converge_fixture(dir.path()).await; - fs::write(&blob, "corrupted content").unwrap(); - - let out = refresh_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - let status = &out.resource_statuses["query.knowledge.find_person"]; - assert_eq!(status.status, ResourceLifecycleStatus::Drifted); - assert!(status.conditions.contains(&"payload_mismatch".to_string())); - let state = read_state_json(dir.path()); - assert!( - state["applied_revision"]["resources"] - .get("query.knowledge.find_person") - .is_none() - ); - } - - #[tokio::test] - async fn refresh_flags_unreadable_payload_as_error() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - let blob = converge_fixture(dir.path()).await; - // A same-named directory yields a non-NotFound IO error portably. - fs::remove_file(&blob).unwrap(); - fs::create_dir(&blob).unwrap(); - - let out = refresh_config_dir(dir.path()).await; - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "catalog_payload_read_error") - ); - let status = &out.resource_statuses["query.knowledge.find_person"]; - assert_eq!(status.status, ResourceLifecycleStatus::Error); - assert!(status.conditions.contains(&"payload_read_error".to_string())); - // Transient IO keeps the digest: no spurious republish. - let state = read_state_json(dir.path()); - assert!( - state["applied_revision"]["resources"] - .get("query.knowledge.find_person") - .is_some() - ); - } - - #[tokio::test] - async fn payload_drift_self_heals_through_refresh_plan_apply() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - let blob = converge_fixture(dir.path()).await; - let original = fs::read_to_string(&blob).unwrap(); - fs::remove_file(&blob).unwrap(); - - let refresh = refresh_config_dir(dir.path()).await; - assert!(refresh.ok, "{:?}", refresh.diagnostics); - - let plan = plan_config_dir(dir.path()).await; - let query_change = plan - .changes - .iter() - .find(|change| change.resource == "query.knowledge.find_person") - .expect("plan must propose recreating the query"); - assert_eq!(query_change.operation, PlanOperation::Create); - assert_eq!(query_change.disposition, Some(ApplyDisposition::Applied)); - - let apply = apply_config_dir(dir.path()).await; - assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); - assert_eq!(fs::read_to_string(&blob).unwrap(), original); - - let status = status_config_dir(dir.path()); - assert!( - !status - .diagnostics - .iter() - .any(|diagnostic| diagnostic.code.starts_with("catalog_payload")), - "{:?}", - status.diagnostics - ); - } - - #[test] - fn verification_skips_graph_and_schema_resources() { - let dir = fixture(); - write_applyable_state(dir.path()); // graph + schema digests only, no blobs - - let out = status_config_dir(dir.path()); - assert!( - !out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code.starts_with("catalog_payload")), - "{:?}", - out.diagnostics - ); - } - - // ---- recovery sidecars + sweep (Stage 4A) ---- - - fn derived_graph_uri(config_dir: &Path, graph_id: &str) -> String { - display_path( - &config_dir - .join(CLUSTER_GRAPHS_DIR) - .join(format!("{graph_id}.omni")), - ) - } - - fn write_create_sidecar( - config_dir: &Path, - graph_id: &str, - desired_schema_digest: &str, - operation_id: &str, - ) -> PathBuf { - let dir = config_dir.join(CLUSTER_RECOVERIES_DIR); - fs::create_dir_all(&dir).unwrap(); - let path = dir.join(format!("{operation_id}.json")); - fs::write( - &path, - serde_json::to_string_pretty(&json!({ - "schema_version": 1, - "operation_id": operation_id, - "started_at": "1970-01-01T00:00:00Z", - "kind": "graph_create", - "graph_id": graph_id, - "graph_uri": derived_graph_uri(config_dir, graph_id), - "desired_schema_digest": desired_schema_digest, - })) - .unwrap(), - ) - .unwrap(); - path - } - - #[tokio::test] - async fn sweep_removes_sidecar_when_root_absent() { - let dir = fixture(); - write_applyable_state(dir.path()); - let sidecar = write_create_sidecar(dir.path(), "knowledge", "irrelevant", "01ROW1"); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - // Row 1: nothing moved; intent removed, run proceeds normally. - assert!(!sidecar.exists()); - assert!(out.converged); - } - - #[tokio::test] - async fn sweep_rolls_forward_completed_create() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_state_resources(dir.path(), &[]); // state predates the create - let desired = validate_config_dir(dir.path()); - let schema_digest = desired.resource_digests["schema.knowledge"].clone(); - let sidecar = write_create_sidecar(dir.path(), "knowledge", &schema_digest, "01ROW4"); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") - ); - // Row 4: ledger converged to observable reality, audit recorded, - // sidecar retired after the CAS landed. - let state = read_state_json(dir.path()); - assert_eq!( - state["applied_revision"]["resources"]["schema.knowledge"]["digest"], - schema_digest - ); - assert!( - state["recovery_records"] - .as_object() - .unwrap() - .values() - .any(|record| record["outcome"] == "rolled_forward" - && record["graph_id"] == "knowledge") - ); - assert!(!sidecar.exists()); - // With the graph rolled forward, the same run converges the catalog. - assert!(out.converged, "{out:?}"); - } - - #[tokio::test] - async fn sweep_completes_already_recorded_create() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_applyable_state(dir.path()); // state already records graph+schema - let desired = validate_config_dir(dir.path()); - let sidecar = write_create_sidecar( - dir.path(), - "knowledge", - &desired.resource_digests["schema.knowledge"], - "01ROW2", - ); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - // Row 2: outcome was already durable; no audit entry, sidecar retired. - assert!(!sidecar.exists()); - let state = read_state_json(dir.path()); - assert!( - state["recovery_records"] - .as_object() - .is_none_or(|records| records.is_empty()), - "{state}" - ); - } - - #[tokio::test] - async fn sweep_keeps_sidecar_for_incomplete_root() { - let dir = fixture(); - write_applyable_state(dir.path()); - // A root that exists but cannot be opened: the engine's partial-init gap. - let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); - fs::create_dir_all(&root).unwrap(); - fs::write(root.join("_schema.pg"), "junk").unwrap(); - let sidecar = write_create_sidecar(dir.path(), "knowledge", "whatever", "01ROW5"); - - let out = apply_config_dir(dir.path()).await; - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "graph_create_incomplete") - ); - // Row 5: never auto-delete; sidecar and root stay for the operator, - // and the Error status is persisted by the run's state write. - assert!(sidecar.exists()); - assert!(root.exists()); - let state = read_state_json(dir.path()); - assert_eq!(state["resource_statuses"]["graph.knowledge"]["status"], "error"); - assert!( - state["resource_statuses"]["graph.knowledge"]["conditions"] - .as_array() - .unwrap() - .iter() - .any(|condition| condition == "graph_create_incomplete") - ); - } - - #[tokio::test] - async fn sweep_flags_unexpected_schema_as_pending() { - let dir = fixture(); - write_state_resources(dir.path(), &[]); - // Live graph exists with a schema the sidecar never intended. - let graph_dir = dir.path().join(CLUSTER_GRAPHS_DIR); - fs::create_dir_all(&graph_dir).unwrap(); - Omnigraph::init( - &derived_graph_uri(dir.path(), "knowledge"), - "\nnode Other {\n name: String @key\n}\n", - ) - .await - .unwrap(); - let desired = validate_config_dir(dir.path()); - let sidecar = write_create_sidecar( - dir.path(), - "knowledge", - &desired.resource_digests["schema.knowledge"], - "01ROW6", - ); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); // warning, not error - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "cluster_recovery_pending") - ); - // Row 6: refuse to guess; sidecar kept, Drifted persisted. - assert!(sidecar.exists()); - let state = read_state_json(dir.path()); - assert_eq!( - state["resource_statuses"]["graph.knowledge"]["status"], - "drifted" - ); - assert!( - state["resource_statuses"]["graph.knowledge"]["conditions"] - .as_array() - .unwrap() - .iter() - .any(|condition| condition == "actual_applied_state_pending") - ); - } - - #[tokio::test] - async fn apply_blocks_create_while_recovery_pending() { - let dir = fixture(); - write_state_resources(dir.path(), &[]); - // A kept (row 5) sidecar: partial root that cannot be opened. - let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); - fs::create_dir_all(&root).unwrap(); - fs::write(root.join("_schema.pg"), "junk").unwrap(); - let sidecar = write_create_sidecar(dir.path(), "knowledge", "whatever", "01PEND"); - - let out = apply_config_dir(dir.path()).await; - assert!(!out.ok); // row 5 is an error condition - let by_resource: BTreeMap<&str, &PlanChange> = out - .changes - .iter() - .map(|change| (change.resource.as_str(), change)) - .collect(); - // The pending recovery blocks the create and its dependents; the - // executor never attempts the init. - assert_eq!( - by_resource["graph.knowledge"].disposition, - Some(ApplyDisposition::Blocked) - ); - assert_eq!( - by_resource["graph.knowledge"].reason.as_deref(), - Some("cluster_recovery_pending") - ); - assert_eq!( - by_resource["query.knowledge.find_person"].reason.as_deref(), - Some("cluster_recovery_pending") - ); - assert_eq!( - by_resource["policy.base"].reason.as_deref(), - Some("cluster_recovery_pending") - ); - assert!(sidecar.exists()); - // The sweep's Error status is what persists — not a generic Blocked. - let state = read_state_json(dir.path()); - assert_eq!(state["resource_statuses"]["graph.knowledge"]["status"], "error"); - } - - #[tokio::test] - async fn plan_embeds_migration_preview_for_schema_update() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_applyable_state(dir.path()); - fs::write( - dir.path().join("people.pg"), - "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n", - ) - .unwrap(); - - let out = plan_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - let schema_change = out - .changes - .iter() - .find(|change| change.resource == "schema.knowledge") - .unwrap(); - let migration = schema_change.migration.as_ref().expect("preview embedded"); - assert!(migration.supported); - assert!( - serde_json::to_string(&migration.steps) - .unwrap() - .contains("add_property"), - "{migration:?}" - ); - } - - #[tokio::test] - async fn plan_warns_when_preview_unavailable() { - let dir = fixture(); - write_applyable_state(dir.path()); // digests recorded, but no live root - fs::write( - dir.path().join("people.pg"), - "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n", - ) - .unwrap(); - - let out = plan_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - let schema_change = out - .changes - .iter() - .find(|change| change.resource == "schema.knowledge") - .unwrap(); - assert!(schema_change.migration.is_none()); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "schema_preview_unavailable") - ); - } - - fn write_schema_apply_sidecar( - config_dir: &Path, - graph_id: &str, - desired_schema_digest: &str, - operation_id: &str, - ) -> PathBuf { - let dir = config_dir.join(CLUSTER_RECOVERIES_DIR); - fs::create_dir_all(&dir).unwrap(); - let path = dir.join(format!("{operation_id}.json")); - fs::write( - &path, - serde_json::to_string_pretty(&json!({ - "schema_version": 1, - "operation_id": operation_id, - "started_at": "1970-01-01T00:00:00Z", - "kind": "schema_apply", - "graph_id": graph_id, - "graph_uri": derived_graph_uri(config_dir, graph_id), - "desired_schema_digest": desired_schema_digest, - })) - .unwrap(), - ) - .unwrap(); - path - } - - const SCHEMA_V2: &str = "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n"; - - #[tokio::test] - async fn sweep_retires_schema_sidecar_when_ledger_consistent() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_applyable_state(dir.path()); // state digest == live digest - let sidecar = - write_schema_apply_sidecar(dir.path(), "knowledge", "never-applied", "01SROW1"); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!(!sidecar.exists()); - let state = read_state_json(dir.path()); - assert!( - state["recovery_records"] - .as_object() - .is_none_or(|records| records.is_empty()) - ); - } - - #[tokio::test] - async fn sweep_rolls_forward_completed_schema_apply() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_applyable_state(dir.path()); - // The schema apply completed on the graph out-of-process... - let graph_uri = derived_graph_uri(dir.path(), "knowledge"); - let db = Omnigraph::open(&graph_uri).await.unwrap(); - db.apply_schema(SCHEMA_V2).await.unwrap(); - // ...the desired config matches it, and the sidecar records the intent. - fs::write(dir.path().join("people.pg"), SCHEMA_V2).unwrap(); - let desired = validate_config_dir(dir.path()); - let v2_digest = desired.resource_digests["schema.knowledge"].clone(); - let sidecar = write_schema_apply_sidecar(dir.path(), "knowledge", &v2_digest, "01SROW3"); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") - ); - assert!(!sidecar.exists()); - let state = read_state_json(dir.path()); - assert_eq!( - state["applied_revision"]["resources"]["schema.knowledge"]["digest"], - v2_digest - ); - assert!( - state["recovery_records"] - .as_object() - .unwrap() - .values() - .any(|record| record["kind"] == "schema_apply" - && record["outcome"] == "rolled_forward") - ); - assert!(out.converged, "{out:?}"); - } - - #[tokio::test] - async fn sweep_flags_unexpected_schema_apply_state_as_pending() { - let dir = fixture(); - init_derived_graph(dir.path()).await; // live = v1 - write_state_resources(dir.path(), &[("schema.knowledge", "stale-digest")]); - // Sidecar intended a digest that is neither live nor recorded. - let sidecar = - write_schema_apply_sidecar(dir.path(), "knowledge", "intended-digest", "01SROW6"); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); // warnings only - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "cluster_recovery_pending") - ); - assert!(sidecar.exists()); - let state = read_state_json(dir.path()); - assert_eq!( - state["resource_statuses"]["schema.knowledge"]["status"], - "drifted" - ); - } - - #[tokio::test] - async fn sweep_keeps_schema_sidecar_for_unopenable_root() { - let dir = fixture(); - write_applyable_state(dir.path()); - let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); - fs::create_dir_all(&root).unwrap(); // exists, won't open - let sidecar = - write_schema_apply_sidecar(dir.path(), "knowledge", "whatever", "01SROWX"); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); // warning: cannot verify - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "cluster_recovery_pending") - ); - assert!(sidecar.exists()); - } - - /// Seed: converged knowledge subtree + a stale `old` graph subtree with a - /// real directory on disk. - fn seed_deletable_state(config_dir: &Path) { - write_applyable_state(config_dir); - let state = read_state_json(config_dir); - let g = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] - .as_str() - .unwrap() - .to_string(); - let sc = state["applied_revision"]["resources"]["schema.knowledge"]["digest"] - .as_str() - .unwrap() - .to_string(); - write_state_resources( - config_dir, - &[ - ("graph.knowledge", g.as_str()), - ("schema.knowledge", sc.as_str()), - ("graph.old", "3333"), - ("schema.old", "4444"), - ("query.old.q", "5555"), - ], - ); - let root = config_dir.join(CLUSTER_GRAPHS_DIR).join("old.omni"); - fs::create_dir_all(&root).unwrap(); - fs::write(root.join("_schema.pg"), "stale").unwrap(); - } - - #[tokio::test] - async fn apply_executes_approved_graph_delete() { - let dir = fixture(); - seed_deletable_state(dir.path()); - let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; - assert!(approved.ok, "{:?}", approved.diagnostics); - let approval_id = approved.approval_id.clone().unwrap(); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.converged, "{out:?}"); - let by_resource: BTreeMap<&str, &PlanChange> = out - .changes - .iter() - .map(|change| (change.resource.as_str(), change)) - .collect(); - assert_eq!(by_resource["graph.old"].disposition, Some(ApplyDisposition::Applied)); - assert_eq!(by_resource["schema.old"].disposition, Some(ApplyDisposition::Applied)); - assert_eq!(by_resource["query.old.q"].disposition, Some(ApplyDisposition::Applied)); - // The root is gone; the subtree is tombstoned out of the ledger. - assert!(!dir.path().join(CLUSTER_GRAPHS_DIR).join("old.omni").exists()); - let state = read_state_json(dir.path()); - let resources = state["applied_revision"]["resources"].as_object().unwrap(); - assert!(!resources.contains_key("graph.old")); - assert!(!resources.contains_key("schema.old")); - assert!(!resources.contains_key("query.old.q")); - assert_eq!(state["observations"]["graph.old"]["kind"], "tombstone"); - assert_eq!(state["observations"]["graph.old"]["approval_id"], approval_id); - // Approval consumed in BOTH stores: ledger summary + artifact file. - assert!(state["approval_records"][&approval_id]["consumed_at"].is_string()); - let artifact: serde_json::Value = serde_json::from_str( - &fs::read_to_string( - dir.path() - .join(CLUSTER_APPROVALS_DIR) - .join(format!("{approval_id}.json")), - ) - .unwrap(), - ) - .unwrap(); - assert!(artifact["consumed_at"].is_string(), "{artifact}"); - // Sidecar retired. - assert!( - fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) - .map(|mut entries| entries.next().is_none()) - .unwrap_or(true) - ); - // A consumed approval authorizes nothing further (idempotent re-apply). - let again = apply_config_dir(dir.path()).await; - assert!(again.ok && again.converged && !again.state_written, "{again:?}"); - } - - fn write_delete_sidecar( - config_dir: &Path, - graph_id: &str, - approval_id: Option<&str>, - operation_id: &str, - ) -> PathBuf { - let dir = config_dir.join(CLUSTER_RECOVERIES_DIR); - fs::create_dir_all(&dir).unwrap(); - let path = dir.join(format!("{operation_id}.json")); - fs::write( - &path, - serde_json::to_string_pretty(&json!({ - "schema_version": 1, - "operation_id": operation_id, - "started_at": "1970-01-01T00:00:00Z", - "kind": "graph_delete", - "graph_id": graph_id, - "graph_uri": derived_graph_uri(config_dir, graph_id), - "desired_schema_digest": "", - "approval_id": approval_id, - })) - .unwrap(), - ) - .unwrap(); - path - } - - #[tokio::test] - async fn sweep_retires_delete_sidecar_when_tombstoned() { - let dir = fixture(); - write_applyable_state(dir.path()); // no graph.old in state, no root - let sidecar = write_delete_sidecar(dir.path(), "old", None, "01DROW7"); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!(!sidecar.exists()); - let state = read_state_json(dir.path()); - assert!( - state["recovery_records"] - .as_object() - .is_none_or(|records| records.is_empty()) - ); - } - - #[tokio::test] - async fn sweep_rolls_forward_completed_delete() { - let dir = fixture(); - seed_deletable_state(dir.path()); - // Approve, then simulate: root removed, state stale, sidecar present. - let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; - let approval_id = approved.approval_id.unwrap(); - fs::remove_dir_all(dir.path().join(CLUSTER_GRAPHS_DIR).join("old.omni")).unwrap(); - let sidecar = write_delete_sidecar(dir.path(), "old", Some(&approval_id), "01DROW7B"); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") - ); - assert!(!sidecar.exists()); - let state = read_state_json(dir.path()); - assert!( - !state["applied_revision"]["resources"] - .as_object() - .unwrap() - .contains_key("graph.old") - ); - assert_eq!(state["observations"]["graph.old"]["kind"], "tombstone"); - assert!(state["approval_records"][&approval_id]["consumed_at"].is_string()); - assert!( - state["recovery_records"] - .as_object() - .unwrap() - .values() - .any(|record| record["kind"] == "graph_delete" - && record["outcome"] == "rolled_forward") - ); - // The artifact file is marked consumed post-CAS. - let artifact: serde_json::Value = serde_json::from_str( - &fs::read_to_string( - dir.path() - .join(CLUSTER_APPROVALS_DIR) - .join(format!("{approval_id}.json")), - ) - .unwrap(), - ) - .unwrap(); - assert!(artifact["consumed_at"].is_string()); - assert!(out.converged, "{out:?}"); - } - - #[tokio::test] - async fn sweep_reproposes_incomplete_delete() { - let dir = fixture(); - seed_deletable_state(dir.path()); // root present - let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; - assert!(approved.ok); - let sidecar = write_delete_sidecar(dir.path(), "old", approved.approval_id.as_deref(), "01DROW8"); - - // Row 8: the stale intent is retired with a warning, and the same run - // re-executes the still-approved delete to completion. - let out = apply_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "graph_delete_incomplete") - ); - assert!(!sidecar.exists()); - assert!(!dir.path().join(CLUSTER_GRAPHS_DIR).join("old.omni").exists()); - assert!(out.converged, "{out:?}"); - } - - // ---- policy bindings in the applied revision (5A) ---- - - #[tokio::test] - async fn apply_records_policy_bindings() { - let dir = fixture(); - write_applyable_state(dir.path()); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok && out.converged, "{:?}", out.diagnostics); - let state = read_state_json(dir.path()); - assert_eq!( - state["applied_revision"]["resources"]["policy.base"]["applies_to"], - serde_json::json!(["graph.knowledge"]), - "{state}" - ); - // Non-policy entries carry no bindings field at all. - assert!( - state["applied_revision"]["resources"]["query.knowledge.find_person"] - .get("applies_to") - .is_none() - ); - } - - #[tokio::test] - async fn binding_change_is_a_visible_plan_change() { - let dir = fixture(); - write_applyable_state(dir.path()); - let converge = apply_config_dir(dir.path()).await; - assert!(converge.converged, "{converge:?}"); - // Edit ONLY applies_to: the policy file digest is unchanged. - fs::write( - dir.path().join(CLUSTER_CONFIG_FILE), - r#" -version: 1 -metadata: - name: test -state: - backend: cluster - lock: true -graphs: - knowledge: - schema: ./people.pg - queries: - find_person: - file: ./people.gq -policies: - base: - file: ./base.policy.yaml - applies_to: [cluster, knowledge] -"#, - ) - .unwrap(); - - let plan = plan_config_dir(dir.path()).await; - let change = plan - .changes - .iter() - .find(|change| change.resource == "policy.base") - .expect("binding change must be visible in plan"); - assert!(change.binding_change); - assert_eq!(change.operation, PlanOperation::Update); - assert_eq!(change.before_digest, change.after_digest); - - let out = apply_config_dir(dir.path()).await; - assert!(out.ok && out.converged, "{out:?}"); - let state = read_state_json(dir.path()); - assert_eq!( - state["applied_revision"]["resources"]["policy.base"]["applies_to"], - serde_json::json!(["cluster", "graph.knowledge"]) - ); - // Idempotent: a second run sees no changes. - let again = apply_config_dir(dir.path()).await; - assert!(again.changes.is_empty() && !again.state_written, "{again:?}"); - } - - #[tokio::test] - async fn pre_5a_state_backfills_bindings() { - let dir = fixture(); - write_applyable_state(dir.path()); - let converge = apply_config_dir(dir.path()).await; - assert!(converge.converged, "{converge:?}"); - // Strip the bindings from the state entry (a pre-5A ledger). - let mut state: serde_json::Value = serde_json::from_str( - &fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(), - ) - .unwrap(); - state["applied_revision"]["resources"]["policy.base"] - .as_object_mut() - .unwrap() - .remove("applies_to"); - fs::write( - dir.path().join(CLUSTER_STATE_FILE), - serde_json::to_string_pretty(&state).unwrap(), - ) - .unwrap(); - - let plan = plan_config_dir(dir.path()).await; - assert!( - plan.changes - .iter() - .any(|change| change.resource == "policy.base" && change.binding_change), - "{plan:?}" - ); - let out = apply_config_dir(dir.path()).await; - assert!(out.ok && out.converged, "{out:?}"); - let healed = read_state_json(dir.path()); - assert_eq!( - healed["applied_revision"]["resources"]["policy.base"]["applies_to"], - serde_json::json!(["graph.knowledge"]) - ); - } - - #[tokio::test] - async fn bindings_survive_refresh() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_applyable_state(dir.path()); - let converge = apply_config_dir(dir.path()).await; - assert!(converge.converged, "{converge:?}"); - - let refresh = refresh_config_dir(dir.path()).await; - assert!(refresh.ok, "{:?}", refresh.diagnostics); - let state = read_state_json(dir.path()); - assert_eq!( - state["applied_revision"]["resources"]["policy.base"]["applies_to"], - serde_json::json!(["graph.knowledge"]) - ); - } - - // ---- serving snapshot (5B read-only loader) ---- - - #[tokio::test] - async fn serving_snapshot_reads_converged_cluster() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_applyable_state(dir.path()); - let converge = apply_config_dir(dir.path()).await; - assert!(converge.converged, "{converge:?}"); - - let snapshot = read_serving_snapshot(dir.path()).expect("converged cluster must serve"); - assert_eq!(snapshot.graphs.len(), 1); - assert_eq!(snapshot.graphs[0].graph_id, "knowledge"); - assert!(snapshot.graphs[0].root.ends_with("graphs/knowledge.omni")); - assert_eq!(snapshot.queries.len(), 1); - assert_eq!(snapshot.queries[0].name, "find_person"); - assert!(snapshot.queries[0].source.contains("query find_person")); - assert_eq!(snapshot.policies.len(), 1); - assert_eq!(snapshot.policies[0].applies_to, vec!["graph.knowledge"]); - assert!(snapshot.policies[0].blob_path.exists()); - } - - #[test] - fn serving_snapshot_refuses_missing_state() { - let dir = fixture(); - let err = read_serving_snapshot(dir.path()).unwrap_err(); - assert!( - err.iter().any(|diagnostic| diagnostic.code == "cluster_state_missing"), - "{err:?}" - ); - } - - #[tokio::test] - async fn serving_snapshot_refuses_pending_recovery() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_applyable_state(dir.path()); - apply_config_dir(dir.path()).await; - write_schema_apply_sidecar(dir.path(), "knowledge", "whatever", "01SERVE"); - - let err = read_serving_snapshot(dir.path()).unwrap_err(); - assert!( - err.iter().any(|diagnostic| diagnostic.code == "cluster_recovery_pending"), - "{err:?}" - ); - } - - #[tokio::test] - async fn serving_snapshot_refuses_tampered_blob_and_stripped_bindings() { - let dir = fixture(); - init_derived_graph(dir.path()).await; - write_applyable_state(dir.path()); - apply_config_dir(dir.path()).await; - // Tamper with the query blob... - let snapshot = read_serving_snapshot(dir.path()).unwrap(); - let desired = validate_config_dir(dir.path()); - let query_digest = &desired.resource_digests["query.knowledge.find_person"]; - let blob = dir - .path() - .join(CLUSTER_RESOURCES_DIR) - .join("query/knowledge/find_person") - .join(format!("{query_digest}.gq")); - fs::write(&blob, "tampered").unwrap(); - // ...and strip the policy bindings (pre-5A ledger). - let mut state: serde_json::Value = serde_json::from_str( - &fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(), - ) - .unwrap(); - state["applied_revision"]["resources"]["policy.base"] - .as_object_mut() - .unwrap() - .remove("applies_to"); - fs::write( - dir.path().join(CLUSTER_STATE_FILE), - serde_json::to_string_pretty(&state).unwrap(), - ) - .unwrap(); - - let err = read_serving_snapshot(dir.path()).unwrap_err(); - assert!( - err.iter() - .any(|diagnostic| diagnostic.code == "catalog_payload_digest_mismatch"), - "{err:?}" - ); - assert!( - err.iter().any(|diagnostic| diagnostic.code == "policy_bindings_missing"), - "{err:?}" - ); - let _ = snapshot; // the pre-tamper read succeeded - } - - #[test] - fn serving_snapshot_refuses_empty_cluster() { - let dir = fixture(); - write_state_resources(dir.path(), &[]); // state exists, no graphs - - let err = read_serving_snapshot(dir.path()).unwrap_err(); - assert!( - err.iter().any(|diagnostic| diagnostic.code == "cluster_empty"), - "{err:?}" - ); - } - - // ---- query discovery (Terraform-style declaration) ---- - - #[test] - fn queries_directory_discovers_every_declaration() { - let dir = tempfile::tempdir().unwrap(); - fs::write(dir.path().join("people.pg"), "\nnode Person {\n name: String @key\n}\n").unwrap(); - fs::create_dir(dir.path().join("queries")).unwrap(); - fs::write( - dir.path().join("queries/people.gq"), - "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n\nquery all_people() {\n match { $p: Person }\n return { $p.name }\n}\n", - ) - .unwrap(); - fs::write( - dir.path().join("queries/extra.gq"), - "\nquery count_people() {\n match { $p: Person }\n return { count($p) }\n}\n", - ) - .unwrap(); - fs::write(dir.path().join("queries/notes.txt"), "ignored").unwrap(); - fs::write( - dir.path().join("cluster.yaml"), - "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: ./queries/\n", - ) - .unwrap(); - - let out = validate_config_dir(dir.path()); - assert!(out.ok, "{:?}", out.diagnostics); - let names: Vec<&str> = out - .resource_digests - .keys() - .filter_map(|address| address.strip_prefix("query.knowledge.")) - .collect(); - assert_eq!(names, vec!["all_people", "count_people", "find_person"]); - } - - #[test] - fn queries_list_and_single_file_forms_discover() { - let dir = tempfile::tempdir().unwrap(); - fs::write(dir.path().join("people.pg"), "\nnode Person {\n name: String @key\n}\n").unwrap(); - fs::write( - dir.path().join("a.gq"), - "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", - ) - .unwrap(); - fs::write( - dir.path().join("b.gq"), - "\nquery all_people() {\n match { $p: Person }\n return { $p.name }\n}\n", - ) - .unwrap(); - fs::write( - dir.path().join("cluster.yaml"), - "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: [./a.gq, ./b.gq]\n", - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.resource_digests.contains_key("query.knowledge.find_person")); - assert!(out.resource_digests.contains_key("query.knowledge.all_people")); - - // Single-file string form - fs::write( - dir.path().join("cluster.yaml"), - "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: ./a.gq\n", - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(out.ok, "{:?}", out.diagnostics); - assert!(out.resource_digests.contains_key("query.knowledge.find_person")); - assert!(!out.resource_digests.contains_key("query.knowledge.all_people")); - } - - #[test] - fn query_discovery_rejects_duplicates_and_parse_errors() { - let dir = tempfile::tempdir().unwrap(); - fs::write(dir.path().join("people.pg"), "\nnode Person {\n name: String @key\n}\n").unwrap(); - let decl = "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n"; - fs::write(dir.path().join("a.gq"), decl).unwrap(); - fs::write(dir.path().join("b.gq"), decl).unwrap(); - fs::write( - dir.path().join("cluster.yaml"), - "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: [./a.gq, ./b.gq]\n", - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "duplicate_query_name"), - "{:?}", - out.diagnostics - ); - - fs::write(dir.path().join("broken.gq"), "query {{{ nope").unwrap(); - fs::write( - dir.path().join("cluster.yaml"), - "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: ./broken.gq\n", - ) - .unwrap(); - let out = validate_config_dir(dir.path()); - assert!(!out.ok); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "query_parse_error"), - "{:?}", - out.diagnostics - ); - } - - #[test] - fn status_warns_on_pending_recovery_sidecar() { - let dir = fixture(); - write_applyable_state(dir.path()); - write_create_sidecar(dir.path(), "knowledge", "irrelevant", "01STATUS"); - - let out = status_config_dir(dir.path()); - assert!(out.ok, "{:?}", out.diagnostics); - assert!( - out.diagnostics - .iter() - .any(|diagnostic| diagnostic.code == "cluster_recovery_pending" - && diagnostic.severity == DiagnosticSeverity::Warning) - ); - } - - #[tokio::test] - async fn plan_annotates_apply_dispositions() { - let dir = fixture(); - let out = plan_config_dir(dir.path()).await; - assert!(out.ok, "{:?}", out.diagnostics); - let by_resource: BTreeMap<&str, &PlanChange> = out - .changes - .iter() - .map(|change| (change.resource.as_str(), change)) - .collect(); - // Stage 4A: graph/schema creates are executable, and dependents ride - // the same run — plan previews exactly that. - assert_eq!( - by_resource["graph.knowledge"].disposition, - Some(ApplyDisposition::Applied) - ); - assert_eq!( - by_resource["schema.knowledge"].disposition, - Some(ApplyDisposition::Applied) - ); - assert_eq!( - by_resource["query.knowledge.find_person"].disposition, - Some(ApplyDisposition::Applied) - ); - assert_eq!( - by_resource["policy.base"].disposition, - Some(ApplyDisposition::Applied) - ); - } -} +#[path = "tests.rs"] +mod tests; diff --git a/crates/omnigraph-cluster/src/tests.rs b/crates/omnigraph-cluster/src/tests.rs new file mode 100644 index 0000000..a03c522 --- /dev/null +++ b/crates/omnigraph-cluster/src/tests.rs @@ -0,0 +1,3019 @@ +//! In-source test suite, moved verbatim from lib.rs (modularization). +//! Indentation is preserved exactly — embedded raw-string fixtures +//! (cluster.yaml/JSON bodies) are content, not formatting. +#![allow(clippy::all)] + + use std::fs; + use std::path::Path; + + use omnigraph::db::Omnigraph; + use serde_json::json; + use tempfile::tempdir; + + use super::*; + + const SCHEMA: &str = r#" +node Person { + name: String @key + age: I32? +} +"#; + + const QUERY: &str = r#" +query find_person($name: String) { + match { $p: Person { name: $name } } + return { $p.name, $p.age } +} +"#; + + fn fixture() -> tempfile::TempDir { + let dir = tempdir().unwrap(); + fs::write(dir.path().join("people.pg"), SCHEMA).unwrap(); + fs::write(dir.path().join("people.gq"), QUERY).unwrap(); + fs::write(dir.path().join("base.policy.yaml"), "rules: []\n").unwrap(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +metadata: + name: test +state: + backend: cluster + lock: true +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +policies: + base: + file: ./base.policy.yaml + applies_to: [knowledge] +"#, + ) + .unwrap(); + dir + } + + async fn init_derived_graph(root: &Path) { + let graph_dir = root.join(CLUSTER_GRAPHS_DIR); + fs::create_dir_all(&graph_dir).unwrap(); + let graph = graph_dir.join("knowledge.omni"); + Omnigraph::init(graph.to_string_lossy().as_ref(), SCHEMA) + .await + .unwrap(); + } + + fn write_lock_file(config_dir: &Path, lock_id: &str, operation: &str) { + let state_dir = config_dir.join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("lock.json"), + json!({ + "version": 1, + "lock_id": lock_id, + "operation": operation, + "created_at": "1970-01-01T00:00:00Z", + "pid": 123 + }) + .to_string(), + ) + .unwrap(); + } + + #[test] + fn valid_minimal_config() { + let dir = fixture(); + let out = validate_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.resource_digests.contains_key("graph.knowledge")); + assert!(out.resource_digests.contains_key("schema.knowledge")); + assert!( + out.dependencies + .iter() + .any(|dep| dep.from == "policy.base" && dep.to == "graph.knowledge") + ); + } + + #[test] + fn unknown_field_rejection() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\ngraphs: {}\nwat: true\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert!(out.diagnostics[0].message.contains("unknown field")); + } + + #[test] + fn future_phase_field_rejection() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\ngraphs: {}\npipelines: {}\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert_eq!(out.diagnostics[0].code, "future_phase_field"); + } + + #[test] + fn duplicate_yaml_key_rejection() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\ngraphs: {}\ngraphs: {}\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert_eq!(out.diagnostics[0].code, "duplicate_yaml_key"); + } + + #[test] + fn duplicate_yaml_key_rejection_keeps_quoted_hashes() { + let diagnostics = + duplicate_key_diagnostics("\"name#display\": one\n\"name#display\": two\n"); + assert_eq!(diagnostics.len(), 1); + assert_eq!(diagnostics[0].code, "duplicate_yaml_key"); + } + + #[test] + fn missing_schema_query_and_policy_files() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +graphs: + knowledge: + schema: ./missing.pg + queries: + find_person: { file: ./missing.gq } +policies: + base: + file: ./missing.policy.yaml + applies_to: [knowledge] +"#, + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + let codes: BTreeSet<_> = out.diagnostics.iter().map(|d| d.code.as_str()).collect(); + assert!(codes.contains("schema_file_missing")); + assert!(codes.contains("query_file_missing")); + assert!(codes.contains("policy_file_missing")); + } + + #[test] + fn wrong_kind_and_dangling_refs_fail() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg +policies: + base: + file: ./base.policy.yaml + applies_to: [query.knowledge.find_person, missing] +"#, + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + let codes: BTreeSet<_> = out.diagnostics.iter().map(|d| d.code.as_str()).collect(); + assert!(codes.contains("wrong_kind_reference")); + assert!(codes.contains("dangling_graph_reference")); + } + + #[test] + fn query_key_mismatch_fails() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg + queries: + different: { file: ./people.gq } +"#, + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert_eq!(out.diagnostics[0].code, "query_key_mismatch"); + } + + #[test] + fn query_typecheck_failure_fails() { + let dir = fixture(); + fs::write( + dir.path().join("people.gq"), + "query find_person() { match { $d: DoesNotExist } return { $d.name } }\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "query_typecheck_error") + ); + } + + #[tokio::test] + async fn missing_state_plans_creates() { + let dir = fixture(); + let out = plan_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!out.state_observations.state_found); + assert!(!out.state_observations.locked); + assert!(out.state_observations.lock_acquired); + assert!( + out.changes + .iter() + .all(|c| c.operation == PlanOperation::Create) + ); + assert!(out.changes.iter().any(|c| c.resource == "graph.knowledge")); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[tokio::test] + async fn config_digest_ignores_yaml_comments_and_formatting() { + let dir = fixture(); + let first = plan_config_dir(dir.path()).await; + assert!(first.ok, "{:?}", first.diagnostics); + + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +# Same semantic config as the fixture, intentionally rendered differently. +version: 1 +metadata: { name: test } +state: { backend: cluster, lock: true } +graphs: + knowledge: + schema: ./people.pg + queries: { find_person: { file: ./people.gq } } +policies: + base: + file: ./base.policy.yaml + applies_to: + - knowledge +"#, + ) + .unwrap(); + + let second = plan_config_dir(dir.path()).await; + assert!(second.ok, "{:?}", second.diagnostics); + assert_eq!( + first.desired_revision.config_digest, + second.desired_revision.config_digest + ); + } + + #[tokio::test] + async fn existing_state_plans_update_and_delete_deterministically() { + let dir = fixture(); + let first = plan_config_dir(dir.path()).await; + let state_dir = dir.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + serde_json::to_string_pretty(&json!({ + "version": 1, + "applied_revision": { + "config_digest": "old", + "resources": { + "graph.knowledge": { "digest": first.resource_digests["graph.knowledge"] }, + "policy.old": { "digest": "abc" }, + "schema.knowledge": { "digest": "old-schema" } + } + } + })) + .unwrap(), + ) + .unwrap(); + + let out = plan_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + let rendered: Vec<_> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), &change.operation)) + .collect(); + assert_eq!( + rendered, + vec![ + ("policy.base", &PlanOperation::Create), + ("policy.old", &PlanOperation::Delete), + ("query.knowledge.find_person", &PlanOperation::Create), + ("schema.knowledge", &PlanOperation::Update), + ] + ); + } + + #[tokio::test] + async fn old_minimal_state_json_still_plans_with_default_revision() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{ + "version": 1, + "applied_revision": { + "config_digest": "old", + "resources": { + "graph.knowledge": { "digest": "old-graph" } + } + } +}"#, + ) + .unwrap(); + + let out = plan_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.state_observations.state_revision, 0); + assert!(out.state_observations.state_cas.is_some()); + assert!(out.changes.iter().any(|change| { + change.resource == "graph.knowledge" && change.operation == PlanOperation::Update + })); + } + + #[test] + fn extended_state_json_status_surfaces_statuses() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + let state = r#"{ + "version": 1, + "state_revision": 42, + "applied_revision": { + "config_digest": "applied-config", + "resources": { + "graph.knowledge": { "digest": "graph-digest" } + } + }, + "resource_statuses": { + "graph.knowledge": { + "status": "applied", + "conditions": ["healthy"], + "message": "ready" + } + }, + "approval_records": {}, + "recovery_records": {}, + "observations": { + "graph.knowledge": { "manifest_version": 12 } + } +}"#; + fs::write(state_dir.join("state.json"), state).unwrap(); + + let out = status_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.state_observations.state_found); + assert_eq!(out.state_observations.state_revision, 42); + assert_eq!( + out.state_observations.state_cas.as_deref(), + Some(format!("sha256:{}", sha256_hex(state.as_bytes())).as_str()) + ); + assert_eq!( + out.resource_digests + .get("graph.knowledge") + .map(String::as_str), + Some("graph-digest") + ); + assert_eq!( + out.resource_statuses["graph.knowledge"].status, + ResourceLifecycleStatus::Applied + ); + } + + #[test] + fn missing_state_status_succeeds_with_warning() { + let dir = fixture(); + let out = status_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!out.state_observations.state_found); + assert_eq!(out.state_observations.state_revision, 0); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_missing") + ); + } + + #[test] + fn invalid_state_status_fails() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write(state_dir.join("state.json"), "{").unwrap(); + + let out = status_config_dir(dir.path()); + assert!(!out.ok); + assert!(out.state_observations.state_found); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "invalid_state_json") + ); + } + + #[test] + fn status_surfaces_full_lock_metadata() { + let dir = fixture(); + write_lock_file(dir.path(), "held-lock", "refresh"); + + let out = status_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.state_observations.locked); + assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); + assert_eq!( + out.state_observations.lock_operation.as_deref(), + Some("refresh") + ); + assert_eq!( + out.state_observations.lock_created_at.as_deref(), + Some("1970-01-01T00:00:00Z") + ); + assert_eq!(out.state_observations.lock_pid, Some(123)); + assert!(out.state_observations.lock_age_seconds.is_some()); + } + + #[test] + fn force_unlock_matching_id_removes_lock() { + let dir = fixture(); + write_lock_file(dir.path(), "held-lock", "plan"); + + let out = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.lock_removed); + assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); + assert_eq!( + out.state_observations.lock_operation.as_deref(), + Some("plan") + ); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn force_unlock_wrong_id_fails_and_preserves_lock() { + let dir = fixture(); + write_lock_file(dir.path(), "held-lock", "plan"); + + let out = force_unlock_config_dir(dir.path(), "other-lock"); + assert!(!out.ok); + assert!(!out.lock_removed); + assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_id_mismatch") + ); + assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn force_unlock_missing_lock_fails() { + let dir = fixture(); + + let out = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(!out.ok); + assert!(!out.lock_removed); + assert!(!out.state_observations.locked); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_missing") + ); + } + + #[test] + fn force_unlock_invalid_lock_json_fails_and_preserves_lock() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write(state_dir.join("lock.json"), "{").unwrap(); + + let out = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(!out.ok); + assert!(!out.lock_removed); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "invalid_state_lock") + ); + assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn force_unlock_unsupported_lock_version_fails_and_preserves_lock() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("lock.json"), + r#"{"version":2,"lock_id":"held-lock","operation":"plan","created_at":"1970-01-01T00:00:00Z","pid":123}"#, + ) + .unwrap(); + + let out = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(!out.ok); + assert!(!out.lock_removed); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "unsupported_state_lock_version") + ); + assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn force_unlock_external_state_backend_rejected() { + let dir = fixture(); + write_lock_file(dir.path(), "held-lock", "plan"); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +state: + backend: s3://state-bucket/cluster +graphs: + knowledge: + schema: ./people.pg +"#, + ) + .unwrap(); + + let out = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(!out.ok); + assert!(!out.lock_removed); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "unsupported_state_backend") + ); + assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[tokio::test] + async fn plan_succeeds_after_force_unlock() { + let dir = fixture(); + write_lock_file(dir.path(), "held-lock", "plan"); + + let locked = plan_config_dir(dir.path()).await; + assert!(!locked.ok); + assert!( + locked + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_held") + ); + + let unlocked = force_unlock_config_dir(dir.path(), "held-lock"); + assert!(unlocked.ok, "{:?}", unlocked.diagnostics); + + let out = plan_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + } + + #[tokio::test] + async fn plan_reports_state_cas_revision_and_removes_lock() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + let state = r#"{ + "version": 1, + "state_revision": 7, + "applied_revision": { + "config_digest": "old", + "resources": { + "graph.knowledge": { "digest": "old-graph" } + } + } +}"#; + fs::write(state_dir.join("state.json"), state).unwrap(); + + let out = plan_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.state_observations.state_revision, 7); + assert_eq!( + out.state_observations.state_cas.as_deref(), + Some(format!("sha256:{}", sha256_hex(state.as_bytes())).as_str()) + ); + assert!(!out.state_observations.locked); + assert!(out.state_observations.lock_id.is_none()); + assert!(out.state_observations.lock_acquired); + assert!(out.state_observations.acquired_lock_id.is_some()); + assert!( + !dir.path().join(CLUSTER_LOCK_FILE).exists(), + "plan must release lock before returning" + ); + } + + #[tokio::test] + async fn existing_lock_makes_plan_fail() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("lock.json"), + r#"{ + "version": 1, + "lock_id": "held-lock", + "operation": "plan", + "created_at": "2026-06-08T00:00:00Z", + "pid": 123 +}"#, + ) + .unwrap(); + + let out = plan_config_dir(dir.path()).await; + assert!(!out.ok); + assert!(out.state_observations.locked); + assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); + assert!(!out.state_observations.lock_acquired); + assert!(out.state_observations.acquired_lock_id.is_none()); + assert_eq!( + out.state_observations.lock_operation.as_deref(), + Some("plan") + ); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_held") + ); + assert!(out.diagnostics.iter().any(|diagnostic| { + diagnostic.code == "state_lock_held" + && diagnostic.message.contains("force-unlock held-lock") + })); + } + + #[tokio::test] + async fn state_lock_false_bypasses_lock_with_warning() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +state: + backend: cluster + lock: false +graphs: + knowledge: + schema: ./people.pg +"#, + ) + .unwrap(); + + let out = plan_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!out.state_observations.locked); + assert!(!out.state_observations.lock_acquired); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_disabled") + ); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[test] + fn external_state_backend_rejected() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\nstate:\n backend: s3://bucket/state\ngraphs: {}\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert_eq!(out.diagnostics[0].code, "unsupported_state_backend"); + } + + #[tokio::test] + async fn external_state_backend_plan_rejected() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\nstate:\n backend: s3://bucket/state\ngraphs: {}\n", + ) + .unwrap(); + let out = plan_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "unsupported_state_backend") + ); + } + + #[tokio::test] + async fn import_missing_state_creates_state_with_graph_observation() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + + let out = import_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.state_observations.state_revision, 1); + assert!(out.state_observations.state_cas.is_some()); + assert!(!out.state_observations.locked); + assert!(out.state_observations.lock_acquired); + assert!(out.state_observations.acquired_lock_id.is_some()); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + assert_eq!( + out.resource_digests + .get("schema.knowledge") + .map(String::as_str), + Some(sha256_hex(SCHEMA.as_bytes()).as_str()) + ); + assert!(out.observations["graph.knowledge"]["manifest_version"].is_number()); + assert_eq!( + out.observations["graph.knowledge"]["schema_matches_desired"], + true + ); + + let state: serde_json::Value = + serde_json::from_str(&fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap()) + .unwrap(); + assert_eq!(state["state_revision"], 1); + assert_eq!( + state["resource_statuses"]["graph.knowledge"]["status"], + "applied" + ); + } + + #[tokio::test] + async fn import_existing_state_fails() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + + let out = import_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_already_exists") + ); + } + + #[tokio::test] + async fn refresh_missing_state_fails() { + let dir = fixture(); + let out = refresh_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_missing") + ); + } + + #[tokio::test] + async fn refresh_existing_minimal_state_increments_revision_and_updates_cas() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"config_digest":"old","resources":{"graph.knowledge":{"digest":"old"}}}}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.state_observations.state_revision, 1); + assert!(out.state_observations.state_cas.is_some()); + assert!(!out.state_observations.locked); + assert!(out.state_observations.lock_acquired); + assert_eq!( + out.resource_statuses["graph.knowledge"].status, + ResourceLifecycleStatus::Applied + ); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[tokio::test] + async fn refresh_records_live_schema_digest_and_manifest_version() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"state_revision":4,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.state_observations.state_revision, 5); + assert_eq!( + out.observations["graph.knowledge"]["schema_digest"], + sha256_hex(SCHEMA.as_bytes()) + ); + assert!(out.observations["graph.knowledge"]["manifest_version"].is_u64()); + } + + #[tokio::test] + async fn missing_derived_graph_root_marks_drifted_and_plans_creates() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{"graph.knowledge":{"digest":"old-graph"},"schema.knowledge":{"digest":"old-schema"}}}}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!( + out.resource_statuses["graph.knowledge"].status, + ResourceLifecycleStatus::Drifted + ); + assert!(!out.resource_digests.contains_key("graph.knowledge")); + assert_eq!(out.observations["graph.knowledge"]["exists"], false); + + let plan = plan_config_dir(dir.path()).await; + assert!(plan.ok, "{:?}", plan.diagnostics); + assert!(plan.changes.iter().any(|change| { + change.resource == "graph.knowledge" && change.operation == PlanOperation::Create + })); + assert!(plan.changes.iter().any(|change| { + change.resource == "schema.knowledge" && change.operation == PlanOperation::Create + })); + } + + #[tokio::test] + async fn live_schema_mismatch_marks_drifted_and_causes_plan_update() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + fs::write( + dir.path().join("people.pg"), + SCHEMA.replace("age: I32?", "age: I32?\n nickname: String?"), + ) + .unwrap(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{"graph.knowledge":{"digest":"old-graph"},"schema.knowledge":{"digest":"old-schema"}}}}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!( + out.resource_statuses["schema.knowledge"].status, + ResourceLifecycleStatus::Drifted + ); + assert_eq!( + out.observations["graph.knowledge"]["schema_matches_desired"], + false + ); + + let plan = plan_config_dir(dir.path()).await; + assert!(plan.ok, "{:?}", plan.diagnostics); + assert!(plan.changes.iter().any(|change| { + change.resource == "schema.knowledge" && change.operation == PlanOperation::Update + })); + } + + #[tokio::test] + async fn existing_lock_makes_refresh_fail() { + let dir = fixture(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + fs::write( + state_dir.join("lock.json"), + r#"{"version":1,"lock_id":"held-lock","operation":"refresh","created_at":"2026-06-08T00:00:00Z","pid":123}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(!out.ok); + assert!(out.state_observations.locked); + assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); + assert!(!out.state_observations.lock_acquired); + assert_eq!( + out.state_observations.lock_operation.as_deref(), + Some("refresh") + ); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_held") + ); + assert!(out.diagnostics.iter().any(|diagnostic| { + diagnostic.code == "state_lock_held" + && diagnostic.message.contains("force-unlock held-lock") + })); + } + + #[tokio::test] + async fn state_lock_false_bypasses_refresh_lock_with_warning() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +state: + backend: cluster + lock: false +graphs: + knowledge: + schema: ./people.pg +"#, + ) + .unwrap(); + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!out.state_observations.locked); + assert!(!out.state_observations.lock_acquired); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_disabled") + ); + } + + #[tokio::test] + async fn external_state_backend_refresh_rejected() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\nstate:\n backend: s3://bucket/state\ngraphs: {}\n", + ) + .unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "unsupported_state_backend") + ); + } + + #[tokio::test] + async fn import_graph_open_error_does_not_create_state() { + let dir = fixture(); + fs::create_dir_all(dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni")).unwrap(); + + let out = import_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "graph_observation_error") + ); + assert!(!dir.path().join(CLUSTER_STATE_FILE).exists()); + } + + // ---- config-only apply (Stage 3A) ---- + + /// Seed a state.json that simulates "graph exists with the desired schema, + /// queries/policies not yet applied" by borrowing the desired digests. + fn write_applyable_state(config_dir: &Path) { + let out = validate_config_dir(config_dir); + assert!(out.ok, "{:?}", out.diagnostics); + let schema_digest = out.resource_digests.get("schema.knowledge").unwrap().clone(); + let graph_composite = + graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); + write_state_resources( + config_dir, + &[ + ("graph.knowledge", graph_composite.as_str()), + ("schema.knowledge", schema_digest.as_str()), + ], + ); + } + + fn write_state_resources(config_dir: &Path, resources: &[(&str, &str)]) { + let resource_map: serde_json::Map<String, serde_json::Value> = resources + .iter() + .map(|(address, digest)| ((*address).to_string(), json!({ "digest": digest }))) + .collect(); + let state_dir = config_dir.join(CLUSTER_STATE_DIR); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + serde_json::to_string_pretty(&json!({ + "version": 1, + "state_revision": 1, + "applied_revision": { "resources": resource_map } + })) + .unwrap(), + ) + .unwrap(); + } + + fn read_state_json(config_dir: &Path) -> serde_json::Value { + serde_json::from_str(&fs::read_to_string(config_dir.join(CLUSTER_STATE_FILE)).unwrap()) + .unwrap() + } + + fn query_payload_path(config_dir: &Path, digest: &str) -> std::path::PathBuf { + config_dir + .join(CLUSTER_RESOURCES_DIR) + .join("query/knowledge/find_person") + .join(format!("{digest}.gq")) + } + + fn policy_payload_path(config_dir: &Path, digest: &str) -> std::path::PathBuf { + config_dir + .join(CLUSTER_RESOURCES_DIR) + .join("policy/base") + .join(format!("{digest}.yaml")) + } + + #[tokio::test] + async fn apply_without_state_fails_with_state_missing() { + let dir = fixture(); + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_missing" + && diagnostic.message.contains("cluster import")) + ); + assert!(!dir.path().join(CLUSTER_STATE_FILE).exists()); + assert!(!dir.path().join(CLUSTER_RESOURCES_DIR).exists()); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[tokio::test] + async fn apply_writes_payloads_state_and_statuses() { + let dir = fixture(); + write_applyable_state(dir.path()); + let desired = validate_config_dir(dir.path()); + let query_digest = desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap() + .clone(); + let policy_digest = desired.resource_digests.get("policy.base").unwrap().clone(); + let schema_digest = desired + .resource_digests + .get("schema.knowledge") + .unwrap() + .clone(); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(out.applied_count, 2); + assert_eq!(out.deferred_count, 0); + assert!(out.converged); + assert!(out.state_written); + + let query_blob = query_payload_path(dir.path(), &query_digest); + assert_eq!(fs::read_to_string(&query_blob).unwrap(), QUERY); + let policy_blob = policy_payload_path(dir.path(), &policy_digest); + assert_eq!(fs::read_to_string(&policy_blob).unwrap(), "rules: []\n"); + + let state = read_state_json(dir.path()); + assert_eq!(state["state_revision"], 2); + let resources = &state["applied_revision"]["resources"]; + assert_eq!( + resources["query.knowledge.find_person"]["digest"], + query_digest + ); + assert_eq!(resources["policy.base"]["digest"], policy_digest); + let expected_composite = graph_digest( + "knowledge", + Some(&schema_digest), + Some( + &[("find_person".to_string(), query_digest.clone())] + .into_iter() + .collect(), + ), + ); + assert_eq!(resources["graph.knowledge"]["digest"], expected_composite); + assert_eq!( + state["applied_revision"]["config_digest"], + desired_revision_digest(&out) + ); + assert_eq!( + state["resource_statuses"]["query.knowledge.find_person"]["status"], + "applied" + ); + assert_eq!(state["resource_statuses"]["policy.base"]["status"], "applied"); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + fn desired_revision_digest(out: &ApplyOutput) -> String { + out.desired_revision.config_digest.clone().unwrap() + } + + #[tokio::test] + async fn apply_update_changes_query_digest_and_keeps_old_blob() { + let dir = fixture(); + let desired = validate_config_dir(dir.path()); + let schema_digest = desired + .resource_digests + .get("schema.knowledge") + .unwrap() + .clone(); + let old_digest = "0".repeat(64); + let graph_composite = + graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", graph_composite.as_str()), + ("schema.knowledge", schema_digest.as_str()), + ("query.knowledge.find_person", old_digest.as_str()), + ], + ); + let old_blob = query_payload_path(dir.path(), &old_digest); + fs::create_dir_all(old_blob.parent().unwrap()).unwrap(); + fs::write(&old_blob, "old query source").unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + let new_digest = desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap(); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["query.knowledge.find_person"]["digest"], + *new_digest + ); + assert_eq!(fs::read_to_string(&old_blob).unwrap(), "old query source"); + assert!(query_payload_path(dir.path(), new_digest).exists()); + } + + #[tokio::test] + async fn apply_deletes_removed_resources_but_keeps_blobs() { + let dir = fixture(); + let desired = validate_config_dir(dir.path()); + let schema_digest = desired + .resource_digests + .get("schema.knowledge") + .unwrap() + .clone(); + let stale_query_digest = "1".repeat(64); + let stale_policy_digest = "2".repeat(64); + let graph_composite = + graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", graph_composite.as_str()), + ("schema.knowledge", schema_digest.as_str()), + ("query.knowledge.orphan", stale_query_digest.as_str()), + ("policy.old", stale_policy_digest.as_str()), + ], + ); + let stale_blob = dir + .path() + .join(CLUSTER_RESOURCES_DIR) + .join("policy/old") + .join(format!("{stale_policy_digest}.yaml")); + fs::create_dir_all(stale_blob.parent().unwrap()).unwrap(); + fs::write(&stale_blob, "old policy").unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.converged); + let state = read_state_json(dir.path()); + let resources = &state["applied_revision"]["resources"]; + assert!(resources.get("query.knowledge.orphan").is_none()); + assert!(resources.get("policy.old").is_none()); + assert!( + state["resource_statuses"] + .get("query.knowledge.orphan") + .is_none() + ); + // Deleted resources leave their content-addressed blobs in place; GC is + // a later stage. + assert_eq!(fs::read_to_string(&stale_blob).unwrap(), "old policy"); + // The composite no longer includes the orphan query. + let query_digest = desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap() + .clone(); + let expected_composite = graph_digest( + "knowledge", + Some(&schema_digest), + Some(&[("find_person".to_string(), query_digest)].into_iter().collect()), + ); + assert_eq!(resources["graph.knowledge"]["digest"], expected_composite); + } + + #[tokio::test] + async fn apply_schema_update_and_dependent_query_in_one_run() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + // Schema update + a query update that depends on the new field: one + // apply executes the schema migration first, then the catalog write. + fs::write(dir.path().join("people.pg"), SCHEMA_V2).unwrap(); + fs::write( + dir.path().join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name, $p.bio }\n}\n", + ) + .unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.converged, "{out:?}"); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + assert_eq!( + by_resource["schema.knowledge"].disposition, + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["query.knowledge.find_person"].disposition, + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["graph.knowledge"].disposition, + Some(ApplyDisposition::Derived) + ); + // The live graph carries the new schema. + let db = Omnigraph::open_read_only(&derived_graph_uri(dir.path(), "knowledge")) + .await + .unwrap(); + let desired = validate_config_dir(dir.path()); + assert_eq!( + sha256_hex(db.schema_source().as_bytes()), + desired.resource_digests["schema.knowledge"] + ); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + desired.resource_digests["schema.knowledge"] + ); + // Sidecar retired after the CAS landed. + assert!( + !dir.path().join(CLUSTER_RECOVERIES_DIR).exists() + || fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) + .unwrap() + .next() + .is_none() + ); + } + + #[tokio::test] + async fn apply_unsupported_schema_change_fails_loudly() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + // Property type changes are unsupported by the engine planner. + fs::write( + dir.path().join("people.pg"), + "\nnode Person {\n name: String @key\n age: I64?\n}\n", + ) + .unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!(out.diagnostics.iter().any(|diagnostic| { + diagnostic.code == "schema_apply_failed" + && diagnostic.message.contains("changing property type") + })); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + assert_eq!( + by_resource["schema.knowledge"].disposition, + Some(ApplyDisposition::Blocked) + ); + assert_eq!( + by_resource["schema.knowledge"].reason.as_deref(), + Some("schema_apply_failed") + ); + // The live schema and the ledger are unchanged. + let state = read_state_json(dir.path()); + let desired = validate_config_dir(dir.path()); + assert_ne!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + desired.resource_digests["schema.knowledge"] + ); + // Second run: the sweep retires the stale sidecar (ledger consistent) + // and the run fails just as loudly — idempotent loudness. + let second = apply_config_dir(dir.path()).await; + assert!(!second.ok); + assert!( + second + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "schema_apply_failed") + ); + } + + #[tokio::test] + async fn apply_blocks_schema_update_while_recovery_pending() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_state_resources(dir.path(), &[("schema.knowledge", "stale-digest")]); + fs::write(dir.path().join("people.pg"), SCHEMA_V2).unwrap(); + // A pending sidecar whose intent matches neither live nor recorded. + write_schema_apply_sidecar(dir.path(), "knowledge", "intended-digest", "01PENDS"); + + let out = apply_config_dir(dir.path()).await; + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + assert_eq!( + by_resource["schema.knowledge"].disposition, + Some(ApplyDisposition::Blocked) + ); + assert_eq!( + by_resource["schema.knowledge"].reason.as_deref(), + Some("cluster_recovery_pending") + ); + } + + #[tokio::test] + async fn apply_creates_graph_and_unblocks_dependents() { + let dir = fixture(); + write_state_resources(dir.path(), &[]); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.converged, "{out:?}"); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + // Stage 4A: the create executes, and its dependents apply in-run. + assert_eq!( + by_resource["graph.knowledge"].disposition, + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["schema.knowledge"].disposition, + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["query.knowledge.find_person"].disposition, + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["policy.base"].disposition, + Some(ApplyDisposition::Applied) + ); + // The graph exists on disk and opens; state records everything. + let graph_uri = derived_graph_uri(dir.path(), "knowledge"); + let db = Omnigraph::open_read_only(&graph_uri).await.unwrap(); + let desired = validate_config_dir(dir.path()); + assert_eq!( + sha256_hex(db.schema_source().as_bytes()), + desired.resource_digests["schema.knowledge"] + ); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + desired.resource_digests["schema.knowledge"] + ); + assert_eq!( + state["resource_statuses"]["graph.knowledge"]["status"], + "applied" + ); + // The create's sidecar was retired after the state CAS landed. + assert!( + !dir.path().join(CLUSTER_RECOVERIES_DIR).exists() + || fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) + .unwrap() + .next() + .is_none() + ); + } + + #[tokio::test] + async fn apply_create_failure_blocks_dependents_and_keeps_sidecar() { + let dir = fixture(); + write_state_resources(dir.path(), &[]); + // Make the init fail its strict preflight: a junk _schema.pg already + // sits at the derived root (the engine refuses to overwrite it). + let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); + fs::create_dir_all(&root).unwrap(); + fs::write(root.join("_schema.pg"), "junk").unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "graph_create_failed") + ); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + // Dependents are demoted: the run tells the truth about what executed. + assert_eq!( + by_resource["graph.knowledge"].disposition, + Some(ApplyDisposition::Blocked) + ); + assert_eq!( + by_resource["query.knowledge.find_person"].disposition, + Some(ApplyDisposition::Blocked) + ); + assert_eq!( + by_resource["query.knowledge.find_person"].reason.as_deref(), + Some("dependency_not_applied") + ); + assert_eq!( + by_resource["policy.base"].disposition, + Some(ApplyDisposition::Blocked) + ); + assert!(!out.converged); + // The sidecar stays for the sweep to classify next run. + assert!( + fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) + .unwrap() + .next() + .is_some() + ); + // No graph digests moved. + let state = read_state_json(dir.path()); + assert!( + state["applied_revision"]["resources"] + .as_object() + .unwrap() + .is_empty() + ); + } + + #[tokio::test] + async fn apply_blocks_graph_delete_without_approval() { + let dir = fixture(); + let desired = validate_config_dir(dir.path()); + let schema_digest = desired + .resource_digests + .get("schema.knowledge") + .unwrap() + .clone(); + let graph_composite = + graph_digest("knowledge", Some(&schema_digest), Some(&BTreeMap::new())); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", graph_composite.as_str()), + ("schema.knowledge", schema_digest.as_str()), + ("graph.old", "3333"), + ("schema.old", "4444"), + ("query.old.q", "5555"), + ], + ); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!out.converged); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + // Stage 4C: deletes are gated, not deferred — every subtree change + // blocks on the single graph-level approval. + assert_eq!( + by_resource["graph.old"].disposition, + Some(ApplyDisposition::Blocked) + ); + assert_eq!( + by_resource["graph.old"].reason.as_deref(), + Some("approval_required") + ); + assert_eq!( + by_resource["schema.old"].reason.as_deref(), + Some("approval_required") + ); + assert_eq!( + by_resource["query.old.q"].reason.as_deref(), + Some("approval_required") + ); + // State intact; nothing destroyed without the artifact. + let state = read_state_json(dir.path()); + let resources = &state["applied_revision"]["resources"]; + assert_eq!(resources["graph.old"]["digest"], "3333"); + assert_eq!(resources["schema.old"]["digest"], "4444"); + assert_eq!(resources["query.old.q"]["digest"], "5555"); + } + + #[tokio::test] + async fn approve_writes_digest_bound_artifact() { + let dir = fixture(); + write_applyable_state(dir.path()); + // Seed a deletable subtree. + let state = read_state_json(dir.path()); + let graph_digest_str = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + let schema_digest_str = state["applied_revision"]["resources"]["schema.knowledge"] + ["digest"] + .as_str() + .unwrap() + .to_string(); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", graph_digest_str.as_str()), + ("schema.knowledge", schema_digest_str.as_str()), + ("graph.old", "3333"), + ("schema.old", "4444"), + ], + ); + + let out = approve_config_dir(dir.path(), "graph.old", "andrew").await; + assert!(out.ok, "{:?}", out.diagnostics); + let approval_id = out.approval_id.clone().unwrap(); + let artifact: serde_json::Value = serde_json::from_str( + &fs::read_to_string( + dir.path() + .join(CLUSTER_APPROVALS_DIR) + .join(format!("{approval_id}.json")), + ) + .unwrap(), + ) + .unwrap(); + assert_eq!(artifact["resource"], "graph.old"); + assert_eq!(artifact["operation"], "delete"); + assert_eq!(artifact["approved_by"], "andrew"); + assert_eq!(artifact["bound_before_digest"], "3333"); + assert!(artifact["bound_after_digest"].is_null()); + assert!(artifact["bound_config_digest"].is_string()); + assert!(artifact["consumed_at"].is_null()); + + // A non-gated address is refused. + let not_gated = approve_config_dir(dir.path(), "query.knowledge.find_person", "andrew").await; + assert!(!not_gated.ok); + assert!( + not_gated + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "approval_not_required") + ); + } + + #[tokio::test] + async fn stale_approval_is_ignored() { + let dir = fixture(); + write_applyable_state(dir.path()); + let state = read_state_json(dir.path()); + let graph_digest_str = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + let schema_digest_str = state["applied_revision"]["resources"]["schema.knowledge"] + ["digest"] + .as_str() + .unwrap() + .to_string(); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", graph_digest_str.as_str()), + ("schema.knowledge", schema_digest_str.as_str()), + ("graph.old", "3333"), + ], + ); + let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; + assert!(approved.ok, "{:?}", approved.diagnostics); + // The config moves after approval: the bound config digest no longer + // matches and the artifact authorizes nothing. + fs::write(dir.path().join("base.policy.yaml"), "rules: [] # moved\n").unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "approval_stale"), + "{:?}", + out.diagnostics + ); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + assert_eq!( + by_resource["graph.old"].reason.as_deref(), + Some("approval_required") + ); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["graph.old"]["digest"], + "3333" + ); + } + + #[tokio::test] + async fn compute_approvals_one_gate_per_subtree() { + let dir = fixture(); + write_applyable_state(dir.path()); + let state = read_state_json(dir.path()); + let g = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + let sc = state["applied_revision"]["resources"]["schema.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + write_state_resources( + dir.path(), + &[ + ("graph.knowledge", g.as_str()), + ("schema.knowledge", sc.as_str()), + ("graph.old", "3333"), + ("schema.old", "4444"), + ("query.old.q", "5555"), + ], + ); + let plan = plan_config_dir(dir.path()).await; + let gated: Vec<&str> = plan + .approvals_required + .iter() + .map(|gate| gate.resource.as_str()) + .collect(); + assert_eq!(gated, vec!["graph.old"], "{plan:?}"); + assert!(!plan.approvals_required[0].satisfied); + } + + #[tokio::test] + async fn apply_is_idempotent() { + let dir = fixture(); + write_applyable_state(dir.path()); + + let first = apply_config_dir(dir.path()).await; + assert!(first.ok, "{:?}", first.diagnostics); + assert!(first.state_written); + let state_after_first = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); + + let second = apply_config_dir(dir.path()).await; + assert!(second.ok, "{:?}", second.diagnostics); + assert!(second.changes.is_empty()); + assert_eq!(second.applied_count, 0); + assert!(second.converged); + assert!(!second.state_written); + let state_after_second = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); + assert_eq!(state_after_first, state_after_second); + assert_eq!(second.state_observations.state_revision, 2); + } + + #[tokio::test] + async fn apply_respects_held_lock() { + let dir = fixture(); + write_applyable_state(dir.path()); + write_lock_file(dir.path(), "held-lock", "plan"); + + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_held") + ); + // The held lock survives a refused apply, and nothing was written. + assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); + assert!(!dir.path().join(CLUSTER_RESOURCES_DIR).exists()); + let state = read_state_json(dir.path()); + assert_eq!(state["state_revision"], 1); + } + + #[tokio::test] + async fn apply_state_lock_false_bypasses_with_warning() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +state: + backend: cluster + lock: false +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +"#, + ) + .unwrap(); + write_applyable_state(dir.path()); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.state_written); + assert!(!out.state_observations.lock_acquired); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_lock_disabled") + ); + assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); + } + + #[tokio::test] + async fn apply_skips_existing_payload_blob() { + let dir = fixture(); + write_applyable_state(dir.path()); + let desired = validate_config_dir(dir.path()); + let query_digest = desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap() + .clone(); + // Content-addressed blobs are trusted by name: an existing file is + // never rewritten. + let blob = query_payload_path(dir.path(), &query_digest); + fs::create_dir_all(blob.parent().unwrap()).unwrap(); + fs::write(&blob, "pre-existing").unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert_eq!(fs::read_to_string(&blob).unwrap(), "pre-existing"); + } + + #[tokio::test] + async fn apply_invalid_config_fails_before_lock() { + let dir = fixture(); + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + "version: 1\nnot_a_field: true\n", + ) + .unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + // Config errors bail before the lock or any state directory exists. + assert!(!dir.path().join(CLUSTER_STATE_DIR).exists()); + } + + /// When the state write fails after payloads landed, the output must + /// report the statuses actually on disk — not the unpersisted in-memory + /// mutations (phantom `applied` entries would mislead automation that + /// reads `resource_statuses` independently of `ok`). + #[cfg(unix)] + #[tokio::test] + async fn apply_state_write_failure_reports_persisted_statuses() { + use std::os::unix::fs::PermissionsExt; + + let dir = fixture(); + // lock: false so the only write into __cluster/ is state.json itself. + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +state: + backend: cluster + lock: false +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +"#, + ) + .unwrap(); + write_applyable_state(dir.path()); + // Pre-create the payload blob so the payload phase is a no-op and the + // failure lands exactly at the state write. + let desired = validate_config_dir(dir.path()); + let query_digest = desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap(); + let blob = query_payload_path(dir.path(), query_digest); + fs::create_dir_all(blob.parent().unwrap()).unwrap(); + fs::write(&blob, QUERY).unwrap(); + + let state_dir = dir.path().join(CLUSTER_STATE_DIR); + fs::set_permissions(&state_dir, fs::Permissions::from_mode(0o555)).unwrap(); + // Running as root ignores permission bits; skip rather than flake. + if fs::write(state_dir.join("probe"), b"x").is_ok() { + let _ = fs::remove_file(state_dir.join("probe")); + fs::set_permissions(&state_dir, fs::Permissions::from_mode(0o755)).unwrap(); + eprintln!("skipping: permissions are not enforced (running as root)"); + return; + } + + let out = apply_config_dir(dir.path()).await; + fs::set_permissions(&state_dir, fs::Permissions::from_mode(0o755)).unwrap(); + + assert!(!out.ok); + assert!(!out.state_written); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "state_write_error"), + "{:?}", + out.diagnostics + ); + // The seeded state has no statuses; the failed apply must not invent + // the in-memory `applied` ones it failed to persist. + assert!( + out.resource_statuses.is_empty(), + "unpersisted statuses leaked into output: {:?}", + out.resource_statuses + ); + } + + // ---- catalog payload verification (Stage 3B) ---- + + /// Converge a fixture dir and return the query blob path. + async fn converge_fixture(config_dir: &Path) -> std::path::PathBuf { + write_applyable_state(config_dir); + let out = apply_config_dir(config_dir).await; + assert!(out.ok && out.converged, "{:?}", out.diagnostics); + let desired = validate_config_dir(config_dir); + query_payload_path( + config_dir, + desired + .resource_digests + .get("query.knowledge.find_person") + .unwrap(), + ) + } + + #[tokio::test] + async fn status_reports_missing_payload_read_only() { + let dir = fixture(); + let blob = converge_fixture(dir.path()).await; + let state_before = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); + fs::remove_file(&blob).unwrap(); + + let out = status_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.diagnostics.iter().any(|diagnostic| { + diagnostic.code == "catalog_payload_missing" + && diagnostic.path == "query.knowledge.find_person" + })); + // Read-only: persisted statuses and state bytes untouched. + assert_eq!( + out.resource_statuses["query.knowledge.find_person"].status, + ResourceLifecycleStatus::Applied + ); + assert_eq!( + fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(), + state_before + ); + } + + #[tokio::test] + async fn refresh_removes_digest_and_drifts_on_missing_payload() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let blob = converge_fixture(dir.path()).await; + fs::remove_file(&blob).unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "catalog_payload_missing") + ); + let status = &out.resource_statuses["query.knowledge.find_person"]; + assert_eq!(status.status, ResourceLifecycleStatus::Drifted); + assert!(status.conditions.contains(&"payload_missing".to_string())); + let state = read_state_json(dir.path()); + assert!( + state["applied_revision"]["resources"] + .get("query.knowledge.find_person") + .is_none(), + "{state}" + ); + } + + #[tokio::test] + async fn refresh_drifts_on_corrupted_payload() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let blob = converge_fixture(dir.path()).await; + fs::write(&blob, "corrupted content").unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + let status = &out.resource_statuses["query.knowledge.find_person"]; + assert_eq!(status.status, ResourceLifecycleStatus::Drifted); + assert!(status.conditions.contains(&"payload_mismatch".to_string())); + let state = read_state_json(dir.path()); + assert!( + state["applied_revision"]["resources"] + .get("query.knowledge.find_person") + .is_none() + ); + } + + #[tokio::test] + async fn refresh_flags_unreadable_payload_as_error() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let blob = converge_fixture(dir.path()).await; + // A same-named directory yields a non-NotFound IO error portably. + fs::remove_file(&blob).unwrap(); + fs::create_dir(&blob).unwrap(); + + let out = refresh_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "catalog_payload_read_error") + ); + let status = &out.resource_statuses["query.knowledge.find_person"]; + assert_eq!(status.status, ResourceLifecycleStatus::Error); + assert!(status.conditions.contains(&"payload_read_error".to_string())); + // Transient IO keeps the digest: no spurious republish. + let state = read_state_json(dir.path()); + assert!( + state["applied_revision"]["resources"] + .get("query.knowledge.find_person") + .is_some() + ); + } + + #[tokio::test] + async fn payload_drift_self_heals_through_refresh_plan_apply() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + let blob = converge_fixture(dir.path()).await; + let original = fs::read_to_string(&blob).unwrap(); + fs::remove_file(&blob).unwrap(); + + let refresh = refresh_config_dir(dir.path()).await; + assert!(refresh.ok, "{:?}", refresh.diagnostics); + + let plan = plan_config_dir(dir.path()).await; + let query_change = plan + .changes + .iter() + .find(|change| change.resource == "query.knowledge.find_person") + .expect("plan must propose recreating the query"); + assert_eq!(query_change.operation, PlanOperation::Create); + assert_eq!(query_change.disposition, Some(ApplyDisposition::Applied)); + + let apply = apply_config_dir(dir.path()).await; + assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); + assert_eq!(fs::read_to_string(&blob).unwrap(), original); + + let status = status_config_dir(dir.path()); + assert!( + !status + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code.starts_with("catalog_payload")), + "{:?}", + status.diagnostics + ); + } + + #[test] + fn verification_skips_graph_and_schema_resources() { + let dir = fixture(); + write_applyable_state(dir.path()); // graph + schema digests only, no blobs + + let out = status_config_dir(dir.path()); + assert!( + !out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code.starts_with("catalog_payload")), + "{:?}", + out.diagnostics + ); + } + + // ---- recovery sidecars + sweep (Stage 4A) ---- + + fn derived_graph_uri(config_dir: &Path, graph_id: &str) -> String { + display_path( + &config_dir + .join(CLUSTER_GRAPHS_DIR) + .join(format!("{graph_id}.omni")), + ) + } + + fn write_create_sidecar( + config_dir: &Path, + graph_id: &str, + desired_schema_digest: &str, + operation_id: &str, + ) -> PathBuf { + let dir = config_dir.join(CLUSTER_RECOVERIES_DIR); + fs::create_dir_all(&dir).unwrap(); + let path = dir.join(format!("{operation_id}.json")); + fs::write( + &path, + serde_json::to_string_pretty(&json!({ + "schema_version": 1, + "operation_id": operation_id, + "started_at": "1970-01-01T00:00:00Z", + "kind": "graph_create", + "graph_id": graph_id, + "graph_uri": derived_graph_uri(config_dir, graph_id), + "desired_schema_digest": desired_schema_digest, + })) + .unwrap(), + ) + .unwrap(); + path + } + + #[tokio::test] + async fn sweep_removes_sidecar_when_root_absent() { + let dir = fixture(); + write_applyable_state(dir.path()); + let sidecar = write_create_sidecar(dir.path(), "knowledge", "irrelevant", "01ROW1"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + // Row 1: nothing moved; intent removed, run proceeds normally. + assert!(!sidecar.exists()); + assert!(out.converged); + } + + #[tokio::test] + async fn sweep_rolls_forward_completed_create() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_state_resources(dir.path(), &[]); // state predates the create + let desired = validate_config_dir(dir.path()); + let schema_digest = desired.resource_digests["schema.knowledge"].clone(); + let sidecar = write_create_sidecar(dir.path(), "knowledge", &schema_digest, "01ROW4"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") + ); + // Row 4: ledger converged to observable reality, audit recorded, + // sidecar retired after the CAS landed. + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + schema_digest + ); + assert!( + state["recovery_records"] + .as_object() + .unwrap() + .values() + .any(|record| record["outcome"] == "rolled_forward" + && record["graph_id"] == "knowledge") + ); + assert!(!sidecar.exists()); + // With the graph rolled forward, the same run converges the catalog. + assert!(out.converged, "{out:?}"); + } + + #[tokio::test] + async fn sweep_completes_already_recorded_create() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); // state already records graph+schema + let desired = validate_config_dir(dir.path()); + let sidecar = write_create_sidecar( + dir.path(), + "knowledge", + &desired.resource_digests["schema.knowledge"], + "01ROW2", + ); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + // Row 2: outcome was already durable; no audit entry, sidecar retired. + assert!(!sidecar.exists()); + let state = read_state_json(dir.path()); + assert!( + state["recovery_records"] + .as_object() + .is_none_or(|records| records.is_empty()), + "{state}" + ); + } + + #[tokio::test] + async fn sweep_keeps_sidecar_for_incomplete_root() { + let dir = fixture(); + write_applyable_state(dir.path()); + // A root that exists but cannot be opened: the engine's partial-init gap. + let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); + fs::create_dir_all(&root).unwrap(); + fs::write(root.join("_schema.pg"), "junk").unwrap(); + let sidecar = write_create_sidecar(dir.path(), "knowledge", "whatever", "01ROW5"); + + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "graph_create_incomplete") + ); + // Row 5: never auto-delete; sidecar and root stay for the operator, + // and the Error status is persisted by the run's state write. + assert!(sidecar.exists()); + assert!(root.exists()); + let state = read_state_json(dir.path()); + assert_eq!(state["resource_statuses"]["graph.knowledge"]["status"], "error"); + assert!( + state["resource_statuses"]["graph.knowledge"]["conditions"] + .as_array() + .unwrap() + .iter() + .any(|condition| condition == "graph_create_incomplete") + ); + } + + #[tokio::test] + async fn sweep_flags_unexpected_schema_as_pending() { + let dir = fixture(); + write_state_resources(dir.path(), &[]); + // Live graph exists with a schema the sidecar never intended. + let graph_dir = dir.path().join(CLUSTER_GRAPHS_DIR); + fs::create_dir_all(&graph_dir).unwrap(); + Omnigraph::init( + &derived_graph_uri(dir.path(), "knowledge"), + "\nnode Other {\n name: String @key\n}\n", + ) + .await + .unwrap(); + let desired = validate_config_dir(dir.path()); + let sidecar = write_create_sidecar( + dir.path(), + "knowledge", + &desired.resource_digests["schema.knowledge"], + "01ROW6", + ); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); // warning, not error + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_pending") + ); + // Row 6: refuse to guess; sidecar kept, Drifted persisted. + assert!(sidecar.exists()); + let state = read_state_json(dir.path()); + assert_eq!( + state["resource_statuses"]["graph.knowledge"]["status"], + "drifted" + ); + assert!( + state["resource_statuses"]["graph.knowledge"]["conditions"] + .as_array() + .unwrap() + .iter() + .any(|condition| condition == "actual_applied_state_pending") + ); + } + + #[tokio::test] + async fn apply_blocks_create_while_recovery_pending() { + let dir = fixture(); + write_state_resources(dir.path(), &[]); + // A kept (row 5) sidecar: partial root that cannot be opened. + let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); + fs::create_dir_all(&root).unwrap(); + fs::write(root.join("_schema.pg"), "junk").unwrap(); + let sidecar = write_create_sidecar(dir.path(), "knowledge", "whatever", "01PEND"); + + let out = apply_config_dir(dir.path()).await; + assert!(!out.ok); // row 5 is an error condition + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + // The pending recovery blocks the create and its dependents; the + // executor never attempts the init. + assert_eq!( + by_resource["graph.knowledge"].disposition, + Some(ApplyDisposition::Blocked) + ); + assert_eq!( + by_resource["graph.knowledge"].reason.as_deref(), + Some("cluster_recovery_pending") + ); + assert_eq!( + by_resource["query.knowledge.find_person"].reason.as_deref(), + Some("cluster_recovery_pending") + ); + assert_eq!( + by_resource["policy.base"].reason.as_deref(), + Some("cluster_recovery_pending") + ); + assert!(sidecar.exists()); + // The sweep's Error status is what persists — not a generic Blocked. + let state = read_state_json(dir.path()); + assert_eq!(state["resource_statuses"]["graph.knowledge"]["status"], "error"); + } + + #[tokio::test] + async fn plan_embeds_migration_preview_for_schema_update() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + fs::write( + dir.path().join("people.pg"), + "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n", + ) + .unwrap(); + + let out = plan_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + let schema_change = out + .changes + .iter() + .find(|change| change.resource == "schema.knowledge") + .unwrap(); + let migration = schema_change.migration.as_ref().expect("preview embedded"); + assert!(migration.supported); + assert!( + serde_json::to_string(&migration.steps) + .unwrap() + .contains("add_property"), + "{migration:?}" + ); + } + + #[tokio::test] + async fn plan_warns_when_preview_unavailable() { + let dir = fixture(); + write_applyable_state(dir.path()); // digests recorded, but no live root + fs::write( + dir.path().join("people.pg"), + "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n", + ) + .unwrap(); + + let out = plan_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + let schema_change = out + .changes + .iter() + .find(|change| change.resource == "schema.knowledge") + .unwrap(); + assert!(schema_change.migration.is_none()); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "schema_preview_unavailable") + ); + } + + fn write_schema_apply_sidecar( + config_dir: &Path, + graph_id: &str, + desired_schema_digest: &str, + operation_id: &str, + ) -> PathBuf { + let dir = config_dir.join(CLUSTER_RECOVERIES_DIR); + fs::create_dir_all(&dir).unwrap(); + let path = dir.join(format!("{operation_id}.json")); + fs::write( + &path, + serde_json::to_string_pretty(&json!({ + "schema_version": 1, + "operation_id": operation_id, + "started_at": "1970-01-01T00:00:00Z", + "kind": "schema_apply", + "graph_id": graph_id, + "graph_uri": derived_graph_uri(config_dir, graph_id), + "desired_schema_digest": desired_schema_digest, + })) + .unwrap(), + ) + .unwrap(); + path + } + + const SCHEMA_V2: &str = "\nnode Person {\n name: String @key\n age: I32?\n bio: String?\n}\n"; + + #[tokio::test] + async fn sweep_retires_schema_sidecar_when_ledger_consistent() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); // state digest == live digest + let sidecar = + write_schema_apply_sidecar(dir.path(), "knowledge", "never-applied", "01SROW1"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!sidecar.exists()); + let state = read_state_json(dir.path()); + assert!( + state["recovery_records"] + .as_object() + .is_none_or(|records| records.is_empty()) + ); + } + + #[tokio::test] + async fn sweep_rolls_forward_completed_schema_apply() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + // The schema apply completed on the graph out-of-process... + let graph_uri = derived_graph_uri(dir.path(), "knowledge"); + let db = Omnigraph::open(&graph_uri).await.unwrap(); + db.apply_schema(SCHEMA_V2).await.unwrap(); + // ...the desired config matches it, and the sidecar records the intent. + fs::write(dir.path().join("people.pg"), SCHEMA_V2).unwrap(); + let desired = validate_config_dir(dir.path()); + let v2_digest = desired.resource_digests["schema.knowledge"].clone(); + let sidecar = write_schema_apply_sidecar(dir.path(), "knowledge", &v2_digest, "01SROW3"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") + ); + assert!(!sidecar.exists()); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["schema.knowledge"]["digest"], + v2_digest + ); + assert!( + state["recovery_records"] + .as_object() + .unwrap() + .values() + .any(|record| record["kind"] == "schema_apply" + && record["outcome"] == "rolled_forward") + ); + assert!(out.converged, "{out:?}"); + } + + #[tokio::test] + async fn sweep_flags_unexpected_schema_apply_state_as_pending() { + let dir = fixture(); + init_derived_graph(dir.path()).await; // live = v1 + write_state_resources(dir.path(), &[("schema.knowledge", "stale-digest")]); + // Sidecar intended a digest that is neither live nor recorded. + let sidecar = + write_schema_apply_sidecar(dir.path(), "knowledge", "intended-digest", "01SROW6"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); // warnings only + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_pending") + ); + assert!(sidecar.exists()); + let state = read_state_json(dir.path()); + assert_eq!( + state["resource_statuses"]["schema.knowledge"]["status"], + "drifted" + ); + } + + #[tokio::test] + async fn sweep_keeps_schema_sidecar_for_unopenable_root() { + let dir = fixture(); + write_applyable_state(dir.path()); + let root = dir.path().join(CLUSTER_GRAPHS_DIR).join("knowledge.omni"); + fs::create_dir_all(&root).unwrap(); // exists, won't open + let sidecar = + write_schema_apply_sidecar(dir.path(), "knowledge", "whatever", "01SROWX"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); // warning: cannot verify + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_pending") + ); + assert!(sidecar.exists()); + } + + /// Seed: converged knowledge subtree + a stale `old` graph subtree with a + /// real directory on disk. + fn seed_deletable_state(config_dir: &Path) { + write_applyable_state(config_dir); + let state = read_state_json(config_dir); + let g = state["applied_revision"]["resources"]["graph.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + let sc = state["applied_revision"]["resources"]["schema.knowledge"]["digest"] + .as_str() + .unwrap() + .to_string(); + write_state_resources( + config_dir, + &[ + ("graph.knowledge", g.as_str()), + ("schema.knowledge", sc.as_str()), + ("graph.old", "3333"), + ("schema.old", "4444"), + ("query.old.q", "5555"), + ], + ); + let root = config_dir.join(CLUSTER_GRAPHS_DIR).join("old.omni"); + fs::create_dir_all(&root).unwrap(); + fs::write(root.join("_schema.pg"), "stale").unwrap(); + } + + #[tokio::test] + async fn apply_executes_approved_graph_delete() { + let dir = fixture(); + seed_deletable_state(dir.path()); + let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; + assert!(approved.ok, "{:?}", approved.diagnostics); + let approval_id = approved.approval_id.clone().unwrap(); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.converged, "{out:?}"); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + assert_eq!(by_resource["graph.old"].disposition, Some(ApplyDisposition::Applied)); + assert_eq!(by_resource["schema.old"].disposition, Some(ApplyDisposition::Applied)); + assert_eq!(by_resource["query.old.q"].disposition, Some(ApplyDisposition::Applied)); + // The root is gone; the subtree is tombstoned out of the ledger. + assert!(!dir.path().join(CLUSTER_GRAPHS_DIR).join("old.omni").exists()); + let state = read_state_json(dir.path()); + let resources = state["applied_revision"]["resources"].as_object().unwrap(); + assert!(!resources.contains_key("graph.old")); + assert!(!resources.contains_key("schema.old")); + assert!(!resources.contains_key("query.old.q")); + assert_eq!(state["observations"]["graph.old"]["kind"], "tombstone"); + assert_eq!(state["observations"]["graph.old"]["approval_id"], approval_id); + // Approval consumed in BOTH stores: ledger summary + artifact file. + assert!(state["approval_records"][&approval_id]["consumed_at"].is_string()); + let artifact: serde_json::Value = serde_json::from_str( + &fs::read_to_string( + dir.path() + .join(CLUSTER_APPROVALS_DIR) + .join(format!("{approval_id}.json")), + ) + .unwrap(), + ) + .unwrap(); + assert!(artifact["consumed_at"].is_string(), "{artifact}"); + // Sidecar retired. + assert!( + fs::read_dir(dir.path().join(CLUSTER_RECOVERIES_DIR)) + .map(|mut entries| entries.next().is_none()) + .unwrap_or(true) + ); + // A consumed approval authorizes nothing further (idempotent re-apply). + let again = apply_config_dir(dir.path()).await; + assert!(again.ok && again.converged && !again.state_written, "{again:?}"); + } + + fn write_delete_sidecar( + config_dir: &Path, + graph_id: &str, + approval_id: Option<&str>, + operation_id: &str, + ) -> PathBuf { + let dir = config_dir.join(CLUSTER_RECOVERIES_DIR); + fs::create_dir_all(&dir).unwrap(); + let path = dir.join(format!("{operation_id}.json")); + fs::write( + &path, + serde_json::to_string_pretty(&json!({ + "schema_version": 1, + "operation_id": operation_id, + "started_at": "1970-01-01T00:00:00Z", + "kind": "graph_delete", + "graph_id": graph_id, + "graph_uri": derived_graph_uri(config_dir, graph_id), + "desired_schema_digest": "", + "approval_id": approval_id, + })) + .unwrap(), + ) + .unwrap(); + path + } + + #[tokio::test] + async fn sweep_retires_delete_sidecar_when_tombstoned() { + let dir = fixture(); + write_applyable_state(dir.path()); // no graph.old in state, no root + let sidecar = write_delete_sidecar(dir.path(), "old", None, "01DROW7"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!(!sidecar.exists()); + let state = read_state_json(dir.path()); + assert!( + state["recovery_records"] + .as_object() + .is_none_or(|records| records.is_empty()) + ); + } + + #[tokio::test] + async fn sweep_rolls_forward_completed_delete() { + let dir = fixture(); + seed_deletable_state(dir.path()); + // Approve, then simulate: root removed, state stale, sidecar present. + let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; + let approval_id = approved.approval_id.unwrap(); + fs::remove_dir_all(dir.path().join(CLUSTER_GRAPHS_DIR).join("old.omni")).unwrap(); + let sidecar = write_delete_sidecar(dir.path(), "old", Some(&approval_id), "01DROW7B"); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_rolled_forward") + ); + assert!(!sidecar.exists()); + let state = read_state_json(dir.path()); + assert!( + !state["applied_revision"]["resources"] + .as_object() + .unwrap() + .contains_key("graph.old") + ); + assert_eq!(state["observations"]["graph.old"]["kind"], "tombstone"); + assert!(state["approval_records"][&approval_id]["consumed_at"].is_string()); + assert!( + state["recovery_records"] + .as_object() + .unwrap() + .values() + .any(|record| record["kind"] == "graph_delete" + && record["outcome"] == "rolled_forward") + ); + // The artifact file is marked consumed post-CAS. + let artifact: serde_json::Value = serde_json::from_str( + &fs::read_to_string( + dir.path() + .join(CLUSTER_APPROVALS_DIR) + .join(format!("{approval_id}.json")), + ) + .unwrap(), + ) + .unwrap(); + assert!(artifact["consumed_at"].is_string()); + assert!(out.converged, "{out:?}"); + } + + #[tokio::test] + async fn sweep_reproposes_incomplete_delete() { + let dir = fixture(); + seed_deletable_state(dir.path()); // root present + let approved = approve_config_dir(dir.path(), "graph.old", "andrew").await; + assert!(approved.ok); + let sidecar = write_delete_sidecar(dir.path(), "old", approved.approval_id.as_deref(), "01DROW8"); + + // Row 8: the stale intent is retired with a warning, and the same run + // re-executes the still-approved delete to completion. + let out = apply_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "graph_delete_incomplete") + ); + assert!(!sidecar.exists()); + assert!(!dir.path().join(CLUSTER_GRAPHS_DIR).join("old.omni").exists()); + assert!(out.converged, "{out:?}"); + } + + // ---- policy bindings in the applied revision (5A) ---- + + #[tokio::test] + async fn apply_records_policy_bindings() { + let dir = fixture(); + write_applyable_state(dir.path()); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok && out.converged, "{:?}", out.diagnostics); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["policy.base"]["applies_to"], + serde_json::json!(["graph.knowledge"]), + "{state}" + ); + // Non-policy entries carry no bindings field at all. + assert!( + state["applied_revision"]["resources"]["query.knowledge.find_person"] + .get("applies_to") + .is_none() + ); + } + + #[tokio::test] + async fn binding_change_is_a_visible_plan_change() { + let dir = fixture(); + write_applyable_state(dir.path()); + let converge = apply_config_dir(dir.path()).await; + assert!(converge.converged, "{converge:?}"); + // Edit ONLY applies_to: the policy file digest is unchanged. + fs::write( + dir.path().join(CLUSTER_CONFIG_FILE), + r#" +version: 1 +metadata: + name: test +state: + backend: cluster + lock: true +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +policies: + base: + file: ./base.policy.yaml + applies_to: [cluster, knowledge] +"#, + ) + .unwrap(); + + let plan = plan_config_dir(dir.path()).await; + let change = plan + .changes + .iter() + .find(|change| change.resource == "policy.base") + .expect("binding change must be visible in plan"); + assert!(change.binding_change); + assert_eq!(change.operation, PlanOperation::Update); + assert_eq!(change.before_digest, change.after_digest); + + let out = apply_config_dir(dir.path()).await; + assert!(out.ok && out.converged, "{out:?}"); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["policy.base"]["applies_to"], + serde_json::json!(["cluster", "graph.knowledge"]) + ); + // Idempotent: a second run sees no changes. + let again = apply_config_dir(dir.path()).await; + assert!(again.changes.is_empty() && !again.state_written, "{again:?}"); + } + + #[tokio::test] + async fn pre_5a_state_backfills_bindings() { + let dir = fixture(); + write_applyable_state(dir.path()); + let converge = apply_config_dir(dir.path()).await; + assert!(converge.converged, "{converge:?}"); + // Strip the bindings from the state entry (a pre-5A ledger). + let mut state: serde_json::Value = serde_json::from_str( + &fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(), + ) + .unwrap(); + state["applied_revision"]["resources"]["policy.base"] + .as_object_mut() + .unwrap() + .remove("applies_to"); + fs::write( + dir.path().join(CLUSTER_STATE_FILE), + serde_json::to_string_pretty(&state).unwrap(), + ) + .unwrap(); + + let plan = plan_config_dir(dir.path()).await; + assert!( + plan.changes + .iter() + .any(|change| change.resource == "policy.base" && change.binding_change), + "{plan:?}" + ); + let out = apply_config_dir(dir.path()).await; + assert!(out.ok && out.converged, "{out:?}"); + let healed = read_state_json(dir.path()); + assert_eq!( + healed["applied_revision"]["resources"]["policy.base"]["applies_to"], + serde_json::json!(["graph.knowledge"]) + ); + } + + #[tokio::test] + async fn bindings_survive_refresh() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + let converge = apply_config_dir(dir.path()).await; + assert!(converge.converged, "{converge:?}"); + + let refresh = refresh_config_dir(dir.path()).await; + assert!(refresh.ok, "{:?}", refresh.diagnostics); + let state = read_state_json(dir.path()); + assert_eq!( + state["applied_revision"]["resources"]["policy.base"]["applies_to"], + serde_json::json!(["graph.knowledge"]) + ); + } + + // ---- serving snapshot (5B read-only loader) ---- + + #[tokio::test] + async fn serving_snapshot_reads_converged_cluster() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + let converge = apply_config_dir(dir.path()).await; + assert!(converge.converged, "{converge:?}"); + + let snapshot = read_serving_snapshot(dir.path()).expect("converged cluster must serve"); + assert_eq!(snapshot.graphs.len(), 1); + assert_eq!(snapshot.graphs[0].graph_id, "knowledge"); + assert!(snapshot.graphs[0].root.ends_with("graphs/knowledge.omni")); + assert_eq!(snapshot.queries.len(), 1); + assert_eq!(snapshot.queries[0].name, "find_person"); + assert!(snapshot.queries[0].source.contains("query find_person")); + assert_eq!(snapshot.policies.len(), 1); + assert_eq!(snapshot.policies[0].applies_to, vec!["graph.knowledge"]); + assert!(snapshot.policies[0].blob_path.exists()); + } + + #[test] + fn serving_snapshot_refuses_missing_state() { + let dir = fixture(); + let err = read_serving_snapshot(dir.path()).unwrap_err(); + assert!( + err.iter().any(|diagnostic| diagnostic.code == "cluster_state_missing"), + "{err:?}" + ); + } + + #[tokio::test] + async fn serving_snapshot_refuses_pending_recovery() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + apply_config_dir(dir.path()).await; + write_schema_apply_sidecar(dir.path(), "knowledge", "whatever", "01SERVE"); + + let err = read_serving_snapshot(dir.path()).unwrap_err(); + assert!( + err.iter().any(|diagnostic| diagnostic.code == "cluster_recovery_pending"), + "{err:?}" + ); + } + + #[tokio::test] + async fn serving_snapshot_refuses_tampered_blob_and_stripped_bindings() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + apply_config_dir(dir.path()).await; + // Tamper with the query blob... + let snapshot = read_serving_snapshot(dir.path()).unwrap(); + let desired = validate_config_dir(dir.path()); + let query_digest = &desired.resource_digests["query.knowledge.find_person"]; + let blob = dir + .path() + .join(CLUSTER_RESOURCES_DIR) + .join("query/knowledge/find_person") + .join(format!("{query_digest}.gq")); + fs::write(&blob, "tampered").unwrap(); + // ...and strip the policy bindings (pre-5A ledger). + let mut state: serde_json::Value = serde_json::from_str( + &fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(), + ) + .unwrap(); + state["applied_revision"]["resources"]["policy.base"] + .as_object_mut() + .unwrap() + .remove("applies_to"); + fs::write( + dir.path().join(CLUSTER_STATE_FILE), + serde_json::to_string_pretty(&state).unwrap(), + ) + .unwrap(); + + let err = read_serving_snapshot(dir.path()).unwrap_err(); + assert!( + err.iter() + .any(|diagnostic| diagnostic.code == "catalog_payload_digest_mismatch"), + "{err:?}" + ); + assert!( + err.iter().any(|diagnostic| diagnostic.code == "policy_bindings_missing"), + "{err:?}" + ); + let _ = snapshot; // the pre-tamper read succeeded + } + + #[test] + fn serving_snapshot_refuses_empty_cluster() { + let dir = fixture(); + write_state_resources(dir.path(), &[]); // state exists, no graphs + + let err = read_serving_snapshot(dir.path()).unwrap_err(); + assert!( + err.iter().any(|diagnostic| diagnostic.code == "cluster_empty"), + "{err:?}" + ); + } + + // ---- query discovery (Terraform-style declaration) ---- + + #[test] + fn queries_directory_discovers_every_declaration() { + let dir = tempfile::tempdir().unwrap(); + fs::write(dir.path().join("people.pg"), "\nnode Person {\n name: String @key\n}\n").unwrap(); + fs::create_dir(dir.path().join("queries")).unwrap(); + fs::write( + dir.path().join("queries/people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n\nquery all_people() {\n match { $p: Person }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + dir.path().join("queries/extra.gq"), + "\nquery count_people() {\n match { $p: Person }\n return { count($p) }\n}\n", + ) + .unwrap(); + fs::write(dir.path().join("queries/notes.txt"), "ignored").unwrap(); + fs::write( + dir.path().join("cluster.yaml"), + "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: ./queries/\n", + ) + .unwrap(); + + let out = validate_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + let names: Vec<&str> = out + .resource_digests + .keys() + .filter_map(|address| address.strip_prefix("query.knowledge.")) + .collect(); + assert_eq!(names, vec!["all_people", "count_people", "find_person"]); + } + + #[test] + fn queries_list_and_single_file_forms_discover() { + let dir = tempfile::tempdir().unwrap(); + fs::write(dir.path().join("people.pg"), "\nnode Person {\n name: String @key\n}\n").unwrap(); + fs::write( + dir.path().join("a.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + dir.path().join("b.gq"), + "\nquery all_people() {\n match { $p: Person }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + dir.path().join("cluster.yaml"), + "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: [./a.gq, ./b.gq]\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.resource_digests.contains_key("query.knowledge.find_person")); + assert!(out.resource_digests.contains_key("query.knowledge.all_people")); + + // Single-file string form + fs::write( + dir.path().join("cluster.yaml"), + "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: ./a.gq\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!(out.resource_digests.contains_key("query.knowledge.find_person")); + assert!(!out.resource_digests.contains_key("query.knowledge.all_people")); + } + + #[test] + fn query_discovery_rejects_duplicates_and_parse_errors() { + let dir = tempfile::tempdir().unwrap(); + fs::write(dir.path().join("people.pg"), "\nnode Person {\n name: String @key\n}\n").unwrap(); + let decl = "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n"; + fs::write(dir.path().join("a.gq"), decl).unwrap(); + fs::write(dir.path().join("b.gq"), decl).unwrap(); + fs::write( + dir.path().join("cluster.yaml"), + "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: [./a.gq, ./b.gq]\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "duplicate_query_name"), + "{:?}", + out.diagnostics + ); + + fs::write(dir.path().join("broken.gq"), "query {{{ nope").unwrap(); + fs::write( + dir.path().join("cluster.yaml"), + "version: 1\ngraphs:\n knowledge:\n schema: ./people.pg\n queries: ./broken.gq\n", + ) + .unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "query_parse_error"), + "{:?}", + out.diagnostics + ); + } + + #[test] + fn status_warns_on_pending_recovery_sidecar() { + let dir = fixture(); + write_applyable_state(dir.path()); + write_create_sidecar(dir.path(), "knowledge", "irrelevant", "01STATUS"); + + let out = status_config_dir(dir.path()); + assert!(out.ok, "{:?}", out.diagnostics); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "cluster_recovery_pending" + && diagnostic.severity == DiagnosticSeverity::Warning) + ); + } + + #[tokio::test] + async fn plan_annotates_apply_dispositions() { + let dir = fixture(); + let out = plan_config_dir(dir.path()).await; + assert!(out.ok, "{:?}", out.diagnostics); + let by_resource: BTreeMap<&str, &PlanChange> = out + .changes + .iter() + .map(|change| (change.resource.as_str(), change)) + .collect(); + // Stage 4A: graph/schema creates are executable, and dependents ride + // the same run — plan previews exactly that. + assert_eq!( + by_resource["graph.knowledge"].disposition, + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["schema.knowledge"].disposition, + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["query.knowledge.find_person"].disposition, + Some(ApplyDisposition::Applied) + ); + assert_eq!( + by_resource["policy.base"].disposition, + Some(ApplyDisposition::Applied) + ); + } From 5a8047e5d0615f4f21cd62cb4abab1680aeada91 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 05:28:04 +0300 Subject: [PATCH 102/165] refactor(cluster): move the storage backend to store.rs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verbatim move of LocalStateBackend, StateSnapshot, StateLockGuard and their impls — the single home for stored-state I/O (state ledger, lock, recovery sidecars, approval artifacts), where the RFC-006 object-storage port lands next as a focused diff. Visibility bumps (pub(crate)) only; 95 tests green before and after. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 556 +------------------------ crates/omnigraph-cluster/src/store.rs | 561 ++++++++++++++++++++++++++ 2 files changed, 564 insertions(+), 553 deletions(-) create mode 100644 crates/omnigraph-cluster/src/store.rs diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index da80710..17dd8a6 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -19,6 +19,9 @@ use ulid::Ulid; pub mod failpoints; +mod store; +use store::{LocalStateBackend, StateLockGuard, StateSnapshot}; + pub const CLUSTER_CONFIG_FILE: &str = "cluster.yaml"; pub const CLUSTER_GRAPHS_DIR: &str = "graphs"; pub const CLUSTER_STATE_DIR: &str = "__cluster"; @@ -666,25 +669,6 @@ struct SweepOutcome { consumed_approvals: Vec<String>, } -#[derive(Debug)] -struct LocalStateBackend { - state_dir: PathBuf, - state_path: PathBuf, - lock_path: PathBuf, - recoveries_dir: PathBuf, - approvals_dir: PathBuf, -} - -#[derive(Debug)] -struct StateSnapshot { - state: Option<ClusterState>, - state_cas: Option<String>, -} - -#[derive(Debug)] -struct StateLockGuard { - path: PathBuf, -} pub fn validate_config_dir(config_dir: impl AsRef<Path>) -> ValidateOutput { let outcome = load_desired(config_dir.as_ref()); @@ -2436,540 +2420,6 @@ fn validate_cluster_header( } } -impl LocalStateBackend { - fn new(config_dir: &Path) -> Self { - let state_dir = config_dir.join(CLUSTER_STATE_DIR); - Self { - state_path: config_dir.join(CLUSTER_STATE_FILE), - lock_path: config_dir.join(CLUSTER_LOCK_FILE), - recoveries_dir: config_dir.join(CLUSTER_RECOVERIES_DIR), - approvals_dir: config_dir.join(CLUSTER_APPROVALS_DIR), - state_dir, - } - } - - /// List approval artifacts in ULID (filename) order; unparseable files - /// warn and stay on disk for the operator. - fn list_approval_artifacts( - &self, - diagnostics: &mut Vec<Diagnostic>, - ) -> Vec<(PathBuf, ApprovalArtifact)> { - let mut paths = Vec::new(); - match fs::read_dir(&self.approvals_dir) { - Ok(entries) => { - for entry in entries.flatten() { - let path = entry.path(); - if path.extension().is_some_and(|ext| ext == "json") { - paths.push(path); - } - } - } - Err(err) if err.kind() == ErrorKind::NotFound => {} - Err(err) => diagnostics.push(Diagnostic::warning( - "approval_read_error", - CLUSTER_APPROVALS_DIR, - format!("could not list approval artifacts: {err}"), - )), - } - paths.sort(); - let mut artifacts = Vec::new(); - for path in paths { - match fs::read_to_string(&path) - .map_err(|err| err.to_string()) - .and_then(|text| { - serde_json::from_str::<ApprovalArtifact>(&text).map_err(|err| err.to_string()) - }) { - Ok(artifact) if artifact.schema_version == 1 => artifacts.push((path, artifact)), - Ok(artifact) => diagnostics.push(Diagnostic::warning( - "unsupported_approval_version", - display_path(&path), - format!( - "unsupported approval artifact version {}; leaving it in place", - artifact.schema_version - ), - )), - Err(err) => diagnostics.push(Diagnostic::warning( - "invalid_approval_artifact", - display_path(&path), - format!("could not parse approval artifact ({err}); leaving it in place"), - )), - } - } - artifacts - } - - /// Atomically write (or rewrite, e.g. on consumption) an approval artifact. - fn write_approval_artifact(&self, artifact: &ApprovalArtifact) -> Result<PathBuf, Diagnostic> { - fs::create_dir_all(&self.approvals_dir).map_err(|err| { - Diagnostic::error( - "approval_write_error", - CLUSTER_APPROVALS_DIR, - format!("could not create approvals directory: {err}"), - ) - })?; - let target = self - .approvals_dir - .join(format!("{}.json", artifact.approval_id)); - let mut payload = serde_json::to_string_pretty(artifact).map_err(|err| { - Diagnostic::error( - "approval_write_error", - display_path(&target), - format!("could not encode approval artifact: {err}"), - ) - })?; - payload.push('\n'); - let tmp_path = self - .approvals_dir - .join(format!("{}.json.tmp.{}", artifact.approval_id, Ulid::new())); - fs::write(&tmp_path, payload.as_bytes()).map_err(|err| { - Diagnostic::error( - "approval_write_error", - display_path(&tmp_path), - format!("could not write approval artifact: {err}"), - ) - })?; - if let Err(err) = fs::rename(&tmp_path, &target) { - let _ = fs::remove_file(&tmp_path); - return Err(Diagnostic::error( - "approval_write_error", - display_path(&target), - format!("could not move approval artifact into place: {err}"), - )); - } - Ok(target) - } - - /// List recovery sidecars in ULID (filename) order. Unparseable files are - /// reported as warnings and skipped — they stay on disk for the operator. - fn list_recovery_sidecars( - &self, - diagnostics: &mut Vec<Diagnostic>, - ) -> Vec<(PathBuf, RecoverySidecar)> { - let mut paths = Vec::new(); - match fs::read_dir(&self.recoveries_dir) { - Ok(entries) => { - for entry in entries.flatten() { - let path = entry.path(); - if path.extension().is_some_and(|ext| ext == "json") { - paths.push(path); - } - } - } - Err(err) if err.kind() == ErrorKind::NotFound => {} - Err(err) => { - diagnostics.push(Diagnostic::warning( - "recovery_sidecar_read_error", - CLUSTER_RECOVERIES_DIR, - format!("could not list recovery sidecars: {err}"), - )); - } - } - paths.sort(); - let mut sidecars = Vec::new(); - for path in paths { - match fs::read_to_string(&path) - .map_err(|err| err.to_string()) - .and_then(|text| { - serde_json::from_str::<RecoverySidecar>(&text).map_err(|err| err.to_string()) - }) { - Ok(sidecar) if sidecar.schema_version == 1 => sidecars.push((path, sidecar)), - Ok(sidecar) => diagnostics.push(Diagnostic::warning( - "unsupported_recovery_sidecar_version", - display_path(&path), - format!( - "unsupported recovery sidecar version {}; leaving it in place", - sidecar.schema_version - ), - )), - Err(err) => diagnostics.push(Diagnostic::warning( - "invalid_recovery_sidecar", - display_path(&path), - format!("could not parse recovery sidecar ({err}); leaving it in place"), - )), - } - } - sidecars - } - - /// Atomically write (or rewrite) a recovery sidecar; returns its path. - fn write_recovery_sidecar(&self, sidecar: &RecoverySidecar) -> Result<PathBuf, Diagnostic> { - fs::create_dir_all(&self.recoveries_dir).map_err(|err| { - Diagnostic::error( - "recovery_sidecar_write_error", - CLUSTER_RECOVERIES_DIR, - format!("could not create recoveries directory: {err}"), - ) - })?; - let target = self - .recoveries_dir - .join(format!("{}.json", sidecar.operation_id)); - let mut payload = serde_json::to_string_pretty(sidecar).map_err(|err| { - Diagnostic::error( - "recovery_sidecar_write_error", - display_path(&target), - format!("could not encode recovery sidecar: {err}"), - ) - })?; - payload.push('\n'); - let tmp_path = self - .recoveries_dir - .join(format!("{}.json.tmp.{}", sidecar.operation_id, Ulid::new())); - fs::write(&tmp_path, payload.as_bytes()).map_err(|err| { - Diagnostic::error( - "recovery_sidecar_write_error", - display_path(&tmp_path), - format!("could not write recovery sidecar: {err}"), - ) - })?; - if let Err(err) = fs::rename(&tmp_path, &target) { - let _ = fs::remove_file(&tmp_path); - return Err(Diagnostic::error( - "recovery_sidecar_write_error", - display_path(&target), - format!("could not move recovery sidecar into place: {err}"), - )); - } - Ok(target) - } - - fn observations(&self) -> StateObservations { - StateObservations { - state_path: display_path(&self.state_path), - lock_path: display_path(&self.lock_path), - state_found: false, - applied_config_digest: None, - state_revision: 0, - state_cas: None, - resource_count: 0, - locked: false, - lock_id: None, - lock_acquired: false, - acquired_lock_id: None, - lock_operation: None, - lock_created_at: None, - lock_pid: None, - lock_age_seconds: None, - } - } - - fn read_state( - &self, - observations: &mut StateObservations, - ) -> Result<StateSnapshot, Diagnostic> { - let text = match fs::read_to_string(&self.state_path) { - Ok(text) => text, - Err(err) if err.kind() == ErrorKind::NotFound => { - return Ok(StateSnapshot { - state: None, - state_cas: None, - }); - } - Err(err) => { - return Err(Diagnostic::error( - "state_read_error", - CLUSTER_STATE_FILE, - format!("could not read state file: {err}"), - )); - } - }; - - observations.state_found = true; - let state_cas = format!("sha256:{}", sha256_hex(text.as_bytes())); - observations.state_cas = Some(state_cas.clone()); - - let state = serde_json::from_str::<ClusterState>(&text).map_err(|err| { - Diagnostic::error( - "invalid_state_json", - CLUSTER_STATE_FILE, - format!("could not parse state JSON: {err}"), - ) - })?; - - if state.version != 1 { - return Err(Diagnostic::error( - "unsupported_state_version", - "state.version", - format!( - "unsupported cluster state version {}; this build supports version 1", - state.version - ), - )); - } - - observations.applied_config_digest = state.applied_revision.config_digest.clone(); - observations.state_revision = state.state_revision; - observations.resource_count = state.applied_revision.resources.len(); - - Ok(StateSnapshot { - state: Some(state), - state_cas: Some(state_cas), - }) - } - - fn write_state( - &self, - state: &ClusterState, - expected_cas: Option<&str>, - observations: &mut StateObservations, - ) -> Result<(), Diagnostic> { - fs::create_dir_all(&self.state_dir).map_err(|err| { - Diagnostic::error( - "state_write_error", - CLUSTER_STATE_DIR, - format!("could not create cluster state directory: {err}"), - ) - })?; - - let current_cas = self.current_state_cas()?; - if current_cas.as_deref() != expected_cas { - return Err(Diagnostic::error( - "state_cas_mismatch", - CLUSTER_STATE_FILE, - "state.json changed while the command was running; re-run the command against the latest state", - )); - } - - let mut payload = serde_json::to_string_pretty(state).map_err(|err| { - Diagnostic::error( - "state_write_error", - CLUSTER_STATE_FILE, - format!("could not encode state JSON: {err}"), - ) - })?; - payload.push('\n'); - - let tmp_path = self - .state_dir - .join(format!("state.json.tmp.{}", Ulid::new())); - let mut file = OpenOptions::new() - .write(true) - .create_new(true) - .open(&tmp_path) - .map_err(|err| { - Diagnostic::error( - "state_write_error", - display_path(&tmp_path), - format!("could not create temporary state file: {err}"), - ) - })?; - file.write_all(payload.as_bytes()).map_err(|err| { - Diagnostic::error( - "state_write_error", - display_path(&tmp_path), - format!("could not write temporary state file: {err}"), - ) - })?; - file.sync_all().map_err(|err| { - Diagnostic::error( - "state_write_error", - display_path(&tmp_path), - format!("could not sync temporary state file: {err}"), - ) - })?; - drop(file); - - if let Err(err) = fs::rename(&tmp_path, &self.state_path) { - let _ = fs::remove_file(&tmp_path); - return Err(Diagnostic::error( - "state_write_error", - CLUSTER_STATE_FILE, - format!("could not replace state.json atomically: {err}"), - )); - } - - let written = fs::read_to_string(&self.state_path).map_err(|err| { - Diagnostic::error( - "state_write_error", - CLUSTER_STATE_FILE, - format!("could not read state.json after write: {err}"), - ) - })?; - observations.state_found = true; - observations.applied_config_digest = state.applied_revision.config_digest.clone(); - observations.state_revision = state.state_revision; - observations.state_cas = Some(format!("sha256:{}", sha256_hex(written.as_bytes()))); - observations.resource_count = state.applied_revision.resources.len(); - - Ok(()) - } - - fn current_state_cas(&self) -> Result<Option<String>, Diagnostic> { - match fs::read(&self.state_path) { - Ok(bytes) => Ok(Some(format!("sha256:{}", sha256_hex(&bytes)))), - Err(err) if err.kind() == ErrorKind::NotFound => Ok(None), - Err(err) => Err(Diagnostic::error( - "state_read_error", - CLUSTER_STATE_FILE, - format!("could not read state file for CAS check: {err}"), - )), - } - } - - fn acquire_lock( - &self, - operation: &str, - observations: &mut StateObservations, - ) -> Result<StateLockGuard, Diagnostic> { - fs::create_dir_all(&self.state_dir).map_err(|err| { - Diagnostic::error( - "state_lock_error", - CLUSTER_STATE_DIR, - format!("could not create cluster state directory: {err}"), - ) - })?; - - let lock_id = Ulid::new().to_string(); - let lock = StateLockFile { - version: 1, - lock_id: lock_id.clone(), - operation: operation.to_string(), - created_at: OffsetDateTime::now_utc() - .format(&Rfc3339) - .unwrap_or_else(|_| "1970-01-01T00:00:00Z".to_string()), - pid: process::id(), - }; - let payload = serde_json::to_string_pretty(&lock).map_err(|err| { - Diagnostic::error( - "state_lock_error", - CLUSTER_LOCK_FILE, - format!("could not encode state lock: {err}"), - ) - })?; - - match OpenOptions::new() - .write(true) - .create_new(true) - .open(&self.lock_path) - { - Ok(mut file) => { - if let Err(err) = file.write_all(payload.as_bytes()) { - // No guard exists yet, so clean up the create-new file here - // instead of leaving a stale partial lock for the next run. - drop(file); - let _ = fs::remove_file(&self.lock_path); - return Err(Diagnostic::error( - "state_lock_error", - CLUSTER_LOCK_FILE, - format!("could not write state lock: {err}"), - )); - } - observations.lock_acquired = true; - observations.acquired_lock_id = Some(lock_id.clone()); - Ok(StateLockGuard { - path: self.lock_path.clone(), - }) - } - Err(err) if err.kind() == ErrorKind::AlreadyExists => { - self.observe_lock_metadata_lossy(observations); - Err(Diagnostic::error( - "state_lock_held", - CLUSTER_LOCK_FILE, - state_lock_held_message(observations), - )) - } - Err(err) => Err(Diagnostic::error( - "state_lock_error", - CLUSTER_LOCK_FILE, - format!("could not acquire state lock: {err}"), - )), - } - } - - fn force_unlock( - &self, - requested_lock_id: &str, - observations: &mut StateObservations, - ) -> Result<(), Diagnostic> { - let text = match fs::read_to_string(&self.lock_path) { - Ok(text) => text, - Err(err) if err.kind() == ErrorKind::NotFound => { - return Err(Diagnostic::error( - "state_lock_missing", - CLUSTER_LOCK_FILE, - "cluster state lock is not present; nothing was unlocked", - )); - } - Err(err) => { - return Err(Diagnostic::error( - "state_lock_read_error", - CLUSTER_LOCK_FILE, - format!("could not read state lock: {err}"), - )); - } - }; - observations.locked = true; - let lock = parse_lock_file_for_unlock(&text)?; - observations.observe_lock_metadata(&lock); - - if lock.lock_id != requested_lock_id { - return Err(Diagnostic::error( - "state_lock_id_mismatch", - CLUSTER_LOCK_FILE, - format!( - "cluster state lock id is {}; refusing to unlock with requested id {requested_lock_id}", - lock.lock_id - ), - )); - } - - fs::remove_file(&self.lock_path).map_err(|err| { - Diagnostic::error( - "state_unlock_error", - CLUSTER_LOCK_FILE, - format!("could not remove state lock: {err}"), - ) - }) - } - - fn observe_lock( - &self, - observations: &mut StateObservations, - diagnostics: &mut Vec<Diagnostic>, - ) { - if self.lock_path.exists() { - observations.locked = true; - match fs::read_to_string(&self.lock_path) { - Ok(text) => match serde_json::from_str::<StateLockFile>(&text) { - Ok(lock) if lock.version == 1 => { - observations.observe_lock_metadata(&lock); - } - Ok(lock) => diagnostics.push(Diagnostic::warning( - "unsupported_state_lock_version", - CLUSTER_LOCK_FILE, - format!("unsupported cluster state lock version {}", lock.version), - )), - Err(err) => diagnostics.push(Diagnostic::warning( - "invalid_state_lock", - CLUSTER_LOCK_FILE, - format!("could not parse state lock: {err}"), - )), - }, - Err(err) => diagnostics.push(Diagnostic::warning( - "state_lock_read_error", - CLUSTER_LOCK_FILE, - format!("could not read state lock: {err}"), - )), - } - } - } - - fn observe_lock_metadata_lossy(&self, observations: &mut StateObservations) { - observations.locked = true; - if let Ok(text) = fs::read_to_string(&self.lock_path) { - if let Ok(lock) = serde_json::from_str::<StateLockFile>(&text) { - if lock.version == 1 { - observations.observe_lock_metadata(&lock); - } - } - } - } -} - -impl Drop for StateLockGuard { - fn drop(&mut self) { - let _ = fs::remove_file(&self.path); - } -} fn parse_lock_file_for_unlock(text: &str) -> Result<StateLockFile, Diagnostic> { let lock = serde_json::from_str::<StateLockFile>(text).map_err(|err| { diff --git a/crates/omnigraph-cluster/src/store.rs b/crates/omnigraph-cluster/src/store.rs new file mode 100644 index 0000000..f378660 --- /dev/null +++ b/crates/omnigraph-cluster/src/store.rs @@ -0,0 +1,561 @@ +//! The cluster's storage backend: state ledger, lock, recovery +//! sidecars, approval artifacts (moved verbatim from lib.rs in the +//! modularization). The object-storage port (RFC-006) lands here as a +//! follow-up — this module is the single home for stored-state I/O. + +use super::*; + +#[derive(Debug)] +pub(crate) struct LocalStateBackend { + state_dir: PathBuf, + state_path: PathBuf, + lock_path: PathBuf, + recoveries_dir: PathBuf, + approvals_dir: PathBuf, +} + +#[derive(Debug)] +pub(crate) struct StateSnapshot { + pub(crate) state: Option<ClusterState>, + pub(crate) state_cas: Option<String>, +} + +#[derive(Debug)] +pub(crate) struct StateLockGuard { + path: PathBuf, +} + +impl LocalStateBackend { + pub(crate) fn new(config_dir: &Path) -> Self { + let state_dir = config_dir.join(CLUSTER_STATE_DIR); + Self { + state_path: config_dir.join(CLUSTER_STATE_FILE), + lock_path: config_dir.join(CLUSTER_LOCK_FILE), + recoveries_dir: config_dir.join(CLUSTER_RECOVERIES_DIR), + approvals_dir: config_dir.join(CLUSTER_APPROVALS_DIR), + state_dir, + } + } + + /// List approval artifacts in ULID (filename) order; unparseable files + /// warn and stay on disk for the operator. + pub(crate) fn list_approval_artifacts( + &self, + diagnostics: &mut Vec<Diagnostic>, + ) -> Vec<(PathBuf, ApprovalArtifact)> { + let mut paths = Vec::new(); + match fs::read_dir(&self.approvals_dir) { + Ok(entries) => { + for entry in entries.flatten() { + let path = entry.path(); + if path.extension().is_some_and(|ext| ext == "json") { + paths.push(path); + } + } + } + Err(err) if err.kind() == ErrorKind::NotFound => {} + Err(err) => diagnostics.push(Diagnostic::warning( + "approval_read_error", + CLUSTER_APPROVALS_DIR, + format!("could not list approval artifacts: {err}"), + )), + } + paths.sort(); + let mut artifacts = Vec::new(); + for path in paths { + match fs::read_to_string(&path) + .map_err(|err| err.to_string()) + .and_then(|text| { + serde_json::from_str::<ApprovalArtifact>(&text).map_err(|err| err.to_string()) + }) { + Ok(artifact) if artifact.schema_version == 1 => artifacts.push((path, artifact)), + Ok(artifact) => diagnostics.push(Diagnostic::warning( + "unsupported_approval_version", + display_path(&path), + format!( + "unsupported approval artifact version {}; leaving it in place", + artifact.schema_version + ), + )), + Err(err) => diagnostics.push(Diagnostic::warning( + "invalid_approval_artifact", + display_path(&path), + format!("could not parse approval artifact ({err}); leaving it in place"), + )), + } + } + artifacts + } + + /// Atomically write (or rewrite, e.g. on consumption) an approval artifact. + pub(crate) fn write_approval_artifact(&self, artifact: &ApprovalArtifact) -> Result<PathBuf, Diagnostic> { + fs::create_dir_all(&self.approvals_dir).map_err(|err| { + Diagnostic::error( + "approval_write_error", + CLUSTER_APPROVALS_DIR, + format!("could not create approvals directory: {err}"), + ) + })?; + let target = self + .approvals_dir + .join(format!("{}.json", artifact.approval_id)); + let mut payload = serde_json::to_string_pretty(artifact).map_err(|err| { + Diagnostic::error( + "approval_write_error", + display_path(&target), + format!("could not encode approval artifact: {err}"), + ) + })?; + payload.push('\n'); + let tmp_path = self + .approvals_dir + .join(format!("{}.json.tmp.{}", artifact.approval_id, Ulid::new())); + fs::write(&tmp_path, payload.as_bytes()).map_err(|err| { + Diagnostic::error( + "approval_write_error", + display_path(&tmp_path), + format!("could not write approval artifact: {err}"), + ) + })?; + if let Err(err) = fs::rename(&tmp_path, &target) { + let _ = fs::remove_file(&tmp_path); + return Err(Diagnostic::error( + "approval_write_error", + display_path(&target), + format!("could not move approval artifact into place: {err}"), + )); + } + Ok(target) + } + + /// List recovery sidecars in ULID (filename) order. Unparseable files are + /// reported as warnings and skipped — they stay on disk for the operator. + pub(crate) fn list_recovery_sidecars( + &self, + diagnostics: &mut Vec<Diagnostic>, + ) -> Vec<(PathBuf, RecoverySidecar)> { + let mut paths = Vec::new(); + match fs::read_dir(&self.recoveries_dir) { + Ok(entries) => { + for entry in entries.flatten() { + let path = entry.path(); + if path.extension().is_some_and(|ext| ext == "json") { + paths.push(path); + } + } + } + Err(err) if err.kind() == ErrorKind::NotFound => {} + Err(err) => { + diagnostics.push(Diagnostic::warning( + "recovery_sidecar_read_error", + CLUSTER_RECOVERIES_DIR, + format!("could not list recovery sidecars: {err}"), + )); + } + } + paths.sort(); + let mut sidecars = Vec::new(); + for path in paths { + match fs::read_to_string(&path) + .map_err(|err| err.to_string()) + .and_then(|text| { + serde_json::from_str::<RecoverySidecar>(&text).map_err(|err| err.to_string()) + }) { + Ok(sidecar) if sidecar.schema_version == 1 => sidecars.push((path, sidecar)), + Ok(sidecar) => diagnostics.push(Diagnostic::warning( + "unsupported_recovery_sidecar_version", + display_path(&path), + format!( + "unsupported recovery sidecar version {}; leaving it in place", + sidecar.schema_version + ), + )), + Err(err) => diagnostics.push(Diagnostic::warning( + "invalid_recovery_sidecar", + display_path(&path), + format!("could not parse recovery sidecar ({err}); leaving it in place"), + )), + } + } + sidecars + } + + /// Atomically write (or rewrite) a recovery sidecar; returns its path. + pub(crate) fn write_recovery_sidecar(&self, sidecar: &RecoverySidecar) -> Result<PathBuf, Diagnostic> { + fs::create_dir_all(&self.recoveries_dir).map_err(|err| { + Diagnostic::error( + "recovery_sidecar_write_error", + CLUSTER_RECOVERIES_DIR, + format!("could not create recoveries directory: {err}"), + ) + })?; + let target = self + .recoveries_dir + .join(format!("{}.json", sidecar.operation_id)); + let mut payload = serde_json::to_string_pretty(sidecar).map_err(|err| { + Diagnostic::error( + "recovery_sidecar_write_error", + display_path(&target), + format!("could not encode recovery sidecar: {err}"), + ) + })?; + payload.push('\n'); + let tmp_path = self + .recoveries_dir + .join(format!("{}.json.tmp.{}", sidecar.operation_id, Ulid::new())); + fs::write(&tmp_path, payload.as_bytes()).map_err(|err| { + Diagnostic::error( + "recovery_sidecar_write_error", + display_path(&tmp_path), + format!("could not write recovery sidecar: {err}"), + ) + })?; + if let Err(err) = fs::rename(&tmp_path, &target) { + let _ = fs::remove_file(&tmp_path); + return Err(Diagnostic::error( + "recovery_sidecar_write_error", + display_path(&target), + format!("could not move recovery sidecar into place: {err}"), + )); + } + Ok(target) + } + + pub(crate) fn observations(&self) -> StateObservations { + StateObservations { + state_path: display_path(&self.state_path), + lock_path: display_path(&self.lock_path), + state_found: false, + applied_config_digest: None, + state_revision: 0, + state_cas: None, + resource_count: 0, + locked: false, + lock_id: None, + lock_acquired: false, + acquired_lock_id: None, + lock_operation: None, + lock_created_at: None, + lock_pid: None, + lock_age_seconds: None, + } + } + + pub(crate) fn read_state( + &self, + observations: &mut StateObservations, + ) -> Result<StateSnapshot, Diagnostic> { + let text = match fs::read_to_string(&self.state_path) { + Ok(text) => text, + Err(err) if err.kind() == ErrorKind::NotFound => { + return Ok(StateSnapshot { + state: None, + state_cas: None, + }); + } + Err(err) => { + return Err(Diagnostic::error( + "state_read_error", + CLUSTER_STATE_FILE, + format!("could not read state file: {err}"), + )); + } + }; + + observations.state_found = true; + let state_cas = format!("sha256:{}", sha256_hex(text.as_bytes())); + observations.state_cas = Some(state_cas.clone()); + + let state = serde_json::from_str::<ClusterState>(&text).map_err(|err| { + Diagnostic::error( + "invalid_state_json", + CLUSTER_STATE_FILE, + format!("could not parse state JSON: {err}"), + ) + })?; + + if state.version != 1 { + return Err(Diagnostic::error( + "unsupported_state_version", + "state.version", + format!( + "unsupported cluster state version {}; this build supports version 1", + state.version + ), + )); + } + + observations.applied_config_digest = state.applied_revision.config_digest.clone(); + observations.state_revision = state.state_revision; + observations.resource_count = state.applied_revision.resources.len(); + + Ok(StateSnapshot { + state: Some(state), + state_cas: Some(state_cas), + }) + } + + pub(crate) fn write_state( + &self, + state: &ClusterState, + expected_cas: Option<&str>, + observations: &mut StateObservations, + ) -> Result<(), Diagnostic> { + fs::create_dir_all(&self.state_dir).map_err(|err| { + Diagnostic::error( + "state_write_error", + CLUSTER_STATE_DIR, + format!("could not create cluster state directory: {err}"), + ) + })?; + + let current_cas = self.current_state_cas()?; + if current_cas.as_deref() != expected_cas { + return Err(Diagnostic::error( + "state_cas_mismatch", + CLUSTER_STATE_FILE, + "state.json changed while the command was running; re-run the command against the latest state", + )); + } + + let mut payload = serde_json::to_string_pretty(state).map_err(|err| { + Diagnostic::error( + "state_write_error", + CLUSTER_STATE_FILE, + format!("could not encode state JSON: {err}"), + ) + })?; + payload.push('\n'); + + let tmp_path = self + .state_dir + .join(format!("state.json.tmp.{}", Ulid::new())); + let mut file = OpenOptions::new() + .write(true) + .create_new(true) + .open(&tmp_path) + .map_err(|err| { + Diagnostic::error( + "state_write_error", + display_path(&tmp_path), + format!("could not create temporary state file: {err}"), + ) + })?; + file.write_all(payload.as_bytes()).map_err(|err| { + Diagnostic::error( + "state_write_error", + display_path(&tmp_path), + format!("could not write temporary state file: {err}"), + ) + })?; + file.sync_all().map_err(|err| { + Diagnostic::error( + "state_write_error", + display_path(&tmp_path), + format!("could not sync temporary state file: {err}"), + ) + })?; + drop(file); + + if let Err(err) = fs::rename(&tmp_path, &self.state_path) { + let _ = fs::remove_file(&tmp_path); + return Err(Diagnostic::error( + "state_write_error", + CLUSTER_STATE_FILE, + format!("could not replace state.json atomically: {err}"), + )); + } + + let written = fs::read_to_string(&self.state_path).map_err(|err| { + Diagnostic::error( + "state_write_error", + CLUSTER_STATE_FILE, + format!("could not read state.json after write: {err}"), + ) + })?; + observations.state_found = true; + observations.applied_config_digest = state.applied_revision.config_digest.clone(); + observations.state_revision = state.state_revision; + observations.state_cas = Some(format!("sha256:{}", sha256_hex(written.as_bytes()))); + observations.resource_count = state.applied_revision.resources.len(); + + Ok(()) + } + + pub(crate) fn current_state_cas(&self) -> Result<Option<String>, Diagnostic> { + match fs::read(&self.state_path) { + Ok(bytes) => Ok(Some(format!("sha256:{}", sha256_hex(&bytes)))), + Err(err) if err.kind() == ErrorKind::NotFound => Ok(None), + Err(err) => Err(Diagnostic::error( + "state_read_error", + CLUSTER_STATE_FILE, + format!("could not read state file for CAS check: {err}"), + )), + } + } + + pub(crate) fn acquire_lock( + &self, + operation: &str, + observations: &mut StateObservations, + ) -> Result<StateLockGuard, Diagnostic> { + fs::create_dir_all(&self.state_dir).map_err(|err| { + Diagnostic::error( + "state_lock_error", + CLUSTER_STATE_DIR, + format!("could not create cluster state directory: {err}"), + ) + })?; + + let lock_id = Ulid::new().to_string(); + let lock = StateLockFile { + version: 1, + lock_id: lock_id.clone(), + operation: operation.to_string(), + created_at: OffsetDateTime::now_utc() + .format(&Rfc3339) + .unwrap_or_else(|_| "1970-01-01T00:00:00Z".to_string()), + pid: process::id(), + }; + let payload = serde_json::to_string_pretty(&lock).map_err(|err| { + Diagnostic::error( + "state_lock_error", + CLUSTER_LOCK_FILE, + format!("could not encode state lock: {err}"), + ) + })?; + + match OpenOptions::new() + .write(true) + .create_new(true) + .open(&self.lock_path) + { + Ok(mut file) => { + if let Err(err) = file.write_all(payload.as_bytes()) { + // No guard exists yet, so clean up the create-new file here + // instead of leaving a stale partial lock for the next run. + drop(file); + let _ = fs::remove_file(&self.lock_path); + return Err(Diagnostic::error( + "state_lock_error", + CLUSTER_LOCK_FILE, + format!("could not write state lock: {err}"), + )); + } + observations.lock_acquired = true; + observations.acquired_lock_id = Some(lock_id.clone()); + Ok(StateLockGuard { + path: self.lock_path.clone(), + }) + } + Err(err) if err.kind() == ErrorKind::AlreadyExists => { + self.observe_lock_metadata_lossy(observations); + Err(Diagnostic::error( + "state_lock_held", + CLUSTER_LOCK_FILE, + state_lock_held_message(observations), + )) + } + Err(err) => Err(Diagnostic::error( + "state_lock_error", + CLUSTER_LOCK_FILE, + format!("could not acquire state lock: {err}"), + )), + } + } + + pub(crate) fn force_unlock( + &self, + requested_lock_id: &str, + observations: &mut StateObservations, + ) -> Result<(), Diagnostic> { + let text = match fs::read_to_string(&self.lock_path) { + Ok(text) => text, + Err(err) if err.kind() == ErrorKind::NotFound => { + return Err(Diagnostic::error( + "state_lock_missing", + CLUSTER_LOCK_FILE, + "cluster state lock is not present; nothing was unlocked", + )); + } + Err(err) => { + return Err(Diagnostic::error( + "state_lock_read_error", + CLUSTER_LOCK_FILE, + format!("could not read state lock: {err}"), + )); + } + }; + observations.locked = true; + let lock = parse_lock_file_for_unlock(&text)?; + observations.observe_lock_metadata(&lock); + + if lock.lock_id != requested_lock_id { + return Err(Diagnostic::error( + "state_lock_id_mismatch", + CLUSTER_LOCK_FILE, + format!( + "cluster state lock id is {}; refusing to unlock with requested id {requested_lock_id}", + lock.lock_id + ), + )); + } + + fs::remove_file(&self.lock_path).map_err(|err| { + Diagnostic::error( + "state_unlock_error", + CLUSTER_LOCK_FILE, + format!("could not remove state lock: {err}"), + ) + }) + } + + pub(crate) fn observe_lock( + &self, + observations: &mut StateObservations, + diagnostics: &mut Vec<Diagnostic>, + ) { + if self.lock_path.exists() { + observations.locked = true; + match fs::read_to_string(&self.lock_path) { + Ok(text) => match serde_json::from_str::<StateLockFile>(&text) { + Ok(lock) if lock.version == 1 => { + observations.observe_lock_metadata(&lock); + } + Ok(lock) => diagnostics.push(Diagnostic::warning( + "unsupported_state_lock_version", + CLUSTER_LOCK_FILE, + format!("unsupported cluster state lock version {}", lock.version), + )), + Err(err) => diagnostics.push(Diagnostic::warning( + "invalid_state_lock", + CLUSTER_LOCK_FILE, + format!("could not parse state lock: {err}"), + )), + }, + Err(err) => diagnostics.push(Diagnostic::warning( + "state_lock_read_error", + CLUSTER_LOCK_FILE, + format!("could not read state lock: {err}"), + )), + } + } + } + + pub(crate) fn observe_lock_metadata_lossy(&self, observations: &mut StateObservations) { + observations.locked = true; + if let Ok(text) = fs::read_to_string(&self.lock_path) { + if let Ok(lock) = serde_json::from_str::<StateLockFile>(&text) { + if lock.version == 1 { + observations.observe_lock_metadata(&lock); + } + } + } + } +} + +impl Drop for StateLockGuard { + fn drop(&mut self) { + let _ = fs::remove_file(&self.path); + } +} From 00fc5cf5378f8018023bf362d1aa3119c526c2a8 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 05:29:44 +0300 Subject: [PATCH 103/165] refactor(cluster): move the serving snapshot to serve.rs Verbatim move of the Serving* types, read_serving_snapshot, and read_verified_payload; public re-exports preserved (the server's imports are unchanged). 95 tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 187 +------------------------ crates/omnigraph-cluster/src/serve.rs | 189 ++++++++++++++++++++++++++ 2 files changed, 192 insertions(+), 184 deletions(-) create mode 100644 crates/omnigraph-cluster/src/serve.rs diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 17dd8a6..f26110e 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -19,8 +19,11 @@ use ulid::Ulid; pub mod failpoints; +mod serve; mod store; use store::{LocalStateBackend, StateLockGuard, StateSnapshot}; +pub use serve::{ServingGraph, ServingPolicy, ServingQuery, ServingSnapshot, read_serving_snapshot}; +use serve::read_verified_payload; pub const CLUSTER_CONFIG_FILE: &str = "cluster.yaml"; pub const CLUSTER_GRAPHS_DIR: &str = "graphs"; @@ -1817,190 +1820,6 @@ pub async fn approve_config_dir( } } -/// One graph in a serving snapshot: its id and on-disk root. -#[derive(Debug, Clone)] -pub struct ServingGraph { - pub graph_id: String, - pub root: PathBuf, -} - -/// One stored query: its graph binding, registry name, and verified source. -#[derive(Debug, Clone)] -pub struct ServingQuery { - pub graph_id: String, - pub name: String, - pub source: String, -} - -/// One policy bundle: its verified catalog blob path and applied bindings -/// (normalized typed refs: `cluster` | `graph.<id>`). -#[derive(Debug, Clone)] -pub struct ServingPolicy { - pub name: String, - pub blob_path: PathBuf, - pub applies_to: Vec<String>, -} - -/// Everything a server needs to boot from the cluster catalog (RFC-005 §D2). -#[derive(Debug, Clone)] -pub struct ServingSnapshot { - pub graphs: Vec<ServingGraph>, - pub queries: Vec<ServingQuery>, - pub policies: Vec<ServingPolicy>, -} - -/// Read the applied revision as a serving snapshot — the read-only loader for -/// the Phase-5 server boot. All-or-nothing per RFC-005 §D4: every readiness -/// failure is collected and the whole snapshot refused; no partial serving. -/// Takes no lock: the state file is replaced atomically, so this reads a -/// consistent point-in-time ledger. -pub fn read_serving_snapshot(config_dir: impl AsRef<Path>) -> Result<ServingSnapshot, Vec<Diagnostic>> { - let config_dir = config_dir.as_ref().to_path_buf(); - let backend = LocalStateBackend::new(&config_dir); - let mut diagnostics: Vec<Diagnostic> = Vec::new(); - - // A ledger a sweep is about to rewrite must not start serving. - let sidecars = backend.list_recovery_sidecars(&mut diagnostics); - if !sidecars.is_empty() { - diagnostics.push(Diagnostic::error( - "cluster_recovery_pending", - CLUSTER_RECOVERIES_DIR, - format!( - "{} interrupted operation(s) await recovery; run any state-mutating cluster command (e.g. `cluster apply`) to sweep, then retry", - sidecars.len() - ), - )); - } - - let mut observations = backend.observations(); - let state = match backend.read_state(&mut observations) { - Ok(snapshot) => match snapshot.state { - Some(state) => Some(state), - None => { - diagnostics.push(Diagnostic::error( - "cluster_state_missing", - CLUSTER_STATE_FILE, - "no cluster state ledger; run `cluster import` and `cluster apply` first", - )); - None - } - }, - Err(diagnostic) => { - diagnostics.push(diagnostic); - None - } - }; - let Some(state) = state else { - return Err(diagnostics); - }; - - let mut graphs = Vec::new(); - let mut queries = Vec::new(); - let mut policies = Vec::new(); - for (address, entry) in &state.applied_revision.resources { - match resource_kind(address) { - ResourceKind::Graph(graph_id) => { - graphs.push(ServingGraph { - root: config_dir - .join(CLUSTER_GRAPHS_DIR) - .join(format!("{graph_id}.omni")), - graph_id, - }); - } - ResourceKind::Schema(_) => {} - kind @ ResourceKind::Query { .. } => { - let ResourceKind::Query { graph, name } = &kind else { - unreachable!() - }; - match read_verified_payload(&config_dir, &kind, &entry.digest, address) { - Ok(source) => queries.push(ServingQuery { - graph_id: graph.clone(), - name: name.clone(), - source, - }), - Err(diagnostic) => diagnostics.push(diagnostic), - } - } - kind @ ResourceKind::Policy(_) => { - let ResourceKind::Policy(name) = &kind else { - unreachable!() - }; - let Some(applies_to) = entry.applies_to.clone() else { - diagnostics.push(Diagnostic::error( - "policy_bindings_missing", - address.clone(), - "no applied applies_to bindings recorded (ledger predates binding metadata); re-run `cluster apply` to backfill", - )); - continue; - }; - match read_verified_payload(&config_dir, &kind, &entry.digest, address) { - Ok(_) => policies.push(ServingPolicy { - name: name.clone(), - blob_path: payload_path(&config_dir, &kind, &entry.digest) - .expect("policy kind always has a payload path"), - applies_to, - }), - Err(diagnostic) => diagnostics.push(diagnostic), - } - } - ResourceKind::Unknown => {} - } - } - - if graphs.is_empty() { - diagnostics.push(Diagnostic::error( - "cluster_empty", - CLUSTER_STATE_FILE, - "the applied revision records no graphs; apply a cluster with at least one graph before serving from it", - )); - } - if has_errors(&diagnostics) { - return Err(diagnostics); - } - Ok(ServingSnapshot { - graphs, - queries, - policies, - }) -} - -/// Read a catalog blob and verify it against the recorded digest. -fn read_verified_payload( - config_dir: &Path, - kind: &ResourceKind, - digest: &str, - address: &str, -) -> Result<String, Diagnostic> { - let path = payload_path(config_dir, kind, digest) - .expect("query/policy kinds always have a payload path"); - let bytes = fs::read(&path).map_err(|err| { - Diagnostic::error( - "catalog_payload_missing", - address, - format!( - "catalog blob '{}' unreadable ({err}); run `cluster refresh` then `cluster apply`, and restart", - display_path(&path) - ), - ) - })?; - if sha256_hex(&bytes) != digest { - return Err(Diagnostic::error( - "catalog_payload_digest_mismatch", - address, - format!( - "catalog blob '{}' does not match its recorded digest; run `cluster refresh` then `cluster apply`, and restart", - display_path(&path) - ), - )); - } - String::from_utf8(bytes).map_err(|err| { - Diagnostic::error( - "catalog_payload_invalid", - address, - format!("catalog blob is not valid UTF-8: {err}"), - ) - }) -} pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { let parsed = parse_cluster_config(config_dir.as_ref()); diff --git a/crates/omnigraph-cluster/src/serve.rs b/crates/omnigraph-cluster/src/serve.rs new file mode 100644 index 0000000..0152bc4 --- /dev/null +++ b/crates/omnigraph-cluster/src/serve.rs @@ -0,0 +1,189 @@ +//! Phase-5 serving snapshot: the read-only loader a `--cluster` server +//! boots from (moved verbatim from lib.rs in the modularization). + +use super::*; + +/// One graph in a serving snapshot: its id and on-disk root. +#[derive(Debug, Clone)] +pub struct ServingGraph { + pub graph_id: String, + pub root: PathBuf, +} + +/// One stored query: its graph binding, registry name, and verified source. +#[derive(Debug, Clone)] +pub struct ServingQuery { + pub graph_id: String, + pub name: String, + pub source: String, +} + +/// One policy bundle: its verified catalog blob path and applied bindings +/// (normalized typed refs: `cluster` | `graph.<id>`). +#[derive(Debug, Clone)] +pub struct ServingPolicy { + pub name: String, + pub blob_path: PathBuf, + pub applies_to: Vec<String>, +} + +/// Everything a server needs to boot from the cluster catalog (RFC-005 §D2). +#[derive(Debug, Clone)] +pub struct ServingSnapshot { + pub graphs: Vec<ServingGraph>, + pub queries: Vec<ServingQuery>, + pub policies: Vec<ServingPolicy>, +} + +/// Read the applied revision as a serving snapshot — the read-only loader for +/// the Phase-5 server boot. All-or-nothing per RFC-005 §D4: every readiness +/// failure is collected and the whole snapshot refused; no partial serving. +/// Takes no lock: the state file is replaced atomically, so this reads a +/// consistent point-in-time ledger. +pub fn read_serving_snapshot(config_dir: impl AsRef<Path>) -> Result<ServingSnapshot, Vec<Diagnostic>> { + let config_dir = config_dir.as_ref().to_path_buf(); + let backend = LocalStateBackend::new(&config_dir); + let mut diagnostics: Vec<Diagnostic> = Vec::new(); + + // A ledger a sweep is about to rewrite must not start serving. + let sidecars = backend.list_recovery_sidecars(&mut diagnostics); + if !sidecars.is_empty() { + diagnostics.push(Diagnostic::error( + "cluster_recovery_pending", + CLUSTER_RECOVERIES_DIR, + format!( + "{} interrupted operation(s) await recovery; run any state-mutating cluster command (e.g. `cluster apply`) to sweep, then retry", + sidecars.len() + ), + )); + } + + let mut observations = backend.observations(); + let state = match backend.read_state(&mut observations) { + Ok(snapshot) => match snapshot.state { + Some(state) => Some(state), + None => { + diagnostics.push(Diagnostic::error( + "cluster_state_missing", + CLUSTER_STATE_FILE, + "no cluster state ledger; run `cluster import` and `cluster apply` first", + )); + None + } + }, + Err(diagnostic) => { + diagnostics.push(diagnostic); + None + } + }; + let Some(state) = state else { + return Err(diagnostics); + }; + + let mut graphs = Vec::new(); + let mut queries = Vec::new(); + let mut policies = Vec::new(); + for (address, entry) in &state.applied_revision.resources { + match resource_kind(address) { + ResourceKind::Graph(graph_id) => { + graphs.push(ServingGraph { + root: config_dir + .join(CLUSTER_GRAPHS_DIR) + .join(format!("{graph_id}.omni")), + graph_id, + }); + } + ResourceKind::Schema(_) => {} + kind @ ResourceKind::Query { .. } => { + let ResourceKind::Query { graph, name } = &kind else { + unreachable!() + }; + match read_verified_payload(&config_dir, &kind, &entry.digest, address) { + Ok(source) => queries.push(ServingQuery { + graph_id: graph.clone(), + name: name.clone(), + source, + }), + Err(diagnostic) => diagnostics.push(diagnostic), + } + } + kind @ ResourceKind::Policy(_) => { + let ResourceKind::Policy(name) = &kind else { + unreachable!() + }; + let Some(applies_to) = entry.applies_to.clone() else { + diagnostics.push(Diagnostic::error( + "policy_bindings_missing", + address.clone(), + "no applied applies_to bindings recorded (ledger predates binding metadata); re-run `cluster apply` to backfill", + )); + continue; + }; + match read_verified_payload(&config_dir, &kind, &entry.digest, address) { + Ok(_) => policies.push(ServingPolicy { + name: name.clone(), + blob_path: payload_path(&config_dir, &kind, &entry.digest) + .expect("policy kind always has a payload path"), + applies_to, + }), + Err(diagnostic) => diagnostics.push(diagnostic), + } + } + ResourceKind::Unknown => {} + } + } + + if graphs.is_empty() { + diagnostics.push(Diagnostic::error( + "cluster_empty", + CLUSTER_STATE_FILE, + "the applied revision records no graphs; apply a cluster with at least one graph before serving from it", + )); + } + if has_errors(&diagnostics) { + return Err(diagnostics); + } + Ok(ServingSnapshot { + graphs, + queries, + policies, + }) +} + +/// Read a catalog blob and verify it against the recorded digest. +pub(crate) fn read_verified_payload( + config_dir: &Path, + kind: &ResourceKind, + digest: &str, + address: &str, +) -> Result<String, Diagnostic> { + let path = payload_path(config_dir, kind, digest) + .expect("query/policy kinds always have a payload path"); + let bytes = fs::read(&path).map_err(|err| { + Diagnostic::error( + "catalog_payload_missing", + address, + format!( + "catalog blob '{}' unreadable ({err}); run `cluster refresh` then `cluster apply`, and restart", + display_path(&path) + ), + ) + })?; + if sha256_hex(&bytes) != digest { + return Err(Diagnostic::error( + "catalog_payload_digest_mismatch", + address, + format!( + "catalog blob '{}' does not match its recorded digest; run `cluster refresh` then `cluster apply`, and restart", + display_path(&path) + ), + )); + } + String::from_utf8(bytes).map_err(|err| { + Diagnostic::error( + "catalog_payload_invalid", + address, + format!("catalog blob is not valid UTF-8: {err}"), + ) + }) +} From 9c3e09e838695fc0f55a04bf1e04ced25ab3ed4b Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 05:30:55 +0300 Subject: [PATCH 104/165] refactor(cluster): move the recovery sweep to sweep.rs Verbatim move of the sidecar classification (all RFC-004 D3 rows), tombstoning, and approval-consumption helpers. 95 tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 383 +------------------------ crates/omnigraph-cluster/src/sweep.rs | 386 ++++++++++++++++++++++++++ 2 files changed, 388 insertions(+), 381 deletions(-) create mode 100644 crates/omnigraph-cluster/src/sweep.rs diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index f26110e..3faacaa 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -20,10 +20,12 @@ use ulid::Ulid; pub mod failpoints; mod serve; +mod sweep; mod store; use store::{LocalStateBackend, StateLockGuard, StateSnapshot}; pub use serve::{ServingGraph, ServingPolicy, ServingQuery, ServingSnapshot, read_serving_snapshot}; use serve::read_verified_payload; +use sweep::{mark_approvals_consumed, record_approval_consumed, sweep_recovery_sidecars, tombstone_graph_subtree, warn_pending_recovery_sidecars}; pub const CLUSTER_CONFIG_FILE: &str = "cluster.yaml"; pub const CLUSTER_GRAPHS_DIR: &str = "graphs"; @@ -2291,387 +2293,6 @@ fn initial_import_state(desired: &DesiredCluster) -> ClusterState { } } -/// Recovery sweep (RFC-004 §D3): runs at the start of every state-mutating -/// cluster command, under the state lock, before the command's own work. -/// Roll-forward-only — the engine's own sidecars make each graph-level -/// operation atomic within the graph, so the cluster never rolls a graph -/// back; it converges the ledger to observable reality or refuses loudly. -/// Mutations ride the calling command's CAS-checked state write; completed -/// sidecars are deleted only after that write lands. -async fn sweep_recovery_sidecars( - backend: &LocalStateBackend, - state: &mut ClusterState, - diagnostics: &mut Vec<Diagnostic>, -) -> SweepOutcome { - let mut outcome = SweepOutcome::default(); - for (path, sidecar) in backend.list_recovery_sidecars(diagnostics) { - match sidecar.kind { - RecoverySidecarKind::GraphCreate => { - sweep_graph_create_sidecar(path, sidecar, state, diagnostics, &mut outcome).await; - } - RecoverySidecarKind::SchemaApply => { - sweep_schema_apply_sidecar(path, sidecar, state, diagnostics, &mut outcome).await; - } - RecoverySidecarKind::GraphDelete => { - sweep_graph_delete_sidecar(path, sidecar, state, diagnostics, &mut outcome); - } - } - } - outcome -} - -async fn sweep_graph_create_sidecar( - path: PathBuf, - sidecar: RecoverySidecar, - state: &mut ClusterState, - diagnostics: &mut Vec<Diagnostic>, - outcome: &mut SweepOutcome, -) { - let graph_address = graph_address(&sidecar.graph_id); - let schema_addr = schema_address(&sidecar.graph_id); - let graph_path = PathBuf::from(&sidecar.graph_uri); - - // Row 1: nothing moved — the init never landed. The sidecar is pure - // intent; remove it and let the command's own plan re-propose the create. - if !graph_path.exists() { - let _ = fs::remove_file(&path); - return; - } - - match Omnigraph::open_read_only(&sidecar.graph_uri).await { - Ok(db) => { - let live_digest = sha256_hex(db.schema_source().as_bytes()); - let recorded = state - .applied_revision - .resources - .get(&schema_addr) - .map(|resource| resource.digest.clone()); - if recorded.as_deref() == Some(live_digest.as_str()) { - // Row 2: crash fell between the state CAS and sidecar delete. - outcome.completed_sidecars.push(path); - } else if live_digest == sidecar.desired_schema_digest { - // Row 4: the create completed on the graph; roll the cluster - // state forward to observable reality. - state.applied_revision.resources.insert( - schema_addr.clone(), - StateResource { - digest: live_digest.clone(), - applies_to: None, - }, - ); - let query_digests = state_query_digests_for_graph(state, &sidecar.graph_id); - let composite = - graph_digest(&sidecar.graph_id, Some(&live_digest), Some(&query_digests)); - state - .applied_revision - .resources - .insert(graph_address.clone(), StateResource { digest: composite, applies_to: None }); - set_resource_status_applied(state, &graph_address); - set_resource_status_applied(state, &schema_addr); - state.recovery_records.insert( - sidecar.operation_id.clone(), - json!({ - "kind": "graph_create", - "graph_id": sidecar.graph_id, - "outcome": "rolled_forward", - "recovered_at": now_rfc3339(), - "actor": sidecar.actor, - }), - ); - diagnostics.push(Diagnostic::warning( - "cluster_recovery_rolled_forward", - graph_address.clone(), - "an interrupted graph create had completed on the graph; cluster state was rolled forward to match", - )); - outcome.completed_sidecars.push(path); - } else { - // Row 6: the graph moved to something the sidecar did not - // intend. Refuse to guess; require refresh + operator re-plan. - set_resource_status( - state, - &graph_address, - ResourceLifecycleStatus::Drifted, - "actual_applied_state_pending", - "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", - ); - set_resource_status( - state, - &schema_addr, - ResourceLifecycleStatus::Drifted, - "actual_applied_state_pending", - "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", - ); - diagnostics.push(Diagnostic::warning( - "cluster_recovery_pending", - graph_address.clone(), - "an interrupted graph create left unexpected graph state; graph-moving work is blocked until repaired", - )); - outcome.pending_graphs.insert(sidecar.graph_id.clone()); - } - } - Err(err) => { - // Row 5: partial root (the engine's documented init gap). Never - // auto-delete — reconciler deletes are the same data-loss class - // as human deletes; the operator removes the root explicitly. - set_resource_status( - state, - &graph_address, - ResourceLifecycleStatus::Error, - "graph_create_incomplete", - "graph root exists but cannot be opened; remove the graph root and re-run `cluster apply`", - ); - set_resource_status( - state, - &schema_addr, - ResourceLifecycleStatus::Error, - "graph_create_incomplete", - "graph root exists but cannot be opened; remove the graph root and re-run `cluster apply`", - ); - diagnostics.push(Diagnostic::error( - "graph_create_incomplete", - graph_address.clone(), - format!( - "graph root '{}' exists but cannot be opened ({err}); remove the graph root and re-run `cluster apply`", - sidecar.graph_uri - ), - )); - outcome.pending_graphs.insert(sidecar.graph_id.clone()); - } - } -} - -async fn sweep_schema_apply_sidecar( - path: PathBuf, - sidecar: RecoverySidecar, - state: &mut ClusterState, - diagnostics: &mut Vec<Diagnostic>, - outcome: &mut SweepOutcome, -) { - let graph_address = graph_address(&sidecar.graph_id); - let schema_addr = schema_address(&sidecar.graph_id); - - // Digest-based classification: robust to unrelated manifest movement; - // the sidecar's version pins stay forensic. - let live_digest = match Omnigraph::open_read_only(&sidecar.graph_uri).await { - Ok(db) => sha256_hex(db.schema_source().as_bytes()), - Err(err) => { - // Cannot verify the interrupted operation — refuse to guess. - diagnostics.push(Diagnostic::warning( - "cluster_recovery_pending", - graph_address.clone(), - format!( - "an interrupted schema apply cannot be verified (graph '{}' did not open: {err}); graph-moving work is blocked until repaired", - sidecar.graph_uri - ), - )); - outcome.pending_graphs.insert(sidecar.graph_id.clone()); - return; - } - }; - - let recorded = state - .applied_revision - .resources - .get(&schema_addr) - .map(|resource| resource.digest.clone()); - if recorded.as_deref() == Some(live_digest.as_str()) { - // Ledger consistent with the live graph (the apply never landed, or - // landed and was recorded): the sidecar is stale intent — retire it. - outcome.completed_sidecars.push(path); - } else if live_digest == sidecar.desired_schema_digest { - // RFC-004 §D3 row 3: the schema apply completed on the graph; roll - // the cluster state forward to observable reality. - state.applied_revision.resources.insert( - schema_addr.clone(), - StateResource { - digest: live_digest.clone(), - applies_to: None, - }, - ); - let query_digests = state_query_digests_for_graph(state, &sidecar.graph_id); - let composite = graph_digest(&sidecar.graph_id, Some(&live_digest), Some(&query_digests)); - state - .applied_revision - .resources - .insert(graph_address.clone(), StateResource { digest: composite, applies_to: None }); - set_resource_status_applied(state, &graph_address); - set_resource_status_applied(state, &schema_addr); - state.recovery_records.insert( - sidecar.operation_id.clone(), - json!({ - "kind": "schema_apply", - "graph_id": sidecar.graph_id, - "outcome": "rolled_forward", - "recovered_at": now_rfc3339(), - "actor": sidecar.actor, - }), - ); - diagnostics.push(Diagnostic::warning( - "cluster_recovery_rolled_forward", - graph_address.clone(), - "an interrupted schema apply had completed on the graph; cluster state was rolled forward to match", - )); - outcome.completed_sidecars.push(path); - } else { - // Row 6: live schema is neither the recorded nor the desired digest. - set_resource_status( - state, - &graph_address, - ResourceLifecycleStatus::Drifted, - "actual_applied_state_pending", - "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", - ); - set_resource_status( - state, - &schema_addr, - ResourceLifecycleStatus::Drifted, - "actual_applied_state_pending", - "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", - ); - diagnostics.push(Diagnostic::warning( - "cluster_recovery_pending", - graph_address.clone(), - "an interrupted schema apply left unexpected graph state; graph-moving work is blocked until repaired", - )); - outcome.pending_graphs.insert(sidecar.graph_id.clone()); - } -} - -fn sweep_graph_delete_sidecar( - path: PathBuf, - sidecar: RecoverySidecar, - state: &mut ClusterState, - diagnostics: &mut Vec<Diagnostic>, - outcome: &mut SweepOutcome, -) { - let graph_address = graph_address(&sidecar.graph_id); - let root = PathBuf::from(&sidecar.graph_uri); - - if root.exists() { - // Row 8: the delete never completed. Prefix removal is idempotent and - // works on partial roots, so the repair is simply the re-proposed, - // still-approved delete on a later run — retire the stale intent. - diagnostics.push(Diagnostic::warning( - "graph_delete_incomplete", - graph_address, - "a previous graph delete did not complete; it will be re-proposed by plan and can be retried under its approval", - )); - outcome.completed_sidecars.push(path); - return; - } - - if !state.applied_revision.resources.contains_key(&graph_address) { - // Row 7: already tombstoned (or never recorded); crash fell between - // the state CAS and sidecar delete. - outcome.completed_sidecars.push(path); - return; - } - - // Row 7b: the root is gone, the ledger is stale — roll forward the - // tombstone, consume the approval the sidecar carries, audit. - tombstone_graph_subtree(state, &sidecar.graph_id, sidecar.approval_id.as_deref(), sidecar.actor.as_deref()); - state.recovery_records.insert( - sidecar.operation_id.clone(), - json!({ - "kind": "graph_delete", - "graph_id": sidecar.graph_id, - "outcome": "rolled_forward", - "recovered_at": now_rfc3339(), - "actor": sidecar.actor, - }), - ); - if let Some(approval_id) = &sidecar.approval_id { - record_approval_consumed(state, approval_id, &sidecar.operation_id); - outcome.consumed_approvals.push(approval_id.clone()); - } - diagnostics.push(Diagnostic::warning( - "cluster_recovery_rolled_forward", - graph_address, - "an interrupted graph delete had completed on disk; cluster state was rolled forward to match", - )); - outcome.completed_sidecars.push(path); -} - -/// Remove a graph's subtree (graph, schema, queries) from the ledger and -/// leave a tombstone observation. Idempotent. -fn tombstone_graph_subtree( - state: &mut ClusterState, - graph_id: &str, - approval_id: Option<&str>, - actor: Option<&str>, -) { - let graph_addr = graph_address(graph_id); - let schema_addr = schema_address(graph_id); - let query_prefix = format!("query.{graph_id}."); - state.applied_revision.resources.remove(&graph_addr); - state.applied_revision.resources.remove(&schema_addr); - state - .applied_revision - .resources - .retain(|address, _| !address.starts_with(&query_prefix)); - state.resource_statuses.remove(&graph_addr); - state.resource_statuses.remove(&schema_addr); - state - .resource_statuses - .retain(|address, _| !address.starts_with(&query_prefix)); - state.observations.insert( - graph_addr, - json!({ - "kind": "tombstone", - "deleted_at": now_rfc3339(), - "approval_id": approval_id, - "actor": actor, - }), - ); -} - -/// Record approval consumption in the state ledger. The artifact FILE is -/// rewritten with consumed_at only after the state write lands, so a failed -/// CAS leaves the approval valid for the retry. -fn record_approval_consumed(state: &mut ClusterState, approval_id: &str, operation_id: &str) { - state.approval_records.insert( - approval_id.to_string(), - json!({ - "consumed_at": now_rfc3339(), - "consumed_by_operation": operation_id, - }), - ); -} - -/// Mark approval artifact files consumed on disk (post-CAS). -fn mark_approvals_consumed(backend: &LocalStateBackend, approval_ids: &[String]) { - if approval_ids.is_empty() { - return; - } - let mut sink = Vec::new(); - for (_, mut artifact) in backend.list_approval_artifacts(&mut sink) { - if approval_ids.contains(&artifact.approval_id) && artifact.consumed_at.is_none() { - artifact.consumed_at = Some(now_rfc3339()); - let _ = backend.write_approval_artifact(&artifact); - } - } -} - -/// Read-only commands report pending sidecars without acting on them. -fn warn_pending_recovery_sidecars(config_dir: &Path, diagnostics: &mut Vec<Diagnostic>) { - let recoveries_dir = config_dir.join(CLUSTER_RECOVERIES_DIR); - let Ok(entries) = fs::read_dir(&recoveries_dir) else { - return; - }; - let mut names: Vec<String> = entries - .flatten() - .filter(|entry| entry.path().extension().is_some_and(|ext| ext == "json")) - .map(|entry| entry.file_name().to_string_lossy().into_owned()) - .collect(); - names.sort(); - for name in names { - diagnostics.push(Diagnostic::warning( - "cluster_recovery_pending", - format!("{CLUSTER_RECOVERIES_DIR}/{name}"), - "a recovery sidecar from an interrupted apply is pending; the next state-mutating command will classify it", - )); - } -} async fn observe_declared_graphs(desired: &DesiredCluster, state: &mut ClusterState) -> usize { let mut graph_error_count = 0; diff --git a/crates/omnigraph-cluster/src/sweep.rs b/crates/omnigraph-cluster/src/sweep.rs new file mode 100644 index 0000000..77ad8c5 --- /dev/null +++ b/crates/omnigraph-cluster/src/sweep.rs @@ -0,0 +1,386 @@ +//! The recovery sweep: RFC-004's roll-forward-only sidecar +//! classification (moved verbatim from lib.rs in the modularization). + +use super::*; + +/// Recovery sweep (RFC-004 §D3): runs at the start of every state-mutating +/// cluster command, under the state lock, before the command's own work. +/// Roll-forward-only — the engine's own sidecars make each graph-level +/// operation atomic within the graph, so the cluster never rolls a graph +/// back; it converges the ledger to observable reality or refuses loudly. +/// Mutations ride the calling command's CAS-checked state write; completed +/// sidecars are deleted only after that write lands. +pub(crate) async fn sweep_recovery_sidecars( + backend: &LocalStateBackend, + state: &mut ClusterState, + diagnostics: &mut Vec<Diagnostic>, +) -> SweepOutcome { + let mut outcome = SweepOutcome::default(); + for (path, sidecar) in backend.list_recovery_sidecars(diagnostics) { + match sidecar.kind { + RecoverySidecarKind::GraphCreate => { + sweep_graph_create_sidecar(path, sidecar, state, diagnostics, &mut outcome).await; + } + RecoverySidecarKind::SchemaApply => { + sweep_schema_apply_sidecar(path, sidecar, state, diagnostics, &mut outcome).await; + } + RecoverySidecarKind::GraphDelete => { + sweep_graph_delete_sidecar(path, sidecar, state, diagnostics, &mut outcome); + } + } + } + outcome +} + +pub(crate) async fn sweep_graph_create_sidecar( + path: PathBuf, + sidecar: RecoverySidecar, + state: &mut ClusterState, + diagnostics: &mut Vec<Diagnostic>, + outcome: &mut SweepOutcome, +) { + let graph_address = graph_address(&sidecar.graph_id); + let schema_addr = schema_address(&sidecar.graph_id); + let graph_path = PathBuf::from(&sidecar.graph_uri); + + // Row 1: nothing moved — the init never landed. The sidecar is pure + // intent; remove it and let the command's own plan re-propose the create. + if !graph_path.exists() { + let _ = fs::remove_file(&path); + return; + } + + match Omnigraph::open_read_only(&sidecar.graph_uri).await { + Ok(db) => { + let live_digest = sha256_hex(db.schema_source().as_bytes()); + let recorded = state + .applied_revision + .resources + .get(&schema_addr) + .map(|resource| resource.digest.clone()); + if recorded.as_deref() == Some(live_digest.as_str()) { + // Row 2: crash fell between the state CAS and sidecar delete. + outcome.completed_sidecars.push(path); + } else if live_digest == sidecar.desired_schema_digest { + // Row 4: the create completed on the graph; roll the cluster + // state forward to observable reality. + state.applied_revision.resources.insert( + schema_addr.clone(), + StateResource { + digest: live_digest.clone(), + applies_to: None, + }, + ); + let query_digests = state_query_digests_for_graph(state, &sidecar.graph_id); + let composite = + graph_digest(&sidecar.graph_id, Some(&live_digest), Some(&query_digests)); + state + .applied_revision + .resources + .insert(graph_address.clone(), StateResource { digest: composite, applies_to: None }); + set_resource_status_applied(state, &graph_address); + set_resource_status_applied(state, &schema_addr); + state.recovery_records.insert( + sidecar.operation_id.clone(), + json!({ + "kind": "graph_create", + "graph_id": sidecar.graph_id, + "outcome": "rolled_forward", + "recovered_at": now_rfc3339(), + "actor": sidecar.actor, + }), + ); + diagnostics.push(Diagnostic::warning( + "cluster_recovery_rolled_forward", + graph_address.clone(), + "an interrupted graph create had completed on the graph; cluster state was rolled forward to match", + )); + outcome.completed_sidecars.push(path); + } else { + // Row 6: the graph moved to something the sidecar did not + // intend. Refuse to guess; require refresh + operator re-plan. + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Drifted, + "actual_applied_state_pending", + "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", + ); + set_resource_status( + state, + &schema_addr, + ResourceLifecycleStatus::Drifted, + "actual_applied_state_pending", + "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", + ); + diagnostics.push(Diagnostic::warning( + "cluster_recovery_pending", + graph_address.clone(), + "an interrupted graph create left unexpected graph state; graph-moving work is blocked until repaired", + )); + outcome.pending_graphs.insert(sidecar.graph_id.clone()); + } + } + Err(err) => { + // Row 5: partial root (the engine's documented init gap). Never + // auto-delete — reconciler deletes are the same data-loss class + // as human deletes; the operator removes the root explicitly. + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Error, + "graph_create_incomplete", + "graph root exists but cannot be opened; remove the graph root and re-run `cluster apply`", + ); + set_resource_status( + state, + &schema_addr, + ResourceLifecycleStatus::Error, + "graph_create_incomplete", + "graph root exists but cannot be opened; remove the graph root and re-run `cluster apply`", + ); + diagnostics.push(Diagnostic::error( + "graph_create_incomplete", + graph_address.clone(), + format!( + "graph root '{}' exists but cannot be opened ({err}); remove the graph root and re-run `cluster apply`", + sidecar.graph_uri + ), + )); + outcome.pending_graphs.insert(sidecar.graph_id.clone()); + } + } +} + +pub(crate) async fn sweep_schema_apply_sidecar( + path: PathBuf, + sidecar: RecoverySidecar, + state: &mut ClusterState, + diagnostics: &mut Vec<Diagnostic>, + outcome: &mut SweepOutcome, +) { + let graph_address = graph_address(&sidecar.graph_id); + let schema_addr = schema_address(&sidecar.graph_id); + + // Digest-based classification: robust to unrelated manifest movement; + // the sidecar's version pins stay forensic. + let live_digest = match Omnigraph::open_read_only(&sidecar.graph_uri).await { + Ok(db) => sha256_hex(db.schema_source().as_bytes()), + Err(err) => { + // Cannot verify the interrupted operation — refuse to guess. + diagnostics.push(Diagnostic::warning( + "cluster_recovery_pending", + graph_address.clone(), + format!( + "an interrupted schema apply cannot be verified (graph '{}' did not open: {err}); graph-moving work is blocked until repaired", + sidecar.graph_uri + ), + )); + outcome.pending_graphs.insert(sidecar.graph_id.clone()); + return; + } + }; + + let recorded = state + .applied_revision + .resources + .get(&schema_addr) + .map(|resource| resource.digest.clone()); + if recorded.as_deref() == Some(live_digest.as_str()) { + // Ledger consistent with the live graph (the apply never landed, or + // landed and was recorded): the sidecar is stale intent — retire it. + outcome.completed_sidecars.push(path); + } else if live_digest == sidecar.desired_schema_digest { + // RFC-004 §D3 row 3: the schema apply completed on the graph; roll + // the cluster state forward to observable reality. + state.applied_revision.resources.insert( + schema_addr.clone(), + StateResource { + digest: live_digest.clone(), + applies_to: None, + }, + ); + let query_digests = state_query_digests_for_graph(state, &sidecar.graph_id); + let composite = graph_digest(&sidecar.graph_id, Some(&live_digest), Some(&query_digests)); + state + .applied_revision + .resources + .insert(graph_address.clone(), StateResource { digest: composite, applies_to: None }); + set_resource_status_applied(state, &graph_address); + set_resource_status_applied(state, &schema_addr); + state.recovery_records.insert( + sidecar.operation_id.clone(), + json!({ + "kind": "schema_apply", + "graph_id": sidecar.graph_id, + "outcome": "rolled_forward", + "recovered_at": now_rfc3339(), + "actor": sidecar.actor, + }), + ); + diagnostics.push(Diagnostic::warning( + "cluster_recovery_rolled_forward", + graph_address.clone(), + "an interrupted schema apply had completed on the graph; cluster state was rolled forward to match", + )); + outcome.completed_sidecars.push(path); + } else { + // Row 6: live schema is neither the recorded nor the desired digest. + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Drifted, + "actual_applied_state_pending", + "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", + ); + set_resource_status( + state, + &schema_addr, + ResourceLifecycleStatus::Drifted, + "actual_applied_state_pending", + "graph state does not match the interrupted operation; run `cluster refresh` and re-plan", + ); + diagnostics.push(Diagnostic::warning( + "cluster_recovery_pending", + graph_address.clone(), + "an interrupted schema apply left unexpected graph state; graph-moving work is blocked until repaired", + )); + outcome.pending_graphs.insert(sidecar.graph_id.clone()); + } +} + +pub(crate) fn sweep_graph_delete_sidecar( + path: PathBuf, + sidecar: RecoverySidecar, + state: &mut ClusterState, + diagnostics: &mut Vec<Diagnostic>, + outcome: &mut SweepOutcome, +) { + let graph_address = graph_address(&sidecar.graph_id); + let root = PathBuf::from(&sidecar.graph_uri); + + if root.exists() { + // Row 8: the delete never completed. Prefix removal is idempotent and + // works on partial roots, so the repair is simply the re-proposed, + // still-approved delete on a later run — retire the stale intent. + diagnostics.push(Diagnostic::warning( + "graph_delete_incomplete", + graph_address, + "a previous graph delete did not complete; it will be re-proposed by plan and can be retried under its approval", + )); + outcome.completed_sidecars.push(path); + return; + } + + if !state.applied_revision.resources.contains_key(&graph_address) { + // Row 7: already tombstoned (or never recorded); crash fell between + // the state CAS and sidecar delete. + outcome.completed_sidecars.push(path); + return; + } + + // Row 7b: the root is gone, the ledger is stale — roll forward the + // tombstone, consume the approval the sidecar carries, audit. + tombstone_graph_subtree(state, &sidecar.graph_id, sidecar.approval_id.as_deref(), sidecar.actor.as_deref()); + state.recovery_records.insert( + sidecar.operation_id.clone(), + json!({ + "kind": "graph_delete", + "graph_id": sidecar.graph_id, + "outcome": "rolled_forward", + "recovered_at": now_rfc3339(), + "actor": sidecar.actor, + }), + ); + if let Some(approval_id) = &sidecar.approval_id { + record_approval_consumed(state, approval_id, &sidecar.operation_id); + outcome.consumed_approvals.push(approval_id.clone()); + } + diagnostics.push(Diagnostic::warning( + "cluster_recovery_rolled_forward", + graph_address, + "an interrupted graph delete had completed on disk; cluster state was rolled forward to match", + )); + outcome.completed_sidecars.push(path); +} + +/// Remove a graph's subtree (graph, schema, queries) from the ledger and +/// leave a tombstone observation. Idempotent. +pub(crate) fn tombstone_graph_subtree( + state: &mut ClusterState, + graph_id: &str, + approval_id: Option<&str>, + actor: Option<&str>, +) { + let graph_addr = graph_address(graph_id); + let schema_addr = schema_address(graph_id); + let query_prefix = format!("query.{graph_id}."); + state.applied_revision.resources.remove(&graph_addr); + state.applied_revision.resources.remove(&schema_addr); + state + .applied_revision + .resources + .retain(|address, _| !address.starts_with(&query_prefix)); + state.resource_statuses.remove(&graph_addr); + state.resource_statuses.remove(&schema_addr); + state + .resource_statuses + .retain(|address, _| !address.starts_with(&query_prefix)); + state.observations.insert( + graph_addr, + json!({ + "kind": "tombstone", + "deleted_at": now_rfc3339(), + "approval_id": approval_id, + "actor": actor, + }), + ); +} + +/// Record approval consumption in the state ledger. The artifact FILE is +/// rewritten with consumed_at only after the state write lands, so a failed +/// CAS leaves the approval valid for the retry. +pub(crate) fn record_approval_consumed(state: &mut ClusterState, approval_id: &str, operation_id: &str) { + state.approval_records.insert( + approval_id.to_string(), + json!({ + "consumed_at": now_rfc3339(), + "consumed_by_operation": operation_id, + }), + ); +} + +/// Mark approval artifact files consumed on disk (post-CAS). +pub(crate) fn mark_approvals_consumed(backend: &LocalStateBackend, approval_ids: &[String]) { + if approval_ids.is_empty() { + return; + } + let mut sink = Vec::new(); + for (_, mut artifact) in backend.list_approval_artifacts(&mut sink) { + if approval_ids.contains(&artifact.approval_id) && artifact.consumed_at.is_none() { + artifact.consumed_at = Some(now_rfc3339()); + let _ = backend.write_approval_artifact(&artifact); + } + } +} + +/// Read-only commands report pending sidecars without acting on them. +pub(crate) fn warn_pending_recovery_sidecars(config_dir: &Path, diagnostics: &mut Vec<Diagnostic>) { + let recoveries_dir = config_dir.join(CLUSTER_RECOVERIES_DIR); + let Ok(entries) = fs::read_dir(&recoveries_dir) else { + return; + }; + let mut names: Vec<String> = entries + .flatten() + .filter(|entry| entry.path().extension().is_some_and(|ext| ext == "json")) + .map(|entry| entry.file_name().to_string_lossy().into_owned()) + .collect(); + names.sort(); + for name in names { + diagnostics.push(Diagnostic::warning( + "cluster_recovery_pending", + format!("{CLUSTER_RECOVERIES_DIR}/{name}"), + "a recovery sidecar from an interrupted apply is pending; the next state-mutating command will classify it", + )); + } +} From dd17c0c50f1ebd8a350373b5df4c436d5d1fb578 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 05:33:13 +0300 Subject: [PATCH 105/165] refactor(cluster): move diffing and classification to diff.rs Verbatim move of diff_resources, binding-change diffing, blast radius, approval gating, ResourceKind, classify_changes, and demotion. 95 tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/diff.rs | 420 +++++++++++++++++++++++++++ crates/omnigraph-cluster/src/lib.rs | 414 +------------------------- 2 files changed, 422 insertions(+), 412 deletions(-) create mode 100644 crates/omnigraph-cluster/src/diff.rs diff --git a/crates/omnigraph-cluster/src/diff.rs b/crates/omnigraph-cluster/src/diff.rs new file mode 100644 index 0000000..e75db4d --- /dev/null +++ b/crates/omnigraph-cluster/src/diff.rs @@ -0,0 +1,420 @@ +//! Plan/apply classification: resource diffing, dispositions, approval +//! gating, demotion (moved verbatim from lib.rs in the modularization). + +use super::*; + +pub(crate) fn diff_resources( + prior: &BTreeMap<String, String>, + desired: &BTreeMap<String, String>, +) -> Vec<PlanChange> { + let mut changes = Vec::new(); + for (address, after) in desired { + match prior.get(address) { + None => changes.push(PlanChange { + resource: address.clone(), + operation: PlanOperation::Create, + before_digest: None, + after_digest: Some(after.clone()), + disposition: None, + reason: None, + binding_change: false, + migration: None, + }), + Some(before) if before != after => changes.push(PlanChange { + resource: address.clone(), + operation: PlanOperation::Update, + before_digest: Some(before.clone()), + after_digest: Some(after.clone()), + disposition: None, + reason: None, + binding_change: false, + migration: None, + }), + Some(_) => {} + } + } + for (address, before) in prior { + if !desired.contains_key(address) { + changes.push(PlanChange { + resource: address.clone(), + operation: PlanOperation::Delete, + before_digest: Some(before.clone()), + after_digest: None, + disposition: None, + reason: None, + binding_change: false, + migration: None, + }); + } + } + changes.sort_by(|a, b| a.resource.cmp(&b.resource)); + changes +} + +/// Binding-only policy changes: the file digest is unchanged (so +/// `diff_resources` saw nothing) but the applied `applies_to` differs from +/// the desired bindings — including the pre-5A case where the state entry +/// has no bindings recorded yet. These are first-class plan changes: without +/// this pass a binding edit would silently rot or silently converge. +pub(crate) fn append_policy_binding_changes( + changes: &mut Vec<PlanChange>, + prior_state: Option<&ClusterState>, + desired: &DesiredCluster, +) { + let Some(state) = prior_state else { + return; // no state: everything is already a Create carrying bindings + }; + for (address, desired_bindings) in &desired.policy_bindings { + if changes.iter().any(|change| &change.resource == address) { + continue; // content change already covers it + } + let Some(entry) = state.applied_revision.resources.get(address) else { + continue; // not applied yet: the Create covers it + }; + if entry.applies_to.as_ref() == Some(desired_bindings) { + continue; + } + changes.push(PlanChange { + resource: address.clone(), + operation: PlanOperation::Update, + before_digest: Some(entry.digest.clone()), + after_digest: Some(entry.digest.clone()), + disposition: None, + reason: None, + binding_change: true, + migration: None, + }); + } + changes.sort_by(|a, b| a.resource.cmp(&b.resource)); +} + +pub(crate) fn compute_blast_radius( + changes: &[PlanChange], + dependencies: &[Dependency], +) -> Vec<BlastRadius> { + changes + .iter() + .filter_map(|change| { + let affected: Vec<_> = dependencies + .iter() + .filter_map(|dep| (dep.to == change.resource).then_some(dep.from.clone())) + .collect(); + (!affected.is_empty()).then(|| BlastRadius { + resource: change.resource.clone(), + affected, + }) + }) + .collect() +} + +pub(crate) fn compute_approvals( + changes: &[PlanChange], + approved: &BTreeSet<String>, +) -> Vec<ApprovalRequirement> { + // One gate per subtree: the graph.<id> delete carries its schema and + // queries, so a schema delete whose graph is also deleted is not listed. + let graph_deletes: BTreeSet<String> = changes + .iter() + .filter(|change| change.operation == PlanOperation::Delete) + .filter_map(|change| change.resource.strip_prefix("graph.").map(str::to_string)) + .collect(); + changes + .iter() + .filter_map(|change| { + if change.operation != PlanOperation::Delete { + return None; + } + let gated = match resource_kind(&change.resource) { + ResourceKind::Graph(_) => true, + ResourceKind::Schema(graph) => !graph_deletes.contains(&graph), + _ => false, + }; + gated.then(|| ApprovalRequirement { + resource: change.resource.clone(), + reason: "delete may remove deployed graph or schema definition".to_string(), + satisfied: approved.contains(&change.resource), + }) + }) + .collect() +} + +/// Resources with a valid (digest-matching, unconsumed) pending approval. +/// Near-misses — an artifact for the same resource whose bound digests no +/// longer match — warn as `approval_stale` and never authorize anything. +pub(crate) fn approved_resources( + artifacts: &[(PathBuf, ApprovalArtifact)], + changes: &[PlanChange], + config_digest: &str, + diagnostics: &mut Vec<Diagnostic>, +) -> BTreeSet<String> { + let mut approved = BTreeSet::new(); + for change in changes { + let candidates: Vec<&ApprovalArtifact> = artifacts + .iter() + .map(|(_, artifact)| artifact) + .filter(|artifact| artifact.consumed_at.is_none() && artifact.resource == change.resource) + .collect(); + if candidates.is_empty() { + continue; + } + let matched = candidates.iter().any(|artifact| { + artifact.bound_config_digest == config_digest + && artifact.bound_before_digest == change.before_digest + && artifact.bound_after_digest == change.after_digest + }); + if matched { + approved.insert(change.resource.clone()); + } else { + diagnostics.push(Diagnostic::warning( + "approval_stale", + change.resource.clone(), + "an approval artifact exists but its bound digests no longer match the plan; re-run `cluster approve`", + )); + } + } + approved +} + +#[derive(Debug, PartialEq, Eq)] +pub(crate) enum ResourceKind { + Graph(String), + Schema(String), + Query { graph: String, name: String }, + Policy(String), + Unknown, +} + +pub(crate) fn resource_kind(address: &str) -> ResourceKind { + if let Some(graph) = address.strip_prefix("graph.") { + ResourceKind::Graph(graph.to_string()) + } else if let Some(graph) = address.strip_prefix("schema.") { + ResourceKind::Schema(graph.to_string()) + } else if let Some(rest) = address.strip_prefix("query.") { + match rest.split_once('.') { + Some((graph, name)) => ResourceKind::Query { + graph: graph.to_string(), + name: name.to_string(), + }, + None => ResourceKind::Unknown, + } + } else if let Some(name) = address.strip_prefix("policy.") { + ResourceKind::Policy(name.to_string()) + } else { + ResourceKind::Unknown + } +} + +/// Classify every planned change with the disposition config-only apply gives +/// it. Stage 3A executes only query/policy catalog writes; graph/schema +/// movement is a later phase, and `graph.<id>` composite updates whose schema +/// component is unchanged converge automatically once query digests land. +pub(crate) fn classify_changes( + changes: &mut [PlanChange], + dependencies: &[Dependency], + pending_recovery: &BTreeSet<String>, + approved: &BTreeSet<String>, +) { + let mut schema_creates = BTreeSet::new(); + let mut schema_pending = BTreeSet::new(); + let mut graph_creates = BTreeSet::new(); + let mut graph_deletes = BTreeSet::new(); + for change in changes.iter() { + match resource_kind(&change.resource) { + ResourceKind::Schema(graph) => match change.operation { + PlanOperation::Create => { + schema_creates.insert(graph); + } + // Schema updates execute in-run before catalog writes (4B) + // and no longer block dependents; deletes (4C) still do. + PlanOperation::Update => {} + PlanOperation::Delete => { + schema_pending.insert(graph); + } + }, + ResourceKind::Graph(graph) => match change.operation { + PlanOperation::Create => { + graph_creates.insert(graph); + } + PlanOperation::Delete => { + graph_deletes.insert(graph); + } + PlanOperation::Update => {} + }, + _ => {} + } + } + // A schema Create is satisfied by its paired graph create (the init + // carries the schema); a standalone schema Create stays pending. + for graph in &schema_creates { + if !graph_creates.contains(graph) { + schema_pending.insert(graph.clone()); + } + } + // Subtree deletes ride the approved graph delete. + let rides_approved_delete = |graph: &str| { + graph_deletes.contains(graph) + && approved.contains(&graph_address(graph)) + && !pending_recovery.contains(graph) + }; + + for change in changes.iter_mut() { + let (disposition, reason) = match resource_kind(&change.resource) { + ResourceKind::Schema(graph) => match change.operation { + PlanOperation::Create + if graph_creates.contains(&graph) + && !pending_recovery.contains(&graph) => + { + // Applied with the graph create — the init carries it. + (ApplyDisposition::Applied, None) + } + PlanOperation::Update if !pending_recovery.contains(&graph) => { + // Stage 4B: schema updates execute via the engine's + // schema apply (soft drops only; allow_data_loss is 4C). + (ApplyDisposition::Applied, None) + } + PlanOperation::Create | PlanOperation::Update => { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) + } + PlanOperation::Delete if graph_deletes.contains(&graph) => { + if rides_approved_delete(&graph) { + (ApplyDisposition::Applied, None) + } else if pending_recovery.contains(&graph) { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) + } else { + (ApplyDisposition::Blocked, Some("approval_required")) + } + } + _ => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), + }, + ResourceKind::Graph(graph) => match change.operation { + PlanOperation::Create => { + if pending_recovery.contains(&graph) { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) + } else { + (ApplyDisposition::Applied, None) + } + } + PlanOperation::Update if !schema_pending.contains(&graph) => { + (ApplyDisposition::Derived, None) + } + // Stage 4C: an approved graph delete executes (the + // irreversible tier — gated by a digest-bound artifact). + PlanOperation::Delete => { + if pending_recovery.contains(&graph) { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) + } else if rides_approved_delete(&graph) { + (ApplyDisposition::Applied, None) + } else { + (ApplyDisposition::Blocked, Some("approval_required")) + } + } + _ => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), + }, + ResourceKind::Query { graph, .. } => match change.operation { + PlanOperation::Delete => { + if rides_approved_delete(&graph) { + // Tombstoned with the approved graph delete. + (ApplyDisposition::Applied, None) + } else if graph_deletes.contains(&graph) { + (ApplyDisposition::Blocked, Some("approval_required")) + } else { + (ApplyDisposition::Applied, None) + } + } + PlanOperation::Create | PlanOperation::Update => { + if pending_recovery.contains(&graph) { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) + } else if schema_pending.contains(&graph) { + ( + ApplyDisposition::Blocked, + Some("dependency_not_applied"), + ) + } else { + // A graph create in the same plan no longer blocks: + // creates execute first in the same apply run. + (ApplyDisposition::Applied, None) + } + } + }, + ResourceKind::Policy(_) => match change.operation { + PlanOperation::Delete => (ApplyDisposition::Applied, None), + PlanOperation::Create | PlanOperation::Update => { + let blocked_pending = dependencies.iter().any(|dep| { + dep.from == change.resource + && dep + .to + .strip_prefix("graph.") + .is_some_and(|graph| pending_recovery.contains(graph)) + }); + if blocked_pending { + (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) + } else { + (ApplyDisposition::Applied, None) + } + } + }, + ResourceKind::Unknown => { + (ApplyDisposition::Deferred, Some("apply_unsupported_kind")) + } + }; + change.disposition = Some(disposition); + change.reason = reason.map(str::to_string); + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum FailedGraphOrigin { + GraphCreate, + SchemaApply, + GraphDelete, +} + +/// After a graph-moving operation fails mid-run, every change that depended +/// on that graph flips from Applied to Blocked so the output and the +/// persisted statuses tell the truth about what this run actually executed. +/// The originating change carries the failure code; dependents carry +/// `dependency_not_applied`. +pub(crate) fn demote_dependents_of_failed_graphs( + changes: &mut [PlanChange], + failed: &BTreeMap<String, FailedGraphOrigin>, + dependencies: &[Dependency], +) { + for change in changes.iter_mut() { + if change.disposition != Some(ApplyDisposition::Applied) { + continue; + } + let demote_reason = match resource_kind(&change.resource) { + ResourceKind::Graph(graph) => match failed.get(&graph) { + Some(FailedGraphOrigin::GraphCreate) => Some("graph_create_failed"), + Some(FailedGraphOrigin::GraphDelete) => Some("graph_delete_failed"), + Some(FailedGraphOrigin::SchemaApply) => Some("dependency_not_applied"), + None => None, + }, + ResourceKind::Schema(graph) => match failed.get(&graph) { + Some(FailedGraphOrigin::SchemaApply) => Some("schema_apply_failed"), + Some(FailedGraphOrigin::GraphCreate) | Some(FailedGraphOrigin::GraphDelete) => { + Some("dependency_not_applied") + } + None => None, + }, + ResourceKind::Query { graph, .. } if failed.contains_key(&graph) => { + Some("dependency_not_applied") + } + ResourceKind::Policy(_) => { + let blocked = dependencies.iter().any(|dep| { + dep.from == change.resource + && dep + .to + .strip_prefix("graph.") + .is_some_and(|graph| failed.contains_key(graph)) + }); + blocked.then_some("dependency_not_applied") + } + _ => None, + }; + if let Some(reason) = demote_reason { + change.disposition = Some(ApplyDisposition::Blocked); + change.reason = Some(reason.to_string()); + } + } +} diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 3faacaa..c54d245 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -19,12 +19,14 @@ use ulid::Ulid; pub mod failpoints; +mod diff; mod serve; mod sweep; mod store; use store::{LocalStateBackend, StateLockGuard, StateSnapshot}; pub use serve::{ServingGraph, ServingPolicy, ServingQuery, ServingSnapshot, read_serving_snapshot}; use serve::read_verified_payload; +use diff::{FailedGraphOrigin, ResourceKind, append_policy_binding_changes, approved_resources, classify_changes, compute_approvals, compute_blast_radius, demote_dependents_of_failed_graphs, diff_resources, resource_kind}; use sweep::{mark_approvals_consumed, record_approval_consumed, sweep_recovery_sidecars, tombstone_graph_subtree, warn_pending_recovery_sidecars}; pub const CLUSTER_CONFIG_FILE: &str = "cluster.yaml"; @@ -2856,418 +2858,6 @@ fn validate_query_source( } } -fn diff_resources( - prior: &BTreeMap<String, String>, - desired: &BTreeMap<String, String>, -) -> Vec<PlanChange> { - let mut changes = Vec::new(); - for (address, after) in desired { - match prior.get(address) { - None => changes.push(PlanChange { - resource: address.clone(), - operation: PlanOperation::Create, - before_digest: None, - after_digest: Some(after.clone()), - disposition: None, - reason: None, - binding_change: false, - migration: None, - }), - Some(before) if before != after => changes.push(PlanChange { - resource: address.clone(), - operation: PlanOperation::Update, - before_digest: Some(before.clone()), - after_digest: Some(after.clone()), - disposition: None, - reason: None, - binding_change: false, - migration: None, - }), - Some(_) => {} - } - } - for (address, before) in prior { - if !desired.contains_key(address) { - changes.push(PlanChange { - resource: address.clone(), - operation: PlanOperation::Delete, - before_digest: Some(before.clone()), - after_digest: None, - disposition: None, - reason: None, - binding_change: false, - migration: None, - }); - } - } - changes.sort_by(|a, b| a.resource.cmp(&b.resource)); - changes -} - -/// Binding-only policy changes: the file digest is unchanged (so -/// `diff_resources` saw nothing) but the applied `applies_to` differs from -/// the desired bindings — including the pre-5A case where the state entry -/// has no bindings recorded yet. These are first-class plan changes: without -/// this pass a binding edit would silently rot or silently converge. -fn append_policy_binding_changes( - changes: &mut Vec<PlanChange>, - prior_state: Option<&ClusterState>, - desired: &DesiredCluster, -) { - let Some(state) = prior_state else { - return; // no state: everything is already a Create carrying bindings - }; - for (address, desired_bindings) in &desired.policy_bindings { - if changes.iter().any(|change| &change.resource == address) { - continue; // content change already covers it - } - let Some(entry) = state.applied_revision.resources.get(address) else { - continue; // not applied yet: the Create covers it - }; - if entry.applies_to.as_ref() == Some(desired_bindings) { - continue; - } - changes.push(PlanChange { - resource: address.clone(), - operation: PlanOperation::Update, - before_digest: Some(entry.digest.clone()), - after_digest: Some(entry.digest.clone()), - disposition: None, - reason: None, - binding_change: true, - migration: None, - }); - } - changes.sort_by(|a, b| a.resource.cmp(&b.resource)); -} - -fn compute_blast_radius(changes: &[PlanChange], dependencies: &[Dependency]) -> Vec<BlastRadius> { - changes - .iter() - .filter_map(|change| { - let affected: Vec<_> = dependencies - .iter() - .filter_map(|dep| (dep.to == change.resource).then_some(dep.from.clone())) - .collect(); - (!affected.is_empty()).then(|| BlastRadius { - resource: change.resource.clone(), - affected, - }) - }) - .collect() -} - -fn compute_approvals( - changes: &[PlanChange], - approved: &BTreeSet<String>, -) -> Vec<ApprovalRequirement> { - // One gate per subtree: the graph.<id> delete carries its schema and - // queries, so a schema delete whose graph is also deleted is not listed. - let graph_deletes: BTreeSet<String> = changes - .iter() - .filter(|change| change.operation == PlanOperation::Delete) - .filter_map(|change| change.resource.strip_prefix("graph.").map(str::to_string)) - .collect(); - changes - .iter() - .filter_map(|change| { - if change.operation != PlanOperation::Delete { - return None; - } - let gated = match resource_kind(&change.resource) { - ResourceKind::Graph(_) => true, - ResourceKind::Schema(graph) => !graph_deletes.contains(&graph), - _ => false, - }; - gated.then(|| ApprovalRequirement { - resource: change.resource.clone(), - reason: "delete may remove deployed graph or schema definition".to_string(), - satisfied: approved.contains(&change.resource), - }) - }) - .collect() -} - -/// Resources with a valid (digest-matching, unconsumed) pending approval. -/// Near-misses — an artifact for the same resource whose bound digests no -/// longer match — warn as `approval_stale` and never authorize anything. -fn approved_resources( - artifacts: &[(PathBuf, ApprovalArtifact)], - changes: &[PlanChange], - config_digest: &str, - diagnostics: &mut Vec<Diagnostic>, -) -> BTreeSet<String> { - let mut approved = BTreeSet::new(); - for change in changes { - let candidates: Vec<&ApprovalArtifact> = artifacts - .iter() - .map(|(_, artifact)| artifact) - .filter(|artifact| artifact.consumed_at.is_none() && artifact.resource == change.resource) - .collect(); - if candidates.is_empty() { - continue; - } - let matched = candidates.iter().any(|artifact| { - artifact.bound_config_digest == config_digest - && artifact.bound_before_digest == change.before_digest - && artifact.bound_after_digest == change.after_digest - }); - if matched { - approved.insert(change.resource.clone()); - } else { - diagnostics.push(Diagnostic::warning( - "approval_stale", - change.resource.clone(), - "an approval artifact exists but its bound digests no longer match the plan; re-run `cluster approve`", - )); - } - } - approved -} - -#[derive(Debug, PartialEq, Eq)] -enum ResourceKind { - Graph(String), - Schema(String), - Query { graph: String, name: String }, - Policy(String), - Unknown, -} - -fn resource_kind(address: &str) -> ResourceKind { - if let Some(graph) = address.strip_prefix("graph.") { - ResourceKind::Graph(graph.to_string()) - } else if let Some(graph) = address.strip_prefix("schema.") { - ResourceKind::Schema(graph.to_string()) - } else if let Some(rest) = address.strip_prefix("query.") { - match rest.split_once('.') { - Some((graph, name)) => ResourceKind::Query { - graph: graph.to_string(), - name: name.to_string(), - }, - None => ResourceKind::Unknown, - } - } else if let Some(name) = address.strip_prefix("policy.") { - ResourceKind::Policy(name.to_string()) - } else { - ResourceKind::Unknown - } -} - -/// Classify every planned change with the disposition config-only apply gives -/// it. Stage 3A executes only query/policy catalog writes; graph/schema -/// movement is a later phase, and `graph.<id>` composite updates whose schema -/// component is unchanged converge automatically once query digests land. -fn classify_changes( - changes: &mut [PlanChange], - dependencies: &[Dependency], - pending_recovery: &BTreeSet<String>, - approved: &BTreeSet<String>, -) { - let mut schema_creates = BTreeSet::new(); - let mut schema_pending = BTreeSet::new(); - let mut graph_creates = BTreeSet::new(); - let mut graph_deletes = BTreeSet::new(); - for change in changes.iter() { - match resource_kind(&change.resource) { - ResourceKind::Schema(graph) => match change.operation { - PlanOperation::Create => { - schema_creates.insert(graph); - } - // Schema updates execute in-run before catalog writes (4B) - // and no longer block dependents; deletes (4C) still do. - PlanOperation::Update => {} - PlanOperation::Delete => { - schema_pending.insert(graph); - } - }, - ResourceKind::Graph(graph) => match change.operation { - PlanOperation::Create => { - graph_creates.insert(graph); - } - PlanOperation::Delete => { - graph_deletes.insert(graph); - } - PlanOperation::Update => {} - }, - _ => {} - } - } - // A schema Create is satisfied by its paired graph create (the init - // carries the schema); a standalone schema Create stays pending. - for graph in &schema_creates { - if !graph_creates.contains(graph) { - schema_pending.insert(graph.clone()); - } - } - // Subtree deletes ride the approved graph delete. - let rides_approved_delete = |graph: &str| { - graph_deletes.contains(graph) - && approved.contains(&graph_address(graph)) - && !pending_recovery.contains(graph) - }; - - for change in changes.iter_mut() { - let (disposition, reason) = match resource_kind(&change.resource) { - ResourceKind::Schema(graph) => match change.operation { - PlanOperation::Create - if graph_creates.contains(&graph) - && !pending_recovery.contains(&graph) => - { - // Applied with the graph create — the init carries it. - (ApplyDisposition::Applied, None) - } - PlanOperation::Update if !pending_recovery.contains(&graph) => { - // Stage 4B: schema updates execute via the engine's - // schema apply (soft drops only; allow_data_loss is 4C). - (ApplyDisposition::Applied, None) - } - PlanOperation::Create | PlanOperation::Update => { - (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) - } - PlanOperation::Delete if graph_deletes.contains(&graph) => { - if rides_approved_delete(&graph) { - (ApplyDisposition::Applied, None) - } else if pending_recovery.contains(&graph) { - (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) - } else { - (ApplyDisposition::Blocked, Some("approval_required")) - } - } - _ => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), - }, - ResourceKind::Graph(graph) => match change.operation { - PlanOperation::Create => { - if pending_recovery.contains(&graph) { - (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) - } else { - (ApplyDisposition::Applied, None) - } - } - PlanOperation::Update if !schema_pending.contains(&graph) => { - (ApplyDisposition::Derived, None) - } - // Stage 4C: an approved graph delete executes (the - // irreversible tier — gated by a digest-bound artifact). - PlanOperation::Delete => { - if pending_recovery.contains(&graph) { - (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) - } else if rides_approved_delete(&graph) { - (ApplyDisposition::Applied, None) - } else { - (ApplyDisposition::Blocked, Some("approval_required")) - } - } - _ => (ApplyDisposition::Deferred, Some("apply_unsupported_kind")), - }, - ResourceKind::Query { graph, .. } => match change.operation { - PlanOperation::Delete => { - if rides_approved_delete(&graph) { - // Tombstoned with the approved graph delete. - (ApplyDisposition::Applied, None) - } else if graph_deletes.contains(&graph) { - (ApplyDisposition::Blocked, Some("approval_required")) - } else { - (ApplyDisposition::Applied, None) - } - } - PlanOperation::Create | PlanOperation::Update => { - if pending_recovery.contains(&graph) { - (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) - } else if schema_pending.contains(&graph) { - ( - ApplyDisposition::Blocked, - Some("dependency_not_applied"), - ) - } else { - // A graph create in the same plan no longer blocks: - // creates execute first in the same apply run. - (ApplyDisposition::Applied, None) - } - } - }, - ResourceKind::Policy(_) => match change.operation { - PlanOperation::Delete => (ApplyDisposition::Applied, None), - PlanOperation::Create | PlanOperation::Update => { - let blocked_pending = dependencies.iter().any(|dep| { - dep.from == change.resource - && dep - .to - .strip_prefix("graph.") - .is_some_and(|graph| pending_recovery.contains(graph)) - }); - if blocked_pending { - (ApplyDisposition::Blocked, Some("cluster_recovery_pending")) - } else { - (ApplyDisposition::Applied, None) - } - } - }, - ResourceKind::Unknown => { - (ApplyDisposition::Deferred, Some("apply_unsupported_kind")) - } - }; - change.disposition = Some(disposition); - change.reason = reason.map(str::to_string); - } -} - -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -enum FailedGraphOrigin { - GraphCreate, - SchemaApply, - GraphDelete, -} - -/// After a graph-moving operation fails mid-run, every change that depended -/// on that graph flips from Applied to Blocked so the output and the -/// persisted statuses tell the truth about what this run actually executed. -/// The originating change carries the failure code; dependents carry -/// `dependency_not_applied`. -fn demote_dependents_of_failed_graphs( - changes: &mut [PlanChange], - failed: &BTreeMap<String, FailedGraphOrigin>, - dependencies: &[Dependency], -) { - for change in changes.iter_mut() { - if change.disposition != Some(ApplyDisposition::Applied) { - continue; - } - let demote_reason = match resource_kind(&change.resource) { - ResourceKind::Graph(graph) => match failed.get(&graph) { - Some(FailedGraphOrigin::GraphCreate) => Some("graph_create_failed"), - Some(FailedGraphOrigin::GraphDelete) => Some("graph_delete_failed"), - Some(FailedGraphOrigin::SchemaApply) => Some("dependency_not_applied"), - None => None, - }, - ResourceKind::Schema(graph) => match failed.get(&graph) { - Some(FailedGraphOrigin::SchemaApply) => Some("schema_apply_failed"), - Some(FailedGraphOrigin::GraphCreate) | Some(FailedGraphOrigin::GraphDelete) => { - Some("dependency_not_applied") - } - None => None, - }, - ResourceKind::Query { graph, .. } if failed.contains_key(&graph) => { - Some("dependency_not_applied") - } - ResourceKind::Policy(_) => { - let blocked = dependencies.iter().any(|dep| { - dep.from == change.resource - && dep - .to - .strip_prefix("graph.") - .is_some_and(|graph| failed.contains_key(graph)) - }); - blocked.then_some("dependency_not_applied") - } - _ => None, - }; - if let Some(reason) = demote_reason { - change.disposition = Some(ApplyDisposition::Blocked); - change.reason = Some(reason.to_string()); - } - } -} /// Content-addressed catalog path for an applied resource payload. Extensions /// are fixed per kind (`.gq` / `.yaml`) regardless of the source file's name, From dc0a1fc5a5720d44f78a59f3581f2408d8c79705 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 05:37:20 +0300 Subject: [PATCH 106/165] refactor(cluster): move declared-config loading to config.rs Verbatim move of cluster.yaml parsing, query discovery, source digesting, header/id validation, path resolution, and live-graph observation. Two helpers that the cut swept along were relocated to their right homes (state-status helpers back to lib.rs, lock-file helpers to store.rs). 95 tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/config.rs | 881 ++++++++++++++++++++++ crates/omnigraph-cluster/src/lib.rs | 971 +------------------------ crates/omnigraph-cluster/src/store.rs | 27 + 3 files changed, 947 insertions(+), 932 deletions(-) create mode 100644 crates/omnigraph-cluster/src/config.rs diff --git a/crates/omnigraph-cluster/src/config.rs b/crates/omnigraph-cluster/src/config.rs new file mode 100644 index 0000000..ecdc71c --- /dev/null +++ b/crates/omnigraph-cluster/src/config.rs @@ -0,0 +1,881 @@ +//! Declared-configuration loading: cluster.yaml parsing, query +//! discovery, source digesting, validation (moved verbatim from lib.rs +//! in the modularization). Reads the operator's WORKING TREE — stored +//! state never lives here (see store.rs). + +use super::*; + +/// How a graph declares its stored queries. Terraform-style: the `.gq` +/// files ARE the declaration — point at them (or a directory) and every +/// `query <name>` they contain is discovered. The explicit name->file map +/// remains for fine-grained control. +#[derive(Debug, Serialize, Deserialize)] +#[serde(untagged)] +pub(crate) enum QueriesDecl { + /// `queries: ./queries/` — a directory (top-level `*.gq`, sorted) or a + /// single `.gq` file; every declaration inside is registered. + Discover(PathBuf), + /// `queries: [./queries/, ./extra.gq]` — several directories/files. + DiscoverMany(Vec<PathBuf>), + /// `queries: { name: { file: ... } }` — explicit registry. + Explicit(BTreeMap<String, QueryConfig>), +} + +impl Default for QueriesDecl { + fn default() -> Self { + QueriesDecl::Explicit(BTreeMap::new()) + } +} + +/// Expand a graph's query declaration into the canonical name->file map. +/// Discovery reads and parses each `.gq`; unreadable or unparseable files +/// and duplicate query names are loud validation errors — a declaration the +/// tool cannot enumerate is broken, not partially usable. +pub(crate) fn resolve_query_decls( + config_dir: &Path, + graph_id: &str, + decl: &QueriesDecl, + diagnostics: &mut Vec<Diagnostic>, +) -> (BTreeMap<String, QueryConfig>, BTreeMap<PathBuf, String>) { + let paths: Vec<PathBuf> = match decl { + QueriesDecl::Explicit(map) => { + return ( + map.iter() + .map(|(name, config)| { + (name.clone(), QueryConfig { file: config.file.clone() }) + }) + .collect(), + BTreeMap::new(), + ); + } + QueriesDecl::Discover(path) => vec![path.clone()], + QueriesDecl::DiscoverMany(paths) => paths.clone(), + }; + + let mut files: Vec<(PathBuf, PathBuf)> = Vec::new(); // (declared-relative, resolved) + for declared in &paths { + let resolved = resolve_config_path(config_dir, declared); + if resolved.is_dir() { + let mut entries: Vec<PathBuf> = match fs::read_dir(&resolved) { + Ok(read) => read + .flatten() + .map(|entry| entry.path()) + .filter(|path| path.extension().is_some_and(|ext| ext == "gq")) + .collect(), + Err(err) => { + diagnostics.push(Diagnostic::error( + "query_dir_unreadable", + format!("graphs.{graph_id}.queries"), + format!("could not list query directory '{}': {err}", resolved.display()), + )); + continue; + } + }; + entries.sort(); + if entries.is_empty() { + diagnostics.push(Diagnostic::warning( + "query_dir_empty", + format!("graphs.{graph_id}.queries"), + format!("query directory '{}' contains no .gq files", resolved.display()), + )); + } + for path in entries { + let relative = declared.join(path.file_name().expect("dir entries have names")); + files.push((relative, path)); + } + } else { + files.push((declared.clone(), resolved)); + } + } + + let mut registry: BTreeMap<String, QueryConfig> = BTreeMap::new(); + let mut origin: BTreeMap<String, PathBuf> = BTreeMap::new(); + // Content read once at discovery and handed to the caller — the per-query + // digest/typecheck pass reuses it instead of re-reading (no N+1 reads, no + // window for the file to change between enumeration and validation). + let mut contents: BTreeMap<PathBuf, String> = BTreeMap::new(); + for (declared, resolved) in files { + let source = match fs::read_to_string(&resolved) { + Ok(source) => source, + Err(err) => { + diagnostics.push(Diagnostic::error( + "query_file_missing", + format!("graphs.{graph_id}.queries"), + format!("could not read query file '{}': {err}", resolved.display()), + )); + continue; + } + }; + let parsed = match parse_query(&source) { + Ok(parsed) => parsed, + Err(err) => { + diagnostics.push(Diagnostic::error( + "query_parse_error", + format!("graphs.{graph_id}.queries"), + format!("'{}' does not parse: {err}", resolved.display()), + )); + continue; + } + }; + for query_decl in &parsed.queries { + let name = query_decl.name.clone(); + if let Some(previous) = origin.get(&name) { + diagnostics.push(Diagnostic::error( + "duplicate_query_name", + format!("graphs.{graph_id}.queries.{name}"), + format!( + "query '{name}' is declared in both '{}' and '{}'", + previous.display(), + declared.display() + ), + )); + continue; + } + origin.insert(name.clone(), declared.clone()); + registry.insert(name, QueryConfig { file: declared.clone() }); + } + contents.insert(declared, source); + } + (registry, contents) +} + +pub(crate) fn parse_cluster_config(config_dir: &Path) -> ParsedConfig { + let config_dir = config_dir.to_path_buf(); + let config_file = config_dir.join(CLUSTER_CONFIG_FILE); + let mut diagnostics = Vec::new(); + + if !config_dir.is_dir() { + diagnostics.push(Diagnostic::error( + "config_dir_not_found", + display_path(&config_dir), + "`--config` must point at a directory containing cluster.yaml", + )); + return ParsedConfig { + raw: None, + diagnostics, + config_dir, + config_file, + }; + } + + let text = match fs::read_to_string(&config_file) { + Ok(text) => text, + Err(err) => { + diagnostics.push(Diagnostic::error( + "cluster_config_read_error", + CLUSTER_CONFIG_FILE, + format!("could not read cluster.yaml: {err}"), + )); + return ParsedConfig { + raw: None, + diagnostics, + config_dir, + config_file, + }; + } + }; + + diagnostics.extend(duplicate_key_diagnostics(&text)); + diagnostics.extend(future_field_diagnostics(&text)); + if has_errors(&diagnostics) { + return ParsedConfig { + raw: None, + diagnostics, + config_dir, + config_file, + }; + } + + let raw = match serde_yaml::from_str::<RawClusterConfig>(&text) { + Ok(raw) => Some(raw), + Err(err) => { + diagnostics.push(Diagnostic::error( + "invalid_cluster_yaml", + CLUSTER_CONFIG_FILE, + format!("could not parse cluster.yaml: {err}"), + )); + None + } + }; + + ParsedConfig { + raw, + diagnostics, + config_dir, + config_file, + } +} + +pub(crate) fn validate_cluster_header( + raw: &RawClusterConfig, + diagnostics: &mut Vec<Diagnostic>, +) -> ClusterSettings { + if raw.version != 1 { + diagnostics.push(Diagnostic::error( + "unsupported_cluster_config_version", + "version", + format!( + "unsupported cluster config version {}; this build supports version 1", + raw.version + ), + )); + } + if let Some(name) = raw.metadata.name.as_deref() { + if name.trim().is_empty() { + diagnostics.push(Diagnostic::error( + "empty_metadata_name", + "metadata.name", + "metadata.name must not be empty when provided", + )); + } + } + if let Some(backend) = raw.state.backend.as_deref() { + if backend != "cluster" { + diagnostics.push(Diagnostic::error( + "unsupported_state_backend", + "state.backend", + "Stage 2C supports only omitted state.backend or `cluster`", + )); + } + } + + ClusterSettings { + state_lock: raw.state.lock.unwrap_or(true), + } +} + + + +pub(crate) fn state_resource_digests(state: &ClusterState) -> BTreeMap<String, String> { + state + .applied_revision + .resources + .iter() + .map(|(address, resource)| (address.clone(), resource.digest.clone())) + .collect() +} + +pub(crate) fn initial_import_state(desired: &DesiredCluster) -> ClusterState { + ClusterState { + version: 1, + state_revision: 0, + applied_revision: AppliedRevisionState { + config_digest: Some(desired.config_digest.clone()), + resources: BTreeMap::new(), + }, + resource_statuses: BTreeMap::new(), + approval_records: BTreeMap::new(), + recovery_records: BTreeMap::new(), + observations: BTreeMap::new(), + } +} + + +pub(crate) async fn observe_declared_graphs(desired: &DesiredCluster, state: &mut ClusterState) -> usize { + let mut graph_error_count = 0; + for graph in &desired.graphs { + let graph_address = graph_address(&graph.id); + let schema_address = schema_address(&graph.id); + let graph_path = desired + .config_dir + .join(CLUSTER_GRAPHS_DIR) + .join(format!("{}.omni", graph.id)); + let graph_uri = display_path(&graph_path); + let observed_at = now_rfc3339(); + + if !graph_path.exists() { + state.applied_revision.resources.remove(&graph_address); + state.applied_revision.resources.remove(&schema_address); + state.observations.insert( + graph_address.clone(), + graph_observation_json(GraphObservationJson { + address: &graph_address, + graph_uri: &graph_uri, + observed_at: &observed_at, + exists: false, + manifest_version: None, + schema_digest: None, + desired_schema_digest: &graph.schema_digest, + schema_matches_desired: Some(false), + error: Some("derived graph root is missing"), + }), + ); + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Drifted, + "graph_missing", + "derived graph root is missing", + ); + set_resource_status( + state, + &schema_address, + ResourceLifecycleStatus::Drifted, + "graph_missing", + "derived graph root is missing", + ); + continue; + } + + match observe_live_graph(&graph_uri).await { + Ok(observation) => { + let schema_matches = observation.schema_digest == graph.schema_digest; + state.applied_revision.resources.insert( + schema_address.clone(), + StateResource { + digest: observation.schema_digest.clone(), + applies_to: None, + }, + ); + let query_digests = state_query_digests_for_graph(state, &graph.id); + let graph_digest_value = graph_digest( + &graph.id, + Some(&observation.schema_digest), + Some(&query_digests), + ); + state.applied_revision.resources.insert( + graph_address.clone(), + StateResource { + digest: graph_digest_value, + applies_to: None, + }, + ); + state.observations.insert( + graph_address.clone(), + graph_observation_json(GraphObservationJson { + address: &graph_address, + graph_uri: &graph_uri, + observed_at: &observed_at, + exists: true, + manifest_version: Some(observation.manifest_version), + schema_digest: Some(observation.schema_digest.as_str()), + desired_schema_digest: &graph.schema_digest, + schema_matches_desired: Some(schema_matches), + error: None, + }), + ); + if schema_matches { + set_resource_status_applied(state, &graph_address); + set_resource_status_applied(state, &schema_address); + } else { + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Drifted, + "schema_mismatch", + "live schema digest differs from desired schema digest", + ); + set_resource_status( + state, + &schema_address, + ResourceLifecycleStatus::Drifted, + "schema_mismatch", + "live schema digest differs from desired schema digest", + ); + } + } + Err(error) => { + graph_error_count += 1; + state.observations.insert( + graph_address.clone(), + graph_observation_json(GraphObservationJson { + address: &graph_address, + graph_uri: &graph_uri, + observed_at: &observed_at, + exists: true, + manifest_version: None, + schema_digest: None, + desired_schema_digest: &graph.schema_digest, + schema_matches_desired: None, + error: Some(error.as_str()), + }), + ); + set_resource_status( + state, + &graph_address, + ResourceLifecycleStatus::Error, + "graph_observation_error", + error.as_str(), + ); + set_resource_status( + state, + &schema_address, + ResourceLifecycleStatus::Error, + "graph_observation_error", + error.as_str(), + ); + } + } + } + graph_error_count +} + +/// RFC-004 §D7: the data-aware preview — the engine's migration plan for a +/// desired schema against the live graph, computed read-only (no lock). +pub(crate) async fn preview_schema_migration( + graph_uri: &str, + schema_path: &str, +) -> Result<SchemaMigrationPlan, String> { + let source = fs::read_to_string(schema_path).map_err(|err| err.to_string())?; + let db = Omnigraph::open_read_only(graph_uri) + .await + .map_err(|err| err.to_string())?; + let preview = db + .preview_schema_apply_with_options(&source, SchemaApplyOptions::default()) + .await + .map_err(|err| err.to_string())?; + Ok(preview.plan) +} + +struct LiveGraphObservation { + manifest_version: u64, + schema_digest: String, +} + +pub(crate) async fn observe_live_graph(graph_uri: &str) -> Result<LiveGraphObservation, String> { + let db = Omnigraph::open_read_only(graph_uri) + .await + .map_err(|err| err.to_string())?; + let snapshot = db + .snapshot_of(ReadTarget::branch("main")) + .await + .map_err(|err| err.to_string())?; + let schema_source = db.schema_source(); + Ok(LiveGraphObservation { + manifest_version: snapshot.version(), + schema_digest: sha256_hex(schema_source.as_bytes()), + }) +} + +struct GraphObservationJson<'a> { + address: &'a str, + graph_uri: &'a str, + observed_at: &'a str, + exists: bool, + manifest_version: Option<u64>, + schema_digest: Option<&'a str>, + desired_schema_digest: &'a str, + schema_matches_desired: Option<bool>, + error: Option<&'a str>, +} + +pub(crate) fn graph_observation_json(observation: GraphObservationJson<'_>) -> serde_json::Value { + json!({ + "kind": "graph", + "address": observation.address, + "graph_uri": observation.graph_uri, + "observed_at": observation.observed_at, + "exists": observation.exists, + "manifest_version": observation.manifest_version, + "schema_digest": observation.schema_digest, + "desired_schema_digest": observation.desired_schema_digest, + "schema_matches_desired": observation.schema_matches_desired, + "error": observation.error, + }) +} + + +pub(crate) fn load_desired(config_dir: &Path) -> LoadOutcome { + let parsed = parse_cluster_config(config_dir); + let config_dir = parsed.config_dir; + let config_file = parsed.config_file; + let mut diagnostics = parsed.diagnostics; + let Some(raw) = parsed.raw else { + return LoadOutcome { + desired: None, + diagnostics, + config_dir, + config_file, + }; + }; + let settings = validate_cluster_header(&raw, &mut diagnostics); + + let mut resources = BTreeMap::new(); + let mut dependencies = BTreeSet::new(); + let mut graph_query_digests: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new(); + let mut graph_schema_digests: BTreeMap<String, String> = BTreeMap::new(); + + for (graph_id, graph) in &raw.graphs { + validate_id( + "graph id", + &format!("graphs.{graph_id}"), + graph_id, + &mut diagnostics, + ); + let graph_address = graph_address(graph_id); + let schema_address = schema_address(graph_id); + dependencies.insert(Dependency { + from: schema_address.clone(), + to: graph_address.clone(), + }); + + let schema_path = resolve_config_path(&config_dir, &graph.schema); + let schema_source = match fs::read_to_string(&schema_path) { + Ok(source) => { + let digest = sha256_hex(source.as_bytes()); + graph_schema_digests.insert(graph_id.clone(), digest.clone()); + resources.insert( + schema_address.clone(), + ResourceSummary { + address: schema_address.clone(), + kind: "schema".to_string(), + digest, + path: Some(display_path(&schema_path)), + }, + ); + Some(source) + } + Err(err) => { + diagnostics.push(Diagnostic::error( + "schema_file_missing", + format!("graphs.{graph_id}.schema"), + format!( + "could not read schema file '{}': {err}", + schema_path.display() + ), + )); + None + } + }; + + let catalog = schema_source.and_then(|source| match parse_schema(&source) { + Ok(schema) => match build_catalog(&schema) { + Ok(catalog) => Some(catalog), + Err(err) => { + diagnostics.push(Diagnostic::error( + "schema_catalog_error", + format!("graphs.{graph_id}.schema"), + err.to_string(), + )); + None + } + }, + Err(err) => { + diagnostics.push(Diagnostic::error( + "schema_parse_error", + format!("graphs.{graph_id}.schema"), + err.to_string(), + )); + None + } + }); + + let (graph_queries, query_contents) = + resolve_query_decls(&config_dir, graph_id, &graph.queries, &mut diagnostics); + for (query_name, query) in &graph_queries { + validate_id( + "query name", + &format!("graphs.{graph_id}.queries.{query_name}"), + query_name, + &mut diagnostics, + ); + let query_address = query_address(graph_id, query_name); + dependencies.insert(Dependency { + from: query_address.clone(), + to: graph_address.clone(), + }); + dependencies.insert(Dependency { + from: query_address.clone(), + to: schema_address.clone(), + }); + + let query_path = resolve_config_path(&config_dir, &query.file); + let source = match query_contents.get(&query.file) { + Some(cached) => Ok(cached.clone()), + None => fs::read_to_string(&query_path), + }; + match source { + Ok(source) => { + let digest = sha256_hex(source.as_bytes()); + graph_query_digests + .entry(graph_id.clone()) + .or_default() + .insert(query_name.clone(), digest.clone()); + resources.insert( + query_address.clone(), + ResourceSummary { + address: query_address, + kind: "query".to_string(), + digest, + path: Some(display_path(&query_path)), + }, + ); + validate_query_source( + graph_id, + query_name, + &source, + catalog.as_ref(), + &mut diagnostics, + ); + } + Err(err) => diagnostics.push(Diagnostic::error( + "query_file_missing", + format!("graphs.{graph_id}.queries.{query_name}.file"), + format!( + "could not read query file '{}': {err}", + query_path.display() + ), + )), + } + } + } + + for graph_id in raw.graphs.keys() { + let digest = graph_digest( + graph_id, + graph_schema_digests.get(graph_id), + graph_query_digests.get(graph_id), + ); + resources.insert( + graph_address(graph_id), + ResourceSummary { + address: graph_address(graph_id), + kind: "graph".to_string(), + digest, + path: None, + }, + ); + } + + let mut policy_bindings: BTreeMap<String, Vec<String>> = BTreeMap::new(); + for (policy_name, policy) in &raw.policies { + validate_id( + "policy name", + &format!("policies.{policy_name}"), + policy_name, + &mut diagnostics, + ); + if policy.applies_to.is_empty() { + diagnostics.push(Diagnostic::error( + "policy_missing_applies_to", + format!("policies.{policy_name}.applies_to"), + "policy.applies_to must name `cluster` or at least one graph", + )); + } + + let policy_address = policy_address(policy_name); + let mut normalized_bindings: Vec<String> = Vec::new(); + for (idx, target) in policy.applies_to.iter().enumerate() { + match normalize_policy_target(target) { + PolicyTarget::Cluster => { + normalized_bindings.push("cluster".to_string()); + } + PolicyTarget::Graph(graph_id) => { + normalized_bindings.push(graph_address(&graph_id)); + if raw.graphs.contains_key(&graph_id) { + dependencies.insert(Dependency { + from: policy_address.clone(), + to: graph_address(&graph_id), + }); + } else { + diagnostics.push(Diagnostic::error( + "dangling_graph_reference", + format!("policies.{policy_name}.applies_to[{idx}]"), + format!( + "policy references graph `{graph_id}`, but no graph with that id is declared" + ), + )); + } + } + PolicyTarget::WrongKind(kind) => diagnostics.push(Diagnostic::error( + "wrong_kind_reference", + format!("policies.{policy_name}.applies_to[{idx}]"), + format!("policy applies_to expects graph refs or `cluster`, got `{kind}`"), + )), + } + } + + normalized_bindings.sort(); + normalized_bindings.dedup(); + policy_bindings.insert(policy_address.clone(), normalized_bindings); + + let policy_path = resolve_config_path(&config_dir, &policy.file); + match fs::read(&policy_path) { + Ok(bytes) => { + resources.insert( + policy_address.clone(), + ResourceSummary { + address: policy_address, + kind: "policy".to_string(), + digest: sha256_hex(&bytes), + path: Some(display_path(&policy_path)), + }, + ); + } + Err(err) => diagnostics.push(Diagnostic::error( + "policy_file_missing", + format!("policies.{policy_name}.file"), + format!( + "could not read policy file '{}': {err}", + policy_path.display() + ), + )), + } + } + + let mut resource_digests = BTreeMap::new(); + let mut resource_list = Vec::new(); + for (address, resource) in resources { + resource_digests.insert(address, resource.digest.clone()); + resource_list.push(resource); + } + let dependencies: Vec<_> = dependencies.into_iter().collect(); + let graphs = raw + .graphs + .keys() + .map(|graph_id| DesiredGraph { + id: graph_id.clone(), + schema_digest: graph_schema_digests + .get(graph_id) + .cloned() + .unwrap_or_default(), + }) + .collect(); + let config_digest = desired_config_digest(&raw, &resource_digests); + + LoadOutcome { + desired: Some(DesiredCluster { + config_dir: config_dir.clone(), + config_digest, + state_lock: settings.state_lock, + graphs, + resource_digests, + resources: resource_list, + dependencies, + policy_bindings, + }), + diagnostics, + config_dir, + config_file, + } +} + +pub(crate) fn validate_query_source( + graph_id: &str, + query_name: &str, + source: &str, + catalog: Option<&omnigraph_compiler::catalog::Catalog>, + diagnostics: &mut Vec<Diagnostic>, +) { + let path = format!("graphs.{graph_id}.queries.{query_name}"); + match parse_query(source) { + Ok(query_file) => { + let Some(query_decl) = query_file.queries.iter().find(|q| q.name == query_name) else { + diagnostics.push(Diagnostic::error( + "query_key_mismatch", + path, + format!("no `query {query_name}` declaration found in the referenced .gq file"), + )); + return; + }; + if let Some(catalog) = catalog { + if let Err(err) = typecheck_query_decl(catalog, query_decl) { + diagnostics.push(Diagnostic::error( + "query_typecheck_error", + format!("graphs.{graph_id}.queries.{query_name}"), + err.to_string(), + )); + } + } else { + diagnostics.push(Diagnostic::warning( + "query_typecheck_skipped", + format!("graphs.{graph_id}.queries.{query_name}"), + "query parsed, but type-check was skipped because the graph schema is invalid", + )); + } + } + Err(err) => diagnostics.push(Diagnostic::error( + "query_parse_error", + path, + err.to_string(), + )), + } +} + +pub(crate) fn future_field_diagnostics(text: &str) -> Vec<Diagnostic> { + let Ok(value) = serde_yaml::from_str::<serde_yaml::Value>(text) else { + return Vec::new(); + }; + let Some(mapping) = value.as_mapping() else { + return Vec::new(); + }; + let future_fields = [ + "apply", + "env_file", + "providers", + "pipelines", + "embeddings", + "ui", + "aliases", + "bindings", + ]; + mapping + .keys() + .filter_map(|key| key.as_str()) + .filter(|key| future_fields.contains(key)) + .map(|key| { + Diagnostic::error( + "future_phase_field", + key, + format!("`{key}` is reserved for a later cluster-control phase"), + ) + }) + .collect() +} + +pub(crate) fn validate_id(kind: &str, path: &str, value: &str, diagnostics: &mut Vec<Diagnostic>) { + let mut chars = value.chars(); + let valid = chars + .next() + .is_some_and(|ch| ch.is_ascii_alphabetic() || ch == '_') + && chars.all(|ch| ch.is_ascii_alphanumeric() || ch == '_' || ch == '-'); + if !valid { + diagnostics.push(Diagnostic::error( + "invalid_resource_id", + path, + format!("{kind} `{value}` must start with a letter or `_` and contain only ASCII letters, digits, `_`, or `-`"), + )); + } +} + +enum PolicyTarget { + Cluster, + Graph(String), + WrongKind(String), +} + +pub(crate) fn normalize_policy_target(value: &str) -> PolicyTarget { + if value == "cluster" { + PolicyTarget::Cluster + } else if let Some(graph_id) = value.strip_prefix("graph.") { + PolicyTarget::Graph(graph_id.to_string()) + } else if value.contains('.') { + PolicyTarget::WrongKind(value.to_string()) + } else { + PolicyTarget::Graph(value.to_string()) + } +} + +pub(crate) fn graph_address(graph_id: &str) -> String { + format!("graph.{graph_id}") +} + +pub(crate) fn schema_address(graph_id: &str) -> String { + format!("schema.{graph_id}") +} + +pub(crate) fn query_address(graph_id: &str, query_name: &str) -> String { + format!("query.{graph_id}.{query_name}") +} + +pub(crate) fn policy_address(policy_name: &str) -> String { + format!("policy.{policy_name}") +} + +pub(crate) fn resolve_config_path(config_dir: &Path, path: &Path) -> PathBuf { + if path.is_absolute() { + path.to_path_buf() + } else { + config_dir.join(path) + } +} diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index c54d245..8b41fdf 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -19,6 +19,7 @@ use ulid::Ulid; pub mod failpoints; +mod config; mod diff; mod serve; mod sweep; @@ -26,6 +27,7 @@ mod store; use store::{LocalStateBackend, StateLockGuard, StateSnapshot}; pub use serve::{ServingGraph, ServingPolicy, ServingQuery, ServingSnapshot, read_serving_snapshot}; use serve::read_verified_payload; +use config::{QueriesDecl, observe_declared_graphs, validate_cluster_header, future_field_diagnostics, initial_import_state, observe_live_graph, preview_schema_migration, state_resource_digests, graph_address, policy_address, query_address, schema_address, load_desired, normalize_policy_target, parse_cluster_config, resolve_config_path, resolve_query_decls, validate_id, validate_query_source}; use diff::{FailedGraphOrigin, ResourceKind, append_policy_binding_changes, approved_resources, classify_changes, compute_approvals, compute_blast_radius, demote_dependents_of_failed_graphs, diff_resources, resource_kind}; use sweep::{mark_approvals_consumed, record_approval_consumed, sweep_recovery_sidecars, tombstone_graph_subtree, warn_pending_recovery_sidecars}; @@ -430,138 +432,6 @@ struct GraphConfig { /// How a graph declares its stored queries. Terraform-style: the `.gq` /// files ARE the declaration — point at them (or a directory) and every -/// `query <name>` they contain is discovered. The explicit name->file map -/// remains for fine-grained control. -#[derive(Debug, Serialize, Deserialize)] -#[serde(untagged)] -enum QueriesDecl { - /// `queries: ./queries/` — a directory (top-level `*.gq`, sorted) or a - /// single `.gq` file; every declaration inside is registered. - Discover(PathBuf), - /// `queries: [./queries/, ./extra.gq]` — several directories/files. - DiscoverMany(Vec<PathBuf>), - /// `queries: { name: { file: ... } }` — explicit registry. - Explicit(BTreeMap<String, QueryConfig>), -} - -impl Default for QueriesDecl { - fn default() -> Self { - QueriesDecl::Explicit(BTreeMap::new()) - } -} - -/// Expand a graph's query declaration into the canonical name->file map. -/// Discovery reads and parses each `.gq`; unreadable or unparseable files -/// and duplicate query names are loud validation errors — a declaration the -/// tool cannot enumerate is broken, not partially usable. -fn resolve_query_decls( - config_dir: &Path, - graph_id: &str, - decl: &QueriesDecl, - diagnostics: &mut Vec<Diagnostic>, -) -> (BTreeMap<String, QueryConfig>, BTreeMap<PathBuf, String>) { - let paths: Vec<PathBuf> = match decl { - QueriesDecl::Explicit(map) => { - return ( - map.iter() - .map(|(name, config)| { - (name.clone(), QueryConfig { file: config.file.clone() }) - }) - .collect(), - BTreeMap::new(), - ); - } - QueriesDecl::Discover(path) => vec![path.clone()], - QueriesDecl::DiscoverMany(paths) => paths.clone(), - }; - - let mut files: Vec<(PathBuf, PathBuf)> = Vec::new(); // (declared-relative, resolved) - for declared in &paths { - let resolved = resolve_config_path(config_dir, declared); - if resolved.is_dir() { - let mut entries: Vec<PathBuf> = match fs::read_dir(&resolved) { - Ok(read) => read - .flatten() - .map(|entry| entry.path()) - .filter(|path| path.extension().is_some_and(|ext| ext == "gq")) - .collect(), - Err(err) => { - diagnostics.push(Diagnostic::error( - "query_dir_unreadable", - format!("graphs.{graph_id}.queries"), - format!("could not list query directory '{}': {err}", resolved.display()), - )); - continue; - } - }; - entries.sort(); - if entries.is_empty() { - diagnostics.push(Diagnostic::warning( - "query_dir_empty", - format!("graphs.{graph_id}.queries"), - format!("query directory '{}' contains no .gq files", resolved.display()), - )); - } - for path in entries { - let relative = declared.join(path.file_name().expect("dir entries have names")); - files.push((relative, path)); - } - } else { - files.push((declared.clone(), resolved)); - } - } - - let mut registry: BTreeMap<String, QueryConfig> = BTreeMap::new(); - let mut origin: BTreeMap<String, PathBuf> = BTreeMap::new(); - // Content read once at discovery and handed to the caller — the per-query - // digest/typecheck pass reuses it instead of re-reading (no N+1 reads, no - // window for the file to change between enumeration and validation). - let mut contents: BTreeMap<PathBuf, String> = BTreeMap::new(); - for (declared, resolved) in files { - let source = match fs::read_to_string(&resolved) { - Ok(source) => source, - Err(err) => { - diagnostics.push(Diagnostic::error( - "query_file_missing", - format!("graphs.{graph_id}.queries"), - format!("could not read query file '{}': {err}", resolved.display()), - )); - continue; - } - }; - let parsed = match parse_query(&source) { - Ok(parsed) => parsed, - Err(err) => { - diagnostics.push(Diagnostic::error( - "query_parse_error", - format!("graphs.{graph_id}.queries"), - format!("'{}' does not parse: {err}", resolved.display()), - )); - continue; - } - }; - for query_decl in &parsed.queries { - let name = query_decl.name.clone(); - if let Some(previous) = origin.get(&name) { - diagnostics.push(Diagnostic::error( - "duplicate_query_name", - format!("graphs.{graph_id}.queries.{name}"), - format!( - "query '{name}' is declared in both '{}' and '{}'", - previous.display(), - declared.display() - ), - )); - continue; - } - origin.insert(name.clone(), declared.clone()); - registry.insert(name, QueryConfig { file: declared.clone() }); - } - contents.insert(declared, source); - } - (registry, contents) -} - #[derive(Debug, Serialize, Deserialize)] #[serde(deny_unknown_fields)] struct QueryConfig { @@ -2138,725 +2008,6 @@ async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> St } } -fn parse_cluster_config(config_dir: &Path) -> ParsedConfig { - let config_dir = config_dir.to_path_buf(); - let config_file = config_dir.join(CLUSTER_CONFIG_FILE); - let mut diagnostics = Vec::new(); - - if !config_dir.is_dir() { - diagnostics.push(Diagnostic::error( - "config_dir_not_found", - display_path(&config_dir), - "`--config` must point at a directory containing cluster.yaml", - )); - return ParsedConfig { - raw: None, - diagnostics, - config_dir, - config_file, - }; - } - - let text = match fs::read_to_string(&config_file) { - Ok(text) => text, - Err(err) => { - diagnostics.push(Diagnostic::error( - "cluster_config_read_error", - CLUSTER_CONFIG_FILE, - format!("could not read cluster.yaml: {err}"), - )); - return ParsedConfig { - raw: None, - diagnostics, - config_dir, - config_file, - }; - } - }; - - diagnostics.extend(duplicate_key_diagnostics(&text)); - diagnostics.extend(future_field_diagnostics(&text)); - if has_errors(&diagnostics) { - return ParsedConfig { - raw: None, - diagnostics, - config_dir, - config_file, - }; - } - - let raw = match serde_yaml::from_str::<RawClusterConfig>(&text) { - Ok(raw) => Some(raw), - Err(err) => { - diagnostics.push(Diagnostic::error( - "invalid_cluster_yaml", - CLUSTER_CONFIG_FILE, - format!("could not parse cluster.yaml: {err}"), - )); - None - } - }; - - ParsedConfig { - raw, - diagnostics, - config_dir, - config_file, - } -} - -fn validate_cluster_header( - raw: &RawClusterConfig, - diagnostics: &mut Vec<Diagnostic>, -) -> ClusterSettings { - if raw.version != 1 { - diagnostics.push(Diagnostic::error( - "unsupported_cluster_config_version", - "version", - format!( - "unsupported cluster config version {}; this build supports version 1", - raw.version - ), - )); - } - if let Some(name) = raw.metadata.name.as_deref() { - if name.trim().is_empty() { - diagnostics.push(Diagnostic::error( - "empty_metadata_name", - "metadata.name", - "metadata.name must not be empty when provided", - )); - } - } - if let Some(backend) = raw.state.backend.as_deref() { - if backend != "cluster" { - diagnostics.push(Diagnostic::error( - "unsupported_state_backend", - "state.backend", - "Stage 2C supports only omitted state.backend or `cluster`", - )); - } - } - - ClusterSettings { - state_lock: raw.state.lock.unwrap_or(true), - } -} - - -fn parse_lock_file_for_unlock(text: &str) -> Result<StateLockFile, Diagnostic> { - let lock = serde_json::from_str::<StateLockFile>(text).map_err(|err| { - Diagnostic::error( - "invalid_state_lock", - CLUSTER_LOCK_FILE, - format!("could not parse state lock: {err}"), - ) - })?; - if lock.version != 1 { - return Err(Diagnostic::error( - "unsupported_state_lock_version", - CLUSTER_LOCK_FILE, - format!("unsupported cluster state lock version {}", lock.version), - )); - } - Ok(lock) -} - -fn state_lock_held_message(observations: &StateObservations) -> String { - match observations.lock_id.as_deref() { - Some(lock_id) => format!( - "cluster state lock already exists (lock id {lock_id}); run `omnigraph cluster force-unlock {lock_id}` only after confirming no cluster operation is active" - ), - None => "cluster state lock already exists; remove it only after confirming no cluster operation is active".to_string(), - } -} - -fn state_resource_digests(state: &ClusterState) -> BTreeMap<String, String> { - state - .applied_revision - .resources - .iter() - .map(|(address, resource)| (address.clone(), resource.digest.clone())) - .collect() -} - -fn initial_import_state(desired: &DesiredCluster) -> ClusterState { - ClusterState { - version: 1, - state_revision: 0, - applied_revision: AppliedRevisionState { - config_digest: Some(desired.config_digest.clone()), - resources: BTreeMap::new(), - }, - resource_statuses: BTreeMap::new(), - approval_records: BTreeMap::new(), - recovery_records: BTreeMap::new(), - observations: BTreeMap::new(), - } -} - - -async fn observe_declared_graphs(desired: &DesiredCluster, state: &mut ClusterState) -> usize { - let mut graph_error_count = 0; - for graph in &desired.graphs { - let graph_address = graph_address(&graph.id); - let schema_address = schema_address(&graph.id); - let graph_path = desired - .config_dir - .join(CLUSTER_GRAPHS_DIR) - .join(format!("{}.omni", graph.id)); - let graph_uri = display_path(&graph_path); - let observed_at = now_rfc3339(); - - if !graph_path.exists() { - state.applied_revision.resources.remove(&graph_address); - state.applied_revision.resources.remove(&schema_address); - state.observations.insert( - graph_address.clone(), - graph_observation_json(GraphObservationJson { - address: &graph_address, - graph_uri: &graph_uri, - observed_at: &observed_at, - exists: false, - manifest_version: None, - schema_digest: None, - desired_schema_digest: &graph.schema_digest, - schema_matches_desired: Some(false), - error: Some("derived graph root is missing"), - }), - ); - set_resource_status( - state, - &graph_address, - ResourceLifecycleStatus::Drifted, - "graph_missing", - "derived graph root is missing", - ); - set_resource_status( - state, - &schema_address, - ResourceLifecycleStatus::Drifted, - "graph_missing", - "derived graph root is missing", - ); - continue; - } - - match observe_live_graph(&graph_uri).await { - Ok(observation) => { - let schema_matches = observation.schema_digest == graph.schema_digest; - state.applied_revision.resources.insert( - schema_address.clone(), - StateResource { - digest: observation.schema_digest.clone(), - applies_to: None, - }, - ); - let query_digests = state_query_digests_for_graph(state, &graph.id); - let graph_digest_value = graph_digest( - &graph.id, - Some(&observation.schema_digest), - Some(&query_digests), - ); - state.applied_revision.resources.insert( - graph_address.clone(), - StateResource { - digest: graph_digest_value, - applies_to: None, - }, - ); - state.observations.insert( - graph_address.clone(), - graph_observation_json(GraphObservationJson { - address: &graph_address, - graph_uri: &graph_uri, - observed_at: &observed_at, - exists: true, - manifest_version: Some(observation.manifest_version), - schema_digest: Some(observation.schema_digest.as_str()), - desired_schema_digest: &graph.schema_digest, - schema_matches_desired: Some(schema_matches), - error: None, - }), - ); - if schema_matches { - set_resource_status_applied(state, &graph_address); - set_resource_status_applied(state, &schema_address); - } else { - set_resource_status( - state, - &graph_address, - ResourceLifecycleStatus::Drifted, - "schema_mismatch", - "live schema digest differs from desired schema digest", - ); - set_resource_status( - state, - &schema_address, - ResourceLifecycleStatus::Drifted, - "schema_mismatch", - "live schema digest differs from desired schema digest", - ); - } - } - Err(error) => { - graph_error_count += 1; - state.observations.insert( - graph_address.clone(), - graph_observation_json(GraphObservationJson { - address: &graph_address, - graph_uri: &graph_uri, - observed_at: &observed_at, - exists: true, - manifest_version: None, - schema_digest: None, - desired_schema_digest: &graph.schema_digest, - schema_matches_desired: None, - error: Some(error.as_str()), - }), - ); - set_resource_status( - state, - &graph_address, - ResourceLifecycleStatus::Error, - "graph_observation_error", - error.as_str(), - ); - set_resource_status( - state, - &schema_address, - ResourceLifecycleStatus::Error, - "graph_observation_error", - error.as_str(), - ); - } - } - } - graph_error_count -} - -/// RFC-004 §D7: the data-aware preview — the engine's migration plan for a -/// desired schema against the live graph, computed read-only (no lock). -async fn preview_schema_migration( - graph_uri: &str, - schema_path: &str, -) -> Result<SchemaMigrationPlan, String> { - let source = fs::read_to_string(schema_path).map_err(|err| err.to_string())?; - let db = Omnigraph::open_read_only(graph_uri) - .await - .map_err(|err| err.to_string())?; - let preview = db - .preview_schema_apply_with_options(&source, SchemaApplyOptions::default()) - .await - .map_err(|err| err.to_string())?; - Ok(preview.plan) -} - -struct LiveGraphObservation { - manifest_version: u64, - schema_digest: String, -} - -async fn observe_live_graph(graph_uri: &str) -> Result<LiveGraphObservation, String> { - let db = Omnigraph::open_read_only(graph_uri) - .await - .map_err(|err| err.to_string())?; - let snapshot = db - .snapshot_of(ReadTarget::branch("main")) - .await - .map_err(|err| err.to_string())?; - let schema_source = db.schema_source(); - Ok(LiveGraphObservation { - manifest_version: snapshot.version(), - schema_digest: sha256_hex(schema_source.as_bytes()), - }) -} - -struct GraphObservationJson<'a> { - address: &'a str, - graph_uri: &'a str, - observed_at: &'a str, - exists: bool, - manifest_version: Option<u64>, - schema_digest: Option<&'a str>, - desired_schema_digest: &'a str, - schema_matches_desired: Option<bool>, - error: Option<&'a str>, -} - -fn graph_observation_json(observation: GraphObservationJson<'_>) -> serde_json::Value { - json!({ - "kind": "graph", - "address": observation.address, - "graph_uri": observation.graph_uri, - "observed_at": observation.observed_at, - "exists": observation.exists, - "manifest_version": observation.manifest_version, - "schema_digest": observation.schema_digest, - "desired_schema_digest": observation.desired_schema_digest, - "schema_matches_desired": observation.schema_matches_desired, - "error": observation.error, - }) -} - -fn state_query_digests_for_graph(state: &ClusterState, graph_id: &str) -> BTreeMap<String, String> { - let prefix = format!("query.{graph_id}."); - state - .applied_revision - .resources - .iter() - .filter_map(|(address, resource)| { - address - .strip_prefix(&prefix) - .map(|name| (name.to_string(), resource.digest.clone())) - }) - .collect() -} - -fn set_resource_status_applied(state: &mut ClusterState, address: &str) { - state.resource_statuses.insert( - address.to_string(), - ResourceStatusRecord { - status: ResourceLifecycleStatus::Applied, - conditions: Vec::new(), - message: None, - }, - ); -} - -fn set_resource_status( - state: &mut ClusterState, - address: &str, - status: ResourceLifecycleStatus, - condition: &str, - message: &str, -) { - state.resource_statuses.insert( - address.to_string(), - ResourceStatusRecord { - status, - conditions: vec![condition.to_string()], - message: Some(message.to_string()), - }, - ); -} - -fn load_desired(config_dir: &Path) -> LoadOutcome { - let parsed = parse_cluster_config(config_dir); - let config_dir = parsed.config_dir; - let config_file = parsed.config_file; - let mut diagnostics = parsed.diagnostics; - let Some(raw) = parsed.raw else { - return LoadOutcome { - desired: None, - diagnostics, - config_dir, - config_file, - }; - }; - let settings = validate_cluster_header(&raw, &mut diagnostics); - - let mut resources = BTreeMap::new(); - let mut dependencies = BTreeSet::new(); - let mut graph_query_digests: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new(); - let mut graph_schema_digests: BTreeMap<String, String> = BTreeMap::new(); - - for (graph_id, graph) in &raw.graphs { - validate_id( - "graph id", - &format!("graphs.{graph_id}"), - graph_id, - &mut diagnostics, - ); - let graph_address = graph_address(graph_id); - let schema_address = schema_address(graph_id); - dependencies.insert(Dependency { - from: schema_address.clone(), - to: graph_address.clone(), - }); - - let schema_path = resolve_config_path(&config_dir, &graph.schema); - let schema_source = match fs::read_to_string(&schema_path) { - Ok(source) => { - let digest = sha256_hex(source.as_bytes()); - graph_schema_digests.insert(graph_id.clone(), digest.clone()); - resources.insert( - schema_address.clone(), - ResourceSummary { - address: schema_address.clone(), - kind: "schema".to_string(), - digest, - path: Some(display_path(&schema_path)), - }, - ); - Some(source) - } - Err(err) => { - diagnostics.push(Diagnostic::error( - "schema_file_missing", - format!("graphs.{graph_id}.schema"), - format!( - "could not read schema file '{}': {err}", - schema_path.display() - ), - )); - None - } - }; - - let catalog = schema_source.and_then(|source| match parse_schema(&source) { - Ok(schema) => match build_catalog(&schema) { - Ok(catalog) => Some(catalog), - Err(err) => { - diagnostics.push(Diagnostic::error( - "schema_catalog_error", - format!("graphs.{graph_id}.schema"), - err.to_string(), - )); - None - } - }, - Err(err) => { - diagnostics.push(Diagnostic::error( - "schema_parse_error", - format!("graphs.{graph_id}.schema"), - err.to_string(), - )); - None - } - }); - - let (graph_queries, query_contents) = - resolve_query_decls(&config_dir, graph_id, &graph.queries, &mut diagnostics); - for (query_name, query) in &graph_queries { - validate_id( - "query name", - &format!("graphs.{graph_id}.queries.{query_name}"), - query_name, - &mut diagnostics, - ); - let query_address = query_address(graph_id, query_name); - dependencies.insert(Dependency { - from: query_address.clone(), - to: graph_address.clone(), - }); - dependencies.insert(Dependency { - from: query_address.clone(), - to: schema_address.clone(), - }); - - let query_path = resolve_config_path(&config_dir, &query.file); - let source = match query_contents.get(&query.file) { - Some(cached) => Ok(cached.clone()), - None => fs::read_to_string(&query_path), - }; - match source { - Ok(source) => { - let digest = sha256_hex(source.as_bytes()); - graph_query_digests - .entry(graph_id.clone()) - .or_default() - .insert(query_name.clone(), digest.clone()); - resources.insert( - query_address.clone(), - ResourceSummary { - address: query_address, - kind: "query".to_string(), - digest, - path: Some(display_path(&query_path)), - }, - ); - validate_query_source( - graph_id, - query_name, - &source, - catalog.as_ref(), - &mut diagnostics, - ); - } - Err(err) => diagnostics.push(Diagnostic::error( - "query_file_missing", - format!("graphs.{graph_id}.queries.{query_name}.file"), - format!( - "could not read query file '{}': {err}", - query_path.display() - ), - )), - } - } - } - - for graph_id in raw.graphs.keys() { - let digest = graph_digest( - graph_id, - graph_schema_digests.get(graph_id), - graph_query_digests.get(graph_id), - ); - resources.insert( - graph_address(graph_id), - ResourceSummary { - address: graph_address(graph_id), - kind: "graph".to_string(), - digest, - path: None, - }, - ); - } - - let mut policy_bindings: BTreeMap<String, Vec<String>> = BTreeMap::new(); - for (policy_name, policy) in &raw.policies { - validate_id( - "policy name", - &format!("policies.{policy_name}"), - policy_name, - &mut diagnostics, - ); - if policy.applies_to.is_empty() { - diagnostics.push(Diagnostic::error( - "policy_missing_applies_to", - format!("policies.{policy_name}.applies_to"), - "policy.applies_to must name `cluster` or at least one graph", - )); - } - - let policy_address = policy_address(policy_name); - let mut normalized_bindings: Vec<String> = Vec::new(); - for (idx, target) in policy.applies_to.iter().enumerate() { - match normalize_policy_target(target) { - PolicyTarget::Cluster => { - normalized_bindings.push("cluster".to_string()); - } - PolicyTarget::Graph(graph_id) => { - normalized_bindings.push(graph_address(&graph_id)); - if raw.graphs.contains_key(&graph_id) { - dependencies.insert(Dependency { - from: policy_address.clone(), - to: graph_address(&graph_id), - }); - } else { - diagnostics.push(Diagnostic::error( - "dangling_graph_reference", - format!("policies.{policy_name}.applies_to[{idx}]"), - format!( - "policy references graph `{graph_id}`, but no graph with that id is declared" - ), - )); - } - } - PolicyTarget::WrongKind(kind) => diagnostics.push(Diagnostic::error( - "wrong_kind_reference", - format!("policies.{policy_name}.applies_to[{idx}]"), - format!("policy applies_to expects graph refs or `cluster`, got `{kind}`"), - )), - } - } - - normalized_bindings.sort(); - normalized_bindings.dedup(); - policy_bindings.insert(policy_address.clone(), normalized_bindings); - - let policy_path = resolve_config_path(&config_dir, &policy.file); - match fs::read(&policy_path) { - Ok(bytes) => { - resources.insert( - policy_address.clone(), - ResourceSummary { - address: policy_address, - kind: "policy".to_string(), - digest: sha256_hex(&bytes), - path: Some(display_path(&policy_path)), - }, - ); - } - Err(err) => diagnostics.push(Diagnostic::error( - "policy_file_missing", - format!("policies.{policy_name}.file"), - format!( - "could not read policy file '{}': {err}", - policy_path.display() - ), - )), - } - } - - let mut resource_digests = BTreeMap::new(); - let mut resource_list = Vec::new(); - for (address, resource) in resources { - resource_digests.insert(address, resource.digest.clone()); - resource_list.push(resource); - } - let dependencies: Vec<_> = dependencies.into_iter().collect(); - let graphs = raw - .graphs - .keys() - .map(|graph_id| DesiredGraph { - id: graph_id.clone(), - schema_digest: graph_schema_digests - .get(graph_id) - .cloned() - .unwrap_or_default(), - }) - .collect(); - let config_digest = desired_config_digest(&raw, &resource_digests); - - LoadOutcome { - desired: Some(DesiredCluster { - config_dir: config_dir.clone(), - config_digest, - state_lock: settings.state_lock, - graphs, - resource_digests, - resources: resource_list, - dependencies, - policy_bindings, - }), - diagnostics, - config_dir, - config_file, - } -} - -fn validate_query_source( - graph_id: &str, - query_name: &str, - source: &str, - catalog: Option<&omnigraph_compiler::catalog::Catalog>, - diagnostics: &mut Vec<Diagnostic>, -) { - let path = format!("graphs.{graph_id}.queries.{query_name}"); - match parse_query(source) { - Ok(query_file) => { - let Some(query_decl) = query_file.queries.iter().find(|q| q.name == query_name) else { - diagnostics.push(Diagnostic::error( - "query_key_mismatch", - path, - format!("no `query {query_name}` declaration found in the referenced .gq file"), - )); - return; - }; - if let Some(catalog) = catalog { - if let Err(err) = typecheck_query_decl(catalog, query_decl) { - diagnostics.push(Diagnostic::error( - "query_typecheck_error", - format!("graphs.{graph_id}.queries.{query_name}"), - err.to_string(), - )); - } - } else { - diagnostics.push(Diagnostic::warning( - "query_typecheck_skipped", - format!("graphs.{graph_id}.queries.{query_name}"), - "query parsed, but type-check was skipped because the graph schema is invalid", - )); - } - } - Err(err) => diagnostics.push(Diagnostic::error( - "query_parse_error", - path, - err.to_string(), - )), - } -} /// Content-addressed catalog path for an applied resource payload. Extensions @@ -3117,36 +2268,6 @@ fn duplicate_key_diagnostics(text: &str) -> Vec<Diagnostic> { diagnostics } -fn future_field_diagnostics(text: &str) -> Vec<Diagnostic> { - let Ok(value) = serde_yaml::from_str::<serde_yaml::Value>(text) else { - return Vec::new(); - }; - let Some(mapping) = value.as_mapping() else { - return Vec::new(); - }; - let future_fields = [ - "apply", - "env_file", - "providers", - "pipelines", - "embeddings", - "ui", - "aliases", - "bindings", - ]; - mapping - .keys() - .filter_map(|key| key.as_str()) - .filter(|key| future_fields.contains(key)) - .map(|key| { - Diagnostic::error( - "future_phase_field", - key, - format!("`{key}` is reserved for a later cluster-control phase"), - ) - }) - .collect() -} fn strip_comment(line: &str) -> String { let mut in_single_quote = false; @@ -3170,61 +2291,47 @@ fn strip_comment(line: &str) -> String { line.to_string() } -fn validate_id(kind: &str, path: &str, value: &str, diagnostics: &mut Vec<Diagnostic>) { - let mut chars = value.chars(); - let valid = chars - .next() - .is_some_and(|ch| ch.is_ascii_alphabetic() || ch == '_') - && chars.all(|ch| ch.is_ascii_alphanumeric() || ch == '_' || ch == '-'); - if !valid { - diagnostics.push(Diagnostic::error( - "invalid_resource_id", - path, - format!("{kind} `{value}` must start with a letter or `_` and contain only ASCII letters, digits, `_`, or `-`"), - )); - } + +fn state_query_digests_for_graph(state: &ClusterState, graph_id: &str) -> BTreeMap<String, String> { + let prefix = format!("query.{graph_id}."); + state + .applied_revision + .resources + .iter() + .filter_map(|(address, resource)| { + address + .strip_prefix(&prefix) + .map(|name| (name.to_string(), resource.digest.clone())) + }) + .collect() } -enum PolicyTarget { - Cluster, - Graph(String), - WrongKind(String), +fn set_resource_status_applied(state: &mut ClusterState, address: &str) { + state.resource_statuses.insert( + address.to_string(), + ResourceStatusRecord { + status: ResourceLifecycleStatus::Applied, + conditions: Vec::new(), + message: None, + }, + ); } -fn normalize_policy_target(value: &str) -> PolicyTarget { - if value == "cluster" { - PolicyTarget::Cluster - } else if let Some(graph_id) = value.strip_prefix("graph.") { - PolicyTarget::Graph(graph_id.to_string()) - } else if value.contains('.') { - PolicyTarget::WrongKind(value.to_string()) - } else { - PolicyTarget::Graph(value.to_string()) - } -} - -fn graph_address(graph_id: &str) -> String { - format!("graph.{graph_id}") -} - -fn schema_address(graph_id: &str) -> String { - format!("schema.{graph_id}") -} - -fn query_address(graph_id: &str, query_name: &str) -> String { - format!("query.{graph_id}.{query_name}") -} - -fn policy_address(policy_name: &str) -> String { - format!("policy.{policy_name}") -} - -fn resolve_config_path(config_dir: &Path, path: &Path) -> PathBuf { - if path.is_absolute() { - path.to_path_buf() - } else { - config_dir.join(path) - } +fn set_resource_status( + state: &mut ClusterState, + address: &str, + status: ResourceLifecycleStatus, + condition: &str, + message: &str, +) { + state.resource_statuses.insert( + address.to_string(), + ResourceStatusRecord { + status, + conditions: vec![condition.to_string()], + message: Some(message.to_string()), + }, + ); } fn graph_digest( diff --git a/crates/omnigraph-cluster/src/store.rs b/crates/omnigraph-cluster/src/store.rs index f378660..8a95661 100644 --- a/crates/omnigraph-cluster/src/store.rs +++ b/crates/omnigraph-cluster/src/store.rs @@ -559,3 +559,30 @@ impl Drop for StateLockGuard { let _ = fs::remove_file(&self.path); } } + +pub(crate) fn parse_lock_file_for_unlock(text: &str) -> Result<StateLockFile, Diagnostic> { + let lock = serde_json::from_str::<StateLockFile>(text).map_err(|err| { + Diagnostic::error( + "invalid_state_lock", + CLUSTER_LOCK_FILE, + format!("could not parse state lock: {err}"), + ) + })?; + if lock.version != 1 { + return Err(Diagnostic::error( + "unsupported_state_lock_version", + CLUSTER_LOCK_FILE, + format!("unsupported cluster state lock version {}", lock.version), + )); + } + Ok(lock) +} + +pub(crate) fn state_lock_held_message(observations: &StateObservations) -> String { + match observations.lock_id.as_deref() { + Some(lock_id) => format!( + "cluster state lock already exists (lock id {lock_id}); run `omnigraph cluster force-unlock {lock_id}` only after confirming no cluster operation is active" + ), + None => "cluster state lock already exists; remove it only after confirming no cluster operation is active".to_string(), + } +} From db6fe03be1527b0c8f4dbfa74e5ff9033dec2466 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 05:42:02 +0300 Subject: [PATCH 107/165] refactor(cluster): move type definitions to types.rs Verbatim move of the public output/diagnostic types and the internal state/sidecar/approval models; previously-private types and their fields get pub(crate) (they were crate-visible by position before). lib.rs is now the command pipeline + public API. 95 tests green; full workspace gate green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 510 +------------------------- crates/omnigraph-cluster/src/types.rs | 510 ++++++++++++++++++++++++++ 2 files changed, 513 insertions(+), 507 deletions(-) create mode 100644 crates/omnigraph-cluster/src/types.rs diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 8b41fdf..dc66408 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -20,11 +20,14 @@ use ulid::Ulid; pub mod failpoints; mod config; +mod types; mod diff; mod serve; mod sweep; mod store; use store::{LocalStateBackend, StateLockGuard, StateSnapshot}; +pub use types::*; +use types::*; pub use serve::{ServingGraph, ServingPolicy, ServingQuery, ServingSnapshot, read_serving_snapshot}; use serve::read_verified_payload; use config::{QueriesDecl, observe_declared_graphs, validate_cluster_header, future_field_diagnostics, initial_import_state, observe_live_graph, preview_schema_migration, state_resource_digests, graph_address, policy_address, query_address, schema_address, load_desired, normalize_policy_target, parse_cluster_config, resolve_config_path, resolve_query_decls, validate_id, validate_query_source}; @@ -40,513 +43,6 @@ pub const CLUSTER_RESOURCES_DIR: &str = "__cluster/resources"; pub const CLUSTER_RECOVERIES_DIR: &str = "__cluster/recoveries"; pub const CLUSTER_APPROVALS_DIR: &str = "__cluster/approvals"; -#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] -#[serde(rename_all = "snake_case")] -pub enum DiagnosticSeverity { - Error, - Warning, -} - -#[derive(Debug, Clone, Serialize, PartialEq, Eq)] -pub struct Diagnostic { - pub code: String, - pub severity: DiagnosticSeverity, - pub path: String, - pub message: String, -} - -impl Diagnostic { - fn error(code: impl Into<String>, path: impl Into<String>, message: impl Into<String>) -> Self { - Self { - code: code.into(), - severity: DiagnosticSeverity::Error, - path: path.into(), - message: message.into(), - } - } - - fn warning( - code: impl Into<String>, - path: impl Into<String>, - message: impl Into<String>, - ) -> Self { - Self { - code: code.into(), - severity: DiagnosticSeverity::Warning, - path: path.into(), - message: message.into(), - } - } -} - -#[derive(Debug, Clone, Serialize, PartialEq, Eq)] -pub struct ResourceSummary { - pub address: String, - pub kind: String, - pub digest: String, - #[serde(skip_serializing_if = "Option::is_none")] - pub path: Option<String>, -} - -#[derive(Debug, Clone, Serialize, PartialEq, Eq, PartialOrd, Ord)] -pub struct Dependency { - pub from: String, - pub to: String, -} - -#[derive(Debug, Clone, Serialize)] -pub struct ValidateOutput { - pub ok: bool, - pub config_dir: String, - pub config_file: String, - pub resource_digests: BTreeMap<String, String>, - pub resources: Vec<ResourceSummary>, - pub dependencies: Vec<Dependency>, - pub diagnostics: Vec<Diagnostic>, -} - -#[derive(Debug, Clone, Serialize)] -pub struct DesiredRevision { - #[serde(skip_serializing_if = "Option::is_none")] - pub config_digest: Option<String>, -} - -#[derive(Debug, Clone, Serialize)] -pub struct StateObservations { - pub state_path: String, - pub lock_path: String, - pub state_found: bool, - #[serde(skip_serializing_if = "Option::is_none")] - pub applied_config_digest: Option<String>, - pub state_revision: u64, - #[serde(skip_serializing_if = "Option::is_none")] - pub state_cas: Option<String>, - pub resource_count: usize, - pub locked: bool, - #[serde(skip_serializing_if = "Option::is_none")] - pub lock_id: Option<String>, - pub lock_acquired: bool, - #[serde(skip_serializing_if = "Option::is_none")] - pub acquired_lock_id: Option<String>, - #[serde(skip_serializing_if = "Option::is_none")] - pub lock_operation: Option<String>, - #[serde(skip_serializing_if = "Option::is_none")] - pub lock_created_at: Option<String>, - #[serde(skip_serializing_if = "Option::is_none")] - pub lock_pid: Option<u32>, - #[serde(skip_serializing_if = "Option::is_none")] - pub lock_age_seconds: Option<u64>, -} - -impl StateObservations { - fn observe_lock_metadata(&mut self, lock: &StateLockFile) { - self.locked = true; - self.lock_id = Some(lock.lock_id.clone()); - self.lock_operation = Some(lock.operation.clone()); - self.lock_created_at = Some(lock.created_at.clone()); - self.lock_pid = Some(lock.pid); - self.lock_age_seconds = lock_age_seconds(&lock.created_at); - } -} - -#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] -#[serde(rename_all = "snake_case")] -pub enum ResourceLifecycleStatus { - Pending, - Planned, - Applying, - Applied, - Drifted, - Blocked, - Error, -} - -#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] -#[serde(deny_unknown_fields)] -pub struct ResourceStatusRecord { - pub status: ResourceLifecycleStatus, - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub conditions: Vec<String>, - #[serde(default, skip_serializing_if = "Option::is_none")] - pub message: Option<String>, -} - -#[derive(Debug, Clone, Serialize, PartialEq, Eq)] -#[serde(rename_all = "snake_case")] -pub enum PlanOperation { - Create, - Update, - Delete, -} - -/// How `cluster apply` treats a planned change in the current stage. -/// -/// `Applied` changes execute (config-only query/policy catalog writes). -/// `Derived` marks a `graph.<id>` composite-digest update that converges -/// automatically once its applied query digests land in state. `Deferred` -/// changes need a later phase (graph/schema lifecycle or schema content). -/// `Blocked` query/policy changes are gated by an unapplied or missing -/// dependency. -#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)] -#[serde(rename_all = "snake_case")] -pub enum ApplyDisposition { - Applied, - Derived, - Deferred, - Blocked, -} - -#[derive(Debug, Clone, Serialize, PartialEq)] -pub struct PlanChange { - pub resource: String, - pub operation: PlanOperation, - #[serde(skip_serializing_if = "Option::is_none")] - pub before_digest: Option<String>, - #[serde(skip_serializing_if = "Option::is_none")] - pub after_digest: Option<String>, - #[serde(skip_serializing_if = "Option::is_none")] - pub disposition: Option<ApplyDisposition>, - #[serde(skip_serializing_if = "Option::is_none")] - pub reason: Option<String>, - /// True for a policy change whose file digest is unchanged but whose - /// `applies_to` bindings differ from the applied revision (including the - /// pre-5A backfill case). - #[serde(default, skip_serializing_if = "std::ops::Not::not")] - pub binding_change: bool, - /// For schema updates: the engine's migration plan against the live - /// graph (RFC-004 §D7's data-aware preview). Absent when the preview is - /// unavailable (warning `schema_preview_unavailable`). - #[serde(skip_serializing_if = "Option::is_none")] - pub migration: Option<SchemaMigrationPlan>, -} - -#[derive(Debug, Clone, Serialize, PartialEq, Eq)] -pub struct BlastRadius { - pub resource: String, - pub affected: Vec<String>, -} - -#[derive(Debug, Clone, Serialize, PartialEq, Eq)] -pub struct ApprovalRequirement { - pub resource: String, - pub reason: String, - /// True when a valid (digest-matching, unconsumed) approval artifact is - /// pending for this change. - pub satisfied: bool, -} - -#[derive(Debug, Clone, Serialize)] -pub struct PlanOutput { - pub ok: bool, - pub config_dir: String, - pub desired_revision: DesiredRevision, - pub resource_digests: BTreeMap<String, String>, - pub dependencies: Vec<Dependency>, - pub state_observations: StateObservations, - pub changes: Vec<PlanChange>, - pub blast_radius: Vec<BlastRadius>, - pub approvals_required: Vec<ApprovalRequirement>, - pub diagnostics: Vec<Diagnostic>, -} - -#[derive(Debug, Clone, Serialize)] -pub struct StatusOutput { - pub ok: bool, - pub config_dir: String, - pub state_observations: StateObservations, - pub resource_digests: BTreeMap<String, String>, - pub resource_statuses: BTreeMap<String, ResourceStatusRecord>, - pub observations: BTreeMap<String, serde_json::Value>, - pub diagnostics: Vec<Diagnostic>, -} - -#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)] -#[serde(rename_all = "snake_case")] -pub enum StateSyncOperation { - Refresh, - Import, -} - -#[derive(Debug, Clone, Serialize)] -pub struct StateSyncOutput { - pub ok: bool, - pub operation: StateSyncOperation, - pub config_dir: String, - pub state_observations: StateObservations, - pub resource_digests: BTreeMap<String, String>, - pub resource_statuses: BTreeMap<String, ResourceStatusRecord>, - pub observations: BTreeMap<String, serde_json::Value>, - pub diagnostics: Vec<Diagnostic>, -} - -#[derive(Debug, Clone, Serialize)] -pub struct ForceUnlockOutput { - pub ok: bool, - pub config_dir: String, - pub state_observations: StateObservations, - pub lock_removed: bool, - pub diagnostics: Vec<Diagnostic>, -} - -/// Output of config-only `cluster apply`. "Applied" means recorded in the -/// local cluster catalog (`__cluster/`); nothing applied here serves traffic — -/// the server still boots from `omnigraph.yaml` until the server-boot stage. -#[derive(Debug, Clone, Serialize)] -pub struct ApplyOutput { - pub ok: bool, - pub config_dir: String, - #[serde(skip_serializing_if = "Option::is_none")] - pub actor: Option<String>, - pub desired_revision: DesiredRevision, - pub state_observations: StateObservations, - /// Every planned change, with `disposition`/`reason` always populated. - pub changes: Vec<PlanChange>, - pub applied_count: usize, - /// Deferred + Blocked changes (Derived composite updates count as neither). - pub deferred_count: usize, - /// True when state matches the desired revision after this apply. - pub converged: bool, - /// False for a no-op re-apply: state bytes (and revision) were left untouched. - pub state_written: bool, - /// The statuses as persisted: post-apply on success, the pre-apply on-disk - /// snapshot when the state write fails (never unpersisted in-memory state). - pub resource_statuses: BTreeMap<String, ResourceStatusRecord>, - pub diagnostics: Vec<Diagnostic>, -} - -/// A digest-bound human approval for an irreversible operation (RFC-004 -/// §D4). Written by `cluster approve`, consumed by apply. The file is never -/// deleted on consumption — it is rewritten with `consumed_at` and also -/// summarized into the state ledger's `approval_records`, so the audit fact -/// survives the loss of either store (axiom 11). -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct ApprovalArtifact { - schema_version: u32, - approval_id: String, - resource: String, - operation: String, - reason: String, - bound_config_digest: String, - #[serde(default)] - bound_before_digest: Option<String>, - #[serde(default)] - bound_after_digest: Option<String>, - approved_by: String, - created_at: String, - #[serde(default)] - consumed_at: Option<String>, - #[serde(default)] - consumed_by_operation: Option<String>, -} - -#[derive(Debug, Clone, Serialize)] -pub struct ApproveOutput { - pub ok: bool, - pub config_dir: String, - #[serde(skip_serializing_if = "Option::is_none")] - pub approval_id: Option<String>, - #[serde(skip_serializing_if = "Option::is_none")] - pub resource: Option<String>, - #[serde(skip_serializing_if = "Option::is_none")] - pub operation: Option<PlanOperation>, - #[serde(skip_serializing_if = "Option::is_none")] - pub approved_by: Option<String>, - pub diagnostics: Vec<Diagnostic>, -} - -#[derive(Debug, Clone)] -struct DesiredCluster { - config_dir: PathBuf, - config_digest: String, - state_lock: bool, - graphs: Vec<DesiredGraph>, - resource_digests: BTreeMap<String, String>, - resources: Vec<ResourceSummary>, - dependencies: Vec<Dependency>, - /// `policy.<name>` address -> normalized applies_to refs. - policy_bindings: BTreeMap<String, Vec<String>>, -} - -#[derive(Debug, Clone)] -struct DesiredGraph { - id: String, - schema_digest: String, -} - -#[derive(Debug)] -struct ParsedConfig { - raw: Option<RawClusterConfig>, - diagnostics: Vec<Diagnostic>, - config_dir: PathBuf, - config_file: PathBuf, -} - -#[derive(Debug, Clone, Copy)] -struct ClusterSettings { - state_lock: bool, -} - -#[derive(Debug)] -struct LoadOutcome { - desired: Option<DesiredCluster>, - diagnostics: Vec<Diagnostic>, - config_dir: PathBuf, - config_file: PathBuf, -} - -#[derive(Debug, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct RawClusterConfig { - version: u32, - #[serde(default)] - metadata: Metadata, - #[serde(default)] - state: StateConfig, - #[serde(default)] - graphs: BTreeMap<String, GraphConfig>, - #[serde(default)] - policies: BTreeMap<String, PolicyConfig>, -} - -#[derive(Debug, Default, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct Metadata { - name: Option<String>, -} - -#[derive(Debug, Default, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct StateConfig { - backend: Option<String>, - lock: Option<bool>, -} - -#[derive(Debug, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct GraphConfig { - schema: PathBuf, - #[serde(default)] - queries: QueriesDecl, -} - -/// How a graph declares its stored queries. Terraform-style: the `.gq` -/// files ARE the declaration — point at them (or a directory) and every -#[derive(Debug, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct QueryConfig { - file: PathBuf, -} - -#[derive(Debug, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct PolicyConfig { - file: PathBuf, - applies_to: Vec<String>, -} - -// Stage 2A/2B accept these forward-compatible state sections so existing -// ledgers won't churn while approval/recovery semantics are staged later. -#[allow(dead_code)] -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct ClusterState { - version: u32, - #[serde(default)] - state_revision: u64, - applied_revision: AppliedRevisionState, - #[serde(default)] - resource_statuses: BTreeMap<String, ResourceStatusRecord>, - #[serde(default)] - approval_records: BTreeMap<String, serde_json::Value>, - #[serde(default)] - recovery_records: BTreeMap<String, serde_json::Value>, - #[serde(default)] - observations: BTreeMap<String, serde_json::Value>, -} - -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct AppliedRevisionState { - #[serde(default)] - config_digest: Option<String>, - #[serde(default)] - resources: BTreeMap<String, StateResource>, -} - -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct StateResource { - digest: String, - /// Policy resources only: the applied `applies_to` bindings, normalized - /// to typed refs (`cluster` | `graph.<id>`). Recorded so the state - /// ledger is serving-sufficient for the Phase-5 server boot (RFC-005 - /// §D3). Absent on pre-5A entries (backfilled by the next apply) and on - /// non-policy resources. - #[serde(default, skip_serializing_if = "Option::is_none")] - applies_to: Option<Vec<String>>, -} - -#[derive(Debug, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct StateLockFile { - version: u32, - lock_id: String, - operation: String, - created_at: String, - pid: u32, -} - -/// Recovery-intent record for a graph-moving apply operation (RFC-004 §D2). -/// Written under the state lock before the engine call that can create or -/// move a graph manifest; deleted only after the cluster state CAS that -/// records the outcome lands. The sweep (§D3) classifies survivors. -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct RecoverySidecar { - schema_version: u32, - operation_id: String, - started_at: String, - #[serde(default)] - actor: Option<String>, - kind: RecoverySidecarKind, - graph_id: String, - graph_uri: String, - #[serde(default)] - observed_manifest_version: Option<u64>, - #[serde(default)] - expected_manifest_version: Option<u64>, - desired_schema_digest: String, - #[serde(default)] - state_cas_base: Option<String>, - /// For graph_delete: the approval this operation consumes; lets a sweep - /// roll-forward consume it too. - #[serde(default)] - approval_id: Option<String>, -} - -#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] -#[serde(rename_all = "snake_case")] -enum RecoverySidecarKind { - GraphCreate, - SchemaApply, - GraphDelete, -} - -#[derive(Debug, Default)] -struct SweepOutcome { - /// Graphs whose sidecar was kept (rows 5/6): graph-moving work for them - /// is blocked until the operator repairs and re-observes. - pending_graphs: BTreeSet<String>, - /// Sidecars whose outcome is recorded (rows 2/4): deleted only after the - /// command's state write lands, so a CAS failure re-sweeps them. - completed_sidecars: Vec<PathBuf>, - /// Approval artifacts consumed by a roll-forward (delete row 7b): their - /// files are rewritten with consumed_at only after the state write lands. - consumed_approvals: Vec<String>, -} - - pub fn validate_config_dir(config_dir: impl AsRef<Path>) -> ValidateOutput { let outcome = load_desired(config_dir.as_ref()); let (resource_digests, resources, dependencies) = match outcome.desired { diff --git a/crates/omnigraph-cluster/src/types.rs b/crates/omnigraph-cluster/src/types.rs new file mode 100644 index 0000000..c366f04 --- /dev/null +++ b/crates/omnigraph-cluster/src/types.rs @@ -0,0 +1,510 @@ +//! Public output/diagnostic types and internal state/sidecar/approval +//! models (moved verbatim from lib.rs in the modularization). + +use super::*; + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum DiagnosticSeverity { + Error, + Warning, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct Diagnostic { + pub code: String, + pub severity: DiagnosticSeverity, + pub path: String, + pub message: String, +} + +impl Diagnostic { + pub(crate) fn error(code: impl Into<String>, path: impl Into<String>, message: impl Into<String>) -> Self { + Self { + code: code.into(), + severity: DiagnosticSeverity::Error, + path: path.into(), + message: message.into(), + } + } + + pub(crate) fn warning( + code: impl Into<String>, + path: impl Into<String>, + message: impl Into<String>, + ) -> Self { + Self { + code: code.into(), + severity: DiagnosticSeverity::Warning, + path: path.into(), + message: message.into(), + } + } +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct ResourceSummary { + pub address: String, + pub kind: String, + pub digest: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub path: Option<String>, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq, PartialOrd, Ord)] +pub struct Dependency { + pub from: String, + pub to: String, +} + +#[derive(Debug, Clone, Serialize)] +pub struct ValidateOutput { + pub ok: bool, + pub config_dir: String, + pub config_file: String, + pub resource_digests: BTreeMap<String, String>, + pub resources: Vec<ResourceSummary>, + pub dependencies: Vec<Dependency>, + pub diagnostics: Vec<Diagnostic>, +} + +#[derive(Debug, Clone, Serialize)] +pub struct DesiredRevision { + #[serde(skip_serializing_if = "Option::is_none")] + pub config_digest: Option<String>, +} + +#[derive(Debug, Clone, Serialize)] +pub struct StateObservations { + pub state_path: String, + pub lock_path: String, + pub state_found: bool, + #[serde(skip_serializing_if = "Option::is_none")] + pub applied_config_digest: Option<String>, + pub state_revision: u64, + #[serde(skip_serializing_if = "Option::is_none")] + pub state_cas: Option<String>, + pub resource_count: usize, + pub locked: bool, + #[serde(skip_serializing_if = "Option::is_none")] + pub lock_id: Option<String>, + pub lock_acquired: bool, + #[serde(skip_serializing_if = "Option::is_none")] + pub acquired_lock_id: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub lock_operation: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub lock_created_at: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub lock_pid: Option<u32>, + #[serde(skip_serializing_if = "Option::is_none")] + pub lock_age_seconds: Option<u64>, +} + +impl StateObservations { + pub(crate) fn observe_lock_metadata(&mut self, lock: &StateLockFile) { + self.locked = true; + self.lock_id = Some(lock.lock_id.clone()); + self.lock_operation = Some(lock.operation.clone()); + self.lock_created_at = Some(lock.created_at.clone()); + self.lock_pid = Some(lock.pid); + self.lock_age_seconds = lock_age_seconds(&lock.created_at); + } +} + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum ResourceLifecycleStatus { + Pending, + Planned, + Applying, + Applied, + Drifted, + Blocked, + Error, +} + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +pub struct ResourceStatusRecord { + pub status: ResourceLifecycleStatus, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub conditions: Vec<String>, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub message: Option<String>, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum PlanOperation { + Create, + Update, + Delete, +} + +/// How `cluster apply` treats a planned change in the current stage. +/// +/// `Applied` changes execute (config-only query/policy catalog writes). +/// `Derived` marks a `graph.<id>` composite-digest update that converges +/// automatically once its applied query digests land in state. `Deferred` +/// changes need a later phase (graph/schema lifecycle or schema content). +/// `Blocked` query/policy changes are gated by an unapplied or missing +/// dependency. +#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum ApplyDisposition { + Applied, + Derived, + Deferred, + Blocked, +} + +#[derive(Debug, Clone, Serialize, PartialEq)] +pub struct PlanChange { + pub resource: String, + pub operation: PlanOperation, + #[serde(skip_serializing_if = "Option::is_none")] + pub before_digest: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub after_digest: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub disposition: Option<ApplyDisposition>, + #[serde(skip_serializing_if = "Option::is_none")] + pub reason: Option<String>, + /// True for a policy change whose file digest is unchanged but whose + /// `applies_to` bindings differ from the applied revision (including the + /// pre-5A backfill case). + #[serde(default, skip_serializing_if = "std::ops::Not::not")] + pub binding_change: bool, + /// For schema updates: the engine's migration plan against the live + /// graph (RFC-004 §D7's data-aware preview). Absent when the preview is + /// unavailable (warning `schema_preview_unavailable`). + #[serde(skip_serializing_if = "Option::is_none")] + pub migration: Option<SchemaMigrationPlan>, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct BlastRadius { + pub resource: String, + pub affected: Vec<String>, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct ApprovalRequirement { + pub resource: String, + pub reason: String, + /// True when a valid (digest-matching, unconsumed) approval artifact is + /// pending for this change. + pub satisfied: bool, +} + +#[derive(Debug, Clone, Serialize)] +pub struct PlanOutput { + pub ok: bool, + pub config_dir: String, + pub desired_revision: DesiredRevision, + pub resource_digests: BTreeMap<String, String>, + pub dependencies: Vec<Dependency>, + pub state_observations: StateObservations, + pub changes: Vec<PlanChange>, + pub blast_radius: Vec<BlastRadius>, + pub approvals_required: Vec<ApprovalRequirement>, + pub diagnostics: Vec<Diagnostic>, +} + +#[derive(Debug, Clone, Serialize)] +pub struct StatusOutput { + pub ok: bool, + pub config_dir: String, + pub state_observations: StateObservations, + pub resource_digests: BTreeMap<String, String>, + pub resource_statuses: BTreeMap<String, ResourceStatusRecord>, + pub observations: BTreeMap<String, serde_json::Value>, + pub diagnostics: Vec<Diagnostic>, +} + +#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum StateSyncOperation { + Refresh, + Import, +} + +#[derive(Debug, Clone, Serialize)] +pub struct StateSyncOutput { + pub ok: bool, + pub operation: StateSyncOperation, + pub config_dir: String, + pub state_observations: StateObservations, + pub resource_digests: BTreeMap<String, String>, + pub resource_statuses: BTreeMap<String, ResourceStatusRecord>, + pub observations: BTreeMap<String, serde_json::Value>, + pub diagnostics: Vec<Diagnostic>, +} + +#[derive(Debug, Clone, Serialize)] +pub struct ForceUnlockOutput { + pub ok: bool, + pub config_dir: String, + pub state_observations: StateObservations, + pub lock_removed: bool, + pub diagnostics: Vec<Diagnostic>, +} + +/// Output of config-only `cluster apply`. "Applied" means recorded in the +/// local cluster catalog (`__cluster/`); nothing applied here serves traffic — +/// the server still boots from `omnigraph.yaml` until the server-boot stage. +#[derive(Debug, Clone, Serialize)] +pub struct ApplyOutput { + pub ok: bool, + pub config_dir: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub actor: Option<String>, + pub desired_revision: DesiredRevision, + pub state_observations: StateObservations, + /// Every planned change, with `disposition`/`reason` always populated. + pub changes: Vec<PlanChange>, + pub applied_count: usize, + /// Deferred + Blocked changes (Derived composite updates count as neither). + pub deferred_count: usize, + /// True when state matches the desired revision after this apply. + pub converged: bool, + /// False for a no-op re-apply: state bytes (and revision) were left untouched. + pub state_written: bool, + /// The statuses as persisted: post-apply on success, the pre-apply on-disk + /// snapshot when the state write fails (never unpersisted in-memory state). + pub resource_statuses: BTreeMap<String, ResourceStatusRecord>, + pub diagnostics: Vec<Diagnostic>, +} + +/// A digest-bound human approval for an irreversible operation (RFC-004 +/// §D4). Written by `cluster approve`, consumed by apply. The file is never +/// deleted on consumption — it is rewritten with `consumed_at` and also +/// summarized into the state ledger's `approval_records`, so the audit fact +/// survives the loss of either store (axiom 11). +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct ApprovalArtifact { + pub(crate) schema_version: u32, + pub(crate) approval_id: String, + pub(crate) resource: String, + pub(crate) operation: String, + pub(crate) reason: String, + pub(crate) bound_config_digest: String, + #[serde(default)] + pub(crate) bound_before_digest: Option<String>, + #[serde(default)] + pub(crate) bound_after_digest: Option<String>, + pub(crate) approved_by: String, + pub(crate) created_at: String, + #[serde(default)] + pub(crate) consumed_at: Option<String>, + #[serde(default)] + pub(crate) consumed_by_operation: Option<String>, +} + +#[derive(Debug, Clone, Serialize)] +pub struct ApproveOutput { + pub ok: bool, + pub config_dir: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub approval_id: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub resource: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub operation: Option<PlanOperation>, + #[serde(skip_serializing_if = "Option::is_none")] + pub approved_by: Option<String>, + pub diagnostics: Vec<Diagnostic>, +} + +#[derive(Debug, Clone)] +pub(crate) struct DesiredCluster { + pub(crate) config_dir: PathBuf, + pub(crate) config_digest: String, + pub(crate) state_lock: bool, + pub(crate) graphs: Vec<DesiredGraph>, + pub(crate) resource_digests: BTreeMap<String, String>, + pub(crate) resources: Vec<ResourceSummary>, + pub(crate) dependencies: Vec<Dependency>, + /// `policy.<name>` address -> normalized applies_to refs. + pub(crate) policy_bindings: BTreeMap<String, Vec<String>>, +} + +#[derive(Debug, Clone)] +pub(crate) struct DesiredGraph { + pub(crate) id: String, + pub(crate) schema_digest: String, +} + +#[derive(Debug)] +pub(crate) struct ParsedConfig { + pub(crate) raw: Option<RawClusterConfig>, + pub(crate) diagnostics: Vec<Diagnostic>, + pub(crate) config_dir: PathBuf, + pub(crate) config_file: PathBuf, +} + +#[derive(Debug, Clone, Copy)] +pub(crate) struct ClusterSettings { + pub(crate) state_lock: bool, +} + +#[derive(Debug)] +pub(crate) struct LoadOutcome { + pub(crate) desired: Option<DesiredCluster>, + pub(crate) diagnostics: Vec<Diagnostic>, + pub(crate) config_dir: PathBuf, + pub(crate) config_file: PathBuf, +} + +#[derive(Debug, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct RawClusterConfig { + pub(crate) version: u32, + #[serde(default)] + pub(crate) metadata: Metadata, + #[serde(default)] + pub(crate) state: StateConfig, + #[serde(default)] + pub(crate) graphs: BTreeMap<String, GraphConfig>, + #[serde(default)] + pub(crate) policies: BTreeMap<String, PolicyConfig>, +} + +#[derive(Debug, Default, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct Metadata { + pub(crate) name: Option<String>, +} + +#[derive(Debug, Default, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct StateConfig { + pub(crate) backend: Option<String>, + pub(crate) lock: Option<bool>, +} + +#[derive(Debug, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct GraphConfig { + pub(crate) schema: PathBuf, + #[serde(default)] + pub(crate) queries: QueriesDecl, +} + +/// How a graph declares its stored queries. Terraform-style: the `.gq` +/// files ARE the declaration — point at them (or a directory) and every +#[derive(Debug, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct QueryConfig { + pub(crate) file: PathBuf, +} + +#[derive(Debug, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct PolicyConfig { + pub(crate) file: PathBuf, + pub(crate) applies_to: Vec<String>, +} + +// Stage 2A/2B accept these forward-compatible state sections so existing +// ledgers won't churn while approval/recovery semantics are staged later. +#[allow(dead_code)] +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct ClusterState { + pub(crate) version: u32, + #[serde(default)] + pub(crate) state_revision: u64, + pub(crate) applied_revision: AppliedRevisionState, + #[serde(default)] + pub(crate) resource_statuses: BTreeMap<String, ResourceStatusRecord>, + #[serde(default)] + pub(crate) approval_records: BTreeMap<String, serde_json::Value>, + #[serde(default)] + pub(crate) recovery_records: BTreeMap<String, serde_json::Value>, + #[serde(default)] + pub(crate) observations: BTreeMap<String, serde_json::Value>, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct AppliedRevisionState { + #[serde(default)] + pub(crate) config_digest: Option<String>, + #[serde(default)] + pub(crate) resources: BTreeMap<String, StateResource>, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct StateResource { + pub(crate) digest: String, + /// Policy resources only: the applied `applies_to` bindings, normalized + /// to typed refs (`cluster` | `graph.<id>`). Recorded so the state + /// ledger is serving-sufficient for the Phase-5 server boot (RFC-005 + /// §D3). Absent on pre-5A entries (backfilled by the next apply) and on + /// non-policy resources. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub(crate) applies_to: Option<Vec<String>>, +} + +#[derive(Debug, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct StateLockFile { + pub(crate) version: u32, + pub(crate) lock_id: String, + pub(crate) operation: String, + pub(crate) created_at: String, + pub(crate) pid: u32, +} + +/// Recovery-intent record for a graph-moving apply operation (RFC-004 §D2). +/// Written under the state lock before the engine call that can create or +/// move a graph manifest; deleted only after the cluster state CAS that +/// records the outcome lands. The sweep (§D3) classifies survivors. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct RecoverySidecar { + pub(crate) schema_version: u32, + pub(crate) operation_id: String, + pub(crate) started_at: String, + #[serde(default)] + pub(crate) actor: Option<String>, + pub(crate) kind: RecoverySidecarKind, + pub(crate) graph_id: String, + pub(crate) graph_uri: String, + #[serde(default)] + pub(crate) observed_manifest_version: Option<u64>, + #[serde(default)] + pub(crate) expected_manifest_version: Option<u64>, + pub(crate) desired_schema_digest: String, + #[serde(default)] + pub(crate) state_cas_base: Option<String>, + /// For graph_delete: the approval this operation consumes; lets a sweep + /// roll-forward consume it too. + #[serde(default)] + pub(crate) approval_id: Option<String>, +} + +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub(crate) enum RecoverySidecarKind { + GraphCreate, + SchemaApply, + GraphDelete, +} + +#[derive(Debug, Default)] +pub(crate) struct SweepOutcome { + /// Graphs whose sidecar was kept (rows 5/6): graph-moving work for them + /// is blocked until the operator repairs and re-observes. + pub(crate) pending_graphs: BTreeSet<String>, + /// Sidecars whose outcome is recorded (rows 2/4): deleted only after the + /// command's state write lands, so a CAS failure re-sweeps them. + pub(crate) completed_sidecars: Vec<PathBuf>, + /// Approval artifacts consumed by a roll-forward (delete row 7b): their + /// files are rewritten with consumed_at only after the state write lands. + pub(crate) consumed_approvals: Vec<String>, +} From 7f32e6f1bc092a3e2a3d87ec354551f3f61c9ddc Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 10:38:54 +0300 Subject: [PATCH 108/165] ci: raise Test Workspace timeout to 75 minutes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A cold rust-cache (every Cargo.lock change) means a full workspace + failpoints-feature build on the 2-core runner, which now exceeds 45 minutes on slow runner days — and because a timed-out run never saves its cache, an undersized budget self-perpetuates: every retry starts cold and dies identically (observed four consecutive 45-minute cancellations on main and PR #188 after #186's lock bump). Warm-cache runs stay ~15 minutes; 75 is headroom matching the rustfs job's budget, not a target. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- .github/workflows/ci.yml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 15e6d11..a77108f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -127,7 +127,13 @@ jobs: name: Test Workspace needs: classify_changes runs-on: ubuntu-latest - timeout-minutes: 45 + # 75, not 45: a cold rust-cache (every Cargo.lock change) costs a full + # workspace + failpoints-feature build on a 2-core runner, which now + # exceeds 45 minutes on slow runner days. A timed-out run never SAVES + # its cache, so an undersized budget self-perpetuates: every retry + # starts cold and dies the same way (observed 2026-06-11, four runs). + # Warm-cache runs stay ~15 minutes; this is headroom, not a target. + timeout-minutes: 75 permissions: contents: write env: From fd002abaa50380f65fda89567adbf3e9c7043561 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 14:11:14 +0300 Subject: [PATCH 109/165] feat(cluster): port the storage backend to the engine StorageAdapter MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit LocalStateBackend becomes ClusterStore: every stored byte — state ledger, lock, recovery sidecars, approval artifacts — now flows through the engine's StorageAdapter, making file:// and s3:// one code path. Behavior on the file backend is byte-compatible (layout, CAS semantics, diagnostics, lock release timing) and the entire pre-existing suite passes unchanged. Mechanics: the ledger CAS keeps its public sha256 vocabulary while the physical swap is token-conditioned (ETag If-Match on S3 via PR #186's primitives; content-token + temp/rename locally — the pre-port semantics); the lock is a create-only put (genuinely cross-machine on object stores) with deterministic drop-release locally and best-effort spawned release on S3; sidecars/approvals address by URI (SweepOutcome and the executors carry strings); sweep row-1 retirement joins the uniform deferred post-CAS cleanup. ClusterStore also gains the catalog-payload and graph-root methods that commit 2 wires in. Async ripple: status/force-unlock/serving-snapshot and the server's settings loader chain go async (CLI dispatch and ~20 test hosts follow, mechanically). tokio joins the cluster crate's runtime deps for the lock guard's handle. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 4 +- crates/omnigraph-cluster/Cargo.toml | 4 + crates/omnigraph-cluster/src/diff.rs | 2 +- crates/omnigraph-cluster/src/lib.rs | 86 +-- crates/omnigraph-cluster/src/serve.rs | 10 +- crates/omnigraph-cluster/src/store.rs | 791 +++++++++++++++--------- crates/omnigraph-cluster/src/sweep.rs | 22 +- crates/omnigraph-cluster/src/tests.rs | 98 +-- crates/omnigraph-cluster/src/types.rs | 3 +- crates/omnigraph-server/src/lib.rs | 42 +- crates/omnigraph-server/src/main.rs | 3 +- crates/omnigraph-server/tests/server.rs | 101 +-- 12 files changed, 687 insertions(+), 479 deletions(-) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index da3cc44..e9cff0c 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -3720,7 +3720,7 @@ async fn main() -> Result<()> { finish_cluster_approve(&output, json)?; } ClusterCommand::Status { config, json } => { - let output = status_config_dir(config); + let output = status_config_dir(config).await; finish_cluster_status(&output, json)?; } ClusterCommand::Refresh { config, json } => { @@ -3736,7 +3736,7 @@ async fn main() -> Result<()> { config, json, } => { - let output = force_unlock_config_dir(config, lock_id); + let output = force_unlock_config_dir(config, lock_id).await; finish_cluster_force_unlock(&output, json)?; } }, diff --git a/crates/omnigraph-cluster/Cargo.toml b/crates/omnigraph-cluster/Cargo.toml index b5f99c9..973de6d 100644 --- a/crates/omnigraph-cluster/Cargo.toml +++ b/crates/omnigraph-cluster/Cargo.toml @@ -23,6 +23,10 @@ serde_yaml = { workspace = true } sha2 = { workspace = true } thiserror = { workspace = true } time = { workspace = true } +# Runtime handle only — best-effort async lock release in +# StateLockGuard::drop on object-store backends (cluster commands always +# run inside the caller's tokio runtime). +tokio = { workspace = true } ulid = { workspace = true } [dev-dependencies] diff --git a/crates/omnigraph-cluster/src/diff.rs b/crates/omnigraph-cluster/src/diff.rs index e75db4d..593b2fa 100644 --- a/crates/omnigraph-cluster/src/diff.rs +++ b/crates/omnigraph-cluster/src/diff.rs @@ -142,7 +142,7 @@ pub(crate) fn compute_approvals( /// Near-misses — an artifact for the same resource whose bound digests no /// longer match — warn as `approval_stale` and never authorize anything. pub(crate) fn approved_resources( - artifacts: &[(PathBuf, ApprovalArtifact)], + artifacts: &[(String, ApprovalArtifact)], changes: &[PlanChange], config_digest: &str, diagnostics: &mut Vec<Diagnostic>, diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index dc66408..ec1a02a 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -25,7 +25,7 @@ mod diff; mod serve; mod sweep; mod store; -use store::{LocalStateBackend, StateLockGuard, StateSnapshot}; +use store::{ClusterStore, StateLockGuard, StateSnapshot}; pub use types::*; use types::*; pub use serve::{ServingGraph, ServingPolicy, ServingQuery, ServingSnapshot, read_serving_snapshot}; @@ -69,7 +69,7 @@ pub fn validate_config_dir(config_dir: impl AsRef<Path>) -> ValidateOutput { pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { let outcome = load_desired(config_dir.as_ref()); let mut diagnostics = outcome.diagnostics; - let backend = LocalStateBackend::new(&outcome.config_dir); + let backend = ClusterStore::for_config_dir(&outcome.config_dir); let mut observations = backend.observations(); let Some(desired) = outcome.desired else { @@ -107,7 +107,7 @@ pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { } let _lock_guard = if desired.state_lock { - match backend.acquire_lock("plan", &mut observations) { + match backend.acquire_lock("plan", &mut observations).await { Ok(guard) => Some(guard), Err(diagnostic) => { diagnostics.push(diagnostic); @@ -130,7 +130,7 @@ pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { let mut prior_resources = BTreeMap::new(); let mut prior_state: Option<ClusterState> = None; if !has_errors(&diagnostics) { - match backend.read_state(&mut observations) { + match backend.read_state(&mut observations).await { Ok(snapshot) => { if let Some(state) = snapshot.state { prior_resources = state_resource_digests(&state); @@ -151,7 +151,7 @@ pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { } // Plan previews dispositions without sweeping; a pending recovery is // surfaced as the cluster_recovery_pending warning above instead. - let artifacts = backend.list_approval_artifacts(&mut diagnostics); + let artifacts = backend.list_approval_artifacts(&mut diagnostics).await; let approved = approved_resources( &artifacts, &changes, @@ -242,7 +242,7 @@ pub async fn apply_config_dir_with_options( ) -> ApplyOutput { let outcome = load_desired(config_dir.as_ref()); let mut diagnostics = outcome.diagnostics; - let backend = LocalStateBackend::new(&outcome.config_dir); + let backend = ClusterStore::for_config_dir(&outcome.config_dir); let mut observations = backend.observations(); let actor_for_output = options.actor.clone(); @@ -294,7 +294,7 @@ pub async fn apply_config_dir_with_options( // Named guard: the lock must be held until the state outcome is recorded. let _lock_guard = if desired.state_lock { - match backend.acquire_lock("apply", &mut observations) { + match backend.acquire_lock("apply", &mut observations).await { Ok(guard) => Some(guard), Err(diagnostic) => { diagnostics.push(diagnostic); @@ -321,7 +321,7 @@ pub async fn apply_config_dir_with_options( ); } - let snapshot = match backend.read_state(&mut observations) { + let snapshot = match backend.read_state(&mut observations).await { Ok(snapshot) => snapshot, Err(diagnostic) => { diagnostics.push(diagnostic); @@ -361,7 +361,7 @@ pub async fn apply_config_dir_with_options( let prior_resources = state_resource_digests(&state); let mut changes = diff_resources(&prior_resources, &desired.resource_digests); append_policy_binding_changes(&mut changes, Some(&state), &desired); - let approval_artifacts = backend.list_approval_artifacts(&mut diagnostics); + let approval_artifacts = backend.list_approval_artifacts(&mut diagnostics).await; let approved = approved_resources( &approval_artifacts, &changes, @@ -424,7 +424,7 @@ pub async fn apply_config_dir_with_options( }) .filter_map(|change| change.resource.strip_prefix("graph.").map(str::to_string)) .collect(); - let mut completed_op_sidecars: Vec<PathBuf> = Vec::new(); + let mut completed_op_sidecars: Vec<String> = Vec::new(); let mut failed_graphs: BTreeMap<String, FailedGraphOrigin> = BTreeMap::new(); let mut graph_moving_aborted = false; for graph_id in &graph_creates_to_run { @@ -462,7 +462,7 @@ pub async fn apply_config_dir_with_options( state_cas_base: expected_cas.clone(), approval_id: None, }; - let sidecar_path = match backend.write_recovery_sidecar(&sidecar) { + let sidecar_path = match backend.write_recovery_sidecar(&sidecar).await { Ok(path) => path, Err(diagnostic) => { diagnostics.push(diagnostic); @@ -514,7 +514,7 @@ pub async fn apply_config_dir_with_options( Ok(source) => source, Err(diagnostic) => { diagnostics.push(diagnostic); - let _ = fs::remove_file(&sidecar_path); // nothing moved + backend.delete_object(&sidecar_path).await; // nothing moved failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::GraphCreate); graph_moving_aborted = true; continue; @@ -540,7 +540,7 @@ pub async fn apply_config_dir_with_options( if let Ok(db) = Omnigraph::open_read_only(&graph_uri).await { if let Ok(snapshot) = db.snapshot_of(ReadTarget::branch("main")).await { sidecar.expected_manifest_version = Some(snapshot.version()); - if let Err(diagnostic) = backend.write_recovery_sidecar(&sidecar) { + if let Err(diagnostic) = backend.write_recovery_sidecar(&sidecar).await { diagnostics.push(diagnostic); } } @@ -626,7 +626,7 @@ pub async fn apply_config_dir_with_options( state_cas_base: expected_cas.clone(), approval_id: None, }; - let sidecar_path = match backend.write_recovery_sidecar(&sidecar) { + let sidecar_path = match backend.write_recovery_sidecar(&sidecar).await { Ok(path) => path, Err(diagnostic) => { diagnostics.push(diagnostic); @@ -677,7 +677,7 @@ pub async fn apply_config_dir_with_options( Ok(source) => source, Err(diagnostic) => { diagnostics.push(diagnostic); - let _ = fs::remove_file(&sidecar_path); // nothing moved + backend.delete_object(&sidecar_path).await; // nothing moved failed_graphs.insert(graph_id.clone(), FailedGraphOrigin::SchemaApply); graph_moving_aborted = true; continue; @@ -695,7 +695,7 @@ pub async fn apply_config_dir_with_options( { Ok(result) => { sidecar.expected_manifest_version = Some(result.manifest_version); - if let Err(diagnostic) = backend.write_recovery_sidecar(&sidecar) { + if let Err(diagnostic) = backend.write_recovery_sidecar(&sidecar).await { diagnostics.push(diagnostic); } } @@ -871,7 +871,7 @@ pub async fn apply_config_dir_with_options( state_cas_base: expected_cas.clone(), approval_id: approval_id.clone(), }; - let sidecar_path = match backend.write_recovery_sidecar(&sidecar) { + let sidecar_path = match backend.write_recovery_sidecar(&sidecar).await { Ok(path) => path, Err(diagnostic) => { diagnostics.push(diagnostic); @@ -1004,8 +1004,14 @@ pub async fn apply_config_dir_with_options( // persisted-statuses revert contract below is exercised; a cfg_callback // on this point can mutate state.json to simulate a concurrent writer, // making write_state's CAS check fail organically. - let write_result = failpoints::maybe_fail("cluster_apply.before_state_write") - .and_then(|()| backend.write_state(&new_state, expected_cas.as_deref(), &mut observations)); + let write_result = match failpoints::maybe_fail("cluster_apply.before_state_write") { + Ok(()) => { + backend + .write_state(&new_state, expected_cas.as_deref(), &mut observations) + .await + } + Err(diagnostic) => Err(diagnostic), + }; match write_result { Ok(()) => state_written = true, Err(diagnostic) => { @@ -1017,16 +1023,16 @@ pub async fn apply_config_dir_with_options( // Completed (rows 2/4) sweep sidecars are deleted only once their outcome // is durably recorded; on a failed write they stay and re-sweep next run. if !state_write_failed { - for sidecar_path in sweep + for sidecar_uri in sweep .completed_sidecars .iter() .chain(completed_op_sidecars.iter()) { - let _ = fs::remove_file(sidecar_path); + backend.delete_object(sidecar_uri).await; } let mut all_consumed = sweep.consumed_approvals.clone(); all_consumed.extend(consumed_approval_ids.iter().cloned()); - mark_approvals_consumed(&backend, &all_consumed); + mark_approvals_consumed(&backend, &all_consumed).await; } // On a failed state write, report the statuses that are actually on disk // (the pre-apply snapshot), not the in-memory mutations that were never @@ -1082,7 +1088,7 @@ pub async fn approve_config_dir( ) -> ApproveOutput { let outcome = load_desired(config_dir.as_ref()); let mut diagnostics = outcome.diagnostics; - let backend = LocalStateBackend::new(&outcome.config_dir); + let backend = ClusterStore::for_config_dir(&outcome.config_dir); let mut observations = backend.observations(); let fail = |config_dir: String, diagnostics: Vec<Diagnostic>| ApproveOutput { @@ -1103,7 +1109,7 @@ pub async fn approve_config_dir( } let _lock_guard = if desired.state_lock { - match backend.acquire_lock("approve", &mut observations) { + match backend.acquire_lock("approve", &mut observations).await { Ok(guard) => Some(guard), Err(diagnostic) => { diagnostics.push(diagnostic); @@ -1119,7 +1125,7 @@ pub async fn approve_config_dir( None }; - let state = match backend.read_state(&mut observations) { + let state = match backend.read_state(&mut observations).await { Ok(snapshot) => match snapshot.state { Some(state) => state, None => { @@ -1174,7 +1180,7 @@ pub async fn approve_config_dir( consumed_at: None, consumed_by_operation: None, }; - if let Err(diagnostic) = backend.write_approval_artifact(&artifact) { + if let Err(diagnostic) = backend.write_approval_artifact(&artifact).await { diagnostics.push(diagnostic); return fail(display_path(&desired.config_dir), diagnostics); } @@ -1191,12 +1197,12 @@ pub async fn approve_config_dir( } -pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { +pub async fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { let parsed = parse_cluster_config(config_dir.as_ref()); let mut diagnostics = parsed.diagnostics; - let backend = LocalStateBackend::new(&parsed.config_dir); + let backend = ClusterStore::for_config_dir(&parsed.config_dir); let mut observations = backend.observations(); - backend.observe_lock(&mut observations, &mut diagnostics); + backend.observe_lock(&mut observations, &mut diagnostics).await; warn_pending_recovery_sidecars(&parsed.config_dir, &mut diagnostics); let mut resource_digests = BTreeMap::new(); @@ -1206,7 +1212,7 @@ pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { if let Some(raw) = parsed.raw.as_ref() { let _settings = validate_cluster_header(raw, &mut diagnostics); if !has_errors(&diagnostics) { - match backend.read_state(&mut observations) { + match backend.read_state(&mut observations).await { Ok(snapshot) => { if let Some(state) = snapshot.state { // Read-only point-in-time catalog check: report the @@ -1244,20 +1250,20 @@ pub fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { } } -pub fn force_unlock_config_dir( +pub async fn force_unlock_config_dir( config_dir: impl AsRef<Path>, lock_id: impl AsRef<str>, ) -> ForceUnlockOutput { let parsed = parse_cluster_config(config_dir.as_ref()); let mut diagnostics = parsed.diagnostics; - let backend = LocalStateBackend::new(&parsed.config_dir); + let backend = ClusterStore::for_config_dir(&parsed.config_dir); let mut observations = backend.observations(); let mut lock_removed = false; if let Some(raw) = parsed.raw.as_ref() { let _settings = validate_cluster_header(raw, &mut diagnostics); if !has_errors(&diagnostics) { - match backend.force_unlock(lock_id.as_ref(), &mut observations) { + match backend.force_unlock(lock_id.as_ref(), &mut observations).await { Ok(()) => lock_removed = true, Err(diagnostic) => diagnostics.push(diagnostic), } @@ -1284,7 +1290,7 @@ pub async fn import_config_dir(config_dir: impl AsRef<Path>) -> StateSyncOutput async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> StateSyncOutput { let outcome = load_desired(config_dir); let mut diagnostics = outcome.diagnostics; - let backend = LocalStateBackend::new(&outcome.config_dir); + let backend = ClusterStore::for_config_dir(&outcome.config_dir); let mut observations = backend.observations(); let Some(desired) = outcome.desired else { @@ -1315,7 +1321,7 @@ async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> St let operation_label = state_sync_operation_label(operation); let _lock_guard = if desired.state_lock { - match backend.acquire_lock(operation_label, &mut observations) { + match backend.acquire_lock(operation_label, &mut observations).await { Ok(guard) => Some(guard), Err(diagnostic) => { diagnostics.push(diagnostic); @@ -1346,7 +1352,7 @@ async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> St }; } - let snapshot = match backend.read_state(&mut observations) { + let snapshot = match backend.read_state(&mut observations).await { Ok(snapshot) => snapshot, Err(diagnostic) => { diagnostics.push(diagnostic); @@ -1477,14 +1483,14 @@ async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> St state.state_revision = state.state_revision.saturating_add(1); } - match backend.write_state(&state, expected_cas.as_deref(), &mut observations) { + match backend.write_state(&state, expected_cas.as_deref(), &mut observations).await { Ok(()) => { // Completed sweep sidecars are deleted only after their outcome // is durably recorded; on failure they stay and re-sweep. - for sidecar_path in &sweep.completed_sidecars { - let _ = fs::remove_file(sidecar_path); + for sidecar_uri in &sweep.completed_sidecars { + backend.delete_object(sidecar_uri).await; } - mark_approvals_consumed(&backend, &sweep.consumed_approvals); + mark_approvals_consumed(&backend, &sweep.consumed_approvals).await; } Err(diagnostic) => diagnostics.push(diagnostic), } diff --git a/crates/omnigraph-cluster/src/serve.rs b/crates/omnigraph-cluster/src/serve.rs index 0152bc4..8578aee 100644 --- a/crates/omnigraph-cluster/src/serve.rs +++ b/crates/omnigraph-cluster/src/serve.rs @@ -40,13 +40,15 @@ pub struct ServingSnapshot { /// failure is collected and the whole snapshot refused; no partial serving. /// Takes no lock: the state file is replaced atomically, so this reads a /// consistent point-in-time ledger. -pub fn read_serving_snapshot(config_dir: impl AsRef<Path>) -> Result<ServingSnapshot, Vec<Diagnostic>> { +pub async fn read_serving_snapshot( + config_dir: impl AsRef<Path>, +) -> Result<ServingSnapshot, Vec<Diagnostic>> { let config_dir = config_dir.as_ref().to_path_buf(); - let backend = LocalStateBackend::new(&config_dir); + let backend = ClusterStore::for_config_dir(&config_dir); let mut diagnostics: Vec<Diagnostic> = Vec::new(); // A ledger a sweep is about to rewrite must not start serving. - let sidecars = backend.list_recovery_sidecars(&mut diagnostics); + let sidecars = backend.list_recovery_sidecars(&mut diagnostics).await; if !sidecars.is_empty() { diagnostics.push(Diagnostic::error( "cluster_recovery_pending", @@ -59,7 +61,7 @@ pub fn read_serving_snapshot(config_dir: impl AsRef<Path>) -> Result<ServingSnap } let mut observations = backend.observations(); - let state = match backend.read_state(&mut observations) { + let state = match backend.read_state(&mut observations).await { Ok(snapshot) => match snapshot.state { Some(state) => Some(state), None => { diff --git a/crates/omnigraph-cluster/src/store.rs b/crates/omnigraph-cluster/src/store.rs index 8a95661..bf328af 100644 --- a/crates/omnigraph-cluster/src/store.rs +++ b/crates/omnigraph-cluster/src/store.rs @@ -1,230 +1,446 @@ -//! The cluster's storage backend: state ledger, lock, recovery -//! sidecars, approval artifacts (moved verbatim from lib.rs in the -//! modularization). The object-storage port (RFC-006) lands here as a -//! follow-up — this module is the single home for stored-state I/O. +//! The cluster's storage layer: every stored byte (state ledger, lock, +//! recovery sidecars, approval artifacts, catalog payloads) goes through the +//! engine's `StorageAdapter`, so `file://` and `s3://` are one code path +//! (RFC-006). Declared configuration — `cluster.yaml` and the schema/query/ +//! policy sources it references — deliberately does NOT live here: config is +//! read from the operator's working tree (Terraform's config-local / +//! state-remote split). +//! +//! Raw `fs::*` for cluster state outside this module is a deny-list entry. -use super::*; +use std::path::Path; +use std::process; +use std::sync::Arc; -#[derive(Debug)] -pub(crate) struct LocalStateBackend { - state_dir: PathBuf, - state_path: PathBuf, - lock_path: PathBuf, - recoveries_dir: PathBuf, - approvals_dir: PathBuf, +use omnigraph::storage::{StorageAdapter, StorageKind, storage_for_uri, storage_kind_for_uri}; +use time::OffsetDateTime; +use time::format_description::well_known::Rfc3339; +use ulid::Ulid; + +use crate::{ + ApprovalArtifact, CLUSTER_APPROVALS_DIR, CLUSTER_LOCK_FILE, CLUSTER_RECOVERIES_DIR, + CLUSTER_RESOURCES_DIR, CLUSTER_STATE_FILE, ClusterState, Diagnostic, RecoverySidecar, + ResourceKind, StateLockFile, StateObservations, sha256_hex, +}; + +#[derive(Debug, Clone)] +pub(crate) struct ClusterStore { + adapter: Arc<dyn StorageAdapter>, + /// Normalized storage-root URI, no trailing slash: `file:///abs/dir` + /// (the default config-dir layout) or `s3://bucket/prefix`. + root: String, + /// What observations/diagnostics display for stored locations: the plain + /// local path for `file://` roots (byte-compatible with the pre-store + /// outputs), the URI otherwise. + display_root: String, } #[derive(Debug)] pub(crate) struct StateSnapshot { pub(crate) state: Option<ClusterState>, + /// Content identity (`sha256:<hex>`) — the public CAS vocabulary. pub(crate) state_cas: Option<String>, } #[derive(Debug)] pub(crate) struct StateLockGuard { - path: PathBuf, + adapter: Arc<dyn StorageAdapter>, + uri: String, + kind: StorageKind, } -impl LocalStateBackend { - pub(crate) fn new(config_dir: &Path) -> Self { - let state_dir = config_dir.join(CLUSTER_STATE_DIR); - Self { - state_path: config_dir.join(CLUSTER_STATE_FILE), - lock_path: config_dir.join(CLUSTER_LOCK_FILE), - recoveries_dir: config_dir.join(CLUSTER_RECOVERIES_DIR), - approvals_dir: config_dir.join(CLUSTER_APPROVALS_DIR), - state_dir, - } - } - - /// List approval artifacts in ULID (filename) order; unparseable files - /// warn and stay on disk for the operator. - pub(crate) fn list_approval_artifacts( - &self, - diagnostics: &mut Vec<Diagnostic>, - ) -> Vec<(PathBuf, ApprovalArtifact)> { - let mut paths = Vec::new(); - match fs::read_dir(&self.approvals_dir) { - Ok(entries) => { - for entry in entries.flatten() { - let path = entry.path(); - if path.extension().is_some_and(|ext| ext == "json") { - paths.push(path); - } +impl Drop for StateLockGuard { + fn drop(&mut self) { + match self.kind { + // Deterministic release on the file backend (tests assert the + // lock is gone the moment a command returns). + StorageKind::Local => { + let path = self.uri.trim_start_matches("file://"); + let _ = std::fs::remove_file(path); + } + // Object stores need an async delete; best-effort spawn. A crash + // here leaves the lock for `force-unlock` — same as a process + // kill, and the same recovery path. + StorageKind::S3 => { + let adapter = Arc::clone(&self.adapter); + let uri = self.uri.clone(); + if let Ok(handle) = tokio::runtime::Handle::try_current() { + handle.spawn(async move { + let _ = adapter.delete(&uri).await; + }); } } - Err(err) if err.kind() == ErrorKind::NotFound => {} - Err(err) => diagnostics.push(Diagnostic::warning( - "approval_read_error", - CLUSTER_APPROVALS_DIR, - format!("could not list approval artifacts: {err}"), - )), } - paths.sort(); - let mut artifacts = Vec::new(); - for path in paths { - match fs::read_to_string(&path) - .map_err(|err| err.to_string()) - .and_then(|text| { - serde_json::from_str::<ApprovalArtifact>(&text).map_err(|err| err.to_string()) - }) { - Ok(artifact) if artifact.schema_version == 1 => artifacts.push((path, artifact)), - Ok(artifact) => diagnostics.push(Diagnostic::warning( - "unsupported_approval_version", - display_path(&path), - format!( - "unsupported approval artifact version {}; leaving it in place", - artifact.schema_version - ), - )), + } +} + +impl ClusterStore { + /// The default layout: storage root = the config directory itself + /// (`file://<abs config dir>`), byte-compatible with every pre-existing + /// cluster on disk. + pub(crate) fn for_config_dir(config_dir: &Path) -> Self { + let absolute = + std::path::absolute(config_dir).unwrap_or_else(|_| config_dir.to_path_buf()); + let display_root = absolute + .to_string_lossy() + .trim_end_matches('/') + .to_string(); + let root = format!("file://{display_root}"); + let adapter = storage_for_uri(&root) + .expect("local storage adapter construction is infallible for file:// roots"); + Self { + adapter, + root, + display_root, + } + } + + /// An explicit `storage:` root. `file://` URIs and plain paths normalize + /// to the local backend; `s3://bucket/prefix` to the S3 backend (env- + /// driven credentials/endpoint — the same contract as graph storage). + pub(crate) fn for_storage_root(root_uri: &str) -> Result<Self, Diagnostic> { + let trimmed = root_uri.trim_end_matches('/'); + if storage_kind_for_uri(trimmed) == StorageKind::Local { + let path = trimmed.trim_start_matches("file://"); + return Ok(Self::for_config_dir(Path::new(path))); + } + let adapter = storage_for_uri(trimmed).map_err(|err| { + Diagnostic::error( + "storage_root_invalid", + "storage", + format!("could not initialize storage for '{root_uri}': {err}"), + ) + })?; + Ok(Self { + adapter, + root: trimmed.to_string(), + display_root: trimmed.to_string(), + }) + } + + pub(crate) fn kind(&self) -> StorageKind { + storage_kind_for_uri(&self.root) + } + + fn uri(&self, relative: &str) -> String { + format!("{}/{}", self.root, relative) + } + + fn display(&self, relative: &str) -> String { + format!("{}/{}", self.display_root, relative) + } + + /// Derived graph root for `<id>`: `<storage>/graphs/<id>.omni`. A plain + /// local path for `file://` roots (byte-compatible, directly usable by + /// the engine); the S3 URI the engine opens natively otherwise. + pub(crate) fn graph_root(&self, graph_id: &str) -> String { + match self.kind() { + StorageKind::Local => format!("{}/graphs/{graph_id}.omni", self.display_root), + StorageKind::S3 => format!("{}/graphs/{graph_id}.omni", self.root), + } + } + + /// `read_text_versioned`, returning None for a missing object (probed + /// via `exists` — the engine error type doesn't discriminate NotFound). + async fn read_versioned_opt(&self, uri: &str) -> Result<Option<(String, String)>, String> { + match self.adapter.exists(uri).await { + Ok(false) => return Ok(None), + Ok(true) => {} + Err(err) => return Err(err.to_string()), + } + self.adapter + .read_text_versioned(uri) + .await + .map(Some) + .map_err(|err| err.to_string()) + } + + /// JSON object write with the strongest atomicity the backend offers: + /// temp + rename on the filesystem (no torn JSON after a crash; the + /// pre-port behavior), a single atomic PUT on object stores (where + /// copy+delete would be weaker, not stronger). + async fn put_json(&self, relative: &str, payload: &str) -> Result<(), String> { + let target = self.uri(relative); + match self.kind() { + StorageKind::Local => { + let tmp = format!("{target}.tmp.{}", Ulid::new()); + self.adapter + .write_text(&tmp, payload) + .await + .map_err(|err| err.to_string())?; + if let Err(err) = self.adapter.rename_text(&tmp, &target).await { + let _ = self.adapter.delete(&tmp).await; + return Err(err.to_string()); + } + Ok(()) + } + StorageKind::S3 => self + .adapter + .write_text(&target, payload) + .await + .map_err(|err| err.to_string()), + } + } + + /// Shared list-and-parse for the sidecar/approval directories: id + /// (filename) order; unparseable objects warn and stay for the operator. + async fn list_json_dir<T: serde::de::DeserializeOwned>( + &self, + dir: &str, + diagnostics: &mut Vec<Diagnostic>, + list_error_code: &'static str, + parse_error_code: &'static str, + version_ok: impl Fn(&T) -> bool, + version_error_code: &'static str, + ) -> Vec<(String, T)> { + let dir_uri = self.uri(dir); + let mut uris = match self.adapter.list_dir(&dir_uri).await { + Ok(uris) => uris, + Err(err) => { + diagnostics.push(Diagnostic::warning( + list_error_code, + dir, + format!("could not list '{dir}': {err}"), + )); + return Vec::new(); + } + }; + uris.retain(|uri| uri.ends_with(".json")); + uris.sort(); + let mut out = Vec::new(); + for uri in uris { + match self.adapter.read_text(&uri).await { + Ok(text) => match serde_json::from_str::<T>(&text) { + Ok(value) if version_ok(&value) => out.push((uri, value)), + Ok(_) => diagnostics.push(Diagnostic::warning( + version_error_code, + uri.clone(), + "unsupported schema version; leaving it in place".to_string(), + )), + Err(err) => diagnostics.push(Diagnostic::warning( + parse_error_code, + uri.clone(), + format!("could not parse ({err}); leaving it in place"), + )), + }, Err(err) => diagnostics.push(Diagnostic::warning( - "invalid_approval_artifact", - display_path(&path), - format!("could not parse approval artifact ({err}); leaving it in place"), + parse_error_code, + uri.clone(), + format!("could not read ({err}); leaving it in place"), )), } } - artifacts + out } - /// Atomically write (or rewrite, e.g. on consumption) an approval artifact. - pub(crate) fn write_approval_artifact(&self, artifact: &ApprovalArtifact) -> Result<PathBuf, Diagnostic> { - fs::create_dir_all(&self.approvals_dir).map_err(|err| { - Diagnostic::error( - "approval_write_error", - CLUSTER_APPROVALS_DIR, - format!("could not create approvals directory: {err}"), - ) - })?; - let target = self - .approvals_dir - .join(format!("{}.json", artifact.approval_id)); + /// Best-effort object removal (sidecar retirement after a CAS lands, + /// lock cleanup) — failures are recoverable by the next sweep. + pub(crate) async fn delete_object(&self, uri: &str) { + let _ = self.adapter.delete(uri).await; + } + + /// Recursive prefix delete for graph roots (approved deletes). Idempotent; + /// S3 non-atomicity is tolerated by the delete protocol's retry shape. + pub(crate) async fn delete_graph_root(&self, graph_uri: &str) -> Result<(), String> { + self.adapter + .delete_prefix(graph_uri) + .await + .map_err(|err| err.to_string()) + } + + /// Existence probe for graph roots in sweep classification. A bare local + /// path or any URI works — resolved through the same adapter machinery + /// the engine uses. + pub(crate) async fn graph_root_exists(&self, graph_uri: &str) -> bool { + match storage_kind_for_uri(graph_uri) { + StorageKind::Local => Path::new(graph_uri.trim_start_matches("file://")).exists(), + StorageKind::S3 => match storage_for_uri(graph_uri) { + Ok(adapter) => !adapter + .list_dir(graph_uri) + .await + .map(|entries| entries.is_empty()) + .unwrap_or(true), + Err(_) => false, + }, + } + } + + // ---- approvals ---- + + pub(crate) async fn list_approval_artifacts( + &self, + diagnostics: &mut Vec<Diagnostic>, + ) -> Vec<(String, ApprovalArtifact)> { + self.list_json_dir( + CLUSTER_APPROVALS_DIR, + diagnostics, + "approval_read_error", + "invalid_approval_artifact", + |artifact: &ApprovalArtifact| artifact.schema_version == 1, + "unsupported_approval_version", + ) + .await + } + + pub(crate) async fn write_approval_artifact( + &self, + artifact: &ApprovalArtifact, + ) -> Result<String, Diagnostic> { + let relative = format!("{CLUSTER_APPROVALS_DIR}/{}.json", artifact.approval_id); let mut payload = serde_json::to_string_pretty(artifact).map_err(|err| { Diagnostic::error( "approval_write_error", - display_path(&target), + self.display(&relative), format!("could not encode approval artifact: {err}"), ) })?; payload.push('\n'); - let tmp_path = self - .approvals_dir - .join(format!("{}.json.tmp.{}", artifact.approval_id, Ulid::new())); - fs::write(&tmp_path, payload.as_bytes()).map_err(|err| { + self.put_json(&relative, &payload).await.map_err(|err| { Diagnostic::error( "approval_write_error", - display_path(&tmp_path), + self.display(&relative), format!("could not write approval artifact: {err}"), ) })?; - if let Err(err) = fs::rename(&tmp_path, &target) { - let _ = fs::remove_file(&tmp_path); - return Err(Diagnostic::error( - "approval_write_error", - display_path(&target), - format!("could not move approval artifact into place: {err}"), - )); - } - Ok(target) + Ok(self.uri(&relative)) } - /// List recovery sidecars in ULID (filename) order. Unparseable files are - /// reported as warnings and skipped — they stay on disk for the operator. - pub(crate) fn list_recovery_sidecars( + // ---- recovery sidecars ---- + + pub(crate) async fn list_recovery_sidecars( &self, diagnostics: &mut Vec<Diagnostic>, - ) -> Vec<(PathBuf, RecoverySidecar)> { - let mut paths = Vec::new(); - match fs::read_dir(&self.recoveries_dir) { - Ok(entries) => { - for entry in entries.flatten() { - let path = entry.path(); - if path.extension().is_some_and(|ext| ext == "json") { - paths.push(path); - } - } - } - Err(err) if err.kind() == ErrorKind::NotFound => {} - Err(err) => { - diagnostics.push(Diagnostic::warning( - "recovery_sidecar_read_error", - CLUSTER_RECOVERIES_DIR, - format!("could not list recovery sidecars: {err}"), - )); - } - } - paths.sort(); - let mut sidecars = Vec::new(); - for path in paths { - match fs::read_to_string(&path) - .map_err(|err| err.to_string()) - .and_then(|text| { - serde_json::from_str::<RecoverySidecar>(&text).map_err(|err| err.to_string()) - }) { - Ok(sidecar) if sidecar.schema_version == 1 => sidecars.push((path, sidecar)), - Ok(sidecar) => diagnostics.push(Diagnostic::warning( - "unsupported_recovery_sidecar_version", - display_path(&path), - format!( - "unsupported recovery sidecar version {}; leaving it in place", - sidecar.schema_version - ), - )), - Err(err) => diagnostics.push(Diagnostic::warning( - "invalid_recovery_sidecar", - display_path(&path), - format!("could not parse recovery sidecar ({err}); leaving it in place"), - )), - } - } - sidecars + ) -> Vec<(String, RecoverySidecar)> { + self.list_json_dir( + CLUSTER_RECOVERIES_DIR, + diagnostics, + "recovery_sidecar_read_error", + "invalid_recovery_sidecar", + |sidecar: &RecoverySidecar| sidecar.schema_version == 1, + "unsupported_recovery_sidecar_version", + ) + .await } - /// Atomically write (or rewrite) a recovery sidecar; returns its path. - pub(crate) fn write_recovery_sidecar(&self, sidecar: &RecoverySidecar) -> Result<PathBuf, Diagnostic> { - fs::create_dir_all(&self.recoveries_dir).map_err(|err| { - Diagnostic::error( - "recovery_sidecar_write_error", - CLUSTER_RECOVERIES_DIR, - format!("could not create recoveries directory: {err}"), - ) - })?; - let target = self - .recoveries_dir - .join(format!("{}.json", sidecar.operation_id)); + pub(crate) async fn write_recovery_sidecar( + &self, + sidecar: &RecoverySidecar, + ) -> Result<String, Diagnostic> { + let relative = format!("{CLUSTER_RECOVERIES_DIR}/{}.json", sidecar.operation_id); let mut payload = serde_json::to_string_pretty(sidecar).map_err(|err| { Diagnostic::error( "recovery_sidecar_write_error", - display_path(&target), + self.display(&relative), format!("could not encode recovery sidecar: {err}"), ) })?; payload.push('\n'); - let tmp_path = self - .recoveries_dir - .join(format!("{}.json.tmp.{}", sidecar.operation_id, Ulid::new())); - fs::write(&tmp_path, payload.as_bytes()).map_err(|err| { + self.put_json(&relative, &payload).await.map_err(|err| { Diagnostic::error( "recovery_sidecar_write_error", - display_path(&tmp_path), + self.display(&relative), format!("could not write recovery sidecar: {err}"), ) })?; - if let Err(err) = fs::rename(&tmp_path, &target) { - let _ = fs::remove_file(&tmp_path); + Ok(self.uri(&relative)) + } + + // ---- catalog payloads ---- + + /// Content-addressed catalog location for a query/policy payload + /// (extensions fixed per kind, same as the pre-port layout). + pub(crate) fn payload_relative(kind: &ResourceKind, digest: &str) -> Option<String> { + match kind { + ResourceKind::Query { graph, name } => Some(format!( + "{CLUSTER_RESOURCES_DIR}/query/{graph}/{name}/{digest}.gq" + )), + ResourceKind::Policy(name) => Some(format!( + "{CLUSTER_RESOURCES_DIR}/policy/{name}/{digest}.yaml" + )), + _ => None, + } + } + + pub(crate) fn payload_display(&self, kind: &ResourceKind, digest: &str) -> Option<String> { + Self::payload_relative(kind, digest).map(|relative| self.display(&relative)) + } + + pub(crate) async fn payload_exists(&self, kind: &ResourceKind, digest: &str) -> bool { + let Some(relative) = Self::payload_relative(kind, digest) else { + return false; + }; + self.adapter + .exists(&self.uri(&relative)) + .await + .unwrap_or(false) + } + + /// Idempotent content-addressed write: a payload already present at its + /// digest is by definition identical. + pub(crate) async fn write_payload( + &self, + kind: &ResourceKind, + digest: &str, + content: &str, + ) -> Result<(), String> { + let Some(relative) = Self::payload_relative(kind, digest) else { + return Err("resource kind has no payload".to_string()); + }; + if self + .adapter + .exists(&self.uri(&relative)) + .await + .map_err(|err| err.to_string())? + { + return Ok(()); + } + self.put_json(&relative, content).await + } + + /// Read a catalog payload and verify it against its recorded digest. + pub(crate) async fn read_verified_payload( + &self, + kind: &ResourceKind, + digest: &str, + address: &str, + ) -> Result<String, Diagnostic> { + let Some(relative) = Self::payload_relative(kind, digest) else { return Err(Diagnostic::error( - "recovery_sidecar_write_error", - display_path(&target), - format!("could not move recovery sidecar into place: {err}"), + "catalog_payload_missing", + address, + "resource kind has no payload", + )); + }; + let uri = self.uri(&relative); + let text = self.adapter.read_text(&uri).await.map_err(|err| { + Diagnostic::error( + "catalog_payload_missing", + address, + format!( + "catalog blob '{}' unreadable ({err}); run `cluster refresh` then `cluster apply`, and restart", + self.display(&relative) + ), + ) + })?; + if sha256_hex(text.as_bytes()) != digest { + return Err(Diagnostic::error( + "catalog_payload_digest_mismatch", + address, + format!( + "catalog blob '{}' does not match its recorded digest; run `cluster refresh` then `cluster apply`, and restart", + self.display(&relative) + ), )); } - Ok(target) + Ok(text) } + // ---- observations ---- + pub(crate) fn observations(&self) -> StateObservations { StateObservations { - state_path: display_path(&self.state_path), - lock_path: display_path(&self.lock_path), + state_path: self.display(CLUSTER_STATE_FILE), + lock_path: self.display(CLUSTER_LOCK_FILE), state_found: false, applied_config_digest: None, state_revision: 0, @@ -241,13 +457,16 @@ impl LocalStateBackend { } } - pub(crate) fn read_state( + // ---- state ledger ---- + + pub(crate) async fn read_state( &self, observations: &mut StateObservations, ) -> Result<StateSnapshot, Diagnostic> { - let text = match fs::read_to_string(&self.state_path) { - Ok(text) => text, - Err(err) if err.kind() == ErrorKind::NotFound => { + let state_uri = self.uri(CLUSTER_STATE_FILE); + let (text, _version) = match self.read_versioned_opt(&state_uri).await { + Ok(Some(read)) => read, + Ok(None) => { return Ok(StateSnapshot { state: None, state_cas: None, @@ -295,27 +514,32 @@ impl LocalStateBackend { }) } - pub(crate) fn write_state( + /// CAS-guarded ledger replace. The public contract stays content-level + /// (`expected_cas` = `sha256:<hex>` from the snapshot the command read); + /// the physical swap is token-conditioned on a fresh read, so a writer + /// that raced us between the fresh read and the put loses with + /// `state_cas_mismatch` — never a silent overwrite. On S3 the token is + /// the object's ETag and the put is conditional (If-Match); locally it + /// is a content token over the same temp+rename flow as before the port. + pub(crate) async fn write_state( &self, state: &ClusterState, expected_cas: Option<&str>, observations: &mut StateObservations, ) -> Result<(), Diagnostic> { - fs::create_dir_all(&self.state_dir).map_err(|err| { + let state_uri = self.uri(CLUSTER_STATE_FILE); + let current = self.read_versioned_opt(&state_uri).await.map_err(|err| { Diagnostic::error( "state_write_error", - CLUSTER_STATE_DIR, - format!("could not create cluster state directory: {err}"), + CLUSTER_STATE_FILE, + format!("could not read state file before write: {err}"), ) })?; - - let current_cas = self.current_state_cas()?; + let current_cas = current + .as_ref() + .map(|(text, _)| format!("sha256:{}", sha256_hex(text.as_bytes()))); if current_cas.as_deref() != expected_cas { - return Err(Diagnostic::error( - "state_cas_mismatch", - CLUSTER_STATE_FILE, - "state.json changed while the command was running; re-run the command against the latest state", - )); + return Err(state_cas_mismatch()); } let mut payload = serde_json::to_string_pretty(state).map_err(|err| { @@ -327,86 +551,51 @@ impl LocalStateBackend { })?; payload.push('\n'); - let tmp_path = self - .state_dir - .join(format!("state.json.tmp.{}", Ulid::new())); - let mut file = OpenOptions::new() - .write(true) - .create_new(true) - .open(&tmp_path) - .map_err(|err| { - Diagnostic::error( - "state_write_error", - display_path(&tmp_path), - format!("could not create temporary state file: {err}"), - ) - })?; - file.write_all(payload.as_bytes()).map_err(|err| { - Diagnostic::error( - "state_write_error", - display_path(&tmp_path), - format!("could not write temporary state file: {err}"), - ) - })?; - file.sync_all().map_err(|err| { - Diagnostic::error( - "state_write_error", - display_path(&tmp_path), - format!("could not sync temporary state file: {err}"), - ) - })?; - drop(file); - - if let Err(err) = fs::rename(&tmp_path, &self.state_path) { - let _ = fs::remove_file(&tmp_path); - return Err(Diagnostic::error( - "state_write_error", - CLUSTER_STATE_FILE, - format!("could not replace state.json atomically: {err}"), - )); + let written = match current { + None => self + .adapter + .write_text_if_absent(&state_uri, &payload) + .await + .map_err(|err| { + Diagnostic::error( + "state_write_error", + CLUSTER_STATE_FILE, + format!("could not create state.json: {err}"), + ) + })?, + Some((_, version)) => self + .adapter + .write_text_if_match(&state_uri, &payload, &version) + .await + .map_err(|err| { + Diagnostic::error( + "state_write_error", + CLUSTER_STATE_FILE, + format!("could not replace state.json: {err}"), + ) + })? + .is_some(), + }; + if !written { + return Err(state_cas_mismatch()); } - let written = fs::read_to_string(&self.state_path).map_err(|err| { - Diagnostic::error( - "state_write_error", - CLUSTER_STATE_FILE, - format!("could not read state.json after write: {err}"), - ) - })?; observations.state_found = true; observations.applied_config_digest = state.applied_revision.config_digest.clone(); observations.state_revision = state.state_revision; - observations.state_cas = Some(format!("sha256:{}", sha256_hex(written.as_bytes()))); + observations.state_cas = Some(format!("sha256:{}", sha256_hex(payload.as_bytes()))); observations.resource_count = state.applied_revision.resources.len(); - Ok(()) } - pub(crate) fn current_state_cas(&self) -> Result<Option<String>, Diagnostic> { - match fs::read(&self.state_path) { - Ok(bytes) => Ok(Some(format!("sha256:{}", sha256_hex(&bytes)))), - Err(err) if err.kind() == ErrorKind::NotFound => Ok(None), - Err(err) => Err(Diagnostic::error( - "state_read_error", - CLUSTER_STATE_FILE, - format!("could not read state file for CAS check: {err}"), - )), - } - } + // ---- lock ---- - pub(crate) fn acquire_lock( + pub(crate) async fn acquire_lock( &self, operation: &str, observations: &mut StateObservations, ) -> Result<StateLockGuard, Diagnostic> { - fs::create_dir_all(&self.state_dir).map_err(|err| { - Diagnostic::error( - "state_lock_error", - CLUSTER_STATE_DIR, - format!("could not create cluster state directory: {err}"), - ) - })?; - + let lock_uri = self.uri(CLUSTER_LOCK_FILE); let lock_id = Ulid::new().to_string(); let lock = StateLockFile { version: 1, @@ -425,31 +614,18 @@ impl LocalStateBackend { ) })?; - match OpenOptions::new() - .write(true) - .create_new(true) - .open(&self.lock_path) - { - Ok(mut file) => { - if let Err(err) = file.write_all(payload.as_bytes()) { - // No guard exists yet, so clean up the create-new file here - // instead of leaving a stale partial lock for the next run. - drop(file); - let _ = fs::remove_file(&self.lock_path); - return Err(Diagnostic::error( - "state_lock_error", - CLUSTER_LOCK_FILE, - format!("could not write state lock: {err}"), - )); - } + match self.adapter.write_text_if_absent(&lock_uri, &payload).await { + Ok(true) => { observations.lock_acquired = true; - observations.acquired_lock_id = Some(lock_id.clone()); + observations.acquired_lock_id = Some(lock_id); Ok(StateLockGuard { - path: self.lock_path.clone(), + adapter: Arc::clone(&self.adapter), + uri: lock_uri, + kind: self.kind(), }) } - Err(err) if err.kind() == ErrorKind::AlreadyExists => { - self.observe_lock_metadata_lossy(observations); + Ok(false) => { + self.observe_lock_metadata_lossy(observations).await; Err(Diagnostic::error( "state_lock_held", CLUSTER_LOCK_FILE, @@ -459,23 +635,24 @@ impl LocalStateBackend { Err(err) => Err(Diagnostic::error( "state_lock_error", CLUSTER_LOCK_FILE, - format!("could not acquire state lock: {err}"), + format!("could not write state lock: {err}"), )), } } - pub(crate) fn force_unlock( + pub(crate) async fn force_unlock( &self, - requested_lock_id: &str, + lock_id: &str, observations: &mut StateObservations, ) -> Result<(), Diagnostic> { - let text = match fs::read_to_string(&self.lock_path) { - Ok(text) => text, - Err(err) if err.kind() == ErrorKind::NotFound => { + let lock_uri = self.uri(CLUSTER_LOCK_FILE); + let text = match self.read_versioned_opt(&lock_uri).await { + Ok(Some((text, _))) => text, + Ok(None) => { return Err(Diagnostic::error( "state_lock_missing", CLUSTER_LOCK_FILE, - "cluster state lock is not present; nothing was unlocked", + "no cluster state lock is present", )); } Err(err) => { @@ -486,42 +663,41 @@ impl LocalStateBackend { )); } }; - observations.locked = true; let lock = parse_lock_file_for_unlock(&text)?; observations.observe_lock_metadata(&lock); - - if lock.lock_id != requested_lock_id { + observations.locked = true; + if lock.lock_id != lock_id { return Err(Diagnostic::error( "state_lock_id_mismatch", CLUSTER_LOCK_FILE, format!( - "cluster state lock id is {}; refusing to unlock with requested id {requested_lock_id}", + "lock id mismatch: held lock is {}, refusing to remove (pass the exact id from `cluster status`)", lock.lock_id ), )); } - - fs::remove_file(&self.lock_path).map_err(|err| { + self.adapter.delete(&lock_uri).await.map_err(|err| { Diagnostic::error( - "state_unlock_error", + "state_lock_error", CLUSTER_LOCK_FILE, format!("could not remove state lock: {err}"), ) - }) + })?; + observations.locked = false; + Ok(()) } - pub(crate) fn observe_lock( + pub(crate) async fn observe_lock( &self, observations: &mut StateObservations, diagnostics: &mut Vec<Diagnostic>, ) { - if self.lock_path.exists() { - observations.locked = true; - match fs::read_to_string(&self.lock_path) { - Ok(text) => match serde_json::from_str::<StateLockFile>(&text) { - Ok(lock) if lock.version == 1 => { - observations.observe_lock_metadata(&lock); - } + let lock_uri = self.uri(CLUSTER_LOCK_FILE); + match self.read_versioned_opt(&lock_uri).await { + Ok(Some((text, _))) => { + observations.locked = true; + match serde_json::from_str::<StateLockFile>(&text) { + Ok(lock) if lock.version == 1 => observations.observe_lock_metadata(&lock), Ok(lock) => diagnostics.push(Diagnostic::warning( "unsupported_state_lock_version", CLUSTER_LOCK_FILE, @@ -532,19 +708,24 @@ impl LocalStateBackend { CLUSTER_LOCK_FILE, format!("could not parse state lock: {err}"), )), - }, - Err(err) => diagnostics.push(Diagnostic::warning( - "state_lock_read_error", - CLUSTER_LOCK_FILE, - format!("could not read state lock: {err}"), - )), + } } + Ok(None) => {} + Err(err) => diagnostics.push(Diagnostic::warning( + "state_lock_read_error", + CLUSTER_LOCK_FILE, + format!("could not read state lock: {err}"), + )), } } - pub(crate) fn observe_lock_metadata_lossy(&self, observations: &mut StateObservations) { + pub(crate) async fn observe_lock_metadata_lossy( + &self, + observations: &mut StateObservations, + ) { observations.locked = true; - if let Ok(text) = fs::read_to_string(&self.lock_path) { + let lock_uri = self.uri(CLUSTER_LOCK_FILE); + if let Ok(Some((text, _))) = self.read_versioned_opt(&lock_uri).await { if let Ok(lock) = serde_json::from_str::<StateLockFile>(&text) { if lock.version == 1 { observations.observe_lock_metadata(&lock); @@ -554,10 +735,12 @@ impl LocalStateBackend { } } -impl Drop for StateLockGuard { - fn drop(&mut self) { - let _ = fs::remove_file(&self.path); - } +fn state_cas_mismatch() -> Diagnostic { + Diagnostic::error( + "state_cas_mismatch", + CLUSTER_STATE_FILE, + "state.json changed while the command was running; re-run the command against the latest state", + ) } pub(crate) fn parse_lock_file_for_unlock(text: &str) -> Result<StateLockFile, Diagnostic> { diff --git a/crates/omnigraph-cluster/src/sweep.rs b/crates/omnigraph-cluster/src/sweep.rs index 77ad8c5..2cfd7d1 100644 --- a/crates/omnigraph-cluster/src/sweep.rs +++ b/crates/omnigraph-cluster/src/sweep.rs @@ -11,12 +11,12 @@ use super::*; /// Mutations ride the calling command's CAS-checked state write; completed /// sidecars are deleted only after that write lands. pub(crate) async fn sweep_recovery_sidecars( - backend: &LocalStateBackend, + backend: &ClusterStore, state: &mut ClusterState, diagnostics: &mut Vec<Diagnostic>, ) -> SweepOutcome { let mut outcome = SweepOutcome::default(); - for (path, sidecar) in backend.list_recovery_sidecars(diagnostics) { + for (path, sidecar) in backend.list_recovery_sidecars(diagnostics).await { match sidecar.kind { RecoverySidecarKind::GraphCreate => { sweep_graph_create_sidecar(path, sidecar, state, diagnostics, &mut outcome).await; @@ -33,7 +33,7 @@ pub(crate) async fn sweep_recovery_sidecars( } pub(crate) async fn sweep_graph_create_sidecar( - path: PathBuf, + path: String, sidecar: RecoverySidecar, state: &mut ClusterState, diagnostics: &mut Vec<Diagnostic>, @@ -44,9 +44,11 @@ pub(crate) async fn sweep_graph_create_sidecar( let graph_path = PathBuf::from(&sidecar.graph_uri); // Row 1: nothing moved — the init never landed. The sidecar is pure - // intent; remove it and let the command's own plan re-propose the create. + // intent; retire it (deferred to the command's post-CAS cleanup, like + // every other completed sidecar — a failed CAS simply re-sweeps it) and + // let the command's own plan re-propose the create. if !graph_path.exists() { - let _ = fs::remove_file(&path); + outcome.completed_sidecars.push(path); return; } @@ -153,7 +155,7 @@ pub(crate) async fn sweep_graph_create_sidecar( } pub(crate) async fn sweep_schema_apply_sidecar( - path: PathBuf, + path: String, sidecar: RecoverySidecar, state: &mut ClusterState, diagnostics: &mut Vec<Diagnostic>, @@ -250,7 +252,7 @@ pub(crate) async fn sweep_schema_apply_sidecar( } pub(crate) fn sweep_graph_delete_sidecar( - path: PathBuf, + path: String, sidecar: RecoverySidecar, state: &mut ClusterState, diagnostics: &mut Vec<Diagnostic>, @@ -351,15 +353,15 @@ pub(crate) fn record_approval_consumed(state: &mut ClusterState, approval_id: &s } /// Mark approval artifact files consumed on disk (post-CAS). -pub(crate) fn mark_approvals_consumed(backend: &LocalStateBackend, approval_ids: &[String]) { +pub(crate) async fn mark_approvals_consumed(backend: &ClusterStore, approval_ids: &[String]) { if approval_ids.is_empty() { return; } let mut sink = Vec::new(); - for (_, mut artifact) in backend.list_approval_artifacts(&mut sink) { + for (_, mut artifact) in backend.list_approval_artifacts(&mut sink).await { if approval_ids.contains(&artifact.approval_id) && artifact.consumed_at.is_none() { artifact.consumed_at = Some(now_rfc3339()); - let _ = backend.write_approval_artifact(&artifact); + let _ = backend.write_approval_artifact(&artifact).await; } } } diff --git a/crates/omnigraph-cluster/src/tests.rs b/crates/omnigraph-cluster/src/tests.rs index a03c522..3b7984d 100644 --- a/crates/omnigraph-cluster/src/tests.rs +++ b/crates/omnigraph-cluster/src/tests.rs @@ -351,8 +351,8 @@ policies: })); } - #[test] - fn extended_state_json_status_surfaces_statuses() { + #[tokio::test] + async fn extended_state_json_status_surfaces_statuses() { let dir = fixture(); let state_dir = dir.path().join(CLUSTER_STATE_DIR); fs::create_dir_all(&state_dir).unwrap(); @@ -380,7 +380,7 @@ policies: }"#; fs::write(state_dir.join("state.json"), state).unwrap(); - let out = status_config_dir(dir.path()); + let out = status_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert!(out.state_observations.state_found); assert_eq!(out.state_observations.state_revision, 42); @@ -400,10 +400,10 @@ policies: ); } - #[test] - fn missing_state_status_succeeds_with_warning() { + #[tokio::test] + async fn missing_state_status_succeeds_with_warning() { let dir = fixture(); - let out = status_config_dir(dir.path()); + let out = status_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert!(!out.state_observations.state_found); assert_eq!(out.state_observations.state_revision, 0); @@ -414,14 +414,14 @@ policies: ); } - #[test] - fn invalid_state_status_fails() { + #[tokio::test] + async fn invalid_state_status_fails() { let dir = fixture(); let state_dir = dir.path().join(CLUSTER_STATE_DIR); fs::create_dir_all(&state_dir).unwrap(); fs::write(state_dir.join("state.json"), "{").unwrap(); - let out = status_config_dir(dir.path()); + let out = status_config_dir(dir.path()).await; assert!(!out.ok); assert!(out.state_observations.state_found); assert!( @@ -431,12 +431,12 @@ policies: ); } - #[test] - fn status_surfaces_full_lock_metadata() { + #[tokio::test] + async fn status_surfaces_full_lock_metadata() { let dir = fixture(); write_lock_file(dir.path(), "held-lock", "refresh"); - let out = status_config_dir(dir.path()); + let out = status_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert!(out.state_observations.locked); assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); @@ -452,12 +452,12 @@ policies: assert!(out.state_observations.lock_age_seconds.is_some()); } - #[test] - fn force_unlock_matching_id_removes_lock() { + #[tokio::test] + async fn force_unlock_matching_id_removes_lock() { let dir = fixture(); write_lock_file(dir.path(), "held-lock", "plan"); - let out = force_unlock_config_dir(dir.path(), "held-lock"); + let out = force_unlock_config_dir(dir.path(), "held-lock").await; assert!(out.ok, "{:?}", out.diagnostics); assert!(out.lock_removed); assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); @@ -468,12 +468,12 @@ policies: assert!(!dir.path().join(CLUSTER_LOCK_FILE).exists()); } - #[test] - fn force_unlock_wrong_id_fails_and_preserves_lock() { + #[tokio::test] + async fn force_unlock_wrong_id_fails_and_preserves_lock() { let dir = fixture(); write_lock_file(dir.path(), "held-lock", "plan"); - let out = force_unlock_config_dir(dir.path(), "other-lock"); + let out = force_unlock_config_dir(dir.path(), "other-lock").await; assert!(!out.ok); assert!(!out.lock_removed); assert_eq!(out.state_observations.lock_id.as_deref(), Some("held-lock")); @@ -485,11 +485,11 @@ policies: assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); } - #[test] - fn force_unlock_missing_lock_fails() { + #[tokio::test] + async fn force_unlock_missing_lock_fails() { let dir = fixture(); - let out = force_unlock_config_dir(dir.path(), "held-lock"); + let out = force_unlock_config_dir(dir.path(), "held-lock").await; assert!(!out.ok); assert!(!out.lock_removed); assert!(!out.state_observations.locked); @@ -500,14 +500,14 @@ policies: ); } - #[test] - fn force_unlock_invalid_lock_json_fails_and_preserves_lock() { + #[tokio::test] + async fn force_unlock_invalid_lock_json_fails_and_preserves_lock() { let dir = fixture(); let state_dir = dir.path().join(CLUSTER_STATE_DIR); fs::create_dir_all(&state_dir).unwrap(); fs::write(state_dir.join("lock.json"), "{").unwrap(); - let out = force_unlock_config_dir(dir.path(), "held-lock"); + let out = force_unlock_config_dir(dir.path(), "held-lock").await; assert!(!out.ok); assert!(!out.lock_removed); assert!( @@ -518,8 +518,8 @@ policies: assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); } - #[test] - fn force_unlock_unsupported_lock_version_fails_and_preserves_lock() { + #[tokio::test] + async fn force_unlock_unsupported_lock_version_fails_and_preserves_lock() { let dir = fixture(); let state_dir = dir.path().join(CLUSTER_STATE_DIR); fs::create_dir_all(&state_dir).unwrap(); @@ -529,7 +529,7 @@ policies: ) .unwrap(); - let out = force_unlock_config_dir(dir.path(), "held-lock"); + let out = force_unlock_config_dir(dir.path(), "held-lock").await; assert!(!out.ok); assert!(!out.lock_removed); assert!( @@ -540,8 +540,8 @@ policies: assert!(dir.path().join(CLUSTER_LOCK_FILE).exists()); } - #[test] - fn force_unlock_external_state_backend_rejected() { + #[tokio::test] + async fn force_unlock_external_state_backend_rejected() { let dir = fixture(); write_lock_file(dir.path(), "held-lock", "plan"); fs::write( @@ -557,7 +557,7 @@ graphs: ) .unwrap(); - let out = force_unlock_config_dir(dir.path(), "held-lock"); + let out = force_unlock_config_dir(dir.path(), "held-lock").await; assert!(!out.ok); assert!(!out.lock_removed); assert!( @@ -582,7 +582,7 @@ graphs: .any(|diagnostic| diagnostic.code == "state_lock_held") ); - let unlocked = force_unlock_config_dir(dir.path(), "held-lock"); + let unlocked = force_unlock_config_dir(dir.path(), "held-lock").await; assert!(unlocked.ok, "{:?}", unlocked.diagnostics); let out = plan_config_dir(dir.path()).await; @@ -1886,7 +1886,7 @@ graphs: let state_before = fs::read_to_string(dir.path().join(CLUSTER_STATE_FILE)).unwrap(); fs::remove_file(&blob).unwrap(); - let out = status_config_dir(dir.path()); + let out = status_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert!(out.diagnostics.iter().any(|diagnostic| { diagnostic.code == "catalog_payload_missing" @@ -2001,7 +2001,7 @@ graphs: assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); assert_eq!(fs::read_to_string(&blob).unwrap(), original); - let status = status_config_dir(dir.path()); + let status = status_config_dir(dir.path()).await; assert!( !status .diagnostics @@ -2012,12 +2012,12 @@ graphs: ); } - #[test] - fn verification_skips_graph_and_schema_resources() { + #[tokio::test] + async fn verification_skips_graph_and_schema_resources() { let dir = fixture(); write_applyable_state(dir.path()); // graph + schema digests only, no blobs - let out = status_config_dir(dir.path()); + let out = status_config_dir(dir.path()).await; assert!( !out.diagnostics .iter() @@ -2770,7 +2770,7 @@ policies: let converge = apply_config_dir(dir.path()).await; assert!(converge.converged, "{converge:?}"); - let snapshot = read_serving_snapshot(dir.path()).expect("converged cluster must serve"); + let snapshot = read_serving_snapshot(dir.path()).await.expect("converged cluster must serve"); assert_eq!(snapshot.graphs.len(), 1); assert_eq!(snapshot.graphs[0].graph_id, "knowledge"); assert!(snapshot.graphs[0].root.ends_with("graphs/knowledge.omni")); @@ -2782,10 +2782,10 @@ policies: assert!(snapshot.policies[0].blob_path.exists()); } - #[test] - fn serving_snapshot_refuses_missing_state() { + #[tokio::test] + async fn serving_snapshot_refuses_missing_state() { let dir = fixture(); - let err = read_serving_snapshot(dir.path()).unwrap_err(); + let err = read_serving_snapshot(dir.path()).await.unwrap_err(); assert!( err.iter().any(|diagnostic| diagnostic.code == "cluster_state_missing"), "{err:?}" @@ -2800,7 +2800,7 @@ policies: apply_config_dir(dir.path()).await; write_schema_apply_sidecar(dir.path(), "knowledge", "whatever", "01SERVE"); - let err = read_serving_snapshot(dir.path()).unwrap_err(); + let err = read_serving_snapshot(dir.path()).await.unwrap_err(); assert!( err.iter().any(|diagnostic| diagnostic.code == "cluster_recovery_pending"), "{err:?}" @@ -2814,7 +2814,7 @@ policies: write_applyable_state(dir.path()); apply_config_dir(dir.path()).await; // Tamper with the query blob... - let snapshot = read_serving_snapshot(dir.path()).unwrap(); + let snapshot = read_serving_snapshot(dir.path()).await.unwrap(); let desired = validate_config_dir(dir.path()); let query_digest = &desired.resource_digests["query.knowledge.find_person"]; let blob = dir @@ -2838,7 +2838,7 @@ policies: ) .unwrap(); - let err = read_serving_snapshot(dir.path()).unwrap_err(); + let err = read_serving_snapshot(dir.path()).await.unwrap_err(); assert!( err.iter() .any(|diagnostic| diagnostic.code == "catalog_payload_digest_mismatch"), @@ -2851,12 +2851,12 @@ policies: let _ = snapshot; // the pre-tamper read succeeded } - #[test] - fn serving_snapshot_refuses_empty_cluster() { + #[tokio::test] + async fn serving_snapshot_refuses_empty_cluster() { let dir = fixture(); write_state_resources(dir.path(), &[]); // state exists, no graphs - let err = read_serving_snapshot(dir.path()).unwrap_err(); + let err = read_serving_snapshot(dir.path()).await.unwrap_err(); assert!( err.iter().any(|diagnostic| diagnostic.code == "cluster_empty"), "{err:?}" @@ -2972,13 +2972,13 @@ policies: ); } - #[test] - fn status_warns_on_pending_recovery_sidecar() { + #[tokio::test] + async fn status_warns_on_pending_recovery_sidecar() { let dir = fixture(); write_applyable_state(dir.path()); write_create_sidecar(dir.path(), "knowledge", "irrelevant", "01STATUS"); - let out = status_config_dir(dir.path()); + let out = status_config_dir(dir.path()).await; assert!(out.ok, "{:?}", out.diagnostics); assert!( out.diagnostics diff --git a/crates/omnigraph-cluster/src/types.rs b/crates/omnigraph-cluster/src/types.rs index c366f04..ca960a5 100644 --- a/crates/omnigraph-cluster/src/types.rs +++ b/crates/omnigraph-cluster/src/types.rs @@ -503,7 +503,8 @@ pub(crate) struct SweepOutcome { pub(crate) pending_graphs: BTreeSet<String>, /// Sidecars whose outcome is recorded (rows 2/4): deleted only after the /// command's state write lands, so a CAS failure re-sweeps them. - pub(crate) completed_sidecars: Vec<PathBuf>, + /// Store URIs (the storage layer addresses everything by URI). + pub(crate) completed_sidecars: Vec<String>, /// Approval artifacts consumed by a roll-forward (delete row 7b): their /// files are rewritten with consumed_at only after the state write lands. pub(crate) consumed_approvals: Vec<String>, diff --git a/crates/omnigraph-server/src/lib.rs b/crates/omnigraph-server/src/lib.rs index 0038674..f7fc6b1 100644 --- a/crates/omnigraph-server/src/lib.rs +++ b/crates/omnigraph-server/src/lib.rs @@ -893,12 +893,12 @@ fn format_registry_load_errors(label: &str, errors: &[queries::LoadError]) -> St /// catalog blob content, policy bundles from blob paths with their applied /// bindings. Always multi-graph routing. The unauthenticated/env handling /// matches the omnigraph.yaml path. -fn load_cluster_settings( +async fn load_cluster_settings( cluster_dir: &PathBuf, cli_bind: Option<String>, cli_allow_unauthenticated: bool, ) -> Result<ServerConfig> { - let snapshot = omnigraph_cluster::read_serving_snapshot(cluster_dir).map_err(|diagnostics| { + let snapshot = omnigraph_cluster::read_serving_snapshot(cluster_dir).await.map_err(|diagnostics| { let details = diagnostics .iter() .map(|diagnostic| format!("[{}] {}: {}", diagnostic.code, diagnostic.path, diagnostic.message)) @@ -988,7 +988,7 @@ fn load_cluster_settings( }) } -pub fn load_server_settings( +pub async fn load_server_settings( config_path: Option<&PathBuf>, cli_cluster: Option<&PathBuf>, cli_uri: Option<String>, @@ -1005,7 +1005,7 @@ pub fn load_server_settings( "--cluster is an exclusive boot source; it cannot combine with a graph URI, --target, or --config (axiom 15: a deployment serves from one source)" ); } - return load_cluster_settings(cluster_dir, cli_bind, cli_allow_unauthenticated); + return load_cluster_settings(cluster_dir, cli_bind, cli_allow_unauthenticated).await; } let config = load_config(config_path)?; let bind = cli_bind.unwrap_or_else(|| config.server_bind().to_string()); @@ -3363,8 +3363,8 @@ mod tests { ); } - #[test] - fn server_settings_load_from_yaml_config() { + #[tokio::test] + async fn server_settings_load_from_yaml_config() { let temp = tempdir().unwrap(); let config = temp.path().join("omnigraph.yaml"); fs::write( @@ -3380,7 +3380,7 @@ server: ) .unwrap(); - let settings = load_server_settings(Some(&config), None, None, None, None, false).unwrap(); + let settings = load_server_settings(Some(&config), None, None, None, None, false).await.unwrap(); match &settings.mode { ServerConfigMode::Single { uri, graph_id, .. } => { assert_eq!(uri, "/tmp/demo.omni"); @@ -3391,8 +3391,8 @@ server: assert_eq!(settings.bind, "0.0.0.0:9090"); } - #[test] - fn server_settings_cli_flags_override_yaml_config() { + #[tokio::test] + async fn server_settings_cli_flags_override_yaml_config() { let temp = tempdir().unwrap(); let config = temp.path().join("omnigraph.yaml"); fs::write( @@ -3416,6 +3416,7 @@ server: Some("0.0.0.0:9999".to_string()), false, ) + .await .unwrap(); match &settings.mode { ServerConfigMode::Single { uri, graph_id, .. } => { @@ -3427,8 +3428,8 @@ server: assert_eq!(settings.bind, "0.0.0.0:9999"); } - #[test] - fn server_settings_can_resolve_named_target() { + #[tokio::test] + async fn server_settings_can_resolve_named_target() { let temp = tempdir().unwrap(); let config = temp.path().join("omnigraph.yaml"); fs::write( @@ -3448,6 +3449,7 @@ server: let settings = load_server_settings(Some(&config), None, None, Some("dev".to_string()), None, false) + .await .unwrap(); match &settings.mode { ServerConfigMode::Single { uri, graph_id, .. } => { @@ -3458,9 +3460,9 @@ server: } } - #[test] - fn server_settings_require_uri_from_cli_or_config() { - let error = load_server_settings(None, None, None, None, None, false).unwrap_err(); + #[tokio::test] + async fn server_settings_require_uri_from_cli_or_config() { + let error = load_server_settings(None, None, None, None, None, false).await.unwrap_err(); assert!( error.to_string().contains("no graph to serve"), "expected mode-inference error, got: {error}", @@ -3598,9 +3600,9 @@ server: ); } - #[test] + #[tokio::test] #[serial] - fn unauthenticated_env_var_classification() { + async fn unauthenticated_env_var_classification() { // MR-723 PR A: closes the gap where the env-var read path inside // `load_server_settings` was structurally implemented but not // exercised by any test. Three properties to pin, all in one @@ -3627,7 +3629,7 @@ server: // Truthy values flip Open mode on, even with CLI flag off. for value in ["1", "true", "yes", "TRUE", "anything"] { let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some(value))]); - let settings = load_server_settings(Some(&config_path), None, None, None, None, false) + let settings = load_server_settings(Some(&config_path), None, None, None, None, false).await .expect("settings load should succeed"); assert!( settings.allow_unauthenticated, @@ -3638,7 +3640,7 @@ server: // Falsy values keep refusal behavior, even with CLI flag off. for value in ["0", "false", "FALSE", ""] { let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some(value))]); - let settings = load_server_settings(Some(&config_path), None, None, None, None, false) + let settings = load_server_settings(Some(&config_path), None, None, None, None, false).await .expect("settings load should succeed"); assert!( !settings.allow_unauthenticated, @@ -3648,7 +3650,7 @@ server: // Unset env var: also false. let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", None)]); - let settings = load_server_settings(Some(&config_path), None, None, None, None, false) + let settings = load_server_settings(Some(&config_path), None, None, None, None, false).await .expect("settings load should succeed"); assert!( !settings.allow_unauthenticated, @@ -3659,7 +3661,7 @@ server: // CLI flag wins even when env is falsy — `serve()` honors the // OR of both inputs. let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some("0"))]); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true) + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await .expect("settings load should succeed"); assert!( settings.allow_unauthenticated, diff --git a/crates/omnigraph-server/src/main.rs b/crates/omnigraph-server/src/main.rs index c71ea2f..9000910 100644 --- a/crates/omnigraph-server/src/main.rs +++ b/crates/omnigraph-server/src/main.rs @@ -43,6 +43,7 @@ async fn main() -> Result<()> { cli.target, cli.bind, cli.unauthenticated, - )?; + ) + .await?; serve(settings).await } diff --git a/crates/omnigraph-server/tests/server.rs b/crates/omnigraph-server/tests/server.rs index 7858587..d11c542 100644 --- a/crates/omnigraph-server/tests/server.rs +++ b/crates/omnigraph-server/tests/server.rs @@ -5567,8 +5567,8 @@ mod multi_graph_startup { /// `GraphId` validation runs at startup — a reserved name in /// `omnigraph.yaml` produces a clear error rather than getting /// rejected per-request. - #[test] - fn load_server_settings_rejects_reserved_graph_id() { + #[tokio::test] + async fn load_server_settings_rejects_reserved_graph_id() { let temp = tempfile::tempdir().unwrap(); let config_path = temp.path().join("omnigraph.yaml"); fs::write( @@ -5580,7 +5580,7 @@ graphs: "#, ) .unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, None, false).unwrap_err(); + let err = load_server_settings(Some(&config_path), None, None, None, None, false).await.unwrap_err(); assert!( err.to_string().contains("invalid graph id 'policies'"), "expected reserved-name rejection, got: {err}" @@ -5644,8 +5644,8 @@ graphs: // ── Four-rule mode inference matrix ─────────────────────────────── /// Rule 1: CLI positional URI → Single. - #[test] - fn mode_inference_cli_uri_is_single() { + #[tokio::test] + async fn mode_inference_cli_uri_is_single() { let settings = load_server_settings( None, None, @@ -5654,6 +5654,7 @@ graphs: None, true, // allow unauth so we get past the runtime-state check ) + .await .unwrap(); match settings.mode { ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/cli.omni"), @@ -5662,8 +5663,8 @@ graphs: } /// Rule 2: --target picks one graph from `graphs:` map → Single. - #[test] - fn mode_inference_cli_target_is_single() { + #[tokio::test] + async fn mode_inference_cli_target_is_single() { let temp = tempfile::tempdir().unwrap(); let config_path = temp.path().join("omnigraph.yaml"); fs::write( @@ -5679,6 +5680,7 @@ graphs: .unwrap(); let settings = load_server_settings(Some(&config_path), None, None, Some("alpha".into()), None, true) + .await .unwrap(); match settings.mode { ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/alpha.omni"), @@ -5687,8 +5689,8 @@ graphs: } /// Rule 3: `server.graph` set → Single (target picked from config). - #[test] - fn mode_inference_server_graph_is_single() { + #[tokio::test] + async fn mode_inference_server_graph_is_single() { let temp = tempfile::tempdir().unwrap(); let config_path = temp.path().join("omnigraph.yaml"); fs::write( @@ -5704,7 +5706,7 @@ server: "#, ) .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); match settings.mode { ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/beta.omni"), ServerConfigMode::Multi { .. } => panic!("expected Single (rule 3), got Multi"), @@ -5712,8 +5714,8 @@ server: } /// Rule 4: `--config` + non-empty `graphs:` + no single-mode selector → Multi. - #[test] - fn mode_inference_config_plus_graphs_is_multi() { + #[tokio::test] + async fn mode_inference_config_plus_graphs_is_multi() { let temp = tempfile::tempdir().unwrap(); let config_path = temp.path().join("omnigraph.yaml"); fs::write( @@ -5727,7 +5729,7 @@ graphs: "#, ) .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); match settings.mode { ServerConfigMode::Multi { graphs, .. } => { let ids: Vec<&str> = graphs.iter().map(|g| g.graph_id.as_str()).collect(); @@ -5738,8 +5740,8 @@ graphs: } } - #[test] - fn mode_inference_multi_rejects_top_level_policy_file() { + #[tokio::test] + async fn mode_inference_multi_rejects_top_level_policy_file() { let temp = tempfile::tempdir().unwrap(); let config_path = temp.path().join("omnigraph.yaml"); fs::write( @@ -5753,7 +5755,7 @@ graphs: "#, ) .unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap_err(); + let err = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap_err(); let msg = err.to_string(); assert!( msg.contains("top-level") && msg.contains("policy.file") && msg.contains("not honored"), @@ -5769,8 +5771,8 @@ graphs: ); } - #[test] - fn mode_inference_multi_rejects_top_level_queries() { + #[tokio::test] + async fn mode_inference_multi_rejects_top_level_queries() { // Symmetric to the policy guard: a top-level `queries:` block in // multi-graph mode is not honored (each graph uses its own), so it // is a loud error rather than a silent no-op. @@ -5781,7 +5783,7 @@ graphs: "queries:\n q:\n file: ./q.gq\ngraphs:\n alpha:\n uri: /tmp/alpha.omni\n", ) .unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap_err(); + let err = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap_err(); let msg = err.to_string(); assert!( msg.contains("queries") && msg.contains("not honored"), @@ -5789,8 +5791,8 @@ graphs: ); } - #[test] - fn single_mode_named_graph_rejects_top_level_blocks() { + #[tokio::test] + async fn single_mode_named_graph_rejects_top_level_blocks() { // Serving a graph by name (`--target`/`server.graph`) uses its // per-graph block; a populated top-level block would be silently // shadowed, so boot refuses and names the per-graph location. @@ -5803,6 +5805,7 @@ graphs: .unwrap(); let err = load_server_settings(Some(&config_path), None, None, Some("prod".to_string()), None, true) + .await .unwrap_err(); let msg = err.to_string(); assert!( @@ -5811,8 +5814,8 @@ graphs: ); } - #[test] - fn single_mode_named_graph_uses_per_graph_policy_and_queries() { + #[tokio::test] + async fn single_mode_named_graph_uses_per_graph_policy_and_queries() { // The identity rule: `--target prod` attaches `graphs.prod`'s own // policy + queries, not the top-level ones (which are absent here). let temp = tempfile::tempdir().unwrap(); @@ -5830,6 +5833,7 @@ graphs: .unwrap(); let settings = load_server_settings(Some(&config_path), None, None, Some("prod".to_string()), None, true) + .await .unwrap(); match settings.mode { ServerConfigMode::Single { @@ -5851,8 +5855,8 @@ graphs: } } - #[test] - fn mode_inference_normalizes_multi_graph_uris() { + #[tokio::test] + async fn mode_inference_normalizes_multi_graph_uris() { let temp = tempfile::tempdir().unwrap(); let graph = temp.path().join("alpha.omni"); let config_path = temp.path().join("omnigraph.yaml"); @@ -5868,7 +5872,7 @@ graphs: ), ) .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); match settings.mode { ServerConfigMode::Multi { graphs, .. } => { assert_eq!(graphs[0].uri, graph.to_string_lossy()); @@ -5878,9 +5882,9 @@ graphs: } /// Rule 5: nothing → error with migration hint. - #[test] - fn mode_inference_no_inputs_errors_with_migration_hint() { - let err = load_server_settings(None, None, None, None, None, true).unwrap_err(); + #[tokio::test] + async fn mode_inference_no_inputs_errors_with_migration_hint() { + let err = load_server_settings(None, None, None, None, None, true).await.unwrap_err(); let msg = err.to_string(); assert!( msg.contains("no graph to serve"), @@ -5890,19 +5894,19 @@ graphs: /// Rule 4 sub-case: `--config` with empty `graphs:` map and no /// single-mode selector → rule 5 fires (no graph to serve). - #[test] - fn mode_inference_empty_graphs_map_errors() { + #[tokio::test] + async fn mode_inference_empty_graphs_map_errors() { let temp = tempfile::tempdir().unwrap(); let config_path = temp.path().join("omnigraph.yaml"); fs::write(&config_path, "server:\n bind: 127.0.0.1:8080\n").unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap_err(); + let err = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap_err(); assert!(err.to_string().contains("no graph to serve")); } /// `--config` + `<URI>` together: URI wins → Single (the CLI URI /// takes precedence over the config's graphs map). - #[test] - fn mode_inference_cli_uri_overrides_graphs_map() { + #[tokio::test] + async fn mode_inference_cli_uri_overrides_graphs_map() { let temp = tempfile::tempdir().unwrap(); let config_path = temp.path().join("omnigraph.yaml"); fs::write( @@ -5922,6 +5926,7 @@ graphs: None, true, ) + .await .unwrap(); match settings.mode { ServerConfigMode::Single { uri, .. } => { @@ -5937,8 +5942,8 @@ graphs: } /// Per-graph `policy.file` is resolved relative to the config base_dir. - #[test] - fn per_graph_policy_file_is_resolved_relative_to_base_dir() { + #[tokio::test] + async fn per_graph_policy_file_is_resolved_relative_to_base_dir() { let temp = tempfile::tempdir().unwrap(); let config_path = temp.path().join("omnigraph.yaml"); fs::write( @@ -5954,7 +5959,7 @@ graphs: "#, ) .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); let graphs = match settings.mode { ServerConfigMode::Multi { graphs, .. } => graphs, _ => panic!("expected Multi"), @@ -5972,8 +5977,8 @@ graphs: } /// `server.policy.file` resolves alongside the graphs map. - #[test] - fn server_policy_file_is_resolved_relative_to_base_dir() { + #[tokio::test] + async fn server_policy_file_is_resolved_relative_to_base_dir() { let temp = tempfile::tempdir().unwrap(); let config_path = temp.path().join("omnigraph.yaml"); fs::write( @@ -5988,7 +5993,7 @@ graphs: "#, ) .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); match settings.mode { ServerConfigMode::Multi { server_policy_file, .. @@ -6268,7 +6273,7 @@ graphs: .unwrap(); let settings: ServerConfig = - load_server_settings(Some(&config_path), None, None, None, None, true).unwrap(); + load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); assert!(matches!(settings.mode, ServerConfigMode::Multi { .. })); match settings.mode { @@ -6321,14 +6326,14 @@ graphs: temp } -fn cluster_settings(dir: &Path) -> color_eyre::eyre::Result<omnigraph_server::ServerConfig> { - omnigraph_server::load_server_settings(None, Some(&dir.to_path_buf()), None, None, None, true) +async fn cluster_settings(dir: &Path) -> color_eyre::eyre::Result<omnigraph_server::ServerConfig> { + omnigraph_server::load_server_settings(None, Some(&dir.to_path_buf()), None, None, None, true).await } #[tokio::test] async fn cluster_boot_serves_applied_state() { let temp = converged_cluster_dir("").await; - let settings = cluster_settings(temp.path()).unwrap(); + let settings = cluster_settings(temp.path()).await.unwrap(); let omnigraph_server::ServerConfigMode::Multi { graphs, config_path, @@ -6444,7 +6449,7 @@ graphs: temp }; - let settings = cluster_settings(temp.path()).unwrap(); + let settings = cluster_settings(temp.path()).await.unwrap(); let omnigraph_server::ServerConfigMode::Multi { graphs, server_policy_file, @@ -6482,6 +6487,7 @@ async fn cluster_boot_refusals() { None, true, ) + .await .unwrap_err(); assert!(err.to_string().contains("exclusive boot source"), "{err}"); let err = omnigraph_server::load_server_settings( @@ -6492,6 +6498,7 @@ async fn cluster_boot_refusals() { None, true, ) + .await .unwrap_err(); assert!(err.to_string().contains("exclusive boot source"), "{err}"); @@ -6499,7 +6506,7 @@ async fn cluster_boot_refusals() { let blob_dir = dir.join("__cluster/resources/query/knowledge/find_person"); let blob = fs::read_dir(&blob_dir).unwrap().next().unwrap().unwrap().path(); fs::write(&blob, "tampered").unwrap(); - let err = cluster_settings(&dir).unwrap_err(); + let err = cluster_settings(&dir).await.unwrap_err(); assert!( err.to_string().contains("catalog_payload_digest_mismatch"), "{err}" @@ -6508,6 +6515,6 @@ async fn cluster_boot_refusals() { // Missing state refuses with the import/apply remedy. let empty = tempfile::tempdir().unwrap(); - let err = cluster_settings(empty.path()).unwrap_err(); + let err = cluster_settings(empty.path()).await.unwrap_err(); assert!(err.to_string().contains("cluster_state_missing"), "{err}"); } From 8dc2f1525509c3aae359317d0851e7ddce74f422 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 14:28:04 +0300 Subject: [PATCH 110/165] =?UTF-8?q?feat(cluster):=20the=20storage:=20root?= =?UTF-8?q?=20=E2=80=94=20state,=20catalog,=20and=20graph=20roots=20reloca?= =?UTF-8?q?table?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit cluster.yaml gains an optional storage: URI deciding where everything the cluster STORES lives: the state ledger, lock, content-addressed catalog, recovery sidecars, approval artifacts, and the derived graph roots (<storage>/graphs/<id>.omni). Absent, it defaults to the config directory itself — the original layout, byte-compatible, so pre-existing clusters and the whole test suite are untouched. Declared configuration always stays in the working tree (Terraform's config-local/state-remote split); credentials are env-only, never in cluster.yaml. Every command resolves its store from the declared root (a bad root is a loud invalid_storage_root). Graph-root derivation, the delete executor (prefix delete via the adapter), the sweep's existence probes, the catalog payload write/verify/read paths, and the serving snapshot all flow through ClusterStore — the last raw-fs holdouts for stored state are gone, and the deny-list gains the rule that keeps it that way. Tests: default-layout byte-compat, a file:// root relocating the entire cluster (ledger+catalog+graphs under the new root, nothing under the config dir, serving snapshot follows), invalid-root validation. 98 in-crate + 9 failpoints + full workspace gate green. The s3:// flavor lands with PR 3's gated RustFS e2e. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/config.rs | 40 +++- crates/omnigraph-cluster/src/lib.rs | 241 +++++++++++++------------ crates/omnigraph-cluster/src/serve.rs | 70 +++---- crates/omnigraph-cluster/src/store.rs | 28 +++ crates/omnigraph-cluster/src/sweep.rs | 14 +- crates/omnigraph-cluster/src/tests.rs | 64 +++++++ crates/omnigraph-cluster/src/types.rs | 11 +- docs/dev/invariants.md | 4 + docs/user/cluster-config.md | 14 ++ docs/user/cluster.md | 2 + 10 files changed, 309 insertions(+), 179 deletions(-) diff --git a/crates/omnigraph-cluster/src/config.rs b/crates/omnigraph-cluster/src/config.rs index ecdc71c..acc954d 100644 --- a/crates/omnigraph-cluster/src/config.rs +++ b/crates/omnigraph-cluster/src/config.rs @@ -239,8 +239,33 @@ pub(crate) fn validate_cluster_header( } } + if let Some(storage) = raw.storage.as_deref() { + let trimmed = storage.trim(); + if trimmed.is_empty() { + diagnostics.push(Diagnostic::error( + "invalid_storage_root", + "storage", + "storage must be a non-empty URI (e.g. s3://bucket/prefix) when provided", + )); + } else if let Some(rest) = trimmed.strip_prefix("s3://") { + if rest.trim_start_matches('/').is_empty() { + diagnostics.push(Diagnostic::error( + "invalid_storage_root", + "storage", + "storage s3:// URI must name a bucket", + )); + } + } + } + ClusterSettings { state_lock: raw.state.lock.unwrap_or(true), + storage_root: raw + .storage + .as_deref() + .map(str::trim) + .filter(|storage| !storage.is_empty()) + .map(|storage| storage.trim_end_matches('/').to_string()), } } @@ -271,19 +296,19 @@ pub(crate) fn initial_import_state(desired: &DesiredCluster) -> ClusterState { } -pub(crate) async fn observe_declared_graphs(desired: &DesiredCluster, state: &mut ClusterState) -> usize { +pub(crate) async fn observe_declared_graphs( + desired: &DesiredCluster, + backend: &ClusterStore, + state: &mut ClusterState, +) -> usize { let mut graph_error_count = 0; for graph in &desired.graphs { let graph_address = graph_address(&graph.id); let schema_address = schema_address(&graph.id); - let graph_path = desired - .config_dir - .join(CLUSTER_GRAPHS_DIR) - .join(format!("{}.omni", graph.id)); - let graph_uri = display_path(&graph_path); + let graph_uri = backend.graph_root(&graph.id); let observed_at = now_rfc3339(); - if !graph_path.exists() { + if !backend.graph_root_exists(&graph_uri).await { state.applied_revision.resources.remove(&graph_address); state.applied_revision.resources.remove(&schema_address); state.observations.insert( @@ -737,6 +762,7 @@ pub(crate) fn load_desired(config_dir: &Path) -> LoadOutcome { desired: Some(DesiredCluster { config_dir: config_dir.clone(), config_digest, + storage_root: settings.storage_root.clone(), state_lock: settings.state_lock, graphs, resource_digests, diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index ec1a02a..d97bb5b 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -29,7 +29,6 @@ use store::{ClusterStore, StateLockGuard, StateSnapshot}; pub use types::*; use types::*; pub use serve::{ServingGraph, ServingPolicy, ServingQuery, ServingSnapshot, read_serving_snapshot}; -use serve::read_verified_payload; use config::{QueriesDecl, observe_declared_graphs, validate_cluster_header, future_field_diagnostics, initial_import_state, observe_live_graph, preview_schema_migration, state_resource_digests, graph_address, policy_address, query_address, schema_address, load_desired, normalize_policy_target, parse_cluster_config, resolve_config_path, resolve_query_decls, validate_id, validate_query_source}; use diff::{FailedGraphOrigin, ResourceKind, append_policy_binding_changes, approved_resources, classify_changes, compute_approvals, compute_blast_radius, demote_dependents_of_failed_graphs, diff_resources, resource_kind}; use sweep::{mark_approvals_consumed, record_approval_consumed, sweep_recovery_sidecars, tombstone_graph_subtree, warn_pending_recovery_sidecars}; @@ -43,6 +42,18 @@ pub const CLUSTER_RESOURCES_DIR: &str = "__cluster/resources"; pub const CLUSTER_RECOVERIES_DIR: &str = "__cluster/recoveries"; pub const CLUSTER_APPROVALS_DIR: &str = "__cluster/approvals"; +/// The store for a load outcome: the declared `storage:` root when present, +/// the config directory itself otherwise. A bad root is a loud error. +fn store_for( + config_dir: &Path, + storage_root: Option<&str>, +) -> Result<ClusterStore, Diagnostic> { + match storage_root { + Some(root) => ClusterStore::for_storage_root(root), + None => Ok(ClusterStore::for_config_dir(config_dir)), + } +} + pub fn validate_config_dir(config_dir: impl AsRef<Path>) -> ValidateOutput { let outcome = load_desired(config_dir.as_ref()); let (resource_digests, resources, dependencies) = match outcome.desired { @@ -69,7 +80,17 @@ pub fn validate_config_dir(config_dir: impl AsRef<Path>) -> ValidateOutput { pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { let outcome = load_desired(config_dir.as_ref()); let mut diagnostics = outcome.diagnostics; - let backend = ClusterStore::for_config_dir(&outcome.config_dir); + let storage_root = outcome + .desired + .as_ref() + .and_then(|desired| desired.storage_root.clone()); + let backend = match store_for(&outcome.config_dir, storage_root.as_deref()) { + Ok(backend) => backend, + Err(diagnostic) => { + diagnostics.push(diagnostic); + ClusterStore::for_config_dir(&outcome.config_dir) + } + }; let mut observations = backend.observations(); let Some(desired) = outcome.desired else { @@ -169,12 +190,7 @@ pub async fn plan_config_dir(config_dir: impl AsRef<Path>) -> PlanOutput { let ResourceKind::Schema(graph_id) = resource_kind(&change.resource) else { continue; }; - let graph_uri = display_path( - &desired - .config_dir - .join(CLUSTER_GRAPHS_DIR) - .join(format!("{graph_id}.omni")), - ); + let graph_uri = backend.graph_root(&graph_id); let source_path = desired .resources .iter() @@ -242,7 +258,17 @@ pub async fn apply_config_dir_with_options( ) -> ApplyOutput { let outcome = load_desired(config_dir.as_ref()); let mut diagnostics = outcome.diagnostics; - let backend = ClusterStore::for_config_dir(&outcome.config_dir); + let storage_root = outcome + .desired + .as_ref() + .and_then(|desired| desired.storage_root.clone()); + let backend = match store_for(&outcome.config_dir, storage_root.as_deref()) { + Ok(backend) => backend, + Err(diagnostic) => { + diagnostics.push(diagnostic); + ClusterStore::for_config_dir(&outcome.config_dir) + } + }; let mut observations = backend.observations(); let actor_for_output = options.actor.clone(); @@ -442,12 +468,7 @@ pub async fn apply_config_dir_with_options( else { continue; }; - let graph_uri = display_path( - &desired - .config_dir - .join(CLUSTER_GRAPHS_DIR) - .join(format!("{graph_id}.omni")), - ); + let graph_uri = backend.graph_root(graph_id); let mut sidecar = RecoverySidecar { schema_version: 1, operation_id: Ulid::new().to_string(), @@ -587,12 +608,7 @@ pub async fn apply_config_dir_with_options( else { continue; }; - let graph_uri = display_path( - &desired - .config_dir - .join(CLUSTER_GRAPHS_DIR) - .join(format!("{graph_id}.omni")), - ); + let graph_uri = backend.graph_root(graph_id); // Read-write open: the engine's own recovery sweep runs here, which // is exactly what we want before moving its manifest. let db = match Omnigraph::open(&graph_uri).await { @@ -767,9 +783,9 @@ pub async fn apply_config_dir_with_options( .after_digest .as_deref() .expect("create/update always carries an after digest"); - let Some(target) = payload_path(&desired.config_dir, &kind, digest) else { + if ClusterStore::payload_relative(&kind, digest).is_none() { continue; - }; + } let Some(source) = source_paths.get(change.resource.as_str()) else { diagnostics.push(Diagnostic::error( "resource_payload_write_error", @@ -779,7 +795,8 @@ pub async fn apply_config_dir_with_options( continue; }; if let Err(diagnostic) = - write_resource_payload(&target, Path::new(source), digest, &change.resource) + write_resource_payload(&backend, &kind, Path::new(source), digest, &change.resource) + .await { diagnostics.push(diagnostic); } @@ -844,12 +861,7 @@ pub async fn apply_config_dir_with_options( && artifact.bound_config_digest == desired.config_digest }) .map(|artifact| artifact.approval_id.clone()); - let graph_uri = display_path( - &desired - .config_dir - .join(CLUSTER_GRAPHS_DIR) - .join(format!("{graph_id}.omni")), - ); + let graph_uri = backend.graph_root(graph_id); let observed_manifest_version = match Omnigraph::open_read_only(&graph_uri).await { Ok(db) => match db.snapshot_of(ReadTarget::branch("main")).await { Ok(snapshot) => Some(snapshot.version()), @@ -888,9 +900,10 @@ pub async fn apply_config_dir_with_options( graph_moving_aborted = true; continue; } - match fs::remove_dir_all(PathBuf::from(&graph_uri)) { + // Prefix delete through the storage layer: remove_dir_all locally, + // list+delete on object stores (idempotent; already-gone is fine). + match backend.delete_graph_root(&graph_uri).await { Ok(()) => {} - Err(err) if err.kind() == ErrorKind::NotFound => {} // already gone Err(err) => { diagnostics.push(Diagnostic::error( "graph_delete_failed", @@ -1088,7 +1101,17 @@ pub async fn approve_config_dir( ) -> ApproveOutput { let outcome = load_desired(config_dir.as_ref()); let mut diagnostics = outcome.diagnostics; - let backend = ClusterStore::for_config_dir(&outcome.config_dir); + let storage_root = outcome + .desired + .as_ref() + .and_then(|desired| desired.storage_root.clone()); + let backend = match store_for(&outcome.config_dir, storage_root.as_deref()) { + Ok(backend) => backend, + Err(diagnostic) => { + diagnostics.push(diagnostic); + ClusterStore::for_config_dir(&outcome.config_dir) + } + }; let mut observations = backend.observations(); let fail = |config_dir: String, diagnostics: Vec<Diagnostic>| ApproveOutput { @@ -1200,7 +1223,20 @@ pub async fn approve_config_dir( pub async fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { let parsed = parse_cluster_config(config_dir.as_ref()); let mut diagnostics = parsed.diagnostics; - let backend = ClusterStore::for_config_dir(&parsed.config_dir); + let storage_root = parsed.raw.as_ref().and_then(|raw| { + raw.storage + .as_deref() + .map(str::trim) + .filter(|root| !root.is_empty()) + .map(|root| root.trim_end_matches('/').to_string()) + }); + let backend = match store_for(&parsed.config_dir, storage_root.as_deref()) { + Ok(backend) => backend, + Err(diagnostic) => { + diagnostics.push(diagnostic); + ClusterStore::for_config_dir(&parsed.config_dir) + } + }; let mut observations = backend.observations(); backend.observe_lock(&mut observations, &mut diagnostics).await; warn_pending_recovery_sidecars(&parsed.config_dir, &mut diagnostics); @@ -1219,7 +1255,7 @@ pub async fn status_config_dir(config_dir: impl AsRef<Path>) -> StatusOutput { // findings as diagnostics; persisting Drifted statuses // is refresh's job. Status never writes state. for (address, finding) in - verify_catalog_payloads(&parsed.config_dir, &state) + verify_catalog_payloads(&backend, &state).await { diagnostics.push(payload_finding_diagnostic(&address, &finding)); } @@ -1256,7 +1292,20 @@ pub async fn force_unlock_config_dir( ) -> ForceUnlockOutput { let parsed = parse_cluster_config(config_dir.as_ref()); let mut diagnostics = parsed.diagnostics; - let backend = ClusterStore::for_config_dir(&parsed.config_dir); + let storage_root = parsed.raw.as_ref().and_then(|raw| { + raw.storage + .as_deref() + .map(str::trim) + .filter(|root| !root.is_empty()) + .map(|root| root.trim_end_matches('/').to_string()) + }); + let backend = match store_for(&parsed.config_dir, storage_root.as_deref()) { + Ok(backend) => backend, + Err(diagnostic) => { + diagnostics.push(diagnostic); + ClusterStore::for_config_dir(&parsed.config_dir) + } + }; let mut observations = backend.observations(); let mut lock_removed = false; @@ -1290,7 +1339,17 @@ pub async fn import_config_dir(config_dir: impl AsRef<Path>) -> StateSyncOutput async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> StateSyncOutput { let outcome = load_desired(config_dir); let mut diagnostics = outcome.diagnostics; - let backend = ClusterStore::for_config_dir(&outcome.config_dir); + let storage_root = outcome + .desired + .as_ref() + .and_then(|desired| desired.storage_root.clone()); + let backend = match store_for(&outcome.config_dir, storage_root.as_deref()) { + Ok(backend) => backend, + Err(diagnostic) => { + diagnostics.push(diagnostic); + ClusterStore::for_config_dir(&outcome.config_dir) + } + }; let mut observations = backend.observations(); let Some(desired) = outcome.desired else { @@ -1418,7 +1477,7 @@ async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> St // a drifted query digest first means the live-graph composite recompute // below already excludes it, so the persisted graph.<id> composite stays // consistent and the next plan shows exactly the create + derived update. - for (address, finding) in verify_catalog_payloads(&desired.config_dir, &state) { + for (address, finding) in verify_catalog_payloads(&backend, &state).await { diagnostics.push(payload_finding_diagnostic(&address, &finding)); match finding { PayloadFinding::Missing => { @@ -1455,7 +1514,7 @@ async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> St } } - let graph_error_count = observe_declared_graphs(&desired, &mut state).await; + let graph_error_count = observe_declared_graphs(&desired, &backend, &mut state).await; if graph_error_count > 0 { diagnostics.push(Diagnostic::error( "graph_observation_error", @@ -1512,28 +1571,6 @@ async fn sync_config_dir(config_dir: &Path, operation: StateSyncOperation) -> St -/// Content-addressed catalog path for an applied resource payload. Extensions -/// are fixed per kind (`.gq` / `.yaml`) regardless of the source file's name, -/// so the catalog layout cannot drift with operator file conventions. -fn payload_path(config_dir: &Path, kind: &ResourceKind, digest: &str) -> Option<PathBuf> { - let resources_dir = config_dir.join(CLUSTER_RESOURCES_DIR); - match kind { - ResourceKind::Query { graph, name } => Some( - resources_dir - .join("query") - .join(graph) - .join(name) - .join(format!("{digest}.gq")), - ), - ResourceKind::Policy(name) => Some( - resources_dir - .join("policy") - .join(name) - .join(format!("{digest}.yaml")), - ), - _ => None, - } -} #[derive(Debug, PartialEq, Eq)] enum PayloadFinding { @@ -1547,34 +1584,26 @@ enum PayloadFinding { /// unknown addresses have no payloads and are skipped. Read-only; findings /// are deterministic (BTreeMap order). Payloads are small (queries, policy /// bundles), so a full digest re-hash is cheap. -fn verify_catalog_payloads( - config_dir: &Path, +async fn verify_catalog_payloads( + backend: &ClusterStore, state: &ClusterState, ) -> Vec<(String, PayloadFinding)> { let mut findings = Vec::new(); for (address, resource) in &state.applied_revision.resources { let kind = resource_kind(address); - let Some(path) = payload_path(config_dir, &kind, &resource.digest) else { + if ClusterStore::payload_relative(&kind, &resource.digest).is_none() { continue; - }; - match fs::read(&path) { - Ok(bytes) => { - let actual_digest = sha256_hex(&bytes); + } + match backend.read_payload(&kind, &resource.digest).await { + Ok(Some(text)) => { + let actual_digest = sha256_hex(text.as_bytes()); if actual_digest != resource.digest { findings.push((address.clone(), PayloadFinding::Mismatch { actual_digest })); } } - Err(err) if err.kind() == ErrorKind::NotFound => { - findings.push((address.clone(), PayloadFinding::Missing)); - } + Ok(None) => findings.push((address.clone(), PayloadFinding::Missing)), Err(err) => { - findings.push(( - address.clone(), - PayloadFinding::ReadError(format!( - "could not read catalog payload '{}': {err}", - path.display() - )), - )); + findings.push((address.clone(), PayloadFinding::ReadError(err))); } } } @@ -1606,13 +1635,15 @@ fn payload_finding_diagnostic(address: &str, finding: &PayloadFinding) -> Diagno /// digest-named file is trusted as-is. The digest re-check is the apply-side /// TOCTOU detector — the source file changing between `load_desired` and the /// payload write must fail loudly, never publish mismatched content. -fn write_resource_payload( - target: &Path, +async fn write_resource_payload( + backend: &ClusterStore, + kind: &ResourceKind, source: &Path, expected_digest: &str, resource: &str, ) -> Result<(), Diagnostic> { - if target.exists() { + if backend.payload_exists(kind, expected_digest).await { + // Content-addressed: an existing digest-named object is identical. return Ok(()); } let bytes = fs::read(source).map_err(|err| { @@ -1623,6 +1654,9 @@ fn write_resource_payload( ) })?; if sha256_hex(&bytes) != expected_digest { + // The apply-side TOCTOU detector: the source changing between + // load_desired and this write must fail loudly, never publish + // mismatched content. return Err(Diagnostic::error( "resource_content_changed", resource, @@ -1632,54 +1666,23 @@ fn write_resource_payload( ), )); } - let parent = target.parent().expect("payload path always has a parent"); - fs::create_dir_all(parent).map_err(|err| { + let content = String::from_utf8(bytes).map_err(|err| { Diagnostic::error( "resource_payload_write_error", resource, - format!("could not create payload directory: {err}"), + format!("resource source is not valid UTF-8: {err}"), ) })?; - let file_name = target - .file_name() - .expect("payload path always has a file name") - .to_string_lossy(); - let tmp_path = parent.join(format!("{file_name}.tmp.{}", Ulid::new())); - let mut file = OpenOptions::new() - .write(true) - .create_new(true) - .open(&tmp_path) + backend + .write_payload(kind, expected_digest, &content) + .await .map_err(|err| { Diagnostic::error( "resource_payload_write_error", resource, - format!("could not create temporary payload file: {err}"), + format!("could not write payload: {err}"), ) - })?; - let write_result = file - .write_all(&bytes) - .and_then(|()| file.sync_all()) - .map_err(|err| { - Diagnostic::error( - "resource_payload_write_error", - resource, - format!("could not write payload file: {err}"), - ) - }); - drop(file); - if let Err(diagnostic) = write_result { - let _ = fs::remove_file(&tmp_path); - return Err(diagnostic); - } - if let Err(err) = fs::rename(&tmp_path, target) { - let _ = fs::remove_file(&tmp_path); - return Err(Diagnostic::error( - "resource_payload_write_error", - resource, - format!("could not move payload file into place: {err}"), - )); - } - Ok(()) + }) } /// Recompute the composite `graph.<id>` digests for state-resident graphs from diff --git a/crates/omnigraph-cluster/src/serve.rs b/crates/omnigraph-cluster/src/serve.rs index 8578aee..b459641 100644 --- a/crates/omnigraph-cluster/src/serve.rs +++ b/crates/omnigraph-cluster/src/serve.rs @@ -44,8 +44,24 @@ pub async fn read_serving_snapshot( config_dir: impl AsRef<Path>, ) -> Result<ServingSnapshot, Vec<Diagnostic>> { let config_dir = config_dir.as_ref().to_path_buf(); - let backend = ClusterStore::for_config_dir(&config_dir); let mut diagnostics: Vec<Diagnostic> = Vec::new(); + // The declared storage: root decides where the ledger/catalog/graphs + // live; config parse errors surface through the normal validation path. + let parsed = parse_cluster_config(&config_dir); + let storage_root = parsed.raw.as_ref().and_then(|raw| { + raw.storage + .as_deref() + .map(str::trim) + .filter(|root| !root.is_empty()) + .map(|root| root.trim_end_matches('/').to_string()) + }); + let backend = match storage_root.as_deref() { + Some(root) => match ClusterStore::for_storage_root(root) { + Ok(backend) => backend, + Err(diagnostic) => return Err(vec![diagnostic]), + }, + None => ClusterStore::for_config_dir(&config_dir), + }; // A ledger a sweep is about to rewrite must not start serving. let sidecars = backend.list_recovery_sidecars(&mut diagnostics).await; @@ -89,9 +105,7 @@ pub async fn read_serving_snapshot( match resource_kind(address) { ResourceKind::Graph(graph_id) => { graphs.push(ServingGraph { - root: config_dir - .join(CLUSTER_GRAPHS_DIR) - .join(format!("{graph_id}.omni")), + root: PathBuf::from(backend.graph_root(&graph_id)), graph_id, }); } @@ -100,7 +114,7 @@ pub async fn read_serving_snapshot( let ResourceKind::Query { graph, name } = &kind else { unreachable!() }; - match read_verified_payload(&config_dir, &kind, &entry.digest, address) { + match backend.read_verified_payload(&kind, &entry.digest, address).await { Ok(source) => queries.push(ServingQuery { graph_id: graph.clone(), name: name.clone(), @@ -121,11 +135,14 @@ pub async fn read_serving_snapshot( )); continue; }; - match read_verified_payload(&config_dir, &kind, &entry.digest, address) { + match backend.read_verified_payload(&kind, &entry.digest, address).await { Ok(_) => policies.push(ServingPolicy { name: name.clone(), - blob_path: payload_path(&config_dir, &kind, &entry.digest) - .expect("policy kind always has a payload path"), + blob_path: PathBuf::from( + backend + .payload_display(&kind, &entry.digest) + .expect("policy kind always has a payload path"), + ), applies_to, }), Err(diagnostic) => diagnostics.push(diagnostic), @@ -152,40 +169,3 @@ pub async fn read_serving_snapshot( }) } -/// Read a catalog blob and verify it against the recorded digest. -pub(crate) fn read_verified_payload( - config_dir: &Path, - kind: &ResourceKind, - digest: &str, - address: &str, -) -> Result<String, Diagnostic> { - let path = payload_path(config_dir, kind, digest) - .expect("query/policy kinds always have a payload path"); - let bytes = fs::read(&path).map_err(|err| { - Diagnostic::error( - "catalog_payload_missing", - address, - format!( - "catalog blob '{}' unreadable ({err}); run `cluster refresh` then `cluster apply`, and restart", - display_path(&path) - ), - ) - })?; - if sha256_hex(&bytes) != digest { - return Err(Diagnostic::error( - "catalog_payload_digest_mismatch", - address, - format!( - "catalog blob '{}' does not match its recorded digest; run `cluster refresh` then `cluster apply`, and restart", - display_path(&path) - ), - )); - } - String::from_utf8(bytes).map_err(|err| { - Diagnostic::error( - "catalog_payload_invalid", - address, - format!("catalog blob is not valid UTF-8: {err}"), - ) - }) -} diff --git a/crates/omnigraph-cluster/src/store.rs b/crates/omnigraph-cluster/src/store.rs index bf328af..f52dd29 100644 --- a/crates/omnigraph-cluster/src/store.rs +++ b/crates/omnigraph-cluster/src/store.rs @@ -375,6 +375,34 @@ impl ClusterStore { .unwrap_or(false) } + /// Raw payload read: `Ok(None)` for a missing blob, `Err` for transport + /// failures — callers classify (verify loops need the three-way split). + pub(crate) async fn read_payload( + &self, + kind: &ResourceKind, + digest: &str, + ) -> Result<Option<String>, String> { + let Some(relative) = Self::payload_relative(kind, digest) else { + return Ok(None); + }; + let uri = self.uri(&relative); + match self.adapter.exists(&uri).await { + Ok(false) => return Ok(None), + Ok(true) => {} + Err(err) => return Err(err.to_string()), + } + self.adapter + .read_text(&uri) + .await + .map(Some) + .map_err(|err| { + format!( + "could not read catalog payload '{}': {err}", + self.display(&relative) + ) + }) + } + /// Idempotent content-addressed write: a payload already present at its /// digest is by definition identical. pub(crate) async fn write_payload( diff --git a/crates/omnigraph-cluster/src/sweep.rs b/crates/omnigraph-cluster/src/sweep.rs index 2cfd7d1..7aecb01 100644 --- a/crates/omnigraph-cluster/src/sweep.rs +++ b/crates/omnigraph-cluster/src/sweep.rs @@ -19,13 +19,13 @@ pub(crate) async fn sweep_recovery_sidecars( for (path, sidecar) in backend.list_recovery_sidecars(diagnostics).await { match sidecar.kind { RecoverySidecarKind::GraphCreate => { - sweep_graph_create_sidecar(path, sidecar, state, diagnostics, &mut outcome).await; + sweep_graph_create_sidecar(backend, path, sidecar, state, diagnostics, &mut outcome).await; } RecoverySidecarKind::SchemaApply => { sweep_schema_apply_sidecar(path, sidecar, state, diagnostics, &mut outcome).await; } RecoverySidecarKind::GraphDelete => { - sweep_graph_delete_sidecar(path, sidecar, state, diagnostics, &mut outcome); + sweep_graph_delete_sidecar(backend, path, sidecar, state, diagnostics, &mut outcome).await; } } } @@ -33,6 +33,7 @@ pub(crate) async fn sweep_recovery_sidecars( } pub(crate) async fn sweep_graph_create_sidecar( + backend: &ClusterStore, path: String, sidecar: RecoverySidecar, state: &mut ClusterState, @@ -41,13 +42,12 @@ pub(crate) async fn sweep_graph_create_sidecar( ) { let graph_address = graph_address(&sidecar.graph_id); let schema_addr = schema_address(&sidecar.graph_id); - let graph_path = PathBuf::from(&sidecar.graph_uri); // Row 1: nothing moved — the init never landed. The sidecar is pure // intent; retire it (deferred to the command's post-CAS cleanup, like // every other completed sidecar — a failed CAS simply re-sweeps it) and // let the command's own plan re-propose the create. - if !graph_path.exists() { + if !backend.graph_root_exists(&sidecar.graph_uri).await { outcome.completed_sidecars.push(path); return; } @@ -251,7 +251,8 @@ pub(crate) async fn sweep_schema_apply_sidecar( } } -pub(crate) fn sweep_graph_delete_sidecar( +pub(crate) async fn sweep_graph_delete_sidecar( + backend: &ClusterStore, path: String, sidecar: RecoverySidecar, state: &mut ClusterState, @@ -259,9 +260,8 @@ pub(crate) fn sweep_graph_delete_sidecar( outcome: &mut SweepOutcome, ) { let graph_address = graph_address(&sidecar.graph_id); - let root = PathBuf::from(&sidecar.graph_uri); - if root.exists() { + if backend.graph_root_exists(&sidecar.graph_uri).await { // Row 8: the delete never completed. Prefix removal is idempotent and // works on partial roots, so the repair is simply the re-proposed, // still-approved delete on a later run — retire the stale intent. diff --git a/crates/omnigraph-cluster/src/tests.rs b/crates/omnigraph-cluster/src/tests.rs index 3b7984d..ba7019f 100644 --- a/crates/omnigraph-cluster/src/tests.rs +++ b/crates/omnigraph-cluster/src/tests.rs @@ -2762,6 +2762,70 @@ policies: // ---- serving snapshot (5B read-only loader) ---- + // ---- storage: root (RFC-006) ---- + + #[tokio::test] + async fn storage_root_defaults_to_config_dir_layout() { + let dir = fixture(); + init_derived_graph(dir.path()).await; + write_applyable_state(dir.path()); + let out = apply_config_dir(dir.path()).await; + assert!(out.converged, "{out:?}"); + // No storage: key — the original on-disk layout, byte-compatible. + assert!(dir.path().join(CLUSTER_STATE_FILE).exists()); + assert!(dir.path().join(CLUSTER_RESOURCES_DIR).exists()); + assert!(dir.path().join("graphs/knowledge.omni").exists()); + } + + #[tokio::test] + async fn storage_root_file_uri_relocates_the_cluster() { + let dir = fixture(); + let storage = tempfile::tempdir().unwrap(); + let storage_path = storage.path().to_string_lossy().to_string(); + let mut config = fs::read_to_string(dir.path().join("cluster.yaml")).unwrap(); + config = config.replace("version: 1\n", &format!("version: 1\nstorage: {storage_path}\n")); + fs::write(dir.path().join("cluster.yaml"), config).unwrap(); + + let import = import_config_dir(dir.path()).await; + assert!(import.ok, "{:?}", import.diagnostics); + let out = apply_config_dir(dir.path()).await; + assert!(out.ok && out.converged, "{:?}", out.diagnostics); + + // Everything lives under the declared root; nothing under config dir. + assert!(storage.path().join("__cluster/state.json").exists()); + assert!(storage.path().join("graphs/knowledge.omni").exists()); + assert!(storage.path().join(CLUSTER_RESOURCES_DIR).exists()); + assert!(!dir.path().join(CLUSTER_STATE_FILE).exists()); + assert!(!dir.path().join("graphs").exists()); + + // The serving snapshot follows the root. + let snapshot = read_serving_snapshot(dir.path()).await.unwrap(); + assert!( + snapshot.graphs[0] + .root + .starts_with(storage.path()), + "{:?}", + snapshot.graphs[0].root + ); + } + + #[test] + fn storage_root_invalid_uri_fails_validation() { + let dir = fixture(); + let mut config = fs::read_to_string(dir.path().join("cluster.yaml")).unwrap(); + config = config.replace("version: 1\n", "version: 1\nstorage: \"s3://\"\n"); + fs::write(dir.path().join("cluster.yaml"), config).unwrap(); + let out = validate_config_dir(dir.path()); + assert!(!out.ok); + assert!( + out.diagnostics + .iter() + .any(|diagnostic| diagnostic.code == "invalid_storage_root"), + "{:?}", + out.diagnostics + ); + } + #[tokio::test] async fn serving_snapshot_reads_converged_cluster() { let dir = fixture(); diff --git a/crates/omnigraph-cluster/src/types.rs b/crates/omnigraph-cluster/src/types.rs index ca960a5..e44e2f4 100644 --- a/crates/omnigraph-cluster/src/types.rs +++ b/crates/omnigraph-cluster/src/types.rs @@ -322,6 +322,8 @@ pub struct ApproveOutput { pub(crate) struct DesiredCluster { pub(crate) config_dir: PathBuf, pub(crate) config_digest: String, + /// The declared `storage:` root, if any (None ⇒ the config dir itself). + pub(crate) storage_root: Option<String>, pub(crate) state_lock: bool, pub(crate) graphs: Vec<DesiredGraph>, pub(crate) resource_digests: BTreeMap<String, String>, @@ -345,9 +347,10 @@ pub(crate) struct ParsedConfig { pub(crate) config_file: PathBuf, } -#[derive(Debug, Clone, Copy)] +#[derive(Debug, Clone)] pub(crate) struct ClusterSettings { pub(crate) state_lock: bool, + pub(crate) storage_root: Option<String>, } #[derive(Debug)] @@ -364,6 +367,12 @@ pub(crate) struct RawClusterConfig { pub(crate) version: u32, #[serde(default)] pub(crate) metadata: Metadata, + /// Storage root URI for everything the cluster stores: the state + /// ledger, catalog, sidecars, approvals, and derived graph roots. + /// Absent ⇒ `file://<config-dir>` (the original layout, byte-compatible). + /// `s3://bucket/prefix` puts the whole cluster on object storage. + #[serde(default)] + pub(crate) storage: Option<String>, #[serde(default)] pub(crate) state: StateConfig, #[serde(default)] diff --git a/docs/dev/invariants.md b/docs/dev/invariants.md index 7642fd9..655e360 100644 --- a/docs/dev/invariants.md +++ b/docs/dev/invariants.md @@ -206,6 +206,10 @@ case is exceptional. fits. - Discarding retrieval score/rank before fusion or projection decisions. - Auto-creating placeholder nodes for orphan edges. +- Raw filesystem I/O for cluster-stored state (ledger, lock, sidecars, + approvals, catalog) outside the cluster crate's storage module — every + stored byte goes through the engine `StorageAdapter` so `file://` and + `s3://` stay one code path. - Wire-protocol-specific code in compiler or engine crates. - Cloud-only correctness fixes or forks of the OSS engine for correctness. - Mutating immutable substrate state in place, including Lance fragments or diff --git a/docs/user/cluster-config.md b/docs/user/cluster-config.md index 24d1833..59c9207 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/cluster-config.md @@ -101,6 +101,20 @@ updates all of its queries together. Paths are relative to the config directory — the cluster is one explicit folder, so no `./` prefixes are needed. +`storage:` (optional) is the **storage root URI** for everything the cluster +stores — the state ledger, lock, content-addressed catalog, recovery +sidecars, approval artifacts, and the derived graph roots +(`<storage>/graphs/<id>.omni`). Absent, it defaults to the config directory +itself (the original layout, byte-compatible with pre-existing clusters). +`s3://bucket/prefix` puts the whole cluster on S3-compatible object storage: +the ledger CAS uses conditional writes (verified against AWS S3 semantics and +RustFS), the lock becomes genuinely cross-machine, and graph roots are +engine-native S3 URIs. Credentials are **never** in `cluster.yaml` — the +standard `AWS_*` environment contract applies, identical to graph storage. +Declared configuration (`cluster.yaml` and the schema/query/policy sources it +references) always stays in the working tree: config is versioned in git, +state lives in the store — the Terraform split. + `metadata.name` is a display label. `state.backend` may be omitted or set to `cluster`; external state backends are reserved for a later stage. `state.lock` defaults to `true`. When enabled, `cluster plan`, `cluster apply`, diff --git a/docs/user/cluster.md b/docs/user/cluster.md index 1731f31..19755fb 100644 --- a/docs/user/cluster.md +++ b/docs/user/cluster.md @@ -40,6 +40,8 @@ company-brain/ ```yaml # cluster.yaml version: 1 +# storage: s3://omnigraph-local/clusters/company-brain # optional: put the +# ledger, catalog, and graph data on object storage (default: this folder) metadata: name: company-brain graphs: From f6ae3e4fa30656d745fe711e2cb56943b497a4ae Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 14:33:26 +0300 Subject: [PATCH 111/165] fix(cluster): lock release must complete before a CLI process exits MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Caught by the first live s3 smoke: StateLockGuard's spawned async delete dies with the runtime when a short-lived CLI process exits right after the command — import's lock survived into the next command as state_lock_held. On the multi-thread runtime (the CLI, and the gated s3 tests) block_in_place waits for the delete to complete; current-thread runtimes keep the spawn fallback with force-unlock as the documented recovery, same as a crash. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/store.rs | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/crates/omnigraph-cluster/src/store.rs b/crates/omnigraph-cluster/src/store.rs index f52dd29..4d33d2c 100644 --- a/crates/omnigraph-cluster/src/store.rs +++ b/crates/omnigraph-cluster/src/store.rs @@ -58,16 +58,30 @@ impl Drop for StateLockGuard { let path = self.uri.trim_start_matches("file://"); let _ = std::fs::remove_file(path); } - // Object stores need an async delete; best-effort spawn. A crash - // here leaves the lock for `force-unlock` — same as a process - // kill, and the same recovery path. + // Object stores need an async delete, and it must COMPLETE + // before a short-lived CLI process exits — a spawned task dies + // with the runtime and leaks the lock (caught by the s3 smoke + // test: import's lock survived into the next command). On the + // multi-thread runtime (the CLI and the gated s3 tests), + // block_in_place waits for the delete; on a current-thread + // runtime that's not allowed, so fall back to a spawn — + // best-effort, with `force-unlock` as the documented recovery, + // same as a crash. StorageKind::S3 => { let adapter = Arc::clone(&self.adapter); let uri = self.uri.clone(); if let Ok(handle) = tokio::runtime::Handle::try_current() { - handle.spawn(async move { - let _ = adapter.delete(&uri).await; - }); + if handle.runtime_flavor() == tokio::runtime::RuntimeFlavor::MultiThread { + tokio::task::block_in_place(move || { + handle.block_on(async move { + let _ = adapter.delete(&uri).await; + }); + }); + } else { + handle.spawn(async move { + let _ = adapter.delete(&uri).await; + }); + } } } } From b036073ec603403512762d565897fc8788561153 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 15:03:51 +0300 Subject: [PATCH 112/165] refactor(server): split the test monolith into area suites MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit tests/server.rs (6,517 lines, 110 tests) becomes seven area files — auth_policy, data_routes, schema_routes, stored_queries, multi_graph, boot_settings, s3 — with shared helpers in tests/support/mod.rs. Verbatim moves + visibility bumps (pub on helpers, pub(super)->pub inside the matrix harness); cargo fix stripped the per-file unused imports. All 110 tests pass in their new homes (289 across the crate including lib and openapi). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-server/tests/auth_policy.rs | 915 +++ .../omnigraph-server/tests/boot_settings.rs | 1022 +++ crates/omnigraph-server/tests/data_routes.rs | 1572 ++++ crates/omnigraph-server/tests/multi_graph.rs | 584 ++ crates/omnigraph-server/tests/s3.rs | 77 + .../omnigraph-server/tests/schema_routes.rs | 830 +++ crates/omnigraph-server/tests/server.rs | 6520 ----------------- .../omnigraph-server/tests/stored_queries.rs | 329 + crates/omnigraph-server/tests/support/mod.rs | 1195 +++ 9 files changed, 6524 insertions(+), 6520 deletions(-) create mode 100644 crates/omnigraph-server/tests/auth_policy.rs create mode 100644 crates/omnigraph-server/tests/boot_settings.rs create mode 100644 crates/omnigraph-server/tests/data_routes.rs create mode 100644 crates/omnigraph-server/tests/multi_graph.rs create mode 100644 crates/omnigraph-server/tests/s3.rs create mode 100644 crates/omnigraph-server/tests/schema_routes.rs delete mode 100644 crates/omnigraph-server/tests/server.rs create mode 100644 crates/omnigraph-server/tests/stored_queries.rs create mode 100644 crates/omnigraph-server/tests/support/mod.rs diff --git a/crates/omnigraph-server/tests/auth_policy.rs b/crates/omnigraph-server/tests/auth_policy.rs new file mode 100644 index 0000000..05c0c56 --- /dev/null +++ b/crates/omnigraph-server/tests/auth_policy.rs @@ -0,0 +1,915 @@ +//! Bearer auth, actor resolution, Cedar policy decisions, admission. +//! Moved verbatim from tests/server.rs in the modularization. + +use std::env; +use std::fs; +use std::sync::Arc; + +use axum::body::Body; +use axum::http::header::AUTHORIZATION; +use axum::http::{Method, Request, StatusCode}; +use omnigraph::db::{Omnigraph, ReadTarget}; +use omnigraph::error::OmniError; +use omnigraph::loader::LoadMode; +use omnigraph_server::api::{ + BranchCreateRequest, BranchMergeRequest, ChangeRequest, ErrorOutput, ExportRequest, ReadRequest, SchemaApplyRequest, +}; +use omnigraph_server::{AppState, build_app}; +use serde_json::{Value, json}; +use tower::ServiceExt; + + +mod support; +use support::*; + +#[tokio::test(flavor = "multi_thread")] +async fn healthz_succeeds_after_startup() { + let (_temp, app) = app_for_loaded_graph().await; + let (status, body) = json_response( + &app, + Request::builder() + .uri("/healthz") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + + assert_eq!(status, StatusCode::OK); + assert_eq!(body["status"], "ok"); + assert_eq!(body["version"], env!("CARGO_PKG_VERSION")); + match option_env!("OMNIGRAPH_SOURCE_VERSION") { + Some(source_version) => assert_eq!(body["source_version"], source_version), + None => assert!(body.get("source_version").is_none()), + } +} + +#[tokio::test(flavor = "multi_thread")] +async fn protected_routes_require_bearer_token() { + let (_temp, app) = app_for_loaded_graph_with_auth("demo-token").await; + let (status, body) = json_response( + &app, + Request::builder() + .uri("/branches") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + + let error: ErrorOutput = serde_json::from_value(body).unwrap(); + assert_eq!(status, StatusCode::UNAUTHORIZED); + assert_eq!( + error.code, + Some(omnigraph_server::api::ErrorCode::Unauthorized) + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn protected_routes_accept_valid_bearer_token_while_healthz_stays_open() { + let (_temp, app) = app_for_loaded_graph_with_auth("demo-token").await; + + let health = app + .clone() + .oneshot( + Request::builder() + .uri("/healthz") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(health.status(), StatusCode::OK); + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/branches") + .method(Method::GET) + .header("authorization", "Bearer demo-token") + .body(Body::empty()) + .unwrap(), + ) + .await; + + assert_eq!(status, StatusCode::OK); + assert!(body["branches"].is_array()); +} + +#[tokio::test(flavor = "multi_thread")] +async fn protected_routes_accept_any_configured_team_bearer_token() { + let (_temp, app) = app_for_loaded_graph_with_auth_tokens(&[ + ("team-01", "token-one"), + ("team-02", "token-two"), + ]) + .await; + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/branches") + .method(Method::GET) + .header("authorization", "Bearer token-two") + .body(Body::empty()) + .unwrap(), + ) + .await; + + assert_eq!(status, StatusCode::OK); + assert!(body["branches"].is_array()); +} + +#[tokio::test(flavor = "multi_thread")] +async fn bearer_token_resolves_to_correct_actor_for_policy_decisions() { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let policy_path = temp.path().join("policy.yaml"); + fs::write( + &policy_path, + r#" +version: 1 +groups: + readers: [act-a] + writers: [act-b] +protected_branches: [main] +rules: + - id: readers-only + allow: + actors: { group: readers } + actions: [read] + branch_scope: any +"#, + ) + .unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![ + ("act-a".to_string(), "token-a".to_string()), + ("act-b".to_string(), "token-b".to_string()), + ], + Some(&policy_path), + ) + .await + .unwrap(); + let app = build_app(state); + + // act-a is authenticated AND authorized. + let (ok_status, _) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .header("authorization", "Bearer token-a") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(ok_status, StatusCode::OK); + + // act-b is authenticated but policy rejects — proves the resolved actor + // (not some default) was the policy subject. + let (denied_status, denied_body) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .header("authorization", "Bearer token-b") + .body(Body::empty()) + .unwrap(), + ) + .await; + let denied_error: ErrorOutput = serde_json::from_value(denied_body).unwrap(); + assert_eq!(denied_status, StatusCode::FORBIDDEN); + assert_eq!( + denied_error.code, + Some(omnigraph_server::api::ErrorCode::Forbidden) + ); + + // Unknown token: 401, never reaches the policy engine. + let (bad_status, _) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .header("authorization", "Bearer wrong-token") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(bad_status, StatusCode::UNAUTHORIZED); +} + +#[tokio::test(flavor = "multi_thread")] +async fn actor_id_resolves_from_bearer_token_ignoring_client_supplied_headers() { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let policy_path = temp.path().join("policy.yaml"); + // Same readers/writers split as + // `bearer_token_resolves_to_correct_actor_for_policy_decisions` — + // `act-a` can read main, `act-b` cannot. The asymmetry is what + // makes the spoof-up/spoof-down distinction observable. + fs::write( + &policy_path, + r#" +version: 1 +groups: + readers: [act-a] + writers: [act-b] +protected_branches: [main] +rules: + - id: readers-only + allow: + actors: { group: readers } + actions: [read] + branch_scope: any +"#, + ) + .unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![ + ("act-a".to_string(), "token-a".to_string()), + ("act-b".to_string(), "token-b".to_string()), + ], + Some(&policy_path), + ) + .await + .unwrap(); + let app = build_app(state); + + // (1) Spoof-up: bearer for act-b (denied) + X-Actor-Id: act-a (allowed). + // If the server were trusting the header, this would succeed as + // act-a. The contract is: the bearer wins. Expect 403 because + // act-b can't read. + let (spoof_up_status, spoof_up_body) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .header("authorization", "Bearer token-b") + .header("x-actor-id", "act-a") + .body(Body::empty()) + .unwrap(), + ) + .await; + let spoof_up_error: ErrorOutput = serde_json::from_value(spoof_up_body).unwrap(); + assert_eq!( + spoof_up_status, + StatusCode::FORBIDDEN, + "X-Actor-Id must not promote a denied bearer to an allowed actor", + ); + assert_eq!( + spoof_up_error.code, + Some(omnigraph_server::api::ErrorCode::Forbidden), + ); + + // (2) Spoof-down: bearer for act-a (allowed) + X-Actor-Id: act-b (denied). + // If the server were trusting the header, this would fail as act-b. + // The contract is: the bearer wins. Expect 200 because act-a can read. + let (spoof_down_status, _) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .header("authorization", "Bearer token-a") + .header("x-actor-id", "act-b") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!( + spoof_down_status, + StatusCode::OK, + "X-Actor-Id must not demote an allowed bearer to a denied actor", + ); + + // (3) Empty-string spoof attempt: an X-Actor-Id of "" must not + // leak through as the policy subject. Same expectation as (1): + // bearer for act-b is denied regardless of what the header tries. + let (empty_spoof_status, _) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .header("authorization", "Bearer token-b") + .header("x-actor-id", "") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!( + empty_spoof_status, + StatusCode::FORBIDDEN, + "empty X-Actor-Id must not clear the resolved actor", + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn policy_allows_read_but_distinguishes_401_from_403() { + let (_temp, app) = app_for_loaded_graph_with_auth_tokens_and_policy( + &[("act-bruno", "team-token"), ("act-ragnor", "admin-token")], + POLICY_YAML, + ) + .await; + + let (missing_status, missing_body) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + let missing_error: ErrorOutput = serde_json::from_value(missing_body).unwrap(); + assert_eq!(missing_status, StatusCode::UNAUTHORIZED); + assert_eq!( + missing_error.code, + Some(omnigraph_server::api::ErrorCode::Unauthorized) + ); + + let (snapshot_status, snapshot_body) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .header("authorization", "Bearer team-token") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(snapshot_status, StatusCode::OK); + assert_eq!(snapshot_body["branch"], "main"); + + let export_request = ExportRequest { + branch: Some("main".to_string()), + type_names: Vec::new(), + table_keys: Vec::new(), + }; + let (forbidden_status, forbidden_body) = json_response( + &app, + Request::builder() + .uri("/export") + .method(Method::POST) + .header("authorization", "Bearer team-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&export_request).unwrap())) + .unwrap(), + ) + .await; + let forbidden_error: ErrorOutput = serde_json::from_value(forbidden_body).unwrap(); + assert_eq!(forbidden_status, StatusCode::FORBIDDEN); + assert_eq!( + forbidden_error.code, + Some(omnigraph_server::api::ErrorCode::Forbidden) + ); + + let response = app + .clone() + .oneshot( + Request::builder() + .uri("/export") + .method(Method::POST) + .header("authorization", "Bearer admin-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&export_request).unwrap())) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(response.status(), StatusCode::OK); +} + +#[tokio::test(flavor = "multi_thread")] +async fn policy_uses_resolved_branch_for_snapshot_reads() { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let snapshot_id = { + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.resolve_snapshot("main").await.unwrap().to_string() + }; + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, POLICY_PROTECTED_READ_YAML).unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![("act-bruno".to_string(), "team-token".to_string())], + Some(&policy_path), + ) + .await + .unwrap(); + let app = build_app(state); + + let read = ReadRequest { + query_source: fs::read_to_string(fixture("test.gq")).unwrap(), + query_name: Some("get_person".to_string()), + params: Some(json!({ "name": "Alice" })), + branch: None, + snapshot: Some(snapshot_id), + }; + let (status, body) = json_response( + &app, + Request::builder() + .uri("/read") + .method(Method::POST) + .header("authorization", "Bearer team-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&read).unwrap())) + .unwrap(), + ) + .await; + + assert_eq!(status, StatusCode::OK); + assert_eq!(body["target"]["branch"], Value::Null); + assert_eq!( + body["target"]["snapshot"].as_str(), + read.snapshot.as_deref() + ); + assert_eq!(body["row_count"], 1); +} + +#[tokio::test(flavor = "multi_thread")] +async fn policy_blocks_change_on_protected_main_but_allows_unprotected_branch() { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.branch_create_from(ReadTarget::branch("main"), "feature") + .await + .unwrap(); + drop(db); + + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, POLICY_YAML).unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![("act-bruno".to_string(), "team-token".to_string())], + Some(&policy_path), + ) + .await + .unwrap(); + let app = build_app(state); + + let main_change = ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": "Mina", "age": 28 })), + branch: Some("main".to_string()), + }; + let (main_status, main_body) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header("authorization", "Bearer team-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&main_change).unwrap())) + .unwrap(), + ) + .await; + let main_error: ErrorOutput = serde_json::from_value(main_body).unwrap(); + assert_eq!(main_status, StatusCode::FORBIDDEN); + assert_eq!( + main_error.code, + Some(omnigraph_server::api::ErrorCode::Forbidden) + ); + + let feature_change = ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": "Mina", "age": 28 })), + branch: Some("feature".to_string()), + }; + let (feature_status, feature_body) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header("authorization", "Bearer team-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&feature_change).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(feature_status, StatusCode::OK); + assert_eq!(feature_body["branch"], "feature"); + assert_eq!(feature_body["affected_nodes"], 1); +} + +#[tokio::test(flavor = "multi_thread")] +async fn policy_blocks_non_admin_merge_to_main_and_allows_admin() { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.branch_create_from(ReadTarget::branch("main"), "feature") + .await + .unwrap(); + db.load( + "feature", + r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#, + LoadMode::Append, + ) + .await + .unwrap(); + drop(db); + + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, POLICY_YAML).unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![ + ("act-bruno".to_string(), "team-token".to_string()), + ("act-ragnor".to_string(), "admin-token".to_string()), + ], + Some(&policy_path), + ) + .await + .unwrap(); + let app = build_app(state); + + let merge = BranchMergeRequest { + source: "feature".to_string(), + target: Some("main".to_string()), + }; + let (deny_status, deny_body) = json_response( + &app, + Request::builder() + .uri("/branches/merge") + .method(Method::POST) + .header("authorization", "Bearer team-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&merge).unwrap())) + .unwrap(), + ) + .await; + let deny_error: ErrorOutput = serde_json::from_value(deny_body).unwrap(); + assert_eq!(deny_status, StatusCode::FORBIDDEN); + assert_eq!( + deny_error.code, + Some(omnigraph_server::api::ErrorCode::Forbidden) + ); + + let (allow_status, allow_body) = json_response( + &app, + Request::builder() + .uri("/branches/merge") + .method(Method::POST) + .header("authorization", "Bearer admin-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&merge).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(allow_status, StatusCode::OK); + assert_eq!(allow_body["actor_id"], "act-ragnor"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn authenticated_change_stamps_actor_on_commits() { + // With the Run state machine removed, actor_id is recorded + // directly on the commit graph (no intermediate run record). + let (_temp, app) = app_for_loaded_graph_with_auth_tokens(&[("act-andrew", "token-one")]).await; + + let change = ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": "Mina", "age": 28 })), + branch: Some("main".to_string()), + }; + let (change_status, change_body) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header("authorization", "Bearer token-one") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&change).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(change_status, StatusCode::OK); + assert_eq!(change_body["actor_id"], "act-andrew"); + + let (commits_status, commits_body) = json_response( + &app, + Request::builder() + .uri("/commits?branch=main") + .method(Method::GET) + .header("authorization", "Bearer token-one") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(commits_status, StatusCode::OK); + let head = commits_body["commits"] + .as_array() + .unwrap() + .last() + .expect("head commit should exist"); + assert_eq!(head["actor_id"], "act-andrew"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn authenticated_branch_merge_stamps_merge_actor_on_head_commit() { + let (_temp, app) = app_for_loaded_graph_with_auth_tokens(&[ + ("act-andrew", "token-one"), + ("act-ragnor", "token-two"), + ]) + .await; + + let create = BranchCreateRequest { + from: Some("main".to_string()), + name: "feature".to_string(), + }; + let (create_status, _) = json_response( + &app, + Request::builder() + .uri("/branches") + .method(Method::POST) + .header("authorization", "Bearer token-one") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&create).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(create_status, StatusCode::OK); + + let change = ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": "Zoe", "age": 33 })), + branch: Some("feature".to_string()), + }; + let (change_status, _) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header("authorization", "Bearer token-one") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&change).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(change_status, StatusCode::OK); + + let merge = BranchMergeRequest { + source: "feature".to_string(), + target: Some("main".to_string()), + }; + let (merge_status, merge_body) = json_response( + &app, + Request::builder() + .uri("/branches/merge") + .method(Method::POST) + .header("authorization", "Bearer token-two") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&merge).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(merge_status, StatusCode::OK); + assert_eq!(merge_body["actor_id"], "act-ragnor"); + + let (commit_status, commit_body) = json_response( + &app, + Request::builder() + .uri("/commits?branch=main") + .method(Method::GET) + .header("authorization", "Bearer token-two") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(commit_status, StatusCode::OK); + let head = commit_body["commits"] + .as_array() + .unwrap() + .last() + .expect("head commit should exist"); + assert_eq!(head["actor_id"], "act-ragnor"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn engine_layer_policy_fires_via_direct_arc_omnigraph_from_new_single() { + use omnigraph_server::GraphRouting; + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + + // Permit `act-allowed` for change actions; `act-blocked` is not in + // any allowed group — every change request from them must deny. + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, permit_all_policy_yaml(&["act-allowed"])).unwrap(); + let policy_engine = + omnigraph_server::PolicyEngine::load_graph(&policy_path, graph.to_string_lossy().as_ref()) + .unwrap(); + + let workload = omnigraph_server::workload::WorkloadController::new(100, 1_000_000_000); + let state = AppState::new_single( + graph.to_string_lossy().to_string(), + db, + vec![("act-blocked".to_string(), "block-token".to_string())], + Some(policy_engine), + workload, + ); + + // Reach into the routing and pull the engine the same way an + // embedded consumer holding `Arc<Omnigraph>` would. If `new_single` + // failed to apply `with_policy` to the engine, this `mutate_as` + // would succeed — the HTTP-layer is bypassed entirely. + let handle = match state.routing() { + GraphRouting::Single { handle } => Arc::clone(handle), + GraphRouting::Multi { .. } => panic!("expected single-mode routing"), + }; + let engine = Arc::clone(&handle.engine); + + let mut params: omnigraph_compiler::ParamMap = Default::default(); + params.insert( + "name".to_string(), + omnigraph_compiler::Literal::String("EngineLayerBlocked".to_string()), + ); + params.insert("age".to_string(), omnigraph_compiler::Literal::Integer(30)); + let result = engine + .mutate_as( + "main", + MUTATION_QUERIES, + "insert_person", + ¶ms, + Some("act-blocked"), + ) + .await; + match result { + Err(OmniError::Policy(_)) => { /* expected — engine-layer gate fired */ } + Ok(_) => panic!( + "engine-layer policy did NOT fire — act-blocked successfully ran mutate_as via \ + the engine pulled from the registry handle. AppState::new_single failed to apply \ + with_policy to the underlying Omnigraph engine. This is the B2 footgun the \ + with_policy_engine deletion was supposed to close." + ), + Err(other) => panic!("expected OmniError::Policy, got: {other:?}"), + } +} + +#[tokio::test(flavor = "multi_thread")] +async fn oversized_request_body_returns_payload_too_large() { + let (_temp, app) = app_for_loaded_graph().await; + let oversized = "x".repeat(1_100_000); + let response = app + .clone() + .oneshot( + Request::builder() + .uri("/read") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(oversized)) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::PAYLOAD_TOO_LARGE); +} + +#[tokio::test(flavor = "multi_thread")] +async fn default_deny_mode_allows_read_for_authenticated_actor() { + let (_temp, app) = app_for_graph_with_auth_tokens_only( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-andrew", "demo-token")], + ) + .await; + + let (status, _body) = json_response( + &app, + Request::builder() + .uri("/snapshot") + .method(Method::GET) + .header(AUTHORIZATION, "Bearer demo-token") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); +} + +#[tokio::test(flavor = "multi_thread")] +async fn default_deny_mode_rejects_change_with_forbidden() { + let (_temp, app) = app_for_graph_with_auth_tokens_only( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-andrew", "demo-token")], + ) + .await; + + let change = ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": "DefaultDeny", "age": 1 })), + branch: Some("main".to_string()), + }; + let (status, body) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header(AUTHORIZATION, "Bearer demo-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&change).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::FORBIDDEN); + let error: ErrorOutput = serde_json::from_value(body).unwrap(); + assert!( + error.error.contains("default-deny"), + "expected default-deny in error message, got: {}", + error.error + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn default_deny_mode_rejects_schema_apply_with_forbidden() { + let (_temp, app) = app_for_graph_with_auth_tokens_only( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-andrew", "demo-token")], + ) + .await; + + let req = SchemaApplyRequest { + schema_source: additive_schema_with_nickname(), + ..Default::default() + }; + let (status, body) = json_response( + &app, + Request::builder() + .uri("/schema/apply") + .method(Method::POST) + .header(AUTHORIZATION, "Bearer demo-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&req).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::FORBIDDEN); + let error: ErrorOutput = serde_json::from_value(body).unwrap(); + assert!( + error.error.contains("default-deny"), + "expected default-deny in error message, got: {}", + error.error + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn policy_decision_parity_change_admin_on_main_allowed() { + // (act-ragnor, change, main) — admins-change-anywhere rule applies. + // Both SDK and HTTP must allow. Each path uses its own fresh graph + // because allow→side-effects. + let (_t1, graph1, policy1) = build_parity_graph().await; + let sdk = sdk_change_decision(&graph1, &policy1, "act-ragnor").await; + let (_t2, graph2, policy2) = build_parity_graph().await; + let http = http_change_decision(&graph2, &policy2, "act-ragnor", "ragnor-token").await; + assert!( + matches!(sdk, ParityDecision::Allow) && matches!(http, ParityDecision::Allow), + "SDK={sdk:?} HTTP={http:?} — should both Allow", + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn policy_decision_parity_change_team_on_main_denied() { + // (act-bruno, change, main) — no rule grants bruno change on + // protected. Both SDK and HTTP must deny. Same graph is reusable + // because deny→no side-effects. + let (_temp, graph, policy) = build_parity_graph().await; + let sdk = sdk_change_decision(&graph, &policy, "act-bruno").await; + let http = http_change_decision(&graph, &policy, "act-bruno", "bruno-token").await; + assert!( + matches!(sdk, ParityDecision::Deny) && matches!(http, ParityDecision::Deny), + "SDK={sdk:?} HTTP={http:?} — should both Deny", + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn policy_decision_parity_branch_merge_admin_allowed() { + // (act-ragnor, branch_merge, feature→main) — admins-merge-to-protected + // rule applies. Both Allow. Each path uses its own fresh graph — + // a successful merge consumes the feature branch's commit on main. + let (_t1, graph1, policy1) = build_parity_graph().await; + let sdk = sdk_merge_decision(&graph1, &policy1, "act-ragnor").await; + let (_t2, graph2, policy2) = build_parity_graph().await; + let http = http_merge_decision(&graph2, &policy2, "act-ragnor", "ragnor-token").await; + assert!( + matches!(sdk, ParityDecision::Allow) && matches!(http, ParityDecision::Allow), + "SDK={sdk:?} HTTP={http:?} — should both Allow", + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn policy_decision_parity_branch_merge_team_denied() { + // (act-bruno, branch_merge, feature→main) — no rule grants bruno + // branch_merge. Both Deny. + let (_temp, graph, policy) = build_parity_graph().await; + let sdk = sdk_merge_decision(&graph, &policy, "act-bruno").await; + let http = http_merge_decision(&graph, &policy, "act-bruno", "bruno-token").await; + assert!( + matches!(sdk, ParityDecision::Deny) && matches!(http, ParityDecision::Deny), + "SDK={sdk:?} HTTP={http:?} — should both Deny", + ); +} diff --git a/crates/omnigraph-server/tests/boot_settings.rs b/crates/omnigraph-server/tests/boot_settings.rs new file mode 100644 index 0000000..0e75486 --- /dev/null +++ b/crates/omnigraph-server/tests/boot_settings.rs @@ -0,0 +1,1022 @@ +//! Server settings loading and mode inference (single vs multi). +//! Moved verbatim from tests/server.rs in the modularization. + +use std::fs; + +use axum::Router; +use axum::body::{Body, to_bytes}; +use axum::http::{Method, Request, StatusCode}; +use omnigraph::db::Omnigraph; +use omnigraph_server::{AppState, build_app}; +use serde_json::Value; +use tower::ServiceExt; + + +mod support; +use support::*; + +mod multi_graph_startup { + use super::*; + use omnigraph::storage::normalize_root_uri; + use omnigraph_server::{ + GraphHandle, GraphId, GraphKey, GraphRegistry, InsertError, ServerConfig, ServerConfigMode, + load_server_settings, + }; + use std::sync::Arc; + + async fn build_multi_mode_app(graph_ids: &[&str]) -> (Vec<tempfile::TempDir>, Router) { + let mut dirs = Vec::with_capacity(graph_ids.len()); + let mut handles = Vec::with_capacity(graph_ids.len()); + for id in graph_ids { + let dir = tempfile::tempdir().unwrap(); + let graph_uri = dir.path().join(id).to_str().unwrap().to_string(); + let schema = fs::read_to_string(fixture("test.pg")).unwrap(); + let engine = Omnigraph::init(&graph_uri, &schema).await.unwrap(); + handles.push(Arc::new(GraphHandle { + key: GraphKey::cluster(GraphId::try_from(*id).unwrap()), + uri: graph_uri, + engine: Arc::new(engine), + policy: None, + queries: None, + })); + dirs.push(dir); + } + let workload = omnigraph_server::workload::WorkloadController::from_env(); + let state = AppState::new_multi(handles, Vec::new(), None, workload, None).unwrap(); + let app = build_app(state); + (dirs, app) + } + + /// Cluster route `/graphs/{graph_id}/snapshot` resolves to the right + /// engine. Two graphs side by side; assert each responds to its own + /// id and does NOT respond to the other's URL. + #[tokio::test(flavor = "multi_thread")] + async fn cluster_routes_dispatch_per_graph_handle() { + let (_dirs, app) = build_multi_mode_app(&["alpha", "beta"]).await; + for id in ["alpha", "beta"] { + let resp = app + .clone() + .oneshot( + Request::builder() + .method(Method::GET) + .uri(format!("/graphs/{id}/snapshot?branch=main")) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!( + resp.status(), + StatusCode::OK, + "graph '{id}' must respond OK on its cluster snapshot route" + ); + } + } + + /// Unknown graph id under the cluster prefix yields 404 (not 500, + /// not 410 — `Gone` is reserved for the future DELETE flow). + #[tokio::test(flavor = "multi_thread")] + async fn cluster_route_for_unknown_graph_returns_404() { + let (_dirs, app) = build_multi_mode_app(&["alpha"]).await; + let resp = app + .oneshot( + Request::builder() + .method(Method::GET) + .uri("/graphs/nonexistent/snapshot?branch=main") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(resp.status(), StatusCode::NOT_FOUND); + } + + /// Coverage net for cluster-route regressions across every + /// protected handler — not just the few that have inner path + /// params. Bug-1 surfaced because only `/snapshot` was being + /// exercised in cluster mode, leaving the other six protected + /// routes implicitly untested. This sweep hits each one and + /// asserts the response shows the handler was reached: no 404 + /// (router didn't match), no 500 with "Wrong number of path + /// arguments" (path extractor broke), no 500 with "missing + /// extension" (routing middleware didn't inject the handle). + /// + /// Status codes are negative assertions because each handler's + /// happy-path inputs differ — what matters is "the request + /// reached the handler," not "the handler returned 200." The + /// individual handlers' logic is already tested in single mode. + #[tokio::test(flavor = "multi_thread")] + async fn all_protected_cluster_routes_resolve_to_their_handler() { + let (_dirs, app) = build_multi_mode_app(&["alpha"]).await; + + // (method, path, body) — one minimal request per protected + // cluster route. Bodies are valid enough that the router and + // extractors succeed; whether the engine ultimately returns + // 200 or 4xx is per-handler and not what this test pins. + let cases: &[(Method, &str, Option<&str>)] = &[ + (Method::GET, "/graphs/alpha/snapshot?branch=main", None), + (Method::GET, "/graphs/alpha/schema", None), + (Method::GET, "/graphs/alpha/branches", None), + (Method::GET, "/graphs/alpha/commits", None), + ( + Method::POST, + "/graphs/alpha/read", + Some(r#"{"query_source":"query q() { return {} }"}"#), + ), + ( + Method::POST, + "/graphs/alpha/change", + Some(r#"{"query_source":"query q() { return {} }"}"#), + ), + ( + Method::POST, + "/graphs/alpha/export", + Some(r#"{"branch":"main"}"#), + ), + ( + Method::POST, + "/graphs/alpha/schema/apply", + Some(r#"{"schema_source":"","allow_data_loss":false}"#), + ), + (Method::POST, "/graphs/alpha/ingest", Some(r#"{"data":""}"#)), + ( + Method::POST, + "/graphs/alpha/branches/merge", + Some(r#"{"source":"main","target":"main"}"#), + ), + ]; + + for (method, path, body) in cases { + let req_body = body + .map(|s| Body::from(s.to_string())) + .unwrap_or_else(Body::empty); + let req = Request::builder() + .method(method.clone()) + .uri(*path) + .header("content-type", "application/json") + .body(req_body) + .unwrap(); + let resp = app.clone().oneshot(req).await.unwrap(); + let status = resp.status(); + let bytes = to_bytes(resp.into_body(), usize::MAX).await.unwrap(); + let body_str = String::from_utf8_lossy(&bytes); + + assert_ne!( + status, + StatusCode::NOT_FOUND, + "{} {} — router didn't match (cluster-route mounting regression). Body: {}", + method, + path, + body_str, + ); + assert!( + !(status == StatusCode::INTERNAL_SERVER_ERROR + && body_str.contains("Wrong number of path arguments")), + "{} {} — path extractor broke (Bug-1 class regression). Body: {}", + method, + path, + body_str, + ); + assert!( + !(status == StatusCode::INTERNAL_SERVER_ERROR + && body_str.to_lowercase().contains("missing extension")), + "{} {} — routing middleware didn't inject GraphHandle. Body: {}", + method, + path, + body_str, + ); + } + } + + /// Regression for the bot-surfaced path-extractor bug: cluster + /// routes whose inner path also captures a parameter + /// (`/graphs/{graph_id}/branches/{branch}`, + /// `/graphs/{graph_id}/commits/{commit_id}`) must extract the + /// inner param cleanly. Axum 0.8 propagates the outer `{graph_id}` + /// capture into nested handlers, so a `Path<String>` extractor + /// would see two values and fail with "Wrong number of path + /// arguments. Expected 1 but got 2." Today both DELETE branch and + /// GET commit-by-id break in multi-mode because their handlers + /// use bare `Path<String>` — this test pins the fix. + /// + /// The broader `all_protected_cluster_routes_resolve_to_their_handler` + /// test sweeps the full route surface; this one stays narrowly + /// targeted at the inner-path-param shape because that's the + /// specific regression class. + #[tokio::test(flavor = "multi_thread")] + async fn cluster_routes_with_inner_path_params_deserialize_correctly() { + let (_dirs, app) = build_multi_mode_app(&["alpha"]).await; + + // Create a branch we can then delete — DELETE /graphs/alpha/branches/feature + let create_resp = app + .clone() + .oneshot( + Request::builder() + .method(Method::POST) + .uri("/graphs/alpha/branches") + .header("content-type", "application/json") + .body(Body::from(r#"{"name":"feature"}"#)) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!( + create_resp.status(), + StatusCode::OK, + "branch create on the cluster route must succeed before delete can be tested" + ); + + // DELETE /graphs/{graph_id}/branches/{branch} — exercises a handler + // whose only Path extractor (`branch`) is inside a nested route + // that also captures `graph_id`. The handler must pick `branch` + // by name, not by position. + let delete_resp = app + .clone() + .oneshot( + Request::builder() + .method(Method::DELETE) + .uri("/graphs/alpha/branches/feature") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + let delete_status = delete_resp.status(); + let delete_body = to_bytes(delete_resp.into_body(), usize::MAX).await.unwrap(); + assert_eq!( + delete_status, + StatusCode::OK, + "DELETE /graphs/{{id}}/branches/{{branch}} must extract `branch` cleanly. \ + Body: {}", + String::from_utf8_lossy(&delete_body), + ); + + // GET /graphs/{graph_id}/commits/{commit_id} — same shape: the + // handler's only Path extractor is the inner `commit_id`, which + // must deserialize by name even though `graph_id` is also in scope. + // We don't know a real commit_id, but the failure mode under test + // is path extraction, not commit lookup — a 404 from the engine + // is fine; a 500 with "Wrong number of path arguments" is the bug. + let commit_resp = app + .oneshot( + Request::builder() + .method(Method::GET) + .uri("/graphs/alpha/commits/0000000000000000") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + let commit_status = commit_resp.status(); + let commit_body = to_bytes(commit_resp.into_body(), usize::MAX).await.unwrap(); + let body_str = String::from_utf8_lossy(&commit_body); + assert!( + commit_status != StatusCode::INTERNAL_SERVER_ERROR + || !body_str.contains("Wrong number of path arguments"), + "GET /graphs/{{id}}/commits/{{commit_id}} must extract `commit_id` cleanly. \ + Got: {} | {}", + commit_status, + body_str, + ); + } + + /// Flat routes 404 in multi mode — the router only mounts under + /// `/graphs/{graph_id}/...` so `/snapshot` doesn't resolve. + #[tokio::test(flavor = "multi_thread")] + async fn flat_routes_404_in_multi_mode() { + let (_dirs, app) = build_multi_mode_app(&["alpha"]).await; + let resp = app + .oneshot( + Request::builder() + .method(Method::GET) + .uri("/snapshot?branch=main") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(resp.status(), StatusCode::NOT_FOUND); + } + + /// `GraphId` validation runs at startup — a reserved name in + /// `omnigraph.yaml` produces a clear error rather than getting + /// rejected per-request. + #[tokio::test] + async fn load_server_settings_rejects_reserved_graph_id() { + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +graphs: + policies: + uri: /tmp/g1.omni +"#, + ) + .unwrap(); + let err = load_server_settings(Some(&config_path), None, None, None, None, false).await.unwrap_err(); + assert!( + err.to_string().contains("invalid graph id 'policies'"), + "expected reserved-name rejection, got: {err}" + ); + } + + #[tokio::test(flavor = "multi_thread")] + async fn registry_rejects_duplicate_normalized_graph_uris() { + let dir = tempfile::tempdir().unwrap(); + let graph_uri = dir.path().join("same").to_str().unwrap().to_string(); + let schema = fs::read_to_string(fixture("test.pg")).unwrap(); + let engine = Arc::new(Omnigraph::init(&graph_uri, &schema).await.unwrap()); + + let alpha = Arc::new(GraphHandle { + key: GraphKey::cluster(GraphId::try_from("alpha").unwrap()), + uri: graph_uri.clone(), + engine: Arc::clone(&engine), + policy: None, + queries: None, + }); + let beta = Arc::new(GraphHandle { + key: GraphKey::cluster(GraphId::try_from("beta").unwrap()), + uri: format!("file://{graph_uri}/"), + engine, + policy: None, + queries: None, + }); + + match GraphRegistry::from_handles(vec![alpha, beta]) { + Err(InsertError::DuplicateUri(uri)) => { + assert!( + normalize_root_uri(&uri).is_ok(), + "duplicate URI should still be parseable, got {uri}" + ); + } + Err(err) => panic!("expected DuplicateUri for normalized aliases, got {err:?}"), + Ok(_) => panic!("expected DuplicateUri for normalized aliases, got Ok"), + } + } + + #[tokio::test(flavor = "multi_thread")] + async fn registry_stores_canonical_graph_uri() { + let dir = tempfile::tempdir().unwrap(); + let graph_uri = dir.path().join("canonical").to_str().unwrap().to_string(); + let schema = fs::read_to_string(fixture("test.pg")).unwrap(); + let engine = Omnigraph::init(&graph_uri, &schema).await.unwrap(); + let handle = Arc::new(GraphHandle { + key: GraphKey::cluster(GraphId::try_from("alpha").unwrap()), + uri: format!("file://{graph_uri}/"), + engine: Arc::new(engine), + policy: None, + queries: None, + }); + + let registry = GraphRegistry::from_handles(vec![handle]).unwrap(); + let listed = registry.list(); + assert_eq!(listed.len(), 1); + assert_eq!(listed[0].uri, graph_uri); + } + + // ── Four-rule mode inference matrix ─────────────────────────────── + + /// Rule 1: CLI positional URI → Single. + #[tokio::test] + async fn mode_inference_cli_uri_is_single() { + let settings = load_server_settings( + None, + None, + Some("/tmp/cli.omni".to_string()), + None, + None, + true, // allow unauth so we get past the runtime-state check + ) + .await + .unwrap(); + match settings.mode { + ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/cli.omni"), + ServerConfigMode::Multi { .. } => panic!("expected Single (rule 1), got Multi"), + } + } + + /// Rule 2: --target picks one graph from `graphs:` map → Single. + #[tokio::test] + async fn mode_inference_cli_target_is_single() { + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +graphs: + alpha: + uri: /tmp/alpha.omni + beta: + uri: /tmp/beta.omni +"#, + ) + .unwrap(); + let settings = + load_server_settings(Some(&config_path), None, None, Some("alpha".into()), None, true) + .await + .unwrap(); + match settings.mode { + ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/alpha.omni"), + ServerConfigMode::Multi { .. } => panic!("expected Single (rule 2), got Multi"), + } + } + + /// Rule 3: `server.graph` set → Single (target picked from config). + #[tokio::test] + async fn mode_inference_server_graph_is_single() { + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +graphs: + alpha: + uri: /tmp/alpha.omni + beta: + uri: /tmp/beta.omni +server: + graph: beta +"#, + ) + .unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); + match settings.mode { + ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/beta.omni"), + ServerConfigMode::Multi { .. } => panic!("expected Single (rule 3), got Multi"), + } + } + + /// Rule 4: `--config` + non-empty `graphs:` + no single-mode selector → Multi. + #[tokio::test] + async fn mode_inference_config_plus_graphs_is_multi() { + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +graphs: + alpha: + uri: /tmp/alpha.omni + beta: + uri: /tmp/beta.omni +"#, + ) + .unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); + match settings.mode { + ServerConfigMode::Multi { graphs, .. } => { + let ids: Vec<&str> = graphs.iter().map(|g| g.graph_id.as_str()).collect(); + // BTreeMap iteration order is alphabetical. + assert_eq!(ids, vec!["alpha", "beta"]); + } + ServerConfigMode::Single { .. } => panic!("expected Multi (rule 4), got Single"), + } + } + + #[tokio::test] + async fn mode_inference_multi_rejects_top_level_policy_file() { + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +policy: + file: ./policy.yaml +graphs: + alpha: + uri: /tmp/alpha.omni +"#, + ) + .unwrap(); + let err = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap_err(); + let msg = err.to_string(); + assert!( + msg.contains("top-level") && msg.contains("policy.file") && msg.contains("not honored"), + "expected top-level-not-honored guidance, got: {msg}" + ); + assert!( + msg.contains("graphs.<graph_id>"), + "expected per-graph migration guidance, got: {msg}" + ); + assert!( + msg.contains("server.policy.file"), + "expected server policy migration guidance, got: {msg}" + ); + } + + #[tokio::test] + async fn mode_inference_multi_rejects_top_level_queries() { + // Symmetric to the policy guard: a top-level `queries:` block in + // multi-graph mode is not honored (each graph uses its own), so it + // is a loud error rather than a silent no-op. + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + "queries:\n q:\n file: ./q.gq\ngraphs:\n alpha:\n uri: /tmp/alpha.omni\n", + ) + .unwrap(); + let err = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap_err(); + let msg = err.to_string(); + assert!( + msg.contains("queries") && msg.contains("not honored"), + "top-level queries must be rejected in multi-graph mode: {msg}" + ); + } + + #[tokio::test] + async fn single_mode_named_graph_rejects_top_level_blocks() { + // Serving a graph by name (`--target`/`server.graph`) uses its + // per-graph block; a populated top-level block would be silently + // shadowed, so boot refuses and names the per-graph location. + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + "policy:\n file: ./top.yaml\ngraphs:\n prod:\n uri: /tmp/prod.omni\n", + ) + .unwrap(); + let err = + load_server_settings(Some(&config_path), None, None, Some("prod".to_string()), None, true) + .await + .unwrap_err(); + let msg = err.to_string(); + assert!( + msg.contains("prod") && msg.contains("policy.file") && msg.contains("graphs.prod"), + "named single-mode + top-level policy must refuse, naming the graph: {msg}" + ); + } + + #[tokio::test] + async fn single_mode_named_graph_uses_per_graph_policy_and_queries() { + // The identity rule: `--target prod` attaches `graphs.prod`'s own + // policy + queries, not the top-level ones (which are absent here). + let temp = tempfile::tempdir().unwrap(); + fs::write( + temp.path().join("prod.gq"), + "query pq() { match { $u: User } return { $u.name } }", + ) + .unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + "graphs:\n prod:\n uri: /tmp/prod.omni\n policy:\n file: ./prod-policy.yaml\n \ + queries:\n pq:\n file: ./prod.gq\n", + ) + .unwrap(); + let settings = + load_server_settings(Some(&config_path), None, None, Some("prod".to_string()), None, true) + .await + .unwrap(); + match settings.mode { + ServerConfigMode::Single { + graph_id, + policy_file, + queries, + .. + } => { + assert_eq!(graph_id, "prod", "named single-mode keeps graph identity"); + assert!( + policy_file + .as_ref() + .is_some_and(|p| p.ends_with("prod-policy.yaml")), + "per-graph policy attached: {policy_file:?}" + ); + assert!(queries.lookup("pq").is_some(), "per-graph query attached"); + } + other => panic!("expected Single mode, got {other:?}"), + } + } + + #[tokio::test] + async fn mode_inference_normalizes_multi_graph_uris() { + let temp = tempfile::tempdir().unwrap(); + let graph = temp.path().join("alpha.omni"); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + format!( + r#" +graphs: + alpha: + uri: file://{}/ +"#, + graph.display() + ), + ) + .unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); + match settings.mode { + ServerConfigMode::Multi { graphs, .. } => { + assert_eq!(graphs[0].uri, graph.to_string_lossy()); + } + ServerConfigMode::Single { .. } => panic!("expected Multi"), + } + } + + /// Rule 5: nothing → error with migration hint. + #[tokio::test] + async fn mode_inference_no_inputs_errors_with_migration_hint() { + let err = load_server_settings(None, None, None, None, None, true).await.unwrap_err(); + let msg = err.to_string(); + assert!( + msg.contains("no graph to serve"), + "expected migration-hint error, got: {msg}" + ); + } + + /// Rule 4 sub-case: `--config` with empty `graphs:` map and no + /// single-mode selector → rule 5 fires (no graph to serve). + #[tokio::test] + async fn mode_inference_empty_graphs_map_errors() { + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write(&config_path, "server:\n bind: 127.0.0.1:8080\n").unwrap(); + let err = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap_err(); + assert!(err.to_string().contains("no graph to serve")); + } + + /// `--config` + `<URI>` together: URI wins → Single (the CLI URI + /// takes precedence over the config's graphs map). + #[tokio::test] + async fn mode_inference_cli_uri_overrides_graphs_map() { + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +graphs: + alpha: + uri: /tmp/alpha.omni +"#, + ) + .unwrap(); + let settings = load_server_settings( + Some(&config_path), + None, + Some("/tmp/cli-override.omni".to_string()), + None, + None, + true, + ) + .await + .unwrap(); + match settings.mode { + ServerConfigMode::Single { uri, .. } => { + assert_eq!( + uri, "/tmp/cli-override.omni", + "CLI URI must win over graphs: map" + ); + } + ServerConfigMode::Multi { .. } => { + panic!("expected Single (CLI URI wins), got Multi") + } + } + } + + /// Per-graph `policy.file` is resolved relative to the config base_dir. + #[tokio::test] + async fn per_graph_policy_file_is_resolved_relative_to_base_dir() { + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +graphs: + alpha: + uri: /tmp/alpha.omni + policy: + file: ./policies/alpha.yaml + beta: + uri: /tmp/beta.omni +"#, + ) + .unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); + let graphs = match settings.mode { + ServerConfigMode::Multi { graphs, .. } => graphs, + _ => panic!("expected Multi"), + }; + // graphs is BTreeMap-iter order (alphabetical). + let alpha = &graphs[0]; + let beta = &graphs[1]; + assert_eq!(alpha.graph_id, "alpha"); + assert_eq!( + alpha.policy_file.as_ref().unwrap(), + &temp.path().join("policies/alpha.yaml") + ); + assert_eq!(beta.graph_id, "beta"); + assert!(beta.policy_file.is_none()); + } + + /// `server.policy.file` resolves alongside the graphs map. + #[tokio::test] + async fn server_policy_file_is_resolved_relative_to_base_dir() { + let temp = tempfile::tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +server: + policy: + file: ./server-policy.yaml +graphs: + alpha: + uri: /tmp/alpha.omni +"#, + ) + .unwrap(); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); + match settings.mode { + ServerConfigMode::Multi { + server_policy_file, .. + } => { + assert_eq!( + server_policy_file.unwrap(), + temp.path().join("server-policy.yaml") + ); + } + _ => panic!("expected Multi"), + } + } + + /// `GET /graphs` must NOT leak the registry in Open mode without + /// an explicit server policy. Operators who pass `--unauthenticated` + /// opted into trusting the network for graph DATA, not for leaking + /// server topology (graph IDs + URIs, which may contain S3 bucket + /// paths or internal hostnames). Cedar gating the management + /// surface is the documented contract for `server_graphs_list` + /// ("don't leak the registry until the operator explicitly + /// authorizes it"); enforcing that contract in every runtime + /// state — not just `PolicyEnabled` — is the correct-by-design + /// closure of the open-mode hole the bot-review pass surfaced. + /// + /// Today (pre-fix) this returns 200 because `authorize_request`'s + /// no-policy fallback only denies when `actor.is_some()`, so Open + /// mode (`actor: None`) falls through to `Ok(())`. The fix in the + /// next commit tightens the fallback so server-scoped actions + /// always require explicit policy. + /// + /// Sort-order coverage previously lived here; it has moved to + /// `get_graphs_with_server_policy_authorizes_per_cedar` where + /// the response body is now non-empty and operator-authorized. + #[tokio::test(flavor = "multi_thread")] + async fn get_graphs_denied_in_open_mode_without_server_policy() { + let (_dirs, app) = build_multi_mode_app(&["beta", "alpha"]).await; + let resp = app + .oneshot( + Request::builder() + .method(Method::GET) + .uri("/graphs") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + let status = resp.status(); + let body = to_bytes(resp.into_body(), usize::MAX).await.unwrap(); + let body_str = String::from_utf8_lossy(&body); + assert_eq!( + status, + StatusCode::FORBIDDEN, + "GET /graphs must require an explicit server policy in every \ + runtime state; Open-mode bypass would leak server topology. \ + Body: {body_str}", + ); + } + + /// `GET /graphs` returns 405 in single mode (resource exists in the + /// API surface, just not operational without a `graphs:` map). + #[tokio::test(flavor = "multi_thread")] + async fn get_graphs_returns_405_in_single_mode() { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let state = AppState::open(graph.to_string_lossy().to_string()) + .await + .unwrap(); + let app = build_app(state); + let resp = app + .oneshot( + Request::builder() + .method(Method::GET) + .uri("/graphs") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(resp.status(), StatusCode::METHOD_NOT_ALLOWED); + } + + /// `GET /graphs` requires bearer auth when tokens are configured. + #[tokio::test(flavor = "multi_thread")] + async fn get_graphs_requires_bearer_auth_when_configured() { + use omnigraph_server::{GraphHandle, GraphId, GraphKey}; + // Build a multi-mode app with bearer tokens configured. + let dir = tempfile::tempdir().unwrap(); + let graph_uri = dir.path().join("alpha").to_str().unwrap().to_string(); + let schema = fs::read_to_string(fixture("test.pg")).unwrap(); + let engine = Omnigraph::init(&graph_uri, &schema).await.unwrap(); + let handle = Arc::new(GraphHandle { + key: GraphKey::cluster(GraphId::try_from("alpha").unwrap()), + uri: graph_uri, + engine: Arc::new(engine), + policy: None, + queries: None, + }); + let tokens = vec![("act-andrew".to_string(), "secret-token".to_string())]; + let workload = omnigraph_server::workload::WorkloadController::from_env(); + let state = AppState::new_multi(vec![handle], tokens, None, workload, None).unwrap(); + let app = build_app(state); + + // No Authorization header → 401. + let resp_no_auth = app + .clone() + .oneshot( + Request::builder() + .method(Method::GET) + .uri("/graphs") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(resp_no_auth.status(), StatusCode::UNAUTHORIZED); + + // With auth but no server policy → 403 (default-deny, since + // GraphList is not Read). + let resp_authed = app + .oneshot( + Request::builder() + .method(Method::GET) + .uri("/graphs") + .header("authorization", "Bearer secret-token") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(resp_authed.status(), StatusCode::FORBIDDEN); + } + + /// `GET /graphs` with a server policy that allows `graph_list` → 200 + /// and returns the registry sorted alphabetically by `graph_id`. + /// `GET /graphs` with a server policy that does NOT allow + /// `graph_list` (viewer group) → 403. + /// + /// This test owns the alphabetical-sort coverage that previously + /// lived in `get_graphs_lists_registered_graphs_in_multi_mode`. + /// That test now asserts denial in Open mode (server-scoped actions + /// require explicit policy in every runtime state), so the positive + /// body-shape assertions need a home where the response is + /// operator-authorized — here. + #[tokio::test(flavor = "multi_thread")] + async fn get_graphs_with_server_policy_authorizes_per_cedar() { + use omnigraph_policy::PolicyEngine; + use omnigraph_server::{GraphHandle, GraphId, GraphKey}; + + let dir = tempfile::tempdir().unwrap(); + + // Two graphs deliberately registered in non-alphabetical order + // so the test would fail if the handler relied on insertion + // order instead of server-side sorting. + let schema = fs::read_to_string(fixture("test.pg")).unwrap(); + let mut handles = Vec::new(); + for id in ["beta", "alpha"] { + let graph_uri = dir.path().join(id).to_str().unwrap().to_string(); + let engine = Omnigraph::init(&graph_uri, &schema).await.unwrap(); + handles.push(Arc::new(GraphHandle { + key: GraphKey::cluster(GraphId::try_from(id).unwrap()), + uri: graph_uri, + engine: Arc::new(engine), + policy: None, + queries: None, + })); + } + + // Server policy: admins can graph_list, viewers cannot. + let policy_path = dir.path().join("server-policy.yaml"); + fs::write( + &policy_path, + r#" +version: 1 +groups: + admins: [act-andrew] + viewers: [act-bruno] +rules: + - id: admins-list-graphs + allow: + actors: { group: admins } + actions: [graph_list] +"#, + ) + .unwrap(); + let server_policy = PolicyEngine::load_server(&policy_path).unwrap(); + + let tokens = vec![ + ("act-andrew".to_string(), "andrew-token".to_string()), + ("act-bruno".to_string(), "bruno-token".to_string()), + ]; + let workload = omnigraph_server::workload::WorkloadController::from_env(); + let state = + AppState::new_multi(handles, tokens, Some(server_policy), workload, None).unwrap(); + let app = build_app(state); + + // Admin → 200, body returns both graphs alphabetically sorted. + let resp_admin = app + .clone() + .oneshot( + Request::builder() + .method(Method::GET) + .uri("/graphs") + .header("authorization", "Bearer andrew-token") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!( + resp_admin.status(), + StatusCode::OK, + "admin must be allowed graph_list" + ); + let body = to_bytes(resp_admin.into_body(), usize::MAX).await.unwrap(); + let json: Value = serde_json::from_slice(&body).unwrap(); + let graphs = json["graphs"].as_array().unwrap(); + assert_eq!(graphs.len(), 2, "response must list both registered graphs"); + assert_eq!( + graphs[0]["graph_id"].as_str().unwrap(), + "alpha", + "server must sort graphs alphabetically by graph_id (insertion order was 'beta', 'alpha')" + ); + assert_eq!(graphs[1]["graph_id"].as_str().unwrap(), "beta"); + + // Viewer → 403 + let resp_viewer = app + .oneshot( + Request::builder() + .method(Method::GET) + .uri("/graphs") + .header("authorization", "Bearer bruno-token") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!( + resp_viewer.status(), + StatusCode::FORBIDDEN, + "viewer must be denied graph_list (Cedar gate)" + ); + } + + /// Loads an `omnigraph.yaml` with two graphs and verifies multi-mode + /// inference plus graph entry resolution. Cluster-route dispatch is + /// covered by the route tests above. + #[tokio::test(flavor = "multi_thread")] + async fn server_settings_load_multi_graph_config_entries() { + let cfg_dir = tempfile::tempdir().unwrap(); + // Real graph storage dirs (the URIs in the config must point to + // a graph init-able location). + let alpha_dir = cfg_dir.path().join("alpha.omni"); + let beta_dir = cfg_dir.path().join("beta.omni"); + let schema = fs::read_to_string(fixture("test.pg")).unwrap(); + Omnigraph::init(alpha_dir.to_str().unwrap(), &schema) + .await + .unwrap(); + Omnigraph::init(beta_dir.to_str().unwrap(), &schema) + .await + .unwrap(); + + let config_path = cfg_dir.path().join("omnigraph.yaml"); + fs::write( + &config_path, + format!( + r#" +graphs: + alpha: + uri: {alpha} + beta: + uri: {beta} +"#, + alpha = alpha_dir.display(), + beta = beta_dir.display(), + ), + ) + .unwrap(); + + let settings: ServerConfig = + load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); + assert!(matches!(settings.mode, ServerConfigMode::Multi { .. })); + + match settings.mode { + ServerConfigMode::Multi { graphs, .. } => { + assert_eq!(graphs.len(), 2); + let ids: Vec<&str> = graphs.iter().map(|g| g.graph_id.as_str()).collect(); + assert_eq!(ids, vec!["alpha", "beta"]); + } + _ => unreachable!(), + } + } +} diff --git a/crates/omnigraph-server/tests/data_routes.rs b/crates/omnigraph-server/tests/data_routes.rs new file mode 100644 index 0000000..cef2f9a --- /dev/null +++ b/crates/omnigraph-server/tests/data_routes.rs @@ -0,0 +1,1572 @@ +//! Data-plane routes: read/query/change/ingest/branches/snapshot/export. +//! Moved verbatim from tests/server.rs in the modularization. + +use std::fs; +use std::sync::Arc; + +use axum::body::{Body, to_bytes}; +use axum::http::{Method, Request, StatusCode}; +use omnigraph::db::{Omnigraph, ReadTarget}; +use omnigraph::loader::LoadMode; +use omnigraph_server::api::{ + BranchCreateRequest, BranchMergeRequest, ChangeRequest, ErrorOutput, ExportRequest, + IngestRequest, QueryRequest, ReadRequest, +}; +use omnigraph_server::{AppState, build_app}; +use serde_json::{Value, json}; +use serial_test::serial; +use tower::ServiceExt; + + +mod support; +use support::*; + +#[tokio::test(flavor = "multi_thread")] +async fn export_route_returns_jsonl_for_branch_snapshot() { + let token = "demo-token"; + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.branch_create_from(ReadTarget::branch("main"), "feature") + .await + .unwrap(); + db.load( + "feature", + r#"{"type":"Person","data":{"name":"Eve","age":29}}"#, + LoadMode::Append, + ) + .await + .unwrap(); + let expected = db + .export_jsonl("feature", &["Person".to_string()], &[]) + .await + .unwrap(); + drop(db); + + // MR-723: tokens-without-policy is now default-deny. Install a + // permit-all policy alongside the bearer token so /export + // (action=Export) passes Cedar evaluation. The test is exercising + // export semantics, not policy — the policy is just enough to clear + // the State 3 path. + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, permit_all_policy_yaml(&["default"])).unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![("default".to_string(), token.to_string())], + Some(&policy_path), + ) + .await + .unwrap(); + let app = build_app(state); + + let response = app + .clone() + .oneshot( + Request::builder() + .uri("/export") + .method(Method::POST) + .header("content-type", "application/json") + .header("authorization", format!("Bearer {}", token)) + .body(Body::from( + serde_json::to_vec(&ExportRequest { + branch: Some("feature".to_string()), + type_names: vec!["Person".to_string()], + table_keys: Vec::new(), + }) + .unwrap(), + )) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + response.headers().get("content-type").unwrap(), + "application/x-ndjson; charset=utf-8" + ); + let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + let text = String::from_utf8(body.to_vec()).unwrap(); + assert_eq!(text, expected); +} + +#[tokio::test(flavor = "multi_thread")] +async fn snapshot_route_returns_manifest_dataset_version() { + let (temp, app) = app_for_loaded_graph().await; + let graph = graph_path(temp.path()); + let expected_manifest_version = manifest_dataset_version(&graph).await; + + let (snapshot_status, snapshot_body) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + + assert_eq!(snapshot_status, StatusCode::OK); + assert_eq!(snapshot_body["branch"], "main"); + assert_eq!( + snapshot_body["manifest_version"].as_u64().unwrap(), + expected_manifest_version + ); + assert!(snapshot_body["tables"].is_array()); +} + +#[tokio::test(flavor = "multi_thread")] +async fn ingest_creates_branch_returns_metadata_and_stamps_actor() { + let (temp, app) = app_for_loaded_graph_with_auth_tokens(&[("act-andrew", "token-one")]).await; + let graph = graph_path(temp.path()); + let ingest = IngestRequest { + branch: Some("feature-ingest".to_string()), + from: Some("main".to_string()), + mode: Some(LoadMode::Merge), + data: r#"{"type":"Person","data":{"name":"Zoe","age":33}} +{"type":"Person","data":{"name":"Bob","age":26}}"# + .to_string(), + }; + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/ingest") + .method(Method::POST) + .header("authorization", "Bearer token-one") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&ingest).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(body["branch"], "feature-ingest"); + assert_eq!(body["base_branch"], "main"); + assert_eq!(body["branch_created"], true); + assert_eq!(body["mode"], "merge"); + assert_eq!(body["actor_id"], "act-andrew"); + assert_eq!(body["tables"][0]["table_key"], "node:Person"); + assert_eq!(body["tables"][0]["rows_loaded"], 2); + + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + let snapshot = db + .snapshot_of(ReadTarget::branch("feature-ingest")) + .await + .unwrap(); + let person_ds = snapshot.open("node:Person").await.unwrap(); + assert_eq!(person_ds.count_rows(None).await.unwrap(), 5); + let head = db + .list_commits(Some("feature-ingest")) + .await + .unwrap() + .into_iter() + .last() + .unwrap(); + assert_eq!(head.actor_id.as_deref(), Some("act-andrew")); +} + +#[tokio::test(flavor = "multi_thread")] +async fn ingest_existing_branch_skips_branch_create_policy_check() { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + { + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.branch_create_from(ReadTarget::branch("main"), "feature") + .await + .unwrap(); + } + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, POLICY_YAML).unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![("act-bruno".to_string(), "team-token".to_string())], + Some(&policy_path), + ) + .await + .unwrap(); + let app = build_app(state); + let ingest = IngestRequest { + branch: Some("feature".to_string()), + from: Some("other-base".to_string()), + mode: Some(LoadMode::Merge), + data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), + }; + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/ingest") + .method(Method::POST) + .header("authorization", "Bearer team-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&ingest).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(body["branch"], "feature"); + assert_eq!(body["branch_created"], false); + assert_eq!(body["base_branch"], "other-base"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn ingest_without_from_returns_404_for_missing_branch_and_creates_nothing() { + let (temp, app) = app_for_loaded_graph().await; + let graph = graph_path(temp.path()); + let ingest = IngestRequest { + branch: Some("feature-typo".to_string()), + from: None, + mode: Some(LoadMode::Merge), + data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), + }; + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/ingest") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&ingest).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::NOT_FOUND); + let error: ErrorOutput = serde_json::from_value(body).unwrap(); + assert_eq!(error.code, Some(omnigraph_server::api::ErrorCode::NotFound)); + + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + assert!( + !db.branch_list() + .await + .unwrap() + .contains(&"feature-typo".to_string()), + "a 404'd ingest must not create the branch" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn ingest_without_from_loads_into_existing_branch() { + let (temp, app) = app_for_loaded_graph().await; + let graph = graph_path(temp.path()); + { + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.branch_create_from(ReadTarget::branch("main"), "feature") + .await + .unwrap(); + } + let ingest = IngestRequest { + branch: Some("feature".to_string()), + from: None, + mode: Some(LoadMode::Merge), + data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), + }; + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/ingest") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&ingest).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(body["branch"], "feature"); + assert_eq!(body["branch_created"], false); + assert_eq!(body["base_branch"], serde_json::Value::Null); +} + +#[tokio::test(flavor = "multi_thread")] +async fn ingest_denies_missing_branch_without_branch_create_permission() { + let (_temp, app) = app_for_loaded_graph_with_auth_tokens_and_policy( + &[("act-bruno", "team-token")], + POLICY_YAML, + ) + .await; + let ingest = IngestRequest { + branch: Some("feature".to_string()), + from: Some("main".to_string()), + mode: Some(LoadMode::Merge), + data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), + }; + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/ingest") + .method(Method::POST) + .header("authorization", "Bearer team-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&ingest).unwrap())) + .unwrap(), + ) + .await; + let error: ErrorOutput = serde_json::from_value(body).unwrap(); + assert_eq!(status, StatusCode::FORBIDDEN); + assert_eq!( + error.code, + Some(omnigraph_server::api::ErrorCode::Forbidden) + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn ingest_denies_when_actor_lacks_change_permission() { + let (_temp, app) = app_for_loaded_graph_with_auth_tokens_and_policy( + &[("act-bruno", "team-token")], + INGEST_CREATE_ONLY_POLICY_YAML, + ) + .await; + let ingest = IngestRequest { + branch: Some("feature".to_string()), + from: Some("main".to_string()), + mode: Some(LoadMode::Merge), + data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), + }; + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/ingest") + .method(Method::POST) + .header("authorization", "Bearer team-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&ingest).unwrap())) + .unwrap(), + ) + .await; + let error: ErrorOutput = serde_json::from_value(body).unwrap(); + assert_eq!(status, StatusCode::FORBIDDEN); + assert_eq!( + error.code, + Some(omnigraph_server::api::ErrorCode::Forbidden) + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn ingest_rejects_payloads_over_32_mib() { + let (_temp, app) = app_for_loaded_graph().await; + let oversize = IngestRequest { + branch: Some("feature".to_string()), + from: Some("main".to_string()), + mode: Some(LoadMode::Merge), + data: "x".repeat(33 * 1024 * 1024), + }; + + let response = app + .clone() + .oneshot( + Request::builder() + .uri("/ingest") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&oversize).unwrap())) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(response.status(), StatusCode::PAYLOAD_TOO_LARGE); +} + +#[tokio::test(flavor = "multi_thread")] +async fn branch_merge_conflict_response_includes_structured_conflicts() { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.branch_create_from(ReadTarget::branch("main"), "feature") + .await + .unwrap(); + db.mutate( + "main", + MUTATION_QUERIES, + "set_age", + &omnigraph_compiler::json_params_to_param_map( + Some(&json!({"name": "Alice", "age": 31 })), + &omnigraph_compiler::find_named_query(MUTATION_QUERIES, "set_age") + .unwrap() + .params, + omnigraph_compiler::JsonParamMode::Standard, + ) + .unwrap(), + ) + .await + .unwrap(); + db.mutate( + "feature", + MUTATION_QUERIES, + "set_age", + &omnigraph_compiler::json_params_to_param_map( + Some(&json!({"name": "Alice", "age": 32 })), + &omnigraph_compiler::find_named_query(MUTATION_QUERIES, "set_age") + .unwrap() + .params, + omnigraph_compiler::JsonParamMode::Standard, + ) + .unwrap(), + ) + .await + .unwrap(); + drop(db); + + let state = AppState::open(graph.to_string_lossy().to_string()) + .await + .unwrap(); + let app = build_app(state); + let merge = BranchMergeRequest { + source: "feature".to_string(), + target: Some("main".to_string()), + }; + let (status, body) = json_response( + &app, + Request::builder() + .uri("/branches/merge") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&merge).unwrap())) + .unwrap(), + ) + .await; + + let error: ErrorOutput = serde_json::from_value(body).unwrap(); + assert_eq!(status, StatusCode::CONFLICT); + assert_eq!(error.code, Some(omnigraph_server::api::ErrorCode::Conflict)); + assert!(error.error.contains("merge conflict")); + assert!(error.merge_conflicts.iter().any(|conflict| { + conflict.table_key == "node:Person" + && conflict.row_id.as_deref() == Some("Alice") + && conflict.kind == omnigraph_server::api::MergeConflictKindOutput::DivergentUpdate + })); +} + +#[tokio::test(flavor = "multi_thread")] +async fn repeated_read_after_change_sees_updated_state_from_same_app() { + let (_temp, app) = app_for_loaded_graph().await; + + let change = ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": "Mina", "age": 28 })), + branch: Some("main".to_string()), + }; + let (change_status, change_body) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&change).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(change_status, StatusCode::OK); + assert_eq!(change_body["affected_nodes"], 1); + + let read = ReadRequest { + query_source: fs::read_to_string(fixture("test.gq")).unwrap(), + query_name: Some("get_person".to_string()), + params: Some(json!({ "name": "Mina" })), + branch: Some("main".to_string()), + snapshot: None, + }; + let (read_status, read_body) = json_response( + &app, + Request::builder() + .uri("/read") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&read).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(read_status, StatusCode::OK); + assert_eq!(read_body["row_count"], 1); + assert_eq!(read_body["rows"][0]["p.name"], "Mina"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn query_endpoint_runs_inline_read() { + let (_temp, app) = app_for_loaded_graph().await; + + let query = QueryRequest { + query: fs::read_to_string(fixture("test.gq")).unwrap(), + name: Some("get_person".to_string()), + params: Some(json!({ "name": "Alice" })), + branch: Some("main".to_string()), + snapshot: None, + }; + let (status, body) = json_response( + &app, + Request::builder() + .uri("/query") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&query).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(body["query_name"], "get_person"); + assert_eq!(body["row_count"], 1); + assert_eq!(body["rows"][0]["p.name"], "Alice"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn query_endpoint_rejects_mutation_with_400() { + let (_temp, app) = app_for_loaded_graph().await; + + let query = QueryRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": "Should", "age": 1 })), + branch: Some("main".to_string()), + snapshot: None, + }; + let (status, body) = json_response( + &app, + Request::builder() + .uri("/query") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&query).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::BAD_REQUEST); + let err = body["error"].as_str().unwrap_or_default(); + assert!( + err.contains("contains mutations") && err.contains("POST /mutate"), + "expected mutation-rejection message pointing at canonical /mutate, got: {err}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn mutate_endpoint_runs_inline_mutation() { + // Canonical mutation endpoint. Pairs with `/query` on the read side. + // Same wire shape as `/change`, no deprecation signal. + let (_temp, app) = app_for_loaded_graph().await; + + let request = json!({ + "query": MUTATION_QUERIES, + "name": "insert_person", + "params": { "name": "Mutie", "age": 30 }, + "branch": "main", + }); + let response = app + .clone() + .oneshot( + Request::builder() + .uri("/mutate") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&request).unwrap())) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::OK); + // Canonical route is NOT deprecated; no Deprecation header expected. + assert!( + response.headers().get("deprecation").is_none(), + "POST /mutate must not advertise itself as deprecated" + ); + let body_bytes = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + let body: Value = serde_json::from_slice(&body_bytes).unwrap(); + assert_eq!(body["affected_nodes"], 1); + assert_eq!(body["query_name"], "insert_person"); + assert_eq!(body["branch"], "main"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn change_endpoint_emits_deprecation_headers() { + // `/change` is kept indefinitely for back-compat but flagged at runtime + // per RFC 9745 (`Deprecation: true`) + RFC 8288 (`Link: </mutate>; + // rel="successor-version"`). The OpenAPI side is covered by + // `openapi_change_is_deprecated` in tests/openapi.rs. + let (_temp, app) = app_for_loaded_graph().await; + + let request = json!({ + "query": MUTATION_QUERIES, + "name": "insert_person", + "params": { "name": "Legacyer", "age": 33 }, + "branch": "main", + }); + let response = app + .clone() + .oneshot( + Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&request).unwrap())) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + response + .headers() + .get("deprecation") + .and_then(|v| v.to_str().ok()), + Some("true"), + "POST /change must advertise `Deprecation: true` (RFC 9745)" + ); + assert_eq!( + response.headers().get("link").and_then(|v| v.to_str().ok()), + Some("</mutate>; rel=\"successor-version\""), + "POST /change must point at /mutate via `Link` rel=successor-version (RFC 8288)" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn read_endpoint_emits_deprecation_headers() { + // `/read` is kept indefinitely for byte-stable back-compat but flagged + // at runtime per RFC 9745 + RFC 8288. Successor is `/query`. + let (_temp, app) = app_for_loaded_graph().await; + + let request = ReadRequest { + query_source: fs::read_to_string(fixture("test.gq")).unwrap(), + query_name: Some("get_person".to_string()), + params: Some(json!({ "name": "Alice" })), + branch: Some("main".to_string()), + snapshot: None, + }; + let response = app + .clone() + .oneshot( + Request::builder() + .uri("/read") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&request).unwrap())) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + response + .headers() + .get("deprecation") + .and_then(|v| v.to_str().ok()), + Some("true"), + "POST /read must advertise `Deprecation: true` (RFC 9745)" + ); + assert_eq!( + response.headers().get("link").and_then(|v| v.to_str().ok()), + Some("</query>; rel=\"successor-version\""), + "POST /read must point at /query via `Link` rel=successor-version (RFC 8288)" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn query_endpoint_does_not_emit_deprecation_headers() { + // Sanity check the inverse: the canonical `/query` endpoint must not + // carry deprecation signaling, so SDK codegens don't propagate a + // bogus `@deprecated` marker. + let (_temp, app) = app_for_loaded_graph().await; + + let request = QueryRequest { + query: fs::read_to_string(fixture("test.gq")).unwrap(), + name: Some("get_person".to_string()), + params: Some(json!({ "name": "Alice" })), + branch: Some("main".to_string()), + snapshot: None, + }; + let response = app + .clone() + .oneshot( + Request::builder() + .uri("/query") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&request).unwrap())) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::OK); + assert!( + response.headers().get("deprecation").is_none(), + "POST /query is canonical and must not advertise itself as deprecated" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn change_endpoint_accepts_legacy_field_names() { + // The canonical wire field names on /change are `query` and `name`, but + // serde aliases keep the legacy `query_source`/`query_name` payload + // shape working for clients that haven't migrated yet. Pin both shapes. + let (_temp, app) = app_for_loaded_graph().await; + + let legacy_body = json!({ + "query_source": MUTATION_QUERIES, + "query_name": "insert_person", + "params": { "name": "Legacy", "age": 21 }, + "branch": "main", + }); + let (status, body) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&legacy_body).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(body["affected_nodes"], 1); + + let canonical_body = json!({ + "query": MUTATION_QUERIES, + "name": "insert_person", + "params": { "name": "Canonical", "age": 22 }, + "branch": "main", + }); + let (status, body) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&canonical_body).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(body["affected_nodes"], 1); +} + +#[tokio::test(flavor = "multi_thread")] +async fn remote_branch_list_create_merge_flow_works() { + let (_temp, app) = app_for_loaded_graph().await; + + let (list_status, list_body) = json_response( + &app, + Request::builder() + .uri("/branches") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(list_status, StatusCode::OK); + assert_eq!(list_body["branches"], json!(["main"])); + + let create = BranchCreateRequest { + from: Some("main".to_string()), + name: "feature".to_string(), + }; + let (create_status, create_body) = json_response( + &app, + Request::builder() + .uri("/branches") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&create).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(create_status, StatusCode::OK); + assert_eq!(create_body["from"], "main"); + assert_eq!(create_body["name"], "feature"); + + let (list_status, list_body) = json_response( + &app, + Request::builder() + .uri("/branches") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(list_status, StatusCode::OK); + assert_eq!(list_body["branches"], json!(["feature", "main"])); + + let change = ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": "Zoe", "age": 33 })), + branch: Some("feature".to_string()), + }; + let (change_status, change_body) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&change).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(change_status, StatusCode::OK); + assert_eq!(change_body["branch"], "feature"); + assert_eq!(change_body["affected_nodes"], 1); + + let read_main_before = ReadRequest { + query_source: fs::read_to_string(fixture("test.gq")).unwrap(), + query_name: Some("get_person".to_string()), + params: Some(json!({ "name": "Zoe" })), + branch: Some("main".to_string()), + snapshot: None, + }; + let (read_status, read_body) = json_response( + &app, + Request::builder() + .uri("/read") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&read_main_before).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(read_status, StatusCode::OK); + assert_eq!(read_body["row_count"], 0); + + let merge = BranchMergeRequest { + source: "feature".to_string(), + target: Some("main".to_string()), + }; + let (merge_status, merge_body) = json_response( + &app, + Request::builder() + .uri("/branches/merge") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&merge).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(merge_status, StatusCode::OK); + assert_eq!(merge_body["source"], "feature"); + assert_eq!(merge_body["target"], "main"); + assert_eq!(merge_body["outcome"], "fast_forward"); + + let read_main_after = ReadRequest { + query_source: fs::read_to_string(fixture("test.gq")).unwrap(), + query_name: Some("get_person".to_string()), + params: Some(json!({ "name": "Zoe" })), + branch: Some("main".to_string()), + snapshot: None, + }; + let (read_status, read_body) = json_response( + &app, + Request::builder() + .uri("/read") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&read_main_after).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(read_status, StatusCode::OK); + assert_eq!(read_body["row_count"], 1); + assert_eq!(read_body["rows"][0]["p.name"], "Zoe"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn remote_branch_delete_flow_works() { + let (_temp, app) = app_for_loaded_graph().await; + + let create = BranchCreateRequest { + from: Some("main".to_string()), + name: "feature".to_string(), + }; + let (create_status, _) = json_response( + &app, + Request::builder() + .uri("/branches") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&create).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(create_status, StatusCode::OK); + + let (delete_status, delete_body) = json_response( + &app, + Request::builder() + .uri("/branches/feature") + .method(Method::DELETE) + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(delete_status, StatusCode::OK); + assert_eq!(delete_body["name"], "feature"); + + let (list_status, list_body) = json_response( + &app, + Request::builder() + .uri("/branches") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(list_status, StatusCode::OK); + assert_eq!(list_body["branches"], json!(["main"])); +} + +#[tokio::test(flavor = "multi_thread")] +async fn branch_delete_denies_without_policy_permission() { + let (temp, app) = app_for_loaded_graph_with_auth_tokens_and_policy( + &[("act-andrew", "token-admin"), ("act-bruno", "token-team")], + POLICY_YAML, + ) + .await; + let graph = graph_path(temp.path()); + + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.branch_create_from(ReadTarget::branch("main"), "feature") + .await + .unwrap(); + drop(db); + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/branches/feature") + .method(Method::DELETE) + .header("authorization", "Bearer token-team") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::FORBIDDEN); + assert!( + body["error"] + .as_str() + .unwrap() + .contains("policy denied action 'branch_delete'") + ); +} + +#[tokio::test(flavor = "multi_thread")] +#[serial] +async fn remote_read_embeds_string_nearest_queries_with_mock_runtime() { + const EMBED_SCHEMA: &str = r#" +node Doc { + slug: String @key + title: String @index + embedding: Vector(4) @index +} +"#; + const EMBED_QUERY: &str = r#" +query vector_search_string($q: String) { + match { $d: Doc } + return { $d.slug, $d.title } + order { nearest($d.embedding, $q) } + limit 3 +} +"#; + + let alpha = mock_embedding("alpha", 4); + let beta = mock_embedding("beta", 4); + let gamma = mock_embedding("gamma", 4); + let data = format!( + concat!( + r#"{{"type":"Doc","data":{{"slug":"alpha-doc","title":"alpha guide","embedding":[{}]}}}}"#, + "\n", + r#"{{"type":"Doc","data":{{"slug":"beta-doc","title":"beta guide","embedding":[{}]}}}}"#, + "\n", + r#"{{"type":"Doc","data":{{"slug":"gamma-doc","title":"gamma handbook","embedding":[{}]}}}}"# + ), + format_vector(&alpha), + format_vector(&beta), + format_vector(&gamma), + ); + + let _guard = EnvGuard::set(&[ + ("OMNIGRAPH_EMBEDDINGS_MOCK", Some("1")), + ("GEMINI_API_KEY", None), + ]); + let temp = init_graph_with_schema_and_data(EMBED_SCHEMA, &data).await; + let graph = graph_path(temp.path()); + let state = AppState::open(graph.to_string_lossy().to_string()) + .await + .unwrap(); + let app = build_app(state); + + let read = ReadRequest { + query_source: EMBED_QUERY.to_string(), + query_name: Some("vector_search_string".to_string()), + params: Some(json!({ "q": "alpha" })), + branch: Some("main".to_string()), + snapshot: None, + }; + let (status, body) = json_response( + &app, + Request::builder() + .uri("/read") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&read).unwrap())) + .unwrap(), + ) + .await; + + assert_eq!(status, StatusCode::OK); + assert_eq!(body["row_count"], 3); + assert_eq!(body["rows"][0]["d.slug"], "alpha-doc"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn change_conflict_returns_manifest_conflict_409() { + // A write that races with another writer surfaces as HTTP 409 with + // a structured `manifest_conflict` body — `table_key`, `expected`, + // and `actual` — so clients can detect-and-retry without parsing + // the message. + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + + // Build the server first so its handle pins the pre-mutation manifest + // version. Then advance the manifest from outside the server. The + // server's next /change call will capture stale `expected_versions` + // (from its still-pinned snapshot) and the publisher's CAS rejects. + let state = AppState::open(graph.to_string_lossy().to_string()) + .await + .unwrap(); + let app = build_app(state); + + { + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.mutate( + "main", + MUTATION_QUERIES, + "set_age", + &omnigraph_compiler::json_params_to_param_map( + Some(&json!({"name": "Alice", "age": 31 })), + &omnigraph_compiler::find_named_query(MUTATION_QUERIES, "set_age") + .unwrap() + .params, + omnigraph_compiler::JsonParamMode::Standard, + ) + .unwrap(), + ) + .await + .unwrap(); + } + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from( + serde_json::to_vec(&ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("set_age".to_string()), + params: Some(json!({ "name": "Alice", "age": 33 })), + branch: Some("main".to_string()), + }) + .unwrap(), + )) + .unwrap(), + ) + .await; + + assert_eq!(status, StatusCode::CONFLICT); + let error: ErrorOutput = serde_json::from_value(body).unwrap(); + assert_eq!(error.code, Some(omnigraph_server::api::ErrorCode::Conflict)); + let conflict = error + .manifest_conflict + .expect("publisher CAS rejection must populate manifest_conflict body"); + assert_eq!(conflict.table_key, "node:Person"); + assert!( + conflict.actual > conflict.expected, + "actual ({}) should be ahead of expected ({})", + conflict.actual, + conflict.expected, + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn change_concurrent_inserts_same_key_serialize_without_409() { + // PR 2 Phase 2 (MR-686): pin the design fix for the same-key + // concurrency hazard. Pre-fix, in-process concurrent inserts on + // the same `(table, branch)` rejected with 409 manifest_conflict + // because `ensure_expected_version` fired before the per-table + // queue was acquired and saw Lance HEAD already advanced by a + // peer writer. Post-fix, Insert/Merge skip the strict pre-stage + // check (see `MutationOpKind::strict_pre_stage_version_check`); + // the queue serializes commit_staged; Lance's natural rebase + // handles the in-flight stage; the publisher's CAS on a fresh + // per-branch snapshot under the queue catches genuine cross- + // process drift. + // + // This test spawns N concurrent /change inserts on a single + // node type and asserts: every request returns 200 (no 409), + // and the final row count equals the seed count + N (every + // staged batch actually committed). + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let state = AppState::open(graph.to_string_lossy().to_string()) + .await + .unwrap(); + let app = build_app(state); + + // test.jsonl seeds 4 Persons (Alice, Bob, Charlie, Diana). + const SEED_PERSON_ROWS: u64 = 4; + const N: usize = 12; + + let mut handles = Vec::with_capacity(N); + for i in 0..N { + let app = app.clone(); + handles.push(tokio::spawn(async move { + let body = serde_json::to_vec(&ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": format!("racer-{i}"), "age": i as i32 })), + branch: Some("main".to_string()), + }) + .unwrap(); + let req = Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(); + let response = app.oneshot(req).await.unwrap(); + response.status() + })); + } + + let mut statuses = Vec::with_capacity(N); + for h in handles { + statuses.push(h.await.unwrap()); + } + + let bad: Vec<_> = statuses + .iter() + .enumerate() + .filter(|(_, s)| **s != StatusCode::OK) + .collect(); + assert!( + bad.is_empty(), + "expected every concurrent insert to return 200, got non-200 for: {:?}", + bad + ); + + // Verify the inserts actually landed. The status check above only proves + // the publisher CAS didn't reject; the row count proves none of the + // concurrent commits silently overwrote a peer. + let (snapshot_status, snapshot_body) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(snapshot_status, StatusCode::OK); + let person_rows = snapshot_body["tables"] + .as_array() + .and_then(|tables| { + tables + .iter() + .find(|t| t["table_key"].as_str() == Some("node:Person")) + }) + .and_then(|t| t["row_count"].as_u64()) + .expect("snapshot must include node:Person row_count"); + assert_eq!( + person_rows, + SEED_PERSON_ROWS + N as u64, + "expected {} seeded + {} concurrent inserts = {} Person rows; got {}", + SEED_PERSON_ROWS, + N, + SEED_PERSON_ROWS + N as u64, + person_rows, + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn change_concurrent_updates_same_key_serialize_via_publisher_cas() { + // Pin Update RYW semantics under in-process concurrency on the same + // `(table, branch)`. With per-table queue serialization and op-kind-aware + // drift detection at commit time, exactly one of N concurrent UPDATEs + // on the same row commits; the rest are rejected as 409 manifest_conflict. + // + // Pre-fix bug class: in `MutationStaging::commit_all`, after queue + // acquisition, the staged Lance transaction is handed straight to + // `commit_staged`. For a writer whose staged dataset is at V0 but + // Lance HEAD has advanced to V1 (because the queue's prior winner + // already published), Lance's transaction conflict resolver fires + // `RetryableCommitConflict` on Update vs Update on the same row. + // That error gets wrapped as `OmniError::Lance(<string>)` and the + // API surfaces it as **500 internal**, not 409. Users see "internal + // server error" instead of a retryable conflict, breaking the + // documented 409 contract for in-process drift. + // + // Post-fix invariant: `commit_all` does an op-kind-aware drift check + // before each `commit_staged`. For tables whose tracked op_kind has + // `strict_pre_stage_version_check() == true` (Update / Delete / + // SchemaRewrite), if the staged dataset's version doesn't match the + // fresh manifest pin, return `OmniError::manifest_expected_version_mismatch` + // → 409 ExpectedVersionMismatch. The N-1 losers see a clean 409 + // before Lance's commit_staged ever runs. + // + // Why correct-by-design: closing the class "Lance internal conflict + // surfaces as 500 instead of 409" rather than mapping the specific + // Lance error variant. The drift check fires at the right architectural + // layer (engine boundary, under the queue) and respects the existing + // `MutationOpKind` policy. + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let state = AppState::open(graph.to_string_lossy().to_string()) + .await + .unwrap(); + let app = build_app(state); + + // Spawn N=8 concurrent UPDATEs on Alice (from test.jsonl, age=30 at V0) + // writing distinct ages. + const N: usize = 8; + let mut handles = Vec::with_capacity(N); + for i in 0..N { + let app = app.clone(); + let target_age = 100 + i as i32; + handles.push(tokio::spawn(async move { + let body = serde_json::to_vec(&ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("set_age".to_string()), + params: Some(json!({ "name": "Alice", "age": target_age })), + branch: Some("main".to_string()), + }) + .unwrap(); + let req = Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(); + let response = app.oneshot(req).await.unwrap(); + let status = response.status(); + let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + (status, body.to_vec()) + })); + } + + let mut results = Vec::with_capacity(N); + for h in handles { + results.push(h.await.unwrap()); + } + let statuses: Vec<StatusCode> = results.iter().map(|(s, _)| *s).collect(); + + let ok_count = statuses.iter().filter(|s| **s == StatusCode::OK).count(); + let conflict_count = statuses + .iter() + .filter(|s| **s == StatusCode::CONFLICT) + .count(); + let other: Vec<_> = statuses + .iter() + .enumerate() + .filter(|(_, s)| **s != StatusCode::OK && **s != StatusCode::CONFLICT) + .collect(); + + let other_bodies: Vec<(usize, StatusCode, String)> = other + .iter() + .map(|(i, s)| { + let body_str = String::from_utf8_lossy(&results[*i].1).to_string(); + (*i, **s, body_str) + }) + .collect(); + assert!( + other.is_empty(), + "expected only 200 or 409 statuses, got non-200/409 entries: {:?}", + other_bodies + ); + assert_eq!( + ok_count + conflict_count, + N, + "all responses must be 200 or 409 to satisfy the RYW invariant; statuses: {:?}", + statuses + ); + assert_eq!( + ok_count, + 1, + "expected exactly one update to commit and N-1 to receive 409 manifest_conflict \ + (op-kind-aware drift check rejects stale-V0 staged datasets at commit_all entry). \ + Got {} OK + {} 409 + {} other. \ + Pre-fix symptom: 1 OK + (N-1) x 500 because Lance's RetryableCommitConflict for \ + Update vs Update on the same row bubbles up as `OmniError::Lance(<string>)` and \ + the API maps it to 500 internal, not 409. Statuses: {:?}", + ok_count, + conflict_count, + statuses.len() - ok_count - conflict_count, + statuses, + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn change_disjoint_table_concurrency_succeeds_at_http_level() { + // HTTP-level pin for MR-686's disjoint-table promise: concurrent /change + // requests touching different node types must coexist without admission + // rejection or publisher-CAS conflict. The bench harness measures + // throughput; this test is the regression sentinel that catches a + // future change which accidentally re-introduces graph-wide + // serialization on the disjoint path. + // + // Setup: test.jsonl seeds 4 Persons + 2 Companies. Spawn N=4 concurrent + // /change inserts on `node:Person` and N=4 concurrent inserts on + // `node:Company`. All 8 must return 200, and the post-test row counts + // must reflect every insert. + const PERSON_QUERY: &str = r#" +query insert_p($name: String, $age: I32) { + insert Person { name: $name, age: $age } +} +"#; + const COMPANY_QUERY: &str = r#" +query insert_c($name: String) { + insert Company { name: $name } +} +"#; + const SEED_PERSONS: u64 = 4; + const SEED_COMPANIES: u64 = 2; + const PER_TYPE: usize = 4; + + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let state = AppState::open(graph.to_string_lossy().to_string()) + .await + .unwrap(); + let app = build_app(state); + + let mut handles = Vec::with_capacity(PER_TYPE * 2); + for i in 0..PER_TYPE { + let app_p = app.clone(); + handles.push(tokio::spawn(async move { + let body = serde_json::to_vec(&ChangeRequest { + query: PERSON_QUERY.to_string(), + name: Some("insert_p".to_string()), + params: Some(json!({ "name": format!("p-{i}"), "age": i as i32 })), + branch: Some("main".to_string()), + }) + .unwrap(); + let req = Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(); + app_p.oneshot(req).await.unwrap().status() + })); + let app_c = app.clone(); + handles.push(tokio::spawn(async move { + let body = serde_json::to_vec(&ChangeRequest { + query: COMPANY_QUERY.to_string(), + name: Some("insert_c".to_string()), + params: Some(json!({ "name": format!("c-{i}") })), + branch: Some("main".to_string()), + }) + .unwrap(); + let req = Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(); + app_c.oneshot(req).await.unwrap().status() + })); + } + + let mut statuses = Vec::with_capacity(PER_TYPE * 2); + for h in handles { + statuses.push(h.await.unwrap()); + } + + let bad: Vec<_> = statuses + .iter() + .enumerate() + .filter(|(_, s)| **s != StatusCode::OK) + .collect(); + assert!( + bad.is_empty(), + "expected every disjoint /change insert to return 200, got non-200 for: {:?}", + bad, + ); + + // Verify both tables landed every insert. + let (status, body) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + let lookup_count = |table_key: &str| -> u64 { + body["tables"] + .as_array() + .and_then(|tables| { + tables + .iter() + .find(|t| t["table_key"].as_str() == Some(table_key)) + }) + .and_then(|t| t["row_count"].as_u64()) + .unwrap_or_else(|| panic!("snapshot missing {}", table_key)) + }; + assert_eq!( + lookup_count("node:Person"), + SEED_PERSONS + PER_TYPE as u64, + "Person row count after concurrent inserts", + ); + assert_eq!( + lookup_count("node:Company"), + SEED_COMPANIES + PER_TYPE as u64, + "Company row count after concurrent inserts", + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn ingest_per_actor_admission_cap_returns_429() { + // Pin the admission gate on `/ingest`. With per-actor in-flight cap of 1 + // and 8 concurrent requests from the same actor, at least one request + // must be rejected with HTTP 429 and `code: too_many_requests`. + // + // Pre-fix bug class: the admission pattern at `server_change` + // (`crates/omnigraph-server/src/lib.rs:932`) was the only handler + // that called `WorkloadController::try_admit`. A heavy actor sending + // bulk-ingest traffic would exhaust shared engine capacity (Lance I/O + // threads, manifest churn) without ever hitting an admission cap. + // Pinned at the HTTP boundary so future refactors that drop the + // try_admit call from a mutating handler turn this red. + // + // Post-fix invariant: `/ingest`, `/branches/create`, `/branches/delete`, + // `/branches/merge`, and `/schema/apply` all gate on + // `state.workload.try_admit(&actor_arc, est_bytes)` after Cedar + // authorization and before the engine call. Cap exhaustion surfaces as + // 429 with `code: too_many_requests`. + // + // Construct the WorkloadController directly with cap=1 instead of + // mutating `OMNIGRAPH_PER_ACTOR_INFLIGHT_MAX` via EnvGuard. Process-wide + // env vars are visible to concurrently-running tests; the previous + // `EnvGuard + #[serial]` pair leaked the override into any other test + // that called `AppState::open` during the guard's window + // (matrix CI failure on commit 99b0941). Using the explicit + // `AppState::new_with_workload` constructor closes that bug class — + // this test no longer mutates global state and no longer needs + // `#[serial]`. + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + let workload = omnigraph_server::workload::WorkloadController::new( + 1, // per-actor in-flight cap (the fixture under test) + 1_000_000_000, // per-actor byte budget — large so it never bottlenecks + ); + // MR-723: install a permit-all policy alongside the bearer token so + // /ingest (action=Change) passes Cedar evaluation. The test is + // exercising the admission cap, not policy — the policy is just + // enough to clear the State 3 path so the test reaches workload. + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, permit_all_policy_yaml(&["act-flooder"])).unwrap(); + let policy_engine = + omnigraph_server::PolicyEngine::load_graph(&policy_path, graph.to_string_lossy().as_ref()) + .unwrap(); + let state = AppState::new_single( + graph.to_string_lossy().to_string(), + db, + vec![("act-flooder".to_string(), "flooder-token".to_string())], + Some(policy_engine), + workload, + ); + let app = build_app(state); + let _temp = temp; + + // Eight concurrent ingests, all from act-flooder. Only one fits in a + // cap=1 in-flight semaphore; the others must 429. + const N: usize = 8; + let barrier = Arc::new(tokio::sync::Barrier::new(N)); + let mut handles = Vec::with_capacity(N); + for i in 0..N { + let app = app.clone(); + let barrier = Arc::clone(&barrier); + handles.push(tokio::spawn(async move { + // Align the 8 tasks at the barrier so they all attempt + // try_admit close in time. + barrier.wait().await; + + let body = serde_json::to_vec(&IngestRequest { + data: format!( + "{{\"type\":\"Person\",\"data\":{{\"name\":\"flooder-{i}\",\"age\":{i}}}}}\n" + ), + branch: Some("main".to_string()), + from: Some("main".to_string()), + mode: Some(omnigraph::loader::LoadMode::Merge), + }) + .unwrap(); + let req = Request::builder() + .uri("/ingest") + .method(Method::POST) + .header("authorization", "Bearer flooder-token") + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(); + let response = app.oneshot(req).await.unwrap(); + let status = response.status(); + let headers = response.headers().clone(); + let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + (status, headers, body.to_vec()) + })); + } + + let mut results = Vec::with_capacity(N); + for h in handles { + results.push(h.await.unwrap()); + } + let statuses: Vec<StatusCode> = results.iter().map(|(s, _, _)| *s).collect(); + + let too_many: Vec<usize> = statuses + .iter() + .enumerate() + .filter(|(_, s)| **s == StatusCode::TOO_MANY_REQUESTS) + .map(|(i, _)| i) + .collect(); + assert!( + !too_many.is_empty(), + "expected at least one /ingest under cap=1 to return 429; got statuses: {:?}", + statuses, + ); + + // Validate the structured error body for each 429 (body must carry + // the `too_many_requests` code so clients can distinguish it from + // generic conflicts). + for i in &too_many { + let body_value: Value = serde_json::from_slice(&results[*i].2).unwrap(); + let error: ErrorOutput = serde_json::from_value(body_value).unwrap(); + assert_eq!( + error.code, + Some(omnigraph_server::api::ErrorCode::TooManyRequests), + "429 body must carry code=too_many_requests; idx {} got {:?}", + i, + error.code, + ); + } + + // Validate the `Retry-After` header is set on every 429. Pinned by + // the same test so a future refactor that drops the header from + // `IntoResponse for ApiError` turns this red. The constant + // matches `crates/omnigraph-server/src/lib.rs::ApiError::into_response`. + for i in &too_many { + let retry_after = results[*i] + .1 + .get(axum::http::header::RETRY_AFTER) + .and_then(|v| v.to_str().ok()) + .map(str::to_string); + assert!( + retry_after.is_some(), + "429 response must include a Retry-After header; idx {} headers were: {:?}", + i, + results[*i].1, + ); + } +} diff --git a/crates/omnigraph-server/tests/multi_graph.rs b/crates/omnigraph-server/tests/multi_graph.rs new file mode 100644 index 0000000..251f899 --- /dev/null +++ b/crates/omnigraph-server/tests/multi_graph.rs @@ -0,0 +1,584 @@ +//! Cluster-mode boot and the concurrent branch-ops matrix. +//! Moved verbatim from tests/server.rs in the modularization. + +use std::fs; + +use axum::body::{Body, to_bytes}; +use axum::http::{Method, Request, StatusCode}; +use omnigraph_server::api::ErrorOutput; +use omnigraph_server::{AppState, build_app}; +use serde_json::Value; +use tower::ServiceExt; + + +mod support; +use support::*; + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn concurrent_branch_ops_morphological_matrix() { + // Cell a: Merge × Merge, distinct targets. + // Pre-fix on b09a097/22d76db: branch_merge_impl's swap-restore race + // landed feature_a's content in target_b instead of target_a (and + // vice versa — symmetric swap). Identity asserts catch both + // asymmetric and symmetric variants. + { + let cell = "a:merge×merge:distinct-targets"; + let h = matrix::Harness::new().await; + h.create_branch("main", "feature-a-cella").await; + h.insert_person("feature-a-cella", "EveA-cella", 22).await; + h.create_branch("main", "feature-b-cella").await; + h.insert_person("feature-b-cella", "FrankB-cella", 33).await; + h.create_branch("main", "target-a-cella").await; + h.create_branch("main", "target-b-cella").await; + + let (sa, sb) = h + .run_pair( + matrix::op_merge("feature-a-cella".to_string(), "target-a-cella".to_string()), + matrix::op_merge("feature-b-cella".to_string(), "target-b-cella".to_string()), + ) + .await; + assert_eq!(sa.status, StatusCode::OK, "[{}] merge a", cell); + assert_eq!(sb.status, StatusCode::OK, "[{}] merge b", cell); + h.assert_persons("target-a-cella", cell, &["EveA-cella"], &["FrankB-cella"]) + .await; + h.assert_persons("target-b-cella", cell, &["FrankB-cella"], &["EveA-cella"]) + .await; + h.assert_post_op_sentinel(cell, "sentinel-cella").await; + } + + // Cell b: Merge × Merge, same target / distinct sources. + // Both want to land in main. merge_exclusive serializes; both should + // succeed and main should contain BOTH sources' contributions. + { + let cell = "b:merge×merge:same-target-distinct-sources"; + let h = matrix::Harness::new().await; + h.create_branch("main", "src-x-cellb").await; + h.insert_person("src-x-cellb", "Xavier-cellb", 41).await; + h.create_branch("main", "src-y-cellb").await; + h.insert_person("src-y-cellb", "Yvonne-cellb", 42).await; + + let (sa, sb) = h + .run_pair( + matrix::op_merge("src-x-cellb".to_string(), "main".to_string()), + matrix::op_merge("src-y-cellb".to_string(), "main".to_string()), + ) + .await; + assert_eq!(sa.status, StatusCode::OK, "[{}] merge x", cell); + assert_eq!(sb.status, StatusCode::OK, "[{}] merge y", cell); + h.assert_persons("main", cell, &["Xavier-cellb", "Yvonne-cellb"], &[]) + .await; + h.assert_post_op_sentinel(cell, "sentinel-cellb").await; + } + + // Cell c: Merge × Merge, same source / distinct targets (fanout). + // One source merged into two targets simultaneously. merge_exclusive + // serializes; both targets should reflect the source's content. + { + let cell = "c:merge×merge:same-source-distinct-targets"; + let h = matrix::Harness::new().await; + h.create_branch("main", "src-shared-cellc").await; + h.insert_person("src-shared-cellc", "Sharon-cellc", 50) + .await; + h.create_branch("main", "tgt-1-cellc").await; + h.create_branch("main", "tgt-2-cellc").await; + + let (sa, sb) = h + .run_pair( + matrix::op_merge("src-shared-cellc".to_string(), "tgt-1-cellc".to_string()), + matrix::op_merge("src-shared-cellc".to_string(), "tgt-2-cellc".to_string()), + ) + .await; + assert_eq!(sa.status, StatusCode::OK, "[{}] merge into tgt-1", cell); + assert_eq!(sb.status, StatusCode::OK, "[{}] merge into tgt-2", cell); + h.assert_persons("tgt-1-cellc", cell, &["Sharon-cellc"], &[]) + .await; + h.assert_persons("tgt-2-cellc", cell, &["Sharon-cellc"], &[]) + .await; + h.assert_post_op_sentinel(cell, "sentinel-cellc").await; + } + + // Cell d: Merge × Change, both touching main. C2 permits both + // succeed, or exactly one clean 409 if the merge detects target + // movement after planning but before acquiring the queue. + { + let cell = "d:merge×change:into-target"; + let h = matrix::Harness::new().await; + h.create_branch("main", "feature-celld").await; + h.insert_person("feature-celld", "EveD-celld", 22).await; + + let (sa, sb) = h + .run_pair( + matrix::op_merge("feature-celld".to_string(), "main".to_string()), + matrix::op_change_insert("main".to_string(), "FrankD-celld".to_string(), 33), + ) + .await; + assert_eq!(sb.status, StatusCode::OK, "[{}] change", cell); + assert!( + sa.status == StatusCode::OK || sa.status == StatusCode::CONFLICT, + "[{}] merge must be 200 or clean 409, got {}", + cell, + sa.status + ); + if sa.status == StatusCode::OK { + h.assert_persons("main", cell, &["EveD-celld", "FrankD-celld"], &[]) + .await; + } else { + let error: ErrorOutput = serde_json::from_slice(&sa.body).unwrap(); + let conflict = error + .manifest_conflict + .expect("merge 409 must include manifest_conflict"); + assert_eq!( + conflict.table_key, "node:Person", + "[{}] conflict table", + cell + ); + h.assert_persons("main", cell, &["FrankD-celld"], &["EveD-celld"]) + .await; + } + h.assert_post_op_sentinel(cell, "sentinel-celld").await; + } + + // Cell e: Merge × BranchCreateFrom-target. Concurrent fork off the + // merge target while the merge runs. Both should succeed; the new + // branch should have a coherent view (either pre- or post-merge, + // both valid). After both, target = main has the merged content. + { + let cell = "e:merge×branch_create_from:target"; + let h = matrix::Harness::new().await; + h.create_branch("main", "src-celle").await; + h.insert_person("src-celle", "Eve-celle", 22).await; + + let (sa, sb) = h + .run_pair( + matrix::op_merge("src-celle".to_string(), "main".to_string()), + matrix::op_branch_create("main".to_string(), "fork-celle".to_string()), + ) + .await; + assert_eq!(sa.status, StatusCode::OK, "[{}] merge", cell); + assert_eq!(sb.status, StatusCode::OK, "[{}] branch_create_from", cell); + // Main definitely has Eve. + h.assert_persons("main", cell, &["Eve-celle"], &[]).await; + // fork-celle was forked off main at SOME version; main's current + // count is 5 (4 seeded + Eve). fork-celle has either 4 (pre-merge + // snapshot) or 5 (post-merge snapshot); both are valid timings. + let fork_count = h.person_count("fork-celle").await; + assert!( + fork_count == 4 || fork_count == 5, + "[{}] fork-celle row count must be pre- or post-merge view (4 or 5), got {}", + cell, + fork_count + ); + h.assert_post_op_sentinel(cell, "sentinel-celle").await; + } + + // Cell f: BranchCreateFrom × BranchCreateFrom, distinct parents. + // Pre-fix on f925ad1: swap-restore race in branch_create_from_impl + // forked the new branch off the wrong parent. Identity asserts pin + // that fork-from-A inherits A's content, fork-from-B inherits B's. + { + let cell = "f:branch_create_from×branch_create_from:distinct-parents"; + let h = matrix::Harness::new().await; + h.create_branch("main", "alpha-cellf").await; + h.insert_person("alpha-cellf", "Eve-cellf", 22).await; + h.create_branch("main", "beta-cellf").await; + + let (sa, sb) = h + .run_pair( + matrix::op_branch_create("alpha-cellf".to_string(), "gamma-cellf".to_string()), + matrix::op_branch_create("beta-cellf".to_string(), "delta-cellf".to_string()), + ) + .await; + assert_eq!(sa.status, StatusCode::OK, "[{}] gamma create", cell); + assert_eq!(sb.status, StatusCode::OK, "[{}] delta create", cell); + // gamma forks off alpha → must contain Eve. + h.assert_persons("gamma-cellf", cell, &["Eve-cellf"], &[]) + .await; + // delta forks off beta → must NOT contain Eve. + h.assert_persons("delta-cellf", cell, &[], &["Eve-cellf"]) + .await; + h.assert_post_op_sentinel(cell, "sentinel-cellf").await; + } + + // Cell g: BranchCreateFrom × BranchDelete, unrelated branches. + // Disjoint branches; both should complete cleanly without + // interference. + { + let cell = "g:branch_create_from×branch_delete:unrelated"; + let h = matrix::Harness::new().await; + h.create_branch("main", "doomed-cellg").await; + + let (sa, sb) = h + .run_pair( + matrix::op_branch_create("main".to_string(), "newborn-cellg".to_string()), + matrix::op_branch_delete("doomed-cellg".to_string()), + ) + .await; + assert_eq!(sa.status, StatusCode::OK, "[{}] create newborn", cell); + assert_eq!(sb.status, StatusCode::OK, "[{}] delete doomed", cell); + // newborn-cellg exists with main's content. + h.assert_persons("newborn-cellg", cell, &["Alice"], &[]) + .await; + h.assert_post_op_sentinel(cell, "sentinel-cellg").await; + } + + // Cell h: BranchDelete × BranchDelete, distinct branches. Both call + // refresh() internally; verify no deadlock and both deletes land. + { + let cell = "h:branch_delete×branch_delete:distinct"; + let h = matrix::Harness::new().await; + h.create_branch("main", "doomed1-cellh").await; + h.create_branch("main", "doomed2-cellh").await; + + let (sa, sb) = h + .run_pair( + matrix::op_branch_delete("doomed1-cellh".to_string()), + matrix::op_branch_delete("doomed2-cellh".to_string()), + ) + .await; + assert_eq!(sa.status, StatusCode::OK, "[{}] delete 1", cell); + assert_eq!(sb.status, StatusCode::OK, "[{}] delete 2", cell); + // Verify both gone via /branches list (snapshot would still work + // for a deleted branch via parent fallback in some paths, so we + // use the explicit list). + let r = h + .app + .clone() + .oneshot( + Request::builder() + .uri("/branches") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(r.status(), StatusCode::OK); + let body = to_bytes(r.into_body(), usize::MAX).await.unwrap(); + let list_body: Value = serde_json::from_slice(&body).unwrap(); + let branches: Vec<&str> = list_body["branches"] + .as_array() + .unwrap() + .iter() + .filter_map(|v| v.as_str()) + .collect(); + assert!( + !branches.contains(&"doomed1-cellh"), + "[{}] doomed1 still in branch list: {:?}", + cell, + branches + ); + assert!( + !branches.contains(&"doomed2-cellh"), + "[{}] doomed2 still in branch list: {:?}", + cell, + branches + ); + h.assert_post_op_sentinel(cell, "sentinel-cellh").await; + } + + // Cell i: BranchDelete × Change, on a different branch. Delete one + // branch while a /change runs on main. Both should succeed. + { + let cell = "i:branch_delete×change:distinct-branch"; + let h = matrix::Harness::new().await; + h.create_branch("main", "doomed-celli").await; + + let (sa, sb) = h + .run_pair( + matrix::op_branch_delete("doomed-celli".to_string()), + matrix::op_change_insert("main".to_string(), "Pat-celli".to_string(), 44), + ) + .await; + assert_eq!(sa.status, StatusCode::OK, "[{}] delete", cell); + assert_eq!(sb.status, StatusCode::OK, "[{}] change", cell); + h.assert_persons("main", cell, &["Pat-celli"], &[]).await; + h.assert_post_op_sentinel(cell, "sentinel-celli").await; + } + + // Cell j: BranchCreateFrom × Change, both on main. The fork timing + // determines whether the new branch sees the change (pre or post). + // Both valid. Main must contain the inserted row. + { + let cell = "j:branch_create_from×change:on-source"; + let h = matrix::Harness::new().await; + + let (sa, sb) = h + .run_pair( + matrix::op_branch_create("main".to_string(), "twin-cellj".to_string()), + matrix::op_change_insert("main".to_string(), "Quincy-cellj".to_string(), 55), + ) + .await; + assert_eq!(sa.status, StatusCode::OK, "[{}] branch_create", cell); + assert_eq!(sb.status, StatusCode::OK, "[{}] change", cell); + h.assert_persons("main", cell, &["Quincy-cellj"], &[]).await; + // twin-cellj has either pre-change view (no Quincy) or + // post-change view (with Quincy); either is valid. + let twin_has_quincy = h.person_exists("twin-cellj", "Quincy-cellj").await; + let _ = twin_has_quincy; // either valid timing — just ensure no panic + h.assert_post_op_sentinel(cell, "sentinel-cellj").await; + } + + // Cell k: reopen consistency. Run a representative concurrent pair, + // drop the engine, reopen on a separate handle, verify state matches. + { + let cell = "k:reopen-after-pair"; + let h = matrix::Harness::new().await; + h.create_branch("main", "src-cellk").await; + h.insert_person("src-cellk", "Rita-cellk", 36).await; + + let (sa, sb) = h + .run_pair( + matrix::op_merge("src-cellk".to_string(), "main".to_string()), + matrix::op_change_insert("main".to_string(), "Steve-cellk".to_string(), 37), + ) + .await; + assert_eq!(sb.status, StatusCode::OK, "[{}] change", cell); + assert!( + sa.status == StatusCode::OK || sa.status == StatusCode::CONFLICT, + "[{}] merge must be 200 or clean 409, got {}", + cell, + sa.status + ); + if sa.status == StatusCode::OK { + h.assert_persons("main", cell, &["Rita-cellk", "Steve-cellk"], &[]) + .await; + } else { + let error: ErrorOutput = serde_json::from_slice(&sa.body).unwrap(); + let conflict = error + .manifest_conflict + .expect("merge 409 must include manifest_conflict"); + assert_eq!( + conflict.table_key, "node:Person", + "[{}] conflict table", + cell + ); + h.assert_persons("main", cell, &["Steve-cellk"], &["Rita-cellk"]) + .await; + } + + // Reopen via a fresh AppState on the same graph. + let graph_uri = format!("{}/server.omni", h._temp.path().display()); + let reopened = AppState::open(graph_uri.clone()).await.unwrap(); + let app2 = build_app(reopened); + // Sanity: the same identity check via the new app must see + // Rita and Steve. + let r = app2 + .clone() + .oneshot( + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(r.status(), StatusCode::OK, "[{}] reopen snapshot", cell); + let body = to_bytes(r.into_body(), usize::MAX).await.unwrap(); + let v: Value = serde_json::from_slice(&body).unwrap(); + let person_rows = v["tables"] + .as_array() + .and_then(|tables| { + tables + .iter() + .find(|t| t["table_key"].as_str() == Some("node:Person")) + }) + .and_then(|t| t["row_count"].as_u64()) + .expect("reopen snapshot must include node:Person row_count"); + let expected_rows = if sa.status == StatusCode::OK { 6 } else { 5 }; + assert_eq!( + person_rows, expected_rows, + "[{}] reopened main should include seed (4) + committed concurrent writes", + cell, + ); + } +} + +#[tokio::test] +async fn cluster_boot_serves_applied_state() { + let temp = converged_cluster_dir("").await; + let settings = cluster_settings(temp.path()).await.unwrap(); + let omnigraph_server::ServerConfigMode::Multi { + graphs, + config_path, + server_policy_file, + } = settings.mode + else { + panic!("cluster boot must select multi-graph routing"); + }; + assert_eq!(graphs.len(), 1); + assert_eq!(graphs[0].graph_id, "knowledge"); + assert!(server_policy_file.is_none()); + + let state = + omnigraph_server::open_multi_graph_state(graphs, Vec::new(), None, config_path) + .await + .unwrap(); + let app = build_app(state); + + // The management surface keeps its closed-by-default contract: without a + // cluster-scoped policy bundle there is no server-level Cedar engine, so + // GET /graphs refuses even in cluster mode. + let (status, body) = json_response( + &app, + Request::builder().uri("/graphs").body(Body::empty()).unwrap(), + ) + .await; + assert_eq!(status, StatusCode::FORBIDDEN, "{body}"); + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/graphs/knowledge/queries") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK, "{body}"); + assert!( + body["queries"] + .as_array() + .unwrap() + .iter() + .any(|q| q["name"] == "find_person"), + "{body}" + ); + + let (status, body) = json_response( + &app, + Request::builder() + .method(Method::POST) + .uri("/graphs/knowledge/queries/find_person") + .header("content-type", "application/json") + .body(Body::from(r#"{"params":{"name":"nobody"}}"#)) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK, "{body}"); +} + +#[tokio::test] +async fn cluster_boot_wires_policy_bindings_into_cedar_slots() { + let temp = tempfile::tempdir().unwrap(); + drop(temp); + let policy_block = r#"policies: + graph_rules: + file: ./graph.policy.yaml + applies_to: [knowledge] + cluster_rules: + file: ./cluster.policy.yaml + applies_to: [cluster] +"#; + let temp = { + let temp = tempfile::tempdir().unwrap(); + fs::write( + temp.path().join("people.pg"), + "\nnode Person {\n name: String @key\n}\n", + ) + .unwrap(); + fs::write( + temp.path().join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + temp.path().join("graph.policy.yaml"), + permit_all_policy_yaml(&["default"]), + ) + .unwrap(); + fs::write( + temp.path().join("cluster.policy.yaml"), + permit_all_policy_yaml(&["default"]).replace("protected_branches: [main]\n", "protected_branches: [main]\nkind: server\n"), + ) + .unwrap(); + fs::write( + temp.path().join("cluster.yaml"), + format!( + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +{policy_block}"# + ), + ) + .unwrap(); + let import = omnigraph_cluster::import_config_dir(temp.path()).await; + assert!(import.ok, "{:?}", import.diagnostics); + let apply = omnigraph_cluster::apply_config_dir(temp.path()).await; + assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); + temp + }; + + let settings = cluster_settings(temp.path()).await.unwrap(); + let omnigraph_server::ServerConfigMode::Multi { + graphs, + server_policy_file, + .. + } = settings.mode + else { + panic!("cluster boot must select multi-graph routing"); + }; + let graph_policy = graphs[0].policy_file.as_ref().expect("graph-bound bundle"); + assert!( + graph_policy + .to_string_lossy() + .contains("__cluster/resources/policy/graph_rules/"), + "{graph_policy:?}" + ); + let server_policy = server_policy_file.expect("cluster-bound bundle"); + assert!( + server_policy + .to_string_lossy() + .contains("__cluster/resources/policy/cluster_rules/"), + "{server_policy:?}" + ); +} + +#[tokio::test] +async fn cluster_boot_refusals() { + // Mutual exclusion with --config / URI. + let temp = converged_cluster_dir("").await; + let dir = temp.path().to_path_buf(); + let err = omnigraph_server::load_server_settings( + Some(&dir.join("omnigraph.yaml")), + Some(&dir), + None, + None, + None, + true, + ) + .await + .unwrap_err(); + assert!(err.to_string().contains("exclusive boot source"), "{err}"); + let err = omnigraph_server::load_server_settings( + None, + Some(&dir), + Some("file:///tmp/x.omni".to_string()), + None, + None, + true, + ) + .await + .unwrap_err(); + assert!(err.to_string().contains("exclusive boot source"), "{err}"); + + // Tampered catalog blob refuses boot with the remedy. + let blob_dir = dir.join("__cluster/resources/query/knowledge/find_person"); + let blob = fs::read_dir(&blob_dir).unwrap().next().unwrap().unwrap().path(); + fs::write(&blob, "tampered").unwrap(); + let err = cluster_settings(&dir).await.unwrap_err(); + assert!( + err.to_string().contains("catalog_payload_digest_mismatch"), + "{err}" + ); + assert!(err.to_string().contains("cluster refresh"), "{err}"); + + // Missing state refuses with the import/apply remedy. + let empty = tempfile::tempdir().unwrap(); + let err = cluster_settings(empty.path()).await.unwrap_err(); + assert!(err.to_string().contains("cluster_state_missing"), "{err}"); +} diff --git a/crates/omnigraph-server/tests/s3.rs b/crates/omnigraph-server/tests/s3.rs new file mode 100644 index 0000000..b0126a8 --- /dev/null +++ b/crates/omnigraph-server/tests/s3.rs @@ -0,0 +1,77 @@ +//! S3-backed single-graph serving (gated on OMNIGRAPH_S3_TEST_BUCKET). +//! Moved verbatim from tests/server.rs in the modularization. + +use std::fs; + +use axum::body::Body; +use axum::http::{Method, Request, StatusCode}; +use omnigraph::db::Omnigraph; +use omnigraph::loader::{LoadMode, load_jsonl}; +use omnigraph_server::api::ReadRequest; +use omnigraph_server::{AppState, build_app}; +use serde_json::json; + + +mod support; +use support::*; + +#[tokio::test(flavor = "multi_thread")] +async fn server_opens_s3_graph_directly_and_serves_snapshot_and_read() { + let Some(uri) = s3_test_graph_uri("server") else { + eprintln!("skipping s3 server test: OMNIGRAPH_S3_TEST_BUCKET is not set"); + return; + }; + + Omnigraph::init(&uri, &fs::read_to_string(fixture("test.pg")).unwrap()) + .await + .unwrap(); + let mut db = Omnigraph::open(&uri).await.unwrap(); + load_jsonl( + &mut db, + &fs::read_to_string(fixture("test.jsonl")).unwrap(), + LoadMode::Overwrite, + ) + .await + .unwrap(); + + let app = build_app( + AppState::open_with_bearer_token(uri.clone(), Some("s3-token".to_string())) + .await + .unwrap(), + ); + + let (snapshot_status, snapshot_body) = json_response( + &app, + Request::builder() + .uri("/snapshot") + .method(Method::GET) + .header("authorization", "Bearer s3-token") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(snapshot_status, StatusCode::OK); + assert!(snapshot_body["tables"].is_array()); + + let read = ReadRequest { + query_source: fs::read_to_string(fixture("test.gq")).unwrap(), + query_name: Some("get_person".to_string()), + params: Some(json!({ "name": "Alice" })), + branch: Some("main".to_string()), + snapshot: None, + }; + let (read_status, read_body) = json_response( + &app, + Request::builder() + .uri("/read") + .method(Method::POST) + .header("authorization", "Bearer s3-token") + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&read).unwrap())) + .unwrap(), + ) + .await; + assert_eq!(read_status, StatusCode::OK); + assert_eq!(read_body["row_count"], 1); + assert_eq!(read_body["rows"][0]["p.name"], "Alice"); +} diff --git a/crates/omnigraph-server/tests/schema_routes.rs b/crates/omnigraph-server/tests/schema_routes.rs new file mode 100644 index 0000000..d250d8a --- /dev/null +++ b/crates/omnigraph-server/tests/schema_routes.rs @@ -0,0 +1,830 @@ +//! Schema read/apply routes: migrations over HTTP, drift, gating. +//! Moved verbatim from tests/server.rs in the modularization. + +use std::fs; + +use axum::body::Body; +use axum::http::{Method, Request, StatusCode}; +use lance::index::DatasetIndexExt; +use omnigraph::db::{Omnigraph, ReadTarget}; +use omnigraph::loader::LoadMode; +use omnigraph_server::api::{ + ChangeRequest, ErrorOutput, ReadRequest, SchemaApplyRequest, SchemaOutput, +}; +use omnigraph_server::{AppState, build_app}; +use serde_json::json; + + +mod support; +use support::*; + +#[tokio::test] +async fn schema_apply_route_updates_graph_for_authorized_admin() { + let (temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + SCHEMA_APPLY_POLICY_YAML, + ) + .await; + let schema = additive_schema_with_nickname(); + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: schema, + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + + assert_eq!(status, StatusCode::OK); + assert_eq!(payload["applied"], true); + let graph = graph_path(temp.path()); + let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + assert!( + reopened.catalog().node_types["Person"] + .properties + .contains_key("nickname") + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_apply_route_rejects_stored_query_breakage_before_publish() { + let (temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, true)], + &[("act-ragnor", "admin-token")], + STORED_QUERY_SCHEMA_APPLY_POLICY_YAML, + ) + .await; + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: renamed_age_schema(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + assert_eq!(status, StatusCode::BAD_REQUEST, "body: {payload}"); + let message = payload["error"].as_str().unwrap_or_default(); + assert!( + message.contains("find_person") && message.contains("schema check"), + "registry breakage should name the stored query; body: {payload}" + ); + + let reopened = Omnigraph::open(graph_path(temp.path()).to_str().unwrap()) + .await + .unwrap(); + let person = &reopened.catalog().node_types["Person"]; + assert!(person.properties.contains_key("age")); + assert!(!person.properties.contains_key("years")); + + let (invoke_status, invoke_body) = json_response( + &app, + invoke_request( + "find_person", + "admin-token", + json!({ "params": { "name": "Alice" } }), + ), + ) + .await; + assert_eq!(invoke_status, StatusCode::OK, "body: {invoke_body}"); + assert_eq!(invoke_body["row_count"], 1); +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_apply_route_noop_keeps_valid_stored_query_registry() { + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, true)], + &[("act-ragnor", "admin-token")], + STORED_QUERY_SCHEMA_APPLY_POLICY_YAML, + ) + .await; + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: fs::read_to_string(fixture("test.pg")).unwrap(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + assert_eq!(status, StatusCode::OK, "body: {payload}"); + assert_eq!(payload["applied"], false); +} + +#[tokio::test] +async fn schema_apply_route_requires_schema_apply_policy_permission() { + let (_temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + POLICY_YAML, + ) + .await; + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: additive_schema_with_nickname(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + + assert_eq!(status, StatusCode::FORBIDDEN); + assert_eq!( + payload["code"], + serde_json::to_value(omnigraph_server::api::ErrorCode::Forbidden).unwrap() + ); +} + +#[tokio::test] +async fn schema_apply_route_requires_bearer_token_when_policy_enabled() { + let (_temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + SCHEMA_APPLY_POLICY_YAML, + ) + .await; + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: additive_schema_with_nickname(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + + assert_eq!(status, StatusCode::UNAUTHORIZED); + assert_eq!( + payload["code"], + serde_json::to_value(omnigraph_server::api::ErrorCode::Unauthorized).unwrap() + ); +} + +#[tokio::test] +async fn schema_apply_route_can_rename_type() { + let (temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + SCHEMA_APPLY_POLICY_YAML, + ) + .await; + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: renamed_person_schema(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + + assert_eq!(status, StatusCode::OK); + assert_eq!(payload["applied"], true); + let graph = graph_path(temp.path()); + let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + let snapshot = reopened + .snapshot_of(ReadTarget::branch("main")) + .await + .unwrap(); + assert!(snapshot.entry("node:Human").is_some()); + assert!(snapshot.entry("node:Person").is_none()); +} + +#[tokio::test] +async fn schema_apply_route_can_rename_property() { + let (temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + SCHEMA_APPLY_POLICY_YAML, + ) + .await; + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: renamed_age_schema(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + + assert_eq!(status, StatusCode::OK); + assert_eq!(payload["applied"], true); + let graph = graph_path(temp.path()); + let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + let person = &reopened.catalog().node_types["Person"]; + assert!(person.properties.contains_key("years")); + assert!(!person.properties.contains_key("age")); +} + +#[tokio::test] +async fn schema_apply_route_can_add_index() { + let (temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + SCHEMA_APPLY_POLICY_YAML, + ) + .await; + let graph = graph_path(temp.path()); + let before_index_count = { + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + let snapshot = db.snapshot_of(ReadTarget::branch("main")).await.unwrap(); + let dataset = snapshot.open("node:Person").await.unwrap(); + dataset.load_indices().await.unwrap().len() + }; + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: indexed_name_schema(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + + assert_eq!(status, StatusCode::OK); + assert_eq!(payload["applied"], true); + let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + let snapshot = reopened + .snapshot_of(ReadTarget::branch("main")) + .await + .unwrap(); + let dataset = snapshot.open("node:Person").await.unwrap(); + let after_index_count = dataset.load_indices().await.unwrap().len(); + assert!(after_index_count > before_index_count); +} + +#[tokio::test] +async fn schema_apply_route_rejects_unsupported_plan() { + let (_temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + SCHEMA_APPLY_POLICY_YAML, + ) + .await; + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: unsupported_schema_change(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + + assert_eq!(status, StatusCode::BAD_REQUEST); + assert_eq!( + payload["code"], + serde_json::to_value(omnigraph_server::api::ErrorCode::BadRequest).unwrap() + ); +} + +#[tokio::test] +async fn schema_apply_route_rejects_when_non_main_branch_exists() { + let temp = init_graph_with_schema(&fs::read_to_string(fixture("test.pg")).unwrap()).await; + let graph = graph_path(temp.path()); + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.branch_create("feature").await.unwrap(); + drop(db); + + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, SCHEMA_APPLY_POLICY_YAML).unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![("act-ragnor".to_string(), "admin-token".to_string())], + Some(&policy_path), + ) + .await + .unwrap(); + let app = build_app(state); + + let request = Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: additive_schema_with_nickname(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(); + let (status, payload) = json_response(&app, request).await; + + assert_eq!(status, StatusCode::CONFLICT); + assert_eq!( + payload["code"], + serde_json::to_value(omnigraph_server::api::ErrorCode::Conflict).unwrap() + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_drift_returns_conflict_for_snapshot_read_and_change() { + let (temp, app) = app_for_loaded_graph().await; + let graph = graph_path(temp.path()); + fs::write(graph.join("_schema.pg"), drifted_test_schema()).unwrap(); + + let (snapshot_status, snapshot_body) = json_response( + &app, + Request::builder() + .uri("/snapshot?branch=main") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + let snapshot_error: ErrorOutput = serde_json::from_value(snapshot_body).unwrap(); + assert_eq!(snapshot_status, StatusCode::CONFLICT); + assert_eq!( + snapshot_error.code, + Some(omnigraph_server::api::ErrorCode::Conflict) + ); + assert!( + snapshot_error + .error + .contains("schema evolution is locked down in phase 1") + ); + + let read = ReadRequest { + query_source: fs::read_to_string(fixture("test.gq")).unwrap(), + query_name: Some("get_person".to_string()), + params: Some(json!({ "name": "Alice" })), + branch: Some("main".to_string()), + snapshot: None, + }; + let (read_status, read_body) = json_response( + &app, + Request::builder() + .uri("/read") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&read).unwrap())) + .unwrap(), + ) + .await; + let read_error: ErrorOutput = serde_json::from_value(read_body).unwrap(); + assert_eq!(read_status, StatusCode::CONFLICT); + assert_eq!( + read_error.code, + Some(omnigraph_server::api::ErrorCode::Conflict) + ); + assert!( + read_error + .error + .contains("schema evolution is locked down in phase 1") + ); + + let change = ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": "Mina", "age": 28 })), + branch: Some("main".to_string()), + }; + let (change_status, change_body) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&change).unwrap())) + .unwrap(), + ) + .await; + let change_error: ErrorOutput = serde_json::from_value(change_body).unwrap(); + assert_eq!(change_status, StatusCode::CONFLICT); + assert_eq!( + change_error.code, + Some(omnigraph_server::api::ErrorCode::Conflict) + ); + assert!( + change_error + .error + .contains("schema evolution is locked down in phase 1") + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_route_returns_current_source() { + let (_temp, app) = app_for_loaded_graph().await; + let (status, body) = json_response( + &app, + Request::builder() + .uri("/schema") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + + assert_eq!(status, StatusCode::OK); + let output: SchemaOutput = serde_json::from_value(body).unwrap(); + assert!(output.schema_source.contains("node Person")); +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_route_requires_bearer_token_when_auth_configured() { + let (_temp, app) = app_for_loaded_graph_with_auth("demo-token").await; + + let (missing_status, missing_body) = json_response( + &app, + Request::builder() + .uri("/schema") + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await; + let missing_error: ErrorOutput = serde_json::from_value(missing_body).unwrap(); + assert_eq!(missing_status, StatusCode::UNAUTHORIZED); + assert_eq!( + missing_error.code, + Some(omnigraph_server::api::ErrorCode::Unauthorized) + ); + + let (ok_status, ok_body) = json_response( + &app, + Request::builder() + .uri("/schema") + .method(Method::GET) + .header("authorization", "Bearer demo-token") + .body(Body::empty()) + .unwrap(), + ) + .await; + assert_eq!(ok_status, StatusCode::OK); + let output: SchemaOutput = serde_json::from_value(ok_body).unwrap(); + assert!(!output.schema_source.is_empty()); +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_route_denied_when_actor_lacks_read_permission() { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let policy_path = temp.path().join("policy.yaml"); + // Policy grants branch_create only — no read action for act-bruno. + fs::write(&policy_path, INGEST_CREATE_ONLY_POLICY_YAML).unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![("act-bruno".to_string(), "team-token".to_string())], + Some(&policy_path), + ) + .await + .unwrap(); + let app = build_app(state); + + let (status, body) = json_response( + &app, + Request::builder() + .uri("/schema") + .method(Method::GET) + .header("authorization", "Bearer team-token") + .body(Body::empty()) + .unwrap(), + ) + .await; + let error: ErrorOutput = serde_json::from_value(body).unwrap(); + assert_eq!(status, StatusCode::FORBIDDEN); + assert_eq!( + error.code, + Some(omnigraph_server::api::ErrorCode::Forbidden) + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_apply_route_soft_drops_property_via_http() { + let (temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + SCHEMA_APPLY_POLICY_YAML, + ) + .await; + // Load a row that has the column we're about to drop. + let graph = graph_path(temp.path()); + { + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.load( + "main", + r#"{"type":"Person","data":{"name":"PreDrop","age":42}}"#, + LoadMode::Append, + ) + .await + .unwrap(); + } + let pre_version = manifest_dataset_version(&graph).await; + + let (status, payload) = json_response( + &app, + Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: schema_without_age(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(payload["applied"], true); + + // Catalog reflects the drop: `age` is gone from the live schema. + let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + assert!( + !reopened.catalog().node_types["Person"] + .properties + .contains_key("age"), + "catalog should not contain `age` after drop" + ); + + // Soft drop preserves the prior version — `age` is still readable + // via time travel to the pre-drop manifest version. Mirrors the + // SDK-side assertion in `apply_schema_drops_a_nullable_property_softly_preserves_prior_version`. + let pre_drop_snapshot = reopened.snapshot_at_version(pre_version).await.unwrap(); + let pre_drop_ds = pre_drop_snapshot.open("node:Person").await.unwrap(); + let pre_drop_fields = pre_drop_ds + .schema() + .fields + .iter() + .map(|f| f.name.clone()) + .collect::<Vec<_>>(); + assert!( + pre_drop_fields.iter().any(|f| f == "age"), + "soft drop should leave the pre-drop dataset's `age` column \ + time-travel-reachable; got fields {pre_drop_fields:?}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_apply_route_soft_drops_node_type_via_http() { + let (temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + SCHEMA_APPLY_POLICY_YAML, + ) + .await; + let graph = graph_path(temp.path()); + + let (status, payload) = json_response( + &app, + Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: schema_without_company(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(payload["applied"], true); + + let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + assert!( + !reopened.catalog().node_types.contains_key("Company"), + "catalog should not contain `Company` after drop" + ); + assert!( + !reopened.catalog().edge_types.contains_key("WorksAt"), + "catalog should not contain `WorksAt` after cascade" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_apply_route_hard_drops_property_with_allow_data_loss() { + let (temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + SCHEMA_APPLY_POLICY_YAML, + ) + .await; + let graph = graph_path(temp.path()); + { + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.load( + "main", + r#"{"type":"Person","data":{"name":"PreDropHard","age":50}}"#, + LoadMode::Append, + ) + .await + .unwrap(); + } + + // Apply with allow_data_loss=true → Hard mode promotion. + let (status, payload) = json_response( + &app, + Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: schema_without_age(), + allow_data_loss: true, + }) + .unwrap(), + )) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(payload["applied"], true); + + // Catalog reflects the drop. + let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + assert!( + !reopened.catalog().node_types["Person"] + .properties + .contains_key("age"), + "catalog should not contain `age` after Hard drop" + ); + // Plan steps should show DropMode::Hard for property drops. + let steps = payload["steps"].as_array().expect("steps array"); + let drop_step = steps + .iter() + .find(|s| s["kind"] == "drop_property") + .expect("plan should include drop_property step"); + let mode = &drop_step["mode"]; + assert_eq!( + mode, "hard", + "expected hard mode under allow_data_loss=true" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_apply_route_keeps_drops_soft_without_flag() { + // Symmetric to the Hard test: same schema change, but no + // allow_data_loss flag → drops stay Soft (prior column data + // remains time-travel-reachable). Pins the default semantics + // against accidental Hard promotion. + let (temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + SCHEMA_APPLY_POLICY_YAML, + ) + .await; + let graph = graph_path(temp.path()); + + let (status, payload) = json_response( + &app, + Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: schema_without_age(), + allow_data_loss: false, + }) + .unwrap(), + )) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(payload["applied"], true); + + let steps = payload["steps"].as_array().expect("steps array"); + let drop_step = steps + .iter() + .find(|s| s["kind"] == "drop_property") + .expect("plan should include drop_property step"); + let mode = &drop_step["mode"]; + assert_eq!(mode, "soft", "expected soft mode without allow_data_loss"); + let _ = graph; +} + +#[tokio::test(flavor = "multi_thread")] +async fn schema_apply_route_additive_property_preserves_existing_rows() { + // SDK suite covers rename and drop data preservation. Additive + // AddProperty wasn't pinned with a row-count check anywhere. + // Load N rows, apply schema adding nullable property, verify + // every row is still readable and the new column is null. + let (temp, app) = app_for_graph_with_auth_tokens_and_policy( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &[("act-ragnor", "admin-token")], + SCHEMA_APPLY_POLICY_YAML, + ) + .await; + let graph = graph_path(temp.path()); + + // Standard fixture data: 4 Persons + 1 Company. Load it. + let pre_count = { + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.load( + "main", + &fs::read_to_string(fixture("test.jsonl")).unwrap(), + LoadMode::Append, + ) + .await + .unwrap(); + let snap = db + .snapshot_of(omnigraph::db::ReadTarget::branch("main")) + .await + .unwrap(); + snap.entry("node:Person").expect("Person").row_count + }; + assert!(pre_count > 0, "fixture should have loaded Person rows"); + + let (status, payload) = json_response( + &app, + Request::builder() + .method(Method::POST) + .uri("/schema/apply") + .header("content-type", "application/json") + .header("authorization", "Bearer admin-token") + .body(Body::from( + serde_json::to_vec(&SchemaApplyRequest { + schema_source: additive_schema_with_nickname(), + ..Default::default() + }) + .unwrap(), + )) + .unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(payload["applied"], true); + + // Row count preserved. + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + let snap = db + .snapshot_of(omnigraph::db::ReadTarget::branch("main")) + .await + .unwrap(); + let post_count = snap.entry("node:Person").expect("Person").row_count; + assert_eq!( + post_count, pre_count, + "AddProperty should preserve row count", + ); +} diff --git a/crates/omnigraph-server/tests/server.rs b/crates/omnigraph-server/tests/server.rs deleted file mode 100644 index d11c542..0000000 --- a/crates/omnigraph-server/tests/server.rs +++ /dev/null @@ -1,6520 +0,0 @@ -use std::env; -use std::fs; -use std::path::{Path, PathBuf}; -use std::sync::Arc; - -use axum::Router; -use axum::body::{Body, to_bytes}; -use axum::http::header::AUTHORIZATION; -use axum::http::{Method, Request, StatusCode}; -use lance::index::DatasetIndexExt; -use omnigraph::db::{Omnigraph, ReadTarget}; -use omnigraph::error::OmniError; -use omnigraph::loader::{LoadMode, load_jsonl}; -use omnigraph_policy::{PolicyChecker, PolicyEngine}; -use omnigraph_server::api::{ - BranchCreateRequest, BranchMergeRequest, ChangeRequest, ErrorOutput, ExportRequest, - IngestRequest, QueryRequest, ReadRequest, SchemaApplyRequest, SchemaOutput, -}; -use omnigraph_server::queries::{QueryRegistry, RegistrySpec}; -use omnigraph_server::{AppState, build_app}; -use serde_json::{Value, json}; -use serial_test::serial; -use tower::ServiceExt; - -const MUTATION_QUERIES: &str = r#" -query insert_person($name: String, $age: I32) { - insert Person { name: $name, age: $age } -} - -query set_age($name: String, $age: I32) { - update Person set { age: $age } where name = $name -} -"#; - -const POLICY_YAML: &str = r#" -version: 1 -groups: - team: [act-andrew, act-bruno, act-ragnor] - admins: [act-ragnor] -protected_branches: [main] -rules: - - id: team-read - allow: - actors: { group: team } - actions: [read] - branch_scope: any - - id: admins-export - allow: - actors: { group: admins } - actions: [export] - branch_scope: any - - id: team-write-unprotected - allow: - actors: { group: team } - actions: [change] - branch_scope: unprotected - - id: admins-merge - allow: - actors: { group: admins } - actions: [branch_delete, branch_merge] - target_branch_scope: protected -"#; - -const POLICY_PROTECTED_READ_YAML: &str = r#" -version: 1 -groups: - team: [act-bruno] -protected_branches: [main] -rules: - - id: protected-read - allow: - actors: { group: team } - actions: [read] - branch_scope: protected -"#; - -const INGEST_CREATE_ONLY_POLICY_YAML: &str = r#" -version: 1 -groups: - team: [act-bruno] -protected_branches: [main] -rules: - - id: team-branch-create - allow: - actors: { group: team } - actions: [branch_create] - target_branch_scope: unprotected -"#; - -const SCHEMA_APPLY_POLICY_YAML: &str = r#" -version: 1 -groups: - admins: [act-ragnor] -protected_branches: [main] -rules: - - id: admins-schema-apply - allow: - actors: { group: admins } - actions: [schema_apply] - target_branch_scope: protected -"#; - -fn fixture(name: &str) -> PathBuf { - PathBuf::from(env!("CARGO_MANIFEST_DIR")) - .join("../omnigraph/tests/fixtures") - .join(name) -} - -async fn init_loaded_graph() -> tempfile::TempDir { - init_graph_with_schema_and_data( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &fs::read_to_string(fixture("test.jsonl")).unwrap(), - ) - .await -} - -async fn init_graph_with_schema_and_data(schema: &str, data: &str) -> tempfile::TempDir { - let temp = tempfile::tempdir().unwrap(); - let graph = graph_path(temp.path()); - fs::create_dir_all(&graph).unwrap(); - Omnigraph::init(graph.to_str().unwrap(), schema) - .await - .unwrap(); - let mut db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - load_jsonl(&mut db, data, LoadMode::Overwrite) - .await - .unwrap(); - temp -} - -async fn init_graph_with_schema(schema: &str) -> tempfile::TempDir { - let temp = tempfile::tempdir().unwrap(); - let graph = graph_path(temp.path()); - fs::create_dir_all(&graph).unwrap(); - Omnigraph::init(graph.to_str().unwrap(), schema) - .await - .unwrap(); - temp -} - -fn graph_path(root: &Path) -> PathBuf { - root.join("server.omni") -} - -fn stored_query_registry(specs: &[(&str, &str, bool)]) -> QueryRegistry { - QueryRegistry::from_specs( - specs - .iter() - .map(|(name, source, expose)| RegistrySpec { - name: name.to_string(), - source: source.to_string(), - expose: *expose, - tool_name: None, - }) - .collect(), - ) - .expect("specs parse and key==symbol") -} - -#[tokio::test] -async fn server_boots_with_a_valid_stored_query_registry() { - // A stored query that type-checks against the fixture schema - // (`Person { name, age }`) must let the server boot. - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let registry = stored_query_registry(&[( - "find_person", - "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", - false, - )]); - let state = AppState::open_single_with_queries( - graph.to_string_lossy().to_string(), - vec![], - None, - registry, - ) - .await; - assert!(state.is_ok(), "valid registry should boot: {:?}", state.err()); -} - -#[tokio::test] -async fn server_refuses_boot_on_type_broken_stored_query() { - // A stored query referencing a type not in the schema (`Widget`) - // must abort boot, naming the offending query. - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let registry = stored_query_registry(&[( - "ghost", - "query ghost() { match { $w: Widget } return { $w.name } }", - false, - )]); - let result = AppState::open_single_with_queries( - graph.to_string_lossy().to_string(), - vec![], - None, - registry, - ) - .await; - // `AppState` is not `Debug`, so match rather than `expect_err`. - let err = match result { - Ok(_) => panic!("type-broken stored query must refuse boot"), - Err(err) => err, - }; - let msg = err.to_string(); - assert!(msg.contains("ghost"), "error should name the broken query: {msg}"); - assert!( - msg.contains("schema check"), - "error should mention the schema check: {msg}" - ); -} - -/// Build a single-mode app with a stored-query registry plus a bearer→actor -/// pairing and a policy, so invoke tests exercise the `invoke_query` -/// boundary gate and the inner read/change gates together. -async fn app_with_stored_queries( - specs: &[(&str, &str, bool)], - tokens: &[(&str, &str)], - policy: &str, -) -> (tempfile::TempDir, Router) { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, policy).unwrap(); - let registry = stored_query_registry(specs); - let state = AppState::open_single_with_queries( - graph.to_string_lossy().to_string(), - tokens - .iter() - .map(|(actor, token)| ((*actor).to_string(), (*token).to_string())) - .collect(), - Some(&policy_path), - registry, - ) - .await - .unwrap(); - (temp, build_app(state)) -} - -/// - `act-invoke`: invoke_query + read (stored reads, not mutations) -/// - `act-full`: invoke_query + read + change (stored mutations) -/// - `act-noinvoke`: read only, no invoke_query (boundary-denied) -/// - `act-invokeonly`: invoke_query only, no read (clears the boundary, inner read denies) -const INVOKE_POLICY_YAML: &str = r#" -version: 1 -groups: - invokers: ["act-invoke"] - full: ["act-full"] - readers: ["act-noinvoke"] - invoke_only: ["act-invokeonly"] -protected_branches: [main] -rules: - # invoke_query is graph-scoped — its own rules, no branch_scope. - - id: invokers-can-invoke - allow: - actors: { group: invokers } - actions: [invoke_query] - - id: full-can-invoke - allow: - actors: { group: full } - actions: [invoke_query] - - id: invoke-only-can-invoke - allow: - actors: { group: invoke_only } - actions: [invoke_query] - # read / change are branch-scoped. - - id: invokers-can-read - allow: - actors: { group: invokers } - actions: [read] - branch_scope: any - - id: full-can-read-change - allow: - actors: { group: full } - actions: [read, change] - branch_scope: any - - id: readers-can-read - allow: - actors: { group: readers } - actions: [read] - branch_scope: any -"#; - -const STORED_QUERY_SCHEMA_APPLY_POLICY_YAML: &str = r#" -version: 1 -groups: - admins: [act-ragnor] -protected_branches: [main] -rules: - - id: admins-can-invoke - allow: - actors: { group: admins } - actions: [invoke_query] - - id: admins-can-read - allow: - actors: { group: admins } - actions: [read] - branch_scope: any - - id: admins-can-schema-apply - allow: - actors: { group: admins } - actions: [schema_apply] - target_branch_scope: protected -"#; - -const FIND_PERSON_GQ: &str = - "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }"; - -fn invoke_request(name: &str, token: &str, body: Value) -> Request<Body> { - Request::builder() - .uri(format!("/queries/{name}")) - .method(Method::POST) - .header("content-type", "application/json") - .header("authorization", format!("Bearer {token}")) - .body(Body::from(serde_json::to_vec(&body).unwrap())) - .unwrap() -} - -fn invoke_request_bytes( - name: &str, - token: &str, - body: impl Into<Body>, - content_type: Option<&str>, -) -> Request<Body> { - let mut builder = Request::builder() - .uri(format!("/queries/{name}")) - .method(Method::POST) - .header("authorization", format!("Bearer {token}")); - if let Some(content_type) = content_type { - builder = builder.header("content-type", content_type); - } - builder.body(body.into()).unwrap() -} - -#[tokio::test(flavor = "multi_thread")] -async fn invoke_stored_read_returns_rows() { - let (_temp, app) = app_with_stored_queries( - &[("find_person", FIND_PERSON_GQ, false)], - &[("act-invoke", "t-invoke")], - INVOKE_POLICY_YAML, - ) - .await; - let (status, body) = json_response( - &app, - invoke_request("find_person", "t-invoke", json!({ "params": { "name": "Alice" } })), - ) - .await; - assert_eq!(status, StatusCode::OK, "body: {body}"); - assert_eq!(body["query_name"], "find_person"); - assert_eq!(body["row_count"], 1, "Alice is in the fixture; body: {body}"); - assert!(body["rows"].is_array(), "read envelope shape; body: {body}"); -} - -#[tokio::test(flavor = "multi_thread")] -async fn invoke_stored_read_accepts_absent_or_empty_body() { - let no_param_query = "query list_people() { match { $p: Person } return { $p.name } }"; - let (_temp, app) = app_with_stored_queries( - &[("list_people", no_param_query, false)], - &[("act-invoke", "t-invoke")], - INVOKE_POLICY_YAML, - ) - .await; - - let (status, body) = json_response( - &app, - invoke_request_bytes("list_people", "t-invoke", Body::empty(), None), - ) - .await; - assert_eq!(status, StatusCode::OK, "body: {body}"); - assert_eq!(body["query_name"], "list_people"); - - let (status, body) = json_response( - &app, - invoke_request_bytes( - "list_people", - "t-invoke", - Body::empty(), - Some("application/json"), - ), - ) - .await; - assert_eq!(status, StatusCode::OK, "body: {body}"); - - let (status, body) = json_response( - &app, - invoke_request_bytes( - "list_people", - "t-invoke", - Body::from("{}"), - Some("application/json"), - ), - ) - .await; - assert_eq!(status, StatusCode::OK, "body: {body}"); - - let (status, body) = json_response( - &app, - invoke_request_bytes( - "list_people", - "t-invoke", - Body::from("{"), - Some("application/json"), - ), - ) - .await; - assert_eq!(status, StatusCode::BAD_REQUEST, "body: {body}"); - assert!( - body["error"] - .as_str() - .unwrap_or_default() - .contains("invalid stored-query invocation body"), - "malformed JSON should be rejected as bad request; body: {body}" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn invoke_stored_mutation_double_gates_on_change() { - let specs: &[(&str, &str, bool)] = &[( - "add_person", - "query add_person($name: String) { insert Person { name: $name } }", - false, - )]; - let (_temp, app) = app_with_stored_queries( - specs, - &[("act-invoke", "t-invoke"), ("act-full", "t-full")], - INVOKE_POLICY_YAML, - ) - .await; - - // Has invoke_query but NOT change → the inner change gate denies (403). - let (status, body) = json_response( - &app, - invoke_request("add_person", "t-invoke", json!({ "params": { "name": "Eve" } })), - ) - .await; - assert_eq!( - status, - StatusCode::FORBIDDEN, - "invoke_query without change must 403; body: {body}" - ); - - // Has invoke_query + change → applied. - let (status, body) = json_response( - &app, - invoke_request("add_person", "t-full", json!({ "params": { "name": "Eve" } })), - ) - .await; - assert_eq!(status, StatusCode::OK, "body: {body}"); - assert_eq!(body["affected_nodes"], 1, "body: {body}"); -} - -#[tokio::test(flavor = "multi_thread")] -async fn invoke_stored_query_bad_param_is_400() { - let (_temp, app) = app_with_stored_queries( - &[("find_person", FIND_PERSON_GQ, false)], - &[("act-invoke", "t-invoke")], - INVOKE_POLICY_YAML, - ) - .await; - // `name` is declared String; pass a number. - let (status, body) = json_response( - &app, - invoke_request("find_person", "t-invoke", json!({ "params": { "name": 123 } })), - ) - .await; - assert_eq!(status, StatusCode::BAD_REQUEST, "body: {body}"); - assert!( - body["error"].as_str().unwrap_or_default().contains("name"), - "400 should name the offending param; body: {body}" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn invoke_unknown_query_and_denied_actor_return_identical_404() { - let (_temp, app) = app_with_stored_queries( - &[("find_person", FIND_PERSON_GQ, false)], - &[("act-invoke", "t-invoke"), ("act-noinvoke", "t-noinvoke")], - INVOKE_POLICY_YAML, - ) - .await; - - // Authorized actor, unknown query name → 404. - let (unknown_status, unknown_body) = - json_response(&app, invoke_request("does_not_exist", "t-invoke", json!({}))).await; - // Denied actor (no invoke_query), real query name → 404. - let (denied_status, denied_body) = json_response( - &app, - invoke_request("find_person", "t-noinvoke", json!({ "params": { "name": "Alice" } })), - ) - .await; - - assert_eq!(unknown_status, StatusCode::NOT_FOUND); - assert_eq!(denied_status, StatusCode::NOT_FOUND); - assert_eq!( - unknown_body, denied_body, - "deny must be byte-identical to a missing query (no catalog probing)" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn invoke_query_holder_without_read_sees_403_not_404() { - // The 404-hiding is for callers WITHOUT invoke_query. An actor that - // HOLDS invoke_query but lacks `read` clears the boundary gate, then the - // inner read gate denies → 403 for an EXISTING read query, vs 404 for an - // unknown one. Existence is visible to grant-holders by design (the - // documented double-gate); this pins that actual contract. - let (_temp, app) = app_with_stored_queries( - &[("find_person", FIND_PERSON_GQ, false)], - &[("act-invokeonly", "t-invokeonly")], - INVOKE_POLICY_YAML, - ) - .await; - let (exists_status, _) = json_response( - &app, - invoke_request("find_person", "t-invokeonly", json!({ "params": { "name": "Alice" } })), - ) - .await; - let (absent_status, _) = - json_response(&app, invoke_request("does_not_exist", "t-invokeonly", json!({}))).await; - assert_eq!( - exists_status, - StatusCode::FORBIDDEN, - "an existing read query the holder can't read → inner-gate 403" - ); - assert_eq!(absent_status, StatusCode::NOT_FOUND, "unknown query still 404s"); -} - -fn get_request(uri: &str, token: &str) -> Request<Body> { - Request::builder() - .uri(uri) - .method(Method::GET) - .header("authorization", format!("Bearer {token}")) - .body(Body::empty()) - .unwrap() -} - -#[tokio::test(flavor = "multi_thread")] -async fn list_queries_returns_only_exposed_with_typed_params() { - let (_temp, app) = app_with_stored_queries( - &[ - ("find_person", FIND_PERSON_GQ, true), - ( - "add_person", - "query add_person($name: String) { insert Person { name: $name } }", - true, - ), - ("hidden", "query hidden() { match { $p: Person } return { $p.name } }", false), - ], - &[("act-invoke", "t-invoke")], - INVOKE_POLICY_YAML, - ) - .await; - let (status, body) = json_response(&app, get_request("/queries", "t-invoke")).await; - assert_eq!(status, StatusCode::OK, "body: {body}"); - - let entries = body["queries"].as_array().unwrap(); - let names: Vec<&str> = entries.iter().map(|q| q["name"].as_str().unwrap()).collect(); - assert!( - names.contains(&"find_person") && names.contains(&"add_person"), - "exposed queries listed: {names:?}" - ); - assert!(!names.contains(&"hidden"), "non-exposed query hidden from the catalog: {names:?}"); - - let fp = entries.iter().find(|q| q["name"] == "find_person").unwrap(); - assert_eq!(fp["mutation"], false); - assert_eq!(fp["tool_name"], "find_person"); - assert_eq!(fp["params"][0]["name"], "name"); - assert_eq!(fp["params"][0]["kind"], "string"); - let ap = entries.iter().find(|q| q["name"] == "add_person").unwrap(); - assert_eq!(ap["mutation"], true, "stored insert → mutation"); -} - -#[tokio::test(flavor = "multi_thread")] -async fn list_queries_is_read_gated_so_a_non_invoker_can_list() { - // The catalog is read-gated (not invoke_query-gated), so a reader who - // lacks invoke_query still enumerates the exposed queries — the - // documented probe-oracle gap until per-query Cedar filtering lands. - let (_temp, app) = app_with_stored_queries( - &[("find_person", FIND_PERSON_GQ, true)], - &[("act-noinvoke", "t-noinvoke")], - INVOKE_POLICY_YAML, - ) - .await; - let (status, body) = json_response(&app, get_request("/queries", "t-noinvoke")).await; - assert_eq!(status, StatusCode::OK, "read-gated catalog; body: {body}"); - let names: Vec<&str> = body["queries"] - .as_array() - .unwrap() - .iter() - .map(|q| q["name"].as_str().unwrap()) - .collect(); - assert!( - names.contains(&"find_person"), - "a reader lists the catalog despite lacking invoke_query: {names:?}" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn list_queries_is_empty_when_no_registry() { - let (_temp, app) = app_for_loaded_graph_with_auth("demo-token").await; - let (status, body) = json_response(&app, get_request("/queries", "demo-token")).await; - assert_eq!(status, StatusCode::OK, "body: {body}"); - assert!( - body["queries"].as_array().unwrap().is_empty(), - "no stored-query registry → empty catalog" - ); -} - -fn drifted_test_schema() -> String { - fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace("age: I32?", "age: I64?") -} - -async fn manifest_dataset_version(graph: &Path) -> u64 { - Omnigraph::open(graph.to_string_lossy().as_ref()) - .await - .unwrap() - .snapshot_of(ReadTarget::branch("main")) - .await - .unwrap() - .version() -} - -fn s3_test_graph_uri(suite: &str) -> Option<String> { - let bucket = env::var("OMNIGRAPH_S3_TEST_BUCKET").ok()?; - let prefix = env::var("OMNIGRAPH_S3_TEST_PREFIX") - .ok() - .filter(|value| !value.trim().is_empty()) - .unwrap_or_else(|| "omnigraph-itests".to_string()); - let unique = std::time::SystemTime::now() - .duration_since(std::time::UNIX_EPOCH) - .ok()? - .as_nanos(); - Some(format!("s3://{}/{}/{}/{}", bucket, prefix, suite, unique)) -} - -async fn app_for_loaded_graph() -> (tempfile::TempDir, Router) { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let state = AppState::open(graph.to_string_lossy().to_string()) - .await - .unwrap(); - (temp, build_app(state)) -} - -/// Build a permit-all policy YAML that grants every action used by the -/// HTTP-layer tests to the listed actor names. MR-723 default-deny -/// closed the "tokens but no policy" loophole; helpers that used to -/// represent "auth without policy" now install this permit-all policy -/// so test cases retain their pre-MR-723 semantics ("auth required, -/// every action permitted") without conflicting with the new state -/// matrix. Tests that specifically need the State-2 deny path use -/// `app_for_graph_with_auth_tokens_only` instead. -fn permit_all_policy_yaml(actors: &[&str]) -> String { - let members = actors - .iter() - .map(|a| format!("\"{a}\"")) - .collect::<Vec<_>>() - .join(", "); - format!( - r#" -version: 1 -groups: - permitted: [{members}] -protected_branches: [main] -rules: - - id: permit-data - allow: - actors: {{ group: permitted }} - actions: [read, change, export] - branch_scope: any - - id: permit-protected-target-actions - allow: - actors: {{ group: permitted }} - actions: [schema_apply, branch_create, branch_delete, branch_merge] - target_branch_scope: any -"# - ) -} - -async fn app_for_loaded_graph_with_auth(token: &str) -> (tempfile::TempDir, Router) { - // `AppState::new_with_bearer_token(token)` maps the token to actor "default"; - // permit-all policy needs to include that actor. - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, permit_all_policy_yaml(&["default"])).unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![("default".to_string(), token.to_string())], - Some(&policy_path), - ) - .await - .unwrap(); - (temp, build_app(state)) -} - -async fn app_for_loaded_graph_with_auth_tokens( - tokens: &[(&str, &str)], -) -> (tempfile::TempDir, Router) { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let policy_path = temp.path().join("policy.yaml"); - let actors: Vec<&str> = tokens.iter().map(|(actor, _)| *actor).collect(); - fs::write(&policy_path, permit_all_policy_yaml(&actors)).unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - tokens - .iter() - .map(|(actor, token)| ((*actor).to_string(), (*token).to_string())) - .collect(), - Some(&policy_path), - ) - .await - .unwrap(); - (temp, build_app(state)) -} - -async fn app_for_loaded_graph_with_auth_tokens_and_policy( - tokens: &[(&str, &str)], - policy: &str, -) -> (tempfile::TempDir, Router) { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, policy).unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - tokens - .iter() - .map(|(actor, token)| ((*actor).to_string(), (*token).to_string())) - .collect(), - Some(&policy_path), - ) - .await - .unwrap(); - (temp, build_app(state)) -} - -async fn app_for_graph_with_auth_tokens_and_policy( - schema: &str, - tokens: &[(&str, &str)], - policy: &str, -) -> (tempfile::TempDir, Router) { - let temp = init_graph_with_schema(schema).await; - let graph = graph_path(temp.path()); - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, policy).unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - tokens - .iter() - .map(|(actor, token)| ((*actor).to_string(), (*token).to_string())) - .collect(), - Some(&policy_path), - ) - .await - .unwrap(); - (temp, build_app(state)) -} - -/// MR-723 default-deny mode: bearer tokens configured, no policy file. -/// Exercises ServerRuntimeState::DefaultDeny — authenticated requests -/// for Read succeed, every other action is rejected with 403 from -/// `authorize_request`'s state-2 branch. -async fn app_for_graph_with_auth_tokens_only( - schema: &str, - tokens: &[(&str, &str)], -) -> (tempfile::TempDir, Router) { - let temp = init_graph_with_schema(schema).await; - let graph = graph_path(temp.path()); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - tokens - .iter() - .map(|(actor, token)| ((*actor).to_string(), (*token).to_string())) - .collect(), - None, - ) - .await - .unwrap(); - (temp, build_app(state)) -} - -fn additive_schema_with_nickname() -> String { - fs::read_to_string(fixture("test.pg")).unwrap().replace( - " age: I32?\n}", - " age: I32?\n nickname: String?\n}", - ) -} - -fn schema_without_age() -> String { - // Drop the nullable `age` column from the test schema. Used by the - // HTTP soft/hard drop tests below. - fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace(" age: I32?\n", "") -} - -fn schema_without_company() -> String { - // Drop the `Company` node type and the edge referencing it. Used - // by the HTTP DropType test below. Hand-crafted (no template - // string replace) because the fixture interleaves the type and - // its edge. - r#"node Person { - name: String @key - age: I32? -} - -edge Knows: Person -> Person { - since: Date? -} -"# - .to_string() -} - -fn renamed_person_schema() -> String { - fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace("node Person {\n", "node Human @rename_from(\"Person\") {\n") - .replace("edge Knows: Person -> Person", "edge Knows: Human -> Human") - .replace( - "edge WorksAt: Person -> Company", - "edge WorksAt: Human -> Company", - ) -} - -fn renamed_age_schema() -> String { - fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace("age: I32?", "years: I32? @rename_from(\"age\")") -} - -fn indexed_name_schema() -> String { - fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace("name: String @key", "name: String @key @index") -} - -fn unsupported_schema_change() -> String { - fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace("age: I32?", "age: I64?") -} - -async fn json_response(app: &Router, request: Request<Body>) -> (StatusCode, Value) { - let response = app.clone().oneshot(request).await.unwrap(); - let status = response.status(); - let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); - let value = serde_json::from_slice(&body).unwrap(); - (status, value) -} - -#[tokio::test] -async fn schema_apply_route_updates_graph_for_authorized_admin() { - let (temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - SCHEMA_APPLY_POLICY_YAML, - ) - .await; - let schema = additive_schema_with_nickname(); - - let request = Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: schema, - ..Default::default() - }) - .unwrap(), - )) - .unwrap(); - let (status, payload) = json_response(&app, request).await; - - assert_eq!(status, StatusCode::OK); - assert_eq!(payload["applied"], true); - let graph = graph_path(temp.path()); - let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - assert!( - reopened.catalog().node_types["Person"] - .properties - .contains_key("nickname") - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn schema_apply_route_rejects_stored_query_breakage_before_publish() { - let (temp, app) = app_with_stored_queries( - &[("find_person", FIND_PERSON_GQ, true)], - &[("act-ragnor", "admin-token")], - STORED_QUERY_SCHEMA_APPLY_POLICY_YAML, - ) - .await; - - let request = Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: renamed_age_schema(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(); - let (status, payload) = json_response(&app, request).await; - assert_eq!(status, StatusCode::BAD_REQUEST, "body: {payload}"); - let message = payload["error"].as_str().unwrap_or_default(); - assert!( - message.contains("find_person") && message.contains("schema check"), - "registry breakage should name the stored query; body: {payload}" - ); - - let reopened = Omnigraph::open(graph_path(temp.path()).to_str().unwrap()) - .await - .unwrap(); - let person = &reopened.catalog().node_types["Person"]; - assert!(person.properties.contains_key("age")); - assert!(!person.properties.contains_key("years")); - - let (invoke_status, invoke_body) = json_response( - &app, - invoke_request( - "find_person", - "admin-token", - json!({ "params": { "name": "Alice" } }), - ), - ) - .await; - assert_eq!(invoke_status, StatusCode::OK, "body: {invoke_body}"); - assert_eq!(invoke_body["row_count"], 1); -} - -#[tokio::test(flavor = "multi_thread")] -async fn schema_apply_route_noop_keeps_valid_stored_query_registry() { - let (_temp, app) = app_with_stored_queries( - &[("find_person", FIND_PERSON_GQ, true)], - &[("act-ragnor", "admin-token")], - STORED_QUERY_SCHEMA_APPLY_POLICY_YAML, - ) - .await; - - let request = Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: fs::read_to_string(fixture("test.pg")).unwrap(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(); - let (status, payload) = json_response(&app, request).await; - assert_eq!(status, StatusCode::OK, "body: {payload}"); - assert_eq!(payload["applied"], false); -} - -#[tokio::test] -async fn schema_apply_route_requires_schema_apply_policy_permission() { - let (_temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - POLICY_YAML, - ) - .await; - - let request = Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: additive_schema_with_nickname(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(); - let (status, payload) = json_response(&app, request).await; - - assert_eq!(status, StatusCode::FORBIDDEN); - assert_eq!( - payload["code"], - serde_json::to_value(omnigraph_server::api::ErrorCode::Forbidden).unwrap() - ); -} - -#[tokio::test] -async fn schema_apply_route_requires_bearer_token_when_policy_enabled() { - let (_temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - SCHEMA_APPLY_POLICY_YAML, - ) - .await; - - let request = Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: additive_schema_with_nickname(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(); - let (status, payload) = json_response(&app, request).await; - - assert_eq!(status, StatusCode::UNAUTHORIZED); - assert_eq!( - payload["code"], - serde_json::to_value(omnigraph_server::api::ErrorCode::Unauthorized).unwrap() - ); -} - -#[tokio::test] -async fn schema_apply_route_can_rename_type() { - let (temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - SCHEMA_APPLY_POLICY_YAML, - ) - .await; - - let request = Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: renamed_person_schema(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(); - let (status, payload) = json_response(&app, request).await; - - assert_eq!(status, StatusCode::OK); - assert_eq!(payload["applied"], true); - let graph = graph_path(temp.path()); - let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - let snapshot = reopened - .snapshot_of(ReadTarget::branch("main")) - .await - .unwrap(); - assert!(snapshot.entry("node:Human").is_some()); - assert!(snapshot.entry("node:Person").is_none()); -} - -#[tokio::test] -async fn schema_apply_route_can_rename_property() { - let (temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - SCHEMA_APPLY_POLICY_YAML, - ) - .await; - - let request = Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: renamed_age_schema(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(); - let (status, payload) = json_response(&app, request).await; - - assert_eq!(status, StatusCode::OK); - assert_eq!(payload["applied"], true); - let graph = graph_path(temp.path()); - let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - let person = &reopened.catalog().node_types["Person"]; - assert!(person.properties.contains_key("years")); - assert!(!person.properties.contains_key("age")); -} - -#[tokio::test] -async fn schema_apply_route_can_add_index() { - let (temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - SCHEMA_APPLY_POLICY_YAML, - ) - .await; - let graph = graph_path(temp.path()); - let before_index_count = { - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - let snapshot = db.snapshot_of(ReadTarget::branch("main")).await.unwrap(); - let dataset = snapshot.open("node:Person").await.unwrap(); - dataset.load_indices().await.unwrap().len() - }; - - let request = Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: indexed_name_schema(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(); - let (status, payload) = json_response(&app, request).await; - - assert_eq!(status, StatusCode::OK); - assert_eq!(payload["applied"], true); - let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - let snapshot = reopened - .snapshot_of(ReadTarget::branch("main")) - .await - .unwrap(); - let dataset = snapshot.open("node:Person").await.unwrap(); - let after_index_count = dataset.load_indices().await.unwrap().len(); - assert!(after_index_count > before_index_count); -} - -#[tokio::test] -async fn schema_apply_route_rejects_unsupported_plan() { - let (_temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - SCHEMA_APPLY_POLICY_YAML, - ) - .await; - - let request = Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: unsupported_schema_change(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(); - let (status, payload) = json_response(&app, request).await; - - assert_eq!(status, StatusCode::BAD_REQUEST); - assert_eq!( - payload["code"], - serde_json::to_value(omnigraph_server::api::ErrorCode::BadRequest).unwrap() - ); -} - -#[tokio::test] -async fn schema_apply_route_rejects_when_non_main_branch_exists() { - let temp = init_graph_with_schema(&fs::read_to_string(fixture("test.pg")).unwrap()).await; - let graph = graph_path(temp.path()); - let mut db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.branch_create("feature").await.unwrap(); - drop(db); - - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, SCHEMA_APPLY_POLICY_YAML).unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![("act-ragnor".to_string(), "admin-token".to_string())], - Some(&policy_path), - ) - .await - .unwrap(); - let app = build_app(state); - - let request = Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: additive_schema_with_nickname(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(); - let (status, payload) = json_response(&app, request).await; - - assert_eq!(status, StatusCode::CONFLICT); - assert_eq!( - payload["code"], - serde_json::to_value(omnigraph_server::api::ErrorCode::Conflict).unwrap() - ); -} - -struct EnvGuard { - saved: Vec<(&'static str, Option<String>)>, -} - -impl EnvGuard { - fn set(vars: &[(&'static str, Option<&str>)]) -> Self { - let saved = vars - .iter() - .map(|(name, _)| (*name, env::var(name).ok())) - .collect::<Vec<_>>(); - for (name, value) in vars { - unsafe { - match value { - Some(value) => env::set_var(name, value), - None => env::remove_var(name), - } - } - } - Self { saved } - } -} - -impl Drop for EnvGuard { - fn drop(&mut self) { - for (name, value) in self.saved.drain(..) { - unsafe { - match value { - Some(value) => env::set_var(name, value), - None => env::remove_var(name), - } - } - } - } -} - -fn format_vector(values: &[f32]) -> String { - values - .iter() - .map(|value| format!("{:.8}", value)) - .collect::<Vec<_>>() - .join(", ") -} - -fn normalize_vector(mut values: Vec<f32>) -> Vec<f32> { - let norm = values - .iter() - .map(|value| (*value as f64) * (*value as f64)) - .sum::<f64>() - .sqrt() as f32; - if norm > f32::EPSILON { - for value in &mut values { - *value /= norm; - } - } - values -} - -fn fnv1a64(bytes: &[u8]) -> u64 { - let mut hash = 14695981039346656037u64; - for byte in bytes { - hash ^= *byte as u64; - hash = hash.wrapping_mul(1099511628211u64); - } - hash -} - -fn xorshift64(mut x: u64) -> u64 { - x ^= x << 13; - x ^= x >> 7; - x ^= x << 17; - x -} - -fn mock_embedding(input: &str, dim: usize) -> Vec<f32> { - let mut seed = fnv1a64(input.as_bytes()); - let mut out = Vec::with_capacity(dim); - for _ in 0..dim { - seed = xorshift64(seed); - let ratio = (seed as f64 / u64::MAX as f64) as f32; - out.push((ratio * 2.0) - 1.0); - } - normalize_vector(out) -} - -#[tokio::test(flavor = "multi_thread")] -async fn healthz_succeeds_after_startup() { - let (_temp, app) = app_for_loaded_graph().await; - let (status, body) = json_response( - &app, - Request::builder() - .uri("/healthz") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - - assert_eq!(status, StatusCode::OK); - assert_eq!(body["status"], "ok"); - assert_eq!(body["version"], env!("CARGO_PKG_VERSION")); - match option_env!("OMNIGRAPH_SOURCE_VERSION") { - Some(source_version) => assert_eq!(body["source_version"], source_version), - None => assert!(body.get("source_version").is_none()), - } -} - -#[tokio::test(flavor = "multi_thread")] -async fn schema_drift_returns_conflict_for_snapshot_read_and_change() { - let (temp, app) = app_for_loaded_graph().await; - let graph = graph_path(temp.path()); - fs::write(graph.join("_schema.pg"), drifted_test_schema()).unwrap(); - - let (snapshot_status, snapshot_body) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - let snapshot_error: ErrorOutput = serde_json::from_value(snapshot_body).unwrap(); - assert_eq!(snapshot_status, StatusCode::CONFLICT); - assert_eq!( - snapshot_error.code, - Some(omnigraph_server::api::ErrorCode::Conflict) - ); - assert!( - snapshot_error - .error - .contains("schema evolution is locked down in phase 1") - ); - - let read = ReadRequest { - query_source: fs::read_to_string(fixture("test.gq")).unwrap(), - query_name: Some("get_person".to_string()), - params: Some(json!({ "name": "Alice" })), - branch: Some("main".to_string()), - snapshot: None, - }; - let (read_status, read_body) = json_response( - &app, - Request::builder() - .uri("/read") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&read).unwrap())) - .unwrap(), - ) - .await; - let read_error: ErrorOutput = serde_json::from_value(read_body).unwrap(); - assert_eq!(read_status, StatusCode::CONFLICT); - assert_eq!( - read_error.code, - Some(omnigraph_server::api::ErrorCode::Conflict) - ); - assert!( - read_error - .error - .contains("schema evolution is locked down in phase 1") - ); - - let change = ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": "Mina", "age": 28 })), - branch: Some("main".to_string()), - }; - let (change_status, change_body) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&change).unwrap())) - .unwrap(), - ) - .await; - let change_error: ErrorOutput = serde_json::from_value(change_body).unwrap(); - assert_eq!(change_status, StatusCode::CONFLICT); - assert_eq!( - change_error.code, - Some(omnigraph_server::api::ErrorCode::Conflict) - ); - assert!( - change_error - .error - .contains("schema evolution is locked down in phase 1") - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn protected_routes_require_bearer_token() { - let (_temp, app) = app_for_loaded_graph_with_auth("demo-token").await; - let (status, body) = json_response( - &app, - Request::builder() - .uri("/branches") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - - let error: ErrorOutput = serde_json::from_value(body).unwrap(); - assert_eq!(status, StatusCode::UNAUTHORIZED); - assert_eq!( - error.code, - Some(omnigraph_server::api::ErrorCode::Unauthorized) - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn protected_routes_accept_valid_bearer_token_while_healthz_stays_open() { - let (_temp, app) = app_for_loaded_graph_with_auth("demo-token").await; - - let health = app - .clone() - .oneshot( - Request::builder() - .uri("/healthz") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(health.status(), StatusCode::OK); - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/branches") - .method(Method::GET) - .header("authorization", "Bearer demo-token") - .body(Body::empty()) - .unwrap(), - ) - .await; - - assert_eq!(status, StatusCode::OK); - assert!(body["branches"].is_array()); -} - -#[tokio::test(flavor = "multi_thread")] -async fn export_route_returns_jsonl_for_branch_snapshot() { - let token = "demo-token"; - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let mut db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.branch_create_from(ReadTarget::branch("main"), "feature") - .await - .unwrap(); - db.load( - "feature", - r#"{"type":"Person","data":{"name":"Eve","age":29}}"#, - LoadMode::Append, - ) - .await - .unwrap(); - let expected = db - .export_jsonl("feature", &["Person".to_string()], &[]) - .await - .unwrap(); - drop(db); - - // MR-723: tokens-without-policy is now default-deny. Install a - // permit-all policy alongside the bearer token so /export - // (action=Export) passes Cedar evaluation. The test is exercising - // export semantics, not policy — the policy is just enough to clear - // the State 3 path. - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, permit_all_policy_yaml(&["default"])).unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![("default".to_string(), token.to_string())], - Some(&policy_path), - ) - .await - .unwrap(); - let app = build_app(state); - - let response = app - .clone() - .oneshot( - Request::builder() - .uri("/export") - .method(Method::POST) - .header("content-type", "application/json") - .header("authorization", format!("Bearer {}", token)) - .body(Body::from( - serde_json::to_vec(&ExportRequest { - branch: Some("feature".to_string()), - type_names: vec!["Person".to_string()], - table_keys: Vec::new(), - }) - .unwrap(), - )) - .unwrap(), - ) - .await - .unwrap(); - - assert_eq!(response.status(), StatusCode::OK); - assert_eq!( - response.headers().get("content-type").unwrap(), - "application/x-ndjson; charset=utf-8" - ); - let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); - let text = String::from_utf8(body.to_vec()).unwrap(); - assert_eq!(text, expected); -} - -#[tokio::test(flavor = "multi_thread")] -async fn protected_routes_accept_any_configured_team_bearer_token() { - let (_temp, app) = app_for_loaded_graph_with_auth_tokens(&[ - ("team-01", "token-one"), - ("team-02", "token-two"), - ]) - .await; - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/branches") - .method(Method::GET) - .header("authorization", "Bearer token-two") - .body(Body::empty()) - .unwrap(), - ) - .await; - - assert_eq!(status, StatusCode::OK); - assert!(body["branches"].is_array()); -} - -/// Verifies the hashed-token lookup correctly resolves each bearer to its -/// associated actor, and that the resolved actor — not the handler-supplied -/// default — is what the policy engine sees. Two tokens for two distinct -/// actors; policy grants read to actor-A only. Swapping tokens must swap -/// the policy outcome. -#[tokio::test(flavor = "multi_thread")] -async fn bearer_token_resolves_to_correct_actor_for_policy_decisions() { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let policy_path = temp.path().join("policy.yaml"); - fs::write( - &policy_path, - r#" -version: 1 -groups: - readers: [act-a] - writers: [act-b] -protected_branches: [main] -rules: - - id: readers-only - allow: - actors: { group: readers } - actions: [read] - branch_scope: any -"#, - ) - .unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![ - ("act-a".to_string(), "token-a".to_string()), - ("act-b".to_string(), "token-b".to_string()), - ], - Some(&policy_path), - ) - .await - .unwrap(); - let app = build_app(state); - - // act-a is authenticated AND authorized. - let (ok_status, _) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .header("authorization", "Bearer token-a") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(ok_status, StatusCode::OK); - - // act-b is authenticated but policy rejects — proves the resolved actor - // (not some default) was the policy subject. - let (denied_status, denied_body) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .header("authorization", "Bearer token-b") - .body(Body::empty()) - .unwrap(), - ) - .await; - let denied_error: ErrorOutput = serde_json::from_value(denied_body).unwrap(); - assert_eq!(denied_status, StatusCode::FORBIDDEN); - assert_eq!( - denied_error.code, - Some(omnigraph_server::api::ErrorCode::Forbidden) - ); - - // Unknown token: 401, never reaches the policy engine. - let (bad_status, _) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .header("authorization", "Bearer wrong-token") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(bad_status, StatusCode::UNAUTHORIZED); -} - -/// Regression test for MR-731: actor identity comes from the matched -/// bearer token, never from a client-supplied request header. A future -/// "convenience" PR that lets clients override `actor_id` to spoof -/// another identity must break this test. The principle is named in -/// `docs/dev/invariants.md` Hard Invariant 11 and at the actor-resolution -/// site in `omnigraph-server/src/lib.rs::authorize_request`. -/// -/// Two assertions in one fixture: -/// 1. Spoof-up: bearer for a *denied* actor + X-Actor-Id naming an -/// *allowed* actor — policy still denies (proves the spoof header -/// doesn't promote the request). -/// 2. Spoof-down: bearer for an *allowed* actor + X-Actor-Id naming a -/// *denied* actor — policy still allows (proves the server-resolved -/// identity wins; the spoof can't trick the request into a denial -/// either, which would otherwise be a confusing UX trap). -/// -/// Cross-reference: MR-777 covers boundary cases like actor-id -/// *collision* (two distinct tokens minting the same actor_id) and -/// malformed bearer header parsing. See `auth_boundary_case_coverage` -/// suite when it lands; the two tests together pin the full bearer-token -/// → actor identity contract. -#[tokio::test(flavor = "multi_thread")] -async fn actor_id_resolves_from_bearer_token_ignoring_client_supplied_headers() { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let policy_path = temp.path().join("policy.yaml"); - // Same readers/writers split as - // `bearer_token_resolves_to_correct_actor_for_policy_decisions` — - // `act-a` can read main, `act-b` cannot. The asymmetry is what - // makes the spoof-up/spoof-down distinction observable. - fs::write( - &policy_path, - r#" -version: 1 -groups: - readers: [act-a] - writers: [act-b] -protected_branches: [main] -rules: - - id: readers-only - allow: - actors: { group: readers } - actions: [read] - branch_scope: any -"#, - ) - .unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![ - ("act-a".to_string(), "token-a".to_string()), - ("act-b".to_string(), "token-b".to_string()), - ], - Some(&policy_path), - ) - .await - .unwrap(); - let app = build_app(state); - - // (1) Spoof-up: bearer for act-b (denied) + X-Actor-Id: act-a (allowed). - // If the server were trusting the header, this would succeed as - // act-a. The contract is: the bearer wins. Expect 403 because - // act-b can't read. - let (spoof_up_status, spoof_up_body) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .header("authorization", "Bearer token-b") - .header("x-actor-id", "act-a") - .body(Body::empty()) - .unwrap(), - ) - .await; - let spoof_up_error: ErrorOutput = serde_json::from_value(spoof_up_body).unwrap(); - assert_eq!( - spoof_up_status, - StatusCode::FORBIDDEN, - "X-Actor-Id must not promote a denied bearer to an allowed actor", - ); - assert_eq!( - spoof_up_error.code, - Some(omnigraph_server::api::ErrorCode::Forbidden), - ); - - // (2) Spoof-down: bearer for act-a (allowed) + X-Actor-Id: act-b (denied). - // If the server were trusting the header, this would fail as act-b. - // The contract is: the bearer wins. Expect 200 because act-a can read. - let (spoof_down_status, _) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .header("authorization", "Bearer token-a") - .header("x-actor-id", "act-b") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!( - spoof_down_status, - StatusCode::OK, - "X-Actor-Id must not demote an allowed bearer to a denied actor", - ); - - // (3) Empty-string spoof attempt: an X-Actor-Id of "" must not - // leak through as the policy subject. Same expectation as (1): - // bearer for act-b is denied regardless of what the header tries. - let (empty_spoof_status, _) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .header("authorization", "Bearer token-b") - .header("x-actor-id", "") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!( - empty_spoof_status, - StatusCode::FORBIDDEN, - "empty X-Actor-Id must not clear the resolved actor", - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn policy_allows_read_but_distinguishes_401_from_403() { - let (_temp, app) = app_for_loaded_graph_with_auth_tokens_and_policy( - &[("act-bruno", "team-token"), ("act-ragnor", "admin-token")], - POLICY_YAML, - ) - .await; - - let (missing_status, missing_body) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - let missing_error: ErrorOutput = serde_json::from_value(missing_body).unwrap(); - assert_eq!(missing_status, StatusCode::UNAUTHORIZED); - assert_eq!( - missing_error.code, - Some(omnigraph_server::api::ErrorCode::Unauthorized) - ); - - let (snapshot_status, snapshot_body) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .header("authorization", "Bearer team-token") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(snapshot_status, StatusCode::OK); - assert_eq!(snapshot_body["branch"], "main"); - - let export_request = ExportRequest { - branch: Some("main".to_string()), - type_names: Vec::new(), - table_keys: Vec::new(), - }; - let (forbidden_status, forbidden_body) = json_response( - &app, - Request::builder() - .uri("/export") - .method(Method::POST) - .header("authorization", "Bearer team-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&export_request).unwrap())) - .unwrap(), - ) - .await; - let forbidden_error: ErrorOutput = serde_json::from_value(forbidden_body).unwrap(); - assert_eq!(forbidden_status, StatusCode::FORBIDDEN); - assert_eq!( - forbidden_error.code, - Some(omnigraph_server::api::ErrorCode::Forbidden) - ); - - let response = app - .clone() - .oneshot( - Request::builder() - .uri("/export") - .method(Method::POST) - .header("authorization", "Bearer admin-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&export_request).unwrap())) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(response.status(), StatusCode::OK); -} - -#[tokio::test(flavor = "multi_thread")] -async fn policy_uses_resolved_branch_for_snapshot_reads() { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let snapshot_id = { - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.resolve_snapshot("main").await.unwrap().to_string() - }; - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, POLICY_PROTECTED_READ_YAML).unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![("act-bruno".to_string(), "team-token".to_string())], - Some(&policy_path), - ) - .await - .unwrap(); - let app = build_app(state); - - let read = ReadRequest { - query_source: fs::read_to_string(fixture("test.gq")).unwrap(), - query_name: Some("get_person".to_string()), - params: Some(json!({ "name": "Alice" })), - branch: None, - snapshot: Some(snapshot_id), - }; - let (status, body) = json_response( - &app, - Request::builder() - .uri("/read") - .method(Method::POST) - .header("authorization", "Bearer team-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&read).unwrap())) - .unwrap(), - ) - .await; - - assert_eq!(status, StatusCode::OK); - assert_eq!(body["target"]["branch"], Value::Null); - assert_eq!( - body["target"]["snapshot"].as_str(), - read.snapshot.as_deref() - ); - assert_eq!(body["row_count"], 1); -} - -#[tokio::test(flavor = "multi_thread")] -async fn snapshot_route_returns_manifest_dataset_version() { - let (temp, app) = app_for_loaded_graph().await; - let graph = graph_path(temp.path()); - let expected_manifest_version = manifest_dataset_version(&graph).await; - - let (snapshot_status, snapshot_body) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - - assert_eq!(snapshot_status, StatusCode::OK); - assert_eq!(snapshot_body["branch"], "main"); - assert_eq!( - snapshot_body["manifest_version"].as_u64().unwrap(), - expected_manifest_version - ); - assert!(snapshot_body["tables"].is_array()); -} - -#[tokio::test(flavor = "multi_thread")] -async fn schema_route_returns_current_source() { - let (_temp, app) = app_for_loaded_graph().await; - let (status, body) = json_response( - &app, - Request::builder() - .uri("/schema") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - - assert_eq!(status, StatusCode::OK); - let output: SchemaOutput = serde_json::from_value(body).unwrap(); - assert!(output.schema_source.contains("node Person")); -} - -#[tokio::test(flavor = "multi_thread")] -async fn schema_route_requires_bearer_token_when_auth_configured() { - let (_temp, app) = app_for_loaded_graph_with_auth("demo-token").await; - - let (missing_status, missing_body) = json_response( - &app, - Request::builder() - .uri("/schema") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - let missing_error: ErrorOutput = serde_json::from_value(missing_body).unwrap(); - assert_eq!(missing_status, StatusCode::UNAUTHORIZED); - assert_eq!( - missing_error.code, - Some(omnigraph_server::api::ErrorCode::Unauthorized) - ); - - let (ok_status, ok_body) = json_response( - &app, - Request::builder() - .uri("/schema") - .method(Method::GET) - .header("authorization", "Bearer demo-token") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(ok_status, StatusCode::OK); - let output: SchemaOutput = serde_json::from_value(ok_body).unwrap(); - assert!(!output.schema_source.is_empty()); -} - -#[tokio::test(flavor = "multi_thread")] -async fn schema_route_denied_when_actor_lacks_read_permission() { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let policy_path = temp.path().join("policy.yaml"); - // Policy grants branch_create only — no read action for act-bruno. - fs::write(&policy_path, INGEST_CREATE_ONLY_POLICY_YAML).unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![("act-bruno".to_string(), "team-token".to_string())], - Some(&policy_path), - ) - .await - .unwrap(); - let app = build_app(state); - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/schema") - .method(Method::GET) - .header("authorization", "Bearer team-token") - .body(Body::empty()) - .unwrap(), - ) - .await; - let error: ErrorOutput = serde_json::from_value(body).unwrap(); - assert_eq!(status, StatusCode::FORBIDDEN); - assert_eq!( - error.code, - Some(omnigraph_server::api::ErrorCode::Forbidden) - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn policy_blocks_change_on_protected_main_but_allows_unprotected_branch() { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let mut db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.branch_create_from(ReadTarget::branch("main"), "feature") - .await - .unwrap(); - drop(db); - - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, POLICY_YAML).unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![("act-bruno".to_string(), "team-token".to_string())], - Some(&policy_path), - ) - .await - .unwrap(); - let app = build_app(state); - - let main_change = ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": "Mina", "age": 28 })), - branch: Some("main".to_string()), - }; - let (main_status, main_body) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header("authorization", "Bearer team-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&main_change).unwrap())) - .unwrap(), - ) - .await; - let main_error: ErrorOutput = serde_json::from_value(main_body).unwrap(); - assert_eq!(main_status, StatusCode::FORBIDDEN); - assert_eq!( - main_error.code, - Some(omnigraph_server::api::ErrorCode::Forbidden) - ); - - let feature_change = ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": "Mina", "age": 28 })), - branch: Some("feature".to_string()), - }; - let (feature_status, feature_body) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header("authorization", "Bearer team-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&feature_change).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(feature_status, StatusCode::OK); - assert_eq!(feature_body["branch"], "feature"); - assert_eq!(feature_body["affected_nodes"], 1); -} - -#[tokio::test(flavor = "multi_thread")] -async fn policy_blocks_non_admin_merge_to_main_and_allows_admin() { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let mut db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.branch_create_from(ReadTarget::branch("main"), "feature") - .await - .unwrap(); - db.load( - "feature", - r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#, - LoadMode::Append, - ) - .await - .unwrap(); - drop(db); - - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, POLICY_YAML).unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![ - ("act-bruno".to_string(), "team-token".to_string()), - ("act-ragnor".to_string(), "admin-token".to_string()), - ], - Some(&policy_path), - ) - .await - .unwrap(); - let app = build_app(state); - - let merge = BranchMergeRequest { - source: "feature".to_string(), - target: Some("main".to_string()), - }; - let (deny_status, deny_body) = json_response( - &app, - Request::builder() - .uri("/branches/merge") - .method(Method::POST) - .header("authorization", "Bearer team-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&merge).unwrap())) - .unwrap(), - ) - .await; - let deny_error: ErrorOutput = serde_json::from_value(deny_body).unwrap(); - assert_eq!(deny_status, StatusCode::FORBIDDEN); - assert_eq!( - deny_error.code, - Some(omnigraph_server::api::ErrorCode::Forbidden) - ); - - let (allow_status, allow_body) = json_response( - &app, - Request::builder() - .uri("/branches/merge") - .method(Method::POST) - .header("authorization", "Bearer admin-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&merge).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(allow_status, StatusCode::OK); - assert_eq!(allow_body["actor_id"], "act-ragnor"); -} - -#[tokio::test(flavor = "multi_thread")] -async fn authenticated_change_stamps_actor_on_commits() { - // With the Run state machine removed, actor_id is recorded - // directly on the commit graph (no intermediate run record). - let (_temp, app) = app_for_loaded_graph_with_auth_tokens(&[("act-andrew", "token-one")]).await; - - let change = ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": "Mina", "age": 28 })), - branch: Some("main".to_string()), - }; - let (change_status, change_body) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header("authorization", "Bearer token-one") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&change).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(change_status, StatusCode::OK); - assert_eq!(change_body["actor_id"], "act-andrew"); - - let (commits_status, commits_body) = json_response( - &app, - Request::builder() - .uri("/commits?branch=main") - .method(Method::GET) - .header("authorization", "Bearer token-one") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(commits_status, StatusCode::OK); - let head = commits_body["commits"] - .as_array() - .unwrap() - .last() - .expect("head commit should exist"); - assert_eq!(head["actor_id"], "act-andrew"); -} - -#[tokio::test(flavor = "multi_thread")] -async fn ingest_creates_branch_returns_metadata_and_stamps_actor() { - let (temp, app) = app_for_loaded_graph_with_auth_tokens(&[("act-andrew", "token-one")]).await; - let graph = graph_path(temp.path()); - let ingest = IngestRequest { - branch: Some("feature-ingest".to_string()), - from: Some("main".to_string()), - mode: Some(LoadMode::Merge), - data: r#"{"type":"Person","data":{"name":"Zoe","age":33}} -{"type":"Person","data":{"name":"Bob","age":26}}"# - .to_string(), - }; - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/ingest") - .method(Method::POST) - .header("authorization", "Bearer token-one") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&ingest).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - assert_eq!(body["branch"], "feature-ingest"); - assert_eq!(body["base_branch"], "main"); - assert_eq!(body["branch_created"], true); - assert_eq!(body["mode"], "merge"); - assert_eq!(body["actor_id"], "act-andrew"); - assert_eq!(body["tables"][0]["table_key"], "node:Person"); - assert_eq!(body["tables"][0]["rows_loaded"], 2); - - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - let snapshot = db - .snapshot_of(ReadTarget::branch("feature-ingest")) - .await - .unwrap(); - let person_ds = snapshot.open("node:Person").await.unwrap(); - assert_eq!(person_ds.count_rows(None).await.unwrap(), 5); - let head = db - .list_commits(Some("feature-ingest")) - .await - .unwrap() - .into_iter() - .last() - .unwrap(); - assert_eq!(head.actor_id.as_deref(), Some("act-andrew")); -} - -#[tokio::test(flavor = "multi_thread")] -async fn ingest_existing_branch_skips_branch_create_policy_check() { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - { - let mut db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.branch_create_from(ReadTarget::branch("main"), "feature") - .await - .unwrap(); - } - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, POLICY_YAML).unwrap(); - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![("act-bruno".to_string(), "team-token".to_string())], - Some(&policy_path), - ) - .await - .unwrap(); - let app = build_app(state); - let ingest = IngestRequest { - branch: Some("feature".to_string()), - from: Some("other-base".to_string()), - mode: Some(LoadMode::Merge), - data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), - }; - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/ingest") - .method(Method::POST) - .header("authorization", "Bearer team-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&ingest).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - assert_eq!(body["branch"], "feature"); - assert_eq!(body["branch_created"], false); - assert_eq!(body["base_branch"], "other-base"); -} - -/// Regression: branch creation is opt-in by presence of `from`. A request -/// without `from` against a branch that doesn't exist must 404 — not -/// silently fork `main` and land the data on the typo'd branch. -#[tokio::test(flavor = "multi_thread")] -async fn ingest_without_from_returns_404_for_missing_branch_and_creates_nothing() { - let (temp, app) = app_for_loaded_graph().await; - let graph = graph_path(temp.path()); - let ingest = IngestRequest { - branch: Some("feature-typo".to_string()), - from: None, - mode: Some(LoadMode::Merge), - data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), - }; - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/ingest") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&ingest).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::NOT_FOUND); - let error: ErrorOutput = serde_json::from_value(body).unwrap(); - assert_eq!(error.code, Some(omnigraph_server::api::ErrorCode::NotFound)); - - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - assert!( - !db.branch_list() - .await - .unwrap() - .contains(&"feature-typo".to_string()), - "a 404'd ingest must not create the branch" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn ingest_without_from_loads_into_existing_branch() { - let (temp, app) = app_for_loaded_graph().await; - let graph = graph_path(temp.path()); - { - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.branch_create_from(ReadTarget::branch("main"), "feature") - .await - .unwrap(); - } - let ingest = IngestRequest { - branch: Some("feature".to_string()), - from: None, - mode: Some(LoadMode::Merge), - data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), - }; - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/ingest") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&ingest).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - assert_eq!(body["branch"], "feature"); - assert_eq!(body["branch_created"], false); - assert_eq!(body["base_branch"], serde_json::Value::Null); -} - -#[tokio::test(flavor = "multi_thread")] -async fn ingest_denies_missing_branch_without_branch_create_permission() { - let (_temp, app) = app_for_loaded_graph_with_auth_tokens_and_policy( - &[("act-bruno", "team-token")], - POLICY_YAML, - ) - .await; - let ingest = IngestRequest { - branch: Some("feature".to_string()), - from: Some("main".to_string()), - mode: Some(LoadMode::Merge), - data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), - }; - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/ingest") - .method(Method::POST) - .header("authorization", "Bearer team-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&ingest).unwrap())) - .unwrap(), - ) - .await; - let error: ErrorOutput = serde_json::from_value(body).unwrap(); - assert_eq!(status, StatusCode::FORBIDDEN); - assert_eq!( - error.code, - Some(omnigraph_server::api::ErrorCode::Forbidden) - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn ingest_denies_when_actor_lacks_change_permission() { - let (_temp, app) = app_for_loaded_graph_with_auth_tokens_and_policy( - &[("act-bruno", "team-token")], - INGEST_CREATE_ONLY_POLICY_YAML, - ) - .await; - let ingest = IngestRequest { - branch: Some("feature".to_string()), - from: Some("main".to_string()), - mode: Some(LoadMode::Merge), - data: r#"{"type":"Person","data":{"name":"Zoe","age":33}}"#.to_string(), - }; - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/ingest") - .method(Method::POST) - .header("authorization", "Bearer team-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&ingest).unwrap())) - .unwrap(), - ) - .await; - let error: ErrorOutput = serde_json::from_value(body).unwrap(); - assert_eq!(status, StatusCode::FORBIDDEN); - assert_eq!( - error.code, - Some(omnigraph_server::api::ErrorCode::Forbidden) - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn ingest_rejects_payloads_over_32_mib() { - let (_temp, app) = app_for_loaded_graph().await; - let oversize = IngestRequest { - branch: Some("feature".to_string()), - from: Some("main".to_string()), - mode: Some(LoadMode::Merge), - data: "x".repeat(33 * 1024 * 1024), - }; - - let response = app - .clone() - .oneshot( - Request::builder() - .uri("/ingest") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&oversize).unwrap())) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(response.status(), StatusCode::PAYLOAD_TOO_LARGE); -} - -#[tokio::test(flavor = "multi_thread")] -async fn authenticated_branch_merge_stamps_merge_actor_on_head_commit() { - let (_temp, app) = app_for_loaded_graph_with_auth_tokens(&[ - ("act-andrew", "token-one"), - ("act-ragnor", "token-two"), - ]) - .await; - - let create = BranchCreateRequest { - from: Some("main".to_string()), - name: "feature".to_string(), - }; - let (create_status, _) = json_response( - &app, - Request::builder() - .uri("/branches") - .method(Method::POST) - .header("authorization", "Bearer token-one") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&create).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(create_status, StatusCode::OK); - - let change = ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": "Zoe", "age": 33 })), - branch: Some("feature".to_string()), - }; - let (change_status, _) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header("authorization", "Bearer token-one") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&change).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(change_status, StatusCode::OK); - - let merge = BranchMergeRequest { - source: "feature".to_string(), - target: Some("main".to_string()), - }; - let (merge_status, merge_body) = json_response( - &app, - Request::builder() - .uri("/branches/merge") - .method(Method::POST) - .header("authorization", "Bearer token-two") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&merge).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(merge_status, StatusCode::OK); - assert_eq!(merge_body["actor_id"], "act-ragnor"); - - let (commit_status, commit_body) = json_response( - &app, - Request::builder() - .uri("/commits?branch=main") - .method(Method::GET) - .header("authorization", "Bearer token-two") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(commit_status, StatusCode::OK); - let head = commit_body["commits"] - .as_array() - .unwrap() - .last() - .expect("head commit should exist"); - assert_eq!(head["actor_id"], "act-ragnor"); -} - -#[tokio::test(flavor = "multi_thread")] -async fn branch_merge_conflict_response_includes_structured_conflicts() { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let mut db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.branch_create_from(ReadTarget::branch("main"), "feature") - .await - .unwrap(); - db.mutate( - "main", - MUTATION_QUERIES, - "set_age", - &omnigraph_compiler::json_params_to_param_map( - Some(&json!({"name": "Alice", "age": 31 })), - &omnigraph_compiler::find_named_query(MUTATION_QUERIES, "set_age") - .unwrap() - .params, - omnigraph_compiler::JsonParamMode::Standard, - ) - .unwrap(), - ) - .await - .unwrap(); - db.mutate( - "feature", - MUTATION_QUERIES, - "set_age", - &omnigraph_compiler::json_params_to_param_map( - Some(&json!({"name": "Alice", "age": 32 })), - &omnigraph_compiler::find_named_query(MUTATION_QUERIES, "set_age") - .unwrap() - .params, - omnigraph_compiler::JsonParamMode::Standard, - ) - .unwrap(), - ) - .await - .unwrap(); - drop(db); - - let state = AppState::open(graph.to_string_lossy().to_string()) - .await - .unwrap(); - let app = build_app(state); - let merge = BranchMergeRequest { - source: "feature".to_string(), - target: Some("main".to_string()), - }; - let (status, body) = json_response( - &app, - Request::builder() - .uri("/branches/merge") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&merge).unwrap())) - .unwrap(), - ) - .await; - - let error: ErrorOutput = serde_json::from_value(body).unwrap(); - assert_eq!(status, StatusCode::CONFLICT); - assert_eq!(error.code, Some(omnigraph_server::api::ErrorCode::Conflict)); - assert!(error.error.contains("merge conflict")); - assert!(error.merge_conflicts.iter().any(|conflict| { - conflict.table_key == "node:Person" - && conflict.row_id.as_deref() == Some("Alice") - && conflict.kind == omnigraph_server::api::MergeConflictKindOutput::DivergentUpdate - })); -} - -#[tokio::test(flavor = "multi_thread")] -async fn repeated_read_after_change_sees_updated_state_from_same_app() { - let (_temp, app) = app_for_loaded_graph().await; - - let change = ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": "Mina", "age": 28 })), - branch: Some("main".to_string()), - }; - let (change_status, change_body) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&change).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(change_status, StatusCode::OK); - assert_eq!(change_body["affected_nodes"], 1); - - let read = ReadRequest { - query_source: fs::read_to_string(fixture("test.gq")).unwrap(), - query_name: Some("get_person".to_string()), - params: Some(json!({ "name": "Mina" })), - branch: Some("main".to_string()), - snapshot: None, - }; - let (read_status, read_body) = json_response( - &app, - Request::builder() - .uri("/read") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&read).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(read_status, StatusCode::OK); - assert_eq!(read_body["row_count"], 1); - assert_eq!(read_body["rows"][0]["p.name"], "Mina"); -} - -#[tokio::test(flavor = "multi_thread")] -async fn query_endpoint_runs_inline_read() { - let (_temp, app) = app_for_loaded_graph().await; - - let query = QueryRequest { - query: fs::read_to_string(fixture("test.gq")).unwrap(), - name: Some("get_person".to_string()), - params: Some(json!({ "name": "Alice" })), - branch: Some("main".to_string()), - snapshot: None, - }; - let (status, body) = json_response( - &app, - Request::builder() - .uri("/query") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&query).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - assert_eq!(body["query_name"], "get_person"); - assert_eq!(body["row_count"], 1); - assert_eq!(body["rows"][0]["p.name"], "Alice"); -} - -#[tokio::test(flavor = "multi_thread")] -async fn query_endpoint_rejects_mutation_with_400() { - let (_temp, app) = app_for_loaded_graph().await; - - let query = QueryRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": "Should", "age": 1 })), - branch: Some("main".to_string()), - snapshot: None, - }; - let (status, body) = json_response( - &app, - Request::builder() - .uri("/query") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&query).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::BAD_REQUEST); - let err = body["error"].as_str().unwrap_or_default(); - assert!( - err.contains("contains mutations") && err.contains("POST /mutate"), - "expected mutation-rejection message pointing at canonical /mutate, got: {err}" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn mutate_endpoint_runs_inline_mutation() { - // Canonical mutation endpoint. Pairs with `/query` on the read side. - // Same wire shape as `/change`, no deprecation signal. - let (_temp, app) = app_for_loaded_graph().await; - - let request = json!({ - "query": MUTATION_QUERIES, - "name": "insert_person", - "params": { "name": "Mutie", "age": 30 }, - "branch": "main", - }); - let response = app - .clone() - .oneshot( - Request::builder() - .uri("/mutate") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&request).unwrap())) - .unwrap(), - ) - .await - .unwrap(); - - assert_eq!(response.status(), StatusCode::OK); - // Canonical route is NOT deprecated; no Deprecation header expected. - assert!( - response.headers().get("deprecation").is_none(), - "POST /mutate must not advertise itself as deprecated" - ); - let body_bytes = to_bytes(response.into_body(), usize::MAX).await.unwrap(); - let body: Value = serde_json::from_slice(&body_bytes).unwrap(); - assert_eq!(body["affected_nodes"], 1); - assert_eq!(body["query_name"], "insert_person"); - assert_eq!(body["branch"], "main"); -} - -#[tokio::test(flavor = "multi_thread")] -async fn change_endpoint_emits_deprecation_headers() { - // `/change` is kept indefinitely for back-compat but flagged at runtime - // per RFC 9745 (`Deprecation: true`) + RFC 8288 (`Link: </mutate>; - // rel="successor-version"`). The OpenAPI side is covered by - // `openapi_change_is_deprecated` in tests/openapi.rs. - let (_temp, app) = app_for_loaded_graph().await; - - let request = json!({ - "query": MUTATION_QUERIES, - "name": "insert_person", - "params": { "name": "Legacyer", "age": 33 }, - "branch": "main", - }); - let response = app - .clone() - .oneshot( - Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&request).unwrap())) - .unwrap(), - ) - .await - .unwrap(); - - assert_eq!(response.status(), StatusCode::OK); - assert_eq!( - response - .headers() - .get("deprecation") - .and_then(|v| v.to_str().ok()), - Some("true"), - "POST /change must advertise `Deprecation: true` (RFC 9745)" - ); - assert_eq!( - response.headers().get("link").and_then(|v| v.to_str().ok()), - Some("</mutate>; rel=\"successor-version\""), - "POST /change must point at /mutate via `Link` rel=successor-version (RFC 8288)" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn read_endpoint_emits_deprecation_headers() { - // `/read` is kept indefinitely for byte-stable back-compat but flagged - // at runtime per RFC 9745 + RFC 8288. Successor is `/query`. - let (_temp, app) = app_for_loaded_graph().await; - - let request = ReadRequest { - query_source: fs::read_to_string(fixture("test.gq")).unwrap(), - query_name: Some("get_person".to_string()), - params: Some(json!({ "name": "Alice" })), - branch: Some("main".to_string()), - snapshot: None, - }; - let response = app - .clone() - .oneshot( - Request::builder() - .uri("/read") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&request).unwrap())) - .unwrap(), - ) - .await - .unwrap(); - - assert_eq!(response.status(), StatusCode::OK); - assert_eq!( - response - .headers() - .get("deprecation") - .and_then(|v| v.to_str().ok()), - Some("true"), - "POST /read must advertise `Deprecation: true` (RFC 9745)" - ); - assert_eq!( - response.headers().get("link").and_then(|v| v.to_str().ok()), - Some("</query>; rel=\"successor-version\""), - "POST /read must point at /query via `Link` rel=successor-version (RFC 8288)" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn query_endpoint_does_not_emit_deprecation_headers() { - // Sanity check the inverse: the canonical `/query` endpoint must not - // carry deprecation signaling, so SDK codegens don't propagate a - // bogus `@deprecated` marker. - let (_temp, app) = app_for_loaded_graph().await; - - let request = QueryRequest { - query: fs::read_to_string(fixture("test.gq")).unwrap(), - name: Some("get_person".to_string()), - params: Some(json!({ "name": "Alice" })), - branch: Some("main".to_string()), - snapshot: None, - }; - let response = app - .clone() - .oneshot( - Request::builder() - .uri("/query") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&request).unwrap())) - .unwrap(), - ) - .await - .unwrap(); - - assert_eq!(response.status(), StatusCode::OK); - assert!( - response.headers().get("deprecation").is_none(), - "POST /query is canonical and must not advertise itself as deprecated" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn change_endpoint_accepts_legacy_field_names() { - // The canonical wire field names on /change are `query` and `name`, but - // serde aliases keep the legacy `query_source`/`query_name` payload - // shape working for clients that haven't migrated yet. Pin both shapes. - let (_temp, app) = app_for_loaded_graph().await; - - let legacy_body = json!({ - "query_source": MUTATION_QUERIES, - "query_name": "insert_person", - "params": { "name": "Legacy", "age": 21 }, - "branch": "main", - }); - let (status, body) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&legacy_body).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - assert_eq!(body["affected_nodes"], 1); - - let canonical_body = json!({ - "query": MUTATION_QUERIES, - "name": "insert_person", - "params": { "name": "Canonical", "age": 22 }, - "branch": "main", - }); - let (status, body) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&canonical_body).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - assert_eq!(body["affected_nodes"], 1); -} - -#[tokio::test(flavor = "multi_thread")] -async fn remote_branch_list_create_merge_flow_works() { - let (_temp, app) = app_for_loaded_graph().await; - - let (list_status, list_body) = json_response( - &app, - Request::builder() - .uri("/branches") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(list_status, StatusCode::OK); - assert_eq!(list_body["branches"], json!(["main"])); - - let create = BranchCreateRequest { - from: Some("main".to_string()), - name: "feature".to_string(), - }; - let (create_status, create_body) = json_response( - &app, - Request::builder() - .uri("/branches") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&create).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(create_status, StatusCode::OK); - assert_eq!(create_body["from"], "main"); - assert_eq!(create_body["name"], "feature"); - - let (list_status, list_body) = json_response( - &app, - Request::builder() - .uri("/branches") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(list_status, StatusCode::OK); - assert_eq!(list_body["branches"], json!(["feature", "main"])); - - let change = ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": "Zoe", "age": 33 })), - branch: Some("feature".to_string()), - }; - let (change_status, change_body) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&change).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(change_status, StatusCode::OK); - assert_eq!(change_body["branch"], "feature"); - assert_eq!(change_body["affected_nodes"], 1); - - let read_main_before = ReadRequest { - query_source: fs::read_to_string(fixture("test.gq")).unwrap(), - query_name: Some("get_person".to_string()), - params: Some(json!({ "name": "Zoe" })), - branch: Some("main".to_string()), - snapshot: None, - }; - let (read_status, read_body) = json_response( - &app, - Request::builder() - .uri("/read") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&read_main_before).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(read_status, StatusCode::OK); - assert_eq!(read_body["row_count"], 0); - - let merge = BranchMergeRequest { - source: "feature".to_string(), - target: Some("main".to_string()), - }; - let (merge_status, merge_body) = json_response( - &app, - Request::builder() - .uri("/branches/merge") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&merge).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(merge_status, StatusCode::OK); - assert_eq!(merge_body["source"], "feature"); - assert_eq!(merge_body["target"], "main"); - assert_eq!(merge_body["outcome"], "fast_forward"); - - let read_main_after = ReadRequest { - query_source: fs::read_to_string(fixture("test.gq")).unwrap(), - query_name: Some("get_person".to_string()), - params: Some(json!({ "name": "Zoe" })), - branch: Some("main".to_string()), - snapshot: None, - }; - let (read_status, read_body) = json_response( - &app, - Request::builder() - .uri("/read") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&read_main_after).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(read_status, StatusCode::OK); - assert_eq!(read_body["row_count"], 1); - assert_eq!(read_body["rows"][0]["p.name"], "Zoe"); -} - -#[tokio::test(flavor = "multi_thread")] -async fn remote_branch_delete_flow_works() { - let (_temp, app) = app_for_loaded_graph().await; - - let create = BranchCreateRequest { - from: Some("main".to_string()), - name: "feature".to_string(), - }; - let (create_status, _) = json_response( - &app, - Request::builder() - .uri("/branches") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&create).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(create_status, StatusCode::OK); - - let (delete_status, delete_body) = json_response( - &app, - Request::builder() - .uri("/branches/feature") - .method(Method::DELETE) - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(delete_status, StatusCode::OK); - assert_eq!(delete_body["name"], "feature"); - - let (list_status, list_body) = json_response( - &app, - Request::builder() - .uri("/branches") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(list_status, StatusCode::OK); - assert_eq!(list_body["branches"], json!(["main"])); -} - -#[tokio::test(flavor = "multi_thread")] -async fn branch_delete_denies_without_policy_permission() { - let (temp, app) = app_for_loaded_graph_with_auth_tokens_and_policy( - &[("act-andrew", "token-admin"), ("act-bruno", "token-team")], - POLICY_YAML, - ) - .await; - let graph = graph_path(temp.path()); - - let mut db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.branch_create_from(ReadTarget::branch("main"), "feature") - .await - .unwrap(); - drop(db); - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/branches/feature") - .method(Method::DELETE) - .header("authorization", "Bearer token-team") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::FORBIDDEN); - assert!( - body["error"] - .as_str() - .unwrap() - .contains("policy denied action 'branch_delete'") - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn server_opens_s3_graph_directly_and_serves_snapshot_and_read() { - let Some(uri) = s3_test_graph_uri("server") else { - eprintln!("skipping s3 server test: OMNIGRAPH_S3_TEST_BUCKET is not set"); - return; - }; - - Omnigraph::init(&uri, &fs::read_to_string(fixture("test.pg")).unwrap()) - .await - .unwrap(); - let mut db = Omnigraph::open(&uri).await.unwrap(); - load_jsonl( - &mut db, - &fs::read_to_string(fixture("test.jsonl")).unwrap(), - LoadMode::Overwrite, - ) - .await - .unwrap(); - - let app = build_app( - AppState::open_with_bearer_token(uri.clone(), Some("s3-token".to_string())) - .await - .unwrap(), - ); - - let (snapshot_status, snapshot_body) = json_response( - &app, - Request::builder() - .uri("/snapshot") - .method(Method::GET) - .header("authorization", "Bearer s3-token") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(snapshot_status, StatusCode::OK); - assert!(snapshot_body["tables"].is_array()); - - let read = ReadRequest { - query_source: fs::read_to_string(fixture("test.gq")).unwrap(), - query_name: Some("get_person".to_string()), - params: Some(json!({ "name": "Alice" })), - branch: Some("main".to_string()), - snapshot: None, - }; - let (read_status, read_body) = json_response( - &app, - Request::builder() - .uri("/read") - .method(Method::POST) - .header("authorization", "Bearer s3-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&read).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(read_status, StatusCode::OK); - assert_eq!(read_body["row_count"], 1); - assert_eq!(read_body["rows"][0]["p.name"], "Alice"); -} - -#[tokio::test(flavor = "multi_thread")] -#[serial] -async fn remote_read_embeds_string_nearest_queries_with_mock_runtime() { - const EMBED_SCHEMA: &str = r#" -node Doc { - slug: String @key - title: String @index - embedding: Vector(4) @index -} -"#; - const EMBED_QUERY: &str = r#" -query vector_search_string($q: String) { - match { $d: Doc } - return { $d.slug, $d.title } - order { nearest($d.embedding, $q) } - limit 3 -} -"#; - - let alpha = mock_embedding("alpha", 4); - let beta = mock_embedding("beta", 4); - let gamma = mock_embedding("gamma", 4); - let data = format!( - concat!( - r#"{{"type":"Doc","data":{{"slug":"alpha-doc","title":"alpha guide","embedding":[{}]}}}}"#, - "\n", - r#"{{"type":"Doc","data":{{"slug":"beta-doc","title":"beta guide","embedding":[{}]}}}}"#, - "\n", - r#"{{"type":"Doc","data":{{"slug":"gamma-doc","title":"gamma handbook","embedding":[{}]}}}}"# - ), - format_vector(&alpha), - format_vector(&beta), - format_vector(&gamma), - ); - - let _guard = EnvGuard::set(&[ - ("OMNIGRAPH_EMBEDDINGS_MOCK", Some("1")), - ("GEMINI_API_KEY", None), - ]); - let temp = init_graph_with_schema_and_data(EMBED_SCHEMA, &data).await; - let graph = graph_path(temp.path()); - let state = AppState::open(graph.to_string_lossy().to_string()) - .await - .unwrap(); - let app = build_app(state); - - let read = ReadRequest { - query_source: EMBED_QUERY.to_string(), - query_name: Some("vector_search_string".to_string()), - params: Some(json!({ "q": "alpha" })), - branch: Some("main".to_string()), - snapshot: None, - }; - let (status, body) = json_response( - &app, - Request::builder() - .uri("/read") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&read).unwrap())) - .unwrap(), - ) - .await; - - assert_eq!(status, StatusCode::OK); - assert_eq!(body["row_count"], 3); - assert_eq!(body["rows"][0]["d.slug"], "alpha-doc"); -} - -#[tokio::test(flavor = "multi_thread")] -async fn change_conflict_returns_manifest_conflict_409() { - // A write that races with another writer surfaces as HTTP 409 with - // a structured `manifest_conflict` body — `table_key`, `expected`, - // and `actual` — so clients can detect-and-retry without parsing - // the message. - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - - // Build the server first so its handle pins the pre-mutation manifest - // version. Then advance the manifest from outside the server. The - // server's next /change call will capture stale `expected_versions` - // (from its still-pinned snapshot) and the publisher's CAS rejects. - let state = AppState::open(graph.to_string_lossy().to_string()) - .await - .unwrap(); - let app = build_app(state); - - { - let mut db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.mutate( - "main", - MUTATION_QUERIES, - "set_age", - &omnigraph_compiler::json_params_to_param_map( - Some(&json!({"name": "Alice", "age": 31 })), - &omnigraph_compiler::find_named_query(MUTATION_QUERIES, "set_age") - .unwrap() - .params, - omnigraph_compiler::JsonParamMode::Standard, - ) - .unwrap(), - ) - .await - .unwrap(); - } - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from( - serde_json::to_vec(&ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("set_age".to_string()), - params: Some(json!({ "name": "Alice", "age": 33 })), - branch: Some("main".to_string()), - }) - .unwrap(), - )) - .unwrap(), - ) - .await; - - assert_eq!(status, StatusCode::CONFLICT); - let error: ErrorOutput = serde_json::from_value(body).unwrap(); - assert_eq!(error.code, Some(omnigraph_server::api::ErrorCode::Conflict)); - let conflict = error - .manifest_conflict - .expect("publisher CAS rejection must populate manifest_conflict body"); - assert_eq!(conflict.table_key, "node:Person"); - assert!( - conflict.actual > conflict.expected, - "actual ({}) should be ahead of expected ({})", - conflict.actual, - conflict.expected, - ); -} - -#[tokio::test(flavor = "multi_thread", worker_threads = 4)] -async fn change_concurrent_inserts_same_key_serialize_without_409() { - // PR 2 Phase 2 (MR-686): pin the design fix for the same-key - // concurrency hazard. Pre-fix, in-process concurrent inserts on - // the same `(table, branch)` rejected with 409 manifest_conflict - // because `ensure_expected_version` fired before the per-table - // queue was acquired and saw Lance HEAD already advanced by a - // peer writer. Post-fix, Insert/Merge skip the strict pre-stage - // check (see `MutationOpKind::strict_pre_stage_version_check`); - // the queue serializes commit_staged; Lance's natural rebase - // handles the in-flight stage; the publisher's CAS on a fresh - // per-branch snapshot under the queue catches genuine cross- - // process drift. - // - // This test spawns N concurrent /change inserts on a single - // node type and asserts: every request returns 200 (no 409), - // and the final row count equals the seed count + N (every - // staged batch actually committed). - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let state = AppState::open(graph.to_string_lossy().to_string()) - .await - .unwrap(); - let app = build_app(state); - - // test.jsonl seeds 4 Persons (Alice, Bob, Charlie, Diana). - const SEED_PERSON_ROWS: u64 = 4; - const N: usize = 12; - - let mut handles = Vec::with_capacity(N); - for i in 0..N { - let app = app.clone(); - handles.push(tokio::spawn(async move { - let body = serde_json::to_vec(&ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": format!("racer-{i}"), "age": i as i32 })), - branch: Some("main".to_string()), - }) - .unwrap(); - let req = Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(); - let response = app.oneshot(req).await.unwrap(); - response.status() - })); - } - - let mut statuses = Vec::with_capacity(N); - for h in handles { - statuses.push(h.await.unwrap()); - } - - let bad: Vec<_> = statuses - .iter() - .enumerate() - .filter(|(_, s)| **s != StatusCode::OK) - .collect(); - assert!( - bad.is_empty(), - "expected every concurrent insert to return 200, got non-200 for: {:?}", - bad - ); - - // Verify the inserts actually landed. The status check above only proves - // the publisher CAS didn't reject; the row count proves none of the - // concurrent commits silently overwrote a peer. - let (snapshot_status, snapshot_body) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(snapshot_status, StatusCode::OK); - let person_rows = snapshot_body["tables"] - .as_array() - .and_then(|tables| { - tables - .iter() - .find(|t| t["table_key"].as_str() == Some("node:Person")) - }) - .and_then(|t| t["row_count"].as_u64()) - .expect("snapshot must include node:Person row_count"); - assert_eq!( - person_rows, - SEED_PERSON_ROWS + N as u64, - "expected {} seeded + {} concurrent inserts = {} Person rows; got {}", - SEED_PERSON_ROWS, - N, - SEED_PERSON_ROWS + N as u64, - person_rows, - ); -} - -#[tokio::test(flavor = "multi_thread", worker_threads = 4)] -async fn change_concurrent_updates_same_key_serialize_via_publisher_cas() { - // Pin Update RYW semantics under in-process concurrency on the same - // `(table, branch)`. With per-table queue serialization and op-kind-aware - // drift detection at commit time, exactly one of N concurrent UPDATEs - // on the same row commits; the rest are rejected as 409 manifest_conflict. - // - // Pre-fix bug class: in `MutationStaging::commit_all`, after queue - // acquisition, the staged Lance transaction is handed straight to - // `commit_staged`. For a writer whose staged dataset is at V0 but - // Lance HEAD has advanced to V1 (because the queue's prior winner - // already published), Lance's transaction conflict resolver fires - // `RetryableCommitConflict` on Update vs Update on the same row. - // That error gets wrapped as `OmniError::Lance(<string>)` and the - // API surfaces it as **500 internal**, not 409. Users see "internal - // server error" instead of a retryable conflict, breaking the - // documented 409 contract for in-process drift. - // - // Post-fix invariant: `commit_all` does an op-kind-aware drift check - // before each `commit_staged`. For tables whose tracked op_kind has - // `strict_pre_stage_version_check() == true` (Update / Delete / - // SchemaRewrite), if the staged dataset's version doesn't match the - // fresh manifest pin, return `OmniError::manifest_expected_version_mismatch` - // → 409 ExpectedVersionMismatch. The N-1 losers see a clean 409 - // before Lance's commit_staged ever runs. - // - // Why correct-by-design: closing the class "Lance internal conflict - // surfaces as 500 instead of 409" rather than mapping the specific - // Lance error variant. The drift check fires at the right architectural - // layer (engine boundary, under the queue) and respects the existing - // `MutationOpKind` policy. - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let state = AppState::open(graph.to_string_lossy().to_string()) - .await - .unwrap(); - let app = build_app(state); - - // Spawn N=8 concurrent UPDATEs on Alice (from test.jsonl, age=30 at V0) - // writing distinct ages. - const N: usize = 8; - let mut handles = Vec::with_capacity(N); - for i in 0..N { - let app = app.clone(); - let target_age = 100 + i as i32; - handles.push(tokio::spawn(async move { - let body = serde_json::to_vec(&ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("set_age".to_string()), - params: Some(json!({ "name": "Alice", "age": target_age })), - branch: Some("main".to_string()), - }) - .unwrap(); - let req = Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(); - let response = app.oneshot(req).await.unwrap(); - let status = response.status(); - let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); - (status, body.to_vec()) - })); - } - - let mut results = Vec::with_capacity(N); - for h in handles { - results.push(h.await.unwrap()); - } - let statuses: Vec<StatusCode> = results.iter().map(|(s, _)| *s).collect(); - - let ok_count = statuses.iter().filter(|s| **s == StatusCode::OK).count(); - let conflict_count = statuses - .iter() - .filter(|s| **s == StatusCode::CONFLICT) - .count(); - let other: Vec<_> = statuses - .iter() - .enumerate() - .filter(|(_, s)| **s != StatusCode::OK && **s != StatusCode::CONFLICT) - .collect(); - - let other_bodies: Vec<(usize, StatusCode, String)> = other - .iter() - .map(|(i, s)| { - let body_str = String::from_utf8_lossy(&results[*i].1).to_string(); - (*i, **s, body_str) - }) - .collect(); - assert!( - other.is_empty(), - "expected only 200 or 409 statuses, got non-200/409 entries: {:?}", - other_bodies - ); - assert_eq!( - ok_count + conflict_count, - N, - "all responses must be 200 or 409 to satisfy the RYW invariant; statuses: {:?}", - statuses - ); - assert_eq!( - ok_count, - 1, - "expected exactly one update to commit and N-1 to receive 409 manifest_conflict \ - (op-kind-aware drift check rejects stale-V0 staged datasets at commit_all entry). \ - Got {} OK + {} 409 + {} other. \ - Pre-fix symptom: 1 OK + (N-1) x 500 because Lance's RetryableCommitConflict for \ - Update vs Update on the same row bubbles up as `OmniError::Lance(<string>)` and \ - the API maps it to 500 internal, not 409. Statuses: {:?}", - ok_count, - conflict_count, - statuses.len() - ok_count - conflict_count, - statuses, - ); -} - -// ───────────────────────────────────────────────────────────────────────── -// Branch-ops morphological matrix -// -// Table-driven test covering all interesting (op_a, op_b, target_overlap) -// concurrent-pair cells with the C1-C6 invariants asserted uniformly: -// -// C1 — both complete (no deadlock, no hang) -// C2 — status: both 200, or exactly one clean conflict (409/429), no 500 -// C3 — per-target row count -// C4 — per-target row identity (present + absent named persons) -// C5 — engine state remains coherent (subsequent /snapshot is consistent) -// C6 — post-op /change on main succeeds (engine state isn't poisoned) -// -// Cell list (a-k) below. Each cell uses a fresh tempdir + AppState so a -// failure in one doesn't leak into the next. Within a cell, ops align at -// a tokio::sync::Barrier so both reach the engine close in time, and the -// pair is wrapped in tokio::time::timeout(15s) so a deadlock surfaces -// as a clean panic. -// -// Replaces the three narrow concurrent_branch_* tests below; their -// scenarios are folded into cells f, h, i (branch_create_from race), -// cell a (merge race with C4 identity assertions), and cell d -// (concurrent change-during-merge). -// ───────────────────────────────────────────────────────────────────────── - -mod matrix { - use super::*; - use std::time::Duration; - use tokio::sync::Barrier; - - #[derive(Debug)] - pub(super) struct OpStatus { - pub status: StatusCode, - pub body: Vec<u8>, - } - - pub(super) struct Harness { - pub _temp: tempfile::TempDir, - pub app: Router, - } - - impl Harness { - pub async fn new() -> Self { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - // Build the WorkloadController explicitly with defaults rather - // than letting `AppState::open` call - // `WorkloadController::from_env()`. The admission-gate test - // (`ingest_per_actor_admission_cap_returns_429`) sets - // OMNIGRAPH_PER_ACTOR_INFLIGHT_MAX=1 inside an EnvGuard while - // it runs. Process-wide env vars are visible to - // concurrently-running tests; if a matrix cell reads env at - // AppState construction time during that window it picks up - // cap=1 and the second concurrent merge in cell b surfaces - // 429 instead of the expected 200. Constructing the - // controller here with explicit defaults makes cells - // independent of any env mutation other tests perform. - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - let workload = omnigraph_server::workload::WorkloadController::with_defaults(); - let state = AppState::new_with_workload( - graph.to_string_lossy().to_string(), - db, - Vec::new(), - workload, - ); - let app = build_app(state); - Self { _temp: temp, app } - } - - pub async fn create_branch(&self, from: &str, name: &str) { - let body = serde_json::to_vec(&BranchCreateRequest { - from: Some(from.to_string()), - name: name.to_string(), - }) - .unwrap(); - let r = self - .app - .clone() - .oneshot( - Request::builder() - .uri("/branches") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!( - r.status(), - StatusCode::OK, - "setup create_branch {} from {} failed", - name, - from - ); - } - - pub async fn insert_person(&self, branch: &str, name: &str, age: i32) { - let body = serde_json::to_vec(&ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": name, "age": age })), - branch: Some(branch.to_string()), - }) - .unwrap(); - let r = self - .app - .clone() - .oneshot( - Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!( - r.status(), - StatusCode::OK, - "setup insert {} on {} failed", - name, - branch - ); - } - - /// Run two ops concurrently with barrier alignment + 15s deadlock - /// timeout. Returns `(op_a, op_b)`. Panics on timeout. - pub async fn run_pair( - &self, - op_a: impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus>, - op_b: impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus>, - ) -> (OpStatus, OpStatus) { - let barrier = Arc::new(Barrier::new(2)); - let h_a = op_a(self.app.clone(), Arc::clone(&barrier)); - let h_b = op_b(self.app.clone(), Arc::clone(&barrier)); - let result = tokio::time::timeout(Duration::from_secs(15), async { - let a = h_a.await.unwrap(); - let b = h_b.await.unwrap(); - (a, b) - }) - .await; - result.expect("concurrent op pair deadlocked (>15s)") - } - - pub async fn person_count(&self, branch: &str) -> u64 { - let r = self - .app - .clone() - .oneshot( - Request::builder() - .uri(format!("/snapshot?branch={}", branch)) - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(r.status(), StatusCode::OK, "snapshot {} failed", branch); - let body = to_bytes(r.into_body(), usize::MAX).await.unwrap(); - let v: Value = serde_json::from_slice(&body).unwrap(); - v["tables"] - .as_array() - .and_then(|tables| { - tables - .iter() - .find(|t| t["table_key"].as_str() == Some("node:Person")) - }) - .and_then(|t| t["row_count"].as_u64()) - .unwrap_or_else(|| panic!("snapshot {} missing node:Person", branch)) - } - - /// True iff the named Person exists on `branch`. Uses the - /// `get_person` query from `test.gq` for identity rather than - /// just count. - pub async fn person_exists(&self, branch: &str, name: &str) -> bool { - let body = serde_json::to_vec(&ReadRequest { - query_source: include_str!("../../omnigraph/tests/fixtures/test.gq").to_string(), - query_name: Some("get_person".to_string()), - params: Some(json!({ "name": name })), - branch: Some(branch.to_string()), - snapshot: None, - }) - .unwrap(); - let r = self - .app - .clone() - .oneshot( - Request::builder() - .uri("/read") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!( - r.status(), - StatusCode::OK, - "person_exists query for {} on {} failed", - name, - branch - ); - let body = to_bytes(r.into_body(), usize::MAX).await.unwrap(); - let v: Value = serde_json::from_slice(&body).unwrap(); - v["row_count"].as_u64().unwrap_or(0) > 0 - } - - /// Asserts each name in `present` exists on `branch` and each in - /// `absent` does not. Identity-grade check that catches symmetric - /// swap races a row-count assertion would miss. - pub async fn assert_persons( - &self, - branch: &str, - cell: &str, - present: &[&str], - absent: &[&str], - ) { - for name in present { - assert!( - self.person_exists(branch, name).await, - "[{}] expected {} to be present on {}", - cell, - name, - branch - ); - } - for name in absent { - assert!( - !self.person_exists(branch, name).await, - "[{}] expected {} to be absent from {}", - cell, - name, - branch - ); - } - } - - /// C6: insert a uniquely-named sentinel on main and verify it - /// landed. Catches engine-state poisoning where a cell's - /// concurrent ops left the engine half-broken — subsequent - /// /change either deadlocks or returns a non-200. - pub async fn assert_post_op_sentinel(&self, cell: &str, sentinel: &str) { - let body = serde_json::to_vec(&ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": sentinel, "age": 99 })), - branch: Some("main".to_string()), - }) - .unwrap(); - let r = self - .app - .clone() - .oneshot( - Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!( - r.status(), - StatusCode::OK, - "[{}] post-op sentinel /change on main failed (engine poisoned?)", - cell - ); - assert!( - self.person_exists("main", sentinel).await, - "[{}] sentinel {} did not land on main", - cell, - sentinel - ); - } - } - - // Helpers that build the closures for `run_pair`. Each takes a - // Router + Barrier and returns a JoinHandle yielding the status/body. - - pub(super) fn op_merge( - source: String, - target: String, - ) -> impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus> { - move |app: Router, barrier: Arc<Barrier>| { - tokio::spawn(async move { - barrier.wait().await; - let body = serde_json::to_vec(&BranchMergeRequest { - source, - target: Some(target), - }) - .unwrap(); - let response = app - .oneshot( - Request::builder() - .uri("/branches/merge") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(), - ) - .await - .unwrap(); - let status = response.status(); - let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); - OpStatus { - status, - body: body.to_vec(), - } - }) - } - } - - pub(super) fn op_change_insert( - branch: String, - name: String, - age: i32, - ) -> impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus> { - move |app: Router, barrier: Arc<Barrier>| { - tokio::spawn(async move { - barrier.wait().await; - let body = serde_json::to_vec(&ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": name, "age": age })), - branch: Some(branch), - }) - .unwrap(); - let response = app - .oneshot( - Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(), - ) - .await - .unwrap(); - let status = response.status(); - let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); - OpStatus { - status, - body: body.to_vec(), - } - }) - } - } - - pub(super) fn op_branch_create( - from: String, - name: String, - ) -> impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus> { - move |app: Router, barrier: Arc<Barrier>| { - tokio::spawn(async move { - barrier.wait().await; - let body = serde_json::to_vec(&BranchCreateRequest { - from: Some(from), - name, - }) - .unwrap(); - let response = app - .oneshot( - Request::builder() - .uri("/branches") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(), - ) - .await - .unwrap(); - let status = response.status(); - let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); - OpStatus { - status, - body: body.to_vec(), - } - }) - } - } - - pub(super) fn op_branch_delete( - name: String, - ) -> impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus> { - move |app: Router, barrier: Arc<Barrier>| { - tokio::spawn(async move { - barrier.wait().await; - let response = app - .oneshot( - Request::builder() - .uri(format!("/branches/{}", name)) - .method(Method::DELETE) - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - let status = response.status(); - let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); - OpStatus { - status, - body: body.to_vec(), - } - }) - } - } -} - -#[tokio::test(flavor = "multi_thread", worker_threads = 4)] -async fn concurrent_branch_ops_morphological_matrix() { - // Cell a: Merge × Merge, distinct targets. - // Pre-fix on b09a097/22d76db: branch_merge_impl's swap-restore race - // landed feature_a's content in target_b instead of target_a (and - // vice versa — symmetric swap). Identity asserts catch both - // asymmetric and symmetric variants. - { - let cell = "a:merge×merge:distinct-targets"; - let h = matrix::Harness::new().await; - h.create_branch("main", "feature-a-cella").await; - h.insert_person("feature-a-cella", "EveA-cella", 22).await; - h.create_branch("main", "feature-b-cella").await; - h.insert_person("feature-b-cella", "FrankB-cella", 33).await; - h.create_branch("main", "target-a-cella").await; - h.create_branch("main", "target-b-cella").await; - - let (sa, sb) = h - .run_pair( - matrix::op_merge("feature-a-cella".to_string(), "target-a-cella".to_string()), - matrix::op_merge("feature-b-cella".to_string(), "target-b-cella".to_string()), - ) - .await; - assert_eq!(sa.status, StatusCode::OK, "[{}] merge a", cell); - assert_eq!(sb.status, StatusCode::OK, "[{}] merge b", cell); - h.assert_persons("target-a-cella", cell, &["EveA-cella"], &["FrankB-cella"]) - .await; - h.assert_persons("target-b-cella", cell, &["FrankB-cella"], &["EveA-cella"]) - .await; - h.assert_post_op_sentinel(cell, "sentinel-cella").await; - } - - // Cell b: Merge × Merge, same target / distinct sources. - // Both want to land in main. merge_exclusive serializes; both should - // succeed and main should contain BOTH sources' contributions. - { - let cell = "b:merge×merge:same-target-distinct-sources"; - let h = matrix::Harness::new().await; - h.create_branch("main", "src-x-cellb").await; - h.insert_person("src-x-cellb", "Xavier-cellb", 41).await; - h.create_branch("main", "src-y-cellb").await; - h.insert_person("src-y-cellb", "Yvonne-cellb", 42).await; - - let (sa, sb) = h - .run_pair( - matrix::op_merge("src-x-cellb".to_string(), "main".to_string()), - matrix::op_merge("src-y-cellb".to_string(), "main".to_string()), - ) - .await; - assert_eq!(sa.status, StatusCode::OK, "[{}] merge x", cell); - assert_eq!(sb.status, StatusCode::OK, "[{}] merge y", cell); - h.assert_persons("main", cell, &["Xavier-cellb", "Yvonne-cellb"], &[]) - .await; - h.assert_post_op_sentinel(cell, "sentinel-cellb").await; - } - - // Cell c: Merge × Merge, same source / distinct targets (fanout). - // One source merged into two targets simultaneously. merge_exclusive - // serializes; both targets should reflect the source's content. - { - let cell = "c:merge×merge:same-source-distinct-targets"; - let h = matrix::Harness::new().await; - h.create_branch("main", "src-shared-cellc").await; - h.insert_person("src-shared-cellc", "Sharon-cellc", 50) - .await; - h.create_branch("main", "tgt-1-cellc").await; - h.create_branch("main", "tgt-2-cellc").await; - - let (sa, sb) = h - .run_pair( - matrix::op_merge("src-shared-cellc".to_string(), "tgt-1-cellc".to_string()), - matrix::op_merge("src-shared-cellc".to_string(), "tgt-2-cellc".to_string()), - ) - .await; - assert_eq!(sa.status, StatusCode::OK, "[{}] merge into tgt-1", cell); - assert_eq!(sb.status, StatusCode::OK, "[{}] merge into tgt-2", cell); - h.assert_persons("tgt-1-cellc", cell, &["Sharon-cellc"], &[]) - .await; - h.assert_persons("tgt-2-cellc", cell, &["Sharon-cellc"], &[]) - .await; - h.assert_post_op_sentinel(cell, "sentinel-cellc").await; - } - - // Cell d: Merge × Change, both touching main. C2 permits both - // succeed, or exactly one clean 409 if the merge detects target - // movement after planning but before acquiring the queue. - { - let cell = "d:merge×change:into-target"; - let h = matrix::Harness::new().await; - h.create_branch("main", "feature-celld").await; - h.insert_person("feature-celld", "EveD-celld", 22).await; - - let (sa, sb) = h - .run_pair( - matrix::op_merge("feature-celld".to_string(), "main".to_string()), - matrix::op_change_insert("main".to_string(), "FrankD-celld".to_string(), 33), - ) - .await; - assert_eq!(sb.status, StatusCode::OK, "[{}] change", cell); - assert!( - sa.status == StatusCode::OK || sa.status == StatusCode::CONFLICT, - "[{}] merge must be 200 or clean 409, got {}", - cell, - sa.status - ); - if sa.status == StatusCode::OK { - h.assert_persons("main", cell, &["EveD-celld", "FrankD-celld"], &[]) - .await; - } else { - let error: ErrorOutput = serde_json::from_slice(&sa.body).unwrap(); - let conflict = error - .manifest_conflict - .expect("merge 409 must include manifest_conflict"); - assert_eq!( - conflict.table_key, "node:Person", - "[{}] conflict table", - cell - ); - h.assert_persons("main", cell, &["FrankD-celld"], &["EveD-celld"]) - .await; - } - h.assert_post_op_sentinel(cell, "sentinel-celld").await; - } - - // Cell e: Merge × BranchCreateFrom-target. Concurrent fork off the - // merge target while the merge runs. Both should succeed; the new - // branch should have a coherent view (either pre- or post-merge, - // both valid). After both, target = main has the merged content. - { - let cell = "e:merge×branch_create_from:target"; - let h = matrix::Harness::new().await; - h.create_branch("main", "src-celle").await; - h.insert_person("src-celle", "Eve-celle", 22).await; - - let (sa, sb) = h - .run_pair( - matrix::op_merge("src-celle".to_string(), "main".to_string()), - matrix::op_branch_create("main".to_string(), "fork-celle".to_string()), - ) - .await; - assert_eq!(sa.status, StatusCode::OK, "[{}] merge", cell); - assert_eq!(sb.status, StatusCode::OK, "[{}] branch_create_from", cell); - // Main definitely has Eve. - h.assert_persons("main", cell, &["Eve-celle"], &[]).await; - // fork-celle was forked off main at SOME version; main's current - // count is 5 (4 seeded + Eve). fork-celle has either 4 (pre-merge - // snapshot) or 5 (post-merge snapshot); both are valid timings. - let fork_count = h.person_count("fork-celle").await; - assert!( - fork_count == 4 || fork_count == 5, - "[{}] fork-celle row count must be pre- or post-merge view (4 or 5), got {}", - cell, - fork_count - ); - h.assert_post_op_sentinel(cell, "sentinel-celle").await; - } - - // Cell f: BranchCreateFrom × BranchCreateFrom, distinct parents. - // Pre-fix on f925ad1: swap-restore race in branch_create_from_impl - // forked the new branch off the wrong parent. Identity asserts pin - // that fork-from-A inherits A's content, fork-from-B inherits B's. - { - let cell = "f:branch_create_from×branch_create_from:distinct-parents"; - let h = matrix::Harness::new().await; - h.create_branch("main", "alpha-cellf").await; - h.insert_person("alpha-cellf", "Eve-cellf", 22).await; - h.create_branch("main", "beta-cellf").await; - - let (sa, sb) = h - .run_pair( - matrix::op_branch_create("alpha-cellf".to_string(), "gamma-cellf".to_string()), - matrix::op_branch_create("beta-cellf".to_string(), "delta-cellf".to_string()), - ) - .await; - assert_eq!(sa.status, StatusCode::OK, "[{}] gamma create", cell); - assert_eq!(sb.status, StatusCode::OK, "[{}] delta create", cell); - // gamma forks off alpha → must contain Eve. - h.assert_persons("gamma-cellf", cell, &["Eve-cellf"], &[]) - .await; - // delta forks off beta → must NOT contain Eve. - h.assert_persons("delta-cellf", cell, &[], &["Eve-cellf"]) - .await; - h.assert_post_op_sentinel(cell, "sentinel-cellf").await; - } - - // Cell g: BranchCreateFrom × BranchDelete, unrelated branches. - // Disjoint branches; both should complete cleanly without - // interference. - { - let cell = "g:branch_create_from×branch_delete:unrelated"; - let h = matrix::Harness::new().await; - h.create_branch("main", "doomed-cellg").await; - - let (sa, sb) = h - .run_pair( - matrix::op_branch_create("main".to_string(), "newborn-cellg".to_string()), - matrix::op_branch_delete("doomed-cellg".to_string()), - ) - .await; - assert_eq!(sa.status, StatusCode::OK, "[{}] create newborn", cell); - assert_eq!(sb.status, StatusCode::OK, "[{}] delete doomed", cell); - // newborn-cellg exists with main's content. - h.assert_persons("newborn-cellg", cell, &["Alice"], &[]) - .await; - h.assert_post_op_sentinel(cell, "sentinel-cellg").await; - } - - // Cell h: BranchDelete × BranchDelete, distinct branches. Both call - // refresh() internally; verify no deadlock and both deletes land. - { - let cell = "h:branch_delete×branch_delete:distinct"; - let h = matrix::Harness::new().await; - h.create_branch("main", "doomed1-cellh").await; - h.create_branch("main", "doomed2-cellh").await; - - let (sa, sb) = h - .run_pair( - matrix::op_branch_delete("doomed1-cellh".to_string()), - matrix::op_branch_delete("doomed2-cellh".to_string()), - ) - .await; - assert_eq!(sa.status, StatusCode::OK, "[{}] delete 1", cell); - assert_eq!(sb.status, StatusCode::OK, "[{}] delete 2", cell); - // Verify both gone via /branches list (snapshot would still work - // for a deleted branch via parent fallback in some paths, so we - // use the explicit list). - let r = h - .app - .clone() - .oneshot( - Request::builder() - .uri("/branches") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(r.status(), StatusCode::OK); - let body = to_bytes(r.into_body(), usize::MAX).await.unwrap(); - let list_body: Value = serde_json::from_slice(&body).unwrap(); - let branches: Vec<&str> = list_body["branches"] - .as_array() - .unwrap() - .iter() - .filter_map(|v| v.as_str()) - .collect(); - assert!( - !branches.contains(&"doomed1-cellh"), - "[{}] doomed1 still in branch list: {:?}", - cell, - branches - ); - assert!( - !branches.contains(&"doomed2-cellh"), - "[{}] doomed2 still in branch list: {:?}", - cell, - branches - ); - h.assert_post_op_sentinel(cell, "sentinel-cellh").await; - } - - // Cell i: BranchDelete × Change, on a different branch. Delete one - // branch while a /change runs on main. Both should succeed. - { - let cell = "i:branch_delete×change:distinct-branch"; - let h = matrix::Harness::new().await; - h.create_branch("main", "doomed-celli").await; - - let (sa, sb) = h - .run_pair( - matrix::op_branch_delete("doomed-celli".to_string()), - matrix::op_change_insert("main".to_string(), "Pat-celli".to_string(), 44), - ) - .await; - assert_eq!(sa.status, StatusCode::OK, "[{}] delete", cell); - assert_eq!(sb.status, StatusCode::OK, "[{}] change", cell); - h.assert_persons("main", cell, &["Pat-celli"], &[]).await; - h.assert_post_op_sentinel(cell, "sentinel-celli").await; - } - - // Cell j: BranchCreateFrom × Change, both on main. The fork timing - // determines whether the new branch sees the change (pre or post). - // Both valid. Main must contain the inserted row. - { - let cell = "j:branch_create_from×change:on-source"; - let h = matrix::Harness::new().await; - - let (sa, sb) = h - .run_pair( - matrix::op_branch_create("main".to_string(), "twin-cellj".to_string()), - matrix::op_change_insert("main".to_string(), "Quincy-cellj".to_string(), 55), - ) - .await; - assert_eq!(sa.status, StatusCode::OK, "[{}] branch_create", cell); - assert_eq!(sb.status, StatusCode::OK, "[{}] change", cell); - h.assert_persons("main", cell, &["Quincy-cellj"], &[]).await; - // twin-cellj has either pre-change view (no Quincy) or - // post-change view (with Quincy); either is valid. - let twin_has_quincy = h.person_exists("twin-cellj", "Quincy-cellj").await; - let _ = twin_has_quincy; // either valid timing — just ensure no panic - h.assert_post_op_sentinel(cell, "sentinel-cellj").await; - } - - // Cell k: reopen consistency. Run a representative concurrent pair, - // drop the engine, reopen on a separate handle, verify state matches. - { - let cell = "k:reopen-after-pair"; - let h = matrix::Harness::new().await; - h.create_branch("main", "src-cellk").await; - h.insert_person("src-cellk", "Rita-cellk", 36).await; - - let (sa, sb) = h - .run_pair( - matrix::op_merge("src-cellk".to_string(), "main".to_string()), - matrix::op_change_insert("main".to_string(), "Steve-cellk".to_string(), 37), - ) - .await; - assert_eq!(sb.status, StatusCode::OK, "[{}] change", cell); - assert!( - sa.status == StatusCode::OK || sa.status == StatusCode::CONFLICT, - "[{}] merge must be 200 or clean 409, got {}", - cell, - sa.status - ); - if sa.status == StatusCode::OK { - h.assert_persons("main", cell, &["Rita-cellk", "Steve-cellk"], &[]) - .await; - } else { - let error: ErrorOutput = serde_json::from_slice(&sa.body).unwrap(); - let conflict = error - .manifest_conflict - .expect("merge 409 must include manifest_conflict"); - assert_eq!( - conflict.table_key, "node:Person", - "[{}] conflict table", - cell - ); - h.assert_persons("main", cell, &["Steve-cellk"], &["Rita-cellk"]) - .await; - } - - // Reopen via a fresh AppState on the same graph. - let graph_uri = format!("{}/server.omni", h._temp.path().display()); - let reopened = AppState::open(graph_uri.clone()).await.unwrap(); - let app2 = build_app(reopened); - // Sanity: the same identity check via the new app must see - // Rita and Steve. - let r = app2 - .clone() - .oneshot( - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(r.status(), StatusCode::OK, "[{}] reopen snapshot", cell); - let body = to_bytes(r.into_body(), usize::MAX).await.unwrap(); - let v: Value = serde_json::from_slice(&body).unwrap(); - let person_rows = v["tables"] - .as_array() - .and_then(|tables| { - tables - .iter() - .find(|t| t["table_key"].as_str() == Some("node:Person")) - }) - .and_then(|t| t["row_count"].as_u64()) - .expect("reopen snapshot must include node:Person row_count"); - let expected_rows = if sa.status == StatusCode::OK { 6 } else { 5 }; - assert_eq!( - person_rows, expected_rows, - "[{}] reopened main should include seed (4) + committed concurrent writes", - cell, - ); - } -} - -#[tokio::test(flavor = "multi_thread", worker_threads = 4)] -async fn change_disjoint_table_concurrency_succeeds_at_http_level() { - // HTTP-level pin for MR-686's disjoint-table promise: concurrent /change - // requests touching different node types must coexist without admission - // rejection or publisher-CAS conflict. The bench harness measures - // throughput; this test is the regression sentinel that catches a - // future change which accidentally re-introduces graph-wide - // serialization on the disjoint path. - // - // Setup: test.jsonl seeds 4 Persons + 2 Companies. Spawn N=4 concurrent - // /change inserts on `node:Person` and N=4 concurrent inserts on - // `node:Company`. All 8 must return 200, and the post-test row counts - // must reflect every insert. - const PERSON_QUERY: &str = r#" -query insert_p($name: String, $age: I32) { - insert Person { name: $name, age: $age } -} -"#; - const COMPANY_QUERY: &str = r#" -query insert_c($name: String) { - insert Company { name: $name } -} -"#; - const SEED_PERSONS: u64 = 4; - const SEED_COMPANIES: u64 = 2; - const PER_TYPE: usize = 4; - - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let state = AppState::open(graph.to_string_lossy().to_string()) - .await - .unwrap(); - let app = build_app(state); - - let mut handles = Vec::with_capacity(PER_TYPE * 2); - for i in 0..PER_TYPE { - let app_p = app.clone(); - handles.push(tokio::spawn(async move { - let body = serde_json::to_vec(&ChangeRequest { - query: PERSON_QUERY.to_string(), - name: Some("insert_p".to_string()), - params: Some(json!({ "name": format!("p-{i}"), "age": i as i32 })), - branch: Some("main".to_string()), - }) - .unwrap(); - let req = Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(); - app_p.oneshot(req).await.unwrap().status() - })); - let app_c = app.clone(); - handles.push(tokio::spawn(async move { - let body = serde_json::to_vec(&ChangeRequest { - query: COMPANY_QUERY.to_string(), - name: Some("insert_c".to_string()), - params: Some(json!({ "name": format!("c-{i}") })), - branch: Some("main".to_string()), - }) - .unwrap(); - let req = Request::builder() - .uri("/change") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(); - app_c.oneshot(req).await.unwrap().status() - })); - } - - let mut statuses = Vec::with_capacity(PER_TYPE * 2); - for h in handles { - statuses.push(h.await.unwrap()); - } - - let bad: Vec<_> = statuses - .iter() - .enumerate() - .filter(|(_, s)| **s != StatusCode::OK) - .collect(); - assert!( - bad.is_empty(), - "expected every disjoint /change insert to return 200, got non-200 for: {:?}", - bad, - ); - - // Verify both tables landed every insert. - let (status, body) = json_response( - &app, - Request::builder() - .uri("/snapshot?branch=main") - .method(Method::GET) - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - let lookup_count = |table_key: &str| -> u64 { - body["tables"] - .as_array() - .and_then(|tables| { - tables - .iter() - .find(|t| t["table_key"].as_str() == Some(table_key)) - }) - .and_then(|t| t["row_count"].as_u64()) - .unwrap_or_else(|| panic!("snapshot missing {}", table_key)) - }; - assert_eq!( - lookup_count("node:Person"), - SEED_PERSONS + PER_TYPE as u64, - "Person row count after concurrent inserts", - ); - assert_eq!( - lookup_count("node:Company"), - SEED_COMPANIES + PER_TYPE as u64, - "Company row count after concurrent inserts", - ); -} - -#[tokio::test(flavor = "multi_thread", worker_threads = 4)] -async fn ingest_per_actor_admission_cap_returns_429() { - // Pin the admission gate on `/ingest`. With per-actor in-flight cap of 1 - // and 8 concurrent requests from the same actor, at least one request - // must be rejected with HTTP 429 and `code: too_many_requests`. - // - // Pre-fix bug class: the admission pattern at `server_change` - // (`crates/omnigraph-server/src/lib.rs:932`) was the only handler - // that called `WorkloadController::try_admit`. A heavy actor sending - // bulk-ingest traffic would exhaust shared engine capacity (Lance I/O - // threads, manifest churn) without ever hitting an admission cap. - // Pinned at the HTTP boundary so future refactors that drop the - // try_admit call from a mutating handler turn this red. - // - // Post-fix invariant: `/ingest`, `/branches/create`, `/branches/delete`, - // `/branches/merge`, and `/schema/apply` all gate on - // `state.workload.try_admit(&actor_arc, est_bytes)` after Cedar - // authorization and before the engine call. Cap exhaustion surfaces as - // 429 with `code: too_many_requests`. - // - // Construct the WorkloadController directly with cap=1 instead of - // mutating `OMNIGRAPH_PER_ACTOR_INFLIGHT_MAX` via EnvGuard. Process-wide - // env vars are visible to concurrently-running tests; the previous - // `EnvGuard + #[serial]` pair leaked the override into any other test - // that called `AppState::open` during the guard's window - // (matrix CI failure on commit 99b0941). Using the explicit - // `AppState::new_with_workload` constructor closes that bug class — - // this test no longer mutates global state and no longer needs - // `#[serial]`. - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - let workload = omnigraph_server::workload::WorkloadController::new( - 1, // per-actor in-flight cap (the fixture under test) - 1_000_000_000, // per-actor byte budget — large so it never bottlenecks - ); - // MR-723: install a permit-all policy alongside the bearer token so - // /ingest (action=Change) passes Cedar evaluation. The test is - // exercising the admission cap, not policy — the policy is just - // enough to clear the State 3 path so the test reaches workload. - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, permit_all_policy_yaml(&["act-flooder"])).unwrap(); - let policy_engine = - omnigraph_server::PolicyEngine::load_graph(&policy_path, graph.to_string_lossy().as_ref()) - .unwrap(); - let state = AppState::new_single( - graph.to_string_lossy().to_string(), - db, - vec![("act-flooder".to_string(), "flooder-token".to_string())], - Some(policy_engine), - workload, - ); - let app = build_app(state); - let _temp = temp; - - // Eight concurrent ingests, all from act-flooder. Only one fits in a - // cap=1 in-flight semaphore; the others must 429. - const N: usize = 8; - let barrier = Arc::new(tokio::sync::Barrier::new(N)); - let mut handles = Vec::with_capacity(N); - for i in 0..N { - let app = app.clone(); - let barrier = Arc::clone(&barrier); - handles.push(tokio::spawn(async move { - // Align the 8 tasks at the barrier so they all attempt - // try_admit close in time. - barrier.wait().await; - - let body = serde_json::to_vec(&IngestRequest { - data: format!( - "{{\"type\":\"Person\",\"data\":{{\"name\":\"flooder-{i}\",\"age\":{i}}}}}\n" - ), - branch: Some("main".to_string()), - from: Some("main".to_string()), - mode: Some(omnigraph::loader::LoadMode::Merge), - }) - .unwrap(); - let req = Request::builder() - .uri("/ingest") - .method(Method::POST) - .header("authorization", "Bearer flooder-token") - .header("content-type", "application/json") - .body(Body::from(body)) - .unwrap(); - let response = app.oneshot(req).await.unwrap(); - let status = response.status(); - let headers = response.headers().clone(); - let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); - (status, headers, body.to_vec()) - })); - } - - let mut results = Vec::with_capacity(N); - for h in handles { - results.push(h.await.unwrap()); - } - let statuses: Vec<StatusCode> = results.iter().map(|(s, _, _)| *s).collect(); - - let too_many: Vec<usize> = statuses - .iter() - .enumerate() - .filter(|(_, s)| **s == StatusCode::TOO_MANY_REQUESTS) - .map(|(i, _)| i) - .collect(); - assert!( - !too_many.is_empty(), - "expected at least one /ingest under cap=1 to return 429; got statuses: {:?}", - statuses, - ); - - // Validate the structured error body for each 429 (body must carry - // the `too_many_requests` code so clients can distinguish it from - // generic conflicts). - for i in &too_many { - let body_value: Value = serde_json::from_slice(&results[*i].2).unwrap(); - let error: ErrorOutput = serde_json::from_value(body_value).unwrap(); - assert_eq!( - error.code, - Some(omnigraph_server::api::ErrorCode::TooManyRequests), - "429 body must carry code=too_many_requests; idx {} got {:?}", - i, - error.code, - ); - } - - // Validate the `Retry-After` header is set on every 429. Pinned by - // the same test so a future refactor that drops the header from - // `IntoResponse for ApiError` turns this red. The constant - // matches `crates/omnigraph-server/src/lib.rs::ApiError::into_response`. - for i in &too_many { - let retry_after = results[*i] - .1 - .get(axum::http::header::RETRY_AFTER) - .and_then(|v| v.to_str().ok()) - .map(str::to_string); - assert!( - retry_after.is_some(), - "429 response must include a Retry-After header; idx {} headers were: {:?}", - i, - results[*i].1, - ); - } -} - -/// Regression for B2 (MR-668): when an `AppState` is built with a -/// per-graph policy and a custom workload, the engine inside the -/// routing's `GraphHandle` MUST have the same policy applied via -/// `Omnigraph::with_policy`. Pre-fix, `new_with_workload(...).with_policy_engine(p)` -/// installed the policy only on the HTTP-layer `handle.policy`; the -/// underlying `Arc<Omnigraph>` was reused without `with_policy`, so any -/// caller reaching through `state.routing()` could bypass Cedar. -/// -/// This test reaches the engine the same way an embedded SDK consumer -/// or future routing code path would, and asserts the policy still -/// fires. The deny path is "act-blocked has a valid bearer but isn't in -/// the policy's allowed group" — i.e., authenticated-but-unauthorised. -#[tokio::test(flavor = "multi_thread")] -async fn engine_layer_policy_fires_via_direct_arc_omnigraph_from_new_single() { - use omnigraph_server::GraphRouting; - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - - // Permit `act-allowed` for change actions; `act-blocked` is not in - // any allowed group — every change request from them must deny. - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, permit_all_policy_yaml(&["act-allowed"])).unwrap(); - let policy_engine = - omnigraph_server::PolicyEngine::load_graph(&policy_path, graph.to_string_lossy().as_ref()) - .unwrap(); - - let workload = omnigraph_server::workload::WorkloadController::new(100, 1_000_000_000); - let state = AppState::new_single( - graph.to_string_lossy().to_string(), - db, - vec![("act-blocked".to_string(), "block-token".to_string())], - Some(policy_engine), - workload, - ); - - // Reach into the routing and pull the engine the same way an - // embedded consumer holding `Arc<Omnigraph>` would. If `new_single` - // failed to apply `with_policy` to the engine, this `mutate_as` - // would succeed — the HTTP-layer is bypassed entirely. - let handle = match state.routing() { - GraphRouting::Single { handle } => Arc::clone(handle), - GraphRouting::Multi { .. } => panic!("expected single-mode routing"), - }; - let engine = Arc::clone(&handle.engine); - - let mut params: omnigraph_compiler::ParamMap = Default::default(); - params.insert( - "name".to_string(), - omnigraph_compiler::Literal::String("EngineLayerBlocked".to_string()), - ); - params.insert("age".to_string(), omnigraph_compiler::Literal::Integer(30)); - let result = engine - .mutate_as( - "main", - MUTATION_QUERIES, - "insert_person", - ¶ms, - Some("act-blocked"), - ) - .await; - match result { - Err(OmniError::Policy(_)) => { /* expected — engine-layer gate fired */ } - Ok(_) => panic!( - "engine-layer policy did NOT fire — act-blocked successfully ran mutate_as via \ - the engine pulled from the registry handle. AppState::new_single failed to apply \ - with_policy to the underlying Omnigraph engine. This is the B2 footgun the \ - with_policy_engine deletion was supposed to close." - ), - Err(other) => panic!("expected OmniError::Policy, got: {other:?}"), - } -} - -#[tokio::test(flavor = "multi_thread")] -async fn oversized_request_body_returns_payload_too_large() { - let (_temp, app) = app_for_loaded_graph().await; - let oversized = "x".repeat(1_100_000); - let response = app - .clone() - .oneshot( - Request::builder() - .uri("/read") - .method(Method::POST) - .header("content-type", "application/json") - .body(Body::from(oversized)) - .unwrap(), - ) - .await - .unwrap(); - - assert_eq!(response.status(), StatusCode::PAYLOAD_TOO_LARGE); -} - -// ─── MR-723 default-deny mode (State 2: tokens without policy) ────────── -// -// `authorize_request` returns 403 for every action except `Read` when a -// PolicyEngine is not installed but bearer tokens are configured. Pinned -// by the three tests below — Read allowed, Change/SchemaApply denied — -// to prevent regressing back to the pre-MR-723 "tokens configured but -// no policy = fully open" trap. - -#[tokio::test(flavor = "multi_thread")] -async fn default_deny_mode_allows_read_for_authenticated_actor() { - let (_temp, app) = app_for_graph_with_auth_tokens_only( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-andrew", "demo-token")], - ) - .await; - - let (status, _body) = json_response( - &app, - Request::builder() - .uri("/snapshot") - .method(Method::GET) - .header(AUTHORIZATION, "Bearer demo-token") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); -} - -#[tokio::test(flavor = "multi_thread")] -async fn default_deny_mode_rejects_change_with_forbidden() { - let (_temp, app) = app_for_graph_with_auth_tokens_only( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-andrew", "demo-token")], - ) - .await; - - let change = ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": "DefaultDeny", "age": 1 })), - branch: Some("main".to_string()), - }; - let (status, body) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header(AUTHORIZATION, "Bearer demo-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&change).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::FORBIDDEN); - let error: ErrorOutput = serde_json::from_value(body).unwrap(); - assert!( - error.error.contains("default-deny"), - "expected default-deny in error message, got: {}", - error.error - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn default_deny_mode_rejects_schema_apply_with_forbidden() { - let (_temp, app) = app_for_graph_with_auth_tokens_only( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-andrew", "demo-token")], - ) - .await; - - let req = SchemaApplyRequest { - schema_source: additive_schema_with_nickname(), - ..Default::default() - }; - let (status, body) = json_response( - &app, - Request::builder() - .uri("/schema/apply") - .method(Method::POST) - .header(AUTHORIZATION, "Bearer demo-token") - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&req).unwrap())) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::FORBIDDEN); - let error: ErrorOutput = serde_json::from_value(body).unwrap(); - assert!( - error.error.contains("default-deny"), - "expected default-deny in error message, got: {}", - error.error - ); -} - -// ─── SDK ↔ HTTP decision parity (MR-722 PR A) ───────────────────────────── -// -// Engine and HTTP both consult Cedar via `PolicyChecker::check()`; by -// construction they cannot disagree on a decision. These tests pin that -// property explicitly so a future refactor that introduces a separate -// auth path (or copy-pastes Cedar evaluation logic) turns red. -// -// Four cases cover the per-action scope shapes: -// * Change on a protected branch via `mutate_as` / POST /change -// * Change with an actor that has no permit -// * BranchMerge to a protected target via `branch_merge_as` / POST /branches/merge -// * BranchMerge with an actor that has no permit - -const PARITY_POLICY_YAML: &str = r#" -version: 1 -groups: - team: [act-bruno] - admins: [act-ragnor] -protected_branches: [main] -rules: - - id: admins-change-anywhere - allow: - actors: { group: admins } - actions: [change] - branch_scope: any - - id: admins-merge-to-protected - allow: - actors: { group: admins } - actions: [branch_merge] - target_branch_scope: protected -"#; - -#[derive(Clone, Copy, Debug)] -enum ParityDecision { - Allow, - Deny, -} - -async fn build_parity_graph() -> (tempfile::TempDir, PathBuf, PathBuf) { - // Build a graph with `main` loaded and a `feature` branch ready for - // merge. Returns the graph path and a written policy.yaml path. - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - { - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.branch_create_from(ReadTarget::branch("main"), "feature") - .await - .unwrap(); - db.load_as( - "feature", - None, - r#"{"type":"Person","data":{"name":"ParityEve","age":29}}"#, - LoadMode::Append, - None, - ) - .await - .unwrap(); - } - let policy_path = temp.path().join("policy.yaml"); - fs::write(&policy_path, PARITY_POLICY_YAML).unwrap(); - (temp, graph, policy_path) -} - -async fn sdk_change_decision(graph: &Path, policy_path: &Path, actor: &str) -> ParityDecision { - let policy = PolicyEngine::load_graph(policy_path, graph.to_string_lossy().as_ref()).unwrap(); - let db = Omnigraph::open(graph.to_str().unwrap()) - .await - .unwrap() - .with_policy(Arc::new(policy) as Arc<dyn PolicyChecker>); - let mut params: omnigraph_compiler::ParamMap = Default::default(); - // Parameter keys are bare names (no `$` prefix); the runtime resolves - // `$name` references in the query body to `params["name"]`. - params.insert( - "name".to_string(), - omnigraph_compiler::Literal::String("ParityCharlie".to_string()), - ); - params.insert("age".to_string(), omnigraph_compiler::Literal::Integer(30)); - let result = db - .mutate_as( - "main", - MUTATION_QUERIES, - "insert_person", - ¶ms, - Some(actor), - ) - .await; - match result { - Ok(_) => ParityDecision::Allow, - Err(OmniError::Policy(_)) => ParityDecision::Deny, - Err(other) => panic!("unexpected SDK error for change: {other:?}"), - } -} - -async fn http_change_decision( - graph: &Path, - policy_path: &PathBuf, - actor: &str, - token: &str, -) -> ParityDecision { - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![(actor.to_string(), token.to_string())], - Some(policy_path), - ) - .await - .unwrap(); - let app = build_app(state); - let req = ChangeRequest { - query: MUTATION_QUERIES.to_string(), - name: Some("insert_person".to_string()), - params: Some(json!({ "name": "ParityCharlie", "age": 30 })), - branch: Some("main".to_string()), - }; - let (status, _body) = json_response( - &app, - Request::builder() - .uri("/change") - .method(Method::POST) - .header(AUTHORIZATION, format!("Bearer {token}")) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&req).unwrap())) - .unwrap(), - ) - .await; - match status { - StatusCode::OK => ParityDecision::Allow, - StatusCode::FORBIDDEN => ParityDecision::Deny, - other => panic!("unexpected HTTP status for change: {other}"), - } -} - -async fn sdk_merge_decision(graph: &Path, policy_path: &Path, actor: &str) -> ParityDecision { - let policy = PolicyEngine::load_graph(policy_path, graph.to_string_lossy().as_ref()).unwrap(); - let db = Omnigraph::open(graph.to_str().unwrap()) - .await - .unwrap() - .with_policy(Arc::new(policy) as Arc<dyn PolicyChecker>); - let result = db.branch_merge_as("feature", "main", Some(actor)).await; - match result { - Ok(_) => ParityDecision::Allow, - Err(OmniError::Policy(_)) => ParityDecision::Deny, - Err(other) => panic!("unexpected SDK error for branch_merge: {other:?}"), - } -} - -async fn http_merge_decision( - graph: &Path, - policy_path: &PathBuf, - actor: &str, - token: &str, -) -> ParityDecision { - let state = AppState::open_with_bearer_tokens_and_policy( - graph.to_string_lossy().to_string(), - vec![(actor.to_string(), token.to_string())], - Some(policy_path), - ) - .await - .unwrap(); - let app = build_app(state); - let req = BranchMergeRequest { - source: "feature".to_string(), - target: Some("main".to_string()), - }; - let (status, _body) = json_response( - &app, - Request::builder() - .uri("/branches/merge") - .method(Method::POST) - .header(AUTHORIZATION, format!("Bearer {token}")) - .header("content-type", "application/json") - .body(Body::from(serde_json::to_vec(&req).unwrap())) - .unwrap(), - ) - .await; - match status { - StatusCode::OK => ParityDecision::Allow, - StatusCode::FORBIDDEN => ParityDecision::Deny, - other => panic!("unexpected HTTP status for branch_merge: {other}"), - } -} - -#[tokio::test(flavor = "multi_thread")] -async fn policy_decision_parity_change_admin_on_main_allowed() { - // (act-ragnor, change, main) — admins-change-anywhere rule applies. - // Both SDK and HTTP must allow. Each path uses its own fresh graph - // because allow→side-effects. - let (_t1, graph1, policy1) = build_parity_graph().await; - let sdk = sdk_change_decision(&graph1, &policy1, "act-ragnor").await; - let (_t2, graph2, policy2) = build_parity_graph().await; - let http = http_change_decision(&graph2, &policy2, "act-ragnor", "ragnor-token").await; - assert!( - matches!(sdk, ParityDecision::Allow) && matches!(http, ParityDecision::Allow), - "SDK={sdk:?} HTTP={http:?} — should both Allow", - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn policy_decision_parity_change_team_on_main_denied() { - // (act-bruno, change, main) — no rule grants bruno change on - // protected. Both SDK and HTTP must deny. Same graph is reusable - // because deny→no side-effects. - let (_temp, graph, policy) = build_parity_graph().await; - let sdk = sdk_change_decision(&graph, &policy, "act-bruno").await; - let http = http_change_decision(&graph, &policy, "act-bruno", "bruno-token").await; - assert!( - matches!(sdk, ParityDecision::Deny) && matches!(http, ParityDecision::Deny), - "SDK={sdk:?} HTTP={http:?} — should both Deny", - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn policy_decision_parity_branch_merge_admin_allowed() { - // (act-ragnor, branch_merge, feature→main) — admins-merge-to-protected - // rule applies. Both Allow. Each path uses its own fresh graph — - // a successful merge consumes the feature branch's commit on main. - let (_t1, graph1, policy1) = build_parity_graph().await; - let sdk = sdk_merge_decision(&graph1, &policy1, "act-ragnor").await; - let (_t2, graph2, policy2) = build_parity_graph().await; - let http = http_merge_decision(&graph2, &policy2, "act-ragnor", "ragnor-token").await; - assert!( - matches!(sdk, ParityDecision::Allow) && matches!(http, ParityDecision::Allow), - "SDK={sdk:?} HTTP={http:?} — should both Allow", - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn policy_decision_parity_branch_merge_team_denied() { - // (act-bruno, branch_merge, feature→main) — no rule grants bruno - // branch_merge. Both Deny. - let (_temp, graph, policy) = build_parity_graph().await; - let sdk = sdk_merge_decision(&graph, &policy, "act-bruno").await; - let http = http_merge_decision(&graph, &policy, "act-bruno", "bruno-token").await; - assert!( - matches!(sdk, ParityDecision::Deny) && matches!(http, ParityDecision::Deny), - "SDK={sdk:?} HTTP={http:?} — should both Deny", - ); -} - -// ─── MR-694 PR B: HTTP soft + hard drop semantics + data preservation ──── -// -// SDK-level drop semantics are pinned in `crates/omnigraph/tests/schema_apply.rs`. -// These HTTP-side tests mirror the assertions through POST /schema/apply -// and exercise the new `allow_data_loss` field (closes the gap where -// the schema-lint chassis v1.2 shipped Hard mode on the CLI but the -// HTTP request struct had no equivalent field). - -#[tokio::test(flavor = "multi_thread")] -async fn schema_apply_route_soft_drops_property_via_http() { - let (temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - SCHEMA_APPLY_POLICY_YAML, - ) - .await; - // Load a row that has the column we're about to drop. - let graph = graph_path(temp.path()); - { - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.load( - "main", - r#"{"type":"Person","data":{"name":"PreDrop","age":42}}"#, - LoadMode::Append, - ) - .await - .unwrap(); - } - let pre_version = manifest_dataset_version(&graph).await; - - let (status, payload) = json_response( - &app, - Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: schema_without_age(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - assert_eq!(payload["applied"], true); - - // Catalog reflects the drop: `age` is gone from the live schema. - let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - assert!( - !reopened.catalog().node_types["Person"] - .properties - .contains_key("age"), - "catalog should not contain `age` after drop" - ); - - // Soft drop preserves the prior version — `age` is still readable - // via time travel to the pre-drop manifest version. Mirrors the - // SDK-side assertion in `apply_schema_drops_a_nullable_property_softly_preserves_prior_version`. - let pre_drop_snapshot = reopened.snapshot_at_version(pre_version).await.unwrap(); - let pre_drop_ds = pre_drop_snapshot.open("node:Person").await.unwrap(); - let pre_drop_fields = pre_drop_ds - .schema() - .fields - .iter() - .map(|f| f.name.clone()) - .collect::<Vec<_>>(); - assert!( - pre_drop_fields.iter().any(|f| f == "age"), - "soft drop should leave the pre-drop dataset's `age` column \ - time-travel-reachable; got fields {pre_drop_fields:?}" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn schema_apply_route_soft_drops_node_type_via_http() { - let (temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - SCHEMA_APPLY_POLICY_YAML, - ) - .await; - let graph = graph_path(temp.path()); - - let (status, payload) = json_response( - &app, - Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: schema_without_company(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - assert_eq!(payload["applied"], true); - - let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - assert!( - !reopened.catalog().node_types.contains_key("Company"), - "catalog should not contain `Company` after drop" - ); - assert!( - !reopened.catalog().edge_types.contains_key("WorksAt"), - "catalog should not contain `WorksAt` after cascade" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn schema_apply_route_hard_drops_property_with_allow_data_loss() { - let (temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - SCHEMA_APPLY_POLICY_YAML, - ) - .await; - let graph = graph_path(temp.path()); - { - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.load( - "main", - r#"{"type":"Person","data":{"name":"PreDropHard","age":50}}"#, - LoadMode::Append, - ) - .await - .unwrap(); - } - - // Apply with allow_data_loss=true → Hard mode promotion. - let (status, payload) = json_response( - &app, - Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: schema_without_age(), - allow_data_loss: true, - }) - .unwrap(), - )) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - assert_eq!(payload["applied"], true); - - // Catalog reflects the drop. - let reopened = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - assert!( - !reopened.catalog().node_types["Person"] - .properties - .contains_key("age"), - "catalog should not contain `age` after Hard drop" - ); - // Plan steps should show DropMode::Hard for property drops. - let steps = payload["steps"].as_array().expect("steps array"); - let drop_step = steps - .iter() - .find(|s| s["kind"] == "drop_property") - .expect("plan should include drop_property step"); - let mode = &drop_step["mode"]; - assert_eq!( - mode, "hard", - "expected hard mode under allow_data_loss=true" - ); -} - -#[tokio::test(flavor = "multi_thread")] -async fn schema_apply_route_keeps_drops_soft_without_flag() { - // Symmetric to the Hard test: same schema change, but no - // allow_data_loss flag → drops stay Soft (prior column data - // remains time-travel-reachable). Pins the default semantics - // against accidental Hard promotion. - let (temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - SCHEMA_APPLY_POLICY_YAML, - ) - .await; - let graph = graph_path(temp.path()); - - let (status, payload) = json_response( - &app, - Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: schema_without_age(), - allow_data_loss: false, - }) - .unwrap(), - )) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - assert_eq!(payload["applied"], true); - - let steps = payload["steps"].as_array().expect("steps array"); - let drop_step = steps - .iter() - .find(|s| s["kind"] == "drop_property") - .expect("plan should include drop_property step"); - let mode = &drop_step["mode"]; - assert_eq!(mode, "soft", "expected soft mode without allow_data_loss"); - let _ = graph; -} - -#[tokio::test(flavor = "multi_thread")] -async fn schema_apply_route_additive_property_preserves_existing_rows() { - // SDK suite covers rename and drop data preservation. Additive - // AddProperty wasn't pinned with a row-count check anywhere. - // Load N rows, apply schema adding nullable property, verify - // every row is still readable and the new column is null. - let (temp, app) = app_for_graph_with_auth_tokens_and_policy( - &fs::read_to_string(fixture("test.pg")).unwrap(), - &[("act-ragnor", "admin-token")], - SCHEMA_APPLY_POLICY_YAML, - ) - .await; - let graph = graph_path(temp.path()); - - // Standard fixture data: 4 Persons + 1 Company. Load it. - let pre_count = { - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - db.load( - "main", - &fs::read_to_string(fixture("test.jsonl")).unwrap(), - LoadMode::Append, - ) - .await - .unwrap(); - let snap = db - .snapshot_of(omnigraph::db::ReadTarget::branch("main")) - .await - .unwrap(); - snap.entry("node:Person").expect("Person").row_count - }; - assert!(pre_count > 0, "fixture should have loaded Person rows"); - - let (status, payload) = json_response( - &app, - Request::builder() - .method(Method::POST) - .uri("/schema/apply") - .header("content-type", "application/json") - .header("authorization", "Bearer admin-token") - .body(Body::from( - serde_json::to_vec(&SchemaApplyRequest { - schema_source: additive_schema_with_nickname(), - ..Default::default() - }) - .unwrap(), - )) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK); - assert_eq!(payload["applied"], true); - - // Row count preserved. - let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); - let snap = db - .snapshot_of(omnigraph::db::ReadTarget::branch("main")) - .await - .unwrap(); - let post_count = snap.entry("node:Person").expect("Person").row_count; - assert_eq!( - post_count, pre_count, - "AddProperty should preserve row count", - ); -} - -// ─── MR-668: multi-graph startup ────────────────────────────────────────── - -mod multi_graph_startup { - use super::*; - use omnigraph::storage::normalize_root_uri; - use omnigraph_server::{ - GraphHandle, GraphId, GraphKey, GraphRegistry, InsertError, ServerConfig, ServerConfigMode, - load_server_settings, - }; - use std::sync::Arc; - - async fn build_multi_mode_app(graph_ids: &[&str]) -> (Vec<tempfile::TempDir>, Router) { - let mut dirs = Vec::with_capacity(graph_ids.len()); - let mut handles = Vec::with_capacity(graph_ids.len()); - for id in graph_ids { - let dir = tempfile::tempdir().unwrap(); - let graph_uri = dir.path().join(id).to_str().unwrap().to_string(); - let schema = fs::read_to_string(fixture("test.pg")).unwrap(); - let engine = Omnigraph::init(&graph_uri, &schema).await.unwrap(); - handles.push(Arc::new(GraphHandle { - key: GraphKey::cluster(GraphId::try_from(*id).unwrap()), - uri: graph_uri, - engine: Arc::new(engine), - policy: None, - queries: None, - })); - dirs.push(dir); - } - let workload = omnigraph_server::workload::WorkloadController::from_env(); - let state = AppState::new_multi(handles, Vec::new(), None, workload, None).unwrap(); - let app = build_app(state); - (dirs, app) - } - - /// Cluster route `/graphs/{graph_id}/snapshot` resolves to the right - /// engine. Two graphs side by side; assert each responds to its own - /// id and does NOT respond to the other's URL. - #[tokio::test(flavor = "multi_thread")] - async fn cluster_routes_dispatch_per_graph_handle() { - let (_dirs, app) = build_multi_mode_app(&["alpha", "beta"]).await; - for id in ["alpha", "beta"] { - let resp = app - .clone() - .oneshot( - Request::builder() - .method(Method::GET) - .uri(format!("/graphs/{id}/snapshot?branch=main")) - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!( - resp.status(), - StatusCode::OK, - "graph '{id}' must respond OK on its cluster snapshot route" - ); - } - } - - /// Unknown graph id under the cluster prefix yields 404 (not 500, - /// not 410 — `Gone` is reserved for the future DELETE flow). - #[tokio::test(flavor = "multi_thread")] - async fn cluster_route_for_unknown_graph_returns_404() { - let (_dirs, app) = build_multi_mode_app(&["alpha"]).await; - let resp = app - .oneshot( - Request::builder() - .method(Method::GET) - .uri("/graphs/nonexistent/snapshot?branch=main") - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(resp.status(), StatusCode::NOT_FOUND); - } - - /// Coverage net for cluster-route regressions across every - /// protected handler — not just the few that have inner path - /// params. Bug-1 surfaced because only `/snapshot` was being - /// exercised in cluster mode, leaving the other six protected - /// routes implicitly untested. This sweep hits each one and - /// asserts the response shows the handler was reached: no 404 - /// (router didn't match), no 500 with "Wrong number of path - /// arguments" (path extractor broke), no 500 with "missing - /// extension" (routing middleware didn't inject the handle). - /// - /// Status codes are negative assertions because each handler's - /// happy-path inputs differ — what matters is "the request - /// reached the handler," not "the handler returned 200." The - /// individual handlers' logic is already tested in single mode. - #[tokio::test(flavor = "multi_thread")] - async fn all_protected_cluster_routes_resolve_to_their_handler() { - let (_dirs, app) = build_multi_mode_app(&["alpha"]).await; - - // (method, path, body) — one minimal request per protected - // cluster route. Bodies are valid enough that the router and - // extractors succeed; whether the engine ultimately returns - // 200 or 4xx is per-handler and not what this test pins. - let cases: &[(Method, &str, Option<&str>)] = &[ - (Method::GET, "/graphs/alpha/snapshot?branch=main", None), - (Method::GET, "/graphs/alpha/schema", None), - (Method::GET, "/graphs/alpha/branches", None), - (Method::GET, "/graphs/alpha/commits", None), - ( - Method::POST, - "/graphs/alpha/read", - Some(r#"{"query_source":"query q() { return {} }"}"#), - ), - ( - Method::POST, - "/graphs/alpha/change", - Some(r#"{"query_source":"query q() { return {} }"}"#), - ), - ( - Method::POST, - "/graphs/alpha/export", - Some(r#"{"branch":"main"}"#), - ), - ( - Method::POST, - "/graphs/alpha/schema/apply", - Some(r#"{"schema_source":"","allow_data_loss":false}"#), - ), - (Method::POST, "/graphs/alpha/ingest", Some(r#"{"data":""}"#)), - ( - Method::POST, - "/graphs/alpha/branches/merge", - Some(r#"{"source":"main","target":"main"}"#), - ), - ]; - - for (method, path, body) in cases { - let req_body = body - .map(|s| Body::from(s.to_string())) - .unwrap_or_else(Body::empty); - let req = Request::builder() - .method(method.clone()) - .uri(*path) - .header("content-type", "application/json") - .body(req_body) - .unwrap(); - let resp = app.clone().oneshot(req).await.unwrap(); - let status = resp.status(); - let bytes = to_bytes(resp.into_body(), usize::MAX).await.unwrap(); - let body_str = String::from_utf8_lossy(&bytes); - - assert_ne!( - status, - StatusCode::NOT_FOUND, - "{} {} — router didn't match (cluster-route mounting regression). Body: {}", - method, - path, - body_str, - ); - assert!( - !(status == StatusCode::INTERNAL_SERVER_ERROR - && body_str.contains("Wrong number of path arguments")), - "{} {} — path extractor broke (Bug-1 class regression). Body: {}", - method, - path, - body_str, - ); - assert!( - !(status == StatusCode::INTERNAL_SERVER_ERROR - && body_str.to_lowercase().contains("missing extension")), - "{} {} — routing middleware didn't inject GraphHandle. Body: {}", - method, - path, - body_str, - ); - } - } - - /// Regression for the bot-surfaced path-extractor bug: cluster - /// routes whose inner path also captures a parameter - /// (`/graphs/{graph_id}/branches/{branch}`, - /// `/graphs/{graph_id}/commits/{commit_id}`) must extract the - /// inner param cleanly. Axum 0.8 propagates the outer `{graph_id}` - /// capture into nested handlers, so a `Path<String>` extractor - /// would see two values and fail with "Wrong number of path - /// arguments. Expected 1 but got 2." Today both DELETE branch and - /// GET commit-by-id break in multi-mode because their handlers - /// use bare `Path<String>` — this test pins the fix. - /// - /// The broader `all_protected_cluster_routes_resolve_to_their_handler` - /// test sweeps the full route surface; this one stays narrowly - /// targeted at the inner-path-param shape because that's the - /// specific regression class. - #[tokio::test(flavor = "multi_thread")] - async fn cluster_routes_with_inner_path_params_deserialize_correctly() { - let (_dirs, app) = build_multi_mode_app(&["alpha"]).await; - - // Create a branch we can then delete — DELETE /graphs/alpha/branches/feature - let create_resp = app - .clone() - .oneshot( - Request::builder() - .method(Method::POST) - .uri("/graphs/alpha/branches") - .header("content-type", "application/json") - .body(Body::from(r#"{"name":"feature"}"#)) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!( - create_resp.status(), - StatusCode::OK, - "branch create on the cluster route must succeed before delete can be tested" - ); - - // DELETE /graphs/{graph_id}/branches/{branch} — exercises a handler - // whose only Path extractor (`branch`) is inside a nested route - // that also captures `graph_id`. The handler must pick `branch` - // by name, not by position. - let delete_resp = app - .clone() - .oneshot( - Request::builder() - .method(Method::DELETE) - .uri("/graphs/alpha/branches/feature") - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - let delete_status = delete_resp.status(); - let delete_body = to_bytes(delete_resp.into_body(), usize::MAX).await.unwrap(); - assert_eq!( - delete_status, - StatusCode::OK, - "DELETE /graphs/{{id}}/branches/{{branch}} must extract `branch` cleanly. \ - Body: {}", - String::from_utf8_lossy(&delete_body), - ); - - // GET /graphs/{graph_id}/commits/{commit_id} — same shape: the - // handler's only Path extractor is the inner `commit_id`, which - // must deserialize by name even though `graph_id` is also in scope. - // We don't know a real commit_id, but the failure mode under test - // is path extraction, not commit lookup — a 404 from the engine - // is fine; a 500 with "Wrong number of path arguments" is the bug. - let commit_resp = app - .oneshot( - Request::builder() - .method(Method::GET) - .uri("/graphs/alpha/commits/0000000000000000") - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - let commit_status = commit_resp.status(); - let commit_body = to_bytes(commit_resp.into_body(), usize::MAX).await.unwrap(); - let body_str = String::from_utf8_lossy(&commit_body); - assert!( - commit_status != StatusCode::INTERNAL_SERVER_ERROR - || !body_str.contains("Wrong number of path arguments"), - "GET /graphs/{{id}}/commits/{{commit_id}} must extract `commit_id` cleanly. \ - Got: {} | {}", - commit_status, - body_str, - ); - } - - /// Flat routes 404 in multi mode — the router only mounts under - /// `/graphs/{graph_id}/...` so `/snapshot` doesn't resolve. - #[tokio::test(flavor = "multi_thread")] - async fn flat_routes_404_in_multi_mode() { - let (_dirs, app) = build_multi_mode_app(&["alpha"]).await; - let resp = app - .oneshot( - Request::builder() - .method(Method::GET) - .uri("/snapshot?branch=main") - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(resp.status(), StatusCode::NOT_FOUND); - } - - /// `GraphId` validation runs at startup — a reserved name in - /// `omnigraph.yaml` produces a clear error rather than getting - /// rejected per-request. - #[tokio::test] - async fn load_server_settings_rejects_reserved_graph_id() { - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -graphs: - policies: - uri: /tmp/g1.omni -"#, - ) - .unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, None, false).await.unwrap_err(); - assert!( - err.to_string().contains("invalid graph id 'policies'"), - "expected reserved-name rejection, got: {err}" - ); - } - - #[tokio::test(flavor = "multi_thread")] - async fn registry_rejects_duplicate_normalized_graph_uris() { - let dir = tempfile::tempdir().unwrap(); - let graph_uri = dir.path().join("same").to_str().unwrap().to_string(); - let schema = fs::read_to_string(fixture("test.pg")).unwrap(); - let engine = Arc::new(Omnigraph::init(&graph_uri, &schema).await.unwrap()); - - let alpha = Arc::new(GraphHandle { - key: GraphKey::cluster(GraphId::try_from("alpha").unwrap()), - uri: graph_uri.clone(), - engine: Arc::clone(&engine), - policy: None, - queries: None, - }); - let beta = Arc::new(GraphHandle { - key: GraphKey::cluster(GraphId::try_from("beta").unwrap()), - uri: format!("file://{graph_uri}/"), - engine, - policy: None, - queries: None, - }); - - match GraphRegistry::from_handles(vec![alpha, beta]) { - Err(InsertError::DuplicateUri(uri)) => { - assert!( - normalize_root_uri(&uri).is_ok(), - "duplicate URI should still be parseable, got {uri}" - ); - } - Err(err) => panic!("expected DuplicateUri for normalized aliases, got {err:?}"), - Ok(_) => panic!("expected DuplicateUri for normalized aliases, got Ok"), - } - } - - #[tokio::test(flavor = "multi_thread")] - async fn registry_stores_canonical_graph_uri() { - let dir = tempfile::tempdir().unwrap(); - let graph_uri = dir.path().join("canonical").to_str().unwrap().to_string(); - let schema = fs::read_to_string(fixture("test.pg")).unwrap(); - let engine = Omnigraph::init(&graph_uri, &schema).await.unwrap(); - let handle = Arc::new(GraphHandle { - key: GraphKey::cluster(GraphId::try_from("alpha").unwrap()), - uri: format!("file://{graph_uri}/"), - engine: Arc::new(engine), - policy: None, - queries: None, - }); - - let registry = GraphRegistry::from_handles(vec![handle]).unwrap(); - let listed = registry.list(); - assert_eq!(listed.len(), 1); - assert_eq!(listed[0].uri, graph_uri); - } - - // ── Four-rule mode inference matrix ─────────────────────────────── - - /// Rule 1: CLI positional URI → Single. - #[tokio::test] - async fn mode_inference_cli_uri_is_single() { - let settings = load_server_settings( - None, - None, - Some("/tmp/cli.omni".to_string()), - None, - None, - true, // allow unauth so we get past the runtime-state check - ) - .await - .unwrap(); - match settings.mode { - ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/cli.omni"), - ServerConfigMode::Multi { .. } => panic!("expected Single (rule 1), got Multi"), - } - } - - /// Rule 2: --target picks one graph from `graphs:` map → Single. - #[tokio::test] - async fn mode_inference_cli_target_is_single() { - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -graphs: - alpha: - uri: /tmp/alpha.omni - beta: - uri: /tmp/beta.omni -"#, - ) - .unwrap(); - let settings = - load_server_settings(Some(&config_path), None, None, Some("alpha".into()), None, true) - .await - .unwrap(); - match settings.mode { - ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/alpha.omni"), - ServerConfigMode::Multi { .. } => panic!("expected Single (rule 2), got Multi"), - } - } - - /// Rule 3: `server.graph` set → Single (target picked from config). - #[tokio::test] - async fn mode_inference_server_graph_is_single() { - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -graphs: - alpha: - uri: /tmp/alpha.omni - beta: - uri: /tmp/beta.omni -server: - graph: beta -"#, - ) - .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); - match settings.mode { - ServerConfigMode::Single { uri, .. } => assert_eq!(uri, "/tmp/beta.omni"), - ServerConfigMode::Multi { .. } => panic!("expected Single (rule 3), got Multi"), - } - } - - /// Rule 4: `--config` + non-empty `graphs:` + no single-mode selector → Multi. - #[tokio::test] - async fn mode_inference_config_plus_graphs_is_multi() { - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -graphs: - alpha: - uri: /tmp/alpha.omni - beta: - uri: /tmp/beta.omni -"#, - ) - .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); - match settings.mode { - ServerConfigMode::Multi { graphs, .. } => { - let ids: Vec<&str> = graphs.iter().map(|g| g.graph_id.as_str()).collect(); - // BTreeMap iteration order is alphabetical. - assert_eq!(ids, vec!["alpha", "beta"]); - } - ServerConfigMode::Single { .. } => panic!("expected Multi (rule 4), got Single"), - } - } - - #[tokio::test] - async fn mode_inference_multi_rejects_top_level_policy_file() { - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -policy: - file: ./policy.yaml -graphs: - alpha: - uri: /tmp/alpha.omni -"#, - ) - .unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap_err(); - let msg = err.to_string(); - assert!( - msg.contains("top-level") && msg.contains("policy.file") && msg.contains("not honored"), - "expected top-level-not-honored guidance, got: {msg}" - ); - assert!( - msg.contains("graphs.<graph_id>"), - "expected per-graph migration guidance, got: {msg}" - ); - assert!( - msg.contains("server.policy.file"), - "expected server policy migration guidance, got: {msg}" - ); - } - - #[tokio::test] - async fn mode_inference_multi_rejects_top_level_queries() { - // Symmetric to the policy guard: a top-level `queries:` block in - // multi-graph mode is not honored (each graph uses its own), so it - // is a loud error rather than a silent no-op. - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - "queries:\n q:\n file: ./q.gq\ngraphs:\n alpha:\n uri: /tmp/alpha.omni\n", - ) - .unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap_err(); - let msg = err.to_string(); - assert!( - msg.contains("queries") && msg.contains("not honored"), - "top-level queries must be rejected in multi-graph mode: {msg}" - ); - } - - #[tokio::test] - async fn single_mode_named_graph_rejects_top_level_blocks() { - // Serving a graph by name (`--target`/`server.graph`) uses its - // per-graph block; a populated top-level block would be silently - // shadowed, so boot refuses and names the per-graph location. - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - "policy:\n file: ./top.yaml\ngraphs:\n prod:\n uri: /tmp/prod.omni\n", - ) - .unwrap(); - let err = - load_server_settings(Some(&config_path), None, None, Some("prod".to_string()), None, true) - .await - .unwrap_err(); - let msg = err.to_string(); - assert!( - msg.contains("prod") && msg.contains("policy.file") && msg.contains("graphs.prod"), - "named single-mode + top-level policy must refuse, naming the graph: {msg}" - ); - } - - #[tokio::test] - async fn single_mode_named_graph_uses_per_graph_policy_and_queries() { - // The identity rule: `--target prod` attaches `graphs.prod`'s own - // policy + queries, not the top-level ones (which are absent here). - let temp = tempfile::tempdir().unwrap(); - fs::write( - temp.path().join("prod.gq"), - "query pq() { match { $u: User } return { $u.name } }", - ) - .unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - "graphs:\n prod:\n uri: /tmp/prod.omni\n policy:\n file: ./prod-policy.yaml\n \ - queries:\n pq:\n file: ./prod.gq\n", - ) - .unwrap(); - let settings = - load_server_settings(Some(&config_path), None, None, Some("prod".to_string()), None, true) - .await - .unwrap(); - match settings.mode { - ServerConfigMode::Single { - graph_id, - policy_file, - queries, - .. - } => { - assert_eq!(graph_id, "prod", "named single-mode keeps graph identity"); - assert!( - policy_file - .as_ref() - .is_some_and(|p| p.ends_with("prod-policy.yaml")), - "per-graph policy attached: {policy_file:?}" - ); - assert!(queries.lookup("pq").is_some(), "per-graph query attached"); - } - other => panic!("expected Single mode, got {other:?}"), - } - } - - #[tokio::test] - async fn mode_inference_normalizes_multi_graph_uris() { - let temp = tempfile::tempdir().unwrap(); - let graph = temp.path().join("alpha.omni"); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - format!( - r#" -graphs: - alpha: - uri: file://{}/ -"#, - graph.display() - ), - ) - .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); - match settings.mode { - ServerConfigMode::Multi { graphs, .. } => { - assert_eq!(graphs[0].uri, graph.to_string_lossy()); - } - ServerConfigMode::Single { .. } => panic!("expected Multi"), - } - } - - /// Rule 5: nothing → error with migration hint. - #[tokio::test] - async fn mode_inference_no_inputs_errors_with_migration_hint() { - let err = load_server_settings(None, None, None, None, None, true).await.unwrap_err(); - let msg = err.to_string(); - assert!( - msg.contains("no graph to serve"), - "expected migration-hint error, got: {msg}" - ); - } - - /// Rule 4 sub-case: `--config` with empty `graphs:` map and no - /// single-mode selector → rule 5 fires (no graph to serve). - #[tokio::test] - async fn mode_inference_empty_graphs_map_errors() { - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write(&config_path, "server:\n bind: 127.0.0.1:8080\n").unwrap(); - let err = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap_err(); - assert!(err.to_string().contains("no graph to serve")); - } - - /// `--config` + `<URI>` together: URI wins → Single (the CLI URI - /// takes precedence over the config's graphs map). - #[tokio::test] - async fn mode_inference_cli_uri_overrides_graphs_map() { - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -graphs: - alpha: - uri: /tmp/alpha.omni -"#, - ) - .unwrap(); - let settings = load_server_settings( - Some(&config_path), - None, - Some("/tmp/cli-override.omni".to_string()), - None, - None, - true, - ) - .await - .unwrap(); - match settings.mode { - ServerConfigMode::Single { uri, .. } => { - assert_eq!( - uri, "/tmp/cli-override.omni", - "CLI URI must win over graphs: map" - ); - } - ServerConfigMode::Multi { .. } => { - panic!("expected Single (CLI URI wins), got Multi") - } - } - } - - /// Per-graph `policy.file` is resolved relative to the config base_dir. - #[tokio::test] - async fn per_graph_policy_file_is_resolved_relative_to_base_dir() { - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -graphs: - alpha: - uri: /tmp/alpha.omni - policy: - file: ./policies/alpha.yaml - beta: - uri: /tmp/beta.omni -"#, - ) - .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); - let graphs = match settings.mode { - ServerConfigMode::Multi { graphs, .. } => graphs, - _ => panic!("expected Multi"), - }; - // graphs is BTreeMap-iter order (alphabetical). - let alpha = &graphs[0]; - let beta = &graphs[1]; - assert_eq!(alpha.graph_id, "alpha"); - assert_eq!( - alpha.policy_file.as_ref().unwrap(), - &temp.path().join("policies/alpha.yaml") - ); - assert_eq!(beta.graph_id, "beta"); - assert!(beta.policy_file.is_none()); - } - - /// `server.policy.file` resolves alongside the graphs map. - #[tokio::test] - async fn server_policy_file_is_resolved_relative_to_base_dir() { - let temp = tempfile::tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -server: - policy: - file: ./server-policy.yaml -graphs: - alpha: - uri: /tmp/alpha.omni -"#, - ) - .unwrap(); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); - match settings.mode { - ServerConfigMode::Multi { - server_policy_file, .. - } => { - assert_eq!( - server_policy_file.unwrap(), - temp.path().join("server-policy.yaml") - ); - } - _ => panic!("expected Multi"), - } - } - - /// `GET /graphs` must NOT leak the registry in Open mode without - /// an explicit server policy. Operators who pass `--unauthenticated` - /// opted into trusting the network for graph DATA, not for leaking - /// server topology (graph IDs + URIs, which may contain S3 bucket - /// paths or internal hostnames). Cedar gating the management - /// surface is the documented contract for `server_graphs_list` - /// ("don't leak the registry until the operator explicitly - /// authorizes it"); enforcing that contract in every runtime - /// state — not just `PolicyEnabled` — is the correct-by-design - /// closure of the open-mode hole the bot-review pass surfaced. - /// - /// Today (pre-fix) this returns 200 because `authorize_request`'s - /// no-policy fallback only denies when `actor.is_some()`, so Open - /// mode (`actor: None`) falls through to `Ok(())`. The fix in the - /// next commit tightens the fallback so server-scoped actions - /// always require explicit policy. - /// - /// Sort-order coverage previously lived here; it has moved to - /// `get_graphs_with_server_policy_authorizes_per_cedar` where - /// the response body is now non-empty and operator-authorized. - #[tokio::test(flavor = "multi_thread")] - async fn get_graphs_denied_in_open_mode_without_server_policy() { - let (_dirs, app) = build_multi_mode_app(&["beta", "alpha"]).await; - let resp = app - .oneshot( - Request::builder() - .method(Method::GET) - .uri("/graphs") - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - let status = resp.status(); - let body = to_bytes(resp.into_body(), usize::MAX).await.unwrap(); - let body_str = String::from_utf8_lossy(&body); - assert_eq!( - status, - StatusCode::FORBIDDEN, - "GET /graphs must require an explicit server policy in every \ - runtime state; Open-mode bypass would leak server topology. \ - Body: {body_str}", - ); - } - - /// `GET /graphs` returns 405 in single mode (resource exists in the - /// API surface, just not operational without a `graphs:` map). - #[tokio::test(flavor = "multi_thread")] - async fn get_graphs_returns_405_in_single_mode() { - let temp = init_loaded_graph().await; - let graph = graph_path(temp.path()); - let state = AppState::open(graph.to_string_lossy().to_string()) - .await - .unwrap(); - let app = build_app(state); - let resp = app - .oneshot( - Request::builder() - .method(Method::GET) - .uri("/graphs") - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(resp.status(), StatusCode::METHOD_NOT_ALLOWED); - } - - /// `GET /graphs` requires bearer auth when tokens are configured. - #[tokio::test(flavor = "multi_thread")] - async fn get_graphs_requires_bearer_auth_when_configured() { - use omnigraph_server::{GraphHandle, GraphId, GraphKey}; - // Build a multi-mode app with bearer tokens configured. - let dir = tempfile::tempdir().unwrap(); - let graph_uri = dir.path().join("alpha").to_str().unwrap().to_string(); - let schema = fs::read_to_string(fixture("test.pg")).unwrap(); - let engine = Omnigraph::init(&graph_uri, &schema).await.unwrap(); - let handle = Arc::new(GraphHandle { - key: GraphKey::cluster(GraphId::try_from("alpha").unwrap()), - uri: graph_uri, - engine: Arc::new(engine), - policy: None, - queries: None, - }); - let tokens = vec![("act-andrew".to_string(), "secret-token".to_string())]; - let workload = omnigraph_server::workload::WorkloadController::from_env(); - let state = AppState::new_multi(vec![handle], tokens, None, workload, None).unwrap(); - let app = build_app(state); - - // No Authorization header → 401. - let resp_no_auth = app - .clone() - .oneshot( - Request::builder() - .method(Method::GET) - .uri("/graphs") - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(resp_no_auth.status(), StatusCode::UNAUTHORIZED); - - // With auth but no server policy → 403 (default-deny, since - // GraphList is not Read). - let resp_authed = app - .oneshot( - Request::builder() - .method(Method::GET) - .uri("/graphs") - .header("authorization", "Bearer secret-token") - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!(resp_authed.status(), StatusCode::FORBIDDEN); - } - - /// `GET /graphs` with a server policy that allows `graph_list` → 200 - /// and returns the registry sorted alphabetically by `graph_id`. - /// `GET /graphs` with a server policy that does NOT allow - /// `graph_list` (viewer group) → 403. - /// - /// This test owns the alphabetical-sort coverage that previously - /// lived in `get_graphs_lists_registered_graphs_in_multi_mode`. - /// That test now asserts denial in Open mode (server-scoped actions - /// require explicit policy in every runtime state), so the positive - /// body-shape assertions need a home where the response is - /// operator-authorized — here. - #[tokio::test(flavor = "multi_thread")] - async fn get_graphs_with_server_policy_authorizes_per_cedar() { - use omnigraph_policy::PolicyEngine; - use omnigraph_server::{GraphHandle, GraphId, GraphKey}; - - let dir = tempfile::tempdir().unwrap(); - - // Two graphs deliberately registered in non-alphabetical order - // so the test would fail if the handler relied on insertion - // order instead of server-side sorting. - let schema = fs::read_to_string(fixture("test.pg")).unwrap(); - let mut handles = Vec::new(); - for id in ["beta", "alpha"] { - let graph_uri = dir.path().join(id).to_str().unwrap().to_string(); - let engine = Omnigraph::init(&graph_uri, &schema).await.unwrap(); - handles.push(Arc::new(GraphHandle { - key: GraphKey::cluster(GraphId::try_from(id).unwrap()), - uri: graph_uri, - engine: Arc::new(engine), - policy: None, - queries: None, - })); - } - - // Server policy: admins can graph_list, viewers cannot. - let policy_path = dir.path().join("server-policy.yaml"); - fs::write( - &policy_path, - r#" -version: 1 -groups: - admins: [act-andrew] - viewers: [act-bruno] -rules: - - id: admins-list-graphs - allow: - actors: { group: admins } - actions: [graph_list] -"#, - ) - .unwrap(); - let server_policy = PolicyEngine::load_server(&policy_path).unwrap(); - - let tokens = vec![ - ("act-andrew".to_string(), "andrew-token".to_string()), - ("act-bruno".to_string(), "bruno-token".to_string()), - ]; - let workload = omnigraph_server::workload::WorkloadController::from_env(); - let state = - AppState::new_multi(handles, tokens, Some(server_policy), workload, None).unwrap(); - let app = build_app(state); - - // Admin → 200, body returns both graphs alphabetically sorted. - let resp_admin = app - .clone() - .oneshot( - Request::builder() - .method(Method::GET) - .uri("/graphs") - .header("authorization", "Bearer andrew-token") - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!( - resp_admin.status(), - StatusCode::OK, - "admin must be allowed graph_list" - ); - let body = to_bytes(resp_admin.into_body(), usize::MAX).await.unwrap(); - let json: Value = serde_json::from_slice(&body).unwrap(); - let graphs = json["graphs"].as_array().unwrap(); - assert_eq!(graphs.len(), 2, "response must list both registered graphs"); - assert_eq!( - graphs[0]["graph_id"].as_str().unwrap(), - "alpha", - "server must sort graphs alphabetically by graph_id (insertion order was 'beta', 'alpha')" - ); - assert_eq!(graphs[1]["graph_id"].as_str().unwrap(), "beta"); - - // Viewer → 403 - let resp_viewer = app - .oneshot( - Request::builder() - .method(Method::GET) - .uri("/graphs") - .header("authorization", "Bearer bruno-token") - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); - assert_eq!( - resp_viewer.status(), - StatusCode::FORBIDDEN, - "viewer must be denied graph_list (Cedar gate)" - ); - } - - /// Loads an `omnigraph.yaml` with two graphs and verifies multi-mode - /// inference plus graph entry resolution. Cluster-route dispatch is - /// covered by the route tests above. - #[tokio::test(flavor = "multi_thread")] - async fn server_settings_load_multi_graph_config_entries() { - let cfg_dir = tempfile::tempdir().unwrap(); - // Real graph storage dirs (the URIs in the config must point to - // a graph init-able location). - let alpha_dir = cfg_dir.path().join("alpha.omni"); - let beta_dir = cfg_dir.path().join("beta.omni"); - let schema = fs::read_to_string(fixture("test.pg")).unwrap(); - Omnigraph::init(alpha_dir.to_str().unwrap(), &schema) - .await - .unwrap(); - Omnigraph::init(beta_dir.to_str().unwrap(), &schema) - .await - .unwrap(); - - let config_path = cfg_dir.path().join("omnigraph.yaml"); - fs::write( - &config_path, - format!( - r#" -graphs: - alpha: - uri: {alpha} - beta: - uri: {beta} -"#, - alpha = alpha_dir.display(), - beta = beta_dir.display(), - ), - ) - .unwrap(); - - let settings: ServerConfig = - load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); - assert!(matches!(settings.mode, ServerConfigMode::Multi { .. })); - - match settings.mode { - ServerConfigMode::Multi { graphs, .. } => { - assert_eq!(graphs.len(), 2); - let ids: Vec<&str> = graphs.iter().map(|g| g.graph_id.as_str()).collect(); - assert_eq!(ids, vec!["alpha", "beta"]); - } - _ => unreachable!(), - } - } -} - -// ---- Phase 5: cluster-mode boot (RFC-005) ---- - -/// Build and converge a real cluster directory: cluster.yaml + schema + -/// stored query (+ optional policies), then `import` + `apply` so the -/// catalog and state ledger exist exactly as an operator would have them. -async fn converged_cluster_dir(policies_yaml: &str) -> tempfile::TempDir { - let temp = tempfile::tempdir().unwrap(); - fs::write( - temp.path().join("people.pg"), - "\nnode Person {\n name: String @key\n}\n", - ) - .unwrap(); - fs::write( - temp.path().join("people.gq"), - "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", - ) - .unwrap(); - fs::write( - temp.path().join("cluster.yaml"), - format!( - r#" -version: 1 -graphs: - knowledge: - schema: ./people.pg - queries: - find_person: - file: ./people.gq -{policies_yaml}"# - ), - ) - .unwrap(); - let import = omnigraph_cluster::import_config_dir(temp.path()).await; - assert!(import.ok, "{:?}", import.diagnostics); - let apply = omnigraph_cluster::apply_config_dir(temp.path()).await; - assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); - temp -} - -async fn cluster_settings(dir: &Path) -> color_eyre::eyre::Result<omnigraph_server::ServerConfig> { - omnigraph_server::load_server_settings(None, Some(&dir.to_path_buf()), None, None, None, true).await -} - -#[tokio::test] -async fn cluster_boot_serves_applied_state() { - let temp = converged_cluster_dir("").await; - let settings = cluster_settings(temp.path()).await.unwrap(); - let omnigraph_server::ServerConfigMode::Multi { - graphs, - config_path, - server_policy_file, - } = settings.mode - else { - panic!("cluster boot must select multi-graph routing"); - }; - assert_eq!(graphs.len(), 1); - assert_eq!(graphs[0].graph_id, "knowledge"); - assert!(server_policy_file.is_none()); - - let state = - omnigraph_server::open_multi_graph_state(graphs, Vec::new(), None, config_path) - .await - .unwrap(); - let app = build_app(state); - - // The management surface keeps its closed-by-default contract: without a - // cluster-scoped policy bundle there is no server-level Cedar engine, so - // GET /graphs refuses even in cluster mode. - let (status, body) = json_response( - &app, - Request::builder().uri("/graphs").body(Body::empty()).unwrap(), - ) - .await; - assert_eq!(status, StatusCode::FORBIDDEN, "{body}"); - - let (status, body) = json_response( - &app, - Request::builder() - .uri("/graphs/knowledge/queries") - .body(Body::empty()) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK, "{body}"); - assert!( - body["queries"] - .as_array() - .unwrap() - .iter() - .any(|q| q["name"] == "find_person"), - "{body}" - ); - - let (status, body) = json_response( - &app, - Request::builder() - .method(Method::POST) - .uri("/graphs/knowledge/queries/find_person") - .header("content-type", "application/json") - .body(Body::from(r#"{"params":{"name":"nobody"}}"#)) - .unwrap(), - ) - .await; - assert_eq!(status, StatusCode::OK, "{body}"); -} - -#[tokio::test] -async fn cluster_boot_wires_policy_bindings_into_cedar_slots() { - let temp = tempfile::tempdir().unwrap(); - drop(temp); - let policy_block = r#"policies: - graph_rules: - file: ./graph.policy.yaml - applies_to: [knowledge] - cluster_rules: - file: ./cluster.policy.yaml - applies_to: [cluster] -"#; - let temp = { - let temp = tempfile::tempdir().unwrap(); - fs::write( - temp.path().join("people.pg"), - "\nnode Person {\n name: String @key\n}\n", - ) - .unwrap(); - fs::write( - temp.path().join("people.gq"), - "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", - ) - .unwrap(); - fs::write( - temp.path().join("graph.policy.yaml"), - permit_all_policy_yaml(&["default"]), - ) - .unwrap(); - fs::write( - temp.path().join("cluster.policy.yaml"), - permit_all_policy_yaml(&["default"]).replace("protected_branches: [main]\n", "protected_branches: [main]\nkind: server\n"), - ) - .unwrap(); - fs::write( - temp.path().join("cluster.yaml"), - format!( - r#" -version: 1 -graphs: - knowledge: - schema: ./people.pg - queries: - find_person: - file: ./people.gq -{policy_block}"# - ), - ) - .unwrap(); - let import = omnigraph_cluster::import_config_dir(temp.path()).await; - assert!(import.ok, "{:?}", import.diagnostics); - let apply = omnigraph_cluster::apply_config_dir(temp.path()).await; - assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); - temp - }; - - let settings = cluster_settings(temp.path()).await.unwrap(); - let omnigraph_server::ServerConfigMode::Multi { - graphs, - server_policy_file, - .. - } = settings.mode - else { - panic!("cluster boot must select multi-graph routing"); - }; - let graph_policy = graphs[0].policy_file.as_ref().expect("graph-bound bundle"); - assert!( - graph_policy - .to_string_lossy() - .contains("__cluster/resources/policy/graph_rules/"), - "{graph_policy:?}" - ); - let server_policy = server_policy_file.expect("cluster-bound bundle"); - assert!( - server_policy - .to_string_lossy() - .contains("__cluster/resources/policy/cluster_rules/"), - "{server_policy:?}" - ); -} - -#[tokio::test] -async fn cluster_boot_refusals() { - // Mutual exclusion with --config / URI. - let temp = converged_cluster_dir("").await; - let dir = temp.path().to_path_buf(); - let err = omnigraph_server::load_server_settings( - Some(&dir.join("omnigraph.yaml")), - Some(&dir), - None, - None, - None, - true, - ) - .await - .unwrap_err(); - assert!(err.to_string().contains("exclusive boot source"), "{err}"); - let err = omnigraph_server::load_server_settings( - None, - Some(&dir), - Some("file:///tmp/x.omni".to_string()), - None, - None, - true, - ) - .await - .unwrap_err(); - assert!(err.to_string().contains("exclusive boot source"), "{err}"); - - // Tampered catalog blob refuses boot with the remedy. - let blob_dir = dir.join("__cluster/resources/query/knowledge/find_person"); - let blob = fs::read_dir(&blob_dir).unwrap().next().unwrap().unwrap().path(); - fs::write(&blob, "tampered").unwrap(); - let err = cluster_settings(&dir).await.unwrap_err(); - assert!( - err.to_string().contains("catalog_payload_digest_mismatch"), - "{err}" - ); - assert!(err.to_string().contains("cluster refresh"), "{err}"); - - // Missing state refuses with the import/apply remedy. - let empty = tempfile::tempdir().unwrap(); - let err = cluster_settings(empty.path()).await.unwrap_err(); - assert!(err.to_string().contains("cluster_state_missing"), "{err}"); -} diff --git a/crates/omnigraph-server/tests/stored_queries.rs b/crates/omnigraph-server/tests/stored_queries.rs new file mode 100644 index 0000000..e4da1d3 --- /dev/null +++ b/crates/omnigraph-server/tests/stored_queries.rs @@ -0,0 +1,329 @@ +//! Stored-query registry boot, /queries listing, and invocation routes. +//! Moved verbatim from tests/server.rs in the modularization. + + +use axum::body::Body; +use axum::http::StatusCode; +use omnigraph_server::AppState; +use serde_json::json; + + +mod support; +use support::*; + +#[tokio::test] +async fn server_boots_with_a_valid_stored_query_registry() { + // A stored query that type-checks against the fixture schema + // (`Person { name, age }`) must let the server boot. + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let registry = stored_query_registry(&[( + "find_person", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + false, + )]); + let state = AppState::open_single_with_queries( + graph.to_string_lossy().to_string(), + vec![], + None, + registry, + ) + .await; + assert!(state.is_ok(), "valid registry should boot: {:?}", state.err()); +} + +#[tokio::test] +async fn server_refuses_boot_on_type_broken_stored_query() { + // A stored query referencing a type not in the schema (`Widget`) + // must abort boot, naming the offending query. + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let registry = stored_query_registry(&[( + "ghost", + "query ghost() { match { $w: Widget } return { $w.name } }", + false, + )]); + let result = AppState::open_single_with_queries( + graph.to_string_lossy().to_string(), + vec![], + None, + registry, + ) + .await; + // `AppState` is not `Debug`, so match rather than `expect_err`. + let err = match result { + Ok(_) => panic!("type-broken stored query must refuse boot"), + Err(err) => err, + }; + let msg = err.to_string(); + assert!(msg.contains("ghost"), "error should name the broken query: {msg}"); + assert!( + msg.contains("schema check"), + "error should mention the schema check: {msg}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_stored_read_returns_rows() { + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, false)], + &[("act-invoke", "t-invoke")], + INVOKE_POLICY_YAML, + ) + .await; + let (status, body) = json_response( + &app, + invoke_request("find_person", "t-invoke", json!({ "params": { "name": "Alice" } })), + ) + .await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + assert_eq!(body["query_name"], "find_person"); + assert_eq!(body["row_count"], 1, "Alice is in the fixture; body: {body}"); + assert!(body["rows"].is_array(), "read envelope shape; body: {body}"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_stored_read_accepts_absent_or_empty_body() { + let no_param_query = "query list_people() { match { $p: Person } return { $p.name } }"; + let (_temp, app) = app_with_stored_queries( + &[("list_people", no_param_query, false)], + &[("act-invoke", "t-invoke")], + INVOKE_POLICY_YAML, + ) + .await; + + let (status, body) = json_response( + &app, + invoke_request_bytes("list_people", "t-invoke", Body::empty(), None), + ) + .await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + assert_eq!(body["query_name"], "list_people"); + + let (status, body) = json_response( + &app, + invoke_request_bytes( + "list_people", + "t-invoke", + Body::empty(), + Some("application/json"), + ), + ) + .await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + + let (status, body) = json_response( + &app, + invoke_request_bytes( + "list_people", + "t-invoke", + Body::from("{}"), + Some("application/json"), + ), + ) + .await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + + let (status, body) = json_response( + &app, + invoke_request_bytes( + "list_people", + "t-invoke", + Body::from("{"), + Some("application/json"), + ), + ) + .await; + assert_eq!(status, StatusCode::BAD_REQUEST, "body: {body}"); + assert!( + body["error"] + .as_str() + .unwrap_or_default() + .contains("invalid stored-query invocation body"), + "malformed JSON should be rejected as bad request; body: {body}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_stored_mutation_double_gates_on_change() { + let specs: &[(&str, &str, bool)] = &[( + "add_person", + "query add_person($name: String) { insert Person { name: $name } }", + false, + )]; + let (_temp, app) = app_with_stored_queries( + specs, + &[("act-invoke", "t-invoke"), ("act-full", "t-full")], + INVOKE_POLICY_YAML, + ) + .await; + + // Has invoke_query but NOT change → the inner change gate denies (403). + let (status, body) = json_response( + &app, + invoke_request("add_person", "t-invoke", json!({ "params": { "name": "Eve" } })), + ) + .await; + assert_eq!( + status, + StatusCode::FORBIDDEN, + "invoke_query without change must 403; body: {body}" + ); + + // Has invoke_query + change → applied. + let (status, body) = json_response( + &app, + invoke_request("add_person", "t-full", json!({ "params": { "name": "Eve" } })), + ) + .await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + assert_eq!(body["affected_nodes"], 1, "body: {body}"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_stored_query_bad_param_is_400() { + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, false)], + &[("act-invoke", "t-invoke")], + INVOKE_POLICY_YAML, + ) + .await; + // `name` is declared String; pass a number. + let (status, body) = json_response( + &app, + invoke_request("find_person", "t-invoke", json!({ "params": { "name": 123 } })), + ) + .await; + assert_eq!(status, StatusCode::BAD_REQUEST, "body: {body}"); + assert!( + body["error"].as_str().unwrap_or_default().contains("name"), + "400 should name the offending param; body: {body}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_unknown_query_and_denied_actor_return_identical_404() { + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, false)], + &[("act-invoke", "t-invoke"), ("act-noinvoke", "t-noinvoke")], + INVOKE_POLICY_YAML, + ) + .await; + + // Authorized actor, unknown query name → 404. + let (unknown_status, unknown_body) = + json_response(&app, invoke_request("does_not_exist", "t-invoke", json!({}))).await; + // Denied actor (no invoke_query), real query name → 404. + let (denied_status, denied_body) = json_response( + &app, + invoke_request("find_person", "t-noinvoke", json!({ "params": { "name": "Alice" } })), + ) + .await; + + assert_eq!(unknown_status, StatusCode::NOT_FOUND); + assert_eq!(denied_status, StatusCode::NOT_FOUND); + assert_eq!( + unknown_body, denied_body, + "deny must be byte-identical to a missing query (no catalog probing)" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn invoke_query_holder_without_read_sees_403_not_404() { + // The 404-hiding is for callers WITHOUT invoke_query. An actor that + // HOLDS invoke_query but lacks `read` clears the boundary gate, then the + // inner read gate denies → 403 for an EXISTING read query, vs 404 for an + // unknown one. Existence is visible to grant-holders by design (the + // documented double-gate); this pins that actual contract. + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, false)], + &[("act-invokeonly", "t-invokeonly")], + INVOKE_POLICY_YAML, + ) + .await; + let (exists_status, _) = json_response( + &app, + invoke_request("find_person", "t-invokeonly", json!({ "params": { "name": "Alice" } })), + ) + .await; + let (absent_status, _) = + json_response(&app, invoke_request("does_not_exist", "t-invokeonly", json!({}))).await; + assert_eq!( + exists_status, + StatusCode::FORBIDDEN, + "an existing read query the holder can't read → inner-gate 403" + ); + assert_eq!(absent_status, StatusCode::NOT_FOUND, "unknown query still 404s"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn list_queries_returns_only_exposed_with_typed_params() { + let (_temp, app) = app_with_stored_queries( + &[ + ("find_person", FIND_PERSON_GQ, true), + ( + "add_person", + "query add_person($name: String) { insert Person { name: $name } }", + true, + ), + ("hidden", "query hidden() { match { $p: Person } return { $p.name } }", false), + ], + &[("act-invoke", "t-invoke")], + INVOKE_POLICY_YAML, + ) + .await; + let (status, body) = json_response(&app, get_request("/queries", "t-invoke")).await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + + let entries = body["queries"].as_array().unwrap(); + let names: Vec<&str> = entries.iter().map(|q| q["name"].as_str().unwrap()).collect(); + assert!( + names.contains(&"find_person") && names.contains(&"add_person"), + "exposed queries listed: {names:?}" + ); + assert!(!names.contains(&"hidden"), "non-exposed query hidden from the catalog: {names:?}"); + + let fp = entries.iter().find(|q| q["name"] == "find_person").unwrap(); + assert_eq!(fp["mutation"], false); + assert_eq!(fp["tool_name"], "find_person"); + assert_eq!(fp["params"][0]["name"], "name"); + assert_eq!(fp["params"][0]["kind"], "string"); + let ap = entries.iter().find(|q| q["name"] == "add_person").unwrap(); + assert_eq!(ap["mutation"], true, "stored insert → mutation"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn list_queries_is_read_gated_so_a_non_invoker_can_list() { + // The catalog is read-gated (not invoke_query-gated), so a reader who + // lacks invoke_query still enumerates the exposed queries — the + // documented probe-oracle gap until per-query Cedar filtering lands. + let (_temp, app) = app_with_stored_queries( + &[("find_person", FIND_PERSON_GQ, true)], + &[("act-noinvoke", "t-noinvoke")], + INVOKE_POLICY_YAML, + ) + .await; + let (status, body) = json_response(&app, get_request("/queries", "t-noinvoke")).await; + assert_eq!(status, StatusCode::OK, "read-gated catalog; body: {body}"); + let names: Vec<&str> = body["queries"] + .as_array() + .unwrap() + .iter() + .map(|q| q["name"].as_str().unwrap()) + .collect(); + assert!( + names.contains(&"find_person"), + "a reader lists the catalog despite lacking invoke_query: {names:?}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn list_queries_is_empty_when_no_registry() { + let (_temp, app) = app_for_loaded_graph_with_auth("demo-token").await; + let (status, body) = json_response(&app, get_request("/queries", "demo-token")).await; + assert_eq!(status, StatusCode::OK, "body: {body}"); + assert!( + body["queries"].as_array().unwrap().is_empty(), + "no stored-query registry → empty catalog" + ); +} diff --git a/crates/omnigraph-server/tests/support/mod.rs b/crates/omnigraph-server/tests/support/mod.rs new file mode 100644 index 0000000..0e32410 --- /dev/null +++ b/crates/omnigraph-server/tests/support/mod.rs @@ -0,0 +1,1195 @@ +//! Shared helpers for the server integration suites (moved verbatim +//! from the monolithic tests/server.rs in the modularization). +#![allow(dead_code)] + +use std::env; +use std::fs; +use std::path::{Path, PathBuf}; +use std::sync::Arc; + +use axum::Router; +use axum::body::{Body, to_bytes}; +use axum::http::header::AUTHORIZATION; +use axum::http::{Method, Request, StatusCode}; +use omnigraph::db::{Omnigraph, ReadTarget}; +use omnigraph::error::OmniError; +use omnigraph::loader::{LoadMode, load_jsonl}; +use omnigraph_policy::{PolicyChecker, PolicyEngine}; +use omnigraph_server::api::{ + BranchCreateRequest, BranchMergeRequest, ChangeRequest, ReadRequest, +}; +use omnigraph_server::queries::{QueryRegistry, RegistrySpec}; +use omnigraph_server::{AppState, build_app}; +use serde_json::{Value, json}; +use tower::ServiceExt; + + +pub const MUTATION_QUERIES: &str = r#" +query insert_person($name: String, $age: I32) { + insert Person { name: $name, age: $age } +} + +query set_age($name: String, $age: I32) { + update Person set { age: $age } where name = $name +} +"#; + +pub const POLICY_YAML: &str = r#" +version: 1 +groups: + team: [act-andrew, act-bruno, act-ragnor] + admins: [act-ragnor] +protected_branches: [main] +rules: + - id: team-read + allow: + actors: { group: team } + actions: [read] + branch_scope: any + - id: admins-export + allow: + actors: { group: admins } + actions: [export] + branch_scope: any + - id: team-write-unprotected + allow: + actors: { group: team } + actions: [change] + branch_scope: unprotected + - id: admins-merge + allow: + actors: { group: admins } + actions: [branch_delete, branch_merge] + target_branch_scope: protected +"#; + +pub const POLICY_PROTECTED_READ_YAML: &str = r#" +version: 1 +groups: + team: [act-bruno] +protected_branches: [main] +rules: + - id: protected-read + allow: + actors: { group: team } + actions: [read] + branch_scope: protected +"#; + +pub const INGEST_CREATE_ONLY_POLICY_YAML: &str = r#" +version: 1 +groups: + team: [act-bruno] +protected_branches: [main] +rules: + - id: team-branch-create + allow: + actors: { group: team } + actions: [branch_create] + target_branch_scope: unprotected +"#; + +pub const SCHEMA_APPLY_POLICY_YAML: &str = r#" +version: 1 +groups: + admins: [act-ragnor] +protected_branches: [main] +rules: + - id: admins-schema-apply + allow: + actors: { group: admins } + actions: [schema_apply] + target_branch_scope: protected +"#; + +pub fn fixture(name: &str) -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("../omnigraph/tests/fixtures") + .join(name) +} + +pub async fn init_loaded_graph() -> tempfile::TempDir { + init_graph_with_schema_and_data( + &fs::read_to_string(fixture("test.pg")).unwrap(), + &fs::read_to_string(fixture("test.jsonl")).unwrap(), + ) + .await +} + +pub async fn init_graph_with_schema_and_data(schema: &str, data: &str) -> tempfile::TempDir { + let temp = tempfile::tempdir().unwrap(); + let graph = graph_path(temp.path()); + fs::create_dir_all(&graph).unwrap(); + Omnigraph::init(graph.to_str().unwrap(), schema) + .await + .unwrap(); + let mut db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + load_jsonl(&mut db, data, LoadMode::Overwrite) + .await + .unwrap(); + temp +} + +pub async fn init_graph_with_schema(schema: &str) -> tempfile::TempDir { + let temp = tempfile::tempdir().unwrap(); + let graph = graph_path(temp.path()); + fs::create_dir_all(&graph).unwrap(); + Omnigraph::init(graph.to_str().unwrap(), schema) + .await + .unwrap(); + temp +} + +pub fn graph_path(root: &Path) -> PathBuf { + root.join("server.omni") +} + +pub fn stored_query_registry(specs: &[(&str, &str, bool)]) -> QueryRegistry { + QueryRegistry::from_specs( + specs + .iter() + .map(|(name, source, expose)| RegistrySpec { + name: name.to_string(), + source: source.to_string(), + expose: *expose, + tool_name: None, + }) + .collect(), + ) + .expect("specs parse and key==symbol") +} + +pub async fn app_with_stored_queries( + specs: &[(&str, &str, bool)], + tokens: &[(&str, &str)], + policy: &str, +) -> (tempfile::TempDir, Router) { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, policy).unwrap(); + let registry = stored_query_registry(specs); + let state = AppState::open_single_with_queries( + graph.to_string_lossy().to_string(), + tokens + .iter() + .map(|(actor, token)| ((*actor).to_string(), (*token).to_string())) + .collect(), + Some(&policy_path), + registry, + ) + .await + .unwrap(); + (temp, build_app(state)) +} + +pub const INVOKE_POLICY_YAML: &str = r#" +version: 1 +groups: + invokers: ["act-invoke"] + full: ["act-full"] + readers: ["act-noinvoke"] + invoke_only: ["act-invokeonly"] +protected_branches: [main] +rules: + # invoke_query is graph-scoped — its own rules, no branch_scope. + - id: invokers-can-invoke + allow: + actors: { group: invokers } + actions: [invoke_query] + - id: full-can-invoke + allow: + actors: { group: full } + actions: [invoke_query] + - id: invoke-only-can-invoke + allow: + actors: { group: invoke_only } + actions: [invoke_query] + # read / change are branch-scoped. + - id: invokers-can-read + allow: + actors: { group: invokers } + actions: [read] + branch_scope: any + - id: full-can-read-change + allow: + actors: { group: full } + actions: [read, change] + branch_scope: any + - id: readers-can-read + allow: + actors: { group: readers } + actions: [read] + branch_scope: any +"#; + +pub const STORED_QUERY_SCHEMA_APPLY_POLICY_YAML: &str = r#" +version: 1 +groups: + admins: [act-ragnor] +protected_branches: [main] +rules: + - id: admins-can-invoke + allow: + actors: { group: admins } + actions: [invoke_query] + - id: admins-can-read + allow: + actors: { group: admins } + actions: [read] + branch_scope: any + - id: admins-can-schema-apply + allow: + actors: { group: admins } + actions: [schema_apply] + target_branch_scope: protected +"#; + +pub const FIND_PERSON_GQ: &str = + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }"; + +pub fn invoke_request(name: &str, token: &str, body: Value) -> Request<Body> { + Request::builder() + .uri(format!("/queries/{name}")) + .method(Method::POST) + .header("content-type", "application/json") + .header("authorization", format!("Bearer {token}")) + .body(Body::from(serde_json::to_vec(&body).unwrap())) + .unwrap() +} + +pub fn invoke_request_bytes( + name: &str, + token: &str, + body: impl Into<Body>, + content_type: Option<&str>, +) -> Request<Body> { + let mut builder = Request::builder() + .uri(format!("/queries/{name}")) + .method(Method::POST) + .header("authorization", format!("Bearer {token}")); + if let Some(content_type) = content_type { + builder = builder.header("content-type", content_type); + } + builder.body(body.into()).unwrap() +} + +pub fn get_request(uri: &str, token: &str) -> Request<Body> { + Request::builder() + .uri(uri) + .method(Method::GET) + .header("authorization", format!("Bearer {token}")) + .body(Body::empty()) + .unwrap() +} + +pub fn drifted_test_schema() -> String { + fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace("age: I32?", "age: I64?") +} + +pub async fn manifest_dataset_version(graph: &Path) -> u64 { + Omnigraph::open(graph.to_string_lossy().as_ref()) + .await + .unwrap() + .snapshot_of(ReadTarget::branch("main")) + .await + .unwrap() + .version() +} + +pub fn s3_test_graph_uri(suite: &str) -> Option<String> { + let bucket = env::var("OMNIGRAPH_S3_TEST_BUCKET").ok()?; + let prefix = env::var("OMNIGRAPH_S3_TEST_PREFIX") + .ok() + .filter(|value| !value.trim().is_empty()) + .unwrap_or_else(|| "omnigraph-itests".to_string()); + let unique = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .ok()? + .as_nanos(); + Some(format!("s3://{}/{}/{}/{}", bucket, prefix, suite, unique)) +} + +pub async fn app_for_loaded_graph() -> (tempfile::TempDir, Router) { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let state = AppState::open(graph.to_string_lossy().to_string()) + .await + .unwrap(); + (temp, build_app(state)) +} + +pub fn permit_all_policy_yaml(actors: &[&str]) -> String { + let members = actors + .iter() + .map(|a| format!("\"{a}\"")) + .collect::<Vec<_>>() + .join(", "); + format!( + r#" +version: 1 +groups: + permitted: [{members}] +protected_branches: [main] +rules: + - id: permit-data + allow: + actors: {{ group: permitted }} + actions: [read, change, export] + branch_scope: any + - id: permit-protected-target-actions + allow: + actors: {{ group: permitted }} + actions: [schema_apply, branch_create, branch_delete, branch_merge] + target_branch_scope: any +"# + ) +} + +pub async fn app_for_loaded_graph_with_auth(token: &str) -> (tempfile::TempDir, Router) { + // `AppState::new_with_bearer_token(token)` maps the token to actor "default"; + // permit-all policy needs to include that actor. + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, permit_all_policy_yaml(&["default"])).unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![("default".to_string(), token.to_string())], + Some(&policy_path), + ) + .await + .unwrap(); + (temp, build_app(state)) +} + +pub async fn app_for_loaded_graph_with_auth_tokens( + tokens: &[(&str, &str)], +) -> (tempfile::TempDir, Router) { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let policy_path = temp.path().join("policy.yaml"); + let actors: Vec<&str> = tokens.iter().map(|(actor, _)| *actor).collect(); + fs::write(&policy_path, permit_all_policy_yaml(&actors)).unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + tokens + .iter() + .map(|(actor, token)| ((*actor).to_string(), (*token).to_string())) + .collect(), + Some(&policy_path), + ) + .await + .unwrap(); + (temp, build_app(state)) +} + +pub async fn app_for_loaded_graph_with_auth_tokens_and_policy( + tokens: &[(&str, &str)], + policy: &str, +) -> (tempfile::TempDir, Router) { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, policy).unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + tokens + .iter() + .map(|(actor, token)| ((*actor).to_string(), (*token).to_string())) + .collect(), + Some(&policy_path), + ) + .await + .unwrap(); + (temp, build_app(state)) +} + +pub async fn app_for_graph_with_auth_tokens_and_policy( + schema: &str, + tokens: &[(&str, &str)], + policy: &str, +) -> (tempfile::TempDir, Router) { + let temp = init_graph_with_schema(schema).await; + let graph = graph_path(temp.path()); + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, policy).unwrap(); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + tokens + .iter() + .map(|(actor, token)| ((*actor).to_string(), (*token).to_string())) + .collect(), + Some(&policy_path), + ) + .await + .unwrap(); + (temp, build_app(state)) +} + +pub async fn app_for_graph_with_auth_tokens_only( + schema: &str, + tokens: &[(&str, &str)], +) -> (tempfile::TempDir, Router) { + let temp = init_graph_with_schema(schema).await; + let graph = graph_path(temp.path()); + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + tokens + .iter() + .map(|(actor, token)| ((*actor).to_string(), (*token).to_string())) + .collect(), + None, + ) + .await + .unwrap(); + (temp, build_app(state)) +} + +pub fn additive_schema_with_nickname() -> String { + fs::read_to_string(fixture("test.pg")).unwrap().replace( + " age: I32?\n}", + " age: I32?\n nickname: String?\n}", + ) +} + +pub fn schema_without_age() -> String { + // Drop the nullable `age` column from the test schema. Used by the + // HTTP soft/hard drop tests below. + fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace(" age: I32?\n", "") +} + +pub fn schema_without_company() -> String { + // Drop the `Company` node type and the edge referencing it. Used + // by the HTTP DropType test below. Hand-crafted (no template + // string replace) because the fixture interleaves the type and + // its edge. + r#"node Person { + name: String @key + age: I32? +} + +edge Knows: Person -> Person { + since: Date? +} +"# + .to_string() +} + +pub fn renamed_person_schema() -> String { + fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace("node Person {\n", "node Human @rename_from(\"Person\") {\n") + .replace("edge Knows: Person -> Person", "edge Knows: Human -> Human") + .replace( + "edge WorksAt: Person -> Company", + "edge WorksAt: Human -> Company", + ) +} + +pub fn renamed_age_schema() -> String { + fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace("age: I32?", "years: I32? @rename_from(\"age\")") +} + +pub fn indexed_name_schema() -> String { + fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace("name: String @key", "name: String @key @index") +} + +pub fn unsupported_schema_change() -> String { + fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace("age: I32?", "age: I64?") +} + +pub async fn json_response(app: &Router, request: Request<Body>) -> (StatusCode, Value) { + let response = app.clone().oneshot(request).await.unwrap(); + let status = response.status(); + let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + let value = serde_json::from_slice(&body).unwrap(); + (status, value) +} + +pub struct EnvGuard { + saved: Vec<(&'static str, Option<String>)>, +} + +impl EnvGuard { + pub fn set(vars: &[(&'static str, Option<&str>)]) -> Self { + let saved = vars + .iter() + .map(|(name, _)| (*name, env::var(name).ok())) + .collect::<Vec<_>>(); + for (name, value) in vars { + unsafe { + match value { + Some(value) => env::set_var(name, value), + None => env::remove_var(name), + } + } + } + Self { saved } + } +} + +impl Drop for EnvGuard { + fn drop(&mut self) { + for (name, value) in self.saved.drain(..) { + unsafe { + match value { + Some(value) => env::set_var(name, value), + None => env::remove_var(name), + } + } + } + } +} + +pub fn format_vector(values: &[f32]) -> String { + values + .iter() + .map(|value| format!("{:.8}", value)) + .collect::<Vec<_>>() + .join(", ") +} + +pub fn normalize_vector(mut values: Vec<f32>) -> Vec<f32> { + let norm = values + .iter() + .map(|value| (*value as f64) * (*value as f64)) + .sum::<f64>() + .sqrt() as f32; + if norm > f32::EPSILON { + for value in &mut values { + *value /= norm; + } + } + values +} + +pub fn fnv1a64(bytes: &[u8]) -> u64 { + let mut hash = 14695981039346656037u64; + for byte in bytes { + hash ^= *byte as u64; + hash = hash.wrapping_mul(1099511628211u64); + } + hash +} + +pub fn xorshift64(mut x: u64) -> u64 { + x ^= x << 13; + x ^= x >> 7; + x ^= x << 17; + x +} + +pub fn mock_embedding(input: &str, dim: usize) -> Vec<f32> { + let mut seed = fnv1a64(input.as_bytes()); + let mut out = Vec::with_capacity(dim); + for _ in 0..dim { + seed = xorshift64(seed); + let ratio = (seed as f64 / u64::MAX as f64) as f32; + out.push((ratio * 2.0) - 1.0); + } + normalize_vector(out) +} + +pub mod matrix { + use super::*; + use std::time::Duration; + use tokio::sync::Barrier; + + #[derive(Debug)] + pub struct OpStatus { + pub status: StatusCode, + pub body: Vec<u8>, + } + + pub struct Harness { + pub _temp: tempfile::TempDir, + pub app: Router, + } + + impl Harness { + pub async fn new() -> Self { + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + // Build the WorkloadController explicitly with defaults rather + // than letting `AppState::open` call + // `WorkloadController::from_env()`. The admission-gate test + // (`ingest_per_actor_admission_cap_returns_429`) sets + // OMNIGRAPH_PER_ACTOR_INFLIGHT_MAX=1 inside an EnvGuard while + // it runs. Process-wide env vars are visible to + // concurrently-running tests; if a matrix cell reads env at + // AppState construction time during that window it picks up + // cap=1 and the second concurrent merge in cell b surfaces + // 429 instead of the expected 200. Constructing the + // controller here with explicit defaults makes cells + // independent of any env mutation other tests perform. + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + let workload = omnigraph_server::workload::WorkloadController::with_defaults(); + let state = AppState::new_with_workload( + graph.to_string_lossy().to_string(), + db, + Vec::new(), + workload, + ); + let app = build_app(state); + Self { _temp: temp, app } + } + + pub async fn create_branch(&self, from: &str, name: &str) { + let body = serde_json::to_vec(&BranchCreateRequest { + from: Some(from.to_string()), + name: name.to_string(), + }) + .unwrap(); + let r = self + .app + .clone() + .oneshot( + Request::builder() + .uri("/branches") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!( + r.status(), + StatusCode::OK, + "setup create_branch {} from {} failed", + name, + from + ); + } + + pub async fn insert_person(&self, branch: &str, name: &str, age: i32) { + let body = serde_json::to_vec(&ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": name, "age": age })), + branch: Some(branch.to_string()), + }) + .unwrap(); + let r = self + .app + .clone() + .oneshot( + Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!( + r.status(), + StatusCode::OK, + "setup insert {} on {} failed", + name, + branch + ); + } + + /// Run two ops concurrently with barrier alignment + 15s deadlock + /// timeout. Returns `(op_a, op_b)`. Panics on timeout. + pub async fn run_pair( + &self, + op_a: impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus>, + op_b: impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus>, + ) -> (OpStatus, OpStatus) { + let barrier = Arc::new(Barrier::new(2)); + let h_a = op_a(self.app.clone(), Arc::clone(&barrier)); + let h_b = op_b(self.app.clone(), Arc::clone(&barrier)); + let result = tokio::time::timeout(Duration::from_secs(15), async { + let a = h_a.await.unwrap(); + let b = h_b.await.unwrap(); + (a, b) + }) + .await; + result.expect("concurrent op pair deadlocked (>15s)") + } + + pub async fn person_count(&self, branch: &str) -> u64 { + let r = self + .app + .clone() + .oneshot( + Request::builder() + .uri(format!("/snapshot?branch={}", branch)) + .method(Method::GET) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(r.status(), StatusCode::OK, "snapshot {} failed", branch); + let body = to_bytes(r.into_body(), usize::MAX).await.unwrap(); + let v: Value = serde_json::from_slice(&body).unwrap(); + v["tables"] + .as_array() + .and_then(|tables| { + tables + .iter() + .find(|t| t["table_key"].as_str() == Some("node:Person")) + }) + .and_then(|t| t["row_count"].as_u64()) + .unwrap_or_else(|| panic!("snapshot {} missing node:Person", branch)) + } + + /// True iff the named Person exists on `branch`. Uses the + /// `get_person` query from `test.gq` for identity rather than + /// just count. + pub async fn person_exists(&self, branch: &str, name: &str) -> bool { + let body = serde_json::to_vec(&ReadRequest { + query_source: include_str!("../../../omnigraph/tests/fixtures/test.gq").to_string(), + query_name: Some("get_person".to_string()), + params: Some(json!({ "name": name })), + branch: Some(branch.to_string()), + snapshot: None, + }) + .unwrap(); + let r = self + .app + .clone() + .oneshot( + Request::builder() + .uri("/read") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!( + r.status(), + StatusCode::OK, + "person_exists query for {} on {} failed", + name, + branch + ); + let body = to_bytes(r.into_body(), usize::MAX).await.unwrap(); + let v: Value = serde_json::from_slice(&body).unwrap(); + v["row_count"].as_u64().unwrap_or(0) > 0 + } + + /// Asserts each name in `present` exists on `branch` and each in + /// `absent` does not. Identity-grade check that catches symmetric + /// swap races a row-count assertion would miss. + pub async fn assert_persons( + &self, + branch: &str, + cell: &str, + present: &[&str], + absent: &[&str], + ) { + for name in present { + assert!( + self.person_exists(branch, name).await, + "[{}] expected {} to be present on {}", + cell, + name, + branch + ); + } + for name in absent { + assert!( + !self.person_exists(branch, name).await, + "[{}] expected {} to be absent from {}", + cell, + name, + branch + ); + } + } + + /// C6: insert a uniquely-named sentinel on main and verify it + /// landed. Catches engine-state poisoning where a cell's + /// concurrent ops left the engine half-broken — subsequent + /// /change either deadlocks or returns a non-200. + pub async fn assert_post_op_sentinel(&self, cell: &str, sentinel: &str) { + let body = serde_json::to_vec(&ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": sentinel, "age": 99 })), + branch: Some("main".to_string()), + }) + .unwrap(); + let r = self + .app + .clone() + .oneshot( + Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!( + r.status(), + StatusCode::OK, + "[{}] post-op sentinel /change on main failed (engine poisoned?)", + cell + ); + assert!( + self.person_exists("main", sentinel).await, + "[{}] sentinel {} did not land on main", + cell, + sentinel + ); + } + } + + // Helpers that build the closures for `run_pair`. Each takes a + // Router + Barrier and returns a JoinHandle yielding the status/body. + + pub fn op_merge( + source: String, + target: String, + ) -> impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus> { + move |app: Router, barrier: Arc<Barrier>| { + tokio::spawn(async move { + barrier.wait().await; + let body = serde_json::to_vec(&BranchMergeRequest { + source, + target: Some(target), + }) + .unwrap(); + let response = app + .oneshot( + Request::builder() + .uri("/branches/merge") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(), + ) + .await + .unwrap(); + let status = response.status(); + let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + OpStatus { + status, + body: body.to_vec(), + } + }) + } + } + + pub fn op_change_insert( + branch: String, + name: String, + age: i32, + ) -> impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus> { + move |app: Router, barrier: Arc<Barrier>| { + tokio::spawn(async move { + barrier.wait().await; + let body = serde_json::to_vec(&ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": name, "age": age })), + branch: Some(branch), + }) + .unwrap(); + let response = app + .oneshot( + Request::builder() + .uri("/change") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(), + ) + .await + .unwrap(); + let status = response.status(); + let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + OpStatus { + status, + body: body.to_vec(), + } + }) + } + } + + pub fn op_branch_create( + from: String, + name: String, + ) -> impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus> { + move |app: Router, barrier: Arc<Barrier>| { + tokio::spawn(async move { + barrier.wait().await; + let body = serde_json::to_vec(&BranchCreateRequest { + from: Some(from), + name, + }) + .unwrap(); + let response = app + .oneshot( + Request::builder() + .uri("/branches") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(body)) + .unwrap(), + ) + .await + .unwrap(); + let status = response.status(); + let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + OpStatus { + status, + body: body.to_vec(), + } + }) + } + } + + pub fn op_branch_delete( + name: String, + ) -> impl FnOnce(Router, Arc<Barrier>) -> tokio::task::JoinHandle<OpStatus> { + move |app: Router, barrier: Arc<Barrier>| { + tokio::spawn(async move { + barrier.wait().await; + let response = app + .oneshot( + Request::builder() + .uri(format!("/branches/{}", name)) + .method(Method::DELETE) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + let status = response.status(); + let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + OpStatus { + status, + body: body.to_vec(), + } + }) + } + } +} + +pub const PARITY_POLICY_YAML: &str = r#" +version: 1 +groups: + team: [act-bruno] + admins: [act-ragnor] +protected_branches: [main] +rules: + - id: admins-change-anywhere + allow: + actors: { group: admins } + actions: [change] + branch_scope: any + - id: admins-merge-to-protected + allow: + actors: { group: admins } + actions: [branch_merge] + target_branch_scope: protected +"#; + +#[derive(Clone, Copy, Debug)] +pub enum ParityDecision { + Allow, + Deny, +} + +pub async fn build_parity_graph() -> (tempfile::TempDir, PathBuf, PathBuf) { + // Build a graph with `main` loaded and a `feature` branch ready for + // merge. Returns the graph path and a written policy.yaml path. + let temp = init_loaded_graph().await; + let graph = graph_path(temp.path()); + { + let db = Omnigraph::open(graph.to_str().unwrap()).await.unwrap(); + db.branch_create_from(ReadTarget::branch("main"), "feature") + .await + .unwrap(); + db.load_as( + "feature", + None, + r#"{"type":"Person","data":{"name":"ParityEve","age":29}}"#, + LoadMode::Append, + None, + ) + .await + .unwrap(); + } + let policy_path = temp.path().join("policy.yaml"); + fs::write(&policy_path, PARITY_POLICY_YAML).unwrap(); + (temp, graph, policy_path) +} + +pub async fn sdk_change_decision(graph: &Path, policy_path: &Path, actor: &str) -> ParityDecision { + let policy = PolicyEngine::load_graph(policy_path, graph.to_string_lossy().as_ref()).unwrap(); + let db = Omnigraph::open(graph.to_str().unwrap()) + .await + .unwrap() + .with_policy(Arc::new(policy) as Arc<dyn PolicyChecker>); + let mut params: omnigraph_compiler::ParamMap = Default::default(); + // Parameter keys are bare names (no `$` prefix); the runtime resolves + // `$name` references in the query body to `params["name"]`. + params.insert( + "name".to_string(), + omnigraph_compiler::Literal::String("ParityCharlie".to_string()), + ); + params.insert("age".to_string(), omnigraph_compiler::Literal::Integer(30)); + let result = db + .mutate_as( + "main", + MUTATION_QUERIES, + "insert_person", + ¶ms, + Some(actor), + ) + .await; + match result { + Ok(_) => ParityDecision::Allow, + Err(OmniError::Policy(_)) => ParityDecision::Deny, + Err(other) => panic!("unexpected SDK error for change: {other:?}"), + } +} + +pub async fn http_change_decision( + graph: &Path, + policy_path: &PathBuf, + actor: &str, + token: &str, +) -> ParityDecision { + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![(actor.to_string(), token.to_string())], + Some(policy_path), + ) + .await + .unwrap(); + let app = build_app(state); + let req = ChangeRequest { + query: MUTATION_QUERIES.to_string(), + name: Some("insert_person".to_string()), + params: Some(json!({ "name": "ParityCharlie", "age": 30 })), + branch: Some("main".to_string()), + }; + let (status, _body) = json_response( + &app, + Request::builder() + .uri("/change") + .method(Method::POST) + .header(AUTHORIZATION, format!("Bearer {token}")) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&req).unwrap())) + .unwrap(), + ) + .await; + match status { + StatusCode::OK => ParityDecision::Allow, + StatusCode::FORBIDDEN => ParityDecision::Deny, + other => panic!("unexpected HTTP status for change: {other}"), + } +} + +pub async fn sdk_merge_decision(graph: &Path, policy_path: &Path, actor: &str) -> ParityDecision { + let policy = PolicyEngine::load_graph(policy_path, graph.to_string_lossy().as_ref()).unwrap(); + let db = Omnigraph::open(graph.to_str().unwrap()) + .await + .unwrap() + .with_policy(Arc::new(policy) as Arc<dyn PolicyChecker>); + let result = db.branch_merge_as("feature", "main", Some(actor)).await; + match result { + Ok(_) => ParityDecision::Allow, + Err(OmniError::Policy(_)) => ParityDecision::Deny, + Err(other) => panic!("unexpected SDK error for branch_merge: {other:?}"), + } +} + +pub async fn http_merge_decision( + graph: &Path, + policy_path: &PathBuf, + actor: &str, + token: &str, +) -> ParityDecision { + let state = AppState::open_with_bearer_tokens_and_policy( + graph.to_string_lossy().to_string(), + vec![(actor.to_string(), token.to_string())], + Some(policy_path), + ) + .await + .unwrap(); + let app = build_app(state); + let req = BranchMergeRequest { + source: "feature".to_string(), + target: Some("main".to_string()), + }; + let (status, _body) = json_response( + &app, + Request::builder() + .uri("/branches/merge") + .method(Method::POST) + .header(AUTHORIZATION, format!("Bearer {token}")) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&req).unwrap())) + .unwrap(), + ) + .await; + match status { + StatusCode::OK => ParityDecision::Allow, + StatusCode::FORBIDDEN => ParityDecision::Deny, + other => panic!("unexpected HTTP status for branch_merge: {other}"), + } +} + +pub async fn converged_cluster_dir(policies_yaml: &str) -> tempfile::TempDir { + let temp = tempfile::tempdir().unwrap(); + fs::write( + temp.path().join("people.pg"), + "\nnode Person {\n name: String @key\n}\n", + ) + .unwrap(); + fs::write( + temp.path().join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + temp.path().join("cluster.yaml"), + format!( + r#" +version: 1 +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +{policies_yaml}"# + ), + ) + .unwrap(); + let import = omnigraph_cluster::import_config_dir(temp.path()).await; + assert!(import.ok, "{:?}", import.diagnostics); + let apply = omnigraph_cluster::apply_config_dir(temp.path()).await; + assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); + temp +} + +pub async fn cluster_settings(dir: &Path) -> color_eyre::eyre::Result<omnigraph_server::ServerConfig> { + omnigraph_server::load_server_settings(None, Some(&dir.to_path_buf()), None, None, None, true).await +} From 127440d8734ab4582f4f2d64882ccf72381c286e Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 15:08:25 +0300 Subject: [PATCH 113/165] refactor(server): split lib.rs into handlers and settings modules Verbatim moves: route handlers + bearer-auth middleware + per-request authorization + the cluster-prefix OpenAPI rewrite go to handlers.rs; settings resolution (omnigraph.yaml/CLI/env, mode inference, bearer-token sources, runtime-state classification) and its in-source test mod go to settings.rs. lib.rs (1,158 lines) keeps the public types, app/router assembly, and serve(). The ApiDoc derive references handlers::-qualified paths; the one multi-line utoipa attribute the cut orphaned was relocated with its handler. 289 crate tests green, OpenAPI drift check included. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-server/src/handlers.rs | 1666 ++++++++++++++ crates/omnigraph-server/src/lib.rs | 2683 +---------------------- crates/omnigraph-server/src/settings.rs | 988 +++++++++ 3 files changed, 2678 insertions(+), 2659 deletions(-) create mode 100644 crates/omnigraph-server/src/handlers.rs create mode 100644 crates/omnigraph-server/src/settings.rs diff --git a/crates/omnigraph-server/src/handlers.rs b/crates/omnigraph-server/src/handlers.rs new file mode 100644 index 0000000..2ead0e3 --- /dev/null +++ b/crates/omnigraph-server/src/handlers.rs @@ -0,0 +1,1666 @@ +//! HTTP route handlers, the bearer-auth middleware, per-request +//! authorization, and the cluster-prefix OpenAPI rewrite (moved +//! verbatim from lib.rs in the modularization). + +use super::*; + +/// Liveness probe. +/// +/// Returns server status and version. Unauthenticated; safe to call from any +/// caller. Use this to confirm the server is reachable before invoking other +/// endpoints. +#[utoipa::path( + get, + path = "/healthz", + tag = "health", + operation_id = "health", + responses( + (status = 200, description = "Server is healthy", body = HealthOutput), + ), +)] +pub(crate) async fn server_health() -> Json<HealthOutput> { + Json(HealthOutput { + status: "ok".to_string(), + version: SERVER_VERSION.to_string(), + source_version: SERVER_SOURCE_VERSION.map(str::to_string), + }) +} + +#[utoipa::path( + get, + path = "/graphs", + tag = "management", + operation_id = "listGraphs", + responses( + (status = 200, description = "List of registered graphs", body = GraphListResponse), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + (status = 405, description = "Method not allowed (single-graph mode)", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// List every graph currently registered with this server (MR-668). +/// +/// Multi-graph mode only. In single mode, the route returns 405 — there's +/// no registry to enumerate. Cedar-gated by the server-level policy via +/// the `graph_list` action against `Omnigraph::Server::"root"`. +/// +/// Order: alphabetical by `graph_id` (server-sorted so clients see +/// deterministic output across requests). +pub(crate) async fn server_graphs_list( + State(state): State<AppState>, + actor: Option<Extension<ResolvedActor>>, +) -> std::result::Result<Json<GraphListResponse>, ApiError> { + // 405 in single mode — there's no registry to enumerate, and the + // legacy URL surface didn't expose this endpoint. + let registry = match state.routing() { + GraphRouting::Single { .. } => { + return Err(ApiError::method_not_allowed( + "GET /graphs is only available in multi-graph mode", + )); + } + GraphRouting::Multi { registry, .. } => registry, + }; + + // Server-level Cedar gate. `state.server_policy` is loaded from + // `server.policy.file` in `omnigraph.yaml` at startup. When no + // server policy is configured, `authorize_request_server` falls + // through to the MR-723 default-deny semantics (every non-Read + // action denied for an authenticated actor). `GraphList` is not + // `Read`, so without a server policy the request gets 403 — which + // is the right default (don't leak the registry until the operator + // explicitly authorizes it). + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + state.server_policy.as_deref(), + PolicyRequest { + action: PolicyAction::GraphList, + branch: None, + target_branch: None, + }, + )?; + + let mut graphs: Vec<GraphInfo> = registry + .list() + .into_iter() + .map(|handle| GraphInfo { + graph_id: handle.key.graph_id.as_str().to_string(), + uri: handle.uri.clone(), + }) + .collect(); + graphs.sort_by(|a, b| a.graph_id.cmp(&b.graph_id)); + Ok(Json(GraphListResponse { graphs })) +} + +pub(crate) async fn server_openapi(State(state): State<AppState>) -> Json<utoipa::openapi::OpenApi> { + let mut doc = ApiDoc::openapi(); + if !state.requires_bearer_auth() { + strip_security(&mut doc); + } + // MR-668: in multi mode, the protected routes live under + // `/graphs/{graph_id}/...`. Rewrite the doc so the spec matches + // the routes the router actually serves. Public paths (`/healthz`) + // stay flat in both modes. + if matches!(state.routing(), GraphRouting::Multi { .. }) { + nest_paths_under_cluster_prefix(&mut doc); + } + Json(doc) +} + +/// Path prefix used to namespace per-graph routes in multi mode. +/// Kept in sync with the `Router::nest(...)` invocation in `build_app`. +const CLUSTER_PATH_PREFIX: &str = "/graphs/{graph_id}"; + +/// Operation-id prefix applied to every cloned cluster operation. +/// Decision 7 in the implementation plan — keeps operation IDs unique +/// across the spec when both flat and nested variants ever appear in +/// the same generation pass. +const CLUSTER_OPERATION_ID_PREFIX: &str = "cluster_"; + +/// Paths that stay flat in every server mode (public or server-level, +/// no per-graph dependency). Update this list when adding new +/// always-flat endpoints. `/graphs` is the management enumeration — +/// it lives at the root in both single mode (405) and multi mode, and +/// must never be rewritten to `/graphs/{graph_id}/graphs`. +const ALWAYS_FLAT_PATHS: &[&str] = &["/healthz", "/graphs"]; + +/// In multi-mode `server_openapi`, every protected path-item is +/// reattached under the cluster prefix. Operation IDs gain the +/// `cluster_` prefix so SDK generators don't collide if/when both +/// surfaces are merged. Every rewritten operation also declares the +/// required `{graph_id}` path parameter so the served OpenAPI document +/// remains internally valid. +/// +/// Removing the flat protected paths matches the runtime router — +/// in multi mode, requests to `/snapshot` etc. return 404, so the +/// spec must agree. +pub(crate) fn nest_paths_under_cluster_prefix(doc: &mut utoipa::openapi::OpenApi) { + let original = std::mem::take(&mut doc.paths.paths); + let mut rewritten = std::collections::BTreeMap::new(); + for (path, mut item) in original { + if ALWAYS_FLAT_PATHS.contains(&path.as_str()) { + rewritten.insert(path, item); + continue; + } + rename_operation_ids(&mut item, CLUSTER_OPERATION_ID_PREFIX); + add_cluster_graph_id_parameter(&mut item); + let new_path = format!("{CLUSTER_PATH_PREFIX}{path}"); + rewritten.insert(new_path, item); + } + doc.paths.paths = rewritten; +} + +pub(crate) fn add_cluster_graph_id_parameter(item: &mut utoipa::openapi::PathItem) { + for op in path_item_operations_mut(item) { + let parameters = op.parameters.get_or_insert_with(Vec::new); + let has_graph_id = parameters + .iter() + .any(|param| param.name == "graph_id" && param.parameter_in == ParameterIn::Path); + if !has_graph_id { + parameters.insert(0, graph_id_path_parameter()); + } + } +} + +pub(crate) fn graph_id_path_parameter() -> Parameter { + let mut parameter = Parameter::new("graph_id"); + parameter.parameter_in = ParameterIn::Path; + parameter.description = Some("Graph id to route the request to.".to_string()); + parameter.schema = Some(Object::with_type(Type::String).into()); + parameter +} + +/// Prefix every operation_id in this PathItem with `prefix`. +pub(crate) fn rename_operation_ids(item: &mut utoipa::openapi::PathItem, prefix: &str) { + for op in path_item_operations_mut(item) { + if let Some(id) = op.operation_id.as_deref() { + op.operation_id = Some(format!("{prefix}{id}")); + } + } +} + +pub(crate) fn path_item_operations_mut( + item: &mut utoipa::openapi::PathItem, +) -> impl Iterator<Item = &mut utoipa::openapi::path::Operation> { + [ + item.get.as_mut(), + item.post.as_mut(), + item.put.as_mut(), + item.delete.as_mut(), + item.options.as_mut(), + item.head.as_mut(), + item.patch.as_mut(), + item.trace.as_mut(), + ] + .into_iter() + .flatten() +} + +pub(crate) fn strip_security(doc: &mut utoipa::openapi::OpenApi) { + if let Some(components) = doc.components.as_mut() { + components.security_schemes.clear(); + } + for path_item in doc.paths.paths.values_mut() { + for op in [ + path_item.get.as_mut(), + path_item.post.as_mut(), + path_item.put.as_mut(), + path_item.delete.as_mut(), + path_item.options.as_mut(), + path_item.head.as_mut(), + path_item.patch.as_mut(), + path_item.trace.as_mut(), + ] + .into_iter() + .flatten() + { + op.security = None; + } + } +} + +pub(crate) async fn require_bearer_auth( + State(state): State<AppState>, + mut request: Request, + next: Next, +) -> std::result::Result<Response, ApiError> { + if !state.requires_bearer_auth() { + return Ok(next.run(request).await); + } + + let Some(header) = request + .headers() + .get(AUTHORIZATION) + .and_then(|value| value.to_str().ok()) + else { + return Err(ApiError::unauthorized("missing bearer token")); + }; + + let Some(provided_token) = header.strip_prefix("Bearer ") else { + return Err(ApiError::unauthorized("missing bearer token")); + }; + + let Some(actor) = state.authenticate_bearer_token(provided_token) else { + return Err(ApiError::unauthorized("invalid bearer token")); + }; + request.extensions_mut().insert(actor); + + Ok(next.run(request).await) +} + +/// Routing middleware (MR-668). Resolves the active graph for the +/// request and injects `Arc<GraphHandle>` as an extension so handlers can +/// extract it via `Extension<Arc<GraphHandle>>`. +/// +/// **Single mode**: the routing field holds the single handle directly. +/// Routes are flat; every request resolves to that handle, regardless +/// of the URI path. No registry walk, no sentinel key, no +/// programmer-error guard. +/// +/// **Multi mode**: routes are nested under `/graphs/{graph_id}/...`. The +/// middleware extracts `{graph_id}` from the URI path and looks it up in +/// the registry. Returns 404 if the graph is not registered. +/// +/// The middleware fires AFTER `require_bearer_auth`, so the actor is +/// already in the request extensions (or auth was off entirely). +pub(crate) async fn resolve_graph_handle( + State(state): State<AppState>, + mut request: Request, + next: Next, +) -> std::result::Result<Response, ApiError> { + let handle = match &state.routing { + GraphRouting::Single { handle } => Arc::clone(handle), + GraphRouting::Multi { registry, .. } => { + // `Router::nest("/graphs/{graph_id}", inner)` rewrites + // `request.uri().path()` to the inner suffix (e.g. `/snapshot`). + // The pre-rewrite URI is preserved in the `OriginalUri` + // request extension by axum's router; we read from there to + // extract `{graph_id}`. Fall back to the current URI only if + // the extension is missing, which shouldn't happen for + // nested routes but is safe defensive code. + let original_path: String = request + .extensions() + .get::<OriginalUri>() + .map(|OriginalUri(uri)| uri.path().to_string()) + .unwrap_or_else(|| request.uri().path().to_string()); + let graph_id_str = original_path + .strip_prefix("/graphs/") + .and_then(|rest| rest.split('/').next()) + .filter(|s| !s.is_empty()) + .ok_or_else(|| { + ApiError::bad_request( + "cluster route missing /graphs/{graph_id} prefix".to_string(), + ) + })?; + let graph_id = GraphId::try_from(graph_id_str.to_string()) + .map_err(|err| ApiError::bad_request(err.to_string()))?; + let key = GraphKey::cluster(graph_id.clone()); + match registry.get(&key) { + RegistryLookup::Ready(handle) => handle, + RegistryLookup::Gone => { + return Err(ApiError::not_found(format!("graph '{graph_id}' not found"))); + } + } + } + }; + + // Per-request observability. `Span::current().record` would silently + // no-op here because no upstream `#[tracing::instrument(...)]` macro + // declares a `graph_id` field; emit an explicit event instead so the + // routing decision actually lands in logs. + info!(graph_id = %handle.key.graph_id, "graph routed"); + + request.extensions_mut().insert(handle); + Ok(next.run(request).await) +} + +pub(crate) fn log_policy_decision(actor_id: &str, request: &PolicyRequest, decision: &PolicyDecision) { + info!( + actor_id = actor_id, + action = %request.action, + branch = request.branch.as_deref().unwrap_or(""), + target_branch = request.target_branch.as_deref().unwrap_or(""), + allowed = decision.allowed, + matched_rule_id = decision.matched_rule_id.as_deref().unwrap_or(""), + "policy decision" + ); +} + +/// The allow/deny **decision** an authorization check produces, kept +/// separate from the operational failures (`Err`) that can occur while +/// computing it. [`authorize_request`] collapses `Denied` to a 403; a caller +/// that needs to remap a denial without also remapping operational failures +/// (the stored-query invoke handler hides a denial as a 404) matches on this +/// directly, so a real 401 (missing bearer) or 500 (policy-evaluation error) +/// keeps its true status instead of being masked as the denial's response. +pub(crate) enum Authz { + Allowed, + Denied(String), +} + +/// HTTP-layer Cedar policy gate, returning the allow/deny [`Authz`] decision +/// and reserving `Err` for operational failures (401 missing bearer, 500 +/// policy-evaluation error). Two sources of the policy engine: +/// * Per-graph handler — passes `handle.policy.as_deref()` so the +/// graph's Cedar rules govern read/change/branch_*/schema_apply. +/// * Management handler — passes `state.server_policy.as_deref()` so +/// server-level Cedar rules govern `graph_list` (the only shipped +/// server-scoped action; runtime `graph_create` / `graph_delete` +/// are deferred until a managed cluster catalog lands). +/// +/// The MR-731 invariant lives inside this function: actor identity is +/// supplied as a separate argument from the resolved bearer match. The +/// `PolicyRequest` struct itself does not carry identity (the field was +/// dropped from the type), so handlers cannot smuggle it through the +/// request. See `actor_id_resolves_from_bearer_token_ignoring_client_supplied_headers` +/// at `tests/server.rs`. +pub(crate) fn authorize( + actor: Option<&ResolvedActor>, + policy: Option<&PolicyEngine>, + request: PolicyRequest, +) -> std::result::Result<Authz, ApiError> { + let Some(engine) = policy else { + // No PolicyEngine installed. Three runtime states can reach this: + // + // * **Open mode** (`--unauthenticated`): no tokens, no policy. + // Per-graph operations are open by operator opt-in (they + // accepted "trust the network" for graph data). + // * **DefaultDeny mode**: tokens configured but no policy. The + // request went through bearer auth, so `actor` is Some. Only + // per-graph `Read` is permitted; other per-graph actions + // return 403. Closes the "configured auth but forgot the + // policy file" trap from MR-723. + // * Either of the above with a **server-scoped** action + // (`graph_list`, future `graph_create`/`graph_delete`). + // + // Server-scoped actions are always denied here, regardless of + // mode or actor presence. The management surface leaks server + // topology (graph IDs + URIs that may contain S3 bucket paths + // or internal hostnames) — operators who opted into Open mode + // accepted exposure of graph DATA, not exposure of server + // topology. Closing the management surface by default in every + // runtime state means the docstring contract on + // `server_graphs_list` ("don't leak the registry until the + // operator explicitly authorizes it") holds uniformly; the + // operator's only path to enabling it is configuring an + // explicit `server.policy.file` in omnigraph.yaml. + if request.action.resource_kind() == PolicyResourceKind::Server { + return Ok(Authz::Denied( + "server-scoped actions require an explicit `server.policy.file` \ + configured in omnigraph.yaml — the management surface is closed \ + by default in every runtime state, including --unauthenticated, \ + so that server topology is never exposed without operator opt-in." + .to_string(), + )); + } + if actor.is_some() && request.action != PolicyAction::Read { + return Ok(Authz::Denied( + "server runs in default-deny mode (bearer tokens configured but no \ + policy file). Only `read` actions are permitted; configure \ + `policy.file` in omnigraph.yaml to enable other actions." + .to_string(), + )); + } + return Ok(Authz::Allowed); + }; + let Some(actor) = actor else { + return Err(ApiError::unauthorized("missing bearer token")); + }; + // SECURITY INVARIANT (MR-731): actor identity is supplied to the + // policy engine here as a separate argument, sourced from the + // bearer-token match resolved by `require_bearer_auth`. The + // `PolicyRequest` struct itself no longer carries `actor_id` (it + // was dropped from the type), so handlers cannot smuggle identity + // through the request body and there is no overwrite step that + // could be skipped. The principle is codified in + // `docs/dev/invariants.md` Hard Invariant 11 ("clients cannot set + // actor identity directly") and pinned by the regression test + // `actor_id_resolves_from_bearer_token_ignoring_client_supplied_headers` + // in `crates/omnigraph-server/tests/server.rs`. + let actor_id = actor.actor_id.as_ref(); + let decision = engine + .authorize(actor_id, &request) + .map_err(|err| ApiError::internal(format!("policy: {err}")))?; + log_policy_decision(actor_id, &request, &decision); + if decision.allowed { + Ok(Authz::Allowed) + } else { + Ok(Authz::Denied(decision.message)) + } +} + +/// Thin wrapper over [`authorize`] for the handlers that treat any denial as a +/// 403: a denial becomes `ApiError::forbidden`, and operational failures +/// (401 missing bearer, 500 policy-evaluation error) propagate unchanged. The +/// stored-query invoke handler does **not** use this — it consumes the +/// [`Authz`] decision directly to hide a denial as a 404 while letting an +/// operational failure keep its true status. +pub(crate) fn authorize_request( + actor: Option<&ResolvedActor>, + policy: Option<&PolicyEngine>, + request: PolicyRequest, +) -> std::result::Result<(), ApiError> { + match authorize(actor, policy, request)? { + Authz::Allowed => Ok(()), + Authz::Denied(message) => Err(ApiError::forbidden(message)), + } +} + +#[utoipa::path( + get, + path = "/snapshot", + tag = "snapshots", + operation_id = "getSnapshot", + params(SnapshotQuery), + responses( + (status = 200, description = "Database snapshot", body = api::SnapshotOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Read the current snapshot of a branch. +/// +/// Returns the manifest version plus per-table metadata (path, version, row +/// count) for every table on the branch. Defaults to `main` when `branch` is +/// omitted. Read-only. +pub(crate) async fn server_snapshot( + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Query(query): Query<SnapshotQuery>, +) -> std::result::Result<Json<api::SnapshotOutput>, ApiError> { + let branch = query.branch.unwrap_or_else(|| "main".to_string()); + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::Read, + branch: Some(branch.clone()), + target_branch: None, + }, + )?; + let snapshot = { + let db = &handle.engine; + db.snapshot_of(ReadTarget::branch(branch.as_str())) + .await + .map_err(ApiError::from_omni)? + }; + Ok(Json(snapshot_payload(&branch, &snapshot))) +} + +/// Header values that flag a response as coming from a deprecated route +/// (RFC 9745 / RFC 8288) and point at the canonical successor. +pub(crate) fn deprecation_headers(successor_link: &'static str) -> [(HeaderName, HeaderValue); 2] { + [ + ( + HeaderName::from_static("deprecation"), + HeaderValue::from_static("true"), + ), + ( + HeaderName::from_static("link"), + HeaderValue::from_static(successor_link), + ), + ] +} + +#[utoipa::path( + post, + path = "/read", + tag = "queries", + operation_id = "read", + request_body = ReadRequest, + responses( + (status = 200, description = "Query results (response includes `Deprecation: true` + `Link: </query>; rel=\"successor-version\"`)", body = ReadOutput), + (status = 400, description = "Bad request", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +#[deprecated(note = "use POST /query instead; /read is kept indefinitely for byte-stable back-compat")] +/// **Deprecated** — use [`POST /query`](#tag/queries/operation/query) instead. +/// +/// Execute a GQ read query. Behavior is unchanged from prior releases; the +/// route is kept indefinitely for byte-stable back-compat. New integrations +/// should target `POST /query`, which has clean field names (`query` / +/// `name`) and a 400-on-mutation guard. Responses from this route include +/// `Deprecation: true` and `Link: </query>; rel="successor-version"` +/// headers per RFC 9745 / RFC 8288 so SDKs and proxies can surface the +/// signal. +pub(crate) async fn server_read( + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Json(request): Json<ReadRequest>, +) -> std::result::Result<([(HeaderName, HeaderValue); 2], Json<ReadOutput>), ApiError> { + let (selected_name, target, result) = run_query( + handle, + actor.as_ref().map(|Extension(actor)| actor), + &request.query_source, + request.query_name.as_deref(), + request.params.as_ref(), + request.branch, + request.snapshot, + false, // /read predates the D2 rule; legacy callers may submit mutating queries here + ) + .await?; + Ok(( + deprecation_headers("</query>; rel=\"successor-version\""), + Json(api::read_output(selected_name, &target, result)), + )) +} + +#[utoipa::path( + post, + path = "/query", + tag = "queries", + operation_id = "query", + request_body = QueryRequest, + responses( + (status = 200, description = "Query results", body = ReadOutput), + (status = 400, description = "Bad request - also returned when the query body contains mutations; use POST /mutate (or its deprecated alias POST /change) for write queries", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Execute an inline read query (friendlier-named alternative to `POST /read`). +/// +/// Designed for ad-hoc exploration and AI-agent tool-use: short field +/// names (`query`, `name`) match the CLI `-e` flag and the GQ `query` +/// keyword. Mutations (`insert`/`update`/`delete`) are rejected with 400 +/// -- use `POST /mutate` (or its deprecated alias `POST /change`) for +/// write queries. Otherwise behaves identically to `POST /read`: same +/// target semantics (branch xor snapshot), same Cedar action (Read), +/// same response shape. +pub(crate) async fn server_query( + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Json(request): Json<QueryRequest>, +) -> std::result::Result<Json<ReadOutput>, ApiError> { + let (selected_name, target, result) = run_query( + handle, + actor.as_ref().map(|Extension(actor)| actor), + &request.query, + request.name.as_deref(), + request.params.as_ref(), + request.branch, + request.snapshot, + true, // /query is read-only; reject mutations + ) + .await?; + Ok(Json(api::read_output(selected_name, &target, result))) +} + +#[utoipa::path( + post, + path = "/export", + tag = "queries", + operation_id = "export", + request_body = ExportRequest, + responses( + (status = 200, description = "Exported data as NDJSON", content_type = "application/x-ndjson"), + (status = 400, description = "Bad request", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Stream the contents of a branch as NDJSON. +/// +/// Emits one JSON object per line (`application/x-ndjson`). Filter with +/// `type_names` (node/edge type names) and/or `table_keys`; both empty +/// streams the entire branch. Suitable for large exports — the response is +/// streamed, not buffered. Read-only. +pub(crate) async fn server_export( + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Json(request): Json<ExportRequest>, +) -> std::result::Result<Response, ApiError> { + let branch = request.branch.unwrap_or_else(|| "main".to_string()); + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::Export, + branch: Some(branch.clone()), + target_branch: None, + }, + )?; + let engine = Arc::clone(&handle.engine); + let type_names = request.type_names.clone(); + let table_keys = request.table_keys.clone(); + let (tx, rx) = mpsc::unbounded_channel::<std::result::Result<Bytes, io::Error>>(); + tokio::spawn(async move { + let result = { + let mut writer = ExportStreamWriter { sender: tx.clone() }; + engine + .export_jsonl_to_writer(&branch, &type_names, &table_keys, &mut writer) + .await + }; + if let Err(err) = result { + let _ = tx.send(Err(io::Error::other(err.to_string()))); + } + }); + let body = Body::from_stream(stream::unfold(rx, |mut rx| async move { + rx.recv().await.map(|item| (item, rx)) + })); + Ok(( + StatusCode::OK, + [(CONTENT_TYPE, "application/x-ndjson; charset=utf-8")], + body, + ) + .into_response()) +} + +/// Shared implementation behind `POST /mutate` (canonical) and +/// `POST /change` (deprecated alias). Returns the bare `ChangeOutput`; +/// each route handler wraps it (the alias also attaches Deprecation +/// headers). +/// Shared backend for `/mutate` (canonical) and `/change` (deprecated alias). +/// +/// Decoupled from `ChangeRequest` so MR-969's `/queries/{name}` stored-query +/// handler can call this directly with registry-supplied fields without +/// rebuilding the request body. Today's HTTP handlers unpack the request and +/// call here; the registry would do the same. +pub(crate) async fn run_mutate( + state: AppState, + handle: Arc<GraphHandle>, + actor: Option<&ResolvedActor>, + query: &str, + name: Option<&str>, + params_json: Option<&Value>, + branch: String, +) -> std::result::Result<ChangeOutput, ApiError> { + let actor_arc = actor + .map(|a| Arc::clone(&a.actor_id)) + .unwrap_or_else(|| Arc::<str>::from("anonymous")); + let actor_id = actor.map(|a| a.actor_id.as_ref()); + authorize_request( + actor, + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::Change, + branch: Some(branch.clone()), + target_branch: None, + }, + )?; + // Per-actor admission: bound concurrent in-flight mutations and + // estimated bytes per actor. Cedar runs FIRST so denied requests + // don't consume admission slots. Estimate uses the request body + // size as a coarse proxy; engine memory pressure can run higher. + let est_bytes = query.len() as u64 + + params_json + .map(|p| p.to_string().len() as u64) + .unwrap_or(0); + let _admission = state + .workload + .try_admit(&actor_arc, est_bytes) + .map_err(ApiError::from_workload_reject)?; + let (selected_name, query_params) = + select_named_query(query, name).map_err(|err| ApiError::bad_request(err.to_string()))?; + let params = query_params_from_json(&query_params, params_json) + .map_err(|err| ApiError::bad_request(err.to_string()))?; + + let result = { + let db = &handle.engine; + db.mutate_as(&branch, query, &selected_name, ¶ms, actor_id) + .await + .map_err(ApiError::from_omni)? + }; + Ok(ChangeOutput { + branch, + query_name: selected_name, + affected_nodes: result.affected_nodes, + affected_edges: result.affected_edges, + actor_id: actor_id.map(str::to_string), + }) +} + +/// Shared backend for `/query` (canonical) and `/read` (deprecated alias). +/// +/// Mirrors [`run_mutate`]'s decoupled shape so MR-969's stored-query handler +/// can call here with registry-supplied fields. Rejects inline source that +/// contains mutations (D2 rule); callers wanting writes go through +/// [`run_mutate`] instead. +/// +/// Intentionally does **not** take [`AppState`] (unlike [`run_mutate`]): +/// reads are not admission-gated today, so there is no `state.workload` +/// consumer. The signature grows the parameter when Phase 1 (MR-976) adds +/// the request envelope's `expect: { max_rows_scanned: N }` budget, or +/// MR-969 extends per-actor admission to stored-read invocations. +pub(crate) async fn run_query( + handle: Arc<GraphHandle>, + actor: Option<&ResolvedActor>, + query: &str, + name: Option<&str>, + params_json: Option<&Value>, + branch: Option<String>, + snapshot: Option<String>, + reject_mutations: bool, +) -> std::result::Result<(String, ReadTarget, omnigraph_compiler::result::QueryResult), ApiError> { + if branch.is_some() && snapshot.is_some() { + return Err(ApiError::bad_request( + "request may specify branch or snapshot, not both", + )); + } + + let target = read_target_from_request(branch, snapshot); + let policy_branch = match &target { + ReadTarget::Branch(branch) => Some(branch.clone()), + ReadTarget::Snapshot(_) if handle.policy.is_some() && actor.is_some() => { + let db = &handle.engine; + db.resolved_branch_of(target.clone()) + .await + .map(|branch| branch.or_else(|| Some("main".to_string()))) + .map_err(ApiError::from_omni)? + } + ReadTarget::Snapshot(_) => None, + }; + authorize_request( + actor, + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::Read, + branch: policy_branch, + target_branch: None, + }, + )?; + let query_decl = + select_named_query_decl(query, name).map_err(|err| ApiError::bad_request(err.to_string()))?; + if reject_mutations && !query_decl.mutations.is_empty() { + return Err(ApiError::bad_request(format!( + "query '{}' contains mutations (insert/update/delete); use POST /mutate for write queries", + query_decl.name + ))); + } + let selected_name = query_decl.name.clone(); + let params = query_params_from_json(&query_decl.params, params_json) + .map_err(|err| ApiError::bad_request(err.to_string()))?; + + let result = { + let db = &handle.engine; + db.query(target.clone(), query, &selected_name, ¶ms) + .await + .map_err(ApiError::from_omni)? + }; + Ok((selected_name, target, result)) +} + +#[utoipa::path( + post, + path = "/change", + tag = "mutations", + operation_id = "change", + request_body = ChangeRequest, + responses( + (status = 200, description = "Mutation results (response includes `Deprecation: true` + `Link: </mutate>; rel=\"successor-version\"`)", body = ChangeOutput), + (status = 400, description = "Bad request", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + (status = 409, description = "Merge conflict", body = ErrorOutput), + (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +#[deprecated(note = "use POST /mutate instead; /change is kept indefinitely for back-compat")] +/// **Deprecated** — use [`POST /mutate`](#tag/mutations/operation/mutate) instead. +/// +/// Apply a GQ mutation to a branch. Behavior is unchanged; the route is +/// kept indefinitely for back-compat. New integrations should target +/// `POST /mutate`, which has identical semantics and a name that pairs +/// cleanly with `POST /query`. Responses from this route include +/// `Deprecation: true` and `Link: </mutate>; rel="successor-version"` +/// headers per RFC 9745 / RFC 8288 so SDKs and proxies can surface the +/// signal. +pub(crate) async fn server_change( + State(state): State<AppState>, + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Json(request): Json<ChangeRequest>, +) -> std::result::Result<([(HeaderName, HeaderValue); 2], Json<ChangeOutput>), ApiError> { + let branch = request.branch.unwrap_or_else(|| "main".to_string()); + let output = run_mutate( + state, + handle, + actor.as_ref().map(|Extension(actor)| actor), + &request.query, + request.name.as_deref(), + request.params.as_ref(), + branch, + ) + .await?; + Ok(( + deprecation_headers("</mutate>; rel=\"successor-version\""), + Json(output), + )) +} + +#[utoipa::path( + post, + path = "/mutate", + tag = "mutations", + operation_id = "mutate", + request_body = ChangeRequest, + responses( + (status = 200, description = "Mutation results", body = ChangeOutput), + (status = 400, description = "Bad request", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + (status = 409, description = "Merge conflict", body = ErrorOutput), + (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Apply a GQ mutation to a branch (canonical mutation endpoint). +/// +/// Writes to the named `branch` (defaults to `main`). Mutations are atomic +/// per call and produce a new commit. Returns counts of nodes and edges +/// affected. **Destructive**: on success the branch is updated; rejected +/// mutations may still acquire locks briefly. Returns 409 on merge conflict. +/// +/// Pairs with `POST /query` (read-only). The legacy `POST /change` route +/// has identical semantics and is kept as a deprecated alias. +pub(crate) async fn server_mutate( + State(state): State<AppState>, + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Json(request): Json<ChangeRequest>, +) -> std::result::Result<Json<ChangeOutput>, ApiError> { + let branch = request.branch.unwrap_or_else(|| "main".to_string()); + Ok(Json( + run_mutate( + state, + handle, + actor.as_ref().map(|Extension(actor)| actor), + &request.query, + request.name.as_deref(), + request.params.as_ref(), + branch, + ) + .await?, + )) +} + +/// Path parameter for `POST /queries/{name}`. +#[derive(Deserialize)] +pub(crate) struct QueryNamePath { + name: String, +} + +pub(crate) fn parse_optional_invoke_body( + body: Bytes, +) -> std::result::Result<InvokeStoredQueryRequest, ApiError> { + if body.is_empty() { + return Ok(InvokeStoredQueryRequest::default()); + } + serde_json::from_slice::<Option<InvokeStoredQueryRequest>>(&body) + .map(|request| request.unwrap_or_default()) + .map_err(|err| { + ApiError::bad_request(format!("invalid stored-query invocation body: {err}")) + }) +} + +#[utoipa::path( + post, + path = "/queries/{name}", + tag = "queries", + operation_id = "invoke_query", + params(("name" = String, Path, description = "Stored query name (the registry key)")), + request_body = Option<InvokeStoredQueryRequest>, + responses( + (status = 200, description = "Read envelope (ReadOutput) or mutation envelope (ChangeOutput), serialized untagged", body = InvokeStoredQueryResponse), + (status = 400, description = "Bad request (param type error; snapshot on a stored mutation)", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden (the inner `change` gate for a stored mutation)", body = ErrorOutput), + (status = 404, description = "Unknown stored query, or `invoke_query` denied — indistinguishable to a caller without the grant", body = ErrorOutput), + (status = 409, description = "Merge conflict", body = ErrorOutput), + (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), + (status = 500, description = "Policy evaluation error (a denial is reported as 404, not 500)", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Invoke a curated, server-side stored query by name. +/// +/// The query source comes from the graph's `queries:` registry, not the +/// request body — callers send only runtime inputs (`params`, `branch`, +/// `snapshot`). Gated by the `invoke_query` Cedar action at the boundary; +/// a stored *mutation* additionally passes the engine's `change` gate +/// (double-gated). An actor **without** `invoke_query` cannot tell a denied +/// query from a missing one — both return the same 404, so the catalog +/// can't be probed without the grant. Once `invoke_query` is held, the +/// inner `read`/`change` gate may surface a 403 for an existing query the +/// actor can't run (the intended double-gate signal). +pub(crate) async fn server_invoke_query( + State(state): State<AppState>, + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Path(QueryNamePath { name }): Path<QueryNamePath>, + body: Bytes, +) -> std::result::Result<Json<InvokeStoredQueryResponse>, ApiError> { + let req = parse_optional_invoke_body(body)?; + // A caller without `invoke_query` can't tell a denial from a missing + // query: both 404 with this exact message, so the catalog can't be + // probed without the grant. (A caller that holds invoke_query may still + // see the inner gate's 403 for an existing query it can't run — intended.) + const NOT_FOUND: &str = "stored query not found"; + let actor_ref = actor.as_ref().map(|Extension(actor)| actor); + + // Boundary gate (authentication already ran in `require_bearer_auth`). + // A denial is hidden as 404 (deny == missing, so the catalog can't be + // probed without the grant), but operational failures (401 missing bearer, + // 500 policy-evaluation error) propagate with their true status via `?` + // rather than being masked as a missing query. + match authorize( + actor_ref, + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::InvokeQuery, + // Graph-scoped: no branch dimension. The per-branch/snapshot + // access is enforced by the inner read/change gate in the + // runner, so the outer gate must not resolve a branch (doing so + // was wrong for snapshot reads). + branch: None, + target_branch: None, + }, + )? { + Authz::Allowed => {} + Authz::Denied(_) => return Err(ApiError::not_found(NOT_FOUND)), + } + + // Resolve against the per-graph registry (same 404 on a miss). + let stored = handle + .queries + .as_ref() + .and_then(|registry| registry.lookup(&name)) + .ok_or_else(|| ApiError::not_found(NOT_FOUND))?; + + // Detach what we need before `handle` moves into the runner — the + // registry borrow lives inside `handle`. + let source = Arc::clone(&stored.source); + let query_name = stored.name.clone(); + let is_mutation = stored.is_mutation(); + + info!( + graph = %handle.uri, + actor = ?actor_ref.map(|a| a.actor_id.as_ref()), + query = %query_name, + kind = if is_mutation { "mutate" } else { "read" }, + "stored query invoked" + ); + + if is_mutation { + if req.snapshot.is_some() { + return Err(ApiError::bad_request( + "stored mutation cannot target a snapshot", + )); + } + let branch = req.branch.unwrap_or_else(|| "main".to_string()); + let output = run_mutate( + state, + handle, + actor_ref, + &source, + Some(&query_name), + req.params.as_ref(), + branch, + ) + .await?; + Ok(Json(InvokeStoredQueryResponse::Change(output))) + } else { + let (selected, target, result) = run_query( + handle, + actor_ref, + &source, + Some(&query_name), + req.params.as_ref(), + req.branch, + req.snapshot, + true, + ) + .await?; + Ok(Json(InvokeStoredQueryResponse::Read(api::read_output( + selected, &target, result, + )))) + } +} + +#[utoipa::path( + get, + path = "/queries", + tag = "queries", + operation_id = "list_queries", + responses( + (status = 200, description = "Stored-query catalog (the mcp.expose subset, with typed params)", body = QueriesCatalogOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// List the graph's exposed stored queries as a typed tool catalog. +/// +/// Returns the `mcp.expose == true` subset of the `queries:` registry, each +/// with its MCP tool name, read/mutate flag, description/instruction, and +/// typed parameters — enough for a client to register them as tools without +/// fetching `.gq` source. Read-gated; the catalog is graph-wide (branch +/// independent — `read` is authorized against `main`). **Not** Cedar-filtered +/// per query yet, so it can list a query whose `invoke_query` the caller +/// lacks (a known gap until per-query authorization lands). +pub(crate) async fn server_list_queries( + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, +) -> std::result::Result<Json<QueriesCatalogOutput>, ApiError> { + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::Read, + branch: Some("main".to_string()), + target_branch: None, + }, + )?; + let queries = match handle.queries.as_ref() { + Some(registry) => registry + .iter() + .filter(|q| q.expose) + .map(api::query_catalog_entry) + .collect(), + None => Vec::new(), + }; + Ok(Json(QueriesCatalogOutput { queries })) +} + +#[utoipa::path( + get, + path = "/schema", + tag = "schema", + operation_id = "getSchema", + responses( + (status = 200, description = "Current schema source", body = SchemaOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Read the current schema source. +/// +/// Returns the project's schema as a single string in `.pg` source form. +/// Useful for clients that want to introspect available types and tables +/// before constructing GQ queries. Read-only. +pub(crate) async fn server_schema_get( + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, +) -> std::result::Result<Json<SchemaOutput>, ApiError> { + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::Read, + branch: None, + target_branch: None, + }, + )?; + let schema_source = { + let db = &handle.engine; + db.schema_source().to_string() + }; + Ok(Json(SchemaOutput { schema_source })) +} + +#[utoipa::path( + post, + path = "/schema/apply", + tag = "mutations", + operation_id = "applySchema", + request_body = SchemaApplyRequest, + responses( + (status = 200, description = "Schema apply results", body = SchemaApplyOutput), + (status = 400, description = "Bad request", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Apply a schema migration. +/// +/// Diffs `schema_source` against the current schema and applies the resulting +/// migration steps (add/drop type, add/drop column, etc.). **Destructive**: +/// some steps drop data. Returns the list of steps applied; if `applied` is +/// false the diff was unsupported and no changes were made. +pub(crate) async fn server_schema_apply( + State(state): State<AppState>, + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Json(request): Json<SchemaApplyRequest>, +) -> std::result::Result<Json<SchemaApplyOutput>, ApiError> { + let actor_arc = actor + .as_ref() + .map(|Extension(actor)| Arc::clone(&actor.actor_id)) + .unwrap_or_else(|| Arc::<str>::from("anonymous")); + let actor_id = actor + .as_ref() + .map(|Extension(actor)| actor.actor_id.as_ref()); + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::SchemaApply, + branch: None, + target_branch: Some("main".to_string()), + }, + )?; + let est_bytes = request.schema_source.len() as u64; + let _admission = state + .workload + .try_admit(&actor_arc, est_bytes) + .map_err(ApiError::from_workload_reject)?; + let result = { + let db = &handle.engine; + let registry = handle.queries.as_deref(); + let label = handle.key.graph_id.as_str().to_string(); + // Engine-layer policy enforcement (MR-722): pass the resolved + // actor through so apply_schema_as can call enforce() with the + // authoritative identity. With a policy installed in AppState, + // engine-side enforcement re-checks the same decision the + // HTTP-layer authorize_request just made above. PR #3 collapses + // the redundancy. + db.apply_schema_as_with_catalog_check( + &request.schema_source, + omnigraph::db::SchemaApplyOptions { + allow_data_loss: request.allow_data_loss, + }, + actor_id, + |catalog| { + if let Some(registry) = registry { + validate_registry_against_catalog(registry, catalog, &label)?; + } + Ok(()) + }, + ) + .await + .map_err(ApiError::from_omni)? + }; + Ok(Json(schema_apply_output(handle.uri.as_str(), result))) +} + +#[utoipa::path( + post, + path = "/ingest", + tag = "mutations", + operation_id = "ingest", + request_body = IngestRequest, + responses( + (status = 200, description = "Ingest results", body = IngestOutput), + (status = 400, description = "Bad request", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Bulk-load NDJSON data into a branch. +/// +/// `data` is NDJSON with one record per line. `mode` controls behavior on +/// existing rows: `merge` upserts by id (default), `append` blindly inserts, +/// `overwrite` replaces table contents. Branch creation is opt-in by +/// presence of `from`: with `from` set, a missing `branch` is created from +/// it; without `from`, `branch` must already exist — a missing branch is a +/// 404, never an implicit fork. **Destructive** when `mode` is `overwrite` +/// or when the load produces conflicting writes. +pub(crate) async fn server_ingest( + State(state): State<AppState>, + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Json(request): Json<IngestRequest>, +) -> std::result::Result<Json<IngestOutput>, ApiError> { + let branch = request.branch.unwrap_or_else(|| "main".to_string()); + let from = request.from; + let mode = request.mode.unwrap_or(omnigraph::loader::LoadMode::Merge); + let actor_arc = actor + .as_ref() + .map(|Extension(actor)| Arc::clone(&actor.actor_id)) + .unwrap_or_else(|| Arc::<str>::from("anonymous")); + let actor_id = actor + .as_ref() + .map(|Extension(actor)| actor.actor_id.as_ref()); + + let branch_exists = { + let db = &handle.engine; + db.branch_list() + .await + .map_err(ApiError::from_omni)? + .into_iter() + .any(|name| name == branch) + }; + + if !branch_exists { + match from.as_deref() { + // Fork-if-missing is opt-in by presence of `from`; without it a + // typo'd branch name must surface as an error, not silently + // create a fork and land the data there. + None => { + return Err(ApiError::not_found(format!( + "branch '{branch}' not found; pass `from` to create it" + ))); + } + Some(from) => authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::BranchCreate, + branch: Some(from.to_string()), + target_branch: Some(branch.clone()), + }, + )?, + } + } + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::Change, + branch: Some(branch.clone()), + target_branch: None, + }, + )?; + let est_bytes = request.data.len() as u64; + let _admission = state + .workload + .try_admit(&actor_arc, est_bytes) + .map_err(ApiError::from_workload_reject)?; + + let result = { + let db = &handle.engine; + db.load_as(&branch, from.as_deref(), &request.data, mode, actor_id) + .await + .map_err(ApiError::from_omni)? + }; + + Ok(Json(ingest_output( + handle.uri.as_str(), + &result, + mode, + actor_id.map(str::to_string), + ))) +} + +#[utoipa::path( + get, + path = "/branches", + tag = "branches", + operation_id = "listBranches", + responses( + (status = 200, description = "List of branches", body = BranchListOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// List all branches. +/// +/// Returns branch names sorted alphabetically. Read-only. +pub(crate) async fn server_branch_list( + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, +) -> std::result::Result<Json<BranchListOutput>, ApiError> { + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::Read, + branch: None, + target_branch: None, + }, + )?; + let mut branches = { + let db = &handle.engine; + db.branch_list().await.map_err(ApiError::from_omni)? + }; + branches.sort(); + Ok(Json(BranchListOutput { branches })) +} + +#[utoipa::path( + post, + path = "/branches", + tag = "branches", + operation_id = "createBranch", + request_body = BranchCreateRequest, + responses( + (status = 200, description = "Branch created", body = BranchCreateOutput), + (status = 400, description = "Bad request", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + (status = 409, description = "Branch already exists", body = ErrorOutput), + (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Create a new branch. +/// +/// Forks `name` off of `from` (defaults to `main`). The new branch shares +/// table data with its parent until it is mutated. Returns 409 if `name` +/// already exists. +pub(crate) async fn server_branch_create( + State(state): State<AppState>, + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Json(request): Json<BranchCreateRequest>, +) -> std::result::Result<Json<BranchCreateOutput>, ApiError> { + let from = request.from.unwrap_or_else(|| "main".to_string()); + let actor_arc = actor + .as_ref() + .map(|Extension(actor)| Arc::clone(&actor.actor_id)) + .unwrap_or_else(|| Arc::<str>::from("anonymous")); + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::BranchCreate, + branch: Some(from.clone()), + target_branch: Some(request.name.clone()), + }, + )?; + // Branch metadata only — small constant bytes estimate. The Lance + // shallow-clone work is bounded by the parent's manifest size, not + // the request body. + let _admission = state + .workload + .try_admit(&actor_arc, 256) + .map_err(ApiError::from_workload_reject)?; + { + let db = &handle.engine; + db.branch_create_from_as( + ReadTarget::branch(&from), + &request.name, + actor.as_ref().map(|Extension(a)| a.actor_id.as_ref()), + ) + .await + .map_err(ApiError::from_omni)?; + } + Ok(Json(BranchCreateOutput { + uri: handle.uri.clone(), + from, + name: request.name, + actor_id: actor.map(|Extension(actor)| actor.actor_id.as_ref().to_string()), + })) +} + +/// Path-param shape for [`server_branch_delete`]. Named-field +/// deserialization (rather than `Path<String>` or `Path<(String,)>`) +/// keeps the extractor stable across single-mode flat routes and +/// multi-mode nested routes: the `{branch}` capture is picked by +/// name and any other captures in scope (e.g. `{graph_id}` in +/// multi-mode) are ignored without breaking deserialization. +/// +/// Closes the "handler path-extractor type is positional and breaks +/// when route nesting changes" class. +#[derive(Deserialize)] +pub(crate) struct BranchPath { + branch: String, +} + +#[utoipa::path( + delete, + path = "/branches/{branch}", + tag = "branches", + operation_id = "deleteBranch", + params( + ("branch" = String, Path, description = "Branch name to delete"), + ), + responses( + (status = 200, description = "Branch deleted", body = BranchDeleteOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + (status = 404, description = "Branch not found", body = ErrorOutput), + (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Delete a branch. +/// +/// **Irreversible.** Removes the branch pointer; commits remain reachable +/// only if referenced by another branch. Returns 404 if the branch does not +/// exist. +pub(crate) async fn server_branch_delete( + State(state): State<AppState>, + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Path(BranchPath { branch }): Path<BranchPath>, +) -> std::result::Result<Json<BranchDeleteOutput>, ApiError> { + let actor_arc = actor + .as_ref() + .map(|Extension(actor)| Arc::clone(&actor.actor_id)) + .unwrap_or_else(|| Arc::<str>::from("anonymous")); + let actor_id = actor + .as_ref() + .map(|Extension(actor)| actor.actor_id.as_ref()); + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::BranchDelete, + branch: None, + target_branch: Some(branch.clone()), + }, + )?; + // Metadata-only manifest tombstone — small constant estimate. + let _admission = state + .workload + .try_admit(&actor_arc, 256) + .map_err(ApiError::from_workload_reject)?; + { + let db = &handle.engine; + db.branch_delete_as(&branch, actor_id) + .await + .map_err(ApiError::from_omni)?; + } + Ok(Json(BranchDeleteOutput { + uri: handle.uri.clone(), + name: branch, + actor_id: actor_id.map(str::to_string), + })) +} + +#[utoipa::path( + post, + path = "/branches/merge", + tag = "branches", + operation_id = "mergeBranches", + request_body = BranchMergeRequest, + responses( + (status = 200, description = "Branches merged", body = BranchMergeOutput), + (status = 400, description = "Bad request", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + (status = 409, description = "Merge conflict", body = ErrorOutput), + (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Merge one branch into another. +/// +/// Merges `source` into `target` (defaults to `main`). Outcome is one of +/// `already_up_to_date`, `fast_forward`, or `merged`. Returns 409 with the +/// list of conflicts if the merge cannot be completed; the target is left +/// unchanged in that case. **Destructive** to `target` on success. +pub(crate) async fn server_branch_merge( + State(state): State<AppState>, + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Json(request): Json<BranchMergeRequest>, +) -> std::result::Result<Json<BranchMergeOutput>, ApiError> { + let target = request.target.unwrap_or_else(|| "main".to_string()); + let actor_arc = actor + .as_ref() + .map(|Extension(actor)| Arc::clone(&actor.actor_id)) + .unwrap_or_else(|| Arc::<str>::from("anonymous")); + let actor_id = actor + .as_ref() + .map(|Extension(actor)| actor.actor_id.as_ref()); + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::BranchMerge, + branch: Some(request.source.clone()), + target_branch: Some(target.clone()), + }, + )?; + // Merge body is small JSON; the heavy work is in the engine but is + // bounded per-(table, branch) by the writer queue. Small constant + // estimate suffices for the actor in-flight count. + let _admission = state + .workload + .try_admit(&actor_arc, 256) + .map_err(ApiError::from_workload_reject)?; + let outcome = { + let db = &handle.engine; + db.branch_merge_as(&request.source, &target, actor_id) + .await + .map_err(ApiError::from_omni)? + }; + Ok(Json(BranchMergeOutput { + source: request.source, + target, + outcome: outcome.into(), + actor_id: actor_id.map(str::to_string), + })) +} + +#[utoipa::path( + get, + path = "/commits", + tag = "commits", + operation_id = "listCommits", + params(CommitListQuery), + responses( + (status = 200, description = "List of commits", body = CommitListOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// List commits. +/// +/// Filter by `branch` to get the commits on a single branch (most recent +/// first); omit to list across all branches. Read-only. +pub(crate) async fn server_commit_list( + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Query(query): Query<CommitListQuery>, +) -> std::result::Result<Json<CommitListOutput>, ApiError> { + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::Read, + branch: query.branch.clone(), + target_branch: None, + }, + )?; + let commits = { + let db = &handle.engine; + db.list_commits(query.branch.as_deref()) + .await + .map_err(ApiError::from_omni)? + }; + Ok(Json(CommitListOutput { + commits: commits.iter().map(api::commit_output).collect(), + })) +} + +/// Path-param shape for [`server_commit_show`]. See [`BranchPath`] +/// for the design rationale — same pattern, different field name. +#[derive(Deserialize)] +pub(crate) struct CommitPath { + commit_id: String, +} + +#[utoipa::path( + get, + path = "/commits/{commit_id}", + tag = "commits", + operation_id = "getCommit", + params( + ("commit_id" = String, Path, description = "Commit identifier"), + ), + responses( + (status = 200, description = "Commit details", body = api::CommitOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + (status = 404, description = "Commit not found", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] + +/// Get a single commit. +/// +/// Returns the commit's manifest version, parent commit(s), and creation +/// metadata. Read-only. +pub(crate) async fn server_commit_show( + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Path(CommitPath { commit_id }): Path<CommitPath>, +) -> std::result::Result<Json<api::CommitOutput>, ApiError> { + authorize_request( + actor.as_ref().map(|Extension(actor)| actor), + handle.policy.as_deref(), + PolicyRequest { + action: PolicyAction::Read, + branch: None, + target_branch: None, + }, + )?; + let commit = { + let db = &handle.engine; + db.get_commit(&commit_id) + .await + .map_err(ApiError::from_omni)? + }; + Ok(Json(api::commit_output(&commit))) +} + +pub(crate) fn read_target_from_request(branch: Option<String>, snapshot: Option<String>) -> ReadTarget { + if let Some(snapshot) = snapshot { + ReadTarget::snapshot(omnigraph::db::SnapshotId::new(snapshot)) + } else { + ReadTarget::branch(branch.unwrap_or_else(|| "main".to_string())) + } +} + +pub(crate) fn select_named_query_decl( + query_source: &str, + requested_name: Option<&str>, +) -> Result<omnigraph_compiler::query::ast::QueryDecl> { + let parsed = parse_query(query_source)?; + let query = if let Some(name) = requested_name { + parsed + .queries + .into_iter() + .find(|query| query.name == name) + .ok_or_else(|| color_eyre::eyre::eyre!("query '{}' not found", name))? + } else if parsed.queries.len() == 1 { + parsed.queries.into_iter().next().unwrap() + } else { + bail!("query file contains multiple queries; pass --name"); + }; + Ok(query) +} + +pub(crate) fn select_named_query( + query_source: &str, + requested_name: Option<&str>, +) -> Result<(String, Vec<omnigraph_compiler::query::ast::Param>)> { + let query = select_named_query_decl(query_source, requested_name)?; + Ok((query.name, query.params)) +} + +pub(crate) fn query_params_from_json( + query_params: &[omnigraph_compiler::query::ast::Param], + params_json: Option<&Value>, +) -> Result<ParamMap> { + json_params_to_param_map(params_json, query_params, JsonParamMode::Standard) + .map_err(|err| color_eyre::eyre::eyre!(err.to_string())) +} + diff --git a/crates/omnigraph-server/src/lib.rs b/crates/omnigraph-server/src/lib.rs index f7fc6b1..1c70083 100644 --- a/crates/omnigraph-server/src/lib.rs +++ b/crates/omnigraph-server/src/lib.rs @@ -1,4 +1,9 @@ pub mod api; +mod handlers; +mod settings; +pub use settings::{load_server_settings, classify_server_runtime_state, server_config_is_multi, ServerRuntimeState}; +use settings::*; +use handlers::*; pub mod auth; pub mod config; pub mod graph_id; @@ -88,27 +93,27 @@ fn hash_bearer_token(token: &str) -> BearerTokenHash { description = "HTTP API for the Omnigraph graph database", ), paths( - server_health, - server_graphs_list, - server_snapshot, + handlers::server_health, + handlers::server_graphs_list, + handlers::server_snapshot, // deprecated; the #[deprecated] attribute on the handler // surfaces as `deprecated: true` on the OpenAPI operation. - #[allow(deprecated)] server_read, - server_query, - server_export, - #[allow(deprecated)] server_change, - server_mutate, - server_list_queries, - server_invoke_query, - server_schema_apply, - server_schema_get, - server_ingest, - server_branch_list, - server_branch_create, - server_branch_delete, - server_branch_merge, - server_commit_list, - server_commit_show, + #[allow(deprecated)] handlers::server_read, + handlers::server_query, + handlers::server_export, + #[allow(deprecated)] handlers::server_change, + handlers::server_mutate, + handlers::server_list_queries, + handlers::server_invoke_query, + handlers::server_schema_apply, + handlers::server_schema_get, + handlers::server_ingest, + handlers::server_branch_list, + handlers::server_branch_create, + handlers::server_branch_delete, + handlers::server_branch_merge, + handlers::server_commit_list, + handlers::server_commit_show, ), modifiers(&SecurityAddon), )] @@ -888,325 +893,6 @@ fn format_registry_load_errors(label: &str, errors: &[queries::LoadError]) -> St format!("graph '{label}': stored-query registry failed to load:\n {joined}") } -/// Build serving settings from a cluster directory's applied revision -/// (RFC-005 §D2): graphs at derived roots, stored queries from verified -/// catalog blob content, policy bundles from blob paths with their applied -/// bindings. Always multi-graph routing. The unauthenticated/env handling -/// matches the omnigraph.yaml path. -async fn load_cluster_settings( - cluster_dir: &PathBuf, - cli_bind: Option<String>, - cli_allow_unauthenticated: bool, -) -> Result<ServerConfig> { - let snapshot = omnigraph_cluster::read_serving_snapshot(cluster_dir).await.map_err(|diagnostics| { - let details = diagnostics - .iter() - .map(|diagnostic| format!("[{}] {}: {}", diagnostic.code, diagnostic.path, diagnostic.message)) - .collect::<Vec<_>>() - .join("\n "); - eyre!("the cluster at '{}' is not ready to serve:\n {details}", cluster_dir.display()) - })?; - - // Bindings -> Cedar slots. The serving pipeline loads one bundle per - // graph plus one server-level bundle; stacked bundles per scope are a - // later slice — refuse loudly rather than silently merging policy. - let mut server_policy_file: Option<PathBuf> = None; - let mut graph_policy_files: BTreeMap<String, PathBuf> = BTreeMap::new(); - for policy in &snapshot.policies { - for binding in &policy.applies_to { - if binding == "cluster" { - if server_policy_file.replace(policy.blob_path.clone()).is_some() { - bail!( - "multiple policy bundles bind the cluster scope; cluster-mode serving supports one bundle per scope — split or merge bundles (multi-bundle scopes are a later slice)" - ); - } - } else if let Some(graph_id) = binding.strip_prefix("graph.") { - if graph_policy_files - .insert(graph_id.to_string(), policy.blob_path.clone()) - .is_some() - { - bail!( - "multiple policy bundles bind graph '{graph_id}'; cluster-mode serving supports one bundle per scope — split or merge bundles (multi-bundle scopes are a later slice)" - ); - } - } else { - bail!("unrecognized policy binding '{binding}' in the applied revision"); - } - } - } - - let mut graphs = Vec::new(); - for graph in &snapshot.graphs { - let specs: Vec<queries::RegistrySpec> = snapshot - .queries - .iter() - .filter(|query| query.graph_id == graph.graph_id) - .map(|query| queries::RegistrySpec { - name: query.name.clone(), - source: query.source.clone(), - // The §D5 bridge: the cluster registry has no expose flag - // (exposure becomes a policy decision in Phase 6) — cluster - // mode lists every stored query. - expose: true, - tool_name: None, - }) - .collect(); - let registry = QueryRegistry::from_specs(specs).map_err(|errors| { - let details = errors - .iter() - .map(|error| error.to_string()) - .collect::<Vec<_>>() - .join("\n "); - eyre!( - "stored queries in the applied revision failed to parse:\n {details}\nrun `cluster refresh` then `cluster apply`, and restart" - ) - })?; - graphs.push(GraphStartupConfig { - graph_id: graph.graph_id.clone(), - uri: graph.root.to_string_lossy().to_string(), - policy_file: graph_policy_files.get(&graph.graph_id).cloned(), - queries: registry, - }); - } - - let env_unauth = std::env::var("OMNIGRAPH_UNAUTHENTICATED") - .ok() - .map(|v| { - let trimmed = v.trim(); - !trimmed.is_empty() && trimmed != "0" && !trimmed.eq_ignore_ascii_case("false") - }) - .unwrap_or(false); - - Ok(ServerConfig { - mode: ServerConfigMode::Multi { - graphs, - config_path: cluster_dir.clone(), - server_policy_file, - }, - bind: cli_bind.unwrap_or_else(|| "127.0.0.1:8080".to_string()), - allow_unauthenticated: cli_allow_unauthenticated || env_unauth, - }) -} - -pub async fn load_server_settings( - config_path: Option<&PathBuf>, - cli_cluster: Option<&PathBuf>, - cli_uri: Option<String>, - cli_target: Option<String>, - cli_bind: Option<String>, - cli_allow_unauthenticated: bool, -) -> Result<ServerConfig> { - // Rule 0 (RFC-005): --cluster is an exclusive boot source. It is checked - // before anything reads omnigraph.yaml — in cluster mode that file is - // never opened, not even the implicit current-directory search. - if let Some(cluster_dir) = cli_cluster { - if cli_uri.is_some() || cli_target.is_some() || config_path.is_some() { - bail!( - "--cluster is an exclusive boot source; it cannot combine with a graph URI, --target, or --config (axiom 15: a deployment serves from one source)" - ); - } - return load_cluster_settings(cluster_dir, cli_bind, cli_allow_unauthenticated).await; - } - let config = load_config(config_path)?; - let bind = cli_bind.unwrap_or_else(|| config.server_bind().to_string()); - // Either `--unauthenticated` or `OMNIGRAPH_UNAUTHENTICATED=1` flips - // this. Treat any non-empty, non-"0"/"false" string as truthy — - // standard 12-factor "any value is true" reading of the env var. - let env_unauth = std::env::var("OMNIGRAPH_UNAUTHENTICATED") - .ok() - .map(|v| { - let trimmed = v.trim(); - !trimmed.is_empty() && trimmed != "0" && !trimmed.eq_ignore_ascii_case("false") - }) - .unwrap_or(false); - let allow_unauthenticated = cli_allow_unauthenticated || env_unauth; - - // MR-668 decision 2 — four-rule mode inference matrix. - // - // 1. CLI `<URI>` positional → Single (URI = the value) - // 2. CLI `--target <name>` → Single (URI = graphs.<name>.uri) - // 3. `server.graph` in config → Single (URI = graphs.<server.graph>.uri) - // 4. `--config` + non-empty `graphs:` + no single-mode selector - // → Multi (every entry in `graphs:`) - // 5. otherwise → error with migration hint - // - // Rules 1-3 are mutually compatible (CLI URI wins over `--target` - // wins over `server.graph`), reusing the existing - // `resolve_target_uri` precedence. - let has_cli_uri = cli_uri.is_some(); - let has_cli_target = cli_target.is_some(); - let has_server_graph = config.server_graph_name().is_some(); - let has_graphs_map = !config.graphs.is_empty(); - let has_explicit_config = config_path.is_some(); - - let mode = if has_cli_uri || has_cli_target || has_server_graph { - // Rules 1, 2, or 3 → Single mode. - let raw_uri = config.resolve_target_uri( - cli_uri, - cli_target.as_deref(), - config.server_graph_name(), - )?; - let uri = normalize_root_uri(&raw_uri).wrap_err_with(|| { - format!("normalize single-graph URI '{raw_uri}' from server settings") - })?; - // Config follows graph IDENTITY, not mode: a bare URI is anonymous - // (top-level config); a graph chosen by name uses its per-graph - // `graphs.<name>.{policy,queries}`. `resolve_target_uri` already - // errored on an unknown name, so a `Some(name)` here is a known graph. - let selected: Option<&str> = if has_cli_uri { - None - } else { - cli_target.as_deref().or_else(|| config.server_graph_name()) - }; - // A named selection must not leave a populated top-level block - // silently unused — refuse boot and point at the per-graph block. The - // same rule the CLI selection gate enforces, shared via one helper so - // the boot check and `omnigraph queries validate`/`list` can't drift. - config.ensure_top_level_blocks_honored(selected)?; - // Load + identity-check now (no engine needed); the schema - // type-check happens when the engine opens. - let policy_file = config.resolve_policy_file_for(selected); - let queries = QueryRegistry::load(&config, config.query_entries_for(selected)) - .map_err(|errs| color_eyre::eyre::eyre!(format_registry_load_errors(&uri, &errs)))?; - let graph_id = graph_resource_id_for_selection(selected, &uri); - ServerConfigMode::Single { - uri, - graph_id, - policy_file, - queries, - } - } else if has_explicit_config && has_graphs_map { - // Multi mode: every graph uses its per-graph block; top-level - // policy/queries are never honored, so a populated one is an error. - let unhonored = config.populated_top_level_blocks(); - if !unhonored.is_empty() { - bail!( - "multi-graph mode: top-level {} {} not honored — each graph uses its own \ - `graphs.<graph_id>.…` block. Move per-graph rules there (and any \ - `graph_list` policy to `server.policy.file`).", - unhonored.join(" and "), - if unhonored.len() == 1 { "is" } else { "are" }, - ); - } - // Rule 4 → Multi mode. Build a startup config per graph. - let mut graphs = Vec::with_capacity(config.graphs.len()); - for (name, target) in &config.graphs { - // Validate the graph id can construct a `GraphId` newtype. - // Doing this here (not at registry insert) so a malformed - // omnigraph.yaml fails at startup with a clear error. - GraphId::try_from(name.clone()).map_err(|err| { - color_eyre::eyre::eyre!("invalid graph id '{name}' in omnigraph.yaml: {err}") - })?; - let raw_uri = config.resolve_uri_value(&target.uri); - let uri = normalize_root_uri(&raw_uri).wrap_err_with(|| { - format!("normalize URI '{raw_uri}' for graph '{name}' in omnigraph.yaml") - })?; - // Per-graph `queries:`, selected through the shared - // `query_entries_for` so server and CLI resolve identically. - // Load + identity-check now; the schema type-check happens - // when this graph's engine opens. - let queries = QueryRegistry::load(&config, config.query_entries_for(Some(name.as_str()))) - .map_err(|errs| color_eyre::eyre::eyre!(format_registry_load_errors(name, &errs)))?; - graphs.push(GraphStartupConfig { - graph_id: name.clone(), - uri, - policy_file: config.resolve_target_policy_file(name), - queries, - }); - } - let config_path = config_path - .cloned() - .expect("has_explicit_config implies config_path is Some"); - let server_policy_file = config.resolve_server_policy_file(); - ServerConfigMode::Multi { - graphs, - config_path, - server_policy_file, - } - } else { - // Rule 5 → error with migration hint. - bail!( - "no graph to serve: pass a URI (`omnigraph-server <URI>`), select a target \ - (`--target <name> --config omnigraph.yaml`), set `server.graph: <name>` in \ - omnigraph.yaml, or for multi-graph mode add a `graphs:` map to the config \ - file referenced by `--config`." - ); - }; - - Ok(ServerConfig { - mode, - bind, - allow_unauthenticated, - }) -} - -/// Whether the loaded config will run the server in multi-graph mode. -/// Useful for the test that constructs `ServerConfig` directly. -pub fn server_config_is_multi(config: &ServerConfig) -> bool { - matches!(config.mode, ServerConfigMode::Multi { .. }) -} - -/// MR-723 server runtime state, classified from the three-state matrix -/// of (bearer tokens configured) × (policy file configured) at startup. -/// -/// * **Open** — neither tokens nor policy; requires explicit -/// `allow_unauthenticated`. Effectively a "trust the network" dev -/// mode. `serve()` refuses to start in this shape without the flag, -/// so the only way to reach this state at runtime is via deliberate -/// operator opt-in. -/// * **DefaultDeny** — tokens configured but no policy file. The -/// server requires a valid bearer token; once authenticated, every -/// action except `Read` is denied with 403. Closes the "tokens but -/// forgot the policy file" trap. -/// * **PolicyEnabled** — policy file configured and at least one -/// bearer token configured. Cedar evaluates every authenticated -/// request. Policy without tokens is rejected at startup — -/// such a server would 401 every request, which is bug-shaped -/// rather than feature-shaped (operators wanting "deny all -/// unauthenticated traffic" should configure tokens plus a -/// deny-all policy to get meaningful 403s with policy-decision -/// logging instead). -#[derive(Debug, Clone, Copy, Eq, PartialEq)] -pub enum ServerRuntimeState { - Open, - DefaultDeny, - PolicyEnabled, -} - -/// Compute the [`ServerRuntimeState`] from the configured inputs. -/// Pulled out as a pure function so the matrix is unit-testable -/// without standing up the full server. -/// -/// The classifier is the **single source of truth** for "should we -/// start?" — both `serve()`'s single-mode and multi-mode branches -/// call this before constructing their `AppState`. Adding a startup -/// invariant here means both modes enforce it automatically; the -/// alternative (per-constructor `bail!`) drifts the moment a third -/// mode is added. -pub fn classify_server_runtime_state( - has_tokens: bool, - has_policy: bool, - allow_unauthenticated: bool, -) -> Result<ServerRuntimeState> { - match (has_tokens, has_policy, allow_unauthenticated) { - (false, false, false) => bail!( - "server has no bearer tokens and no policy file configured. This is a fully \ - open server — pass `--unauthenticated` (or set OMNIGRAPH_UNAUTHENTICATED=1) \ - if you actually want that, otherwise configure bearer tokens (see \ - docs/user/server.md) and/or `policy.file` in omnigraph.yaml." - ), - (false, false, true) => Ok(ServerRuntimeState::Open), - (true, false, _) => Ok(ServerRuntimeState::DefaultDeny), - (false, true, _) => bail!( - "policy file is configured but no bearer tokens — every request would 401 \ - because no token can ever match. Configure at least one bearer token (see \ - docs/user/server.md), or remove the policy file. To deny all unauthenticated \ - traffic deliberately, configure tokens plus a deny-all Cedar rule — that \ - produces meaningful 403s with policy-decision logging instead of silent 401s." - ), - (true, true, _) => Ok(ServerRuntimeState::PolicyEnabled), - } -} pub fn build_app(state: AppState) -> Router { // The per-graph protected routes, identical in single + multi mode. @@ -1469,2325 +1155,4 @@ async fn shutdown_signal() { info!("shutdown signal received"); } -#[utoipa::path( - get, - path = "/healthz", - tag = "health", - operation_id = "health", - responses( - (status = 200, description = "Server is healthy", body = HealthOutput), - ), -)] -/// Liveness probe. -/// -/// Returns server status and version. Unauthenticated; safe to call from any -/// caller. Use this to confirm the server is reachable before invoking other -/// endpoints. -async fn server_health() -> Json<HealthOutput> { - Json(HealthOutput { - status: "ok".to_string(), - version: SERVER_VERSION.to_string(), - source_version: SERVER_SOURCE_VERSION.map(str::to_string), - }) -} -#[utoipa::path( - get, - path = "/graphs", - tag = "management", - operation_id = "listGraphs", - responses( - (status = 200, description = "List of registered graphs", body = GraphListResponse), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - (status = 405, description = "Method not allowed (single-graph mode)", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// List every graph currently registered with this server (MR-668). -/// -/// Multi-graph mode only. In single mode, the route returns 405 — there's -/// no registry to enumerate. Cedar-gated by the server-level policy via -/// the `graph_list` action against `Omnigraph::Server::"root"`. -/// -/// Order: alphabetical by `graph_id` (server-sorted so clients see -/// deterministic output across requests). -async fn server_graphs_list( - State(state): State<AppState>, - actor: Option<Extension<ResolvedActor>>, -) -> std::result::Result<Json<GraphListResponse>, ApiError> { - // 405 in single mode — there's no registry to enumerate, and the - // legacy URL surface didn't expose this endpoint. - let registry = match state.routing() { - GraphRouting::Single { .. } => { - return Err(ApiError::method_not_allowed( - "GET /graphs is only available in multi-graph mode", - )); - } - GraphRouting::Multi { registry, .. } => registry, - }; - - // Server-level Cedar gate. `state.server_policy` is loaded from - // `server.policy.file` in `omnigraph.yaml` at startup. When no - // server policy is configured, `authorize_request_server` falls - // through to the MR-723 default-deny semantics (every non-Read - // action denied for an authenticated actor). `GraphList` is not - // `Read`, so without a server policy the request gets 403 — which - // is the right default (don't leak the registry until the operator - // explicitly authorizes it). - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - state.server_policy.as_deref(), - PolicyRequest { - action: PolicyAction::GraphList, - branch: None, - target_branch: None, - }, - )?; - - let mut graphs: Vec<GraphInfo> = registry - .list() - .into_iter() - .map(|handle| GraphInfo { - graph_id: handle.key.graph_id.as_str().to_string(), - uri: handle.uri.clone(), - }) - .collect(); - graphs.sort_by(|a, b| a.graph_id.cmp(&b.graph_id)); - Ok(Json(GraphListResponse { graphs })) -} - -async fn server_openapi(State(state): State<AppState>) -> Json<utoipa::openapi::OpenApi> { - let mut doc = ApiDoc::openapi(); - if !state.requires_bearer_auth() { - strip_security(&mut doc); - } - // MR-668: in multi mode, the protected routes live under - // `/graphs/{graph_id}/...`. Rewrite the doc so the spec matches - // the routes the router actually serves. Public paths (`/healthz`) - // stay flat in both modes. - if matches!(state.routing(), GraphRouting::Multi { .. }) { - nest_paths_under_cluster_prefix(&mut doc); - } - Json(doc) -} - -/// Path prefix used to namespace per-graph routes in multi mode. -/// Kept in sync with the `Router::nest(...)` invocation in `build_app`. -const CLUSTER_PATH_PREFIX: &str = "/graphs/{graph_id}"; - -/// Operation-id prefix applied to every cloned cluster operation. -/// Decision 7 in the implementation plan — keeps operation IDs unique -/// across the spec when both flat and nested variants ever appear in -/// the same generation pass. -const CLUSTER_OPERATION_ID_PREFIX: &str = "cluster_"; - -/// Paths that stay flat in every server mode (public or server-level, -/// no per-graph dependency). Update this list when adding new -/// always-flat endpoints. `/graphs` is the management enumeration — -/// it lives at the root in both single mode (405) and multi mode, and -/// must never be rewritten to `/graphs/{graph_id}/graphs`. -const ALWAYS_FLAT_PATHS: &[&str] = &["/healthz", "/graphs"]; - -/// In multi-mode `server_openapi`, every protected path-item is -/// reattached under the cluster prefix. Operation IDs gain the -/// `cluster_` prefix so SDK generators don't collide if/when both -/// surfaces are merged. Every rewritten operation also declares the -/// required `{graph_id}` path parameter so the served OpenAPI document -/// remains internally valid. -/// -/// Removing the flat protected paths matches the runtime router — -/// in multi mode, requests to `/snapshot` etc. return 404, so the -/// spec must agree. -fn nest_paths_under_cluster_prefix(doc: &mut utoipa::openapi::OpenApi) { - let original = std::mem::take(&mut doc.paths.paths); - let mut rewritten = std::collections::BTreeMap::new(); - for (path, mut item) in original { - if ALWAYS_FLAT_PATHS.contains(&path.as_str()) { - rewritten.insert(path, item); - continue; - } - rename_operation_ids(&mut item, CLUSTER_OPERATION_ID_PREFIX); - add_cluster_graph_id_parameter(&mut item); - let new_path = format!("{CLUSTER_PATH_PREFIX}{path}"); - rewritten.insert(new_path, item); - } - doc.paths.paths = rewritten; -} - -fn add_cluster_graph_id_parameter(item: &mut utoipa::openapi::PathItem) { - for op in path_item_operations_mut(item) { - let parameters = op.parameters.get_or_insert_with(Vec::new); - let has_graph_id = parameters - .iter() - .any(|param| param.name == "graph_id" && param.parameter_in == ParameterIn::Path); - if !has_graph_id { - parameters.insert(0, graph_id_path_parameter()); - } - } -} - -fn graph_id_path_parameter() -> Parameter { - let mut parameter = Parameter::new("graph_id"); - parameter.parameter_in = ParameterIn::Path; - parameter.description = Some("Graph id to route the request to.".to_string()); - parameter.schema = Some(Object::with_type(Type::String).into()); - parameter -} - -/// Prefix every operation_id in this PathItem with `prefix`. -fn rename_operation_ids(item: &mut utoipa::openapi::PathItem, prefix: &str) { - for op in path_item_operations_mut(item) { - if let Some(id) = op.operation_id.as_deref() { - op.operation_id = Some(format!("{prefix}{id}")); - } - } -} - -fn path_item_operations_mut( - item: &mut utoipa::openapi::PathItem, -) -> impl Iterator<Item = &mut utoipa::openapi::path::Operation> { - [ - item.get.as_mut(), - item.post.as_mut(), - item.put.as_mut(), - item.delete.as_mut(), - item.options.as_mut(), - item.head.as_mut(), - item.patch.as_mut(), - item.trace.as_mut(), - ] - .into_iter() - .flatten() -} - -fn strip_security(doc: &mut utoipa::openapi::OpenApi) { - if let Some(components) = doc.components.as_mut() { - components.security_schemes.clear(); - } - for path_item in doc.paths.paths.values_mut() { - for op in [ - path_item.get.as_mut(), - path_item.post.as_mut(), - path_item.put.as_mut(), - path_item.delete.as_mut(), - path_item.options.as_mut(), - path_item.head.as_mut(), - path_item.patch.as_mut(), - path_item.trace.as_mut(), - ] - .into_iter() - .flatten() - { - op.security = None; - } - } -} - -async fn require_bearer_auth( - State(state): State<AppState>, - mut request: Request, - next: Next, -) -> std::result::Result<Response, ApiError> { - if !state.requires_bearer_auth() { - return Ok(next.run(request).await); - } - - let Some(header) = request - .headers() - .get(AUTHORIZATION) - .and_then(|value| value.to_str().ok()) - else { - return Err(ApiError::unauthorized("missing bearer token")); - }; - - let Some(provided_token) = header.strip_prefix("Bearer ") else { - return Err(ApiError::unauthorized("missing bearer token")); - }; - - let Some(actor) = state.authenticate_bearer_token(provided_token) else { - return Err(ApiError::unauthorized("invalid bearer token")); - }; - request.extensions_mut().insert(actor); - - Ok(next.run(request).await) -} - -/// Routing middleware (MR-668). Resolves the active graph for the -/// request and injects `Arc<GraphHandle>` as an extension so handlers can -/// extract it via `Extension<Arc<GraphHandle>>`. -/// -/// **Single mode**: the routing field holds the single handle directly. -/// Routes are flat; every request resolves to that handle, regardless -/// of the URI path. No registry walk, no sentinel key, no -/// programmer-error guard. -/// -/// **Multi mode**: routes are nested under `/graphs/{graph_id}/...`. The -/// middleware extracts `{graph_id}` from the URI path and looks it up in -/// the registry. Returns 404 if the graph is not registered. -/// -/// The middleware fires AFTER `require_bearer_auth`, so the actor is -/// already in the request extensions (or auth was off entirely). -async fn resolve_graph_handle( - State(state): State<AppState>, - mut request: Request, - next: Next, -) -> std::result::Result<Response, ApiError> { - let handle = match &state.routing { - GraphRouting::Single { handle } => Arc::clone(handle), - GraphRouting::Multi { registry, .. } => { - // `Router::nest("/graphs/{graph_id}", inner)` rewrites - // `request.uri().path()` to the inner suffix (e.g. `/snapshot`). - // The pre-rewrite URI is preserved in the `OriginalUri` - // request extension by axum's router; we read from there to - // extract `{graph_id}`. Fall back to the current URI only if - // the extension is missing, which shouldn't happen for - // nested routes but is safe defensive code. - let original_path: String = request - .extensions() - .get::<OriginalUri>() - .map(|OriginalUri(uri)| uri.path().to_string()) - .unwrap_or_else(|| request.uri().path().to_string()); - let graph_id_str = original_path - .strip_prefix("/graphs/") - .and_then(|rest| rest.split('/').next()) - .filter(|s| !s.is_empty()) - .ok_or_else(|| { - ApiError::bad_request( - "cluster route missing /graphs/{graph_id} prefix".to_string(), - ) - })?; - let graph_id = GraphId::try_from(graph_id_str.to_string()) - .map_err(|err| ApiError::bad_request(err.to_string()))?; - let key = GraphKey::cluster(graph_id.clone()); - match registry.get(&key) { - RegistryLookup::Ready(handle) => handle, - RegistryLookup::Gone => { - return Err(ApiError::not_found(format!("graph '{graph_id}' not found"))); - } - } - } - }; - - // Per-request observability. `Span::current().record` would silently - // no-op here because no upstream `#[tracing::instrument(...)]` macro - // declares a `graph_id` field; emit an explicit event instead so the - // routing decision actually lands in logs. - info!(graph_id = %handle.key.graph_id, "graph routed"); - - request.extensions_mut().insert(handle); - Ok(next.run(request).await) -} - -fn log_policy_decision(actor_id: &str, request: &PolicyRequest, decision: &PolicyDecision) { - info!( - actor_id = actor_id, - action = %request.action, - branch = request.branch.as_deref().unwrap_or(""), - target_branch = request.target_branch.as_deref().unwrap_or(""), - allowed = decision.allowed, - matched_rule_id = decision.matched_rule_id.as_deref().unwrap_or(""), - "policy decision" - ); -} - -/// The allow/deny **decision** an authorization check produces, kept -/// separate from the operational failures (`Err`) that can occur while -/// computing it. [`authorize_request`] collapses `Denied` to a 403; a caller -/// that needs to remap a denial without also remapping operational failures -/// (the stored-query invoke handler hides a denial as a 404) matches on this -/// directly, so a real 401 (missing bearer) or 500 (policy-evaluation error) -/// keeps its true status instead of being masked as the denial's response. -enum Authz { - Allowed, - Denied(String), -} - -/// HTTP-layer Cedar policy gate, returning the allow/deny [`Authz`] decision -/// and reserving `Err` for operational failures (401 missing bearer, 500 -/// policy-evaluation error). Two sources of the policy engine: -/// * Per-graph handler — passes `handle.policy.as_deref()` so the -/// graph's Cedar rules govern read/change/branch_*/schema_apply. -/// * Management handler — passes `state.server_policy.as_deref()` so -/// server-level Cedar rules govern `graph_list` (the only shipped -/// server-scoped action; runtime `graph_create` / `graph_delete` -/// are deferred until a managed cluster catalog lands). -/// -/// The MR-731 invariant lives inside this function: actor identity is -/// supplied as a separate argument from the resolved bearer match. The -/// `PolicyRequest` struct itself does not carry identity (the field was -/// dropped from the type), so handlers cannot smuggle it through the -/// request. See `actor_id_resolves_from_bearer_token_ignoring_client_supplied_headers` -/// at `tests/server.rs`. -fn authorize( - actor: Option<&ResolvedActor>, - policy: Option<&PolicyEngine>, - request: PolicyRequest, -) -> std::result::Result<Authz, ApiError> { - let Some(engine) = policy else { - // No PolicyEngine installed. Three runtime states can reach this: - // - // * **Open mode** (`--unauthenticated`): no tokens, no policy. - // Per-graph operations are open by operator opt-in (they - // accepted "trust the network" for graph data). - // * **DefaultDeny mode**: tokens configured but no policy. The - // request went through bearer auth, so `actor` is Some. Only - // per-graph `Read` is permitted; other per-graph actions - // return 403. Closes the "configured auth but forgot the - // policy file" trap from MR-723. - // * Either of the above with a **server-scoped** action - // (`graph_list`, future `graph_create`/`graph_delete`). - // - // Server-scoped actions are always denied here, regardless of - // mode or actor presence. The management surface leaks server - // topology (graph IDs + URIs that may contain S3 bucket paths - // or internal hostnames) — operators who opted into Open mode - // accepted exposure of graph DATA, not exposure of server - // topology. Closing the management surface by default in every - // runtime state means the docstring contract on - // `server_graphs_list` ("don't leak the registry until the - // operator explicitly authorizes it") holds uniformly; the - // operator's only path to enabling it is configuring an - // explicit `server.policy.file` in omnigraph.yaml. - if request.action.resource_kind() == PolicyResourceKind::Server { - return Ok(Authz::Denied( - "server-scoped actions require an explicit `server.policy.file` \ - configured in omnigraph.yaml — the management surface is closed \ - by default in every runtime state, including --unauthenticated, \ - so that server topology is never exposed without operator opt-in." - .to_string(), - )); - } - if actor.is_some() && request.action != PolicyAction::Read { - return Ok(Authz::Denied( - "server runs in default-deny mode (bearer tokens configured but no \ - policy file). Only `read` actions are permitted; configure \ - `policy.file` in omnigraph.yaml to enable other actions." - .to_string(), - )); - } - return Ok(Authz::Allowed); - }; - let Some(actor) = actor else { - return Err(ApiError::unauthorized("missing bearer token")); - }; - // SECURITY INVARIANT (MR-731): actor identity is supplied to the - // policy engine here as a separate argument, sourced from the - // bearer-token match resolved by `require_bearer_auth`. The - // `PolicyRequest` struct itself no longer carries `actor_id` (it - // was dropped from the type), so handlers cannot smuggle identity - // through the request body and there is no overwrite step that - // could be skipped. The principle is codified in - // `docs/dev/invariants.md` Hard Invariant 11 ("clients cannot set - // actor identity directly") and pinned by the regression test - // `actor_id_resolves_from_bearer_token_ignoring_client_supplied_headers` - // in `crates/omnigraph-server/tests/server.rs`. - let actor_id = actor.actor_id.as_ref(); - let decision = engine - .authorize(actor_id, &request) - .map_err(|err| ApiError::internal(format!("policy: {err}")))?; - log_policy_decision(actor_id, &request, &decision); - if decision.allowed { - Ok(Authz::Allowed) - } else { - Ok(Authz::Denied(decision.message)) - } -} - -/// Thin wrapper over [`authorize`] for the handlers that treat any denial as a -/// 403: a denial becomes `ApiError::forbidden`, and operational failures -/// (401 missing bearer, 500 policy-evaluation error) propagate unchanged. The -/// stored-query invoke handler does **not** use this — it consumes the -/// [`Authz`] decision directly to hide a denial as a 404 while letting an -/// operational failure keep its true status. -fn authorize_request( - actor: Option<&ResolvedActor>, - policy: Option<&PolicyEngine>, - request: PolicyRequest, -) -> std::result::Result<(), ApiError> { - match authorize(actor, policy, request)? { - Authz::Allowed => Ok(()), - Authz::Denied(message) => Err(ApiError::forbidden(message)), - } -} - -#[utoipa::path( - get, - path = "/snapshot", - tag = "snapshots", - operation_id = "getSnapshot", - params(SnapshotQuery), - responses( - (status = 200, description = "Database snapshot", body = api::SnapshotOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Read the current snapshot of a branch. -/// -/// Returns the manifest version plus per-table metadata (path, version, row -/// count) for every table on the branch. Defaults to `main` when `branch` is -/// omitted. Read-only. -async fn server_snapshot( - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Query(query): Query<SnapshotQuery>, -) -> std::result::Result<Json<api::SnapshotOutput>, ApiError> { - let branch = query.branch.unwrap_or_else(|| "main".to_string()); - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::Read, - branch: Some(branch.clone()), - target_branch: None, - }, - )?; - let snapshot = { - let db = &handle.engine; - db.snapshot_of(ReadTarget::branch(branch.as_str())) - .await - .map_err(ApiError::from_omni)? - }; - Ok(Json(snapshot_payload(&branch, &snapshot))) -} - -/// Header values that flag a response as coming from a deprecated route -/// (RFC 9745 / RFC 8288) and point at the canonical successor. -fn deprecation_headers(successor_link: &'static str) -> [(HeaderName, HeaderValue); 2] { - [ - ( - HeaderName::from_static("deprecation"), - HeaderValue::from_static("true"), - ), - ( - HeaderName::from_static("link"), - HeaderValue::from_static(successor_link), - ), - ] -} - -#[utoipa::path( - post, - path = "/read", - tag = "queries", - operation_id = "read", - request_body = ReadRequest, - responses( - (status = 200, description = "Query results (response includes `Deprecation: true` + `Link: </query>; rel=\"successor-version\"`)", body = ReadOutput), - (status = 400, description = "Bad request", body = ErrorOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -#[deprecated(note = "use POST /query instead; /read is kept indefinitely for byte-stable back-compat")] -/// **Deprecated** — use [`POST /query`](#tag/queries/operation/query) instead. -/// -/// Execute a GQ read query. Behavior is unchanged from prior releases; the -/// route is kept indefinitely for byte-stable back-compat. New integrations -/// should target `POST /query`, which has clean field names (`query` / -/// `name`) and a 400-on-mutation guard. Responses from this route include -/// `Deprecation: true` and `Link: </query>; rel="successor-version"` -/// headers per RFC 9745 / RFC 8288 so SDKs and proxies can surface the -/// signal. -async fn server_read( - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Json(request): Json<ReadRequest>, -) -> std::result::Result<([(HeaderName, HeaderValue); 2], Json<ReadOutput>), ApiError> { - let (selected_name, target, result) = run_query( - handle, - actor.as_ref().map(|Extension(actor)| actor), - &request.query_source, - request.query_name.as_deref(), - request.params.as_ref(), - request.branch, - request.snapshot, - false, // /read predates the D2 rule; legacy callers may submit mutating queries here - ) - .await?; - Ok(( - deprecation_headers("</query>; rel=\"successor-version\""), - Json(api::read_output(selected_name, &target, result)), - )) -} - -#[utoipa::path( - post, - path = "/query", - tag = "queries", - operation_id = "query", - request_body = QueryRequest, - responses( - (status = 200, description = "Query results", body = ReadOutput), - (status = 400, description = "Bad request - also returned when the query body contains mutations; use POST /mutate (or its deprecated alias POST /change) for write queries", body = ErrorOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Execute an inline read query (friendlier-named alternative to `POST /read`). -/// -/// Designed for ad-hoc exploration and AI-agent tool-use: short field -/// names (`query`, `name`) match the CLI `-e` flag and the GQ `query` -/// keyword. Mutations (`insert`/`update`/`delete`) are rejected with 400 -/// -- use `POST /mutate` (or its deprecated alias `POST /change`) for -/// write queries. Otherwise behaves identically to `POST /read`: same -/// target semantics (branch xor snapshot), same Cedar action (Read), -/// same response shape. -async fn server_query( - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Json(request): Json<QueryRequest>, -) -> std::result::Result<Json<ReadOutput>, ApiError> { - let (selected_name, target, result) = run_query( - handle, - actor.as_ref().map(|Extension(actor)| actor), - &request.query, - request.name.as_deref(), - request.params.as_ref(), - request.branch, - request.snapshot, - true, // /query is read-only; reject mutations - ) - .await?; - Ok(Json(api::read_output(selected_name, &target, result))) -} - -#[utoipa::path( - post, - path = "/export", - tag = "queries", - operation_id = "export", - request_body = ExportRequest, - responses( - (status = 200, description = "Exported data as NDJSON", content_type = "application/x-ndjson"), - (status = 400, description = "Bad request", body = ErrorOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Stream the contents of a branch as NDJSON. -/// -/// Emits one JSON object per line (`application/x-ndjson`). Filter with -/// `type_names` (node/edge type names) and/or `table_keys`; both empty -/// streams the entire branch. Suitable for large exports — the response is -/// streamed, not buffered. Read-only. -async fn server_export( - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Json(request): Json<ExportRequest>, -) -> std::result::Result<Response, ApiError> { - let branch = request.branch.unwrap_or_else(|| "main".to_string()); - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::Export, - branch: Some(branch.clone()), - target_branch: None, - }, - )?; - let engine = Arc::clone(&handle.engine); - let type_names = request.type_names.clone(); - let table_keys = request.table_keys.clone(); - let (tx, rx) = mpsc::unbounded_channel::<std::result::Result<Bytes, io::Error>>(); - tokio::spawn(async move { - let result = { - let mut writer = ExportStreamWriter { sender: tx.clone() }; - engine - .export_jsonl_to_writer(&branch, &type_names, &table_keys, &mut writer) - .await - }; - if let Err(err) = result { - let _ = tx.send(Err(io::Error::other(err.to_string()))); - } - }); - let body = Body::from_stream(stream::unfold(rx, |mut rx| async move { - rx.recv().await.map(|item| (item, rx)) - })); - Ok(( - StatusCode::OK, - [(CONTENT_TYPE, "application/x-ndjson; charset=utf-8")], - body, - ) - .into_response()) -} - -/// Shared implementation behind `POST /mutate` (canonical) and -/// `POST /change` (deprecated alias). Returns the bare `ChangeOutput`; -/// each route handler wraps it (the alias also attaches Deprecation -/// headers). -/// Shared backend for `/mutate` (canonical) and `/change` (deprecated alias). -/// -/// Decoupled from `ChangeRequest` so MR-969's `/queries/{name}` stored-query -/// handler can call this directly with registry-supplied fields without -/// rebuilding the request body. Today's HTTP handlers unpack the request and -/// call here; the registry would do the same. -async fn run_mutate( - state: AppState, - handle: Arc<GraphHandle>, - actor: Option<&ResolvedActor>, - query: &str, - name: Option<&str>, - params_json: Option<&Value>, - branch: String, -) -> std::result::Result<ChangeOutput, ApiError> { - let actor_arc = actor - .map(|a| Arc::clone(&a.actor_id)) - .unwrap_or_else(|| Arc::<str>::from("anonymous")); - let actor_id = actor.map(|a| a.actor_id.as_ref()); - authorize_request( - actor, - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::Change, - branch: Some(branch.clone()), - target_branch: None, - }, - )?; - // Per-actor admission: bound concurrent in-flight mutations and - // estimated bytes per actor. Cedar runs FIRST so denied requests - // don't consume admission slots. Estimate uses the request body - // size as a coarse proxy; engine memory pressure can run higher. - let est_bytes = query.len() as u64 - + params_json - .map(|p| p.to_string().len() as u64) - .unwrap_or(0); - let _admission = state - .workload - .try_admit(&actor_arc, est_bytes) - .map_err(ApiError::from_workload_reject)?; - let (selected_name, query_params) = - select_named_query(query, name).map_err(|err| ApiError::bad_request(err.to_string()))?; - let params = query_params_from_json(&query_params, params_json) - .map_err(|err| ApiError::bad_request(err.to_string()))?; - - let result = { - let db = &handle.engine; - db.mutate_as(&branch, query, &selected_name, ¶ms, actor_id) - .await - .map_err(ApiError::from_omni)? - }; - Ok(ChangeOutput { - branch, - query_name: selected_name, - affected_nodes: result.affected_nodes, - affected_edges: result.affected_edges, - actor_id: actor_id.map(str::to_string), - }) -} - -/// Shared backend for `/query` (canonical) and `/read` (deprecated alias). -/// -/// Mirrors [`run_mutate`]'s decoupled shape so MR-969's stored-query handler -/// can call here with registry-supplied fields. Rejects inline source that -/// contains mutations (D2 rule); callers wanting writes go through -/// [`run_mutate`] instead. -/// -/// Intentionally does **not** take [`AppState`] (unlike [`run_mutate`]): -/// reads are not admission-gated today, so there is no `state.workload` -/// consumer. The signature grows the parameter when Phase 1 (MR-976) adds -/// the request envelope's `expect: { max_rows_scanned: N }` budget, or -/// MR-969 extends per-actor admission to stored-read invocations. -async fn run_query( - handle: Arc<GraphHandle>, - actor: Option<&ResolvedActor>, - query: &str, - name: Option<&str>, - params_json: Option<&Value>, - branch: Option<String>, - snapshot: Option<String>, - reject_mutations: bool, -) -> std::result::Result<(String, ReadTarget, omnigraph_compiler::result::QueryResult), ApiError> { - if branch.is_some() && snapshot.is_some() { - return Err(ApiError::bad_request( - "request may specify branch or snapshot, not both", - )); - } - - let target = read_target_from_request(branch, snapshot); - let policy_branch = match &target { - ReadTarget::Branch(branch) => Some(branch.clone()), - ReadTarget::Snapshot(_) if handle.policy.is_some() && actor.is_some() => { - let db = &handle.engine; - db.resolved_branch_of(target.clone()) - .await - .map(|branch| branch.or_else(|| Some("main".to_string()))) - .map_err(ApiError::from_omni)? - } - ReadTarget::Snapshot(_) => None, - }; - authorize_request( - actor, - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::Read, - branch: policy_branch, - target_branch: None, - }, - )?; - let query_decl = - select_named_query_decl(query, name).map_err(|err| ApiError::bad_request(err.to_string()))?; - if reject_mutations && !query_decl.mutations.is_empty() { - return Err(ApiError::bad_request(format!( - "query '{}' contains mutations (insert/update/delete); use POST /mutate for write queries", - query_decl.name - ))); - } - let selected_name = query_decl.name.clone(); - let params = query_params_from_json(&query_decl.params, params_json) - .map_err(|err| ApiError::bad_request(err.to_string()))?; - - let result = { - let db = &handle.engine; - db.query(target.clone(), query, &selected_name, ¶ms) - .await - .map_err(ApiError::from_omni)? - }; - Ok((selected_name, target, result)) -} - -#[utoipa::path( - post, - path = "/change", - tag = "mutations", - operation_id = "change", - request_body = ChangeRequest, - responses( - (status = 200, description = "Mutation results (response includes `Deprecation: true` + `Link: </mutate>; rel=\"successor-version\"`)", body = ChangeOutput), - (status = 400, description = "Bad request", body = ErrorOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - (status = 409, description = "Merge conflict", body = ErrorOutput), - (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -#[deprecated(note = "use POST /mutate instead; /change is kept indefinitely for back-compat")] -/// **Deprecated** — use [`POST /mutate`](#tag/mutations/operation/mutate) instead. -/// -/// Apply a GQ mutation to a branch. Behavior is unchanged; the route is -/// kept indefinitely for back-compat. New integrations should target -/// `POST /mutate`, which has identical semantics and a name that pairs -/// cleanly with `POST /query`. Responses from this route include -/// `Deprecation: true` and `Link: </mutate>; rel="successor-version"` -/// headers per RFC 9745 / RFC 8288 so SDKs and proxies can surface the -/// signal. -async fn server_change( - State(state): State<AppState>, - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Json(request): Json<ChangeRequest>, -) -> std::result::Result<([(HeaderName, HeaderValue); 2], Json<ChangeOutput>), ApiError> { - let branch = request.branch.unwrap_or_else(|| "main".to_string()); - let output = run_mutate( - state, - handle, - actor.as_ref().map(|Extension(actor)| actor), - &request.query, - request.name.as_deref(), - request.params.as_ref(), - branch, - ) - .await?; - Ok(( - deprecation_headers("</mutate>; rel=\"successor-version\""), - Json(output), - )) -} - -#[utoipa::path( - post, - path = "/mutate", - tag = "mutations", - operation_id = "mutate", - request_body = ChangeRequest, - responses( - (status = 200, description = "Mutation results", body = ChangeOutput), - (status = 400, description = "Bad request", body = ErrorOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - (status = 409, description = "Merge conflict", body = ErrorOutput), - (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Apply a GQ mutation to a branch (canonical mutation endpoint). -/// -/// Writes to the named `branch` (defaults to `main`). Mutations are atomic -/// per call and produce a new commit. Returns counts of nodes and edges -/// affected. **Destructive**: on success the branch is updated; rejected -/// mutations may still acquire locks briefly. Returns 409 on merge conflict. -/// -/// Pairs with `POST /query` (read-only). The legacy `POST /change` route -/// has identical semantics and is kept as a deprecated alias. -async fn server_mutate( - State(state): State<AppState>, - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Json(request): Json<ChangeRequest>, -) -> std::result::Result<Json<ChangeOutput>, ApiError> { - let branch = request.branch.unwrap_or_else(|| "main".to_string()); - Ok(Json( - run_mutate( - state, - handle, - actor.as_ref().map(|Extension(actor)| actor), - &request.query, - request.name.as_deref(), - request.params.as_ref(), - branch, - ) - .await?, - )) -} - -/// Path parameter for `POST /queries/{name}`. -#[derive(Deserialize)] -struct QueryNamePath { - name: String, -} - -fn parse_optional_invoke_body( - body: Bytes, -) -> std::result::Result<InvokeStoredQueryRequest, ApiError> { - if body.is_empty() { - return Ok(InvokeStoredQueryRequest::default()); - } - serde_json::from_slice::<Option<InvokeStoredQueryRequest>>(&body) - .map(|request| request.unwrap_or_default()) - .map_err(|err| { - ApiError::bad_request(format!("invalid stored-query invocation body: {err}")) - }) -} - -#[utoipa::path( - post, - path = "/queries/{name}", - tag = "queries", - operation_id = "invoke_query", - params(("name" = String, Path, description = "Stored query name (the registry key)")), - request_body = Option<InvokeStoredQueryRequest>, - responses( - (status = 200, description = "Read envelope (ReadOutput) or mutation envelope (ChangeOutput), serialized untagged", body = InvokeStoredQueryResponse), - (status = 400, description = "Bad request (param type error; snapshot on a stored mutation)", body = ErrorOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden (the inner `change` gate for a stored mutation)", body = ErrorOutput), - (status = 404, description = "Unknown stored query, or `invoke_query` denied — indistinguishable to a caller without the grant", body = ErrorOutput), - (status = 409, description = "Merge conflict", body = ErrorOutput), - (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), - (status = 500, description = "Policy evaluation error (a denial is reported as 404, not 500)", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Invoke a curated, server-side stored query by name. -/// -/// The query source comes from the graph's `queries:` registry, not the -/// request body — callers send only runtime inputs (`params`, `branch`, -/// `snapshot`). Gated by the `invoke_query` Cedar action at the boundary; -/// a stored *mutation* additionally passes the engine's `change` gate -/// (double-gated). An actor **without** `invoke_query` cannot tell a denied -/// query from a missing one — both return the same 404, so the catalog -/// can't be probed without the grant. Once `invoke_query` is held, the -/// inner `read`/`change` gate may surface a 403 for an existing query the -/// actor can't run (the intended double-gate signal). -async fn server_invoke_query( - State(state): State<AppState>, - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Path(QueryNamePath { name }): Path<QueryNamePath>, - body: Bytes, -) -> std::result::Result<Json<InvokeStoredQueryResponse>, ApiError> { - let req = parse_optional_invoke_body(body)?; - // A caller without `invoke_query` can't tell a denial from a missing - // query: both 404 with this exact message, so the catalog can't be - // probed without the grant. (A caller that holds invoke_query may still - // see the inner gate's 403 for an existing query it can't run — intended.) - const NOT_FOUND: &str = "stored query not found"; - let actor_ref = actor.as_ref().map(|Extension(actor)| actor); - - // Boundary gate (authentication already ran in `require_bearer_auth`). - // A denial is hidden as 404 (deny == missing, so the catalog can't be - // probed without the grant), but operational failures (401 missing bearer, - // 500 policy-evaluation error) propagate with their true status via `?` - // rather than being masked as a missing query. - match authorize( - actor_ref, - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::InvokeQuery, - // Graph-scoped: no branch dimension. The per-branch/snapshot - // access is enforced by the inner read/change gate in the - // runner, so the outer gate must not resolve a branch (doing so - // was wrong for snapshot reads). - branch: None, - target_branch: None, - }, - )? { - Authz::Allowed => {} - Authz::Denied(_) => return Err(ApiError::not_found(NOT_FOUND)), - } - - // Resolve against the per-graph registry (same 404 on a miss). - let stored = handle - .queries - .as_ref() - .and_then(|registry| registry.lookup(&name)) - .ok_or_else(|| ApiError::not_found(NOT_FOUND))?; - - // Detach what we need before `handle` moves into the runner — the - // registry borrow lives inside `handle`. - let source = Arc::clone(&stored.source); - let query_name = stored.name.clone(); - let is_mutation = stored.is_mutation(); - - info!( - graph = %handle.uri, - actor = ?actor_ref.map(|a| a.actor_id.as_ref()), - query = %query_name, - kind = if is_mutation { "mutate" } else { "read" }, - "stored query invoked" - ); - - if is_mutation { - if req.snapshot.is_some() { - return Err(ApiError::bad_request( - "stored mutation cannot target a snapshot", - )); - } - let branch = req.branch.unwrap_or_else(|| "main".to_string()); - let output = run_mutate( - state, - handle, - actor_ref, - &source, - Some(&query_name), - req.params.as_ref(), - branch, - ) - .await?; - Ok(Json(InvokeStoredQueryResponse::Change(output))) - } else { - let (selected, target, result) = run_query( - handle, - actor_ref, - &source, - Some(&query_name), - req.params.as_ref(), - req.branch, - req.snapshot, - true, - ) - .await?; - Ok(Json(InvokeStoredQueryResponse::Read(api::read_output( - selected, &target, result, - )))) - } -} - -#[utoipa::path( - get, - path = "/queries", - tag = "queries", - operation_id = "list_queries", - responses( - (status = 200, description = "Stored-query catalog (the mcp.expose subset, with typed params)", body = QueriesCatalogOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// List the graph's exposed stored queries as a typed tool catalog. -/// -/// Returns the `mcp.expose == true` subset of the `queries:` registry, each -/// with its MCP tool name, read/mutate flag, description/instruction, and -/// typed parameters — enough for a client to register them as tools without -/// fetching `.gq` source. Read-gated; the catalog is graph-wide (branch -/// independent — `read` is authorized against `main`). **Not** Cedar-filtered -/// per query yet, so it can list a query whose `invoke_query` the caller -/// lacks (a known gap until per-query authorization lands). -async fn server_list_queries( - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, -) -> std::result::Result<Json<QueriesCatalogOutput>, ApiError> { - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::Read, - branch: Some("main".to_string()), - target_branch: None, - }, - )?; - let queries = match handle.queries.as_ref() { - Some(registry) => registry - .iter() - .filter(|q| q.expose) - .map(api::query_catalog_entry) - .collect(), - None => Vec::new(), - }; - Ok(Json(QueriesCatalogOutput { queries })) -} - -#[utoipa::path( - get, - path = "/schema", - tag = "schema", - operation_id = "getSchema", - responses( - (status = 200, description = "Current schema source", body = SchemaOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Read the current schema source. -/// -/// Returns the project's schema as a single string in `.pg` source form. -/// Useful for clients that want to introspect available types and tables -/// before constructing GQ queries. Read-only. -async fn server_schema_get( - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, -) -> std::result::Result<Json<SchemaOutput>, ApiError> { - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::Read, - branch: None, - target_branch: None, - }, - )?; - let schema_source = { - let db = &handle.engine; - db.schema_source().to_string() - }; - Ok(Json(SchemaOutput { schema_source })) -} - -#[utoipa::path( - post, - path = "/schema/apply", - tag = "mutations", - operation_id = "applySchema", - request_body = SchemaApplyRequest, - responses( - (status = 200, description = "Schema apply results", body = SchemaApplyOutput), - (status = 400, description = "Bad request", body = ErrorOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Apply a schema migration. -/// -/// Diffs `schema_source` against the current schema and applies the resulting -/// migration steps (add/drop type, add/drop column, etc.). **Destructive**: -/// some steps drop data. Returns the list of steps applied; if `applied` is -/// false the diff was unsupported and no changes were made. -async fn server_schema_apply( - State(state): State<AppState>, - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Json(request): Json<SchemaApplyRequest>, -) -> std::result::Result<Json<SchemaApplyOutput>, ApiError> { - let actor_arc = actor - .as_ref() - .map(|Extension(actor)| Arc::clone(&actor.actor_id)) - .unwrap_or_else(|| Arc::<str>::from("anonymous")); - let actor_id = actor - .as_ref() - .map(|Extension(actor)| actor.actor_id.as_ref()); - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::SchemaApply, - branch: None, - target_branch: Some("main".to_string()), - }, - )?; - let est_bytes = request.schema_source.len() as u64; - let _admission = state - .workload - .try_admit(&actor_arc, est_bytes) - .map_err(ApiError::from_workload_reject)?; - let result = { - let db = &handle.engine; - let registry = handle.queries.as_deref(); - let label = handle.key.graph_id.as_str().to_string(); - // Engine-layer policy enforcement (MR-722): pass the resolved - // actor through so apply_schema_as can call enforce() with the - // authoritative identity. With a policy installed in AppState, - // engine-side enforcement re-checks the same decision the - // HTTP-layer authorize_request just made above. PR #3 collapses - // the redundancy. - db.apply_schema_as_with_catalog_check( - &request.schema_source, - omnigraph::db::SchemaApplyOptions { - allow_data_loss: request.allow_data_loss, - }, - actor_id, - |catalog| { - if let Some(registry) = registry { - validate_registry_against_catalog(registry, catalog, &label)?; - } - Ok(()) - }, - ) - .await - .map_err(ApiError::from_omni)? - }; - Ok(Json(schema_apply_output(handle.uri.as_str(), result))) -} - -#[utoipa::path( - post, - path = "/ingest", - tag = "mutations", - operation_id = "ingest", - request_body = IngestRequest, - responses( - (status = 200, description = "Ingest results", body = IngestOutput), - (status = 400, description = "Bad request", body = ErrorOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Bulk-load NDJSON data into a branch. -/// -/// `data` is NDJSON with one record per line. `mode` controls behavior on -/// existing rows: `merge` upserts by id (default), `append` blindly inserts, -/// `overwrite` replaces table contents. Branch creation is opt-in by -/// presence of `from`: with `from` set, a missing `branch` is created from -/// it; without `from`, `branch` must already exist — a missing branch is a -/// 404, never an implicit fork. **Destructive** when `mode` is `overwrite` -/// or when the load produces conflicting writes. -async fn server_ingest( - State(state): State<AppState>, - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Json(request): Json<IngestRequest>, -) -> std::result::Result<Json<IngestOutput>, ApiError> { - let branch = request.branch.unwrap_or_else(|| "main".to_string()); - let from = request.from; - let mode = request.mode.unwrap_or(omnigraph::loader::LoadMode::Merge); - let actor_arc = actor - .as_ref() - .map(|Extension(actor)| Arc::clone(&actor.actor_id)) - .unwrap_or_else(|| Arc::<str>::from("anonymous")); - let actor_id = actor - .as_ref() - .map(|Extension(actor)| actor.actor_id.as_ref()); - - let branch_exists = { - let db = &handle.engine; - db.branch_list() - .await - .map_err(ApiError::from_omni)? - .into_iter() - .any(|name| name == branch) - }; - - if !branch_exists { - match from.as_deref() { - // Fork-if-missing is opt-in by presence of `from`; without it a - // typo'd branch name must surface as an error, not silently - // create a fork and land the data there. - None => { - return Err(ApiError::not_found(format!( - "branch '{branch}' not found; pass `from` to create it" - ))); - } - Some(from) => authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::BranchCreate, - branch: Some(from.to_string()), - target_branch: Some(branch.clone()), - }, - )?, - } - } - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::Change, - branch: Some(branch.clone()), - target_branch: None, - }, - )?; - let est_bytes = request.data.len() as u64; - let _admission = state - .workload - .try_admit(&actor_arc, est_bytes) - .map_err(ApiError::from_workload_reject)?; - - let result = { - let db = &handle.engine; - db.load_as(&branch, from.as_deref(), &request.data, mode, actor_id) - .await - .map_err(ApiError::from_omni)? - }; - - Ok(Json(ingest_output( - handle.uri.as_str(), - &result, - mode, - actor_id.map(str::to_string), - ))) -} - -#[utoipa::path( - get, - path = "/branches", - tag = "branches", - operation_id = "listBranches", - responses( - (status = 200, description = "List of branches", body = BranchListOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// List all branches. -/// -/// Returns branch names sorted alphabetically. Read-only. -async fn server_branch_list( - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, -) -> std::result::Result<Json<BranchListOutput>, ApiError> { - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::Read, - branch: None, - target_branch: None, - }, - )?; - let mut branches = { - let db = &handle.engine; - db.branch_list().await.map_err(ApiError::from_omni)? - }; - branches.sort(); - Ok(Json(BranchListOutput { branches })) -} - -#[utoipa::path( - post, - path = "/branches", - tag = "branches", - operation_id = "createBranch", - request_body = BranchCreateRequest, - responses( - (status = 200, description = "Branch created", body = BranchCreateOutput), - (status = 400, description = "Bad request", body = ErrorOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - (status = 409, description = "Branch already exists", body = ErrorOutput), - (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Create a new branch. -/// -/// Forks `name` off of `from` (defaults to `main`). The new branch shares -/// table data with its parent until it is mutated. Returns 409 if `name` -/// already exists. -async fn server_branch_create( - State(state): State<AppState>, - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Json(request): Json<BranchCreateRequest>, -) -> std::result::Result<Json<BranchCreateOutput>, ApiError> { - let from = request.from.unwrap_or_else(|| "main".to_string()); - let actor_arc = actor - .as_ref() - .map(|Extension(actor)| Arc::clone(&actor.actor_id)) - .unwrap_or_else(|| Arc::<str>::from("anonymous")); - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::BranchCreate, - branch: Some(from.clone()), - target_branch: Some(request.name.clone()), - }, - )?; - // Branch metadata only — small constant bytes estimate. The Lance - // shallow-clone work is bounded by the parent's manifest size, not - // the request body. - let _admission = state - .workload - .try_admit(&actor_arc, 256) - .map_err(ApiError::from_workload_reject)?; - { - let db = &handle.engine; - db.branch_create_from_as( - ReadTarget::branch(&from), - &request.name, - actor.as_ref().map(|Extension(a)| a.actor_id.as_ref()), - ) - .await - .map_err(ApiError::from_omni)?; - } - Ok(Json(BranchCreateOutput { - uri: handle.uri.clone(), - from, - name: request.name, - actor_id: actor.map(|Extension(actor)| actor.actor_id.as_ref().to_string()), - })) -} - -/// Path-param shape for [`server_branch_delete`]. Named-field -/// deserialization (rather than `Path<String>` or `Path<(String,)>`) -/// keeps the extractor stable across single-mode flat routes and -/// multi-mode nested routes: the `{branch}` capture is picked by -/// name and any other captures in scope (e.g. `{graph_id}` in -/// multi-mode) are ignored without breaking deserialization. -/// -/// Closes the "handler path-extractor type is positional and breaks -/// when route nesting changes" class. -#[derive(Deserialize)] -struct BranchPath { - branch: String, -} - -#[utoipa::path( - delete, - path = "/branches/{branch}", - tag = "branches", - operation_id = "deleteBranch", - params( - ("branch" = String, Path, description = "Branch name to delete"), - ), - responses( - (status = 200, description = "Branch deleted", body = BranchDeleteOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - (status = 404, description = "Branch not found", body = ErrorOutput), - (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Delete a branch. -/// -/// **Irreversible.** Removes the branch pointer; commits remain reachable -/// only if referenced by another branch. Returns 404 if the branch does not -/// exist. -async fn server_branch_delete( - State(state): State<AppState>, - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Path(BranchPath { branch }): Path<BranchPath>, -) -> std::result::Result<Json<BranchDeleteOutput>, ApiError> { - let actor_arc = actor - .as_ref() - .map(|Extension(actor)| Arc::clone(&actor.actor_id)) - .unwrap_or_else(|| Arc::<str>::from("anonymous")); - let actor_id = actor - .as_ref() - .map(|Extension(actor)| actor.actor_id.as_ref()); - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::BranchDelete, - branch: None, - target_branch: Some(branch.clone()), - }, - )?; - // Metadata-only manifest tombstone — small constant estimate. - let _admission = state - .workload - .try_admit(&actor_arc, 256) - .map_err(ApiError::from_workload_reject)?; - { - let db = &handle.engine; - db.branch_delete_as(&branch, actor_id) - .await - .map_err(ApiError::from_omni)?; - } - Ok(Json(BranchDeleteOutput { - uri: handle.uri.clone(), - name: branch, - actor_id: actor_id.map(str::to_string), - })) -} - -#[utoipa::path( - post, - path = "/branches/merge", - tag = "branches", - operation_id = "mergeBranches", - request_body = BranchMergeRequest, - responses( - (status = 200, description = "Branches merged", body = BranchMergeOutput), - (status = 400, description = "Bad request", body = ErrorOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - (status = 409, description = "Merge conflict", body = ErrorOutput), - (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Merge one branch into another. -/// -/// Merges `source` into `target` (defaults to `main`). Outcome is one of -/// `already_up_to_date`, `fast_forward`, or `merged`. Returns 409 with the -/// list of conflicts if the merge cannot be completed; the target is left -/// unchanged in that case. **Destructive** to `target` on success. -async fn server_branch_merge( - State(state): State<AppState>, - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Json(request): Json<BranchMergeRequest>, -) -> std::result::Result<Json<BranchMergeOutput>, ApiError> { - let target = request.target.unwrap_or_else(|| "main".to_string()); - let actor_arc = actor - .as_ref() - .map(|Extension(actor)| Arc::clone(&actor.actor_id)) - .unwrap_or_else(|| Arc::<str>::from("anonymous")); - let actor_id = actor - .as_ref() - .map(|Extension(actor)| actor.actor_id.as_ref()); - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::BranchMerge, - branch: Some(request.source.clone()), - target_branch: Some(target.clone()), - }, - )?; - // Merge body is small JSON; the heavy work is in the engine but is - // bounded per-(table, branch) by the writer queue. Small constant - // estimate suffices for the actor in-flight count. - let _admission = state - .workload - .try_admit(&actor_arc, 256) - .map_err(ApiError::from_workload_reject)?; - let outcome = { - let db = &handle.engine; - db.branch_merge_as(&request.source, &target, actor_id) - .await - .map_err(ApiError::from_omni)? - }; - Ok(Json(BranchMergeOutput { - source: request.source, - target, - outcome: outcome.into(), - actor_id: actor_id.map(str::to_string), - })) -} - -#[utoipa::path( - get, - path = "/commits", - tag = "commits", - operation_id = "listCommits", - params(CommitListQuery), - responses( - (status = 200, description = "List of commits", body = CommitListOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// List commits. -/// -/// Filter by `branch` to get the commits on a single branch (most recent -/// first); omit to list across all branches. Read-only. -async fn server_commit_list( - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Query(query): Query<CommitListQuery>, -) -> std::result::Result<Json<CommitListOutput>, ApiError> { - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::Read, - branch: query.branch.clone(), - target_branch: None, - }, - )?; - let commits = { - let db = &handle.engine; - db.list_commits(query.branch.as_deref()) - .await - .map_err(ApiError::from_omni)? - }; - Ok(Json(CommitListOutput { - commits: commits.iter().map(api::commit_output).collect(), - })) -} - -/// Path-param shape for [`server_commit_show`]. See [`BranchPath`] -/// for the design rationale — same pattern, different field name. -#[derive(Deserialize)] -struct CommitPath { - commit_id: String, -} - -#[utoipa::path( - get, - path = "/commits/{commit_id}", - tag = "commits", - operation_id = "getCommit", - params( - ("commit_id" = String, Path, description = "Commit identifier"), - ), - responses( - (status = 200, description = "Commit details", body = api::CommitOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - (status = 404, description = "Commit not found", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] - -/// Get a single commit. -/// -/// Returns the commit's manifest version, parent commit(s), and creation -/// metadata. Read-only. -async fn server_commit_show( - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Path(CommitPath { commit_id }): Path<CommitPath>, -) -> std::result::Result<Json<api::CommitOutput>, ApiError> { - authorize_request( - actor.as_ref().map(|Extension(actor)| actor), - handle.policy.as_deref(), - PolicyRequest { - action: PolicyAction::Read, - branch: None, - target_branch: None, - }, - )?; - let commit = { - let db = &handle.engine; - db.get_commit(&commit_id) - .await - .map_err(ApiError::from_omni)? - }; - Ok(Json(api::commit_output(&commit))) -} - -fn read_target_from_request(branch: Option<String>, snapshot: Option<String>) -> ReadTarget { - if let Some(snapshot) = snapshot { - ReadTarget::snapshot(omnigraph::db::SnapshotId::new(snapshot)) - } else { - ReadTarget::branch(branch.unwrap_or_else(|| "main".to_string())) - } -} - -fn select_named_query_decl( - query_source: &str, - requested_name: Option<&str>, -) -> Result<omnigraph_compiler::query::ast::QueryDecl> { - let parsed = parse_query(query_source)?; - let query = if let Some(name) = requested_name { - parsed - .queries - .into_iter() - .find(|query| query.name == name) - .ok_or_else(|| color_eyre::eyre::eyre!("query '{}' not found", name))? - } else if parsed.queries.len() == 1 { - parsed.queries.into_iter().next().unwrap() - } else { - bail!("query file contains multiple queries; pass --name"); - }; - Ok(query) -} - -fn select_named_query( - query_source: &str, - requested_name: Option<&str>, -) -> Result<(String, Vec<omnigraph_compiler::query::ast::Param>)> { - let query = select_named_query_decl(query_source, requested_name)?; - Ok((query.name, query.params)) -} - -fn query_params_from_json( - query_params: &[omnigraph_compiler::query::ast::Param], - params_json: Option<&Value>, -) -> Result<ParamMap> { - json_params_to_param_map(params_json, query_params, JsonParamMode::Standard) - .map_err(|err| color_eyre::eyre::eyre!(err.to_string())) -} - -fn normalize_bearer_token(value: Option<String>) -> Option<String> { - value - .map(|value| value.trim().to_string()) - .filter(|value| !value.is_empty()) -} - -fn normalize_bearer_actor(value: String) -> Result<String> { - let value = value.trim().to_string(); - if value.is_empty() { - bail!("bearer token actor names must not be blank"); - } - Ok(value) -} - -fn parse_bearer_tokens_json(value: &str) -> Result<Vec<(String, String)>> { - let entries: HashMap<String, String> = serde_json::from_str(value) - .wrap_err("OMNIGRAPH_SERVER_BEARER_TOKENS_JSON must be a JSON object of actor->token")?; - Ok(entries.into_iter().collect()) -} - -fn read_bearer_tokens_file(path: &str) -> Result<Vec<(String, String)>> { - let contents = fs::read_to_string(path) - .wrap_err_with(|| format!("failed to read bearer tokens file at {path}"))?; - parse_bearer_tokens_json(&contents) - .wrap_err_with(|| format!("failed to parse bearer tokens file at {path}")) -} - -fn validate_bearer_tokens(entries: Vec<(String, String)>) -> Result<Vec<(String, String)>> { - let mut seen_actors = HashSet::new(); - let mut seen_tokens = HashSet::new(); - let mut normalized = Vec::with_capacity(entries.len()); - - for (actor, token) in entries { - let actor = normalize_bearer_actor(actor)?; - let Some(token) = normalize_bearer_token(Some(token)) else { - bail!("bearer token for actor '{actor}' must not be blank"); - }; - if !seen_actors.insert(actor.clone()) { - bail!("duplicate bearer token actor '{actor}'"); - } - if !seen_tokens.insert(token.clone()) { - bail!("duplicate bearer token value configured"); - } - normalized.push((actor, token)); - } - - normalized.sort_by(|(left, _), (right, _)| left.cmp(right)); - Ok(normalized) -} - -fn server_bearer_tokens_from_env() -> Result<Vec<(String, String)>> { - let mut entries = Vec::new(); - - if let Some(token) = normalize_bearer_token(std::env::var("OMNIGRAPH_SERVER_BEARER_TOKEN").ok()) - { - entries.push(("default".to_string(), token)); - } - - if let Some(path) = - normalize_bearer_token(std::env::var("OMNIGRAPH_SERVER_BEARER_TOKENS_FILE").ok()) - { - entries.extend(read_bearer_tokens_file(&path)?); - } else if let Some(json) = - normalize_bearer_token(std::env::var("OMNIGRAPH_SERVER_BEARER_TOKENS_JSON").ok()) - { - entries.extend(parse_bearer_tokens_json(&json)?); - } - - validate_bearer_tokens(entries) -} - -#[cfg(test)] -mod tests { - use super::{ - GraphStartupConfig, ServerConfig, ServerConfigMode, ServerRuntimeState, - classify_server_runtime_state, hash_bearer_token, load_server_settings, - normalize_bearer_token, parse_bearer_tokens_json, serve, server_bearer_tokens_from_env, - }; - use serial_test::serial; - use std::env; - use std::fs; - use tempfile::tempdir; - - /// `authorize` returns the allow/deny **decision** (`Authz`) and reserves - /// `Err` for operational failures, so the invoke handler can hide a denial - /// as 404 without also masking a 401/500. Pins each outcome. - #[test] - fn authorize_splits_decision_from_operational_error() { - use super::{Authz, PolicyAction, PolicyCompiler, PolicyConfig, PolicyRequest, ResolvedActor, authorize}; - use std::sync::Arc; - - fn req(action: PolicyAction) -> PolicyRequest { - PolicyRequest { action, branch: None, target_branch: None } - } - let actor = ResolvedActor::cluster_static(Arc::from("act-alice")); - - // --- No policy engine installed (open / default-deny modes) --- - // A server-scoped action is denied in every no-policy state. - assert!(matches!( - authorize(Some(&actor), None, req(PolicyAction::GraphList)).unwrap(), - Authz::Denied(_) - )); - // Authenticated actor + a non-read per-graph action → default-deny. - assert!(matches!( - authorize(Some(&actor), None, req(PolicyAction::Change)).unwrap(), - Authz::Denied(_) - )); - // `read` is the one per-graph action permitted without a policy. - assert!(matches!( - authorize(Some(&actor), None, req(PolicyAction::Read)).unwrap(), - Authz::Allowed - )); - // Open mode (no actor, no policy) → allowed. - assert!(matches!( - authorize(None, None, req(PolicyAction::Read)).unwrap(), - Authz::Allowed - )); - - // --- Policy engine installed --- - let policy: PolicyConfig = serde_yaml::from_str( - "version: 1\n\ - groups:\n team: [act-alice]\n\ - rules:\n - id: team-read\n allow:\n actors: { group: team }\n actions: [read]\n branch_scope: any\n", - ) - .unwrap(); - let engine = PolicyCompiler::compile(&policy, "graph").unwrap(); - - // A matched allow rule → Allowed. - assert!(matches!( - authorize( - Some(&actor), - Some(&engine), - PolicyRequest { action: PolicyAction::Read, branch: Some("main".to_string()), target_branch: None }, - ) - .unwrap(), - Authz::Allowed - )); - // Known actor, no matching allow rule → Denied, carrying the decision message. - match authorize( - Some(&actor), - Some(&engine), - PolicyRequest { action: PolicyAction::Change, branch: Some("main".to_string()), target_branch: None }, - ) - .unwrap() - { - Authz::Denied(message) => assert!(!message.is_empty(), "a deny carries its decision message"), - Authz::Allowed => panic!("change must be denied: only read is allowed"), - } - // Policy installed but no actor → operational failure (`Err`), NOT a - // decision. This is the split that keeps a 401/500 from being masked - // as the denial's response in the invoke handler. - assert!( - authorize(None, Some(&engine), req(PolicyAction::Read)).is_err(), - "a missing actor with a policy installed is an operational error, not a deny" - ); - } - - #[test] - fn hash_bearer_token_produces_32_byte_output() { - let hash = hash_bearer_token("any-token"); - assert_eq!(hash.len(), 32); - } - - /// The single gate both open paths funnel through: it refuses a - /// schema breakage (naming the graph label + query), attaches a clean - /// registry, and collapses an empty one to `None`. Pure over its args - /// (no engine), so it covers the multi-graph path's logic too — the - /// only per-path difference is the `label`, asserted here. - #[test] - fn validate_and_attach_gates_on_schema_and_collapses_empty() { - use crate::queries::{QueryRegistry, RegistrySpec}; - use omnigraph_compiler::catalog::build_catalog; - use omnigraph_compiler::schema::parser::parse_schema; - - let schema = parse_schema("node User {\nname: String\n}\n").unwrap(); - let catalog = build_catalog(&schema).unwrap(); - let spec = |name: &str, source: &str| RegistrySpec { - name: name.to_string(), - source: source.to_string(), - expose: false, - tool_name: None, - }; - - // Empty registry → nothing attached, no error. - let empty = - super::validate_and_attach(QueryRegistry::default(), &catalog, "g").unwrap(); - assert!(empty.is_none()); - - // A query that type-checks → attached. - let ok = QueryRegistry::from_specs(vec![spec( - "find_user", - "query find_user() { match { $u: User } return { $u.name } }", - )]) - .unwrap(); - assert!(super::validate_and_attach(ok, &catalog, "g").unwrap().is_some()); - - // A query referencing a type the schema lacks → boot refusal that - // names both the graph label and the offending query. - let broken = QueryRegistry::from_specs(vec![spec( - "ghost", - "query ghost() { match { $w: Widget } return { $w.name } }", - )]) - .unwrap(); - let err = super::validate_and_attach(broken, &catalog, "graph-x").unwrap_err(); - let msg = err.to_string(); - assert!(msg.contains("graph-x"), "labels the graph: {msg}"); - assert!(msg.contains("ghost"), "names the query: {msg}"); - assert!(msg.contains("schema check"), "mentions the schema check: {msg}"); - } - - #[test] - fn hash_bearer_token_is_deterministic() { - assert_eq!( - hash_bearer_token("stable-input"), - hash_bearer_token("stable-input"), - ); - } - - #[test] - fn hash_bearer_token_differs_for_different_inputs() { - assert_ne!(hash_bearer_token("token-a"), hash_bearer_token("token-b")); - } - - #[test] - fn hash_bearer_token_matches_known_sha256_vector() { - // SHA-256("abc"). If this ever fails, the hash function was swapped. - let hash = hash_bearer_token("abc"); - let hex: String = hash.iter().map(|b| format!("{:02x}", b)).collect(); - assert_eq!( - hex, - "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad" - ); - } - - #[tokio::test] - async fn server_settings_load_from_yaml_config() { - let temp = tempdir().unwrap(); - let config = temp.path().join("omnigraph.yaml"); - fs::write( - &config, - r#" -graphs: - local: - uri: /tmp/demo.omni -server: - graph: local - bind: 0.0.0.0:9090 -"#, - ) - .unwrap(); - - let settings = load_server_settings(Some(&config), None, None, None, None, false).await.unwrap(); - match &settings.mode { - ServerConfigMode::Single { uri, graph_id, .. } => { - assert_eq!(uri, "/tmp/demo.omni"); - assert_eq!(graph_id, "local"); - } - ServerConfigMode::Multi { .. } => panic!("expected Single mode, got Multi"), - } - assert_eq!(settings.bind, "0.0.0.0:9090"); - } - - #[tokio::test] - async fn server_settings_cli_flags_override_yaml_config() { - let temp = tempdir().unwrap(); - let config = temp.path().join("omnigraph.yaml"); - fs::write( - &config, - r#" -graphs: - local: - uri: /tmp/demo.omni -server: - graph: local - bind: 127.0.0.1:8080 -"#, - ) - .unwrap(); - - let settings = load_server_settings( - Some(&config), - None, - Some("/tmp/override.omni".to_string()), - None, - Some("0.0.0.0:9999".to_string()), - false, - ) - .await - .unwrap(); - match &settings.mode { - ServerConfigMode::Single { uri, graph_id, .. } => { - assert_eq!(uri, "/tmp/override.omni"); - assert_eq!(graph_id, "/tmp/override.omni"); - } - ServerConfigMode::Multi { .. } => panic!("expected Single mode, got Multi"), - } - assert_eq!(settings.bind, "0.0.0.0:9999"); - } - - #[tokio::test] - async fn server_settings_can_resolve_named_target() { - let temp = tempdir().unwrap(); - let config = temp.path().join("omnigraph.yaml"); - fs::write( - &config, - r#" -graphs: - local: - uri: ./demo.omni - dev: - uri: http://127.0.0.1:8080 -server: - graph: local - bind: 127.0.0.1:8080 -"#, - ) - .unwrap(); - - let settings = - load_server_settings(Some(&config), None, None, Some("dev".to_string()), None, false) - .await - .unwrap(); - match &settings.mode { - ServerConfigMode::Single { uri, graph_id, .. } => { - assert_eq!(uri, "http://127.0.0.1:8080"); - assert_eq!(graph_id, "dev"); - } - ServerConfigMode::Multi { .. } => panic!("expected Single mode, got Multi"), - } - } - - #[tokio::test] - async fn server_settings_require_uri_from_cli_or_config() { - let error = load_server_settings(None, None, None, None, None, false).await.unwrap_err(); - assert!( - error.to_string().contains("no graph to serve"), - "expected mode-inference error, got: {error}", - ); - } - - #[test] - fn classify_open_requires_explicit_unauthenticated_flag() { - // State 1: no tokens, no policy, no flag → refuse to start. - let error = classify_server_runtime_state(false, false, false).unwrap_err(); - let msg = error.to_string(); - assert!( - msg.contains("--unauthenticated"), - "expected refusal message mentioning --unauthenticated, got: {msg}" - ); - - // Same matrix cell but with the flag set → Open mode permitted. - assert_eq!( - classify_server_runtime_state(false, false, true).unwrap(), - ServerRuntimeState::Open - ); - } - - #[test] - fn classify_tokens_without_policy_is_default_deny() { - // State 2: tokens configured, no policy → DefaultDeny regardless - // of the flag (the flag opts into the fully-open dev mode; it - // doesn't downgrade default-deny back to open). - assert_eq!( - classify_server_runtime_state(true, false, false).unwrap(), - ServerRuntimeState::DefaultDeny - ); - assert_eq!( - classify_server_runtime_state(true, false, true).unwrap(), - ServerRuntimeState::DefaultDeny - ); - } - - #[tokio::test] - #[serial] - async fn serve_refuses_to_start_with_policy_but_no_tokens_multi_mode() { - // Bug 2 from the bot-review pass: multi-mode startup was missing - // the "policy requires tokens" check that single-mode enforces. - // After centralizing the check in `classify_server_runtime_state`, - // both modes get the same enforcement. This test guards the - // multi-mode propagation path. - // - // Sibling test below pins single mode. Together they pin that - // the classifier is called from both branches of `serve()`. - let _guard = EnvGuard::set(&[ - ("OMNIGRAPH_SERVER_BEARER_TOKEN", None), - ("OMNIGRAPH_SERVER_BEARER_TOKENS_FILE", None), - ("OMNIGRAPH_SERVER_BEARER_TOKENS_JSON", None), - ("OMNIGRAPH_SERVER_BEARER_TOKENS_AWS_SECRET", None), - ("OMNIGRAPH_UNAUTHENTICATED", None), - ]); - let temp = tempdir().unwrap(); - // The classifier reads `has_policy_configured` from the config - // shape (does the Option contain a path?), not from file - // existence, so we can hand it a path without writing a real - // policy file — the bail fires before policy load. - let policy_path = temp.path().join("server-policy.yaml"); - let config = ServerConfig { - mode: ServerConfigMode::Multi { - graphs: vec![GraphStartupConfig { - graph_id: "alpha".to_string(), - uri: temp - .path() - .join("alpha.omni") - .to_string_lossy() - .into_owned(), - policy_file: None, - queries: crate::queries::QueryRegistry::default(), - }], - config_path: temp.path().join("omnigraph.yaml"), - server_policy_file: Some(policy_path), - }, - bind: "127.0.0.1:0".to_string(), - allow_unauthenticated: false, - }; - let result = serve(config).await; - let err = result - .expect_err("serve should refuse to start in multi mode with policy but no tokens"); - let msg = format!("{:?}", err); - assert!( - msg.contains("policy file is configured but no bearer tokens"), - "expected policy-without-tokens rejection in multi mode, got: {msg}", - ); - } - - #[tokio::test] - #[serial] - async fn serve_refuses_to_start_in_state_1_without_unauthenticated() { - // MR-723 PR A: pin the integration boundary that the classifier - // is actually called by `serve()` before any side-effecting - // work (Lance dataset open, TcpListener::bind). The classifier - // itself is unit-tested above; this test guards the propagation - // path from `classify_server_runtime_state` through serve's - // `?` so a future refactor that drops the call returns red. - // - // Marked `#[serial]` because we have to clear all bearer-token - // env vars, and another test in this module setting any of them - // concurrently would corrupt the read inside `resolve_token_source`. - let _guard = EnvGuard::set(&[ - ("OMNIGRAPH_SERVER_BEARER_TOKEN", None), - ("OMNIGRAPH_SERVER_BEARER_TOKENS_FILE", None), - ("OMNIGRAPH_SERVER_BEARER_TOKENS_JSON", None), - ("OMNIGRAPH_SERVER_BEARER_TOKENS_AWS_SECRET", None), - ("OMNIGRAPH_UNAUTHENTICATED", None), - ]); - let temp = tempdir().unwrap(); - // Graph path doesn't need to exist — classifier fires before - // `AppState::open_with_bearer_tokens_and_policy`. - let config = ServerConfig { - mode: ServerConfigMode::Single { - uri: temp - .path() - .join("graph.omni") - .to_string_lossy() - .into_owned(), - graph_id: "default".to_string(), - policy_file: None, - queries: crate::queries::QueryRegistry::default(), - }, - bind: "127.0.0.1:0".to_string(), - allow_unauthenticated: false, - }; - let result = serve(config).await; - let err = - result.expect_err("serve should refuse to start in State 1 without --unauthenticated"); - let msg = format!("{:?}", err); - assert!( - msg.contains("no bearer tokens") || msg.contains("policy file"), - "expected refusal message naming the misconfiguration, got: {msg}", - ); - } - - #[tokio::test] - #[serial] - async fn unauthenticated_env_var_classification() { - // MR-723 PR A: closes the gap where the env-var read path inside - // `load_server_settings` was structurally implemented but not - // exercised by any test. Three properties to pin, all in one - // sequential test because `cargo test` runs the mod test suite - // in parallel and `OMNIGRAPH_UNAUTHENTICATED` is process-global - // — interleaving with another test that sets the same env var - // (concurrent classifier tests, even the bearer-token suite - // sharing `EnvGuard`) corrupts the read. Sequential within one - // test fn is the simplest race-free shape. - let temp = tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -graphs: - local: - uri: /tmp/demo-unauth.omni -server: - graph: local -"#, - ) - .unwrap(); - - // Truthy values flip Open mode on, even with CLI flag off. - for value in ["1", "true", "yes", "TRUE", "anything"] { - let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some(value))]); - let settings = load_server_settings(Some(&config_path), None, None, None, None, false).await - .expect("settings load should succeed"); - assert!( - settings.allow_unauthenticated, - "OMNIGRAPH_UNAUTHENTICATED={value:?} should enable Open mode", - ); - } - - // Falsy values keep refusal behavior, even with CLI flag off. - for value in ["0", "false", "FALSE", ""] { - let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some(value))]); - let settings = load_server_settings(Some(&config_path), None, None, None, None, false).await - .expect("settings load should succeed"); - assert!( - !settings.allow_unauthenticated, - "OMNIGRAPH_UNAUTHENTICATED={value:?} should NOT enable Open mode", - ); - } - - // Unset env var: also false. - let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", None)]); - let settings = load_server_settings(Some(&config_path), None, None, None, None, false).await - .expect("settings load should succeed"); - assert!( - !settings.allow_unauthenticated, - "OMNIGRAPH_UNAUTHENTICATED unset should NOT enable Open mode", - ); - drop(_guard); - - // CLI flag wins even when env is falsy — `serve()` honors the - // OR of both inputs. - let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some("0"))]); - let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await - .expect("settings load should succeed"); - assert!( - settings.allow_unauthenticated, - "--unauthenticated CLI flag should win even when env is falsy", - ); - } - - #[test] - fn classify_policy_enabled_requires_tokens() { - // State 3: tokens + policy → PolicyEnabled, regardless of the - // `allow_unauthenticated` flag (Cedar evaluates the bearer, - // the flag is moot once tokens exist). - assert_eq!( - classify_server_runtime_state(true, true, false).unwrap(), - ServerRuntimeState::PolicyEnabled - ); - assert_eq!( - classify_server_runtime_state(true, true, true).unwrap(), - ServerRuntimeState::PolicyEnabled - ); - } - - #[test] - fn classify_policy_without_tokens_is_rejected() { - // Closes the "policy installed but no tokens → silent 401 on - // every request" footgun. The same shape that single-mode - // `open_with_bearer_tokens_and_policy` used to bail on - // privately is now rejected by the classifier so both single - // and multi mode get the same enforcement from one source of - // truth. - for allow_unauthenticated in [false, true] { - let err = - classify_server_runtime_state(false, true, allow_unauthenticated).unwrap_err(); - let msg = err.to_string(); - assert!( - msg.contains("policy file is configured but no bearer tokens"), - "expected policy-without-tokens rejection message; got: {msg}" - ); - assert!( - msg.contains("every request would 401"), - "rejection message must name the failure mode; got: {msg}" - ); - } - } - - #[test] - fn normalize_bearer_token_trims_and_filters_blank_values() { - assert_eq!(normalize_bearer_token(None), None); - assert_eq!(normalize_bearer_token(Some(" ".to_string())), None); - assert_eq!( - normalize_bearer_token(Some(" demo-token ".to_string())).as_deref(), - Some("demo-token") - ); - } - - struct EnvGuard { - saved: Vec<(&'static str, Option<String>)>, - } - - impl EnvGuard { - fn set(vars: &[(&'static str, Option<&str>)]) -> Self { - let saved = vars - .iter() - .map(|(name, _)| (*name, env::var(name).ok())) - .collect::<Vec<_>>(); - for (name, value) in vars { - unsafe { - match value { - Some(value) => env::set_var(name, value), - None => env::remove_var(name), - } - } - } - Self { saved } - } - } - - impl Drop for EnvGuard { - fn drop(&mut self) { - for (name, value) in self.saved.drain(..) { - unsafe { - match value { - Some(value) => env::set_var(name, value), - None => env::remove_var(name), - } - } - } - } - } - - #[test] - fn parse_bearer_tokens_json_reads_actor_token_map() { - let tokens = parse_bearer_tokens_json(r#"{"alice":" token-a ","bob":"token-b"}"#).unwrap(); - assert_eq!(tokens.len(), 2); - assert!(tokens.contains(&("alice".to_string(), " token-a ".to_string()))); - assert!(tokens.contains(&("bob".to_string(), "token-b".to_string()))); - } - - #[test] - #[serial] - fn server_bearer_tokens_from_env_reads_legacy_token_and_token_file() { - let temp = tempdir().unwrap(); - let tokens_path = temp.path().join("tokens.json"); - fs::write( - &tokens_path, - r#"{"team-01":"token-one","team-02":"token-two"}"#, - ) - .unwrap(); - - let _guard = EnvGuard::set(&[ - ("OMNIGRAPH_SERVER_BEARER_TOKEN", Some(" legacy-token ")), - ( - "OMNIGRAPH_SERVER_BEARER_TOKENS_FILE", - Some(tokens_path.to_str().unwrap()), - ), - ("OMNIGRAPH_SERVER_BEARER_TOKENS_JSON", None), - ]); - - let tokens = server_bearer_tokens_from_env().unwrap(); - assert_eq!( - tokens, - vec![ - ("default".to_string(), "legacy-token".to_string()), - ("team-01".to_string(), "token-one".to_string()), - ("team-02".to_string(), "token-two".to_string()), - ] - ); - } -} diff --git a/crates/omnigraph-server/src/settings.rs b/crates/omnigraph-server/src/settings.rs new file mode 100644 index 0000000..6531c3a --- /dev/null +++ b/crates/omnigraph-server/src/settings.rs @@ -0,0 +1,988 @@ +//! Server settings: omnigraph.yaml/CLI/env resolution, mode inference +//! (single vs multi vs cluster), bearer-token sources, and runtime-state +//! classification (moved verbatim from lib.rs in the modularization). + +use super::*; + +/// Build serving settings from a cluster directory's applied revision +/// (RFC-005 §D2): graphs at derived roots, stored queries from verified +/// catalog blob content, policy bundles from blob paths with their applied +/// bindings. Always multi-graph routing. The unauthenticated/env handling +/// matches the omnigraph.yaml path. +pub(crate) async fn load_cluster_settings( + cluster_dir: &PathBuf, + cli_bind: Option<String>, + cli_allow_unauthenticated: bool, +) -> Result<ServerConfig> { + let snapshot = omnigraph_cluster::read_serving_snapshot(cluster_dir).await.map_err(|diagnostics| { + let details = diagnostics + .iter() + .map(|diagnostic| format!("[{}] {}: {}", diagnostic.code, diagnostic.path, diagnostic.message)) + .collect::<Vec<_>>() + .join("\n "); + eyre!("the cluster at '{}' is not ready to serve:\n {details}", cluster_dir.display()) + })?; + + // Bindings -> Cedar slots. The serving pipeline loads one bundle per + // graph plus one server-level bundle; stacked bundles per scope are a + // later slice — refuse loudly rather than silently merging policy. + let mut server_policy_file: Option<PathBuf> = None; + let mut graph_policy_files: BTreeMap<String, PathBuf> = BTreeMap::new(); + for policy in &snapshot.policies { + for binding in &policy.applies_to { + if binding == "cluster" { + if server_policy_file.replace(policy.blob_path.clone()).is_some() { + bail!( + "multiple policy bundles bind the cluster scope; cluster-mode serving supports one bundle per scope — split or merge bundles (multi-bundle scopes are a later slice)" + ); + } + } else if let Some(graph_id) = binding.strip_prefix("graph.") { + if graph_policy_files + .insert(graph_id.to_string(), policy.blob_path.clone()) + .is_some() + { + bail!( + "multiple policy bundles bind graph '{graph_id}'; cluster-mode serving supports one bundle per scope — split or merge bundles (multi-bundle scopes are a later slice)" + ); + } + } else { + bail!("unrecognized policy binding '{binding}' in the applied revision"); + } + } + } + + let mut graphs = Vec::new(); + for graph in &snapshot.graphs { + let specs: Vec<queries::RegistrySpec> = snapshot + .queries + .iter() + .filter(|query| query.graph_id == graph.graph_id) + .map(|query| queries::RegistrySpec { + name: query.name.clone(), + source: query.source.clone(), + // The §D5 bridge: the cluster registry has no expose flag + // (exposure becomes a policy decision in Phase 6) — cluster + // mode lists every stored query. + expose: true, + tool_name: None, + }) + .collect(); + let registry = QueryRegistry::from_specs(specs).map_err(|errors| { + let details = errors + .iter() + .map(|error| error.to_string()) + .collect::<Vec<_>>() + .join("\n "); + eyre!( + "stored queries in the applied revision failed to parse:\n {details}\nrun `cluster refresh` then `cluster apply`, and restart" + ) + })?; + graphs.push(GraphStartupConfig { + graph_id: graph.graph_id.clone(), + uri: graph.root.to_string_lossy().to_string(), + policy_file: graph_policy_files.get(&graph.graph_id).cloned(), + queries: registry, + }); + } + + let env_unauth = std::env::var("OMNIGRAPH_UNAUTHENTICATED") + .ok() + .map(|v| { + let trimmed = v.trim(); + !trimmed.is_empty() && trimmed != "0" && !trimmed.eq_ignore_ascii_case("false") + }) + .unwrap_or(false); + + Ok(ServerConfig { + mode: ServerConfigMode::Multi { + graphs, + config_path: cluster_dir.clone(), + server_policy_file, + }, + bind: cli_bind.unwrap_or_else(|| "127.0.0.1:8080".to_string()), + allow_unauthenticated: cli_allow_unauthenticated || env_unauth, + }) +} + +pub async fn load_server_settings( + config_path: Option<&PathBuf>, + cli_cluster: Option<&PathBuf>, + cli_uri: Option<String>, + cli_target: Option<String>, + cli_bind: Option<String>, + cli_allow_unauthenticated: bool, +) -> Result<ServerConfig> { + // Rule 0 (RFC-005): --cluster is an exclusive boot source. It is checked + // before anything reads omnigraph.yaml — in cluster mode that file is + // never opened, not even the implicit current-directory search. + if let Some(cluster_dir) = cli_cluster { + if cli_uri.is_some() || cli_target.is_some() || config_path.is_some() { + bail!( + "--cluster is an exclusive boot source; it cannot combine with a graph URI, --target, or --config (axiom 15: a deployment serves from one source)" + ); + } + return load_cluster_settings(cluster_dir, cli_bind, cli_allow_unauthenticated).await; + } + let config = load_config(config_path)?; + let bind = cli_bind.unwrap_or_else(|| config.server_bind().to_string()); + // Either `--unauthenticated` or `OMNIGRAPH_UNAUTHENTICATED=1` flips + // this. Treat any non-empty, non-"0"/"false" string as truthy — + // standard 12-factor "any value is true" reading of the env var. + let env_unauth = std::env::var("OMNIGRAPH_UNAUTHENTICATED") + .ok() + .map(|v| { + let trimmed = v.trim(); + !trimmed.is_empty() && trimmed != "0" && !trimmed.eq_ignore_ascii_case("false") + }) + .unwrap_or(false); + let allow_unauthenticated = cli_allow_unauthenticated || env_unauth; + + // MR-668 decision 2 — four-rule mode inference matrix. + // + // 1. CLI `<URI>` positional → Single (URI = the value) + // 2. CLI `--target <name>` → Single (URI = graphs.<name>.uri) + // 3. `server.graph` in config → Single (URI = graphs.<server.graph>.uri) + // 4. `--config` + non-empty `graphs:` + no single-mode selector + // → Multi (every entry in `graphs:`) + // 5. otherwise → error with migration hint + // + // Rules 1-3 are mutually compatible (CLI URI wins over `--target` + // wins over `server.graph`), reusing the existing + // `resolve_target_uri` precedence. + let has_cli_uri = cli_uri.is_some(); + let has_cli_target = cli_target.is_some(); + let has_server_graph = config.server_graph_name().is_some(); + let has_graphs_map = !config.graphs.is_empty(); + let has_explicit_config = config_path.is_some(); + + let mode = if has_cli_uri || has_cli_target || has_server_graph { + // Rules 1, 2, or 3 → Single mode. + let raw_uri = config.resolve_target_uri( + cli_uri, + cli_target.as_deref(), + config.server_graph_name(), + )?; + let uri = normalize_root_uri(&raw_uri).wrap_err_with(|| { + format!("normalize single-graph URI '{raw_uri}' from server settings") + })?; + // Config follows graph IDENTITY, not mode: a bare URI is anonymous + // (top-level config); a graph chosen by name uses its per-graph + // `graphs.<name>.{policy,queries}`. `resolve_target_uri` already + // errored on an unknown name, so a `Some(name)` here is a known graph. + let selected: Option<&str> = if has_cli_uri { + None + } else { + cli_target.as_deref().or_else(|| config.server_graph_name()) + }; + // A named selection must not leave a populated top-level block + // silently unused — refuse boot and point at the per-graph block. The + // same rule the CLI selection gate enforces, shared via one helper so + // the boot check and `omnigraph queries validate`/`list` can't drift. + config.ensure_top_level_blocks_honored(selected)?; + // Load + identity-check now (no engine needed); the schema + // type-check happens when the engine opens. + let policy_file = config.resolve_policy_file_for(selected); + let queries = QueryRegistry::load(&config, config.query_entries_for(selected)) + .map_err(|errs| color_eyre::eyre::eyre!(format_registry_load_errors(&uri, &errs)))?; + let graph_id = graph_resource_id_for_selection(selected, &uri); + ServerConfigMode::Single { + uri, + graph_id, + policy_file, + queries, + } + } else if has_explicit_config && has_graphs_map { + // Multi mode: every graph uses its per-graph block; top-level + // policy/queries are never honored, so a populated one is an error. + let unhonored = config.populated_top_level_blocks(); + if !unhonored.is_empty() { + bail!( + "multi-graph mode: top-level {} {} not honored — each graph uses its own \ + `graphs.<graph_id>.…` block. Move per-graph rules there (and any \ + `graph_list` policy to `server.policy.file`).", + unhonored.join(" and "), + if unhonored.len() == 1 { "is" } else { "are" }, + ); + } + // Rule 4 → Multi mode. Build a startup config per graph. + let mut graphs = Vec::with_capacity(config.graphs.len()); + for (name, target) in &config.graphs { + // Validate the graph id can construct a `GraphId` newtype. + // Doing this here (not at registry insert) so a malformed + // omnigraph.yaml fails at startup with a clear error. + GraphId::try_from(name.clone()).map_err(|err| { + color_eyre::eyre::eyre!("invalid graph id '{name}' in omnigraph.yaml: {err}") + })?; + let raw_uri = config.resolve_uri_value(&target.uri); + let uri = normalize_root_uri(&raw_uri).wrap_err_with(|| { + format!("normalize URI '{raw_uri}' for graph '{name}' in omnigraph.yaml") + })?; + // Per-graph `queries:`, selected through the shared + // `query_entries_for` so server and CLI resolve identically. + // Load + identity-check now; the schema type-check happens + // when this graph's engine opens. + let queries = QueryRegistry::load(&config, config.query_entries_for(Some(name.as_str()))) + .map_err(|errs| color_eyre::eyre::eyre!(format_registry_load_errors(name, &errs)))?; + graphs.push(GraphStartupConfig { + graph_id: name.clone(), + uri, + policy_file: config.resolve_target_policy_file(name), + queries, + }); + } + let config_path = config_path + .cloned() + .expect("has_explicit_config implies config_path is Some"); + let server_policy_file = config.resolve_server_policy_file(); + ServerConfigMode::Multi { + graphs, + config_path, + server_policy_file, + } + } else { + // Rule 5 → error with migration hint. + bail!( + "no graph to serve: pass a URI (`omnigraph-server <URI>`), select a target \ + (`--target <name> --config omnigraph.yaml`), set `server.graph: <name>` in \ + omnigraph.yaml, or for multi-graph mode add a `graphs:` map to the config \ + file referenced by `--config`." + ); + }; + + Ok(ServerConfig { + mode, + bind, + allow_unauthenticated, + }) +} + +/// Whether the loaded config will run the server in multi-graph mode. +/// Useful for the test that constructs `ServerConfig` directly. +pub fn server_config_is_multi(config: &ServerConfig) -> bool { + matches!(config.mode, ServerConfigMode::Multi { .. }) +} + +/// MR-723 server runtime state, classified from the three-state matrix +/// of (bearer tokens configured) × (policy file configured) at startup. +/// +/// * **Open** — neither tokens nor policy; requires explicit +/// `allow_unauthenticated`. Effectively a "trust the network" dev +/// mode. `serve()` refuses to start in this shape without the flag, +/// so the only way to reach this state at runtime is via deliberate +/// operator opt-in. +/// * **DefaultDeny** — tokens configured but no policy file. The +/// server requires a valid bearer token; once authenticated, every +/// action except `Read` is denied with 403. Closes the "tokens but +/// forgot the policy file" trap. +/// * **PolicyEnabled** — policy file configured and at least one +/// bearer token configured. Cedar evaluates every authenticated +/// request. Policy without tokens is rejected at startup — +/// such a server would 401 every request, which is bug-shaped +/// rather than feature-shaped (operators wanting "deny all +/// unauthenticated traffic" should configure tokens plus a +/// deny-all policy to get meaningful 403s with policy-decision +/// logging instead). +#[derive(Debug, Clone, Copy, Eq, PartialEq)] +pub enum ServerRuntimeState { + Open, + DefaultDeny, + PolicyEnabled, +} + +/// Compute the [`ServerRuntimeState`] from the configured inputs. +/// Pulled out as a pure function so the matrix is unit-testable +/// without standing up the full server. +/// +/// The classifier is the **single source of truth** for "should we +/// start?" — both `serve()`'s single-mode and multi-mode branches +/// call this before constructing their `AppState`. Adding a startup +/// invariant here means both modes enforce it automatically; the +/// alternative (per-constructor `bail!`) drifts the moment a third +/// mode is added. +pub fn classify_server_runtime_state( + has_tokens: bool, + has_policy: bool, + allow_unauthenticated: bool, +) -> Result<ServerRuntimeState> { + match (has_tokens, has_policy, allow_unauthenticated) { + (false, false, false) => bail!( + "server has no bearer tokens and no policy file configured. This is a fully \ + open server — pass `--unauthenticated` (or set OMNIGRAPH_UNAUTHENTICATED=1) \ + if you actually want that, otherwise configure bearer tokens (see \ + docs/user/server.md) and/or `policy.file` in omnigraph.yaml." + ), + (false, false, true) => Ok(ServerRuntimeState::Open), + (true, false, _) => Ok(ServerRuntimeState::DefaultDeny), + (false, true, _) => bail!( + "policy file is configured but no bearer tokens — every request would 401 \ + because no token can ever match. Configure at least one bearer token (see \ + docs/user/server.md), or remove the policy file. To deny all unauthenticated \ + traffic deliberately, configure tokens plus a deny-all Cedar rule — that \ + produces meaningful 403s with policy-decision logging instead of silent 401s." + ), + (true, true, _) => Ok(ServerRuntimeState::PolicyEnabled), + } +} + +pub(crate) fn normalize_bearer_token(value: Option<String>) -> Option<String> { + value + .map(|value| value.trim().to_string()) + .filter(|value| !value.is_empty()) +} + +pub(crate) fn normalize_bearer_actor(value: String) -> Result<String> { + let value = value.trim().to_string(); + if value.is_empty() { + bail!("bearer token actor names must not be blank"); + } + Ok(value) +} + +pub(crate) fn parse_bearer_tokens_json(value: &str) -> Result<Vec<(String, String)>> { + let entries: HashMap<String, String> = serde_json::from_str(value) + .wrap_err("OMNIGRAPH_SERVER_BEARER_TOKENS_JSON must be a JSON object of actor->token")?; + Ok(entries.into_iter().collect()) +} + +pub(crate) fn read_bearer_tokens_file(path: &str) -> Result<Vec<(String, String)>> { + let contents = fs::read_to_string(path) + .wrap_err_with(|| format!("failed to read bearer tokens file at {path}"))?; + parse_bearer_tokens_json(&contents) + .wrap_err_with(|| format!("failed to parse bearer tokens file at {path}")) +} + +pub(crate) fn validate_bearer_tokens(entries: Vec<(String, String)>) -> Result<Vec<(String, String)>> { + let mut seen_actors = HashSet::new(); + let mut seen_tokens = HashSet::new(); + let mut normalized = Vec::with_capacity(entries.len()); + + for (actor, token) in entries { + let actor = normalize_bearer_actor(actor)?; + let Some(token) = normalize_bearer_token(Some(token)) else { + bail!("bearer token for actor '{actor}' must not be blank"); + }; + if !seen_actors.insert(actor.clone()) { + bail!("duplicate bearer token actor '{actor}'"); + } + if !seen_tokens.insert(token.clone()) { + bail!("duplicate bearer token value configured"); + } + normalized.push((actor, token)); + } + + normalized.sort_by(|(left, _), (right, _)| left.cmp(right)); + Ok(normalized) +} + +pub(crate) fn server_bearer_tokens_from_env() -> Result<Vec<(String, String)>> { + let mut entries = Vec::new(); + + if let Some(token) = normalize_bearer_token(std::env::var("OMNIGRAPH_SERVER_BEARER_TOKEN").ok()) + { + entries.push(("default".to_string(), token)); + } + + if let Some(path) = + normalize_bearer_token(std::env::var("OMNIGRAPH_SERVER_BEARER_TOKENS_FILE").ok()) + { + entries.extend(read_bearer_tokens_file(&path)?); + } else if let Some(json) = + normalize_bearer_token(std::env::var("OMNIGRAPH_SERVER_BEARER_TOKENS_JSON").ok()) + { + entries.extend(parse_bearer_tokens_json(&json)?); + } + + validate_bearer_tokens(entries) +} + +#[cfg(test)] +mod tests { + use super::{ + GraphStartupConfig, ServerConfig, ServerConfigMode, ServerRuntimeState, + classify_server_runtime_state, hash_bearer_token, load_server_settings, + normalize_bearer_token, parse_bearer_tokens_json, serve, server_bearer_tokens_from_env, + }; + use serial_test::serial; + use std::env; + use std::fs; + use tempfile::tempdir; + + /// `authorize` returns the allow/deny **decision** (`Authz`) and reserves + /// `Err` for operational failures, so the invoke handler can hide a denial + /// as 404 without also masking a 401/500. Pins each outcome. + #[test] + fn authorize_splits_decision_from_operational_error() { + use super::{Authz, PolicyAction, PolicyCompiler, PolicyConfig, PolicyRequest, ResolvedActor, authorize}; + use std::sync::Arc; + + fn req(action: PolicyAction) -> PolicyRequest { + PolicyRequest { action, branch: None, target_branch: None } + } + let actor = ResolvedActor::cluster_static(Arc::from("act-alice")); + + // --- No policy engine installed (open / default-deny modes) --- + // A server-scoped action is denied in every no-policy state. + assert!(matches!( + authorize(Some(&actor), None, req(PolicyAction::GraphList)).unwrap(), + Authz::Denied(_) + )); + // Authenticated actor + a non-read per-graph action → default-deny. + assert!(matches!( + authorize(Some(&actor), None, req(PolicyAction::Change)).unwrap(), + Authz::Denied(_) + )); + // `read` is the one per-graph action permitted without a policy. + assert!(matches!( + authorize(Some(&actor), None, req(PolicyAction::Read)).unwrap(), + Authz::Allowed + )); + // Open mode (no actor, no policy) → allowed. + assert!(matches!( + authorize(None, None, req(PolicyAction::Read)).unwrap(), + Authz::Allowed + )); + + // --- Policy engine installed --- + let policy: PolicyConfig = serde_yaml::from_str( + "version: 1\n\ + groups:\n team: [act-alice]\n\ + rules:\n - id: team-read\n allow:\n actors: { group: team }\n actions: [read]\n branch_scope: any\n", + ) + .unwrap(); + let engine = PolicyCompiler::compile(&policy, "graph").unwrap(); + + // A matched allow rule → Allowed. + assert!(matches!( + authorize( + Some(&actor), + Some(&engine), + PolicyRequest { action: PolicyAction::Read, branch: Some("main".to_string()), target_branch: None }, + ) + .unwrap(), + Authz::Allowed + )); + // Known actor, no matching allow rule → Denied, carrying the decision message. + match authorize( + Some(&actor), + Some(&engine), + PolicyRequest { action: PolicyAction::Change, branch: Some("main".to_string()), target_branch: None }, + ) + .unwrap() + { + Authz::Denied(message) => assert!(!message.is_empty(), "a deny carries its decision message"), + Authz::Allowed => panic!("change must be denied: only read is allowed"), + } + // Policy installed but no actor → operational failure (`Err`), NOT a + // decision. This is the split that keeps a 401/500 from being masked + // as the denial's response in the invoke handler. + assert!( + authorize(None, Some(&engine), req(PolicyAction::Read)).is_err(), + "a missing actor with a policy installed is an operational error, not a deny" + ); + } + + #[test] + fn hash_bearer_token_produces_32_byte_output() { + let hash = hash_bearer_token("any-token"); + assert_eq!(hash.len(), 32); + } + + /// The single gate both open paths funnel through: it refuses a + /// schema breakage (naming the graph label + query), attaches a clean + /// registry, and collapses an empty one to `None`. Pure over its args + /// (no engine), so it covers the multi-graph path's logic too — the + /// only per-path difference is the `label`, asserted here. + #[test] + fn validate_and_attach_gates_on_schema_and_collapses_empty() { + use crate::queries::{QueryRegistry, RegistrySpec}; + use omnigraph_compiler::catalog::build_catalog; + use omnigraph_compiler::schema::parser::parse_schema; + + let schema = parse_schema("node User {\nname: String\n}\n").unwrap(); + let catalog = build_catalog(&schema).unwrap(); + let spec = |name: &str, source: &str| RegistrySpec { + name: name.to_string(), + source: source.to_string(), + expose: false, + tool_name: None, + }; + + // Empty registry → nothing attached, no error. + let empty = + super::validate_and_attach(QueryRegistry::default(), &catalog, "g").unwrap(); + assert!(empty.is_none()); + + // A query that type-checks → attached. + let ok = QueryRegistry::from_specs(vec![spec( + "find_user", + "query find_user() { match { $u: User } return { $u.name } }", + )]) + .unwrap(); + assert!(super::validate_and_attach(ok, &catalog, "g").unwrap().is_some()); + + // A query referencing a type the schema lacks → boot refusal that + // names both the graph label and the offending query. + let broken = QueryRegistry::from_specs(vec![spec( + "ghost", + "query ghost() { match { $w: Widget } return { $w.name } }", + )]) + .unwrap(); + let err = super::validate_and_attach(broken, &catalog, "graph-x").unwrap_err(); + let msg = err.to_string(); + assert!(msg.contains("graph-x"), "labels the graph: {msg}"); + assert!(msg.contains("ghost"), "names the query: {msg}"); + assert!(msg.contains("schema check"), "mentions the schema check: {msg}"); + } + + #[test] + fn hash_bearer_token_is_deterministic() { + assert_eq!( + hash_bearer_token("stable-input"), + hash_bearer_token("stable-input"), + ); + } + + #[test] + fn hash_bearer_token_differs_for_different_inputs() { + assert_ne!(hash_bearer_token("token-a"), hash_bearer_token("token-b")); + } + + #[test] + fn hash_bearer_token_matches_known_sha256_vector() { + // SHA-256("abc"). If this ever fails, the hash function was swapped. + let hash = hash_bearer_token("abc"); + let hex: String = hash.iter().map(|b| format!("{:02x}", b)).collect(); + assert_eq!( + hex, + "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad" + ); + } + + #[tokio::test] + async fn server_settings_load_from_yaml_config() { + let temp = tempdir().unwrap(); + let config = temp.path().join("omnigraph.yaml"); + fs::write( + &config, + r#" +graphs: + local: + uri: /tmp/demo.omni +server: + graph: local + bind: 0.0.0.0:9090 +"#, + ) + .unwrap(); + + let settings = load_server_settings(Some(&config), None, None, None, None, false).await.unwrap(); + match &settings.mode { + ServerConfigMode::Single { uri, graph_id, .. } => { + assert_eq!(uri, "/tmp/demo.omni"); + assert_eq!(graph_id, "local"); + } + ServerConfigMode::Multi { .. } => panic!("expected Single mode, got Multi"), + } + assert_eq!(settings.bind, "0.0.0.0:9090"); + } + + #[tokio::test] + async fn server_settings_cli_flags_override_yaml_config() { + let temp = tempdir().unwrap(); + let config = temp.path().join("omnigraph.yaml"); + fs::write( + &config, + r#" +graphs: + local: + uri: /tmp/demo.omni +server: + graph: local + bind: 127.0.0.1:8080 +"#, + ) + .unwrap(); + + let settings = load_server_settings( + Some(&config), + None, + Some("/tmp/override.omni".to_string()), + None, + Some("0.0.0.0:9999".to_string()), + false, + ) + .await + .unwrap(); + match &settings.mode { + ServerConfigMode::Single { uri, graph_id, .. } => { + assert_eq!(uri, "/tmp/override.omni"); + assert_eq!(graph_id, "/tmp/override.omni"); + } + ServerConfigMode::Multi { .. } => panic!("expected Single mode, got Multi"), + } + assert_eq!(settings.bind, "0.0.0.0:9999"); + } + + #[tokio::test] + async fn server_settings_can_resolve_named_target() { + let temp = tempdir().unwrap(); + let config = temp.path().join("omnigraph.yaml"); + fs::write( + &config, + r#" +graphs: + local: + uri: ./demo.omni + dev: + uri: http://127.0.0.1:8080 +server: + graph: local + bind: 127.0.0.1:8080 +"#, + ) + .unwrap(); + + let settings = + load_server_settings(Some(&config), None, None, Some("dev".to_string()), None, false) + .await + .unwrap(); + match &settings.mode { + ServerConfigMode::Single { uri, graph_id, .. } => { + assert_eq!(uri, "http://127.0.0.1:8080"); + assert_eq!(graph_id, "dev"); + } + ServerConfigMode::Multi { .. } => panic!("expected Single mode, got Multi"), + } + } + + #[tokio::test] + async fn server_settings_require_uri_from_cli_or_config() { + let error = load_server_settings(None, None, None, None, None, false).await.unwrap_err(); + assert!( + error.to_string().contains("no graph to serve"), + "expected mode-inference error, got: {error}", + ); + } + + #[test] + fn classify_open_requires_explicit_unauthenticated_flag() { + // State 1: no tokens, no policy, no flag → refuse to start. + let error = classify_server_runtime_state(false, false, false).unwrap_err(); + let msg = error.to_string(); + assert!( + msg.contains("--unauthenticated"), + "expected refusal message mentioning --unauthenticated, got: {msg}" + ); + + // Same matrix cell but with the flag set → Open mode permitted. + assert_eq!( + classify_server_runtime_state(false, false, true).unwrap(), + ServerRuntimeState::Open + ); + } + + #[test] + fn classify_tokens_without_policy_is_default_deny() { + // State 2: tokens configured, no policy → DefaultDeny regardless + // of the flag (the flag opts into the fully-open dev mode; it + // doesn't downgrade default-deny back to open). + assert_eq!( + classify_server_runtime_state(true, false, false).unwrap(), + ServerRuntimeState::DefaultDeny + ); + assert_eq!( + classify_server_runtime_state(true, false, true).unwrap(), + ServerRuntimeState::DefaultDeny + ); + } + + #[tokio::test] + #[serial] + async fn serve_refuses_to_start_with_policy_but_no_tokens_multi_mode() { + // Bug 2 from the bot-review pass: multi-mode startup was missing + // the "policy requires tokens" check that single-mode enforces. + // After centralizing the check in `classify_server_runtime_state`, + // both modes get the same enforcement. This test guards the + // multi-mode propagation path. + // + // Sibling test below pins single mode. Together they pin that + // the classifier is called from both branches of `serve()`. + let _guard = EnvGuard::set(&[ + ("OMNIGRAPH_SERVER_BEARER_TOKEN", None), + ("OMNIGRAPH_SERVER_BEARER_TOKENS_FILE", None), + ("OMNIGRAPH_SERVER_BEARER_TOKENS_JSON", None), + ("OMNIGRAPH_SERVER_BEARER_TOKENS_AWS_SECRET", None), + ("OMNIGRAPH_UNAUTHENTICATED", None), + ]); + let temp = tempdir().unwrap(); + // The classifier reads `has_policy_configured` from the config + // shape (does the Option contain a path?), not from file + // existence, so we can hand it a path without writing a real + // policy file — the bail fires before policy load. + let policy_path = temp.path().join("server-policy.yaml"); + let config = ServerConfig { + mode: ServerConfigMode::Multi { + graphs: vec![GraphStartupConfig { + graph_id: "alpha".to_string(), + uri: temp + .path() + .join("alpha.omni") + .to_string_lossy() + .into_owned(), + policy_file: None, + queries: crate::queries::QueryRegistry::default(), + }], + config_path: temp.path().join("omnigraph.yaml"), + server_policy_file: Some(policy_path), + }, + bind: "127.0.0.1:0".to_string(), + allow_unauthenticated: false, + }; + let result = serve(config).await; + let err = result + .expect_err("serve should refuse to start in multi mode with policy but no tokens"); + let msg = format!("{:?}", err); + assert!( + msg.contains("policy file is configured but no bearer tokens"), + "expected policy-without-tokens rejection in multi mode, got: {msg}", + ); + } + + #[tokio::test] + #[serial] + async fn serve_refuses_to_start_in_state_1_without_unauthenticated() { + // MR-723 PR A: pin the integration boundary that the classifier + // is actually called by `serve()` before any side-effecting + // work (Lance dataset open, TcpListener::bind). The classifier + // itself is unit-tested above; this test guards the propagation + // path from `classify_server_runtime_state` through serve's + // `?` so a future refactor that drops the call returns red. + // + // Marked `#[serial]` because we have to clear all bearer-token + // env vars, and another test in this module setting any of them + // concurrently would corrupt the read inside `resolve_token_source`. + let _guard = EnvGuard::set(&[ + ("OMNIGRAPH_SERVER_BEARER_TOKEN", None), + ("OMNIGRAPH_SERVER_BEARER_TOKENS_FILE", None), + ("OMNIGRAPH_SERVER_BEARER_TOKENS_JSON", None), + ("OMNIGRAPH_SERVER_BEARER_TOKENS_AWS_SECRET", None), + ("OMNIGRAPH_UNAUTHENTICATED", None), + ]); + let temp = tempdir().unwrap(); + // Graph path doesn't need to exist — classifier fires before + // `AppState::open_with_bearer_tokens_and_policy`. + let config = ServerConfig { + mode: ServerConfigMode::Single { + uri: temp + .path() + .join("graph.omni") + .to_string_lossy() + .into_owned(), + graph_id: "default".to_string(), + policy_file: None, + queries: crate::queries::QueryRegistry::default(), + }, + bind: "127.0.0.1:0".to_string(), + allow_unauthenticated: false, + }; + let result = serve(config).await; + let err = + result.expect_err("serve should refuse to start in State 1 without --unauthenticated"); + let msg = format!("{:?}", err); + assert!( + msg.contains("no bearer tokens") || msg.contains("policy file"), + "expected refusal message naming the misconfiguration, got: {msg}", + ); + } + + #[tokio::test] + #[serial] + async fn unauthenticated_env_var_classification() { + // MR-723 PR A: closes the gap where the env-var read path inside + // `load_server_settings` was structurally implemented but not + // exercised by any test. Three properties to pin, all in one + // sequential test because `cargo test` runs the mod test suite + // in parallel and `OMNIGRAPH_UNAUTHENTICATED` is process-global + // — interleaving with another test that sets the same env var + // (concurrent classifier tests, even the bearer-token suite + // sharing `EnvGuard`) corrupts the read. Sequential within one + // test fn is the simplest race-free shape. + let temp = tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +graphs: + local: + uri: /tmp/demo-unauth.omni +server: + graph: local +"#, + ) + .unwrap(); + + // Truthy values flip Open mode on, even with CLI flag off. + for value in ["1", "true", "yes", "TRUE", "anything"] { + let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some(value))]); + let settings = load_server_settings(Some(&config_path), None, None, None, None, false).await + .expect("settings load should succeed"); + assert!( + settings.allow_unauthenticated, + "OMNIGRAPH_UNAUTHENTICATED={value:?} should enable Open mode", + ); + } + + // Falsy values keep refusal behavior, even with CLI flag off. + for value in ["0", "false", "FALSE", ""] { + let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some(value))]); + let settings = load_server_settings(Some(&config_path), None, None, None, None, false).await + .expect("settings load should succeed"); + assert!( + !settings.allow_unauthenticated, + "OMNIGRAPH_UNAUTHENTICATED={value:?} should NOT enable Open mode", + ); + } + + // Unset env var: also false. + let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", None)]); + let settings = load_server_settings(Some(&config_path), None, None, None, None, false).await + .expect("settings load should succeed"); + assert!( + !settings.allow_unauthenticated, + "OMNIGRAPH_UNAUTHENTICATED unset should NOT enable Open mode", + ); + drop(_guard); + + // CLI flag wins even when env is falsy — `serve()` honors the + // OR of both inputs. + let _guard = EnvGuard::set(&[("OMNIGRAPH_UNAUTHENTICATED", Some("0"))]); + let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await + .expect("settings load should succeed"); + assert!( + settings.allow_unauthenticated, + "--unauthenticated CLI flag should win even when env is falsy", + ); + } + + #[test] + fn classify_policy_enabled_requires_tokens() { + // State 3: tokens + policy → PolicyEnabled, regardless of the + // `allow_unauthenticated` flag (Cedar evaluates the bearer, + // the flag is moot once tokens exist). + assert_eq!( + classify_server_runtime_state(true, true, false).unwrap(), + ServerRuntimeState::PolicyEnabled + ); + assert_eq!( + classify_server_runtime_state(true, true, true).unwrap(), + ServerRuntimeState::PolicyEnabled + ); + } + + #[test] + fn classify_policy_without_tokens_is_rejected() { + // Closes the "policy installed but no tokens → silent 401 on + // every request" footgun. The same shape that single-mode + // `open_with_bearer_tokens_and_policy` used to bail on + // privately is now rejected by the classifier so both single + // and multi mode get the same enforcement from one source of + // truth. + for allow_unauthenticated in [false, true] { + let err = + classify_server_runtime_state(false, true, allow_unauthenticated).unwrap_err(); + let msg = err.to_string(); + assert!( + msg.contains("policy file is configured but no bearer tokens"), + "expected policy-without-tokens rejection message; got: {msg}" + ); + assert!( + msg.contains("every request would 401"), + "rejection message must name the failure mode; got: {msg}" + ); + } + } + + #[test] + fn normalize_bearer_token_trims_and_filters_blank_values() { + assert_eq!(normalize_bearer_token(None), None); + assert_eq!(normalize_bearer_token(Some(" ".to_string())), None); + assert_eq!( + normalize_bearer_token(Some(" demo-token ".to_string())).as_deref(), + Some("demo-token") + ); + } + + struct EnvGuard { + saved: Vec<(&'static str, Option<String>)>, + } + + impl EnvGuard { + fn set(vars: &[(&'static str, Option<&str>)]) -> Self { + let saved = vars + .iter() + .map(|(name, _)| (*name, env::var(name).ok())) + .collect::<Vec<_>>(); + for (name, value) in vars { + unsafe { + match value { + Some(value) => env::set_var(name, value), + None => env::remove_var(name), + } + } + } + Self { saved } + } + } + + impl Drop for EnvGuard { + fn drop(&mut self) { + for (name, value) in self.saved.drain(..) { + unsafe { + match value { + Some(value) => env::set_var(name, value), + None => env::remove_var(name), + } + } + } + } + } + + #[test] + fn parse_bearer_tokens_json_reads_actor_token_map() { + let tokens = parse_bearer_tokens_json(r#"{"alice":" token-a ","bob":"token-b"}"#).unwrap(); + assert_eq!(tokens.len(), 2); + assert!(tokens.contains(&("alice".to_string(), " token-a ".to_string()))); + assert!(tokens.contains(&("bob".to_string(), "token-b".to_string()))); + } + + #[test] + #[serial] + fn server_bearer_tokens_from_env_reads_legacy_token_and_token_file() { + let temp = tempdir().unwrap(); + let tokens_path = temp.path().join("tokens.json"); + fs::write( + &tokens_path, + r#"{"team-01":"token-one","team-02":"token-two"}"#, + ) + .unwrap(); + + let _guard = EnvGuard::set(&[ + ("OMNIGRAPH_SERVER_BEARER_TOKEN", Some(" legacy-token ")), + ( + "OMNIGRAPH_SERVER_BEARER_TOKENS_FILE", + Some(tokens_path.to_str().unwrap()), + ), + ("OMNIGRAPH_SERVER_BEARER_TOKENS_JSON", None), + ]); + + let tokens = server_bearer_tokens_from_env().unwrap(); + assert_eq!( + tokens, + vec![ + ("default".to_string(), "legacy-token".to_string()), + ("team-01".to_string(), "token-one".to_string()), + ("team-02".to_string(), "token-two".to_string()), + ] + ); + } +} From 916015c4164cac79fcdf4ce8f3af7e7c9ffe115f Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 15:14:27 +0300 Subject: [PATCH 114/165] refactor(cli): split main.rs into cli/helpers/output modules Verbatim moves: the clap surface (every command/subcommand/arg struct) to cli.rs, resolution helpers (config/actor/graph/branch/query, remote HTTP, env/token, scaffolding) to helpers.rs, human/JSON formatting to output.rs, the in-source test mod to main_tests.rs via #[path]. main.rs (1,184 lines) keeps main() and the dispatch match. Visibility bumps only; 22 binary tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/cli.rs | 650 +++++ crates/omnigraph-cli/src/helpers.rs | 1085 +++++++++ crates/omnigraph-cli/src/main.rs | 3031 +----------------------- crates/omnigraph-cli/src/main_tests.rs | 416 ++++ crates/omnigraph-cli/src/output.rs | 830 +++++++ 5 files changed, 2990 insertions(+), 3022 deletions(-) create mode 100644 crates/omnigraph-cli/src/cli.rs create mode 100644 crates/omnigraph-cli/src/helpers.rs create mode 100644 crates/omnigraph-cli/src/main_tests.rs create mode 100644 crates/omnigraph-cli/src/output.rs diff --git a/crates/omnigraph-cli/src/cli.rs b/crates/omnigraph-cli/src/cli.rs new file mode 100644 index 0000000..6b59559 --- /dev/null +++ b/crates/omnigraph-cli/src/cli.rs @@ -0,0 +1,650 @@ +//! The clap surface: every command, subcommand, and argument struct +//! (moved verbatim from main.rs in the modularization). + +use super::*; + +pub(crate) const DEFAULT_BEARER_TOKEN_ENV: &str = "OMNIGRAPH_BEARER_TOKEN"; + +#[derive(Debug, Parser)] +#[command(name = "omnigraph")] +#[command(about = "Omnigraph graph database CLI")] +#[command(version = env!("CARGO_PKG_VERSION"), disable_version_flag = true)] +pub(crate) struct Cli { + /// Actor identity for direct-engine writes (MR-722). Overrides + /// `cli.actor` from `omnigraph.yaml`. When the configured policy + /// is in effect, Cedar evaluates this actor against the requested + /// action and scope; with policy configured but neither this flag + /// nor `cli.actor` set, the engine-layer footgun guard fires and + /// the write is denied (no silent bypass). Has no effect on remote + /// HTTP writes — those resolve their actor server-side from the + /// bearer token. + #[arg(long = "as", global = true, value_name = "ACTOR")] + pub(crate) as_actor: Option<String>, + + #[command(subcommand)] + pub(crate) command: Command, +} + +#[derive(Debug, Subcommand)] +pub(crate) enum Command { + /// Print the CLI version + Version, + /// Generate, clean, or refresh explicit seed embeddings + Embed(EmbedArgs), + /// Initialize a new graph from a schema + Init { + #[arg(long)] + schema: PathBuf, + /// Graph URI (local path or s3://) + uri: String, + /// Overwrite existing schema artifacts at the URI. Without + /// this flag, init refuses to touch a URI that already holds + /// `_schema.pg`, `_schema.ir.json`, or `__schema_state.json` + /// — closes the re-init footgun (MR-668 follow-up). With the + /// flag, the operator opts in to destructive semantics. + #[arg(long)] + force: bool, + }, + /// Load data into a graph (local or remote) + Load { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + data: PathBuf, + /// Target branch (defaults to main). Without --from it must exist. + #[arg(long)] + branch: Option<String>, + /// Base branch to fork --branch from when it doesn't exist yet. + /// Without this flag a missing branch is an error, never a fork. + #[arg(long)] + from: Option<String>, + /// How existing rows are handled: overwrite | append | merge. + /// Required — overwrite is destructive, so there is no default. + #[arg(long)] + mode: CliLoadMode, + #[arg(long)] + json: bool, + }, + /// Deprecated alias of `load --from <base>` (defaults: --mode merge, --from main) + Ingest { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + data: PathBuf, + #[arg(long)] + branch: Option<String>, + #[arg(long)] + from: Option<String>, + #[arg(long, default_value = "merge")] + mode: CliLoadMode, + #[arg(long)] + json: bool, + }, + /// Branch operations + Branch { + #[command(subcommand)] + command: BranchCommand, + }, + /// Schema planning operations + Schema { + #[command(subcommand)] + command: SchemaCommand, + }, + /// Validate queries against a schema (offline) or repo (repo-backed). + /// + /// Canonical name is `lint` (matches the `omnigraph_compiler::lint` + /// module and the `OG-XXX-NNN` lint-code vocabulary). Replaces the + /// deprecated `omnigraph query lint` / `omnigraph query check` / + /// `omnigraph check` invocations — each is kept as an argv-level + /// shim that prints a one-line stderr warning and rewrites to + /// `omnigraph lint`. Aliases are deliberately *not* exposed via + /// clap's `visible_alias` because that would advertise two + /// equivalent canonical names, which agents emit interchangeably + /// (see MR-981). + Lint { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + query: PathBuf, + #[arg(long)] + schema: Option<PathBuf>, + #[arg(long)] + json: bool, + }, + /// Operate on the server-side stored-query registry (`queries:`). + Queries { + #[command(subcommand)] + command: QueriesCommand, + }, + /// Show graph snapshot + Snapshot { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + branch: Option<String>, + #[arg(long)] + json: bool, + }, + /// Export a full graph snapshot as JSONL + Export { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + branch: Option<String>, + #[arg(long, hide = true)] + jsonl: bool, + #[arg(long = "type")] + type_names: Vec<String>, + #[arg(long = "table")] + table_keys: Vec<String>, + }, + /// Commit history operations + Commit { + #[command(subcommand)] + command: CommitCommand, + }, + /// Execute a read query against a branch or snapshot. + /// + /// Canonical read endpoint. The previous name `omnigraph read` is + /// kept as a visible alias and prints a one-line deprecation warning + /// when used. Pairs with `omnigraph mutate` on the write side. + #[command(visible_alias = "read")] + Query { + /// Graph URI + #[arg(long)] + uri: Option<String>, + #[arg(hide = true)] + legacy_uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long, conflicts_with_all = ["query", "query_string"])] + alias: Option<String>, + #[arg(long, conflicts_with_all = ["alias", "query_string"])] + query: Option<PathBuf>, + /// Inline GQ source — alternative to `--query <path>` and `--alias <name>`. + #[arg(short = 'e', long = "query-string", value_name = "GQ", conflicts_with_all = ["query", "alias"])] + query_string: Option<String>, + #[arg(long)] + name: Option<String>, + #[command(flatten)] + params: ParamsArgs, + #[arg(long, conflicts_with = "snapshot")] + branch: Option<String>, + #[arg(long, conflicts_with = "branch")] + snapshot: Option<String>, + #[arg(long, conflicts_with = "json")] + format: Option<ReadOutputFormat>, + #[arg(long, conflicts_with = "format")] + json: bool, + #[arg()] + alias_args: Vec<String>, + }, + /// Execute a graph mutation query against a branch. + /// + /// Canonical mutation endpoint. The previous name `omnigraph change` + /// is kept as a visible alias and prints a one-line deprecation + /// warning when used. Pairs with `omnigraph query` on the read side. + #[command(visible_alias = "change")] + Mutate { + /// Graph URI + #[arg(long)] + uri: Option<String>, + #[arg(hide = true)] + legacy_uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long, conflicts_with_all = ["query", "query_string"])] + alias: Option<String>, + #[arg(long, conflicts_with_all = ["alias", "query_string"])] + query: Option<PathBuf>, + /// Inline GQ source — alternative to `--query <path>` and `--alias <name>`. + #[arg(short = 'e', long = "query-string", value_name = "GQ", conflicts_with_all = ["query", "alias"])] + query_string: Option<String>, + #[arg(long)] + name: Option<String>, + #[command(flatten)] + params: ParamsArgs, + #[arg(long)] + branch: Option<String>, + #[arg(long)] + json: bool, + #[arg()] + alias_args: Vec<String>, + }, + /// Policy administration and diagnostics + Policy { + #[command(subcommand)] + command: PolicyCommand, + }, + /// Compact small Lance fragments in every table of the graph + Optimize { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + json: bool, + }, + /// Classify and explicitly repair manifest/head drift + Repair { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + /// Publish verified maintenance drift. Without this flag, repair only + /// previews what it would do. + #[arg(long)] + confirm: bool, + /// Also publish suspicious or unverifiable drift. Requires + /// `--confirm`; use only after operator review. + #[arg(long, requires = "confirm")] + force: bool, + #[arg(long)] + json: bool, + }, + /// Remove old Lance versions from every table of the graph (destructive) + Cleanup { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + /// Number of recent versions to keep per table. Either `--keep` or + /// `--older-than` (or both) must be set. + #[arg(long)] + keep: Option<u32>, + /// Only remove versions older than this duration. Accepts Go-style + /// durations: `7d`, `24h`, `90m`. At least one of --keep / --older-than. + #[arg(long)] + older_than: Option<String>, + /// Required to actually run; without it, prints what would be removed + #[arg(long)] + confirm: bool, + #[arg(long)] + json: bool, + }, + /// Validate and plan read-only cluster configuration. + Cluster { + #[command(subcommand)] + command: ClusterCommand, + }, + /// Manage graphs on a multi-graph server (MR-668) + Graphs { + #[command(subcommand)] + command: GraphsCommand, + }, +} + +#[derive(Debug, Subcommand)] +pub(crate) enum ClusterCommand { + /// Validate cluster.yaml and referenced schemas, queries, and policy files. + Validate { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, + /// Produce a read-only plan by diffing cluster.yaml against __cluster/state.json. + Plan { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, + /// Apply the config-only (query/policy) subset of the plan to the local + /// cluster catalog. Graph/schema changes are deferred to a later stage. + Apply { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, + /// Record a digest-bound approval for a gated (irreversible) change, + /// e.g. a graph delete. Requires the global --as actor. + Approve { + /// Typed resource address of the gated change (e.g. graph.scratch). + resource: String, + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, + /// Read the local JSON state ledger without scanning live graph resources. + Status { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, + /// Refresh existing local JSON state from declared graph observations. + Refresh { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, + /// Import initial local JSON state from declared graph observations. + Import { + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, + /// Remove a held local JSON state lock after operator confirmation. + ForceUnlock { + /// Exact lock id from cluster status or a state_lock_held diagnostic. + lock_id: String, + /// Cluster config directory containing cluster.yaml. + #[arg(long, default_value = ".")] + config: PathBuf, + /// Emit JSON instead of human text. + #[arg(long)] + json: bool, + }, +} + +/// Operations on the graph registry of a multi-graph server (MR-668). +/// +/// All operations target a remote multi-graph server URL (http:// or +/// https://). Local-URI invocations return a clear error. To add or +/// remove graphs, operators edit `omnigraph.yaml` directly and restart +/// the server — runtime mutation is not exposed in v0.6.0. +#[derive(Debug, Subcommand)] +pub(crate) enum GraphsCommand { + /// List every graph registered with the multi-graph server. + List { + /// Remote server URL (e.g. `https://server.example.com`). + #[arg(long)] + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + json: bool, + }, +} + +#[derive(Debug, Subcommand)] +pub(crate) enum BranchCommand { + /// Create a new branch + Create { + /// Graph URI + #[arg(long)] + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + from: Option<String>, + name: String, + #[arg(long)] + json: bool, + }, + /// List branches + List { + /// Graph URI + #[arg(long)] + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + json: bool, + }, + /// Delete a branch + Delete { + /// Graph URI + #[arg(long)] + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + name: String, + #[arg(long)] + json: bool, + }, + /// Merge a source branch into a target branch + Merge { + /// Graph URI + #[arg(long)] + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + source: String, + #[arg(long)] + into: Option<String>, + #[arg(long)] + json: bool, + }, +} + +#[derive(Debug, Subcommand)] +pub(crate) enum SchemaCommand { + /// Plan a schema migration against the accepted persisted schema + Plan { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + schema: PathBuf, + #[arg(long)] + json: bool, + /// Show the plan as it would execute with `--allow-data-loss`. + /// Promotes every `DropMode::Soft` step to `DropMode::Hard` + /// so the plan output reflects the destructive intent. + #[arg(long, default_value_t = false)] + allow_data_loss: bool, + }, + /// Apply a supported schema migration + Apply { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + schema: PathBuf, + #[arg(long)] + json: bool, + /// Allow destructive (data-loss) schema changes. + /// + /// Without this flag, drops are "soft": the column or table + /// is removed from the current manifest version but prior + /// versions are retained, so `snapshot_at_version(pre_drop)` + /// can still read the dropped data until `omnigraph cleanup` + /// runs. With this flag, drops are "hard": `cleanup_old_versions` + /// runs on the affected datasets immediately after the apply, + /// making the prior data unreachable. + #[arg(long, default_value_t = false)] + allow_data_loss: bool, + }, + /// Show the current accepted schema source + #[command(alias = "get")] + Show { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + json: bool, + }, +} + +#[derive(Debug, Subcommand)] + +pub(crate) enum CommitCommand { + /// List graph commits + List { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + branch: Option<String>, + #[arg(long)] + json: bool, + }, + /// Show a graph commit + Show { + /// Graph URI + #[arg(long)] + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + commit_id: String, + #[arg(long)] + json: bool, + }, +} + +#[derive(Debug, Subcommand)] +pub(crate) enum PolicyCommand { + /// Validate policy YAML and compiled Cedar policy state + Validate { + #[arg(long)] + config: Option<PathBuf>, + }, + /// Run declarative policy tests from policy.tests.yaml + Test { + #[arg(long)] + config: Option<PathBuf>, + }, + /// Explain one policy decision locally + Explain { + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + actor: String, + #[arg(long)] + action: PolicyAction, + #[arg(long)] + branch: Option<String>, + #[arg(long = "target-branch")] + target_branch: Option<String>, + }, +} + +#[derive(Debug, Subcommand)] +pub(crate) enum QueriesCommand { + /// Type-check the stored-query registry against the live schema. + /// + /// Distinct from `omnigraph lint` (which lints one `.gq` file): + /// this validates the whole `queries:` registry — opening the graph + /// to read its schema and confirming every stored query still + /// type-checks. Exits non-zero on any breakage. + Validate { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + json: bool, + }, + /// List the registered stored queries (name, MCP exposure, params). + List { + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + json: bool, + }, +} + +#[derive(Debug, Args, Clone)] +pub(crate) struct ParamsArgs { + #[arg(long, conflicts_with = "params_file")] + pub(crate) params: Option<String>, + #[arg(long, conflicts_with = "params")] + pub(crate) params_file: Option<PathBuf>, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize, ValueEnum)] +#[serde(rename_all = "snake_case")] +pub(crate) enum CliLoadMode { + Overwrite, + Append, + Merge, +} + +impl From<CliLoadMode> for LoadMode { + fn from(value: CliLoadMode) -> Self { + match value { + CliLoadMode::Overwrite => LoadMode::Overwrite, + CliLoadMode::Append => LoadMode::Append, + CliLoadMode::Merge => LoadMode::Merge, + } + } +} + +impl CliLoadMode { + pub(crate) fn as_str(self) -> &'static str { + match self { + CliLoadMode::Overwrite => "overwrite", + CliLoadMode::Append => "append", + CliLoadMode::Merge => "merge", + } + } +} + diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs new file mode 100644 index 0000000..be356a9 --- /dev/null +++ b/crates/omnigraph-cli/src/helpers.rs @@ -0,0 +1,1085 @@ +//! Resolution helpers: config/actor/graph/branch/query resolution, +//! remote HTTP, env/token handling, scaffolding (moved verbatim from +//! main.rs in the modularization). + +use super::*; + +pub(crate) fn ensure_local_graph_parent(uri: &str) -> Result<()> { + if !uri.contains("://") { + fs::create_dir_all(uri)?; + } + Ok(()) +} + +pub(crate) fn is_remote_uri(uri: &str) -> bool { + uri.starts_with("http://") || uri.starts_with("https://") +} + +pub(crate) fn remote_url(base: &str, path: &str) -> String { + format!("{}{}", base.trim_end_matches('/'), path) +} + +pub(crate) fn remote_branch_url(base: &str, branch: &str) -> Result<String> { + let mut url = reqwest::Url::parse(&format!("{}/", base.trim_end_matches('/')))?; + url.path_segments_mut() + .map_err(|_| color_eyre::eyre::eyre!("invalid remote base url"))? + .extend(["branches", branch]); + Ok(url.to_string()) +} + +pub(crate) fn normalize_bearer_token(value: Option<String>) -> Option<String> { + value + .map(|value| value.trim().to_string()) + .filter(|value| !value.is_empty()) +} + +pub(crate) fn bearer_token_from_env(var_name: &str) -> Option<String> { + normalize_bearer_token(std::env::var(var_name).ok()) +} + +pub(crate) fn parse_env_assignment(line: &str) -> Option<(String, String)> { + let line = line.trim(); + if line.is_empty() || line.starts_with('#') { + return None; + } + + let line = line.strip_prefix("export ").unwrap_or(line).trim(); + let (name, value) = line.split_once('=')?; + let name = name.trim(); + if name.is_empty() { + return None; + } + + let value = value.trim(); + let value = if value.len() >= 2 + && ((value.starts_with('"') && value.ends_with('"')) + || (value.starts_with('\'') && value.ends_with('\''))) + { + &value[1..value.len() - 1] + } else { + value + }; + + Some((name.to_string(), value.to_string())) +} + +pub(crate) fn bearer_token_from_env_file(path: &Path, var_name: &str) -> Result<Option<String>> { + if !path.exists() { + return Ok(None); + } + + for line in fs::read_to_string(path)?.lines() { + let Some((name, value)) = parse_env_assignment(line) else { + continue; + }; + if name == var_name { + return Ok(normalize_bearer_token(Some(value))); + } + } + + Ok(None) +} + +pub(crate) fn load_env_file_into_process(path: &Path) -> Result<()> { + if !path.exists() { + return Ok(()); + } + + for line in fs::read_to_string(path)?.lines() { + let Some((name, value)) = parse_env_assignment(line) else { + continue; + }; + if std::env::var_os(&name).is_none() { + unsafe { + std::env::set_var(name, value); + } + } + } + + Ok(()) +} + +pub(crate) fn load_cli_config(config_path: Option<&PathBuf>) -> Result<OmnigraphConfig> { + let config = load_config(config_path)?; + if let Some(path) = config.resolve_auth_env_file() { + load_env_file_into_process(&path)?; + } + Ok(config) +} + +#[derive(Debug, Clone)] +pub(crate) struct ResolvedCliGraph { + pub(crate) uri: String, + pub(crate) selected: Option<String>, + pub(crate) graph_id: String, + pub(crate) policy_file: Option<PathBuf>, + pub(crate) is_remote: bool, +} + +impl ResolvedCliGraph { + pub(crate) fn selected(&self) -> Option<&str> { + self.selected.as_deref() + } +} + +pub(crate) struct ResolvedPolicyContext { + pub(crate) policy_file: PathBuf, + pub(crate) graph_id: String, +} + +pub(crate) fn resolve_policy_context(config: &OmnigraphConfig) -> Result<ResolvedPolicyContext> { + let selected = config.resolve_policy_tooling_graph_selection()?; + let policy_file = config.resolve_policy_file_for(selected).ok_or_else(|| { + color_eyre::eyre::eyre!( + "policy.file or graphs.<name>.policy.file must be set in omnigraph.yaml" + ) + })?; + let graph_id = match selected { + Some(name) => graph_resource_id_for_selection(Some(name), ""), + None => graph_resource_id_for_selection(None, "default"), + }; + Ok(ResolvedPolicyContext { + policy_file, + graph_id, + }) +} + +pub(crate) fn resolve_policy_engine(context: &ResolvedPolicyContext) -> Result<PolicyEngine> { + PolicyEngine::load_graph(&context.policy_file, &context.graph_id) +} + +pub(crate) fn resolve_policy_engine_for_graph(graph: &ResolvedCliGraph) -> Result<PolicyEngine> { + let policy_file = graph.policy_file.as_ref().ok_or_else(|| { + color_eyre::eyre::eyre!( + "policy.file or graphs.<name>.policy.file must be set in omnigraph.yaml" + ) + })?; + PolicyEngine::load_graph(policy_file, &graph.graph_id) +} + +pub(crate) async fn open_local_db_with_policy(graph: &ResolvedCliGraph) -> Result<Omnigraph> { + let db = Omnigraph::open(&graph.uri).await?; + if graph.policy_file.is_some() { + let engine = Arc::new(resolve_policy_engine_for_graph(graph)?); + Ok(db.with_policy(engine as Arc<dyn omnigraph_policy::PolicyChecker>)) + } else { + Ok(db) + } +} + +pub(crate) fn resolve_cluster_actor(cli_as: Option<&str>) -> Result<Option<String>> { + if let Some(actor) = cli_as { + return Ok(Some(actor.to_string())); + } + let config = load_config(None).wrap_err( + "resolving the default actor from the per-operator omnigraph.yaml (pass --as <ACTOR> to skip this lookup)", + )?; + Ok(config.cli.actor.clone()) +} + +pub(crate) fn resolve_cli_actor<'a>(cli_as: Option<&'a str>, config: &'a OmnigraphConfig) -> Option<&'a str> { + cli_as.or(config.cli.actor.as_deref()) +} + +pub(crate) fn resolve_policy_tests_path(context: &ResolvedPolicyContext) -> PathBuf { + context.policy_file.with_file_name("policy.tests.yaml") +} + +pub(crate) fn normalize_policy_graph_uri(uri: &str) -> Result<String> { + if is_remote_uri(uri) { + Ok(uri.trim_end_matches('/').to_string()) + } else { + Ok(normalize_root_uri(uri)?) + } +} + +pub(crate) fn resolve_remote_bearer_token( + config: &OmnigraphConfig, + explicit_uri: Option<&str>, + explicit_target: Option<&str>, +) -> Result<Option<String>> { + let scoped_env = + config.graph_bearer_token_env(explicit_uri, explicit_target, config.cli_graph_name()); + let mut env_names = Vec::new(); + if let Some(name) = scoped_env { + env_names.push(name.to_string()); + } + if env_names + .iter() + .all(|name| name != DEFAULT_BEARER_TOKEN_ENV) + { + env_names.push(DEFAULT_BEARER_TOKEN_ENV.to_string()); + } + + let env_file = config.resolve_auth_env_file(); + for env_name in env_names { + if let Some(token) = bearer_token_from_env(&env_name) { + return Ok(Some(token)); + } + if let Some(path) = env_file.as_ref() { + if let Some(token) = bearer_token_from_env_file(path, &env_name)? { + return Ok(Some(token)); + } + } + } + + Ok(None) +} + +pub(crate) fn build_http_client() -> Result<reqwest::Client> { + Ok(reqwest::Client::new()) +} + +pub(crate) fn apply_bearer_token( + request: reqwest::RequestBuilder, + token: Option<&str>, +) -> reqwest::RequestBuilder { + if let Some(token) = token { + request.header(AUTHORIZATION, format!("Bearer {}", token)) + } else { + request + } +} + +pub(crate) async fn remote_json<T: DeserializeOwned>( + client: &reqwest::Client, + method: Method, + url: String, + body: Option<Value>, + bearer_token: Option<&str>, +) -> Result<T> { + let request = apply_bearer_token(client.request(method, url), bearer_token); + let request = if let Some(body) = body { + request.json(&body) + } else { + request + }; + let response = request.send().await?; + let status = response.status(); + let text = response.text().await?; + if !status.is_success() { + if let Ok(error) = serde_json::from_str::<ErrorOutput>(&text) { + bail!(error.error); + } + bail!("server returned {}: {}", status, text); + } + Ok(serde_json::from_str(&text)?) +} + +pub(crate) fn resolve_uri( + config: &OmnigraphConfig, + cli_uri: Option<String>, + cli_target: Option<&str>, +) -> Result<String> { + config.resolve_target_uri(cli_uri, cli_target, config.cli_graph_name()) +} + +pub(crate) fn resolve_cli_graph( + config: &OmnigraphConfig, + cli_uri: Option<String>, + cli_target: Option<&str>, +) -> Result<ResolvedCliGraph> { + let selected = if cli_uri.is_some() { + None + } else { + cli_target + .map(str::to_string) + .or_else(|| config.cli_graph_name().map(str::to_string)) + }; + config.resolve_graph_selection(selected.as_deref())?; + let uri = resolve_uri(config, cli_uri, cli_target)?; + let normalized_uri = normalize_policy_graph_uri(&uri)?; + let graph_id = graph_resource_id_for_selection(selected.as_deref(), &normalized_uri); + Ok(ResolvedCliGraph { + graph_id, + is_remote: is_remote_uri(&uri), + policy_file: config.resolve_policy_file_for(selected.as_deref()), + selected, + uri, + }) +} + +pub(crate) fn resolve_local_graph( + config: &OmnigraphConfig, + cli_uri: Option<String>, + cli_target: Option<&str>, + operation: &str, +) -> Result<ResolvedCliGraph> { + let graph = resolve_cli_graph(config, cli_uri, cli_target)?; + if graph.is_remote { + bail!( + "{} is only supported against local graph URIs in this milestone", + operation + ); + } + Ok(graph) +} + +pub(crate) fn parse_duration_arg(s: &str) -> Result<std::time::Duration> { + let s = s.trim(); + if s.is_empty() { + bail!("duration is empty"); + } + let (num_part, unit) = match s + .char_indices() + .rev() + .find(|(_, c)| c.is_ascii_alphabetic()) + { + Some((i, _)) => ( + &s[..i + 1 - s[i..].chars().next().unwrap().len_utf8()], + &s[i..], + ), + None => (s, ""), + }; + let n: u64 = num_part + .parse() + .map_err(|e| color_eyre::eyre::eyre!("invalid duration '{}': {}", s, e))?; + let secs = match unit { + "" | "s" => n, + "m" => n * 60, + "h" => n * 60 * 60, + "d" => n * 60 * 60 * 24, + "w" => n * 60 * 60 * 24 * 7, + _ => bail!("unknown duration unit '{}'. Supported: s, m, h, d, w", unit), + }; + Ok(std::time::Duration::from_secs(secs)) +} + +pub(crate) fn resolve_local_uri( + config: &OmnigraphConfig, + cli_uri: Option<String>, + cli_target: Option<&str>, + operation: &str, +) -> Result<String> { + Ok(resolve_local_graph(config, cli_uri, cli_target, operation)?.uri) +} + +pub(crate) fn resolve_branch( + config: &OmnigraphConfig, + cli_branch: Option<String>, + alias_branch: Option<String>, + default_branch: &str, +) -> String { + cli_branch + .or(alias_branch) + .or_else(|| config.cli.branch.clone()) + .unwrap_or_else(|| default_branch.to_string()) +} + +pub(crate) fn resolve_read_target( + config: &OmnigraphConfig, + cli_branch: Option<String>, + cli_snapshot: Option<String>, + alias_branch: Option<String>, +) -> Result<ReadTarget> { + if cli_branch.is_some() && cli_snapshot.is_some() { + bail!("read target may specify branch or snapshot, not both"); + } + Ok(read_target_from_cli( + cli_branch + .or(alias_branch) + .or_else(|| config.cli.branch.clone()), + cli_snapshot, + )) +} + +pub(crate) fn resolve_query_path( + config: &OmnigraphConfig, + explicit_query: Option<&PathBuf>, + alias_query: Option<&str>, +) -> Result<PathBuf> { + explicit_query + .map(PathBuf::from) + .or_else(|| alias_query.map(PathBuf::from)) + .ok_or_else(|| { + color_eyre::eyre::eyre!( + "exactly one of --query, --query-string, or --alias must be provided" + ) + }) + .and_then(|query_path| config.resolve_query_path(&query_path)) +} + +pub(crate) fn resolve_query_source( + config: &OmnigraphConfig, + explicit_query: Option<&PathBuf>, + inline_query: Option<&str>, + alias_query: Option<&str>, +) -> Result<String> { + if let Some(inline) = inline_query { + if inline.trim().is_empty() { + bail!("--query-string must not be empty"); + } + return Ok(inline.to_string()); + } + Ok(fs::read_to_string(resolve_query_path( + config, + explicit_query, + alias_query, + )?)?) +} + +pub(crate) fn parse_alias_value(value: &str) -> Value { + serde_json::from_str(value).unwrap_or_else(|_| Value::String(value.to_string())) +} + +pub(crate) fn merged_params_json( + alias_name: Option<&str>, + alias_arg_names: &[String], + alias_arg_values: &[String], + explicit: Option<Value>, +) -> Result<Option<Value>> { + if alias_arg_values.len() > alias_arg_names.len() { + let alias = alias_name.unwrap_or("<alias>"); + bail!( + "alias '{}' expects at most {} args but got {}", + alias, + alias_arg_names.len(), + alias_arg_values.len() + ); + } + + let mut merged = serde_json::Map::new(); + for (arg_name, arg_value) in alias_arg_names.iter().zip(alias_arg_values.iter()) { + merged.insert(arg_name.clone(), parse_alias_value(arg_value)); + } + + match explicit { + Some(Value::Object(object)) => { + for (key, value) in object { + merged.insert(key, value); + } + } + Some(_) => bail!("params JSON must be an object"), + None => {} + } + + if merged.is_empty() { + Ok(None) + } else { + Ok(Some(Value::Object(merged))) + } +} + +pub(crate) fn resolve_read_format( + config: &OmnigraphConfig, + cli_format: Option<ReadOutputFormat>, + json: bool, + alias_format: Option<ReadOutputFormat>, +) -> ReadOutputFormat { + if json { + ReadOutputFormat::Json + } else { + cli_format + .or(alias_format) + .unwrap_or_else(|| config.cli_output_format()) + } +} + +pub(crate) fn resolve_alias<'a>( + config: &'a OmnigraphConfig, + alias_name: Option<&'a str>, + expected: AliasCommand, +) -> Result<Option<(&'a str, &'a omnigraph_server::AliasConfig)>> { + let Some(alias_name) = alias_name else { + return Ok(None); + }; + let alias = config.alias(alias_name)?; + if alias.command != expected { + bail!( + "alias '{}' is a {:?} alias, not a {:?} alias", + alias_name, + alias.command, + expected + ); + } + Ok(Some((alias_name, alias))) +} + +pub(crate) fn normalize_legacy_alias_uri( + uri: Option<String>, + target_available: bool, + alias_name: Option<&str>, + mut alias_args: Vec<String>, +) -> (Option<String>, Vec<String>) { + let Some(candidate) = uri else { + return (None, alias_args); + }; + + if alias_name.is_some() && target_available { + alias_args.insert(0, candidate); + return (None, alias_args); + } + + (Some(candidate), alias_args) +} + +pub(crate) fn scaffold_config_if_missing(uri: &str) -> Result<()> { + let path = inferred_config_path(uri)?; + if path.exists() { + return Ok(()); + } + + fs::write( + path, + format!( + "\ +project: + name: Omnigraph Project + +graphs: + local: + uri: {} + # bearer_token_env: OMNIGRAPH_BEARER_TOKEN + +server: + graph: local + bind: 127.0.0.1:8080 + +cli: + graph: local + branch: main + output_format: table + table_max_column_width: 80 + table_cell_layout: truncate + +query: + roots: + - queries + - . + +aliases: + # owner: + # command: read + # query: context.gq + # name: decision_owner + # args: [slug] + # graph: local + # branch: main + # format: kv + # + # attach_trace: + # command: change + # query: mutations.gq + # name: attach_trace + # args: [decision_slug, trace_slug] + # graph: local + # branch: main + +# auth: +# env_file: ./.env.omni +# +# policy: +# file: ./policy.yaml +", + yaml_string(uri), + ), + )?; + Ok(()) +} + +pub(crate) fn inferred_config_path(uri: &str) -> Result<PathBuf> { + if uri.contains("://") { + return Ok(omnigraph_server::config::default_config_path()); + } + + let path = Path::new(uri); + let base = if path.is_absolute() { + path.parent() + .map(Path::to_path_buf) + .unwrap_or(std::env::current_dir()?) + } else { + std::env::current_dir()?.join(path.parent().unwrap_or_else(|| Path::new("."))) + }; + Ok(base.join(omnigraph_server::config::DEFAULT_CONFIG_FILE)) +} + +pub(crate) fn read_target_from_cli(branch: Option<String>, snapshot: Option<String>) -> ReadTarget { + if let Some(snapshot) = snapshot { + ReadTarget::snapshot(SnapshotId::new(snapshot)) + } else { + ReadTarget::branch(branch.unwrap_or_else(|| "main".to_string())) + } +} + +pub(crate) fn load_params_json(params: &ParamsArgs) -> Result<Option<Value>> { + match (¶ms.params, ¶ms.params_file) { + (Some(inline), None) => Ok(Some(serde_json::from_str(inline)?)), + (None, Some(path)) => Ok(Some(serde_json::from_str(&fs::read_to_string(path)?)?)), + (None, None) => Ok(None), + (Some(_), Some(_)) => bail!("only one of --params or --params-file may be provided"), + } +} + +pub(crate) fn select_named_query( + query_source: &str, + requested_name: Option<&str>, +) -> Result<(String, Vec<omnigraph_compiler::query::ast::Param>)> { + let parsed = parse_query(query_source)?; + let query = if let Some(name) = requested_name { + parsed + .queries + .into_iter() + .find(|query| query.name == name) + .ok_or_else(|| color_eyre::eyre::eyre!("query '{}' not found", name))? + } else if parsed.queries.len() == 1 { + parsed.queries.into_iter().next().unwrap() + } else { + bail!("query file contains multiple queries; pass --name"); + }; + + Ok((query.name, query.params)) +} + +pub(crate) fn query_params_from_json( + query_params: &[omnigraph_compiler::query::ast::Param], + params_json: Option<&Value>, +) -> Result<ParamMap> { + json_params_to_param_map(params_json, query_params, JsonParamMode::Standard) + .map_err(|err| color_eyre::eyre::eyre!(err.to_string())) +} + +pub(crate) async fn execute_query_lint( + config: &OmnigraphConfig, + cli_uri: Option<String>, + cli_target: Option<&str>, + schema_path: Option<&PathBuf>, + query_path: &PathBuf, +) -> Result<QueryLintOutput> { + let resolved_query_path = resolve_query_path(config, Some(query_path), None)?; + let query_source = fs::read_to_string(&resolved_query_path)?; + let query_path = resolved_query_path.to_string_lossy().into_owned(); + + if let Some(schema_path) = schema_path { + let schema_source = fs::read_to_string(schema_path)?; + let schema = + parse_schema(&schema_source).map_err(|err| color_eyre::eyre::eyre!(err.to_string()))?; + let catalog = + build_catalog(&schema).map_err(|err| color_eyre::eyre::eyre!(err.to_string()))?; + return Ok(lint_query_file( + &catalog, + &query_source, + query_path, + QueryLintSchemaSource::file(schema_path.to_string_lossy().into_owned()), + )); + } + + let has_graph_target = + cli_uri.is_some() || cli_target.is_some() || config.cli_graph_name().is_some(); + if !has_graph_target { + bail!("query lint requires --schema <schema.pg> or a resolvable graph target"); + } + + let uri = resolve_local_uri(config, cli_uri, cli_target, "query lint")?; + let db = Omnigraph::open(&uri).await?; + Ok(lint_query_file( + &db.catalog(), + &query_source, + query_path, + QueryLintSchemaSource::graph(uri), + )) +} + +pub(crate) fn resolve_selected_graph( + config: &OmnigraphConfig, + cli_uri: Option<String>, + cli_target: Option<&str>, + operation: &str, +) -> Result<(String, Option<String>)> { + let graph = resolve_local_graph(config, cli_uri, cli_target, operation)?; + Ok((graph.uri, graph.selected)) +} + +pub(crate) fn load_registry_or_report( + config: &OmnigraphConfig, + selected: Option<&str>, +) -> Result<QueryRegistry> { + QueryRegistry::load(config, config.query_entries_for(selected)).map_err(|errors| { + color_eyre::eyre::eyre!( + "stored-query registry failed to load:\n {}", + errors + .iter() + .map(|e| e.to_string()) + .collect::<Vec<_>>() + .join("\n ") + ) + }) +} + +pub(crate) fn graph_query_registry_names(config: &OmnigraphConfig) -> Vec<&str> { + config + .graphs + .iter() + .filter_map(|(name, graph)| (!graph.queries.is_empty()).then_some(name.as_str())) + .collect() +} + +pub(crate) fn resolve_registry_selection_for_list( + config: &OmnigraphConfig, + target: Option<&str>, +) -> Result<Option<String>> { + let selected = target + .map(str::to_string) + .or_else(|| config.cli_graph_name().map(str::to_string)); + if let Some(name) = selected.as_deref() { + config.resolve_graph_selection(Some(name))?; + return Ok(selected); + } + + if !config.query_entries().is_empty() { + return Ok(None); + } + + let graph_names = graph_query_registry_names(config); + if graph_names.is_empty() { + return Ok(None); + } + + bail!( + "stored-query registries are configured for graph{} {} but no graph was selected. Pass `--target {}` or set `cli.graph`.", + if graph_names.len() == 1 { "" } else { "s" }, + graph_names.join(", "), + graph_names[0], + ) +} + +pub(crate) fn validate_registry_for_catalog( + registry: &QueryRegistry, + catalog: &omnigraph_compiler::catalog::Catalog, + label: &str, +) -> omnigraph::error::Result<()> { + let report = check(registry, catalog); + if report.has_breakages() { + return Err(omnigraph::error::OmniError::manifest( + format_check_breakages(label, &report), + )); + } + Ok(()) +} + +pub(crate) async fn execute_queries_validate( + uri: Option<String>, + target: Option<String>, + config_path: Option<&PathBuf>, + json: bool, +) -> Result<()> { + let config = load_cli_config(config_path)?; + // One selection drives both the schema URI and the registry, so a + // positional URI and a `--target` can't validate different graphs. + let (uri, selected) = + resolve_selected_graph(&config, uri, target.as_deref(), "queries validate")?; + let registry = load_registry_or_report(&config, selected.as_deref())?; + let db = Omnigraph::open(&uri).await?; + let report = check(®istry, &db.catalog()); + + let output = QueriesValidateOutput { + ok: !report.has_breakages(), + breakages: report + .breakages + .iter() + .map(|b| QueriesIssue { + query: b.query.clone(), + message: b.message.clone(), + }) + .collect(), + warnings: report + .warnings + .iter() + .map(|w| QueriesIssue { + query: w.query.clone(), + message: w.message.clone(), + }) + .collect(), + }; + + if json { + print_json(&output)?; + } else { + if output.breakages.is_empty() { + println!( + "OK {} stored quer{} type-check against the schema", + registry.len(), + if registry.len() == 1 { "y" } else { "ies" } + ); + } + for issue in &output.breakages { + println!("ERROR query '{}': {}", issue.query, issue.message); + } + for issue in &output.warnings { + println!("WARN query '{}': {}", issue.query, issue.message); + } + } + + if report.has_breakages() { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + +pub(crate) fn execute_queries_list( + target: Option<String>, + config_path: Option<&PathBuf>, + json: bool, +) -> Result<()> { + let config = load_cli_config(config_path)?; + let selected = resolve_registry_selection_for_list(&config, target.as_deref())?; + let registry = load_registry_or_report(&config, selected.as_deref())?; + + let output = QueriesListOutput { + queries: registry + .iter() + .map(|q| QueriesListItem { + name: q.name.clone(), + mcp_expose: q.expose, + tool_name: q.tool_name.clone(), + mutation: q.is_mutation(), + params: q + .decl + .params + .iter() + .map(|p| QueriesParam { + name: p.name.clone(), + type_name: p.type_name.clone(), + nullable: p.nullable, + }) + .collect(), + }) + .collect(), + }; + + if json { + print_json(&output)?; + } else if output.queries.is_empty() { + println!("(no stored queries registered)"); + } else { + for q in &output.queries { + let kind = if q.mutation { "mutation" } else { "read" }; + let params = q + .params + .iter() + .map(|p| { + format!( + "${}: {}{}", + p.name, + p.type_name, + if p.nullable { "?" } else { "" } + ) + }) + .collect::<Vec<_>>() + .join(", "); + let mcp = if q.mcp_expose { + format!(" [mcp: {}]", q.tool_name.as_deref().unwrap_or(&q.name)) + } else { + String::new() + }; + println!("{kind} {}({params}){mcp}", q.name); + } + } + Ok(()) +} + +pub(crate) async fn execute_read( + uri: &str, + query_source: &str, + query_name: Option<&str>, + target: ReadTarget, + params_json: Option<&Value>, +) -> Result<ReadOutput> { + let (selected_name, query_params) = select_named_query(query_source, query_name)?; + let params = query_params_from_json(&query_params, params_json)?; + let db = Omnigraph::open(uri).await?; + let result = db + .query(target.clone(), query_source, &selected_name, ¶ms) + .await?; + Ok(read_output(selected_name, &target, result)) +} + +pub(crate) async fn execute_read_remote( + client: &reqwest::Client, + uri: &str, + query_source: &str, + query_name: Option<&str>, + target: ReadTarget, + params_json: Option<&Value>, + bearer_token: Option<&str>, +) -> Result<ReadOutput> { + let (branch, snapshot) = match &target { + ReadTarget::Branch(branch) => (Some(branch.clone()), None), + ReadTarget::Snapshot(snapshot) => (None, Some(snapshot.as_str().to_string())), + }; + remote_json( + client, + Method::POST, + remote_url(uri, "/read"), + Some(serde_json::to_value(ReadRequest { + query_source: query_source.to_string(), + query_name: query_name.map(ToOwned::to_owned), + params: params_json.cloned(), + branch, + snapshot, + })?), + bearer_token, + ) + .await +} + +pub(crate) async fn execute_change( + graph: &ResolvedCliGraph, + query_source: &str, + query_name: Option<&str>, + branch: &str, + params_json: Option<&Value>, + config: &OmnigraphConfig, + cli_as_actor: Option<&str>, +) -> Result<ChangeOutput> { + let (selected_name, query_params) = select_named_query(query_source, query_name)?; + let params = query_params_from_json(&query_params, params_json)?; + let db = open_local_db_with_policy(graph).await?; + let actor = resolve_cli_actor(cli_as_actor, config); + let result = db + .mutate_as(branch, query_source, &selected_name, ¶ms, actor) + .await?; + Ok(ChangeOutput { + branch: branch.to_string(), + query_name: selected_name, + affected_nodes: result.affected_nodes, + affected_edges: result.affected_edges, + actor_id: actor.map(String::from), + }) +} + +pub(crate) fn legacy_change_request_body( + query_source: &str, + query_name: Option<&str>, + branch: &str, + params_json: Option<&Value>, +) -> Value { + let mut body = serde_json::json!({ + "query_source": query_source, + "branch": branch, + }); + if let Some(name) = query_name { + body["query_name"] = Value::String(name.to_string()); + } + if let Some(params) = params_json { + body["params"] = params.clone(); + } + body +} + +pub(crate) async fn execute_change_remote( + client: &reqwest::Client, + uri: &str, + query_source: &str, + query_name: Option<&str>, + branch: &str, + params_json: Option<&Value>, + bearer_token: Option<&str>, +) -> Result<ChangeOutput> { + remote_json( + client, + Method::POST, + remote_url(uri, "/change"), + Some(legacy_change_request_body( + query_source, + query_name, + branch, + params_json, + )), + bearer_token, + ) + .await +} + +pub(crate) async fn execute_export_to_writer<W: Write>( + uri: &str, + branch: &str, + type_names: &[String], + table_keys: &[String], + writer: &mut W, +) -> Result<()> { + let db = Omnigraph::open(uri).await?; + db.export_jsonl_to_writer(branch, type_names, table_keys, writer) + .await?; + writer.flush()?; + Ok(()) +} + +pub(crate) async fn execute_export_remote_to_writer<W: Write>( + client: &reqwest::Client, + uri: &str, + branch: &str, + type_names: &[String], + table_keys: &[String], + bearer_token: Option<&str>, + writer: &mut W, +) -> Result<()> { + let request = apply_bearer_token( + client.request(Method::POST, remote_url(uri, "/export")), + bearer_token, + ) + .json(&ExportRequest { + branch: Some(branch.to_string()), + type_names: type_names.to_vec(), + table_keys: table_keys.to_vec(), + }); + let mut response = request.send().await?; + let status = response.status(); + if !status.is_success() { + let text = response.text().await?; + if let Ok(error) = serde_json::from_str::<ErrorOutput>(&text) { + bail!(error.error); + } + bail!("server returned {}: {}", status, text); + } + + while let Some(chunk) = response.chunk().await? { + writer.write_all(&chunk)?; + } + writer.flush()?; + Ok(()) +} + +pub(crate) fn rewrite_deprecated_argv(args: Vec<OsString>) -> Vec<OsString> { + if args.len() >= 3 { + let sub = args[1].to_str(); + let sub2 = args[2].to_str(); + if sub == Some("query") && matches!(sub2, Some("lint") | Some("check")) { + let suffix = sub2.unwrap(); + eprintln!( + "warning: `omnigraph query {suffix}` is deprecated; use `omnigraph lint` instead" + ); + // Drop the leading `query` token AND normalize `check` -> `lint`. + // `check` is no longer a clap visible_alias (MR-981 §6), so the + // rewritten argv must reach the canonical `lint` subcommand + // directly. Result for `omnigraph query check --query foo.gq`: + // `omnigraph lint --query foo.gq`. + let mut out = Vec::with_capacity(args.len() - 1); + out.push(args[0].clone()); + out.push(OsString::from("lint")); + out.extend(args[3..].iter().cloned()); + return out; + } + } + if let Some(sub) = args.get(1).and_then(|s| s.to_str()) { + match sub { + "read" => { + eprintln!("warning: `omnigraph read` is deprecated; use `omnigraph query` instead") + } + "change" => eprintln!( + "warning: `omnigraph change` is deprecated; use `omnigraph mutate` instead" + ), + "check" => { + eprintln!("warning: `omnigraph check` is deprecated; use `omnigraph lint` instead"); + // Rewrite the top-level subcommand to `lint`; pass through the rest. + let mut out = Vec::with_capacity(args.len()); + out.push(args[0].clone()); + out.push(OsString::from("lint")); + out.extend(args[2..].iter().cloned()); + return out; + } + _ => {} + } + } + args +} diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index e9cff0c..0d6ce03 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -47,2613 +47,12 @@ mod read_format; use embed::{EmbedArgs, EmbedOutput, execute_embed}; use read_format::{ReadRenderOptions, render_read}; -const DEFAULT_BEARER_TOKEN_ENV: &str = "OMNIGRAPH_BEARER_TOKEN"; - -#[derive(Debug, Parser)] -#[command(name = "omnigraph")] -#[command(about = "Omnigraph graph database CLI")] -#[command(version = env!("CARGO_PKG_VERSION"), disable_version_flag = true)] -struct Cli { - /// Actor identity for direct-engine writes (MR-722). Overrides - /// `cli.actor` from `omnigraph.yaml`. When the configured policy - /// is in effect, Cedar evaluates this actor against the requested - /// action and scope; with policy configured but neither this flag - /// nor `cli.actor` set, the engine-layer footgun guard fires and - /// the write is denied (no silent bypass). Has no effect on remote - /// HTTP writes — those resolve their actor server-side from the - /// bearer token. - #[arg(long = "as", global = true, value_name = "ACTOR")] - as_actor: Option<String>, - - #[command(subcommand)] - command: Command, -} - -#[derive(Debug, Subcommand)] -enum Command { - /// Print the CLI version - Version, - /// Generate, clean, or refresh explicit seed embeddings - Embed(EmbedArgs), - /// Initialize a new graph from a schema - Init { - #[arg(long)] - schema: PathBuf, - /// Graph URI (local path or s3://) - uri: String, - /// Overwrite existing schema artifacts at the URI. Without - /// this flag, init refuses to touch a URI that already holds - /// `_schema.pg`, `_schema.ir.json`, or `__schema_state.json` - /// — closes the re-init footgun (MR-668 follow-up). With the - /// flag, the operator opts in to destructive semantics. - #[arg(long)] - force: bool, - }, - /// Load data into a graph (local or remote) - Load { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - data: PathBuf, - /// Target branch (defaults to main). Without --from it must exist. - #[arg(long)] - branch: Option<String>, - /// Base branch to fork --branch from when it doesn't exist yet. - /// Without this flag a missing branch is an error, never a fork. - #[arg(long)] - from: Option<String>, - /// How existing rows are handled: overwrite | append | merge. - /// Required — overwrite is destructive, so there is no default. - #[arg(long)] - mode: CliLoadMode, - #[arg(long)] - json: bool, - }, - /// Deprecated alias of `load --from <base>` (defaults: --mode merge, --from main) - Ingest { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - data: PathBuf, - #[arg(long)] - branch: Option<String>, - #[arg(long)] - from: Option<String>, - #[arg(long, default_value = "merge")] - mode: CliLoadMode, - #[arg(long)] - json: bool, - }, - /// Branch operations - Branch { - #[command(subcommand)] - command: BranchCommand, - }, - /// Schema planning operations - Schema { - #[command(subcommand)] - command: SchemaCommand, - }, - /// Validate queries against a schema (offline) or repo (repo-backed). - /// - /// Canonical name is `lint` (matches the `omnigraph_compiler::lint` - /// module and the `OG-XXX-NNN` lint-code vocabulary). Replaces the - /// deprecated `omnigraph query lint` / `omnigraph query check` / - /// `omnigraph check` invocations — each is kept as an argv-level - /// shim that prints a one-line stderr warning and rewrites to - /// `omnigraph lint`. Aliases are deliberately *not* exposed via - /// clap's `visible_alias` because that would advertise two - /// equivalent canonical names, which agents emit interchangeably - /// (see MR-981). - Lint { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - query: PathBuf, - #[arg(long)] - schema: Option<PathBuf>, - #[arg(long)] - json: bool, - }, - /// Operate on the server-side stored-query registry (`queries:`). - Queries { - #[command(subcommand)] - command: QueriesCommand, - }, - /// Show graph snapshot - Snapshot { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - branch: Option<String>, - #[arg(long)] - json: bool, - }, - /// Export a full graph snapshot as JSONL - Export { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - branch: Option<String>, - #[arg(long, hide = true)] - jsonl: bool, - #[arg(long = "type")] - type_names: Vec<String>, - #[arg(long = "table")] - table_keys: Vec<String>, - }, - /// Commit history operations - Commit { - #[command(subcommand)] - command: CommitCommand, - }, - /// Execute a read query against a branch or snapshot. - /// - /// Canonical read endpoint. The previous name `omnigraph read` is - /// kept as a visible alias and prints a one-line deprecation warning - /// when used. Pairs with `omnigraph mutate` on the write side. - #[command(visible_alias = "read")] - Query { - /// Graph URI - #[arg(long)] - uri: Option<String>, - #[arg(hide = true)] - legacy_uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long, conflicts_with_all = ["query", "query_string"])] - alias: Option<String>, - #[arg(long, conflicts_with_all = ["alias", "query_string"])] - query: Option<PathBuf>, - /// Inline GQ source — alternative to `--query <path>` and `--alias <name>`. - #[arg(short = 'e', long = "query-string", value_name = "GQ", conflicts_with_all = ["query", "alias"])] - query_string: Option<String>, - #[arg(long)] - name: Option<String>, - #[command(flatten)] - params: ParamsArgs, - #[arg(long, conflicts_with = "snapshot")] - branch: Option<String>, - #[arg(long, conflicts_with = "branch")] - snapshot: Option<String>, - #[arg(long, conflicts_with = "json")] - format: Option<ReadOutputFormat>, - #[arg(long, conflicts_with = "format")] - json: bool, - #[arg()] - alias_args: Vec<String>, - }, - /// Execute a graph mutation query against a branch. - /// - /// Canonical mutation endpoint. The previous name `omnigraph change` - /// is kept as a visible alias and prints a one-line deprecation - /// warning when used. Pairs with `omnigraph query` on the read side. - #[command(visible_alias = "change")] - Mutate { - /// Graph URI - #[arg(long)] - uri: Option<String>, - #[arg(hide = true)] - legacy_uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long, conflicts_with_all = ["query", "query_string"])] - alias: Option<String>, - #[arg(long, conflicts_with_all = ["alias", "query_string"])] - query: Option<PathBuf>, - /// Inline GQ source — alternative to `--query <path>` and `--alias <name>`. - #[arg(short = 'e', long = "query-string", value_name = "GQ", conflicts_with_all = ["query", "alias"])] - query_string: Option<String>, - #[arg(long)] - name: Option<String>, - #[command(flatten)] - params: ParamsArgs, - #[arg(long)] - branch: Option<String>, - #[arg(long)] - json: bool, - #[arg()] - alias_args: Vec<String>, - }, - /// Policy administration and diagnostics - Policy { - #[command(subcommand)] - command: PolicyCommand, - }, - /// Compact small Lance fragments in every table of the graph - Optimize { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - json: bool, - }, - /// Classify and explicitly repair manifest/head drift - Repair { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - /// Publish verified maintenance drift. Without this flag, repair only - /// previews what it would do. - #[arg(long)] - confirm: bool, - /// Also publish suspicious or unverifiable drift. Requires - /// `--confirm`; use only after operator review. - #[arg(long, requires = "confirm")] - force: bool, - #[arg(long)] - json: bool, - }, - /// Remove old Lance versions from every table of the graph (destructive) - Cleanup { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - /// Number of recent versions to keep per table. Either `--keep` or - /// `--older-than` (or both) must be set. - #[arg(long)] - keep: Option<u32>, - /// Only remove versions older than this duration. Accepts Go-style - /// durations: `7d`, `24h`, `90m`. At least one of --keep / --older-than. - #[arg(long)] - older_than: Option<String>, - /// Required to actually run; without it, prints what would be removed - #[arg(long)] - confirm: bool, - #[arg(long)] - json: bool, - }, - /// Validate and plan read-only cluster configuration. - Cluster { - #[command(subcommand)] - command: ClusterCommand, - }, - /// Manage graphs on a multi-graph server (MR-668) - Graphs { - #[command(subcommand)] - command: GraphsCommand, - }, -} - -#[derive(Debug, Subcommand)] -enum ClusterCommand { - /// Validate cluster.yaml and referenced schemas, queries, and policy files. - Validate { - /// Cluster config directory containing cluster.yaml. - #[arg(long, default_value = ".")] - config: PathBuf, - /// Emit JSON instead of human text. - #[arg(long)] - json: bool, - }, - /// Produce a read-only plan by diffing cluster.yaml against __cluster/state.json. - Plan { - /// Cluster config directory containing cluster.yaml. - #[arg(long, default_value = ".")] - config: PathBuf, - /// Emit JSON instead of human text. - #[arg(long)] - json: bool, - }, - /// Apply the config-only (query/policy) subset of the plan to the local - /// cluster catalog. Graph/schema changes are deferred to a later stage. - Apply { - /// Cluster config directory containing cluster.yaml. - #[arg(long, default_value = ".")] - config: PathBuf, - /// Emit JSON instead of human text. - #[arg(long)] - json: bool, - }, - /// Record a digest-bound approval for a gated (irreversible) change, - /// e.g. a graph delete. Requires the global --as actor. - Approve { - /// Typed resource address of the gated change (e.g. graph.scratch). - resource: String, - /// Cluster config directory containing cluster.yaml. - #[arg(long, default_value = ".")] - config: PathBuf, - /// Emit JSON instead of human text. - #[arg(long)] - json: bool, - }, - /// Read the local JSON state ledger without scanning live graph resources. - Status { - /// Cluster config directory containing cluster.yaml. - #[arg(long, default_value = ".")] - config: PathBuf, - /// Emit JSON instead of human text. - #[arg(long)] - json: bool, - }, - /// Refresh existing local JSON state from declared graph observations. - Refresh { - /// Cluster config directory containing cluster.yaml. - #[arg(long, default_value = ".")] - config: PathBuf, - /// Emit JSON instead of human text. - #[arg(long)] - json: bool, - }, - /// Import initial local JSON state from declared graph observations. - Import { - /// Cluster config directory containing cluster.yaml. - #[arg(long, default_value = ".")] - config: PathBuf, - /// Emit JSON instead of human text. - #[arg(long)] - json: bool, - }, - /// Remove a held local JSON state lock after operator confirmation. - ForceUnlock { - /// Exact lock id from cluster status or a state_lock_held diagnostic. - lock_id: String, - /// Cluster config directory containing cluster.yaml. - #[arg(long, default_value = ".")] - config: PathBuf, - /// Emit JSON instead of human text. - #[arg(long)] - json: bool, - }, -} - -/// Operations on the graph registry of a multi-graph server (MR-668). -/// -/// All operations target a remote multi-graph server URL (http:// or -/// https://). Local-URI invocations return a clear error. To add or -/// remove graphs, operators edit `omnigraph.yaml` directly and restart -/// the server — runtime mutation is not exposed in v0.6.0. -#[derive(Debug, Subcommand)] -enum GraphsCommand { - /// List every graph registered with the multi-graph server. - List { - /// Remote server URL (e.g. `https://server.example.com`). - #[arg(long)] - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - json: bool, - }, -} - -#[derive(Debug, Subcommand)] -enum BranchCommand { - /// Create a new branch - Create { - /// Graph URI - #[arg(long)] - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - from: Option<String>, - name: String, - #[arg(long)] - json: bool, - }, - /// List branches - List { - /// Graph URI - #[arg(long)] - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - json: bool, - }, - /// Delete a branch - Delete { - /// Graph URI - #[arg(long)] - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - name: String, - #[arg(long)] - json: bool, - }, - /// Merge a source branch into a target branch - Merge { - /// Graph URI - #[arg(long)] - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - source: String, - #[arg(long)] - into: Option<String>, - #[arg(long)] - json: bool, - }, -} - -#[derive(Debug, Subcommand)] -enum SchemaCommand { - /// Plan a schema migration against the accepted persisted schema - Plan { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - schema: PathBuf, - #[arg(long)] - json: bool, - /// Show the plan as it would execute with `--allow-data-loss`. - /// Promotes every `DropMode::Soft` step to `DropMode::Hard` - /// so the plan output reflects the destructive intent. - #[arg(long, default_value_t = false)] - allow_data_loss: bool, - }, - /// Apply a supported schema migration - Apply { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - schema: PathBuf, - #[arg(long)] - json: bool, - /// Allow destructive (data-loss) schema changes. - /// - /// Without this flag, drops are "soft": the column or table - /// is removed from the current manifest version but prior - /// versions are retained, so `snapshot_at_version(pre_drop)` - /// can still read the dropped data until `omnigraph cleanup` - /// runs. With this flag, drops are "hard": `cleanup_old_versions` - /// runs on the affected datasets immediately after the apply, - /// making the prior data unreachable. - #[arg(long, default_value_t = false)] - allow_data_loss: bool, - }, - /// Show the current accepted schema source - #[command(alias = "get")] - Show { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - json: bool, - }, -} - -#[derive(Debug, Subcommand)] - -enum CommitCommand { - /// List graph commits - List { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - branch: Option<String>, - #[arg(long)] - json: bool, - }, - /// Show a graph commit - Show { - /// Graph URI - #[arg(long)] - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - commit_id: String, - #[arg(long)] - json: bool, - }, -} - -#[derive(Debug, Subcommand)] -enum PolicyCommand { - /// Validate policy YAML and compiled Cedar policy state - Validate { - #[arg(long)] - config: Option<PathBuf>, - }, - /// Run declarative policy tests from policy.tests.yaml - Test { - #[arg(long)] - config: Option<PathBuf>, - }, - /// Explain one policy decision locally - Explain { - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - actor: String, - #[arg(long)] - action: PolicyAction, - #[arg(long)] - branch: Option<String>, - #[arg(long = "target-branch")] - target_branch: Option<String>, - }, -} - -#[derive(Debug, Subcommand)] -enum QueriesCommand { - /// Type-check the stored-query registry against the live schema. - /// - /// Distinct from `omnigraph lint` (which lints one `.gq` file): - /// this validates the whole `queries:` registry — opening the graph - /// to read its schema and confirming every stored query still - /// type-checks. Exits non-zero on any breakage. - Validate { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - json: bool, - }, - /// List the registered stored queries (name, MCP exposure, params). - List { - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - json: bool, - }, -} - -#[derive(Debug, Args, Clone)] -struct ParamsArgs { - #[arg(long, conflicts_with = "params_file")] - params: Option<String>, - #[arg(long, conflicts_with = "params")] - params_file: Option<PathBuf>, -} - -#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize, ValueEnum)] -#[serde(rename_all = "snake_case")] -enum CliLoadMode { - Overwrite, - Append, - Merge, -} - -impl From<CliLoadMode> for LoadMode { - fn from(value: CliLoadMode) -> Self { - match value { - CliLoadMode::Overwrite => LoadMode::Overwrite, - CliLoadMode::Append => LoadMode::Append, - CliLoadMode::Merge => LoadMode::Merge, - } - } -} - -impl CliLoadMode { - fn as_str(self) -> &'static str { - match self { - CliLoadMode::Overwrite => "overwrite", - CliLoadMode::Append => "append", - CliLoadMode::Merge => "merge", - } - } -} - -#[derive(Debug, Serialize)] -struct LoadOutput { - uri: String, - branch: String, - mode: &'static str, - /// Present only when `--from` was given; echoes the requested base. - #[serde(skip_serializing_if = "Option::is_none")] - base_branch: Option<String>, - branch_created: bool, - nodes_loaded: usize, - edges_loaded: usize, - node_types_loaded: usize, - edge_types_loaded: usize, -} - -/// Map a remote `/ingest` response onto the CLI's load output. Table keys -/// carry `node:`/`edge:` prefixes, so the per-kind sums are derivable -/// client-side without the catalog. -fn load_output_from_tables( - uri: &str, - branch: &str, - mode: CliLoadMode, - output: &IngestOutput, -) -> LoadOutput { - let mut nodes_loaded = 0; - let mut edges_loaded = 0; - let mut node_types_loaded = 0; - let mut edge_types_loaded = 0; - for table in &output.tables { - if table.table_key.starts_with("node:") { - nodes_loaded += table.rows_loaded; - node_types_loaded += 1; - } else if table.table_key.starts_with("edge:") { - edges_loaded += table.rows_loaded; - edge_types_loaded += 1; - } - } - LoadOutput { - uri: uri.to_string(), - branch: branch.to_string(), - mode: mode.as_str(), - base_branch: output.base_branch.clone(), - branch_created: output.branch_created, - nodes_loaded, - edges_loaded, - node_types_loaded, - edge_types_loaded, - } -} - -#[derive(Debug, Serialize)] -struct SchemaPlanOutput<'a> { - uri: &'a str, - supported: bool, - step_count: usize, - steps: &'a [SchemaMigrationStep], -} - -fn print_schema_apply_human(output: &SchemaApplyOutput) { - println!("schema apply for {}", output.uri); - println!("supported: {}", if output.supported { "yes" } else { "no" }); - println!("applied: {}", if output.applied { "yes" } else { "no" }); - println!("manifest_version: {}", output.manifest_version); - if output.steps.is_empty() { - println!("no schema changes"); - return; - } - for step in &output.steps { - println!("- {}", render_schema_plan_step(step)); - } -} - -fn query_kind_label(kind: QueryLintQueryKind) -> &'static str { - match kind { - QueryLintQueryKind::Read => "read", - QueryLintQueryKind::Mutation => "mutation", - } -} - -fn severity_label(severity: QueryLintSeverity) -> &'static str { - match severity { - QueryLintSeverity::Error => "ERROR", - QueryLintSeverity::Warning => "WARN ", - QueryLintSeverity::Info => "INFO ", - } -} - -fn print_query_lint_human(output: &QueryLintOutput) { - for result in &output.results { - match result.status { - QueryLintStatus::Ok => { - println!( - "OK query `{}` ({})", - result.name, - query_kind_label(result.kind) - ); - } - QueryLintStatus::Error => { - println!( - "ERROR query `{}`: {}", - result.name, - result.error.as_deref().unwrap_or("unknown error") - ); - } - } - - for warning in &result.warnings { - println!("WARN query `{}`: {}", result.name, warning); - } - } - - for finding in &output.findings { - println!("{} {}", severity_label(finding.severity), finding.message); - } - - println!( - "INFO Lint complete: {} queries processed ({} error(s), {} warning(s), {} info item(s))", - output.queries_processed, output.errors, output.warnings, output.infos - ); -} - -fn finish_query_lint(output: &QueryLintOutput, json: bool) -> Result<()> { - if json { - print_json(output)?; - } else { - print_query_lint_human(output); - } - - if output.status == QueryLintStatus::Error { - io::stdout().flush()?; - std::process::exit(1); - } - - Ok(()) -} - -fn ensure_local_graph_parent(uri: &str) -> Result<()> { - if !uri.contains("://") { - fs::create_dir_all(uri)?; - } - Ok(()) -} - -fn print_json<T: Serialize>(value: &T) -> Result<()> { - println!("{}", serde_json::to_string_pretty(value)?); - Ok(()) -} - -fn print_cluster_validate_human(output: &ValidateOutput) { - if output.ok { - println!( - "cluster config valid: {} resource(s), {} dependency edge(s)", - output.resources.len(), - output.dependencies.len() - ); - } else { - println!("cluster config invalid"); - } - print_cluster_diagnostics(&output.diagnostics); -} - -fn print_cluster_plan_human(output: &PlanOutput) { - if output.ok { - println!( - "cluster plan: {} change(s), {} approval gate(s)", - output.changes.len(), - output.approvals_required.len() - ); - for change in &output.changes { - let bindings = if change.binding_change { " [bindings]" } else { "" }; - println!(" {:?} {}{bindings}", change.operation, change.resource); - if let Some(migration) = &change.migration { - if !migration.supported { - println!(" migration UNSUPPORTED:"); - } - for step in &migration.steps { - println!( - " {}", - serde_json::to_string(step).unwrap_or_else(|_| format!("{step:?}")) - ); - } - } - } - if output.changes.is_empty() { - println!(" no changes"); - } - } else { - println!("cluster plan failed"); - } - print_cluster_diagnostics(&output.diagnostics); -} - -fn print_cluster_apply_human(output: &ApplyOutput) { - if output.ok { - println!( - "cluster apply: {} applied, {} deferred/blocked", - output.applied_count, output.deferred_count - ); - } else { - println!("cluster apply failed"); - } - // The change list prints on failure too: an operator debugging a partial - // apply (payload or state-write error) needs to see what was attempted. - print_cluster_apply_changes(&output.changes); - if output.ok { - let state = &output.state_observations; - println!( - " state: revision {}, converged: {}, written: {}", - state.state_revision, output.converged, output.state_written - ); - println!(" note: cluster-booted servers (--cluster) serve this on their next restart; omnigraph.yaml deployments are unaffected"); - } - print_cluster_diagnostics(&output.diagnostics); -} - -fn print_cluster_apply_changes(changes: &[omnigraph_cluster::PlanChange]) { - for change in changes { - let bindings = if change.binding_change { " [bindings]" } else { "" }; - match (&change.disposition, change.reason.as_deref()) { - (Some(disposition), Some(reason)) => println!( - " {:?} {}{bindings} [{disposition:?}: {reason}]", - change.operation, change.resource - ), - (Some(disposition), None) => println!( - " {:?} {}{bindings} [{disposition:?}]", - change.operation, change.resource - ), - _ => println!(" {:?} {}{bindings}", change.operation, change.resource), - } - } - if changes.is_empty() { - println!(" no changes"); - } -} - -fn print_cluster_status_human(output: &StatusOutput) { - if output.ok { - let state = &output.state_observations; - if state.state_found { - println!( - "cluster state: revision {}, {} resource(s)", - state.state_revision, state.resource_count - ); - if let Some(digest) = state.applied_config_digest.as_deref() { - println!(" applied config: {digest}"); - } - if state.locked { - println!(" lock: held{}", cluster_lock_summary(state)); - } else { - println!(" lock: not held"); - } - } else { - println!("cluster state missing"); - } - } else { - println!("cluster status failed"); - } - print_cluster_diagnostics(&output.diagnostics); -} - -fn print_cluster_state_sync_human(output: &StateSyncOutput) { - let operation = match output.operation { - omnigraph_cluster::StateSyncOperation::Refresh => "refresh", - omnigraph_cluster::StateSyncOperation::Import => "import", - }; - if output.ok { - let state = &output.state_observations; - println!( - "cluster {operation}: revision {}, {} resource(s)", - state.state_revision, state.resource_count - ); - if let Some(cas) = state.state_cas.as_deref() { - println!(" state_cas: {cas}"); - } - if state.locked { - println!(" lock: acquired{}", cluster_lock_summary(state)); - } else { - println!(" lock: not acquired"); - } - } else { - println!("cluster {operation} failed"); - } - print_cluster_diagnostics(&output.diagnostics); -} - -fn print_cluster_force_unlock_human(output: &ForceUnlockOutput) { - if output.ok { - if output.lock_removed { - println!( - "cluster force-unlock: removed lock{}", - cluster_lock_summary(&output.state_observations) - ); - } else { - println!("cluster force-unlock: no lock removed"); - } - } else { - println!("cluster force-unlock failed"); - if output.state_observations.locked { - println!( - " lock: held{}", - cluster_lock_summary(&output.state_observations) - ); - } - } - print_cluster_diagnostics(&output.diagnostics); -} - -fn cluster_lock_summary(state: &omnigraph_cluster::StateObservations) -> String { - let Some(lock_id) = state.lock_id.as_deref() else { - return String::new(); - }; - let mut parts = vec![format!("id={lock_id}")]; - if let Some(operation) = state.lock_operation.as_deref() { - parts.push(format!("operation={operation}")); - } - if let Some(pid) = state.lock_pid { - parts.push(format!("pid={pid}")); - } - if let Some(created_at) = state.lock_created_at.as_deref() { - parts.push(format!("created_at={created_at}")); - } - if let Some(age_seconds) = state.lock_age_seconds { - parts.push(format!("age_seconds={age_seconds}")); - } - format!(" ({})", parts.join(", ")) -} - -fn print_cluster_diagnostics(diagnostics: &[omnigraph_cluster::Diagnostic]) { - for diagnostic in diagnostics { - let label = match diagnostic.severity { - DiagnosticSeverity::Error => "ERROR", - DiagnosticSeverity::Warning => "WARN ", - }; - println!( - "{label} {} {}: {}", - diagnostic.code, diagnostic.path, diagnostic.message - ); - } -} - -fn finish_cluster_validate(output: &ValidateOutput, json: bool) -> Result<()> { - if json { - print_json(output)?; - } else { - print_cluster_validate_human(output); - } - if !output.ok { - io::stdout().flush()?; - std::process::exit(1); - } - Ok(()) -} - -fn finish_cluster_plan(output: &PlanOutput, json: bool) -> Result<()> { - if json { - print_json(output)?; - } else { - print_cluster_plan_human(output); - } - if !output.ok { - io::stdout().flush()?; - std::process::exit(1); - } - Ok(()) -} - -fn finish_cluster_apply(output: &ApplyOutput, json: bool) -> Result<()> { - if json { - print_json(output)?; - } else { - print_cluster_apply_human(output); - } - if !output.ok { - io::stdout().flush()?; - std::process::exit(1); - } - Ok(()) -} - -fn finish_cluster_approve(output: &ApproveOutput, json: bool) -> Result<()> { - if json { - print_json(output)?; - } else if output.ok { - println!( - "cluster approve: {} {} approved by {} (approval {})", - output - .operation - .as_ref() - .map(|operation| format!("{operation:?}").to_lowercase()) - .unwrap_or_default(), - output.resource.as_deref().unwrap_or("?"), - output.approved_by.as_deref().unwrap_or("?"), - output.approval_id.as_deref().unwrap_or("?"), - ); - print_cluster_diagnostics(&output.diagnostics); - } else { - println!("cluster approve failed"); - print_cluster_diagnostics(&output.diagnostics); - } - if !output.ok { - io::stdout().flush()?; - std::process::exit(1); - } - Ok(()) -} - -fn finish_cluster_status(output: &StatusOutput, json: bool) -> Result<()> { - if json { - print_json(output)?; - } else { - print_cluster_status_human(output); - } - if !output.ok { - io::stdout().flush()?; - std::process::exit(1); - } - Ok(()) -} - -fn finish_cluster_state_sync(output: &StateSyncOutput, json: bool) -> Result<()> { - if json { - print_json(output)?; - } else { - print_cluster_state_sync_human(output); - } - if !output.ok { - io::stdout().flush()?; - std::process::exit(1); - } - Ok(()) -} - -fn finish_cluster_force_unlock(output: &ForceUnlockOutput, json: bool) -> Result<()> { - if json { - print_json(output)?; - } else { - print_cluster_force_unlock_human(output); - } - if !output.ok { - io::stdout().flush()?; - std::process::exit(1); - } - Ok(()) -} - -fn is_remote_uri(uri: &str) -> bool { - uri.starts_with("http://") || uri.starts_with("https://") -} - -fn remote_url(base: &str, path: &str) -> String { - format!("{}{}", base.trim_end_matches('/'), path) -} - -fn remote_branch_url(base: &str, branch: &str) -> Result<String> { - let mut url = reqwest::Url::parse(&format!("{}/", base.trim_end_matches('/')))?; - url.path_segments_mut() - .map_err(|_| color_eyre::eyre::eyre!("invalid remote base url"))? - .extend(["branches", branch]); - Ok(url.to_string()) -} - -fn normalize_bearer_token(value: Option<String>) -> Option<String> { - value - .map(|value| value.trim().to_string()) - .filter(|value| !value.is_empty()) -} - -fn bearer_token_from_env(var_name: &str) -> Option<String> { - normalize_bearer_token(std::env::var(var_name).ok()) -} - -fn parse_env_assignment(line: &str) -> Option<(String, String)> { - let line = line.trim(); - if line.is_empty() || line.starts_with('#') { - return None; - } - - let line = line.strip_prefix("export ").unwrap_or(line).trim(); - let (name, value) = line.split_once('=')?; - let name = name.trim(); - if name.is_empty() { - return None; - } - - let value = value.trim(); - let value = if value.len() >= 2 - && ((value.starts_with('"') && value.ends_with('"')) - || (value.starts_with('\'') && value.ends_with('\''))) - { - &value[1..value.len() - 1] - } else { - value - }; - - Some((name.to_string(), value.to_string())) -} - -fn bearer_token_from_env_file(path: &Path, var_name: &str) -> Result<Option<String>> { - if !path.exists() { - return Ok(None); - } - - for line in fs::read_to_string(path)?.lines() { - let Some((name, value)) = parse_env_assignment(line) else { - continue; - }; - if name == var_name { - return Ok(normalize_bearer_token(Some(value))); - } - } - - Ok(None) -} - -fn load_env_file_into_process(path: &Path) -> Result<()> { - if !path.exists() { - return Ok(()); - } - - for line in fs::read_to_string(path)?.lines() { - let Some((name, value)) = parse_env_assignment(line) else { - continue; - }; - if std::env::var_os(&name).is_none() { - unsafe { - std::env::set_var(name, value); - } - } - } - - Ok(()) -} - -fn load_cli_config(config_path: Option<&PathBuf>) -> Result<OmnigraphConfig> { - let config = load_config(config_path)?; - if let Some(path) = config.resolve_auth_env_file() { - load_env_file_into_process(&path)?; - } - Ok(config) -} - -#[derive(Debug, Clone)] -struct ResolvedCliGraph { - uri: String, - selected: Option<String>, - graph_id: String, - policy_file: Option<PathBuf>, - is_remote: bool, -} - -impl ResolvedCliGraph { - fn selected(&self) -> Option<&str> { - self.selected.as_deref() - } -} - -struct ResolvedPolicyContext { - policy_file: PathBuf, - graph_id: String, -} - -fn resolve_policy_context(config: &OmnigraphConfig) -> Result<ResolvedPolicyContext> { - let selected = config.resolve_policy_tooling_graph_selection()?; - let policy_file = config.resolve_policy_file_for(selected).ok_or_else(|| { - color_eyre::eyre::eyre!( - "policy.file or graphs.<name>.policy.file must be set in omnigraph.yaml" - ) - })?; - let graph_id = match selected { - Some(name) => graph_resource_id_for_selection(Some(name), ""), - None => graph_resource_id_for_selection(None, "default"), - }; - Ok(ResolvedPolicyContext { - policy_file, - graph_id, - }) -} - -fn resolve_policy_engine(context: &ResolvedPolicyContext) -> Result<PolicyEngine> { - PolicyEngine::load_graph(&context.policy_file, &context.graph_id) -} - -fn resolve_policy_engine_for_graph(graph: &ResolvedCliGraph) -> Result<PolicyEngine> { - let policy_file = graph.policy_file.as_ref().ok_or_else(|| { - color_eyre::eyre::eyre!( - "policy.file or graphs.<name>.policy.file must be set in omnigraph.yaml" - ) - })?; - PolicyEngine::load_graph(policy_file, &graph.graph_id) -} - -/// Open a local graph and install the policy resolved for the same graph -/// identity that produced the URI. A named graph uses -/// `graphs.<name>.policy.file`; an explicit positional URI is anonymous and -/// uses the legacy top-level `policy.file`. -async fn open_local_db_with_policy(graph: &ResolvedCliGraph) -> Result<Omnigraph> { - let db = Omnigraph::open(&graph.uri).await?; - if graph.policy_file.is_some() { - let engine = Arc::new(resolve_policy_engine_for_graph(graph)?); - Ok(db.with_policy(engine as Arc<dyn omnigraph_policy::PolicyChecker>)) - } else { - Ok(db) - } -} - -/// Actor resolution for cluster operations. Cluster FACTS stay unlayered -/// (cluster.yaml only), but the operator's identity is a per-operator fact — -/// the per-operator config's permanent job. An explicit --as never touches -/// any config (containers and CI stay config-free); without it, the standard -/// cwd omnigraph.yaml search supplies `cli.actor`, and a malformed config -/// fails loudly rather than silently dropping attribution. Deliberately -/// `load_config`, NOT `load_cli_config`: the latter also loads -/// `auth.env_file` into the process env — a second thing, violating the -/// documented "exactly one thing" contract. -fn resolve_cluster_actor(cli_as: Option<&str>) -> Result<Option<String>> { - if let Some(actor) = cli_as { - return Ok(Some(actor.to_string())); - } - let config = load_config(None).wrap_err( - "resolving the default actor from the per-operator omnigraph.yaml (pass --as <ACTOR> to skip this lookup)", - )?; - Ok(config.cli.actor.clone()) -} - -/// Resolve the CLI's effective actor identity for engine-layer policy -/// (MR-722). Precedence: `--as <ACTOR>` (top-level flag) overrides -/// `cli.actor` from `omnigraph.yaml`; both unset returns `None`. When -/// policy is configured and this returns `None`, the engine-layer -/// footgun guard intentionally denies — silent bypass via "I forgot the -/// actor" is what the guard prevents. -fn resolve_cli_actor<'a>(cli_as: Option<&'a str>, config: &'a OmnigraphConfig) -> Option<&'a str> { - cli_as.or(config.cli.actor.as_deref()) -} - -fn resolve_policy_tests_path(context: &ResolvedPolicyContext) -> PathBuf { - context.policy_file.with_file_name("policy.tests.yaml") -} - -fn normalize_policy_graph_uri(uri: &str) -> Result<String> { - if is_remote_uri(uri) { - Ok(uri.trim_end_matches('/').to_string()) - } else { - Ok(normalize_root_uri(uri)?) - } -} - -fn resolve_remote_bearer_token( - config: &OmnigraphConfig, - explicit_uri: Option<&str>, - explicit_target: Option<&str>, -) -> Result<Option<String>> { - let scoped_env = - config.graph_bearer_token_env(explicit_uri, explicit_target, config.cli_graph_name()); - let mut env_names = Vec::new(); - if let Some(name) = scoped_env { - env_names.push(name.to_string()); - } - if env_names - .iter() - .all(|name| name != DEFAULT_BEARER_TOKEN_ENV) - { - env_names.push(DEFAULT_BEARER_TOKEN_ENV.to_string()); - } - - let env_file = config.resolve_auth_env_file(); - for env_name in env_names { - if let Some(token) = bearer_token_from_env(&env_name) { - return Ok(Some(token)); - } - if let Some(path) = env_file.as_ref() { - if let Some(token) = bearer_token_from_env_file(path, &env_name)? { - return Ok(Some(token)); - } - } - } - - Ok(None) -} - -fn build_http_client() -> Result<reqwest::Client> { - Ok(reqwest::Client::new()) -} - -fn apply_bearer_token( - request: reqwest::RequestBuilder, - token: Option<&str>, -) -> reqwest::RequestBuilder { - if let Some(token) = token { - request.header(AUTHORIZATION, format!("Bearer {}", token)) - } else { - request - } -} - -async fn remote_json<T: DeserializeOwned>( - client: &reqwest::Client, - method: Method, - url: String, - body: Option<Value>, - bearer_token: Option<&str>, -) -> Result<T> { - let request = apply_bearer_token(client.request(method, url), bearer_token); - let request = if let Some(body) = body { - request.json(&body) - } else { - request - }; - let response = request.send().await?; - let status = response.status(); - let text = response.text().await?; - if !status.is_success() { - if let Ok(error) = serde_json::from_str::<ErrorOutput>(&text) { - bail!(error.error); - } - bail!("server returned {}: {}", status, text); - } - Ok(serde_json::from_str(&text)?) -} - -fn resolve_uri( - config: &OmnigraphConfig, - cli_uri: Option<String>, - cli_target: Option<&str>, -) -> Result<String> { - config.resolve_target_uri(cli_uri, cli_target, config.cli_graph_name()) -} - -fn resolve_cli_graph( - config: &OmnigraphConfig, - cli_uri: Option<String>, - cli_target: Option<&str>, -) -> Result<ResolvedCliGraph> { - let selected = if cli_uri.is_some() { - None - } else { - cli_target - .map(str::to_string) - .or_else(|| config.cli_graph_name().map(str::to_string)) - }; - config.resolve_graph_selection(selected.as_deref())?; - let uri = resolve_uri(config, cli_uri, cli_target)?; - let normalized_uri = normalize_policy_graph_uri(&uri)?; - let graph_id = graph_resource_id_for_selection(selected.as_deref(), &normalized_uri); - Ok(ResolvedCliGraph { - graph_id, - is_remote: is_remote_uri(&uri), - policy_file: config.resolve_policy_file_for(selected.as_deref()), - selected, - uri, - }) -} - -fn resolve_local_graph( - config: &OmnigraphConfig, - cli_uri: Option<String>, - cli_target: Option<&str>, - operation: &str, -) -> Result<ResolvedCliGraph> { - let graph = resolve_cli_graph(config, cli_uri, cli_target)?; - if graph.is_remote { - bail!( - "{} is only supported against local graph URIs in this milestone", - operation - ); - } - Ok(graph) -} - -/// Parse a Go-style compact duration: `7d`, `24h`, `30m`, `90s`, or a plain -/// integer as seconds. Used by the `cleanup --older-than` flag. -fn parse_duration_arg(s: &str) -> Result<std::time::Duration> { - let s = s.trim(); - if s.is_empty() { - bail!("duration is empty"); - } - let (num_part, unit) = match s - .char_indices() - .rev() - .find(|(_, c)| c.is_ascii_alphabetic()) - { - Some((i, _)) => ( - &s[..i + 1 - s[i..].chars().next().unwrap().len_utf8()], - &s[i..], - ), - None => (s, ""), - }; - let n: u64 = num_part - .parse() - .map_err(|e| color_eyre::eyre::eyre!("invalid duration '{}': {}", s, e))?; - let secs = match unit { - "" | "s" => n, - "m" => n * 60, - "h" => n * 60 * 60, - "d" => n * 60 * 60 * 24, - "w" => n * 60 * 60 * 24 * 7, - _ => bail!("unknown duration unit '{}'. Supported: s, m, h, d, w", unit), - }; - Ok(std::time::Duration::from_secs(secs)) -} - -fn resolve_local_uri( - config: &OmnigraphConfig, - cli_uri: Option<String>, - cli_target: Option<&str>, - operation: &str, -) -> Result<String> { - Ok(resolve_local_graph(config, cli_uri, cli_target, operation)?.uri) -} - -fn resolve_branch( - config: &OmnigraphConfig, - cli_branch: Option<String>, - alias_branch: Option<String>, - default_branch: &str, -) -> String { - cli_branch - .or(alias_branch) - .or_else(|| config.cli.branch.clone()) - .unwrap_or_else(|| default_branch.to_string()) -} - -fn resolve_read_target( - config: &OmnigraphConfig, - cli_branch: Option<String>, - cli_snapshot: Option<String>, - alias_branch: Option<String>, -) -> Result<ReadTarget> { - if cli_branch.is_some() && cli_snapshot.is_some() { - bail!("read target may specify branch or snapshot, not both"); - } - Ok(read_target_from_cli( - cli_branch - .or(alias_branch) - .or_else(|| config.cli.branch.clone()), - cli_snapshot, - )) -} - -fn resolve_query_path( - config: &OmnigraphConfig, - explicit_query: Option<&PathBuf>, - alias_query: Option<&str>, -) -> Result<PathBuf> { - explicit_query - .map(PathBuf::from) - .or_else(|| alias_query.map(PathBuf::from)) - .ok_or_else(|| { - color_eyre::eyre::eyre!( - "exactly one of --query, --query-string, or --alias must be provided" - ) - }) - .and_then(|query_path| config.resolve_query_path(&query_path)) -} - -fn resolve_query_source( - config: &OmnigraphConfig, - explicit_query: Option<&PathBuf>, - inline_query: Option<&str>, - alias_query: Option<&str>, -) -> Result<String> { - if let Some(inline) = inline_query { - if inline.trim().is_empty() { - bail!("--query-string must not be empty"); - } - return Ok(inline.to_string()); - } - Ok(fs::read_to_string(resolve_query_path( - config, - explicit_query, - alias_query, - )?)?) -} - -fn parse_alias_value(value: &str) -> Value { - serde_json::from_str(value).unwrap_or_else(|_| Value::String(value.to_string())) -} - -fn merged_params_json( - alias_name: Option<&str>, - alias_arg_names: &[String], - alias_arg_values: &[String], - explicit: Option<Value>, -) -> Result<Option<Value>> { - if alias_arg_values.len() > alias_arg_names.len() { - let alias = alias_name.unwrap_or("<alias>"); - bail!( - "alias '{}' expects at most {} args but got {}", - alias, - alias_arg_names.len(), - alias_arg_values.len() - ); - } - - let mut merged = serde_json::Map::new(); - for (arg_name, arg_value) in alias_arg_names.iter().zip(alias_arg_values.iter()) { - merged.insert(arg_name.clone(), parse_alias_value(arg_value)); - } - - match explicit { - Some(Value::Object(object)) => { - for (key, value) in object { - merged.insert(key, value); - } - } - Some(_) => bail!("params JSON must be an object"), - None => {} - } - - if merged.is_empty() { - Ok(None) - } else { - Ok(Some(Value::Object(merged))) - } -} - -fn print_load_human(payload: &LoadOutput) { - println!( - "loaded {} on branch {} with {}: {} nodes across {} node types, {} edges across {} edge types", - payload.uri, - payload.branch, - payload.mode, - payload.nodes_loaded, - payload.node_types_loaded, - payload.edges_loaded, - payload.edge_types_loaded - ); - if payload.branch_created { - if let Some(base) = &payload.base_branch { - println!("branch {} created from {}", payload.branch, base); - } - } -} - -fn print_ingest_human(output: &IngestOutput) { - println!( - "ingested {} into branch {} from {} with {} ({})", - output.uri, - output.branch, - output.base_branch.as_deref().unwrap_or("main"), - output.mode.as_str(), - if output.branch_created { - "branch created" - } else { - "branch exists" - } - ); - for table in &output.tables { - println!("{} rows_loaded={}", table.table_key, table.rows_loaded); - } - if let Some(actor_id) = &output.actor_id { - println!("actor_id: {}", actor_id); - } -} - -fn print_schema_plan_human(uri: &str, plan: &SchemaMigrationPlan) { - println!("schema plan for {}", uri); - println!("supported: {}", if plan.supported { "yes" } else { "no" }); - if plan.steps.is_empty() { - println!("no schema changes"); - return; - } - for step in &plan.steps { - println!("- {}", render_schema_plan_step(step)); - } -} - -fn render_schema_plan_step(step: &SchemaMigrationStep) -> String { - match step { - SchemaMigrationStep::AddType { type_kind, name } => { - format!("add {} type '{}'", schema_type_kind_label(*type_kind), name) - } - SchemaMigrationStep::RenameType { - type_kind, - from, - to, - } => format!( - "rename {} type '{}' -> '{}'", - schema_type_kind_label(*type_kind), - from, - to - ), - SchemaMigrationStep::AddProperty { - type_kind, - type_name, - property_name, - property_type, - } => format!( - "add property '{}.{}' ({}) on {} '{}'", - type_name, - property_name, - render_prop_type(property_type), - schema_type_kind_label(*type_kind), - type_name - ), - SchemaMigrationStep::RenameProperty { - type_kind, - type_name, - from, - to, - } => format!( - "rename property '{}.{}' -> '{}.{}' on {} '{}'", - type_name, - from, - type_name, - to, - schema_type_kind_label(*type_kind), - type_name - ), - SchemaMigrationStep::AddConstraint { - type_kind, - type_name, - constraint, - } => format!( - "add constraint {} on {} '{}'", - render_constraint(constraint), - schema_type_kind_label(*type_kind), - type_name - ), - SchemaMigrationStep::UpdateTypeMetadata { - type_kind, - name, - annotations, - } => format!( - "update metadata on {} '{}' ({})", - schema_type_kind_label(*type_kind), - name, - render_annotations(annotations) - ), - SchemaMigrationStep::UpdatePropertyMetadata { - type_kind, - type_name, - property_name, - annotations, - } => format!( - "update metadata on property '{}.{}' of {} '{}' ({})", - type_name, - property_name, - schema_type_kind_label(*type_kind), - type_name, - render_annotations(annotations) - ), - SchemaMigrationStep::DropType { - type_kind, - name, - mode, - } => format!( - "drop {} type '{}' ({} mode)", - schema_type_kind_label(*type_kind), - name, - drop_mode_label(*mode), - ), - SchemaMigrationStep::DropProperty { - type_kind, - type_name, - property_name, - mode, - } => format!( - "drop property '{}.{}' of {} '{}' ({} mode)", - type_name, - property_name, - schema_type_kind_label(*type_kind), - type_name, - drop_mode_label(*mode), - ), - SchemaMigrationStep::UnsupportedChange { entity, reason, .. } => { - // When a schema-lint code is attached, render code + tier - // so operators see at-a-glance the kind of risk (destructive - // / validated / safe) — not just the rule identifier. - // Reach the diagnostic via the `diagnostic()` helper so the - // CLI doesn't need to know how the lookup works. - match step.diagnostic() { - Some(diag) => format!( - "unsupported change on {} [{}, {}]: {}", - entity, - diag.code, - schema_lint_tier_label(diag.tier), - reason, - ), - None => format!("unsupported change on {}: {}", entity, reason), - } - } - } -} - -fn schema_type_kind_label(kind: omnigraph_compiler::SchemaTypeKind) -> &'static str { - match kind { - omnigraph_compiler::SchemaTypeKind::Interface => "interface", - omnigraph_compiler::SchemaTypeKind::Node => "node", - omnigraph_compiler::SchemaTypeKind::Edge => "edge", - } -} - -fn schema_lint_tier_label(tier: omnigraph_compiler::SafetyTier) -> &'static str { - match tier { - omnigraph_compiler::SafetyTier::Safe => "safe", - omnigraph_compiler::SafetyTier::Validated => "validated", - omnigraph_compiler::SafetyTier::Destructive => "destructive", - } -} - -fn drop_mode_label(mode: omnigraph_compiler::DropMode) -> &'static str { - match mode { - omnigraph_compiler::DropMode::Soft => "soft", - omnigraph_compiler::DropMode::Hard => "hard", - } -} - -fn render_prop_type(prop_type: &omnigraph_compiler::PropType) -> String { - let base = if let Some(values) = &prop_type.enum_values { - format!("Enum({})", values.join("|")) - } else { - prop_type.scalar.to_string() - }; - let base = if prop_type.list { - format!("[{}]", base) - } else { - base - }; - if prop_type.nullable { - format!("{}?", base) - } else { - base - } -} - -fn render_constraint(constraint: &omnigraph_compiler::schema::ast::Constraint) -> String { - match constraint { - omnigraph_compiler::schema::ast::Constraint::Key(columns) => { - format!("@key({})", columns.join(", ")) - } - omnigraph_compiler::schema::ast::Constraint::Unique(columns) => { - format!("@unique({})", columns.join(", ")) - } - omnigraph_compiler::schema::ast::Constraint::Index(columns) => { - format!("@index({})", columns.join(", ")) - } - omnigraph_compiler::schema::ast::Constraint::Range { property, min, max } => { - format!("@range({}, {:?}, {:?})", property, min, max) - } - omnigraph_compiler::schema::ast::Constraint::Check { property, pattern } => { - format!("@check({}, {:?})", property, pattern) - } - } -} - -fn render_annotations(annotations: &[omnigraph_compiler::schema::ast::Annotation]) -> String { - annotations - .iter() - .map(|annotation| match &annotation.value { - Some(value) => format!("@{}({})", annotation.name, value), - None => format!("@{}", annotation.name), - }) - .collect::<Vec<_>>() - .join(", ") -} - -fn print_embed_human(output: &EmbedOutput) { - println!( - "embedded {} rows (selected {}, cleaned {}) from {} -> {} [{} {}d]", - output.embedded_rows, - output.selected_rows, - output.cleaned_rows, - output.input, - output.output, - output.mode, - output.dimension - ); -} - -fn print_snapshot_human(branch: &str, manifest_version: u64, entries: &[SnapshotTableOutput]) { - println!("branch: {}", branch); - println!("manifest_version: {}", manifest_version); - for entry in entries { - println!( - "{} v{} branch={} rows={}", - entry.table_key, - entry.table_version, - entry.table_branch.as_deref().unwrap_or("main"), - entry.row_count - ); - } -} - -fn print_read_output( - output: &ReadOutput, - format: ReadOutputFormat, - config: &OmnigraphConfig, -) -> Result<()> { - println!( - "{}", - render_read( - output, - format, - &ReadRenderOptions { - max_column_width: config.table_max_column_width(), - cell_layout: config.table_cell_layout(), - }, - )? - ); - Ok(()) -} - -fn print_change_human(output: &ChangeOutput) { - println!( - "changed {} via {}: {} nodes, {} edges", - output.branch, output.query_name, output.affected_nodes, output.affected_edges - ); - if let Some(actor_id) = &output.actor_id { - println!("actor_id: {}", actor_id); - } -} - -fn print_commit_list_human(commits: &[CommitOutput]) { - for commit in commits { - let branch = commit.manifest_branch.as_deref().unwrap_or("main"); - println!( - "{} branch={} version={}{}", - commit.graph_commit_id, - branch, - commit.manifest_version, - commit - .actor_id - .as_deref() - .map(|actor| format!(" actor={}", actor)) - .unwrap_or_default() - ); - } -} - -fn print_commit_human(commit: &CommitOutput) { - println!("graph_commit_id: {}", commit.graph_commit_id); - println!( - "manifest_branch: {}", - commit.manifest_branch.as_deref().unwrap_or("main") - ); - println!("manifest_version: {}", commit.manifest_version); - if let Some(parent_commit_id) = &commit.parent_commit_id { - println!("parent_commit_id: {}", parent_commit_id); - } - if let Some(merged_parent_commit_id) = &commit.merged_parent_commit_id { - println!("merged_parent_commit_id: {}", merged_parent_commit_id); - } - if let Some(actor_id) = &commit.actor_id { - println!("actor_id: {}", actor_id); - } - println!("created_at: {}", commit.created_at); -} - -fn print_policy_explain(decision: &PolicyDecision, actor_id: &str, request: &PolicyRequest) { - println!( - "decision: {}", - if decision.allowed { "allow" } else { "deny" } - ); - println!("actor: {}", actor_id); - println!("action: {}", request.action); - if let Some(branch) = &request.branch { - println!("branch: {}", branch); - } - if let Some(target_branch) = &request.target_branch { - println!("target_branch: {}", target_branch); - } - if let Some(rule_id) = &decision.matched_rule_id { - println!("matched_rule: {}", rule_id); - } - println!("message: {}", decision.message); -} - -fn resolve_read_format( - config: &OmnigraphConfig, - cli_format: Option<ReadOutputFormat>, - json: bool, - alias_format: Option<ReadOutputFormat>, -) -> ReadOutputFormat { - if json { - ReadOutputFormat::Json - } else { - cli_format - .or(alias_format) - .unwrap_or_else(|| config.cli_output_format()) - } -} - -fn resolve_alias<'a>( - config: &'a OmnigraphConfig, - alias_name: Option<&'a str>, - expected: AliasCommand, -) -> Result<Option<(&'a str, &'a omnigraph_server::AliasConfig)>> { - let Some(alias_name) = alias_name else { - return Ok(None); - }; - let alias = config.alias(alias_name)?; - if alias.command != expected { - bail!( - "alias '{}' is a {:?} alias, not a {:?} alias", - alias_name, - alias.command, - expected - ); - } - Ok(Some((alias_name, alias))) -} - -fn normalize_legacy_alias_uri( - uri: Option<String>, - target_available: bool, - alias_name: Option<&str>, - mut alias_args: Vec<String>, -) -> (Option<String>, Vec<String>) { - let Some(candidate) = uri else { - return (None, alias_args); - }; - - if alias_name.is_some() && target_available { - alias_args.insert(0, candidate); - return (None, alias_args); - } - - (Some(candidate), alias_args) -} - -fn scaffold_config_if_missing(uri: &str) -> Result<()> { - let path = inferred_config_path(uri)?; - if path.exists() { - return Ok(()); - } - - fs::write( - path, - format!( - "\ -project: - name: Omnigraph Project - -graphs: - local: - uri: {} - # bearer_token_env: OMNIGRAPH_BEARER_TOKEN - -server: - graph: local - bind: 127.0.0.1:8080 - -cli: - graph: local - branch: main - output_format: table - table_max_column_width: 80 - table_cell_layout: truncate - -query: - roots: - - queries - - . - -aliases: - # owner: - # command: read - # query: context.gq - # name: decision_owner - # args: [slug] - # graph: local - # branch: main - # format: kv - # - # attach_trace: - # command: change - # query: mutations.gq - # name: attach_trace - # args: [decision_slug, trace_slug] - # graph: local - # branch: main - -# auth: -# env_file: ./.env.omni -# -# policy: -# file: ./policy.yaml -", - yaml_string(uri), - ), - )?; - Ok(()) -} - -fn yaml_string(value: &str) -> String { - format!("'{}'", value.replace('\'', "''")) -} - -fn inferred_config_path(uri: &str) -> Result<PathBuf> { - if uri.contains("://") { - return Ok(omnigraph_server::config::default_config_path()); - } - - let path = Path::new(uri); - let base = if path.is_absolute() { - path.parent() - .map(Path::to_path_buf) - .unwrap_or(std::env::current_dir()?) - } else { - std::env::current_dir()?.join(path.parent().unwrap_or_else(|| Path::new("."))) - }; - Ok(base.join(omnigraph_server::config::DEFAULT_CONFIG_FILE)) -} - -fn read_target_from_cli(branch: Option<String>, snapshot: Option<String>) -> ReadTarget { - if let Some(snapshot) = snapshot { - ReadTarget::snapshot(SnapshotId::new(snapshot)) - } else { - ReadTarget::branch(branch.unwrap_or_else(|| "main".to_string())) - } -} - -fn load_params_json(params: &ParamsArgs) -> Result<Option<Value>> { - match (¶ms.params, ¶ms.params_file) { - (Some(inline), None) => Ok(Some(serde_json::from_str(inline)?)), - (None, Some(path)) => Ok(Some(serde_json::from_str(&fs::read_to_string(path)?)?)), - (None, None) => Ok(None), - (Some(_), Some(_)) => bail!("only one of --params or --params-file may be provided"), - } -} - -fn select_named_query( - query_source: &str, - requested_name: Option<&str>, -) -> Result<(String, Vec<omnigraph_compiler::query::ast::Param>)> { - let parsed = parse_query(query_source)?; - let query = if let Some(name) = requested_name { - parsed - .queries - .into_iter() - .find(|query| query.name == name) - .ok_or_else(|| color_eyre::eyre::eyre!("query '{}' not found", name))? - } else if parsed.queries.len() == 1 { - parsed.queries.into_iter().next().unwrap() - } else { - bail!("query file contains multiple queries; pass --name"); - }; - - Ok((query.name, query.params)) -} - -fn query_params_from_json( - query_params: &[omnigraph_compiler::query::ast::Param], - params_json: Option<&Value>, -) -> Result<ParamMap> { - json_params_to_param_map(params_json, query_params, JsonParamMode::Standard) - .map_err(|err| color_eyre::eyre::eyre!(err.to_string())) -} - -async fn execute_query_lint( - config: &OmnigraphConfig, - cli_uri: Option<String>, - cli_target: Option<&str>, - schema_path: Option<&PathBuf>, - query_path: &PathBuf, -) -> Result<QueryLintOutput> { - let resolved_query_path = resolve_query_path(config, Some(query_path), None)?; - let query_source = fs::read_to_string(&resolved_query_path)?; - let query_path = resolved_query_path.to_string_lossy().into_owned(); - - if let Some(schema_path) = schema_path { - let schema_source = fs::read_to_string(schema_path)?; - let schema = - parse_schema(&schema_source).map_err(|err| color_eyre::eyre::eyre!(err.to_string()))?; - let catalog = - build_catalog(&schema).map_err(|err| color_eyre::eyre::eyre!(err.to_string()))?; - return Ok(lint_query_file( - &catalog, - &query_source, - query_path, - QueryLintSchemaSource::file(schema_path.to_string_lossy().into_owned()), - )); - } - - let has_graph_target = - cli_uri.is_some() || cli_target.is_some() || config.cli_graph_name().is_some(); - if !has_graph_target { - bail!("query lint requires --schema <schema.pg> or a resolvable graph target"); - } - - let uri = resolve_local_uri(config, cli_uri, cli_target, "query lint")?; - let db = Omnigraph::open(&uri).await?; - Ok(lint_query_file( - &db.catalog(), - &query_source, - query_path, - QueryLintSchemaSource::graph(uri), - )) -} - -#[derive(serde::Serialize)] -struct QueriesIssue { - query: String, - message: String, -} - -#[derive(serde::Serialize)] -struct QueriesValidateOutput { - ok: bool, - breakages: Vec<QueriesIssue>, - warnings: Vec<QueriesIssue>, -} - -#[derive(serde::Serialize)] -struct QueriesParam { - name: String, - #[serde(rename = "type")] - type_name: String, - nullable: bool, -} - -#[derive(serde::Serialize)] -struct QueriesListItem { - name: String, - mcp_expose: bool, - tool_name: Option<String>, - mutation: bool, - params: Vec<QueriesParam>, -} - -#[derive(serde::Serialize)] -struct QueriesListOutput { - queries: Vec<QueriesListItem>, -} - -/// Resolve the selected graph to `(local URI, registry selection)` from one -/// precedence, so a command's schema and its stored-query registry can never -/// come from different graphs. A **positional URI is anonymous** (top-level -/// registry, ignoring the configured default graph); otherwise `--target` -/// or the configured `cli.graph` names the graph (its per-graph block). -/// Mirrors the server's single-mode identity rule. -fn resolve_selected_graph( - config: &OmnigraphConfig, - cli_uri: Option<String>, - cli_target: Option<&str>, - operation: &str, -) -> Result<(String, Option<String>)> { - let graph = resolve_local_graph(config, cli_uri, cli_target, operation)?; - Ok((graph.uri, graph.selected)) -} - -/// Load the stored-query registry for an already-resolved graph selection -/// (`None` = anonymous → top-level; `Some(name)` = that graph's block). -fn load_registry_or_report( - config: &OmnigraphConfig, - selected: Option<&str>, -) -> Result<QueryRegistry> { - QueryRegistry::load(config, config.query_entries_for(selected)).map_err(|errors| { - color_eyre::eyre::eyre!( - "stored-query registry failed to load:\n {}", - errors - .iter() - .map(|e| e.to_string()) - .collect::<Vec<_>>() - .join("\n ") - ) - }) -} - -fn graph_query_registry_names(config: &OmnigraphConfig) -> Vec<&str> { - config - .graphs - .iter() - .filter_map(|(name, graph)| (!graph.queries.is_empty()).then_some(name.as_str())) - .collect() -} - -fn resolve_registry_selection_for_list( - config: &OmnigraphConfig, - target: Option<&str>, -) -> Result<Option<String>> { - let selected = target - .map(str::to_string) - .or_else(|| config.cli_graph_name().map(str::to_string)); - if let Some(name) = selected.as_deref() { - config.resolve_graph_selection(Some(name))?; - return Ok(selected); - } - - if !config.query_entries().is_empty() { - return Ok(None); - } - - let graph_names = graph_query_registry_names(config); - if graph_names.is_empty() { - return Ok(None); - } - - bail!( - "stored-query registries are configured for graph{} {} but no graph was selected. Pass `--target {}` or set `cli.graph`.", - if graph_names.len() == 1 { "" } else { "s" }, - graph_names.join(", "), - graph_names[0], - ) -} - -fn validate_registry_for_catalog( - registry: &QueryRegistry, - catalog: &omnigraph_compiler::catalog::Catalog, - label: &str, -) -> omnigraph::error::Result<()> { - let report = check(registry, catalog); - if report.has_breakages() { - return Err(omnigraph::error::OmniError::manifest( - format_check_breakages(label, &report), - )); - } - Ok(()) -} - -async fn execute_queries_validate( - uri: Option<String>, - target: Option<String>, - config_path: Option<&PathBuf>, - json: bool, -) -> Result<()> { - let config = load_cli_config(config_path)?; - // One selection drives both the schema URI and the registry, so a - // positional URI and a `--target` can't validate different graphs. - let (uri, selected) = - resolve_selected_graph(&config, uri, target.as_deref(), "queries validate")?; - let registry = load_registry_or_report(&config, selected.as_deref())?; - let db = Omnigraph::open(&uri).await?; - let report = check(®istry, &db.catalog()); - - let output = QueriesValidateOutput { - ok: !report.has_breakages(), - breakages: report - .breakages - .iter() - .map(|b| QueriesIssue { - query: b.query.clone(), - message: b.message.clone(), - }) - .collect(), - warnings: report - .warnings - .iter() - .map(|w| QueriesIssue { - query: w.query.clone(), - message: w.message.clone(), - }) - .collect(), - }; - - if json { - print_json(&output)?; - } else { - if output.breakages.is_empty() { - println!( - "OK {} stored quer{} type-check against the schema", - registry.len(), - if registry.len() == 1 { "y" } else { "ies" } - ); - } - for issue in &output.breakages { - println!("ERROR query '{}': {}", issue.query, issue.message); - } - for issue in &output.warnings { - println!("WARN query '{}': {}", issue.query, issue.message); - } - } - - if report.has_breakages() { - io::stdout().flush()?; - std::process::exit(1); - } - Ok(()) -} - -fn execute_queries_list( - target: Option<String>, - config_path: Option<&PathBuf>, - json: bool, -) -> Result<()> { - let config = load_cli_config(config_path)?; - let selected = resolve_registry_selection_for_list(&config, target.as_deref())?; - let registry = load_registry_or_report(&config, selected.as_deref())?; - - let output = QueriesListOutput { - queries: registry - .iter() - .map(|q| QueriesListItem { - name: q.name.clone(), - mcp_expose: q.expose, - tool_name: q.tool_name.clone(), - mutation: q.is_mutation(), - params: q - .decl - .params - .iter() - .map(|p| QueriesParam { - name: p.name.clone(), - type_name: p.type_name.clone(), - nullable: p.nullable, - }) - .collect(), - }) - .collect(), - }; - - if json { - print_json(&output)?; - } else if output.queries.is_empty() { - println!("(no stored queries registered)"); - } else { - for q in &output.queries { - let kind = if q.mutation { "mutation" } else { "read" }; - let params = q - .params - .iter() - .map(|p| { - format!( - "${}: {}{}", - p.name, - p.type_name, - if p.nullable { "?" } else { "" } - ) - }) - .collect::<Vec<_>>() - .join(", "); - let mcp = if q.mcp_expose { - format!(" [mcp: {}]", q.tool_name.as_deref().unwrap_or(&q.name)) - } else { - String::new() - }; - println!("{kind} {}({params}){mcp}", q.name); - } - } - Ok(()) -} - -async fn execute_read( - uri: &str, - query_source: &str, - query_name: Option<&str>, - target: ReadTarget, - params_json: Option<&Value>, -) -> Result<ReadOutput> { - let (selected_name, query_params) = select_named_query(query_source, query_name)?; - let params = query_params_from_json(&query_params, params_json)?; - let db = Omnigraph::open(uri).await?; - let result = db - .query(target.clone(), query_source, &selected_name, ¶ms) - .await?; - Ok(read_output(selected_name, &target, result)) -} - -async fn execute_read_remote( - client: &reqwest::Client, - uri: &str, - query_source: &str, - query_name: Option<&str>, - target: ReadTarget, - params_json: Option<&Value>, - bearer_token: Option<&str>, -) -> Result<ReadOutput> { - let (branch, snapshot) = match &target { - ReadTarget::Branch(branch) => (Some(branch.clone()), None), - ReadTarget::Snapshot(snapshot) => (None, Some(snapshot.as_str().to_string())), - }; - remote_json( - client, - Method::POST, - remote_url(uri, "/read"), - Some(serde_json::to_value(ReadRequest { - query_source: query_source.to_string(), - query_name: query_name.map(ToOwned::to_owned), - params: params_json.cloned(), - branch, - snapshot, - })?), - bearer_token, - ) - .await -} - -async fn execute_change( - graph: &ResolvedCliGraph, - query_source: &str, - query_name: Option<&str>, - branch: &str, - params_json: Option<&Value>, - config: &OmnigraphConfig, - cli_as_actor: Option<&str>, -) -> Result<ChangeOutput> { - let (selected_name, query_params) = select_named_query(query_source, query_name)?; - let params = query_params_from_json(&query_params, params_json)?; - let db = open_local_db_with_policy(graph).await?; - let actor = resolve_cli_actor(cli_as_actor, config); - let result = db - .mutate_as(branch, query_source, &selected_name, ¶ms, actor) - .await?; - Ok(ChangeOutput { - branch: branch.to_string(), - query_name: selected_name, - affected_nodes: result.affected_nodes, - affected_edges: result.affected_edges, - actor_id: actor.map(String::from), - }) -} - -/// Build the JSON body for `POST /change` using the legacy wire shape. -/// -/// `ChangeRequest`'s Rust field names are now `query` / `name` (the canonical -/// wire shape going forward), but old `omnigraph-server` builds still require -/// the legacy `query_source` / `query_name` keys on `/change`. Hand-rolling -/// the JSON with the legacy names keeps a newer CLI talking to an older -/// server intact -- the same byte-stability contract we apply to -/// `execute_read_remote` against `/read`. -fn legacy_change_request_body( - query_source: &str, - query_name: Option<&str>, - branch: &str, - params_json: Option<&Value>, -) -> Value { - let mut body = serde_json::json!({ - "query_source": query_source, - "branch": branch, - }); - if let Some(name) = query_name { - body["query_name"] = Value::String(name.to_string()); - } - if let Some(params) = params_json { - body["params"] = params.clone(); - } - body -} - -async fn execute_change_remote( - client: &reqwest::Client, - uri: &str, - query_source: &str, - query_name: Option<&str>, - branch: &str, - params_json: Option<&Value>, - bearer_token: Option<&str>, -) -> Result<ChangeOutput> { - remote_json( - client, - Method::POST, - remote_url(uri, "/change"), - Some(legacy_change_request_body( - query_source, - query_name, - branch, - params_json, - )), - bearer_token, - ) - .await -} - -async fn execute_export_to_writer<W: Write>( - uri: &str, - branch: &str, - type_names: &[String], - table_keys: &[String], - writer: &mut W, -) -> Result<()> { - let db = Omnigraph::open(uri).await?; - db.export_jsonl_to_writer(branch, type_names, table_keys, writer) - .await?; - writer.flush()?; - Ok(()) -} - -async fn execute_export_remote_to_writer<W: Write>( - client: &reqwest::Client, - uri: &str, - branch: &str, - type_names: &[String], - table_keys: &[String], - bearer_token: Option<&str>, - writer: &mut W, -) -> Result<()> { - let request = apply_bearer_token( - client.request(Method::POST, remote_url(uri, "/export")), - bearer_token, - ) - .json(&ExportRequest { - branch: Some(branch.to_string()), - type_names: type_names.to_vec(), - table_keys: table_keys.to_vec(), - }); - let mut response = request.send().await?; - let status = response.status(); - if !status.is_success() { - let text = response.text().await?; - if let Ok(error) = serde_json::from_str::<ErrorOutput>(&text) { - bail!(error.error); - } - bail!("server returned {}: {}", status, text); - } - - while let Some(chunk) = response.chunk().await? { - writer.write_all(&chunk)?; - } - writer.flush()?; - Ok(()) -} - -/// Rewrite deprecated CLI invocations into their canonical form. -/// -/// The current rename pass moves four subcommands: -/// - `omnigraph read` -> `omnigraph query` (clap `visible_alias` handles parsing; we warn) -/// - `omnigraph change` -> `omnigraph mutate` (clap `visible_alias` handles parsing; we warn) -/// - `omnigraph check` -> `omnigraph lint` (rewrite required; no visible_alias by design) -/// - `omnigraph query lint` -> `omnigraph lint` (rewrite required; `query` is now the read-runner) -/// - `omnigraph query check` -> `omnigraph lint` (rewrite required) -/// -/// `check` is *not* a clap visible_alias on `lint` even though they're -/// semantically equivalent. Visible aliases create two canonical names -/// that agents emit interchangeably depending on training-data drift -/// (see MR-981 §6 for the policy). The argv-shim + stderr warning -/// pattern preserves back-compat for human users while pointing every -/// caller at the single canonical name in `--help`. -/// -/// Returns the (possibly rewritten) argv that clap should parse. -fn rewrite_deprecated_argv(args: Vec<OsString>) -> Vec<OsString> { - if args.len() >= 3 { - let sub = args[1].to_str(); - let sub2 = args[2].to_str(); - if sub == Some("query") && matches!(sub2, Some("lint") | Some("check")) { - let suffix = sub2.unwrap(); - eprintln!( - "warning: `omnigraph query {suffix}` is deprecated; use `omnigraph lint` instead" - ); - // Drop the leading `query` token AND normalize `check` -> `lint`. - // `check` is no longer a clap visible_alias (MR-981 §6), so the - // rewritten argv must reach the canonical `lint` subcommand - // directly. Result for `omnigraph query check --query foo.gq`: - // `omnigraph lint --query foo.gq`. - let mut out = Vec::with_capacity(args.len() - 1); - out.push(args[0].clone()); - out.push(OsString::from("lint")); - out.extend(args[3..].iter().cloned()); - return out; - } - } - if let Some(sub) = args.get(1).and_then(|s| s.to_str()) { - match sub { - "read" => { - eprintln!("warning: `omnigraph read` is deprecated; use `omnigraph query` instead") - } - "change" => eprintln!( - "warning: `omnigraph change` is deprecated; use `omnigraph mutate` instead" - ), - "check" => { - eprintln!("warning: `omnigraph check` is deprecated; use `omnigraph lint` instead"); - // Rewrite the top-level subcommand to `lint`; pass through the rest. - let mut out = Vec::with_capacity(args.len()); - out.push(args[0].clone()); - out.push(OsString::from("lint")); - out.extend(args[2..].iter().cloned()); - return out; - } - _ => {} - } - } - args -} +mod cli; +mod helpers; +mod output; +use cli::*; +use helpers::*; +use output::*; #[tokio::main] async fn main() -> Result<()> { @@ -3779,419 +1178,7 @@ async fn main() -> Result<()> { Ok(()) } + #[cfg(test)] -mod tests { - use std::fs; - - use super::{ - DEFAULT_BEARER_TOKEN_ENV, apply_bearer_token, bearer_token_from_env_file, - legacy_change_request_body, load_cli_config, load_env_file_into_process, - normalize_bearer_token, parse_env_assignment, resolve_cli_graph, resolve_policy_context, - resolve_remote_bearer_token, - }; - use omnigraph_server::load_config; - use reqwest::header::AUTHORIZATION; - use serde_json::json; - use tempfile::tempdir; - - #[test] - fn legacy_change_request_body_uses_legacy_field_names() { - // `execute_change_remote` hits `POST /change`, which old - // `omnigraph-server` builds deserialize as `ChangeRequest` with - // **required** `query_source` and optional `query_name` keys. - // Newer servers accept both spellings via serde alias, but a - // newer CLI must still emit the legacy keys on the wire so it - // can talk to an old server during a rolling upgrade. - let body = legacy_change_request_body( - "query insert_person($n: String) { insert Person { name: $n } }", - Some("insert_person"), - "main", - Some(&json!({ "n": "Alice" })), - ); - assert_eq!( - body["query_source"].as_str(), - Some("query insert_person($n: String) { insert Person { name: $n } }"), - ); - assert_eq!(body["query_name"].as_str(), Some("insert_person")); - assert_eq!(body["branch"].as_str(), Some("main")); - assert_eq!(body["params"]["n"].as_str(), Some("Alice")); - // Crucially, the **new** field names must NOT appear -- old - // servers would silently treat them as unknown fields and then - // fail on missing required `query_source`. - assert!( - body.get("query").is_none(), - "legacy /change body must not carry the renamed `query` key; got {body}" - ); - assert!( - body.get("name").is_none(), - "legacy /change body must not carry the renamed `name` key; got {body}" - ); - } - - #[test] - fn legacy_change_request_body_omits_optional_fields_when_unset() { - let body = legacy_change_request_body( - "query find() { match { $p: Person } return { $p.name } }", - None, - "main", - None, - ); - assert_eq!(body["branch"].as_str(), Some("main")); - assert!(body.get("query_name").is_none()); - assert!(body.get("params").is_none()); - } - - #[test] - fn apply_bearer_token_adds_header_when_configured() { - let client = reqwest::Client::new(); - let request = apply_bearer_token(client.get("http://example.com"), Some("demo-token")) - .build() - .unwrap(); - assert_eq!( - request - .headers() - .get(AUTHORIZATION) - .and_then(|value| value.to_str().ok()), - Some("Bearer demo-token") - ); - } - - #[test] - fn apply_bearer_token_leaves_request_unchanged_when_not_configured() { - let client = reqwest::Client::new(); - let request = apply_bearer_token(client.get("http://example.com"), None) - .build() - .unwrap(); - assert!(request.headers().get(AUTHORIZATION).is_none()); - } - - #[test] - fn normalize_bearer_token_trims_and_filters_blank_values() { - assert_eq!(normalize_bearer_token(None), None); - assert_eq!(normalize_bearer_token(Some(" ".to_string())), None); - assert_eq!( - normalize_bearer_token(Some(" demo-token ".to_string())).as_deref(), - Some("demo-token") - ); - } - - #[test] - fn parse_env_assignment_supports_plain_and_exported_values() { - assert_eq!( - parse_env_assignment("DEMO_TOKEN=demo-token"), - Some(("DEMO_TOKEN".to_string(), "demo-token".to_string())) - ); - assert_eq!( - parse_env_assignment("export DEMO_TOKEN=\"quoted-token\""), - Some(("DEMO_TOKEN".to_string(), "quoted-token".to_string())) - ); - assert_eq!(parse_env_assignment("# comment"), None); - assert_eq!(parse_env_assignment(" "), None); - } - - #[test] - fn bearer_token_from_env_file_reads_named_value() { - let temp = tempdir().unwrap(); - let env_file = temp.path().join(".env.omni"); - fs::write( - &env_file, - "FIRST=ignore\nexport DEMO_TOKEN=\" demo-token \"\n", - ) - .unwrap(); - - assert_eq!( - bearer_token_from_env_file(&env_file, "DEMO_TOKEN") - .unwrap() - .as_deref(), - Some("demo-token") - ); - assert_eq!( - bearer_token_from_env_file(&env_file, "MISSING").unwrap(), - None - ); - } - - #[test] - fn load_env_file_into_process_sets_missing_values_without_overriding_existing_ones() { - let temp = tempdir().unwrap(); - let env_file = temp.path().join(".env.omni"); - fs::write( - &env_file, - "AUTOLOAD_ONLY=from-file\nAUTOLOAD_PRESET=from-file\n", - ) - .unwrap(); - - let missing_key = "AUTOLOAD_ONLY"; - let preset_key = "AUTOLOAD_PRESET"; - let previous_missing = std::env::var_os(missing_key); - let previous_preset = std::env::var_os(preset_key); - - unsafe { - std::env::remove_var(missing_key); - std::env::set_var(preset_key, "from-env"); - } - - load_env_file_into_process(&env_file).unwrap(); - - assert_eq!(std::env::var(missing_key).unwrap(), "from-file"); - assert_eq!(std::env::var(preset_key).unwrap(), "from-env"); - - unsafe { - if let Some(value) = previous_missing { - std::env::set_var(missing_key, value); - } else { - std::env::remove_var(missing_key); - } - - if let Some(value) = previous_preset { - std::env::set_var(preset_key, value); - } else { - std::env::remove_var(preset_key); - } - } - } - - #[test] - fn resolve_remote_bearer_token_uses_scoped_env_file_with_global_fallback() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - r#" -graphs: - demo: - uri: https://example.com - bearer_token_env: DEMO_TOKEN -auth: - env_file: .env.omni -cli: - graph: demo -"#, - ) - .unwrap(); - fs::write( - temp.path().join(".env.omni"), - "DEMO_TOKEN=scoped-token\nOMNIGRAPH_BEARER_TOKEN=global-token\n", - ) - .unwrap(); - - let previous = std::env::var_os(DEFAULT_BEARER_TOKEN_ENV); - unsafe { - std::env::remove_var(DEFAULT_BEARER_TOKEN_ENV); - } - - let config_path = temp.path().join("omnigraph.yaml"); - let config = load_config(Some(&config_path)).unwrap(); - - assert_eq!( - resolve_remote_bearer_token(&config, None, Some("demo")) - .unwrap() - .as_deref(), - Some("scoped-token") - ); - assert_eq!( - resolve_remote_bearer_token(&config, Some("https://override.example.com"), None) - .unwrap() - .as_deref(), - Some("global-token") - ); - - unsafe { - if let Some(value) = previous { - std::env::set_var(DEFAULT_BEARER_TOKEN_ENV, value); - } else { - std::env::remove_var(DEFAULT_BEARER_TOKEN_ENV); - } - } - } - - #[test] - fn load_cli_config_autoloads_env_file_into_process() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - r#" -auth: - env_file: .env.omni -graphs: - demo: - uri: s3://bucket/prefix -"#, - ) - .unwrap(); - fs::write( - temp.path().join(".env.omni"), - "AUTOLOAD_FROM_CONFIG=loaded\n", - ) - .unwrap(); - - let key = "AUTOLOAD_FROM_CONFIG"; - let previous = std::env::var_os(key); - unsafe { - std::env::remove_var(key); - } - - let config_path = temp.path().join("omnigraph.yaml"); - let config = load_cli_config(Some(&config_path)).unwrap(); - - assert_eq!( - config.resolve_target_uri(None, Some("demo"), None).unwrap(), - "s3://bucket/prefix" - ); - assert_eq!(std::env::var(key).unwrap(), "loaded"); - - unsafe { - if let Some(value) = previous { - std::env::set_var(key, value); - } else { - std::env::remove_var(key); - } - } - } - - #[test] - fn graph_identity_resolve_policy_context_named_cli_graph_uses_graph_key_not_project_name_or_uri() - { - let temp = tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -project: - name: misleading-project -graphs: - local: - uri: /tmp/local-policy-graph.omni - policy: - file: ./policy.yaml -cli: - graph: local -"#, - ) - .unwrap(); - - let config = load_config(Some(&config_path)).unwrap(); - let context = resolve_policy_context(&config).unwrap(); - assert_eq!(context.graph_id, "local"); - } - - #[test] - fn graph_identity_resolve_policy_context_server_graph_uses_graph_key_when_cli_graph_absent() { - let temp = tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -project: - name: misleading-project -graphs: - local: - uri: /tmp/local-policy-graph.omni - policy: - file: ./server-policy.yaml -server: - graph: local -"#, - ) - .unwrap(); - - let config = load_config(Some(&config_path)).unwrap(); - let context = resolve_policy_context(&config).unwrap(); - assert_eq!(context.graph_id, "local"); - assert!(context.policy_file.ends_with("server-policy.yaml")); - } - - #[test] - fn graph_identity_resolve_policy_context_anonymous_uses_top_level_default_identity() { - let temp = tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -project: - name: misleading-project -graphs: - local: - uri: /tmp/local-policy-graph.omni -policy: - file: ./top-policy.yaml -"#, - ) - .unwrap(); - - let config = load_config(Some(&config_path)).unwrap(); - let context = resolve_policy_context(&config).unwrap(); - assert_eq!(context.graph_id, "default"); - assert!(context.policy_file.ends_with("top-policy.yaml")); - } - - #[test] - fn graph_identity_resolve_cli_graph_named_target_uses_graph_key_not_project_name_or_uri() { - let temp = tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -project: - name: misleading-project -graphs: - prod: - uri: s3://bucket/prod-graph/ - policy: - file: ./prod-policy.yaml -"#, - ) - .unwrap(); - - let config = load_config(Some(&config_path)).unwrap(); - let graph = resolve_cli_graph(&config, None, Some("prod")).unwrap(); - assert_eq!(graph.selected(), Some("prod")); - assert_eq!(graph.graph_id, "prod"); - assert_eq!(graph.uri, "s3://bucket/prod-graph/"); - } - - #[test] - fn graph_identity_resolve_cli_graph_positional_uri_uses_anonymous_normalized_uri() { - let temp = tempdir().unwrap(); - let config_path = temp.path().join("omnigraph.yaml"); - fs::write( - &config_path, - r#" -project: - name: misleading-project -graphs: - local: - uri: /tmp/configured-graph.omni - policy: - file: ./policy.yaml -cli: - graph: local -"#, - ) - .unwrap(); - - let config = load_config(Some(&config_path)).unwrap(); - let local_graph_path = temp.path().join("explicit-graph.omni"); - let local_graph = resolve_cli_graph( - &config, - Some(format!("file://{}", local_graph_path.display())), - None, - ) - .unwrap(); - assert_eq!(local_graph.selected(), None); - assert_eq!( - local_graph.graph_id, - local_graph_path.to_string_lossy().as_ref() - ); - assert_eq!(local_graph.policy_file, None); - - let s3_graph = resolve_cli_graph( - &config, - Some("s3://bucket/anonymous-graph/".to_string()), - None, - ) - .unwrap(); - assert_eq!(s3_graph.selected(), None); - assert_eq!(s3_graph.graph_id, "s3://bucket/anonymous-graph"); - assert_eq!(s3_graph.policy_file, None); - } -} +#[path = "main_tests.rs"] +mod tests; diff --git a/crates/omnigraph-cli/src/main_tests.rs b/crates/omnigraph-cli/src/main_tests.rs new file mode 100644 index 0000000..0bbb593 --- /dev/null +++ b/crates/omnigraph-cli/src/main_tests.rs @@ -0,0 +1,416 @@ +//! In-source test suite for the CLI binary (moved verbatim from +//! main.rs; `use super::*` resolves through the #[path] declaration). + + use std::fs; + + use super::{ + DEFAULT_BEARER_TOKEN_ENV, apply_bearer_token, bearer_token_from_env_file, + legacy_change_request_body, load_cli_config, load_env_file_into_process, + normalize_bearer_token, parse_env_assignment, resolve_cli_graph, resolve_policy_context, + resolve_remote_bearer_token, + }; + use omnigraph_server::load_config; + use reqwest::header::AUTHORIZATION; + use serde_json::json; + use tempfile::tempdir; + + #[test] + fn legacy_change_request_body_uses_legacy_field_names() { + // `execute_change_remote` hits `POST /change`, which old + // `omnigraph-server` builds deserialize as `ChangeRequest` with + // **required** `query_source` and optional `query_name` keys. + // Newer servers accept both spellings via serde alias, but a + // newer CLI must still emit the legacy keys on the wire so it + // can talk to an old server during a rolling upgrade. + let body = legacy_change_request_body( + "query insert_person($n: String) { insert Person { name: $n } }", + Some("insert_person"), + "main", + Some(&json!({ "n": "Alice" })), + ); + assert_eq!( + body["query_source"].as_str(), + Some("query insert_person($n: String) { insert Person { name: $n } }"), + ); + assert_eq!(body["query_name"].as_str(), Some("insert_person")); + assert_eq!(body["branch"].as_str(), Some("main")); + assert_eq!(body["params"]["n"].as_str(), Some("Alice")); + // Crucially, the **new** field names must NOT appear -- old + // servers would silently treat them as unknown fields and then + // fail on missing required `query_source`. + assert!( + body.get("query").is_none(), + "legacy /change body must not carry the renamed `query` key; got {body}" + ); + assert!( + body.get("name").is_none(), + "legacy /change body must not carry the renamed `name` key; got {body}" + ); + } + + #[test] + fn legacy_change_request_body_omits_optional_fields_when_unset() { + let body = legacy_change_request_body( + "query find() { match { $p: Person } return { $p.name } }", + None, + "main", + None, + ); + assert_eq!(body["branch"].as_str(), Some("main")); + assert!(body.get("query_name").is_none()); + assert!(body.get("params").is_none()); + } + + #[test] + fn apply_bearer_token_adds_header_when_configured() { + let client = reqwest::Client::new(); + let request = apply_bearer_token(client.get("http://example.com"), Some("demo-token")) + .build() + .unwrap(); + assert_eq!( + request + .headers() + .get(AUTHORIZATION) + .and_then(|value| value.to_str().ok()), + Some("Bearer demo-token") + ); + } + + #[test] + fn apply_bearer_token_leaves_request_unchanged_when_not_configured() { + let client = reqwest::Client::new(); + let request = apply_bearer_token(client.get("http://example.com"), None) + .build() + .unwrap(); + assert!(request.headers().get(AUTHORIZATION).is_none()); + } + + #[test] + fn normalize_bearer_token_trims_and_filters_blank_values() { + assert_eq!(normalize_bearer_token(None), None); + assert_eq!(normalize_bearer_token(Some(" ".to_string())), None); + assert_eq!( + normalize_bearer_token(Some(" demo-token ".to_string())).as_deref(), + Some("demo-token") + ); + } + + #[test] + fn parse_env_assignment_supports_plain_and_exported_values() { + assert_eq!( + parse_env_assignment("DEMO_TOKEN=demo-token"), + Some(("DEMO_TOKEN".to_string(), "demo-token".to_string())) + ); + assert_eq!( + parse_env_assignment("export DEMO_TOKEN=\"quoted-token\""), + Some(("DEMO_TOKEN".to_string(), "quoted-token".to_string())) + ); + assert_eq!(parse_env_assignment("# comment"), None); + assert_eq!(parse_env_assignment(" "), None); + } + + #[test] + fn bearer_token_from_env_file_reads_named_value() { + let temp = tempdir().unwrap(); + let env_file = temp.path().join(".env.omni"); + fs::write( + &env_file, + "FIRST=ignore\nexport DEMO_TOKEN=\" demo-token \"\n", + ) + .unwrap(); + + assert_eq!( + bearer_token_from_env_file(&env_file, "DEMO_TOKEN") + .unwrap() + .as_deref(), + Some("demo-token") + ); + assert_eq!( + bearer_token_from_env_file(&env_file, "MISSING").unwrap(), + None + ); + } + + #[test] + fn load_env_file_into_process_sets_missing_values_without_overriding_existing_ones() { + let temp = tempdir().unwrap(); + let env_file = temp.path().join(".env.omni"); + fs::write( + &env_file, + "AUTOLOAD_ONLY=from-file\nAUTOLOAD_PRESET=from-file\n", + ) + .unwrap(); + + let missing_key = "AUTOLOAD_ONLY"; + let preset_key = "AUTOLOAD_PRESET"; + let previous_missing = std::env::var_os(missing_key); + let previous_preset = std::env::var_os(preset_key); + + unsafe { + std::env::remove_var(missing_key); + std::env::set_var(preset_key, "from-env"); + } + + load_env_file_into_process(&env_file).unwrap(); + + assert_eq!(std::env::var(missing_key).unwrap(), "from-file"); + assert_eq!(std::env::var(preset_key).unwrap(), "from-env"); + + unsafe { + if let Some(value) = previous_missing { + std::env::set_var(missing_key, value); + } else { + std::env::remove_var(missing_key); + } + + if let Some(value) = previous_preset { + std::env::set_var(preset_key, value); + } else { + std::env::remove_var(preset_key); + } + } + } + + #[test] + fn resolve_remote_bearer_token_uses_scoped_env_file_with_global_fallback() { + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("omnigraph.yaml"), + r#" +graphs: + demo: + uri: https://example.com + bearer_token_env: DEMO_TOKEN +auth: + env_file: .env.omni +cli: + graph: demo +"#, + ) + .unwrap(); + fs::write( + temp.path().join(".env.omni"), + "DEMO_TOKEN=scoped-token\nOMNIGRAPH_BEARER_TOKEN=global-token\n", + ) + .unwrap(); + + let previous = std::env::var_os(DEFAULT_BEARER_TOKEN_ENV); + unsafe { + std::env::remove_var(DEFAULT_BEARER_TOKEN_ENV); + } + + let config_path = temp.path().join("omnigraph.yaml"); + let config = load_config(Some(&config_path)).unwrap(); + + assert_eq!( + resolve_remote_bearer_token(&config, None, Some("demo")) + .unwrap() + .as_deref(), + Some("scoped-token") + ); + assert_eq!( + resolve_remote_bearer_token(&config, Some("https://override.example.com"), None) + .unwrap() + .as_deref(), + Some("global-token") + ); + + unsafe { + if let Some(value) = previous { + std::env::set_var(DEFAULT_BEARER_TOKEN_ENV, value); + } else { + std::env::remove_var(DEFAULT_BEARER_TOKEN_ENV); + } + } + } + + #[test] + fn load_cli_config_autoloads_env_file_into_process() { + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("omnigraph.yaml"), + r#" +auth: + env_file: .env.omni +graphs: + demo: + uri: s3://bucket/prefix +"#, + ) + .unwrap(); + fs::write( + temp.path().join(".env.omni"), + "AUTOLOAD_FROM_CONFIG=loaded\n", + ) + .unwrap(); + + let key = "AUTOLOAD_FROM_CONFIG"; + let previous = std::env::var_os(key); + unsafe { + std::env::remove_var(key); + } + + let config_path = temp.path().join("omnigraph.yaml"); + let config = load_cli_config(Some(&config_path)).unwrap(); + + assert_eq!( + config.resolve_target_uri(None, Some("demo"), None).unwrap(), + "s3://bucket/prefix" + ); + assert_eq!(std::env::var(key).unwrap(), "loaded"); + + unsafe { + if let Some(value) = previous { + std::env::set_var(key, value); + } else { + std::env::remove_var(key); + } + } + } + + #[test] + fn graph_identity_resolve_policy_context_named_cli_graph_uses_graph_key_not_project_name_or_uri() + { + let temp = tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +project: + name: misleading-project +graphs: + local: + uri: /tmp/local-policy-graph.omni + policy: + file: ./policy.yaml +cli: + graph: local +"#, + ) + .unwrap(); + + let config = load_config(Some(&config_path)).unwrap(); + let context = resolve_policy_context(&config).unwrap(); + assert_eq!(context.graph_id, "local"); + } + + #[test] + fn graph_identity_resolve_policy_context_server_graph_uses_graph_key_when_cli_graph_absent() { + let temp = tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +project: + name: misleading-project +graphs: + local: + uri: /tmp/local-policy-graph.omni + policy: + file: ./server-policy.yaml +server: + graph: local +"#, + ) + .unwrap(); + + let config = load_config(Some(&config_path)).unwrap(); + let context = resolve_policy_context(&config).unwrap(); + assert_eq!(context.graph_id, "local"); + assert!(context.policy_file.ends_with("server-policy.yaml")); + } + + #[test] + fn graph_identity_resolve_policy_context_anonymous_uses_top_level_default_identity() { + let temp = tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +project: + name: misleading-project +graphs: + local: + uri: /tmp/local-policy-graph.omni +policy: + file: ./top-policy.yaml +"#, + ) + .unwrap(); + + let config = load_config(Some(&config_path)).unwrap(); + let context = resolve_policy_context(&config).unwrap(); + assert_eq!(context.graph_id, "default"); + assert!(context.policy_file.ends_with("top-policy.yaml")); + } + + #[test] + fn graph_identity_resolve_cli_graph_named_target_uses_graph_key_not_project_name_or_uri() { + let temp = tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +project: + name: misleading-project +graphs: + prod: + uri: s3://bucket/prod-graph/ + policy: + file: ./prod-policy.yaml +"#, + ) + .unwrap(); + + let config = load_config(Some(&config_path)).unwrap(); + let graph = resolve_cli_graph(&config, None, Some("prod")).unwrap(); + assert_eq!(graph.selected(), Some("prod")); + assert_eq!(graph.graph_id, "prod"); + assert_eq!(graph.uri, "s3://bucket/prod-graph/"); + } + + #[test] + fn graph_identity_resolve_cli_graph_positional_uri_uses_anonymous_normalized_uri() { + let temp = tempdir().unwrap(); + let config_path = temp.path().join("omnigraph.yaml"); + fs::write( + &config_path, + r#" +project: + name: misleading-project +graphs: + local: + uri: /tmp/configured-graph.omni + policy: + file: ./policy.yaml +cli: + graph: local +"#, + ) + .unwrap(); + + let config = load_config(Some(&config_path)).unwrap(); + let local_graph_path = temp.path().join("explicit-graph.omni"); + let local_graph = resolve_cli_graph( + &config, + Some(format!("file://{}", local_graph_path.display())), + None, + ) + .unwrap(); + assert_eq!(local_graph.selected(), None); + assert_eq!( + local_graph.graph_id, + local_graph_path.to_string_lossy().as_ref() + ); + assert_eq!(local_graph.policy_file, None); + + let s3_graph = resolve_cli_graph( + &config, + Some("s3://bucket/anonymous-graph/".to_string()), + None, + ) + .unwrap(); + assert_eq!(s3_graph.selected(), None); + assert_eq!(s3_graph.graph_id, "s3://bucket/anonymous-graph"); + assert_eq!(s3_graph.policy_file, None); + } diff --git a/crates/omnigraph-cli/src/output.rs b/crates/omnigraph-cli/src/output.rs new file mode 100644 index 0000000..f77e50f --- /dev/null +++ b/crates/omnigraph-cli/src/output.rs @@ -0,0 +1,830 @@ +//! Human/JSON output formatting for every command (moved verbatim from +//! main.rs in the modularization). + +use super::*; + +#[derive(Debug, Serialize)] +pub(crate) struct LoadOutput { + pub(crate) uri: String, + pub(crate) branch: String, + pub(crate) mode: &'static str, + /// Present only when `--from` was given; echoes the requested base. + #[serde(skip_serializing_if = "Option::is_none")] + pub(crate) base_branch: Option<String>, + pub(crate) branch_created: bool, + pub(crate) nodes_loaded: usize, + pub(crate) edges_loaded: usize, + pub(crate) node_types_loaded: usize, + pub(crate) edge_types_loaded: usize, +} + +pub(crate) fn load_output_from_tables( + uri: &str, + branch: &str, + mode: CliLoadMode, + output: &IngestOutput, +) -> LoadOutput { + let mut nodes_loaded = 0; + let mut edges_loaded = 0; + let mut node_types_loaded = 0; + let mut edge_types_loaded = 0; + for table in &output.tables { + if table.table_key.starts_with("node:") { + nodes_loaded += table.rows_loaded; + node_types_loaded += 1; + } else if table.table_key.starts_with("edge:") { + edges_loaded += table.rows_loaded; + edge_types_loaded += 1; + } + } + LoadOutput { + uri: uri.to_string(), + branch: branch.to_string(), + mode: mode.as_str(), + base_branch: output.base_branch.clone(), + branch_created: output.branch_created, + nodes_loaded, + edges_loaded, + node_types_loaded, + edge_types_loaded, + } +} + +#[derive(Debug, Serialize)] +pub(crate) struct SchemaPlanOutput<'a> { + pub(crate) uri: &'a str, + pub(crate) supported: bool, + pub(crate) step_count: usize, + pub(crate) steps: &'a [SchemaMigrationStep], +} + +pub(crate) fn print_schema_apply_human(output: &SchemaApplyOutput) { + println!("schema apply for {}", output.uri); + println!("supported: {}", if output.supported { "yes" } else { "no" }); + println!("applied: {}", if output.applied { "yes" } else { "no" }); + println!("manifest_version: {}", output.manifest_version); + if output.steps.is_empty() { + println!("no schema changes"); + return; + } + for step in &output.steps { + println!("- {}", render_schema_plan_step(step)); + } +} + +pub(crate) fn query_kind_label(kind: QueryLintQueryKind) -> &'static str { + match kind { + QueryLintQueryKind::Read => "read", + QueryLintQueryKind::Mutation => "mutation", + } +} + +pub(crate) fn severity_label(severity: QueryLintSeverity) -> &'static str { + match severity { + QueryLintSeverity::Error => "ERROR", + QueryLintSeverity::Warning => "WARN ", + QueryLintSeverity::Info => "INFO ", + } +} + +pub(crate) fn print_query_lint_human(output: &QueryLintOutput) { + for result in &output.results { + match result.status { + QueryLintStatus::Ok => { + println!( + "OK query `{}` ({})", + result.name, + query_kind_label(result.kind) + ); + } + QueryLintStatus::Error => { + println!( + "ERROR query `{}`: {}", + result.name, + result.error.as_deref().unwrap_or("unknown error") + ); + } + } + + for warning in &result.warnings { + println!("WARN query `{}`: {}", result.name, warning); + } + } + + for finding in &output.findings { + println!("{} {}", severity_label(finding.severity), finding.message); + } + + println!( + "INFO Lint complete: {} queries processed ({} error(s), {} warning(s), {} info item(s))", + output.queries_processed, output.errors, output.warnings, output.infos + ); +} + +pub(crate) fn finish_query_lint(output: &QueryLintOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_query_lint_human(output); + } + + if output.status == QueryLintStatus::Error { + io::stdout().flush()?; + std::process::exit(1); + } + + Ok(()) +} + +pub(crate) fn print_json<T: Serialize>(value: &T) -> Result<()> { + println!("{}", serde_json::to_string_pretty(value)?); + Ok(()) +} + +pub(crate) fn print_cluster_validate_human(output: &ValidateOutput) { + if output.ok { + println!( + "cluster config valid: {} resource(s), {} dependency edge(s)", + output.resources.len(), + output.dependencies.len() + ); + } else { + println!("cluster config invalid"); + } + print_cluster_diagnostics(&output.diagnostics); +} + +pub(crate) fn print_cluster_plan_human(output: &PlanOutput) { + if output.ok { + println!( + "cluster plan: {} change(s), {} approval gate(s)", + output.changes.len(), + output.approvals_required.len() + ); + for change in &output.changes { + let bindings = if change.binding_change { " [bindings]" } else { "" }; + println!(" {:?} {}{bindings}", change.operation, change.resource); + if let Some(migration) = &change.migration { + if !migration.supported { + println!(" migration UNSUPPORTED:"); + } + for step in &migration.steps { + println!( + " {}", + serde_json::to_string(step).unwrap_or_else(|_| format!("{step:?}")) + ); + } + } + } + if output.changes.is_empty() { + println!(" no changes"); + } + } else { + println!("cluster plan failed"); + } + print_cluster_diagnostics(&output.diagnostics); +} + +pub(crate) fn print_cluster_apply_human(output: &ApplyOutput) { + if output.ok { + println!( + "cluster apply: {} applied, {} deferred/blocked", + output.applied_count, output.deferred_count + ); + } else { + println!("cluster apply failed"); + } + // The change list prints on failure too: an operator debugging a partial + // apply (payload or state-write error) needs to see what was attempted. + print_cluster_apply_changes(&output.changes); + if output.ok { + let state = &output.state_observations; + println!( + " state: revision {}, converged: {}, written: {}", + state.state_revision, output.converged, output.state_written + ); + println!(" note: cluster-booted servers (--cluster) serve this on their next restart; omnigraph.yaml deployments are unaffected"); + } + print_cluster_diagnostics(&output.diagnostics); +} + +pub(crate) fn print_cluster_apply_changes(changes: &[omnigraph_cluster::PlanChange]) { + for change in changes { + let bindings = if change.binding_change { " [bindings]" } else { "" }; + match (&change.disposition, change.reason.as_deref()) { + (Some(disposition), Some(reason)) => println!( + " {:?} {}{bindings} [{disposition:?}: {reason}]", + change.operation, change.resource + ), + (Some(disposition), None) => println!( + " {:?} {}{bindings} [{disposition:?}]", + change.operation, change.resource + ), + _ => println!(" {:?} {}{bindings}", change.operation, change.resource), + } + } + if changes.is_empty() { + println!(" no changes"); + } +} + +pub(crate) fn print_cluster_status_human(output: &StatusOutput) { + if output.ok { + let state = &output.state_observations; + if state.state_found { + println!( + "cluster state: revision {}, {} resource(s)", + state.state_revision, state.resource_count + ); + if let Some(digest) = state.applied_config_digest.as_deref() { + println!(" applied config: {digest}"); + } + if state.locked { + println!(" lock: held{}", cluster_lock_summary(state)); + } else { + println!(" lock: not held"); + } + } else { + println!("cluster state missing"); + } + } else { + println!("cluster status failed"); + } + print_cluster_diagnostics(&output.diagnostics); +} + +pub(crate) fn print_cluster_state_sync_human(output: &StateSyncOutput) { + let operation = match output.operation { + omnigraph_cluster::StateSyncOperation::Refresh => "refresh", + omnigraph_cluster::StateSyncOperation::Import => "import", + }; + if output.ok { + let state = &output.state_observations; + println!( + "cluster {operation}: revision {}, {} resource(s)", + state.state_revision, state.resource_count + ); + if let Some(cas) = state.state_cas.as_deref() { + println!(" state_cas: {cas}"); + } + if state.locked { + println!(" lock: acquired{}", cluster_lock_summary(state)); + } else { + println!(" lock: not acquired"); + } + } else { + println!("cluster {operation} failed"); + } + print_cluster_diagnostics(&output.diagnostics); +} + +pub(crate) fn print_cluster_force_unlock_human(output: &ForceUnlockOutput) { + if output.ok { + if output.lock_removed { + println!( + "cluster force-unlock: removed lock{}", + cluster_lock_summary(&output.state_observations) + ); + } else { + println!("cluster force-unlock: no lock removed"); + } + } else { + println!("cluster force-unlock failed"); + if output.state_observations.locked { + println!( + " lock: held{}", + cluster_lock_summary(&output.state_observations) + ); + } + } + print_cluster_diagnostics(&output.diagnostics); +} + +pub(crate) fn cluster_lock_summary(state: &omnigraph_cluster::StateObservations) -> String { + let Some(lock_id) = state.lock_id.as_deref() else { + return String::new(); + }; + let mut parts = vec![format!("id={lock_id}")]; + if let Some(operation) = state.lock_operation.as_deref() { + parts.push(format!("operation={operation}")); + } + if let Some(pid) = state.lock_pid { + parts.push(format!("pid={pid}")); + } + if let Some(created_at) = state.lock_created_at.as_deref() { + parts.push(format!("created_at={created_at}")); + } + if let Some(age_seconds) = state.lock_age_seconds { + parts.push(format!("age_seconds={age_seconds}")); + } + format!(" ({})", parts.join(", ")) +} + +pub(crate) fn print_cluster_diagnostics(diagnostics: &[omnigraph_cluster::Diagnostic]) { + for diagnostic in diagnostics { + let label = match diagnostic.severity { + DiagnosticSeverity::Error => "ERROR", + DiagnosticSeverity::Warning => "WARN ", + }; + println!( + "{label} {} {}: {}", + diagnostic.code, diagnostic.path, diagnostic.message + ); + } +} + +pub(crate) fn finish_cluster_validate(output: &ValidateOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_validate_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + +pub(crate) fn finish_cluster_plan(output: &PlanOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_plan_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + +pub(crate) fn finish_cluster_apply(output: &ApplyOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_apply_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + +pub(crate) fn finish_cluster_approve(output: &ApproveOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else if output.ok { + println!( + "cluster approve: {} {} approved by {} (approval {})", + output + .operation + .as_ref() + .map(|operation| format!("{operation:?}").to_lowercase()) + .unwrap_or_default(), + output.resource.as_deref().unwrap_or("?"), + output.approved_by.as_deref().unwrap_or("?"), + output.approval_id.as_deref().unwrap_or("?"), + ); + print_cluster_diagnostics(&output.diagnostics); + } else { + println!("cluster approve failed"); + print_cluster_diagnostics(&output.diagnostics); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + +pub(crate) fn finish_cluster_status(output: &StatusOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_status_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + +pub(crate) fn finish_cluster_state_sync(output: &StateSyncOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_state_sync_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + +pub(crate) fn finish_cluster_force_unlock(output: &ForceUnlockOutput, json: bool) -> Result<()> { + if json { + print_json(output)?; + } else { + print_cluster_force_unlock_human(output); + } + if !output.ok { + io::stdout().flush()?; + std::process::exit(1); + } + Ok(()) +} + +pub(crate) fn print_load_human(payload: &LoadOutput) { + println!( + "loaded {} on branch {} with {}: {} nodes across {} node types, {} edges across {} edge types", + payload.uri, + payload.branch, + payload.mode, + payload.nodes_loaded, + payload.node_types_loaded, + payload.edges_loaded, + payload.edge_types_loaded + ); + if payload.branch_created { + if let Some(base) = &payload.base_branch { + println!("branch {} created from {}", payload.branch, base); + } + } +} + +pub(crate) fn print_ingest_human(output: &IngestOutput) { + println!( + "ingested {} into branch {} from {} with {} ({})", + output.uri, + output.branch, + output.base_branch.as_deref().unwrap_or("main"), + output.mode.as_str(), + if output.branch_created { + "branch created" + } else { + "branch exists" + } + ); + for table in &output.tables { + println!("{} rows_loaded={}", table.table_key, table.rows_loaded); + } + if let Some(actor_id) = &output.actor_id { + println!("actor_id: {}", actor_id); + } +} + +pub(crate) fn print_schema_plan_human(uri: &str, plan: &SchemaMigrationPlan) { + println!("schema plan for {}", uri); + println!("supported: {}", if plan.supported { "yes" } else { "no" }); + if plan.steps.is_empty() { + println!("no schema changes"); + return; + } + for step in &plan.steps { + println!("- {}", render_schema_plan_step(step)); + } +} + +pub(crate) fn render_schema_plan_step(step: &SchemaMigrationStep) -> String { + match step { + SchemaMigrationStep::AddType { type_kind, name } => { + format!("add {} type '{}'", schema_type_kind_label(*type_kind), name) + } + SchemaMigrationStep::RenameType { + type_kind, + from, + to, + } => format!( + "rename {} type '{}' -> '{}'", + schema_type_kind_label(*type_kind), + from, + to + ), + SchemaMigrationStep::AddProperty { + type_kind, + type_name, + property_name, + property_type, + } => format!( + "add property '{}.{}' ({}) on {} '{}'", + type_name, + property_name, + render_prop_type(property_type), + schema_type_kind_label(*type_kind), + type_name + ), + SchemaMigrationStep::RenameProperty { + type_kind, + type_name, + from, + to, + } => format!( + "rename property '{}.{}' -> '{}.{}' on {} '{}'", + type_name, + from, + type_name, + to, + schema_type_kind_label(*type_kind), + type_name + ), + SchemaMigrationStep::AddConstraint { + type_kind, + type_name, + constraint, + } => format!( + "add constraint {} on {} '{}'", + render_constraint(constraint), + schema_type_kind_label(*type_kind), + type_name + ), + SchemaMigrationStep::UpdateTypeMetadata { + type_kind, + name, + annotations, + } => format!( + "update metadata on {} '{}' ({})", + schema_type_kind_label(*type_kind), + name, + render_annotations(annotations) + ), + SchemaMigrationStep::UpdatePropertyMetadata { + type_kind, + type_name, + property_name, + annotations, + } => format!( + "update metadata on property '{}.{}' of {} '{}' ({})", + type_name, + property_name, + schema_type_kind_label(*type_kind), + type_name, + render_annotations(annotations) + ), + SchemaMigrationStep::DropType { + type_kind, + name, + mode, + } => format!( + "drop {} type '{}' ({} mode)", + schema_type_kind_label(*type_kind), + name, + drop_mode_label(*mode), + ), + SchemaMigrationStep::DropProperty { + type_kind, + type_name, + property_name, + mode, + } => format!( + "drop property '{}.{}' of {} '{}' ({} mode)", + type_name, + property_name, + schema_type_kind_label(*type_kind), + type_name, + drop_mode_label(*mode), + ), + SchemaMigrationStep::UnsupportedChange { entity, reason, .. } => { + // When a schema-lint code is attached, render code + tier + // so operators see at-a-glance the kind of risk (destructive + // / validated / safe) — not just the rule identifier. + // Reach the diagnostic via the `diagnostic()` helper so the + // CLI doesn't need to know how the lookup works. + match step.diagnostic() { + Some(diag) => format!( + "unsupported change on {} [{}, {}]: {}", + entity, + diag.code, + schema_lint_tier_label(diag.tier), + reason, + ), + None => format!("unsupported change on {}: {}", entity, reason), + } + } + } +} + +pub(crate) fn schema_type_kind_label(kind: omnigraph_compiler::SchemaTypeKind) -> &'static str { + match kind { + omnigraph_compiler::SchemaTypeKind::Interface => "interface", + omnigraph_compiler::SchemaTypeKind::Node => "node", + omnigraph_compiler::SchemaTypeKind::Edge => "edge", + } +} + +pub(crate) fn schema_lint_tier_label(tier: omnigraph_compiler::SafetyTier) -> &'static str { + match tier { + omnigraph_compiler::SafetyTier::Safe => "safe", + omnigraph_compiler::SafetyTier::Validated => "validated", + omnigraph_compiler::SafetyTier::Destructive => "destructive", + } +} + +pub(crate) fn drop_mode_label(mode: omnigraph_compiler::DropMode) -> &'static str { + match mode { + omnigraph_compiler::DropMode::Soft => "soft", + omnigraph_compiler::DropMode::Hard => "hard", + } +} + +pub(crate) fn render_prop_type(prop_type: &omnigraph_compiler::PropType) -> String { + let base = if let Some(values) = &prop_type.enum_values { + format!("Enum({})", values.join("|")) + } else { + prop_type.scalar.to_string() + }; + let base = if prop_type.list { + format!("[{}]", base) + } else { + base + }; + if prop_type.nullable { + format!("{}?", base) + } else { + base + } +} + +pub(crate) fn render_constraint(constraint: &omnigraph_compiler::schema::ast::Constraint) -> String { + match constraint { + omnigraph_compiler::schema::ast::Constraint::Key(columns) => { + format!("@key({})", columns.join(", ")) + } + omnigraph_compiler::schema::ast::Constraint::Unique(columns) => { + format!("@unique({})", columns.join(", ")) + } + omnigraph_compiler::schema::ast::Constraint::Index(columns) => { + format!("@index({})", columns.join(", ")) + } + omnigraph_compiler::schema::ast::Constraint::Range { property, min, max } => { + format!("@range({}, {:?}, {:?})", property, min, max) + } + omnigraph_compiler::schema::ast::Constraint::Check { property, pattern } => { + format!("@check({}, {:?})", property, pattern) + } + } +} + +pub(crate) fn render_annotations(annotations: &[omnigraph_compiler::schema::ast::Annotation]) -> String { + annotations + .iter() + .map(|annotation| match &annotation.value { + Some(value) => format!("@{}({})", annotation.name, value), + None => format!("@{}", annotation.name), + }) + .collect::<Vec<_>>() + .join(", ") +} + +pub(crate) fn print_embed_human(output: &EmbedOutput) { + println!( + "embedded {} rows (selected {}, cleaned {}) from {} -> {} [{} {}d]", + output.embedded_rows, + output.selected_rows, + output.cleaned_rows, + output.input, + output.output, + output.mode, + output.dimension + ); +} + +pub(crate) fn print_snapshot_human(branch: &str, manifest_version: u64, entries: &[SnapshotTableOutput]) { + println!("branch: {}", branch); + println!("manifest_version: {}", manifest_version); + for entry in entries { + println!( + "{} v{} branch={} rows={}", + entry.table_key, + entry.table_version, + entry.table_branch.as_deref().unwrap_or("main"), + entry.row_count + ); + } +} + +pub(crate) fn print_read_output( + output: &ReadOutput, + format: ReadOutputFormat, + config: &OmnigraphConfig, +) -> Result<()> { + println!( + "{}", + render_read( + output, + format, + &ReadRenderOptions { + max_column_width: config.table_max_column_width(), + cell_layout: config.table_cell_layout(), + }, + )? + ); + Ok(()) +} + +pub(crate) fn print_change_human(output: &ChangeOutput) { + println!( + "changed {} via {}: {} nodes, {} edges", + output.branch, output.query_name, output.affected_nodes, output.affected_edges + ); + if let Some(actor_id) = &output.actor_id { + println!("actor_id: {}", actor_id); + } +} + +pub(crate) fn print_commit_list_human(commits: &[CommitOutput]) { + for commit in commits { + let branch = commit.manifest_branch.as_deref().unwrap_or("main"); + println!( + "{} branch={} version={}{}", + commit.graph_commit_id, + branch, + commit.manifest_version, + commit + .actor_id + .as_deref() + .map(|actor| format!(" actor={}", actor)) + .unwrap_or_default() + ); + } +} + +pub(crate) fn print_commit_human(commit: &CommitOutput) { + println!("graph_commit_id: {}", commit.graph_commit_id); + println!( + "manifest_branch: {}", + commit.manifest_branch.as_deref().unwrap_or("main") + ); + println!("manifest_version: {}", commit.manifest_version); + if let Some(parent_commit_id) = &commit.parent_commit_id { + println!("parent_commit_id: {}", parent_commit_id); + } + if let Some(merged_parent_commit_id) = &commit.merged_parent_commit_id { + println!("merged_parent_commit_id: {}", merged_parent_commit_id); + } + if let Some(actor_id) = &commit.actor_id { + println!("actor_id: {}", actor_id); + } + println!("created_at: {}", commit.created_at); +} + +pub(crate) fn print_policy_explain(decision: &PolicyDecision, actor_id: &str, request: &PolicyRequest) { + println!( + "decision: {}", + if decision.allowed { "allow" } else { "deny" } + ); + println!("actor: {}", actor_id); + println!("action: {}", request.action); + if let Some(branch) = &request.branch { + println!("branch: {}", branch); + } + if let Some(target_branch) = &request.target_branch { + println!("target_branch: {}", target_branch); + } + if let Some(rule_id) = &decision.matched_rule_id { + println!("matched_rule: {}", rule_id); + } + println!("message: {}", decision.message); +} + +pub(crate) fn yaml_string(value: &str) -> String { + format!("'{}'", value.replace('\'', "''")) +} + +#[derive(serde::Serialize)] +pub(crate) struct QueriesIssue { + pub(crate) query: String, + pub(crate) message: String, +} + +#[derive(serde::Serialize)] +pub(crate) struct QueriesValidateOutput { + pub(crate) ok: bool, + pub(crate) breakages: Vec<QueriesIssue>, + pub(crate) warnings: Vec<QueriesIssue>, +} + +#[derive(serde::Serialize)] +pub(crate) struct QueriesParam { + pub(crate) name: String, + #[serde(rename = "type")] + pub(crate) type_name: String, + pub(crate) nullable: bool, +} + +#[derive(serde::Serialize)] +pub(crate) struct QueriesListItem { + pub(crate) name: String, + pub(crate) mcp_expose: bool, + pub(crate) tool_name: Option<String>, + pub(crate) mutation: bool, + pub(crate) params: Vec<QueriesParam>, +} + +#[derive(serde::Serialize)] +pub(crate) struct QueriesListOutput { + pub(crate) queries: Vec<QueriesListItem>, +} From d5e75df2724a1dd4319c2208bda32a127211d9b8 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 15:16:51 +0300 Subject: [PATCH 115/165] refactor(cli): split the test monolith into command-area suites MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit tests/cli.rs (4,548 lines, 112 tests) becomes five area files — cli_cluster (24), cli_cluster_e2e (10, the spawned-binary lifecycle compositions), cli_data (49), cli_schema_config (16), cli_queries (13) — with the file-local helpers joining the existing tests/support harness. Verbatim moves + visibility bumps; 161 crate tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/cli.rs | 4548 ----------------- crates/omnigraph-cli/tests/cli_cluster.rs | 884 ++++ crates/omnigraph-cli/tests/cli_cluster_e2e.rs | 621 +++ crates/omnigraph-cli/tests/cli_data.rs | 1631 ++++++ crates/omnigraph-cli/tests/cli_queries.rs | 535 ++ .../omnigraph-cli/tests/cli_schema_config.rs | 500 ++ crates/omnigraph-cli/tests/support/mod.rs | 350 ++ 7 files changed, 4521 insertions(+), 4548 deletions(-) delete mode 100644 crates/omnigraph-cli/tests/cli.rs create mode 100644 crates/omnigraph-cli/tests/cli_cluster.rs create mode 100644 crates/omnigraph-cli/tests/cli_cluster_e2e.rs create mode 100644 crates/omnigraph-cli/tests/cli_data.rs create mode 100644 crates/omnigraph-cli/tests/cli_queries.rs create mode 100644 crates/omnigraph-cli/tests/cli_schema_config.rs diff --git a/crates/omnigraph-cli/tests/cli.rs b/crates/omnigraph-cli/tests/cli.rs deleted file mode 100644 index 3f21b8a..0000000 --- a/crates/omnigraph-cli/tests/cli.rs +++ /dev/null @@ -1,4548 +0,0 @@ -use std::fs; - -use lance::Dataset; -use lance::index::DatasetIndexExt; -use omnigraph::db::{Omnigraph, ReadTarget}; -use serde_json::Value; -use tempfile::tempdir; - -mod support; - -use support::*; - -const POLICY_YAML: &str = r#" -version: 1 -groups: - team: [act-andrew, act-bruno] - admins: [act-andrew] -protected_branches: [main] -rules: - - id: team-read - allow: - actors: { group: team } - actions: [read] - branch_scope: any - - id: team-write - allow: - actors: { group: team } - actions: [change] - branch_scope: unprotected - - id: admins-promote - allow: - actors: { group: admins } - actions: [branch_merge] - target_branch_scope: protected -"#; - -const POLICY_TESTS_YAML: &str = r#" -version: 1 -cases: - - id: allow-feature-write - actor: act-andrew - action: change - branch: feature - expect: allow - - id: deny-main-write - actor: act-bruno - action: change - branch: main - expect: deny -"#; - -fn manifest_dataset_version(graph: &std::path::Path) -> u64 { - tokio::runtime::Runtime::new().unwrap().block_on(async { - Omnigraph::open(graph.to_string_lossy().as_ref()) - .await - .unwrap() - .snapshot_of(ReadTarget::branch("main")) - .await - .unwrap() - .version() - }) -} - -fn forge_person_delete_drift(graph: &std::path::Path) -> (u64, u64) { - tokio::runtime::Runtime::new().unwrap().block_on(async { - let uri = graph.to_string_lossy(); - let db = Omnigraph::open(uri.as_ref()).await.unwrap(); - let snap = db - .snapshot_of(ReadTarget::branch("main")) - .await - .unwrap(); - let entry = snap.entry("node:Person").unwrap(); - let full_path = format!("{}/{}", uri.trim_end_matches('/'), entry.table_path); - let mut ds = Dataset::open(&full_path).await.unwrap(); - let deleted = ds.delete("name = 'Alice'").await.unwrap(); - assert_eq!(deleted.num_deleted_rows, 1); - let head = deleted.new_dataset.version().version; - assert!(head > entry.table_version); - (entry.table_version, head) - }) -} - -fn write_policy_config_fixture(root: &std::path::Path) -> (std::path::PathBuf, std::path::PathBuf) { - let config = root.join("omnigraph.yaml"); - let policy = root.join("policy.yaml"); - fs::write( - &config, - r#" -project: - name: policy-test-graph -policy: - file: ./policy.yaml -"#, - ) - .unwrap(); - fs::write(&policy, POLICY_YAML).unwrap(); - fs::write(root.join("policy.tests.yaml"), POLICY_TESTS_YAML).unwrap(); - (config, policy) -} - -fn write_cluster_config_fixture(root: &std::path::Path) { - fs::write( - root.join("people.pg"), - r#" -node Person { - name: String @key - age: I32? -} -"#, - ) - .unwrap(); - fs::write( - root.join("people.gq"), - r#" -query find_person($name: String) { - match { $p: Person { name: $name } } - return { $p.name, $p.age } -} -"#, - ) - .unwrap(); - fs::write(root.join("base.policy.yaml"), "rules: []\n").unwrap(); - fs::write( - root.join("cluster.yaml"), - r#" -version: 1 -metadata: - name: company-brain -state: - backend: cluster - lock: true -graphs: - knowledge: - schema: ./people.pg - queries: - find_person: - file: ./people.gq -policies: - base: - file: ./base.policy.yaml - applies_to: [knowledge] -"#, - ) - .unwrap(); -} - -fn init_cluster_derived_graph(root: &std::path::Path) { - init_named_cluster_graph(root, "knowledge", "people.pg"); -} - -fn init_named_cluster_graph(root: &std::path::Path, graph_id: &str, schema_file: &str) { - let graph_dir = root.join("graphs"); - fs::create_dir_all(&graph_dir).unwrap(); - output_success( - cli() - .arg("init") - .arg("--schema") - .arg(root.join(schema_file)) - .arg(graph_dir.join(format!("{graph_id}.omni"))), - ); -} - -fn write_cluster_lock(root: &std::path::Path, lock_id: &str, operation: &str) { - let state_dir = root.join("__cluster"); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("lock.json"), - format!( - r#"{{"version":1,"lock_id":"{lock_id}","operation":"{operation}","created_at":"1970-01-01T00:00:00Z","pid":123}}"# - ), - ) - .unwrap(); -} - -#[test] -fn version_command_prints_current_cli_version() { - let output = output_success(cli().arg("version")); - let stdout = stdout_string(&output); - - assert_eq!( - stdout.trim(), - format!("omnigraph {}", env!("CARGO_PKG_VERSION")) - ); -} - -#[test] -fn cluster_validate_config_success() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - - let output = output_success( - cli() - .arg("cluster") - .arg("validate") - .arg("--config") - .arg(temp.path()), - ); - let stdout = stdout_string(&output); - assert!(stdout.contains("cluster config valid"), "{stdout}"); -} - -#[test] -fn cluster_validate_json_is_stable() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - - let json = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("validate") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(json["ok"], true); - assert!(json["resource_digests"]["graph.knowledge"].is_string()); - assert!(json["resource_digests"]["query.knowledge.find_person"].is_string()); - assert_eq!(json["dependencies"][0]["from"], "policy.base"); - assert_eq!(json["dependencies"][0]["to"], "graph.knowledge"); -} - -#[test] -fn cluster_plan_json_reads_inferred_local_state() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - let state_dir = temp.path().join("__cluster"); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#" -{ - "version": 1, - "applied_revision": { - "config_digest": "old", - "resources": { - "graph.knowledge": { "digest": "old-graph" }, - "policy.old": { "digest": "old-policy" } - } - } -} -"#, - ) - .unwrap(); - - let json = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("plan") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(json["ok"], true); - assert_eq!(json["state_observations"]["state_found"], true); - assert!( - json["changes"] - .as_array() - .unwrap() - .iter() - .any(|change| change["resource"] == "policy.old" && change["operation"] == "delete"), - "plan should read state and delete stale resources: {json}" - ); -} - -#[test] -fn cluster_status_json_reports_missing_state() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - - let json = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("status") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(json["ok"], true); - assert_eq!(json["state_observations"]["state_found"], false); - assert!( - json["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["code"] == "state_missing"), - "missing state should be a warning diagnostic: {json}" - ); -} - -#[test] -fn cluster_status_json_reports_lock_metadata() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - write_cluster_lock(temp.path(), "held-lock", "refresh"); - - let json = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("status") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(json["ok"], true); - assert_eq!(json["state_observations"]["locked"], true); - assert_eq!(json["state_observations"]["lock_id"], "held-lock"); - assert_eq!(json["state_observations"]["lock_operation"], "refresh"); - assert_eq!(json["state_observations"]["lock_pid"], 123); - assert_eq!( - json["state_observations"]["lock_created_at"], - "1970-01-01T00:00:00Z" - ); - assert!(json["state_observations"]["lock_age_seconds"].is_number()); -} - -#[test] -fn cluster_status_json_reports_extended_state() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - let state_dir = temp.path().join("__cluster"); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#" -{ - "version": 1, - "state_revision": 5, - "applied_revision": { - "config_digest": "applied", - "resources": { - "graph.knowledge": { "digest": "graph-digest" } - } - }, - "resource_statuses": { - "graph.knowledge": { "status": "applied", "conditions": ["healthy"] } - }, - "approval_records": {}, - "recovery_records": {}, - "observations": {} -} -"#, - ) - .unwrap(); - - let json = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("status") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(json["ok"], true); - assert_eq!(json["state_observations"]["state_revision"], 5); - assert!( - json["state_observations"]["state_cas"] - .as_str() - .unwrap() - .starts_with("sha256:") - ); - assert_eq!(json["resource_digests"]["graph.knowledge"], "graph-digest"); - assert_eq!( - json["resource_statuses"]["graph.knowledge"]["status"], - "applied" - ); -} - -#[test] -fn cluster_plan_json_includes_state_cas_revision_and_lock_observation() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - let state_dir = temp.path().join("__cluster"); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#" -{ - "version": 1, - "state_revision": 9, - "applied_revision": { - "config_digest": "old", - "resources": { - "graph.knowledge": { "digest": "old-graph" } - } - } -} -"#, - ) - .unwrap(); - - let json = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("plan") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(json["ok"], true); - assert_eq!(json["state_observations"]["state_revision"], 9); - assert!( - json["state_observations"]["state_cas"] - .as_str() - .unwrap() - .starts_with("sha256:") - ); - assert_eq!(json["state_observations"]["locked"], false); - assert_eq!(json["state_observations"]["lock_acquired"], true); - assert!(json["state_observations"]["acquired_lock_id"].is_string()); - assert!(!state_dir.join("lock.json").exists()); -} - -#[test] -fn cluster_plan_locked_state_exits_nonzero() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - write_cluster_lock(temp.path(), "held-lock", "plan"); - - let output = output_failure( - cli() - .arg("cluster") - .arg("plan") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - ); - let json = parse_stdout_json(&output); - assert_eq!(json["ok"], false); - assert_eq!(json["state_observations"]["locked"], true); - assert_eq!(json["state_observations"]["lock_acquired"], false); - assert_eq!(json["state_observations"]["lock_id"], "held-lock"); - assert_eq!(json["state_observations"]["lock_operation"], "plan"); - assert_eq!(json["state_observations"]["lock_pid"], 123); - assert_eq!( - json["state_observations"]["lock_created_at"], - "1970-01-01T00:00:00Z" - ); - assert!(json["state_observations"]["lock_age_seconds"].is_number()); - assert!( - json["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["code"] == "state_lock_held" - && diagnostic["message"] - .as_str() - .unwrap() - .contains("force-unlock held-lock")), - "locked state should produce a useful diagnostic: {json}" - ); -} - -#[test] -fn cluster_force_unlock_json_removes_lock() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - write_cluster_lock(temp.path(), "held-lock", "plan"); - - let json = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("force-unlock") - .arg("held-lock") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(json["ok"], true); - assert_eq!(json["lock_removed"], true); - assert_eq!(json["state_observations"]["lock_id"], "held-lock"); - assert_eq!(json["state_observations"]["lock_operation"], "plan"); - assert!(!temp.path().join("__cluster/lock.json").exists()); -} - -#[test] -fn cluster_force_unlock_wrong_id_exits_nonzero() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - write_cluster_lock(temp.path(), "held-lock", "plan"); - - let json = parse_stdout_json(&output_failure( - cli() - .arg("cluster") - .arg("force-unlock") - .arg("other-lock") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(json["ok"], false); - assert_eq!(json["lock_removed"], false); - assert!( - json["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["code"] == "state_lock_id_mismatch") - ); - assert!(temp.path().join("__cluster/lock.json").exists()); -} - -#[test] -fn cluster_locked_plan_then_force_unlock_then_plan_succeeds() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - write_cluster_lock(temp.path(), "held-lock", "plan"); - - let locked = parse_stdout_json(&output_failure( - cli() - .arg("cluster") - .arg("plan") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(locked["ok"], false); - assert_eq!(locked["state_observations"]["lock_id"], "held-lock"); - - let unlocked = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("force-unlock") - .arg("held-lock") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(unlocked["lock_removed"], true); - - let planned = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("plan") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(planned["ok"], true); -} - -#[test] -fn cluster_import_json_bootstraps_missing_state() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - init_cluster_derived_graph(temp.path()); - - let json = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("import") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(json["ok"], true); - assert_eq!(json["operation"], "import"); - assert_eq!(json["state_observations"]["state_revision"], 1); - assert!( - json["state_observations"]["state_cas"] - .as_str() - .unwrap() - .starts_with("sha256:") - ); - assert_eq!(json["state_observations"]["locked"], false); - assert_eq!(json["state_observations"]["lock_acquired"], true); - assert!(json["state_observations"]["acquired_lock_id"].is_string()); - assert!(json["observations"]["graph.knowledge"]["manifest_version"].is_number()); - assert_eq!( - json["resource_statuses"]["graph.knowledge"]["status"], - "applied" - ); - assert!(temp.path().join("__cluster/state.json").exists()); - assert!(!temp.path().join("__cluster/lock.json").exists()); -} - -#[test] -fn cluster_refresh_json_updates_revision_cas_and_removes_lock() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - init_cluster_derived_graph(temp.path()); - let state_dir = temp.path().join("__cluster"); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#" -{ - "version": 1, - "state_revision": 2, - "applied_revision": { "resources": {} } -} -"#, - ) - .unwrap(); - - let json = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("refresh") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(json["ok"], true); - assert_eq!(json["operation"], "refresh"); - assert_eq!(json["state_observations"]["state_revision"], 3); - assert!( - json["state_observations"]["state_cas"] - .as_str() - .unwrap() - .starts_with("sha256:") - ); - assert_eq!(json["state_observations"]["locked"], false); - assert_eq!(json["state_observations"]["lock_acquired"], true); - assert!(json["state_observations"]["acquired_lock_id"].is_string()); - assert!(!state_dir.join("lock.json").exists()); -} - -#[test] -fn cluster_refresh_missing_state_exits_nonzero() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - - let output = output_failure( - cli() - .arg("cluster") - .arg("refresh") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - ); - let json = parse_stdout_json(&output); - assert_eq!(json["ok"], false); - assert!( - json["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["code"] == "state_missing"), - "missing state should produce a useful diagnostic: {json}" - ); -} - -#[test] -fn cluster_import_existing_state_exits_nonzero() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - let state_dir = temp.path().join("__cluster"); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#"{"version":1,"applied_revision":{"resources":{}}}"#, - ) - .unwrap(); - - let output = output_failure( - cli() - .arg("cluster") - .arg("import") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - ); - let json = parse_stdout_json(&output); - assert_eq!(json["ok"], false); - assert!( - json["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["code"] == "state_already_exists"), - "existing state should produce a useful diagnostic: {json}" - ); -} - -#[test] -fn cluster_refresh_and_import_locked_state_exit_nonzero() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - let state_dir = temp.path().join("__cluster"); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - r#"{"version":1,"applied_revision":{"resources":{}}}"#, - ) - .unwrap(); - fs::write( - state_dir.join("lock.json"), - r#"{"version":1,"lock_id":"held-lock","operation":"refresh","created_at":"2026-06-08T00:00:00Z","pid":123}"#, - ) - .unwrap(); - - let refresh = parse_stdout_json(&output_failure( - cli() - .arg("cluster") - .arg("refresh") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(refresh["state_observations"]["locked"], true); - assert_eq!(refresh["state_observations"]["lock_id"], "held-lock"); - assert_eq!(refresh["state_observations"]["lock_acquired"], false); - assert!( - refresh["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["code"] == "state_lock_held") - ); - - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - let state_dir = temp.path().join("__cluster"); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("lock.json"), - r#"{"version":1,"lock_id":"held-lock","operation":"import","created_at":"2026-06-08T00:00:00Z","pid":123}"#, - ) - .unwrap(); - - let imported = parse_stdout_json(&output_failure( - cli() - .arg("cluster") - .arg("import") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(imported["state_observations"]["locked"], true); - assert_eq!(imported["state_observations"]["lock_id"], "held-lock"); - assert_eq!(imported["state_observations"]["lock_acquired"], false); - assert!( - imported["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["code"] == "state_lock_held") - ); -} - -#[test] -fn cluster_validate_invalid_config_exits_nonzero() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("cluster.yaml"), - "version: 1\ngraphs: {}\npipelines: {}\n", - ) - .unwrap(); - - let output = output_failure( - cli() - .arg("cluster") - .arg("validate") - .arg("--config") - .arg(temp.path()), - ); - let stdout = stdout_string(&output); - assert!(stdout.contains("future_phase_field"), "{stdout}"); -} - -/// Seed an applyable state: schema digest borrowed from `cluster validate`, -/// graph entry present (composite recomputed by apply), queries/policies -/// pending. -fn write_cluster_applyable_state(root: &std::path::Path) -> serde_json::Value { - let validate = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("validate") - .arg("--config") - .arg(root) - .arg("--json"), - )); - let schema_digest = validate["resource_digests"]["schema.knowledge"] - .as_str() - .unwrap() - .to_string(); - let state_dir = root.join("__cluster"); - fs::create_dir_all(&state_dir).unwrap(); - fs::write( - state_dir.join("state.json"), - format!( - r#"{{ - "version": 1, - "state_revision": 1, - "applied_revision": {{ - "resources": {{ - "graph.knowledge": {{ "digest": "seed" }}, - "schema.knowledge": {{ "digest": "{schema_digest}" }} - }} - }} -}} -"# - ), - ) - .unwrap(); - validate -} - -#[test] -fn cluster_apply_json_applies_query_and_policy() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - let validate = write_cluster_applyable_state(temp.path()); - - let json = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("apply") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(json["ok"], true, "{json}"); - assert_eq!(json["applied_count"], 2, "{json}"); - assert_eq!(json["converged"], true, "{json}"); - assert_eq!(json["state_written"], true, "{json}"); - assert_eq!( - json["resource_statuses"]["query.knowledge.find_person"]["status"], - "applied" - ); - - let query_digest = validate["resource_digests"]["query.knowledge.find_person"] - .as_str() - .unwrap(); - let payload = temp - .path() - .join("__cluster/resources/query/knowledge/find_person") - .join(format!("{query_digest}.gq")); - assert!(payload.exists(), "missing payload {}", payload.display()); - - let state: serde_json::Value = serde_json::from_str( - &fs::read_to_string(temp.path().join("__cluster/state.json")).unwrap(), - ) - .unwrap(); - assert_eq!(state["state_revision"], 2); - assert_eq!( - state["applied_revision"]["resources"]["query.knowledge.find_person"]["digest"], - *query_digest - ); -} - -#[test] -fn cluster_apply_missing_state_exits_nonzero() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - - let output = output_failure( - cli() - .arg("cluster") - .arg("apply") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - ); - let json = parse_stdout_json(&output); - assert_eq!(json["ok"], false); - assert!( - json["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["code"] == "state_missing"), - "{json}" - ); - assert!(!temp.path().join("__cluster/resources").exists()); -} - -#[test] -fn cluster_apply_locked_exits_nonzero() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - write_cluster_applyable_state(temp.path()); - write_cluster_lock(temp.path(), "held-lock", "plan"); - - let output = output_failure( - cli() - .arg("cluster") - .arg("apply") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - ); - let json = parse_stdout_json(&output); - assert_eq!(json["ok"], false); - assert!( - json["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["code"] == "state_lock_held"), - "{json}" - ); - assert!(temp.path().join("__cluster/lock.json").exists()); - assert!(!temp.path().join("__cluster/resources").exists()); -} - -fn cluster_json(root: &std::path::Path, command: &str) -> serde_json::Value { - parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg(command) - .arg("--config") - .arg(root) - .arg("--json"), - )) -} - -/// End-to-end lifecycle against a REAL derived graph: import observes the live -/// graph, plan/apply converge the query+policy catalog, status reports it, -/// refresh re-observes without un-converging, and a query edit round-trips. -/// This is the composition test — every step passes individually elsewhere; -/// this catches the seams (e.g. refresh and apply recomputing the graph -/// composite digest differently would silently re-open the plan forever). -#[test] -fn cluster_e2e_lifecycle_import_apply_status_refresh_converges() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - init_cluster_derived_graph(temp.path()); - - let import = cluster_json(temp.path(), "import"); - assert_eq!(import["ok"], true, "{import}"); - assert_eq!(import["state_observations"]["state_revision"], 1); - - let plan = cluster_json(temp.path(), "plan"); - let changes = plan["changes"].as_array().unwrap(); - assert_eq!(changes.len(), 3, "{plan}"); - let disposition_of = |resource: &str| { - changes - .iter() - .find(|change| change["resource"] == resource) - .unwrap_or_else(|| panic!("missing change for {resource}: {plan}"))["disposition"] - .clone() - }; - assert_eq!(disposition_of("graph.knowledge"), "derived"); - assert_eq!(disposition_of("query.knowledge.find_person"), "applied"); - assert_eq!(disposition_of("policy.base"), "applied"); - - let apply = cluster_json(temp.path(), "apply"); - assert_eq!(apply["ok"], true, "{apply}"); - assert_eq!(apply["applied_count"], 2, "{apply}"); - assert_eq!(apply["converged"], true, "{apply}"); - - let status = cluster_json(temp.path(), "status"); - assert_eq!( - status["resource_statuses"]["query.knowledge.find_person"]["status"], - "applied" - ); - assert_eq!(status["resource_statuses"]["policy.base"]["status"], "applied"); - assert!( - status["state_observations"]["applied_config_digest"].is_string(), - "converged apply must record the applied config digest: {status}" - ); - - // Refresh re-observes the live graph; it must not undo apply's work. - let refresh = cluster_json(temp.path(), "refresh"); - assert_eq!(refresh["ok"], true, "{refresh}"); - let replan = cluster_json(temp.path(), "plan"); - assert!( - replan["changes"].as_array().unwrap().is_empty(), - "refresh after a converged apply must not re-open the plan: {replan}" - ); - - // A query edit round-trips: plan update -> apply -> converged again. - fs::write( - temp.path().join("people.gq"), - r#" -query find_person($name: String) { - match { $p: Person { name: $name } } - return { $p.name } -} -"#, - ) - .unwrap(); - let apply_edit = cluster_json(temp.path(), "apply"); - assert_eq!(apply_edit["applied_count"], 1, "{apply_edit}"); - assert_eq!(apply_edit["converged"], true, "{apply_edit}"); - - let final_apply = cluster_json(temp.path(), "apply"); - assert_eq!(final_apply["state_written"], false, "{final_apply}"); - assert!(final_apply["changes"].as_array().unwrap().is_empty()); -} - -/// The operator workflow across the Stage 3A boundary: a schema change is -/// deferred by cluster apply, executed by `omnigraph schema apply` against -/// the graph, picked up by `cluster refresh`, and the next apply re-converges. -#[test] -fn cluster_e2e_schema_change_applied_by_cluster() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - init_cluster_derived_graph(temp.path()); - let import = cluster_json(temp.path(), "import"); - assert_eq!(import["ok"], true, "{import}"); - let apply = cluster_json(temp.path(), "apply"); - assert_eq!(apply["converged"], true, "{apply}"); - - // Additive schema change: Stage 4B applies it from the cluster — no - // manual schema apply, no refresh round-trip. - fs::write( - temp.path().join("people.pg"), - r#" -node Person { - name: String @key - age: I32? - bio: String? -} -"#, - ) - .unwrap(); - - // Plan previews the real migration steps (RFC-004 §D7). - let plan = cluster_json(temp.path(), "plan"); - let schema_change = change_for(&plan, "schema.knowledge"); - assert_eq!(schema_change["disposition"], "applied", "{plan}"); - let migration = &schema_change["migration"]; - assert_eq!(migration["supported"], true, "{plan}"); - assert!( - migration["steps"] - .as_array() - .unwrap() - .iter() - .any(|step| step["kind"] == "add_property"), - "{plan}" - ); - - let evolve = cluster_json(temp.path(), "apply"); - assert_eq!(evolve["ok"], true, "{evolve}"); - assert_eq!(evolve["converged"], true, "{evolve}"); - assert_eq!(change_for(&evolve, "schema.knowledge")["disposition"], "applied"); - - // The live graph carries the new schema; the plan is empty. - let schema_show = output_success( - cli() - .arg("schema") - .arg("show") - .arg(temp.path().join("graphs/knowledge.omni")), - ); - assert!(stdout_string(&schema_show).contains("bio"), "live schema updated"); - let replan = cluster_json(temp.path(), "plan"); - assert!( - replan["changes"].as_array().unwrap().is_empty(), - "one cluster apply converges a schema change: {replan}" - ); -} - -/// Lock-recovery composition: a held lock refuses apply, force-unlock clears -/// it, and the retried apply converges. -#[test] -fn cluster_e2e_force_unlock_unblocks_apply() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - write_cluster_applyable_state(temp.path()); - write_cluster_lock(temp.path(), "stuck-lock", "apply"); - - let refused = parse_stdout_json(&output_failure( - cli() - .arg("cluster") - .arg("apply") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(refused["ok"], false); - - let unlocked = parse_stdout_json(&output_success( - cli() - .arg("cluster") - .arg("force-unlock") - .arg("stuck-lock") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(unlocked["lock_removed"], true, "{unlocked}"); - - let retried = cluster_json(temp.path(), "apply"); - assert_eq!(retried["ok"], true, "{retried}"); - assert_eq!(retried["converged"], true, "{retried}"); -} - -/// Two-graph fixture: `knowledge` (people) + `engineering` (services), a -/// policy spanning both graphs, and a cluster-scoped policy with no graph -/// dependencies. -fn write_multi_graph_cluster_fixture(root: &std::path::Path) { - write_cluster_config_fixture(root); - fs::write( - root.join("services.pg"), - r#" -node Service { - name: String @key -} -"#, - ) - .unwrap(); - fs::write( - root.join("services.gq"), - r#" -query find_service($name: String) { - match { $s: Service { name: $name } } - return { $s.name } -} -"#, - ) - .unwrap(); - fs::write(root.join("cluster_wide.policy.yaml"), "rules: []\n").unwrap(); - fs::write(root.join("shared.policy.yaml"), "rules: []\n").unwrap(); - fs::write( - root.join("cluster.yaml"), - r#" -version: 1 -metadata: - name: company-brain -state: - backend: cluster - lock: true -graphs: - knowledge: - schema: ./people.pg - queries: - find_person: - file: ./people.gq - engineering: - schema: ./services.pg - queries: - find_service: - file: ./services.gq -policies: - shared: - file: ./shared.policy.yaml - applies_to: [knowledge, engineering] - cluster_wide: - file: ./cluster_wide.policy.yaml - applies_to: [cluster] -"#, - ) - .unwrap(); -} - -fn change_for<'j>(json: &'j serde_json::Value, resource: &str) -> &'j serde_json::Value { - json["changes"] - .as_array() - .unwrap() - .iter() - .find(|change| change["resource"] == resource) - .unwrap_or_else(|| panic!("missing change for {resource}: {json}")) -} - -/// The spec's resilience claim — "state is reconstructable from the -/// self-describing cluster" — proven end to end: lose the ledger, re-import -/// from the live graph, re-apply, and converge onto the same content-addressed -/// catalog blobs. -#[test] -fn cluster_e2e_lost_state_reimport_recovers_catalog() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - init_cluster_derived_graph(temp.path()); - let import = cluster_json(temp.path(), "import"); - assert_eq!(import["ok"], true, "{import}"); - let apply = cluster_json(temp.path(), "apply"); - assert_eq!(apply["converged"], true, "{apply}"); - - let query_digest = change_for(&apply, "query.knowledge.find_person")["after_digest"] - .as_str() - .unwrap() - .to_string(); - let blob = temp - .path() - .join("__cluster/resources/query/knowledge/find_person") - .join(format!("{query_digest}.gq")); - let blob_content = fs::read_to_string(&blob).unwrap(); - - // Disaster: the state ledger is lost. - fs::remove_file(temp.path().join("__cluster/state.json")).unwrap(); - - let reimport = cluster_json(temp.path(), "import"); - assert_eq!(reimport["ok"], true, "{reimport}"); - assert_eq!(reimport["state_observations"]["state_revision"], 1); - // Import observes graph/schema only; query/policy digests are not invented. - assert!( - reimport["resource_digests"] - .get("query.knowledge.find_person") - .is_none(), - "{reimport}" - ); - - let plan = cluster_json(temp.path(), "plan"); - assert_eq!( - change_for(&plan, "query.knowledge.find_person")["disposition"], - "applied" - ); - assert_eq!(change_for(&plan, "policy.base")["disposition"], "applied"); - - let reapply = cluster_json(temp.path(), "apply"); - assert_eq!(reapply["ok"], true, "{reapply}"); - assert_eq!(reapply["converged"], true, "{reapply}"); - assert!( - reapply["state_observations"]["applied_config_digest"].is_string(), - "{reapply}" - ); - // The catalog blob was reused, not rewritten with different content. - assert_eq!(fs::read_to_string(&blob).unwrap(), blob_content); - - let replan = cluster_json(temp.path(), "plan"); - assert!(replan["changes"].as_array().unwrap().is_empty(), "{replan}"); -} - -/// The Sarah/Bob violation made visible: a schema change applied directly to -/// the graph (no config change) must surface as drift through refresh, status, -/// and plan — and apply must never silently "correct" it. -#[test] -fn cluster_e2e_out_of_band_schema_drift_then_apply_converges_it() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - init_cluster_derived_graph(temp.path()); - let import = cluster_json(temp.path(), "import"); - assert_eq!(import["ok"], true, "{import}"); - let apply = cluster_json(temp.path(), "apply"); - assert_eq!(apply["converged"], true, "{apply}"); - - // Out-of-band: the live graph evolves, cluster.yaml stays put. - fs::write( - temp.path().join("people_v2.pg"), - r#" -node Person { - name: String @key - age: I32? - bio: String? -} -"#, - ) - .unwrap(); - output_success( - cli() - .arg("schema") - .arg("apply") - .arg(temp.path().join("graphs/knowledge.omni")) - .arg("--schema") - .arg(temp.path().join("people_v2.pg")) - .arg("--json"), - ); - - // Drift is visible... - let refresh = cluster_json(temp.path(), "refresh"); - assert_eq!( - refresh["resource_statuses"]["schema.knowledge"]["status"], - "drifted" - ); - // ...the plan proposes converging back to desired, with a migration - // preview (a soft drop of the out-of-band field)... - let plan = cluster_json(temp.path(), "plan"); - let schema_change = change_for(&plan, "schema.knowledge"); - assert_eq!(schema_change["disposition"], "applied", "{plan}"); - assert!( - schema_change["migration"]["steps"] - .as_array() - .unwrap() - .iter() - .any(|step| step["kind"] == "drop_property" && step["mode"] == "soft"), - "{plan}" - ); - // ...and apply converges the live schema back (axiom 8: drift correction - // is gated like any change; a soft migration is the recoverable tier). - let converge = cluster_json(temp.path(), "apply"); - assert_eq!(converge["ok"], true, "{converge}"); - assert_eq!(converge["converged"], true, "{converge}"); - let schema_show = output_success( - cli() - .arg("schema") - .arg("show") - .arg(temp.path().join("graphs/knowledge.omni")), - ); - assert!( - !stdout_string(&schema_show).contains("bio"), - "out-of-band field soft-dropped back to desired" - ); - let replan = cluster_json(temp.path(), "plan"); - assert!(replan["changes"].as_array().unwrap().is_empty(), "{replan}"); -} - -/// Disaster input fails closed: a destroyed graph root drifts the ledger, -/// the plan proposes deferred creates, and apply moves nothing. -#[test] -fn cluster_e2e_graph_root_destruction_drifts_then_apply_recreates_empty_graph() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - init_cluster_derived_graph(temp.path()); - let import = cluster_json(temp.path(), "import"); - assert_eq!(import["ok"], true, "{import}"); - let apply = cluster_json(temp.path(), "apply"); - assert_eq!(apply["converged"], true, "{apply}"); - let query_digest = change_for(&apply, "query.knowledge.find_person")["after_digest"] - .as_str() - .unwrap() - .to_string(); - - fs::remove_dir_all(temp.path().join("graphs/knowledge.omni")).unwrap(); - - // Missing root is drift, not an error. - let refresh = cluster_json(temp.path(), "refresh"); - assert_eq!(refresh["ok"], true, "{refresh}"); - assert_eq!( - refresh["resource_statuses"]["graph.knowledge"]["status"], - "drifted" - ); - assert!( - refresh["resource_statuses"]["graph.knowledge"]["conditions"] - .as_array() - .unwrap() - .iter() - .any(|condition| condition == "graph_missing"), - "{refresh}" - ); - // Graph/schema digests removed; query/policy digests preserved. - assert!(refresh["resource_digests"].get("graph.knowledge").is_none()); - assert!(refresh["resource_digests"].get("schema.knowledge").is_none()); - assert!( - refresh["resource_digests"] - .get("query.knowledge.find_person") - .is_some(), - "{refresh}" - ); - - let plan = cluster_json(temp.path(), "plan"); - assert_eq!(change_for(&plan, "graph.knowledge")["operation"], "create"); - // Stage 4A: the re-create is executable and the plan says so — nothing - // hidden about converging a destroyed root back to an EMPTY graph (the - // data was already lost; this is declarative convergence, RFC-004 §D1). - assert_eq!(change_for(&plan, "graph.knowledge")["disposition"], "applied"); - assert_eq!(change_for(&plan, "schema.knowledge")["disposition"], "applied"); - // Converged-then-destroyed: query/policy are already in state at the - // desired digests, so they are not changes at all. - assert_eq!(plan["changes"].as_array().unwrap().len(), 2, "{plan}"); - - let recreate = cluster_json(temp.path(), "apply"); - assert_eq!(recreate["ok"], true, "{recreate}"); - assert_eq!(recreate["converged"], true, "{recreate}"); - // The empty graph is back on disk; catalog state survived throughout. - assert!(temp.path().join("graphs/knowledge.omni").exists()); - let state: serde_json::Value = serde_json::from_str( - &fs::read_to_string(temp.path().join("__cluster/state.json")).unwrap(), - ) - .unwrap(); - assert_eq!( - state["applied_revision"]["resources"]["query.knowledge.find_person"]["digest"], - query_digest - ); - assert!( - temp.path() - .join("__cluster/resources/query/knowledge/find_person") - .join(format!("{query_digest}.gq")) - .exists() - ); -} - -/// The disposition matrix as a system under Stage 4A: a fresh multi-graph -/// config converges in ONE apply (both graphs created, spanning and -/// cluster-scoped policies applied), and a later mixed run — schema update -/// (deferred), its dependent query (blocked), an independent query update -/// (applied), its composite (derived) — shows all four dispositions at once -/// before the graph-plane schema apply closes the loop. -#[test] -fn cluster_e2e_multi_graph_mixed_dispositions_then_approve_and_converge() { - let temp = tempdir().unwrap(); - write_multi_graph_cluster_fixture(temp.path()); - // No manual init: Stage 4A creates both graphs. - - let import = cluster_json(temp.path(), "import"); - assert_eq!(import["ok"], true, "{import}"); - - let apply = cluster_json(temp.path(), "apply"); - assert_eq!(apply["ok"], true, "{apply}"); - assert_eq!(apply["converged"], true, "{apply}"); - assert_eq!(change_for(&apply, "graph.knowledge")["disposition"], "applied"); - assert_eq!( - change_for(&apply, "graph.engineering")["disposition"], - "applied" - ); - assert_eq!( - change_for(&apply, "query.engineering.find_service")["disposition"], - "applied" - ); - // The graph-spanning and cluster-scoped policies ride the same run. - assert_eq!(change_for(&apply, "policy.shared")["disposition"], "applied"); - assert_eq!( - change_for(&apply, "policy.cluster_wide")["disposition"], - "applied" - ); - assert!(temp.path().join("graphs/knowledge.omni").exists()); - assert!(temp.path().join("graphs/engineering.omni").exists()); - - // Mixed run: a graph REMOVAL (4C territory — deferred) gates its query - // delete (blocked), while a knowledge query update is independent - // (applied) and re-derives its composite. All four dispositions at once. - fs::write( - temp.path().join("cluster.yaml"), - r#" -version: 1 -metadata: - name: company-brain -state: - backend: cluster - lock: true -graphs: - knowledge: - schema: ./people.pg - queries: - find_person: - file: ./people.gq -policies: - shared: - file: ./shared.policy.yaml - applies_to: [knowledge] - cluster_wide: - file: ./cluster_wide.policy.yaml - applies_to: [cluster] -"#, - ) - .unwrap(); - fs::write( - temp.path().join("people.gq"), - "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", - ) - .unwrap(); - - let mixed = cluster_json(temp.path(), "apply"); - assert_eq!(mixed["ok"], true, "{mixed}"); - assert_eq!(mixed["converged"], false, "{mixed}"); - // Stage 4C: deletes are gated on a digest-bound approval, one gate per - // subtree (the graph-level approval carries schema + queries). - assert_eq!( - change_for(&mixed, "graph.engineering")["disposition"], - "blocked" - ); - assert_eq!( - change_for(&mixed, "graph.engineering")["reason"], - "approval_required" - ); - assert_eq!( - change_for(&mixed, "schema.engineering")["reason"], - "approval_required" - ); - assert_eq!( - change_for(&mixed, "query.engineering.find_service")["reason"], - "approval_required" - ); - let gate_plan = cluster_json(temp.path(), "plan"); - let gates = gate_plan["approvals_required"].as_array().unwrap(); - assert_eq!(gates.len(), 1, "{gate_plan}"); - assert_eq!(gates[0]["resource"], "graph.engineering"); - assert_eq!(gates[0]["satisfied"], false); - assert_eq!( - change_for(&mixed, "query.knowledge.find_person")["disposition"], - "applied" - ); - // 5A: policy.shared's applies_to narrowed with an unchanged file digest - // — now a first-class binding change, applied in the same run. - assert_eq!(change_for(&mixed, "policy.shared")["binding_change"], true); - assert_eq!(change_for(&mixed, "policy.shared")["disposition"], "applied"); - assert_eq!( - change_for(&mixed, "graph.knowledge")["disposition"], - "derived" - ); - // Deterministic ordering: changes sorted by resource address. - let order: Vec<&str> = mixed["changes"] - .as_array() - .unwrap() - .iter() - .map(|change| change["resource"].as_str().unwrap()) - .collect(); - let mut sorted = order.clone(); - sorted.sort_unstable(); - assert_eq!(order, sorted, "{mixed}"); - // The conclusion: an apply without approval stays blocked; the approved - // delete converges the cluster, tombstoning the removed graph. - let still_blocked = cluster_json(temp.path(), "apply"); - assert_eq!(still_blocked["converged"], false, "{still_blocked}"); - - let approve = parse_stdout_json(&output_success( - cli() - .arg("--as") - .arg("andrew") - .arg("cluster") - .arg("approve") - .arg("graph.engineering") - .arg("--config") - .arg(temp.path()) - .arg("--json"), - )); - assert_eq!(approve["ok"], true, "{approve}"); - assert_eq!(approve["approved_by"], "andrew"); - - let converge = cluster_json(temp.path(), "apply"); - assert_eq!(converge["ok"], true, "{converge}"); - assert_eq!(converge["converged"], true, "{converge}"); - assert!(!temp.path().join("graphs/engineering.omni").exists()); - - let status = cluster_json(temp.path(), "status"); - assert_eq!(status["observations"]["graph.engineering"]["kind"], "tombstone"); - let final_plan = cluster_json(temp.path(), "plan"); - assert!( - final_plan["changes"].as_array().unwrap().is_empty(), - "{final_plan}" - ); -} - -/// An approval without an approver is meaningless: approve requires --as. -#[test] -fn cluster_e2e_approve_requires_actor() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - - let output = output_failure( - cli() - .arg("cluster") - .arg("approve") - .arg("graph.knowledge") - .arg("--config") - .arg(temp.path()), - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!(stderr.contains("--as"), "{stderr}"); -} - -/// Stage 4A headline: a declared graph is created by `cluster apply` itself — -/// no manual `omnigraph init` anywhere in the flow. -#[test] -fn cluster_e2e_declared_graph_created_by_apply() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - - let import = cluster_json(temp.path(), "import"); - assert_eq!(import["ok"], true, "{import}"); - - let apply = cluster_json(temp.path(), "apply"); - assert_eq!(apply["ok"], true, "{apply}"); - assert_eq!(apply["converged"], true, "{apply}"); - assert_eq!(change_for(&apply, "graph.knowledge")["disposition"], "applied"); - assert!(temp.path().join("graphs/knowledge.omni").exists()); - - // The created graph is a real graph: the per-graph CLI can open it. - let snapshot = output_success( - cli() - .arg("snapshot") - .arg(temp.path().join("graphs/knowledge.omni")), - ); - assert!(!stdout_string(&snapshot).is_empty()); - - let plan = cluster_json(temp.path(), "plan"); - assert!(plan["changes"].as_array().unwrap().is_empty(), "{plan}"); - let status = cluster_json(temp.path(), "status"); - assert_eq!( - status["resource_statuses"]["graph.knowledge"]["status"], - "applied" - ); -} - -/// Catalog payload drift self-heals across the command surface: status warns -/// read-only, refresh persists the drift and drops the digest, apply -/// republishes the blob, status comes back clean. -#[test] -fn cluster_e2e_payload_drift_self_heals() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - init_cluster_derived_graph(temp.path()); - let import = cluster_json(temp.path(), "import"); - assert_eq!(import["ok"], true, "{import}"); - let apply = cluster_json(temp.path(), "apply"); - assert_eq!(apply["converged"], true, "{apply}"); - - let query_digest = change_for(&apply, "query.knowledge.find_person")["after_digest"] - .as_str() - .unwrap() - .to_string(); - let blob = temp - .path() - .join("__cluster/resources/query/knowledge/find_person") - .join(format!("{query_digest}.gq")); - fs::remove_file(&blob).unwrap(); - - let status = cluster_json(temp.path(), "status"); - assert_eq!(status["ok"], true, "{status}"); - assert!( - status["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["code"] == "catalog_payload_missing"), - "{status}" - ); - - let refresh = cluster_json(temp.path(), "refresh"); - assert_eq!(refresh["ok"], true, "{refresh}"); - assert_eq!( - refresh["resource_statuses"]["query.knowledge.find_person"]["status"], - "drifted" - ); - - let heal = cluster_json(temp.path(), "apply"); - assert_eq!(heal["ok"], true, "{heal}"); - assert_eq!(heal["converged"], true, "{heal}"); - assert!(blob.exists(), "blob republished"); - - let clean = cluster_json(temp.path(), "status"); - assert!( - !clean["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| { - diagnostic["code"] - .as_str() - .is_some_and(|code| code.starts_with("catalog_payload")) - }), - "{clean}" - ); -} - -#[test] -fn short_version_flag_prints_current_cli_version() { - let output = output_success(cli().arg("-v")); - let stdout = stdout_string(&output); - - assert_eq!( - stdout.trim(), - format!("omnigraph {}", env!("CARGO_PKG_VERSION")) - ); -} - -#[test] -fn long_version_flag_prints_current_cli_version() { - let output = output_success(cli().arg("--version")); - let stdout = stdout_string(&output); - - assert_eq!( - stdout.trim(), - format!("omnigraph {}", env!("CARGO_PKG_VERSION")) - ); -} - -#[test] -fn embed_seed_fills_missing_and_preserves_existing_vectors_by_default() { - let temp = tempdir().unwrap(); - let seed = write_seed_fixture(temp.path()); - - let output = output_success( - cli() - .env("OMNIGRAPH_EMBEDDINGS_MOCK", "1") - .arg("embed") - .arg("--seed") - .arg(&seed) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["mode"], "fill_missing"); - assert_eq!(payload["embedded_rows"], 1); - assert_eq!(payload["selected_rows"], 2); - - let embedded = read_embedded_rows(temp.path().join("build/seed.embedded.jsonl")); - assert_eq!( - embedded[0]["data"]["embedding"].as_array().unwrap().len(), - 4 - ); - assert_eq!( - embedded[1]["data"]["embedding"], - serde_json::json!([0.1, 0.2]) - ); -} - -#[test] -fn embed_clean_removes_selected_embeddings() { - let temp = tempdir().unwrap(); - let seed = write_seed_fixture(temp.path()); - - let output = output_success( - cli() - .arg("embed") - .arg("--seed") - .arg(&seed) - .arg("--clean") - .arg("--select") - .arg("Decision:slug=dec-beta") - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["mode"], "clean"); - assert_eq!(payload["cleaned_rows"], 1); - - let embedded = read_embedded_rows(temp.path().join("build/seed.embedded.jsonl")); - assert!(embedded[0]["data"].get("embedding").is_none()); - assert!(embedded[1]["data"].get("embedding").is_none()); -} - -#[test] -fn embed_select_reembeds_only_matching_rows() { - let temp = tempdir().unwrap(); - let seed = write_seed_fixture(temp.path()); - - let output = output_success( - cli() - .env("OMNIGRAPH_EMBEDDINGS_MOCK", "1") - .arg("embed") - .arg("--seed") - .arg(&seed) - .arg("--select") - .arg("Decision:slug=dec-beta") - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["mode"], "reembed_selected"); - assert_eq!(payload["embedded_rows"], 1); - assert_eq!(payload["selected_rows"], 1); - - let embedded = read_embedded_rows(temp.path().join("build/seed.embedded.jsonl")); - assert!(embedded[0]["data"].get("embedding").is_none()); - assert_ne!( - embedded[1]["data"]["embedding"], - serde_json::json!([0.1, 0.2]) - ); - assert_eq!( - embedded[1]["data"]["embedding"].as_array().unwrap().len(), - 4 - ); -} - -#[test] -fn embed_seed_preserves_non_entity_rows() { - let temp = tempdir().unwrap(); - let seed = write_seed_fixture_with_edge(temp.path()); - - let output = output_success( - cli() - .env("OMNIGRAPH_EMBEDDINGS_MOCK", "1") - .arg("embed") - .arg("--seed") - .arg(&seed) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["rows"], 3); - assert_eq!(payload["embedded_rows"], 1); - - let embedded = read_embedded_rows(temp.path().join("build/seed.embedded.jsonl")); - assert_eq!(embedded.len(), 3); - assert_eq!(embedded[2]["edge"], "Triggered"); - assert_eq!(embedded[2]["from"], "sig-alpha"); - assert_eq!(embedded[2]["to"], "dec-alpha"); -} - -#[test] -fn init_creates_graph_successfully_on_missing_local_directory() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema = fixture("test.pg"); - - let output = output_success(cli().arg("init").arg("--schema").arg(&schema).arg(&graph)); - let stdout = stdout_string(&output); - - assert!(stdout.contains("initialized")); - assert!(graph.join("_schema.pg").exists()); - assert!(graph.join("__manifest").exists()); - assert!(temp.path().join("omnigraph.yaml").exists()); -} - -#[test] -fn repair_json_reports_noop_on_clean_graph() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - let output = output_success(cli().arg("repair").arg("--json").arg(&graph)); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["confirm"], false); - assert_eq!(payload["force"], false); - assert_eq!(payload["manifest_version"], Value::Null); - let tables = payload["tables"].as_array().unwrap(); - assert_eq!(tables.len(), 4); - assert!(tables.iter().all(|table| { - table["classification"] == "no_drift" && table["action"] == "no_op" - })); -} - -#[test] -fn repair_confirm_json_refuses_suspicious_drift_with_nonzero_exit_then_force_succeeds() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - let graph_manifest_before = manifest_dataset_version(&graph); - let (table_manifest_before, table_head_before) = forge_person_delete_drift(&graph); - - let refused = output_failure( - cli() - .arg("repair") - .arg("--confirm") - .arg("--json") - .arg(&graph), - ); - let refused_payload: Value = serde_json::from_slice(&refused.stdout).unwrap(); - assert_eq!(refused_payload["manifest_version"], Value::Null); - let person = refused_payload["tables"] - .as_array() - .unwrap() - .iter() - .find(|table| table["table_key"] == "node:Person") - .unwrap(); - assert_eq!(person["classification"], "suspicious"); - assert_eq!(person["action"], "refused"); - assert!( - String::from_utf8_lossy(&refused.stderr).contains("repair refused"), - "stderr should explain the non-zero exit; got: {}", - String::from_utf8_lossy(&refused.stderr) - ); - assert_eq!(manifest_dataset_version(&graph), graph_manifest_before); - - let forced = output_success( - cli() - .arg("repair") - .arg("--force") - .arg("--confirm") - .arg("--json") - .arg(&graph), - ); - let forced_payload: Value = serde_json::from_slice(&forced.stdout).unwrap(); - let forced_manifest = forced_payload["manifest_version"].as_u64().unwrap(); - assert!(forced_manifest > graph_manifest_before); - let person = forced_payload["tables"] - .as_array() - .unwrap() - .iter() - .find(|table| table["table_key"] == "node:Person") - .unwrap(); - assert_eq!(person["classification"], "suspicious"); - assert_eq!(person["action"], "forced"); - assert_eq!(person["manifest_version"], table_manifest_before); - assert_eq!(person["lance_head_version"], table_head_before); - assert_eq!(manifest_dataset_version(&graph), forced_manifest); -} - -#[test] -fn schema_plan_json_reports_supported_additive_change() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema_path = temp.path().join("next.pg"); - init_graph(&graph); - - let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace( - " age: I32?\n}", - " age: I32?\n nickname: String?\n}", - ); - fs::write(&schema_path, next_schema).unwrap(); - - let output = output_success( - cli() - .arg("schema") - .arg("plan") - .arg("--schema") - .arg(&schema_path) - .arg("--json") - .arg(&graph), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["supported"], true); - assert_eq!(payload["step_count"], 1); - assert_eq!(payload["steps"][0]["kind"], "add_property"); - assert_eq!(payload["steps"][0]["type_kind"], "node"); - assert_eq!(payload["steps"][0]["type_name"], "Person"); - assert_eq!(payload["steps"][0]["property_name"], "nickname"); -} - -#[test] -fn schema_plan_json_reports_unsupported_type_change() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema_path = temp.path().join("breaking.pg"); - init_graph(&graph); - - let breaking_schema = fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace("age: I32?", "age: I64?"); - fs::write(&schema_path, breaking_schema).unwrap(); - - let output = output_success( - cli() - .arg("schema") - .arg("plan") - .arg("--schema") - .arg(&schema_path) - .arg("--json") - .arg(&graph), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["supported"], false); - assert!(payload["steps"].as_array().unwrap().iter().any(|step| { - step["kind"] == "unsupported_change" - && step["entity"] - .as_str() - .unwrap_or_default() - .contains("Person.age") - })); -} - -#[test] -fn schema_apply_json_applies_supported_migration() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema_path = temp.path().join("next.pg"); - init_graph(&graph); - - let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace( - " age: I32?\n}", - " age: I32?\n nickname: String?\n}", - ); - fs::write(&schema_path, next_schema).unwrap(); - - let output = output_success( - cli() - .arg("schema") - .arg("apply") - .arg("--schema") - .arg(&schema_path) - .arg("--json") - .arg(&graph), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["supported"], true); - assert_eq!(payload["applied"], true); - assert_eq!(payload["step_count"], 1); - - let db = tokio::runtime::Runtime::new() - .unwrap() - .block_on(Omnigraph::open(graph.to_string_lossy().as_ref())) - .unwrap(); - assert!( - db.catalog().node_types["Person"] - .properties - .contains_key("nickname") - ); -} - -#[test] -fn schema_apply_human_reports_noop() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema_path = fixture("test.pg"); - init_graph(&graph); - - let output = output_success( - cli() - .arg("schema") - .arg("apply") - .arg("--schema") - .arg(&schema_path) - .arg(&graph), - ); - let stdout = stdout_string(&output); - - assert!(stdout.contains("applied: no")); - assert!(stdout.contains("no schema changes")); -} - -#[test] -fn schema_apply_json_renames_type_and_updates_snapshot() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema_path = temp.path().join("rename.pg"); - init_graph(&graph); - - let renamed_schema = fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace("node Person {\n", "node Human @rename_from(\"Person\") {\n") - .replace("edge Knows: Person -> Person", "edge Knows: Human -> Human") - .replace( - "edge WorksAt: Person -> Company", - "edge WorksAt: Human -> Company", - ); - fs::write(&schema_path, renamed_schema).unwrap(); - - let output = output_success( - cli() - .arg("schema") - .arg("apply") - .arg("--schema") - .arg(&schema_path) - .arg("--json") - .arg(&graph), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["applied"], true); - - let db = tokio::runtime::Runtime::new() - .unwrap() - .block_on(Omnigraph::open(graph.to_string_lossy().as_ref())) - .unwrap(); - let snapshot = tokio::runtime::Runtime::new() - .unwrap() - .block_on(db.snapshot_of(ReadTarget::branch("main"))) - .unwrap(); - assert!(snapshot.entry("node:Human").is_some()); - assert!(snapshot.entry("node:Person").is_none()); -} - -#[test] -fn schema_apply_json_renames_property_and_updates_catalog() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema_path = temp.path().join("rename-property.pg"); - init_graph(&graph); - - let renamed_schema = fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace("age: I32?", "years: I32? @rename_from(\"age\")"); - fs::write(&schema_path, renamed_schema).unwrap(); - - let output = output_success( - cli() - .arg("schema") - .arg("apply") - .arg("--schema") - .arg(&schema_path) - .arg("--json") - .arg(&graph), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["applied"], true); - - let db = tokio::runtime::Runtime::new() - .unwrap() - .block_on(Omnigraph::open(graph.to_string_lossy().as_ref())) - .unwrap(); - let person = &db.catalog().node_types["Person"]; - assert!(person.properties.contains_key("years")); - assert!(!person.properties.contains_key("age")); -} - -#[test] -fn schema_apply_json_adds_index_for_existing_property() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema_path = temp.path().join("index.pg"); - init_graph(&graph); - - let before_index_count = tokio::runtime::Runtime::new().unwrap().block_on(async { - let db = Omnigraph::open(graph.to_string_lossy().as_ref()) - .await - .unwrap(); - let snapshot = db.snapshot_of(ReadTarget::branch("main")).await.unwrap(); - let dataset = snapshot.open("node:Person").await.unwrap(); - dataset.load_indices().await.unwrap().len() - }); - - let indexed_schema = fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace("name: String @key", "name: String @key @index"); - fs::write(&schema_path, indexed_schema).unwrap(); - - let output = output_success( - cli() - .arg("schema") - .arg("apply") - .arg("--schema") - .arg(&schema_path) - .arg("--json") - .arg(&graph), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["applied"], true); - - let after_index_count = tokio::runtime::Runtime::new().unwrap().block_on(async { - let db = Omnigraph::open(graph.to_string_lossy().as_ref()) - .await - .unwrap(); - let snapshot = db.snapshot_of(ReadTarget::branch("main")).await.unwrap(); - let dataset = snapshot.open("node:Person").await.unwrap(); - dataset.load_indices().await.unwrap().len() - }); - assert!(after_index_count > before_index_count); -} - -#[test] -fn schema_apply_rejects_unsupported_plan() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema_path = temp.path().join("breaking.pg"); - init_graph(&graph); - - let breaking_schema = fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace("age: I32?", "age: I64?"); - fs::write(&schema_path, breaking_schema).unwrap(); - - let output = output_failure( - cli() - .arg("schema") - .arg("apply") - .arg("--schema") - .arg(&schema_path) - .arg(&graph), - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!(stderr.contains("changing property type")); -} - -#[test] -fn schema_apply_rejects_when_non_main_branch_exists() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema_path = temp.path().join("next.pg"); - init_graph(&graph); - output_success( - cli() - .arg("branch") - .arg("create") - .arg("--from") - .arg("main") - .arg("--uri") - .arg(&graph) - .arg("feature"), - ); - - let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace( - " age: I32?\n}", - " age: I32?\n nickname: String?\n}", - ); - fs::write(&schema_path, next_schema).unwrap(); - - let output = output_failure( - cli() - .arg("schema") - .arg("apply") - .arg("--schema") - .arg(&schema_path) - .arg(&graph), - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!(stderr.contains("schema apply requires a graph with only main")); -} - -#[test] -fn query_lint_json_with_schema_reports_warnings() { - let temp = tempdir().unwrap(); - let schema_path = temp.path().join("schema.pg"); - let query_path = temp.path().join("queries.gq"); - write_file( - &schema_path, - r#" -node Policy { - slug: String @key - name: String? - effectiveTo: DateTime? -} -"#, - ); - write_query_file( - &query_path, - r#" -query update_policy($slug: String, $name: String) { - update Policy set { name: $name } where slug = $slug -} -"#, - ); - - let output = output_success( - cli() - .arg("query") - .arg("lint") - .arg("--query") - .arg(&query_path) - .arg("--schema") - .arg(&schema_path) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["status"], "ok"); - assert_eq!(payload["schema_source"]["kind"], "file"); - assert_eq!(payload["queries_processed"], 1); - assert_eq!(payload["warnings"], 1); - assert_eq!(payload["findings"][0]["code"], "L201"); - assert_eq!( - payload["findings"][0]["message"], - "Policy.effectiveTo exists in schema but no update query sets it" - ); -} - -#[test] -fn query_check_alias_matches_lint_output() { - let temp = tempdir().unwrap(); - let schema_path = temp.path().join("schema.pg"); - let query_path = temp.path().join("queries.gq"); - write_file( - &schema_path, - r#" -node Person { - name: String -} -"#, - ); - write_query_file( - &query_path, - r#" -query list_people() { - match { $p: Person } - return { $p.name } -} -"#, - ); - - let lint_output = output_success( - cli() - .arg("query") - .arg("lint") - .arg("--query") - .arg(&query_path) - .arg("--schema") - .arg(&schema_path) - .arg("--json"), - ); - let check_output = output_success( - cli() - .arg("query") - .arg("check") - .arg("--query") - .arg(&query_path) - .arg("--schema") - .arg(&schema_path) - .arg("--json"), - ); - - assert_eq!(stdout_string(&lint_output), stdout_string(&check_output)); -} - -/// `omnigraph lint` is the canonical top-level lint command after the -/// query/mutate rename. `omnigraph query lint` and `omnigraph query check` -/// are kept as deprecated argv shims (warning + rewrite). All three must -/// produce identical stdout output. -#[test] -fn lint_top_level_matches_deprecated_query_lint_output() { - let temp = tempdir().unwrap(); - let schema_path = temp.path().join("schema.pg"); - let query_path = temp.path().join("queries.gq"); - write_file( - &schema_path, - r#" -node Person { - name: String -} -"#, - ); - write_query_file( - &query_path, - r#" -query list_people() { - match { $p: Person } - return { $p.name } -} -"#, - ); - - let canonical = output_success( - cli() - .arg("lint") - .arg("--query") - .arg(&query_path) - .arg("--schema") - .arg(&schema_path) - .arg("--json"), - ); - let deprecated_lint = output_success( - cli() - .arg("query") - .arg("lint") - .arg("--query") - .arg(&query_path) - .arg("--schema") - .arg(&schema_path) - .arg("--json"), - ); - let deprecated_check = output_success( - cli() - .arg("query") - .arg("check") - .arg("--query") - .arg(&query_path) - .arg("--schema") - .arg(&schema_path) - .arg("--json"), - ); - - assert_eq!(stdout_string(&canonical), stdout_string(&deprecated_lint)); - assert_eq!(stdout_string(&canonical), stdout_string(&deprecated_check)); - - // Canonical form must NOT emit the deprecation warning. - let canonical_stderr = String::from_utf8(canonical.stderr).unwrap(); - assert!( - !canonical_stderr.contains("deprecated"), - "`omnigraph lint` is canonical and must not warn; got stderr: {canonical_stderr}" - ); - - // Deprecated forms MUST emit the one-line warning, pointing at the - // new top-level `omnigraph lint`. - let lint_stderr = String::from_utf8(deprecated_lint.stderr).unwrap(); - assert!( - lint_stderr.contains("`omnigraph query lint` is deprecated") - && lint_stderr.contains("`omnigraph lint`"), - "expected deprecation warning pointing at `omnigraph lint`; got: {lint_stderr}" - ); - let check_stderr = String::from_utf8(deprecated_check.stderr).unwrap(); - assert!( - check_stderr.contains("`omnigraph query check` is deprecated") - && check_stderr.contains("`omnigraph lint`"), - "expected deprecation warning pointing at `omnigraph lint`; got: {check_stderr}" - ); -} - -/// Bare `omnigraph check` is NOT a clap `visible_alias` on `lint` (MR-981 §6: -/// visible aliases give agents two canonical names to emit interchangeably). -/// It's an argv-level shim: rewrites to `omnigraph lint`, prints a one-line -/// stderr deprecation warning, and produces identical stdout to the canonical -/// invocation. Cargo/Go users typing `check` keep working; help text shows -/// only `lint`. -#[test] -fn deprecated_check_top_level_rewrites_to_lint() { - let temp = tempdir().unwrap(); - let schema_path = temp.path().join("schema.pg"); - let query_path = temp.path().join("queries.gq"); - write_file( - &schema_path, - r#" -node Person { - name: String -} -"#, - ); - write_query_file( - &query_path, - r#" -query list_people() { - match { $p: Person } - return { $p.name } -} -"#, - ); - - let canonical = output_success( - cli() - .arg("lint") - .arg("--query") - .arg(&query_path) - .arg("--schema") - .arg(&schema_path) - .arg("--json"), - ); - let deprecated_check = output_success( - cli() - .arg("check") - .arg("--query") - .arg(&query_path) - .arg("--schema") - .arg(&schema_path) - .arg("--json"), - ); - - assert_eq!(stdout_string(&canonical), stdout_string(&deprecated_check)); - - let check_stderr = String::from_utf8(deprecated_check.stderr).unwrap(); - assert!( - check_stderr.contains("`omnigraph check` is deprecated") - && check_stderr.contains("`omnigraph lint`"), - "expected `omnigraph check` deprecation warning pointing at `omnigraph lint`; got: {check_stderr}" - ); - - // `check` must NOT appear in the canonical `omnigraph --help` output — - // agents copy the surface from help text and would otherwise emit both - // names interchangeably. - let help = cli().arg("--help").output().unwrap(); - let stdout = String::from_utf8(help.stdout).unwrap(); - let check_aliased = stdout - .lines() - .any(|line| line.trim_start().starts_with("lint") && line.contains("check")); - assert!( - !check_aliased, - "`check` must not be advertised as a visible alias of `lint`; help output: {stdout}" - ); -} - -/// `omnigraph read` and `omnigraph change` are kept as visible clap -/// aliases for the new canonical `query` / `mutate` subcommands, plus an -/// argv-level deprecation warning. The warning is emitted to stderr; the -/// command otherwise behaves identically to the canonical form. -#[test] -fn deprecated_read_and_change_subcommands_emit_warnings() { - // Both subcommands require `--query`/`--query-string`/`--alias`, so - // invoking them with no args will exit non-zero. That's fine -- - // we only care that the deprecation warning is printed before the - // argument-required error. - let output = cli().arg("read").output().unwrap(); - let stderr = String::from_utf8(output.stderr).unwrap(); - assert!( - stderr.contains("`omnigraph read` is deprecated") && stderr.contains("`omnigraph query`"), - "expected `omnigraph read` deprecation warning; got: {stderr}" - ); - - let output = cli().arg("change").output().unwrap(); - let stderr = String::from_utf8(output.stderr).unwrap(); - assert!( - stderr.contains("`omnigraph change` is deprecated") - && stderr.contains("`omnigraph mutate`"), - "expected `omnigraph change` deprecation warning; got: {stderr}" - ); - - // Sanity check the inverse: the canonical names must NOT print the - // deprecation banner. - let output = cli().arg("query").arg("--help").output().unwrap(); - let stderr = String::from_utf8(output.stderr).unwrap(); - assert!( - !stderr.contains("deprecated"), - "`omnigraph query` is canonical and must not warn; got: {stderr}" - ); - let output = cli().arg("mutate").arg("--help").output().unwrap(); - let stderr = String::from_utf8(output.stderr).unwrap(); - assert!( - !stderr.contains("deprecated"), - "`omnigraph mutate` is canonical and must not warn; got: {stderr}" - ); -} - -#[test] -fn query_lint_can_use_local_graph_via_positional_uri() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let query_path = temp.path().join("queries.gq"); - init_graph(&graph); - write_query_file( - &query_path, - r#" -query list_people() { - match { $p: Person } - return { $p.name } -} -"#, - ); - - let output = output_success( - cli() - .arg("query") - .arg("lint") - .arg("--query") - .arg(&query_path) - .arg("--json") - .arg(&graph), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["status"], "ok"); - assert_eq!(payload["schema_source"]["kind"], "graph"); - assert_eq!( - payload["schema_source"]["uri"].as_str(), - Some(graph.to_string_lossy().as_ref()) - ); -} - -#[test] -fn query_lint_can_resolve_graph_and_query_from_config() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let config_path = temp.path().join("omnigraph.yaml"); - init_graph(&graph); - write_query_file( - &temp.path().join("queries.gq"), - r#" -query list_people() { - match { $p: Person } - return { $p.name } -} -"#, - ); - write_config(&config_path, &local_yaml_config(&graph)); - - let output = output_success( - cli() - .arg("query") - .arg("lint") - .arg("--query") - .arg("queries.gq") - .arg("--config") - .arg(&config_path) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["status"], "ok"); - assert_eq!(payload["schema_source"]["kind"], "graph"); - assert_eq!( - payload["schema_source"]["uri"].as_str(), - Some(graph.to_string_lossy().as_ref()) - ); -} - -#[test] -fn query_lint_rejects_http_targets_without_schema() { - let temp = tempdir().unwrap(); - let query_path = temp.path().join("queries.gq"); - write_query_file( - &query_path, - r#" -query list_people() { - match { $p: Person } - return { $p.name } -} -"#, - ); - - let output = output_failure( - cli() - .arg("query") - .arg("lint") - .arg("--query") - .arg(&query_path) - .arg("http://127.0.0.1:8080"), - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - stderr.contains("query lint is only supported against local graph URIs in this milestone") - ); -} - -#[test] -fn query_lint_requires_schema_or_resolvable_graph_target() { - let temp = tempdir().unwrap(); - let query_path = temp.path().join("queries.gq"); - write_query_file( - &query_path, - r#" -query list_people() { - match { $p: Person } - return { $p.name } -} -"#, - ); - - let output = output_failure( - cli() - .arg("query") - .arg("lint") - .arg("--query") - .arg(&query_path), - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - stderr.contains("query lint requires --schema <schema.pg> or a resolvable graph target") - ); -} - -#[test] -fn query_lint_human_output_reports_warnings() { - let temp = tempdir().unwrap(); - let schema_path = temp.path().join("schema.pg"); - let query_path = temp.path().join("queries.gq"); - write_file( - &schema_path, - r#" -node Policy { - slug: String @key - name: String? - effectiveTo: DateTime? -} -"#, - ); - write_query_file( - &query_path, - r#" -query update_policy($slug: String, $name: String) { - update Policy set { name: $name } where slug = $slug -} -"#, - ); - - let output = output_success( - cli() - .arg("query") - .arg("lint") - .arg("--query") - .arg(&query_path) - .arg("--schema") - .arg(&schema_path), - ); - let stdout = stdout_string(&output); - - assert!(stdout.contains("OK query `update_policy` (mutation)")); - assert!( - stdout.contains("WARN Policy.effectiveTo exists in schema but no update query sets it") - ); - assert!(stdout.contains( - "INFO Lint complete: 1 queries processed (0 error(s), 1 warning(s), 0 info item(s))" - )); -} - -#[test] -fn query_lint_human_output_reports_strict_validation_errors() { - let temp = tempdir().unwrap(); - let schema_path = temp.path().join("schema.pg"); - let query_path = temp.path().join("queries.gq"); - write_file( - &schema_path, - r#" -node Policy { - slug: String @key - name: String? -} -"#, - ); - write_query_file( - &query_path, - r#" -query bad_update($slug: String) { - update Policy set { priority_level: "high" } where slug = $slug -} -"#, - ); - - let output = output_failure( - cli() - .arg("query") - .arg("lint") - .arg("--query") - .arg(&query_path) - .arg("--schema") - .arg(&schema_path), - ); - let stdout = stdout_string(&output); - - assert!(stdout.contains("ERROR query `bad_update`:")); - assert!(stdout.contains("Policy")); - assert!(stdout.contains( - "INFO Lint complete: 1 queries processed (1 error(s), 0 warning(s), 0 info item(s))" - )); -} - -#[test] -fn load_json_outputs_summary_for_main_branch() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - let data = fixture("test.jsonl"); - - let output = output_success( - cli() - .arg("load") - .arg("--mode") - .arg("overwrite") - .arg("--data") - .arg(&data) - .arg("--json") - .arg(&graph), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["branch"], "main"); - assert_eq!(payload["mode"], "overwrite"); - assert_eq!(payload["nodes_loaded"], 6); - assert_eq!(payload["edges_loaded"], 5); - assert_eq!(payload["node_types_loaded"], 2); - assert_eq!(payload["edge_types_loaded"], 2); -} - -#[test] -fn load_into_feature_branch_with_merge_mode_succeeds() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - output_success( - cli() - .arg("branch") - .arg("create") - .arg("--uri") - .arg(&graph) - .arg("--from") - .arg("main") - .arg("feature"), - ); - - let feature_data = temp.path().join("feature.jsonl"); - write_jsonl( - &feature_data, - r#"{"type":"Person","data":{"name":"Alice","age":31}}"#, - ); - - let output = output_success( - cli() - .arg("load") - .arg("--data") - .arg(&feature_data) - .arg("--branch") - .arg("feature") - .arg("--mode") - .arg("merge") - .arg(&graph), - ); - let stdout = stdout_string(&output); - - assert!(stdout.contains("branch feature")); - assert!(stdout.contains("with merge")); - assert!(stdout.contains("1 nodes across 1 node types")); -} - -#[test] -fn read_json_outputs_rows_for_named_query() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - let queries = fixture("test.gq"); - - let output = output_success( - cli() - .arg("read") - .arg(&graph) - .arg("--query") - .arg(&queries) - .arg("--name") - .arg("get_person") - .arg("--params") - .arg(r#"{"name":"Alice"}"#) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["query_name"], "get_person"); - assert_eq!(payload["target"]["branch"], "main"); - assert_eq!(payload["row_count"], 1); - assert_eq!(payload["rows"][0]["p.name"], "Alice"); -} - -#[test] -fn export_jsonl_outputs_source_rows_for_selected_branch_and_type() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - output_success( - cli() - .arg("branch") - .arg("create") - .arg("--uri") - .arg(&graph) - .arg("--from") - .arg("main") - .arg("feature"), - ); - - let feature_data = temp.path().join("feature-export.jsonl"); - write_jsonl( - &feature_data, - r#"{"type":"Person","data":{"name":"Eve","age":29}}"#, - ); - output_success( - cli() - .arg("load") - .arg("--data") - .arg(&feature_data) - .arg("--branch") - .arg("feature") - .arg("--mode") - .arg("append") - .arg(&graph), - ); - - let output = output_success( - cli() - .arg("export") - .arg(&graph) - .arg("--branch") - .arg("feature") - .arg("--type") - .arg("Person") - .arg("--jsonl"), - ); - let rows = stdout_string(&output) - .lines() - .map(|line| serde_json::from_str::<Value>(line).unwrap()) - .collect::<Vec<_>>(); - - assert_eq!(rows.len(), 5); - assert!(rows.iter().all(|row| row["type"] == "Person")); - assert!(rows.iter().all(|row| row.get("edge").is_none())); - assert!( - rows.iter() - .any(|row| row["data"]["name"].as_str() == Some("Eve")) - ); -} - -#[test] -fn policy_validate_accepts_valid_policy_file() { - let temp = tempdir().unwrap(); - let (config, _) = write_policy_config_fixture(temp.path()); - - let output = output_success( - cli() - .arg("policy") - .arg("validate") - .arg("--config") - .arg(&config), - ); - let stdout = stdout_string(&output); - - assert!(stdout.contains("policy valid:")); - assert!(stdout.contains("policy.yaml")); - assert!(stdout.contains("[2 actors]")); -} - -#[test] -fn policy_validate_fails_for_invalid_policy_file() { - let temp = tempdir().unwrap(); - let config = temp.path().join("omnigraph.yaml"); - let policy = temp.path().join("policy.yaml"); - fs::write( - &config, - r#" -project: - name: policy-test-graph -policy: - file: ./policy.yaml -"#, - ) - .unwrap(); - fs::write( - &policy, - r#" -version: 1 -groups: - team: [act-andrew] -rules: - - id: duplicate - allow: - actors: { group: team } - actions: [read] - branch_scope: any - - id: duplicate - allow: - actors: { group: team } - actions: [export] - branch_scope: any -"#, - ) - .unwrap(); - - let output = output_failure( - cli() - .arg("policy") - .arg("validate") - .arg("--config") - .arg(&config), - ); - let stderr = String::from_utf8(output.stderr).unwrap(); - assert!(stderr.contains("duplicate policy rule id")); -} - -#[test] -fn policy_test_runs_declarative_cases() { - let temp = tempdir().unwrap(); - let (config, _) = write_policy_config_fixture(temp.path()); - - let output = output_success(cli().arg("policy").arg("test").arg("--config").arg(&config)); - let stdout = stdout_string(&output); - - assert!(stdout.contains("policy tests passed: 2 cases")); -} - -#[test] -fn policy_explain_reports_decision_and_matched_rule() { - let temp = tempdir().unwrap(); - let (config, _) = write_policy_config_fixture(temp.path()); - - let allow = output_success( - cli() - .arg("policy") - .arg("explain") - .arg("--config") - .arg(&config) - .arg("--actor") - .arg("act-andrew") - .arg("--action") - .arg("change") - .arg("--branch") - .arg("feature"), - ); - let allow_stdout = stdout_string(&allow); - assert!(allow_stdout.contains("decision: allow")); - assert!(allow_stdout.contains("matched_rule: team-write")); - - let deny = output_success( - cli() - .arg("policy") - .arg("explain") - .arg("--config") - .arg(&config) - .arg("--actor") - .arg("act-bruno") - .arg("--action") - .arg("change") - .arg("--branch") - .arg("main"), - ); - let deny_stdout = stdout_string(&deny); - assert!(deny_stdout.contains("decision: deny")); - assert!(deny_stdout.contains("message: policy denied action 'change' on branch 'main'")); -} - -#[test] -fn read_can_resolve_uri_from_config() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let config = temp.path().join("omnigraph.yaml"); - init_graph(&graph); - load_fixture(&graph); - write_config(&config, &local_yaml_config(&graph)); - - let output = output_success( - cli() - .arg("read") - .arg("--config") - .arg(&config) - .arg("--query") - .arg(fixture("test.gq")) - .arg("--name") - .arg("get_person") - .arg("--params") - .arg(r#"{"name":"Alice"}"#) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["row_count"], 1); -} - -#[test] -fn read_alias_from_yaml_config_runs_with_kv_output() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let config = temp.path().join("omnigraph.yaml"); - let query = temp.path().join("aliases.gq"); - init_graph(&graph); - load_fixture(&graph); - write_query_file( - &query, - &std::fs::read_to_string(fixture("test.gq")).unwrap(), - ); - write_config( - &config, - &format!( - "{}aliases:\n owner:\n command: read\n query: aliases.gq\n name: get_person\n args: [name]\n format: kv\n", - local_yaml_config(&graph) - ), - ); - - let output = output_success( - cli() - .arg("read") - .arg("--config") - .arg(&config) - .arg("--alias") - .arg("owner") - .arg("Alice"), - ); - let stdout = stdout_string(&output); - - assert!(stdout.contains("row 1")); - assert!(stdout.contains("p.name: Alice")); -} - -#[test] -fn read_alias_uses_alias_target_without_cli_default_and_accepts_url_like_arg() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let config = temp.path().join("omnigraph.yaml"); - let query = temp.path().join("aliases.gq"); - let data = temp.path().join("url-like.jsonl"); - init_graph(&graph); - write_jsonl( - &data, - r#"{"type":"Person","data":{"name":"https://example.com","age":30}}"#, - ); - output_success( - cli() - .arg("load") - .arg("--mode") - .arg("overwrite") - .arg("--data") - .arg(&data) - .arg(&graph), - ); - write_query_file( - &query, - &std::fs::read_to_string(fixture("test.gq")).unwrap(), - ); - write_config( - &config, - &format!( - "graphs:\n local:\n uri: '{}'\nquery:\n roots:\n - .\npolicy: {{}}\naliases:\n owner:\n command: read\n query: aliases.gq\n name: get_person\n args: [name]\n graph: local\n format: kv\n", - graph.to_string_lossy() - ), - ); - - let output = output_success( - cli() - .arg("read") - .arg("--config") - .arg(&config) - .arg("--alias") - .arg("owner") - .arg("https://example.com"), - ); - let stdout = stdout_string(&output); - - assert!(stdout.contains("row 1")); - assert!(stdout.contains("p.name: https://example.com")); -} - -#[test] -fn change_alias_from_yaml_config_persists_changes() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let config = temp.path().join("omnigraph.yaml"); - let query = temp.path().join("mutations.gq"); - init_graph(&graph); - load_fixture(&graph); - write_query_file( - &query, - r#" -query insert_person($name: String, $age: I32) { - insert Person { name: $name, age: $age } -} -"#, - ); - write_config( - &config, - &format!( - "{}aliases:\n add_person:\n command: change\n query: mutations.gq\n name: insert_person\n args: [name, age]\n", - local_yaml_config(&graph) - ), - ); - - let output = output_success( - cli() - .arg("change") - .arg("--config") - .arg(&config) - .arg("--alias") - .arg("add_person") - .arg("Eve") - .arg("29") - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["affected_nodes"], 1); - - let verify = output_success( - cli() - .arg("read") - .arg(&graph) - .arg("--query") - .arg(fixture("test.gq")) - .arg("--name") - .arg("get_person") - .arg("--params") - .arg(r#"{"name":"Eve"}"#) - .arg("--json"), - ); - let verify_payload: Value = serde_json::from_slice(&verify.stdout).unwrap(); - assert_eq!(verify_payload["row_count"], 1); -} - -#[test] -fn read_csv_format_outputs_header_and_row_values() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - let output = output_success( - cli() - .arg("read") - .arg(&graph) - .arg("--query") - .arg(fixture("test.gq")) - .arg("--name") - .arg("get_person") - .arg("--params") - .arg(r#"{"name":"Alice"}"#) - .arg("--format") - .arg("csv"), - ); - let stdout = stdout_string(&output); - - assert!(stdout.lines().next().unwrap().contains("p.name")); - assert!(stdout.contains("Alice")); -} - -#[test] -fn read_jsonl_format_outputs_metadata_header_first() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - let output = output_success( - cli() - .arg("read") - .arg(&graph) - .arg("--query") - .arg(fixture("test.gq")) - .arg("--name") - .arg("get_person") - .arg("--params") - .arg(r#"{"name":"Alice"}"#) - .arg("--format") - .arg("jsonl"), - ); - let stdout = stdout_string(&output); - let mut lines = stdout.lines(); - assert!(lines.next().unwrap().contains("\"kind\":\"metadata\"")); - assert!(lines.next().unwrap().contains("\"p.name\":\"Alice\"")); -} - -#[test] -fn change_json_outputs_affected_counts_and_persists() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - let mutation_file = temp.path().join("mutations.gq"); - write_query_file( - &mutation_file, - r#" -query insert_person($name: String, $age: I32) { - insert Person { name: $name, age: $age } -} -"#, - ); - - let output = output_success( - cli() - .arg("change") - .arg(&graph) - .arg("--query") - .arg(&mutation_file) - .arg("--params") - .arg(r#"{"name":"Eve","age":29}"#) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["branch"], "main"); - assert_eq!(payload["query_name"], "insert_person"); - assert_eq!(payload["affected_nodes"], 1); - assert_eq!(payload["affected_edges"], 0); - - let verify = output_success( - cli() - .arg("read") - .arg(&graph) - .arg("--query") - .arg(fixture("test.gq")) - .arg("--name") - .arg("get_person") - .arg("--params") - .arg(r#"{"name":"Eve"}"#) - .arg("--json"), - ); - let verify_payload: Value = serde_json::from_slice(&verify.stdout).unwrap(); - assert_eq!(verify_payload["row_count"], 1); - assert_eq!(verify_payload["rows"][0]["p.name"], "Eve"); -} - -#[test] -fn change_can_resolve_uri_and_branch_from_config() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let config = temp.path().join("omnigraph.yaml"); - init_graph(&graph); - load_fixture(&graph); - write_config(&config, &local_yaml_config(&graph)); - let mutation_file = temp.path().join("config-mutations.gq"); - write_query_file( - &mutation_file, - r#" -query insert_person($name: String, $age: I32) { - insert Person { name: $name, age: $age } -} -"#, - ); - - let output = output_success( - cli() - .arg("change") - .arg("--config") - .arg(&config) - .arg("--query") - .arg(&mutation_file) - .arg("--params") - .arg(r#"{"name":"Mia","age":30}"#) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["branch"], "main"); - assert_eq!(payload["affected_nodes"], 1); -} - -#[test] -fn read_requires_name_for_multi_query_files() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - let output = output_failure( - cli() - .arg("read") - .arg(&graph) - .arg("--query") - .arg(fixture("test.gq")), - ); - let stderr = String::from_utf8(output.stderr).unwrap(); - assert!(stderr.contains("multiple queries")); -} - -#[test] -fn read_supports_inline_query_string() { - let temp = tempdir().unwrap(); - let repo = graph_path(temp.path()); - init_graph(&repo); - load_fixture(&repo); - - let output = output_success( - cli() - .arg("read") - .arg(&repo) - .arg("-e") - .arg("query find($name: String) { match { $p: Person { name: $name } } return { $p.name, $p.age } }") - .arg("--params") - .arg(r#"{"name":"Alice"}"#) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["query_name"], "find"); - assert_eq!(payload["row_count"], 1); - assert_eq!(payload["rows"][0]["p.name"], "Alice"); -} - -#[test] -fn change_supports_inline_query_string() { - let temp = tempdir().unwrap(); - let repo = graph_path(temp.path()); - init_graph(&repo); - load_fixture(&repo); - - let output = output_success( - cli() - .arg("change") - .arg(&repo) - .arg("--query-string") - .arg("query add($name: String, $age: I32) { insert Person { name: $name, age: $age } }") - .arg("--params") - .arg(r#"{"name":"Inline","age":42}"#) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["query_name"], "add"); - assert_eq!(payload["affected_nodes"], 1); - - let verify = output_success( - cli() - .arg("read") - .arg(&repo) - .arg("-e") - .arg("query find($name: String) { match { $p: Person { name: $name } } return { $p.name } }") - .arg("--params") - .arg(r#"{"name":"Inline"}"#) - .arg("--json"), - ); - let verify_payload: Value = serde_json::from_slice(&verify.stdout).unwrap(); - assert_eq!(verify_payload["row_count"], 1); -} - -#[test] -fn read_rejects_query_string_combined_with_query() { - let temp = tempdir().unwrap(); - let repo = graph_path(temp.path()); - init_graph(&repo); - load_fixture(&repo); - - let output = output_failure( - cli() - .arg("read") - .arg(&repo) - .arg("--query") - .arg(fixture("test.gq")) - .arg("-e") - .arg("query whatever() { match { $p: Person } return { $p.name } }"), - ); - let stderr = String::from_utf8(output.stderr).unwrap(); - assert!( - stderr.contains("cannot be used") || stderr.contains("conflict"), - "expected clap conflict error, got: {stderr}" - ); -} - -#[test] -fn read_rejects_empty_query_string() { - let temp = tempdir().unwrap(); - let repo = graph_path(temp.path()); - init_graph(&repo); - load_fixture(&repo); - - let output = output_failure(cli().arg("read").arg(&repo).arg("-e").arg("")); - let stderr = String::from_utf8(output.stderr).unwrap(); - assert!( - stderr.contains("must not be empty"), - "expected empty-string rejection, got: {stderr}" - ); -} - -#[test] -fn branch_create_json_outputs_source_and_name() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - - let output = output_success( - cli() - .arg("branch") - .arg("create") - .arg("--uri") - .arg(&graph) - .arg("--from") - .arg("main") - .arg("feature") - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["from"], "main"); - assert_eq!(payload["name"], "feature"); - assert_eq!(payload["uri"], graph.to_string_lossy().as_ref()); -} - -#[test] -fn branch_list_outputs_sorted_branches() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - - output_success( - cli() - .arg("branch") - .arg("create") - .arg("--uri") - .arg(&graph) - .arg("--from") - .arg("main") - .arg("zeta"), - ); - output_success( - cli() - .arg("branch") - .arg("create") - .arg("--uri") - .arg(&graph) - .arg("--from") - .arg("main") - .arg("alpha"), - ); - - let output = output_success(cli().arg("branch").arg("list").arg("--uri").arg(&graph)); - let stdout = stdout_string(&output); - let lines = stdout - .lines() - .map(str::trim) - .filter(|line| !line.is_empty()) - .collect::<Vec<_>>(); - - assert_eq!(lines, vec!["alpha", "main", "zeta"]); -} - -#[test] -fn branch_delete_json_outputs_name_and_removes_branch() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - - output_success( - cli() - .arg("branch") - .arg("create") - .arg("--uri") - .arg(&graph) - .arg("--from") - .arg("main") - .arg("feature"), - ); - - let output = output_success( - cli() - .arg("branch") - .arg("delete") - .arg("--uri") - .arg(&graph) - .arg("feature") - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["name"], "feature"); - assert_eq!(payload["uri"], graph.to_string_lossy().as_ref()); - - let listed = output_success(cli().arg("branch").arg("list").arg("--uri").arg(&graph)); - let stdout = stdout_string(&listed); - let lines = stdout - .lines() - .map(str::trim) - .filter(|line| !line.is_empty()) - .collect::<Vec<_>>(); - assert_eq!(lines, vec!["main"]); -} - -#[test] -fn branch_delete_rejects_main() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - - let output = output_failure( - cli() - .arg("branch") - .arg("delete") - .arg("--uri") - .arg(&graph) - .arg("main"), - ); - let stderr = String::from_utf8(output.stderr).unwrap(); - assert!(stderr.contains("cannot delete branch 'main'")); -} - -#[test] -fn branch_merge_defaults_target_to_main() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - output_success( - cli() - .arg("branch") - .arg("create") - .arg("--uri") - .arg(&graph) - .arg("--from") - .arg("main") - .arg("feature"), - ); - - let feature_data = temp.path().join("feature.jsonl"); - write_jsonl( - &feature_data, - r#"{"type":"Person","data":{"name":"Eve","age":29}}"#, - ); - output_success( - cli() - .arg("load") - .arg("--data") - .arg(&feature_data) - .arg("--branch") - .arg("feature") - .arg("--mode") - .arg("append") - .arg(&graph), - ); - - let merge_output = output_success( - cli() - .arg("branch") - .arg("merge") - .arg("--uri") - .arg(&graph) - .arg("feature") - .arg("--json"), - ); - let merge_payload: Value = serde_json::from_slice(&merge_output.stdout).unwrap(); - assert_eq!(merge_payload["source"], "feature"); - assert_eq!(merge_payload["target"], "main"); - assert_eq!(merge_payload["outcome"], "fast_forward"); - - let snapshot_output = output_success( - cli() - .arg("snapshot") - .arg(&graph) - .arg("--branch") - .arg("main") - .arg("--json"), - ); - let snapshot: Value = serde_json::from_slice(&snapshot_output.stdout).unwrap(); - let person_row_count = snapshot["tables"] - .as_array() - .unwrap() - .iter() - .find(|table| table["table_key"] == "node:Person") - .unwrap()["row_count"] - .as_u64() - .unwrap(); - assert_eq!(person_row_count, 5); -} - -#[test] -fn branch_merge_supports_explicit_target() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - output_success( - cli() - .arg("branch") - .arg("create") - .arg("--uri") - .arg(&graph) - .arg("--from") - .arg("main") - .arg("feature"), - ); - output_success( - cli() - .arg("branch") - .arg("create") - .arg("--uri") - .arg(&graph) - .arg("--from") - .arg("main") - .arg("experiment"), - ); - - let feature_data = temp.path().join("feature-explicit.jsonl"); - write_jsonl( - &feature_data, - r#"{"type":"Person","data":{"name":"Frank","age":41}}"#, - ); - output_success( - cli() - .arg("load") - .arg("--data") - .arg(&feature_data) - .arg("--branch") - .arg("feature") - .arg("--mode") - .arg("append") - .arg(&graph), - ); - - let merge_output = output_success( - cli() - .arg("branch") - .arg("merge") - .arg("--uri") - .arg(&graph) - .arg("feature") - .arg("--into") - .arg("experiment") - .arg("--json"), - ); - let merge_payload: Value = serde_json::from_slice(&merge_output.stdout).unwrap(); - assert_eq!(merge_payload["target"], "experiment"); - assert_eq!(merge_payload["outcome"], "fast_forward"); -} - -#[test] -fn snapshot_json_returns_manifest_version_and_tables() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - let output = output_success(cli().arg("snapshot").arg(&graph).arg("--json")); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["branch"], "main"); - assert_eq!( - payload["manifest_version"].as_u64().unwrap(), - manifest_dataset_version(&graph) - ); - assert!(payload["tables"].as_array().unwrap().len() >= 4); -} - -fn write_seed_fixture(root: &std::path::Path) -> std::path::PathBuf { - fs::create_dir_all(root.join("data")).unwrap(); - fs::create_dir_all(root.join("build")).unwrap(); - let raw_seed = root.join("data/seed.jsonl"); - let seed = root.join("seed.yaml"); - - fs::write( - &raw_seed, - concat!( - "{\"type\":\"Decision\",\"data\":{\"slug\":\"dec-alpha\",\"intent\":\"Alpha ship\"}}\n", - "{\"type\":\"Decision\",\"data\":{\"slug\":\"dec-beta\",\"intent\":\"Beta ship\",\"embedding\":[0.1,0.2]}}\n" - ), - ) - .unwrap(); - - fs::write( - &seed, - concat!( - "graph:\n", - " slug: mr-context-graph\n", - "sources:\n", - " raw_seed: ./data/seed.jsonl\n", - "artifacts:\n", - " embedded_seed: ./build/seed.embedded.jsonl\n", - "embeddings:\n", - " model: gemini-embedding-2-preview\n", - " dimension: 4\n", - " types:\n", - " Decision:\n", - " target: embedding\n", - " fields: [slug, intent]\n" - ), - ) - .unwrap(); - - seed -} - -fn write_seed_fixture_with_edge(root: &std::path::Path) -> std::path::PathBuf { - let seed = write_seed_fixture(root); - let raw_seed = root.join("data/seed.jsonl"); - fs::write( - &raw_seed, - concat!( - "{\"type\":\"Decision\",\"data\":{\"slug\":\"dec-alpha\",\"intent\":\"Alpha ship\"}}\n", - "{\"type\":\"Decision\",\"data\":{\"slug\":\"dec-beta\",\"intent\":\"Beta ship\",\"embedding\":[0.1,0.2]}}\n", - "{\"edge\":\"Triggered\",\"from\":\"sig-alpha\",\"to\":\"dec-alpha\"}\n" - ), - ) - .unwrap(); - seed -} - -fn read_embedded_rows(path: std::path::PathBuf) -> Vec<Value> { - fs::read_to_string(path) - .unwrap() - .lines() - .filter(|line| !line.trim().is_empty()) - .map(|line| serde_json::from_str(line).unwrap()) - .collect() -} - -#[test] -fn snapshot_can_resolve_uri_from_config() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let config = temp.path().join("omnigraph.yaml"); - init_graph(&graph); - load_fixture(&graph); - write_config(&config, &local_yaml_config(&graph)); - - let output = output_success( - cli() - .arg("snapshot") - .arg("--config") - .arg(&config) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["branch"], "main"); -} - -#[test] -fn snapshot_human_output_includes_branch_and_table_summaries() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - let output = output_success(cli().arg("snapshot").arg(&graph)); - let stdout = stdout_string(&output); - - assert!(stdout.contains("branch: main")); - assert!(stdout.contains("manifest_version:")); - assert!(stdout.contains("node:Person v")); - assert!(stdout.contains("edge:Knows v")); -} - -#[test] -fn commit_show_accepts_long_uri_flag() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - let list = output_success(cli().arg("commit").arg("list").arg(&graph).arg("--json")); - let list_payload: Value = serde_json::from_slice(&list.stdout).unwrap(); - let commit_id = list_payload["commits"][0]["graph_commit_id"] - .as_str() - .unwrap() - .to_string(); - - let output = output_success( - cli() - .arg("commit") - .arg("show") - .arg("--uri") - .arg(&graph) - .arg(&commit_id) - .arg("--json"), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - - assert_eq!(payload["graph_commit_id"], commit_id); - assert!(payload["manifest_version"].as_u64().unwrap() >= 1); -} - -#[test] -fn cli_fails_for_missing_graph() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - - let output = output_failure(cli().arg("snapshot").arg(&graph)); - let stderr = String::from_utf8(output.stderr).unwrap(); - assert!( - stderr.contains("_schema.pg") - || stderr.contains("No such file") - || stderr.contains("not found") - ); -} - -#[test] -fn cli_fails_for_missing_schema_or_data_file() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let missing_schema = temp.path().join("missing.pg"); - let missing_data = temp.path().join("missing.jsonl"); - - let init_output = output_failure( - cli() - .arg("init") - .arg("--schema") - .arg(&missing_schema) - .arg(&graph), - ); - assert!( - String::from_utf8(init_output.stderr) - .unwrap() - .contains("No such file") - ); - - init_graph(&graph); - let load_output = output_failure( - cli() - .arg("load") - .arg("--mode") - .arg("overwrite") - .arg("--data") - .arg(&missing_data) - .arg(&graph), - ); - assert!( - String::from_utf8(load_output.stderr) - .unwrap() - .contains("No such file") - ); -} - -#[test] -fn cli_fails_for_invalid_merge_requests() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - load_fixture(&graph); - - let missing_branch = output_failure( - cli() - .arg("branch") - .arg("merge") - .arg("--uri") - .arg(&graph) - .arg("missing"), - ); - let missing_branch_stderr = String::from_utf8(missing_branch.stderr).unwrap(); - assert!( - missing_branch_stderr.contains("missing") - || missing_branch_stderr.contains("head commit") - || missing_branch_stderr.contains("not found") - ); - - let same_branch = output_failure( - cli() - .arg("branch") - .arg("merge") - .arg("--uri") - .arg(&graph) - .arg("main") - .arg("--into") - .arg("main"), - ); - assert!( - String::from_utf8(same_branch.stderr) - .unwrap() - .contains("distinct source and target") - ); -} - -// `omnigraph run list/show/publish/abort` subcommands removed -// alongside the run state machine. Direct-to-target writes leave nothing -// for these CLIs to manage. Audit history is now visible via -// `omnigraph commit list` reading the commit graph. - -// ─── MR-694 PR B: --allow-data-loss flag end-to-end ────────────────────── -// -// The schema-lint chassis v1.2 (PR #100) shipped the `--allow-data-loss` -// flag at the CLI layer; the SDK suite verifies promotion to Hard mode -// via `apply_schema_with_options(.., SchemaApplyOptions { allow_data_loss })`. -// These CLI tests close the integration gap so a future change that -// drops the flag wiring in `main.rs` turns red. - -#[test] -fn schema_apply_allow_data_loss_flag_promotes_drops_to_hard() { - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema_path = temp.path().join("drop-age.pg"); - init_graph(&graph); - - // Drop the nullable `age` column. - let next_schema = fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace(" age: I32?\n", ""); - fs::write(&schema_path, next_schema).unwrap(); - - let output = output_success( - cli() - .arg("schema") - .arg("apply") - .arg("--schema") - .arg(&schema_path) - .arg("--allow-data-loss") - .arg("--json") - .arg(&graph), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["applied"], true); - - let drop_step = payload["steps"] - .as_array() - .unwrap() - .iter() - .find(|s| s["kind"] == "drop_property") - .expect("plan should include a drop_property step"); - assert_eq!( - drop_step["mode"], "hard", - "--allow-data-loss should promote Soft → Hard; full step: {drop_step}", - ); -} - -#[test] -fn schema_apply_without_allow_data_loss_keeps_soft_drops() { - // Symmetric to the above: same schema change without the flag → - // drops stay Soft. Pins default semantics against accidental Hard - // promotion if a future refactor changes the option threading. - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - let schema_path = temp.path().join("drop-age-soft.pg"); - init_graph(&graph); - - let next_schema = fs::read_to_string(fixture("test.pg")) - .unwrap() - .replace(" age: I32?\n", ""); - fs::write(&schema_path, next_schema).unwrap(); - - let output = output_success( - cli() - .arg("schema") - .arg("apply") - .arg("--schema") - .arg(&schema_path) - .arg("--json") - .arg(&graph), - ); - let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["applied"], true); - - let drop_step = payload["steps"] - .as_array() - .unwrap() - .iter() - .find(|s| s["kind"] == "drop_property") - .expect("plan should include a drop_property step"); - assert_eq!( - drop_step["mode"], "soft", - "no flag should leave drops Soft; full step: {drop_step}", - ); -} - -#[test] -fn schema_plan_parity_cli_and_sdk() { - // Same .pg through `Omnigraph::plan_schema_with_options` (SDK) and - // `omnigraph schema plan --json` (CLI). Asserts the steps array is - // byte-identical after JSON round-trip. HTTP doesn't expose a - // separate /schema/plan route — that side of parity is covered by - // the HTTP soft/hard drop tests, which exercise apply with - // identical fixtures. - let temp = tempdir().unwrap(); - let graph = graph_path(temp.path()); - init_graph(&graph); - let schema_path = temp.path().join("plan-parity.pg"); - let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace( - " age: I32?\n}", - " age: I32?\n nickname: String?\n}", - ); - fs::write(&schema_path, &next_schema).unwrap(); - - // CLI side. - let cli_output = output_success( - cli() - .arg("schema") - .arg("plan") - .arg("--schema") - .arg(&schema_path) - .arg("--json") - .arg(&graph), - ); - let cli_payload: Value = serde_json::from_slice(&cli_output.stdout).unwrap(); - - // SDK side: open graph, call plan_schema. - let plan = tokio::runtime::Runtime::new().unwrap().block_on(async { - let db = Omnigraph::open(graph.to_string_lossy().as_ref()) - .await - .unwrap(); - db.plan_schema(&next_schema).await.unwrap() - }); - let sdk_steps = serde_json::to_value(&plan.steps).unwrap(); - - assert_eq!( - cli_payload["steps"], sdk_steps, - "CLI plan steps must match SDK plan steps for identical input", - ); - assert_eq!(cli_payload["supported"], plan.supported); -} - -// ─── MR-668 PR 8 — omnigraph graphs subcommand ───────────────────────────── - -/// `omnigraph graphs --help` lists only the read-only `list` -/// subcommand. Runtime add (`create`) and remove (`delete`) are -/// deferred — operators add/remove graphs by editing `omnigraph.yaml` -/// and restarting. This test pins the deferral against accidental -/// re-introduction. -#[test] -fn graphs_subcommand_help_lists_list_only() { - let output = output_success(cli().arg("graphs").arg("--help")); - let stdout = stdout_string(&output); - assert!( - stdout.contains("list"), - "expected `list` subcommand in help output:\n{stdout}" - ); - let lowered = stdout.to_lowercase(); - assert!( - !lowered.contains("create a new graph"), - "graph create should not be in v0.6.0 help; got:\n{stdout}" - ); - assert!( - !lowered.contains("delete a graph"), - "graph delete should not be in v0.6.0 help; got:\n{stdout}" - ); -} - -/// `omnigraph graphs list` against a local URI errors with a clear -/// message — the CLI only operates against remote multi-graph servers. -#[test] -fn graphs_list_against_local_uri_errors_with_remote_only_message() { - let output = output_failure( - cli() - .arg("graphs") - .arg("list") - .arg("--uri") - .arg("/tmp/local"), - ); - let stderr = String::from_utf8_lossy(&output.stderr).into_owned(); - assert!( - stderr.contains("remote multi-graph server URL"), - "expected 'remote multi-graph server URL' rejection in stderr; got:\n{stderr}" - ); -} - -fn queries_test_config(graph_uri: &str, entry: &str, gq_file: &str) -> String { - format!( - "graphs:\n local:\n uri: '{}'\n queries:\n {entry}:\n file: ./{gq_file}\n\ - cli:\n graph: local\npolicy: {{}}\n", - graph_uri.replace('\'', "''") - ) -} - -#[test] -fn queries_validate_exits_zero_on_clean_registry() { - let graph = SystemGraph::loaded(); - graph.write_query( - "find_person.gq", - "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", - ); - let config = graph.write_config( - "omnigraph.yaml", - &queries_test_config( - &graph.path().to_string_lossy(), - "find_person", - "find_person.gq", - ), - ); - let output = output_success( - cli() - .arg("queries") - .arg("validate") - .arg("--config") - .arg(&config), - ); - let stdout = stdout_string(&output); - assert!(stdout.contains("OK"), "stdout:\n{stdout}"); -} - -#[test] -fn queries_validate_exits_nonzero_on_type_broken_query() { - let graph = SystemGraph::loaded(); - // `Widget` is not in the fixture schema. - graph.write_query( - "ghost.gq", - "query ghost() { match { $w: Widget } return { $w.name } }", - ); - let config = graph.write_config( - "omnigraph.yaml", - &queries_test_config(&graph.path().to_string_lossy(), "ghost", "ghost.gq"), - ); - let output = output_failure( - cli() - .arg("queries") - .arg("validate") - .arg("--config") - .arg(&config), - ); - let stdout = stdout_string(&output); - assert!( - stdout.contains("ghost"), - "validation should name the broken query; stdout:\n{stdout}" - ); -} - -#[test] -fn queries_list_prints_registered_query() { - let graph = SystemGraph::loaded(); - graph.write_query( - "find_person.gq", - "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", - ); - // Exposed with an explicit tool name so the list shows the MCP suffix. - let config = graph.write_config( - "omnigraph.yaml", - &format!( - concat!( - "graphs:\n", - " local:\n", - " uri: '{}'\n", - " queries:\n", - " find_person:\n", - " file: ./find_person.gq\n", - " mcp: {{ expose: true, tool_name: lookup_person }}\n", - "cli:\n", - " graph: local\n", - "policy: {{}}\n", - ), - graph.path().to_string_lossy().replace('\'', "''") - ), - ); - let output = output_success( - cli() - .arg("queries") - .arg("list") - .arg("--config") - .arg(&config), - ); - let stdout = stdout_string(&output); - assert!(stdout.contains("find_person"), "stdout:\n{stdout}"); - assert!( - stdout.contains("$name: String"), - "list should show typed params; stdout:\n{stdout}" - ); - assert!( - stdout.contains("[mcp: lookup_person]"), - "list should show the MCP tool name for exposed queries; stdout:\n{stdout}" - ); -} - -#[test] -fn queries_list_requires_graph_selection_for_per_graph_only_registries() { - let graph = SystemGraph::loaded(); - graph.write_query( - "find_person.gq", - "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", - ); - let config = graph.write_config( - "omnigraph.yaml", - &format!( - concat!( - "graphs:\n", - " local:\n", - " uri: '{}'\n", - " queries:\n", - " find_person:\n", - " file: ./find_person.gq\n", - "policy: {{}}\n", - ), - graph.path().to_string_lossy().replace('\'', "''") - ), - ); - - let output = output_failure( - cli() - .arg("queries") - .arg("list") - .arg("--config") - .arg(&config), - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - stderr.contains("local") && stderr.contains("--target local"), - "error must name the graph and give a concrete selection hint; stderr:\n{stderr}" - ); -} - -#[test] -fn queries_list_without_graph_selection_lists_top_level_registry() { - let graph = SystemGraph::loaded(); - graph.write_query( - "top_find.gq", - "query top_find($name: String) { match { $p: Person { name: $name } } return { $p.age } }", - ); - let config = graph.write_config( - "omnigraph.yaml", - concat!( - "queries:\n", - " top_find:\n", - " file: ./top_find.gq\n", - "policy: {}\n", - ), - ); - - let output = output_success( - cli() - .arg("queries") - .arg("list") - .arg("--config") - .arg(&config), - ); - let stdout = stdout_string(&output); - assert!(stdout.contains("top_find"), "stdout:\n{stdout}"); -} - -#[test] -fn queries_list_unknown_target_errors() { - // `queries list` opens no graph URI, so unknown-graph validation can't ride - // along on URI resolution the way it does for every other command. An - // unknown `--target` must still error (naming the graph) instead of - // silently falling back to the top-level registry and showing the wrong - // (or empty) catalog. - let graph = SystemGraph::loaded(); - graph.write_query( - "find_person.gq", - "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", - ); - let config = graph.write_config( - "omnigraph.yaml", - &queries_test_config( - &graph.path().to_string_lossy(), - "find_person", - "find_person.gq", - ), - ); - let output = output_failure( - cli() - .arg("queries") - .arg("list") - .arg("--target") - .arg("nonexistent") - .arg("--config") - .arg(&config), - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - stderr.contains("nonexistent"), - "error must name the unknown graph; stderr:\n{stderr}" - ); -} - -#[test] -fn queries_commands_reject_named_graph_with_populated_top_level_block() { - // A named graph (here via `cli.graph`) uses its own `graphs.<name>` block, - // so a populated top-level `queries:` block would be silently ignored — a - // config the server REFUSES to boot. `queries validate`/`list` must reject - // it too (matching boot) instead of validating/listing the per-graph block - // and giving a false green. - let graph = SystemGraph::loaded(); - graph.write_query( - "find_person.gq", - "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", - ); - let config = graph.write_config( - "omnigraph.yaml", - &format!( - concat!( - "graphs:\n", - " local:\n", - " uri: '{}'\n", - " queries:\n", - " find_person:\n", - " file: ./find_person.gq\n", - "cli:\n", - " graph: local\n", - "queries:\n", // populated top-level block: the coherence violation - " legacy:\n", - " file: ./legacy.gq\n", - "policy: {{}}\n", - ), - graph.path().to_string_lossy().replace('\'', "''") - ), - ); - // Both resolve `local` from cli.graph (no positional URI), so both must - // error and name the graph + the ignored block — like server boot does. - for sub in ["validate", "list"] { - let output = output_failure(cli().arg("queries").arg(sub).arg("--config").arg(&config)); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - stderr.contains("local") && stderr.contains("queries"), - "`queries {sub}` must reject a named graph with a populated top-level block; stderr:\n{stderr}" - ); - } -} - -#[test] -fn queries_validate_exits_nonzero_on_duplicate_tool_name() { - // Two exposed queries claiming one MCP tool name is a load-time - // collision — `queries validate` must fail (offline, before the engine - // opens) and name both queries plus the contested tool. - let graph = SystemGraph::loaded(); - graph.write_query( - "a.gq", - "query a() { match { $p: Person } return { $p.name } }", - ); - graph.write_query( - "b.gq", - "query b() { match { $p: Person } return { $p.name } }", - ); - let config = graph.write_config( - "omnigraph.yaml", - &format!( - concat!( - "graphs:\n", - " local:\n", - " uri: '{}'\n", - " queries:\n", - " a:\n", - " file: ./a.gq\n", - " mcp: {{ expose: true, tool_name: dup }}\n", - " b:\n", - " file: ./b.gq\n", - " mcp: {{ expose: true, tool_name: dup }}\n", - "cli:\n", - " graph: local\n", - "policy: {{}}\n", - ), - graph.path().to_string_lossy().replace('\'', "''") - ), - ); - let output = output_failure( - cli() - .arg("queries") - .arg("validate") - .arg("--config") - .arg(&config), - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - stderr.contains("dup") && stderr.contains("'a'") && stderr.contains("'b'"), - "duplicate tool name should be reported naming both queries; stderr:\n{stderr}" - ); -} - -#[test] -fn queries_validate_positional_uri_ignores_default_graph() { - // A positional URI is anonymous → the schema AND the registry both come - // from top-level, even when `cli.graph` names a graph whose per-graph - // queries would fail. Pins that the URI and registry can't diverge. - let graph = SystemGraph::loaded(); - graph.write_query( - "clean.gq", - "query clean($name: String) { match { $p: Person { name: $name } } return { $p.age } }", - ); - // `Widget` is not in the fixture schema — the default graph's per-graph - // query would break validate if it were (wrongly) selected. - graph.write_query( - "broken.gq", - "query broken() { match { $w: Widget } return { $w.name } }", - ); - let config = graph.write_config( - "omnigraph.yaml", - concat!( - "cli:\n graph: prod\n", - "graphs:\n", - " prod:\n", - " uri: /nonexistent-prod.omni\n", - " queries:\n", - " broken:\n", - " file: ./broken.gq\n", - "queries:\n", - " clean:\n", - " file: ./clean.gq\n", - "policy: {}\n", - ), - ); - // Positional URI = the real loaded graph; selection is anonymous, so the - // CLEAN top-level registry validates (not prod's broken one). - let output = output_success( - cli() - .arg("queries") - .arg("validate") - .arg(graph.path()) - .arg("--config") - .arg(&config), - ); - let stdout = stdout_string(&output); - assert!( - stdout.contains("OK"), - "positional URI must validate the top-level registry, not the cli.graph default; stdout:\n{stdout}" - ); -} - -// ---- per-operator local config (omnigraph.yaml) vs the cluster surfaces ---- - -/// Cluster ops resolve operator identity per-operator: --as wins, and -/// without it the cwd omnigraph.yaml's `cli.actor` is the default. -#[test] -fn cluster_apply_uses_cli_actor_from_local_config() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - fs::write( - temp.path().join("omnigraph.yaml"), - "cli:\n actor: act-local\n", - ) - .unwrap(); - // Phase 1: import once (setup, not under test). - let output = cli() - .current_dir(temp.path()) - .arg("cluster") - .arg("import") - .arg("--config") - .arg(temp.path()) - .output() - .unwrap(); - assert!(output.status.success(), "{output:?}"); - - // Phase 2: apply alone, capturing the echoed actor (idempotent re-runs). - let apply = |extra: &[&str]| { - let mut command = cli(); - command.current_dir(temp.path()); - for arg in extra { - command.arg(arg); - } - let output = command - .arg("cluster") - .arg("apply") - .arg("--config") - .arg(temp.path()) - .arg("--json") - .output() - .unwrap(); - let json: serde_json::Value = - serde_json::from_str(String::from_utf8_lossy(&output.stdout).trim()).unwrap(); - json["actor"].clone() - }; - assert_eq!(apply(&[]), "act-local", "cli.actor is the no-flag default"); - assert_eq!(apply(&["--as", "andrew"]), "andrew", "--as overrides cli.actor"); -} - -#[test] -fn cluster_approve_uses_cli_actor_fallback() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - fs::write( - temp.path().join("omnigraph.yaml"), - "cli:\n actor: act-local\n", - ) - .unwrap(); - // Converge, then remove the graph so a gated delete is pending. - for command in ["import", "apply"] { - let output = cli() - .current_dir(temp.path()) - .arg("cluster") - .arg(command) - .arg("--config") - .arg(temp.path()) - .output() - .unwrap(); - assert!(output.status.success(), "cluster {command} failed"); - } - fs::write(temp.path().join("cluster.yaml"), "version: 1\ngraphs: {}\n").unwrap(); - - let output = cli() - .current_dir(temp.path()) - .arg("cluster") - .arg("approve") - .arg("graph.knowledge") - .arg("--config") - .arg(temp.path()) - .arg("--json") - .output() - .unwrap(); - assert!(output.status.success(), "{output:?}"); - let json: serde_json::Value = - serde_json::from_str(String::from_utf8_lossy(&output.stdout).trim()).unwrap(); - assert_eq!(json["approved_by"], "act-local"); - - // With neither flag nor config: refused with the actionable message. - let bare = tempdir().unwrap(); - write_cluster_config_fixture(bare.path()); - let output = output_failure( - cli() - .current_dir(bare.path()) - .arg("cluster") - .arg("approve") - .arg("graph.knowledge") - .arg("--config") - .arg(bare.path()), - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!(stderr.contains("--as"), "{stderr}"); - assert!(stderr.contains("cli.actor"), "{stderr}"); -} - -/// A malformed omnigraph.yaml in the cwd must never break cluster commands; -/// it is read for exactly one thing (the actor default when --as is absent), -/// and only that path fails loudly. -#[test] -fn cluster_commands_ignore_malformed_local_config() { - let temp = tempdir().unwrap(); - write_cluster_config_fixture(temp.path()); - fs::write(temp.path().join("omnigraph.yaml"), "{{{{ not yaml").unwrap(); - - for command in ["validate", "plan", "status"] { - let output = cli() - .current_dir(temp.path()) - .arg("cluster") - .arg(command) - .arg("--config") - .arg(temp.path()) - .arg("--json") - .output() - .unwrap(); - assert!( - output.status.success() || command == "plan", // plan warns state-missing pre-import; still must not config-error - "cluster {command} affected by malformed omnigraph.yaml: {output:?}" - ); - assert!( - !String::from_utf8_lossy(&output.stderr).contains("omnigraph.yaml"), - "cluster {command} touched omnigraph.yaml" - ); - } - // import + apply with an explicit --as: the config is never loaded. - for (command, args) in [("import", vec![]), ("apply", vec!["--as", "andrew"])] { - let mut invocation = cli(); - invocation.current_dir(temp.path()); - for arg in &args { - invocation.arg(arg); - } - let output = invocation - .arg("cluster") - .arg(command) - .arg("--config") - .arg(temp.path()) - .output() - .unwrap(); - assert!( - output.status.success(), - "cluster {command} affected by malformed omnigraph.yaml: {}", - String::from_utf8_lossy(&output.stderr) - ); - } - // Only the no-flag actor lookup is allowed to fail, and loudly. - let output = output_failure( - cli() - .current_dir(temp.path()) - .arg("cluster") - .arg("apply") - .arg("--config") - .arg(temp.path()), - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - stderr.contains("omnigraph.yaml") && stderr.contains("--as"), - "the actor-default config read must fail loudly and actionably: {stderr}" - ); -} - -/// A well-formed omnigraph.yaml with a CONFLICTING world view (different -/// graphs, server bind) leaks nothing into cluster outputs. -#[test] -fn cluster_commands_ignore_conflicting_local_config() { - let baseline = tempdir().unwrap(); - write_cluster_config_fixture(baseline.path()); - let with_config = tempdir().unwrap(); - write_cluster_config_fixture(with_config.path()); - fs::write( - with_config.path().join("omnigraph.yaml"), - r#" -server: - bind: 0.0.0.0:9999 -graphs: - phantom: - uri: ./phantom.omni -"#, - ) - .unwrap(); - - let validate = |dir: &std::path::Path| { - let output = cli() - .current_dir(dir) - .arg("cluster") - .arg("validate") - .arg("--config") - .arg(dir) - .arg("--json") - .output() - .unwrap(); - assert!(output.status.success(), "{output:?}"); - serde_json::from_str::<serde_json::Value>(String::from_utf8_lossy(&output.stdout).trim()) - .unwrap() - }; - let (a, b) = (validate(baseline.path()), validate(with_config.path())); - // Compare the path-free invariants (paths embed each tempdir). - for key in ["ok", "diagnostics", "resource_digests", "dependencies"] { - assert_eq!(a[key], b[key], "conflicting omnigraph.yaml leaked into cluster validate ({key})"); - } - let leaked = b.to_string(); - assert!(!leaked.contains("phantom") && !leaked.contains("9999"), "{leaked}"); -} diff --git a/crates/omnigraph-cli/tests/cli_cluster.rs b/crates/omnigraph-cli/tests/cli_cluster.rs new file mode 100644 index 0000000..be7675a --- /dev/null +++ b/crates/omnigraph-cli/tests/cli_cluster.rs @@ -0,0 +1,884 @@ +//! Cluster command surface: validate/plan/apply/approve/status/sync/force-unlock. +//! Moved verbatim from tests/cli.rs in the modularization. + +use std::fs; + +use tempfile::tempdir; + +mod support; + +use support::*; + + +#[test] +fn cluster_validate_config_success() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let output = output_success( + cli() + .arg("cluster") + .arg("validate") + .arg("--config") + .arg(temp.path()), + ); + let stdout = stdout_string(&output); + assert!(stdout.contains("cluster config valid"), "{stdout}"); +} + +#[test] +fn cluster_validate_json_is_stable() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("validate") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert!(json["resource_digests"]["graph.knowledge"].is_string()); + assert!(json["resource_digests"]["query.knowledge.find_person"].is_string()); + assert_eq!(json["dependencies"][0]["from"], "policy.base"); + assert_eq!(json["dependencies"][0]["to"], "graph.knowledge"); +} + +#[test] +fn cluster_plan_json_reads_inferred_local_state() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#" +{ + "version": 1, + "applied_revision": { + "config_digest": "old", + "resources": { + "graph.knowledge": { "digest": "old-graph" }, + "policy.old": { "digest": "old-policy" } + } + } +} +"#, + ) + .unwrap(); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("plan") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["state_observations"]["state_found"], true); + assert!( + json["changes"] + .as_array() + .unwrap() + .iter() + .any(|change| change["resource"] == "policy.old" && change["operation"] == "delete"), + "plan should read state and delete stale resources: {json}" + ); +} + +#[test] +fn cluster_status_json_reports_missing_state() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("status") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["state_observations"]["state_found"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_missing"), + "missing state should be a warning diagnostic: {json}" + ); +} + +#[test] +fn cluster_status_json_reports_lock_metadata() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_lock(temp.path(), "held-lock", "refresh"); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("status") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["state_observations"]["locked"], true); + assert_eq!(json["state_observations"]["lock_id"], "held-lock"); + assert_eq!(json["state_observations"]["lock_operation"], "refresh"); + assert_eq!(json["state_observations"]["lock_pid"], 123); + assert_eq!( + json["state_observations"]["lock_created_at"], + "1970-01-01T00:00:00Z" + ); + assert!(json["state_observations"]["lock_age_seconds"].is_number()); +} + +#[test] +fn cluster_status_json_reports_extended_state() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#" +{ + "version": 1, + "state_revision": 5, + "applied_revision": { + "config_digest": "applied", + "resources": { + "graph.knowledge": { "digest": "graph-digest" } + } + }, + "resource_statuses": { + "graph.knowledge": { "status": "applied", "conditions": ["healthy"] } + }, + "approval_records": {}, + "recovery_records": {}, + "observations": {} +} +"#, + ) + .unwrap(); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("status") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["state_observations"]["state_revision"], 5); + assert!( + json["state_observations"]["state_cas"] + .as_str() + .unwrap() + .starts_with("sha256:") + ); + assert_eq!(json["resource_digests"]["graph.knowledge"], "graph-digest"); + assert_eq!( + json["resource_statuses"]["graph.knowledge"]["status"], + "applied" + ); +} + +#[test] +fn cluster_plan_json_includes_state_cas_revision_and_lock_observation() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#" +{ + "version": 1, + "state_revision": 9, + "applied_revision": { + "config_digest": "old", + "resources": { + "graph.knowledge": { "digest": "old-graph" } + } + } +} +"#, + ) + .unwrap(); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("plan") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["state_observations"]["state_revision"], 9); + assert!( + json["state_observations"]["state_cas"] + .as_str() + .unwrap() + .starts_with("sha256:") + ); + assert_eq!(json["state_observations"]["locked"], false); + assert_eq!(json["state_observations"]["lock_acquired"], true); + assert!(json["state_observations"]["acquired_lock_id"].is_string()); + assert!(!state_dir.join("lock.json").exists()); +} + +#[test] +fn cluster_plan_locked_state_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_lock(temp.path(), "held-lock", "plan"); + + let output = output_failure( + cli() + .arg("cluster") + .arg("plan") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + ); + let json = parse_stdout_json(&output); + assert_eq!(json["ok"], false); + assert_eq!(json["state_observations"]["locked"], true); + assert_eq!(json["state_observations"]["lock_acquired"], false); + assert_eq!(json["state_observations"]["lock_id"], "held-lock"); + assert_eq!(json["state_observations"]["lock_operation"], "plan"); + assert_eq!(json["state_observations"]["lock_pid"], 123); + assert_eq!( + json["state_observations"]["lock_created_at"], + "1970-01-01T00:00:00Z" + ); + assert!(json["state_observations"]["lock_age_seconds"].is_number()); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_lock_held" + && diagnostic["message"] + .as_str() + .unwrap() + .contains("force-unlock held-lock")), + "locked state should produce a useful diagnostic: {json}" + ); +} + +#[test] +fn cluster_force_unlock_json_removes_lock() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_lock(temp.path(), "held-lock", "plan"); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("force-unlock") + .arg("held-lock") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["lock_removed"], true); + assert_eq!(json["state_observations"]["lock_id"], "held-lock"); + assert_eq!(json["state_observations"]["lock_operation"], "plan"); + assert!(!temp.path().join("__cluster/lock.json").exists()); +} + +#[test] +fn cluster_force_unlock_wrong_id_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_lock(temp.path(), "held-lock", "plan"); + + let json = parse_stdout_json(&output_failure( + cli() + .arg("cluster") + .arg("force-unlock") + .arg("other-lock") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], false); + assert_eq!(json["lock_removed"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_lock_id_mismatch") + ); + assert!(temp.path().join("__cluster/lock.json").exists()); +} + +#[test] +fn cluster_locked_plan_then_force_unlock_then_plan_succeeds() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_lock(temp.path(), "held-lock", "plan"); + + let locked = parse_stdout_json(&output_failure( + cli() + .arg("cluster") + .arg("plan") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(locked["ok"], false); + assert_eq!(locked["state_observations"]["lock_id"], "held-lock"); + + let unlocked = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("force-unlock") + .arg("held-lock") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(unlocked["lock_removed"], true); + + let planned = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("plan") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(planned["ok"], true); +} + +#[test] +fn cluster_import_json_bootstraps_missing_state() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("import") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["operation"], "import"); + assert_eq!(json["state_observations"]["state_revision"], 1); + assert!( + json["state_observations"]["state_cas"] + .as_str() + .unwrap() + .starts_with("sha256:") + ); + assert_eq!(json["state_observations"]["locked"], false); + assert_eq!(json["state_observations"]["lock_acquired"], true); + assert!(json["state_observations"]["acquired_lock_id"].is_string()); + assert!(json["observations"]["graph.knowledge"]["manifest_version"].is_number()); + assert_eq!( + json["resource_statuses"]["graph.knowledge"]["status"], + "applied" + ); + assert!(temp.path().join("__cluster/state.json").exists()); + assert!(!temp.path().join("__cluster/lock.json").exists()); +} + +#[test] +fn cluster_refresh_json_updates_revision_cas_and_removes_lock() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#" +{ + "version": 1, + "state_revision": 2, + "applied_revision": { "resources": {} } +} +"#, + ) + .unwrap(); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("refresh") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true); + assert_eq!(json["operation"], "refresh"); + assert_eq!(json["state_observations"]["state_revision"], 3); + assert!( + json["state_observations"]["state_cas"] + .as_str() + .unwrap() + .starts_with("sha256:") + ); + assert_eq!(json["state_observations"]["locked"], false); + assert_eq!(json["state_observations"]["lock_acquired"], true); + assert!(json["state_observations"]["acquired_lock_id"].is_string()); + assert!(!state_dir.join("lock.json").exists()); +} + +#[test] +fn cluster_refresh_missing_state_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let output = output_failure( + cli() + .arg("cluster") + .arg("refresh") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + ); + let json = parse_stdout_json(&output); + assert_eq!(json["ok"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_missing"), + "missing state should produce a useful diagnostic: {json}" + ); +} + +#[test] +fn cluster_import_existing_state_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + + let output = output_failure( + cli() + .arg("cluster") + .arg("import") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + ); + let json = parse_stdout_json(&output); + assert_eq!(json["ok"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_already_exists"), + "existing state should produce a useful diagnostic: {json}" + ); +} + +#[test] +fn cluster_refresh_and_import_locked_state_exit_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + r#"{"version":1,"applied_revision":{"resources":{}}}"#, + ) + .unwrap(); + fs::write( + state_dir.join("lock.json"), + r#"{"version":1,"lock_id":"held-lock","operation":"refresh","created_at":"2026-06-08T00:00:00Z","pid":123}"#, + ) + .unwrap(); + + let refresh = parse_stdout_json(&output_failure( + cli() + .arg("cluster") + .arg("refresh") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(refresh["state_observations"]["locked"], true); + assert_eq!(refresh["state_observations"]["lock_id"], "held-lock"); + assert_eq!(refresh["state_observations"]["lock_acquired"], false); + assert!( + refresh["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_lock_held") + ); + + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let state_dir = temp.path().join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("lock.json"), + r#"{"version":1,"lock_id":"held-lock","operation":"import","created_at":"2026-06-08T00:00:00Z","pid":123}"#, + ) + .unwrap(); + + let imported = parse_stdout_json(&output_failure( + cli() + .arg("cluster") + .arg("import") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(imported["state_observations"]["locked"], true); + assert_eq!(imported["state_observations"]["lock_id"], "held-lock"); + assert_eq!(imported["state_observations"]["lock_acquired"], false); + assert!( + imported["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_lock_held") + ); +} + +#[test] +fn cluster_validate_invalid_config_exits_nonzero() { + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("cluster.yaml"), + "version: 1\ngraphs: {}\npipelines: {}\n", + ) + .unwrap(); + + let output = output_failure( + cli() + .arg("cluster") + .arg("validate") + .arg("--config") + .arg(temp.path()), + ); + let stdout = stdout_string(&output); + assert!(stdout.contains("future_phase_field"), "{stdout}"); +} + +#[test] +fn cluster_apply_json_applies_query_and_policy() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let validate = write_cluster_applyable_state(temp.path()); + + let json = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(json["ok"], true, "{json}"); + assert_eq!(json["applied_count"], 2, "{json}"); + assert_eq!(json["converged"], true, "{json}"); + assert_eq!(json["state_written"], true, "{json}"); + assert_eq!( + json["resource_statuses"]["query.knowledge.find_person"]["status"], + "applied" + ); + + let query_digest = validate["resource_digests"]["query.knowledge.find_person"] + .as_str() + .unwrap(); + let payload = temp + .path() + .join("__cluster/resources/query/knowledge/find_person") + .join(format!("{query_digest}.gq")); + assert!(payload.exists(), "missing payload {}", payload.display()); + + let state: serde_json::Value = serde_json::from_str( + &fs::read_to_string(temp.path().join("__cluster/state.json")).unwrap(), + ) + .unwrap(); + assert_eq!(state["state_revision"], 2); + assert_eq!( + state["applied_revision"]["resources"]["query.knowledge.find_person"]["digest"], + *query_digest + ); +} + +#[test] +fn cluster_apply_missing_state_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let output = output_failure( + cli() + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + ); + let json = parse_stdout_json(&output); + assert_eq!(json["ok"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_missing"), + "{json}" + ); + assert!(!temp.path().join("__cluster/resources").exists()); +} + +#[test] +fn cluster_apply_locked_exits_nonzero() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_applyable_state(temp.path()); + write_cluster_lock(temp.path(), "held-lock", "plan"); + + let output = output_failure( + cli() + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + ); + let json = parse_stdout_json(&output); + assert_eq!(json["ok"], false); + assert!( + json["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "state_lock_held"), + "{json}" + ); + assert!(temp.path().join("__cluster/lock.json").exists()); + assert!(!temp.path().join("__cluster/resources").exists()); +} + +#[test] +fn cluster_apply_uses_cli_actor_from_local_config() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + fs::write( + temp.path().join("omnigraph.yaml"), + "cli:\n actor: act-local\n", + ) + .unwrap(); + // Phase 1: import once (setup, not under test). + let output = cli() + .current_dir(temp.path()) + .arg("cluster") + .arg("import") + .arg("--config") + .arg(temp.path()) + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + + // Phase 2: apply alone, capturing the echoed actor (idempotent re-runs). + let apply = |extra: &[&str]| { + let mut command = cli(); + command.current_dir(temp.path()); + for arg in extra { + command.arg(arg); + } + let output = command + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json") + .output() + .unwrap(); + let json: serde_json::Value = + serde_json::from_str(String::from_utf8_lossy(&output.stdout).trim()).unwrap(); + json["actor"].clone() + }; + assert_eq!(apply(&[]), "act-local", "cli.actor is the no-flag default"); + assert_eq!(apply(&["--as", "andrew"]), "andrew", "--as overrides cli.actor"); +} + +#[test] +fn cluster_approve_uses_cli_actor_fallback() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + fs::write( + temp.path().join("omnigraph.yaml"), + "cli:\n actor: act-local\n", + ) + .unwrap(); + // Converge, then remove the graph so a gated delete is pending. + for command in ["import", "apply"] { + let output = cli() + .current_dir(temp.path()) + .arg("cluster") + .arg(command) + .arg("--config") + .arg(temp.path()) + .output() + .unwrap(); + assert!(output.status.success(), "cluster {command} failed"); + } + fs::write(temp.path().join("cluster.yaml"), "version: 1\ngraphs: {}\n").unwrap(); + + let output = cli() + .current_dir(temp.path()) + .arg("cluster") + .arg("approve") + .arg("graph.knowledge") + .arg("--config") + .arg(temp.path()) + .arg("--json") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + let json: serde_json::Value = + serde_json::from_str(String::from_utf8_lossy(&output.stdout).trim()).unwrap(); + assert_eq!(json["approved_by"], "act-local"); + + // With neither flag nor config: refused with the actionable message. + let bare = tempdir().unwrap(); + write_cluster_config_fixture(bare.path()); + let output = output_failure( + cli() + .current_dir(bare.path()) + .arg("cluster") + .arg("approve") + .arg("graph.knowledge") + .arg("--config") + .arg(bare.path()), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stderr.contains("--as"), "{stderr}"); + assert!(stderr.contains("cli.actor"), "{stderr}"); +} + +#[test] +fn cluster_commands_ignore_malformed_local_config() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + fs::write(temp.path().join("omnigraph.yaml"), "{{{{ not yaml").unwrap(); + + for command in ["validate", "plan", "status"] { + let output = cli() + .current_dir(temp.path()) + .arg("cluster") + .arg(command) + .arg("--config") + .arg(temp.path()) + .arg("--json") + .output() + .unwrap(); + assert!( + output.status.success() || command == "plan", // plan warns state-missing pre-import; still must not config-error + "cluster {command} affected by malformed omnigraph.yaml: {output:?}" + ); + assert!( + !String::from_utf8_lossy(&output.stderr).contains("omnigraph.yaml"), + "cluster {command} touched omnigraph.yaml" + ); + } + // import + apply with an explicit --as: the config is never loaded. + for (command, args) in [("import", vec![]), ("apply", vec!["--as", "andrew"])] { + let mut invocation = cli(); + invocation.current_dir(temp.path()); + for arg in &args { + invocation.arg(arg); + } + let output = invocation + .arg("cluster") + .arg(command) + .arg("--config") + .arg(temp.path()) + .output() + .unwrap(); + assert!( + output.status.success(), + "cluster {command} affected by malformed omnigraph.yaml: {}", + String::from_utf8_lossy(&output.stderr) + ); + } + // Only the no-flag actor lookup is allowed to fail, and loudly. + let output = output_failure( + cli() + .current_dir(temp.path()) + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("omnigraph.yaml") && stderr.contains("--as"), + "the actor-default config read must fail loudly and actionably: {stderr}" + ); +} + +#[test] +fn cluster_commands_ignore_conflicting_local_config() { + let baseline = tempdir().unwrap(); + write_cluster_config_fixture(baseline.path()); + let with_config = tempdir().unwrap(); + write_cluster_config_fixture(with_config.path()); + fs::write( + with_config.path().join("omnigraph.yaml"), + r#" +server: + bind: 0.0.0.0:9999 +graphs: + phantom: + uri: ./phantom.omni +"#, + ) + .unwrap(); + + let validate = |dir: &std::path::Path| { + let output = cli() + .current_dir(dir) + .arg("cluster") + .arg("validate") + .arg("--config") + .arg(dir) + .arg("--json") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + serde_json::from_str::<serde_json::Value>(String::from_utf8_lossy(&output.stdout).trim()) + .unwrap() + }; + let (a, b) = (validate(baseline.path()), validate(with_config.path())); + // Compare the path-free invariants (paths embed each tempdir). + for key in ["ok", "diagnostics", "resource_digests", "dependencies"] { + assert_eq!(a[key], b[key], "conflicting omnigraph.yaml leaked into cluster validate ({key})"); + } + let leaked = b.to_string(); + assert!(!leaked.contains("phantom") && !leaked.contains("9999"), "{leaked}"); +} diff --git a/crates/omnigraph-cli/tests/cli_cluster_e2e.rs b/crates/omnigraph-cli/tests/cli_cluster_e2e.rs new file mode 100644 index 0000000..36b476a --- /dev/null +++ b/crates/omnigraph-cli/tests/cli_cluster_e2e.rs @@ -0,0 +1,621 @@ +//! Cluster lifecycle compositions over the spawned binary (recovery, drift, convergence). +//! Moved verbatim from tests/cli.rs in the modularization. + +use std::fs; + +use tempfile::tempdir; + +mod support; + +use support::*; + + +#[test] +fn cluster_e2e_lifecycle_import_apply_status_refresh_converges() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + assert_eq!(import["state_observations"]["state_revision"], 1); + + let plan = cluster_json(temp.path(), "plan"); + let changes = plan["changes"].as_array().unwrap(); + assert_eq!(changes.len(), 3, "{plan}"); + let disposition_of = |resource: &str| { + changes + .iter() + .find(|change| change["resource"] == resource) + .unwrap_or_else(|| panic!("missing change for {resource}: {plan}"))["disposition"] + .clone() + }; + assert_eq!(disposition_of("graph.knowledge"), "derived"); + assert_eq!(disposition_of("query.knowledge.find_person"), "applied"); + assert_eq!(disposition_of("policy.base"), "applied"); + + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["ok"], true, "{apply}"); + assert_eq!(apply["applied_count"], 2, "{apply}"); + assert_eq!(apply["converged"], true, "{apply}"); + + let status = cluster_json(temp.path(), "status"); + assert_eq!( + status["resource_statuses"]["query.knowledge.find_person"]["status"], + "applied" + ); + assert_eq!(status["resource_statuses"]["policy.base"]["status"], "applied"); + assert!( + status["state_observations"]["applied_config_digest"].is_string(), + "converged apply must record the applied config digest: {status}" + ); + + // Refresh re-observes the live graph; it must not undo apply's work. + let refresh = cluster_json(temp.path(), "refresh"); + assert_eq!(refresh["ok"], true, "{refresh}"); + let replan = cluster_json(temp.path(), "plan"); + assert!( + replan["changes"].as_array().unwrap().is_empty(), + "refresh after a converged apply must not re-open the plan: {replan}" + ); + + // A query edit round-trips: plan update -> apply -> converged again. + fs::write( + temp.path().join("people.gq"), + r#" +query find_person($name: String) { + match { $p: Person { name: $name } } + return { $p.name } +} +"#, + ) + .unwrap(); + let apply_edit = cluster_json(temp.path(), "apply"); + assert_eq!(apply_edit["applied_count"], 1, "{apply_edit}"); + assert_eq!(apply_edit["converged"], true, "{apply_edit}"); + + let final_apply = cluster_json(temp.path(), "apply"); + assert_eq!(final_apply["state_written"], false, "{final_apply}"); + assert!(final_apply["changes"].as_array().unwrap().is_empty()); +} + +#[test] +fn cluster_e2e_schema_change_applied_by_cluster() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["converged"], true, "{apply}"); + + // Additive schema change: Stage 4B applies it from the cluster — no + // manual schema apply, no refresh round-trip. + fs::write( + temp.path().join("people.pg"), + r#" +node Person { + name: String @key + age: I32? + bio: String? +} +"#, + ) + .unwrap(); + + // Plan previews the real migration steps (RFC-004 §D7). + let plan = cluster_json(temp.path(), "plan"); + let schema_change = change_for(&plan, "schema.knowledge"); + assert_eq!(schema_change["disposition"], "applied", "{plan}"); + let migration = &schema_change["migration"]; + assert_eq!(migration["supported"], true, "{plan}"); + assert!( + migration["steps"] + .as_array() + .unwrap() + .iter() + .any(|step| step["kind"] == "add_property"), + "{plan}" + ); + + let evolve = cluster_json(temp.path(), "apply"); + assert_eq!(evolve["ok"], true, "{evolve}"); + assert_eq!(evolve["converged"], true, "{evolve}"); + assert_eq!(change_for(&evolve, "schema.knowledge")["disposition"], "applied"); + + // The live graph carries the new schema; the plan is empty. + let schema_show = output_success( + cli() + .arg("schema") + .arg("show") + .arg(temp.path().join("graphs/knowledge.omni")), + ); + assert!(stdout_string(&schema_show).contains("bio"), "live schema updated"); + let replan = cluster_json(temp.path(), "plan"); + assert!( + replan["changes"].as_array().unwrap().is_empty(), + "one cluster apply converges a schema change: {replan}" + ); +} + +#[test] +fn cluster_e2e_force_unlock_unblocks_apply() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + write_cluster_applyable_state(temp.path()); + write_cluster_lock(temp.path(), "stuck-lock", "apply"); + + let refused = parse_stdout_json(&output_failure( + cli() + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(refused["ok"], false); + + let unlocked = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("force-unlock") + .arg("stuck-lock") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(unlocked["lock_removed"], true, "{unlocked}"); + + let retried = cluster_json(temp.path(), "apply"); + assert_eq!(retried["ok"], true, "{retried}"); + assert_eq!(retried["converged"], true, "{retried}"); +} + +#[test] +fn cluster_e2e_lost_state_reimport_recovers_catalog() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["converged"], true, "{apply}"); + + let query_digest = change_for(&apply, "query.knowledge.find_person")["after_digest"] + .as_str() + .unwrap() + .to_string(); + let blob = temp + .path() + .join("__cluster/resources/query/knowledge/find_person") + .join(format!("{query_digest}.gq")); + let blob_content = fs::read_to_string(&blob).unwrap(); + + // Disaster: the state ledger is lost. + fs::remove_file(temp.path().join("__cluster/state.json")).unwrap(); + + let reimport = cluster_json(temp.path(), "import"); + assert_eq!(reimport["ok"], true, "{reimport}"); + assert_eq!(reimport["state_observations"]["state_revision"], 1); + // Import observes graph/schema only; query/policy digests are not invented. + assert!( + reimport["resource_digests"] + .get("query.knowledge.find_person") + .is_none(), + "{reimport}" + ); + + let plan = cluster_json(temp.path(), "plan"); + assert_eq!( + change_for(&plan, "query.knowledge.find_person")["disposition"], + "applied" + ); + assert_eq!(change_for(&plan, "policy.base")["disposition"], "applied"); + + let reapply = cluster_json(temp.path(), "apply"); + assert_eq!(reapply["ok"], true, "{reapply}"); + assert_eq!(reapply["converged"], true, "{reapply}"); + assert!( + reapply["state_observations"]["applied_config_digest"].is_string(), + "{reapply}" + ); + // The catalog blob was reused, not rewritten with different content. + assert_eq!(fs::read_to_string(&blob).unwrap(), blob_content); + + let replan = cluster_json(temp.path(), "plan"); + assert!(replan["changes"].as_array().unwrap().is_empty(), "{replan}"); +} + +#[test] +fn cluster_e2e_out_of_band_schema_drift_then_apply_converges_it() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["converged"], true, "{apply}"); + + // Out-of-band: the live graph evolves, cluster.yaml stays put. + fs::write( + temp.path().join("people_v2.pg"), + r#" +node Person { + name: String @key + age: I32? + bio: String? +} +"#, + ) + .unwrap(); + output_success( + cli() + .arg("schema") + .arg("apply") + .arg(temp.path().join("graphs/knowledge.omni")) + .arg("--schema") + .arg(temp.path().join("people_v2.pg")) + .arg("--json"), + ); + + // Drift is visible... + let refresh = cluster_json(temp.path(), "refresh"); + assert_eq!( + refresh["resource_statuses"]["schema.knowledge"]["status"], + "drifted" + ); + // ...the plan proposes converging back to desired, with a migration + // preview (a soft drop of the out-of-band field)... + let plan = cluster_json(temp.path(), "plan"); + let schema_change = change_for(&plan, "schema.knowledge"); + assert_eq!(schema_change["disposition"], "applied", "{plan}"); + assert!( + schema_change["migration"]["steps"] + .as_array() + .unwrap() + .iter() + .any(|step| step["kind"] == "drop_property" && step["mode"] == "soft"), + "{plan}" + ); + // ...and apply converges the live schema back (axiom 8: drift correction + // is gated like any change; a soft migration is the recoverable tier). + let converge = cluster_json(temp.path(), "apply"); + assert_eq!(converge["ok"], true, "{converge}"); + assert_eq!(converge["converged"], true, "{converge}"); + let schema_show = output_success( + cli() + .arg("schema") + .arg("show") + .arg(temp.path().join("graphs/knowledge.omni")), + ); + assert!( + !stdout_string(&schema_show).contains("bio"), + "out-of-band field soft-dropped back to desired" + ); + let replan = cluster_json(temp.path(), "plan"); + assert!(replan["changes"].as_array().unwrap().is_empty(), "{replan}"); +} + +#[test] +fn cluster_e2e_graph_root_destruction_drifts_then_apply_recreates_empty_graph() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["converged"], true, "{apply}"); + let query_digest = change_for(&apply, "query.knowledge.find_person")["after_digest"] + .as_str() + .unwrap() + .to_string(); + + fs::remove_dir_all(temp.path().join("graphs/knowledge.omni")).unwrap(); + + // Missing root is drift, not an error. + let refresh = cluster_json(temp.path(), "refresh"); + assert_eq!(refresh["ok"], true, "{refresh}"); + assert_eq!( + refresh["resource_statuses"]["graph.knowledge"]["status"], + "drifted" + ); + assert!( + refresh["resource_statuses"]["graph.knowledge"]["conditions"] + .as_array() + .unwrap() + .iter() + .any(|condition| condition == "graph_missing"), + "{refresh}" + ); + // Graph/schema digests removed; query/policy digests preserved. + assert!(refresh["resource_digests"].get("graph.knowledge").is_none()); + assert!(refresh["resource_digests"].get("schema.knowledge").is_none()); + assert!( + refresh["resource_digests"] + .get("query.knowledge.find_person") + .is_some(), + "{refresh}" + ); + + let plan = cluster_json(temp.path(), "plan"); + assert_eq!(change_for(&plan, "graph.knowledge")["operation"], "create"); + // Stage 4A: the re-create is executable and the plan says so — nothing + // hidden about converging a destroyed root back to an EMPTY graph (the + // data was already lost; this is declarative convergence, RFC-004 §D1). + assert_eq!(change_for(&plan, "graph.knowledge")["disposition"], "applied"); + assert_eq!(change_for(&plan, "schema.knowledge")["disposition"], "applied"); + // Converged-then-destroyed: query/policy are already in state at the + // desired digests, so they are not changes at all. + assert_eq!(plan["changes"].as_array().unwrap().len(), 2, "{plan}"); + + let recreate = cluster_json(temp.path(), "apply"); + assert_eq!(recreate["ok"], true, "{recreate}"); + assert_eq!(recreate["converged"], true, "{recreate}"); + // The empty graph is back on disk; catalog state survived throughout. + assert!(temp.path().join("graphs/knowledge.omni").exists()); + let state: serde_json::Value = serde_json::from_str( + &fs::read_to_string(temp.path().join("__cluster/state.json")).unwrap(), + ) + .unwrap(); + assert_eq!( + state["applied_revision"]["resources"]["query.knowledge.find_person"]["digest"], + query_digest + ); + assert!( + temp.path() + .join("__cluster/resources/query/knowledge/find_person") + .join(format!("{query_digest}.gq")) + .exists() + ); +} + +#[test] +fn cluster_e2e_multi_graph_mixed_dispositions_then_approve_and_converge() { + let temp = tempdir().unwrap(); + write_multi_graph_cluster_fixture(temp.path()); + // No manual init: Stage 4A creates both graphs. + + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["ok"], true, "{apply}"); + assert_eq!(apply["converged"], true, "{apply}"); + assert_eq!(change_for(&apply, "graph.knowledge")["disposition"], "applied"); + assert_eq!( + change_for(&apply, "graph.engineering")["disposition"], + "applied" + ); + assert_eq!( + change_for(&apply, "query.engineering.find_service")["disposition"], + "applied" + ); + // The graph-spanning and cluster-scoped policies ride the same run. + assert_eq!(change_for(&apply, "policy.shared")["disposition"], "applied"); + assert_eq!( + change_for(&apply, "policy.cluster_wide")["disposition"], + "applied" + ); + assert!(temp.path().join("graphs/knowledge.omni").exists()); + assert!(temp.path().join("graphs/engineering.omni").exists()); + + // Mixed run: a graph REMOVAL (4C territory — deferred) gates its query + // delete (blocked), while a knowledge query update is independent + // (applied) and re-derives its composite. All four dispositions at once. + fs::write( + temp.path().join("cluster.yaml"), + r#" +version: 1 +metadata: + name: company-brain +state: + backend: cluster + lock: true +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +policies: + shared: + file: ./shared.policy.yaml + applies_to: [knowledge] + cluster_wide: + file: ./cluster_wide.policy.yaml + applies_to: [cluster] +"#, + ) + .unwrap(); + fs::write( + temp.path().join("people.gq"), + "\nquery find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + + let mixed = cluster_json(temp.path(), "apply"); + assert_eq!(mixed["ok"], true, "{mixed}"); + assert_eq!(mixed["converged"], false, "{mixed}"); + // Stage 4C: deletes are gated on a digest-bound approval, one gate per + // subtree (the graph-level approval carries schema + queries). + assert_eq!( + change_for(&mixed, "graph.engineering")["disposition"], + "blocked" + ); + assert_eq!( + change_for(&mixed, "graph.engineering")["reason"], + "approval_required" + ); + assert_eq!( + change_for(&mixed, "schema.engineering")["reason"], + "approval_required" + ); + assert_eq!( + change_for(&mixed, "query.engineering.find_service")["reason"], + "approval_required" + ); + let gate_plan = cluster_json(temp.path(), "plan"); + let gates = gate_plan["approvals_required"].as_array().unwrap(); + assert_eq!(gates.len(), 1, "{gate_plan}"); + assert_eq!(gates[0]["resource"], "graph.engineering"); + assert_eq!(gates[0]["satisfied"], false); + assert_eq!( + change_for(&mixed, "query.knowledge.find_person")["disposition"], + "applied" + ); + // 5A: policy.shared's applies_to narrowed with an unchanged file digest + // — now a first-class binding change, applied in the same run. + assert_eq!(change_for(&mixed, "policy.shared")["binding_change"], true); + assert_eq!(change_for(&mixed, "policy.shared")["disposition"], "applied"); + assert_eq!( + change_for(&mixed, "graph.knowledge")["disposition"], + "derived" + ); + // Deterministic ordering: changes sorted by resource address. + let order: Vec<&str> = mixed["changes"] + .as_array() + .unwrap() + .iter() + .map(|change| change["resource"].as_str().unwrap()) + .collect(); + let mut sorted = order.clone(); + sorted.sort_unstable(); + assert_eq!(order, sorted, "{mixed}"); + // The conclusion: an apply without approval stays blocked; the approved + // delete converges the cluster, tombstoning the removed graph. + let still_blocked = cluster_json(temp.path(), "apply"); + assert_eq!(still_blocked["converged"], false, "{still_blocked}"); + + let approve = parse_stdout_json(&output_success( + cli() + .arg("--as") + .arg("andrew") + .arg("cluster") + .arg("approve") + .arg("graph.engineering") + .arg("--config") + .arg(temp.path()) + .arg("--json"), + )); + assert_eq!(approve["ok"], true, "{approve}"); + assert_eq!(approve["approved_by"], "andrew"); + + let converge = cluster_json(temp.path(), "apply"); + assert_eq!(converge["ok"], true, "{converge}"); + assert_eq!(converge["converged"], true, "{converge}"); + assert!(!temp.path().join("graphs/engineering.omni").exists()); + + let status = cluster_json(temp.path(), "status"); + assert_eq!(status["observations"]["graph.engineering"]["kind"], "tombstone"); + let final_plan = cluster_json(temp.path(), "plan"); + assert!( + final_plan["changes"].as_array().unwrap().is_empty(), + "{final_plan}" + ); +} + +#[test] +fn cluster_e2e_approve_requires_actor() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let output = output_failure( + cli() + .arg("cluster") + .arg("approve") + .arg("graph.knowledge") + .arg("--config") + .arg(temp.path()), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stderr.contains("--as"), "{stderr}"); +} + +#[test] +fn cluster_e2e_declared_graph_created_by_apply() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["ok"], true, "{apply}"); + assert_eq!(apply["converged"], true, "{apply}"); + assert_eq!(change_for(&apply, "graph.knowledge")["disposition"], "applied"); + assert!(temp.path().join("graphs/knowledge.omni").exists()); + + // The created graph is a real graph: the per-graph CLI can open it. + let snapshot = output_success( + cli() + .arg("snapshot") + .arg(temp.path().join("graphs/knowledge.omni")), + ); + assert!(!stdout_string(&snapshot).is_empty()); + + let plan = cluster_json(temp.path(), "plan"); + assert!(plan["changes"].as_array().unwrap().is_empty(), "{plan}"); + let status = cluster_json(temp.path(), "status"); + assert_eq!( + status["resource_statuses"]["graph.knowledge"]["status"], + "applied" + ); +} + +#[test] +fn cluster_e2e_payload_drift_self_heals() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["converged"], true, "{apply}"); + + let query_digest = change_for(&apply, "query.knowledge.find_person")["after_digest"] + .as_str() + .unwrap() + .to_string(); + let blob = temp + .path() + .join("__cluster/resources/query/knowledge/find_person") + .join(format!("{query_digest}.gq")); + fs::remove_file(&blob).unwrap(); + + let status = cluster_json(temp.path(), "status"); + assert_eq!(status["ok"], true, "{status}"); + assert!( + status["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "catalog_payload_missing"), + "{status}" + ); + + let refresh = cluster_json(temp.path(), "refresh"); + assert_eq!(refresh["ok"], true, "{refresh}"); + assert_eq!( + refresh["resource_statuses"]["query.knowledge.find_person"]["status"], + "drifted" + ); + + let heal = cluster_json(temp.path(), "apply"); + assert_eq!(heal["ok"], true, "{heal}"); + assert_eq!(heal["converged"], true, "{heal}"); + assert!(blob.exists(), "blob republished"); + + let clean = cluster_json(temp.path(), "status"); + assert!( + !clean["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| { + diagnostic["code"] + .as_str() + .is_some_and(|code| code.starts_with("catalog_payload")) + }), + "{clean}" + ); +} diff --git a/crates/omnigraph-cli/tests/cli_data.rs b/crates/omnigraph-cli/tests/cli_data.rs new file mode 100644 index 0000000..841bedf --- /dev/null +++ b/crates/omnigraph-cli/tests/cli_data.rs @@ -0,0 +1,1631 @@ +//! Data commands: load/read/change/branch/commit/export/snapshot/policy/embed/maintenance. +//! Moved verbatim from tests/cli.rs in the modularization. + +use std::fs; + +use serde_json::Value; +use tempfile::tempdir; + +mod support; + +use support::*; + + +#[test] +fn short_version_flag_prints_current_cli_version() { + let output = output_success(cli().arg("-v")); + let stdout = stdout_string(&output); + + assert_eq!( + stdout.trim(), + format!("omnigraph {}", env!("CARGO_PKG_VERSION")) + ); +} + +#[test] +fn long_version_flag_prints_current_cli_version() { + let output = output_success(cli().arg("--version")); + let stdout = stdout_string(&output); + + assert_eq!( + stdout.trim(), + format!("omnigraph {}", env!("CARGO_PKG_VERSION")) + ); +} + +#[test] +fn embed_seed_fills_missing_and_preserves_existing_vectors_by_default() { + let temp = tempdir().unwrap(); + let seed = write_seed_fixture(temp.path()); + + let output = output_success( + cli() + .env("OMNIGRAPH_EMBEDDINGS_MOCK", "1") + .arg("embed") + .arg("--seed") + .arg(&seed) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["mode"], "fill_missing"); + assert_eq!(payload["embedded_rows"], 1); + assert_eq!(payload["selected_rows"], 2); + + let embedded = read_embedded_rows(temp.path().join("build/seed.embedded.jsonl")); + assert_eq!( + embedded[0]["data"]["embedding"].as_array().unwrap().len(), + 4 + ); + assert_eq!( + embedded[1]["data"]["embedding"], + serde_json::json!([0.1, 0.2]) + ); +} + +#[test] +fn embed_clean_removes_selected_embeddings() { + let temp = tempdir().unwrap(); + let seed = write_seed_fixture(temp.path()); + + let output = output_success( + cli() + .arg("embed") + .arg("--seed") + .arg(&seed) + .arg("--clean") + .arg("--select") + .arg("Decision:slug=dec-beta") + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["mode"], "clean"); + assert_eq!(payload["cleaned_rows"], 1); + + let embedded = read_embedded_rows(temp.path().join("build/seed.embedded.jsonl")); + assert!(embedded[0]["data"].get("embedding").is_none()); + assert!(embedded[1]["data"].get("embedding").is_none()); +} + +#[test] +fn embed_select_reembeds_only_matching_rows() { + let temp = tempdir().unwrap(); + let seed = write_seed_fixture(temp.path()); + + let output = output_success( + cli() + .env("OMNIGRAPH_EMBEDDINGS_MOCK", "1") + .arg("embed") + .arg("--seed") + .arg(&seed) + .arg("--select") + .arg("Decision:slug=dec-beta") + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["mode"], "reembed_selected"); + assert_eq!(payload["embedded_rows"], 1); + assert_eq!(payload["selected_rows"], 1); + + let embedded = read_embedded_rows(temp.path().join("build/seed.embedded.jsonl")); + assert!(embedded[0]["data"].get("embedding").is_none()); + assert_ne!( + embedded[1]["data"]["embedding"], + serde_json::json!([0.1, 0.2]) + ); + assert_eq!( + embedded[1]["data"]["embedding"].as_array().unwrap().len(), + 4 + ); +} + +#[test] +fn embed_seed_preserves_non_entity_rows() { + let temp = tempdir().unwrap(); + let seed = write_seed_fixture_with_edge(temp.path()); + + let output = output_success( + cli() + .env("OMNIGRAPH_EMBEDDINGS_MOCK", "1") + .arg("embed") + .arg("--seed") + .arg(&seed) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["rows"], 3); + assert_eq!(payload["embedded_rows"], 1); + + let embedded = read_embedded_rows(temp.path().join("build/seed.embedded.jsonl")); + assert_eq!(embedded.len(), 3); + assert_eq!(embedded[2]["edge"], "Triggered"); + assert_eq!(embedded[2]["from"], "sig-alpha"); + assert_eq!(embedded[2]["to"], "dec-alpha"); +} + +#[test] +fn repair_json_reports_noop_on_clean_graph() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + let output = output_success(cli().arg("repair").arg("--json").arg(&graph)); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["confirm"], false); + assert_eq!(payload["force"], false); + assert_eq!(payload["manifest_version"], Value::Null); + let tables = payload["tables"].as_array().unwrap(); + assert_eq!(tables.len(), 4); + assert!(tables.iter().all(|table| { + table["classification"] == "no_drift" && table["action"] == "no_op" + })); +} + +#[test] +fn repair_confirm_json_refuses_suspicious_drift_with_nonzero_exit_then_force_succeeds() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + let graph_manifest_before = manifest_dataset_version(&graph); + let (table_manifest_before, table_head_before) = forge_person_delete_drift(&graph); + + let refused = output_failure( + cli() + .arg("repair") + .arg("--confirm") + .arg("--json") + .arg(&graph), + ); + let refused_payload: Value = serde_json::from_slice(&refused.stdout).unwrap(); + assert_eq!(refused_payload["manifest_version"], Value::Null); + let person = refused_payload["tables"] + .as_array() + .unwrap() + .iter() + .find(|table| table["table_key"] == "node:Person") + .unwrap(); + assert_eq!(person["classification"], "suspicious"); + assert_eq!(person["action"], "refused"); + assert!( + String::from_utf8_lossy(&refused.stderr).contains("repair refused"), + "stderr should explain the non-zero exit; got: {}", + String::from_utf8_lossy(&refused.stderr) + ); + assert_eq!(manifest_dataset_version(&graph), graph_manifest_before); + + let forced = output_success( + cli() + .arg("repair") + .arg("--force") + .arg("--confirm") + .arg("--json") + .arg(&graph), + ); + let forced_payload: Value = serde_json::from_slice(&forced.stdout).unwrap(); + let forced_manifest = forced_payload["manifest_version"].as_u64().unwrap(); + assert!(forced_manifest > graph_manifest_before); + let person = forced_payload["tables"] + .as_array() + .unwrap() + .iter() + .find(|table| table["table_key"] == "node:Person") + .unwrap(); + assert_eq!(person["classification"], "suspicious"); + assert_eq!(person["action"], "forced"); + assert_eq!(person["manifest_version"], table_manifest_before); + assert_eq!(person["lance_head_version"], table_head_before); + assert_eq!(manifest_dataset_version(&graph), forced_manifest); +} + +#[test] +fn query_lint_json_with_schema_reports_warnings() { + let temp = tempdir().unwrap(); + let schema_path = temp.path().join("schema.pg"); + let query_path = temp.path().join("queries.gq"); + write_file( + &schema_path, + r#" +node Policy { + slug: String @key + name: String? + effectiveTo: DateTime? +} +"#, + ); + write_query_file( + &query_path, + r#" +query update_policy($slug: String, $name: String) { + update Policy set { name: $name } where slug = $slug +} +"#, + ); + + let output = output_success( + cli() + .arg("query") + .arg("lint") + .arg("--query") + .arg(&query_path) + .arg("--schema") + .arg(&schema_path) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["status"], "ok"); + assert_eq!(payload["schema_source"]["kind"], "file"); + assert_eq!(payload["queries_processed"], 1); + assert_eq!(payload["warnings"], 1); + assert_eq!(payload["findings"][0]["code"], "L201"); + assert_eq!( + payload["findings"][0]["message"], + "Policy.effectiveTo exists in schema but no update query sets it" + ); +} + +#[test] +fn lint_top_level_matches_deprecated_query_lint_output() { + let temp = tempdir().unwrap(); + let schema_path = temp.path().join("schema.pg"); + let query_path = temp.path().join("queries.gq"); + write_file( + &schema_path, + r#" +node Person { + name: String +} +"#, + ); + write_query_file( + &query_path, + r#" +query list_people() { + match { $p: Person } + return { $p.name } +} +"#, + ); + + let canonical = output_success( + cli() + .arg("lint") + .arg("--query") + .arg(&query_path) + .arg("--schema") + .arg(&schema_path) + .arg("--json"), + ); + let deprecated_lint = output_success( + cli() + .arg("query") + .arg("lint") + .arg("--query") + .arg(&query_path) + .arg("--schema") + .arg(&schema_path) + .arg("--json"), + ); + let deprecated_check = output_success( + cli() + .arg("query") + .arg("check") + .arg("--query") + .arg(&query_path) + .arg("--schema") + .arg(&schema_path) + .arg("--json"), + ); + + assert_eq!(stdout_string(&canonical), stdout_string(&deprecated_lint)); + assert_eq!(stdout_string(&canonical), stdout_string(&deprecated_check)); + + // Canonical form must NOT emit the deprecation warning. + let canonical_stderr = String::from_utf8(canonical.stderr).unwrap(); + assert!( + !canonical_stderr.contains("deprecated"), + "`omnigraph lint` is canonical and must not warn; got stderr: {canonical_stderr}" + ); + + // Deprecated forms MUST emit the one-line warning, pointing at the + // new top-level `omnigraph lint`. + let lint_stderr = String::from_utf8(deprecated_lint.stderr).unwrap(); + assert!( + lint_stderr.contains("`omnigraph query lint` is deprecated") + && lint_stderr.contains("`omnigraph lint`"), + "expected deprecation warning pointing at `omnigraph lint`; got: {lint_stderr}" + ); + let check_stderr = String::from_utf8(deprecated_check.stderr).unwrap(); + assert!( + check_stderr.contains("`omnigraph query check` is deprecated") + && check_stderr.contains("`omnigraph lint`"), + "expected deprecation warning pointing at `omnigraph lint`; got: {check_stderr}" + ); +} + +#[test] +fn deprecated_check_top_level_rewrites_to_lint() { + let temp = tempdir().unwrap(); + let schema_path = temp.path().join("schema.pg"); + let query_path = temp.path().join("queries.gq"); + write_file( + &schema_path, + r#" +node Person { + name: String +} +"#, + ); + write_query_file( + &query_path, + r#" +query list_people() { + match { $p: Person } + return { $p.name } +} +"#, + ); + + let canonical = output_success( + cli() + .arg("lint") + .arg("--query") + .arg(&query_path) + .arg("--schema") + .arg(&schema_path) + .arg("--json"), + ); + let deprecated_check = output_success( + cli() + .arg("check") + .arg("--query") + .arg(&query_path) + .arg("--schema") + .arg(&schema_path) + .arg("--json"), + ); + + assert_eq!(stdout_string(&canonical), stdout_string(&deprecated_check)); + + let check_stderr = String::from_utf8(deprecated_check.stderr).unwrap(); + assert!( + check_stderr.contains("`omnigraph check` is deprecated") + && check_stderr.contains("`omnigraph lint`"), + "expected `omnigraph check` deprecation warning pointing at `omnigraph lint`; got: {check_stderr}" + ); + + // `check` must NOT appear in the canonical `omnigraph --help` output — + // agents copy the surface from help text and would otherwise emit both + // names interchangeably. + let help = cli().arg("--help").output().unwrap(); + let stdout = String::from_utf8(help.stdout).unwrap(); + let check_aliased = stdout + .lines() + .any(|line| line.trim_start().starts_with("lint") && line.contains("check")); + assert!( + !check_aliased, + "`check` must not be advertised as a visible alias of `lint`; help output: {stdout}" + ); +} + +#[test] +fn deprecated_read_and_change_subcommands_emit_warnings() { + // Both subcommands require `--query`/`--query-string`/`--alias`, so + // invoking them with no args will exit non-zero. That's fine -- + // we only care that the deprecation warning is printed before the + // argument-required error. + let output = cli().arg("read").output().unwrap(); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!( + stderr.contains("`omnigraph read` is deprecated") && stderr.contains("`omnigraph query`"), + "expected `omnigraph read` deprecation warning; got: {stderr}" + ); + + let output = cli().arg("change").output().unwrap(); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!( + stderr.contains("`omnigraph change` is deprecated") + && stderr.contains("`omnigraph mutate`"), + "expected `omnigraph change` deprecation warning; got: {stderr}" + ); + + // Sanity check the inverse: the canonical names must NOT print the + // deprecation banner. + let output = cli().arg("query").arg("--help").output().unwrap(); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!( + !stderr.contains("deprecated"), + "`omnigraph query` is canonical and must not warn; got: {stderr}" + ); + let output = cli().arg("mutate").arg("--help").output().unwrap(); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!( + !stderr.contains("deprecated"), + "`omnigraph mutate` is canonical and must not warn; got: {stderr}" + ); +} + +#[test] +fn query_lint_can_use_local_graph_via_positional_uri() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let query_path = temp.path().join("queries.gq"); + init_graph(&graph); + write_query_file( + &query_path, + r#" +query list_people() { + match { $p: Person } + return { $p.name } +} +"#, + ); + + let output = output_success( + cli() + .arg("query") + .arg("lint") + .arg("--query") + .arg(&query_path) + .arg("--json") + .arg(&graph), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["status"], "ok"); + assert_eq!(payload["schema_source"]["kind"], "graph"); + assert_eq!( + payload["schema_source"]["uri"].as_str(), + Some(graph.to_string_lossy().as_ref()) + ); +} + +#[test] +fn query_lint_can_resolve_graph_and_query_from_config() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let config_path = temp.path().join("omnigraph.yaml"); + init_graph(&graph); + write_query_file( + &temp.path().join("queries.gq"), + r#" +query list_people() { + match { $p: Person } + return { $p.name } +} +"#, + ); + write_config(&config_path, &local_yaml_config(&graph)); + + let output = output_success( + cli() + .arg("query") + .arg("lint") + .arg("--query") + .arg("queries.gq") + .arg("--config") + .arg(&config_path) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["status"], "ok"); + assert_eq!(payload["schema_source"]["kind"], "graph"); + assert_eq!( + payload["schema_source"]["uri"].as_str(), + Some(graph.to_string_lossy().as_ref()) + ); +} + +#[test] +fn query_lint_rejects_http_targets_without_schema() { + let temp = tempdir().unwrap(); + let query_path = temp.path().join("queries.gq"); + write_query_file( + &query_path, + r#" +query list_people() { + match { $p: Person } + return { $p.name } +} +"#, + ); + + let output = output_failure( + cli() + .arg("query") + .arg("lint") + .arg("--query") + .arg(&query_path) + .arg("http://127.0.0.1:8080"), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("query lint is only supported against local graph URIs in this milestone") + ); +} + +#[test] +fn query_lint_requires_schema_or_resolvable_graph_target() { + let temp = tempdir().unwrap(); + let query_path = temp.path().join("queries.gq"); + write_query_file( + &query_path, + r#" +query list_people() { + match { $p: Person } + return { $p.name } +} +"#, + ); + + let output = output_failure( + cli() + .arg("query") + .arg("lint") + .arg("--query") + .arg(&query_path), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("query lint requires --schema <schema.pg> or a resolvable graph target") + ); +} + +#[test] +fn query_lint_human_output_reports_warnings() { + let temp = tempdir().unwrap(); + let schema_path = temp.path().join("schema.pg"); + let query_path = temp.path().join("queries.gq"); + write_file( + &schema_path, + r#" +node Policy { + slug: String @key + name: String? + effectiveTo: DateTime? +} +"#, + ); + write_query_file( + &query_path, + r#" +query update_policy($slug: String, $name: String) { + update Policy set { name: $name } where slug = $slug +} +"#, + ); + + let output = output_success( + cli() + .arg("query") + .arg("lint") + .arg("--query") + .arg(&query_path) + .arg("--schema") + .arg(&schema_path), + ); + let stdout = stdout_string(&output); + + assert!(stdout.contains("OK query `update_policy` (mutation)")); + assert!( + stdout.contains("WARN Policy.effectiveTo exists in schema but no update query sets it") + ); + assert!(stdout.contains( + "INFO Lint complete: 1 queries processed (0 error(s), 1 warning(s), 0 info item(s))" + )); +} + +#[test] +fn query_lint_human_output_reports_strict_validation_errors() { + let temp = tempdir().unwrap(); + let schema_path = temp.path().join("schema.pg"); + let query_path = temp.path().join("queries.gq"); + write_file( + &schema_path, + r#" +node Policy { + slug: String @key + name: String? +} +"#, + ); + write_query_file( + &query_path, + r#" +query bad_update($slug: String) { + update Policy set { priority_level: "high" } where slug = $slug +} +"#, + ); + + let output = output_failure( + cli() + .arg("query") + .arg("lint") + .arg("--query") + .arg(&query_path) + .arg("--schema") + .arg(&schema_path), + ); + let stdout = stdout_string(&output); + + assert!(stdout.contains("ERROR query `bad_update`:")); + assert!(stdout.contains("Policy")); + assert!(stdout.contains( + "INFO Lint complete: 1 queries processed (1 error(s), 0 warning(s), 0 info item(s))" + )); +} + +#[test] +fn load_json_outputs_summary_for_main_branch() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + let data = fixture("test.jsonl"); + + let output = output_success( + cli() + .arg("load") + .arg("--mode") + .arg("overwrite") + .arg("--data") + .arg(&data) + .arg("--json") + .arg(&graph), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["branch"], "main"); + assert_eq!(payload["mode"], "overwrite"); + assert_eq!(payload["nodes_loaded"], 6); + assert_eq!(payload["edges_loaded"], 5); + assert_eq!(payload["node_types_loaded"], 2); + assert_eq!(payload["edge_types_loaded"], 2); +} + +#[test] +fn load_into_feature_branch_with_merge_mode_succeeds() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + output_success( + cli() + .arg("branch") + .arg("create") + .arg("--uri") + .arg(&graph) + .arg("--from") + .arg("main") + .arg("feature"), + ); + + let feature_data = temp.path().join("feature.jsonl"); + write_jsonl( + &feature_data, + r#"{"type":"Person","data":{"name":"Alice","age":31}}"#, + ); + + let output = output_success( + cli() + .arg("load") + .arg("--data") + .arg(&feature_data) + .arg("--branch") + .arg("feature") + .arg("--mode") + .arg("merge") + .arg(&graph), + ); + let stdout = stdout_string(&output); + + assert!(stdout.contains("branch feature")); + assert!(stdout.contains("with merge")); + assert!(stdout.contains("1 nodes across 1 node types")); +} + +#[test] +fn read_json_outputs_rows_for_named_query() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + let queries = fixture("test.gq"); + + let output = output_success( + cli() + .arg("read") + .arg(&graph) + .arg("--query") + .arg(&queries) + .arg("--name") + .arg("get_person") + .arg("--params") + .arg(r#"{"name":"Alice"}"#) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["query_name"], "get_person"); + assert_eq!(payload["target"]["branch"], "main"); + assert_eq!(payload["row_count"], 1); + assert_eq!(payload["rows"][0]["p.name"], "Alice"); +} + +#[test] +fn export_jsonl_outputs_source_rows_for_selected_branch_and_type() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + output_success( + cli() + .arg("branch") + .arg("create") + .arg("--uri") + .arg(&graph) + .arg("--from") + .arg("main") + .arg("feature"), + ); + + let feature_data = temp.path().join("feature-export.jsonl"); + write_jsonl( + &feature_data, + r#"{"type":"Person","data":{"name":"Eve","age":29}}"#, + ); + output_success( + cli() + .arg("load") + .arg("--data") + .arg(&feature_data) + .arg("--branch") + .arg("feature") + .arg("--mode") + .arg("append") + .arg(&graph), + ); + + let output = output_success( + cli() + .arg("export") + .arg(&graph) + .arg("--branch") + .arg("feature") + .arg("--type") + .arg("Person") + .arg("--jsonl"), + ); + let rows = stdout_string(&output) + .lines() + .map(|line| serde_json::from_str::<Value>(line).unwrap()) + .collect::<Vec<_>>(); + + assert_eq!(rows.len(), 5); + assert!(rows.iter().all(|row| row["type"] == "Person")); + assert!(rows.iter().all(|row| row.get("edge").is_none())); + assert!( + rows.iter() + .any(|row| row["data"]["name"].as_str() == Some("Eve")) + ); +} + +#[test] +fn policy_validate_accepts_valid_policy_file() { + let temp = tempdir().unwrap(); + let (config, _) = write_policy_config_fixture(temp.path()); + + let output = output_success( + cli() + .arg("policy") + .arg("validate") + .arg("--config") + .arg(&config), + ); + let stdout = stdout_string(&output); + + assert!(stdout.contains("policy valid:")); + assert!(stdout.contains("policy.yaml")); + assert!(stdout.contains("[2 actors]")); +} + +#[test] +fn policy_validate_fails_for_invalid_policy_file() { + let temp = tempdir().unwrap(); + let config = temp.path().join("omnigraph.yaml"); + let policy = temp.path().join("policy.yaml"); + fs::write( + &config, + r#" +project: + name: policy-test-graph +policy: + file: ./policy.yaml +"#, + ) + .unwrap(); + fs::write( + &policy, + r#" +version: 1 +groups: + team: [act-andrew] +rules: + - id: duplicate + allow: + actors: { group: team } + actions: [read] + branch_scope: any + - id: duplicate + allow: + actors: { group: team } + actions: [export] + branch_scope: any +"#, + ) + .unwrap(); + + let output = output_failure( + cli() + .arg("policy") + .arg("validate") + .arg("--config") + .arg(&config), + ); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!(stderr.contains("duplicate policy rule id")); +} + +#[test] +fn policy_test_runs_declarative_cases() { + let temp = tempdir().unwrap(); + let (config, _) = write_policy_config_fixture(temp.path()); + + let output = output_success(cli().arg("policy").arg("test").arg("--config").arg(&config)); + let stdout = stdout_string(&output); + + assert!(stdout.contains("policy tests passed: 2 cases")); +} + +#[test] +fn policy_explain_reports_decision_and_matched_rule() { + let temp = tempdir().unwrap(); + let (config, _) = write_policy_config_fixture(temp.path()); + + let allow = output_success( + cli() + .arg("policy") + .arg("explain") + .arg("--config") + .arg(&config) + .arg("--actor") + .arg("act-andrew") + .arg("--action") + .arg("change") + .arg("--branch") + .arg("feature"), + ); + let allow_stdout = stdout_string(&allow); + assert!(allow_stdout.contains("decision: allow")); + assert!(allow_stdout.contains("matched_rule: team-write")); + + let deny = output_success( + cli() + .arg("policy") + .arg("explain") + .arg("--config") + .arg(&config) + .arg("--actor") + .arg("act-bruno") + .arg("--action") + .arg("change") + .arg("--branch") + .arg("main"), + ); + let deny_stdout = stdout_string(&deny); + assert!(deny_stdout.contains("decision: deny")); + assert!(deny_stdout.contains("message: policy denied action 'change' on branch 'main'")); +} + +#[test] +fn read_can_resolve_uri_from_config() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let config = temp.path().join("omnigraph.yaml"); + init_graph(&graph); + load_fixture(&graph); + write_config(&config, &local_yaml_config(&graph)); + + let output = output_success( + cli() + .arg("read") + .arg("--config") + .arg(&config) + .arg("--query") + .arg(fixture("test.gq")) + .arg("--name") + .arg("get_person") + .arg("--params") + .arg(r#"{"name":"Alice"}"#) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["row_count"], 1); +} + +#[test] +fn read_csv_format_outputs_header_and_row_values() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + let output = output_success( + cli() + .arg("read") + .arg(&graph) + .arg("--query") + .arg(fixture("test.gq")) + .arg("--name") + .arg("get_person") + .arg("--params") + .arg(r#"{"name":"Alice"}"#) + .arg("--format") + .arg("csv"), + ); + let stdout = stdout_string(&output); + + assert!(stdout.lines().next().unwrap().contains("p.name")); + assert!(stdout.contains("Alice")); +} + +#[test] +fn read_jsonl_format_outputs_metadata_header_first() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + let output = output_success( + cli() + .arg("read") + .arg(&graph) + .arg("--query") + .arg(fixture("test.gq")) + .arg("--name") + .arg("get_person") + .arg("--params") + .arg(r#"{"name":"Alice"}"#) + .arg("--format") + .arg("jsonl"), + ); + let stdout = stdout_string(&output); + let mut lines = stdout.lines(); + assert!(lines.next().unwrap().contains("\"kind\":\"metadata\"")); + assert!(lines.next().unwrap().contains("\"p.name\":\"Alice\"")); +} + +#[test] +fn change_json_outputs_affected_counts_and_persists() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + let mutation_file = temp.path().join("mutations.gq"); + write_query_file( + &mutation_file, + r#" +query insert_person($name: String, $age: I32) { + insert Person { name: $name, age: $age } +} +"#, + ); + + let output = output_success( + cli() + .arg("change") + .arg(&graph) + .arg("--query") + .arg(&mutation_file) + .arg("--params") + .arg(r#"{"name":"Eve","age":29}"#) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["branch"], "main"); + assert_eq!(payload["query_name"], "insert_person"); + assert_eq!(payload["affected_nodes"], 1); + assert_eq!(payload["affected_edges"], 0); + + let verify = output_success( + cli() + .arg("read") + .arg(&graph) + .arg("--query") + .arg(fixture("test.gq")) + .arg("--name") + .arg("get_person") + .arg("--params") + .arg(r#"{"name":"Eve"}"#) + .arg("--json"), + ); + let verify_payload: Value = serde_json::from_slice(&verify.stdout).unwrap(); + assert_eq!(verify_payload["row_count"], 1); + assert_eq!(verify_payload["rows"][0]["p.name"], "Eve"); +} + +#[test] +fn change_can_resolve_uri_and_branch_from_config() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let config = temp.path().join("omnigraph.yaml"); + init_graph(&graph); + load_fixture(&graph); + write_config(&config, &local_yaml_config(&graph)); + let mutation_file = temp.path().join("config-mutations.gq"); + write_query_file( + &mutation_file, + r#" +query insert_person($name: String, $age: I32) { + insert Person { name: $name, age: $age } +} +"#, + ); + + let output = output_success( + cli() + .arg("change") + .arg("--config") + .arg(&config) + .arg("--query") + .arg(&mutation_file) + .arg("--params") + .arg(r#"{"name":"Mia","age":30}"#) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["branch"], "main"); + assert_eq!(payload["affected_nodes"], 1); +} + +#[test] +fn read_requires_name_for_multi_query_files() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + let output = output_failure( + cli() + .arg("read") + .arg(&graph) + .arg("--query") + .arg(fixture("test.gq")), + ); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!(stderr.contains("multiple queries")); +} + +#[test] +fn read_supports_inline_query_string() { + let temp = tempdir().unwrap(); + let repo = graph_path(temp.path()); + init_graph(&repo); + load_fixture(&repo); + + let output = output_success( + cli() + .arg("read") + .arg(&repo) + .arg("-e") + .arg("query find($name: String) { match { $p: Person { name: $name } } return { $p.name, $p.age } }") + .arg("--params") + .arg(r#"{"name":"Alice"}"#) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["query_name"], "find"); + assert_eq!(payload["row_count"], 1); + assert_eq!(payload["rows"][0]["p.name"], "Alice"); +} + +#[test] +fn change_supports_inline_query_string() { + let temp = tempdir().unwrap(); + let repo = graph_path(temp.path()); + init_graph(&repo); + load_fixture(&repo); + + let output = output_success( + cli() + .arg("change") + .arg(&repo) + .arg("--query-string") + .arg("query add($name: String, $age: I32) { insert Person { name: $name, age: $age } }") + .arg("--params") + .arg(r#"{"name":"Inline","age":42}"#) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["query_name"], "add"); + assert_eq!(payload["affected_nodes"], 1); + + let verify = output_success( + cli() + .arg("read") + .arg(&repo) + .arg("-e") + .arg("query find($name: String) { match { $p: Person { name: $name } } return { $p.name } }") + .arg("--params") + .arg(r#"{"name":"Inline"}"#) + .arg("--json"), + ); + let verify_payload: Value = serde_json::from_slice(&verify.stdout).unwrap(); + assert_eq!(verify_payload["row_count"], 1); +} + +#[test] +fn read_rejects_query_string_combined_with_query() { + let temp = tempdir().unwrap(); + let repo = graph_path(temp.path()); + init_graph(&repo); + load_fixture(&repo); + + let output = output_failure( + cli() + .arg("read") + .arg(&repo) + .arg("--query") + .arg(fixture("test.gq")) + .arg("-e") + .arg("query whatever() { match { $p: Person } return { $p.name } }"), + ); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!( + stderr.contains("cannot be used") || stderr.contains("conflict"), + "expected clap conflict error, got: {stderr}" + ); +} + +#[test] +fn read_rejects_empty_query_string() { + let temp = tempdir().unwrap(); + let repo = graph_path(temp.path()); + init_graph(&repo); + load_fixture(&repo); + + let output = output_failure(cli().arg("read").arg(&repo).arg("-e").arg("")); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!( + stderr.contains("must not be empty"), + "expected empty-string rejection, got: {stderr}" + ); +} + +#[test] +fn branch_create_json_outputs_source_and_name() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + + let output = output_success( + cli() + .arg("branch") + .arg("create") + .arg("--uri") + .arg(&graph) + .arg("--from") + .arg("main") + .arg("feature") + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["from"], "main"); + assert_eq!(payload["name"], "feature"); + assert_eq!(payload["uri"], graph.to_string_lossy().as_ref()); +} + +#[test] +fn branch_list_outputs_sorted_branches() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + + output_success( + cli() + .arg("branch") + .arg("create") + .arg("--uri") + .arg(&graph) + .arg("--from") + .arg("main") + .arg("zeta"), + ); + output_success( + cli() + .arg("branch") + .arg("create") + .arg("--uri") + .arg(&graph) + .arg("--from") + .arg("main") + .arg("alpha"), + ); + + let output = output_success(cli().arg("branch").arg("list").arg("--uri").arg(&graph)); + let stdout = stdout_string(&output); + let lines = stdout + .lines() + .map(str::trim) + .filter(|line| !line.is_empty()) + .collect::<Vec<_>>(); + + assert_eq!(lines, vec!["alpha", "main", "zeta"]); +} + +#[test] +fn branch_delete_json_outputs_name_and_removes_branch() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + + output_success( + cli() + .arg("branch") + .arg("create") + .arg("--uri") + .arg(&graph) + .arg("--from") + .arg("main") + .arg("feature"), + ); + + let output = output_success( + cli() + .arg("branch") + .arg("delete") + .arg("--uri") + .arg(&graph) + .arg("feature") + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["name"], "feature"); + assert_eq!(payload["uri"], graph.to_string_lossy().as_ref()); + + let listed = output_success(cli().arg("branch").arg("list").arg("--uri").arg(&graph)); + let stdout = stdout_string(&listed); + let lines = stdout + .lines() + .map(str::trim) + .filter(|line| !line.is_empty()) + .collect::<Vec<_>>(); + assert_eq!(lines, vec!["main"]); +} + +#[test] +fn branch_delete_rejects_main() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + + let output = output_failure( + cli() + .arg("branch") + .arg("delete") + .arg("--uri") + .arg(&graph) + .arg("main"), + ); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!(stderr.contains("cannot delete branch 'main'")); +} + +#[test] +fn branch_merge_defaults_target_to_main() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + output_success( + cli() + .arg("branch") + .arg("create") + .arg("--uri") + .arg(&graph) + .arg("--from") + .arg("main") + .arg("feature"), + ); + + let feature_data = temp.path().join("feature.jsonl"); + write_jsonl( + &feature_data, + r#"{"type":"Person","data":{"name":"Eve","age":29}}"#, + ); + output_success( + cli() + .arg("load") + .arg("--data") + .arg(&feature_data) + .arg("--branch") + .arg("feature") + .arg("--mode") + .arg("append") + .arg(&graph), + ); + + let merge_output = output_success( + cli() + .arg("branch") + .arg("merge") + .arg("--uri") + .arg(&graph) + .arg("feature") + .arg("--json"), + ); + let merge_payload: Value = serde_json::from_slice(&merge_output.stdout).unwrap(); + assert_eq!(merge_payload["source"], "feature"); + assert_eq!(merge_payload["target"], "main"); + assert_eq!(merge_payload["outcome"], "fast_forward"); + + let snapshot_output = output_success( + cli() + .arg("snapshot") + .arg(&graph) + .arg("--branch") + .arg("main") + .arg("--json"), + ); + let snapshot: Value = serde_json::from_slice(&snapshot_output.stdout).unwrap(); + let person_row_count = snapshot["tables"] + .as_array() + .unwrap() + .iter() + .find(|table| table["table_key"] == "node:Person") + .unwrap()["row_count"] + .as_u64() + .unwrap(); + assert_eq!(person_row_count, 5); +} + +#[test] +fn branch_merge_supports_explicit_target() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + output_success( + cli() + .arg("branch") + .arg("create") + .arg("--uri") + .arg(&graph) + .arg("--from") + .arg("main") + .arg("feature"), + ); + output_success( + cli() + .arg("branch") + .arg("create") + .arg("--uri") + .arg(&graph) + .arg("--from") + .arg("main") + .arg("experiment"), + ); + + let feature_data = temp.path().join("feature-explicit.jsonl"); + write_jsonl( + &feature_data, + r#"{"type":"Person","data":{"name":"Frank","age":41}}"#, + ); + output_success( + cli() + .arg("load") + .arg("--data") + .arg(&feature_data) + .arg("--branch") + .arg("feature") + .arg("--mode") + .arg("append") + .arg(&graph), + ); + + let merge_output = output_success( + cli() + .arg("branch") + .arg("merge") + .arg("--uri") + .arg(&graph) + .arg("feature") + .arg("--into") + .arg("experiment") + .arg("--json"), + ); + let merge_payload: Value = serde_json::from_slice(&merge_output.stdout).unwrap(); + assert_eq!(merge_payload["target"], "experiment"); + assert_eq!(merge_payload["outcome"], "fast_forward"); +} + +#[test] +fn snapshot_json_returns_manifest_version_and_tables() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + let output = output_success(cli().arg("snapshot").arg(&graph).arg("--json")); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["branch"], "main"); + assert_eq!( + payload["manifest_version"].as_u64().unwrap(), + manifest_dataset_version(&graph) + ); + assert!(payload["tables"].as_array().unwrap().len() >= 4); +} + +#[test] +fn snapshot_can_resolve_uri_from_config() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let config = temp.path().join("omnigraph.yaml"); + init_graph(&graph); + load_fixture(&graph); + write_config(&config, &local_yaml_config(&graph)); + + let output = output_success( + cli() + .arg("snapshot") + .arg("--config") + .arg(&config) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["branch"], "main"); +} + +#[test] +fn snapshot_human_output_includes_branch_and_table_summaries() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + let output = output_success(cli().arg("snapshot").arg(&graph)); + let stdout = stdout_string(&output); + + assert!(stdout.contains("branch: main")); + assert!(stdout.contains("manifest_version:")); + assert!(stdout.contains("node:Person v")); + assert!(stdout.contains("edge:Knows v")); +} + +#[test] +fn commit_show_accepts_long_uri_flag() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + let list = output_success(cli().arg("commit").arg("list").arg(&graph).arg("--json")); + let list_payload: Value = serde_json::from_slice(&list.stdout).unwrap(); + let commit_id = list_payload["commits"][0]["graph_commit_id"] + .as_str() + .unwrap() + .to_string(); + + let output = output_success( + cli() + .arg("commit") + .arg("show") + .arg("--uri") + .arg(&graph) + .arg(&commit_id) + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["graph_commit_id"], commit_id); + assert!(payload["manifest_version"].as_u64().unwrap() >= 1); +} + +#[test] +fn cli_fails_for_missing_graph() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + + let output = output_failure(cli().arg("snapshot").arg(&graph)); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!( + stderr.contains("_schema.pg") + || stderr.contains("No such file") + || stderr.contains("not found") + ); +} + +#[test] +fn cli_fails_for_missing_schema_or_data_file() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let missing_schema = temp.path().join("missing.pg"); + let missing_data = temp.path().join("missing.jsonl"); + + let init_output = output_failure( + cli() + .arg("init") + .arg("--schema") + .arg(&missing_schema) + .arg(&graph), + ); + assert!( + String::from_utf8(init_output.stderr) + .unwrap() + .contains("No such file") + ); + + init_graph(&graph); + let load_output = output_failure( + cli() + .arg("load") + .arg("--mode") + .arg("overwrite") + .arg("--data") + .arg(&missing_data) + .arg(&graph), + ); + assert!( + String::from_utf8(load_output.stderr) + .unwrap() + .contains("No such file") + ); +} + +#[test] +fn cli_fails_for_invalid_merge_requests() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + let missing_branch = output_failure( + cli() + .arg("branch") + .arg("merge") + .arg("--uri") + .arg(&graph) + .arg("missing"), + ); + let missing_branch_stderr = String::from_utf8(missing_branch.stderr).unwrap(); + assert!( + missing_branch_stderr.contains("missing") + || missing_branch_stderr.contains("head commit") + || missing_branch_stderr.contains("not found") + ); + + let same_branch = output_failure( + cli() + .arg("branch") + .arg("merge") + .arg("--uri") + .arg(&graph) + .arg("main") + .arg("--into") + .arg("main"), + ); + assert!( + String::from_utf8(same_branch.stderr) + .unwrap() + .contains("distinct source and target") + ); +} diff --git a/crates/omnigraph-cli/tests/cli_queries.rs b/crates/omnigraph-cli/tests/cli_queries.rs new file mode 100644 index 0000000..8a1e553 --- /dev/null +++ b/crates/omnigraph-cli/tests/cli_queries.rs @@ -0,0 +1,535 @@ +//! Stored-query commands and alias resolution. +//! Moved verbatim from tests/cli.rs in the modularization. + + +use serde_json::Value; +use tempfile::tempdir; + +mod support; + +use support::*; + + +#[test] +fn query_check_alias_matches_lint_output() { + let temp = tempdir().unwrap(); + let schema_path = temp.path().join("schema.pg"); + let query_path = temp.path().join("queries.gq"); + write_file( + &schema_path, + r#" +node Person { + name: String +} +"#, + ); + write_query_file( + &query_path, + r#" +query list_people() { + match { $p: Person } + return { $p.name } +} +"#, + ); + + let lint_output = output_success( + cli() + .arg("query") + .arg("lint") + .arg("--query") + .arg(&query_path) + .arg("--schema") + .arg(&schema_path) + .arg("--json"), + ); + let check_output = output_success( + cli() + .arg("query") + .arg("check") + .arg("--query") + .arg(&query_path) + .arg("--schema") + .arg(&schema_path) + .arg("--json"), + ); + + assert_eq!(stdout_string(&lint_output), stdout_string(&check_output)); +} + +#[test] +fn read_alias_from_yaml_config_runs_with_kv_output() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let config = temp.path().join("omnigraph.yaml"); + let query = temp.path().join("aliases.gq"); + init_graph(&graph); + load_fixture(&graph); + write_query_file( + &query, + &std::fs::read_to_string(fixture("test.gq")).unwrap(), + ); + write_config( + &config, + &format!( + "{}aliases:\n owner:\n command: read\n query: aliases.gq\n name: get_person\n args: [name]\n format: kv\n", + local_yaml_config(&graph) + ), + ); + + let output = output_success( + cli() + .arg("read") + .arg("--config") + .arg(&config) + .arg("--alias") + .arg("owner") + .arg("Alice"), + ); + let stdout = stdout_string(&output); + + assert!(stdout.contains("row 1")); + assert!(stdout.contains("p.name: Alice")); +} + +#[test] +fn read_alias_uses_alias_target_without_cli_default_and_accepts_url_like_arg() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let config = temp.path().join("omnigraph.yaml"); + let query = temp.path().join("aliases.gq"); + let data = temp.path().join("url-like.jsonl"); + init_graph(&graph); + write_jsonl( + &data, + r#"{"type":"Person","data":{"name":"https://example.com","age":30}}"#, + ); + output_success( + cli() + .arg("load") + .arg("--mode") + .arg("overwrite") + .arg("--data") + .arg(&data) + .arg(&graph), + ); + write_query_file( + &query, + &std::fs::read_to_string(fixture("test.gq")).unwrap(), + ); + write_config( + &config, + &format!( + "graphs:\n local:\n uri: '{}'\nquery:\n roots:\n - .\npolicy: {{}}\naliases:\n owner:\n command: read\n query: aliases.gq\n name: get_person\n args: [name]\n graph: local\n format: kv\n", + graph.to_string_lossy() + ), + ); + + let output = output_success( + cli() + .arg("read") + .arg("--config") + .arg(&config) + .arg("--alias") + .arg("owner") + .arg("https://example.com"), + ); + let stdout = stdout_string(&output); + + assert!(stdout.contains("row 1")); + assert!(stdout.contains("p.name: https://example.com")); +} + +#[test] +fn change_alias_from_yaml_config_persists_changes() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let config = temp.path().join("omnigraph.yaml"); + let query = temp.path().join("mutations.gq"); + init_graph(&graph); + load_fixture(&graph); + write_query_file( + &query, + r#" +query insert_person($name: String, $age: I32) { + insert Person { name: $name, age: $age } +} +"#, + ); + write_config( + &config, + &format!( + "{}aliases:\n add_person:\n command: change\n query: mutations.gq\n name: insert_person\n args: [name, age]\n", + local_yaml_config(&graph) + ), + ); + + let output = output_success( + cli() + .arg("change") + .arg("--config") + .arg(&config) + .arg("--alias") + .arg("add_person") + .arg("Eve") + .arg("29") + .arg("--json"), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["affected_nodes"], 1); + + let verify = output_success( + cli() + .arg("read") + .arg(&graph) + .arg("--query") + .arg(fixture("test.gq")) + .arg("--name") + .arg("get_person") + .arg("--params") + .arg(r#"{"name":"Eve"}"#) + .arg("--json"), + ); + let verify_payload: Value = serde_json::from_slice(&verify.stdout).unwrap(); + assert_eq!(verify_payload["row_count"], 1); +} + +#[test] +fn queries_validate_exits_zero_on_clean_registry() { + let graph = SystemGraph::loaded(); + graph.write_query( + "find_person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + &queries_test_config( + &graph.path().to_string_lossy(), + "find_person", + "find_person.gq", + ), + ); + let output = output_success( + cli() + .arg("queries") + .arg("validate") + .arg("--config") + .arg(&config), + ); + let stdout = stdout_string(&output); + assert!(stdout.contains("OK"), "stdout:\n{stdout}"); +} + +#[test] +fn queries_validate_exits_nonzero_on_type_broken_query() { + let graph = SystemGraph::loaded(); + // `Widget` is not in the fixture schema. + graph.write_query( + "ghost.gq", + "query ghost() { match { $w: Widget } return { $w.name } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + &queries_test_config(&graph.path().to_string_lossy(), "ghost", "ghost.gq"), + ); + let output = output_failure( + cli() + .arg("queries") + .arg("validate") + .arg("--config") + .arg(&config), + ); + let stdout = stdout_string(&output); + assert!( + stdout.contains("ghost"), + "validation should name the broken query; stdout:\n{stdout}" + ); +} + +#[test] +fn queries_list_prints_registered_query() { + let graph = SystemGraph::loaded(); + graph.write_query( + "find_person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + // Exposed with an explicit tool name so the list shows the MCP suffix. + let config = graph.write_config( + "omnigraph.yaml", + &format!( + concat!( + "graphs:\n", + " local:\n", + " uri: '{}'\n", + " queries:\n", + " find_person:\n", + " file: ./find_person.gq\n", + " mcp: {{ expose: true, tool_name: lookup_person }}\n", + "cli:\n", + " graph: local\n", + "policy: {{}}\n", + ), + graph.path().to_string_lossy().replace('\'', "''") + ), + ); + let output = output_success( + cli() + .arg("queries") + .arg("list") + .arg("--config") + .arg(&config), + ); + let stdout = stdout_string(&output); + assert!(stdout.contains("find_person"), "stdout:\n{stdout}"); + assert!( + stdout.contains("$name: String"), + "list should show typed params; stdout:\n{stdout}" + ); + assert!( + stdout.contains("[mcp: lookup_person]"), + "list should show the MCP tool name for exposed queries; stdout:\n{stdout}" + ); +} + +#[test] +fn queries_list_requires_graph_selection_for_per_graph_only_registries() { + let graph = SystemGraph::loaded(); + graph.write_query( + "find_person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + &format!( + concat!( + "graphs:\n", + " local:\n", + " uri: '{}'\n", + " queries:\n", + " find_person:\n", + " file: ./find_person.gq\n", + "policy: {{}}\n", + ), + graph.path().to_string_lossy().replace('\'', "''") + ), + ); + + let output = output_failure( + cli() + .arg("queries") + .arg("list") + .arg("--config") + .arg(&config), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("local") && stderr.contains("--target local"), + "error must name the graph and give a concrete selection hint; stderr:\n{stderr}" + ); +} + +#[test] +fn queries_list_without_graph_selection_lists_top_level_registry() { + let graph = SystemGraph::loaded(); + graph.write_query( + "top_find.gq", + "query top_find($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + concat!( + "queries:\n", + " top_find:\n", + " file: ./top_find.gq\n", + "policy: {}\n", + ), + ); + + let output = output_success( + cli() + .arg("queries") + .arg("list") + .arg("--config") + .arg(&config), + ); + let stdout = stdout_string(&output); + assert!(stdout.contains("top_find"), "stdout:\n{stdout}"); +} + +#[test] +fn queries_list_unknown_target_errors() { + // `queries list` opens no graph URI, so unknown-graph validation can't ride + // along on URI resolution the way it does for every other command. An + // unknown `--target` must still error (naming the graph) instead of + // silently falling back to the top-level registry and showing the wrong + // (or empty) catalog. + let graph = SystemGraph::loaded(); + graph.write_query( + "find_person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + &queries_test_config( + &graph.path().to_string_lossy(), + "find_person", + "find_person.gq", + ), + ); + let output = output_failure( + cli() + .arg("queries") + .arg("list") + .arg("--target") + .arg("nonexistent") + .arg("--config") + .arg(&config), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("nonexistent"), + "error must name the unknown graph; stderr:\n{stderr}" + ); +} + +#[test] +fn queries_commands_reject_named_graph_with_populated_top_level_block() { + // A named graph (here via `cli.graph`) uses its own `graphs.<name>` block, + // so a populated top-level `queries:` block would be silently ignored — a + // config the server REFUSES to boot. `queries validate`/`list` must reject + // it too (matching boot) instead of validating/listing the per-graph block + // and giving a false green. + let graph = SystemGraph::loaded(); + graph.write_query( + "find_person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + &format!( + concat!( + "graphs:\n", + " local:\n", + " uri: '{}'\n", + " queries:\n", + " find_person:\n", + " file: ./find_person.gq\n", + "cli:\n", + " graph: local\n", + "queries:\n", // populated top-level block: the coherence violation + " legacy:\n", + " file: ./legacy.gq\n", + "policy: {{}}\n", + ), + graph.path().to_string_lossy().replace('\'', "''") + ), + ); + // Both resolve `local` from cli.graph (no positional URI), so both must + // error and name the graph + the ignored block — like server boot does. + for sub in ["validate", "list"] { + let output = output_failure(cli().arg("queries").arg(sub).arg("--config").arg(&config)); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("local") && stderr.contains("queries"), + "`queries {sub}` must reject a named graph with a populated top-level block; stderr:\n{stderr}" + ); + } +} + +#[test] +fn queries_validate_exits_nonzero_on_duplicate_tool_name() { + // Two exposed queries claiming one MCP tool name is a load-time + // collision — `queries validate` must fail (offline, before the engine + // opens) and name both queries plus the contested tool. + let graph = SystemGraph::loaded(); + graph.write_query( + "a.gq", + "query a() { match { $p: Person } return { $p.name } }", + ); + graph.write_query( + "b.gq", + "query b() { match { $p: Person } return { $p.name } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + &format!( + concat!( + "graphs:\n", + " local:\n", + " uri: '{}'\n", + " queries:\n", + " a:\n", + " file: ./a.gq\n", + " mcp: {{ expose: true, tool_name: dup }}\n", + " b:\n", + " file: ./b.gq\n", + " mcp: {{ expose: true, tool_name: dup }}\n", + "cli:\n", + " graph: local\n", + "policy: {{}}\n", + ), + graph.path().to_string_lossy().replace('\'', "''") + ), + ); + let output = output_failure( + cli() + .arg("queries") + .arg("validate") + .arg("--config") + .arg(&config), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("dup") && stderr.contains("'a'") && stderr.contains("'b'"), + "duplicate tool name should be reported naming both queries; stderr:\n{stderr}" + ); +} + +#[test] +fn queries_validate_positional_uri_ignores_default_graph() { + // A positional URI is anonymous → the schema AND the registry both come + // from top-level, even when `cli.graph` names a graph whose per-graph + // queries would fail. Pins that the URI and registry can't diverge. + let graph = SystemGraph::loaded(); + graph.write_query( + "clean.gq", + "query clean($name: String) { match { $p: Person { name: $name } } return { $p.age } }", + ); + // `Widget` is not in the fixture schema — the default graph's per-graph + // query would break validate if it were (wrongly) selected. + graph.write_query( + "broken.gq", + "query broken() { match { $w: Widget } return { $w.name } }", + ); + let config = graph.write_config( + "omnigraph.yaml", + concat!( + "cli:\n graph: prod\n", + "graphs:\n", + " prod:\n", + " uri: /nonexistent-prod.omni\n", + " queries:\n", + " broken:\n", + " file: ./broken.gq\n", + "queries:\n", + " clean:\n", + " file: ./clean.gq\n", + "policy: {}\n", + ), + ); + // Positional URI = the real loaded graph; selection is anonymous, so the + // CLEAN top-level registry validates (not prod's broken one). + let output = output_success( + cli() + .arg("queries") + .arg("validate") + .arg(graph.path()) + .arg("--config") + .arg(&config), + ); + let stdout = stdout_string(&output); + assert!( + stdout.contains("OK"), + "positional URI must validate the top-level registry, not the cli.graph default; stdout:\n{stdout}" + ); +} diff --git a/crates/omnigraph-cli/tests/cli_schema_config.rs b/crates/omnigraph-cli/tests/cli_schema_config.rs new file mode 100644 index 0000000..a0dac0a --- /dev/null +++ b/crates/omnigraph-cli/tests/cli_schema_config.rs @@ -0,0 +1,500 @@ +//! init/config scaffolding, schema plan/apply, graphs listing, version. +//! Moved verbatim from tests/cli.rs in the modularization. + +use std::fs; + +use lance::index::DatasetIndexExt; +use omnigraph::db::{Omnigraph, ReadTarget}; +use serde_json::Value; +use tempfile::tempdir; + +mod support; + +use support::*; + + +#[test] +fn version_command_prints_current_cli_version() { + let output = output_success(cli().arg("version")); + let stdout = stdout_string(&output); + + assert_eq!( + stdout.trim(), + format!("omnigraph {}", env!("CARGO_PKG_VERSION")) + ); +} + +#[test] +fn init_creates_graph_successfully_on_missing_local_directory() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema = fixture("test.pg"); + + let output = output_success(cli().arg("init").arg("--schema").arg(&schema).arg(&graph)); + let stdout = stdout_string(&output); + + assert!(stdout.contains("initialized")); + assert!(graph.join("_schema.pg").exists()); + assert!(graph.join("__manifest").exists()); + assert!(temp.path().join("omnigraph.yaml").exists()); +} + +#[test] +fn schema_plan_json_reports_supported_additive_change() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema_path = temp.path().join("next.pg"); + init_graph(&graph); + + let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace( + " age: I32?\n}", + " age: I32?\n nickname: String?\n}", + ); + fs::write(&schema_path, next_schema).unwrap(); + + let output = output_success( + cli() + .arg("schema") + .arg("plan") + .arg("--schema") + .arg(&schema_path) + .arg("--json") + .arg(&graph), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["supported"], true); + assert_eq!(payload["step_count"], 1); + assert_eq!(payload["steps"][0]["kind"], "add_property"); + assert_eq!(payload["steps"][0]["type_kind"], "node"); + assert_eq!(payload["steps"][0]["type_name"], "Person"); + assert_eq!(payload["steps"][0]["property_name"], "nickname"); +} + +#[test] +fn schema_plan_json_reports_unsupported_type_change() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema_path = temp.path().join("breaking.pg"); + init_graph(&graph); + + let breaking_schema = fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace("age: I32?", "age: I64?"); + fs::write(&schema_path, breaking_schema).unwrap(); + + let output = output_success( + cli() + .arg("schema") + .arg("plan") + .arg("--schema") + .arg(&schema_path) + .arg("--json") + .arg(&graph), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["supported"], false); + assert!(payload["steps"].as_array().unwrap().iter().any(|step| { + step["kind"] == "unsupported_change" + && step["entity"] + .as_str() + .unwrap_or_default() + .contains("Person.age") + })); +} + +#[test] +fn schema_apply_json_applies_supported_migration() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema_path = temp.path().join("next.pg"); + init_graph(&graph); + + let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace( + " age: I32?\n}", + " age: I32?\n nickname: String?\n}", + ); + fs::write(&schema_path, next_schema).unwrap(); + + let output = output_success( + cli() + .arg("schema") + .arg("apply") + .arg("--schema") + .arg(&schema_path) + .arg("--json") + .arg(&graph), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + + assert_eq!(payload["supported"], true); + assert_eq!(payload["applied"], true); + assert_eq!(payload["step_count"], 1); + + let db = tokio::runtime::Runtime::new() + .unwrap() + .block_on(Omnigraph::open(graph.to_string_lossy().as_ref())) + .unwrap(); + assert!( + db.catalog().node_types["Person"] + .properties + .contains_key("nickname") + ); +} + +#[test] +fn schema_apply_human_reports_noop() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema_path = fixture("test.pg"); + init_graph(&graph); + + let output = output_success( + cli() + .arg("schema") + .arg("apply") + .arg("--schema") + .arg(&schema_path) + .arg(&graph), + ); + let stdout = stdout_string(&output); + + assert!(stdout.contains("applied: no")); + assert!(stdout.contains("no schema changes")); +} + +#[test] +fn schema_apply_json_renames_type_and_updates_snapshot() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema_path = temp.path().join("rename.pg"); + init_graph(&graph); + + let renamed_schema = fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace("node Person {\n", "node Human @rename_from(\"Person\") {\n") + .replace("edge Knows: Person -> Person", "edge Knows: Human -> Human") + .replace( + "edge WorksAt: Person -> Company", + "edge WorksAt: Human -> Company", + ); + fs::write(&schema_path, renamed_schema).unwrap(); + + let output = output_success( + cli() + .arg("schema") + .arg("apply") + .arg("--schema") + .arg(&schema_path) + .arg("--json") + .arg(&graph), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["applied"], true); + + let db = tokio::runtime::Runtime::new() + .unwrap() + .block_on(Omnigraph::open(graph.to_string_lossy().as_ref())) + .unwrap(); + let snapshot = tokio::runtime::Runtime::new() + .unwrap() + .block_on(db.snapshot_of(ReadTarget::branch("main"))) + .unwrap(); + assert!(snapshot.entry("node:Human").is_some()); + assert!(snapshot.entry("node:Person").is_none()); +} + +#[test] +fn schema_apply_json_renames_property_and_updates_catalog() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema_path = temp.path().join("rename-property.pg"); + init_graph(&graph); + + let renamed_schema = fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace("age: I32?", "years: I32? @rename_from(\"age\")"); + fs::write(&schema_path, renamed_schema).unwrap(); + + let output = output_success( + cli() + .arg("schema") + .arg("apply") + .arg("--schema") + .arg(&schema_path) + .arg("--json") + .arg(&graph), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["applied"], true); + + let db = tokio::runtime::Runtime::new() + .unwrap() + .block_on(Omnigraph::open(graph.to_string_lossy().as_ref())) + .unwrap(); + let person = &db.catalog().node_types["Person"]; + assert!(person.properties.contains_key("years")); + assert!(!person.properties.contains_key("age")); +} + +#[test] +fn schema_apply_json_adds_index_for_existing_property() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema_path = temp.path().join("index.pg"); + init_graph(&graph); + + let before_index_count = tokio::runtime::Runtime::new().unwrap().block_on(async { + let db = Omnigraph::open(graph.to_string_lossy().as_ref()) + .await + .unwrap(); + let snapshot = db.snapshot_of(ReadTarget::branch("main")).await.unwrap(); + let dataset = snapshot.open("node:Person").await.unwrap(); + dataset.load_indices().await.unwrap().len() + }); + + let indexed_schema = fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace("name: String @key", "name: String @key @index"); + fs::write(&schema_path, indexed_schema).unwrap(); + + let output = output_success( + cli() + .arg("schema") + .arg("apply") + .arg("--schema") + .arg(&schema_path) + .arg("--json") + .arg(&graph), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["applied"], true); + + let after_index_count = tokio::runtime::Runtime::new().unwrap().block_on(async { + let db = Omnigraph::open(graph.to_string_lossy().as_ref()) + .await + .unwrap(); + let snapshot = db.snapshot_of(ReadTarget::branch("main")).await.unwrap(); + let dataset = snapshot.open("node:Person").await.unwrap(); + dataset.load_indices().await.unwrap().len() + }); + assert!(after_index_count > before_index_count); +} + +#[test] +fn schema_apply_rejects_unsupported_plan() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema_path = temp.path().join("breaking.pg"); + init_graph(&graph); + + let breaking_schema = fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace("age: I32?", "age: I64?"); + fs::write(&schema_path, breaking_schema).unwrap(); + + let output = output_failure( + cli() + .arg("schema") + .arg("apply") + .arg("--schema") + .arg(&schema_path) + .arg(&graph), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stderr.contains("changing property type")); +} + +#[test] +fn schema_apply_rejects_when_non_main_branch_exists() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema_path = temp.path().join("next.pg"); + init_graph(&graph); + output_success( + cli() + .arg("branch") + .arg("create") + .arg("--from") + .arg("main") + .arg("--uri") + .arg(&graph) + .arg("feature"), + ); + + let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace( + " age: I32?\n}", + " age: I32?\n nickname: String?\n}", + ); + fs::write(&schema_path, next_schema).unwrap(); + + let output = output_failure( + cli() + .arg("schema") + .arg("apply") + .arg("--schema") + .arg(&schema_path) + .arg(&graph), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stderr.contains("schema apply requires a graph with only main")); +} + +#[test] +fn schema_apply_allow_data_loss_flag_promotes_drops_to_hard() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema_path = temp.path().join("drop-age.pg"); + init_graph(&graph); + + // Drop the nullable `age` column. + let next_schema = fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace(" age: I32?\n", ""); + fs::write(&schema_path, next_schema).unwrap(); + + let output = output_success( + cli() + .arg("schema") + .arg("apply") + .arg("--schema") + .arg(&schema_path) + .arg("--allow-data-loss") + .arg("--json") + .arg(&graph), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["applied"], true); + + let drop_step = payload["steps"] + .as_array() + .unwrap() + .iter() + .find(|s| s["kind"] == "drop_property") + .expect("plan should include a drop_property step"); + assert_eq!( + drop_step["mode"], "hard", + "--allow-data-loss should promote Soft → Hard; full step: {drop_step}", + ); +} + +#[test] +fn schema_apply_without_allow_data_loss_keeps_soft_drops() { + // Symmetric to the above: same schema change without the flag → + // drops stay Soft. Pins default semantics against accidental Hard + // promotion if a future refactor changes the option threading. + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + let schema_path = temp.path().join("drop-age-soft.pg"); + init_graph(&graph); + + let next_schema = fs::read_to_string(fixture("test.pg")) + .unwrap() + .replace(" age: I32?\n", ""); + fs::write(&schema_path, next_schema).unwrap(); + + let output = output_success( + cli() + .arg("schema") + .arg("apply") + .arg("--schema") + .arg(&schema_path) + .arg("--json") + .arg(&graph), + ); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["applied"], true); + + let drop_step = payload["steps"] + .as_array() + .unwrap() + .iter() + .find(|s| s["kind"] == "drop_property") + .expect("plan should include a drop_property step"); + assert_eq!( + drop_step["mode"], "soft", + "no flag should leave drops Soft; full step: {drop_step}", + ); +} + +#[test] +fn schema_plan_parity_cli_and_sdk() { + // Same .pg through `Omnigraph::plan_schema_with_options` (SDK) and + // `omnigraph schema plan --json` (CLI). Asserts the steps array is + // byte-identical after JSON round-trip. HTTP doesn't expose a + // separate /schema/plan route — that side of parity is covered by + // the HTTP soft/hard drop tests, which exercise apply with + // identical fixtures. + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + let schema_path = temp.path().join("plan-parity.pg"); + let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace( + " age: I32?\n}", + " age: I32?\n nickname: String?\n}", + ); + fs::write(&schema_path, &next_schema).unwrap(); + + // CLI side. + let cli_output = output_success( + cli() + .arg("schema") + .arg("plan") + .arg("--schema") + .arg(&schema_path) + .arg("--json") + .arg(&graph), + ); + let cli_payload: Value = serde_json::from_slice(&cli_output.stdout).unwrap(); + + // SDK side: open graph, call plan_schema. + let plan = tokio::runtime::Runtime::new().unwrap().block_on(async { + let db = Omnigraph::open(graph.to_string_lossy().as_ref()) + .await + .unwrap(); + db.plan_schema(&next_schema).await.unwrap() + }); + let sdk_steps = serde_json::to_value(&plan.steps).unwrap(); + + assert_eq!( + cli_payload["steps"], sdk_steps, + "CLI plan steps must match SDK plan steps for identical input", + ); + assert_eq!(cli_payload["supported"], plan.supported); +} + +#[test] +fn graphs_subcommand_help_lists_list_only() { + let output = output_success(cli().arg("graphs").arg("--help")); + let stdout = stdout_string(&output); + assert!( + stdout.contains("list"), + "expected `list` subcommand in help output:\n{stdout}" + ); + let lowered = stdout.to_lowercase(); + assert!( + !lowered.contains("create a new graph"), + "graph create should not be in v0.6.0 help; got:\n{stdout}" + ); + assert!( + !lowered.contains("delete a graph"), + "graph delete should not be in v0.6.0 help; got:\n{stdout}" + ); +} + +#[test] +fn graphs_list_against_local_uri_errors_with_remote_only_message() { + let output = output_failure( + cli() + .arg("graphs") + .arg("list") + .arg("--uri") + .arg("/tmp/local"), + ); + let stderr = String::from_utf8_lossy(&output.stderr).into_owned(); + assert!( + stderr.contains("remote multi-graph server URL"), + "expected 'remote multi-graph server URL' rejection in stderr; got:\n{stderr}" + ); +} diff --git a/crates/omnigraph-cli/tests/support/mod.rs b/crates/omnigraph-cli/tests/support/mod.rs index 653be11..586bf93 100644 --- a/crates/omnigraph-cli/tests/support/mod.rs +++ b/crates/omnigraph-cli/tests/support/mod.rs @@ -317,3 +317,353 @@ impl SystemGraph { spawn_server_with_config_env(config, envs) } } + +// ---- helpers moved from the monolithic tests/cli.rs ---- +#[allow(unused_imports)] +use lance::Dataset; +#[allow(unused_imports)] +use lance::index::DatasetIndexExt; +#[allow(unused_imports)] +use omnigraph::db::{Omnigraph, ReadTarget}; + +pub const POLICY_YAML: &str = r#" +version: 1 +groups: + team: [act-andrew, act-bruno] + admins: [act-andrew] +protected_branches: [main] +rules: + - id: team-read + allow: + actors: { group: team } + actions: [read] + branch_scope: any + - id: team-write + allow: + actors: { group: team } + actions: [change] + branch_scope: unprotected + - id: admins-promote + allow: + actors: { group: admins } + actions: [branch_merge] + target_branch_scope: protected +"#; + +pub const POLICY_TESTS_YAML: &str = r#" +version: 1 +cases: + - id: allow-feature-write + actor: act-andrew + action: change + branch: feature + expect: allow + - id: deny-main-write + actor: act-bruno + action: change + branch: main + expect: deny +"#; + +pub fn manifest_dataset_version(graph: &std::path::Path) -> u64 { + tokio::runtime::Runtime::new().unwrap().block_on(async { + Omnigraph::open(graph.to_string_lossy().as_ref()) + .await + .unwrap() + .snapshot_of(ReadTarget::branch("main")) + .await + .unwrap() + .version() + }) +} + +pub fn forge_person_delete_drift(graph: &std::path::Path) -> (u64, u64) { + tokio::runtime::Runtime::new().unwrap().block_on(async { + let uri = graph.to_string_lossy(); + let db = Omnigraph::open(uri.as_ref()).await.unwrap(); + let snap = db + .snapshot_of(ReadTarget::branch("main")) + .await + .unwrap(); + let entry = snap.entry("node:Person").unwrap(); + let full_path = format!("{}/{}", uri.trim_end_matches('/'), entry.table_path); + let mut ds = Dataset::open(&full_path).await.unwrap(); + let deleted = ds.delete("name = 'Alice'").await.unwrap(); + assert_eq!(deleted.num_deleted_rows, 1); + let head = deleted.new_dataset.version().version; + assert!(head > entry.table_version); + (entry.table_version, head) + }) +} + +pub fn write_policy_config_fixture(root: &std::path::Path) -> (std::path::PathBuf, std::path::PathBuf) { + let config = root.join("omnigraph.yaml"); + let policy = root.join("policy.yaml"); + fs::write( + &config, + r#" +project: + name: policy-test-graph +policy: + file: ./policy.yaml +"#, + ) + .unwrap(); + fs::write(&policy, POLICY_YAML).unwrap(); + fs::write(root.join("policy.tests.yaml"), POLICY_TESTS_YAML).unwrap(); + (config, policy) +} + +pub fn write_cluster_config_fixture(root: &std::path::Path) { + fs::write( + root.join("people.pg"), + r#" +node Person { + name: String @key + age: I32? +} +"#, + ) + .unwrap(); + fs::write( + root.join("people.gq"), + r#" +query find_person($name: String) { + match { $p: Person { name: $name } } + return { $p.name, $p.age } +} +"#, + ) + .unwrap(); + fs::write(root.join("base.policy.yaml"), "rules: []\n").unwrap(); + fs::write( + root.join("cluster.yaml"), + r#" +version: 1 +metadata: + name: company-brain +state: + backend: cluster + lock: true +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq +policies: + base: + file: ./base.policy.yaml + applies_to: [knowledge] +"#, + ) + .unwrap(); +} + +pub fn init_cluster_derived_graph(root: &std::path::Path) { + init_named_cluster_graph(root, "knowledge", "people.pg"); +} + +pub fn init_named_cluster_graph(root: &std::path::Path, graph_id: &str, schema_file: &str) { + let graph_dir = root.join("graphs"); + fs::create_dir_all(&graph_dir).unwrap(); + output_success( + cli() + .arg("init") + .arg("--schema") + .arg(root.join(schema_file)) + .arg(graph_dir.join(format!("{graph_id}.omni"))), + ); +} + +pub fn write_cluster_lock(root: &std::path::Path, lock_id: &str, operation: &str) { + let state_dir = root.join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("lock.json"), + format!( + r#"{{"version":1,"lock_id":"{lock_id}","operation":"{operation}","created_at":"1970-01-01T00:00:00Z","pid":123}}"# + ), + ) + .unwrap(); +} + +pub fn write_cluster_applyable_state(root: &std::path::Path) -> serde_json::Value { + let validate = parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg("validate") + .arg("--config") + .arg(root) + .arg("--json"), + )); + let schema_digest = validate["resource_digests"]["schema.knowledge"] + .as_str() + .unwrap() + .to_string(); + let state_dir = root.join("__cluster"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join("state.json"), + format!( + r#"{{ + "version": 1, + "state_revision": 1, + "applied_revision": {{ + "resources": {{ + "graph.knowledge": {{ "digest": "seed" }}, + "schema.knowledge": {{ "digest": "{schema_digest}" }} + }} + }} +}} +"# + ), + ) + .unwrap(); + validate +} + +pub fn cluster_json(root: &std::path::Path, command: &str) -> serde_json::Value { + parse_stdout_json(&output_success( + cli() + .arg("cluster") + .arg(command) + .arg("--config") + .arg(root) + .arg("--json"), + )) +} + +pub fn write_multi_graph_cluster_fixture(root: &std::path::Path) { + write_cluster_config_fixture(root); + fs::write( + root.join("services.pg"), + r#" +node Service { + name: String @key +} +"#, + ) + .unwrap(); + fs::write( + root.join("services.gq"), + r#" +query find_service($name: String) { + match { $s: Service { name: $name } } + return { $s.name } +} +"#, + ) + .unwrap(); + fs::write(root.join("cluster_wide.policy.yaml"), "rules: []\n").unwrap(); + fs::write(root.join("shared.policy.yaml"), "rules: []\n").unwrap(); + fs::write( + root.join("cluster.yaml"), + r#" +version: 1 +metadata: + name: company-brain +state: + backend: cluster + lock: true +graphs: + knowledge: + schema: ./people.pg + queries: + find_person: + file: ./people.gq + engineering: + schema: ./services.pg + queries: + find_service: + file: ./services.gq +policies: + shared: + file: ./shared.policy.yaml + applies_to: [knowledge, engineering] + cluster_wide: + file: ./cluster_wide.policy.yaml + applies_to: [cluster] +"#, + ) + .unwrap(); +} + +pub fn change_for<'j>(json: &'j serde_json::Value, resource: &str) -> &'j serde_json::Value { + json["changes"] + .as_array() + .unwrap() + .iter() + .find(|change| change["resource"] == resource) + .unwrap_or_else(|| panic!("missing change for {resource}: {json}")) +} + +pub fn write_seed_fixture(root: &std::path::Path) -> std::path::PathBuf { + fs::create_dir_all(root.join("data")).unwrap(); + fs::create_dir_all(root.join("build")).unwrap(); + let raw_seed = root.join("data/seed.jsonl"); + let seed = root.join("seed.yaml"); + + fs::write( + &raw_seed, + concat!( + "{\"type\":\"Decision\",\"data\":{\"slug\":\"dec-alpha\",\"intent\":\"Alpha ship\"}}\n", + "{\"type\":\"Decision\",\"data\":{\"slug\":\"dec-beta\",\"intent\":\"Beta ship\",\"embedding\":[0.1,0.2]}}\n" + ), + ) + .unwrap(); + + fs::write( + &seed, + concat!( + "graph:\n", + " slug: mr-context-graph\n", + "sources:\n", + " raw_seed: ./data/seed.jsonl\n", + "artifacts:\n", + " embedded_seed: ./build/seed.embedded.jsonl\n", + "embeddings:\n", + " model: gemini-embedding-2-preview\n", + " dimension: 4\n", + " types:\n", + " Decision:\n", + " target: embedding\n", + " fields: [slug, intent]\n" + ), + ) + .unwrap(); + + seed +} + +pub fn write_seed_fixture_with_edge(root: &std::path::Path) -> std::path::PathBuf { + let seed = write_seed_fixture(root); + let raw_seed = root.join("data/seed.jsonl"); + fs::write( + &raw_seed, + concat!( + "{\"type\":\"Decision\",\"data\":{\"slug\":\"dec-alpha\",\"intent\":\"Alpha ship\"}}\n", + "{\"type\":\"Decision\",\"data\":{\"slug\":\"dec-beta\",\"intent\":\"Beta ship\",\"embedding\":[0.1,0.2]}}\n", + "{\"edge\":\"Triggered\",\"from\":\"sig-alpha\",\"to\":\"dec-alpha\"}\n" + ), + ) + .unwrap(); + seed +} + +pub fn read_embedded_rows(path: std::path::PathBuf) -> Vec<Value> { + fs::read_to_string(path) + .unwrap() + .lines() + .filter(|line| !line.trim().is_empty()) + .map(|line| serde_json::from_str(line).unwrap()) + .collect() +} + +pub fn queries_test_config(graph_uri: &str, entry: &str, gq_file: &str) -> String { + format!( + "graphs:\n local:\n uri: '{}'\n queries:\n {entry}:\n file: ./{gq_file}\n\ + cli:\n graph: local\npolicy: {{}}\n", + graph_uri.replace('\'', "''") + ) +} From 4a3f8e3a962b3e93cc5dd68fed5dc5e32bc972e6 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 15:21:44 +0300 Subject: [PATCH 116/165] ci: point the RustFS server smoke at the renamed s3 test target The test-split renamed tests/server.rs away; the job now targets --test s3. Also fixes a stale name filter (s3_repo vs the actual s3_graph test): a substring filter matching nothing passes vacuously, so this step had been running zero tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- .github/workflows/ci.yml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a77108f..2ed00d1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -351,7 +351,10 @@ jobs: run: cargo test --locked -p omnigraph-engine --test s3_storage -- --nocapture - name: Run RustFS server smoke - run: cargo test --locked -p omnigraph-server --test server server_opens_s3_repo_directly_and_serves_snapshot_and_read -- --nocapture + # The exact test name (not a loose substring): a filter that matches + # nothing passes vacuously, which silently ran zero tests here for a + # while (the old filter said s3_repo; the test says s3_graph). + run: cargo test --locked -p omnigraph-server --test s3 server_opens_s3_graph_directly_and_serves_snapshot_and_read -- --nocapture - name: Run RustFS CLI smoke run: cargo test --locked -p omnigraph-cli --test system_local local_cli_s3_end_to_end_init_load_read_flow -- --nocapture From 58855c0a7c61972fb6272aa5d205bbe375b3f7c0 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 15:56:22 +0300 Subject: [PATCH 117/165] feat(cluster,server): inline policy content + config-free --cluster URI boot MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two serving changes that complete RFC-006's read side: ServingPolicy carries the policy bundle CONTENT (digest-verified at snapshot read) instead of a blob path — the catalog may live on object storage, and the server must not re-read mutable state after the snapshot. The server grows a PolicySource enum: File for omnigraph.yaml deployments (unchanged), Inline for cluster boots, wired through PolicyEngine::load_{graph,server}_from_source. read_serving_snapshot_from_storage(uri) reads the applied revision straight from a storage root, and --cluster accepts a scheme-qualified URI (s3://bucket/prefix): config-free serving — a serving box needs only the URI and credentials; the ledger and catalog on the bucket ARE the deployment artifact. Bare paths keep the config-directory behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cluster/src/lib.rs | 2 +- crates/omnigraph-cluster/src/serve.rs | 33 +++++++++---- crates/omnigraph-cluster/src/tests.rs | 4 +- crates/omnigraph-server/src/lib.rs | 47 ++++++++++++++----- crates/omnigraph-server/src/main.rs | 8 ++-- crates/omnigraph-server/src/settings.rs | 44 ++++++++++++----- .../omnigraph-server/tests/boot_settings.rs | 24 +++++----- crates/omnigraph-server/tests/multi_graph.rs | 34 +++++++------- 8 files changed, 127 insertions(+), 69 deletions(-) diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index d97bb5b..1422dad 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -28,7 +28,7 @@ mod store; use store::{ClusterStore, StateLockGuard, StateSnapshot}; pub use types::*; use types::*; -pub use serve::{ServingGraph, ServingPolicy, ServingQuery, ServingSnapshot, read_serving_snapshot}; +pub use serve::{ServingGraph, ServingPolicy, ServingQuery, ServingSnapshot, read_serving_snapshot, read_serving_snapshot_from_storage}; use config::{QueriesDecl, observe_declared_graphs, validate_cluster_header, future_field_diagnostics, initial_import_state, observe_live_graph, preview_schema_migration, state_resource_digests, graph_address, policy_address, query_address, schema_address, load_desired, normalize_policy_target, parse_cluster_config, resolve_config_path, resolve_query_decls, validate_id, validate_query_source}; use diff::{FailedGraphOrigin, ResourceKind, append_policy_binding_changes, approved_resources, classify_changes, compute_approvals, compute_blast_radius, demote_dependents_of_failed_graphs, diff_resources, resource_kind}; use sweep::{mark_approvals_consumed, record_approval_consumed, sweep_recovery_sidecars, tombstone_graph_subtree, warn_pending_recovery_sidecars}; diff --git a/crates/omnigraph-cluster/src/serve.rs b/crates/omnigraph-cluster/src/serve.rs index b459641..4abd0bf 100644 --- a/crates/omnigraph-cluster/src/serve.rs +++ b/crates/omnigraph-cluster/src/serve.rs @@ -23,7 +23,10 @@ pub struct ServingQuery { #[derive(Debug, Clone)] pub struct ServingPolicy { pub name: String, - pub blob_path: PathBuf, + /// The policy bundle CONTENT, digest-verified against the applied + /// revision at read time. Content, not a path: the catalog may live on + /// object storage, and the server must not re-read mutable state. + pub source: String, pub applies_to: Vec<String>, } @@ -44,7 +47,6 @@ pub async fn read_serving_snapshot( config_dir: impl AsRef<Path>, ) -> Result<ServingSnapshot, Vec<Diagnostic>> { let config_dir = config_dir.as_ref().to_path_buf(); - let mut diagnostics: Vec<Diagnostic> = Vec::new(); // The declared storage: root decides where the ledger/catalog/graphs // live; config parse errors surface through the normal validation path. let parsed = parse_cluster_config(&config_dir); @@ -62,6 +64,25 @@ pub async fn read_serving_snapshot( }, None => ClusterStore::for_config_dir(&config_dir), }; + read_snapshot_with_store(backend).await +} + +/// Read the applied revision directly from a storage root URI — config-free +/// serving: a `--cluster s3://bucket/prefix` server needs no local files at +/// all, only the bucket and credentials. The ledger and catalog ARE the +/// deployment artifact. +pub async fn read_serving_snapshot_from_storage( + storage_root: &str, +) -> Result<ServingSnapshot, Vec<Diagnostic>> { + let backend = + ClusterStore::for_storage_root(storage_root).map_err(|diagnostic| vec![diagnostic])?; + read_snapshot_with_store(backend).await +} + +async fn read_snapshot_with_store( + backend: ClusterStore, +) -> Result<ServingSnapshot, Vec<Diagnostic>> { + let mut diagnostics: Vec<Diagnostic> = Vec::new(); // A ledger a sweep is about to rewrite must not start serving. let sidecars = backend.list_recovery_sidecars(&mut diagnostics).await; @@ -136,13 +157,9 @@ pub async fn read_serving_snapshot( continue; }; match backend.read_verified_payload(&kind, &entry.digest, address).await { - Ok(_) => policies.push(ServingPolicy { + Ok(source) => policies.push(ServingPolicy { name: name.clone(), - blob_path: PathBuf::from( - backend - .payload_display(&kind, &entry.digest) - .expect("policy kind always has a payload path"), - ), + source, applies_to, }), Err(diagnostic) => diagnostics.push(diagnostic), diff --git a/crates/omnigraph-cluster/src/tests.rs b/crates/omnigraph-cluster/src/tests.rs index ba7019f..63e7da7 100644 --- a/crates/omnigraph-cluster/src/tests.rs +++ b/crates/omnigraph-cluster/src/tests.rs @@ -2843,7 +2843,9 @@ policies: assert!(snapshot.queries[0].source.contains("query find_person")); assert_eq!(snapshot.policies.len(), 1); assert_eq!(snapshot.policies[0].applies_to, vec!["graph.knowledge"]); - assert!(snapshot.policies[0].blob_path.exists()); + // Content, not a path: the catalog may live on object storage. + // The fixture bundle is `rules: []` — assert the verified text. + assert!(snapshot.policies[0].source.contains("rules:")); } #[tokio::test] diff --git a/crates/omnigraph-server/src/lib.rs b/crates/omnigraph-server/src/lib.rs index 1c70083..3bde2a7 100644 --- a/crates/omnigraph-server/src/lib.rs +++ b/crates/omnigraph-server/src/lib.rs @@ -193,18 +193,28 @@ pub enum ServerConfigMode { config_path: PathBuf, /// `server.policy.file` (server-level Cedar policy for the /// management endpoints). Wired into `GET /graphs` authorization. - server_policy_file: Option<PathBuf>, + server_policy: Option<PolicySource>, }, } +/// Where a Cedar policy bundle comes from at startup. File-based for +/// omnigraph.yaml deployments; inline (digest-verified catalog content) +/// for cluster-mode boots, where the catalog may live on object storage +/// and the server must not re-read mutable state after the snapshot. +#[derive(Debug, Clone)] +pub enum PolicySource { + File(PathBuf), + Inline(String), +} + /// One graph's startup-time configuration: id, opened URI, optional -/// per-graph policy file path. Constructed by `load_server_settings` +/// per-graph policy source. Constructed by `load_server_settings` /// in multi mode; consumed by `serve`'s parallel open loop. #[derive(Debug, Clone)] pub struct GraphStartupConfig { pub graph_id: String, pub uri: String, - pub policy_file: Option<PathBuf>, + pub policy: Option<PolicySource>, /// Per-graph stored-query registry, loaded and identity-checked at /// settings-build time; type-checked against the schema when this /// graph's engine opens. @@ -994,9 +1004,9 @@ pub async fn serve(config: ServerConfig) -> Result<()> { ServerConfigMode::Single { policy_file, .. } => policy_file.is_some(), ServerConfigMode::Multi { graphs, - server_policy_file, + server_policy, .. - } => server_policy_file.is_some() || graphs.iter().any(|g| g.policy_file.is_some()), + } => server_policy.is_some() || graphs.iter().any(|g| g.policy.is_some()), }; let runtime_state = classify_server_runtime_state( !tokens.is_empty(), @@ -1045,7 +1055,7 @@ pub async fn serve(config: ServerConfig) -> Result<()> { ServerConfigMode::Multi { graphs, config_path, - server_policy_file, + server_policy, } => { info!( bind = %bind, @@ -1054,7 +1064,7 @@ pub async fn serve(config: ServerConfig) -> Result<()> { config = %config_path.display(), "serving omnigraph" ); - open_multi_graph_state(graphs, tokens, server_policy_file.as_ref(), config_path).await? + open_multi_graph_state(graphs, tokens, server_policy.as_ref(), config_path).await? } }; @@ -1065,6 +1075,14 @@ pub async fn serve(config: ServerConfig) -> Result<()> { Ok(()) } +/// Load a graph-scoped policy bundle from either source kind. +fn load_graph_policy(source: &PolicySource, graph_id: &str) -> Result<PolicyEngine> { + match source { + PolicySource::File(path) => Ok(PolicyEngine::load_graph(path, graph_id)?), + PolicySource::Inline(text) => Ok(PolicyEngine::load_graph_from_source(text, graph_id)?), + } +} + /// Parallel open of every graph in the startup config, with bounded /// concurrency (`buffer_unordered(4)`). Fail-fast — the first open error /// aborts startup; other in-flight opens are dropped (their `Omnigraph` @@ -1076,7 +1094,7 @@ pub async fn serve(config: ServerConfig) -> Result<()> { pub async fn open_multi_graph_state( graphs: Vec<GraphStartupConfig>, tokens: Vec<(String, String)>, - server_policy_file: Option<&PathBuf>, + server_policy_source: Option<&PolicySource>, config_path: PathBuf, ) -> Result<AppState> { use futures::{StreamExt, TryStreamExt}; @@ -1089,8 +1107,11 @@ pub async fn open_multi_graph_state( // The placeholder graph_id `"server"` is the sentinel the Cedar // resource-model refactor maps to the singleton // `Omnigraph::Server::"root"` entity at evaluation time. - let server_policy = match server_policy_file { - Some(path) => Some(PolicyEngine::load_server(path)?), + let server_policy = match server_policy_source { + Some(PolicySource::File(path)) => Some(PolicyEngine::load_server(path)?), + Some(PolicySource::Inline(source)) => { + Some(PolicyEngine::load_server_from_source(source)?) + } None => None, }; @@ -1128,9 +1149,9 @@ async fn open_single_graph(cfg: GraphStartupConfig) -> Result<Arc<GraphHandle>> // owned `Arc`, so no borrow of `db` survives into the match. let queries = validate_and_attach(cfg.queries, &db.catalog(), graph_id.as_str())?; - let (policy_arc, db) = match &cfg.policy_file { - Some(path) => { - let policy = PolicyEngine::load_graph(path, graph_id.as_str())?; + let (policy_arc, db) = match &cfg.policy { + Some(source) => { + let policy = load_graph_policy(source, graph_id.as_str())?; let policy_arc: Arc<PolicyEngine> = Arc::new(policy); let checker = Arc::clone(&policy_arc) as Arc<dyn omnigraph_policy::PolicyChecker>; (Some(policy_arc), db.with_policy(checker)) diff --git a/crates/omnigraph-server/src/main.rs b/crates/omnigraph-server/src/main.rs index 9000910..a138d12 100644 --- a/crates/omnigraph-server/src/main.rs +++ b/crates/omnigraph-server/src/main.rs @@ -14,10 +14,10 @@ struct Cli { target: Option<String>, #[arg(long)] config: Option<PathBuf>, - /// Boot from a cluster directory (the applied revision in - /// __cluster/state.json + content-addressed catalog blobs) instead of - /// omnigraph.yaml. Exclusive: cannot combine with <URI>, --target, or - /// --config. + /// Boot from a cluster: either a config directory (storage resolved + /// through cluster.yaml) or a storage-root URI directly + /// (s3://bucket/prefix — config-free serving from the bucket). + /// Exclusive: cannot combine with <URI>, --target, or --config. #[arg(long)] cluster: Option<PathBuf>, #[arg(long)] diff --git a/crates/omnigraph-server/src/settings.rs b/crates/omnigraph-server/src/settings.rs index 6531c3a..59c437b 100644 --- a/crates/omnigraph-server/src/settings.rs +++ b/crates/omnigraph-server/src/settings.rs @@ -14,7 +14,19 @@ pub(crate) async fn load_cluster_settings( cli_bind: Option<String>, cli_allow_unauthenticated: bool, ) -> Result<ServerConfig> { - let snapshot = omnigraph_cluster::read_serving_snapshot(cluster_dir).await.map_err(|diagnostics| { + // `--cluster` accepts either a config directory (the ledger location is + // resolved through cluster.yaml's `storage:` key) or a storage-root URI + // directly (`s3://bucket/prefix`) — config-free serving: the ledger and + // catalog on the bucket ARE the deployment artifact. + // Any scheme-qualified argument (s3://, file://) is a storage root; a + // bare path is a config directory. + let cluster_arg = cluster_dir.to_string_lossy(); + let snapshot = if cluster_arg.contains("://") { + omnigraph_cluster::read_serving_snapshot_from_storage(cluster_arg.as_ref()).await + } else { + omnigraph_cluster::read_serving_snapshot(cluster_dir).await + } + .map_err(|diagnostics| { let details = diagnostics .iter() .map(|diagnostic| format!("[{}] {}: {}", diagnostic.code, diagnostic.path, diagnostic.message)) @@ -26,19 +38,25 @@ pub(crate) async fn load_cluster_settings( // Bindings -> Cedar slots. The serving pipeline loads one bundle per // graph plus one server-level bundle; stacked bundles per scope are a // later slice — refuse loudly rather than silently merging policy. - let mut server_policy_file: Option<PathBuf> = None; - let mut graph_policy_files: BTreeMap<String, PathBuf> = BTreeMap::new(); + let mut server_policy: Option<PolicySource> = None; + let mut graph_policies: BTreeMap<String, PolicySource> = BTreeMap::new(); for policy in &snapshot.policies { for binding in &policy.applies_to { if binding == "cluster" { - if server_policy_file.replace(policy.blob_path.clone()).is_some() { + if server_policy + .replace(PolicySource::Inline(policy.source.clone())) + .is_some() + { bail!( "multiple policy bundles bind the cluster scope; cluster-mode serving supports one bundle per scope — split or merge bundles (multi-bundle scopes are a later slice)" ); } } else if let Some(graph_id) = binding.strip_prefix("graph.") { - if graph_policy_files - .insert(graph_id.to_string(), policy.blob_path.clone()) + if graph_policies + .insert( + graph_id.to_string(), + PolicySource::Inline(policy.source.clone()), + ) .is_some() { bail!( @@ -80,7 +98,7 @@ pub(crate) async fn load_cluster_settings( graphs.push(GraphStartupConfig { graph_id: graph.graph_id.clone(), uri: graph.root.to_string_lossy().to_string(), - policy_file: graph_policy_files.get(&graph.graph_id).cloned(), + policy: graph_policies.get(&graph.graph_id).cloned(), queries: registry, }); } @@ -97,7 +115,7 @@ pub(crate) async fn load_cluster_settings( mode: ServerConfigMode::Multi { graphs, config_path: cluster_dir.clone(), - server_policy_file, + server_policy, }, bind: cli_bind.unwrap_or_else(|| "127.0.0.1:8080".to_string()), allow_unauthenticated: cli_allow_unauthenticated || env_unauth, @@ -226,18 +244,18 @@ pub async fn load_server_settings( graphs.push(GraphStartupConfig { graph_id: name.clone(), uri, - policy_file: config.resolve_target_policy_file(name), + policy: config.resolve_target_policy_file(name).map(PolicySource::File), queries, }); } let config_path = config_path .cloned() .expect("has_explicit_config implies config_path is Some"); - let server_policy_file = config.resolve_server_policy_file(); + let server_policy = config.resolve_server_policy_file().map(PolicySource::File); ServerConfigMode::Multi { graphs, config_path, - server_policy_file, + server_policy, } } else { // Rule 5 → error with migration hint. @@ -729,11 +747,11 @@ server: .join("alpha.omni") .to_string_lossy() .into_owned(), - policy_file: None, + policy: None, queries: crate::queries::QueryRegistry::default(), }], config_path: temp.path().join("omnigraph.yaml"), - server_policy_file: Some(policy_path), + server_policy: Some(crate::PolicySource::File(policy_path)), }, bind: "127.0.0.1:0".to_string(), allow_unauthenticated: false, diff --git a/crates/omnigraph-server/tests/boot_settings.rs b/crates/omnigraph-server/tests/boot_settings.rs index 0e75486..3869d27 100644 --- a/crates/omnigraph-server/tests/boot_settings.rs +++ b/crates/omnigraph-server/tests/boot_settings.rs @@ -702,12 +702,14 @@ graphs: let alpha = &graphs[0]; let beta = &graphs[1]; assert_eq!(alpha.graph_id, "alpha"); - assert_eq!( - alpha.policy_file.as_ref().unwrap(), - &temp.path().join("policies/alpha.yaml") - ); + let omnigraph_server::PolicySource::File(alpha_policy) = + alpha.policy.as_ref().unwrap() + else { + panic!("yaml-configured policy must stay file-based"); + }; + assert_eq!(alpha_policy, &temp.path().join("policies/alpha.yaml")); assert_eq!(beta.graph_id, "beta"); - assert!(beta.policy_file.is_none()); + assert!(beta.policy.is_none()); } /// `server.policy.file` resolves alongside the graphs map. @@ -729,13 +731,11 @@ graphs: .unwrap(); let settings = load_server_settings(Some(&config_path), None, None, None, None, true).await.unwrap(); match settings.mode { - ServerConfigMode::Multi { - server_policy_file, .. - } => { - assert_eq!( - server_policy_file.unwrap(), - temp.path().join("server-policy.yaml") - ); + ServerConfigMode::Multi { server_policy, .. } => { + let omnigraph_server::PolicySource::File(path) = server_policy.unwrap() else { + panic!("yaml-configured server policy must stay file-based"); + }; + assert_eq!(path, temp.path().join("server-policy.yaml")); } _ => panic!("expected Multi"), } diff --git a/crates/omnigraph-server/tests/multi_graph.rs b/crates/omnigraph-server/tests/multi_graph.rs index 251f899..5ad847f 100644 --- a/crates/omnigraph-server/tests/multi_graph.rs +++ b/crates/omnigraph-server/tests/multi_graph.rs @@ -401,14 +401,14 @@ async fn cluster_boot_serves_applied_state() { let omnigraph_server::ServerConfigMode::Multi { graphs, config_path, - server_policy_file, + server_policy, } = settings.mode else { panic!("cluster boot must select multi-graph routing"); }; assert_eq!(graphs.len(), 1); assert_eq!(graphs[0].graph_id, "knowledge"); - assert!(server_policy_file.is_none()); + assert!(server_policy.is_none()); let state = omnigraph_server::open_multi_graph_state(graphs, Vec::new(), None, config_path) @@ -516,26 +516,26 @@ graphs: let settings = cluster_settings(temp.path()).await.unwrap(); let omnigraph_server::ServerConfigMode::Multi { graphs, - server_policy_file, + server_policy, .. } = settings.mode else { panic!("cluster boot must select multi-graph routing"); }; - let graph_policy = graphs[0].policy_file.as_ref().expect("graph-bound bundle"); - assert!( - graph_policy - .to_string_lossy() - .contains("__cluster/resources/policy/graph_rules/"), - "{graph_policy:?}" - ); - let server_policy = server_policy_file.expect("cluster-bound bundle"); - assert!( - server_policy - .to_string_lossy() - .contains("__cluster/resources/policy/cluster_rules/"), - "{server_policy:?}" - ); + // Cluster boots carry policy CONTENT (digest-verified catalog blobs), + // not paths — the catalog may live on object storage. + let omnigraph_server::PolicySource::Inline(graph_policy) = + graphs[0].policy.as_ref().expect("graph-bound bundle") + else { + panic!("cluster-mode graph policy must be inline content"); + }; + assert!(graph_policy.contains("actors:"), "{graph_policy:?}"); + let omnigraph_server::PolicySource::Inline(server_policy) = + server_policy.expect("cluster-bound bundle") + else { + panic!("cluster-mode server policy must be inline content"); + }; + assert!(server_policy.contains("kind: server"), "{server_policy:?}"); } #[tokio::test] From 8d7aed065f6ea49157c0ab6fc9ec16532abad297 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 15:56:40 +0300 Subject: [PATCH 118/165] test(cluster,server): gated object-storage cluster e2e + CI wiring + docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit s3_cluster.rs runs the full control-plane lifecycle against a real bucket (CI: containerized RustFS; locally the RustFS binary): import → lock released (pins the drop-time release regression caught on the first live smoke) → apply (graph roots + catalog on the bucket, nothing local) → serving snapshots from both the config dir and the bare URI → schema evolution → approved delete (prefix removal) → empty-cluster refusal. The server suite gains the config-free boot test: --cluster s3://… with zero local files serves a stored query over HTTP. CI: the rustfs job runs both suites; the classify filter covers the cluster store/serve modules and the new test files. The server smoke drops its name filter — every test in the s3 target is bucket-gated, and a filter matching nothing passes vacuously (which silently ran zero tests for a while). Docs: deployment.md gains the Bucket-no-volume shape as the preferred cloud deployment; cluster.md/server.md document --cluster <uri>; testing.md maps the new suite. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- .github/workflows/ci.yml | 16 +- crates/omnigraph-cluster/tests/s3_cluster.rs | 162 +++++++++++++++++++ crates/omnigraph-server/tests/s3.rs | 102 ++++++++++++ docs/dev/testing.md | 5 +- docs/user/cluster.md | 6 + docs/user/deployment.md | 31 +++- docs/user/server.md | 4 +- 7 files changed, 311 insertions(+), 15 deletions(-) create mode 100644 crates/omnigraph-cluster/tests/s3_cluster.rs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2ed00d1..b2b18d8 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -89,7 +89,9 @@ jobs: crates/omnigraph/src/storage.rs) run_rustfs_ci=true ;; crates/omnigraph/src/db/manifest.rs|crates/omnigraph/src/db/manifest/*) run_rustfs_ci=true ;; crates/omnigraph/tests/s3_storage.rs|crates/omnigraph/tests/helpers/*) run_rustfs_ci=true ;; - crates/omnigraph-server/tests/server.rs) run_rustfs_ci=true ;; + crates/omnigraph-cluster/src/store.rs|crates/omnigraph-cluster/src/serve.rs) run_rustfs_ci=true ;; + crates/omnigraph-cluster/tests/s3_cluster.rs) run_rustfs_ci=true ;; + crates/omnigraph-server/tests/s3.rs|crates/omnigraph-server/tests/support/*) run_rustfs_ci=true ;; crates/omnigraph-cli/tests/system_local.rs) run_rustfs_ci=true ;; esac done @@ -351,10 +353,14 @@ jobs: run: cargo test --locked -p omnigraph-engine --test s3_storage -- --nocapture - name: Run RustFS server smoke - # The exact test name (not a loose substring): a filter that matches - # nothing passes vacuously, which silently ran zero tests here for a - # while (the old filter said s3_repo; the test says s3_graph). - run: cargo test --locked -p omnigraph-server --test s3 server_opens_s3_graph_directly_and_serves_snapshot_and_read -- --nocapture + # No name filter: every test in the s3 target is bucket-gated, and a + # filter matching nothing passes vacuously (which silently ran zero + # tests here for a while — the old filter said s3_repo, the test + # said s3_graph). + run: cargo test --locked -p omnigraph-server --test s3 -- --nocapture + + - name: Run RustFS cluster e2e + run: cargo test --locked -p omnigraph-cluster --test s3_cluster -- --nocapture - name: Run RustFS CLI smoke run: cargo test --locked -p omnigraph-cli --test system_local local_cli_s3_end_to_end_init_load_read_flow -- --nocapture diff --git a/crates/omnigraph-cluster/tests/s3_cluster.rs b/crates/omnigraph-cluster/tests/s3_cluster.rs new file mode 100644 index 0000000..3c7cef3 --- /dev/null +++ b/crates/omnigraph-cluster/tests/s3_cluster.rs @@ -0,0 +1,162 @@ +//! Cluster-on-object-storage end-to-end (RFC-006): the full control-plane +//! lifecycle with `storage: s3://…` — import, apply (graph roots + catalog +//! on the bucket), serving snapshots from both the config dir and the bare +//! storage URI, schema evolution, and the approved delete (prefix removal). +//! +//! Gated like every S3 suite: skips unless `OMNIGRAPH_S3_TEST_BUCKET` is +//! set (CI runs it against containerized RustFS; locally use the RustFS +//! binary + `AWS_*` env, see docs/dev/testing.md). +//! +//! Runtime flavor is multi_thread on purpose: the state-lock guard's +//! drop-time release uses block_in_place on object stores, which is the +//! production (CLI) runtime shape — and the lock-release regression this +//! suite pins (a spawned delete dying with a short-lived runtime) only +//! reproduces realistically under it. + +use std::env; +use std::fs; + +use omnigraph_cluster::{ + apply_config_dir, import_config_dir, read_serving_snapshot, + read_serving_snapshot_from_storage, status_config_dir, validate_config_dir, +}; +use ulid::Ulid; + +const SCHEMA_V1: &str = "node Person {\n name: String @key\n}\n"; +const SCHEMA_V2: &str = "node Person {\n name: String @key\n title: String?\n}\n"; +const FIND_PERSON_GQ: &str = "query find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n"; +const POLICY_YAML: &str = r#" +version: 1 +actors: + - id: act-admin + roles: [admin] +rules: + - effect: permit + actions: [read, change, schema_apply, branch_create, branch_delete, branch_merge] + roles: [admin] +"#; + +/// Unique per-run storage root under the test bucket, or None to skip. +fn s3_storage_root(suite: &str) -> Option<String> { + let bucket = env::var("OMNIGRAPH_S3_TEST_BUCKET").ok()?; + Some(format!("s3://{bucket}/cluster-e2e/{suite}-{}", Ulid::new())) +} + +fn write_cluster_fixture(dir: &std::path::Path, storage_root: &str, schema: &str) { + fs::write(dir.join("people.pg"), schema).unwrap(); + fs::create_dir_all(dir.join("queries")).unwrap(); + fs::write(dir.join("queries/people.gq"), FIND_PERSON_GQ).unwrap(); + fs::write(dir.join("intel.policy.yaml"), POLICY_YAML).unwrap(); + fs::write( + dir.join("cluster.yaml"), + format!( + r#"version: 1 +storage: {storage_root} +graphs: + knowledge: + schema: people.pg + queries: queries/ +policies: + intel: + file: intel.policy.yaml + applies_to: [graph.knowledge] +"# + ), + ) + .unwrap(); +} + +#[tokio::test(flavor = "multi_thread")] +async fn s3_cluster_full_lifecycle_import_apply_serve_evolve_delete() { + let Some(root) = s3_storage_root("lifecycle") else { + eprintln!("skipping s3 cluster e2e: OMNIGRAPH_S3_TEST_BUCKET is not set"); + return; + }; + let dir = tempfile::tempdir().unwrap(); + write_cluster_fixture(dir.path(), &root, SCHEMA_V1); + + // validate is config-only and must pass before any bucket I/O. + let validate = validate_config_dir(dir.path()); + assert!(validate.ok, "{:?}", validate.diagnostics); + + let import = import_config_dir(dir.path()).await; + assert!(import.ok, "{:?}", import.diagnostics); + + // The lock-release regression (caught live on the first smoke): the + // guard's drop must COMPLETE its bucket delete before the command + // returns — a follow-up command finding `state_lock_held` means the + // release was spawned into a dying runtime. + let status = status_config_dir(dir.path()).await; + assert!(status.ok, "{:?}", status.diagnostics); + assert!( + !status.state_observations.locked, + "import leaked the state lock on the bucket: {:?}", + status.state_observations + ); + + let apply = apply_config_dir(dir.path()).await; + assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); + + // Nothing stored locally: the config dir holds only declared sources. + assert!(!dir.path().join("__cluster").exists()); + assert!(!dir.path().join("graphs").exists()); + + // Serving snapshot resolves through cluster.yaml's storage: key… + let via_config = read_serving_snapshot(dir.path()).await.unwrap(); + assert_eq!(via_config.graphs.len(), 1); + let graph_root = via_config.graphs[0].root.to_string_lossy().to_string(); + assert!( + graph_root.starts_with("s3://") && graph_root.ends_with("graphs/knowledge.omni"), + "{graph_root}" + ); + assert_eq!(via_config.queries.len(), 1); + assert_eq!(via_config.policies.len(), 1); + assert!( + via_config.policies[0].source.contains("act-admin"), + "policy must carry verified content, not a path" + ); + + // …and config-free, straight from the bucket URI (the deployment + // payoff: a server needs only the URI and credentials). + let via_uri = read_serving_snapshot_from_storage(&root).await.unwrap(); + assert_eq!(via_uri.graphs.len(), 1); + assert_eq!( + via_uri.graphs[0].root.to_string_lossy(), + via_config.graphs[0].root.to_string_lossy() + ); + assert_eq!(via_uri.policies.len(), 1); + + // Schema evolution converges on the bucket. + write_cluster_fixture(dir.path(), &root, SCHEMA_V2); + let evolve = apply_config_dir(dir.path()).await; + assert!(evolve.ok && evolve.converged, "{:?}", evolve.diagnostics); + + // Approved delete: drop the graph from the config; the plan demands an + // approval, the approved apply prefix-deletes the bucket root. + fs::write( + dir.path().join("cluster.yaml"), + format!("version: 1\nstorage: {root}\ngraphs: {{}}\n"), + ) + .unwrap(); + let plan = omnigraph_cluster::plan_config_dir(dir.path()).await; + assert!(plan.ok, "{:?}", plan.diagnostics); + let approval = plan + .approvals_required + .first() + .expect("graph delete requires approval"); + let approve = omnigraph_cluster::approve_config_dir( + dir.path(), + &approval.resource, + "e2e-operator", + ) + .await; + assert!(approve.ok, "{:?}", approve.diagnostics); + let delete = apply_config_dir(dir.path()).await; + assert!(delete.ok && delete.converged, "{:?}", delete.diagnostics); + + let after = read_serving_snapshot_from_storage(&root).await; + assert!( + after.is_err(), + "an empty cluster must refuse to serve, got {after:?}" + ); +} diff --git a/crates/omnigraph-server/tests/s3.rs b/crates/omnigraph-server/tests/s3.rs index b0126a8..2c61125 100644 --- a/crates/omnigraph-server/tests/s3.rs +++ b/crates/omnigraph-server/tests/s3.rs @@ -75,3 +75,105 @@ async fn server_opens_s3_graph_directly_and_serves_snapshot_and_read() { assert_eq!(read_body["row_count"], 1); assert_eq!(read_body["rows"][0]["p.name"], "Alice"); } + +/// Config-free cluster serving (RFC-006): boot `--cluster s3://bucket/prefix` +/// with NO local files at all — the ledger and catalog on the bucket are the +/// whole deployment artifact. The fixture cluster is applied from a temp +/// config dir, which is then dropped before the server boots from the URI. +#[tokio::test(flavor = "multi_thread")] +async fn server_boots_cluster_from_bare_storage_uri_and_serves_query() { + let Some(bucket) = std::env::var("OMNIGRAPH_S3_TEST_BUCKET").ok() else { + eprintln!("skipping s3 cluster-serving test: OMNIGRAPH_S3_TEST_BUCKET is not set"); + return; + }; + let unique = format!( + "{}-{}", + std::process::id(), + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() + ); + let root = format!("s3://{bucket}/cluster-serve/{unique}"); + + // Apply a one-graph cluster onto the bucket, seed it, then DROP the + // config dir — the boot below must need nothing local. + { + let dir = tempfile::tempdir().unwrap(); + fs::write( + dir.path().join("people.pg"), + "node Person {\n name: String @key\n}\n", + ) + .unwrap(); + fs::write( + dir.path().join("people.gq"), + "query find_person($name: String) {\n match { $p: Person { name: $name } }\n return { $p.name }\n}\n", + ) + .unwrap(); + fs::write( + dir.path().join("cluster.yaml"), + format!( + "version: 1\nstorage: {root}\ngraphs:\n knowledge:\n schema: people.pg\n queries:\n find_person:\n file: people.gq\n" + ), + ) + .unwrap(); + let import = omnigraph_cluster::import_config_dir(dir.path()).await; + assert!(import.ok, "{:?}", import.diagnostics); + let apply = omnigraph_cluster::apply_config_dir(dir.path()).await; + assert!(apply.ok && apply.converged, "{:?}", apply.diagnostics); + + let graph_uri = format!("{root}/graphs/knowledge.omni"); + let mut db = Omnigraph::open(&graph_uri).await.unwrap(); + load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Ada\"}}\n", + LoadMode::Overwrite, + ) + .await + .unwrap(); + } + + let settings = omnigraph_server::load_server_settings( + None, + Some(&std::path::PathBuf::from(&root)), + None, + None, + None, + true, + ) + .await + .unwrap(); + let omnigraph_server::ServerConfigMode::Multi { + graphs, + config_path, + server_policy, + } = settings.mode + else { + panic!("cluster boot must select multi-graph routing"); + }; + let state = omnigraph_server::open_multi_graph_state( + graphs, + Vec::new(), + server_policy.as_ref(), + config_path, + ) + .await + .unwrap(); + let app = build_app(state); + + let response = tower::ServiceExt::oneshot( + app, + Request::builder() + .method(Method::POST) + .uri("/graphs/knowledge/queries/find_person") + .header("content-type", "application/json") + .body(Body::from(json!({"params": {"name": "Ada"}}).to_string())) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(response.status(), StatusCode::OK); + let bytes = axum::body::to_bytes(response.into_body(), usize::MAX).await.unwrap(); + let value: serde_json::Value = serde_json::from_slice(&bytes).unwrap(); + assert_eq!(value["rows"][0]["p.name"], "Ada", "{value}"); +} diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 848594a..a817428 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -8,7 +8,7 @@ This file is the always-on map of the test surface. **Consult it before every ta |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | | `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish; includes the `cluster_e2e_*` lifecycle compositions over the spawned binary — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `system_local.rs` (incl. the full-cycle cluster lifecycle with a spawned `--cluster` server — declare→serve→evolve→drift-heal→approved-delete — and applied-policy enforcement over HTTP), `system_remote.rs`, share `tests/support/mod.rs` | -| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows), and 5A policy binding metadata (applies_to in the applied revision, binding-change diffing + convergence, pre-5A backfill), and the 5B serving-snapshot read API (converged read, refusal rows) | +| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated); `tests/s3_cluster.rs` (bucket-gated full lifecycle on object storage) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows), and 5A policy binding metadata (applies_to in the applied revision, binding-change diffing + convergence, pre-5A backfill), and the 5B serving-snapshot read API (converged read, refusal rows) | | `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level; incl. cluster-mode boot — converged-dir serving, policy binding wiring, boot refusals), `openapi.rs` (OpenAPI drift / regeneration) | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | @@ -64,7 +64,8 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav CI runs three S3-backed tests against a containerized RustFS server (`.github/workflows/ci.yml` → `rustfs_integration` job): - `cargo test -p omnigraph-engine --test s3_storage` -- `cargo test -p omnigraph-server --test server server_opens_s3_graph_directly_and_serves_snapshot_and_read` +- `cargo test -p omnigraph-server --test s3` (single-graph serving + config-free `--cluster s3://` boot) +- `cargo test -p omnigraph-cluster --test s3_cluster` (full control-plane lifecycle on the bucket) - `cargo test -p omnigraph-cli --test system_local local_cli_s3_end_to_end_init_load_read_flow` Locally, set `OMNIGRAPH_S3_TEST_BUCKET` (and the usual `AWS_*` vars including `AWS_ENDPOINT_URL_S3` for non-AWS) before running. Without those, S3 tests skip gracefully. diff --git a/docs/user/cluster.md b/docs/user/cluster.md index 19755fb..0d6dac5 100644 --- a/docs/user/cluster.md +++ b/docs/user/cluster.md @@ -84,6 +84,12 @@ OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-reader":"s3cret"}' \ omnigraph-server --cluster company-brain --bind 0.0.0.0:8080 ``` +`--cluster` accepts either a **config directory** (the storage root resolves +through `cluster.yaml`'s `storage:` key) or a **storage-root URI directly** +(`--cluster s3://bucket/prefix`) — config-free serving: a serving box needs +only the URI and credentials, no checkout of the config repo. The ledger and +catalog on the bucket are the deployment artifact. + `--cluster` is an **exclusive boot source**: it cannot be combined with a graph URI, `--target`, or `--config`, and `omnigraph.yaml` is never read in this mode. Routing is always multi-graph: diff --git a/docs/user/deployment.md b/docs/user/deployment.md index 00f8272..ece7b5d 100644 --- a/docs/user/deployment.md +++ b/docs/user/deployment.md @@ -47,10 +47,31 @@ omnigraph-server s3://my-bucket/graphs/example/releases/2026-04-10-v0.1.0 \ ## Cluster Mode in Containers (AWS, Railway) -A cluster-booted deployment serves a **cluster directory** (config + state -ledger + content-addressed catalog + graph data) from a mounted volume — the -one structural difference from the stateless S3 single-graph shape, which -needs no volume at all. The container contract: +A cluster-booted deployment has **two shapes** since the `storage:` root +(RFC-006): + +- **Bucket, no volume (preferred for cloud)** — the cluster's ledger, + catalog, and graph data live under an object-storage root + (`storage: s3://bucket/prefix` in `cluster.yaml`). The server boots + **config-free** from the bare URI; the container needs no volume at all: + + ```bash + docker run -d \ + -e OMNIGRAPH_CLUSTER=s3://my-bucket/clusters/company-brain \ + -e AWS_ACCESS_KEY_ID=... -e AWS_SECRET_ACCESS_KEY=... \ + -e OMNIGRAPH_SERVER_BEARER_TOKEN=... \ + -p 8080:8080 <image> + ``` + + Day-2 runs from any operator checkout of the config repo: + `omnigraph cluster apply --config ./company-brain` (the `storage:` key + routes every stored byte to the bucket), then restart the service. The + state lock is genuinely cross-machine on object storage, so CI and + operator shells contend safely. + +- **Volume (file-rooted)** — the original shape: the whole cluster + directory on a mounted volume. Still fully supported; the container + contract: ```bash docker run -d \ @@ -102,8 +123,6 @@ above). ### Constraints (current honest list) -- **Cluster directories are local-filesystem** — the volume is mandatory; - S3-hosted cluster dirs are not supported. - **No hot reload** — applied changes serve on the next restart. - **Single-writer apply** — run `cluster apply` from one place at a time (the state lock enforces this; CI or one operator shell, not both). diff --git a/docs/user/server.md b/docs/user/server.md index 0922e74..391b7ae 100644 --- a/docs/user/server.md +++ b/docs/user/server.md @@ -16,7 +16,7 @@ Axum 0.8 + tokio + utoipa-generated OpenAPI. **Two modes** (v0.6.0+): single-gra ### Cluster-booted multi mode (Phase 5) -`omnigraph-server --cluster <dir>` boots from the cluster catalog's **applied +`omnigraph-server --cluster <dir-or-uri>` boots from the cluster catalog's **applied revision** (`state.json` + content-addressed blobs) instead of `omnigraph.yaml` — an exclusive boot source: combining it with `<URI>`, `--target`, or `--config` is a startup error, and `omnigraph.yaml` is never @@ -27,7 +27,7 @@ for what is read and the fail-fast readiness rules. `--bind`, Mode inference: -0. CLI `--cluster <dir>` → **multi, cluster-booted** (exclusive) +0. CLI `--cluster <dir | s3://…>` → **multi, cluster-booted** (exclusive; a scheme-qualified argument reads the ledger straight from the storage root, no local config) 1. CLI positional `<URI>` → single 2. CLI `--target <name>` → single 3. `server.graph` in config → single From d531f609997edafc46050ee398f78db4606927ca Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 18:29:55 +0300 Subject: [PATCH 119/165] =?UTF-8?q?docs(rfc):=20RFC-007=20=E2=80=94=20per-?= =?UTF-8?q?operator=20config,=20the=20operator=20slice=20of=20RFC-002?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Terraform-style operator/project split: ~/.omnigraph/config.yaml for identity (operator.actor in the --as cascade), credentials keyed by server name (env -> 0600 credentials file; no inline secrets), and operator-owned named servers that project configs reference but cannot redefine. Explicitly a staged subset of RFC-002: adopts its settled decisions (one dir, keyed credentials, env precedence), defers GraphLocator/use/state-layer, and encodes the ten confirmed PR #139 findings as design rules (compat shims, key-level merges, atomic writes, the project-layer trust boundary). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/index.md | 1 + docs/dev/rfc-007-operator-config.md | 256 ++++++++++++++++++++++++++++ 2 files changed, 257 insertions(+) create mode 100644 docs/dev/rfc-007-operator-config.md diff --git a/docs/dev/index.md b/docs/dev/index.md index 4bc1e6a..7e50777 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -76,6 +76,7 @@ Working documents for in-flight feature work. Removed when the work lands. | Future cluster control plane — declarative as-code config, JSON state ledger, reconciler | [cluster-config-specs.md](cluster-config-specs.md), [cluster-axioms.md](cluster-axioms.md), [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md) | | Cluster graph & schema apply — Phase 4 sidecars, roll-forward recovery, approval artifacts | [rfc-004-cluster-graph-schema-apply.md](rfc-004-cluster-graph-schema-apply.md) | | Server boots from cluster state — Phase 5 mode switch, applied-revision serving | [rfc-005-server-cluster-boot.md](rfc-005-server-cluster-boot.md) | +| Per-operator config — `~/.omnigraph/` identity, keyed credentials, named servers (the operator slice of RFC-002) | [rfc-007-operator-config.md](rfc-007-operator-config.md) | ## Boundary diff --git a/docs/dev/rfc-007-operator-config.md b/docs/dev/rfc-007-operator-config.md new file mode 100644 index 0000000..5c6b6b2 --- /dev/null +++ b/docs/dev/rfc-007-operator-config.md @@ -0,0 +1,256 @@ +# RFC: Per-Operator Config — the Operator Slice of RFC-002 + +**Status:** Proposed +**Date:** 2026-06-11 +**Builds on:** [rfc-002-config-cli-architecture.md](rfc-002-config-cli-architecture.md) (Proposed; implementation parked — PRs #139/#162 closed over review findings), [rfc-005-server-cluster-boot.md](rfc-005-server-cluster-boot.md) (Landed), RFC-006 storage roots (#186/#190/#194, landed). The #139 review record is a normative input: every design rule in §D6 traces to a confirmed finding. +**Target release:** unversioned (staged; see Sequencing). + +## Summary + +Give OmniGraph the operator half of the Terraform config split. Terraform +separates `~/.terraformrc` (who I am, my credentials, my CLI behavior) from +the working directory's `*.tf` (what the project declares). OmniGraph today +has only the project half: `./omnigraph.yaml` in the current working +directory (or `--config <path>`), and nothing else — no home-level config, +no walk-up, no env override for the CLI. Operator identity and credentials +must be re-declared in every directory an operator works from, and — worse — +they end up in files that live next to repo-committed project config. + +This RFC introduces **`~/.omnigraph/config.yaml`** (the operator layer) and +a **keyed credentials chain**, scoped deliberately small: + +1. **Operator identity** — a default actor for every `--as` cascade. +2. **Credentials by server name** — no more inventing env-var names per + server; secrets never inline, never in the project layer. +3. **Named servers** — operator-owned endpoint definitions that project + configs can reference but not redefine. + +It is explicitly a **subset of RFC-002**, sequenced to land. RFC-002 settled +the right long-term decisions (one `~/.omnigraph/` dir, credentials keyed by +server name, `OMNIGRAPH_CONFIG`/`OMNIGRAPH_HOME` env precedence) but its +implementation arrived as one 4,800-line PR mixing a crate extraction with +behavior changes, and died over ten confirmed findings. This RFC adopts +RFC-002's settled decisions verbatim where they apply, defers everything +else (`GraphLocator`, multi-homing, `omnigraph use`, the State layer), and +encodes the #139 findings as design rules so the same failures cannot recur. + +## Motivation + +Three concrete pains, all hit in real operation this cycle: + +- **Identity repetition.** The cluster actor cascade (#180) resolves + `--as` from the per-operator `omnigraph.yaml` — which means every + operator hand-maintains a copy in every working directory (the + `~/exp/intel` setup needed exactly this). A repo-committed + `omnigraph.yaml` cannot carry `as: act-andrew` without claiming every + contributor is Andrew. +- **Credential ergonomics.** `bearer_token_env` forces three coordinated + steps per server (invent a var name, reference it in config, set it in + the secret store). The peer group — AWS profiles, `gh hosts`, kubeconfig + users — keys secrets by the server's *name*. +- **Cluster-era working shape.** With clusters on object storage (RFC-006), + the project directory is a *declaration checkout* — operators run + `cluster apply --config ./checkout` from anywhere. The things that are + about the *operator* (who am I, which servers do I know, how do I like + output formatted) have no home that travels with them. + +## Non-Goals + +- **`GraphLocator` / multi-homed graph resolution** (RFC-002 §1) — the + biggest and riskiest part of config-v2; untouched here. +- **`omnigraph use` / the State layer** (`~/.omnigraph/state/`) — deferred + with it (finding #2 showed its precedence interacts badly with scaffolds; + that problem belongs to the slice that introduces it). +- **OS keychain integration** — the credentials *chain* (§D4) leaves a slot + for it; this RFC ships env + file sources only. +- **Project-file walk-up.** Terraform does not walk up from subdirectories + and neither do we — `--config` (or running in the directory) stays the + explicit, deterministic story. Rejected, not deferred: walk-up makes "which + config am I using" a function of cwd depth, the class of surprise this RFC + exists to remove. +- **Renaming or removing anything.** No flag renames, no key renames, no + schema-version bumps (findings #1, #3, #10). + +## Background (verified against main) + +- **Project-config lookup today** (`crates/omnigraph-server/src/config.rs:529-553`, + shared by CLI and server): `--config <path>`, else `./omnigraph.yaml` in + cwd, else built-in defaults. Relative paths inside the file resolve + against the file's own directory (`base_dir`). No env var, no home file, + no walk-up. +- **Side-effect on load** (`crates/omnigraph-cli/src/helpers.rs:102-108`): + `load_cli_config` also loads `auth.env_file` into the process env — + this is how `OMNIGRAPH_BEARER_TOKEN` reaches remote commands today. +- **Actor resolution** (`helpers.rs:170`, #180): `--as` flag, else the + project config's actor — currently the end of the chain. +- **Existing credential mechanism**: `TargetConfig.bearer_token_env` names + an env var; `auth.env_file` points at a git-ignored dotenv. Both keep + working indefinitely (RFC-002 already committed to this; finding #3 + showed what happens otherwise). +- **`OMNIGRAPH_CONFIG`** exists today only as the *container entrypoint's* + translation to the server's `--config`. The CLI does not read it. + +## Design + +### D1. Files and discovery + +``` +~/.omnigraph/config.yaml # the operator layer (this RFC) +~/.omnigraph/credentials # keyed secrets, 0600, git-irrelevant (§D4) +./omnigraph.yaml # the project layer (unchanged) +``` + +Discovery order for the operator file: `$OMNIGRAPH_HOME/config.yaml` if +`OMNIGRAPH_HOME` is set, else `~/.omnigraph/config.yaml`. Absent file = +empty layer, never an error. `~` is expanded wherever paths are read +(finding #9 — today a literal `./~/...` directory gets created). + +`OMNIGRAPH_CONFIG=<path>` becomes a first-class override for the *project* +file in the CLI (highest precedence below the `--config` flag), aligning the +CLI with the container contract that already uses this variable for the +server. One name, one meaning, both binaries. + +Per RFC-002 §4 (adopted verbatim): `~/.omnigraph/` is the one canonical +dir — cache/state subdirectories arrive with their own slices; XDG roots are +not part of the mental model (`$XDG_CONFIG_HOME` may be honored as a +fallback read location if set, but is never written to). + +### D2. The operator schema (v1 of this layer) + +```yaml +# ~/.omnigraph/config.yaml — about the OPERATOR, never about a project +operator: + actor: act-andrew # default for every --as cascade + +servers: # operator-owned endpoint definitions + intel-dev: + url: http://127.0.0.1:8080 + prod: + url: https://graph.modernrelay.ai + # No token here, ever. Resolution: §D4. + +defaults: + output: table # read --format default +``` + +Unknown keys are a **warning, not an error** in this layer (an operator file +written by a newer CLI must not brick an older one; contrast with +`cluster.yaml`, where unknown keys are deliberately fatal because they +change what a *plan* means). + +### D3. Precedence and the merge rule + +``` +flag > env > project omnigraph.yaml > operator config > built-in +``` + +with exactly one principled inversion (§D5): **credentials and endpoint +definitions never come from the project layer when an operator-layer +definition exists for the same server name.** + +Merging is **key-level**: scalars override per key; maps (`servers:`, +`graphs:`) merge per *entry*, and entries merge per *field* (finding #13 — +`merge_map` replacing whole entries silently dropped sibling fields). A +project file referencing `server: prod` composes with the operator's +`servers.prod.url`; it does not need to re-declare it and cannot +accidentally clobber half of it. + +Concretely for the two flows this slice touches: + +- **Actor**: `--as` > project `as:`/actor key (unchanged semantics) > + `operator.actor` > none (commands that need an actor keep failing loudly). +- **Output format**: `--format` > project default > `defaults.output` > + `table`. + +### D4. Credentials: keyed by server name, by-reference always + +Adopted from RFC-002 §5 unchanged, minus the keychain (a later source in +the same chain). For a server named `<name>`, the resolution chain is: + +1. `OMNIGRAPH_TOKEN_<NAME>` (uppercased, `-`→`_`) — explicit env, wins. +2. `[<name>]` section in `~/.omnigraph/credentials` (INI-style, `0600`; + the loader refuses a group/world-readable file). +3. The legacy pair — `bearer_token_env` + `auth.env_file` — exactly as + today, for configs that already use it. + +No inline secrets in any YAML file, operator or project (the existing +invariant 12 posture extended to disk). A future `omnigraph login <name>` +writes/rotates one section of the credentials file via temp + rename +(finding #7: every operator-layer write is atomic), creating it `0600`. + +### D5. The trust boundary (the security findings, made structural) + +Findings #4, #5, #6 share one root cause: the project layer — a file that +arrives with a *repo checkout* — could redirect where requests go and what +secrets they carry. The rules: + +1. **A project file may *reference* a server by name; it may not *redefine* + an operator-defined server.** If `./omnigraph.yaml` declares + `servers.prod.url` and `~/.omnigraph/config.yaml` also defines `prod`, + the operator definition wins and the CLI warns about the shadowed + project entry. A project-only server name keeps working (legacy compat), + but the keyed-credentials chain (§D4 steps 1–2) never resolves for it — + only the legacy explicit `bearer_token_env` does. Net effect: a malicious + checkout cannot point `prod` at an attacker host and harvest the + operator's `prod` token. +2. **`auth.env_file` keeps auto-loading (compat), but project-layer + env-files cannot *override* variables already set in the process or by + the operator layer** — first-set-wins, operator-before-project (the + existing real-env-wins rule, extended one layer down). Finding #5's + injection becomes a no-op against any var the operator actually uses. +3. **A token is sent only to the server it is keyed to.** The legacy + single `OMNIGRAPH_BEARER_TOKEN` fallback keeps working for the + single-server shape, but when a request resolves through a *named* + server, only that name's chain applies (finding #6's broadcast). + +### D6. Compatibility rules (the #139 findings as law) + +| Rule | Source finding | +|---|---| +| No flag or key is removed or renamed; new behavior is additive | #1, #3 | +| A config that loads today loads identically after this RFC; new validation applies only to new keys | #3, #8, #10 | +| Every operator-layer file write is temp + rename, never in-place | #7 | +| `~` expands wherever a path is read | #9 | +| Map merges are per-entry, per-field — never wholesale replace | #13 | +| One resolution path per concern — the actor chain and the token chain each have exactly one implementation, called by CLI and server alike | #11, #12 | +| Each slice lands as its own PR with the workspace gate green; no slice mixes mechanical moves with behavior changes | #139's disposition | + +## Sequencing + +Three PRs, each independently useful, each landable without the next: + +1. **PR 1 — the operator file + identity.** Loader for + `~/.omnigraph/config.yaml` (+ `OMNIGRAPH_HOME`, `~`-expansion, warn-only + unknown keys), `operator.actor` joining the `--as` cascade, + `defaults.output` joining the format cascade, `OMNIGRAPH_CONFIG` env for + the CLI's project file. Docs: `cli-reference.md` gains the layer table. +2. **PR 2 — keyed credentials.** `servers:` in the operator layer, the + §D4 chain (env + credentials file), the §D5 trust rules, and + `omnigraph login <name>` (atomic write, `0600`). Legacy mechanisms + untouched and tested-as-untouched. +3. **PR 3 — project references.** `server: <name>` in project + graph/target entries resolving through operator-defined servers, with + the shadowing warning. This is the *bridge* toward RFC-002's locator — + it gives multi-server addressing a safe, minimal form without the + `GraphLocator` rework. + +## Open questions + +- Should `operator.actor` apply to *local* (embedded-engine) writes too, or + only where a server/cluster boundary exists? Leaning yes-everywhere: one + identity chain (§D6 one-path rule), and local audit rows get better. +- Does `defaults.output` belong in slice 1, or is identity-only an even + cleaner first PR? (Cost of including it is one cascade hop; value is + immediate.) +- `omnigraph config view --resolved` (RFC-002 had it; #139 shipped a + version) — slice 1 or slice 2? It materially helps debugging precedence, + which argues early. + +## Relationship to RFC-002 + +RFC-002 remains the umbrella architecture. This RFC implements its §2 +(layered config, global-first), §4 (file naming / one dir), and §5 +(credentials) in their minimal load-bearing form, and explicitly defers §1 +(`GraphLocator`/targets), §3 (roles), and the State layer. If/when the +locator work resumes, it builds on these layers rather than re-landing +them. RFC-002's header should gain a pointer here once this merges. From 320311e75958569aba00b136647ab5f58cb7472a Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 19:33:19 +0300 Subject: [PATCH 120/165] =?UTF-8?q?docs(rfc):=20RFC-008=20=E2=80=94=20depr?= =?UTF-8?q?ecate=20omnigraph.yaml,=20one=20concern=20per=20config=20surfac?= =?UTF-8?q?e?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The file is three unrelated concerns wearing one filename — server deployment config, project/CLI conveniences, operator identity — and the mixture is the root cause of a recurring problem class (per-operator copies of project files, checkout-supplied credential redirection, init scaffold pollution). End state: two single-owner surfaces — cluster config (team, repo) and operator config (person, $HOME) — plus the zero-config flags/env tier. Complete key-by-key migration map over the verified OmnigraphConfig surface; staged retirement per the repo's Hyrum rules (warn with per-key guidance -> `config migrate` tool -> stop scaffolding -> opt-in strict -> removal at the next major). RFC-007's project-layer framing is amended to transitional accordingly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/index.md | 1 + docs/dev/rfc-007-operator-config.md | 12 +- docs/dev/rfc-008-deprecate-omnigraph-yaml.md | 174 +++++++++++++++++++ 3 files changed, 186 insertions(+), 1 deletion(-) create mode 100644 docs/dev/rfc-008-deprecate-omnigraph-yaml.md diff --git a/docs/dev/index.md b/docs/dev/index.md index 7e50777..b23326b 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -77,6 +77,7 @@ Working documents for in-flight feature work. Removed when the work lands. | Cluster graph & schema apply — Phase 4 sidecars, roll-forward recovery, approval artifacts | [rfc-004-cluster-graph-schema-apply.md](rfc-004-cluster-graph-schema-apply.md) | | Server boots from cluster state — Phase 5 mode switch, applied-revision serving | [rfc-005-server-cluster-boot.md](rfc-005-server-cluster-boot.md) | | Per-operator config — `~/.omnigraph/` identity, keyed credentials, named servers (the operator slice of RFC-002) | [rfc-007-operator-config.md](rfc-007-operator-config.md) | +| Deprecate `omnigraph.yaml` — one concern per config surface; key-by-key migration map and staged retirement | [rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md) | ## Boundary diff --git a/docs/dev/rfc-007-operator-config.md b/docs/dev/rfc-007-operator-config.md index 5c6b6b2..d2d9724 100644 --- a/docs/dev/rfc-007-operator-config.md +++ b/docs/dev/rfc-007-operator-config.md @@ -246,7 +246,17 @@ Three PRs, each independently useful, each landable without the next: version) — slice 1 or slice 2? It materially helps debugging precedence, which argues early. -## Relationship to RFC-002 +## Relationship to RFC-002 and RFC-008 + +**RFC-008 supersedes this RFC's "project layer" framing**: with +`omnigraph.yaml` deprecated +([rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md)), +the project layer *is* the cluster checkout. References to project +`omnigraph.yaml` in §D3/§D5 describe the transitional window only; the +trust-boundary rules apply unchanged to whatever the project layer is at a +given stage. Sequencing couples them: RFC-007 PRs 1–2 must land before +RFC-008's migration stages can begin (the operator layer is what keys +migrate *to*). RFC-002 remains the umbrella architecture. This RFC implements its §2 (layered config, global-first), §4 (file naming / one dir), and §5 diff --git a/docs/dev/rfc-008-deprecate-omnigraph-yaml.md b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md new file mode 100644 index 0000000..49e2c4b --- /dev/null +++ b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md @@ -0,0 +1,174 @@ +# RFC: Deprecate `omnigraph.yaml` — One Concern per Config Surface + +**Status:** Proposed +**Date:** 2026-06-11 +**Builds on:** [rfc-007-operator-config.md](rfc-007-operator-config.md) (the +operator layer that absorbs the identity/credential keys), +[rfc-005-server-cluster-boot.md](rfc-005-server-cluster-boot.md) (Landed — +cluster-booted serving), RFC-006 storage roots (landed: #186/#190/#194). +**Supersedes in part:** RFC-007's "project layer" framing (§Relationship +below) and [rfc-002-config-cli-architecture.md](rfc-002-config-cli-architecture.md)'s +assumption that `omnigraph.yaml` remains the project manifest. +**Target release:** staged; final removal at the next major (see Sequencing). + +## Summary + +Retire `omnigraph.yaml`. It is three unrelated concerns wearing one +filename — server deployment config, project/CLI conveniences, and operator +identity — and the mixture is not a cosmetic wart but the root cause of a +recurring class of problems: operators keeping personal copies of "project" +files, repo checkouts able to carry credential-adjacent keys (the #139 +security findings), `omnigraph init` scaffolding config into unrelated +directories, and every config discussion needing a paragraph to establish +which of the three files is meant. + +The end state is **two config surfaces with single owners**: + +| Surface | Owner | Declares | +|---|---|---| +| **Cluster config** (`cluster.yaml` + catalog) | the team, in a repo | what the system *is*: graphs, schemas, queries, policies, storage | +| **Operator config** (`~/.omnigraph/`) | one person, in `$HOME` | who *I* am: identity, credentials, known servers, ergonomics | + +plus **flags/env** for the zero-config tier (one graph, one server, no +control plane) — which already works today with no file at all. + +`omnigraph.yaml` has no role left once every key has a better home. This +RFC gives each key that home, and stages the retirement so that no working +setup breaks without a loud warning, a migration command, and a full +deprecation cycle first. + +## Motivation + +- **It breaks the ownership logic.** A config file must have one owner. A + file that carries `graphs:` (team-owned, reviewable) next to `cli.actor` + (one person's identity) and `auth.env_file` (credential loading) can be + neither safely committed nor sensibly personal. Every real deployment + this cycle tripped on it: per-operator copies in `~/exp/intel`, + graph-scoped alias URIs that only make sense per-person, the #139 + findings where a checkout-supplied file could redirect tokens. +- **The cluster made it redundant.** Since RFC-005/006, a cluster + deployment serves from the applied catalog — `--cluster` mode does not + read `omnigraph.yaml` *at all*. Stored queries, policies, bindings, and + graph addressing all have authoritative homes. What remains in + `omnigraph.yaml` for cluster users is dead weight that can silently + disagree with what is actually serving. +- **Two declarative dialects is one too many.** `cluster.yaml` and + `omnigraph.yaml` both declare graphs/queries/policies with different + schemas, different validation strictness, and different lifecycle + guarantees. Maintaining, documenting, and testing both — and explaining + when each applies — is a permanent tax (the "programming integrated over + time" lens says: this forks on every config-surface change). + +## Non-Goals + +- **Breaking anyone now.** Every `omnigraph.yaml` that works today keeps + working through the entire deprecation window, with warnings. +- **Retiring the zero-config tier.** `omnigraph-server s3://bucket/g.omni + --bind …` plus env vars stays first-class forever — that tier needs *no* + file, which is the point. +- **Forcing the control plane on single-graph users.** The migration target + for a multi-graph yaml deployment is a *minimal* cluster (file-rooted, + no bucket required, `cluster.yaml` barely longer than the `graphs:` map + it replaces) — but a single graph never needs even that. +- **Touching `cluster.yaml`** — its schema and strictness are unchanged. + +## Where every key goes (the complete migration map) + +The full `OmnigraphConfig` surface (verified against +`crates/omnigraph-server/src/config.rs:182-207`): + +| `omnigraph.yaml` key | Concern | New home | +|---|---|---| +| `graphs.<name>.uri` | what exists / where | `cluster.yaml` `graphs:` (storage-root-derived) — or a flag/env for the zero-config tier | +| `graphs.<name>.queries`, top-level `queries:` | what exists | cluster catalog (`.gq` discovery, RFC-004/#183) | +| `graphs.<name>.policy.file`, top-level `policy.file`, `server.policy.file` | what's enforced | `cluster.yaml` `policies:` + `applies_to` bindings | +| `server.bind` | deployment runtime | `--bind` / env (already authoritative; the key is a default) | +| `server.graph` | deployment runtime | `--target`-style flag / env in the zero-config tier; meaningless under cluster boot | +| `graphs.<name>.bearer_token_env`, `auth.env_file` | credentials | operator credentials chain (RFC-007 §D4) | +| `cli.actor` | identity | `operator.actor` (RFC-007 §D3) | +| `cli.output_format`, `cli.table_*` | personal ergonomics | `defaults:` in operator config (RFC-007 §D2) | +| `cli.graph`, `cli.branch` | personal targeting | operator config: named servers + a per-operator default target (RFC-007 PR 3) | +| `aliases.<name>` | personal ergonomics over shared queries | operator config `aliases:` — the *queries* they invoke are cluster-owned; the *shorthand* is personal | +| `query.roots` | discovery convenience | obsolete — cluster query discovery (#183) replaced it | +| `project.name` | label | dropped (the cluster's `metadata.name` is the deployment label) | + +Two placements worth defending: + +- **Aliases are operator config, not cluster config.** The stored query is + the shared contract (catalog-owned, digest-pinned); an alias is one + person's shorthand with their favorite default params and target. Putting + aliases in the cluster would force team review on personal ergonomics; + leaving them per-directory recreates today's problem. Per-operator, + keyed by server/graph name, is the AWS-profile shape. +- **Multi-graph serving without a control plane migrates to a minimal + cluster, not to a new file.** The honest cost: `cluster import` + `apply` + once, on a `file://` root next to the graphs. The honest benefit: one + declarative dialect, one validation path, one serving source — and the + upgrade path to buckets/approvals is a one-line `storage:` change instead + of a re-platform. + +## Deprecation mechanics + +Per Hyrum's Law (the repo's own deny-list: shipped observable behavior is +contract), retirement is staged, loud, and tooled: + +1. **Warn.** Loading `omnigraph.yaml` emits a one-line deprecation notice + naming the replacement for each key actually present in the file (not a + generic banner — the migration map above, applied to *your* file). + Suppressible per-process (`OMNIGRAPH_SUPPRESS_YAML_DEPRECATION=1`) for + CI logs during the window. +2. **Migrate.** `omnigraph config migrate` reads an existing + `omnigraph.yaml` and writes the split: the team half as a ready-to-review + `cluster.yaml` (+ moves query/policy files into the checkout layout), + the personal half merged into `~/.omnigraph/config.yaml` — printing a + diff-style summary and touching nothing without `--write`. The command + is the test of the migration map's completeness: any key it cannot + place is a bug in this RFC. +3. **Stop scaffolding.** `omnigraph init` stops generating + `omnigraph.yaml` (it currently scaffolds one into cwd — the source of + the test-pollution bug). `omnigraph cluster init` (new, small) scaffolds + a minimal `cluster.yaml` instead. +4. **Opt-in strict.** `OMNIGRAPH_NO_LEGACY_CONFIG=1` turns the warning into + an error — for teams that finished migrating and want regressions caught. +5. **Remove at the next major.** Loading the file becomes an error pointing + at `config migrate`. The `OmnigraphConfig` code path, the dual + query-registry loaders, and the yaml-mode server boot source are deleted + — the payoff that makes the whole exercise worth it. + +Stages 1–3 can land in one release once RFC-007 PRs 1–2 exist (the operator +layer must exist before anything can migrate *to* it). Stage 4 the release +after. Stage 5 at the major, with the removal listed in release notes from +stage 1 onward. + +## What this deletes, eventually + +- The `OmnigraphConfig` struct and its 12-key surface, the + `load_config`/`load_cli_config` pair and its env-side-effect, the + scaffolder, and the legacy resolution paths (`resolve_cli_graph`'s dual + modes — finding #11's root cause). +- The yaml-mode multi-graph server boot (`ServerConfigMode::Multi` keeps + existing — cluster boot constructs it — but its `omnigraph.yaml` source + goes). +- An entire class of documentation ("which file does X go in?") and the + #139 security surface (a checkout cannot hijack what no longer loads). + +## Relationship to RFC-007 and RFC-002 + +RFC-007 ships the operator layer this RFC migrates *to*; its "project +layer" language should be read as transitional — after this RFC, the +project layer **is** the cluster checkout, and RFC-007's PR 3 (project +`server:` references) applies to `cluster.yaml`-adjacent operator targeting +rather than to `omnigraph.yaml`. RFC-002's locator/state-layer work, if +resumed, targets the two-surface world directly. RFC-002's file-naming +decisions (`~/.omnigraph/` as the one dir) are unaffected. + +## Open questions + +- **Window length**: one minor release between warn (stage 1) and strict + (stage 4), or two? Cookbooks, skills, and the deployment docs all need + the same pass; the migration command makes a short window defensible. +- **`omnigraph login` vs `config migrate` ordering** — both write + `~/.omnigraph/`; whichever lands first establishes the file-locking and + atomic-write helpers the other reuses. +- **Does the MCP server config** (RFC-003) reference `omnigraph.yaml` + anywhere that needs the same treatment? To be audited in stage 1. From 08ce8dc34d4df48ac428e4db267f0f05fd26cf8a Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 19:54:34 +0300 Subject: [PATCH 121/165] docs(rfc): align RFC-007 with RFC-008's two-surface architecture MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit RFC-007 now speaks the end-state language throughout: the operator surface is one half of the two-surface split (cluster config / operator config), not a layer over a living omnigraph.yaml. The precedence cascade drops the project layer (cluster config carries no operator-resolvable keys — a checkout can never supply identity); legacy omnigraph.yaml appears only as the RFC-008 deprecation-window slot. The trust boundary is restated as closed-by-construction in the end state, with the rules governing the window. PR 3 becomes operator targeting (--server + operator aliases — the replacement RFC-008 needs before legacy aliases migrate), and the schema example gains the aliases block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/rfc-007-operator-config.md | 158 ++++++++++++++++------------ 1 file changed, 92 insertions(+), 66 deletions(-) diff --git a/docs/dev/rfc-007-operator-config.md b/docs/dev/rfc-007-operator-config.md index d2d9724..52a446e 100644 --- a/docs/dev/rfc-007-operator-config.md +++ b/docs/dev/rfc-007-operator-config.md @@ -3,27 +3,28 @@ **Status:** Proposed **Date:** 2026-06-11 **Builds on:** [rfc-002-config-cli-architecture.md](rfc-002-config-cli-architecture.md) (Proposed; implementation parked — PRs #139/#162 closed over review findings), [rfc-005-server-cluster-boot.md](rfc-005-server-cluster-boot.md) (Landed), RFC-006 storage roots (#186/#190/#194, landed). The #139 review record is a normative input: every design rule in §D6 traces to a confirmed finding. +**Paired with:** [rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md) — together they define the two-surface architecture this RFC's operator half belongs to. **Target release:** unversioned (staged; see Sequencing). ## Summary -Give OmniGraph the operator half of the Terraform config split. Terraform -separates `~/.terraformrc` (who I am, my credentials, my CLI behavior) from -the working directory's `*.tf` (what the project declares). OmniGraph today -has only the project half: `./omnigraph.yaml` in the current working -directory (or `--config <path>`), and nothing else — no home-level config, -no walk-up, no env override for the CLI. Operator identity and credentials -must be re-declared in every directory an operator works from, and — worse — -they end up in files that live next to repo-committed project config. +Give OmniGraph the operator half of the **two-surface config architecture** +(RFC-008): **cluster config** (team-owned, in a repo — what the system *is*) +and **operator config** (person-owned, in `$HOME` — who *I* am). This is +Terraform's split: `~/.terraformrc` for the operator, the checkout for the +declaration. OmniGraph today has neither half cleanly — `omnigraph.yaml` +mixes both concerns (RFC-008 retires it), and there is no home-level config +at all: identity and credentials get re-declared per working directory, in +files that sit next to repo-committed config. -This RFC introduces **`~/.omnigraph/config.yaml`** (the operator layer) and -a **keyed credentials chain**, scoped deliberately small: +This RFC introduces **`~/.omnigraph/config.yaml`** (the operator surface) +and a **keyed credentials chain**, scoped deliberately small: 1. **Operator identity** — a default actor for every `--as` cascade. 2. **Credentials by server name** — no more inventing env-var names per - server; secrets never inline, never in the project layer. -3. **Named servers** — operator-owned endpoint definitions that project - configs can reference but not redefine. + server; secrets never inline, never in any repo-committed file. +3. **Named servers** — operator-owned endpoint definitions; nothing a + checkout supplies can redefine them. It is explicitly a **subset of RFC-002**, sequenced to land. RFC-002 settled the right long-term decisions (one `~/.omnigraph/` dir, credentials keyed by @@ -63,11 +64,14 @@ Three concrete pains, all hit in real operation this cycle: that problem belongs to the slice that introduces it). - **OS keychain integration** — the credentials *chain* (§D4) leaves a slot for it; this RFC ships env + file sources only. -- **Project-file walk-up.** Terraform does not walk up from subdirectories +- **Config-file walk-up.** Terraform does not walk up from subdirectories and neither do we — `--config` (or running in the directory) stays the - explicit, deterministic story. Rejected, not deferred: walk-up makes "which - config am I using" a function of cwd depth, the class of surprise this RFC - exists to remove. + explicit, deterministic story for cluster checkouts. Rejected, not + deferred: walk-up makes "which config am I using" a function of cwd + depth, the class of surprise this RFC exists to remove. +- **Retiring `omnigraph.yaml`** — that is RFC-008's job, with its own + staging. This RFC builds the destination; during RFC-008's deprecation + window the legacy file keeps loading exactly as today. - **Renaming or removing anything.** No flag renames, no key renames, no schema-version bumps (findings #1, #3, #10). @@ -95,9 +99,10 @@ Three concrete pains, all hit in real operation this cycle: ### D1. Files and discovery ``` -~/.omnigraph/config.yaml # the operator layer (this RFC) +~/.omnigraph/config.yaml # the operator surface (this RFC) ~/.omnigraph/credentials # keyed secrets, 0600, git-irrelevant (§D4) -./omnigraph.yaml # the project layer (unchanged) +./cluster.yaml + checkout # the team surface (unchanged; RFC-004..006) +./omnigraph.yaml # legacy, loads as today through RFC-008's window ``` Discovery order for the operator file: `$OMNIGRAPH_HOME/config.yaml` if @@ -105,10 +110,12 @@ Discovery order for the operator file: `$OMNIGRAPH_HOME/config.yaml` if empty layer, never an error. `~` is expanded wherever paths are read (finding #9 — today a literal `./~/...` directory gets created). -`OMNIGRAPH_CONFIG=<path>` becomes a first-class override for the *project* -file in the CLI (highest precedence below the `--config` flag), aligning the +`OMNIGRAPH_CONFIG=<path>` becomes a first-class override for the `--config` +argument in the CLI (highest precedence below the flag itself), aligning the CLI with the container contract that already uses this variable for the -server. One name, one meaning, both binaries. +server. One name, one meaning, both binaries — it points at whatever the +command's `--config` would (a cluster checkout for cluster commands; the +legacy file during RFC-008's window). Per RFC-002 §4 (adopted verbatim): `~/.omnigraph/` is the one canonical dir — cache/state subdirectories arrive with their own slices; XDG roots are @@ -118,7 +125,7 @@ fallback read location if set, but is never written to). ### D2. The operator schema (v1 of this layer) ```yaml -# ~/.omnigraph/config.yaml — about the OPERATOR, never about a project +# ~/.omnigraph/config.yaml — about the OPERATOR, never about the system operator: actor: act-andrew # default for every --as cascade @@ -129,6 +136,12 @@ servers: # operator-owned endpoint definitions url: https://graph.modernrelay.ai # No token here, ever. Resolution: §D4. +aliases: # personal shorthand over CLUSTER-owned queries + triage: # (the query is the shared contract; the alias, + server: intel-dev # its defaults, and its name are mine — RFC-008) + graph: spike + query: weekly_triage + defaults: output: table # read --format default ``` @@ -140,27 +153,32 @@ change what a *plan* means). ### D3. Precedence and the merge rule +The end-state cascade is short, because the team surface (cluster config) +deliberately carries **no operator-resolvable keys** — no actor, no tokens, +no output preferences. Identity can never come from a checkout: + ``` -flag > env > project omnigraph.yaml > operator config > built-in +flag > env > operator config > built-in ``` -with exactly one principled inversion (§D5): **credentials and endpoint -definitions never come from the project layer when an operator-layer -definition exists for the same server name.** +During RFC-008's deprecation window, a legacy `omnigraph.yaml` slots in +between env and operator config (its keys win over operator defaults, +preserving today's behavior for unmigrated setups) — with the §D5 +credential inversion: **credentials and endpoint definitions never come +from a legacy/checkout file when an operator-layer definition exists for +the same server name.** Merging is **key-level**: scalars override per key; maps (`servers:`, -`graphs:`) merge per *entry*, and entries merge per *field* (finding #13 — -`merge_map` replacing whole entries silently dropped sibling fields). A -project file referencing `server: prod` composes with the operator's -`servers.prod.url`; it does not need to re-declare it and cannot -accidentally clobber half of it. +`aliases:`) merge per *entry*, and entries merge per *field* (finding #13 — +`merge_map` replacing whole entries silently dropped sibling fields). Concretely for the two flows this slice touches: -- **Actor**: `--as` > project `as:`/actor key (unchanged semantics) > - `operator.actor` > none (commands that need an actor keep failing loudly). -- **Output format**: `--format` > project default > `defaults.output` > - `table`. +- **Actor**: `--as` > legacy `cli.actor` (window only, unchanged semantics) + > `operator.actor` > none (commands that need an actor keep failing + loudly). +- **Output format**: `--format` > legacy default (window only) > + `defaults.output` > `table`. ### D4. Credentials: keyed by server name, by-reference always @@ -173,29 +191,33 @@ the same chain). For a server named `<name>`, the resolution chain is: 3. The legacy pair — `bearer_token_env` + `auth.env_file` — exactly as today, for configs that already use it. -No inline secrets in any YAML file, operator or project (the existing -invariant 12 posture extended to disk). A future `omnigraph login <name>` +No inline secrets in any YAML file, anywhere (the existing invariant 12 +posture extended to disk). A future `omnigraph login <name>` writes/rotates one section of the credentials file via temp + rename (finding #7: every operator-layer write is atomic), creating it `0600`. ### D5. The trust boundary (the security findings, made structural) -Findings #4, #5, #6 share one root cause: the project layer — a file that -arrives with a *repo checkout* — could redirect where requests go and what -secrets they carry. The rules: +Findings #4, #5, #6 share one root cause: a file that arrives with a +*repo checkout* could redirect where requests go and what secrets they +carry. In the end state this is closed by construction — cluster config has +no server/credential keys at all, and the operator surface never comes from +a checkout. The rules below therefore govern the **RFC-008 window** (while +legacy `omnigraph.yaml` still loads) and stand as the permanent law for any +future checkout-supplied surface: -1. **A project file may *reference* a server by name; it may not *redefine* - an operator-defined server.** If `./omnigraph.yaml` declares - `servers.prod.url` and `~/.omnigraph/config.yaml` also defines `prod`, - the operator definition wins and the CLI warns about the shadowed - project entry. A project-only server name keeps working (legacy compat), - but the keyed-credentials chain (§D4 steps 1–2) never resolves for it — - only the legacy explicit `bearer_token_env` does. Net effect: a malicious - checkout cannot point `prod` at an attacker host and harvest the - operator's `prod` token. -2. **`auth.env_file` keeps auto-loading (compat), but project-layer +1. **A checkout-supplied file may *reference* a server by name; it may not + *redefine* an operator-defined server.** If a legacy `./omnigraph.yaml` + declares `servers.prod.url` and `~/.omnigraph/config.yaml` also defines + `prod`, the operator definition wins and the CLI warns about the + shadowed entry. A legacy-only server name keeps working (compat), but + the keyed-credentials chain (§D4 steps 1–2) never resolves for it — + only the legacy explicit `bearer_token_env` does. Net effect: a + malicious checkout cannot point `prod` at an attacker host and harvest + the operator's `prod` token. +2. **`auth.env_file` keeps auto-loading (compat), but checkout-layer env-files cannot *override* variables already set in the process or by - the operator layer** — first-set-wins, operator-before-project (the + the operator layer** — first-set-wins, operator-before-checkout (the existing real-env-wins rule, extended one layer down). Finding #5's injection becomes a no-op against any var the operator actually uses. 3. **A token is sent only to the server it is keyed to.** The legacy @@ -223,16 +245,22 @@ Three PRs, each independently useful, each landable without the next: `~/.omnigraph/config.yaml` (+ `OMNIGRAPH_HOME`, `~`-expansion, warn-only unknown keys), `operator.actor` joining the `--as` cascade, `defaults.output` joining the format cascade, `OMNIGRAPH_CONFIG` env for - the CLI's project file. Docs: `cli-reference.md` gains the layer table. + the CLI's `--config`. Docs: `cli-reference.md` gains the two-surface + table. 2. **PR 2 — keyed credentials.** `servers:` in the operator layer, the §D4 chain (env + credentials file), the §D5 trust rules, and `omnigraph login <name>` (atomic write, `0600`). Legacy mechanisms untouched and tested-as-untouched. -3. **PR 3 — project references.** `server: <name>` in project - graph/target entries resolving through operator-defined servers, with - the shadowing warning. This is the *bridge* toward RFC-002's locator — - it gives multi-server addressing a safe, minimal form without the - `GraphLocator` rework. +3. **PR 3 — operator targeting.** `--server <name>` on remote-capable + commands and `aliases:` in the operator layer (server + graph + query + + default params), resolving through operator-defined servers. This is + the *bridge* toward RFC-002's locator — multi-server addressing in a + safe, minimal form without the `GraphLocator` rework — and the + replacement RFC-008 needs before legacy aliases can migrate. + +RFC-008's deprecation stages begin only after PRs 1–2 are on main: the +operator surface must exist before `config migrate` has somewhere to move +keys to. ## Open questions @@ -248,15 +276,13 @@ Three PRs, each independently useful, each landable without the next: ## Relationship to RFC-002 and RFC-008 -**RFC-008 supersedes this RFC's "project layer" framing**: with -`omnigraph.yaml` deprecated +**RFC-008 is the other half of this design**: this RFC builds the operator +surface; RFC-008 retires the mixed-ownership file ([rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md)), -the project layer *is* the cluster checkout. References to project -`omnigraph.yaml` in §D3/§D5 describe the transitional window only; the -trust-boundary rules apply unchanged to whatever the project layer is at a -given stage. Sequencing couples them: RFC-007 PRs 1–2 must land before -RFC-008's migration stages can begin (the operator layer is what keys -migrate *to*). +leaving exactly two config surfaces — cluster (team) and operator (person). +Every mention of `omnigraph.yaml` in this RFC describes the deprecation +window only. Sequencing couples them: RFC-007 PRs 1–2 land first, then +RFC-008's migration stages run against them. RFC-002 remains the umbrella architecture. This RFC implements its §2 (layered config, global-first), §4 (file naming / one dir), and §5 From be4bd46212de836668c545c16f0d2ce2bbd079da Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 20:29:02 +0300 Subject: [PATCH 122/165] =?UTF-8?q?feat(cli):=20the=20operator=20config=20?= =?UTF-8?q?surface=20=E2=80=94=20identity=20and=20output=20defaults=20(RFC?= =?UTF-8?q?-007=20PR=201)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ~/.omnigraph/config.yaml joins the resolution chains as the operator surface: operator.actor becomes the last hop of THE actor chain (--as > legacy cli.actor during the RFC-008 window > operator.actor > none, one implementation for direct-engine and cluster commands alike) and defaults.output joins the read-format cascade below every more-specific source. Discovery honors $OMNIGRAPH_HOME (tilde-expanded, #139 finding 9); an absent file is an empty layer; unknown keys WARN and load (a file written for later slices must not break this CLI); malformed YAML is a loud error. The module is CLI-only — the server never reads operator config (invariant 11 by construction). $OMNIGRAPH_CONFIG becomes a first-class stand-in for --config in load_config (flag > env > ./omnigraph.yaml), one meaning in both binaries. The test harness pins hermeticity: spawned binaries get a nonexistent OMNIGRAPH_HOME by default so no test ever reads the developer's real operator config. New coverage: loader unit tests, the env-precedence matrix on load_config_in, and spawned-binary e2es for the actor chain (operator wins with no flag/legacy key; legacy outranks it; --as wins) and the format cascade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/helpers.rs | 52 +++++- crates/omnigraph-cli/src/main.rs | 19 +- crates/omnigraph-cli/src/operator.rs | 212 ++++++++++++++++++++++ crates/omnigraph-cli/tests/cli_cluster.rs | 67 +++++++ crates/omnigraph-cli/tests/cli_data.rs | 46 +++++ crates/omnigraph-cli/tests/support/mod.rs | 16 +- crates/omnigraph-server/src/config.rs | 69 +++++-- 7 files changed, 445 insertions(+), 36 deletions(-) create mode 100644 crates/omnigraph-cli/src/operator.rs diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index be356a9..85eb42a 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -3,6 +3,7 @@ //! main.rs in the modularization). use super::*; +use crate::operator; pub(crate) fn ensure_local_graph_parent(uri: &str) -> Result<()> { if !uri.contains("://") { @@ -167,18 +168,40 @@ pub(crate) async fn open_local_db_with_policy(graph: &ResolvedCliGraph) -> Resul } } +/// THE actor chain (RFC-007 §D3) — every command that needs an identity +/// resolves through this one function (one path per concern): +/// `--as` > legacy `cli.actor` in omnigraph.yaml (RFC-008 window) > +/// `operator.actor` in ~/.omnigraph/config.yaml > none. +pub(crate) fn resolve_actor( + cli_as: Option<&str>, + legacy_config_actor: Option<&str>, +) -> Result<Option<String>> { + if let Some(actor) = cli_as { + return Ok(Some(actor.to_string())); + } + if let Some(actor) = legacy_config_actor { + return Ok(Some(actor.to_string())); + } + Ok(operator::load_operator_config()? + .actor() + .map(str::to_string)) +} + pub(crate) fn resolve_cluster_actor(cli_as: Option<&str>) -> Result<Option<String>> { if let Some(actor) = cli_as { return Ok(Some(actor.to_string())); } let config = load_config(None).wrap_err( - "resolving the default actor from the per-operator omnigraph.yaml (pass --as <ACTOR> to skip this lookup)", + "resolving the default actor from omnigraph.yaml (pass --as <ACTOR> to skip this lookup)", )?; - Ok(config.cli.actor.clone()) + resolve_actor(None, config.cli.actor.as_deref()) } -pub(crate) fn resolve_cli_actor<'a>(cli_as: Option<&'a str>, config: &'a OmnigraphConfig) -> Option<&'a str> { - cli_as.or(config.cli.actor.as_deref()) +pub(crate) fn resolve_cli_actor( + cli_as: Option<&str>, + config: &OmnigraphConfig, +) -> Result<Option<String>> { + resolve_actor(cli_as, config.cli.actor.as_deref()) } pub(crate) fn resolve_policy_tests_path(context: &ResolvedPolicyContext) -> PathBuf { @@ -460,6 +483,9 @@ pub(crate) fn merged_params_json( } } +/// The format cascade (RFC-007 §D3): `--json` > `--format` > alias format > +/// legacy `cli.output_format` (RFC-008 window) > operator `defaults.output` +/// > table. pub(crate) fn resolve_read_format( config: &OmnigraphConfig, cli_format: Option<ReadOutputFormat>, @@ -467,12 +493,17 @@ pub(crate) fn resolve_read_format( alias_format: Option<ReadOutputFormat>, ) -> ReadOutputFormat { if json { - ReadOutputFormat::Json - } else { - cli_format - .or(alias_format) - .unwrap_or_else(|| config.cli_output_format()) + return ReadOutputFormat::Json; } + cli_format + .or(alias_format) + .or(config.cli.output_format) + .or_else(|| { + operator::load_operator_config() + .ok() + .and_then(|operator| operator.output()) + }) + .unwrap_or_default() } pub(crate) fn resolve_alias<'a>( @@ -935,7 +966,8 @@ pub(crate) async fn execute_change( let (selected_name, query_params) = select_named_query(query_source, query_name)?; let params = query_params_from_json(&query_params, params_json)?; let db = open_local_db_with_policy(graph).await?; - let actor = resolve_cli_actor(cli_as_actor, config); + let actor = resolve_cli_actor(cli_as_actor, config)?; + let actor = actor.as_deref(); let result = db .mutate_as(branch, query_source, &selected_name, ¶ms, actor) .await?; diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 0d6ce03..bef111f 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -42,6 +42,7 @@ use serde::de::DeserializeOwned; use serde_json::Value; mod embed; +mod operator; mod read_format; use embed::{EmbedArgs, EmbedOutput, execute_embed}; @@ -129,7 +130,8 @@ async fn main() -> Result<()> { load_output_from_tables(&uri, &branch, mode, &output) } else { let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); + let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; + let actor = actor.as_deref(); let result = db .load_file_as( &branch, @@ -196,7 +198,8 @@ async fn main() -> Result<()> { .await? } else { let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); + let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; + let actor = actor.as_deref(); let result = db .load_file_as( &branch, @@ -243,7 +246,8 @@ async fn main() -> Result<()> { .await? } else { let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); + let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; + let actor = actor.as_deref(); db.branch_create_from_as(ReadTarget::branch(&from), &name, actor) .await?; BranchCreateOutput { @@ -316,7 +320,8 @@ async fn main() -> Result<()> { .await? } else { let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); + let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; + let actor = actor.as_deref(); db.branch_delete_as(&name, actor).await?; BranchDeleteOutput { uri: uri.clone(), @@ -358,7 +363,8 @@ async fn main() -> Result<()> { .await? } else { let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); + let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; + let actor = actor.as_deref(); let outcome = db.branch_merge_as(&source, &into, actor).await?; BranchMergeOutput { source: source.clone(), @@ -514,7 +520,8 @@ async fn main() -> Result<()> { .await? } else { let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config); + let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; + let actor = actor.as_deref(); let registry = load_registry_or_report(&config, graph.selected())?; let registry = (!registry.is_empty()).then_some(registry); let label = graph.selected().unwrap_or(&uri).to_string(); diff --git a/crates/omnigraph-cli/src/operator.rs b/crates/omnigraph-cli/src/operator.rs new file mode 100644 index 0000000..bac37b3 --- /dev/null +++ b/crates/omnigraph-cli/src/operator.rs @@ -0,0 +1,212 @@ +//! The operator config surface (RFC-007): `~/.omnigraph/config.yaml` — who +//! the operator IS (identity, ergonomics), never what the system is (that's +//! cluster config) and never a project file (nothing here arrives with a +//! repo checkout). +//! +//! PR-1 scope: `operator.actor` + `defaults.output`. Unknown keys WARN and +//! are preserved-by-ignoring — a file written for a newer CLI (servers, +//! aliases, credentials keys from later slices) must load cleanly on this +//! one. Contrast with `cluster.yaml`, where unknown keys are fatal because +//! they change what a plan means. +//! +//! This module is CLI-only by design: the server never reads operator +//! config (server-side identity comes from bearer auth — invariant 11 +//! holds by construction). + +use std::env; +use std::path::{Path, PathBuf}; + +use color_eyre::Result; +use color_eyre::eyre::eyre; +use serde::Deserialize; + +use omnigraph_server::config::ReadOutputFormat; + +pub(crate) const OPERATOR_HOME_ENV: &str = "OMNIGRAPH_HOME"; +pub(crate) const OPERATOR_DIR: &str = ".omnigraph"; +pub(crate) const OPERATOR_CONFIG_FILE: &str = "config.yaml"; + +#[derive(Debug, Default, Deserialize)] +pub(crate) struct OperatorConfig { + #[serde(default)] + pub(crate) operator: OperatorIdentity, + #[serde(default)] + pub(crate) defaults: OperatorDefaults, + /// Everything this CLI version doesn't know. Warned once at load, + /// otherwise ignored (forward compatibility within the operator layer). + #[serde(flatten)] + unknown: serde_yaml::Mapping, +} + +#[derive(Debug, Default, Deserialize)] +pub(crate) struct OperatorIdentity { + /// Default actor for every `--as` cascade (CLI direct-engine writes and + /// cluster commands alike): `--as` > legacy config actor (RFC-008 + /// window) > this > none. + pub(crate) actor: Option<String>, + #[serde(flatten)] + unknown: serde_yaml::Mapping, +} + +#[derive(Debug, Default, Deserialize)] +pub(crate) struct OperatorDefaults { + /// Default read output format, below every more-specific source. + pub(crate) output: Option<ReadOutputFormat>, + #[serde(flatten)] + unknown: serde_yaml::Mapping, +} + +impl OperatorConfig { + pub(crate) fn actor(&self) -> Option<&str> { + self.operator.actor.as_deref() + } + + pub(crate) fn output(&self) -> Option<ReadOutputFormat> { + self.defaults.output + } +} + +/// The operator dir: `$OMNIGRAPH_HOME` if set (tilde-expanded), else +/// `~/.omnigraph`. Returns None when no home directory is resolvable +/// (degenerate environments — the layer is simply absent). +pub(crate) fn operator_dir() -> Option<PathBuf> { + if let Some(home_override) = env::var_os(OPERATOR_HOME_ENV) { + let raw = home_override.to_string_lossy().into_owned(); + return Some(expand_tilde(&raw)); + } + env::home_dir().map(|home| home.join(OPERATOR_DIR)) +} + +/// Load the operator layer. Absent file (or unresolvable home) is an empty +/// layer, never an error; a present-but-malformed file is a loud error (the +/// operator owns it and can fix it); unknown keys warn to stderr once. +pub(crate) fn load_operator_config() -> Result<OperatorConfig> { + let Some(dir) = operator_dir() else { + return Ok(OperatorConfig::default()); + }; + load_operator_config_at(&dir.join(OPERATOR_CONFIG_FILE)) +} + +pub(crate) fn load_operator_config_at(path: &Path) -> Result<OperatorConfig> { + let text = match std::fs::read_to_string(path) { + Ok(text) => text, + Err(err) if err.kind() == std::io::ErrorKind::NotFound => { + return Ok(OperatorConfig::default()); + } + Err(err) => { + return Err(eyre!( + "could not read operator config '{}': {err}", + path.display() + )); + } + }; + let config: OperatorConfig = serde_yaml::from_str(&text).map_err(|err| { + eyre!( + "could not parse operator config '{}': {err}", + path.display() + ) + })?; + for warning in config.unknown_key_warnings() { + eprintln!("warning: {warning} in operator config '{}'", path.display()); + } + Ok(config) +} + +impl OperatorConfig { + fn unknown_key_warnings(&self) -> Vec<String> { + let mut warnings = Vec::new(); + let mut collect = |mapping: &serde_yaml::Mapping, prefix: &str| { + for key in mapping.keys() { + if let Some(name) = key.as_str() { + warnings.push(format!( + "unknown key `{prefix}{name}` (newer CLI feature or typo); ignored" + )); + } + } + }; + collect(&self.unknown, ""); + collect(&self.operator.unknown, "operator."); + collect(&self.defaults.unknown, "defaults."); + warnings + } +} + +/// Expand a leading `~` / `~/` to the home directory (PR #139 finding 9: +/// a literal `./~/…` path silently created a directory named `~`). +pub(crate) fn expand_tilde(raw: &str) -> PathBuf { + if raw == "~" { + return env::home_dir().unwrap_or_else(|| PathBuf::from(raw)); + } + if let Some(rest) = raw.strip_prefix("~/") { + if let Some(home) = env::home_dir() { + return home.join(rest); + } + } + PathBuf::from(raw) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::fs; + + #[test] + fn absent_file_is_an_empty_layer() { + let dir = tempfile::tempdir().unwrap(); + let config = load_operator_config_at(&dir.path().join("config.yaml")).unwrap(); + assert!(config.actor().is_none()); + assert!(config.output().is_none()); + } + + #[test] + fn parses_identity_and_defaults() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("config.yaml"); + fs::write( + &path, + "operator:\n actor: act-andrew\ndefaults:\n output: json\n", + ) + .unwrap(); + let config = load_operator_config_at(&path).unwrap(); + assert_eq!(config.actor(), Some("act-andrew")); + assert_eq!(config.output(), Some(ReadOutputFormat::Json)); + } + + #[test] + fn unknown_keys_warn_but_load() { + // A file written for a later slice (servers/aliases) must load + // cleanly today — warn-only forward compatibility. + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("config.yaml"); + fs::write( + &path, + "operator:\n actor: act-a\n color: green\nservers:\n prod:\n url: https://example.com\naliases: {}\n", + ) + .unwrap(); + let config = load_operator_config_at(&path).unwrap(); + assert_eq!(config.actor(), Some("act-a")); + let warnings = config.unknown_key_warnings(); + assert_eq!(warnings.len(), 3, "{warnings:?}"); + assert!(warnings.iter().any(|w| w.contains("`servers`"))); + assert!(warnings.iter().any(|w| w.contains("`aliases`"))); + assert!(warnings.iter().any(|w| w.contains("`operator.color`"))); + } + + #[test] + fn malformed_yaml_is_a_loud_error() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("config.yaml"); + fs::write(&path, "operator: [not, a, mapping\n").unwrap(); + let err = load_operator_config_at(&path).unwrap_err(); + assert!(err.to_string().contains("could not parse operator config")); + } + + #[test] + fn expand_tilde_resolves_home_prefix() { + let home = env::home_dir().unwrap(); + assert_eq!(expand_tilde("~"), home); + assert_eq!(expand_tilde("~/x/y"), home.join("x/y")); + assert_eq!(expand_tilde("/abs/path"), PathBuf::from("/abs/path")); + assert_eq!(expand_tilde("rel/path"), PathBuf::from("rel/path")); + } +} diff --git a/crates/omnigraph-cli/tests/cli_cluster.rs b/crates/omnigraph-cli/tests/cli_cluster.rs index be7675a..bfadf40 100644 --- a/crates/omnigraph-cli/tests/cli_cluster.rs +++ b/crates/omnigraph-cli/tests/cli_cluster.rs @@ -726,6 +726,73 @@ fn cluster_apply_uses_cli_actor_from_local_config() { assert_eq!(apply(&["--as", "andrew"]), "andrew", "--as overrides cli.actor"); } +/// RFC-007 PR 1: the operator layer joins the actor chain — +/// `--as` > legacy `cli.actor` (RFC-008 window) > `operator.actor` > none. +#[test] +fn cluster_apply_uses_operator_actor_from_omnigraph_home() { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + let operator_home = tempdir().unwrap(); + fs::write( + operator_home.path().join("config.yaml"), + "operator:\n actor: act-operator\n", + ) + .unwrap(); + + let output = cli() + .current_dir(temp.path()) + .env("OMNIGRAPH_HOME", operator_home.path()) + .arg("cluster") + .arg("import") + .arg("--config") + .arg(temp.path()) + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + + let apply = |extra: &[&str]| { + let mut command = cli(); + command + .current_dir(temp.path()) + .env("OMNIGRAPH_HOME", operator_home.path()); + for arg in extra { + command.arg(arg); + } + let output = command + .arg("cluster") + .arg("apply") + .arg("--config") + .arg(temp.path()) + .arg("--json") + .output() + .unwrap(); + let json: serde_json::Value = + serde_json::from_str(String::from_utf8_lossy(&output.stdout).trim()).unwrap(); + json["actor"].clone() + }; + + // No --as, no omnigraph.yaml: the operator identity applies. + assert_eq!( + apply(&[]), + "act-operator", + "operator.actor is the no-flag, no-legacy-config default" + ); + // --as still wins over everything. + assert_eq!(apply(&["--as", "andrew"]), "andrew"); + + // A legacy cli.actor (RFC-008 window) outranks the operator layer. + fs::write( + temp.path().join("omnigraph.yaml"), + "cli:\n actor: act-legacy\n", + ) + .unwrap(); + assert_eq!( + apply(&[]), + "act-legacy", + "legacy cli.actor wins over operator.actor during the deprecation window" + ); +} + #[test] fn cluster_approve_uses_cli_actor_fallback() { let temp = tempdir().unwrap(); diff --git a/crates/omnigraph-cli/tests/cli_data.rs b/crates/omnigraph-cli/tests/cli_data.rs index 841bedf..203a7c2 100644 --- a/crates/omnigraph-cli/tests/cli_data.rs +++ b/crates/omnigraph-cli/tests/cli_data.rs @@ -984,6 +984,52 @@ fn read_csv_format_outputs_header_and_row_values() { assert!(stdout.contains("Alice")); } +/// RFC-007 PR 1: the format cascade's operator hop — `defaults.output` in +/// ~/.omnigraph/config.yaml applies when nothing more specific is given, +/// and `--format` still wins over it. +#[test] +fn read_uses_operator_default_output_format() { + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + let operator_home = tempdir().unwrap(); + fs::write( + operator_home.path().join("config.yaml"), + "defaults:\n output: csv\n", + ) + .unwrap(); + + let read = |extra: &[&str]| { + let mut command = cli(); + command + .env("OMNIGRAPH_HOME", operator_home.path()) + .arg("read") + .arg(&graph) + .arg("--query") + .arg(fixture("test.gq")) + .arg("--name") + .arg("get_person") + .arg("--params") + .arg(r#"{"name":"Alice"}"#); + for arg in extra { + command.arg(arg); + } + stdout_string(&output_success(&mut command)) + }; + + let stdout = read(&[]); + assert!( + stdout.lines().next().unwrap().contains("p.name") && stdout.contains("Alice"), + "operator defaults.output: csv applies with no --format: {stdout}" + ); + let stdout = read(&["--format", "jsonl"]); + assert!( + stdout.starts_with('{'), + "--format wins over the operator default: {stdout}" + ); +} + #[test] fn read_jsonl_format_outputs_metadata_header_first() { let temp = tempdir().unwrap(); diff --git a/crates/omnigraph-cli/tests/support/mod.rs b/crates/omnigraph-cli/tests/support/mod.rs index 586bf93..41e46c7 100644 --- a/crates/omnigraph-cli/tests/support/mod.rs +++ b/crates/omnigraph-cli/tests/support/mod.rs @@ -12,12 +12,24 @@ use reqwest::blocking::Client; use serde_json::Value; use tempfile::{TempDir, tempdir}; +/// Hermetic default: point OMNIGRAPH_HOME at a path that exists on no +/// machine, so spawned binaries never read the developer's real +/// ~/.omnigraph/ (an absent operator config is an empty layer). Tests +/// exercising the operator layer override the var explicitly. +pub const HERMETIC_OPERATOR_HOME: &str = "/nonexistent/omnigraph-test-home"; + pub fn cli() -> Command { - Command::cargo_bin("omnigraph").unwrap() + let mut command = Command::cargo_bin("omnigraph").unwrap(); + command.env("OMNIGRAPH_HOME", HERMETIC_OPERATOR_HOME); + command.env_remove("OMNIGRAPH_CONFIG"); + command } pub fn cli_process() -> StdCommand { - StdCommand::new(assert_cmd::cargo::cargo_bin("omnigraph")) + let mut command = StdCommand::new(assert_cmd::cargo::cargo_bin("omnigraph")); + command.env("OMNIGRAPH_HOME", HERMETIC_OPERATOR_HOME); + command.env_remove("OMNIGRAPH_CONFIG"); + command } fn server_process() -> StdCommand { diff --git a/crates/omnigraph-server/src/config.rs b/crates/omnigraph-server/src/config.rs index b308b72..52bac2e 100644 --- a/crates/omnigraph-server/src/config.rs +++ b/crates/omnigraph-server/src/config.rs @@ -526,12 +526,23 @@ pub fn default_config_path() -> PathBuf { PathBuf::from(DEFAULT_CONFIG_FILE) } +/// `OMNIGRAPH_CONFIG` env var: a first-class stand-in for `--config`, one +/// name with one meaning in both binaries (the container entrypoint already +/// uses it for the server; RFC-007 §D1 extends it to the CLI). +pub const CONFIG_PATH_ENV: &str = "OMNIGRAPH_CONFIG"; + pub fn load_config(config_path: Option<&PathBuf>) -> Result<OmnigraphConfig> { - load_config_in(&env::current_dir()?, config_path) + let env_path = env::var_os(CONFIG_PATH_ENV).map(PathBuf::from); + load_config_in(&env::current_dir()?, config_path, env_path.as_ref()) } -fn load_config_in(cwd: &Path, config_path: Option<&PathBuf>) -> Result<OmnigraphConfig> { - let explicit_path = config_path.cloned(); +fn load_config_in( + cwd: &Path, + config_path: Option<&PathBuf>, + env_path: Option<&PathBuf>, +) -> Result<OmnigraphConfig> { + // Precedence: explicit --config flag > $OMNIGRAPH_CONFIG > ./omnigraph.yaml. + let explicit_path = config_path.or(env_path).cloned(); let config_path = explicit_path.or_else(|| { let default_path = cwd.join(DEFAULT_CONFIG_FILE); default_path.exists().then_some(default_path) @@ -575,6 +586,28 @@ mod tests { ReadOutputFormat, TableCellLayout, graph_resource_id_for_selection, load_config_in, }; + #[test] + fn env_config_path_stands_in_for_the_flag_but_loses_to_it() { + let temp = tempdir().unwrap(); + let flag_path = temp.path().join("flag.yaml"); + let env_path = temp.path().join("env.yaml"); + fs::write(&flag_path, "cli:\n actor: act-flag\n").unwrap(); + fs::write(&env_path, "cli:\n actor: act-env\n").unwrap(); + + // $OMNIGRAPH_CONFIG used when no flag… + let config = load_config_in(temp.path(), None, Some(&env_path)).unwrap(); + assert_eq!(config.cli.actor.as_deref(), Some("act-env")); + + // …loses to an explicit --config… + let config = load_config_in(temp.path(), Some(&flag_path), Some(&env_path)).unwrap(); + assert_eq!(config.cli.actor.as_deref(), Some("act-flag")); + + // …and beats the cwd default file. + fs::write(temp.path().join("omnigraph.yaml"), "cli:\n actor: act-cwd\n").unwrap(); + let config = load_config_in(temp.path(), None, Some(&env_path)).unwrap(); + assert_eq!(config.cli.actor.as_deref(), Some("act-env")); + } + #[test] fn load_config_reads_yaml_defaults_from_current_dir() { let temp = tempdir().unwrap(); @@ -598,7 +631,7 @@ policy: {} ) .unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); assert_eq!(config.cli_graph_name(), Some("local")); assert_eq!(config.cli_branch(), "main"); assert_eq!(config.cli_output_format(), ReadOutputFormat::Kv); @@ -633,7 +666,7 @@ policy: {} ) .unwrap(); - let config = load_config_in(&child, None).unwrap(); + let config = load_config_in(&child, None, None).unwrap(); assert!(config.graphs.is_empty()); } @@ -657,7 +690,7 @@ policy: {} "graphs:\n local:\n uri: ./demo.omni\n", ) .unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); // A known graph passes through unchanged. assert_eq!(config.resolve_graph_selection(Some("local")).unwrap(), Some("local")); @@ -680,7 +713,7 @@ policy: {} "graphs:\n local:\n uri: ./demo.omni\npolicy:\n file: ./top.yaml\n", ) .unwrap(); - let incoherent = load_config_in(temp2.path(), None).unwrap(); + let incoherent = load_config_in(temp2.path(), None, None).unwrap(); let err = incoherent .resolve_graph_selection(Some("local")) .unwrap_err() @@ -705,7 +738,7 @@ policy: {} server:\n graph: local\ncli:\n graph: prod\n", ) .unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); assert_eq!( config.resolve_policy_tooling_graph_selection().unwrap(), Some("prod") @@ -717,7 +750,7 @@ policy: {} "graphs:\n local:\n uri: ./local.omni\nserver:\n graph: local\n", ) .unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); assert_eq!( config.resolve_policy_tooling_graph_selection().unwrap(), Some("local") @@ -725,7 +758,7 @@ policy: {} let temp = tempdir().unwrap(); fs::write(temp.path().join("omnigraph.yaml"), "policy: {}\n").unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); assert_eq!(config.resolve_policy_tooling_graph_selection().unwrap(), None); let temp = tempdir().unwrap(); @@ -734,7 +767,7 @@ policy: {} "graphs:\n local:\n uri: ./local.omni\nserver:\n graph: ghost\n", ) .unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); let err = config .resolve_policy_tooling_graph_selection() .unwrap_err() @@ -760,7 +793,7 @@ policy: {} ) .unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); let resolved = config.resolve_query_path(Path::new("test.gq")).unwrap(); assert_eq!(resolved, temp.path().join("queries").join("test.gq")); } @@ -777,7 +810,7 @@ policy: {} fs::write(ambient_dir.join("local.gq"), "query ambient { return {} }").unwrap(); let config = - load_config_in(&ambient_dir, Some(&config_dir.join("omnigraph.yaml"))).unwrap(); + load_config_in(&ambient_dir, Some(&config_dir.join("omnigraph.yaml")), None).unwrap(); let resolved = config.resolve_query_path(Path::new("local.gq")).unwrap(); assert_eq!(resolved, config_dir.join("local.gq")); @@ -807,7 +840,7 @@ queries: ) .unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); // Per-graph registry (multi-graph mode). let prod = config.target_query_entries("prod").unwrap(); @@ -848,7 +881,7 @@ queries: policy:\n file: ./prod.yaml\n bare:\n uri: s3://b/bare\n", ) .unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); // Named graph with its own policy → per-graph (not top-level). assert!( @@ -884,7 +917,7 @@ queries: ) .unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); // Additive: no `queries:` anywhere → empty registries everywhere. assert!(config.query_entries().is_empty()); assert!( @@ -904,7 +937,7 @@ queries: ) .unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); assert_eq!( config.resolve_policy_file().unwrap(), temp.path().join("policy.yaml") @@ -927,7 +960,7 @@ cli: ) .unwrap(); - let config = load_config_in(temp.path(), None).unwrap(); + let config = load_config_in(temp.path(), None, None).unwrap(); assert_eq!( config.graph_bearer_token_env( Some("https://override.example.com"), From 9427fb510e8ec74303caaf253c6afd0148e35a90 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 20:32:04 +0300 Subject: [PATCH 123/165] docs(cli): the two config surfaces + the operator file reference cli-reference.md gains the config-surfaces table (cluster / operator / flags-env, with omnigraph.yaml marked as the legacy combined file per RFC-008) and the operator config.yaml reference; audit.md documents the unified actor chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/user/audit.md | 2 +- docs/user/cli-reference.md | 31 ++++++++++++++++++++++++++++++- 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/docs/user/audit.md b/docs/user/audit.md index 52cecde..845c2e0 100644 --- a/docs/user/audit.md +++ b/docs/user/audit.md @@ -3,5 +3,5 @@ - `Omnigraph::audit_actor_id: Option<String>` is the actor in effect. - `_as` variants of every write API let callers override the actor: `mutate_as`, `load_as`, `branch_merge_as`, `apply_schema_as`, etc. - Actor IDs are persisted on `GraphCommit.actor_id` with split storage in `_graph_commit_actors.lance` (the commit graph is split into `_graph_commits.lance` for the linkage and `_graph_commit_actors.lance` for the actor map). -- HTTP server uses the bearer-token actor automatically; CLI uses the local user / explicit env (no implicit actor). +- HTTP server uses the bearer-token actor automatically. The CLI resolves one actor chain everywhere: `--as` > legacy `cli.actor` in `omnigraph.yaml` > `operator.actor` in `~/.omnigraph/config.yaml` > none (RFC-007). - Pre-v0.4.0 graphs also stored actor IDs on `RunRecord.actor_id` in `_graph_runs.lance` / `_graph_run_actors.lance`. The Run state machine was removed in MR-771; those files are inert post-v0.4.0. The v2→v3 manifest migration sweeps any stale `__run__*` branches on first write-open (MR-770); the inert dataset bytes remain until a `delete_prefix` primitive lands. diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 74d772f..1dbc1ff 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -27,7 +27,36 @@ Top-level command families and subcommands. Graph-targeting commands accept eith | `policy validate \| test \| explain` | Cedar tooling. Selects `cli.graph`, else `server.graph`, else top-level `policy.file` | | `version` / `-v` | print `omnigraph 0.3.x` | -## `omnigraph.yaml` schema +## Config surfaces + +Two config surfaces with single owners (RFC-007/RFC-008), plus a zero-config +tier: + +| Surface | Owner | Location | Declares | +|---|---|---|---| +| Cluster config | the team, in a repo | `cluster.yaml` + checkout ([cluster-config.md](cluster-config.md)) | what the system **is**: graphs, schemas, queries, policies, storage | +| Operator config | one person | `~/.omnigraph/config.yaml` (override dir with `$OMNIGRAPH_HOME`) | who **I** am: identity, ergonomics | +| Flags / env | per invocation | — | everything, explicitly | + +`omnigraph.yaml` (below) is the legacy combined file — fully supported +today, slated for staged deprecation (RFC-008); its keys' future homes are +listed there. + +### `~/.omnigraph/config.yaml` (operator) + +```yaml +operator: + actor: act-andrew # default identity for every --as cascade: + # --as > legacy cli.actor > operator.actor > none +defaults: + output: table # read format default, below --json/--format/alias/legacy +``` + +Absent file = empty layer. Unknown keys warn and load (a file written for a +newer CLI works on an older one). `$OMNIGRAPH_CONFIG=<path>` stands in for +`--config` (the flag wins) in both the CLI and the server. + +## `omnigraph.yaml` schema (legacy combined file) ```yaml project: { name } From a819ab500e3c4e652089c27c40784e9e02e60fde Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 21:24:51 +0300 Subject: [PATCH 124/165] =?UTF-8?q?feat(cli):=20keyed=20credentials=20?= =?UTF-8?q?=E2=80=94=20servers:,=20the=20token=20chain,=20login/logout=20(?= =?UTF-8?q?RFC-007=20PR=202)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The operator config gains servers: (name -> url; never a token). A remote command whose URL prefix-matches an operator server resolves its bearer token through the keyed chain first — OMNIGRAPH_TOKEN_<NAME> env, then the [<name>] section of ~/.omnigraph/credentials (created 0600 via temp+rename, #139 finding 7; group/world-readable files refused loudly) — falling through to the legacy chain unchanged. URL keying makes §D5 rule 3 structural: a token is only ever sent to the server it is keyed to. Longest-prefix matching with a path-boundary check (http://h:8080 never matches http://h:8080-evil). Inserting the keyed hop above the legacy chain is safe by construction — no existing setup can have servers: defined. omnigraph login <name> stores/rotates one section (token from --token or one stdin line — the pipe flow keeps secrets out of shell history); omnigraph logout removes it, idempotently; logging in before declaring the server warns instead of failing (the gh model). Coverage: URL-match/no-substring-trap, credentials round-trip preserving sibling sections, 0600 write + over-permissive refusal, env-name mapping; the legacy resolve test is now hermetic against a real ~/.omnigraph and asserts byte-identical legacy behavior with no servers defined; one spawned-binary e2e walks the whole lifecycle against an authed server: refusal -> wrong-token login (stdin) -> rotate (--token) -> authorized read -> env-beats-file -> non-matching-URL negative -> logout revokes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/cli.rs | 22 ++ crates/omnigraph-cli/src/helpers.rs | 31 ++ crates/omnigraph-cli/src/main.rs | 23 ++ crates/omnigraph-cli/src/main_tests.rs | 11 + crates/omnigraph-cli/src/operator.rs | 324 ++++++++++++++++++++- crates/omnigraph-cli/src/output.rs | 45 +++ crates/omnigraph-cli/tests/support/mod.rs | 9 + crates/omnigraph-cli/tests/system_local.rs | 115 ++++++++ docs/dev/rfc-007-operator-config.md | 4 +- docs/user/cli-reference.md | 23 ++ 10 files changed, 603 insertions(+), 4 deletions(-) diff --git a/crates/omnigraph-cli/src/cli.rs b/crates/omnigraph-cli/src/cli.rs index 6b59559..7708c0a 100644 --- a/crates/omnigraph-cli/src/cli.rs +++ b/crates/omnigraph-cli/src/cli.rs @@ -29,6 +29,28 @@ pub(crate) struct Cli { pub(crate) enum Command { /// Print the CLI version Version, + /// Store a bearer token for a named server in ~/.omnigraph/credentials + /// (0600). Token from --token or one line on stdin: + /// `echo $TOKEN | omnigraph login prod`. The keyed token applies to + /// requests whose URL matches the server's `url` in the operator + /// config's `servers:` map. + Login { + /// Server name (keys the credential; declare its url under + /// `servers:` in ~/.omnigraph/config.yaml) + name: String, + /// The token. Prefer piping via stdin over this flag (shell + /// history). + #[arg(long)] + token: Option<String>, + #[arg(long)] + json: bool, + }, + /// Remove a named server's stored credential. Idempotent. + Logout { + name: String, + #[arg(long)] + json: bool, + }, /// Generate, clean, or refresh explicit seed embeddings Embed(EmbedArgs), /// Initialize a new graph from a schema diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index 85eb42a..b837192 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -221,6 +221,21 @@ pub(crate) fn resolve_remote_bearer_token( explicit_uri: Option<&str>, explicit_target: Option<&str>, ) -> Result<Option<String>> { + // The keyed hop (RFC-007 §D4, gh-host model): when the effective remote + // URL belongs to an operator-defined server, that server's keyed chain + // applies first — OMNIGRAPH_TOKEN_<NAME> env, then the 0600 credentials + // file. Ok(None) falls through to the legacy chain unchanged, and the + // keyed token is structurally scoped to its own server (§D5 rule 3): + // a URL matching no operator server never sees it. + if let Some(remote_url) = effective_remote_url(config, explicit_uri, explicit_target) { + let operator_config = operator::load_operator_config()?; + if let Some(server) = operator_config.find_server_for_url(&remote_url) { + if let Some(token) = operator::resolve_keyed_token(server)? { + return Ok(Some(token)); + } + } + } + let scoped_env = config.graph_bearer_token_env(explicit_uri, explicit_target, config.cli_graph_name()); let mut env_names = Vec::new(); @@ -249,6 +264,22 @@ pub(crate) fn resolve_remote_bearer_token( Ok(None) } +/// The remote base URL a token resolution is FOR — the same scoping +/// `graph_bearer_token_env` uses: an explicit http(s) `--uri` wins, else +/// the config-resolved target's uri (when remote). Local URIs → None. +fn effective_remote_url( + config: &OmnigraphConfig, + explicit_uri: Option<&str>, + explicit_target: Option<&str>, +) -> Option<String> { + if let Some(uri) = explicit_uri { + return is_remote_uri(uri).then(|| uri.to_string()); + } + let target = config.resolve_target_name(explicit_uri, explicit_target, config.cli_graph_name())?; + let uri = &config.graphs.get(target)?.uri; + is_remote_uri(uri).then(|| uri.clone()) +} + pub(crate) fn build_http_client() -> Result<reqwest::Client> { Ok(reqwest::Client::new()) } diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index bef111f..85fe537 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -73,6 +73,29 @@ async fn main() -> Result<()> { }; let http_client = build_http_client()?; match cli.command { + Command::Login { name, token, json } => { + let token = match token { + Some(token) => token, + None => { + let mut line = String::new(); + std::io::stdin().read_line(&mut line)?; + line + } + }; + let Some(token) = normalize_bearer_token(Some(token)) else { + color_eyre::eyre::bail!( + "no token provided: pass --token <TOKEN> or pipe it on stdin (echo $TOKEN | omnigraph login {name})" + ); + }; + let operator_config = crate::operator::load_operator_config()?; + let declared = operator_config.servers.contains_key(&name); + let path = crate::operator::write_credential(&name, &token)?; + finish_login(&name, &path, declared, json)?; + } + Command::Logout { name, json } => { + let path = crate::operator::remove_credential(&name)?; + finish_logout(&name, &path, json)?; + } Command::Version => { println!("omnigraph {}", env!("CARGO_PKG_VERSION")); } diff --git a/crates/omnigraph-cli/src/main_tests.rs b/crates/omnigraph-cli/src/main_tests.rs index 0bbb593..8380c36 100644 --- a/crates/omnigraph-cli/src/main_tests.rs +++ b/crates/omnigraph-cli/src/main_tests.rs @@ -195,8 +195,14 @@ cli: .unwrap(); let previous = std::env::var_os(DEFAULT_BEARER_TOKEN_ENV); + let previous_home = std::env::var_os("OMNIGRAPH_HOME"); unsafe { std::env::remove_var(DEFAULT_BEARER_TOKEN_ENV); + // Hermetic: the keyed hop (RFC-007 PR 2) must not pick up a real + // ~/.omnigraph on the developer's machine — and with no operator + // servers defined, the legacy chain below must behave + // byte-identically to pre-PR-2 (tested-as-untouched). + std::env::set_var("OMNIGRAPH_HOME", temp.path().join("no-operator-config")); } let config_path = temp.path().join("omnigraph.yaml"); @@ -221,6 +227,11 @@ cli: } else { std::env::remove_var(DEFAULT_BEARER_TOKEN_ENV); } + if let Some(value) = previous_home { + std::env::set_var("OMNIGRAPH_HOME", value); + } else { + std::env::remove_var("OMNIGRAPH_HOME"); + } } } diff --git a/crates/omnigraph-cli/src/operator.rs b/crates/omnigraph-cli/src/operator.rs index bac37b3..1b95e24 100644 --- a/crates/omnigraph-cli/src/operator.rs +++ b/crates/omnigraph-cli/src/operator.rs @@ -13,6 +13,7 @@ //! config (server-side identity comes from bearer auth — invariant 11 //! holds by construction). +use std::collections::BTreeMap; use std::env; use std::path::{Path, PathBuf}; @@ -32,12 +33,24 @@ pub(crate) struct OperatorConfig { pub(crate) operator: OperatorIdentity, #[serde(default)] pub(crate) defaults: OperatorDefaults, + /// Operator-owned endpoint definitions (RFC-007 §D2/§D4): name → url. + /// The name keys the credential chain; nothing a repo checkout supplies + /// can redefine an entry here. No tokens in this file, ever. + #[serde(default)] + pub(crate) servers: BTreeMap<String, OperatorServer>, /// Everything this CLI version doesn't know. Warned once at load, /// otherwise ignored (forward compatibility within the operator layer). #[serde(flatten)] unknown: serde_yaml::Mapping, } +#[derive(Debug, Deserialize)] +pub(crate) struct OperatorServer { + pub(crate) url: String, + #[serde(flatten)] + unknown: serde_yaml::Mapping, +} + #[derive(Debug, Default, Deserialize)] pub(crate) struct OperatorIdentity { /// Default actor for every `--as` cascade (CLI direct-engine writes and @@ -64,6 +77,26 @@ impl OperatorConfig { pub(crate) fn output(&self) -> Option<ReadOutputFormat> { self.defaults.output } + + /// The gh-host model: which operator server (if any) does this request + /// URL belong to? Longest-prefix match after trailing-slash + /// normalization, so `url: http://h:8080` matches + /// `http://h:8080/graphs/spike` but never `http://h:8080-evil`. + pub(crate) fn find_server_for_url(&self, request_url: &str) -> Option<&str> { + let request = request_url.trim_end_matches('/'); + let mut best: Option<(&str, usize)> = None; + for (name, server) in &self.servers { + let base = server.url.trim_end_matches('/'); + let matches = request == base + || request + .strip_prefix(base) + .is_some_and(|rest| rest.starts_with('/')); + if matches && best.is_none_or(|(_, len)| base.len() > len) { + best = Some((name, base.len())); + } + } + best.map(|(name, _)| name) + } } /// The operator dir: `$OMNIGRAPH_HOME` if set (tilde-expanded), else @@ -127,10 +160,216 @@ impl OperatorConfig { collect(&self.unknown, ""); collect(&self.operator.unknown, "operator."); collect(&self.defaults.unknown, "defaults."); + for (name, server) in &self.servers { + collect(&server.unknown, &format!("servers.{name}.")); + } warnings } } +// ---- keyed credentials (RFC-007 §D4) ---- + +pub(crate) const CREDENTIALS_FILE: &str = "credentials"; +const TOKEN_ENV_PREFIX: &str = "OMNIGRAPH_TOKEN_"; + +pub(crate) fn credentials_path() -> Option<PathBuf> { + operator_dir().map(|dir| dir.join(CREDENTIALS_FILE)) +} + +/// `intel-dev` → `OMNIGRAPH_TOKEN_INTEL_DEV`. +pub(crate) fn token_env_name(server: &str) -> String { + let mut name = String::from(TOKEN_ENV_PREFIX); + for c in server.chars() { + name.push(match c { + '-' => '_', + other => other.to_ascii_uppercase(), + }); + } + name +} + +/// The keyed token chain for a named server (§D4 steps 1–2): +/// `OMNIGRAPH_TOKEN_<NAME>` env → `[<name>]` in the credentials file. +/// `Ok(None)` means "no keyed token" — callers fall through to the legacy +/// chain; a present-but-unreadable/over-permissive credentials file is a +/// loud error, never a silent skip. +pub(crate) fn resolve_keyed_token(server: &str) -> Result<Option<String>> { + if let Ok(token) = env::var(token_env_name(server)) { + let token = token.trim(); + if !token.is_empty() { + return Ok(Some(token.to_string())); + } + } + let Some(path) = credentials_path() else { + return Ok(None); + }; + read_credential_at(&path, server) +} + +pub(crate) fn read_credential_at(path: &Path, server: &str) -> Result<Option<String>> { + let text = match std::fs::read_to_string(path) { + Ok(text) => text, + Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(err) => { + return Err(eyre!( + "could not read credentials file '{}': {err}", + path.display() + )); + } + }; + refuse_over_permissive(path)?; + let mut in_section = false; + for line in text.lines() { + let line = line.trim(); + if line.is_empty() || line.starts_with('#') { + continue; + } + if let Some(section) = line.strip_prefix('[').and_then(|l| l.strip_suffix(']')) { + in_section = section.trim() == server; + continue; + } + if in_section { + if let Some((key, value)) = line.split_once('=') { + if key.trim() == "token" { + let value = unquote(value.trim()); + if value.is_empty() { + return Ok(None); + } + return Ok(Some(value.to_string())); + } + } + } + } + Ok(None) +} + +/// Write (or rotate) one server's token, preserving every other section. +/// Temp file + rename (#139 finding 7), created 0600. +pub(crate) fn write_credential(server: &str, token: &str) -> Result<PathBuf> { + let path = credentials_path() + .ok_or_else(|| eyre!("no home directory resolvable for the credentials file"))?; + rewrite_credentials_at(&path, server, Some(token))?; + Ok(path) +} + +/// Remove one server's section. Idempotent: absent file or section is fine. +pub(crate) fn remove_credential(server: &str) -> Result<PathBuf> { + let path = credentials_path() + .ok_or_else(|| eyre!("no home directory resolvable for the credentials file"))?; + rewrite_credentials_at(&path, server, None)?; + Ok(path) +} + +pub(crate) fn rewrite_credentials_at( + path: &Path, + server: &str, + token: Option<&str>, +) -> Result<()> { + let existing = match std::fs::read_to_string(path) { + Ok(text) => { + refuse_over_permissive(path)?; + text + } + Err(err) if err.kind() == std::io::ErrorKind::NotFound => String::new(), + Err(err) => { + return Err(eyre!( + "could not read credentials file '{}': {err}", + path.display() + )); + } + }; + + // Drop the target section (if present), keep everything else verbatim. + let mut out = String::new(); + let mut in_target = false; + for line in existing.lines() { + let trimmed = line.trim(); + if let Some(section) = trimmed.strip_prefix('[').and_then(|l| l.strip_suffix(']')) { + in_target = section.trim() == server; + if in_target { + continue; + } + } + if !in_target { + out.push_str(line); + out.push('\n'); + } + } + if let Some(token) = token { + if !out.is_empty() && !out.ends_with("\n\n") { + out.push('\n'); + } + out.push_str(&format!("[{server}]\ntoken = {token}\n")); + } + + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent)?; + } + let tmp = path.with_extension(format!("tmp.{}", std::process::id())); + write_owner_only(&tmp, &out)?; + std::fs::rename(&tmp, path).map_err(|err| { + let _ = std::fs::remove_file(&tmp); + eyre!( + "could not move credentials file into place '{}': {err}", + path.display() + ) + })?; + Ok(()) +} + +#[cfg(unix)] +fn write_owner_only(path: &Path, content: &str) -> Result<()> { + use std::io::Write; + use std::os::unix::fs::OpenOptionsExt; + let mut file = std::fs::OpenOptions::new() + .write(true) + .create(true) + .truncate(true) + .mode(0o600) + .open(path)?; + file.write_all(content.as_bytes())?; + Ok(()) +} + +#[cfg(not(unix))] +fn write_owner_only(path: &Path, content: &str) -> Result<()> { + std::fs::write(path, content)?; + Ok(()) +} + +/// Secrets are operator-private: refuse a credentials file other accounts +/// can read (the chain errs loudly rather than using a leaked secret). +#[cfg(unix)] +fn refuse_over_permissive(path: &Path) -> Result<()> { + use std::os::unix::fs::PermissionsExt; + let mode = std::fs::metadata(path)?.permissions().mode(); + if mode & 0o077 != 0 { + return Err(eyre!( + "credentials file '{}' is group/world-accessible (mode {:o}); run `chmod 600 {}`", + path.display(), + mode & 0o777, + path.display() + )); + } + Ok(()) +} + +#[cfg(not(unix))] +fn refuse_over_permissive(_path: &Path) -> Result<()> { + Ok(()) +} + +fn unquote(value: &str) -> &str { + if value.len() >= 2 + && ((value.starts_with('"') && value.ends_with('"')) + || (value.starts_with('\'') && value.ends_with('\''))) + { + &value[1..value.len() - 1] + } else { + value + } +} + /// Expand a leading `~` / `~/` to the home directory (PR #139 finding 9: /// a literal `./~/…` path silently created a directory named `~`). pub(crate) fn expand_tilde(raw: &str) -> PathBuf { @@ -186,10 +425,12 @@ mod tests { let config = load_operator_config_at(&path).unwrap(); assert_eq!(config.actor(), Some("act-a")); let warnings = config.unknown_key_warnings(); - assert_eq!(warnings.len(), 3, "{warnings:?}"); - assert!(warnings.iter().any(|w| w.contains("`servers`"))); + // `servers` became a known key in PR 2; `aliases` stays unknown + // until PR 3. + assert_eq!(warnings.len(), 2, "{warnings:?}"); assert!(warnings.iter().any(|w| w.contains("`aliases`"))); assert!(warnings.iter().any(|w| w.contains("`operator.color`"))); + assert_eq!(config.servers["prod"].url, "https://example.com"); } #[test] @@ -201,6 +442,85 @@ mod tests { assert!(err.to_string().contains("could not parse operator config")); } + #[test] + fn find_server_for_url_longest_prefix_no_substring_traps() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("config.yaml"); + fs::write( + &path, + "servers:\n dev:\n url: http://h:8080\n dev-spike:\n url: http://h:8080/graphs/spike\n", + ) + .unwrap(); + let config = load_operator_config_at(&path).unwrap(); + assert_eq!(config.find_server_for_url("http://h:8080"), Some("dev")); + assert_eq!( + config.find_server_for_url("http://h:8080/graphs/other"), + Some("dev") + ); + // longest prefix wins + assert_eq!( + config.find_server_for_url("http://h:8080/graphs/spike/queries/q"), + Some("dev-spike") + ); + // no substring trap: a different port/host must not match + assert_eq!(config.find_server_for_url("http://h:8080-evil/x"), None); + assert_eq!(config.find_server_for_url("http://other:9999"), None); + } + + #[test] + fn token_env_name_uppercases_and_underscores() { + assert_eq!(token_env_name("intel-dev"), "OMNIGRAPH_TOKEN_INTEL_DEV"); + assert_eq!(token_env_name("prod"), "OMNIGRAPH_TOKEN_PROD"); + } + + #[test] + fn credentials_roundtrip_rotate_remove_preserving_other_sections() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("credentials"); + + rewrite_credentials_at(&path, "prod", Some("tok-1")).unwrap(); + rewrite_credentials_at(&path, "dev", Some("tok-dev")).unwrap(); + assert_eq!( + read_credential_at(&path, "prod").unwrap().as_deref(), + Some("tok-1") + ); + + // rotate prod; dev preserved + rewrite_credentials_at(&path, "prod", Some("tok-2")).unwrap(); + assert_eq!( + read_credential_at(&path, "prod").unwrap().as_deref(), + Some("tok-2") + ); + assert_eq!( + read_credential_at(&path, "dev").unwrap().as_deref(), + Some("tok-dev") + ); + + // remove prod; dev preserved; removal is idempotent + rewrite_credentials_at(&path, "prod", None).unwrap(); + rewrite_credentials_at(&path, "prod", None).unwrap(); + assert_eq!(read_credential_at(&path, "prod").unwrap(), None); + assert_eq!( + read_credential_at(&path, "dev").unwrap().as_deref(), + Some("tok-dev") + ); + } + + #[cfg(unix)] + #[test] + fn credentials_written_0600_and_over_permissive_refused() { + use std::os::unix::fs::PermissionsExt; + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("credentials"); + rewrite_credentials_at(&path, "prod", Some("tok")).unwrap(); + let mode = fs::metadata(&path).unwrap().permissions().mode(); + assert_eq!(mode & 0o777, 0o600, "written {:o}", mode & 0o777); + + fs::set_permissions(&path, fs::Permissions::from_mode(0o644)).unwrap(); + let err = read_credential_at(&path, "prod").unwrap_err(); + assert!(err.to_string().contains("chmod 600"), "{err}"); + } + #[test] fn expand_tilde_resolves_home_prefix() { let home = env::home_dir().unwrap(); diff --git a/crates/omnigraph-cli/src/output.rs b/crates/omnigraph-cli/src/output.rs index f77e50f..04df60a 100644 --- a/crates/omnigraph-cli/src/output.rs +++ b/crates/omnigraph-cli/src/output.rs @@ -828,3 +828,48 @@ pub(crate) struct QueriesListItem { pub(crate) struct QueriesListOutput { pub(crate) queries: Vec<QueriesListItem>, } + +pub(crate) fn finish_login( + server: &str, + credentials_path: &std::path::Path, + declared: bool, + json: bool, +) -> Result<()> { + if json { + print_json(&serde_json::json!({ + "server": server, + "credentials_path": credentials_path.display().to_string(), + "declared": declared, + }))?; + } else { + println!( + "stored credential for '{server}' in {}", + credentials_path.display() + ); + } + if !declared { + eprintln!( + "note: '{server}' is not declared under servers: in the operator config; the token applies once you add `servers:\n {server}:\n url: <server url>` to ~/.omnigraph/config.yaml" + ); + } + Ok(()) +} + +pub(crate) fn finish_logout( + server: &str, + credentials_path: &std::path::Path, + json: bool, +) -> Result<()> { + if json { + print_json(&serde_json::json!({ + "server": server, + "credentials_path": credentials_path.display().to_string(), + }))?; + } else { + println!( + "removed credential for '{server}' from {}", + credentials_path.display() + ); + } + Ok(()) +} diff --git a/crates/omnigraph-cli/tests/support/mod.rs b/crates/omnigraph-cli/tests/support/mod.rs index 41e46c7..b11e94d 100644 --- a/crates/omnigraph-cli/tests/support/mod.rs +++ b/crates/omnigraph-cli/tests/support/mod.rs @@ -259,6 +259,15 @@ pub fn spawn_server_with_cluster_env(cluster_dir: &Path, envs: &[(&str, &str)]) spawn_server_process(command) } +pub fn spawn_server_with_env(graph: &Path, envs: &[(&str, &str)]) -> TestServer { + let mut command = server_process(); + command.arg(graph); + for (name, value) in envs { + command.env(name, value); + } + spawn_server_process(command) +} + pub fn spawn_server_with_config_env(config: &Path, envs: &[(&str, &str)]) -> TestServer { let mut command = server_process(); command.arg("--config").arg(config); diff --git a/crates/omnigraph-cli/tests/system_local.rs b/crates/omnigraph-cli/tests/system_local.rs index 46f6fcf..5eb739f 100644 --- a/crates/omnigraph-cli/tests/system_local.rs +++ b/crates/omnigraph-cli/tests/system_local.rs @@ -2309,3 +2309,118 @@ fn cluster_server_boot_ignores_local_config_in_cwd() { let response = reqwest::blocking::get(format!("{}/healthz", server.base_url)).unwrap(); assert!(response.status().is_success()); } + +/// RFC-007 PR 2: keyed credentials end to end — `login` stores a 0600 +/// credential, the URL-matched server's token chain authenticates remote +/// reads (env > file), a non-matching URL never sees the token (§D5 rule +/// 3), and `logout` revokes. +#[test] +fn local_cli_keyed_credentials_authenticate_url_matched_server() { + let graph = SystemGraph::loaded(); + let server = spawn_server_with_env( + graph.path(), + &[("OMNIGRAPH_SERVER_BEARER_TOKEN", "secret-tok")], + ); + let operator_home = tempfile::tempdir().unwrap(); + let write_server_url = |url: &str| { + fs::write( + operator_home.path().join("config.yaml"), + format!("servers:\n test-srv:\n url: {url}\n"), + ) + .unwrap(); + }; + write_server_url(&server.base_url); + + let remote_read = |envs: &[(&str, &str)]| { + let mut command = cli(); + command.env("OMNIGRAPH_HOME", operator_home.path()); + for (name, value) in envs { + command.env(name, value); + } + command + .arg("read") + .arg(&server.base_url) + .arg("--query") + .arg(fixture("test.gq")) + .arg("--name") + .arg("get_person") + .arg("--params") + .arg(r#"{"name":"Alice"}"#) + .arg("--json") + .output() + .unwrap() + }; + + // No credential anywhere: the server refuses. + let output = remote_read(&[]); + assert!(!output.status.success(), "{output:?}"); + + // login with a WRONG token (via stdin, the documented pipe flow). + let output = cli() + .env("OMNIGRAPH_HOME", operator_home.path()) + .arg("login") + .arg("test-srv") + .write_stdin("wrong-tok\n") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + let output = remote_read(&[]); + assert!(!output.status.success(), "wrong token must not authenticate"); + + // Re-login rotates to the right token (via --token); 0600 on disk. + let output = cli() + .env("OMNIGRAPH_HOME", operator_home.path()) + .arg("login") + .arg("test-srv") + .arg("--token") + .arg("secret-tok") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + let credentials = operator_home.path().join("credentials"); + let text = fs::read_to_string(&credentials).unwrap(); + assert!(text.contains("[test-srv]"), "{text}"); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = fs::metadata(&credentials).unwrap().permissions().mode(); + assert_eq!(mode & 0o777, 0o600, "{:o}", mode & 0o777); + } + let output = remote_read(&[]); + assert!( + output.status.success(), + "keyed credential must authenticate the URL-matched server: {output:?}" + ); + let payload: serde_json::Value = + serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["rows"][0]["p.name"], "Alice"); + + // OMNIGRAPH_TOKEN_<NAME> env outranks the credentials file. + let output = remote_read(&[("OMNIGRAPH_TOKEN_TEST_SRV", "env-wrong")]); + assert!( + !output.status.success(), + "keyed env token must outrank the credentials file" + ); + + // §D5 rule 3: a URL matching no operator server never sees the token. + write_server_url("http://127.0.0.1:1"); + let output = remote_read(&[]); + assert!( + !output.status.success(), + "token keyed to another url must not be sent here" + ); + write_server_url(&server.base_url); + + // logout revokes; idempotent. + for _ in 0..2 { + let output = cli() + .env("OMNIGRAPH_HOME", operator_home.path()) + .arg("logout") + .arg("test-srv") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + } + let output = remote_read(&[]); + assert!(!output.status.success(), "logout must revoke access"); +} diff --git a/docs/dev/rfc-007-operator-config.md b/docs/dev/rfc-007-operator-config.md index 52a446e..5abf4e1 100644 --- a/docs/dev/rfc-007-operator-config.md +++ b/docs/dev/rfc-007-operator-config.md @@ -241,13 +241,13 @@ future checkout-supplied surface: Three PRs, each independently useful, each landable without the next: -1. **PR 1 — the operator file + identity.** Loader for +1. **PR 1 — the operator file + identity** *(landed: #196)*. Loader for `~/.omnigraph/config.yaml` (+ `OMNIGRAPH_HOME`, `~`-expansion, warn-only unknown keys), `operator.actor` joining the `--as` cascade, `defaults.output` joining the format cascade, `OMNIGRAPH_CONFIG` env for the CLI's `--config`. Docs: `cli-reference.md` gains the two-surface table. -2. **PR 2 — keyed credentials.** `servers:` in the operator layer, the +2. **PR 2 — keyed credentials** *(landed)*. `servers:` in the operator layer, the §D4 chain (env + credentials file), the §D5 trust rules, and `omnigraph login <name>` (atomic write, `0600`). Legacy mechanisms untouched and tested-as-untouched. diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 1dbc1ff..c41a15c 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -48,6 +48,9 @@ listed there. operator: actor: act-andrew # default identity for every --as cascade: # --as > legacy cli.actor > operator.actor > none +servers: # operator-owned endpoints; names key the credentials + prod: + url: https://graph.example.com # no tokens in this file, ever defaults: output: table # read format default, below --json/--format/alias/legacy ``` @@ -56,6 +59,26 @@ Absent file = empty layer. Unknown keys warn and load (a file written for a newer CLI works on an older one). `$OMNIGRAPH_CONFIG=<path>` stands in for `--config` (the flag wins) in both the CLI and the server. +#### Credentials keyed by server name + +`omnigraph login <name>` stores a bearer token in +`~/.omnigraph/credentials` (created `0600`; group/world-readable files are +refused). Token from `--token`, or — preferred, keeps it out of shell +history — one line on stdin: `echo $TOKEN | omnigraph login prod`. +`omnigraph logout <name>` removes it (idempotent). + +A remote command whose URL prefix-matches an operator server's `url` (the +`gh` host model — no flags needed) resolves its token through: + +| Order | Source | +|---|---| +| 1 | `OMNIGRAPH_TOKEN_<NAME>` env (`prod` → `OMNIGRAPH_TOKEN_PROD`) | +| 2 | `[<name>]` section in `~/.omnigraph/credentials` | +| 3 | the legacy chain unchanged (`bearer_token_env` → `OMNIGRAPH_BEARER_TOKEN` → `auth.env_file`) | + +A token is only ever sent to the server it is keyed to: URLs matching no +operator server use the legacy chain alone. + ## `omnigraph.yaml` schema (legacy combined file) ```yaml From 65160cc060c17052bc4bcaa78c28216c2ccf345c Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 22:15:19 +0300 Subject: [PATCH 125/165] =?UTF-8?q?docs(rfc):=20aliases=20are=20bindings,?= =?UTF-8?q?=20not=20content=20=E2=80=94=20the=20ratified=20alias=20model?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit RFC-007 §D2 gains the model the alias design reasoned through: stored queries are content + its canonical team-owned name; legacy omnigraph.yaml aliases conflate a personal name with a local-file content pointer (the muddle RFC-008 retires); operator aliases are pure bindings (server, graph, stored-query NAME, arg mapping, defaults) — an alias that carries content competes with the catalog, one that references a name composes with it. The three senses of 'global' are resolved explicitly: cross-graph globality is strengthened (one $HOME file vs per-directory), team-shared shorthand is deliberately NOT an alias mechanism (the shared name IS the catalog name), cross-machine follows the dotfile. Collision rule: legacy wins during the RFC-008 window, with a warning. RFC-008's migration row for aliases sharpens accordingly: a legacy alias splits — content to the catalog (via cluster apply), binding to the operator layer; config migrate proposes both halves. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/rfc-007-operator-config.md | 43 ++++++++++++++++++-- docs/dev/rfc-008-deprecate-omnigraph-yaml.md | 2 +- 2 files changed, 40 insertions(+), 5 deletions(-) diff --git a/docs/dev/rfc-007-operator-config.md b/docs/dev/rfc-007-operator-config.md index 5abf4e1..96daad8 100644 --- a/docs/dev/rfc-007-operator-config.md +++ b/docs/dev/rfc-007-operator-config.md @@ -137,10 +137,13 @@ servers: # operator-owned endpoint definitions # No token here, ever. Resolution: §D4. aliases: # personal shorthand over CLUSTER-owned queries - triage: # (the query is the shared contract; the alias, - server: intel-dev # its defaults, and its name are mine — RFC-008) - graph: spike - query: weekly_triage + triage: + server: intel-dev # required: names an operator server above + graph: spike # optional (omit for single-mode servers) + query: weekly_triage # STORED query name on that server — never a file + args: [since] # positional CLI args -> params, in order + params: { limit: 20 } # optional fixed defaults (positionals/--params win) + format: table # optional; feeds the format cascade defaults: output: table # read --format default @@ -151,6 +154,38 @@ written by a newer CLI must not brick an older one; contrast with `cluster.yaml`, where unknown keys are deliberately fatal because they change what a *plan* means). +#### Aliases are bindings, not content + +Three things must not be conflated: + +- **Stored queries (the cluster catalog)** are *content plus its canonical, + team-owned name* — reviewed, digest-pinned, invocable by name over HTTP. +- **Legacy `omnigraph.yaml` aliases** conflate a personal name with a + pointer to query *content in a local file* — which is why they break + across directories and can drift from the catalog. RFC-008 retires them. +- **Operator aliases** are pure **bindings, zero content**: a personal name + → (server, graph, stored-query *name*, arg mapping, defaults). An alias + that carries content competes with the catalog; an alias that references + a name composes with it. + +The three senses of "global", resolved by this split: + +1. **Across graphs/servers** — preserved and strengthened: today's aliases + are "global" only within one per-directory config file; operator + aliases live in one `$HOME` file, each binding self-contained, usable + from any cwd. +2. **Across operators (team-shared shorthand)** — deliberately *no alias + mechanism*: the shared name IS the stored query's catalog name. A team + that wants a shorter shared name renames the query in `cluster.yaml` + (reviewed, one name). A parallel team-alias namespace would be two + shared names for one thing — pure drift surface. +3. **Across machines** — dotfile the one operator file; bindings carry no + local-file dependencies. + +Collision rule during the RFC-008 window: a legacy file-alias with the +same name **wins**, with a warning naming both definitions — consistent +with §D3's legacy-outranks-operator ordering. + ### D3. Precedence and the merge rule The end-state cascade is short, because the team surface (cluster config) diff --git a/docs/dev/rfc-008-deprecate-omnigraph-yaml.md b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md index 49e2c4b..d496df8 100644 --- a/docs/dev/rfc-008-deprecate-omnigraph-yaml.md +++ b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md @@ -88,7 +88,7 @@ The full `OmnigraphConfig` surface (verified against | `cli.actor` | identity | `operator.actor` (RFC-007 §D3) | | `cli.output_format`, `cli.table_*` | personal ergonomics | `defaults:` in operator config (RFC-007 §D2) | | `cli.graph`, `cli.branch` | personal targeting | operator config: named servers + a per-operator default target (RFC-007 PR 3) | -| `aliases.<name>` | personal ergonomics over shared queries | operator config `aliases:` — the *queries* they invoke are cluster-owned; the *shorthand* is personal | +| `aliases.<name>` | a personal name conflated with a content pointer | **splits in two** (RFC-007 §D2 "bindings, not content"): the referenced `.gq` file's *content* becomes a catalog stored query (team-reviewed); the *binding* becomes an operator alias referencing that name. `config migrate` proposes both halves but cannot publish catalog content itself — that is a `cluster apply` | | `query.roots` | discovery convenience | obsolete — cluster query discovery (#183) replaced it | | `project.name` | label | dropped (the cluster's `metadata.name` is the deployment label) | From 2b33ab64f2a33260c917bbe3bfe13f1abacfbc5e Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 22:19:25 +0300 Subject: [PATCH 126/165] feat(cli): --server <name> targeting (RFC-007 PR 3, part 1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Global flags --server (operator-defined server name) and --graph (graph id on a multi-graph server, requires --server) resolve to the effective remote URI through one helper and feed the ordinary uri slot — graph resolution and the PR-2 keyed-token URL match work unchanged; the flag is sugar for a URI the operator already owns. Exclusive with a positional URI and --target (loud error, never silent precedence). Unknown names fail listing the servers that ARE defined. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/cli.rs | 11 ++++++ crates/omnigraph-cli/src/helpers.rs | 51 ++++++++++++++++++++++++++++ crates/omnigraph-cli/src/main.rs | 28 +++++++++++++++ crates/omnigraph-cli/src/operator.rs | 19 +++++++++++ 4 files changed, 109 insertions(+) diff --git a/crates/omnigraph-cli/src/cli.rs b/crates/omnigraph-cli/src/cli.rs index 7708c0a..feb08e8 100644 --- a/crates/omnigraph-cli/src/cli.rs +++ b/crates/omnigraph-cli/src/cli.rs @@ -21,6 +21,17 @@ pub(crate) struct Cli { #[arg(long = "as", global = true, value_name = "ACTOR")] pub(crate) as_actor: Option<String>, + /// Target an operator-defined server by name (RFC-007): resolves to + /// its `url` from `servers:` in ~/.omnigraph/config.yaml. Exclusive + /// with a positional URI or `--target`. + #[arg(long, global = true, value_name = "NAME")] + pub(crate) server: Option<String>, + + /// Graph id on a multi-graph `--server` (appends `/graphs/<id>` to + /// the server url). Requires --server. + #[arg(long, global = true, value_name = "GRAPH_ID", requires = "server")] + pub(crate) graph: Option<String>, + #[command(subcommand)] pub(crate) command: Command, } diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index b837192..7adac16 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -264,6 +264,57 @@ pub(crate) fn resolve_remote_bearer_token( Ok(None) } +/// `--server <name>` (RFC-007 PR 3): resolve an operator-defined server +/// name (+ optional `--graph` for multi-graph servers) to the effective +/// remote URI. The result feeds the ordinary `uri` slot, so graph +/// resolution and the keyed-token URL match work unchanged — the flag is +/// sugar for a URI the operator already owns. Unknown names fail loudly, +/// listing what IS defined. +pub(crate) fn resolve_server_flag( + server: Option<&str>, + graph: Option<&str>, +) -> Result<Option<String>> { + let Some(server) = server else { + return Ok(None); + }; + let operator_config = operator::load_operator_config()?; + let Some(entry) = operator_config.servers.get(server) else { + let known = operator_config + .servers + .keys() + .map(String::as_str) + .collect::<Vec<_>>() + .join(", "); + color_eyre::eyre::bail!( + "unknown server '{server}' — servers defined in the operator config: [{known}] (add it under servers: in ~/.omnigraph/config.yaml)" + ); + }; + let base = entry.url.trim_end_matches('/'); + Ok(Some(match graph { + Some(graph) => format!("{base}/graphs/{graph}"), + None => base.to_string(), + })) +} + +/// Apply `--server`/`--graph` to a command's uri/target slots: exclusive +/// with both (loud error, not silent precedence), no-op when absent. +pub(crate) fn apply_server_flag( + server: Option<&str>, + graph: Option<&str>, + uri: Option<String>, + target: Option<&str>, +) -> Result<Option<String>> { + if server.is_none() { + return Ok(uri); + } + if uri.is_some() || target.is_some() { + color_eyre::eyre::bail!( + "--server is exclusive with a positional URI and --target — pick one way to address the graph" + ); + } + resolve_server_flag(server, graph) +} + /// The remote base URL a token resolution is FOR — the same scoping /// `graph_bearer_token_env` uses: an explicit http(s) `--uri` wins, else /// the config-resolved target's uri (when remote). Local URIs → None. diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 85fe537..0ee3851 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -130,6 +130,8 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let graph = resolve_cli_graph(&config, uri, target.as_deref())?; @@ -198,6 +200,8 @@ async fn main() -> Result<()> { use `omnigraph load --from <base> --mode <mode>` (ingest defaults: --from main --mode merge)" ); let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let graph = resolve_cli_graph(&config, uri, target.as_deref())?; @@ -250,6 +254,8 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let graph = resolve_cli_graph(&config, uri, target.as_deref())?; @@ -293,6 +299,8 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let graph = resolve_cli_graph(&config, uri, target.as_deref())?; @@ -328,6 +336,8 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let graph = resolve_cli_graph(&config, uri, target.as_deref())?; @@ -367,6 +377,8 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let graph = resolve_cli_graph(&config, uri, target.as_deref())?; @@ -417,6 +429,8 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let uri = resolve_uri(&config, uri, target.as_deref())?; @@ -456,6 +470,8 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let uri = resolve_uri(&config, uri, target.as_deref())?; @@ -519,6 +535,8 @@ async fn main() -> Result<()> { allow_data_loss, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let graph = resolve_cli_graph(&config, uri, target.as_deref())?; @@ -576,6 +594,8 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let uri = resolve_uri(&config, uri, target.as_deref())?; @@ -640,6 +660,8 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let uri = resolve_uri(&config, uri, target.as_deref())?; @@ -675,6 +697,8 @@ async fn main() -> Result<()> { table_keys, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let uri = resolve_uri(&config, uri, target.as_deref())?; @@ -736,6 +760,7 @@ async fn main() -> Result<()> { let target_name = target .as_deref() .or_else(|| alias_config.and_then(|alias| alias.graph.as_deref())); + let uri = apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target_name)?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target_name)?; let graph = resolve_cli_graph(&config, uri, target_name)?; let uri = graph.uri.clone(); @@ -822,6 +847,7 @@ async fn main() -> Result<()> { let target_name = target .as_deref() .or_else(|| alias_config.and_then(|alias| alias.graph.as_deref())); + let uri = apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target_name)?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target_name)?; let graph = resolve_cli_graph(&config, uri, target_name)?; let uri = graph.uri.clone(); @@ -1177,6 +1203,8 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; + let uri = + apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; let uri = resolve_uri(&config, uri, target.as_deref())?; diff --git a/crates/omnigraph-cli/src/operator.rs b/crates/omnigraph-cli/src/operator.rs index 1b95e24..64b5756 100644 --- a/crates/omnigraph-cli/src/operator.rs +++ b/crates/omnigraph-cli/src/operator.rs @@ -467,6 +467,25 @@ mod tests { assert_eq!(config.find_server_for_url("http://other:9999"), None); } + #[test] + fn server_lookup_supports_targeting() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("config.yaml"); + fs::write( + &path, + "servers:\n intel-dev:\n url: http://127.0.0.1:8080/\n", + ) + .unwrap(); + let config = load_operator_config_at(&path).unwrap(); + // the --server resolution shape: bare url and graph-scoped url + let base = config.servers["intel-dev"].url.trim_end_matches('/'); + assert_eq!(base, "http://127.0.0.1:8080"); + assert_eq!( + format!("{base}/graphs/spike"), + "http://127.0.0.1:8080/graphs/spike" + ); + } + #[test] fn token_env_name_uppercases_and_underscores() { assert_eq!(token_env_name("intel-dev"), "OMNIGRAPH_TOKEN_INTEL_DEV"); From dc91c55970ae94362e6f5f51e23355db8fb8abf6 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 22:25:42 +0300 Subject: [PATCH 127/165] =?UTF-8?q?feat(cli):=20operator=20aliases=20?= =?UTF-8?q?=E2=80=94=20pure=20bindings=20invoking=20stored=20queries=20(RF?= =?UTF-8?q?C-007=20PR=203,=20part=202)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit aliases: in the operator config bind a personal name to (server, graph, stored-query NAME, positional arg mapping, fixed param defaults, format) — zero content, per the ratified bindings-not-content model. Invocation goes through the server's stored-query endpoint (POST {base}/graphs/{g}/queries/{name}) with the keyed credential resolving via the ordinary URL match; param precedence --params > positionals > fixed defaults; the result renders through the existing format cascade with the alias's format as its hop. A legacy omnigraph.yaml alias with the same name wins during the RFC-008 window, with a warning naming both. E2e (spawned policy-gated server, invoke_query granted via a per-graph bundle): the alias invokes with name + one positional and nothing else — server, graph, query, and token all from the operator layer; --server/ --graph explicit targeting; unknown --server lists defined names; --server exclusive with a positional URI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/helpers.rs | 51 +++++++++ crates/omnigraph-cli/src/main.rs | 28 +++++ crates/omnigraph-cli/src/operator.rs | 35 +++++- crates/omnigraph-cli/tests/system_local.rs | 121 +++++++++++++++++++++ docs/dev/rfc-007-operator-config.md | 2 +- docs/user/cli-reference.md | 25 ++++- 6 files changed, 256 insertions(+), 6 deletions(-) diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index 7adac16..a48f2e4 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -296,6 +296,57 @@ pub(crate) fn resolve_server_flag( })) } +/// Execute an OPERATOR alias (RFC-007 PR 3): a pure binding invoking a +/// stored query by name on a named server — POST {base}/queries/{name}. +/// Param precedence: --params > positional args > the alias's fixed +/// params. The keyed token applies via the ordinary URL match. +pub(crate) async fn execute_operator_alias( + client: &reqwest::Client, + config: &OmnigraphConfig, + alias_name: &str, + alias: &crate::operator::OperatorAlias, + alias_args: &[String], + explicit_params: Option<Value>, +) -> Result<ReadOutput> { + let uri = resolve_server_flag(Some(&alias.server), alias.graph.as_deref())? + .expect("server name is present"); + let bearer_token = resolve_remote_bearer_token(config, Some(&uri), None)?; + + let mut params = serde_json::Map::new(); + for (key, value) in &alias.params { + let Some(key) = key.as_str() else { + bail!("alias '{alias_name}': params keys must be strings"); + }; + params.insert(key.to_string(), serde_json::to_value(value)?); + } + if alias_args.len() > alias.args.len() { + bail!( + "alias '{alias_name}' takes {} positional arg(s) ({}), got {}", + alias.args.len(), + alias.args.join(", "), + alias_args.len() + ); + } + for (name, value) in alias.args.iter().zip(alias_args) { + params.insert(name.clone(), parse_alias_value(value)); + } + if let Some(Value::Object(explicit)) = explicit_params { + for (key, value) in explicit { + params.insert(key, value); + } + } + + let body = (!params.is_empty()).then(|| serde_json::json!({ "params": params })); + remote_json( + client, + Method::POST, + remote_url(&uri, &format!("/queries/{}", alias.query)), + body, + bearer_token.as_deref(), + ) + .await +} + /// Apply `--server`/`--graph` to a command's uri/target slots: exclusive /// with both (loud error, not silent precedence), no-op when absent. pub(crate) fn apply_server_flag( diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 0ee3851..284cff8 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -746,6 +746,34 @@ async fn main() -> Result<()> { } let config = load_cli_config(config.as_ref())?; + // Operator aliases (RFC-007 PR 3): pure bindings to stored + // queries. A legacy file-alias with the same name wins during + // the RFC-008 window (with a warning); an alias name found + // only in the operator layer takes the invoke path here. + if let Some(alias_name) = alias.as_deref() { + let operator_config = crate::operator::load_operator_config()?; + if let Some(operator_alias) = operator_config.aliases.get(alias_name) { + if config.alias(alias_name).is_ok() { + eprintln!( + "warning: alias '{alias_name}' is defined in both omnigraph.yaml (legacy, wins during the deprecation window) and the operator config; the legacy definition applies" + ); + } else { + let output = execute_operator_alias( + &http_client, + &config, + alias_name, + operator_alias, + &alias_args, + load_params_json(¶ms)?, + ) + .await?; + let format = + resolve_read_format(&config, format, json, operator_alias.format); + print_read_output(&output, format, &config)?; + return Ok(()); + } + } + } let alias = resolve_alias(&config, alias.as_deref(), AliasCommand::Read)?; let alias_name = alias.as_ref().map(|(name, _)| *name); let alias_config = alias.as_ref().map(|(_, alias)| *alias); diff --git a/crates/omnigraph-cli/src/operator.rs b/crates/omnigraph-cli/src/operator.rs index 64b5756..16f5550 100644 --- a/crates/omnigraph-cli/src/operator.rs +++ b/crates/omnigraph-cli/src/operator.rs @@ -38,12 +38,38 @@ pub(crate) struct OperatorConfig { /// can redefine an entry here. No tokens in this file, ever. #[serde(default)] pub(crate) servers: BTreeMap<String, OperatorServer>, + /// Personal alias bindings (RFC-007 PR 3); see OperatorAlias. + #[serde(default)] + pub(crate) aliases: BTreeMap<String, OperatorAlias>, /// Everything this CLI version doesn't know. Warned once at load, /// otherwise ignored (forward compatibility within the operator layer). #[serde(flatten)] unknown: serde_yaml::Mapping, } +/// A personal alias: a pure BINDING to a stored query on a named server — +/// never content, never a file (RFC-007 §D2 "Aliases are bindings, not +/// content"). The stored query is the team's contract; the alias, its +/// defaults, and its name are the operator's. +#[derive(Debug, Deserialize)] +pub(crate) struct OperatorAlias { + /// Names an entry under `servers:`. + pub(crate) server: String, + /// Graph id for multi-graph servers (appends `/graphs/<id>`). + pub(crate) graph: Option<String>, + /// The STORED query's name on that server. + pub(crate) query: String, + /// Positional CLI args bind to these param names, in order. + #[serde(default)] + pub(crate) args: Vec<String>, + /// Fixed default params; positionals and `--params` override per key. + #[serde(default)] + pub(crate) params: serde_yaml::Mapping, + pub(crate) format: Option<ReadOutputFormat>, + #[serde(flatten)] + unknown: serde_yaml::Mapping, +} + #[derive(Debug, Deserialize)] pub(crate) struct OperatorServer { pub(crate) url: String, @@ -163,6 +189,9 @@ impl OperatorConfig { for (name, server) in &self.servers { collect(&server.unknown, &format!("servers.{name}.")); } + for (name, alias) in &self.aliases { + collect(&alias.unknown, &format!("aliases.{name}.")); + } warnings } } @@ -425,10 +454,8 @@ mod tests { let config = load_operator_config_at(&path).unwrap(); assert_eq!(config.actor(), Some("act-a")); let warnings = config.unknown_key_warnings(); - // `servers` became a known key in PR 2; `aliases` stays unknown - // until PR 3. - assert_eq!(warnings.len(), 2, "{warnings:?}"); - assert!(warnings.iter().any(|w| w.contains("`aliases`"))); + // `servers` (PR 2) and `aliases` (PR 3) are known keys now. + assert_eq!(warnings.len(), 1, "{warnings:?}"); assert!(warnings.iter().any(|w| w.contains("`operator.color`"))); assert_eq!(config.servers["prod"].url, "https://example.com"); } diff --git a/crates/omnigraph-cli/tests/system_local.rs b/crates/omnigraph-cli/tests/system_local.rs index 5eb739f..28ed7a3 100644 --- a/crates/omnigraph-cli/tests/system_local.rs +++ b/crates/omnigraph-cli/tests/system_local.rs @@ -2424,3 +2424,124 @@ fn local_cli_keyed_credentials_authenticate_url_matched_server() { let output = remote_read(&[]); assert!(!output.status.success(), "logout must revoke access"); } + +/// RFC-007 PR 3: --server targeting and operator aliases (pure bindings to +/// stored queries) end to end, with the keyed credential from PR 2. +#[test] +fn local_cli_operator_alias_and_server_flag_invoke_stored_query() { + let graph = SystemGraph::loaded(); + graph.write_query( + "stored-find-person.gq", + "query find_person($name: String) { match { $p: Person { name: $name } } return { $p.name } }", + ); + // invoke_query is policy-gated (anti-probing 404 without the grant), + // so the server gets a per-graph bundle granting it to the operator. + graph.write_file( + "graph.policy.yaml", + "version: 1\ngroups:\n ops: [\"act-op\"]\nprotected_branches: [main]\nrules:\n - id: allow-invoke\n allow:\n actors: { group: ops }\n actions: [invoke_query]\n - id: allow-read\n allow:\n actors: { group: ops }\n actions: [read]\n branch_scope: any\n", + ); + let config = graph.write_config( + "omnigraph-server.yaml", + &format!( + "graphs:\n local:\n uri: {}\n policy:\n file: ./graph.policy.yaml\n queries:\n find_person:\n file: ./stored-find-person.gq\n", + yaml_string(&graph.path().to_string_lossy()) + ), + ); + let server = spawn_server_with_config_env( + &config, + &[( + "OMNIGRAPH_SERVER_BEARER_TOKENS_JSON", + r#"{"act-op":"srv-tok"}"#, + )], + ); + + let operator_home = tempfile::tempdir().unwrap(); + fs::write( + operator_home.path().join("config.yaml"), + format!( + "servers:\n dev:\n url: {}\naliases:\n who:\n server: dev\n graph: local\n query: find_person\n args: [name]\n", + server.base_url + ), + ) + .unwrap(); + fs::write( + operator_home.path().join("credentials"), + "[dev]\ntoken = srv-tok\n", + ) + .unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + fs::set_permissions( + operator_home.path().join("credentials"), + fs::Permissions::from_mode(0o600), + ) + .unwrap(); + } + + // The operator alias: name + positional arg, nothing else — server, + // graph, stored query, and token all resolve from the operator layer. + let output = cli() + .env("OMNIGRAPH_HOME", operator_home.path()) + .arg("query") + .arg("--alias") + .arg("who") + .arg("Alice") + .arg("--json") + .output() + .unwrap(); + assert!( + output.status.success(), + "operator alias must invoke the stored query: {output:?}" + ); + let payload: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["rows"][0]["p.name"], "Alice", "{payload}"); + + // --server/--graph: the same stored query via explicit targeting. + let output = cli() + .env("OMNIGRAPH_HOME", operator_home.path()) + .arg("query") + .arg("--server") + .arg("dev") + .arg("--graph") + .arg("local") + .arg("--query-string") + .arg("query q($name: String) { match { $p: Person { name: $name } } return { $p.name } }") + .arg("--params") + .arg(r#"{"name":"Alice"}"#) + .arg("--json") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + + // Unknown --server errors listing what IS defined. + let output = cli() + .env("OMNIGRAPH_HOME", operator_home.path()) + .arg("query") + .arg("--server") + .arg("nope") + .arg("--query-string") + .arg("query q() { match { $p: Person } return { $p.name } }") + .output() + .unwrap(); + assert!(!output.status.success()); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stderr.contains("unknown server 'nope'") && stderr.contains("dev"), "{stderr}"); + + // --server is exclusive with a positional URI. + let output = cli() + .env("OMNIGRAPH_HOME", operator_home.path()) + .arg("query") + .arg(&server.base_url) + .arg("--server") + .arg("dev") + .arg("--query-string") + .arg("query q() { match { $p: Person } return { $p.name } }") + .output() + .unwrap(); + assert!(!output.status.success()); + assert!( + String::from_utf8_lossy(&output.stderr).contains("exclusive"), + "{output:?}" + ); +} diff --git a/docs/dev/rfc-007-operator-config.md b/docs/dev/rfc-007-operator-config.md index 96daad8..5bd8afb 100644 --- a/docs/dev/rfc-007-operator-config.md +++ b/docs/dev/rfc-007-operator-config.md @@ -286,7 +286,7 @@ Three PRs, each independently useful, each landable without the next: §D4 chain (env + credentials file), the §D5 trust rules, and `omnigraph login <name>` (atomic write, `0600`). Legacy mechanisms untouched and tested-as-untouched. -3. **PR 3 — operator targeting.** `--server <name>` on remote-capable +3. **PR 3 — operator targeting** *(landed)*. `--server <name>` on remote-capable commands and `aliases:` in the operator layer (server + graph + query + default params), resolving through operator-defined servers. This is the *bridge* toward RFC-002's locator — multi-server addressing in a diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index c41a15c..b113ef3 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -2,7 +2,7 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` schema. For a quick-start guide, see [cli.md](cli.md). -Top-level command families and subcommands. Graph-targeting commands accept either a positional `URI`, `--uri`, or a `--target <name>` resolved against `omnigraph.yaml`; `cluster` commands use `--config <dir>`. +Top-level command families and subcommands. Graph-targeting commands accept a positional `URI`, `--uri`, a `--target <name>` resolved against `omnigraph.yaml`, or `--server <name>` (an operator-defined server from `~/.omnigraph/config.yaml`, optionally with `--graph <id>` for multi-graph servers; exclusive with the other forms); `cluster` commands use `--config <dir>`. ## Top-level commands @@ -67,6 +67,29 @@ refused). Token from `--token`, or — preferred, keeps it out of shell history — one line on stdin: `echo $TOKEN | omnigraph login prod`. `omnigraph logout <name>` removes it (idempotent). +#### Operator aliases — bindings, not content + +An operator alias is a personal name for *invoking a stored query on a +named server* — it carries no query content (the stored query in the +catalog is the team's contract; the alias, its defaults, and its name are +yours): + +```yaml +aliases: + triage: + server: intel-dev # names an entry under servers: + graph: spike # optional (multi-graph servers) + query: weekly_triage # the STORED query's name — never a file + args: [since] # positional args -> params, in order + params: { limit: 20 } # fixed defaults; positionals/--params win + format: table +``` + +`omnigraph query --alias triage 2026-06-01` invokes +`POST <server>/graphs/spike/queries/weekly_triage` with the keyed +credential. A legacy `omnigraph.yaml` alias with the same name wins during +the deprecation window (with a warning). + A remote command whose URL prefix-matches an operator server's `url` (the `gh` host model — no flags needed) resolves its token through: From 20ddfc61c1605d790ab59bbcd7f108d138c7b37b Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 22:29:57 +0300 Subject: [PATCH 128/165] fix(cli): reclaim the hidden legacy-uri positional for operator aliases MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Caught on the live smoke: with --alias, the first bare CLI arg lands in the hidden legacy_uri positional, so an operator alias's positional param never bound ('parameter not provided' from the server). An operator alias always knows its target, so the existing normalize_legacy_alias_uri reclaims the swallowed positional as the first alias arg — same rule the legacy path already applies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/main.rs | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 284cff8..4306b67 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -758,6 +758,15 @@ async fn main() -> Result<()> { "warning: alias '{alias_name}' is defined in both omnigraph.yaml (legacy, wins during the deprecation window) and the operator config; the legacy definition applies" ); } else { + // The hidden legacy-uri positional swallows the first + // bare arg; an operator alias always knows its target, + // so reclaim it as the first positional param. + let (_, alias_args) = normalize_legacy_alias_uri( + legacy_uri.clone(), + true, + Some(alias_name), + alias_args.clone(), + ); let output = execute_operator_alias( &http_client, &config, From c89d268b236d96d270803e602e631f42f48a4e4d Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 23:28:33 +0300 Subject: [PATCH 129/165] feat(config): per-key deprecation warnings on legacy omnigraph.yaml load (RFC-008 stage 1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Loading a legacy file (flag, env, or cwd-found — never on defaults) emits one stderr block listing each key actually present with its destination from RFC-008's migration map — the map applied to YOUR file, not a generic banner. Once per process; both binaries warn (cluster-mode boots never reach load_config, silent by construction); suppressible via OMNIGRAPH_SUPPRESS_YAML_DEPRECATION=1 for CI logs during the window. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- .../omnigraph-cli/tests/cli_schema_config.rs | 40 +++++++++ crates/omnigraph-server/src/config.rs | 88 ++++++++++++++++++- 2 files changed, 127 insertions(+), 1 deletion(-) diff --git a/crates/omnigraph-cli/tests/cli_schema_config.rs b/crates/omnigraph-cli/tests/cli_schema_config.rs index a0dac0a..a55aaed 100644 --- a/crates/omnigraph-cli/tests/cli_schema_config.rs +++ b/crates/omnigraph-cli/tests/cli_schema_config.rs @@ -498,3 +498,43 @@ fn graphs_list_against_local_uri_errors_with_remote_only_message() { "expected 'remote multi-graph server URL' rejection in stderr; got:\n{stderr}" ); } + +/// RFC-008 stage 1: loading a legacy omnigraph.yaml emits the per-key +/// deprecation block (the migration map applied to THIS file), suppressible +/// via OMNIGRAPH_SUPPRESS_YAML_DEPRECATION. +#[test] +fn legacy_config_load_warns_per_key_and_suppression_silences() { + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("omnigraph.yaml"), + "cli:\n actor: act-x\ngraphs:\n g:\n uri: /tmp/never-opened\n", + ) + .unwrap(); + + // `graphs list --json` loads the config and exits without touching the + // graph URI. + let output = cli() + .current_dir(temp.path()) + .arg("graphs") + .arg("list") + .arg("--json") + .output() + .unwrap(); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("deprecated (RFC-008)") && stderr.contains("`cli.actor` -> `operator.actor`"), + "{stderr}" + ); + assert!(stderr.contains("config migrate"), "{stderr}"); + + let output = cli() + .current_dir(temp.path()) + .env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1") + .arg("graphs") + .arg("list") + .arg("--json") + .output() + .unwrap(); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(!stderr.contains("deprecated (RFC-008)"), "{stderr}"); +} diff --git a/crates/omnigraph-server/src/config.rs b/crates/omnigraph-server/src/config.rs index 52bac2e..ea7ce30 100644 --- a/crates/omnigraph-server/src/config.rs +++ b/crates/omnigraph-server/src/config.rs @@ -549,7 +549,9 @@ fn load_config_in( }); let mut config = if let Some(path) = &config_path { - serde_yaml::from_str::<OmnigraphConfig>(&fs::read_to_string(path)?)? + let text = fs::read_to_string(path)?; + warn_yaml_deprecation_once(path, &text); + serde_yaml::from_str::<OmnigraphConfig>(&text)? } else { OmnigraphConfig::default() }; @@ -563,6 +565,74 @@ fn load_config_in( Ok(config) } +/// RFC-008 stage 1: suppress the legacy-config deprecation warning +/// (one process), for CI logs during the deprecation window. +pub const SUPPRESS_YAML_DEPRECATION_ENV: &str = "OMNIGRAPH_SUPPRESS_YAML_DEPRECATION"; + +/// RFC-008's migration map (the "Where every key goes" table), applied to +/// the keys actually present in a loaded file — never a generic banner. +/// Keys are `(yaml pointer, destination)`; the pointer is matched against +/// the file's real top-level/nested keys. +const YAML_DEPRECATION_MAP: &[(&str, &str)] = &[ + ("graphs", "cluster.yaml `graphs:` (team surface) — or flags/env for the zero-config tier"), + ("queries", "the cluster catalog (`.gq` discovery in cluster.yaml)"), + ("policy", "cluster.yaml `policies:` + `applies_to` bindings"), + ("server", "flags/env (`--bind`); meaningless under cluster boot"), + ("auth", "the operator credentials chain (`omnigraph login <server>`)"), + ("aliases", "operator `aliases:` (bindings) + catalog stored queries (content)"), + ("query", "obsolete — cluster query discovery replaced `query.roots`"), + ("project", "cluster.yaml `metadata.name`"), + ("cli.actor", "`operator.actor` in ~/.omnigraph/config.yaml"), + ("cli.output_format", "`defaults.output` in ~/.omnigraph/config.yaml"), + ("cli.table_max_column_width", "`defaults.table_max_column_width` in ~/.omnigraph/config.yaml"), + ("cli.table_cell_layout", "`defaults.table_cell_layout` in ~/.omnigraph/config.yaml"), + ("cli.graph", "explicit `--target`/`--server` (no operator default-target yet)"), + ("cli.branch", "explicit `--branch`"), +]; + +/// Emit the per-key deprecation block once per process when a legacy +/// `omnigraph.yaml` is actually loaded. `omnigraph config migrate` +/// produces the split these lines describe. +fn warn_yaml_deprecation_once(path: &Path, text: &str) { + static WARNED: std::sync::OnceLock<()> = std::sync::OnceLock::new(); + if env::var_os(SUPPRESS_YAML_DEPRECATION_ENV).is_some() { + return; + } + let lines = yaml_deprecation_lines(text); + if lines.is_empty() { + return; + } + WARNED.get_or_init(|| { + eprintln!( + "warning: '{}' is deprecated (RFC-008) — its keys have new homes; run `omnigraph config migrate` for the split, set {SUPPRESS_YAML_DEPRECATION_ENV}=1 to silence:", + path.display() + ); + for line in &lines { + eprintln!(" {line}"); + } + }); +} + +fn yaml_deprecation_lines(text: &str) -> Vec<String> { + let Ok(mapping) = serde_yaml::from_str::<serde_yaml::Mapping>(text) else { + return Vec::new(); + }; + let present = |pointer: &str| -> bool { + match pointer.split_once('.') { + None => mapping.contains_key(pointer), + Some((outer, inner)) => mapping + .get(outer) + .and_then(|value| value.as_mapping()) + .is_some_and(|nested| nested.contains_key(inner)), + } + }; + YAML_DEPRECATION_MAP + .iter() + .filter(|(pointer, _)| present(pointer)) + .map(|(pointer, destination)| format!("`{pointer}` -> {destination}")) + .collect() +} + fn absolute_base_dir(cwd: &Path, path: &Path) -> Result<PathBuf> { let path = if path.is_absolute() { path.to_path_buf() @@ -608,6 +678,22 @@ mod tests { assert_eq!(config.cli.actor.as_deref(), Some("act-env")); } + #[test] + fn yaml_deprecation_lines_name_present_keys_only() { + let lines = super::yaml_deprecation_lines( + "graphs:\n g:\n uri: /tmp/x\ncli:\n actor: a\n branch: main\n", + ); + let joined = lines.join("\n"); + assert!(joined.contains("`graphs` ->"), "{joined}"); + assert!(joined.contains("`cli.actor` -> `operator.actor`"), "{joined}"); + assert!(joined.contains("`cli.branch` ->"), "{joined}"); + assert!(!joined.contains("`aliases`"), "{joined}"); + assert!(!joined.contains("`cli.output_format`"), "{joined}"); + + assert!(super::yaml_deprecation_lines("").is_empty()); + assert!(super::yaml_deprecation_lines("not: [valid").is_empty()); + } + #[test] fn load_config_reads_yaml_defaults_from_current_dir() { let temp = tempdir().unwrap(); From cd1f17539628f50078357ff66cc3b5432f56c958 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 23:32:05 +0300 Subject: [PATCH 130/165] =?UTF-8?q?feat(cli):=20omnigraph=20config=20migra?= =?UTF-8?q?te=20=E2=80=94=20the=20RFC-008=20split=20(stage=202)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reads a legacy omnigraph.yaml and produces the three-section split: team half as a ready-to-review cluster.yaml proposal (graphs with TODO schema pointers — the legacy file never knew schemas — per-graph queries directories, policies with applies_to bindings), personal half as an operator-config merge (actor, output/table defaults — OperatorDefaults gains the two table keys with their cascade hops — remote graphs with bearer_token_env become servers entries plus a printed login step, and legacy aliases split per the RFC: content to the catalog as a manual step, binding to an operator alias), plus a dropped-keys section with reasons. Touches nothing without --write; with it, the operator merge is key-level (existing entries always win; prior file backed up), and cluster.yaml is emitted only when absent (else cluster.yaml.proposed). --json emits the report structurally. The completeness contract is a unit test: every top-level key of the legacy schema must classify somewhere, or the RFC-008 map has a bug. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/cli.rs | 23 + crates/omnigraph-cli/src/main.rs | 37 ++ crates/omnigraph-cli/src/migrate.rs | 408 ++++++++++++++++++ crates/omnigraph-cli/src/operator.rs | 4 + crates/omnigraph-cli/src/output.rs | 23 +- .../omnigraph-cli/tests/cli_schema_config.rs | 71 +++ 6 files changed, 562 insertions(+), 4 deletions(-) create mode 100644 crates/omnigraph-cli/src/migrate.rs diff --git a/crates/omnigraph-cli/src/cli.rs b/crates/omnigraph-cli/src/cli.rs index feb08e8..7b976b4 100644 --- a/crates/omnigraph-cli/src/cli.rs +++ b/crates/omnigraph-cli/src/cli.rs @@ -56,6 +56,12 @@ pub(crate) enum Command { #[arg(long)] json: bool, }, + /// Legacy-config tooling (RFC-008): split omnigraph.yaml into its + /// two destinations. + Config { + #[command(subcommand)] + command: ConfigCommand, + }, /// Remove a named server's stored credential. Idempotent. Logout { name: String, @@ -681,3 +687,20 @@ impl CliLoadMode { } } +#[derive(Debug, Subcommand)] +pub(crate) enum ConfigCommand { + /// Propose (and with --write, apply) the RFC-008 split of a legacy + /// omnigraph.yaml: team half -> a ready-to-review cluster.yaml, + /// personal half -> ~/.omnigraph/config.yaml (key-level merge, + /// existing entries always win). Touches nothing without --write. + Migrate { + /// Path to the legacy omnigraph.yaml (default: ./omnigraph.yaml) + #[arg(long)] + config: Option<PathBuf>, + /// Apply the split instead of only printing it + #[arg(long)] + write: bool, + #[arg(long)] + json: bool, + }, +} diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 4306b67..133a8a0 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -42,6 +42,7 @@ use serde::de::DeserializeOwned; use serde_json::Value; mod embed; +mod migrate; mod operator; mod read_format; @@ -73,6 +74,42 @@ async fn main() -> Result<()> { }; let http_client = build_http_client()?; match cli.command { + Command::Config { command } => match command { + ConfigCommand::Migrate { config, write, json } => { + let path = migrate::legacy_config_path(config.as_ref()); + if !path.exists() { + bail!( + "no legacy config at '{}' — nothing to migrate", + path.display() + ); + } + let legacy = load_config(Some(&path))?; + let report = migrate::build_report(&legacy, &path); + if write { + let legacy_dir = path + .parent() + .filter(|parent| !parent.as_os_str().is_empty()) + .unwrap_or(std::path::Path::new(".")) + .to_path_buf(); + let written = migrate::apply_report(&report, &legacy_dir)?; + if json { + print_json(&serde_json::json!({ + "report": report, + "written": written, + }))?; + } else { + print!("{}", migrate::render_report(&report)); + for line in written { + println!("wrote: {line}"); + } + } + } else if json { + print_json(&report)?; + } else { + print!("{}", migrate::render_report(&report)); + } + } + }, Command::Login { name, token, json } => { let token = match token { Some(token) => token, diff --git a/crates/omnigraph-cli/src/migrate.rs b/crates/omnigraph-cli/src/migrate.rs new file mode 100644 index 0000000..3891061 --- /dev/null +++ b/crates/omnigraph-cli/src/migrate.rs @@ -0,0 +1,408 @@ +//! `omnigraph config migrate` (RFC-008 stage 2): split a legacy +//! `omnigraph.yaml` into its two destinations — the team half as a +//! ready-to-review `cluster.yaml` proposal, the personal half merged into +//! `~/.omnigraph/config.yaml` — and name what's obsolete. The command is +//! the completeness test of RFC-008's migration map: any key it cannot +//! place is a bug in the RFC. +//! +//! Touches nothing without `--write`. Referenced `.gq`/policy files are +//! never moved; manual steps are printed instead. + +use std::collections::BTreeMap; +use std::path::{Path, PathBuf}; + +use color_eyre::Result; +use color_eyre::eyre::eyre; +use omnigraph_server::OmnigraphConfig; +use serde::Serialize; + +use crate::operator; + +#[derive(Debug, Serialize)] +pub(crate) struct MigrateReport { + pub(crate) source: String, + /// The ready-to-review cluster.yaml text (None when the legacy file + /// declares nothing team-shaped). + pub(crate) cluster_yaml: Option<String>, + /// Operator keys to merge: dotted key -> YAML value text. + pub(crate) operator_merge: BTreeMap<String, String>, + /// Keys with no destination, and why. + pub(crate) dropped: Vec<DroppedKey>, + /// Steps the command will not do for you. + pub(crate) manual_steps: Vec<String>, +} + +#[derive(Debug, Serialize)] +pub(crate) struct DroppedKey { + pub(crate) key: String, + pub(crate) reason: String, +} + +/// Classify a parsed legacy config into the report. Pure — no I/O. +pub(crate) fn build_report(config: &OmnigraphConfig, source: &Path) -> MigrateReport { + let mut dropped = Vec::new(); + let mut manual_steps = Vec::new(); + let mut operator_merge: BTreeMap<String, String> = BTreeMap::new(); + + // ---- personal half ---- + if let Some(actor) = &config.cli.actor { + operator_merge.insert("operator.actor".into(), actor.clone()); + } + if let Some(format) = config.cli.output_format { + operator_merge.insert( + "defaults.output".into(), + serde_yaml::to_string(&format).unwrap_or_default().trim().to_string(), + ); + } + if let Some(width) = config.cli.table_max_column_width { + operator_merge.insert("defaults.table_max_column_width".into(), width.to_string()); + } + if let Some(layout) = config.cli.table_cell_layout { + operator_merge.insert( + "defaults.table_cell_layout".into(), + serde_yaml::to_string(&layout).unwrap_or_default().trim().to_string(), + ); + } + if config.cli.graph.is_some() { + dropped.push(DroppedKey { + key: "cli.graph".into(), + reason: "no operator default-target yet — address graphs explicitly via --target/--server (RFC-002 locator territory)".into(), + }); + } + if config.cli.branch.is_some() { + dropped.push(DroppedKey { + key: "cli.branch".into(), + reason: "pass --branch explicitly".into(), + }); + } + + // Remote graphs with a token env become operator servers (the keyed + // chain replaces invented env-var names). + for (name, target) in &config.graphs { + if target.uri.starts_with("http://") || target.uri.starts_with("https://") { + operator_merge.insert(format!("servers.{name}.url"), target.uri.clone()); + if target.bearer_token_env.is_some() { + manual_steps.push(format!( + "store the '{name}' token in the keyed chain: echo $TOKEN | omnigraph login {name} (replaces bearer_token_env)" + )); + } + } + } + if config.auth.env_file.is_some() { + manual_steps.push( + "auth.env_file keeps working during the window; prefer `omnigraph login <server>` per server going forward".into(), + ); + } + + // Legacy aliases split: content -> catalog stored query, binding -> + // operator alias referencing the name. + for (name, alias) in &config.aliases { + let query_name = alias.name.clone().unwrap_or_else(|| name.clone()); + operator_merge.insert( + format!("aliases.{name}"), + format!( + "{{ server: TODO-server-name, graph: {}, query: {query_name}, args: [{}] }}", + alias.graph.as_deref().unwrap_or("TODO-graph-id"), + alias.args.join(", ") + ), + ); + manual_steps.push(format!( + "alias '{name}': move its query content ('{}') into the cluster checkout's queries/ so '{query_name}' becomes a catalog stored query", + alias.query + )); + } + + // ---- team half ---- + let has_team_content = !config.graphs.is_empty() + || !config.queries.is_empty() + || config.policy.file.is_some() + || config.server.policy.file.is_some(); + let cluster_yaml = has_team_content.then(|| { + let mut out = String::from("version: 1\n"); + if let Some(name) = &config.project.name { + out.push_str(&format!("metadata:\n name: {name}\n")); + } + out.push_str("# storage: s3://bucket/prefix # or omit: this folder is the root\n"); + if !config.graphs.is_empty() || !config.queries.is_empty() { + out.push_str("graphs:\n"); + } + // Single-graph top-level queries belong to a graph the legacy file + // never named; propose one. + if !config.queries.is_empty() && config.graphs.is_empty() { + out.push_str(" default: # TODO: pick the graph id\n schema: # TODO: path to this graph's .pg schema\n queries: queries/\n"); + } + for (name, target) in &config.graphs { + out.push_str(&format!(" {name}:\n")); + out.push_str(" schema: # TODO: path to this graph's .pg schema\n"); + if !target.queries.is_empty() { + out.push_str(" queries: queries/ # move the .gq files here\n"); + } + out.push_str(&format!( + " # legacy root: {} — the cluster manages graph roots under its storage; run `omnigraph cluster import` after reviewing\n", + target.uri + )); + } + let mut policies: Vec<(String, String, String)> = Vec::new(); + if let Some(file) = &config.policy.file { + policies.push(("default".into(), file.clone(), "graph.<id> # TODO: bind".into())); + } + if let Some(file) = &config.server.policy.file { + policies.push(("server".into(), file.clone(), "cluster".into())); + } + for (name, target) in &config.graphs { + if let Some(file) = &target.policy.file { + policies.push((name.clone(), file.clone(), format!("graph.{name}"))); + } + } + if !policies.is_empty() { + out.push_str("policies:\n"); + for (name, file, binding) in policies { + out.push_str(&format!( + " {name}:\n file: {file}\n applies_to: [{binding}]\n" + )); + } + } + out + }); + + if !config.query.roots.is_empty() { + dropped.push(DroppedKey { + key: "query.roots".into(), + reason: "obsolete — cluster query discovery (queries: <dir>) replaced it".into(), + }); + } + if config.server.bind.is_some() || config.server.graph.is_some() { + dropped.push(DroppedKey { + key: "server.bind / server.graph".into(), + reason: "deployment runtime — pass --bind / target flags or env".into(), + }); + } + if config.project.name.is_some() && cluster_yaml.is_none() { + dropped.push(DroppedKey { + key: "project.name".into(), + reason: "the cluster's metadata.name is the deployment label".into(), + }); + } + + MigrateReport { + source: source.display().to_string(), + cluster_yaml, + operator_merge, + dropped, + manual_steps, + } +} + +pub(crate) fn render_report(report: &MigrateReport) -> String { + let mut out = format!("migration plan for {}\n", report.source); + if let Some(cluster) = &report.cluster_yaml { + out.push_str("\n== team half -> cluster.yaml (ready to review) ==\n"); + out.push_str(cluster); + } + if !report.operator_merge.is_empty() { + out.push_str("\n== personal half -> ~/.omnigraph/config.yaml ==\n"); + for (key, value) in &report.operator_merge { + out.push_str(&format!(" {key}: {value}\n")); + } + } + if !report.dropped.is_empty() { + out.push_str("\n== no destination ==\n"); + for dropped in &report.dropped { + out.push_str(&format!(" {} — {}\n", dropped.key, dropped.reason)); + } + } + if !report.manual_steps.is_empty() { + out.push_str("\n== manual steps ==\n"); + for step in &report.manual_steps { + out.push_str(&format!(" - {step}\n")); + } + } + out.push_str("\n(nothing written; pass --write to apply the operator merge and emit cluster.yaml)\n"); + out +} + +/// `--write`: merge the personal half into the operator config (key-level, +/// existing entries always win; the prior file is backed up) and write the +/// team half to cluster.yaml in the legacy config's directory (or +/// cluster.yaml.proposed when one already exists). +pub(crate) fn apply_report(report: &MigrateReport, legacy_dir: &Path) -> Result<Vec<String>> { + let mut written = Vec::new(); + + if !report.operator_merge.is_empty() { + let dir = operator::operator_dir() + .ok_or_else(|| eyre!("no home directory resolvable for the operator config"))?; + std::fs::create_dir_all(&dir)?; + let path = dir.join(operator::OPERATOR_CONFIG_FILE); + let existing_text = std::fs::read_to_string(&path).unwrap_or_default(); + let mut mapping: serde_yaml::Mapping = if existing_text.trim().is_empty() { + serde_yaml::Mapping::new() + } else { + serde_yaml::from_str(&existing_text) + .map_err(|err| eyre!("operator config '{}' does not parse: {err}", path.display()))? + }; + let mut merged_any = false; + for (dotted, value_text) in &report.operator_merge { + if merge_dotted_if_absent(&mut mapping, dotted, value_text)? { + merged_any = true; + } + } + if merged_any { + if !existing_text.is_empty() { + let backup = path.with_extension("yaml.bak"); + std::fs::write(&backup, &existing_text)?; + written.push(format!("backed up prior operator config to {}", backup.display())); + } + let rendered = serde_yaml::to_string(&mapping)?; + let tmp = path.with_extension(format!("yaml.tmp.{}", std::process::id())); + std::fs::write(&tmp, &rendered)?; + std::fs::rename(&tmp, &path)?; + written.push(format!("merged personal keys into {}", path.display())); + } else { + written.push("operator config already carries every personal key (nothing merged)".into()); + } + } + + if let Some(cluster) = &report.cluster_yaml { + let target = legacy_dir.join("cluster.yaml"); + let target = if target.exists() { + legacy_dir.join("cluster.yaml.proposed") + } else { + target + }; + std::fs::write(&target, cluster)?; + written.push(format!("wrote team-half proposal to {}", target.display())); + } + + Ok(written) +} + +/// Set `a.b.c` in the mapping only when absent; returns whether it wrote. +fn merge_dotted_if_absent( + mapping: &mut serde_yaml::Mapping, + dotted: &str, + value_text: &str, +) -> Result<bool> { + let value: serde_yaml::Value = + serde_yaml::from_str(value_text).unwrap_or(serde_yaml::Value::String(value_text.into())); + let parts: Vec<&str> = dotted.split('.').collect(); + let mut current = mapping; + for part in &parts[..parts.len() - 1] { + let key = serde_yaml::Value::String((*part).into()); + let entry = current + .entry(key) + .or_insert_with(|| serde_yaml::Value::Mapping(serde_yaml::Mapping::new())); + current = entry + .as_mapping_mut() + .ok_or_else(|| eyre!("operator config key '{dotted}' collides with a non-mapping"))?; + } + let leaf = serde_yaml::Value::String(parts[parts.len() - 1].into()); + if current.contains_key(&leaf) { + return Ok(false); + } + current.insert(leaf, value); + Ok(true) +} + +pub(crate) fn legacy_config_path(explicit: Option<&PathBuf>) -> PathBuf { + explicit.cloned().unwrap_or_else(|| PathBuf::from("omnigraph.yaml")) +} + +#[cfg(test)] +mod tests { + use super::*; + use omnigraph_server::config::load_config; + + fn full_legacy_fixture(dir: &Path) -> PathBuf { + let path = dir.join("omnigraph.yaml"); + std::fs::write( + &path, + r#" +project: { name: brain } +graphs: + prod: + uri: https://graph.example.com + bearer_token_env: PROD_TOKEN + policy: { file: ./prod.policy.yaml } + queries: + find: { file: ./find.gq } + local: + uri: /tmp/local.omni +server: { bind: "0.0.0.0:9999", policy: { file: ./server.policy.yaml } } +auth: { env_file: .env.omni } +cli: + graph: prod + branch: main + actor: act-me + output_format: json + table_max_column_width: 40 +query: { roots: ["."] } +aliases: + triage: { command: query, query: ./triage.gq, name: weekly_triage, args: [since], graph: prod } +policy: { file: ./top.policy.yaml } +queries: + top_q: { file: ./top.gq } +"#, + ) + .unwrap(); + path + } + + /// The RFC-008 completeness contract: every top-level key of the + /// legacy schema must appear in the report somewhere (team half, + /// operator merge, dropped, or manual steps). + #[test] + fn every_legacy_key_is_classified() { + let dir = tempfile::tempdir().unwrap(); + let path = full_legacy_fixture(dir.path()); + let config = load_config(Some(&path)).unwrap(); + let report = build_report(&config, &path); + let rendered = render_report(&report); + + let serialized = + serde_yaml::to_value(OmnigraphConfig::default()).expect("default serializes"); + for key in serialized.as_mapping().unwrap().keys() { + let key = key.as_str().unwrap(); + assert!( + rendered.contains(key) + || report.operator_merge.keys().any(|k| k.contains(key)) + || matches!(key, "graphs" | "queries" | "policy" | "project") + && report.cluster_yaml.is_some(), + "legacy key '{key}' is unclassified — fix the RFC-008 map: {rendered}" + ); + } + + // spot checks on each section + assert_eq!(report.operator_merge["operator.actor"], "act-me"); + assert_eq!(report.operator_merge["defaults.output"], "json"); + assert_eq!( + report.operator_merge["servers.prod.url"], + "https://graph.example.com" + ); + assert!(report.operator_merge["aliases.triage"].contains("query: weekly_triage")); + let cluster = report.cluster_yaml.as_deref().unwrap(); + assert!(cluster.contains("version: 1")); + assert!(cluster.contains("name: brain")); + assert!(cluster.contains(" prod:")); + assert!(cluster.contains("applies_to: [cluster]")); + assert!(cluster.contains("applies_to: [graph.prod]")); + assert!(report.dropped.iter().any(|d| d.key == "query.roots")); + assert!(report.dropped.iter().any(|d| d.key.contains("server.bind"))); + assert!( + report + .manual_steps + .iter() + .any(|s| s.contains("omnigraph login prod")) + ); + } + + #[test] + fn merge_dotted_never_clobbers_existing() { + let mut mapping: serde_yaml::Mapping = + serde_yaml::from_str("operator:\n actor: keep-me\n").unwrap(); + assert!(!merge_dotted_if_absent(&mut mapping, "operator.actor", "new").unwrap()); + assert!(merge_dotted_if_absent(&mut mapping, "defaults.output", "json").unwrap()); + let text = serde_yaml::to_string(&mapping).unwrap(); + assert!(text.contains("keep-me") && !text.contains("new")); + assert!(text.contains("output: json")); + } +} diff --git a/crates/omnigraph-cli/src/operator.rs b/crates/omnigraph-cli/src/operator.rs index 16f5550..fb8658d 100644 --- a/crates/omnigraph-cli/src/operator.rs +++ b/crates/omnigraph-cli/src/operator.rs @@ -91,6 +91,10 @@ pub(crate) struct OperatorIdentity { pub(crate) struct OperatorDefaults { /// Default read output format, below every more-specific source. pub(crate) output: Option<ReadOutputFormat>, + /// Table rendering preferences (below the legacy cli.table_* keys + /// during the RFC-008 window). + pub(crate) table_max_column_width: Option<usize>, + pub(crate) table_cell_layout: Option<omnigraph_server::config::TableCellLayout>, #[serde(flatten)] unknown: serde_yaml::Mapping, } diff --git a/crates/omnigraph-cli/src/output.rs b/crates/omnigraph-cli/src/output.rs index 04df60a..964307b 100644 --- a/crates/omnigraph-cli/src/output.rs +++ b/crates/omnigraph-cli/src/output.rs @@ -716,10 +716,7 @@ pub(crate) fn print_read_output( render_read( output, format, - &ReadRenderOptions { - max_column_width: config.table_max_column_width(), - cell_layout: config.table_cell_layout(), - }, + &resolve_table_render_options(config), )? ); Ok(()) @@ -873,3 +870,21 @@ pub(crate) fn finish_logout( } Ok(()) } + +/// Table prefs cascade (RFC-007/008): legacy cli.table_* (window) > +/// operator defaults.table_* > built-in. +pub(crate) fn resolve_table_render_options(config: &OmnigraphConfig) -> ReadRenderOptions { + let operator = crate::operator::load_operator_config().unwrap_or_default(); + ReadRenderOptions { + max_column_width: config + .cli + .table_max_column_width + .or(operator.defaults.table_max_column_width) + .unwrap_or(80), + cell_layout: config + .cli + .table_cell_layout + .or(operator.defaults.table_cell_layout) + .unwrap_or_default(), + } +} diff --git a/crates/omnigraph-cli/tests/cli_schema_config.rs b/crates/omnigraph-cli/tests/cli_schema_config.rs index a55aaed..20281f6 100644 --- a/crates/omnigraph-cli/tests/cli_schema_config.rs +++ b/crates/omnigraph-cli/tests/cli_schema_config.rs @@ -538,3 +538,74 @@ fn legacy_config_load_warns_per_key_and_suppression_silences() { let stderr = String::from_utf8_lossy(&output.stderr); assert!(!stderr.contains("deprecated (RFC-008)"), "{stderr}"); } + +/// RFC-008 stage 2: `config migrate` proposes the split read-only, applies +/// it with --write (operator merge never clobbers; cluster.yaml emitted), +/// and a second --write is idempotent. +#[test] +fn config_migrate_splits_legacy_config() { + let temp = tempdir().unwrap(); + fs::write( + temp.path().join("omnigraph.yaml"), + "graphs:\n prod:\n uri: https://graph.example.com\n bearer_token_env: PROD_TOKEN\ncli:\n actor: act-me\n output_format: json\npolicy:\n file: ./top.policy.yaml\n", + ) + .unwrap(); + let operator_home = tempfile::tempdir().unwrap(); + fs::write( + operator_home.path().join("config.yaml"), + "operator:\n actor: act-existing\n", + ) + .unwrap(); + + // Read-only proposal: names both halves, writes nothing. + let output = cli() + .current_dir(temp.path()) + .env("OMNIGRAPH_HOME", operator_home.path()) + .env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1") + .arg("config") + .arg("migrate") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + let stdout = String::from_utf8_lossy(&output.stdout); + assert!(stdout.contains("team half -> cluster.yaml"), "{stdout}"); + assert!(stdout.contains("operator.actor: act-me"), "{stdout}"); + assert!(stdout.contains("omnigraph login prod"), "{stdout}"); + assert!(!temp.path().join("cluster.yaml").exists()); + + // --write: cluster.yaml lands; the existing operator actor is KEPT. + let output = cli() + .current_dir(temp.path()) + .env("OMNIGRAPH_HOME", operator_home.path()) + .env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1") + .arg("config") + .arg("migrate") + .arg("--write") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + let cluster = fs::read_to_string(temp.path().join("cluster.yaml")).unwrap(); + assert!(cluster.contains("version: 1") && cluster.contains(" prod:"), "{cluster}"); + let operator_text = + fs::read_to_string(operator_home.path().join("config.yaml")).unwrap(); + assert!(operator_text.contains("act-existing"), "{operator_text}"); + assert!(!operator_text.contains("act-me"), "existing keys win: {operator_text}"); + assert!(operator_text.contains("output: json"), "{operator_text}"); + assert!( + operator_text.contains("url: https://graph.example.com"), + "{operator_text}" + ); + + // Second --write: cluster.yaml exists -> proposal file, no clobber. + let output = cli() + .current_dir(temp.path()) + .env("OMNIGRAPH_HOME", operator_home.path()) + .env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1") + .arg("config") + .arg("migrate") + .arg("--write") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + assert!(temp.path().join("cluster.yaml.proposed").exists()); +} From 5ba9656666c5512780c14a448f2ecb1c1e48d467 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 23:34:04 +0300 Subject: [PATCH 131/165] feat(cli): init stops scaffolding omnigraph.yaml; cluster init replaces it (RFC-008 stage 3) omnigraph init no longer writes a legacy config into cwd (the source of the earlier test-pollution bug, and a scaffold for a deprecated file); the scaffolder is deleted. omnigraph cluster init scaffolds the replacement: a minimal valid cluster.yaml (version: 1, optional metadata.name / storage:, a commented graphs example), refusing to overwrite. The scaffold validates clean via cluster validate in the e2e. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/cli.rs | 17 +++++ crates/omnigraph-cli/src/helpers.rs | 63 ------------------- crates/omnigraph-cli/src/main.rs | 38 ++++++++++- crates/omnigraph-cli/tests/cli_cluster.rs | 47 ++++++++++++++ .../omnigraph-cli/tests/cli_schema_config.rs | 3 +- 5 files changed, 103 insertions(+), 65 deletions(-) diff --git a/crates/omnigraph-cli/src/cli.rs b/crates/omnigraph-cli/src/cli.rs index 7b976b4..6b89f0e 100644 --- a/crates/omnigraph-cli/src/cli.rs +++ b/crates/omnigraph-cli/src/cli.rs @@ -345,6 +345,23 @@ pub(crate) enum Command { #[derive(Debug, Subcommand)] pub(crate) enum ClusterCommand { + /// Scaffold a minimal cluster.yaml in the config directory (refuses + /// if one exists). The cluster checkout replaces the legacy + /// omnigraph.yaml scaffold (RFC-008 stage 3). + Init { + /// Directory to scaffold into (default: .) + #[arg(long, default_value = ".")] + config: PathBuf, + /// Optional deployment label (metadata.name) + #[arg(long)] + name: Option<String>, + /// Optional storage root URI (s3://bucket/prefix); omit for the + /// config-dir layout + #[arg(long)] + storage: Option<String>, + #[arg(long)] + json: bool, + }, /// Validate cluster.yaml and referenced schemas, queries, and policy files. Validate { /// Cluster config directory containing cluster.yaml. diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index a48f2e4..67fb6ea 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -677,69 +677,6 @@ pub(crate) fn normalize_legacy_alias_uri( (Some(candidate), alias_args) } -pub(crate) fn scaffold_config_if_missing(uri: &str) -> Result<()> { - let path = inferred_config_path(uri)?; - if path.exists() { - return Ok(()); - } - - fs::write( - path, - format!( - "\ -project: - name: Omnigraph Project - -graphs: - local: - uri: {} - # bearer_token_env: OMNIGRAPH_BEARER_TOKEN - -server: - graph: local - bind: 127.0.0.1:8080 - -cli: - graph: local - branch: main - output_format: table - table_max_column_width: 80 - table_cell_layout: truncate - -query: - roots: - - queries - - . - -aliases: - # owner: - # command: read - # query: context.gq - # name: decision_owner - # args: [slug] - # graph: local - # branch: main - # format: kv - # - # attach_trace: - # command: change - # query: mutations.gq - # name: attach_trace - # args: [decision_slug, trace_slug] - # graph: local - # branch: main - -# auth: -# env_file: ./.env.omni -# -# policy: -# file: ./policy.yaml -", - yaml_string(uri), - ), - )?; - Ok(()) -} pub(crate) fn inferred_config_path(uri: &str) -> Result<PathBuf> { if uri.contains("://") { diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 133a8a0..074ca25 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -153,7 +153,6 @@ async fn main() -> Result<()> { omnigraph::db::InitOptions { force }, ) .await?; - scaffold_config_if_missing(&uri)?; println!("initialized {}", uri); } Command::Load { @@ -1218,6 +1217,43 @@ async fn main() -> Result<()> { } } Command::Cluster { command } => match command { + ClusterCommand::Init { + config, + name, + storage, + json, + } => { + let target = config.join("cluster.yaml"); + if target.exists() { + bail!( + "'{}' already exists — cluster init refuses to overwrite", + target.display() + ); + } + std::fs::create_dir_all(&config)?; + let mut scaffold = String::from("version: 1\n"); + if let Some(name) = &name { + scaffold.push_str(&format!("metadata:\n name: {name}\n")); + } + match &storage { + Some(root) => scaffold.push_str(&format!("storage: {root}\n")), + None => scaffold.push_str( + "# storage: s3://bucket/prefix # omit: this folder is the storage root\n", + ), + } + scaffold.push_str( + "graphs: {}\n# graphs:\n# knowledge:\n# schema: knowledge.pg\n# queries: queries/\n", + ); + std::fs::write(&target, scaffold)?; + if json { + print_json(&serde_json::json!({ "created": target.display().to_string() }))?; + } else { + println!( + "created {} — declare graphs, then `omnigraph cluster import` and `apply`", + target.display() + ); + } + } ClusterCommand::Validate { config, json } => { let output = validate_config_dir(config); finish_cluster_validate(&output, json)?; diff --git a/crates/omnigraph-cli/tests/cli_cluster.rs b/crates/omnigraph-cli/tests/cli_cluster.rs index bfadf40..9f0326a 100644 --- a/crates/omnigraph-cli/tests/cli_cluster.rs +++ b/crates/omnigraph-cli/tests/cli_cluster.rs @@ -949,3 +949,50 @@ graphs: let leaked = b.to_string(); assert!(!leaked.contains("phantom") && !leaked.contains("9999"), "{leaked}"); } + +/// RFC-008 stage 3: `cluster init` scaffolds a minimal valid cluster.yaml +/// (the replacement for the retired omnigraph.yaml scaffold) and refuses +/// to overwrite. +#[test] +fn cluster_init_scaffolds_minimal_valid_config_and_refuses_overwrite() { + let temp = tempdir().unwrap(); + let output = cli() + .arg("cluster") + .arg("init") + .arg("--config") + .arg(temp.path()) + .arg("--name") + .arg("brain") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + let text = fs::read_to_string(temp.path().join("cluster.yaml")).unwrap(); + assert!(text.contains("version: 1") && text.contains("name: brain"), "{text}"); + + // The scaffold validates clean. + let output = cli() + .arg("cluster") + .arg("validate") + .arg("--config") + .arg(temp.path()) + .arg("--json") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + let payload: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(payload["ok"], true, "{payload}"); + + // Refuses a second run. + let output = cli() + .arg("cluster") + .arg("init") + .arg("--config") + .arg(temp.path()) + .output() + .unwrap(); + assert!(!output.status.success()); + assert!( + String::from_utf8_lossy(&output.stderr).contains("refuses to overwrite"), + "{output:?}" + ); +} diff --git a/crates/omnigraph-cli/tests/cli_schema_config.rs b/crates/omnigraph-cli/tests/cli_schema_config.rs index 20281f6..3e2a2b9 100644 --- a/crates/omnigraph-cli/tests/cli_schema_config.rs +++ b/crates/omnigraph-cli/tests/cli_schema_config.rs @@ -36,7 +36,8 @@ fn init_creates_graph_successfully_on_missing_local_directory() { assert!(stdout.contains("initialized")); assert!(graph.join("_schema.pg").exists()); assert!(graph.join("__manifest").exists()); - assert!(temp.path().join("omnigraph.yaml").exists()); + // RFC-008 stage 3: init no longer scaffolds the legacy config file. + assert!(!temp.path().join("omnigraph.yaml").exists()); } #[test] From 3adbc65af2c0ae8028fa3e20048d9015d1fd89a5 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 23:37:12 +0300 Subject: [PATCH 132/165] docs(cli): config migrate, cluster init, the legacy-file deprecation notice Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/rfc-008-deprecate-omnigraph-yaml.md | 6 +++--- docs/user/cli-reference.md | 10 +++++++++- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/docs/dev/rfc-008-deprecate-omnigraph-yaml.md b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md index d496df8..e9c37a9 100644 --- a/docs/dev/rfc-008-deprecate-omnigraph-yaml.md +++ b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md @@ -112,19 +112,19 @@ Two placements worth defending: Per Hyrum's Law (the repo's own deny-list: shipped observable behavior is contract), retirement is staged, loud, and tooled: -1. **Warn.** Loading `omnigraph.yaml` emits a one-line deprecation notice +1. **Warn** *(landed)*. Loading `omnigraph.yaml` emits a one-line deprecation notice naming the replacement for each key actually present in the file (not a generic banner — the migration map above, applied to *your* file). Suppressible per-process (`OMNIGRAPH_SUPPRESS_YAML_DEPRECATION=1`) for CI logs during the window. -2. **Migrate.** `omnigraph config migrate` reads an existing +2. **Migrate** *(landed)*. `omnigraph config migrate` reads an existing `omnigraph.yaml` and writes the split: the team half as a ready-to-review `cluster.yaml` (+ moves query/policy files into the checkout layout), the personal half merged into `~/.omnigraph/config.yaml` — printing a diff-style summary and touching nothing without `--write`. The command is the test of the migration map's completeness: any key it cannot place is a bug in this RFC. -3. **Stop scaffolding.** `omnigraph init` stops generating +3. **Stop scaffolding** *(landed)*. `omnigraph init` stops generating `omnigraph.yaml` (it currently scaffolds one into cwd — the source of the test-pollution bug). `omnigraph cluster init` (new, small) scaffolds a minimal `cluster.yaml` instead. diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index b113ef3..e787ca2 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -8,7 +8,7 @@ Top-level command families and subcommands. Graph-targeting commands accept a po | Command | Purpose | |---|---| -| `init` | `--schema <pg>` → initialize a graph (also scaffolds `omnigraph.yaml` if missing) | +| `init` | `--schema <pg>` → initialize a graph (no longer scaffolds `omnigraph.yaml` — RFC-008; use `cluster init` for a config scaffold) | | `load` | bulk load a branch, local or remote (`--mode overwrite\|append\|merge` is **required** — overwrite is destructive, so there is no default). Without `--from` the target branch must exist; `--from <base>` forks a missing `--branch` from `<base>` first | | `ingest` | deprecated alias of `load --from <base>` (defaults: `--from main --mode merge`); prints a one-line warning to stderr | | `query` (alias: `read`) | run named read query; source via `--query <path>`, `-e`/`--query-string <GQ>`, or `--alias <name>` (exactly one). `read` is the deprecated previous name and prints a one-line warning to stderr | @@ -19,6 +19,8 @@ Top-level command families and subcommands. Graph-targeting commands accept a po | `commit list \| show` | inspect commit graph | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | +| `cluster init` | scaffold a minimal `cluster.yaml` (`--name`, `--storage`); refuses to overwrite | +| `config migrate` | propose (or `--write`: apply) the RFC-008 split of a legacy `omnigraph.yaml` — team half → ready-to-review `cluster.yaml`, personal half → `~/.omnigraph/config.yaml` (key-level merge, existing entries win), plus dropped-key reasons and manual steps | | `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | declarative cluster control plane. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`, annotates dispositions, and embeds real schema-migration previews; `apply` converges the cluster — stored-query/policy catalog writes (content-addressed under `__cluster/resources/`), graph creates, schema updates (soft drops only; `--as` records the actor), and graph deletes behind a digest-bound approval from `cluster approve <resource> --as <actor>` (`apply`/`approve` default the actor from the per-operator `omnigraph.yaml`'s `cli.actor` when `--as` is omitted; nothing else in that file affects cluster commands); what apply converges is what an `omnigraph-server --cluster <dir>` deployment serves on its next restart (omnigraph.yaml deployments are unaffected); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) | | `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review | @@ -104,6 +106,12 @@ operator server use the legacy chain alone. ## `omnigraph.yaml` schema (legacy combined file) +> **Deprecated (RFC-008).** Loading this file prints a per-key notice +> naming each present key's new home (suppress in CI with +> `OMNIGRAPH_SUPPRESS_YAML_DEPRECATION=1`); `omnigraph config migrate` +> produces the split. The file keeps working through the deprecation +> window. + ```yaml project: { name } graphs: From 5328c91341210cab18bf80b5d83753a7d806f90e Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Thu, 11 Jun 2026 23:45:18 +0300 Subject: [PATCH 133/165] =?UTF-8?q?refactor(cli):=20drop=20cluster=20init?= =?UTF-8?q?=20=E2=80=94=20no=20replacement=20scaffold?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Andrew's call, and the right one by the repo's own lens: a minimal cluster.yaml is five lines; a generator is a second copy of the schema to keep in sync forever, emitting a file that is unusable until hand-edited anyway (graphs: {} cannot apply or serve). Terraform has no config scaffolder either. New users copy from the cluster quick-start; migrants get a ready-to-review cluster.yaml from config migrate. RFC-008 stage 3 becomes purely subtractive. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/cli.rs | 17 -------- crates/omnigraph-cli/src/main.rs | 37 ---------------- crates/omnigraph-cli/tests/cli_cluster.rs | 46 -------------------- docs/dev/rfc-008-deprecate-omnigraph-yaml.md | 10 +++-- docs/user/cli-reference.md | 3 +- 5 files changed, 8 insertions(+), 105 deletions(-) diff --git a/crates/omnigraph-cli/src/cli.rs b/crates/omnigraph-cli/src/cli.rs index 6b89f0e..7b976b4 100644 --- a/crates/omnigraph-cli/src/cli.rs +++ b/crates/omnigraph-cli/src/cli.rs @@ -345,23 +345,6 @@ pub(crate) enum Command { #[derive(Debug, Subcommand)] pub(crate) enum ClusterCommand { - /// Scaffold a minimal cluster.yaml in the config directory (refuses - /// if one exists). The cluster checkout replaces the legacy - /// omnigraph.yaml scaffold (RFC-008 stage 3). - Init { - /// Directory to scaffold into (default: .) - #[arg(long, default_value = ".")] - config: PathBuf, - /// Optional deployment label (metadata.name) - #[arg(long)] - name: Option<String>, - /// Optional storage root URI (s3://bucket/prefix); omit for the - /// config-dir layout - #[arg(long)] - storage: Option<String>, - #[arg(long)] - json: bool, - }, /// Validate cluster.yaml and referenced schemas, queries, and policy files. Validate { /// Cluster config directory containing cluster.yaml. diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 074ca25..8178e65 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -1217,43 +1217,6 @@ async fn main() -> Result<()> { } } Command::Cluster { command } => match command { - ClusterCommand::Init { - config, - name, - storage, - json, - } => { - let target = config.join("cluster.yaml"); - if target.exists() { - bail!( - "'{}' already exists — cluster init refuses to overwrite", - target.display() - ); - } - std::fs::create_dir_all(&config)?; - let mut scaffold = String::from("version: 1\n"); - if let Some(name) = &name { - scaffold.push_str(&format!("metadata:\n name: {name}\n")); - } - match &storage { - Some(root) => scaffold.push_str(&format!("storage: {root}\n")), - None => scaffold.push_str( - "# storage: s3://bucket/prefix # omit: this folder is the storage root\n", - ), - } - scaffold.push_str( - "graphs: {}\n# graphs:\n# knowledge:\n# schema: knowledge.pg\n# queries: queries/\n", - ); - std::fs::write(&target, scaffold)?; - if json { - print_json(&serde_json::json!({ "created": target.display().to_string() }))?; - } else { - println!( - "created {} — declare graphs, then `omnigraph cluster import` and `apply`", - target.display() - ); - } - } ClusterCommand::Validate { config, json } => { let output = validate_config_dir(config); finish_cluster_validate(&output, json)?; diff --git a/crates/omnigraph-cli/tests/cli_cluster.rs b/crates/omnigraph-cli/tests/cli_cluster.rs index 9f0326a..3b2eed3 100644 --- a/crates/omnigraph-cli/tests/cli_cluster.rs +++ b/crates/omnigraph-cli/tests/cli_cluster.rs @@ -950,49 +950,3 @@ graphs: assert!(!leaked.contains("phantom") && !leaked.contains("9999"), "{leaked}"); } -/// RFC-008 stage 3: `cluster init` scaffolds a minimal valid cluster.yaml -/// (the replacement for the retired omnigraph.yaml scaffold) and refuses -/// to overwrite. -#[test] -fn cluster_init_scaffolds_minimal_valid_config_and_refuses_overwrite() { - let temp = tempdir().unwrap(); - let output = cli() - .arg("cluster") - .arg("init") - .arg("--config") - .arg(temp.path()) - .arg("--name") - .arg("brain") - .output() - .unwrap(); - assert!(output.status.success(), "{output:?}"); - let text = fs::read_to_string(temp.path().join("cluster.yaml")).unwrap(); - assert!(text.contains("version: 1") && text.contains("name: brain"), "{text}"); - - // The scaffold validates clean. - let output = cli() - .arg("cluster") - .arg("validate") - .arg("--config") - .arg(temp.path()) - .arg("--json") - .output() - .unwrap(); - assert!(output.status.success(), "{output:?}"); - let payload: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap(); - assert_eq!(payload["ok"], true, "{payload}"); - - // Refuses a second run. - let output = cli() - .arg("cluster") - .arg("init") - .arg("--config") - .arg(temp.path()) - .output() - .unwrap(); - assert!(!output.status.success()); - assert!( - String::from_utf8_lossy(&output.stderr).contains("refuses to overwrite"), - "{output:?}" - ); -} diff --git a/docs/dev/rfc-008-deprecate-omnigraph-yaml.md b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md index e9c37a9..a59be52 100644 --- a/docs/dev/rfc-008-deprecate-omnigraph-yaml.md +++ b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md @@ -125,9 +125,13 @@ contract), retirement is staged, loud, and tooled: is the test of the migration map's completeness: any key it cannot place is a bug in this RFC. 3. **Stop scaffolding** *(landed)*. `omnigraph init` stops generating - `omnigraph.yaml` (it currently scaffolds one into cwd — the source of - the test-pollution bug). `omnigraph cluster init` (new, small) scaffolds - a minimal `cluster.yaml` instead. + `omnigraph.yaml` (it scaffolded one into cwd — the source of the + test-pollution bug). **No replacement scaffold**: a minimal + `cluster.yaml` is five lines; a generator would be a second copy of the + schema to keep in sync, producing a file that is unusable until + hand-edited anyway (Terraform has no config scaffolder either). New + users copy from the cluster quick-start; migrants get a ready-to-review + `cluster.yaml` from `config migrate`. 4. **Opt-in strict.** `OMNIGRAPH_NO_LEGACY_CONFIG=1` turns the warning into an error — for teams that finished migrating and want regressions caught. 5. **Remove at the next major.** Loading the file becomes an error pointing diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index e787ca2..62e3e0d 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -8,7 +8,7 @@ Top-level command families and subcommands. Graph-targeting commands accept a po | Command | Purpose | |---|---| -| `init` | `--schema <pg>` → initialize a graph (no longer scaffolds `omnigraph.yaml` — RFC-008; use `cluster init` for a config scaffold) | +| `init` | `--schema <pg>` → initialize a graph (no longer scaffolds `omnigraph.yaml` — RFC-008; start cluster configs from the [cluster.md](cluster.md) quick-start or `config migrate`) | | `load` | bulk load a branch, local or remote (`--mode overwrite\|append\|merge` is **required** — overwrite is destructive, so there is no default). Without `--from` the target branch must exist; `--from <base>` forks a missing `--branch` from `<base>` first | | `ingest` | deprecated alias of `load --from <base>` (defaults: `--from main --mode merge`); prints a one-line warning to stderr | | `query` (alias: `read`) | run named read query; source via `--query <path>`, `-e`/`--query-string <GQ>`, or `--alias <name>` (exactly one). `read` is the deprecated previous name and prints a one-line warning to stderr | @@ -19,7 +19,6 @@ Top-level command families and subcommands. Graph-targeting commands accept a po | `commit list \| show` | inspect commit graph | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | -| `cluster init` | scaffold a minimal `cluster.yaml` (`--name`, `--storage`); refuses to overwrite | | `config migrate` | propose (or `--write`: apply) the RFC-008 split of a legacy `omnigraph.yaml` — team half → ready-to-review `cluster.yaml`, personal half → `~/.omnigraph/config.yaml` (key-level merge, existing entries win), plus dropped-key reasons and manual steps | | `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | declarative cluster control plane. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`, annotates dispositions, and embeds real schema-migration previews; `apply` converges the cluster — stored-query/policy catalog writes (content-addressed under `__cluster/resources/`), graph creates, schema updates (soft drops only; `--as` records the actor), and graph deletes behind a digest-bound approval from `cluster approve <resource> --as <actor>` (`apply`/`approve` default the actor from the per-operator `omnigraph.yaml`'s `cli.actor` when `--as` is omitted; nothing else in that file affects cluster commands); what apply converges is what an `omnigraph-server --cluster <dir>` deployment serves on its next restart (omnigraph.yaml deployments are unaffected); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) | From 4c50170c773de9a36ee126c551f814a91545656b Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Fri, 12 Jun 2026 00:03:10 +0300 Subject: [PATCH 134/165] feat(config): OMNIGRAPH_NO_LEGACY_CONFIG strict mode (RFC-008 stage 4) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Opt-in: with the env set, loading a legacy omnigraph.yaml is a hard error pointing at config migrate — the regression guard for migrated teams (a stray legacy file would otherwise silently outrank operator config during the window) and the rehearsal for stage 5's removal. Strict refuses the FILE, never its absence: flag-less invocations on migrated setups are untouched. Inert unless set. The RFC's stages-1-3-then-4 release gap collapsed honestly: no version boundary was crossed between them, so all four ship in the same release (noted in the RFC). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- .../omnigraph-cli/tests/cli_schema_config.rs | 36 +++++++++ crates/omnigraph-server/src/config.rs | 74 ++++++++++++++----- docs/dev/rfc-008-deprecate-omnigraph-yaml.md | 2 +- docs/user/cli-reference.md | 4 +- 4 files changed, 95 insertions(+), 21 deletions(-) diff --git a/crates/omnigraph-cli/tests/cli_schema_config.rs b/crates/omnigraph-cli/tests/cli_schema_config.rs index 3e2a2b9..710c856 100644 --- a/crates/omnigraph-cli/tests/cli_schema_config.rs +++ b/crates/omnigraph-cli/tests/cli_schema_config.rs @@ -610,3 +610,39 @@ fn config_migrate_splits_legacy_config() { assert!(output.status.success(), "{output:?}"); assert!(temp.path().join("cluster.yaml.proposed").exists()); } + +/// RFC-008 stage 4: OMNIGRAPH_NO_LEGACY_CONFIG refuses a present legacy +/// file (pointing at config migrate) but changes nothing on migrated +/// setups with no file. +#[test] +fn strict_mode_refuses_legacy_file_but_not_its_absence() { + let temp = tempdir().unwrap(); + fs::write(temp.path().join("omnigraph.yaml"), "cli:\n actor: a\n").unwrap(); + let output = cli() + .current_dir(temp.path()) + .env("OMNIGRAPH_NO_LEGACY_CONFIG", "1") + .arg("graphs") + .arg("list") + .arg("--json") + .output() + .unwrap(); + assert!(!output.status.success()); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("OMNIGRAPH_NO_LEGACY_CONFIG") && stderr.contains("config migrate"), + "{stderr}" + ); + + // Migrated setup (no file): strict mode is a no-op — a config-loading + // command that tolerates empty defaults succeeds. + let clean = tempdir().unwrap(); + let output = cli() + .current_dir(clean.path()) + .env("OMNIGRAPH_NO_LEGACY_CONFIG", "1") + .arg("queries") + .arg("list") + .arg("--json") + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); +} diff --git a/crates/omnigraph-server/src/config.rs b/crates/omnigraph-server/src/config.rs index ea7ce30..15b957d 100644 --- a/crates/omnigraph-server/src/config.rs +++ b/crates/omnigraph-server/src/config.rs @@ -531,15 +531,24 @@ pub fn default_config_path() -> PathBuf { /// uses it for the server; RFC-007 §D1 extends it to the CLI). pub const CONFIG_PATH_ENV: &str = "OMNIGRAPH_CONFIG"; +/// RFC-008 stage 4 — opt-in strict mode: when set, loading a legacy +/// `omnigraph.yaml` is a hard error instead of a warning. For teams that +/// finished migrating and want regressions caught (a stray legacy file +/// would otherwise silently outrank operator config during the window). +/// The rehearsal for stage 5's removal. +pub const NO_LEGACY_CONFIG_ENV: &str = "OMNIGRAPH_NO_LEGACY_CONFIG"; + pub fn load_config(config_path: Option<&PathBuf>) -> Result<OmnigraphConfig> { let env_path = env::var_os(CONFIG_PATH_ENV).map(PathBuf::from); - load_config_in(&env::current_dir()?, config_path, env_path.as_ref()) + let strict = env::var_os(NO_LEGACY_CONFIG_ENV).is_some(); + load_config_in(&env::current_dir()?, config_path, env_path.as_ref(), strict) } fn load_config_in( cwd: &Path, config_path: Option<&PathBuf>, env_path: Option<&PathBuf>, + strict_no_legacy: bool, ) -> Result<OmnigraphConfig> { // Precedence: explicit --config flag > $OMNIGRAPH_CONFIG > ./omnigraph.yaml. let explicit_path = config_path.or(env_path).cloned(); @@ -549,6 +558,14 @@ fn load_config_in( }); let mut config = if let Some(path) = &config_path { + if strict_no_legacy { + // Strict refuses the FILE, not its absence — flag-less + // invocations on migrated setups keep working. + bail!( + "legacy config '{}' refused: {NO_LEGACY_CONFIG_ENV} is set (RFC-008 strict mode); run `omnigraph config migrate`, then remove the file — or unset the variable", + path.display() + ); + } let text = fs::read_to_string(path)?; warn_yaml_deprecation_once(path, &text); serde_yaml::from_str::<OmnigraphConfig>(&text)? @@ -665,19 +682,38 @@ mod tests { fs::write(&env_path, "cli:\n actor: act-env\n").unwrap(); // $OMNIGRAPH_CONFIG used when no flag… - let config = load_config_in(temp.path(), None, Some(&env_path)).unwrap(); + let config = load_config_in(temp.path(), None, Some(&env_path), false).unwrap(); assert_eq!(config.cli.actor.as_deref(), Some("act-env")); // …loses to an explicit --config… - let config = load_config_in(temp.path(), Some(&flag_path), Some(&env_path)).unwrap(); + let config = load_config_in(temp.path(), Some(&flag_path), Some(&env_path), false).unwrap(); assert_eq!(config.cli.actor.as_deref(), Some("act-flag")); // …and beats the cwd default file. fs::write(temp.path().join("omnigraph.yaml"), "cli:\n actor: act-cwd\n").unwrap(); - let config = load_config_in(temp.path(), None, Some(&env_path)).unwrap(); + let config = load_config_in(temp.path(), None, Some(&env_path), false).unwrap(); assert_eq!(config.cli.actor.as_deref(), Some("act-env")); } + #[test] + fn strict_mode_refuses_the_file_not_its_absence() { + let temp = tempdir().unwrap(); + // No file: strict mode changes nothing (defaults load). + let config = load_config_in(temp.path(), None, None, true).unwrap(); + assert!(config.cli.actor.is_none()); + + // File present: strict refuses with the migrate pointer. + fs::write(temp.path().join("omnigraph.yaml"), "cli:\n actor: a\n").unwrap(); + let err = load_config_in(temp.path(), None, None, true).unwrap_err(); + let message = err.to_string(); + assert!( + message.contains("OMNIGRAPH_NO_LEGACY_CONFIG") && message.contains("config migrate"), + "{message}" + ); + // Without strict, the same file loads. + assert!(load_config_in(temp.path(), None, None, false).is_ok()); + } + #[test] fn yaml_deprecation_lines_name_present_keys_only() { let lines = super::yaml_deprecation_lines( @@ -717,7 +753,7 @@ policy: {} ) .unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); assert_eq!(config.cli_graph_name(), Some("local")); assert_eq!(config.cli_branch(), "main"); assert_eq!(config.cli_output_format(), ReadOutputFormat::Kv); @@ -752,7 +788,7 @@ policy: {} ) .unwrap(); - let config = load_config_in(&child, None, None).unwrap(); + let config = load_config_in(&child, None, None, false).unwrap(); assert!(config.graphs.is_empty()); } @@ -776,7 +812,7 @@ policy: {} "graphs:\n local:\n uri: ./demo.omni\n", ) .unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); // A known graph passes through unchanged. assert_eq!(config.resolve_graph_selection(Some("local")).unwrap(), Some("local")); @@ -799,7 +835,7 @@ policy: {} "graphs:\n local:\n uri: ./demo.omni\npolicy:\n file: ./top.yaml\n", ) .unwrap(); - let incoherent = load_config_in(temp2.path(), None, None).unwrap(); + let incoherent = load_config_in(temp2.path(), None, None, false).unwrap(); let err = incoherent .resolve_graph_selection(Some("local")) .unwrap_err() @@ -824,7 +860,7 @@ policy: {} server:\n graph: local\ncli:\n graph: prod\n", ) .unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); assert_eq!( config.resolve_policy_tooling_graph_selection().unwrap(), Some("prod") @@ -836,7 +872,7 @@ policy: {} "graphs:\n local:\n uri: ./local.omni\nserver:\n graph: local\n", ) .unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); assert_eq!( config.resolve_policy_tooling_graph_selection().unwrap(), Some("local") @@ -844,7 +880,7 @@ policy: {} let temp = tempdir().unwrap(); fs::write(temp.path().join("omnigraph.yaml"), "policy: {}\n").unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); assert_eq!(config.resolve_policy_tooling_graph_selection().unwrap(), None); let temp = tempdir().unwrap(); @@ -853,7 +889,7 @@ policy: {} "graphs:\n local:\n uri: ./local.omni\nserver:\n graph: ghost\n", ) .unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); let err = config .resolve_policy_tooling_graph_selection() .unwrap_err() @@ -879,7 +915,7 @@ policy: {} ) .unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); let resolved = config.resolve_query_path(Path::new("test.gq")).unwrap(); assert_eq!(resolved, temp.path().join("queries").join("test.gq")); } @@ -896,7 +932,7 @@ policy: {} fs::write(ambient_dir.join("local.gq"), "query ambient { return {} }").unwrap(); let config = - load_config_in(&ambient_dir, Some(&config_dir.join("omnigraph.yaml")), None).unwrap(); + load_config_in(&ambient_dir, Some(&config_dir.join("omnigraph.yaml")), None, false).unwrap(); let resolved = config.resolve_query_path(Path::new("local.gq")).unwrap(); assert_eq!(resolved, config_dir.join("local.gq")); @@ -926,7 +962,7 @@ queries: ) .unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); // Per-graph registry (multi-graph mode). let prod = config.target_query_entries("prod").unwrap(); @@ -967,7 +1003,7 @@ queries: policy:\n file: ./prod.yaml\n bare:\n uri: s3://b/bare\n", ) .unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); // Named graph with its own policy → per-graph (not top-level). assert!( @@ -1003,7 +1039,7 @@ queries: ) .unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); // Additive: no `queries:` anywhere → empty registries everywhere. assert!(config.query_entries().is_empty()); assert!( @@ -1023,7 +1059,7 @@ queries: ) .unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); assert_eq!( config.resolve_policy_file().unwrap(), temp.path().join("policy.yaml") @@ -1046,7 +1082,7 @@ cli: ) .unwrap(); - let config = load_config_in(temp.path(), None, None).unwrap(); + let config = load_config_in(temp.path(), None, None, false).unwrap(); assert_eq!( config.graph_bearer_token_env( Some("https://override.example.com"), diff --git a/docs/dev/rfc-008-deprecate-omnigraph-yaml.md b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md index a59be52..72baaf6 100644 --- a/docs/dev/rfc-008-deprecate-omnigraph-yaml.md +++ b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md @@ -132,7 +132,7 @@ contract), retirement is staged, loud, and tooled: hand-edited anyway (Terraform has no config scaffolder either). New users copy from the cluster quick-start; migrants get a ready-to-review `cluster.yaml` from `config migrate`. -4. **Opt-in strict.** `OMNIGRAPH_NO_LEGACY_CONFIG=1` turns the warning into +4. **Opt-in strict** *(landed — the release gap to stages 1–3 collapsed: no version boundary was crossed between them, so all four ship in the same release)*. `OMNIGRAPH_NO_LEGACY_CONFIG=1` turns the warning into an error — for teams that finished migrating and want regressions caught. 5. **Remove at the next major.** Loading the file becomes an error pointing at `config migrate`. The `OmnigraphConfig` code path, the dual diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 62e3e0d..b419adf 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -109,7 +109,9 @@ operator server use the legacy chain alone. > naming each present key's new home (suppress in CI with > `OMNIGRAPH_SUPPRESS_YAML_DEPRECATION=1`); `omnigraph config migrate` > produces the split. The file keeps working through the deprecation -> window. +> window. Migrated teams can set `OMNIGRAPH_NO_LEGACY_CONFIG=1` to turn +> any legacy-file load into a hard error (regression guard; the file's +> absence is always fine). ```yaml project: { name } From aabb3dca2eca9dcf19e21886992fd17855726156 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Fri, 12 Jun 2026 13:44:51 +0300 Subject: [PATCH 135/165] fix(storage): flush before acking in local write_text_if_absent MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit tokio's async File buffers writes internally: write_all only fills the buffer, and the actual OS write happens in a background task after drop — so write_text_if_absent could return Ok(true) with the file created but still EMPTY, and an immediate reader saw EOF. Caught twice in CI as 'EOF while parsing a value' reading state.json right after cluster import (the cluster's first state-write routes here since the storage port); also an invariant-6 violation (acknowledged before the write reached the OS). The other local write paths use tokio::fs::write, which flushes internally — this was the one miss. Fix: flush().await before Ok, with the same remove-on-failure cleanup as the write itself. Regression test is a best-effort tight loop (the window is timing-dependent; the two CI failures are the recorded red) asserting read-after-ack never sees a short file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph/src/storage.rs | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/crates/omnigraph/src/storage.rs b/crates/omnigraph/src/storage.rs index 978d1ce..187a6d6 100644 --- a/crates/omnigraph/src/storage.rs +++ b/crates/omnigraph/src/storage.rs @@ -139,6 +139,18 @@ impl StorageAdapter for LocalStorageAdapter { let _ = tokio::fs::remove_file(&path).await; return Err(err.into()); } + // tokio's async File buffers internally: without an explicit flush, + // write_all only fills the buffer and the actual OS write happens in + // a background task AFTER this fn returns — a reader can then see + // the created-but-still-empty file (caught twice in CI as an + // "EOF while parsing" on a state.json read right after import). + // Flushing before Ok restores write-then-read consistency, matching + // tokio::fs::write (which flushes internally) used by every other + // write path here. + if let Err(err) = file.flush().await { + let _ = tokio::fs::remove_file(&path).await; + return Err(err.into()); + } Ok(true) } @@ -599,6 +611,25 @@ fn env_var_truthy(key: &str) -> bool { #[cfg(test)] mod tests { + /// Regression for the write_text_if_absent buffering bug: a reader + /// immediately after Ok(true) must never see the created file empty. + /// The failure is timing-dependent (tokio's background write task), so + /// this loop is a best-effort local reproducer — the recorded red is + /// two CI failures ("EOF while parsing" on a state.json read right + /// after cluster import). + #[tokio::test(flavor = "multi_thread")] + async fn write_text_if_absent_is_read_consistent_immediately() { + let dir = tempfile::tempdir().unwrap(); + let adapter = super::storage_for_uri(&format!("file://{}", dir.path().display())).unwrap(); + let payload = "x".repeat(64 * 1024); + for i in 0..200 { + let uri = format!("file://{}/f{}.json", dir.path().display(), i); + assert!(adapter.write_text_if_absent(&uri, &payload).await.unwrap()); + let read = std::fs::read_to_string(dir.path().join(format!("f{i}.json"))).unwrap(); + assert_eq!(read.len(), payload.len(), "iteration {i}: short read"); + } + } + #[tokio::test] async fn local_versioned_cas_roundtrip() { let dir = tempfile::tempdir().unwrap(); From b24bb16d0cc110e7eef828e20ce252e87e90448d Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Fri, 12 Jun 2026 13:45:33 +0300 Subject: [PATCH 136/165] ci(codeowners): restore ragnorc to engineering and docs roles MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-adds ragnorc to both roles in the source of truth and regenerates CODEOWNERS + the ownership tables. This also resolves the standing inconsistency from #169: branch-protection.json's bypass_pull_request_allowances still listed ragnorc after his codeowners removal — the two lists are in sync again (no protection change needed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- .github/CODEOWNERS | 14 +++++++------- .github/codeowners-roles.yml | 2 ++ docs/dev/codeowners.md | 18 +++++++++--------- 3 files changed, 18 insertions(+), 16 deletions(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 3650f9e..ce8510c 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -8,11 +8,11 @@ # CI fails if this file drifts from its source, and rejects PRs that # edit this file directly without also editing the yml. -* @aaltshuler +* @aaltshuler @ragnorc -crates/** @aaltshuler -docs/** @aaltshuler -README.md @aaltshuler -AGENTS.md @aaltshuler -CLAUDE.md @aaltshuler -SECURITY.md @aaltshuler +crates/** @aaltshuler @ragnorc +docs/** @aaltshuler @ragnorc +README.md @aaltshuler @ragnorc +AGENTS.md @aaltshuler @ragnorc +CLAUDE.md @aaltshuler @ragnorc +SECURITY.md @aaltshuler @ragnorc diff --git a/.github/codeowners-roles.yml b/.github/codeowners-roles.yml index ed43c4a..65f2400 100644 --- a/.github/codeowners-roles.yml +++ b/.github/codeowners-roles.yml @@ -22,6 +22,7 @@ roles: compiler. members: - aaltshuler + - ragnorc docs: description: > @@ -29,6 +30,7 @@ roles: AGENTS.md, CLAUDE.md symlink, SECURITY.md). members: - aaltshuler + - ragnorc # Path → role mapping. GitHub CODEOWNERS uses "last match wins" # semantics — when multiple patterns match a file, only the last diff --git a/docs/dev/codeowners.md b/docs/dev/codeowners.md index 80d59e9..707f4f4 100644 --- a/docs/dev/codeowners.md +++ b/docs/dev/codeowners.md @@ -14,20 +14,20 @@ The tables below are **generated** from `.github/codeowners-roles.yml` by `.gith | Path | Owners | Role(s) | |---|---|---| -| `*` | @aaltshuler | engineering | -| `crates/**` | @aaltshuler | engineering | -| `docs/**` | @aaltshuler | docs | -| `README.md` | @aaltshuler | docs | -| `AGENTS.md` | @aaltshuler | docs | -| `CLAUDE.md` | @aaltshuler | docs | -| `SECURITY.md` | @aaltshuler | docs | +| `*` | @aaltshuler @ragnorc | engineering | +| `crates/**` | @aaltshuler @ragnorc | engineering | +| `docs/**` | @aaltshuler @ragnorc | docs | +| `README.md` | @aaltshuler @ragnorc | docs | +| `AGENTS.md` | @aaltshuler @ragnorc | docs | +| `CLAUDE.md` | @aaltshuler @ragnorc | docs | +| `SECURITY.md` | @aaltshuler @ragnorc | docs | **Roles**: | Role | Members | Description | |---|---|---| -| `engineering` | @aaltshuler | All production code under crates/**. Engine, CLI, server, compiler. | -| `docs` | @aaltshuler | Documentation under docs/**, plus repo-level docs (README.md, AGENTS.md, CLAUDE.md symlink, SECURITY.md). | +| `engineering` | @aaltshuler @ragnorc | All production code under crates/**. Engine, CLI, server, compiler. | +| `docs` | @aaltshuler @ragnorc | Documentation under docs/**, plus repo-level docs (README.md, AGENTS.md, CLAUDE.md symlink, SECURITY.md). | <!-- END GENERATED OWNERSHIP --> From dedd647cdef3fe30867e286ac076c7762e13be51 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Fri, 12 Jun 2026 13:21:03 +0300 Subject: [PATCH 137/165] release: bump workspace to 0.7.0 All six crate manifests + their path-dependency constraints, Cargo.lock, the regenerated openapi.json version metadata, AGENTS.md's surveyed version, and the v0.7.0 release notes (object-storage clusters, config-free --cluster serving, the operator config surface, keyed credentials, operator targeting/aliases, and the omnigraph.yaml deprecation stages). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- AGENTS.md | 12 ++-- Cargo.lock | 12 ++-- crates/omnigraph-cli/Cargo.toml | 12 ++-- crates/omnigraph-cluster/Cargo.toml | 6 +- crates/omnigraph-compiler/Cargo.toml | 2 +- crates/omnigraph-policy/Cargo.toml | 2 +- crates/omnigraph-server/Cargo.toml | 10 ++-- crates/omnigraph/Cargo.toml | 8 +-- docs/releases/v0.7.0.md | 90 ++++++++++++++++++++++++++++ openapi.json | 2 +- 10 files changed, 123 insertions(+), 33 deletions(-) create mode 100644 docs/releases/v0.7.0.md diff --git a/AGENTS.md b/AGENTS.md index b335955..87d6a46 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -16,7 +16,7 @@ Tools that support `@`-imports (Claude Code) auto-include all three files via th `CLAUDE.md` is a symlink to this file — there is exactly one source of truth. Edit `AGENTS.md`. -**Version surveyed:** 0.6.2 +**Version surveyed:** 0.7.0 **Workspace crates:** `omnigraph-compiler`, `omnigraph` (engine), `omnigraph-policy`, `omnigraph-cluster`, `omnigraph-cli`, `omnigraph-server` **Storage substrate:** Lance 6.x (columnar, versioned, branchable) **License:** MIT @@ -33,8 +33,8 @@ OmniGraph is a typed property-graph engine built as a coordination layer over ma - **Multi-modal querying**: vector ANN (`nearest`), full-text (`search`/`fuzzy`/`match_text`/`bm25`), Reciprocal Rank Fusion (`rrf`), and graph traversal (`Expand`, anti-join `not { … }`) in one runtime. - **Branches and commits across the whole graph**: Git-style — every successful publish appends to a commit DAG; merges are three-way at the row level. - **Atomic per-query writes**: `mutate_as` and `load` accumulate insert/update batches into an in-memory `MutationStaging.pending` per touched table; one `stage_*` + `commit_staged` per table runs at end-of-query, then `ManifestBatchPublisher::publish` commits the manifest atomically with per-table `expected_table_versions` CAS. A mid-query failure leaves Lance HEAD untouched on staged tables — no drift, no run state machine, no staging branches. Deletes still inline-commit; D₂ at parse time prevents inserts/updates and deletes from coexisting in one query. -- **HTTP server**: Axum + utoipa OpenAPI, bearer auth (SHA-256 hashed, optional AWS Secrets Manager). Cedar policy enforcement is engine-wide — every `_as` writer calls `Omnigraph::enforce(action, scope, actor)`, so HTTP, CLI, and embedded SDK consumers all hit the same gate. **Two modes** (v0.6.0+): single-graph (legacy flat routes) and multi-graph (`/graphs/{graph_id}/...` cluster routes + read-only `GET /graphs` enumeration). Per-graph + server-level Cedar policies. Runtime add/remove (`POST /graphs`, `DELETE /graphs/{id}`) is not exposed — operators edit `omnigraph.yaml` and restart. -- **CLI** driven by a single `omnigraph.yaml`; multi-format output (json/jsonl/csv/kv/table). +- **HTTP server**: Axum + utoipa OpenAPI, bearer auth (SHA-256 hashed, optional AWS Secrets Manager). Cedar policy enforcement is engine-wide — every `_as` writer calls `Omnigraph::enforce(action, scope, actor)`, so HTTP, CLI, and embedded SDK consumers all hit the same gate. **Two modes** (v0.6.0+): single-graph (legacy flat routes) and multi-graph (`/graphs/{graph_id}/...` cluster routes + read-only `GET /graphs` enumeration). Per-graph + server-level Cedar policies. Multi-graph mode boots from a cluster directory (`--cluster <dir | s3://…>`, RFC-005) or the legacy `omnigraph.yaml` `graphs:` map. Runtime add/remove (`POST /graphs`, `DELETE /graphs/{id}`) is not exposed — operators run `cluster apply` (or edit the legacy file) and restart. +- **CLI** with two-surface config (RFC-008): the team-owned cluster directory (`cluster.yaml`) plus the per-operator `~/.omnigraph/config.yaml` (servers, credentials, actor, aliases). The legacy combined `omnigraph.yaml` still loads with per-key deprecation warnings — `config migrate` proposes the split, `OMNIGRAPH_NO_LEGACY_CONFIG=1` enforces strict mode. **Never extend `omnigraph.yaml`.** Multi-format output (json/jsonl/csv/kv/table). Throughout the docs, capabilities are split into **L1 — Inherited from Lance** vs **L2 — Added by OmniGraph**. @@ -90,7 +90,7 @@ Full diagram and concurrency model: [docs/dev/architecture.md](docs/dev/architec | Cedar policy actions, scopes, CLI | [docs/user/policy.md](docs/user/policy.md) | | HTTP server endpoints, auth, error model, body limits | [docs/user/server.md](docs/user/server.md) | | CLI quick-start | [docs/user/cli.md](docs/user/cli.md) | -| CLI command surface and `omnigraph.yaml` schema | [docs/user/cli-reference.md](docs/user/cli-reference.md) | +| CLI command surface and config schemas (`~/.omnigraph/config.yaml`, legacy `omnigraph.yaml`) | [docs/user/cli-reference.md](docs/user/cli-reference.md) | | Audit / actor tracking | [docs/user/audit.md](docs/user/audit.md) | | Error taxonomy and result serialization | [docs/user/errors.md](docs/user/errors.md) | | Install (binary / Homebrew / source / channels) | [docs/user/install.md](docs/user/install.md) | @@ -258,8 +258,8 @@ omnigraph policy explain --actor act-alice --action change --branch main | Three-way row-level merge | — | `OrderedTableCursor` + `StagedTableWriter`, structured `MergeConflictKind` | | Change feeds | — | `diff_between` / `diff_commits` with manifest fast path + ID streaming | | Cedar policy | — | Per-graph actions plus server-scoped actions (see [docs/user/policy.md](docs/user/policy.md) for the current list), branch / target_branch / protected scopes, validate/test/explain CLI. **Engine-wide enforcement** (MR-722): every `_as` writer (`apply_schema_as`, `mutate_as`, `load_as` — the deprecated `ingest_as` shims route through it — `branch_create_as` / `branch_create_from_as`, `branch_delete_as`, `branch_merge_as`) calls `Omnigraph::enforce(action, scope, actor)` — HTTP, CLI, embedded SDK all hit the same gate. | -| HTTP server | — | Axum, OpenAPI via utoipa, bearer auth (SHA-256, AWS Secrets Manager option), `authorize_request` at the HTTP boundary (resolves bearer→actor, applies admission control), NDJSON streaming export, **multi-graph mode (v0.6.0+) with cluster routes + read-only `GET /graphs` enumeration + per-graph + server-level Cedar policies. Add/remove graphs by editing `omnigraph.yaml` and restarting.** | -| CLI with config | — | `omnigraph.yaml`, aliases, multi-format output (json/jsonl/csv/kv/table) | +| HTTP server | — | Axum, OpenAPI via utoipa, bearer auth (SHA-256, AWS Secrets Manager option), `authorize_request` at the HTTP boundary (resolves bearer→actor, applies admission control), NDJSON streaming export, **multi-graph mode (v0.6.0+) with cluster routes + read-only `GET /graphs` enumeration + per-graph + server-level Cedar policies. Multi-graph boots from a cluster directory (`--cluster`) or the legacy `omnigraph.yaml`; add/remove graphs via `cluster apply` (or by editing the legacy file) and restarting.** | +| CLI with config | — | two-surface config (team `cluster.yaml` dir + per-operator `~/.omnigraph/config.yaml`; legacy `omnigraph.yaml` deprecated per RFC-008), aliases, multi-format output (json/jsonl/csv/kv/table) | | Audit / actor tracking | — | `_as` write APIs + actor map in commit graph | | Local RustFS bootstrap | — | `scripts/local-rustfs-bootstrap.sh` one-shot S3-backed dev environment | diff --git a/Cargo.lock b/Cargo.lock index b1cf0ef..2099055 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4543,7 +4543,7 @@ dependencies = [ [[package]] name = "omnigraph-cli" -version = "0.6.2" +version = "0.7.0" dependencies = [ "assert_cmd", "clap", @@ -4566,7 +4566,7 @@ dependencies = [ [[package]] name = "omnigraph-cluster" -version = "0.6.2" +version = "0.7.0" dependencies = [ "fail", "omnigraph-compiler", @@ -4584,7 +4584,7 @@ dependencies = [ [[package]] name = "omnigraph-compiler" -version = "0.6.2" +version = "0.7.0" dependencies = [ "ahash", "arrow-array", @@ -4605,7 +4605,7 @@ dependencies = [ [[package]] name = "omnigraph-engine" -version = "0.6.2" +version = "0.7.0" dependencies = [ "arc-swap", "arrow-array", @@ -4648,7 +4648,7 @@ dependencies = [ [[package]] name = "omnigraph-policy" -version = "0.6.2" +version = "0.7.0" dependencies = [ "cedar-policy", "clap", @@ -4661,7 +4661,7 @@ dependencies = [ [[package]] name = "omnigraph-server" -version = "0.6.2" +version = "0.7.0" dependencies = [ "arc-swap", "async-trait", diff --git a/crates/omnigraph-cli/Cargo.toml b/crates/omnigraph-cli/Cargo.toml index 901d69c..1670fb2 100644 --- a/crates/omnigraph-cli/Cargo.toml +++ b/crates/omnigraph-cli/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-cli" -version = "0.6.2" +version = "0.7.0" edition = "2024" description = "CLI for the Omnigraph graph database." license = "MIT" @@ -13,11 +13,11 @@ name = "omnigraph" path = "src/main.rs" [dependencies] -omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.2" } -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } -omnigraph-cluster = { path = "../omnigraph-cluster", version = "0.6.2" } -omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.2" } -omnigraph-server = { path = "../omnigraph-server", version = "0.6.2" } +omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.7.0" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.7.0" } +omnigraph-cluster = { path = "../omnigraph-cluster", version = "0.7.0" } +omnigraph-policy = { path = "../omnigraph-policy", version = "0.7.0" } +omnigraph-server = { path = "../omnigraph-server", version = "0.7.0" } clap = { workspace = true } color-eyre = { workspace = true } serde = { workspace = true } diff --git a/crates/omnigraph-cluster/Cargo.toml b/crates/omnigraph-cluster/Cargo.toml index 973de6d..05a9308 100644 --- a/crates/omnigraph-cluster/Cargo.toml +++ b/crates/omnigraph-cluster/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-cluster" -version = "0.6.2" +version = "0.7.0" edition = "2024" description = "Cluster configuration validation, planning, and config-only apply for Omnigraph." license = "MIT" @@ -14,8 +14,8 @@ documentation = "https://docs.rs/omnigraph-cluster" failpoints = ["dep:fail", "fail/failpoints"] [dependencies] -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } -omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.2" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.7.0" } +omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.7.0" } fail = { workspace = true, optional = true } serde = { workspace = true } serde_json = { workspace = true } diff --git a/crates/omnigraph-compiler/Cargo.toml b/crates/omnigraph-compiler/Cargo.toml index 8db46e6..bbf03f1 100644 --- a/crates/omnigraph-compiler/Cargo.toml +++ b/crates/omnigraph-compiler/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-compiler" -version = "0.6.2" +version = "0.7.0" edition = "2024" description = "Schema/query compiler for Omnigraph. Zero Lance dependency." license = "MIT" diff --git a/crates/omnigraph-policy/Cargo.toml b/crates/omnigraph-policy/Cargo.toml index 0df2a12..907ce07 100644 --- a/crates/omnigraph-policy/Cargo.toml +++ b/crates/omnigraph-policy/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-policy" -version = "0.6.2" +version = "0.7.0" edition = "2024" description = "Policy / authorization layer for Omnigraph — Cedar-backed PolicyEngine, PolicyChecker trait, ResourceScope enum." license = "MIT" diff --git a/crates/omnigraph-server/Cargo.toml b/crates/omnigraph-server/Cargo.toml index 5393221..614711e 100644 --- a/crates/omnigraph-server/Cargo.toml +++ b/crates/omnigraph-server/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-server" -version = "0.6.2" +version = "0.7.0" edition = "2024" description = "HTTP server for the Omnigraph graph database." license = "MIT" @@ -19,10 +19,10 @@ default = [] aws = ["dep:aws-config", "dep:aws-sdk-secretsmanager"] [dependencies] -omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.6.2" } -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } -omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.2" } -omnigraph-cluster = { path = "../omnigraph-cluster", version = "0.6.2" } +omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.7.0" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.7.0" } +omnigraph-policy = { path = "../omnigraph-policy", version = "0.7.0" } +omnigraph-cluster = { path = "../omnigraph-cluster", version = "0.7.0" } axum = { workspace = true } clap = { workspace = true } color-eyre = { workspace = true } diff --git a/crates/omnigraph/Cargo.toml b/crates/omnigraph/Cargo.toml index a4a2fe0..7ee9bda 100644 --- a/crates/omnigraph/Cargo.toml +++ b/crates/omnigraph/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "omnigraph-engine" -version = "0.6.2" +version = "0.7.0" edition = "2024" description = "Runtime engine for the Omnigraph graph database." license = "MIT" @@ -16,8 +16,8 @@ default = [] failpoints = ["dep:fail", "fail/failpoints"] [dependencies] -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } -omnigraph-policy = { path = "../omnigraph-policy", version = "0.6.2" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.7.0" } +omnigraph-policy = { path = "../omnigraph-policy", version = "0.7.0" } lance = { workspace = true } lance-datafusion = { workspace = true } datafusion = { workspace = true } @@ -52,7 +52,7 @@ chrono = { workspace = true } arc-swap = { workspace = true } [dev-dependencies] -omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.6.2" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.7.0" } tokio = { workspace = true } lance-namespace-impls = { workspace = true } serial_test = "3" diff --git a/docs/releases/v0.7.0.md b/docs/releases/v0.7.0.md new file mode 100644 index 0000000..4048041 --- /dev/null +++ b/docs/releases/v0.7.0.md @@ -0,0 +1,90 @@ +# Omnigraph v0.7.0 + +v0.7.0 takes the cluster control plane to object storage and overhauls the +configuration architecture around two single-owner surfaces. A cluster — +state ledger, content-addressed catalog, and graph data — can now live +entirely on an S3-compatible bucket, and a server can boot from that bucket +with no local files at all. Operator identity, credentials, and personal +aliases move to a home-level config; the legacy combined `omnigraph.yaml` +enters a guided, staged deprecation. + +## Highlights + +- **Clusters on object storage (`storage:`).** `cluster.yaml` gains an + optional `storage: s3://bucket/prefix` root. Every stored byte — the + state ledger, lock, recovery sidecars, approval artifacts, catalog blobs, + and the derived graph roots (`<storage>/graphs/<id>.omni`) — flows + through one storage layer, so `file://` (the default, byte-compatible + with existing clusters) and `s3://` are a single code path. The ledger's + compare-and-swap uses S3 conditional writes (`If-Match` / + `If-None-Match`), verified against AWS semantics, RustFS, and + Tigris-backed stores; the state lock is genuinely cross-machine on + object storage. +- **Config-free serving: `--cluster s3://bucket/prefix`.** The server + accepts a bare storage-root URI and boots from the applied revision on + the bucket — the ledger and catalog are the whole deployment artifact. + Policy bundles serve as digest-verified *content* from the catalog + (never re-read from disk), closing the last gap for fully remote + clusters. The preferred container shape becomes **bucket, no volume** + (see `docs/user/deployment.md`). +- **Per-operator configuration (`~/.omnigraph/`).** A home-level config + carries operator identity (`operator.actor`, the new last hop of the + `--as` chain), output defaults, named servers, and personal aliases. + `$OMNIGRAPH_HOME` relocates it; `$OMNIGRAPH_CONFIG` now stands in for + `--config` in both binaries. +- **Credentials keyed by server name.** `omnigraph login <server>` stores + a bearer token in `~/.omnigraph/credentials` (created `0600`; over- + permissive files are refused). Token resolution for a request whose URL + matches an operator-defined server: `OMNIGRAPH_TOKEN_<NAME>` env → the + credentials file → the legacy `bearer_token_env` chain unchanged. A + token is only ever sent to the server it is keyed to. +- **Operator targeting and aliases.** `--server <name>` (with `--graph + <id>` for multi-graph servers) addresses operator-defined endpoints on + every remote-capable command. Operator aliases are pure *bindings* — + personal name → (server, graph, stored-query name, default params) — + invoking catalog-owned stored queries; they carry no query content. +- **`omnigraph.yaml` deprecation begins.** Loading the legacy file prints + a per-key notice naming each present key's new home + (`OMNIGRAPH_SUPPRESS_YAML_DEPRECATION=1` to silence in CI). + `omnigraph config migrate` proposes — and with `--write`, applies — the + split: team half to a ready-to-review `cluster.yaml`, personal half + merged into the operator config (existing entries always win). + `omnigraph init` no longer scaffolds the file. Migrated teams can set + `OMNIGRAPH_NO_LEGACY_CONFIG=1` to turn any legacy-file load into a hard + error. The file itself keeps working until its removal at the next + major version. + +## Breaking / behavior changes + +- `omnigraph init` no longer writes an `omnigraph.yaml` into the working + directory. Start cluster configs from the documentation templates, or + run `omnigraph config migrate` against an existing legacy file. +- Loading a legacy `omnigraph.yaml` now emits a deprecation block on + stderr (suppressible; see above). Output on stdout is unchanged. +- `ServingPolicy` (cluster crate API) carries verified policy *content* + instead of a blob path; `read_serving_snapshot` and several cluster + command entry points are now async. + +## Upgrade notes + +- Existing clusters need no migration: an absent `storage:` key keeps the + config-directory layout byte-for-byte. +- Existing `omnigraph.yaml` setups keep working through the deprecation + window; `omnigraph config migrate` produces the recommended split. +- Operator setup is three lines: + `mkdir -p ~/.omnigraph`, write `operator.actor` (and `servers:`) into + `~/.omnigraph/config.yaml`, then `echo $TOKEN | omnigraph login <server>`. + +## Internals + +- The cluster, server, and CLI crates were modularized (the 7.9k-line + cluster `lib.rs` is now eight focused modules; the server and CLI test + monoliths split into per-area suites) — pure code movement, no behavior + change. +- New gated end-to-end suites run the full cluster lifecycle against a + real S3-compatible store in CI, including a lock-release regression and + a config-free server boot from a bare bucket URI. +- The deployment guide gains the bucket-no-volume container recipe for + AWS and Railway, validated against a live Railway deployment + (Railway buckets are S3-compatible and pass the conditional-write + contract test). diff --git a/openapi.json b/openapi.json index 85c5b8d..6e3dd03 100644 --- a/openapi.json +++ b/openapi.json @@ -7,7 +7,7 @@ "name": "MIT", "identifier": "MIT" }, - "version": "0.6.2" + "version": "0.7.0" }, "paths": { "/branches": { From 98c6147c38fe97e109acac96cd0d1fe59325834a Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Fri, 12 Jun 2026 13:22:34 +0300 Subject: [PATCH 138/165] docs(testing): bring the test map up to release truth MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lands an orphaned-but-accurate working-tree edit (the engine table rows for forbidden_apis.rs, lance_surface_guards.rs, traversal_indexed, proptest_equivalence, ordering, literal_filters, policy_engine_chassis — all real files; 21 -> 28 count) and replaces the stale pre-modularization crate rows: the CLI and server entries now describe the per-area suites (#192/#193 splits) plus this cycle's additions (RFC-008 deprecation coverage, keyed-credential auth, hermetic OMNIGRAPH_HOME harness, the bucket-gated s3 suites). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/testing.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/docs/dev/testing.md b/docs/dev/testing.md index a817428..f2b33de 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -6,10 +6,10 @@ This file is the always-on map of the test surface. **Consult it before every ta | Crate | Path | Style | |---|---|---| -| `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (21 files), fixture-driven, share `tests/helpers/mod.rs` | -| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | `cli.rs` (unit-ish; includes the `cluster_e2e_*` lifecycle compositions over the spawned binary — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `system_local.rs` (incl. the full-cycle cluster lifecycle with a spawned `--cluster` server — declare→serve→evolve→drift-heal→approved-delete — and applied-policy enforcement over HTTP), `system_remote.rs`, share `tests/support/mod.rs` | +| `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (28 files), fixture-driven, share `tests/helpers/mod.rs` | +| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | Per-area suites (post-modularization): `cli_cluster.rs` (cluster command surface + operator-actor cascade), `cli_cluster_e2e.rs` (spawned-binary lifecycle compositions — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `cli_data.rs` (load/read/change/branch/commit/export/snapshot/policy/embed/maintenance + operator format cascade), `cli_schema_config.rs` (init/config, schema plan/apply, RFC-008 deprecation warnings + `config migrate` + strict mode), `cli_queries.rs`, `system_local.rs` (full-cycle cluster lifecycle with a spawned `--cluster` server, applied-policy enforcement over HTTP, keyed-credential auth, operator aliases), `system_remote.rs`; share `tests/support/mod.rs` (hermetic `OMNIGRAPH_HOME` by default) | | `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated); `tests/s3_cluster.rs` (bucket-gated full lifecycle on object storage) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows), and 5A policy binding metadata (applies_to in the applied revision, binding-change diffing + convergence, pre-5A backfill), and the 5B serving-snapshot read API (converged read, refusal rows) | -| `omnigraph-server` | `crates/omnigraph-server/tests/` | `server.rs` (HTTP-level; incl. cluster-mode boot — converged-dir serving, policy binding wiring, boot refusals), `openapi.rs` (OpenAPI drift / regeneration) | +| `omnigraph-server` | `crates/omnigraph-server/tests/` | Per-area suites (post-modularization): `auth_policy.rs`, `data_routes.rs`, `schema_routes.rs`, `stored_queries.rs`, `multi_graph.rs` (cluster-mode boot — converged serving, policy binding wiring, boot refusals — + the concurrent branch-ops matrix), `boot_settings.rs` (mode inference, PolicySource), `s3.rs` (bucket-gated: single-graph serving + config-free `--cluster s3://` boot), `openapi.rs` (OpenAPI drift / regeneration); share `tests/support/mod.rs` | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | The engine's `tests/` is the principal coverage surface; most graph-shaped behavior is exercised there. @@ -23,18 +23,25 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav | `merge_truth_table.rs` | Merge-pair truth table (MR-786): all 9×9 `(left_op, right_op)` cells from `{noop, addNode, removeNode, addEdge, removeEdge, setProperty, dropProperty, addLabel, removeLabel}`. Adding a new op to `OpVariant` forces a compile error in `build_case` until the new row + column are dispositioned. 36 executable cells run through real `branch_merge` with a structured oracle (`MergeOutcome` / `MergeConflictKind` + graph-state assert); 45 cells involving `dropProperty`/`addLabel`/`removeLabel` are recorded as `Unsupported` until the mutation grammar grows. | | `writes.rs` | Direct-publish writes: cancellation, non-strict insert/merge rebase under the per-table queue, strict stale-write conflicts, multi-statement atomicity, MR-794 staged-write rewire (D₂ rejection, insert+update coalesce, multi-append coalesce, partial-failure recovery, load RI/cardinality recovery) | | `staged_writes.rs` | TableStore staged-write primitives (`stage_append`, `stage_merge_insert`, `commit_staged`, `scan_with_staged`, `count_rows_with_staged`) — primitive-level only; engine code uses the in-memory `MutationStaging` accumulator instead | +| `forbidden_apis.rs` | Defense-in-depth source-walk guard: engine code (`exec/`, `db/omnigraph/`, `loader/`, `changes/`) must not reach around the sealed storage trait to Lance inline-commit APIs; `// forbidden-api-allow: <reason>` sentinel exempts reviewed lines | +| `lance_surface_guards.rs` | Pins the Lance API surfaces omnigraph depends on (named runtime + compile-only guards; see [lance.md](lance.md)) — the first smoke check on any Lance version bump; e.g. `compact_files_still_fails_on_blob_columns` turns red when the upstream blob-compaction fix lands | | `lifecycle.rs` | Graph lifecycle, schema state | | `point_in_time.rs` | Snapshots, time travel (`snapshot_at_version`, `entity_at`) | | `changes.rs` | `diff_between` / `diff_commits` | | `consistency.rs` | Cross-table snapshot isolation, atomic publish | | `schema_apply.rs` | Migration plan + apply, schema-apply lock | | `search.rs` | FTS / vector / hybrid (`bm25`, `nearest`, `rrf`) | -| `traversal.rs` | `Expand`, variable-length hops, anti-join | +| `traversal.rs` | `Expand`, variable-length hops, anti-join (CSR path — `OMNIGRAPH_TRAVERSAL_MODE` unset) | +| `traversal_indexed.rs` | BTREE-indexed Expand (`execute_expand_indexed`) forced via `OMNIGRAPH_TRAVERSAL_MODE`, asserted semantically equal to the CSR path; own binary, all `#[serial]` so env writes never race | +| `proptest_equivalence.rs` | Property-based query-correctness invariants over generated graphs (shared key alphabet forces cross-type id collisions, cycles, self-loops) — pins Expand-mode equivalence so a future fork divergence fails loudly instead of silently; `#[serial]` | +| `ordering.rs` | ORDER BY contract: descending, multi-key precedence, deterministic key-column tie-break (total order, so `ORDER … LIMIT` is deterministic), NULL placement (`nulls_first = !descending`) | +| `literal_filters.rs` | Execution goldens for non-string/non-integer scalar literal filters (F64/F32/Bool/Date/DateTime) across both the in-memory comparison arm and the Lance-pushdown arm | | `aggregation.rs` | `count`, `sum`, `avg`, `min`, `max` | | `export.rs` | NDJSON streaming export filters | | `s3_storage.rs` | S3-backed graph (skipped unless `OMNIGRAPH_S3_TEST_BUCKET` is set) | | `lance_version_columns.rs` | Per-row `_row_last_updated_at_version` behavior | | `validators.rs` | Schema constraint enforcement (enum, range, unique, cardinality) across JSONL, insert, update paths | +| `policy_engine_chassis.rs` | Engine-layer Cedar enforcement (MR-722): allow + deny through every `_as` writer via the SDK directly — no HTTP — proving embedded and CLI callers hit the same gate as the server, with action × scope shapes matching `authorize_request` | | `maintenance.rs` | `optimize` (compaction), `repair` (explicit uncovered-drift publish), and `cleanup` (version GC): empty/idempotent/no-op edges, policy validation, head preservation; `optimize` publishes its own compaction (`optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds`), skips pre-existing uncovered drift (`optimize_skips_preexisting_manifest_head_drift`), and refuses to run while a `__recovery` sidecar is pending (`optimize_defers_when_recovery_sidecar_is_pending`); `repair` previews/heals verified maintenance drift, refuses raw semantic drift without `--force`, and forced repair publishes only by explicit operator choice | | `failpoints.rs` | Failure-injection coverage (gated on `failpoints` feature). Includes the five per-writer Phase B → recovery integration tests (`recovery_rolls_forward_after_finalize_publisher_failure`, `schema_apply_phase_b_failure_recovered_on_next_open`, `branch_merge_phase_b_failure_recovered_on_next_open`, `ensure_indices_phase_b_failure_recovered_on_next_open`, `optimize_phase_b_failure_recovered_on_next_open`). | | `recovery.rs` | Open-time recovery sweep — sidecar I/O, classifier dispatch (NoMovement / RolledPastExpected / UnexpectedAtP1 / UnexpectedMultistep / InvariantViolation), all-or-nothing decision, roll-forward via `ManifestBatchPublisher::publish`, roll-back via `Dataset::restore`, audit row in `_graph_commit_recoveries.lance`, `OpenMode::ReadOnly` skip path | From 9002cfd5b9077fcb8de970892a2509d9a0bcf067 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Fri, 12 Jun 2026 17:33:11 +0300 Subject: [PATCH 139/165] =?UTF-8?q?docs(rfc):=20RFC-009=20=E2=80=94=20unif?= =?UTF-8?q?y=20CLI=20access=20paths;=20align=20the=20RFC=20corpus?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adopts the unify-embedded/remote draft as RFC-009 with three alignment amendments: (1) the promised 'companion config-authority RFC' is RFC-008, already landed through stage 4 — referenced, not re-proposed; (2) open question 3 is answered by the two-surface architecture (embedded graphs list enumerates the cluster catalog via read_serving_snapshot, never omnigraph.yaml); (3) Phase 2 salvages PR #139's reviewed-clean omnigraph-api-types extraction instead of rebuilding. Adds the cycle's two no-referee bugs (alias positional, write-if-absent flush) as concrete parity-matrix motivation, and RFC-007's addressing/credential chains as RemoteClient constructor inputs. Corpus alignment: RFC-002's header now maps each of its pieces to the successor that landed or superseded it (007/008/009) with a do-not- implement-from-here-unchecked warning; RFC-007 gains the RFC-009 relationship; RFC-008 stage 5 notes the Phases-4/5 easing; dev index row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- docs/dev/index.md | 1 + docs/dev/rfc-002-config-cli-architecture.md | 2 +- docs/dev/rfc-007-operator-config.md | 6 + docs/dev/rfc-008-deprecate-omnigraph-yaml.md | 2 +- docs/dev/rfc-009-unify-access-paths.md | 213 +++++++++++++++++++ 5 files changed, 222 insertions(+), 2 deletions(-) create mode 100644 docs/dev/rfc-009-unify-access-paths.md diff --git a/docs/dev/index.md b/docs/dev/index.md index b23326b..b1dc4fb 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -78,6 +78,7 @@ Working documents for in-flight feature work. Removed when the work lands. | Server boots from cluster state — Phase 5 mode switch, applied-revision serving | [rfc-005-server-cluster-boot.md](rfc-005-server-cluster-boot.md) | | Per-operator config — `~/.omnigraph/` identity, keyed credentials, named servers (the operator slice of RFC-002) | [rfc-007-operator-config.md](rfc-007-operator-config.md) | | Deprecate `omnigraph.yaml` — one concern per config surface; key-by-key migration map and staged retirement | [rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md) | +| Unify CLI embedded/remote access paths — parity referee, shared wire-DTO crate, `GraphClient` trait, declared plane capabilities | [rfc-009-unify-access-paths.md](rfc-009-unify-access-paths.md) | ## Boundary diff --git a/docs/dev/rfc-002-config-cli-architecture.md b/docs/dev/rfc-002-config-cli-architecture.md index 0a8e573..8095eda 100644 --- a/docs/dev/rfc-002-config-cli-architecture.md +++ b/docs/dev/rfc-002-config-cli-architecture.md @@ -1,6 +1,6 @@ # RFC: Config & CLI Architecture — Layered Config, Client Targeting, File Naming -**Status:** Proposed +**Status:** Proposed (umbrella; implementation parked — PRs #139/#162). Its pieces have since landed or been superseded piecemeal: layered config/file-naming/credentials → [rfc-007-operator-config.md](rfc-007-operator-config.md) (landed); the project-manifest role of `omnigraph.yaml` → deprecated by [rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md) (stages 1–4 landed); the `omnigraph-api-types` extraction and client unification → [rfc-009-unify-access-paths.md](rfc-009-unify-access-paths.md) (proposed; salvages #139's clean extraction). Still exclusively here: `GraphLocator`/multi-homing (§1), roles (§3), the State layer / `omnigraph use`. Do not implement from this document without checking those successors first. **Date:** 2026-05-30 **Tickets:** MR-668 (multi-graph server, shipped — the dependency this builds on), MR-969 (stored queries + MCP — supplies the in-repo agent tool surface), MR-973 (quickstart / onboarding), MR-974 (agent setup surface), MR-981 (agent-friendly CLI hardening) **Target release:** v0.8.x (tentative; phased — see Rollout) diff --git a/docs/dev/rfc-007-operator-config.md b/docs/dev/rfc-007-operator-config.md index 5bd8afb..1cbc0ef 100644 --- a/docs/dev/rfc-007-operator-config.md +++ b/docs/dev/rfc-007-operator-config.md @@ -319,6 +319,12 @@ Every mention of `omnigraph.yaml` in this RFC describes the deprecation window only. Sequencing couples them: RFC-007 PRs 1–2 land first, then RFC-008's migration stages run against them. +[rfc-009-unify-access-paths.md](rfc-009-unify-access-paths.md) consumes +this RFC's surfaces: the actor chain and keyed-credential chain become +constructor-time inputs of its `RemoteClient`/`EmbeddedClient`, and +`--server`/operator aliases resolve to the same (base URL, credential) +pair before its `GraphClient` trait is touched. + RFC-002 remains the umbrella architecture. This RFC implements its §2 (layered config, global-first), §4 (file naming / one dir), and §5 (credentials) in their minimal load-bearing form, and explicitly defers §1 diff --git a/docs/dev/rfc-008-deprecate-omnigraph-yaml.md b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md index 72baaf6..4b5bf35 100644 --- a/docs/dev/rfc-008-deprecate-omnigraph-yaml.md +++ b/docs/dev/rfc-008-deprecate-omnigraph-yaml.md @@ -134,7 +134,7 @@ contract), retirement is staged, loud, and tooled: `cluster.yaml` from `config migrate`. 4. **Opt-in strict** *(landed — the release gap to stages 1–3 collapsed: no version boundary was crossed between them, so all four ship in the same release)*. `OMNIGRAPH_NO_LEGACY_CONFIG=1` turns the warning into an error — for teams that finished migrating and want regressions caught. -5. **Remove at the next major.** Loading the file becomes an error pointing +5. **Remove at the next major** *(eased by [rfc-009-unify-access-paths.md](rfc-009-unify-access-paths.md) Phases 4–5: declared plane capabilities and route alignment shrink the yaml-boot removal diff)*. Loading the file becomes an error pointing at `config migrate`. The `OmnigraphConfig` code path, the dual query-registry loaders, and the yaml-mode server boot source are deleted — the payoff that makes the whole exercise worth it. diff --git a/docs/dev/rfc-009-unify-access-paths.md b/docs/dev/rfc-009-unify-access-paths.md new file mode 100644 index 0000000..43dcdf2 --- /dev/null +++ b/docs/dev/rfc-009-unify-access-paths.md @@ -0,0 +1,213 @@ +# RFC: Unify the CLI's Embedded and Remote Access Paths + +**Status:** Proposed +**Date:** 2026-06-12 +**Audience:** engine/CLI/server maintainers +**Builds on:** [rfc-007-operator-config.md](rfc-007-operator-config.md) +(landed — `--server` targeting and operator aliases are remote-addressing +surfaces the unified client must treat as first-class), +[rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md) +(stages 1–4 landed — the config-authority demotion this RFC's earlier +drafts promised as a "companion" already happened; the remaining sliver, +removing the yaml-mode server boot source, is RFC-008 stage 5 and is +*eased* by Phases 4–5 here), [rfc-002-config-cli-architecture.md](rfc-002-config-cli-architecture.md) +(umbrella; see Prior Art for what is salvageable from its parked +implementation). +**Sequencing:** post-v0.7.0 (the release cut comes first). + +## Summary + +Collapse the CLI's per-command `is_remote` forks into one execution path coded +against a `GraphClient` trait with two implementations (embedded engine, HTTP), +sharing one wire-DTO crate with the server. Establish an executable parity +referee *before* the refactor. This is the same cure, in the same order, that +fixed the storage layer: one contract, one implementation where semantics are +one thing, an executable contract where two implementations must exist. + +## Motivation — validated facts + +Graph **semantics** cannot drift between paths today: both converge on the same +engine `_as` entry points (verified: `omnigraph-server/src/handlers.rs` calls +`mutate_as`, `apply_schema_as_with_catalog_check`, `load_as`, +`branch_create_from_as`, `branch_delete_as`, `branch_merge_as` — the same +functions the CLI's embedded arm calls), and Cedar enforcement lives inside +those writers. What *can* drift is everything around them: + +1. **15 `is_remote` forks** in `crates/omnigraph-cli/src/main.rs`, each + duplicating request shaping and output mapping per command. (RFC-007 PR 3 + threaded `apply_server_flag` through exactly these sites — the duplication + is measured, not estimated.) +2. **Triple DTO construction.** "The result of a load" is built in three + places: the server handler (engine result → HTTP response), the CLI remote + arm (HTTP response → `LoadOutput` via `load_output_from_tables`), and the + CLI local arm (engine result → hand-built `LoadOutput`). Three mappings + that agree only by discipline — the exact shape of the storage-adapter bug + class (one prose contract, N implementations, no referee). +3. **The remote `load` arm rides the deprecated `/ingest` route.** A + non-deprecated CLI verb coupled to a deprecated endpoint turns the + endpoint's eventual removal into a surprise CLI breakage. +4. **Plane restrictions are accidental, not declared.** `init` / `optimize` / + `repair` / `cleanup` / `cluster *` are storage-only and `graphs list` is + server-only by code shape; pointing `optimize` at an `https://` target + fails with whatever `Omnigraph::open` says about an https URI. Per Hyrum, + that accidental error text is already someone's dependency. +5. **Parity pinning is thin.** One explicit parity test + (`cli_schema_config.rs::schema_plan_parity_cli_and_sdk`), flow coverage in + `system_local.rs`, and the OpenAPI drift test. No systematic + per-verb embedded-vs-remote comparison exists. Two bugs from the current + cycle argue the referee's value concretely: the operator-alias positional + bug (the hidden `legacy_uri` positional swallowed the first arg — local + and remote disagreed until a live smoke caught it) and the + `write_text_if_absent` flush bug (one of N implementations of an + unwritten contract) would both have failed a parity matrix. + +## Design + +Ordered so each phase is independently shippable and the referee exists before +anything moves — mirroring the storage collapse, where the pinned contract +tests gated the swap, and the test-monolith modularization (#192/#193), which +makes Phase 3 tractable: the CLI dispatch is 1,184 lines today, not 4,200. + +### Phase 1 — Parity matrix (the referee; do first, no refactor) + +A CLI integration test (extend the `system_local.rs` harness, which already +spawns both binaries): one fixture graph; for every forked verb, run the +command once against the local URI and once against a spawned server with +identical inputs; diff the `--json` outputs against an explicit allowlist of +transport-only fields (e.g. resolved URI). Assert identical exit codes for the +shared error cases. + +This pins today's behavior so Phase 3 can't silently change it, and catches +every future fork drift. It also incidentally covers utoipa annotation↔route +mismatches (a lying `#[utoipa::path]` makes the remote leg 404). + +### Phase 2 — One wire-DTO crate + +Move the HTTP request/response types and the single `engine result → DTO` +mapping per verb into a shared crate (working name `omnigraph-api-types`), +carrying serde + utoipa `ToSchema` derives: + +- `omnigraph-server` handlers serialize these types; utoipa derives + `openapi.json` from them (the existing `openapi.rs` regeneration test stays + the spec referee). +- The CLI embedded path constructs them via the shared mapping. +- The CLI remote path deserializes the literal same types. + +The mapping then exists once, next to the type — it cannot fork. Spec codegen +remains exactly where it belongs: foreign-language clients (the TS SDK +pipeline). Generating a Rust client from the spec is explicitly rejected — it +would round-trip Rust types through a lossy intermediate when compile-time +type sharing is available. + +**Prior art to salvage:** PR #139's review explicitly found the +`omnigraph-api-types` extraction *clean* ("the crate extraction itself is +clean and the openapi.json byte-identical claim holds" — +[pr-139 findings](rfc-002-config-cli-architecture.md)); it was the behavior +changes bundled alongside that killed the PR. Seed this phase from the +extraction commits on `ragnorc/scrutinize-rfc-002` rather than rebuilding — +cherry-picked narrowly, never relanded wholesale. + +Boundary note: this does NOT violate "transport/auth stay at the boundary" +(invariants §11). The shared crate holds plain serde DTOs; it depends on +neither axum nor the engine's internals. The engine crate does not depend on +it — the `engine result → DTO` mapping lives in the shared crate (or the CLI/ +server side), taking engine result types as input. + +### Phase 3 — `GraphClient` trait, two implementations + +```text +trait GraphClient // verb-level: load, mutate, query, branch_*, schema_*, export, commit_* + ├── EmbeddedClient // wraps Omnigraph + the shared mapping; actor: explicit (--as cascade, RFC-007) + └── RemoteClient // reqwest + bearer; actor: resolved server-side from the token +``` + +Each CLI command body is written once against the trait; the 15 forks become +2 impls × 1 contract. Actor resolution is a constructor-time difference of the +impls, never a per-verb branch — the trust model (storage credentials = +self-declared actor via the RFC-007 actor chain; server = token-resolved +actor via the RFC-007 keyed-credential chain) is a feature, not drift. +`RemoteClient` construction is where RFC-007's addressing lands once: +positional URI, `--target`, `--server <name>`, and operator aliases all +resolve to the same (base URL, credential) pair before the trait is touched. +The Phase 1 matrix becomes the trait's conformance suite, run against both +impls. + +### Phase 4 — Declared plane capabilities + +Each CLI command declares `Storage | Server | Both`. Dispatch checks the +resolved target against the declaration and fails with one deliberate message +("maintenance commands operate on storage directly; use a storage URI, not a +server target") instead of today's incidental errors. The declaration table is +also documentation: it makes the control-plane/data-plane split (maintenance +and cluster commands must work with the server down) explicit in code. +"Server" targets include operator-config named servers (RFC-007), not only +literal `http(s)://` URIs. + +### Phase 5 — Route alignment + +Add a canonical `/load` endpoint (the handler already exists behind the +`/ingest` shim); point `RemoteClient` at it; keep `/ingest` on its existing +deprecation path. While here, check whether the server uses `utoipa-axum`'s +router-coupled registration (`OpenApiRouter`/`routes!`); if it hand-mounts +routes beside `#[utoipa::path]` annotations, prefer migrating registration so +path annotations and mount points are the same declaration (the modularization +already hit one orphaned-attribute incident of exactly this class). + +## Non-goals + +- **No localhost-server funnel for the embedded path.** Routing embedded use + through a daemon would destroy the embedded/CLI/test story to buy parity the + trait + matrix already provide. +- **No trust-model unification.** `--as` vs bearer-resolved actors stay. +- **No spec-codegen for the Rust client** (see Phase 2). +- **No change to plane-restricted command availability** — maintenance stays + storage-direct by design; Phase 4 only makes the restriction explicit. +- **No config-authority work** — that was RFC-008, already landed through + stage 4; this RFC neither accelerates nor blocks stage 5, though Phases 4–5 + make the eventual yaml-boot removal a smaller diff. + +## Compatibility + +- CLI `--json` output is observable contract; Phase 1 freezes it before + Phase 3 moves code. Any field-level unification that *changes* output is a + deliberate, release-noted decision, not a refactor side effect. +- Error-message text for mis-planed commands changes in Phase 4 — release-note + it (Hyrum). +- `openapi.json` should be byte-stable through Phase 2 if the DTO move is + faithful; the regeneration test enforces this. + +## Testing + +- Phase 1 matrix is the spine; it must stay green, textually unchanged, + through Phases 2–3 (the storage-collapse playbook). +- Phase 2: `openapi.rs` byte-stability + existing server tests. +- Phase 4: one test per capability class asserting the deliberate error. +- Phase 5: parity matrix leg for `load` flips to `/load`; an `/ingest` shim + test stays until removal. + +## Open questions + +1. Crate granularity: one `omnigraph-api-types` crate vs folding into an + existing one. (Leaning separate: server and CLI both depend on it; the + engine must not. The #139 extraction already answered this with a separate + crate that reviewed clean.) +2. Does the `query`/`read` streaming path (NDJSON export) fit the trait, or is + export a documented per-impl method? (Streaming over HTTP vs an iterator + over the embedded engine differ in shape, not content.) +3. ~~Whether `graphs list` belongs on the trait.~~ **Answered by the + two-surface architecture**: the embedded impl enumerates the **cluster + catalog** (`read_serving_snapshot` exists for exactly this), never + `omnigraph.yaml` (deprecated, RFC-008). `graphs list` becomes + `Both`-capability: remote = `GET /graphs` (policy-gated), embedded = + catalog enumeration from a cluster storage root. + +## Relationship to prior work + +The third application of the same principle in this lineage: storage adapters +(collapsed to one implementation + an executable contract — and its CAS/flush +bugs were exactly the no-referee class), recovery liveness, and now access +paths. RFC-007 supplied the addressing and credential surfaces `RemoteClient` +consumes; RFC-008 removed the competing config authority; RFC-002 remains the +umbrella whose remaining unimplemented pieces (`GraphLocator`, the State +layer) would build on the trait introduced here rather than on per-command +forks. From 08c9b03d406fe93a76f7c229f12609dae61d38fc Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Fri, 12 Jun 2026 17:50:46 +0300 Subject: [PATCH 140/165] test(cli): the embedded/remote parity matrix (RFC-009 Phase 1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The referee before any unification moves: every forked verb runs once against the local graph and once against a spawned server on a twin copy of the same fixture, with the SAME actor (--as locally; bearer-resolved remotely) and the SAME Cedar bundle on both arms — like-for-like enforcement is part of the harness (a tokens-only server is default-deny by design; comparing that against a bare local arm measures configuration, not the fork). Declared-volatile fields (ids, wall-clock, transport locations) scrub to placeholders; everything else must match exactly, and exit codes must match for shared failures. Headline result: 11 rows green with an EMPTY divergence ledger — the arms agree on every verb today. The ledger (KNOWN_DIVERGENCES) exists so any future divergence is pinned or filed, never silently repaired; repairs are Phase 3's job, gated by this referee staying green. One engine observation surfaced and filed (#207): inline execution with a declared-but-unbound param matches ALL rows on both arms, while the stored-query invoke path hard-errors — a cross-path asymmetry the matrix pins as agreeing behavior pending a deliberate fix. Documented exclusions (graphs list, ingest/load-over-/ingest, storage-plane verbs) map to RFC-009 Phases 4-5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/tests/parity_matrix.rs | 255 ++++++++++++++++++++ crates/omnigraph-cli/tests/support/mod.rs | 167 +++++++++++++ docs/dev/rfc-009-unify-access-paths.md | 11 +- docs/dev/testing.md | 2 +- 4 files changed, 433 insertions(+), 2 deletions(-) create mode 100644 crates/omnigraph-cli/tests/parity_matrix.rs diff --git a/crates/omnigraph-cli/tests/parity_matrix.rs b/crates/omnigraph-cli/tests/parity_matrix.rs new file mode 100644 index 0000000..75ba49e --- /dev/null +++ b/crates/omnigraph-cli/tests/parity_matrix.rs @@ -0,0 +1,255 @@ +//! RFC-009 Phase 1 — the embedded/remote parity referee. +//! +//! For every CLI verb with an `is_remote` fork, run the identical +//! invocation against (a) the local graph directly and (b) a spawned +//! server on a twin copy of the same graph, with the SAME actor on both +//! arms (local `--as act-parity`; remote bearer token resolving to +//! `act-parity`). Scrub the declared-volatile allowlist +//! (`support::scrub_volatile` — ids, wall-clock, transport locations); +//! everything else must match exactly. +//! +//! This test PINS behavior; it does not idealize it. Genuine divergences +//! discovered here are recorded in `KNOWN_DIVERGENCES` below (and filed), +//! never silently repaired — repairs are Phase 3's job, gated by this +//! referee staying green through the refactor. + +use tempfile::TempDir; + +mod support; +use support::*; + +/// Divergences between the arms that exist today, pinned as expectations. +/// Removing an entry requires the corresponding behavior change to be a +/// deliberate, release-noted decision (RFC-009 Compatibility). +const KNOWN_DIVERGENCES: &[&str] = &[ + // populated by the rows below as they are written +]; + +/// One matched setup per row: twin graphs + the SAME Cedar bundle on both +/// arms (the local arm via --config top-level policy.file; the server via +/// its config). Returns everything a row needs. +struct Parity { + _temp: TempDir, + local: std::path::PathBuf, + local_cfg: std::path::PathBuf, + server: TestServer, +} + +fn parity() -> Parity { + let (temp, local, remote) = twin_graphs(); + let (local_cfg, server_cfg) = parity_configs(temp.path(), &local, &remote); + let server = spawn_server_with_config_env( + &server_cfg, + &[( + "OMNIGRAPH_SERVER_BEARER_TOKENS_JSON", + r#"{"act-parity":"parity-tok"}"#, + )], + ); + Parity { + _temp: temp, + local, + local_cfg, + server, + } +} + +impl Parity { + fn run(&self, args: &[&str]) -> (std::process::Output, std::process::Output) { + run_both_with_config(&self.local, Some(&self.local_cfg), &self.server.base_url, args) + } +} + +fn assert_parity(verb: &str, local: &std::process::Output, remote: &std::process::Output) { + assert_eq!( + local.status.code(), + remote.status.code(), + "{verb}: exit codes diverge\nlocal: {local:?}\nremote: {remote:?}" + ); + if local.status.success() { + let local_json = scrubbed_json(local); + let remote_json = scrubbed_json(remote); + assert_eq!( + local_json, remote_json, + "{verb}: scrubbed JSON diverges (left=local, right=remote)" + ); + } +} + +#[test] +fn parity_query() { + let p = parity(); + let query = fixture("test.gq"); + let (l, r) = p.run(&[ + "query", + "--query", + query.to_str().unwrap(), + "--name", + "get_person", + "--params", + r#"{"name":"Alice"}"#, + "--json", + ], + ); + assert_parity("query", &l, &r); +} + +#[test] +fn parity_schema_show() { + let p = parity(); + let (l, r) = p.run(&["schema", "show", "--json"]); + assert_parity("schema show", &l, &r); +} + +#[test] +fn parity_snapshot() { + let p = parity(); + let (l, r) = p.run(&["snapshot", "--json"]); + assert_parity("snapshot", &l, &r); +} + +#[test] +fn parity_branch_list() { + let p = parity(); + let (l, r) = p.run(&["branch", "list", "--json"]); + assert_parity("branch list", &l, &r); +} + +#[test] +fn parity_commit_list() { + let p = parity(); + let (l, r) = p.run(&["commit", "list", "--json"]); + assert_parity("commit list", &l, &r); +} + +#[test] +fn parity_mutate() { + let p = parity(); + let (l, r) = p.run(&[ + "mutate", + "-e", + "query add($name: String, $age: I32) { insert Person { name: $name, age: $age } }", + "--params", + r#"{"name":"Parity","age":7}"#, + "--json", + ], + ); + assert_parity("mutate", &l, &r); +} + +#[test] +fn parity_branch_create_delete() { + let p = parity(); + let (l, r) = p.run(&["branch", "create", "--from", "main", "parity-branch", "--json"], + ); + assert_parity("branch create", &l, &r); + let (l, r) = p.run(&["branch", "delete", "parity-branch", "--json"], + ); + assert_parity("branch delete", &l, &r); +} + +#[test] +fn parity_branch_merge() { + let p = parity(); + let (l, r) = p.run(&["branch", "create", "--from", "main", "feature", "--json"], + ); + assert_parity("branch create (merge setup)", &l, &r); + let (l, r) = p.run(&["branch", "merge", "feature", "--into", "main", "--json"], + ); + assert_parity("branch merge", &l, &r); +} + +#[test] +fn parity_load() { + let p = parity(); + let data = p.local.parent().unwrap().join("rows.jsonl"); + std::fs::write( + &data, + "{\"type\":\"Person\",\"data\":{\"name\":\"Loaded\",\"age\":1}}\n", + ) + .unwrap(); + let (l, r) = p.run(&[ + "load", + "--mode", + "merge", + "--data", + data.to_str().unwrap(), + "--json", + ], + ); + assert_parity("load", &l, &r); +} + +// ---- error parity: exit codes must match for shared failure cases ---- + +#[test] +fn parity_errors_share_exit_codes() { + let p = parity(); + + // unknown branch on merge + let (l, r) = p.run(&["branch", "merge", "no-such-branch", "--into", "main", "--json"], + ); + assert_eq!( + (l.status.success(), r.status.success()), + (false, false), + "merge of unknown branch must fail on both arms\nlocal {l:?}\nremote {r:?}" + ); + + // unknown query name in the source + let query = fixture("test.gq"); + let (l, r) = p.run(&[ + "query", + "--query", + query.to_str().unwrap(), + "--name", + "no_such_query", + "--json", + ], + ); + assert_eq!( + (l.status.success(), r.status.success()), + (false, false), + "unknown query name must fail on both arms\nlocal {l:?}\nremote {r:?}" + ); + + // Discovery (parity HOLDS, behavior surprising): an inline query run + // with a declared-but-unbound param does NOT error on either arm — it + // returns every row (the filter drops), while the stored-query invoke + // path hard-errors 'parameter not provided'. Pinned here as agreeing + // behavior; the cross-path asymmetry is filed separately. + let (l, r) = p.run(&[ + "query", + "--query", + query.to_str().unwrap(), + "--name", + "get_person", + "--json", + ], + ); + assert_eq!( + (l.status.success(), r.status.success()), + (true, true), + "unbound-param inline query currently SUCCEEDS on both arms (matches-all)" + ); +} + +// ---- documented exclusions (not bugs; the Phase 4 capability table) ---- +// +// - `graphs list`: server-only today; becomes Both-capability when the +// embedded arm enumerates the cluster catalog (RFC-009 open Q3, answered). +// - `ingest`: deprecated alias of load; the remote `load` arm itself rides +// the deprecated /ingest route today (RFC-009 Phase 5 flips it to /load — +// this matrix's `parity_load` row is where that flip becomes visible). +// - `init`, `optimize`, `repair`, `cleanup`, `cluster *`: storage-plane by +// design (must work with the server down); Phase 4 declares this. +#[allow(dead_code)] +const EXCLUSIONS_DOCUMENTED: () = (); + +#[test] +fn known_divergences_ledger_is_current() { + // The ledger exists so removals are deliberate: an empty list with all + // rows green means the arms agree everywhere the matrix looks. + assert!( + KNOWN_DIVERGENCES.is_empty(), + "divergences are pinned: {KNOWN_DIVERGENCES:?}" + ); +} diff --git a/crates/omnigraph-cli/tests/support/mod.rs b/crates/omnigraph-cli/tests/support/mod.rs index b11e94d..b36288c 100644 --- a/crates/omnigraph-cli/tests/support/mod.rs +++ b/crates/omnigraph-cli/tests/support/mod.rs @@ -688,3 +688,170 @@ pub fn queries_test_config(graph_uri: &str, entry: &str, gq_file: &str) -> Strin graph_uri.replace('\'', "''") ) } + +// ---- RFC-009 Phase 1: parity-matrix harness ---- + +/// Twin graphs for embedded-vs-remote comparison: the same loaded fixture +/// copied to two roots, so write verbs can run once per arm on identical +/// state. Returns (tempdir-guard, local_graph, remote_graph). +pub fn twin_graphs() -> (TempDir, PathBuf, PathBuf) { + let temp = tempdir().unwrap(); + let seed = temp.path().join("seed"); + fs::create_dir_all(&seed).unwrap(); + let graph = seed.join("server.omni"); + init_graph(&graph); + load_fixture(&graph); + let local = temp.path().join("local.omni"); + let remote = temp.path().join("remote.omni"); + copy_dir(&graph, &local); + copy_dir(&graph, &remote); + (temp, local, remote) +} + +pub fn copy_dir(from: &Path, to: &Path) { + fs::create_dir_all(to).unwrap(); + for entry in fs::read_dir(from).unwrap() { + let entry = entry.unwrap(); + let target = to.join(entry.file_name()); + if entry.file_type().unwrap().is_dir() { + copy_dir(&entry.path(), &target); + } else { + fs::copy(entry.path(), &target).unwrap(); + } + } +} + +/// Scrub declared-volatile fields (RFC-009 Phase 1 allowlist) so the rest +/// of the JSON must match exactly. Key-based, recursive; both arms get the +/// same placeholders. Everything NOT listed here is contract. +pub fn scrub_volatile(value: &mut serde_json::Value) { + const VOLATILE_KEYS: &[&str] = &[ + // identity-bearing per-instance values + "commit_id", "id", "parent_id", "merge_parent_id", "snapshot", + // wall-clock + "committed_at", "created_at", "timestamp", + // transport / location + "uri", "path", + ]; + match value { + serde_json::Value::Object(map) => { + for (key, val) in map.iter_mut() { + if VOLATILE_KEYS.contains(&key.as_str()) && !val.is_null() { + *val = serde_json::Value::String(format!("<volatile:{key}>")); + } else { + scrub_volatile(val); + } + } + } + serde_json::Value::Array(items) => { + for item in items { + scrub_volatile(item); + } + } + _ => {} + } +} + +pub const PARITY_ACTOR: &str = "act-parity"; +pub const PARITY_TOKEN: &str = "parity-tok"; + +/// Identical Cedar bundle for BOTH arms — like-for-like enforcement is part +/// of the parity contract (a bare local arm is permissive while a +/// tokens-only server is default-deny; comparing those would measure +/// configuration, not the fork). +pub fn parity_policy_yaml() -> String { + r#"version: 1 +groups: + parity: ["act-parity"] +protected_branches: [] +rules: + - id: reads + allow: + actors: { group: parity } + actions: [read, export, invoke_query] + - id: read-scope + allow: + actors: { group: parity } + actions: [read, export] + branch_scope: any + - id: writes + allow: + actors: { group: parity } + actions: [change] + branch_scope: any + - id: branching + allow: + actors: { group: parity } + actions: [schema_apply, branch_create, branch_delete, branch_merge] + target_branch_scope: any +"# + .to_string() +} + +/// Per-arm config files carrying the same policy. Both arms address the +/// graph by positional URI, so the TOP-LEVEL policy.file applies on each +/// side (single-graph semantics). +pub fn parity_configs(root: &Path, _local_graph: &Path, remote_graph: &Path) -> (PathBuf, PathBuf) { + let policy = root.join("parity.policy.yaml"); + fs::write(&policy, parity_policy_yaml()).unwrap(); + let local_cfg = root.join("local.omnigraph.yaml"); + fs::write( + &local_cfg, + format!("policy:\n file: {}\n", policy.display()), + ) + .unwrap(); + let server_cfg = root.join("server.omnigraph.yaml"); + fs::write( + &server_cfg, + format!( + "server:\n graph: parity\ngraphs:\n parity:\n uri: {}\n policy:\n file: {}\n", + remote_graph.display(), + policy.display() + ), + ) + .unwrap(); + (local_cfg, server_cfg) +} + +/// Run one CLI invocation per arm with identical verb args: locally against +/// `local_graph` (--as actor) and remotely against a server URL whose token +/// resolves to the same actor. Returns raw Outputs for exit-code + JSON +/// comparison by the caller. +pub fn run_both( + local_graph: &Path, + server_url: &str, + args: &[&str], +) -> (std::process::Output, std::process::Output) { + run_both_with_config(local_graph, None, server_url, args) +} + +pub fn run_both_with_config( + local_graph: &Path, + local_config: Option<&Path>, + server_url: &str, + args: &[&str], +) -> (std::process::Output, std::process::Output) { + let mut local = cli(); + local.arg(args[0]).arg(local_graph).args(&args[1..]).arg("--as").arg(PARITY_ACTOR); + if let Some(config) = local_config { + local.arg("--config").arg(config); + } + let local_out = local.output().unwrap(); + + let mut remote = cli(); + remote + .env("OMNIGRAPH_BEARER_TOKEN", PARITY_TOKEN) + .arg(args[0]) + .arg(server_url) + .args(&args[1..]); + let remote_out = remote.output().unwrap(); + (local_out, remote_out) +} + +/// Parse, scrub, and pretty-print for diffable assertion messages. +pub fn scrubbed_json(output: &std::process::Output) -> String { + let mut value: serde_json::Value = serde_json::from_slice(&output.stdout) + .unwrap_or_else(|e| panic!("non-JSON stdout ({e}): {output:?}")); + scrub_volatile(&mut value); + serde_json::to_string_pretty(&value).unwrap() +} diff --git a/docs/dev/rfc-009-unify-access-paths.md b/docs/dev/rfc-009-unify-access-paths.md index 43dcdf2..cada723 100644 --- a/docs/dev/rfc-009-unify-access-paths.md +++ b/docs/dev/rfc-009-unify-access-paths.md @@ -68,7 +68,7 @@ anything moves — mirroring the storage collapse, where the pinned contract tests gated the swap, and the test-monolith modularization (#192/#193), which makes Phase 3 tractable: the CLI dispatch is 1,184 lines today, not 4,200. -### Phase 1 — Parity matrix (the referee; do first, no refactor) +### Phase 1 — Parity matrix (the referee; do first, no refactor) *(landed)* A CLI integration test (extend the `system_local.rs` harness, which already spawns both binaries): one fixture graph; for every forked verb, run the @@ -81,6 +81,15 @@ This pins today's behavior so Phase 3 can't silently change it, and catches every future fork drift. It also incidentally covers utoipa annotation↔route mismatches (a lying `#[utoipa::path]` makes the remote leg 404). +**Phase 1 outcome (landed):** `crates/omnigraph-cli/tests/parity_matrix.rs` +— 11 rows green with an **empty divergence ledger**: with matched Cedar +policy on both arms, embedded and remote agree on every forked verb's +scrubbed JSON and exit codes. Two findings along the way: like-for-like +requires the same policy bundle on both arms (a tokens-only server is +default-deny by design — the harness encodes this), and inline execution's +unbound-param matches-all vs the invoke path's hard error is a cross-path +asymmetry, filed as #207 and pinned (not repaired) by the matrix. + ### Phase 2 — One wire-DTO crate Move the HTTP request/response types and the single `engine result → DTO` diff --git a/docs/dev/testing.md b/docs/dev/testing.md index f2b33de..866af69 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -7,7 +7,7 @@ This file is the always-on map of the test surface. **Consult it before every ta | Crate | Path | Style | |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (28 files), fixture-driven, share `tests/helpers/mod.rs` | -| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | Per-area suites (post-modularization): `cli_cluster.rs` (cluster command surface + operator-actor cascade), `cli_cluster_e2e.rs` (spawned-binary lifecycle compositions — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `cli_data.rs` (load/read/change/branch/commit/export/snapshot/policy/embed/maintenance + operator format cascade), `cli_schema_config.rs` (init/config, schema plan/apply, RFC-008 deprecation warnings + `config migrate` + strict mode), `cli_queries.rs`, `system_local.rs` (full-cycle cluster lifecycle with a spawned `--cluster` server, applied-policy enforcement over HTTP, keyed-credential auth, operator aliases), `system_remote.rs`; share `tests/support/mod.rs` (hermetic `OMNIGRAPH_HOME` by default) | +| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | Per-area suites (post-modularization): `cli_cluster.rs` (cluster command surface + operator-actor cascade), `cli_cluster_e2e.rs` (spawned-binary lifecycle compositions — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `cli_data.rs` (load/read/change/branch/commit/export/snapshot/policy/embed/maintenance + operator format cascade), `cli_schema_config.rs` (init/config, schema plan/apply, RFC-008 deprecation warnings + `config migrate` + strict mode), `cli_queries.rs`, `parity_matrix.rs` (RFC-009 Phase 1: the embedded-vs-remote referee — every forked verb run against both arms with matched Cedar policy and the same actor, scrubbed-JSON + exit-code equality; divergences are pinned in its `KNOWN_DIVERGENCES` ledger, never silently repaired), `system_local.rs` (full-cycle cluster lifecycle with a spawned `--cluster` server, applied-policy enforcement over HTTP, keyed-credential auth, operator aliases), `system_remote.rs`; share `tests/support/mod.rs` (hermetic `OMNIGRAPH_HOME` by default) | | `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated); `tests/s3_cluster.rs` (bucket-gated full lifecycle on object storage) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows), and 5A policy binding metadata (applies_to in the applied revision, binding-change diffing + convergence, pre-5A backfill), and the 5B serving-snapshot read API (converged read, refusal rows) | | `omnigraph-server` | `crates/omnigraph-server/tests/` | Per-area suites (post-modularization): `auth_policy.rs`, `data_routes.rs`, `schema_routes.rs`, `stored_queries.rs`, `multi_graph.rs` (cluster-mode boot — converged serving, policy binding wiring, boot refusals — + the concurrent branch-ops matrix), `boot_settings.rs` (mode inference, PolicySource), `s3.rs` (bucket-gated: single-graph serving + config-free `--cluster s3://` boot), `openapi.rs` (OpenAPI drift / regeneration); share `tests/support/mod.rs` | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | From 446b46d54841578b2d32da9f82993bee136ec895 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford <ragnor.comerford@gmail.com> Date: Sat, 13 Jun 2026 11:20:08 +0200 Subject: [PATCH 141/165] Recovery liveness, storage fault-injection matrix, and one storage implementation over object_store (#203) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * test(engine): pin the long-lived-handle heal contract for sidecar-covered drift A Phase B -> Phase C failure (commit_staged advanced Lance HEAD, manifest publish did not land, recovery sidecar persists) currently wedges every subsequent staged write on the same engine handle: the commit-time drift guard rejects with 'run omnigraph repair', but repair itself refuses while a recovery sidecar is pending, so a long-lived server can only recover by restart. The documented contract (writes.md 'Long-running servers', invariants.md invariant 5) says refresh-time roll-forward closes this residual without restart -- but no write path runs it. Two red tests pin the intended contract at the write entry points: a follow-up load (the POST /ingest shape: shared handle, no reopen) and a follow-up mutation must heal roll-forward-eligible sidecars in-process and then succeed. Currently failing with: table 'node:Company' has Lance HEAD version 2 ahead of manifest version 1; run `omnigraph repair` before writing The fix lands in the next commit. * fix(engine): heal pending recovery sidecars at the staged-write entry points Close the long-lived-process gap in the recovery protocol: a Phase B -> Phase C residual (per-table commit_staged landed, manifest publish did not, sidecar persists) previously recovered only at the next ReadWrite open or via an explicit refresh() that no production write path called, so a long-lived server wedged every subsequent write on the commit-time drift guard until restart. New recovery::heal_pending_sidecars_roll_forward: - one list_dir of __recovery/ at write entry (empty -> immediate return, the steady state), so the per-write cost is one storage list; - per sidecar, acquires the same per-(table_key, table_branch) write queues every sidecar writer holds from before write_sidecar until after delete_sidecar, then re-checks sidecar existence -- this serializes the heal against live writers instead of rolling an in-flight sidecar forward from under its writer (which would fail that writer's publish CAS spuriously). Lock order queues -> coordinator matches every writer's commit->publish path. This is the queue-acquisition design recovery.rs and write_queue.rs already documented for in-process recovery; - processes in RollForwardOnly mode: the common residual rolls forward in-process; rollback-eligible sidecars still defer to the next ReadWrite open (Dataset::restore is unsafe under concurrency). Wire it into load_as and mutate_as (before the inline delete path can advance any HEAD), and rebase Omnigraph::refresh onto the same helper so refresh stops racing live writers' sidecars. The maintenance entry points (apply_schema_as, branch_merge_as, ensure_indices) intentionally keep their strict fail-loud preconditions for now; wiring the same heal there is a follow-up with its own tests. Turns the previous commit's two red tests green. * fix(engine): name the right recovery path in the commit-time drift guard The drift guard's 'run omnigraph repair before writing' advice is a dead end when the drift is covered by a pending recovery sidecar: repair refuses while a sidecar is pending. With the write-entry heal in place, reaching this guard with sidecar-covered drift means the heal deferred it (rollback-eligible), and the actual recovery path is a read-write reopen. Distinguish the two classes on the error path only (one sidecar list, after the conflict is already certain); a listing failure falls back to the uncovered-drift wording rather than masking the conflict. Pinned by extending refresh_defers_rollback_eligible_sidecar_to_next_open with a write attempt against the deferred sidecar. * docs: write-entry in-process sidecar heal — contract and coverage Update the recovery contract docs to match the previous two commits: invariant 5 now states that the staged-write entry points and refresh run in-process roll-forward recovery (long-lived processes converge on the next write, not at restart); writes.md 'Long-running servers' describes the heal's queue-acquisition concurrency contract, the improved drift-guard error, and the entry points that intentionally do not heal yet; testing.md indexes the new failpoint tests; AGENTS.md capability matrix drops the claim that in-process recovery is entirely future work (only the rollback path remains with the background reconciler). * test(engine): pin the entry heal contract for schema apply and branch merge Without the write-entry heal, the two maintenance writers do worse than wedge on sidecar-covered drift -- they proceed and decide its fate implicitly: - schema apply re-plans table rewrites from the manifest pin, orphaning the drifted Phase-B commit (its rows silently vanish from the rewritten table) while the stale sidecar lingers to misclassify against the post-apply pins; - branch merge publishes over the drift, making the failed writer's commit visible as an unattributed side effect (no recovery audit row), and leaves the stale sidecar behind. Two red tests pin the intended contract: both entry points heal the sidecar first (attributed roll-forward), then run on the converged state. Currently failing on the stale-sidecar / dropped-rows assertions; the fix lands in the next commit. * fix(engine): heal pending recovery sidecars at the schema-apply and branch-merge entries Extend the write-entry heal to the remaining two write entry points. Unlike load/mutate (which wedge on the drift guard), these proceeded over sidecar-covered drift and decided its fate implicitly: - schema apply re-planned table rewrites from the manifest pin, orphaning the drifted Phase-B commit -- its rows silently vanished from the rewritten table -- while the stale sidecar lingered to misclassify against the post-apply pins; - branch merge published over the drift, making the failed writer's commit visible without a recovery audit row, and left the stale sidecar behind. Both now run the same queue-serialized roll-forward heal at entry, before their own sidecar exists, so recovery is attributed (audit row) and deterministic. ensure_indices stays heal-free: it runs inside the load / schema-apply flows after their entry heal. Turns the previous commit's two red tests green. Docs updated in the same change (invariant 5, writes.md, testing.md, AGENTS.md). * test(engine): pin Phase A sidecar-write failure semantics Storage fault-injection matrix, row 1: a sidecar PUT failure (S3 PutObject / fs write) in Phase A. New failpoint recovery.sidecar_write at the top of write_sidecar -- the single choke point all five sidecar writers go through -- models the storage error backend-generically. Also adds the other three storage-fault failpoints used by the following commits (recovery.sidecar_delete, recovery.sidecar_list, recovery.record_audit); each is a no-op without the failpoints feature. Pinned contract: every writer writes its sidecar BEFORE its first HEAD-advancing commit, so a put failure aborts with zero drift (no sidecar, Lance HEAD == manifest pin, no rows) and a transient fault never wedges the graph -- the same handle writes/merges normally once it clears. Covered for load (the staging writer) and branch_merge (the multi-table writer, forced onto the RewriteMerged path by diverging both sides). * test(engine): pin Phase D delete, list, and audit-append storage-fault semantics Storage fault-injection matrix, rows 2/3/5, plus the real-backend run: - recovery.sidecar_delete: a Phase D delete failure (S3 DeleteObject) must NOT fail the user's write -- the manifest publish already landed, so the caller's data is durable. The swallowed failure leaves a stale sidecar; the next write's entry heal consumes it via the stale-sidecar audit-recovery path (RolledForward, attributed). - recovery.sidecar_list: a __recovery/ list failure (S3 ListObjectsV2) is loud at every consumer -- the write-entry heal fails the write and the open-time sweep fails the open. Silently skipping recovery over a pending sidecar would be consumer tolerance of drift. Once the fault clears, open recovers the pending sidecar normally. - recovery.record_audit: an audit write failure after the roll-forward's manifest publish aborts that recovery attempt and keeps the sidecar; re-entry detects the already-published manifest, records exactly ONE RolledForward audit row, and converges -- the retry tolerance documented on record_audit, exercised end-to-end. - s3_load_recovers_after_publisher_failure_without_reopen: the same-handle heal scenario on a real bucket (gated on OMNIGRAPH_S3_TEST_BUCKET, skips locally), exercising sidecar put/list/delete through S3StorageAdapter instead of the local-FS adapter. CI wiring lands in a follow-up commit. * test(engine): refuse corrupt recovery sidecars loudly Storage fault-injection matrix, row 4 (no failpoint needed -- the corrupt file is written by hand, sibling to the unknown-schema-version refusal test): a truncated/garbage __recovery/{ulid}.json must be refused loudly by both the write-entry heal (the write fails naming the parse error) and the open-time sweep (ReadWrite open fails naming the file), with the file left on disk for operator inspection. Read-only opens still work -- the sweep is skipped there. * test(engine): run the S3 sidecar-lifecycle coverage in CI + document the fault matrix - ci.yml rustfs_integration: new step running the bucket-gated failpoints tests (name filter s3_) against the RustFS container, so sidecar put/list/delete are exercised through S3StorageAdapter on every storage-affecting PR. - writes.md: sidecar I/O failure semantics -- Phase A put failure aborts with zero drift; Phase D delete failure is swallowed (write already durable) and healed by the next write; list failures are loud at heal and open; corrupt sidecars are refused with the file kept for inspection; audit-append failures are retried to exactly one audit row. - testing.md: index the storage-fault matrix in the failpoints.rs row and the new RustFS CI line. * test(engine): pin read-visibility of acknowledged local if-absent writes The cluster lib test import_missing_state_creates_state_with_graph_- observation flakes at ~50% under full-workspace load ('EOF while parsing a value' reading back the state.json its own import just acknowledged). Root cause is in the engine's local storage adapter: write_text_if_absent writes through a buffered tokio::fs::File and returns when write_all resolves -- which, per tokio's documented File semantics, means the bytes reached tokio's internal buffer, not the file. The actual write completes in a background blocking task after drop, so a caller that acknowledges success and reads the object back can see an empty or partial file. Under load the window widens; the red run fails at iteration 0 with 0 of 8192 bytes on disk. The regression test pins the contract at the adapter boundary: when write_text_if_absent resolves, the full contents are visible to any reader; a losing second claim leaves the winner's object untouched. The fix lands in the next commit. * fix(engine): publish local storage writes with atomic visibility Close the class, not the instance. The local adapter admitted three ways for a reader to observe a write that was acknowledged or visible before its bytes were complete: 1. write_text_if_absent acknowledged success when the buffered tokio::fs::File write_all resolved -- i.e. when the bytes reached tokio's internal buffer, not the file. A caller reading back its own acknowledged write could see an empty object (the ~50% cluster import flake under full-workspace load; the regression test failed at iteration 0 with 0 of 8192 bytes visible). 2. The same call published its CLAIM (create_new) before its CONTENT, so concurrent readers saw an empty claimed file in the window. 3. write_text (plain tokio::fs::write) exposed truncated content mid-replace -- silently falsifying write_sidecar's 'readers either see the complete sidecar or none' contract on local FS (true on S3, where PutObject is atomic). A flush in write_text_if_absent would have fixed only (1). Instead, both local write paths now publish complete temp files atomically: rename for replace (write_text -- the idiom write_text_if_match already used) and hard_link for no-replace (write_text_if_absent -- link fails AlreadyExists, so exactly one of N concurrent claimants wins and the winner's object is fully readable at the instant it becomes visible). The local adapter now honors the same object-level atomic-visibility contract as the S3 adapter, which is what every caller (recovery sidecar protocol, cluster state CAS) was written against. Crash-orphaned *.tmp.* files are inert: the sidecar sweep filters to .json, and cluster state reads address state.json by name. fsync/durability policy is unchanged (no fsync before, none now); this fix is about visibility ordering, not power-loss durability. Pre-existing on main (landed with the multi-graph server mode change, PR #119); surfaced by this branch's heal work only because one extra list_dir per write shifted test timing. Cluster lib suite: 12/25 failures before, 0/25 after. Turns the previous commit's red test green. * refactor(engine): one storage implementation over object_store for every backend Collapse LocalStorageAdapter (hand-rolled tokio::fs) and S3StorageAdapter into a single ObjectStorageAdapter backed by Arc<dyn object_store::ObjectStore> -- LocalFileSystem for local URIs, the existing AmazonS3 build for s3://, plus a pub in_memory() constructor (full contract including TRUE conditional updates; the in-memory test backend testing.md asked for at the adapter level). Why: the acknowledged-before-visible bug showed the two-impl shape has no referee -- one prose contract, two independent answers. Upstream LocalFileSystem::put_opts is byte-for-byte the staged-temp+rename/ hard_link idiom that fix converged on, and Lance's own commit protocol is built on the same primitives (put-if-not-exists / rename-if-not- exists), so the substrate-aligned move is to stop hand-rolling it. The per-backend residue shrinks to a UriCodec (URI <-> object path) and one capability flag. Semantics preserved by construction, with three deliberate deltas: - exists() is now object-store-semantics everywhere (head + non-empty prefix fallback): an EMPTY local directory no longer 'exists'. The only dir-shaped caller (_graph_commits.lance probes) self-heals via ensure_commit_graph_initialized where it previously wedged loudly. - A directory at an object path reads as NotFound, not as an IO error ('only objects exist'). The cluster unreadable-payload test used a same-named directory as a portable non-NotFound trigger; it now uses chmod 000, which still models genuine transient IO. - write_text_if_match keeps content-token semantics on local (PutMode::Update is NotImplemented upstream for LocalFileSystem in 0.12.5 and 0.13.2); the capability flag gates the token SOURCE in read_text_versioned too -- an ETag token with content-compare writes would lose every CAS. delete_prefix keeps a local remove_dir_all branch: directories are a local-FS concept, and list+delete would leave empty skeletons that cluster graph_root_exists (raw Path::exists) reports as still present. LocalStorageAdapter remains as a delegating shim so the pinned contract tests gate this swap textually unchanged; the shim and the test parameterization over local + in-memory land next. Cargo gains the explicit 'fs' feature (already transitively enabled by lance). * test(engine): one executable storage contract, run against every backend Remove the LocalStorageAdapter delegation shim and migrate its construction sites to ObjectStorageAdapter::local(). Replace the per-backend duplicated tests with a single contract_suite asserting the trait's promises (atomic replace, exists incl. the dataset-root prefix probe, one-winner if_absent, versioned CAS with loud CAS-lost, rename, list round-trip with no sibling-prefix bleed, idempotent delete/delete_prefix), run against the local backend and the new in-memory backend -- which implements true conditional updates, so the strong-CAS path is exercised without a bucket. The bucket-gated S3 variant already exists (s3_adapter_conditional_writes_contract). New local-specific pins for the deliberate semantic edges of the collapse: empty directories are not objects (exists=false; the Lance dataset-root probe shape is the non-empty case), file://-anchored and spaces-in-path list output round-trips byte-identically into read_text, dot-segment paths are lexically absolutized (the CLI's ./graph.omni shape), and upstream rename creating missing destination parents. The acknowledged-write visibility regression test stays, now documenting that the cross-API std::fs read-back is the point. * refactor(cluster): drop put_json's per-backend atomicity branch The local temp+rename dance predates the storage adapter guaranteeing atomic visibility; now that write_text publishes via a staged temp + rename on the filesystem (and a single atomic PUT on object stores) by contract, the branch duplicated upstream behavior. One call, both backends. * docs: storage adapter collapse — contract, in-memory backend, local CAS gap - testing.md: the 'no MemStorage backend' note is half-closed — ObjectStorageAdapter::in_memory() covers the text-object layer with the full contract (true conditional updates); Lance datasets bypass the adapter, so the engine substrate ask stays open. - invariants.md: truth-matrix Tests row updated; new Known Gap for local write_text_if_match (upstream PutMode::Update is unimplemented for LocalFileSystem; content-token emulation is safe only under the cluster lock protocol — close before admitting a lock-free caller). - writes.md: backend notes for the unified adapter (name#N staging residue invisible to the sweep, backend-wrapped error text with exists()-probing for missing-vs-error, loud permission failures). * docs: finish renaming the storage adapters in user docs and test comments storage.md's URI-scheme table and the S3 failpoint test's doc comment still named the deleted LocalStorageAdapter/S3StorageAdapter; both now describe the unified ObjectStorageAdapter over object_store, including the relative-path absolutization note for local URIs. * test(engine): pin branch-awareness of the drift guard's recovery advice A pending sidecar on ANOTHER branch does not cover this branch's drift: with a deferred feature-branch sidecar on disk and genuinely uncovered drift on main, the main write's error must still point at omnigraph repair -- a read-write reopen recovers the sidecar but cannot repair main's uncovered drift. Currently red: the guard matches sidecar pins by table_key only, so the feature sidecar flips main's advice to the reopen path. Fix in the next commit. Surfaced by external review of the drift-guard change. * fix(engine): branch-aware sidecar matching in the drift guard's advice The commit-time drift guard's sidecar-covered check matched pins by table_key alone, so a pending sidecar on another branch flipped this branch's uncovered-drift advice from 'run omnigraph repair' to the reopen path -- and a reopen recovers that sidecar but cannot repair this branch's drift. Compare the pin's table_branch too. Turns the previous commit's red test green. Surfaced by external review of the drift-guard change. * test(engine): pin heal non-interference with a live schema apply The write-entry heal's schema-staging reconcile runs before any queue acquisition, so a load on the same handle, overlapping a schema apply parked between its staging write and manifest commit, promotes the apply's staging files (new catalog live against the old manifest), classifies the LIVE apply's sidecar, and publishes its registrations out from under it. The resumed apply then collides with its own stolen commit. Currently red with: Lance("Concurrent modification: table version 3 already exists for node:Tag") The fix (per-sidecar reconcile under the sidecar's write-queue guards, plus a serialization key the schema-apply writer and the heal both acquire) lands in the next commit. Surfaced by external review of the write-entry heal. * fix(engine): serialize the heal's schema-staging reconcile with live schema applies The write-entry heal ran recover_schema_state_files up front, before acquiring any queue guards. Overlapping a live schema apply parked between its staging write and manifest commit, the heal promoted the apply's staging files (new catalog live against the old manifest), classified the LIVE apply's sidecar, and published its registrations — the resumed apply then collided with its own stolen commit. Correct by construction: - New schema-apply serialization queue key, acquired by the schema- apply writer (alongside its per-table keys) from before write_sidecar until after delete_sidecar. Per-table keys alone don't cover a registration-only migration, which pins no existing tables but has a sidecar and staging files on disk. - The heal reconciles schema staging lazily, PER SchemaApply sidecar, after acquiring that sidecar's guards (including the serialization key) and re-confirming the sidecar exists — a sidecar that survives the queue wait belongs to a dead writer, so the reconcile can no longer race a live apply. Recomputing per sidecar also removes the staleness of one up-front result across a multi-sidecar pass. - Omnigraph::refresh drops its up-front reconcile-and-pass-through (same race, and a pre-promoted result would make the heal's guarded reconcile see clean staging and wrongly defer the sidecar): it now reconciles standalone only when NO sidecar exists — which cannot race a live apply, whose sidecar always precedes its staging files — and otherwise defers entirely to the heal. The open-time sweep keeps its precomputed reconcile: open has no concurrent writers. Turns the previous commit's red test green. Surfaced by external review of the write-entry heal. Self-audit addendum folded in: refresh's no-sidecar gate had a TOCTOU (a live apply could write its sidecar + staging between the empty check and the reconcile) — the standalone reconcile now holds the serialization key across the list-then-reconcile pair. The remaining residual is cross-process only (in-process queues cannot serialize against a writer in another process; the open-time sweep has the same pre-existing exposure) and is now an explicit Known Gap in invariants.md rather than an implicit one. * test(engine): pin catalog reload after the heal recovers a schema apply When the write-entry heal rolls a crashed apply's SchemaApply sidecar forward on the same handle, disk and manifest move to the new schema (staging promoted, registrations published) but the handle's in-memory schema_source/catalog do not. Subsequent writes then validate against the stale catalog and reject rows of types the graph already has. Currently red with: record 1: unknown node type 'Tag' refresh() reloads after its heal; the write entry points must too. Fix in the next commit. Surfaced by external review of the write-entry heal. * fix(engine): reload the in-memory catalog after the heal recovers a schema apply heal_pending_recovery_sidecars refreshed the coordinator and invalidated the runtime cache after processing sidecars, but never reloaded schema_source/catalog — so a write whose entry heal rolled a crashed SchemaApply sidecar forward proceeded to validate against the OLD schema while disk and manifest were already on the new one. reload_schema_if_source_changed is the same post-heal step refresh() already runs; it no-ops on the (overwhelmingly common) non-schema heal because the on-disk source is unchanged. Turns the previous commit's red test green. Surfaced by external review of the write-entry heal. * test(engine): pin that a deleted-branch sidecar cannot wedge the graph A rollback-eligible sidecar pinned to a branch is deferred by every roll-forward-only pass; if the branch is then deleted, the sidecar survives, referencing a branch with no manifest tree. The heal (every write entry) and the open-time sweep (every ReadWrite open) both fail opening the dead branch, and repair refuses while a sidecar is pending -- a terminal read-only state with manual sidecar surgery as the only exit. Currently red with: Lance("Not found: .../__manifest/tree/feature/_versions") The branch's tree and forks are already reclaimed, so the pinned drift is unreachable and the sidecar is provably moot; the fix classifies it as an orphaned-branch terminal state (audit + discard) in both passes. Surfaced by review (P1, verified by repro). * fix(engine): classify deleted-branch sidecars as orphaned instead of wedging A deferred (rollback-eligible) sidecar pinned to a branch survives branch_delete; both the write-entry heal and the open-time sweep then failed unconditionally opening the dead branch -- every write and every ReadWrite open errored, and repair refuses while a sidecar pends. Terminal state, manual sidecar surgery the only exit. The branch's tree and per-table forks are already reclaimed at delete, so the drift the sidecar pins is unreachable and the sidecar is provably moot. Both passes now check the sidecar's branch against the manifest's branch list (the authority -- deliberately NOT inferred from a Not-found on open, which could be a transient storage error masking real recovery intent) and discard orphans with an OrphanedBranchDiscarded audit row, commit appended on main since the sidecar's own branch no longer has a commit graph. The open-time half is pre-existing; the write-entry heal made it hot. Turns the previous commit's red test green. Surfaced by review (P1, verified by repro). * chore: harden review nits — vacuous CI filter, root-runner skip, liveness note - ci.yml: the RustFS sidecar-lifecycle step now fails loudly if the 's3_' name filter matches zero tests (cargo passes vacuously on an empty filter; the step exists specifically to prove S3 sidecar I/O coverage). The pre-existing CLI smoke step has the same shape and is left for a follow-up. - cluster unreadable-payload test: cfg(unix) + a skip-with-log when running as root (mode 000 is still readable to root, common in container dev runners), so the test degrades instead of failing. - refresh: document the one-pass-late convergence for legacy staging residue while non-SchemaApply sidecars pend, so nobody 'fixes' it by re-running the reconcile unserialized — the exact race the serialization key closes. * test(engine): pin orphan-discard idempotency across a delete fault discard_orphaned_branch_sidecar writes its audit row and main commit before deleting the sidecar; a Phase D delete fault leaves the sidecar on disk with the audit already durable, and the retry repeated the whole path -- a second OrphanedBranchDiscarded audit row (and commit) for the same operation. Currently red: 2 rows after one fault + retry. The retry must only finish the delete. Fix next. Also promotes the recovery-audit kinds reader into the shared test helpers (it was recovery.rs-local). Surfaced by external review of the orphan-discard fix. * fix(engine): orphan-discard idempotency + heal reports acted-vs-deferred Two review findings on the recovery surface: - discard_orphaned_branch_sidecar now checks the audit table for an existing (operation_id, OrphanedBranchDiscarded) row before appending the commit + audit pair, so a Phase D delete fault retries ONLY the delete instead of duplicating audit rows and commit-graph entries. Cold path: the list scan runs only when an orphaned sidecar exists. Turns the previous commit's red test green (exactly one audit row across fault + retry). - process_sidecar returns whether durable state changed; the heal sets processed_any only for sidecars that were actually rolled forward / rolled back / audit-recovered (orphan discards count). Deferred sidecars (rollback-eligible, invariant-violating, unpromoted SchemaApply) no longer trigger a per-write schema reload + full runtime-cache invalidation while they pend -- the cache is snapshot-keyed so this was waste, not corruption, but it was paid on every write until reopen. Acted-paths' processed=true remains pinned by load_after_schema_apply_phase_b_failure_uses_recovered_catalog (the reload depends on it). Surfaced by external review. * test(engine): pin the orphan-discard audit-append fault leg as documented tolerance The orphan discard's commit append and audit append are two writes; a failure between them leaves a recovery commit with no audit row, and the retry (keyed on the audit row, the operator-facing record) appends a second commit before the audit lands. This is the same not-atomic-pair-write tolerance record_audit documents and the manifest->commit-graph Known Gap covers for every publish: bounded commit-graph noise, audit row exactly-once under clean failures. Keying idempotency on commit rows instead would need an operation_id column on _graph_commits, and audit-before-commit would dangle the graph_commit_id join -- both worse than the documented residual. Make the tolerance explicit instead of implicit: docstring names the window, a failpoint sits inside it, and the new test pins convergence across the fault (sidecar consumed, exactly one audit row), completing the orphan-discard fault matrix alongside the delete-fault leg. Surfaced by external review of the orphan-discard idempotency. * test(engine): pin honest drift-guard advice when sidecar listing fails The guard's unwrap_or(false) conflated 'classified as uncovered' with 'could not classify': a transient list fault on the guard's second list (the entry heal's first list having succeeded) confidently routed the operator to omnigraph repair even when the heal had just deferred a rollback-eligible sidecar -- and repair refuses while a sidecar is pending. Currently red: the error says 'run omnigraph repair' with no mention of the reopen path. The fix names both paths plus the failure cause when classification is impossible. Surfaced by external review of the drift-guard fallback. * fix(engine): admit ambiguity in the drift guard when sidecar listing fails Replace the unwrap_or(false) fallback with a tri-state: covered -> reopen advice; uncovered -> repair advice; listing FAILED -> say the drift could not be classified, name the cause, and give both paths in order ('run repair, or reopen read-write if repair reports a pending sidecar'). The old fallback confidently routed a transient list fault to repair, which refuses while a sidecar is pending -- a self- correcting but pointless detour. The conflict itself is still always raised; only the advice degrades honestly. Turns the previous commit's red test green. Surfaced by external review of the drift-guard fallback. --- .github/workflows/ci.yml | 14 + AGENTS.md | 2 +- Cargo.toml | 2 +- crates/omnigraph-cluster/src/store.rs | 31 +- crates/omnigraph-cluster/src/tests.rs | 22 +- crates/omnigraph/src/db/manifest.rs | 3 +- crates/omnigraph/src/db/manifest/recovery.rs | 326 ++++- crates/omnigraph/src/db/omnigraph.rs | 152 +- .../src/db/omnigraph/schema_apply.rs | 29 +- crates/omnigraph/src/db/recovery_audit.rs | 7 + crates/omnigraph/src/exec/merge.rs | 7 + crates/omnigraph/src/exec/mutation.rs | 8 + crates/omnigraph/src/exec/staging.rs | 48 +- crates/omnigraph/src/loader/mod.rs | 7 + crates/omnigraph/src/storage.rs | 1070 ++++++++------ crates/omnigraph/tests/failpoints.rs | 1236 +++++++++++++++++ crates/omnigraph/tests/helpers/recovery.rs | 33 + crates/omnigraph/tests/recovery.rs | 212 +++ docs/dev/invariants.md | 35 +- docs/dev/testing.md | 5 +- docs/dev/writes.md | 74 +- docs/user/storage.md | 4 +- 22 files changed, 2778 insertions(+), 549 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b2b18d8..56ef3e3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -365,6 +365,20 @@ jobs: - name: Run RustFS CLI smoke run: cargo test --locked -p omnigraph-cli --test system_local local_cli_s3_end_to_end_init_load_read_flow -- --nocapture + - name: Run RustFS recovery-sidecar lifecycle + # Sidecar put/list/delete through the S3 storage backend on a + # real bucket (the failpoint only wedges the publisher; the + # sidecar I/O is exercised for real). Name filter `s3_` matches + # the bucket-gated tests in the failpoints target only; the + # grep guards against the filter going vacuous (cargo passes + # with 0 tests matched) if those tests are ever renamed. + run: | + output=$(cargo test --locked -p omnigraph-engine --features failpoints --test failpoints s3_ -- --nocapture 2>&1); status=$? + echo "$output" + [ "$status" -eq 0 ] || exit "$status" + echo "$output" | grep -Eq "test result: ok\. [1-9][0-9]* passed" \ + || { echo "::error::filter 's3_' matched no tests — vacuous pass"; exit 1; } + - name: Dump RustFS logs on failure if: failure() run: docker logs rustfs diff --git a/AGENTS.md b/AGENTS.md index 87d6a46..d9e0c45 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -240,7 +240,7 @@ omnigraph policy explain --actor act-alice --action change --branch main | Columnar storage on object store | ✅ Arrow/Lance | URI normalization, S3 env-var plumbing | | Per-dataset versioning + time travel | ✅ | `snapshot_at_version`, `entity_at`, snapshot-pinned reads across many tables | | Per-dataset branches | ✅ | **Graph-level** branches (atomic across all sub-tables), lazy fork, system branch filtering | -| Atomic single-dataset commits | ✅ | **Multi-table publish via three layers**, NOT a single Lance primitive: (1) per-table Lance `commit_staged` for the data write, (2) `__manifest` row-level CAS via `ManifestBatchPublisher` for cross-table ordering, (3) the open-time recovery sweep for the residual gap between (1) and (2). All three layers ship; the five migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`, `optimize_all_tables`) write a `__recovery/{ulid}.json` sidecar before Phase B and delete it after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the sweep in `db/manifest/recovery.rs`: classify, decide all-or-nothing per sidecar, roll forward via single `ManifestBatchPublisher::publish` or roll back via `Dataset::restore` followed by a manifest publish of the restored version (so both directions converge to `manifest == HEAD` — no residual drift), and record an audit row in `_graph_commit_recoveries.lance` (queryable via `omnigraph commit list --filter actor=omnigraph:recovery`). Continuous in-process recovery (no restart needed between Phase B failure and recovery) is the goal of a future background reconciler. Engine writes route through a sealed `TableStorage` trait (`db.storage()`) exposing only `stage_*` + `commit_staged` + reads; the inline-commit residuals (`delete_where`, `create_vector_index`) are split onto a separate sealed `InlineCommitResidual` trait reached via `db.storage_inline_residual()` (MR-854), so the default surface cannot couple a write with a HEAD advance — §1 holds by construction. `delete_where` and `create_vector_index` stay inline until upstream Lance ships a public two-phase API ([#6658](https://github.com/lance-format/lance/issues/6658), [#6666](https://github.com/lance-format/lance/issues/6666)); `LoadMode::Overwrite` uses Lance `Overwrite` staged transactions. | +| Atomic single-dataset commits | ✅ | **Multi-table publish via three layers**, NOT a single Lance primitive: (1) per-table Lance `commit_staged` for the data write, (2) `__manifest` row-level CAS via `ManifestBatchPublisher` for cross-table ordering, (3) the open-time recovery sweep for the residual gap between (1) and (2). All three layers ship; the five migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`, `optimize_all_tables`) write a `__recovery/{ulid}.json` sidecar before Phase B and delete it after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the sweep in `db/manifest/recovery.rs`: classify, decide all-or-nothing per sidecar, roll forward via single `ManifestBatchPublisher::publish` or roll back via `Dataset::restore` followed by a manifest publish of the restored version (so both directions converge to `manifest == HEAD` — no residual drift), and record an audit row in `_graph_commit_recoveries.lance` (queryable via `omnigraph commit list --filter actor=omnigraph:recovery`). The write entry points (`load_as`, `mutate_as`, `apply_schema_as`, `branch_merge_as`) and `refresh` additionally run an in-process roll-forward-only heal (serialized against live writers via the per-table write queues), so a long-lived server converges on its next write without restart; only rollback-eligible sidecars still defer to the next read-write open (a future background reconciler's goal). Engine writes route through a sealed `TableStorage` trait (`db.storage()`) exposing only `stage_*` + `commit_staged` + reads; the inline-commit residuals (`delete_where`, `create_vector_index`) are split onto a separate sealed `InlineCommitResidual` trait reached via `db.storage_inline_residual()` (MR-854), so the default surface cannot couple a write with a HEAD advance — §1 holds by construction. `delete_where` and `create_vector_index` stay inline until upstream Lance ships a public two-phase API ([#6658](https://github.com/lance-format/lance/issues/6658), [#6666](https://github.com/lance-format/lance/issues/6666)); `LoadMode::Overwrite` uses Lance `Overwrite` staged transactions. | | Compaction (`compact_files`) | ✅ | `omnigraph optimize` orchestrates over all node/edge tables, bounded concurrency; **publishes each compacted table's new version to `__manifest`** (so the manifest tracks the Lance HEAD — required for reads to observe compaction and for schema apply / strict writes to pass their HEAD-vs-manifest precondition), under the per-`(table, main)` write queue with `SidecarKind::Optimize` recovery coverage; **refuses on an unrecovered graph** (errors if a `__recovery` sidecar is pending); **skips uncovered HEAD > manifest drift** with `DriftNeedsRepair` instead of interpreting it; **skips blob-bearing tables** (reported via `TableOptimizeStats.skipped`, not silent), gated on `LANCE_SUPPORTS_BLOB_COMPACTION` until the upstream blob-v2 compaction-decode bug is fixed (see [docs/dev/invariants.md](docs/dev/invariants.md) Known Gaps) | | Repair uncovered drift | — | `omnigraph repair` explicitly classifies uncovered table `HEAD > manifest` drift: verified maintenance drift (`ReserveFragments`/`Rewrite`) can be published with `--confirm`; suspicious or unverifiable drift requires `--force --confirm`. Sidecar-covered crash residuals still recover automatically on open. | | Cleanup (`cleanup_old_versions`) | ✅ | `omnigraph cleanup` with `--keep` / `--older-than` policy | diff --git a/Cargo.toml b/Cargo.toml index 17990ea..918ac05 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -63,7 +63,7 @@ base64 = "0.22" ariadne = "0.4" regex = "1" reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] } -object_store = { version = "0.12.5", default-features = false, features = ["aws"] } +object_store = { version = "0.12.5", default-features = false, features = ["aws", "fs"] } fail = "0.5" time = { version = "0.3", features = ["formatting"] } axum = { version = "0.8", features = ["json", "macros"] } diff --git a/crates/omnigraph-cluster/src/store.rs b/crates/omnigraph-cluster/src/store.rs index 4d33d2c..620df96 100644 --- a/crates/omnigraph-cluster/src/store.rs +++ b/crates/omnigraph-cluster/src/store.rs @@ -169,31 +169,16 @@ impl ClusterStore { .map_err(|err| err.to_string()) } - /// JSON object write with the strongest atomicity the backend offers: - /// temp + rename on the filesystem (no torn JSON after a crash; the - /// pre-port behavior), a single atomic PUT on object stores (where - /// copy+delete would be weaker, not stronger). + /// JSON object write. Atomic visibility is the storage adapter's + /// contract on every backend (staged temp + rename on the filesystem, + /// a single atomic PUT on object stores) — no torn JSON after a crash, + /// no per-backend branch needed here. async fn put_json(&self, relative: &str, payload: &str) -> Result<(), String> { let target = self.uri(relative); - match self.kind() { - StorageKind::Local => { - let tmp = format!("{target}.tmp.{}", Ulid::new()); - self.adapter - .write_text(&tmp, payload) - .await - .map_err(|err| err.to_string())?; - if let Err(err) = self.adapter.rename_text(&tmp, &target).await { - let _ = self.adapter.delete(&tmp).await; - return Err(err.to_string()); - } - Ok(()) - } - StorageKind::S3 => self - .adapter - .write_text(&target, payload) - .await - .map_err(|err| err.to_string()), - } + self.adapter + .write_text(&target, payload) + .await + .map_err(|err| err.to_string()) } /// Shared list-and-parse for the sidecar/approval directories: id diff --git a/crates/omnigraph-cluster/src/tests.rs b/crates/omnigraph-cluster/src/tests.rs index 63e7da7..805ecda 100644 --- a/crates/omnigraph-cluster/src/tests.rs +++ b/crates/omnigraph-cluster/src/tests.rs @@ -1950,13 +1950,29 @@ graphs: } #[tokio::test] + #[cfg(unix)] async fn refresh_flags_unreadable_payload_as_error() { let dir = fixture(); init_derived_graph(dir.path()).await; let blob = converge_fixture(dir.path()).await; - // A same-named directory yields a non-NotFound IO error portably. - fs::remove_file(&blob).unwrap(); - fs::create_dir(&blob).unwrap(); + // Make the payload unreadable without removing it: permission + // denied is a genuine non-NotFound IO error. (A same-named + // directory no longer triggers this path: object-store semantics + // classify a directory at an object path as NotFound — "only + // objects exist" — which is the missing-payload case, not the + // unreadable one.) + let mut perms = fs::metadata(&blob).unwrap().permissions(); + std::os::unix::fs::PermissionsExt::set_mode(&mut perms, 0o000); + fs::set_permissions(&blob, perms).unwrap(); + // Root reads straight through mode 000 (container dev runners + // commonly run as root): skip rather than fail — the contract + // under test needs a genuine permission error. + if fs::read(&blob).is_ok() { + eprintln!( + "skipping refresh_flags_unreadable_payload_as_error: running as root (mode 000 is still readable)" + ); + return; + } let out = refresh_config_dir(dir.path()).await; assert!(!out.ok); diff --git a/crates/omnigraph/src/db/manifest.rs b/crates/omnigraph/src/db/manifest.rs index 5bf1f87..6cd271a 100644 --- a/crates/omnigraph/src/db/manifest.rs +++ b/crates/omnigraph/src/db/manifest.rs @@ -36,7 +36,8 @@ use publisher::{GraphNamespacePublisher, ManifestBatchPublisher}; pub(crate) use recovery::{ RecoveryMode, RecoverySidecar, RecoverySidecarHandle, SidecarKind, SidecarTablePin, SidecarTableRegistration, SidecarTombstone, delete_sidecar, has_schema_apply_sidecar, - list_sidecars, new_sidecar, recover_manifest_drift, write_sidecar, + heal_pending_sidecars_roll_forward, list_sidecars, new_sidecar, recover_manifest_drift, + schema_apply_serial_queue_key, write_sidecar, }; pub use state::SubTableEntry; #[cfg(test)] diff --git a/crates/omnigraph/src/db/manifest/recovery.rs b/crates/omnigraph/src/db/manifest/recovery.rs index 3b0f147..d49e86a 100644 --- a/crates/omnigraph/src/db/manifest/recovery.rs +++ b/crates/omnigraph/src/db/manifest/recovery.rs @@ -28,10 +28,11 @@ //! CreateIndex/Merge — see `check_restore_txn` at lance-6.0.1 //! `src/io/commit/conflict_resolver.rs:986`. The hazard is documented //! by `tests/staged_writes.rs::lance_restore_loses_to_concurrent_append_via_orphaning`. -//! This module sidesteps the hazard by running recovery only at -//! `Omnigraph::open` (before any other writers can race). A future -//! continuous in-process recovery reconciler will need to guard via -//! per-(table_key, branch) queue acquisition. +//! The open-time sweep sidesteps the hazard by running before any +//! other writers can race; the in-process heal +//! ([`heal_pending_sidecars_roll_forward`]) never restores (roll- +//! forward only) and guards via per-(table_key, branch) queue +//! acquisition. use std::collections::HashMap; @@ -316,8 +317,8 @@ pub(crate) fn sidecar_uri(root_uri: &str, operation_id: &str) -> String { /// Write a sidecar atomically and return a handle for later deletion. /// /// The atomicity contract is inherited from [`StorageAdapter::write_text`]: -/// LocalStorageAdapter writes via `tokio::fs::write` (whole-file replace); -/// S3StorageAdapter writes via PutObject (atomic at the object level). +/// the local backend publishes via a staged temp file + rename (atomic on +/// POSIX); object stores write via PutObject (atomic at the object level). /// Both are sufficient for sidecar semantics — readers either see the /// complete sidecar or none. pub(crate) async fn write_sidecar( @@ -325,6 +326,9 @@ pub(crate) async fn write_sidecar( storage: &dyn StorageAdapter, sidecar: &RecoverySidecar, ) -> Result<RecoverySidecarHandle> { + // Failpoint: models a storage put failure (S3 PutObject / fs write) + // in Phase A — every writer must abort before any HEAD advance. + crate::failpoints::maybe_fail("recovery.sidecar_write")?; debug_assert_eq!(sidecar.schema_version, SIDECAR_SCHEMA_VERSION); let uri = sidecar_uri(root_uri, &sidecar.operation_id); let json = serde_json::to_string_pretty(sidecar).map_err(|err| { @@ -342,6 +346,10 @@ pub(crate) async fn delete_sidecar( handle: &RecoverySidecarHandle, storage: &dyn StorageAdapter, ) -> Result<()> { + // Failpoint: models a storage delete failure (S3 DeleteObject) in + // Phase D — callers swallow it (the write already published) and the + // stale sidecar is healed by the next write or open. + crate::failpoints::maybe_fail("recovery.sidecar_delete")?; storage.delete(&handle.sidecar_uri).await } @@ -356,6 +364,10 @@ pub(crate) async fn list_sidecars( root_uri: &str, storage: &dyn StorageAdapter, ) -> Result<Vec<RecoverySidecar>> { + // Failpoint: models a storage list failure (S3 ListObjectsV2) — every + // consumer (open-time sweep, write-entry heal) must fail loudly + // rather than silently skipping recovery. + crate::failpoints::maybe_fail("recovery.sidecar_list")?; let dir = recovery_dir_uri(root_uri); let mut uris = storage.list_dir(&dir).await?; // Sort by URI so the sweep processes sidecars deterministically. @@ -534,6 +546,240 @@ pub(crate) async fn restore_table_to_version( Ok(()) } +/// In-process heal for pending recovery sidecars — the entry point for +/// long-lived handles (`Omnigraph::refresh` and the staged-write entry +/// points `load_as` / `mutate_as`). +/// +/// Steady-state cost is one `list_dir` of `__recovery/` (typically +/// empty → immediate return), so write entry points can afford to call +/// this on every request. When sidecars exist, each is processed in +/// `RecoveryMode::RollForwardOnly`: the common Phase B → Phase C +/// residual (per-table `commit_staged` landed, manifest publish did +/// not) rolls forward in-process; rollback-eligible or invariant- +/// violating sidecars are deferred to the next ReadWrite open, exactly +/// as `Omnigraph::refresh` documents. +/// +/// Concurrency: unlike the open-time sweep, this runs while other +/// writers may be in flight. Every sidecar writer (mutation/load +/// finalize, schema_apply, branch_merge, ensure_indices, optimize) +/// acquires its per-`(table_key, table_branch)` write queues *before* +/// `write_sidecar` and holds them until after `delete_sidecar` — so +/// acquiring the same queues here blocks until that writer either +/// finished (sidecar deleted; the existence re-check skips it) or died +/// (sidecar is genuinely orphaned; safe to process). Without this, the +/// heal could observe a live writer's sidecar in its commit→publish +/// window, roll it forward, and fail that writer's own publish CAS. +/// Lock order is queues → coordinator, matching every writer's +/// commit→publish path. +/// +/// The schema-staging reconcile runs lazily, per SchemaApply sidecar, +/// AFTER that sidecar's queue guards are held and its existence is +/// re-confirmed — never up front. An up-front reconcile can promote a +/// LIVE schema apply's staging files and steal its commit (pinned by +/// `tests/failpoints.rs::heal_does_not_promote_live_schema_apply_staging`). +/// +/// Returns `true` when at least one sidecar was processed (the caller +/// should invalidate per-snapshot caches). +pub(crate) async fn heal_pending_sidecars_roll_forward( + root_uri: &str, + storage: std::sync::Arc<dyn StorageAdapter>, + coordinator: &tokio::sync::RwLock<GraphCoordinator>, + write_queue: &crate::db::write_queue::WriteQueueManager, +) -> Result<bool> { + let sidecars = list_sidecars(root_uri, storage.as_ref()).await?; + if sidecars.is_empty() { + return Ok(false); + } + let mut processed_any = false; + for sidecar in sidecars { + // Serialize against a possibly-live writer (see fn docs). Guards + // are scoped per sidecar so two sidecars never hold queues + // simultaneously (no cross-sidecar lock-order surface). + let mut queue_keys: Vec<crate::db::write_queue::TableQueueKey> = sidecar + .tables + .iter() + .map(|pin| (pin.table_key.clone(), pin.table_branch.clone())) + .collect(); + let is_schema_apply = matches!(sidecar.writer_kind, SidecarKind::SchemaApply); + if is_schema_apply { + // A SchemaApply sidecar's per-table pins don't cover a + // registration-only migration (no existing tables touched, + // but staging files + a sidecar on disk). The schema-apply + // writer holds this serialization key from before its + // sidecar write until after its sidecar delete, so blocking + // on it — then re-checking sidecar existence — guarantees + // the writer is gone before the reconcile below mutates + // schema staging. + queue_keys.push(schema_apply_serial_queue_key()); + } + let _guards = write_queue.acquire_many(&queue_keys).await; + // Re-check after the wait: the writer we blocked on may have + // completed Phase C and deleted its sidecar. + if !storage + .exists(&sidecar_uri(root_uri, &sidecar.operation_id)) + .await? + { + continue; + } + // Schema-staging reconcile, per SchemaApply sidecar, UNDER the + // sidecar's guards: a sidecar still on disk after the queue wait + // belongs to a dead writer, so promoting its staging files can no + // longer race the live apply's own renames or steal its commit. + // It also re-runs per sidecar, so a multi-sidecar pass never + // classifies against a reconcile result an earlier roll-forward + // staled. Non-SchemaApply sidecars never consult the value. + let schema_state_recovery = if is_schema_apply { + let snapshot = { + let mut coord = coordinator.write().await; + coord.refresh().await?; + coord.snapshot() + }; + crate::db::schema_state::recover_schema_state_files( + root_uri, + std::sync::Arc::clone(&storage), + &snapshot, + ) + .await? + } else { + SchemaStateRecovery::Noop + }; + // Fresh per-branch snapshot — same rationale as + // `recover_manifest_drift`: classify against the branch the + // sidecar's writer targeted, refreshed after any prior + // sidecar's roll-forward. + let branch_snapshot = match sidecar.branch.as_deref() { + Some(b) => { + // Orphan check against the manifest's branch list (the + // authority) BEFORE opening: a deferred sidecar whose + // branch was deleted would otherwise wedge every write + // on the dead-branch open. + let (branch_exists, main_version) = { + let mut coord = coordinator.write().await; + coord.refresh().await?; + let exists = coord.all_branches().await?.iter().any(|name| name == b); + (exists, coord.snapshot().version()) + }; + if !branch_exists { + discard_orphaned_branch_sidecar( + root_uri, + storage.as_ref(), + &sidecar, + main_version, + ) + .await?; + processed_any = true; + continue; + } + let mut branch_coord = + GraphCoordinator::open_branch(root_uri, b, std::sync::Arc::clone(&storage)) + .await?; + branch_coord.refresh().await?; + branch_coord.snapshot() + } + None => { + let mut coord = coordinator.write().await; + coord.refresh().await?; + coord.snapshot() + } + }; + if process_sidecar( + root_uri, + storage.as_ref(), + &branch_snapshot, + &sidecar, + RecoveryMode::RollForwardOnly, + schema_state_recovery, + ) + .await? + { + processed_any = true; + } + } + // Re-read coordinator state so the caller's handle observes the + // post-heal manifest. + coordinator.write().await.refresh().await?; + Ok(processed_any) +} + +/// Discard a sidecar whose branch no longer exists in the manifest (the +/// authority — callers must key the orphan classification off the branch +/// LIST, never off a `Not found` from an open, which could be a transient +/// storage error masking real recovery intent). The branch's tree and +/// per-table forks are already reclaimed, so the drift the sidecar pins is +/// unreachable and the sidecar is provably moot; leaving it would wedge +/// every heal (write entry) and every ReadWrite open on a dead-branch +/// open, with `repair` refusing while it pends. Records an +/// `OrphanedBranchDiscarded` audit row (commit appended on main — the +/// sidecar's own branch has no commit graph anymore). +async fn discard_orphaned_branch_sidecar( + root_uri: &str, + storage: &dyn StorageAdapter, + sidecar: &RecoverySidecar, + manifest_version: u64, +) -> Result<()> { + warn!( + operation_id = sidecar.operation_id.as_str(), + writer_kind = ?sidecar.writer_kind, + branch = sidecar.branch.as_deref().unwrap_or("<none>"), + "recovery: discarding sidecar for a deleted branch (drift unreachable; audit recorded)" + ); + let mut audit = RecoveryAudit::open(root_uri).await?; + // Idempotency across a Phase D delete fault: the audit row + commit + // land before the sidecar delete, so a failed delete re-enters here + // with the audit already durable. Append only once per operation — + // the retry's sole remaining job is finishing the delete. (Cold + // path: the list scan runs only when an orphaned sidecar exists.) + // + // Documented residual: the commit append and the audit append are + // two writes. A failure BETWEEN them leaves a recovery commit with + // no audit row; the retry (keyed on the audit row, the operator- + // facing record) appends a second commit before the audit lands — + // bounded commit-graph noise, audit row still exactly-once. Same + // not-atomic-pair-write tolerance as `record_audit` and the + // manifest→commit-graph Known Gap; keying on commit rows instead + // would need an operation_id column on `_graph_commits`, and + // audit-before-commit would dangle the `graph_commit_id` join. + let already_recorded = audit.list().await?.iter().any(|record| { + record.operation_id == sidecar.operation_id + && record.recovery_kind == RecoveryKind::OrphanedBranchDiscarded + }); + if !already_recorded { + let mut graph = CommitGraph::open(root_uri).await?; + let graph_commit_id = graph + .append_commit(None, manifest_version, Some(RECOVERY_ACTOR)) + .await?; + // Failpoint: the residual window above — commit appended, audit + // not yet durable. + crate::failpoints::maybe_fail("recovery.orphan_discard_audit_append")?; + audit + .append(RecoveryAuditRecord { + graph_commit_id, + recovery_kind: RecoveryKind::OrphanedBranchDiscarded, + recovery_for_actor: sidecar.actor_id.clone(), + operation_id: sidecar.operation_id.clone(), + sidecar_writer_kind: format!("{:?}", sidecar.writer_kind), + per_table_outcomes: Vec::new(), + created_at: now_micros()?, + }) + .await?; + } + let handle = RecoverySidecarHandle { + operation_id: sidecar.operation_id.clone(), + sidecar_uri: sidecar_uri(root_uri, &sidecar.operation_id), + }; + delete_sidecar(&handle, storage).await +} + +/// The write-queue key serializing schema-apply's sidecar lifecycle +/// against the write-entry heal. The schema-apply writer acquires it +/// (alongside its per-table keys) from before `write_sidecar` until +/// after `delete_sidecar`; the heal acquires it before reconciling +/// schema staging or processing a SchemaApply sidecar. The name cannot +/// collide with real table keys (those are `node:`/`edge:`-prefixed). +pub(crate) fn schema_apply_serial_queue_key() -> crate::db::write_queue::TableQueueKey { + ("__schema_apply__".to_string(), None) +} + /// Open-time recovery sweep — the entry point invoked from /// `Omnigraph::open` (gated on `OpenMode::ReadWrite`). /// @@ -549,9 +795,10 @@ pub(crate) async fn restore_table_to_version( /// /// Concurrency: today recovery runs synchronously in `Omnigraph::open` /// *before* the engine is wrapped in the server's `Arc<RwLock<Omnigraph>>`. -/// No request handlers can race. A future per-(table_key, branch) writer -/// queue model (paired with a background reconciler) will need to acquire -/// queues before the sweep restores or publishes. +/// No request handlers can race, so this sweep does NOT acquire write +/// queues. In-process callers (refresh, write entry points) must use +/// [`heal_pending_sidecars_roll_forward`] instead, which serializes +/// against live writers via per-(table_key, branch) queue acquisition. pub(crate) async fn recover_manifest_drift( root_uri: &str, storage: std::sync::Arc<dyn StorageAdapter>, @@ -578,6 +825,21 @@ pub(crate) async fn recover_manifest_drift( for sidecar in sidecars { let branch_snapshot = match sidecar.branch.as_deref() { Some(b) => { + // Orphan check against the manifest's branch list (the + // authority) BEFORE opening — same classification as the + // write-entry heal: a deferred sidecar whose branch was + // deleted would otherwise fail every ReadWrite open. + coordinator.refresh().await?; + if !coordinator.all_branches().await?.iter().any(|name| name == b) { + discard_orphaned_branch_sidecar( + root_uri, + storage.as_ref(), + &sidecar, + coordinator.snapshot().version(), + ) + .await?; + continue; + } let mut branch_coord = GraphCoordinator::open_branch(root_uri, b, std::sync::Arc::clone(&storage)) .await?; @@ -611,7 +873,11 @@ async fn process_sidecar( sidecar: &RecoverySidecar, mode: RecoveryMode, schema_state_recovery: SchemaStateRecovery, -) -> Result<()> { +) -> Result<bool> { + // Returns whether durable state changed (roll-forward, roll-back, or + // stale-sidecar audit recovery). `false` = the sidecar was deferred + // untouched -- callers must not treat that as a completed heal (no + // schema reload / cache invalidation is warranted). let mut states = Vec::with_capacity(sidecar.tables.len()); for pin in &sidecar.tables { let lance_head = open_lance_head(&pin.table_path, pin.table_branch.as_deref()).await?; @@ -655,7 +921,7 @@ async fn process_sidecar( writer_kind = ?sidecar.writer_kind, "recovery: deferring sidecar with invariant violation to next ReadWrite open" ); - Ok(()) + Ok(false) } }, SidecarDecision::RollBack => { @@ -701,7 +967,8 @@ async fn process_sidecar( return record_audit_recovery_rollforward( root_uri, storage, snapshot, sidecar, &states, ) - .await; + .await + .map(|()| true); } if matches!(mode, RecoveryMode::RollForwardOnly) { // In-process recovery cannot run Dataset::restore safely @@ -713,14 +980,16 @@ async fn process_sidecar( writer_kind = ?sidecar.writer_kind, "recovery: deferring rollback-eligible sidecar to next ReadWrite open" ); - return Ok(()); + return Ok(false); } warn!( operation_id = sidecar.operation_id.as_str(), writer_kind = ?sidecar.writer_kind, "recovery: rolling back sidecar (mixed or unexpected state)" ); - roll_back_sidecar(root_uri, storage, snapshot, sidecar, &states).await + roll_back_sidecar(root_uri, storage, snapshot, sidecar, &states) + .await + .map(|()| true) } SidecarDecision::RollForward => { if matches!(sidecar.writer_kind, SidecarKind::SchemaApply) @@ -733,7 +1002,9 @@ async fn process_sidecar( "recovery: rolling back SchemaApply sidecar because schema staging \ files were not promoted in this recovery pass" ); - roll_back_sidecar(root_uri, storage, snapshot, sidecar, &states).await + roll_back_sidecar(root_uri, storage, snapshot, sidecar, &states) + .await + .map(|()| true) } RecoveryMode::RollForwardOnly => { warn!( @@ -741,7 +1012,7 @@ async fn process_sidecar( "recovery: deferring SchemaApply sidecar because schema staging files \ were not promoted in this recovery pass" ); - Ok(()) + Ok(false) } }; } @@ -788,7 +1059,7 @@ async fn process_sidecar( ) .await?; delete_sidecar_by_operation_id(root_uri, storage, &sidecar.operation_id).await?; - Ok(()) + Ok(true) } } } @@ -1134,6 +1405,11 @@ async fn record_audit( kind: RecoveryKind, outcomes: Vec<TableOutcome>, ) -> Result<()> { + // Failpoint: models an audit write failure after the roll-forward / + // roll-back publish already landed — the sweep aborts, the sidecar + // stays, and re-entry records the audit row (see the retry note in + // the doc comment above). + crate::failpoints::maybe_fail("recovery.record_audit")?; // Non-main recovery commits must be appended on the sidecar branch's // commit graph, otherwise parent_commit_id comes from the global // main head. BranchMerge additionally records the source branch's @@ -1260,7 +1536,7 @@ pub(crate) fn new_sidecar( #[cfg(test)] mod tests { use super::*; - use crate::storage::LocalStorageAdapter; + use crate::storage::ObjectStorageAdapter; use crate::table_store::TableStore; use arrow_array::{Int32Array, RecordBatch, StringArray}; use arrow_schema::{DataType, Field, Schema}; @@ -1573,7 +1849,7 @@ mod tests { #[tokio::test] async fn list_sidecars_returns_empty_when_dir_missing() { let dir = tempfile::tempdir().unwrap(); - let storage = LocalStorageAdapter::default(); + let storage = ObjectStorageAdapter::local(); let result = list_sidecars(dir.path().to_str().unwrap(), &storage) .await .unwrap(); @@ -1583,10 +1859,10 @@ mod tests { #[tokio::test] async fn write_then_list_then_delete_round_trip() { let dir = tempfile::tempdir().unwrap(); - // Create the __recovery/ subdir so write_sidecar's parent exists - // (LocalStorageAdapter::write_text doesn't mkdir parents). - std::fs::create_dir(dir.path().join(RECOVERY_DIR_NAME)).unwrap(); - let storage = LocalStorageAdapter::default(); + // No pre-created __recovery/ subdir: the storage backend creates + // missing parents on put, which is what the first sidecar write + // of a fresh graph relies on. + let storage = ObjectStorageAdapter::local(); let root = dir.path().to_str().unwrap(); let sidecar = new_sidecar( @@ -1617,7 +1893,7 @@ mod tests { "noise", ) .unwrap(); - let storage = LocalStorageAdapter::default(); + let storage = ObjectStorageAdapter::local(); let result = list_sidecars(dir.path().to_str().unwrap(), &storage) .await .unwrap(); @@ -1633,7 +1909,7 @@ mod tests { async fn list_sidecars_returns_deterministic_order() { let dir = tempfile::tempdir().unwrap(); std::fs::create_dir(dir.path().join(RECOVERY_DIR_NAME)).unwrap(); - let storage = LocalStorageAdapter::default(); + let storage = ObjectStorageAdapter::local(); let root = dir.path().to_str().unwrap(); // Write sidecars in REVERSE chronological order (newest first). diff --git a/crates/omnigraph/src/db/omnigraph.rs b/crates/omnigraph/src/db/omnigraph.rs index 50f5d34..779a2e0 100644 --- a/crates/omnigraph/src/db/omnigraph.rs +++ b/crates/omnigraph/src/db/omnigraph.rs @@ -378,10 +378,11 @@ impl Omnigraph { recover_schema_state_files(&root, Arc::clone(&storage), &coordinator.snapshot()) .await?; // Recovery sweep: close the Phase B → Phase C residual on - // any sidecar left over from a crashed writer. Continuous - // in-process recovery for long-running servers (no restart - // required between Phase B failure and recovery) is a - // separate background-reconciler effort. + // any sidecar left over from a crashed writer. Long-running + // processes additionally converge in-process: the staged- + // write entry points and `refresh` run the roll-forward-only + // heal (`heal_pending_sidecars_roll_forward`); only + // rollback-eligible sidecars wait for this open-time sweep. crate::db::manifest::recover_manifest_drift( &root, Arc::clone(&storage), @@ -755,7 +756,7 @@ impl Omnigraph { /// /// Composition mirrors `Omnigraph::open_with_storage_and_mode`'s /// recovery sequence, in the same order, with one restriction: the - /// manifest-drift sweep runs in `RollForwardOnly` mode (rollback / + /// manifest-drift heal runs in `RollForwardOnly` mode (rollback / /// abort cases defer to the next ReadWrite open because /// `Dataset::restore` is unsafe under concurrency). Each step: /// @@ -767,49 +768,123 @@ impl Omnigraph { /// SchemaApply roll-forward doesn't publish the manifest while /// the staging files remain unrenamed (which would corrupt the /// graph: data on new schema, catalog on old). - /// 3. `recover_manifest_drift(... RollForwardOnly)` — close the + /// 3. `heal_pending_sidecars_roll_forward` — close the /// finalize→publisher residual via roll-forward; defer rollback - /// work to next ReadWrite open. + /// work to next ReadWrite open. Serializes against live writers + /// by acquiring each sidecar's per-(table_key, branch) write + /// queues, so refresh never rolls forward an in-flight writer's + /// sidecar from under it. /// 4. `runtime_cache.invalidate_all` — drop stale per-snapshot caches. /// /// Steady state cost: one `list_dir` of `__recovery/` (typically /// returns empty → early return for both passes). No additional /// Lance reads. /// - /// Engine-internal callers that already hold an in-flight sidecar - /// (e.g. `schema_apply` mid-write) MUST use + /// The staged-write entry points (`load_as`, `mutate_as`) run the + /// same heal via + /// [`heal_pending_recovery_sidecars`](Self::heal_pending_recovery_sidecars), + /// so a long-lived server converges on the next write without an + /// explicit refresh. Engine-internal callers that already hold an + /// in-flight sidecar (e.g. `schema_apply` mid-write) MUST use /// [`refresh_coordinator_only`](Self::refresh_coordinator_only) to /// avoid the recovery sweep racing their own sidecar. pub async fn refresh(&self) -> Result<()> { - // Scope the coord write guard to the recovery section only. + // Standalone schema-staging reconcile ONLY when no recovery + // sidecar exists (legacy/manual staging residue). When sidecars + // exist, the heal below owns the reconcile — per SchemaApply + // sidecar, under that sidecar's queue guards — because an + // unserialized reconcile can promote a LIVE schema apply's + // staging files from under it, and a pre-promoted result would + // make the heal's own guarded reconcile see clean staging and + // wrongly defer the sidecar. The no-sidecar case cannot race a + // live apply: its sidecar is on disk before its staging files. + // + // Scope the coord write guard to the schema-state section only. // `reload_schema_if_source_changed` (below) acquires // `self.coordinator.read().await` when the on-disk schema source // has drifted from the cached `schema_source`. Tokio's RwLock is // not reentrant, so holding the write across that call deadlocks. // Pinned by `composite_flow_schema_apply_then_branch_ops_no_deadlock_in_refresh`. + // The heal also takes the lock itself (queues → coordinator + // order), so it must run after this guard is released. { - let mut coord = self.coordinator.write().await; - coord.refresh().await?; - let schema_state_recovery = recover_schema_state_files( - &self.root_uri, - Arc::clone(&self.storage), - &coord.snapshot(), - ) - .await?; - crate::db::manifest::recover_manifest_drift( - &self.root_uri, - Arc::clone(&self.storage), - &mut *coord, - crate::db::manifest::RecoveryMode::RollForwardOnly, - schema_state_recovery, - ) - .await?; - } // ← write guard released before reload's read acquisition + // Hold the schema-apply serialization key across the + // list-then-reconcile pair: without it, a live apply can + // write its sidecar + staging between the empty check and + // the reconcile (the same race, through a smaller window). + // Queue before coordinator — the documented lock order. + // + // Liveness note: with a pending NON-SchemaApply sidecar + // (e.g. a Mutation residual), this gate skips the standalone + // reconcile and the heal below reconciles only per + // SchemaApply sidecar — so pre-sidecar-era orphaned staging + // residue waits for the NEXT refresh after the sidecars are + // consumed. Convergence holds, one pass late. Do not "fix" + // by re-running the reconcile unserialized here: that is + // exactly the live-apply race this block exists to close. + let _serial = self + .write_queue + .acquire(&crate::db::manifest::schema_apply_serial_queue_key()) + .await; + if crate::db::manifest::list_sidecars(&self.root_uri, self.storage.as_ref()) + .await? + .is_empty() + { + let mut coord = self.coordinator.write().await; + coord.refresh().await?; + recover_schema_state_files( + &self.root_uri, + Arc::clone(&self.storage), + &coord.snapshot(), + ) + .await?; + } + } // ← guards released before the heal's queue acquisition + crate::db::manifest::heal_pending_sidecars_roll_forward( + &self.root_uri, + Arc::clone(&self.storage), + &self.coordinator, + &self.write_queue, + ) + .await?; self.reload_schema_if_source_changed().await?; self.runtime_cache.invalidate_all().await; Ok(()) } + /// Write-entry heal: converge any pending recovery sidecars (a + /// previously failed writer's Phase B → Phase C residual) before + /// starting a new staged write, so a long-lived process (the HTTP + /// server, an embedded handle) recovers on its next write instead + /// of wedging every write on the commit-time drift guard until + /// restart. Roll-forward only; rollback-eligible sidecars defer to + /// the next ReadWrite open exactly as [`refresh`](Self::refresh) + /// does. + /// + /// Steady-state cost: one `list_dir` of `__recovery/` (typically + /// empty → immediate return). See + /// `recovery::heal_pending_sidecars_roll_forward` for the + /// concurrency contract (per-table write-queue acquisition). + pub(crate) async fn heal_pending_recovery_sidecars(&self) -> Result<()> { + let processed = crate::db::manifest::heal_pending_sidecars_roll_forward( + &self.root_uri, + Arc::clone(&self.storage), + &self.coordinator, + &self.write_queue, + ) + .await?; + if processed { + // A rolled-forward SchemaApply sidecar moved disk + manifest + // to the new schema (staging promoted, registrations + // published); the in-memory catalog must follow or the very + // write that triggered the heal validates against the stale + // schema. Same post-heal step as `refresh`. + self.reload_schema_if_source_changed().await?; + self.runtime_cache.invalidate_all().await; + } + Ok(()) + } + async fn reload_schema_if_source_changed(&self) -> Result<()> { let schema_path = schema_source_uri(&self.root_uri); let schema_source = self.storage.read_text(&schema_path).await?; @@ -1951,7 +2026,7 @@ mod tests { use serde_json::Value; use std::sync::{Arc, Mutex}; - use crate::storage::{LocalStorageAdapter, StorageAdapter, join_uri}; + use crate::storage::{ObjectStorageAdapter, StorageAdapter, join_uri}; const TEST_SCHEMA: &str = r#" node Person { @@ -1967,9 +2042,9 @@ edge Knows: Person -> Person { edge WorksAt: Person -> Company "#; - #[derive(Debug, Default)] + #[derive(Debug)] struct RecordingStorageAdapter { - inner: LocalStorageAdapter, + inner: ObjectStorageAdapter, reads: Mutex<Vec<String>>, writes: Mutex<Vec<String>>, exists_checks: Mutex<Vec<String>>, @@ -1977,6 +2052,19 @@ edge WorksAt: Person -> Company deletes: Mutex<Vec<String>>, } + impl Default for RecordingStorageAdapter { + fn default() -> Self { + Self { + inner: ObjectStorageAdapter::local(), + reads: Mutex::default(), + writes: Mutex::default(), + exists_checks: Mutex::default(), + renames: Mutex::default(), + deletes: Mutex::default(), + } + } + } + impl RecordingStorageAdapter { fn reads(&self) -> Vec<String> { self.reads.lock().unwrap().clone() @@ -2052,7 +2140,7 @@ edge WorksAt: Person -> Company #[derive(Debug)] struct InitRaceStorageAdapter { - inner: LocalStorageAdapter, + inner: ObjectStorageAdapter, root: String, barrier: Arc<tokio::sync::Barrier>, } @@ -2117,7 +2205,7 @@ edge WorksAt: Person -> Company let uri = dir.path().to_str().unwrap().to_string(); let root = normalize_root_uri(&uri).unwrap(); let storage: Arc<dyn StorageAdapter> = Arc::new(InitRaceStorageAdapter { - inner: LocalStorageAdapter, + inner: ObjectStorageAdapter::local(), root, barrier: Arc::new(tokio::sync::Barrier::new(2)), }); diff --git a/crates/omnigraph/src/db/omnigraph/schema_apply.rs b/crates/omnigraph/src/db/omnigraph/schema_apply.rs index 506db36..f965ad4 100644 --- a/crates/omnigraph/src/db/omnigraph/schema_apply.rs +++ b/crates/omnigraph/src/db/omnigraph/schema_apply.rs @@ -144,6 +144,14 @@ where actor, )?; + // Converge any pending recovery sidecar before planning: a table + // rewrite over sidecar-covered drift would otherwise re-plan from + // the manifest pin and orphan the drifted Phase-B commit (silently + // dropping its rows) while the stale sidecar lingers to misclassify + // against the post-apply pins. Runs before the apply's own sidecar + // exists, so the heal can never observe it. + db.heal_pending_recovery_sidecars().await?; + acquire_schema_apply_lock(db).await?; let result = apply_schema_with_lock(db, desired_schema_source, options, validate_catalog).await; let release_result = release_schema_apply_lock(db).await; @@ -428,19 +436,30 @@ where // per-table acquisitions are uncontended. They exist for symmetry // with future MR-870 recovery, which will need queue acquisition // before any `Dataset::restore` it issues for SchemaApply sidecars. - let schema_apply_queue_keys: Vec<(String, Option<String>)> = recovery_pins + let mut schema_apply_queue_keys: Vec<(String, Option<String>)> = recovery_pins .iter() .map(|pin| (pin.table_key.clone(), pin.table_branch.clone())) .collect(); + // The serialization key the write-entry heal acquires before touching + // schema staging or a SchemaApply sidecar. Per-table keys alone don't + // cover a registration-only migration (no pins, but a sidecar and + // staging files on disk) — without this, a concurrent write's heal can + // promote this apply's staging files and publish its registrations out + // from under it. Acquired whenever a sidecar will be written, held + // through Phase D (the guards live to the end of this function). + let writes_sidecar = !(recovery_pins.is_empty() + && sidecar_registrations.is_empty() + && sidecar_tombstones.is_empty()); + if writes_sidecar { + schema_apply_queue_keys + .push(crate::db::manifest::schema_apply_serial_queue_key()); + } let _schema_apply_queue_guards = db .write_queue() .acquire_many(&schema_apply_queue_keys) .await; - let recovery_handle = if recovery_pins.is_empty() - && sidecar_registrations.is_empty() - && sidecar_tombstones.is_empty() - { + let recovery_handle = if !writes_sidecar { None } else { // `branch=None` because schema_apply publishes against main — diff --git a/crates/omnigraph/src/db/recovery_audit.rs b/crates/omnigraph/src/db/recovery_audit.rs index b9e8e7b..2aab6bc 100644 --- a/crates/omnigraph/src/db/recovery_audit.rs +++ b/crates/omnigraph/src/db/recovery_audit.rs @@ -43,6 +43,11 @@ const RECOVERIES_DIR: &str = "_graph_commit_recoveries.lance"; pub(crate) enum RecoveryKind { RolledForward, RolledBack, + /// The sidecar's branch no longer exists in the manifest: its tree + /// and forks are reclaimed, the pinned drift is unreachable, and the + /// sidecar is provably moot — discarded with this audit row instead + /// of wedging every heal/sweep on a dead-branch open. + OrphanedBranchDiscarded, } impl RecoveryKind { @@ -50,6 +55,7 @@ impl RecoveryKind { match self { RecoveryKind::RolledForward => "RolledForward", RecoveryKind::RolledBack => "RolledBack", + RecoveryKind::OrphanedBranchDiscarded => "OrphanedBranchDiscarded", } } @@ -57,6 +63,7 @@ impl RecoveryKind { match s { "RolledForward" => Ok(RecoveryKind::RolledForward), "RolledBack" => Ok(RecoveryKind::RolledBack), + "OrphanedBranchDiscarded" => Ok(RecoveryKind::OrphanedBranchDiscarded), other => Err(OmniError::manifest_internal(format!( "unknown recovery_kind '{}' in _graph_commit_recoveries.lance", other diff --git a/crates/omnigraph/src/exec/merge.rs b/crates/omnigraph/src/exec/merge.rs index f245d15..ea16b15 100644 --- a/crates/omnigraph/src/exec/merge.rs +++ b/crates/omnigraph/src/exec/merge.rs @@ -1081,6 +1081,13 @@ impl Omnigraph { actor_id, )?; self.ensure_schema_apply_idle("branch_merge").await?; + // Converge any pending recovery sidecar before the merge + // captures its target snapshot: the merge's publish would + // otherwise make the drifted Phase-B commit visible as an + // unattributed side effect (manifest catches up to HEAD with no + // recovery audit row) and leave the stale sidecar behind. Runs + // before the merge's own sidecar exists. + self.heal_pending_recovery_sidecars().await?; self.branch_merge_impl(source, target, actor_id).await } diff --git a/crates/omnigraph/src/exec/mutation.rs b/crates/omnigraph/src/exec/mutation.rs index e537d0d..e9051c4 100644 --- a/crates/omnigraph/src/exec/mutation.rs +++ b/crates/omnigraph/src/exec/mutation.rs @@ -715,6 +715,14 @@ impl Omnigraph { actor_id: Option<&str>, ) -> Result<MutationResult> { self.ensure_schema_state_valid().await?; + // Converge any pending recovery sidecar (a previously failed + // writer's Phase B → Phase C residual) before executing: the + // inline delete path advances Lance HEAD during execution and + // the staged path's commit-time drift guard refuses + // sidecar-covered drift, so a long-lived handle must heal here + // — not at restart. One `list_dir` when no sidecars exist (the + // steady state). + self.heal_pending_recovery_sidecars().await?; let requested = Self::normalize_branch_name(branch)?; // Reject internal `__run__*` / system-prefixed branches at the // public write boundary. Direct-publish paths assert this diff --git a/crates/omnigraph/src/exec/staging.rs b/crates/omnigraph/src/exec/staging.rs index a3932b0..cbfd52d 100644 --- a/crates/omnigraph/src/exec/staging.rs +++ b/crates/omnigraph/src/exec/staging.rs @@ -599,9 +599,53 @@ impl StagedMutation { ))); } if head > current { + // Error path only: tell the operator which drift class + // this is. Uncovered drift (external raw Lance write, + // pre-fix maintenance) goes through `omnigraph repair`. + // Sidecar-covered drift reaching this guard means the + // write-entry heal deferred it (rollback-eligible), and + // `repair` refuses while a sidecar is pending — the + // recovery path is a read-write reopen. A list failure + // must not mask the conflict — and must not pick a + // class confidently either: "could not classify" names + // both paths and the cause, never routing the operator + // to a command that will refuse. + let action = match crate::db::manifest::list_sidecars( + db.root_uri(), + db.storage_adapter(), + ) + .await + { + Ok(sidecars) => { + let covered = sidecars.iter().any(|sidecar| { + sidecar.tables.iter().any(|pin| { + // Branch-aware: a sidecar pinning the + // same table on ANOTHER branch does not + // cover this branch's drift — a reopen + // would recover that sidecar but leave + // this drift for `repair`. + pin.table_key == entry.table_key + && pin.table_branch == entry.path.table_branch + }) + }); + if covered { + "a pending recovery sidecar requires rollback — reopen the \ + graph read-write (e.g. restart the server) to recover" + .to_string() + } else { + "run `omnigraph repair` before writing".to_string() + } + } + Err(list_err) => format!( + "could not classify the drift (sidecar listing failed: {}); \ + run `omnigraph repair`, or reopen the graph read-write if \ + repair reports a pending recovery sidecar", + list_err + ), + }; return Err(OmniError::manifest_conflict(format!( - "table '{}' has Lance HEAD version {} ahead of manifest version {}; run `omnigraph repair` before writing", - entry.table_key, head, current + "table '{}' has Lance HEAD version {} ahead of manifest version {}; {}", + entry.table_key, head, current, action ))); } diff --git a/crates/omnigraph/src/loader/mod.rs b/crates/omnigraph/src/loader/mod.rs index 09c2f7c..69ada79 100644 --- a/crates/omnigraph/src/loader/mod.rs +++ b/crates/omnigraph/src/loader/mod.rs @@ -188,6 +188,13 @@ impl Omnigraph { actor_id, )?; self.ensure_schema_state_valid().await?; + // Converge any pending recovery sidecar (a previously failed + // writer's Phase B → Phase C residual) before staging anything: + // without this, sidecar-covered drift wedges every load on the + // commit-time drift guard until a process restart — `repair` + // refuses while a sidecar is pending. One `list_dir` when no + // sidecars exist (the steady state). + self.heal_pending_recovery_sidecars().await?; // Reject internal `__run__*` / system-prefixed branches at the // public write boundary. Direct-publish paths assert this // explicitly so a caller can't write to legacy or system diff --git a/crates/omnigraph/src/storage.rs b/crates/omnigraph/src/storage.rs index 187a6d6..1f96b39 100644 --- a/crates/omnigraph/src/storage.rs +++ b/crates/omnigraph/src/storage.rs @@ -1,14 +1,15 @@ use std::env; use std::fmt::Debug; -use std::path::{Path, PathBuf}; +use std::path::{Component, Path, PathBuf}; use std::sync::Arc; use async_trait::async_trait; use futures::TryStreamExt; use object_store::aws::AmazonS3Builder; +use object_store::local::LocalFileSystem; +use object_store::memory::InMemory; use object_store::path::Path as ObjectPath; use object_store::{DynObjectStore, ObjectStore, PutMode, PutPayload}; -use tokio::io::AsyncWriteExt; use url::Url; use crate::error::{OmniError, Result}; @@ -38,20 +39,28 @@ pub trait StorageAdapter: Debug + Send + Sync { /// List all files (non-recursively, files only) directly under `dir_uri`. /// Returns full URIs (same scheme as `dir_uri`). The result is unordered. /// Returns Ok(empty) if the directory does not exist or is empty. + /// Consumers must tolerate non-payload residue appearing in storage + /// (backend staging files are filtered by the backend, but crash residue + /// of any future producer may not be) — filter by suffix, never assume + /// every entry is yours. async fn list_dir(&self, dir_uri: &str) -> Result<Vec<String>>; - /// Read a text object together with its backend version token (S3: the - /// object's ETag; local: sha256 of the content). The token is opaque — - /// valid only for `write_text_if_match` against the same adapter. + /// Read a text object together with its backend version token (stores + /// with conditional-update support: the object's ETag; local: sha256 of + /// the content). The token is opaque — valid only for + /// `write_text_if_match` against the same adapter. async fn read_text_versioned(&self, uri: &str) -> Result<(String, String)>; /// Replace the object at `uri` only if its current version still matches /// `expected_version` (obtained from a prior versioned read/write on this /// adapter). Returns `Ok(Some(new_version))` on success and `Ok(None)` /// when the precondition failed (a concurrent writer won — the CAS-lost - /// case callers must surface, never swallow). S3 uses a conditional put - /// (If-Match); local compares content then replaces via temp + rename — - /// the same single-machine semantics the callers had before this trait, - /// safe under the callers' own lock protocol but not a cross-process - /// barrier by itself. + /// case callers must surface, never swallow). Stores with conditional + /// updates (S3, in-memory) use a true conditional put (If-Match); the + /// local filesystem has no such primitive (`PutMode::Update` is + /// unimplemented upstream), so local compares content then replaces via + /// an atomic staged write — the same single-machine semantics the + /// callers had before this trait, safe under the callers' own lock + /// protocol but not a cross-process barrier by itself (see the Known + /// Gaps entry in docs/dev/invariants.md). async fn write_text_if_match( &self, uri: &str, @@ -59,14 +68,18 @@ pub trait StorageAdapter: Debug + Send + Sync { expected_version: &str, ) -> Result<Option<String>>; /// Recursively delete every object under `prefix_uri`. Returns Ok(()) - /// when nothing exists there (idempotent). Local: `remove_dir_all`; - /// S3: list + delete (NOT atomic — callers must tolerate partial - /// prefixes on crash, which the cluster delete protocol does by retry). + /// when nothing exists there (idempotent). Local: `remove_dir_all` + /// (directories are a local-FS concept; list+delete would leave empty + /// directory skeletons that local existence probes report as present); + /// object stores: list + delete (NOT atomic — callers must tolerate + /// partial prefixes on crash, which the cluster delete protocol does by + /// retry). async fn delete_prefix(&self, prefix_uri: &str) -> Result<()>; } -/// Version token for local files: content identity. ETags are unavailable -/// on the filesystem; sha256 is stable, cheap at these object sizes, and +/// Version token for local files: content identity. The local filesystem +/// backend reports mtime-derived ETags too coarse for CAS (sub-granularity +/// rewrites collide); sha256 is stable, cheap at these object sizes, and /// already the cluster ledger's CAS vocabulary. fn local_version_token(bytes: &[u8]) -> String { use sha2::{Digest, Sha256}; @@ -80,13 +93,34 @@ pub enum StorageKind { S3, } -#[derive(Debug, Default)] -pub struct LocalStorageAdapter; - +/// The one storage implementation: every backend is an +/// [`object_store::ObjectStore`], so the semantics (atomic-visibility puts, +/// conditional creates, path-delimited listing) are upstream-maintained and +/// identical across backends by construction. The per-backend residue is +/// confined to [`UriCodec`] (URI ↔ object path mapping) and the +/// `supports_conditional_update` capability flag (false only for the local +/// filesystem, where upstream `PutMode::Update` is unimplemented). #[derive(Debug)] -pub struct S3StorageAdapter { - bucket: String, +pub struct ObjectStorageAdapter { store: Arc<DynObjectStore>, + codec: UriCodec, + /// Whether the backend implements `PutMode::Update` (ETag-conditioned + /// put). Gates BOTH the version-token source in `read_text_versioned` + /// and the `write_text_if_match` strategy — the two must agree or every + /// CAS loses. + supports_conditional_update: bool, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +enum UriCodec { + /// Plain absolute/relative paths or `file://` URIs, mapped onto a + /// root-anchored [`LocalFileSystem`]. + Local, + /// `s3://{bucket}/{key}` URIs, mapped onto a bucket-scoped store. + S3 { bucket: String }, + /// Opaque keys for the in-memory test/embedded backend; leading + /// slashes are stripped. + Memory, } #[derive(Debug, Clone, PartialEq, Eq)] @@ -95,357 +129,22 @@ struct S3Location { key: String, } -#[async_trait] -impl StorageAdapter for LocalStorageAdapter { - async fn read_text(&self, uri: &str) -> Result<String> { - let path = local_path_from_uri(uri)?; - Ok(tokio::fs::read_to_string(&path).await?) - } - - async fn write_text(&self, uri: &str, contents: &str) -> Result<()> { - let path = local_path_from_uri(uri)?; - // Ensure parent directory exists. S3 has no equivalent (PutObject - // is path-agnostic). For local fs, callers like the recovery - // sidecar protocol expect transparent directory creation under - // the graph root (the `__recovery/` directory doesn't pre-exist; - // first sidecar write creates it). - if let Some(parent) = path.parent() { - if !parent.as_os_str().is_empty() { - tokio::fs::create_dir_all(parent).await?; - } - } - tokio::fs::write(&path, contents).await?; - Ok(()) - } - - async fn write_text_if_absent(&self, uri: &str, contents: &str) -> Result<bool> { - let path = local_path_from_uri(uri)?; - if let Some(parent) = path.parent() { - if !parent.as_os_str().is_empty() { - tokio::fs::create_dir_all(parent).await?; - } - } - let mut file = match tokio::fs::OpenOptions::new() - .write(true) - .create_new(true) - .open(&path) - .await - { - Ok(file) => file, - Err(err) if err.kind() == std::io::ErrorKind::AlreadyExists => return Ok(false), - Err(err) => return Err(err.into()), - }; - if let Err(err) = file.write_all(contents.as_bytes()).await { - let _ = tokio::fs::remove_file(&path).await; - return Err(err.into()); - } - // tokio's async File buffers internally: without an explicit flush, - // write_all only fills the buffer and the actual OS write happens in - // a background task AFTER this fn returns — a reader can then see - // the created-but-still-empty file (caught twice in CI as an - // "EOF while parsing" on a state.json read right after import). - // Flushing before Ok restores write-then-read consistency, matching - // tokio::fs::write (which flushes internally) used by every other - // write path here. - if let Err(err) = file.flush().await { - let _ = tokio::fs::remove_file(&path).await; - return Err(err.into()); - } - Ok(true) - } - - async fn exists(&self, uri: &str) -> Result<bool> { - Ok(local_path_from_uri(uri)?.exists()) - } - - async fn rename_text(&self, from_uri: &str, to_uri: &str) -> Result<()> { - let from = local_path_from_uri(from_uri)?; - let to = local_path_from_uri(to_uri)?; - tokio::fs::rename(&from, &to).await?; - Ok(()) - } - - async fn delete(&self, uri: &str) -> Result<()> { - let path = local_path_from_uri(uri)?; - match tokio::fs::remove_file(&path).await { - Ok(()) => Ok(()), - Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()), - Err(err) => Err(err.into()), +impl ObjectStorageAdapter { + /// Local-filesystem backend rooted at `/`. URIs are plain paths or + /// `file://` URIs; relative paths are lexically absolutized against the + /// current working directory. + pub fn local() -> Self { + Self { + store: Arc::new(LocalFileSystem::new()), + codec: UriCodec::Local, + supports_conditional_update: false, } } - async fn list_dir(&self, dir_uri: &str) -> Result<Vec<String>> { - let path = local_path_from_uri(dir_uri)?; - let mut out = Vec::new(); - let mut entries = match tokio::fs::read_dir(&path).await { - Ok(e) => e, - Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(out), - Err(err) => return Err(err.into()), - }; - let dir_str = dir_uri.trim_end_matches('/'); - while let Some(entry) = entries.next_entry().await? { - let ft = entry.file_type().await?; - if !ft.is_file() { - continue; - } - if let Some(name) = entry.file_name().to_str() { - out.push(format!("{}/{}", dir_str, name)); - } - } - Ok(out) - } - - async fn read_text_versioned(&self, uri: &str) -> Result<(String, String)> { - let path = local_path_from_uri(uri)?; - let bytes = tokio::fs::read(&path).await?; - let version = local_version_token(&bytes); - let text = String::from_utf8(bytes).map_err(|err| { - OmniError::manifest_internal(format!("storage read failed for '{}': {}", uri, err)) - })?; - Ok((text, version)) - } - - async fn write_text_if_match( - &self, - uri: &str, - contents: &str, - expected_version: &str, - ) -> Result<Option<String>> { - let path = local_path_from_uri(uri)?; - let current = match tokio::fs::read(&path).await { - Ok(bytes) => bytes, - Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(None), - Err(err) => return Err(err.into()), - }; - if local_version_token(¤t) != expected_version { - return Ok(None); - } - let tmp = path.with_extension(format!("tmp.{}", ulid::Ulid::new())); - tokio::fs::write(&tmp, contents.as_bytes()).await?; - if let Err(err) = tokio::fs::rename(&tmp, &path).await { - let _ = tokio::fs::remove_file(&tmp).await; - return Err(err.into()); - } - Ok(Some(local_version_token(contents.as_bytes()))) - } - - async fn delete_prefix(&self, prefix_uri: &str) -> Result<()> { - let path = local_path_from_uri(prefix_uri)?; - match tokio::fs::remove_dir_all(&path).await { - Ok(()) => Ok(()), - Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()), - Err(err) => Err(err.into()), - } - } -} - -#[async_trait] -impl StorageAdapter for S3StorageAdapter { - async fn read_text(&self, uri: &str) -> Result<String> { - let location = self.object_path(uri)?; - let bytes = self - .store - .get(&location) - .await - .map_err(|err| storage_backend_error("read", uri, err))? - .bytes() - .await - .map_err(|err| storage_backend_error("read", uri, err))?; - - String::from_utf8(bytes.to_vec()).map_err(|err| { - OmniError::manifest_internal(format!("storage read failed for '{}': {}", uri, err)) - }) - } - - async fn write_text(&self, uri: &str, contents: &str) -> Result<()> { - let location = self.object_path(uri)?; - self.store - .put(&location, PutPayload::from(contents.as_bytes().to_vec())) - .await - .map_err(|err| storage_backend_error("write", uri, err))?; - Ok(()) - } - - async fn write_text_if_absent(&self, uri: &str, contents: &str) -> Result<bool> { - let location = self.object_path(uri)?; - match self - .store - .put_opts( - &location, - PutPayload::from(contents.as_bytes().to_vec()), - PutMode::Create.into(), - ) - .await - { - Ok(_) => Ok(true), - Err(object_store::Error::AlreadyExists { .. }) - | Err(object_store::Error::Precondition { .. }) => Ok(false), - Err(err) => Err(storage_backend_error("write_if_absent", uri, err)), - } - } - - async fn exists(&self, uri: &str) -> Result<bool> { - let location = self.object_path(uri)?; - match self.store.head(&location).await { - Ok(_) => Ok(true), - Err(object_store::Error::NotFound { .. }) => { - let mut entries = self.store.list(Some(&location)); - let has_prefix_entries = entries - .try_next() - .await - .map_err(|err| storage_backend_error("exists", uri, err))? - .is_some(); - Ok(has_prefix_entries) - } - Err(err) => Err(storage_backend_error("exists", uri, err)), - } - } - - async fn rename_text(&self, from_uri: &str, to_uri: &str) -> Result<()> { - // S3 has no atomic rename. Copy then delete; if the copy succeeds and - // the delete fails (or the process crashes between them), both - // source and destination exist with the same content. Recovery code - // must tolerate this case — see schema_state::recover_schema_state_files. - let from = self.object_path(from_uri)?; - let to = self.object_path(to_uri)?; - self.store - .copy(&from, &to) - .await - .map_err(|err| storage_backend_error("rename:copy", from_uri, err))?; - self.store - .delete(&from) - .await - .map_err(|err| storage_backend_error("rename:delete", from_uri, err))?; - Ok(()) - } - - async fn delete(&self, uri: &str) -> Result<()> { - let location = self.object_path(uri)?; - match self.store.delete(&location).await { - Ok(()) => Ok(()), - Err(object_store::Error::NotFound { .. }) => Ok(()), - Err(err) => Err(storage_backend_error("delete", uri, err)), - } - } - - async fn list_dir(&self, dir_uri: &str) -> Result<Vec<String>> { - // Normalize: ensure the URI describes a directory (trailing '/') so - // we don't match sibling paths with a shared prefix - // (e.g. listing `__recovery` shouldn't match `__recovery_log/...`). - let dir_with_slash = if dir_uri.ends_with('/') { - dir_uri.to_string() - } else { - format!("{}/", dir_uri) - }; - // object_store::Path strips the trailing '/'; re-add it for filtering. - let prefix_loc = self.object_path(&dir_with_slash)?; - let prefix_with_slash = format!("{}/", prefix_loc.as_ref()); - - let mut entries = self.store.list(Some(&prefix_loc)); - let mut out = Vec::new(); - let bucket_root = format!("{}{}/", S3_SCHEME_PREFIX, self.bucket); - while let Some(meta) = entries - .try_next() - .await - .map_err(|err| storage_backend_error("list_dir", dir_uri, err))? - { - let key_str = meta.location.as_ref(); - // Require the directory boundary to filter out sibling-prefix - // matches (object_store's `list` is prefix-based, not dir-based). - if !key_str.starts_with(&prefix_with_slash) { - continue; - } - let suffix = &key_str[prefix_with_slash.len()..]; - // Non-recursive: skip anything inside a sub-directory. - if suffix.contains('/') { - continue; - } - out.push(format!("{}{}", bucket_root, key_str)); - } - Ok(out) - } - - async fn read_text_versioned(&self, uri: &str) -> Result<(String, String)> { - let location = self.object_path(uri)?; - let result = self - .store - .get(&location) - .await - .map_err(|err| storage_backend_error("read", uri, err))?; - let etag = result.meta.e_tag.clone(); - let bytes = result - .bytes() - .await - .map_err(|err| storage_backend_error("read", uri, err))?; - // Every S3-compatible store we target returns ETags; fall back to a - // content token rather than failing if one ever omits it. - let version = etag.unwrap_or_else(|| local_version_token(&bytes)); - let text = String::from_utf8(bytes.to_vec()).map_err(|err| { - OmniError::manifest_internal(format!("storage read failed for '{}': {}", uri, err)) - })?; - Ok((text, version)) - } - - async fn write_text_if_match( - &self, - uri: &str, - contents: &str, - expected_version: &str, - ) -> Result<Option<String>> { - let location = self.object_path(uri)?; - let mode = PutMode::Update(object_store::UpdateVersion { - e_tag: Some(expected_version.to_string()), - version: None, - }); - match self - .store - .put_opts( - &location, - PutPayload::from(contents.as_bytes().to_vec()), - mode.into(), - ) - .await - { - Ok(result) => Ok(Some( - result - .e_tag - .unwrap_or_else(|| local_version_token(contents.as_bytes())), - )), - Err(object_store::Error::Precondition { .. }) - | Err(object_store::Error::NotFound { .. }) => Ok(None), - Err(err) => Err(storage_backend_error("write_if_match", uri, err)), - } - } - - async fn delete_prefix(&self, prefix_uri: &str) -> Result<()> { - let dir_with_slash = if prefix_uri.ends_with('/') { - prefix_uri.to_string() - } else { - format!("{}/", prefix_uri) - }; - let prefix_loc = self.object_path(&dir_with_slash)?; - let mut entries = self.store.list(Some(&prefix_loc)); - let mut locations = Vec::new(); - while let Some(meta) = entries - .try_next() - .await - .map_err(|err| storage_backend_error("delete_prefix", prefix_uri, err))? - { - locations.push(meta.location); - } - for location in locations { - match self.store.delete(&location).await { - Ok(()) => {} - Err(object_store::Error::NotFound { .. }) => {} - Err(err) => return Err(storage_backend_error("delete_prefix", prefix_uri, err)), - } - } - Ok(()) - } -} - -impl S3StorageAdapter { - fn from_root_uri(root_uri: &str) -> Result<Self> { + /// S3 backend scoped to the bucket named in `root_uri`. Credentials and + /// endpoint come from the standard `AWS_*` environment variables (the + /// same ones Lance reads for its dataset stores). + pub fn s3_from_root_uri(root_uri: &str) -> Result<Self> { let location = parse_s3_uri(root_uri)?; let mut builder = AmazonS3Builder::from_env().with_bucket_name(&location.bucket); @@ -471,29 +170,311 @@ impl S3StorageAdapter { })?; Ok(Self { - bucket: location.bucket, store: Arc::new(store), + codec: UriCodec::S3 { + bucket: location.bucket, + }, + supports_conditional_update: true, }) } + /// In-memory backend for tests and embedded experiments. Implements the + /// FULL contract including true conditional updates (unlike the local + /// filesystem), so contract tests exercise the strong-CAS path without a + /// bucket. State lives only as long as the adapter. + pub fn in_memory() -> Self { + Self { + store: Arc::new(InMemory::new()), + codec: UriCodec::Memory, + supports_conditional_update: true, + } + } + fn object_path(&self, uri: &str) -> Result<ObjectPath> { - let location = parse_s3_uri(uri)?; - if location.bucket != self.bucket { - return Err(OmniError::manifest_internal(format!( - "s3 storage bucket mismatch for '{}': expected '{}', found '{}'", - uri, self.bucket, location.bucket - ))); + match &self.codec { + UriCodec::Local => { + let path = absolutize_lexically(local_path_from_uri(uri)?)?; + ObjectPath::from_absolute_path(&path).map_err(|err| { + OmniError::manifest_internal(format!( + "invalid local object path for '{}': {}", + uri, err + )) + }) + } + UriCodec::S3 { bucket } => { + let location = parse_s3_uri(uri)?; + if &location.bucket != bucket { + return Err(OmniError::manifest_internal(format!( + "s3 storage bucket mismatch for '{}': expected '{}', found '{}'", + uri, bucket, location.bucket + ))); + } + if location.key.is_empty() { + return Err(OmniError::manifest_internal(format!( + "s3 storage path is empty for '{}'", + uri + ))); + } + ObjectPath::parse(&location.key).map_err(|err| { + OmniError::manifest_internal(format!( + "invalid s3 object path for '{}': {}", + uri, err + )) + }) + } + UriCodec::Memory => { + ObjectPath::parse(uri.trim_start_matches('/')).map_err(|err| { + OmniError::manifest_internal(format!( + "invalid memory object path for '{}': {}", + uri, err + )) + }) + } } - if location.key.is_empty() { - return Err(OmniError::manifest_internal(format!( - "s3 storage path is empty for '{}'", - uri - ))); - } - ObjectPath::parse(&location.key).map_err(|err| { - OmniError::manifest_internal(format!("invalid s3 object path for '{}': {}", uri, err)) + } +} + +#[async_trait] +impl StorageAdapter for ObjectStorageAdapter { + async fn read_text(&self, uri: &str) -> Result<String> { + let location = self.object_path(uri)?; + let bytes = self + .store + .get(&location) + .await + .map_err(|err| storage_backend_error("read", uri, err))? + .bytes() + .await + .map_err(|err| storage_backend_error("read", uri, err))?; + + String::from_utf8(bytes.to_vec()).map_err(|err| { + OmniError::manifest_internal(format!("storage read failed for '{}': {}", uri, err)) }) } + + async fn write_text(&self, uri: &str, contents: &str) -> Result<()> { + // Atomic visibility is the backend's contract: object stores via + // PutObject; LocalFileSystem via an internal staged-temp + rename + // (a reader sees the old object or the new one, never a truncated + // in-progress write). Callers (sidecar protocol, cluster state) + // assume it. + let location = self.object_path(uri)?; + self.store + .put(&location, PutPayload::from(contents.as_bytes().to_vec())) + .await + .map_err(|err| storage_backend_error("write", uri, err))?; + Ok(()) + } + + async fn write_text_if_absent(&self, uri: &str, contents: &str) -> Result<bool> { + // PutMode::Create: atomic no-replace publish on every backend — + // exactly one of N concurrent claimants wins, and the winner's + // object is fully readable at the instant it becomes visible + // (LocalFileSystem stages the temp file completely, then + // hard_links it; pinned by + // `local_write_text_if_absent_is_read_visible_on_return`). + let location = self.object_path(uri)?; + match self + .store + .put_opts( + &location, + PutPayload::from(contents.as_bytes().to_vec()), + PutMode::Create.into(), + ) + .await + { + Ok(_) => Ok(true), + Err(object_store::Error::AlreadyExists { .. }) + | Err(object_store::Error::Precondition { .. }) => Ok(false), + Err(err) => Err(storage_backend_error("write_if_absent", uri, err)), + } + } + + async fn exists(&self, uri: &str) -> Result<bool> { + // head() answers for objects; the list fallback answers for + // "directory-shaped" URIs (e.g. a Lance dataset root, whose + // `_versions/*.manifest` makes any committed dataset non-empty). + // Object-store semantics throughout: only objects exist — + // an EMPTY local directory does not (callers that probe local + // directories use std::fs directly). + let location = self.object_path(uri)?; + match self.store.head(&location).await { + Ok(_) => Ok(true), + Err(object_store::Error::NotFound { .. }) => { + let mut entries = self.store.list(Some(&location)); + let has_prefix_entries = entries + .try_next() + .await + .map_err(|err| storage_backend_error("exists", uri, err))? + .is_some(); + Ok(has_prefix_entries) + } + Err(err) => Err(storage_backend_error("exists", uri, err)), + } + } + + async fn rename_text(&self, from_uri: &str, to_uri: &str) -> Result<()> { + // ObjectStore::rename: LocalFileSystem overrides it with an atomic + // fs::rename (creating missing destination parents); object stores + // use the default copy + delete — if the copy succeeds and the + // delete fails (or the process crashes between them), both source + // and destination exist with the same content. Recovery code must + // tolerate this case — see schema_state::recover_schema_state_files. + let from = self.object_path(from_uri)?; + let to = self.object_path(to_uri)?; + self.store + .rename(&from, &to) + .await + .map_err(|err| storage_backend_error("rename", from_uri, err))?; + Ok(()) + } + + async fn delete(&self, uri: &str) -> Result<()> { + let location = self.object_path(uri)?; + match self.store.delete(&location).await { + Ok(()) => Ok(()), + Err(object_store::Error::NotFound { .. }) => Ok(()), + Err(err) => Err(storage_backend_error("delete", uri, err)), + } + } + + async fn list_dir(&self, dir_uri: &str) -> Result<Vec<String>> { + // list_with_delimiter is non-recursive and path-delimited on every + // backend (no sibling-prefix bleed: listing `__recovery` cannot + // match `__recovery_log/...`), and returns Ok(empty) for a missing + // directory. Output URIs are anchored on the INPUT `dir_uri` plus + // the entry filename, so the strings round-trip byte-identically + // into read_text/delete regardless of scheme (plain path, file://, + // s3://). + let anchor = dir_uri.trim_end_matches('/'); + let prefix = self.object_path(anchor)?; + let listing = self + .store + .list_with_delimiter(Some(&prefix)) + .await + .map_err(|err| storage_backend_error("list_dir", dir_uri, err))?; + let mut out = Vec::with_capacity(listing.objects.len()); + for meta in listing.objects { + if let Some(name) = meta.location.filename() { + out.push(format!("{}/{}", anchor, name)); + } + } + Ok(out) + } + + async fn read_text_versioned(&self, uri: &str) -> Result<(String, String)> { + let location = self.object_path(uri)?; + let result = self + .store + .get(&location) + .await + .map_err(|err| storage_backend_error("read", uri, err))?; + let etag = result.meta.e_tag.clone(); + let bytes = result + .bytes() + .await + .map_err(|err| storage_backend_error("read", uri, err))?; + // The token SOURCE must agree with the write_text_if_match strategy + // below: conditional-update backends compare ETags server-side, so + // the token is the ETag; the local emulation compares content, so + // the token is the content hash. Mixing them makes every CAS lose. + let version = if self.supports_conditional_update { + // Every S3-compatible store we target returns ETags; fall back + // to a content token rather than failing if one ever omits it. + etag.unwrap_or_else(|| local_version_token(&bytes)) + } else { + local_version_token(&bytes) + }; + let text = String::from_utf8(bytes.to_vec()).map_err(|err| { + OmniError::manifest_internal(format!("storage read failed for '{}': {}", uri, err)) + })?; + Ok((text, version)) + } + + async fn write_text_if_match( + &self, + uri: &str, + contents: &str, + expected_version: &str, + ) -> Result<Option<String>> { + let location = self.object_path(uri)?; + if self.supports_conditional_update { + let mode = PutMode::Update(object_store::UpdateVersion { + e_tag: Some(expected_version.to_string()), + version: None, + }); + return match self + .store + .put_opts( + &location, + PutPayload::from(contents.as_bytes().to_vec()), + mode.into(), + ) + .await + { + Ok(result) => Ok(Some( + result + .e_tag + .unwrap_or_else(|| local_version_token(contents.as_bytes())), + )), + Err(object_store::Error::Precondition { .. }) + | Err(object_store::Error::NotFound { .. }) => Ok(None), + Err(err) => Err(storage_backend_error("write_if_match", uri, err)), + }; + } + // Local emulation: content-compare then atomic replace. NOT a + // cross-process CAS (check-then-act gap) — safe under the callers' + // lock protocol only; tracked in docs/dev/invariants.md Known Gaps. + let current = match self.store.get(&location).await { + Ok(result) => result + .bytes() + .await + .map_err(|err| storage_backend_error("read", uri, err))?, + Err(object_store::Error::NotFound { .. }) => return Ok(None), + Err(err) => return Err(storage_backend_error("read", uri, err)), + }; + if local_version_token(¤t) != expected_version { + return Ok(None); + } + self.store + .put(&location, PutPayload::from(contents.as_bytes().to_vec())) + .await + .map_err(|err| storage_backend_error("write_if_match", uri, err))?; + Ok(Some(local_version_token(contents.as_bytes()))) + } + + async fn delete_prefix(&self, prefix_uri: &str) -> Result<()> { + // Directories are a local-FS concept: a list+delete loop would + // leave empty directory skeletons that local existence probes + // (cluster graph_root_exists uses std Path::exists) report as + // still-present. remove_dir_all reclaims them in one call. + if self.codec == UriCodec::Local { + let path = absolutize_lexically(local_path_from_uri(prefix_uri)?)?; + return match tokio::fs::remove_dir_all(&path).await { + Ok(()) => Ok(()), + Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()), + Err(err) => Err(err.into()), + }; + } + let prefix = self.object_path(prefix_uri.trim_end_matches('/'))?; + let mut entries = self.store.list(Some(&prefix)); + let mut locations = Vec::new(); + while let Some(meta) = entries + .try_next() + .await + .map_err(|err| storage_backend_error("delete_prefix", prefix_uri, err))? + { + locations.push(meta.location); + } + for location in locations { + match self.store.delete(&location).await { + Ok(()) => {} + Err(object_store::Error::NotFound { .. }) => {} + Err(err) => return Err(storage_backend_error("delete_prefix", prefix_uri, err)), + } + } + Ok(()) + } } pub fn storage_kind_for_uri(uri: &str) -> StorageKind { @@ -506,8 +487,8 @@ pub fn storage_kind_for_uri(uri: &str) -> StorageKind { pub fn storage_for_uri(uri: &str) -> Result<Arc<dyn StorageAdapter>> { match storage_kind_for_uri(uri) { - StorageKind::Local => Ok(Arc::new(LocalStorageAdapter)), - StorageKind::S3 => Ok(Arc::new(S3StorageAdapter::from_root_uri(uri)?)), + StorageKind::Local => Ok(Arc::new(ObjectStorageAdapter::local())), + StorageKind::S3 => Ok(Arc::new(ObjectStorageAdapter::s3_from_root_uri(uri)?)), } } @@ -553,6 +534,38 @@ fn local_path_from_uri(uri: &str) -> Result<PathBuf> { Ok(PathBuf::from(uri)) } +/// Lexically absolutize a local path: join relative paths onto the current +/// working directory and fold `.` / `..` components, without touching the +/// filesystem. Required because `object_store::path::Path` rejects +/// relative and dot segments, while callers (the CLI in particular) pass +/// paths like `./graph.omni` verbatim. +fn absolutize_lexically(path: PathBuf) -> Result<PathBuf> { + let joined = if path.is_absolute() { + path + } else { + std::env::current_dir() + .map_err(|err| { + OmniError::manifest_internal(format!( + "cannot resolve relative storage path '{}': {}", + path.display(), + err + )) + })? + .join(path) + }; + let mut out = PathBuf::new(); + for component in joined.components() { + match component { + Component::CurDir => {} + Component::ParentDir => { + out.pop(); + } + other => out.push(other), + } + } + Ok(out) +} + fn local_path_from_file_uri(uri: &str) -> Result<PathBuf> { let url = Url::parse(uri).map_err(|err| { OmniError::manifest_internal(format!("invalid file uri '{}': {}", uri, err)) @@ -610,17 +623,185 @@ fn env_var_truthy(key: &str) -> bool { #[cfg(test)] mod tests { + use super::*; - /// Regression for the write_text_if_absent buffering bug: a reader - /// immediately after Ok(true) must never see the created file empty. - /// The failure is timing-dependent (tokio's background write task), so - /// this loop is a best-effort local reproducer — the recorded red is - /// two CI failures ("EOF while parsing" on a state.json read right - /// after cluster import). + /// The executable backend contract: every assertion here must hold for + /// EVERY backend (the divergence class this adapter closed was "two + /// implementations, one prose contract, no referee"). The S3 variant + /// runs bucket-gated in `tests/s3_storage.rs` + /// (`s3_adapter_conditional_writes_contract`). + async fn contract_suite(adapter: &dyn StorageAdapter, root: &str) { + // Write/read round-trip; replace is in-place and atomic. + let a = format!("{root}/contract/a.json"); + adapter.write_text(&a, "v1").await.unwrap(); + assert_eq!(adapter.read_text(&a).await.unwrap(), "v1"); + adapter.write_text(&a, "v2").await.unwrap(); + assert_eq!(adapter.read_text(&a).await.unwrap(), "v2"); + + // exists: object yes; missing no; non-empty prefix yes (the + // directory-shaped probe Lance dataset roots rely on). + assert!(adapter.exists(&a).await.unwrap()); + assert!( + !adapter + .exists(&format!("{root}/contract/missing.json")) + .await + .unwrap() + ); + assert!(adapter.exists(&format!("{root}/contract")).await.unwrap()); + + // if_absent: exactly one claim wins; the loser leaves the winner's + // object untouched. + let claim = format!("{root}/contract/claim.json"); + assert!(adapter.write_text_if_absent(&claim, "first").await.unwrap()); + assert!(!adapter.write_text_if_absent(&claim, "second").await.unwrap()); + assert_eq!(adapter.read_text(&claim).await.unwrap(), "first"); + + // Versioned CAS: fresh token wins, stale token loses with Ok(None) + // (never a silent overwrite), missing object can't match. + let state = format!("{root}/contract/state.json"); + adapter.write_text(&state, "s1").await.unwrap(); + let (text, v1) = adapter.read_text_versioned(&state).await.unwrap(); + assert_eq!(text, "s1"); + let v2 = adapter + .write_text_if_match(&state, "s2", &v1) + .await + .unwrap() + .expect("fresh token must win"); + assert_ne!(v2, v1); + assert!( + adapter + .write_text_if_match(&state, "s3", &v1) + .await + .unwrap() + .is_none() + ); + assert_eq!(adapter.read_text(&state).await.unwrap(), "s2"); + assert!( + adapter + .write_text_if_match(&format!("{root}/contract/absent.json"), "x", &v1) + .await + .unwrap() + .is_none() + ); + + // rename: destination is replaced; source is gone. + let src = format!("{root}/contract/src.json"); + adapter.write_text(&src, "moved").await.unwrap(); + adapter.rename_text(&src, &a).await.unwrap(); + assert_eq!(adapter.read_text(&a).await.unwrap(), "moved"); + assert!(!adapter.exists(&src).await.unwrap()); + + // list_dir: direct children only, no sibling-prefix bleed, output + // URIs round-trip verbatim into read_text, missing dir is empty. + let dir_uri = format!("{root}/contract/list"); + adapter + .write_text(&format!("{dir_uri}/one.json"), "1") + .await + .unwrap(); + adapter + .write_text(&format!("{dir_uri}/two.json"), "2") + .await + .unwrap(); + adapter + .write_text(&format!("{dir_uri}/sub/three.json"), "3") + .await + .unwrap(); + adapter + .write_text(&format!("{root}/contract/list_log/x.json"), "x") + .await + .unwrap(); + let mut listed = adapter.list_dir(&dir_uri).await.unwrap(); + listed.sort(); + assert_eq!( + listed, + vec![ + format!("{dir_uri}/one.json"), + format!("{dir_uri}/two.json") + ] + ); + for uri in &listed { + adapter.read_text(uri).await.unwrap(); + } + assert!( + adapter + .list_dir(&format!("{root}/contract/nope")) + .await + .unwrap() + .is_empty() + ); + + // delete: idempotent. + adapter.delete(&claim).await.unwrap(); + adapter.delete(&claim).await.unwrap(); + assert!(!adapter.exists(&claim).await.unwrap()); + + // delete_prefix: recursive + idempotent; nothing under the prefix + // (including local directory skeletons) survives. + adapter + .delete_prefix(&format!("{root}/contract")) + .await + .unwrap(); + assert!(!adapter.exists(&a).await.unwrap()); + assert!(!adapter.exists(&format!("{root}/contract")).await.unwrap()); + adapter + .delete_prefix(&format!("{root}/contract")) + .await + .unwrap(); + } + + #[tokio::test] + async fn contract_suite_local() { + let dir = tempfile::tempdir().unwrap(); + let adapter = ObjectStorageAdapter::local(); + contract_suite(&adapter, dir.path().to_str().unwrap()).await; + } + + #[tokio::test] + async fn contract_suite_in_memory() { + // InMemory implements true conditional updates, so this runs the + // strong-CAS path (ETag tokens + PutMode::Update) without a bucket. + let adapter = ObjectStorageAdapter::in_memory(); + contract_suite(&adapter, "mem-root").await; + } + + /// `write_text_if_absent` must make the contents visible to any + /// subsequent reader before it returns — callers acknowledge + /// success the moment it resolves (cluster state bootstrap reads + /// the file back; init ownership claims depend on it). + /// Regression: the previous hand-rolled local adapter wrote through a + /// buffered `tokio::fs::File` without flushing, so the bytes could + /// still be in flight on the blocking pool while a reader saw an empty + /// or partial file. Reads back through `std::fs` deliberately — + /// cross-API visibility is the point. + #[tokio::test] + async fn local_write_text_if_absent_is_read_visible_on_return() { + let dir = tempfile::tempdir().unwrap(); + let adapter = ObjectStorageAdapter::local(); + let payload = "x".repeat(8 * 1024); + for i in 0..1000 { + let path = dir.path().join(format!("obj-{i}.json")); + let uri = format!("{}", path.display()); + assert!(adapter.write_text_if_absent(&uri, &payload).await.unwrap()); + let read = std::fs::read_to_string(&path).unwrap(); + assert_eq!( + read.len(), + payload.len(), + "iteration {i}: write_text_if_absent returned before its \ + contents reached the file" + ); + } + } + + /// Regression for the write_text_if_absent buffering bug, via the + /// `storage_for_uri` + `file://` construction path and a multi-thread + /// runtime (complements `local_write_text_if_absent_is_read_visible_- + /// on_return`, which uses the direct constructor and plain paths): a + /// reader immediately after Ok(true) must never see the created file + /// empty or short. #[tokio::test(flavor = "multi_thread")] async fn write_text_if_absent_is_read_consistent_immediately() { let dir = tempfile::tempdir().unwrap(); - let adapter = super::storage_for_uri(&format!("file://{}", dir.path().display())).unwrap(); + let adapter = storage_for_uri(&format!("file://{}", dir.path().display())).unwrap(); let payload = "x".repeat(64 * 1024); for i in 0..200 { let uri = format!("file://{}/f{}.json", dir.path().display(), i); @@ -630,55 +811,73 @@ mod tests { } } + /// Object-store semantics on the local filesystem: only objects exist. + /// An empty directory is not an object and not a non-empty prefix — + /// callers that genuinely probe local directories use std::fs. #[tokio::test] - async fn local_versioned_cas_roundtrip() { + async fn local_exists_is_object_semantics_for_directories() { let dir = tempfile::tempdir().unwrap(); - let uri = format!("{}/state.json", dir.path().display()); - let adapter = LocalStorageAdapter; - adapter.write_text(&uri, "v1").await.unwrap(); - let (text, version) = adapter.read_text_versioned(&uri).await.unwrap(); - assert_eq!(text, "v1"); - - // Matching token replaces and returns the next token. - let next = adapter - .write_text_if_match(&uri, "v2", &version) - .await - .unwrap() - .expect("fresh token must win"); - assert_ne!(next, version); - // The stale token must lose (CAS-lost is Ok(None), never silent). + let probe = dir.path().join("maybe-dataset"); + let adapter = ObjectStorageAdapter::local(); + std::fs::create_dir(&probe).unwrap(); assert!( - adapter - .write_text_if_match(&uri, "v3", &version) - .await - .unwrap() - .is_none() + !adapter.exists(probe.to_str().unwrap()).await.unwrap(), + "an empty directory is not an object" ); - let (text, _) = adapter.read_text_versioned(&uri).await.unwrap(); - assert_eq!(text, "v2"); - // Missing object: precondition can't hold. - let missing = format!("{}/absent.json", dir.path().display()); + std::fs::write(probe.join("1.manifest"), "m").unwrap(); assert!( - adapter - .write_text_if_match(&missing, "x", &version) - .await - .unwrap() - .is_none() + adapter.exists(probe.to_str().unwrap()).await.unwrap(), + "a non-empty prefix exists (the Lance dataset-root probe shape)" ); } + /// list_dir output is anchored on the INPUT dir_uri, so `file://` + /// anchors and paths with spaces round-trip byte-identically into + /// read_text — the cluster store passes file://-schemed roots. #[tokio::test] - async fn local_delete_prefix_is_recursive_and_idempotent() { + async fn local_list_round_trips_file_scheme_and_spaces() { let dir = tempfile::tempdir().unwrap(); - let root = format!("{}/tree", dir.path().display()); - let adapter = LocalStorageAdapter; - adapter.write_text(&format!("{root}/a.txt"), "a").await.unwrap(); - adapter.write_text(&format!("{root}/sub/b.txt"), "b").await.unwrap(); - adapter.delete_prefix(&root).await.unwrap(); - assert!(!adapter.exists(&format!("{root}/a.txt")).await.unwrap()); - adapter.delete_prefix(&root).await.unwrap(); // absent -> Ok + let root = dir.path().join("with space"); + let adapter = ObjectStorageAdapter::local(); + let plain = format!("{}/x.json", root.display()); + adapter.write_text(&plain, "x").await.unwrap(); + + let listed = adapter.list_dir(root.to_str().unwrap()).await.unwrap(); + assert_eq!(listed, vec![plain.clone()]); + assert_eq!(adapter.read_text(&listed[0]).await.unwrap(), "x"); + + let file_anchor = format!("file://{}", root.display()); + let listed = adapter.list_dir(&file_anchor).await.unwrap(); + assert_eq!(listed, vec![format!("{file_anchor}/x.json")]); + assert_eq!(adapter.read_text(&listed[0]).await.unwrap(), "x"); + } + + /// Relative and dot-segment paths are lexically absolutized before + /// hitting the object-path layer (which rejects them) — the CLI passes + /// `./graph.omni`-shaped URIs verbatim. + #[tokio::test] + async fn local_paths_with_dot_segments_are_absolutized() { + let dir = tempfile::tempdir().unwrap(); + let adapter = ObjectStorageAdapter::local(); + let uri = format!("{}/sub/../dotted.json", dir.path().display()); + adapter.write_text(&uri, "x").await.unwrap(); + assert_eq!(adapter.read_text(&uri).await.unwrap(), "x"); + assert!(dir.path().join("dotted.json").exists()); + } + + /// Upstream local rename creates missing destination parents — more + /// lenient than the previous bare fs::rename; pinned so an upstream + /// regression is loud. + #[tokio::test] + async fn local_rename_creates_missing_destination_parents() { + let dir = tempfile::tempdir().unwrap(); + let adapter = ObjectStorageAdapter::local(); + let src = format!("{}/src.json", dir.path().display()); + adapter.write_text(&src, "x").await.unwrap(); + let dst = format!("{}/new-sub/dst.json", dir.path().display()); + adapter.rename_text(&src, &dst).await.unwrap(); + assert_eq!(adapter.read_text(&dst).await.unwrap(), "x"); } - use super::*; #[test] fn storage_backend_selection_is_scheme_aware() { @@ -732,15 +931,4 @@ mod tests { assert_eq!(location.key, "graph/_schema.pg"); } - #[tokio::test] - async fn local_write_text_if_absent_creates_once_without_overwrite() { - let dir = tempfile::tempdir().unwrap(); - let uri = dir.path().join("claim.txt"); - let uri = uri.to_str().unwrap(); - let storage = LocalStorageAdapter; - - assert!(storage.write_text_if_absent(uri, "first").await.unwrap()); - assert!(!storage.write_text_if_absent(uri, "second").await.unwrap()); - assert_eq!(storage.read_text(uri).await.unwrap(), "first"); - } } diff --git a/crates/omnigraph/tests/failpoints.rs b/crates/omnigraph/tests/failpoints.rs index 3be0a56..b45cfa0 100644 --- a/crates/omnigraph/tests/failpoints.rs +++ b/crates/omnigraph/tests/failpoints.rs @@ -1190,6 +1190,1222 @@ async fn refresh_runs_roll_forward_recovery_in_process() { assert_eq!(helpers::count_rows(&db, "node:Person").await, 2); } +/// The long-lived-process contract for `load`: a Phase B → Phase C +/// failure (per-table `commit_staged` advanced Lance HEAD, manifest +/// publish did not land, sidecar persists) must not wedge subsequent +/// loads on the same engine handle. This is the server shape — `POST +/// /ingest` calls `load_as` on a shared handle with no reopen between +/// requests — so the follow-up load must heal the sidecar-covered +/// drift in-process: no restart, no explicit `refresh()`, no +/// `omnigraph repair`. +#[tokio::test] +async fn load_after_finalize_publisher_failure_heals_without_reopen() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + + // Failed multi-table load: Person + Company + WorksAt all run + // commit_staged (Lance HEAD advances on three tables), then the + // publisher is wedged before the manifest commit. + { + let _failpoint = ScopedFailPoint::new("mutation.post_finalize_pre_publisher", "return"); + let err = load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Alice","age":30}} +{"type":"Person","data":{"name":"Bob","age":25}} +{"type":"Company","data":{"name":"Acme"}} +{"edge":"WorksAt","from":"Alice","to":"Acme"} +"#, + LoadMode::Merge, + ) + .await + .unwrap_err(); + assert!( + err.to_string() + .contains("injected failpoint triggered: mutation.post_finalize_pre_publisher"), + "unexpected error: {err}" + ); + let recovery_dir = dir.path().join("__recovery"); + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 1, + "exactly one sidecar must persist after the finalize failure" + ); + } + + // Follow-up load on the SAME handle, touching the drifted tables. + // Must succeed without manual intervention. + load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Carol","age":41}} +{"type":"Company","data":{"name":"Globex"}} +"#, + LoadMode::Merge, + ) + .await + .expect( + "a follow-up load on the same handle must heal sidecar-covered \ + drift in-process instead of demanding repair/restart", + ); + + // Both batches are visible: the first load rolled forward, the + // second landed normally on top of it. + assert_eq!(helpers::count_rows(&db, "node:Person").await, 3); + assert_eq!(helpers::count_rows(&db, "node:Company").await, 2); + assert_eq!(helpers::count_rows(&db, "edge:WorksAt").await, 1); + + // The sidecar was consumed by the in-process roll-forward. + let recovery_dir = dir.path().join("__recovery"); + if recovery_dir.exists() { + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 0, + "sidecar must be consumed by the in-process roll-forward" + ); + } +} + +/// Phase A storage-fault contract: a sidecar PUT failure (S3 PutObject / +/// fs write, injected at `recovery.sidecar_write`) must abort the load +/// BEFORE any Lance HEAD advances — no sidecar, no drift, nothing to +/// recover — and the same handle must write normally once the fault +/// clears (a transient storage error never wedges the graph). +#[tokio::test] +async fn sidecar_write_failure_aborts_load_with_no_head_advance() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + + let person_uri = node_table_uri(&uri, "Person"); + let pre_head = lance::Dataset::open(&person_uri) + .await + .unwrap() + .version() + .version; + + { + let _failpoint = ScopedFailPoint::new("recovery.sidecar_write", "return"); + let err = load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Alice","age":30}} +{"type":"Company","data":{"name":"Acme"}} +"#, + LoadMode::Merge, + ) + .await + .unwrap_err(); + assert!( + err.to_string() + .contains("injected failpoint triggered: recovery.sidecar_write"), + "unexpected error: {err}" + ); + } + + // Phase A ordering: the sidecar write precedes the first + // commit_staged, so the failed load left no sidecar and moved no + // Lance HEAD — manifest and HEAD agree, nothing to recover. + let recovery_dir = dir.path().join("__recovery"); + if recovery_dir.exists() { + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 0, + "a Phase A put failure must not leave a sidecar" + ); + } + let post_head = lance::Dataset::open(&person_uri) + .await + .unwrap() + .version() + .version; + assert_eq!( + pre_head, post_head, + "a Phase A put failure must abort before any Lance HEAD advance" + ); + let manifest_pin = db + .snapshot_of(omnigraph::db::ReadTarget::branch("main")) + .await + .unwrap() + .entry("node:Person") + .unwrap() + .table_version; + assert_eq!(manifest_pin, post_head, "no drift after a Phase A abort"); + assert_eq!(helpers::count_rows(&db, "node:Person").await, 0); + + // Fault cleared: the same handle writes normally — no wedge, no + // recovery required. + load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Alice","age":30}} +{"type":"Company","data":{"name":"Acme"}} +"#, + LoadMode::Merge, + ) + .await + .expect("a transient sidecar put failure must not wedge later writes"); + assert_eq!(helpers::count_rows(&db, "node:Person").await, 1); + assert_eq!(helpers::count_rows(&db, "node:Company").await, 1); +} + +/// Real-backend coverage of the sidecar lifecycle: the same-handle heal +/// scenario on an S3-compatible store, exercising sidecar put / list / +/// delete through the S3 object-store backend instead of the +/// local filesystem backend. Skips unless `OMNIGRAPH_S3_TEST_BUCKET` is set +/// (same gate as `s3_storage.rs`); CI runs it against RustFS. +#[tokio::test] +async fn s3_load_recovers_after_publisher_failure_without_reopen() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let Some(uri) = helpers::s3_test_graph_uri("failpoints") else { + eprintln!( + "skipping s3_load_recovers_after_publisher_failure_without_reopen: \ + OMNIGRAPH_S3_TEST_BUCKET is not set" + ); + return; + }; + + let _scenario = FailScenario::setup(); + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + + // Failed load: commit_staged lands on S3, manifest publish does not; + // the sidecar PUT went through the S3 adapter. + { + let _failpoint = ScopedFailPoint::new("mutation.post_finalize_pre_publisher", "return"); + let err = load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Alice","age":30}} +{"type":"Company","data":{"name":"Acme"}} +"#, + LoadMode::Merge, + ) + .await + .err() + .expect("finalize failpoint must fail the load"); + assert!( + err.to_string() + .contains("injected failpoint triggered: mutation.post_finalize_pre_publisher"), + "unexpected error: {err}" + ); + } + + // Same-handle follow-up load: the entry heal LISTs __recovery/ on + // S3, rolls the sidecar forward, DELETEs it, and the write lands. + load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Bob","age":25}} +"#, + LoadMode::Merge, + ) + .await + .expect("the same-handle heal must converge on an S3-backed graph"); + + assert_eq!(helpers::count_rows(&db, "node:Person").await, 2); + assert_eq!(helpers::count_rows(&db, "node:Company").await, 1); + + // Reopen cross-check: nothing left for the open-time sweep, state + // converged (the heal consumed the sidecar on S3). + drop(db); + let db = Omnigraph::open(&uri).await.unwrap(); + assert_eq!(helpers::count_rows(&db, "node:Person").await, 2); +} + +/// Storage-fault contract for the recovery AUDIT write (injected at +/// `recovery.record_audit`): a failure after the roll-forward's manifest +/// publish aborts that recovery attempt loudly and keeps the sidecar; +/// re-entry detects the already-published manifest (stale-sidecar path), +/// records exactly one `RolledForward` audit row, and converges — the +/// documented retry tolerance in `record_audit`'s contract, exercised +/// end-to-end through a real injected failure. +#[tokio::test] +async fn record_audit_failure_after_roll_forward_converges_on_next_write() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + + // Pending sidecar with real drift. + { + let _failpoint = ScopedFailPoint::new("mutation.post_finalize_pre_publisher", "return"); + load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Alice","age":30}} +"#, + LoadMode::Merge, + ) + .await + .err() + .expect("finalize failpoint must fail the load"); + } + + // The next write's heal rolls forward (manifest publish lands) but + // the audit write fails — the write must fail loudly and the sidecar + // must survive for the retry. + { + let _failpoint = ScopedFailPoint::new("recovery.record_audit", "return"); + let err = load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Bob","age":25}} +"#, + LoadMode::Merge, + ) + .await + .err() + .expect("an audit write failure mid-heal must fail the write"); + assert!( + err.to_string() + .contains("injected failpoint triggered: recovery.record_audit"), + "unexpected error: {err}" + ); + let recovery_dir = dir.path().join("__recovery"); + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 1, + "the sidecar must survive an audit write failure so the retry can record it" + ); + } + + // Fault cleared: the next write converges — stale-sidecar audit + // recovery (manifest already advanced) + the write itself. + load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Carol","age":41}} +"#, + LoadMode::Merge, + ) + .await + .expect("recovery must converge once the audit fault clears"); + + let recovery_dir = dir.path().join("__recovery"); + if recovery_dir.exists() { + assert_eq!(std::fs::read_dir(&recovery_dir).unwrap().count(), 0); + } + // Alice (rolled forward) + Carol (clean). Bob's write failed before + // staging anything — the heal error aborted his load at entry. + assert_eq!(helpers::count_rows(&db, "node:Person").await, 2); + // Exactly one audit row despite two recovery attempts: the first + // attempt's audit failed before any row landed; the retry recorded + // the roll-forward once. + let audit_uri = format!( + "{}/_graph_commit_recoveries.lance", + uri.trim_end_matches('/') + ); + let audit_rows = lance::Dataset::open(&audit_uri) + .await + .expect("audit dataset exists after the retried recovery") + .count_rows(None) + .await + .unwrap(); + assert_eq!(audit_rows, 1, "exactly one recovery audit row"); +} + +/// Storage-fault contract for the `__recovery/` LIST (S3 ListObjectsV2, +/// injected at `recovery.sidecar_list`): every consumer fails loudly — +/// the write-entry heal fails the write, the open-time sweep fails the +/// open — rather than silently skipping recovery over a pending sidecar +/// (which would be consumer tolerance of drift). Once the fault clears, +/// open recovers normally. +#[tokio::test] +async fn sidecar_list_failure_fails_write_and_open_loudly_then_clears() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + + // Pending sidecar via the usual finalize → publisher failure. + { + let _failpoint = ScopedFailPoint::new("mutation.post_finalize_pre_publisher", "return"); + let err = load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Alice","age":30}} +"#, + LoadMode::Merge, + ) + .await + .unwrap_err(); + assert!( + err.to_string() + .contains("injected failpoint triggered: mutation.post_finalize_pre_publisher"), + "unexpected error: {err}" + ); + let recovery_dir = dir.path().join("__recovery"); + assert_eq!(std::fs::read_dir(&recovery_dir).unwrap().count(), 1); + } + + let _failpoint = ScopedFailPoint::new("recovery.sidecar_list", "return"); + + // Write-entry heal: the list failure surfaces as the write's error — + // no silent skip that would proceed over the pending sidecar. + let err = load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Bob","age":25}} +"#, + LoadMode::Merge, + ) + .await + .unwrap_err(); + assert!( + err.to_string() + .contains("injected failpoint triggered: recovery.sidecar_list"), + "the write-entry heal must surface a list failure loudly; got: {err}" + ); + + // Open-time sweep: a fresh ReadWrite open fails on the same fault. + drop(db); + let err = Omnigraph::open(&uri) + .await + .err() + .expect("open must fail while the sidecar list fault is active"); + assert!( + err.to_string() + .contains("injected failpoint triggered: recovery.sidecar_list"), + "the open-time sweep must surface a list failure loudly; got: {err}" + ); + + // Fault cleared: open recovers the pending sidecar normally. + drop(_failpoint); + let db = Omnigraph::open(&uri).await.unwrap(); + let recovery_dir = dir.path().join("__recovery"); + if recovery_dir.exists() { + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 0, + "open after the fault clears must recover the sidecar" + ); + } + assert_eq!(helpers::count_rows(&db, "node:Person").await, 1); +} + +/// Phase D storage-fault contract: a sidecar DELETE failure (S3 +/// DeleteObject, injected at `recovery.sidecar_delete`) after a +/// successful manifest publish must NOT fail the user's write — the +/// data is durable and visible. The stale sidecar it leaves behind is +/// consumed by the next write's entry heal (attributed `RolledForward` +/// audit row), not by an operator. +#[tokio::test] +async fn sidecar_delete_failure_keeps_write_success_and_next_write_heals() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + + { + let _failpoint = ScopedFailPoint::new("recovery.sidecar_delete", "return"); + // The load itself must succeed: commit_staged + manifest publish + // landed; only the Phase D cleanup failed (swallowed + logged). + load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Alice","age":30}} +"#, + LoadMode::Merge, + ) + .await + .expect("a Phase D delete failure must not fail a write that already published"); + assert_eq!(helpers::count_rows(&db, "node:Person").await, 1); + let recovery_dir = dir.path().join("__recovery"); + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 1, + "the swallowed delete leaves a stale sidecar behind" + ); + } + + // Fault cleared: the next write's entry heal consumes the stale + // sidecar (manifest pin already caught up — the stale-sidecar + // roll-forward audit path) and the write lands. + load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Bob","age":25}} +"#, + LoadMode::Merge, + ) + .await + .expect("a stale sidecar from a failed Phase D delete must not block later writes"); + + let recovery_dir = dir.path().join("__recovery"); + if recovery_dir.exists() { + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 0, + "the stale sidecar must be consumed by the next write's heal" + ); + } + assert_eq!(helpers::count_rows(&db, "node:Person").await, 2); +} + +/// Phase A storage-fault contract for branch_merge — the multi-table +/// writer where sidecar-before-commit ordering matters most. A sidecar +/// PUT failure must abort the merge before any target-table HEAD moves; +/// retrying after the fault clears merges cleanly. +#[tokio::test] +async fn sidecar_write_failure_aborts_branch_merge_with_no_head_advance() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Alice","age":30}} +"#, + LoadMode::Append, + ) + .await + .unwrap(); + + db.branch_create("feature").await.unwrap(); + // Diverge BOTH sides so Person is a RewriteMerged candidate (the + // merge path that pins a recovery sidecar; an unchanged target would + // adopt source state without one). + helpers::mutate_branch( + &mut db, + "feature", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Eve")], &[("$age", 22)]), + ) + .await + .unwrap(); + helpers::mutate_main( + &mut db, + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Mallory")], &[("$age", 35)]), + ) + .await + .unwrap(); + + let person_uri = node_table_uri(&uri, "Person"); + let pre_head = lance::Dataset::open(&person_uri) + .await + .unwrap() + .version() + .version; + + { + let _failpoint = ScopedFailPoint::new("recovery.sidecar_write", "return"); + let err = db.branch_merge("feature", "main").await.unwrap_err(); + assert!( + err.to_string() + .contains("injected failpoint triggered: recovery.sidecar_write"), + "unexpected error: {err}" + ); + } + + let recovery_dir = dir.path().join("__recovery"); + if recovery_dir.exists() { + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 0, + "a Phase A put failure must not leave a sidecar" + ); + } + let post_head = lance::Dataset::open(&person_uri) + .await + .unwrap() + .version() + .version; + assert_eq!( + pre_head, post_head, + "a Phase A put failure must abort the merge before any target \ + Lance HEAD advance" + ); + assert_eq!(helpers::count_rows(&db, "node:Person").await, 2); + + // Fault cleared: the merge lands cleanly. + db.branch_merge("feature", "main") + .await + .expect("a transient sidecar put failure must not wedge the merge"); + assert_eq!(helpers::count_rows(&db, "node:Person").await, 3); +} + +/// Same contract as +/// `load_after_finalize_publisher_failure_heals_without_reopen`, for the +/// mutation entry point: after a failed mutation leaves a sidecar, the +/// next mutation on the same handle heals it in-process — no explicit +/// `refresh()` (which `refresh_runs_roll_forward_recovery_in_process` +/// covers), no reopen. +#[tokio::test] +async fn mutation_after_finalize_publisher_failure_heals_without_reopen() { + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + + { + let _failpoint = ScopedFailPoint::new("mutation.post_finalize_pre_publisher", "return"); + let err = mutate_main( + &mut db, + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Eve")], &[("$age", 22)]), + ) + .await + .unwrap_err(); + assert!( + err.to_string() + .contains("injected failpoint triggered: mutation.post_finalize_pre_publisher"), + "unexpected error: {err}" + ); + let recovery_dir = dir.path().join("__recovery"); + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 1, + "exactly one sidecar must persist after the finalize failure" + ); + } + + // Follow-up mutation on the SAME handle, same table. No refresh, no + // reopen — the write entry point heals the drift itself. + mutate_main( + &mut db, + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Frank")], &[("$age", 33)]), + ) + .await + .expect( + "a follow-up mutation on the same handle must heal sidecar-covered \ + drift in-process instead of demanding repair/restart", + ); + + // Eve rolled forward, Frank landed normally. + assert_eq!(helpers::count_rows(&db, "node:Person").await, 2); + + let recovery_dir = dir.path().join("__recovery"); + if recovery_dir.exists() { + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 0, + "sidecar must be consumed by the in-process roll-forward" + ); + } +} + +/// Same heal contract as the load/mutation variants, for the schema +/// apply entry point: a pending roll-forward-eligible sidecar (here +/// from a failed load) must be healed in-process before the migration +/// runs, so a long-lived handle can evolve the schema without a +/// restart after a Phase B → Phase C failure. +#[tokio::test] +async fn schema_apply_after_finalize_publisher_failure_heals_without_reopen() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + + { + let _failpoint = ScopedFailPoint::new("mutation.post_finalize_pre_publisher", "return"); + let err = load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Alice","age":30}} +{"type":"Company","data":{"name":"Acme"}} +"#, + LoadMode::Merge, + ) + .await + .unwrap_err(); + assert!( + err.to_string() + .contains("injected failpoint triggered: mutation.post_finalize_pre_publisher"), + "unexpected error: {err}" + ); + let recovery_dir = dir.path().join("__recovery"); + assert_eq!(std::fs::read_dir(&recovery_dir).unwrap().count(), 1); + } + + // Additive migration on the SAME handle. Must heal the load's + // sidecar first, then apply normally. + let desired = format!("{}\nnode Tag {{ name: String @key }}\n", helpers::TEST_SCHEMA); + db.apply_schema(&desired).await.expect( + "schema apply on the same handle must heal sidecar-covered \ + drift in-process instead of failing until restart", + ); + + // The failed load rolled forward; the migration landed. + assert_eq!(helpers::count_rows(&db, "node:Person").await, 1); + assert_eq!(helpers::count_rows(&db, "node:Company").await, 1); + assert_eq!(helpers::count_rows(&db, "node:Tag").await, 0); + + // No sidecar remains (the load's was consumed by the heal; schema + // apply deleted its own after publish). + let recovery_dir = dir.path().join("__recovery"); + if recovery_dir.exists() { + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 0, + "no sidecar may remain after heal + successful schema apply" + ); + } +} + +/// Same heal contract for the branch-merge entry point: a pending +/// roll-forward-eligible sidecar on the target branch must be healed +/// (with its recovery audit row) before the merge reads its target +/// snapshot — not silently folded into the merge's publish. +#[tokio::test] +async fn branch_merge_after_finalize_publisher_failure_heals_without_reopen() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Alice","age":30}} +"#, + LoadMode::Append, + ) + .await + .unwrap(); + + // A feature branch with its own write, to merge back later. + db.branch_create("feature").await.unwrap(); + helpers::mutate_branch( + &mut db, + "feature", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Eve")], &[("$age", 22)]), + ) + .await + .unwrap(); + + // Failed load on MAIN: Person drifts ahead of the manifest with a + // sidecar covering it. + { + let _failpoint = ScopedFailPoint::new("mutation.post_finalize_pre_publisher", "return"); + let err = load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Bob","age":25}} +"#, + LoadMode::Merge, + ) + .await + .unwrap_err(); + assert!( + err.to_string() + .contains("injected failpoint triggered: mutation.post_finalize_pre_publisher"), + "unexpected error: {err}" + ); + let recovery_dir = dir.path().join("__recovery"); + assert_eq!(std::fs::read_dir(&recovery_dir).unwrap().count(), 1); + } + + // Merge on the SAME handle. The entry heal must consume the load's + // sidecar (publishing Bob with a recovery audit row) BEFORE the + // merge captures its target snapshot. + db.branch_merge("feature", "main").await.expect( + "branch merge on the same handle must heal sidecar-covered \ + drift in-process instead of failing or folding it silently", + ); + + // No sidecar remains: the heal consumed the load's sidecar; the + // merge deleted its own after publish. Without the entry heal the + // merge's publish makes the drifted commit visible as a side effect + // (manifest catches up to HEAD) and the stale sidecar lingers + // until some later sweep — recovery must be attributed, not + // incidental. + let recovery_dir = dir.path().join("__recovery"); + if recovery_dir.exists() { + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 0, + "the load's sidecar must be consumed by the entry heal, not left behind" + ); + } + + // All three writes are visible on main: Alice (clean load), Bob + // (rolled forward), Eve (merged). + assert_eq!(helpers::count_rows(&db, "node:Person").await, 3); +} + +/// Discarding an orphaned-branch sidecar must be idempotent across a +/// Phase D delete failure: the audit row + commit land before the +/// sidecar delete, so a delete fault leaves the sidecar on disk with +/// the audit already written — the retry must NOT append a second +/// audit row for the same operation, only finish the delete. +#[tokio::test] +async fn orphaned_branch_discard_is_idempotent_across_delete_failure() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Alice\",\"age\":30}}\n", + LoadMode::Merge, + ) + .await + .unwrap(); + db.branch_create("feature").await.unwrap(); + helpers::mutate_branch( + &mut db, + "feature", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Eve")], &[("$age", 22)]), + ) + .await + .unwrap(); + + // Deferred-shape sidecar pinned to feature (head < expected ⇒ + // invariant violation ⇒ every roll-forward-only pass defers it). + let person_uri = node_table_uri(&uri, "Person"); + let sidecar_json = format!( + r#"{{ + "schema_version": 1, + "operation_id": "01H000000000000000000000ID", + "started_at": "0", + "branch": "feature", + "actor_id": null, + "writer_kind": "Mutation", + "tables": [ + {{ + "table_key": "node:Person", + "table_path": "{person_uri}", + "expected_version": 999, + "post_commit_pin": 1000, + "table_branch": "feature" + }} + ] + }}"# + ); + let recovery_dir = dir.path().join("__recovery"); + std::fs::create_dir_all(&recovery_dir).unwrap(); + std::fs::write( + recovery_dir.join("01H000000000000000000000ID.json"), + &sidecar_json, + ) + .unwrap(); + + // Orphan the sidecar. + db.branch_delete("feature").await.unwrap(); + + // First write: the discard path writes its audit row, then the + // sidecar delete fails (injected). The write fails loudly. + { + let _failpoint = ScopedFailPoint::new("recovery.sidecar_delete", "return"); + let err = load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Bob\",\"age\":25}}\n", + LoadMode::Merge, + ) + .await + .err() + .expect("a sidecar-delete fault mid-discard must fail the write"); + assert!( + err.to_string() + .contains("injected failpoint triggered: recovery.sidecar_delete"), + "unexpected error: {err}" + ); + assert_eq!(std::fs::read_dir(&recovery_dir).unwrap().count(), 1); + } + + // Retry: must finish the delete WITHOUT a second audit row. + load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Bob\",\"age\":25}}\n", + LoadMode::Merge, + ) + .await + .expect("the retry must complete the orphan discard and the write"); + assert_eq!(std::fs::read_dir(&recovery_dir).unwrap().count(), 0); + let orphan_rows = helpers::recovery::recovery_audit_kinds(dir.path()) + .await + .into_iter() + .filter(|kind| kind == "OrphanedBranchDiscarded") + .count(); + assert_eq!( + orphan_rows, 1, + "exactly one OrphanedBranchDiscarded audit row despite the delete-fault retry" + ); +} + +/// When the commit-time drift guard cannot LIST sidecars to classify +/// the drift (transient storage fault on the guard's list, after the +/// entry heal's list succeeded), it must say so and name BOTH recovery +/// paths — not confidently route to `omnigraph repair`, which refuses +/// while a sidecar is pending. Sequenced failpoint: first list (entry +/// heal) passes, second list (the guard) fails. +#[tokio::test] +async fn drift_guard_names_both_paths_when_sidecar_list_fails() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"alice\",\"age\":30}}\n", + LoadMode::Append, + ) + .await + .unwrap(); + + // Rollback-eligible (deferred) sidecar covering main's Person drift — + // same shape as refresh_defers_rollback_eligible_sidecar_to_next_open. + let snapshot = db + .snapshot_of(omnigraph::db::ReadTarget::branch("main")) + .await + .unwrap(); + let entry = snapshot.entry("node:Person").unwrap(); + let person_uri = format!("{}/{}", uri.trim_end_matches('/'), entry.table_path); + let manifest_pin = entry.table_version; + let mut ds = lance::Dataset::open(&person_uri).await.unwrap(); + helpers::lance_delete_inline(&mut ds, "1 = 2").await; + let head_after_drift = ds.version().version; + let sidecar_json = format!( + r#"{{ + "schema_version": 1, + "operation_id": "01H0000000000000000000LSTF", + "started_at": "0", + "branch": null, + "actor_id": null, + "writer_kind": "Mutation", + "tables": [ + {{ + "table_key":"node:Person", + "table_path":"{}", + "expected_version":{}, + "post_commit_pin":{} + }} + ] + }}"#, + person_uri, + manifest_pin - 1, + head_after_drift, + ); + let recovery_dir = dir.path().join("__recovery"); + std::fs::create_dir_all(&recovery_dir).unwrap(); + std::fs::write( + recovery_dir.join("01H0000000000000000000LSTF.json"), + &sidecar_json, + ) + .unwrap(); + + // First list (entry heal) passes and defers the sidecar; second + // list (the guard's classification) fails. + let _failpoint = ScopedFailPoint::new("recovery.sidecar_list", "1*off->1*return"); + let err = load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"bob\",\"age\":25}}\n", + LoadMode::Merge, + ) + .await + .err() + .expect("drift must still fail the write"); + let msg = err.to_string(); + assert!( + msg.contains("could not classify the drift") + && msg.contains("omnigraph repair") + && msg.contains("reopen the graph read-write"), + "an unclassifiable drift must name BOTH recovery paths, not \ + confidently route to repair; got: {msg}" + ); +} + +/// The other half of the orphan-discard fault matrix: the audit append +/// fails AFTER the recovery commit landed. The retry (keyed on the +/// audit row, the operator-facing record) must converge to exactly one +/// audit row and a consumed sidecar. The second recovery commit the +/// retry appends is the documented not-atomic-pair-write tolerance +/// (same class as `record_audit` and the manifest→commit-graph Known +/// Gap): bounded commit-graph noise, never a lost or duplicated audit +/// record under clean failures. +#[tokio::test] +async fn orphaned_branch_discard_converges_across_audit_append_failure() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Alice\",\"age\":30}}\n", + LoadMode::Merge, + ) + .await + .unwrap(); + db.branch_create("feature").await.unwrap(); + helpers::mutate_branch( + &mut db, + "feature", + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Eve")], &[("$age", 22)]), + ) + .await + .unwrap(); + + // Deferred-shape sidecar pinned to feature, then orphaned. + let person_uri = node_table_uri(&uri, "Person"); + let sidecar_json = format!( + r#"{{ + "schema_version": 1, + "operation_id": "01H000000000000000000000AF", + "started_at": "0", + "branch": "feature", + "actor_id": null, + "writer_kind": "Mutation", + "tables": [ + {{ + "table_key": "node:Person", + "table_path": "{person_uri}", + "expected_version": 999, + "post_commit_pin": 1000, + "table_branch": "feature" + }} + ] + }}"# + ); + let recovery_dir = dir.path().join("__recovery"); + std::fs::create_dir_all(&recovery_dir).unwrap(); + std::fs::write( + recovery_dir.join("01H000000000000000000000AF.json"), + &sidecar_json, + ) + .unwrap(); + db.branch_delete("feature").await.unwrap(); + + // First write: the recovery commit lands, then the audit append + // fails (injected). The write fails loudly; the sidecar survives so + // the discard is retried with the audit still owed. + { + let _failpoint = ScopedFailPoint::new("recovery.orphan_discard_audit_append", "return"); + let err = load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Bob\",\"age\":25}}\n", + LoadMode::Merge, + ) + .await + .err() + .expect("an audit-append fault mid-discard must fail the write"); + assert!( + err.to_string() + .contains("injected failpoint triggered: recovery.orphan_discard_audit_append"), + "unexpected error: {err}" + ); + assert_eq!( + std::fs::read_dir(&recovery_dir).unwrap().count(), + 1, + "the sidecar must survive an audit-append fault so the discard is retried" + ); + let orphan_rows = helpers::recovery::recovery_audit_kinds(dir.path()) + .await + .into_iter() + .filter(|kind| kind == "OrphanedBranchDiscarded") + .count(); + assert_eq!(orphan_rows, 0, "no audit row landed before the fault"); + } + + // Retry: converges — sidecar consumed, exactly one audit row. + load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Bob\",\"age\":25}}\n", + LoadMode::Merge, + ) + .await + .expect("the retry must complete the orphan discard and the write"); + assert_eq!(std::fs::read_dir(&recovery_dir).unwrap().count(), 0); + let orphan_rows = helpers::recovery::recovery_audit_kinds(dir.path()) + .await + .into_iter() + .filter(|kind| kind == "OrphanedBranchDiscarded") + .count(); + assert_eq!( + orphan_rows, 1, + "exactly one OrphanedBranchDiscarded audit row despite the audit-fault retry" + ); +} + +/// After the write-entry heal rolls a SchemaApply sidecar forward (a +/// crashed apply on the SAME handle: staging promoted, registrations +/// published), the handle's in-memory catalog must be reloaded — disk +/// and manifest are on the new schema, and validating subsequent +/// writes against the stale catalog rejects rows of types the graph +/// already has. +#[tokio::test] +async fn load_after_schema_apply_phase_b_failure_uses_recovered_catalog() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let mut db = Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap(); + load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"alice\",\"age\":30}}\n", + LoadMode::Append, + ) + .await + .unwrap(); + + // v2: a Person property (rewritten_tables work) + a new Tag type + // (table-set change, keeps the staging disambiguator decisive). + let v2_schema = r#"node Person { + name: String @key + age: I32? + city: String? +} + +node Company { + name: String @key +} + +node Tag { + label: String @key +} + +edge Knows: Person -> Person { + since: Date? +} + +edge WorksAt: Person -> Company +"#; + { + let _failpoint = ScopedFailPoint::new("schema_apply.after_staging_write", "return"); + let err = db.apply_schema(v2_schema).await.unwrap_err(); + assert!( + err.to_string() + .contains("injected failpoint triggered: schema_apply.after_staging_write"), + "unexpected error: {err}" + ); + let recovery_dir = dir.path().join("__recovery"); + assert_eq!(std::fs::read_dir(&recovery_dir).unwrap().count(), 1); + } + + // Same handle: a load of the NEW type. The entry heal rolls the + // apply forward (staging promoted, manifest registers node:Tag) — + // and the loader must then validate against the RECOVERED catalog, + // not the stale in-memory one. + load_jsonl( + &mut db, + "{\"type\":\"Tag\",\"data\":{\"label\":\"t1\"}}\n", + LoadMode::Merge, + ) + .await + .expect( + "after the heal rolls the schema apply forward, the same handle \ + must accept rows of the recovered schema's types", + ); + assert_eq!(helpers::count_rows(&db, "node:Tag").await, 1); + let recovery_dir = dir.path().join("__recovery"); + if recovery_dir.exists() { + assert_eq!(std::fs::read_dir(&recovery_dir).unwrap().count(), 0); + } +} + +/// A concurrent write's entry heal must NOT promote a LIVE schema +/// apply's staging files. The apply pauses just after writing its +/// staging files (sidecar on disk from Phase A, staging on disk, +/// manifest not yet committed); a load on the same handle fires the +/// heal in that window. If the heal's schema-staging reconcile runs +/// unserialized, it promotes the staging files from under the live +/// apply — putting the NEW catalog live against the OLD manifest — and +/// the resumed apply's own renames then fail on the missing sources: +/// an error (and a corrupted catalog) for an otherwise-healthy apply. +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn heal_does_not_promote_live_schema_apply_staging() { + use omnigraph::loader::LoadMode; + use std::sync::Arc; + + let _scenario = FailScenario::setup(); + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + + let db = Arc::new(Omnigraph::init(&uri, helpers::TEST_SCHEMA).await.unwrap()); + + // Pause the apply right after its staging files land (its sidecar is + // already on disk from Phase A; the manifest commit has not run). + let failpoint = ScopedFailPoint::new("schema_apply.after_staging_write", "pause"); + + let apply_db = Arc::clone(&db); + let desired = format!("{}\nnode Tag {{ name: String @key }}\n", helpers::TEST_SCHEMA); + let apply = tokio::spawn(async move { apply_db.apply_schema(&desired).await }); + + // Wait until the apply is parked in the window: staging on disk. + let staging_pg = dir.path().join("_schema.pg.staging"); + for _ in 0..500 { + if staging_pg.exists() { + break; + } + tokio::time::sleep(std::time::Duration::from_millis(10)).await; + } + assert!(staging_pg.exists(), "schema apply never reached the paused window"); + + // Concurrent load on the same handle: its entry heal runs while the + // apply is paused. The load itself may fail (schema apply in + // progress) — what matters is what its heal does to the live apply. + let load_db = Arc::clone(&db); + let load = tokio::spawn(async move { + load_db + .load_as( + "main", + None, + "{\"type\":\"Person\",\"data\":{\"name\":\"Alice\",\"age\":30}}\n", + LoadMode::Merge, + None, + ) + .await + }); + + // Give the load's heal time to act inside the window. Broken code + // completes the load here (its heal promoted the staging files and + // stole the apply's commit); fixed code leaves the load blocked on + // the schema-apply serialization key until the apply finishes. + tokio::time::sleep(std::time::Duration::from_millis(500)).await; + drop(failpoint); + + let apply_result = apply.await.unwrap(); + let _ = tokio::time::timeout(std::time::Duration::from_secs(30), load) + .await + .expect("load must complete once the apply releases its guards") + .unwrap(); + apply_result.expect( + "a concurrent write's heal must not promote the live schema \ + apply's staging files out from under it", + ); + + // The migration landed and nothing recovery-shaped remains. + assert_eq!(helpers::count_rows(&db, "node:Tag").await, 0); + let recovery_dir = dir.path().join("__recovery"); + if recovery_dir.exists() { + assert_eq!(std::fs::read_dir(&recovery_dir).unwrap().count(), 0); + } +} + /// Refresh-time recovery must NOT call `Dataset::restore` — it can /// silently orphan a concurrent writer's commit. Sidecars that would /// require rollback must be left on disk for the next ReadWrite open. @@ -1306,6 +2522,26 @@ async fn refresh_defers_rollback_eligible_sidecar_to_next_open() { pre_head={pre_head}, post_head={post_head}", ); + // A write attempt while the rollback-eligible sidecar is deferred: + // the write-entry heal defers it again (roll-forward-only), and the + // commit-time drift guard must name the actual recovery path (a + // read-write reopen) — NOT `omnigraph repair`, which refuses while + // a sidecar is pending. + let err = mutate_main( + &mut db, + MUTATION_QUERIES, + "insert_person", + &mixed_params(&[("$name", "Grace")], &[("$age", 50)]), + ) + .await + .unwrap_err(); + assert!( + err.to_string() + .contains("a pending recovery sidecar requires rollback"), + "drift guard must point at a read-write reopen for sidecar-covered \ + rollback-eligible drift; got: {err}" + ); + // Cross-check: drop the engine and reopen — full sweep handles // the rollback (will use Dataset::restore safely; no concurrent // writers at open time). diff --git a/crates/omnigraph/tests/helpers/recovery.rs b/crates/omnigraph/tests/helpers/recovery.rs index 90d9a25..4cb45e0 100644 --- a/crates/omnigraph/tests/helpers/recovery.rs +++ b/crates/omnigraph/tests/helpers/recovery.rs @@ -143,6 +143,39 @@ pub fn sidecar_operation_ids(graph_root: &Path) -> Vec<String> { ids } +/// Recovery-audit rows' `recovery_kind` values at `graph_root`, in +/// storage order. Empty when the audit dataset doesn't exist yet. +pub async fn recovery_audit_kinds(graph_root: &Path) -> Vec<String> { + let recoveries_dir = graph_root.join("_graph_commit_recoveries.lance"); + if !recoveries_dir.exists() { + return Vec::new(); + } + let ds = Dataset::open(recoveries_dir.to_str().unwrap()) + .await + .expect("recoveries dataset opens"); + let batches: Vec<RecordBatch> = ds + .scan() + .try_into_stream() + .await + .unwrap() + .try_collect() + .await + .unwrap(); + let mut out = Vec::new(); + for batch in batches { + let kinds = batch + .column_by_name("recovery_kind") + .expect("recovery_kind column present") + .as_any() + .downcast_ref::<StringArray>() + .expect("recovery_kind is Utf8"); + for i in 0..kinds.len() { + out.push(kinds.value(i).to_string()); + } + } + out +} + pub async fn branch_head_commit_id(graph_root: &Path, branch: &str) -> Result<String> { let graph = match branch { "main" => CommitGraph::open(&graph_uri(graph_root)).await?, diff --git a/crates/omnigraph/tests/recovery.rs b/crates/omnigraph/tests/recovery.rs index 37d46cb..b5ca58f 100644 --- a/crates/omnigraph/tests/recovery.rs +++ b/crates/omnigraph/tests/recovery.rs @@ -134,6 +134,218 @@ async fn recovery_refuses_unknown_schema_version_on_open() { ); } +#[tokio::test] +async fn recovery_refuses_corrupt_sidecar_on_open_and_write() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + + // A truncated/garbage sidecar — e.g. a crashed writer or a partial + // local-FS write (S3 PutObject is atomic; local fs::write is not). + write_sidecar_file(dir.path(), "01H000000000000000000000CC", "{not json"); + + // A live handle's write-entry heal must surface the parse failure + // loudly instead of proceeding over a sidecar it cannot interpret. + let err = load_jsonl( + &mut db, + r#"{"type":"Person","data":{"name":"Alice","age":30}} +"#, + LoadMode::Merge, + ) + .await + .err() + .expect("expected the write to fail on the corrupt sidecar"); + assert!( + err.to_string().contains("is not valid JSON"), + "expected the corrupt-sidecar parse error, got: {}", + err, + ); + + // A fresh ReadWrite open fails the same way. + drop(db); + let err = Omnigraph::open(uri) + .await + .err() + .expect("expected open to fail because of the corrupt sidecar"); + let msg = err.to_string(); + assert!( + msg.contains("01H000000000000000000000CC") && msg.contains("is not valid JSON"), + "expected the corrupt-sidecar parse error naming the file, got: {}", + msg, + ); + // The file must remain on disk for inspection — never auto-deleted. + assert!( + list_recovery_dir(dir.path()).contains(&"01H000000000000000000000CC.json".to_string()), + "corrupt sidecar should remain on disk after refusal" + ); + + // Read-only open still works — the sweep is skipped entirely. + let _db = Omnigraph::open_read_only(uri).await.unwrap(); +} + +/// The commit-time drift guard's advice must be branch-aware: a pending +/// sidecar on ANOTHER branch does not cover this branch's drift. With a +/// deferred feature-branch sidecar on disk and genuinely uncovered drift +/// on main, the main write must still point at `omnigraph repair` — a +/// read-write reopen recovers the sidecar but cannot repair main's +/// uncovered drift. +#[tokio::test] +async fn drift_guard_advice_ignores_other_branch_sidecars() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, TEST_SCHEMA).await.unwrap(); + load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Alice\",\"age\":30}}\n", + LoadMode::Merge, + ) + .await + .unwrap(); + db.branch_create("feature").await.unwrap(); + // A real feature write forks Person's Lance dataset onto the branch + // (the heal classifies a feature sidecar against the forked head). + db.mutate( + "feature", + helpers::MUTATION_QUERIES, + "insert_person", + &helpers::mixed_params(&[("$name", "eve")], &[("$age", 22)]), + ) + .await + .unwrap(); + + // A sidecar pinning node:Person ON FEATURE, shaped so the write-entry + // heal defers it (head < expected_version classifies as an invariant + // violation; roll-forward-only mode leaves it for the next ReadWrite + // open) — it persists through the write attempt below. + let person_uri = node_table_uri(uri, "Person"); + let sidecar_json = format!( + r#"{{ + "schema_version": 1, + "operation_id": "01H000000000000000000000XB", + "started_at": "0", + "branch": "feature", + "actor_id": null, + "writer_kind": "Mutation", + "tables": [ + {{ + "table_key": "node:Person", + "table_path": "{person_uri}", + "expected_version": 999, + "post_commit_pin": 1000, + "table_branch": "feature" + }} + ] + }}"# + ); + write_sidecar_file(dir.path(), "01H000000000000000000000XB", &sidecar_json); + + // Genuinely uncovered drift on MAIN's Person (raw Lance write + // bypassing the manifest — the `omnigraph repair` class). + let mut ds = Dataset::open(&person_uri).await.unwrap(); + let _ = helpers::lance_delete_inline(&mut ds, "1 = 2").await; + + let err = load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Bob\",\"age\":25}}\n", + LoadMode::Merge, + ) + .await + .err() + .expect("uncovered main drift must fail the write"); + assert!( + err.to_string().contains("run `omnigraph repair`"), + "a feature-branch sidecar must not flip main's uncovered-drift \ + advice to the reopen path; got: {err}" + ); +} + +/// A deferred sidecar pinned to a branch that is subsequently DELETED +/// must not wedge the graph: the branch's tree and forks are reclaimed, +/// so the pinned drift is unreachable and the sidecar is provably moot. +/// Both the write-entry heal and the open-time sweep must classify it +/// as orphaned (audit + discard) instead of failing to open the dead +/// branch on every write and every ReadWrite open — a terminal state, +/// since `repair` refuses while a sidecar is pending. +#[tokio::test] +async fn deleted_branch_sidecar_does_not_wedge_writes_or_open() { + use omnigraph::loader::{LoadMode, load_jsonl}; + + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap().to_string(); + let mut db = Omnigraph::init(&uri, TEST_SCHEMA).await.unwrap(); + load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Alice\",\"age\":30}}\n", + LoadMode::Merge, + ) + .await + .unwrap(); + db.branch_create("feature").await.unwrap(); + db.mutate( + "feature", + helpers::MUTATION_QUERIES, + "insert_person", + &helpers::mixed_params(&[("$name", "eve")], &[("$age", 22)]), + ) + .await + .unwrap(); + + // A rollback-eligible (deferred) sidecar pinned to feature — shaped + // so every roll-forward-only pass leaves it on disk. + let person_uri = node_table_uri(&uri, "Person"); + let sidecar_json = format!( + r#"{{ + "schema_version": 1, + "operation_id": "01H000000000000000000000DB", + "started_at": "0", + "branch": "feature", + "actor_id": null, + "writer_kind": "Mutation", + "tables": [ + {{ + "table_key": "node:Person", + "table_path": "{person_uri}", + "expected_version": 999, + "post_commit_pin": 1000, + "table_branch": "feature" + }} + ] + }}"# + ); + write_sidecar_file(dir.path(), "01H000000000000000000000DB", &sidecar_json); + + // Branch delete defers the rollback-eligible sidecar and proceeds — + // the sidecar now references a branch that no longer exists. + db.branch_delete("feature").await.unwrap(); + + // The next write's heal must classify the orphan and discard it, + // not fail opening the dead branch. + load_jsonl( + &mut db, + "{\"type\":\"Person\",\"data\":{\"name\":\"Bob\",\"age\":25}}\n", + LoadMode::Merge, + ) + .await + .expect("a write after deleting a sidecar-pinned branch must succeed"); + assert_eq!( + list_recovery_dir(dir.path()).len(), + 0, + "the orphaned sidecar must be discarded (with an audit row), not left to wedge" + ); + + // And a fresh ReadWrite open must succeed too (the sweep shares the + // same classification). + drop(db); + let db = Omnigraph::open(&uri) + .await + .expect("ReadWrite open after deleting a sidecar-pinned branch must succeed"); + assert_eq!(helpers::count_rows(&db, "node:Person").await, 2); +} + #[tokio::test] async fn read_only_open_skips_recovery_sweep() { let dir = tempfile::tempdir().unwrap(); diff --git a/docs/dev/invariants.md b/docs/dev/invariants.md index 655e360..b3bcfaf 100644 --- a/docs/dev/invariants.md +++ b/docs/dev/invariants.md @@ -42,10 +42,12 @@ Use it this way: 5. **Recovery is part of the commit protocol.** Writers that can advance Lance HEAD before manifest publish must write `__recovery/{ulid}.json` sidecars. - `Omnigraph::open` in read-write mode runs the all-or-nothing sweep, and - `refresh` runs roll-forward-only recovery for long-lived processes. Do not - add a new writer kind without sidecar coverage or an explicit proof that no - Lance HEAD can move before manifest publish. + `Omnigraph::open` in read-write mode runs the all-or-nothing sweep; the + write entry points (`load_as`, `mutate_as`, `apply_schema_as`, + `branch_merge_as`) and `refresh` run roll-forward-only recovery in-process, + so a long-lived process converges on its next write rather than at restart. Do not add a new writer kind without + sidecar coverage or an explicit proof that no Lance HEAD can move before + manifest publish. 6. **Strong consistency is the default.** Reads are snapshot-isolated, writes are durable before acknowledgement, and branch reads observe the current @@ -106,7 +108,7 @@ Use it this way: | Index lifecycle | `ensure_indices` is explicit today; reconciler-based convergence is roadmap | [indexes.md](../user/indexes.md), [maintenance.md](../user/maintenance.md) | | Traversal IDs | Runtime still builds `TypeIndex`; Lance stable row-id based graph IDs are roadmap | [architecture.md](architecture.md), [query-language.md](../user/query-language.md) | | Auth | Bearer token hashing and server-side actor resolution are implemented at the HTTP boundary | [server.md](../user/server.md), [policy.md](../user/policy.md) | -| Tests | Tempdir-backed Lance tests are the current substrate; there is no `MemStorage` test backend | [testing.md](testing.md) | +| Tests | Tempdir-backed Lance tests are the current substrate; the storage adapter has an in-memory backend for adapter-level contract tests, but Lance datasets bypass it | [testing.md](testing.md) | The branch-delete reconciler is authority-derived: it reclaims orphaned forks today and degrades to a no-op if Lance ships an atomic multi-dataset branch @@ -146,6 +148,29 @@ them explicit. Remove the skip when the upstream Lance fix lands — the `lance_surface_guards.rs::compact_files_still_fails_on_blob_columns` guard turns red on that bump to force it. +- **Recovery is serialized against live writers in-process only:** the + write-entry heal (and `refresh`) serialize against a live writer's sidecar + lifetime via the per-`(table, branch)` write queues plus the schema-apply + serialization key — all in-process primitives. A recovery pass in one + process cannot serialize against a live writer in another (the open-time + sweep has the same exposure, and always has): it may roll a live foreign + writer's sidecar forward, which degrades to publisher-CAS contention for + data writes but can race the schema-staging promotion for a foreign live + schema apply. Multi-process writers on one graph are already documented + one-winner-CAS territory; closing this fully needs a cross-process + serialization primitive (e.g. lease-based use of the schema-apply lock + branch) — design it before promoting multi-process write topologies. +- **Local `write_text_if_match` is not a cross-process CAS:** object-store + backends use a true conditional put (ETag If-Match; the in-memory test + backend too), but upstream `object_store` leaves `PutMode::Update` + unimplemented for `LocalFileSystem`, so the local path emulates CAS with + a content-token compare followed by an atomic replace — a check-then-act + gap plus content-token ABA. Every current caller goes through the cluster + lock protocol first, which makes this safe. A lock-free caller would get + S3-correct but local-racy behavior — the same divergence shape as the + acknowledged-before-visible bug this branch fixed. Close it (local CAS + primitive, or a trait-level lock requirement) before admitting any + lock-free `if_match` caller. - **Manifest→commit-graph publish atomicity:** a graph commit advances `__manifest` (the visibility authority) and then appends `_graph_commits` as two separate writes (`commit_updates_with_actor_with_expected`, failpoint diff --git a/docs/dev/testing.md b/docs/dev/testing.md index f2b33de..d2d08f3 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -43,7 +43,7 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav | `validators.rs` | Schema constraint enforcement (enum, range, unique, cardinality) across JSONL, insert, update paths | | `policy_engine_chassis.rs` | Engine-layer Cedar enforcement (MR-722): allow + deny through every `_as` writer via the SDK directly — no HTTP — proving embedded and CLI callers hit the same gate as the server, with action × scope shapes matching `authorize_request` | | `maintenance.rs` | `optimize` (compaction), `repair` (explicit uncovered-drift publish), and `cleanup` (version GC): empty/idempotent/no-op edges, policy validation, head preservation; `optimize` publishes its own compaction (`optimize_publishes_compaction_to_manifest_so_schema_apply_succeeds`), skips pre-existing uncovered drift (`optimize_skips_preexisting_manifest_head_drift`), and refuses to run while a `__recovery` sidecar is pending (`optimize_defers_when_recovery_sidecar_is_pending`); `repair` previews/heals verified maintenance drift, refuses raw semantic drift without `--force`, and forced repair publishes only by explicit operator choice | -| `failpoints.rs` | Failure-injection coverage (gated on `failpoints` feature). Includes the five per-writer Phase B → recovery integration tests (`recovery_rolls_forward_after_finalize_publisher_failure`, `schema_apply_phase_b_failure_recovered_on_next_open`, `branch_merge_phase_b_failure_recovered_on_next_open`, `ensure_indices_phase_b_failure_recovered_on_next_open`, `optimize_phase_b_failure_recovered_on_next_open`). | +| `failpoints.rs` | Failure-injection coverage (gated on `failpoints` feature). Includes the five per-writer Phase B → recovery integration tests (`recovery_rolls_forward_after_finalize_publisher_failure`, `schema_apply_phase_b_failure_recovered_on_next_open`, `branch_merge_phase_b_failure_recovered_on_next_open`, `ensure_indices_phase_b_failure_recovered_on_next_open`, `optimize_phase_b_failure_recovered_on_next_open`) and the write-entry in-process heal contract (the four `*_after_finalize_publisher_failure_heals_without_reopen` tests — load, mutation, schema apply, branch merge: a follow-up write on the same handle rolls a sidecar-covered residual forward without reopen/refresh) and the storage-fault matrix for the sidecar lifecycle (`recovery.sidecar_{write,delete,list}` / `recovery.record_audit` failpoints: Phase A put failure aborts with zero drift, Phase D delete failure is swallowed and healed by the next write, list failures are loud at heal and open, audit-append failures are retried to exactly one audit row; plus the bucket-gated `s3_load_recovers_after_publisher_failure_without_reopen`). | | `recovery.rs` | Open-time recovery sweep — sidecar I/O, classifier dispatch (NoMovement / RolledPastExpected / UnexpectedAtP1 / UnexpectedMultistep / InvariantViolation), all-or-nothing decision, roll-forward via `ManifestBatchPublisher::publish`, roll-back via `Dataset::restore`, audit row in `_graph_commit_recoveries.lance`, `OpenMode::ReadOnly` skip path | | `composite_flow.rs` | Compositional/narrative end-to-end stories — multi-step flows that compose mechanics covered by other test files. Catches integration regressions where individual operations all pass their unit tests but their composition breaks (sequential merges, post-merge main writes, time-travel through merge DAG, reopen consistency over multi-merge histories, post-optimize and post-cleanup strict writes). | @@ -57,7 +57,7 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav - **CLI** — `crates/omnigraph-cli/tests/support/mod.rs`: `Command`-style wrapper for invoking `omnigraph`, server-process spawning, fixture resolution, output assertion helpers. - **Server** — no shared helpers; server tests call the `Omnigraph` engine API directly and exercise endpoints over the wire. -> Note: there is **no `MemStorage` or in-memory backend** today. Tests use `tempfile::tempdir()` for local FS. If you find yourself needing one for layer isolation, that's an architectural ask — keep it explicit in [docs/dev/invariants.md](invariants.md) under known gaps. +> Note: the storage adapter has an in-memory backend (`ObjectStorageAdapter::in_memory()`, full contract including true conditional updates) used by the adapter contract tests in `storage.rs`. It covers only the text-object layer (sidecars, schema staging, cluster state) — **Lance datasets bypass the adapter**, so engine integration tests still use `tempfile::tempdir()`. An in-memory Lance substrate remains an architectural ask — keep it explicit in [docs/dev/invariants.md](invariants.md) under known gaps. ## Failpoints (fault injection) @@ -74,6 +74,7 @@ CI runs three S3-backed tests against a containerized RustFS server (`.github/wo - `cargo test -p omnigraph-server --test s3` (single-graph serving + config-free `--cluster s3://` boot) - `cargo test -p omnigraph-cluster --test s3_cluster` (full control-plane lifecycle on the bucket) - `cargo test -p omnigraph-cli --test system_local local_cli_s3_end_to_end_init_load_read_flow` +- `cargo test -p omnigraph-engine --features failpoints --test failpoints s3_` (recovery-sidecar lifecycle on a real bucket) Locally, set `OMNIGRAPH_S3_TEST_BUCKET` (and the usual `AWS_*` vars including `AWS_ENDPOINT_URL_S3` for non-AWS) before running. Without those, S3 tests skip gracefully. diff --git a/docs/dev/writes.md b/docs/dev/writes.md index 5647d82..82d6ba8 100644 --- a/docs/dev/writes.md +++ b/docs/dev/writes.md @@ -215,19 +215,43 @@ Triggers for the residual: transient Lance write errors during finalize (object-store retry budget exhaustion, disk full); persistent publisher contention exceeding `PUBLISHER_RETRY_BUDGET = 5` retries. -**Long-running servers**: `Omnigraph::refresh` runs roll-forward-only -recovery in-process — the common Phase B → Phase C residual closes -without a restart. The next mutation on the same handle (after refresh) -no longer surfaces `ExpectedVersionMismatch` for the failed table. +**Long-running servers**: the write entry points (`load_as`, +`mutate_as`, `apply_schema_as`, `branch_merge_as`) and +`Omnigraph::refresh` run roll-forward-only recovery in-process +(`recovery::heal_pending_sidecars_roll_forward`) — the common +Phase B → Phase C residual closes on the next write, without a +restart and without an explicit refresh. The heal lists `__recovery/` +(one `list_dir`; empty in the steady state) and, per sidecar, acquires +the same per-`(table_key, table_branch)` write queues every sidecar +writer holds from before `write_sidecar` until after `delete_sidecar` — +so it serializes against a live writer instead of rolling its +in-flight sidecar forward from under it (a sidecar whose queues can be +acquired belongs to a writer that finished or died; an existence +re-check after the wait skips the finished case). Lock order is +queues → coordinator, matching every writer's commit→publish path. +Pinned by the four +`tests/failpoints.rs::*_after_finalize_publisher_failure_heals_without_reopen` +tests (load, mutation, schema apply, branch merge). The maintenance +entries need the heal for more than liveness: without it, a schema +apply re-plans rewrites from the manifest pin and orphans the drifted +Phase-B commit (dropping its rows), and a branch merge publishes the +drift as an unattributed side effect — both while the stale sidecar +lingers to misclassify later. Sidecars that would require a `Dataset::restore` (mixed / unexpected state) are deferred to the next `OpenMode::ReadWrite` open: restore is unsafe under concurrency because Lance's `check_restore_txn` accepts the restore against in-flight Append/Update/Delete commits and silently orphans them (pinned by `tests/staged_writes.rs::lance_restore_loses_to_concurrent_append_via_orphaning`). +When such a deferred sidecar blocks a write, the commit-time drift +guard says so explicitly ("a pending recovery sidecar requires +rollback — reopen the graph read-write") instead of pointing at +`omnigraph repair`, which refuses while a sidecar is pending. Continuous in-process recovery for the rollback path is the goal of a -future background reconciler with per-(table, branch) writer-queue -acquisition. +future background reconciler. `ensure_indices` does not heal at entry +itself — it runs inside the load / schema-apply flows after their +entry heal, and its strict preconditions still fail loudly on drift +when invoked directly. The publisher-CAS contract is unchanged: a *concurrent writer* that advances any of our touched tables between snapshot capture and @@ -235,6 +259,44 @@ publisher commit produces exactly one winner. The residual above is about *our* abandoned commits in the failure path, not about concurrency races. +**Sidecar I/O failure semantics** (all sidecar I/O goes through the +backend-generic `StorageAdapter`; the contracts below are pinned by the +storage-fault failpoints `recovery.sidecar_{write,delete,list}` / +`recovery.record_audit` and their tests in `tests/failpoints.rs` and +`tests/recovery.rs`): + +- **Phase A put fails** (S3 PutObject / fs write): the writer aborts + before its first HEAD-advancing commit — no sidecar, no drift, + nothing to recover; a transient fault never wedges later writes. +- **Phase D delete fails** (S3 DeleteObject): swallowed with a warning — + the write already published, so failing the caller would report an + error for a durable write. The stale sidecar is consumed by the next + write's entry heal (or the next open) via the stale-sidecar + audit-recovery path, recorded as `RolledForward`. +- **`__recovery/` list fails** (S3 ListObjectsV2): loud at every + consumer — the write-entry heal fails the write, the open-time sweep + fails the open. Silently skipping recovery would be consumer + tolerance of drift. +- **Corrupt / unparseable sidecar**: refused loudly by heal and open + alike; the file stays on disk for operator inspection (read-only + opens still work — the sweep is skipped there). +- **Audit append fails after a roll-forward publish**: that recovery + attempt errors and keeps the sidecar; re-entry sees the + already-published manifest, records exactly one `RolledForward` + audit row, and deletes the sidecar (the retry tolerance documented + on `record_audit`). + +Backend notes (the adapter is one implementation over `object_store` +for every backend): local writes stage through `name#<digits>` temp +files that the backend filters from listings and refuses to address — +crash residue of that shape is invisible to the sweep, harmless, and +reclaimed by `delete_prefix`/manual cleanup. Storage errors are +backend-wrapped text without a typed NotFound discriminant — callers +that need missing-vs-error (the cluster store) probe `exists()` first. +`exists()` itself is object-store semantics everywhere: only objects +(or non-empty prefixes) exist, and a permission failure is a loud +error, not a silent `false`. + ## Conflict shape Concurrent writers to the same `(table, branch)` produce exactly one diff --git a/docs/user/storage.md b/docs/user/storage.md index 2c57a92..9cc2356 100644 --- a/docs/user/storage.md +++ b/docs/user/storage.md @@ -104,8 +104,8 @@ The split — L2 owns the cross-dataset catalog; L1 owns the per-dataset interna | Scheme | Backend | Notes | |---|---|---| -| local path / `file://` | `LocalStorageAdapter` (tokio) | Normalized to absolute paths | -| `s3://bucket/prefix` | `S3StorageAdapter` (object_store) | Honors `AWS_ENDPOINT_URL_S3`, `AWS_ALLOW_HTTP`, `AWS_S3_FORCE_PATH_STYLE` | +| local path / `file://` | `ObjectStorageAdapter` over `object_store::LocalFileSystem` | Normalized to absolute paths; relative and dot-segment paths are lexically absolutized | +| `s3://bucket/prefix` | `ObjectStorageAdapter` over `object_store` S3 | Honors `AWS_ENDPOINT_URL_S3`, `AWS_ALLOW_HTTP`, `AWS_S3_FORCE_PATH_STYLE` | | `http(s)://host:port` | HTTP client to `omnigraph-server` | Used by CLI as a target, not a storage backend | ## Object-store env vars (S3-compatible) From 4821e7208f294df385fe8ecf8f5b75e5e14584c9 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Sat, 13 Jun 2026 17:03:20 +0300 Subject: [PATCH 142/165] refactor(api): extract omnigraph-api-types crate (RFC-009 Phase 2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The HTTP wire DTOs and their engine-result -> DTO mappings move from omnigraph-server's api module into a new omnigraph-api-types crate that both server and CLI can depend on (engine must not — DAG: api-types -> engine, never the reverse). The crate holds plain serde/utoipa types only; the transport-coupled error->status mapping stays in the server (lib.rs/ handlers). The one server-runtime coupling (query_catalog_entry, which maps a StoredQuery — not a wire type) stays behind in api.rs, now calling the crate's pub param_descriptor. api.rs becomes a thin `pub use omnigraph_api_types::*` re-export, so every omnigraph_server::api::Foo path (handlers, the OpenApi schema list, CLI imports) resolves unchanged. openapi.json regenerates BYTE-IDENTICAL (the Phase-2 referee: 77 openapi tests green, zero diff). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- Cargo.lock | 12 + Cargo.toml | 1 + crates/omnigraph-api-types/Cargo.toml | 16 + crates/omnigraph-api-types/src/lib.rs | 697 +++++++++++++++++++++++++ crates/omnigraph-server/Cargo.toml | 1 + crates/omnigraph-server/src/api.rs | 701 +------------------------- 6 files changed, 737 insertions(+), 691 deletions(-) create mode 100644 crates/omnigraph-api-types/Cargo.toml create mode 100644 crates/omnigraph-api-types/src/lib.rs diff --git a/Cargo.lock b/Cargo.lock index 2099055..bb95af9 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4541,6 +4541,17 @@ dependencies = [ "tokio", ] +[[package]] +name = "omnigraph-api-types" +version = "0.7.0" +dependencies = [ + "omnigraph-compiler", + "omnigraph-engine", + "serde", + "serde_json", + "utoipa", +] + [[package]] name = "omnigraph-cli" version = "0.7.0" @@ -4674,6 +4685,7 @@ dependencies = [ "futures", "lance", "lance-index", + "omnigraph-api-types", "omnigraph-cluster", "omnigraph-compiler", "omnigraph-engine", diff --git a/Cargo.toml b/Cargo.toml index 918ac05..76b37e0 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -4,6 +4,7 @@ members = [ "crates/omnigraph-compiler", "crates/omnigraph", "crates/omnigraph-cli", + "crates/omnigraph-api-types", "crates/omnigraph-cluster", "crates/omnigraph-policy", "crates/omnigraph-server", diff --git a/crates/omnigraph-api-types/Cargo.toml b/crates/omnigraph-api-types/Cargo.toml new file mode 100644 index 0000000..d69d4fe --- /dev/null +++ b/crates/omnigraph-api-types/Cargo.toml @@ -0,0 +1,16 @@ +[package] +name = "omnigraph-api-types" +version = "0.7.0" +edition = "2024" +description = "Shared HTTP wire DTOs for Omnigraph — request/response types and engine-result → DTO mappings used by both omnigraph-server and omnigraph-cli (RFC-009). Plain serde/utoipa types; no transport or server internals." +license = "MIT" +repository = "https://github.com/ModernRelay/omnigraph" +homepage = "https://github.com/ModernRelay/omnigraph" +documentation = "https://docs.rs/omnigraph-api-types" + +[dependencies] +omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.7.0" } +omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.7.0" } +serde = { workspace = true } +serde_json = { workspace = true } +utoipa = { workspace = true } diff --git a/crates/omnigraph-api-types/src/lib.rs b/crates/omnigraph-api-types/src/lib.rs new file mode 100644 index 0000000..910d86b --- /dev/null +++ b/crates/omnigraph-api-types/src/lib.rs @@ -0,0 +1,697 @@ +//! Shared HTTP wire DTOs (RFC-009 Phase 2) — moved from +//! omnigraph-server's api module so server and CLI share one definition +//! and one engine-result -> DTO mapping per verb. Plain serde/utoipa +//! types; no transport, no server internals. + +use omnigraph::db::{GraphCommit, MergeOutcome, ReadTarget, SchemaApplyResult, Snapshot}; +use omnigraph::error::{MergeConflict, MergeConflictKind}; +use omnigraph::loader::{LoadMode, LoadResult}; +use omnigraph_compiler::SchemaMigrationStep; +use omnigraph_compiler::query::ast::Param; +use omnigraph_compiler::result::QueryResult; +use omnigraph_compiler::types::{PropType, ScalarType}; +use serde::{Deserialize, Serialize}; +use serde_json::Value; +use utoipa::{IntoParams, ToSchema}; + +/// Shadow enum for documenting [`LoadMode`] in the OpenAPI schema. +#[derive(ToSchema)] +#[schema(as = LoadMode)] +#[allow(dead_code)] +enum LoadModeSchema { + /// Overwrite existing data. + #[schema(rename = "overwrite")] + Overwrite, + /// Append to existing data. + #[schema(rename = "append")] + Append, + /// Merge by id key (upsert). + #[schema(rename = "merge")] + Merge, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct SnapshotTableOutput { + pub table_key: String, + pub table_path: String, + pub table_version: u64, + pub table_branch: Option<String>, + pub row_count: u64, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct SnapshotOutput { + pub branch: String, + pub manifest_version: u64, + pub tables: Vec<SnapshotTableOutput>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct BranchCreateRequest { + /// Parent branch to fork from. Defaults to `main`. + pub from: Option<String>, + /// Name of the new branch. Must not already exist. + pub name: String, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct BranchCreateOutput { + pub uri: String, + pub from: String, + pub name: String, + pub actor_id: Option<String>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct BranchListOutput { + pub branches: Vec<String>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct BranchDeleteOutput { + pub uri: String, + pub name: String, + pub actor_id: Option<String>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct BranchMergeRequest { + /// Source branch whose commits will be merged. + pub source: String, + /// Target branch that will receive the merge. Defaults to `main`. + pub target: Option<String>, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ToSchema)] +#[serde(rename_all = "snake_case")] +pub enum BranchMergeOutcome { + AlreadyUpToDate, + FastForward, + Merged, +} + +impl From<MergeOutcome> for BranchMergeOutcome { + fn from(value: MergeOutcome) -> Self { + match value { + MergeOutcome::AlreadyUpToDate => Self::AlreadyUpToDate, + MergeOutcome::FastForward => Self::FastForward, + MergeOutcome::Merged => Self::Merged, + } + } +} + +impl BranchMergeOutcome { + pub fn as_str(self) -> &'static str { + match self { + Self::AlreadyUpToDate => "already_up_to_date", + Self::FastForward => "fast_forward", + Self::Merged => "merged", + } + } +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct BranchMergeOutput { + pub source: String, + pub target: String, + pub outcome: BranchMergeOutcome, + pub actor_id: Option<String>, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ToSchema)] +#[serde(rename_all = "snake_case")] +pub enum MergeConflictKindOutput { + DivergentInsert, + DivergentUpdate, + DeleteVsUpdate, + OrphanEdge, + UniqueViolation, + CardinalityViolation, + ValueConstraintViolation, +} + +impl MergeConflictKindOutput { + pub fn as_str(self) -> &'static str { + match self { + Self::DivergentInsert => "divergent_insert", + Self::DivergentUpdate => "divergent_update", + Self::DeleteVsUpdate => "delete_vs_update", + Self::OrphanEdge => "orphan_edge", + Self::UniqueViolation => "unique_violation", + Self::CardinalityViolation => "cardinality_violation", + Self::ValueConstraintViolation => "value_constraint_violation", + } + } +} + +impl From<MergeConflictKind> for MergeConflictKindOutput { + fn from(value: MergeConflictKind) -> Self { + match value { + MergeConflictKind::DivergentInsert => Self::DivergentInsert, + MergeConflictKind::DivergentUpdate => Self::DivergentUpdate, + MergeConflictKind::DeleteVsUpdate => Self::DeleteVsUpdate, + MergeConflictKind::OrphanEdge => Self::OrphanEdge, + MergeConflictKind::UniqueViolation => Self::UniqueViolation, + MergeConflictKind::CardinalityViolation => Self::CardinalityViolation, + MergeConflictKind::ValueConstraintViolation => Self::ValueConstraintViolation, + } + } +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct MergeConflictOutput { + pub table_key: String, + pub row_id: Option<String>, + pub kind: MergeConflictKindOutput, + pub message: String, +} + +impl From<&MergeConflict> for MergeConflictOutput { + fn from(value: &MergeConflict) -> Self { + Self { + table_key: value.table_key.clone(), + row_id: value.row_id.clone(), + kind: value.kind.into(), + message: value.message.clone(), + } + } +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct ReadTargetOutput { + pub branch: Option<String>, + pub snapshot: Option<String>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct ReadOutput { + pub query_name: String, + pub target: ReadTargetOutput, + pub row_count: usize, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub columns: Vec<String>, + pub rows: Value, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct ChangeOutput { + pub branch: String, + pub query_name: String, + pub affected_nodes: usize, + pub affected_edges: usize, + pub actor_id: Option<String>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct IngestTableOutput { + pub table_key: String, + pub rows_loaded: usize, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct IngestOutput { + pub uri: String, + pub branch: String, + /// Base branch a fork was requested from (the request's `from`), echoed + /// even when the branch already existed. `null` when `from` was absent. + pub base_branch: Option<String>, + pub branch_created: bool, + #[schema(value_type = LoadModeSchema)] + pub mode: LoadMode, + pub tables: Vec<IngestTableOutput>, + pub actor_id: Option<String>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct CommitOutput { + pub graph_commit_id: String, + pub manifest_branch: Option<String>, + pub manifest_version: u64, + pub parent_commit_id: Option<String>, + pub merged_parent_commit_id: Option<String>, + pub actor_id: Option<String>, + /// Commit creation time as Unix epoch microseconds. + #[schema(example = 1714000000000000i64)] + pub created_at: i64, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct CommitListOutput { + pub commits: Vec<CommitOutput>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct ReadRequest { + /// GQ query source. May declare one or more named queries; pick one with + /// `query_name` if there is more than one. + #[schema( + example = "query get_person($name: String) {\n match {\n $p: Person { name: $name }\n }\n return { $p.name, $p.age }\n}" + )] + pub query_source: String, + /// Name of the query to run when `query_source` declares multiple. Optional + /// when only one query is declared. + pub query_name: Option<String>, + /// JSON object whose keys match the query's declared parameters. + pub params: Option<Value>, + /// Branch to read from. Mutually exclusive with `snapshot`. Defaults to `main`. + pub branch: Option<String>, + /// Snapshot id to read from. Mutually exclusive with `branch`. + pub snapshot: Option<String>, +} + +/// Inline read-query request for `POST /query`. +/// +/// Friendlier-named alternative to [`ReadRequest`] for ad-hoc reads and +/// AI-agent integration. Mutations are rejected with 400 — use `POST +/// /mutate` (or its deprecated alias `POST /change`) for write queries. +/// Field names are deliberately short (`query`, `name`) to match the GQ +/// keyword and the CLI `-e` flag. +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct QueryRequest { + /// GQ read-query source. May declare one or more named queries; pick one + /// with `name` when more than one is declared. Mutations + /// (`insert`/`update`/`delete`) get 400 — use `POST /mutate` (or its + /// deprecated alias `POST /change`) instead. + #[schema(example = "query get_person($name: String) {\n match {\n $p: Person { name: $name }\n }\n return { $p.name, $p.age }\n}")] + pub query: String, + /// Name of the query to run when `query` declares multiple. Optional when + /// only one query is declared. + pub name: Option<String>, + /// JSON object whose keys match the query's declared parameters. + pub params: Option<Value>, + /// Branch to read from. Mutually exclusive with `snapshot`. Defaults to `main`. + pub branch: Option<String>, + /// Snapshot id to read from. Mutually exclusive with `branch`. + pub snapshot: Option<String>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct ChangeRequest { + /// GQ mutation source containing `insert`, `update`, or `delete` statements. + /// May declare multiple named mutations; pick one with `name`. + /// + /// Accepts the legacy field name `query_source` as a deserialization alias. + #[schema( + example = "query insert_person($name: String, $age: I32) {\n insert Person { name: $name, age: $age }\n}" + )] + #[serde(alias = "query_source")] + pub query: String, + /// Name of the mutation to run when `query` declares multiple. + /// + /// Accepts the legacy field name `query_name` as a deserialization alias. + #[serde(default, alias = "query_name")] + pub name: Option<String>, + /// JSON object whose keys match the mutation's declared parameters. + #[serde(default)] + pub params: Option<Value>, + /// Target branch. Defaults to `main`. + #[serde(default)] + pub branch: Option<String>, +} + +/// Body for `POST /queries/{name}` — invokes the server-side stored query +/// named in the path. The query source and name come from the registry, +/// never the body; only the runtime inputs are supplied here. +#[derive(Debug, Clone, Default, Serialize, Deserialize, ToSchema)] +pub struct InvokeStoredQueryRequest { + /// JSON object whose keys match the stored query's declared parameters. + #[serde(default)] + pub params: Option<Value>, + /// Branch to run against. Defaults to `main`; for a stored mutation the + /// write targets this branch. + #[serde(default)] + pub branch: Option<String>, + /// Snapshot id to read from (read queries only — rejected for a stored + /// mutation). Mutually exclusive with `branch`. + #[serde(default)] + pub snapshot: Option<String>, +} + +/// Response for `POST /queries/{name}`: the read envelope for a stored +/// read, or the mutation envelope for a stored mutation. Serialized +/// **untagged**, so the wire shape is exactly [`ReadOutput`] or +/// [`ChangeOutput`] — classification follows the stored query, not a +/// wrapper field. +#[derive(Debug, Serialize, ToSchema)] +#[serde(untagged)] +pub enum InvokeStoredQueryResponse { + Read(ReadOutput), + Change(ChangeOutput), +} + +/// The kind of a stored-query parameter, decomposed so a client (e.g. an +/// MCP server) can build a typed input schema with a closed `match` and +/// never re-parse omnigraph's type spelling. `bigint`/`date`/`datetime`/ +/// `blob` are carried as JSON strings on the wire: a 64-bit integer past +/// 2^53 loses precision as a JSON number, and Date/DateTime are ISO +/// strings, Blob a blob-URI string. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ToSchema)] +#[serde(rename_all = "snake_case")] +pub enum ParamKind { + String, + Bool, + Int, + #[serde(rename = "bigint")] + BigInt, + Float, + Date, + #[serde(rename = "datetime")] + DateTime, + Blob, + Vector, + List, +} + +/// One declared parameter of a stored query, projected for the catalog. +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct ParamDescriptor { + pub name: String, + pub kind: ParamKind, + /// Element kind when `kind == list` (always a scalar — the grammar + /// forbids lists of vectors or nested lists). + #[serde(skip_serializing_if = "Option::is_none")] + pub item_kind: Option<ParamKind>, + /// Dimension when `kind == vector`. + #[serde(skip_serializing_if = "Option::is_none")] + pub vector_dim: Option<u32>, + /// `false` → the caller must supply it; `true` → optional. + pub nullable: bool, +} + +/// One entry in the stored-query catalog (`GET /queries`). +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct QueryCatalogEntry { + /// Registry key / invoke path segment (`POST /queries/{name}`). + pub name: String, + /// MCP tool id (the `tool_name` override, else `name`). + pub tool_name: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub description: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub instruction: Option<String>, + /// `true` for a stored mutation → an MCP read-only hint of `false`. + pub mutation: bool, + pub params: Vec<ParamDescriptor>, +} + +/// Response for `GET /queries`: the `mcp.expose` subset of a graph's +/// stored-query registry, each with typed parameters. +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct QueriesCatalogOutput { + pub queries: Vec<QueryCatalogEntry>, +} + +/// Total map from a resolved scalar to its catalog kind. Exhaustive on +/// purpose: a new `ScalarType` is a compile error here until catalogued. +fn scalar_kind(scalar: ScalarType) -> ParamKind { + match scalar { + ScalarType::String => ParamKind::String, + ScalarType::Bool => ParamKind::Bool, + ScalarType::I32 | ScalarType::U32 => ParamKind::Int, + ScalarType::I64 | ScalarType::U64 => ParamKind::BigInt, + ScalarType::F32 | ScalarType::F64 => ParamKind::Float, + ScalarType::Date => ParamKind::Date, + ScalarType::DateTime => ParamKind::DateTime, + ScalarType::Blob => ParamKind::Blob, + ScalarType::Vector(_) => ParamKind::Vector, + } +} + +pub fn param_descriptor(param: &Param) -> ParamDescriptor { + match PropType::from_param_type_name(¶m.type_name, param.nullable) { + Some(pt) if pt.list => ParamDescriptor { + name: param.name.clone(), + kind: ParamKind::List, + item_kind: Some(scalar_kind(pt.scalar)), + vector_dim: None, + nullable: param.nullable, + }, + Some(pt) => { + let (kind, vector_dim) = match pt.scalar { + ScalarType::Vector(dim) => (ParamKind::Vector, Some(dim)), + other => (scalar_kind(other), None), + }; + ParamDescriptor { + name: param.name.clone(), + kind, + item_kind: None, + vector_dim, + nullable: param.nullable, + } + } + // Unreachable for a parsed query (every declared param type is + // grammatical); fall back to an opaque string so the field is still + // usable rather than dropped. + None => ParamDescriptor { + name: param.name.clone(), + kind: ParamKind::String, + item_kind: None, + vector_dim: None, + nullable: param.nullable, + }, + } +} + + +#[derive(Debug, Clone, Default, Serialize, Deserialize, ToSchema)] +pub struct SchemaApplyRequest { + /// Project schema in `.pg` source form. The diff against the current + /// schema produces the migration steps that will be applied. + #[schema( + example = "node Person {\n name: String @key\n age: I32?\n}\n\nedge Knows: Person -> Person" + )] + pub schema_source: String, + /// When true, promote every `DropMode::Soft` step in the plan to + /// `DropMode::Hard`, making the prior column data unreachable + /// after the apply. Matches the CLI's `--allow-data-loss` flag. + /// Defaults to `false` (drops remain reversible via time travel). + #[serde(default)] + pub allow_data_loss: bool, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct SchemaApplyOutput { + pub uri: String, + pub supported: bool, + pub applied: bool, + pub step_count: usize, + pub manifest_version: u64, + #[schema(value_type = Vec<Value>)] + pub steps: Vec<SchemaMigrationStep>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct SchemaOutput { + pub schema_source: String, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct IngestRequest { + /// Target branch. Defaults to `main`. Without `from`, the branch must + /// already exist — a missing branch is a 404, never an implicit fork. + pub branch: Option<String>, + /// Parent branch used to create `branch` if it does not exist. Branch + /// creation is opt-in by presence of this field; omit it to require an + /// existing branch. + pub from: Option<String>, + /// How existing rows are handled. Defaults to `merge`. + #[schema(value_type = Option<LoadModeSchema>)] + pub mode: Option<LoadMode>, + /// NDJSON payload: one record per line, each shaped + /// `{"type": "<TypeName>", "data": {...}}`. + #[schema( + example = "{\"type\": \"Person\", \"data\": {\"name\": \"Alice\", \"age\": 30}}\n{\"type\": \"Person\", \"data\": {\"name\": \"Bob\", \"age\": 25}}" + )] + pub data: String, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct ExportRequest { + /// Branch to export. Defaults to `main`. + pub branch: Option<String>, + /// Restrict the export to these node/edge type names. Empty exports all types. + #[serde(default)] + pub type_names: Vec<String>, + /// Restrict the export to these table keys. Empty exports all tables. + #[serde(default)] + pub table_keys: Vec<String>, +} + +#[derive(Debug, Clone, Deserialize, IntoParams)] +pub struct SnapshotQuery { + pub branch: Option<String>, +} + +#[derive(Debug, Clone, Deserialize, IntoParams)] +pub struct CommitListQuery { + pub branch: Option<String>, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct HealthOutput { + pub status: String, + pub version: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub source_version: Option<String>, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ToSchema)] +#[serde(rename_all = "snake_case")] +pub enum ErrorCode { + Unauthorized, + Forbidden, + BadRequest, + NotFound, + /// 405 Method Not Allowed — the route exists but the active server + /// mode doesn't serve this method (e.g. `GET /graphs` in single-graph + /// mode). Distinct from 404 so clients can tell "wrong context" from + /// "no such resource." + MethodNotAllowed, + Conflict, + /// 429 Too Many Requests — per-actor admission cap exceeded. + /// Clients should respect the `Retry-After` header. + TooManyRequests, + Internal, +} + +/// Structured details for a publisher-level OCC failure. Surfaces alongside +/// HTTP 409 when a write was rejected because the caller's pre-write view of +/// one table's manifest version was stale relative to the current head. The +/// expected/actual fields tell the client which table to refresh. +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct ManifestConflictOutput { + pub table_key: String, + pub expected: u64, + pub actual: u64, +} + +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct ErrorOutput { + pub error: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub code: Option<ErrorCode>, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub merge_conflicts: Vec<MergeConflictOutput>, + /// Set when the conflict is a publisher CAS rejection + /// (`ManifestConflictDetails::ExpectedVersionMismatch`). The caller's + /// pre-write view of `table_key` was at version `expected` but the + /// manifest is now at `actual`. Refresh and retry. + #[serde(skip_serializing_if = "Option::is_none")] + pub manifest_conflict: Option<ManifestConflictOutput>, +} + +pub fn snapshot_payload(branch: &str, snapshot: &Snapshot) -> SnapshotOutput { + let mut entries: Vec<_> = snapshot.entries().cloned().collect(); + entries.sort_by(|a, b| a.table_key.cmp(&b.table_key)); + let tables = entries + .iter() + .map(|entry| SnapshotTableOutput { + table_key: entry.table_key.clone(), + table_path: entry.table_path.clone(), + table_version: entry.table_version, + table_branch: entry.table_branch.clone(), + row_count: entry.row_count, + }) + .collect::<Vec<_>>(); + SnapshotOutput { + branch: branch.to_string(), + manifest_version: snapshot.version(), + tables, + } +} + +pub fn schema_apply_output(uri: &str, result: SchemaApplyResult) -> SchemaApplyOutput { + SchemaApplyOutput { + uri: uri.to_string(), + supported: result.supported, + applied: result.applied, + step_count: result.steps.len(), + manifest_version: result.manifest_version, + steps: result.steps, + } +} + +pub fn commit_output(commit: &GraphCommit) -> CommitOutput { + CommitOutput { + graph_commit_id: commit.graph_commit_id.clone(), + manifest_branch: commit.manifest_branch.clone(), + manifest_version: commit.manifest_version, + parent_commit_id: commit.parent_commit_id.clone(), + merged_parent_commit_id: commit.merged_parent_commit_id.clone(), + actor_id: commit.actor_id.clone(), + created_at: commit.created_at, + } +} + +pub fn read_output(query_name: String, target: &ReadTarget, result: QueryResult) -> ReadOutput { + let columns = result + .schema() + .fields() + .iter() + .map(|field| field.name().clone()) + .collect(); + ReadOutput { + query_name, + target: read_target_output(target), + row_count: result.num_rows(), + columns, + rows: result.to_rust_json(), + } +} + +pub fn ingest_output( + uri: &str, + result: &LoadResult, + mode: LoadMode, + actor_id: Option<String>, +) -> IngestOutput { + IngestOutput { + uri: uri.to_string(), + branch: result.branch.clone(), + base_branch: result.base_branch.clone(), + branch_created: result.branch_created, + mode, + tables: result + .to_ingest_tables() + .into_iter() + .map(|table| IngestTableOutput { + table_key: table.table_key, + rows_loaded: table.rows_loaded, + }) + .collect(), + actor_id, + } +} + +pub fn read_target_output(target: &ReadTarget) -> ReadTargetOutput { + match target { + ReadTarget::Branch(branch) => ReadTargetOutput { + branch: Some(branch.clone()), + snapshot: None, + }, + ReadTarget::Snapshot(snapshot) => ReadTargetOutput { + branch: None, + snapshot: Some(snapshot.as_str().to_string()), + }, + } +} + +// ─── MR-668 — management endpoint shapes ────────────────────────────────── + +/// One entry in the response from `GET /graphs`. Cluster operators +/// consume this list to discover which graphs the server is currently +/// serving. The shape is intentionally minimal — `graph_id` and `uri` +/// are the only fields a routing client needs. +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct GraphInfo { + pub graph_id: String, + pub uri: String, +} + +/// Response from `GET /graphs`. Lists every graph registered with the +/// server in alphabetical order by `graph_id` (sorted server-side so +/// clients get deterministic output across requests). +#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] +pub struct GraphListResponse { + pub graphs: Vec<GraphInfo>, +} diff --git a/crates/omnigraph-server/Cargo.toml b/crates/omnigraph-server/Cargo.toml index 614711e..a6a0717 100644 --- a/crates/omnigraph-server/Cargo.toml +++ b/crates/omnigraph-server/Cargo.toml @@ -22,6 +22,7 @@ aws = ["dep:aws-config", "dep:aws-sdk-secretsmanager"] omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.7.0" } omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.7.0" } omnigraph-policy = { path = "../omnigraph-policy", version = "0.7.0" } +omnigraph-api-types = { path = "../omnigraph-api-types", version = "0.7.0" } omnigraph-cluster = { path = "../omnigraph-cluster", version = "0.7.0" } axum = { workspace = true } clap = { workspace = true } diff --git a/crates/omnigraph-server/src/api.rs b/crates/omnigraph-server/src/api.rs index ff3cf67..cf0d604 100644 --- a/crates/omnigraph-server/src/api.rs +++ b/crates/omnigraph-server/src/api.rs @@ -1,452 +1,14 @@ -use omnigraph::db::{GraphCommit, MergeOutcome, ReadTarget, SchemaApplyResult, Snapshot}; -use omnigraph::error::{MergeConflict, MergeConflictKind}; -use omnigraph::loader::{LoadMode, LoadResult}; +//! HTTP wire DTOs. The types and their engine-result -> DTO mappings live +//! in the shared `omnigraph-api-types` crate (RFC-009 Phase 2) so the CLI +//! and server share one definition; re-exported here so every +//! `omnigraph_server::api::*` path (handlers, the OpenApi schema list, +//! CLI imports) keeps resolving unchanged. Only `query_catalog_entry` +//! stays — it maps the server's runtime `StoredQuery` (not a wire type) +//! into the shared `QueryCatalogEntry` DTO. + +pub use omnigraph_api_types::*; + use crate::queries::StoredQuery; -use omnigraph_compiler::SchemaMigrationStep; -use omnigraph_compiler::query::ast::Param; -use omnigraph_compiler::result::QueryResult; -use omnigraph_compiler::types::{PropType, ScalarType}; -use serde::{Deserialize, Serialize}; -use serde_json::Value; -use utoipa::{IntoParams, ToSchema}; - -/// Shadow enum for documenting [`LoadMode`] in the OpenAPI schema. -#[derive(ToSchema)] -#[schema(as = LoadMode)] -#[allow(dead_code)] -enum LoadModeSchema { - /// Overwrite existing data. - #[schema(rename = "overwrite")] - Overwrite, - /// Append to existing data. - #[schema(rename = "append")] - Append, - /// Merge by id key (upsert). - #[schema(rename = "merge")] - Merge, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct SnapshotTableOutput { - pub table_key: String, - pub table_path: String, - pub table_version: u64, - pub table_branch: Option<String>, - pub row_count: u64, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct SnapshotOutput { - pub branch: String, - pub manifest_version: u64, - pub tables: Vec<SnapshotTableOutput>, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct BranchCreateRequest { - /// Parent branch to fork from. Defaults to `main`. - pub from: Option<String>, - /// Name of the new branch. Must not already exist. - pub name: String, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct BranchCreateOutput { - pub uri: String, - pub from: String, - pub name: String, - pub actor_id: Option<String>, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct BranchListOutput { - pub branches: Vec<String>, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct BranchDeleteOutput { - pub uri: String, - pub name: String, - pub actor_id: Option<String>, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct BranchMergeRequest { - /// Source branch whose commits will be merged. - pub source: String, - /// Target branch that will receive the merge. Defaults to `main`. - pub target: Option<String>, -} - -#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ToSchema)] -#[serde(rename_all = "snake_case")] -pub enum BranchMergeOutcome { - AlreadyUpToDate, - FastForward, - Merged, -} - -impl From<MergeOutcome> for BranchMergeOutcome { - fn from(value: MergeOutcome) -> Self { - match value { - MergeOutcome::AlreadyUpToDate => Self::AlreadyUpToDate, - MergeOutcome::FastForward => Self::FastForward, - MergeOutcome::Merged => Self::Merged, - } - } -} - -impl BranchMergeOutcome { - pub fn as_str(self) -> &'static str { - match self { - Self::AlreadyUpToDate => "already_up_to_date", - Self::FastForward => "fast_forward", - Self::Merged => "merged", - } - } -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct BranchMergeOutput { - pub source: String, - pub target: String, - pub outcome: BranchMergeOutcome, - pub actor_id: Option<String>, -} - -#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ToSchema)] -#[serde(rename_all = "snake_case")] -pub enum MergeConflictKindOutput { - DivergentInsert, - DivergentUpdate, - DeleteVsUpdate, - OrphanEdge, - UniqueViolation, - CardinalityViolation, - ValueConstraintViolation, -} - -impl MergeConflictKindOutput { - pub fn as_str(self) -> &'static str { - match self { - Self::DivergentInsert => "divergent_insert", - Self::DivergentUpdate => "divergent_update", - Self::DeleteVsUpdate => "delete_vs_update", - Self::OrphanEdge => "orphan_edge", - Self::UniqueViolation => "unique_violation", - Self::CardinalityViolation => "cardinality_violation", - Self::ValueConstraintViolation => "value_constraint_violation", - } - } -} - -impl From<MergeConflictKind> for MergeConflictKindOutput { - fn from(value: MergeConflictKind) -> Self { - match value { - MergeConflictKind::DivergentInsert => Self::DivergentInsert, - MergeConflictKind::DivergentUpdate => Self::DivergentUpdate, - MergeConflictKind::DeleteVsUpdate => Self::DeleteVsUpdate, - MergeConflictKind::OrphanEdge => Self::OrphanEdge, - MergeConflictKind::UniqueViolation => Self::UniqueViolation, - MergeConflictKind::CardinalityViolation => Self::CardinalityViolation, - MergeConflictKind::ValueConstraintViolation => Self::ValueConstraintViolation, - } - } -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct MergeConflictOutput { - pub table_key: String, - pub row_id: Option<String>, - pub kind: MergeConflictKindOutput, - pub message: String, -} - -impl From<&MergeConflict> for MergeConflictOutput { - fn from(value: &MergeConflict) -> Self { - Self { - table_key: value.table_key.clone(), - row_id: value.row_id.clone(), - kind: value.kind.into(), - message: value.message.clone(), - } - } -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct ReadTargetOutput { - pub branch: Option<String>, - pub snapshot: Option<String>, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct ReadOutput { - pub query_name: String, - pub target: ReadTargetOutput, - pub row_count: usize, - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub columns: Vec<String>, - pub rows: Value, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct ChangeOutput { - pub branch: String, - pub query_name: String, - pub affected_nodes: usize, - pub affected_edges: usize, - pub actor_id: Option<String>, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct IngestTableOutput { - pub table_key: String, - pub rows_loaded: usize, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct IngestOutput { - pub uri: String, - pub branch: String, - /// Base branch a fork was requested from (the request's `from`), echoed - /// even when the branch already existed. `null` when `from` was absent. - pub base_branch: Option<String>, - pub branch_created: bool, - #[schema(value_type = LoadModeSchema)] - pub mode: LoadMode, - pub tables: Vec<IngestTableOutput>, - pub actor_id: Option<String>, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct CommitOutput { - pub graph_commit_id: String, - pub manifest_branch: Option<String>, - pub manifest_version: u64, - pub parent_commit_id: Option<String>, - pub merged_parent_commit_id: Option<String>, - pub actor_id: Option<String>, - /// Commit creation time as Unix epoch microseconds. - #[schema(example = 1714000000000000i64)] - pub created_at: i64, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct CommitListOutput { - pub commits: Vec<CommitOutput>, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct ReadRequest { - /// GQ query source. May declare one or more named queries; pick one with - /// `query_name` if there is more than one. - #[schema( - example = "query get_person($name: String) {\n match {\n $p: Person { name: $name }\n }\n return { $p.name, $p.age }\n}" - )] - pub query_source: String, - /// Name of the query to run when `query_source` declares multiple. Optional - /// when only one query is declared. - pub query_name: Option<String>, - /// JSON object whose keys match the query's declared parameters. - pub params: Option<Value>, - /// Branch to read from. Mutually exclusive with `snapshot`. Defaults to `main`. - pub branch: Option<String>, - /// Snapshot id to read from. Mutually exclusive with `branch`. - pub snapshot: Option<String>, -} - -/// Inline read-query request for `POST /query`. -/// -/// Friendlier-named alternative to [`ReadRequest`] for ad-hoc reads and -/// AI-agent integration. Mutations are rejected with 400 — use `POST -/// /mutate` (or its deprecated alias `POST /change`) for write queries. -/// Field names are deliberately short (`query`, `name`) to match the GQ -/// keyword and the CLI `-e` flag. -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct QueryRequest { - /// GQ read-query source. May declare one or more named queries; pick one - /// with `name` when more than one is declared. Mutations - /// (`insert`/`update`/`delete`) get 400 — use `POST /mutate` (or its - /// deprecated alias `POST /change`) instead. - #[schema(example = "query get_person($name: String) {\n match {\n $p: Person { name: $name }\n }\n return { $p.name, $p.age }\n}")] - pub query: String, - /// Name of the query to run when `query` declares multiple. Optional when - /// only one query is declared. - pub name: Option<String>, - /// JSON object whose keys match the query's declared parameters. - pub params: Option<Value>, - /// Branch to read from. Mutually exclusive with `snapshot`. Defaults to `main`. - pub branch: Option<String>, - /// Snapshot id to read from. Mutually exclusive with `branch`. - pub snapshot: Option<String>, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct ChangeRequest { - /// GQ mutation source containing `insert`, `update`, or `delete` statements. - /// May declare multiple named mutations; pick one with `name`. - /// - /// Accepts the legacy field name `query_source` as a deserialization alias. - #[schema( - example = "query insert_person($name: String, $age: I32) {\n insert Person { name: $name, age: $age }\n}" - )] - #[serde(alias = "query_source")] - pub query: String, - /// Name of the mutation to run when `query` declares multiple. - /// - /// Accepts the legacy field name `query_name` as a deserialization alias. - #[serde(default, alias = "query_name")] - pub name: Option<String>, - /// JSON object whose keys match the mutation's declared parameters. - #[serde(default)] - pub params: Option<Value>, - /// Target branch. Defaults to `main`. - #[serde(default)] - pub branch: Option<String>, -} - -/// Body for `POST /queries/{name}` — invokes the server-side stored query -/// named in the path. The query source and name come from the registry, -/// never the body; only the runtime inputs are supplied here. -#[derive(Debug, Clone, Default, Serialize, Deserialize, ToSchema)] -pub struct InvokeStoredQueryRequest { - /// JSON object whose keys match the stored query's declared parameters. - #[serde(default)] - pub params: Option<Value>, - /// Branch to run against. Defaults to `main`; for a stored mutation the - /// write targets this branch. - #[serde(default)] - pub branch: Option<String>, - /// Snapshot id to read from (read queries only — rejected for a stored - /// mutation). Mutually exclusive with `branch`. - #[serde(default)] - pub snapshot: Option<String>, -} - -/// Response for `POST /queries/{name}`: the read envelope for a stored -/// read, or the mutation envelope for a stored mutation. Serialized -/// **untagged**, so the wire shape is exactly [`ReadOutput`] or -/// [`ChangeOutput`] — classification follows the stored query, not a -/// wrapper field. -#[derive(Debug, Serialize, ToSchema)] -#[serde(untagged)] -pub enum InvokeStoredQueryResponse { - Read(ReadOutput), - Change(ChangeOutput), -} - -/// The kind of a stored-query parameter, decomposed so a client (e.g. an -/// MCP server) can build a typed input schema with a closed `match` and -/// never re-parse omnigraph's type spelling. `bigint`/`date`/`datetime`/ -/// `blob` are carried as JSON strings on the wire: a 64-bit integer past -/// 2^53 loses precision as a JSON number, and Date/DateTime are ISO -/// strings, Blob a blob-URI string. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ToSchema)] -#[serde(rename_all = "snake_case")] -pub enum ParamKind { - String, - Bool, - Int, - #[serde(rename = "bigint")] - BigInt, - Float, - Date, - #[serde(rename = "datetime")] - DateTime, - Blob, - Vector, - List, -} - -/// One declared parameter of a stored query, projected for the catalog. -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct ParamDescriptor { - pub name: String, - pub kind: ParamKind, - /// Element kind when `kind == list` (always a scalar — the grammar - /// forbids lists of vectors or nested lists). - #[serde(skip_serializing_if = "Option::is_none")] - pub item_kind: Option<ParamKind>, - /// Dimension when `kind == vector`. - #[serde(skip_serializing_if = "Option::is_none")] - pub vector_dim: Option<u32>, - /// `false` → the caller must supply it; `true` → optional. - pub nullable: bool, -} - -/// One entry in the stored-query catalog (`GET /queries`). -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct QueryCatalogEntry { - /// Registry key / invoke path segment (`POST /queries/{name}`). - pub name: String, - /// MCP tool id (the `tool_name` override, else `name`). - pub tool_name: String, - #[serde(skip_serializing_if = "Option::is_none")] - pub description: Option<String>, - #[serde(skip_serializing_if = "Option::is_none")] - pub instruction: Option<String>, - /// `true` for a stored mutation → an MCP read-only hint of `false`. - pub mutation: bool, - pub params: Vec<ParamDescriptor>, -} - -/// Response for `GET /queries`: the `mcp.expose` subset of a graph's -/// stored-query registry, each with typed parameters. -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct QueriesCatalogOutput { - pub queries: Vec<QueryCatalogEntry>, -} - -/// Total map from a resolved scalar to its catalog kind. Exhaustive on -/// purpose: a new `ScalarType` is a compile error here until catalogued. -fn scalar_kind(scalar: ScalarType) -> ParamKind { - match scalar { - ScalarType::String => ParamKind::String, - ScalarType::Bool => ParamKind::Bool, - ScalarType::I32 | ScalarType::U32 => ParamKind::Int, - ScalarType::I64 | ScalarType::U64 => ParamKind::BigInt, - ScalarType::F32 | ScalarType::F64 => ParamKind::Float, - ScalarType::Date => ParamKind::Date, - ScalarType::DateTime => ParamKind::DateTime, - ScalarType::Blob => ParamKind::Blob, - ScalarType::Vector(_) => ParamKind::Vector, - } -} - -fn param_descriptor(param: &Param) -> ParamDescriptor { - match PropType::from_param_type_name(¶m.type_name, param.nullable) { - Some(pt) if pt.list => ParamDescriptor { - name: param.name.clone(), - kind: ParamKind::List, - item_kind: Some(scalar_kind(pt.scalar)), - vector_dim: None, - nullable: param.nullable, - }, - Some(pt) => { - let (kind, vector_dim) = match pt.scalar { - ScalarType::Vector(dim) => (ParamKind::Vector, Some(dim)), - other => (scalar_kind(other), None), - }; - ParamDescriptor { - name: param.name.clone(), - kind, - item_kind: None, - vector_dim, - nullable: param.nullable, - } - } - // Unreachable for a parsed query (every declared param type is - // grammatical); fall back to an opaque string so the field is still - // usable rather than dropped. - None => ParamDescriptor { - name: param.name.clone(), - kind: ParamKind::String, - item_kind: None, - vector_dim: None, - nullable: param.nullable, - }, - } -} /// Project a loaded stored query into its catalog entry (typed params, /// MCP tool name, read/mutate flag, description/instruction). @@ -460,246 +22,3 @@ pub fn query_catalog_entry(query: &StoredQuery) -> QueryCatalogEntry { params: query.decl.params.iter().map(param_descriptor).collect(), } } - -#[derive(Debug, Clone, Default, Serialize, Deserialize, ToSchema)] -pub struct SchemaApplyRequest { - /// Project schema in `.pg` source form. The diff against the current - /// schema produces the migration steps that will be applied. - #[schema( - example = "node Person {\n name: String @key\n age: I32?\n}\n\nedge Knows: Person -> Person" - )] - pub schema_source: String, - /// When true, promote every `DropMode::Soft` step in the plan to - /// `DropMode::Hard`, making the prior column data unreachable - /// after the apply. Matches the CLI's `--allow-data-loss` flag. - /// Defaults to `false` (drops remain reversible via time travel). - #[serde(default)] - pub allow_data_loss: bool, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct SchemaApplyOutput { - pub uri: String, - pub supported: bool, - pub applied: bool, - pub step_count: usize, - pub manifest_version: u64, - #[schema(value_type = Vec<Value>)] - pub steps: Vec<SchemaMigrationStep>, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct SchemaOutput { - pub schema_source: String, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct IngestRequest { - /// Target branch. Defaults to `main`. Without `from`, the branch must - /// already exist — a missing branch is a 404, never an implicit fork. - pub branch: Option<String>, - /// Parent branch used to create `branch` if it does not exist. Branch - /// creation is opt-in by presence of this field; omit it to require an - /// existing branch. - pub from: Option<String>, - /// How existing rows are handled. Defaults to `merge`. - #[schema(value_type = Option<LoadModeSchema>)] - pub mode: Option<LoadMode>, - /// NDJSON payload: one record per line, each shaped - /// `{"type": "<TypeName>", "data": {...}}`. - #[schema( - example = "{\"type\": \"Person\", \"data\": {\"name\": \"Alice\", \"age\": 30}}\n{\"type\": \"Person\", \"data\": {\"name\": \"Bob\", \"age\": 25}}" - )] - pub data: String, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct ExportRequest { - /// Branch to export. Defaults to `main`. - pub branch: Option<String>, - /// Restrict the export to these node/edge type names. Empty exports all types. - #[serde(default)] - pub type_names: Vec<String>, - /// Restrict the export to these table keys. Empty exports all tables. - #[serde(default)] - pub table_keys: Vec<String>, -} - -#[derive(Debug, Clone, Deserialize, IntoParams)] -pub struct SnapshotQuery { - pub branch: Option<String>, -} - -#[derive(Debug, Clone, Deserialize, IntoParams)] -pub struct CommitListQuery { - pub branch: Option<String>, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct HealthOutput { - pub status: String, - pub version: String, - #[serde(skip_serializing_if = "Option::is_none")] - pub source_version: Option<String>, -} - -#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ToSchema)] -#[serde(rename_all = "snake_case")] -pub enum ErrorCode { - Unauthorized, - Forbidden, - BadRequest, - NotFound, - /// 405 Method Not Allowed — the route exists but the active server - /// mode doesn't serve this method (e.g. `GET /graphs` in single-graph - /// mode). Distinct from 404 so clients can tell "wrong context" from - /// "no such resource." - MethodNotAllowed, - Conflict, - /// 429 Too Many Requests — per-actor admission cap exceeded. - /// Clients should respect the `Retry-After` header. - TooManyRequests, - Internal, -} - -/// Structured details for a publisher-level OCC failure. Surfaces alongside -/// HTTP 409 when a write was rejected because the caller's pre-write view of -/// one table's manifest version was stale relative to the current head. The -/// expected/actual fields tell the client which table to refresh. -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct ManifestConflictOutput { - pub table_key: String, - pub expected: u64, - pub actual: u64, -} - -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct ErrorOutput { - pub error: String, - #[serde(skip_serializing_if = "Option::is_none")] - pub code: Option<ErrorCode>, - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub merge_conflicts: Vec<MergeConflictOutput>, - /// Set when the conflict is a publisher CAS rejection - /// (`ManifestConflictDetails::ExpectedVersionMismatch`). The caller's - /// pre-write view of `table_key` was at version `expected` but the - /// manifest is now at `actual`. Refresh and retry. - #[serde(skip_serializing_if = "Option::is_none")] - pub manifest_conflict: Option<ManifestConflictOutput>, -} - -pub fn snapshot_payload(branch: &str, snapshot: &Snapshot) -> SnapshotOutput { - let mut entries: Vec<_> = snapshot.entries().cloned().collect(); - entries.sort_by(|a, b| a.table_key.cmp(&b.table_key)); - let tables = entries - .iter() - .map(|entry| SnapshotTableOutput { - table_key: entry.table_key.clone(), - table_path: entry.table_path.clone(), - table_version: entry.table_version, - table_branch: entry.table_branch.clone(), - row_count: entry.row_count, - }) - .collect::<Vec<_>>(); - SnapshotOutput { - branch: branch.to_string(), - manifest_version: snapshot.version(), - tables, - } -} - -pub fn schema_apply_output(uri: &str, result: SchemaApplyResult) -> SchemaApplyOutput { - SchemaApplyOutput { - uri: uri.to_string(), - supported: result.supported, - applied: result.applied, - step_count: result.steps.len(), - manifest_version: result.manifest_version, - steps: result.steps, - } -} - -pub fn commit_output(commit: &GraphCommit) -> CommitOutput { - CommitOutput { - graph_commit_id: commit.graph_commit_id.clone(), - manifest_branch: commit.manifest_branch.clone(), - manifest_version: commit.manifest_version, - parent_commit_id: commit.parent_commit_id.clone(), - merged_parent_commit_id: commit.merged_parent_commit_id.clone(), - actor_id: commit.actor_id.clone(), - created_at: commit.created_at, - } -} - -pub fn read_output(query_name: String, target: &ReadTarget, result: QueryResult) -> ReadOutput { - let columns = result - .schema() - .fields() - .iter() - .map(|field| field.name().clone()) - .collect(); - ReadOutput { - query_name, - target: read_target_output(target), - row_count: result.num_rows(), - columns, - rows: result.to_rust_json(), - } -} - -pub fn ingest_output( - uri: &str, - result: &LoadResult, - mode: LoadMode, - actor_id: Option<String>, -) -> IngestOutput { - IngestOutput { - uri: uri.to_string(), - branch: result.branch.clone(), - base_branch: result.base_branch.clone(), - branch_created: result.branch_created, - mode, - tables: result - .to_ingest_tables() - .into_iter() - .map(|table| IngestTableOutput { - table_key: table.table_key, - rows_loaded: table.rows_loaded, - }) - .collect(), - actor_id, - } -} - -pub fn read_target_output(target: &ReadTarget) -> ReadTargetOutput { - match target { - ReadTarget::Branch(branch) => ReadTargetOutput { - branch: Some(branch.clone()), - snapshot: None, - }, - ReadTarget::Snapshot(snapshot) => ReadTargetOutput { - branch: None, - snapshot: Some(snapshot.as_str().to_string()), - }, - } -} - -// ─── MR-668 — management endpoint shapes ────────────────────────────────── - -/// One entry in the response from `GET /graphs`. Cluster operators -/// consume this list to discover which graphs the server is currently -/// serving. The shape is intentionally minimal — `graph_id` and `uri` -/// are the only fields a routing client needs. -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct GraphInfo { - pub graph_id: String, - pub uri: String, -} - -/// Response from `GET /graphs`. Lists every graph registered with the -/// server in alphabetical order by `graph_id` (sorted server-side so -/// clients get deterministic output across requests). -#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] -pub struct GraphListResponse { - pub graphs: Vec<GraphInfo>, -} From adbb2a181c36b3b775d899699e8225eb69d5f9dd Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Sat, 13 Jun 2026 17:05:32 +0300 Subject: [PATCH 143/165] refactor(cli): consume omnigraph-api-types directly; unify the load mapping MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The CLI's wire-DTO imports repoint from omnigraph_server::api to omnigraph-api-types (the server's other exports — queries registry, config types — still come from omnigraph-server). The local Load arm's inline LoadOutput hand-construction in main.rs is extracted into load_output_from_result next to load_output_from_tables in output.rs, so both '-> LoadOutput' mappings (engine LoadResult for local, wire IngestOutput for remote) live in one place. Deviation from the plan, with reason: LoadOutput stays CLI-side rather than moving into the wire-DTO crate — it is a rendered CLI output type, not an HTTP wire DTO, and its mapping consumes a CLI clap type (CliLoadMode). The shared crate stays strictly wire DTOs. Shapes unchanged: the parity matrix passes textually unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- Cargo.lock | 1 + crates/omnigraph-cli/Cargo.toml | 1 + crates/omnigraph-cli/src/main.rs | 16 +++------------- crates/omnigraph-cli/src/output.rs | 29 +++++++++++++++++++++++++++-- 4 files changed, 32 insertions(+), 15 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index bb95af9..994bb5e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4561,6 +4561,7 @@ dependencies = [ "color-eyre", "lance", "lance-index", + "omnigraph-api-types", "omnigraph-cluster", "omnigraph-compiler", "omnigraph-engine", diff --git a/crates/omnigraph-cli/Cargo.toml b/crates/omnigraph-cli/Cargo.toml index 1670fb2..e21b21e 100644 --- a/crates/omnigraph-cli/Cargo.toml +++ b/crates/omnigraph-cli/Cargo.toml @@ -15,6 +15,7 @@ path = "src/main.rs" [dependencies] omnigraph = { package = "omnigraph-engine", path = "../omnigraph", version = "0.7.0" } omnigraph-compiler = { path = "../omnigraph-compiler", version = "0.7.0" } +omnigraph-api-types = { path = "../omnigraph-api-types", version = "0.7.0" } omnigraph-cluster = { path = "../omnigraph-cluster", version = "0.7.0" } omnigraph-policy = { path = "../omnigraph-policy", version = "0.7.0" } omnigraph-server = { path = "../omnigraph-server", version = "0.7.0" } diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 8178e65..0b518eb 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -22,7 +22,7 @@ use omnigraph_compiler::{ QueryLintSeverity, QueryLintStatus, SchemaMigrationPlan, SchemaMigrationStep, build_catalog, json_params_to_param_map, lint_query_file, }; -use omnigraph_server::api::{ +use omnigraph_api_types::{ BranchCreateOutput, BranchCreateRequest, BranchDeleteOutput, BranchListOutput, BranchMergeOutput, BranchMergeRequest, ChangeOutput, CommitListOutput, CommitOutput, ErrorOutput, ExportRequest, GraphListResponse, IngestOutput, IngestRequest, ReadOutput, @@ -188,7 +188,7 @@ async fn main() -> Result<()> { bearer_token.as_deref(), ) .await?; - load_output_from_tables(&uri, &branch, mode, &output) + load_output_from_tables(&uri, &branch, mode.as_str(), &output) } else { let db = open_local_db_with_policy(&graph).await?; let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; @@ -202,17 +202,7 @@ async fn main() -> Result<()> { actor, ) .await?; - LoadOutput { - uri: uri.clone(), - branch: branch.clone(), - mode: mode.as_str(), - base_branch: result.base_branch.clone(), - branch_created: result.branch_created, - nodes_loaded: result.nodes_loaded.values().sum(), - edges_loaded: result.edges_loaded.values().sum(), - node_types_loaded: result.nodes_loaded.len(), - edge_types_loaded: result.edges_loaded.len(), - } + load_output_from_result(&uri, &branch, mode.as_str(), &result) }; if json { print_json(&payload)?; diff --git a/crates/omnigraph-cli/src/output.rs b/crates/omnigraph-cli/src/output.rs index 964307b..c6acd32 100644 --- a/crates/omnigraph-cli/src/output.rs +++ b/crates/omnigraph-cli/src/output.rs @@ -21,7 +21,7 @@ pub(crate) struct LoadOutput { pub(crate) fn load_output_from_tables( uri: &str, branch: &str, - mode: CliLoadMode, + mode: &'static str, output: &IngestOutput, ) -> LoadOutput { let mut nodes_loaded = 0; @@ -40,7 +40,7 @@ pub(crate) fn load_output_from_tables( LoadOutput { uri: uri.to_string(), branch: branch.to_string(), - mode: mode.as_str(), + mode, base_branch: output.base_branch.clone(), branch_created: output.branch_created, nodes_loaded, @@ -50,6 +50,31 @@ pub(crate) fn load_output_from_tables( } } +/// The local arm's twin of `load_output_from_tables`: build the same +/// `LoadOutput` from the engine `LoadResult` directly (the remote arm only +/// has the wire `IngestOutput`'s table list; the local arm has the full +/// result). Both load mappings live here, next to the struct — RFC-009 +/// Phase 2's "one place" for the `-> LoadOutput` mapping that used to fork +/// between this file and main.rs's inline construction. +pub(crate) fn load_output_from_result( + uri: &str, + branch: &str, + mode: &'static str, + result: &omnigraph::loader::LoadResult, +) -> LoadOutput { + LoadOutput { + uri: uri.to_string(), + branch: branch.to_string(), + mode, + base_branch: result.base_branch.clone(), + branch_created: result.branch_created, + nodes_loaded: result.nodes_loaded.values().sum(), + edges_loaded: result.edges_loaded.values().sum(), + node_types_loaded: result.nodes_loaded.len(), + edge_types_loaded: result.edges_loaded.len(), + } +} + #[derive(Debug, Serialize)] pub(crate) struct SchemaPlanOutput<'a> { pub(crate) uri: &'a str, From 3e2502c35e83f88b7351bbd0d4a8b63d4b803aff Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Sat, 13 Jun 2026 17:10:00 +0300 Subject: [PATCH 144/165] docs: omnigraph-api-types in the crate list; RFC-009 Phase 2 outcome Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- AGENTS.md | 2 +- docs/dev/rfc-009-unify-access-paths.md | 11 ++++++++++- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index d9e0c45..ae6e744 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,7 +17,7 @@ Tools that support `@`-imports (Claude Code) auto-include all three files via th `CLAUDE.md` is a symlink to this file — there is exactly one source of truth. Edit `AGENTS.md`. **Version surveyed:** 0.7.0 -**Workspace crates:** `omnigraph-compiler`, `omnigraph` (engine), `omnigraph-policy`, `omnigraph-cluster`, `omnigraph-cli`, `omnigraph-server` +**Workspace crates:** `omnigraph-compiler`, `omnigraph` (engine), `omnigraph-policy`, `omnigraph-api-types` (shared HTTP wire DTOs), `omnigraph-cluster`, `omnigraph-cli`, `omnigraph-server` **Storage substrate:** Lance 6.x (columnar, versioned, branchable) **License:** MIT **Toolchain:** Rust stable, edition 2024 diff --git a/docs/dev/rfc-009-unify-access-paths.md b/docs/dev/rfc-009-unify-access-paths.md index cada723..8b8251b 100644 --- a/docs/dev/rfc-009-unify-access-paths.md +++ b/docs/dev/rfc-009-unify-access-paths.md @@ -90,7 +90,7 @@ default-deny by design — the harness encodes this), and inline execution's unbound-param matches-all vs the invoke path's hard error is a cross-path asymmetry, filed as #207 and pinned (not repaired) by the matrix. -### Phase 2 — One wire-DTO crate +### Phase 2 — One wire-DTO crate *(landed)* Move the HTTP request/response types and the single `engine result → DTO` mapping per verb into a shared crate (working name `omnigraph-api-types`), @@ -122,6 +122,15 @@ neither axum nor the engine's internals. The engine crate does not depend on it — the `engine result → DTO` mapping lives in the shared crate (or the CLI/ server side), taking engine result types as input. +**Phase 2 outcome (landed):** `crates/omnigraph-api-types` holds the wire +DTOs + their `engine-result → DTO` mappings; `omnigraph-server::api` is a +`pub use` re-export (so `openapi.json` is byte-identical — the referee +passed with zero diff), and the CLI consumes the crate directly. One +deliberate refinement of the original sketch: `LoadOutput` is a rendered +CLI output type, not a wire DTO, so it stayed CLI-side — both its mappings +(local `LoadResult`, remote `IngestOutput`) now sit together in +`output.rs`. The parity matrix passed textually unchanged. + ### Phase 3 — `GraphClient` trait, two implementations ```text From 25d74d689d0801962f8897d9d05d4303974fc932 Mon Sep 17 00:00:00 2001 From: aaltshuler <andrew@collectivelab.io> Date: Sat, 13 Jun 2026 17:44:23 +0300 Subject: [PATCH 145/165] refactor(cli): GraphClient enum + read verbs (RFC-009 Phase 3a) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The embedded-vs-remote split gets one home: a GraphClient enum (Embedded { uri } | Remote { http, base_url, token }) with a resolve() factory that absorbs the shared preamble (apply_server_flag -> token -> URI/remoteness) and a verb method per command. The five uniform read forks — branch list, commit list, commit show, schema show, snapshot — collapse from per-command if-graph-is-remote else to one line each (main.rs: -113/+47). Behavior identical per verb (local reads still open WITHOUT policy, as today); the Phase-1 parity matrix is the referee and passes textually unchanged. Enum, not the RFC trait: only two variants ever, and inherent async methods avoid async_trait boxing and the apply_schema closure that is not object-safe (3b) — same one-body-two-impls collapse, less ceremony. Scope: the uniform reads only. The query verb (policy-open + operator- alias early-return + param merge) joins the write verbs in 3b; export/streaming and graphs-list in 3c, where the now-shared execute_*_remote/execute_* pairs get retired. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/client.rs | 197 +++++++++++++++++++++++++++++ crates/omnigraph-cli/src/main.rs | 160 +++++++---------------- 2 files changed, 244 insertions(+), 113 deletions(-) create mode 100644 crates/omnigraph-cli/src/client.rs diff --git a/crates/omnigraph-cli/src/client.rs b/crates/omnigraph-cli/src/client.rs new file mode 100644 index 0000000..feeaf16 --- /dev/null +++ b/crates/omnigraph-cli/src/client.rs @@ -0,0 +1,197 @@ +//! `GraphClient` — the one place the embedded-vs-remote split lives +//! (RFC-009 Phase 3). A CLI command body calls a verb method; the +//! enum routes to the engine (local URI) or HTTP (remote URI). The +//! 15 per-command `if graph.is_remote { … } else { … }` forks collapse +//! into two arms here. +//! +//! Phase 3a scope: the factory + the uniform read verbs (snapshot, +//! schema show, branch list, commit list/show — all of which open the +//! local engine WITHOUT policy today, preserved exactly). Write verbs +//! and the policy-bearing `query`/`mutate` arrive in 3b (the Embedded +//! variant will grow the policy context then); export + graphs-list in +//! 3c. Behavior is unchanged per verb — the Phase-1 parity matrix is the +//! referee and stays textually unchanged. +//! +//! Enum, not a trait (RFC sketch said "trait"): only two variants ever, +//! and inherent async methods sidestep `async_trait` boxing plus the +//! `apply_schema` catalog-validator closure that is not object-safe. +//! Same one-body-two-impls collapse, less ceremony. + +use reqwest::Method; +use color_eyre::Result; +use omnigraph::db::{Omnigraph, ReadTarget}; +use omnigraph_api_types::{ + BranchListOutput, CommitListOutput, CommitOutput, SchemaOutput, SnapshotOutput, commit_output, + snapshot_payload, +}; + +use crate::helpers::{ + apply_server_flag, build_http_client, is_remote_uri, remote_json, remote_url, + resolve_remote_bearer_token, +}; +use omnigraph_server::config::OmnigraphConfig; + +pub(crate) enum GraphClient { + /// Local engine at `uri`. Reads open the dataset per call (no policy + /// attached — matches today's read behavior; the write verbs in 3b + /// add a policy-bearing context). + Embedded { uri: String }, + /// Remote HTTP server. The actor is resolved server-side from the + /// token; the client never sets identity. + Remote { + http: reqwest::Client, + base_url: String, + token: Option<String>, + }, +} + +impl GraphClient { + /// Resolve the addressing (positional URI / `--target` / `--server`) + /// and credential once, then pick the variant by URI scheme — the + /// single branch point that replaces every per-command `is_remote` + /// fork. Mirrors the read verbs' current preamble (`resolve_uri` + /// path, not the policy-bearing `resolve_cli_graph`). + pub(crate) fn resolve( + config: &OmnigraphConfig, + server: Option<&str>, + graph: Option<&str>, + uri: Option<String>, + target: Option<&str>, + ) -> Result<Self> { + let uri = apply_server_flag(server, graph, uri, target)?; + let token = resolve_remote_bearer_token(config, uri.as_deref(), target)?; + let uri = crate::helpers::resolve_uri(config, uri, target)?; + if is_remote_uri(&uri) { + Ok(GraphClient::Remote { + http: build_http_client()?, + base_url: uri, + token, + }) + } else { + Ok(GraphClient::Embedded { uri }) + } + } + + pub(crate) async fn branch_list(&self) -> Result<BranchListOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + remote_json( + http, + Method::GET, + remote_url(base_url, "/branches"), + None, + token.as_deref(), + ) + .await + } + GraphClient::Embedded { uri } => { + let db = Omnigraph::open(uri).await?; + let mut branches = db.branch_list().await?; + branches.sort(); + Ok(BranchListOutput { branches }) + } + } + } + + pub(crate) async fn snapshot(&self, branch: &str) -> Result<SnapshotOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + remote_json( + http, + Method::GET, + format!("{}?branch={}", remote_url(base_url, "/snapshot"), branch), + None, + token.as_deref(), + ) + .await + } + GraphClient::Embedded { uri } => { + let db = Omnigraph::open(uri).await?; + let snapshot = db.snapshot_of(ReadTarget::branch(branch)).await?; + Ok(snapshot_payload(branch, &snapshot)) + } + } + } + + pub(crate) async fn schema_source(&self) -> Result<SchemaOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + remote_json( + http, + Method::GET, + remote_url(base_url, "/schema"), + None, + token.as_deref(), + ) + .await + } + GraphClient::Embedded { uri } => { + let db = Omnigraph::open(uri).await?; + Ok(SchemaOutput { + schema_source: db.schema_source().to_string(), + }) + } + } + } + + pub(crate) async fn list_commits(&self, branch: Option<&str>) -> Result<CommitListOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + let url = match branch { + Some(branch) => format!("{}?branch={}", remote_url(base_url, "/commits"), branch), + None => remote_url(base_url, "/commits"), + }; + remote_json(http, Method::GET, url, None, token.as_deref()).await + } + GraphClient::Embedded { uri } => { + let db = Omnigraph::open(uri).await?; + let commits = db + .list_commits(branch) + .await? + .iter() + .map(commit_output) + .collect::<Vec<_>>(); + Ok(CommitListOutput { commits }) + } + } + } + + pub(crate) async fn get_commit(&self, commit_id: &str) -> Result<CommitOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + remote_json( + http, + Method::GET, + remote_url(base_url, &format!("/commits/{commit_id}")), + None, + token.as_deref(), + ) + .await + } + GraphClient::Embedded { uri } => { + let db = Omnigraph::open(uri).await?; + Ok(commit_output(&db.get_commit(commit_id).await?)) + } + } + } +} diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 0b518eb..e979622 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -23,12 +23,11 @@ use omnigraph_compiler::{ json_params_to_param_map, lint_query_file, }; use omnigraph_api_types::{ - BranchCreateOutput, BranchCreateRequest, BranchDeleteOutput, BranchListOutput, - BranchMergeOutput, BranchMergeRequest, ChangeOutput, CommitListOutput, CommitOutput, + BranchCreateOutput, BranchCreateRequest, BranchDeleteOutput, + BranchMergeOutput, BranchMergeRequest, ChangeOutput, CommitOutput, ErrorOutput, ExportRequest, GraphListResponse, IngestOutput, IngestRequest, ReadOutput, - ReadRequest, SchemaApplyOutput, SchemaApplyRequest, SchemaOutput, SnapshotOutput, - SnapshotTableOutput, commit_output, ingest_output, read_output, schema_apply_output, - snapshot_payload, + ReadRequest, SchemaApplyOutput, SchemaApplyRequest, + SnapshotTableOutput, ingest_output, read_output, schema_apply_output, }; use omnigraph_server::queries::{QueryRegistry, check, format_check_breakages}; use omnigraph_server::{ @@ -50,6 +49,7 @@ use embed::{EmbedArgs, EmbedOutput, execute_embed}; use read_format::{ReadRenderOptions, render_read}; mod cli; +mod client; mod helpers; mod output; use cli::*; @@ -325,27 +325,14 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let graph = resolve_cli_graph(&config, uri, target.as_deref())?; - let uri = graph.uri.clone(); - let payload = if graph.is_remote { - remote_json::<BranchListOutput>( - &http_client, - Method::GET, - remote_url(&uri, "/branches"), - None, - bearer_token.as_deref(), - ) - .await? - } else { - let db = Omnigraph::open(&uri).await?; - let mut branches = db.branch_list().await?; - branches.sort(); - BranchListOutput { branches } - }; + let client = client::GraphClient::resolve( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + )?; + let payload = client.branch_list().await?; if json { print_json(&payload)?; } else { @@ -455,37 +442,18 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; - let commits = if is_remote_uri(&uri) { - remote_json::<CommitListOutput>( - &http_client, - Method::GET, - if let Some(branch) = branch.as_deref() { - format!("{}?branch={}", remote_url(&uri, "/commits"), branch) - } else { - remote_url(&uri, "/commits") - }, - None, - bearer_token.as_deref(), - ) - .await? - .commits - } else { - let db = Omnigraph::open(&uri).await?; - db.list_commits(branch.as_deref()) - .await? - .iter() - .map(commit_output) - .collect::<Vec<_>>() - }; + let client = client::GraphClient::resolve( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + )?; + let payload = client.list_commits(branch.as_deref()).await?; if json { - print_json(&CommitListOutput { commits })?; + print_json(&payload)?; } else { - print_commit_list_human(&commits); + print_commit_list_human(&payload.commits); } } CommitCommand::Show { @@ -496,24 +464,14 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; - let commit = if is_remote_uri(&uri) { - remote_json::<CommitOutput>( - &http_client, - Method::GET, - remote_url(&uri, &format!("/commits/{}", commit_id)), - None, - bearer_token.as_deref(), - ) - .await? - } else { - let db = Omnigraph::open(&uri).await?; - commit_output(&db.get_commit(&commit_id).await?) - }; + let client = client::GraphClient::resolve( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + )?; + let commit = client.get_commit(&commit_id).await?; if json { print_json(&commit)?; } else { @@ -620,26 +578,14 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; - let output = if is_remote_uri(&uri) { - remote_json::<SchemaOutput>( - &http_client, - Method::GET, - remote_url(&uri, "/schema"), - None, - bearer_token.as_deref(), - ) - .await? - } else { - let db = Omnigraph::open(&uri).await?; - SchemaOutput { - schema_source: db.schema_source().to_string(), - } - }; + let client = client::GraphClient::resolve( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + )?; + let output = client.schema_source().await?; if json { print_json(&output)?; } else { @@ -686,27 +632,15 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; + let client = client::GraphClient::resolve( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + )?; let branch = resolve_branch(&config, branch, None, "main"); - let payload = if is_remote_uri(&uri) { - remote_json::<SnapshotOutput>( - &http_client, - Method::GET, - format!("{}?branch={}", remote_url(&uri, "/snapshot"), branch), - None, - bearer_token.as_deref(), - ) - .await? - } else { - let db = Omnigraph::open(&uri).await?; - let snapshot = db.snapshot_of(ReadTarget::branch(branch.as_str())).await?; - snapshot_payload(&branch, &snapshot) - }; - + let payload = client.snapshot(&branch).await?; if json { print_json(&payload)?; } else { From 81b66f9427953f4fc736cead9d4986421aad91c5 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sat, 13 Jun 2026 19:23:41 +0300 Subject: [PATCH 146/165] ci: run Test Workspace only on main, not on pull requests (#212) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The full workspace + failpoints suite was the slowest PR gate (~15min warm, up to the 75min cold ceiling) and dominated PR turnaround. Gate the `test` job with `if: github.event_name != 'pull_request'` so it runs only on push to `main` (post-merge), on `v*` tags, and on manual `workflow_dispatch`. `RustFS S3 Integration` needs `test`, so it becomes push-/dispatch-only by the same cascade. Drop `Test Workspace` from the required-check list in branch-protection.json: a required context that never reports on PRs (the job no longer runs there) would leave every PR permanently pending — the job-never-reports trap the policy already documents. Trade-off accepted deliberately (chosen by the maintainer): a regression the suite would catch now lands on `main` and reddens the post-merge run instead of being blocked pre-merge, so `main` can briefly break. Mitigations documented in ci.md: run `cargo test --workspace --locked` locally before merging non-trivial changes (or trigger the workflow on your branch via workflow_dispatch), and regenerate openapi.json locally for server/API changes (the auto-regen step lived in the now-PR-skipped test job). The fast PR gates remain: Classify Changes, Check AGENTS.md Links, the AWS-feature build/test, and the two CODEOWNERS checks. NOTE: an admin must run ./scripts/apply-branch-protection.sh after this merges, or GitHub keeps requiring the now-unreported Test Workspace context. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- .github/branch-protection.json | 1 - .github/workflows/ci.yml | 20 ++++++++++++++++++++ docs/dev/branch-protection.md | 2 +- docs/dev/ci.md | 3 +++ 4 files changed, 24 insertions(+), 2 deletions(-) diff --git a/.github/branch-protection.json b/.github/branch-protection.json index c039e32..aa1ab19 100644 --- a/.github/branch-protection.json +++ b/.github/branch-protection.json @@ -5,7 +5,6 @@ "contexts": [ "Classify Changes", "Check AGENTS.md Links", - "Test Workspace", "Test omnigraph-server --features aws", "CODEOWNERS matches source", "CODEOWNERS not hand-edited" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 56ef3e3..fca08da 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -128,6 +128,23 @@ jobs: test: name: Test Workspace needs: classify_changes + # PR latency: the full workspace + failpoints build/test is the slowest + # gate (~15min warm, up to the 75min ceiling cold) and dominated PR + # turnaround. It now runs only on push to `main` (post-merge), on tags, + # and on manual `workflow_dispatch` — NOT on pull_request. Trade-off + # accepted deliberately: a regression is caught on the `main` run after + # merge rather than before it, so `main` can briefly go red. Mitigations: + # (1) `Test Workspace` is removed from required PR checks in + # `.github/branch-protection.json` (a required check that never + # reports would leave every PR permanently pending); + # (2) run the full suite locally before merging risky changes + # (`cargo test --workspace --locked`), or trigger this workflow via + # the Actions "Run workflow" button (workflow_dispatch) on your branch; + # (3) openapi.json is no longer auto-regenerated on PRs (that step lived + # here) — regenerate it locally for server/API changes + # (`OMNIGRAPH_UPDATE_OPENAPI=1 cargo test -p omnigraph-server --test openapi`) + # or the strict drift check fails the post-merge `main` run. + if: github.event_name != 'pull_request' runs-on: ubuntu-latest # 75, not 45: a cold rust-cache (every Cargo.lock change) costs a full # workspace + failpoints-feature build on a 2-core runner, which now @@ -274,6 +291,9 @@ jobs: rustfs_integration: name: RustFS S3 Integration + # `needs: test` means this is push-/dispatch-only too: on pull_request the + # `test` job is skipped, so this dependent is skipped with it. S3 + # integration runs post-merge on `main`, alongside the workspace suite. needs: - classify_changes - test diff --git a/docs/dev/branch-protection.md b/docs/dev/branch-protection.md index 2b6cc37..1d1c094 100644 --- a/docs/dev/branch-protection.md +++ b/docs/dev/branch-protection.md @@ -8,7 +8,7 @@ This page explains what the policy says and how to change it. | Setting | Value | Why | |---|---|---| -| **Required status checks (strict)** | `Classify Changes`, `Check AGENTS.md Links`, `Test Workspace`, `Test omnigraph-server --features aws`, `CODEOWNERS matches source`, `CODEOWNERS not hand-edited` | Every PR must pass workspace tests, AGENTS.md link integrity, and the CODEOWNERS hygiene checks. The two CODEOWNERS contexts must equal the job `name:` values in `.github/workflows/codeowners.yml` **verbatim** — a context naming a job that never reports (the old `CODEOWNERS / drift` used the job *id*, and the job was path-filtered) leaves every PR permanently pending and forces admin overrides. `strict: true` requires the branch to be up-to-date with `main` before merge. | +| **Required status checks (strict)** | `Classify Changes`, `Check AGENTS.md Links`, `Test omnigraph-server --features aws`, `CODEOWNERS matches source`, `CODEOWNERS not hand-edited` | Every PR must pass the AWS-feature build/test, AGENTS.md link integrity, and the CODEOWNERS hygiene checks. **`Test Workspace` is deliberately NOT required** — it runs only on push to `main` (post-merge), tags, and manual `workflow_dispatch`, to keep PR turnaround fast (it was the ~15min+ slow gate). It is therefore *not* listed here: a required check that never reports on PRs (the `test` job is `if: github.event_name != 'pull_request'`) would leave every PR permanently pending — the same job-never-reports trap the CODEOWNERS contexts call out below. The trade-off (a regression lands on `main` and is caught by the post-merge run, so `main` can briefly go red) and its mitigations are documented in [ci.md](ci.md). The two CODEOWNERS contexts must equal the job `name:` values in `.github/workflows/codeowners.yml` **verbatim** — a context naming a job that never reports (the old `CODEOWNERS / drift` used the job *id*, and the job was path-filtered) leaves every PR permanently pending and forces admin overrides. `strict: true` requires the branch to be up-to-date with `main` before merge. | | **Required approving reviews** | `1` | At least one reviewer. With a 2-person team, going higher would block all merges when one person is unavailable. | | **Require code-owner reviews** | `true` | The reviewer must be a code owner per `.github/CODEOWNERS`. This is what makes the codeowners chassis enforced. | | **Dismiss stale reviews on new commits** | `true` | A push after approval invalidates the prior review. Prevents the "approve, then sneak in unreviewed changes" pattern. | diff --git a/docs/dev/ci.md b/docs/dev/ci.md index 1124cb4..2e80f40 100644 --- a/docs/dev/ci.md +++ b/docs/dev/ci.md @@ -3,6 +3,9 @@ `.github/workflows/`: - **ci.yml**: text-only changes skip; otherwise `cargo test --workspace --locked` on ubuntu-latest with protobuf compiler. OpenAPI-drift check that auto-commits the regenerated `openapi.json` for same-repository PRs. Also runs the AGENTS.md cross-link integrity check (`scripts/check-agents-md.sh`). + - **`Test Workspace` does not run on pull requests.** The job is gated `if: github.event_name != 'pull_request'`, so the full workspace + failpoints suite runs only on push to `main` (post-merge), on `v*` tags, and on manual `workflow_dispatch`. This was a deliberate PR-latency trade-off — it was the slowest gate (~15min warm, up to the 75min cold ceiling). `RustFS S3 Integration` `needs: test`, so it is push-/dispatch-only for the same reason. The fast PR gates remain: `Classify Changes`, `Check AGENTS.md Links`, `Test omnigraph-server --features aws`, and the two CODEOWNERS checks. `Test Workspace` is correspondingly **not** in the required-check list (`.github/branch-protection.json`); see [branch-protection.md](branch-protection.md). + - **Consequences to internalize:** (1) a regression that the suite would catch now lands on `main` and turns the post-merge run red, rather than being blocked pre-merge — `main` can briefly break, so run `cargo test --workspace --locked` locally before merging anything non-trivial, or trigger this workflow on your branch via the Actions "Run workflow" button. (2) `openapi.json` is no longer auto-regenerated on PRs (that step is inside the `test` job); for server/API changes, regenerate it locally with `OMNIGRAPH_UPDATE_OPENAPI=1 cargo test -p omnigraph-server --test openapi` and commit it, or the strict drift check fails the post-merge `main` run. + - **Applying this policy:** removing `Test Workspace` from the JSON is inert until an admin runs `./scripts/apply-branch-protection.sh`. **Run it immediately after this change merges** — until then GitHub still requires a `Test Workspace` context that no longer reports on PRs, which leaves every open PR permanently pending (the job-never-reports trap). - **AWS feature build job**: `cargo build/test -p omnigraph-server --features aws` on ubuntu-latest. - **Windows binary build job**: `cargo build --release --locked -p omnigraph-cli -p omnigraph-server` on windows-latest with smoke checks for `omnigraph.exe version`, `omnigraph-server.exe --help`, and PowerShell installer syntax. - **RustFS S3 integration**: spins up RustFS in Docker, runs `s3_storage`, `server_opens_s3_graph_directly_and_serves_snapshot_and_read`, and `local_cli_s3_end_to_end_init_load_read_flow`. From d32c1ac191fbf88ef0fc249b1c6aa393cbd8cef1 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sat, 13 Jun 2026 19:25:57 +0300 Subject: [PATCH 147/165] refactor(cli): collapse write/query forks onto GraphClient (RFC-009 Phase 3b) (#211) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 3a put the GraphClient enum in place and collapsed the five uniform read forks. 3b folds the remaining data-plane forks onto the same enum: load, ingest, mutate, query, branch create/delete/merge, and schema apply. The wrinkle 3a deferred was the local policy attachment. Reads and query open the local engine without a policy; writes open through open_local_db_with_policy and attribute a resolved actor. So the Embedded variant grows an optional policy context (graph/actor) filled by a second factory, resolve_with_policy; resolve() leaves it empty. open_embedded picks the open path from whether the context is present, preserving both of today's behaviors exactly. query still uses resolve() (no policy), as the read path did. apply_schema takes the catalog-validator closure as impl FnOnce(&Catalog) — the embedded arm runs it inside apply_schema_as_with_catalog_check, the remote arm ignores it (the server runs its own check). That non-object-safe closure is why GraphClient is an enum, not a trait. The stored-query registry is still built caller-side and only for the local path. load and ingest stay separate methods: same operation, but load surfaces the CLI LoadOutput (two distinct per-arm mappings preserved) while ingest surfaces the wire IngestOutput. The now-fully-dead execute_read/ execute_read_remote and execute_change/execute_change_remote pairs are retired (legacy_change_request_body stays — client.rs uses it); the export pair remains for 3c. The Phase-1 parity matrix is unchanged and green; full workspace tests pass. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --- crates/omnigraph-cli/src/client.rs | 465 ++++++++++++++++++++++++++-- crates/omnigraph-cli/src/helpers.rs | 95 ------ crates/omnigraph-cli/src/main.rs | 349 ++++++--------------- 3 files changed, 541 insertions(+), 368 deletions(-) diff --git a/crates/omnigraph-cli/src/client.rs b/crates/omnigraph-cli/src/client.rs index feeaf16..02daae4 100644 --- a/crates/omnigraph-cli/src/client.rs +++ b/crates/omnigraph-cli/src/client.rs @@ -4,38 +4,56 @@ //! 15 per-command `if graph.is_remote { … } else { … }` forks collapse //! into two arms here. //! -//! Phase 3a scope: the factory + the uniform read verbs (snapshot, -//! schema show, branch list, commit list/show — all of which open the -//! local engine WITHOUT policy today, preserved exactly). Write verbs -//! and the policy-bearing `query`/`mutate` arrive in 3b (the Embedded -//! variant will grow the policy context then); export + graphs-list in -//! 3c. Behavior is unchanged per verb — the Phase-1 parity matrix is the -//! referee and stays textually unchanged. +//! Phase 3a put the factory + the uniform read verbs in place. Phase 3b +//! adds the data-plane writes (`load`/`ingest`/`mutate`/`branch_*`/ +//! `apply_schema`) and `query`. The wrinkle 3a deferred: writes open the +//! local engine WITH policy (`open_local_db_with_policy`) and carry a +//! resolved actor, while reads/`query` open WITHOUT policy. So the +//! `Embedded` variant grows an optional policy context (`graph`/`actor`) +//! and a second factory (`resolve_with_policy`) fills it; `resolve()` +//! leaves it empty. The open path picks itself from whether `graph` is +//! set, preserving today's two behaviors exactly. Export + graphs-list +//! land in 3c. Behavior is unchanged per verb — the Phase-1 parity matrix +//! is the referee and stays textually unchanged. //! //! Enum, not a trait (RFC sketch said "trait"): only two variants ever, //! and inherent async methods sidestep `async_trait` boxing plus the //! `apply_schema` catalog-validator closure that is not object-safe. //! Same one-body-two-impls collapse, less ceremony. -use reqwest::Method; use color_eyre::Result; use omnigraph::db::{Omnigraph, ReadTarget}; use omnigraph_api_types::{ - BranchListOutput, CommitListOutput, CommitOutput, SchemaOutput, SnapshotOutput, commit_output, + BranchCreateOutput, BranchCreateRequest, BranchDeleteOutput, BranchListOutput, + BranchMergeOutput, BranchMergeRequest, ChangeOutput, CommitListOutput, CommitOutput, + IngestOutput, IngestRequest, ReadOutput, ReadRequest, SchemaApplyOutput, SchemaApplyRequest, + SchemaOutput, SnapshotOutput, commit_output, ingest_output, read_output, schema_apply_output, snapshot_payload, }; +use omnigraph_compiler::catalog::Catalog; +use reqwest::Method; +use serde_json::Value; +use crate::cli::CliLoadMode; use crate::helpers::{ - apply_server_flag, build_http_client, is_remote_uri, remote_json, remote_url, - resolve_remote_bearer_token, + ResolvedCliGraph, apply_server_flag, build_http_client, is_remote_uri, + legacy_change_request_body, open_local_db_with_policy, query_params_from_json, remote_branch_url, + remote_json, remote_url, resolve_cli_actor, resolve_cli_graph, resolve_remote_bearer_token, + select_named_query, }; +use crate::output::{LoadOutput, load_output_from_result, load_output_from_tables}; use omnigraph_server::config::OmnigraphConfig; pub(crate) enum GraphClient { - /// Local engine at `uri`. Reads open the dataset per call (no policy - /// attached — matches today's read behavior; the write verbs in 3b - /// add a policy-bearing context). - Embedded { uri: String }, + /// Local engine at `uri`. Reads (`resolve()`) leave `graph`/`actor` + /// empty and open without policy; writes (`resolve_with_policy()`) + /// fill them, opening through `open_local_db_with_policy` and + /// attributing the resolved actor. + Embedded { + uri: String, + graph: Option<ResolvedCliGraph>, + actor: Option<String>, + }, /// Remote HTTP server. The actor is resolved server-side from the /// token; the client never sets identity. Remote { @@ -50,7 +68,8 @@ impl GraphClient { /// and credential once, then pick the variant by URI scheme — the /// single branch point that replaces every per-command `is_remote` /// fork. Mirrors the read verbs' current preamble (`resolve_uri` - /// path, not the policy-bearing `resolve_cli_graph`). + /// path, not the policy-bearing `resolve_cli_graph`). Used by reads + /// and `query` (which opens without policy, like the reads). pub(crate) fn resolve( config: &OmnigraphConfig, server: Option<&str>, @@ -68,7 +87,76 @@ impl GraphClient { token, }) } else { - Ok(GraphClient::Embedded { uri }) + Ok(GraphClient::Embedded { + uri, + graph: None, + actor: None, + }) + } + } + + /// Write-path factory: the same addressing/credential resolution as + /// `resolve()`, but through the stricter `resolve_cli_graph` (which + /// carries `policy_file`/`graph_id`/`selected`), and with the actor + /// resolved up front. The embedded arm then opens WITH policy. The + /// resolution order matches the write arms exactly: server flag → + /// bearer token → graph. + pub(crate) fn resolve_with_policy( + config: &OmnigraphConfig, + server: Option<&str>, + graph: Option<&str>, + uri: Option<String>, + target: Option<&str>, + cli_as: Option<&str>, + ) -> Result<Self> { + let uri = apply_server_flag(server, graph, uri, target)?; + let token = resolve_remote_bearer_token(config, uri.as_deref(), target)?; + let resolved = resolve_cli_graph(config, uri, target)?; + if resolved.is_remote { + Ok(GraphClient::Remote { + http: build_http_client()?, + base_url: resolved.uri, + token, + }) + } else { + let actor = resolve_cli_actor(cli_as, config)?; + Ok(GraphClient::Embedded { + uri: resolved.uri.clone(), + graph: Some(resolved), + actor, + }) + } + } + + /// The graph URI (local path / remote base URL) this client addresses. + pub(crate) fn uri(&self) -> &str { + match self { + GraphClient::Embedded { uri, .. } => uri, + GraphClient::Remote { base_url, .. } => base_url, + } + } + + /// The selected graph name, when a policy-bearing embedded client was + /// resolved against a named graph. `None` for remote and for reads. + pub(crate) fn selected(&self) -> Option<&str> { + match self { + GraphClient::Embedded { graph, .. } => graph.as_ref().and_then(ResolvedCliGraph::selected), + GraphClient::Remote { .. } => None, + } + } + + pub(crate) fn is_remote(&self) -> bool { + matches!(self, GraphClient::Remote { .. }) + } + + /// Open the local engine the way the resolved client demands: with + /// policy when a `graph` context is present (write path), bare + /// otherwise (read/`query` path). Captures today's two open paths in + /// one place so each verb stays a single match arm. + async fn open_embedded(uri: &str, graph: &Option<ResolvedCliGraph>) -> Result<Omnigraph> { + match graph { + Some(graph) => open_local_db_with_policy(graph).await, + None => Ok(Omnigraph::open(uri).await?), } } @@ -88,7 +176,7 @@ impl GraphClient { ) .await } - GraphClient::Embedded { uri } => { + GraphClient::Embedded { uri, .. } => { let db = Omnigraph::open(uri).await?; let mut branches = db.branch_list().await?; branches.sort(); @@ -113,7 +201,7 @@ impl GraphClient { ) .await } - GraphClient::Embedded { uri } => { + GraphClient::Embedded { uri, .. } => { let db = Omnigraph::open(uri).await?; let snapshot = db.snapshot_of(ReadTarget::branch(branch)).await?; Ok(snapshot_payload(branch, &snapshot)) @@ -137,7 +225,7 @@ impl GraphClient { ) .await } - GraphClient::Embedded { uri } => { + GraphClient::Embedded { uri, .. } => { let db = Omnigraph::open(uri).await?; Ok(SchemaOutput { schema_source: db.schema_source().to_string(), @@ -159,7 +247,7 @@ impl GraphClient { }; remote_json(http, Method::GET, url, None, token.as_deref()).await } - GraphClient::Embedded { uri } => { + GraphClient::Embedded { uri, .. } => { let db = Omnigraph::open(uri).await?; let commits = db .list_commits(branch) @@ -188,10 +276,343 @@ impl GraphClient { ) .await } - GraphClient::Embedded { uri } => { + GraphClient::Embedded { uri, .. } => { let db = Omnigraph::open(uri).await?; Ok(commit_output(&db.get_commit(commit_id).await?)) } } } + + /// `load` — bulk-load `data` (a file path) onto `branch`, forking from + /// `from` if missing. Returns the CLI `LoadOutput`; each arm keeps its + /// own mapping (remote sums the wire `IngestOutput.tables`, embedded + /// reads the richer `LoadResult` directly) — preserved exactly. + pub(crate) async fn load( + &self, + branch: &str, + from: Option<&str>, + data: &str, + mode: CliLoadMode, + ) -> Result<LoadOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + let data = std::fs::read_to_string(data)?; + let output = remote_json::<IngestOutput>( + http, + Method::POST, + remote_url(base_url, "/ingest"), + Some(serde_json::to_value(IngestRequest { + branch: Some(branch.to_string()), + from: from.map(ToOwned::to_owned), + mode: Some(mode.into()), + data, + })?), + token.as_deref(), + ) + .await?; + Ok(load_output_from_tables(base_url, branch, mode.as_str(), &output)) + } + GraphClient::Embedded { uri, graph, actor } => { + let db = Self::open_embedded(uri, graph).await?; + let result = db + .load_file_as(branch, from, data, mode.into(), actor.as_deref()) + .await?; + Ok(load_output_from_result(uri, branch, mode.as_str(), &result)) + } + } + } + + /// `ingest` — the deprecated alias of `load`. Same operation, but the + /// surfaced shape is the wire `IngestOutput` (printed by + /// `print_ingest_human`), so it is its own method. The embedded arm + /// echoes `actor_id: None` in the output exactly as the legacy arm did + /// (the actor is still attributed on the commit via `load_file_as`). + pub(crate) async fn ingest( + &self, + branch: &str, + from: &str, + data: &str, + mode: CliLoadMode, + ) -> Result<IngestOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + let data = std::fs::read_to_string(data)?; + remote_json( + http, + Method::POST, + remote_url(base_url, "/ingest"), + Some(serde_json::to_value(IngestRequest { + branch: Some(branch.to_string()), + from: Some(from.to_string()), + mode: Some(mode.into()), + data, + })?), + token.as_deref(), + ) + .await + } + GraphClient::Embedded { uri, graph, actor } => { + let db = Self::open_embedded(uri, graph).await?; + let result = db + .load_file_as(branch, Some(from), data, mode.into(), actor.as_deref()) + .await?; + Ok(ingest_output(uri, &result, mode.into(), None)) + } + } + } + + /// `mutate` — run a change query against `branch`. Folds + /// `execute_change` / `execute_change_remote` + the legacy request body. + pub(crate) async fn mutate( + &self, + branch: &str, + query_source: &str, + query_name: Option<&str>, + params_json: Option<&Value>, + ) -> Result<ChangeOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + remote_json( + http, + Method::POST, + remote_url(base_url, "/change"), + Some(legacy_change_request_body( + query_source, + query_name, + branch, + params_json, + )), + token.as_deref(), + ) + .await + } + GraphClient::Embedded { uri, graph, actor } => { + let (selected_name, query_params) = select_named_query(query_source, query_name)?; + let params = query_params_from_json(&query_params, params_json)?; + let db = Self::open_embedded(uri, graph).await?; + let actor = actor.as_deref(); + let result = db + .mutate_as(branch, query_source, &selected_name, ¶ms, actor) + .await?; + Ok(ChangeOutput { + branch: branch.to_string(), + query_name: selected_name, + affected_nodes: result.affected_nodes, + affected_edges: result.affected_edges, + actor_id: actor.map(String::from), + }) + } + } + } + + /// `query` — run a read query against `target`. Folds `execute_read` / + /// `execute_read_remote`; the embedded arm opens WITHOUT policy (reads + /// never attach one), so this verb resolves via `resolve()`. + pub(crate) async fn query( + &self, + target: ReadTarget, + query_source: &str, + query_name: Option<&str>, + params_json: Option<&Value>, + ) -> Result<ReadOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + let (branch, snapshot) = match &target { + ReadTarget::Branch(branch) => (Some(branch.clone()), None), + ReadTarget::Snapshot(snapshot) => (None, Some(snapshot.as_str().to_string())), + }; + remote_json( + http, + Method::POST, + remote_url(base_url, "/read"), + Some(serde_json::to_value(ReadRequest { + query_source: query_source.to_string(), + query_name: query_name.map(ToOwned::to_owned), + params: params_json.cloned(), + branch, + snapshot, + })?), + token.as_deref(), + ) + .await + } + GraphClient::Embedded { uri, graph, .. } => { + let (selected_name, query_params) = select_named_query(query_source, query_name)?; + let params = query_params_from_json(&query_params, params_json)?; + let db = Self::open_embedded(uri, graph).await?; + let result = db + .query(target.clone(), query_source, &selected_name, ¶ms) + .await?; + Ok(read_output(selected_name, &target, result)) + } + } + } + + pub(crate) async fn branch_create_from( + &self, + from: &str, + name: &str, + ) -> Result<BranchCreateOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + remote_json( + http, + Method::POST, + remote_url(base_url, "/branches"), + Some(serde_json::to_value(BranchCreateRequest { + from: Some(from.to_string()), + name: name.to_string(), + })?), + token.as_deref(), + ) + .await + } + GraphClient::Embedded { uri, graph, actor } => { + let db = Self::open_embedded(uri, graph).await?; + let actor = actor.as_deref(); + db.branch_create_from_as(ReadTarget::branch(from), name, actor) + .await?; + Ok(BranchCreateOutput { + uri: uri.clone(), + from: from.to_string(), + name: name.to_string(), + actor_id: actor.map(String::from), + }) + } + } + } + + pub(crate) async fn branch_delete(&self, name: &str) -> Result<BranchDeleteOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + remote_json( + http, + Method::DELETE, + remote_branch_url(base_url, name)?, + None, + token.as_deref(), + ) + .await + } + GraphClient::Embedded { uri, graph, actor } => { + let db = Self::open_embedded(uri, graph).await?; + let actor = actor.as_deref(); + db.branch_delete_as(name, actor).await?; + Ok(BranchDeleteOutput { + uri: uri.clone(), + name: name.to_string(), + actor_id: actor.map(String::from), + }) + } + } + } + + pub(crate) async fn branch_merge(&self, source: &str, into: &str) -> Result<BranchMergeOutput> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + remote_json( + http, + Method::POST, + remote_url(base_url, "/branches/merge"), + Some(serde_json::to_value(BranchMergeRequest { + source: source.to_string(), + target: Some(into.to_string()), + })?), + token.as_deref(), + ) + .await + } + GraphClient::Embedded { uri, graph, actor } => { + let db = Self::open_embedded(uri, graph).await?; + let actor = actor.as_deref(); + let outcome = db.branch_merge_as(source, into, actor).await?; + Ok(BranchMergeOutput { + source: source.to_string(), + target: into.to_string(), + outcome: outcome.into(), + actor_id: actor.map(String::from), + }) + } + } + } + + /// `apply_schema` — apply `schema_source`. The embedded arm runs the + /// caller's catalog validator (stored-query registry check) inside the + /// engine's `apply_schema_as_with_catalog_check`; the remote arm runs + /// the server's own check and IGNORES `validate`. The `impl FnOnce` + /// validator is exactly why this is an enum, not a trait (non-object- + /// safe). + pub(crate) async fn apply_schema<F>( + &self, + schema_source: &str, + allow_data_loss: bool, + validate: F, + ) -> Result<SchemaApplyOutput> + where + F: FnOnce(&Catalog) -> omnigraph::error::Result<()>, + { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + // MR-694 PR B: SchemaApplyRequest carries allow_data_loss so + // Hard-mode drops are no longer CLI-only; the server's + // `server_schema_apply` honors it (and runs its own catalog + // check, so `validate` does not apply here). + remote_json::<SchemaApplyOutput>( + http, + Method::POST, + remote_url(base_url, "/schema/apply"), + Some(serde_json::to_value(SchemaApplyRequest { + schema_source: schema_source.to_string(), + allow_data_loss, + })?), + token.as_deref(), + ) + .await + } + GraphClient::Embedded { uri, graph, actor } => { + let db = Self::open_embedded(uri, graph).await?; + let result = db + .apply_schema_as_with_catalog_check( + schema_source, + omnigraph::db::SchemaApplyOptions { allow_data_loss }, + actor.as_deref(), + validate, + ) + .await?; + Ok(schema_apply_output(uri, result)) + } + } + } } diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index 67fb6ea..e9dfcc1 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -979,77 +979,6 @@ pub(crate) fn execute_queries_list( Ok(()) } -pub(crate) async fn execute_read( - uri: &str, - query_source: &str, - query_name: Option<&str>, - target: ReadTarget, - params_json: Option<&Value>, -) -> Result<ReadOutput> { - let (selected_name, query_params) = select_named_query(query_source, query_name)?; - let params = query_params_from_json(&query_params, params_json)?; - let db = Omnigraph::open(uri).await?; - let result = db - .query(target.clone(), query_source, &selected_name, ¶ms) - .await?; - Ok(read_output(selected_name, &target, result)) -} - -pub(crate) async fn execute_read_remote( - client: &reqwest::Client, - uri: &str, - query_source: &str, - query_name: Option<&str>, - target: ReadTarget, - params_json: Option<&Value>, - bearer_token: Option<&str>, -) -> Result<ReadOutput> { - let (branch, snapshot) = match &target { - ReadTarget::Branch(branch) => (Some(branch.clone()), None), - ReadTarget::Snapshot(snapshot) => (None, Some(snapshot.as_str().to_string())), - }; - remote_json( - client, - Method::POST, - remote_url(uri, "/read"), - Some(serde_json::to_value(ReadRequest { - query_source: query_source.to_string(), - query_name: query_name.map(ToOwned::to_owned), - params: params_json.cloned(), - branch, - snapshot, - })?), - bearer_token, - ) - .await -} - -pub(crate) async fn execute_change( - graph: &ResolvedCliGraph, - query_source: &str, - query_name: Option<&str>, - branch: &str, - params_json: Option<&Value>, - config: &OmnigraphConfig, - cli_as_actor: Option<&str>, -) -> Result<ChangeOutput> { - let (selected_name, query_params) = select_named_query(query_source, query_name)?; - let params = query_params_from_json(&query_params, params_json)?; - let db = open_local_db_with_policy(graph).await?; - let actor = resolve_cli_actor(cli_as_actor, config)?; - let actor = actor.as_deref(); - let result = db - .mutate_as(branch, query_source, &selected_name, ¶ms, actor) - .await?; - Ok(ChangeOutput { - branch: branch.to_string(), - query_name: selected_name, - affected_nodes: result.affected_nodes, - affected_edges: result.affected_edges, - actor_id: actor.map(String::from), - }) -} - pub(crate) fn legacy_change_request_body( query_source: &str, query_name: Option<&str>, @@ -1069,30 +998,6 @@ pub(crate) fn legacy_change_request_body( body } -pub(crate) async fn execute_change_remote( - client: &reqwest::Client, - uri: &str, - query_source: &str, - query_name: Option<&str>, - branch: &str, - params_json: Option<&Value>, - bearer_token: Option<&str>, -) -> Result<ChangeOutput> { - remote_json( - client, - Method::POST, - remote_url(uri, "/change"), - Some(legacy_change_request_body( - query_source, - query_name, - branch, - params_json, - )), - bearer_token, - ) - .await -} - pub(crate) async fn execute_export_to_writer<W: Write>( uri: &str, branch: &str, diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index e979622..53eb4c7 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -23,11 +23,8 @@ use omnigraph_compiler::{ json_params_to_param_map, lint_query_file, }; use omnigraph_api_types::{ - BranchCreateOutput, BranchCreateRequest, BranchDeleteOutput, - BranchMergeOutput, BranchMergeRequest, ChangeOutput, CommitOutput, - ErrorOutput, ExportRequest, GraphListResponse, IngestOutput, IngestRequest, ReadOutput, - ReadRequest, SchemaApplyOutput, SchemaApplyRequest, - SnapshotTableOutput, ingest_output, read_output, schema_apply_output, + ChangeOutput, CommitOutput, ErrorOutput, ExportRequest, GraphListResponse, IngestOutput, + ReadOutput, SchemaApplyOutput, SnapshotTableOutput, }; use omnigraph_server::queries::{QueryRegistry, check, format_check_breakages}; use omnigraph_server::{ @@ -166,44 +163,18 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let graph = resolve_cli_graph(&config, uri, target.as_deref())?; - let uri = graph.uri.clone(); + let client = client::GraphClient::resolve_with_policy( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + cli.as_actor.as_deref(), + )?; let branch = resolve_branch(&config, branch, None, "main"); - let payload = if graph.is_remote { - let data = fs::read_to_string(&data)?; - let output = remote_json::<IngestOutput>( - &http_client, - Method::POST, - remote_url(&uri, "/ingest"), - Some(serde_json::to_value(IngestRequest { - branch: Some(branch.clone()), - from: from.clone(), - mode: Some(mode.into()), - data, - })?), - bearer_token.as_deref(), - ) + let payload = client + .load(&branch, from.as_deref(), &data.to_string_lossy(), mode) .await?; - load_output_from_tables(&uri, &branch, mode.as_str(), &output) - } else { - let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; - let actor = actor.as_deref(); - let result = db - .load_file_as( - &branch, - from.as_deref(), - &data.to_string_lossy(), - mode.into(), - actor, - ) - .await?; - load_output_from_result(&uri, &branch, mode.as_str(), &result) - }; if json { print_json(&payload)?; } else { @@ -226,44 +197,19 @@ async fn main() -> Result<()> { use `omnigraph load --from <base> --mode <mode>` (ingest defaults: --from main --mode merge)" ); let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let graph = resolve_cli_graph(&config, uri, target.as_deref())?; - let uri = graph.uri.clone(); + let client = client::GraphClient::resolve_with_policy( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + cli.as_actor.as_deref(), + )?; let branch = resolve_branch(&config, branch, None, "main"); let from = resolve_branch(&config, from, None, "main"); - let payload = if graph.is_remote { - let data = fs::read_to_string(&data)?; - remote_json::<IngestOutput>( - &http_client, - Method::POST, - remote_url(&uri, "/ingest"), - Some(serde_json::to_value(IngestRequest { - branch: Some(branch.clone()), - from: Some(from.clone()), - mode: Some(mode.into()), - data, - })?), - bearer_token.as_deref(), - ) - .await? - } else { - let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; - let actor = actor.as_deref(); - let result = db - .load_file_as( - &branch, - Some(&from), - &data.to_string_lossy(), - mode.into(), - actor, - ) - .await?; - ingest_output(&uri, &result, mode.into(), None) - }; + let payload = client + .ingest(&branch, &from, &data.to_string_lossy(), mode) + .await?; if json { print_json(&payload)?; } else { @@ -280,38 +226,16 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let graph = resolve_cli_graph(&config, uri, target.as_deref())?; - let uri = graph.uri.clone(); + let client = client::GraphClient::resolve_with_policy( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + cli.as_actor.as_deref(), + )?; let from = resolve_branch(&config, from, None, "main"); - let payload = if graph.is_remote { - remote_json::<BranchCreateOutput>( - &http_client, - Method::POST, - remote_url(&uri, "/branches"), - Some(serde_json::to_value(BranchCreateRequest { - from: Some(from.clone()), - name: name.clone(), - })?), - bearer_token.as_deref(), - ) - .await? - } else { - let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; - let actor = actor.as_deref(); - db.branch_create_from_as(ReadTarget::branch(&from), &name, actor) - .await?; - BranchCreateOutput { - uri: uri.clone(), - from: from.clone(), - name: name.clone(), - actor_id: actor.map(String::from), - } - }; + let payload = client.branch_create_from(&from, &name).await?; if json { print_json(&payload)?; } else { @@ -349,32 +273,15 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let graph = resolve_cli_graph(&config, uri, target.as_deref())?; - let uri = graph.uri.clone(); - let payload = if graph.is_remote { - remote_json::<BranchDeleteOutput>( - &http_client, - Method::DELETE, - remote_branch_url(&uri, &name)?, - None, - bearer_token.as_deref(), - ) - .await? - } else { - let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; - let actor = actor.as_deref(); - db.branch_delete_as(&name, actor).await?; - BranchDeleteOutput { - uri: uri.clone(), - name: name.clone(), - actor_id: actor.map(String::from), - } - }; + let client = client::GraphClient::resolve_with_policy( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + cli.as_actor.as_deref(), + )?; + let payload = client.branch_delete(&name).await?; if json { print_json(&payload)?; } else { @@ -390,37 +297,16 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let graph = resolve_cli_graph(&config, uri, target.as_deref())?; - let uri = graph.uri.clone(); + let client = client::GraphClient::resolve_with_policy( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + cli.as_actor.as_deref(), + )?; let into = resolve_branch(&config, into, None, "main"); - let payload = if graph.is_remote { - remote_json::<BranchMergeOutput>( - &http_client, - Method::POST, - remote_url(&uri, "/branches/merge"), - Some(serde_json::to_value(BranchMergeRequest { - source: source.clone(), - target: Some(into.clone()), - })?), - bearer_token.as_deref(), - ) - .await? - } else { - let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; - let actor = actor.as_deref(); - let outcome = db.branch_merge_as(&source, &into, actor).await?; - BranchMergeOutput { - source: source.clone(), - target: into.clone(), - outcome: outcome.into(), - actor_id: actor.map(String::from), - } - }; + let payload = client.branch_merge(&source, &into).await?; if json { print_json(&payload)?; } else { @@ -519,52 +405,34 @@ async fn main() -> Result<()> { allow_data_loss, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let graph = resolve_cli_graph(&config, uri, target.as_deref())?; - let uri = graph.uri.clone(); + let client = client::GraphClient::resolve_with_policy( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + cli.as_actor.as_deref(), + )?; let schema_source = fs::read_to_string(&schema)?; - let output = if graph.is_remote { - // MR-694 PR B: SchemaApplyRequest gained an - // allow_data_loss field so Hard-mode drops are no - // longer CLI-only. The previous bail is gone; the - // field is forwarded into the JSON payload, and - // the server's `server_schema_apply` honors it. - remote_json::<SchemaApplyOutput>( - &http_client, - Method::POST, - remote_url(&uri, "/schema/apply"), - Some(serde_json::to_value(SchemaApplyRequest { - schema_source: schema_source.clone(), - allow_data_loss, - })?), - bearer_token.as_deref(), - ) - .await? + // The stored-query registry check is an embedded-only concern + // (the remote arm ignores the validator — the server runs its + // own check); build it only for the local path so the remote + // path keeps its no-registry-load behavior. + let registry = if client.is_remote() { + None } else { - let db = open_local_db_with_policy(&graph).await?; - let actor = resolve_cli_actor(cli.as_actor.as_deref(), &config)?; - let actor = actor.as_deref(); - let registry = load_registry_or_report(&config, graph.selected())?; - let registry = (!registry.is_empty()).then_some(registry); - let label = graph.selected().unwrap_or(&uri).to_string(); - let result = db - .apply_schema_as_with_catalog_check( - &schema_source, - omnigraph::db::SchemaApplyOptions { allow_data_loss }, - actor, - |catalog| { - if let Some(registry) = registry.as_ref() { - validate_registry_for_catalog(registry, catalog, &label)?; - } - Ok(()) - }, - ) - .await?; - schema_apply_output(&uri, result) + let registry = load_registry_or_report(&config, client.selected())?; + (!registry.is_empty()).then_some(registry) }; + let label = client.selected().unwrap_or(client.uri()).to_string(); + let output = client + .apply_schema(&schema_source, allow_data_loss, |catalog| { + if let Some(registry) = registry.as_ref() { + validate_registry_for_catalog(registry, catalog, &label)?; + } + Ok(()) + }) + .await?; if json { print_json(&output)?; } else { @@ -757,10 +625,13 @@ async fn main() -> Result<()> { let target_name = target .as_deref() .or_else(|| alias_config.and_then(|alias| alias.graph.as_deref())); - let uri = apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target_name)?; - let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target_name)?; - let graph = resolve_cli_graph(&config, uri, target_name)?; - let uri = graph.uri.clone(); + let client = client::GraphClient::resolve( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target_name, + )?; let query_source = resolve_query_source( &config, query.as_ref(), @@ -782,27 +653,14 @@ async fn main() -> Result<()> { alias_config.and_then(|alias| alias.branch.clone()), )?; let query_name = name.or_else(|| alias_config.and_then(|alias| alias.name.clone())); - let output = if graph.is_remote { - execute_read_remote( - &http_client, - &uri, + let output = client + .query( + target, &query_source, query_name.as_deref(), - target, - params_json.as_ref(), - bearer_token.as_deref(), - ) - .await? - } else { - execute_read( - &uri, - &query_source, - query_name.as_deref(), - target, params_json.as_ref(), ) - .await? - }; + .await?; let format = resolve_read_format( &config, format, @@ -844,10 +702,14 @@ async fn main() -> Result<()> { let target_name = target .as_deref() .or_else(|| alias_config.and_then(|alias| alias.graph.as_deref())); - let uri = apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target_name)?; - let bearer_token = resolve_remote_bearer_token(&config, uri.as_deref(), target_name)?; - let graph = resolve_cli_graph(&config, uri, target_name)?; - let uri = graph.uri.clone(); + let client = client::GraphClient::resolve_with_policy( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target_name, + cli.as_actor.as_deref(), + )?; let query_source = resolve_query_source( &config, query.as_ref(), @@ -869,29 +731,14 @@ async fn main() -> Result<()> { "main", ); let query_name = name.or_else(|| alias_config.and_then(|alias| alias.name.clone())); - let output = if graph.is_remote { - execute_change_remote( - &http_client, - &uri, + let output = client + .mutate( + &branch, &query_source, query_name.as_deref(), - &branch, params_json.as_ref(), - bearer_token.as_deref(), ) - .await? - } else { - execute_change( - &graph, - &query_source, - query_name.as_deref(), - &branch, - params_json.as_ref(), - &config, - cli.as_actor.as_deref(), - ) - .await? - }; + .await?; if json { print_json(&output)?; } else { From 45500a690a9bef49bf05993bb947d9ccfa2728d9 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sat, 13 Jun 2026 21:03:45 +0300 Subject: [PATCH 148/165] refactor(cli): collapse export + graphs-list onto GraphClient (RFC-009 Phase 3c) (#213) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The last two embedded-vs-remote forks move onto the enum, so every such `if` in the CLI now lives in client.rs — the point of the refactor. - `export<W: Write>`: the streaming verb 3b deferred (writes to a writer, chunks the HTTP response body, rather than returning a DTO). Embedded calls db.export_jsonl_to_writer; Remote streams the chunked body through. Opens WITHOUT policy (like reads), so it routes via resolve(). - `list_graphs`: remote-only by design (no local enumeration endpoint), so the Embedded arm keeps the loud "requires a remote multi-graph server" bail verbatim. Routing it through the enum still buys the shared resolve() addressing/token preamble the arm hand-rolled. Retire the now-orphaned execute_export_to_writer / execute_export_remote_to_writer pair, and sweep two pre-existing dead fns while in the files: inferred_config_path (helpers.rs) and yaml_string (output.rs, shadowed by test-local copies). parity_matrix gains one row, parity_export — the single intended matrix change in this phase. Export is a JSONL stream, not a single --json doc, so it compares the two arms' output line-wise (sorted; twin graphs are byte-copies so rows need no scrubbing). graphs-list gets no row: its remote-only behavior is a documented exclusion, not an equality case. Full workspace tests pass; all 12 parity rows green. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- crates/omnigraph-cli/src/client.rs | 93 ++++++++++++++++++++- crates/omnigraph-cli/src/helpers.rs | 65 -------------- crates/omnigraph-cli/src/main.rs | 60 +++++-------- crates/omnigraph-cli/src/output.rs | 4 - crates/omnigraph-cli/tests/parity_matrix.rs | 29 +++++++ 5 files changed, 137 insertions(+), 114 deletions(-) diff --git a/crates/omnigraph-cli/src/client.rs b/crates/omnigraph-cli/src/client.rs index 02daae4..5ca6351 100644 --- a/crates/omnigraph-cli/src/client.rs +++ b/crates/omnigraph-cli/src/client.rs @@ -21,14 +21,17 @@ //! `apply_schema` catalog-validator closure that is not object-safe. //! Same one-body-two-impls collapse, less ceremony. +use std::io::Write; + use color_eyre::Result; +use color_eyre::eyre::bail; use omnigraph::db::{Omnigraph, ReadTarget}; use omnigraph_api_types::{ BranchCreateOutput, BranchCreateRequest, BranchDeleteOutput, BranchListOutput, BranchMergeOutput, BranchMergeRequest, ChangeOutput, CommitListOutput, CommitOutput, - IngestOutput, IngestRequest, ReadOutput, ReadRequest, SchemaApplyOutput, SchemaApplyRequest, - SchemaOutput, SnapshotOutput, commit_output, ingest_output, read_output, schema_apply_output, - snapshot_payload, + ErrorOutput, ExportRequest, GraphListResponse, IngestOutput, IngestRequest, ReadOutput, + ReadRequest, SchemaApplyOutput, SchemaApplyRequest, SchemaOutput, SnapshotOutput, commit_output, + ingest_output, read_output, schema_apply_output, snapshot_payload, }; use omnigraph_compiler::catalog::Catalog; use reqwest::Method; @@ -36,7 +39,7 @@ use serde_json::Value; use crate::cli::CliLoadMode; use crate::helpers::{ - ResolvedCliGraph, apply_server_flag, build_http_client, is_remote_uri, + ResolvedCliGraph, apply_bearer_token, apply_server_flag, build_http_client, is_remote_uri, legacy_change_request_body, open_local_db_with_policy, query_params_from_json, remote_branch_url, remote_json, remote_url, resolve_cli_actor, resolve_cli_graph, resolve_remote_bearer_token, select_named_query, @@ -615,4 +618,86 @@ impl GraphClient { } } } + + /// `export` — stream the branch as JSONL into `writer`. The streaming + /// shape (a `W: Write`, not a returned DTO) is why this lands in 3c + /// rather than 3b. Opens WITHOUT policy (like reads), so it is reached + /// via `resolve()`; the Embedded arm opens bare. The Remote arm streams + /// the chunked response body straight through (no buffering the whole + /// export in memory). + pub(crate) async fn export<W: Write>( + &self, + branch: &str, + type_names: &[String], + table_keys: &[String], + writer: &mut W, + ) -> Result<()> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + let request = apply_bearer_token( + http.request(Method::POST, remote_url(base_url, "/export")), + token.as_deref(), + ) + .json(&ExportRequest { + branch: Some(branch.to_string()), + type_names: type_names.to_vec(), + table_keys: table_keys.to_vec(), + }); + let mut response = request.send().await?; + let status = response.status(); + if !status.is_success() { + let text = response.text().await?; + if let Ok(error) = serde_json::from_str::<ErrorOutput>(&text) { + bail!(error.error); + } + bail!("server returned {}: {}", status, text); + } + while let Some(chunk) = response.chunk().await? { + writer.write_all(&chunk)?; + } + writer.flush()?; + Ok(()) + } + GraphClient::Embedded { uri, .. } => { + let db = Omnigraph::open(uri).await?; + db.export_jsonl_to_writer(branch, type_names, table_keys, writer) + .await?; + writer.flush()?; + Ok(()) + } + } + } + + /// `graphs list` — enumerate the graphs a remote multi-graph server + /// serves (`GET /graphs`). Remote-only by design: there is no local + /// enumeration endpoint, so the Embedded arm fails loudly pointing the + /// operator at `omnigraph.yaml`. Routing it through the enum still buys + /// the shared `resolve()` addressing/token preamble. + pub(crate) async fn list_graphs(&self) -> Result<GraphListResponse> { + match self { + GraphClient::Remote { + http, + base_url, + token, + } => { + remote_json( + http, + Method::GET, + remote_url(base_url, "/graphs"), + None, + token.as_deref(), + ) + .await + } + GraphClient::Embedded { .. } => bail!( + "`omnigraph graphs list` requires a remote multi-graph server URL \ + (http:// or https://). To enumerate local graphs, read `omnigraph.yaml` \ + directly." + ), + } + } } diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index e9dfcc1..84c8867 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -678,22 +678,6 @@ pub(crate) fn normalize_legacy_alias_uri( } -pub(crate) fn inferred_config_path(uri: &str) -> Result<PathBuf> { - if uri.contains("://") { - return Ok(omnigraph_server::config::default_config_path()); - } - - let path = Path::new(uri); - let base = if path.is_absolute() { - path.parent() - .map(Path::to_path_buf) - .unwrap_or(std::env::current_dir()?) - } else { - std::env::current_dir()?.join(path.parent().unwrap_or_else(|| Path::new("."))) - }; - Ok(base.join(omnigraph_server::config::DEFAULT_CONFIG_FILE)) -} - pub(crate) fn read_target_from_cli(branch: Option<String>, snapshot: Option<String>) -> ReadTarget { if let Some(snapshot) = snapshot { ReadTarget::snapshot(SnapshotId::new(snapshot)) @@ -998,55 +982,6 @@ pub(crate) fn legacy_change_request_body( body } -pub(crate) async fn execute_export_to_writer<W: Write>( - uri: &str, - branch: &str, - type_names: &[String], - table_keys: &[String], - writer: &mut W, -) -> Result<()> { - let db = Omnigraph::open(uri).await?; - db.export_jsonl_to_writer(branch, type_names, table_keys, writer) - .await?; - writer.flush()?; - Ok(()) -} - -pub(crate) async fn execute_export_remote_to_writer<W: Write>( - client: &reqwest::Client, - uri: &str, - branch: &str, - type_names: &[String], - table_keys: &[String], - bearer_token: Option<&str>, - writer: &mut W, -) -> Result<()> { - let request = apply_bearer_token( - client.request(Method::POST, remote_url(uri, "/export")), - bearer_token, - ) - .json(&ExportRequest { - branch: Some(branch.to_string()), - type_names: type_names.to_vec(), - table_keys: table_keys.to_vec(), - }); - let mut response = request.send().await?; - let status = response.status(); - if !status.is_success() { - let text = response.text().await?; - if let Ok(error) = serde_json::from_str::<ErrorOutput>(&text) { - bail!(error.error); - } - bail!("server returned {}: {}", status, text); - } - - while let Some(chunk) = response.chunk().await? { - writer.write_all(&chunk)?; - } - writer.flush()?; - Ok(()) -} - pub(crate) fn rewrite_deprecated_argv(args: Vec<OsString>) -> Vec<OsString> { if args.len() >= 3 { let sub = args[1].to_str(); diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 53eb4c7..fd67fb3 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -23,8 +23,8 @@ use omnigraph_compiler::{ json_params_to_param_map, lint_query_file, }; use omnigraph_api_types::{ - ChangeOutput, CommitOutput, ErrorOutput, ExportRequest, GraphListResponse, IngestOutput, - ReadOutput, SchemaApplyOutput, SnapshotTableOutput, + ChangeOutput, CommitOutput, ErrorOutput, IngestOutput, ReadOutput, SchemaApplyOutput, + SnapshotTableOutput, }; use omnigraph_server::queries::{QueryRegistry, check, format_check_breakages}; use omnigraph_server::{ @@ -525,11 +525,13 @@ async fn main() -> Result<()> { table_keys, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; + let client = client::GraphClient::resolve( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + )?; let branch = resolve_branch(&config, branch, None, "main"); if jsonl { eprintln!("warning: --jsonl is deprecated; `omnigraph export` always emits JSONL"); @@ -537,21 +539,9 @@ async fn main() -> Result<()> { let stdout = io::stdout(); let mut stdout = stdout.lock(); - if is_remote_uri(&uri) { - execute_export_remote_to_writer( - &http_client, - &uri, - &branch, - &type_names, - &table_keys, - bearer_token.as_deref(), - &mut stdout, - ) + client + .export(&branch, &type_names, &table_keys, &mut stdout) .await?; - } else { - execute_export_to_writer(&uri, &branch, &type_names, &table_keys, &mut stdout) - .await?; - } } Command::Query { uri, @@ -1047,26 +1037,14 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = - apply_server_flag(cli.server.as_deref(), cli.graph.as_deref(), uri, target.as_deref())?; - let bearer_token = - resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; - if !is_remote_uri(&uri) { - bail!( - "`omnigraph graphs list` requires a remote multi-graph server URL \ - (http:// or https://). To enumerate local graphs, read `omnigraph.yaml` \ - directly." - ); - } - let payload = remote_json::<GraphListResponse>( - &http_client, - Method::GET, - remote_url(&uri, "/graphs"), - None, - bearer_token.as_deref(), - ) - .await?; + let client = client::GraphClient::resolve( + &config, + cli.server.as_deref(), + cli.graph.as_deref(), + uri, + target.as_deref(), + )?; + let payload = client.list_graphs().await?; if json { print_json(&payload)?; } else { diff --git a/crates/omnigraph-cli/src/output.rs b/crates/omnigraph-cli/src/output.rs index c6acd32..446c6ca 100644 --- a/crates/omnigraph-cli/src/output.rs +++ b/crates/omnigraph-cli/src/output.rs @@ -812,10 +812,6 @@ pub(crate) fn print_policy_explain(decision: &PolicyDecision, actor_id: &str, re println!("message: {}", decision.message); } -pub(crate) fn yaml_string(value: &str) -> String { - format!("'{}'", value.replace('\'', "''")) -} - #[derive(serde::Serialize)] pub(crate) struct QueriesIssue { pub(crate) query: String, diff --git a/crates/omnigraph-cli/tests/parity_matrix.rs b/crates/omnigraph-cli/tests/parity_matrix.rs index 75ba49e..b65c46e 100644 --- a/crates/omnigraph-cli/tests/parity_matrix.rs +++ b/crates/omnigraph-cli/tests/parity_matrix.rs @@ -179,6 +179,35 @@ fn parity_load() { assert_parity("load", &l, &r); } +#[test] +fn parity_export() { + let p = parity(); + let (l, r) = p.run(&["export"]); + // export emits a JSONL STREAM, not a single `--json` document, so the + // scrubbed-single-doc `assert_parity` doesn't apply — compare line-wise. + // The twin graphs are byte-copies of one loaded fixture, so rows carry + // identical ids/versions and need no scrubbing; sort the lines so any + // cross-arm row-ordering difference doesn't masquerade as a divergence. + assert_eq!( + l.status.code(), + r.status.code(), + "export: exit codes diverge\nlocal {l:?}\nremote {r:?}" + ); + assert!(l.status.success(), "export local arm failed: {l:?}"); + let mut local_lines: Vec<&str> = std::str::from_utf8(&l.stdout).unwrap().lines().collect(); + let mut remote_lines: Vec<&str> = std::str::from_utf8(&r.stdout).unwrap().lines().collect(); + assert!( + !local_lines.is_empty(), + "export produced no rows — the parity check would be vacuous" + ); + local_lines.sort_unstable(); + remote_lines.sort_unstable(); + assert_eq!( + local_lines, remote_lines, + "export: JSONL streams diverge (left=local, right=remote)" + ); +} + // ---- error parity: exit codes must match for shared failure cases ---- #[test] From be4f29c0d057dfb2eda84a2d9629548d4a820139 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sat, 13 Jun 2026 21:03:56 +0300 Subject: [PATCH 149/165] =?UTF-8?q?docs(rfc):=20RFC-010=20=E2=80=94=20rest?= =?UTF-8?q?ructure=20the=20CLI=20around=20explicit=20planes=20(#214)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The CLI silently spans three planes (data / storage-maintenance / control) and forces the operator to name a graph differently per plane: the graph you query as `--server prod --graph knowledge` you must maintain as `s3://bucket/knowledge.omni`. Plane restrictions (graphs list is server-only, optimize is storage-only) are accidental — discovered by hitting a cryptic error, not declared. RFC-010 proposes: one graph-addressing model across every verb, a declared per-subcommand capability surface (expanding RFC-009 Phase 4), and plane-grouped --help. Storage maintenance stays off the wire deliberately (no HTTP routes for optimize/cleanup/repair). CLI-internal only — no engine, server, or wire change. Incorporates the Codex review thread (kept verbatim with per-point Resolution notes): sharpened resolver authority rule (operator/legacy target must be direct storage; cluster-managed graphs via explicit --cluster --graph), per-subcommand capability table (schema plan vs show/apply, queries validate vs list, session/tooling classified), graphs list aligned to RFC-009's both-later target, init promoted to an explicit cluster-apply signpost, and a Test plan that extends the existing CLI suites and pins the new wrong-plane error strings. Linked from docs/dev/index.md. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- docs/dev/index.md | 1 + docs/dev/rfc-010-cli-planes-restructure.md | 381 +++++++++++++++++++++ 2 files changed, 382 insertions(+) create mode 100644 docs/dev/rfc-010-cli-planes-restructure.md diff --git a/docs/dev/index.md b/docs/dev/index.md index b1dc4fb..2b2ddf0 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -79,6 +79,7 @@ Working documents for in-flight feature work. Removed when the work lands. | Per-operator config — `~/.omnigraph/` identity, keyed credentials, named servers (the operator slice of RFC-002) | [rfc-007-operator-config.md](rfc-007-operator-config.md) | | Deprecate `omnigraph.yaml` — one concern per config surface; key-by-key migration map and staged retirement | [rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md) | | Unify CLI embedded/remote access paths — parity referee, shared wire-DTO crate, `GraphClient` trait, declared plane capabilities | [rfc-009-unify-access-paths.md](rfc-009-unify-access-paths.md) | +| Restructure the CLI around explicit planes — one graph-addressing model, declared capability surface, plane-grouped help (expands RFC-009 Phase 4) | [rfc-010-cli-planes-restructure.md](rfc-010-cli-planes-restructure.md) | ## Boundary diff --git a/docs/dev/rfc-010-cli-planes-restructure.md b/docs/dev/rfc-010-cli-planes-restructure.md new file mode 100644 index 0000000..3764d09 --- /dev/null +++ b/docs/dev/rfc-010-cli-planes-restructure.md @@ -0,0 +1,381 @@ +# RFC: Restructure the CLI Around Explicit Planes + +**Status:** Proposed +**Date:** 2026-06-13 +**Audience:** CLI/server/cluster maintainers +**Builds on:** [rfc-009-unify-access-paths.md](rfc-009-unify-access-paths.md) +(Phases 3a–3c landed — the embedded/remote data-plane fork is now one +`GraphClient` enum; this RFC **expands RFC-009 Phase 4** from a narrow +embedded-vs-remote capability table into the full plane model, and leaves +Phase 5 route alignment where it is), +[rfc-007-operator-config.md](rfc-007-operator-config.md) (operator +`--server`/`--graph`/`--target` addressing — the surfaces this RFC makes +uniform across planes), +[rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md). +**Sequencing:** post-v0.7.0, after RFC-009 Phase 3c (done). + +## Summary + +The CLI silently spans **three planes** — data, storage/maintenance, and +control — and forces the operator to know which plane each verb lives on *and* +address a graph differently per plane. The same graph you query as +`--server prod --graph knowledge` you must maintain as +`s3://bucket/knowledge.omni`. Plane restrictions (`graphs list` is server-only, +`optimize` is storage-only) are *accidental* — discovered by hitting a cryptic +error, not *declared*. + +This RFC makes the plane model **explicit and coherent** with three moves: + +1. **One graph-addressing model** across every verb (`--target`/`--graph`/ + positional URI/`--server`), resolving to a storage URI for maintenance and a + remote client for data — instead of two different ways to name one graph. +2. **A declared, per-subcommand capability surface** (RFC-009 Phase 4): each + verb declares its plane(s); wrong-plane invocations get an honest "this is + storage-plane, `--server` doesn't apply" error from one table, not scattered + `bail!`s. +3. **Plane-grouped `--help`** so the model is legible at a glance. + +No new server feature. Storage maintenance stays off the wire — deliberately. + +## Current state of affairs + +The CLI has 23 top-level commands. They divide into three planes, addressed +three different ways: + +| Plane | Verbs | Reaches the graph by | Addressing surface | +|---|---|---|---| +| **Data** | `query`, `mutate`, `load`, `ingest`, `branch *`, `snapshot`, `export`, `commit *`, `schema show/apply`, `graphs list` | embedded engine **or** HTTP server (one `GraphClient`, 15 call sites) | positional URI **or** `--target` / `--graph` / `--server` (config aliases) | +| **Storage / maintenance** | `init`, `optimize`, `repair`, `cleanup`, `schema plan`, `queries validate` | embedded engine **only**, directly on storage (`file://` or `s3://`) | positional URI **or** `--target` — **no `--server` / `--graph`** | +| **Control** | `cluster validate/plan/apply/approve/status/refresh/import/force-unlock` | a cluster **directory** (`file://` or `s3://`), not a graph URI | `--config <dir>` | + +### What's confusing (validated facts) + +1. **Two names for one graph.** Data verbs resolve `--server prod --graph + knowledge` through `apply_server_flag` (16 call sites). Maintenance verbs use + `resolve_uri`/`resolve_local_uri` and accept only a positional URI or + `--target` — so to compact the graph you *query* as `--server prod --graph + knowledge` you must *type* `s3://bucket/knowledge.omni`. One graph, two + addressing vocabularies. + +2. **Plane restrictions are accidental, not declared.** `graphs list` is + server-only and `optimize`/`repair`/`cleanup`/`init` are storage-only purely + by code shape. Point `optimize` at an `https://` URL and you get whatever + `Omnigraph::open` says about an https URI — accidental error text that, per + Hyrum's Law, is already someone's dependency. The capability is real but + unstated. + +3. **The split is per-subcommand, and the family names hide it.** `schema plan` + is storage-only (`resolve_local_uri`) while `schema show`/`schema apply` are + data-plane (the graph client). `queries validate` opens the graph to + typecheck while `queries list` only reads the registry config. The plane is + a property of the *subcommand*, not the family. + +4. **Maintenance has no server/cluster counterpart at all.** There is no HTTP + route and no `cluster` subcommand for `optimize`/`cleanup`/`repair` (verified: + nothing in the server route table, nothing in `omnigraph-cluster/src`). For a + server-backed deployment you run the *same CLI* against the storage URI, + out-of-band from the serving process. This is correct (maintenance is + heavyweight, destructive, single-operator — it should not be a multi-tenant + HTTP surface), but it is **undocumented in the CLI's own shape**, so it reads + as an omission rather than a decision. + +5. **`init` has a hidden control-plane twin.** Bare `init` creates a single + graph from storage; in cluster mode the equivalent is `cluster apply` + (graph-creation stage, with ledger/recovery/approval semantics). Same intent, + two entry points, no signpost between them. + +6. **Flat `--help`.** All 23 commands list as one undifferentiated block, so the + plane a verb belongs to is tribal knowledge. + +The net effect: a new operator must already know OmniGraph's plane architecture +to predict which flags work on which verb and how to name a graph. The CLI does +not teach its own model. + +## Target CLI ergonomics + +The throughline: **you name a graph one way, and the CLI tells you what works +where.** Simple examples of the end state: + +### One name for a graph, everywhere + +A config target `knowledge` works on every verb that touches that graph: + +```bash +omnigraph query --target knowledge --query q.gq # data (embedded or remote, auto) +omnigraph load --target knowledge --data rows.jsonl # data +omnigraph optimize --target knowledge # maintenance (resolves to its storage URI) +omnigraph cleanup --target knowledge --keep 10 --confirm +omnigraph repair --target knowledge --confirm +``` + +The positional URI form still works everywhere, unchanged: + +```bash +omnigraph optimize s3://bucket/knowledge.omni +``` + +### Data plane: same command, embedded or remote + +You don't pick "local vs server" syntax — resolution decides: + +```bash +omnigraph query ./local.omni --query q.gq # opens engine directly +omnigraph query --server prod --graph knowledge --query q.gq # over HTTP +omnigraph query --target knowledge --query q.gq # whichever the config says +``` + +### Maintenance: `--target` must resolve to direct storage (loud if not) + +```bash +$ omnigraph optimize --target prod +error: `--target prod` resolves to a remote server (https://prod…). + `optimize` is a storage-plane command and needs direct storage access. + Pass the graph's s3://… URI, or use --cluster <dir> --graph <id>. +``` + +Cluster-managed graphs get an explicit, intentional path (no implicit +`cluster.yaml` peeking): + +```bash +omnigraph optimize --cluster ./cluster --graph knowledge +``` + +### Wrong-plane = one honest, stable error + +```bash +$ omnigraph optimize --server prod +error: `optimize` is a storage-plane command; `--server` addresses the data + plane and does not apply here. Use --target <name> or a storage URI. + +$ omnigraph graphs list ./local.omni +error: `graphs list` needs a remote multi-graph server (http/https) today. + (Embedded cluster-catalog enumeration is planned — RFC-009.) +``` + +### `--help` teaches the model + +``` +DATA PLANE run against a graph (embedded or --server) + query mutate load branch snapshot export commit schema show schema apply + +STORAGE / MAINTENANCE direct storage access; no server + init optimize repair cleanup schema plan queries validate + +CONTROL PLANE manage a cluster directory + cluster + +INSPECT / SESSION + graphs list queries list lint policy embed login logout config +``` + +### Exceptions, signposted (not silent) + +```bash +omnigraph init --schema s.pg ./new.omni # plain path: fine + +$ omnigraph init --target knowledge --schema s.pg # cluster-managed target: redirected +error: `knowledge` is a cluster-managed graph. Create it via `cluster apply` + (which records ledger + recovery + approvals), not `init`. +``` + +**In one line:** one way to name a graph, the right flags accepted per verb, and +a CLI that tells you its planes instead of making you memorize them. + +## Proposed shape (mechanism) + +### One addressing model for every graph-addressing verb + +Route **all** graph-addressing verbs — data *and* maintenance — through one +resolver that turns `(positional URI | --target | --graph | --server)` into +either a **storage URI** (`file://`/`s3://`) → embedded execution, or a **remote +`GraphClient`** → HTTP execution, per the verb's declared plane. + +**Authority rule (the precedence must not be silent).** `--target` is an +operator/legacy target lookup; `cluster.yaml` is a *different* authority surface +(read only by `cluster` commands and `--cluster` boot). A maintenance verb must +not quietly consult both and invent a precedence. The rule: + +- A maintenance verb's `--target` resolves through the **operator/legacy** + config and its URI must already be **direct storage**; a target that resolves + to a remote (`http(s)://`) URL **fails loudly** (see the example above). +- **Cluster-managed graphs are addressed explicitly** via `--cluster <dir> + --graph <id>`, so reading cluster state is an intentional mode — never an + implicit fallback between operator config and `cluster.yaml`. + +### A declared, per-subcommand capability surface (RFC-009 Phase 4, expanded) + +One table, **per subcommand** (family-level rows hide exactly the cases the +table exists to make non-accidental): + +| Command | Data (embedded) | Data (remote) | Storage (direct) | Config / session | Notes | +|---|---|---|---|---|---| +| `query`, `mutate`, `load`, `ingest` | ✅ | ✅ | — | — | `ingest` is the deprecated alias of `load` | +| `branch create/list/delete/merge` | ✅ | ✅ | — | — | | +| `snapshot`, `export`, `commit list/show` | ✅ | ✅ | — | — | | +| `schema show` | ✅ | ✅ | — | — | | +| `schema apply` | ✅ | ✅ | — | — | declarative alternative: `cluster apply` | +| `schema plan` | — | — | ✅ | — | local resolver today | +| `queries validate` | — | — | ✅ | — | opens the graph to typecheck | +| `init` | — | — | ✅ | — | cluster-managed graphs → `cluster apply` | +| `optimize`, `repair`, `cleanup` | — | — | ✅ | — | | +| `graphs list` | (later) | ✅ | — | — | remote today; embedded-cluster later (RFC-009) | +| `queries list` | — | — | — | ✅ | reads the registry config; no graph | +| `lint` | — | — | ✅ | ✅ | `--schema` file, or opens a local graph | +| `policy validate/test/explain` | — | — | — | ✅ | reads policy files + config | +| `embed` | — | — | — | ✅ | local tooling (files + embedding API) | +| `login`, `logout`, `config`, `version` | — | — | — | ✅ | session / config; no graph | + +The resolver consults this table. A wrong-plane invocation produces one honest, +stable message instead of N ad-hoc `bail!`s and accidental `open` errors. + +### Plane-grouped `--help` + +Group the command list by plane (the `--help` block shown under Target CLI +ergonomics). Cosmetic, zero behavior change, highest legibility-per-line. + +### Maintenance stays off the wire (decision, not omission) + +This RFC **does not** add server routes for `optimize`/`cleanup`/`repair`: + +- **Serving = the server.** Multi-tenant, safe-for-many-callers data plane. +- **Storage maintenance = the CLI against storage**, addressed uniformly, + run by an operator or a scheduled job with storage access. + +Adding maintenance-over-HTTP would re-introduce a heavyweight, destructive +multi-tenant surface and *add* a plane rather than clarify the three we have. +A future cluster-driven maintenance reconciler (scheduled compaction/GC as a +control-plane policy) is explicitly **out of scope** — net-new design (who runs +it, with what resource bounds), not a CLI restructure. + +### `init` is an explicit exception (decision) + +Direct-storage `init` against a plain URI/target stays. But if a target resolves +to a **cluster-managed** graph root, `init` **refuses and signposts** `cluster +apply` (which records ledger, recovery, and approval artifacts) rather than +initializing that root out of band. This closes the "hidden twin" of the current +state. + +## Compatibility + +Additive and low-risk: + +- **`--target`/`--graph` on maintenance verbs** is new capability; the positional + URI form keeps working unchanged. +- **Grouped `--help`** is cosmetic. +- **Capability-surface error text** changes the message you get on a wrong-plane + or misaddressed invocation. Per Hyrum's Law that text is observable; the change + is deliberate, release-noted, and replaces an *accidental* `Omnigraph::open` + string with a *stable, declared* one — a net improvement, but flagged. + +No engine, server, or wire-protocol change. The work is CLI-internal: the shared +resolver, the capability table, and help grouping. + +## Test plan + +Extend the existing CLI suites rather than adding a duplicate harness: + +- **`parity_matrix.rs`** — capability exclusions (the per-subcommand plane table + becomes the source of truth for which verbs are remote-only / storage-only). +- **`cli_data.rs`** — maintenance wrong-plane errors (`optimize --server`, + `optimize --target <remote>`), and `--target` resolving to direct storage. +- **`cli_schema_config.rs`** — `graphs list` plane behavior, `schema plan` + vs `schema show/apply` plane split, and plane-grouped `--help` output. +- **`system_local.rs`** — `--server` / operator-targeting edge cases end-to-end. + +Pin the new wrong-plane error strings deliberately: this RFC is intentionally +replacing accidental `Omnigraph::open` strings with stable capability errors, and +those strings become observable behavior (Hyrum). + +## Relationship to RFC-009 + +RFC-009 Phase 4 was scoped as "declared plane capabilities" for the +embedded-vs-remote axis only. This RFC **subsumes and broadens** that phase into +the full three-plane, per-subcommand model (adds uniform maintenance addressing, +the authority rule, and help grouping). RFC-009 Phase 5 (remote `load` → +`/load` route alignment) is unaffected and remains in RFC-009. + +**`graphs list` reconciliation:** RFC-009's answered open question (pinned in +`parity_matrix.rs`'s exclusions comment) targets `graphs list` becoming +Both-capability once the embedded arm enumerates the cluster catalog. This RFC +**aligns** with that rather than superseding it: the capability table shows +`graphs list` as remote today, embedded-cluster later. + +## Open questions + +1. **Capability-table location** — a CLI-internal const, or surfaced (e.g. in + `--help` and a machine-readable `omnigraph capabilities` for tooling)? +2. **`--cluster <dir> --graph <id>` for maintenance** — does the maintenance + command resolve the storage URI from the applied cluster state, or from the + declared `cluster.yaml`? (Applied state is the truth the server serves; + declared config may be ahead of it.) + +## Review comments (Codex, 2026-06-13) + +Overall take: the direction is right. The planes already exist; making them +declared in code, help text, and error messages should reduce operator surprise. +Keeping storage maintenance off HTTP is also the right boundary: `optimize`, +`repair`, and `cleanup` are direct-storage operator actions, not a multi-tenant +serving surface. + +Before implementation, tighten these points: + +1. **Resolver authority needs a sharper rule.** The proposal says maintenance + resolves storage URIs "from `cluster.yaml` / operator config", but those are + different authority surfaces. Today `--target` is an operator/legacy + graph-target lookup; cluster config is read by `cluster` commands and by + `--cluster` server boot. Do not make a maintenance command silently consult + both and pick a precedence. Either: + - `--target` on maintenance means an operator/legacy target whose URI is + already direct storage, with remote targets failing loudly; or + - add an explicit cluster-root/config resolver for this case, so reading + cluster state is an intentional mode. + + **Resolution (accepted):** both — `--target` resolves through operator/legacy + config and must be direct storage (remote → loud fail); cluster-managed graphs + use the explicit `--cluster <dir> --graph <id>` resolver. See *Authority + rule* under Proposed shape. + +2. **`graphs list` conflicts with RFC-009's target shape.** This RFC classifies + `graphs list` as remote-only, while RFC-009's answered open question says it + becomes Both-capability once the embedded arm enumerates the cluster catalog. + Pick one direction here: either this RFC explicitly supersedes that target, + or the capability table should show `graphs list` as remote today and + embedded-cluster later. + + **Resolution (accepted):** align, don't supersede. The table shows `graphs + list` remote-today / embedded-cluster-later. See *Relationship to RFC-009*. + +3. **The capability table should be per subcommand, not per family.** The + family-level rows hide the exact cases the table is supposed to make + non-accidental. At minimum, call out: + - `schema plan` as local/storage-backed today, while `schema show` and + `schema apply` route through the graph client; + - `queries validate` versus `queries list`, which do not have the same + plane shape; + - `lint`, `policy`, `embed`, `login`, `logout`, `config`, and `version`, so + enumeration/session/tooling commands are intentionally classified instead + of falling outside the model. + + **Resolution (accepted):** the capability table is now per-subcommand and + classifies every command, including the session/tooling group. + +4. **`init` should be an explicit exception.** Direct-storage `init` is fine. + A cluster-managed graph should be created by `cluster apply`, with ledger, + recovery, and approval semantics. If a named target resolves to a + cluster-managed graph root, `init` should signpost `cluster apply` rather + than quietly initializing that root out of band. + + **Resolution (accepted):** promoted from open question to a decision. See + *`init` is an explicit exception*. + +Testing notes for the implementation slice: + +- Extend the existing CLI suites rather than adding a new duplicate harness: + `parity_matrix.rs` for capability exclusions, `cli_data.rs` for maintenance + wrong-plane errors, `cli_schema_config.rs` for `graphs list` / help behavior, + and `system_local.rs` for `--server` / operator-targeting edge cases. +- Pin the new wrong-plane error strings deliberately. This RFC is intentionally + replacing accidental `Omnigraph::open` strings with stable capability errors, + and those strings become observable behavior. + + **Resolution (accepted):** captured as the *Test plan* section. From 2ddb88fad998cb8d4c90fe95cd6c321449dd34d2 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sat, 13 Jun 2026 22:24:09 +0300 Subject: [PATCH 150/165] =?UTF-8?q?docs(rfc):=20RFC-010=20=E2=80=94=20appl?= =?UTF-8?q?y=20verification-comment=20current-state=20fixups=20(#215)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Folds in the Codex verification review (kept verbatim with per-point Resolution notes): - `graphs list` is marked remote-only today in the current-state table (the embedded arm bails; it rides GraphClient only to share the resolver). - `init` is noted as positional-URI-only today (no `--target`); adding `--target` to init is part of the proposal, entangled with the init→cluster apply signpost, not current state. - Validated-fact #1 now describes the post-collapse reality (`GraphClient::resolve*`; only the two factories call `apply_server_flag`), dropping the stale "16 call sites" count. - The Authority rule carries a flag-shape caveat: `--graph` is already a global flag requiring `--server`, so the cluster-managed resolver and its flag shape are deferred to a later slice; the illustrative `--cluster <dir> --graph <id>` spelling is marked not-final. Docs-only; no code change. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- docs/dev/rfc-010-cli-planes-restructure.md | 80 ++++++++++++++++++++-- 1 file changed, 74 insertions(+), 6 deletions(-) diff --git a/docs/dev/rfc-010-cli-planes-restructure.md b/docs/dev/rfc-010-cli-planes-restructure.md index 3764d09..ad7f7b9 100644 --- a/docs/dev/rfc-010-cli-planes-restructure.md +++ b/docs/dev/rfc-010-cli-planes-restructure.md @@ -44,19 +44,26 @@ three different ways: | Plane | Verbs | Reaches the graph by | Addressing surface | |---|---|---|---| -| **Data** | `query`, `mutate`, `load`, `ingest`, `branch *`, `snapshot`, `export`, `commit *`, `schema show/apply`, `graphs list` | embedded engine **or** HTTP server (one `GraphClient`, 15 call sites) | positional URI **or** `--target` / `--graph` / `--server` (config aliases) | -| **Storage / maintenance** | `init`, `optimize`, `repair`, `cleanup`, `schema plan`, `queries validate` | embedded engine **only**, directly on storage (`file://` or `s3://`) | positional URI **or** `--target` — **no `--server` / `--graph`** | +| **Data** | `query`, `mutate`, `load`, `ingest`, `branch *`, `snapshot`, `export`, `commit *`, `schema show/apply` (and `graphs list`, **remote-only today** — see note) | embedded engine **or** HTTP server (one `GraphClient`) | positional URI **or** `--target` / `--graph` / `--server` (config aliases) | +| **Storage / maintenance** | `init`, `optimize`, `repair`, `cleanup`, `schema plan`, `queries validate` | embedded engine **only**, directly on storage (`file://` or `s3://`) | positional URI **or** `--target` — **no `--server` / `--graph`** (except `init`, which today takes **only a required positional URI** — no `--target`) | | **Control** | `cluster validate/plan/apply/approve/status/refresh/import/force-unlock` | a cluster **directory** (`file://` or `s3://`), not a graph URI | `--config <dir>` | ### What's confusing (validated facts) 1. **Two names for one graph.** Data verbs resolve `--server prod --graph - knowledge` through `apply_server_flag` (16 call sites). Maintenance verbs use + knowledge` through `GraphClient::resolve*` (the embedded/remote fork collapsed + in RFC-009 Phases 3a–3c; only the two `GraphClient` factories call + `apply_server_flag`). Maintenance verbs instead use `resolve_uri`/`resolve_local_uri` and accept only a positional URI or `--target` — so to compact the graph you *query* as `--server prod --graph knowledge` you must *type* `s3://bucket/knowledge.omni`. One graph, two addressing vocabularies. + > **Note (`graphs list`).** It is routed through `GraphClient` only to share + > the addressing/token resolver; its embedded arm fails loudly, so it is + > **remote-only today** (the later capability table and *Relationship to + > RFC-009* record it as remote-now / embedded-cluster-later). + 2. **Plane restrictions are accidental, not declared.** `graphs list` is server-only and `optimize`/`repair`/`cleanup`/`init` are storage-only purely by code shape. Point `optimize` at an `https://` URL and you get whatever @@ -198,9 +205,19 @@ not quietly consult both and invent a precedence. The rule: - A maintenance verb's `--target` resolves through the **operator/legacy** config and its URI must already be **direct storage**; a target that resolves to a remote (`http(s)://`) URL **fails loudly** (see the example above). -- **Cluster-managed graphs are addressed explicitly** via `--cluster <dir> - --graph <id>`, so reading cluster state is an intentional mode — never an - implicit fallback between operator config and `cluster.yaml`. +- **Cluster-managed graphs are addressed explicitly** via a cluster-root + + graph-id pair (spelled `--cluster <dir> --graph <id>` for illustration), so + reading cluster state is an intentional mode — never an implicit fallback + between operator config and `cluster.yaml`. + + > **Flag-shape caveat (deferred).** `--graph` is *already* a global flag that + > `requires = "server"` and appends `/graphs/<id>` to a **remote** URL — a + > different meaning, and clap won't permit `--graph` without `--server`. So the + > cluster-maintenance addressing needs either a distinct flag (e.g. + > `--cluster-graph <id>`) or an explicit global-flag migration. This is why + > the cluster-managed resolver is **deferred to a later slice** (it also rides + > the applied-state-vs-declared-config open question below); the + > operator/legacy `--target` path lands first. ### A declared, per-subcommand capability surface (RFC-009 Phase 4, expanded) @@ -379,3 +396,54 @@ Testing notes for the implementation slice: and those strings become observable behavior. **Resolution (accepted):** captured as the *Test plan* section. + +## Verification comments (Codex, 2026-06-13) + +Follow-up verification against the current CLI/server code found a few +remaining current-state nits. These are doc-shape issues, not objections to the +proposal: + +1. **Current-state table overstates `graphs list`.** The table under *Current + state of affairs* still lists `graphs list` with data verbs that reach the + graph by embedded engine or HTTP. Current code routes it through `GraphClient` + only to share the resolver, but the embedded arm fails loudly; the later + RFC text correctly says remote today / embedded-cluster later. Make the + current-state row match that. + + **Resolution (accepted):** the Data row now marks `graphs list` **remote-only + today**, with a note that it rides `GraphClient` only to share the resolver. + +2. **Current-state table overstates `init` addressing.** `init` is grouped with + maintenance verbs whose addressing surface is positional URI or `--target`. + Current `init` only accepts a required positional URI and has no `--target` + or config path. The proposal can add that capability, but the current-state + table should not describe it as already present. + + **Resolution (accepted):** the Storage row now calls out that `init` takes + **only a required positional URI** today (no `--target`); adding `--target` to + `init` is part of the proposal, entangled with the `init`→`cluster apply` + signpost, not current state. + +3. **`apply_server_flag` call-site count is stale.** The text says data verbs + resolve `--server prod --graph knowledge` through `apply_server_flag` at + 16 call sites. Current code has the fork collapsed: data verbs call + `GraphClient::resolve*`, and only the two `GraphClient` factories call + `apply_server_flag`. Rephrase the verified fact around `GraphClient`, not + the old pre-collapse call-site count. + + **Resolution (accepted):** validated-fact #1 now describes the post-collapse + reality (`GraphClient::resolve*`; the two factories call `apply_server_flag`), + dropping the stale count. + +4. **`--cluster <dir> --graph <id>` collides with today's global `--graph` + semantics.** The target ergonomics section proposes that flag shape for + maintenance, but current `--graph` is a global flag that requires + `--server` and appends `/graphs/<id>` to a remote server URL. Either choose + a separate cluster-maintenance graph flag shape, or call out the clap/global + flag migration explicitly as part of the implementation. + + **Resolution (accepted):** the *Authority rule* now carries a flag-shape + caveat — the cluster-managed resolver (and its flag shape, e.g. + `--cluster-graph` vs a `--graph` migration) is **deferred to a later slice**; + the operator/legacy `--target` path lands first. The illustrative + `--cluster <dir> --graph <id>` spelling is marked as not-final. From 106356ab251bebf0414c18cbfc1d522adbfe66ae Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sat, 13 Jun 2026 22:45:58 +0300 Subject: [PATCH 151/165] =?UTF-8?q?feat(cli):=20RFC-010=20Slice=201=20?= =?UTF-8?q?=E2=80=94=20declared=20plane=20capability=20surface=20+=20hones?= =?UTF-8?q?t=20addressing=20(#217)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(cli): declared plane capability surface + wrong-plane guard (RFC-010 Slice 1) New `planes.rs` is the single source of truth for which plane each subcommand belongs to (Data / Storage / Control / Session). `command_plane` is an exhaustive match — adding a `Command` variant is a compile error until its plane is declared, so the surface cannot silently drift from the command set. It descends into the nested enums where the plane differs per subcommand (`schema plan` is storage while `schema show/apply` are data; `queries validate` opens the graph while `queries list` reads only config). `guard_addressing` runs once in `main` before dispatch: the data-plane addressing flags `--server`/`--graph` on any non-data verb now fail with one declared, pinned error instead of being silently ignored (`optimize --server prod` previously dropped `--server`). `init`'s message drops the `--target` half since it takes only a positional URI today. Test: `cli_schema_config::schema_plan_with_server_flag_errors_wrong_plane` pins the per-subcommand label, proving the guard descends into the nested enum. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): storage-plane verbs fail loudly on a remote target (RFC-010 Slice 1) `optimize`/`repair`/`cleanup` switch from `resolve_uri` to `resolve_local_uri`, so a `--target` (or positional URI) that resolves to a remote server now fails with a declared storage-plane message instead of whatever `Omnigraph::open` said about an `http(s)://` URI. The `resolve_local_graph` bail is reworded to that storage-plane message, so every storage verb already on the local resolver (`schema plan`, `queries validate`, `lint`) speaks with one voice. Net: `optimize --target knowledge` resolves to the graph's storage URI and runs embedded; `optimize --target prod` (remote) fails loudly; `optimize --server` is caught earlier by the guard. Positional-URI invocations are unchanged. Tests (pinned strings, per RFC-010's test plan): optimize happy path on a local graph, `optimize --server` wrong-plane error, `optimize <https>` storage-plane error; the existing `query_lint_rejects_http_targets_without_schema` assertion is updated to the new shared message. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- crates/omnigraph-cli/src/helpers.rs | 7 +- crates/omnigraph-cli/src/main.rs | 11 +- crates/omnigraph-cli/src/planes.rs | 151 ++++++++++++++++++ crates/omnigraph-cli/tests/cli_data.rs | 47 +++++- .../omnigraph-cli/tests/cli_schema_config.rs | 22 +++ 5 files changed, 232 insertions(+), 6 deletions(-) create mode 100644 crates/omnigraph-cli/src/planes.rs diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index 84c8867..9991486 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -464,8 +464,11 @@ pub(crate) fn resolve_local_graph( let graph = resolve_cli_graph(config, cli_uri, cli_target)?; if graph.is_remote { bail!( - "{} is only supported against local graph URIs in this milestone", - operation + "`{}` is a storage-plane command and needs direct storage access; \ + the resolved target is a remote server ({}). Pass the graph's \ + file:// or s3:// URI.", + operation, + graph.uri ); } Ok(graph) diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index fd67fb3..6c94132 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -49,6 +49,7 @@ mod cli; mod client; mod helpers; mod output; +mod planes; use cli::*; use helpers::*; use output::*; @@ -70,6 +71,10 @@ async fn main() -> Result<()> { Cli::from_arg_matches(&matches)? }; let http_client = build_http_client()?; + // RFC-010 Slice 1: reject data-plane addressing flags (--server/--graph) on + // a verb that doesn't live on the data plane, from one declared table — + // before any per-command dispatch. + planes::guard_addressing(&cli)?; match cli.command { Command::Config { command } => match command { ConfigCommand::Migrate { config, write, json } => { @@ -781,7 +786,7 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; + let uri = resolve_local_uri(&config, uri, target.as_deref(), "optimize")?; let db = Omnigraph::open(&uri).await?; let stats = db.optimize().await?; if json { @@ -823,7 +828,7 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; + let uri = resolve_local_uri(&config, uri, target.as_deref(), "repair")?; let db = Omnigraph::open(&uri).await?; let stats = db .repair(omnigraph::db::RepairOptions { confirm, force }) @@ -907,7 +912,7 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = resolve_uri(&config, uri, target.as_deref())?; + let uri = resolve_local_uri(&config, uri, target.as_deref(), "cleanup")?; let older_than_dur = older_than.as_deref().map(parse_duration_arg).transpose()?; diff --git a/crates/omnigraph-cli/src/planes.rs b/crates/omnigraph-cli/src/planes.rs new file mode 100644 index 0000000..81328d3 --- /dev/null +++ b/crates/omnigraph-cli/src/planes.rs @@ -0,0 +1,151 @@ +//! Declared CLI "planes" (RFC-010 Slice 1). +//! +//! Every subcommand belongs to exactly one plane. This classification is the +//! single source of truth the wrong-plane guard consumes — and that later +//! RFC-010 slices (the capability surface, plane-grouped help) will consume +//! too. The `command_plane` match is **exhaustive on purpose**: adding a +//! `Command` variant is a compile error until its plane is declared, so the +//! surface cannot silently drift from the command set. +//! +//! See [docs/dev/rfc-010-cli-planes-restructure.md]. + +use color_eyre::Result; +use color_eyre::eyre::bail; + +use crate::cli::{Cli, Command, QueriesCommand, SchemaCommand}; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Plane { + /// Runs against a graph, embedded **or** via `--server` (the `GraphClient` + /// axis). The only plane on which the data-plane addressing flags + /// (`--server`/`--graph`) apply. + Data, + /// Direct storage access; no server. Maintenance + local-only inspection + /// that must work with the server down. + Storage, + /// Operates on a cluster directory, not a graph URI. + Control, + /// Touches no graph at all — session / config / local tooling. + Session, +} + +impl std::fmt::Display for Plane { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(match self { + Plane::Data => "data", + Plane::Storage => "storage", + Plane::Control => "control", + Plane::Session => "session", + }) + } +} + +/// The plane a subcommand belongs to. Exhaustive — a new `Command` variant +/// will not compile until classified. Descends into the nested enums where +/// the plane differs per subcommand (`schema plan` is storage while `schema +/// show`/`apply` are data; `queries validate` opens the graph while `queries +/// list` only reads config). +pub(crate) fn command_plane(cmd: &Command) -> Plane { + match cmd { + Command::Query { .. } + | Command::Mutate { .. } + | Command::Load { .. } + | Command::Ingest { .. } + | Command::Branch { .. } + | Command::Snapshot { .. } + | Command::Export { .. } + | Command::Commit { .. } + | Command::Graphs { .. } => Plane::Data, + Command::Schema { + command: SchemaCommand::Show { .. } | SchemaCommand::Apply { .. }, + } => Plane::Data, + Command::Schema { + command: SchemaCommand::Plan { .. }, + } => Plane::Storage, + Command::Queries { + command: QueriesCommand::Validate { .. }, + } => Plane::Storage, + Command::Queries { + command: QueriesCommand::List { .. }, + } => Plane::Session, + Command::Init { .. } + | Command::Optimize { .. } + | Command::Repair { .. } + | Command::Cleanup { .. } + | Command::Lint { .. } => Plane::Storage, + Command::Cluster { .. } => Plane::Control, + Command::Policy { .. } + | Command::Embed(_) + | Command::Login { .. } + | Command::Logout { .. } + | Command::Config { .. } + | Command::Version => Plane::Session, + } +} + +/// User-facing label for a subcommand (descends one level for the nested +/// families so messages read `schema plan`, `queries validate`, etc.). +pub(crate) fn command_label(cmd: &Command) -> &'static str { + match cmd { + Command::Version => "version", + Command::Login { .. } => "login", + Command::Logout { .. } => "logout", + Command::Config { .. } => "config", + Command::Embed(_) => "embed", + Command::Init { .. } => "init", + Command::Load { .. } => "load", + Command::Ingest { .. } => "ingest", + Command::Branch { .. } => "branch", + Command::Schema { command } => match command { + SchemaCommand::Plan { .. } => "schema plan", + SchemaCommand::Apply { .. } => "schema apply", + SchemaCommand::Show { .. } => "schema show", + }, + Command::Lint { .. } => "lint", + Command::Queries { command } => match command { + QueriesCommand::Validate { .. } => "queries validate", + QueriesCommand::List { .. } => "queries list", + }, + Command::Snapshot { .. } => "snapshot", + Command::Export { .. } => "export", + Command::Commit { .. } => "commit", + Command::Query { .. } => "query", + Command::Mutate { .. } => "mutate", + Command::Policy { .. } => "policy", + Command::Optimize { .. } => "optimize", + Command::Repair { .. } => "repair", + Command::Cleanup { .. } => "cleanup", + Command::Cluster { .. } => "cluster", + Command::Graphs { .. } => "graphs", + } +} + +/// Reject the data-plane addressing flags (`--server`/`--graph`) on any verb +/// that does not live on the data plane. This replaces the old silent-ignore +/// — e.g. `optimize --server prod` previously dropped `--server` and tried to +/// resolve a default target, failing (if at all) with an unrelated message. +/// Now it fails with one honest, declared error. RFC-010 Slice 1. +pub(crate) fn guard_addressing(cli: &Cli) -> Result<()> { + if cli.server.is_none() && cli.graph.is_none() { + return Ok(()); + } + let plane = command_plane(&cli.command); + if plane == Plane::Data { + return Ok(()); + } + let label = command_label(&cli.command); + let how = match plane { + // `init` is the one storage verb with no `--target` today (it takes a + // required positional URI), so its remediation drops the `--target` half. + Plane::Storage => match cli.command { + Command::Init { .. } => "Pass a storage URI.", + _ => "Use --target <name> or a storage URI.", + }, + Plane::Control => "It operates on a cluster directory (pass --config <dir>).", + Plane::Session => "It does not address a graph.", + Plane::Data => unreachable!("data plane returned early"), + }; + bail!( + "`{label}` is a {plane}-plane command; --server/--graph address the data plane and do not apply. {how}" + ); +} diff --git a/crates/omnigraph-cli/tests/cli_data.rs b/crates/omnigraph-cli/tests/cli_data.rs index 203a7c2..01dbeb3 100644 --- a/crates/omnigraph-cli/tests/cli_data.rs +++ b/crates/omnigraph-cli/tests/cli_data.rs @@ -142,6 +142,47 @@ fn embed_seed_preserves_non_entity_rows() { assert_eq!(embedded[2]["to"], "dec-alpha"); } +#[test] +fn optimize_json_succeeds_on_local_graph() { + // Happy path for the resolve_local_uri swap (RFC-010 Slice 1): a positional + // local path still resolves and runs embedded. + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + + let output = output_success(cli().arg("optimize").arg("--json").arg(&graph)); + let payload: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert!(payload["tables"].as_array().is_some()); +} + +#[test] +fn optimize_with_server_flag_errors_wrong_plane() { + // RFC-010 Slice 1: --server is a data-plane addressing flag; on a + // storage-plane verb the guard rejects it loudly (was: silently ignored). + let output = output_failure(cli().arg("optimize").arg("--server").arg("prod")); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("`optimize` is a storage-plane command") + && stderr.contains("--server/--graph address the data plane and do not apply") + && stderr.contains("Use --target <name> or a storage URI."), + "wrong-plane guard message not found; got: {stderr}" + ); +} + +#[test] +fn optimize_with_remote_target_errors_storage_plane() { + // RFC-010 Slice 1: a maintenance verb pointed at a remote URI fails loudly + // and declaratively (was: whatever Omnigraph::open said about an https URI). + let output = output_failure(cli().arg("optimize").arg("https://graph.example.invalid")); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("`optimize` is a storage-plane command and needs direct storage access") + && stderr.contains("remote server"), + "storage-plane remote-target message not found; got: {stderr}" + ); +} + #[test] fn repair_json_reports_noop_on_clean_graph() { let temp = tempdir().unwrap(); @@ -542,8 +583,12 @@ query list_people() { .arg("http://127.0.0.1:8080"), ); let stderr = String::from_utf8_lossy(&output.stderr); + // RFC-010 Slice 1: the storage-plane verbs now share one declared message + // (was: "query lint is only supported against local graph URIs …"). assert!( - stderr.contains("query lint is only supported against local graph URIs in this milestone") + stderr.contains("`query lint` is a storage-plane command and needs direct storage access") + && stderr.contains("remote server"), + "storage-plane remote-target message not found; got: {stderr}" ); } diff --git a/crates/omnigraph-cli/tests/cli_schema_config.rs b/crates/omnigraph-cli/tests/cli_schema_config.rs index 710c856..c962321 100644 --- a/crates/omnigraph-cli/tests/cli_schema_config.rs +++ b/crates/omnigraph-cli/tests/cli_schema_config.rs @@ -72,6 +72,28 @@ fn schema_plan_json_reports_supported_additive_change() { assert_eq!(payload["steps"][0]["property_name"], "nickname"); } +#[test] +fn schema_plan_with_server_flag_errors_wrong_plane() { + // RFC-010 Slice 1: `schema plan` is storage-plane while `schema show/apply` + // are data-plane — the guard rejects --server on plan with the per-subcommand + // label (proving command_plane/command_label descend into the nested enum). + let output = output_failure( + cli() + .arg("schema") + .arg("plan") + .arg("--schema") + .arg(fixture("test.pg")) + .arg("--server") + .arg("prod"), + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("`schema plan` is a storage-plane command") + && stderr.contains("Use --target <name> or a storage URI."), + "schema plan wrong-plane message not found; got: {stderr}" + ); +} + #[test] fn schema_plan_json_reports_unsupported_type_change() { let temp = tempdir().unwrap(); From 4187d56f8a3d259d3b5714dcd6247f22ce19c825 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sat, 13 Jun 2026 22:58:51 +0300 Subject: [PATCH 152/165] fix(cli): align lint plane label + document the plane model (RFC-010 follow-up) (#218) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the Greptile review on #217: P1 — `lint` reported two different names. `command_label` returns `lint`, but `execute_query_lint` passed `"query lint"` as the resolver operation string, so `lint --server` said `lint` while `lint <https>` said `query lint`. Both were pinned by tests. `query lint` is the *deprecated* alias (argv-rewritten to `lint`), so the canonical name is `lint`: switch both user-facing strings in `execute_query_lint` (the storage-plane bail label and the requires-schema-or-target usage message) to `lint`, and update the two pinned assertions in `cli_data.rs`. P2 — user-doc debt (AGENTS.md rule 1: error text is observable behavior). Document the plane model in `cli-reference.md` (new *Command planes* section: data vs storage/maintenance vs control, which addressing flags apply, and the declared wrong-plane / remote-target errors), and add an addressing note to `maintenance.md` cross-referencing it. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- crates/omnigraph-cli/src/helpers.rs | 4 ++-- crates/omnigraph-cli/tests/cli_data.rs | 4 ++-- docs/user/cli-reference.md | 15 +++++++++++++++ docs/user/maintenance.md | 2 ++ 4 files changed, 21 insertions(+), 4 deletions(-) diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index 9991486..e9809b5 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -754,10 +754,10 @@ pub(crate) async fn execute_query_lint( let has_graph_target = cli_uri.is_some() || cli_target.is_some() || config.cli_graph_name().is_some(); if !has_graph_target { - bail!("query lint requires --schema <schema.pg> or a resolvable graph target"); + bail!("lint requires --schema <schema.pg> or a resolvable graph target"); } - let uri = resolve_local_uri(config, cli_uri, cli_target, "query lint")?; + let uri = resolve_local_uri(config, cli_uri, cli_target, "lint")?; let db = Omnigraph::open(&uri).await?; Ok(lint_query_file( &db.catalog(), diff --git a/crates/omnigraph-cli/tests/cli_data.rs b/crates/omnigraph-cli/tests/cli_data.rs index 01dbeb3..f558018 100644 --- a/crates/omnigraph-cli/tests/cli_data.rs +++ b/crates/omnigraph-cli/tests/cli_data.rs @@ -586,7 +586,7 @@ query list_people() { // RFC-010 Slice 1: the storage-plane verbs now share one declared message // (was: "query lint is only supported against local graph URIs …"). assert!( - stderr.contains("`query lint` is a storage-plane command and needs direct storage access") + stderr.contains("`lint` is a storage-plane command and needs direct storage access") && stderr.contains("remote server"), "storage-plane remote-target message not found; got: {stderr}" ); @@ -615,7 +615,7 @@ query list_people() { ); let stderr = String::from_utf8_lossy(&output.stderr); assert!( - stderr.contains("query lint requires --schema <schema.pg> or a resolvable graph target") + stderr.contains("lint requires --schema <schema.pg> or a resolvable graph target") ); } diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index b419adf..3843b2e 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -28,6 +28,21 @@ Top-level command families and subcommands. Graph-targeting commands accept a po | `policy validate \| test \| explain` | Cedar tooling. Selects `cli.graph`, else `server.graph`, else top-level `policy.file` | | `version` / `-v` | print `omnigraph 0.3.x` | +## Command planes + +Every command lives on one **plane**, which determines how it reaches a graph and which addressing flags apply (RFC-010): + +- **Data plane** — `query`, `mutate`, `load`, `ingest`, `branch *`, `snapshot`, `export`, `commit *`, `schema show`, `schema apply` (and `graphs list`, remote-only today). Run against a graph **embedded or via a server**: accept a positional `URI` / `--target` / `--server` (+ `--graph` for multi-graph servers). +- **Storage / maintenance plane** — `init`, `optimize`, `repair`, `cleanup`, `schema plan`, `queries validate`, `lint`. Run with **direct storage access** (`file://` / `s3://`), never through a server. They accept a positional `URI` or `--target`, but **not** `--server` / `--graph`, and a `--target` that resolves to a remote (`http(s)://`) server is rejected. (`init` takes only a positional `URI` today — no `--target`.) +- **Control plane** — `cluster *`. Operates on a cluster directory via `--config <dir>`. + +These restrictions are enforced and reported, not silent: + +- A data-plane addressing flag on a non-data verb fails loudly, e.g.: ``optimize is a storage-plane command; --server/--graph address the data plane and do not apply. Use --target <name> or a storage URI.`` +- A storage-plane verb pointed at a remote target fails loudly, e.g.: ``optimize is a storage-plane command and needs direct storage access; the resolved target is a remote server (https://…). Pass the graph's file:// or s3:// URI.`` + +To maintain a server-backed graph, run the maintenance verbs from a host with storage access against the graph's storage URI (or `--target`), out-of-band from the serving process — there are no server routes for `optimize` / `repair` / `cleanup` by design. + ## Config surfaces Two config surfaces with single owners (RFC-007/RFC-008), plus a zero-config diff --git a/docs/user/maintenance.md b/docs/user/maintenance.md index e69bba3..3386582 100644 --- a/docs/user/maintenance.md +++ b/docs/user/maintenance.md @@ -2,6 +2,8 @@ `db/omnigraph/optimize.rs` and `db/omnigraph/repair.rs`. +**Addressing (RFC-010).** `optimize`, `repair`, and `cleanup` are **storage-plane** CLI commands: they run with direct storage access against a positional `URI` or `--target`, never through a server, and reject `--server` / `--graph` or a `--target` that resolves to a remote (`http(s)://`) URL with a declared error. There are no server routes for them by design — to maintain a server-backed graph, run them out-of-band against the graph's storage URI. See the *Command planes* section of [cli-reference.md](cli-reference.md). + ## `optimize_all_tables(db)` — non-destructive - Lance `compact_files()` on every node + edge table on `main`, then **publishes the compacted version to the `__manifest`** so the manifest's `table_version` tracks the compacted Lance HEAD. Reads pin the manifest version, so without this publish compaction would be invisible to readers *and* would break the HEAD-vs-manifest precondition of the next schema apply / strict update/delete ("stale view … refresh and retry"). The publish advances the graph version (a system-attributed commit) only for tables that actually compacted. From d6cf5b298c0e80753e8b95dbdc472fb18d47cbb5 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sun, 14 Jun 2026 01:49:40 +0300 Subject: [PATCH 153/165] feat(cli): plane-grouped --help + clap 4.6.1 (RFC-010 Slice 2) (#220) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * chore(deps): bump clap to 4.6.1 Workspace constraint "4" → "4.6" so the resolver picks up the 4.6 line (a plain `cargo update` stayed on 4.5.x). clap 4.5.58 → 4.6.1 (clap_builder 4.6.0, clap_derive 4.6.1). Minor bump, no API breakage; the workspace builds and all CLI suites pass unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): group --help by plane (RFC-010 Slice 2) Slice 1 declared the planes (the command_plane table + the wrong-plane guard); this makes them visible in `--help`. clap can't print labeled heading rows between subcommand groups (verified against the source — help_heading is args-only, {subcommands} is one flat block), so per the chosen approach: cluster + legend. - Reorder the `Command` enum into plane bands (clap lists subcommands in declaration order): data (query, mutate, load, branch, snapshot, export, commit, schema, graphs) → storage/local-graph ops (init, optimize, repair, cleanup, lint, queries) → control (cluster) → session (policy, embed, login, logout, config, version). No magic display_order numbers — the source order IS the help order, with band comments for readers. The band placement matches `command_plane` (lint/queries are storage-plane: they reject --server), so the help grouping and the guard agree. - Add an `after_help` legend on `Cli` naming the planes. Written to describe the planes (not enumerate every command) so it doesn't drift. Help-polish (post-review): hide the deprecated `ingest` from the list (still a valid command); trim the long `login` and `--as` descriptions to one line each so the columns don't blow up. The behavioral source of truth for planes stays `planes::command_plane`; this ordering is its cosmetic counterpart. Test: `help_groups_commands_by_plane` pins the legend phrase + the cluster ordering (query < optimize < cluster). Doc: a line under cli-reference's *Command planes* section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): qualify mixed-plane commands in the --help legend Addresses the Greptile P2 on #220: the legend placed `schema` entirely in Data and `queries` entirely in Storage, but per `command_plane` the subcommands differ — `schema plan` is storage-plane (rejects --server) and `queries list` is session (no graph). A user reading the legend then running `schema plan --server` would hit a rejection contradicting it. The Commands list is one entry per top-level command (necessarily coarse), so the legend carries the nuance: `schema [plan: storage]` and `queries [list: session]`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- Cargo.lock | 24 +- Cargo.toml | 2 +- crates/omnigraph-cli/src/cli.rs | 370 +++++++++--------- .../omnigraph-cli/tests/cli_schema_config.rs | 32 ++ docs/user/cli-reference.md | 2 + 5 files changed, 239 insertions(+), 191 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 994bb5e..21403b0 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -83,9 +83,9 @@ dependencies = [ [[package]] name = "anstream" -version = "0.6.21" +version = "1.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "43d5b281e737544384e969a5ccad3f1cdd24b48086a0fc1b2a5262a26b8f4f4a" +checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d" dependencies = [ "anstyle", "anstyle-parse", @@ -104,9 +104,9 @@ checksum = "5192cca8006f1fd4f7237516f40fa183bb07f8fbdfedaa0036de5ea9b0b45e78" [[package]] name = "anstyle-parse" -version = "0.2.7" +version = "1.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4e7644824f0aa2c7b9384579234ef10eb7efb6a0deb83f9630a49594dd9c15c2" +checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e" dependencies = [ "utf8parse", ] @@ -1314,9 +1314,9 @@ dependencies = [ [[package]] name = "clap" -version = "4.5.58" +version = "4.6.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "63be97961acde393029492ce0be7a1af7e323e6bae9511ebfac33751be5e6806" +checksum = "1ddb117e43bbf7dacf0a4190fef4d345b9bad68dfc649cb349e7d17d28428e51" dependencies = [ "clap_builder", "clap_derive", @@ -1324,9 +1324,9 @@ dependencies = [ [[package]] name = "clap_builder" -version = "4.5.58" +version = "4.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7f13174bda5dfd69d7e947827e5af4b0f2f94a4a3ee92912fba07a66150f21e2" +checksum = "714a53001bf66416adb0e2ef5ac857140e7dc3a0c48fb28b2f10762fc4b5069f" dependencies = [ "anstream", "anstyle", @@ -1336,9 +1336,9 @@ dependencies = [ [[package]] name = "clap_derive" -version = "4.5.55" +version = "4.6.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a92793da1a46a5f2a02a6f4c46c6496b28c43638adea8306fcb0caa1634f24e5" +checksum = "f2ce8604710f6733aa641a2b3731eaa1e8b3d9973d5e3565da11800813f997a9" dependencies = [ "heck", "proc-macro2", @@ -5323,9 +5323,9 @@ dependencies = [ [[package]] name = "quote" -version = "1.0.44" +version = "1.0.45" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "21b2ebcf727b7760c461f091f9f0f539b77b8e87f2fd88131e7f1b433b3cece4" +checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924" dependencies = [ "proc-macro2", ] diff --git a/Cargo.toml b/Cargo.toml index 76b37e0..56cdde5 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -48,7 +48,7 @@ pest = "2" pest_derive = "2" thiserror = "2" tokio = { version = "1", features = ["rt-multi-thread", "macros", "time", "net", "signal", "sync"] } -clap = { version = "4", features = ["derive"] } +clap = { version = "4.6", features = ["derive"] } serde = { version = "1", features = ["derive"] } serde_json = "1" serde_yaml = "0.9" diff --git a/crates/omnigraph-cli/src/cli.rs b/crates/omnigraph-cli/src/cli.rs index 7b976b4..670ed7c 100644 --- a/crates/omnigraph-cli/src/cli.rs +++ b/crates/omnigraph-cli/src/cli.rs @@ -9,15 +9,24 @@ pub(crate) const DEFAULT_BEARER_TOKEN_ENV: &str = "OMNIGRAPH_BEARER_TOKEN"; #[command(name = "omnigraph")] #[command(about = "Omnigraph graph database CLI")] #[command(version = env!("CARGO_PKG_VERSION"), disable_version_flag = true)] +// Subcommands are listed grouped by plane (clap renders them in declaration +// order). clap can't print labeled headings between subcommand groups, so this +// legend names the planes; the grouping is the variant order in `Command`. +#[command(after_help = "\ +COMMANDS BY PLANE:\n \ +Data — run against a graph, embedded or via --server (query, mutate, load, \ +branch, snapshot, export, commit, schema [plan: storage], graphs).\n \ +Storage — direct storage or local files; reject --server (init, optimize, \ +repair, cleanup, lint, queries [list: session]).\n \ +Control — manage a cluster directory via --config (cluster).\n \ +Session — no graph; local config & tooling (policy, embed, login, logout, \ +config, version).\n\ +See the 'Command planes' section of the CLI reference for which flags apply where.")] pub(crate) struct Cli { - /// Actor identity for direct-engine writes (MR-722). Overrides - /// `cli.actor` from `omnigraph.yaml`. When the configured policy - /// is in effect, Cedar evaluates this actor against the requested - /// action and scope; with policy configured but neither this flag - /// nor `cli.actor` set, the engine-layer footgun guard fires and - /// the write is denied (no silent bypass). Has no effect on remote - /// HTTP writes — those resolve their actor server-side from the - /// bearer token. + /// Actor id for direct-engine writes; overrides `cli.actor`. No effect on + /// remote writes (the server resolves the actor from the bearer token). + /// With a policy configured but no actor set, the write is denied — see + /// docs/user/policy.md. #[arg(long = "as", global = true, value_name = "ACTOR")] pub(crate) as_actor: Option<String>, @@ -38,170 +47,7 @@ pub(crate) struct Cli { #[derive(Debug, Subcommand)] pub(crate) enum Command { - /// Print the CLI version - Version, - /// Store a bearer token for a named server in ~/.omnigraph/credentials - /// (0600). Token from --token or one line on stdin: - /// `echo $TOKEN | omnigraph login prod`. The keyed token applies to - /// requests whose URL matches the server's `url` in the operator - /// config's `servers:` map. - Login { - /// Server name (keys the credential; declare its url under - /// `servers:` in ~/.omnigraph/config.yaml) - name: String, - /// The token. Prefer piping via stdin over this flag (shell - /// history). - #[arg(long)] - token: Option<String>, - #[arg(long)] - json: bool, - }, - /// Legacy-config tooling (RFC-008): split omnigraph.yaml into its - /// two destinations. - Config { - #[command(subcommand)] - command: ConfigCommand, - }, - /// Remove a named server's stored credential. Idempotent. - Logout { - name: String, - #[arg(long)] - json: bool, - }, - /// Generate, clean, or refresh explicit seed embeddings - Embed(EmbedArgs), - /// Initialize a new graph from a schema - Init { - #[arg(long)] - schema: PathBuf, - /// Graph URI (local path or s3://) - uri: String, - /// Overwrite existing schema artifacts at the URI. Without - /// this flag, init refuses to touch a URI that already holds - /// `_schema.pg`, `_schema.ir.json`, or `__schema_state.json` - /// — closes the re-init footgun (MR-668 follow-up). With the - /// flag, the operator opts in to destructive semantics. - #[arg(long)] - force: bool, - }, - /// Load data into a graph (local or remote) - Load { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - data: PathBuf, - /// Target branch (defaults to main). Without --from it must exist. - #[arg(long)] - branch: Option<String>, - /// Base branch to fork --branch from when it doesn't exist yet. - /// Without this flag a missing branch is an error, never a fork. - #[arg(long)] - from: Option<String>, - /// How existing rows are handled: overwrite | append | merge. - /// Required — overwrite is destructive, so there is no default. - #[arg(long)] - mode: CliLoadMode, - #[arg(long)] - json: bool, - }, - /// Deprecated alias of `load --from <base>` (defaults: --mode merge, --from main) - Ingest { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - data: PathBuf, - #[arg(long)] - branch: Option<String>, - #[arg(long)] - from: Option<String>, - #[arg(long, default_value = "merge")] - mode: CliLoadMode, - #[arg(long)] - json: bool, - }, - /// Branch operations - Branch { - #[command(subcommand)] - command: BranchCommand, - }, - /// Schema planning operations - Schema { - #[command(subcommand)] - command: SchemaCommand, - }, - /// Validate queries against a schema (offline) or repo (repo-backed). - /// - /// Canonical name is `lint` (matches the `omnigraph_compiler::lint` - /// module and the `OG-XXX-NNN` lint-code vocabulary). Replaces the - /// deprecated `omnigraph query lint` / `omnigraph query check` / - /// `omnigraph check` invocations — each is kept as an argv-level - /// shim that prints a one-line stderr warning and rewrites to - /// `omnigraph lint`. Aliases are deliberately *not* exposed via - /// clap's `visible_alias` because that would advertise two - /// equivalent canonical names, which agents emit interchangeably - /// (see MR-981). - Lint { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - query: PathBuf, - #[arg(long)] - schema: Option<PathBuf>, - #[arg(long)] - json: bool, - }, - /// Operate on the server-side stored-query registry (`queries:`). - Queries { - #[command(subcommand)] - command: QueriesCommand, - }, - /// Show graph snapshot - Snapshot { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - branch: Option<String>, - #[arg(long)] - json: bool, - }, - /// Export a full graph snapshot as JSONL - Export { - /// Graph URI - uri: Option<String>, - #[arg(long)] - target: Option<String>, - #[arg(long)] - config: Option<PathBuf>, - #[arg(long)] - branch: Option<String>, - #[arg(long, hide = true)] - jsonl: bool, - #[arg(long = "type")] - type_names: Vec<String>, - #[arg(long = "table")] - table_keys: Vec<String>, - }, - /// Commit history operations - Commit { - #[command(subcommand)] - command: CommitCommand, - }, + // ── Data plane ── run against a graph (embedded or via --server). /// Execute a read query against a branch or snapshot. /// /// Canonical read endpoint. The previous name `omnigraph read` is @@ -274,10 +120,115 @@ pub(crate) enum Command { #[arg()] alias_args: Vec<String>, }, - /// Policy administration and diagnostics - Policy { + /// Load data into a graph (local or remote) + Load { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + data: PathBuf, + /// Target branch (defaults to main). Without --from it must exist. + #[arg(long)] + branch: Option<String>, + /// Base branch to fork --branch from when it doesn't exist yet. + /// Without this flag a missing branch is an error, never a fork. + #[arg(long)] + from: Option<String>, + /// How existing rows are handled: overwrite | append | merge. + /// Required — overwrite is destructive, so there is no default. + #[arg(long)] + mode: CliLoadMode, + #[arg(long)] + json: bool, + }, + /// Deprecated alias of `load --from <base>` (defaults: --mode merge, --from main) + #[command(hide = true)] + Ingest { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + data: PathBuf, + #[arg(long)] + branch: Option<String>, + #[arg(long)] + from: Option<String>, + #[arg(long, default_value = "merge")] + mode: CliLoadMode, + #[arg(long)] + json: bool, + }, + /// Branch operations + Branch { #[command(subcommand)] - command: PolicyCommand, + command: BranchCommand, + }, + /// Show graph snapshot + Snapshot { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + branch: Option<String>, + #[arg(long)] + json: bool, + }, + /// Export a full graph snapshot as JSONL + Export { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + branch: Option<String>, + #[arg(long, hide = true)] + jsonl: bool, + #[arg(long = "type")] + type_names: Vec<String>, + #[arg(long = "table")] + table_keys: Vec<String>, + }, + /// Commit history operations + Commit { + #[command(subcommand)] + command: CommitCommand, + }, + /// Schema planning operations + Schema { + #[command(subcommand)] + command: SchemaCommand, + }, + /// Manage graphs on a multi-graph server (MR-668) + Graphs { + #[command(subcommand)] + command: GraphsCommand, + }, + + // ── Storage / local graph ops ── direct storage or local files; reject --server. + /// Initialize a new graph from a schema + Init { + #[arg(long)] + schema: PathBuf, + /// Graph URI (local path or s3://) + uri: String, + /// Overwrite existing schema artifacts at the URI. Without + /// this flag, init refuses to touch a URI that already holds + /// `_schema.pg`, `_schema.ir.json`, or `__schema_state.json` + /// — closes the re-init footgun (MR-668 follow-up). With the + /// flag, the operator opts in to destructive semantics. + #[arg(long)] + force: bool, }, /// Compact small Lance fragments in every table of the graph Optimize { @@ -331,16 +282,79 @@ pub(crate) enum Command { #[arg(long)] json: bool, }, + /// Validate queries against a schema (offline) or repo (repo-backed). + /// + /// Canonical name is `lint` (matches the `omnigraph_compiler::lint` + /// module and the `OG-XXX-NNN` lint-code vocabulary). Replaces the + /// deprecated `omnigraph query lint` / `omnigraph query check` / + /// `omnigraph check` invocations — each is kept as an argv-level + /// shim that prints a one-line stderr warning and rewrites to + /// `omnigraph lint`. Aliases are deliberately *not* exposed via + /// clap's `visible_alias` because that would advertise two + /// equivalent canonical names, which agents emit interchangeably + /// (see MR-981). + Lint { + /// Graph URI + uri: Option<String>, + #[arg(long)] + target: Option<String>, + #[arg(long)] + config: Option<PathBuf>, + #[arg(long)] + query: PathBuf, + #[arg(long)] + schema: Option<PathBuf>, + #[arg(long)] + json: bool, + }, + /// Operate on the server-side stored-query registry (`queries:`). + Queries { + #[command(subcommand)] + command: QueriesCommand, + }, + + // ── Control plane ── manage a cluster directory (--config <dir>). /// Validate and plan read-only cluster configuration. Cluster { #[command(subcommand)] command: ClusterCommand, }, - /// Manage graphs on a multi-graph server (MR-668) - Graphs { + + // ── Session / config ── no graph addressing; local tooling. + /// Policy administration and diagnostics + Policy { #[command(subcommand)] - command: GraphsCommand, + command: PolicyCommand, }, + /// Generate, clean, or refresh explicit seed embeddings + Embed(EmbedArgs), + /// Store a bearer token for a named server (0600 credentials file). Token + /// via --token or piped on stdin; see the CLI reference for token resolution. + Login { + /// Server name (keys the credential; declare its url under + /// `servers:` in ~/.omnigraph/config.yaml) + name: String, + /// The token. Prefer piping via stdin over this flag (shell + /// history). + #[arg(long)] + token: Option<String>, + #[arg(long)] + json: bool, + }, + /// Remove a named server's stored credential. Idempotent. + Logout { + name: String, + #[arg(long)] + json: bool, + }, + /// Legacy-config tooling (RFC-008): split omnigraph.yaml into its + /// two destinations. + Config { + #[command(subcommand)] + command: ConfigCommand, + }, + /// Print the CLI version + Version, } #[derive(Debug, Subcommand)] diff --git a/crates/omnigraph-cli/tests/cli_schema_config.rs b/crates/omnigraph-cli/tests/cli_schema_config.rs index c962321..b15987f 100644 --- a/crates/omnigraph-cli/tests/cli_schema_config.rs +++ b/crates/omnigraph-cli/tests/cli_schema_config.rs @@ -24,6 +24,38 @@ fn version_command_prints_current_cli_version() { ); } +#[test] +fn help_groups_commands_by_plane() { + // RFC-010 Slice 2: `--help` clusters commands by plane (declaration order + // in the Command enum) and explains the planes in an after_help legend. + // Pinned lightly — the legend phrase + the cluster ordering — to avoid + // brittle full-text assertions on clap's help body. + let output = output_success(cli().arg("--help")); + let stdout = stdout_string(&output); + + assert!( + stdout.contains("COMMANDS BY PLANE"), + "plane legend (after_help) missing from --help:\n{stdout}" + ); + + // The Commands list precedes the legend, so first occurrences sit in the + // list and must appear in plane order: a data verb, then a storage verb, + // then the control verb. + let pos = |needle: &str| { + stdout + .find(needle) + .unwrap_or_else(|| panic!("'{needle}' not found in --help:\n{stdout}")) + }; + assert!( + pos("query") < pos("optimize"), + "data commands should be listed before storage commands" + ); + assert!( + pos("optimize") < pos("cluster"), + "storage commands should be listed before the control command" + ); +} + #[test] fn init_creates_graph_successfully_on_missing_local_directory() { let temp = tempdir().unwrap(); diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 3843b2e..5be5ee3 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -43,6 +43,8 @@ These restrictions are enforced and reported, not silent: To maintain a server-backed graph, run the maintenance verbs from a host with storage access against the graph's storage URI (or `--target`), out-of-band from the serving process — there are no server routes for `optimize` / `repair` / `cleanup` by design. +`omnigraph --help` lists commands **clustered by plane** (data → storage → control → session) with a plane legend at the bottom. + ## Config surfaces Two config surfaces with single owners (RFC-007/RFC-008), plus a zero-config From 6144bb18d6452d7277c433bd69b6e6943d326bd4 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sun, 14 Jun 2026 02:52:21 +0300 Subject: [PATCH 154/165] feat(cli): cluster-managed maintenance addressing + init signpost (RFC-010 Slice 3) (#221) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(cluster): cluster_root_for_graph_uri detection helper (RFC-010 Slice 3) Public helper the CLI uses to refuse `init` into a cluster-managed location: given a graph storage URI of the cluster layout (`<root>/graphs/<id>.omni`), return the cluster root if `<root>` holds `__cluster/state.json`, else None. Cheap by construction — a URI that doesn't match the `<root>/graphs/<id>.omni` shape returns None with zero I/O, so ordinary `init` targets never probe storage. Works for file:// and s3:// via the storage adapter. Adds two ClusterStore accessors (`display_root`, `has_state`). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): cluster-managed maintenance addressing + init signpost (RFC-010 Slice 3) Two cluster-graph-aware CLI behaviors, sharing the cluster-resolution path. Maintenance addressing. `optimize`/`repair`/`cleanup` gain `--cluster <dir|s3://…> --cluster-graph <id>`, which resolves the graph's storage URI from the served cluster snapshot (the same truth a `--cluster` server boots from — `read_serving_snapshot*`) and opens it embedded. The operator no longer hand-types `<storage>/graphs/<id>.omni`. A distinct flag is required because the global `--graph` is `requires = server` and means a remote multi-graph id. clap enforces both-or-neither and exclusion with the positional URI / `--target`; an unserved graph errors loudly, pointing at `cluster apply`. init signpost. `init` refuses a cluster-managed positional path (the `<root>/graphs/<id>.omni` layout where `<root>` holds `__cluster/state.json`, detected by `cluster_root_for_graph_uri`) and points at `cluster apply` — graphs in an established cluster are created with ledger/recovery/approvals, not by hand. The check is gated on the path shape, so ordinary `init` does no extra I/O and existing pre-apply cluster-graph inits are unaffected. planes guard remediation now also mentions `--cluster … --cluster-graph …` (the two Slice-1 guard-string tests track it). Docs updated (cli-reference Command planes, maintenance.md, cluster.md §7); the stale "no S3-hosted cluster directories" limitation is dropped (RFC-006 landed it). Tests (cli_cluster.rs, reusing the apply-a-cluster fixture): resolve by id, unknown-id error, `--cluster` requires `--cluster-graph`, init refusal + signpost, and ordinary init still works. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(cli): resolve cluster graphs from the state ledger, not the serving snapshot Addresses the Greptile review on #221. `read_serving_snapshot*` does all-or-nothing serving validation — recovery-sidecar checks plus a digest verify of every catalog payload (query .gq, policy blobs). Using it to resolve a maintenance target coupled `optimize`/`repair`/`cleanup` to the readiness of unrelated resources: a single corrupt policy blob, or a pending recovery sweep, would block the command before it could touch the graph — worst for `repair`, the tool you reach for *when the cluster is degraded*. Add `omnigraph_cluster::resolve_graph_storage_uri(cluster, graph_id)`: read the state ledger, confirm the graph is in the applied revision, return `graph_root(id)` — the URI is deterministically derivable, no catalog validation. The CLI's cluster resolver now calls it. Test: `optimize --cluster … --cluster-graph …` still resolves after the catalog payloads (`__cluster/resources/`) are removed — the ledger-only path is not blocked by degraded/unrelated catalog state. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- crates/omnigraph-cli/src/cli.rs | 21 +++ crates/omnigraph-cli/src/helpers.rs | 31 ++++ crates/omnigraph-cli/src/main.rs | 46 +++++- crates/omnigraph-cli/src/planes.rs | 2 +- crates/omnigraph-cli/tests/cli_cluster.rs | 135 ++++++++++++++++++ crates/omnigraph-cli/tests/cli_data.rs | 2 +- .../omnigraph-cli/tests/cli_schema_config.rs | 2 +- crates/omnigraph-cluster/src/lib.rs | 2 +- crates/omnigraph-cluster/src/serve.rs | 128 +++++++++++++++++ crates/omnigraph-cluster/src/store.rs | 15 ++ docs/user/cli-reference.md | 7 +- docs/user/cluster.md | 22 ++- docs/user/maintenance.md | 2 +- 13 files changed, 401 insertions(+), 14 deletions(-) diff --git a/crates/omnigraph-cli/src/cli.rs b/crates/omnigraph-cli/src/cli.rs index 670ed7c..f5e8c26 100644 --- a/crates/omnigraph-cli/src/cli.rs +++ b/crates/omnigraph-cli/src/cli.rs @@ -238,6 +238,13 @@ pub(crate) enum Command { target: Option<String>, #[arg(long)] config: Option<PathBuf>, + /// Cluster directory or storage-root URI; with --cluster-graph, resolves + /// the graph's storage URI from the served cluster state. + #[arg(long, conflicts_with_all = ["uri", "target"], requires = "cluster_graph")] + cluster: Option<String>, + /// Graph id within --cluster. + #[arg(long, requires = "cluster")] + cluster_graph: Option<String>, #[arg(long)] json: bool, }, @@ -249,6 +256,13 @@ pub(crate) enum Command { target: Option<String>, #[arg(long)] config: Option<PathBuf>, + /// Cluster directory or storage-root URI; with --cluster-graph, resolves + /// the graph's storage URI from the served cluster state. + #[arg(long, conflicts_with_all = ["uri", "target"], requires = "cluster_graph")] + cluster: Option<String>, + /// Graph id within --cluster. + #[arg(long, requires = "cluster")] + cluster_graph: Option<String>, /// Publish verified maintenance drift. Without this flag, repair only /// previews what it would do. #[arg(long)] @@ -268,6 +282,13 @@ pub(crate) enum Command { target: Option<String>, #[arg(long)] config: Option<PathBuf>, + /// Cluster directory or storage-root URI; with --cluster-graph, resolves + /// the graph's storage URI from the served cluster state. + #[arg(long, conflicts_with_all = ["uri", "target"], requires = "cluster_graph")] + cluster: Option<String>, + /// Graph id within --cluster. + #[arg(long, requires = "cluster")] + cluster_graph: Option<String>, /// Number of recent versions to keep per table. Either `--keep` or /// `--older-than` (or both) must be set. #[arg(long)] diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index e9809b5..7e1ca15 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -513,6 +513,37 @@ pub(crate) fn resolve_local_uri( Ok(resolve_local_graph(config, cli_uri, cli_target, operation)?.uri) } +/// Resolve a storage-plane verb's target to a direct storage URI (RFC-010 +/// Slice 3). `--cluster <dir|uri> --cluster-graph <id>` resolves the graph's +/// storage URI from the **served cluster state** (the truth a `--cluster` +/// server serves); otherwise the ordinary positional-URI / `--target` path. +/// clap enforces both-or-neither and exclusion with `uri`/`--target`, so the +/// mismatched arm is defensive. +pub(crate) async fn resolve_storage_uri( + config: &OmnigraphConfig, + cli_uri: Option<String>, + cli_target: Option<&str>, + cluster: Option<&str>, + cluster_graph: Option<&str>, + operation: &str, +) -> Result<String> { + match (cluster, cluster_graph) { + (Some(cluster), Some(graph_id)) => resolve_cluster_graph_uri(cluster, graph_id).await, + (None, None) => resolve_local_uri(config, cli_uri, cli_target, operation), + _ => bail!("--cluster and --cluster-graph must be given together"), + } +} + +/// Look up a graph's storage URI from a cluster's applied state ledger. Uses +/// the lightweight `resolve_graph_storage_uri` (NOT the full serving-snapshot +/// validation), so maintenance — especially `repair` — works even when an +/// unrelated catalog payload is corrupt or a recovery sweep is pending. +async fn resolve_cluster_graph_uri(cluster: &str, graph_id: &str) -> Result<String> { + omnigraph_cluster::resolve_graph_storage_uri(cluster, graph_id) + .await + .map_err(|diagnostic| color_eyre::eyre::eyre!("{}", diagnostic.message)) +} + pub(crate) fn resolve_branch( config: &OmnigraphConfig, cli_branch: Option<String>, diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 6c94132..c3a67d4 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -147,6 +147,16 @@ async fn main() -> Result<()> { } } Command::Init { schema, uri, force } => { + // RFC-010 Slice 3: graphs inside an established cluster are created + // by `cluster apply` (which records ledger/recovery/approvals), not + // by hand-running `init` into the cluster's storage layout. + if let Some(root) = omnigraph_cluster::cluster_root_for_graph_uri(&uri).await { + bail!( + "`{uri}` is inside cluster `{root}`. Graphs in a cluster are created by \ + `cluster apply` (which records ledger, recovery, and approvals), not `init`. \ + Declare the graph in cluster.yaml and run `cluster apply`." + ); + } let schema_source = fs::read_to_string(&schema)?; ensure_local_graph_parent(&uri)?; Omnigraph::init_with_options( @@ -783,10 +793,20 @@ async fn main() -> Result<()> { uri, target, config, + cluster, + cluster_graph, json, } => { let config = load_cli_config(config.as_ref())?; - let uri = resolve_local_uri(&config, uri, target.as_deref(), "optimize")?; + let uri = resolve_storage_uri( + &config, + uri, + target.as_deref(), + cluster.as_deref(), + cluster_graph.as_deref(), + "optimize", + ) + .await?; let db = Omnigraph::open(&uri).await?; let stats = db.optimize().await?; if json { @@ -823,12 +843,22 @@ async fn main() -> Result<()> { uri, target, config, + cluster, + cluster_graph, confirm, force, json, } => { let config = load_cli_config(config.as_ref())?; - let uri = resolve_local_uri(&config, uri, target.as_deref(), "repair")?; + let uri = resolve_storage_uri( + &config, + uri, + target.as_deref(), + cluster.as_deref(), + cluster_graph.as_deref(), + "repair", + ) + .await?; let db = Omnigraph::open(&uri).await?; let stats = db .repair(omnigraph::db::RepairOptions { confirm, force }) @@ -906,13 +936,23 @@ async fn main() -> Result<()> { uri, target, config, + cluster, + cluster_graph, keep, older_than, confirm, json, } => { let config = load_cli_config(config.as_ref())?; - let uri = resolve_local_uri(&config, uri, target.as_deref(), "cleanup")?; + let uri = resolve_storage_uri( + &config, + uri, + target.as_deref(), + cluster.as_deref(), + cluster_graph.as_deref(), + "cleanup", + ) + .await?; let older_than_dur = older_than.as_deref().map(parse_duration_arg).transpose()?; diff --git a/crates/omnigraph-cli/src/planes.rs b/crates/omnigraph-cli/src/planes.rs index 81328d3..7c81dfb 100644 --- a/crates/omnigraph-cli/src/planes.rs +++ b/crates/omnigraph-cli/src/planes.rs @@ -139,7 +139,7 @@ pub(crate) fn guard_addressing(cli: &Cli) -> Result<()> { // required positional URI), so its remediation drops the `--target` half. Plane::Storage => match cli.command { Command::Init { .. } => "Pass a storage URI.", - _ => "Use --target <name> or a storage URI.", + _ => "Use --target <name>, a storage URI, or --cluster <dir> --cluster-graph <id>.", }, Plane::Control => "It operates on a cluster directory (pass --config <dir>).", Plane::Session => "It does not address a graph.", diff --git a/crates/omnigraph-cli/tests/cli_cluster.rs b/crates/omnigraph-cli/tests/cli_cluster.rs index 3b2eed3..9205b84 100644 --- a/crates/omnigraph-cli/tests/cli_cluster.rs +++ b/crates/omnigraph-cli/tests/cli_cluster.rs @@ -950,3 +950,138 @@ graphs: assert!(!leaked.contains("phantom") && !leaked.contains("9999"), "{leaked}"); } + +// ── RFC-010 Slice 3: cluster-managed maintenance addressing + init signpost ── + +/// Stand up an applied, served cluster with the `knowledge` graph and return +/// its directory guard. Mirrors the e2e setup (fixture → init → import → apply). +fn applied_knowledge_cluster() -> tempfile::TempDir { + let temp = tempdir().unwrap(); + write_cluster_config_fixture(temp.path()); + init_cluster_derived_graph(temp.path()); + let import = cluster_json(temp.path(), "import"); + assert_eq!(import["ok"], true, "{import}"); + let apply = cluster_json(temp.path(), "apply"); + assert_eq!(apply["converged"], true, "{apply}"); + temp +} + +#[test] +fn optimize_resolves_a_cluster_graph_by_id() { + let temp = applied_knowledge_cluster(); + // No hand-typed storage path: address the graph by cluster dir + id. + let out = output_success( + cli() + .arg("optimize") + .arg("--cluster") + .arg(temp.path()) + .arg("--cluster-graph") + .arg("knowledge") + .arg("--json"), + ); + let payload = parse_stdout_json(&out); + assert!( + payload["tables"].as_array().is_some(), + "optimize did not run against the resolved cluster graph: {payload}" + ); +} + +#[test] +fn optimize_unknown_cluster_graph_id_errors() { + let temp = applied_knowledge_cluster(); + let out = output_failure( + cli() + .arg("optimize") + .arg("--cluster") + .arg(temp.path()) + .arg("--cluster-graph") + .arg("does-not-exist") + .arg("--json"), + ); + let stderr = String::from_utf8_lossy(&out.stderr); + assert!( + stderr.contains("is not applied in cluster") && stderr.contains("cluster apply"), + "expected an unapplied-graph error pointing at cluster apply; got: {stderr}" + ); +} + +#[test] +fn cluster_flag_requires_cluster_graph() { + // clap enforces both-or-neither. + let out = output_failure( + cli() + .arg("optimize") + .arg("--cluster") + .arg(".") + .arg("--json"), + ); + let stderr = String::from_utf8_lossy(&out.stderr); + assert!( + stderr.contains("cluster-graph") || stderr.contains("required"), + "expected --cluster to require --cluster-graph; got: {stderr}" + ); +} + +#[test] +fn init_refuses_a_cluster_managed_path_and_signposts_cluster_apply() { + let temp = applied_knowledge_cluster(); + // Hand-init a NEW graph into the established cluster's storage layout. + let out = output_failure( + cli() + .arg("init") + .arg("--schema") + .arg(temp.path().join("people.pg")) + .arg(temp.path().join("graphs").join("sneaky.omni")), + ); + let stderr = String::from_utf8_lossy(&out.stderr); + assert!( + stderr.contains("cluster apply"), + "init into a cluster-managed path should signpost `cluster apply`; got: {stderr}" + ); + // And it did not create the graph. + assert!(!temp.path().join("graphs").join("sneaky.omni").exists()); +} + +#[test] +fn init_outside_a_cluster_still_works() { + // Regression guard: ordinary init (no cluster layout) is unaffected. + let temp = tempdir().unwrap(); + let schema = fixture("test.pg"); + let out = output_success( + cli() + .arg("init") + .arg("--schema") + .arg(&schema) + .arg(temp.path().join("plain.omni")), + ); + assert!(stdout_string(&out).contains("initialized")); +} + +#[test] +fn optimize_by_cluster_works_when_catalog_payloads_are_degraded() { + // Robustness (Greptile, #221): maintenance resolves the graph URI from the + // state ledger alone, so an unrelated corrupt/missing catalog payload (or a + // pending recovery sweep) does NOT block it — unlike the full serving-snapshot + // read. This is what keeps `repair --cluster` usable on a degraded cluster. + let temp = applied_knowledge_cluster(); + // Remove the verified catalog payloads (queries/policies) — a serving read + // would refuse with a catalog-payload diagnostic; the ledger-only resolve + // must not care. + let resources = temp.path().join("__cluster").join("resources"); + if resources.exists() { + fs::remove_dir_all(&resources).unwrap(); + } + let out = output_success( + cli() + .arg("optimize") + .arg("--cluster") + .arg(temp.path()) + .arg("--cluster-graph") + .arg("knowledge") + .arg("--json"), + ); + assert!( + parse_stdout_json(&out)["tables"].as_array().is_some(), + "optimize should resolve via the ledger despite degraded catalog payloads" + ); +} diff --git a/crates/omnigraph-cli/tests/cli_data.rs b/crates/omnigraph-cli/tests/cli_data.rs index f558018..8d1f80a 100644 --- a/crates/omnigraph-cli/tests/cli_data.rs +++ b/crates/omnigraph-cli/tests/cli_data.rs @@ -165,7 +165,7 @@ fn optimize_with_server_flag_errors_wrong_plane() { assert!( stderr.contains("`optimize` is a storage-plane command") && stderr.contains("--server/--graph address the data plane and do not apply") - && stderr.contains("Use --target <name> or a storage URI."), + && stderr.contains("Use --target <name>, a storage URI, or --cluster <dir> --cluster-graph <id>."), "wrong-plane guard message not found; got: {stderr}" ); } diff --git a/crates/omnigraph-cli/tests/cli_schema_config.rs b/crates/omnigraph-cli/tests/cli_schema_config.rs index b15987f..f4735c1 100644 --- a/crates/omnigraph-cli/tests/cli_schema_config.rs +++ b/crates/omnigraph-cli/tests/cli_schema_config.rs @@ -121,7 +121,7 @@ fn schema_plan_with_server_flag_errors_wrong_plane() { let stderr = String::from_utf8_lossy(&output.stderr); assert!( stderr.contains("`schema plan` is a storage-plane command") - && stderr.contains("Use --target <name> or a storage URI."), + && stderr.contains("Use --target <name>, a storage URI, or --cluster <dir> --cluster-graph <id>."), "schema plan wrong-plane message not found; got: {stderr}" ); } diff --git a/crates/omnigraph-cluster/src/lib.rs b/crates/omnigraph-cluster/src/lib.rs index 1422dad..0c0f4e6 100644 --- a/crates/omnigraph-cluster/src/lib.rs +++ b/crates/omnigraph-cluster/src/lib.rs @@ -28,7 +28,7 @@ mod store; use store::{ClusterStore, StateLockGuard, StateSnapshot}; pub use types::*; use types::*; -pub use serve::{ServingGraph, ServingPolicy, ServingQuery, ServingSnapshot, read_serving_snapshot, read_serving_snapshot_from_storage}; +pub use serve::{ServingGraph, ServingPolicy, ServingQuery, ServingSnapshot, cluster_root_for_graph_uri, read_serving_snapshot, read_serving_snapshot_from_storage, resolve_graph_storage_uri}; use config::{QueriesDecl, observe_declared_graphs, validate_cluster_header, future_field_diagnostics, initial_import_state, observe_live_graph, preview_schema_migration, state_resource_digests, graph_address, policy_address, query_address, schema_address, load_desired, normalize_policy_target, parse_cluster_config, resolve_config_path, resolve_query_decls, validate_id, validate_query_source}; use diff::{FailedGraphOrigin, ResourceKind, append_policy_binding_changes, approved_resources, classify_changes, compute_approvals, compute_blast_radius, demote_dependents_of_failed_graphs, diff_resources, resource_kind}; use sweep::{mark_approvals_consumed, record_approval_consumed, sweep_recovery_sidecars, tombstone_graph_subtree, warn_pending_recovery_sidecars}; diff --git a/crates/omnigraph-cluster/src/serve.rs b/crates/omnigraph-cluster/src/serve.rs index 4abd0bf..241ab41 100644 --- a/crates/omnigraph-cluster/src/serve.rs +++ b/crates/omnigraph-cluster/src/serve.rs @@ -79,6 +79,87 @@ pub async fn read_serving_snapshot_from_storage( read_snapshot_with_store(backend).await } +/// Cluster root for a graph **storage URI** of the cluster layout +/// (`<root>/graphs/<id>.omni`), if `<root>` is actually a cluster (holds +/// `__cluster/state.json`); otherwise `None`. Used by the CLI to refuse +/// `init` into a cluster-managed location — graphs there are created by +/// `cluster apply`, not `init`. +/// +/// Cheap by construction: a URI that does not match the `<root>/graphs/<id>.omni` +/// shape returns `None` without any I/O, so ordinary `init` targets +/// (`./kb.omni`, `s3://bucket/kb.omni`) never probe storage. Works for +/// `file://` and `s3://` via the storage adapter. +pub async fn cluster_root_for_graph_uri(graph_uri: &str) -> Option<String> { + let root = cluster_root_of_graph_layout(graph_uri)?; + let store = ClusterStore::for_storage_root(&root).ok()?; + store + .has_state() + .await + .then(|| store.display_root().to_string()) +} + +/// Resolve a graph's **storage URI** (`<root>/graphs/<id>.omni`) from a cluster's +/// applied state ledger — the lightweight path for storage-plane maintenance +/// (`optimize`/`repair`/`cleanup`). +/// +/// Unlike [`read_serving_snapshot`], this deliberately does NOT validate catalog +/// payloads or recovery readiness: maintenance only needs the derivable graph +/// root, and must not be blocked by an unrelated corrupt policy/query blob or a +/// pending recovery sweep — a degraded cluster is exactly when an operator +/// reaches for `repair`. It reads the state ledger, confirms the graph is in the +/// applied revision, and returns `graph_root(id)`. +/// +/// `cluster` is a config directory or a storage-root URI (`s3://…`, config-free), +/// mirroring the server's `--cluster` dispatch. +pub async fn resolve_graph_storage_uri(cluster: &str, graph_id: &str) -> Result<String, Diagnostic> { + let backend = if cluster.contains("://") { + ClusterStore::for_storage_root(cluster)? + } else { + ClusterStore::for_config_dir(Path::new(cluster)) + }; + let mut observations = backend.observations(); + let snapshot = backend.read_state(&mut observations).await?; + let state = snapshot.state.ok_or_else(|| { + Diagnostic::error( + "cluster_state_missing", + CLUSTER_STATE_FILE, + format!("cluster `{cluster}` has no applied state; run `cluster apply` first"), + ) + })?; + let address = format!("graph.{graph_id}"); + if !state.applied_revision.resources.contains_key(&address) { + let applied: Vec<&str> = state + .applied_revision + .resources + .keys() + .filter_map(|a| a.strip_prefix("graph.")) + .collect(); + return Err(Diagnostic::error( + "graph_not_applied", + address, + format!( + "graph `{graph_id}` is not applied in cluster `{cluster}` (applied graphs: [{}]); \ + declare it in cluster.yaml and run `cluster apply`, or check the id", + applied.join(", ") + ), + )); + } + Ok(backend.graph_root(graph_id)) +} + +/// Split `<root>/graphs/<id>.omni` → `<root>`, gating on the exact cluster +/// graph-layout shape (a single `<id>` segment, no nested path). `None` for +/// anything else — no I/O is done for non-cluster-shaped URIs. +fn cluster_root_of_graph_layout(graph_uri: &str) -> Option<String> { + let trimmed = graph_uri.trim_end_matches('/'); + let rest = trimmed.strip_suffix(".omni")?; + let (root, id) = rest.rsplit_once("/graphs/")?; + if root.is_empty() || id.is_empty() || id.contains('/') { + return None; + } + Some(root.to_string()) +} + async fn read_snapshot_with_store( backend: ClusterStore, ) -> Result<ServingSnapshot, Vec<Diagnostic>> { @@ -186,3 +267,50 @@ async fn read_snapshot_with_store( }) } +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn graph_layout_gating_does_no_io_for_non_cluster_shapes() { + // Only `<root>/graphs/<id>.omni` matches; everything else is None. + assert_eq!( + cluster_root_of_graph_layout("/data/cluster/graphs/kb.omni").as_deref(), + Some("/data/cluster") + ); + assert_eq!( + cluster_root_of_graph_layout("s3://bucket/prefix/graphs/kb.omni").as_deref(), + Some("s3://bucket/prefix") + ); + assert_eq!(cluster_root_of_graph_layout("./kb.omni"), None); + assert_eq!(cluster_root_of_graph_layout("s3://bucket/kb.omni"), None); + // nested id under graphs/ is not the cluster layout + assert_eq!(cluster_root_of_graph_layout("/c/graphs/a/b.omni"), None); + // not a .omni graph + assert_eq!(cluster_root_of_graph_layout("/c/graphs/kb"), None); + } + + #[tokio::test] + async fn cluster_root_detected_only_when_state_ledger_present() { + let temp = tempfile::tempdir().unwrap(); + let root = temp.path(); + std::fs::create_dir_all(root.join("graphs")).unwrap(); + let graph_uri = format!("{}/graphs/kb.omni", root.to_string_lossy()); + + // No __cluster/state.json yet → not a cluster. + assert_eq!(cluster_root_for_graph_uri(&graph_uri).await, None); + + // Lay down the state ledger → now it's a cluster-managed location. + std::fs::create_dir_all(root.join("__cluster")).unwrap(); + std::fs::write(root.join(CLUSTER_STATE_FILE), "{}").unwrap(); + let detected = cluster_root_for_graph_uri(&graph_uri).await; + assert!(detected.is_some(), "expected cluster root to be detected"); + + // A non-cluster-shaped target never probes and is always None. + assert_eq!( + cluster_root_for_graph_uri(&format!("{}/plain.omni", root.to_string_lossy())).await, + None + ); + } +} + diff --git a/crates/omnigraph-cluster/src/store.rs b/crates/omnigraph-cluster/src/store.rs index 620df96..5129397 100644 --- a/crates/omnigraph-cluster/src/store.rs +++ b/crates/omnigraph-cluster/src/store.rs @@ -154,6 +154,21 @@ impl ClusterStore { } } + /// Display-form storage root (plain local path for `file://`, URI for S3). + pub(crate) fn display_root(&self) -> &str { + &self.display_root + } + + /// Whether this root holds the cluster state ledger (`__cluster/state.json`) + /// — i.e. is an actual cluster, not just any directory. Probed via the + /// adapter (`file://` or `s3://`), failures read as "not a cluster". + pub(crate) async fn has_state(&self) -> bool { + self.adapter + .exists(&self.uri(CLUSTER_STATE_FILE)) + .await + .unwrap_or(false) + } + /// `read_text_versioned`, returning None for a missing object (probed /// via `exists` — the engine error type doesn't discriminate NotFound). async fn read_versioned_opt(&self, uri: &str) -> Result<Option<(String, String)>, String> { diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 5be5ee3..e9216b8 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -33,15 +33,16 @@ Top-level command families and subcommands. Graph-targeting commands accept a po Every command lives on one **plane**, which determines how it reaches a graph and which addressing flags apply (RFC-010): - **Data plane** — `query`, `mutate`, `load`, `ingest`, `branch *`, `snapshot`, `export`, `commit *`, `schema show`, `schema apply` (and `graphs list`, remote-only today). Run against a graph **embedded or via a server**: accept a positional `URI` / `--target` / `--server` (+ `--graph` for multi-graph servers). -- **Storage / maintenance plane** — `init`, `optimize`, `repair`, `cleanup`, `schema plan`, `queries validate`, `lint`. Run with **direct storage access** (`file://` / `s3://`), never through a server. They accept a positional `URI` or `--target`, but **not** `--server` / `--graph`, and a `--target` that resolves to a remote (`http(s)://`) server is rejected. (`init` takes only a positional `URI` today — no `--target`.) +- **Storage / maintenance plane** — `init`, `optimize`, `repair`, `cleanup`, `schema plan`, `queries validate`, `lint`. Run with **direct storage access** (`file://` / `s3://`), never through a server. They accept a positional `URI` or `--target`, but **not** `--server` / `--graph`, and a `--target` that resolves to a remote (`http(s)://`) server is rejected. (`init` takes only a positional `URI` today — no `--target`.) `optimize` / `repair` / `cleanup` also accept **`--cluster <dir|s3://…> --cluster-graph <id>`**, which resolves the graph's storage URI from the served cluster state (so you needn't know the `<storage>/graphs/<id>.omni` layout). - **Control plane** — `cluster *`. Operates on a cluster directory via `--config <dir>`. These restrictions are enforced and reported, not silent: -- A data-plane addressing flag on a non-data verb fails loudly, e.g.: ``optimize is a storage-plane command; --server/--graph address the data plane and do not apply. Use --target <name> or a storage URI.`` +- A data-plane addressing flag on a non-data verb fails loudly, e.g.: ``optimize is a storage-plane command; --server/--graph address the data plane and do not apply. Use --target <name>, a storage URI, or --cluster <dir> --cluster-graph <id>.`` - A storage-plane verb pointed at a remote target fails loudly, e.g.: ``optimize is a storage-plane command and needs direct storage access; the resolved target is a remote server (https://…). Pass the graph's file:// or s3:// URI.`` +- `init` into an **established cluster's** storage layout (`<root>/graphs/<id>.omni` where `<root>` holds `__cluster/state.json`) is refused — graphs in a cluster are created by `cluster apply` (which records ledger / recovery / approvals), not `init`. -To maintain a server-backed graph, run the maintenance verbs from a host with storage access against the graph's storage URI (or `--target`), out-of-band from the serving process — there are no server routes for `optimize` / `repair` / `cleanup` by design. +To maintain a server-backed graph, run the maintenance verbs from a host with storage access against the graph's storage URI (`--target`, or `--cluster … --cluster-graph …`), out-of-band from the serving process — there are no server routes for `optimize` / `repair` / `cleanup` by design. `omnigraph --help` lists commands **clustered by plane** (data → storage → control → session) with a plane legend at the bottom. diff --git a/docs/user/cluster.md b/docs/user/cluster.md index 0d6dac5..93b5ddf 100644 --- a/docs/user/cluster.md +++ b/docs/user/cluster.md @@ -251,12 +251,28 @@ with an in-flight apply. loads). It just no longer describes the deployment — a server boots from one source or the other, never a merge of both. +## 7. Maintaining a cluster graph + +Storage maintenance (`optimize` / `repair` / `cleanup`) is **not** a control-plane +operation — it runs out-of-band, with direct storage access, against the graph's +roots. Address a cluster graph by name instead of hand-typing its storage path: + +```bash +omnigraph optimize --cluster ./company-brain --cluster-graph knowledge +omnigraph cleanup --cluster ./company-brain --cluster-graph knowledge --keep 10 --confirm +# --cluster also takes the storage-root URI directly (config-free): +omnigraph optimize --cluster s3://bucket/clusters/company-brain --cluster-graph knowledge +``` + +The graph's storage URI is resolved from the **served cluster state** (the same +truth a `--cluster` server boots from); a graph that hasn't been applied yet is +not resolvable. Run these from a host with storage access — there are no server +routes for them. Conversely, **`init` refuses** a cluster-managed path: graphs in +a cluster are created by `cluster apply`, not by hand. + ## What the control plane does not do (yet) - **No hot reload** — applied changes serve on the next restart. -- **No S3-hosted cluster directories** — the config dir, ledger, catalog, - and derived graph roots are local-filesystem paths today. (Individual - *graphs* on S3 are a server feature outside cluster mode.) - **No data operations** — rows move through `omnigraph load / ingest / mutate` against the graph roots, with branches and merges as usual. - **Stored-query exposure is all-or-nothing per cluster** — every applied diff --git a/docs/user/maintenance.md b/docs/user/maintenance.md index 3386582..8b97657 100644 --- a/docs/user/maintenance.md +++ b/docs/user/maintenance.md @@ -2,7 +2,7 @@ `db/omnigraph/optimize.rs` and `db/omnigraph/repair.rs`. -**Addressing (RFC-010).** `optimize`, `repair`, and `cleanup` are **storage-plane** CLI commands: they run with direct storage access against a positional `URI` or `--target`, never through a server, and reject `--server` / `--graph` or a `--target` that resolves to a remote (`http(s)://`) URL with a declared error. There are no server routes for them by design — to maintain a server-backed graph, run them out-of-band against the graph's storage URI. See the *Command planes* section of [cli-reference.md](cli-reference.md). +**Addressing (RFC-010).** `optimize`, `repair`, and `cleanup` are **storage-plane** CLI commands: they run with direct storage access against a positional `URI`, `--target`, or **`--cluster <dir|s3://…> --cluster-graph <id>`** (which resolves the graph's storage URI from the served cluster state, so you needn't know the `<storage>/graphs/<id>.omni` layout). They never run through a server, and reject `--server` / `--graph` or a `--target` that resolves to a remote (`http(s)://`) URL with a declared error. There are no server routes for them by design — to maintain a server-backed graph, run them out-of-band against the graph's storage URI. See the *Command planes* section of [cli-reference.md](cli-reference.md). ## `optimize_all_tables(db)` — non-destructive From 8726ca92ecc72bc226f5f77f1afee41c39f501c9 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sun, 14 Jun 2026 03:32:16 +0300 Subject: [PATCH 155/165] feat: canonical POST /load, deprecate /ingest (RFC-009 Phase 5) (#222) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(server): canonical POST /load, deprecate /ingest (RFC-009 Phase 5) The CLI's non-deprecated `load` verb rode the deprecated `/ingest` route, so `/ingest`'s eventual removal would silently break it. Add a canonical `/load`, mirroring the shipped `/mutate`↔`/change` and `/query`↔`/read` pattern. - Extract `server_ingest`'s body into a shared `run_ingest` (branch-exists / fork-if-`from`, Cedar auth, admission, `load_as`, `IngestOutput` mapping). - `server_load` (canonical) → `run_ingest`, `Json<IngestOutput>`. - `server_ingest` (deprecated) → `run_ingest` + `#[deprecated]` + RFC 9745/8288 `Deprecation: true` / `Link: </load>; rel="successor-version"` headers. - Router mounts `/load` (same 32 MB body limit) beside `/ingest`; OpenAPI `paths(...)` gains `server_load` and flags `server_ingest` deprecated. `/load` reuses `IngestRequest`/`IngestOutput`, exactly as canonical `/mutate` reuses `Change*` — a DTO rename is a separate, larger change (out of scope). openapi.json regenerated. Tests: openapi `/load` present + not deprecated, `/ingest` deprecated, `/load` bearer-secured; data_routes `/load` happy path + `/ingest` deprecation headers. Existing `/ingest` route tests stay green (the shim is unchanged). Docs: server.md endpoint table; RFC-009 Phase 5 marked landed (incl. the hand-mount-vs-utoipa-axum registration finding). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): point remote load at /load (RFC-009 Phase 5) `GraphClient::load`'s remote arm now POSTs to the canonical `/load` route instead of the deprecated `/ingest`; the deprecated `ingest` verb keeps riding `/ingest`. `parity_load` exercises `/load` on the remote arm (its documented flip); the matrix exclusions comment is updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- crates/omnigraph-cli/src/client.rs | 5 +- crates/omnigraph-cli/tests/parity_matrix.rs | 6 +- crates/omnigraph-server/src/handlers.rs | 140 +++++++++++++------ crates/omnigraph-server/src/lib.rs | 18 ++- crates/omnigraph-server/tests/data_routes.rs | 77 ++++++++++ crates/omnigraph-server/tests/openapi.rs | 28 ++++ docs/dev/rfc-009-unify-access-paths.md | 21 +-- docs/user/server.md | 3 +- openapi.json | 84 ++++++++++- 9 files changed, 325 insertions(+), 57 deletions(-) diff --git a/crates/omnigraph-cli/src/client.rs b/crates/omnigraph-cli/src/client.rs index 5ca6351..d9e7726 100644 --- a/crates/omnigraph-cli/src/client.rs +++ b/crates/omnigraph-cli/src/client.rs @@ -304,10 +304,13 @@ impl GraphClient { token, } => { let data = std::fs::read_to_string(data)?; + // RFC-009 Phase 5: the canonical `load` verb targets the + // canonical `/load` route (the deprecated `ingest` verb below + // still rides `/ingest`). let output = remote_json::<IngestOutput>( http, Method::POST, - remote_url(base_url, "/ingest"), + remote_url(base_url, "/load"), Some(serde_json::to_value(IngestRequest { branch: Some(branch.to_string()), from: from.map(ToOwned::to_owned), diff --git a/crates/omnigraph-cli/tests/parity_matrix.rs b/crates/omnigraph-cli/tests/parity_matrix.rs index b65c46e..65a584f 100644 --- a/crates/omnigraph-cli/tests/parity_matrix.rs +++ b/crates/omnigraph-cli/tests/parity_matrix.rs @@ -265,9 +265,9 @@ fn parity_errors_share_exit_codes() { // // - `graphs list`: server-only today; becomes Both-capability when the // embedded arm enumerates the cluster catalog (RFC-009 open Q3, answered). -// - `ingest`: deprecated alias of load; the remote `load` arm itself rides -// the deprecated /ingest route today (RFC-009 Phase 5 flips it to /load — -// this matrix's `parity_load` row is where that flip becomes visible). +// - `ingest`: deprecated alias of load; its remote arm rides the deprecated +// /ingest route. The canonical `load` verb targets `/load` (RFC-009 Phase 5, +// landed) — `parity_load` exercises it on the remote arm. // - `init`, `optimize`, `repair`, `cleanup`, `cluster *`: storage-plane by // design (must work with the server down); Phase 4 declares this. #[allow(dead_code)] diff --git a/crates/omnigraph-server/src/handlers.rs b/crates/omnigraph-server/src/handlers.rs index 2ead0e3..94f4743 100644 --- a/crates/omnigraph-server/src/handlers.rs +++ b/crates/omnigraph-server/src/handlers.rs @@ -1183,46 +1183,22 @@ pub(crate) async fn server_schema_apply( Ok(Json(schema_apply_output(handle.uri.as_str(), result))) } -#[utoipa::path( - post, - path = "/ingest", - tag = "mutations", - operation_id = "ingest", - request_body = IngestRequest, - responses( - (status = 200, description = "Ingest results", body = IngestOutput), - (status = 400, description = "Bad request", body = ErrorOutput), - (status = 401, description = "Unauthorized", body = ErrorOutput), - (status = 403, description = "Forbidden", body = ErrorOutput), - (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), - ), - security(("bearer_token" = [])), -)] -/// Bulk-load NDJSON data into a branch. -/// -/// `data` is NDJSON with one record per line. `mode` controls behavior on -/// existing rows: `merge` upserts by id (default), `append` blindly inserts, -/// `overwrite` replaces table contents. Branch creation is opt-in by -/// presence of `from`: with `from` set, a missing `branch` is created from -/// it; without `from`, `branch` must already exist — a missing branch is a -/// 404, never an implicit fork. **Destructive** when `mode` is `overwrite` -/// or when the load produces conflicting writes. -pub(crate) async fn server_ingest( - State(state): State<AppState>, - Extension(handle): Extension<Arc<GraphHandle>>, - actor: Option<Extension<ResolvedActor>>, - Json(request): Json<IngestRequest>, -) -> std::result::Result<Json<IngestOutput>, ApiError> { +/// Shared body for `POST /load` (canonical) and `POST /ingest` (deprecated): +/// branch-exists / fork-if-`from` check, Cedar authorization, admission, the +/// bulk `load_as`, and the `IngestOutput` mapping. +async fn run_ingest( + state: AppState, + handle: Arc<GraphHandle>, + actor: Option<&ResolvedActor>, + request: IngestRequest, +) -> std::result::Result<IngestOutput, ApiError> { let branch = request.branch.unwrap_or_else(|| "main".to_string()); let from = request.from; let mode = request.mode.unwrap_or(omnigraph::loader::LoadMode::Merge); let actor_arc = actor - .as_ref() - .map(|Extension(actor)| Arc::clone(&actor.actor_id)) + .map(|actor| Arc::clone(&actor.actor_id)) .unwrap_or_else(|| Arc::<str>::from("anonymous")); - let actor_id = actor - .as_ref() - .map(|Extension(actor)| actor.actor_id.as_ref()); + let actor_id = actor.map(|actor| actor.actor_id.as_ref()); let branch_exists = { let db = &handle.engine; @@ -1244,7 +1220,7 @@ pub(crate) async fn server_ingest( ))); } Some(from) => authorize_request( - actor.as_ref().map(|Extension(actor)| actor), + actor, handle.policy.as_deref(), PolicyRequest { action: PolicyAction::BranchCreate, @@ -1255,7 +1231,7 @@ pub(crate) async fn server_ingest( } } authorize_request( - actor.as_ref().map(|Extension(actor)| actor), + actor, handle.policy.as_deref(), PolicyRequest { action: PolicyAction::Change, @@ -1276,12 +1252,98 @@ pub(crate) async fn server_ingest( .map_err(ApiError::from_omni)? }; - Ok(Json(ingest_output( + Ok(ingest_output( handle.uri.as_str(), &result, mode, actor_id.map(str::to_string), - ))) + )) +} + +#[utoipa::path( + post, + path = "/load", + tag = "mutations", + operation_id = "load", + request_body = IngestRequest, + responses( + (status = 200, description = "Load results", body = IngestOutput), + (status = 400, description = "Bad request", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +/// Bulk-load NDJSON data into a branch (canonical load endpoint). +/// +/// `data` is NDJSON with one record per line. `mode` controls behavior on +/// existing rows: `merge` upserts by id (default), `append` blindly inserts, +/// `overwrite` replaces table contents. Branch creation is opt-in by +/// presence of `from`: with `from` set, a missing `branch` is created from +/// it; without `from`, `branch` must already exist — a missing branch is a +/// 404, never an implicit fork. **Destructive** when `mode` is `overwrite` +/// or when the load produces conflicting writes. +/// +/// The legacy `POST /ingest` route has identical semantics and is kept as a +/// deprecated alias. +pub(crate) async fn server_load( + State(state): State<AppState>, + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Json(request): Json<IngestRequest>, +) -> std::result::Result<Json<IngestOutput>, ApiError> { + Ok(Json( + run_ingest( + state, + handle, + actor.as_ref().map(|Extension(actor)| actor), + request, + ) + .await?, + )) +} + +#[utoipa::path( + post, + path = "/ingest", + tag = "mutations", + operation_id = "ingest", + request_body = IngestRequest, + responses( + (status = 200, description = "Load results (response includes `Deprecation: true` + `Link: </load>; rel=\"successor-version\"`)", body = IngestOutput), + (status = 400, description = "Bad request", body = ErrorOutput), + (status = 401, description = "Unauthorized", body = ErrorOutput), + (status = 403, description = "Forbidden", body = ErrorOutput), + (status = 429, description = "Per-actor admission cap exceeded; honor `Retry-After` header", body = ErrorOutput), + ), + security(("bearer_token" = [])), +)] +#[deprecated(note = "use POST /load instead; /ingest is kept indefinitely for back-compat")] +/// **Deprecated** — use [`POST /load`](#tag/mutations/operation/load) instead. +/// +/// Bulk-load NDJSON data into a branch. Behavior is unchanged; the route is +/// kept indefinitely for back-compat. New integrations should target +/// `POST /load`, which has identical semantics. Responses from this route +/// include `Deprecation: true` and `Link: </load>; rel="successor-version"` +/// headers per RFC 9745 / RFC 8288 so SDKs and proxies can surface the signal. +pub(crate) async fn server_ingest( + State(state): State<AppState>, + Extension(handle): Extension<Arc<GraphHandle>>, + actor: Option<Extension<ResolvedActor>>, + Json(request): Json<IngestRequest>, +) -> std::result::Result<([(HeaderName, HeaderValue); 2], Json<IngestOutput>), ApiError> { + let output = run_ingest( + state, + handle, + actor.as_ref().map(|Extension(actor)| actor), + request, + ) + .await?; + Ok(( + deprecation_headers("</load>; rel=\"successor-version\""), + Json(output), + )) } #[utoipa::path( diff --git a/crates/omnigraph-server/src/lib.rs b/crates/omnigraph-server/src/lib.rs index 3bde2a7..3761e91 100644 --- a/crates/omnigraph-server/src/lib.rs +++ b/crates/omnigraph-server/src/lib.rs @@ -107,7 +107,10 @@ fn hash_bearer_token(token: &str) -> BearerTokenHash { handlers::server_invoke_query, handlers::server_schema_apply, handlers::server_schema_get, - handlers::server_ingest, + handlers::server_load, + // deprecated; the #[deprecated] attribute on the handler surfaces as + // `deprecated: true` on the OpenAPI operation. + #[allow(deprecated)] handlers::server_ingest, handlers::server_branch_list, handlers::server_branch_create, handlers::server_branch_delete, @@ -934,9 +937,20 @@ pub fn build_app(state: AppState) -> Router { .route("/queries/{name}", post(server_invoke_query)) .route("/schema", get(server_schema_get)) .route("/schema/apply", post(server_schema_apply)) + .route( + "/load", + post(server_load).layer(DefaultBodyLimit::max(INGEST_REQUEST_BODY_LIMIT_BYTES)), + ) + // /ingest is the deprecated alias of /load; its handler carries + // #[deprecated] (OpenAPI operation flagged) and emits RFC 9745 + // Deprecation + RFC 8288 Link headers. Suppress the call-site warning. .route( "/ingest", - post(server_ingest).layer(DefaultBodyLimit::max(INGEST_REQUEST_BODY_LIMIT_BYTES)), + post({ + #[allow(deprecated)] + server_ingest + }) + .layer(DefaultBodyLimit::max(INGEST_REQUEST_BODY_LIMIT_BYTES)), ) .route( "/branches", diff --git a/crates/omnigraph-server/tests/data_routes.rs b/crates/omnigraph-server/tests/data_routes.rs index cef2f9a..5dc47c1 100644 --- a/crates/omnigraph-server/tests/data_routes.rs +++ b/crates/omnigraph-server/tests/data_routes.rs @@ -620,6 +620,83 @@ async fn change_endpoint_emits_deprecation_headers() { ); } +#[tokio::test(flavor = "multi_thread")] +async fn load_endpoint_loads_into_existing_branch() { + // Canonical bulk-load endpoint (RFC-009 Phase 5). Same wire shape as + // /ingest, no deprecation signal. + let (_temp, app) = app_for_loaded_graph().await; + let request = IngestRequest { + branch: Some("main".to_string()), + from: None, + mode: Some(LoadMode::Merge), + data: r#"{"type":"Person","data":{"name":"Loaded","age":7}}"#.to_string(), + }; + let response = app + .clone() + .oneshot( + Request::builder() + .uri("/load") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&request).unwrap())) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::OK); + assert!( + response.headers().get("deprecation").is_none(), + "POST /load must not advertise itself as deprecated" + ); + let body_bytes = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + let body: Value = serde_json::from_slice(&body_bytes).unwrap(); + assert_eq!(body["branch"], "main"); + assert_eq!(body["tables"][0]["table_key"], "node:Person"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn ingest_endpoint_emits_deprecation_headers() { + // `/ingest` is the deprecated alias of `/load` (RFC-009 Phase 5): flagged + // at runtime per RFC 9745 (`Deprecation: true`) + RFC 8288 (`Link: </load>; + // rel="successor-version"`). The OpenAPI side is covered by + // `openapi_ingest_is_deprecated` in tests/openapi.rs. + let (_temp, app) = app_for_loaded_graph().await; + let request = IngestRequest { + branch: Some("main".to_string()), + from: None, + mode: Some(LoadMode::Merge), + data: r#"{"type":"Person","data":{"name":"Legacyer","age":33}}"#.to_string(), + }; + let response = app + .clone() + .oneshot( + Request::builder() + .uri("/ingest") + .method(Method::POST) + .header("content-type", "application/json") + .body(Body::from(serde_json::to_vec(&request).unwrap())) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + response + .headers() + .get("deprecation") + .and_then(|v| v.to_str().ok()), + Some("true"), + "POST /ingest must advertise `Deprecation: true` (RFC 9745)" + ); + assert_eq!( + response.headers().get("link").and_then(|v| v.to_str().ok()), + Some("</load>; rel=\"successor-version\""), + "POST /ingest must point at /load via `Link` rel=successor-version (RFC 8288)" + ); +} + #[tokio::test(flavor = "multi_thread")] async fn read_endpoint_emits_deprecation_headers() { // `/read` is kept indefinitely for byte-stable back-compat but flagged diff --git a/crates/omnigraph-server/tests/openapi.rs b/crates/omnigraph-server/tests/openapi.rs index 3d13e74..ac1fb59 100644 --- a/crates/omnigraph-server/tests/openapi.rs +++ b/crates/omnigraph-server/tests/openapi.rs @@ -172,6 +172,7 @@ const EXPECTED_PATHS: &[&str] = &[ "/queries/{name}", "/schema", "/schema/apply", + "/load", "/ingest", "/branches", "/branches/{branch}", @@ -300,6 +301,32 @@ fn openapi_ingest_is_post() { assert!(doc["paths"]["/ingest"]["post"].is_object()); } +#[test] +fn openapi_load_is_not_deprecated() { + // RFC-009 Phase 5: /load is the canonical bulk-load endpoint. + let doc = openapi_json(); + assert!(doc["paths"]["/load"]["post"].is_object()); + let deprecated = doc["paths"]["/load"]["post"] + .get("deprecated") + .and_then(serde_json::Value::as_bool) + .unwrap_or(false); + assert!( + !deprecated, + "/load is the canonical load endpoint and must not be deprecated" + ); +} + +#[test] +fn openapi_ingest_is_deprecated() { + // RFC-009 Phase 5: /ingest is now the deprecated alias of /load. + let doc = openapi_json(); + assert_eq!( + doc["paths"]["/ingest"]["post"]["deprecated"], + serde_json::Value::Bool(true), + "/ingest must be flagged deprecated now that /load is canonical" + ); +} + #[test] fn openapi_branches_supports_get_and_post() { let doc = openapi_json(); @@ -705,6 +732,7 @@ fn protected_endpoints_reference_bearer_token_security() { ("/schema/apply", "post"), ("/queries", "get"), ("/queries/{name}", "post"), + ("/load", "post"), ("/ingest", "post"), ("/export", "post"), ("/snapshot", "get"), diff --git a/docs/dev/rfc-009-unify-access-paths.md b/docs/dev/rfc-009-unify-access-paths.md index 8b8251b..9b2d842 100644 --- a/docs/dev/rfc-009-unify-access-paths.md +++ b/docs/dev/rfc-009-unify-access-paths.md @@ -161,15 +161,20 @@ and cluster commands must work with the server down) explicit in code. "Server" targets include operator-config named servers (RFC-007), not only literal `http(s)://` URIs. -### Phase 5 — Route alignment +### Phase 5 — Route alignment (landed) -Add a canonical `/load` endpoint (the handler already exists behind the -`/ingest` shim); point `RemoteClient` at it; keep `/ingest` on its existing -deprecation path. While here, check whether the server uses `utoipa-axum`'s -router-coupled registration (`OpenApiRouter`/`routes!`); if it hand-mounts -routes beside `#[utoipa::path]` annotations, prefer migrating registration so -path annotations and mount points are the same declaration (the modularization -already hit one orphaned-attribute incident of exactly this class). +Added a canonical `POST /load` (shared `run_ingest` body; the deprecated +`/ingest` is now a thin alias carrying `#[deprecated]` + RFC 9745/8288 +`Deprecation`/`Link: </load>` headers, exactly mirroring `/mutate`↔`/change`) +and pointed the CLI's remote `load` arm at it; `/ingest` stays on its +deprecation path. `/load` reuses `IngestRequest`/`IngestOutput` (as canonical +`/mutate` reuses `Change*`); a DTO rename is a separate change. + +Registration finding: the server **hand-mounts** routes (`.route(...)`) beside a +manual `#[openapi(paths(...))]` list, not `utoipa-axum`'s `OpenApiRouter`/ +`routes!`. This PR followed the existing manual pattern (one `.route` + one +`paths(...)` entry + the `#[utoipa::path]` annotation) rather than migrating +registration — the migration is a worthwhile but orthogonal cleanup, deferred. ## Non-goals diff --git a/docs/user/server.md b/docs/user/server.md index 391b7ae..a2e6705 100644 --- a/docs/user/server.md +++ b/docs/user/server.md @@ -56,7 +56,8 @@ Per-graph endpoints — same body shape across modes; URLs differ: | POST | `/queries/{name}` | `/graphs/{id}/queries/{name}` | bearer + `invoke_query` (+ `change` for a stored mutation) | invoke a named query from the `queries:` registry; deny == 404 | `server_invoke_query` | | GET | `/schema` | `/graphs/{id}/schema` | bearer + `read` | get current `.pg` source | `server_schema_get` | | POST | `/schema/apply` | `/graphs/{id}/schema/apply` | bearer + `schema_apply` (target=`main`) | migrate | `server_schema_apply` | -| POST | `/ingest` | `/graphs/{id}/ingest` | bearer + `branch_create` (only when `from` is set and the branch is created) + `change` | bulk load; branch creation is opt-in via `from` — without it a missing `branch` is a 404, never an implicit fork | `server_ingest` (32 MB body limit) | +| POST | `/load` | `/graphs/{id}/load` | bearer + `branch_create` (only when `from` is set and the branch is created) + `change` | bulk load (canonical); branch creation is opt-in via `from` — without it a missing `branch` is a 404, never an implicit fork | `server_load` (32 MB body limit) | +| POST | `/ingest` | `/graphs/{id}/ingest` | bearer + `branch_create` (only when `from` is set and the branch is created) + `change` | **deprecated** alias of `/load` (carries `Deprecation: true` + `Link: </load>; rel="successor-version"`) | `server_ingest` (32 MB body limit) | | GET | `/branches` | `/graphs/{id}/branches` | bearer + `read` | list branches | `server_branch_list` | | POST | `/branches` | `/graphs/{id}/branches` | bearer + `branch_create` | create | `server_branch_create` | | DELETE | `/branches/{branch}` | `/graphs/{id}/branches/{branch}` | bearer + `branch_delete` | delete | `server_branch_delete` | diff --git a/openapi.json b/openapi.json index 6e3dd03..4f0309f 100644 --- a/openapi.json +++ b/openapi.json @@ -670,8 +670,8 @@ "tags": [ "mutations" ], - "summary": "Bulk-load NDJSON data into a branch.", - "description": "`data` is NDJSON with one record per line. `mode` controls behavior on\nexisting rows: `merge` upserts by id (default), `append` blindly inserts,\n`overwrite` replaces table contents. Branch creation is opt-in by\npresence of `from`: with `from` set, a missing `branch` is created from\nit; without `from`, `branch` must already exist — a missing branch is a\n404, never an implicit fork. **Destructive** when `mode` is `overwrite`\nor when the load produces conflicting writes.", + "summary": "**Deprecated** — use [`POST /load`](#tag/mutations/operation/load) instead.", + "description": "Bulk-load NDJSON data into a branch. Behavior is unchanged; the route is\nkept indefinitely for back-compat. New integrations should target\n`POST /load`, which has identical semantics. Responses from this route\ninclude `Deprecation: true` and `Link: </load>; rel=\"successor-version\"`\nheaders per RFC 9745 / RFC 8288 so SDKs and proxies can surface the signal.", "operationId": "ingest", "requestBody": { "content": { @@ -685,7 +685,85 @@ }, "responses": { "200": { - "description": "Ingest results", + "description": "Load results (response includes `Deprecation: true` + `Link: </load>; rel=\"successor-version\"`)", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/IngestOutput" + } + } + } + }, + "400": { + "description": "Bad request", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + }, + "401": { + "description": "Unauthorized", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + }, + "403": { + "description": "Forbidden", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + }, + "429": { + "description": "Per-actor admission cap exceeded; honor `Retry-After` header", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOutput" + } + } + } + } + }, + "deprecated": true, + "security": [ + { + "bearer_token": [] + } + ] + } + }, + "/load": { + "post": { + "tags": [ + "mutations" + ], + "summary": "Bulk-load NDJSON data into a branch (canonical load endpoint).", + "description": "`data` is NDJSON with one record per line. `mode` controls behavior on\nexisting rows: `merge` upserts by id (default), `append` blindly inserts,\n`overwrite` replaces table contents. Branch creation is opt-in by\npresence of `from`: with `from` set, a missing `branch` is created from\nit; without `from`, `branch` must already exist — a missing branch is a\n404, never an implicit fork. **Destructive** when `mode` is `overwrite`\nor when the load produces conflicting writes.\n\nThe legacy `POST /ingest` route has identical semantics and is kept as a\ndeprecated alias.", + "operationId": "load", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/IngestRequest" + } + } + }, + "required": true + }, + "responses": { + "200": { + "description": "Load results", "content": { "application/json": { "schema": { From d46e50dd6d0cbf01d54f68730e52c56991b9b477 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sun, 14 Jun 2026 13:52:14 +0300 Subject: [PATCH 156/165] docs(user): restructure user docs into topic sections (Phase 1) (#223) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Move the 23 flat docs/user/*.md files into topic subdirectories so the user guide is organized by area (schema, queries, search, branching, cli, operations, clusters, concepts, reference) instead of a flat list. This is a pure structural move — whole files relocated, every cross-doc link recomputed, no prose rewrites or content splits (those follow in Phase 2). - 19 `git mv`s (install.md, deployment.md stay top-level); history preserved (renames detected at 92–100% similarity). - All intra-doc links, AGENTS.md's topic table (52 pointers), and the docs/dev + docs/releases back-links recomputed via relpath from each file's new location. - docs/user/index.md rewritten as a sectioned nav hub. - Fixed 5 doc-path references in Rust (comments + two user-facing server settings error strings) to point at the new locations. Verified: zero broken .md links across tracked docs; check-agents-md.sh green (with the untracked scratch docs set aside); touched crates build. Note: the public site (omnigraph-web) imports docs/ via a flat-only script; its import-docs.mjs needs a subdir-aware update before the next re-sync. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- AGENTS.md | 40 +++++----- crates/omnigraph-cli/src/cli.rs | 2 +- crates/omnigraph-server/src/settings.rs | 4 +- crates/omnigraph/src/exec/query.rs | 2 +- crates/omnigraph/tests/ordering.rs | 2 +- docs/dev/architecture.md | 4 +- docs/dev/cluster-config-specs.md | 4 +- docs/dev/execution.md | 2 +- docs/dev/index.md | 24 +++--- docs/dev/invariants.md | 16 ++-- docs/dev/rfc-001-queries-envelope-mcp.md | 2 +- docs/dev/writes.md | 2 +- docs/user/{ => branching}/changes.md | 0 .../index.md} | 2 +- docs/user/{ => branching}/transactions.md | 12 +-- docs/user/{cli.md => cli/index.md} | 0 .../{cli-reference.md => cli/reference.md} | 8 +- .../{cluster-config.md => clusters/config.md} | 2 +- docs/user/{cluster.md => clusters/index.md} | 10 +-- docs/user/{ => concepts}/storage.md | 0 docs/user/deployment.md | 2 +- docs/user/index.md | 73 ++++++++++++------- docs/user/{ => operations}/audit.md | 0 docs/user/{ => operations}/errors.md | 2 +- docs/user/{ => operations}/maintenance.md | 8 +- docs/user/{ => operations}/policy.md | 0 docs/user/{ => operations}/server.md | 6 +- .../{query-language.md => queries/index.md} | 4 +- docs/user/{ => reference}/constants.md | 0 .../{schema-language.md => schema/index.md} | 0 docs/user/{schema-lint.md => schema/lint.md} | 0 docs/user/{ => search}/embeddings.md | 0 docs/user/{ => search}/indexes.md | 2 +- 33 files changed, 126 insertions(+), 109 deletions(-) rename docs/user/{ => branching}/changes.md (100%) rename docs/user/{branches-commits.md => branching/index.md} (95%) rename docs/user/{ => branching}/transactions.md (93%) rename docs/user/{cli.md => cli/index.md} (100%) rename docs/user/{cli-reference.md => cli/reference.md} (97%) rename docs/user/{cluster-config.md => clusters/config.md} (99%) rename docs/user/{cluster.md => clusters/index.md} (97%) rename docs/user/{ => concepts}/storage.md (100%) rename docs/user/{ => operations}/audit.md (100%) rename docs/user/{ => operations}/errors.md (92%) rename docs/user/{ => operations}/maintenance.md (92%) rename docs/user/{ => operations}/policy.md (100%) rename docs/user/{ => operations}/server.md (94%) rename docs/user/{query-language.md => queries/index.md} (97%) rename docs/user/{ => reference}/constants.md (100%) rename docs/user/{schema-language.md => schema/index.md} (100%) rename docs/user/{schema-lint.md => schema/lint.md} (100%) rename docs/user/{ => search}/embeddings.md (100%) rename docs/user/{ => search}/indexes.md (92%) diff --git a/AGENTS.md b/AGENTS.md index ae6e744..79e4fa7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,32 +73,32 @@ Full diagram and concurrency model: [docs/dev/architecture.md](docs/dev/architec | **Lance docs index — fetch upstream Lance docs by problem domain** | **[docs/dev/lance.md](docs/dev/lance.md)** | | **Test coverage map — what's covered, what helpers to reuse, before-every-task checklist** | **[docs/dev/testing.md](docs/dev/testing.md)** | | Architecture, L1/L2 framing, concurrency model | [docs/dev/architecture.md](docs/dev/architecture.md) | -| Storage layout, `__manifest` schema, URI schemes, S3 env vars | [docs/user/storage.md](docs/user/storage.md) | -| `.pg` schema language, types, constraints, annotations, migration planning | [docs/user/schema-language.md](docs/user/schema-language.md) | -| Schema-lint codes (`OG-XXX-NNN`), families, severity, suppression | [docs/user/schema-lint.md](docs/user/schema-lint.md) | -| `.gq` query language, MATCH/RETURN/ORDER, search funcs, mutations, IR ops, lint codes | [docs/user/query-language.md](docs/user/query-language.md) | -| Indexes (BTREE / inverted / vector / graph topology) | [docs/user/indexes.md](docs/user/indexes.md) | -| Embeddings (compiler + engine clients, env vars, `@embed`) | [docs/user/embeddings.md](docs/user/embeddings.md) | -| Branches, commit graph, snapshots, system branches | [docs/user/branches-commits.md](docs/user/branches-commits.md) | -| Transactions and atomicity (per-query atomic; branches as multi-query transactions) | [docs/user/transactions.md](docs/user/transactions.md) | +| Storage layout, `__manifest` schema, URI schemes, S3 env vars | [docs/user/storage.md](docs/user/concepts/storage.md) | +| `.pg` schema language, types, constraints, annotations, migration planning | [docs/user/schema-language.md](docs/user/schema/index.md) | +| Schema-lint codes (`OG-XXX-NNN`), families, severity, suppression | [docs/user/schema-lint.md](docs/user/schema/lint.md) | +| `.gq` query language, MATCH/RETURN/ORDER, search funcs, mutations, IR ops, lint codes | [docs/user/query-language.md](docs/user/queries/index.md) | +| Indexes (BTREE / inverted / vector / graph topology) | [docs/user/indexes.md](docs/user/search/indexes.md) | +| Embeddings (compiler + engine clients, env vars, `@embed`) | [docs/user/embeddings.md](docs/user/search/embeddings.md) | +| Branches, commit graph, snapshots, system branches | [docs/user/branches-commits.md](docs/user/branching/index.md) | +| Transactions and atomicity (per-query atomic; branches as multi-query transactions) | [docs/user/transactions.md](docs/user/branching/transactions.md) | | Direct-publish write path (staging, D2, recovery sidecars; the former Run state machine) | [docs/dev/writes.md](docs/dev/writes.md) | | Three-way merge and conflict kinds | [docs/dev/merge.md](docs/dev/merge.md) | -| Diff / change feed (`diff_between`, `diff_commits`) | [docs/user/changes.md](docs/user/changes.md) | +| Diff / change feed (`diff_between`, `diff_commits`) | [docs/user/changes.md](docs/user/branching/changes.md) | | Query execution, mutation execution, bulk loader, `load` vs `ingest` | [docs/dev/execution.md](docs/dev/execution.md) | -| `optimize` (compaction) and `cleanup` (version GC) | [docs/user/maintenance.md](docs/user/maintenance.md) | -| Cluster operator guide (deploy/manage clusters, approvals, recovery, serving) | [docs/user/cluster.md](docs/user/cluster.md) | -| Cedar policy actions, scopes, CLI | [docs/user/policy.md](docs/user/policy.md) | -| HTTP server endpoints, auth, error model, body limits | [docs/user/server.md](docs/user/server.md) | -| CLI quick-start | [docs/user/cli.md](docs/user/cli.md) | -| CLI command surface and config schemas (`~/.omnigraph/config.yaml`, legacy `omnigraph.yaml`) | [docs/user/cli-reference.md](docs/user/cli-reference.md) | -| Audit / actor tracking | [docs/user/audit.md](docs/user/audit.md) | -| Error taxonomy and result serialization | [docs/user/errors.md](docs/user/errors.md) | +| `optimize` (compaction) and `cleanup` (version GC) | [docs/user/maintenance.md](docs/user/operations/maintenance.md) | +| Cluster operator guide (deploy/manage clusters, approvals, recovery, serving) | [docs/user/cluster.md](docs/user/clusters/index.md) | +| Cedar policy actions, scopes, CLI | [docs/user/policy.md](docs/user/operations/policy.md) | +| HTTP server endpoints, auth, error model, body limits | [docs/user/server.md](docs/user/operations/server.md) | +| CLI quick-start | [docs/user/cli.md](docs/user/cli/index.md) | +| CLI command surface and config schemas (`~/.omnigraph/config.yaml`, legacy `omnigraph.yaml`) | [docs/user/cli-reference.md](docs/user/cli/reference.md) | +| Audit / actor tracking | [docs/user/audit.md](docs/user/operations/audit.md) | +| Error taxonomy and result serialization | [docs/user/errors.md](docs/user/operations/errors.md) | | Install (binary / Homebrew / source / channels) | [docs/user/install.md](docs/user/install.md) | | Deployment (binary / container / RustFS bootstrap / auth / build variants) | [docs/user/deployment.md](docs/user/deployment.md) | | CI / release workflows | [docs/dev/ci.md](docs/dev/ci.md) | | Code ownership (CODEOWNERS source of truth, roles, regeneration) | [docs/dev/codeowners.md](docs/dev/codeowners.md) | | Branch protection policy (declarative, applied via `scripts/apply-branch-protection.sh`) | [docs/dev/branch-protection.md](docs/dev/branch-protection.md) | -| Constants & tunables cheat sheet | [docs/user/constants.md](docs/user/constants.md) | +| Constants & tunables cheat sheet | [docs/user/constants.md](docs/user/reference/constants.md) | | Per-version release notes | [docs/releases/](docs/releases/) | --- @@ -257,7 +257,7 @@ omnigraph policy explain --actor act-alice --action change --branch main | Per-query atomic writes | — | In-memory `MutationStaging.pending` accumulator + `stage_*` / `commit_staged` per touched table at end-of-query + publisher CAS via `commit_with_expected` (single manifest commit per `mutate_as` / `load`); D₂ parse-time rule keeps inserts/updates and deletes from mixing | | Three-way row-level merge | — | `OrderedTableCursor` + `StagedTableWriter`, structured `MergeConflictKind` | | Change feeds | — | `diff_between` / `diff_commits` with manifest fast path + ID streaming | -| Cedar policy | — | Per-graph actions plus server-scoped actions (see [docs/user/policy.md](docs/user/policy.md) for the current list), branch / target_branch / protected scopes, validate/test/explain CLI. **Engine-wide enforcement** (MR-722): every `_as` writer (`apply_schema_as`, `mutate_as`, `load_as` — the deprecated `ingest_as` shims route through it — `branch_create_as` / `branch_create_from_as`, `branch_delete_as`, `branch_merge_as`) calls `Omnigraph::enforce(action, scope, actor)` — HTTP, CLI, embedded SDK all hit the same gate. | +| Cedar policy | — | Per-graph actions plus server-scoped actions (see [docs/user/policy.md](docs/user/operations/policy.md) for the current list), branch / target_branch / protected scopes, validate/test/explain CLI. **Engine-wide enforcement** (MR-722): every `_as` writer (`apply_schema_as`, `mutate_as`, `load_as` — the deprecated `ingest_as` shims route through it — `branch_create_as` / `branch_create_from_as`, `branch_delete_as`, `branch_merge_as`) calls `Omnigraph::enforce(action, scope, actor)` — HTTP, CLI, embedded SDK all hit the same gate. | | HTTP server | — | Axum, OpenAPI via utoipa, bearer auth (SHA-256, AWS Secrets Manager option), `authorize_request` at the HTTP boundary (resolves bearer→actor, applies admission control), NDJSON streaming export, **multi-graph mode (v0.6.0+) with cluster routes + read-only `GET /graphs` enumeration + per-graph + server-level Cedar policies. Multi-graph boots from a cluster directory (`--cluster`) or the legacy `omnigraph.yaml`; add/remove graphs via `cluster apply` (or by editing the legacy file) and restarting.** | | CLI with config | — | two-surface config (team `cluster.yaml` dir + per-operator `~/.omnigraph/config.yaml`; legacy `omnigraph.yaml` deprecated per RFC-008), aliases, multi-format output (json/jsonl/csv/kv/table) | | Audit / actor tracking | — | `_as` write APIs + actor map in commit graph | @@ -282,7 +282,7 @@ Rules: 7. **Re-verify before recommending.** If you cite a flag, env var, endpoint, or constant to the user or in code, grep for it in source first. Memory and docs go stale; the code is authoritative. 8. **Keep AGENTS.md short.** This file is always loaded into agent context, so every added line has a recurring context-window cost. Prefer pointers and terse invariants here; put detail in `docs/`. 9. **Keep AGENTS.md a map, not an encyclopedia.** New deep content goes into `docs/`. Add an entry to "Where to find each topic" instead of pasting prose into this file. The "Always-on rules" section is the exception — it's for invariants that should always be in scope. -10. **Re-read on schema/query/IR changes.** Edits to `schema.pest`, `query.pest`, `ir/lower.rs`, `query/typecheck.rs`, or `query/lint.rs` should trigger a re-read of [docs/user/schema-language.md](docs/user/schema-language.md), [docs/user/query-language.md](docs/user/query-language.md), and [docs/dev/execution.md](docs/dev/execution.md) to confirm they still describe reality. +10. **Re-read on schema/query/IR changes.** Edits to `schema.pest`, `query.pest`, `ir/lower.rs`, `query/typecheck.rs`, or `query/lint.rs` should trigger a re-read of [docs/user/schema-language.md](docs/user/schema/index.md), [docs/user/query-language.md](docs/user/queries/index.md), and [docs/dev/execution.md](docs/dev/execution.md) to confirm they still describe reality. 11. **Always make smaller commits.** Each commit does one thing, compiles, and passes tests; mechanical refactors land separately from the behavior changes they enable. 12. **Test-first for bug fixes.** When fixing an identified bug, write a regression test that reproduces the failure first. Confirm it fails against the current code with the predicted symptom (not an unrelated error). Then land the fix in a separate commit and confirm the test turns green. The test commit lands just before the fix commit so the red → green pair is visible in `git log` and a reviewer can check out the test commit alone and reproduce the failure. 13. **Correct by design over symptomatic patches.** When a bug surfaces, identify the root cause and make the fix correct by construction. Don't patch the symptom. If the design admits the bug class, the fix is to close the class, not to add a guard around the latest instance. A symptomatic patch is acceptable only as a stop-gap, with an explicit note in the commit message and a follow-up issue tracking the design fix. diff --git a/crates/omnigraph-cli/src/cli.rs b/crates/omnigraph-cli/src/cli.rs index f5e8c26..28010d2 100644 --- a/crates/omnigraph-cli/src/cli.rs +++ b/crates/omnigraph-cli/src/cli.rs @@ -26,7 +26,7 @@ pub(crate) struct Cli { /// Actor id for direct-engine writes; overrides `cli.actor`. No effect on /// remote writes (the server resolves the actor from the bearer token). /// With a policy configured but no actor set, the write is denied — see - /// docs/user/policy.md. + /// docs/user/operations/policy.md. #[arg(long = "as", global = true, value_name = "ACTOR")] pub(crate) as_actor: Option<String>, diff --git a/crates/omnigraph-server/src/settings.rs b/crates/omnigraph-server/src/settings.rs index 59c437b..890c5da 100644 --- a/crates/omnigraph-server/src/settings.rs +++ b/crates/omnigraph-server/src/settings.rs @@ -327,14 +327,14 @@ pub fn classify_server_runtime_state( "server has no bearer tokens and no policy file configured. This is a fully \ open server — pass `--unauthenticated` (or set OMNIGRAPH_UNAUTHENTICATED=1) \ if you actually want that, otherwise configure bearer tokens (see \ - docs/user/server.md) and/or `policy.file` in omnigraph.yaml." + docs/user/operations/server.md) and/or `policy.file` in omnigraph.yaml." ), (false, false, true) => Ok(ServerRuntimeState::Open), (true, false, _) => Ok(ServerRuntimeState::DefaultDeny), (false, true, _) => bail!( "policy file is configured but no bearer tokens — every request would 401 \ because no token can ever match. Configure at least one bearer token (see \ - docs/user/server.md), or remove the policy file. To deny all unauthenticated \ + docs/user/operations/server.md), or remove the policy file. To deny all unauthenticated \ traffic deliberately, configure tokens plus a deny-all Cedar rule — that \ produces meaningful 403s with policy-decision logging instead of silent 401s." ), diff --git a/crates/omnigraph/src/exec/query.rs b/crates/omnigraph/src/exec/query.rs index 5bc18f2..ae2a824 100644 --- a/crates/omnigraph/src/exec/query.rs +++ b/crates/omnigraph/src/exec/query.rs @@ -763,7 +763,7 @@ fn traversal_indexed_override() -> Option<bool> { /// Max source-row frontier for which Expand uses the BTREE-indexed path. /// Larger frontiers fall back to the in-memory CSR (dense / whole-graph). See -/// `docs/user/constants.md`. +/// `docs/user/reference/constants.md`. const DEFAULT_EXPAND_INDEXED_MAX_FRONTIER: usize = 1024; /// Max hop count for the indexed path (each hop is one indexed scan; very deep /// traversals fan out toward whole-graph and are better served by CSR). diff --git a/crates/omnigraph/tests/ordering.rs b/crates/omnigraph/tests/ordering.rs index 4e9296b..2684b1c 100644 --- a/crates/omnigraph/tests/ordering.rs +++ b/crates/omnigraph/tests/ordering.rs @@ -7,7 +7,7 @@ //! keys yield a TOTAL, deterministic order (and `ORDER … LIMIT` is //! deterministic). NULL placement is `nulls_first = !descending` (NULLs first //! under ASC, last under DESC). Both are documented in -//! `docs/user/query-language.md`. +//! `docs/user/queries/index.md`. mod helpers; diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md index 9d31545..4e8d3c6 100644 --- a/docs/dev/architecture.md +++ b/docs/dev/architecture.md @@ -10,7 +10,7 @@ Three views, increasing zoom: 2. **Layer view** — the eight-layer stack inside one OmniGraph process. 3. **Component zoom-ins** — what's inside each layer. -For runtime flows (read query, mutation), see [`docs/dev/execution.md`](execution.md). For the on-disk layout of a graph, see [`docs/user/storage.md`](../user/storage.md). +For runtime flows (read query, mutation), see [`docs/dev/execution.md`](execution.md). For the on-disk layout of a graph, see [`docs/user/storage.md`](../user/concepts/storage.md). L1 (orange in the diagrams) is what we inherit from Lance; L2 (blue) is what OmniGraph adds. The L1/L2 framing is also called out in prose at the bottom of this doc. @@ -280,7 +280,7 @@ flowchart LR eng --> wq ``` -The server applies Cedar policy at the HTTP boundary today. The roadmap, called out in [docs/dev/invariants.md](invariants.md) as a known gap, is to push policy into the planner as predicates. After Cedar, mutating handlers go through `WorkloadController` (per-actor admission cap + byte budget; PR 2 / MR-686) before reaching the engine. The engine itself holds an `Arc<WriteQueueManager>` so concurrent mutations on the same `(table, branch)` serialize at the queue, while disjoint keys run in parallel — see [docs/user/server.md](../user/server.md) "Per-actor admission control" and [docs/dev/writes.md](writes.md). The CLI bypasses the HTTP layer (and admission) and calls the engine API directly. +The server applies Cedar policy at the HTTP boundary today. The roadmap, called out in [docs/dev/invariants.md](invariants.md) as a known gap, is to push policy into the planner as predicates. After Cedar, mutating handlers go through `WorkloadController` (per-actor admission cap + byte budget; PR 2 / MR-686) before reaching the engine. The engine itself holds an `Arc<WriteQueueManager>` so concurrent mutations on the same `(table, branch)` serialize at the queue, while disjoint keys run in parallel — see [docs/user/server.md](../user/operations/server.md) "Per-actor admission control" and [docs/dev/writes.md](writes.md). The CLI bypasses the HTTP layer (and admission) and calls the engine API directly. Code paths: diff --git a/docs/dev/cluster-config-specs.md b/docs/dev/cluster-config-specs.md index d248be2..b9dfde8 100644 --- a/docs/dev/cluster-config-specs.md +++ b/docs/dev/cluster-config-specs.md @@ -3,11 +3,11 @@ **Status:** Draft / thinking-in-progress **Type:** Architecture direction **Date:** 2026-06-07 -**Relationship:** generalizes today's `omnigraph.yaml` graph/query/policy configuration surface ([CLI reference](../user/cli-reference.md), [server docs](../user/server.md)) into a future cluster control plane. The distilled rules are in [cluster-axioms.md](cluster-axioms.md); detailed downstream implementation spec and blast-radius assessment in [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md). This is a proposed architecture, not an implemented RFC. +**Relationship:** generalizes today's `omnigraph.yaml` graph/query/policy configuration surface ([CLI reference](../user/cli/reference.md), [server docs](../user/operations/server.md)) into a future cluster control plane. The distilled rules are in [cluster-axioms.md](cluster-axioms.md); detailed downstream implementation spec and blast-radius assessment in [cluster-config-implementation-spec.md](cluster-config-implementation-spec.md). This is a proposed architecture, not an implemented RFC. > **Implementation status.** The examples below describe the full target schema. > Stage 2B only accepts the read-only subset documented in -> [cluster-config.md](../user/cluster-config.md). Future-phase fields such as +> [cluster-config.md](../user/clusters/config.md). Future-phase fields such as > `env_file`, `apply`, `providers`, `pipelines`, `embeddings`, `ui`, `aliases`, > and `bindings` are intentionally rejected with typed diagnostics until their > reconciler semantics are implemented. diff --git a/docs/dev/execution.md b/docs/dev/execution.md index 0e8e3fc..237a7af 100644 --- a/docs/dev/execution.md +++ b/docs/dev/execution.md @@ -177,4 +177,4 @@ For all three modes, a mid-load failure (RI / cardinality violation, validation ## Embeddings during load -If a node type has `@embed` properties, the loader calls the engine embedding client (Gemini, RETRIEVAL_DOCUMENT) per row to populate the vector column. See [embeddings.md](../user/embeddings.md). +If a node type has `@embed` properties, the loader calls the engine embedding client (Gemini, RETRIEVAL_DOCUMENT) per row to populate the vector column. See [embeddings.md](../user/search/embeddings.md). diff --git a/docs/dev/index.md b/docs/dev/index.md index 2b2ddf0..ac8c07f 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -20,13 +20,13 @@ constraints. User-facing behavior should still be documented through | Area | Read | |---|---| | System structure, L1/L2 framing, component diagrams | [architecture.md](architecture.md) | -| On-disk layout, manifest schema, URI behavior | [storage.md](../user/storage.md) | +| On-disk layout, manifest schema, URI behavior | [storage.md](../user/concepts/storage.md) | | Direct-publish writes, D2, staged writes, recovery sidecars | [writes.md](writes.md) | | Query execution, mutation execution, loader flow | [execution.md](execution.md) | -| Index lifecycle and graph topology indexes | [indexes.md](../user/indexes.md) | -| Branch and commit internals | [branches-commits.md](../user/branches-commits.md) | +| Index lifecycle and graph topology indexes | [indexes.md](../user/search/indexes.md) | +| Branch and commit internals | [branches-commits.md](../user/branching/index.md) | | Three-way merge implementation and conflicts | [merge.md](merge.md) | -| Diff/change-feed implementation | [changes.md](../user/changes.md) | +| Diff/change-feed implementation | [changes.md](../user/branching/changes.md) | | Branch protection policy | [branch-protection.md](branch-protection.md) | | CODEOWNERS source of truth | [codeowners.md](codeowners.md) | @@ -34,14 +34,14 @@ constraints. User-facing behavior should still be documented through | Area | Read | |---|---| -| Schema grammar, catalog, migration planner | [schema-language.md](../user/schema-language.md) | -| Query grammar, IR, lints, mutation restrictions | [query-language.md](../user/query-language.md) | -| Embedding client and `@embed` integration | [embeddings.md](../user/embeddings.md) | -| Cedar policy surface and server gating | [policy.md](../user/policy.md) | -| Server auth, OpenAPI, endpoint handlers | [server.md](../user/server.md) | -| Error taxonomy and serialization | [errors.md](../user/errors.md) | -| Constants and tunables | [constants.md](../user/constants.md) | -| Transaction model public contract | [transactions.md](../user/transactions.md) | +| Schema grammar, catalog, migration planner | [schema-language.md](../user/schema/index.md) | +| Query grammar, IR, lints, mutation restrictions | [query-language.md](../user/queries/index.md) | +| Embedding client and `@embed` integration | [embeddings.md](../user/search/embeddings.md) | +| Cedar policy surface and server gating | [policy.md](../user/operations/policy.md) | +| Server auth, OpenAPI, endpoint handlers | [server.md](../user/operations/server.md) | +| Error taxonomy and serialization | [errors.md](../user/operations/errors.md) | +| Constants and tunables | [constants.md](../user/reference/constants.md) | +| Transaction model public contract | [transactions.md](../user/branching/transactions.md) | ## Project Operations diff --git a/docs/dev/invariants.md b/docs/dev/invariants.md index b3bcfaf..a0bcc6d 100644 --- a/docs/dev/invariants.md +++ b/docs/dev/invariants.md @@ -58,7 +58,7 @@ Use it this way: branch they read even when index coverage is partial. Expensive index work should converge from manifest state instead of extending the critical write path. Scalar staged index builds and vector inline residuals are documented - in [writes.md](writes.md) and [indexes.md](../user/indexes.md). + in [writes.md](writes.md) and [indexes.md](../user/search/indexes.md). 8. **Schema identity survives renames.** Accepted schema identity must remain stable across type and property renames. Rename support belongs in migration @@ -100,14 +100,14 @@ Use it this way: |---|---|---| | Multi-table commit | Manifest CAS plus recovery sidecars; not a single Lance primitive | [writes.md](writes.md), [architecture.md](architecture.md) | | Constructive mutations | In-memory `MutationStaging`, one end-of-query table commit per touched table, then one manifest publish | [writes.md](writes.md), [execution.md](execution.md) | -| Deletes | Inline-commit residual; delete-only queries allowed, mixed insert/update/delete rejected by D2 | [query-language.md](../user/query-language.md), [writes.md](writes.md) | -| Branch delete | Manifest is the single authority, flipped atomically first; per-table forks + commit-graph branch are derived state, reclaimed best-effort (`force_delete_branch`) with the `cleanup` reconciler as the guaranteed backstop. Reusing a name whose reclaim failed before `cleanup` surfaces an actionable error | [branches-commits.md](../user/branches-commits.md), [maintenance.md](../user/maintenance.md) | -| Schema validation | Type checks, required fields, defaults, edge endpoint checks, and edge cardinality are enforced on write paths | [schema-language.md](../user/schema-language.md), [execution.md](execution.md) | -| Unique constraints | Intra-batch and write-path checks exist; intake and branch-merge derive the composite key through one shared function (`loader::composite_unique_key`, a separator-free `Vec<String>` tuple) and fail loudly on an un-keyable column type rather than silently exempting it; full cross-version uniqueness against already-committed rows is still a gap | [schema-language.md](../user/schema-language.md) | +| Deletes | Inline-commit residual; delete-only queries allowed, mixed insert/update/delete rejected by D2 | [query-language.md](../user/queries/index.md), [writes.md](writes.md) | +| Branch delete | Manifest is the single authority, flipped atomically first; per-table forks + commit-graph branch are derived state, reclaimed best-effort (`force_delete_branch`) with the `cleanup` reconciler as the guaranteed backstop. Reusing a name whose reclaim failed before `cleanup` surfaces an actionable error | [branches-commits.md](../user/branching/index.md), [maintenance.md](../user/operations/maintenance.md) | +| Schema validation | Type checks, required fields, defaults, edge endpoint checks, and edge cardinality are enforced on write paths | [schema-language.md](../user/schema/index.md), [execution.md](execution.md) | +| Unique constraints | Intra-batch and write-path checks exist; intake and branch-merge derive the composite key through one shared function (`loader::composite_unique_key`, a separator-free `Vec<String>` tuple) and fail loudly on an un-keyable column type rather than silently exempting it; full cross-version uniqueness against already-committed rows is still a gap | [schema-language.md](../user/schema/index.md) | | Storage trait | `TableStorage` (via `db.storage()`) is staged-only; the inline-commit residuals (`delete_where`, `create_vector_index`) are split onto a separate sealed `InlineCommitResidual` trait reached via `db.storage_inline_residual()` (MR-854), so §1 holds by construction; capability/stat surfaces are roadmap | [writes.md](writes.md), [architecture.md](architecture.md) | -| Index lifecycle | `ensure_indices` is explicit today; reconciler-based convergence is roadmap | [indexes.md](../user/indexes.md), [maintenance.md](../user/maintenance.md) | -| Traversal IDs | Runtime still builds `TypeIndex`; Lance stable row-id based graph IDs are roadmap | [architecture.md](architecture.md), [query-language.md](../user/query-language.md) | -| Auth | Bearer token hashing and server-side actor resolution are implemented at the HTTP boundary | [server.md](../user/server.md), [policy.md](../user/policy.md) | +| Index lifecycle | `ensure_indices` is explicit today; reconciler-based convergence is roadmap | [indexes.md](../user/search/indexes.md), [maintenance.md](../user/operations/maintenance.md) | +| Traversal IDs | Runtime still builds `TypeIndex`; Lance stable row-id based graph IDs are roadmap | [architecture.md](architecture.md), [query-language.md](../user/queries/index.md) | +| Auth | Bearer token hashing and server-side actor resolution are implemented at the HTTP boundary | [server.md](../user/operations/server.md), [policy.md](../user/operations/policy.md) | | Tests | Tempdir-backed Lance tests are the current substrate; the storage adapter has an in-memory backend for adapter-level contract tests, but Lance datasets bypass it | [testing.md](testing.md) | The branch-delete reconciler is authority-derived: it reclaims orphaned forks diff --git a/docs/dev/rfc-001-queries-envelope-mcp.md b/docs/dev/rfc-001-queries-envelope-mcp.md index b5d62d4..94d15e8 100644 --- a/docs/dev/rfc-001-queries-envelope-mcp.md +++ b/docs/dev/rfc-001-queries-envelope-mcp.md @@ -348,4 +348,4 @@ Callers move at their own pace. The envelope upgrades + URL rename ship in v0.6. - RFC 8288 (`Link` relations, `successor-version`) - MCP spec: [modelcontextprotocol.io](https://modelcontextprotocol.io) - [invariants.md](./invariants.md) — substrate boundaries this work respects -- [../user/server.md](../user/server.md) — current HTTP surface (post-MR-656 picks up the `/query`+`/mutate` rename and deprecation) +- [../user/server.md](../user/operations/server.md) — current HTTP surface (post-MR-656 picks up the `/query`+`/mutate` rename and deprecation) diff --git a/docs/dev/writes.md b/docs/dev/writes.md index 82d6ba8..c3511e0 100644 --- a/docs/dev/writes.md +++ b/docs/dev/writes.md @@ -305,7 +305,7 @@ success and one failure. The losing writer's error is `ManifestConflictDetails::ExpectedVersionMismatch { table_key, expected, actual }`. The HTTP server maps this to **409 Conflict** with body `{"error": "...", "code": "conflict", "manifest_conflict": { "table_key": -"...", "expected": N, "actual": M }}` — see [docs/user/server.md](../user/server.md). +"...", "expected": N, "actual": M }}` — see [docs/user/server.md](../user/operations/server.md). ## Audit diff --git a/docs/user/changes.md b/docs/user/branching/changes.md similarity index 100% rename from docs/user/changes.md rename to docs/user/branching/changes.md diff --git a/docs/user/branches-commits.md b/docs/user/branching/index.md similarity index 95% rename from docs/user/branches-commits.md rename to docs/user/branching/index.md index a4044cb..17d17b2 100644 --- a/docs/user/branches-commits.md +++ b/docs/user/branching/index.md @@ -10,7 +10,7 @@ OmniGraph builds *graph branches* on top by branching every sub-table coherently - `branch_create(name)` / `branch_create_from(target, name)` — disallowed name `main`; fails if branch exists; ensures the schema-apply lock is idle. Atomic and authority-first like `branch_delete`: it flips the `__manifest` branch (authority), then creates the derived commit-graph branch, force-dropping any orphaned commit-graph ref left by an incomplete prior delete (the manifest branch is fresh, so a same-named commit-graph branch is provably a zombie). If commit-graph creation fails, the manifest branch is rolled back so the name never half-exists. - `branch_list()` — returns public branches, **filters the internal** `__schema_apply_lock__` branch. -- `branch_delete(name)` — refuses if there are descendants on the branch, or if it is the current branch. The manifest is the single authority for branch existence: deletion flips the `__manifest` branch ref first (one atomic op), after which the branch is gone from every snapshot. The owned per-table forks and the commit-graph branch are derived state, reclaimed best-effort with `force_delete_branch` after the flip. A failure during that reclaim (transient object-store error) does not fail the call or block the authority flip; the leftover forks are unreachable orphans that the [`cleanup`](maintenance.md) reconciler converges. One consequence: if a delete's best-effort reclaim fails, reusing that branch name before the next `cleanup` surfaces a clear error pointing at `cleanup` (the stale fork would otherwise collide on first write). +- `branch_delete(name)` — refuses if there are descendants on the branch, or if it is the current branch. The manifest is the single authority for branch existence: deletion flips the `__manifest` branch ref first (one atomic op), after which the branch is gone from every snapshot. The owned per-table forks and the commit-graph branch are derived state, reclaimed best-effort with `force_delete_branch` after the flip. A failure during that reclaim (transient object-store error) does not fail the call or block the authority flip; the leftover forks are unreachable orphans that the [`cleanup`](../operations/maintenance.md) reconciler converges. One consequence: if a delete's best-effort reclaim fails, reusing that branch name before the next `cleanup` surfaces a clear error pointing at `cleanup` (the stale fork would otherwise collide on first write). - **Lazy forking**: a branch only forks a sub-table when that sub-table is first mutated on it. Pure-read branches share fragments with their source. A fork collision is classified by the manifest authority, not by Lance branch versions: if the live manifest already records the fork on the active branch, a concurrent first-write won and the caller gets a retryable "refresh and retry"; if the manifest does not, a physical branch there is an orphan and the caller is pointed at `cleanup`. - `sync_branch(branch)` — re-binds the in-memory handle to the latest head of the branch. diff --git a/docs/user/transactions.md b/docs/user/branching/transactions.md similarity index 93% rename from docs/user/transactions.md rename to docs/user/branching/transactions.md index 39a86c4..a5515da 100644 --- a/docs/user/transactions.md +++ b/docs/user/branching/transactions.md @@ -2,7 +2,7 @@ OmniGraph does not have `BEGIN` / `COMMIT` / `ROLLBACK`. Branches do that job. This page explains the model, when to use which primitive, and shows worked examples for the patterns that come up most. -The architectural rule lives in [`docs/dev/invariants.md`](../dev/invariants.md): +The architectural rule lives in [`docs/dev/invariants.md`](../../dev/invariants.md): > **Mutations publish at one boundary.** A `mutate_as` or `load` operation > accumulates constructive writes, commits each touched table at the end, then @@ -161,8 +161,8 @@ This is the workflow MR-797 / agentic loops are designed around: **branches are ## See also -- [`docs/user/branches-commits.md`](branches-commits.md) — branch and commit-graph mechanics. -- [`docs/dev/merge.md`](../dev/merge.md) — three-way merge details and conflict kinds. -- [`docs/user/query-language.md`](query-language.md) — `.gq` syntax for the multi-statement queries used above. -- [`docs/dev/writes.md`](../dev/writes.md) — the per-query commit pipeline that gives single-query atomicity. -- [`docs/dev/invariants.md`](../dev/invariants.md) — the architectural rule. +- [`docs/user/branches-commits.md`](index.md) — branch and commit-graph mechanics. +- [`docs/dev/merge.md`](../../dev/merge.md) — three-way merge details and conflict kinds. +- [`docs/user/query-language.md`](../queries/index.md) — `.gq` syntax for the multi-statement queries used above. +- [`docs/dev/writes.md`](../../dev/writes.md) — the per-query commit pipeline that gives single-query atomicity. +- [`docs/dev/invariants.md`](../../dev/invariants.md) — the architectural rule. diff --git a/docs/user/cli.md b/docs/user/cli/index.md similarity index 100% rename from docs/user/cli.md rename to docs/user/cli/index.md diff --git a/docs/user/cli-reference.md b/docs/user/cli/reference.md similarity index 97% rename from docs/user/cli-reference.md rename to docs/user/cli/reference.md index e9216b8..bb73225 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli/reference.md @@ -1,6 +1,6 @@ # CLI Reference (`omnigraph`) -A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` schema. For a quick-start guide, see [cli.md](cli.md). +A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` schema. For a quick-start guide, see [cli.md](index.md). Top-level command families and subcommands. Graph-targeting commands accept a positional `URI`, `--uri`, a `--target <name>` resolved against `omnigraph.yaml`, or `--server <name>` (an operator-defined server from `~/.omnigraph/config.yaml`, optionally with `--graph <id>` for multi-graph servers; exclusive with the other forms); `cluster` commands use `--config <dir>`. @@ -8,7 +8,7 @@ Top-level command families and subcommands. Graph-targeting commands accept a po | Command | Purpose | |---|---| -| `init` | `--schema <pg>` → initialize a graph (no longer scaffolds `omnigraph.yaml` — RFC-008; start cluster configs from the [cluster.md](cluster.md) quick-start or `config migrate`) | +| `init` | `--schema <pg>` → initialize a graph (no longer scaffolds `omnigraph.yaml` — RFC-008; start cluster configs from the [cluster.md](../clusters/index.md) quick-start or `config migrate`) | | `load` | bulk load a branch, local or remote (`--mode overwrite\|append\|merge` is **required** — overwrite is destructive, so there is no default). Without `--from` the target branch must exist; `--from <base>` forks a missing `--branch` from `<base>` first | | `ingest` | deprecated alias of `load --from <base>` (defaults: `--from main --mode merge`); prints a one-line warning to stderr | | `query` (alias: `read`) | run named read query; source via `--query <path>`, `-e`/`--query-string <GQ>`, or `--alias <name>` (exactly one). `read` is the deprecated previous name and prints a one-line warning to stderr | @@ -53,7 +53,7 @@ tier: | Surface | Owner | Location | Declares | |---|---|---|---| -| Cluster config | the team, in a repo | `cluster.yaml` + checkout ([cluster-config.md](cluster-config.md)) | what the system **is**: graphs, schemas, queries, policies, storage | +| Cluster config | the team, in a repo | `cluster.yaml` + checkout ([cluster-config.md](../clusters/config.md)) | what the system **is**: graphs, schemas, queries, policies, storage | | Operator config | one person | `~/.omnigraph/config.yaml` (override dir with `$OMNIGRAPH_HOME`) | who **I** am: identity, ergonomics | | Flags / env | per invocation | — | everything, explicitly | @@ -204,7 +204,7 @@ creates one only when it is missing. Both observe declared graphs read-only at `<config-dir>/graphs/<graph-id>.omni`. External state backends, graph/schema apply, automatic stale-lock breaking, `plan --refresh`, pipelines, UI specs, embeddings, aliases, and bindings are reserved for later stages. See -[cluster-config.md](cluster-config.md). +[cluster-config.md](../clusters/config.md). ## Output formats (`query` command, alias: `read`) diff --git a/docs/user/cluster-config.md b/docs/user/clusters/config.md similarity index 99% rename from docs/user/cluster-config.md rename to docs/user/clusters/config.md index 59c9207..5b2e0d5 100644 --- a/docs/user/cluster-config.md +++ b/docs/user/clusters/config.md @@ -3,7 +3,7 @@ **Status:** Phase 5 — cluster-booted serving (`omnigraph-server --cluster`). > New to the cluster tooling? Start with the operator how-to guide, -> [cluster.md](cluster.md) — this document is the reference. +> [cluster.md](index.md) — this document is the reference. Cluster config is the future control-plane configuration surface for a whole OmniGraph deployment. In this stage, OmniGraph can validate a local diff --git a/docs/user/cluster.md b/docs/user/clusters/index.md similarity index 97% rename from docs/user/cluster.md rename to docs/user/clusters/index.md index 93b5ddf..0617753 100644 --- a/docs/user/cluster.md +++ b/docs/user/clusters/index.md @@ -7,8 +7,8 @@ destructive changes, and recovering from crashes. It is a **how-to**. The reference for every `cluster.yaml` key, command flag, state-file field, and diagnostic code is -[cluster-config.md](cluster-config.md); the HTTP surface is -[server.md](server.md). +[cluster-config.md](config.md); the HTTP surface is +[server.md](../operations/server.md). ## The model in one paragraph @@ -102,7 +102,7 @@ curl -H 'authorization: Bearer s3cret' \ Bearer tokens and the bind address are deliberately *not* cluster facts — they are per-replica, set by flag or environment -([server.md](server.md#modes) for the token sources). +([server.md](../operations/server.md#modes) for the token sources). ## 2. The day-2 loop: edit → plan → apply → restart @@ -237,7 +237,7 @@ with an in-flight apply. directory; boot is read-only. Roll out a change by `apply` once, then restarting replicas (serving is static per process — there is no hot reload yet). Container/cloud recipes (AWS ECS+EFS, Railway volumes): - [deployment.md](deployment.md#cluster-mode-in-containers-aws-railway). + [deployment.md](../deployment.md#cluster-mode-in-containers-aws-railway). - **The directory is the deployable unit**: config, catalog, ledger, approvals, and graph data all live under it. Back it up as a whole; version the *config files* (not `__cluster/` or `graphs/`) in git. @@ -282,4 +282,4 @@ a cluster are created by `cluster apply`, not by hand. reserved and rejected loudly. For the full reference — every key, flag, status, disposition, and -diagnostic — see [cluster-config.md](cluster-config.md). +diagnostic — see [cluster-config.md](config.md). diff --git a/docs/user/storage.md b/docs/user/concepts/storage.md similarity index 100% rename from docs/user/storage.md rename to docs/user/concepts/storage.md diff --git a/docs/user/deployment.md b/docs/user/deployment.md index ece7b5d..7f134c5 100644 --- a/docs/user/deployment.md +++ b/docs/user/deployment.md @@ -17,7 +17,7 @@ The server also has two **boot sources**: `omnigraph.yaml` (graph targets declared in the per-operator config) or a **cluster directory** (`omnigraph-server --cluster <dir>`), which serves the cluster control plane's applied revision — see -[cluster-config.md](cluster-config.md#serving-from-the-cluster-the-mode-switch). +[cluster-config.md](clusters/config.md#serving-from-the-cluster-the-mode-switch). The two are exclusive per deployment; switching is a restart with a different flag. diff --git a/docs/user/index.md b/docs/user/index.md index 956fa0b..c47b79b 100644 --- a/docs/user/index.md +++ b/docs/user/index.md @@ -2,44 +2,62 @@ **Audience:** users, CLI users, HTTP clients, and self-hosting operators -This is the public-facing entry point. These docs should describe behavior, -commands, configuration, and operational contracts without requiring knowledge -of MRs, internal recovery mechanics, or contributor-only invariants. +This is the public-facing entry point. These docs describe behavior, commands, +configuration, and operational contracts without requiring knowledge of internal +recovery mechanics or contributor-only invariants. They are organized by topic — +start with install, then follow the section that matches your task. -## Start Here +## Start here | Goal | Read | |---|---| | Install OmniGraph | [install.md](install.md) | -| Run the CLI locally | [cli.md](cli.md) | -| Look up every CLI flag and config field | [cli-reference.md](cli-reference.md) | -| Deploy and operate a cluster (how-to guide) | [cluster.md](cluster.md) | -| Validate and plan cluster config | [cluster-config.md](cluster-config.md) | -| Write schemas | [schema-language.md](schema-language.md) | -| Read schema-lint diagnostic codes | [schema-lint.md](schema-lint.md) | -| Write queries and mutations | [query-language.md](query-language.md) | -| Use embeddings | [embeddings.md](embeddings.md) | +| Run the CLI | [cli/index.md](cli/index.md) | +| Look up every CLI flag and config field | [cli/reference.md](cli/reference.md) | -## Operate A Graph +## Schema & queries | Goal | Read | |---|---| -| Understand graph layout and URI support | [storage.md](storage.md) | -| Work with branches, commits, and snapshots | [branches-commits.md](branches-commits.md) | -| Coordinate multi-query workflows | [transactions.md](transactions.md) | -| Read diffs and change feeds | [changes.md](changes.md) | -| Build and use indexes | [indexes.md](indexes.md) | -| Compact and clean old versions | [maintenance.md](maintenance.md) | -| Interpret errors and output formats | [errors.md](errors.md) | +| Write schemas (the `.pg` language) | [schema/index.md](schema/index.md) | +| Read schema-lint diagnostic codes | [schema/lint.md](schema/lint.md) | +| Write queries and mutations (the `.gq` language) | [queries/index.md](queries/index.md) | +| Use vector / full-text / hybrid search | [search/indexes.md](search/indexes.md) | +| Generate embeddings | [search/embeddings.md](search/embeddings.md) | +| Build and use indexes | [search/indexes.md](search/indexes.md) | -## Run The Server +## Branching & version control + +| Goal | Read | +|---|---| +| Work with branches, commits, and snapshots | [branching/index.md](branching/index.md) | +| Coordinate multi-query workflows | [branching/transactions.md](branching/transactions.md) | +| Read diffs and change feeds | [branching/changes.md](branching/changes.md) | + +## Operations | Goal | Read | |---|---| | Deploy the binary or container | [deployment.md](deployment.md) | -| Use HTTP endpoints | [server.md](server.md) | -| Configure Cedar authorization | [policy.md](policy.md) | -| Track actors and audit behavior | [audit.md](audit.md) | +| Use HTTP endpoints | [operations/server.md](operations/server.md) | +| Compact, repair, and clean old versions | [operations/maintenance.md](operations/maintenance.md) | +| Configure Cedar authorization | [operations/policy.md](operations/policy.md) | +| Track actors and audit behavior | [operations/audit.md](operations/audit.md) | +| Interpret errors and output formats | [operations/errors.md](operations/errors.md) | + +## Clusters + +| Goal | Read | +|---|---| +| Deploy and operate a cluster (how-to) | [clusters/index.md](clusters/index.md) | +| Reference every `cluster.yaml` key and command | [clusters/config.md](clusters/config.md) | + +## Concepts & reference + +| Goal | Read | +|---|---| +| Understand graph layout and URI support | [concepts/storage.md](concepts/storage.md) | +| Look up constants and tunables | [reference/constants.md](reference/constants.md) | ## Releases @@ -48,7 +66,6 @@ changes between versions, not for contributor design history. ## Boundary -User docs should focus on stable behavior. If a paragraph needs to explain -internal sidecars, Lance API blockers, MR numbers, test strategy, or review -rules, it probably belongs in [docs/dev/index.md](../dev/index.md) or a developer-area document -instead. +User docs focus on stable behavior. If a paragraph needs to explain internal +sidecars, Lance API blockers, or test strategy, it probably belongs in +[docs/dev/index.md](../dev/index.md) or a developer-area document instead. diff --git a/docs/user/audit.md b/docs/user/operations/audit.md similarity index 100% rename from docs/user/audit.md rename to docs/user/operations/audit.md diff --git a/docs/user/errors.md b/docs/user/operations/errors.md similarity index 92% rename from docs/user/errors.md rename to docs/user/operations/errors.md index 8373b0d..fad39a7 100644 --- a/docs/user/errors.md +++ b/docs/user/operations/errors.md @@ -9,7 +9,7 @@ - `Manifest(ManifestError { kind: BadRequest|NotFound|Conflict|Internal, details: Option<ManifestConflictDetails>, … })` - `ManifestConflictDetails::ExpectedVersionMismatch { table_key, expected, actual }` — caller's `expected_table_versions` did not match the manifest's current latest non-tombstoned version (set by `OmniError::manifest_expected_version_mismatch`). - `ManifestConflictDetails::RowLevelCasContention` — Lance row-level CAS rejected the publish because a concurrent writer landed the same `object_id`. Retried internally by the publisher; only surfaces if the retry budget exhausts. - - **D₂ parse-time rejection** (MR-794): a single mutation query that mixes inserts/updates with deletes errors out *before any I/O* with kind `BadRequest`. Message: `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes`. See [docs/user/query-language.md](query-language.md) for the rule and [docs/dev/writes.md](../dev/writes.md) for the underlying staged-write rationale. + - **D₂ parse-time rejection** (MR-794): a single mutation query that mixes inserts/updates with deletes errors out *before any I/O* with kind `BadRequest`. Message: `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes`. See [docs/user/query-language.md](../queries/index.md) for the rule and [docs/dev/writes.md](../../dev/writes.md) for the underlying staged-write rationale. - `MergeConflicts(Vec<MergeConflict>)` Compiler-side `NanoError` covers parse / catalog / type / storage / plan / execution / arrow / lance / IO / manifest / unique-constraint, each with structured spans (`SourceSpan { start, end }`) for ariadne-style diagnostics. diff --git a/docs/user/maintenance.md b/docs/user/operations/maintenance.md similarity index 92% rename from docs/user/maintenance.md rename to docs/user/operations/maintenance.md index 8b97657..eeeb002 100644 --- a/docs/user/maintenance.md +++ b/docs/user/operations/maintenance.md @@ -2,7 +2,7 @@ `db/omnigraph/optimize.rs` and `db/omnigraph/repair.rs`. -**Addressing (RFC-010).** `optimize`, `repair`, and `cleanup` are **storage-plane** CLI commands: they run with direct storage access against a positional `URI`, `--target`, or **`--cluster <dir|s3://…> --cluster-graph <id>`** (which resolves the graph's storage URI from the served cluster state, so you needn't know the `<storage>/graphs/<id>.omni` layout). They never run through a server, and reject `--server` / `--graph` or a `--target` that resolves to a remote (`http(s)://`) URL with a declared error. There are no server routes for them by design — to maintain a server-backed graph, run them out-of-band against the graph's storage URI. See the *Command planes* section of [cli-reference.md](cli-reference.md). +**Addressing (RFC-010).** `optimize`, `repair`, and `cleanup` are **storage-plane** CLI commands: they run with direct storage access against a positional `URI`, `--target`, or **`--cluster <dir|s3://…> --cluster-graph <id>`** (which resolves the graph's storage URI from the served cluster state, so you needn't know the `<storage>/graphs/<id>.omni` layout). They never run through a server, and reject `--server` / `--graph` or a `--target` that resolves to a remote (`http(s)://`) URL with a declared error. There are no server routes for them by design — to maintain a server-backed graph, run them out-of-band against the graph's storage URI. See the *Command planes* section of [cli-reference.md](../cli/reference.md). ## `optimize_all_tables(db)` — non-destructive @@ -13,7 +13,7 @@ - **Uncovered drift is skipped, not interpreted.** If a table's Lance HEAD is ahead of the version recorded in `__manifest` and no recovery sidecar covers that movement, `optimize` reports `skipped: Some(DriftNeedsRepair)` with the manifest/head versions and leaves the table untouched. Run `omnigraph repair` to classify and explicitly publish that drift. - Bounded by `OMNIGRAPH_MAINTENANCE_CONCURRENCY` (default 8). - Returns `[TableOptimizeStats { table_key, fragments_removed, fragments_added, committed, skipped, manifest_version, lance_head_version }]`. -- **Blob tables are skipped.** A table that declares any `Blob` property is not compacted: it is reported with `skipped: Some(BlobColumnsUnsupportedByLance)` (and logged via `tracing::warn`) instead of compacted, and the rest of the sweep proceeds normally. The current Lance `compact_files` mis-decodes blob-v2 columns under its forced `BlobHandling::AllBinary` read; **reads and writes are unaffected** — only compaction is. This is gated by `LANCE_SUPPORTS_BLOB_COMPACTION` (`db/omnigraph/optimize.rs`) and removed when the upstream Lance fix lands (see [docs/dev/lance.md](../dev/lance.md)). Consequence: fragment count and deleted-row space on blob tables are not reclaimed until then; query results are never affected. +- **Blob tables are skipped.** A table that declares any `Blob` property is not compacted: it is reported with `skipped: Some(BlobColumnsUnsupportedByLance)` (and logged via `tracing::warn`) instead of compacted, and the rest of the sweep proceeds normally. The current Lance `compact_files` mis-decodes blob-v2 columns under its forced `BlobHandling::AllBinary` read; **reads and writes are unaffected** — only compaction is. This is gated by `LANCE_SUPPORTS_BLOB_COMPACTION` (`db/omnigraph/optimize.rs`) and removed when the upstream Lance fix lands (see [docs/dev/lance.md](../../dev/lance.md)). Consequence: fragment count and deleted-row space on blob tables are not reclaimed until then; query results are never affected. ## `repair_all_tables(db, options)` — explicit @@ -36,7 +36,7 @@ any failed tables; rerun `cleanup` to retry them. - CLI guards with `--confirm`; without it, prints a preview line. - **Recovery floor:** `--keep < 3` may garbage-collect Lance versions that the open-time recovery sweep needs as a rollback target (the sweep restores to the branch's manifest-pinned table version, which is HEAD-1 in the typical Phase B → Phase C drift case). Default `--keep 10` is safe. -- **Orphaned-branch reconciliation:** before the version GC, cleanup runs `reconcile_orphaned_branches`, which `force_delete_branch`es any per-table or commit-graph Lance branch absent from the manifest branch list. These orphans arise when a `branch_delete` flips the manifest authority but a downstream best-effort reclaim does not complete (see [branches-commits.md](branches-commits.md)). The reconciler is authority-derived and idempotent (it no-ops once nothing is orphaned), runs regardless of the `keep_versions` / `older_than` values (those gate version GC only), and never reclaims `main` or system-branch forks. Reclaimed forks are logged via `tracing::info`. +- **Orphaned-branch reconciliation:** before the version GC, cleanup runs `reconcile_orphaned_branches`, which `force_delete_branch`es any per-table or commit-graph Lance branch absent from the manifest branch list. These orphans arise when a `branch_delete` flips the manifest authority but a downstream best-effort reclaim does not complete (see [branches-commits.md](../branching/index.md)). The reconciler is authority-derived and idempotent (it no-ops once nothing is orphaned), runs regardless of the `keep_versions` / `older_than` values (those gate version GC only), and never reclaims `main` or system-branch forks. Reclaimed forks are logged via `tracing::info`. ## Tombstones @@ -44,6 +44,6 @@ Logical sub-table delete markers in `__manifest`; `tombstone_object_id(table_key ## Internal schema migrations (`db/manifest/migrations.rs`) -Version evolutions of the on-disk `__manifest` shape are reconciled automatically on the first write under a new binary. `INTERNAL_MANIFEST_SCHEMA_VERSION` declares the shape the binary expects; the on-disk stamp `omnigraph:internal_schema_version` (Lance schema-level metadata) records the on-disk shape. The publisher's open-for-write path calls `migrate_internal_schema` before reading state; reads are side-effect-free. No operator action is required for in-place upgrades. See [storage.md → Internal schema versioning](storage.md) for the full mechanism. +Version evolutions of the on-disk `__manifest` shape are reconciled automatically on the first write under a new binary. `INTERNAL_MANIFEST_SCHEMA_VERSION` declares the shape the binary expects; the on-disk stamp `omnigraph:internal_schema_version` (Lance schema-level metadata) records the on-disk shape. The publisher's open-for-write path calls `migrate_internal_schema` before reading state; reads are side-effect-free. No operator action is required for in-place upgrades. See [storage.md → Internal schema versioning](../concepts/storage.md) for the full mechanism. A binary opening a manifest stamped at a version *higher* than it knows about refuses to publish with a clear "upgrade omnigraph first" error — old binaries cannot clobber a newer schema. diff --git a/docs/user/policy.md b/docs/user/operations/policy.md similarity index 100% rename from docs/user/policy.md rename to docs/user/operations/policy.md diff --git a/docs/user/server.md b/docs/user/operations/server.md similarity index 94% rename from docs/user/server.md rename to docs/user/operations/server.md index a2e6705..8e63e99 100644 --- a/docs/user/server.md +++ b/docs/user/operations/server.md @@ -21,7 +21,7 @@ revision** (`state.json` + content-addressed blobs) instead of `omnigraph.yaml` — an exclusive boot source: combining it with `<URI>`, `--target`, or `--config` is a startup error, and `omnigraph.yaml` is never read in this mode. Always multi-graph routing. See -[cluster-config.md](cluster-config.md#serving-from-the-cluster-the-mode-switch) +[cluster-config.md](../clusters/config.md#serving-from-the-cluster-the-mode-switch) for what is read and the fail-fast readiness rules. `--bind`, `--unauthenticated`, and the bearer-token env vars work identically. @@ -36,7 +36,7 @@ Mode inference: ### Stored-query validation at startup -If a graph declares a `queries:` registry (see [cli-reference](cli-reference.md)), the server **loads and type-checks every stored query against that graph's live schema at startup** and **refuses to boot** if any query references a type or property the schema lacks — the same fail-loud posture as a malformed policy file, so schema drift surfaces at the deploy boundary rather than at invocation. Two MCP-exposed queries claiming the same tool name is likewise a boot error. Non-blocking advisories (e.g. an MCP-exposed query with a vector parameter an agent cannot supply) are logged. Validate offline before deploying with `omnigraph queries validate`. Discover the exposed queries as a typed tool catalog with `GET /queries`, and invoke one over HTTP with `POST /queries/{name}` (both below). +If a graph declares a `queries:` registry (see [cli-reference](../cli/reference.md)), the server **loads and type-checks every stored query against that graph's live schema at startup** and **refuses to boot** if any query references a type or property the schema lacks — the same fail-loud posture as a malformed policy file, so schema drift surfaces at the deploy boundary rather than at invocation. Two MCP-exposed queries claiming the same tool name is likewise a boot error. Non-blocking advisories (e.g. an MCP-exposed query with a vector parameter an agent cannot supply) are logged. Validate offline before deploying with `omnigraph queries validate`. Discover the exposed queries as a typed tool catalog with `GET /queries`, and invoke one over HTTP with `POST /queries/{name}` (both below). ## Endpoint inventory @@ -209,7 +209,7 @@ admission-gated. policy file without tokens is also rejected at startup. In open mode `/openapi.json` strips the security scheme. -See [deployment.md](deployment.md) for token-source operational details. +See [deployment.md](../deployment.md) for token-source operational details. ## Tracing & observability diff --git a/docs/user/query-language.md b/docs/user/queries/index.md similarity index 97% rename from docs/user/query-language.md rename to docs/user/queries/index.md index bcab67c..0942d50 100644 --- a/docs/user/query-language.md +++ b/docs/user/queries/index.md @@ -72,7 +72,7 @@ A single mutation query must be **either insert/update-only or delete-only**. Mi > `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes. This restriction lifts when Lance exposes a two-phase delete API (tracked: MR-793 / Lance-upstream).` -Reason: under the staged-write rewire (MR-794), inserts and updates accumulate in memory and commit at end-of-query, while deletes still inline-commit (Lance v6.0.1 has no public two-phase delete). Mixing creates ordering hazards (same-row insert→delete becomes a no-op because the staged insert isn't visible to delete; cascading deletes of just-inserted edges break referential integrity by silent design). Until the MR-A Lance v7 bump migrates `delete_where` to staged (`DeleteBuilder::execute_uncommitted` first ships in `v7.0.0-beta.10`), the parse-time rejection keeps both paths atomic and correct. See [docs/dev/writes.md](../dev/writes.md), [docs/dev/lance.md](../dev/lance.md), and [docs/dev/invariants.md](../dev/invariants.md). +Reason: under the staged-write rewire (MR-794), inserts and updates accumulate in memory and commit at end-of-query, while deletes still inline-commit (Lance v6.0.1 has no public two-phase delete). Mixing creates ordering hazards (same-row insert→delete becomes a no-op because the staged insert isn't visible to delete; cascading deletes of just-inserted edges break referential integrity by silent design). Until the MR-A Lance v7 bump migrates `delete_where` to staged (`DeleteBuilder::execute_uncommitted` first ships in `v7.0.0-beta.10`), the parse-time rejection keeps both paths atomic and correct. See [docs/dev/writes.md](../../dev/writes.md), [docs/dev/lance.md](../../dev/lance.md), and [docs/dev/invariants.md](../../dev/invariants.md). ## IR (Intermediate Representation) @@ -81,7 +81,7 @@ Reason: under the staged-write rewire (MR-794), inserts and updates accumulate i Pipeline operations: - `NodeScan { variable, type_name, filters }` -- `Expand { src_var, dst_var, edge_type, direction (Out|In), dst_type, min_hops, max_hops, dst_filters }` — destination filters are pushed *into* the expand so Lance scalar pushdown can prune. Executed one of two ways, chosen per-expand by a cost model over cheap manifest counts (frontier size, |E|, source-vertex count, hops) plus index coverage: selective traversals (small frontier relative to the source set) resolve neighbors from the persisted `src`/`dst` BTREE (one indexed scan per hop); dense / deep / large-frontier traversals — or those whose BTREE coverage is degraded so a full scan would be paid per hop — use the in-memory CSR adjacency index. Both produce identical results. The `OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER` / `OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS` ceilings bound the *initial dispatch* frontier/hops (beyond them CSR is always used); the cost model estimates total indexed work as ~`hops × frontier × fanout` and prices dense fan-out toward CSR — they are not a hard per-hop bound. `OMNIGRAPH_TRAVERSAL_MODE=indexed|csr` forces a mode (see [constants](constants.md)). +- `Expand { src_var, dst_var, edge_type, direction (Out|In), dst_type, min_hops, max_hops, dst_filters }` — destination filters are pushed *into* the expand so Lance scalar pushdown can prune. Executed one of two ways, chosen per-expand by a cost model over cheap manifest counts (frontier size, |E|, source-vertex count, hops) plus index coverage: selective traversals (small frontier relative to the source set) resolve neighbors from the persisted `src`/`dst` BTREE (one indexed scan per hop); dense / deep / large-frontier traversals — or those whose BTREE coverage is degraded so a full scan would be paid per hop — use the in-memory CSR adjacency index. Both produce identical results. The `OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER` / `OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS` ceilings bound the *initial dispatch* frontier/hops (beyond them CSR is always used); the cost model estimates total indexed work as ~`hops × frontier × fanout` and prices dense fan-out toward CSR — they are not a hard per-hop bound. `OMNIGRAPH_TRAVERSAL_MODE=indexed|csr` forces a mode (see [constants](../reference/constants.md)). - `Filter { left, op, right }` - `AntiJoin { outer_var, inner: Vec<IROp> }` — for `not { … }` diff --git a/docs/user/constants.md b/docs/user/reference/constants.md similarity index 100% rename from docs/user/constants.md rename to docs/user/reference/constants.md diff --git a/docs/user/schema-language.md b/docs/user/schema/index.md similarity index 100% rename from docs/user/schema-language.md rename to docs/user/schema/index.md diff --git a/docs/user/schema-lint.md b/docs/user/schema/lint.md similarity index 100% rename from docs/user/schema-lint.md rename to docs/user/schema/lint.md diff --git a/docs/user/embeddings.md b/docs/user/search/embeddings.md similarity index 100% rename from docs/user/embeddings.md rename to docs/user/search/embeddings.md diff --git a/docs/user/indexes.md b/docs/user/search/indexes.md similarity index 92% rename from docs/user/indexes.md rename to docs/user/search/indexes.md index df898c4..fde9488 100644 --- a/docs/user/indexes.md +++ b/docs/user/search/indexes.md @@ -23,4 +23,4 @@ This is OmniGraph-specific (not Lance): - `CsrIndex`: Compressed Sparse Row representation of edges per edge type — `offsets[i]..offsets[i+1]` slices into `targets`. - `GraphIndex { type_indices, csr (out), csc (in) }` — built on demand from a snapshot's edge tables, **lazily**: only when an `Expand` the planner routes to the CSR path (dense / large frontier) or an `AntiJoin` actually needs it. - Cached in `RuntimeCache::graph_indices` (LRU, max 8 entries, keyed by snapshot id + edge table versions). -- Selective `Expand`s resolve neighbors from the persisted `src`/`dst` BTREE instead (one indexed scan per hop) and never trigger the CSR build; see [query-language](query-language.md) → Expand. Pure scans, and queries served entirely by the indexed traversal path, skip it. +- Selective `Expand`s resolve neighbors from the persisted `src`/`dst` BTREE instead (one indexed scan per hop) and never trigger the CSR build; see [query-language](../queries/index.md) → Expand. Pure scans, and queries served entirely by the indexed traversal path, skip it. From 612741b387a8f13a65f3514ae49ac095196676cb Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sun, 14 Jun 2026 13:53:46 +0300 Subject: [PATCH 157/165] docs(user): split language/branching pages + add front-door pages (Phase 2) (#225) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content build-out on top of the Phase 1 topic move. No behavior changes. Splits (existing content relocated, cross-linked): - queries/index.md → mutations/index.md (insert/update/delete + the inserts-vs-deletes rule) and search/index.md (the multi-modal search functions + a hybrid-ranking overview tying nearest/bm25/rrf together). queries/index.md now covers the read shape and points at both. - branching/index.md → branching/time-travel.md (snapshots/time travel) and branching/merge.md (three-way merge + the 7 conflict kinds, verified against error.rs MergeConflictKind). New pages (written from the code, user-facing): - quickstart.md — init → load → query → branch, with verified CLI flags. - concepts/index.md — what OmniGraph is + the L1/L2 (Lance/OmniGraph) framing. Expanded operations/audit.md from a 7-line struct dump into a real actor-tracking page (server token-resolved vs CLI --as chain; reading the trail; the omnigraph:recovery reserved actor). Index wiring: docs/user/index.md and AGENTS.md's topic table link every new page; also normalized AGENTS.md's docs/user link display text to match the Phase 1 retargeted paths. Verified: zero broken .md links; check-agents-md.sh green (57 links, 54 docs). Deferred to Phase 3: de-dev polish (grammar paths, IR internals still in queries/branching), guides/, and a possible reference/config.md split (the config schema is already coherent in cli/reference.md). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- AGENTS.md | 46 +++++++++-------- docs/user/branching/index.md | 8 ++- docs/user/branching/merge.md | 47 +++++++++++++++++ docs/user/branching/time-travel.md | 31 ++++++++++++ docs/user/concepts/index.md | 49 ++++++++++++++++++ docs/user/index.md | 12 +++-- docs/user/mutations/index.md | 52 +++++++++++++++++++ docs/user/operations/audit.md | 51 ++++++++++++++++--- docs/user/queries/index.md | 41 +++------------ docs/user/quickstart.md | 81 ++++++++++++++++++++++++++++++ docs/user/search/index.md | 48 ++++++++++++++++++ 11 files changed, 399 insertions(+), 67 deletions(-) create mode 100644 docs/user/branching/merge.md create mode 100644 docs/user/branching/time-travel.md create mode 100644 docs/user/concepts/index.md create mode 100644 docs/user/mutations/index.md create mode 100644 docs/user/quickstart.md create mode 100644 docs/user/search/index.md diff --git a/AGENTS.md b/AGENTS.md index 79e4fa7..065e28a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,32 +73,38 @@ Full diagram and concurrency model: [docs/dev/architecture.md](docs/dev/architec | **Lance docs index — fetch upstream Lance docs by problem domain** | **[docs/dev/lance.md](docs/dev/lance.md)** | | **Test coverage map — what's covered, what helpers to reuse, before-every-task checklist** | **[docs/dev/testing.md](docs/dev/testing.md)** | | Architecture, L1/L2 framing, concurrency model | [docs/dev/architecture.md](docs/dev/architecture.md) | -| Storage layout, `__manifest` schema, URI schemes, S3 env vars | [docs/user/storage.md](docs/user/concepts/storage.md) | -| `.pg` schema language, types, constraints, annotations, migration planning | [docs/user/schema-language.md](docs/user/schema/index.md) | -| Schema-lint codes (`OG-XXX-NNN`), families, severity, suppression | [docs/user/schema-lint.md](docs/user/schema/lint.md) | -| `.gq` query language, MATCH/RETURN/ORDER, search funcs, mutations, IR ops, lint codes | [docs/user/query-language.md](docs/user/queries/index.md) | -| Indexes (BTREE / inverted / vector / graph topology) | [docs/user/indexes.md](docs/user/search/indexes.md) | -| Embeddings (compiler + engine clients, env vars, `@embed`) | [docs/user/embeddings.md](docs/user/search/embeddings.md) | -| Branches, commit graph, snapshots, system branches | [docs/user/branches-commits.md](docs/user/branching/index.md) | -| Transactions and atomicity (per-query atomic; branches as multi-query transactions) | [docs/user/transactions.md](docs/user/branching/transactions.md) | +| Storage layout, `__manifest` schema, URI schemes, S3 env vars | [docs/user/concepts/storage.md](docs/user/concepts/storage.md) | +| `.pg` schema language, types, constraints, annotations, migration planning | [docs/user/schema/index.md](docs/user/schema/index.md) | +| Schema-lint codes (`OG-XXX-NNN`), families, severity, suppression | [docs/user/schema/lint.md](docs/user/schema/lint.md) | +| `.gq` query language, MATCH/RETURN/ORDER, IR ops, lint codes | [docs/user/queries/index.md](docs/user/queries/index.md) | +| Mutations — insert/update/delete, D2, atomicity | [docs/user/mutations/index.md](docs/user/mutations/index.md) | +| Search funcs (`nearest`/`bm25`/`rrf`), hybrid ranking | [docs/user/search/index.md](docs/user/search/index.md) | +| Indexes (BTREE / inverted / vector / graph topology) | [docs/user/search/indexes.md](docs/user/search/indexes.md) | +| Embeddings (compiler + engine clients, env vars, `@embed`) | [docs/user/search/embeddings.md](docs/user/search/embeddings.md) | +| Concepts — what OmniGraph is, L1/L2 framing | [docs/user/concepts/index.md](docs/user/concepts/index.md) | +| Quickstart — init → load → query → branch | [docs/user/quickstart.md](docs/user/quickstart.md) | +| Branches, commit graph, system branches | [docs/user/branching/index.md](docs/user/branching/index.md) | +| Snapshots & time travel | [docs/user/branching/time-travel.md](docs/user/branching/time-travel.md) | +| Three-way merge and conflict kinds (user-facing) | [docs/user/branching/merge.md](docs/user/branching/merge.md) | +| Transactions and atomicity (per-query atomic; branches as multi-query transactions) | [docs/user/branching/transactions.md](docs/user/branching/transactions.md) | | Direct-publish write path (staging, D2, recovery sidecars; the former Run state machine) | [docs/dev/writes.md](docs/dev/writes.md) | | Three-way merge and conflict kinds | [docs/dev/merge.md](docs/dev/merge.md) | -| Diff / change feed (`diff_between`, `diff_commits`) | [docs/user/changes.md](docs/user/branching/changes.md) | +| Diff / change feed (`diff_between`, `diff_commits`) | [docs/user/branching/changes.md](docs/user/branching/changes.md) | | Query execution, mutation execution, bulk loader, `load` vs `ingest` | [docs/dev/execution.md](docs/dev/execution.md) | -| `optimize` (compaction) and `cleanup` (version GC) | [docs/user/maintenance.md](docs/user/operations/maintenance.md) | -| Cluster operator guide (deploy/manage clusters, approvals, recovery, serving) | [docs/user/cluster.md](docs/user/clusters/index.md) | -| Cedar policy actions, scopes, CLI | [docs/user/policy.md](docs/user/operations/policy.md) | -| HTTP server endpoints, auth, error model, body limits | [docs/user/server.md](docs/user/operations/server.md) | -| CLI quick-start | [docs/user/cli.md](docs/user/cli/index.md) | -| CLI command surface and config schemas (`~/.omnigraph/config.yaml`, legacy `omnigraph.yaml`) | [docs/user/cli-reference.md](docs/user/cli/reference.md) | -| Audit / actor tracking | [docs/user/audit.md](docs/user/operations/audit.md) | -| Error taxonomy and result serialization | [docs/user/errors.md](docs/user/operations/errors.md) | +| `optimize` (compaction) and `cleanup` (version GC) | [docs/user/operations/maintenance.md](docs/user/operations/maintenance.md) | +| Cluster operator guide (deploy/manage clusters, approvals, recovery, serving) | [docs/user/clusters/index.md](docs/user/clusters/index.md) | +| Cedar policy actions, scopes, CLI | [docs/user/operations/policy.md](docs/user/operations/policy.md) | +| HTTP server endpoints, auth, error model, body limits | [docs/user/operations/server.md](docs/user/operations/server.md) | +| CLI quick-start | [docs/user/cli/index.md](docs/user/cli/index.md) | +| CLI command surface and config schemas (`~/.omnigraph/config.yaml`, legacy `omnigraph.yaml`) | [docs/user/cli/reference.md](docs/user/cli/reference.md) | +| Audit / actor tracking | [docs/user/operations/audit.md](docs/user/operations/audit.md) | +| Error taxonomy and result serialization | [docs/user/operations/errors.md](docs/user/operations/errors.md) | | Install (binary / Homebrew / source / channels) | [docs/user/install.md](docs/user/install.md) | | Deployment (binary / container / RustFS bootstrap / auth / build variants) | [docs/user/deployment.md](docs/user/deployment.md) | | CI / release workflows | [docs/dev/ci.md](docs/dev/ci.md) | | Code ownership (CODEOWNERS source of truth, roles, regeneration) | [docs/dev/codeowners.md](docs/dev/codeowners.md) | | Branch protection policy (declarative, applied via `scripts/apply-branch-protection.sh`) | [docs/dev/branch-protection.md](docs/dev/branch-protection.md) | -| Constants & tunables cheat sheet | [docs/user/constants.md](docs/user/reference/constants.md) | +| Constants & tunables cheat sheet | [docs/user/reference/constants.md](docs/user/reference/constants.md) | | Per-version release notes | [docs/releases/](docs/releases/) | --- @@ -257,7 +263,7 @@ omnigraph policy explain --actor act-alice --action change --branch main | Per-query atomic writes | — | In-memory `MutationStaging.pending` accumulator + `stage_*` / `commit_staged` per touched table at end-of-query + publisher CAS via `commit_with_expected` (single manifest commit per `mutate_as` / `load`); D₂ parse-time rule keeps inserts/updates and deletes from mixing | | Three-way row-level merge | — | `OrderedTableCursor` + `StagedTableWriter`, structured `MergeConflictKind` | | Change feeds | — | `diff_between` / `diff_commits` with manifest fast path + ID streaming | -| Cedar policy | — | Per-graph actions plus server-scoped actions (see [docs/user/policy.md](docs/user/operations/policy.md) for the current list), branch / target_branch / protected scopes, validate/test/explain CLI. **Engine-wide enforcement** (MR-722): every `_as` writer (`apply_schema_as`, `mutate_as`, `load_as` — the deprecated `ingest_as` shims route through it — `branch_create_as` / `branch_create_from_as`, `branch_delete_as`, `branch_merge_as`) calls `Omnigraph::enforce(action, scope, actor)` — HTTP, CLI, embedded SDK all hit the same gate. | +| Cedar policy | — | Per-graph actions plus server-scoped actions (see [docs/user/operations/policy.md](docs/user/operations/policy.md) for the current list), branch / target_branch / protected scopes, validate/test/explain CLI. **Engine-wide enforcement** (MR-722): every `_as` writer (`apply_schema_as`, `mutate_as`, `load_as` — the deprecated `ingest_as` shims route through it — `branch_create_as` / `branch_create_from_as`, `branch_delete_as`, `branch_merge_as`) calls `Omnigraph::enforce(action, scope, actor)` — HTTP, CLI, embedded SDK all hit the same gate. | | HTTP server | — | Axum, OpenAPI via utoipa, bearer auth (SHA-256, AWS Secrets Manager option), `authorize_request` at the HTTP boundary (resolves bearer→actor, applies admission control), NDJSON streaming export, **multi-graph mode (v0.6.0+) with cluster routes + read-only `GET /graphs` enumeration + per-graph + server-level Cedar policies. Multi-graph boots from a cluster directory (`--cluster`) or the legacy `omnigraph.yaml`; add/remove graphs via `cluster apply` (or by editing the legacy file) and restarting.** | | CLI with config | — | two-surface config (team `cluster.yaml` dir + per-operator `~/.omnigraph/config.yaml`; legacy `omnigraph.yaml` deprecated per RFC-008), aliases, multi-format output (json/jsonl/csv/kv/table) | | Audit / actor tracking | — | `_as` write APIs + actor map in commit graph | @@ -282,7 +288,7 @@ Rules: 7. **Re-verify before recommending.** If you cite a flag, env var, endpoint, or constant to the user or in code, grep for it in source first. Memory and docs go stale; the code is authoritative. 8. **Keep AGENTS.md short.** This file is always loaded into agent context, so every added line has a recurring context-window cost. Prefer pointers and terse invariants here; put detail in `docs/`. 9. **Keep AGENTS.md a map, not an encyclopedia.** New deep content goes into `docs/`. Add an entry to "Where to find each topic" instead of pasting prose into this file. The "Always-on rules" section is the exception — it's for invariants that should always be in scope. -10. **Re-read on schema/query/IR changes.** Edits to `schema.pest`, `query.pest`, `ir/lower.rs`, `query/typecheck.rs`, or `query/lint.rs` should trigger a re-read of [docs/user/schema-language.md](docs/user/schema/index.md), [docs/user/query-language.md](docs/user/queries/index.md), and [docs/dev/execution.md](docs/dev/execution.md) to confirm they still describe reality. +10. **Re-read on schema/query/IR changes.** Edits to `schema.pest`, `query.pest`, `ir/lower.rs`, `query/typecheck.rs`, or `query/lint.rs` should trigger a re-read of [docs/user/schema/index.md](docs/user/schema/index.md), [docs/user/queries/index.md](docs/user/queries/index.md), and [docs/dev/execution.md](docs/dev/execution.md) to confirm they still describe reality. 11. **Always make smaller commits.** Each commit does one thing, compiles, and passes tests; mechanical refactors land separately from the behavior changes they enable. 12. **Test-first for bug fixes.** When fixing an identified bug, write a regression test that reproduces the failure first. Confirm it fails against the current code with the predicted symptom (not an unrelated error). Then land the fix in a separate commit and confirm the test turns green. The test commit lands just before the fix commit so the red → green pair is visible in `git log` and a reviewer can check out the test commit alone and reproduce the failure. 13. **Correct by design over symptomatic patches.** When a bug surfaces, identify the root cause and make the fix correct by construction. Don't patch the symptom. If the design admits the bug class, the fix is to close the class, not to add a guard around the latest instance. A symptomatic patch is acceptable only as a stop-gap, with an explicit note in the commit message and a follow-up issue tracking the design fix. diff --git a/docs/user/branching/index.md b/docs/user/branching/index.md index 17d17b2..a0f1a6e 100644 --- a/docs/user/branching/index.md +++ b/docs/user/branching/index.md @@ -43,11 +43,9 @@ Notes: ## L2 — Snapshots & time travel -- `snapshot()` — current snapshot for the bound branch; cached. -- `snapshot_of(target)` — snapshot at a `ReadTarget` (branch | snapshot id). -- `snapshot_at_version(v: u64)` — historical snapshot from any manifest version. -- `entity_at(table_key, id, version)` — single-entity time travel without building a full snapshot. -- A `Snapshot` is a `(version, HashMap<table_key, SubTableEntry>)` — cheap to build, snapshot-isolated cross-table reads. +Reading a branch at a past version, or a single entity at a past version, is +covered on the [time travel](time-travel.md) page. Merging branches and the +conflict kinds are on the [merge](merge.md) page. ## L2 — Internal system branches diff --git a/docs/user/branching/merge.md b/docs/user/branching/merge.md new file mode 100644 index 0000000..fde2fab --- /dev/null +++ b/docs/user/branching/merge.md @@ -0,0 +1,47 @@ +# Merging Branches + +Merging integrates the changes on one branch into another. OmniGraph merges are +**three-way and row-level**: it compares both branches against their common +ancestor and merges each node/edge table row by row, then publishes the result as +**one atomic commit** across the whole graph. + +```bash +omnigraph branch merge review/2026-04-25 --into main s3://bucket/graph.omni +``` + +`branch merge <source> [--into <target>]` merges `<source>` into `<target>` +(default `main`). + +## Outcomes + +A merge resolves to one of three outcomes: + +- **Already up to date** — the target already contains every change on the source; + nothing to do. +- **Fast-forward** — the target has no changes the source lacks, so the target + simply advances to the source. +- **Merged** — both sides diverged; a new merge commit is created with two parents. + +## Conflicts + +When both branches changed the same data incompatibly, the merge fails with a +structured list of conflicts (the HTTP server returns `409` with a +`merge_conflicts[]` array). No partial result is published — the merge is +all-or-nothing. The conflict kinds are: + +| Kind | Meaning | +|---|---| +| `DivergentInsert` | The same id was inserted on both branches. | +| `DivergentUpdate` | The same row was updated differently on both branches. | +| `DeleteVsUpdate` | One side deleted a row the other side updated. | +| `OrphanEdge` | An edge references a node the other side deleted. | +| `UniqueViolation` | The merged result would violate a unique constraint. | +| `CardinalityViolation` | The merged result would violate an edge cardinality constraint. | +| `ValueConstraintViolation` | The merged result would violate a value constraint (enum/range). | + +Each conflict carries the table, the row id (when applicable), the kind, and a +message. Resolve conflicts by reconciling the two branches — typically by making +the conflicting change on one side and re-merging. + +See [branches & commits](index.md) for the branch and commit-DAG model, and +[changes](changes.md) for diffing two branches before you merge. diff --git a/docs/user/branching/time-travel.md b/docs/user/branching/time-travel.md new file mode 100644 index 0000000..e6bd52d --- /dev/null +++ b/docs/user/branching/time-travel.md @@ -0,0 +1,31 @@ +# Snapshots & Time Travel + +Every read in OmniGraph happens against a **snapshot** — a consistent, cross-table +view of the graph at one manifest version. A query holds one snapshot for its whole +lifetime, so it never sees a partial write from a concurrent commit (see +[transactions](transactions.md)). + +## Reading the past + +- **Current head** — by default a read targets the current head of the bound branch. +- **By snapshot id** — read a branch or a specific snapshot id (`--snapshot` on + `omnigraph read`). +- **By version** — reconstruct a historical snapshot from any past manifest version. +- **Single entity** — look up one entity at a past version without building a full + snapshot (cheaper when you only need one node or edge). + +Snapshots are cheap to build: a snapshot is just the set of visible sub-table +versions at a manifest version, so cross-table reads stay snapshot-isolated. + +## CLI + +```bash +# Read a query against a past snapshot +omnigraph read --query ./q.gq --name find --snapshot <snapshot-id> s3://bucket/graph.omni +``` + +Time travel composes with branches: every branch has its own version history, and +you can read any branch at any of its past versions. Commits and the commit DAG +that these versions correspond to are described in +[branches & commits](index.md); diffing two versions is on the +[changes](changes.md) page. diff --git a/docs/user/concepts/index.md b/docs/user/concepts/index.md new file mode 100644 index 0000000..8bc3d7e --- /dev/null +++ b/docs/user/concepts/index.md @@ -0,0 +1,49 @@ +# Concepts + +OmniGraph is a typed property-graph engine built as a coordination layer over the +[Lance](https://lance.org) columnar storage format. It gives you a schema-checked +graph with vector, full-text, and graph queries in one runtime, plus Git-style +branches and commits across the whole graph. + +## The data model + +- A graph has **node types** and **edge types**, declared in a + [schema](../schema/index.md). +- Each node type and each edge type is stored as its **own Lance dataset** — + columnar, versioned, on local disk or object storage. +- A single `__manifest` table coordinates all of those datasets, so the graph has + one coherent version even though it spans many datasets. + +This split is what lets a graph commit be **atomic across every type at once**: a +publish flips every relevant dataset's version together in one manifest write, so +readers never see a half-applied change. See [storage](storage.md) for the layout. + +## Two layers: inherited vs. added + +Throughout the docs, capabilities are framed as **L1** (inherited from Lance) or +**L2** (added by OmniGraph): + +| | L1 — from Lance | L2 — added by OmniGraph | +|---|---|---| +| Storage | Columnar Arrow datasets on object storage | Per-type datasets coordinated as one graph | +| Versioning | Per-dataset versions + time travel | [Snapshots](../branching/time-travel.md) across all types at once | +| Branches | Per-dataset branches | [Graph-level branches](../branching/index.md), atomic across types | +| Commits | Per-dataset commits | [Commit DAG](../branching/index.md) for the whole graph; three-way [merge](../branching/merge.md) | +| Indexes | Scalar / vector / full-text indexes | Built per relevant column; graph topology index for traversal | +| Search | Vector + full-text primitives | [`nearest` / `bm25` / `rrf`](../search/index.md) in one query, plus graph traversal | +| Querying | — | The [`.gq` query language](../queries/index.md) and [`.pg` schema language](../schema/index.md) | + +## How the pieces fit + +- The **schema** (`.pg`) and **query** (`.gq`) languages are compiled to a typed + intermediate representation. +- The **engine** runs queries and mutations against Lance, coordinates the manifest, + maintains the commit graph, and builds indexes. +- The **CLI** ([`omnigraph`](../cli/index.md)) and the + **HTTP server** ([`operations/server.md`](../operations/server.md)) are two front + ends over the same engine, so embedded and remote behavior match. +- [Cedar policy](../operations/policy.md) enforcement is engine-wide — every writer + goes through the same authorization gate regardless of front end. + +For deployment-scale topics — multi-graph servers, control-plane operations, +recovery — see [clusters](../clusters/index.md). diff --git a/docs/user/index.md b/docs/user/index.md index c47b79b..cabd98a 100644 --- a/docs/user/index.md +++ b/docs/user/index.md @@ -12,6 +12,8 @@ start with install, then follow the section that matches your task. | Goal | Read | |---|---| | Install OmniGraph | [install.md](install.md) | +| Run the core loop end to end | [quickstart.md](quickstart.md) | +| Understand the model | [concepts/index.md](concepts/index.md) | | Run the CLI | [cli/index.md](cli/index.md) | | Look up every CLI flag and config field | [cli/reference.md](cli/reference.md) | @@ -21,8 +23,9 @@ start with install, then follow the section that matches your task. |---|---| | Write schemas (the `.pg` language) | [schema/index.md](schema/index.md) | | Read schema-lint diagnostic codes | [schema/lint.md](schema/lint.md) | -| Write queries and mutations (the `.gq` language) | [queries/index.md](queries/index.md) | -| Use vector / full-text / hybrid search | [search/indexes.md](search/indexes.md) | +| Write queries (the `.gq` language) | [queries/index.md](queries/index.md) | +| Write data — inserts, updates, deletes | [mutations/index.md](mutations/index.md) | +| Use vector / full-text / hybrid search | [search/index.md](search/index.md) | | Generate embeddings | [search/embeddings.md](search/embeddings.md) | | Build and use indexes | [search/indexes.md](search/indexes.md) | @@ -30,7 +33,9 @@ start with install, then follow the section that matches your task. | Goal | Read | |---|---| -| Work with branches, commits, and snapshots | [branching/index.md](branching/index.md) | +| Work with branches and commits | [branching/index.md](branching/index.md) | +| Read past versions (time travel) | [branching/time-travel.md](branching/time-travel.md) | +| Merge branches and resolve conflicts | [branching/merge.md](branching/merge.md) | | Coordinate multi-query workflows | [branching/transactions.md](branching/transactions.md) | | Read diffs and change feeds | [branching/changes.md](branching/changes.md) | @@ -56,6 +61,7 @@ start with install, then follow the section that matches your task. | Goal | Read | |---|---| +| Understand the model and L1/L2 framing | [concepts/index.md](concepts/index.md) | | Understand graph layout and URI support | [concepts/storage.md](concepts/storage.md) | | Look up constants and tunables | [reference/constants.md](reference/constants.md) | diff --git a/docs/user/mutations/index.md b/docs/user/mutations/index.md new file mode 100644 index 0000000..2602ae5 --- /dev/null +++ b/docs/user/mutations/index.md @@ -0,0 +1,52 @@ +# Mutations + +Write statements live inside a `query` declaration whose body is one or more +mutation statements (the [query language](../queries/index.md) covers the read +shape and shared declaration syntax). + +``` +query onboard($name: String, $title: String) { + insert Person { name: $name, title: $title } +} +``` + +An edge type is inserted the same way — its endpoint columns are just +properties in the assignment block (`insert WorksAt { person: $p, org: $o }`). + +## Statements + +- `insert <Type> { prop: <value>, … }` +- `update <Type> set { prop: <value>, … } where <prop> <op> <value>` +- `delete <Type> where <prop> <op> <value>` + +`<value>` is a literal, `$param`, or `now()`. + +## Atomicity + +A change query publishes **one commit** at the end of the query. Multiple +insert/update statements accumulate in memory and commit together — a mid-query +failure leaves the graph untouched. See [transactions](../branching/transactions.md) +for the per-query atomicity contract and [branches](../branching/index.md) for +multi-query workflows. + +## Inserts/updates and deletes cannot mix in one query + +A single change query must be **either insert/update-only or delete-only**. +Mixing the two is rejected at parse time, before any I/O: + +> `mutation '<name>' on the same query mixes inserts/updates and deletes; split +> into separate mutations: (1) inserts and updates, then (2) deletes.` + +Run two separate queries instead — the inserts/updates first, then the deletes. +The restriction exists because inserts/updates and deletes commit through +different paths today, and mixing them in one query creates ordering hazards +(e.g. a same-row insert-then-delete, or a cascading delete of a just-inserted +edge). Keeping the two kinds in separate queries keeps each one atomic and +correct. + +## Bulk loading + +For loading data from files rather than inline statements, use +[`omnigraph load`](../cli/index.md) (`--mode overwrite|append|merge`) — it is the +single bulk-write command and applies the same schema validation and atomic +publish as inline mutations. diff --git a/docs/user/operations/audit.md b/docs/user/operations/audit.md index 845c2e0..7e8b24d 100644 --- a/docs/user/operations/audit.md +++ b/docs/user/operations/audit.md @@ -1,7 +1,46 @@ -# Audit / Actor tracking +# Audit & Actor Tracking -- `Omnigraph::audit_actor_id: Option<String>` is the actor in effect. -- `_as` variants of every write API let callers override the actor: `mutate_as`, `load_as`, `branch_merge_as`, `apply_schema_as`, etc. -- Actor IDs are persisted on `GraphCommit.actor_id` with split storage in `_graph_commit_actors.lance` (the commit graph is split into `_graph_commits.lance` for the linkage and `_graph_commit_actors.lance` for the actor map). -- HTTP server uses the bearer-token actor automatically. The CLI resolves one actor chain everywhere: `--as` > legacy `cli.actor` in `omnigraph.yaml` > `operator.actor` in `~/.omnigraph/config.yaml` > none (RFC-007). -- Pre-v0.4.0 graphs also stored actor IDs on `RunRecord.actor_id` in `_graph_runs.lance` / `_graph_run_actors.lance`. The Run state machine was removed in MR-771; those files are inert post-v0.4.0. The v2→v3 manifest migration sweeps any stale `__run__*` branches on first write-open (MR-770); the inert dataset bytes remain until a `delete_prefix` primitive lands. +Every write in OmniGraph records **who made it**. The actor id is persisted on the +graph commit, so the commit history is an audit trail of which actor changed the +graph and when. + +## Where the actor comes from + +The actor is resolved differently depending on the front end, but it always lands +on the commit: + +- **HTTP server** — the actor is resolved **server-side from the bearer token**. A + client cannot set its own actor id; it is derived from the authenticated token. + See [policy](policy.md) for how tokens map to actors. +- **CLI / embedded** — the actor is self-declared through one resolution chain: + + 1. `--as <actor>` on the command, + 2. then `operator.actor` in `~/.omnigraph/config.yaml` (see the + [CLI reference](../cli/reference.md)), + 3. otherwise none. + +This difference is intentional: storage credentials imply a self-declared actor, +while a server resolves the actor from a token it trusts. + +## Reading the audit trail + +Actor ids are stored on each commit in the [commit graph](../branching/index.md). +List commits to see who made each change: + +```bash +omnigraph commit list graph.omni +``` + +System-initiated writes use reserved actor ids — for example, automatic recovery +of an interrupted write records `omnigraph:recovery`, so operator changes and +machine repairs are distinguishable in the history: + +```bash +omnigraph commit list --filter actor=omnigraph:recovery graph.omni +``` + +## What is tracked + +Every successful publish — load, change, branch merge, and schema apply — appends a +commit carrying the resolving actor. Because publishes are atomic, the actor on a +commit is exactly the actor responsible for that whole change. diff --git a/docs/user/queries/index.md b/docs/user/queries/index.md index 0942d50..c00d1a9 100644 --- a/docs/user/queries/index.md +++ b/docs/user/queries/index.md @@ -13,8 +13,11 @@ query <name>($p1: T1, $p2: T2?, …) Two body shapes: -- **Read**: `match { … } return { … } [order { … }] [limit N]` -- **Mutation**: one or more of `insert | update | delete` statements +- **Read**: `match { … } return { … } [order { … }] [limit N]` — covered on this page. +- **Mutation**: one or more of `insert | update | delete` statements — see [mutations](../mutations/index.md). + +Multi-modal search functions (`nearest`, `bm25`, `rrf`, …) used inside `match`, +`return`, and `order` are documented on the [search](../search/index.md) page. Param types reuse all schema scalars; trailing `?` makes a param optional. The compiler reserves `$__nanograph_now` for `now()`. @@ -25,21 +28,6 @@ Param types reuse all schema scalars; trailing `?` makes a param optional. The c - **Filter**: `<expr> <op> <expr>` with operators `>=`, `<=`, `!=`, `>`, `<`, `=`, and string `contains`. - **Negation**: `not { clause+ }` — desugars to anti-join over the inner pipeline. -## Search clauses (multi-modal) - -Used inside MATCH or as expressions inside RETURN/ORDER: - -| Function | Purpose | Underlying Lance facility | -|---|---|---| -| `nearest($x.vec, $q)` | k-NN vector search (cosine) | Lance vector index (IVF / HNSW) | -| `search(field, q)` | Generic FTS | Inverted index | -| `fuzzy(field, q [, max_edits])` | Levenshtein-tolerant text search | Inverted index | -| `match_text(field, q)` | Pattern match | Inverted index | -| `bm25(field, q)` | BM25 scoring | Inverted index | -| `rrf(rank_a, rank_b [, k])` | Reciprocal Rank Fusion of two rankings (default k=60) | OmniGraph fuses scored rankings | - -`nearest()` requires a `LIMIT`; the compiler resolves the query vector via the param map (or via the runtime embedding client when bound to a text input). - ## RETURN clause `return { <expr> [as <alias>], … }` with expressions: @@ -48,7 +36,7 @@ Used inside MATCH or as expressions inside RETURN/ORDER: - Literals: string, int, float, bool, list - `now()` - Aggregates: `count`, `sum`, `avg`, `min`, `max` -- All search functions above (so you can return a score column) +- [Search functions](../search/index.md) (so you can return a score column) - `AliasRef` — re-use a previous projection alias ## ORDER & LIMIT @@ -58,21 +46,8 @@ Used inside MATCH or as expressions inside RETURN/ORDER: - **Total, deterministic order.** Rows with equal user-sort keys are broken by the bound entities' key columns (`<var>.id`, ascending) appended as a final tie-break, so the result is a *total* order — reproducible across runs, and `order … limit N` returns a deterministic top-N even when ties straddle the cutoff. (Aggregate results have no entity-key columns; their group rows are already distinct on the projected group keys.) - **NULL placement** is *nulls-first ascending, nulls-last descending* (i.e. `nulls_first = !descending`): a NULL sorts as if smaller than any value. -## Mutation statements - -- `insert <Type> { prop: <value>, … }` -- `update <Type> set { prop: <value>, … } where <prop> <op> <value>` -- `delete <Type> where <prop> <op> <value>` - -`<value>` is a literal, `$param`, or `now()`. Multi-statement mutations execute atomically (added in v0.2.0). - -### D₂ — mixed insert/update + delete is rejected at parse time - -A single mutation query must be **either insert/update-only or delete-only**. Mixed → rejected before any I/O with the message: - -> `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes. This restriction lifts when Lance exposes a two-phase delete API (tracked: MR-793 / Lance-upstream).` - -Reason: under the staged-write rewire (MR-794), inserts and updates accumulate in memory and commit at end-of-query, while deletes still inline-commit (Lance v6.0.1 has no public two-phase delete). Mixing creates ordering hazards (same-row insert→delete becomes a no-op because the staged insert isn't visible to delete; cascading deletes of just-inserted edges break referential integrity by silent design). Until the MR-A Lance v7 bump migrates `delete_where` to staged (`DeleteBuilder::execute_uncommitted` first ships in `v7.0.0-beta.10`), the parse-time rejection keeps both paths atomic and correct. See [docs/dev/writes.md](../../dev/writes.md), [docs/dev/lance.md](../../dev/lance.md), and [docs/dev/invariants.md](../../dev/invariants.md). +Write statements (`insert` / `update` / `delete`) are documented on the +[mutations](../mutations/index.md) page. ## IR (Intermediate Representation) diff --git a/docs/user/quickstart.md b/docs/user/quickstart.md new file mode 100644 index 0000000..b39ff1b --- /dev/null +++ b/docs/user/quickstart.md @@ -0,0 +1,81 @@ +# Quickstart + +This walks the core loop end to end: define a schema, initialize a graph, load +data, query it, and use a branch. It uses a local file-backed graph; swap the +path for an `s3://…` URI to run the same flow against object storage. + +[Install](install.md) the `omnigraph` CLI first. + +## 1. Write a schema + +A schema (`.pg`) declares your node and edge types. Save this as `schema.pg`: + +``` +node Person { + name: String, + title: String?, +} +``` + +See the [schema language](schema/index.md) for types, constraints, and edges. + +## 2. Initialize the graph + +```bash +omnigraph init --schema schema.pg graph.omni +``` + +`init` creates an empty graph at the given URI with your schema applied. + +## 3. Load data + +`load` is the single bulk-write command. `--mode` is required +(`overwrite | append | merge`): + +```bash +omnigraph load --data people.jsonl --mode overwrite graph.omni +``` + +`people.jsonl` is newline-delimited JSON, one record per line. For finer-grained +or inline writes, see [mutations](mutations/index.md). + +## 4. Query + +Write a query (`.gq`) — save as `queries.gq`: + +```gq +query find_people($title: String) { + match { $p: Person { title: $title } } + return { $p.name } +} +``` + +Run it: + +```bash +omnigraph read --query queries.gq --name find_people \ + --params '{"title":"Engineer"}' --format table graph.omni +``` + +The [query language](queries/index.md) covers `match`/`return`/`order`, and +[search](search/index.md) covers vector and full-text search. + +## 5. Work on a branch + +Branches isolate changes until you merge them — Git-style, across the whole graph: + +```bash +omnigraph branch create review/new-hires graph.omni +omnigraph load --data new-hires.jsonl --mode append --branch review/new-hires graph.omni +# inspect the branch, then integrate it +omnigraph branch merge review/new-hires --into main graph.omni +``` + +See [branches & commits](branching/index.md) and [merging](branching/merge.md). + +## Next steps + +- [CLI reference](cli/reference.md) — every command and flag. +- [Schema language](schema/index.md) and [query language](queries/index.md). +- [Operating a cluster](clusters/index.md) and [running the server](operations/server.md) + for multi-graph, multi-user deployments. diff --git a/docs/user/search/index.md b/docs/user/search/index.md new file mode 100644 index 0000000..280e9e8 --- /dev/null +++ b/docs/user/search/index.md @@ -0,0 +1,48 @@ +# Search + +OmniGraph runs vector, full-text, and hybrid search in the same runtime as graph +traversal — a single [query](../queries/index.md) can combine a vector `nearest`, +a `bm25` text score, and an `Expand` traversal. Search functions are used inside +`match` (to filter), or as expressions inside `return` / `order` (to score and +rank). + +## Functions + +| Function | Purpose | Backing index | +|---|---|---| +| `nearest($x.vec, $q)` | k-NN vector search (cosine) | vector index (IVF / HNSW) | +| `search(field, q)` | Generic full-text search | inverted (FTS) index | +| `fuzzy(field, q [, max_edits])` | Levenshtein-tolerant text search | inverted index | +| `match_text(field, q)` | Pattern match | inverted index | +| `bm25(field, q)` | BM25 relevance scoring | inverted index | +| `rrf(rank_a, rank_b [, k])` | Reciprocal Rank Fusion of two rankings (default `k=60`) | fuses scored rankings | + +- `nearest()` requires a `limit`. The query vector is resolved from the param map, + or embedded from a text input at runtime via the configured + [embedding client](embeddings.md). +- Scores and ranks propagate as ordinary columns, so you can `return` a score and + `order` by it. + +## Hybrid ranking with `rrf` + +Reciprocal Rank Fusion combines two independent rankings (typically one vector and +one text) into a single fused ranking, without needing the two score scales to be +comparable. Rank each retrieval separately, then fuse: + +```gq +query hybrid($q: String) { + match { $d: Document { } } + return { + $d, + rrf( nearest($d.embedding, $q), bm25($d.body, $q) ) as score + } + order { score desc } + limit 10 +} +``` + +## Indexes and embeddings + +Search functions only work when the backing index exists — see +[indexes](indexes.md) for building vector and inverted indexes, and +[embeddings](embeddings.md) for generating the vectors `nearest` searches over. From 77dffdae928331545b480edd96821ec27a36c8a9 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sun, 14 Jun 2026 14:39:25 +0300 Subject: [PATCH 158/165] =?UTF-8?q?docs(user):=20de-dev=20polish=20?= =?UTF-8?q?=E2=80=94=20strip=20internal=20scaffolding=20from=20user=20docs?= =?UTF-8?q?=20(Phase=203a)=20(#226)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove developer-only scaffolding that leaked into the public user/operator docs, while preserving every user-facing behavior, command, flag, endpoint, constant, and env var. No behavior changes. Removed across 18 files: - internal ticket / sequencing refs (MR-NNN, RFC-NNN, "Phase N"); - source-code paths (crates/**/*.rs, *.pest) and internal struct/function dumps (e.g. the QueryIR / GraphCommit / SchemaMigrationPlan Rust types, internal fn names like fork_branch_from_state, optimize_all_tables); - Lance-internal blocker prose (upstream issue numbers, blob-decode cause, sidecar Phase-B/C mechanics) — keeping the user-visible behavior (e.g. "optimize skips Blob-column tables; reads/writes unaffected"); - pre-v0.4.0 Run-state-machine archaeology. Internal IR/lowering/recovery-internals sections were either trimmed to a brief user-facing note (e.g. "Traversal execution", "interrupted writes recover automatically; recovery commits are recorded under actor omnigraph:recovery") or removed. Kept: all language syntax, lint codes, Cedar actions/scopes, endpoints, error taxonomy, every constant and env var (verified none dropped from the constants cheat-sheet), and the operator-facing explanations of on-disk artifacts. Residual "legacy" mentions are all user-facing (the deprecated omnigraph.yaml, the legacy token chain, old command names). Verified: zero internal-scaffolding leaks (MR/RFC/Phase/.rs/.pest = 0) across docs/user; zero broken links; check-agents-md.sh green. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- docs/user/branching/changes.md | 2 +- docs/user/branching/index.md | 49 ++++++-------------- docs/user/branching/transactions.md | 6 +-- docs/user/cli/index.md | 2 +- docs/user/cli/reference.md | 13 +++--- docs/user/clusters/config.md | 2 - docs/user/concepts/storage.md | 44 +++++++++--------- docs/user/deployment.md | 3 +- docs/user/operations/errors.md | 2 +- docs/user/operations/maintenance.md | 56 +++++++++++------------ docs/user/operations/policy.md | 40 ++++++++-------- docs/user/operations/server.md | 71 ++++++++++++++--------------- docs/user/queries/index.md | 33 ++------------ docs/user/reference/constants.md | 40 ++++++++-------- docs/user/schema/index.md | 37 ++++++--------- docs/user/schema/lint.md | 43 ++++++++--------- docs/user/search/embeddings.md | 4 +- docs/user/search/indexes.md | 11 ++--- 18 files changed, 192 insertions(+), 266 deletions(-) diff --git a/docs/user/branching/changes.md b/docs/user/branching/changes.md index 58739e2..a9bceec 100644 --- a/docs/user/branching/changes.md +++ b/docs/user/branching/changes.md @@ -1,6 +1,6 @@ # Change Detection / Diff -`changes/mod.rs`. Three-level algorithm: +Diffing two read targets uses a three-level algorithm: 1. **Manifest diff**: skip sub-tables whose `(table_version, table_branch)` is unchanged. 2. **Lineage check**: diff --git a/docs/user/branching/index.md b/docs/user/branching/index.md index a0f1a6e..20ea125 100644 --- a/docs/user/branching/index.md +++ b/docs/user/branching/index.md @@ -2,44 +2,24 @@ ## L1 — Lance per-dataset branches -Lance supports branching at the dataset level: a branch is a named lineage of versions, and `fork_branch_from_state(source_branch, target_branch, source_version)` creates a copy-on-write fork. +Lance supports branching at the dataset level: a branch is a named lineage of versions, and a copy-on-write fork creates a new branch from a source branch at a given version. ## L2 — Graph-level branches OmniGraph builds *graph branches* on top by branching every sub-table coherently: -- `branch_create(name)` / `branch_create_from(target, name)` — disallowed name `main`; fails if branch exists; ensures the schema-apply lock is idle. Atomic and authority-first like `branch_delete`: it flips the `__manifest` branch (authority), then creates the derived commit-graph branch, force-dropping any orphaned commit-graph ref left by an incomplete prior delete (the manifest branch is fresh, so a same-named commit-graph branch is provably a zombie). If commit-graph creation fails, the manifest branch is rolled back so the name never half-exists. -- `branch_list()` — returns public branches, **filters the internal** `__schema_apply_lock__` branch. -- `branch_delete(name)` — refuses if there are descendants on the branch, or if it is the current branch. The manifest is the single authority for branch existence: deletion flips the `__manifest` branch ref first (one atomic op), after which the branch is gone from every snapshot. The owned per-table forks and the commit-graph branch are derived state, reclaimed best-effort with `force_delete_branch` after the flip. A failure during that reclaim (transient object-store error) does not fail the call or block the authority flip; the leftover forks are unreachable orphans that the [`cleanup`](../operations/maintenance.md) reconciler converges. One consequence: if a delete's best-effort reclaim fails, reusing that branch name before the next `cleanup` surfaces a clear error pointing at `cleanup` (the stale fork would otherwise collide on first write). -- **Lazy forking**: a branch only forks a sub-table when that sub-table is first mutated on it. Pure-read branches share fragments with their source. A fork collision is classified by the manifest authority, not by Lance branch versions: if the live manifest already records the fork on the active branch, a concurrent first-write won and the caller gets a retryable "refresh and retry"; if the manifest does not, a physical branch there is an orphan and the caller is pointed at `cleanup`. -- `sync_branch(branch)` — re-binds the in-memory handle to the latest head of the branch. +- **Create** (`branch create` / `branch create --from <target>`) — the name `main` is disallowed; fails if the branch exists. Atomic: the new branch becomes visible all-or-nothing, so a name never half-exists. +- **List** (`branch list`) — returns public branches, **filtering the internal** `__schema_apply_lock__` branch. +- **Delete** (`branch delete`) — refuses if there are descendants on the branch, or if it is the current branch. Once deleted, the branch is gone from every snapshot. The owned per-table forks are reclaimed best-effort; if that reclaim hits a transient object-store error, the leftover storage is reclaimed later by the [`cleanup`](../operations/maintenance.md) command. One consequence: if a delete's reclaim fails, reusing that branch name before the next `cleanup` surfaces a clear error pointing at `cleanup`. +- **Lazy forking**: a branch only forks a sub-table when that sub-table is first mutated on it. Pure-read branches share storage with their source. If two writers race to first-write the same branch, the loser gets a retryable "refresh and retry". -## L2 — Commit graph (`db/commit_graph.rs`) +## L2 — Commit graph -In-memory shape of a graph commit: +Each graph commit carries a ULID id, the manifest branch and version it published, its parent commit (two parents for a merge commit, one for a linear commit), the actor who made it, and a creation timestamp. -``` -GraphCommit { - graph_commit_id: ULID, - manifest_branch: Option<String>, - manifest_version: u64, - parent_commit_id: Option<String>, - merged_parent_commit_id: Option<String>, // populated for merge commits - actor_id: Option<String>, // joined in-memory from _graph_commit_actors.lance, NOT a column on _graph_commits.lance - created_at: i64 (microseconds since epoch), -} -``` - -Storage is split across two Lance datasets (both with stable row IDs): - -- `_graph_commits.lance` — every column above *except* `actor_id`. -- `_graph_commit_actors.lance` — optional separate `(graph_commit_id, actor_id)` map, created on demand. The `actor_id` field above is populated by joining this dataset in-memory at load time. - -Notes: - -- Every successful publish (load / change / merge / schema_apply) appends one commit. +- Every successful publish (load / change / merge / schema apply) appends one commit. - Merge commits have two parents; linear commits have one. -- API: `list_commits(branch)`, `get_commit(id)`, `head_commit_id_for_branch(branch)`. +- Inspect history with `commit list` and `commit show`. ## L2 — Snapshots & time travel @@ -49,13 +29,12 @@ conflict kinds are on the [merge](merge.md) page. ## L2 — Internal system branches -Internal or legacy branch refs: - -- `__schema_apply_lock__` — serializes schema migrations; filtered from `branch_list()` but visible to internals. -- `__run__<run-id>` — legacy from the pre-v0.4.0 Run state machine (removed in MR-771). These are swept off `__manifest` on the first read-write open by the v2→v3 internal-schema migration (MR-770), and `__run__*` is no longer a reserved name. Known limitation: a pre-v0.4.0 graph opened **read-only** still surfaces any stale `__run__*` branch in `branch_list()` until its first read-write open (the migration is write-path-only, like all manifest migrations). +- `__schema_apply_lock__` — serializes schema migrations; filtered from `branch list` but used internally. ## L2 — Recovery audit trail -The five migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`, `optimize_all_tables`) protect their multi-table commits with a sidecar at `__recovery/{ulid}.json` written before Phase B and deleted after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the recovery sweep in `crates/omnigraph/src/db/manifest/recovery.rs`: classify per-table state, decide all-or-nothing per sidecar, roll forward / back, record an audit row. +Interrupted multi-table writes are recovered automatically the next time the graph is opened read-write. Recovery commits are recorded in the audit trail under the actor `omnigraph:recovery`, so you can find them with: -Audit rows live in `_graph_commit_recoveries.lance` (sibling to `_graph_commits.lance`) and reference the commit graph by `graph_commit_id`. The linked recovery commit is identified by that same `graph_commit_id`, and `actor_id="omnigraph:recovery"` is stored in `_graph_commit_actors.lance` (joined by `graph_commit_id`) — `_graph_commits.lance` itself does not carry the `actor_id` column. To find recoveries for a specific original actor: `omnigraph commit list --filter actor=omnigraph:recovery`, then join to `_graph_commit_recoveries.lance` by `graph_commit_id` to read `recovery_for_actor`. Schema: see `crates/omnigraph/src/db/recovery_audit.rs`. +```bash +omnigraph commit list --filter actor=omnigraph:recovery +``` diff --git a/docs/user/branching/transactions.md b/docs/user/branching/transactions.md index a5515da..6e6b1c4 100644 --- a/docs/user/branching/transactions.md +++ b/docs/user/branching/transactions.md @@ -107,7 +107,7 @@ Properties: - Each query on the branch is its own publisher commit — so they're individually atomic. Per-query CAS works on branches just like on main. - The branch lives on disk. Process crash mid-workflow? Re-open and resume. - Multiple agents can work on different branches in parallel without blocking each other. -- The merge is a three-way merge at the row level. Conflicts surface as `OmniError::MergeConflicts(Vec<MergeConflict>)`, with structured kinds (`DivergentInsert`, `DivergentUpdate`, `DeleteVsUpdate`, …) so callers can handle them programmatically. +- The merge is a three-way merge at the row level. Conflicts surface as structured merge-conflict kinds (`DivergentInsert`, `DivergentUpdate`, `DeleteVsUpdate`, …) so callers can handle them programmatically. ### 4. Coordinating multiple agents @@ -129,14 +129,14 @@ omnigraph branch merge agent-b/work --into main graph.omni Each agent sees a consistent snapshot of `main` at the time it forked. The first merge to `main` lands as a fast-forward (or a no-op if no concurrent change). The second merge runs three-way: rows touched by both branches surface as `MergeConflict`s for the caller to resolve. -This is the workflow MR-797 / agentic loops are designed around: **branches are the unit of "an agent's working set."** +This is the workflow agentic loops are designed around: **branches are the unit of "an agent's working set."** ## Failure modes | Scenario | What happens | Caller action | |---|---|---| | Single query fails mid-flight | Publisher never publishes; target unchanged | Read the error, decide whether to retry | -| Concurrent writers race the same `(table, branch)` | Publisher CAS rejects the loser with `ManifestConflictDetails::ExpectedVersionMismatch` | Refresh handle, retry the query | +| Concurrent writers race the same `(table, branch)` | Publisher CAS rejects the loser with a version-mismatch conflict | Refresh handle, retry the query | | Branch with N successful mutations, then merge fails (three-way conflict) | Each individual mutation already committed on the branch; merge surfaces `MergeConflicts` | Inspect, decide whether to keep working on the branch, abandon it (`branch_delete`), or resolve and re-merge | | Process crashes mid-branch-workflow | Each completed mutation on the branch is durable | Re-open the graph, continue where you left off | diff --git a/docs/user/cli/index.md b/docs/user/cli/index.md index a6ce442..6813744 100644 --- a/docs/user/cli/index.md +++ b/docs/user/cli/index.md @@ -106,7 +106,7 @@ omnigraph commit list graph.omni --json omnigraph commit show --uri graph.omni <commit-id> --json ``` -(The legacy `omnigraph run list/show/publish/abort` subcommands were removed in MR-771; mutations and loads publish atomically and the commit graph (`omnigraph commit list`) is the audit surface.) +(Mutations and loads publish atomically; the commit graph (`omnigraph commit list`) is the audit surface.) `query lint` and `query check` are the same command surface. In v1, graph-backed lint uses local or `s3://` graph URIs; HTTP targets are only supported when you diff --git a/docs/user/cli/reference.md b/docs/user/cli/reference.md index bb73225..77feaf1 100644 --- a/docs/user/cli/reference.md +++ b/docs/user/cli/reference.md @@ -8,7 +8,7 @@ Top-level command families and subcommands. Graph-targeting commands accept a po | Command | Purpose | |---|---| -| `init` | `--schema <pg>` → initialize a graph (no longer scaffolds `omnigraph.yaml` — RFC-008; start cluster configs from the [cluster.md](../clusters/index.md) quick-start or `config migrate`) | +| `init` | `--schema <pg>` → initialize a graph (no longer scaffolds `omnigraph.yaml`; start cluster configs from the [cluster.md](../clusters/index.md) quick-start or `config migrate`) | | `load` | bulk load a branch, local or remote (`--mode overwrite\|append\|merge` is **required** — overwrite is destructive, so there is no default). Without `--from` the target branch must exist; `--from <base>` forks a missing `--branch` from `<base>` first | | `ingest` | deprecated alias of `load --from <base>` (defaults: `--from main --mode merge`); prints a one-line warning to stderr | | `query` (alias: `read`) | run named read query; source via `--query <path>`, `-e`/`--query-string <GQ>`, or `--alias <name>` (exactly one). `read` is the deprecated previous name and prints a one-line warning to stderr | @@ -19,7 +19,7 @@ Top-level command families and subcommands. Graph-targeting commands accept a po | `commit list \| show` | inspect commit graph | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | -| `config migrate` | propose (or `--write`: apply) the RFC-008 split of a legacy `omnigraph.yaml` — team half → ready-to-review `cluster.yaml`, personal half → `~/.omnigraph/config.yaml` (key-level merge, existing entries win), plus dropped-key reasons and manual steps | +| `config migrate` | propose (or `--write`: apply) the split of a legacy `omnigraph.yaml` — team half → ready-to-review `cluster.yaml`, personal half → `~/.omnigraph/config.yaml` (key-level merge, existing entries win), plus dropped-key reasons and manual steps | | `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | declarative cluster control plane. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`, annotates dispositions, and embeds real schema-migration previews; `apply` converges the cluster — stored-query/policy catalog writes (content-addressed under `__cluster/resources/`), graph creates, schema updates (soft drops only; `--as` records the actor), and graph deletes behind a digest-bound approval from `cluster approve <resource> --as <actor>` (`apply`/`approve` default the actor from the per-operator `omnigraph.yaml`'s `cli.actor` when `--as` is omitted; nothing else in that file affects cluster commands); what apply converges is what an `omnigraph-server --cluster <dir>` deployment serves on its next restart (omnigraph.yaml deployments are unaffected); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) | | `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review | @@ -30,7 +30,7 @@ Top-level command families and subcommands. Graph-targeting commands accept a po ## Command planes -Every command lives on one **plane**, which determines how it reaches a graph and which addressing flags apply (RFC-010): +Every command lives on one **plane**, which determines how it reaches a graph and which addressing flags apply: - **Data plane** — `query`, `mutate`, `load`, `ingest`, `branch *`, `snapshot`, `export`, `commit *`, `schema show`, `schema apply` (and `graphs list`, remote-only today). Run against a graph **embedded or via a server**: accept a positional `URI` / `--target` / `--server` (+ `--graph` for multi-graph servers). - **Storage / maintenance plane** — `init`, `optimize`, `repair`, `cleanup`, `schema plan`, `queries validate`, `lint`. Run with **direct storage access** (`file://` / `s3://`), never through a server. They accept a positional `URI` or `--target`, but **not** `--server` / `--graph`, and a `--target` that resolves to a remote (`http(s)://`) server is rejected. (`init` takes only a positional `URI` today — no `--target`.) `optimize` / `repair` / `cleanup` also accept **`--cluster <dir|s3://…> --cluster-graph <id>`**, which resolves the graph's storage URI from the served cluster state (so you needn't know the `<storage>/graphs/<id>.omni` layout). @@ -48,8 +48,7 @@ To maintain a server-backed graph, run the maintenance verbs from a host with st ## Config surfaces -Two config surfaces with single owners (RFC-007/RFC-008), plus a zero-config -tier: +Two config surfaces with single owners, plus a zero-config tier: | Surface | Owner | Location | Declares | |---|---|---|---| @@ -58,7 +57,7 @@ tier: | Flags / env | per invocation | — | everything, explicitly | `omnigraph.yaml` (below) is the legacy combined file — fully supported -today, slated for staged deprecation (RFC-008); its keys' future homes are +today, slated for staged deprecation; its keys' future homes are listed there. ### `~/.omnigraph/config.yaml` (operator) @@ -123,7 +122,7 @@ operator server use the legacy chain alone. ## `omnigraph.yaml` schema (legacy combined file) -> **Deprecated (RFC-008).** Loading this file prints a per-key notice +> **Deprecated.** Loading this file prints a per-key notice > naming each present key's new home (suppress in CI with > `OMNIGRAPH_SUPPRESS_YAML_DEPRECATION=1`); `omnigraph config migrate` > produces the split. The file keeps working through the deprecation diff --git a/docs/user/clusters/config.md b/docs/user/clusters/config.md index 5b2e0d5..63d9d8d 100644 --- a/docs/user/clusters/config.md +++ b/docs/user/clusters/config.md @@ -1,7 +1,5 @@ # Cluster Config -**Status:** Phase 5 — cluster-booted serving (`omnigraph-server --cluster`). - > New to the cluster tooling? Start with the operator how-to guide, > [cluster.md](index.md) — this document is the reference. diff --git a/docs/user/concepts/storage.md b/docs/user/concepts/storage.md index 9cc2356..68bfbcc 100644 --- a/docs/user/concepts/storage.md +++ b/docs/user/concepts/storage.md @@ -7,47 +7,45 @@ Every node type and every edge type is its own Lance dataset: - **Columnar Arrow storage**: each property is a column; nullable per Arrow schema. - **Fragments**: data is partitioned into fragments; new writes create new fragments. - **Manifest versioning**: every commit produces a new dataset version; old versions remain readable. -- **Stable row IDs**: `enable_stable_row_ids: true` is set on every Lance dataset OmniGraph creates — node and edge data tables, `__manifest`, `_graph_commits.lance`, `_graph_commit_recoveries.lance`, and any future system tables. This is an architectural invariant: the flag is one-way at dataset create per Lance's row-id-lineage spec, so a future change that introduces a Lance dataset must preserve it. Consequences: `_row_created_at_version` and `_row_last_updated_at_version` are available on every dataset (load-bearing for change-feed validators); `CreateIndex × Rewrite` is not a retryable conflict, so indices survive `omnigraph optimize` without needing the Fragment Reuse Index; readers must use a Lance build that recognises the flag (our pinned 4.0.0 is fine). Pre-0.4.x graphs created before this code path settled may have datasets without the flag and cannot be retrofitted in place — the supported path is dump-and-reload. The `stage_overwrite` rewrite path (used by `schema_apply`) preserves the flag through `Operation::Overwrite`; pinned by `stage_overwrite_preserves_stable_row_ids` in `crates/omnigraph/tests/staged_writes.rs`. +- **Stable row IDs**: stable row IDs are enabled on every Lance dataset OmniGraph creates — node and edge data tables, `__manifest`, the commit-graph datasets, and any future system tables. This is an architectural invariant: the flag is one-way at dataset create, so a future change that introduces a Lance dataset must preserve it. Consequences: `_row_created_at_version` and `_row_last_updated_at_version` are available on every dataset (load-bearing for change-feed validators); indices survive `omnigraph optimize`. Pre-0.4.x graphs created before this code path settled may have datasets without the flag and cannot be retrofitted in place — the supported path is dump-and-reload. The rewrite path used by `schema_apply` preserves the flag. - **Append / delete / `merge_insert`**: native Lance write modes. - **Per-dataset branches** (Lance native): copy-on-write at the dataset level. -- **Object-store agnostic**: file://, s3://, gs://, az://, http (read-only via Lance) — OmniGraph wires file:// and s3:// (`storage.rs`). +- **Object-store agnostic**: file://, s3://, gs://, az://, http (read-only via Lance) — OmniGraph wires file:// and s3://. ## L2 — Multi-dataset coordination via `__manifest` OmniGraph is **not** a single Lance dataset; it is a *graph* of datasets coordinated through one append-only manifest table. - **Manifest table**: `__manifest/` Lance dataset. -- **Layout** (`db/manifest/layout.rs`, `db/manifest/state.rs`): +- **Layout**: - `nodes/{fnv1a64-hex(type_name)}` — one Lance dataset per node type - `edges/{fnv1a64-hex(edge_type_name)}` — one Lance dataset per edge type - `__manifest/` — the catalog of all sub-tables and their published versions - `_graph_commits.lance` / `_graph_commit_actors.lance` — the commit graph and its actor map - - (legacy `_graph_runs.lance` / `_graph_run_actors.lance` from pre-v0.4.0 graphs are inert; the run state machine was removed in MR-771. The v2→v3 manifest migration sweeps stale `__run__*` branches on first write-open; the inert dataset bytes themselves remain until a `delete_prefix` storage primitive lands) + - (legacy `_graph_runs.lance` / `_graph_run_actors.lance` from pre-v0.4.0 graphs are inert; the run state machine was removed. The internal schema migration sweeps stale `__run__*` branches on first write-open; the inert dataset bytes themselves remain until a prefix-delete storage primitive lands) - **Manifest row schema** (`object_id, object_type, location, metadata, base_objects, table_key, table_version, table_branch, row_count`): - `object_type` ∈ `table | table_version | table_tombstone` - `table_key` ∈ `node:<TypeName> | edge:<EdgeName>` - `table_branch` is `null` for the main lineage and the branch name otherwise - **Snapshot reconstruction**: latest visible `table_version` per `(table_key, table_branch)` minus tombstones — rows where `object_type = table_tombstone`, whose own `table_version` (acting as the tombstone version) is `>= the entry's table_version`. -- **Atomic publish**: multi-dataset commits publish via a `ManifestBatchPublisher` so a single write to `__manifest` flips all the new sub-table versions visible at once. -- **Row-level CAS on the merge-insert join key**: `object_id` carries `lance-schema:unenforced-primary-key=true` so Lance's bloom-filter conflict resolver rejects two concurrent commits that land the same `object_id` row. Without this annotation, Lance's transparent rebase would admit silent duplicates of `version:T@v=N` from racing publishers (see `.context/merge-insert-cas-granularity.md`). -- **Optimistic concurrency control on publish**: `ManifestBatchPublisher::publish` accepts a `expected_table_versions: HashMap<table_key, u64>` map. Each entry asserts the manifest's current latest non-tombstoned version for that table is exactly what the caller observed; mismatches surface as `OmniError::Manifest` with `ManifestConflictDetails::ExpectedVersionMismatch { table_key, expected, actual }`. Empty map preserves the legacy "best-effort publish" semantics. The publisher uses `conflict_retries(0)` against Lance and owns retry itself (`PUBLISHER_RETRY_BUDGET = 5`), re-running the pre-check on each iteration so concurrent advances surface as `ExpectedVersionMismatch` rather than being silently rebased through. +- **Atomic publish**: multi-dataset commits publish so that a single write to `__manifest` flips all the new sub-table versions visible at once. +- **Row-level CAS on the merge-insert join key**: `object_id` carries an unenforced-primary-key annotation so Lance's bloom-filter conflict resolver rejects two concurrent commits that land the same `object_id` row. Without this annotation, Lance's transparent rebase would admit silent duplicates from racing publishers. +- **Optimistic concurrency control on publish**: a publish asserts the manifest's current latest non-tombstoned version for each touched table is exactly what the caller observed; mismatches surface as an `ExpectedVersionMismatch` manifest conflict naming the table and the expected/actual versions. Concurrent advances surface as a conflict rather than being silently rebased through. -### Internal schema versioning (`db/manifest/migrations.rs`) +### Internal schema versioning -The on-disk shape of `__manifest` is reconciled with the binary via a single stamp + dispatcher. `INTERNAL_MANIFEST_SCHEMA_VERSION` declares the shape this binary writes; the on-disk stamp `omnigraph:internal_schema_version` lives in the manifest dataset's schema-level metadata (Lance `update_schema_metadata`). +The on-disk shape of `__manifest` is reconciled with the binary via a single version stamp held in the manifest dataset's schema-level metadata. -- **`init_manifest_graph`** stamps the current version at creation, so newly initialized graphs never need migration. -- **Publisher open-for-write path** (`load_publish_state`) calls `migrate_internal_schema(&mut dataset)` before reading state. When the on-disk stamp matches the binary, this is a single metadata read with no writes; otherwise the dispatcher walks `match`-arm steps forward (1→2, 2→3, …) until the stamp matches, then proceeds with the publish. Reads stay side-effect-free. +- **Graph creation** stamps the current version, so newly initialized graphs never need migration. +- **The open-for-write path** migrates the on-disk stamp before reading state. When the stamp matches the binary, this is a single metadata read with no writes; otherwise the migration walks steps forward (1→2, 2→3, …) until the stamp matches, then proceeds with the publish. Reads stay side-effect-free. - **Forward-version protection**: a stamp *higher* than the binary's known version triggers a clear "upgrade omnigraph first" error. An old binary cannot clobber a newer schema by silently treating "unknown stamp" as "missing stamp". -- **Idempotency**: each migration step is safe to re-run. A crash between two metadata updates inside a single step leaves the partial state; the next open re-runs the step and the second update lands. The dispatcher itself is a cheap stamp-read on the steady-state path. - -Adding a new on-disk shape change is one constant bump (`INTERNAL_MANIFEST_SCHEMA_VERSION`), one match arm in `migrate_internal_schema`, and one test. No code outside this module branches on the stamp. +- **Idempotency**: each migration step is safe to re-run. A crash between two metadata updates inside a single step leaves the partial state; the next open re-runs the step and the second update lands. | Stamp | Shape change | |---|---| -| v1 (implicit, pre-stamp) | `__manifest.object_id` had no PK annotation; publisher had no row-level CAS protection. | -| v2 | `__manifest.object_id` carries `lance-schema:unenforced-primary-key=true`; row-level CAS engaged. Stamped as `omnigraph:internal_schema_version=2`. | -| v3 | One-time sweep of legacy `__run__*` staging branches (pre-v0.4.0 Run state machine, removed MR-771) off `__manifest`. Runs at `Omnigraph::open(ReadWrite)` and on publish. Stamped as `omnigraph:internal_schema_version=3`. | +| v1 (implicit, pre-stamp) | `__manifest.object_id` had no PK annotation; no row-level CAS protection. | +| v2 | `__manifest.object_id` carries an unenforced-primary-key annotation; row-level CAS engaged. | +| v3 | One-time sweep of legacy `__run__*` staging branches (pre-v0.4.0 Run state machine, removed) off `__manifest`. Runs at read-write open and on publish. | ## On-disk layout @@ -92,20 +90,20 @@ flowchart TB - **Graph root** is one directory (or S3 prefix). Everything below is part of one OmniGraph graph. - **`__manifest/`** is a Lance dataset whose rows describe which sub-table version is published at which graph-branch. Reading a snapshot starts here. - **`nodes/`** and **`edges/`** are sibling directories holding one Lance dataset per declared type. Names are `fnv1a64-hex` of the type name to keep paths fixed-length and case-safe. -- **`_graph_commits.lance`** is an L2 dataset that records the graph-level commit DAG, with a paired `_graph_commit_actors.lance` for the actor map. (Pre-v0.4.0 graphs also have inert `_graph_runs.lance` / `_graph_run_actors.lance` from the removed Run state machine; the v2→v3 migration sweeps their stale `__run__*` branches, and the dataset bytes are reclaimed once `delete_prefix` lands.) -- **`_graph_commit_recoveries.lance`** — one row per recovery sweep action. Joined to `_graph_commits.lance` by `graph_commit_id`; the linked commit row carries `actor_id=omnigraph:recovery`. Operators correlate recoveries with the original mutations they rolled forward / back via this join. See `crates/omnigraph/src/db/recovery_audit.rs`. -- **`__recovery/{ulid}.json`** — transient sidecar files written by the five migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`, `optimize_all_tables`) before Phase B begins, deleted after Phase C succeeds. A sidecar persisting after process exit means the writer crashed in the Phase B → Phase C window; the next `Omnigraph::open` recovery sweep processes it. Steady-state directory is empty. See `crates/omnigraph/src/db/manifest/recovery.rs`. +- **`_graph_commits.lance`** is an L2 dataset that records the graph-level commit DAG, with a paired `_graph_commit_actors.lance` for the actor map. (Pre-v0.4.0 graphs also have inert `_graph_runs.lance` / `_graph_run_actors.lance` from the removed Run state machine; the internal schema migration sweeps their stale `__run__*` branches, and the dataset bytes are reclaimed once a prefix-delete primitive lands.) +- **`_graph_commit_recoveries.lance`** — one row per crash-recovery action. Joined to `_graph_commits.lance` by `graph_commit_id`; the linked commit row carries `actor_id=omnigraph:recovery`. Operators correlate recoveries with the original mutations they rolled forward / back via this join. +- **`__recovery/{ulid}.json`** — transient sidecar files written by a writer before it advances the underlying dataset, deleted once the matching manifest publish succeeds. A sidecar persisting after process exit means the writer crashed mid-commit; the next read-write open processes it. Steady-state directory is empty. - **`_refs/branches/{name}.json`** is graph-level branch metadata — pointers from a branch name to the manifest version it heads. - **Inside each Lance dataset** (orange): the standard Lance directory layout. `_versions/{n}.manifest` records every commit; `data/` holds the actual Arrow fragments; `_indices/{uuid}/` holds index segments with their own `fragment_bitmap` for partial coverage; `_refs/` holds Lance-native per-dataset branches and tags. The split — L2 owns the cross-dataset catalog; L1 owns the per-dataset internals — means that schema work (which adds or removes datasets) updates `__manifest`, while data work (which adds fragments) updates `_versions/` inside the affected dataset and then bumps `__manifest`. -## URI scheme support (`storage.rs`) +## URI scheme support | Scheme | Backend | Notes | |---|---|---| -| local path / `file://` | `ObjectStorageAdapter` over `object_store::LocalFileSystem` | Normalized to absolute paths; relative and dot-segment paths are lexically absolutized | -| `s3://bucket/prefix` | `ObjectStorageAdapter` over `object_store` S3 | Honors `AWS_ENDPOINT_URL_S3`, `AWS_ALLOW_HTTP`, `AWS_S3_FORCE_PATH_STYLE` | +| local path / `file://` | local filesystem | Normalized to absolute paths; relative and dot-segment paths are lexically absolutized | +| `s3://bucket/prefix` | S3 object store | Honors `AWS_ENDPOINT_URL_S3`, `AWS_ALLOW_HTTP`, `AWS_S3_FORCE_PATH_STYLE` | | `http(s)://host:port` | HTTP client to `omnigraph-server` | Used by CLI as a target, not a storage backend | ## Object-store env vars (S3-compatible) diff --git a/docs/user/deployment.md b/docs/user/deployment.md index 7f134c5..71cd5c8 100644 --- a/docs/user/deployment.md +++ b/docs/user/deployment.md @@ -47,8 +47,7 @@ omnigraph-server s3://my-bucket/graphs/example/releases/2026-04-10-v0.1.0 \ ## Cluster Mode in Containers (AWS, Railway) -A cluster-booted deployment has **two shapes** since the `storage:` root -(RFC-006): +A cluster-booted deployment has **two shapes** since the `storage:` root: - **Bucket, no volume (preferred for cloud)** — the cluster's ledger, catalog, and graph data live under an object-storage root diff --git a/docs/user/operations/errors.md b/docs/user/operations/errors.md index fad39a7..48f1fc9 100644 --- a/docs/user/operations/errors.md +++ b/docs/user/operations/errors.md @@ -9,7 +9,7 @@ - `Manifest(ManifestError { kind: BadRequest|NotFound|Conflict|Internal, details: Option<ManifestConflictDetails>, … })` - `ManifestConflictDetails::ExpectedVersionMismatch { table_key, expected, actual }` — caller's `expected_table_versions` did not match the manifest's current latest non-tombstoned version (set by `OmniError::manifest_expected_version_mismatch`). - `ManifestConflictDetails::RowLevelCasContention` — Lance row-level CAS rejected the publish because a concurrent writer landed the same `object_id`. Retried internally by the publisher; only surfaces if the retry budget exhausts. - - **D₂ parse-time rejection** (MR-794): a single mutation query that mixes inserts/updates with deletes errors out *before any I/O* with kind `BadRequest`. Message: `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes`. See [docs/user/query-language.md](../queries/index.md) for the rule and [docs/dev/writes.md](../../dev/writes.md) for the underlying staged-write rationale. + - **D₂ parse-time rejection**: a single mutation query that mixes inserts/updates with deletes errors out *before any I/O* with kind `BadRequest`. Message: `mutation '<name>' on the same query mixes inserts/updates and deletes; split into separate mutations: (1) inserts and updates, then (2) deletes`. See [query-language.md](../queries/index.md) for the rule. - `MergeConflicts(Vec<MergeConflict>)` Compiler-side `NanoError` covers parse / catalog / type / storage / plan / execution / arrow / lance / IO / manifest / unique-constraint, each with structured spans (`SourceSpan { start, end }`) for ariadne-style diagnostics. diff --git a/docs/user/operations/maintenance.md b/docs/user/operations/maintenance.md index eeeb002..a804e31 100644 --- a/docs/user/operations/maintenance.md +++ b/docs/user/operations/maintenance.md @@ -1,49 +1,47 @@ # Maintenance: Optimize, Repair & Cleanup -`db/omnigraph/optimize.rs` and `db/omnigraph/repair.rs`. +**Addressing.** `optimize`, `repair`, and `cleanup` are **storage-plane** CLI commands: they run with direct storage access against a positional `URI`, `--target`, or **`--cluster <dir|s3://…> --cluster-graph <id>`** (which resolves the graph's storage URI from the served cluster state, so you needn't know the `<storage>/graphs/<id>.omni` layout). They never run through a server, and reject `--server` / `--graph` or a `--target` that resolves to a remote (`http(s)://`) URL with a declared error. There are no server routes for them by design — to maintain a server-backed graph, run them out-of-band against the graph's storage URI. See the *Command planes* section of [cli-reference.md](../cli/reference.md). -**Addressing (RFC-010).** `optimize`, `repair`, and `cleanup` are **storage-plane** CLI commands: they run with direct storage access against a positional `URI`, `--target`, or **`--cluster <dir|s3://…> --cluster-graph <id>`** (which resolves the graph's storage URI from the served cluster state, so you needn't know the `<storage>/graphs/<id>.omni` layout). They never run through a server, and reject `--server` / `--graph` or a `--target` that resolves to a remote (`http(s)://`) URL with a declared error. There are no server routes for them by design — to maintain a server-backed graph, run them out-of-band against the graph's storage URI. See the *Command planes* section of [cli-reference.md](../cli/reference.md). +## `optimize` — non-destructive -## `optimize_all_tables(db)` — non-destructive - -- Lance `compact_files()` on every node + edge table on `main`, then **publishes the compacted version to the `__manifest`** so the manifest's `table_version` tracks the compacted Lance HEAD. Reads pin the manifest version, so without this publish compaction would be invisible to readers *and* would break the HEAD-vs-manifest precondition of the next schema apply / strict update/delete ("stale view … refresh and retry"). The publish advances the graph version (a system-attributed commit) only for tables that actually compacted. -- Rewrites small fragments into fewer large ones; old fragments remain reachable via older manifests until `cleanup` runs. -- Each table's compact→publish runs under its per-`(table, main)` write queue (serializing with concurrent mutations — compaction is a Lance `Rewrite` op that retryable-conflicts with a concurrent merge/update/delete on overlapping fragments). The Lance-HEAD-before-manifest-publish gap is covered by a `SidecarKind::Optimize` recovery sidecar (loose-match): a crash in that window rolls the compacted version forward on the next `Omnigraph::open` (compaction is content-preserving, so roll-forward is always safe). -- **Requires a recovered graph.** `optimize` refuses (errors) when an unresolved recovery sidecar is present under `__recovery` — operating on an unrecovered graph could publish a partial write the open-time recovery sweep would roll back. Reopen the graph to run the recovery sweep, then re-run `optimize`. -- **Uncovered drift is skipped, not interpreted.** If a table's Lance HEAD is ahead of the version recorded in `__manifest` and no recovery sidecar covers that movement, `optimize` reports `skipped: Some(DriftNeedsRepair)` with the manifest/head versions and leaves the table untouched. Run `omnigraph repair` to classify and explicitly publish that drift. +- Compacts every node + edge table on `main`, then **publishes the compacted version to the `__manifest`** so the manifest's recorded version tracks the compacted state. Reads pin the manifest version, so without this publish compaction would be invisible to readers *and* would break the version precondition of the next schema apply / strict update/delete ("stale view … refresh and retry"). The publish advances the graph version (a system-attributed commit) only for tables that actually compacted. +- Rewrites small fragments into fewer large ones; old fragments remain reachable via older versions until `cleanup` runs. +- Each table's compact→publish serializes with concurrent mutations on the same table. A crash mid-operation is recovered automatically on the next open (compaction is content-preserving, so roll-forward is always safe). +- **Requires a recovered graph.** `optimize` refuses (errors) when a pending crash-recovery operation is present — operating on an unrecovered graph could publish a partial write that recovery would roll back. Reopen the graph to run recovery, then re-run `optimize`. +- **Uncovered drift is skipped, not interpreted.** If a table's underlying version is ahead of the version recorded in `__manifest` and no crash-recovery record covers that movement, `optimize` reports `skipped: DriftNeedsRepair` with the manifest/head versions and leaves the table untouched. Run `omnigraph repair` to classify and explicitly publish that drift. - Bounded by `OMNIGRAPH_MAINTENANCE_CONCURRENCY` (default 8). -- Returns `[TableOptimizeStats { table_key, fragments_removed, fragments_added, committed, skipped, manifest_version, lance_head_version }]`. -- **Blob tables are skipped.** A table that declares any `Blob` property is not compacted: it is reported with `skipped: Some(BlobColumnsUnsupportedByLance)` (and logged via `tracing::warn`) instead of compacted, and the rest of the sweep proceeds normally. The current Lance `compact_files` mis-decodes blob-v2 columns under its forced `BlobHandling::AllBinary` read; **reads and writes are unaffected** — only compaction is. This is gated by `LANCE_SUPPORTS_BLOB_COMPACTION` (`db/omnigraph/optimize.rs`) and removed when the upstream Lance fix lands (see [docs/dev/lance.md](../../dev/lance.md)). Consequence: fragment count and deleted-row space on blob tables are not reclaimed until then; query results are never affected. +- Returns per-table stats: `table_key, fragments_removed, fragments_added, committed, skipped, manifest_version, lance_head_version`. +- **Blob tables are skipped.** A table that declares any `Blob` property is not compacted: it is reported with `skipped: BlobColumnsUnsupportedByLance` (and logged) instead of compacted, and the rest of the sweep proceeds normally. **Reads and writes are unaffected** — only compaction is. Consequence: fragment count and deleted-row space on blob tables are not reclaimed; query results are never affected. -## `repair_all_tables(db, options)` — explicit +## `repair` — explicit -- Handles **uncovered manifest/head drift**: a table's Lance HEAD is ahead of the manifest pin and no recovery sidecar records the writer intent. -- Preview by default. `omnigraph repair --json <uri>` reports each table's `classification`, `action`, manifest/head versions, Lance operation names, and any classification error. `--confirm` publishes only verified maintenance drift; if any suspicious or unverifiable table is refused, the CLI prints the per-table output and exits non-zero. `--force --confirm` also publishes suspicious or unverifiable drift after operator review. -- Classifies drift by reading Lance transactions from `manifest_version + 1` through `lance_head_version`. Only `ReserveFragments` and `Rewrite` are verified maintenance. Semantic operations such as `Append`, `Delete`, `Update`, `Merge`, or missing transaction history are not auto-healed. -- Publishes repair by advancing `__manifest` to the existing Lance HEAD; it does **not** rewrite Lance data. If the publish succeeds, normal reads and strict writes use the repaired version. If it fails, no new data-side partial state was created. -- Requires a clean recovery state. Pending `__recovery` sidecars still belong to automatic sidecar recovery, not manual repair. +- Handles **uncovered manifest/head drift**: a table's underlying version is ahead of the manifest pin and no crash-recovery record explains the movement. +- Preview by default. `omnigraph repair --json <uri>` reports each table's `classification`, `action`, manifest/head versions, underlying operation names, and any classification error. `--confirm` publishes only verified maintenance drift; if any suspicious or unverifiable table is refused, the CLI prints the per-table output and exits non-zero. `--force --confirm` also publishes suspicious or unverifiable drift after operator review. +- Classifies drift by reading the table's transaction history from `manifest_version + 1` through the current head. Only fragment-reservation and rewrite (compaction) operations are verified maintenance. Semantic operations such as append, delete, update, merge, or missing transaction history are not auto-healed. +- Publishes repair by advancing `__manifest` to the existing head; it does **not** rewrite data. If the publish succeeds, normal reads and strict writes use the repaired version. If it fails, no new data-side partial state was created. +- Requires a clean recovery state. A pending crash-recovery operation still belongs to automatic recovery, not manual repair. -## `cleanup_all_tables(db, options)` — destructive +## `cleanup` — destructive -- Lance `cleanup_old_versions()` per table. -- Removes manifests (and their unique fragments) older than the retention policy. -- `CleanupPolicyOptions { keep_versions: Option<u32>, older_than: Option<Duration> }` — at least one is required. -- Returns `[TableCleanupStats { table_key, bytes_removed, old_versions_removed, error }]`. +- Garbage-collects old versions per table. +- Removes versions (and their unique fragments) older than the retention policy. +- Policy options `keep_versions` and `older_than` — at least one is required. +- Returns per-table stats: `table_key, bytes_removed, old_versions_removed, error`. - **Fault-isolated per table.** A single table's transient failure (version GC or - orphan reclaim) is recorded on that table's stats row (`error: Some(..)`, logged - via `tracing`) and never aborts the healthy tables — cleanup is the convergence + orphan reclaim) is recorded on that table's stats row (with an `error`) and logged, + and never aborts the healthy tables — cleanup is the convergence backstop, so it does as much as it can and converges on re-run. The CLI reports any failed tables; rerun `cleanup` to retry them. - CLI guards with `--confirm`; without it, prints a preview line. -- **Recovery floor:** `--keep < 3` may garbage-collect Lance versions that the open-time recovery sweep needs as a rollback target (the sweep restores to the branch's manifest-pinned table version, which is HEAD-1 in the typical Phase B → Phase C drift case). Default `--keep 10` is safe. -- **Orphaned-branch reconciliation:** before the version GC, cleanup runs `reconcile_orphaned_branches`, which `force_delete_branch`es any per-table or commit-graph Lance branch absent from the manifest branch list. These orphans arise when a `branch_delete` flips the manifest authority but a downstream best-effort reclaim does not complete (see [branches-commits.md](../branching/index.md)). The reconciler is authority-derived and idempotent (it no-ops once nothing is orphaned), runs regardless of the `keep_versions` / `older_than` values (those gate version GC only), and never reclaims `main` or system-branch forks. Reclaimed forks are logged via `tracing::info`. +- **Recovery floor:** `--keep < 3` may garbage-collect versions that crash recovery needs as a rollback target. Default `--keep 10` is safe. +- **Orphaned-branch reconciliation:** before the version GC, cleanup reclaims any per-table or commit-graph branch absent from the manifest branch list. These orphans arise when a `branch_delete` flips the manifest authority but a downstream best-effort reclaim does not complete (see [branches-commits.md](../branching/index.md)). The reconciler is idempotent (it no-ops once nothing is orphaned), runs regardless of the `keep_versions` / `older_than` values (those gate version GC only), and never reclaims `main` or system-branch forks. Reclaimed forks are logged. ## Tombstones -Logical sub-table delete markers in `__manifest`; `tombstone_object_id(table_key, version)` excludes a sub-table version from snapshot reconstruction. +Logical sub-table delete markers in `__manifest` that exclude a sub-table version from snapshot reconstruction. -## Internal schema migrations (`db/manifest/migrations.rs`) +## Internal schema migrations -Version evolutions of the on-disk `__manifest` shape are reconciled automatically on the first write under a new binary. `INTERNAL_MANIFEST_SCHEMA_VERSION` declares the shape the binary expects; the on-disk stamp `omnigraph:internal_schema_version` (Lance schema-level metadata) records the on-disk shape. The publisher's open-for-write path calls `migrate_internal_schema` before reading state; reads are side-effect-free. No operator action is required for in-place upgrades. See [storage.md → Internal schema versioning](../concepts/storage.md) for the full mechanism. +Version evolutions of the on-disk `__manifest` shape are reconciled automatically on the first write under a new binary. An on-disk stamp records the shape; the binary migrates it forward before reading state, and reads are side-effect-free. No operator action is required for in-place upgrades. See [storage.md → Internal schema versioning](../concepts/storage.md) for the full mechanism. A binary opening a manifest stamped at a version *higher* than it knows about refuses to publish with a clear "upgrade omnigraph first" error — old binaries cannot clobber a newer schema. diff --git a/docs/user/operations/policy.md b/docs/user/operations/policy.md index 91684d8..159ed4d 100644 --- a/docs/user/operations/policy.md +++ b/docs/user/operations/policy.md @@ -13,7 +13,7 @@ Per-graph actions (bind to `Omnigraph::Graph::"<graph_id>"`): 5. `branch_create` 6. `branch_delete` 7. `branch_merge` -8. `admin` — reserved for policy-management surfaces (hot reload, audit log, approvals). No call site today; see MR-724 for the reservation rationale. +8. `admin` — reserved for policy-management surfaces (hot reload, audit log, approvals). No call site today. 9. `invoke_query` — gates invoking a server-side stored query (the `queries:` registry). Graph-scoped (like `admin`) — per-branch access is enforced by the inner `read` / `change` gate, so a rule that sets `branch_scope` on `invoke_query` is rejected. Coarse in this release: an `invoke_query` allow rule permits any stored query on the graph; a future, additive refinement adds an optional per-query-name scope without changing rules written against the coarse action. Enforced at `POST /queries/{name}` (see [server](server.md)). A stored *mutation* is double-gated: `invoke_query` to reach the tool, plus `change` for the write itself (the engine `_as` writers still enforce per the query body). Server-scoped action (v0.6.0+; binds to `Omnigraph::Server::"root"`): @@ -113,20 +113,20 @@ Policy is a property of the **engine**, not the transport. Every mutating write — `mutate_as`, `load_as` (the deprecated `ingest_as` shims route through it), `apply_schema_as`, `branch_create_as`, `branch_create_from_as`, `branch_delete_as`, -`branch_merge_as` — calls `Omnigraph::enforce(action, scope, actor)` at -the head of the method. The gate fires identically whether the call +`branch_merge_as` — consults the policy gate at the head of the method. +The gate fires identically whether the call originates from the HTTP server, the CLI, or an embedded SDK consumer. -When no `PolicyChecker` is installed (the dev/embedded default) the gate +When no policy is installed (the dev/embedded default) the gate is a strict no-op; when one is installed and the call site forgets to thread an actor through, the gate fails closed rather than silently bypassing. -## Server runtime states (MR-723) +## Server runtime states The HTTP server classifies its startup configuration into one of three states based on whether bearer tokens are configured and whether a policy file is set. The state determines what happens to a request that -reaches `authorize_request()` without a matching policy permit. +reaches the authorization gate without a matching policy permit. | State | Tokens | Policy file | Behavior | |---|---|---|---| @@ -134,21 +134,17 @@ reaches `authorize_request()` without a matching policy permit. | **DefaultDeny** | yes | no | Every authenticated request for an action other than `read` is rejected with HTTP 403. Closes the "tokens but forgot the policy file" trap — an operator who sets up auth and forgot to point at a policy file used to ship the illusion of protection. | | **PolicyEnabled** | yes | yes | Authenticated requests that reach a configured policy engine are evaluated by Cedar. Server-scoped actions still require `server.policy.file`. | -The classifier is `classify_server_runtime_state` in -`crates/omnigraph-server/src/lib.rs`; it returns `Err` for the "no -tokens, no policy, no flag" cell and for "policy file, no tokens" so the -server refuses to start instead of silently shipping an open instance or -a policy-protected server that can only 401. Tests pin every cell of the -matrix and the State-2 deny path. +The server refuses to start for the "no tokens, no policy, no flag" cell +and for "policy file, no tokens" — instead of silently shipping an open +instance or a policy-protected server that can only 401. -Server-side, `authorize_request()` still runs at the HTTP boundary — +Server-side, request authorization still runs at the HTTP boundary — that's where actor identity is resolved from the bearer token and where admission control / per-actor rate limits live. Engine-layer enforcement is the **defense in depth** layer: it catches CLI direct-engine writes, embedded SDK consumers, and any future transport that hasn't (or won't) -re-implement HTTP's authorize_request. Both layers consult the same -Cedar policy via the same `PolicyChecker` trait, so decisions cannot -disagree. +re-implement the HTTP boundary's authorization. Both layers consult the same +Cedar policy, so decisions cannot disagree. ## Coarse vs. fine enforcement @@ -157,19 +153,19 @@ responsibilities: | Layer | Question it answers | Where it fires | |---|---|---| -| **Engine-layer (coarse)** | Can this actor invoke this action against this branch / branch-transition? | `Omnigraph::enforce(action, scope, actor)` at the head of every `_as` writer; one Cedar decision per call. | -| **Query-layer (fine)** | For the rows / types this action actually touches, which can the actor see or modify? | Per-row predicates pushed into DataFusion at plan time. **Not yet implemented — see MR-725.** | +| **Engine-layer (coarse)** | Can this actor invoke this action against this branch / branch-transition? | The policy gate at the head of every `_as` writer; one Cedar decision per call. | +| **Query-layer (fine)** | For the rows / types this action actually touches, which can the actor see or modify? | Per-row predicates pushed into the query plan. **Not yet implemented.** | -The engine-layer gate keeps `ResourceScope` deliberately at branch -granularity (`Graph`, `Branch`, `TargetBranch`, `BranchTransition`). +The engine-layer gate keeps its resource scope deliberately at branch +granularity (graph, branch, target branch, branch transition). Per-type and per-row authority is the query-layer's job; conflating them -in `ResourceScope` would create two places per-type policy could be +in the engine-layer scope would create two places per-type policy could be evaluated and a drift surface between them. ## Actor identity (signed-claim-only) The actor identity used for every policy decision comes from the matched bearer token — never from a client-supplied request header, query parameter, or body field. The server resolves the token at the auth middleware boundary, looks up the actor it was minted for, and overwrites whatever the handler may have placed in the policy request. Clients cannot set `actor_id` directly. -This is intentional. Trusting client-supplied identity for authorization is "asking the attacker if they're an admin" — Supabase's RLS history names the same footgun. The chokepoint lives in `authorize_request` in `crates/omnigraph-server/src/lib.rs` and is named in `docs/dev/invariants.md` Hard Invariant 11. A regression test asserts the contract: a request with `Authorization: Bearer <token-for-actor-A>` plus `X-Actor-Id: actor-B` always evaluates as actor A, never as actor B. +This is intentional. Trusting client-supplied identity for authorization is "asking the attacker if they're an admin" — Supabase's RLS history names the same footgun. The chokepoint lives at the server's auth boundary: a request with `Authorization: Bearer <token-for-actor-A>` plus `X-Actor-Id: actor-B` always evaluates as actor A, never as actor B. If you find yourself wanting to let clients override `actor_id` for impersonation, delegation, or service-account flows — that's a feature, but it needs explicit design (e.g., signed delegation claims, an `On-Behalf-Of` audit trail). It is not a convenience knob. diff --git a/docs/user/operations/server.md b/docs/user/operations/server.md index 8e63e99..0eb2ae8 100644 --- a/docs/user/operations/server.md +++ b/docs/user/operations/server.md @@ -1,10 +1,10 @@ # HTTP Server (`omnigraph-server`) -Axum 0.8 + tokio + utoipa-generated OpenAPI. **Two modes** (v0.6.0+): single-graph (legacy) and multi-graph (MR-668), with **two boot sources** for multi mode: `omnigraph.yaml` or — exclusively — a cluster directory (`--cluster`, RFC-005). Mode is inferred from CLI args + config shape. +Axum 0.8 + tokio + utoipa-generated OpenAPI. **Two modes** (v0.6.0+): single-graph and multi-graph, with **two boot sources** for multi mode: `omnigraph.yaml` or — exclusively — a cluster directory (`--cluster`). Mode is inferred from CLI args + config shape. ## Modes -### Single-graph mode (legacy) +### Single-graph mode `omnigraph-server <URI>` or `omnigraph-server --target <name> --config omnigraph.yaml`. Routes are flat — `/snapshot`, `/read`, `/branches`, etc. @@ -14,10 +14,10 @@ Axum 0.8 + tokio + utoipa-generated OpenAPI. **Two modes** (v0.6.0+): single-gra `omnigraph-server --config omnigraph.yaml` with a non-empty `graphs:` map and **no** single-mode selector (no `server.graph`, no `<URI>`, no `--target`). The server opens every configured graph in parallel at startup (bounded concurrency = 4, fail-fast on the first open error). Routes are nested under `/graphs/{graph_id}/...`. Bare flat paths return 404 in multi mode. -### Cluster-booted multi mode (Phase 5) +### Cluster-booted multi mode `omnigraph-server --cluster <dir-or-uri>` boots from the cluster catalog's **applied -revision** (`state.json` + content-addressed blobs) instead of +revision** instead of `omnigraph.yaml` — an exclusive boot source: combining it with `<URI>`, `--target`, or `--config` is a startup error, and `omnigraph.yaml` is never read in this mode. Always multi-graph routing. See @@ -42,34 +42,34 @@ If a graph declares a `queries:` registry (see [cli-reference](../cli/reference. Per-graph endpoints — same body shape across modes; URLs differ: -| Method | Single-mode path | Multi-mode path | Auth | Action | Handler | -|---|---|---|---|---|---| -| GET | `/healthz` | `/healthz` | none | — | `server_health` | -| GET | `/openapi.json` | `/openapi.json` | none | — | `server_openapi` (strips security if auth disabled; in multi mode emits cluster paths with `cluster_` operation-id prefix) | -| GET | `/snapshot?branch=` | `/graphs/{id}/snapshot?branch=` | bearer + `read` | snapshot of branch | `server_snapshot` | -| POST | `/query` | `/graphs/{id}/query` | bearer + `read` | inline read query (canonical; clean field names `query`/`name`; mutations → 400) | `server_query` | -| POST | `/read` | `/graphs/{id}/read` | bearer + `read` | **deprecated** alias of `/query` (legacy field names `query_source`/`query_name`, byte-stable response; carries `Deprecation: true` + `Link: </query>; rel="successor-version"`) | `server_read` | -| POST | `/export` | `/graphs/{id}/export` | bearer + `export` | NDJSON stream | `server_export` | -| POST | `/mutate` | `/graphs/{id}/mutate` | bearer + `change` | mutation (canonical; `query`/`name`; accepts legacy `query_source`/`query_name` as serde aliases) | `server_mutate` | -| POST | `/change` | `/graphs/{id}/change` | bearer + `change` | **deprecated** alias of `/mutate` (carries `Deprecation: true` + `Link: </mutate>; rel="successor-version"`) | `server_change` | -| GET | `/queries` | `/graphs/{id}/queries` | bearer + `read` | list the `mcp.expose` stored queries as a typed tool catalog | `server_list_queries` | -| POST | `/queries/{name}` | `/graphs/{id}/queries/{name}` | bearer + `invoke_query` (+ `change` for a stored mutation) | invoke a named query from the `queries:` registry; deny == 404 | `server_invoke_query` | -| GET | `/schema` | `/graphs/{id}/schema` | bearer + `read` | get current `.pg` source | `server_schema_get` | -| POST | `/schema/apply` | `/graphs/{id}/schema/apply` | bearer + `schema_apply` (target=`main`) | migrate | `server_schema_apply` | -| POST | `/load` | `/graphs/{id}/load` | bearer + `branch_create` (only when `from` is set and the branch is created) + `change` | bulk load (canonical); branch creation is opt-in via `from` — without it a missing `branch` is a 404, never an implicit fork | `server_load` (32 MB body limit) | -| POST | `/ingest` | `/graphs/{id}/ingest` | bearer + `branch_create` (only when `from` is set and the branch is created) + `change` | **deprecated** alias of `/load` (carries `Deprecation: true` + `Link: </load>; rel="successor-version"`) | `server_ingest` (32 MB body limit) | -| GET | `/branches` | `/graphs/{id}/branches` | bearer + `read` | list branches | `server_branch_list` | -| POST | `/branches` | `/graphs/{id}/branches` | bearer + `branch_create` | create | `server_branch_create` | -| DELETE | `/branches/{branch}` | `/graphs/{id}/branches/{branch}` | bearer + `branch_delete` | delete | `server_branch_delete` | -| POST | `/branches/merge` | `/graphs/{id}/branches/merge` | bearer + `branch_merge` | merge `source → target` | `server_branch_merge` | -| GET | `/commits?branch=` | `/graphs/{id}/commits?branch=` | bearer + `read` | list | `server_commit_list` | -| GET | `/commits/{commit_id}` | `/graphs/{id}/commits/{commit_id}` | bearer + `read` | show | `server_commit_show` | +| Method | Single-mode path | Multi-mode path | Auth | Action | +|---|---|---|---|---| +| GET | `/healthz` | `/healthz` | none | — | +| GET | `/openapi.json` | `/openapi.json` | none | — (strips security if auth disabled; in multi mode emits cluster paths with `cluster_` operation-id prefix) | +| GET | `/snapshot?branch=` | `/graphs/{id}/snapshot?branch=` | bearer + `read` | snapshot of branch | +| POST | `/query` | `/graphs/{id}/query` | bearer + `read` | inline read query (canonical; clean field names `query`/`name`; mutations → 400) | +| POST | `/read` | `/graphs/{id}/read` | bearer + `read` | **deprecated** alias of `/query` (legacy field names `query_source`/`query_name`, byte-stable response; carries `Deprecation: true` + `Link: </query>; rel="successor-version"`) | +| POST | `/export` | `/graphs/{id}/export` | bearer + `export` | NDJSON stream | +| POST | `/mutate` | `/graphs/{id}/mutate` | bearer + `change` | mutation (canonical; `query`/`name`; accepts legacy `query_source`/`query_name` as serde aliases) | +| POST | `/change` | `/graphs/{id}/change` | bearer + `change` | **deprecated** alias of `/mutate` (carries `Deprecation: true` + `Link: </mutate>; rel="successor-version"`) | +| GET | `/queries` | `/graphs/{id}/queries` | bearer + `read` | list the `mcp.expose` stored queries as a typed tool catalog | +| POST | `/queries/{name}` | `/graphs/{id}/queries/{name}` | bearer + `invoke_query` (+ `change` for a stored mutation) | invoke a named query from the `queries:` registry; deny == 404 | +| GET | `/schema` | `/graphs/{id}/schema` | bearer + `read` | get current `.pg` source | +| POST | `/schema/apply` | `/graphs/{id}/schema/apply` | bearer + `schema_apply` (target=`main`) | migrate | +| POST | `/load` | `/graphs/{id}/load` | bearer + `branch_create` (only when `from` is set and the branch is created) + `change` | bulk load (canonical); branch creation is opt-in via `from` — without it a missing `branch` is a 404, never an implicit fork (32 MB body limit) | +| POST | `/ingest` | `/graphs/{id}/ingest` | bearer + `branch_create` (only when `from` is set and the branch is created) + `change` | **deprecated** alias of `/load` (carries `Deprecation: true` + `Link: </load>; rel="successor-version"`) (32 MB body limit) | +| GET | `/branches` | `/graphs/{id}/branches` | bearer + `read` | list branches | +| POST | `/branches` | `/graphs/{id}/branches` | bearer + `branch_create` | create | +| DELETE | `/branches/{branch}` | `/graphs/{id}/branches/{branch}` | bearer + `branch_delete` | delete | +| POST | `/branches/merge` | `/graphs/{id}/branches/merge` | bearer + `branch_merge` | merge `source → target` | +| GET | `/commits?branch=` | `/graphs/{id}/commits?branch=` | bearer + `read` | list | +| GET | `/commits/{commit_id}` | `/graphs/{id}/commits/{commit_id}` | bearer + `read` | show | Server-level management endpoints (v0.6.0+): -| Method | Path | Auth | Action | Handler | -|---|---|---|---|---| -| GET | `/graphs` | bearer + `graph_list` on `Server::"root"` | list registered graphs | `server_graphs_list` (405 in single mode) | +| Method | Path | Auth | Action | +|---|---|---|---| +| GET | `/graphs` | bearer + `graph_list` on `Server::"root"` | list registered graphs (405 in single mode) | ### Stored-query catalog (`GET /queries`) @@ -96,9 +96,8 @@ or remove graphs by stopping the server, editing the `graphs:` map in `omnigraph.yaml`, then restarting. The server treats `omnigraph.yaml` as operator-owned configuration and never writes it. -A future release may introduce a managed registry (Lance-backed, -catalog-style: reserve → init → publish with recovery sidecars) and -re-expose runtime mutation on top of it. +A future release may introduce a managed registry and re-expose runtime +mutation on top of it. ## Inline read queries (`POST /query`) @@ -154,7 +153,7 @@ Only `/export` streams (`application/x-ndjson`, MPSC channel + `Body::from_strea Uniform `ErrorOutput { error, code?, merge_conflicts[], manifest_conflict? }` with `code ∈ unauthorized | forbidden | bad_request | not_found | conflict | too_many_requests | internal`. Merge conflicts attach structured `MergeConflictOutput { table_key, row_id?, kind, message }`. -`manifest_conflict` is set on **publisher CAS rejections** (HTTP 409): the +`manifest_conflict` is set on **concurrent-write rejections** (HTTP 409): the caller's pre-write view of one table's manifest version was stale. `ManifestConflictOutput { table_key, expected, actual }` tells the client which table to refresh and retry. This is the conflict shape produced by @@ -169,8 +168,8 @@ Disjoint `(table, branch)` writes from different actors now run concurrently, guarded only by the engine's per-(table, branch) write queue. To keep one heavy actor from exhausting shared capacity (Lance I/O, manifest -churn, network), the server gates mutating handlers through a -`WorkloadController` configured per-process from environment variables: +churn, network), the server gates mutating handlers through per-process +admission limits configured from environment variables: | Env var | Default | Purpose | |---|---|---| @@ -199,7 +198,7 @@ admission-gated. ## Auth model (`bearer + SHA-256`) - Tokens are SHA-256 hashed on startup; plaintext is never persisted in memory. -- Constant-time comparison via `subtle::ConstantTimeEq`. +- Constant-time comparison. - Three sources, in precedence: 1. `OMNIGRAPH_SERVER_BEARER_TOKENS_AWS_SECRET` — AWS Secrets Manager (build with `--features aws`) 2. `OMNIGRAPH_SERVER_BEARER_TOKENS_FILE` or `OMNIGRAPH_SERVER_BEARER_TOKENS_JSON` — JSON `{actor_id: token, …}` diff --git a/docs/user/queries/index.md b/docs/user/queries/index.md index c00d1a9..c8a70c5 100644 --- a/docs/user/queries/index.md +++ b/docs/user/queries/index.md @@ -1,7 +1,5 @@ # Query Language (`.gq`) -Pest grammar at `crates/omnigraph-compiler/src/query/query.pest`. AST in `query/ast.rs`. Type checker in `query/typecheck.rs`. Lowering in `ir/lower.rs`. - ## Query declarations ``` @@ -49,40 +47,19 @@ Param types reuse all schema scalars; trailing `?` makes a param optional. The c Write statements (`insert` / `update` / `delete`) are documented on the [mutations](../mutations/index.md) page. -## IR (Intermediate Representation) +## Traversal execution -`QueryIR { name, params, pipeline: Vec<IROp>, return_exprs, order_by, limit }` +Variable-length traversals (`Expand`) are executed one of two ways, chosen per-expand by a cost model over cheap manifest counts (frontier size, edge count, source-vertex count, hops) plus index coverage: selective traversals (small frontier relative to the source set) resolve neighbors from the persisted `src`/`dst` BTREE (one indexed scan per hop); dense / deep / large-frontier traversals — or those whose BTREE coverage is degraded so a full scan would be paid per hop — use an in-memory CSR adjacency index. Both produce identical results. The `OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER` / `OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS` ceilings bound the *initial dispatch* frontier/hops (beyond them CSR is always used); the cost model estimates total indexed work as ~`hops × frontier × fanout` and prices dense fan-out toward CSR — they are not a hard per-hop bound. `OMNIGRAPH_TRAVERSAL_MODE=indexed|csr` forces a mode (see [constants](../reference/constants.md)). -Pipeline operations: - -- `NodeScan { variable, type_name, filters }` -- `Expand { src_var, dst_var, edge_type, direction (Out|In), dst_type, min_hops, max_hops, dst_filters }` — destination filters are pushed *into* the expand so Lance scalar pushdown can prune. Executed one of two ways, chosen per-expand by a cost model over cheap manifest counts (frontier size, |E|, source-vertex count, hops) plus index coverage: selective traversals (small frontier relative to the source set) resolve neighbors from the persisted `src`/`dst` BTREE (one indexed scan per hop); dense / deep / large-frontier traversals — or those whose BTREE coverage is degraded so a full scan would be paid per hop — use the in-memory CSR adjacency index. Both produce identical results. The `OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER` / `OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS` ceilings bound the *initial dispatch* frontier/hops (beyond them CSR is always used); the cost model estimates total indexed work as ~`hops × frontier × fanout` and prices dense fan-out toward CSR — they are not a hard per-hop bound. `OMNIGRAPH_TRAVERSAL_MODE=indexed|csr` forces a mode (see [constants](../reference/constants.md)). -- `Filter { left, op, right }` -- `AntiJoin { outer_var, inner: Vec<IROp> }` — for `not { … }` - -Lowering: - -1. Partition MATCH clauses (bindings, traversals, filters, negations). -2. Identify "deferred" bindings (a destination of a traversal that has filters) so the Expand can carry the filter as a pushdown. -3. Emit NodeScan for the first binding, then Expand operations, then remaining Filter operations, then AntiJoins for negations. -4. Translate RETURN / ORDER expressions; preserve LIMIT. - -## Linting & validation (`query/lint.rs`) +## Linting & validation Codes seen so far: - **Q000** (Error): parse error - **L201** (Warning): nullable property never set by any UPDATE — "{type}.{prop} exists in schema but no update query sets it" - (Warning): mutation declares no params — hardcoded mutations are easy to miss -- Plus all type errors from `typecheck_query_decl()` (undefined types, mismatched operators, undefined edges, etc.) +- Plus all type errors from type checking (undefined types, mismatched operators, undefined edges, etc.) -Output: - -``` -QueryLintOutput { status, schema_source, query_path, - queries_processed, errors, warnings, infos, - results: [{ name, kind, status, error?, warnings[] }], - findings: [{ severity, code, message, type_name?, property?, query_names[] }] } -``` +Lint output reports an overall status, per-query results (name, kind, status, any error and warnings), and structured findings (severity, code, message, and the type/property/query they apply to). CLI exits non-zero only on `status = Error`. diff --git a/docs/user/reference/constants.md b/docs/user/reference/constants.md index f523042..2cad0d1 100644 --- a/docs/user/reference/constants.md +++ b/docs/user/reference/constants.md @@ -1,26 +1,26 @@ # Constants & Tunables (cheat sheet) -| Name | Value | Where | +| Name | Value | Area | |---|---|---| -| `MANIFEST_DIR` | `__manifest` | `db/manifest/layout.rs` | -| Commit graph dir | `_graph_commits.lance` | `db/commit_graph.rs` | -| Run registry dir (legacy, removed MR-771) | `_graph_runs.lance` | inert post-v0.4.0; bytes remain until a `delete_prefix` primitive lands | -| Run branch prefix (legacy, removed MR-771/MR-770) | `__run__` | swept off `__manifest` by the v2→v3 migration; no longer a reserved name | -| Schema apply lock | `__schema_apply_lock__` | `db/mod.rs` | -| Manifest publisher retry budget | `PUBLISHER_RETRY_BUDGET = 5` | `db/manifest/publisher.rs` | -| Internal manifest schema version | `INTERNAL_MANIFEST_SCHEMA_VERSION = 3` | `db/manifest/migrations.rs` | -| Merge stage batch | `MERGE_STAGE_BATCH_ROWS = 8192` | `exec/merge.rs` | -| Maintenance concurrency | `OMNIGRAPH_MAINTENANCE_CONCURRENCY=8` | `db/omnigraph/optimize.rs` | -| Lance blob compaction support | `LANCE_SUPPORTS_BLOB_COMPACTION = false` | `db/omnigraph/optimize.rs` | -| Graph index cache size | `8` (LRU) | `runtime_cache.rs` | -| Expand indexed-path frontier ceiling | `OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER=1024` | `exec/query.rs` | -| Expand indexed-path hop ceiling | `OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS=6` | `exec/query.rs` | -| Expand CSR-build cost factor | `CSR_BUILD_FACTOR = 1.5` | `exec/query.rs` | -| Expand mode override | `OMNIGRAPH_TRAVERSAL_MODE` (`indexed`\|`csr`; unset = cost-based auto) | `exec/query.rs` | -| Default body limit | `1 MB` | `omnigraph-server/lib.rs` | -| Ingest body limit | `32 MB` | `omnigraph-server/lib.rs` | -| Engine embed model | `gemini-embedding-2-preview` | `omnigraph/embedding.rs` | -| Compiler embed model | `text-embedding-3-small` | `omnigraph-compiler/embedding.rs` | +| `MANIFEST_DIR` | `__manifest` | manifest layout | +| Commit graph dir | `_graph_commits.lance` | commit graph | +| Run registry dir (legacy, removed) | `_graph_runs.lance` | inert post-v0.4.0; bytes remain until a prefix-delete primitive lands | +| Run branch prefix (legacy, removed) | `__run__` | swept off `__manifest` by the internal schema migration; no longer a reserved name | +| Schema apply lock | `__schema_apply_lock__` | schema apply | +| Manifest publisher retry budget | `PUBLISHER_RETRY_BUDGET = 5` | manifest publish | +| Internal manifest schema version | `INTERNAL_MANIFEST_SCHEMA_VERSION = 3` | manifest migrations | +| Merge stage batch | `MERGE_STAGE_BATCH_ROWS = 8192` | merge execution | +| Maintenance concurrency | `OMNIGRAPH_MAINTENANCE_CONCURRENCY=8` | optimize/cleanup | +| Lance blob compaction support | `LANCE_SUPPORTS_BLOB_COMPACTION = false` | optimize | +| Graph index cache size | `8` (LRU) | runtime cache | +| Expand indexed-path frontier ceiling | `OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER=1024` | traversal | +| Expand indexed-path hop ceiling | `OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS=6` | traversal | +| Expand CSR-build cost factor | `CSR_BUILD_FACTOR = 1.5` | traversal | +| Expand mode override | `OMNIGRAPH_TRAVERSAL_MODE` (`indexed`\|`csr`; unset = cost-based auto) | traversal | +| Default body limit | `1 MB` | HTTP server | +| Ingest body limit | `32 MB` | HTTP server | +| Engine embed model | `gemini-embedding-2-preview` | engine embedding | +| Compiler embed model | `text-embedding-3-small` | compiler embedding | | Embed timeout | `30 000 ms` | both clients | | Embed retries | `4` | both clients | | Embed retry backoff | `200 ms` | both clients | diff --git a/docs/user/schema/index.md b/docs/user/schema/index.md index 4250676..d0fcd1b 100644 --- a/docs/user/schema/index.md +++ b/docs/user/schema/index.md @@ -1,7 +1,5 @@ # Schema Language (`.pg`) -Pest grammar at `crates/omnigraph-compiler/src/schema/schema.pest`. AST at `schema/ast.rs`. Catalog at `catalog/mod.rs`. - ## Top-level declarations - `interface <Name> { property* }` — reusable property contracts. @@ -47,37 +45,28 @@ Edge bodies only allow `@unique` and `@index`. - `@<ident>` or `@<ident>(<literal>)` on any declaration or property. - Known annotations: - - `@embed` on a Vector property — names the *source* property whose text gets embedded into this vector at ingest (`embed_sources` map in NodeType). + - `@embed` on a Vector property — names the *source* property whose text gets embedded into this vector at ingest. - `@description("…")`, `@instruction("…")` on query declarations (carried through to clients). - Custom annotations are accepted by the parser and surfaced in catalog metadata; unrecognized annotations don't fail compilation. -## Catalog construction +## Table layout -- Pass 0: collect interfaces. -- Pass 1: collect nodes, expand `implements`, build constraint and `@embed` mappings, build the Arrow schema for each node table (`id: Utf8` plus all properties; blob columns get `LargeBinary`). -- Pass 2: collect edges, validate that `from_type` / `to_type` exist, normalize edge names case-insensitively for lookup, validate constraints for edges. Edge Arrow schema: `id: Utf8, src: Utf8, dst: Utf8` plus edge properties. - -## Schema IR & stable type IDs - -- `SCHEMA_IR_VERSION = 1` (`catalog/schema_ir.rs`). -- Each interface/node/edge currently gets a `stable_type_id` from a kind+name hash. -- Rename-preserving accepted IDs are an architectural invariant, but the current hash-on-name implementation is a known gap until migration carries IDs across `@rename_from`. -- Serialized as JSON for diff/migration plans. +- Each node type compiles to a table with an `id: Utf8` column plus all declared properties (blob columns are stored as `LargeBinary`); `implements` clauses expand the interface's properties into the node. +- Each edge type compiles to a table with `id: Utf8, src: Utf8, dst: Utf8` plus the edge's own properties. Edge endpoint types (`from`/`to`) must exist, and edge names are matched case-insensitively. ## Schema migration planning -`plan_schema_migration(accepted, desired) -> SchemaMigrationPlan { supported, steps[] }` with step types: +A migration plan compares the accepted schema against the desired one and reports whether the change is supported plus the ordered steps it requires: -- `AddType { type_kind, name }` -- `RenameType { type_kind, from, to }` -- `AddProperty { type_kind, type_name, property_name, property_type }` -- `RenameProperty { type_kind, type_name, from, to }` -- `AddConstraint { type_kind, type_name, constraint }` -- `UpdateTypeMetadata { … annotations }` -- `UpdatePropertyMetadata { … annotations }` -- `UnsupportedChange { entity, reason }` (forces `supported=false`) +- Add a type +- Rename a type +- Add a property +- Rename a property +- Add a constraint +- Update type or property metadata (annotations) +- Unsupported change (reports the entity and reason; forces the plan to unsupported) -`apply_schema()` returns `SchemaApplyResult { supported, applied, manifest_version, steps }` and is gated by an internal `__schema_apply_lock__` system branch so concurrent schema applies serialize. +Applying a plan reports whether it was supported, the steps applied, and the resulting manifest version. Concurrent schema applies serialize so they can't interleave. ## Destructive drops — `--allow-data-loss` diff --git a/docs/user/schema/lint.md b/docs/user/schema/lint.md index a1495fd..6635e9f 100644 --- a/docs/user/schema/lint.md +++ b/docs/user/schema/lint.md @@ -2,29 +2,26 @@ The migration planner emits **code-tagged diagnostics** for every schema change it rejects. Codes have the form `OG-XXX-NNN` and identify the rule (not the message); operators reference them in suppression directives, severity overrides, and CI reports. -This page is the catalog of codes shipped today. The chassis behind it is tracked in [MR-694](https://linear.app/modernrelay/issue/MR-694). +This page is the catalog of codes shipped today. -## What's shipped in v0 +## What's shipped -- Stable code attached to every rejection the planner emits (today: 5 of 17 paths — the rest carry `code: None` and are tagged as future work). +- Stable code attached to every rejection the planner emits (today: 5 of 17 paths — the rest are tagged as future work). - Code appears in the user-visible error message: `[OG-DS-104] removing property 'Person.age' is not supported …`. - CLI `omnigraph schema plan` shows the code on `unsupported change …` lines. -- Tests in `tests/schema_apply.rs` assert on codes, not on free-text prose. ## What's not shipped yet -- Severity configuration in `omnigraph.yaml` (planned: `lint: { OG-DS-103: error }`). +- Severity configuration (planned: `lint: { OG-DS-103: error }`). - `@allow(OG-XXX-NNN, "rationale")` suppression directives. -- Pre-migration checks (the `migration_check { … }` block — MR-941). -- The CD / VE / LK / NM families (MR-942..945). -- CI integration (MR-946). -- Cost-class annotations (MR-944). +- Pre-migration checks (the `migration_check { … }` block). +- The CD / VE / LK / NM families. +- CI integration. +- Cost-class annotations. -See the parent chassis issue (MR-694) for the design and the per-family sub-issues for what's planned. +## Code catalog -## Code catalog (v0) - -The chassis defines ten families. Today only DS and MF have emitted codes. The remaining families are reserved for future PRs. +The chassis defines ten families. Today only DS and MF have emitted codes. The remaining families are reserved for future releases. | Code | Family | Tier | Default severity | Meaning | |---|---|---|---|---| @@ -37,24 +34,22 @@ The chassis defines ten families. Today only DS and MF have emitted codes. The r | `OG-MF-104` | Maybe-fail | validated | error | tighten nullable to non-nullable (reserved) | | `OG-MF-106` | Maybe-fail | destructive | error | narrowing scalar type | -The full code catalog source of truth lives in `crates/omnigraph-compiler/src/lint/codes.rs`. CI-level invariants (uniqueness, format, family coverage) are unit-tested in the same module. - ## Families The ten chassis families: | Prefix | Family | Status | |---|---|---| -| **DS** | Destructive (data-loss) | shipped, v0 | -| **MF** | Maybe-fail / data-dependent | shipped, v0 | -| **CD** | Constraint deletion (relaxation warning) | tracked in MR-942 | +| **DS** | Destructive (data-loss) | shipped | +| **MF** | Maybe-fail / data-dependent | shipped | +| **CD** | Constraint deletion (relaxation warning) | planned | | **BC** | Backward-incompatible (rename) | implicit in `@rename_from`; codify later | -| **NM** | Naming conventions | tracked in MR-945 | -| **OW** | Ownership (per-resource Cedar) | tracked in MR-722 | -| **NL** | Non-linear (branch-merge divergence) | stubbed in MR-947 | -| **VE** | Vector / embedding | tracked in MR-943 | -| **ED** | Edge / graph topology | tracked in MR-701, MR-943 | -| **LK** | Lock duration / cost | tracked in MR-944 | +| **NM** | Naming conventions | planned | +| **OW** | Ownership (per-resource Cedar) | planned | +| **NL** | Non-linear (branch-merge divergence) | planned | +| **VE** | Vector / embedding | planned | +| **ED** | Edge / graph topology | planned | +| **LK** | Lock duration / cost | planned | ## Prior art diff --git a/docs/user/search/embeddings.md b/docs/user/search/embeddings.md index 382e683..31455c4 100644 --- a/docs/user/search/embeddings.md +++ b/docs/user/search/embeddings.md @@ -2,14 +2,14 @@ OmniGraph has **two** embedding clients with different defaults and purposes. -## Compiler-side client (`omnigraph-compiler/src/embedding.rs`) — query-time normalization +## Compiler-side client — query-time normalization - Default model: `text-embedding-3-small` (OpenAI-style schema) - Env: `NANOGRAPH_EMBED_MODEL`, `OPENAI_API_KEY`, `OPENAI_BASE_URL` (default `https://api.openai.com/v1`), `NANOGRAPH_EMBEDDINGS_MOCK`, `NANOGRAPH_EMBED_TIMEOUT_MS=30000`, `NANOGRAPH_EMBED_RETRY_ATTEMPTS=4`, `NANOGRAPH_EMBED_RETRY_BACKOFF_MS=200` - Methods: `embed_text(input, expected_dim)`, `embed_texts(inputs, expected_dim)` - Mock mode: deterministic FNV-1a + xorshift64 → L2-normalized vectors -## Engine-side client (`omnigraph/src/embedding.rs`) — runtime ingest +## Engine-side client — runtime ingest - Model: `gemini-embedding-2-preview` - Env: `GEMINI_API_KEY`, `OMNIGRAPH_GEMINI_BASE_URL` (default Google generativelanguage v1beta), `OMNIGRAPH_EMBED_TIMEOUT_MS=30000`, `OMNIGRAPH_EMBED_RETRY_ATTEMPTS=4`, `OMNIGRAPH_EMBED_RETRY_BACKOFF_MS=200`, `OMNIGRAPH_EMBEDDINGS_MOCK` diff --git a/docs/user/search/indexes.md b/docs/user/search/indexes.md index fde9488..84b968d 100644 --- a/docs/user/search/indexes.md +++ b/docs/user/search/indexes.md @@ -15,12 +15,11 @@ - **Lazy branch forking for indexes**: a branch that hasn't mutated a sub-table doesn't need its own index — the main lineage's index is reused until the first write triggers a copy-on-write fork. - Vector index parameters (metric, nlist, nprobe, etc.) are not exposed in the schema; they default at the Lance layer and are picked up automatically when an index is asked for on a Vector column. -## L2 — Graph topology index (`graph_index/mod.rs`) +## L2 — Graph topology index This is OmniGraph-specific (not Lance): -- `TypeIndex`: dense `u32 ↔ String id` mapping per node type. -- `CsrIndex`: Compressed Sparse Row representation of edges per edge type — `offsets[i]..offsets[i+1]` slices into `targets`. -- `GraphIndex { type_indices, csr (out), csc (in) }` — built on demand from a snapshot's edge tables, **lazily**: only when an `Expand` the planner routes to the CSR path (dense / large frontier) or an `AntiJoin` actually needs it. -- Cached in `RuntimeCache::graph_indices` (LRU, max 8 entries, keyed by snapshot id + edge table versions). -- Selective `Expand`s resolve neighbors from the persisted `src`/`dst` BTREE instead (one indexed scan per hop) and never trigger the CSR build; see [query-language](../queries/index.md) → Expand. Pure scans, and queries served entirely by the indexed traversal path, skip it. +- A Compressed Sparse Row (CSR) adjacency representation of edges, with both out- (CSR) and in- (CSC) directions, plus a dense per-node-type id mapping. +- Built on demand from a snapshot's edge tables, **lazily**: only when an `Expand` the planner routes to the CSR path (dense / large frontier) or an `AntiJoin` actually needs it. +- Cached per snapshot (LRU, keyed by snapshot id + edge table versions), so repeat traversals over the same snapshot reuse it. +- Selective `Expand`s resolve neighbors from the persisted `src`/`dst` BTREE instead (one indexed scan per hop) and never trigger the CSR build; see [query-language](../queries/index.md) → Traversal execution. Pure scans, and queries served entirely by the indexed traversal path, skip it. From 1bed9980524095aac169398554b3da8d3e6ee577 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford <ragnor.comerford@gmail.com> Date: Sun, 14 Jun 2026 16:31:19 +0200 Subject: [PATCH 159/165] fix(engine): scalar index coverage + filter literal coercion (query latency) (#216) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(engine): lower date/datetime filter literals as typed Arrow scalars `literal_to_expr` lowered `Date`/`DateTime` query literals as Utf8 strings, relying on DataFusion implicit casts. Against a physical `Date32`/`Date64` column that can coerce the column side (`CAST(col AS Utf8)`), which defeats a scalar BTREE and degrades the scan to a full filtered read. Lower to typed `Date32`/`Date64` scalars instead (reusing the loader's `parse_date32_literal`/`parse_date64_literal`, already used by the in-memory comparison arm), so the predicate stays a direct column comparison and the index is used. Malformed literals fall back to the Utf8 string so pushdown behavior never regresses. Tests: unit goldens asserting the lowered literal is typed (red before, green after) + inline-binding pushdown equality in literal_filters confirming the epoch conversion selects the right rows. * fix(engine): build scalar BTREE for enum and orderable-scalar @index columns `build_indices_on_dataset_for_catalog` only handled `String` (-> FTS) and `Vector` (-> vector). Enums are physically `String`, so an enum `@index` column (e.g. `status`) got an FTS inverted index, which Lance never consults for `=`; and `DateTime`/`Date`/numeric/`Bool` `@index` columns fell through and built nothing. Both meant equality/range filters degraded to full scans with `indices_loaded=0`. Dispatch index kind by property type via a shared `node_prop_index_kind`: enum + orderable scalar -> BTREE, free-text String -> FTS, Vector -> vector, list/Blob -> none. The helper is shared by the builder and `needs_index_work_node` so they cannot drift — the latter decides recovery- sidecar pinning, and under-reporting would leave a HEAD-advancing index build uncovered (invariant 5). Tests: scalar_indexes.rs asserts enum/DateTime/numeric @index columns report `IndexCoverage::Indexed` while free-text String/un-annotated columns stay `Degraded` (negative control). Docs: docs/user/indexes.md. * feat(engine): reindex in optimize to keep index coverage current A scalar/FTS/vector index only covers the fragments it was built over. Rows appended after the build (e.g. `ingest --mode merge`, whose commit does not rebuild an existing index) are scanned unindexed, and `compact_files` rewrites fragments out of coverage. Nothing folded them back in, so coverage decayed as the graph grew — even the id/src/dst BTREEs that power traversal. `optimize_one_table` now runs Lance `optimize_indices` after `compact_files` (incremental merge, not retrain — the same compact->optimize_indices sequence LanceDB's `optimize()` uses) and enters the publish path on compaction work OR stale index coverage (new `TableStore::has_unindexed_fragments`, reusing the fragment_bitmap logic). `optimize_indices` is a committing call with no uncommitted variant in lance-6.0.1, so it is an inline-commit residual covered by the existing `SidecarKind::Optimize` recovery sidecar spanning both ops. Blob-bearing tables are still skipped (the Lance blob-compaction bug is compaction-specific; reindex-for-blob deferred as a noted follow-up). Tests: maintenance.rs asserts an appended fragment is uncovered before and covered after optimize, and idempotency holds (second pass is a no-op). lance_surface_guards pins the `optimize_indices` signature and its incremental- coverage behavior. The existing optimize Phase-B recovery failpoint now also exercises a crash after reindex. Docs: maintenance.md, writes.md, invariants.md, lance.md, AGENTS.md. * fix(engine): coerce pushdown filter literals to the column type Filter literals were pushed to Lance in their natural Arrow type (every integer Int64, every float Float64). Against a narrower indexed column DataFusion widens to the literal's type and casts the COLUMN (`CAST(n32 AS Int64)`), which defeats the scalar BTREE and degrades to a full filtered read. A physical-plan probe confirms it: an Int32 column filtered by an i32 literal uses `ScalarIndexQuery`; by an i64 literal it does not. Thread the scan's `arrow_schema` through `build_lance_filter_expr` -> `ir_filter_to_expr` and coerce each literal operand to the opposite column's exact Arrow type, reusing `projection::literal_to_array` + `arrow_cast` (the same path the in-memory arm uses, so the two arms agree). Coercion never demotes a filter to None: on failure it falls back to the natural literal, because a node scan has no in-memory fallback for inline filters. Supersedes the date-specific change in e4ef67b (PR1): the probe shows dates were never index-defeated — temporal coercion casts the LITERAL, not the column — so PR1's index-use rationale was wrong though harmless. The generic coercion subsumes it; `literal_to_expr`'s date arms revert to the natural Utf8 fallback, and its unit tests now assert the live coerced path. Tests: surface guard `scalar_index_use_requires_matched_literal_type` pins the substrate behavior (matched -> index, widened -> column-cast full scan); unit tests cover Int32/UInt32/Float32 coercion, range op, reversed operand order, and the natural fallback; `literal_filters` adds an I32 column with equality + range and an F32 pushdown case. * fix(engine): only coerce filter literals when the cast is lossless The literal coercion in f064121 narrowed unconditionally. typecheck permits numeric cross-type comparisons (`types_compatible`), so an out-of-domain literal reaches `literal_to_typed_expr` and casts lossily: a fractional float vs an integer column truncates (`{ count: 2.7 }` -> `count = 2`, wrongly matching the count=2 row) and an out-of-range integer overflows to null (`count < 3e9` on I32 -> `count < NULL` -> empty). Both silently change results, and a node scan has no in-memory fallback for inline filters. Add a lossless guard for integer targets: round-trip the cast back to the natural type and, on mismatch, return None so the caller keeps the natural literal (correct via DataFusion coercion; the index is just unused for that out-of-domain predicate). Float targets stay coerced -- narrowing F64 -> F32 is the column's own precision domain, not a value error. Resolves the two valid review findings on PR #216 (Codex float truncation, Greptile out-of-range). Tests: unit cases for fractional/out-of-range fallback vs whole-float/in-range coerce vs F32 exemption; e2e `{ count: 2.7 }` returns no rows. --- AGENTS.md | 4 +- crates/omnigraph/src/db/omnigraph.rs | 2 +- crates/omnigraph/src/db/omnigraph/optimize.rs | 46 ++- .../omnigraph/src/db/omnigraph/table_ops.rs | 132 +++++-- crates/omnigraph/src/exec/projection.rs | 6 +- crates/omnigraph/src/exec/query.rs | 336 ++++++++++++++++-- crates/omnigraph/src/table_store.rs | 30 ++ .../omnigraph/tests/lance_surface_guards.rs | 197 ++++++++++ crates/omnigraph/tests/literal_filters.rs | 59 ++- crates/omnigraph/tests/maintenance.rs | 68 ++++ crates/omnigraph/tests/scalar_indexes.rs | 74 ++++ docs/dev/invariants.md | 2 +- docs/dev/lance.md | 3 +- docs/dev/writes.md | 15 +- docs/user/operations/maintenance.md | 7 +- docs/user/search/indexes.md | 21 +- 16 files changed, 917 insertions(+), 85 deletions(-) create mode 100644 crates/omnigraph/tests/scalar_indexes.rs diff --git a/AGENTS.md b/AGENTS.md index 065e28a..427d976 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -247,10 +247,10 @@ omnigraph policy explain --actor act-alice --action change --branch main | Per-dataset versioning + time travel | ✅ | `snapshot_at_version`, `entity_at`, snapshot-pinned reads across many tables | | Per-dataset branches | ✅ | **Graph-level** branches (atomic across all sub-tables), lazy fork, system branch filtering | | Atomic single-dataset commits | ✅ | **Multi-table publish via three layers**, NOT a single Lance primitive: (1) per-table Lance `commit_staged` for the data write, (2) `__manifest` row-level CAS via `ManifestBatchPublisher` for cross-table ordering, (3) the open-time recovery sweep for the residual gap between (1) and (2). All three layers ship; the five migrated writers (`MutationStaging::finalize`, `schema_apply`, `branch_merge`, `ensure_indices`, `optimize_all_tables`) write a `__recovery/{ulid}.json` sidecar before Phase B and delete it after Phase C. The next `Omnigraph::open` (gated on `OpenMode::ReadWrite`) runs the sweep in `db/manifest/recovery.rs`: classify, decide all-or-nothing per sidecar, roll forward via single `ManifestBatchPublisher::publish` or roll back via `Dataset::restore` followed by a manifest publish of the restored version (so both directions converge to `manifest == HEAD` — no residual drift), and record an audit row in `_graph_commit_recoveries.lance` (queryable via `omnigraph commit list --filter actor=omnigraph:recovery`). The write entry points (`load_as`, `mutate_as`, `apply_schema_as`, `branch_merge_as`) and `refresh` additionally run an in-process roll-forward-only heal (serialized against live writers via the per-table write queues), so a long-lived server converges on its next write without restart; only rollback-eligible sidecars still defer to the next read-write open (a future background reconciler's goal). Engine writes route through a sealed `TableStorage` trait (`db.storage()`) exposing only `stage_*` + `commit_staged` + reads; the inline-commit residuals (`delete_where`, `create_vector_index`) are split onto a separate sealed `InlineCommitResidual` trait reached via `db.storage_inline_residual()` (MR-854), so the default surface cannot couple a write with a HEAD advance — §1 holds by construction. `delete_where` and `create_vector_index` stay inline until upstream Lance ships a public two-phase API ([#6658](https://github.com/lance-format/lance/issues/6658), [#6666](https://github.com/lance-format/lance/issues/6666)); `LoadMode::Overwrite` uses Lance `Overwrite` staged transactions. | -| Compaction (`compact_files`) | ✅ | `omnigraph optimize` orchestrates over all node/edge tables, bounded concurrency; **publishes each compacted table's new version to `__manifest`** (so the manifest tracks the Lance HEAD — required for reads to observe compaction and for schema apply / strict writes to pass their HEAD-vs-manifest precondition), under the per-`(table, main)` write queue with `SidecarKind::Optimize` recovery coverage; **refuses on an unrecovered graph** (errors if a `__recovery` sidecar is pending); **skips uncovered HEAD > manifest drift** with `DriftNeedsRepair` instead of interpreting it; **skips blob-bearing tables** (reported via `TableOptimizeStats.skipped`, not silent), gated on `LANCE_SUPPORTS_BLOB_COMPACTION` until the upstream blob-v2 compaction-decode bug is fixed (see [docs/dev/invariants.md](docs/dev/invariants.md) Known Gaps) | +| Compaction (`compact_files`) + reindex (`optimize_indices`) | ✅ | `omnigraph optimize` orchestrates over all node/edge tables, bounded concurrency; per table runs `compact_files` **then Lance `optimize_indices`** (folds appended/rewritten fragments back into existing indexes — incremental merge, not retrain) and **publishes the resulting version to `__manifest`** (so the manifest tracks the Lance HEAD — required for reads to observe the work and for schema apply / strict writes to pass their HEAD-vs-manifest precondition), under the per-`(table, main)` write queue with `SidecarKind::Optimize` recovery coverage spanning both ops; **commits even with no compaction work if index coverage is stale**; **refuses on an unrecovered graph**; **skips uncovered HEAD > manifest drift** with `DriftNeedsRepair`; **skips blob-bearing tables** (reported via `TableOptimizeStats.skipped`, not silent; reindex is skipped for them too today), gated on `LANCE_SUPPORTS_BLOB_COMPACTION` until the upstream blob-v2 compaction-decode bug is fixed (see [docs/dev/invariants.md](docs/dev/invariants.md) Known Gaps) | | Repair uncovered drift | — | `omnigraph repair` explicitly classifies uncovered table `HEAD > manifest` drift: verified maintenance drift (`ReserveFragments`/`Rewrite`) can be published with `--confirm`; suspicious or unverifiable drift requires `--force --confirm`. Sidecar-covered crash residuals still recover automatically on open. | | Cleanup (`cleanup_old_versions`) | ✅ | `omnigraph cleanup` with `--keep` / `--older-than` policy | -| BTREE / inverted (FTS) / vector indexes | ✅ | `ensure_indices` builds them on every relevant column; idempotent; lazy across branches | +| BTREE / inverted (FTS) / vector indexes | ✅ | `ensure_indices` builds them per `@index`/`@key` column, dispatched by type via `node_prop_index_kind` (enum + orderable scalar → BTREE, free-text String → FTS, Vector → vector); idempotent; lazy across branches. Coverage of fragments appended after build is restored by `optimize`'s `optimize_indices` pass (see Compaction row). | | `merge_insert` upsert | ✅ | `LoadMode::Merge`, mutation `update`/`insert`/`delete` lowering | | Vector search | ✅ | `nearest()` query op; embedding pipeline (Gemini / OpenAI clients); `@embed` in schema | | Full-text search | ✅ | `search/fuzzy/match_text/bm25` query ops | diff --git a/crates/omnigraph/src/db/omnigraph.rs b/crates/omnigraph/src/db/omnigraph.rs index 779a2e0..6c80117 100644 --- a/crates/omnigraph/src/db/omnigraph.rs +++ b/crates/omnigraph/src/db/omnigraph.rs @@ -16,7 +16,7 @@ use lance::dataset::scanner::ColumnOrdering; use lance::datatypes::BlobKind; use omnigraph_compiler::catalog::{Catalog, EdgeType, NodeType}; use omnigraph_compiler::schema::parser::parse_schema; -use omnigraph_compiler::types::ScalarType; +use omnigraph_compiler::types::{PropType, ScalarType}; use omnigraph_compiler::{ DropMode, SchemaIR, SchemaMigrationPlan, SchemaMigrationStep, SchemaTypeKind, build_catalog_from_ir, build_schema_ir, plan_schema_migration, diff --git a/crates/omnigraph/src/db/omnigraph/optimize.rs b/crates/omnigraph/src/db/omnigraph/optimize.rs index 21629a8..9195256 100644 --- a/crates/omnigraph/src/db/omnigraph/optimize.rs +++ b/crates/omnigraph/src/db/omnigraph/optimize.rs @@ -32,6 +32,8 @@ use lance::dataset::cleanup::{CleanupPolicy, RemovalStats}; use lance::dataset::optimize::{ CompactionMetrics, CompactionOptions, compact_files, plan_compaction, }; +use lance::index::DatasetIndexExt; +use lance_index::optimize::OptimizeOptions; use super::*; @@ -361,16 +363,22 @@ async fn optimize_one_table( } // Precise "will it compact?" check — `plan_compaction` also accounts for - // deletion materialization (which can rewrite even a single fragment). A - // steady-state already-compacted table yields an empty plan and is never - // pinned in a sidecar (a zero-commit pin would classify NoMovement on - // recovery and force an all-or-nothing rollback). Uncovered pre-existing - // drift is skipped above and must go through explicit repair. + // deletion materialization (which can rewrite even a single fragment). let options = CompactionOptions::default(); let plan = plan_compaction(&ds, &options) .await .map_err(|e| OmniError::Lance(e.to_string()))?; - if plan.num_tasks() == 0 { + let will_compact = plan.num_tasks() > 0; + // Even when there is nothing to compact, the table may still have index + // work: rows appended since the index was built (e.g. via `ingest --mode + // merge`) are scanned unindexed until folded in. Either compaction or stale + // index coverage is enough to enter the publish path. If NEITHER, this + // table is a no-op and must NOT be pinned in a sidecar — a zero-commit pin + // classifies NoMovement on recovery and forces an all-or-nothing rollback + // of sibling tables' legitimate work. Uncovered pre-existing manifest/HEAD + // drift is skipped above and must go through explicit repair. + let needs_reindex = TableStore::has_unindexed_fragments(&ds).await?; + if !will_compact && !needs_reindex { return Ok(TableOptimizeStats::compacted( table_key, &CompactionMetrics::default(), @@ -378,8 +386,9 @@ async fn optimize_one_table( )); } - // Phase A: recovery sidecar BEFORE compaction advances the Lance HEAD, so a - // crash before the manifest publish rolls forward on next open. + // Phase A: recovery sidecar BEFORE any HEAD-advancing op (compaction or + // index optimize), so a crash before the manifest publish rolls forward on + // next open. let sidecar = crate::db::manifest::new_sidecar( crate::db::manifest::SidecarKind::Optimize, None, @@ -398,11 +407,26 @@ async fn optimize_one_table( let handle = crate::db::manifest::write_sidecar(db.root_uri(), db.storage_adapter(), &sidecar).await?; - // Phase B: compaction (reserve-fragments + rewrite commits advance HEAD). + // Phase B: compaction (if any) then incremental index optimize — both + // advance Lance HEAD inside the sidecar window. `compact_files` rewrites + // fragments and drops them from existing index segments' coverage; + // `optimize_indices` folds the rewritten and any previously-unindexed + // fragments back in (Lance's incremental merge, not a full retrain). This + // is the same compact -> optimize_indices sequencing LanceDB's `optimize()` + // uses. `optimize_indices` is an inline-commit residual: lance-6.0.1 + // exposes no uncommitted variant, so like `compact_files` it commits + // directly and relies on the sidecar for recovery. let version_before = ds.version().version; - let metrics: CompactionMetrics = compact_files(&mut ds, options, None) + let metrics: CompactionMetrics = if will_compact { + compact_files(&mut ds, options, None) + .await + .map_err(|e| OmniError::Lance(e.to_string()))? + } else { + CompactionMetrics::default() + }; + ds.optimize_indices(&OptimizeOptions::default()) .await - .map_err(|e| OmniError::Lance(e.to_string()))?; + .map_err(|e| OmniError::Lance(format!("optimize_indices on {}: {}", table_key, e)))?; let version_after = ds.version().version; let committed = version_after != version_before; diff --git a/crates/omnigraph/src/db/omnigraph/table_ops.rs b/crates/omnigraph/src/db/omnigraph/table_ops.rs index f7a365a..3f40c1d 100644 --- a/crates/omnigraph/src/db/omnigraph/table_ops.rs +++ b/crates/omnigraph/src/db/omnigraph/table_ops.rs @@ -310,6 +310,48 @@ pub(super) async fn ensure_indices_for_branch(db: &Omnigraph, branch: Option<&st Ok(()) } +/// The single scalar/vector index a node property receives from a one-column +/// `@index`/`@key` declaration, or `None` when the property type is not +/// indexable here (a list column or `Blob`). +/// +/// Shared by `build_indices_on_dataset_for_catalog` (which builds the index) +/// and `needs_index_work_node` (which checks coverage to decide recovery- +/// sidecar pinning) so the two cannot drift: an enum or orderable scalar the +/// builder gives a BTREE must also be reported as "needs work" until that +/// BTREE exists, or the HEAD-advancing build would run without sidecar cover. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +enum NodePropIndexKind { + Btree, + Fts, + Vector, +} + +fn node_prop_index_kind(prop_type: &PropType) -> Option<NodePropIndexKind> { + if prop_type.list { + return None; + } + // Enums are physically `String` but filtered by equality, so they take a + // scalar BTREE, not an FTS inverted index (Lance never consults an inverted + // index for `=`/range). Free-text Strings keep FTS for + // `search()`/`match_text`/`bm25`. + let is_enum = prop_type.enum_values.is_some(); + match prop_type.scalar { + ScalarType::String if !is_enum => Some(NodePropIndexKind::Fts), + ScalarType::Vector(_) => Some(NodePropIndexKind::Vector), + ScalarType::String + | ScalarType::DateTime + | ScalarType::Date + | ScalarType::I32 + | ScalarType::I64 + | ScalarType::U32 + | ScalarType::U64 + | ScalarType::F32 + | ScalarType::F64 + | ScalarType::Bool => Some(NodePropIndexKind::Btree), + ScalarType::Blob => None, + } +} + /// Returns true if the node table is missing at least one declared /// scalar/vector index that `build_indices_on_dataset_for_catalog` would /// build AND has at least one row (the ensure_indices loop has @@ -318,11 +360,12 @@ pub(super) async fn ensure_indices_for_branch(db: &Omnigraph, branch: Option<&st /// would force `NoMovement` classification on recovery and trigger the /// all-or-nothing rollback of sibling tables' legitimate index work). /// -/// Per the actual `build_indices_on_dataset_for_catalog` implementation -/// (this file, ~line 419-491), nodes get BTree (id) + per-prop FTS -/// (@search String) + per-prop Vector indices; edges get BTree only -/// (id, src, dst). The two helpers mirror that asymmetry — see the -/// `needs_index_work_edge` doc comment. +/// Per `build_indices_on_dataset_for_catalog`, nodes get BTree (id) plus, for +/// each one-column `@index`/`@key` property, the index `node_prop_index_kind` +/// assigns: a scalar BTREE for enums and orderable scalars +/// (DateTime/Date/numeric/Bool), FTS for free-text Strings, or a Vector index. +/// Edges get BTree only (id, src, dst). This helper and the builder share +/// `node_prop_index_kind` so they cannot drift — see its doc comment. async fn needs_index_work_node( db: &Omnigraph, type_name: &str, @@ -359,14 +402,23 @@ async fn needs_index_work_node( let Some(prop_type) = node_type.properties.get(prop_name) else { continue; }; - if matches!(prop_type.scalar, ScalarType::String) && !prop_type.list { - if !db.storage().has_fts_index(&ds, prop_name).await? { - return Ok(true); + match node_prop_index_kind(prop_type) { + Some(NodePropIndexKind::Fts) => { + if !db.storage().has_fts_index(&ds, prop_name).await? { + return Ok(true); + } } - } else if matches!(prop_type.scalar, ScalarType::Vector(_)) && !prop_type.list { - if !db.storage().has_vector_index(&ds, prop_name).await? { - return Ok(true); + Some(NodePropIndexKind::Vector) => { + if !db.storage().has_vector_index(&ds, prop_name).await? { + return Ok(true); + } } + Some(NodePropIndexKind::Btree) => { + if !db.storage().has_btree_index(&ds, prop_name).await? { + return Ok(true); + } + } + None => {} } } Ok(false) @@ -615,30 +667,44 @@ pub(super) async fn build_indices_on_dataset_for_catalog( } let prop_name = &index_cols[0]; if let Some(prop_type) = node_type.properties.get(prop_name) { - if matches!(prop_type.scalar, ScalarType::String) && !prop_type.list { - if !db.storage().has_fts_index(ds, prop_name).await? { - stage_and_commit_inverted(db, table_key, ds, prop_name.as_str()) - .await?; + match node_prop_index_kind(prop_type) { + Some(NodePropIndexKind::Fts) => { + if !db.storage().has_fts_index(ds, prop_name).await? { + stage_and_commit_inverted(db, table_key, ds, prop_name.as_str()) + .await?; + } } - } else if matches!(prop_type.scalar, ScalarType::Vector(_)) && !prop_type.list { - if !db.storage().has_vector_index(ds, prop_name).await? { - // Inline-commit residual: lance-6.0.1 does not - // expose `build_index_metadata_from_segments` as - // `pub`, so vector indices cannot be staged from - // outside the lance crate. Document at the call - // site; companion ticket to lance-format/lance#6658. - let new_snap = db - .storage_inline_residual() - .create_vector_index(ds.clone(), prop_name.as_str()) - .await - .map_err(|e| { - OmniError::Lance(format!( - "create Vector index on {}({}): {}", - table_key, prop_name, e - )) - })?; - *ds = new_snap; + Some(NodePropIndexKind::Vector) => { + if !db.storage().has_vector_index(ds, prop_name).await? { + // Inline-commit residual: lance-6.0.1 does not + // expose `build_index_metadata_from_segments` as + // `pub`, so vector indices cannot be staged from + // outside the lance crate. Document at the call + // site; companion ticket to lance-format/lance#6658. + let new_snap = db + .storage_inline_residual() + .create_vector_index(ds.clone(), prop_name.as_str()) + .await + .map_err(|e| { + OmniError::Lance(format!( + "create Vector index on {}({}): {}", + table_key, prop_name, e + )) + })?; + *ds = new_snap; + } } + // Enum + orderable scalars (DateTime/Date/numeric/Bool) + // get a BTREE so `=`, range, IN, and IS NULL are index- + // accelerated instead of degrading to a full scan. + Some(NodePropIndexKind::Btree) => { + if !db.storage().has_btree_index(ds, prop_name).await? { + stage_and_commit_btree(db, table_key, ds, &[prop_name.as_str()]) + .await?; + } + } + // List or Blob column: not indexable as a scalar here. + None => {} } } } diff --git a/crates/omnigraph/src/exec/projection.rs b/crates/omnigraph/src/exec/projection.rs index 7280ec5..bb6e665 100644 --- a/crates/omnigraph/src/exec/projection.rs +++ b/crates/omnigraph/src/exec/projection.rs @@ -72,7 +72,11 @@ fn evaluate_expr(batch: &RecordBatch, expr: &IRExpr, params: &ParamMap) -> Resul } /// Create a constant array from a literal value. -fn literal_to_array(lit: &Literal, num_rows: usize) -> Result<ArrayRef> { +/// +/// `pub(super)` so the pushdown arm (`query.rs::literal_to_typed_expr`) can build +/// a literal in the same natural Arrow type and cast it to the column type through +/// the identical `arrow_cast` path used here, keeping the two filter arms in sync. +pub(super) fn literal_to_array(lit: &Literal, num_rows: usize) -> Result<ArrayRef> { Ok(match lit { Literal::Null => arrow_array::new_null_array(&DataType::Utf8, num_rows), Literal::String(s) => Arc::new(StringArray::from(vec![s.as_str(); num_rows])) as ArrayRef, diff --git a/crates/omnigraph/src/exec/query.rs b/crates/omnigraph/src/exec/query.rs index ae2a824..4c1822f 100644 --- a/crates/omnigraph/src/exec/query.rs +++ b/crates/omnigraph/src/exec/query.rs @@ -1289,10 +1289,12 @@ async fn expand_hydrate_and_align( params: &ParamMap, ) -> Result<()> { // Pushable destination filters are applied by `hydrate_nodes`; the rest - // (`ir_filter_to_expr` → None) are applied in memory after hconcat. + // (`ir_filter_to_expr` → None) are applied in memory after hconcat. The + // schema arg only affects a pushable literal's TYPE, never Some-vs-None, so + // `None` here yields the same pushable/non-pushable split as `hydrate_nodes`. let non_pushable: Vec<&IRFilter> = dst_filters .iter() - .filter(|f| ir_filter_to_expr(f, params).is_none()) + .filter(|f| ir_filter_to_expr(f, params, None).is_none()) .collect(); // Unique destination ids (first-seen order) for one batched hydration. @@ -1506,7 +1508,8 @@ async fn hydrate_nodes( // `id IN (ids)` AND any pushable destination filters, as a structured Expr. let id_list: Vec<datafusion::prelude::Expr> = ids.iter().map(|id| lit(id.clone())).collect(); let mut filter_expr = col("id").in_list(id_list, false); - if let Some(dst_expr) = build_lance_filter_expr(dst_filters, params) { + if let Some(dst_expr) = build_lance_filter_expr(dst_filters, params, Some(&node_type.arrow_schema)) + { filter_expr = filter_expr.and(dst_expr); } @@ -1747,21 +1750,23 @@ async fn execute_node_scan( let table_key = format!("node:{}", type_name); let ds = snapshot.open(&table_key).await?; + let node_type = &catalog.node_types[type_name]; + // Lower the IR filters to a DataFusion `Expr` and apply via // `Scanner::filter_expr` inside the configure closure. The string // pushdown path (`build_lance_filter` → `scanner.filter(&str)`) is // gone for node scans — structured Expr unlocks `CompOp::Contains` // pushdown (via `array_has`) and lets DF 53's optimizer rules // (vectorized IN-list, PhysicalExprSimplifier, CASE-NULL shortcut) - // reach our predicates. Other call sites that still take string SQL - // (hydrate_nodes for the Expand pushdown, count_rows, the mutation - // delete path) migrate in follow-up MRs. - let filter_expr = build_lance_filter_expr(filters, params); + // reach our predicates. Passing the node's `arrow_schema` lets the lowering + // coerce literals to each column's exact type so narrow-numeric BTREEs are + // used. Other call sites that still take string SQL (count_rows, the + // mutation delete path) migrate in follow-up MRs. + let filter_expr = build_lance_filter_expr(filters, params, Some(&node_type.arrow_schema)); // Blob columns must be excluded from scan when a filter is present // (Lance bug: BlobsDescriptions + filter triggers a projection assertion). // We exclude blob columns and add metadata post-scan via take_blobs_by_indices. - let node_type = &catalog.node_types[type_name]; let has_blobs = !node_type.blob_properties.is_empty(); let non_blob_cols: Vec<&str> = node_type .arrow_schema @@ -1990,13 +1995,14 @@ pub(super) fn literal_to_sql(lit: &Literal) -> String { pub(super) fn build_lance_filter_expr( filters: &[IRFilter], params: &ParamMap, + schema: Option<&Schema>, ) -> Option<datafusion::prelude::Expr> { use datafusion::logical_expr::Operator; use datafusion::prelude::Expr; let mut acc: Option<Expr> = None; for f in filters { - let Some(e) = ir_filter_to_expr(f, params) else { + let Some(e) = ir_filter_to_expr(f, params, schema) else { continue; }; acc = Some(match acc { @@ -2017,6 +2023,7 @@ pub(super) fn build_lance_filter_expr( pub(super) fn ir_filter_to_expr( filter: &IRFilter, params: &ParamMap, + schema: Option<&Schema>, ) -> Option<datafusion::prelude::Expr> { use datafusion::functions_nested::expr_fn::array_has; @@ -2027,14 +2034,22 @@ pub(super) fn ir_filter_to_expr( // List-contains: `prop CONTAINS value` lowers to `array_has(prop, value)`. // This is the case the old SQL-string pushdown had to return None for // ("Can't pushdown list contains"); with structured Expr it pushes down fine. + // (Element-type coercion for the contained value is deferred — list columns + // are not scalar-indexed, so the index-eligibility concern below does not apply.) if matches!(filter.op, CompOp::Contains) { - let left = ir_expr_to_expr(&filter.left, params)?; - let right = ir_expr_to_expr(&filter.right, params)?; + let left = ir_expr_to_expr(&filter.left, params, None)?; + let right = ir_expr_to_expr(&filter.right, params, None)?; return Some(array_has(left, right)); } - let left = ir_expr_to_expr(&filter.left, params)?; - let right = ir_expr_to_expr(&filter.right, params)?; + // A literal/param operand is coerced to the OTHER operand's column type so + // the predicate stays a direct `col OP literal` and the scalar index is used. + // Without this, DataFusion widens a narrow column (`CAST(col AS Int64)`), + // which defeats the BTREE (validated by `probe_scalar_index_use_under_literal_type`). + let left_col_type = prop_data_type(&filter.left, schema); + let right_col_type = prop_data_type(&filter.right, schema); + let left = ir_expr_to_expr(&filter.left, params, right_col_type.as_ref())?; + let right = ir_expr_to_expr(&filter.right, params, left_col_type.as_ref())?; Some(match filter.op { CompOp::Eq => left.eq(right), CompOp::Ne => left.not_eq(right), @@ -2052,19 +2067,91 @@ pub(super) fn ir_filter_to_expr( pub(super) fn ir_expr_to_expr( expr: &IRExpr, params: &ParamMap, + target: Option<&arrow_schema::DataType>, ) -> Option<datafusion::prelude::Expr> { - use datafusion::prelude::{col, lit}; + use datafusion::prelude::col; match expr { IRExpr::PropAccess { property, .. } => Some(col(property)), - IRExpr::Literal(l) => literal_to_expr(l), - IRExpr::Param(name) => params.get(name).and_then(literal_to_expr), + IRExpr::Literal(l) => literal_to_expr_coerced(l, target), + IRExpr::Param(name) => params + .get(name) + .and_then(|l| literal_to_expr_coerced(l, target)), _ => None, } } -/// Convert a Literal to a DataFusion `Expr`. Returns `None` for List -/// (which the existing SQL path also can't pushdown — falls through to -/// post-scan in-memory application). +/// The Arrow type of a `PropAccess` operand, looked up in the scan's schema, or +/// `None` if the expr is not a column or the schema/field is unavailable. +fn prop_data_type(expr: &IRExpr, schema: Option<&Schema>) -> Option<arrow_schema::DataType> { + match expr { + IRExpr::PropAccess { property, .. } => schema? + .field_with_name(property) + .ok() + .map(|f| f.data_type().clone()), + _ => None, + } +} + +/// Lower a literal for pushdown, coercing it to `target` (the comparison +/// column's Arrow type) when known. Falls back to the natural-type +/// `literal_to_expr` on a missing target or any coercion failure, so a filter is +/// never demoted to `None` by coercion (a node scan has no in-memory fallback for +/// inline filters — see `execute_node_scan`). +fn literal_to_expr_coerced( + lit: &Literal, + target: Option<&arrow_schema::DataType>, +) -> Option<datafusion::prelude::Expr> { + if let Some(target) = target { + if let Some(e) = literal_to_typed_expr(lit, target) { + return Some(e); + } + } + literal_to_expr(lit) +} + +/// Build a literal as a typed Arrow scalar matching `target`, reusing the same +/// `literal_to_array` + `arrow_cast` path as the in-memory arm +/// (`projection.rs::evaluate_filter`) so the two arms agree. Returns `None` on +/// any failure (unbuildable literal, incompatible cast) — the caller then falls +/// back to the natural-type literal. +/// +/// Lossless-only for integer targets: typecheck permits numeric cross-type +/// comparisons (`types_compatible`), so a fractional float or out-of-range +/// integer can reach here. Casting those to a narrower integer would truncate +/// (`2.7 -> 2`) or overflow to null, silently changing which rows match. We +/// round-trip the cast and, on mismatch, return `None` so the caller keeps the +/// natural literal — correct via DataFusion coercion, the index just goes unused +/// for that out-of-domain predicate. Float targets are exempt: narrowing +/// `F64 -> F32` is the column's own precision domain, not a value error. +fn literal_to_typed_expr( + lit: &Literal, + target: &arrow_schema::DataType, +) -> Option<datafusion::prelude::Expr> { + use datafusion::prelude::lit as df_lit; + use datafusion::scalar::ScalarValue; + + let arr = super::projection::literal_to_array(lit, 1).ok()?; + if arr.data_type() == target { + return Some(df_lit(ScalarValue::try_from_array(&arr, 0).ok()?)); + } + let casted = arrow_cast::cast::cast(&arr, target).ok()?; + if target.is_integer() { + let back = arrow_cast::cast::cast(&casted, arr.data_type()).ok()?; + let original = ScalarValue::try_from_array(&arr, 0).ok()?; + let round_tripped = ScalarValue::try_from_array(&back, 0).ok()?; + if original != round_tripped { + return None; + } + } + Some(df_lit(ScalarValue::try_from_array(&casted, 0).ok()?)) +} + +/// Convert a Literal to a DataFusion `Expr` in its NATURAL Arrow type. This is +/// the fallback used when the comparison column's type is unknown (no schema) or +/// when coercion to it fails; the typed, column-matched coercion that keeps +/// scalar indexes usable lives in `literal_to_typed_expr`. Returns `None` for +/// List (the SQL path also could not pushdown it — falls through to post-scan +/// in-memory application). fn literal_to_expr(lit: &Literal) -> Option<datafusion::prelude::Expr> { use datafusion::prelude::lit as df_lit; Some(match lit { @@ -2073,9 +2160,12 @@ fn literal_to_expr(lit: &Literal) -> Option<datafusion::prelude::Expr> { Literal::Integer(n) => df_lit(*n), Literal::Float(f) => df_lit(*f), Literal::Bool(b) => df_lit(*b), - // Date/DateTime stored as strings; pass through as string literals - // — Lance/DataFusion handles the comparison against typed columns - // via implicit cast, matching the existing string-SQL behavior. + // Date/DateTime pass through as strings here. Against a typed Date + // column DataFusion casts the LITERAL (`CAST(Utf8 AS Date32)`), which is + // index-safe (proven by `scalar_index_use_requires_matched_literal_type`). + // At real pushdown sites the schema is known, so `literal_to_typed_expr` + // produces a typed Date32/Date64 anyway; this branch is only the + // no-schema fallback. Literal::Date(s) => df_lit(s.clone()), Literal::DateTime(s) => df_lit(s.clone()), Literal::List(_) => return None, @@ -2285,3 +2375,205 @@ mod expand_chooser_tests { assert_eq!(choose_expand_mode(&i), ExpandMode::Csr); } } + +#[cfg(test)] +mod literal_lowering_tests { + use super::*; + use datafusion::prelude::Expr; + use datafusion::scalar::ScalarValue; + + // With the column type known, the generic coercion types a date literal to + // the column's Date32/Date64 (the live pushdown path). Without a target it + // is the natural Utf8 fallback, which is still index-safe for dates because + // DataFusion casts the LITERAL, not the column (proven by + // `lance_surface_guards::scalar_index_use_requires_matched_literal_type`). + #[test] + fn date_literals_coerce_to_typed_arrow_scalars() { + use arrow_schema::DataType; + let dt = literal_to_expr_coerced( + &Literal::DateTime("2024-06-01T12:00:00Z".into()), + Some(&DataType::Date64), + ) + .unwrap(); + assert!( + matches!(dt, Expr::Literal(ScalarValue::Date64(Some(_)), ..)), + "DateTime vs Date64 column must coerce to a typed Date64, got {dt:?}" + ); + let d = literal_to_expr_coerced(&Literal::Date("2024-06-01".into()), Some(&DataType::Date32)) + .unwrap(); + assert!( + matches!(d, Expr::Literal(ScalarValue::Date32(Some(_)), ..)), + "Date vs Date32 column must coerce to a typed Date32, got {d:?}" + ); + let nat = literal_to_expr_coerced(&Literal::Date("2024-06-01".into()), None).unwrap(); + assert!( + matches!(nat, Expr::Literal(ScalarValue::Utf8(Some(_)), ..)), + "no target should keep the natural Utf8 date literal, got {nat:?}" + ); + } + + // A malformed date string makes coercion fail, so it falls back to the + // natural Utf8 literal rather than dropping the predicate to None. + #[test] + fn malformed_date_literal_falls_back_to_string() { + use arrow_schema::DataType; + let bad = literal_to_expr_coerced( + &Literal::DateTime("not-a-date".into()), + Some(&DataType::Date64), + ) + .unwrap(); + assert!( + matches!(bad, Expr::Literal(ScalarValue::Utf8(Some(_)), ..)), + "malformed DateTime literal should fall back to a Utf8 literal, got {bad:?}" + ); + } + + // With a column target, a literal lowers to the column's EXACT Arrow type + // (not its natural width), so DataFusion does not widen and cast the column + // — keeping the scalar BTREE usable. See + // `lance_surface_guards::scalar_index_use_requires_matched_literal_type`. + #[test] + fn integer_literal_coerces_to_narrow_column_type() { + use arrow_schema::DataType; + let i32_lit = literal_to_expr_coerced(&Literal::Integer(5), Some(&DataType::Int32)).unwrap(); + assert!( + matches!(i32_lit, Expr::Literal(ScalarValue::Int32(Some(5)), ..)), + "integer literal vs Int32 column must lower to Int32, got {i32_lit:?}" + ); + let u32_lit = literal_to_expr_coerced(&Literal::Integer(7), Some(&DataType::UInt32)).unwrap(); + assert!( + matches!(u32_lit, Expr::Literal(ScalarValue::UInt32(Some(7)), ..)), + "integer literal vs UInt32 column must lower to UInt32, got {u32_lit:?}" + ); + } + + #[test] + fn float_literal_coerces_to_f32_column_type() { + use arrow_schema::DataType; + let f32_lit = + literal_to_expr_coerced(&Literal::Float(1.5), Some(&DataType::Float32)).unwrap(); + assert!( + matches!(f32_lit, Expr::Literal(ScalarValue::Float32(Some(_)), ..)), + "float literal vs Float32 column must lower to Float32, got {f32_lit:?}" + ); + } + + // Lossless guard: a fractional float against an integer column must NOT + // truncate (2.7 -> 2). Fall back to the natural Float64 so the comparison + // stays exact (no integer equals 2.7). + #[test] + fn fractional_float_vs_int_column_falls_back_not_truncate() { + use arrow_schema::DataType; + let e = literal_to_expr_coerced(&Literal::Float(2.7), Some(&DataType::Int32)).unwrap(); + assert!( + matches!(e, Expr::Literal(ScalarValue::Float64(Some(_)), ..)), + "fractional float vs Int32 must fall back to natural Float64, got {e:?}" + ); + } + + // A whole-number float IS lossless against an integer column, so it coerces. + #[test] + fn whole_float_vs_int_column_coerces() { + use arrow_schema::DataType; + let e = literal_to_expr_coerced(&Literal::Float(2.0), Some(&DataType::Int32)).unwrap(); + assert!( + matches!(e, Expr::Literal(ScalarValue::Int32(Some(2)), ..)), + "whole-number float vs Int32 is lossless and must coerce to Int32(2), got {e:?}" + ); + } + + // Lossless guard: an integer literal outside the column's range must NOT + // overflow to null; fall back to the natural Int64 (correct via DataFusion). + #[test] + fn out_of_range_int_vs_narrow_column_falls_back() { + use arrow_schema::DataType; + let e = literal_to_expr_coerced(&Literal::Integer(3_000_000_000), Some(&DataType::Int32)) + .unwrap(); + assert!( + matches!(e, Expr::Literal(ScalarValue::Int64(Some(3_000_000_000)), ..)), + "out-of-range integer vs Int32 must fall back to natural Int64, got {e:?}" + ); + } + + // Float targets are exempt from the lossless guard: narrowing to the column's + // own precision is the correct comparison domain, even when the value is not + // exactly representable in F32 (0.1). + #[test] + fn float_vs_f32_column_coerces_even_when_not_exactly_representable() { + use arrow_schema::DataType; + let e = literal_to_expr_coerced(&Literal::Float(0.1), Some(&DataType::Float32)).unwrap(); + assert!( + matches!(e, Expr::Literal(ScalarValue::Float32(Some(_)), ..)), + "float target must coerce 0.1 to Float32 (exempt from lossless guard), got {e:?}" + ); + } + + // No target (caller without a schema) keeps the natural width — the existing + // fallback, so behavior never regresses where the column type is unknown. + #[test] + fn literal_without_target_keeps_natural_width() { + let nat = literal_to_expr_coerced(&Literal::Integer(5), None).unwrap(); + assert!( + matches!(nat, Expr::Literal(ScalarValue::Int64(Some(5)), ..)), + "no target should keep the natural Int64 width, got {nat:?}" + ); + } + + // True if either operand of a binary comparison is an Int32 literal. + fn binary_has_int32_literal(e: &Expr) -> bool { + if let Expr::BinaryExpr(b) = e { + [b.left.as_ref(), b.right.as_ref()] + .iter() + .any(|side| matches!(side, Expr::Literal(ScalarValue::Int32(Some(_)), ..))) + } else { + false + } + } + + fn int32_schema() -> arrow_schema::Schema { + use arrow_schema::{DataType, Field}; + arrow_schema::Schema::new(vec![Field::new("count", DataType::Int32, true)]) + } + + fn count_prop() -> IRExpr { + IRExpr::PropAccess { + variable: "m".into(), + property: "count".into(), + } + } + + // Coercion is operator-independent: a range comparison's literal coerces to + // the column type just like equality does, so range filters on a narrow + // numeric column keep the BTREE. + #[test] + fn ir_filter_coerces_literal_for_range_op() { + let schema = int32_schema(); + let filter = IRFilter { + left: count_prop(), + op: CompOp::Ge, + right: IRExpr::Literal(Literal::Integer(2)), + }; + let expr = ir_filter_to_expr(&filter, &ParamMap::new(), Some(&schema)).unwrap(); + assert!( + binary_has_int32_literal(&expr), + "range-op literal must coerce to the Int32 column type, got {expr:?}" + ); + } + + // The column may be on either side; the literal coerces to the opposite + // operand's column type regardless of order (`5 < count`). + #[test] + fn ir_filter_coerces_literal_when_column_is_on_the_right() { + let schema = int32_schema(); + let filter = IRFilter { + left: IRExpr::Literal(Literal::Integer(2)), + op: CompOp::Lt, + right: count_prop(), + }; + let expr = ir_filter_to_expr(&filter, &ParamMap::new(), Some(&schema)).unwrap(); + assert!( + binary_has_int32_literal(&expr), + "reversed-operand literal must coerce to the Int32 column type, got {expr:?}" + ); + } +} diff --git a/crates/omnigraph/src/table_store.rs b/crates/omnigraph/src/table_store.rs index 65123c0..b6e8c4d 100644 --- a/crates/omnigraph/src/table_store.rs +++ b/crates/omnigraph/src/table_store.rs @@ -705,6 +705,36 @@ impl TableStore { Ok(IndexCoverage::Indexed) } + /// True if any non-system index on `ds` leaves at least one current + /// fragment uncovered, i.e. rows that the index does not yet account for + /// (appended after the index was built, or rewritten by compaction). Such + /// fragments are scanned unindexed until a reindex (`optimize_indices`) + /// folds them in. Returns false when every index covers every fragment, or + /// when the table has no (non-system) indices to optimize. A `None` + /// `fragment_bitmap` means Lance cannot report coverage for that index, so + /// we do not treat it as uncovered (mirrors `key_column_index_coverage`). + /// + /// Used by `optimize` to decide whether an otherwise-already-compacted + /// table still has index work to do. + pub async fn has_unindexed_fragments(ds: &Dataset) -> Result<bool> { + let indices = ds + .load_indices() + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + let frag_ids: Vec<u32> = ds.fragments().iter().map(|f| f.id as u32).collect(); + for index in indices.iter() { + if is_system_index(index) { + continue; + } + if let Some(bitmap) = index.fragment_bitmap.as_ref() { + if frag_ids.iter().any(|id| !bitmap.contains(*id)) { + return Ok(true); + } + } + } + Ok(false) + } + pub async fn count_rows(&self, ds: &Dataset, filter: Option<String>) -> Result<usize> { ds.count_rows(filter) .await diff --git a/crates/omnigraph/tests/lance_surface_guards.rs b/crates/omnigraph/tests/lance_surface_guards.rs index 370f9e7..fdb977c 100644 --- a/crates/omnigraph/tests/lance_surface_guards.rs +++ b/crates/omnigraph/tests/lance_surface_guards.rs @@ -36,6 +36,7 @@ use lance::dataset::{MergeInsertBuilder, WhenMatched, WhenNotMatched, WriteMode, use lance::index::DatasetIndexExt; use lance_file::version::LanceFileVersion; use lance_index::IndexType; +use lance_index::optimize::OptimizeOptions; use lance_index::scalar::ScalarIndexParams; use lance_namespace::LanceNamespace; use lance_table::io::commit::ManifestNamingScheme; @@ -541,3 +542,199 @@ async fn fragment_deletion_metadata_is_available() { per-fragment deletions and would need to read the deletion vector.", ); } + +// --- Guard 14: Dataset::optimize_indices signature ---------------------------- +// +// `db/omnigraph/optimize.rs::optimize_one_table` calls +// `ds.optimize_indices(&OptimizeOptions::default())` (via `DatasetIndexExt`) to +// fold appended/compacted fragments back into existing indexes. If Lance +// changes the receiver, the options type, or the return shape, this fails to +// compile. Compile-only. + +#[allow( + dead_code, + unreachable_code, + unused_variables, + unused_mut, + clippy::diverging_sub_expression +)] +async fn _compile_optimize_indices_signature() -> lance::Result<()> { + let mut ds: Dataset = unimplemented!(); + let options = OptimizeOptions::default(); + // `&mut self`, `&OptimizeOptions`, returns `Result<()>` (mutates in place + // and commits — there is no uncommitted variant in this Lance, which is why + // optimize treats it as an inline-commit residual under a recovery sidecar). + let _: () = ds.optimize_indices(&options).await?; + Ok(()) +} + +// --- Guard 15: optimize_indices extends fragment coverage ---------------------- +// +// PR3's reindex assumes `optimize_indices` folds fragments appended AFTER an +// index was built into that index (incremental merge, not retrain). This pins +// that Lance behavior at the surface layer so a regression turns red here, the +// first smoke check on a Lance bump, before the slower engine suite. + +#[tokio::test] +async fn optimize_indices_extends_fragment_coverage() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().join("guard_optimize_indices.lance"); + let uri = uri.to_str().unwrap(); + + // Fragment 0: alice, bob. Build a BTREE over `value` covering only it. + let mut ds = fresh_dataset(uri).await; + ds.create_index_builder(&["value"], IndexType::BTree, &ScalarIndexParams::default()) + .replace(true) + .await + .unwrap(); + + // Append a second fragment the existing index does not cover. + let schema = Arc::new(Schema::new(vec![ + Field::new("id", DataType::Utf8, false), + Field::new("value", DataType::Int32, false), + ])); + let batch = RecordBatch::try_new( + schema.clone(), + vec![ + Arc::new(StringArray::from(vec!["carol"])), + Arc::new(Int32Array::from(vec![3])), + ], + ) + .unwrap(); + let reader = RecordBatchIterator::new(vec![Ok(batch)], schema); + let params = WriteParams { + mode: WriteMode::Append, + enable_stable_row_ids: true, + data_storage_version: Some(LanceFileVersion::V2_2), + ..Default::default() + }; + Dataset::write(reader, uri, Some(params)).await.unwrap(); + + let mut ds = Dataset::open(uri).await.unwrap(); + assert!( + value_index_uncovered_count(&ds).await > 0, + "appended fragment should be uncovered by the BTREE before optimize_indices" + ); + + ds.optimize_indices(&OptimizeOptions::default()) + .await + .unwrap(); + + assert_eq!( + value_index_uncovered_count(&ds).await, + 0, + "optimize_indices must fold the appended fragment into the existing index \ + (incremental coverage); if this regresses, PR3's reindex no longer keeps \ + coverage current — revisit db/omnigraph/optimize.rs and docs/dev/lance.md." + ); +} + +/// Count current fragments not covered by the single-column `value` BTREE — +/// mirrors `TableStore::has_unindexed_fragments` (load_indices + +/// `fragment_bitmap.contains`), pinned by Guard 11. +async fn value_index_uncovered_count(ds: &Dataset) -> usize { + let indices = ds.load_indices().await.unwrap(); + let frag_ids: Vec<u32> = ds.fragments().iter().map(|f| f.id as u32).collect(); + let value_fid = ds.schema().field("value").unwrap().id; + for index in indices.iter() { + if index.fields.len() == 1 && index.fields[0] == value_fid { + if let Some(bitmap) = index.fragment_bitmap.as_ref() { + return frag_ids.iter().filter(|id| !bitmap.contains(**id)).count(); + } + } + } + // No `value` index found — treat as fully uncovered so a missing index + // is never mistaken for full coverage. + frag_ids.len() +} + +// --- Guard 16: scalar index use requires a literal matching the column type --- +// +// Pins the substrate behavior the pushdown literal-coercion fix relies on +// (`query.rs::literal_to_typed_expr`): Lance uses the BTREE only when the filter +// is `column OP literal` with a matching type. A width-mismatched literal makes +// DataFusion widen and cast the COLUMN (`CAST(n32 AS Int64)`), which drops the +// scalar index and full-scans. Temporal columns are immune (DataFusion casts the +// Utf8 LITERAL to the date type, not the column). If a Lance/DataFusion bump +// changes either coercion direction, this turns red — re-validate the fix. +#[tokio::test] +async fn scalar_index_use_requires_matched_literal_type() { + use datafusion::physical_plan::displayable; + use datafusion::prelude::{col, lit}; + use datafusion::scalar::ScalarValue; + + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().join("probe_literal_type.lance"); + let uri = uri.to_str().unwrap(); + + let schema = Arc::new(Schema::new(vec![ + Field::new("id", DataType::Utf8, false), + Field::new("n32", DataType::Int32, false), + Field::new("d32", DataType::Date32, false), + ])); + let batch = RecordBatch::try_new( + schema.clone(), + vec![ + Arc::new(StringArray::from(vec!["a", "b", "c", "d"])), + Arc::new(Int32Array::from(vec![1, 5, 9, 13])), + Arc::new(arrow_array::Date32Array::from(vec![19000, 19723, 20000, 20500])), + ], + ) + .unwrap(); + let reader = RecordBatchIterator::new(vec![Ok(batch)], schema); + let params = WriteParams { + mode: WriteMode::Create, + enable_stable_row_ids: true, + data_storage_version: Some(LanceFileVersion::V2_2), + ..Default::default() + }; + let mut ds = Dataset::write(reader, uri, Some(params)).await.unwrap(); + for c in ["n32", "d32"] { + ds.create_index_builder(&[c], IndexType::BTree, &ScalarIndexParams::default()) + .replace(true) + .await + .unwrap(); + } + + async fn plan_str(ds: &Dataset, filter: datafusion::prelude::Expr) -> String { + let mut scanner = ds.scan(); + scanner.filter_expr(filter); + let plan = scanner.create_plan().await.unwrap(); + format!("{}", displayable(plan.as_ref()).indent(true)) + } + + // (label, filter, expect_index_used) + let cases = [ + ("n32 = 5i32 (matched Int32)", col("n32").eq(lit(5i32)), true), + ("n32 = 5i64 (widened Int64)", col("n32").eq(lit(5i64)), false), + ( + "d32 = Date32 (matched)", + col("d32").eq(lit(ScalarValue::Date32(Some(19723)))), + true, + ), + ( + "d32 = '2024-01-01' (Utf8 vs Date32)", + col("d32").eq(lit("2024-01-01")), + true, + ), + ]; + + for (label, filter, expect_index) in cases { + let s = plan_str(&ds, filter).await; + let uses_index = s.contains("ScalarIndexQuery"); + assert_eq!( + uses_index, expect_index, + "[{label}] expected scalar-index use = {expect_index}, got {uses_index}.\n\ + A change here means Lance/DataFusion shifted its coercion or index \ + pushdown; re-validate query.rs::literal_to_typed_expr.\nplan:\n{s}" + ); + } + + // The widened case must show the index-defeating column CAST (the precise + // shape the fix avoids by coercing the literal to the column type). + let widened = plan_str(&ds, col("n32").eq(lit(5i64))).await; + assert!( + widened.contains("CAST(n32 AS Int64)"), + "expected a column-side cast in the widened plan, got:\n{widened}" + ); +} diff --git a/crates/omnigraph/tests/literal_filters.rs b/crates/omnigraph/tests/literal_filters.rs index a0b2bd7..d486f28 100644 --- a/crates/omnigraph/tests/literal_filters.rs +++ b/crates/omnigraph/tests/literal_filters.rs @@ -19,6 +19,7 @@ node Metric { name: String @key score: F64? ratio: F32? + count: I32? active: Bool? born: Date? seen: DateTime? @@ -26,10 +27,10 @@ node Metric { "#; // Seeds partition every predicate, so a dropped filter returns all 4 rows. -const DATA: &str = r#"{"type":"Metric","data":{"name":"m1","score":2.5,"ratio":0.5,"active":true,"born":"2024-06-01","seen":"2024-06-01T12:00:00Z"}} -{"type":"Metric","data":{"name":"m2","score":1.0,"ratio":0.25,"active":false,"born":"2023-01-01","seen":"2023-01-01T00:00:00Z"}} -{"type":"Metric","data":{"name":"m3","score":3.0,"ratio":0.75,"active":true,"born":"2025-01-01","seen":"2025-01-01T00:00:00Z"}} -{"type":"Metric","data":{"name":"m4","score":0.5,"ratio":0.1,"active":false,"born":"2022-12-31","seen":"2022-01-01T00:00:00Z"}}"#; +const DATA: &str = r#"{"type":"Metric","data":{"name":"m1","score":2.5,"ratio":0.5,"count":1,"active":true,"born":"2024-06-01","seen":"2024-06-01T12:00:00Z"}} +{"type":"Metric","data":{"name":"m2","score":1.0,"ratio":0.25,"count":2,"active":false,"born":"2023-01-01","seen":"2023-01-01T00:00:00Z"}} +{"type":"Metric","data":{"name":"m3","score":3.0,"ratio":0.75,"count":3,"active":true,"born":"2025-01-01","seen":"2025-01-01T00:00:00Z"}} +{"type":"Metric","data":{"name":"m4","score":0.5,"ratio":0.1,"count":4,"active":false,"born":"2022-12-31","seen":"2022-01-01T00:00:00Z"}}"#; async fn metric_db(dir: &tempfile::TempDir) -> Omnigraph { let uri = dir.path().to_str().unwrap(); @@ -67,6 +68,50 @@ query inline() { match { $m: Metric { score: 3.0 } } return { $m.name } } assert_eq!(sorted_metric_names(&mut db, q, "inline").await, vec!["m3"]); } +// Inline-binding equality is the Lance-pushdown arm. With the literal coerced to +// the column's exact Arrow type, a narrow-numeric column (I32) and an F32 column +// must still select the right rows — the coercion changes the literal's type, not +// the result set. (The index-use win this enables is pinned at the Lance-surface +// layer by `lance_surface_guards::scalar_index_use_requires_matched_literal_type`.) +#[tokio::test] +async fn int_and_f32_literal_pushdown_coercion() { + let dir = tempfile::tempdir().unwrap(); + let mut db = metric_db(&dir).await; + let q = r#" +query count_eq() { match { $m: Metric { count: 2 } } return { $m.name } } +query ratio_eq() { match { $m: Metric { ratio: 0.25 } } return { $m.name } } +query count_ge() { match { $m: Metric $m.count >= 3 } return { $m.name } } +"#; + // I32 column, integer literal coerced Int64 -> Int32: count == 2 is m2 only. + assert_eq!(sorted_metric_names(&mut db, q, "count_eq").await, vec!["m2"]); + // F32 column, float literal coerced Float64 -> Float32: ratio == 0.25 is m2. + assert_eq!(sorted_metric_names(&mut db, q, "ratio_eq").await, vec!["m2"]); + // Range on the I32 column: count 3,4 >= 3 -> m3, m4 (coercion is op-independent). + assert_eq!( + sorted_metric_names(&mut db, q, "count_ge").await, + vec!["m3", "m4"] + ); +} + +// A fractional float against an integer column must not be truncated by the +// pushdown coercion (`2.7 -> 2` would wrongly match the count=2 row). The +// lossless guard falls back to the natural Float64 literal, so `count = 2.7` +// matches no integer and returns no rows. +#[tokio::test] +async fn fractional_float_equality_on_int_column_returns_no_rows() { + let dir = tempfile::tempdir().unwrap(); + let mut db = metric_db(&dir).await; + let q = r#" +query count_frac() { match { $m: Metric { count: 2.7 } } return { $m.name } } +"#; + assert!( + sorted_metric_names(&mut db, q, "count_frac") + .await + .is_empty(), + "count = 2.7 must match no integer rows (no truncation to count = 2)" + ); +} + #[tokio::test] async fn bool_literal_filters_execute() { let dir = tempfile::tempdir().unwrap(); @@ -88,9 +133,15 @@ async fn date_and_datetime_literal_filters_execute() { let q = r#" query born_ge() { match { $m: Metric $m.born >= date("2024-01-01") } return { $m.name } } query seen_lt() { match { $m: Metric $m.seen < datetime("2024-01-01T00:00:00Z") } return { $m.name } } +query born_eq() { match { $m: Metric { born: date("2024-06-01") } } return { $m.name } } +query seen_eq() { match { $m: Metric { seen: datetime("2024-06-01T12:00:00Z") } } return { $m.name } } "#; // born: m1 2024-06, m3 2025 >= 2024-01-01 assert_eq!(sorted_metric_names(&mut db, q, "born_ge").await, vec!["m1", "m3"]); // seen: m2 2023, m4 2022 < 2024-01-01 assert_eq!(sorted_metric_names(&mut db, q, "seen_lt").await, vec!["m2", "m4"]); + // Inline-binding equality exercises the Lance-pushdown arm with a typed + // Date32/Date64 literal: the epoch conversion must select exactly m1. + assert_eq!(sorted_metric_names(&mut db, q, "born_eq").await, vec!["m1"]); + assert_eq!(sorted_metric_names(&mut db, q, "seen_eq").await, vec!["m1"]); } diff --git a/crates/omnigraph/tests/maintenance.rs b/crates/omnigraph/tests/maintenance.rs index 13c9de7..deb4d2d 100644 --- a/crates/omnigraph/tests/maintenance.rs +++ b/crates/omnigraph/tests/maintenance.rs @@ -14,9 +14,11 @@ use omnigraph::db::{ SkipReason, }; use omnigraph::loader::{LoadMode, load_jsonl}; +use omnigraph::table_store::{IndexCoverage, TableStore}; use helpers::{ MUTATION_QUERIES, TEST_DATA, TEST_SCHEMA, count_rows, init_and_load, mixed_params, mutate_main, + snapshot_main, }; /// Filesystem URI of a node sub-table, mirroring the engine's layout @@ -131,6 +133,72 @@ async fn optimize_after_load_then_again_is_idempotent() { } } +// PR3 (Workstream B): an existing scalar index does not cover fragments +// appended after it was built (build_indices is existence-gated), so those +// rows are scanned unindexed. `optimize` must fold them back in via Lance's +// incremental `optimize_indices`, restoring full coverage. +#[tokio::test] +async fn optimize_reindexes_fragments_appended_after_index_build() { + const SCHEMA: &str = r#" +node Doc { + slug: String @key + rank: I32 @index +} +"#; + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, SCHEMA).await.unwrap(); + + // First load builds the id + rank BTREEs over the initial fragment. + load_jsonl( + &mut db, + "{\"type\":\"Doc\",\"data\":{\"slug\":\"d1\",\"rank\":1}}\n\ + {\"type\":\"Doc\",\"data\":{\"slug\":\"d2\",\"rank\":2}}", + LoadMode::Merge, + ) + .await + .unwrap(); + + // A second load with NEW keys appends a fragment the existing BTREEs do not + // cover (the existence gate skips re-building an index that already exists). + load_jsonl( + &mut db, + "{\"type\":\"Doc\",\"data\":{\"slug\":\"d3\",\"rank\":3}}\n\ + {\"type\":\"Doc\",\"data\":{\"slug\":\"d4\",\"rank\":4}}", + LoadMode::Merge, + ) + .await + .unwrap(); + + // Precondition: the appended fragment is unindexed. + { + let snap = snapshot_main(&db).await.unwrap(); + let ds = snap.open("node:Doc").await.unwrap(); + assert!( + TableStore::has_unindexed_fragments(&ds).await.unwrap(), + "appended fragment should be unindexed before optimize" + ); + } + + db.optimize().await.unwrap(); + + // Postcondition: optimize_indices folded the appended fragment in, so every + // index covers every fragment and `rank` reports fully Indexed. + let snap = snapshot_main(&db).await.unwrap(); + let ds = snap.open("node:Doc").await.unwrap(); + assert!( + !TableStore::has_unindexed_fragments(&ds).await.unwrap(), + "optimize must extend index coverage to all fragments" + ); + assert_eq!( + TableStore::key_column_index_coverage(&ds, "rank") + .await + .unwrap(), + IndexCoverage::Indexed, + "rank BTREE must cover all fragments after optimize" + ); +} + // Regression: `optimize` must not crash on a graph that has a `Blob` table. // // Lance `compact_files` forces `BlobHandling::AllBinary`, which mis-decodes diff --git a/crates/omnigraph/tests/scalar_indexes.rs b/crates/omnigraph/tests/scalar_indexes.rs new file mode 100644 index 0000000..8d8a3f0 --- /dev/null +++ b/crates/omnigraph/tests/scalar_indexes.rs @@ -0,0 +1,74 @@ +//! Coverage for `build_indices_on_dataset_for_catalog`'s per-property index +//! dispatch: which scalar/vector index each `@index`/`@key` column gets. +//! +//! The observable signal is `TableStore::key_column_index_coverage`, which +//! reports `Indexed` only when a BTREE covers the column (the same helper the +//! traversal chooser uses). Enums and orderable scalars must get a BTREE so +//! `=`/range/IN/IS NULL are index-accelerated; free-text Strings keep FTS +//! (which `key_column_index_coverage` does not count as a BTREE, by design). + +mod helpers; + +use omnigraph::db::Omnigraph; +use omnigraph::loader::{LoadMode, load_jsonl}; +use omnigraph::table_store::{IndexCoverage, TableStore}; + +use helpers::*; + +const SCHEMA: &str = r#" +node Item { + slug: String @key + status: enum(active, archived) @index + published: DateTime @index + rank: I32 @index + title: String @index + note: String? +} +"#; + +const DATA: &str = r#"{"type":"Item","data":{"slug":"a","status":"active","published":"2024-06-01T00:00:00Z","rank":1,"title":"alpha","note":"n1"}} +{"type":"Item","data":{"slug":"b","status":"archived","published":"2023-01-01T00:00:00Z","rank":2,"title":"beta","note":"n2"}} +{"type":"Item","data":{"slug":"c","status":"active","published":"2025-02-02T00:00:00Z","rank":3,"title":"gamma","note":"n3"}}"#; + +// Enums and orderable scalars (DateTime, numeric) get a BTREE from load's +// build-indices pass, so a `=`/range filter on them uses the index. Free-text +// String `@index` keeps FTS (no BTREE), and an un-annotated column has no +// scalar index — both report `Degraded`, which is the negative control that +// keeps this test from being vacuously green. +#[tokio::test] +async fn node_scalar_and_enum_index_columns_get_btree() { + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().to_str().unwrap(); + let mut db = Omnigraph::init(uri, SCHEMA).await.unwrap(); + load_jsonl(&mut db, DATA, LoadMode::Overwrite).await.unwrap(); + + let snap = snapshot_main(&db).await.unwrap(); + let ds = snap.open("node:Item").await.unwrap(); + + for col in ["status", "published", "rank"] { + let cov = TableStore::key_column_index_coverage(&ds, col).await.unwrap(); + assert_eq!( + cov, + IndexCoverage::Indexed, + "column '{col}' (enum/DateTime/numeric @index) must get a BTREE, got {cov:?}" + ); + } + + // Free-text String @index -> FTS, which is not a BTREE -> Degraded. + let title_cov = TableStore::key_column_index_coverage(&ds, "title") + .await + .unwrap(); + assert!( + matches!(title_cov, IndexCoverage::Degraded { .. }), + "free-text String @index should keep FTS (no BTREE), got {title_cov:?}" + ); + + // No @index annotation -> no scalar index at all -> Degraded. + let note_cov = TableStore::key_column_index_coverage(&ds, "note") + .await + .unwrap(); + assert!( + matches!(note_cov, IndexCoverage::Degraded { .. }), + "un-annotated column should have no scalar index, got {note_cov:?}" + ); +} diff --git a/docs/dev/invariants.md b/docs/dev/invariants.md index a0bcc6d..c840309 100644 --- a/docs/dev/invariants.md +++ b/docs/dev/invariants.md @@ -105,7 +105,7 @@ Use it this way: | Schema validation | Type checks, required fields, defaults, edge endpoint checks, and edge cardinality are enforced on write paths | [schema-language.md](../user/schema/index.md), [execution.md](execution.md) | | Unique constraints | Intra-batch and write-path checks exist; intake and branch-merge derive the composite key through one shared function (`loader::composite_unique_key`, a separator-free `Vec<String>` tuple) and fail loudly on an un-keyable column type rather than silently exempting it; full cross-version uniqueness against already-committed rows is still a gap | [schema-language.md](../user/schema/index.md) | | Storage trait | `TableStorage` (via `db.storage()`) is staged-only; the inline-commit residuals (`delete_where`, `create_vector_index`) are split onto a separate sealed `InlineCommitResidual` trait reached via `db.storage_inline_residual()` (MR-854), so §1 holds by construction; capability/stat surfaces are roadmap | [writes.md](writes.md), [architecture.md](architecture.md) | -| Index lifecycle | `ensure_indices` is explicit today; reconciler-based convergence is roadmap | [indexes.md](../user/search/indexes.md), [maintenance.md](../user/operations/maintenance.md) | +| Index lifecycle | Index *creation* per `@index`/`@key` property is dispatched by type (enum + orderable scalar → BTREE, free-text String → FTS, Vector → vector) via `node_prop_index_kind`; index *coverage maintenance* exists — `optimize` runs Lance `optimize_indices` after compaction to fold appended/rewritten fragments into existing indexes (still an explicit maintenance call, not yet a background reconciler) | [indexes.md](../user/search/indexes.md), [maintenance.md](../user/operations/maintenance.md) | | Traversal IDs | Runtime still builds `TypeIndex`; Lance stable row-id based graph IDs are roadmap | [architecture.md](architecture.md), [query-language.md](../user/queries/index.md) | | Auth | Bearer token hashing and server-side actor resolution are implemented at the HTTP boundary | [server.md](../user/operations/server.md), [policy.md](../user/operations/policy.md) | | Tests | Tempdir-backed Lance tests are the current substrate; the storage adapter has an in-memory backend for adapter-level contract tests, but Lance datasets bypass it | [testing.md](testing.md) | diff --git a/docs/dev/lance.md b/docs/dev/lance.md index a4e311f..9544e80 100644 --- a/docs/dev/lance.md +++ b/docs/dev/lance.md @@ -169,6 +169,7 @@ Migration from Lance 4.0.0 → 6.0.1 landed in this cycle (DataFusion 52 → 53, - **`Dataset::checkout_version(N).await?.restore().await?`**: `restore()` takes `&mut self` and returns `Result<()>` (mutates in place, does not consume + return a new dataset). The recovery rollback hammer at `db/manifest/recovery.rs:505-522` continues to work. Pinned by `lance_surface_guards.rs::_compile_checkout_version_then_restore_signature`. - **`DatasetBuilder::from_namespace(...).with_branch(...).with_version(...).load()`** surface preserved (the namespace builder chain at `db/manifest/namespace.rs:162-174`). Pinned by `lance_surface_guards.rs::_compile_dataset_builder_from_namespace_signature`. - **`compact_files(&mut ds, CompactionOptions::default(), None)`** signature stable. `CompactionOptions` still does not expose `data_storage_version`; `compact_files` builds its own `WriteParams { ..Default::default() }`. Note: `LanceFileVersion::default()` is now V2_1 in v6, so optimize-rewritten fragments come out at V2_1 by default (was V2_0 in v4). Existing explicit V2_2 pins on creates/appends still apply. +- **`Dataset::optimize_indices(&mut self, &lance_index::optimize::OptimizeOptions)`** (via `DatasetIndexExt`) is a depended-on surface as of the index-coverage work: `db/omnigraph/optimize.rs` calls it after `compact_files` to fold appended/rewritten fragments into existing indexes (incremental merge, not retrain). It is a **committing** call (mutates in place, advances HEAD; no uncommitted variant in v6.0.1), so optimize treats it as an inline-commit residual under the `SidecarKind::Optimize` recovery sidecar. Signature pinned by `lance_surface_guards.rs::_compile_optimize_indices_signature`; the incremental-coverage behavior pinned by `optimize_indices_extends_fragment_coverage` (appended fragment uncovered before, covered after). - **`Dataset::delete(predicate)` returns `DeleteResult { new_dataset: Arc<Dataset>, num_deleted_rows: u64 }`** — unchanged shape. Pinned by `lance_surface_guards.rs::_compile_delete_result_field_shape`. MR-A will repurpose this guard to the staged two-phase variant once `DeleteBuilder::execute_uncommitted` migration lands. - **File reader read methods now async** (Lance PR #6710, v6.0). No effect — omnigraph reaches Lance exclusively through `Dataset::scan` and the staged-write API. - **Tokenizer vendored as `lance-tokenizer`** (Lance PR #6512, v6.0). No effect — no direct tokenizer imports. @@ -178,6 +179,6 @@ Migration from Lance 4.0.0 → 6.0.1 landed in this cycle (DataFusion 52 → 53, - **`Dataset::force_delete_branch`** (`branches().delete(name, force=true)`, dataset.rs:524) tolerates a missing branch-*contents* ref (vs plain `delete_branch`'s `RefNotFound`), but on the local store still errors `NotFound` if the branch `tree/` directory is fully absent (`remove_dir_all`'s NotFound is not caught for Lance's native error variant, refs.rs:526-549). Both variants still refuse a branch with referencing descendants (`RefConflict`). `TableStore::force_delete_branch` wraps this to be fully idempotent (tolerates already-absent). The single-authority branch-delete redesign uses it for orphan reclamation (eager best-effort reclaim + cleanup reconciler). Pinned by `lance_surface_guards.rs::force_delete_branch_semantics`. Branch delete is "flip the ref atomically, then `remove_dir_all(tree/{branch})`"; branch-exclusive data lives under `tree/{branch}/` so a drop reclaims it immediately without touching `main`. - **Lance blob-v2 `compact_files` bug** (no public issue found as of 2026-06): `compact_files` disables binary-copy for blob datasets and forces `BlobHandling::AllBinary` on the read side; the v2.1+ structural decoder then mis-counts column infos for the blob-v2 struct and fails with `Invalid user input: there were more fields in the schema than provided column indices / infos` (`lance-encoding/src/decoder.rs::ColumnInfoIter::expect_next`). This fails even a pristine uniform-V2_2 multi-fragment blob table; vector/list/scalar/ragged columns and mixed file versions all compact fine. Reads/queries use descriptor handling (`BlobHandling::default()`) and are unaffected. `optimize` skips blob-bearing tables behind `LANCE_SUPPORTS_BLOB_COMPACTION = false` (`db/omnigraph/optimize.rs`), reporting `SkipReason::BlobColumnsUnsupportedByLance`. Pinned by `lance_surface_guards.rs::compact_files_still_fails_on_blob_columns`, which turns red when the bug is fixed → flip the gate, remove the skip branch + the `maintenance.rs::optimize_skips_blob_table_and_reports_skip` skip assertions. -Surface guards added: `crates/omnigraph/tests/lance_surface_guards.rs` (10 named guards; 5 runtime + 5 compile-only). Future Lance bumps re-run this file first as the smoke check. Two additional guards from the original plan deferred to follow-up (`manifest_cas_returns_row_level_contention_variant` needs full publisher-race harness; `table_version_metadata_byte_compatible_with_v4` needs `pub(crate)` reach extension). +Surface guards added: `crates/omnigraph/tests/lance_surface_guards.rs` (10 named guards; 5 runtime + 5 compile-only; plus the index-coverage work's `_compile_optimize_indices_signature` and `optimize_indices_extends_fragment_coverage`). Future Lance bumps re-run this file first as the smoke check. Two additional guards from the original plan deferred to follow-up (`manifest_cas_returns_row_level_contention_variant` needs full publisher-race harness; `table_version_metadata_byte_compatible_with_v4` needs `pub(crate)` reach extension). Bump this date stanza on the next alignment pass. diff --git a/docs/dev/writes.md b/docs/dev/writes.md index c3511e0..ccfd5bc 100644 --- a/docs/dev/writes.md +++ b/docs/dev/writes.md @@ -80,10 +80,17 @@ deferred to a follow-up cycle — tracked). Three writers have been migrated onto staged primitives: * **`ensure_indices`** (`db/omnigraph/table_ops.rs::build_indices_on_dataset_for_catalog`) - — scalar indices (BTree, Inverted) now use `stage_create_*_index` + - `commit_staged`. Vector indices stay inline (residual — Lance - `build_index_metadata_from_segments` is `pub(crate)` in 6.0.1; - companion ticket to lance-format/lance#6658 needed). + — scalar indices (BTree, Inverted) use `stage_create_*_index` + + `commit_staged`. Which index a `@index`/`@key` property gets is dispatched by + type via `node_prop_index_kind` (enum + orderable scalar → BTree, free-text + String → Inverted/FTS, Vector → vector). Vector indices stay inline (residual + — Lance `build_index_metadata_from_segments` is `pub(crate)` in 6.0.1; + companion ticket to lance-format/lance#6658 needed). This build is + existence-gated (it creates a *missing* index over current fragments); folding + fragments appended afterward into an *existing* index is `optimize`'s + `optimize_indices` pass — an inline-commit residual, not a staged write (Lance + exposes no uncommitted index-optimize), covered by the optimize recovery + sidecar (see [maintenance.md](../user/operations/maintenance.md)). * **`branch_merge::publish_rewritten_merge_table`** (`exec/merge.rs`) — merge_insert now uses `stage_merge_insert` + `commit_staged`. Deletes stay inline (Lance #6658 residual). diff --git a/docs/user/operations/maintenance.md b/docs/user/operations/maintenance.md index a804e31..4f065e5 100644 --- a/docs/user/operations/maintenance.md +++ b/docs/user/operations/maintenance.md @@ -4,14 +4,15 @@ ## `optimize` — non-destructive -- Compacts every node + edge table on `main`, then **publishes the compacted version to the `__manifest`** so the manifest's recorded version tracks the compacted state. Reads pin the manifest version, so without this publish compaction would be invisible to readers *and* would break the version precondition of the next schema apply / strict update/delete ("stale view … refresh and retry"). The publish advances the graph version (a system-attributed commit) only for tables that actually compacted. +- Compacts every node + edge table on `main`, then reindexes them, then **publishes the resulting version to the `__manifest`** so the manifest's recorded version tracks the compacted-and-reindexed state. Reads pin the manifest version, so without this publish the work would be invisible to readers *and* would break the version precondition of the next schema apply / strict update/delete ("stale view … refresh and retry"). The publish advances the graph version (a system-attributed commit) only for tables that actually changed. - Rewrites small fragments into fewer large ones; old fragments remain reachable via older versions until `cleanup` runs. -- Each table's compact→publish serializes with concurrent mutations on the same table. A crash mid-operation is recovered automatically on the next open (compaction is content-preserving, so roll-forward is always safe). +- **Reindex (index coverage maintenance).** A scalar/FTS/vector index only covers the fragments it was built over. Rows appended after the index was built (e.g. by `load --mode merge`, whose commit does not rebuild an already-existing index) are scanned unindexed, and compaction itself rewrites fragments out of an index's coverage. `optimize` runs Lance's incremental `optimize_indices` after compaction to fold those fragments back in (a delta merge, not a full retrain), restoring full coverage so equality/range/traversal predicates stay index-accelerated. This is why a table with **no compaction work but stale index coverage still commits** a new version under `optimize`. Run `optimize` on a cadence at least as frequent as your freshness window so recently-loaded rows do not linger in the unindexed flat-scan tail. +- Each table's compact→reindex→publish serializes with concurrent mutations on the same table. A crash mid-operation is recovered automatically on the next open (both compaction and reindex are content-preserving, so roll-forward is always safe). - **Requires a recovered graph.** `optimize` refuses (errors) when a pending crash-recovery operation is present — operating on an unrecovered graph could publish a partial write that recovery would roll back. Reopen the graph to run recovery, then re-run `optimize`. - **Uncovered drift is skipped, not interpreted.** If a table's underlying version is ahead of the version recorded in `__manifest` and no crash-recovery record covers that movement, `optimize` reports `skipped: DriftNeedsRepair` with the manifest/head versions and leaves the table untouched. Run `omnigraph repair` to classify and explicitly publish that drift. - Bounded by `OMNIGRAPH_MAINTENANCE_CONCURRENCY` (default 8). - Returns per-table stats: `table_key, fragments_removed, fragments_added, committed, skipped, manifest_version, lance_head_version`. -- **Blob tables are skipped.** A table that declares any `Blob` property is not compacted: it is reported with `skipped: BlobColumnsUnsupportedByLance` (and logged) instead of compacted, and the rest of the sweep proceeds normally. **Reads and writes are unaffected** — only compaction is. Consequence: fragment count and deleted-row space on blob tables are not reclaimed; query results are never affected. +- **Blob tables are skipped.** A table that declares any `Blob` property is not compacted: it is reported with `skipped: BlobColumnsUnsupportedByLance` (and logged) instead of compacted, and the rest of the sweep proceeds normally. **Reads and writes are unaffected** — only compaction is. Consequence: fragment count and deleted-row space on blob tables are not reclaimed; query results are never affected. A skipped blob table is also **not reindexed** in the same sweep (the skip happens before the reindex step), so its index coverage on appended rows is not refreshed by `optimize` today. ## `repair` — explicit diff --git a/docs/user/search/indexes.md b/docs/user/search/indexes.md index 84b968d..ebd69b1 100644 --- a/docs/user/search/indexes.md +++ b/docs/user/search/indexes.md @@ -4,10 +4,27 @@ | Index | Use | Notes | |---|---|---| -| **BTREE scalar** | range / equality on any scalar | created on `@key`, `@index(...)`, and on key columns by `ensure_indices()` | -| **Inverted (FTS)** | `search`, `fuzzy`, `match_text`, `bm25` | created on text columns referenced by FTS queries | +| **BTREE scalar** | `=` / range / `IN` / `IS NULL` on a scalar | always on the node `id` and edge `src`/`dst`; and on each one-column `@index`/`@key` property that is an **enum** or an **orderable scalar** (`DateTime`/`Date`/`I32`/`I64`/`U32`/`U64`/`F32`/`F64`/`Bool`) | +| **Inverted (FTS)** | `search`, `fuzzy`, `match_text`, `bm25` | created on **free-text** (non-enum) `String` `@index`/`@key` columns | | **Vector** | `nearest()` k-NN | Lance picks IVF_PQ vs HNSW family by configuration; OmniGraph stores as FixedSizeList(Float32, dim) | +The per-property index a column gets is decided by `node_prop_index_kind` (shared +by the builder and the sidecar-pinning coverage check so they cannot drift): +enums and orderable scalars → BTREE, free-text Strings → FTS, `Vector` → vector, +list/`Blob` columns → none. + +> **Free-text Strings are not equality-indexed.** A non-enum `String` column +> (including a `String @key` slug) gets an FTS inverted index, which Lance does +> **not** consult for `=`/range — only for `search`/`match_text`/`bm25`. So an +> equality filter on a free-text String falls back to a full scan. If you filter +> a String identifier by equality on a large table, model it so the value is the +> node id, or track it as a follow-up to also build a BTREE on such columns. + +> **Coverage and cost.** Each indexed column adds index files and build time, and +> an index only covers the fragments it was built over. Rows appended after the +> index was built (e.g. by `ingest --mode merge`) are scanned unindexed until a +> reindex extends coverage; see [maintenance](maintenance.md) → `optimize`. + ## L2 — OmniGraph orchestration - `ensure_indices()` / `ensure_indices_on(branch)` — idempotent build of BTREE + inverted indexes for the current head; safe to re-run. From 35e87ab882a7a358b0470e8d6644062143bde9dc Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sun, 14 Jun 2026 20:21:23 +0300 Subject: [PATCH 160/165] =?UTF-8?q?docs(rfc):=20RFC-011=20=E2=80=94=20CLI?= =?UTF-8?q?=20refactoring=20(one=20addressing=20&=20config=20model)=20(#22?= =?UTF-8?q?8)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs(rfc): RFC-011 — CLI refactoring (one addressing & config model) A maintainer-internal RFC (Status: Proposed) for the post-omnigraph.yaml CLI: one ontology (store/server/cluster; cluster vs operator config; catalog; context; capability); addressing = scope + --graph with the access path *derived*; served is the default front door and direct storage is privileged (admin/break-glass); stateless per command; definitions named, payloads passed. Includes the full end-state command taxonomy (by capability), a current-state appendix, migration, invariants check, and the resolved Decisions (with two deferred). Completes the config/CLI lineage RFC-007 → RFC-008 → RFC-009 → RFC-010. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(rfc): RFC-011 — address Greptile review (4 doc fixes) - P1: end-state taxonomy `schema apply` annotation said "Open Q10" — now points at the resolved Decision 10 (cluster graphs via cluster apply). - P1: add the `alias <name>` verb (Decision 4) to the end-state taxonomy's local section — it was claimed "full command set" but omitted. - P2: Decision 11's bulk-data-plane reference now carries the "PR #219, not yet merged" caveat (matches the Relationship section). - P2: footnote now states the `check`→`lint` argv-shim is removed (its end-state disposition was unspecified). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- docs/dev/index.md | 1 + docs/dev/rfc-011-cli-refactoring.md | 754 ++++++++++++++++++++++++++++ 2 files changed, 755 insertions(+) create mode 100644 docs/dev/rfc-011-cli-refactoring.md diff --git a/docs/dev/index.md b/docs/dev/index.md index ac8c07f..c41853e 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -80,6 +80,7 @@ Working documents for in-flight feature work. Removed when the work lands. | Deprecate `omnigraph.yaml` — one concern per config surface; key-by-key migration map and staged retirement | [rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md) | | Unify CLI embedded/remote access paths — parity referee, shared wire-DTO crate, `GraphClient` trait, declared plane capabilities | [rfc-009-unify-access-paths.md](rfc-009-unify-access-paths.md) | | Restructure the CLI around explicit planes — one graph-addressing model, declared capability surface, plane-grouped help (expands RFC-009 Phase 4) | [rfc-010-cli-planes-restructure.md](rfc-010-cli-planes-restructure.md) | +| CLI refactoring — one addressing & config model post-`omnigraph.yaml`: scope + `--graph` + derived access path, served-default / privileged-direct, contexts, named queries, capability classifier (completes RFC-008) | [rfc-011-cli-refactoring.md](rfc-011-cli-refactoring.md) | ## Boundary diff --git a/docs/dev/rfc-011-cli-refactoring.md b/docs/dev/rfc-011-cli-refactoring.md new file mode 100644 index 0000000..c067149 --- /dev/null +++ b/docs/dev/rfc-011-cli-refactoring.md @@ -0,0 +1,754 @@ +# RFC-011: CLI refactoring — one addressing & config model + +**Status:** Proposed +**Date:** 2026-06-14 +**Audience:** CLI/server maintainers +**Builds on:** [rfc-007-operator-config.md](rfc-007-operator-config.md) +(per-operator config, keyed credentials, named servers), +[rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md) +(the legacy file this RFC finishes removing), +[rfc-009-unify-access-paths.md](rfc-009-unify-access-paths.md) +(`GraphClient` — embedded ≡ remote at the execution layer), +[rfc-010-cli-planes-restructure.md](rfc-010-cli-planes-restructure.md) +(declared planes + the wrong-plane guard this RFC subsumes). +**Sequencing:** lands as / after RFC-008 stage 5 (the `omnigraph.yaml` removal). + +## Summary + +Refactor the CLI around one coherent model once `omnigraph.yaml` is gone. The +shape: + +- **One ontology** (store, server, cluster; cluster config vs operator config; + catalog; context; capability) where each term names exactly one concept. +- **Addressing = scope + `--graph`, with the access path *derived*.** A command + resolves a *scope* (operator defaults, an optional named *context*, or one + explicit primitive address — `--store` / `--server` / `--cluster`), selects a + graph inside it with `--graph`, and the **served-vs-direct access path falls out + of the scope's bindings × the verb's capability** — it is never a per-command + toggle and never inferred from a URI scheme. +- **Served is the front door; direct storage is privileged.** The everyday scope + is a *server* (a bearer token, no bucket credentials). Reading or writing a + remote store/cluster directly is an explicit, credentialed, admin/break-glass + act — never the default, never baked into everyday operator config. +- **The CLI is stateless per command.** No `current_context` pointer, no + `USE`-style mode; every command is fully determined by its flags + static + config. You *select* a graph, you do not *switch into* one. +- **Definitions are named; payloads are passed.** Queries (`.gq`) and schema + (`.pg`) live in the catalog and are invoked by name; params and bulk data are + the only per-call inputs. + +This removes `--target`, `--cluster-graph`, `--uri` scheme-dispatch, and the +plane guard's "a `--target` that resolves to a remote URL" special case — and it +collapses the four-plane vocabulary, for users, into a single capability rule. + +## Motivation: the legacy file pollutes the taxonomy + +Today the CLI exposes four overlapping addressing forms but the system has only +three real entities; the mismatch is the whole problem, and `omnigraph.yaml` is +the carrier: + +1. **`--target` straddles kinds.** It resolves through the legacy + `omnigraph.yaml` `graphs:` map (`config.rs::resolve_target_uri`), and that + `.uri` can be a **storage location** (`file`/`s3`) *or* a **remote server** + (`http`). One flag, two access paths with different capability and trust + models. The wrong-plane guard's storage-plane remote rejection + (`helpers.rs:467`) exists *only* to compensate for this overload. +2. **Scheme-inferred transport.** `<URI>`/`--uri` has the same disease a level + down: `is_remote_uri` (`helpers.rs:15`) silently picks embedded vs remote from + the scheme. Transport is guessed from a string, not declared. +3. **No single environment concept.** Defaults are smeared across the deprecated + `omnigraph.yaml` (`cli.graph`, `server.graph`) with no clean way to name or + switch environments. + +Removing `omnigraph.yaml` is the moment to fix all three at once. + +## Ontology + +Every term is one concept. The rest of this RFC uses them precisely. + +### Entities — the things that exist + +- **Graph** — a typed property graph (node/edge types over Lance); the thing you + query and mutate. *Example: the `knowledge` graph.* +- **Store** — the storage location of a **single** graph: its Lance datasets at a + `file://`/`s3://` URI. Addressed directly with `--store`. *Example: + `s3://acme/clusters/brain/graphs/knowledge.omni`.* +- **Cluster** — a storage root holding **many** graphs plus the catalog and + control-plane state (state ledger, approvals, recovery). Managed as-code by the + team. *Example: the `brain` cluster at `s3://acme/clusters/brain`.* +- **Server** — an `omnigraph-server` process serving graphs over HTTP with bearer + auth and Cedar policy; boots from a bare graph or a cluster. *Example: `prod` at + `https://graph.example.com`, serving the `brain` cluster.* + +### Config & catalog — the descriptions + +- **Cluster config** — `cluster.yaml` in the cluster root, declaring the **desired + state** (graphs, schemas, stored queries, policies, storage), applied with + `cluster apply`. Team-owned; the source of truth for *what the system is*. +- **Catalog** — the **applied** registry the cluster owns in storage: the graphs, + stored queries, and policies `cluster apply` materialized. What a server serves + and what `query <name>` resolves against. *(Cluster config is the spec; the + catalog is the applied result.)* +- **Operator config** — `~/.omnigraph/config.yaml`, your **personal** file: + identity (actor), default graph, named servers/clusters, output prefs, optional + contexts. Declares *who I am*, never what the system is. +- **Context** — an optional named bundle of **defaults inside the operator + config** (one of {cluster, server, store} + a default graph). Config data, + **not state**: selecting one fills in omitted flags for a command; it does not + put you "in" a mode. Chosen per command (`--context <name>`) or per shell + (`OMNIGRAPH_CONTEXT`). +- **Credential** — a bearer token keyed to a **server name**, resolved via + `OMNIGRAPH_TOKEN_<NAME>` or `~/.omnigraph/credentials` (`0600`); sent only to + the server it is keyed to. (Per RFC-007 — the operator config holds endpoints, + never tokens.) + +### What you run — definitions vs payloads + +- **Schema** — the `.pg` type definitions for a graph; authored as a file, applied + via `schema apply` (or `cluster apply`). +- **Stored query** — a named query in the catalog, the team's reusable contract; + invoked by name. *Example: `find_people`.* +- **Query file (`.gq`)** — an authoring artifact holding `query <name>` + declarations; becomes a stored query when `cluster apply` adopts it. For + authoring/ad-hoc, not everyday invocation. +- **Payload** — the per-call inputs that vary each run: params (`--params`, + positional args) and bulk data (`--data`). Never part of config. + +### How a command resolves + +- **Scope** — the resolved environment a command addresses: operator defaults, a + named context, or one explicit primitive address. +- **Access path** — **served** (through a server) or **direct** (open storage + in-process). Derived from scope × capability; see "Access path" below. +- **Capability** — what a verb requires: `any`, `served`, `direct`, `control`, + or `local`. +- **Target shape** — whether the verb is **graph-scoped** (selects one graph + inside the scope), **scope-scoped** (operates on the whole server/cluster + scope), or **local** (does not resolve scope or graph). +- **Actor** — the identity a write is attributed to: server-resolved from the + bearer token (served), or `--as` ?? `operator.actor` (direct). + +### The relationships that prevent confusion + +- **Exactly two config surfaces:** **cluster config** (team) and **operator + config** (personal). Nothing else is "a config." +- A **context is not a third config** — it lives *inside* the operator config, and + it is **defaults, not state**. +- A **catalog is not config** — it is the *applied state* the cluster owns. +- A **store is one graph; a cluster is many graphs** + catalog + control state. +- A **graph is the logical thing**; store/server/cluster are ways to reach it. +- "State" elsewhere is not the context: *graph state* is committed data in Lance; + *cluster state* is the applied control-plane ledger. Neither is operator config. + +## Design + +### First principles + +> Addressing should be 1:1 with the system's real entities; the access path +> (served vs direct) should be **derived**, never inferred from a string or +> toggled per command; the CLI should be **terse by config and stateless per +> command**; and **definitions are named while payloads are passed**. + +Every command answers four orthogonal questions — kept orthogonal here: + +| Axis | Question | Today | Target | +|---|---|---|---| +| Scope | which environment? | `omnigraph.yaml` defaults / `--target` | operator defaults · `--context` · one primitive | +| Target shape | whole scope or one graph? | implicit in command family | declared per verb | +| Graph | which graph in it? | tangled into the address | `--graph` only for graph-scoped server/cluster verbs | +| Access path | served or direct? | inferred from scheme / target | **derived** from scope × capability | +| Actor | who am I? | `--as` > `cli.actor` (yaml) > `operator.actor` | `--as`/`operator.actor` (direct) · token (served) | + +### A scope binds one entity — and served is the default + +A scope (a context, the flat defaults, or one primitive flag) binds **exactly one +of** {server, cluster, store}. Server and cluster scopes may contain many graphs +and can carry a `default_graph`; a store scope is already one graph and does not +accept `--graph`. They differ by privilege, and **the everyday default is a +server**: + +- **server** → served (the everyday scope). A bearer token, **no storage + credentials**. Data verbs run through it, policy-enforced; maintenance verbs are + unavailable from this scope — there is no server route for them, so you must + name storage explicitly. This is what a normal operator's config binds. +- **cluster** → direct storage to a managed cluster, for **control, + maintenance, and graph-backed validation only** (`cluster *`, + `optimize`/`repair`/`cleanup`/`schema plan`, graph-backed `lint`, and + `queries validate`). Data verbs are **not** run directly against a cluster — + they go served, or `--store` for ad-hoc. **Privileged:** requires bucket + credentials, so it appears only in a maintainer's config or as an explicit + `--cluster` flag — never in an everyday operator's defaults. +- **store** → one graph's storage, direct. A **local file** store is ordinary + local dev; a **remote `s3://`** store is break-glass. No catalog (named queries + do not resolve — the ad-hoc lane). + +A scope names **one** thing, so there is no independent `server`+`cluster` pair +that could disagree (the audit's coherence hazard is gone by construction — the +default is just a server). And the storage root lives only where it must: + +### Direct storage access is privileged (the storage-root rule) + +> The storage root (`s3://…`) is **server-and-admin knowledge, never +> everyday-operator knowledge.** Everyday operator config binds a server (a bearer +> token, no bucket credentials). Direct remote access — opening a cluster root or +> an `s3://` store — is always **explicit and privileged**: you name +> `--cluster`/`--store`, and only someone with bucket credentials can. The CLI +> never opens a remote store from a default scope. + +This is the least-privilege posture — revoke a bearer token, don't rotate bucket +keys; only the **server process** and an occasional **maintenance admin** ever +hold storage credentials. It makes "use the server, not raw storage" +**structural**, not advisory: direct access requires credentials a normal operator +does not have *and* a flag they must type. The only storage root in an everyday +setup is the one the **server** boots from; operators never see it. (Local *file* +stores for dev are unaffected — a local file is not the production bucket.) + +### Access path is derived, not chosen + +The two access paths are genuinely different — not two transports for one thing: + +- **Served** (through a server): the server resolves your actor from a token and + enforces Cedar policy at the HTTP boundary. In cluster mode the **catalog and + config** (graph set, stored queries, policy bundles) are pinned to the applied + serving revision and move only on restart; **graph data** is read through the + server's engine handle against the requested branch/snapshot (it is not frozen + at boot, though a long-running server will not observe *out-of-band direct + writes* to storage until its handle refreshes). No storage credentials needed. +- **Direct** (open the Lance storage in-process): a **privileged** path — it needs + your own storage credentials, so only an admin/maintainer (or a local-dev file + store) takes it. Actor self-declared (`--as` ?? `operator.actor`), reads **live + storage HEAD**. There is **no server-side identity/auth gate** — but engine-level + Cedar policy *is* still enforced when the graph selection provides a policy + (enforcement is engine-wide; embedded `_as` writers call the same `enforce`). + "Direct" means "no HTTP boundary," not "unpoliced." + +Because they differ in authority, freshness, and availability, a graph reached via +a server and that graph's raw storage are **different things you name +differently** — not one identity you flip. Making the access path a per-command +toggle (`--via`) is the `--target` mistake in new clothes; it is rejected. + +> **The access path follows from the scope and the verb.** A **server** scope → +> served (data/catalog). A **cluster** scope → direct control, maintenance, and +> validation. A **store** scope → direct ad-hoc data (no catalog). The verb's +> capability picks which applies and rejects the mismatches. + +State the bound plainly: the everyday data path +(`query`/`mutate`/`load`/`branch`/`export`/`commit`) against a served graph +**never needs direct storage access**, and direct access is legitimate only in +bounded places: **bootstrap** (`init`), **storage-native maintenance** +(`optimize`/`repair`/`cleanup`/`schema plan`), **graph-backed validation** +(`lint`), **catalog validation** (`queries validate`), the **control plane** +(`cluster *`), **local dev** with no server, and **break-glass** (recovery, or +checking whether a long-running server's handle lags live HEAD). Everything else +is served. This is what makes "discourage direct storage" enforceable rather +than aspirational. + +This list is expected to **shrink**: Decision 11 moves +`optimize`/`cleanup` (and healthy-path `repair`) to server-managed jobs, which +would leave direct access to just standalone/local dev, the control plane, and +break-glass — and remove the last routine reason an admin needs bucket +credentials. + +### Capability semantics + +The CLI validates through verb capability, not plane jargon: + +| Capability | Meaning | Examples | +|---|---|---| +| `any` | graph-scoped data; served via a server scope; direct only against a **store** scope (local dev / break-glass); **errors on a cluster scope** | `query`, `mutate`, `load`, `export`, branch reads, `schema show/apply` | +| `served` | requires an HTTP server; may be graph-scoped or scope-scoped | `graphs list`, `queries list` | +| `direct` | graph-scoped storage-native or graph-backed validation; no server form exists | `init`, `optimize`, `repair`, `cleanup`, `schema plan`, graph-backed `lint` | +| `control` | cluster-scoped catalog/control-plane work; addresses the cluster, not a single raw store | `cluster *`, `queries validate` | +| `local` | does not address a graph or scope | `config`, `context`, `lint --query ... --schema ...` | + +`any` does **not** mean "the user picks": the resolver picks from the scope. +Internally the exhaustive `command_plane` match (`planes.rs`) stays as the drift +guard; user-facing errors speak in terms of what the command needs. + +### Definitions vs payloads + +Queries and schema are **definitions** — contracts that live in the catalog and +are invoked **by name**; params and data are **payloads** passed per call. So the +everyday form is `omnigraph query <name> [params]`, not +`omnigraph query --file find.gq`. A `.gq` path on a routine query is a smell: the +query is not in the catalog yet. Lifecycle: **author a `.gq` → `cluster apply` +adopts it → invoke by name thereafter.** + +Named queries resolve through a **server** (which serves the cluster's catalog). +`queries list` is therefore a served catalog read. `queries validate` is a +control/catalog check against the cluster-owned query definitions. A bare +`--store` has **no catalog**, so it is the ad-hoc lane (`-e` / `--file`), and +`--cluster` does not invoke stored queries. So named-query invocation is a +**served** convenience; direct access (`--store`) is always ad-hoc. + +| Kind | Examples | How it enters a command | +|---|---|---| +| Definition | stored query, schema | named in the catalog; authored as a file, adopted by `cluster apply` | +| Payload | params, bulk data | passed per call (`--params`, positional args, `--data`) | +| Authoring / ad-hoc | a `.gq` you're writing | `-e '…'`, `--file new.gq`, `lint --query new.gq --schema schema.pg`, `schema apply --schema` | + +### Resolution rule + +1. If the verb is `local`, reject graph/scope flags and run without resolving a + scope. +2. If a primitive address is supplied (`--store`/`--server`/`--cluster`), use it + and ignore operator-config scope defaults. *(A **named** primitive — `--server + prod`, `--cluster brain` — still resolves through the operator-config registry; + a **literal** — `--server https://…`, `--store s3://…` — bypasses it. Per + Decision 2: a value containing `://` is a literal, otherwise a config-name + lookup.)* +3. Else if `--context <name>` (or `OMNIGRAPH_CONTEXT`) selects a context, use it. +4. Else use the operator config's flat defaults. Error only if neither resolves. + *(No sticky "current" pointer — each command resolves scope fresh.)* +5. Resolve the graph only for **graph-scoped** verbs. Server/cluster scopes: + exactly one graph in scope → use it; else `default_graph`; else require + `--graph <id>`. Store scopes are already one graph, so `--graph` is rejected. + **Scope-scoped** verbs (`graphs list`, `queries list`, `queries validate`, + and `cluster *`) do not select a graph unless their own resource argument says + otherwise. +6. Derive the access path from capability × scope: + - `direct` verb → the scope's cluster/store; if the scope is a server, error + (name storage explicitly — it is privileged). + - `served` verb → the scope's server; if the scope is a cluster/store, error. + - `control` verb → the scope's cluster; if the scope is a server/store, error + (name a cluster explicitly — it is privileged). + - `any` verb → **served** if the scope is a server; **direct** against a + **store** scope (ad-hoc); on a **cluster** scope, error — cluster is + maintenance-only, so use a server for data or `--store` for ad-hoc. +7. Reject mismatches with an error naming the missing axis. + +Good errors: + +```text +scope "prod" has 4 graphs; pass --graph <id> or set default_graph +optimize needs direct storage access; scope "prod" is a server — name storage with --cluster s3://… or --store (requires storage credentials) +graphs list enumerates a server scope; do not pass --graph +--store opens raw storage directly, bypassing any server (no HTTP auth gate, live HEAD); for recovery/inspection +``` + +### Config shape (operator config) + +`~/.omnigraph/config.yaml` — your personal file; the cluster config +(`cluster.yaml` + catalog) is the separate, team-owned surface. The default-graph +key is `default_graph` everywhere (the per-command flag is `--graph`). + +**Everyday operator — binds a server, holds no storage root:** + +```yaml +defaults: + server: prod + default_graph: knowledge + output: table +servers: + prod: { url: https://graph.example.com } # token keyed by name (RFC-007); no creds here + staging: { url: https://staging.example.com } +contexts: # optional, only for multiple environments + staging: { server: staging, default_graph: knowledge } +``` + +A normal operator never has a storage root or bucket credentials. Their default +scope is served; `optimize`/`repair`/`cleanup` error with a pointer to name +storage explicitly. + +**Maintainer — opts into a cluster root (and has bucket credentials):** + +```yaml +contexts: + brain-admin: { cluster: brain, default_graph: knowledge } # direct; admin/control/maintenance +clusters: + brain: { root: s3://acme/clusters/brain } # the s3:// root lives ONLY here +``` + +The `clusters:` block — the only place a storage root appears in operator config — +is **admin-only and opt-in**, absent from a normal operator's file. Equivalently, +skip config and name it per command: +`omnigraph optimize --cluster s3://acme/clusters/brain --graph knowledge`. The +cluster stays the source of truth for the managed catalog; tokens live in the +keyed credential store, never in this file. + +### Command shape + +Assume the everyday flat defaults: server `prod`, default graph `knowledge`. + +| Intent | Command | Path | +|---|---|---| +| Run a catalog query | `omnigraph query find_people` | served | +| …with params | `omnigraph query find_people --params '{"title":"Eng"}'` | served | +| Another graph in scope | `omnigraph query find_people --graph archive` | served | +| Write | `omnigraph load --data batch.jsonl --mode append` | served | +| A different environment | `omnigraph --context staging query find_people` | served | +| One-off server, no config | `omnigraph query find_people --server https://graph.example.com --graph knowledge` | served | +| Maintain (admin, explicit storage) | `omnigraph optimize --cluster s3://acme/clusters/brain --graph knowledge` | direct (privileged) | +| Maintain (admin, via admin context) | `omnigraph --context brain-admin optimize --graph knowledge` | direct (privileged) | +| List catalog queries | `omnigraph queries list` | served | +| Validate cluster query catalog | `omnigraph queries validate --cluster s3://acme/clusters/brain` | control (privileged) | +| Offline query lint | `omnigraph lint --query new.gq --schema schema.pg` | local | +| Graph-backed query lint | `omnigraph lint --query new.gq --cluster s3://acme/clusters/brain --graph knowledge` | direct (privileged) | +| Local dev, no server | `omnigraph query -e 'match { … } return { … }' --store graph.omni` | direct (local file) | +| Break-glass: raw storage of a served graph | `omnigraph query --file find.gq --store s3://acme/clusters/brain/graphs/knowledge.omni` | direct (privileged, rare) | + +Note what the everyday rows are: **all served.** `optimize` does *not* appear in +the default-scope rows — from a server scope it errors and points you to name +storage (see the resolution rule), so maintenance is always a deliberate, +credentialed act. There is no "force served/direct" row — you never toggle the +path on a configured graph; the only way to reach raw storage is to *name it* +(`--cluster`/`--store`), which makes the privileged bypass unmistakable. Everyday +rows invoke a query **by name**; a `.gq` file appears only where there is no +catalog (bare store, break-glass) via `-e`/`--file`. + +## Before / after + +**Before** = best available today (legacy `omnigraph.yaml` `--target`, `.gq` +files, `--cluster-graph`, scheme inference). **After** = this model. + +| Intent | Before | After | +|---|---|---| +| Run a query | `omnigraph query --target knowledge --query find.gq --name find_people` | `omnigraph query find_people` | +| Another graph | `omnigraph query --target archive --query find.gq --name find_people` | `omnigraph query find_people --graph archive` | +| Load | `omnigraph load --data b.jsonl --mode append --target knowledge` | `omnigraph load --data b.jsonl --mode append` | +| Maintain (admin) | `omnigraph optimize --cluster brain --cluster-graph knowledge` | `omnigraph optimize --cluster s3://acme/clusters/brain --graph knowledge` | +| Another environment | edit `omnigraph.yaml`, or re-address with full URIs | `--context staging …` or `OMNIGRAPH_CONTEXT=staging` | +| One-off remote | `omnigraph query --uri https://… --query find.gq` *(scheme→remote)* | `omnigraph query find_people --server https://… --graph knowledge` | +| Raw storage of a served graph | `omnigraph query s3://…/knowledge.omni --query find.gq` *(looks like a normal query)* | `omnigraph query --file find.gq --store s3://…/knowledge.omni` *(explicit bypass)* | + +**Removed:** `--target`; `--cluster-graph` (`--graph` is the graph selector only +for graph-scoped server/cluster verbs); `--uri` http-scheme dispatch; `--via` +(never ships); everyday `--query <file>` (definitions are named); +`omnigraph.yaml` and its `cli.graph`/`server.graph` defaults. + +## Server-side corollary + +The same ontology applies to `omnigraph-server` boot: with `omnigraph.yaml` gone, +a server boots from a single bare graph URI **or** a cluster (`--cluster <dir|s3>`, +RFC-005), never a `graphs:` map. The store/server/cluster ontology is then +consistent across CLI and server. + +## Migration & compatibility + +Addressing flags and config keys are observable contract (Hyrum); every removal is +staged and release-noted. + +- **`config migrate`** (shipped) maps each legacy `graphs:` entry **by what it + actually is**: `http(s)` URIs → a `server:` (the recommended everyday shape); + `file` URIs → a local `store:`; an `s3://` **graph** URI → an **admin** `store:` + (it is a single graph, not a cluster); an `s3://` **cluster root** (one that + carries cluster state) → an **admin** `cluster:`. Everyday `s3://` graph usage + migrates with a **warning** — prefer serving it via a server rather than + re-establishing direct remote access. It reports dropped keys. +- **Operators move to a server-default scope.** Where a legacy setup pointed + `cli.graph` at an `s3://` graph for everyday use, migration flags it: the + recommended shape is a `server:` scope (bearer token, no bucket creds), with the + `s3://` root kept only in a maintainer's config — not every operator's. +- **`--target`** warns for one release, then errors; **`OMNIGRAPH_NO_LEGACY_CONFIG=1`** + (already the strict switch) becomes the default — loading `omnigraph.yaml` is a + hard error. +- **`--cluster-graph` → `--graph`**: `--cluster-graph` is accepted with a warning + for one release, then removed. +- **`--graph` meaning change**: today `--graph` is "graph id on a multi-graph + server" (paired with `--server`); it generalizes to "select the graph for + graph-scoped verbs in server/cluster scopes." Existing `--server --graph` + usage keeps working (it is a strict superset); release-note the broadened + meaning and the fact that store/scope-scoped verbs reject it. +- **`--uri http://…`** warns, then errors with a pointer to `--server`. +- **`--as` on served paths**: today global `--as` is accepted (a no-op on remote + writes — the server resolves the actor from the token); rejecting it on the + served path is staged — warn for one release, then error. +- **`--alias`** → the `alias` namespace (`omnigraph alias <name>`, Decision 4); + the old `--alias` flag warns for one release, then is removed. + +## Non-goals + +- **No change to the direct/served capability split.** Maintenance stays + storage-direct by design (no server routes for `optimize`/`repair`/`cleanup`); + this RFC only makes the split explicit. +- **No new transport.** Addressing surface, not protocol. +- **No positional sigil grammar** (`@server/graph`, `%cluster/graph`). Considered + and rejected: explicit flags are more discoverable; contexts already give + brevity. Revisit only on demonstrated expert-terseness demand. + +## Decisions + +The questions this RFC opened are resolved as follows. Two are explicitly +deferred (see below); they do not block the model. + +1. **Local-dev path → embedded `--store` scope.** Local dev runs the engine + in-process against a `--store <file>` (or a store-scoped context); `omnigraph + serve` stays available but is not required. Consistent with embedded ≡ remote + (RFC-009). +2. **Primitives are one flag, typed by content.** `--server` and `--cluster` + accept either a config name or a literal URI: a value containing `://` is a + literal (bypasses the registry); otherwise it is a config-name lookup (error if + unknown). `--store` is always a URI. (Replaces the earlier "literal-vs-named" + question — no `--server-url`/`--cluster-root` split.) +3. **Stored invocation: `query <name>` (read) / `mutate <name>` (write), one + catalog namespace.** A name maps to one definition; the verb asserts its kind + and the CLI errors on mismatch (`'apply_labels' is a mutation — use + omnigraph mutate apply_labels`). No `invoke` verb. +4. **Aliases live under an `alias` namespace** — `omnigraph alias <name> [args]`, + never bare top-level. An alias can therefore neither shadow nor be shadowed by a + built-in (current or future) verb. +6. **Context merge: scope wholesale, prefs layered.** The entity binding + + `default_graph` come *wholesale* from the active scope (a context, or flat + defaults if none) — never per-key merged across the entity dimension (that would + yield "server *and* cluster"). Only non-scope preferences (`output`, table + layout) take flat defaults as a base. Precedence: explicit flag > context > flat + defaults. +7. **No default graph → error + list candidates.** A graph-scoped verb with no + `--graph`, no `default_graph`, and >1 graph in scope errors and lists candidates + (served: `GET /graphs`; cluster-direct: catalog enumeration). If enumeration is + policy-gated/unavailable, it says so and asks for `--graph`. Never auto-pick. +9. **Diagnostics & safety.** Writes echo the resolved scope + access path to stderr + (suppress with `--quiet`). Destructive verbs (`cleanup`, overwrite `load`, + `branch delete`) require confirmation when the scope is not local; `--yes` skips + it; **no TTY without `--yes` errors** (never silently proceed). `--json`/CI never + prompt — destructive without `--yes` errors. +10. **Cluster graphs evolve only via `cluster apply`.** `schema apply` (an `any` + verb) targets standalone graphs; against a cluster-managed graph it errors and + points at `cluster apply` (which records ledger/recovery/approvals — RFC-004). + Mirrors `init`'s refusal of a cluster-managed path. +11. **Maintenance moves server-side (committed direction).** `optimize`/`cleanup` + (and healthy-path `repair`) become server/cluster-managed async jobs — + policy-gated, audited, single-coordinator — with `direct` retained only as + break-glass (`repair` when the server is down). Runs out-of-band (a worker + + async job routes, the `POST …` / `GET …/{id}` shape of the bulk-data-plane RFC + (`docs/rfcs/0001-bulk-data-plane.md`, PR #219, not yet merged)), never inline in + serving; `schema plan` is + excluded (≈ `cluster plan` in cluster mode). The **mechanism** (job routes, + worker, scheduling) is a follow-up RFC; until it lands the capability table above + stands, and maintenance is `direct`. When it lands, the maintenance verbs' + capability becomes "served-job + direct break-glass." + +## Deferred + +Non-blocking; settle when convenient. + +- **D5 — combined admin scope.** A scope binds one entity; admins read via a + server scope and maintain via `--cluster`. A `deployments: { … }` object + (server + cluster validated coherent, referenced by a context) is revisited only + if admin ergonomics demand it — and Decision 11 largely removes the need. +- **D8 — the `context` command surface.** `context list` / `context show` + (read-only inspection) are additive diagnostics, shippable anytime; they don't + touch the grammar or resolution. The *no sticky `context use`* constraint holds + regardless — it is a design principle, not a command. + +## Safety + +Dropping the sticky `current_context` pointer removes the main footgun — a +destructive command silently inheriting a "current" environment from an earlier +session. Because each command resolves scope fresh, what is on the command line is +what runs. Two guards remain (a flat default or `OMNIGRAPH_CONTEXT` can still point +at prod): echo the resolved scope + access path on writes, and require +confirmation (or `--yes`) for destructive verbs when the resolved scope is not +local (Decision 9). The most dangerous direct writes (`cleanup`, overwrite +`load`) are *structurally* rare now — unavailable from the everyday server scope, +and gated behind bucket credentials plus an explicit `--cluster`/`--store` — so a +normal operator's setup mostly cannot issue them by accident at all. + +## Invariants & deny-list check + +- **§10 query semantics first-class / §11 transport at the boundary:** preserved — + addressing resolves CLI-side to a `GraphClient`; no transport concepts leak into + engine crates. +- **§12 no client-set actor:** strengthened — the served path's actor stays + token-resolved and `--as` is rejected there; direct self-declares. +- **Least privilege (security posture):** everyday operators hold a revocable + bearer token, not bucket credentials; only the server process and maintenance + admins hold storage creds. Direct remote access is structural opt-in, not a + default — narrowing the blast radius of a leaked operator config. +- **§6 strong consistency:** both paths are snapshot-isolated per query; this RFC + changes addressing, not isolation. +- **Deny-list (no state that drifts):** contexts and aliases are static config + sugar that resolve to canonical scopes; they declare nothing the cluster or + server doesn't already own. No sticky session state is introduced. +- No Hard Invariant is weakened; the change is CLI surface + config removal. + +## Relationship to prior work + +The completion of the config/CLI lineage: RFC-007 added the operator config and +keyed credentials; RFC-008 demoted `omnigraph.yaml`; RFC-009 unified execution +behind `GraphClient`; RFC-010 declared the planes. This RFC removes the last +legacy addressing surface so the plane model becomes a clean function of the three +real entities, and folds the planes into a single capability rule. It is adjacent +to the public-track bulk-data-plane RFC (`docs/rfcs/0001-bulk-data-plane.md`, +PR #219, not yet merged), which canonicalizes `load`/`export` verbs; this RFC +canonicalizes how every verb *addresses* a graph. + +## Appendix: target CLI taxonomy (end state) + +The full command set under this model, organized by **capability** (the new +classifying axis) instead of plane — the end-state counterpart to the +current-taxonomy appendix below. Every command, with its end-state addressing. + +``` +omnigraph +│ +├─ any — data verbs · served by default (server scope, or --server <url|name>); +│ --graph selects the graph in scope; --store forces ad-hoc direct (no catalog) +│ ├─ query (alias: read*) invoke a stored query by NAME; -e/--file for ad-hoc +│ ├─ mutate (alias: change*) invoke a stored mutation by name; -e/--file for ad-hoc +│ ├─ load bulk write — --data, --mode required; --from forks a missing branch +│ ├─ export dump graph data (NDJSON / Arrow) +│ ├─ snapshot current per-table versions +│ ├─ branch { create | list | delete | merge } merge takes --into <target> +│ ├─ commit { list | show } inspect the commit graph +│ └─ schema { show (alias: get) | apply } cluster graphs evolve via cluster apply (Decision 10) +│ +├─ served — needs a server (errors on a store/cluster scope) +│ ├─ graphs list enumerate the graphs a server serves +│ └─ queries list list stored queries in the served catalog +│ +├─ direct — storage-native, PRIVILEGED · --cluster <root> | --store <uri> + bucket creds; never a server +│ ├─ init bootstrap a graph (--store <uri>); refuses a cluster-managed path +│ ├─ optimize compaction; --graph selects +│ ├─ repair publish uncovered drift; --confirm / --force +│ ├─ cleanup version GC; --keep / --older-than / --confirm +│ ├─ schema plan migration preview (reads storage directly) +│ └─ lint --query <path> graph-backed query lint (with --graph on cluster scope) +│ +├─ control — cluster/catalog control, PRIVILEGED · --cluster <dir|s3> +│ ├─ cluster { validate | plan | apply | approve | status | refresh | import | force-unlock } +│ apply/approve take --as <actor>; force-unlock takes <LOCK_ID> +│ └─ queries validate validate cluster-owned stored queries against graph schemas +│ +└─ local — no graph + ├─ policy { validate | test | explain } offline Cedar tooling + ├─ context { list | show } read-only; NO mutating `use` (no sticky state) + ├─ alias <name> [args] personal shortcut; expands to its bound stored-query call (D4) + ├─ config { migrate } finish the omnigraph.yaml split (RFC-008) + ├─ login / logout per-server bearer credentials + ├─ embed offline embedding pipeline + ├─ lint --query <path> --schema <path> file-only query lint + └─ version (-v) +``` + +`*` `read`/`change` remain as deprecated aliases (warn on use); `ingest` and the +`check`→`lint` argv-shim are **removed**. `get` aliases `schema show`. + +### Addressing forms (end state) + +Three scope forms — one per real entity — plus the graph selector. No `--target`, +no `--cluster-graph`, no `--uri` scheme-dispatch, no `--via`. + +| Form | Resolves to | Access | Privilege | +|---|---|---|---| +| **server scope** — operator default, a `--context`, or `--server <url\|name>` | a served endpoint + keyed token | served | everyday (bearer token) | +| **cluster scope** — an admin context, or `--cluster <root>` | a managed cluster's storage + catalog | direct | privileged (bucket creds) | +| **store scope** — `--store <uri>` | one graph's storage (no catalog) | direct | local-dev (file) / break-glass (s3) | +| **`--graph <id>`** | selects the graph for graph-scoped verbs in server/cluster scopes; invalid for store scopes and scope-scoped verbs | — | — | + +Resolution: explicit primitive (`--server`/`--cluster`/`--store`) → `--context` / +`OMNIGRAPH_CONTEXT` → operator flat defaults. Access path is then derived from the +scope kind × the verb's capability (see the Resolution rule); it is never inferred +from a URI scheme and never toggled. + +### What moved vs today + +| Command(s) | Today (plane) | End state (capability) | +|---|---|---| +| `query`/`mutate`/`load`/`export`/`snapshot`/`branch`/`commit`/`schema show`/`schema apply` | Data | **`any`** (served-default; `--store` ad-hoc) | +| `graphs list` | Data (remote-only) | **`served`** | +| `queries list` | Session | **`served`** (catalog read) | +| `init`/`optimize`/`repair`/`cleanup`/`schema plan`/graph-backed `lint` | Storage | **`direct`** (privileged) | +| `queries validate` | Storage | **`control`** (catalog validation) | +| `cluster *` | Control | **control** (unchanged) | +| `policy *`/`embed`/`login`/`logout`/`config`/`version`/offline `lint --query --schema` | Session | **`local`** | +| `ingest`; `--target`; `--cluster-graph`; `--uri http` dispatch | present | **removed** | +| — | — | **added:** `context { list | show }` (read-only) | + +Cross-capability families: `schema` (`plan` is `direct`, `show`/`apply` are +`any`), `queries` (`list` is `served`, `validate` is `control`), and `lint` +(offline with `--schema` is `local`, graph-backed is `direct`) split per +subcommand/mode, exactly where their authority and data dependencies differ. + +## Appendix: current CLI taxonomy (today) + +The **as-is** command surface this RFC transforms, kept so the RFC is +self-contained. The source of truth is the exhaustive `command_plane` match in +`crates/omnigraph-cli/src/planes.rs`. +Where it disagrees with the design above (four planes, `--target`, +`--cluster-graph`, scheme-inferred transport), the design is the *target* and this +is *today*. + +### The four planes (today) + +| Plane | What it touches | Addressing accepted | +|---|---|---| +| **Data** | a graph — embedded **or** via a server | `<URI>` · `--target` · `--server` (+`--graph`) | +| **Storage** | direct storage, no server | `<URI>` · `--target` (local/S3 only) · some also `--cluster`+`--cluster-graph` | +| **Control** | a cluster *directory* | `--config <dir>` | +| **Session** | no graph | — | + +`--server`/`--graph` are gated strictly to the data plane; `guard_addressing` +(`planes.rs:128`) rejects them elsewhere (RFC-010 Slice 1). + +### Command tree by plane (today) + +``` +omnigraph +├─ DATA ────────── run against a graph; embedded or --server +│ ├─ query (alias: read) · mutate (alias: change) · load · ingest (hidden, deprecated) +│ ├─ branch { create | list | delete | merge } · snapshot · export · commit { list | show } +│ ├─ graphs { list } (remote-only) +│ └─ schema { show (alias: get) | apply } ← show/apply are DATA +├─ STORAGE ─────── direct file://|s3:// access; --server rejected +│ ├─ init · optimize · repair · cleanup (optimize/repair/cleanup also: --cluster --cluster-graph) +│ ├─ lint (check shim) · schema plan ← plan is STORAGE +│ └─ queries validate +├─ CONTROL ─────── cluster directory via --config <dir> +│ └─ cluster { validate | plan | apply | approve | status | refresh | import | force-unlock } +└─ SESSION ─────── no graph + ├─ policy { validate | test | explain } · embed · login / logout + ├─ config { migrate } · queries list ← list is SESSION + └─ version (-v) +``` + +`read`/`change` are visible clap aliases (deprecated names, warn); `check` is an +argv-shim → `lint`; `get` aliases `schema show`; `ingest` is hidden but runs. + +### Cross-plane families (today) + +- **`schema`**: `schema plan` is Storage; `schema show`/`apply` are Data. +- **`queries`**: `queries validate` is Storage; `queries list` is Session. + +### Addressing forms (today) + +| Form | Looks up in | Resolves to | Source | +|---|---|---|---| +| `<URI>` / `--uri` | nothing (explicit) | the literal URI | — | +| `--target <name>` | `omnigraph.yaml` `graphs:` | that graph's `uri` (local / S3 / **http**) | `config.rs::resolve_target_uri` | +| `--server <name>` (+`--graph`) | `~/.omnigraph/config.yaml` `servers:` | a remote server URL | `helpers.rs::resolve_server_flag` | +| `--cluster <dir\|s3> --cluster-graph <id>` | served cluster state | the graph's storage URI | `helpers.rs` (RFC-010 Slice 3) | + +Precedence (`resolve_target_uri`): explicit `<URI>`/`--uri` → `--target` → +`cli.graph` default → error. `is_remote_uri` (`helpers.rs:15`) then selects +`GraphClient::Remote` vs `Embedded` (`client.rs:86`). + +### Enforcement points (today) + +- **`guard_addressing`** (`planes.rs:128`): `--server`/`--graph` on a non-data verb + fails with a declared message. +- **Storage-plane remote rejection** (`helpers.rs:467`): a storage verb whose + `--target` resolves to `http(s)://` is rejected. +- **`init` into a cluster layout** is refused (use `cluster apply`). + +## Audit comments + +Reviewed against the current CLI taxonomy, `planes.rs`, `cli.rs`, `helpers.rs`, +`client.rs`, RFC-007/RFC-010, and the user-facing CLI/server docs. + +### Validated + +- The target taxonomy now has a stable classifier: `any`, `served`, `direct`, + `control`, and `local` are all declared capabilities. +- Cluster scope is coherent: it is privileged direct storage for control, + maintenance, and validation, not a direct data path. `any` data verbs served by + default and reject cluster scope. +- Graph selection is no longer universal. Graph-scoped verbs select a graph; + scope-scoped verbs such as `graphs list`, `queries list`, `queries validate`, + and `cluster *` address the whole server/cluster scope. +- The current-state appendix still matches the implemented CLI: four planes, + `--target`, `--cluster-graph`, scheme-inferred transport, `schema plan` as + Storage, and `schema show/apply` as Data. + +Decisions and deferrals are tracked in [Decisions](#decisions) above — not +duplicated here. From 6fa04efc765289d7d457b61985686c3ed05e30d9 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Sun, 14 Jun 2026 21:23:39 +0300 Subject: [PATCH 161/165] =?UTF-8?q?docs(rfc):=20RFC-011=20=E2=80=94=20rena?= =?UTF-8?q?me=20"context"=20=E2=86=92=20"profile"=20(#233)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "context" (borrowed from kubectl) collides with agent/LLM "context". Rename the named-operator-defaults concept to "profile" (AWS-CLI precedent: --profile, OMNIGRAPH_PROFILE, profiles:). Doc-only — the concept was unshipped. "scope" (the resolved addressing target a profile populates) is unchanged. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- docs/dev/index.md | 2 +- docs/dev/rfc-011-cli-refactoring.md | 72 ++++++++++++++--------------- 2 files changed, 37 insertions(+), 37 deletions(-) diff --git a/docs/dev/index.md b/docs/dev/index.md index c41853e..8f4cf86 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -80,7 +80,7 @@ Working documents for in-flight feature work. Removed when the work lands. | Deprecate `omnigraph.yaml` — one concern per config surface; key-by-key migration map and staged retirement | [rfc-008-deprecate-omnigraph-yaml.md](rfc-008-deprecate-omnigraph-yaml.md) | | Unify CLI embedded/remote access paths — parity referee, shared wire-DTO crate, `GraphClient` trait, declared plane capabilities | [rfc-009-unify-access-paths.md](rfc-009-unify-access-paths.md) | | Restructure the CLI around explicit planes — one graph-addressing model, declared capability surface, plane-grouped help (expands RFC-009 Phase 4) | [rfc-010-cli-planes-restructure.md](rfc-010-cli-planes-restructure.md) | -| CLI refactoring — one addressing & config model post-`omnigraph.yaml`: scope + `--graph` + derived access path, served-default / privileged-direct, contexts, named queries, capability classifier (completes RFC-008) | [rfc-011-cli-refactoring.md](rfc-011-cli-refactoring.md) | +| CLI refactoring — one addressing & config model post-`omnigraph.yaml`: scope + `--graph` + derived access path, served-default / privileged-direct, profiles, named queries, capability classifier (completes RFC-008) | [rfc-011-cli-refactoring.md](rfc-011-cli-refactoring.md) | ## Boundary diff --git a/docs/dev/rfc-011-cli-refactoring.md b/docs/dev/rfc-011-cli-refactoring.md index c067149..768509b 100644 --- a/docs/dev/rfc-011-cli-refactoring.md +++ b/docs/dev/rfc-011-cli-refactoring.md @@ -19,9 +19,9 @@ Refactor the CLI around one coherent model once `omnigraph.yaml` is gone. The shape: - **One ontology** (store, server, cluster; cluster config vs operator config; - catalog; context; capability) where each term names exactly one concept. + catalog; profile; capability) where each term names exactly one concept. - **Addressing = scope + `--graph`, with the access path *derived*.** A command - resolves a *scope* (operator defaults, an optional named *context*, or one + resolves a *scope* (operator defaults, an optional named *profile*, or one explicit primitive address — `--store` / `--server` / `--cluster`), selects a graph inside it with `--graph`, and the **served-vs-direct access path falls out of the scope's bindings × the verb's capability** — it is never a per-command @@ -30,7 +30,7 @@ shape: is a *server* (a bearer token, no bucket credentials). Reading or writing a remote store/cluster directly is an explicit, credentialed, admin/break-glass act — never the default, never baked into everyday operator config. -- **The CLI is stateless per command.** No `current_context` pointer, no +- **The CLI is stateless per command.** No `current_profile` pointer, no `USE`-style mode; every command is fully determined by its flags + static config. You *select* a graph, you do not *switch into* one. - **Definitions are named; payloads are passed.** Queries (`.gq`) and schema @@ -91,12 +91,12 @@ Every term is one concept. The rest of this RFC uses them precisely. catalog is the applied result.)* - **Operator config** — `~/.omnigraph/config.yaml`, your **personal** file: identity (actor), default graph, named servers/clusters, output prefs, optional - contexts. Declares *who I am*, never what the system is. -- **Context** — an optional named bundle of **defaults inside the operator + profiles. Declares *who I am*, never what the system is. +- **Profile** — an optional named bundle of **defaults inside the operator config** (one of {cluster, server, store} + a default graph). Config data, **not state**: selecting one fills in omitted flags for a command; it does not - put you "in" a mode. Chosen per command (`--context <name>`) or per shell - (`OMNIGRAPH_CONTEXT`). + put you "in" a mode. Chosen per command (`--profile <name>`) or per shell + (`OMNIGRAPH_PROFILE`). - **Credential** — a bearer token keyed to a **server name**, resolved via `OMNIGRAPH_TOKEN_<NAME>` or `~/.omnigraph/credentials` (`0600`); sent only to the server it is keyed to. (Per RFC-007 — the operator config holds endpoints, @@ -117,7 +117,7 @@ Every term is one concept. The rest of this RFC uses them precisely. ### How a command resolves - **Scope** — the resolved environment a command addresses: operator defaults, a - named context, or one explicit primitive address. + named profile, or one explicit primitive address. - **Access path** — **served** (through a server) or **direct** (open storage in-process). Derived from scope × capability; see "Access path" below. - **Capability** — what a verb requires: `any`, `served`, `direct`, `control`, @@ -132,12 +132,12 @@ Every term is one concept. The rest of this RFC uses them precisely. - **Exactly two config surfaces:** **cluster config** (team) and **operator config** (personal). Nothing else is "a config." -- A **context is not a third config** — it lives *inside* the operator config, and +- A **profile is not a third config** — it lives *inside* the operator config, and it is **defaults, not state**. - A **catalog is not config** — it is the *applied state* the cluster owns. - A **store is one graph; a cluster is many graphs** + catalog + control state. - A **graph is the logical thing**; store/server/cluster are ways to reach it. -- "State" elsewhere is not the context: *graph state* is committed data in Lance; +- "State" elsewhere is not the profile: *graph state* is committed data in Lance; *cluster state* is the applied control-plane ledger. Neither is operator config. ## Design @@ -153,7 +153,7 @@ Every command answers four orthogonal questions — kept orthogonal here: | Axis | Question | Today | Target | |---|---|---|---| -| Scope | which environment? | `omnigraph.yaml` defaults / `--target` | operator defaults · `--context` · one primitive | +| Scope | which environment? | `omnigraph.yaml` defaults / `--target` | operator defaults · `--profile` · one primitive | | Target shape | whole scope or one graph? | implicit in command family | declared per verb | | Graph | which graph in it? | tangled into the address | `--graph` only for graph-scoped server/cluster verbs | | Access path | served or direct? | inferred from scheme / target | **derived** from scope × capability | @@ -161,7 +161,7 @@ Every command answers four orthogonal questions — kept orthogonal here: ### A scope binds one entity — and served is the default -A scope (a context, the flat defaults, or one primitive flag) binds **exactly one +A scope (a profile, the flat defaults, or one primitive flag) binds **exactly one of** {server, cluster, store}. Server and cluster scopes may contain many graphs and can carry a `default_graph`; a store scope is already one graph and does not accept `--graph`. They differ by privilege, and **the everyday default is a @@ -259,7 +259,7 @@ The CLI validates through verb capability, not plane jargon: | `served` | requires an HTTP server; may be graph-scoped or scope-scoped | `graphs list`, `queries list` | | `direct` | graph-scoped storage-native or graph-backed validation; no server form exists | `init`, `optimize`, `repair`, `cleanup`, `schema plan`, graph-backed `lint` | | `control` | cluster-scoped catalog/control-plane work; addresses the cluster, not a single raw store | `cluster *`, `queries validate` | -| `local` | does not address a graph or scope | `config`, `context`, `lint --query ... --schema ...` | +| `local` | does not address a graph or scope | `config`, `profile`, `lint --query ... --schema ...` | `any` does **not** mean "the user picks": the resolver picks from the scope. Internally the exhaustive `command_plane` match (`planes.rs`) stays as the drift @@ -297,7 +297,7 @@ control/catalog check against the cluster-owned query definitions. A bare a **literal** — `--server https://…`, `--store s3://…` — bypasses it. Per Decision 2: a value containing `://` is a literal, otherwise a config-name lookup.)* -3. Else if `--context <name>` (or `OMNIGRAPH_CONTEXT`) selects a context, use it. +3. Else if `--profile <name>` (or `OMNIGRAPH_PROFILE`) selects a profile, use it. 4. Else use the operator config's flat defaults. Error only if neither resolves. *(No sticky "current" pointer — each command resolves scope fresh.)* 5. Resolve the graph only for **graph-scoped** verbs. Server/cluster scopes: @@ -342,7 +342,7 @@ defaults: servers: prod: { url: https://graph.example.com } # token keyed by name (RFC-007); no creds here staging: { url: https://staging.example.com } -contexts: # optional, only for multiple environments +profiles: # optional, only for multiple environments staging: { server: staging, default_graph: knowledge } ``` @@ -353,7 +353,7 @@ storage explicitly. **Maintainer — opts into a cluster root (and has bucket credentials):** ```yaml -contexts: +profiles: brain-admin: { cluster: brain, default_graph: knowledge } # direct; admin/control/maintenance clusters: brain: { root: s3://acme/clusters/brain } # the s3:// root lives ONLY here @@ -376,10 +376,10 @@ Assume the everyday flat defaults: server `prod`, default graph `knowledge`. | …with params | `omnigraph query find_people --params '{"title":"Eng"}'` | served | | Another graph in scope | `omnigraph query find_people --graph archive` | served | | Write | `omnigraph load --data batch.jsonl --mode append` | served | -| A different environment | `omnigraph --context staging query find_people` | served | +| A different environment | `omnigraph --profile staging query find_people` | served | | One-off server, no config | `omnigraph query find_people --server https://graph.example.com --graph knowledge` | served | | Maintain (admin, explicit storage) | `omnigraph optimize --cluster s3://acme/clusters/brain --graph knowledge` | direct (privileged) | -| Maintain (admin, via admin context) | `omnigraph --context brain-admin optimize --graph knowledge` | direct (privileged) | +| Maintain (admin, via admin profile) | `omnigraph --profile brain-admin optimize --graph knowledge` | direct (privileged) | | List catalog queries | `omnigraph queries list` | served | | Validate cluster query catalog | `omnigraph queries validate --cluster s3://acme/clusters/brain` | control (privileged) | | Offline query lint | `omnigraph lint --query new.gq --schema schema.pg` | local | @@ -407,7 +407,7 @@ files, `--cluster-graph`, scheme inference). **After** = this model. | Another graph | `omnigraph query --target archive --query find.gq --name find_people` | `omnigraph query find_people --graph archive` | | Load | `omnigraph load --data b.jsonl --mode append --target knowledge` | `omnigraph load --data b.jsonl --mode append` | | Maintain (admin) | `omnigraph optimize --cluster brain --cluster-graph knowledge` | `omnigraph optimize --cluster s3://acme/clusters/brain --graph knowledge` | -| Another environment | edit `omnigraph.yaml`, or re-address with full URIs | `--context staging …` or `OMNIGRAPH_CONTEXT=staging` | +| Another environment | edit `omnigraph.yaml`, or re-address with full URIs | `--profile staging …` or `OMNIGRAPH_PROFILE=staging` | | One-off remote | `omnigraph query --uri https://… --query find.gq` *(scheme→remote)* | `omnigraph query find_people --server https://… --graph knowledge` | | Raw storage of a served graph | `omnigraph query s3://…/knowledge.omni --query find.gq` *(looks like a normal query)* | `omnigraph query --file find.gq --store s3://…/knowledge.omni` *(explicit bypass)* | @@ -463,7 +463,7 @@ staged and release-noted. this RFC only makes the split explicit. - **No new transport.** Addressing surface, not protocol. - **No positional sigil grammar** (`@server/graph`, `%cluster/graph`). Considered - and rejected: explicit flags are more discoverable; contexts already give + and rejected: explicit flags are more discoverable; profiles already give brevity. Revisit only on demonstrated expert-terseness demand. ## Decisions @@ -472,7 +472,7 @@ The questions this RFC opened are resolved as follows. Two are explicitly deferred (see below); they do not block the model. 1. **Local-dev path → embedded `--store` scope.** Local dev runs the engine - in-process against a `--store <file>` (or a store-scoped context); `omnigraph + in-process against a `--store <file>` (or a store-scoped profile); `omnigraph serve` stays available but is not required. Consistent with embedded ≡ remote (RFC-009). 2. **Primitives are one flag, typed by content.** `--server` and `--cluster` @@ -487,11 +487,11 @@ deferred (see below); they do not block the model. 4. **Aliases live under an `alias` namespace** — `omnigraph alias <name> [args]`, never bare top-level. An alias can therefore neither shadow nor be shadowed by a built-in (current or future) verb. -6. **Context merge: scope wholesale, prefs layered.** The entity binding + - `default_graph` come *wholesale* from the active scope (a context, or flat +6. **Profile merge: scope wholesale, prefs layered.** The entity binding + + `default_graph` come *wholesale* from the active scope (a profile, or flat defaults if none) — never per-key merged across the entity dimension (that would yield "server *and* cluster"). Only non-scope preferences (`output`, table - layout) take flat defaults as a base. Precedence: explicit flag > context > flat + layout) take flat defaults as a base. Precedence: explicit flag > profile > flat defaults. 7. **No default graph → error + list candidates.** A graph-scoped verb with no `--graph`, no `default_graph`, and >1 graph in scope errors and lists candidates @@ -524,19 +524,19 @@ Non-blocking; settle when convenient. - **D5 — combined admin scope.** A scope binds one entity; admins read via a server scope and maintain via `--cluster`. A `deployments: { … }` object - (server + cluster validated coherent, referenced by a context) is revisited only + (server + cluster validated coherent, referenced by a profile) is revisited only if admin ergonomics demand it — and Decision 11 largely removes the need. -- **D8 — the `context` command surface.** `context list` / `context show` +- **D8 — the `profile` command surface.** `profile list` / `profile show` (read-only inspection) are additive diagnostics, shippable anytime; they don't - touch the grammar or resolution. The *no sticky `context use`* constraint holds + touch the grammar or resolution. The *no sticky `profile use`* constraint holds regardless — it is a design principle, not a command. ## Safety -Dropping the sticky `current_context` pointer removes the main footgun — a +Dropping the sticky `current_profile` pointer removes the main footgun — a destructive command silently inheriting a "current" environment from an earlier session. Because each command resolves scope fresh, what is on the command line is -what runs. Two guards remain (a flat default or `OMNIGRAPH_CONTEXT` can still point +what runs. Two guards remain (a flat default or `OMNIGRAPH_PROFILE` can still point at prod): echo the resolved scope + access path on writes, and require confirmation (or `--yes`) for destructive verbs when the resolved scope is not local (Decision 9). The most dangerous direct writes (`cleanup`, overwrite @@ -557,7 +557,7 @@ normal operator's setup mostly cannot issue them by accident at all. default — narrowing the blast radius of a leaked operator config. - **§6 strong consistency:** both paths are snapshot-isolated per query; this RFC changes addressing, not isolation. -- **Deny-list (no state that drifts):** contexts and aliases are static config +- **Deny-list (no state that drifts):** profiles and aliases are static config sugar that resolve to canonical scopes; they declare nothing the cluster or server doesn't already own. No sticky session state is introduced. - No Hard Invariant is weakened; the change is CLI surface + config removal. @@ -612,7 +612,7 @@ omnigraph │ └─ local — no graph ├─ policy { validate | test | explain } offline Cedar tooling - ├─ context { list | show } read-only; NO mutating `use` (no sticky state) + ├─ profile { list | show } read-only; NO mutating `use` (no sticky state) ├─ alias <name> [args] personal shortcut; expands to its bound stored-query call (D4) ├─ config { migrate } finish the omnigraph.yaml split (RFC-008) ├─ login / logout per-server bearer credentials @@ -631,13 +631,13 @@ no `--cluster-graph`, no `--uri` scheme-dispatch, no `--via`. | Form | Resolves to | Access | Privilege | |---|---|---|---| -| **server scope** — operator default, a `--context`, or `--server <url\|name>` | a served endpoint + keyed token | served | everyday (bearer token) | -| **cluster scope** — an admin context, or `--cluster <root>` | a managed cluster's storage + catalog | direct | privileged (bucket creds) | +| **server scope** — operator default, a `--profile`, or `--server <url\|name>` | a served endpoint + keyed token | served | everyday (bearer token) | +| **cluster scope** — an admin profile, or `--cluster <root>` | a managed cluster's storage + catalog | direct | privileged (bucket creds) | | **store scope** — `--store <uri>` | one graph's storage (no catalog) | direct | local-dev (file) / break-glass (s3) | | **`--graph <id>`** | selects the graph for graph-scoped verbs in server/cluster scopes; invalid for store scopes and scope-scoped verbs | — | — | -Resolution: explicit primitive (`--server`/`--cluster`/`--store`) → `--context` / -`OMNIGRAPH_CONTEXT` → operator flat defaults. Access path is then derived from the +Resolution: explicit primitive (`--server`/`--cluster`/`--store`) → `--profile` / +`OMNIGRAPH_PROFILE` → operator flat defaults. Access path is then derived from the scope kind × the verb's capability (see the Resolution rule); it is never inferred from a URI scheme and never toggled. @@ -653,7 +653,7 @@ from a URI scheme and never toggled. | `cluster *` | Control | **control** (unchanged) | | `policy *`/`embed`/`login`/`logout`/`config`/`version`/offline `lint --query --schema` | Session | **`local`** | | `ingest`; `--target`; `--cluster-graph`; `--uri http` dispatch | present | **removed** | -| — | — | **added:** `context { list | show }` (read-only) | +| — | — | **added:** `profile { list | show }` (read-only) | Cross-capability families: `schema` (`plan` is `direct`, `show`/`apply` are `any`), `queries` (`list` is `served`, `validate` is `control`), and `lint` From 7963499995a0f97b26af0df81b5349dc9b142631 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford <ragnor.comerford@gmail.com> Date: Sun, 14 Jun 2026 20:37:12 +0200 Subject: [PATCH 162/165] fix(cli): unify remote URL builder, fix branch delete //branches 404 (#230) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * test(cli): reproduce branch-delete //branches 404 (failing) Regression test for the `branch delete` 404 over a multi-graph `--server`/`--graph` target: the composed URL must be `<base>/branches/<name>` with no empty `//` segment. Fails against the current `remote_branch_url`, which appends a trailing slash before extending path segments and so emits `…/graphs/p9-os//branches/tmpbranch`. The next commit fixes it. left: "http://host/graphs/p9-os//branches/tmpbranch" right: "http://host/graphs/p9-os/branches/tmpbranch" * fix(cli): unify remote URL builder, close the //branches 404 class Correct-by-design fix for the failing test in the previous commit. The bug was not specific to `branch delete`: URL assembly was scattered across a string-concat `remote_url`, a url-crate `remote_branch_url`, and several `format!` interpolations that left dynamic path/query components un-encoded (commit id in the path, branch in the query string). `branch delete` was the instance that surfaced because it is the only verb that puts a dynamic value in the path. Replace both helpers with one `remote_url(base, segments, query)` that every remote call routes through. Callers pass structured segments and query pairs, so trailing-slash normalization (pop_if_empty) and per-segment / per-value percent-encoding live in one place. A stray `//` or an un-encoded dynamic component is no longer representable, closing the whole class rather than the reported instance. Migrates the previous commit's failing test to the new builder and adds the single-graph, trailing-slash, slash-in-name, commit-id-path, and query-value cases (the last two cover the previously latent siblings). All 16 callsites migrated; `remote_branch_url` removed. --- crates/omnigraph-cli/src/client.rs | 34 ++++----- crates/omnigraph-cli/src/helpers.rs | 110 ++++++++++++++++++++++++++-- 2 files changed, 119 insertions(+), 25 deletions(-) diff --git a/crates/omnigraph-cli/src/client.rs b/crates/omnigraph-cli/src/client.rs index d9e7726..4faaa11 100644 --- a/crates/omnigraph-cli/src/client.rs +++ b/crates/omnigraph-cli/src/client.rs @@ -40,7 +40,7 @@ use serde_json::Value; use crate::cli::CliLoadMode; use crate::helpers::{ ResolvedCliGraph, apply_bearer_token, apply_server_flag, build_http_client, is_remote_uri, - legacy_change_request_body, open_local_db_with_policy, query_params_from_json, remote_branch_url, + legacy_change_request_body, open_local_db_with_policy, query_params_from_json, remote_json, remote_url, resolve_cli_actor, resolve_cli_graph, resolve_remote_bearer_token, select_named_query, }; @@ -173,7 +173,7 @@ impl GraphClient { remote_json( http, Method::GET, - remote_url(base_url, "/branches"), + remote_url(base_url, &["branches"], &[])?, None, token.as_deref(), ) @@ -198,7 +198,7 @@ impl GraphClient { remote_json( http, Method::GET, - format!("{}?branch={}", remote_url(base_url, "/snapshot"), branch), + remote_url(base_url, &["snapshot"], &[("branch", branch)])?, None, token.as_deref(), ) @@ -222,7 +222,7 @@ impl GraphClient { remote_json( http, Method::GET, - remote_url(base_url, "/schema"), + remote_url(base_url, &["schema"], &[])?, None, token.as_deref(), ) @@ -245,8 +245,8 @@ impl GraphClient { token, } => { let url = match branch { - Some(branch) => format!("{}?branch={}", remote_url(base_url, "/commits"), branch), - None => remote_url(base_url, "/commits"), + Some(branch) => remote_url(base_url, &["commits"], &[("branch", branch)])?, + None => remote_url(base_url, &["commits"], &[])?, }; remote_json(http, Method::GET, url, None, token.as_deref()).await } @@ -273,7 +273,7 @@ impl GraphClient { remote_json( http, Method::GET, - remote_url(base_url, &format!("/commits/{commit_id}")), + remote_url(base_url, &["commits", commit_id], &[])?, None, token.as_deref(), ) @@ -310,7 +310,7 @@ impl GraphClient { let output = remote_json::<IngestOutput>( http, Method::POST, - remote_url(base_url, "/load"), + remote_url(base_url, &["load"], &[])?, Some(serde_json::to_value(IngestRequest { branch: Some(branch.to_string()), from: from.map(ToOwned::to_owned), @@ -354,7 +354,7 @@ impl GraphClient { remote_json( http, Method::POST, - remote_url(base_url, "/ingest"), + remote_url(base_url, &["ingest"], &[])?, Some(serde_json::to_value(IngestRequest { branch: Some(branch.to_string()), from: Some(from.to_string()), @@ -393,7 +393,7 @@ impl GraphClient { remote_json( http, Method::POST, - remote_url(base_url, "/change"), + remote_url(base_url, &["change"], &[])?, Some(legacy_change_request_body( query_source, query_name, @@ -446,7 +446,7 @@ impl GraphClient { remote_json( http, Method::POST, - remote_url(base_url, "/read"), + remote_url(base_url, &["read"], &[])?, Some(serde_json::to_value(ReadRequest { query_source: query_source.to_string(), query_name: query_name.map(ToOwned::to_owned), @@ -484,7 +484,7 @@ impl GraphClient { remote_json( http, Method::POST, - remote_url(base_url, "/branches"), + remote_url(base_url, &["branches"], &[])?, Some(serde_json::to_value(BranchCreateRequest { from: Some(from.to_string()), name: name.to_string(), @@ -518,7 +518,7 @@ impl GraphClient { remote_json( http, Method::DELETE, - remote_branch_url(base_url, name)?, + remote_url(base_url, &["branches", name], &[])?, None, token.as_deref(), ) @@ -547,7 +547,7 @@ impl GraphClient { remote_json( http, Method::POST, - remote_url(base_url, "/branches/merge"), + remote_url(base_url, &["branches", "merge"], &[])?, Some(serde_json::to_value(BranchMergeRequest { source: source.to_string(), target: Some(into.to_string()), @@ -598,7 +598,7 @@ impl GraphClient { remote_json::<SchemaApplyOutput>( http, Method::POST, - remote_url(base_url, "/schema/apply"), + remote_url(base_url, &["schema", "apply"], &[])?, Some(serde_json::to_value(SchemaApplyRequest { schema_source: schema_source.to_string(), allow_data_loss, @@ -642,7 +642,7 @@ impl GraphClient { token, } => { let request = apply_bearer_token( - http.request(Method::POST, remote_url(base_url, "/export")), + http.request(Method::POST, remote_url(base_url, &["export"], &[])?), token.as_deref(), ) .json(&ExportRequest { @@ -690,7 +690,7 @@ impl GraphClient { remote_json( http, Method::GET, - remote_url(base_url, "/graphs"), + remote_url(base_url, &["graphs"], &[])?, None, token.as_deref(), ) diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index 7e1ca15..6a381b1 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -16,15 +16,41 @@ pub(crate) fn is_remote_uri(uri: &str) -> bool { uri.starts_with("http://") || uri.starts_with("https://") } -pub(crate) fn remote_url(base: &str, path: &str) -> String { - format!("{}{}", base.trim_end_matches('/'), path) -} - -pub(crate) fn remote_branch_url(base: &str, branch: &str) -> Result<String> { - let mut url = reqwest::Url::parse(&format!("{}/", base.trim_end_matches('/')))?; +/// THE one way the CLI composes a remote request URL. Every remote call +/// routes through here so URL assembly has a single mechanism instead of +/// per-callsite string interpolation. +/// +/// - `base` is the resolved server root (single-graph) or `…/graphs/{id}` +/// (multi-graph). +/// - `segments` are appended as individual percent-encoded path segments, so +/// a dynamic component (branch name, commit id, query name) is always one +/// safe segment — e.g. a branch `etl/zendesk/run-1` becomes `%2F`-escaped. +/// - `query` pairs are percent-encoded values. +/// +/// Trailing-slash normalization happens exactly once via `pop_if_empty`: +/// `Url::parse` normalizes a path-less base (`http://host`) to a single empty +/// trailing segment, and a `…/graphs/{id}/` base keeps its own. `extend` +/// appends *after* the last segment, so without dropping a trailing empty one +/// the join emits `…/graphs/{id}//branches/{name}` — the empty `//` segment +/// misses the route and 404s. Because callers pass structured segments rather +/// than a pre-joined string, neither a stray `//` nor an un-encoded dynamic +/// component is representable here. +pub(crate) fn remote_url( + base: &str, + segments: &[&str], + query: &[(&str, &str)], +) -> Result<String> { + let mut url = reqwest::Url::parse(base.trim_end_matches('/'))?; url.path_segments_mut() .map_err(|_| color_eyre::eyre::eyre!("invalid remote base url"))? - .extend(["branches", branch]); + .pop_if_empty() + .extend(segments); + if !query.is_empty() { + let mut pairs = url.query_pairs_mut(); + for (key, value) in query { + pairs.append_pair(key, value); + } + } Ok(url.to_string()) } @@ -340,7 +366,7 @@ pub(crate) async fn execute_operator_alias( remote_json( client, Method::POST, - remote_url(&uri, &format!("/queries/{}", alias.query)), + remote_url(&uri, &["queries", &alias.query], &[])?, body, bearer_token.as_deref(), ) @@ -1059,3 +1085,71 @@ pub(crate) fn rewrite_deprecated_argv(args: Vec<OsString>) -> Vec<OsString> { } args } + +#[cfg(test)] +mod tests { + use super::*; + + // `branch delete` interpolates the branch into the URL path. The composed + // path must be exactly `<base-path>/branches/<name>` with no empty `//` + // segment — an empty segment misses the + // `/graphs/{graph_id}/branches/{branch}` route and 404s. + #[test] + fn remote_url_multi_graph_base_has_no_double_slash() { + let url = remote_url("http://host/graphs/p9-os", &["branches", "tmpbranch"], &[]).unwrap(); + assert_eq!(url, "http://host/graphs/p9-os/branches/tmpbranch"); + assert!( + !url.contains("//branches"), + "double slash before branches: {url}" + ); + } + + #[test] + fn remote_url_single_graph_base_has_no_double_slash() { + let url = remote_url("http://host", &["branches", "tmpbranch"], &[]).unwrap(); + assert_eq!(url, "http://host/branches/tmpbranch"); + } + + #[test] + fn remote_url_tolerates_trailing_slash_on_base() { + let url = remote_url("http://host/graphs/p9-os/", &["branches", "tmpbranch"], &[]).unwrap(); + assert_eq!(url, "http://host/graphs/p9-os/branches/tmpbranch"); + } + + #[test] + fn remote_url_encodes_slashes_in_path_segment() { + let url = remote_url( + "http://host/graphs/p9-os", + &["branches", "etl/zendesk/run-1"], + &[], + ) + .unwrap(); + assert_eq!( + url, + "http://host/graphs/p9-os/branches/etl%2Fzendesk%2Frun-1" + ); + } + + // Sibling cases the unified builder closes by construction: a dynamic + // commit id in the path, and a branch name carried as a query value, are + // both percent-encoded instead of interpolated raw. + #[test] + fn remote_url_encodes_dynamic_path_segment_for_commits() { + let url = remote_url("http://host/graphs/p9-os", &["commits", "a/b c"], &[]).unwrap(); + assert_eq!(url, "http://host/graphs/p9-os/commits/a%2Fb%20c"); + } + + #[test] + fn remote_url_encodes_query_values() { + let url = remote_url( + "http://host/graphs/p9-os", + &["snapshot"], + &[("branch", "feature&x=1")], + ) + .unwrap(); + assert_eq!( + url, + "http://host/graphs/p9-os/snapshot?branch=feature%26x%3D1" + ); + } +} From 67baf615d99a9122c69f579037a3acda4f162237 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford <ragnor.comerford@gmail.com> Date: Sun, 14 Jun 2026 20:42:24 +0200 Subject: [PATCH 163/165] =?UTF-8?q?build(deps):=20bump=20Lance=206.0.1=20?= =?UTF-8?q?=E2=86=92=207.0.0=20(correct-by-design=20substrate=20alignment)?= =?UTF-8?q?=20(#229)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * build(deps): bump Lance 6.0.1 → 7.0.0 (object_store 0.13.2, roaring 0.11.4) Arrow stays 58 and DataFusion stays 53 (no change). The only transitive bump is object_store 0.12.5 → 0.13.2. 141 upstream commits reviewed; no fixes lost (the 6.0.x release-branch backports are all forward-ported into 7.0.0). - object_store 0.13 moved get/put/head/rename/delete behind a new ObjectStoreExt trait (list/list_with_delimiter/put_opts stay on the core trait). Add `use object_store::ObjectStoreExt` in storage.rs and db/manifest/namespace.rs; no call-site changes. Mirrors Lance's own migration in PR #6672. - roaring pinned to 0.11.4 (cargo update -p roaring --precise 0.11.4). Lance 7.0.0's UpdatedFragmentOffsets newtype (lance#6650) derives Eq over HashMap<u64, RoaringBitmap>, which needs RoaringBitmap: Eq, added in roaring 0.11.4; the loose `roaring = "0.11"` constraint otherwise resolves 0.11.3 and lance itself fails to compile. - lance#6774: merge-insert INSERT rows now stamp _row_created_at_version with the commit version (was a fallback of 1). Flip the lance_version_columns assertion to `== v2` and correct the changes/mod.rs rationale comment. Production change-detection keys on _row_last_updated_at_version + ID membership, so its logic is unaffected. Refs lance#6650, lance#6774, lance#6672. * fix(storage): pin WriteParams::auto_cleanup = None (lance#6755 default flip) lance#6755 flipped the WriteParams::auto_cleanup default from on (a full cleanup pass every 20th commit) to None. On 6.0.1 the on-by-default hook could silently GC versions that __manifest pins for snapshots/time-travel. OmniGraph owns cleanup explicitly (optimize.rs::cleanup_all_tables) and never set auto_cleanup, so it was relying on a default that is both wrong for our snapshot model and now changed upstream. Pin auto_cleanup: None explicitly at all 11 production WriteParams sites (table_store ×6, commit_graph ×2, recovery_audit ×1, manifest/graph ×2 — the __manifest + sub-table Create paths). Removes the dependency on a default-flag value and locks in the snapshot-safe behavior regardless of future upstream re-flips. Refs lance#6755. * test(lance): pin BTREE range-boundary correctness (lance#6796) lance#6796 (issue #6792) fixed a BTREE scalar-index range-query bound inclusiveness bug: `x <= hi AND x > lo` returned the wrong boundary row. Add lance_surface_guards.rs::btree_range_query_boundary_is_correct, which reproduces the exact #6792 shape (5 rows + an explicit BTREE drives the index path even on tiny data) and pins the corrected inclusive-<= / exclusive-> semantics. It turns red if a future Lance regression reintroduces the bug. OmniGraph today builds BTREE only on string @key columns and queries them by equality/IN, so its current patterns do not hit this; the guard protects any future BTREE-range path (BTREE-on-properties, range-on-key). Refs lance#6796. * docs(dev): align Lance docs + invariants to 7.0.0 - docs/dev/lance.md: new 2026-06-14 alignment stanza for the 6.0.1 → 7.0.0 bump (object_store ObjectStoreExt move, roaring 0.11.4, #6774/#6796/#6755 behavior, #6658 shipped → MR-A unblocked but separate, #6666 + blob compaction still open); prior 6.0.1 stanza demoted to historical. - AGENTS.md: storage substrate 6.x → 7.x (line + architecture diagram). - docs/dev/invariants.md: deletes/vector known gap updated — the staged two-phase delete API (lance#6658) now exists and MR-A is unblocked, but delete_where stays inline and D2 stays in place until the migration lands; create_vector_index still gated on lance#6666. * fix(storage): skip Lance auto-cleanup on commit paths for legacy datasets Addresses PR #229 review (Codex P1). `WriteParams::auto_cleanup` is create-time config with no effect on existing datasets (Lance write.rs docs), so the previous `auto_cleanup: None` change alone did NOT protect graphs created before the v7 bump: 6.0.1 defaulted auto_cleanup ON, leaving `lance.auto_cleanup.*` config on those datasets, and Lance's per-commit hook (io/commit.rs: `if !commit_config.skip_auto_cleanup`) fires off that stored config — so omnigraph's own writes would GC versions the __manifest pins for snapshots/time-travel. Skip the hook on every commit path, covering new and legacy datasets alike: - commit_staged: CommitBuilder::with_skip_auto_cleanup(true) — the staged data path. - __manifest publisher: MergeInsertBuilder::skip_auto_cleanup(true). - all 11 WriteParams: skip_auto_cleanup: true (direct Dataset::write/append paths; auto_cleanup: None retained so new datasets store no cleanup config at all). Tests: - lance_surface_guards::skip_auto_cleanup_suppresses_version_gc — substrate: negative control (config GCs v1 without skip) + with-skip survival. - staged_writes::commit_staged_skips_auto_cleanup_so_pinned_versions_survive — omnigraph usage: commit_staged on a legacy-config dataset preserves the pinned create version. Refs lance#6755. * test(lance): assert created_at-preserved + updated_at-bumped on merge_insert UPDATE Addresses PR #229 review follow-up. `lance_merge_insert_update_preserves_created_at_version` documented (in a comment) that a merge_insert UPDATE preserves created_at and bumps updated_at, but only asserted the value change — leaving the change-feed invariant unguarded. Add the two missing assertions: - bob created_at == v1 (preserved across UPDATE; what the test name promises; lance#6774 only changed INSERT-row stamping). - bob updated_at == v2 (bumped to the commit version) — the invariant OmniGraph's insert/update classification relies on (changes/mod.rs keys on _row_last_updated_at_version). A regression here would silently drop updates from the diff/change feed. --- AGENTS.md | 4 +- Cargo.lock | 1613 ++++++++++++++--- Cargo.toml | 18 +- crates/omnigraph/src/changes/mod.rs | 12 +- crates/omnigraph/src/db/commit_graph.rs | 4 + crates/omnigraph/src/db/manifest/graph.rs | 4 + crates/omnigraph/src/db/manifest/namespace.rs | 4 +- crates/omnigraph/src/db/manifest/publisher.rs | 6 + crates/omnigraph/src/db/recovery_audit.rs | 2 + crates/omnigraph/src/storage.rs | 2 +- crates/omnigraph/src/table_store.rs | 21 + .../omnigraph/tests/lance_surface_guards.rs | 182 +- .../omnigraph/tests/lance_version_columns.rs | 33 +- crates/omnigraph/tests/staged_writes.rs | 51 + docs/dev/invariants.md | 17 +- docs/dev/lance.md | 19 +- 16 files changed, 1708 insertions(+), 284 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 427d976..7e42a2a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -18,7 +18,7 @@ Tools that support `@`-imports (Claude Code) auto-include all three files via th **Version surveyed:** 0.7.0 **Workspace crates:** `omnigraph-compiler`, `omnigraph` (engine), `omnigraph-policy`, `omnigraph-api-types` (shared HTTP wire DTOs), `omnigraph-cluster`, `omnigraph-cli`, `omnigraph-server` -**Storage substrate:** Lance 6.x (columnar, versioned, branchable) +**Storage substrate:** Lance 7.x (columnar, versioned, branchable) **License:** MIT **Toolchain:** Rust stable, edition 2024 @@ -53,7 +53,7 @@ CLI (omnigraph) HTTP Server (omnigraph-server, Axum) omnigraph (engine) ── ManifestCoordinator, CommitGraph, RunRegistry, GraphIndex (CSR/CSC), exec │ ▼ - Lance 6.x ── columnar Arrow, fragments, per-dataset versions/branches, indexes + Lance 7.x ── columnar Arrow, fragments, per-dataset versions/branches, indexes │ ▼ Object store (file / s3 / RustFS / MinIO / S3-compat) diff --git a/Cargo.lock b/Cargo.lock index 21403b0..33dd652 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -23,9 +23,9 @@ version = "0.8.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b169f7a6d4742236a0a00c541b845991d0ac43e546831af1249753ab4c3aa3a0" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "cipher", - "cpufeatures", + "cpufeatures 0.2.17", ] [[package]] @@ -34,7 +34,7 @@ version = "0.8.12" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "const-random", "getrandom 0.3.4", "once_cell", @@ -137,6 +137,15 @@ version = "1.0.102" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c" +[[package]] +name = "approx" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cab112f0a86d568ea0e627cc1d6be74a1e9cd55214684db5561995f6dad897c6" +dependencies = [ + "num-traits", +] + [[package]] name = "ar_archive_writer" version = "0.5.1" @@ -708,11 +717,11 @@ dependencies = [ "bytes", "form_urlencoded", "hex", - "hmac", + "hmac 0.12.1", "http 0.2.12", "http 1.4.0", "percent-encoding", - "sha2", + "sha2 0.10.9", "time", "tracing", ] @@ -980,7 +989,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "bb531853791a215d7c62a30daf0dde835f381ab5de4589cfe7c649d2cbe92bd6" dependencies = [ "addr2line", - "cfg-if", + "cfg-if 1.0.4", "libc", "miniz_oxide", "object", @@ -1071,7 +1080,7 @@ version = "0.10.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "46502ad458c9a52b69d4d4d32775c788b7a1b85e8bc9d482d92250fc0e3f8efe" dependencies = [ - "digest", + "digest 0.10.7", ] [[package]] @@ -1083,9 +1092,9 @@ dependencies = [ "arrayref", "arrayvec 0.7.6", "cc", - "cfg-if", + "cfg-if 1.0.4", "constant_time_eq", - "cpufeatures", + "cpufeatures 0.2.17", ] [[package]] @@ -1097,6 +1106,15 @@ dependencies = [ "generic-array", ] +[[package]] +name = "block-buffer" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa" +dependencies = [ + "hybrid-array", +] + [[package]] name = "block-padding" version = "0.3.3" @@ -1266,6 +1284,12 @@ dependencies = [ "smol_str", ] +[[package]] +name = "cfg-if" +version = "0.1.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822" + [[package]] name = "cfg-if" version = "1.0.4" @@ -1278,6 +1302,17 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724" +[[package]] +name = "chacha20" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6f8d983286843e49675a4b7a2d174efe136dc93a18d69130dd18198a6c167601" +dependencies = [ + "cfg-if 1.0.4", + "cpufeatures 0.3.0", + "rand_core 0.10.1", +] + [[package]] name = "chrono" version = "0.4.44" @@ -1308,7 +1343,7 @@ version = "0.4.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad" dependencies = [ - "crypto-common", + "crypto-common 0.1.7", "inout", ] @@ -1361,6 +1396,12 @@ dependencies = [ "cc", ] +[[package]] +name = "cmov" +version = "0.5.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a" + [[package]] name = "color-eyre" version = "0.6.5" @@ -1394,6 +1435,25 @@ version = "1.0.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b05b61dc5112cbb17e4b6cd61790d9845d13888356391624cbe7e41efeac1e75" +[[package]] +name = "colored" +version = "3.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "combine" +version = "4.6.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ba5a308b75df32fe02788e748662718f03fde005016435c444eea572398219fd" +dependencies = [ + "bytes", + "memchr", +] + [[package]] name = "comfy-table" version = "7.2.2" @@ -1436,6 +1496,12 @@ version = "0.9.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" +[[package]] +name = "const-oid" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c" + [[package]] name = "const-random" version = "0.1.18" @@ -1456,12 +1522,37 @@ dependencies = [ "tiny-keccak", ] +[[package]] +name = "const-str" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "18f12cc9948ed9604230cdddc7c86e270f9401ccbe3c2e98a4378c5e7632212f" + +[[package]] +name = "const_panic" +version = "0.2.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e262cdaac42494e3ae34c43969f9cdeb7da178bdb4b66fa6a1ea2edb4c8ae652" +dependencies = [ + "typewit", +] + [[package]] name = "constant_time_eq" version = "0.4.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b" +[[package]] +name = "core-foundation" +version = "0.9.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f" +dependencies = [ + "core-foundation-sys", + "libc", +] + [[package]] name = "core-foundation" version = "0.10.1" @@ -1478,6 +1569,15 @@ version = "0.8.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" +[[package]] +name = "countio" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9702aee5d1d744c01d82f6915644f950f898e014903385464c773b96fefdecb" +dependencies = [ + "futures-io", +] + [[package]] name = "cpufeatures" version = "0.2.17" @@ -1487,6 +1587,15 @@ dependencies = [ "libc", ] +[[package]] +name = "cpufeatures" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b2a41393f66f16b0823bb79094d54ac5fbd34ab292ddafb9a0456ac9f87d201" +dependencies = [ + "libc", +] + [[package]] name = "crc32c" version = "0.6.8" @@ -1502,7 +1611,7 @@ version = "1.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9481c1c90cbf2ac953f07c8d4a58aa3945c425b7185c9154d67a65e4230da511" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", ] [[package]] @@ -1574,6 +1683,15 @@ dependencies = [ "typenum", ] +[[package]] +name = "crypto-common" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453" +dependencies = [ + "hybrid-array", +] + [[package]] name = "csv" version = "1.4.0" @@ -1595,6 +1713,31 @@ dependencies = [ "memchr", ] +[[package]] +name = "ctor" +version = "0.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "424e0138278faeb2b401f174ad17e715c829512d74f3d1e81eb43365c2e0590e" +dependencies = [ + "ctor-proc-macro", + "dtor", +] + +[[package]] +name = "ctor-proc-macro" +version = "0.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52560adf09603e58c9a7ee1fe1dcb95a16927b17c127f0ac02d6e768a0e25bc1" + +[[package]] +name = "ctutils" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7d5515a3834141de9eafb9717ad39eea8247b5674e6066c404e8c4b365d2a29e" +dependencies = [ + "cmov", +] + [[package]] name = "darling" version = "0.23.0" @@ -1635,7 +1778,7 @@ version = "6.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5041cc499144891f3790297212f32a74fb938e5136a14943f338ef9e0ae276cf" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "crossbeam-utils", "hashbrown 0.14.5", "lock_api", @@ -1681,7 +1824,7 @@ dependencies = [ "futures", "itertools 0.14.0", "log", - "object_store 0.13.2", + "object_store", "parking_lot", "rand 0.9.2", "regex", @@ -1712,7 +1855,7 @@ dependencies = [ "futures", "itertools 0.14.0", "log", - "object_store 0.13.2", + "object_store", "parking_lot", "tokio", ] @@ -1737,7 +1880,7 @@ dependencies = [ "futures", "itertools 0.14.0", "log", - "object_store 0.13.2", + "object_store", ] [[package]] @@ -1756,7 +1899,7 @@ dependencies = [ "itertools 0.14.0", "libc", "log", - "object_store 0.13.2", + "object_store", "paste", "sqlparser", "tokio", @@ -1797,7 +1940,7 @@ dependencies = [ "glob", "itertools 0.14.0", "log", - "object_store 0.13.2", + "object_store", "rand 0.9.2", "tokio", "url", @@ -1823,7 +1966,7 @@ dependencies = [ "datafusion-session", "futures", "itertools 0.14.0", - "object_store 0.13.2", + "object_store", "tokio", ] @@ -1845,7 +1988,7 @@ dependencies = [ "datafusion-physical-plan", "datafusion-session", "futures", - "object_store 0.13.2", + "object_store", "regex", "tokio", ] @@ -1868,7 +2011,7 @@ dependencies = [ "datafusion-physical-plan", "datafusion-session", "futures", - "object_store 0.13.2", + "object_store", "serde_json", "tokio", "tokio-stream", @@ -1896,7 +2039,7 @@ dependencies = [ "datafusion-physical-expr-common", "futures", "log", - "object_store 0.13.2", + "object_store", "parking_lot", "rand 0.9.2", "tempfile", @@ -1965,7 +2108,7 @@ dependencies = [ "num-traits", "rand 0.9.2", "regex", - "sha2", + "sha2 0.10.9", "unicode-segmentation", "uuid", ] @@ -2284,7 +2427,7 @@ version = "0.7.10" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" dependencies = [ - "const-oid", + "const-oid 0.9.6", "pem-rfc7468", "zeroize", ] @@ -2311,12 +2454,24 @@ version = "0.10.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" dependencies = [ - "block-buffer", - "const-oid", - "crypto-common", + "block-buffer 0.10.4", + "const-oid 0.9.6", + "crypto-common 0.1.7", "subtle", ] +[[package]] +name = "digest" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" +dependencies = [ + "block-buffer 0.12.1", + "const-oid 0.10.2", + "crypto-common 0.2.2", + "ctutils", +] + [[package]] name = "dirs" version = "6.0.0" @@ -2358,6 +2513,21 @@ dependencies = [ "const-random", ] +[[package]] +name = "dtor" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "404d02eeb088a82cfd873006cb713fe411306c7d182c344905e101fb1167d301" +dependencies = [ + "dtor-proc-macro", +] + +[[package]] +name = "dtor-proc-macro" +version = "0.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f678cf4a922c215c63e0de95eb1ff08a958a81d47e485cf9da1e27bf6305cfa5" + [[package]] name = "dunce" version = "1.0.5" @@ -2403,7 +2573,7 @@ version = "0.8.35" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "75030f3c4f45dafd7586dd6780965a8c7e8e285a5ecb86713e63a79c5b2766f3" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", ] [[package]] @@ -2578,9 +2748,9 @@ checksum = "42703706b716c37f96a77aea830392ad231f44c9e9a67872fa5548707e11b11c" [[package]] name = "fsst" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "83cf860f6a6bf0a6a60fdfe5a36c75121fad5ea4332d1d12deee3e65b6047727" +checksum = "bcd0ce0249ac12fd44fcde62d435c36d881952c2f0df4d1de24b45e1dbba5ddb" dependencies = [ "arrow-array", "rand 0.9.2", @@ -2689,6 +2859,15 @@ dependencies = [ "slab", ] +[[package]] +name = "gearhash" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c8cf82cf76cd16485e56295a1377c775ce708c9f1a0be6b029076d60a245d213" +dependencies = [ + "cfg-if 0.1.10", +] + [[package]] name = "generator" version = "0.8.8" @@ -2696,7 +2875,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "52f04ae4152da20c76fe800fa48659201d5cf627c5149ca0b707b69d7eef6cf9" dependencies = [ "cc", - "cfg-if", + "cfg-if 1.0.4", "libc", "log", "rustversion", @@ -2720,10 +2899,10 @@ version = "0.2.17" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "js-sys", "libc", - "wasi", + "wasi 0.11.1+wasi-snapshot-preview1", "wasm-bindgen", ] @@ -2733,7 +2912,7 @@ version = "0.3.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "js-sys", "libc", "r-efi 5.3.0", @@ -2747,11 +2926,14 @@ version = "0.4.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", + "js-sys", "libc", "r-efi 6.0.0", + "rand_core 0.10.1", "wasip2", "wasip3", + "wasm-bindgen", ] [[package]] @@ -2760,6 +2942,26 @@ version = "0.32.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e629b9b98ef3dd8afe6ca2bd0f89306cec16d43d907889945bc5d6687f2f13c7" +[[package]] +name = "git-version" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ad568aa3db0fcbc81f2f116137f263d7304f512a1209b35b85150d3ef88ad19" +dependencies = [ + "git-version-macro", +] + +[[package]] +name = "git-version-macro" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53010ccb100b96a67bc32c0175f0ed1426b31b655d562898e57325f81c023ac0" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + [[package]] name = "glob" version = "0.3.3" @@ -2822,7 +3024,7 @@ version = "2.7.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6ea2d84b969582b4b1864a92dc5d27cd2b77b622a8d79306834f1be5ba20d84b" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "crunchy", "num-traits", "zerocopy", @@ -2866,6 +3068,12 @@ version = "0.17.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" +[[package]] +name = "heapify" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0049b265b7f201ca9ab25475b22b47fe444060126a51abe00f77d986fc5cc52e" + [[package]] name = "heck" version = "0.5.0" @@ -2884,22 +3092,44 @@ version = "0.4.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" +[[package]] +name = "hf-xet" +version = "1.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "430b33fa84f92796d4d263070b6c0d3ca219df7b9a0e1853ee431029b1612bcd" +dependencies = [ + "async-trait", + "bytes", + "http 1.4.0", + "more-asserts", + "serde", + "thiserror", + "tokio", + "tokio-util", + "tracing", + "uuid", + "xet-client", + "xet-core-structures", + "xet-data", + "xet-runtime", +] + [[package]] name = "hmac" version = "0.12.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6c49c37c09c17a53d937dfbb742eb3a961d65a994e6bcdcf37e7399d0cc8ab5e" dependencies = [ - "digest", + "digest 0.10.7", ] [[package]] -name = "home" -version = "0.5.12" +name = "hmac" +version = "0.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cc627f471c528ff0c4a49e1d5e60450c8f6461dd6d10ba9dcd3a61d3dff7728d" +checksum = "6303bc9732ae41b04cb554b844a762b4115a61bfaa81e3e83050991eeb56863f" dependencies = [ - "windows-sys 0.61.2", + "digest 0.11.3", ] [[package]] @@ -2975,6 +3205,15 @@ version = "2.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "135b12329e5e3ce057a9f972339ea52bc954fe1e9358ef27f95e89716fbc5424" +[[package]] +name = "hybrid-array" +version = "0.4.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9155a582abd142abc056962c29e3ce5ff2ad5469f4246b537ed42c5deba857da" +dependencies = [ + "typenum", +] + [[package]] name = "hyper" version = "0.14.32" @@ -3073,9 +3312,11 @@ dependencies = [ "percent-encoding", "pin-project-lite", "socket2 0.6.2", + "system-configuration", "tokio", "tower-service", "tracing", + "windows-registry", ] [[package]] @@ -3271,7 +3512,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4d09b98f7eace8982db770e4408e7470b028ce513ac28fecdc6bf4c30fe92b62" dependencies = [ "bitflags", - "cfg-if", + "cfg-if 1.0.4", "libc", ] @@ -3329,10 +3570,12 @@ checksum = "1a3546dc96b6d42c5f24902af9e2538e82e39ad350b0c766eb3fbf2d8f3d8359" dependencies = [ "jiff-static", "jiff-tzdb-platform", + "js-sys", "log", "portable-atomic", "portable-atomic-util", "serde_core", + "wasm-bindgen", "windows-sys 0.61.2", ] @@ -3362,6 +3605,55 @@ dependencies = [ "jiff-tzdb", ] +[[package]] +name = "jni" +version = "0.22.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5efd9a482cf3a427f00d6b35f14332adc7902ce91efb778580e180ff90fa3498" +dependencies = [ + "cfg-if 1.0.4", + "combine", + "jni-macros", + "jni-sys", + "log", + "simd_cesu8", + "thiserror", + "walkdir", + "windows-link", +] + +[[package]] +name = "jni-macros" +version = "0.22.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a00109accc170f0bdb141fed3e393c565b6f5e072365c3bd58f5b062591560a3" +dependencies = [ + "proc-macro2", + "quote", + "rustc_version", + "simd_cesu8", + "syn 2.0.117", +] + +[[package]] +name = "jni-sys" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c6377a88cb3910bee9b0fa88d4f42e1d2da8e79915598f65fb0c7ee14c878af2" +dependencies = [ + "jni-sys-macros", +] + +[[package]] +name = "jni-sys-macros" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "38c0b942f458fe50cdac086d2f946512305e5631e720728f2a61aabcd47a6264" +dependencies = [ + "quote", + "syn 2.0.117", +] + [[package]] name = "jobserver" version = "0.1.34" @@ -3402,30 +3694,32 @@ dependencies = [ "serde_json", ] -[[package]] -name = "jsonwebtoken" -version = "9.3.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5a87cc7a48537badeae96744432de36f4be2b4a34a05a5ef32e9dd8a1c169dde" -dependencies = [ - "base64", - "js-sys", - "pem", - "ring", - "serde", - "serde_json", - "simple_asn1", -] - [[package]] name = "keccak" version = "0.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cb26cec98cce3a3d96cbb7bced3c4b16e3d13f27ec56dbd62cbc8f39cfb9d653" dependencies = [ - "cpufeatures", + "cpufeatures 0.2.17", ] +[[package]] +name = "konst" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f660d5f887e3562f9ab6f4a14988795b694099d66b4f5dedc02d197ba9becb1d" +dependencies = [ + "const_panic", + "konst_proc_macros", + "typewit", +] + +[[package]] +name = "konst_proc_macros" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e037a2e1d8d5fdbd49b16a4ea09d5d6401c1f29eca5ff29d03d3824dba16256a" + [[package]] name = "lalrpop" version = "0.22.2" @@ -3460,10 +3754,11 @@ dependencies = [ [[package]] name = "lance" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d34e854994e84d043897f5ec9fb609221e9e69e3fd52996cd715d979fcd349f6" +checksum = "3944aca86f4c78f4da04af1c2bf33e664a2826b7af72972ad200d6b9de59019f" dependencies = [ + "arc-swap", "arrow", "arrow-arith", "arrow-array", @@ -3478,9 +3773,11 @@ dependencies = [ "async-trait", "async_cell", "aws-credential-types", + "bitpacking", "byteorder", "bytes", "chrono", + "crossbeam-queue", "crossbeam-skiplist", "dashmap", "datafusion", @@ -3507,13 +3804,14 @@ dependencies = [ "lance-tokenizer", "log", "moka", - "object_store 0.12.5", + "object_store", "permutation", "pin-project", "prost", "prost-build", "prost-types", "rand 0.9.2", + "rayon", "roaring", "semver", "serde", @@ -3529,13 +3827,12 @@ dependencies = [ [[package]] name = "lance-arrow" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7827fe404358c27d120ee8ea8ef7b9415c2911d54072bec83dd689d750ae65da" +checksum = "253f4a0a70580c985b91e65e9ca6cad644825a4078de28d8efbacf3ffbd7ecdc" dependencies = [ "arrow-array", "arrow-buffer", - "arrow-cast", "arrow-data", "arrow-ipc", "arrow-ord", @@ -3552,9 +3849,9 @@ dependencies = [ [[package]] name = "lance-bitpacking" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2cd0b31570d50fe13c7e4e36b03e1f1c99c3d8e5a34845b24b0665b51b40570d" +checksum = "80c4d12521b1945041dd515a56aa0854973138e7ac12111c92572e33e4ecb593" dependencies = [ "arrayref", "paste", @@ -3563,9 +3860,9 @@ dependencies = [ [[package]] name = "lance-core" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b128c213c676cb8e03c62a68670642770825171e64097cc2da97cbb19fe35d29" +checksum = "13f84020da5a484e2f07dd1796e09785ed7cd889857ebc4cb77e32ef214ee594" dependencies = [ "arrow-array", "arrow-buffer", @@ -3573,7 +3870,6 @@ dependencies = [ "async-trait", "byteorder", "bytes", - "chrono", "datafusion-common", "datafusion-sql", "deepsize", @@ -3582,10 +3878,9 @@ dependencies = [ "lance-arrow", "libc", "log", - "mock_instant", "moka", "num_cpus", - "object_store 0.12.5", + "object_store", "pin-project", "prost", "rand 0.9.2", @@ -3602,9 +3897,9 @@ dependencies = [ [[package]] name = "lance-datafusion" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e03b2de71cbcd09b10bf1a17c83cacbc0176ecd97203fb72b9e59d9b8f9a3743" +checksum = "7460597a66534a75987993d4dac5bc330586d99c5b79ae73367dbcbd4e29e576" dependencies = [ "arrow", "arrow-array", @@ -3628,16 +3923,15 @@ dependencies = [ "pin-project", "prost", "prost-build", - "snafu", "tokio", "tracing", ] [[package]] name = "lance-datagen" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2fe7c7ea7fd397e495a1646fec360e46ee0cbd75718f1c0e887aad657c5f2944" +checksum = "046f5506ed2271cd941a050de7bf535dd3aedc291aadec836a63fa56c5926e3b" dependencies = [ "arrow", "arrow-array", @@ -3655,9 +3949,9 @@ dependencies = [ [[package]] name = "lance-encoding" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "fe3f8070835b407d8db9ea8728386bc3207ba23c66a9c22d344e231ef12b77ca" +checksum = "7af54edf43dcf9d6a56cc636eb35d457e68373c6448dca3f0891b3325b4a24e6" dependencies = [ "arrow-arith", "arrow-array", @@ -3682,9 +3976,7 @@ dependencies = [ "num-traits", "prost", "prost-build", - "prost-types", "rand 0.9.2", - "snafu", "strum", "tokio", "tracing", @@ -3694,9 +3986,9 @@ dependencies = [ [[package]] name = "lance-file" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a6dfcf654549330df3aef708cd7c12e170feecddd34d6c19dd005b4153213268" +checksum = "0772ae2d6207995dc1eb28aff9507f78e90b3362b58f311da001e9dc25f3d736" dependencies = [ "arrow-arith", "arrow-array", @@ -3717,21 +4009,21 @@ dependencies = [ "lance-io", "log", "num-traits", - "object_store 0.12.5", + "object_store", "prost", "prost-build", "prost-types", - "snafu", "tokio", "tracing", ] [[package]] name = "lance-index" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4fb8ad0bd10efa2608634a2518b7dd501231e76c56a65fbd6519e23914cc425a" +checksum = "e71fbfb51096a903cb524fe0da716f5f15fbc4a6b6f84cd6dec21abf319c5e84" dependencies = [ + "arc-swap", "arrow", "arrow-arith", "arrow-array", @@ -3750,7 +4042,6 @@ dependencies = [ "datafusion-common", "datafusion-expr", "datafusion-physical-expr", - "datafusion-sql", "deepsize", "dirs", "fst", @@ -3772,7 +4063,7 @@ dependencies = [ "log", "ndarray", "num-traits", - "object_store 0.12.5", + "object_store", "prost", "prost-build", "prost-types", @@ -3784,7 +4075,6 @@ dependencies = [ "serde", "serde_json", "smallvec", - "snafu", "tempfile", "tokio", "tracing", @@ -3794,9 +4084,9 @@ dependencies = [ [[package]] name = "lance-io" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ef5314703fa8c8baed04193cc669da80ab42521c6319d3cc921a4a997690dcc0" +checksum = "bab8c98ef1b870b20541d27f3ca4efdf7c9f5c25214233be07d231ba88900219" dependencies = [ "arrow", "arrow-arith", @@ -3820,10 +4110,9 @@ dependencies = [ "lance-arrow", "lance-core", "lance-namespace", - "libc", "log", "moka", - "object_store 0.12.5", + "object_store", "object_store_opendal", "opendal", "path_abs", @@ -3831,7 +4120,6 @@ dependencies = [ "prost", "rand 0.9.2", "serde", - "snafu", "tempfile", "tokio", "tracing", @@ -3840,9 +4128,9 @@ dependencies = [ [[package]] name = "lance-linalg" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "51aa9b73279f505b2bec0f194c7a2390ca74ad3260131e631a7bef8d97d54b2e" +checksum = "6b4c51cad0ac780b02dc4da48528479e7693c03e8d05390510bbc69ca2a9a1f1" dependencies = [ "arrow-array", "arrow-buffer", @@ -3858,31 +4146,29 @@ dependencies = [ [[package]] name = "lance-namespace" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "39cd01581f55ce45c49cbe494ee86c7ba7ca4ca3654690fd820941cd9105a46e" +checksum = "014e8332ca0615506342e0d3af608639864b68396973be14239f09c9f21f1fc2" dependencies = [ "arrow", "async-trait", "bytes", "lance-core", "lance-namespace-reqwest-client", - "serde", "snafu", ] [[package]] name = "lance-namespace-impls" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c2cb89f3933060f01350ad05a5a3fbda952e8ba638799bf8ac4cd2368416ee46" +checksum = "e8d1231906a3cf92dd3dcda7d14a09c4835af6cd2bcd76dfd2481e87f20a282d" dependencies = [ "arrow", "arrow-ipc", "arrow-schema", "async-trait", "bytes", - "chrono", "futures", "lance", "lance-core", @@ -3892,10 +4178,9 @@ dependencies = [ "lance-namespace", "lance-table", "log", - "object_store 0.12.5", + "object_store", "rand 0.9.2", "serde_json", - "snafu", "tokio", "url", ] @@ -3906,7 +4191,7 @@ version = "0.7.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6369eee4682fb11edf538388b43c61ce288b8302fe89bb40944d7daa7faaae99" dependencies = [ - "reqwest", + "reqwest 0.12.28", "serde", "serde_json", "serde_repr", @@ -3916,9 +4201,9 @@ dependencies = [ [[package]] name = "lance-table" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5db70650465a1af174b7dfe6948ec91a3d466ada12e11274eb66e51132173aa0" +checksum = "b16f1355904aea4ebb04ffc70c58c97901e10bde44452b4b021de4a1f329250d" dependencies = [ "arrow", "arrow-array", @@ -3936,7 +4221,7 @@ dependencies = [ "lance-file", "lance-io", "log", - "object_store 0.12.5", + "object_store", "prost", "prost-build", "prost-types", @@ -3955,9 +4240,9 @@ dependencies = [ [[package]] name = "lance-tokenizer" -version = "6.0.1" +version = "7.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "eb08ef9382c9d58036c323db2c19cc097e02d1d0d87714fc7176b5d3b36a31aa" +checksum = "b39b7f5ed9d0c0b716bf599b559d888267ed1dfe4c4e29d3648b51d2a28940cf" dependencies = [ "rust-stemmers", "serde", @@ -4140,7 +4425,7 @@ version = "0.7.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "419e0dc8046cb947daa77eb95ae174acfbddb7673b4151f56d1eed8e93fbfaca" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "generator", "scoped-tls", "tracing", @@ -4215,8 +4500,17 @@ version = "0.10.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d89e7ee0cfbedfc4da3340218492196241d89eefb6dab27de5df917a6d2e78cf" dependencies = [ - "cfg-if", - "digest", + "cfg-if 1.0.4", + "digest 0.10.7", +] + +[[package]] +name = "mea" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2640d335e7273dacdcf51044026139b2e269c3bb0dfc3f8cb3496b85e3f6a42c" +dependencies = [ + "slab", ] [[package]] @@ -4231,7 +4525,7 @@ version = "7.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5f98efec8807c63c752b5bd61f862c165c115b0a35685bdcfd9238c7aeb592b7" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "miette-derive", "serde", "unicode-width 0.1.14", @@ -4281,16 +4575,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a69bcab0ad47271a0234d9422b131806bf3968021e5dc9328caf2d4cd58557fc" dependencies = [ "libc", - "wasi", + "wasi 0.11.1+wasi-snapshot-preview1", "windows-sys 0.61.2", ] -[[package]] -name = "mock_instant" -version = "0.6.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dce6dd36094cac388f119d2e9dc82dc730ef91c32a6222170d630e5414b956e6" - [[package]] name = "moka" version = "0.12.15" @@ -4311,6 +4599,12 @@ dependencies = [ "uuid", ] +[[package]] +name = "more-asserts" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fafa6961cabd9c63bcd77a45d7e3b7f3b552b70417831fb0f56db717e72407e" + [[package]] name = "multimap" version = "0.10.1" @@ -4362,6 +4656,15 @@ version = "0.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "61807f77802ff30975e01f4f071c8ba10c022052f98b3294119f3e615d13e5be" +[[package]] +name = "ntapi" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c3b335231dfd352ffb0f8017f3b6027a4917f7df785ea2143d8af2adc66980ae" +dependencies = [ + "winapi", +] + [[package]] name = "nu-ansi-term" version = "0.50.3" @@ -4452,6 +4755,34 @@ dependencies = [ "libc", ] +[[package]] +name = "objc2-core-foundation" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536" +dependencies = [ + "bitflags", +] + +[[package]] +name = "objc2-io-kit" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "33fafba39597d6dc1fb709123dfa8289d39406734be322956a69f0931c73bb15" +dependencies = [ + "libc", + "objc2-core-foundation", +] + +[[package]] +name = "objc2-system-configuration" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7216bd11cbda54ccabcab84d523dc93b858ec75ecfb3a7d89513fa22464da396" +dependencies = [ + "objc2-core-foundation", +] + [[package]] name = "object" version = "0.37.3" @@ -4463,16 +4794,18 @@ dependencies = [ [[package]] name = "object_store" -version = "0.12.5" +version = "0.13.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "fbfbfff40aeccab00ec8a910b57ca8ecf4319b335c542f2edcd19dd25a1e2a00" +checksum = "622acbc9100d3c10e2ee15804b0caa40e55c933d5aa53814cd520805b7958a49" dependencies = [ "async-trait", "base64", "bytes", "chrono", "form_urlencoded", - "futures", + "futures-channel", + "futures-core", + "futures-util", "http 1.4.0", "http-body-util", "httparse", @@ -4482,11 +4815,11 @@ dependencies = [ "md-5", "parking_lot", "percent-encoding", - "quick-xml 0.38.4", - "rand 0.9.2", - "reqwest", + "quick-xml 0.39.4", + "rand 0.10.1", + "reqwest 0.12.28", "ring", - "rustls-pemfile", + "rustls-pki-types", "serde", "serde_json", "serde_urlencoded", @@ -4499,43 +4832,18 @@ dependencies = [ "web-time", ] -[[package]] -name = "object_store" -version = "0.13.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "622acbc9100d3c10e2ee15804b0caa40e55c933d5aa53814cd520805b7958a49" -dependencies = [ - "async-trait", - "bytes", - "chrono", - "futures-channel", - "futures-core", - "futures-util", - "http 1.4.0", - "humantime", - "itertools 0.14.0", - "parking_lot", - "percent-encoding", - "thiserror", - "tokio", - "tracing", - "url", - "walkdir", - "wasm-bindgen-futures", - "web-time", -] - [[package]] name = "object_store_opendal" -version = "0.55.0" +version = "0.56.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "113ab0769e972eee585e57407b98de08bda5354fa28e8ba4d89038d6cb6a8991" +checksum = "08298874eee5935c95bcaa393148834f9c53d904461ca15584a041d8a1c907c2" dependencies = [ "async-trait", "bytes", "chrono", "futures", - "object_store 0.12.5", + "mea", + "object_store", "opendal", "pin-project", "tokio", @@ -4568,7 +4876,7 @@ dependencies = [ "omnigraph-policy", "omnigraph-server", "predicates", - "reqwest", + "reqwest 0.12.28", "serde", "serde_json", "serde_yaml", @@ -4586,7 +4894,7 @@ dependencies = [ "serde", "serde_json", "serde_yaml", - "sha2", + "sha2 0.10.9", "tempfile", "thiserror", "time", @@ -4607,10 +4915,10 @@ dependencies = [ "arrow-select", "pest", "pest_derive", - "reqwest", + "reqwest 0.12.28", "serde", "serde_json", - "sha2", + "sha2 0.10.9", "thiserror", "tokio", ] @@ -4639,16 +4947,16 @@ dependencies = [ "lance-namespace", "lance-namespace-impls", "lance-table", - "object_store 0.12.5", + "object_store", "omnigraph-compiler", "omnigraph-policy", "proptest", "regex", - "reqwest", + "reqwest 0.12.28", "serde", "serde_json", "serial_test", - "sha2", + "sha2 0.10.9", "tempfile", "thiserror", "time", @@ -4696,7 +5004,7 @@ dependencies = [ "serde_json", "serde_yaml", "serial_test", - "sha2", + "sha2 0.10.9", "subtle", "tempfile", "thiserror", @@ -4721,33 +5029,226 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" [[package]] -name = "opendal" -version = "0.55.0" +name = "oneshot" +version = "0.1.13" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d075ab8a203a6ab4bc1bce0a4b9fe486a72bf8b939037f4b78d95386384bc80a" +checksum = "269bca4c2591a28585d6bf10d9ed0332b7d76900a1b02bec41bdc3a2cdcda107" + +[[package]] +name = "opendal" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "97b31d3d8e99a85d83b73ec26647f5607b80578ed9375810b6e44ffa3590a236" +dependencies = [ + "ctor", + "opendal-core", + "opendal-layer-concurrent-limit", + "opendal-layer-logging", + "opendal-layer-retry", + "opendal-layer-timeout", + "opendal-service-azblob", + "opendal-service-azdls", + "opendal-service-gcs", + "opendal-service-hf", + "opendal-service-oss", + "opendal-service-s3", +] + +[[package]] +name = "opendal-core" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1849dd2687e173e776d3af5fce1ba3ae47b9dd37a09d1c4deba850ef45fe00ca" dependencies = [ "anyhow", - "backon", "base64", "bytes", - "crc32c", "futures", - "getrandom 0.2.17", "http 1.4.0", "http-body 1.0.1", "jiff", "log", "md-5", + "mea", "percent-encoding", "quick-xml 0.38.4", - "reqsign", - "reqwest", + "reqsign-core", + "reqwest 0.13.4", "serde", "serde_json", - "sha2", "tokio", "url", "uuid", + "web-time", +] + +[[package]] +name = "opendal-layer-concurrent-limit" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "048b1b29c503263bdd80a9afe46a68cd02ea9bd361185b1feab4b151078998e9" +dependencies = [ + "futures", + "http 1.4.0", + "mea", + "opendal-core", +] + +[[package]] +name = "opendal-layer-logging" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2645adc988b12eda106e2679ae529facfbbaa868ceb706f6f8125c6af15c47b" +dependencies = [ + "log", + "opendal-core", +] + +[[package]] +name = "opendal-layer-retry" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4eac134ffa4ddda6131a640a84a5315996424b9416c85052f8c64c1a33b70ad4" +dependencies = [ + "backon", + "log", + "opendal-core", +] + +[[package]] +name = "opendal-layer-timeout" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "619586ab7480c2e3009f6d18eabab18957bc094778fd130bcc38924970a90f4c" +dependencies = [ + "opendal-core", + "tokio", +] + +[[package]] +name = "opendal-service-azblob" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7452bf3ec61cfd81ac9ad9ada17825931e9e371d44a045c6bfab9596c0a2ac3b" +dependencies = [ + "base64", + "bytes", + "http 1.4.0", + "log", + "opendal-core", + "opendal-service-azure-common", + "quick-xml 0.38.4", + "reqsign-azure-storage", + "reqsign-core", + "reqsign-file-read-tokio", + "serde", + "sha2 0.10.9", + "uuid", +] + +[[package]] +name = "opendal-service-azdls" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f9884c2d8cf8ba2bb077d79c877dac5863ba3bab9e2c9c1e41a2e0491404772" +dependencies = [ + "bytes", + "http 1.4.0", + "log", + "opendal-core", + "opendal-service-azure-common", + "quick-xml 0.38.4", + "reqsign-azure-storage", + "reqsign-core", + "reqsign-file-read-tokio", + "serde", + "serde_json", +] + +[[package]] +name = "opendal-service-azure-common" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ffb0e45d6c8dcf66ce2da20e241bcb80e6e540e109a4ff20f318f6c9b4c54e0c" +dependencies = [ + "http 1.4.0", + "opendal-core", +] + +[[package]] +name = "opendal-service-gcs" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "70a49477a10163431896d106136117f5670717f9c9e49cf6f710528800c6633a" +dependencies = [ + "async-trait", + "bytes", + "http 1.4.0", + "log", + "opendal-core", + "percent-encoding", + "quick-xml 0.38.4", + "reqsign-core", + "reqsign-file-read-tokio", + "reqsign-google", + "serde", + "serde_json", + "tokio", +] + +[[package]] +name = "opendal-service-hf" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b2ab7a2a8a11dfe257ef4db5c0de798acbcd0d6429c37382dad2154bc06a388" +dependencies = [ + "bytes", + "hf-xet", + "http 1.4.0", + "log", + "opendal-core", + "percent-encoding", + "reqwest 0.13.4", + "serde", + "serde_json", +] + +[[package]] +name = "opendal-service-oss" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29c8a917829ad06d21b639558532cb0101fe49b040d946d673a73018683fac05" +dependencies = [ + "bytes", + "http 1.4.0", + "log", + "opendal-core", + "quick-xml 0.38.4", + "reqsign-aliyun-oss", + "reqsign-core", + "reqsign-file-read-tokio", + "serde", +] + +[[package]] +name = "opendal-service-s3" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9dadddeb9bb50b0d30927dd914c298c4ddca47e4c1cfa7674d311f0cf9b051c8" +dependencies = [ + "base64", + "bytes", + "crc32c", + "http 1.4.0", + "log", + "md-5", + "opendal-core", + "quick-xml 0.38.4", + "reqsign-aws-v4", + "reqsign-core", + "reqsign-file-read-tokio", + "serde", + "url", ] [[package]] @@ -4781,6 +5282,15 @@ dependencies = [ "hashbrown 0.14.5", ] +[[package]] +name = "os_str_bytes" +version = "6.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e2355d85b9a3786f481747ced0e0ff2ba35213a1f9bd406ed906554d7af805a1" +dependencies = [ + "memchr", +] + [[package]] name = "outref" version = "0.5.2" @@ -4815,7 +5325,7 @@ version = "0.9.12" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "libc", "redox_syscall", "smallvec", @@ -4846,8 +5356,8 @@ version = "0.12.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8ed6a7761f76e3b9f92dfb0a60a6a6477c61024b775147ff0973a02653abaf2" dependencies = [ - "digest", - "hmac", + "digest 0.10.7", + "hmac 0.12.1", ] [[package]] @@ -4921,7 +5431,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "89815c69d36021a140146f26659a81d6c2afa33d216d736dd4be5381a7362220" dependencies = [ "pest", - "sha2", + "sha2 0.10.9", ] [[package]] @@ -5033,7 +5543,7 @@ dependencies = [ "der", "pbkdf2", "scrypt", - "sha2", + "sha2 0.10.9", "spki", ] @@ -5248,9 +5758,9 @@ checksum = "a1d01941d82fa2ab50be1e79e6714289dd7cde78eba4c074bc5a4374f650dfe0" [[package]] name = "quick-xml" -version = "0.37.5" +version = "0.38.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "331e97a1af0bf59823e6eadffe373d7b27f485be8748f71471c662c1f269b7fb" +checksum = "b66c2058c55a409d601666cffe35f04333cf1013010882cec174a7467cd4e21c" dependencies = [ "memchr", "serde", @@ -5258,9 +5768,19 @@ dependencies = [ [[package]] name = "quick-xml" -version = "0.38.4" +version = "0.39.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b66c2058c55a409d601666cffe35f04333cf1013010882cec174a7467cd4e21c" +checksum = "cdcc8dd4e2f670d309a5f0e83fe36dfdc05af317008fea29144da1a2ac858e5e" +dependencies = [ + "memchr", + "serde", +] + +[[package]] +name = "quick-xml" +version = "0.40.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2474bd2e5029e7ccb6abb2ba48cf2383a333851dedf495901544281590c7da7f" dependencies = [ "memchr", "serde", @@ -5292,6 +5812,7 @@ version = "0.11.13" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f1906b49b0c3bc04b5fe5d86a77925ae6524a19b816ae38ce1e426255f1d8a31" dependencies = [ + "aws-lc-rs", "bytes", "getrandom 0.3.4", "lru-slab", @@ -5369,6 +5890,17 @@ dependencies = [ "rand_core 0.9.5", ] +[[package]] +name = "rand" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2e8e8bcc7961af1fdac401278c6a831614941f6164ee3bf4ce61b7edb162207" +dependencies = [ + "chacha20", + "getrandom 0.4.2", + "rand_core 0.10.1", +] + [[package]] name = "rand_chacha" version = "0.3.1" @@ -5407,6 +5939,12 @@ dependencies = [ "getrandom 0.3.4", ] +[[package]] +name = "rand_core" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69" + [[package]] name = "rand_distr" version = "0.5.1" @@ -5480,6 +6018,15 @@ dependencies = [ "crossbeam-utils", ] +[[package]] +name = "redb" +version = "3.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4ba239c1c1693315d3cc0e601db3b3965543afbf48c41730fdca2f069f510f4a" +dependencies = [ + "libc", +] + [[package]] name = "redox_syscall" version = "0.5.18" @@ -5556,34 +6103,116 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a" [[package]] -name = "reqsign" -version = "0.16.5" +name = "reqsign-aliyun-oss" +version = "3.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "43451dbf3590a7590684c25fb8d12ecdcc90ed3ac123433e500447c7d77ed701" +checksum = "372266b4733756738eeb199a98188037d27a0989980e2600ae7ce1faf00a867d" dependencies = [ "anyhow", - "async-trait", - "base64", - "chrono", "form_urlencoded", - "getrandom 0.2.17", - "hex", - "hmac", - "home", "http 1.4.0", - "jsonwebtoken", "log", - "once_cell", "percent-encoding", - "quick-xml 0.37.5", - "rand 0.8.5", - "reqwest", - "rsa", + "reqsign-core", "rust-ini", "serde", "serde_json", +] + +[[package]] +name = "reqsign-aws-v4" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b75624bd8a466e37ddc0a7b6c33ac859a85347c153a916e1dd9d0b68338f74a" +dependencies = [ + "anyhow", + "bytes", + "form_urlencoded", + "hex", + "http 1.4.0", + "log", + "percent-encoding", + "quick-xml 0.40.1", + "reqsign-core", + "rust-ini", + "serde", + "serde_json", + "serde_urlencoded", "sha1", - "sha2", +] + +[[package]] +name = "reqsign-azure-storage" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "62b96928e73ad984de1d99e382749d09e5dab7dd707b767974f7e40aa926b82f" +dependencies = [ + "anyhow", + "base64", + "bytes", + "form_urlencoded", + "http 1.4.0", + "log", + "pem", + "percent-encoding", + "reqsign-core", + "rsa", + "serde", + "serde_json", + "sha1", +] + +[[package]] +name = "reqsign-core" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a5fa5cb48808693614d1701fcd3db0b30fa292e0f18e122ae068b6d32eaeed3f" +dependencies = [ + "anyhow", + "base64", + "bytes", + "form_urlencoded", + "futures", + "hex", + "hmac 0.13.0", + "http 1.4.0", + "jiff", + "log", + "percent-encoding", + "rsa", + "serde", + "serde_json", + "sha1", + "sha2 0.11.0", + "windows-sys 0.61.2", +] + +[[package]] +name = "reqsign-file-read-tokio" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6a4b6f3a3fd29ffcc99a90aec585a65217783badfd73acddf847b63ae683bda9" +dependencies = [ + "anyhow", + "reqsign-core", + "tokio", +] + +[[package]] +name = "reqsign-google" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eb215d0876a18b6bd9cdd380b589e5292aaa638ca15266de794b1122d898b6b2" +dependencies = [ + "form_urlencoded", + "http 1.4.0", + "log", + "percent-encoding", + "reqsign-aws-v4", + "reqsign-core", + "rsa", + "serde", + "serde_json", "tokio", ] @@ -5629,11 +6258,65 @@ dependencies = [ "url", "wasm-bindgen", "wasm-bindgen-futures", - "wasm-streams", + "wasm-streams 0.4.2", "web-sys", "webpki-roots", ] +[[package]] +name = "reqwest" +version = "0.13.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "219c5811de6525e5416c7d5d53bb656d3afdbc6c5af816e0802bcfa42dbdc1c3" +dependencies = [ + "base64", + "bytes", + "futures-core", + "futures-util", + "http 1.4.0", + "http-body 1.0.1", + "http-body-util", + "hyper 1.8.1", + "hyper-rustls 0.27.7", + "hyper-util", + "js-sys", + "log", + "percent-encoding", + "pin-project-lite", + "quinn", + "rustls 0.23.36", + "rustls-pki-types", + "rustls-platform-verifier", + "serde", + "serde_json", + "sync_wrapper", + "tokio", + "tokio-rustls 0.26.4", + "tokio-util", + "tower", + "tower-http", + "tower-service", + "url", + "wasm-bindgen", + "wasm-bindgen-futures", + "wasm-streams 0.5.0", + "web-sys", +] + +[[package]] +name = "reqwest-middleware" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "07bc3f1384cffa4f274dad2d4ddd73aed32fed8f786d96c6be8aa4e5fd3c3b58" +dependencies = [ + "anyhow", + "async-trait", + "http 1.4.0", + "reqwest 0.13.4", + "thiserror", + "tower-service", +] + [[package]] name = "ring" version = "0.17.14" @@ -5641,7 +6324,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a4689e6c2294d81e88dc6261c768b63bc4fcdb852be6d1352498b114f61383b7" dependencies = [ "cc", - "cfg-if", + "cfg-if 1.0.4", "getrandom 0.2.17", "libc", "untrusted", @@ -5650,9 +6333,9 @@ dependencies = [ [[package]] name = "roaring" -version = "0.11.3" +version = "0.11.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8ba9ce64a8f45d7fc86358410bb1a82e8c987504c0d4900e9141d69a9f26c885" +checksum = "1dedc5658c6ecb3bdb5ef5f3295bb9253f42dcf3fd1402c03f6b1f7659c3c4a9" dependencies = [ "bytemuck", "byteorder", @@ -5664,15 +6347,15 @@ version = "0.9.10" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b8573f03f5883dcaebdfcf4725caa1ecb9c15b2ef50c43a07b816e06799bb12d" dependencies = [ - "const-oid", - "digest", + "const-oid 0.9.6", + "digest 0.10.7", "num-bigint-dig", "num-integer", "num-traits", "pkcs1", "pkcs8", "rand_core 0.6.4", - "sha2", + "sha2 0.10.9", "signature", "spki", "subtle", @@ -5685,7 +6368,7 @@ version = "0.21.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "796e8d2b6696392a43bea58116b667fb4c29727dc5abd27d6acf338bb4f688c7" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "ordered-multimap", ] @@ -5778,15 +6461,6 @@ dependencies = [ "security-framework", ] -[[package]] -name = "rustls-pemfile" -version = "2.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dce314e5fee3f39953d46bb63bb8a46d40c2f8fb7cc5a3b6cab2bde9721d6e50" -dependencies = [ - "rustls-pki-types", -] - [[package]] name = "rustls-pki-types" version = "1.14.0" @@ -5797,6 +6471,33 @@ dependencies = [ "zeroize", ] +[[package]] +name = "rustls-platform-verifier" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "26d1e2536ce4f35f4846aa13bff16bd0ff40157cdb14cc056c7b14ba41233ba0" +dependencies = [ + "core-foundation 0.10.1", + "core-foundation-sys", + "jni", + "log", + "once_cell", + "rustls 0.23.36", + "rustls-native-certs", + "rustls-platform-verifier-android", + "rustls-webpki 0.103.9", + "security-framework", + "security-framework-sys", + "webpki-root-certs", + "windows-sys 0.61.2", +] + +[[package]] +name = "rustls-platform-verifier-android" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f87165f0995f63a9fbeea62b64d10b4d9d8e78ec6d7d51fb2125fda7bb36788f" + [[package]] name = "rustls-webpki" version = "0.101.7" @@ -5843,6 +6544,12 @@ version = "1.0.23" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f" +[[package]] +name = "safe-transmute" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3944826ff8fa8093089aba3acb4ef44b9446a99a16f3bf4e74af3f77d340ab7d" + [[package]] name = "salsa20" version = "0.10.2" @@ -5923,7 +6630,7 @@ checksum = "0516a385866c09368f0b5bcd1caff3366aace790fcd46e2bb032697bb172fd1f" dependencies = [ "pbkdf2", "salsa20", - "sha2", + "sha2 0.10.9", ] [[package]] @@ -5949,7 +6656,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d" dependencies = [ "bitflags", - "core-foundation", + "core-foundation 0.10.1", "core-foundation-sys", "libc", "security-framework-sys", @@ -6127,13 +6834,13 @@ dependencies = [ [[package]] name = "sha1" -version = "0.10.6" +version = "0.11.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e3bf829a2d51ab4a5ddf1352d8470c140cadc8301b2ae1789db023f01cedd6ba" +checksum = "aacc4cc499359472b4abe1bf11d0b12e688af9a805fa5e3016f9a386dc2d0214" dependencies = [ - "cfg-if", - "cpufeatures", - "digest", + "cfg-if 1.0.4", + "cpufeatures 0.3.0", + "digest 0.11.3", ] [[package]] @@ -6142,9 +6849,30 @@ version = "0.10.9" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" dependencies = [ - "cfg-if", - "cpufeatures", - "digest", + "cfg-if 1.0.4", + "cpufeatures 0.2.17", + "digest 0.10.7", + "sha2-asm", +] + +[[package]] +name = "sha2" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4" +dependencies = [ + "cfg-if 1.0.4", + "cpufeatures 0.3.0", + "digest 0.11.3", +] + +[[package]] +name = "sha2-asm" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b845214d6175804686b2bd482bcffe96651bb2d1200742b712003504a2dac1ab" +dependencies = [ + "cc", ] [[package]] @@ -6153,7 +6881,7 @@ version = "0.10.8" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "75872d278a8f37ef87fa0ddbda7802605cb18344497949862c0d4dcb291eba60" dependencies = [ - "digest", + "digest 0.10.7", "keccak", ] @@ -6166,6 +6894,17 @@ dependencies = [ "lazy_static", ] +[[package]] +name = "shellexpand" +version = "3.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32824fab5e16e6c4d86dc1ba84489390419a39f97699852b66480bb87d297ed8" +dependencies = [ + "bstr", + "dirs", + "os_str_bytes", +] + [[package]] name = "shlex" version = "1.3.0" @@ -6188,7 +6927,7 @@ version = "2.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de" dependencies = [ - "digest", + "digest 0.10.7", "rand_core 0.6.4", ] @@ -6198,24 +6937,22 @@ version = "0.3.8" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e320a6c5ad31d271ad523dcf3ad13e2767ad8b1cb8f047f75a8aeaf8da139da2" +[[package]] +name = "simd_cesu8" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94f90157bb87cddf702797c5dadfa0be7d266cdf49e22da2fcaa32eff75b2c33" +dependencies = [ + "rustc_version", + "simdutf8", +] + [[package]] name = "simdutf8" version = "0.1.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e" -[[package]] -name = "simple_asn1" -version = "0.6.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0d585997b0ac10be3c5ee635f1bab02d512760d14b7c468801ac8a01d9ae5f1d" -dependencies = [ - "num-bigint", - "num-traits", - "thiserror", - "time", -] - [[package]] name = "siphasher" version = "1.0.2" @@ -6335,12 +7072,28 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "08d74a23609d509411d10e2176dc2a4346e3b4aea2e7b1869f19fdedbc71c013" dependencies = [ "cc", - "cfg-if", + "cfg-if 1.0.4", "libc", "psm", "windows-sys 0.59.0", ] +[[package]] +name = "static_assertions" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f" + +[[package]] +name = "statrs" +version = "0.18.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a3fe7c28c6512e766b0874335db33c94ad7b8f9054228ae1c2abd47ce7d335e" +dependencies = [ + "approx", + "num-traits", +] + [[package]] name = "std_prelude" version = "0.2.12" @@ -6399,6 +7152,12 @@ version = "2.6.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292" +[[package]] +name = "symlink" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7973cce6668464ea31f176d85b13c7ab3bba2cb3b77a2ed26abd7801688010a" + [[package]] name = "syn" version = "1.0.109" @@ -6441,6 +7200,41 @@ dependencies = [ "syn 2.0.117", ] +[[package]] +name = "sysinfo" +version = "0.38.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92ab6a2f8bfe508deb3c6406578252e491d299cbbf3bc0529ecc3313aee4a52f" +dependencies = [ + "libc", + "memchr", + "ntapi", + "objc2-core-foundation", + "objc2-io-kit", + "windows", +] + +[[package]] +name = "system-configuration" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a13f3d0daba03132c0aa9767f98351b3488edc2c100cda2d2ec2b04f3d8d3c8b" +dependencies = [ + "bitflags", + "core-foundation 0.9.4", + "system-configuration-sys", +] + +[[package]] +name = "system-configuration-sys" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e1d1b10ced5ca923a1fcb8d03e96b8d3268065d724548c0211415ff6ac6bac4" +dependencies = [ + "core-foundation-sys", + "libc", +] + [[package]] name = "tagptr" version = "0.2.0" @@ -6516,7 +7310,7 @@ version = "1.1.9" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f60246a4944f24f6e018aa17cdeffb7818b76356965d03b07d6a9886e8962185" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", ] [[package]] @@ -6612,6 +7406,17 @@ dependencies = [ "syn 2.0.117", ] +[[package]] +name = "tokio-retry" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4a129d95275ebf4c493ec53bf0f8cd95f5ac161bc4f381700809a54f595d4470" +dependencies = [ + "pin-project-lite", + "rand 0.10.1", + "tokio", +] + [[package]] name = "tokio-rustls" version = "0.24.1" @@ -6721,6 +7526,19 @@ dependencies = [ "tracing-core", ] +[[package]] +name = "tracing-appender" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "050686193eb999b4bb3bc2acfa891a13da00f79734704c4b8b4ef1a10b368a3c" +dependencies = [ + "crossbeam-channel", + "symlink", + "thiserror", + "time", + "tracing-subscriber", +] + [[package]] name = "tracing-attributes" version = "0.1.31" @@ -6763,6 +7581,16 @@ dependencies = [ "tracing-core", ] +[[package]] +name = "tracing-serde" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "704b1aeb7be0d0a84fc9828cae51dab5970fee5088f83d1dd7ee6f6246fc6ff1" +dependencies = [ + "serde", + "tracing-core", +] + [[package]] name = "tracing-subscriber" version = "0.3.22" @@ -6773,12 +7601,15 @@ dependencies = [ "nu-ansi-term", "once_cell", "regex-automata", + "serde", + "serde_json", "sharded-slab", "smallvec", "thread_local", "tracing", "tracing-core", "tracing-log", + "tracing-serde", ] [[package]] @@ -6804,9 +7635,15 @@ checksum = "6af6ae20167a9ece4bcb41af5b80f8a1f1df981f6391189ce00fd257af04126a" [[package]] name = "typenum" -version = "1.19.0" +version = "1.20.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "562d481066bde0658276a35467c4af00bdc6ee726305698a55b86e61d7ad82bb" +checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" + +[[package]] +name = "typewit" +version = "1.15.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "214ca0b2191785cbc06209b9ca1861e048e39b5ba33574b3cedd58363d5bb5f6" [[package]] name = "ucd-trie" @@ -7027,6 +7864,15 @@ version = "0.11.1+wasi-snapshot-preview1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" +[[package]] +name = "wasi" +version = "0.14.7+wasi-0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "883478de20367e224c0090af9cf5f9fa85bed63a95c1abf3afc5c083ebc06e8c" +dependencies = [ + "wasip2", +] + [[package]] name = "wasip2" version = "1.0.2+wasi-0.2.9" @@ -7045,13 +7891,22 @@ dependencies = [ "wit-bindgen", ] +[[package]] +name = "wasite" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "66fe902b4a6b8028a753d5424909b764ccf79b7a209eac9bf97e59cda9f71a42" +dependencies = [ + "wasi 0.14.7+wasi-0.2.4", +] + [[package]] name = "wasm-bindgen" version = "0.2.108" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "64024a30ec1e37399cf85a7ffefebdb72205ca1c972291c51512360d90bd8566" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "once_cell", "rustversion", "wasm-bindgen-macro", @@ -7064,7 +7919,7 @@ version = "0.4.58" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "70a6e77fd0ae8029c9ea0063f87c46fde723e7d887703d74ad2616d792e51e6f" dependencies = [ - "cfg-if", + "cfg-if 1.0.4", "futures-util", "js-sys", "once_cell", @@ -7139,6 +7994,19 @@ dependencies = [ "web-sys", ] +[[package]] +name = "wasm-streams" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9d1ec4f6517c9e11ae630e200b2b65d193279042e28edd4a2cda233e46670bbb" +dependencies = [ + "futures-util", + "js-sys", + "wasm-bindgen", + "wasm-bindgen-futures", + "web-sys", +] + [[package]] name = "wasmparser" version = "0.244.0" @@ -7171,6 +8039,15 @@ dependencies = [ "wasm-bindgen", ] +[[package]] +name = "webpki-root-certs" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f31141ce3fc3e300ae89b78c0dd67f9708061d1d2eda54b8209346fd6be9a92c" +dependencies = [ + "rustls-pki-types", +] + [[package]] name = "webpki-roots" version = "1.0.6" @@ -7180,6 +8057,35 @@ dependencies = [ "rustls-pki-types", ] +[[package]] +name = "whoami" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6a5b12f9df4f978d2cfdb1bd3bac52433f44393342d7ee9c25f5a1c14c0f45d" +dependencies = [ + "libc", + "libredox", + "objc2-system-configuration", + "wasite", + "web-sys", +] + +[[package]] +name = "winapi" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419" +dependencies = [ + "winapi-i686-pc-windows-gnu", + "winapi-x86_64-pc-windows-gnu", +] + +[[package]] +name = "winapi-i686-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" + [[package]] name = "winapi-util" version = "0.1.11" @@ -7189,6 +8095,33 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "winapi-x86_64-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" + +[[package]] +name = "windows" +version = "0.62.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "527fadee13e0c05939a6a05d5bd6eec6cd2e3dbd648b9f8e447c6518133d8580" +dependencies = [ + "windows-collections", + "windows-core", + "windows-future", + "windows-numerics", +] + +[[package]] +name = "windows-collections" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "23b2d95af1a8a14a3c7367e1ed4fc9c20e0a26e79551b1454d72583c97cc6610" +dependencies = [ + "windows-core", +] + [[package]] name = "windows-core" version = "0.62.2" @@ -7202,6 +8135,17 @@ dependencies = [ "windows-strings", ] +[[package]] +name = "windows-future" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e1d6f90251fe18a279739e78025bd6ddc52a7e22f921070ccdc67dde84c605cb" +dependencies = [ + "windows-core", + "windows-link", + "windows-threading", +] + [[package]] name = "windows-implement" version = "0.60.2" @@ -7230,6 +8174,27 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" +[[package]] +name = "windows-numerics" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e2e40844ac143cdb44aead537bbf727de9b044e107a0f1220392177d15b0f26" +dependencies = [ + "windows-core", + "windows-link", +] + +[[package]] +name = "windows-registry" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "02752bf7fbdcce7f2a27a742f798510f3e5ad88dbe84871e5168e2120c3d5720" +dependencies = [ + "windows-link", + "windows-result", + "windows-strings", +] + [[package]] name = "windows-result" version = "0.4.1" @@ -7317,6 +8282,15 @@ dependencies = [ "windows_x86_64_msvc 0.53.1", ] +[[package]] +name = "windows-threading" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3949bd5b99cafdf1c7ca86b43ca564028dfe27d66958f2470940f73d86d75b37" +dependencies = [ + "windows-link", +] + [[package]] name = "windows_aarch64_gnullvm" version = "0.52.6" @@ -7516,6 +8490,153 @@ dependencies = [ "tap", ] +[[package]] +name = "xet-client" +version = "1.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3e1e496dcbe6a09017acdfaf48e1a646735e7ff5b2a49e2c7e081cca77a59bc8" +dependencies = [ + "anyhow", + "async-trait", + "base64", + "bytes", + "clap", + "crc32fast", + "futures", + "http 1.4.0", + "hyper 1.8.1", + "lazy_static", + "more-asserts", + "rand 0.10.1", + "redb", + "reqwest 0.13.4", + "reqwest-middleware", + "serde", + "serde_json", + "serde_repr", + "statrs", + "tempfile", + "thiserror", + "tokio", + "tokio-retry", + "tracing", + "tracing-subscriber", + "url", + "urlencoding", + "web-time", + "xet-core-structures", + "xet-runtime", +] + +[[package]] +name = "xet-core-structures" +version = "1.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb838aa8eb67d730af301584cf003caad407487606058292a6750711b603fbee" +dependencies = [ + "async-trait", + "base64", + "blake3", + "bytemuck", + "bytes", + "clap", + "countio", + "csv", + "futures", + "futures-util", + "getrandom 0.4.2", + "heapify", + "itertools 0.14.0", + "lazy_static", + "lz4_flex", + "more-asserts", + "rand 0.10.1", + "regex", + "safe-transmute", + "serde", + "static_assertions", + "tempfile", + "thiserror", + "tokio", + "tokio-util", + "tracing", + "uuid", + "web-time", + "xet-runtime", +] + +[[package]] +name = "xet-data" +version = "1.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67fd409bef621411a9d9013798540bb8036cb2678f03ab39af89a5e88034ed8c" +dependencies = [ + "anyhow", + "async-trait", + "bytes", + "chrono", + "clap", + "gearhash", + "http 1.4.0", + "itertools 0.14.0", + "lazy_static", + "more-asserts", + "rand 0.10.1", + "serde", + "serde_json", + "sha2 0.10.9", + "tempfile", + "thiserror", + "tokio", + "tokio-util", + "tracing", + "url", + "uuid", + "walkdir", + "xet-client", + "xet-core-structures", + "xet-runtime", +] + +[[package]] +name = "xet-runtime" +version = "1.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "15d8f121c33866f7648b737abe70d0e2dd9c0af4ffdd7219207531d0283aa63d" +dependencies = [ + "anyhow", + "async-trait", + "bytes", + "chrono", + "colored", + "const-str", + "ctor", + "dirs", + "futures", + "git-version", + "humantime", + "konst", + "lazy_static", + "libc", + "more-asserts", + "oneshot", + "pin-project", + "rand 0.10.1", + "reqwest 0.13.4", + "serde", + "serde_json", + "shellexpand", + "sysinfo", + "thiserror", + "tokio", + "tokio-util", + "tracing", + "tracing-appender", + "tracing-subscriber", + "whoami", + "winapi", +] + [[package]] name = "xmlparser" version = "0.13.6" diff --git a/Cargo.toml b/Cargo.toml index 56cdde5..c442242 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -31,14 +31,14 @@ datafusion-common = "53" datafusion-expr = "53" datafusion-functions-aggregate = "53" -lance = { version = "6.0.1", default-features = false, features = ["aws"] } -lance-datafusion = "6.0.1" -lance-file = "6.0.1" -lance-index = "6.0.1" -lance-linalg = "6.0.1" -lance-namespace = "6.0.1" -lance-namespace-impls = "6.0.1" -lance-table = "6.0.1" +lance = { version = "7.0.0", default-features = false, features = ["aws"] } +lance-datafusion = "7.0.0" +lance-file = "7.0.0" +lance-index = "7.0.0" +lance-linalg = "7.0.0" +lance-namespace = "7.0.0" +lance-namespace-impls = "7.0.0" +lance-table = "7.0.0" ulid = "1" futures = "0.3" @@ -64,7 +64,7 @@ base64 = "0.22" ariadne = "0.4" regex = "1" reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] } -object_store = { version = "0.12.5", default-features = false, features = ["aws", "fs"] } +object_store = { version = "0.13.2", default-features = false, features = ["aws", "fs"] } fail = "0.5" time = { version = "0.3", features = ["formatting"] } axum = { version = "0.8", features = ["json", "macros"] } diff --git a/crates/omnigraph/src/changes/mod.rs b/crates/omnigraph/src/changes/mod.rs index d4a3fe7..2e9bc02 100644 --- a/crates/omnigraph/src/changes/mod.rs +++ b/crates/omnigraph/src/changes/mod.rs @@ -248,12 +248,12 @@ async fn diff_table_same_lineage( // Inserts + Updates: use _row_last_updated_at_version to find all rows // touched since Vf, then classify by checking whether the ID existed at Vf. // - // Why not _row_created_at_version for inserts: Lance's merge_insert stamps - // new rows with _row_created_at_version = dataset_creation_version (v1), - // not the merge_insert commit version. This makes _row_created_at_version - // unreliable for detecting inserts from merge_insert writes. Using - // _row_last_updated_at_version catches all touched rows regardless of - // write mode, and ID-set membership distinguishes inserts from updates. + // We key on _row_last_updated_at_version because one scan over it catches + // every row touched in the window — inserts and updates alike — regardless + // of write mode, and ID-set membership at Vf then distinguishes inserts from + // updates. (lance#6774 made merge_insert stamp new rows' _row_created_at_version + // with the commit version, so created_at became reliable too; last_updated + // stays the right key since it also covers updates.) if wants_inserts || wants_updates { let filter_sql = format!( "_row_last_updated_at_version > {} AND _row_last_updated_at_version <= {}", diff --git a/crates/omnigraph/src/db/commit_graph.rs b/crates/omnigraph/src/db/commit_graph.rs index 9531a64..3d90e54 100644 --- a/crates/omnigraph/src/db/commit_graph.rs +++ b/crates/omnigraph/src/db/commit_graph.rs @@ -57,6 +57,8 @@ impl CommitGraph { mode: WriteMode::Create, enable_stable_row_ids: true, data_storage_version: Some(LanceFileVersion::V2_2), + auto_cleanup: None, + skip_auto_cleanup: true, ..Default::default() }; let dataset = Dataset::write(reader, &uri as &str, Some(params)) @@ -430,6 +432,8 @@ async fn create_commit_actor_dataset(root_uri: &str) -> Result<Dataset> { mode: WriteMode::Create, enable_stable_row_ids: true, data_storage_version: Some(LanceFileVersion::V2_2), + auto_cleanup: None, + skip_auto_cleanup: true, ..Default::default() }; match Dataset::write(reader, &uri as &str, Some(params)).await { diff --git a/crates/omnigraph/src/db/manifest/graph.rs b/crates/omnigraph/src/db/manifest/graph.rs index 6c414aa..da2c641 100644 --- a/crates/omnigraph/src/db/manifest/graph.rs +++ b/crates/omnigraph/src/db/manifest/graph.rs @@ -31,6 +31,8 @@ pub(super) async fn init_manifest_graph( mode: WriteMode::Create, enable_stable_row_ids: true, data_storage_version: Some(LanceFileVersion::V2_2), + auto_cleanup: None, + skip_auto_cleanup: true, ..Default::default() }; let manifest_path = manifest_uri(root); @@ -127,6 +129,8 @@ async fn create_empty_dataset(uri: &str, schema: &SchemaRef) -> Result<Dataset> enable_stable_row_ids: true, data_storage_version: Some(LanceFileVersion::V2_2), allow_external_blob_outside_bases: true, + auto_cleanup: None, + skip_auto_cleanup: true, ..Default::default() }; Dataset::write(reader, uri, Some(params)) diff --git a/crates/omnigraph/src/db/manifest/namespace.rs b/crates/omnigraph/src/db/manifest/namespace.rs index 80d206f..5e907ba 100644 --- a/crates/omnigraph/src/db/manifest/namespace.rs +++ b/crates/omnigraph/src/db/manifest/namespace.rs @@ -10,7 +10,9 @@ use lance_namespace::models::{ }; use lance_namespace::{Error as LanceNamespaceError, LanceNamespace, NamespaceError}; use lance_table::io::commit::ManifestNamingScheme; -use object_store::{Error as ObjectStoreError, ObjectStore as _, PutMode, PutOptions, path::Path}; +use object_store::{ + Error as ObjectStoreError, ObjectStore as _, ObjectStoreExt, PutMode, PutOptions, path::Path, +}; use crate::error::{OmniError, Result}; diff --git a/crates/omnigraph/src/db/manifest/publisher.rs b/crates/omnigraph/src/db/manifest/publisher.rs index d13dd08..288f4be 100644 --- a/crates/omnigraph/src/db/manifest/publisher.rs +++ b/crates/omnigraph/src/db/manifest/publisher.rs @@ -381,6 +381,12 @@ impl GraphNamespacePublisher { // the publisher loop above, where each attempt re-runs the pre-check. merge_builder.conflict_retries(0); merge_builder.use_index(false); + // Skip Lance's auto-cleanup hook: `__manifest` versions are the snapshot + // / time-travel authority and must never be GC'd by Lance's per-commit + // hook. A `__manifest` created before the v7 bump (6.0.1 defaulted + // auto_cleanup ON) still carries the stored config, so this skip is + // load-bearing on upgraded graphs, not just defensive. + merge_builder.skip_auto_cleanup(true); let (new_dataset, _stats) = merge_builder .try_build() .map_err(|e| OmniError::Lance(e.to_string()))? diff --git a/crates/omnigraph/src/db/recovery_audit.rs b/crates/omnigraph/src/db/recovery_audit.rs index 2aab6bc..05d84b8 100644 --- a/crates/omnigraph/src/db/recovery_audit.rs +++ b/crates/omnigraph/src/db/recovery_audit.rs @@ -189,6 +189,8 @@ async fn create_recoveries_dataset(root_uri: &str) -> Result<Dataset> { mode: WriteMode::Create, enable_stable_row_ids: true, data_storage_version: Some(LanceFileVersion::V2_2), + auto_cleanup: None, + skip_auto_cleanup: true, ..Default::default() }; match Dataset::write(reader, &uri as &str, Some(params)).await { diff --git a/crates/omnigraph/src/storage.rs b/crates/omnigraph/src/storage.rs index 1f96b39..357f990 100644 --- a/crates/omnigraph/src/storage.rs +++ b/crates/omnigraph/src/storage.rs @@ -9,7 +9,7 @@ use object_store::aws::AmazonS3Builder; use object_store::local::LocalFileSystem; use object_store::memory::InMemory; use object_store::path::Path as ObjectPath; -use object_store::{DynObjectStore, ObjectStore, PutMode, PutPayload}; +use object_store::{DynObjectStore, ObjectStore, ObjectStoreExt, PutMode, PutPayload}; use url::Url; use crate::error::{OmniError, Result}; diff --git a/crates/omnigraph/src/table_store.rs b/crates/omnigraph/src/table_store.rs index b6e8c4d..b458aec 100644 --- a/crates/omnigraph/src/table_store.rs +++ b/crates/omnigraph/src/table_store.rs @@ -775,6 +775,8 @@ impl TableStore { let params = WriteParams { mode: WriteMode::Append, allow_external_blob_outside_bases: true, + auto_cleanup: None, + skip_auto_cleanup: true, ..Default::default() }; ds.append(reader, Some(params)) @@ -794,6 +796,8 @@ impl TableStore { let params = WriteParams { mode: WriteMode::Append, allow_external_blob_outside_bases: true, + auto_cleanup: None, + skip_auto_cleanup: true, ..Default::default() }; ds.append(reader, Some(params)) @@ -807,6 +811,8 @@ impl TableStore { enable_stable_row_ids: true, data_storage_version: Some(LanceFileVersion::V2_2), allow_external_blob_outside_bases: true, + auto_cleanup: None, + skip_auto_cleanup: true, ..Default::default() }; Dataset::write(reader, dataset_uri, Some(params)) @@ -897,6 +903,8 @@ impl TableStore { let params = WriteParams { mode: WriteMode::Append, allow_external_blob_outside_bases: true, + auto_cleanup: None, + skip_auto_cleanup: true, ..Default::default() }; let transaction = InsertBuilder::new(Arc::new(ds.clone())) @@ -1070,7 +1078,16 @@ impl TableStore { ds: Arc<Dataset>, transaction: Transaction, ) -> Result<Dataset> { + // Skip Lance's auto-cleanup hook on every commit. OmniGraph owns version + // GC explicitly (optimize.rs::cleanup_all_tables); Lance's hook fires off + // the *dataset's stored* `lance.auto_cleanup.*` config, which graphs + // created before the v7 bump (6.0.1 defaulted auto_cleanup ON) still + // carry — so `WriteParams::auto_cleanup = None` alone does NOT stop it on + // upgraded graphs. Skipping here covers the staged write path (the main + // data path) for new and legacy datasets alike, preventing Lance from + // GC'ing versions the __manifest still pins for snapshots/time-travel. CommitBuilder::new(ds) + .with_skip_auto_cleanup(true) .execute(transaction) .await .map_err(|e| OmniError::Lance(e.to_string())) @@ -1117,6 +1134,8 @@ impl TableStore { mode: WriteMode::Overwrite, enable_stable_row_ids: true, allow_external_blob_outside_bases: true, + auto_cleanup: None, + skip_auto_cleanup: true, ..Default::default() }; let transaction = InsertBuilder::new(Arc::new(ds.clone())) @@ -1533,6 +1552,8 @@ impl TableStore { enable_stable_row_ids: true, data_storage_version: Some(LanceFileVersion::V2_2), allow_external_blob_outside_bases: true, + auto_cleanup: None, + skip_auto_cleanup: true, ..Default::default() }; Dataset::write(reader, dataset_uri, Some(params)) diff --git a/crates/omnigraph/tests/lance_surface_guards.rs b/crates/omnigraph/tests/lance_surface_guards.rs index fdb977c..d473142 100644 --- a/crates/omnigraph/tests/lance_surface_guards.rs +++ b/crates/omnigraph/tests/lance_surface_guards.rs @@ -32,7 +32,10 @@ use lance::dataset::builder::DatasetBuilder; use lance::dataset::optimize::{CompactionOptions, compact_files}; use lance::dataset::transaction::Operation; use lance::dataset::write::delete::DeleteResult; -use lance::dataset::{MergeInsertBuilder, WhenMatched, WhenNotMatched, WriteMode, WriteParams}; +use lance::dataset::{ + CommitBuilder, InsertBuilder, MergeInsertBuilder, WhenMatched, WhenNotMatched, WriteMode, + WriteParams, +}; use lance::index::DatasetIndexExt; use lance_file::version::LanceFileVersion; use lance_index::IndexType; @@ -738,3 +741,180 @@ async fn scalar_index_use_requires_matched_literal_type() { "expected a column-side cast in the widened plan, got:\n{widened}" ); } + +// --- Guard 17: BTREE scalar-index range-boundary correctness (lance#6796) ----- +// +// lance#6796 (issue #6792) fixed a BTREE range-query bound-inclusiveness bug: +// `price <= 10 AND price > 5` returned the wrong boundary row (5.0 instead of +// 10.0). OmniGraph today builds BTREE only on string `@key` columns and queries +// them by equality/IN, not range, so its current patterns do not hit this — the +// guard protects any future BTREE-range path. It reproduces the exact #6792 shape +// (5 rows + an explicit BTREE drives the index path even on tiny data, per the +// upstream repro) and pins the corrected inclusive-`<=` / exclusive-`>` semantics. +#[tokio::test] +async fn btree_range_query_boundary_is_correct() { + use arrow_array::Float64Array; + use futures::TryStreamExt; + + let dir = tempfile::tempdir().unwrap(); + let uri = dir.path().join("guard17.lance"); + let schema = Arc::new(Schema::new(vec![ + Field::new("id", DataType::Utf8, false), + Field::new("price", DataType::Float64, false), + ])); + let batch = RecordBatch::try_new( + schema.clone(), + vec![ + Arc::new(StringArray::from(vec!["a", "b", "c", "d", "e"])), + Arc::new(Float64Array::from(vec![1.0, 5.0, 10.0, 15.0, 20.0])), + ], + ) + .unwrap(); + let reader = RecordBatchIterator::new(vec![Ok(batch)], schema); + let params = WriteParams { + mode: WriteMode::Create, + enable_stable_row_ids: true, + data_storage_version: Some(LanceFileVersion::V2_2), + ..Default::default() + }; + let mut ds = Dataset::write(reader, uri.to_str().unwrap(), Some(params)) + .await + .unwrap(); + + // Build the BTREE on the numeric column so the range filter resolves through + // the scalar index (the path lance#6796 fixed). + ds.create_index_builder(&["price"], IndexType::BTree, &ScalarIndexParams::default()) + .replace(true) + .await + .unwrap(); + + let mut scanner = ds.scan(); + scanner.filter("price <= 10.0 AND price > 5.0").unwrap(); + let batches: Vec<RecordBatch> = scanner + .try_into_stream() + .await + .unwrap() + .try_collect() + .await + .unwrap(); + let mut got: Vec<f64> = Vec::new(); + for b in &batches { + let col = b + .column_by_name("price") + .unwrap() + .as_any() + .downcast_ref::<Float64Array>() + .unwrap(); + for i in 0..col.len() { + got.push(col.value(i)); + } + } + got.sort_by(|a, b| a.partial_cmp(b).unwrap()); + assert_eq!( + got, + vec![10.0], + "BTREE range `price <= 10 AND price > 5` must return exactly [10.0] \ + (lance#6796 / issue #6792 boundary fix); got {got:?}. If this regressed, \ + Lance reintroduced the range-bound inclusiveness bug.", + ); +} + +// --- Guard 18: skip_auto_cleanup suppresses version GC (lance#6755 / PR #229) -- +// +// After the v7 bump, OmniGraph relies on `CommitBuilder::with_skip_auto_cleanup` +// (`commit_staged`) and `MergeInsertBuilder::skip_auto_cleanup` (the `__manifest` +// publisher) to stop Lance's per-commit auto-cleanup hook from GC'ing versions +// the `__manifest` pins for snapshots/time-travel. This is load-bearing for +// graphs created BEFORE the bump: 6.0.1 defaulted `WriteParams::auto_cleanup` ON, +// so those datasets carry `lance.auto_cleanup.*` config that `auto_cleanup = None` +// on new writes cannot retroactively clear — only the per-commit skip stops it. +// +// Pins both halves: WITHOUT the skip the aggressive config GCs v1; WITH the skip +// (the exact call `commit_staged` makes) v1 survives. +#[tokio::test] +async fn skip_auto_cleanup_suppresses_version_gc() { + use std::collections::HashMap; + + // The cleanup config 6.0.1 stored by default, made aggressive: fire on every + // commit, delete anything older than now. + async fn set_legacy_cleanup(ds: &mut Dataset) { + let mut cfg = HashMap::new(); + cfg.insert("lance.auto_cleanup.interval".to_string(), "1".to_string()); + cfg.insert("lance.auto_cleanup.older_than".to_string(), "0ms".to_string()); + ds.update_config(cfg).await.unwrap(); + } + fn row(i: i32) -> (Arc<Schema>, RecordBatch) { + let schema = Arc::new(Schema::new(vec![ + Field::new("id", DataType::Utf8, false), + Field::new("value", DataType::Int32, false), + ])); + let batch = RecordBatch::try_new( + schema.clone(), + vec![ + Arc::new(StringArray::from(vec![format!("k{i}")])), + Arc::new(Int32Array::from(vec![i])), + ], + ) + .unwrap(); + (schema, batch) + } + + // Negative control: WITHOUT skip, the legacy config GCs the pinned v1. + let ctrl = tempfile::tempdir().unwrap(); + let curi = ctrl.path().join("g18_ctrl.lance"); + let curi = curi.to_str().unwrap(); + let mut ds = fresh_dataset(curi).await; + let v1 = ds.version().version; + set_legacy_cleanup(&mut ds).await; + for i in 0..5 { + let (schema, batch) = row(i); + let reader = RecordBatchIterator::new(vec![Ok(batch)], schema); + ds.append( + reader, + Some(WriteParams { + mode: WriteMode::Append, + ..Default::default() + }), + ) + .await + .unwrap(); + } + assert!( + ds.checkout_version(v1).await.is_err(), + "negative control: without skip_auto_cleanup, the legacy auto_cleanup \ + config should have GC'd pinned v{v1}; if this fails the config is not \ + firing and the positive assertion below proves nothing." + ); + + // The guarantee: WITH the per-commit skip, v1 survives. Mirrors + // `TableStore::commit_staged` (InsertBuilder::execute_uncommitted + + // CommitBuilder::with_skip_auto_cleanup(true)). + let keep = tempfile::tempdir().unwrap(); + let kuri = keep.path().join("g18.lance"); + let kuri = kuri.to_str().unwrap(); + let mut ds = fresh_dataset(kuri).await; + let v1 = ds.version().version; + set_legacy_cleanup(&mut ds).await; + for i in 0..5 { + let (_schema, batch) = row(i); + let tx = InsertBuilder::new(Arc::new(ds.clone())) + .with_params(&WriteParams { + mode: WriteMode::Append, + ..Default::default() + }) + .execute_uncommitted(vec![batch]) + .await + .unwrap(); + ds = CommitBuilder::new(Arc::new(ds.clone())) + .with_skip_auto_cleanup(true) + .execute(tx) + .await + .unwrap(); + } + assert!( + ds.checkout_version(v1).await.is_ok(), + "v{v1} was GC'd despite CommitBuilder::with_skip_auto_cleanup(true) — the \ + commit_staged / publisher skip is the only thing protecting \ + __manifest-pinned versions on upgraded (pre-bump) graphs." + ); +} diff --git a/crates/omnigraph/tests/lance_version_columns.rs b/crates/omnigraph/tests/lance_version_columns.rs index b9367b9..fbe0cb4 100644 --- a/crates/omnigraph/tests/lance_version_columns.rs +++ b/crates/omnigraph/tests/lance_version_columns.rs @@ -191,14 +191,16 @@ async fn lance_merge_insert_new_row_stamps_created_at_version() { let eve = rows.iter().find(|r| r.0 == "eve").unwrap(); eprintln!("Eve: created_at_version={}, v1={}, v2={}", eve.2, v1, v2); - // Lance behavior (as of 3.0.1): merge_insert stamps new rows with - // _row_created_at_version = dataset_creation_version (v1), NOT the - // merge_insert commit version (v2). This is why Omnigraph's change - // detection uses _row_last_updated_at_version + ID set membership - // to classify inserts vs updates, not _row_created_at_version alone. + // Lance behavior (7.0.0, lance#6774): merge_insert stamps new INSERT + // rows with _row_created_at_version = the commit version (v2). Earlier + // Lance used a fallback of the dataset creation version; #6774 changed + // it so created_at reflects when the row actually entered the dataset. + // Omnigraph's change detection keys on _row_last_updated_at_version + ID + // set membership (see changes/mod.rs), so this stamping change leaves + // insert-vs-update classification unaffected. assert_eq!( - eve.2, v1, - "Lance merge_insert stamps new rows with created_at = dataset creation version, not commit version" + eve.2, v2, + "Lance merge_insert stamps new rows with created_at = commit version (lance#6774)" ); assert_eq!( eve.3, v2, @@ -258,11 +260,24 @@ async fn lance_merge_insert_update_preserves_created_at_version() { assert_eq!(alice.2, v1, "alice created_at should still be v1"); assert_eq!(alice.3, v1, "alice updated_at should still be v1"); - // Bob: updated via merge_insert - // created_at should be preserved (v1), updated_at should be bumped (v2) + // Bob: updated via merge_insert. eprintln!( "Bob: created_at={}, updated_at={}, v1={}, v2={}", bob.2, bob.3, v1, v2 ); assert_eq!(bob.1, 99, "bob's value should be updated to 99"); + // created_at is preserved across an UPDATE (lance#6774 only changed the + // INSERT-row stamping), which is what this test's name promises. + assert_eq!( + bob.2, v1, + "bob created_at must be preserved across a merge_insert UPDATE" + ); + // updated_at bumps to the commit version on UPDATE — the change-feed + // invariant OmniGraph's insert/update classification relies on + // (changes/mod.rs keys on _row_last_updated_at_version). If this regresses, + // the diff/change feed silently misses updates. + assert_eq!( + bob.3, v2, + "bob updated_at must bump to the commit version on a merge_insert UPDATE" + ); } diff --git a/crates/omnigraph/tests/staged_writes.rs b/crates/omnigraph/tests/staged_writes.rs index 3771ad4..cf0e04c 100644 --- a/crates/omnigraph/tests/staged_writes.rs +++ b/crates/omnigraph/tests/staged_writes.rs @@ -1046,3 +1046,54 @@ async fn lance_restore_loses_to_concurrent_append_via_orphaning() { let v2_ids = collect_ids(&v2_batches); assert_eq!(v2_ids, vec!["alice".to_string(), "bob".to_string()]); } + +/// Regression for PR #229: `commit_staged` must skip Lance's per-commit +/// auto-cleanup hook. A graph created BEFORE the v7 bump (6.0.1 defaulted +/// `WriteParams::auto_cleanup` ON) carries `lance.auto_cleanup.*` config on its +/// datasets that `auto_cleanup = None` on new writes cannot retroactively clear; +/// Lance's hook fires off that *stored* config at commit time. Without the skip, +/// the engine's own writes would GC the versions `__manifest` pins for +/// snapshots/time-travel. (The substrate negative control — that the config +/// really does GC without the skip — lives in +/// `lance_surface_guards.rs::skip_auto_cleanup_suppresses_version_gc`.) +#[tokio::test] +async fn commit_staged_skips_auto_cleanup_so_pinned_versions_survive() { + use std::collections::HashMap; + + let dir = tempfile::tempdir().unwrap(); + let uri = format!("{}/people.lance", dir.path().to_str().unwrap()); + let store = TableStore::new(dir.path().to_str().unwrap()); + + let mut ds = TableStore::write_dataset(&uri, person_batch(&[("seed", Some(0))])) + .await + .unwrap(); + let v1 = ds.version().version; + + // Simulate a pre-bump dataset: aggressive legacy auto_cleanup config (fire on + // every commit, delete anything older than now). + let mut cfg = HashMap::new(); + cfg.insert("lance.auto_cleanup.interval".to_string(), "1".to_string()); + cfg.insert("lance.auto_cleanup.older_than".to_string(), "0ms".to_string()); + ds.update_config(cfg).await.unwrap(); + + // Several writes through the engine's staged commit path. + for i in 0..5i32 { + let name = format!("p{i}"); + let staged = store + .stage_append(&ds, person_batch(&[(name.as_str(), Some(i))]), &[]) + .await + .unwrap(); + ds = store + .commit_staged(Arc::new(ds.clone()), staged.transaction) + .await + .unwrap(); + } + + // `commit_staged` sets `with_skip_auto_cleanup(true)`, so the legacy config + // must NOT have GC'd the `__manifest`-pinned create version. + assert!( + ds.checkout_version(v1).await.is_ok(), + "commit_staged must skip Lance auto-cleanup so a pre-bump graph's pinned \ + v{v1} survives; it was GC'd" + ); +} diff --git a/docs/dev/invariants.md b/docs/dev/invariants.md index c840309..878adfe 100644 --- a/docs/dev/invariants.md +++ b/docs/dev/invariants.md @@ -132,13 +132,18 @@ them explicit. new writer cannot couple a write with a HEAD advance through the default surface. The dead legacy methods (`append_batch` on the trait, `merge_insert_batch{,es}`, `create_{btree,inverted}_index`) were removed. The - remaining residuals are `delete_where` (gated on MR-A — Lance v7.x bump) - and `create_vector_index` (gated on Lance #6666); see - [lance.md](lance.md) and [writes.md](writes.md). New write paths should use - the staged shape unless a documented Lance blocker applies. + remaining residuals are `delete_where` and `create_vector_index`. The Lance + 6.0.1 → 7.0.0 bump landed, so the staged two-phase delete API + (`DeleteBuilder::execute_uncommitted`, Lance #6658) is now available and MR-A + is unblocked — but the migration itself is still pending, so `delete_where` + stays inline for now. `create_vector_index` remains gated on Lance #6666 + (still open). See [lance.md](lance.md) and [writes.md](writes.md). New write + paths should use the staged shape unless a documented Lance blocker applies. - **Deletes and vector indexes:** `delete_where` and vector index creation still - advance Lance HEAD inline because the required public Lance APIs are missing. - Keep D2 and recovery coverage in place until those residuals are removed. + advance Lance HEAD inline. The public delete two-phase API now exists (Lance + #6658 shipped in 7.0.0), so the delete residual is unblocked pending the MR-A + migration; vector index creation is still blocked (Lance #6666 open). Keep D2 + and recovery coverage in place until those residuals are removed. - **Blob-column compaction:** Lance `compact_files` mis-decodes blob-v2 columns under its forced `BlobHandling::AllBinary` read ("more fields in the schema than provided column indices"), so `optimize` skips any table with a `Blob` diff --git a/docs/dev/lance.md b/docs/dev/lance.md index 9544e80..2ad1273 100644 --- a/docs/dev/lance.md +++ b/docs/dev/lance.md @@ -156,7 +156,22 @@ If a future need pulls one of these into scope, add a row to the matching domain When Lance ships a major release that changes any of the above (file format bump, new index type, transaction semantics change, new branching primitive), refresh this index in the same change as the omnigraph upgrade. Stale Lance pointers are worse than no pointers. -### Last alignment audit: 2026-05-22 (Lance 6.0.1 upstream; omnigraph pinned at 6.0.1) +### Last alignment audit: 2026-06-14 (Lance 7.0.0 upstream; omnigraph pinned at 7.0.0) + +Migration from Lance 6.0.1 → 7.0.0 landed in this cycle. **Arrow stayed 58, DataFusion stayed 53** (no change) — the only transitive bump is `object_store` 0.12.5 → 0.13.2. 141 upstream commits reviewed (6.0.1 → 7.0.0); no fixes lost (the 6.0.x release-branch backports are all forward-ported into 7.0.0). Behavior-affecting findings: + +- **object_store 0.13 moved convenience methods behind a new `ObjectStoreExt` trait** (`get`/`put`/`head`/`rename`/`delete`; `list`/`list_with_delimiter`/`put_opts` stay on the core `ObjectStore` trait). Fix = add `use object_store::ObjectStoreExt;` to `storage.rs` and `db/manifest/namespace.rs`; no call-site changes. Mirrors Lance's own migration in PR #6672. The local-FS `PutMode::Update` gap is unchanged (still unimplemented upstream), so `storage.rs::write_text_if_match`'s local content-token emulation stays. +- **`roaring` must be pinned to 0.11.4** (`cargo update -p roaring --precise 0.11.4`). Lance 7.0.0's `UpdatedFragmentOffsets` newtype (PR #6650) derives `Eq` over `HashMap<u64, RoaringBitmap>`, which needs `RoaringBitmap: Eq` — added only in roaring 0.11.4 (roaring-rs PR #341). Lance's loose `roaring = "0.11"` constraint otherwise resolves the broken 0.11.3 and **lance itself fails to compile** (`RoaringBitmap: Eq is not satisfied`). roaring is transitive (no direct workspace dep); the pin lives only in `Cargo.lock`. +- **`_row_created_at_version` for merge-insert INSERT rows now = the commit version** (PR #6774; was a fallback of 1 / dataset-creation version). Flipped `lance_version_columns.rs::lance_merge_insert_new_row_stamps_created_at_version` to assert `== v2`. Production change-detection keys on `_row_last_updated_at_version` + ID-set membership, so classification logic is unaffected (the `changes/mod.rs` rationale comment was corrected). +- **BTREE range-query bound inclusiveness fixed** (PR #6796, issue #6792): `x <= hi AND x > lo` returned the wrong boundary row on 6.0.1. omnigraph today builds BTREE only on string `@key` columns (`id`/`src`/`dst`) and queries them by equality/IN, not range, so its *current* query patterns almost certainly never hit this bug — but the corrected boundary semantics are a contract we rely on the moment a BTREE-range path appears (BTREE-on-properties via the index-type tickets, or a range-on-key query). Pinned by `lance_surface_guards.rs::btree_range_query_boundary_is_correct` (reproduces #6792's 5-row + BTREE shape). +- **`WriteParams::auto_cleanup` default flipped from on (every-20-commits) to `None`** (PR #6755). On 6.0.1 the on-by-default hook could GC versions the `__manifest` pins for snapshots/time-travel. omnigraph owns cleanup explicitly (`optimize.rs::cleanup_all_tables`). Two parts to the fix, because `auto_cleanup` is **create-time config only and has no effect on existing datasets** (Lance `write.rs` docs): (1) `auto_cleanup: None` at all 11 `WriteParams` sites so *new* datasets store no cleanup config; (2) — the load-bearing half — `skip_auto_cleanup: true` on every commit path, because graphs created **before** the bump still carry the on-config in their datasets, and Lance's hook fires off the *dataset's stored* config at commit time (`io/commit.rs`: `if !commit_config.skip_auto_cleanup`). So the staged commit path (`commit_staged` → `CommitBuilder::with_skip_auto_cleanup(true)`), the `__manifest` publisher (`MergeInsertBuilder::skip_auto_cleanup(true)`), and the direct `WriteParams` paths all skip the hook. Without this, an upgraded graph would still auto-cleanup and delete `__manifest`-pinned versions. Pinned by `lance_surface_guards.rs::skip_auto_cleanup_suppresses_version_gc` (negative control + with-skip survival). +- **Lance #6658 SHIPPED in 7.0.0** (`DeleteBuilder::execute_uncommitted`, exposed via PR #6781) → MR-A (migrate `delete_where` to the staged two-phase API, retire the parse-time D2 rule) is now **unblocked**, tracked separately (dev-graph `iss-950`). The bump itself keeps `delete_where` inline; the `_compile_delete_result_field_shape` guard is left untouched until MR-A. +- **Still NOT fixed in 7.0.0:** vector-index two-phase (Lance #6666 open) — `create_vector_index` inline residual retained; blob-column compaction — `compact_files_still_fails_on_blob_columns` guard still red on a fix, `optimize` still skips blob tables behind `LANCE_SUPPORTS_BLOB_COMPACTION`. +- **No Lance-API surface omnigraph uses changed 6.0.1 → 7.0.0** (verified by a clean engine build; the only compile break was object_store). `CleanupPolicy`, `WriteParams` (apart from the `auto_cleanup` default), `CompactionOptions`, the namespace models (resolved via `lance-namespace-reqwest-client` 0.7.7, unchanged across the bump), `Operation`, `ManifestLocation`, and `MergeInsertBuilder` shapes are all stable. + +Bump this date stanza on the next alignment pass. + +### Prior alignment audit: 2026-05-22 (Lance 6.0.1 upstream; omnigraph pinned at 6.0.1) Migration from Lance 4.0.0 → 6.0.1 landed in this cycle (DataFusion 52 → 53, Arrow 57 → 58, lance-tokenizer 6.0.1 added, tantivy* removed). Direct 4 → 6 jump; v5.x was not used as an intermediate (rationale in `~/.claude/plans/shimmering-percolating-duckling.md`). Behavior-affecting findings: @@ -180,5 +195,3 @@ Migration from Lance 4.0.0 → 6.0.1 landed in this cycle (DataFusion 52 → 53, - **Lance blob-v2 `compact_files` bug** (no public issue found as of 2026-06): `compact_files` disables binary-copy for blob datasets and forces `BlobHandling::AllBinary` on the read side; the v2.1+ structural decoder then mis-counts column infos for the blob-v2 struct and fails with `Invalid user input: there were more fields in the schema than provided column indices / infos` (`lance-encoding/src/decoder.rs::ColumnInfoIter::expect_next`). This fails even a pristine uniform-V2_2 multi-fragment blob table; vector/list/scalar/ragged columns and mixed file versions all compact fine. Reads/queries use descriptor handling (`BlobHandling::default()`) and are unaffected. `optimize` skips blob-bearing tables behind `LANCE_SUPPORTS_BLOB_COMPACTION = false` (`db/omnigraph/optimize.rs`), reporting `SkipReason::BlobColumnsUnsupportedByLance`. Pinned by `lance_surface_guards.rs::compact_files_still_fails_on_blob_columns`, which turns red when the bug is fixed → flip the gate, remove the skip branch + the `maintenance.rs::optimize_skips_blob_table_and_reports_skip` skip assertions. Surface guards added: `crates/omnigraph/tests/lance_surface_guards.rs` (10 named guards; 5 runtime + 5 compile-only; plus the index-coverage work's `_compile_optimize_indices_signature` and `optimize_indices_extends_fragment_coverage`). Future Lance bumps re-run this file first as the smoke check. Two additional guards from the original plan deferred to follow-up (`manifest_cas_returns_row_level_contention_variant` needs full publisher-race harness; `table_version_metadata_byte_compatible_with_v4` needs `pub(crate)` reach extension). - -Bump this date stanza on the next alignment pass. From ceb37dd4cb9fdd7685ffbabbcbed85c5a9fbcfd6 Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Mon, 15 Jun 2026 02:37:38 +0300 Subject: [PATCH 164/165] fix(engine): close the 2 Lance 7.0.0 alignment failures (immutable PK + native namespace) (#236) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(engine): make the v1→v2 manifest migration idempotent under Lance 7's immutable unenforced primary key Lance 7 (dataset/transaction.rs) makes the unenforced primary key immutable once set: any write touching the reserved `lance-schema:unenforced-primary-key` field metadata after the PK is set errors "cannot be changed once set" — even re-applying the same value. `migrate_v1_to_v2` previously relied on the old Lance 6 idempotency (re-applying the annotation was a no-op-ish bump), which it needs for crash-recovery: a v1 graph that crashes after the field-set but before the stamp bump re-enters the migration with the PK already present. Under Lance 7 that re-entry now errors, so a real pre-v0.4.0 graph crashing in that window could never complete its migration. Guard the field-set with `schema().unenforced_primary_key().is_empty()` so a genuine first-set still runs but a re-set is skipped — restoring crash-idempotency by construction. (Fresh graphs bake the PK into manifest_schema() at init and never run this migration.) The existing test_publish_migrates_pre_stamp_manifest_to_current_version is the regression guard: red under Lance 7 before this change, green after. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(engine): realign the native-namespace surface guard to Lance 7 (TableNotFound) `test_directory_namespace_direct_publish_cannot_replace_native_omnigraph_write_path` pokes Lance's NATIVE DirectoryNamespace (not omnigraph's production write path, which is the manifest merge_insert publisher) to document that it cannot replace omnigraph's authority. Lance 7's DirectoryNamespace routes list/describe/create_table_version through `check_table_status`, which now reports an omnigraph-manifest-tracked table as absent — so all three return TableNotFound for `node:Person` (observed). The native namespace is now fully decoupled from omnigraph's manifest: it cannot enumerate, inspect, or publish over omnigraph's tables. This strengthens the guard's thesis. Realigned the assertions to the v7 behavior and kept the authority check (omnigraph's refresh ignores the direct append; row_count stays 0). Test-only; no production impact. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(lance): document the 2 runtime behavior changes in the 7.0.0 alignment stanza The #229 stanza verified a clean engine *build* but not the test suite, and claimed "no Lance API surface omnigraph uses changed." Two runtime behaviors did, caught only by the full test suite: - the unenforced primary key is immutable once set in v7 (transaction.rs) — broke the v1→v2 manifest migration's crash-idempotency; fixed by an is-set guard; - the native DirectoryNamespace returns TableNotFound for omnigraph manifest-tracked tables (dir.rs) — test-only; the surface guard was realigned. Corrects the over-broad "no surface changed" claim, adds both findings, and notes the lesson: a clean build is not a clean alignment — run cargo test --workspace before declaring a Lance bump done. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- .../omnigraph/src/db/manifest/migrations.rs | 33 ++++--- crates/omnigraph/src/db/manifest/tests.rs | 90 ++++++++++++------- docs/dev/lance.md | 6 +- 3 files changed, 82 insertions(+), 47 deletions(-) diff --git a/crates/omnigraph/src/db/manifest/migrations.rs b/crates/omnigraph/src/db/manifest/migrations.rs index e2801fe..a49135b 100644 --- a/crates/omnigraph/src/db/manifest/migrations.rs +++ b/crates/omnigraph/src/db/manifest/migrations.rs @@ -113,20 +113,27 @@ pub(super) async fn migrate_internal_schema(dataset: &mut Dataset) -> Result<()> /// so the merge-insert conflict resolver enforces row-level CAS at commit /// time, then bump the stamp. /// -/// Both steps are idempotent under retry: re-applying the field annotation -/// at its current value is a no-op-ish bump in Lance, and the stamp is a -/// simple key-value write. A crash between the two leaves the field set -/// without a stamp; the next open re-runs this fn and only the stamp lands. +/// Idempotent under crash-retry by construction. Lance 7 makes the unenforced +/// primary key **immutable once set**: any write that touches the reserved +/// `lance-schema:unenforced-primary-key` field metadata after the PK is set +/// errors ("cannot be changed once set", `lance::dataset::transaction`), even +/// re-applying the same value. A crash between the field-set and the stamp +/// bump leaves the field set without a stamp, so the next open re-enters here +/// with the PK already present — we must therefore set it only when absent. +/// (Fresh graphs bake the PK into `manifest_schema()` at init and never run +/// this migration; only genuine pre-v0.4.0 graphs do.) async fn migrate_v1_to_v2(dataset: &mut Dataset) -> Result<()> { - dataset - .update_field_metadata() - .update( - "object_id", - [(OBJECT_ID_PK_KEY.to_string(), "true".to_string())], - ) - .map_err(|e| OmniError::Lance(e.to_string()))? - .await - .map_err(|e| OmniError::Lance(e.to_string()))?; + if dataset.schema().unenforced_primary_key().is_empty() { + dataset + .update_field_metadata() + .update( + "object_id", + [(OBJECT_ID_PK_KEY.to_string(), "true".to_string())], + ) + .map_err(|e| OmniError::Lance(e.to_string()))? + .await + .map_err(|e| OmniError::Lance(e.to_string()))?; + } set_stamp(dataset, 2).await } diff --git a/crates/omnigraph/src/db/manifest/tests.rs b/crates/omnigraph/src/db/manifest/tests.rs index 885a2a8..b396866 100644 --- a/crates/omnigraph/src/db/manifest/tests.rs +++ b/crates/omnigraph/src/db/manifest/tests.rs @@ -336,40 +336,66 @@ async fn test_directory_namespace_direct_publish_cannot_replace_native_omnigraph .await .unwrap(); - let versions = namespace - .list_table_versions(ListTableVersionsRequest { - id: Some(vec!["node:Person".to_string()]), - descending: Some(true), - ..Default::default() - }) - .await - .unwrap(); - assert_eq!( - versions.versions[0].version as u64, - person_entry.table_version + // Lance 7: the native `DirectoryNamespace` no longer recognizes omnigraph's + // manifest-tracked tables. `check_table_status` reports the table absent + // (`lance-namespace-impls` dir.rs), so list / describe / create_table_version + // all return `TableNotFound`. The native path is fully decoupled from + // omnigraph's manifest authority: it cannot enumerate, inspect, or publish + // over omnigraph's tables. (Pre-v7 the native namespace could still *see* the + // published version but never replace the write path; v7 only widens the gap.) + let assert_table_not_found = |what: &str, dbg: String| { + assert!( + dbg.contains("TableNotFound") && dbg.contains("node:Person"), + "{what}: expected TableNotFound for node:Person, got: {dbg}" + ); + }; + assert_table_not_found( + "list_table_versions", + format!( + "{:?}", + namespace + .list_table_versions(ListTableVersionsRequest { + id: Some(vec!["node:Person".to_string()]), + descending: Some(true), + ..Default::default() + }) + .await + .unwrap_err() + ), + ); + assert_table_not_found( + "describe_table_version", + format!( + "{:?}", + namespace + .describe_table_version(DescribeTableVersionRequest { + id: Some(vec!["node:Person".to_string()]), + version: Some(person_version as i64), + ..Default::default() + }) + .await + .unwrap_err() + ), + ); + assert_table_not_found( + "create_table_version", + format!( + "{:?}", + namespace + .create_table_version(version_metadata.to_create_table_version_request( + "node:Person", + person_version, + 1, + None, + )) + .await + .unwrap_err() + ), ); - let err = namespace - .describe_table_version(DescribeTableVersionRequest { - id: Some(vec!["node:Person".to_string()]), - version: Some(person_version as i64), - ..Default::default() - }) - .await - .unwrap_err(); - assert!(err.to_string().contains("not found")); - - let err = namespace - .create_table_version(version_metadata.to_create_table_version_request( - "node:Person", - person_version, - 1, - None, - )) - .await - .unwrap_err(); - assert!(err.to_string().contains("already exists")); - + // omnigraph's manifest stays authoritative: refresh ignores the direct + // `person_ds.append` above (it was never manifest-published), so the row + // count stays 0 and the version is unchanged. mc.refresh().await.unwrap(); assert_eq!( mc.snapshot().entry("node:Person").unwrap().table_version, diff --git a/docs/dev/lance.md b/docs/dev/lance.md index 2ad1273..daa0435 100644 --- a/docs/dev/lance.md +++ b/docs/dev/lance.md @@ -156,7 +156,7 @@ If a future need pulls one of these into scope, add a row to the matching domain When Lance ships a major release that changes any of the above (file format bump, new index type, transaction semantics change, new branching primitive), refresh this index in the same change as the omnigraph upgrade. Stale Lance pointers are worse than no pointers. -### Last alignment audit: 2026-06-14 (Lance 7.0.0 upstream; omnigraph pinned at 7.0.0) +### Last alignment audit: 2026-06-15 (Lance 7.0.0 upstream; omnigraph pinned at 7.0.0) Migration from Lance 6.0.1 → 7.0.0 landed in this cycle. **Arrow stayed 58, DataFusion stayed 53** (no change) — the only transitive bump is `object_store` 0.12.5 → 0.13.2. 141 upstream commits reviewed (6.0.1 → 7.0.0); no fixes lost (the 6.0.x release-branch backports are all forward-ported into 7.0.0). Behavior-affecting findings: @@ -166,8 +166,10 @@ Migration from Lance 6.0.1 → 7.0.0 landed in this cycle. **Arrow stayed 58, Da - **BTREE range-query bound inclusiveness fixed** (PR #6796, issue #6792): `x <= hi AND x > lo` returned the wrong boundary row on 6.0.1. omnigraph today builds BTREE only on string `@key` columns (`id`/`src`/`dst`) and queries them by equality/IN, not range, so its *current* query patterns almost certainly never hit this bug — but the corrected boundary semantics are a contract we rely on the moment a BTREE-range path appears (BTREE-on-properties via the index-type tickets, or a range-on-key query). Pinned by `lance_surface_guards.rs::btree_range_query_boundary_is_correct` (reproduces #6792's 5-row + BTREE shape). - **`WriteParams::auto_cleanup` default flipped from on (every-20-commits) to `None`** (PR #6755). On 6.0.1 the on-by-default hook could GC versions the `__manifest` pins for snapshots/time-travel. omnigraph owns cleanup explicitly (`optimize.rs::cleanup_all_tables`). Two parts to the fix, because `auto_cleanup` is **create-time config only and has no effect on existing datasets** (Lance `write.rs` docs): (1) `auto_cleanup: None` at all 11 `WriteParams` sites so *new* datasets store no cleanup config; (2) — the load-bearing half — `skip_auto_cleanup: true` on every commit path, because graphs created **before** the bump still carry the on-config in their datasets, and Lance's hook fires off the *dataset's stored* config at commit time (`io/commit.rs`: `if !commit_config.skip_auto_cleanup`). So the staged commit path (`commit_staged` → `CommitBuilder::with_skip_auto_cleanup(true)`), the `__manifest` publisher (`MergeInsertBuilder::skip_auto_cleanup(true)`), and the direct `WriteParams` paths all skip the hook. Without this, an upgraded graph would still auto-cleanup and delete `__manifest`-pinned versions. Pinned by `lance_surface_guards.rs::skip_auto_cleanup_suppresses_version_gc` (negative control + with-skip survival). - **Lance #6658 SHIPPED in 7.0.0** (`DeleteBuilder::execute_uncommitted`, exposed via PR #6781) → MR-A (migrate `delete_where` to the staged two-phase API, retire the parse-time D2 rule) is now **unblocked**, tracked separately (dev-graph `iss-950`). The bump itself keeps `delete_where` inline; the `_compile_delete_result_field_shape` guard is left untouched until MR-A. +- **The unenforced primary key is now immutable once set** (`lance::dataset::transaction`, ~L2472–2480: `if !primary_key_before.is_empty() && (writes_primary_key || primary_key_after != primary_key_before) → "the unenforced primary key is a reserved key and cannot be changed once set"`). omnigraph marks `__manifest.object_id` as the unenforced PK (`lance-schema:unenforced-primary-key`) for merge-insert row-level CAS — baked into `manifest_schema()` at init, and added by the `migrate_v1_to_v2` internal-schema migration for pre-v0.4.0 graphs. The migration relied on Lance 6's idempotent re-apply for crash-recovery (a crash after the field-set but before the stamp bump re-enters the migration with the PK already present); under v7 that re-apply errors, so a real v1 graph could never finish migrating. Fixed by guarding the set with `schema().unenforced_primary_key().is_empty()` (`db/manifest/migrations.rs`). Regression: `db::manifest::tests::test_publish_migrates_pre_stamp_manifest_to_current_version` (was red under v7). +- **Native `DirectoryNamespace` no longer recognizes omnigraph's manifest-tracked tables** (`lance-namespace-impls` dir.rs ~L1310): `list/describe/create_table_version` route through `check_table_status`, which reports an omnigraph table absent → `TableNotFound`. omnigraph production never uses Lance's native namespace (its publisher writes `__manifest` directly via merge_insert; its own `namespace.rs` impls are custom), so this is test-only — the `test_directory_namespace_direct_publish_cannot_replace_native_omnigraph_write_path` surface guard was realigned to the v7 behavior (it now asserts the native namespace is fully decoupled, which only strengthens the guard's thesis). - **Still NOT fixed in 7.0.0:** vector-index two-phase (Lance #6666 open) — `create_vector_index` inline residual retained; blob-column compaction — `compact_files_still_fails_on_blob_columns` guard still red on a fix, `optimize` still skips blob tables behind `LANCE_SUPPORTS_BLOB_COMPACTION`. -- **No Lance-API surface omnigraph uses changed 6.0.1 → 7.0.0** (verified by a clean engine build; the only compile break was object_store). `CleanupPolicy`, `WriteParams` (apart from the `auto_cleanup` default), `CompactionOptions`, the namespace models (resolved via `lance-namespace-reqwest-client` 0.7.7, unchanged across the bump), `Operation`, `ManifestLocation`, and `MergeInsertBuilder` shapes are all stable. +- **No Lance API surface omnigraph uses changed at *compile* time** (the only compile break was object_store) — but **two runtime behaviors did** (the unenforced-PK immutability and the native-namespace `TableNotFound`, above), each caught by the full engine test suite rather than the build. `CleanupPolicy`, `WriteParams` (apart from the `auto_cleanup` default), `CompactionOptions`, the namespace models (resolved via `lance-namespace-reqwest-client` 0.7.7, unchanged across the bump), `Operation`, `ManifestLocation`, and `MergeInsertBuilder` shapes are all stable. Lesson: a clean build is not a clean alignment — run `cargo test --workspace` before declaring a Lance bump done. Bump this date stanza on the next alignment pass. From a4d08a41847ecceb1aaed632983ac94457e384cc Mon Sep 17 00:00:00 2001 From: Andrew Altshuler <andrew@collectivelab.io> Date: Mon, 15 Jun 2026 02:37:55 +0300 Subject: [PATCH 165/165] =?UTF-8?q?feat(cli):=20RFC-011=20Slice=20A=20?= =?UTF-8?q?=E2=80=94=20additive=20scope/profile=20addressing=20(#235)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(cli): RFC-011 Slice A — operator-config scope structs (profiles/clusters/defaults) Additive operator-config surface for the RFC-011 scope model. No behavior change yet — these structs are parsed but not consumed until the scope resolver lands. - OperatorConfig gains `profiles:` (name → OperatorProfile) and `clusters:` (name → OperatorCluster { root }) — the latter the only place a storage root appears in operator config (RFC-011 storage-root rule). - OperatorDefaults gains `server` and `default_graph` (the flat-default scope). - OperatorProfile binds one of {server, cluster, store} + default_graph; `binding()` validates exactly-one on use and returns a ScopeBinding. - Accessors profile()/cluster_root()/default_server()/default_graph(); unknown-key warnings extended to the new blocks (forward-compat preserved — old configs still load, new keys are no longer "unknown"). Tests: parse profiles/clusters/scope-defaults, binding rejects zero/multiple entities, unknown keys in a profile warn. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): RFC-011 Slice A — scope resolver + --profile/--store, wired (additive) Translate the new scope inputs into the existing addressing tuple, in front of the unchanged resolvers. Purely additive: an explicit address (--uri/--target/--server/--store) passes straight through, so every existing invocation is byte-for-byte unchanged. - scope.rs: resolve_scope() with the RFC-011 precedence (explicit > --profile / OMNIGRAPH_PROFILE > flat defaults.server), producing the effective (server, graph, uri, target) for data verbs and (cluster, cluster_graph) for maintenance. Plane×scope capability check (server scope rejected on a maintenance verb; cluster scope rejected on a data verb; store rejects --graph) fires only on the new paths. 9 unit tests. - cli.rs: global --profile <NAME> and --store <URI>. (--graph keeps requires=server for now; profile/default graph comes from default_graph — profile+--graph override is deferred to the --cluster-graph rework.) - client.rs: the two GraphClient factories call resolve_scope (Plane::Data) up front; the explicit branch reproduces today's behavior exactly. - main.rs: the 15 data call sites forward --profile/--store; the 3 maintenance verbs consult the scope (Plane::Storage) only when no explicit per-command address is given, so cluster-binding profiles and --store reach optimize/repair/cleanup. Verified: the full omnigraph-cli suite (221 tests) stays green untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test+docs(cli): RFC-011 Slice A — end-to-end scope test + reference docs - cli_data.rs: prove --store and a --profile store binding drive a read identically to the legacy positional URI (the additive-coexistence contract), end to end against a local graph (no server needed). - cli/reference.md: document profiles/clusters/defaults.server/default_graph, the --profile/--store flags, and a "Scopes & profiles" section; note the model coexists with legacy addressing (nothing removed yet). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --- crates/omnigraph-cli/src/cli.rs | 14 ++ crates/omnigraph-cli/src/client.rs | 31 +++ crates/omnigraph-cli/src/main.rs | 161 ++++++++++--- crates/omnigraph-cli/src/operator.rs | 192 +++++++++++++++ crates/omnigraph-cli/src/scope.rs | 321 +++++++++++++++++++++++++ crates/omnigraph-cli/tests/cli_data.rs | 54 +++++ docs/user/cli/reference.md | 32 ++- 7 files changed, 777 insertions(+), 28 deletions(-) create mode 100644 crates/omnigraph-cli/src/scope.rs diff --git a/crates/omnigraph-cli/src/cli.rs b/crates/omnigraph-cli/src/cli.rs index 28010d2..53f6026 100644 --- a/crates/omnigraph-cli/src/cli.rs +++ b/crates/omnigraph-cli/src/cli.rs @@ -41,6 +41,20 @@ pub(crate) struct Cli { #[arg(long, global = true, value_name = "GRAPH_ID", requires = "server")] pub(crate) graph: Option<String>, + /// Select a named scope bundle (RFC-011) from `profiles:` in + /// ~/.omnigraph/config.yaml: fills in this command's omitted addressing + /// (server/cluster/store + default graph). Falls back to + /// $OMNIGRAPH_PROFILE. Config data, not state — every command resolves + /// scope fresh. + #[arg(long, global = true, value_name = "NAME")] + pub(crate) profile: Option<String>, + + /// Address a single graph's storage directly (RFC-011): a `file://` / + /// `s3://` store URI. Explicit, ad-hoc direct access — bypasses any + /// server. Exclusive with a positional URI / `--target` / `--server`. + #[arg(long, global = true, value_name = "URI")] + pub(crate) store: Option<String>, + #[command(subcommand)] pub(crate) command: Command, } diff --git a/crates/omnigraph-cli/src/client.rs b/crates/omnigraph-cli/src/client.rs index 4faaa11..ca09f88 100644 --- a/crates/omnigraph-cli/src/client.rs +++ b/crates/omnigraph-cli/src/client.rs @@ -79,7 +79,23 @@ impl GraphClient { graph: Option<&str>, uri: Option<String>, target: Option<&str>, + profile: Option<&str>, + store: Option<&str>, ) -> Result<Self> { + // RFC-011: a scope (profile / --store / operator defaults) may stand in + // for omitted addressing. The explicit branch passes server/graph/uri/ + // target straight through, so existing invocations are unchanged. + let scope = crate::scope::resolve_scope( + &crate::operator::load_operator_config()?, + crate::planes::Plane::Data, + crate::scope::ScopeFlags { profile, store, server, graph, uri, target }, + )?; + let (server, graph, uri, target) = ( + scope.server.as_deref(), + scope.graph.as_deref(), + scope.uri, + scope.target.as_deref(), + ); let uri = apply_server_flag(server, graph, uri, target)?; let token = resolve_remote_bearer_token(config, uri.as_deref(), target)?; let uri = crate::helpers::resolve_uri(config, uri, target)?; @@ -111,7 +127,22 @@ impl GraphClient { uri: Option<String>, target: Option<&str>, cli_as: Option<&str>, + profile: Option<&str>, + store: Option<&str>, ) -> Result<Self> { + // RFC-011 scope translation (see `resolve`); explicit addressing passes + // through unchanged. + let scope = crate::scope::resolve_scope( + &crate::operator::load_operator_config()?, + crate::planes::Plane::Data, + crate::scope::ScopeFlags { profile, store, server, graph, uri, target }, + )?; + let (server, graph, uri, target) = ( + scope.server.as_deref(), + scope.graph.as_deref(), + scope.uri, + scope.target.as_deref(), + ); let uri = apply_server_flag(server, graph, uri, target)?; let token = resolve_remote_bearer_token(config, uri.as_deref(), target)?; let resolved = resolve_cli_graph(config, uri, target)?; diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index c3a67d4..e7cf9bd 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -49,6 +49,7 @@ mod cli; mod client; mod helpers; mod output; +mod scope; mod planes; use cli::*; use helpers::*; @@ -185,6 +186,8 @@ async fn main() -> Result<()> { uri, target.as_deref(), cli.as_actor.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let branch = resolve_branch(&config, branch, None, "main"); let payload = client @@ -219,6 +222,8 @@ async fn main() -> Result<()> { uri, target.as_deref(), cli.as_actor.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let branch = resolve_branch(&config, branch, None, "main"); let from = resolve_branch(&config, from, None, "main"); @@ -248,6 +253,8 @@ async fn main() -> Result<()> { uri, target.as_deref(), cli.as_actor.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let from = resolve_branch(&config, from, None, "main"); let payload = client.branch_create_from(&from, &name).await?; @@ -270,6 +277,8 @@ async fn main() -> Result<()> { cli.graph.as_deref(), uri, target.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let payload = client.branch_list().await?; if json { @@ -295,6 +304,8 @@ async fn main() -> Result<()> { uri, target.as_deref(), cli.as_actor.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let payload = client.branch_delete(&name).await?; if json { @@ -319,6 +330,8 @@ async fn main() -> Result<()> { uri, target.as_deref(), cli.as_actor.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let into = resolve_branch(&config, into, None, "main"); let payload = client.branch_merge(&source, &into).await?; @@ -349,6 +362,8 @@ async fn main() -> Result<()> { cli.graph.as_deref(), uri, target.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let payload = client.list_commits(branch.as_deref()).await?; if json { @@ -371,6 +386,8 @@ async fn main() -> Result<()> { cli.graph.as_deref(), uri, target.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let commit = client.get_commit(&commit_id).await?; if json { @@ -427,6 +444,8 @@ async fn main() -> Result<()> { uri, target.as_deref(), cli.as_actor.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let schema_source = fs::read_to_string(&schema)?; // The stored-query registry check is an embedded-only concern @@ -467,6 +486,8 @@ async fn main() -> Result<()> { cli.graph.as_deref(), uri, target.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let output = client.schema_source().await?; if json { @@ -521,6 +542,8 @@ async fn main() -> Result<()> { cli.graph.as_deref(), uri, target.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let branch = resolve_branch(&config, branch, None, "main"); let payload = client.snapshot(&branch).await?; @@ -546,6 +569,8 @@ async fn main() -> Result<()> { cli.graph.as_deref(), uri, target.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let branch = resolve_branch(&config, branch, None, "main"); if jsonl { @@ -636,6 +661,8 @@ async fn main() -> Result<()> { cli.graph.as_deref(), uri, target_name, + cli.profile.as_deref(), + cli.store.as_deref(), )?; let query_source = resolve_query_source( &config, @@ -714,6 +741,8 @@ async fn main() -> Result<()> { uri, target_name, cli.as_actor.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let query_source = resolve_query_source( &config, @@ -798,15 +827,41 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = resolve_storage_uri( - &config, - uri, - target.as_deref(), - cluster.as_deref(), - cluster_graph.as_deref(), - "optimize", - ) - .await?; + let uri = if uri.is_some() || target.is_some() || cluster.is_some() { + resolve_storage_uri( + &config, + uri, + target.as_deref(), + cluster.as_deref(), + cluster_graph.as_deref(), + "optimize", + ) + .await? + } else { + // RFC-011: no explicit per-command address — consult the scope + // (a --profile cluster binding, --store, or operator defaults). + let scope = scope::resolve_scope( + &operator::load_operator_config()?, + planes::Plane::Storage, + scope::ScopeFlags { + profile: cli.profile.as_deref(), + store: cli.store.as_deref(), + server: None, + graph: cli.graph.as_deref(), + uri: None, + target: None, + }, + )?; + resolve_storage_uri( + &config, + scope.uri, + scope.target.as_deref(), + scope.cluster.as_deref(), + scope.cluster_graph.as_deref(), + "optimize", + ) + .await? + }; let db = Omnigraph::open(&uri).await?; let stats = db.optimize().await?; if json { @@ -850,15 +905,40 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = resolve_storage_uri( - &config, - uri, - target.as_deref(), - cluster.as_deref(), - cluster_graph.as_deref(), - "repair", - ) - .await?; + let uri = if uri.is_some() || target.is_some() || cluster.is_some() { + resolve_storage_uri( + &config, + uri, + target.as_deref(), + cluster.as_deref(), + cluster_graph.as_deref(), + "repair", + ) + .await? + } else { + // RFC-011: no explicit per-command address — consult the scope. + let scope = scope::resolve_scope( + &operator::load_operator_config()?, + planes::Plane::Storage, + scope::ScopeFlags { + profile: cli.profile.as_deref(), + store: cli.store.as_deref(), + server: None, + graph: cli.graph.as_deref(), + uri: None, + target: None, + }, + )?; + resolve_storage_uri( + &config, + scope.uri, + scope.target.as_deref(), + scope.cluster.as_deref(), + scope.cluster_graph.as_deref(), + "repair", + ) + .await? + }; let db = Omnigraph::open(&uri).await?; let stats = db .repair(omnigraph::db::RepairOptions { confirm, force }) @@ -944,15 +1024,40 @@ async fn main() -> Result<()> { json, } => { let config = load_cli_config(config.as_ref())?; - let uri = resolve_storage_uri( - &config, - uri, - target.as_deref(), - cluster.as_deref(), - cluster_graph.as_deref(), - "cleanup", - ) - .await?; + let uri = if uri.is_some() || target.is_some() || cluster.is_some() { + resolve_storage_uri( + &config, + uri, + target.as_deref(), + cluster.as_deref(), + cluster_graph.as_deref(), + "cleanup", + ) + .await? + } else { + // RFC-011: no explicit per-command address — consult the scope. + let scope = scope::resolve_scope( + &operator::load_operator_config()?, + planes::Plane::Storage, + scope::ScopeFlags { + profile: cli.profile.as_deref(), + store: cli.store.as_deref(), + server: None, + graph: cli.graph.as_deref(), + uri: None, + target: None, + }, + )?; + resolve_storage_uri( + &config, + scope.uri, + scope.target.as_deref(), + scope.cluster.as_deref(), + scope.cluster_graph.as_deref(), + "cleanup", + ) + .await? + }; let older_than_dur = older_than.as_deref().map(parse_duration_arg).transpose()?; @@ -1088,6 +1193,8 @@ async fn main() -> Result<()> { cli.graph.as_deref(), uri, target.as_deref(), + cli.profile.as_deref(), + cli.store.as_deref(), )?; let payload = client.list_graphs().await?; if json { diff --git a/crates/omnigraph-cli/src/operator.rs b/crates/omnigraph-cli/src/operator.rs index fb8658d..e48af50 100644 --- a/crates/omnigraph-cli/src/operator.rs +++ b/crates/omnigraph-cli/src/operator.rs @@ -41,6 +41,17 @@ pub(crate) struct OperatorConfig { /// Personal alias bindings (RFC-007 PR 3); see OperatorAlias. #[serde(default)] pub(crate) aliases: BTreeMap<String, OperatorAlias>, + /// Named scope bundles (RFC-011): each binds exactly one of + /// {server, cluster, store} plus an optional default graph. Config data, + /// not state — selecting one (`--profile`/`OMNIGRAPH_PROFILE`) fills in a + /// command's omitted addressing; it never puts you "in" a mode. + #[serde(default)] + pub(crate) profiles: BTreeMap<String, OperatorProfile>, + /// Managed-cluster storage roots (RFC-011): name → root URI. The ONLY + /// place a storage root appears in operator config — admin-only and + /// opt-in; a normal operator's file has none. + #[serde(default)] + pub(crate) clusters: BTreeMap<String, OperatorCluster>, /// Everything this CLI version doesn't know. Warned once at load, /// otherwise ignored (forward compatibility within the operator layer). #[serde(flatten)] @@ -95,10 +106,58 @@ pub(crate) struct OperatorDefaults { /// during the RFC-008 window). pub(crate) table_max_column_width: Option<usize>, pub(crate) table_cell_layout: Option<omnigraph_server::config::TableCellLayout>, + /// Default server scope (RFC-011): the everyday addressing when no + /// `--profile` / primitive / legacy address is given. Names an entry + /// under `servers:`. + pub(crate) server: Option<String>, + /// Default graph selected within a server/cluster scope when no + /// `--graph` is passed (RFC-011). + pub(crate) default_graph: Option<String>, #[serde(flatten)] unknown: serde_yaml::Mapping, } +/// A named scope bundle (RFC-011): exactly one of {server, cluster, store} +/// plus an optional default graph. Validated on use (`binding()`), not at +/// parse time, so an unknown CLI's profile still loads. +#[derive(Debug, Default, Deserialize)] +pub(crate) struct OperatorProfile { + /// Names an entry under `servers:` — a served scope. + pub(crate) server: Option<String>, + /// Names an entry under `clusters:` — a privileged direct cluster scope. + pub(crate) cluster: Option<String>, + /// A single graph's storage URI — a direct store scope. + pub(crate) store: Option<String>, + /// Default graph within a server/cluster scope (ignored for a store, + /// which is already one graph). + pub(crate) default_graph: Option<String>, + #[serde(flatten)] + unknown: serde_yaml::Mapping, +} + +/// A managed-cluster storage root (RFC-011). +#[derive(Debug, Default, Deserialize)] +pub(crate) struct OperatorCluster { + /// The cluster's storage-root URI (`file://` / `s3://`). + pub(crate) root: String, + #[serde(flatten)] + unknown: serde_yaml::Mapping, +} + +/// The one entity a profile (or flat default) binds. Exactly one variant — +/// the scope resolver consumes this; "exactly one of server/cluster/store" +/// is enforced when producing it. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) enum ScopeBinding { + /// Served scope: a server name (resolved against `servers:`) or a literal URL. + Server(String), + /// Direct cluster scope: a cluster name (resolved against `clusters:`) or a + /// literal root URI. + Cluster(String), + /// Direct store scope: a single graph's storage URI. + Store(String), +} + impl OperatorConfig { pub(crate) fn actor(&self) -> Option<&str> { self.operator.actor.as_deref() @@ -127,6 +186,57 @@ impl OperatorConfig { } best.map(|(name, _)| name) } + + /// A named profile, if defined (RFC-011). + pub(crate) fn profile(&self, name: &str) -> Option<&OperatorProfile> { + self.profiles.get(name) + } + + /// The storage root of a named cluster, if defined (RFC-011). + pub(crate) fn cluster_root(&self, name: &str) -> Option<&str> { + self.clusters.get(name).map(|c| c.root.as_str()) + } + + /// The flat-default server scope name, if set (RFC-011). + pub(crate) fn default_server(&self) -> Option<&str> { + self.defaults.server.as_deref() + } + + /// The flat-default graph within a server/cluster scope, if set (RFC-011). + pub(crate) fn default_graph(&self) -> Option<&str> { + self.defaults.default_graph.as_deref() + } +} + +impl OperatorProfile { + /// The single entity this profile binds, or a loud error if it binds zero + /// or more than one of {server, cluster, store} (Decision 6: a scope binds + /// exactly one entity). Validated here, on use, rather than at parse time. + pub(crate) fn binding(&self, profile_name: &str) -> Result<ScopeBinding> { + let set: Vec<&str> = [ + self.server.as_ref().map(|_| "server"), + self.cluster.as_ref().map(|_| "cluster"), + self.store.as_ref().map(|_| "store"), + ] + .into_iter() + .flatten() + .collect(); + match set.as_slice() { + ["server"] => Ok(ScopeBinding::Server(self.server.clone().unwrap())), + ["cluster"] => Ok(ScopeBinding::Cluster(self.cluster.clone().unwrap())), + ["store"] => Ok(ScopeBinding::Store(self.store.clone().unwrap())), + [] => Err(eyre!( + "profile '{profile_name}' binds no scope; set exactly one of \ + `server`, `cluster`, or `store`" + )), + many => Err(eyre!( + "profile '{profile_name}' binds {} scopes ({}); a profile must \ + bind exactly one of `server`, `cluster`, or `store`", + many.len(), + many.join(", ") + )), + } + } } /// The operator dir: `$OMNIGRAPH_HOME` if set (tilde-expanded), else @@ -196,6 +306,12 @@ impl OperatorConfig { for (name, alias) in &self.aliases { collect(&alias.unknown, &format!("aliases.{name}.")); } + for (name, profile) in &self.profiles { + collect(&profile.unknown, &format!("profiles.{name}.")); + } + for (name, cluster) in &self.clusters { + collect(&cluster.unknown, &format!("clusters.{name}.")); + } warnings } } @@ -464,6 +580,82 @@ mod tests { assert_eq!(config.servers["prod"].url, "https://example.com"); } + #[test] + fn parses_profiles_clusters_and_scope_defaults() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("config.yaml"); + let yaml = "\ +defaults: + server: prod + default_graph: knowledge +servers: + prod: + url: https://example.com +clusters: + brain: + root: s3://acme/clusters/brain +profiles: + staging: + server: staging + default_graph: knowledge + brain-admin: + cluster: brain + default_graph: knowledge +"; + fs::write(&path, yaml).unwrap(); + let config = load_operator_config_at(&path).unwrap(); + assert_eq!(config.default_server(), Some("prod")); + assert_eq!(config.default_graph(), Some("knowledge")); + assert_eq!(config.cluster_root("brain"), Some("s3://acme/clusters/brain")); + assert_eq!( + config.profile("staging").unwrap().binding("staging").unwrap(), + ScopeBinding::Server("staging".into()) + ); + assert_eq!( + config + .profile("brain-admin") + .unwrap() + .binding("brain-admin") + .unwrap(), + ScopeBinding::Cluster("brain".into()) + ); + // No unknown-key warnings for the new blocks. + assert!(config.unknown_key_warnings().is_empty(), "{:?}", config.unknown_key_warnings()); + } + + #[test] + fn profile_binding_rejects_zero_or_multiple_entities() { + let none = OperatorProfile::default(); + let err = none.binding("p").unwrap_err().to_string(); + assert!(err.contains("binds no scope"), "{err}"); + + let two = OperatorProfile { + server: Some("prod".into()), + store: Some("graph.omni".into()), + ..Default::default() + }; + let err = two.binding("p").unwrap_err().to_string(); + assert!(err.contains("binds 2 scopes"), "{err}"); + assert!(err.contains("server") && err.contains("store"), "{err}"); + } + + #[test] + fn unknown_keys_in_a_profile_warn() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("config.yaml"); + fs::write( + &path, + "profiles:\n p:\n server: prod\n flavour: spicy\n", + ) + .unwrap(); + let config = load_operator_config_at(&path).unwrap(); + let warnings = config.unknown_key_warnings(); + assert!( + warnings.iter().any(|w| w.contains("`profiles.p.flavour`")), + "{warnings:?}" + ); + } + #[test] fn malformed_yaml_is_a_loud_error() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/omnigraph-cli/src/scope.rs b/crates/omnigraph-cli/src/scope.rs new file mode 100644 index 0000000..19ac48d --- /dev/null +++ b/crates/omnigraph-cli/src/scope.rs @@ -0,0 +1,321 @@ +//! RFC-011 Slice A scope resolution. +//! +//! Translates the new scope inputs (`--profile` / `--store` / operator-config +//! `profiles`/`clusters`/`defaults`) into the SAME effective addressing tuple +//! the existing `GraphClient` factories (`client.rs`) and the maintenance +//! resolver (`helpers::resolve_storage_uri`) already consume. This is a +//! translation layer that sits *in front* of those resolvers — it is purely +//! additive: an explicit legacy address (`--uri`/`--target`/`--server`/ +//! `--store`) wins and reproduces today's behavior exactly, so existing +//! invocations are unaffected. +//! +//! The access path (served vs direct) is never chosen here; it falls out of the +//! scope's binding × the verb's plane. The plane→scope capability check rejects +//! mismatches (e.g. a server scope on a maintenance verb) only on the *new* +//! resolution paths. + +use std::env; + +use color_eyre::Result; +use color_eyre::eyre::{bail, eyre}; + +use crate::operator::{OperatorConfig, ScopeBinding}; +use crate::planes::Plane; + +pub(crate) const PROFILE_ENV: &str = "OMNIGRAPH_PROFILE"; + +/// The effective addressing a command should use, in the terms the existing +/// resolvers consume. Data/served verbs read `server`/`graph`/`uri`/`target`; +/// maintenance verbs read `cluster`/`cluster_graph`. +#[derive(Debug, Default, PartialEq, Eq)] +pub(crate) struct ResolvedScope { + pub(crate) server: Option<String>, + pub(crate) graph: Option<String>, + pub(crate) uri: Option<String>, + pub(crate) target: Option<String>, + pub(crate) cluster: Option<String>, + pub(crate) cluster_graph: Option<String>, +} + +/// The raw addressing inputs for one command: the global scope flags plus the +/// command's own positional/`--target` address. +pub(crate) struct ScopeFlags<'a> { + pub(crate) profile: Option<&'a str>, + pub(crate) store: Option<&'a str>, + pub(crate) server: Option<&'a str>, + pub(crate) graph: Option<&'a str>, + pub(crate) uri: Option<String>, + pub(crate) target: Option<&'a str>, +} + +/// Resolve the scope for a command on `plane`. Precedence (RFC-011): +/// 1. explicit legacy/primitive address (`uri`/`target`/`--server`/`--store`) → passthrough; +/// 2. `--profile` / `OMNIGRAPH_PROFILE`; +/// 3. flat `defaults.server` + `defaults.default_graph`; +/// 4. nothing — downstream behaves as today. +pub(crate) fn resolve_scope( + op: &OperatorConfig, + plane: Plane, + flags: ScopeFlags<'_>, +) -> Result<ResolvedScope> { + // 1. Any explicit address wins; reproduce today's behavior untouched. + // `--store` is an explicit store URI — fold it into `uri`. + if flags.uri.is_some() || flags.target.is_some() || flags.server.is_some() || flags.store.is_some() + { + return Ok(ResolvedScope { + server: flags.server.map(str::to_string), + graph: flags.graph.map(str::to_string), + uri: flags.store.map(str::to_string).or(flags.uri), + target: flags.target.map(str::to_string), + ..Default::default() + }); + } + + // 2. A named profile (flag, else env). + let profile_name = flags + .profile + .map(str::to_string) + .or_else(|| env::var(PROFILE_ENV).ok().filter(|s| !s.is_empty())); + if let Some(name) = profile_name { + let profile = op.profile(&name).ok_or_else(|| { + eyre!("unknown profile '{name}' (not defined under `profiles:` in operator config)") + })?; + let binding = profile.binding(&name)?; + let graph = flags + .graph + .map(str::to_string) + .or_else(|| profile.default_graph.clone()); + return scope_from_binding(op, plane, binding, graph, &format!("profile '{name}'")); + } + + // 3. Flat default server scope. + if let Some(server) = op.default_server() { + let graph = flags + .graph + .map(str::to_string) + .or_else(|| op.default_graph().map(str::to_string)); + return scope_from_binding( + op, + plane, + ScopeBinding::Server(server.to_string()), + graph, + "operator defaults", + ); + } + + // 4. Nothing resolved — leave the tuple empty; downstream falls through to + // today's behavior (legacy `cli.graph` default or a no-address error). + Ok(ResolvedScope::default()) +} + +/// Map a resolved binding to the effective tuple, enforcing scope × plane +/// capability (RFC-011): a server scope is served (data only); a cluster scope +/// is privileged direct (maintenance/control only); a store scope is direct +/// (either). +fn scope_from_binding( + op: &OperatorConfig, + plane: Plane, + binding: ScopeBinding, + graph: Option<String>, + source: &str, +) -> Result<ResolvedScope> { + match binding { + ScopeBinding::Server(server) => { + if plane == Plane::Storage { + bail!( + "this command needs direct storage access, but {source} resolves a \ + server scope; name storage explicitly with --store <uri> (or a \ + --cluster/--cluster-graph for a managed graph)" + ); + } + Ok(ResolvedScope { + server: Some(server), + graph, + ..Default::default() + }) + } + ScopeBinding::Cluster(cluster) => { + if plane == Plane::Data { + bail!( + "{source} resolves a cluster scope, which is maintenance-only; run \ + data commands through a server, or use --store <uri> for ad-hoc \ + direct access" + ); + } + // A cluster binding is a config name (resolved against `clusters:`) + // or a literal root URI. + let root = if let Some(root) = op.cluster_root(&cluster) { + root.to_string() + } else if cluster.contains("://") { + cluster + } else { + bail!( + "unknown cluster '{cluster}' ({source}); define it under `clusters:` \ + in operator config, or use a literal root URI" + ); + }; + Ok(ResolvedScope { + cluster: Some(root), + cluster_graph: graph, + ..Default::default() + }) + } + ScopeBinding::Store(uri) => { + if graph.is_some() { + bail!( + "--graph does not apply to a store scope ({source}): a store is already \ + a single graph" + ); + } + Ok(ResolvedScope { + uri: Some(uri), + ..Default::default() + }) + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn cfg(yaml: &str) -> OperatorConfig { + serde_yaml::from_str(yaml).unwrap() + } + + fn flags<'a>() -> ScopeFlags<'a> { + ScopeFlags { + profile: None, + store: None, + server: None, + graph: None, + uri: None, + target: None, + } + } + + #[test] + fn explicit_legacy_address_wins_unchanged() { + let op = cfg("defaults:\n server: prod\nservers:\n prod:\n url: https://x\n"); + // A positional URI given → profile/defaults are ignored entirely. + let scope = resolve_scope( + &op, + Plane::Data, + ScopeFlags { + uri: Some("graph.omni".into()), + ..flags() + }, + ) + .unwrap(); + assert_eq!(scope.uri.as_deref(), Some("graph.omni")); + assert_eq!(scope.server, None); + } + + #[test] + fn store_flag_folds_into_uri_and_rejects_graph() { + let op = OperatorConfig::default(); + let scope = resolve_scope( + &op, + Plane::Data, + ScopeFlags { + store: Some("s3://b/g.omni"), + ..flags() + }, + ) + .unwrap(); + assert_eq!(scope.uri.as_deref(), Some("s3://b/g.omni")); + } + + #[test] + fn flat_default_server_drives_data_verbs() { + let op = cfg("defaults:\n server: prod\n default_graph: knowledge\nservers:\n prod:\n url: https://x\n"); + let scope = resolve_scope(&op, Plane::Data, flags()).unwrap(); + assert_eq!(scope.server.as_deref(), Some("prod")); + assert_eq!(scope.graph.as_deref(), Some("knowledge")); + } + + #[test] + fn profile_server_scope_with_graph_override() { + let op = cfg( + "servers:\n staging:\n url: https://s\nprofiles:\n staging:\n server: staging\n default_graph: knowledge\n", + ); + let scope = resolve_scope( + &op, + Plane::Data, + ScopeFlags { + profile: Some("staging"), + graph: Some("archive"), + ..flags() + }, + ) + .unwrap(); + assert_eq!(scope.server.as_deref(), Some("staging")); + assert_eq!(scope.graph.as_deref(), Some("archive")); // flag beats profile default + } + + #[test] + fn profile_cluster_scope_resolves_root_for_maintenance() { + let op = cfg( + "clusters:\n brain:\n root: s3://acme/brain\nprofiles:\n admin:\n cluster: brain\n default_graph: knowledge\n", + ); + let scope = resolve_scope( + &op, + Plane::Storage, + ScopeFlags { + profile: Some("admin"), + ..flags() + }, + ) + .unwrap(); + assert_eq!(scope.cluster.as_deref(), Some("s3://acme/brain")); + assert_eq!(scope.cluster_graph.as_deref(), Some("knowledge")); + } + + #[test] + fn server_scope_on_maintenance_verb_errors() { + let op = cfg("defaults:\n server: prod\nservers:\n prod:\n url: https://x\n"); + let err = resolve_scope(&op, Plane::Storage, flags()).unwrap_err().to_string(); + assert!(err.contains("direct storage access"), "{err}"); + } + + #[test] + fn cluster_scope_on_data_verb_errors() { + let op = cfg( + "clusters:\n brain:\n root: s3://acme/brain\nprofiles:\n admin:\n cluster: brain\n", + ); + let err = resolve_scope( + &op, + Plane::Data, + ScopeFlags { + profile: Some("admin"), + ..flags() + }, + ) + .unwrap_err() + .to_string(); + assert!(err.contains("maintenance-only"), "{err}"); + } + + #[test] + fn unknown_profile_is_a_loud_error() { + let op = OperatorConfig::default(); + let err = resolve_scope( + &op, + Plane::Data, + ScopeFlags { + profile: Some("nope"), + ..flags() + }, + ) + .unwrap_err() + .to_string(); + assert!(err.contains("unknown profile 'nope'"), "{err}"); + } + + #[test] + fn no_address_resolves_empty_for_legacy_fallthrough() { + let op = OperatorConfig::default(); + let scope = resolve_scope(&op, Plane::Data, flags()).unwrap(); + assert_eq!(scope, ResolvedScope::default()); + } +} diff --git a/crates/omnigraph-cli/tests/cli_data.rs b/crates/omnigraph-cli/tests/cli_data.rs index 8d1f80a..99a3038 100644 --- a/crates/omnigraph-cli/tests/cli_data.rs +++ b/crates/omnigraph-cli/tests/cli_data.rs @@ -3,6 +3,7 @@ use std::fs; +use assert_cmd::Command; use serde_json::Value; use tempfile::tempdir; @@ -801,6 +802,59 @@ fn read_json_outputs_rows_for_named_query() { assert_eq!(payload["rows"][0]["p.name"], "Alice"); } +#[test] +fn read_via_store_flag_and_profile_match_positional_uri() { + // RFC-011 Slice A: the new scope addressing (--store, and a --profile that + // binds a store) drives a read identically to the legacy positional URI — + // the scope layer is additive, not a behavior change. + let temp = tempdir().unwrap(); + let graph = graph_path(temp.path()); + init_graph(&graph); + load_fixture(&graph); + let queries = fixture("test.gq"); + + let read_rows = |cmd: &mut Command| -> Value { + let output = output_success( + cmd.arg("--query") + .arg(&queries) + .arg("--name") + .arg("get_person") + .arg("--params") + .arg(r#"{"name":"Alice"}"#) + .arg("--json"), + ); + serde_json::from_slice(&output.stdout).unwrap() + }; + + // Baseline: positional URI. + let baseline = read_rows(cli().arg("query").arg(&graph)); + assert_eq!(baseline["rows"][0]["p.name"], "Alice"); + + // --store names the same graph directly. + let via_store = read_rows(cli().arg("query").arg("--store").arg(&graph)); + assert_eq!(via_store["rows"], baseline["rows"]); + + // A profile binding that store, selected with --profile (no positional). + let home = temp.path().join("op-home"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::write( + home.join("config.yaml"), + format!( + "profiles:\n local:\n store: '{}'\n", + graph.to_string_lossy() + ), + ) + .unwrap(); + let via_profile = read_rows( + cli() + .env("OMNIGRAPH_HOME", &home) + .arg("query") + .arg("--profile") + .arg("local"), + ); + assert_eq!(via_profile["rows"], baseline["rows"]); +} + #[test] fn export_jsonl_outputs_source_rows_for_selected_branch_and_type() { let temp = tempdir().unwrap(); diff --git a/docs/user/cli/reference.md b/docs/user/cli/reference.md index 77feaf1..44d5ad4 100644 --- a/docs/user/cli/reference.md +++ b/docs/user/cli/reference.md @@ -2,7 +2,7 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` schema. For a quick-start guide, see [cli.md](index.md). -Top-level command families and subcommands. Graph-targeting commands accept a positional `URI`, `--uri`, a `--target <name>` resolved against `omnigraph.yaml`, or `--server <name>` (an operator-defined server from `~/.omnigraph/config.yaml`, optionally with `--graph <id>` for multi-graph servers; exclusive with the other forms); `cluster` commands use `--config <dir>`. +Top-level command families and subcommands. Graph-targeting commands accept a positional `URI`, `--uri`, a `--target <name>` resolved against `omnigraph.yaml`, `--server <name>` (an operator-defined server from `~/.omnigraph/config.yaml`, optionally with `--graph <id>` for multi-graph servers; exclusive with the other forms), `--store <uri>` (a single graph's storage directly), or `--profile <name>` / `$OMNIGRAPH_PROFILE` (a named scope bundle; see [Scopes & profiles](#scopes--profiles-rfc-011)); `cluster` commands use `--config <dir>`. ## Top-level commands @@ -71,12 +71,42 @@ servers: # operator-owned endpoints; names key the credentials url: https://graph.example.com # no tokens in this file, ever defaults: output: table # read format default, below --json/--format/alias/legacy + server: prod # the everyday scope when no address is given (RFC-011) + default_graph: knowledge # graph selected in a server/cluster scope +clusters: # admin-only: managed-cluster storage roots (RFC-011). + brain: # the ONLY place a storage root lives in this file. + root: s3://acme/clusters/brain +profiles: # named scope bundles (RFC-011); pick with --profile + staging: { server: staging, default_graph: knowledge } # a served scope + brain-admin: { cluster: brain, default_graph: knowledge } # a direct cluster scope ``` Absent file = empty layer. Unknown keys warn and load (a file written for a newer CLI works on an older one). `$OMNIGRAPH_CONFIG=<path>` stands in for `--config` (the flag wins) in both the CLI and the server. +#### Scopes & profiles (RFC-011) + +A command resolves a **scope** — a server, a cluster, or a store — then selects a +graph in it; the served-vs-direct access path is derived from the scope, not +toggled. The scope comes from one of (highest precedence first): an explicit +address (a positional URI, `--target`, `--server`, or `--store <uri>`); a named +`--profile <name>` (or `$OMNIGRAPH_PROFILE`); or the flat `defaults.server` + +`defaults.default_graph`. A **profile** binds exactly one of `server` / `cluster` +/ `store` plus an optional default graph — config data, not state: every command +resolves its scope fresh, there is no sticky "current" mode. + +- `--store <uri>` addresses a single graph's storage directly (ad-hoc / break-glass). +- A `cluster`-bound profile reaches `optimize` / `repair` / `cleanup` for a managed + graph (resolving its storage root from `clusters:`), the same as + `--cluster <root> --cluster-graph <id>`. +- A `server`-bound scope on a maintenance verb, or a `cluster`-bound scope on a + data verb, is rejected with a message pointing at the right addressing. + +This model **coexists** with the legacy addressing (`--uri` / `--target` / +`--cluster-graph` / `omnigraph.yaml`) — nothing is removed yet; an explicit legacy +address always wins. + #### Credentials keyed by server name `omnigraph login <name>` stores a bearer token in