mirror of
https://github.com/ModernRelay/omnigraph.git
synced 2026-06-24 02:38:06 +02:00
615 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Ragnor Comerford
|
adc36adf32
|
Merge branch 'main' into ragnorc/omnigraph-mcp-crate
Folds in v0.7.1 (release #290 + optimize/write-path/internal-table-compaction fixes #288/#291/#297) under the MCP branch. Conflict resolutions (5 files): - crates/omnigraph-server/Cargo.toml: take main's 0.7.1 path-dep constraints; keep our omnigraph-mcp dep (bumped to 0.7.1) + http dep. - crates/omnigraph-server/src/handlers.rs: keep our server_list_queries doc-comment (exposed @mcp(expose) subset, invoke_query-gated) — it supersedes main's pre-@mcp(expose) text, since this branch adds the per-query expose flag. - docs/user/operations/server.md: keep our GET /queries description (invoke_query gate + @mcp(expose) exposure) over main's read-gated/list-all text. - docs/dev/index.md: keep both in-flight RFC rows; renumber this branch's tenancy RFC 013 -> 014 (rfc-014-tenancy-cells.md) since main now owns RFC-013 (rfc-013-write-path-latency.md). Title + index link updated; link-check green. - openapi.json: regenerated from merged source (OMNIGRAPH_UPDATE_OPENAPI=1) — now info.version 0.7.1 with our invoke_query/@mcp schema. Coherence: omnigraph-mcp bumped 0.7.0 -> 0.7.1 to match the workspace; Cargo.lock updated. cargo build --workspace green; server/mcp/api-types/compiler suites green (schema_routes.rs reopen-after-apply flakes under parallel IO on a near-full disk, passes single-threaded — a pre-existing main test, unchanged by the merge). |
||
|
Ragnor Comerford
|
6d4606a830
|
fix(engine): optimize survives a cross-process write race on the same table (#297)
* test(engine): cross-process optimize-vs-write race — RED
Two regression tests for the prod bug: a direct `optimize` process racing a
served write on the same table fails, because the in-process write queue does
not serialize across processes and the data-table optimize path has no retry.
- optimize_survives_concurrent_insert_advancing_manifest: a concurrent insert
advances the manifest while optimize is paused between compact and publish;
optimize's equality-CAS publish then fails "expected X but current Y".
- optimize_survives_concurrent_delete_before_compaction: a concurrent delete
commits before optimize compacts; Lance rebases the compaction past it
cleanly, so optimize again fails the publish CAS (the genuine Lance
Rewrite-vs-Rewrite overlap is rarer and shares the internal path's retry).
Both fail today with ExpectedVersionMismatch. Adds the `optimize.before_compact`
failpoint seam + a wait_for_sidecar helper; serializes the optimize failpoint
tests (shared failpoint name). The fix lands next.
* fix(engine): optimize survives a cross-process write race on the same table
The data-table optimize path trusted the in-process write queue and skipped a
retry, so a CLI `optimize` racing a served write (separate processes = separate
queues) failed: either the Lance Rewrite lost ("preempted by concurrent Update")
or the manifest publish lost the strict equality CAS ("expected X but current Y").
Unify both compaction paths on the internal path's reopen+replan shape, with a
two-level retry that matches the two failure points:
- Outer loop (reopen+replan): a genuine Lance Rewrite-vs-Update/Delete same-
fragment conflict means our compaction did not commit — reopen at the new HEAD
and re-plan. Lance rebases the common disjoint case (a concurrent insert/delete
on other fragments) for free, so this fires only on real overlap.
- Inner loop (Phase C, monotonic publish): the manifest advanced between our
compaction and our publish. The compaction is already committed at Lance HEAD N,
so we must NOT reopen (that trips the HEAD>manifest drift guard on our own work).
Re-read the current manifest version C: if C >= N the manifest already includes
our compaction (versions are linear) — no-op; else fast-forward to N. Monotonic,
not the strict equality CAS that manufactured the conflict.
The Phase-A sidecar is written once and reused across reopen attempts (every
Phase-B commit is content-preserving, so recovery rolls the observed HEAD forward
or safely rolls the compaction back). The in-process queue is kept — it is now an
in-process contention reducer, not the cross-process correctness guard. Shares the
COMPACTION_RETRY_BUDGET constant + is_retryable_lance_conflict with the internal
path; adds is_retryable_manifest_conflict for the publish loop. No writer_epoch.
Turns the prior commit's two race tests green.
* docs(rfc-013): two-op-class principle + the found+fixed optimize-vs-write race
§6.6 records the maintenance vs logical op-class distinction (maintenance commutes
→ Lance rebase + reopen/replan + monotonic manifest fast-forward, no writer_epoch;
logical → strict cross-process OCC + epoch) and the prod optimize-vs-served-write
race that motivated it, now landed. Adds the matching mechanic row to §4.2.
* fix(engine): retry must not misclassify optimize's own HEAD drift
Review catch on the cross-process optimize fix: the outer retry loop re-ran the
`lance_head > manifest` drift guard every iteration. After a partial Phase-B commit
(the auto_cleanup strip or compaction commits, then a later op hits a retryable
conflict), the reopened attempt saw HEAD ahead of the manifest — from OUR own
sidecar-covered work, not an external writer — and deleted the sidecar + returned
`skipped_for_drift`, stranding uncovered drift that then needs `repair`.
Track `head_advanced` (did one of our Phase-B ops already commit). The drift guard
now fires only when `!head_advanced` (genuine pre-existing external drift); once we
have advanced HEAD, a reopened HEAD>manifest is our work that the monotonic publish
fast-forwards. The no-op early-return likewise publishes prior committed work instead
of dropping it when `head_advanced`.
Regression test `optimize_retry_does_not_misclassify_own_head_drift` injects one
retryable reindex conflict after the compaction commits (new `optimize.inject_
reindex_conflict` seam); red→green verified by negative control (reverting the gate
reproduces `skipped_for_drift: Some(DriftNeedsRepair)`).
Also de-flake `optimize_survives_concurrent_insert_advancing_manifest`: pause at
`before_compact` (not post-compact) so the concurrent insert lands while HEAD==
manifest — otherwise it could race optimize's committed-but-unpublished compaction
and hit the write-path "HEAD ahead of manifest" guard.
* fix(engine): optimize publish converges on retry-budget exhaustion
Review catch (greptile): the monotonic Phase-C publish loop returned an error on its
final iteration's retryable manifest conflict, even though that conflict can itself
mean a concurrent writer published a version that already includes our (content-
preserving) compaction — i.e. the postcondition ("the manifest reflects our
compaction") is already met. Recovery covered it (no data loss), but the operator
saw a spurious error and had to re-run.
Restructure the loop to re-read `current` on every retryable conflict and, on budget
exhaustion, do a final `current >= state.version` convergence check before surfacing
the error — the §6.6 "postcondition is the state, not winning the CAS" principle.
Factor the repeated current-version read into `current_manifest_version`.
|
||
|
Ragnor Comerford
|
5cfae9acc1
|
docs(rfc-013): latency = (serial_hops + ops/concurrency)·RTT — concurrency-cap correction + Lance-metadata comparison (#292)
* feat(engine): compact the internal __manifest/_graph_commits tables in optimize `optimize` iterated node/edge catalog tables only, so the two internal system tables (`__manifest`, `_graph_commits`) accumulated one fragment per commit and were never compacted -- making every write's metadata scan O(fragments), which grows forever on a long-lived graph (RFC-013 step 2). `optimize_all_tables` now also compacts both internal tables via a new `compact_internal_table`. They are not catalog-tracked (readers open them at their latest Lance HEAD), so it is a much simpler path than `optimize_one_table`: compact in place, no manifest publish (nothing to publish to), no recovery sidecar (a single atomic Lance commit -- no HEAD-before-publish gap), and no optimize_indices (they carry no Lance index, only object_id's unenforced-PK metadata). No application lock: Lance's compact_files auto-retries its Rewrite against any concurrent writer (the canonical LanceDB pattern; Rewrite vs Append is compatible, vs Update a retryable same-fragment conflict Lance rebases), and a coordinator refresh afterwards makes the warm handle observe the compacted HEAD. Compacts both tables even though Phase 7 (iss-991) will later fold _graph_commits into __manifest -- a one-call throwaway for the full interim win; __manifest compaction is also the prerequisite for Phase 7's graph_head contention. Cleanup (version GC) of the internal tables is deliberately NOT included here: it needs the Q8 cleanup-resurrection watermark first (deferred). maintenance.rs: optimize now returns 6 stats (4 data + 2 internal); adds optimize_compacts_internal_tables (sheds fragments, leaks no recovery sidecar, graph coherent for reads + strict writes after). * test(engine): un-ignore the internal-table scan LOCK (step 2 acceptance) `internal_table_scans_are_flat_in_history` was the RED, #[ignore]'d acceptance gate staged in PR #288. With internal-table compaction landed, a write's __manifest/_graph_commits scan is flat in commit-history depth on a compacted graph (measured __manifest 4->2, _graph_commits 7->3 across depth 10->100, vs the pre-step-2 RED 34->214 / 29->207). The test now compacts at each depth before measuring and runs green every-PR. * docs: RFC-013 step 2 internal-table compaction landed - invariants.md: close the compaction half of the read-path-rederivation known gap (optimize now compacts the internal tables; cleanup half still deferred). - maintenance.md: optimize covers __manifest/_graph_commits (no publish, no sidecar); not yet in cleanup. - rfc-013 §9: split step 2 into 2a (compaction, landed) and 2b (cleanup + Q8 watermark, deferred — debated; MTT-overlap + hot-path liability). - testing.md: the internal-table LOCK is now green every-PR. * fix(engine): guard absent _graph_commits + always compact internal tables Addresses PR #291 review findings: - Greptile (P1): optimize unconditionally opened `_graph_commits` for compaction, but a graph can validly have none (the coordinator opens it as `Option`, gated on `storage.exists`, for graphs predating the commit graph). `Dataset::open` on the absent table errored and failed the whole optimize. Guard the `_graph_commits` compaction with the same `storage_adapter().exists()` check the coordinator uses; `__manifest` always exists so it stays unguarded. Regression test `optimize_tolerates_absent_graph_commits_table` (empty graph so no publish recreates the table before the guard). - Cursor (low): the `table_tasks.is_empty()` early return skipped internal-table compaction for a schema with no node/edge types. Removed it so the internal tables are compacted regardless of the data-table set. - Codex (auto-cleanup, P1): documented — `compact_files` commits with a default `CommitConfig` (no skip_auto_cleanup) and `CompactionOptions` exposes no override, so on a graph storing an *on* auto_cleanup config the commit would fire version GC. Both internal tables are created with `auto_cleanup: None`, so new graphs are safe; the only exposure is pre-fix upgraded graphs, identical to the existing data-table optimize path, with step 2b's watermark as the comprehensive guard. Added a comment in `compact_internal_table` recording this. * docs(rfc-013): serial-hop correction — wall-clock is the ~110-hop backbone, not op count Latency-slope measurement on the deployed edge binary ( |
||
|
Ragnor Comerford
|
f2b792e0ae
|
(feat): compact the internal manifest/commit-graph tables in optimize (#291)
* feat(engine): compact the internal __manifest/_graph_commits tables in optimize
`optimize` iterated node/edge catalog tables only, so the two internal system
tables (`__manifest`, `_graph_commits`) accumulated one fragment per commit and
were never compacted -- making every write's metadata scan O(fragments), which
grows forever on a long-lived graph (RFC-013 step 2).
`optimize_all_tables` now also compacts both internal tables via a new
`compact_internal_table`. They are not catalog-tracked (readers open them at
their latest Lance HEAD), so it is a much simpler path than `optimize_one_table`:
compact in place, no manifest publish (nothing to publish to), no recovery
sidecar (a single atomic Lance commit -- no HEAD-before-publish gap), and no
optimize_indices (they carry no Lance index, only object_id's unenforced-PK
metadata). No application lock: Lance's compact_files auto-retries its Rewrite
against any concurrent writer (the canonical LanceDB pattern; Rewrite vs Append
is compatible, vs Update a retryable same-fragment conflict Lance rebases), and a
coordinator refresh afterwards makes the warm handle observe the compacted HEAD.
Compacts both tables even though Phase 7 (iss-991) will later fold _graph_commits
into __manifest -- a one-call throwaway for the full interim win; __manifest
compaction is also the prerequisite for Phase 7's graph_head contention. Cleanup
(version GC) of the internal tables is deliberately NOT included here: it needs
the Q8 cleanup-resurrection watermark first (deferred).
maintenance.rs: optimize now returns 6 stats (4 data + 2 internal); adds
optimize_compacts_internal_tables (sheds fragments, leaks no recovery sidecar,
graph coherent for reads + strict writes after).
* test(engine): un-ignore the internal-table scan LOCK (step 2 acceptance)
`internal_table_scans_are_flat_in_history` was the RED, #[ignore]'d acceptance
gate staged in PR #288. With internal-table compaction landed, a write's
__manifest/_graph_commits scan is flat in commit-history depth on a compacted
graph (measured __manifest 4->2, _graph_commits 7->3 across depth 10->100, vs the
pre-step-2 RED 34->214 / 29->207). The test now compacts at each depth before
measuring and runs green every-PR.
* docs: RFC-013 step 2 internal-table compaction landed
- invariants.md: close the compaction half of the read-path-rederivation known
gap (optimize now compacts the internal tables; cleanup half still deferred).
- maintenance.md: optimize covers __manifest/_graph_commits (no publish, no
sidecar); not yet in cleanup.
- rfc-013 §9: split step 2 into 2a (compaction, landed) and 2b (cleanup + Q8
watermark, deferred — debated; MTT-overlap + hot-path liability).
- testing.md: the internal-table LOCK is now green every-PR.
* fix(engine): guard absent _graph_commits + always compact internal tables
Addresses PR #291 review findings:
- Greptile (P1): optimize unconditionally opened `_graph_commits` for compaction,
but a graph can validly have none (the coordinator opens it as `Option`, gated on
`storage.exists`, for graphs predating the commit graph). `Dataset::open` on the
absent table errored and failed the whole optimize. Guard the `_graph_commits`
compaction with the same `storage_adapter().exists()` check the coordinator uses;
`__manifest` always exists so it stays unguarded. Regression test
`optimize_tolerates_absent_graph_commits_table` (empty graph so no publish
recreates the table before the guard).
- Cursor (low): the `table_tasks.is_empty()` early return skipped internal-table
compaction for a schema with no node/edge types. Removed it so the internal
tables are compacted regardless of the data-table set.
- Codex (auto-cleanup, P1): documented — `compact_files` commits with a default
`CommitConfig` (no skip_auto_cleanup) and `CompactionOptions` exposes no override,
so on a graph storing an *on* auto_cleanup config the commit would fire version
GC. Both internal tables are created with `auto_cleanup: None`, so new graphs are
safe; the only exposure is pre-fix upgraded graphs, identical to the existing
data-table optimize path, with step 2b's watermark as the comprehensive guard.
Added a comment in `compact_internal_table` recording this.
* fix(engine): retry publish on RetryableCommitConflict (compaction vs publish)
Step 2 compacts `__manifest` with no app-level lock (Lance OCC arbitrates,
validated against LanceDB + the lance-7.0.0 conflict resolver). compact_files'
`Operation::Rewrite` auto-retries 20x (CommitConfig default num_retries=20), so a
live publish usually wins the race and the compaction rebases. But the publish
runs its merge-insert with conflict_retries(0) = one rebase attempt; if the
compaction commits first AND the merge touched a fragment the Rewrite rewrote,
Lance preempts the publish with `Error::RetryableCommitConflict` — a DIFFERENT
variant from the row-level `TooMuchWriteContention` the publisher already retries.
Left unhandled, that surfaces a transient error to the caller, i.e. a maintenance
compaction (physical op) failing a live write (logical op) — invariant 7.
Map `LanceError::RetryableCommitConflict` to a new
`ManifestConflictDetails::RetryableCommitConflict` and treat it as retryable in the
publisher's outer loop (reload fresh state + re-merge), alongside
RowLevelCasContention. `ExpectedVersionMismatch` still propagates (a genuine
expectation break must not be blindly retried). This also hardens multi-process
concurrent writers generally, not just compaction.
Normal publishes are insert-only (new object_ids -> new fragments, disjoint from
rewritten old ones), so the conflict is rare; the guard covers the
same-fragment-update edge and multi-process writers. Unit tests in publisher.rs
pin the mapping + the retry-predicate contract.
* revert: publisher RetryableCommitConflict handling (it was the wrong side)
Reverts
|
||
|
Andrew Altshuler
|
fff441196c
|
docs(dev): update coherence ledger — cookbooks drift resolved, omnigraph-ts mechanism (#294)
Some checks failed
CI / Classify Changes (push) Has been cancelled
CI / Check AGENTS.md Links (push) Has been cancelled
CI / Container Entrypoint (push) Has been cancelled
Release Edge / Prepare edge release (push) Has been cancelled
CI / Test Workspace (push) Has been cancelled
CI / Test omnigraph-server --features aws (push) Has been cancelled
CI / RustFS S3 Integration (push) Has been cancelled
Release Edge / Build edge omnigraph-linux-x86_64 (push) Has been cancelled
Release Edge / Build edge omnigraph-macos-arm64 (push) Has been cancelled
Release Edge / Build edge omnigraph-windows-x86_64 (push) Has been cancelled
Release Edge / Smoke Windows installer (push) Has been cancelled
- omnigraph-cookbooks `bearer_token_env` chain: RESOLVED by cookbooks PR #26 (deleted docs/best-practices.md in the 0.7 restructure). - omnigraph-ts catalog `mcp.expose` description: documented why there is no hand-fix — the SDK syncs openapi.json from a *tagged* omnigraph release, and the fix landed on main after the v0.7.1 tag, so it flows in on the next SDK version bump (v0.7.2+) rather than an out-of-band patch. Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
9c792649e2
|
docs(user): coherence cleanup aligned with 0.7.1 (#293)
* docs(cli): fix cluster apply semantics — converges graphs+schema, not config-only `cluster apply` creates graphs, applies schema updates (soft drops), writes stored-query/policy catalog resources, and executes approved graph deletes in one ordered run. Both the user docs and the shipped CLI help text still described it as a "Stage 3A" config-only (query/policy) subset that defers graph/schema changes "to a later stage" — wrong since the graph/schema executor landed. - docs/user/cli/reference.md: rewrite the cluster paragraph to describe apply's actual converge behavior; keep deferred for the genuinely-unsupported case (standalone schema deletes); drop the stale "Stage 3A" / "reserved for later stages" framing. - crates/omnigraph-cli/src/cli.rs: fix the `cluster apply` help text to match. Part of the docs/user coherence cleanup (docs/dev/docs-issues.md, P1). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 * docs(server): align stored-query exposure with cluster-only behavior server.md documented a per-query expose knob ("`mcp.expose` defaults to true; set `mcp: { expose: false }` to hide from the catalog") that does not exist in the only deployment mode. Cluster-only serving lists every stored query: the cluster registry has no expose field (`QueryConfig { file }`) and the boot bridge hardcodes `expose: true` for all cluster queries (omnigraph-server settings), and there is no GQ-level expose annotation. This contradicted clusters/config.md, which already states the correct behavior. Replace the knob bullet with the cluster truth (every applied query is listed; per-query exposure may become a Cedar-policy decision later) and drop the "`mcp.expose` stored queries" phrasing from the catalog description, the endpoint table, and the intro. The `mcp_expose` JSON catalog field is unchanged (still emitted, always true in cluster mode). Part of the docs/user coherence cleanup (docs/dev/docs-issues.md, P1). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 * docs(schema): split direct/embedded vs cluster-managed schema apply schema/index.md claimed `allow_data_loss` is "honored uniformly across transports" and listed HTTP `POST /schema/apply` among them. But that route is 409-disabled for cluster-backed serving (already documented in server.md), and cluster-managed graphs evolve only through `cluster apply` with soft drops — there is no cluster HTTP data-loss path. Scope the data-loss flag to the direct/embedded path (`schema apply --store`, SDK), and add a paragraph: cluster-managed graphs use `cluster apply` (soft drops only); HTTP `POST /schema/apply` is 409 for cluster serving; direct apply against a cluster-managed path is refused. Cross-refs server + cluster docs. Part of the docs/user coherence cleanup (docs/dev/docs-issues.md, P2). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 * docs(server): document /load as canonical in limits + admission prose The endpoint table already listed both `/load` (canonical) and `/ingest` (deprecated alias) at 32 MB, but the admission-control, body-limit, rate-limit, and manifest-conflict prose named only `/ingest` — and the constants page called the limit "Ingest body limit". Add `/load` alongside (or ahead of) `/ingest` everywhere, and rename the constant to "Load (bulk-write) body limit" noting the `/ingest` alias shares it. Part of the docs/user coherence cleanup (docs/dev/docs-issues.md, P2). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 * docs(cli): drop stale bearer-token keys + fix version string The "Bearer token resolution (CLI)" section still listed removed omnigraph.yaml keys (`graphs.<name>.bearer_token_env`, `auth.env_file`) — config surfaces that no longer exist and that implied plaintext tokens in config. Replace it with a pointer to the keyed-credential model documented above (`OMNIGRAPH_TOKEN_<NAME>` → `~/.omnigraph/credentials` → `OMNIGRAPH_BEARER_TOKEN`). Also fix the `version` row: the CLI prints 0.7.x, not 0.3.x. Part of the docs/user coherence cleanup (docs/dev/docs-issues.md, P2 + smaller). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 * docs: route-spelling note + drop stale stage/deferred crumbs - server.md: add a one-line note that the per-graph subsections name routes in shorthand (`GET /queries`, `POST /query`, `POST /mutate`, `POST /queries/{name}`) but every one is served under `/graphs/{id}/…` — the endpoint table is already fully-qualified. - clusters/config.md: redefine the `deferred` plan disposition as an unsupported change (e.g. a standalone schema delete) instead of "graph/schema change, later phase" (graph creates and schema updates apply now); drop the "Stage 2C" label from the lock-recovery note. - search/indexes.md: `ingest --mode merge` → canonical `load --mode merge`. Part of the docs/user coherence cleanup (docs/dev/docs-issues.md, P2 + smaller). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 * docs(dev): track user-docs coherence ledger; mark 2026-06-20 findings resolved Convert the scratch review notes into a tracked living ledger and link it from the dev index. All ten findings from the 2026-06-20 docs/user sweep are validated and fixed in this branch (P1 cluster-apply semantics + stored-query exposure; P2 schema-apply paths, /load canonical, bearer-token keys, route shorthand; plus version/ingest/deferred/stage crumbs). The verification grep checklist is retained for future audits. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 * docs(api): align GET /queries OpenAPI contract with cluster-only behavior Greptile P1 on #293: the prose fix in server.md left the OpenAPI surface stale. The utoipa annotations (handlers.rs, omnigraph-api-types QueriesCatalogOutput) still described the catalog as "the `mcp.expose == true` subset", and those drive the checked-in openapi.json — so SDK consumers read a contract the cluster-only server does not honor (it lists every stored query). Update the three Rust doc-comment/annotation strings to "every stored query" and regenerate openapi.json (OMNIGRAPH_UPDATE_OPENAPI=1; drift test green) in the same change, per AGENTS.md rule 4. Ledger updated: this finding resolved, plus the cross-repo drift it surfaced (omnigraph-ts generated spec/types and omnigraph-cookbooks best-practices bearer_token_env) tracked as open follow-ups. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ragnor Comerford
|
8dab7e2e61
|
test(mcp): harden symptomatic MCP fixes to construction-enforced
Four guards/refactors that convert previously convention-enforced MCP
invariants into ones a future edit can't silently break:
- H1 (loopback Host set): standalone.rs surface-guard asserts our loopback
allow-list is a superset of rmcp's own default set, so an rmcp bump that
adds a loopback form turns red instead of 403'ing that client. Pins the
::1 regression by construction rather than by a hand-kept literal list.
- H2 (one stored-query gate): extract a single invoke_query_request() in
handlers.rs used by GET /queries, POST /queries/{name}, and (imported) the
MCP tools/list + tools/call stored paths. REST and MCP can no longer drift
on which Cedar action governs the catalog. Deletes mcp.rs's local copy.
- H3 (expose chokepoint): tests/mcp.rs source-walk guard bans .lookup( and
registry.iter( in mcp.rs, so the agent surface can only reach stored
queries through exposed()/exposed_by_name() — an @mcp(expose:false) query
cannot leak back into tools/list via the expose-ignoring registry methods.
- H4 (list-gate relaxation lower-bound): a permit-all actor must see every
built-in tool, pinning the other end of the list-gate fix (no callable
tool may be hidden from tools/list).
cargo test -p omnigraph-mcp -p omnigraph-server green (the lone schema_routes
flake was disk-pressure during the clean build; passes in isolation).
|
||
|
Ragnor Comerford
|
f6d2cc03e3
|
write-path cost gate + opener bypass (#288)
* docs(rfc): RFC-013 write-path latency design + index link * perf(engine): open write-path tables directly, bypassing the namespace builder Write opens routed through DatasetBuilder::from_namespace, whose describe_table opened the whole dataset just to return a location and then re-resolved the latest version — an O(commit-depth) double latest-resolution per table open that missed Lance's O(1) version-hint fast path. On an object store this dominated write latency (~70%, RFC-013 section 2.4). TableStore::open_dataset_head_for_write now delegates to the direct opener (open_dataset_head: Dataset::open by URI + checkout_branch, routed through the tracked opener so cost tests can count it; a no-op in production). The manifest already holds every sub-table's location, so the namespace catalog lookup was redundant; ensure_expected_version still validates head == pinned for strict ops. This completes PR #268's open-by-location migration on the write side. With both reads (PR #268) and now writes bypassing it, nothing in production routes through the per-table Lance namespace. The dead open chain (load_table_from_namespace, open_table_head_for_write) is deleted and the StagedTableNamespace contract apparatus is gated #[cfg(test)], mirroring the already-test-only read namespace; __manifest commit coordination (GraphNamespacePublisher) is a separate component and is unaffected. See docs/dev/rfc-013-write-path-latency.md sections 2.4 and 9 (step 3a). * test(engine): write-path cost-budget gate on a shared harness Adds tests/helpers/cost.rs, a store-agnostic cost harness (IoCounts/StagedCounts, measure/measure_with_staged, assert_flat, local_graph/s3_graph) that the read-side warm_read_cost.rs, write_cost.rs, and write_cost_s3.rs share, so the IOTracker / task-local plumbing lives in exactly one place instead of duplicated per test. write_cost.rs (local, every-PR) gates the internal-table scan term flat in commit-history depth (a RED #[ignore]'d LOCK, the acceptance for bringing the internal tables into compaction) plus green guards: a single insert's data writes are bounded, a per-write read-op ceiling fails the moment a round-trip is added, and a keyed insert routes through stage_merge_insert once with no stage_append or vector-index build. write_cost_s3.rs (bucket-gated, rustfs CI) gates the data-table opener term flat across depth — the object-store-RPC phenomenon local FS cannot reproduce, and the red->green proof of the opener bypass. Wired into the rustfs_integration CI job and its path filter. Guards the "hot-path cost is bounded by work, not history" invariant on writes. See docs/dev/rfc-013-write-path-latency.md section 5.1, docs/dev/testing.md. * docs(rfc): RFC-013 step 3a landed; write-skew coupling; cost-gate test map - Section 9: mark step 1 (gate + harness) and step 3a (opener bypass) landed; record the per-table namespace retirement to test-only and the corrected measurement note (the opener win is S3-only; the local data-table growth is the merge-insert/RI fragment scan, a compaction term, not the opener). - Sections 7.1/6/11/5.5/10: correct the cross-table write-skew analysis after a prototype proved the scoped expected-set fix is a no-op against the per-object_id manifest (disjoint writers never share a row, so Lance never conflicts, the publisher never retries, and the expected check is a non-atomic pre-check evaluated once against stale state). The fix needs a shared contention row (Phase-7 graph_head / a minimal head row / commit-time re-validation), so it is coupled to that row, not standalone; that contention is load-bearing for correctness, not a drawback. Split the concurrent face (read-set + head) from the sequential face (inbound-RI validation on node removal) -- two different fixes. - testing.md: add write_cost.rs / helpers/cost.rs / write_cost_s3.rs to the test map; document the local-vs-S3 backend split; extend the cost-budget checklist item to the write/open path and point at the shared harness. * test(engine): isolate the opener in the S3 cost gate; fail loud on S3 setup errors Addresses two PR review findings on the bucket-gated write_cost_s3 gate: - The data-table opener was not isolated: `data_reads` also counts the merge-insert/RI scan, which reads O(fragment-count) and so grows with history for a different reason (compaction's domain, not the opener) -- the same term that made the local data-table count grow. The flat assertion would false-RED or misattribute scan growth to the opener on rustfs. Fix: compact (db.optimize) before each measurement so the table holds ~1 fragment, bounding the scan and leaving the opener's latest-version resolution as the only history-varying term. Compaction preserves version history, so the opener still faces a deep _versions/ chain -- the thing under test. - s3_graph used `.ok()?`, so when OMNIGRAPH_S3_TEST_BUCKET was set but the store was down/misconfigured, init/seed failures collapsed to None and the gate skipped + passed vacuously. Fix: skip only when the bucket env var is absent; once it is set, init/seed failures panic (mirrors tests/s3_storage.rs). * test(engine): isolate the S3 opener with a per-prefix IO probe (correct-by-design) Replaces the fixture-bounded isolation (compact-before-measure) from the prior commit with the root fix: a path-classifying ObjectStore wrapper (PrefixCounter) that attributes each data-table read to the opener term (_versions/.manifest) vs the scan term (data/*.lance). IoCounts now exposes data_opener_reads / data_scan_reads, so write_cost_s3 asserts the opener flat *directly* -- no compaction or fixture massaging, and the assertion measures the opener, not the conflated total. Closes the "harness conflates two IO terms" class: any cost test (read or write) can now isolate the opener. PrefixCounter implements only the object_store 0.13 core ObjectStore methods; the convenience surface (get/put/head/...) routes through get_opts/put_opts via ObjectStoreExt's blanket impl, so every read/write is still counted. Validated locally (every-PR) by write_cost::data_table_reads_split_into_flat_opener_ and_growing_scan: opener stays flat (7 -> 3) while scan grows (11 -> 91) and opener + scan == data_total exactly -- proving the classifier and confirming the local data-table growth is the fragment scan, not the opener. warm_read_cost (12 tests) stays green under the shared-harness change. * refactor(tests): remove cost-harness duplication and namespace cfg(test) noise Branch self-review (no behavior change) — pay down three liabilities the write-path work left: - warm_read_cost.rs kept its own probes() (three IOTrackers + a QueryIoProbes + a probe counter) and read raw .stats().read_iops — duplicating the shared helpers::cost harness this branch introduced. Migrated all 12 tests onto measure()/IoCounts; deleted the local probes(). (This also makes IoCounts' version_probes field used rather than dead.) - insert_cost was copy-pasted verbatim into write_cost.rs and write_cost_s3.rs. Hoisted to helpers::cost::measure_insert so the measured write is defined once. - The per-table Lance namespace (namespace.rs) became entirely test-only after step 3a, but was gated with ~22 per-item #[cfg(test)] attributes. Collapsed to a single `#[cfg(test)] mod namespace;` and stripped the per-item attributes; merged the import groups the gating had split. Verified: lib in-source 162 passed; write_cost 4 + warm_read_cost 12 passed; forbidden_apis passed. |
||
|
Andrew Altshuler
|
b38b36e48f
|
release: v0.7.1 (#290)
Patch release over v0.7.0: three correctness fixes (#283 camelCase filters, #284 cluster-apply crash-loop, #277 branch-merge OOM on embedding tables), the #280 queries-list catalog-metadata improvement, and the #268 warm-read perf fix. No breaking changes, no on-disk format change, no migration. Version coherence: all 7 crate manifests + path-dep constraints, Cargo.lock, openapi.json, and AGENTS.md surveyed version bumped 0.7.0 -> 0.7.1. Full workspace gate green (1472 tests). Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ragnor Comerford
|
fbf455a250
|
Merge branch 'main' into ragnorc/omnigraph-mcp-crate
Bring the MCP feature branch up to date with main (14 commits). One conflict — compiler/parser.rs: main's `NanoError` → `CompilerError` rename vs this branch's `@mcp` / per-param `@description` parser additions; resolved by keeping the new parsing under the renamed error type. The CLI `queries list` change (#280, surfacing `@description`/`@instruction`) auto-merged with this branch's `mcp_expose`/`tool_name` columns. |
||
|
Andrew Altshuler
|
2d34d7c432
|
test(engine): pin camelCase @index → scalar-index routing (#283 follow-up) (#286)
Greptile P2 on #285: the #283 tests prove correctness (right rows, type-level coercion) but not that a camelCase @index equality actually reaches the scalar-index path — a result-only test passes on a silent full-scan fallback, exactly the gap testing.md warns about and the bug-case-fix.md validation checklist (step 5) promised to close. Add lance-surface Guard 20 (mirrors Guard 16): build a BTREE on a camelCase column and assert the scan plan contains `ScalarIndexQuery` under the fix's case-preserving `ident()` expr, and that the pre-fix `col()` expr fails to plan (it normalizes `repoName` → a nonexistent `reponame`). A regression that breaks camelCase index routing — or reverts to `col()` — turns this red instead of degrading to a full scan. The existing e2e (`camelcase_property_filter_executes`) already guards the engine call-site (a `col()` revert errors there). Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
57348cf7fa
|
fix(engine): preserve identifier case in filter pushdown (#283) (#285)
* test(engine): regression tests for #283 camelCase property filters Red against current code. A query (or chained mutation) that filters on a camelCase schema field lints and plans cleanly but fails at run time with "No field named reponame" because the identifier's case is destroyed at the engine->Lance boundary. Coverage added: - query.rs unit: ir_filter_to_expr on a camelCase property must emit an Expr::Column named `repoName`, not `reponame` (red); plus a green coercion guard that a camelCase int column still gets a coerced literal. - mutation.rs unit: predicate_to_sql must emit the column UNQUOTED and case-preserved (green guard documenting the committed-scan contract). - literal_filters.rs e2e: a camelCase @index field with an inline-binding pushdown filter returns the seeded row (red — read pushdown). - writes.rs e2e: an update+delete on a camelCase predicate, and a chained update that re-reads the pending side of scan_with_pending by the same camelCase predicate (red — pending MemTable scan). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 * fix(engine): preserve identifier case in filter pushdown (#283) Two engine->Lance boundaries lowercased camelCase column identifiers, breaking any filter on a camelCase schema field even though the IR, compiler, projection, and in-memory filtering all preserve case. Read pushdown (exec/query.rs, ir_expr_to_expr): build the column reference with datafusion::prelude::ident() instead of col(). col() routes through SQL identifier normalization and lowercases an unquoted identifier (`repoName` -> `reponame`); ident() builds an unqualified, case-preserved Column. Property refs here are always bare column names, so there is no qualified-name handling to lose. No-op for the lowercase columns that work today. Pending mutation scan (table_store.rs, scan_pending_batches): the committed-scan consumer (Lance Scanner::filter(&str)) preserves an unquoted identifier's case but treats a double-quoted "col" as a string literal, so predicate_to_sql must keep the column unquoted. The pending side splices that same unquoted predicate into a DataFusion `SELECT ... WHERE`, which would lowercase it. Make that path case-preserving by disabling sql_parser.enable_ident_normalization on its SessionContext rather than quoting (quoting would match zero committed rows). predicate_to_sql gains only a clarifying comment; its emitted string is unchanged. Full engine suite green (579 tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 * docs(dev): case study for #283 camelCase filter bug Record the root cause, the two-boundary fix (read pushdown col→ident; pending mutation scan ident-normalization off), and why the obvious symmetric "quote the column" fix is wrong (Lance reads a double-quoted column as a string literal and silently matches zero committed rows). Linked from a new "Case Studies" section in the dev index so the link check passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ1Hf4eXLsJmeLUkTYBEw7 --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
3feb23af05
|
feat(cli): surface stored-query @description/@instruction in queries list (#280)
* test: e2e coverage for @description/@instruction surfaces Add end-to-end tests pinning the two annotation surfaces as they exist today, at their real boundaries: - engine (lifecycle.rs): schema-level @description (node/edge/property) and @instruction (node/edge) persist verbatim into the on-disk _schema.ir.json through Omnigraph::init; property-level @instruction aborts init and writes no schema IR. - server (stored_queries.rs): query-level @description/@instruction on a stored query surface as typed QueryCatalogEntry fields over GET /queries, and a query declaring neither omits both fields. No behavior change — these document the current contract. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): surface stored-query @description/@instruction in `queries list` A stored query's @description/@instruction are its catalog metadata — what it does and how to invoke it. The HTTP GET /queries catalog already carries them, but `omnigraph queries list` dropped both fields in human and --json output even though they were available on the registry entry. Carry description/instruction on QueriesListItem (Option, skipped when None) and copy them from the query decl. Human output prints an indented `description:` / `instruction:` line per query when present; --json includes the fields when present and omits them otherwise — matching the HTTP catalog shape documented in docs/user/operations/server.md. Tests (cli_queries.rs): a query with both annotations surfaces them in human + --json; a query with neither prints no annotation lines and omits both JSON fields. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cli): document `queries list` output incl. description/instruction Per AGENTS.md maintenance Rule 1, document the user-visible `queries list` output alongside the field addition. The `queries` command family had no row in the CLI reference top-level table; add one covering `list` (human + --json shapes, with description/instruction shown only when declared, matching the HTTP GET /queries catalog) and `validate`. Addresses the Greptile P2 review finding on PR #280. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): indent multiline stored-query annotations in `queries list` A `@description`/`@instruction` value can be multiline (GQ string literals admit newlines), which made the human `queries list` output break back to the left margin on continuation lines. Indent continuation lines to align under the first via a `print_query_annotation` helper. Addresses review feedback from @martin-g on PR #280. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ragnor Comerford <ragnor.comerford@gmail.com> |
||
|
Andrew Altshuler
|
7fd23c54a3
|
fix(cluster): stop cluster-apply crash-loops from the recovery-sidecar trap (#284)
* fix(cluster): stop cluster-apply crash-loops from the recovery-sidecar trap A `cluster apply` carrying a schema change against a graph that has non-main branches, or an unsupported "needs backfill" migration, armed a recovery sidecar *before* calling the engine, then left it behind when the engine rejected the apply pre-movement. The server refuses to boot while any sidecar is pending, and re-running apply re-armed a fresh sidecar — an unescapable crash loop. None of the engine rejections are bugs; the trap is in the apply/serve choreography. Three coordinated changes: 1. Preview before arming the sidecar. `cluster apply` now runs `preview_schema_apply_with_options` before `write_recovery_sidecar`, so parser/planner rejections (non-main branches, unsupported plan) fail loudly without leaving recovery work behind. The post-preview engine error path now deletes the sidecar when the live schema still matches the recorded digest (nothing moved), and keeps it only on real mid-movement failure — both branches covered by new engine-failpoint tests (cluster failpoints now enable omnigraph/failpoints). 2. Per-graph quarantine at serve time instead of whole-cluster refusal. A graph-attributed pending sidecar, an unopenable graph root, a query parse failure, or an unresolvable embedding provider now quarantines just that graph (logged loudly at every boot layer) while healthy graphs serve; `/graphs` lists only ready graphs and quarantined routes 404. Cluster-global problems (missing/unreadable state, malformed or unattributable sidecars, shared-catalog or cluster-policy errors, zero healthy graphs) stay fail-fast. `--require-all-graphs` / OMNIGRAPH_REQUIRE_ALL_GRAPHS=1 restores all-or-nothing boot. 3. Backfill embedding-provider profile metadata on apply. Mirrors the existing policy-binding backfill: a pre-5A ledger missing `embedding_profile` is now detected as a metadata-only change and backfilled by a no-op apply, instead of bricking serve with `embedding_provider_profile_missing` forever. Tests: trap (no sidecar after a rejected apply), both digest-cleanup branches, per-graph quarantine (cluster + server), embedding backfill. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: resilient cluster boot + recovery-sidecar trap fix Amend RFC-005 D4 readiness posture (cluster-global fail-fast vs graph-local quarantine; deviation #5 for --require-all-graphs), add the v0.7.0 release note, and update the user cluster/server/deployment docs and the OMNIGRAPH_REQUIRE_ALL_GRAPHS env var. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cluster): surface sidecar-cleanup failures; document severity promotion Address Greptile review on PR #284: - The pre-movement sidecar cleanup fast-path discarded `delete_object`'s result, so a transient delete failure left the graph quarantined with no signal. Add `try_delete_object` (Result-returning) and emit a `recovery_sidecar_cleanup_failed` warning diagnostic on failure; the fire-and-forget `delete_object` now delegates to it. - Document why the serve-time loop promotes every `list_recovery_sidecars` diagnostic to a cluster-fatal error (the listing only emits genuine read/parse/version failures, as warnings, whose blast radius serving cannot prove) and note the promote-by-code path if that ever changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ragnor Comerford
|
7168ee0ed0
|
fix(engine): stop branch-merge fast-forward OOM on embedding tables (#277)
* fix(engine): stop branch-merge fast-forward OOM on embedding tables A branch→main fast-forward merge of a forked, embedding-bearing table re-derived the whole branch row-by-row: it lumped new + changed rows into one Lance `merge_insert`, i.e. a full-outer hash join over the entire delta that exhausts the DataFusion memory pool (8k rows × 3072-dim → `Resources exhausted: 188MB HashJoinInput, 100MB pool`), so the merge hung/failed instead of completing. Fix the data path on existing, substrate-supported primitives: - Adopt-with-delta split: new rows → `stage_append` (a streaming `Operation::Append`, no hash join), only genuinely-changed rows → a bounded `stage_merge_insert`, deletes inline. New `AdoptDelta` / `compute_adopt_delta` / `publish_adopted_delta` replace the combined `compute_source_delta` path; the three-way merge path is untouched. - Stream the append via `stage_append_stream` → `execute_uncommitted_stream` (the substrate-blessed bulk-append path), removing the `Vec`+`concat` full-delta materialization. Blob-aware via `scan_stream_for_rewrite`. Exposed on the sealed `TableStorage` trait. - Lazy row-signature: stop stringifying every row's embedding eagerly; compute the signature only for the `(Some,Some)` changed-candidate arm. - Index coverage is reconciler-owned: the adopt path no longer rebuilds vector/FTS indexes inline; `optimize`/`ensure_indices` folds the new rows in (reads stay correct via brute-force tail). Post-merge index-coverage contract documented in docs/user/branching/merge.md. - Recovery pin: new `CandidateTableState::AdoptWithDelta` is classified and pinned so the append's HEAD advance is sidecar-covered (invariant 5); the `BranchMerge` sidecar's loose classification covers the two-commit shape. The regression gate is structural, not a brittle size threshold: task-local write probes assert an append-only fast-forward merge does 0 `stage_merge_insert` (the OOM hash join), appends via `stage_append`, and streams (0 whole-delta materialization). Plus functional correctness, blob round-trip, index-defer, and a Phase-B failpoint recovery test. Residual: the classify-time staging round-trip is still O(N) in memory (architecturally required for the all-or-nothing multi-table publish); bounding it fully is the fragment-adopt follow-up. * test(engine): partial branch-merge Phase B must roll back (RED regression) A branch-merge per-table publish is a multi-commit sequence — adopt: append → upsert → delete; three-way: merge_insert → delete → index — each step advancing Lance HEAD before the single manifest publish. Add four failpoint sites at those windows and four regression tests (mixed delta: a fresh id, a modified base id, a removed base id) asserting that a crash mid-sequence rolls the whole merge BACK on the next open and a re-run re-applies the full delta. RED against current code: the loose `BranchMerge` classification rolls any `lance_head > manifest_pinned` forward, so the partial is published and the merge recorded — the rolled-back-to-base assertion fails with the partial state visible (e.g. bob appended, dave not deleted). The fix lands next. The failpoint sites are no-ops unless the `failpoints` feature activates them. * fix(engine): roll back partial branch-merge Phase B (recovery WAL confirmation) A branch-merge publishes each table with several Lance commits (adopt: append → upsert → delete; three-way: merge_insert → delete → index), then one manifest publish makes them atomic. Recovery classified `BranchMerge` loosely: any `lance_head > manifest_pinned` with a matching CAS pin rolled *forward* to the observed HEAD. So a crash mid-sequence published a partial delta (e.g. the append without its sibling upsert/delete) and recorded the merge as complete — silent data loss; a re-merge sees "already up to date" and never repairs it. Fix: make the recovery sidecar a two-phase WAL for `BranchMerge`. After the whole per-table publish loop completes, stamp each pin's `confirmed_version` with its exact achieved Lance version (a second sidecar write), then publish the manifest. Recovery now: - rolls FORWARD only to a pin's `confirmed_version` (set ⇒ Phase B finished); - rolls BACK (`TableClassification::IncompletePhaseB`) when the HEAD moved but no confirmation was recorded ⇒ a partial publish ⇒ all-or-nothing restore to the manifest pin, so a re-run re-applies the full delta. Scope: `BranchMerge` only. Other loose writers (`SchemaApply`, `EnsureIndices`, `Optimize`) keep the loose roll-forward — their drift is derived state (index coverage, compaction) a partial roll-forward never corrupts, so confirmation would be cost without benefit. This is the write-ahead intent record + idempotent roll-forward that the fast-forward-main commit model requires to be crash-atomic across N tables; version-recorded (not phase-count-derived), so it survives later changes to the per-table commit sequence. Regression tests (failpoints): four partial-window crashes — adopt after-append / after-upsert, three-way after-merge / after-delete — each with a mixed delta (new id, modified id, removed id) now roll the whole merge back; the existing complete-Phase-B tests still roll forward. * fix(engine): scope merge index docs to fast-forward; record append probe after write Two PR-review fixes: - docs(merge): the "a merge does not build indexes inline" note only holds for the fast-forward / adopt path (deferred to the reconciler). The three-way `Merged` path still rebuilds indexes inline in its publish, so a Merged-outcome merge of an embedding table pays the build up front. Scope the doc so a Merged-outcome user isn't surprised or led to skip a post-merge optimize. - `stage_append` recorded its instrumentation probe before the fallible `execute_uncommitted`, so a failed staging write left the call/row counters inflated — and diverged from `stage_append_stream`, which records after the transaction is built. Record after the write succeeds. * fix(engine): record stage_merge_insert / vector-index probes after write too The prior commit moved `stage_append`'s instrumentation probe to after the write, but left the two sibling write primitives with the identical ordering bug: `stage_merge_insert` recorded before `execute_uncommitted`, and `create_vector_index` before the index build. A failed write on either would inflate the probe counter. Move both to record only after the write succeeds, so all write-primitive probes share one rule (record-after-success) — closing the class rather than the single instance the review flagged. * docs(engine): mark the fragment-adopt excision boundary in the merge code Comment the transitional row-level merge code so a future fragment-adopt implementation (Lance branch-merge/rebase #7263 + UUID branch paths #7185) knows exactly what it deletes and what it keeps: - `AdoptDelta` / `compute_adopt_delta` / `publish_adopted_delta` — the row-level re-derivation; removed wholesale when a fast-forward merge becomes a fragment graft (adopt the source table version's fragments + indexes by reference). - `stage_append_stream` — its only caller is that merge append; dead with it unless re-adopted as a general bulk-append path. - `confirm_sidecar_phase_b` — the boundary marker: this SURVIVES. The recovery sidecar is the cross-table WAL a fast-forward-main commit still needs; only the within-table multi-commit reason for `IncompletePhaseB` narrows once each table is a single graft commit. Keep the sidecar; only simplify the classifier. Comments only; no behavior change. * test(engine): pre-upgrade v1 branch-merge sidecar must roll forward (RED) Phase-B confirmation made the recovery classifier strict for every BranchMerge sidecar — including ones written by a binary that predates confirmation. A pre-upgrade crash in the Phase-B→C gap can leave such a sidecar over a COMPLETED merge; the new classifier reads its absent confirmed_version as a partial and rolls it back, silently discarding the finished merge (greptile P1 / Cursor High). This regression test synthesizes that sidecar realistically: crash after Phase B (real sidecar + advanced Lance HEAD), downgrade the on-disk JSON to the pre-confirmation v1 shape (schema_version=1, strip confirmed_version), reopen. RED: the merge rolls back, `bob` is discarded (left ["alice"], want ["alice","bob"]). The versioning fix lands next. * fix(engine): version the recovery sidecar; read pre-confirmation merges as loose Phase-B confirmation changed how a BranchMerge sidecar's absent confirmed_version is interpreted (roll forward → roll back) without versioning the artifact, so the new classifier silently discarded completed pre-upgrade merges (greptile P1 / Cursor High). A capability flag would not fix the symmetric direction — keeping schema_version=1, an OLD binary reading a NEW sidecar sails through its already-shipped strict gate, ignores the unknown flag, and applies loose semantics to a new partial → the same data loss on downgrade. Use the versioning system instead. - Bump SIDECAR_SCHEMA_VERSION 1 → 2; add a fixed CONFIRMATION_SCHEMA_VERSION = 2 (the generation at which confirmation shipped — pinned, so a later v3 keeps v2 confirmation-aware). - Make the read gate version-aware (`parse_sidecar`): refuse only versions NEWER than this binary; accept and interpret older ones with their original semantics — no operator toil draining pre-upgrade sidecars. Rename `SidecarSchemaError.supported_version` → `max_supported_version` and reword. - Dispatch classification by version: the strict BranchMerge confirmation path is gated on `schema_version >= CONFIRMATION_SCHEMA_VERSION`; a v1 BranchMerge sidecar falls through to the existing loose roll-forward. Thread `sidecar.schema_version` from `process_sidecar`. This is bidirectionally safe: a new binary interprets v1 (loose) and v2 (strict) and refuses the future; an old binary's `!= 1` gate already refuses v2, so it never misreads a new sidecar. The flag was an additive-field pattern misapplied to a semantics change; versioning is the correct mechanism. Honest residual (any approach): an old *partial* sidecar still rolls forward — v1 carries no confirmation, so partialness is undetectable in it. The fix stops us from interpreting old sidecars under new rules; it can't retrofit information they never had. * fix(engine): harden recovery — mode resolver, loud divergence check, publish classified version Three correct-by-design fixes from the holistic review of the recovery path, all in recovery.rs (each closes a class, not an instance): A. Resolve the classification mode from `(kind, schema_version)` once, instead of a kind×version match accreting fall-through guards in `classify_table`. New `ClassificationMode { Strict, Loose, Confirmed }` + an exhaustive `SidecarKind::classification_mode` — adding a writer kind or version floor is now one arm in the resolver (the compiler forces it), not a guard threaded through the classifier. No behavior change; existing classify/decide tests are the guard. B. `confirm_sidecar_phase_b` now errors loudly when a pinned table has no achieved version in the publish `updates`, instead of silently skipping it (which left the pin unconfirmed → `IncompletePhaseB` → a silent rollback of a COMPLETE merge). Guards the implicit `pins ⊆ updates` invariant against a future divergence between the two filters (invariants 9/13). + a unit test. C. Recovery roll-forward publishes the version classification OBSERVED (`state.lance_head`), not a fresh HEAD re-read at publish time. For a Confirmed pin classify already validated `lance_head == confirmed_version`, so this publishes the recorded WAL intent by construction and closes the classify→publish re-derivation/TOCTOU for every writer (invariant 15). `push_table_update_at_head` → `push_table_update(target_version: Option<u64>)`: roll-forward pins the classified version; roll-back keeps `None` (publishes the restore commit it just made). In-scope behavior is preserved, so the existing roll-forward integration tests are the guard; the drift-hardening is correct-by-construction (deterministic mid-sweep drift injection isn't feasible — a sync failpoint can't do an async Lance write). |
||
|
aaltshuler
|
f2c512ae26 |
chore: remove CODEOWNERS chassis and the code-owner review gate
Some checks failed
CI / Classify Changes (push) Has been cancelled
CI / Check AGENTS.md Links (push) Has been cancelled
CI / Container Entrypoint (push) Has been cancelled
Release Edge / Prepare edge release (push) Has been cancelled
CI / Test Workspace (push) Has been cancelled
CI / Test omnigraph-server --features aws (push) Has been cancelled
CI / RustFS S3 Integration (push) Has been cancelled
Release Edge / Build edge omnigraph-linux-x86_64 (push) Has been cancelled
Release Edge / Build edge omnigraph-macos-arm64 (push) Has been cancelled
Release Edge / Build edge omnigraph-windows-x86_64 (push) Has been cancelled
Release Edge / Smoke Windows installer (push) Has been cancelled
The repo is a 2-person team where both maintainers own every path, so the CODEOWNERS machinery (generated CODEOWNERS, roles yml, render script, the two drift/hand-edit CI jobs) gated nothing real while adding friction: every PR showed "Review required" and own-PRs merged only via admin/bypass override. Remove the whole chassis and drop the review gate: - delete .github/CODEOWNERS, codeowners-roles.yml, render-codeowners.py, the CODEOWNERS workflow, and docs/dev/codeowners.md - branch-protection.json: drop the two CODEOWNERS required status checks, set require_code_owner_reviews=false and required_approving_review_count=0 (CI checks are the gate; maintainers merge their own PRs once green) - scrub CODEOWNERS references from AGENTS.md, docs indexes, branch-protection and ci docs, GOVERNANCE.md, and CONTRIBUTING.md The policy change is inert until an admin runs scripts/apply-branch-protection.sh. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
a9a5453520
|
Merge pull request #278 from ModernRelay/codex/compiler-error-rename
Rename compiler error and fix cluster config warnings |
||
|
aaltshuler
|
f118b67402 | address compiler error review comments | ||
|
aaltshuler
|
4590c91f9d | rename compiler NanoError and fix cluster config warnings | ||
|
Ragnor Comerford
|
916dc46c0e
|
fix(server): align stored-query MCP discovery gates | ||
|
Ragnor Comerford
|
c06343362a
|
docs(mcp): document the MCP surface, authoring controls, and skill (v0.8.0)
Document the per-graph MCP surface (POST /graphs/{id}/mcp, shipped in the
preceding commits and landing under v0.8.0) and the `.gq` authoring controls
that shape stored-query tools.
- New docs/user/operations/mcp.md: the client-facing guide — transport, tool
catalog (built-ins + stored queries), projection modes, structured output,
authorization (call-authoritative + list-relaxation), Host/Origin policy, the
protocol-version contract.
- docs/user/operations/server.md: the /mcp endpoint + an "MCP surface" section;
docs/user/index.md: a "Connect an MCP agent" pointer.
- docs/user/queries/index.md: an Annotations section — query @description /
@instruction / @mcp(expose, tool_name) and per-parameter @description.
- AGENTS.md: topic-table row + MCP note on the HTTP-server capability row.
- docs/dev/testing.md: the omnigraph-mcp crate + server tests/mcp.rs.
- docs/dev/rfc-005 §D5: retire the "cluster = everything exposed" bridge —
cluster mode honors source `@mcp(expose: …)`; presentation vs authorization
split made explicit.
- skills/omnigraph: server-policy.md MCP section; stored-queries.md corrected
(per-query controls now ship via @mcp, not "planned"); SKILL.md MCP triggers,
Deep Dives row, version → 0.8.0.
- docs/releases/v0.8.0.md: the MCP surface + authoring-controls release notes.
Crate version manifests are deliberately NOT bumped — that is the v0.8.0
release-cut step; this lands on the feature branch.
|
||
|
Ragnor Comerford
|
c8e91c11f0
|
feat(mcp): per-query @mcp(...) annotation + per-param @description + @instruction folding
Wire the `.gq` authoring surface that controls how a stored query is projected
as an MCP tool. All of it rides in the query source (content-addressed,
re-parsed at boot), so there is no cluster.yaml / catalog / serving-snapshot
plumbing — and it is orthogonal to Cedar `invoke_query` (presentation, not
authorization).
- Per-parameter `@description("…")` (leading the variable) → carried on
`Param.description`, mapped through `param_descriptor`, and emitted on the
outer JSON-Schema property by `param_json_schema`, so it shows up in both the
MCP tool input schema and the `GET /queries` catalog.
- Query `@mcp(expose: <bool>, tool_name: "<name>")` → parsed into
`QueryDecl.mcp`; `StoredQuery::is_exposed()` / `effective_tool_name()` resolve
from it. `expose: false` hides a query from the agent surface (`tools/list`,
`stored_query_list`, run-by-name) while keeping it HTTP/service-callable.
- `@instruction` is folded into the MCP tool description (after `@description`),
so the agent-facing how/when-to-use guidance reaches `tools/list`.
- Removes the now-dead `RegistrySpec.{expose, tool_name}` fields (server + CLI);
`settings.rs` no longer hardcodes `expose: true`. Test helpers express
exposure by injecting `@mcp(expose: false)` into the source (the real path).
openapi.json regenerated: `ParamDescriptor` gains an optional `description`.
Tests: compiler parser (param @description, @mcp parse + duplicate rejection),
api-types schema_equivalence (description on the outer property), server mcp
(folded description + param docs + @mcp tool rename, list==call). Full
workspace gate green.
|
||
|
Ragnor Comerford
|
bcd0d9c867
|
feat(mcp): MCP server surface — Streamable-HTTP transport + tool/resource projection (RFC-003)
Add the `omnigraph-mcp` crate (stateless Streamable-HTTP transport, `McpBackend`
seam, fail-closed Host/Origin policy) and the server backend projecting built-in
operations and the per-graph stored-query registry as MCP tools + resources over
`POST /graphs/{id}/mcp`. Every tool delegates to the same engine/handler
functions the REST routes use and is gated by the same Cedar `authorize` path;
reads/writes carry structured output.
Includes three correctness fixes from review + live testing:
- tools/list is a faithful relaxation of the per-call gate: a built-in whose
authorization depends on a caller-chosen branch is shown iff the actor could
invoke it on some branch, via PolicyEngine::permits_on_any_branch (capability
probe through the same Cedar authorizer). A fabricated-`main` probe wrongly
hid graph_mutate under the canonical "protect main, write unprotected" policy.
- The stored-query surface honors mode + `expose` on call as well as on list:
resolve_stored_tool is the single membership test, so the meta pair
(stored_query_list/stored_query_run) is callable only in `meta` mode and
stored_query_run resolves exposed-only. An `expose:false` query is unreachable
by name on the agent surface (it stays HTTP/service-callable).
- The loopback Host allow-list is the full set [127.0.0.1, ::1, localhost]
(matches rmcp's default), so an IPv6 loopback `Host: [::1]` is accepted
regardless of which stack the server bound.
The protocol-version contract is documented (initialize negotiates the version
in its body, so the MCP-Protocol-Version header is validated on non-init
requests only) and pinned by a test.
Tests: omnigraph-mcp/tests/standalone.rs, omnigraph-server/tests/mcp.rs,
omnigraph-policy permits_on_any_branch unit test, omnigraph-api-types schema
projection. Full workspace gate green.
|
||
|
Ragnor Comerford
|
5243c048aa
|
perf(engine): remove the per-query metadata re-derivation tax on warm reads (#268)
* test(engine): add read-path IO instrumentation seam for warm-read cost tests
Prerequisite seam for the query-latency fixes. Adds
crates/omnigraph/src/instrumentation.rs:
- CountingStorageAdapter: a StorageAdapter decorator counting per-method
reads (read_text/exists/read_text_versioned/list_dir), for the
schema-contract reads on the query path.
- A per-query task-local (QueryIoProbes) carrying Lance WrappingObjectStore
wrappers per open category plus a probe counter, delivered via
with_query_io_probes. open_dataset_tracked attaches the wrapper so the
open itself is counted (ObjectStoreParams.object_store_wrapper).
Wires the wrappers into the manifest open (open_manifest_dataset) and the
commit-graph opens (CommitGraph::open/open_at_branch). Production leaves
the task-local unset, so nothing attaches.
Makes Omnigraph::open_with_storage public so tests can inject the counting
adapter. lance-io is a dev-dependency (IOTracker named only in tests). No
runtime behavior change.
* test(engine): warm same-branch read should reuse the coordinator (red)
Cost-budget test using Lance IOTracker at the object-store boundary (the
LanceDB IO-counted-test pattern). On a 20-commit-deep graph, a warm
same-branch query re-opens a fresh coordinator, which opens both the commit
graph and __manifest. Asserts the read opens the commit graph zero times
and performs exactly one cheap version probe; today it does neither (it
scans the commit graph on re-open and never probes). The freshness guard
already passes. Adds the commit_many helper for history-depth fixtures.
Red half of the Fix 1 red->green pair; turns green with the next commit.
* perf(engine): same-branch reads reuse the warm coordinator (Fix 1)
query()/resolved_target re-opened a fresh GraphCoordinator from storage on
every read (full __manifest scan + two commit-graph scans), so a warm
read's cost grew with commit history (invariant 15) though the data was
unchanged.
resolved_target now serves same-branch reads from the warm in-memory
coordinator, gated by a cheap version probe (latest_version_id, one
object-store op) instead of a full re-open:
- fresh (probe == cached version): return the in-memory snapshot under the
read lock, with a synthetic (branch, version) id and no commit-graph
access (reads pin the snapshot by manifest version, not the commit DAG;
invariant 2).
- stale: take the write lock, re-probe (double-checked; tokio RwLock has no
read->write upgrade), then refresh_manifest_only (no commit-graph scan),
preserving strong consistency for external writers (invariant 6).
Cross-branch and snapshot targets keep the existing cold-resolve path.
Adds ManifestCoordinator/GraphCoordinator::probe_latest_version and
GraphCoordinator::refresh_manifest_only. Nothing on the read path needs a
real commit ULID (only RuntimeCache keys on the id, where synthetic is
consistent), per a caller audit.
A warm same-branch read on a 20-commit graph now does zero commit-graph
opens and exactly one probe (down from a deep commit-graph scan) and still
observes external commits. The residual per-table __manifest scans are
removed later by Fix 2.
* test(engine): warm query should validate the schema contract once (red)
ensure_schema_state_valid runs twice per query (query()/run_query_at AND
resolved_target/snapshot_at_version), each reading 3 contract files + 2
existence probes. A warm query thus does 6 read_text + 4 exists where one
validation (3 + 2) suffices, measured via CountingStorageAdapter. Adds a
drift guard (schema_source_drift_is_caught_on_read) that already passes.
Red half of the finding-A red->green pair.
* perf(engine): validate the schema contract once per query (finding A)
ensure_schema_state_valid ran on every query AND again inside
resolved_target / snapshot_at_version, so each query validated the schema
contract twice (~10 storage ops). Removes the redundant query()/
run_query_at() calls; the validation inside resolved_target /
snapshot_at_version still runs, so drift is detected exactly as before.
A source-only fast path was rejected: a long-lived handle must detect
external drift of the schema source, IR, OR state on its next operation
(lifecycle::long_lived_handle_rejects_schema_*), which a source-only
compare would miss. So the only safe latency win is not validating twice.
A warm query now does one validation (3 read_text + 2 exists) instead of
two (6 + 4).
* test(engine): warm + multi-table reads should do zero manifest scans (red)
After Fix 1 a warm same-branch read still scans __manifest ~44 times at
20-commit depth: not from resolution (Fix 1 removed that) but from the
per-table open path, which routes through the Lance namespace and full-scans
__manifest twice per touched table (describe_table + describe_table_version).
Tightens the warm test to assert manifest read_iops == 0 and adds a
multi-table (traversal) test asserting the same, pinning the "2 tables = 2x"
tax. Red half of the Fix 2 red->green pair.
* perf(engine): open touched tables by location+version, not via the namespace (Fix 2)
SubTableEntry::open routed every read-path table open through
DatasetBuilder::from_namespace(BranchManifestNamespace), whose describe_table
full-scans __manifest and, with managed_versioning, makes Lance scan again
(describe_table_version) -- two full __manifest scans per touched table. That
was the residual that made warm-read manifest IO grow with history and the
'2 tables = 2x' multi-table tax.
The resolved Snapshot already holds each table's path/version/branch, so open
directly: from_uri(table_uri_for_path(root, path, branch)).with_version(v).
The branch-qualified location is the dataset that physically holds the version
(main: {path}; branch: {path}/tree/{branch}, Lance native-branch storage), and
with_version resolves it within THAT dataset's _versions. 0 namespace calls +
1 HEAD via the native ConditionalPutCommitHandler.
The read namespace (BranchManifestNamespace) is now unused in production
(writes use StagedTableNamespace), so it, its constructor, and the helpers only
it used (to_namespace_version, publish_requests, their imports) are gated
#[cfg(test)] -- retained to validate the namespace contract in unit tests.
Removes the dead open_table_at_version_from_manifest.
Warm same-branch + multi-table reads now scan __manifest zero times; branch +
time-travel reads stay correct (branching.rs, point_in_time.rs, 2 lib
regression tests); production-lib warnings unchanged (baseline).
* test(engine): cost-budget coverage for branch-warm and stale-refresh reads (matrix)
Extends the read-path cost-budget tests across more of the morphological matrix:
- warm_branch_read_does_no_manifest_scans: a warm read on a non-main branch
(handle synced to it) scans __manifest zero times, exercising Fix 2's
branch-owned-table open (tree/{branch} + with_version) on Fix 1's warm path --
the cell that regressed when the open used with_branch against the base.
- stale_read_refreshes_manifest_only: an external commit makes the next read
take the stale path, which re-reads the manifest (read_iops > 0) but never
scans the commit graph (refresh_manifest_only), pinning Fix 1's manifest-only
refresh.
Cold paths (cross-branch, time-travel) stay behavior-covered (branching.rs,
point_in_time.rs) and are cold by design (Fix 1 warm-paths only same-branch), so
there is no manifest==0 contract to assert there.
* test(engine): same-branch write after external commit must not fork the commit DAG (red)
* fix(engine): refresh commit-graph head before append to prevent same-branch DAG fork
A same-branch write that follows an external commit committed a fresh manifest
version (commit_all rebases the pin from a fresh coordinator) but appended off
the coordinator's stale in-memory commit-graph head, forking the commit DAG (the
new commit and the external commit shared a parent). Pre-existing for non-strict
inserts; widened to strict ops by Fix 1's refresh_manifest_only freshening the
read-time pin. record_graph_commit now refreshes the commit-graph head from
storage before append_commit, so the parent is the true current head.
record_merge_commit is unaffected (it passes explicit parents).
* perf(engine): hold open Dataset handles + share one Session per graph (Fix 3)
A warm same-branch read still re-opened every touched table per query (the
"never warms up" residual after Fix 1+2). A per-graph held-handle cache keyed by
(table_path, branch, version) now serves repeat reads with zero table opens, and
one shared lance::Session per graph warms metadata/index caches across opens.
Validated against LanceDB upstream (rust/lancedb/src/table/dataset.rs
DatasetConsistencyWrapper): hold an Arc<Dataset> and reuse it for 0-IO warm
reads; one Session per connection threaded into opens; writers never serve from
the read cache; time-travel bypasses. One adaptation: omnigraph keys by version
(snapshot-pins-version model) where LanceDB keys per-table+HEAD, reusing the
in-repo GraphIndexCache LRU template.
- ReadCaches (session + TableHandleCache) injected onto live-Branch-read
snapshots in resolved_target; Snapshot::open serves from the cache or opens
once with the session on a miss (via the instrumented open_table_dataset).
- Writes (resolved_branch_target -> open HEAD) and time-travel / Snapshot-id
reads bypass the cache. Version-in-key makes a write a new key (old handle ages
out via LRU); invalidate_all at branch-switch/refresh is hygiene only.
- Cost tests: a 2nd identical warm read does 0 table opens; a write re-opens only
the changed table at its new version.
Full engine suite green.
* test(engine): forbid raw data opens in the read/exec layer (P2 guard)
Extend the forbidden-API guard with Dataset::open / DatasetBuilder::from_uri /
from_namespace so the read/exec layer (exec/, loader/, changes/, db/omnigraph/)
cannot bypass Snapshot::open and the held-handle cache (Fix 3). The instrumented
opener (instrumentation.rs) is allow-listed; two legitimate non-read opens (a
test editing __manifest, Hard-drop version GC) carry sentinels. The
storage/manifest layers stay allow-listed.
Lean P2 scope, per LanceDB-upstream + minimize-liability: the data-read boundary
already exists (SubTableEntry::open); this guard pins it so a future read cannot
open around the cache. Centralizing all internal opens behind one opener is
deferred.
* docs(dev): invariant 15 (one source of truth, cheaply derived) + cost-budget testing
Records the principle behind the query-latency work: Lance and the manifest are
the source of truth, everything else a derived view held warm and refreshed by a
cheap probe; the two failure modes (a drifting parallel copy, and cold
re-derivation whose cost grows with history) are deny-listed. Adds the
cost-budget testing discipline (assert a warm read's open/IO count is flat at
commit-history depth, the LanceDB IO-counted pattern) and the warm_read_cost.rs
row. Updates the read-path-re-derivation known gap to reflect what Fix 1/2/3 +
finding A close, and adds the commit-graph-parent-under-concurrency gap.
* fix(engine): branch-incarnation identity + unified invalidation + shared LruMap (PR #268 review)
Phase 6 A-D, correct-by-design responses to the Codex/Greptile P2 review comments. A: warm-read freshness and the table-handle cache key use the manifest incarnation (e_tag, manifest-timestamp fallback, then version), so a deleted+recreated non-main branch reusing a version number cannot be served stale; main stays version-cheap, non-main loads latest_manifest; a detected stale refresh also invalidates read caches; two regression tests force the version collision. B: unify the two cache invalidations into Omnigraph::invalidate_read_caches() at the four sites. C: assert the stale path's probe count. D: shared LruMap behind both caches with unconditional eviction, plus a unit test. Full engine suite green; multi-process lineage fork and O(history) write refresh remain known gaps for Phase 6E/7.
|
||
|
Andrew Altshuler
|
d0e06a6ff6
|
docs: audit pass — drop pre-0.7.0 release notes; scrub RFC refs from user docs (#272)
* docs: audit pass — drop pre-0.7.0 release notes; scrub RFC refs from user docs
- Delete the pre-0.7.0 release-notes archive (v0.2.0 … v0.6.2); keep v0.7.0.
- Rewrite every inline "RFC-0NN" citation in docs/user/** into durable
plain language (the behavior is the contract, not the planning doc):
cli/index.md, cli/reference.md, clusters/index.md, operations/{maintenance,
policy,server}.md. Updated the in-page "Scopes & profiles" anchor to match
the de-RFC'd heading.
No sub-0.7.0 version caveats or stale Lance-version refs were present in
docs/user/**. Dev docs, AGENTS.md, and instruction files are out of scope for
this pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: second alignment pass — drop residual pre-cluster-only framing
- cli/reference.md: rewrite the server-scope graph-resolution rule — an
omnigraph-server is always cluster-backed, so GET /graphs always answers and
--graph is required; the bare-URL path is only the fallback for an
unavailable/non-omnigraph endpoint (was "a single-graph / flat server …
uses its bare URL as before").
- embeddings.md: "Direct single-graph serving" → "Direct (--store) access"
(there is no single-graph serving mode under cluster-only).
- clusters/{config,index}.md: drop the removed --target flag from the
"--cluster cannot combine with …" clauses.
Verified: no Linear tickets, no RFC refs, no single-graph-as-current, no
--target-as-combinable in docs/user/**.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Andrew Altshuler
|
b6131393b7
|
docs(readme): drop em-dashes, Cursor→Codex, rename agent section (#274)
* docs(readme): drop em-dashes, Cursor→Codex, rename agent section - Replace all 20 em-dashes with context-appropriate punctuation (colons, semicolons, parens, commas) — removes the AI-slop tell. - Cursor → Codex (the agent-host examples and the MCP host list). - "Drive it with an AI agent" → "Set it up with an AI agent". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(readme): wordmark header + simplify query examples - Add the compact wordmark header (light/dark SVG, subtitle, nav row, restyled badges) from the header-redesign work; bring the wordmark assets with it. - Rewrite the Query and mutate examples to lead with the short, config-default form (no repeated --server/--graph) and aliases — showing how simple it is, not crazy-long lines. The verbose --server/--graph/--store form is demoted to a one-line "ad-hoc target" note. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
b55ca02131
|
docs(readme): one sentence per line in the intro (#271)
Adjacent source lines collapse into run-on text when rendered. Add hard line breaks so the headline, subhead, and each intro sentence land on their own line. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
ccd13eca7c
|
docs(readme): make Key capabilities a table (#270)
Match the "What you can build" section's scannable two-column format. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
8cb127d8fa
|
docs(readme): server-first rewrite — deploy, agents, on-prem RustFS (#269)
* docs(readme): server-first rewrite — deploy, agents, on-prem RustFS Reframe the README around the actual product: a self-hostable, multigraph graph server for context assembly and multi-agent coordination, deployed Terraform-style on your own object storage (on-prem via RustFS, or S3/R2/GCS). - Lead with key capabilities and what you can build, not a local toy. - Promote "Drive it with an AI agent" (skill + a docs-first setup prompt) above the manual deploy walkthrough — agents are the primary operator. - "Deploy" is the hero: cluster.yaml → object store → validate/plan/apply → omnigraph-server, with RustFS as the on-prem path front and center. - "Query and mutate": stored queries by name + branch/review/merge. - Security & governance as scannable bullets; Clients & SDKs as a table. - Embedded local graph demoted to a clearly-labeled "quick test" (Signal → Indicates → Pattern), explicitly dev/experiment-only. - Drop the "serve/served/serving" vocabulary tic in favor of deploy/run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(readme): add the server boot command to Deploy §3 (Greptile P1) The "Converge and run" step showed only the converge half — the code block ended at `cluster apply` with no `omnigraph-server` command, leaving a linear reader without a way to actually start the server. Add the boot line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(readme): simplify the server boot command Drop the inline OMNIGRAPH_SERVER_BEARER_TOKEN prefix from the Deploy hero — the example cluster declares a policy so the server boots without it, and bearer auth is covered in Security & governance. Leaves a single clean line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(readme): boot the server from the cluster dir, not a raw s3:// URI Pointing --cluster at the bucket hardcodes the storage URI in the run command. Boot from the config directory instead; the storage URI lives once in cluster.yaml and the server resolves it — single source of truth, and consistent with the cluster apply commands above. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Ragnor Comerford
|
c43b81d318
|
docs(rfc-013): add reader/writer scaling (§5.8) — split read fleet vs write tier
The substrate makes read/write scaling asymmetric: reads are object-store-backed, snapshot-isolated, stateless -> horizontal to N replicas with zero coordination; writes are serialized per (table,branch) by one-winner manifest CAS -> scale by partitioning (branches/graphs/cells), with a single active coordinator per (cell,graph,branch). Adds the CQRS deployment split (read fleet / write tier / maintenance / heavy reads), read-your-write via commit_id/snapshot_id pinning, and gates pooled WRITES (not reads) on closing the cross-process-CAS gap. |
||
|
Ragnor Comerford
|
0f58329ab7
|
docs(rfc-013): tenancy model — cluster-as-tenant cells, pooled compute
General server/topology/auth/deployment RFC resolving the half-built tenancy ambiguity (cluster-only server vs pooled tenant_id scaffolding). Decision: the cluster is the tenant is the cell — silo the data (own storage/catalog/ policy/tokens), pool the compute (one process : N cells). No row-level pooling (no engine RLS). - §5.1 CellRuntime lifts today's per-cluster runtime into a value. - §5.2/§5.3 AppState holds a CellRegistry; resolve_cell is one new outer middleware hop before auth; the per-graph + Cedar + MCP stack is unchanged. - §5.4 per-cell CellAuth (Static | Oidc TokenVerifier); WorkOS org -> cell 1:1 with per-cell OAuth audience (cross-tenant token replay fails on aud). - §5.5 Cedar stays per-graph/per-cell; default-deny-read becomes safe; no tenant dimension needed. - §5.6 control plane = Cell Registry (metadata only) + provisioning-as-code; cell hot-load is the one safe runtime mutation (cell-granular, not graph). - §5.7 tiered dedicated/pooled/on-prem on one binary; §7 backward-compatible (today's single-cluster server = a one-cell map). MCP (rfc-003) is one consumer, not the driver. Linked from docs/dev/index.md. |
||
|
Ragnor Comerford
|
86fbb62d12
|
docs(rfc-003): fold external review into correct-by-design fixes
An external review pass raised 8 findings; verified 7 valid (2 confirmed
against the engine coercer). Folded them in as class-closing fixes rather than
point patches:
- §9.1 (③④, the headline): the JSON-Schema generator was a second hand-written
copy of the engine's input contract — Blob (base64 vs URI string) and nullable
(explicit null) were two drifts of one class. Move the projection to a single
param_json_schema in omnigraph-api-types (next to ParamKind/ParamDescriptor),
fix Blob -> {"type":"string","format":"uri"} (query_input.rs:449 / api-types:354
say blob-URI string) and nullable -> anyOf[..,null] (query_input.rs:273,296),
and lock it to json_value_to_literal_typed with a schema/engine equivalence
test so any future drift is a CI failure.
- §7/§4 (①): replace the fail-open "empty allowed_origins => skip" with a total
OriginPolicy and a single McpHostPolicy::from_bind constructor (remote default
DenyBrowsers, enforced by origin_guard independent of rmcp's empty-list quirk).
No absent-=>-skip state can be constructed.
- §6/§12/§16 (②): make the non-paginated list seam a stated contract (Vec<T>,
no nextCursor; meta mode bounds large catalogs) and drop the pagination claims
the signature couldn't express.
- §9.3 (⑦): built-in/stored tool-name collision becomes a cluster validate/boot
error (fold built-in names into the registry uniqueness check), not a silent
skip — per the invariants deny-list.
- §9.2 (⑥): stored_query_mode folded into the one per-graph mcp: block (Phase 6),
not a floating key; not configurable until that surface exists.
- §10/§1 (⑧): scope derives from the per-graph mount; server-scoped `health`
becomes graph-scoped `graph_health` (server liveness stays REST /healthz).
- §13 (⑤, doc-only): OpenAI row corrected to the `authorization` field; Phase-1
reachability via static bearer is unchanged.
§17 records the locked decisions; the validation header notes the review pass.
|
||
|
Ragnor Comerford
|
3771a29dd4
|
docs(rfc-003): align with main after merge (cluster-only, api-types, /load)
Re-validated RFC-003 against the merged tree (RFC-009/010/011/012). Folds in
the deltas that landed on main since the RFC was written:
- §15 routing: RFC-011 made the server cluster-only. GraphRouting::Single/Multi
and the `match state.routing()` branch are gone; build_app always nests
per_graph_protected under /graphs/{graph_id}. So /mcp is always
/graphs/{id}/mcp (even a single-graph boot is a one-graph registry keyed by
`default`). §15.1's per-graph model is now the only model. Line refs repointed
to lib.rs:876/929/953/954/148.
- §9 stored queries: param/catalog DTOs moved to omnigraph-api-types (RFC-009),
re-exported via api.rs — citations repointed (lib.rs:355/373/496). The legacy
omnigraph.yaml `queries:` declaration source is removed; cluster.yaml
graphs.<id>.queries is the sole source. Flag the §D5 bridge: cluster boot
forces expose=true/tool_name=None today, so per-query expose/tool_name is a
planned phase (added to §17 deferred).
- §11 load: /ingest -> canonical POST /load (RFC-009 Phase 5); graph_load reuses
run_ingest via server_load (handlers.rs:1320), /ingest is a deprecated alias.
- Mechanical: policy/handler/identity/ApiError line-number drift repointed
(policy:16/251, handlers:313/334/711/645/913, identity:186, lib.rs:280).
No architectural change — RFC-011 confirms the per-graph model; the rest is
re-citation and the expose/tool_name caveat.
|
||
|
Ragnor Comerford
|
c08e8dbac4
|
Merge remote-tracking branch 'origin/main' into ragnorc/omnigraph-mcp-crate | ||
|
Andrew Altshuler
|
e510937a7e
|
docs(readme): remove CI status badge (#267)
The CI badge read "failing" largely from concurrency-cancelled runs (rapid doc pushes cancel in-flight runs → cancelled conclusion) and a now-fixed S3-job break, not from a broken default branch. It's been more noise than signal; drop it. CI status is still visible on the Actions tab. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Ragnor Comerford
|
a68896c4a0
|
docs(readme): embedded quick-start run-through + trimmed Clients (#266)
* docs(readme): add embedded quick-start run-through + trim Clients
- Quick start: a copy-pasteable embedded (local file) run-through
(schema → init → load → query → branch), followed by a "Storage backends"
table that surfaces the one thing that changes across embedded / S3 object
storage / RustFS-MinIO — the graph address — with a pointer to cluster mode
for served multi-graph deployments.
- Clients: collapse to a two-line pointer (npm packages + omnigraph-ts repo);
Python SDK marked coming soon.
* docs(readme,quickstart): fix init addressing + drop non-parsing schema commas
Addresses PR review (both verified against source):
- README "Storage backends": `init` takes the graph address as a positional
argument, not `--store` (`Command::Init { uri: String }`) — Codex. Table now
shows bare addresses and a note on which flag each verb uses.
- docs/user/quickstart.md: drop trailing commas in the schema. The .pg grammar
(`prop_decl = { ident ~ ":" ~ type_ref ~ annotation* }`, node body
`(prop_decl | body_constraint)*`, comma not in WHITESPACE) has no comma rule,
so `name: String,` fails to parse — Greptile.
|
||
|
Ragnor Comerford
|
ee4986e9a1
|
docs: onboarding-first README + in-repo agent skill + drop RustFS script (#257)
* docs: optimize README for dev onboarding; fix 0.7.0 staleness The README's setup half drifted from the shipped 0.7.0 CLI and led with the heaviest path (Docker + RustFS). This reworks it for fast, correct onboarding: README.md - New zero-dependency "Your first graph in 60 seconds" hero: a fully copy-pasteable local file-backed loop (schema → init → load → query → branch). - Add a correct "Serve it" section (cluster apply + omnigraph-server --cluster); the server is cluster-only on main, so the old positional-URI boot is gone. - Demote the RustFS bootstrap to "rehearse the S3 path locally"; reframe the storage bullet as "filesystem or any S3-compatible store (AWS S3, R2, MinIO, RustFS)" — RustFS is a provider, not a storage class. - Fix crate/MCP descriptions (query/mutate/load, not read/change/ingest). docs/user/quickstart.md - Fix the query example: `read --name <q> … <uri>` is removed — the query name is positional and the graph is addressed with `--store` (`omnigraph query find_people --query queries.gq --store graph.omni`). scripts/local-rustfs-bootstrap.sh - Convert to cluster mode: write a cluster.yaml (storage: s3://…), then validate → import → apply, load the fixture into the derived root with the now-required --mode, and serve with `omnigraph-server --cluster`. The old flow (`load` without --mode, `omnigraph-server <URI>` positional boot) no longer works on a cluster-only server. * docs: move agent skill into the repo, add agent-setup snippet, drop rustfs script skills/omnigraph - The operational skill (formerly `omnigraph-best-practices` in the cookbooks repo) now lives with the engine it documents, co-versioned. Renamed to `omnigraph`; repository metadata repointed here. - Broadened the description to trigger on intent — storing/retrieving/querying knowledge, agent memory, building a knowledge graph, operating Omnigraph — as well as on CLI/artifact sightings (stays ≤1024 chars). - Install: `npx skills add ModernRelay/omnigraph@omnigraph`. README - New "Set it up with an AI agent" paste snippet: installs the skill, reads the docs (URL), browses the cookbooks, and asks the user about a use case before standing up a first graph. - "Agent skill & starter graphs" section points at skills/omnigraph + cookbooks. Drop scripts/local-rustfs-bootstrap.sh - Not CI-tested (so it rotted: it broke on the cluster-only migration — positional server boot, load without --mode), demoed the now-optional S3 path, and was the most fragile artifact in the repo. Replaced with a "Testing against S3 locally" guide in deployment.md (docker run RustFS/MinIO + AWS_* env + cluster-on-S3). README/AGENTS references updated. |
||
|
Andrew Altshuler
|
05cb73eda6
|
test(cli): pass --yes in the S3 e2e overwrite load (RFC-011 Decision 9) (#265)
The RustFS S3 integration job was red on `local_cli_s3_end_to_end_init_load_read_flow`:
the test runs `load --mode overwrite` against an `s3://` target, which the
RFC-011 Decision 9 destructive-write guard now refuses without `--yes`
("refusing destructive `load --mode overwrite` against non-local target …").
The guard is intended and already covered in cli_data.rs; this test slipped
through because it only runs in the bucket-gated RustFS CI job, not the default
local gate. Add `--yes` to match the established pattern used by the other
overwrite-against-non-local tests in this file (lines ~1305, ~1331).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Andrew Altshuler
|
1493ea4ce6
|
docs(readme): widen --server to <name|url>; show --as on cluster apply (#264)
Two accuracy nits from the #263 bot review, against the cookbook's authoritative addressing reference: `--server` accepts a name OR a literal `http(s)://` URL (prose had narrowed it to `<name>`), and `cluster apply` should show `--as <actor>` so the control-plane action's attribution is explicit. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
df0b0eadd1
|
docs(readme): make Common Commands cluster-first, not store-first (#263)
Follow-up to #262: the command examples led with single-file `--store ./graph.omni` for everything (init/load/query/mutate/branch), which reads as a single-graph-file product — the opposite of the cluster-first paradigm. Reframe so the everyday loop is the headline: declare a cluster → `cluster apply` → `omnigraph-server --cluster …` → work against the served graph with `--server <name> --graph <id>`, invoking stored queries by name. `--store` is demoted to a clearly-secondary "Local / ad-hoc" note for standalone-graph iteration. Folds the former separate "Serving" section into step 1. All served examples verified to resolve correctly (`→ http://…/graphs/<id> (served)`). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
a83da2ccfd
|
docs(readme): align with cluster-first paradigm + RFC-011 CLI ergonomics (#262)
* docs(readme): align with the cluster-first paradigm + RFC-011 CLI ergonomics The README's command examples and crate list predated 0.7.0. Update them: - Common Commands: capability/addressing model — positional/`--store` for direct storage, `--server` for served graphs (no positional `http(s)://`), `query`/ `mutate` invoke a stored query by name (positional is the query name), `load` requires `--mode`. Drop the false "same URI works for local/s3/http" claim and the deprecated `read`/`change` + `--name` forms; mention `~/.omnigraph/config.yaml`. - New "Serving (cluster-first)" section: a deployment is a cluster.yaml converged with `cluster apply` and served by `omnigraph-server --cluster <dir|s3://…>` (no single-graph mode; config-free boot from a bucket). - Fix the stale docs link (`docs/user/cli.md` → `cli/index.md` + the CLI reference) after the docs were restructured into topic sections. - Workspace Crates: list all seven (add omnigraph-policy, omnigraph-api-types, omnigraph-cluster) with cluster-first framing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(agents): fix the stale `read --name` quick-reference example AGENTS.md (always-loaded; symlinked to CLAUDE.md) still showed the deprecated `omnigraph read --query … --name … <s3-uri>` form. Update it to the RFC-011 shape: `omnigraph query --query … <name> … --store <uri>` (read→query, --name→ positional query name, positional URI→--store). Addresses the bot-review finding on #262. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
e539aa1693
|
ci(release): single-writer release publish — fix the matrix finalize race (#261)
The release matrix ran three jobs (linux/macos/windows) that each called `softprops/action-gh-release` to create-or-update the SAME release. Concurrent "Finalizing release" calls exhausted the action's retries, so whole platforms' assets were dropped (on v0.7.0: linux won; macOS and Windows failed and their binaries never attached). Split build from publish: - the matrix now only uploads workflow artifacts (`actions/upload-artifact`); - a single `publish_release` job downloads them all and makes one `action-gh-release` call — the sole writer, so no race. Also add a `tag` workflow_dispatch input (resolved as `inputs.tag || github.ref_name`) so a tag can be re-published without re-cutting it — used to finish v0.7.0. `update_homebrew_tap` and `smoke_windows_installer` now depend on `publish_release` and use the resolved tag (not `GITHUB_REF_NAME`, which is the branch on a dispatch). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
7540c86fa0
|
ci(publish): add omnigraph-api-types + omnigraph-cluster to the crates.io publish order (#260)
The publish-crates workflow's list predated two workspace crates: the shared
wire-DTO crate `omnigraph-api-types` (RFC-009) and `omnigraph-cluster`. On the
v0.7.0 tag it published compiler/policy/engine, then failed on `omnigraph-server`
("no matching package named `omnigraph-api-types`") because that dependency was
never published.
Insert both in dependency order (after omnigraph-engine, before
omnigraph-server). The workflow is idempotent (per-crate version check), so a
re-dispatch for v0.7.0 skips the three already-published crates and finishes
api-types → cluster → server → cli.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Andrew Altshuler
|
80ef9964fe
|
docs(releases): finalize v0.7.0 notes (#259)
Reconcile the v0.7.0 release notes with what 0.7.0 actually ships. The draft was mid-cycle; two facts changed and a late-cycle arc was missing. - omnigraph.yaml is REMOVED (not deprecated): drop the deprecation-window framing (config migrate, OMNIGRAPH_NO_LEGACY_CONFIG, OMNIGRAPH_SUPPRESS_YAML_ DEPRECATION); the two-surface config (cluster.yaml + ~/.omnigraph/config.yaml) is the only config. - Cluster-only server: the server boots only from --cluster; no single-graph flat-route / positional-URI / omnigraph.yaml-graphs boot. Deprecated-route Link headers are the sibling-relative form (<load>, not </load>). - Add the RFC-011 tail: defaults.store, profile list/show, schema-apply refusal (CLI signpost + server 409), read-only aliases, the any/served/direct/control/ local capability vocabulary, removed legacy data-plane addressing. - New "Engine & substrate" section: Lance 6→7, indexed traversal, scalar-index/ query-latency, index-materialization-deferred, recovery liveness, branch-fork self-heal, composite @unique. - New "Embeddings (RFC-012)" section + breaking bullet: provider-independent client (OpenRouter default), @embed same-space validation, the default-provider flip (OMNIGRAPH_EMBED_PROVIDER=gemini for Gemini-direct; OMNIGRAPH_GEMINI_BASE_URL dropped). - Upgrade notes: replace the false "omnigraph.yaml keeps working / config migrate" guidance with the manual cluster.yaml + operator-config path; add server --cluster and embeddings notes. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
51852fdbed
|
Merge pull request #248 from ModernRelay/ragnorc/shaping-config-integration
RFC-012: embedding provider independence + remove dead client |
||
|
aaltshuler
|
e33041c8b7 | Fix aws server startup config test | ||
|
aaltshuler
|
4f8c71fa23 |
Merge remote-tracking branch 'origin/main' into ragnorc/shaping-config-integration
# Conflicts: # crates/omnigraph-cluster/src/lib.rs # crates/omnigraph-cluster/src/serve.rs # crates/omnigraph-server/src/lib.rs # crates/omnigraph-server/src/settings.rs # docs/user/clusters/config.md |
||
|
aaltshuler
|
16e4a833c0 | Wire cluster embedding providers | ||
|
Andrew Altshuler
|
b5658dc696
|
[codex] fix RFC-011 follow-up regressions (#258)
* fix rfc-011 follow-up regressions
* test(cli): remove served schema-apply tests obsoleted by the cluster 409
This PR disables server-side schema apply for cluster-backed serving (409 →
`omnigraph cluster apply`). Two system_local tests still drove *served* schema
apply against a spawned `--cluster` server and asserted the pre-409 behavior, so
they failed under `cargo test --workspace`:
- `local_cli_schema_apply_enforces_engine_layer_policy` — expected a per-actor
policy `denied`/allow on the served route; the route now 409s for everyone
before policy runs.
- `local_cli_schema_apply_rejects_stored_query_breakage_before_publish` —
expected a served apply to reject a stored-query breakage; the route now 409s
before any apply.
Both exercise a path the PR intentionally removed. Their surviving coverage:
the 409 itself is pinned by `schema_routes::schema_apply_route_refuses_cluster_backed_server_mode`
(asserts 409 + no mutation); stored-query-breakage-before-publish stays covered
by `schema_routes::schema_apply_route_rejects_stored_query_breakage_before_publish`
(single-mode); engine-layer schema_apply Cedar enforcement stays covered by
`policy_engine_chassis`. Remove the obsolete served versions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(server): report the cluster-backed schema-apply 409 after the Cedar gate
The 409 ("schema apply is disabled for cluster-backed serving") fired at the top
of `server_schema_apply`, before `authorize_request`. An authenticated-but-
unauthorized actor therefore learned the server is cluster-backed (409) instead
of getting a normal 403 — leaking topology before authorization, against the
same posture that keeps `GET /graphs` default-deny.
Move the 409 below the Cedar gate so the route reports 401 → 403 → 409: an
unauthorized actor gets 403, and only an actor authorized for `schema_apply`
sees the actionable "use `omnigraph cluster apply`" 409. (An open/unauthenticated
server still 409s, as it has no topology to protect.)
Regression: `schema_apply_route_cluster_backed_denies_unauthorized_actor_before_409`
(POLICY_YAML grants no schema_apply → act-ragnor gets 403, not 409). Addresses the
bot-review finding on #258.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
aaltshuler
|
8d2128438e |
fix(cli): quote @embed annotation values in schema show so they round-trip
`render_annotations` emitted `@embed` values unquoted — `@embed(title,
model=openai/text-embedding-3-large)`. The parser stores values via
`decode_string_literal` (quotes stripped) and `annotation_kwarg` requires a
quoted `literal`, so the rendered output did not re-parse: a `model` containing
`/` or `-` is not a valid bare token. `schema show` therefore produced schema
text that `schema apply`/lint would reject.
Re-quote the positional value and every kwarg value as string literals, so
`schema show` reproduces `@embed("title", model="openai/text-embedding-3-large")`
and round-trips. Regression: `render_annotations_quotes_values_so_embed_round_trips`
parses the rendered form back through the schema grammar.
Addresses the bot-review finding on #248.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|