* feat(cli): RFC-011 Slice A — operator-config scope structs (profiles/clusters/defaults)
Additive operator-config surface for the RFC-011 scope model. No behavior
change yet — these structs are parsed but not consumed until the scope
resolver lands.
- OperatorConfig gains `profiles:` (name → OperatorProfile) and `clusters:`
(name → OperatorCluster { root }) — the latter the only place a storage
root appears in operator config (RFC-011 storage-root rule).
- OperatorDefaults gains `server` and `default_graph` (the flat-default scope).
- OperatorProfile binds one of {server, cluster, store} + default_graph;
`binding()` validates exactly-one on use and returns a ScopeBinding.
- Accessors profile()/cluster_root()/default_server()/default_graph();
unknown-key warnings extended to the new blocks (forward-compat preserved —
old configs still load, new keys are no longer "unknown").
Tests: parse profiles/clusters/scope-defaults, binding rejects zero/multiple
entities, unknown keys in a profile warn.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(cli): RFC-011 Slice A — scope resolver + --profile/--store, wired (additive)
Translate the new scope inputs into the existing addressing tuple, in front of
the unchanged resolvers. Purely additive: an explicit address
(--uri/--target/--server/--store) passes straight through, so every existing
invocation is byte-for-byte unchanged.
- scope.rs: resolve_scope() with the RFC-011 precedence (explicit > --profile /
OMNIGRAPH_PROFILE > flat defaults.server), producing the effective
(server, graph, uri, target) for data verbs and (cluster, cluster_graph) for
maintenance. Plane×scope capability check (server scope rejected on a
maintenance verb; cluster scope rejected on a data verb; store rejects --graph)
fires only on the new paths. 9 unit tests.
- cli.rs: global --profile <NAME> and --store <URI>. (--graph keeps
requires=server for now; profile/default graph comes from default_graph —
profile+--graph override is deferred to the --cluster-graph rework.)
- client.rs: the two GraphClient factories call resolve_scope (Plane::Data) up
front; the explicit branch reproduces today's behavior exactly.
- main.rs: the 15 data call sites forward --profile/--store; the 3 maintenance
verbs consult the scope (Plane::Storage) only when no explicit per-command
address is given, so cluster-binding profiles and --store reach
optimize/repair/cleanup.
Verified: the full omnigraph-cli suite (221 tests) stays green untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test+docs(cli): RFC-011 Slice A — end-to-end scope test + reference docs
- cli_data.rs: prove --store and a --profile store binding drive a read
identically to the legacy positional URI (the additive-coexistence contract),
end to end against a local graph (no server needed).
- cli/reference.md: document profiles/clusters/defaults.server/default_graph,
the --profile/--store flags, and a "Scopes & profiles" section; note the model
coexists with legacy addressing (nothing removed yet).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* test(cli): reproduce branch-delete //branches 404 (failing)
Regression test for the `branch delete` 404 over a multi-graph
`--server`/`--graph` target: the composed URL must be `<base>/branches/<name>`
with no empty `//` segment. Fails against the current `remote_branch_url`,
which appends a trailing slash before extending path segments and so emits
`…/graphs/p9-os//branches/tmpbranch`. The next commit fixes it.
left: "http://host/graphs/p9-os//branches/tmpbranch"
right: "http://host/graphs/p9-os/branches/tmpbranch"
* fix(cli): unify remote URL builder, close the //branches 404 class
Correct-by-design fix for the failing test in the previous commit. The bug was
not specific to `branch delete`: URL assembly was scattered across a
string-concat `remote_url`, a url-crate `remote_branch_url`, and several
`format!` interpolations that left dynamic path/query components un-encoded
(commit id in the path, branch in the query string). `branch delete` was the
instance that surfaced because it is the only verb that puts a dynamic value in
the path.
Replace both helpers with one `remote_url(base, segments, query)` that every
remote call routes through. Callers pass structured segments and query pairs,
so trailing-slash normalization (pop_if_empty) and per-segment / per-value
percent-encoding live in one place. A stray `//` or an un-encoded dynamic
component is no longer representable, closing the whole class rather than the
reported instance.
Migrates the previous commit's failing test to the new builder and adds the
single-graph, trailing-slash, slash-in-name, commit-id-path, and query-value
cases (the last two cover the previously latent siblings). All 16 callsites
migrated; `remote_branch_url` removed.
Move the 23 flat docs/user/*.md files into topic subdirectories so the
user guide is organized by area (schema, queries, search, branching, cli,
operations, clusters, concepts, reference) instead of a flat list. This is
a pure structural move — whole files relocated, every cross-doc link
recomputed, no prose rewrites or content splits (those follow in Phase 2).
- 19 `git mv`s (install.md, deployment.md stay top-level); history preserved
(renames detected at 92–100% similarity).
- All intra-doc links, AGENTS.md's topic table (52 pointers), and the
docs/dev + docs/releases back-links recomputed via relpath from each
file's new location.
- docs/user/index.md rewritten as a sectioned nav hub.
- Fixed 5 doc-path references in Rust (comments + two user-facing server
settings error strings) to point at the new locations.
Verified: zero broken .md links across tracked docs; check-agents-md.sh
green (with the untracked scratch docs set aside); touched crates build.
Note: the public site (omnigraph-web) imports docs/ via a flat-only script;
its import-docs.mjs needs a subdir-aware update before the next re-sync.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(cluster): cluster_root_for_graph_uri detection helper (RFC-010 Slice 3)
Public helper the CLI uses to refuse `init` into a cluster-managed location:
given a graph storage URI of the cluster layout (`<root>/graphs/<id>.omni`),
return the cluster root if `<root>` holds `__cluster/state.json`, else None.
Cheap by construction — a URI that doesn't match the `<root>/graphs/<id>.omni`
shape returns None with zero I/O, so ordinary `init` targets never probe
storage. Works for file:// and s3:// via the storage adapter. Adds two
ClusterStore accessors (`display_root`, `has_state`).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(cli): cluster-managed maintenance addressing + init signpost (RFC-010 Slice 3)
Two cluster-graph-aware CLI behaviors, sharing the cluster-resolution path.
Maintenance addressing. `optimize`/`repair`/`cleanup` gain
`--cluster <dir|s3://…> --cluster-graph <id>`, which resolves the graph's
storage URI from the served cluster snapshot (the same truth a `--cluster`
server boots from — `read_serving_snapshot*`) and opens it embedded. The
operator no longer hand-types `<storage>/graphs/<id>.omni`. A distinct flag is
required because the global `--graph` is `requires = server` and means a remote
multi-graph id. clap enforces both-or-neither and exclusion with the positional
URI / `--target`; an unserved graph errors loudly, pointing at `cluster apply`.
init signpost. `init` refuses a cluster-managed positional path (the
`<root>/graphs/<id>.omni` layout where `<root>` holds `__cluster/state.json`,
detected by `cluster_root_for_graph_uri`) and points at `cluster apply` — graphs
in an established cluster are created with ledger/recovery/approvals, not by
hand. The check is gated on the path shape, so ordinary `init` does no extra I/O
and existing pre-apply cluster-graph inits are unaffected.
planes guard remediation now also mentions `--cluster … --cluster-graph …`
(the two Slice-1 guard-string tests track it). Docs updated (cli-reference
Command planes, maintenance.md, cluster.md §7); the stale "no S3-hosted cluster
directories" limitation is dropped (RFC-006 landed it).
Tests (cli_cluster.rs, reusing the apply-a-cluster fixture): resolve by id,
unknown-id error, `--cluster` requires `--cluster-graph`, init refusal +
signpost, and ordinary init still works.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(cli): resolve cluster graphs from the state ledger, not the serving snapshot
Addresses the Greptile review on #221. `read_serving_snapshot*` does
all-or-nothing serving validation — recovery-sidecar checks plus a digest
verify of every catalog payload (query .gq, policy blobs). Using it to resolve
a maintenance target coupled `optimize`/`repair`/`cleanup` to the readiness of
unrelated resources: a single corrupt policy blob, or a pending recovery sweep,
would block the command before it could touch the graph — worst for `repair`,
the tool you reach for *when the cluster is degraded*.
Add `omnigraph_cluster::resolve_graph_storage_uri(cluster, graph_id)`: read the
state ledger, confirm the graph is in the applied revision, return
`graph_root(id)` — the URI is deterministically derivable, no catalog
validation. The CLI's cluster resolver now calls it.
Test: `optimize --cluster … --cluster-graph …` still resolves after the catalog
payloads (`__cluster/resources/`) are removed — the ledger-only path is not
blocked by degraded/unrelated catalog state.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* chore(deps): bump clap to 4.6.1
Workspace constraint "4" → "4.6" so the resolver picks up the 4.6 line
(a plain `cargo update` stayed on 4.5.x). clap 4.5.58 → 4.6.1
(clap_builder 4.6.0, clap_derive 4.6.1). Minor bump, no API breakage; the
workspace builds and all CLI suites pass unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(cli): group --help by plane (RFC-010 Slice 2)
Slice 1 declared the planes (the command_plane table + the wrong-plane
guard); this makes them visible in `--help`. clap can't print labeled
heading rows between subcommand groups (verified against the source —
help_heading is args-only, {subcommands} is one flat block), so per the
chosen approach: cluster + legend.
- Reorder the `Command` enum into plane bands (clap lists subcommands in
declaration order): data (query, mutate, load, branch, snapshot, export,
commit, schema, graphs) → storage/local-graph ops (init, optimize,
repair, cleanup, lint, queries) → control (cluster) → session (policy,
embed, login, logout, config, version). No magic display_order numbers —
the source order IS the help order, with band comments for readers. The
band placement matches `command_plane` (lint/queries are storage-plane:
they reject --server), so the help grouping and the guard agree.
- Add an `after_help` legend on `Cli` naming the planes. Written to
describe the planes (not enumerate every command) so it doesn't drift.
Help-polish (post-review): hide the deprecated `ingest` from the list
(still a valid command); trim the long `login` and `--as` descriptions to
one line each so the columns don't blow up.
The behavioral source of truth for planes stays `planes::command_plane`;
this ordering is its cosmetic counterpart.
Test: `help_groups_commands_by_plane` pins the legend phrase + the cluster
ordering (query < optimize < cluster). Doc: a line under cli-reference's
*Command planes* section.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(cli): qualify mixed-plane commands in the --help legend
Addresses the Greptile P2 on #220: the legend placed `schema` entirely in
Data and `queries` entirely in Storage, but per `command_plane` the
subcommands differ — `schema plan` is storage-plane (rejects --server) and
`queries list` is session (no graph). A user reading the legend then running
`schema plan --server` would hit a rejection contradicting it. The Commands
list is one entry per top-level command (necessarily coarse), so the legend
carries the nuance: `schema [plan: storage]` and `queries [list: session]`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Addresses the Greptile review on #217:
P1 — `lint` reported two different names. `command_label` returns `lint`, but
`execute_query_lint` passed `"query lint"` as the resolver operation string, so
`lint --server` said `lint` while `lint <https>` said `query lint`. Both were
pinned by tests. `query lint` is the *deprecated* alias (argv-rewritten to
`lint`), so the canonical name is `lint`: switch both user-facing strings in
`execute_query_lint` (the storage-plane bail label and the
requires-schema-or-target usage message) to `lint`, and update the two pinned
assertions in `cli_data.rs`.
P2 — user-doc debt (AGENTS.md rule 1: error text is observable behavior).
Document the plane model in `cli-reference.md` (new *Command planes* section:
data vs storage/maintenance vs control, which addressing flags apply, and the
declared wrong-plane / remote-target errors), and add an addressing note to
`maintenance.md` cross-referencing it.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(cli): declared plane capability surface + wrong-plane guard (RFC-010 Slice 1)
New `planes.rs` is the single source of truth for which plane each subcommand
belongs to (Data / Storage / Control / Session). `command_plane` is an
exhaustive match — adding a `Command` variant is a compile error until its
plane is declared, so the surface cannot silently drift from the command set.
It descends into the nested enums where the plane differs per subcommand
(`schema plan` is storage while `schema show/apply` are data; `queries
validate` opens the graph while `queries list` reads only config).
`guard_addressing` runs once in `main` before dispatch: the data-plane
addressing flags `--server`/`--graph` on any non-data verb now fail with one
declared, pinned error instead of being silently ignored (`optimize --server
prod` previously dropped `--server`). `init`'s message drops the `--target`
half since it takes only a positional URI today.
Test: `cli_schema_config::schema_plan_with_server_flag_errors_wrong_plane`
pins the per-subcommand label, proving the guard descends into the nested enum.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(cli): storage-plane verbs fail loudly on a remote target (RFC-010 Slice 1)
`optimize`/`repair`/`cleanup` switch from `resolve_uri` to `resolve_local_uri`,
so a `--target` (or positional URI) that resolves to a remote server now fails
with a declared storage-plane message instead of whatever `Omnigraph::open`
said about an `http(s)://` URI. The `resolve_local_graph` bail is reworded to
that storage-plane message, so every storage verb already on the local resolver
(`schema plan`, `queries validate`, `lint`) speaks with one voice.
Net: `optimize --target knowledge` resolves to the graph's storage URI and runs
embedded; `optimize --target prod` (remote) fails loudly; `optimize --server`
is caught earlier by the guard. Positional-URI invocations are unchanged.
Tests (pinned strings, per RFC-010's test plan): optimize happy path on a local
graph, `optimize --server` wrong-plane error, `optimize <https>` storage-plane
error; the existing `query_lint_rejects_http_targets_without_schema` assertion
is updated to the new shared message.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The last two embedded-vs-remote forks move onto the enum, so every such
`if` in the CLI now lives in client.rs — the point of the refactor.
- `export<W: Write>`: the streaming verb 3b deferred (writes to a writer,
chunks the HTTP response body, rather than returning a DTO). Embedded
calls db.export_jsonl_to_writer; Remote streams the chunked body through.
Opens WITHOUT policy (like reads), so it routes via resolve().
- `list_graphs`: remote-only by design (no local enumeration endpoint), so
the Embedded arm keeps the loud "requires a remote multi-graph server"
bail verbatim. Routing it through the enum still buys the shared
resolve() addressing/token preamble the arm hand-rolled.
Retire the now-orphaned execute_export_to_writer /
execute_export_remote_to_writer pair, and sweep two pre-existing dead fns
while in the files: inferred_config_path (helpers.rs) and yaml_string
(output.rs, shadowed by test-local copies).
parity_matrix gains one row, parity_export — the single intended matrix
change in this phase. Export is a JSONL stream, not a single --json doc,
so it compares the two arms' output line-wise (sorted; twin graphs are
byte-copies so rows need no scrubbing). graphs-list gets no row: its
remote-only behavior is a documented exclusion, not an equality case.
Full workspace tests pass; all 12 parity rows green.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Phase 3a put the GraphClient enum in place and collapsed the five uniform
read forks. 3b folds the remaining data-plane forks onto the same enum:
load, ingest, mutate, query, branch create/delete/merge, and schema apply.
The wrinkle 3a deferred was the local policy attachment. Reads and query
open the local engine without a policy; writes open through
open_local_db_with_policy and attribute a resolved actor. So the Embedded
variant grows an optional policy context (graph/actor) filled by a second
factory, resolve_with_policy; resolve() leaves it empty. open_embedded
picks the open path from whether the context is present, preserving both
of today's behaviors exactly. query still uses resolve() (no policy), as
the read path did.
apply_schema takes the catalog-validator closure as impl FnOnce(&Catalog)
— the embedded arm runs it inside apply_schema_as_with_catalog_check, the
remote arm ignores it (the server runs its own check). That non-object-safe
closure is why GraphClient is an enum, not a trait. The stored-query
registry is still built caller-side and only for the local path.
load and ingest stay separate methods: same operation, but load surfaces
the CLI LoadOutput (two distinct per-arm mappings preserved) while ingest
surfaces the wire IngestOutput. The now-fully-dead execute_read/
execute_read_remote and execute_change/execute_change_remote pairs are
retired (legacy_change_request_body stays — client.rs uses it); the export
pair remains for 3c.
The Phase-1 parity matrix is unchanged and green; full workspace tests pass.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The embedded-vs-remote split gets one home: a GraphClient enum
(Embedded { uri } | Remote { http, base_url, token }) with a resolve()
factory that absorbs the shared preamble (apply_server_flag -> token ->
URI/remoteness) and a verb method per command. The five uniform read
forks — branch list, commit list, commit show, schema show, snapshot —
collapse from per-command if-graph-is-remote else to one line each
(main.rs: -113/+47). Behavior identical per verb (local reads still open
WITHOUT policy, as today); the Phase-1 parity matrix is the referee and
passes textually unchanged.
Enum, not the RFC trait: only two variants ever, and inherent async
methods avoid async_trait boxing and the apply_schema closure that is not
object-safe (3b) — same one-body-two-impls collapse, less ceremony.
Scope: the uniform reads only. The query verb (policy-open + operator-
alias early-return + param merge) joins the write verbs in 3b;
export/streaming and graphs-list in 3c, where the now-shared
execute_*_remote/execute_* pairs get retired.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The CLI's wire-DTO imports repoint from omnigraph_server::api to
omnigraph-api-types (the server's other exports — queries registry,
config types — still come from omnigraph-server). The local Load arm's
inline LoadOutput hand-construction in main.rs is extracted into
load_output_from_result next to load_output_from_tables in output.rs, so
both '-> LoadOutput' mappings (engine LoadResult for local, wire
IngestOutput for remote) live in one place.
Deviation from the plan, with reason: LoadOutput stays CLI-side rather
than moving into the wire-DTO crate — it is a rendered CLI output type,
not an HTTP wire DTO, and its mapping consumes a CLI clap type
(CliLoadMode). The shared crate stays strictly wire DTOs. Shapes
unchanged: the parity matrix passes textually unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The referee before any unification moves: every forked verb runs once
against the local graph and once against a spawned server on a twin copy
of the same fixture, with the SAME actor (--as locally; bearer-resolved
remotely) and the SAME Cedar bundle on both arms — like-for-like
enforcement is part of the harness (a tokens-only server is default-deny
by design; comparing that against a bare local arm measures
configuration, not the fork). Declared-volatile fields (ids, wall-clock,
transport locations) scrub to placeholders; everything else must match
exactly, and exit codes must match for shared failures.
Headline result: 11 rows green with an EMPTY divergence ledger — the
arms agree on every verb today. The ledger (KNOWN_DIVERGENCES) exists so
any future divergence is pinned or filed, never silently repaired;
repairs are Phase 3's job, gated by this referee staying green.
One engine observation surfaced and filed (#207): inline execution with
a declared-but-unbound param matches ALL rows on both arms, while the
stored-query invoke path hard-errors — a cross-path asymmetry the matrix
pins as agreeing behavior pending a deliberate fix. Documented
exclusions (graphs list, ingest/load-over-/ingest, storage-plane verbs)
map to RFC-009 Phases 4-5.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
All six crate manifests + their path-dependency constraints, Cargo.lock,
the regenerated openapi.json version metadata, AGENTS.md's surveyed
version, and the v0.7.0 release notes (object-storage clusters,
config-free --cluster serving, the operator config surface, keyed
credentials, operator targeting/aliases, and the omnigraph.yaml
deprecation stages).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Opt-in: with the env set, loading a legacy omnigraph.yaml is a hard
error pointing at config migrate — the regression guard for migrated
teams (a stray legacy file would otherwise silently outrank operator
config during the window) and the rehearsal for stage 5's removal.
Strict refuses the FILE, never its absence: flag-less invocations on
migrated setups are untouched. Inert unless set.
The RFC's stages-1-3-then-4 release gap collapsed honestly: no version
boundary was crossed between them, so all four ship in the same release
(noted in the RFC).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Andrew's call, and the right one by the repo's own lens: a minimal
cluster.yaml is five lines; a generator is a second copy of the schema to
keep in sync forever, emitting a file that is unusable until hand-edited
anyway (graphs: {} cannot apply or serve). Terraform has no config
scaffolder either. New users copy from the cluster quick-start; migrants
get a ready-to-review cluster.yaml from config migrate. RFC-008 stage 3
becomes purely subtractive.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
omnigraph init no longer writes a legacy config into cwd (the source of
the earlier test-pollution bug, and a scaffold for a deprecated file);
the scaffolder is deleted. omnigraph cluster init scaffolds the
replacement: a minimal valid cluster.yaml (version: 1, optional
metadata.name / storage:, a commented graphs example), refusing to
overwrite. The scaffold validates clean via cluster validate in the e2e.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reads a legacy omnigraph.yaml and produces the three-section split: team
half as a ready-to-review cluster.yaml proposal (graphs with TODO schema
pointers — the legacy file never knew schemas — per-graph queries
directories, policies with applies_to bindings), personal half as an
operator-config merge (actor, output/table defaults — OperatorDefaults
gains the two table keys with their cascade hops — remote graphs with
bearer_token_env become servers entries plus a printed login step, and
legacy aliases split per the RFC: content to the catalog as a manual
step, binding to an operator alias), plus a dropped-keys section with
reasons. Touches nothing without --write; with it, the operator merge is
key-level (existing entries always win; prior file backed up), and
cluster.yaml is emitted only when absent (else cluster.yaml.proposed).
--json emits the report structurally.
The completeness contract is a unit test: every top-level key of the
legacy schema must classify somewhere, or the RFC-008 map has a bug.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Loading a legacy file (flag, env, or cwd-found — never on defaults) emits
one stderr block listing each key actually present with its destination
from RFC-008's migration map — the map applied to YOUR file, not a
generic banner. Once per process; both binaries warn (cluster-mode boots
never reach load_config, silent by construction); suppressible via
OMNIGRAPH_SUPPRESS_YAML_DEPRECATION=1 for CI logs during the window.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Caught on the live smoke: with --alias, the first bare CLI arg lands in
the hidden legacy_uri positional, so an operator alias's positional param
never bound ('parameter not provided' from the server). An operator alias
always knows its target, so the existing normalize_legacy_alias_uri
reclaims the swallowed positional as the first alias arg — same rule the
legacy path already applies.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
aliases: in the operator config bind a personal name to (server, graph,
stored-query NAME, positional arg mapping, fixed param defaults, format)
— zero content, per the ratified bindings-not-content model. Invocation
goes through the server's stored-query endpoint (POST
{base}/graphs/{g}/queries/{name}) with the keyed credential resolving via
the ordinary URL match; param precedence --params > positionals > fixed
defaults; the result renders through the existing format cascade with the
alias's format as its hop. A legacy omnigraph.yaml alias with the same
name wins during the RFC-008 window, with a warning naming both.
E2e (spawned policy-gated server, invoke_query granted via a per-graph
bundle): the alias invokes with name + one positional and nothing else —
server, graph, query, and token all from the operator layer; --server/
--graph explicit targeting; unknown --server lists defined names;
--server exclusive with a positional URI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Global flags --server (operator-defined server name) and --graph (graph id
on a multi-graph server, requires --server) resolve to the effective
remote URI through one helper and feed the ordinary uri slot — graph
resolution and the PR-2 keyed-token URL match work unchanged; the flag is
sugar for a URI the operator already owns. Exclusive with a positional
URI and --target (loud error, never silent precedence). Unknown names
fail listing the servers that ARE defined.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The operator config gains servers: (name -> url; never a token). A remote
command whose URL prefix-matches an operator server resolves its bearer
token through the keyed chain first — OMNIGRAPH_TOKEN_<NAME> env, then the
[<name>] section of ~/.omnigraph/credentials (created 0600 via temp+rename,
#139 finding 7; group/world-readable files refused loudly) — falling
through to the legacy chain unchanged. URL keying makes §D5 rule 3
structural: a token is only ever sent to the server it is keyed to.
Longest-prefix matching with a path-boundary check (http://h:8080 never
matches http://h:8080-evil). Inserting the keyed hop above the legacy chain
is safe by construction — no existing setup can have servers: defined.
omnigraph login <name> stores/rotates one section (token from --token or
one stdin line — the pipe flow keeps secrets out of shell history);
omnigraph logout removes it, idempotently; logging in before declaring the
server warns instead of failing (the gh model).
Coverage: URL-match/no-substring-trap, credentials round-trip preserving
sibling sections, 0600 write + over-permissive refusal, env-name mapping;
the legacy resolve test is now hermetic against a real ~/.omnigraph and
asserts byte-identical legacy behavior with no servers defined; one
spawned-binary e2e walks the whole lifecycle against an authed server:
refusal -> wrong-token login (stdin) -> rotate (--token) -> authorized read
-> env-beats-file -> non-matching-URL negative -> logout revokes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
~/.omnigraph/config.yaml joins the resolution chains as the operator
surface: operator.actor becomes the last hop of THE actor chain (--as >
legacy cli.actor during the RFC-008 window > operator.actor > none, one
implementation for direct-engine and cluster commands alike) and
defaults.output joins the read-format cascade below every more-specific
source. Discovery honors $OMNIGRAPH_HOME (tilde-expanded, #139 finding 9);
an absent file is an empty layer; unknown keys WARN and load (a file
written for later slices must not break this CLI); malformed YAML is a
loud error. The module is CLI-only — the server never reads operator
config (invariant 11 by construction).
$OMNIGRAPH_CONFIG becomes a first-class stand-in for --config in
load_config (flag > env > ./omnigraph.yaml), one meaning in both binaries.
The test harness pins hermeticity: spawned binaries get a nonexistent
OMNIGRAPH_HOME by default so no test ever reads the developer's real
operator config. New coverage: loader unit tests, the env-precedence
matrix on load_config_in, and spawned-binary e2es for the actor chain
(operator wins with no flag/legacy key; legacy outranks it; --as wins) and
the format cascade.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verbatim moves: the clap surface (every command/subcommand/arg struct) to
cli.rs, resolution helpers (config/actor/graph/branch/query, remote HTTP,
env/token, scaffolding) to helpers.rs, human/JSON formatting to output.rs,
the in-source test mod to main_tests.rs via #[path]. main.rs (1,184 lines)
keeps main() and the dispatch match. Visibility bumps only; 22 binary
tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LocalStateBackend becomes ClusterStore: every stored byte — state ledger,
lock, recovery sidecars, approval artifacts — now flows through the
engine's StorageAdapter, making file:// and s3:// one code path. Behavior
on the file backend is byte-compatible (layout, CAS semantics, diagnostics,
lock release timing) and the entire pre-existing suite passes unchanged.
Mechanics: the ledger CAS keeps its public sha256 vocabulary while the
physical swap is token-conditioned (ETag If-Match on S3 via PR #186's
primitives; content-token + temp/rename locally — the pre-port semantics);
the lock is a create-only put (genuinely cross-machine on object stores)
with deterministic drop-release locally and best-effort spawned release on
S3; sidecars/approvals address by URI (SweepOutcome and the executors carry
strings); sweep row-1 retirement joins the uniform deferred post-CAS
cleanup. ClusterStore also gains the catalog-payload and graph-root
methods that commit 2 wires in.
Async ripple: status/force-unlock/serving-snapshot and the server's
settings loader chain go async (CLI dispatch and ~20 test hosts follow,
mechanically). tokio joins the cluster crate's runtime deps for the lock
guard's handle.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
omnigraph load is now the single data-write command:
- works against remote graphs (POSTs the server's /ingest endpoint with the
same bearer/actor resolution as other remote commands) — previously load
was the only data command forced to open Lance storage directly
- --from <base> opts into fork-if-missing for --branch (the former ingest
semantics); without --from a missing branch is an error, never a fork
- --mode is now required: overwrite is destructive, so there is no implicit
default (the old silent default was overwrite)
- output gains base_branch/branch_created (and table sums on remote loads)
omnigraph ingest stays as a deprecated alias (defaults preserved: --from
main --mode merge) that prints a one-line warning to stderr, matching the
read/change deprecation convention; removal in a later release.
Docs updated in the same change: cli.md, cli-reference.md, policy.md,
audit.md, execution.md (unified load section), AGENTS.md quick-flow,
README.md.
BREAKING CHANGE: scripts running omnigraph load without --mode must now
pass it explicitly (previously defaulted to the destructive overwrite).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Branch creation becomes opt-in by presence of the request's 'from' field.
Previously the handler defaulted from to 'main' and always auto-created a
missing branch — a typo'd branch name silently forked main and landed the
data there, with the client none the wiser. Now a request without 'from'
against a missing branch returns 404 branch-not-found and creates nothing;
with 'from' set, fork-if-missing behaves as before. The BranchCreate
authority is only consulted when a fork will actually happen.
The handler calls the unified load_as directly (the deprecated ingest_as
shim is no longer used in the server). IngestOutput.base_branch becomes
nullable: it echoes the request's 'from' and is null when absent. OpenAPI
regenerated; the CLI's local ingest arm moves to load_file_as + the new
converter shape.
BREAKING CHANGE: clients that relied on implicit fork-from-main with 'from'
omitted must now pass from='main' explicitly. IngestOutput.base_branch is
now nullable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
load_as/load_file_as gain a base: Option<&str> parameter: with Some(base) a
missing target branch is forked from base first (the former ingest
semantics); with None the target branch must exist — staging fails on an
unknown branch, so a typo'd name can never create one. LoadResult gains
branch/base_branch/branch_created metadata (additive).
The ingest family (ingest, ingest_as, ingest_file, ingest_file_as) becomes
#[deprecated] shims over load_as that preserve the historical contract
exactly (from: None still means fork from main; base recorded even when no
fork happened). IngestResult and to_ingest_tables stay for the shims and
the server until the removal release.
The layered policy check is unchanged: Change on the target branch always,
BranchCreate additionally when a fork actually happens (enforced inside
branch_create_from_as with the actor threaded through).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- resolve_cluster_actor uses load_config directly: load_cli_config also
loads auth.env_file into the process env — a second thing, violating the
documented 'exactly one thing' omnigraph.yaml contract for cluster ops.
- resolve_cli_actor gets its doc comment back (the inserted helper had
absorbed the contiguous /// block).
- The actor-default test imports once as setup and asserts on apply alone,
idempotently, instead of re-importing inside the assertion helper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
local_cli_s3_end_to_end_init_load_read_flow ran `omnigraph init` without a
current_dir, so init's project scaffold landed in crates/omnigraph-cli/ —
poisoning any later test that resolves a graph target from the cwd config
(query_lint_requires_schema_or_resolvable_graph_target fails determinis-
tically once the file exists). Only manifests when OMNIGRAPH_S3_TEST_BUCKET
is set, which is why local FS runs and CI's scoped rustfs job never caught
it. The init and load calls now run inside the test's tempdir.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A --cluster server process whose cwd contains a MALFORMED omnigraph.yaml
boots and serves — proving mode-inference rule 0 returns before any config
search can run. New spawn_server_with_cluster_in support helper sets the
spawned server's cwd explicitly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cluster FACTS stay unlayered (cluster.yaml only), but the operator's
identity is a per-operator fact — exactly the per-operator omnigraph.yaml's
permanent job, and the cascade every data-plane write already uses. cluster
apply/approve now resolve: --as flag wins and skips any config read
entirely (containers and CI stay config-free); without it, the standard cwd
search supplies cli.actor, with a malformed config failing loudly and
actionably ('pass --as to skip this lookup') rather than silently dropping
attribution. approve's no-actor error now names both sources.
Tests pin the contract from both sides: cli.actor is the no-flag default
for apply (echoed actor) and approve (approved_by), the flag overrides it,
a malformed omnigraph.yaml in cwd breaks nothing except the no-flag actor
lookup, and a conflicting well-formed one leaks nothing into cluster
outputs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- The drift-heal verification now asserts `schema show` succeeded and
produced a schema before checking the rogue field's absence (a failed
command previously made the negative assertion vacuously pass).
- cluster_cli documents why it deliberately does not assert exit codes
(blocked applies exit non-zero by contract while emitting the structured
output callers assert on).
- The comprehensive lifecycle e2es honor OMNIGRAPH_SKIP_SYSTEM_E2E=1
(graceful skip-with-message, the S3-gate pattern) for constrained
sandboxes; requirements + suppression documented in testing.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two system tests composing the whole Phase 1-5 surface with real binaries:
- local_cluster_full_lifecycle_declare_serve_evolve_delete: declare two
graphs -> one apply creates and converges them -> the --cluster server
serves both stored queries -> schema+query evolve in one apply (migration
previewed in plan) -> restart serves the new shape -> out-of-band schema
drift observed by refresh and converged back by apply (rogue field
soft-dropped) -> approved graph delete -> restart serves the survivor and
404s the tombstoned graph -> final plan empty. Catches composition
regressions where each stage passes its own tests but the lifecycle
breaks (the composite_flow.rs principle at the control-plane level).
- local_cluster_serving_enforces_applied_policy_bindings: applied policy
bundles gate serving per their bindings over HTTP with bearer-resolved
actors — the cluster-bound bundle owns graph_list (admin 200, reader 403,
anonymous 401), the graph-bound bundle owns invoke_query (reader gets
rows; denied invocation is the documented anti-probing 404).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The standing caveat ('applied means recorded in the cluster catalog —
nothing more; the server still boots from omnigraph.yaml') retires: cluster
docs gain the 'Serving from the cluster' section (exclusivity, applied-
revision serving, fail-fast readiness, restart-to-pick-up, expose-all
bridge), server.md gains mode-inference rule 0 and the cluster-booted multi
mode, deployment.md the boot-source choice, and the CLI's apply note plus
the cli-reference cluster row (stale back to Stage 3A) now describe the full
convergence surface. RFC-005 flips to Landed with four implementation
deviations recorded.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Phase-5 contract end to end with real binaries: cluster import + apply
via the CLI, seed a row through the graph plane, boot omnigraph-server with
--cluster (no omnigraph.yaml anywhere), and the applied stored query serves
the row over HTTP through the multi-graph routes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Slice 5A of RFC-005: the state ledger becomes serving-sufficient for the
Phase-5 server boot. StateResource gains an optional applies_to (normalized
typed refs: cluster | graph.<id>), written by apply for every applied policy
create/update from the desired config's validated bindings.
The hole this closes: applies_to is not part of the policy file digest, so a
binding-only edit previously produced NO plan change at all (a 4C e2e even
asserted that — the gap, not a contract). Binding changes are now
first-class: a post-diff pass emits an Update with equal before/after
digests and a binding_change marker (visible in plan/apply JSON and human
output as [bindings]), classification/execution treat it as an ordinary
catalog-tier applied change (payload skips naturally — the blob is
unchanged), and convergence requires zero binding divergence, so stale
bindings can never report converged. Pre-5A ledger entries (no bindings
recorded) surface as the same backfill Update; one apply heals them, exactly
the remedy RFC-005's boot-error path names.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stage 4C execution half (RFC-004 §D5/§D6 + sweep rows 7/7b/8): an approved
graph.<id> delete — and its riding schema/query deletes — classifies Applied
and executes LAST in the run, sidecar-fenced: pre-op manifest pin (best
effort; partial roots still delete), approval_id carried in the sidecar,
recursive root removal (NotFound tolerated), subtree tombstoned out of the
ledger with a tombstone observation, the approval consumed in the same state
CAS (ledger summary) and its artifact file rewritten with consumed_at only
after the CAS lands — a failed run consumes nothing and the approval stays
valid for the retry.
Sweep rows: already-tombstoned intents retire (7); a completed delete with a
stale ledger rolls forward — tombstone + approval consumption + audit entry
(7b, idempotent); a still-present root retires the stale intent with a
graph_delete_incomplete warning and the still-approved delete re-executes in
the same run (8) — prefix removal is idempotent, so retry IS the repair.
The multi-graph mixed e2e gets its conclusion: blocked without approval,
cluster approve graph.engineering --as andrew, converge, tombstone visible
in status. Phase 4's disposition matrix is now fully executable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RFC-004 §D4, gate half: graph deletes (and their subtree) now classify
Blocked/approval_required instead of Deferred; the new cluster approve
command (requires the global --as actor) writes
__cluster/approvals/{ulid}.json bound to the desired config digest and the
change's before/after digests, so config or state drift invalidates the
artifact automatically (approval_stale warning, never authorizes). One gate
per subtree: compute_approvals lists only the graph-level delete, and
ApprovalRequirement gains a satisfied flag surfaced by plan. Consumption and
the delete executor land next — until then approved deletes stay blocked so
a gate-only build can never strip state without removing the root.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stage 4B (RFC-004 §D1/§D5): schema.<id> Update changes classify Applied and
execute after graph creates, sequentially and sidecar-fenced — read-write
open (the engine's own recovery runs first), pre-op manifest pin recorded,
apply_schema_as with allow_data_loss: false (soft drops only; hard drops wait
for 4C's approval artifacts), post-op pin rewritten into the sidecar, sidecar
retired only after the final state CAS. Queries gated on a same-plan schema
update unblock (the migration lands first in the same run); failures —
unsupported migrations, lock contention, user branches — surface as
schema_apply_failed with the engine's message, demote dependents via the
origin-aware demotion helper, and stop further graph-moving work.
Schema evolution is now fully cluster-driven (the defer -> manual schema
apply -> refresh loop is gone), and out-of-band schema drift is converged
back by apply as an ordinary soft migration (axiom 8: drift correction is
gated like any change; the recoverable tier needs no approval) — both pinned
by reworked e2es. The multi-graph mixed e2e's deferred row is now
delete-shaped, pre-staging the 4C surface.
Actor: cluster apply accepts the CLI's global --as via the new ApplyOptions /
apply_config_dir_with_options (apply_config_dir delegates unchanged); the
actor is echoed in ApplyOutput and recorded in sidecars and audit entries,
and threads to apply_schema_as so Cedar fires wherever a checker is
installed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RFC-004 §D7's data-aware preview: for every schema update, plan opens the
live graph read-only and embeds the engine's migration plan (supported flag
+ typed steps) in the change record; the human renderer prints the steps.
Preview failures (unreachable graph, planner error) degrade to the digest
diff with a schema_preview_unavailable warning — planning never blocks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mechanical conversion ahead of Stage 4B (plan will preview schema migrations
against live graphs): signature, CLI dispatch, and test callers. Zero
behavior change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stage 4A (RFC-004 §D1/§D5): graph.<id> Create — and its paired schema Create,
which the init carries — classify Applied and execute first in the run,
sequentially and sidecar-fenced: sidecar written before Omnigraph::init at
the derived root, rewritten with the post-init manifest pin, deleted only
after the final state CAS lands. Dependent queries and policies no longer
block on a graph create in the same plan — creates run first, so they apply
in the same run; a create failure demotes them to blocked
(dependency_not_applied) and stops further graph-moving work (loud partials),
with the sidecar left for the sweep to classify. Graphs with a kept recovery
sidecar (rows 5/6) classify Blocked/cluster_recovery_pending, and the sweep's
Drifted/Error statuses are never clobbered by a generic Blocked.
Schema source is re-read and digest-verified under the lock before the init
(the write_resource_payload TOCTOU posture). Plan previews the same
dispositions. e2e fallout updated: a fresh multi-graph config now converges
in one apply; a destroyed root is re-created as an EMPTY graph by the next
apply (declarative convergence — visible in plan, called out in docs); the
new cluster_e2e_declared_graph_created_by_apply pins the no-manual-init flow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mechanical conversion ahead of Stage 4A graph create (which calls the async
Omnigraph::init from inside apply): the fn signature, the CLI dispatch arm,
and every test caller (#[test] -> #[tokio::test]). Zero behavior change; all
60 lib tests and 3 failpoint tests green before and after.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
status warns read-only -> refresh persists drift and drops the digest ->
apply republishes the blob -> status clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four lifecycle compositions over the spawned binary that pin spec claims no
single-command test proves:
- Lost ledger: delete state.json -> re-import from the live graph -> re-apply
converges onto the same content-addressed blobs (axiom 5's reconstructable-
state resilience edge, end to end).
- Out-of-band schema apply (the Sarah/Bob violation): refresh marks
graph/schema Drifted with schema_mismatch, status and plan surface it, and
cluster apply refuses to silently correct it — state keeps the LIVE schema
digest (drift correction is gated, axiom 8).
- Destroyed graph root: refresh records graph_missing drift and drops
graph/schema digests while preserving query/policy; plan proposes deferred
creates only; apply moves nothing and the catalog stays intact.
- Two graphs (one live, one not yet created) + a graph-spanning policy + a
cluster-scoped policy: a single apply yields all four dispositions at once
(applied/derived/deferred/blocked, deterministically ordered), then the
second graph appears, refresh observes it, and apply converges.
Helpers: init_named_cluster_graph generalizes init_cluster_derived_graph;
write_multi_graph_cluster_fixture builds the two-graph config.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Two review findings (greptile, PR #165):
- ApplyOutput.resource_statuses on a failed state write now carries the
pre-apply on-disk snapshot instead of the in-memory mutations that were
never persisted, so automation reading the field independently of `ok`
cannot see phantom applied/blocked statuses. Regression test forces the
state write to fail via a read-only __cluster dir (unix-only, skips when
permissions are not enforced).
- Human-mode `cluster apply` prints the classified changes list on failure
too, so an operator debugging a partial apply without --json sees what was
attempted.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three composition tests over the spawned binary against a real derived graph:
- import -> plan (dispositions) -> apply -> status -> refresh -> plan-empty,
then a query edit round-trip. Pins that refresh and apply recompute the
graph composite digest identically — divergence would silently re-open
the plan forever and no single-command test would catch it.
- The Stage 3A operator workflow across the control/data-plane boundary:
cluster apply defers a schema change, omnigraph schema apply executes it,
cluster refresh observes it, the next cluster apply re-converges.
- Held lock refuses apply, force-unlock clears it, retried apply converges.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
