diff --git a/docs/dev/rfc-002-config-cli-architecture.md b/docs/dev/rfc-002-config-cli-architecture.md index 845ac61..115b42f 100644 --- a/docs/dev/rfc-002-config-cli-architecture.md +++ b/docs/dev/rfc-002-config-cli-architecture.md @@ -4,6 +4,20 @@ **Supersedes:** the original additive-only draft (2026-05-30). This revision **embraces breaking changes** to remove ambiguity and conflation rather than carrying every legacy shape forward. It is gated behind a config `version:` field and ships compat aliases for the highest-traffic legacy keys, but it does not pretend the end-state is purely additive. Incorporates an implementation-readiness review: endpoint-bound credentials, layer identity trust, route-unification specifics, restored `query.roots`, and right-sized auth scope. **Target release:** v0.8.x (phased — see Rollout) +**Implementation status — V1a "locator core" landed.** Shipped: the typed +`GraphLocator` (`storage:` XOR `server:`+`graph_id:`), a `servers:` map, version-gated +config strictness (no `version:` = lenient + deprecation-warned; `version: 1` = unknown +keys rejected at any depth, via `serde_ignored`), `omnigraph-server` rejecting remote +graph entries (embedded-only), and the `--graph` flag. **Divergences from this proposal as +built:** `--target` was **removed outright (no deprecated alias)** — a clean rename, not the +alias proposed below; and `uri:` is honored-but-deprecation-warned (not auto-rewritten to +`storage.uri`). **Crate-location corrections** (the "Reconciliation"/"Implementation" +sections below predate V0): the config schema now lives in the extracted `omnigraph-config` +crate, and `QueryRegistry` was extracted to `omnigraph-queries` (not "kept in +`omnigraph-server`"). Deferred: layered global-first config + merge/provenance + `config +view`, the `cli:`→`defaults:` / `server:`→`serve:` renames (V1-remainder), route unification ++ remote client (V2), and the auth model (V3). + ## Summary OmniGraph today reads one config file, `omnigraph.yaml`, from both the CLI (operating the embedded engine) and `omnigraph-server` (hosting graphs). The CLI **can** already reach a *single-graph* server — point a graph entry's URI at the endpoint and set `bearer_token_env` — but it **cannot address a specific graph on a multi-graph server**, has no named-server credential model, and does not work without a project file in the current directory. Those are the real gaps. diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 425fcee..e8122d7 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -39,6 +39,13 @@ The engine's `tests/` is the principal coverage surface; most graph-shaped behav | `recovery.rs` | Open-time recovery sweep — sidecar I/O, classifier dispatch (NoMovement / RolledPastExpected / UnexpectedAtP1 / UnexpectedMultistep / InvariantViolation), all-or-nothing decision, roll-forward via `ManifestBatchPublisher::publish`, roll-back via `Dataset::restore`, audit row in `_graph_commit_recoveries.lance`, `OpenMode::ReadOnly` skip path | | `composite_flow.rs` | Compositional/narrative end-to-end stories — multi-step flows that compose mechanics covered by other test files. Catches integration regressions where individual operations all pass their unit tests but their composition breaks (sequential merges, post-merge main writes, time-travel through merge DAG, reopen consistency over multi-merge histories). | +**Config & CLI (RFC-002 V1a)** — these live outside the engine `tests/` dir: the +version-gated config-strictness and `deprecation_warnings` tests are inline in +`omnigraph-config` (`src/lib.rs` test module); the `GraphLocator` discriminant + +graph-identity tests are inline in `omnigraph-cli` (`src/main.rs`) plus +`tests/cli.rs` / `tests/system_local.rs`; the server embedded-only rejection tests +are in `omnigraph-server/tests/server.rs` (`multi_graph_startup`). + ## Fixtures `crates/omnigraph/tests/fixtures/` holds the canonical schema (`.pg`), seed data (`.jsonl`), and queries (`.gq`) shared across tests. Reuse these before inventing new ones — the helpers harness already knows how to load them. diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 8263919..a5bb37c 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -2,7 +2,7 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` schema. For a quick-start guide, see [cli.md](cli.md). -17 top-level command families, 40+ subcommands. All commands accept either a positional `URI`, `--uri`, or a `--target ` resolved against `omnigraph.yaml`. +17 top-level command families, 40+ subcommands. All commands accept either a positional `URI`, `--uri`, or a `--graph ` resolved against `omnigraph.yaml`. ## Top-level commands @@ -20,7 +20,7 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` sc | `run list \| show \| publish \| abort` | transactional run ops | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | -| `queries validate \| list` | operate on the server-side stored-query registry (the `queries:` block). `validate` type-checks every stored query against the live schema offline (opens the selected graph; exits non-zero on any breakage), catching schema drift without restarting the server; `list` prints the selected registry's query names, MCP exposure, and typed params. For per-graph registries, pass `--target ` or set `cli.graph`; with no graph selection, `list` shows only top-level `queries:`. Distinct from `lint`, which validates a single `.gq` file | +| `queries validate \| list` | operate on the server-side stored-query registry (the `queries:` block). `validate` type-checks every stored query against the live schema offline (opens the selected graph; exits non-zero on any breakage), catching schema drift without restarting the server; `list` prints the selected registry's query names, MCP exposure, and typed params. For per-graph registries, pass `--graph ` or set `cli.graph`; with no graph selection, `list` shows only top-level `queries:`. Distinct from `lint`, which validates a single `.gq` file | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns; `--json` reports a `skipped` field) | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC | | `embed` | offline JSONL embedding pipeline | @@ -30,11 +30,21 @@ A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` sc ## `omnigraph.yaml` schema ```yaml +version: 1 # omit for the legacy schema (lenient, deprecation-warned); + # `1` = strict: unknown/typo'd keys are rejected at any depth project: { name } +servers: # named remote endpoints (referenced by graphs.<>.server) + : { endpoint: } graphs: : - uri: + # Embedded XOR remote. Embedded → `storage:`; remote → `server:` (+ optional `graph_id:`). + storage: # embedded; a bare string, or a block { uri, region, endpoint } + # server: # remote: a `servers:` entry (mutually exclusive with storage) + # graph_id: # the graph's id on that server (defaults to the entry key) + # uri: # DEPRECATED legacy spelling of storage/server; warns at load bearer_token_env: + branch: # optional default branch + snapshot: # optional read-pinned snapshot queries: # per-graph stored-query registry (server-role; multi-graph mode) : # key MUST equal the `query ` symbol inside the .gq file: # relative to this config's directory diff --git a/docs/user/cli.md b/docs/user/cli.md index b6f2c09..1433651 100644 --- a/docs/user/cli.md +++ b/docs/user/cli.md @@ -60,7 +60,7 @@ Read through the HTTP API: ```bash omnigraph query \ - --target http://127.0.0.1:8080 \ + --graph http://127.0.0.1:8080 \ --query ./queries.gq \ --name get_person \ --params '{"name":"Alice"}' diff --git a/docs/user/policy.md b/docs/user/policy.md index ec0d214..5c4d575 100644 --- a/docs/user/policy.md +++ b/docs/user/policy.md @@ -48,7 +48,7 @@ graphs: ``` **Config follows graph identity, not server mode.** A graph served by **name** -(`--target ` or `server.graph`) uses its own `graphs..policy.file`, +(`--graph ` or `server.graph`) uses its own `graphs..policy.file`, exactly as in multi-graph mode. Top-level `policy.file` applies only to an **anonymous** graph — one served by a bare `` with no `graphs:` entry. Serving a **named** graph (single- or multi-graph mode) while top-level diff --git a/docs/user/server.md b/docs/user/server.md index 67b5afe..a13fe9b 100644 --- a/docs/user/server.md +++ b/docs/user/server.md @@ -6,22 +6,27 @@ Axum 0.8 + tokio + utoipa-generated OpenAPI. **Two modes** (v0.6.0+): single-gra ### Single-graph mode (legacy) -`omnigraph-server ` or `omnigraph-server --target --config omnigraph.yaml`. Routes are flat — `/snapshot`, `/read`, `/branches`, etc. +`omnigraph-server ` or `omnigraph-server --graph --config omnigraph.yaml`. Routes are flat — `/snapshot`, `/read`, `/branches`, etc. -**Config follows graph identity.** A bare `` is an *anonymous* graph and uses the **top-level** `policy.file` / `queries:`. A graph chosen by **name** (`--target` / `server.graph`) uses its own `graphs..{policy.file, queries}` — the same block multi-graph mode uses. ⚠️ *Changed from v0.6.0, which always used top-level config in single mode: a named-graph config that puts `policy`/`queries` at top-level now **refuses boot** and points you at `graphs..…` (move the block there). Bare-`` single mode is unchanged.* +**Config follows graph identity.** A bare `` is an *anonymous* graph and uses the **top-level** `policy.file` / `queries:`. A graph chosen by **name** (`--graph` / `server.graph`) uses its own `graphs..{policy.file, queries}` — the same block multi-graph mode uses. ⚠️ *Changed from v0.6.0, which always used top-level config in single mode: a named-graph config that puts `policy`/`queries` at top-level now **refuses boot** and points you at `graphs..…` (move the block there). Bare-`` single mode is unchanged.* ### Multi-graph mode (v0.6.0+) -`omnigraph-server --config omnigraph.yaml` with a non-empty `graphs:` map and **no** single-mode selector (no `server.graph`, no ``, no `--target`). The server opens every configured graph in parallel at startup (bounded concurrency = 4, fail-fast on the first open error). Routes are nested under `/graphs/{graph_id}/...`. Bare flat paths return 404 in multi mode. +`omnigraph-server --config omnigraph.yaml` with a non-empty `graphs:` map and **no** single-mode selector (no `server.graph`, no ``, no `--graph`). The server opens every configured graph in parallel at startup (bounded concurrency = 4, fail-fast on the first open error). Routes are nested under `/graphs/{graph_id}/...`. Bare flat paths return 404 in multi mode. Mode inference (four-rule matrix): 1. CLI positional `` → single -2. CLI `--target ` → single +2. CLI `--graph ` → single 3. `server.graph` in config → single 4. `--config` + non-empty `graphs:` + no single-mode selector → **multi** 5. otherwise → error with migration hint +**Embedded graphs only.** In either mode, a graph entry that targets a remote +`server:` (or a remote `http(s)://` `uri:`) is rejected at startup — +omnigraph-server serves embedded (self-contained) graphs and does not proxy +another server. Use an embedded `storage:` graph, or point a client at the remote. + ### Stored-query validation at startup If a graph declares a `queries:` registry (see [cli-reference](cli-reference.md)), the server **loads and type-checks every stored query against that graph's live schema at startup** and **refuses to boot** if any query references a type or property the schema lacks — the same fail-loud posture as a malformed policy file, so schema drift surfaces at the deploy boundary rather than at invocation. Two MCP-exposed queries claiming the same tool name is likewise a boot error. Non-blocking advisories (e.g. an MCP-exposed query with a vector parameter an agent cannot supply) are logged. Validate offline before deploying with `omnigraph queries validate`. Discover the exposed queries as a typed tool catalog with `GET /queries`, and invoke one over HTTP with `POST /queries/{name}` (both below). diff --git a/omnigraph.example.yaml b/omnigraph.example.yaml index 8f3d893..f964573 100644 --- a/omnigraph.example.yaml +++ b/omnigraph.example.yaml @@ -1,8 +1,14 @@ +version: 1 + +servers: + prod: + endpoint: http://127.0.0.1:8080 + graphs: local: - uri: ./repo.omni + storage: ./repo.omni dev: - uri: http://127.0.0.1:8080 + server: prod bearer_token_env: OMNIGRAPH_BEARER_TOKEN cli: