diff --git a/AGENTS.md b/AGENTS.md index b4453be..a4ad21c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -33,8 +33,8 @@ OmniGraph is a typed property-graph engine built as a coordination layer over ma - **Multi-modal querying**: vector ANN (`nearest`), full-text (`search`/`fuzzy`/`match_text`/`bm25`), Reciprocal Rank Fusion (`rrf`), and graph traversal (`Expand`, anti-join `not { … }`) in one runtime. - **Branches and commits across the whole graph**: Git-style — every successful publish appends to a commit DAG; merges are three-way at the row level. - **Atomic per-query writes**: `mutate_as` and `load` accumulate insert/update batches into an in-memory `MutationStaging.pending` per touched table; one `stage_*` + `commit_staged` per table runs at end-of-query, then `ManifestBatchPublisher::publish` commits the manifest atomically with per-table `expected_table_versions` CAS. A mid-query failure leaves Lance HEAD untouched on staged tables — no drift, no run state machine, no staging branches. Deletes still inline-commit; D₂ at parse time prevents inserts/updates and deletes from coexisting in one query. -- **HTTP server**: Axum + utoipa OpenAPI, bearer auth (SHA-256 hashed, optional AWS Secrets Manager). Cedar policy enforcement is engine-wide — every `_as` writer calls `Omnigraph::enforce(action, scope, actor)`, so HTTP, CLI, and embedded SDK consumers all hit the same gate. **Two modes** (v0.6.0+): single-graph (legacy flat routes) and multi-graph (`/graphs/{graph_id}/...` cluster routes + read-only `GET /graphs` enumeration). Per-graph + server-level Cedar policies. Multi-graph mode boots from a cluster directory (`--cluster `, RFC-005) or the legacy `omnigraph.yaml` `graphs:` map. Runtime add/remove (`POST /graphs`, `DELETE /graphs/{id}`) is not exposed — operators run `cluster apply` (or edit the legacy file) and restart. -- **CLI** with two-surface config (RFC-008): the team-owned cluster directory (`cluster.yaml`) plus the per-operator `~/.omnigraph/config.yaml` (servers, credentials, actor, aliases). The legacy combined `omnigraph.yaml` still loads with per-key deprecation warnings — `config migrate` proposes the split, `OMNIGRAPH_NO_LEGACY_CONFIG=1` enforces strict mode. **Never extend `omnigraph.yaml`.** Multi-format output (json/jsonl/csv/kv/table). +- **HTTP server**: Axum + utoipa OpenAPI, bearer auth (SHA-256 hashed, optional AWS Secrets Manager). Cedar policy enforcement is engine-wide — every `_as` writer calls `Omnigraph::enforce(action, scope, actor)`, so HTTP, CLI, and embedded SDK consumers all hit the same gate. **Cluster-only boot** (RFC-011): the server always boots from a cluster directory (`--cluster `, RFC-005) and serves N graphs (N ≥ 1) under multi-graph routes (`/graphs/{graph_id}/...` + read-only `GET /graphs` enumeration); there are no single-graph flat routes and no positional-URI boot. Per-graph + server-level Cedar policies. Runtime add/remove (`POST /graphs`, `DELETE /graphs/{id}`) is not exposed — operators run `cluster apply` and restart. +- **CLI** with two-surface config (RFC-007/008): the team-owned cluster directory (`cluster.yaml`) plus the per-operator `~/.omnigraph/config.yaml` (servers, clusters, credentials, actor, profiles, aliases, defaults). Graphs are addressed via `--store`/`--server`/`--cluster`/`--profile`/operator defaults (RFC-011). Multi-format output (json/jsonl/csv/kv/table). Throughout the docs, capabilities are split into **L1 — Inherited from Lance** vs **L2 — Added by OmniGraph**. @@ -96,7 +96,7 @@ Full diagram and concurrency model: [docs/dev/architecture.md](docs/dev/architec | Cedar policy actions, scopes, CLI | [docs/user/operations/policy.md](docs/user/operations/policy.md) | | HTTP server endpoints, auth, error model, body limits | [docs/user/operations/server.md](docs/user/operations/server.md) | | CLI quick-start | [docs/user/cli/index.md](docs/user/cli/index.md) | -| CLI command surface and config schemas (`~/.omnigraph/config.yaml`, legacy `omnigraph.yaml`) | [docs/user/cli/reference.md](docs/user/cli/reference.md) | +| CLI command surface and config schema (`~/.omnigraph/config.yaml`) | [docs/user/cli/reference.md](docs/user/cli/reference.md) | | Audit / actor tracking | [docs/user/operations/audit.md](docs/user/operations/audit.md) | | Error taxonomy and result serialization | [docs/user/operations/errors.md](docs/user/operations/errors.md) | | Install (binary / Homebrew / source / channels) | [docs/user/install.md](docs/user/install.md) | @@ -265,8 +265,8 @@ omnigraph policy explain --actor act-alice --action change --branch main | Three-way row-level merge | — | `OrderedTableCursor` + `StagedTableWriter`, structured `MergeConflictKind` | | Change feeds | — | `diff_between` / `diff_commits` with manifest fast path + ID streaming | | Cedar policy | — | Per-graph actions plus server-scoped actions (see [docs/user/operations/policy.md](docs/user/operations/policy.md) for the current list), branch / target_branch / protected scopes, validate/test/explain CLI. **Engine-wide enforcement** (MR-722): every `_as` writer (`apply_schema_as`, `mutate_as`, `load_as` — the deprecated `ingest_as` shims route through it — `branch_create_as` / `branch_create_from_as`, `branch_delete_as`, `branch_merge_as`) calls `Omnigraph::enforce(action, scope, actor)` — HTTP, CLI, embedded SDK all hit the same gate. | -| HTTP server | — | Axum, OpenAPI via utoipa, bearer auth (SHA-256, AWS Secrets Manager option), `authorize_request` at the HTTP boundary (resolves bearer→actor, applies admission control), NDJSON streaming export, **multi-graph mode (v0.6.0+) with cluster routes + read-only `GET /graphs` enumeration + per-graph + server-level Cedar policies. Multi-graph boots from a cluster directory (`--cluster`) or the legacy `omnigraph.yaml`; add/remove graphs via `cluster apply` (or by editing the legacy file) and restarting.** | -| CLI with config | — | two-surface config (team `cluster.yaml` dir + per-operator `~/.omnigraph/config.yaml`; legacy `omnigraph.yaml` deprecated per RFC-008), aliases, multi-format output (json/jsonl/csv/kv/table) | +| HTTP server | — | Axum, OpenAPI via utoipa, bearer auth (SHA-256, AWS Secrets Manager option), `authorize_request` at the HTTP boundary (resolves bearer→actor, applies admission control), NDJSON streaming export, **cluster-only boot (RFC-011): always `--cluster `, serving N graphs (N ≥ 1) under multi-graph routes + read-only `GET /graphs` enumeration + per-graph + server-level Cedar policies. Add/remove graphs via `cluster apply` and restart.** | +| CLI with config | — | two-surface config (team `cluster.yaml` dir + per-operator `~/.omnigraph/config.yaml`), scope addressing (`--store`/`--server`/`--cluster`/`--profile`/defaults, RFC-011), aliases, multi-format output (json/jsonl/csv/kv/table) | | Audit / actor tracking | — | `_as` write APIs + actor map in commit graph | | Local RustFS bootstrap | — | `scripts/local-rustfs-bootstrap.sh` one-shot S3-backed dev environment | diff --git a/crates/omnigraph-cli/src/cli.rs b/crates/omnigraph-cli/src/cli.rs index ae33d14..81b330b 100644 --- a/crates/omnigraph-cli/src/cli.rs +++ b/crates/omnigraph-cli/src/cli.rs @@ -352,12 +352,6 @@ pub(crate) enum Command { #[arg(long)] json: bool, }, - /// Legacy-config tooling (RFC-008): split omnigraph.yaml into its - /// two destinations. - Config { - #[command(subcommand)] - command: ConfigCommand, - }, /// Print the CLI version Version, } @@ -661,20 +655,3 @@ impl CliLoadMode { } } -#[derive(Debug, Subcommand)] -pub(crate) enum ConfigCommand { - /// Propose (and with --write, apply) the RFC-008 split of a legacy - /// omnigraph.yaml: team half -> a ready-to-review cluster.yaml, - /// personal half -> ~/.omnigraph/config.yaml (key-level merge, - /// existing entries always win). Touches nothing without --write. - Migrate { - /// Path to the legacy omnigraph.yaml (default: ./omnigraph.yaml) - #[arg(long)] - config: Option, - /// Apply the split instead of only printing it - #[arg(long)] - write: bool, - #[arg(long)] - json: bool, - }, -} diff --git a/crates/omnigraph-cli/src/helpers.rs b/crates/omnigraph-cli/src/helpers.rs index 5d06b2a..ac4f5c2 100644 --- a/crates/omnigraph-cli/src/helpers.rs +++ b/crates/omnigraph-cli/src/helpers.rs @@ -119,6 +119,16 @@ pub(crate) fn bearer_token_from_env(var_name: &str) -> Option { normalize_bearer_token(std::env::var(var_name).ok()) } +/// The Cedar resource id for a graph selection: the explicit graph name when one +/// is given, else the normalized URI (the anonymous fallback). Used by the +/// `policy` tooling to address a graph's bundle. +pub(crate) fn graph_resource_id_for_selection( + selected_graph: Option<&str>, + normalized_uri: &str, +) -> String { + selected_graph.unwrap_or(normalized_uri).to_string() +} + #[derive(Debug, Clone)] pub(crate) struct ResolvedCliGraph { pub(crate) uri: String, @@ -991,6 +1001,18 @@ pub(crate) fn rewrite_deprecated_argv(args: Vec) -> Vec { mod tests { use super::*; + #[test] + fn graph_resource_id_for_selection_uses_name_or_anonymous_uri() { + assert_eq!( + graph_resource_id_for_selection(Some("local"), "/tmp/graph.omni"), + "local" + ); + assert_eq!( + graph_resource_id_for_selection(None, "/tmp/graph.omni"), + "/tmp/graph.omni" + ); + } + // RFC-011 Decision 9: locality classifier for the destructive-confirm gate. #[test] fn uri_is_local_truth_table() { diff --git a/crates/omnigraph-cli/src/main.rs b/crates/omnigraph-cli/src/main.rs index 1628816..d9c5720 100644 --- a/crates/omnigraph-cli/src/main.rs +++ b/crates/omnigraph-cli/src/main.rs @@ -24,8 +24,7 @@ use omnigraph_api_types::{ }; use omnigraph_server::queries::{QueryRegistry, check}; use omnigraph_server::{ - PolicyAction, PolicyDecision, PolicyEngine, PolicyRequest, - PolicyTestConfig, ReadOutputFormat, graph_resource_id_for_selection, load_config, + PolicyAction, PolicyDecision, PolicyEngine, PolicyRequest, PolicyTestConfig, }; use reqwest::Method; use reqwest::header::AUTHORIZATION; @@ -34,12 +33,11 @@ use serde::de::DeserializeOwned; use serde_json::Value; mod embed; -mod migrate; mod operator; mod read_format; use embed::{EmbedArgs, EmbedOutput, execute_embed}; -use read_format::{ReadRenderOptions, render_read}; +use read_format::{ReadOutputFormat, ReadRenderOptions, render_read}; mod cli; mod client; @@ -73,42 +71,6 @@ async fn main() -> Result<()> { // before any per-command dispatch. planes::guard_addressing(&cli)?; match cli.command { - Command::Config { command } => match command { - ConfigCommand::Migrate { config, write, json } => { - let path = migrate::legacy_config_path(config.as_ref()); - if !path.exists() { - bail!( - "no legacy config at '{}' — nothing to migrate", - path.display() - ); - } - let legacy = load_config(Some(&path))?; - let report = migrate::build_report(&legacy, &path); - if write { - let legacy_dir = path - .parent() - .filter(|parent| !parent.as_os_str().is_empty()) - .unwrap_or(std::path::Path::new(".")) - .to_path_buf(); - let written = migrate::apply_report(&report, &legacy_dir)?; - if json { - print_json(&serde_json::json!({ - "report": report, - "written": written, - }))?; - } else { - print!("{}", migrate::render_report(&report)); - for line in written { - println!("wrote: {line}"); - } - } - } else if json { - print_json(&report)?; - } else { - print!("{}", migrate::render_report(&report)); - } - } - }, Command::Login { name, token, json } => { let token = match token { Some(token) => token, diff --git a/crates/omnigraph-cli/src/migrate.rs b/crates/omnigraph-cli/src/migrate.rs deleted file mode 100644 index 7410381..0000000 --- a/crates/omnigraph-cli/src/migrate.rs +++ /dev/null @@ -1,408 +0,0 @@ -//! `omnigraph config migrate` (RFC-008 stage 2): split a legacy -//! `omnigraph.yaml` into its two destinations — the team half as a -//! ready-to-review `cluster.yaml` proposal, the personal half merged into -//! `~/.omnigraph/config.yaml` — and name what's obsolete. The command is -//! the completeness test of RFC-008's migration map: any key it cannot -//! place is a bug in the RFC. -//! -//! Touches nothing without `--write`. Referenced `.gq`/policy files are -//! never moved; manual steps are printed instead. - -use std::collections::BTreeMap; -use std::path::{Path, PathBuf}; - -use color_eyre::Result; -use color_eyre::eyre::eyre; -use omnigraph_server::OmnigraphConfig; -use serde::Serialize; - -use crate::operator; - -#[derive(Debug, Serialize)] -pub(crate) struct MigrateReport { - pub(crate) source: String, - /// The ready-to-review cluster.yaml text (None when the legacy file - /// declares nothing team-shaped). - pub(crate) cluster_yaml: Option, - /// Operator keys to merge: dotted key -> YAML value text. - pub(crate) operator_merge: BTreeMap, - /// Keys with no destination, and why. - pub(crate) dropped: Vec, - /// Steps the command will not do for you. - pub(crate) manual_steps: Vec, -} - -#[derive(Debug, Serialize)] -pub(crate) struct DroppedKey { - pub(crate) key: String, - pub(crate) reason: String, -} - -/// Classify a parsed legacy config into the report. Pure — no I/O. -pub(crate) fn build_report(config: &OmnigraphConfig, source: &Path) -> MigrateReport { - let mut dropped = Vec::new(); - let mut manual_steps = Vec::new(); - let mut operator_merge: BTreeMap = BTreeMap::new(); - - // ---- personal half ---- - if let Some(actor) = &config.cli.actor { - operator_merge.insert("operator.actor".into(), actor.clone()); - } - if let Some(format) = config.cli.output_format { - operator_merge.insert( - "defaults.output".into(), - serde_yaml::to_string(&format).unwrap_or_default().trim().to_string(), - ); - } - if let Some(width) = config.cli.table_max_column_width { - operator_merge.insert("defaults.table_max_column_width".into(), width.to_string()); - } - if let Some(layout) = config.cli.table_cell_layout { - operator_merge.insert( - "defaults.table_cell_layout".into(), - serde_yaml::to_string(&layout).unwrap_or_default().trim().to_string(), - ); - } - if config.cli.graph.is_some() { - dropped.push(DroppedKey { - key: "cli.graph".into(), - reason: "address graphs explicitly via --store/--server, or set defaults.default_graph in the operator config".into(), - }); - } - if config.cli.branch.is_some() { - dropped.push(DroppedKey { - key: "cli.branch".into(), - reason: "pass --branch explicitly".into(), - }); - } - - // Remote graphs with a token env become operator servers (the keyed - // chain replaces invented env-var names). - for (name, target) in &config.graphs { - if target.uri.starts_with("http://") || target.uri.starts_with("https://") { - operator_merge.insert(format!("servers.{name}.url"), target.uri.clone()); - if target.bearer_token_env.is_some() { - manual_steps.push(format!( - "store the '{name}' token in the keyed chain: echo $TOKEN | omnigraph login {name} (replaces bearer_token_env)" - )); - } - } - } - if config.auth.env_file.is_some() { - manual_steps.push( - "auth.env_file keeps working during the window; prefer `omnigraph login ` per server going forward".into(), - ); - } - - // Legacy aliases split: content -> catalog stored query, binding -> - // operator alias referencing the name. - for (name, alias) in &config.aliases { - let query_name = alias.name.clone().unwrap_or_else(|| name.clone()); - operator_merge.insert( - format!("aliases.{name}"), - format!( - "{{ server: TODO-server-name, graph: {}, query: {query_name}, args: [{}] }}", - alias.graph.as_deref().unwrap_or("TODO-graph-id"), - alias.args.join(", ") - ), - ); - manual_steps.push(format!( - "alias '{name}': move its query content ('{}') into the cluster checkout's queries/ so '{query_name}' becomes a catalog stored query", - alias.query - )); - } - - // ---- team half ---- - let has_team_content = !config.graphs.is_empty() - || !config.queries.is_empty() - || config.policy.file.is_some() - || config.server.policy.file.is_some(); - let cluster_yaml = has_team_content.then(|| { - let mut out = String::from("version: 1\n"); - if let Some(name) = &config.project.name { - out.push_str(&format!("metadata:\n name: {name}\n")); - } - out.push_str("# storage: s3://bucket/prefix # or omit: this folder is the root\n"); - if !config.graphs.is_empty() || !config.queries.is_empty() { - out.push_str("graphs:\n"); - } - // Single-graph top-level queries belong to a graph the legacy file - // never named; propose one. - if !config.queries.is_empty() && config.graphs.is_empty() { - out.push_str(" default: # TODO: pick the graph id\n schema: # TODO: path to this graph's .pg schema\n queries: queries/\n"); - } - for (name, target) in &config.graphs { - out.push_str(&format!(" {name}:\n")); - out.push_str(" schema: # TODO: path to this graph's .pg schema\n"); - if !target.queries.is_empty() { - out.push_str(" queries: queries/ # move the .gq files here\n"); - } - out.push_str(&format!( - " # legacy root: {} — the cluster manages graph roots under its storage; run `omnigraph cluster import` after reviewing\n", - target.uri - )); - } - let mut policies: Vec<(String, String, String)> = Vec::new(); - if let Some(file) = &config.policy.file { - policies.push(("default".into(), file.clone(), "graph. # TODO: bind".into())); - } - if let Some(file) = &config.server.policy.file { - policies.push(("server".into(), file.clone(), "cluster".into())); - } - for (name, target) in &config.graphs { - if let Some(file) = &target.policy.file { - policies.push((name.clone(), file.clone(), format!("graph.{name}"))); - } - } - if !policies.is_empty() { - out.push_str("policies:\n"); - for (name, file, binding) in policies { - out.push_str(&format!( - " {name}:\n file: {file}\n applies_to: [{binding}]\n" - )); - } - } - out - }); - - if !config.query.roots.is_empty() { - dropped.push(DroppedKey { - key: "query.roots".into(), - reason: "obsolete — cluster query discovery (queries: ) replaced it".into(), - }); - } - if config.server.bind.is_some() || config.server.graph.is_some() { - dropped.push(DroppedKey { - key: "server.bind / server.graph".into(), - reason: "deployment runtime — pass --bind / target flags or env".into(), - }); - } - if config.project.name.is_some() && cluster_yaml.is_none() { - dropped.push(DroppedKey { - key: "project.name".into(), - reason: "the cluster's metadata.name is the deployment label".into(), - }); - } - - MigrateReport { - source: source.display().to_string(), - cluster_yaml, - operator_merge, - dropped, - manual_steps, - } -} - -pub(crate) fn render_report(report: &MigrateReport) -> String { - let mut out = format!("migration plan for {}\n", report.source); - if let Some(cluster) = &report.cluster_yaml { - out.push_str("\n== team half -> cluster.yaml (ready to review) ==\n"); - out.push_str(cluster); - } - if !report.operator_merge.is_empty() { - out.push_str("\n== personal half -> ~/.omnigraph/config.yaml ==\n"); - for (key, value) in &report.operator_merge { - out.push_str(&format!(" {key}: {value}\n")); - } - } - if !report.dropped.is_empty() { - out.push_str("\n== no destination ==\n"); - for dropped in &report.dropped { - out.push_str(&format!(" {} — {}\n", dropped.key, dropped.reason)); - } - } - if !report.manual_steps.is_empty() { - out.push_str("\n== manual steps ==\n"); - for step in &report.manual_steps { - out.push_str(&format!(" - {step}\n")); - } - } - out.push_str("\n(nothing written; pass --write to apply the operator merge and emit cluster.yaml)\n"); - out -} - -/// `--write`: merge the personal half into the operator config (key-level, -/// existing entries always win; the prior file is backed up) and write the -/// team half to cluster.yaml in the legacy config's directory (or -/// cluster.yaml.proposed when one already exists). -pub(crate) fn apply_report(report: &MigrateReport, legacy_dir: &Path) -> Result> { - let mut written = Vec::new(); - - if !report.operator_merge.is_empty() { - let dir = operator::operator_dir() - .ok_or_else(|| eyre!("no home directory resolvable for the operator config"))?; - std::fs::create_dir_all(&dir)?; - let path = dir.join(operator::OPERATOR_CONFIG_FILE); - let existing_text = std::fs::read_to_string(&path).unwrap_or_default(); - let mut mapping: serde_yaml::Mapping = if existing_text.trim().is_empty() { - serde_yaml::Mapping::new() - } else { - serde_yaml::from_str(&existing_text) - .map_err(|err| eyre!("operator config '{}' does not parse: {err}", path.display()))? - }; - let mut merged_any = false; - for (dotted, value_text) in &report.operator_merge { - if merge_dotted_if_absent(&mut mapping, dotted, value_text)? { - merged_any = true; - } - } - if merged_any { - if !existing_text.is_empty() { - let backup = path.with_extension("yaml.bak"); - std::fs::write(&backup, &existing_text)?; - written.push(format!("backed up prior operator config to {}", backup.display())); - } - let rendered = serde_yaml::to_string(&mapping)?; - let tmp = path.with_extension(format!("yaml.tmp.{}", std::process::id())); - std::fs::write(&tmp, &rendered)?; - std::fs::rename(&tmp, &path)?; - written.push(format!("merged personal keys into {}", path.display())); - } else { - written.push("operator config already carries every personal key (nothing merged)".into()); - } - } - - if let Some(cluster) = &report.cluster_yaml { - let target = legacy_dir.join("cluster.yaml"); - let target = if target.exists() { - legacy_dir.join("cluster.yaml.proposed") - } else { - target - }; - std::fs::write(&target, cluster)?; - written.push(format!("wrote team-half proposal to {}", target.display())); - } - - Ok(written) -} - -/// Set `a.b.c` in the mapping only when absent; returns whether it wrote. -fn merge_dotted_if_absent( - mapping: &mut serde_yaml::Mapping, - dotted: &str, - value_text: &str, -) -> Result { - let value: serde_yaml::Value = - serde_yaml::from_str(value_text).unwrap_or(serde_yaml::Value::String(value_text.into())); - let parts: Vec<&str> = dotted.split('.').collect(); - let mut current = mapping; - for part in &parts[..parts.len() - 1] { - let key = serde_yaml::Value::String((*part).into()); - let entry = current - .entry(key) - .or_insert_with(|| serde_yaml::Value::Mapping(serde_yaml::Mapping::new())); - current = entry - .as_mapping_mut() - .ok_or_else(|| eyre!("operator config key '{dotted}' collides with a non-mapping"))?; - } - let leaf = serde_yaml::Value::String(parts[parts.len() - 1].into()); - if current.contains_key(&leaf) { - return Ok(false); - } - current.insert(leaf, value); - Ok(true) -} - -pub(crate) fn legacy_config_path(explicit: Option<&PathBuf>) -> PathBuf { - explicit.cloned().unwrap_or_else(|| PathBuf::from("omnigraph.yaml")) -} - -#[cfg(test)] -mod tests { - use super::*; - use omnigraph_server::config::load_config; - - fn full_legacy_fixture(dir: &Path) -> PathBuf { - let path = dir.join("omnigraph.yaml"); - std::fs::write( - &path, - r#" -project: { name: brain } -graphs: - prod: - uri: https://graph.example.com - bearer_token_env: PROD_TOKEN - policy: { file: ./prod.policy.yaml } - queries: - find: { file: ./find.gq } - local: - uri: /tmp/local.omni -server: { bind: "0.0.0.0:9999", policy: { file: ./server.policy.yaml } } -auth: { env_file: .env.omni } -cli: - graph: prod - branch: main - actor: act-me - output_format: json - table_max_column_width: 40 -query: { roots: ["."] } -aliases: - triage: { command: query, query: ./triage.gq, name: weekly_triage, args: [since], graph: prod } -policy: { file: ./top.policy.yaml } -queries: - top_q: { file: ./top.gq } -"#, - ) - .unwrap(); - path - } - - /// The RFC-008 completeness contract: every top-level key of the - /// legacy schema must appear in the report somewhere (team half, - /// operator merge, dropped, or manual steps). - #[test] - fn every_legacy_key_is_classified() { - let dir = tempfile::tempdir().unwrap(); - let path = full_legacy_fixture(dir.path()); - let config = load_config(Some(&path)).unwrap(); - let report = build_report(&config, &path); - let rendered = render_report(&report); - - let serialized = - serde_yaml::to_value(OmnigraphConfig::default()).expect("default serializes"); - for key in serialized.as_mapping().unwrap().keys() { - let key = key.as_str().unwrap(); - assert!( - rendered.contains(key) - || report.operator_merge.keys().any(|k| k.contains(key)) - || matches!(key, "graphs" | "queries" | "policy" | "project") - && report.cluster_yaml.is_some(), - "legacy key '{key}' is unclassified — fix the RFC-008 map: {rendered}" - ); - } - - // spot checks on each section - assert_eq!(report.operator_merge["operator.actor"], "act-me"); - assert_eq!(report.operator_merge["defaults.output"], "json"); - assert_eq!( - report.operator_merge["servers.prod.url"], - "https://graph.example.com" - ); - assert!(report.operator_merge["aliases.triage"].contains("query: weekly_triage")); - let cluster = report.cluster_yaml.as_deref().unwrap(); - assert!(cluster.contains("version: 1")); - assert!(cluster.contains("name: brain")); - assert!(cluster.contains(" prod:")); - assert!(cluster.contains("applies_to: [cluster]")); - assert!(cluster.contains("applies_to: [graph.prod]")); - assert!(report.dropped.iter().any(|d| d.key == "query.roots")); - assert!(report.dropped.iter().any(|d| d.key.contains("server.bind"))); - assert!( - report - .manual_steps - .iter() - .any(|s| s.contains("omnigraph login prod")) - ); - } - - #[test] - fn merge_dotted_never_clobbers_existing() { - let mut mapping: serde_yaml::Mapping = - serde_yaml::from_str("operator:\n actor: keep-me\n").unwrap(); - assert!(!merge_dotted_if_absent(&mut mapping, "operator.actor", "new").unwrap()); - assert!(merge_dotted_if_absent(&mut mapping, "defaults.output", "json").unwrap()); - let text = serde_yaml::to_string(&mapping).unwrap(); - assert!(text.contains("keep-me") && !text.contains("new")); - assert!(text.contains("output: json")); - } -} diff --git a/crates/omnigraph-cli/src/operator.rs b/crates/omnigraph-cli/src/operator.rs index 929779e..96b4dc1 100644 --- a/crates/omnigraph-cli/src/operator.rs +++ b/crates/omnigraph-cli/src/operator.rs @@ -21,7 +21,7 @@ use color_eyre::Result; use color_eyre::eyre::{bail, eyre}; use serde::Deserialize; -use omnigraph_server::config::ReadOutputFormat; +use crate::read_format::{ReadOutputFormat, TableCellLayout}; pub(crate) const OPERATOR_HOME_ENV: &str = "OMNIGRAPH_HOME"; pub(crate) const OPERATOR_DIR: &str = ".omnigraph"; @@ -102,10 +102,9 @@ pub(crate) struct OperatorIdentity { pub(crate) struct OperatorDefaults { /// Default read output format, below every more-specific source. pub(crate) output: Option, - /// Table rendering preferences (below the legacy cli.table_* keys - /// during the RFC-008 window). + /// Table rendering preferences for `--format table`. pub(crate) table_max_column_width: Option, - pub(crate) table_cell_layout: Option, + pub(crate) table_cell_layout: Option, /// Default server scope (RFC-011): the everyday addressing when no /// `--profile` / primitive / legacy address is given. Names an entry /// under `servers:`. Mutually exclusive with `store` — a scope binds one diff --git a/crates/omnigraph-cli/src/planes.rs b/crates/omnigraph-cli/src/planes.rs index 45f96e2..70b8dc5 100644 --- a/crates/omnigraph-cli/src/planes.rs +++ b/crates/omnigraph-cli/src/planes.rs @@ -132,7 +132,6 @@ pub(crate) fn command_plane(cmd: &Command) -> Plane { Command::Embed(_) | Command::Login { .. } | Command::Logout { .. } - | Command::Config { .. } | Command::Version => Plane::Session, } } @@ -144,7 +143,6 @@ pub(crate) fn command_label(cmd: &Command) -> &'static str { Command::Version => "version", Command::Login { .. } => "login", Command::Logout { .. } => "logout", - Command::Config { .. } => "config", Command::Embed(_) => "embed", Command::Init { .. } => "init", Command::Load { .. } => "load", diff --git a/crates/omnigraph-cli/src/read_format.rs b/crates/omnigraph-cli/src/read_format.rs index b205b19..3ffa9e6 100644 --- a/crates/omnigraph-cli/src/read_format.rs +++ b/crates/omnigraph-cli/src/read_format.rs @@ -1,9 +1,31 @@ +use clap::ValueEnum; use color_eyre::eyre::Result; -use omnigraph_server::ReadOutputFormat; use omnigraph_server::api::ReadOutput; -use omnigraph_server::config::TableCellLayout; +use serde::{Deserialize, Serialize}; use serde_json::{Map, Value}; +/// Output rendering format for read-shaped commands (`read`/`query`/`alias`). +/// A CLI presentation concern — lives here, not in the server. +#[derive(Debug, Clone, Copy, Default, Eq, PartialEq, Serialize, Deserialize, ValueEnum)] +#[serde(rename_all = "snake_case")] +pub enum ReadOutputFormat { + #[default] + Table, + Kv, + Csv, + Jsonl, + Json, +} + +/// How an over-wide table cell is laid out when rendering `--format table`. +#[derive(Debug, Clone, Copy, Default, Eq, PartialEq, Serialize, Deserialize, ValueEnum)] +#[serde(rename_all = "snake_case")] +pub enum TableCellLayout { + #[default] + Truncate, + Wrap, +} + pub struct ReadRenderOptions { pub max_column_width: usize, pub cell_layout: TableCellLayout, diff --git a/crates/omnigraph-cli/tests/cli_schema_config.rs b/crates/omnigraph-cli/tests/cli_schema_config.rs index 8a9ee47..15c6c46 100644 --- a/crates/omnigraph-cli/tests/cli_schema_config.rs +++ b/crates/omnigraph-cli/tests/cli_schema_config.rs @@ -562,74 +562,3 @@ fn graphs_list_against_local_uri_errors_with_remote_only_message() { ); } -/// RFC-008 stage 2: `config migrate` proposes the split read-only, applies -/// it with --write (operator merge never clobbers; cluster.yaml emitted), -/// and a second --write is idempotent. -#[test] -fn config_migrate_splits_legacy_config() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - "graphs:\n prod:\n uri: https://graph.example.com\n bearer_token_env: PROD_TOKEN\ncli:\n actor: act-me\n output_format: json\npolicy:\n file: ./top.policy.yaml\n", - ) - .unwrap(); - let operator_home = tempfile::tempdir().unwrap(); - fs::write( - operator_home.path().join("config.yaml"), - "operator:\n actor: act-existing\n", - ) - .unwrap(); - - // Read-only proposal: names both halves, writes nothing. - let output = cli() - .current_dir(temp.path()) - .env("OMNIGRAPH_HOME", operator_home.path()) - .env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1") - .arg("config") - .arg("migrate") - .output() - .unwrap(); - assert!(output.status.success(), "{output:?}"); - let stdout = String::from_utf8_lossy(&output.stdout); - assert!(stdout.contains("team half -> cluster.yaml"), "{stdout}"); - assert!(stdout.contains("operator.actor: act-me"), "{stdout}"); - assert!(stdout.contains("omnigraph login prod"), "{stdout}"); - assert!(!temp.path().join("cluster.yaml").exists()); - - // --write: cluster.yaml lands; the existing operator actor is KEPT. - let output = cli() - .current_dir(temp.path()) - .env("OMNIGRAPH_HOME", operator_home.path()) - .env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1") - .arg("config") - .arg("migrate") - .arg("--write") - .output() - .unwrap(); - assert!(output.status.success(), "{output:?}"); - let cluster = fs::read_to_string(temp.path().join("cluster.yaml")).unwrap(); - assert!(cluster.contains("version: 1") && cluster.contains(" prod:"), "{cluster}"); - let operator_text = - fs::read_to_string(operator_home.path().join("config.yaml")).unwrap(); - assert!(operator_text.contains("act-existing"), "{operator_text}"); - assert!(!operator_text.contains("act-me"), "existing keys win: {operator_text}"); - assert!(operator_text.contains("output: json"), "{operator_text}"); - assert!( - operator_text.contains("url: https://graph.example.com"), - "{operator_text}" - ); - - // Second --write: cluster.yaml exists -> proposal file, no clobber. - let output = cli() - .current_dir(temp.path()) - .env("OMNIGRAPH_HOME", operator_home.path()) - .env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1") - .arg("config") - .arg("migrate") - .arg("--write") - .output() - .unwrap(); - assert!(output.status.success(), "{output:?}"); - assert!(temp.path().join("cluster.yaml.proposed").exists()); -} - diff --git a/crates/omnigraph-server/src/config.rs b/crates/omnigraph-server/src/config.rs deleted file mode 100644 index 15b957d..0000000 --- a/crates/omnigraph-server/src/config.rs +++ /dev/null @@ -1,1103 +0,0 @@ -use std::collections::BTreeMap; -use std::env; -use std::fs; -use std::path::{Path, PathBuf}; - -use clap::ValueEnum; -use color_eyre::eyre::{Result, bail}; -use serde::{Deserialize, Serialize}; - -pub const DEFAULT_CONFIG_FILE: &str = "omnigraph.yaml"; - -pub fn graph_resource_id_for_selection( - selected_graph: Option<&str>, - normalized_uri: &str, -) -> String { - selected_graph.unwrap_or(normalized_uri).to_string() -} - -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct ProjectConfig { - pub name: Option, -} - -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct TargetConfig { - pub uri: String, - pub bearer_token_env: Option, - /// Per-graph Cedar policy file (MR-668). In single-graph mode this - /// field is unused — the top-level `policy.file` applies. In - /// multi-graph mode, each `graphs..policy.file` governs that - /// graph's HTTP-layer Cedar enforcement. - #[serde(default)] - pub policy: PolicySettings, - /// Per-graph stored-query registry: an inline `name -> entry` - /// map. Mirrors the per-graph `policy` shape — each - /// `graphs..queries` declares that graph's stored queries. Absent - /// (or empty) = no stored queries for the graph. v1 is inline-only; - /// an external `queries.yaml` manifest indirection is a deferred - /// convenience. - #[serde(default)] - pub queries: BTreeMap, -} - -#[derive(Debug, Clone, Copy, Default, Eq, PartialEq, Serialize, Deserialize, ValueEnum)] -#[serde(rename_all = "snake_case")] -pub enum ReadOutputFormat { - #[default] - Table, - Kv, - Csv, - Jsonl, - Json, -} - -#[derive(Debug, Clone, Copy, Default, Eq, PartialEq, Serialize, Deserialize, ValueEnum)] -#[serde(rename_all = "snake_case")] -pub enum TableCellLayout { - #[default] - Truncate, - Wrap, -} - -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct CliDefaults { - #[serde(rename = "graph")] - pub graph: Option, - pub branch: Option, - pub output_format: Option, - pub table_max_column_width: Option, - pub table_cell_layout: Option, - /// Default actor identity for CLI direct-engine writes (MR-722). - /// Used when `policy.file` is configured and the operator hasn't - /// passed `--as ` on the command line. With policy configured - /// and neither this nor `--as` set, the engine-layer footgun guard - /// fires (no silent bypass). - pub actor: Option, -} - -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct ServerDefaults { - #[serde(rename = "graph")] - pub graph: Option, - pub bind: Option, - /// Server-level Cedar policy (MR-668). Governs management endpoints - /// — currently `GET /graphs`; future runtime add/remove endpoints - /// will plug in here too. In single-graph mode this is unused — the - /// top-level `policy.file` covers the single graph. - #[serde(default)] - pub policy: PolicySettings, -} - -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct AuthDefaults { - pub env_file: Option, -} - -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct QueryDefaults { - #[serde(default)] - pub roots: Vec, -} - -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -pub struct PolicySettings { - pub file: Option, -} - -/// One stored-query registry entry. The map **key** is the query's -/// identity — it must equal the `query ` symbol declared inside -/// the referenced `.gq` file (asserted when the registry loads). -/// Renaming the key (or the symbol) is a breaking change to callers, by -/// design. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct QueryEntry { - /// Path to the `.gq` file (relative to the config's `base_dir`). The - /// file may declare several queries; the registry selects the one - /// whose symbol matches the map key. - pub file: String, - #[serde(default)] - pub mcp: McpSettings, -} - -/// MCP exposure for a stored query. A *deployment* concern (the same -/// `.gq` may be exposed in one graph and hidden in another), so it lives -/// in YAML rather than in the `.gq` source. **Default `expose: true`** — -/// declaring a query in the manifest *is* the opt-in, so it appears in the -/// MCP tool catalog (`GET /queries`) by default; set `expose: false` to -/// keep a query HTTP/service-callable but hidden from the agent tool list. -/// `expose` governs catalog membership only — it is **not** an -/// authorization gate (invocation is gated by `invoke_query`), so a hidden -/// query is still invocable by name with the right permission. -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct McpSettings { - #[serde(default = "mcp_expose_default")] - pub expose: bool, - pub tool_name: Option, -} - -fn mcp_expose_default() -> bool { - true -} - -impl Default for McpSettings { - fn default() -> Self { - Self { - expose: mcp_expose_default(), - tool_name: None, - } - } -} - -#[derive(Debug, Clone, Copy, Eq, PartialEq, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -pub enum AliasCommand { - /// Read alias (canonical: `query`). The legacy spelling `read` is - /// kept as the variant name for back-compat with serialized configs - /// and external SDK callers; `query` is accepted on the wire via the - /// serde alias. - #[serde(alias = "query")] - Read, - /// Mutation alias (canonical: `mutate`). The legacy spelling `change` - /// is kept as the variant name for back-compat; `mutate` is accepted - /// on the wire via the serde alias. - #[serde(alias = "mutate")] - Change, -} - -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct AliasConfig { - pub command: AliasCommand, - pub query: String, - pub name: Option, - #[serde(default)] - pub args: Vec, - #[serde(rename = "graph")] - pub graph: Option, - pub branch: Option, - pub format: Option, -} - -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct OmnigraphConfig { - #[serde(default)] - pub project: ProjectConfig, - #[serde(default, rename = "graphs")] - pub graphs: BTreeMap, - #[serde(default)] - pub server: ServerDefaults, - #[serde(default)] - pub auth: AuthDefaults, - #[serde(default)] - pub cli: CliDefaults, - #[serde(default)] - pub query: QueryDefaults, - #[serde(default)] - pub aliases: BTreeMap, - #[serde(default)] - pub policy: PolicySettings, - /// Top-level stored-query registry, used in single-graph - /// mode — mirrors how the top-level `policy` applies to the single - /// graph. In multi-graph mode this is unused; each graph's - /// `graphs..queries` applies instead. - #[serde(default)] - pub queries: BTreeMap, - #[serde(skip)] - base_dir: PathBuf, -} - -impl Default for OmnigraphConfig { - fn default() -> Self { - Self { - project: ProjectConfig::default(), - graphs: BTreeMap::new(), - server: ServerDefaults::default(), - auth: AuthDefaults::default(), - cli: CliDefaults::default(), - query: QueryDefaults::default(), - aliases: BTreeMap::new(), - policy: PolicySettings::default(), - queries: BTreeMap::new(), - base_dir: PathBuf::new(), - } - } -} - -impl OmnigraphConfig { - pub fn base_dir(&self) -> &Path { - &self.base_dir - } - - pub fn cli_branch(&self) -> &str { - self.cli.branch.as_deref().unwrap_or("main") - } - - pub fn cli_output_format(&self) -> ReadOutputFormat { - self.cli.output_format.unwrap_or_default() - } - - pub fn table_max_column_width(&self) -> usize { - self.cli.table_max_column_width.unwrap_or(80) - } - - pub fn table_cell_layout(&self) -> TableCellLayout { - self.cli.table_cell_layout.unwrap_or_default() - } - - pub fn cli_graph_name(&self) -> Option<&str> { - self.cli.graph.as_deref() - } - - pub fn server_graph_name(&self) -> Option<&str> { - self.server.graph.as_deref() - } - - pub fn server_bind(&self) -> &str { - self.server.bind.as_deref().unwrap_or("127.0.0.1:8080") - } - - pub fn resolve_target_name<'a>( - &self, - explicit_uri: Option<&str>, - explicit_target: Option<&'a str>, - default_target: Option<&'a str>, - ) -> Option<&'a str> { - explicit_target.or_else(|| { - if explicit_uri.is_some() { - None - } else { - default_target - } - }) - } - - pub fn graph_bearer_token_env( - &self, - explicit_uri: Option<&str>, - explicit_target: Option<&str>, - default_target: Option<&str>, - ) -> Option<&str> { - let target_name = - self.resolve_target_name(explicit_uri, explicit_target, default_target)?; - self.graphs - .get(target_name) - .and_then(|target| target.bearer_token_env.as_deref()) - } - - pub fn resolve_auth_env_file(&self) -> Option { - self.auth - .env_file - .as_deref() - .map(|path| self.resolve_config_path(path)) - } - - pub fn resolve_policy_file(&self) -> Option { - self.policy - .file - .as_deref() - .map(|path| self.resolve_config_path(path)) - } - - /// Resolve the per-graph policy file path for the named target, - /// relative to the config file's `base_dir`. Returns `None` if the - /// target is unknown or no per-graph `policy.file` is set. - pub fn resolve_target_policy_file(&self, target_name: &str) -> Option { - let target = self.graphs.get(target_name)?; - target - .policy - .file - .as_deref() - .map(|path| self.resolve_config_path(path)) - } - - /// The top-level stored-query registry entries (single-graph mode). - pub fn query_entries(&self) -> &BTreeMap { - &self.queries - } - - /// The per-graph stored-query registry entries for a named target - /// (multi-graph mode). Returns `None` if the target is unknown. - pub fn target_query_entries( - &self, - target_name: &str, - ) -> Option<&BTreeMap> { - self.graphs.get(target_name).map(|target| &target.queries) - } - - /// The stored-query registry entries that apply for a graph - /// selection — the single definition of "which `queries:` block - /// governs graph X", shared by server boot and the CLI so the two - /// can't drift. A named graph present in `graphs:` uses its - /// per-graph block; everything else (no selection, or a name that is - /// not a known graph, e.g. a bare URI) falls back to the top-level - /// block (single-graph mode). - pub fn query_entries_for(&self, graph: Option<&str>) -> &BTreeMap { - match graph { - Some(name) if self.graphs.contains_key(name) => &self.graphs[name].queries, - _ => &self.queries, - } - } - - /// The single CLI gate that turns a raw graph selection into a *validated* - /// one — the fallible counterpart to the infallible - /// [`OmnigraphConfig::query_entries_for`]. Both `queries` subcommands route - /// their selection through here so neither can skip a check the other (or - /// server boot) applies: - /// * a known name passes through, but only after the same coherence check - /// server boot enforces - /// ([`OmnigraphConfig::ensure_top_level_blocks_honored`]) — a named graph - /// with a populated top-level block is rejected; - /// * an unknown name errors with the **same** message - /// [`OmnigraphConfig::resolve_target_uri`] produces, so a command that - /// opens no URI rejects an unknown `--target` exactly like the - /// URI-resolving commands do; - /// * an anonymous selection (`None`, e.g. a bare URI) stays anonymous, - /// resolving to the top-level registry downstream (top-level honored). - pub fn resolve_graph_selection<'a>(&self, graph: Option<&'a str>) -> Result> { - match graph { - Some(name) if self.graphs.contains_key(name) => { - self.ensure_top_level_blocks_honored(Some(name))?; - Ok(Some(name)) - } - Some(name) => bail!("graph '{}' not found in {}", name, DEFAULT_CONFIG_FILE), - None => Ok(None), - } - } - - pub fn resolve_policy_tooling_graph_selection(&self) -> Result> { - self.resolve_graph_selection(self.cli_graph_name().or_else(|| self.server_graph_name())) - } - - /// The policy file that applies for a graph selection — the policy - /// sibling of [`OmnigraphConfig::query_entries_for`], so policy and - /// queries resolve by the same identity rule. A named graph in - /// `graphs:` uses its per-graph `policy.file` with **no** top-level - /// fallback (a named graph with no per-graph policy has no policy — - /// that keeps the boot-time coherence check meaningful); anything else - /// (no selection, or a bare URI) uses the top-level `policy.file`. - pub fn resolve_policy_file_for(&self, graph: Option<&str>) -> Option { - match graph { - Some(name) if self.graphs.contains_key(name) => self.resolve_target_policy_file(name), - _ => self.resolve_policy_file(), - } - } - - /// Names of any top-level config blocks (`policy.file`, `queries:`) - /// that are populated. Used by the boot-time coherence check: when a - /// **named** graph is served (single-mode by name, or multi-mode), - /// the top-level blocks are not honored, so a populated one is a - /// configuration error rather than a silent no-op. - pub fn populated_top_level_blocks(&self) -> Vec<&'static str> { - let mut blocks = Vec::new(); - if self.policy.file.is_some() { - blocks.push("policy.file"); - } - if !self.queries.is_empty() { - blocks.push("queries"); - } - blocks - } - - /// A named graph uses its own `graphs.` block, so a populated - /// top-level block would be silently ignored — a config error. The single - /// definition of that rule, shared by server boot and the CLI selection - /// gate ([`OmnigraphConfig::resolve_graph_selection`]) so the two can't - /// drift. An anonymous selection (`None`, e.g. a bare URI) legitimately - /// honors the top-level blocks, so it is never rejected here. - pub fn ensure_top_level_blocks_honored(&self, selected: Option<&str>) -> Result<()> { - if let Some(name) = selected { - let unhonored = self.populated_top_level_blocks(); - if !unhonored.is_empty() { - bail!( - "named graph '{name}' uses its own `graphs.{name}.…` block, but top-level {} \ - {} set and would be ignored. Move it to `graphs.{name}` (e.g. \ - `graphs.{name}.policy.file`, `graphs.{name}.queries`).", - unhonored.join(" and "), - if unhonored.len() == 1 { "is" } else { "are" }, - ); - } - } - Ok(()) - } - - /// Resolve a stored-query `.gq` file path (from a registry entry), - /// relative to the config's `base_dir`. Mirrors policy-file - /// resolution; the registry loader calls this to turn each entry's - /// `file:` value into an absolute path. - pub fn resolve_query_file(&self, value: &str) -> PathBuf { - self.resolve_config_path(value) - } - - /// Resolve the server-level policy file path (used by management - /// endpoints). Returns `None` if `server.policy.file` is not set. - pub fn resolve_server_policy_file(&self) -> Option { - self.server - .policy - .file - .as_deref() - .map(|path| self.resolve_config_path(path)) - } - - /// Resolve a raw config-supplied URI (which may be relative) to its - /// absolute form. URIs containing `://` are passed through as-is; - /// relative paths are joined with the config file's `base_dir`. - pub fn resolve_uri_value(&self, value: &str) -> String { - self.resolve_config_uri(value) - } - - pub fn resolve_policy_tests_file(&self) -> Option { - let policy_file = self.resolve_policy_file()?; - Some(policy_file.with_file_name("policy.tests.yaml")) - } - - pub fn alias(&self, name: &str) -> Result<&AliasConfig> { - self.aliases - .get(name) - .ok_or_else(|| color_eyre::eyre::eyre!("alias '{}' not found", name)) - } - - pub fn resolve_target_uri( - &self, - explicit_uri: Option, - explicit_target: Option<&str>, - default_target: Option<&str>, - ) -> Result { - if let Some(uri) = explicit_uri { - return Ok(uri); - } - - let target_name = explicit_target.or(default_target).ok_or_else(|| { - color_eyre::eyre::eyre!("URI must be provided via , --target, or config") - })?; - let target = self.graphs.get(target_name).ok_or_else(|| { - color_eyre::eyre::eyre!( - "graph '{}' not found in {}", - target_name, - DEFAULT_CONFIG_FILE - ) - })?; - Ok(self.resolve_config_uri(&target.uri)) - } - - pub fn resolve_query_path(&self, query: &Path) -> Result { - if query.is_absolute() { - return Ok(query.to_path_buf()); - } - - let direct = self.base_dir.join(query); - if direct.exists() { - return Ok(direct); - } - - for root in &self.query.roots { - let candidate = self.base_dir.join(root).join(query); - if candidate.exists() { - return Ok(candidate); - } - } - - bail!("query file '{}' not found", query.display()); - } - - fn resolve_config_uri(&self, value: &str) -> String { - if value.contains("://") { - return value.to_string(); - } - - let path = Path::new(value); - if path.is_absolute() { - value.to_string() - } else { - self.base_dir.join(path).to_string_lossy().to_string() - } - } - - fn resolve_config_path(&self, value: &str) -> PathBuf { - let path = Path::new(value); - if path.is_absolute() { - path.to_path_buf() - } else { - self.base_dir.join(path) - } - } -} - -pub fn default_config_path() -> PathBuf { - PathBuf::from(DEFAULT_CONFIG_FILE) -} - -/// `OMNIGRAPH_CONFIG` env var: a first-class stand-in for `--config`, one -/// name with one meaning in both binaries (the container entrypoint already -/// uses it for the server; RFC-007 §D1 extends it to the CLI). -pub const CONFIG_PATH_ENV: &str = "OMNIGRAPH_CONFIG"; - -/// RFC-008 stage 4 — opt-in strict mode: when set, loading a legacy -/// `omnigraph.yaml` is a hard error instead of a warning. For teams that -/// finished migrating and want regressions caught (a stray legacy file -/// would otherwise silently outrank operator config during the window). -/// The rehearsal for stage 5's removal. -pub const NO_LEGACY_CONFIG_ENV: &str = "OMNIGRAPH_NO_LEGACY_CONFIG"; - -pub fn load_config(config_path: Option<&PathBuf>) -> Result { - let env_path = env::var_os(CONFIG_PATH_ENV).map(PathBuf::from); - let strict = env::var_os(NO_LEGACY_CONFIG_ENV).is_some(); - load_config_in(&env::current_dir()?, config_path, env_path.as_ref(), strict) -} - -fn load_config_in( - cwd: &Path, - config_path: Option<&PathBuf>, - env_path: Option<&PathBuf>, - strict_no_legacy: bool, -) -> Result { - // Precedence: explicit --config flag > $OMNIGRAPH_CONFIG > ./omnigraph.yaml. - let explicit_path = config_path.or(env_path).cloned(); - let config_path = explicit_path.or_else(|| { - let default_path = cwd.join(DEFAULT_CONFIG_FILE); - default_path.exists().then_some(default_path) - }); - - let mut config = if let Some(path) = &config_path { - if strict_no_legacy { - // Strict refuses the FILE, not its absence — flag-less - // invocations on migrated setups keep working. - bail!( - "legacy config '{}' refused: {NO_LEGACY_CONFIG_ENV} is set (RFC-008 strict mode); run `omnigraph config migrate`, then remove the file — or unset the variable", - path.display() - ); - } - let text = fs::read_to_string(path)?; - warn_yaml_deprecation_once(path, &text); - serde_yaml::from_str::(&text)? - } else { - OmnigraphConfig::default() - }; - - config.base_dir = if let Some(path) = config_path { - absolute_base_dir(cwd, &path)? - } else { - cwd.to_path_buf() - }; - - Ok(config) -} - -/// RFC-008 stage 1: suppress the legacy-config deprecation warning -/// (one process), for CI logs during the deprecation window. -pub const SUPPRESS_YAML_DEPRECATION_ENV: &str = "OMNIGRAPH_SUPPRESS_YAML_DEPRECATION"; - -/// RFC-008's migration map (the "Where every key goes" table), applied to -/// the keys actually present in a loaded file — never a generic banner. -/// Keys are `(yaml pointer, destination)`; the pointer is matched against -/// the file's real top-level/nested keys. -const YAML_DEPRECATION_MAP: &[(&str, &str)] = &[ - ("graphs", "cluster.yaml `graphs:` (team surface) — or flags/env for the zero-config tier"), - ("queries", "the cluster catalog (`.gq` discovery in cluster.yaml)"), - ("policy", "cluster.yaml `policies:` + `applies_to` bindings"), - ("server", "flags/env (`--bind`); meaningless under cluster boot"), - ("auth", "the operator credentials chain (`omnigraph login `)"), - ("aliases", "operator `aliases:` (bindings) + catalog stored queries (content)"), - ("query", "obsolete — cluster query discovery replaced `query.roots`"), - ("project", "cluster.yaml `metadata.name`"), - ("cli.actor", "`operator.actor` in ~/.omnigraph/config.yaml"), - ("cli.output_format", "`defaults.output` in ~/.omnigraph/config.yaml"), - ("cli.table_max_column_width", "`defaults.table_max_column_width` in ~/.omnigraph/config.yaml"), - ("cli.table_cell_layout", "`defaults.table_cell_layout` in ~/.omnigraph/config.yaml"), - ("cli.graph", "explicit `--target`/`--server` (no operator default-target yet)"), - ("cli.branch", "explicit `--branch`"), -]; - -/// Emit the per-key deprecation block once per process when a legacy -/// `omnigraph.yaml` is actually loaded. `omnigraph config migrate` -/// produces the split these lines describe. -fn warn_yaml_deprecation_once(path: &Path, text: &str) { - static WARNED: std::sync::OnceLock<()> = std::sync::OnceLock::new(); - if env::var_os(SUPPRESS_YAML_DEPRECATION_ENV).is_some() { - return; - } - let lines = yaml_deprecation_lines(text); - if lines.is_empty() { - return; - } - WARNED.get_or_init(|| { - eprintln!( - "warning: '{}' is deprecated (RFC-008) — its keys have new homes; run `omnigraph config migrate` for the split, set {SUPPRESS_YAML_DEPRECATION_ENV}=1 to silence:", - path.display() - ); - for line in &lines { - eprintln!(" {line}"); - } - }); -} - -fn yaml_deprecation_lines(text: &str) -> Vec { - let Ok(mapping) = serde_yaml::from_str::(text) else { - return Vec::new(); - }; - let present = |pointer: &str| -> bool { - match pointer.split_once('.') { - None => mapping.contains_key(pointer), - Some((outer, inner)) => mapping - .get(outer) - .and_then(|value| value.as_mapping()) - .is_some_and(|nested| nested.contains_key(inner)), - } - }; - YAML_DEPRECATION_MAP - .iter() - .filter(|(pointer, _)| present(pointer)) - .map(|(pointer, destination)| format!("`{pointer}` -> {destination}")) - .collect() -} - -fn absolute_base_dir(cwd: &Path, path: &Path) -> Result { - let path = if path.is_absolute() { - path.to_path_buf() - } else { - cwd.join(path) - }; - Ok(path - .parent() - .map(Path::to_path_buf) - .unwrap_or_else(|| cwd.to_path_buf())) -} - -#[cfg(test)] -mod tests { - use std::fs; - use std::path::{Path, PathBuf}; - - use tempfile::tempdir; - - use super::{ - ReadOutputFormat, TableCellLayout, graph_resource_id_for_selection, load_config_in, - }; - - #[test] - fn env_config_path_stands_in_for_the_flag_but_loses_to_it() { - let temp = tempdir().unwrap(); - let flag_path = temp.path().join("flag.yaml"); - let env_path = temp.path().join("env.yaml"); - fs::write(&flag_path, "cli:\n actor: act-flag\n").unwrap(); - fs::write(&env_path, "cli:\n actor: act-env\n").unwrap(); - - // $OMNIGRAPH_CONFIG used when no flag… - let config = load_config_in(temp.path(), None, Some(&env_path), false).unwrap(); - assert_eq!(config.cli.actor.as_deref(), Some("act-env")); - - // …loses to an explicit --config… - let config = load_config_in(temp.path(), Some(&flag_path), Some(&env_path), false).unwrap(); - assert_eq!(config.cli.actor.as_deref(), Some("act-flag")); - - // …and beats the cwd default file. - fs::write(temp.path().join("omnigraph.yaml"), "cli:\n actor: act-cwd\n").unwrap(); - let config = load_config_in(temp.path(), None, Some(&env_path), false).unwrap(); - assert_eq!(config.cli.actor.as_deref(), Some("act-env")); - } - - #[test] - fn strict_mode_refuses_the_file_not_its_absence() { - let temp = tempdir().unwrap(); - // No file: strict mode changes nothing (defaults load). - let config = load_config_in(temp.path(), None, None, true).unwrap(); - assert!(config.cli.actor.is_none()); - - // File present: strict refuses with the migrate pointer. - fs::write(temp.path().join("omnigraph.yaml"), "cli:\n actor: a\n").unwrap(); - let err = load_config_in(temp.path(), None, None, true).unwrap_err(); - let message = err.to_string(); - assert!( - message.contains("OMNIGRAPH_NO_LEGACY_CONFIG") && message.contains("config migrate"), - "{message}" - ); - // Without strict, the same file loads. - assert!(load_config_in(temp.path(), None, None, false).is_ok()); - } - - #[test] - fn yaml_deprecation_lines_name_present_keys_only() { - let lines = super::yaml_deprecation_lines( - "graphs:\n g:\n uri: /tmp/x\ncli:\n actor: a\n branch: main\n", - ); - let joined = lines.join("\n"); - assert!(joined.contains("`graphs` ->"), "{joined}"); - assert!(joined.contains("`cli.actor` -> `operator.actor`"), "{joined}"); - assert!(joined.contains("`cli.branch` ->"), "{joined}"); - assert!(!joined.contains("`aliases`"), "{joined}"); - assert!(!joined.contains("`cli.output_format`"), "{joined}"); - - assert!(super::yaml_deprecation_lines("").is_empty()); - assert!(super::yaml_deprecation_lines("not: [valid").is_empty()); - } - - #[test] - fn load_config_reads_yaml_defaults_from_current_dir() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - r#" -graphs: - local: - uri: ./demo.omni - bearer_token_env: DEMO_TOKEN -auth: - env_file: .env.omni -cli: - graph: local - branch: main - output_format: kv - table_max_column_width: 40 - table_cell_layout: wrap -policy: {} -"#, - ) - .unwrap(); - - let config = load_config_in(temp.path(), None, None, false).unwrap(); - assert_eq!(config.cli_graph_name(), Some("local")); - assert_eq!(config.cli_branch(), "main"); - assert_eq!(config.cli_output_format(), ReadOutputFormat::Kv); - assert_eq!(config.table_max_column_width(), 40); - assert_eq!(config.table_cell_layout(), TableCellLayout::Wrap); - assert_eq!( - config.graph_bearer_token_env(None, None, config.cli_graph_name()), - Some("DEMO_TOKEN") - ); - assert_eq!( - config.resolve_auth_env_file().unwrap(), - temp.path().join(".env.omni") - ); - assert_eq!( - PathBuf::from( - config - .resolve_target_uri(None, None, config.cli_graph_name()) - .unwrap() - ), - temp.path().join("./demo.omni") - ); - } - - #[test] - fn load_config_does_not_walk_parent_directories() { - let temp = tempdir().unwrap(); - let child = temp.path().join("child"); - fs::create_dir_all(&child).unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - "graphs:\n local:\n uri: ./demo.omni\n", - ) - .unwrap(); - - let config = load_config_in(&child, None, None, false).unwrap(); - assert!(config.graphs.is_empty()); - } - - #[test] - fn graph_resource_id_for_selection_uses_name_or_anonymous_uri() { - assert_eq!( - graph_resource_id_for_selection(Some("local"), "/tmp/graph.omni"), - "local" - ); - assert_eq!( - graph_resource_id_for_selection(None, "/tmp/graph.omni"), - "/tmp/graph.omni" - ); - } - - #[test] - fn resolve_graph_selection_validates_membership_and_coherence() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - "graphs:\n local:\n uri: ./demo.omni\n", - ) - .unwrap(); - let config = load_config_in(temp.path(), None, None, false).unwrap(); - - // A known graph passes through unchanged. - assert_eq!(config.resolve_graph_selection(Some("local")).unwrap(), Some("local")); - // An anonymous selection stays anonymous (→ top-level registry downstream). - assert_eq!(config.resolve_graph_selection(None).unwrap(), None); - // An unknown name errors, naming the graph (matching resolve_target_uri). - let err = config.resolve_graph_selection(Some("ghost")).unwrap_err().to_string(); - assert!( - err.contains("ghost") && err.contains("not found"), - "unknown graph must error naming it: {err}" - ); - - // Coherence: a named graph plus a populated top-level block is the - // config server boot refuses, so the gate rejects it too (shared rule - // via ensure_top_level_blocks_honored). An anonymous selection still - // passes — top-level is honored when no graph is named. - let temp2 = tempdir().unwrap(); - fs::write( - temp2.path().join("omnigraph.yaml"), - "graphs:\n local:\n uri: ./demo.omni\npolicy:\n file: ./top.yaml\n", - ) - .unwrap(); - let incoherent = load_config_in(temp2.path(), None, None, false).unwrap(); - let err = incoherent - .resolve_graph_selection(Some("local")) - .unwrap_err() - .to_string(); - assert!( - err.contains("local") && err.contains("policy.file"), - "named graph + populated top-level block must be rejected, naming both: {err}" - ); - assert_eq!( - incoherent.resolve_graph_selection(None).unwrap(), - None, - "anonymous selection still honors top-level" - ); - } - - #[test] - fn policy_tooling_graph_selection_prefers_cli_then_server_and_validates() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - "graphs:\n local:\n uri: ./local.omni\n prod:\n uri: ./prod.omni\n\ - server:\n graph: local\ncli:\n graph: prod\n", - ) - .unwrap(); - let config = load_config_in(temp.path(), None, None, false).unwrap(); - assert_eq!( - config.resolve_policy_tooling_graph_selection().unwrap(), - Some("prod") - ); - - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - "graphs:\n local:\n uri: ./local.omni\nserver:\n graph: local\n", - ) - .unwrap(); - let config = load_config_in(temp.path(), None, None, false).unwrap(); - assert_eq!( - config.resolve_policy_tooling_graph_selection().unwrap(), - Some("local") - ); - - let temp = tempdir().unwrap(); - fs::write(temp.path().join("omnigraph.yaml"), "policy: {}\n").unwrap(); - let config = load_config_in(temp.path(), None, None, false).unwrap(); - assert_eq!(config.resolve_policy_tooling_graph_selection().unwrap(), None); - - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - "graphs:\n local:\n uri: ./local.omni\nserver:\n graph: ghost\n", - ) - .unwrap(); - let config = load_config_in(temp.path(), None, None, false).unwrap(); - let err = config - .resolve_policy_tooling_graph_selection() - .unwrap_err() - .to_string(); - assert!( - err.contains("ghost") && err.contains("not found"), - "unknown server.graph must use graph-selection validation: {err}" - ); - } - - #[test] - fn resolve_query_path_searches_config_roots() { - let temp = tempdir().unwrap(); - fs::create_dir_all(temp.path().join("queries")).unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - "query:\n roots:\n - queries\npolicy: {}\n", - ) - .unwrap(); - fs::write( - temp.path().join("queries").join("test.gq"), - "query q { return {} }", - ) - .unwrap(); - - let config = load_config_in(temp.path(), None, None, false).unwrap(); - let resolved = config.resolve_query_path(Path::new("test.gq")).unwrap(); - assert_eq!(resolved, temp.path().join("queries").join("test.gq")); - } - - #[test] - fn resolve_query_path_prefers_config_base_dir_over_ambient_cwd() { - let workspace = tempdir().unwrap(); - let config_dir = workspace.path().join("config"); - let ambient_dir = workspace.path().join("ambient"); - fs::create_dir_all(&config_dir).unwrap(); - fs::create_dir_all(&ambient_dir).unwrap(); - fs::write(config_dir.join("omnigraph.yaml"), "policy: {}\n").unwrap(); - fs::write(config_dir.join("local.gq"), "query local { return {} }").unwrap(); - fs::write(ambient_dir.join("local.gq"), "query ambient { return {} }").unwrap(); - - let config = - load_config_in(&ambient_dir, Some(&config_dir.join("omnigraph.yaml")), None, false).unwrap(); - let resolved = config.resolve_query_path(Path::new("local.gq")).unwrap(); - - assert_eq!(resolved, config_dir.join("local.gq")); - } - - #[test] - fn queries_block_round_trips_inline_and_per_graph() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - r#" -graphs: - prod: - uri: s3://bucket/prod - queries: - find_user: - file: ./queries/find_user.gq - mcp: - expose: true - tool_name: lookup_user - internal_audit: - file: ./queries/audit.gq -queries: - single_mode_q: - file: ./q.gq -"#, - ) - .unwrap(); - - let config = load_config_in(temp.path(), None, None, false).unwrap(); - - // Per-graph registry (multi-graph mode). - let prod = config.target_query_entries("prod").unwrap(); - assert_eq!(prod.len(), 2); - let find_user = &prod["find_user"]; - assert_eq!(find_user.file, "./queries/find_user.gq"); - assert!(find_user.mcp.expose); - assert_eq!(find_user.mcp.tool_name.as_deref(), Some("lookup_user")); - // Default exposure is true (the manifest entry is the opt-in); tool_name absent. - let audit = &prod["internal_audit"]; - assert!(audit.mcp.expose); - assert!(audit.mcp.tool_name.is_none()); - - // Top-level registry (single-graph mode). - assert_eq!(config.query_entries().len(), 1); - - // The shared selector resolves the same blocks the server boot - // and the CLI use: a known graph → its per-graph block; no - // selection or an unknown name → the top-level block (the latter - // pins the behavior of the CLI's now-deleted fallback arm). - assert_eq!(config.query_entries_for(Some("prod")).len(), 2); - assert_eq!(config.query_entries_for(None).len(), 1); - assert_eq!(config.query_entries_for(Some("nonexistent")).len(), 1); - - // Path resolution joins against base_dir, like policy files. - assert_eq!( - config.resolve_query_file(&find_user.file), - temp.path().join("./queries/find_user.gq") - ); - } - - #[test] - fn resolve_policy_file_for_follows_identity() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - "policy:\n file: ./top.yaml\ngraphs:\n prod:\n uri: s3://b/prod\n \ - policy:\n file: ./prod.yaml\n bare:\n uri: s3://b/bare\n", - ) - .unwrap(); - let config = load_config_in(temp.path(), None, None, false).unwrap(); - - // Named graph with its own policy → per-graph (not top-level). - assert!( - config - .resolve_policy_file_for(Some("prod")) - .unwrap() - .ends_with("prod.yaml") - ); - // Named graph with NO per-graph policy → None (no top-level fallback; - // load-bearing for the boot coherence check). - assert!(config.resolve_policy_file_for(Some("bare")).is_none()); - // Anonymous (bare URI) or an unknown name → top-level. - assert!( - config - .resolve_policy_file_for(None) - .unwrap() - .ends_with("top.yaml") - ); - assert!( - config - .resolve_policy_file_for(Some("nope")) - .unwrap() - .ends_with("top.yaml") - ); - } - - #[test] - fn queries_block_absent_yields_empty_registry() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - "graphs:\n local:\n uri: ./demo.omni\n", - ) - .unwrap(); - - let config = load_config_in(temp.path(), None, None, false).unwrap(); - // Additive: no `queries:` anywhere → empty registries everywhere. - assert!(config.query_entries().is_empty()); - assert!( - config - .target_query_entries("local") - .unwrap() - .is_empty() - ); - } - - #[test] - fn policy_block_accepts_non_empty_mapping() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - "policy:\n file: ./policy.yaml\n", - ) - .unwrap(); - - let config = load_config_in(temp.path(), None, None, false).unwrap(); - assert_eq!( - config.resolve_policy_file().unwrap(), - temp.path().join("policy.yaml") - ); - } - - #[test] - fn scoped_auth_env_ignores_default_target_when_uri_is_explicit() { - let temp = tempdir().unwrap(); - fs::write( - temp.path().join("omnigraph.yaml"), - r#" -graphs: - demo: - uri: https://example.com - bearer_token_env: DEMO_TOKEN -cli: - graph: demo -"#, - ) - .unwrap(); - - let config = load_config_in(temp.path(), None, None, false).unwrap(); - assert_eq!( - config.graph_bearer_token_env( - Some("https://override.example.com"), - None, - config.cli_graph_name() - ), - None - ); - assert_eq!( - config.graph_bearer_token_env( - Some("https://override.example.com"), - Some("demo"), - config.cli_graph_name() - ), - Some("DEMO_TOKEN") - ); - } -} diff --git a/crates/omnigraph-server/src/lib.rs b/crates/omnigraph-server/src/lib.rs index 4cd6492..e6f63dc 100644 --- a/crates/omnigraph-server/src/lib.rs +++ b/crates/omnigraph-server/src/lib.rs @@ -5,7 +5,6 @@ pub use settings::{load_server_settings, classify_server_runtime_state, ServerRu use settings::*; use handlers::*; pub mod auth; -pub mod config; pub mod graph_id; pub mod identity; pub mod policy; @@ -46,11 +45,6 @@ use axum::response::{IntoResponse, Response}; use axum::routing::{delete, get, post}; use axum::{Json, Router}; use color_eyre::eyre::{Result, WrapErr, bail, eyre}; -pub use config::{ - AliasCommand, AliasConfig, CliDefaults, DEFAULT_CONFIG_FILE, OmnigraphConfig, PolicySettings, - ProjectConfig, QueryDefaults, ReadOutputFormat, ServerDefaults, TableCellLayout, TargetConfig, - graph_resource_id_for_selection, load_config, -}; use futures::stream; use omnigraph::db::{Omnigraph, ReadTarget}; use omnigraph::error::{ManifestConflictDetails, ManifestErrorKind, OmniError}; @@ -879,18 +873,6 @@ fn validate_and_attach( }) } -/// Format every load error (parse / identity failure) into a multi-line -/// boot-abort message. -fn format_registry_load_errors(label: &str, errors: &[queries::LoadError]) -> String { - let joined = errors - .iter() - .map(|e| e.to_string()) - .collect::>() - .join("\n "); - format!("graph '{label}': stored-query registry failed to load:\n {joined}") -} - - pub fn build_app(state: AppState) -> Router { // The per-graph protected routes, identical in single + multi mode. // Two middleware layers wrap them (outer first, inner last): diff --git a/crates/omnigraph-server/src/queries.rs b/crates/omnigraph-server/src/queries.rs index bf131c8..09d2491 100644 --- a/crates/omnigraph-server/src/queries.rs +++ b/crates/omnigraph-server/src/queries.rs @@ -13,7 +13,6 @@ //! Renaming either is a breaking change to callers, by design. use std::collections::BTreeMap; -use std::fs; use std::sync::Arc; use omnigraph_compiler::catalog::Catalog; @@ -22,8 +21,6 @@ use omnigraph_compiler::query::parser::parse_query; use omnigraph_compiler::query::typecheck::typecheck_query_decl; use omnigraph_compiler::types::{PropType, ScalarType}; -use crate::config::{OmnigraphConfig, QueryEntry}; - /// One loaded stored query. `source` is the full `.gq` file text — the /// invocation handler hands it to `run_query` / `run_mutate` verbatim, /// which reuse the same parse/IR/exec path as the inline routes (no @@ -68,8 +65,9 @@ pub struct QueryRegistry { by_name: BTreeMap, } -/// In-memory registry entry before file I/O. Used by [`QueryRegistry::load`] -/// (after reading each `.gq` from disk) and directly by tests. +/// In-memory registry spec: a query's name + already-read `.gq` source. The +/// input to [`QueryRegistry::from_specs`] — built by the server's cluster boot +/// and by the CLI's `queries` tooling from a cluster serving snapshot. #[derive(Debug, Clone)] pub struct RegistrySpec { pub name: String, @@ -169,47 +167,6 @@ impl QueryRegistry { } } - /// Read each registry entry's `.gq` file from disk and build the - /// registry. `entries` is either the top-level `queries` map (single - /// mode) or a graph's `queries` map (multi mode); `config` resolves - /// each entry's relative `file:` path against `base_dir`. - pub fn load( - config: &OmnigraphConfig, - entries: &BTreeMap, - ) -> Result> { - let mut specs = Vec::with_capacity(entries.len()); - let mut errors = Vec::new(); - for (name, entry) in entries { - let path = config.resolve_query_file(&entry.file); - match fs::read_to_string(&path) { - Ok(source) => specs.push(RegistrySpec { - name: name.clone(), - source, - expose: entry.mcp.expose, - tool_name: entry.mcp.tool_name.clone(), - }), - Err(err) => errors.push(LoadError { - query: Some(name.clone()), - message: format!("cannot read '{}': {err}", path.display()), - }), - } - } - - // Parse/identity/uniqueness-check the readable specs even when some - // files failed to read, so every broken entry (I/O, parse, identity, - // tool-name collision) surfaces in one pass rather than one per - // restart. I/O errors come first (in `entries` key order), then the - // spec errors. A non-empty `errors` always fails the load. - match Self::from_specs(specs) { - Ok(registry) if errors.is_empty() => Ok(registry), - Ok(_) => Err(errors), - Err(spec_errors) => { - errors.extend(spec_errors); - Err(errors) - } - } - } - pub fn lookup(&self, name: &str) -> Option<&StoredQuery> { self.by_name.get(name) } @@ -653,36 +610,4 @@ embedding: Vector(4) assert!(entry2.params.is_empty(), "no declared params → empty list"); } - // --- load() error collection (file I/O + parse in one pass) --- - - #[test] - fn load_collects_io_and_parse_errors_in_one_pass() { - use crate::config::load_config; - let temp = tempfile::tempdir().unwrap(); - std::fs::write( - temp.path().join("good.gq"), - "query good() { match { $u: User } return { $u.name } }", - ) - .unwrap(); - std::fs::write(temp.path().join("broken.gq"), "query broken( {{ not valid").unwrap(); - // `missing.gq` is deliberately not written (an I/O failure). - std::fs::write( - temp.path().join("omnigraph.yaml"), - "queries:\n good:\n file: ./good.gq\n \ - missing:\n file: ./missing.gq\n broken:\n file: ./broken.gq\n", - ) - .unwrap(); - let config = load_config(Some(&temp.path().join("omnigraph.yaml"))).unwrap(); - - let errors = QueryRegistry::load(&config, config.query_entries()).unwrap_err(); - let joined = errors.iter().map(|e| e.to_string()).collect::>().join("\n"); - // Both the missing file AND the parse error surface in one pass — - // the I/O failure must not mask the parse failure. - assert!(joined.contains("missing"), "I/O error must surface: {joined}"); - assert!( - joined.contains("broken") && joined.contains("parse error"), - "the parse error in a readable file must surface in the same pass: {joined}" - ); - assert!(!joined.contains("'good'"), "the valid entry is not an error: {joined}"); - } } diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md index 4e8d3c6..004a98a 100644 --- a/docs/dev/architecture.md +++ b/docs/dev/architecture.md @@ -1,6 +1,6 @@ # Architecture -OmniGraph is a typed property-graph engine built as a coordination layer over many Lance datasets, with Git-style branches and commits across the whole graph, multi-modal querying (vector + FTS + BM25 + RRF + graph traversal) in one runtime, an HTTP server with Cedar policy, and a CLI driven by a single `omnigraph.yaml`. +OmniGraph is a typed property-graph engine built as a coordination layer over many Lance datasets, with Git-style branches and commits across the whole graph, multi-modal querying (vector + FTS + BM25 + RRF + graph traversal) in one runtime, an HTTP server with Cedar policy, and a CLI driven by a per-operator `~/.omnigraph/config.yaml` plus team-owned cluster directories. ## Reading guide diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 38b81f2..8d6a305 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -7,7 +7,7 @@ This file is the always-on map of the test surface. **Consult it before every ta | Crate | Path | Style | |---|---|---| | `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (28 files), fixture-driven, share `tests/helpers/mod.rs` | -| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | Per-area suites (post-modularization): `cli_cluster.rs` (cluster command surface + operator-actor cascade), `cli_cluster_e2e.rs` (spawned-binary lifecycle compositions — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `cli_data.rs` (load/read/change/branch/commit/export/snapshot/policy/embed/maintenance + operator format cascade), `cli_schema_config.rs` (init/config, schema plan/apply, RFC-008 deprecation warnings + `config migrate` + strict mode), `cli_queries.rs`, `parity_matrix.rs` (RFC-009 Phase 1: the embedded-vs-remote referee — every forked verb run against both arms with matched Cedar policy and the same actor, scrubbed-JSON + exit-code equality; divergences are pinned in its `KNOWN_DIVERGENCES` ledger, never silently repaired), `system_local.rs` (full-cycle cluster lifecycle with a spawned `--cluster` server, applied-policy enforcement over HTTP, keyed-credential auth, operator aliases), `system_remote.rs`; share `tests/support/mod.rs` (hermetic `OMNIGRAPH_HOME` by default) | +| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | Per-area suites (post-modularization): `cli_cluster.rs` (cluster command surface + operator-actor cascade), `cli_cluster_e2e.rs` (spawned-binary lifecycle compositions — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `cli_data.rs` (load/read/change/branch/commit/export/snapshot/policy/embed/maintenance + operator format cascade), `cli_schema_config.rs` (init/config, schema plan/apply), `cli_queries.rs`, `parity_matrix.rs` (RFC-009 Phase 1: the embedded-vs-remote referee — every forked verb run against both arms with matched Cedar policy and the same actor, scrubbed-JSON + exit-code equality; divergences are pinned in its `KNOWN_DIVERGENCES` ledger, never silently repaired), `system_local.rs` (full-cycle cluster lifecycle with a spawned `--cluster` server, applied-policy enforcement over HTTP, keyed-credential auth, operator aliases), `system_remote.rs`; share `tests/support/mod.rs` (hermetic `OMNIGRAPH_HOME` by default) | | `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated); `tests/s3_cluster.rs` (bucket-gated full lifecycle on object storage) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows), and 5A policy binding metadata (applies_to in the applied revision, binding-change diffing + convergence, pre-5A backfill), and the 5B serving-snapshot read API (converged read, refusal rows) | | `omnigraph-server` | `crates/omnigraph-server/tests/` | Per-area suites (post-modularization): `auth_policy.rs`, `data_routes.rs`, `schema_routes.rs`, `stored_queries.rs`, `multi_graph.rs` (cluster-mode boot — converged serving, policy binding wiring, boot refusals — + the concurrent branch-ops matrix), `boot_settings.rs` (mode inference, PolicySource), `s3.rs` (bucket-gated: single-graph serving + config-free `--cluster s3://` boot), `openapi.rs` (OpenAPI drift / regeneration); share `tests/support/mod.rs` | | `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint | diff --git a/docs/user/cli/index.md b/docs/user/cli/index.md index f77a65e..d8bf66e 100644 --- a/docs/user/cli/index.md +++ b/docs/user/cli/index.md @@ -78,20 +78,26 @@ literal URL); a positional `http(s)://` URI is rejected. If the server requires auth, set its bearer token and `omnigraph login ` (or `OMNIGRAPH_BEARER_TOKEN`). -## Multi-graph servers (v0.6.0+) +## Multi-graph servers -Against a multi-graph server (started with `--config omnigraph.yaml` referencing a non-empty `graphs:` map), use `omnigraph graphs list` to enumerate the registered graphs. The server must configure bearer tokens and `server.policy.file` with a rule that allows `graph_list`; `/graphs` is closed by default even when the server runs with `--unauthenticated`. +A server boots from a cluster directory (`omnigraph-server --cluster `) and +serves every graph the cluster declares. Use `omnigraph graphs list` to enumerate +them. The cluster's server-level policy must allow `graph_list`; `/graphs` is +closed by default even when the server runs with `--unauthenticated`. ```bash OMNIGRAPH_BEARER_TOKEN=admin-token \ - omnigraph graphs list --uri http://server.example.com --json + omnigraph graphs list --server http://server.example.com --json ``` -For config-driven clients, set the remote graph's `bearer_token_env` to an environment variable containing a token whose actor is authorized by `server.policy.file`. +For an operator-defined server, store its token with `omnigraph login ` (or +`OMNIGRAPH_TOKEN_`); the actor must be authorized by the cluster's +server-level policy. -`list` rejects local URI targets — it's for remote multi-graph servers only. +`list` rejects local (`--store`) targets — it's for remote multi-graph servers only. -Runtime add/remove is **not** in v0.6.0. To add a graph, stop the server, add a `graphs.` entry to `omnigraph.yaml`, then restart. To remove, stop the server, delete the entry, restart. +Runtime add/remove via API is not exposed. To add or remove a graph, edit the +cluster's `cluster.yaml`, run `omnigraph cluster apply`, then restart the server. Per-graph addressing: select a graph on a multi-graph server with `--graph`: @@ -107,9 +113,9 @@ omnigraph check --query queries.gq graph.omni --json omnigraph schema plan --schema next.pg graph.omni --json omnigraph schema apply --schema next.pg graph.omni --json -omnigraph policy validate --config omnigraph.yaml -omnigraph policy test --config omnigraph.yaml -omnigraph policy explain --config omnigraph.yaml --actor act-alice --action read --branch main +omnigraph policy validate --cluster ./company-brain --graph knowledge +omnigraph policy test --cluster ./company-brain --graph knowledge --tests policy.tests.yaml +omnigraph policy explain --cluster ./company-brain --graph knowledge --actor act-alice --action read --branch main omnigraph commit list graph.omni --json omnigraph commit show --uri graph.omni --json @@ -123,34 +129,29 @@ also pass `--schema`. ## Config -`omnigraph.yaml` lets the CLI and server share named graphs, defaults, and -query roots: +Configuration has two surfaces with single owners (see the +[CLI reference](reference.md#config-surfaces) for the full schema): + +- **`~/.omnigraph/config.yaml`** — your personal operator config: default actor + (`--as`), named servers + credentials, clusters, profiles, aliases, and + default scope (`defaults.server` / `defaults.store` / `default_graph`). It + decides *who you are* and *what you address by default*. +- **`cluster.yaml`** (a team-owned cluster directory) — declares *what the system + is*: graphs, schemas, stored queries, policies, and storage. A server boots + from it (`--cluster `); see the [cluster guide](../clusters/index.md). ```yaml -graphs: - local: - uri: demo.omni +# ~/.omnigraph/config.yaml +operator: + actor: act-andrew +servers: dev: - uri: http://127.0.0.1:8080 - bearer_token_env: OMNIGRAPH_BEARER_TOKEN - -cli: - graph: local - branch: main - -query: - roots: - - queries - - . + url: http://127.0.0.1:8080 +defaults: + server: dev + default_graph: knowledge ``` -The config file can also define: - -- server bind defaults -- auth env files -- query aliases for common read and change commands -- `policy.file` for Cedar authorization rules - When policy is enabled, `schema apply` is authorized through the `schema_apply` action and is typically limited to admins on protected `main`. @@ -168,6 +169,6 @@ one-line warning to stderr and otherwise behave identically. | `omnigraph query lint` | `omnigraph lint` | Same flags. The argv-level shim rewrites `query lint` to `lint`. | | `omnigraph query check` | `omnigraph check` | `check` is a visible alias of `omnigraph lint`. | -The `command:` field in `aliases.` in `omnigraph.yaml` accepts both -`read` / `change` (legacy) and `query` / `mutate` (canonical); the two +The `command:` field in `aliases.` in `~/.omnigraph/config.yaml` accepts +both `read` / `change` (legacy) and `query` / `mutate` (canonical); the two spellings are interchangeable on the wire via serde aliases. diff --git a/docs/user/cli/reference.md b/docs/user/cli/reference.md index 9dd128d..2429bdf 100644 --- a/docs/user/cli/reference.md +++ b/docs/user/cli/reference.md @@ -1,6 +1,6 @@ # CLI Reference (`omnigraph`) -A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` schema. For a quick-start guide, see [cli.md](index.md). +A reference for the `omnigraph` binary's command surface and the per-operator `~/.omnigraph/config.yaml` schema. For a quick-start guide, see [cli.md](index.md). Top-level command families and subcommands. Graph-targeting commands accept a positional `file://`/`s3://` URI, `--server ` (an operator-defined server from `~/.omnigraph/config.yaml` by name, or a literal `http(s)://` URL, optionally with `--graph ` for multi-graph servers; exclusive with a positional URI), `--store ` (a single graph's storage directly), or `--profile ` / `$OMNIGRAPH_PROFILE` (a named scope bundle; see [Scopes & profiles](#scopes--profiles-rfc-011)); `cluster` commands use `--config `. A remote server is addressed only with `--server` — a positional `http(s)://` URI is rejected. **`query`/`mutate` are the exception**: their positional is a stored-query *name* (RFC-011 D3), not a graph URI, so they address the graph only via `--store`/`--server`/`--profile`/defaults. @@ -8,7 +8,7 @@ Top-level command families and subcommands. Graph-targeting commands accept a po | Command | Purpose | |---|---| -| `init` | `--schema ` → initialize a graph (no longer scaffolds `omnigraph.yaml`; start cluster configs from the [cluster.md](../clusters/index.md) quick-start or `config migrate`) | +| `init` | `--schema ` → initialize a graph (start cluster configs from the [cluster.md](../clusters/index.md) quick-start) | | `load` | bulk load a branch, local or remote (`--mode overwrite\|append\|merge` is **required** — overwrite is destructive, so there is no default). Without `--from` the target branch must exist; `--from ` forks a missing `--branch` from `` first | | `ingest` | deprecated alias of `load --from ` (defaults: `--from main --mode merge`); prints a one-line warning to stderr | | `query ` (alias: `read`) | run a read query. **Catalog lane** (default): `` is a stored query invoked **by name** from the served catalog (served-only — address with `--server`/`--profile`; the verb asserts the query is a read). **Ad-hoc lane**: with `--query ` or `-e`/`--query-string `, runs that source (the positional `` then selects which query in it). No positional graph URI — address via `--store`/`--server`/`--profile`. `read` is the deprecated previous name (one-line stderr warning) | @@ -20,13 +20,12 @@ Top-level command families and subcommands. Graph-targeting commands accept a po | `commit list \| show` | inspect commit graph | | `schema plan \| apply \| show (alias: get)` | migrations | | `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` | -| `config migrate` | propose (or `--write`: apply) the split of a legacy `omnigraph.yaml` — team half → ready-to-review `cluster.yaml`, personal half → `~/.omnigraph/config.yaml` (key-level merge, existing entries win), plus dropped-key reasons and manual steps | -| `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | declarative cluster control plane. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`, annotates dispositions, and embeds real schema-migration previews; `apply` converges the cluster — stored-query/policy catalog writes (content-addressed under `__cluster/resources/`), graph creates, schema updates (soft drops only; `--as` records the actor), and graph deletes behind a digest-bound approval from `cluster approve --as ` (`apply`/`approve` default the actor from the per-operator `omnigraph.yaml`'s `cli.actor` when `--as` is omitted; nothing else in that file affects cluster commands); what apply converges is what an `omnigraph-server --cluster ` deployment serves on its next restart (`--cluster` is the server's only boot source — RFC-011 cluster-only); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock ` manually removes a held local state lock by exact id | +| `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | declarative cluster control plane. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`, annotates dispositions, and embeds real schema-migration previews; `apply` converges the cluster — stored-query/policy catalog writes (content-addressed under `__cluster/resources/`), graph creates, schema updates (soft drops only; `--as` records the actor), and graph deletes behind a digest-bound approval from `cluster approve --as ` (`apply`/`approve` default the actor from `~/.omnigraph/config.yaml`'s `operator.actor` when `--as` is omitted); what apply converges is what an `omnigraph-server --cluster ` deployment serves on its next restart (`--cluster` is the server's only boot source — RFC-011 cluster-only); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock ` manually removes a held local state lock by exact id | | `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) | | `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review | | `cleanup --keep N --older-than 7d --confirm` | destructive version GC (`--confirm` to execute; also needs `--yes` against a non-local `s3://` target — see *Write diagnostics & destructive confirmation*) | | `embed` | offline JSONL embedding pipeline | -| `policy validate \| test \| explain` | Cedar tooling. Selects `cli.graph`, else `server.graph`, else top-level `policy.file` | +| `policy validate \| test \| explain` | Cedar tooling against a cluster's applied policies (`--cluster `; `--graph ` picks a graph's bundle when several apply). `test` takes `--tests `; `explain` takes `--actor`/`--action`/`--branch`/`--target-branch` | | `version` / `-v` | print `omnigraph 0.3.x` | ## Command capabilities @@ -69,21 +68,16 @@ Two config surfaces with single owners, plus a zero-config tier: | Operator config | one person | `~/.omnigraph/config.yaml` (override dir with `$OMNIGRAPH_HOME`) | who **I** am: identity, ergonomics | | Flags / env | per invocation | — | everything, explicitly | -`omnigraph.yaml` (below) is the legacy combined file — fully supported -today, slated for staged deprecation; its keys' future homes are -listed there. - ### `~/.omnigraph/config.yaml` (operator) ```yaml operator: - actor: act-andrew # default identity for every --as cascade: - # --as > legacy cli.actor > operator.actor > none + actor: act-andrew # default identity for the --as cascade: --as > operator.actor > none servers: # operator-owned endpoints; names key the credentials prod: url: https://graph.example.com # no tokens in this file, ever defaults: - output: table # read format default, below --json/--format/alias/legacy + output: table # read format default, below --json/--format/alias server: prod # the everyday SERVED scope when no address is given (RFC-011) # store: file:///data/dev.omni # OR a zero-flag LOCAL default (mutually # # exclusive with `server`); the local-dev @@ -98,8 +92,8 @@ profiles: # named scope bundles (RFC-011); pick with --profile ``` Absent file = empty layer. Unknown keys warn and load (a file written for a -newer CLI works on an older one). `$OMNIGRAPH_CONFIG=` stands in for -`--config` (the flag wins) in both the CLI and the server. +newer CLI works on an older one). Override the config directory with +`$OMNIGRAPH_HOME`. #### Scopes & profiles (RFC-011) @@ -131,7 +125,7 @@ sticky "current" mode. `--target`, `--cluster-graph`, and the positional-`http(s)://`→remote dispatch have been **removed** (`--graph` is now the one graph selector across server and -cluster scopes); `omnigraph.yaml`'s `cli.graph` default still works and an +cluster scopes); operator `defaults`/`--profile` supply the no-flag scope and an explicit address always wins. #### Credentials keyed by server name @@ -164,8 +158,7 @@ aliases: `POST /graphs/spike/queries/weekly_triage` with the keyed credential. Aliases live in their own `alias` namespace (RFC-011 Decision 4), so an alias can never shadow — or be shadowed by — a built-in verb. (The old -`--alias ` flag on `query`/`mutate` was removed; legacy `omnigraph.yaml` -`aliases:` no longer have a CLI entry point.) +`--alias ` flag on `query`/`mutate` was removed.) A remote command whose URL prefix-matches an operator server's `url` (the `gh` host model — no flags needed) resolves its token through: @@ -174,61 +167,10 @@ A remote command whose URL prefix-matches an operator server's `url` (the |---|---| | 1 | `OMNIGRAPH_TOKEN_` env (`prod` → `OMNIGRAPH_TOKEN_PROD`) | | 2 | `[]` section in `~/.omnigraph/credentials` | -| 3 | the legacy chain unchanged (`bearer_token_env` → `OMNIGRAPH_BEARER_TOKEN` → `auth.env_file`) | +| 3 | the default `OMNIGRAPH_BEARER_TOKEN` env | -A token is only ever sent to the server it is keyed to: URLs matching no -operator server use the legacy chain alone. - -## `omnigraph.yaml` schema (legacy combined file) - -> **Deprecated.** Loading this file prints a per-key notice -> naming each present key's new home (suppress in CI with -> `OMNIGRAPH_SUPPRESS_YAML_DEPRECATION=1`); `omnigraph config migrate` -> produces the split. The file keeps working through the deprecation -> window. Migrated teams can set `OMNIGRAPH_NO_LEGACY_CONFIG=1` to turn -> any legacy-file load into a hard error (regression guard; the file's -> absence is always fine). - -```yaml -project: { name } -graphs: - : - uri: - bearer_token_env: - queries: # per-graph stored-query registry (server-role; multi-graph mode) - : # key MUST equal the `query ` symbol inside the .gq - file: # relative to this config's directory - mcp: - expose: true # default true: listed in the MCP catalog (GET /queries); set false to hide (still HTTP-callable) - tool_name: # optional MCP tool-name override (defaults to ; - # must be unique across exposed queries) -server: - graph: - bind: -cli: - graph: - branch: - output_format: json|jsonl|csv|kv|table - table_max_column_width: 80 - table_cell_layout: truncate|wrap -query: - roots: [, …] # search path for .gq files -auth: - env_file: .env.omni -aliases: # legacy file-aliases — parsed but no longer - : # reachable from the CLI (RFC-011 D4 removed - command: read|change|query|mutate # the `--alias` flag). Use operator - query: # aliases (`~/.omnigraph/config.yaml` - name: # `aliases:`) via `omnigraph alias `. - args: [, …] - graph: - branch: - format: -queries: # top-level registry — applies only to a bare-URI (anonymous) graph; a graph served by name uses its `graphs..queries`. Mirrors top-level `policy`. - : { file: } # mcp.expose defaults to true -policy: - file: policy.yaml -``` +A keyed token is only ever sent to the server it is keyed to: a URL matching no +operator server falls back to `OMNIGRAPH_BEARER_TOKEN` alone. ## Cluster config preview @@ -251,8 +193,8 @@ apply, refresh, and import acquire `__cluster/lock.json` by default and release it before returning. `cluster apply` executes only stored-query/policy catalog writes (content-addressed under `__cluster/resources/`) and requires an existing `state.json`; graph/schema changes are deferred with warnings, and -applied resources do not serve traffic — the server still boots from -`omnigraph.yaml`. `cluster status` reads state only and reports any existing +applied resources do not serve traffic until an `omnigraph-server --cluster +` restart picks them up. `cluster status` reads state only and reports any existing lock metadata. `force-unlock` removes a lock only when the supplied id exactly matches the lock file. `refresh` requires an existing `state.json`; `import` creates one only when it is missing. Both observe declared graphs read-only at @@ -271,7 +213,7 @@ embeddings, aliases, and bindings are reserved for later stages. See ## Param resolution -Precedence (high to low): explicit `--params` / `--params-file`, alias positional args, `omnigraph.yaml` defaults. JS-safe-integer handling is built in (`is_js_safe_integer_i64`, `JS_MAX_SAFE_INTEGER_U64`) so 64-bit ids round-trip safely through JSON clients. +Precedence (high to low): explicit `--params` / `--params-file`, alias positional args. JS-safe-integer handling is built in (`is_js_safe_integer_i64`, `JS_MAX_SAFE_INTEGER_U64`) so 64-bit ids round-trip safely through JSON clients. ## Bearer token resolution (CLI) diff --git a/docs/user/clusters/config.md b/docs/user/clusters/config.md index df2b236..d5016b3 100644 --- a/docs/user/clusters/config.md +++ b/docs/user/clusters/config.md @@ -12,8 +12,9 @@ that ledger, manually remove a held local state lock by exact lock id, and catalog writes, **graph creation** (a declared graph that does not exist yet is initialized by apply at the derived root), **schema updates** (soft drops only), and — behind an explicit, digest-bound **approval** — **graph -deletion**. It does not perform data-loss schema migrations, start servers, -or serve anything it applies: the server still boots from `omnigraph.yaml`. +deletion**. It does not perform data-loss schema migrations or start servers: +a separate `omnigraph-server --cluster ` serves the applied revision on +its next (re)start. ## Commands @@ -31,26 +32,24 @@ omnigraph cluster force-unlock --config company-brain --json `--config` points at a directory, not a file. The directory must contain `cluster.yaml`. When omitted, it defaults to the current directory. -## Relationship to `omnigraph.yaml` +## Relationship to `~/.omnigraph/config.yaml` -`cluster.yaml` does not replace `omnigraph.yaml`, and the two never describe -the same fact. `omnigraph.yaml` is the permanent **per-operator** layer (CLI -defaults, the operator's identity and credential references, graph targets -for data-plane commands); `cluster.yaml` is the shared desired state of a +`cluster.yaml` and the per-operator `~/.omnigraph/config.yaml` never describe +the same fact. The operator config is the permanent **per-operator** layer +(the operator's identity and credential references, named servers/clusters, +profiles, and CLI defaults); `cluster.yaml` is the shared desired state of a whole deployment, read only by the `cluster` commands via `--config`. The exact contract: -- **Cluster commands read `omnigraph.yaml` for exactly one thing**: the - `cli.actor` default used by `apply`/`approve` when `--as` is omitted — - operator identity is a per-operator fact. With `--as` present, no config - is read at all. Nothing else (its graph set, targets, bind, queries, - policies) ever influences a cluster command; a malformed `omnigraph.yaml` - breaks only the no-flag actor lookup, loudly. -- **A `--cluster` server reads `omnigraph.yaml` for nothing** — not even the - implicit current-directory search runs (mode-inference rule 0). Boot from - cluster state XOR `omnigraph.yaml`, never a merge. -- **The other direction is ergonomics, not coupling**: a per-operator +- **Cluster commands read the operator config for exactly one thing**: the + `operator.actor` default used by `apply`/`approve` when `--as` is omitted — + operator identity is a per-operator fact. With `--as` present, the operator + config is not needed. Nothing else in it influences a cluster command. +- **No legacy `omnigraph.yaml`**: the CLI does not read `omnigraph.yaml` at + all, and a `--cluster` server reads only the cluster catalog — boot is + cluster-only. +- **The other direction is ergonomics, not coupling**: per-operator data-plane commands address a cluster graph by its derived storage root (`company-brain/graphs/knowledge.omni`) with `--store ` — an ordinary local path, no special handling. @@ -234,12 +233,11 @@ Deletes remove the resource from state; their old payload blobs stay on disk (garbage collection is a later stage). Re-running a converged apply is a no-op: no state write, no revision change (`state_written: false`). -**Applied means serving — for deployments that opt in.** A server started -with `--cluster ` boots from the applied revision (see +**Applied means serving.** A server started with `--cluster ` boots from +the applied revision (see [Serving from the cluster](#serving-from-the-cluster-the-mode-switch)); it -picks up newly applied state on its next restart. Deployments still booting -from `omnigraph.yaml` are untouched: for them, applied means recorded in the -catalog, nothing more. +picks up newly applied state on its next restart. Until that restart, applied +means recorded in the catalog, nothing more. ### Graph creation diff --git a/docs/user/clusters/index.md b/docs/user/clusters/index.md index d5c744a..c59ff9d 100644 --- a/docs/user/clusters/index.md +++ b/docs/user/clusters/index.md @@ -117,7 +117,7 @@ omnigraph cluster apply --config company-brain --as andrew `--as ` attributes the run: it is recorded in recovery sidecars and audit entries and threaded into the engine's commit history. Set -`cli: { actor: }` in your per-operator `omnigraph.yaml` to make it the +`operator: { actor: }` in your `~/.omnigraph/config.yaml` to make it the default when `--as` is omitted (the flag always wins; `approve` requires one of the two). @@ -244,12 +244,12 @@ with an in-flight apply. - **CI-driven convergence**: `validate` and `plan --json` are read-only and safe in pipelines; gate `apply --as ci` on plan review. Approvals are the human step by design — keep `cluster approve` out of automation. -- **`omnigraph.yaml` still has a job**: per-operator settings — your - `cli.actor` default for `--as`, CLI defaults, credentials, and data-plane - ergonomics (address a cluster graph by its derived root like - `company-brain/graphs/knowledge.omni` with `--store` for loads). It just no - longer describes the deployment — a server boots from one source or the - other, never a merge of both. +- **`~/.omnigraph/config.yaml` is the per-operator config**: your + `operator.actor` default for `--as`, named servers/clusters, credentials, + profiles, and data-plane ergonomics (address a cluster graph by its derived + root like `company-brain/graphs/knowledge.omni` with `--store` for loads). The + cluster directory's `cluster.yaml` is the **sole deployment declaration** — the + server boots from the cluster only. ## 7. Maintaining a cluster graph diff --git a/docs/user/deployment.md b/docs/user/deployment.md index b3b810c..21b8087 100644 --- a/docs/user/deployment.md +++ b/docs/user/deployment.md @@ -13,13 +13,10 @@ Omnigraph supports two broad deployment shapes: The server binary and container image expose the same HTTP surface. -The server also has two **boot sources**: `omnigraph.yaml` (graph targets -declared in the per-operator config) or a **cluster directory** -(`omnigraph-server --cluster `), which serves the cluster control +The server has a single **boot source**: a **cluster directory** +(`omnigraph-server --cluster `), which serves the cluster control plane's applied revision — see [cluster-config.md](clusters/config.md#serving-from-the-cluster-the-mode-switch). -The two are exclusive per deployment; switching is a restart with a different -flag. ## Binary Deployment diff --git a/docs/user/operations/policy.md b/docs/user/operations/policy.md index ced1c60..c6096d0 100644 --- a/docs/user/operations/policy.md +++ b/docs/user/operations/policy.md @@ -20,7 +20,7 @@ Server-scoped action (v0.6.0+; binds to `Omnigraph::Server::"root"`): 10. `graph_list` — `GET /graphs` registry enumeration (multi-graph mode) -Server-scoped actions cannot use `branch_scope` or `target_branch_scope` — they operate on the registry, not on a graph's branches. A rule cannot mix server-scoped and per-graph actions; split into separate rules. (Runtime `graph_create` / `graph_delete` are reserved but not shipped in v0.6.0; operators add/remove graphs by editing `omnigraph.yaml` and restarting.) +Server-scoped actions cannot use `branch_scope` or `target_branch_scope` — they operate on the registry, not on a graph's branches. A rule cannot mix server-scoped and per-graph actions; split into separate rules. (Runtime `graph_create` / `graph_delete` over HTTP are reserved but not shipped; operators add/remove graphs by editing the cluster's `cluster.yaml`, running `omnigraph cluster apply`, and restarting the server.) ## Scope kinds @@ -28,38 +28,34 @@ Server-scoped actions cannot use `branch_scope` or `target_branch_scope` — the - `target_branch_scope` — applied to destination (`schema_apply`, branch ops, run ops) - `protected_branches` — named list with special rules; rule scopes are `any | protected | unprotected` -## Per-graph vs. server-level policy (multi-graph mode) +## Per-graph vs. server-level policy -In multi mode (`omnigraph.yaml` with a non-empty `graphs:` map), policy files attach at two levels: +A server boots from a cluster (`--cluster `), and the cluster's +`cluster.yaml` declares its policy bundles in a `policies:` section. Each bundle +names the scopes it `applies_to`: a graph id (per-graph rules — `read`, `change`, +`branch_*`, `schema_apply`) or the literal `cluster` (server-level rules — +`graph_list`). ```yaml -server: - policy: - file: server-policy.yaml # server-level: graph_list - -graphs: +# cluster.yaml +policies: + base: + file: base.policy.yaml + applies_to: [cluster, knowledge] # cluster-level + the `knowledge` graph alpha: - uri: s3://tenant-bucket/alpha - policy: - file: policies/alpha.yaml # per-graph: read, change, branch_*, schema_apply - beta: - uri: s3://tenant-bucket/beta - # no per-graph policy → no engine-layer Cedar enforcement on beta + file: policies/alpha.yaml + applies_to: [alpha] # per-graph: alpha only ``` -**Config follows graph identity, not server mode.** A graph served by **name** -(`--target ` or `server.graph`) uses its own `graphs..policy.file`, -exactly as in multi-graph mode. Top-level `policy.file` applies only to an -**anonymous** graph — one served by a bare `` with no `graphs:` entry. -Serving a **named** graph (single- or multi-graph mode) while top-level -`policy.file` (or `queries:`) is populated **refuses boot**, naming the block, -since the top-level value would otherwise be silently shadowed by the per-graph -block. Move per-graph rules to `graphs..policy.file` and `graph_list` -rules to `server.policy.file`. +A graph with no bundle bound to it has no engine-layer Cedar enforcement. Each +graph's HTTP request flows through its bound bundle; the management endpoint +(`GET /graphs`) flows through the `cluster`-scoped bundle. When no bundle binds +`cluster`, `GET /graphs` is denied in every runtime state, including +`--unauthenticated`; with bearer tokens configured it returns 403 after admission +control because `graph_list` is not a `read`-equivalent action. The operator must +bind a `cluster`-scoped bundle granting `graph_list` to expose `/graphs`. -Each graph's HTTP request flows through its own per-graph policy. The management endpoint (`GET /graphs`) flows through the server-level policy. When `server.policy.file` is unset, `GET /graphs` is denied in every runtime state, including `--unauthenticated`; with bearer tokens configured, it returns 403 after admission control because `graph_list` is not a `read`-equivalent action. The operator must explicitly authorize via `server-policy.yaml` to expose `/graphs`. - -Example server-level policy: +Example `cluster`-scoped bundle: ```yaml version: 1 @@ -72,38 +68,26 @@ rules: actions: [graph_list] ``` -## Configuration +Each per-graph rule may use at most one of `branch_scope` or +`target_branch_scope`. Server-scoped rules (`graph_list`) take neither — they +have no branch context. -`omnigraph.yaml`: +## Actor for direct-engine writes -```yaml -policy: - file: policy.yaml # Cedar rules + groups - tests: policy.tests.yaml # declarative test cases - -cli: - actor: act-andrew # default actor for CLI direct-engine writes -``` - -Each per-graph rule may use at most one of `branch_scope` or `target_branch_scope`. Server-scoped rules (`graph_list`) take neither — they have no branch context. - -`cli.actor` is the default actor identity for CLI direct-engine writes -when `policy.file` is configured. Override per-invocation with `--as -` (top-level flag) — `--as` wins, otherwise `cli.actor` is used, -otherwise no actor. With policy configured and neither set, the -engine-layer footgun guard intentionally denies the write (silent bypass -via "I forgot the actor" is exactly what the guard prevents). Remote -HTTP writes ignore both — they resolve their actor server-side from the -bearer token. +The default actor identity for CLI direct-engine (`--store`) writes is +`operator.actor` in `~/.omnigraph/config.yaml`. Override per-invocation with +`--as ` — `--as` wins, otherwise `operator.actor`, otherwise no actor. +Remote HTTP writes ignore both — they resolve their actor server-side from the +bearer token. (Direct-store access carries no Cedar policy under RFC-011; policy +lives in the cluster/server.) ## CLI -Policy tooling resolves its graph like server single-mode policy: `cli.graph` -wins, otherwise `server.graph` is used, otherwise the top-level `policy.file` -is validated/tested/explained as the anonymous policy. +Policy tooling reads a cluster's applied policy bundles: pass `--cluster `, +and `--graph ` to pick a graph's bundle when several apply. - `omnigraph policy validate` — parse + count actors, exit 1 on parse error. -- `omnigraph policy test` — run cases in `policy.tests.yaml`, exit 1 on any expectation mismatch. +- `omnigraph policy test --tests ` — run the declarative cases in `` against the selected bundle, exit 1 on any expectation mismatch. - `omnigraph policy explain --actor … --action … [--branch …] [--target-branch …]` — show decision and matched rule. - `omnigraph --as ` — set the actor for the duration of one invocation. Effective for `change`, `load` (and its deprecated `ingest` alias), `branch create|delete|merge`, and `schema apply` against a direct (`--store`) graph. **Rejected** on a served write (`--server`): the actor is bearer-token-resolved server-side, so `--as` can't set it there. @@ -132,7 +116,7 @@ reaches the authorization gate without a matching policy permit. |---|---|---|---| | **Open** | no | no | Every request is permitted. Refuses to start unless `--unauthenticated` or `OMNIGRAPH_UNAUTHENTICATED=1` is set — the operator must explicitly opt in. | | **DefaultDeny** | yes | no | Every authenticated request for an action other than `read` is rejected with HTTP 403. Closes the "tokens but forgot the policy file" trap — an operator who sets up auth and forgot to point at a policy file used to ship the illusion of protection. | -| **PolicyEnabled** | yes | yes | Authenticated requests that reach a configured policy engine are evaluated by Cedar. Server-scoped actions still require `server.policy.file`. | +| **PolicyEnabled** | yes | yes | Authenticated requests that reach a configured policy engine are evaluated by Cedar. Server-scoped actions still require a `cluster`-scoped policy bundle. | The server refuses to start for the "no tokens, no policy, no flag" cell and for "policy file, no tokens" — instead of silently shipping an open