mirror of
https://github.com/ModernRelay/omnigraph.git
synced 2026-06-18 02:24:27 +02:00
feat!: delete the legacy OmnigraphConfig + config migrate; finish the omnigraph.yaml docs sweep (#252)
* refactor(cli): own ReadOutputFormat/TableCellLayout in the CLI The two output-presentation enums lived in `omnigraph-server::config` and were re-exported for the CLI, even though the server never used them. Move both definitions into `omnigraph-cli/src/read_format.rs` (where the renderer already lives) and drop them from the server's public re-export. This is a step toward deleting the legacy `omnigraph-server::config` module entirely — a CLI presentation concern has no business in the server crate. No behavior change. The server keeps private copies in `config.rs` only for the soon-to-be-deleted legacy `CliDefaults`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli)!: remove the `config migrate` command and migrate.rs `config migrate` was the last CLI consumer of the legacy `omnigraph.yaml` (`OmnigraphConfig` + `load_config`). With the excision complete there is no legacy file to split, so the whole `omnigraph config` command group is removed along with `migrate.rs`. The `OmnigraphConfig` type, `load_config`, and the deprecation machinery are deleted next. - Remove `Command::Config` / `ConfigCommand` from the clap surface and the dispatch arm; drop `mod migrate;` and the now-unused `load_config` import. - Drop the `Command::Config` arms in `planes.rs`. - Delete the `config_migrate_splits_legacy_config` integration test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(server)!: delete the legacy OmnigraphConfig type and load_config With `config migrate` gone, nothing loads `omnigraph.yaml` anymore. Delete the entire `omnigraph-server::config` module: the `OmnigraphConfig` type and its sub-structs (`ProjectConfig`, `TargetConfig`, `CliDefaults`, `ServerDefaults`, `AuthDefaults`, `QueryDefaults`, `AliasConfig`, `AliasCommand`, `PolicySettings`, `QueryEntry`, `McpSettings`), `load_config`, and the RFC-008 deprecation machinery (`OMNIGRAPH_CONFIG`, `OMNIGRAPH_NO_LEGACY_CONFIG`, `OMNIGRAPH_SUPPRESS_YAML_DEPRECATION`, the deprecation map + warner). - `QueryRegistry::load` (the only `OmnigraphConfig`/`QueryEntry` consumer; its only caller was its own test) is removed — server boot and the CLI both build registries via `QueryRegistry::from_specs`. - `graph_resource_id_for_selection` (CLI-only) moves into the CLI (`helpers.rs`), with its unit test; the server no longer exports it. - Drop the already-dead `format_registry_load_errors` helper (config-adjacent). No behavior change — every deleted item was unreachable after the excision. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: purge the legacy omnigraph.yaml surface from the docs Finish the RFC-011 excision in the docs: the CLI no longer reads omnigraph.yaml and the server boots cluster-only, so every doc that described the legacy file as a live config is now wrong. - AGENTS.md: rewrite the HTTP-server line to cluster-only boot (drop the single-graph/flat-route and omnigraph.yaml-boot framing); rewrite the CLI two-surface-config passage (drop `config migrate`, the deprecation env vars, and "Never extend omnigraph.yaml"); fix the topic table + capability rows. - cli/reference.md: delete the entire "omnigraph.yaml schema (legacy combined file)" section and the `config migrate` row; re-home the `policy` row, the bearer-token chain, the actor/format/param-precedence references, and the `--config` mentions to the operator config + `--cluster`. - cli/index.md: rewrite the multi-graph-server + add-graph paragraphs to cluster (`--cluster` + `cluster apply`); fix the policy examples to `--cluster`; replace the `## Config` omnigraph.yaml example with the operator/cluster two-surface model. - operations/policy.md: rewrite per-graph-vs-server-level policy to the cluster `policies:`/`applies_to` model; re-home the actor + CLI tooling sections. - clusters/config.md, clusters/index.md, deployment.md: server boots from the cluster only; per-operator facts come from ~/.omnigraph/config.yaml. - architecture.md, testing.md: drop the stale omnigraph.yaml / deleted-test references. RFCs, design specs, and prior release notes are left as historical records. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
0bee746a31
commit
4601e5f4bf
20 changed files with 177 additions and 1950 deletions
10
AGENTS.md
10
AGENTS.md
|
|
@ -33,8 +33,8 @@ OmniGraph is a typed property-graph engine built as a coordination layer over ma
|
|||
- **Multi-modal querying**: vector ANN (`nearest`), full-text (`search`/`fuzzy`/`match_text`/`bm25`), Reciprocal Rank Fusion (`rrf`), and graph traversal (`Expand`, anti-join `not { … }`) in one runtime.
|
||||
- **Branches and commits across the whole graph**: Git-style — every successful publish appends to a commit DAG; merges are three-way at the row level.
|
||||
- **Atomic per-query writes**: `mutate_as` and `load` accumulate insert/update batches into an in-memory `MutationStaging.pending` per touched table; one `stage_*` + `commit_staged` per table runs at end-of-query, then `ManifestBatchPublisher::publish` commits the manifest atomically with per-table `expected_table_versions` CAS. A mid-query failure leaves Lance HEAD untouched on staged tables — no drift, no run state machine, no staging branches. Deletes still inline-commit; D₂ at parse time prevents inserts/updates and deletes from coexisting in one query.
|
||||
- **HTTP server**: Axum + utoipa OpenAPI, bearer auth (SHA-256 hashed, optional AWS Secrets Manager). Cedar policy enforcement is engine-wide — every `_as` writer calls `Omnigraph::enforce(action, scope, actor)`, so HTTP, CLI, and embedded SDK consumers all hit the same gate. **Two modes** (v0.6.0+): single-graph (legacy flat routes) and multi-graph (`/graphs/{graph_id}/...` cluster routes + read-only `GET /graphs` enumeration). Per-graph + server-level Cedar policies. Multi-graph mode boots from a cluster directory (`--cluster <dir | s3://…>`, RFC-005) or the legacy `omnigraph.yaml` `graphs:` map. Runtime add/remove (`POST /graphs`, `DELETE /graphs/{id}`) is not exposed — operators run `cluster apply` (or edit the legacy file) and restart.
|
||||
- **CLI** with two-surface config (RFC-008): the team-owned cluster directory (`cluster.yaml`) plus the per-operator `~/.omnigraph/config.yaml` (servers, credentials, actor, aliases). The legacy combined `omnigraph.yaml` still loads with per-key deprecation warnings — `config migrate` proposes the split, `OMNIGRAPH_NO_LEGACY_CONFIG=1` enforces strict mode. **Never extend `omnigraph.yaml`.** Multi-format output (json/jsonl/csv/kv/table).
|
||||
- **HTTP server**: Axum + utoipa OpenAPI, bearer auth (SHA-256 hashed, optional AWS Secrets Manager). Cedar policy enforcement is engine-wide — every `_as` writer calls `Omnigraph::enforce(action, scope, actor)`, so HTTP, CLI, and embedded SDK consumers all hit the same gate. **Cluster-only boot** (RFC-011): the server always boots from a cluster directory (`--cluster <dir | s3://…>`, RFC-005) and serves N graphs (N ≥ 1) under multi-graph routes (`/graphs/{graph_id}/...` + read-only `GET /graphs` enumeration); there are no single-graph flat routes and no positional-URI boot. Per-graph + server-level Cedar policies. Runtime add/remove (`POST /graphs`, `DELETE /graphs/{id}`) is not exposed — operators run `cluster apply` and restart.
|
||||
- **CLI** with two-surface config (RFC-007/008): the team-owned cluster directory (`cluster.yaml`) plus the per-operator `~/.omnigraph/config.yaml` (servers, clusters, credentials, actor, profiles, aliases, defaults). Graphs are addressed via `--store`/`--server`/`--cluster`/`--profile`/operator defaults (RFC-011). Multi-format output (json/jsonl/csv/kv/table).
|
||||
|
||||
Throughout the docs, capabilities are split into **L1 — Inherited from Lance** vs **L2 — Added by OmniGraph**.
|
||||
|
||||
|
|
@ -96,7 +96,7 @@ Full diagram and concurrency model: [docs/dev/architecture.md](docs/dev/architec
|
|||
| Cedar policy actions, scopes, CLI | [docs/user/operations/policy.md](docs/user/operations/policy.md) |
|
||||
| HTTP server endpoints, auth, error model, body limits | [docs/user/operations/server.md](docs/user/operations/server.md) |
|
||||
| CLI quick-start | [docs/user/cli/index.md](docs/user/cli/index.md) |
|
||||
| CLI command surface and config schemas (`~/.omnigraph/config.yaml`, legacy `omnigraph.yaml`) | [docs/user/cli/reference.md](docs/user/cli/reference.md) |
|
||||
| CLI command surface and config schema (`~/.omnigraph/config.yaml`) | [docs/user/cli/reference.md](docs/user/cli/reference.md) |
|
||||
| Audit / actor tracking | [docs/user/operations/audit.md](docs/user/operations/audit.md) |
|
||||
| Error taxonomy and result serialization | [docs/user/operations/errors.md](docs/user/operations/errors.md) |
|
||||
| Install (binary / Homebrew / source / channels) | [docs/user/install.md](docs/user/install.md) |
|
||||
|
|
@ -265,8 +265,8 @@ omnigraph policy explain --actor act-alice --action change --branch main
|
|||
| Three-way row-level merge | — | `OrderedTableCursor` + `StagedTableWriter`, structured `MergeConflictKind` |
|
||||
| Change feeds | — | `diff_between` / `diff_commits` with manifest fast path + ID streaming |
|
||||
| Cedar policy | — | Per-graph actions plus server-scoped actions (see [docs/user/operations/policy.md](docs/user/operations/policy.md) for the current list), branch / target_branch / protected scopes, validate/test/explain CLI. **Engine-wide enforcement** (MR-722): every `_as` writer (`apply_schema_as`, `mutate_as`, `load_as` — the deprecated `ingest_as` shims route through it — `branch_create_as` / `branch_create_from_as`, `branch_delete_as`, `branch_merge_as`) calls `Omnigraph::enforce(action, scope, actor)` — HTTP, CLI, embedded SDK all hit the same gate. |
|
||||
| HTTP server | — | Axum, OpenAPI via utoipa, bearer auth (SHA-256, AWS Secrets Manager option), `authorize_request` at the HTTP boundary (resolves bearer→actor, applies admission control), NDJSON streaming export, **multi-graph mode (v0.6.0+) with cluster routes + read-only `GET /graphs` enumeration + per-graph + server-level Cedar policies. Multi-graph boots from a cluster directory (`--cluster`) or the legacy `omnigraph.yaml`; add/remove graphs via `cluster apply` (or by editing the legacy file) and restarting.** |
|
||||
| CLI with config | — | two-surface config (team `cluster.yaml` dir + per-operator `~/.omnigraph/config.yaml`; legacy `omnigraph.yaml` deprecated per RFC-008), aliases, multi-format output (json/jsonl/csv/kv/table) |
|
||||
| HTTP server | — | Axum, OpenAPI via utoipa, bearer auth (SHA-256, AWS Secrets Manager option), `authorize_request` at the HTTP boundary (resolves bearer→actor, applies admission control), NDJSON streaming export, **cluster-only boot (RFC-011): always `--cluster <dir | s3://…>`, serving N graphs (N ≥ 1) under multi-graph routes + read-only `GET /graphs` enumeration + per-graph + server-level Cedar policies. Add/remove graphs via `cluster apply` and restart.** |
|
||||
| CLI with config | — | two-surface config (team `cluster.yaml` dir + per-operator `~/.omnigraph/config.yaml`), scope addressing (`--store`/`--server`/`--cluster`/`--profile`/defaults, RFC-011), aliases, multi-format output (json/jsonl/csv/kv/table) |
|
||||
| Audit / actor tracking | — | `_as` write APIs + actor map in commit graph |
|
||||
| Local RustFS bootstrap | — | `scripts/local-rustfs-bootstrap.sh` one-shot S3-backed dev environment |
|
||||
|
||||
|
|
|
|||
|
|
@ -352,12 +352,6 @@ pub(crate) enum Command {
|
|||
#[arg(long)]
|
||||
json: bool,
|
||||
},
|
||||
/// Legacy-config tooling (RFC-008): split omnigraph.yaml into its
|
||||
/// two destinations.
|
||||
Config {
|
||||
#[command(subcommand)]
|
||||
command: ConfigCommand,
|
||||
},
|
||||
/// Print the CLI version
|
||||
Version,
|
||||
}
|
||||
|
|
@ -661,20 +655,3 @@ impl CliLoadMode {
|
|||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Subcommand)]
|
||||
pub(crate) enum ConfigCommand {
|
||||
/// Propose (and with --write, apply) the RFC-008 split of a legacy
|
||||
/// omnigraph.yaml: team half -> a ready-to-review cluster.yaml,
|
||||
/// personal half -> ~/.omnigraph/config.yaml (key-level merge,
|
||||
/// existing entries always win). Touches nothing without --write.
|
||||
Migrate {
|
||||
/// Path to the legacy omnigraph.yaml (default: ./omnigraph.yaml)
|
||||
#[arg(long)]
|
||||
config: Option<PathBuf>,
|
||||
/// Apply the split instead of only printing it
|
||||
#[arg(long)]
|
||||
write: bool,
|
||||
#[arg(long)]
|
||||
json: bool,
|
||||
},
|
||||
}
|
||||
|
|
|
|||
|
|
@ -119,6 +119,16 @@ pub(crate) fn bearer_token_from_env(var_name: &str) -> Option<String> {
|
|||
normalize_bearer_token(std::env::var(var_name).ok())
|
||||
}
|
||||
|
||||
/// The Cedar resource id for a graph selection: the explicit graph name when one
|
||||
/// is given, else the normalized URI (the anonymous fallback). Used by the
|
||||
/// `policy` tooling to address a graph's bundle.
|
||||
pub(crate) fn graph_resource_id_for_selection(
|
||||
selected_graph: Option<&str>,
|
||||
normalized_uri: &str,
|
||||
) -> String {
|
||||
selected_graph.unwrap_or(normalized_uri).to_string()
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub(crate) struct ResolvedCliGraph {
|
||||
pub(crate) uri: String,
|
||||
|
|
@ -991,6 +1001,18 @@ pub(crate) fn rewrite_deprecated_argv(args: Vec<OsString>) -> Vec<OsString> {
|
|||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn graph_resource_id_for_selection_uses_name_or_anonymous_uri() {
|
||||
assert_eq!(
|
||||
graph_resource_id_for_selection(Some("local"), "/tmp/graph.omni"),
|
||||
"local"
|
||||
);
|
||||
assert_eq!(
|
||||
graph_resource_id_for_selection(None, "/tmp/graph.omni"),
|
||||
"/tmp/graph.omni"
|
||||
);
|
||||
}
|
||||
|
||||
// RFC-011 Decision 9: locality classifier for the destructive-confirm gate.
|
||||
#[test]
|
||||
fn uri_is_local_truth_table() {
|
||||
|
|
|
|||
|
|
@ -24,8 +24,7 @@ use omnigraph_api_types::{
|
|||
};
|
||||
use omnigraph_server::queries::{QueryRegistry, check};
|
||||
use omnigraph_server::{
|
||||
PolicyAction, PolicyDecision, PolicyEngine, PolicyRequest,
|
||||
PolicyTestConfig, ReadOutputFormat, graph_resource_id_for_selection, load_config,
|
||||
PolicyAction, PolicyDecision, PolicyEngine, PolicyRequest, PolicyTestConfig,
|
||||
};
|
||||
use reqwest::Method;
|
||||
use reqwest::header::AUTHORIZATION;
|
||||
|
|
@ -34,12 +33,11 @@ use serde::de::DeserializeOwned;
|
|||
use serde_json::Value;
|
||||
|
||||
mod embed;
|
||||
mod migrate;
|
||||
mod operator;
|
||||
mod read_format;
|
||||
|
||||
use embed::{EmbedArgs, EmbedOutput, execute_embed};
|
||||
use read_format::{ReadRenderOptions, render_read};
|
||||
use read_format::{ReadOutputFormat, ReadRenderOptions, render_read};
|
||||
|
||||
mod cli;
|
||||
mod client;
|
||||
|
|
@ -73,42 +71,6 @@ async fn main() -> Result<()> {
|
|||
// before any per-command dispatch.
|
||||
planes::guard_addressing(&cli)?;
|
||||
match cli.command {
|
||||
Command::Config { command } => match command {
|
||||
ConfigCommand::Migrate { config, write, json } => {
|
||||
let path = migrate::legacy_config_path(config.as_ref());
|
||||
if !path.exists() {
|
||||
bail!(
|
||||
"no legacy config at '{}' — nothing to migrate",
|
||||
path.display()
|
||||
);
|
||||
}
|
||||
let legacy = load_config(Some(&path))?;
|
||||
let report = migrate::build_report(&legacy, &path);
|
||||
if write {
|
||||
let legacy_dir = path
|
||||
.parent()
|
||||
.filter(|parent| !parent.as_os_str().is_empty())
|
||||
.unwrap_or(std::path::Path::new("."))
|
||||
.to_path_buf();
|
||||
let written = migrate::apply_report(&report, &legacy_dir)?;
|
||||
if json {
|
||||
print_json(&serde_json::json!({
|
||||
"report": report,
|
||||
"written": written,
|
||||
}))?;
|
||||
} else {
|
||||
print!("{}", migrate::render_report(&report));
|
||||
for line in written {
|
||||
println!("wrote: {line}");
|
||||
}
|
||||
}
|
||||
} else if json {
|
||||
print_json(&report)?;
|
||||
} else {
|
||||
print!("{}", migrate::render_report(&report));
|
||||
}
|
||||
}
|
||||
},
|
||||
Command::Login { name, token, json } => {
|
||||
let token = match token {
|
||||
Some(token) => token,
|
||||
|
|
|
|||
|
|
@ -1,408 +0,0 @@
|
|||
//! `omnigraph config migrate` (RFC-008 stage 2): split a legacy
|
||||
//! `omnigraph.yaml` into its two destinations — the team half as a
|
||||
//! ready-to-review `cluster.yaml` proposal, the personal half merged into
|
||||
//! `~/.omnigraph/config.yaml` — and name what's obsolete. The command is
|
||||
//! the completeness test of RFC-008's migration map: any key it cannot
|
||||
//! place is a bug in the RFC.
|
||||
//!
|
||||
//! Touches nothing without `--write`. Referenced `.gq`/policy files are
|
||||
//! never moved; manual steps are printed instead.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use color_eyre::Result;
|
||||
use color_eyre::eyre::eyre;
|
||||
use omnigraph_server::OmnigraphConfig;
|
||||
use serde::Serialize;
|
||||
|
||||
use crate::operator;
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub(crate) struct MigrateReport {
|
||||
pub(crate) source: String,
|
||||
/// The ready-to-review cluster.yaml text (None when the legacy file
|
||||
/// declares nothing team-shaped).
|
||||
pub(crate) cluster_yaml: Option<String>,
|
||||
/// Operator keys to merge: dotted key -> YAML value text.
|
||||
pub(crate) operator_merge: BTreeMap<String, String>,
|
||||
/// Keys with no destination, and why.
|
||||
pub(crate) dropped: Vec<DroppedKey>,
|
||||
/// Steps the command will not do for you.
|
||||
pub(crate) manual_steps: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub(crate) struct DroppedKey {
|
||||
pub(crate) key: String,
|
||||
pub(crate) reason: String,
|
||||
}
|
||||
|
||||
/// Classify a parsed legacy config into the report. Pure — no I/O.
|
||||
pub(crate) fn build_report(config: &OmnigraphConfig, source: &Path) -> MigrateReport {
|
||||
let mut dropped = Vec::new();
|
||||
let mut manual_steps = Vec::new();
|
||||
let mut operator_merge: BTreeMap<String, String> = BTreeMap::new();
|
||||
|
||||
// ---- personal half ----
|
||||
if let Some(actor) = &config.cli.actor {
|
||||
operator_merge.insert("operator.actor".into(), actor.clone());
|
||||
}
|
||||
if let Some(format) = config.cli.output_format {
|
||||
operator_merge.insert(
|
||||
"defaults.output".into(),
|
||||
serde_yaml::to_string(&format).unwrap_or_default().trim().to_string(),
|
||||
);
|
||||
}
|
||||
if let Some(width) = config.cli.table_max_column_width {
|
||||
operator_merge.insert("defaults.table_max_column_width".into(), width.to_string());
|
||||
}
|
||||
if let Some(layout) = config.cli.table_cell_layout {
|
||||
operator_merge.insert(
|
||||
"defaults.table_cell_layout".into(),
|
||||
serde_yaml::to_string(&layout).unwrap_or_default().trim().to_string(),
|
||||
);
|
||||
}
|
||||
if config.cli.graph.is_some() {
|
||||
dropped.push(DroppedKey {
|
||||
key: "cli.graph".into(),
|
||||
reason: "address graphs explicitly via --store/--server, or set defaults.default_graph in the operator config".into(),
|
||||
});
|
||||
}
|
||||
if config.cli.branch.is_some() {
|
||||
dropped.push(DroppedKey {
|
||||
key: "cli.branch".into(),
|
||||
reason: "pass --branch explicitly".into(),
|
||||
});
|
||||
}
|
||||
|
||||
// Remote graphs with a token env become operator servers (the keyed
|
||||
// chain replaces invented env-var names).
|
||||
for (name, target) in &config.graphs {
|
||||
if target.uri.starts_with("http://") || target.uri.starts_with("https://") {
|
||||
operator_merge.insert(format!("servers.{name}.url"), target.uri.clone());
|
||||
if target.bearer_token_env.is_some() {
|
||||
manual_steps.push(format!(
|
||||
"store the '{name}' token in the keyed chain: echo $TOKEN | omnigraph login {name} (replaces bearer_token_env)"
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
if config.auth.env_file.is_some() {
|
||||
manual_steps.push(
|
||||
"auth.env_file keeps working during the window; prefer `omnigraph login <server>` per server going forward".into(),
|
||||
);
|
||||
}
|
||||
|
||||
// Legacy aliases split: content -> catalog stored query, binding ->
|
||||
// operator alias referencing the name.
|
||||
for (name, alias) in &config.aliases {
|
||||
let query_name = alias.name.clone().unwrap_or_else(|| name.clone());
|
||||
operator_merge.insert(
|
||||
format!("aliases.{name}"),
|
||||
format!(
|
||||
"{{ server: TODO-server-name, graph: {}, query: {query_name}, args: [{}] }}",
|
||||
alias.graph.as_deref().unwrap_or("TODO-graph-id"),
|
||||
alias.args.join(", ")
|
||||
),
|
||||
);
|
||||
manual_steps.push(format!(
|
||||
"alias '{name}': move its query content ('{}') into the cluster checkout's queries/ so '{query_name}' becomes a catalog stored query",
|
||||
alias.query
|
||||
));
|
||||
}
|
||||
|
||||
// ---- team half ----
|
||||
let has_team_content = !config.graphs.is_empty()
|
||||
|| !config.queries.is_empty()
|
||||
|| config.policy.file.is_some()
|
||||
|| config.server.policy.file.is_some();
|
||||
let cluster_yaml = has_team_content.then(|| {
|
||||
let mut out = String::from("version: 1\n");
|
||||
if let Some(name) = &config.project.name {
|
||||
out.push_str(&format!("metadata:\n name: {name}\n"));
|
||||
}
|
||||
out.push_str("# storage: s3://bucket/prefix # or omit: this folder is the root\n");
|
||||
if !config.graphs.is_empty() || !config.queries.is_empty() {
|
||||
out.push_str("graphs:\n");
|
||||
}
|
||||
// Single-graph top-level queries belong to a graph the legacy file
|
||||
// never named; propose one.
|
||||
if !config.queries.is_empty() && config.graphs.is_empty() {
|
||||
out.push_str(" default: # TODO: pick the graph id\n schema: # TODO: path to this graph's .pg schema\n queries: queries/\n");
|
||||
}
|
||||
for (name, target) in &config.graphs {
|
||||
out.push_str(&format!(" {name}:\n"));
|
||||
out.push_str(" schema: # TODO: path to this graph's .pg schema\n");
|
||||
if !target.queries.is_empty() {
|
||||
out.push_str(" queries: queries/ # move the .gq files here\n");
|
||||
}
|
||||
out.push_str(&format!(
|
||||
" # legacy root: {} — the cluster manages graph roots under its storage; run `omnigraph cluster import` after reviewing\n",
|
||||
target.uri
|
||||
));
|
||||
}
|
||||
let mut policies: Vec<(String, String, String)> = Vec::new();
|
||||
if let Some(file) = &config.policy.file {
|
||||
policies.push(("default".into(), file.clone(), "graph.<id> # TODO: bind".into()));
|
||||
}
|
||||
if let Some(file) = &config.server.policy.file {
|
||||
policies.push(("server".into(), file.clone(), "cluster".into()));
|
||||
}
|
||||
for (name, target) in &config.graphs {
|
||||
if let Some(file) = &target.policy.file {
|
||||
policies.push((name.clone(), file.clone(), format!("graph.{name}")));
|
||||
}
|
||||
}
|
||||
if !policies.is_empty() {
|
||||
out.push_str("policies:\n");
|
||||
for (name, file, binding) in policies {
|
||||
out.push_str(&format!(
|
||||
" {name}:\n file: {file}\n applies_to: [{binding}]\n"
|
||||
));
|
||||
}
|
||||
}
|
||||
out
|
||||
});
|
||||
|
||||
if !config.query.roots.is_empty() {
|
||||
dropped.push(DroppedKey {
|
||||
key: "query.roots".into(),
|
||||
reason: "obsolete — cluster query discovery (queries: <dir>) replaced it".into(),
|
||||
});
|
||||
}
|
||||
if config.server.bind.is_some() || config.server.graph.is_some() {
|
||||
dropped.push(DroppedKey {
|
||||
key: "server.bind / server.graph".into(),
|
||||
reason: "deployment runtime — pass --bind / target flags or env".into(),
|
||||
});
|
||||
}
|
||||
if config.project.name.is_some() && cluster_yaml.is_none() {
|
||||
dropped.push(DroppedKey {
|
||||
key: "project.name".into(),
|
||||
reason: "the cluster's metadata.name is the deployment label".into(),
|
||||
});
|
||||
}
|
||||
|
||||
MigrateReport {
|
||||
source: source.display().to_string(),
|
||||
cluster_yaml,
|
||||
operator_merge,
|
||||
dropped,
|
||||
manual_steps,
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn render_report(report: &MigrateReport) -> String {
|
||||
let mut out = format!("migration plan for {}\n", report.source);
|
||||
if let Some(cluster) = &report.cluster_yaml {
|
||||
out.push_str("\n== team half -> cluster.yaml (ready to review) ==\n");
|
||||
out.push_str(cluster);
|
||||
}
|
||||
if !report.operator_merge.is_empty() {
|
||||
out.push_str("\n== personal half -> ~/.omnigraph/config.yaml ==\n");
|
||||
for (key, value) in &report.operator_merge {
|
||||
out.push_str(&format!(" {key}: {value}\n"));
|
||||
}
|
||||
}
|
||||
if !report.dropped.is_empty() {
|
||||
out.push_str("\n== no destination ==\n");
|
||||
for dropped in &report.dropped {
|
||||
out.push_str(&format!(" {} — {}\n", dropped.key, dropped.reason));
|
||||
}
|
||||
}
|
||||
if !report.manual_steps.is_empty() {
|
||||
out.push_str("\n== manual steps ==\n");
|
||||
for step in &report.manual_steps {
|
||||
out.push_str(&format!(" - {step}\n"));
|
||||
}
|
||||
}
|
||||
out.push_str("\n(nothing written; pass --write to apply the operator merge and emit cluster.yaml)\n");
|
||||
out
|
||||
}
|
||||
|
||||
/// `--write`: merge the personal half into the operator config (key-level,
|
||||
/// existing entries always win; the prior file is backed up) and write the
|
||||
/// team half to cluster.yaml in the legacy config's directory (or
|
||||
/// cluster.yaml.proposed when one already exists).
|
||||
pub(crate) fn apply_report(report: &MigrateReport, legacy_dir: &Path) -> Result<Vec<String>> {
|
||||
let mut written = Vec::new();
|
||||
|
||||
if !report.operator_merge.is_empty() {
|
||||
let dir = operator::operator_dir()
|
||||
.ok_or_else(|| eyre!("no home directory resolvable for the operator config"))?;
|
||||
std::fs::create_dir_all(&dir)?;
|
||||
let path = dir.join(operator::OPERATOR_CONFIG_FILE);
|
||||
let existing_text = std::fs::read_to_string(&path).unwrap_or_default();
|
||||
let mut mapping: serde_yaml::Mapping = if existing_text.trim().is_empty() {
|
||||
serde_yaml::Mapping::new()
|
||||
} else {
|
||||
serde_yaml::from_str(&existing_text)
|
||||
.map_err(|err| eyre!("operator config '{}' does not parse: {err}", path.display()))?
|
||||
};
|
||||
let mut merged_any = false;
|
||||
for (dotted, value_text) in &report.operator_merge {
|
||||
if merge_dotted_if_absent(&mut mapping, dotted, value_text)? {
|
||||
merged_any = true;
|
||||
}
|
||||
}
|
||||
if merged_any {
|
||||
if !existing_text.is_empty() {
|
||||
let backup = path.with_extension("yaml.bak");
|
||||
std::fs::write(&backup, &existing_text)?;
|
||||
written.push(format!("backed up prior operator config to {}", backup.display()));
|
||||
}
|
||||
let rendered = serde_yaml::to_string(&mapping)?;
|
||||
let tmp = path.with_extension(format!("yaml.tmp.{}", std::process::id()));
|
||||
std::fs::write(&tmp, &rendered)?;
|
||||
std::fs::rename(&tmp, &path)?;
|
||||
written.push(format!("merged personal keys into {}", path.display()));
|
||||
} else {
|
||||
written.push("operator config already carries every personal key (nothing merged)".into());
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(cluster) = &report.cluster_yaml {
|
||||
let target = legacy_dir.join("cluster.yaml");
|
||||
let target = if target.exists() {
|
||||
legacy_dir.join("cluster.yaml.proposed")
|
||||
} else {
|
||||
target
|
||||
};
|
||||
std::fs::write(&target, cluster)?;
|
||||
written.push(format!("wrote team-half proposal to {}", target.display()));
|
||||
}
|
||||
|
||||
Ok(written)
|
||||
}
|
||||
|
||||
/// Set `a.b.c` in the mapping only when absent; returns whether it wrote.
|
||||
fn merge_dotted_if_absent(
|
||||
mapping: &mut serde_yaml::Mapping,
|
||||
dotted: &str,
|
||||
value_text: &str,
|
||||
) -> Result<bool> {
|
||||
let value: serde_yaml::Value =
|
||||
serde_yaml::from_str(value_text).unwrap_or(serde_yaml::Value::String(value_text.into()));
|
||||
let parts: Vec<&str> = dotted.split('.').collect();
|
||||
let mut current = mapping;
|
||||
for part in &parts[..parts.len() - 1] {
|
||||
let key = serde_yaml::Value::String((*part).into());
|
||||
let entry = current
|
||||
.entry(key)
|
||||
.or_insert_with(|| serde_yaml::Value::Mapping(serde_yaml::Mapping::new()));
|
||||
current = entry
|
||||
.as_mapping_mut()
|
||||
.ok_or_else(|| eyre!("operator config key '{dotted}' collides with a non-mapping"))?;
|
||||
}
|
||||
let leaf = serde_yaml::Value::String(parts[parts.len() - 1].into());
|
||||
if current.contains_key(&leaf) {
|
||||
return Ok(false);
|
||||
}
|
||||
current.insert(leaf, value);
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
pub(crate) fn legacy_config_path(explicit: Option<&PathBuf>) -> PathBuf {
|
||||
explicit.cloned().unwrap_or_else(|| PathBuf::from("omnigraph.yaml"))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use omnigraph_server::config::load_config;
|
||||
|
||||
fn full_legacy_fixture(dir: &Path) -> PathBuf {
|
||||
let path = dir.join("omnigraph.yaml");
|
||||
std::fs::write(
|
||||
&path,
|
||||
r#"
|
||||
project: { name: brain }
|
||||
graphs:
|
||||
prod:
|
||||
uri: https://graph.example.com
|
||||
bearer_token_env: PROD_TOKEN
|
||||
policy: { file: ./prod.policy.yaml }
|
||||
queries:
|
||||
find: { file: ./find.gq }
|
||||
local:
|
||||
uri: /tmp/local.omni
|
||||
server: { bind: "0.0.0.0:9999", policy: { file: ./server.policy.yaml } }
|
||||
auth: { env_file: .env.omni }
|
||||
cli:
|
||||
graph: prod
|
||||
branch: main
|
||||
actor: act-me
|
||||
output_format: json
|
||||
table_max_column_width: 40
|
||||
query: { roots: ["."] }
|
||||
aliases:
|
||||
triage: { command: query, query: ./triage.gq, name: weekly_triage, args: [since], graph: prod }
|
||||
policy: { file: ./top.policy.yaml }
|
||||
queries:
|
||||
top_q: { file: ./top.gq }
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
path
|
||||
}
|
||||
|
||||
/// The RFC-008 completeness contract: every top-level key of the
|
||||
/// legacy schema must appear in the report somewhere (team half,
|
||||
/// operator merge, dropped, or manual steps).
|
||||
#[test]
|
||||
fn every_legacy_key_is_classified() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let path = full_legacy_fixture(dir.path());
|
||||
let config = load_config(Some(&path)).unwrap();
|
||||
let report = build_report(&config, &path);
|
||||
let rendered = render_report(&report);
|
||||
|
||||
let serialized =
|
||||
serde_yaml::to_value(OmnigraphConfig::default()).expect("default serializes");
|
||||
for key in serialized.as_mapping().unwrap().keys() {
|
||||
let key = key.as_str().unwrap();
|
||||
assert!(
|
||||
rendered.contains(key)
|
||||
|| report.operator_merge.keys().any(|k| k.contains(key))
|
||||
|| matches!(key, "graphs" | "queries" | "policy" | "project")
|
||||
&& report.cluster_yaml.is_some(),
|
||||
"legacy key '{key}' is unclassified — fix the RFC-008 map: {rendered}"
|
||||
);
|
||||
}
|
||||
|
||||
// spot checks on each section
|
||||
assert_eq!(report.operator_merge["operator.actor"], "act-me");
|
||||
assert_eq!(report.operator_merge["defaults.output"], "json");
|
||||
assert_eq!(
|
||||
report.operator_merge["servers.prod.url"],
|
||||
"https://graph.example.com"
|
||||
);
|
||||
assert!(report.operator_merge["aliases.triage"].contains("query: weekly_triage"));
|
||||
let cluster = report.cluster_yaml.as_deref().unwrap();
|
||||
assert!(cluster.contains("version: 1"));
|
||||
assert!(cluster.contains("name: brain"));
|
||||
assert!(cluster.contains(" prod:"));
|
||||
assert!(cluster.contains("applies_to: [cluster]"));
|
||||
assert!(cluster.contains("applies_to: [graph.prod]"));
|
||||
assert!(report.dropped.iter().any(|d| d.key == "query.roots"));
|
||||
assert!(report.dropped.iter().any(|d| d.key.contains("server.bind")));
|
||||
assert!(
|
||||
report
|
||||
.manual_steps
|
||||
.iter()
|
||||
.any(|s| s.contains("omnigraph login prod"))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn merge_dotted_never_clobbers_existing() {
|
||||
let mut mapping: serde_yaml::Mapping =
|
||||
serde_yaml::from_str("operator:\n actor: keep-me\n").unwrap();
|
||||
assert!(!merge_dotted_if_absent(&mut mapping, "operator.actor", "new").unwrap());
|
||||
assert!(merge_dotted_if_absent(&mut mapping, "defaults.output", "json").unwrap());
|
||||
let text = serde_yaml::to_string(&mapping).unwrap();
|
||||
assert!(text.contains("keep-me") && !text.contains("new"));
|
||||
assert!(text.contains("output: json"));
|
||||
}
|
||||
}
|
||||
|
|
@ -21,7 +21,7 @@ use color_eyre::Result;
|
|||
use color_eyre::eyre::{bail, eyre};
|
||||
use serde::Deserialize;
|
||||
|
||||
use omnigraph_server::config::ReadOutputFormat;
|
||||
use crate::read_format::{ReadOutputFormat, TableCellLayout};
|
||||
|
||||
pub(crate) const OPERATOR_HOME_ENV: &str = "OMNIGRAPH_HOME";
|
||||
pub(crate) const OPERATOR_DIR: &str = ".omnigraph";
|
||||
|
|
@ -102,10 +102,9 @@ pub(crate) struct OperatorIdentity {
|
|||
pub(crate) struct OperatorDefaults {
|
||||
/// Default read output format, below every more-specific source.
|
||||
pub(crate) output: Option<ReadOutputFormat>,
|
||||
/// Table rendering preferences (below the legacy cli.table_* keys
|
||||
/// during the RFC-008 window).
|
||||
/// Table rendering preferences for `--format table`.
|
||||
pub(crate) table_max_column_width: Option<usize>,
|
||||
pub(crate) table_cell_layout: Option<omnigraph_server::config::TableCellLayout>,
|
||||
pub(crate) table_cell_layout: Option<TableCellLayout>,
|
||||
/// Default server scope (RFC-011): the everyday addressing when no
|
||||
/// `--profile` / primitive / legacy address is given. Names an entry
|
||||
/// under `servers:`. Mutually exclusive with `store` — a scope binds one
|
||||
|
|
|
|||
|
|
@ -132,7 +132,6 @@ pub(crate) fn command_plane(cmd: &Command) -> Plane {
|
|||
Command::Embed(_)
|
||||
| Command::Login { .. }
|
||||
| Command::Logout { .. }
|
||||
| Command::Config { .. }
|
||||
| Command::Version => Plane::Session,
|
||||
}
|
||||
}
|
||||
|
|
@ -144,7 +143,6 @@ pub(crate) fn command_label(cmd: &Command) -> &'static str {
|
|||
Command::Version => "version",
|
||||
Command::Login { .. } => "login",
|
||||
Command::Logout { .. } => "logout",
|
||||
Command::Config { .. } => "config",
|
||||
Command::Embed(_) => "embed",
|
||||
Command::Init { .. } => "init",
|
||||
Command::Load { .. } => "load",
|
||||
|
|
|
|||
|
|
@ -1,9 +1,31 @@
|
|||
use clap::ValueEnum;
|
||||
use color_eyre::eyre::Result;
|
||||
use omnigraph_server::ReadOutputFormat;
|
||||
use omnigraph_server::api::ReadOutput;
|
||||
use omnigraph_server::config::TableCellLayout;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::{Map, Value};
|
||||
|
||||
/// Output rendering format for read-shaped commands (`read`/`query`/`alias`).
|
||||
/// A CLI presentation concern — lives here, not in the server.
|
||||
#[derive(Debug, Clone, Copy, Default, Eq, PartialEq, Serialize, Deserialize, ValueEnum)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum ReadOutputFormat {
|
||||
#[default]
|
||||
Table,
|
||||
Kv,
|
||||
Csv,
|
||||
Jsonl,
|
||||
Json,
|
||||
}
|
||||
|
||||
/// How an over-wide table cell is laid out when rendering `--format table`.
|
||||
#[derive(Debug, Clone, Copy, Default, Eq, PartialEq, Serialize, Deserialize, ValueEnum)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum TableCellLayout {
|
||||
#[default]
|
||||
Truncate,
|
||||
Wrap,
|
||||
}
|
||||
|
||||
pub struct ReadRenderOptions {
|
||||
pub max_column_width: usize,
|
||||
pub cell_layout: TableCellLayout,
|
||||
|
|
|
|||
|
|
@ -562,74 +562,3 @@ fn graphs_list_against_local_uri_errors_with_remote_only_message() {
|
|||
);
|
||||
}
|
||||
|
||||
/// RFC-008 stage 2: `config migrate` proposes the split read-only, applies
|
||||
/// it with --write (operator merge never clobbers; cluster.yaml emitted),
|
||||
/// and a second --write is idempotent.
|
||||
#[test]
|
||||
fn config_migrate_splits_legacy_config() {
|
||||
let temp = tempdir().unwrap();
|
||||
fs::write(
|
||||
temp.path().join("omnigraph.yaml"),
|
||||
"graphs:\n prod:\n uri: https://graph.example.com\n bearer_token_env: PROD_TOKEN\ncli:\n actor: act-me\n output_format: json\npolicy:\n file: ./top.policy.yaml\n",
|
||||
)
|
||||
.unwrap();
|
||||
let operator_home = tempfile::tempdir().unwrap();
|
||||
fs::write(
|
||||
operator_home.path().join("config.yaml"),
|
||||
"operator:\n actor: act-existing\n",
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
// Read-only proposal: names both halves, writes nothing.
|
||||
let output = cli()
|
||||
.current_dir(temp.path())
|
||||
.env("OMNIGRAPH_HOME", operator_home.path())
|
||||
.env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1")
|
||||
.arg("config")
|
||||
.arg("migrate")
|
||||
.output()
|
||||
.unwrap();
|
||||
assert!(output.status.success(), "{output:?}");
|
||||
let stdout = String::from_utf8_lossy(&output.stdout);
|
||||
assert!(stdout.contains("team half -> cluster.yaml"), "{stdout}");
|
||||
assert!(stdout.contains("operator.actor: act-me"), "{stdout}");
|
||||
assert!(stdout.contains("omnigraph login prod"), "{stdout}");
|
||||
assert!(!temp.path().join("cluster.yaml").exists());
|
||||
|
||||
// --write: cluster.yaml lands; the existing operator actor is KEPT.
|
||||
let output = cli()
|
||||
.current_dir(temp.path())
|
||||
.env("OMNIGRAPH_HOME", operator_home.path())
|
||||
.env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1")
|
||||
.arg("config")
|
||||
.arg("migrate")
|
||||
.arg("--write")
|
||||
.output()
|
||||
.unwrap();
|
||||
assert!(output.status.success(), "{output:?}");
|
||||
let cluster = fs::read_to_string(temp.path().join("cluster.yaml")).unwrap();
|
||||
assert!(cluster.contains("version: 1") && cluster.contains(" prod:"), "{cluster}");
|
||||
let operator_text =
|
||||
fs::read_to_string(operator_home.path().join("config.yaml")).unwrap();
|
||||
assert!(operator_text.contains("act-existing"), "{operator_text}");
|
||||
assert!(!operator_text.contains("act-me"), "existing keys win: {operator_text}");
|
||||
assert!(operator_text.contains("output: json"), "{operator_text}");
|
||||
assert!(
|
||||
operator_text.contains("url: https://graph.example.com"),
|
||||
"{operator_text}"
|
||||
);
|
||||
|
||||
// Second --write: cluster.yaml exists -> proposal file, no clobber.
|
||||
let output = cli()
|
||||
.current_dir(temp.path())
|
||||
.env("OMNIGRAPH_HOME", operator_home.path())
|
||||
.env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1")
|
||||
.arg("config")
|
||||
.arg("migrate")
|
||||
.arg("--write")
|
||||
.output()
|
||||
.unwrap();
|
||||
assert!(output.status.success(), "{output:?}");
|
||||
assert!(temp.path().join("cluster.yaml.proposed").exists());
|
||||
}
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -5,7 +5,6 @@ pub use settings::{load_server_settings, classify_server_runtime_state, ServerRu
|
|||
use settings::*;
|
||||
use handlers::*;
|
||||
pub mod auth;
|
||||
pub mod config;
|
||||
pub mod graph_id;
|
||||
pub mod identity;
|
||||
pub mod policy;
|
||||
|
|
@ -46,11 +45,6 @@ use axum::response::{IntoResponse, Response};
|
|||
use axum::routing::{delete, get, post};
|
||||
use axum::{Json, Router};
|
||||
use color_eyre::eyre::{Result, WrapErr, bail, eyre};
|
||||
pub use config::{
|
||||
AliasCommand, AliasConfig, CliDefaults, DEFAULT_CONFIG_FILE, OmnigraphConfig, PolicySettings,
|
||||
ProjectConfig, QueryDefaults, ReadOutputFormat, ServerDefaults, TableCellLayout, TargetConfig,
|
||||
graph_resource_id_for_selection, load_config,
|
||||
};
|
||||
use futures::stream;
|
||||
use omnigraph::db::{Omnigraph, ReadTarget};
|
||||
use omnigraph::error::{ManifestConflictDetails, ManifestErrorKind, OmniError};
|
||||
|
|
@ -879,18 +873,6 @@ fn validate_and_attach(
|
|||
})
|
||||
}
|
||||
|
||||
/// Format every load error (parse / identity failure) into a multi-line
|
||||
/// boot-abort message.
|
||||
fn format_registry_load_errors(label: &str, errors: &[queries::LoadError]) -> String {
|
||||
let joined = errors
|
||||
.iter()
|
||||
.map(|e| e.to_string())
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n ");
|
||||
format!("graph '{label}': stored-query registry failed to load:\n {joined}")
|
||||
}
|
||||
|
||||
|
||||
pub fn build_app(state: AppState) -> Router {
|
||||
// The per-graph protected routes, identical in single + multi mode.
|
||||
// Two middleware layers wrap them (outer first, inner last):
|
||||
|
|
|
|||
|
|
@ -13,7 +13,6 @@
|
|||
//! Renaming either is a breaking change to callers, by design.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::fs;
|
||||
use std::sync::Arc;
|
||||
|
||||
use omnigraph_compiler::catalog::Catalog;
|
||||
|
|
@ -22,8 +21,6 @@ use omnigraph_compiler::query::parser::parse_query;
|
|||
use omnigraph_compiler::query::typecheck::typecheck_query_decl;
|
||||
use omnigraph_compiler::types::{PropType, ScalarType};
|
||||
|
||||
use crate::config::{OmnigraphConfig, QueryEntry};
|
||||
|
||||
/// One loaded stored query. `source` is the full `.gq` file text — the
|
||||
/// invocation handler hands it to `run_query` / `run_mutate` verbatim,
|
||||
/// which reuse the same parse/IR/exec path as the inline routes (no
|
||||
|
|
@ -68,8 +65,9 @@ pub struct QueryRegistry {
|
|||
by_name: BTreeMap<String, StoredQuery>,
|
||||
}
|
||||
|
||||
/// In-memory registry entry before file I/O. Used by [`QueryRegistry::load`]
|
||||
/// (after reading each `.gq` from disk) and directly by tests.
|
||||
/// In-memory registry spec: a query's name + already-read `.gq` source. The
|
||||
/// input to [`QueryRegistry::from_specs`] — built by the server's cluster boot
|
||||
/// and by the CLI's `queries` tooling from a cluster serving snapshot.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct RegistrySpec {
|
||||
pub name: String,
|
||||
|
|
@ -169,47 +167,6 @@ impl QueryRegistry {
|
|||
}
|
||||
}
|
||||
|
||||
/// Read each registry entry's `.gq` file from disk and build the
|
||||
/// registry. `entries` is either the top-level `queries` map (single
|
||||
/// mode) or a graph's `queries` map (multi mode); `config` resolves
|
||||
/// each entry's relative `file:` path against `base_dir`.
|
||||
pub fn load(
|
||||
config: &OmnigraphConfig,
|
||||
entries: &BTreeMap<String, QueryEntry>,
|
||||
) -> Result<Self, Vec<LoadError>> {
|
||||
let mut specs = Vec::with_capacity(entries.len());
|
||||
let mut errors = Vec::new();
|
||||
for (name, entry) in entries {
|
||||
let path = config.resolve_query_file(&entry.file);
|
||||
match fs::read_to_string(&path) {
|
||||
Ok(source) => specs.push(RegistrySpec {
|
||||
name: name.clone(),
|
||||
source,
|
||||
expose: entry.mcp.expose,
|
||||
tool_name: entry.mcp.tool_name.clone(),
|
||||
}),
|
||||
Err(err) => errors.push(LoadError {
|
||||
query: Some(name.clone()),
|
||||
message: format!("cannot read '{}': {err}", path.display()),
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
// Parse/identity/uniqueness-check the readable specs even when some
|
||||
// files failed to read, so every broken entry (I/O, parse, identity,
|
||||
// tool-name collision) surfaces in one pass rather than one per
|
||||
// restart. I/O errors come first (in `entries` key order), then the
|
||||
// spec errors. A non-empty `errors` always fails the load.
|
||||
match Self::from_specs(specs) {
|
||||
Ok(registry) if errors.is_empty() => Ok(registry),
|
||||
Ok(_) => Err(errors),
|
||||
Err(spec_errors) => {
|
||||
errors.extend(spec_errors);
|
||||
Err(errors)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn lookup(&self, name: &str) -> Option<&StoredQuery> {
|
||||
self.by_name.get(name)
|
||||
}
|
||||
|
|
@ -653,36 +610,4 @@ embedding: Vector(4)
|
|||
assert!(entry2.params.is_empty(), "no declared params → empty list");
|
||||
}
|
||||
|
||||
// --- load() error collection (file I/O + parse in one pass) ---
|
||||
|
||||
#[test]
|
||||
fn load_collects_io_and_parse_errors_in_one_pass() {
|
||||
use crate::config::load_config;
|
||||
let temp = tempfile::tempdir().unwrap();
|
||||
std::fs::write(
|
||||
temp.path().join("good.gq"),
|
||||
"query good() { match { $u: User } return { $u.name } }",
|
||||
)
|
||||
.unwrap();
|
||||
std::fs::write(temp.path().join("broken.gq"), "query broken( {{ not valid").unwrap();
|
||||
// `missing.gq` is deliberately not written (an I/O failure).
|
||||
std::fs::write(
|
||||
temp.path().join("omnigraph.yaml"),
|
||||
"queries:\n good:\n file: ./good.gq\n \
|
||||
missing:\n file: ./missing.gq\n broken:\n file: ./broken.gq\n",
|
||||
)
|
||||
.unwrap();
|
||||
let config = load_config(Some(&temp.path().join("omnigraph.yaml"))).unwrap();
|
||||
|
||||
let errors = QueryRegistry::load(&config, config.query_entries()).unwrap_err();
|
||||
let joined = errors.iter().map(|e| e.to_string()).collect::<Vec<_>>().join("\n");
|
||||
// Both the missing file AND the parse error surface in one pass —
|
||||
// the I/O failure must not mask the parse failure.
|
||||
assert!(joined.contains("missing"), "I/O error must surface: {joined}");
|
||||
assert!(
|
||||
joined.contains("broken") && joined.contains("parse error"),
|
||||
"the parse error in a readable file must surface in the same pass: {joined}"
|
||||
);
|
||||
assert!(!joined.contains("'good'"), "the valid entry is not an error: {joined}");
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Architecture
|
||||
|
||||
OmniGraph is a typed property-graph engine built as a coordination layer over many Lance datasets, with Git-style branches and commits across the whole graph, multi-modal querying (vector + FTS + BM25 + RRF + graph traversal) in one runtime, an HTTP server with Cedar policy, and a CLI driven by a single `omnigraph.yaml`.
|
||||
OmniGraph is a typed property-graph engine built as a coordination layer over many Lance datasets, with Git-style branches and commits across the whole graph, multi-modal querying (vector + FTS + BM25 + RRF + graph traversal) in one runtime, an HTTP server with Cedar policy, and a CLI driven by a per-operator `~/.omnigraph/config.yaml` plus team-owned cluster directories.
|
||||
|
||||
## Reading guide
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ This file is the always-on map of the test surface. **Consult it before every ta
|
|||
| Crate | Path | Style |
|
||||
|---|---|---|
|
||||
| `omnigraph` (engine) | `crates/omnigraph/tests/` | Integration tests (28 files), fixture-driven, share `tests/helpers/mod.rs` |
|
||||
| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | Per-area suites (post-modularization): `cli_cluster.rs` (cluster command surface + operator-actor cascade), `cli_cluster_e2e.rs` (spawned-binary lifecycle compositions — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `cli_data.rs` (load/read/change/branch/commit/export/snapshot/policy/embed/maintenance + operator format cascade), `cli_schema_config.rs` (init/config, schema plan/apply, RFC-008 deprecation warnings + `config migrate` + strict mode), `cli_queries.rs`, `parity_matrix.rs` (RFC-009 Phase 1: the embedded-vs-remote referee — every forked verb run against both arms with matched Cedar policy and the same actor, scrubbed-JSON + exit-code equality; divergences are pinned in its `KNOWN_DIVERGENCES` ledger, never silently repaired), `system_local.rs` (full-cycle cluster lifecycle with a spawned `--cluster` server, applied-policy enforcement over HTTP, keyed-credential auth, operator aliases), `system_remote.rs`; share `tests/support/mod.rs` (hermetic `OMNIGRAPH_HOME` by default) |
|
||||
| `omnigraph-cli` | `crates/omnigraph-cli/tests/` | Per-area suites (post-modularization): `cli_cluster.rs` (cluster command surface + operator-actor cascade), `cli_cluster_e2e.rs` (spawned-binary lifecycle compositions — lost-state re-import recovery, out-of-band drift, graph-root destruction, multi-graph mixed-disposition convergence), `cli_data.rs` (load/read/change/branch/commit/export/snapshot/policy/embed/maintenance + operator format cascade), `cli_schema_config.rs` (init/config, schema plan/apply), `cli_queries.rs`, `parity_matrix.rs` (RFC-009 Phase 1: the embedded-vs-remote referee — every forked verb run against both arms with matched Cedar policy and the same actor, scrubbed-JSON + exit-code equality; divergences are pinned in its `KNOWN_DIVERGENCES` ledger, never silently repaired), `system_local.rs` (full-cycle cluster lifecycle with a spawned `--cluster` server, applied-policy enforcement over HTTP, keyed-credential auth, operator aliases), `system_remote.rs`; share `tests/support/mod.rs` (hermetic `OMNIGRAPH_HOME` by default) |
|
||||
| `omnigraph-cluster` | mostly in-source `#[cfg(test)] mod tests`; `tests/failpoints.rs` (feature-gated); `tests/s3_cluster.rs` (bucket-gated full lifecycle on object storage) | Cluster config parser, local JSON state diff, state CAS/lock handling/recovery, read-only validate/plan/status plus explicit refresh/import graph observations, config-only apply (content-addressed payload publish, disposition gating, composite-digest convergence, idempotent re-apply), catalog payload verification (status read-only, refresh drift + self-heal), failpoint crash-mid-apply / CAS-race coverage, Stage 4A graph creation (create executor, recovery sidecars + sweep rows, create crash windows), Stage 4B schema apply (migration previews in plan, schema executor, schema-apply sweep classification, schema crash windows), Stage 4C gated deletes (digest-bound approvals, delete executor + tombstones, delete sweep rows, delete crash windows), and 5A policy binding metadata (applies_to in the applied revision, binding-change diffing + convergence, pre-5A backfill), and the 5B serving-snapshot read API (converged read, refusal rows) |
|
||||
| `omnigraph-server` | `crates/omnigraph-server/tests/` | Per-area suites (post-modularization): `auth_policy.rs`, `data_routes.rs`, `schema_routes.rs`, `stored_queries.rs`, `multi_graph.rs` (cluster-mode boot — converged serving, policy binding wiring, boot refusals — + the concurrent branch-ops matrix), `boot_settings.rs` (mode inference, PolicySource), `s3.rs` (bucket-gated: single-graph serving + config-free `--cluster s3://` boot), `openapi.rs` (OpenAPI drift / regeneration); share `tests/support/mod.rs` |
|
||||
| `omnigraph-compiler` | mostly in-source `#[cfg(test)] mod tests` | Parser, type-checker, IR lowering, lint |
|
||||
|
|
|
|||
|
|
@ -78,20 +78,26 @@ literal URL); a positional `http(s)://` URI is rejected. If the server requires
|
|||
auth, set its bearer token and `omnigraph login <server>` (or
|
||||
`OMNIGRAPH_BEARER_TOKEN`).
|
||||
|
||||
## Multi-graph servers (v0.6.0+)
|
||||
## Multi-graph servers
|
||||
|
||||
Against a multi-graph server (started with `--config omnigraph.yaml` referencing a non-empty `graphs:` map), use `omnigraph graphs list` to enumerate the registered graphs. The server must configure bearer tokens and `server.policy.file` with a rule that allows `graph_list`; `/graphs` is closed by default even when the server runs with `--unauthenticated`.
|
||||
A server boots from a cluster directory (`omnigraph-server --cluster <dir>`) and
|
||||
serves every graph the cluster declares. Use `omnigraph graphs list` to enumerate
|
||||
them. The cluster's server-level policy must allow `graph_list`; `/graphs` is
|
||||
closed by default even when the server runs with `--unauthenticated`.
|
||||
|
||||
```bash
|
||||
OMNIGRAPH_BEARER_TOKEN=admin-token \
|
||||
omnigraph graphs list --uri http://server.example.com --json
|
||||
omnigraph graphs list --server http://server.example.com --json
|
||||
```
|
||||
|
||||
For config-driven clients, set the remote graph's `bearer_token_env` to an environment variable containing a token whose actor is authorized by `server.policy.file`.
|
||||
For an operator-defined server, store its token with `omnigraph login <name>` (or
|
||||
`OMNIGRAPH_TOKEN_<NAME>`); the actor must be authorized by the cluster's
|
||||
server-level policy.
|
||||
|
||||
`list` rejects local URI targets — it's for remote multi-graph servers only.
|
||||
`list` rejects local (`--store`) targets — it's for remote multi-graph servers only.
|
||||
|
||||
Runtime add/remove is **not** in v0.6.0. To add a graph, stop the server, add a `graphs.<id>` entry to `omnigraph.yaml`, then restart. To remove, stop the server, delete the entry, restart.
|
||||
Runtime add/remove via API is not exposed. To add or remove a graph, edit the
|
||||
cluster's `cluster.yaml`, run `omnigraph cluster apply`, then restart the server.
|
||||
|
||||
Per-graph addressing: select a graph on a multi-graph server with `--graph`:
|
||||
|
||||
|
|
@ -107,9 +113,9 @@ omnigraph check --query queries.gq graph.omni --json
|
|||
|
||||
omnigraph schema plan --schema next.pg graph.omni --json
|
||||
omnigraph schema apply --schema next.pg graph.omni --json
|
||||
omnigraph policy validate --config omnigraph.yaml
|
||||
omnigraph policy test --config omnigraph.yaml
|
||||
omnigraph policy explain --config omnigraph.yaml --actor act-alice --action read --branch main
|
||||
omnigraph policy validate --cluster ./company-brain --graph knowledge
|
||||
omnigraph policy test --cluster ./company-brain --graph knowledge --tests policy.tests.yaml
|
||||
omnigraph policy explain --cluster ./company-brain --graph knowledge --actor act-alice --action read --branch main
|
||||
|
||||
omnigraph commit list graph.omni --json
|
||||
omnigraph commit show --uri graph.omni <commit-id> --json
|
||||
|
|
@ -123,34 +129,29 @@ also pass `--schema`.
|
|||
|
||||
## Config
|
||||
|
||||
`omnigraph.yaml` lets the CLI and server share named graphs, defaults, and
|
||||
query roots:
|
||||
Configuration has two surfaces with single owners (see the
|
||||
[CLI reference](reference.md#config-surfaces) for the full schema):
|
||||
|
||||
- **`~/.omnigraph/config.yaml`** — your personal operator config: default actor
|
||||
(`--as`), named servers + credentials, clusters, profiles, aliases, and
|
||||
default scope (`defaults.server` / `defaults.store` / `default_graph`). It
|
||||
decides *who you are* and *what you address by default*.
|
||||
- **`cluster.yaml`** (a team-owned cluster directory) — declares *what the system
|
||||
is*: graphs, schemas, stored queries, policies, and storage. A server boots
|
||||
from it (`--cluster <dir>`); see the [cluster guide](../clusters/index.md).
|
||||
|
||||
```yaml
|
||||
graphs:
|
||||
local:
|
||||
uri: demo.omni
|
||||
# ~/.omnigraph/config.yaml
|
||||
operator:
|
||||
actor: act-andrew
|
||||
servers:
|
||||
dev:
|
||||
uri: http://127.0.0.1:8080
|
||||
bearer_token_env: OMNIGRAPH_BEARER_TOKEN
|
||||
|
||||
cli:
|
||||
graph: local
|
||||
branch: main
|
||||
|
||||
query:
|
||||
roots:
|
||||
- queries
|
||||
- .
|
||||
url: http://127.0.0.1:8080
|
||||
defaults:
|
||||
server: dev
|
||||
default_graph: knowledge
|
||||
```
|
||||
|
||||
The config file can also define:
|
||||
|
||||
- server bind defaults
|
||||
- auth env files
|
||||
- query aliases for common read and change commands
|
||||
- `policy.file` for Cedar authorization rules
|
||||
|
||||
When policy is enabled, `schema apply` is authorized through the
|
||||
`schema_apply` action and is typically limited to admins on protected `main`.
|
||||
|
||||
|
|
@ -168,6 +169,6 @@ one-line warning to stderr and otherwise behave identically.
|
|||
| `omnigraph query lint` | `omnigraph lint` | Same flags. The argv-level shim rewrites `query lint` to `lint`. |
|
||||
| `omnigraph query check` | `omnigraph check` | `check` is a visible alias of `omnigraph lint`. |
|
||||
|
||||
The `command:` field in `aliases.<name>` in `omnigraph.yaml` accepts both
|
||||
`read` / `change` (legacy) and `query` / `mutate` (canonical); the two
|
||||
The `command:` field in `aliases.<name>` in `~/.omnigraph/config.yaml` accepts
|
||||
both `read` / `change` (legacy) and `query` / `mutate` (canonical); the two
|
||||
spellings are interchangeable on the wire via serde aliases.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# CLI Reference (`omnigraph`)
|
||||
|
||||
A reference for the `omnigraph` binary's command surface and `omnigraph.yaml` schema. For a quick-start guide, see [cli.md](index.md).
|
||||
A reference for the `omnigraph` binary's command surface and the per-operator `~/.omnigraph/config.yaml` schema. For a quick-start guide, see [cli.md](index.md).
|
||||
|
||||
Top-level command families and subcommands. Graph-targeting commands accept a positional `file://`/`s3://` URI, `--server <name|url>` (an operator-defined server from `~/.omnigraph/config.yaml` by name, or a literal `http(s)://` URL, optionally with `--graph <id>` for multi-graph servers; exclusive with a positional URI), `--store <uri>` (a single graph's storage directly), or `--profile <name>` / `$OMNIGRAPH_PROFILE` (a named scope bundle; see [Scopes & profiles](#scopes--profiles-rfc-011)); `cluster` commands use `--config <dir>`. A remote server is addressed only with `--server` — a positional `http(s)://` URI is rejected. **`query`/`mutate` are the exception**: their positional is a stored-query *name* (RFC-011 D3), not a graph URI, so they address the graph only via `--store`/`--server`/`--profile`/defaults.
|
||||
|
||||
|
|
@ -8,7 +8,7 @@ Top-level command families and subcommands. Graph-targeting commands accept a po
|
|||
|
||||
| Command | Purpose |
|
||||
|---|---|
|
||||
| `init` | `--schema <pg>` → initialize a graph (no longer scaffolds `omnigraph.yaml`; start cluster configs from the [cluster.md](../clusters/index.md) quick-start or `config migrate`) |
|
||||
| `init` | `--schema <pg>` → initialize a graph (start cluster configs from the [cluster.md](../clusters/index.md) quick-start) |
|
||||
| `load` | bulk load a branch, local or remote (`--mode overwrite\|append\|merge` is **required** — overwrite is destructive, so there is no default). Without `--from` the target branch must exist; `--from <base>` forks a missing `--branch` from `<base>` first |
|
||||
| `ingest` | deprecated alias of `load --from <base>` (defaults: `--from main --mode merge`); prints a one-line warning to stderr |
|
||||
| `query <name>` (alias: `read`) | run a read query. **Catalog lane** (default): `<name>` is a stored query invoked **by name** from the served catalog (served-only — address with `--server`/`--profile`; the verb asserts the query is a read). **Ad-hoc lane**: with `--query <path>` or `-e`/`--query-string <GQ>`, runs that source (the positional `<name>` then selects which query in it). No positional graph URI — address via `--store`/`--server`/`--profile`. `read` is the deprecated previous name (one-line stderr warning) |
|
||||
|
|
@ -20,13 +20,12 @@ Top-level command families and subcommands. Graph-targeting commands accept a po
|
|||
| `commit list \| show` | inspect commit graph |
|
||||
| `schema plan \| apply \| show (alias: get)` | migrations |
|
||||
| `lint` (alias: `check`) | offline / graph-backed query validation. Replaces `query lint` / `query check`, which are kept as deprecated argv-level shims that print a one-line warning and rewrite to `omnigraph lint` |
|
||||
| `config migrate` | propose (or `--write`: apply) the split of a legacy `omnigraph.yaml` — team half → ready-to-review `cluster.yaml`, personal half → `~/.omnigraph/config.yaml` (key-level merge, existing entries win), plus dropped-key reasons and manual steps |
|
||||
| `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | declarative cluster control plane. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`, annotates dispositions, and embeds real schema-migration previews; `apply` converges the cluster — stored-query/policy catalog writes (content-addressed under `__cluster/resources/`), graph creates, schema updates (soft drops only; `--as` records the actor), and graph deletes behind a digest-bound approval from `cluster approve <resource> --as <actor>` (`apply`/`approve` default the actor from the per-operator `omnigraph.yaml`'s `cli.actor` when `--as` is omitted; nothing else in that file affects cluster commands); what apply converges is what an `omnigraph-server --cluster <dir>` deployment serves on its next restart (`--cluster` is the server's only boot source — RFC-011 cluster-only); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id |
|
||||
| `cluster validate \| plan \| apply \| approve \| status \| refresh \| import \| force-unlock` | declarative cluster control plane. `validate` checks a local `cluster.yaml` folder and referenced schema/query/policy files; `plan` diffs it against local JSON state at `__cluster/state.json`, annotates dispositions, and embeds real schema-migration previews; `apply` converges the cluster — stored-query/policy catalog writes (content-addressed under `__cluster/resources/`), graph creates, schema updates (soft drops only; `--as` records the actor), and graph deletes behind a digest-bound approval from `cluster approve <resource> --as <actor>` (`apply`/`approve` default the actor from `~/.omnigraph/config.yaml`'s `operator.actor` when `--as` is omitted); what apply converges is what an `omnigraph-server --cluster <dir>` deployment serves on its next restart (`--cluster` is the server's only boot source — RFC-011 cluster-only); `status` reads the state ledger; `refresh`/`import` explicitly update local JSON state from read-only graph observations; `force-unlock <LOCK_ID>` manually removes a held local state lock by exact id |
|
||||
| `optimize` | non-destructive Lance compaction (skips tables with `Blob` columns or uncovered drift; `--json` reports `skipped`) |
|
||||
| `repair [--confirm] [--force]` | preview or explicitly publish uncovered manifest/head drift. `--confirm` heals verified maintenance drift and exits non-zero if suspicious/unverifiable drift is refused; `--force --confirm` publishes suspicious/unverifiable drift after operator review |
|
||||
| `cleanup --keep N --older-than 7d --confirm` | destructive version GC (`--confirm` to execute; also needs `--yes` against a non-local `s3://` target — see *Write diagnostics & destructive confirmation*) |
|
||||
| `embed` | offline JSONL embedding pipeline |
|
||||
| `policy validate \| test \| explain` | Cedar tooling. Selects `cli.graph`, else `server.graph`, else top-level `policy.file` |
|
||||
| `policy validate \| test \| explain` | Cedar tooling against a cluster's applied policies (`--cluster <dir>`; `--graph <id>` picks a graph's bundle when several apply). `test` takes `--tests <file>`; `explain` takes `--actor`/`--action`/`--branch`/`--target-branch` |
|
||||
| `version` / `-v` | print `omnigraph 0.3.x` |
|
||||
|
||||
## Command capabilities
|
||||
|
|
@ -69,21 +68,16 @@ Two config surfaces with single owners, plus a zero-config tier:
|
|||
| Operator config | one person | `~/.omnigraph/config.yaml` (override dir with `$OMNIGRAPH_HOME`) | who **I** am: identity, ergonomics |
|
||||
| Flags / env | per invocation | — | everything, explicitly |
|
||||
|
||||
`omnigraph.yaml` (below) is the legacy combined file — fully supported
|
||||
today, slated for staged deprecation; its keys' future homes are
|
||||
listed there.
|
||||
|
||||
### `~/.omnigraph/config.yaml` (operator)
|
||||
|
||||
```yaml
|
||||
operator:
|
||||
actor: act-andrew # default identity for every --as cascade:
|
||||
# --as > legacy cli.actor > operator.actor > none
|
||||
actor: act-andrew # default identity for the --as cascade: --as > operator.actor > none
|
||||
servers: # operator-owned endpoints; names key the credentials
|
||||
prod:
|
||||
url: https://graph.example.com # no tokens in this file, ever
|
||||
defaults:
|
||||
output: table # read format default, below --json/--format/alias/legacy
|
||||
output: table # read format default, below --json/--format/alias
|
||||
server: prod # the everyday SERVED scope when no address is given (RFC-011)
|
||||
# store: file:///data/dev.omni # OR a zero-flag LOCAL default (mutually
|
||||
# # exclusive with `server`); the local-dev
|
||||
|
|
@ -98,8 +92,8 @@ profiles: # named scope bundles (RFC-011); pick with --profile
|
|||
```
|
||||
|
||||
Absent file = empty layer. Unknown keys warn and load (a file written for a
|
||||
newer CLI works on an older one). `$OMNIGRAPH_CONFIG=<path>` stands in for
|
||||
`--config` (the flag wins) in both the CLI and the server.
|
||||
newer CLI works on an older one). Override the config directory with
|
||||
`$OMNIGRAPH_HOME`.
|
||||
|
||||
#### Scopes & profiles (RFC-011)
|
||||
|
||||
|
|
@ -131,7 +125,7 @@ sticky "current" mode.
|
|||
|
||||
`--target`, `--cluster-graph`, and the positional-`http(s)://`→remote dispatch
|
||||
have been **removed** (`--graph` is now the one graph selector across server and
|
||||
cluster scopes); `omnigraph.yaml`'s `cli.graph` default still works and an
|
||||
cluster scopes); operator `defaults`/`--profile` supply the no-flag scope and an
|
||||
explicit address always wins.
|
||||
|
||||
#### Credentials keyed by server name
|
||||
|
|
@ -164,8 +158,7 @@ aliases:
|
|||
`POST <server>/graphs/spike/queries/weekly_triage` with the keyed
|
||||
credential. Aliases live in their own `alias` namespace (RFC-011 Decision 4),
|
||||
so an alias can never shadow — or be shadowed by — a built-in verb. (The old
|
||||
`--alias <name>` flag on `query`/`mutate` was removed; legacy `omnigraph.yaml`
|
||||
`aliases:` no longer have a CLI entry point.)
|
||||
`--alias <name>` flag on `query`/`mutate` was removed.)
|
||||
|
||||
A remote command whose URL prefix-matches an operator server's `url` (the
|
||||
`gh` host model — no flags needed) resolves its token through:
|
||||
|
|
@ -174,61 +167,10 @@ A remote command whose URL prefix-matches an operator server's `url` (the
|
|||
|---|---|
|
||||
| 1 | `OMNIGRAPH_TOKEN_<NAME>` env (`prod` → `OMNIGRAPH_TOKEN_PROD`) |
|
||||
| 2 | `[<name>]` section in `~/.omnigraph/credentials` |
|
||||
| 3 | the legacy chain unchanged (`bearer_token_env` → `OMNIGRAPH_BEARER_TOKEN` → `auth.env_file`) |
|
||||
| 3 | the default `OMNIGRAPH_BEARER_TOKEN` env |
|
||||
|
||||
A token is only ever sent to the server it is keyed to: URLs matching no
|
||||
operator server use the legacy chain alone.
|
||||
|
||||
## `omnigraph.yaml` schema (legacy combined file)
|
||||
|
||||
> **Deprecated.** Loading this file prints a per-key notice
|
||||
> naming each present key's new home (suppress in CI with
|
||||
> `OMNIGRAPH_SUPPRESS_YAML_DEPRECATION=1`); `omnigraph config migrate`
|
||||
> produces the split. The file keeps working through the deprecation
|
||||
> window. Migrated teams can set `OMNIGRAPH_NO_LEGACY_CONFIG=1` to turn
|
||||
> any legacy-file load into a hard error (regression guard; the file's
|
||||
> absence is always fine).
|
||||
|
||||
```yaml
|
||||
project: { name }
|
||||
graphs:
|
||||
<name>:
|
||||
uri: <local|s3://|http(s)://>
|
||||
bearer_token_env: <ENV_NAME>
|
||||
queries: # per-graph stored-query registry (server-role; multi-graph mode)
|
||||
<query-name>: # key MUST equal the `query <name>` symbol inside the .gq
|
||||
file: <path-to-.gq> # relative to this config's directory
|
||||
mcp:
|
||||
expose: true # default true: listed in the MCP catalog (GET /queries); set false to hide (still HTTP-callable)
|
||||
tool_name: <name> # optional MCP tool-name override (defaults to <query-name>;
|
||||
# must be unique across exposed queries)
|
||||
server:
|
||||
graph: <name>
|
||||
bind: <ip:port>
|
||||
cli:
|
||||
graph: <name>
|
||||
branch: <name>
|
||||
output_format: json|jsonl|csv|kv|table
|
||||
table_max_column_width: 80
|
||||
table_cell_layout: truncate|wrap
|
||||
query:
|
||||
roots: [<dir>, …] # search path for .gq files
|
||||
auth:
|
||||
env_file: .env.omni
|
||||
aliases: # legacy file-aliases — parsed but no longer
|
||||
<alias>: # reachable from the CLI (RFC-011 D4 removed
|
||||
command: read|change|query|mutate # the `--alias` flag). Use operator
|
||||
query: <path-to-.gq> # aliases (`~/.omnigraph/config.yaml`
|
||||
name: <query-name> # `aliases:`) via `omnigraph alias <name>`.
|
||||
args: [<positional-name>, …]
|
||||
graph: <name>
|
||||
branch: <name>
|
||||
format: <output-format>
|
||||
queries: # top-level registry — applies only to a bare-URI (anonymous) graph; a graph served by name uses its `graphs.<id>.queries`. Mirrors top-level `policy`.
|
||||
<query-name>: { file: <path-to-.gq> } # mcp.expose defaults to true
|
||||
policy:
|
||||
file: policy.yaml
|
||||
```
|
||||
A keyed token is only ever sent to the server it is keyed to: a URL matching no
|
||||
operator server falls back to `OMNIGRAPH_BEARER_TOKEN` alone.
|
||||
|
||||
## Cluster config preview
|
||||
|
||||
|
|
@ -251,8 +193,8 @@ apply, refresh, and import acquire `__cluster/lock.json` by default and release
|
|||
it before returning. `cluster apply` executes only stored-query/policy catalog
|
||||
writes (content-addressed under `__cluster/resources/`) and requires an
|
||||
existing `state.json`; graph/schema changes are deferred with warnings, and
|
||||
applied resources do not serve traffic — the server still boots from
|
||||
`omnigraph.yaml`. `cluster status` reads state only and reports any existing
|
||||
applied resources do not serve traffic until an `omnigraph-server --cluster
|
||||
<dir>` restart picks them up. `cluster status` reads state only and reports any existing
|
||||
lock metadata. `force-unlock` removes a lock only when the supplied id exactly
|
||||
matches the lock file. `refresh` requires an existing `state.json`; `import`
|
||||
creates one only when it is missing. Both observe declared graphs read-only at
|
||||
|
|
@ -271,7 +213,7 @@ embeddings, aliases, and bindings are reserved for later stages. See
|
|||
|
||||
## Param resolution
|
||||
|
||||
Precedence (high to low): explicit `--params` / `--params-file`, alias positional args, `omnigraph.yaml` defaults. JS-safe-integer handling is built in (`is_js_safe_integer_i64`, `JS_MAX_SAFE_INTEGER_U64`) so 64-bit ids round-trip safely through JSON clients.
|
||||
Precedence (high to low): explicit `--params` / `--params-file`, alias positional args. JS-safe-integer handling is built in (`is_js_safe_integer_i64`, `JS_MAX_SAFE_INTEGER_U64`) so 64-bit ids round-trip safely through JSON clients.
|
||||
|
||||
## Bearer token resolution (CLI)
|
||||
|
||||
|
|
|
|||
|
|
@ -12,8 +12,9 @@ that ledger, manually remove a held local state lock by exact lock id, and
|
|||
catalog writes, **graph creation** (a declared graph that does not exist yet
|
||||
is initialized by apply at the derived root), **schema updates** (soft drops
|
||||
only), and — behind an explicit, digest-bound **approval** — **graph
|
||||
deletion**. It does not perform data-loss schema migrations, start servers,
|
||||
or serve anything it applies: the server still boots from `omnigraph.yaml`.
|
||||
deletion**. It does not perform data-loss schema migrations or start servers:
|
||||
a separate `omnigraph-server --cluster <dir>` serves the applied revision on
|
||||
its next (re)start.
|
||||
|
||||
## Commands
|
||||
|
||||
|
|
@ -31,26 +32,24 @@ omnigraph cluster force-unlock <LOCK_ID> --config company-brain --json
|
|||
`--config` points at a directory, not a file. The directory must contain
|
||||
`cluster.yaml`. When omitted, it defaults to the current directory.
|
||||
|
||||
## Relationship to `omnigraph.yaml`
|
||||
## Relationship to `~/.omnigraph/config.yaml`
|
||||
|
||||
`cluster.yaml` does not replace `omnigraph.yaml`, and the two never describe
|
||||
the same fact. `omnigraph.yaml` is the permanent **per-operator** layer (CLI
|
||||
defaults, the operator's identity and credential references, graph targets
|
||||
for data-plane commands); `cluster.yaml` is the shared desired state of a
|
||||
`cluster.yaml` and the per-operator `~/.omnigraph/config.yaml` never describe
|
||||
the same fact. The operator config is the permanent **per-operator** layer
|
||||
(the operator's identity and credential references, named servers/clusters,
|
||||
profiles, and CLI defaults); `cluster.yaml` is the shared desired state of a
|
||||
whole deployment, read only by the `cluster` commands via `--config`.
|
||||
|
||||
The exact contract:
|
||||
|
||||
- **Cluster commands read `omnigraph.yaml` for exactly one thing**: the
|
||||
`cli.actor` default used by `apply`/`approve` when `--as` is omitted —
|
||||
operator identity is a per-operator fact. With `--as` present, no config
|
||||
is read at all. Nothing else (its graph set, targets, bind, queries,
|
||||
policies) ever influences a cluster command; a malformed `omnigraph.yaml`
|
||||
breaks only the no-flag actor lookup, loudly.
|
||||
- **A `--cluster` server reads `omnigraph.yaml` for nothing** — not even the
|
||||
implicit current-directory search runs (mode-inference rule 0). Boot from
|
||||
cluster state XOR `omnigraph.yaml`, never a merge.
|
||||
- **The other direction is ergonomics, not coupling**: a per-operator
|
||||
- **Cluster commands read the operator config for exactly one thing**: the
|
||||
`operator.actor` default used by `apply`/`approve` when `--as` is omitted —
|
||||
operator identity is a per-operator fact. With `--as` present, the operator
|
||||
config is not needed. Nothing else in it influences a cluster command.
|
||||
- **No legacy `omnigraph.yaml`**: the CLI does not read `omnigraph.yaml` at
|
||||
all, and a `--cluster` server reads only the cluster catalog — boot is
|
||||
cluster-only.
|
||||
- **The other direction is ergonomics, not coupling**: per-operator
|
||||
data-plane commands address a cluster graph by its derived storage root
|
||||
(`company-brain/graphs/knowledge.omni`) with `--store <uri>` — an ordinary
|
||||
local path, no special handling.
|
||||
|
|
@ -234,12 +233,11 @@ Deletes remove the resource from state; their old payload blobs stay on disk
|
|||
(garbage collection is a later stage). Re-running a converged apply is a no-op:
|
||||
no state write, no revision change (`state_written: false`).
|
||||
|
||||
**Applied means serving — for deployments that opt in.** A server started
|
||||
with `--cluster <dir>` boots from the applied revision (see
|
||||
**Applied means serving.** A server started with `--cluster <dir>` boots from
|
||||
the applied revision (see
|
||||
[Serving from the cluster](#serving-from-the-cluster-the-mode-switch)); it
|
||||
picks up newly applied state on its next restart. Deployments still booting
|
||||
from `omnigraph.yaml` are untouched: for them, applied means recorded in the
|
||||
catalog, nothing more.
|
||||
picks up newly applied state on its next restart. Until that restart, applied
|
||||
means recorded in the catalog, nothing more.
|
||||
|
||||
### Graph creation
|
||||
|
||||
|
|
|
|||
|
|
@ -117,7 +117,7 @@ omnigraph cluster apply --config company-brain --as andrew
|
|||
|
||||
`--as <actor>` attributes the run: it is recorded in recovery sidecars and
|
||||
audit entries and threaded into the engine's commit history. Set
|
||||
`cli: { actor: <you> }` in your per-operator `omnigraph.yaml` to make it the
|
||||
`operator: { actor: <you> }` in your `~/.omnigraph/config.yaml` to make it the
|
||||
default when `--as` is omitted (the flag always wins; `approve` requires one
|
||||
of the two).
|
||||
|
||||
|
|
@ -244,12 +244,12 @@ with an in-flight apply.
|
|||
- **CI-driven convergence**: `validate` and `plan --json` are read-only and
|
||||
safe in pipelines; gate `apply --as ci` on plan review. Approvals are the
|
||||
human step by design — keep `cluster approve` out of automation.
|
||||
- **`omnigraph.yaml` still has a job**: per-operator settings — your
|
||||
`cli.actor` default for `--as`, CLI defaults, credentials, and data-plane
|
||||
ergonomics (address a cluster graph by its derived root like
|
||||
`company-brain/graphs/knowledge.omni` with `--store` for loads). It just no
|
||||
longer describes the deployment — a server boots from one source or the
|
||||
other, never a merge of both.
|
||||
- **`~/.omnigraph/config.yaml` is the per-operator config**: your
|
||||
`operator.actor` default for `--as`, named servers/clusters, credentials,
|
||||
profiles, and data-plane ergonomics (address a cluster graph by its derived
|
||||
root like `company-brain/graphs/knowledge.omni` with `--store` for loads). The
|
||||
cluster directory's `cluster.yaml` is the **sole deployment declaration** — the
|
||||
server boots from the cluster only.
|
||||
|
||||
## 7. Maintaining a cluster graph
|
||||
|
||||
|
|
|
|||
|
|
@ -13,13 +13,10 @@ Omnigraph supports two broad deployment shapes:
|
|||
|
||||
The server binary and container image expose the same HTTP surface.
|
||||
|
||||
The server also has two **boot sources**: `omnigraph.yaml` (graph targets
|
||||
declared in the per-operator config) or a **cluster directory**
|
||||
(`omnigraph-server --cluster <dir>`), which serves the cluster control
|
||||
The server has a single **boot source**: a **cluster directory**
|
||||
(`omnigraph-server --cluster <dir | s3://…>`), which serves the cluster control
|
||||
plane's applied revision — see
|
||||
[cluster-config.md](clusters/config.md#serving-from-the-cluster-the-mode-switch).
|
||||
The two are exclusive per deployment; switching is a restart with a different
|
||||
flag.
|
||||
|
||||
## Binary Deployment
|
||||
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@ Server-scoped action (v0.6.0+; binds to `Omnigraph::Server::"root"`):
|
|||
|
||||
10. `graph_list` — `GET /graphs` registry enumeration (multi-graph mode)
|
||||
|
||||
Server-scoped actions cannot use `branch_scope` or `target_branch_scope` — they operate on the registry, not on a graph's branches. A rule cannot mix server-scoped and per-graph actions; split into separate rules. (Runtime `graph_create` / `graph_delete` are reserved but not shipped in v0.6.0; operators add/remove graphs by editing `omnigraph.yaml` and restarting.)
|
||||
Server-scoped actions cannot use `branch_scope` or `target_branch_scope` — they operate on the registry, not on a graph's branches. A rule cannot mix server-scoped and per-graph actions; split into separate rules. (Runtime `graph_create` / `graph_delete` over HTTP are reserved but not shipped; operators add/remove graphs by editing the cluster's `cluster.yaml`, running `omnigraph cluster apply`, and restarting the server.)
|
||||
|
||||
## Scope kinds
|
||||
|
||||
|
|
@ -28,38 +28,34 @@ Server-scoped actions cannot use `branch_scope` or `target_branch_scope` — the
|
|||
- `target_branch_scope` — applied to destination (`schema_apply`, branch ops, run ops)
|
||||
- `protected_branches` — named list with special rules; rule scopes are `any | protected | unprotected`
|
||||
|
||||
## Per-graph vs. server-level policy (multi-graph mode)
|
||||
## Per-graph vs. server-level policy
|
||||
|
||||
In multi mode (`omnigraph.yaml` with a non-empty `graphs:` map), policy files attach at two levels:
|
||||
A server boots from a cluster (`--cluster <dir>`), and the cluster's
|
||||
`cluster.yaml` declares its policy bundles in a `policies:` section. Each bundle
|
||||
names the scopes it `applies_to`: a graph id (per-graph rules — `read`, `change`,
|
||||
`branch_*`, `schema_apply`) or the literal `cluster` (server-level rules —
|
||||
`graph_list`).
|
||||
|
||||
```yaml
|
||||
server:
|
||||
policy:
|
||||
file: server-policy.yaml # server-level: graph_list
|
||||
|
||||
graphs:
|
||||
# cluster.yaml
|
||||
policies:
|
||||
base:
|
||||
file: base.policy.yaml
|
||||
applies_to: [cluster, knowledge] # cluster-level + the `knowledge` graph
|
||||
alpha:
|
||||
uri: s3://tenant-bucket/alpha
|
||||
policy:
|
||||
file: policies/alpha.yaml # per-graph: read, change, branch_*, schema_apply
|
||||
beta:
|
||||
uri: s3://tenant-bucket/beta
|
||||
# no per-graph policy → no engine-layer Cedar enforcement on beta
|
||||
file: policies/alpha.yaml
|
||||
applies_to: [alpha] # per-graph: alpha only
|
||||
```
|
||||
|
||||
**Config follows graph identity, not server mode.** A graph served by **name**
|
||||
(`--target <name>` or `server.graph`) uses its own `graphs.<name>.policy.file`,
|
||||
exactly as in multi-graph mode. Top-level `policy.file` applies only to an
|
||||
**anonymous** graph — one served by a bare `<URI>` with no `graphs:` entry.
|
||||
Serving a **named** graph (single- or multi-graph mode) while top-level
|
||||
`policy.file` (or `queries:`) is populated **refuses boot**, naming the block,
|
||||
since the top-level value would otherwise be silently shadowed by the per-graph
|
||||
block. Move per-graph rules to `graphs.<graph_id>.policy.file` and `graph_list`
|
||||
rules to `server.policy.file`.
|
||||
A graph with no bundle bound to it has no engine-layer Cedar enforcement. Each
|
||||
graph's HTTP request flows through its bound bundle; the management endpoint
|
||||
(`GET /graphs`) flows through the `cluster`-scoped bundle. When no bundle binds
|
||||
`cluster`, `GET /graphs` is denied in every runtime state, including
|
||||
`--unauthenticated`; with bearer tokens configured it returns 403 after admission
|
||||
control because `graph_list` is not a `read`-equivalent action. The operator must
|
||||
bind a `cluster`-scoped bundle granting `graph_list` to expose `/graphs`.
|
||||
|
||||
Each graph's HTTP request flows through its own per-graph policy. The management endpoint (`GET /graphs`) flows through the server-level policy. When `server.policy.file` is unset, `GET /graphs` is denied in every runtime state, including `--unauthenticated`; with bearer tokens configured, it returns 403 after admission control because `graph_list` is not a `read`-equivalent action. The operator must explicitly authorize via `server-policy.yaml` to expose `/graphs`.
|
||||
|
||||
Example server-level policy:
|
||||
Example `cluster`-scoped bundle:
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
|
|
@ -72,38 +68,26 @@ rules:
|
|||
actions: [graph_list]
|
||||
```
|
||||
|
||||
## Configuration
|
||||
Each per-graph rule may use at most one of `branch_scope` or
|
||||
`target_branch_scope`. Server-scoped rules (`graph_list`) take neither — they
|
||||
have no branch context.
|
||||
|
||||
`omnigraph.yaml`:
|
||||
## Actor for direct-engine writes
|
||||
|
||||
```yaml
|
||||
policy:
|
||||
file: policy.yaml # Cedar rules + groups
|
||||
tests: policy.tests.yaml # declarative test cases
|
||||
|
||||
cli:
|
||||
actor: act-andrew # default actor for CLI direct-engine writes
|
||||
```
|
||||
|
||||
Each per-graph rule may use at most one of `branch_scope` or `target_branch_scope`. Server-scoped rules (`graph_list`) take neither — they have no branch context.
|
||||
|
||||
`cli.actor` is the default actor identity for CLI direct-engine writes
|
||||
when `policy.file` is configured. Override per-invocation with `--as
|
||||
<ACTOR>` (top-level flag) — `--as` wins, otherwise `cli.actor` is used,
|
||||
otherwise no actor. With policy configured and neither set, the
|
||||
engine-layer footgun guard intentionally denies the write (silent bypass
|
||||
via "I forgot the actor" is exactly what the guard prevents). Remote
|
||||
HTTP writes ignore both — they resolve their actor server-side from the
|
||||
bearer token.
|
||||
The default actor identity for CLI direct-engine (`--store`) writes is
|
||||
`operator.actor` in `~/.omnigraph/config.yaml`. Override per-invocation with
|
||||
`--as <ACTOR>` — `--as` wins, otherwise `operator.actor`, otherwise no actor.
|
||||
Remote HTTP writes ignore both — they resolve their actor server-side from the
|
||||
bearer token. (Direct-store access carries no Cedar policy under RFC-011; policy
|
||||
lives in the cluster/server.)
|
||||
|
||||
## CLI
|
||||
|
||||
Policy tooling resolves its graph like server single-mode policy: `cli.graph`
|
||||
wins, otherwise `server.graph` is used, otherwise the top-level `policy.file`
|
||||
is validated/tested/explained as the anonymous policy.
|
||||
Policy tooling reads a cluster's applied policy bundles: pass `--cluster <dir>`,
|
||||
and `--graph <id>` to pick a graph's bundle when several apply.
|
||||
|
||||
- `omnigraph policy validate` — parse + count actors, exit 1 on parse error.
|
||||
- `omnigraph policy test` — run cases in `policy.tests.yaml`, exit 1 on any expectation mismatch.
|
||||
- `omnigraph policy test --tests <file>` — run the declarative cases in `<file>` against the selected bundle, exit 1 on any expectation mismatch.
|
||||
- `omnigraph policy explain --actor … --action … [--branch …] [--target-branch …]` — show decision and matched rule.
|
||||
- `omnigraph --as <ACTOR> <subcommand>` — set the actor for the duration of one invocation. Effective for `change`, `load` (and its deprecated `ingest` alias), `branch create|delete|merge`, and `schema apply` against a direct (`--store`) graph. **Rejected** on a served write (`--server`): the actor is bearer-token-resolved server-side, so `--as` can't set it there.
|
||||
|
||||
|
|
@ -132,7 +116,7 @@ reaches the authorization gate without a matching policy permit.
|
|||
|---|---|---|---|
|
||||
| **Open** | no | no | Every request is permitted. Refuses to start unless `--unauthenticated` or `OMNIGRAPH_UNAUTHENTICATED=1` is set — the operator must explicitly opt in. |
|
||||
| **DefaultDeny** | yes | no | Every authenticated request for an action other than `read` is rejected with HTTP 403. Closes the "tokens but forgot the policy file" trap — an operator who sets up auth and forgot to point at a policy file used to ship the illusion of protection. |
|
||||
| **PolicyEnabled** | yes | yes | Authenticated requests that reach a configured policy engine are evaluated by Cedar. Server-scoped actions still require `server.policy.file`. |
|
||||
| **PolicyEnabled** | yes | yes | Authenticated requests that reach a configured policy engine are evaluated by Cedar. Server-scoped actions still require a `cluster`-scoped policy bundle. |
|
||||
|
||||
The server refuses to start for the "no tokens, no policy, no flag" cell
|
||||
and for "policy file, no tokens" — instead of silently shipping an open
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue