omnigraph/crates/omnigraph-cli/tests/cli_schema_config.rs

709 lines
23 KiB
Rust
Raw Normal View History

//! init/config scaffolding, schema plan/apply, graphs listing, version.
//! Moved verbatim from tests/cli.rs in the modularization.
use std::fs;
use lance::index::DatasetIndexExt;
use omnigraph::db::{Omnigraph, ReadTarget};
use serde_json::Value;
use tempfile::tempdir;
mod support;
use support::*;
#[test]
fn version_command_prints_current_cli_version() {
let output = output_success(cli().arg("version"));
let stdout = stdout_string(&output);
assert_eq!(
stdout.trim(),
format!("omnigraph {}", env!("CARGO_PKG_VERSION"))
);
}
feat(cli): plane-grouped --help + clap 4.6.1 (RFC-010 Slice 2) (#220) * chore(deps): bump clap to 4.6.1 Workspace constraint "4" → "4.6" so the resolver picks up the 4.6 line (a plain `cargo update` stayed on 4.5.x). clap 4.5.58 → 4.6.1 (clap_builder 4.6.0, clap_derive 4.6.1). Minor bump, no API breakage; the workspace builds and all CLI suites pass unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): group --help by plane (RFC-010 Slice 2) Slice 1 declared the planes (the command_plane table + the wrong-plane guard); this makes them visible in `--help`. clap can't print labeled heading rows between subcommand groups (verified against the source — help_heading is args-only, {subcommands} is one flat block), so per the chosen approach: cluster + legend. - Reorder the `Command` enum into plane bands (clap lists subcommands in declaration order): data (query, mutate, load, branch, snapshot, export, commit, schema, graphs) → storage/local-graph ops (init, optimize, repair, cleanup, lint, queries) → control (cluster) → session (policy, embed, login, logout, config, version). No magic display_order numbers — the source order IS the help order, with band comments for readers. The band placement matches `command_plane` (lint/queries are storage-plane: they reject --server), so the help grouping and the guard agree. - Add an `after_help` legend on `Cli` naming the planes. Written to describe the planes (not enumerate every command) so it doesn't drift. Help-polish (post-review): hide the deprecated `ingest` from the list (still a valid command); trim the long `login` and `--as` descriptions to one line each so the columns don't blow up. The behavioral source of truth for planes stays `planes::command_plane`; this ordering is its cosmetic counterpart. Test: `help_groups_commands_by_plane` pins the legend phrase + the cluster ordering (query < optimize < cluster). Doc: a line under cli-reference's *Command planes* section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): qualify mixed-plane commands in the --help legend Addresses the Greptile P2 on #220: the legend placed `schema` entirely in Data and `queries` entirely in Storage, but per `command_plane` the subcommands differ — `schema plan` is storage-plane (rejects --server) and `queries list` is session (no graph). A user reading the legend then running `schema plan --server` would hit a rejection contradicting it. The Commands list is one entry per top-level command (necessarily coarse), so the legend carries the nuance: `schema [plan: storage]` and `queries [list: session]`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 01:49:40 +03:00
#[test]
feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) (#237) * feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) User-facing CLI errors and --help now speak a single "capability" vocabulary — what a command needs — instead of the internal four-plane jargon. Behavior is unchanged: the --server/--graph allow set is identical (the served-graph capabilities `any` ∪ `served` = the old `Data` plane, since `graphs` was already allowed). Only error text and the --help legend change. - planes.rs: add `Capability { Any, Served, Direct, Control, Local }` derived from the existing exhaustive `command_plane` classifier (which stays as the drift guard) plus the one Data→Served refinement (`graphs`). `guard_addressing` now allows `--server`/`--graph` on `{Any, Served}` and rejects elsewhere with a capability-worded message. The mapping reflects *current* behavior (`queries list` → Local, `queries validate` → Direct); it converges to the RFC end-state table when later slices re-route those verbs. - scope.rs: `resolve_scope` takes `Capability` instead of `Plane`, so the whole addressing path speaks one vocabulary; call sites in client.rs (Any) and the 3 maintenance verbs in main.rs (Direct) updated. - helpers.rs: the storage-direct remote rejection reworded to "direct (storage-native) command". - cli.rs: the --help legend is now "COMMANDS BY CAPABILITY". - Tests: the 5 assertions pinning the old plane text updated; added planes.rs unit tests proving the allow set is exactly {Any, Served} (behavior-preservation), the per-verb mapping, and distinct capability phrases. Full omnigraph-cli suite: 225 green (222 + 3 new), zero behavior-test changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(cli): capability vocabulary in the CLI reference + maintenance addressing Rename the reference's "Command planes" section to "Command capabilities" (any/served/direct/control/local), reword the error examples, and update the maintenance doc's addressing note + its section cross-link to match Slice B. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 03:02:07 +03:00
fn help_groups_commands_by_capability() {
// RFC-010 Slice 2 / RFC-011 Slice B: `--help` clusters commands (declaration
// order in the Command enum) and explains the capability each needs in an
// after_help legend. Pinned lightly — the legend phrase + the cluster
// ordering — to avoid brittle full-text assertions on clap's help body.
feat(cli): plane-grouped --help + clap 4.6.1 (RFC-010 Slice 2) (#220) * chore(deps): bump clap to 4.6.1 Workspace constraint "4" → "4.6" so the resolver picks up the 4.6 line (a plain `cargo update` stayed on 4.5.x). clap 4.5.58 → 4.6.1 (clap_builder 4.6.0, clap_derive 4.6.1). Minor bump, no API breakage; the workspace builds and all CLI suites pass unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): group --help by plane (RFC-010 Slice 2) Slice 1 declared the planes (the command_plane table + the wrong-plane guard); this makes them visible in `--help`. clap can't print labeled heading rows between subcommand groups (verified against the source — help_heading is args-only, {subcommands} is one flat block), so per the chosen approach: cluster + legend. - Reorder the `Command` enum into plane bands (clap lists subcommands in declaration order): data (query, mutate, load, branch, snapshot, export, commit, schema, graphs) → storage/local-graph ops (init, optimize, repair, cleanup, lint, queries) → control (cluster) → session (policy, embed, login, logout, config, version). No magic display_order numbers — the source order IS the help order, with band comments for readers. The band placement matches `command_plane` (lint/queries are storage-plane: they reject --server), so the help grouping and the guard agree. - Add an `after_help` legend on `Cli` naming the planes. Written to describe the planes (not enumerate every command) so it doesn't drift. Help-polish (post-review): hide the deprecated `ingest` from the list (still a valid command); trim the long `login` and `--as` descriptions to one line each so the columns don't blow up. The behavioral source of truth for planes stays `planes::command_plane`; this ordering is its cosmetic counterpart. Test: `help_groups_commands_by_plane` pins the legend phrase + the cluster ordering (query < optimize < cluster). Doc: a line under cli-reference's *Command planes* section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): qualify mixed-plane commands in the --help legend Addresses the Greptile P2 on #220: the legend placed `schema` entirely in Data and `queries` entirely in Storage, but per `command_plane` the subcommands differ — `schema plan` is storage-plane (rejects --server) and `queries list` is session (no graph). A user reading the legend then running `schema plan --server` would hit a rejection contradicting it. The Commands list is one entry per top-level command (necessarily coarse), so the legend carries the nuance: `schema [plan: storage]` and `queries [list: session]`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 01:49:40 +03:00
let output = output_success(cli().arg("--help"));
let stdout = stdout_string(&output);
assert!(
feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) (#237) * feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) User-facing CLI errors and --help now speak a single "capability" vocabulary — what a command needs — instead of the internal four-plane jargon. Behavior is unchanged: the --server/--graph allow set is identical (the served-graph capabilities `any` ∪ `served` = the old `Data` plane, since `graphs` was already allowed). Only error text and the --help legend change. - planes.rs: add `Capability { Any, Served, Direct, Control, Local }` derived from the existing exhaustive `command_plane` classifier (which stays as the drift guard) plus the one Data→Served refinement (`graphs`). `guard_addressing` now allows `--server`/`--graph` on `{Any, Served}` and rejects elsewhere with a capability-worded message. The mapping reflects *current* behavior (`queries list` → Local, `queries validate` → Direct); it converges to the RFC end-state table when later slices re-route those verbs. - scope.rs: `resolve_scope` takes `Capability` instead of `Plane`, so the whole addressing path speaks one vocabulary; call sites in client.rs (Any) and the 3 maintenance verbs in main.rs (Direct) updated. - helpers.rs: the storage-direct remote rejection reworded to "direct (storage-native) command". - cli.rs: the --help legend is now "COMMANDS BY CAPABILITY". - Tests: the 5 assertions pinning the old plane text updated; added planes.rs unit tests proving the allow set is exactly {Any, Served} (behavior-preservation), the per-verb mapping, and distinct capability phrases. Full omnigraph-cli suite: 225 green (222 + 3 new), zero behavior-test changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(cli): capability vocabulary in the CLI reference + maintenance addressing Rename the reference's "Command planes" section to "Command capabilities" (any/served/direct/control/local), reword the error examples, and update the maintenance doc's addressing note + its section cross-link to match Slice B. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 03:02:07 +03:00
stdout.contains("COMMANDS BY CAPABILITY"),
"capability legend (after_help) missing from --help:\n{stdout}"
feat(cli): plane-grouped --help + clap 4.6.1 (RFC-010 Slice 2) (#220) * chore(deps): bump clap to 4.6.1 Workspace constraint "4" → "4.6" so the resolver picks up the 4.6 line (a plain `cargo update` stayed on 4.5.x). clap 4.5.58 → 4.6.1 (clap_builder 4.6.0, clap_derive 4.6.1). Minor bump, no API breakage; the workspace builds and all CLI suites pass unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): group --help by plane (RFC-010 Slice 2) Slice 1 declared the planes (the command_plane table + the wrong-plane guard); this makes them visible in `--help`. clap can't print labeled heading rows between subcommand groups (verified against the source — help_heading is args-only, {subcommands} is one flat block), so per the chosen approach: cluster + legend. - Reorder the `Command` enum into plane bands (clap lists subcommands in declaration order): data (query, mutate, load, branch, snapshot, export, commit, schema, graphs) → storage/local-graph ops (init, optimize, repair, cleanup, lint, queries) → control (cluster) → session (policy, embed, login, logout, config, version). No magic display_order numbers — the source order IS the help order, with band comments for readers. The band placement matches `command_plane` (lint/queries are storage-plane: they reject --server), so the help grouping and the guard agree. - Add an `after_help` legend on `Cli` naming the planes. Written to describe the planes (not enumerate every command) so it doesn't drift. Help-polish (post-review): hide the deprecated `ingest` from the list (still a valid command); trim the long `login` and `--as` descriptions to one line each so the columns don't blow up. The behavioral source of truth for planes stays `planes::command_plane`; this ordering is its cosmetic counterpart. Test: `help_groups_commands_by_plane` pins the legend phrase + the cluster ordering (query < optimize < cluster). Doc: a line under cli-reference's *Command planes* section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): qualify mixed-plane commands in the --help legend Addresses the Greptile P2 on #220: the legend placed `schema` entirely in Data and `queries` entirely in Storage, but per `command_plane` the subcommands differ — `schema plan` is storage-plane (rejects --server) and `queries list` is session (no graph). A user reading the legend then running `schema plan --server` would hit a rejection contradicting it. The Commands list is one entry per top-level command (necessarily coarse), so the legend carries the nuance: `schema [plan: storage]` and `queries [list: session]`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 01:49:40 +03:00
);
// The Commands list precedes the legend, so first occurrences sit in the
feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) (#237) * feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) User-facing CLI errors and --help now speak a single "capability" vocabulary — what a command needs — instead of the internal four-plane jargon. Behavior is unchanged: the --server/--graph allow set is identical (the served-graph capabilities `any` ∪ `served` = the old `Data` plane, since `graphs` was already allowed). Only error text and the --help legend change. - planes.rs: add `Capability { Any, Served, Direct, Control, Local }` derived from the existing exhaustive `command_plane` classifier (which stays as the drift guard) plus the one Data→Served refinement (`graphs`). `guard_addressing` now allows `--server`/`--graph` on `{Any, Served}` and rejects elsewhere with a capability-worded message. The mapping reflects *current* behavior (`queries list` → Local, `queries validate` → Direct); it converges to the RFC end-state table when later slices re-route those verbs. - scope.rs: `resolve_scope` takes `Capability` instead of `Plane`, so the whole addressing path speaks one vocabulary; call sites in client.rs (Any) and the 3 maintenance verbs in main.rs (Direct) updated. - helpers.rs: the storage-direct remote rejection reworded to "direct (storage-native) command". - cli.rs: the --help legend is now "COMMANDS BY CAPABILITY". - Tests: the 5 assertions pinning the old plane text updated; added planes.rs unit tests proving the allow set is exactly {Any, Served} (behavior-preservation), the per-verb mapping, and distinct capability phrases. Full omnigraph-cli suite: 225 green (222 + 3 new), zero behavior-test changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(cli): capability vocabulary in the CLI reference + maintenance addressing Rename the reference's "Command planes" section to "Command capabilities" (any/served/direct/control/local), reword the error examples, and update the maintenance doc's addressing note + its section cross-link to match Slice B. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 03:02:07 +03:00
// list and must appear in order: an `any` data verb, then a `direct` verb,
// then the `control` verb.
feat(cli): plane-grouped --help + clap 4.6.1 (RFC-010 Slice 2) (#220) * chore(deps): bump clap to 4.6.1 Workspace constraint "4" → "4.6" so the resolver picks up the 4.6 line (a plain `cargo update` stayed on 4.5.x). clap 4.5.58 → 4.6.1 (clap_builder 4.6.0, clap_derive 4.6.1). Minor bump, no API breakage; the workspace builds and all CLI suites pass unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): group --help by plane (RFC-010 Slice 2) Slice 1 declared the planes (the command_plane table + the wrong-plane guard); this makes them visible in `--help`. clap can't print labeled heading rows between subcommand groups (verified against the source — help_heading is args-only, {subcommands} is one flat block), so per the chosen approach: cluster + legend. - Reorder the `Command` enum into plane bands (clap lists subcommands in declaration order): data (query, mutate, load, branch, snapshot, export, commit, schema, graphs) → storage/local-graph ops (init, optimize, repair, cleanup, lint, queries) → control (cluster) → session (policy, embed, login, logout, config, version). No magic display_order numbers — the source order IS the help order, with band comments for readers. The band placement matches `command_plane` (lint/queries are storage-plane: they reject --server), so the help grouping and the guard agree. - Add an `after_help` legend on `Cli` naming the planes. Written to describe the planes (not enumerate every command) so it doesn't drift. Help-polish (post-review): hide the deprecated `ingest` from the list (still a valid command); trim the long `login` and `--as` descriptions to one line each so the columns don't blow up. The behavioral source of truth for planes stays `planes::command_plane`; this ordering is its cosmetic counterpart. Test: `help_groups_commands_by_plane` pins the legend phrase + the cluster ordering (query < optimize < cluster). Doc: a line under cli-reference's *Command planes* section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): qualify mixed-plane commands in the --help legend Addresses the Greptile P2 on #220: the legend placed `schema` entirely in Data and `queries` entirely in Storage, but per `command_plane` the subcommands differ — `schema plan` is storage-plane (rejects --server) and `queries list` is session (no graph). A user reading the legend then running `schema plan --server` would hit a rejection contradicting it. The Commands list is one entry per top-level command (necessarily coarse), so the legend carries the nuance: `schema [plan: storage]` and `queries [list: session]`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 01:49:40 +03:00
let pos = |needle: &str| {
stdout
.find(needle)
.unwrap_or_else(|| panic!("'{needle}' not found in --help:\n{stdout}"))
};
assert!(
pos("query") < pos("optimize"),
feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) (#237) * feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) User-facing CLI errors and --help now speak a single "capability" vocabulary — what a command needs — instead of the internal four-plane jargon. Behavior is unchanged: the --server/--graph allow set is identical (the served-graph capabilities `any` ∪ `served` = the old `Data` plane, since `graphs` was already allowed). Only error text and the --help legend change. - planes.rs: add `Capability { Any, Served, Direct, Control, Local }` derived from the existing exhaustive `command_plane` classifier (which stays as the drift guard) plus the one Data→Served refinement (`graphs`). `guard_addressing` now allows `--server`/`--graph` on `{Any, Served}` and rejects elsewhere with a capability-worded message. The mapping reflects *current* behavior (`queries list` → Local, `queries validate` → Direct); it converges to the RFC end-state table when later slices re-route those verbs. - scope.rs: `resolve_scope` takes `Capability` instead of `Plane`, so the whole addressing path speaks one vocabulary; call sites in client.rs (Any) and the 3 maintenance verbs in main.rs (Direct) updated. - helpers.rs: the storage-direct remote rejection reworded to "direct (storage-native) command". - cli.rs: the --help legend is now "COMMANDS BY CAPABILITY". - Tests: the 5 assertions pinning the old plane text updated; added planes.rs unit tests proving the allow set is exactly {Any, Served} (behavior-preservation), the per-verb mapping, and distinct capability phrases. Full omnigraph-cli suite: 225 green (222 + 3 new), zero behavior-test changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(cli): capability vocabulary in the CLI reference + maintenance addressing Rename the reference's "Command planes" section to "Command capabilities" (any/served/direct/control/local), reword the error examples, and update the maintenance doc's addressing note + its section cross-link to match Slice B. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 03:02:07 +03:00
"data (any) commands should be listed before direct commands"
feat(cli): plane-grouped --help + clap 4.6.1 (RFC-010 Slice 2) (#220) * chore(deps): bump clap to 4.6.1 Workspace constraint "4" → "4.6" so the resolver picks up the 4.6 line (a plain `cargo update` stayed on 4.5.x). clap 4.5.58 → 4.6.1 (clap_builder 4.6.0, clap_derive 4.6.1). Minor bump, no API breakage; the workspace builds and all CLI suites pass unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): group --help by plane (RFC-010 Slice 2) Slice 1 declared the planes (the command_plane table + the wrong-plane guard); this makes them visible in `--help`. clap can't print labeled heading rows between subcommand groups (verified against the source — help_heading is args-only, {subcommands} is one flat block), so per the chosen approach: cluster + legend. - Reorder the `Command` enum into plane bands (clap lists subcommands in declaration order): data (query, mutate, load, branch, snapshot, export, commit, schema, graphs) → storage/local-graph ops (init, optimize, repair, cleanup, lint, queries) → control (cluster) → session (policy, embed, login, logout, config, version). No magic display_order numbers — the source order IS the help order, with band comments for readers. The band placement matches `command_plane` (lint/queries are storage-plane: they reject --server), so the help grouping and the guard agree. - Add an `after_help` legend on `Cli` naming the planes. Written to describe the planes (not enumerate every command) so it doesn't drift. Help-polish (post-review): hide the deprecated `ingest` from the list (still a valid command); trim the long `login` and `--as` descriptions to one line each so the columns don't blow up. The behavioral source of truth for planes stays `planes::command_plane`; this ordering is its cosmetic counterpart. Test: `help_groups_commands_by_plane` pins the legend phrase + the cluster ordering (query < optimize < cluster). Doc: a line under cli-reference's *Command planes* section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): qualify mixed-plane commands in the --help legend Addresses the Greptile P2 on #220: the legend placed `schema` entirely in Data and `queries` entirely in Storage, but per `command_plane` the subcommands differ — `schema plan` is storage-plane (rejects --server) and `queries list` is session (no graph). A user reading the legend then running `schema plan --server` would hit a rejection contradicting it. The Commands list is one entry per top-level command (necessarily coarse), so the legend carries the nuance: `schema [plan: storage]` and `queries [list: session]`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 01:49:40 +03:00
);
assert!(
pos("optimize") < pos("cluster"),
feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) (#237) * feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) User-facing CLI errors and --help now speak a single "capability" vocabulary — what a command needs — instead of the internal four-plane jargon. Behavior is unchanged: the --server/--graph allow set is identical (the served-graph capabilities `any` ∪ `served` = the old `Data` plane, since `graphs` was already allowed). Only error text and the --help legend change. - planes.rs: add `Capability { Any, Served, Direct, Control, Local }` derived from the existing exhaustive `command_plane` classifier (which stays as the drift guard) plus the one Data→Served refinement (`graphs`). `guard_addressing` now allows `--server`/`--graph` on `{Any, Served}` and rejects elsewhere with a capability-worded message. The mapping reflects *current* behavior (`queries list` → Local, `queries validate` → Direct); it converges to the RFC end-state table when later slices re-route those verbs. - scope.rs: `resolve_scope` takes `Capability` instead of `Plane`, so the whole addressing path speaks one vocabulary; call sites in client.rs (Any) and the 3 maintenance verbs in main.rs (Direct) updated. - helpers.rs: the storage-direct remote rejection reworded to "direct (storage-native) command". - cli.rs: the --help legend is now "COMMANDS BY CAPABILITY". - Tests: the 5 assertions pinning the old plane text updated; added planes.rs unit tests proving the allow set is exactly {Any, Served} (behavior-preservation), the per-verb mapping, and distinct capability phrases. Full omnigraph-cli suite: 225 green (222 + 3 new), zero behavior-test changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(cli): capability vocabulary in the CLI reference + maintenance addressing Rename the reference's "Command planes" section to "Command capabilities" (any/served/direct/control/local), reword the error examples, and update the maintenance doc's addressing note + its section cross-link to match Slice B. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 03:02:07 +03:00
"direct commands should be listed before the control command"
feat(cli): plane-grouped --help + clap 4.6.1 (RFC-010 Slice 2) (#220) * chore(deps): bump clap to 4.6.1 Workspace constraint "4" → "4.6" so the resolver picks up the 4.6 line (a plain `cargo update` stayed on 4.5.x). clap 4.5.58 → 4.6.1 (clap_builder 4.6.0, clap_derive 4.6.1). Minor bump, no API breakage; the workspace builds and all CLI suites pass unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): group --help by plane (RFC-010 Slice 2) Slice 1 declared the planes (the command_plane table + the wrong-plane guard); this makes them visible in `--help`. clap can't print labeled heading rows between subcommand groups (verified against the source — help_heading is args-only, {subcommands} is one flat block), so per the chosen approach: cluster + legend. - Reorder the `Command` enum into plane bands (clap lists subcommands in declaration order): data (query, mutate, load, branch, snapshot, export, commit, schema, graphs) → storage/local-graph ops (init, optimize, repair, cleanup, lint, queries) → control (cluster) → session (policy, embed, login, logout, config, version). No magic display_order numbers — the source order IS the help order, with band comments for readers. The band placement matches `command_plane` (lint/queries are storage-plane: they reject --server), so the help grouping and the guard agree. - Add an `after_help` legend on `Cli` naming the planes. Written to describe the planes (not enumerate every command) so it doesn't drift. Help-polish (post-review): hide the deprecated `ingest` from the list (still a valid command); trim the long `login` and `--as` descriptions to one line each so the columns don't blow up. The behavioral source of truth for planes stays `planes::command_plane`; this ordering is its cosmetic counterpart. Test: `help_groups_commands_by_plane` pins the legend phrase + the cluster ordering (query < optimize < cluster). Doc: a line under cli-reference's *Command planes* section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): qualify mixed-plane commands in the --help legend Addresses the Greptile P2 on #220: the legend placed `schema` entirely in Data and `queries` entirely in Storage, but per `command_plane` the subcommands differ — `schema plan` is storage-plane (rejects --server) and `queries list` is session (no graph). A user reading the legend then running `schema plan --server` would hit a rejection contradicting it. The Commands list is one entry per top-level command (necessarily coarse), so the legend carries the nuance: `schema [plan: storage]` and `queries [list: session]`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 01:49:40 +03:00
);
}
#[test]
fn init_creates_graph_successfully_on_missing_local_directory() {
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema = fixture("test.pg");
let output = output_success(cli().arg("init").arg("--schema").arg(&schema).arg(&graph));
let stdout = stdout_string(&output);
assert!(stdout.contains("initialized"));
assert!(graph.join("_schema.pg").exists());
assert!(graph.join("__manifest").exists());
// RFC-008 stage 3: init no longer scaffolds the legacy config file.
assert!(!temp.path().join("omnigraph.yaml").exists());
}
#[test]
fn schema_plan_json_reports_supported_additive_change() {
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema_path = temp.path().join("next.pg");
init_graph(&graph);
let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace(
" age: I32?\n}",
" age: I32?\n nickname: String?\n}",
);
fs::write(&schema_path, next_schema).unwrap();
let output = output_success(
cli()
.arg("schema")
.arg("plan")
.arg("--schema")
.arg(&schema_path)
.arg("--json")
.arg(&graph),
);
let payload: Value = serde_json::from_slice(&output.stdout).unwrap();
assert_eq!(payload["supported"], true);
assert_eq!(payload["step_count"], 1);
assert_eq!(payload["steps"][0]["kind"], "add_property");
assert_eq!(payload["steps"][0]["type_kind"], "node");
assert_eq!(payload["steps"][0]["type_name"], "Person");
assert_eq!(payload["steps"][0]["property_name"], "nickname");
}
feat(cli): RFC-010 Slice 1 — declared plane capability surface + honest addressing (#217) * feat(cli): declared plane capability surface + wrong-plane guard (RFC-010 Slice 1) New `planes.rs` is the single source of truth for which plane each subcommand belongs to (Data / Storage / Control / Session). `command_plane` is an exhaustive match — adding a `Command` variant is a compile error until its plane is declared, so the surface cannot silently drift from the command set. It descends into the nested enums where the plane differs per subcommand (`schema plan` is storage while `schema show/apply` are data; `queries validate` opens the graph while `queries list` reads only config). `guard_addressing` runs once in `main` before dispatch: the data-plane addressing flags `--server`/`--graph` on any non-data verb now fail with one declared, pinned error instead of being silently ignored (`optimize --server prod` previously dropped `--server`). `init`'s message drops the `--target` half since it takes only a positional URI today. Test: `cli_schema_config::schema_plan_with_server_flag_errors_wrong_plane` pins the per-subcommand label, proving the guard descends into the nested enum. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): storage-plane verbs fail loudly on a remote target (RFC-010 Slice 1) `optimize`/`repair`/`cleanup` switch from `resolve_uri` to `resolve_local_uri`, so a `--target` (or positional URI) that resolves to a remote server now fails with a declared storage-plane message instead of whatever `Omnigraph::open` said about an `http(s)://` URI. The `resolve_local_graph` bail is reworded to that storage-plane message, so every storage verb already on the local resolver (`schema plan`, `queries validate`, `lint`) speaks with one voice. Net: `optimize --target knowledge` resolves to the graph's storage URI and runs embedded; `optimize --target prod` (remote) fails loudly; `optimize --server` is caught earlier by the guard. Positional-URI invocations are unchanged. Tests (pinned strings, per RFC-010's test plan): optimize happy path on a local graph, `optimize --server` wrong-plane error, `optimize <https>` storage-plane error; the existing `query_lint_rejects_http_targets_without_schema` assertion is updated to the new shared message. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 22:45:58 +03:00
#[test]
fn schema_plan_with_server_flag_errors_wrong_plane() {
// RFC-010 Slice 1: `schema plan` is storage-plane while `schema show/apply`
// are data-plane — the guard rejects --server on plan with the per-subcommand
// label (proving command_plane/command_label descend into the nested enum).
let output = output_failure(
cli()
.arg("schema")
.arg("plan")
.arg("--schema")
.arg(fixture("test.pg"))
.arg("--server")
.arg("prod"),
);
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(
feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) (#237) * feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) User-facing CLI errors and --help now speak a single "capability" vocabulary — what a command needs — instead of the internal four-plane jargon. Behavior is unchanged: the --server/--graph allow set is identical (the served-graph capabilities `any` ∪ `served` = the old `Data` plane, since `graphs` was already allowed). Only error text and the --help legend change. - planes.rs: add `Capability { Any, Served, Direct, Control, Local }` derived from the existing exhaustive `command_plane` classifier (which stays as the drift guard) plus the one Data→Served refinement (`graphs`). `guard_addressing` now allows `--server`/`--graph` on `{Any, Served}` and rejects elsewhere with a capability-worded message. The mapping reflects *current* behavior (`queries list` → Local, `queries validate` → Direct); it converges to the RFC end-state table when later slices re-route those verbs. - scope.rs: `resolve_scope` takes `Capability` instead of `Plane`, so the whole addressing path speaks one vocabulary; call sites in client.rs (Any) and the 3 maintenance verbs in main.rs (Direct) updated. - helpers.rs: the storage-direct remote rejection reworded to "direct (storage-native) command". - cli.rs: the --help legend is now "COMMANDS BY CAPABILITY". - Tests: the 5 assertions pinning the old plane text updated; added planes.rs unit tests proving the allow set is exactly {Any, Served} (behavior-preservation), the per-verb mapping, and distinct capability phrases. Full omnigraph-cli suite: 225 green (222 + 3 new), zero behavior-test changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(cli): capability vocabulary in the CLI reference + maintenance addressing Rename the reference's "Command planes" section to "Command capabilities" (any/served/direct/control/local), reword the error examples, and update the maintenance doc's addressing note + its section cross-link to match Slice B. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 03:02:07 +03:00
stderr.contains("`schema plan` is a direct (storage-native) command")
&& stderr.contains("Pass a storage URI."),
feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) (#237) * feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) User-facing CLI errors and --help now speak a single "capability" vocabulary — what a command needs — instead of the internal four-plane jargon. Behavior is unchanged: the --server/--graph allow set is identical (the served-graph capabilities `any` ∪ `served` = the old `Data` plane, since `graphs` was already allowed). Only error text and the --help legend change. - planes.rs: add `Capability { Any, Served, Direct, Control, Local }` derived from the existing exhaustive `command_plane` classifier (which stays as the drift guard) plus the one Data→Served refinement (`graphs`). `guard_addressing` now allows `--server`/`--graph` on `{Any, Served}` and rejects elsewhere with a capability-worded message. The mapping reflects *current* behavior (`queries list` → Local, `queries validate` → Direct); it converges to the RFC end-state table when later slices re-route those verbs. - scope.rs: `resolve_scope` takes `Capability` instead of `Plane`, so the whole addressing path speaks one vocabulary; call sites in client.rs (Any) and the 3 maintenance verbs in main.rs (Direct) updated. - helpers.rs: the storage-direct remote rejection reworded to "direct (storage-native) command". - cli.rs: the --help legend is now "COMMANDS BY CAPABILITY". - Tests: the 5 assertions pinning the old plane text updated; added planes.rs unit tests proving the allow set is exactly {Any, Served} (behavior-preservation), the per-verb mapping, and distinct capability phrases. Full omnigraph-cli suite: 225 green (222 + 3 new), zero behavior-test changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(cli): capability vocabulary in the CLI reference + maintenance addressing Rename the reference's "Command planes" section to "Command capabilities" (any/served/direct/control/local), reword the error examples, and update the maintenance doc's addressing note + its section cross-link to match Slice B. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 03:02:07 +03:00
"schema plan wrong-capability message not found; got: {stderr}"
feat(cli): RFC-010 Slice 1 — declared plane capability surface + honest addressing (#217) * feat(cli): declared plane capability surface + wrong-plane guard (RFC-010 Slice 1) New `planes.rs` is the single source of truth for which plane each subcommand belongs to (Data / Storage / Control / Session). `command_plane` is an exhaustive match — adding a `Command` variant is a compile error until its plane is declared, so the surface cannot silently drift from the command set. It descends into the nested enums where the plane differs per subcommand (`schema plan` is storage while `schema show/apply` are data; `queries validate` opens the graph while `queries list` reads only config). `guard_addressing` runs once in `main` before dispatch: the data-plane addressing flags `--server`/`--graph` on any non-data verb now fail with one declared, pinned error instead of being silently ignored (`optimize --server prod` previously dropped `--server`). `init`'s message drops the `--target` half since it takes only a positional URI today. Test: `cli_schema_config::schema_plan_with_server_flag_errors_wrong_plane` pins the per-subcommand label, proving the guard descends into the nested enum. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli): storage-plane verbs fail loudly on a remote target (RFC-010 Slice 1) `optimize`/`repair`/`cleanup` switch from `resolve_uri` to `resolve_local_uri`, so a `--target` (or positional URI) that resolves to a remote server now fails with a declared storage-plane message instead of whatever `Omnigraph::open` said about an `http(s)://` URI. The `resolve_local_graph` bail is reworded to that storage-plane message, so every storage verb already on the local resolver (`schema plan`, `queries validate`, `lint`) speaks with one voice. Net: `optimize --target knowledge` resolves to the graph's storage URI and runs embedded; `optimize --target prod` (remote) fails loudly; `optimize --server` is caught earlier by the guard. Positional-URI invocations are unchanged. Tests (pinned strings, per RFC-010's test plan): optimize happy path on a local graph, `optimize --server` wrong-plane error, `optimize <https>` storage-plane error; the existing `query_lint_rejects_http_targets_without_schema` assertion is updated to the new shared message. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 22:45:58 +03:00
);
}
#[test]
fn schema_plan_json_reports_unsupported_type_change() {
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema_path = temp.path().join("breaking.pg");
init_graph(&graph);
let breaking_schema = fs::read_to_string(fixture("test.pg"))
.unwrap()
.replace("age: I32?", "age: I64?");
fs::write(&schema_path, breaking_schema).unwrap();
let output = output_success(
cli()
.arg("schema")
.arg("plan")
.arg("--schema")
.arg(&schema_path)
.arg("--json")
.arg(&graph),
);
let payload: Value = serde_json::from_slice(&output.stdout).unwrap();
assert_eq!(payload["supported"], false);
assert!(payload["steps"].as_array().unwrap().iter().any(|step| {
step["kind"] == "unsupported_change"
&& step["entity"]
.as_str()
.unwrap_or_default()
.contains("Person.age")
}));
}
#[test]
fn schema_apply_json_applies_supported_migration() {
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema_path = temp.path().join("next.pg");
init_graph(&graph);
let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace(
" age: I32?\n}",
" age: I32?\n nickname: String?\n}",
);
fs::write(&schema_path, next_schema).unwrap();
let output = output_success(
cli()
.arg("schema")
.arg("apply")
.arg("--schema")
.arg(&schema_path)
.arg("--json")
.arg(&graph),
);
let payload: Value = serde_json::from_slice(&output.stdout).unwrap();
assert_eq!(payload["supported"], true);
assert_eq!(payload["applied"], true);
assert_eq!(payload["step_count"], 1);
let db = tokio::runtime::Runtime::new()
.unwrap()
.block_on(Omnigraph::open(graph.to_string_lossy().as_ref()))
.unwrap();
assert!(
db.catalog().node_types["Person"]
.properties
.contains_key("nickname")
);
}
#[test]
fn schema_apply_human_reports_noop() {
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema_path = fixture("test.pg");
init_graph(&graph);
let output = output_success(
cli()
.arg("schema")
.arg("apply")
.arg("--schema")
.arg(&schema_path)
.arg(&graph),
);
let stdout = stdout_string(&output);
assert!(stdout.contains("applied: no"));
assert!(stdout.contains("no schema changes"));
}
#[test]
fn schema_apply_json_renames_type_and_updates_snapshot() {
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema_path = temp.path().join("rename.pg");
init_graph(&graph);
let renamed_schema = fs::read_to_string(fixture("test.pg"))
.unwrap()
.replace("node Person {\n", "node Human @rename_from(\"Person\") {\n")
.replace("edge Knows: Person -> Person", "edge Knows: Human -> Human")
.replace(
"edge WorksAt: Person -> Company",
"edge WorksAt: Human -> Company",
);
fs::write(&schema_path, renamed_schema).unwrap();
let output = output_success(
cli()
.arg("schema")
.arg("apply")
.arg("--schema")
.arg(&schema_path)
.arg("--json")
.arg(&graph),
);
let payload: Value = serde_json::from_slice(&output.stdout).unwrap();
assert_eq!(payload["applied"], true);
let db = tokio::runtime::Runtime::new()
.unwrap()
.block_on(Omnigraph::open(graph.to_string_lossy().as_ref()))
.unwrap();
let snapshot = tokio::runtime::Runtime::new()
.unwrap()
.block_on(db.snapshot_of(ReadTarget::branch("main")))
.unwrap();
assert!(snapshot.entry("node:Human").is_some());
assert!(snapshot.entry("node:Person").is_none());
}
#[test]
fn schema_apply_json_renames_property_and_updates_catalog() {
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema_path = temp.path().join("rename-property.pg");
init_graph(&graph);
let renamed_schema = fs::read_to_string(fixture("test.pg"))
.unwrap()
.replace("age: I32?", "years: I32? @rename_from(\"age\")");
fs::write(&schema_path, renamed_schema).unwrap();
let output = output_success(
cli()
.arg("schema")
.arg("apply")
.arg("--schema")
.arg(&schema_path)
.arg("--json")
.arg(&graph),
);
let payload: Value = serde_json::from_slice(&output.stdout).unwrap();
assert_eq!(payload["applied"], true);
let db = tokio::runtime::Runtime::new()
.unwrap()
.block_on(Omnigraph::open(graph.to_string_lossy().as_ref()))
.unwrap();
let person = &db.catalog().node_types["Person"];
assert!(person.properties.contains_key("years"));
assert!(!person.properties.contains_key("age"));
}
#[test]
fn schema_apply_json_adds_index_for_existing_property() {
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema_path = temp.path().join("index.pg");
init_graph(&graph);
let before_index_count = tokio::runtime::Runtime::new().unwrap().block_on(async {
let db = Omnigraph::open(graph.to_string_lossy().as_ref())
.await
.unwrap();
let snapshot = db.snapshot_of(ReadTarget::branch("main")).await.unwrap();
let dataset = snapshot.open("node:Person").await.unwrap();
dataset.load_indices().await.unwrap().len()
});
let indexed_schema = fs::read_to_string(fixture("test.pg"))
.unwrap()
.replace("name: String @key", "name: String @key @index");
fs::write(&schema_path, indexed_schema).unwrap();
let output = output_success(
cli()
.arg("schema")
.arg("apply")
.arg("--schema")
.arg(&schema_path)
.arg("--json")
.arg(&graph),
);
let payload: Value = serde_json::from_slice(&output.stdout).unwrap();
assert_eq!(payload["applied"], true);
let after_index_count = tokio::runtime::Runtime::new().unwrap().block_on(async {
let db = Omnigraph::open(graph.to_string_lossy().as_ref())
.await
.unwrap();
let snapshot = db.snapshot_of(ReadTarget::branch("main")).await.unwrap();
let dataset = snapshot.open("node:Person").await.unwrap();
dataset.load_indices().await.unwrap().len()
});
Index materialization is derived state: defer off the write path, reconcile via optimize (iss-848) (#246) * test(engine): reproduce empty-table Vector @index aborting schema apply A Vector (IVF) index trains k-means centroids over the column, so Lance cannot build it on 0 vectors ("Creating empty vector indices with train=False is not yet implemented"). schema apply reconciles a table's whole index set whenever any @index on it changes, so adding an unrelated scalar @index materializes the dormant empty vector index and aborts the entire migration (all-or-nothing). This regression test inits a 0-row Doc with a Vector @index, adds a scalar @index, and asserts the apply succeeds (then loads one embedded row and asserts the deferred index materializes). It fails today at the apply step with the vector-index abort; the fix lands in the next commit. Refs dev-graph iss-empty-vector-index-schema-apply, iss-848. * fix(engine): defer Vector @index on an empty table instead of aborting schema apply build_indices_on_dataset_for_catalog materialized a declared Vector @index unconditionally. On a 0-row table Lance cannot train the IVF index ("Creating empty vector indices with train=False is not yet implemented"), so any later migration that touches the table (e.g. adding an unrelated scalar @index, which reconciles the table's whole index set) aborted the entire migration on the dormant vector index — all-or-nothing. Guard the vector arm with a row-count check, matching the guard ensure_indices_for_branch and the branch-merge rebuild already use: an untrainable column becomes a pending index that a later ensure_indices / optimize materializes once the table has rows. Reads stay correct meanwhile (vector search degrades to a brute-force scan). Stop-gap: the residual rows-present-but-vectors-null window and the full decoupling (intent recorded at apply, an idempotent coverage reconciler) are dev-graph iss-848. Turns the green half of the regression test added in the previous commit. Refs dev-graph iss-empty-vector-index-schema-apply, iss-848, iss-687. * docs(invariants): record the logical-contract-over-physical-state principle The bug class behind the empty-table vector-index abort (and the schema-apply vs optimize version drift) is one shape: a physical operation allowed to fail a logical one. Several hard invariants (2, 5, 7, 13) and deny-list items are already instances of this, but the unifying rule was never written down. Add it to docs/dev/invariants.md as a "Governing principle" section above the hard invariants, naming which invariants and deny-list items instantiate it and the smell to watch for (a logical operation gated on a physical fact). Add a one-line always-on rule (7) in AGENTS.md so it stays in working memory, with the qualifier that genuine logical conflicts still fail loudly — the licence to lag covers physical convergence, not correctness. Audience-neutral: no private ticket refs. check-agents-md.sh passes. * test(engine): index build must tolerate rows with null vectors (load-before-embed) Loading rows whose vector column is null into a `Vector @index` table fails today: build_indices (reached via the loader's prepare_updates_for_commit) calls create_vector_index, and Lance's IVF KMeans errors "cannot train 1 centroids with 0 vectors". The same abort hits ensure_indices/optimize/schema apply/merge, since they all funnel through build_indices_on_dataset_for_catalog. This test loads two null-embedding rows and calls ensure_indices; it must not abort (the untrainable vector column is deferred, sibling indexes still build). Fails today at the load step; fixed in the next commit. Refs dev-graph iss-848, iss-empty-vector-index-schema-apply. * fix(engine): defer unbuildable index columns instead of aborting the write path build_indices_on_dataset_for_catalog is the chokepoint every write path funnels through (load/mutate via prepare_updates_for_commit, schema apply, ensure_indices, optimize, branch merge). Its vector arm called create_vector_index unconditionally, so a column with no trainable vectors yet — an empty table, or rows loaded before `embed` populates them — aborted the whole operation with Lance's IVF KMeans error. Fault-isolate the vector build: on failure, record the column as a PendingIndex (table, column, reason), log it, and continue building the sibling indexes; a later ensure_indices/optimize materializes it once the column is trainable, and reads use brute-force meanwhile. Manifest/CAS/IO errors at the publish boundary still propagate. Isolating at the single chokepoint realizes the governing principle (physical index state never fails a logical operation) for every write path, and supersedes the earlier symptomatic count_rows==0 stop-gap (removed) — closing the residual rows-present-but-vectors-null window it left open. Surfacing pending index status rather than failing is the database norm (Postgres indisvalid, LanceDB list_indices). ensure_indices and the build_indices wrappers now return Vec<PendingIndex>; optimize surfaces it in a later commit. Refs dev-graph iss-848, iss-951 (vector index stays inline-commit until lance#6666). * test(engine): index-only schema apply must not touch table data Adding an @index to an existing column should be a pure metadata change once index materialization moves to the reconciler (iss-848): the apply records the intent in the catalog/IR but builds nothing inline, so the table's manifest version is unchanged. Today the indexed_tables block builds the index inline and bumps the version (4 -> 5). Fixed in the next commit. Refs dev-graph iss-848. * fix(engine): schema apply records index intent only; index-only apply is metadata Schema apply no longer builds indexes inline. The four build_indices calls (added/renamed/rewritten/index-only tables) are removed; the @index/@key intent is already persisted in the catalog/IR the apply writes, and the physical index is materialized off the critical path by ensure_indices/optimize (iss-848). Concretely: - AddConstraint (an @index addition — every other added constraint plans as UnsupportedChange) becomes a pure metadata step alongside the metadata-only steps: it touches no table data, so the table version is unchanged. - added/renamed/rewritten tables still write their data; only the trailing index build is gone. The rewritten table's coverage is restored later by optimize_indices. - recovery_pins drops index-only tables (they no longer advance Lance HEAD) and keeps rewritten tables; their post_commit_pin = expected+1 is now exact (one rewrite commit), strengthening recovery classification. - the now-orphaned Omnigraph::build_indices_on_dataset_for_catalog wrapper is removed. A migration can no longer abort on an index build, for any index type at any cardinality. Turns the green half of index_only_constraint_apply_touches_no_table_data. Refs dev-graph iss-848. * test(engine): optimize must converge a declared-but-unbuilt index After iss-848, adding an @index post-data is a metadata-only apply that defers the physical build, so the column is declared-indexed but unbuilt (reads scan). `optimize` — the operator's cron reconciler — must materialize it. Today optimize only maintains coverage of EXISTING indexes (optimize_indices) and never creates missing ones, so the rank BTREE stays Degraded after optimize. Fixed next commit. Refs dev-graph iss-848. * fix(engine): optimize materializes declared-but-unbuilt indexes (the reconciler) `omnigraph optimize` is the operator's cron reconciler. It already compacts and folds new fragments into EXISTING indexes (optimize_indices); now it also builds declared-but-missing indexes, so the indexes schema apply / load defer (iss-848) converge on the next optimize. Done inside optimize_one_table (not by composing the all-tables ensure_indices, which is drift-blind and would re-publish the uncovered HEAD>manifest drift that optimize deliberately skips): after the per-table drift/blob skips and under the queue + Optimize sidecar already held, a needs_index_create gate (reusing needs_index_work_node/edge — "declared index missing AND row_count > 0", so empty tables stay no-ops) admits index-only work, and Phase B builds the missing index over the just-compacted layout via the build chokepoint. An untrainable vector column fault-isolates into the new TableOptimizeStats.pending_indexes (the list_indices/indisvalid analog operators read), not a failure. committed now reflects index commits, so the existing post-publish cache invalidation covers them. LanceDB's optimize only maintains existing indexes; creating declared-but-missing ones is the L2 behavior omnigraph's declarative @index needs. Turns the green half of optimize_materializes_index_declared_but_unbuilt. Refs dev-graph iss-848. * docs: index materialization is deferred to the reconciler (iss-848) Update the index-lifecycle docs to reflect the new contract: @index/@key declares intent and the physical index is derived state that never fails a logical operation. Schema apply builds nothing (records intent only); load/mutate build inline through one chokepoint that defers an untrainable Vector column as pending; optimize/ensure_indices is the reconciler that creates declared-but-missing indexes and maintains coverage, reporting still-pending columns. Touches: dev/invariants.md (truth-matrix Index-lifecycle row), AGENTS.md (capability matrix), user/search/indexes.md (L2 orchestration), user/operations/ maintenance.md (optimize reconciler bullet), dev/testing.md (new tests). * test(server): schema_apply_route_can_add_index reflects deferred index build iss-848 made schema apply record @index intent without building the physical index inline. The route test asserted the index count increased after apply; on an empty graph it now stays unchanged (the build is deferred to ensure_indices/optimize). Assert the new contract: apply succeeds and the physical index count is unchanged. * fix(engine): precheck vector trainability — don't pin or swallow (PR review) Two issues Cursor Bugbot caught in the chokepoint fault-isolation: 1. (HIGH) Pending vector pins roll back siblings. needs_index_work_node counted a missing vector index as work whenever the table had rows, so a column with no trainable vectors got pinned in the EnsureIndices recovery sidecar — but the build deferred it (zero commit). On a crash before manifest publish the classifier sees NoMovement and the all-or-nothing decision (recovery.rs decide()) rolls back the WHOLE sidecar, undoing a sibling table's committed index work. 2. (MED) Vector build swallowed fatal errors. The match arm converted every create_vector_index error into a deferred PendingIndex, hiding genuine I/O/manifest/Lance failures as "pending". Fix both with one trainability precheck (vector_column_trainable: >=1 non-null vector, the ivf_flat(1) minimum) used identically by needs_index_work_node and the build arm: an untrainable column is never counted as work (so never pinned — no zero-commit pin) and never attempted (so it can't fail); only a trainable column is built, and then any error PROPAGATES (stays fatal). The deferred column is still recorded as a PendingIndex with a clear reason. Refs dev-graph iss-848. * feat(cli): surface pending index column + reason in optimize output (PR review) Codex (P2): pending_indexes was documented as visible in `optimize --json` but the CLI projection never emitted it — operators would lose the only signal that optimize has deferred index work. Greptile (P2): the stat dropped the reason, so operators saw which column was stuck, not why. Carry the reason: TableOptimizeStats.pending_indexes is now Vec<PendingIndex> (column + reason), and `omnigraph optimize --json` emits {column, reason} per pending index; human output prints a "↳ index pending on '<col>': <reason>" line. Refs dev-graph iss-848. * test: align CLI index-add test with deferred build; cover post-rename reconcile - schema_apply_json_adds_index_for_existing_property (cli_schema_config.rs): the CLI analog of the server test — asserted the index count grew after apply; under iss-848 the apply defers the build, so the count is unchanged on an empty graph. Assert the deferred contract. (The only full-suite failure.) - optimize_materializes_index_after_type_rename (maintenance.rs, new): covers the gap Greptile flagged — a RenameType writes the renamed table with rows but no indexes (inline build removed in Commit B); assert the rank index is Degraded post-rename and Indexed after optimize reconciles it. Refs dev-graph iss-848. * test(engine): in-source apply tests reflect deferred index materialization The two db::omnigraph in-source unit tests asserted the old "schema apply builds / preserves indexes inline" behavior (the only remaining full-suite failures): - test_apply_schema_defers_index_then_reconciler_builds_it (was test_apply_schema_adds_index_for_existing_property): apply records the @index intent but builds nothing; assert the BTREE on `age` is absent after apply and present after ensure_indices. (Uses `age`, unindexed in TEST_SCHEMA — `name @key` is already FTS-indexed at seed.) - test_apply_schema_rewrite_defers_index_then_reconciler_restores (was test_apply_schema_rewrite_preserves_existing_indices): an AddProperty rewrite no longer rebuilds indexes inline; assert ensure_indices restores id BTREE + name FTS after the rewrite. Verified by grep that these + the server/CLI tests are the complete set of "apply builds an index" assertions; all other index-presence tests run after load/ensure_indices/primitives, which still build. Refs dev-graph iss-848. * fix(engine): optimize always reports pending indexes, not only on create-work (PR review) Cursor Bugbot (MED): pending_indexes was filled only when needs_index_create was true, but the vector trainability precheck makes needs_index_work_node exclude an untrainable Vector column. So a table whose sole missing index is untrainable, but which optimize still compacts or reindexes, returned an empty pending_indexes — contradicting the documented operator contract for deferred columns. Run the (idempotent) build chokepoint unconditionally once past the no-op gate, rather than gating it on needs_index_create. It skips existing indexes, builds any buildable missing one, and reports an untrainable column as pending whether the table entered for compaction, reindex, or index creation. needs_index_create still gates the no-op decision (so an index-only table still enters the path). Refs dev-graph iss-848. * test(engine): reframe staged-BTREE-failure failpoint onto the reconciler path ensure_indices_stage_btree_failure_leaves_existing_tables_writable fired `ensure_indices.post_stage_pre_commit_btree` and expected `apply_schema` (adding a type) to fail mid-BTREE-build. iss-848 removed apply's inline index build, so that apply now succeeds and the test's unwrap_err panicked — it exercised a removed code path. Reframe onto where BTREE builds happen now: seed Person, add an `@index` on `age` (apply records intent, defers the build), then `ensure_indices` builds the deferred BTREE and the failpoint fires between stage and commit. Person's HEAD is unchanged (no drift) and its EnsureIndices sidecar pins NoMovement; a write to a different, unpinned table (Company) is unaffected (mutations/loads heal roll-forward and proceed, unlike optimize/repair which refuse on a pending sidecar). Preserves the original coverage (staged-index stage failure leaves other tables writable, no drift) in the new architecture. Refs dev-graph iss-848. * feat(server): converge deferred indexes promptly after schema apply (iss-848) Schema apply records @index intent but defers the physical build. On a long-lived server, spawn a detached best-effort ensure_indices after a successful apply so the indexes converge promptly instead of waiting for the operator's next optimize. Fire-and-forget: it never blocks or fails the apply response, and a failure is logged (the index still converges on the next optimize). Guarded on result.applied. The CLI is one-shot, so it has no equivalent; its convergence path is the optimize cadence. handle.engine is already an Arc, so the spawn takes an owned clone. Convergence itself is covered by the engine ensure_indices/optimize tests; the existing empty-graph schema-apply route tests confirm the response is unaffected (the spawn is a read-only no-op on an empty table). Refs dev-graph iss-848. * docs(maintenance): list pending_indexes in optimize per-table stats (consistency)
2026-06-15 18:48:43 +02:00
// iss-848: `schema apply` records the `@index` intent but defers the physical
// index build (materialized later by ensure_indices/optimize; on this empty
// table nothing builds anyway). So the physical index count is unchanged.
assert_eq!(
after_index_count, before_index_count,
"schema apply records @index intent but defers the physical build (iss-848)"
);
}
#[test]
fn schema_apply_rejects_unsupported_plan() {
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema_path = temp.path().join("breaking.pg");
init_graph(&graph);
let breaking_schema = fs::read_to_string(fixture("test.pg"))
.unwrap()
.replace("age: I32?", "age: I64?");
fs::write(&schema_path, breaking_schema).unwrap();
let output = output_failure(
cli()
.arg("schema")
.arg("apply")
.arg("--schema")
.arg(&schema_path)
.arg(&graph),
);
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(stderr.contains("changing property type"));
}
#[test]
fn schema_apply_rejects_when_non_main_branch_exists() {
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema_path = temp.path().join("next.pg");
init_graph(&graph);
output_success(
cli()
.arg("branch")
.arg("create")
.arg("--from")
.arg("main")
.arg("--uri")
.arg(&graph)
.arg("feature"),
);
let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace(
" age: I32?\n}",
" age: I32?\n nickname: String?\n}",
);
fs::write(&schema_path, next_schema).unwrap();
let output = output_failure(
cli()
.arg("schema")
.arg("apply")
.arg("--schema")
.arg(&schema_path)
.arg(&graph),
);
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(stderr.contains("schema apply requires a graph with only main"));
}
#[test]
fn schema_apply_allow_data_loss_flag_promotes_drops_to_hard() {
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema_path = temp.path().join("drop-age.pg");
init_graph(&graph);
// Drop the nullable `age` column.
let next_schema = fs::read_to_string(fixture("test.pg"))
.unwrap()
.replace(" age: I32?\n", "");
fs::write(&schema_path, next_schema).unwrap();
let output = output_success(
cli()
.arg("schema")
.arg("apply")
.arg("--schema")
.arg(&schema_path)
.arg("--allow-data-loss")
.arg("--json")
.arg(&graph),
);
let payload: Value = serde_json::from_slice(&output.stdout).unwrap();
assert_eq!(payload["applied"], true);
let drop_step = payload["steps"]
.as_array()
.unwrap()
.iter()
.find(|s| s["kind"] == "drop_property")
.expect("plan should include a drop_property step");
assert_eq!(
drop_step["mode"], "hard",
"--allow-data-loss should promote Soft → Hard; full step: {drop_step}",
);
}
#[test]
fn schema_apply_without_allow_data_loss_keeps_soft_drops() {
// Symmetric to the above: same schema change without the flag →
// drops stay Soft. Pins default semantics against accidental Hard
// promotion if a future refactor changes the option threading.
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
let schema_path = temp.path().join("drop-age-soft.pg");
init_graph(&graph);
let next_schema = fs::read_to_string(fixture("test.pg"))
.unwrap()
.replace(" age: I32?\n", "");
fs::write(&schema_path, next_schema).unwrap();
let output = output_success(
cli()
.arg("schema")
.arg("apply")
.arg("--schema")
.arg(&schema_path)
.arg("--json")
.arg(&graph),
);
let payload: Value = serde_json::from_slice(&output.stdout).unwrap();
assert_eq!(payload["applied"], true);
let drop_step = payload["steps"]
.as_array()
.unwrap()
.iter()
.find(|s| s["kind"] == "drop_property")
.expect("plan should include a drop_property step");
assert_eq!(
drop_step["mode"], "soft",
"no flag should leave drops Soft; full step: {drop_step}",
);
}
#[test]
fn schema_plan_parity_cli_and_sdk() {
// Same .pg through `Omnigraph::plan_schema_with_options` (SDK) and
// `omnigraph schema plan --json` (CLI). Asserts the steps array is
// byte-identical after JSON round-trip. HTTP doesn't expose a
// separate /schema/plan route — that side of parity is covered by
// the HTTP soft/hard drop tests, which exercise apply with
// identical fixtures.
let temp = tempdir().unwrap();
let graph = graph_path(temp.path());
init_graph(&graph);
let schema_path = temp.path().join("plan-parity.pg");
let next_schema = fs::read_to_string(fixture("test.pg")).unwrap().replace(
" age: I32?\n}",
" age: I32?\n nickname: String?\n}",
);
fs::write(&schema_path, &next_schema).unwrap();
// CLI side.
let cli_output = output_success(
cli()
.arg("schema")
.arg("plan")
.arg("--schema")
.arg(&schema_path)
.arg("--json")
.arg(&graph),
);
let cli_payload: Value = serde_json::from_slice(&cli_output.stdout).unwrap();
// SDK side: open graph, call plan_schema.
let plan = tokio::runtime::Runtime::new().unwrap().block_on(async {
let db = Omnigraph::open(graph.to_string_lossy().as_ref())
.await
.unwrap();
db.plan_schema(&next_schema).await.unwrap()
});
let sdk_steps = serde_json::to_value(&plan.steps).unwrap();
assert_eq!(
cli_payload["steps"], sdk_steps,
"CLI plan steps must match SDK plan steps for identical input",
);
assert_eq!(cli_payload["supported"], plan.supported);
}
#[test]
fn graphs_subcommand_help_lists_list_only() {
let output = output_success(cli().arg("graphs").arg("--help"));
let stdout = stdout_string(&output);
assert!(
stdout.contains("list"),
"expected `list` subcommand in help output:\n{stdout}"
);
let lowered = stdout.to_lowercase();
assert!(
!lowered.contains("create a new graph"),
"graph create should not be in v0.6.0 help; got:\n{stdout}"
);
assert!(
!lowered.contains("delete a graph"),
"graph delete should not be in v0.6.0 help; got:\n{stdout}"
);
}
#[test]
fn graphs_list_against_local_uri_errors_with_remote_only_message() {
let output = output_failure(
cli()
.arg("graphs")
.arg("list")
.arg("--uri")
.arg("/tmp/local"),
);
let stderr = String::from_utf8_lossy(&output.stderr).into_owned();
assert!(
stderr.contains("remote multi-graph server URL"),
"expected 'remote multi-graph server URL' rejection in stderr; got:\n{stderr}"
);
}
/// RFC-008 stage 1: loading a legacy omnigraph.yaml emits the per-key
/// deprecation block (the migration map applied to THIS file), suppressible
/// via OMNIGRAPH_SUPPRESS_YAML_DEPRECATION.
#[test]
fn legacy_config_load_warns_per_key_and_suppression_silences() {
let temp = tempdir().unwrap();
fs::write(
temp.path().join("omnigraph.yaml"),
"cli:\n actor: act-x\ngraphs:\n g:\n uri: /tmp/never-opened\n",
)
.unwrap();
// `graphs list --json` loads the config and exits without touching the
// graph URI.
let output = cli()
.current_dir(temp.path())
.arg("graphs")
.arg("list")
.arg("--json")
.output()
.unwrap();
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(
stderr.contains("deprecated (RFC-008)") && stderr.contains("`cli.actor` -> `operator.actor`"),
"{stderr}"
);
assert!(stderr.contains("config migrate"), "{stderr}");
let output = cli()
.current_dir(temp.path())
.env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1")
.arg("graphs")
.arg("list")
.arg("--json")
.output()
.unwrap();
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(!stderr.contains("deprecated (RFC-008)"), "{stderr}");
}
/// RFC-008 stage 2: `config migrate` proposes the split read-only, applies
/// it with --write (operator merge never clobbers; cluster.yaml emitted),
/// and a second --write is idempotent.
#[test]
fn config_migrate_splits_legacy_config() {
let temp = tempdir().unwrap();
fs::write(
temp.path().join("omnigraph.yaml"),
"graphs:\n prod:\n uri: https://graph.example.com\n bearer_token_env: PROD_TOKEN\ncli:\n actor: act-me\n output_format: json\npolicy:\n file: ./top.policy.yaml\n",
)
.unwrap();
let operator_home = tempfile::tempdir().unwrap();
fs::write(
operator_home.path().join("config.yaml"),
"operator:\n actor: act-existing\n",
)
.unwrap();
// Read-only proposal: names both halves, writes nothing.
let output = cli()
.current_dir(temp.path())
.env("OMNIGRAPH_HOME", operator_home.path())
.env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1")
.arg("config")
.arg("migrate")
.output()
.unwrap();
assert!(output.status.success(), "{output:?}");
let stdout = String::from_utf8_lossy(&output.stdout);
assert!(stdout.contains("team half -> cluster.yaml"), "{stdout}");
assert!(stdout.contains("operator.actor: act-me"), "{stdout}");
assert!(stdout.contains("omnigraph login prod"), "{stdout}");
assert!(!temp.path().join("cluster.yaml").exists());
// --write: cluster.yaml lands; the existing operator actor is KEPT.
let output = cli()
.current_dir(temp.path())
.env("OMNIGRAPH_HOME", operator_home.path())
.env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1")
.arg("config")
.arg("migrate")
.arg("--write")
.output()
.unwrap();
assert!(output.status.success(), "{output:?}");
let cluster = fs::read_to_string(temp.path().join("cluster.yaml")).unwrap();
assert!(cluster.contains("version: 1") && cluster.contains(" prod:"), "{cluster}");
let operator_text =
fs::read_to_string(operator_home.path().join("config.yaml")).unwrap();
assert!(operator_text.contains("act-existing"), "{operator_text}");
assert!(!operator_text.contains("act-me"), "existing keys win: {operator_text}");
assert!(operator_text.contains("output: json"), "{operator_text}");
assert!(
operator_text.contains("url: https://graph.example.com"),
"{operator_text}"
);
// Second --write: cluster.yaml exists -> proposal file, no clobber.
let output = cli()
.current_dir(temp.path())
.env("OMNIGRAPH_HOME", operator_home.path())
.env("OMNIGRAPH_SUPPRESS_YAML_DEPRECATION", "1")
.arg("config")
.arg("migrate")
.arg("--write")
.output()
.unwrap();
assert!(output.status.success(), "{output:?}");
assert!(temp.path().join("cluster.yaml.proposed").exists());
}
/// RFC-008 stage 4: OMNIGRAPH_NO_LEGACY_CONFIG refuses a present legacy
/// file (pointing at config migrate) but changes nothing on migrated
/// setups with no file.
#[test]
fn strict_mode_refuses_legacy_file_but_not_its_absence() {
let temp = tempdir().unwrap();
fs::write(temp.path().join("omnigraph.yaml"), "cli:\n actor: a\n").unwrap();
let output = cli()
.current_dir(temp.path())
.env("OMNIGRAPH_NO_LEGACY_CONFIG", "1")
.arg("graphs")
.arg("list")
.arg("--json")
.output()
.unwrap();
assert!(!output.status.success());
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(
stderr.contains("OMNIGRAPH_NO_LEGACY_CONFIG") && stderr.contains("config migrate"),
"{stderr}"
);
// Migrated setup (no file): strict mode is a no-op — a config-loading
// command that tolerates empty defaults succeeds.
let clean = tempdir().unwrap();
let output = cli()
.current_dir(clean.path())
.env("OMNIGRAPH_NO_LEGACY_CONFIG", "1")
.arg("queries")
.arg("list")
.arg("--json")
.output()
.unwrap();
assert!(output.status.success(), "{output:?}");
}