mirror of
https://github.com/ModernRelay/omnigraph.git
synced 2026-06-18 02:24:27 +02:00
8 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Andrew Altshuler
|
2ed05d2cb1
|
feat(cli): --quiet/--yes globals; echo resolved write target; gate non-local destructive writes (#243)
RFC-011 Decision 9. Writes echo their resolved target + access path to stderr (suppress with --quiet). Destructive writes (cleanup, overwrite load, branch delete) against a non-local scope require consent: --yes, a TTY prompt, or a hard refusal for non-TTY/--json runs. Local file:// writes unaffected. |
||
|
Andrew Altshuler
|
a09045028f
|
feat(cli)!: unify graph selection under --graph; --cluster is a global scope; remove --cluster-graph (#241)
RFC-011: --graph is the single graph selector across server and cluster scopes; --cluster becomes a global scope primitive; --cluster-graph removed. Maintenance dispatch unified through resolve_scope. Wrong-address guard validates each scope flag against the verb it can consume. |
||
|
Andrew Altshuler
|
bc2a989a7b
|
feat(cli)!: remove legacy data-plane addressing (--target, positional http→remote, --as-on-served) (#238)
* feat(cli): --server accepts a literal URL (RFC-011 Decision 2) `resolve_server_flag` now treats a `--server` value containing `://` as a literal base URL (trailing slash trimmed; `--graph` appends `/graphs/<id>`), bypassing the operator-config `servers:` registry; a bare name still resolves through the registry. This is the replacement the upcoming `--uri http(s)://` deprecation points at, and a small ergonomic win on its own (`--server https://host` with no config entry). Token resolution for a literal-URL server falls to the legacy OMNIGRAPH_BEARER_TOKEN chain, same as a positional URL today. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(cli): address the parity-matrix arms with global --store/--server flags Prep for removing the positional-http→remote dispatch. The parity harness addressed both arms with a positional graph right after the verb (`omnigraph <verb> <addr> <args…>`), which only parses for top-level verbs — for nested subcommands (`schema show`, `branch list`, …) the address landed in the subcommand slot and BOTH arms failed identically, so the test passed vacuously (matching exit codes, never comparing output). Address both arms with the global flags instead — local `--store <graph>` (embedded), remote `--server <url>` (served) — appended after the verb + args, valid regardless of nesting. The previously-vacuous nested-verb parity checks now actually compare embedded vs remote (and pass — parity holds), and the remote arm no longer relies on the positional-URL dispatch that's about to be removed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli)!: --as on a served write is a hard error (was a silent no-op) A served write resolves the actor server-side from the bearer token, so `--as` could never set identity there — it was silently ignored. It now errors (in the remote write factory, before any HTTP call), pointing the user at removing `--as` or writing directly with `--store`. Reads don't carry `--as`, so this is write-path only. BREAKING for any script that passed `--as` to a remote write (it was a no-op, so behavior is unchanged except the now-explicit error). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli)!: a positional/--uri http(s):// URL no longer dispatches to a server Remote graphs must be addressed with `--server <url>` (or a named server / a profile binding one). A positional or `--uri` `http(s)://` URL on a data verb now errors instead of silently routing to the remote HTTP client — the scheme no longer carries transport semantics. The discriminator is `via_server`: a remote URL produced by a server scope is fine; a remote URL from a positional/`--uri` source is rejected (`reject_positional_remote` in both GraphClient factories). Storage verbs are unaffected — they already reject remote URIs through `resolve_local_graph` with the existing "direct (storage-native)" error. Migrated the gh-host keyed-credential system test to `--server <url>` (the literal URL still prefix-matches the operator server for token resolution). BREAKING: scripts addressing a server by a bare URL must switch to `--server <url>`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli)!: remove the --target flag (use --store / --profile / --server) Removes the legacy named-graph flag and threads its parameter out of the whole resolver chain. `--target` resolved a graph name through `omnigraph.yaml`'s `graphs:` map; its replacements (`--store <uri>`, `--profile <name>`, `--server <name>`) all ship. - Drops the 22 `target` clap fields + the `--cluster` exclusion that named it. - Threads `target`/`cli_target` out of `resolve_uri`/`resolve_cli_graph`/ `resolve_local_graph`/`resolve_local_uri`/`resolve_storage_uri`/ `resolve_remote_bearer_token`/`apply_server_flag`/`execute_query_lint`/ `resolve_selected_graph`/`resolve_registry_selection_for_list`/ `execute_queries_{validate,list}`, the two `GraphClient` factories, and `ScopeFlags`/`ResolvedScope`. - Keeps the shared `OmnigraphConfig::resolve_target_uri` 3-arg (server boot uses it); the CLI passes None for the explicit-target arm. The `cli.graph` default (omnigraph.yaml bare-command fallback) is unchanged — its removal belongs to the omnigraph.yaml excision. - Operator/file aliases that bind a `graph` name still work: the name is now resolved to a URI inline (a positional URI wins). - Error messages and `--graph`/`--server`/`--store` help text no longer name `--target`; the queries-list selection hint points at `cli.graph`. BREAKING. Tests updated (named-target resolution rewritten onto `cli.graph`; positional-URI tests unchanged). Full omnigraph-cli suite green (228). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(cli): drop --target and positional-http addressing; --as-on-served is an error Update the user docs for the legacy data-plane addressing removals: - the CLI `--target` flag is gone — address graphs with a positional URI, `--store`, `--profile`, or `--server <name|url>`; - a positional `http(s)://` URI no longer dispatches to a server (use `--server`); - `--as` on a served write is now rejected (was a silent no-op). Touches cli/reference.md (addressing intro, capability table, error examples, scopes), cli/index.md (the remote-read example → --server), operations/maintenance + policy, and the cluster docs' data-plane load guidance. The server's own `--target` boot flag is unchanged (server.md untouched). Also fixes a pre-existing broken maintenance link in search/indexes.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(cli): --store is loudly exclusive with a positional URI / --server; test graphs→Served Address two Greptile findings on the RFC-011 slices: - Slice A (P1): `--store` combined with a positional URI silently dropped the URI (`scope.rs` did `store.or(uri)`); `--store` + `--server` errored with a misleading "positional URI" message. Now both combinations fail loudly with a declared `--store is exclusive with a positional URI and --server` error. - Slice B (P2): the `command_capability` unit test never exercised the one Data→Served refinement (`graphs`); added the assertion so deleting that guard can't pass silently. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
7eeced3e88
|
feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local) (#237)
* feat(cli): RFC-011 Slice B — capability vocabulary (any/served/direct/control/local)
User-facing CLI errors and --help now speak a single "capability" vocabulary —
what a command needs — instead of the internal four-plane jargon. Behavior is
unchanged: the --server/--graph allow set is identical (the served-graph
capabilities `any` ∪ `served` = the old `Data` plane, since `graphs` was already
allowed). Only error text and the --help legend change.
- planes.rs: add `Capability { Any, Served, Direct, Control, Local }` derived from
the existing exhaustive `command_plane` classifier (which stays as the drift
guard) plus the one Data→Served refinement (`graphs`). `guard_addressing` now
allows `--server`/`--graph` on `{Any, Served}` and rejects elsewhere with a
capability-worded message. The mapping reflects *current* behavior (`queries
list` → Local, `queries validate` → Direct); it converges to the RFC end-state
table when later slices re-route those verbs.
- scope.rs: `resolve_scope` takes `Capability` instead of `Plane`, so the whole
addressing path speaks one vocabulary; call sites in client.rs (Any) and the 3
maintenance verbs in main.rs (Direct) updated.
- helpers.rs: the storage-direct remote rejection reworded to "direct
(storage-native) command".
- cli.rs: the --help legend is now "COMMANDS BY CAPABILITY".
- Tests: the 5 assertions pinning the old plane text updated; added planes.rs unit
tests proving the allow set is exactly {Any, Served} (behavior-preservation),
the per-verb mapping, and distinct capability phrases.
Full omnigraph-cli suite: 225 green (222 + 3 new), zero behavior-test changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(cli): capability vocabulary in the CLI reference + maintenance addressing
Rename the reference's "Command planes" section to "Command capabilities"
(any/served/direct/control/local), reword the error examples, and update the
maintenance doc's addressing note + its section cross-link to match Slice B.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Ragnor Comerford
|
1bed998052
|
fix(engine): scalar index coverage + filter literal coercion (query latency) (#216)
* fix(engine): lower date/datetime filter literals as typed Arrow scalars `literal_to_expr` lowered `Date`/`DateTime` query literals as Utf8 strings, relying on DataFusion implicit casts. Against a physical `Date32`/`Date64` column that can coerce the column side (`CAST(col AS Utf8)`), which defeats a scalar BTREE and degrades the scan to a full filtered read. Lower to typed `Date32`/`Date64` scalars instead (reusing the loader's `parse_date32_literal`/`parse_date64_literal`, already used by the in-memory comparison arm), so the predicate stays a direct column comparison and the index is used. Malformed literals fall back to the Utf8 string so pushdown behavior never regresses. Tests: unit goldens asserting the lowered literal is typed (red before, green after) + inline-binding pushdown equality in literal_filters confirming the epoch conversion selects the right rows. * fix(engine): build scalar BTREE for enum and orderable-scalar @index columns `build_indices_on_dataset_for_catalog` only handled `String` (-> FTS) and `Vector` (-> vector). Enums are physically `String`, so an enum `@index` column (e.g. `status`) got an FTS inverted index, which Lance never consults for `=`; and `DateTime`/`Date`/numeric/`Bool` `@index` columns fell through and built nothing. Both meant equality/range filters degraded to full scans with `indices_loaded=0`. Dispatch index kind by property type via a shared `node_prop_index_kind`: enum + orderable scalar -> BTREE, free-text String -> FTS, Vector -> vector, list/Blob -> none. The helper is shared by the builder and `needs_index_work_node` so they cannot drift — the latter decides recovery- sidecar pinning, and under-reporting would leave a HEAD-advancing index build uncovered (invariant 5). Tests: scalar_indexes.rs asserts enum/DateTime/numeric @index columns report `IndexCoverage::Indexed` while free-text String/un-annotated columns stay `Degraded` (negative control). Docs: docs/user/indexes.md. * feat(engine): reindex in optimize to keep index coverage current A scalar/FTS/vector index only covers the fragments it was built over. Rows appended after the build (e.g. `ingest --mode merge`, whose commit does not rebuild an existing index) are scanned unindexed, and `compact_files` rewrites fragments out of coverage. Nothing folded them back in, so coverage decayed as the graph grew — even the id/src/dst BTREEs that power traversal. `optimize_one_table` now runs Lance `optimize_indices` after `compact_files` (incremental merge, not retrain — the same compact->optimize_indices sequence LanceDB's `optimize()` uses) and enters the publish path on compaction work OR stale index coverage (new `TableStore::has_unindexed_fragments`, reusing the fragment_bitmap logic). `optimize_indices` is a committing call with no uncommitted variant in lance-6.0.1, so it is an inline-commit residual covered by the existing `SidecarKind::Optimize` recovery sidecar spanning both ops. Blob-bearing tables are still skipped (the Lance blob-compaction bug is compaction-specific; reindex-for-blob deferred as a noted follow-up). Tests: maintenance.rs asserts an appended fragment is uncovered before and covered after optimize, and idempotency holds (second pass is a no-op). lance_surface_guards pins the `optimize_indices` signature and its incremental- coverage behavior. The existing optimize Phase-B recovery failpoint now also exercises a crash after reindex. Docs: maintenance.md, writes.md, invariants.md, lance.md, AGENTS.md. * fix(engine): coerce pushdown filter literals to the column type Filter literals were pushed to Lance in their natural Arrow type (every integer Int64, every float Float64). Against a narrower indexed column DataFusion widens to the literal's type and casts the COLUMN (`CAST(n32 AS Int64)`), which defeats the scalar BTREE and degrades to a full filtered read. A physical-plan probe confirms it: an Int32 column filtered by an i32 literal uses `ScalarIndexQuery`; by an i64 literal it does not. Thread the scan's `arrow_schema` through `build_lance_filter_expr` -> `ir_filter_to_expr` and coerce each literal operand to the opposite column's exact Arrow type, reusing `projection::literal_to_array` + `arrow_cast` (the same path the in-memory arm uses, so the two arms agree). Coercion never demotes a filter to None: on failure it falls back to the natural literal, because a node scan has no in-memory fallback for inline filters. Supersedes the date-specific change in |
||
|
Andrew Altshuler
|
77dffdae92
|
docs(user): de-dev polish — strip internal scaffolding from user docs (Phase 3a) (#226)
Remove developer-only scaffolding that leaked into the public user/operator docs, while preserving every user-facing behavior, command, flag, endpoint, constant, and env var. No behavior changes. Removed across 18 files: - internal ticket / sequencing refs (MR-NNN, RFC-NNN, "Phase N"); - source-code paths (crates/**/*.rs, *.pest) and internal struct/function dumps (e.g. the QueryIR / GraphCommit / SchemaMigrationPlan Rust types, internal fn names like fork_branch_from_state, optimize_all_tables); - Lance-internal blocker prose (upstream issue numbers, blob-decode cause, sidecar Phase-B/C mechanics) — keeping the user-visible behavior (e.g. "optimize skips Blob-column tables; reads/writes unaffected"); - pre-v0.4.0 Run-state-machine archaeology. Internal IR/lowering/recovery-internals sections were either trimmed to a brief user-facing note (e.g. "Traversal execution", "interrupted writes recover automatically; recovery commits are recorded under actor omnigraph:recovery") or removed. Kept: all language syntax, lint codes, Cedar actions/scopes, endpoints, error taxonomy, every constant and env var (verified none dropped from the constants cheat-sheet), and the operator-facing explanations of on-disk artifacts. Residual "legacy" mentions are all user-facing (the deprecated omnigraph.yaml, the legacy token chain, old command names). Verified: zero internal-scaffolding leaks (MR/RFC/Phase/.rs/.pest = 0) across docs/user; zero broken links; check-agents-md.sh green. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
612741b387
|
docs(user): split language/branching pages + add front-door pages (Phase 2) (#225)
Content build-out on top of the Phase 1 topic move. No behavior changes. Splits (existing content relocated, cross-linked): - queries/index.md → mutations/index.md (insert/update/delete + the inserts-vs-deletes rule) and search/index.md (the multi-modal search functions + a hybrid-ranking overview tying nearest/bm25/rrf together). queries/index.md now covers the read shape and points at both. - branching/index.md → branching/time-travel.md (snapshots/time travel) and branching/merge.md (three-way merge + the 7 conflict kinds, verified against error.rs MergeConflictKind). New pages (written from the code, user-facing): - quickstart.md — init → load → query → branch, with verified CLI flags. - concepts/index.md — what OmniGraph is + the L1/L2 (Lance/OmniGraph) framing. Expanded operations/audit.md from a 7-line struct dump into a real actor-tracking page (server token-resolved vs CLI --as chain; reading the trail; the omnigraph:recovery reserved actor). Index wiring: docs/user/index.md and AGENTS.md's topic table link every new page; also normalized AGENTS.md's docs/user link display text to match the Phase 1 retargeted paths. Verified: zero broken .md links; check-agents-md.sh green (57 links, 54 docs). Deferred to Phase 3: de-dev polish (grammar paths, IR internals still in queries/branching), guides/, and a possible reference/config.md split (the config schema is already coherent in cli/reference.md). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Andrew Altshuler
|
d46e50dd6d
|
docs(user): restructure user docs into topic sections (Phase 1) (#223)
Move the 23 flat docs/user/*.md files into topic subdirectories so the user guide is organized by area (schema, queries, search, branching, cli, operations, clusters, concepts, reference) instead of a flat list. This is a pure structural move — whole files relocated, every cross-doc link recomputed, no prose rewrites or content splits (those follow in Phase 2). - 19 `git mv`s (install.md, deployment.md stay top-level); history preserved (renames detected at 92–100% similarity). - All intra-doc links, AGENTS.md's topic table (52 pointers), and the docs/dev + docs/releases back-links recomputed via relpath from each file's new location. - docs/user/index.md rewritten as a sectioned nav hub. - Fixed 5 doc-path references in Rust (comments + two user-facing server settings error strings) to point at the new locations. Verified: zero broken .md links across tracked docs; check-agents-md.sh green (with the untracked scratch docs set aside); touched crates build. Note: the public site (omnigraph-web) imports docs/ via a flat-only script; its import-docs.mjs needs a subdir-aware update before the next re-sync. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |