omnigraph/docs/releases/v0.8.1.md

# Omnigraph v0.8.1

A reliability and features release. The storage substrate moves to Lance
9.0.0-beta.15, adopting two upstream fixes that matter for embedding-heavy
workloads; filtered vector search now applies filters before the search;
schema migration learns enum widening; and the query language gains
undirected edge traversal. **The on-disk format is unchanged (internal schema
v4)** — v0.8.0 graphs are served as-is, no rebuild.

**Channel note:** this release ships as binaries, Homebrew, and the
installer. crates.io publication is deferred for one release: the substrate
is pinned to a Lance pre-release, which exists as a git tag rather than a
registry version, and published crates can only reference registry versions.
`cargo install` users can build the tag directly —
`cargo install --git https://github.com/ModernRelay/omnigraph --tag v0.8.1 omnigraph-cli`
— or use the installer. Registry publication resumes at v0.9.0 with Lance
9.0.0 stable.

## Highlights

- **Storage substrate: Lance 9.0.0-beta.15.** The Lance team resolved two
  issues upstream that this release adopts (lance#7480, lance#7320); both
  could surface on embedding-heavy graphs under routine write-then-maintain
  patterns. If you have seen filtered reads fail with a record-batch length
  error after refresh-style updates and deletes, or keyed lookups fail after
  `optimize` on a recently-updated table, upgrading the binary resolves both
  — **the data on disk was always intact, and no repair step is needed.**
  The interim vendored patch omnigraph carried for the first issue is
  retired now that the fix is upstream. The bump was validated against a
  full review of the upstream changes (382 commits) plus the complete local
  and CI test matrix, including S3 integration.

- **Filtered vector search returns what it should.** Combining a `match`
  filter with `nearest()` (or `bm25()`) previously applied the filter after
  the ANN top-k, so a selective filter could return far fewer rows than
  requested. Filters now apply *before* the search: `limit k` means the
  top-k of *matching* rows. Filtered search results will change — they are
  now correct.

- **Undirected edge traversal.** `$a <edgeName> $b` matches an edge in either
  direction with set semantics (a pair connected both ways appears once), for
  same-endpoint-type edges (e.g. `Related: Issue -> Issue`); asymmetric edges
  are rejected at typecheck (T22). Composes with hop bounds
  (`$a <knows>{1,3} $b`) and `not { }` ("no edge in either direction"). One
  pattern replaces the query-both-directions-and-merge workaround.

- **Enum widening in schema migration.** Adding variants to an `enum(...)`
  property is now a supported, metadata-only migration step — `schema plan`
  shows `extend enum`, `apply` touches no table data, and the new variants
  are accepted immediately on every write surface. Narrowing, variant
  renames, and enum↔String conversions still refuse (OG-MF-106).

- **Embedding validation hardened.** Non-finite embeddings (NaN/Inf) and
  zero vectors are rejected at the client boundary instead of being stored;
  the JSONL loader rejects non-numeric vector elements (previously coerced
  to 0.0); and forcing the mock embedding provider over an explicitly
  configured real one now logs a warning.

- **Blob-bearing tables rejoin maintenance.** With Lance 9's blob-column
  compaction support, `optimize` now includes tables with `Blob` properties
  — fragment reclamation and index folding cover them like any other table.

- **Self-service upgrade refusals.** Opening a graph from an older storage
  format now names the release line that wrote it (e.g. "created by omnigraph
  0.6.2 to 0.7.2") and the exact export → init → load commands, instead of
  asking the operator to identify the right binary themselves.

## Behavior changes

- Filtered `nearest()`/`bm25()` result sets change (see above) — previously
  missing results now appear.
- New inverted indexes are written in Lance's FTS v2 format (readable by
  omnigraph ≥ 0.8.0 binaries), and text-search `AND` semantics were refined
  upstream, so `match_text`/`bm25` rankings may shift slightly.
- Loads that previously succeeded with non-numeric vector elements or
  non-finite embeddings now fail loudly at write time.

## Upgrade notes

- **No storage-format change.** v0.8.0 graphs open unchanged; downgrade to
  v0.8.0 remains possible (same internal schema v4).
- Upgrading the binary is the complete fix for the two substrate issues
  above — no data repair, no rebuild.
- The substrate is a rev-pinned Lance pre-release (9.0.0-beta.15), adopted
  ahead of the stable release to deliver the fixes above promptly. v0.9.0
  moves to Lance 9.0.0 stable.

## Developer-facing

- Scenario benchmark harness (`cargo bench -p omnigraph-engine --bench
  scenarios`): cold subprocess runs measuring wall-clock + peak RSS, JSON-line
  results with a persistent log — a decision instrument, never a CI gate.
- A gated cross-version upgrade test (`OMNIGRAPH_OLD_BIN`) proves a genuine
  v3-format graph is refused with the release-named message and rebuilds via
  the documented path; run on demand, not in CI.
- One dataset-open chokepoint (`instrumentation::open_dataset`) with the
  shared per-graph Lance `Session` attached to write-side opens — the local
  merge-scan cost term is now flat in commit history.