mirror of
https://github.com/ModernRelay/omnigraph.git
synced 2026-07-12 03:12:11 +02:00
281525cf7a
Lance's scanner defaults to prefilter=false: a filter riding the same scanner as nearest()/bm25() is applied AFTER the ANN/FTS top-k, so `limit k` meant top-k of the whole table and a selective predicate silently starved results (the deny-list's silent-partial-result shape; measured by the nearest-prefilter bench scenario: 20k rows, s=0.05, k=10 -> 1000 matching rows exist, 0 returned). Set prefilter(true) whenever a structured filter is pushed to the scanner: one flag governs both the vector and FTS sources, plain scans ignore it, and it re-enables scalar-index acceleration for the predicate under nearest. The red test turns green: filtered nearest now returns the top-k of MATCHING rows. Docs state the filters-before-search contract explicitly (docs/user/search/index.md). Closes iss-nearest-postfilter-starves-results.
52 lines
2.1 KiB
Markdown
52 lines
2.1 KiB
Markdown
# Search
|
|
|
|
OmniGraph runs vector, full-text, and hybrid search in the same runtime as graph
|
|
traversal — a single [query](../queries/index.md) can combine a vector `nearest`,
|
|
a `bm25` text score, and an `Expand` traversal. Search functions are used inside
|
|
`match` (to filter), or as expressions inside `return` / `order` (to score and
|
|
rank).
|
|
|
|
## Functions
|
|
|
|
| Function | Purpose | Backing index |
|
|
|---|---|---|
|
|
| `nearest($x.vec, $q)` | k-NN vector search (cosine) | vector index (IVF / HNSW) |
|
|
| `search(field, q)` | Generic full-text search | inverted (FTS) index |
|
|
| `fuzzy(field, q [, max_edits])` | Levenshtein-tolerant text search | inverted index |
|
|
| `match_text(field, q)` | Pattern match | inverted index |
|
|
| `bm25(field, q)` | BM25 relevance scoring | inverted index |
|
|
| `rrf(rank_a, rank_b [, k])` | Reciprocal Rank Fusion of two rankings (default `k=60`) | fuses scored rankings |
|
|
|
|
- `nearest()` requires a `limit`. The query vector is resolved from the param map,
|
|
or embedded from a text input at runtime via the configured
|
|
[embedding client](embeddings.md).
|
|
- Match filters apply **before** the search: combining a `match` predicate with
|
|
`nearest()` (or `bm25()`) returns the top-`limit` of the *matching* rows —
|
|
never a post-filtered remainder of the global top-k. A selective filter
|
|
narrows the candidate set; it cannot starve the result count.
|
|
- Scores and ranks propagate as ordinary columns, so you can `return` a score and
|
|
`order` by it.
|
|
|
|
## Hybrid ranking with `rrf`
|
|
|
|
Reciprocal Rank Fusion combines two independent rankings (typically one vector and
|
|
one text) into a single fused ranking, without needing the two score scales to be
|
|
comparable. Rank each retrieval separately, then fuse:
|
|
|
|
```gq
|
|
query hybrid($q: String) {
|
|
match { $d: Document { } }
|
|
return {
|
|
$d,
|
|
rrf( nearest($d.embedding, $q), bm25($d.body, $q) ) as score
|
|
}
|
|
order { score desc }
|
|
limit 10
|
|
}
|
|
```
|
|
|
|
## Indexes and embeddings
|
|
|
|
Search functions only work when the backing index exists — see
|
|
[indexes](indexes.md) for building vector and inverted indexes, and
|
|
[embeddings](embeddings.md) for generating the vectors `nearest` searches over.
|