fix(engine): reject non-finite and zero-norm embeddings

Validate every component is_finite() (naming the offending index) and
reject zero-norm vectors before normalizing, via the existing
Err(String) channel — callers already map it through EmbedCallError
{retryable:false} into OmniError::manifest_internal, so no new error
plumbing. The red tests turn green; docs state the finite/non-zero
contract in embeddings.md.

Closes iss-embedding-nan-validation.
This commit is contained in:
aaltshuler 2026-07-05 05:06:44 +03:00 committed by Andrew Altshuler
parent 3b00d81626
commit 10835ca788
2 changed files with 23 additions and 1 deletions

View file

@ -13,7 +13,10 @@ query vectors and document vectors share one model and one vector space.
| `gemini` | `POST {base}/models/{model}:embedContent`, `x-goog-api-key`, with `RETRIEVAL_QUERY` / `RETRIEVAL_DOCUMENT` task types | Reaching Google's `generativelanguage` API directly |
| `mock` | none — deterministic offline vectors | Tests and local dev without a key |
Vectors are stored L2-normalized as `FixedSizeList(Float32, dim)`; the requested output dimension is driven by
Vectors are stored L2-normalized as `FixedSizeList(Float32, dim)`. Embedding
values must be finite — a vector containing NaN/Inf, or an all-zero vector
(no direction under cosine distance), is rejected at the client boundary
rather than persisted; the requested output dimension is driven by
the target column width and sent as Gemini `outputDimensionality` / OpenAI `dimensions`.
## Configuration (cluster)