apunkt/omnigraph
1
0
Fork 0
mirror of https://github.com/ModernRelay/omnigraph.git synced 2026-06-15 01:55:13 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
omnigraph/docs/user/search/embeddings.md
Andrew Altshuler d46e50dd6d
docs(user): restructure user docs into topic sections (Phase 1) (#223) 
Move the 23 flat docs/user/*.md files into topic subdirectories so the
user guide is organized by area (schema, queries, search, branching, cli,
operations, clusters, concepts, reference) instead of a flat list. This is
a pure structural move — whole files relocated, every cross-doc link
recomputed, no prose rewrites or content splits (those follow in Phase 2).

- 19 `git mv`s (install.md, deployment.md stay top-level); history preserved
  (renames detected at 92–100% similarity).
- All intra-doc links, AGENTS.md's topic table (52 pointers), and the
  docs/dev + docs/releases back-links recomputed via relpath from each
  file's new location.
- docs/user/index.md rewritten as a sectioned nav hub.
- Fixed 5 doc-path references in Rust (comments + two user-facing server
  settings error strings) to point at the new locations.

Verified: zero broken .md links across tracked docs; check-agents-md.sh
green (with the untracked scratch docs set aside); touched crates build.

Note: the public site (omnigraph-web) imports docs/ via a flat-only script;
its import-docs.mjs needs a subdir-aware update before the next re-sync.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 13:52:14 +03:00

1.8 KiB
Raw Blame History

Embeddings

OmniGraph has two embedding clients with different defaults and purposes.

Compiler-side client (omnigraph-compiler/src/embedding.rs) — query-time normalization

  • Default model: text-embedding-3-small (OpenAI-style schema)
  • Env: NANOGRAPH_EMBED_MODEL, OPENAI_API_KEY, OPENAI_BASE_URL (default https://api.openai.com/v1), NANOGRAPH_EMBEDDINGS_MOCK, NANOGRAPH_EMBED_TIMEOUT_MS=30000, NANOGRAPH_EMBED_RETRY_ATTEMPTS=4, NANOGRAPH_EMBED_RETRY_BACKOFF_MS=200
  • Methods: embed_text(input, expected_dim), embed_texts(inputs, expected_dim)
  • Mock mode: deterministic FNV-1a + xorshift64 → L2-normalized vectors

Engine-side client (omnigraph/src/embedding.rs) — runtime ingest

  • Model: gemini-embedding-2-preview
  • Env: GEMINI_API_KEY, OMNIGRAPH_GEMINI_BASE_URL (default Google generativelanguage v1beta), OMNIGRAPH_EMBED_TIMEOUT_MS=30000, OMNIGRAPH_EMBED_RETRY_ATTEMPTS=4, OMNIGRAPH_EMBED_RETRY_BACKOFF_MS=200, OMNIGRAPH_EMBEDDINGS_MOCK
  • Two task types: embed_query_text (RETRIEVAL_QUERY) and embed_document_text (RETRIEVAL_DOCUMENT)
  • Exponential backoff with retryable detection (timeouts, 429, 5xx)

Schema integration

Mark a Vector property with @embed("source_text_property"). At ingest, the engine pulls the source text and writes the embedding into the vector column. Stored as L2-normalized FixedSizeList(Float32, dim).

CLI omnigraph embed (offline file pipeline)

Operates on JSONL files (not on a graph). Three modes (mutually exclusive):

  • (default) fill_missing — only embed rows whose target field is empty
  • --reembed-all — overwrite all
  • --clean — strip embeddings

Inputs are either a single seed manifest YAML or --input/--output/--spec. Selectors --type T, --select T:field=value filter rows. Streams JSONL → JSONL.