Wire cluster embedding providers

This commit is contained in:
aaltshuler 2026-06-16 04:02:08 +03:00
parent 8d2128438e
commit 16e4a833c0
16 changed files with 1132 additions and 165 deletions

View file

@ -13,7 +13,8 @@ catalog writes, **graph creation** (a declared graph that does not exist yet
is initialized by apply at the derived root), **schema updates** (soft drops
only), and — behind an explicit, digest-bound **approval** — **graph
deletion**. It does not perform data-loss schema migrations, start servers,
or serve anything it applies: the server still boots from `omnigraph.yaml`.
or run data loads. A server can boot from the applied ledger with
`omnigraph-server --cluster <config-dir | storage-root>`.
## Commands
@ -57,7 +58,7 @@ The exact contract:
## Supported `cluster.yaml`
Stage 3A accepts only this resource subset:
The current config surface accepts this resource subset:
```yaml
version: 1
@ -68,9 +69,18 @@ state:
backend: cluster
lock: true
providers:
embedding:
default:
kind: openai-compatible
base_url: https://openrouter.ai/api/v1
model: openai/text-embedding-3-large
api_key: ${OPENROUTER_API_KEY}
graphs:
knowledge:
schema: knowledge.pg
embedding_provider: default
queries: queries/ # discover every `query <name>` in queries/*.gq
policies:
@ -99,6 +109,17 @@ updates all of its queries together. Paths are relative to the config
directory — the cluster is one explicit folder, so no `./` prefixes are
needed.
`providers.embedding.<name>` defines a query-time embedding provider profile
for cluster-served graphs. A graph opts in with `embedding_provider: <name>`;
bare names normalize to `provider.embedding.<name>`. Supported provider
`kind` values are `openai-compatible` (default/OpenRouter-compatible),
`openai` (OpenAI's own host), `gemini`, and `mock`. Real providers require
`api_key: ${ENV_VAR}`; inline secrets are rejected. The env var is resolved
only when a `--cluster` server boots, so `cluster validate`, `plan`, and
`apply` do not need deployment secrets. `mock` is deterministic and does not
require `api_key`. Vector dimensions stay schema-driven by the target
`Vector(N)` column, not the provider profile.
`storage:` (optional) is the **storage root URI** for everything the cluster
stores — the state ledger, lock, content-addressed catalog, recovery
sidecars, approval artifacts, and the derived graph roots
@ -133,10 +154,12 @@ operation is active.
- stored-query parsing and query-name matching
- stored-query type-checking against the desired schema
- policy `applies_to` graph references
- embedding provider profiles and graph `embedding_provider` references
Fields reserved for later phases, such as `pipelines`, `embeddings`, `ui`,
`aliases`, and `bindings`, fail with a typed diagnostic instead of being
silently ignored.
Fields reserved for later phases, such as `pipelines`, top-level
`embeddings`, `ui`, `aliases`, and `bindings`, fail with a typed diagnostic
instead of being silently ignored. Under `providers`, only `embedding` is
supported today; other provider namespaces fail as unsupported config.
## Planning
@ -156,9 +179,21 @@ resource is planned as a create. If present, the file must use this shape:
"applied_revision": {
"config_digest": "...",
"resources": {
"graph.knowledge": { "digest": "..." },
"schema.knowledge": { "digest": "..." },
"query.knowledge.find_experts": { "digest": "..." },
"provider.embedding.default": {
"digest": "...",
"embedding_profile": {
"kind": "openai-compatible",
"base_url": "https://openrouter.ai/api/v1",
"model": "openai/text-embedding-3-large",
"api_key": "${OPENROUTER_API_KEY}"
}
},
"graph.knowledge": {
"digest": "...",
"embedding_provider": "provider.embedding.default"
},
"policy.base": {
"digest": "...",
"applies_to": ["cluster", "graph.knowledge"]