` cast in `tests/phase_1/embedder_trait.rs` fails
- to resolve after the rewrite.**
- Mitigation: the rewrite preserves the `Embedder` symbol at the same
- module path; only its provenance changes (now generated by
- `trait_variant::make` instead of by `pub use LocalEmbedder as
- Embedder;`). The macro is specifically designed so that the generated
- trait is dyn-compatible at the Send-bound boundary. Verified by the
- identical pattern already working for `MemoryStore` after `0001a`.
-- **`cargo rm async-trait` updates `Cargo.lock` but accidentally bumps
- other crates.**
- Mitigation: run `cargo rm async-trait` and then immediately inspect
- the resulting `Cargo.lock` diff. The expected diff is the removal of
- the `[[package]] name = "async-trait"` block and its hash. Anything
- else is a red flag and should be reverted before committing
- (`git checkout -- Cargo.lock` then `cargo update -p async-trait
- --precise=remove` -- or fall back to manual edit + `cargo check`).
-- **A new workspace crate added in parallel with this work declares
- `async-trait` and the dependency removal silently re-introduces it
- later.**
- Mitigation: the verification step `grep -rn "async-trait"
- --include="Cargo.toml" crates/` is part of the acceptance criteria; a
- rebase that reintroduces the line will fail this gate.
-- **MCP server uses `Embedder` somewhere we missed.**
- Mitigation: full workspace grep (`grep -rn "Embedder" crates/`)
- returns no hits inside `crates/vestige-mcp/` for the trait names; the
- MCP layer uses the concrete `EmbeddingService` from
- `crates/vestige-core/src/embeddings/` for ad-hoc embedding calls. The
- trait surface is purely internal to `vestige-core`.
-
----
-
-## Out-of-Band Notes
-
-- **No other workspace crate declares `async-trait` as a direct
- dependency.** Verified by
- `grep -rn "async-trait" --include="Cargo.toml" crates/` returning
- exactly one hit at `crates/vestige-core/Cargo.toml:119`. There is
- nothing to clean up in `crates/vestige-mcp/Cargo.toml` or elsewhere.
-- **Order matters across the three Phase 1 amendment sub-plans:**
- `0001a` (trait rewrite) -> `0001b` (sqlite split) -> `0001c` (this
- one, async-trait sunset). Reversing the order is possible in
- principle but would force re-editing the embedder rewrite twice and
- leaves the `async-trait` dep behind until very late.
-- **This sub-plan amends `feat/storage-trait-phase1` (tip 790c0c8 plus
- whatever commits `0001a` and `0001b` added).** The branch has not
- been opened upstream yet, so amending in place is safe; no force-push
- to a public PR.
-- **After this sub-plan lands, the branch is reviewed and merged before
- Phase 2 sub-plans (`0002a-` through `0002i-`) begin implementation.**
- Phase 2 introduces no async-trait usage; the Postgres backend follows
- the same `trait_variant::make` pattern (see ADR 0002 D1).
-- **`trait-variant` 0.1 stays in `Cargo.toml`.** It is the only crate
- this sub-plan keeps; `async-trait` is the only one it removes.
-
----
-
-## Self-Contained `/goal` Brief
-
-For a fresh Claude Code session executing this sub-plan without prior
-conversation context:
-
-1. Check out branch `feat/storage-trait-phase1` (or a worktree off
- of it after `0001a` and `0001b` are merged into it).
-2. Read this file (`docs/plans/0001c-async-trait-sunset.md`) in full.
-3. Read `docs/plans/0001a-trait-rewrite.md` sections "Trait declaration
- rewrite" and "Impl block migration" -- they document the exact
- pattern this sub-plan mirrors for the embedder.
-4. Run the prerequisite audit grep listed under "Prerequisites". If it
- returns more than the five hits documented there, stop and report;
- the upstream state does not match what this sub-plan assumes.
-5. Execute Commit 1 (rewrite `embedder/mod.rs`), then Commit 2 (drop
- the attribute on the FastembedEmbedder impl), then Commit 3
- (`cargo rm async-trait`). Run the verification commands listed
- above after each commit; do not proceed if any test or clippy gate
- fails.
-6. Verify every box in "Acceptance Criteria" is ticked.
-7. Report file paths touched, test counts, and the final two grep
- results (commands 7 and 8 from "Verification") in the closing
- message.
diff --git a/docs/plans/0002-phase-2-postgres-backend.md b/docs/plans/0002-phase-2-postgres-backend.md
deleted file mode 100644
index ed2a186..0000000
--- a/docs/plans/0002-phase-2-postgres-backend.md
+++ /dev/null
@@ -1,1278 +0,0 @@
-# Phase 2 Plan: PostgreSQL Backend
-
-> **Supersession Notice (2026-05-26):** This master plan is now archival. Execution is governed by:
-> - **ADR**: [`docs/adr/0002-phase-2-execution.md`](../adr/0002-phase-2-execution.md) -- binding decisions
-> - **Sub-plans** (executable briefs):
-> - Phase 1 amendment: [0001a-trait-rewrite.md](0001a-trait-rewrite.md), [0001b-sqlite-split.md](0001b-sqlite-split.md), [0001c-async-trait-sunset.md](0001c-async-trait-sunset.md)
-> - Phase 2: 0002a..0002i (skeleton, pool+config, migrations, store impl, hybrid search, migrate CLI, reembed, tests+benches, runbook)
->
-> **Deltas vs body**: trait uses `trait_variant`, error type is `MemoryStoreError`/`MemoryStoreResult`, `connect` is `(url, max_connections)` only, the core table is `knowledge_nodes` (not `memories`) and gains `owner_user_id` + `visibility` + `shared_with_groups` + `codebase`, plus `users`/`groups`/`group_memberships` tables. See ADR 0002 D1-D8.
-
-**Status**: Draft
-**Depends on**: Phase 1 (MemoryStore + Embedder traits, embedding_model registry, domain columns)
-**Related**: docs/adr/0001-pluggable-storage-and-network-access.md (Phase 2), docs/prd/001-getting-centralized-vestige.md, docs/plans/local-dev-postgres-setup.md (local cluster provisioning)
-
----
-
-## Scope
-
-### In scope
-
-- `PgMemoryStore` struct implementing the Phase 1 `MemoryStore` trait against `sqlx::PgPool`, including compile-time checked queries via `sqlx::query!` / `sqlx::query_as!`.
-- First-class `pgvector` integration: typed `Vector` columns, HNSW index (`vector_cosine_ops`, `m = 16`, `ef_construction = 64`), and use of the cosine-distance operator `<=>`.
-- First-class Postgres FTS: GENERATED `tsvector` column (`search_vec`) with `setweight` (A=content, B=node_type, C=tags), GIN index, and `websearch_to_tsquery` at query time.
-- Hybrid search via Reciprocal Rank Fusion (RRF) expressed as a single SQL statement with CTEs for FTS and vector subqueries, with optional domain filter through array overlap (`&&`).
-- sqlx migrations directory at `crates/vestige-core/migrations/postgres/`, numbered `{NNNN}_{name}.up.sql` / `{NNNN}_{name}.down.sql`, runnable by `sqlx::migrate!` at startup and by `sqlx-cli`.
-- Offline query cache committed under `crates/vestige-core/.sqlx/` so a DATABASE_URL is not required at build time.
-- Backend selection via `vestige.toml`: `[storage]` section with `backend = "sqlite" | "postgres"` plus the per-backend subsection (`[storage.sqlite]`, `[storage.postgres]`). Exclusive at compile time via `postgres-backend` feature, exclusive at runtime via the enum.
-- CLI: `vestige migrate --from sqlite --to postgres --sqlite-path --postgres-url ` -- streaming copy with progress output.
-- CLI: `vestige migrate --reembed --model=` -- O(n) re-embed under a new `Embedder`, registry update, HNSW rebuild.
-- Testcontainer-based integration tests using the `pgvector/pgvector:pg16` image, behind the `postgres-backend` feature so SQLite-only builds remain untouched.
-- `PgMemoryStore` parity with `SqliteMemoryStore` across every public `MemoryStore` method defined in Phase 1.
-
-### Out of scope
-
-- Phase 3 (network access): HTTP MCP transport, API key auth, `vestige keys` CLI. The `api_keys` DDL is declared by Phase 3; Phase 2 does not create it.
-- Phase 4 (emergent domain classification): `DomainClassifier`, HDBSCAN, discover / rename / merge CLI. Phase 2 provisions the `domains` and `domain_scores` columns and the `domains` table structure so Phase 4 slots in without further migration, but does not compute or classify.
-- Phase 5 (federation): cross-node sync. The `review_events` table is declared in Phase 1; Phase 2 only references it where FSRS writes happen.
-- Changes to the cognitive engine, Phase 1 traits, or the embedding pipeline itself. Phase 2 only adds a backend.
-- SQLCipher parity for Postgres. Operator responsibility (TLS to Postgres, pgcrypto, disk-level encryption) is out of scope for this phase.
-
----
-
-## Prerequisites
-
-### Expected Phase 1 artifacts (consumed, not produced)
-
-Phase 2 treats all of the following as fixed interfaces. Each path is the expected Phase 1 location.
-
-- `crates/vestige-core/src/storage/mod.rs` -- re-exports the trait and the two concrete backends.
-- `crates/vestige-core/src/storage/memory_store.rs` -- defines the `MemoryStore` trait (generated by `trait_variant::make` from `LocalMemoryStore`) with the full CRUD, search, FSRS, graph, and domain surface from the PRD. Phase 2 implements every method here.
-- `crates/vestige-core/src/storage/types.rs` -- shared value types: `MemoryRecord`, `SchedulingState`, `SearchQuery`, `SearchResult`, `MemoryEdge`, `Domain`, `StoreStats`, `HealthStatus`.
-- `crates/vestige-core/src/storage/error.rs` -- `StoreError` enum plus `pub type StoreResult = Result`. Phase 2 extends this with `StoreError::Postgres(sqlx::Error)` and `StoreError::Migrate(sqlx::migrate::MigrateError)` via `From` impls (the variants themselves MUST live behind `#[cfg(feature = "postgres-backend")]`).
-- `crates/vestige-core/src/embedder/mod.rs` -- `Embedder` trait with `embed`, `model_name`, `dimension`, `model_hash`. Phase 2 calls `model_name()`, `dimension()`, and `model_hash()` for the registry.
-- `crates/vestige-core/src/storage/sqlite.rs` -- `SqliteMemoryStore: MemoryStore`. Phase 2's `migrate --from sqlite --to postgres` uses this as the source.
-- `crates/vestige-core/src/storage/registry.rs` -- `EmbeddingModelRegistry` abstraction that both backends implement. Phase 2 supplies a Postgres version writing to `embedding_model`.
-- `crates/vestige-core/migrations/sqlite/` -- V12 (Phase 1) adds `domains TEXT` (JSON-encoded array), `domain_scores TEXT` (JSON), `embedding_model(name, dimension, hash, created_at)`, and `review_events(id, memory_id, timestamp, rating, prior_state, new_state)`. Phase 2 mirrors every column and table in Postgres.
-
-If any of the above is missing when Phase 2 starts, the first action is to surface the gap back to Phase 1 -- do NOT backfill a partial trait in Phase 2.
-
-### Required crates (declared in Phase 2, not installed by this doc)
-
-The agent running Phase 2 uses `cargo add` in `crates/vestige-core/` for each dependency below. Exact versions and feature flags:
-
-- `sqlx@0.8` with features `runtime-tokio`, `tls-rustls`, `postgres`, `uuid`, `chrono`, `json`, `migrate`, `macros`. Optional (gated by `postgres-backend`).
-- `pgvector@0.4` with features `sqlx`. Optional (gated by `postgres-backend`).
-- `deadpool` is NOT needed; `sqlx::PgPool` is the pool.
-- `toml@0.8` (no features) for `vestige.toml` parsing. Moved to non-optional because both backends share the config surface.
-- `figment@0.10` with features `toml`, `env` -- optional, only if Phase 1 has not already picked a config loader. If Phase 1 ships a loader, skip `figment` and reuse.
-- `dirs@6` -- already a transitive `directories` dependency; reuse existing.
-- `tokio-stream@0.1` (no features). Used by migrate commands for streamed iteration.
-- `indicatif@0.17` (no features). Progress bars for the migrate CLI.
-- `futures@0.3` with features `std`. Consumed by sqlx stream combinators.
-
-Dev-only (under `[dev-dependencies]` in `crates/vestige-core/Cargo.toml`, gated by `postgres-backend`):
-
-- `testcontainers@0.22` with features `blocking` off, `async` on (default).
-- `testcontainers-modules@0.10` with features `postgres`.
-- `tokio@1` features `macros`, `rt-multi-thread` (already present for core tests).
-- `criterion@0.5` already present; add a new `[[bench]]` entry.
-
-Feature additions in `crates/vestige-core/Cargo.toml`:
-
-```
-[features]
-postgres-backend = ["dep:sqlx", "dep:pgvector", "dep:tokio-stream", "dep:futures"]
-```
-
-`postgres-backend` is OFF by default. `default = ["embeddings", "vector-search", "bundled-sqlite"]` stays unchanged. `vestige-mcp` forwards a new feature `postgres-backend = ["vestige-core/postgres-backend"]`.
-
-### External tooling
-
-- PostgreSQL 16 or newer (uses `gen_random_uuid()` from `pgcrypto` bundled via `CREATE EXTENSION pgcrypto` in migration 0001; pgvector HNSW indexes require pgvector 0.5+).
-- The `pgvector` extension installed in the target database (our migration issues `CREATE EXTENSION IF NOT EXISTS vector`).
-- `sqlx-cli@0.8` installed on the developer machine for `cargo sqlx prepare --workspace` and `cargo sqlx migrate add` (not a build-time requirement once `.sqlx/` is committed).
-- Docker or Podman reachable by the test harness for `testcontainers-modules::postgres` to spin up `pgvector/pgvector:pg16`.
-- A local Postgres cluster for `sqlx prepare`, manual migration work, and `vestige migrate --to postgres` smoke runs. The recipe for standing one up on Arch/CachyOS (install, initdb, role + db, pgvector, connection string at `~/.vestige_pg_pw`) lives in `docs/plans/local-dev-postgres-setup.md`. Postgres 18 from the Arch repo satisfies the "16 or newer" requirement above. Phase 2 work assumes `DATABASE_URL` points at that cluster once migrations are applied.
-
-### Assumed Rust toolchain
-
-- Rust 2024 edition.
-- MSRV 1.91 (per `CLAUDE.md`). `sqlx 0.8` is compatible.
-- `rustflags` unchanged. No `nightly`-only features.
-
----
-
-## Deliverables
-
-1. Feature gate `postgres-backend` in `crates/vestige-core/Cargo.toml` and `crates/vestige-mcp/Cargo.toml` that cleanly disables all Postgres code paths when off.
-2. `crates/vestige-core/src/storage/postgres/mod.rs` -- `PgMemoryStore` struct and `MemoryStore` trait impl (public entry point).
-3. `crates/vestige-core/src/storage/postgres/pool.rs` -- `PgMemoryStore::connect(config)` and pool configuration.
-4. `crates/vestige-core/src/storage/postgres/search.rs` -- RRF hybrid search query builder and row -> `SearchResult` mapping.
-5. `crates/vestige-core/src/storage/postgres/migrations.rs` -- wraps `sqlx::migrate!("./migrations/postgres")` and surfaces typed errors.
-6. `crates/vestige-core/src/storage/postgres/registry.rs` -- Postgres `EmbeddingModelRegistry` implementation writing `embedding_model`.
-7. `crates/vestige-core/migrations/postgres/0001_init.up.sql` + `0001_init.down.sql` -- extensions, `memories`, `scheduling`, `edges`, `domains`, `embedding_model`, `review_events`, all indexes.
-8. `crates/vestige-core/migrations/postgres/0002_hnsw.up.sql` + `0002_hnsw.down.sql` -- HNSW index creation separated so it can be `CREATE INDEX CONCURRENTLY` during reembed.
-9. `crates/vestige-core/src/config.rs` -- `VestigeConfig`, `StorageConfig`, `SqliteConfig`, `PostgresConfig`, `EmbeddingsConfig`, plus a single `VestigeConfig::load(path: Option<&Path>)` returning `Result`.
-10. `crates/vestige-core/src/storage/postgres/migrate_cli.rs` -- streaming SQLite-to-Postgres copy, domain-aware, with `indicatif` progress.
-11. `crates/vestige-core/src/storage/postgres/reembed.rs` -- `ReembedPlan` and its driver; re-encodes all memories via a supplied `Embedder`, updates `embedding_model`, rebuilds HNSW.
-12. `crates/vestige-mcp/src/bin/cli.rs` -- two new `clap` subcommands `Migrate` (union of `--from/--to` and `--reembed` variants, one subcommand or two, see Open Questions) wired to deliverables 10 and 11.
-13. `crates/vestige-core/.sqlx/` -- offline query cache, committed.
-14. `tests/phase_2/` -- six integration test files listed in the Test Plan.
-15. `crates/vestige-core/benches/pg_hybrid_search.rs` -- Criterion benches for RRF search at 1k and 100k memories, gated by `postgres-backend`.
-16. `docs/runbook/postgres.md` -- brief ops note covering extension install, `max_connections`, backup discipline, and rollback caveats. (Short; only required for the "rollback of migrate" deliverable.)
-
----
-
-## Detailed Task Breakdown
-
-### D1. `postgres-backend` feature gate
-
-- **File**: `crates/vestige-core/Cargo.toml`, `crates/vestige-mcp/Cargo.toml`
-- **Depends on**: nothing; this is the first change.
-- **Rust snippets**:
-
-```toml
-# crates/vestige-core/Cargo.toml
-[features]
-default = ["embeddings", "vector-search", "bundled-sqlite"]
-bundled-sqlite = ["rusqlite/bundled"]
-encryption = ["rusqlite/bundled-sqlcipher"]
-postgres-backend = [
- "dep:sqlx",
- "dep:pgvector",
- "dep:tokio-stream",
- "dep:futures",
-]
-
-[dependencies]
-sqlx = { version = "0.8", default-features = false, features = [
- "runtime-tokio", "tls-rustls", "postgres", "uuid", "chrono",
- "json", "migrate", "macros",
-], optional = true }
-pgvector = { version = "0.4", features = ["sqlx"], optional = true }
-tokio-stream = { version = "0.1", optional = true }
-futures = { version = "0.3", optional = true }
-toml = "0.8"
-indicatif = "0.17"
-```
-
-- **Behavior notes**: keep the two backends mutually compilable per `CLAUDE.md`. Every `use sqlx::...` sits under `#[cfg(feature = "postgres-backend")]`. Every module under `crates/vestige-core/src/storage/postgres/` carries `#![cfg(feature = "postgres-backend")]` as its file-level attribute.
-
-### D2. `PgMemoryStore` core struct
-
-- **File**: `crates/vestige-core/src/storage/postgres/mod.rs`
-- **Depends on**: D1, Phase 1 `MemoryStore` trait and value types.
-- **Signatures**:
-
-```rust
-#![cfg(feature = "postgres-backend")]
-
-use std::sync::Arc;
-use std::time::Duration;
-
-use chrono::{DateTime, Utc};
-use pgvector::Vector;
-use sqlx::postgres::{PgConnectOptions, PgPoolOptions};
-use sqlx::PgPool;
-use uuid::Uuid;
-
-use crate::embedder::Embedder;
-use crate::storage::error::{StoreError, StoreResult};
-use crate::storage::types::{
- Domain, HealthStatus, MemoryEdge, MemoryRecord, SchedulingState,
- SearchQuery, SearchResult, StoreStats,
-};
-use crate::storage::memory_store::LocalMemoryStore;
-
-pub mod migrations;
-pub mod pool;
-pub mod registry;
-pub mod search;
-pub mod migrate_cli;
-pub mod reembed;
-
-/// Postgres-backed implementation of `MemoryStore`.
-///
-/// Cheaply cloneable. Methods take `&self`; interior state lives inside
-/// the `PgPool` (which already provides `Sync` via `Arc` internally).
-#[derive(Clone)]
-pub struct PgMemoryStore {
- pool: PgPool,
- embedding_dim: i32,
- embedding_model: Arc,
-}
-
-#[derive(Debug, Clone)]
-pub struct EmbeddingModelDescriptor {
- pub name: String,
- pub dimension: i32,
- pub hash: String,
-}
-
-impl PgMemoryStore {
- /// Construct a new store. Runs migrations, reads the registry, validates
- /// that the embedder matches the registered model.
- pub async fn connect(
- url: &str,
- max_connections: u32,
- embedder: &dyn Embedder,
- ) -> StoreResult;
-
- /// Low-level constructor for tests: supply an existing pool, skip migrate.
- pub async fn from_pool(
- pool: PgPool,
- embedder: &dyn Embedder,
- ) -> StoreResult;
-
- /// Accessor used by migrate/reembed CLI.
- pub fn pool(&self) -> &PgPool { &self.pool }
-
- pub fn embedding_dim(&self) -> i32 { self.embedding_dim }
-}
-
-#[trait_variant::make(crate::storage::memory_store::MemoryStore: Send)]
-impl LocalMemoryStore for PgMemoryStore {
- async fn init(&self) -> StoreResult<()>;
- async fn health_check(&self) -> StoreResult;
-
- async fn insert(&self, record: &MemoryRecord) -> StoreResult;
- async fn get(&self, id: Uuid) -> StoreResult>;
- async fn update(&self, record: &MemoryRecord) -> StoreResult<()>;
- async fn delete(&self, id: Uuid) -> StoreResult<()>;
-
- async fn search(&self, query: &SearchQuery) -> StoreResult>;
- async fn fts_search(&self, text: &str, limit: usize) -> StoreResult>;
- async fn vector_search(&self, embedding: &[f32], limit: usize) -> StoreResult>;
-
- async fn get_scheduling(&self, memory_id: Uuid) -> StoreResult>;
- async fn update_scheduling(&self, state: &SchedulingState) -> StoreResult<()>;
- async fn get_due_memories(
- &self,
- before: DateTime,
- limit: usize,
- ) -> StoreResult>;
-
- async fn add_edge(&self, edge: &MemoryEdge) -> StoreResult<()>;
- async fn get_edges(&self, node_id: Uuid, edge_type: Option<&str>) -> StoreResult>;
- async fn remove_edge(&self, source: Uuid, target: Uuid, edge_type: &str) -> StoreResult<()>;
- async fn get_neighbors(&self, node_id: Uuid, depth: usize) -> StoreResult>;
-
- async fn list_domains(&self) -> StoreResult>;
- async fn get_domain(&self, id: &str) -> StoreResult>;
- async fn upsert_domain(&self, domain: &Domain) -> StoreResult<()>;
- async fn delete_domain(&self, id: &str) -> StoreResult<()>;
- async fn classify(&self, embedding: &[f32]) -> StoreResult>;
-
- async fn count(&self) -> StoreResult;
- async fn get_stats(&self) -> StoreResult;
- async fn vacuum(&self) -> StoreResult<()>;
-}
-```
-
-- **SQL (inline within impl methods)**: every call uses `sqlx::query!` or `sqlx::query_as!` for compile-time validation. Examples:
-
-```rust
-// insert
-sqlx::query!(
- r#"
- INSERT INTO memories (
- id, domains, domain_scores, content, node_type, tags,
- embedding, metadata, created_at, updated_at
- ) VALUES ($1, $2, $3, $4, $5, $6, $7::vector, $8, $9, $10)
- "#,
- record.id,
- &record.domains as &[String],
- serde_json::to_value(&record.domain_scores)?,
- record.content,
- record.node_type,
- &record.tags as &[String],
- record.embedding.as_ref().map(|v| Vector::from(v.clone())) as Option,
- record.metadata,
- record.created_at,
- record.updated_at,
-)
-.execute(&self.pool)
-.await?;
-```
-
-- **Behavior notes**:
- - `StoreError` gets two new variants behind the feature:
-
-```rust
-#[cfg(feature = "postgres-backend")]
-#[error("postgres error: {0}")]
-Postgres(#[from] sqlx::Error),
-
-#[cfg(feature = "postgres-backend")]
-#[error("postgres migration error: {0}")]
-Migrate(#[from] sqlx::migrate::MigrateError),
-```
-
- - `classify()` on Postgres implements the PRD's cosine-similarity-to-centroid computation inside SQL using `1 - (centroid <=> $1::vector)` over the `domains` table and returns rows sorted descending. This mirrors the behavior a `DomainClassifier` in Phase 4 uses; Phase 2 ships the backend capability but does not call it.
- - Connection pool defaults (see D3): `max_connections = 10`, `acquire_timeout = 30s`, `idle_timeout = 600s`, `test_before_acquire = false` (cheap queries; avoid per-acquire roundtrip).
- - All methods are `async fn` and use sqlx's `tokio` runtime feature; no blocking `block_on`.
-
-### D3. Pool construction and config wiring
-
-- **File**: `crates/vestige-core/src/storage/postgres/pool.rs`
-- **Depends on**: D1, D2, D9.
-- **Signatures**:
-
-```rust
-#![cfg(feature = "postgres-backend")]
-
-use sqlx::postgres::{PgConnectOptions, PgPoolOptions};
-use sqlx::{ConnectOptions, PgPool};
-use std::str::FromStr;
-use std::time::Duration;
-
-use crate::config::PostgresConfig;
-use crate::storage::error::{StoreError, StoreResult};
-
-pub async fn build_pool(cfg: &PostgresConfig) -> StoreResult {
- let mut opts = PgConnectOptions::from_str(&cfg.url)?;
- opts = opts
- .application_name("vestige")
- .statement_cache_capacity(256)
- .log_statements(tracing::log::LevelFilter::Debug);
-
- let pool = PgPoolOptions::new()
- .max_connections(cfg.max_connections.unwrap_or(10))
- .min_connections(0)
- .acquire_timeout(Duration::from_secs(cfg.acquire_timeout_secs.unwrap_or(30)))
- .idle_timeout(Some(Duration::from_secs(600)))
- .max_lifetime(Some(Duration::from_secs(1800)))
- .test_before_acquire(false)
- .connect_with(opts)
- .await?;
-
- Ok(pool)
-}
-```
-
-- **Behavior notes**: acquire timeout chosen to exceed the 30-second testcontainer spin-up requirement. `application_name = "vestige"` makes `pg_stat_activity` readable from `psql` during debugging.
-
-### D4. sqlx migrations directory
-
-- **File**: `crates/vestige-core/migrations/postgres/0001_init.up.sql`, `0001_init.down.sql`, `0002_hnsw.up.sql`, `0002_hnsw.down.sql`.
-- **Depends on**: none (pure SQL).
-
-`0001_init.up.sql`:
-
-```sql
--- Extensions
-CREATE EXTENSION IF NOT EXISTS pgcrypto;
-CREATE EXTENSION IF NOT EXISTS vector;
-
--- Embedding model registry
--- Mirrors the SQLite table created in Phase 1.
-CREATE TABLE embedding_model (
- id SMALLINT PRIMARY KEY DEFAULT 1 CHECK (id = 1),
- name TEXT NOT NULL,
- dimension INTEGER NOT NULL CHECK (dimension > 0),
- hash TEXT NOT NULL,
- created_at TIMESTAMPTZ NOT NULL DEFAULT now()
-);
-
--- Domains table (populated by Phase 4 DomainClassifier; Phase 2 only creates
--- the empty table so list/get/upsert/delete work against both backends).
-CREATE TABLE domains (
- id TEXT PRIMARY KEY,
- label TEXT NOT NULL,
- centroid vector,
- top_terms TEXT[] NOT NULL DEFAULT '{}',
- memory_count INTEGER NOT NULL DEFAULT 0,
- created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
- metadata JSONB NOT NULL DEFAULT '{}'::jsonb
-);
-
--- Core memories table
-CREATE TABLE memories (
- id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
- domains TEXT[] NOT NULL DEFAULT '{}',
- domain_scores JSONB NOT NULL DEFAULT '{}'::jsonb,
- content TEXT NOT NULL,
- node_type TEXT NOT NULL DEFAULT 'general',
- tags TEXT[] NOT NULL DEFAULT '{}',
- embedding vector,
- metadata JSONB NOT NULL DEFAULT '{}'::jsonb,
- created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
- updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
- search_vec TSVECTOR GENERATED ALWAYS AS (
- setweight(to_tsvector('english', coalesce(content, '')), 'A') ||
- setweight(to_tsvector('english', coalesce(node_type, '')), 'B') ||
- setweight(to_tsvector('english', coalesce(array_to_string(tags, ' '), '')), 'C')
- ) STORED
-);
-
--- FSRS scheduling state (1:1 with memories)
-CREATE TABLE scheduling (
- memory_id UUID PRIMARY KEY REFERENCES memories(id) ON DELETE CASCADE,
- stability DOUBLE PRECISION NOT NULL DEFAULT 0.0,
- difficulty DOUBLE PRECISION NOT NULL DEFAULT 0.0,
- retrievability DOUBLE PRECISION NOT NULL DEFAULT 1.0,
- last_review TIMESTAMPTZ,
- next_review TIMESTAMPTZ,
- reps INTEGER NOT NULL DEFAULT 0,
- lapses INTEGER NOT NULL DEFAULT 0
-);
-
--- Graph edges (spreading activation)
-CREATE TABLE edges (
- source_id UUID NOT NULL REFERENCES memories(id) ON DELETE CASCADE,
- target_id UUID NOT NULL REFERENCES memories(id) ON DELETE CASCADE,
- edge_type TEXT NOT NULL DEFAULT 'related',
- weight DOUBLE PRECISION NOT NULL DEFAULT 1.0,
- created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
- PRIMARY KEY (source_id, target_id, edge_type)
-);
-
--- FSRS review event log (Phase 1 creates this; Phase 2 mirrors it for Postgres).
--- Append-only. Used for future federation (Phase 5).
-CREATE TABLE review_events (
- id BIGSERIAL PRIMARY KEY,
- memory_id UUID NOT NULL REFERENCES memories(id) ON DELETE CASCADE,
- timestamp TIMESTAMPTZ NOT NULL DEFAULT now(),
- rating SMALLINT NOT NULL,
- prior_state JSONB NOT NULL,
- new_state JSONB NOT NULL
-);
-
--- Indexes on memories (vector index is declared separately in 0002_hnsw.up.sql)
-CREATE INDEX idx_memories_fts ON memories USING GIN (search_vec);
-CREATE INDEX idx_memories_domains ON memories USING GIN (domains);
-CREATE INDEX idx_memories_tags ON memories USING GIN (tags);
-CREATE INDEX idx_memories_node_type ON memories (node_type);
-CREATE INDEX idx_memories_created ON memories (created_at);
-CREATE INDEX idx_memories_updated ON memories (updated_at);
-
--- Indexes on scheduling
-CREATE INDEX idx_scheduling_next_review ON scheduling (next_review);
-CREATE INDEX idx_scheduling_last_review ON scheduling (last_review);
-
--- Indexes on edges
-CREATE INDEX idx_edges_target ON edges (target_id);
-CREATE INDEX idx_edges_source ON edges (source_id);
-CREATE INDEX idx_edges_type ON edges (edge_type);
-
--- Indexes on review_events
-CREATE INDEX idx_review_events_memory ON review_events (memory_id);
-CREATE INDEX idx_review_events_ts ON review_events (timestamp);
-
--- Update trigger on memories.updated_at
-CREATE OR REPLACE FUNCTION memories_set_updated_at() RETURNS TRIGGER AS $$
-BEGIN
- NEW.updated_at := now();
- RETURN NEW;
-END;
-$$ LANGUAGE plpgsql;
-
-CREATE TRIGGER trg_memories_updated_at
-BEFORE UPDATE ON memories
-FOR EACH ROW EXECUTE FUNCTION memories_set_updated_at();
-```
-
-`0001_init.down.sql`:
-
-```sql
-DROP TRIGGER IF EXISTS trg_memories_updated_at ON memories;
-DROP FUNCTION IF EXISTS memories_set_updated_at();
-
-DROP INDEX IF EXISTS idx_review_events_ts;
-DROP INDEX IF EXISTS idx_review_events_memory;
-DROP INDEX IF EXISTS idx_edges_type;
-DROP INDEX IF EXISTS idx_edges_source;
-DROP INDEX IF EXISTS idx_edges_target;
-DROP INDEX IF EXISTS idx_scheduling_last_review;
-DROP INDEX IF EXISTS idx_scheduling_next_review;
-DROP INDEX IF EXISTS idx_memories_updated;
-DROP INDEX IF EXISTS idx_memories_created;
-DROP INDEX IF EXISTS idx_memories_node_type;
-DROP INDEX IF EXISTS idx_memories_tags;
-DROP INDEX IF EXISTS idx_memories_domains;
-DROP INDEX IF EXISTS idx_memories_fts;
-
-DROP TABLE IF EXISTS review_events;
-DROP TABLE IF EXISTS edges;
-DROP TABLE IF EXISTS scheduling;
-DROP TABLE IF EXISTS memories;
-DROP TABLE IF EXISTS domains;
-DROP TABLE IF EXISTS embedding_model;
-```
-
-`0002_hnsw.up.sql` (separated so reembed can drop-and-recreate without touching the rest of the schema):
-
-```sql
--- HNSW index on memories.embedding.
--- pgvector requires the column to have a typmod (fixed dimension) for HNSW.
--- The dimension is stamped by the application at startup via ALTER TABLE
--- using the embedder's dimension() method (see PgMemoryStore::connect).
--- We express the index with the generic vector_cosine_ops operator class.
-CREATE INDEX idx_memories_embedding_hnsw
- ON memories USING hnsw (embedding vector_cosine_ops)
- WITH (m = 16, ef_construction = 64);
-```
-
-`0002_hnsw.down.sql`:
-
-```sql
-DROP INDEX IF EXISTS idx_memories_embedding_hnsw;
-```
-
-- **Behavior notes**:
- - pgvector HNSW requires a typmod. `PgMemoryStore::connect` runs `ALTER TABLE memories ALTER COLUMN embedding TYPE vector($N)` with `$N = embedder.dimension()` exactly once, guarded by a check against `embedding_model` (first startup ever) or validated against it on subsequent starts. If `embedder.dimension()` differs from the stored one and `embedding_model` is non-empty, return `StoreError::EmbeddingDimensionMismatch` -- the user must run `vestige migrate --reembed`.
- - `ALTER COLUMN ... TYPE vector($N)` on a populated column fails unless the data fits; that is the desired safety net.
- - The `tsvector` GENERATED column uses `array_to_string(tags, ' ')` rather than `array_to_tsvector` from the PRD sketch, because `array_to_tsvector` is not a core function in Postgres 16 and would require an extension. The behavior is equivalent for weight C.
- - `gen_random_uuid()` comes from `pgcrypto`. In Postgres 13+ it is also available from core; we keep the extension for older compatibility paths.
- - MVCC: all table writes are transactional; no explicit locks. `INSERT ... ON CONFLICT DO UPDATE` is used in `upsert_domain`, `update_scheduling`, and edge idempotency.
-
-### D5. Hybrid search via RRF
-
-- **File**: `crates/vestige-core/src/storage/postgres/search.rs`
-- **Depends on**: D2, D4.
-- **Signatures**:
-
-```rust
-#![cfg(feature = "postgres-backend")]
-
-use pgvector::Vector;
-use sqlx::PgPool;
-use uuid::Uuid;
-
-use crate::storage::error::StoreResult;
-use crate::storage::types::{SearchQuery, SearchResult};
-
-const RRF_K: i32 = 60; // constant from Cormack et al. 2009
-const OVERFETCH_MULT: i64 = 3; // matches Phase 1 SQLite overfetch
-
-pub(crate) async fn rrf_search(
- pool: &PgPool,
- query: &SearchQuery,
-) -> StoreResult>;
-```
-
-SQL for the full hybrid RRF query. Placeholders:
-- `$1` = text query (string, may be empty)
-- `$2` = embedding (vector)
-- `$3` = overfetch limit per branch (int)
-- `$4` = final limit (int)
-- `$5` = domain filter (text[] or NULL)
-- `$6` = node_type filter (text[] or NULL)
-- `$7` = tag filter (text[] or NULL)
-
-```sql
-WITH params AS (
- SELECT
- $1::text AS q_text,
- $2::vector AS q_vec,
- $3::int AS overfetch,
- $4::int AS final_limit,
- $5::text[] AS dom_filter,
- $6::text[] AS nt_filter,
- $7::text[] AS tag_filter
-),
-fts AS (
- SELECT m.id,
- ts_rank_cd(m.search_vec, websearch_to_tsquery('english', p.q_text)) AS score,
- ROW_NUMBER() OVER (
- ORDER BY ts_rank_cd(m.search_vec, websearch_to_tsquery('english', p.q_text)) DESC
- ) AS rank
- FROM memories m, params p
- WHERE p.q_text <> ''
- AND m.search_vec @@ websearch_to_tsquery('english', p.q_text)
- AND (p.dom_filter IS NULL OR m.domains && p.dom_filter)
- AND (p.nt_filter IS NULL OR m.node_type = ANY(p.nt_filter))
- AND (p.tag_filter IS NULL OR m.tags && p.tag_filter)
- LIMIT (SELECT overfetch FROM params)
-),
-vec AS (
- SELECT m.id,
- 1 - (m.embedding <=> p.q_vec) AS score,
- ROW_NUMBER() OVER (
- ORDER BY m.embedding <=> p.q_vec
- ) AS rank
- FROM memories m, params p
- WHERE m.embedding IS NOT NULL
- AND p.q_vec IS NOT NULL
- AND (p.dom_filter IS NULL OR m.domains && p.dom_filter)
- AND (p.nt_filter IS NULL OR m.node_type = ANY(p.nt_filter))
- AND (p.tag_filter IS NULL OR m.tags && p.tag_filter)
- LIMIT (SELECT overfetch FROM params)
-),
-fused AS (
- SELECT COALESCE(f.id, v.id) AS id,
- COALESCE(1.0 / (60 + f.rank), 0.0)
- + COALESCE(1.0 / (60 + v.rank), 0.0) AS rrf_score,
- f.score AS fts_score,
- v.score AS vector_score
- FROM fts f FULL OUTER JOIN vec v ON f.id = v.id
-)
-SELECT m.id AS "id!: Uuid",
- m.domains AS "domains!: Vec",
- m.domain_scores AS "domain_scores!: serde_json::Value",
- m.content AS "content!",
- m.node_type AS "node_type!",
- m.tags AS "tags!: Vec",
- m.embedding AS "embedding?: Vector",
- m.metadata AS "metadata!: serde_json::Value",
- m.created_at AS "created_at!: chrono::DateTime",
- m.updated_at AS "updated_at!: chrono::DateTime",
- fused.rrf_score AS "rrf_score!: f64",
- fused.fts_score AS "fts_score?: f64",
- fused.vector_score AS "vector_score?: f64"
-FROM fused
-JOIN memories m ON m.id = fused.id
-ORDER BY fused.rrf_score DESC
-LIMIT (SELECT final_limit FROM params);
-```
-
-- **Behavior notes**:
- - `OVERFETCH_MULT * query.limit` is passed as `$3`. Final `$4` is `query.limit`.
- - Empty text query is allowed; the `fts` CTE returns zero rows (`p.q_text <> ''`) and the result degrades to pure vector search, which matches `vector_search` behavior.
- - Null embedding is allowed; the `vec` CTE returns zero rows and the result degrades to pure FTS, which matches `fts_search` behavior.
- - `fts_search` and `vector_search` are separate public methods on the trait. Each uses a simpler single-CTE query derived from the above by removing the other branch. Implementing them as thin wrappers over `rrf_search` with nullified inputs is acceptable but adds one extra plan per call; the explicit implementations win on latency.
- - `min_retrievability` in `SearchQuery` is applied as a final filter by joining on `scheduling` in the outer `SELECT`. Adding that join unconditionally regresses simple searches; add it only when `query.min_retrievability.is_some()`.
-
-### D6. `embedding_model` registry impl
-
-- **File**: `crates/vestige-core/src/storage/postgres/registry.rs`
-- **Depends on**: D1, D4 (table exists), Phase 1 `EmbeddingModelRegistry` trait.
-- **Signatures**:
-
-```rust
-#![cfg(feature = "postgres-backend")]
-
-use sqlx::PgPool;
-
-use crate::embedder::Embedder;
-use crate::storage::error::{StoreError, StoreResult};
-
-pub(crate) async fn ensure_registry(
- pool: &PgPool,
- embedder: &dyn Embedder,
-) -> StoreResult<()> {
- let row = sqlx::query!(
- r#"SELECT name, dimension, hash FROM embedding_model WHERE id = 1"#
- )
- .fetch_optional(pool)
- .await?;
-
- match row {
- None => {
- sqlx::query!(
- r#"
- INSERT INTO embedding_model (id, name, dimension, hash)
- VALUES (1, $1, $2, $3)
- "#,
- embedder.model_name(),
- embedder.dimension() as i32,
- embedder.model_hash(),
- )
- .execute(pool)
- .await?;
-
- // First-ever run: stamp the vector column typmod.
- let ddl = format!(
- "ALTER TABLE memories ALTER COLUMN embedding TYPE vector({})",
- embedder.dimension()
- );
- sqlx::query(&ddl).execute(pool).await?;
- Ok(())
- }
- Some(r) if r.name == embedder.model_name()
- && r.dimension == embedder.dimension() as i32
- && r.hash == embedder.model_hash() => Ok(()),
- Some(r) => Err(StoreError::EmbeddingMismatch {
- expected: format!("{} ({}d, {})", r.name, r.dimension, r.hash),
- got: format!(
- "{} ({}d, {})",
- embedder.model_name(),
- embedder.dimension(),
- embedder.model_hash()
- ),
- }),
- }
-}
-
-pub(crate) async fn update_registry(
- pool: &PgPool,
- embedder: &dyn Embedder,
-) -> StoreResult<()> {
- // Used only by `vestige migrate --reembed` after a full re-encode.
- sqlx::query!(
- r#"
- UPDATE embedding_model
- SET name = $1, dimension = $2, hash = $3, created_at = now()
- WHERE id = 1
- "#,
- embedder.model_name(),
- embedder.dimension() as i32,
- embedder.model_hash(),
- )
- .execute(pool)
- .await?;
- Ok(())
-}
-```
-
-- **Behavior notes**:
- - `StoreError::EmbeddingMismatch { expected, got }` already exists in Phase 1; Phase 2 just constructs it.
- - The `ALTER TABLE ... TYPE vector(N)` DDL is only issued on first init. On subsequent inits the existing typmod already matches.
- - Re-embed flow also uses this module, but the DDL path is different -- see D11.
-
-### D7. `VestigeConfig`: `vestige.toml` backend selection
-
-- **File**: `crates/vestige-core/src/config.rs` (Phase 1 may already own this file; Phase 2 extends, not replaces)
-- **Depends on**: D1.
-- **Signatures**:
-
-```rust
-use std::path::{Path, PathBuf};
-
-use serde::Deserialize;
-
-#[derive(Debug, Clone, Deserialize)]
-pub struct VestigeConfig {
- #[serde(default)]
- pub embeddings: EmbeddingsConfig,
- #[serde(default)]
- pub storage: StorageConfig,
- #[serde(default)]
- pub server: ServerConfig,
- #[serde(default)]
- pub auth: AuthConfig,
-}
-
-#[derive(Debug, Clone, Deserialize)]
-pub struct EmbeddingsConfig {
- pub provider: String, // "fastembed"
- pub model: String, // "BAAI/bge-base-en-v1.5"
-}
-
-#[derive(Debug, Clone, Deserialize)]
-#[serde(tag = "backend", rename_all = "lowercase")]
-pub enum StorageConfig {
- Sqlite(SqliteConfig),
- #[cfg(feature = "postgres-backend")]
- Postgres(PostgresConfig),
-}
-
-#[derive(Debug, Clone, Deserialize)]
-pub struct SqliteConfig {
- pub path: PathBuf,
-}
-
-#[cfg(feature = "postgres-backend")]
-#[derive(Debug, Clone, Deserialize)]
-pub struct PostgresConfig {
- pub url: String,
- #[serde(default)]
- pub max_connections: Option