feat(storage): phase 1 -- extract MemoryStore and Embedder traits (ADR 0001)

Introduce two trait boundaries that the rest of the stack now sits above,
landing Phase 1 of ADR 0001 (pluggable storage and network access).
Rebased onto v2.1.22 Sanhedrin from the original April work.

MemoryStore / LocalMemoryStore (crates/vestige-core/src/storage/memory_store.rs):
  One trait, ~25 methods, covering CRUD, hybrid / FTS / vector search,
  FSRS scheduling, graph edges, and the forthcoming domain surface.
  trait_variant::make generates a Send-bound MemoryStore alias over the
  base LocalMemoryStore so Arc<dyn MemoryStore> works under tokio/axum.
  Storage errors map through a dedicated MemoryStoreError.

Embedder / LocalEmbedder (crates/vestige-core/src/embedder/):
  Pluggable text-to-vector encoder. FastembedEmbedder wraps the existing
  EmbeddingService; storage never calls fastembed directly anymore.
  Embedder::signature() produces the ModelSignature consumed by the
  store's embedding_model registry.

SqliteMemoryStore (crates/vestige-core/src/storage/sqlite.rs):
  Storage renamed to SqliteMemoryStore; the old name lives on as a
  pub type alias so Arc<Storage> consumers in vestige-mcp stay intact.
  All existing inherent methods are untouched; the trait impl is
  purely additive and dispatches into them. The db_path field added
  by v2.1.1 portable-sync is preserved.

Migration V14 (crates/vestige-core/src/storage/migrations.rs):
  Renumbered from V12 (the original April number) to V14 to slot in
  cleanly after upstream's V12 (v2.1.1 sync_tombstones) and V13
  (v2.1.2 purge tombstones).
  - embedding_model registry table (CHECK id = 1, code enforces the
    single-row invariant).
  - knowledge_nodes.domains / domain_scores TEXT columns (JSON arrays
    default '[]' / '{}'), domains catalogue table, supporting indexes.
  Phase 4 populates these columns; Phase 1 just exposes the schema.

Consolidation and other cognitive pathways now accept a
&dyn LocalMemoryStore (sync) or Arc<dyn MemoryStore> (async) rather
than a concrete Storage.

Tests:
  - trait-method unit tests colocated in sqlite.rs and migrations.rs
  - embedder/fastembed.rs tests for name/dimension/hash stability
  - new integration crate tests/phase_1 (added to workspace members):
    trait_round_trip (8), embedding_model_registry (7),
    domain_column_migration (5), cognitive_module_isolation (4),
    send_bound_variant (2), embedder_trait (2).

Acceptance gate post-rebase:
  - cargo build --workspace --all-targets: ok
  - cargo clippy --workspace --all-targets -- -D warnings: clean
  - cargo test -p vestige-core --lib: 428 pass
  - cargo test -p vestige-phase-1-tests: 28 pass
  - cargo test -p vestige-mcp --lib: 380 pass (Storage alias preserves
    every existing call site)

Co-existence with v2.1.1 portable-sync: this trait extraction is
additive. Portable-sync's tombstone migrations (V12, V13) remain
on the concrete SqliteMemoryStore; Phase 2 (Postgres) will decide
which of those surfaces graduate into the trait.
This commit is contained in:
Jan De Landtsheer 2026-04-21 21:43:52 +02:00 committed by Sam Valladares
parent 01fa882760
commit 5715f585fd
17 changed files with 3282 additions and 44 deletions

115
Cargo.lock generated
View file

@ -143,6 +143,12 @@ dependencies = [
"syn",
]
[[package]]
name = "arrayref"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "76a2e8124351fda1ef8aaaa3bbd7ebbcb486bbcd4225aca0aa0d84bb2db8fecb"
[[package]]
name = "arrayvec"
version = "0.7.6"
@ -158,6 +164,17 @@ dependencies = [
"stable_deref_trait",
]
[[package]]
name = "async-trait"
version = "0.1.89"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9035ad2d096bed7955a320ee7e2230574d28fd3c3a0f186cbea1ff3c7eed5dbb"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "atomic-waker"
version = "1.1.2"
@ -311,6 +328,20 @@ dependencies = [
"core2",
]
[[package]]
name = "blake3"
version = "1.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4d2d5991425dfd0785aed03aedcf0b321d61975c9b5b3689c774a2610ae0b51e"
dependencies = [
"arrayref",
"arrayvec",
"cc",
"cfg-if",
"constant_time_eq",
"cpufeatures 0.3.0",
]
[[package]]
name = "block"
version = "0.1.6"
@ -642,6 +673,12 @@ dependencies = [
"windows-sys 0.59.0",
]
[[package]]
name = "constant_time_eq"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b"
[[package]]
name = "core-foundation"
version = "0.9.4"
@ -697,6 +734,15 @@ dependencies = [
"libc",
]
[[package]]
name = "cpufeatures"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b2a41393f66f16b0823bb79094d54ac5fbd34ab292ddafb9a0456ac9f87d201"
dependencies = [
"libc",
]
[[package]]
name = "crc32fast"
version = "1.5.0"
@ -2282,12 +2328,10 @@ dependencies = [
[[package]]
name = "js-sys"
version = "0.3.95"
version = "0.3.91"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2964e92d1d9dc3364cae4d718d93f227e3abb088e747d92e0395bfdedf1c12ca"
checksum = "b49715b7073f385ba4bc528e5747d02e66cb39c6146efb66b781f131f0fb399c"
dependencies = [
"cfg-if",
"futures-util",
"once_cell",
"wasm-bindgen",
]
@ -3181,9 +3225,9 @@ checksum = "c33a9471896f1c69cecef8d20cbe2f7accd12527ce60845ff44c153bb2a21b49"
[[package]]
name = "portable-atomic-util"
version = "0.2.6"
version = "0.2.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "091397be61a01d4be58e7841595bd4bfedb15f1cd54977d79b8271e94ed799a3"
checksum = "c2a106d1259c23fac8e543272398ae0e3c0b8d33c88ed73d0cc71b0f1d902618"
dependencies = [
"portable-atomic",
]
@ -3822,7 +3866,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3bf829a2d51ab4a5ddf1352d8470c140cadc8301b2ae1789db023f01cedd6ba"
dependencies = [
"cfg-if",
"cpufeatures",
"cpufeatures 0.2.17",
"digest",
]
@ -3833,7 +3877,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283"
dependencies = [
"cfg-if",
"cpufeatures",
"cpufeatures 0.2.17",
"digest",
]
@ -4356,6 +4400,17 @@ dependencies = [
"tracing-serde",
]
[[package]]
name = "trait-variant"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70977707304198400eb4835a78f6a9f928bf41bba420deb8fdb175cd965d77a7"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "try-lock"
version = "0.2.5"
@ -4631,6 +4686,8 @@ checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
name = "vestige-core"
version = "2.1.26"
dependencies = [
"async-trait",
"blake3",
"candle-core",
"chrono",
"criterion",
@ -4646,6 +4703,7 @@ dependencies = [
"thiserror 2.0.18",
"tokio",
"tracing",
"trait-variant",
"usearch",
"uuid",
]
@ -4692,6 +4750,19 @@ dependencies = [
"vestige-core",
]
[[package]]
name = "vestige-phase-1-tests"
version = "0.0.1"
dependencies = [
"chrono",
"rusqlite",
"serde_json",
"tempfile",
"tokio",
"uuid",
"vestige-core",
]
[[package]]
name = "walkdir"
version = "2.5.0"
@ -4737,9 +4808,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen"
version = "0.2.118"
version = "0.2.114"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0bf938a0bacb0469e83c1e148908bd7d5a6010354cf4fb73279b7447422e3a89"
checksum = "6532f9a5c1ece3798cb1c2cfdba640b9b3ba884f5db45973a6f442510a87d38e"
dependencies = [
"cfg-if",
"once_cell",
@ -4750,19 +4821,23 @@ dependencies = [
[[package]]
name = "wasm-bindgen-futures"
version = "0.4.68"
version = "0.4.64"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f371d383f2fb139252e0bfac3b81b265689bf45b6874af544ffa4c975ac1ebf8"
checksum = "e9c5522b3a28661442748e09d40924dfb9ca614b21c00d3fd135720e48b67db8"
dependencies = [
"cfg-if",
"futures-util",
"js-sys",
"once_cell",
"wasm-bindgen",
"web-sys",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.118"
version = "0.2.114"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eeff24f84126c0ec2db7a449f0c2ec963c6a49efe0698c4242929da037ca28ed"
checksum = "18a2d50fcf105fb33bb15f00e7a77b772945a2ee45dcf454961fd843e74c18e6"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
@ -4770,9 +4845,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.118"
version = "0.2.114"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d08065faf983b2b80a79fd87d8254c409281cf7de75fc4b773019824196c904"
checksum = "03ce4caeaac547cdf713d280eda22a730824dd11e6b8c3ca9e42247b25c631e3"
dependencies = [
"bumpalo",
"proc-macro2",
@ -4783,9 +4858,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.118"
version = "0.2.114"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5fd04d9e306f1907bd13c6361b5c6bfc7b3b3c095ed3f8a9246390f8dbdee129"
checksum = "75a326b8c223ee17883a4251907455a2431acc2791c98c26279376490c378c16"
dependencies = [
"unicode-ident",
]
@ -4839,9 +4914,9 @@ dependencies = [
[[package]]
name = "web-sys"
version = "0.3.95"
version = "0.3.91"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4f2dfbb17949fa2088e5d39408c48368947b86f7834484e87b73de55bc14d97d"
checksum = "854ba17bb104abfb26ba36da9729addc7ce7f06f5c0f90f3c391f8461cca21f9"
dependencies = [
"js-sys",
"wasm-bindgen",