vestige/tests/e2e/src/harness/db_manager.rs

387 lines
11 KiB
Rust
Raw Normal View History

//! Test Database Manager
//!
//! Provides isolated database instances for testing:
//! - Temporary databases that are automatically cleaned up
//! - Pre-seeded databases with test data
//! - Database snapshots and restoration
//! - Concurrent test isolation
use std::path::PathBuf;
use tempfile::TempDir;
feat(v2.0.5): Intentional Amnesia — active forgetting via top-down inhibitory control First AI memory system to model forgetting as a neuroscience-grounded PROCESS rather than passive decay. Adds the `suppress` MCP tool (#24), Rac1 cascade worker, migration V10, and dashboard forgetting indicators. Based on: - Anderson, Hanslmayr & Quaegebeur (2025), Nat Rev Neurosci — right lateral PFC as the domain-general inhibitory controller; SIF compounds with each stopping attempt. - Cervantes-Sandoval et al. (2020), Front Cell Neurosci PMC7477079 — Rac1 GTPase as the active synaptic destabilization mechanism. What's new: * `suppress` MCP tool — each call compounds `suppression_count` and subtracts a `0.15 × count` penalty (saturating at 80%) from retrieval scores during hybrid search. Distinct from delete (removes) and demote (one-shot). * Rac1 cascade worker — background sweep piggybacks the 6h consolidation loop, walks `memory_connections` edges from recently-suppressed seeds, applies attenuated FSRS decay to co-activated neighbors. You don't just forget Jake — you fade the café, the roommate, the birthday. * 24h labile window — reversible via `suppress({id, reverse: true})` within 24 hours. Matches Nader reconsolidation semantics. * Migration V10 — additive-only (`suppression_count`, `suppressed_at` + partial indices). All v2.0.x DBs upgrade seamlessly on first launch. * Dashboard: `ForgettingIndicator.svelte` pulses when suppressions are active. 3D graph nodes dim to 20% opacity when suppressed. New WebSocket events: `MemorySuppressed`, `MemoryUnsuppressed`, `Rac1CascadeSwept`. Heartbeat carries `suppressed_count`. * Search pipeline: SIF penalty inserted into the accessibility stage so it stacks on top of passive FSRS decay. * Tool count bumped 23 → 24. Cognitive modules 29 → 30. Memories persist — they are INHIBITED, not erased. `memory.get(id)` returns full content through any number of suppressions. The 24h labile window is a grace period for regret. Also fixes issue #31 (dashboard graph view buggy) as a companion UI bug discovered during the v2.0.5 audit cycle: * Root cause: node glow `SpriteMaterial` had no `map`, so `THREE.Sprite` rendered as a solid-coloured 1×1 plane. Additive blending + `UnrealBloomPass(0.8, 0.4, 0.85)` amplified the square edges into hard-edged glowing cubes. * Fix: shared 128×128 radial-gradient `CanvasTexture` singleton used as the sprite map. Retuned bloom to `(0.55, 0.6, 0.2)`. Halved fog density (0.008 → 0.0035). Edges bumped from dark navy `0x4a4a7a` to brand violet `0x8b5cf6` with higher opacity. Added explicit `scene.background` and a 2000-point starfield for depth. * 21 regression tests added in `ui-fixes.test.ts` locking every invariant in (shared texture singleton, depthWrite:false, scale ×6, bloom magic numbers via source regex, starfield presence). Tests: 1,284 Rust (+47) + 171 Vitest (+21) = 1,455 total, 0 failed Clippy: clean across all targets, zero warnings Release binary: 22.6MB, `cargo build --release -p vestige-mcp` green Versions: workspace aligned at 2.0.5 across all 6 crates/packages Closes #31
2026-04-14 17:30:30 -05:00
use vestige_core::{KnowledgeNode, Rating, Storage};
/// Helper to create IngestInput (works around non_exhaustive)
#[allow(clippy::too_many_arguments)]
fn make_ingest_input(
content: String,
node_type: String,
tags: Vec<String>,
sentiment_score: f64,
sentiment_magnitude: f64,
source: Option<String>,
valid_from: Option<chrono::DateTime<chrono::Utc>>,
valid_until: Option<chrono::DateTime<chrono::Utc>>,
) -> vestige_core::IngestInput {
vestige_core::IngestInput {
content,
node_type,
tags,
sentiment_score,
sentiment_magnitude,
source,
valid_from,
valid_until,
feat(connectors): external-source connector layer + GitHub Issues (#57) Make Vestige a durable, local, semantically-searchable retrieval layer over an external system of record (GitHub Issues first), citing back to the canonical record. Unlike a live ticket-system MCP proxy, Vestige keeps a durable embedded index: searchable offline, joinable with the rest of memory, temporally versioned, and re-syncable idempotently with no duplication. Phases 1-2 of #57 plus a GitHub reference connector and source-aware search: - Source envelope on KnowledgeNode/IngestInput (source_system, source_id, source_url, source_updated_at, content_hash, synced_at, source_project, source_type, source_author). Migration V17: nullable columns (additive), partial UNIQUE index on (source_system, source_id), connector_cursors table. - Idempotent sync primitives in vestige-core: upsert_by_source (content-hash change detection), connector cursor checkpoints, reconcile_source_tombstones (invalidate-don't-delete via bitemporal valid_until). - Connector contract + run_sync driver + GitHub Issues connector behind the optional `connectors` feature (on by default in vestige-mcp, off in the core library default so non-connector consumers link no HTTP client). - source_sync MCP tool ({"repo": "owner/name"}); token from GITHUB_TOKEN env only. Search results gain a sourceRecord citation for connector memories. Adversarial review fixes: GitHub `since` Z-form (the `+00:00` offset corrupted the cursor server-side), un-tombstone clears superseded_by too, cursor never advances past a failing record, Link next-url host-pinned (token-leak guard), records_seen counts new records only. Verified: cargo check/test/clippy -D warnings green across the workspace (default and connectors features); 483 core tests pass. Version bump to 2.1.27 and tag deferred to release. Refs #57 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 01:21:59 -05:00
source_envelope: None,
}
}
/// Manager for test databases
///
/// Creates isolated database instances for each test to prevent interference.
/// Automatically cleans up temporary databases when dropped.
///
/// # Example
///
/// ```rust,ignore
/// let mut db = TestDatabaseManager::new_temp();
///
/// // Use the storage
/// db.storage.ingest(IngestInput { ... });
///
/// // Database is automatically deleted when `db` goes out of scope
/// ```
pub struct TestDatabaseManager {
/// The storage instance
pub storage: Storage,
/// Temporary directory (kept alive to prevent premature deletion)
_temp_dir: Option<TempDir>,
/// Path to the database file
db_path: PathBuf,
/// Snapshot data for restore operations
snapshot: Option<Vec<KnowledgeNode>>,
}
impl TestDatabaseManager {
/// Create a new test database in a temporary directory
///
/// The database is automatically deleted when the manager is dropped.
pub fn new_temp() -> Self {
let temp_dir = TempDir::new().expect("Failed to create temp directory");
let db_path = temp_dir.path().join("test_vestige.db");
let storage = Storage::new(Some(db_path.clone())).expect("Failed to create test storage");
Self {
storage,
_temp_dir: Some(temp_dir),
db_path,
snapshot: None,
}
}
/// Create a test database at a specific path
///
/// The database is NOT automatically deleted.
pub fn new_at_path(path: PathBuf) -> Self {
let storage = Storage::new(Some(path.clone())).expect("Failed to create test storage");
Self {
storage,
_temp_dir: None,
db_path: path,
snapshot: None,
}
}
/// Get the database path
pub fn path(&self) -> &PathBuf {
&self.db_path
}
/// Check if the database is empty
pub fn is_empty(&self) -> bool {
self.storage
.get_stats()
.map(|s| s.total_nodes == 0)
.unwrap_or(true)
}
/// Get the number of nodes in the database
pub fn node_count(&self) -> i64 {
feat(v2.0.5): Intentional Amnesia — active forgetting via top-down inhibitory control First AI memory system to model forgetting as a neuroscience-grounded PROCESS rather than passive decay. Adds the `suppress` MCP tool (#24), Rac1 cascade worker, migration V10, and dashboard forgetting indicators. Based on: - Anderson, Hanslmayr & Quaegebeur (2025), Nat Rev Neurosci — right lateral PFC as the domain-general inhibitory controller; SIF compounds with each stopping attempt. - Cervantes-Sandoval et al. (2020), Front Cell Neurosci PMC7477079 — Rac1 GTPase as the active synaptic destabilization mechanism. What's new: * `suppress` MCP tool — each call compounds `suppression_count` and subtracts a `0.15 × count` penalty (saturating at 80%) from retrieval scores during hybrid search. Distinct from delete (removes) and demote (one-shot). * Rac1 cascade worker — background sweep piggybacks the 6h consolidation loop, walks `memory_connections` edges from recently-suppressed seeds, applies attenuated FSRS decay to co-activated neighbors. You don't just forget Jake — you fade the café, the roommate, the birthday. * 24h labile window — reversible via `suppress({id, reverse: true})` within 24 hours. Matches Nader reconsolidation semantics. * Migration V10 — additive-only (`suppression_count`, `suppressed_at` + partial indices). All v2.0.x DBs upgrade seamlessly on first launch. * Dashboard: `ForgettingIndicator.svelte` pulses when suppressions are active. 3D graph nodes dim to 20% opacity when suppressed. New WebSocket events: `MemorySuppressed`, `MemoryUnsuppressed`, `Rac1CascadeSwept`. Heartbeat carries `suppressed_count`. * Search pipeline: SIF penalty inserted into the accessibility stage so it stacks on top of passive FSRS decay. * Tool count bumped 23 → 24. Cognitive modules 29 → 30. Memories persist — they are INHIBITED, not erased. `memory.get(id)` returns full content through any number of suppressions. The 24h labile window is a grace period for regret. Also fixes issue #31 (dashboard graph view buggy) as a companion UI bug discovered during the v2.0.5 audit cycle: * Root cause: node glow `SpriteMaterial` had no `map`, so `THREE.Sprite` rendered as a solid-coloured 1×1 plane. Additive blending + `UnrealBloomPass(0.8, 0.4, 0.85)` amplified the square edges into hard-edged glowing cubes. * Fix: shared 128×128 radial-gradient `CanvasTexture` singleton used as the sprite map. Retuned bloom to `(0.55, 0.6, 0.2)`. Halved fog density (0.008 → 0.0035). Edges bumped from dark navy `0x4a4a7a` to brand violet `0x8b5cf6` with higher opacity. Added explicit `scene.background` and a 2000-point starfield for depth. * 21 regression tests added in `ui-fixes.test.ts` locking every invariant in (shared texture singleton, depthWrite:false, scale ×6, bloom magic numbers via source regex, starfield presence). Tests: 1,284 Rust (+47) + 171 Vitest (+21) = 1,455 total, 0 failed Clippy: clean across all targets, zero warnings Release binary: 22.6MB, `cargo build --release -p vestige-mcp` green Versions: workspace aligned at 2.0.5 across all 6 crates/packages Closes #31
2026-04-14 17:30:30 -05:00
self.storage.get_stats().map(|s| s.total_nodes).unwrap_or(0)
}
// ========================================================================
// SEEDING METHODS
// ========================================================================
/// Seed the database with a specified number of test nodes
pub fn seed_nodes(&mut self, count: usize) -> Vec<String> {
let mut ids = Vec::with_capacity(count);
for i in 0..count {
let input = make_ingest_input(
format!("Test memory content {}", i),
"fact".to_string(),
vec![format!("test-{}", i % 5)],
0.0,
0.0,
None,
None,
None,
);
if let Ok(node) = self.storage.ingest(input) {
ids.push(node.id);
}
}
ids
}
/// Seed with diverse node types
pub fn seed_diverse(&mut self, count_per_type: usize) -> Vec<String> {
let types = ["fact", "concept", "procedure", "event", "code"];
let mut ids = Vec::with_capacity(count_per_type * types.len());
for node_type in types {
for i in 0..count_per_type {
let input = make_ingest_input(
format!("Test {} content {}", node_type, i),
node_type.to_string(),
vec![node_type.to_string()],
0.0,
0.0,
None,
None,
None,
);
if let Ok(node) = self.storage.ingest(input) {
ids.push(node.id);
}
}
}
ids
}
/// Seed with nodes having various retention states
pub fn seed_with_retention_states(&mut self) -> Vec<String> {
let mut ids = Vec::new();
// New node (never reviewed)
let input = make_ingest_input(
"New memory - never reviewed".to_string(),
"fact".to_string(),
vec!["new".to_string()],
0.0,
0.0,
None,
None,
None,
);
if let Ok(node) = self.storage.ingest(input) {
ids.push(node.id);
}
// Well-learned node (multiple good reviews)
let input = make_ingest_input(
"Well-learned memory - reviewed multiple times".to_string(),
"fact".to_string(),
vec!["learned".to_string()],
0.0,
0.0,
None,
None,
None,
);
if let Ok(node) = self.storage.ingest(input) {
let _ = self.storage.mark_reviewed(&node.id, Rating::Good);
let _ = self.storage.mark_reviewed(&node.id, Rating::Good);
let _ = self.storage.mark_reviewed(&node.id, Rating::Easy);
ids.push(node.id);
}
// Struggling node (multiple lapses)
let input = make_ingest_input(
"Struggling memory - has lapses".to_string(),
"fact".to_string(),
vec!["struggling".to_string()],
0.0,
0.0,
None,
None,
None,
);
if let Ok(node) = self.storage.ingest(input) {
let _ = self.storage.mark_reviewed(&node.id, Rating::Again);
let _ = self.storage.mark_reviewed(&node.id, Rating::Hard);
let _ = self.storage.mark_reviewed(&node.id, Rating::Again);
ids.push(node.id);
}
ids
}
/// Seed with emotional memories (different sentiment magnitudes)
pub fn seed_emotional(&mut self, count: usize) -> Vec<String> {
let mut ids = Vec::with_capacity(count);
for i in 0..count {
let magnitude = (i as f64) / (count as f64);
let input = make_ingest_input(
format!("Emotional memory with magnitude {:.2}", magnitude),
"event".to_string(),
vec!["emotional".to_string()],
if i % 2 == 0 { 0.8 } else { -0.8 },
magnitude,
None,
None,
None,
);
if let Ok(node) = self.storage.ingest(input) {
ids.push(node.id);
}
}
ids
}
// ========================================================================
// SNAPSHOT/RESTORE
// ========================================================================
/// Take a snapshot of current database state
pub fn take_snapshot(&mut self) {
feat(v2.0.5): Intentional Amnesia — active forgetting via top-down inhibitory control First AI memory system to model forgetting as a neuroscience-grounded PROCESS rather than passive decay. Adds the `suppress` MCP tool (#24), Rac1 cascade worker, migration V10, and dashboard forgetting indicators. Based on: - Anderson, Hanslmayr & Quaegebeur (2025), Nat Rev Neurosci — right lateral PFC as the domain-general inhibitory controller; SIF compounds with each stopping attempt. - Cervantes-Sandoval et al. (2020), Front Cell Neurosci PMC7477079 — Rac1 GTPase as the active synaptic destabilization mechanism. What's new: * `suppress` MCP tool — each call compounds `suppression_count` and subtracts a `0.15 × count` penalty (saturating at 80%) from retrieval scores during hybrid search. Distinct from delete (removes) and demote (one-shot). * Rac1 cascade worker — background sweep piggybacks the 6h consolidation loop, walks `memory_connections` edges from recently-suppressed seeds, applies attenuated FSRS decay to co-activated neighbors. You don't just forget Jake — you fade the café, the roommate, the birthday. * 24h labile window — reversible via `suppress({id, reverse: true})` within 24 hours. Matches Nader reconsolidation semantics. * Migration V10 — additive-only (`suppression_count`, `suppressed_at` + partial indices). All v2.0.x DBs upgrade seamlessly on first launch. * Dashboard: `ForgettingIndicator.svelte` pulses when suppressions are active. 3D graph nodes dim to 20% opacity when suppressed. New WebSocket events: `MemorySuppressed`, `MemoryUnsuppressed`, `Rac1CascadeSwept`. Heartbeat carries `suppressed_count`. * Search pipeline: SIF penalty inserted into the accessibility stage so it stacks on top of passive FSRS decay. * Tool count bumped 23 → 24. Cognitive modules 29 → 30. Memories persist — they are INHIBITED, not erased. `memory.get(id)` returns full content through any number of suppressions. The 24h labile window is a grace period for regret. Also fixes issue #31 (dashboard graph view buggy) as a companion UI bug discovered during the v2.0.5 audit cycle: * Root cause: node glow `SpriteMaterial` had no `map`, so `THREE.Sprite` rendered as a solid-coloured 1×1 plane. Additive blending + `UnrealBloomPass(0.8, 0.4, 0.85)` amplified the square edges into hard-edged glowing cubes. * Fix: shared 128×128 radial-gradient `CanvasTexture` singleton used as the sprite map. Retuned bloom to `(0.55, 0.6, 0.2)`. Halved fog density (0.008 → 0.0035). Edges bumped from dark navy `0x4a4a7a` to brand violet `0x8b5cf6` with higher opacity. Added explicit `scene.background` and a 2000-point starfield for depth. * 21 regression tests added in `ui-fixes.test.ts` locking every invariant in (shared texture singleton, depthWrite:false, scale ×6, bloom magic numbers via source regex, starfield presence). Tests: 1,284 Rust (+47) + 171 Vitest (+21) = 1,455 total, 0 failed Clippy: clean across all targets, zero warnings Release binary: 22.6MB, `cargo build --release -p vestige-mcp` green Versions: workspace aligned at 2.0.5 across all 6 crates/packages Closes #31
2026-04-14 17:30:30 -05:00
let nodes = self.storage.get_all_nodes(10000, 0).unwrap_or_default();
self.snapshot = Some(nodes);
}
/// Restore from the last snapshot
///
/// Note: This clears the database and re-inserts all nodes from snapshot.
/// IDs will NOT be preserved (new UUIDs are generated).
pub fn restore_snapshot(&mut self) -> bool {
if let Some(nodes) = self.snapshot.take() {
// Clear current data by recreating storage
// Delete the database file first
let _ = std::fs::remove_file(&self.db_path);
self.storage = Storage::new(Some(self.db_path.clone()))
.expect("Failed to recreate storage for restore");
// Re-insert nodes
for node in nodes {
let input = make_ingest_input(
node.content,
node.node_type,
node.tags,
node.sentiment_score,
node.sentiment_magnitude,
node.source,
node.valid_from,
node.valid_until,
);
let _ = self.storage.ingest(input);
}
true
} else {
false
}
}
/// Check if a snapshot exists
pub fn has_snapshot(&self) -> bool {
self.snapshot.is_some()
}
// ========================================================================
// CLEANUP
// ========================================================================
/// Clear all data from the database
pub fn clear(&mut self) {
// Get all node IDs and delete them
if let Ok(nodes) = self.storage.get_all_nodes(10000, 0) {
for node in nodes {
let _ = self.storage.delete_node(&node.id);
}
}
}
/// Recreate the database (useful for testing migrations)
pub fn recreate(&mut self) {
// Delete the database file
let _ = std::fs::remove_file(&self.db_path);
// Recreate storage
feat(v2.0.5): Intentional Amnesia — active forgetting via top-down inhibitory control First AI memory system to model forgetting as a neuroscience-grounded PROCESS rather than passive decay. Adds the `suppress` MCP tool (#24), Rac1 cascade worker, migration V10, and dashboard forgetting indicators. Based on: - Anderson, Hanslmayr & Quaegebeur (2025), Nat Rev Neurosci — right lateral PFC as the domain-general inhibitory controller; SIF compounds with each stopping attempt. - Cervantes-Sandoval et al. (2020), Front Cell Neurosci PMC7477079 — Rac1 GTPase as the active synaptic destabilization mechanism. What's new: * `suppress` MCP tool — each call compounds `suppression_count` and subtracts a `0.15 × count` penalty (saturating at 80%) from retrieval scores during hybrid search. Distinct from delete (removes) and demote (one-shot). * Rac1 cascade worker — background sweep piggybacks the 6h consolidation loop, walks `memory_connections` edges from recently-suppressed seeds, applies attenuated FSRS decay to co-activated neighbors. You don't just forget Jake — you fade the café, the roommate, the birthday. * 24h labile window — reversible via `suppress({id, reverse: true})` within 24 hours. Matches Nader reconsolidation semantics. * Migration V10 — additive-only (`suppression_count`, `suppressed_at` + partial indices). All v2.0.x DBs upgrade seamlessly on first launch. * Dashboard: `ForgettingIndicator.svelte` pulses when suppressions are active. 3D graph nodes dim to 20% opacity when suppressed. New WebSocket events: `MemorySuppressed`, `MemoryUnsuppressed`, `Rac1CascadeSwept`. Heartbeat carries `suppressed_count`. * Search pipeline: SIF penalty inserted into the accessibility stage so it stacks on top of passive FSRS decay. * Tool count bumped 23 → 24. Cognitive modules 29 → 30. Memories persist — they are INHIBITED, not erased. `memory.get(id)` returns full content through any number of suppressions. The 24h labile window is a grace period for regret. Also fixes issue #31 (dashboard graph view buggy) as a companion UI bug discovered during the v2.0.5 audit cycle: * Root cause: node glow `SpriteMaterial` had no `map`, so `THREE.Sprite` rendered as a solid-coloured 1×1 plane. Additive blending + `UnrealBloomPass(0.8, 0.4, 0.85)` amplified the square edges into hard-edged glowing cubes. * Fix: shared 128×128 radial-gradient `CanvasTexture` singleton used as the sprite map. Retuned bloom to `(0.55, 0.6, 0.2)`. Halved fog density (0.008 → 0.0035). Edges bumped from dark navy `0x4a4a7a` to brand violet `0x8b5cf6` with higher opacity. Added explicit `scene.background` and a 2000-point starfield for depth. * 21 regression tests added in `ui-fixes.test.ts` locking every invariant in (shared texture singleton, depthWrite:false, scale ×6, bloom magic numbers via source regex, starfield presence). Tests: 1,284 Rust (+47) + 171 Vitest (+21) = 1,455 total, 0 failed Clippy: clean across all targets, zero warnings Release binary: 22.6MB, `cargo build --release -p vestige-mcp` green Versions: workspace aligned at 2.0.5 across all 6 crates/packages Closes #31
2026-04-14 17:30:30 -05:00
self.storage =
Storage::new(Some(self.db_path.clone())).expect("Failed to recreate storage");
}
}
impl Drop for TestDatabaseManager {
fn drop(&mut self) {
// Storage is dropped automatically
// TempDir (if Some) will clean up the temp directory
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_temp_database_creation() {
let db = TestDatabaseManager::new_temp();
assert!(db.is_empty());
assert!(db.path().exists());
}
#[test]
fn test_seed_nodes() {
let mut db = TestDatabaseManager::new_temp();
let ids = db.seed_nodes(10);
assert_eq!(ids.len(), 10);
assert_eq!(db.node_count(), 10);
}
#[test]
fn test_seed_diverse() {
let mut db = TestDatabaseManager::new_temp();
let ids = db.seed_diverse(3);
// 5 types * 3 each = 15
assert_eq!(ids.len(), 15);
assert_eq!(db.node_count(), 15);
}
#[test]
fn test_clear_database() {
let mut db = TestDatabaseManager::new_temp();
db.seed_nodes(5);
assert_eq!(db.node_count(), 5);
db.clear();
assert!(db.is_empty());
}
#[test]
fn test_snapshot_restore() {
let mut db = TestDatabaseManager::new_temp();
db.seed_nodes(5);
db.take_snapshot();
assert!(db.has_snapshot());
db.clear();
assert!(db.is_empty());
db.restore_snapshot();
assert_eq!(db.node_count(), 5);
}
}