Commit graph

28 commits

Author SHA1 Message Date
Error42-redacted
8d7b375b31 fix: rewrite explore_connections to use SQLite storage instead of empty in-memory structures
explore_connections (chain, associations, bridges) returned empty results because
it read from CognitiveEngine in-memory data structures that were never populated.
memory_graph works because it queries SQLite directly via storage.get_memory_subgraph().

This rewrites explore.rs to follow the same pattern as graph.rs:
- Remove CognitiveEngine dependency, use Storage directly
- associations: queries depth-1 subgraph for neighbors
- chain: pulls neighborhoods around both endpoints, builds adjacency list,
  runs BFS with priority queue (ported from chains.rs algorithm)
- bridges: reuses chain logic, extracts intermediate nodes by frequency

Also updates the call site in server.rs to pass storage instead of cognitive.

Adds integration tests that insert actual connections and verify non-empty results.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 02:00:06 -08:00
Sam Valladares
ec2af6e71b fix: comprehensive audit fixes for dashboard and backend
Backend:
- Emit WebSocket events from REST delete/promote/demote handlers
- Emit DreamStarted/ConsolidationStarted from MCP tool dispatch
- Add path validation in backup_to() for defense-in-depth

Dashboard:
- Fix ConnectionDiscovered field names (source_id/target_id)
- Fix $effect → onMount in settings (prevents infinite loop)
- Fix $derived → $derived.by in RetentionCurve
- Fix field name mismatches in settings (nodesProcessed, etc.)
- Fix nested <button> → <span role="button"> in memories
- Fix unhandled Promise rejection in stats consolidation
- Add missing EVENT_TYPE_COLORS entries
- Add Three.js resource disposal and event listener cleanup
- Eliminate duplicate root page, redirect to /graph
- Update nav links and keyboard shortcuts to /graph

All 734+ tests passing, 22MB binary, zero build warnings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 15:50:47 -06:00
Sam Valladares
c2d28f3433 feat: Vestige v2.0.0 "Cognitive Leap" — 3D dashboard, HyDE search, WebSocket events
The biggest release in Vestige history. Complete visual and cognitive overhaul.

Dashboard:
- SvelteKit 2 + Three.js 3D neural visualization at localhost:3927/dashboard
- 7 interactive pages: Graph, Memories, Timeline, Feed, Explore, Intentions, Stats
- WebSocket event bus with 16 event types, real-time 3D animations
- Bloom post-processing, GPU instanced rendering, force-directed layout
- Dream visualization mode, FSRS retention curves, command palette (Cmd+K)
- Keyboard shortcuts, responsive mobile layout, PWA installable
- Single binary deployment via include_dir! (22MB)

Engine:
- HyDE query expansion (intent classification + 3-5 semantic variants + centroid)
- fastembed 5.11 with optional Nomic v2 MoE + Qwen3 reranker + Metal GPU
- Emotional memory module (#29)
- Criterion benchmark suite

Backend:
- Axum WebSocket at /ws with heartbeat + event broadcast
- 7 new REST endpoints for cognitive operations
- Event emission from MCP tools via shared broadcast channel
- CORS for SvelteKit dev mode

Distribution:
- GitHub issue templates (bug report, feature request)
- CHANGELOG with comprehensive v2.0 release notes
- README updated with dashboard docs, architecture diagram, comparison table

734 tests passing, zero warnings, 22MB release binary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 03:07:25 -06:00
Sam Valladares
5b90a73055 feat: Vestige v1.9.1 AUTONOMIC — self-regulating memory with graph visualization
Retention Target System: auto-GC low-retention memories during consolidation
(VESTIGE_RETENTION_TARGET env var, default 0.8). Auto-Promote: memories
accessed 3+ times in 24h get frequency-dependent potentiation. Waking SWR
Tagging: promoted memories get preferential 70/30 dream replay. Improved
Consolidation Scheduler: triggers on 6h staleness or 2h active use.

New tools: memory_health (retention dashboard with distribution buckets,
trend tracking, recommendations) and memory_graph (subgraph export with
Fruchterman-Reingold force-directed layout, up to 200 nodes).

Dream connections now persist to database via save_connection(), enabling
memory_graph traversal. Schema Migration V8 adds waking_tag, utility_score,
times_retrieved/useful columns and retention_snapshots table. 21 MCP tools.

v1.9.1 fixes: ConnectionRecord export, UTF-8 safe truncation, link_type
normalization, utility_score clamping, only-new-connections persistence,
70/30 split capacity fill, nonexistent center_id error handling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 02:02:06 -06:00
Sam Valladares
c29023dd80 feat: Vestige v1.7.0 — 18 tools, automation triggers, SQLite perf
Tool consolidation: 23 → 18 tools
- ingest merged into smart_ingest (single + batch mode)
- session_checkpoint merged into smart_ingest batch (items param)
- promote_memory/demote_memory merged into memory(action=promote/demote)
- health_check/stats merged into system_status

Automation triggers in system_status:
- lastDreamTimestamp, savesSinceLastDream, lastBackupTimestamp,
  lastConsolidationTimestamp — enables Claude to conditionally
  trigger dream/backup/gc/find_duplicates at session start
- Migration v6: dream_history table (dreams were in-memory only)
- DreamHistoryRecord struct + save/query methods
- Dream persistence in dream.rs (non-fatal on failure)

SQLite performance:
- PRAGMA mmap_size = 256MB (2-5x read speedup)
- PRAGMA journal_size_limit = 64MB (prevents WAL bloat)
- PRAGMA optimize = 0x10002 (fresh query planner stats on connect)
- FTS5 segment merge during consolidation (20-40% keyword boost)
- PRAGMA optimize during consolidation cycle

1,152 tests passing, 0 failures, release build clean.
2026-02-20 21:59:52 -06:00
Sam Valladares
ce520bb246 chore: license AGPL-3.0, zero clippy warnings, CHANGELOG through v1.6.0
License:
- Replace MIT/Apache-2.0 with AGPL-3.0-only across all crates and npm packages
- Replace LICENSE file with official GNU AGPL-3.0 text
- Remove LICENSE-MIT and LICENSE-APACHE

Code quality:
- Fix all 44 clippy warnings (zero remaining)
- Collapsible if statements, redundant closures, manual Option::map
- Remove duplicate #[allow(dead_code)] attributes in deprecated tool modules
- Add Default impl for CognitiveEngine
- Replace manual sort_by with sort_by_key

Documentation:
- Update CHANGELOG with v1.2.0, v1.3.0, v1.5.0, v1.6.0 entries
- Update README with v1.6.0 highlights and accurate stats (52K lines, 1100+ tests)
- Add fastembed-rs/ to .gitignore
- Add fastembed-rs to workspace exclude

1115 tests passing, zero warnings, RUSTFLAGS="-Dwarnings" clean.
2026-02-19 03:00:39 -06:00
Sam Valladares
495a88331f feat: Vestige v1.6.0 — 6x storage reduction, neural reranking, instant startup
Four internal optimizations for dramatically better performance:

1. F16 vector quantization (ScalarKind::F16 in USearch) — 2x storage savings
2. Matryoshka 256-dim truncation (768→256) — 3x embedding storage savings
3. Convex Combination fusion (0.3 keyword / 0.7 semantic) replacing RRF
4. Cross-encoder reranker (Jina Reranker v1 Turbo via fastembed TextRerank)

Combined: 6x vector storage reduction, ~20% better retrieval quality.
Cross-encoder loads in background — server starts instantly.
Old 768-dim embeddings auto-migrated on load.

614 tests pass, zero warnings.
2026-02-19 01:09:39 -06:00
Sam Valladares
927f41c3e4 feat: Vestige v1.5.0 — Cognitive Engine, memory dreaming, graph exploration, predictive retrieval
28-module CognitiveEngine with full neuroscience pipeline on every tool call.
FSRS-6 now fully automatic: periodic consolidation (6h timer + inline every
100 tool calls), real retrievability formula, episodic-to-semantic auto-merge,
cross-memory reinforcement, Park et al. triple retrieval scoring, ACT-R
base-level activation, personalized w20 optimization.

New tools (19 → 23):
- dream: memory consolidation via replay, discovers hidden connections
- explore_connections: graph traversal (chain, associations, bridges)
- predict: proactive retrieval based on context and activity patterns
- restore: memory restore from JSON backups

All existing tools upgraded with cognitive pre/post processing pipelines.
33 files changed, ~4,100 lines added.
2026-02-18 23:34:15 -06:00
Sam Valladares
04a3062328 feat: Vestige v1.3.0 — importance scoring, session checkpoints, duplicate detection
3 new MCP tools (16 → 19 total):
- importance_score: 4-channel neuroscience importance scoring (novelty/arousal/reward/attention)
- session_checkpoint: batch smart_ingest up to 20 items with PE Gating
- find_duplicates: cosine similarity clustering with union-find for dedup

CLI: vestige ingest command for memory ingestion via command line
Core: made get_node_embedding public, added get_all_embeddings for dedup scanning

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 05:02:09 -06:00
Sam Valladares
34f5e8d52a feat: Vestige v1.2.0 — dashboard, temporal tools, maintenance tools, detail levels
Add web dashboard (axum) on port 3927 with memory browser, search, and
system stats. New MCP tools: memory_timeline, memory_changelog,
health_check, consolidate, stats, backup, export, gc. Search now supports
detail_level (brief/summary/full) to control token usage. Add backup_to()
and get_recent_state_transitions() to storage layer. Bump to v1.2.0.
2026-02-12 04:33:05 -06:00
Sam Valladares
a92fb2b6ed release: v1.1.3 — security hardening, edition 2024, dependency updates
Security:
- Fix RUSTSEC-2026-0007 (bytes integer overflow)
- Restrict SQLite database file permissions to 0600 on Unix
- Add 100KB size limit to intention descriptions (DoS prevention)
- Redact JSON-RPC payloads from debug logs (data leakage prevention)
- Update SECURITY.md with encryption docs and supported versions

Modernization:
- Upgrade Rust edition 2021 → 2024, MSRV 1.75 → 1.85
- Upgrade actions/checkout@v4 → v5, codecov/codecov-action@v3 → v5
- Update all dependencies to latest compatible versions
- Fix edition 2024 match ergonomics in compression.rs

Clippy fixes:
- Rename from_str → parse_name to avoid shadowing FromStr trait
- Replace .max().min() with .clamp()
- Replace sort_by with sort_by_key

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 03:19:07 -06:00
Sam Valladares
6a5c3771fb feat: P0/P1 fixes — backup, export, gc, performance, auto-consolidation, encryption
P0 fixes:
- Add `vestige backup <path>` — full DB copy with WAL checkpoint flush
- Add `vestige export --format json|jsonl [--tags] [--since] <path>` —
  paginated memory export with tag/date filtering
- Add `vestige gc --min-retention 0.1 [--max-age-days] [--dry-run] [--yes]`
  — bulk cleanup of stale memories with safety prompts
- Fix apply_decay() scaling: batched pagination (500 rows/batch) with
  explicit transactions instead of loading all nodes into memory
- Fix hidden MCP resources: memory://insights and memory://consolidation-log
  now listed in resources/list (were implemented but undiscoverable)

P1 fixes:
- Add auto-consolidation on server startup: FSRS-6 decay runs in background
  after 2s delay, only if last consolidation was >6 hours ago
- Add encryption at rest via SQLCipher feature flag: use --features encryption
  with VESTIGE_ENCRYPTION_KEY env var (bundled-sqlite and encryption are
  mutually exclusive)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 03:02:32 -06:00
Sam Valladares
a680fa7d2f fix: dedup on ingest, Intel Mac CI, npm versions, remove dead TS package
- Route ingest tool through smart_ingest (Prediction Error Gating) to
  prevent duplicate memories when content is similar to existing entries
- Fix Intel Mac release build: use macos-13 runner for x86_64-apple-darwin
  (macos-latest is now ARM64, causing silent cross-compile failures)
- Sync npm package version to 1.1.2 (was 1.0.0 in package.json, 1.1.0
  in postinstall.js BINARY_VERSION)
- Add vestige-restore to npm makeExecutable list
- Remove abandoned packages/core/ TypeScript package (pre-Rust implementation
  referencing FSRS-5, chromadb, ollama — 32K lines of dead code)
- Sync workspace Cargo.toml version to 1.1.2

Closes #5

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 02:57:03 -06:00
Sam Valladares
ee39e62a35 chore: bump crate versions to 1.1.2 2026-01-27 02:57:01 -06:00
Sam Valladares
e06dd3d69a chore: cleanup dead code warnings and apply clippy fixes for v1.1.1
- Add #![allow(dead_code)] to deprecated tool modules (kept for
  backwards compatibility but not exposed in MCP tool list)
- Mark unused functions with #[allow(dead_code)] annotations
- Fix unused variable warnings (prefix with _)
- Apply clippy auto-fixes for redundant closures and derives
- Fix test to account for protocol version negotiation
- Reorganize tools/mod.rs to clarify active vs deprecated tools

Security review: LOW RISK - no critical vulnerabilities found
Dead code review: deprecated tools properly annotated

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 01:23:27 -06:00
Sam Valladares
bfa91474a6 fix: resolve UTF-8 string slicing bugs and feature flag issues
- Fix silent errors in stdio.rs: clients now receive fallback error
  responses instead of hanging when JSON serialization fails
- Fix UTF-8 panics in keyword.rs: use char-aware slicing instead of
  byte offsets for query sanitization and term highlighting
- Fix UTF-8 panics in prospective_memory.rs: replace hard-coded byte
  offsets with char-aware slicing for natural language parsing
- Fix UTF-8 panics in git.rs: convert byte positions to char positions
  before slicing commit messages
- Fix feature flag bug in vestige-mcp: add proper [features] section
  to forward embeddings and vector-search features from vestige-core,
  enabling the #[cfg(feature = "embeddings")] initialization code

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 01:14:59 -06:00
Sam Valladares
f10367ecd0 fix: add explicit embedding initialization with error logging
When embeddings fail to initialize in the MCP server context (e.g., due
to working directory issues), the error was silently swallowed and
smart_ingest would fall back to regular ingest without explanation.

Changes:
- Add init_embeddings() method to Storage for explicit initialization
- Initialize embeddings at MCP server startup with error logging
- Add check_ready() method to EmbeddingService for error access
- Log warning when is_ready() returns false

Now users will see clear error messages like:
"Failed to initialize embedding service: ..."
"Hint: Check FASTEMBED_CACHE_PATH or ensure ~/.fastembed_cache exists"

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 01:04:06 -06:00
Sam Valladares
6ccbc7d2c0 fix: implement MCP protocol version negotiation
Claude Desktop requests protocol version 2025-06-18 but Vestige was
responding with 2025-11-25, causing Claude Desktop to disconnect.

Now the server negotiates: if the client requests an older version,
use it. This maintains backward compatibility with older clients.

Fixes: Claude Desktop "Server transport closed unexpectedly" error

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 00:59:53 -06:00
Sam Valladares
ab1a3a885a chore: bump vestige-mcp version to 1.1.1
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 00:56:42 -06:00
Sam Valladares
b779b54c5c fix: reduce visible tools from 30 to 8
Users were seeing 30 tools in /mcp which was overwhelming. Now only 8
essential tools are listed:

1. search - unified hybrid search
2. memory - get/delete/state operations
3. codebase - patterns and decisions
4. intention - prospective memory
5. ingest - add memories
6. smart_ingest - intelligent ingestion
7. promote_memory - thumbs up
8. demote_memory - thumbs down

Deprecated tools (recall, semantic_search, etc.) still work internally
for backward compatibility but are no longer listed.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 00:42:19 -06:00
Sam Valladares
3fb6a77bc7 fix(cli): use Cargo.toml version instead of hardcoded
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 02:06:22 -06:00
Sam Valladares
1b9004f88e chore(v1.1): move stats tools to CLI, add feedback protocol
- Remove get_stats, health_check, run_consolidation from MCP server
- These tools now CLI-only: vestige stats, vestige health, vestige consolidate
- Add feedback protocol to server instructions (promote/demote guidance)
- Update README with CLI Admin Commands section and Feedback Tools table
- Update release note: "8 Cognitive Primitives + 3 CLI Admin Commands"

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 02:05:06 -06:00
Sam Valladares
086cc2fa2c chore: bump version to 1.1.0
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 01:44:30 -06:00
Sam Valladares
8bb6500985 feat(v1.1): consolidate 29 tools → 8 unified tools + CLI
Tool Consolidation:
- search: merges recall, semantic_search, hybrid_search
- memory: merges get_knowledge, delete_knowledge, get_memory_state
- codebase: merges remember_pattern, remember_decision, get_codebase_context
- intention: merges all 5 intention tools into action-based API

New CLI Binary:
- vestige stats [--tagging] [--states]
- vestige health
- vestige consolidate
- vestige restore <file>

Documentation:
- Verify all neuroscience claims against codebase
- Fix Memory States table: "Retention" → "Accessibility" with formula
- Clarify Spreading Activation: embedding similarity vs full network module
- Update Synaptic Tagging: clarify 9h/2h implementation vs biology
- Add comprehensive FAQ with 30+ questions
- Add storage modes: global, per-project, multi-Claude household
- Add CLAUDE.md setup instructions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 01:31:58 -06:00
Sam Valladares
7440f21402 Add vestige-restore binary for backup restoration 2026-01-25 13:52:01 -06:00
Sam Valladares
bbd1c15b4a Add Prediction Error Gating and smart_ingest tool (26 tools total)
Implements neuroscience-inspired memory gating based on prediction error:
- New smart_ingest MCP tool that auto-decides CREATE/UPDATE/SUPERSEDE
- PredictionErrorGate evaluates semantic similarity vs existing memories
- Automatically supersedes demoted memories with similar new content
- Reinforces near-identical memories instead of creating duplicates
- Adds promote_memory/demote_memory/request_feedback tools

Thresholds:
- >0.92 similarity = Reinforce existing
- >0.75 similarity = Update/Merge
- <0.75 similarity = Create new
- Demoted + similar = Auto-supersede

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 13:30:03 -06:00
Sam Valladares
5337efdfa7 Switch embedding model from BGE to nomic-embed-text-v1.5
- Replace BGE-base-en-v1.5 with nomic-embed-text-v1.5
- 8192 token context window (vs 512 for BGE)
- Matryoshka representation learning support
- Fully open source with training data released
- Same 768 dimensions, no schema changes required

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 03:11:15 -06:00
Sam Valladares
f9c60eb5a7 Initial commit: Vestige v1.0.0 - Cognitive memory MCP server
FSRS-6 spaced repetition, spreading activation, synaptic tagging,
hippocampal indexing, and 130 years of memory research.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 01:31:03 -06:00