Commit graph

21 commits

Author SHA1 Message Date
Bot
f3e25f7503 fix: push type filters into SQL WHERE clause + expose in MCP search
Type filtering (include_types/exclude_types) was applied post-fetch after
the database LIMIT, which could return zero results when all top-N
results were of the filtered type. This pushes type filters into the SQL
WHERE clause in keyword_search_with_scores() so they apply before the
limit. Semantic results still get post-fetch filtering as a safety net
since the vector index cannot filter by type.

Also adds hybrid_search_filtered() as the new primary method, with the
original hybrid_search() delegating to it with no filters for backward
compatibility. The MCP search tool now exposes include_types and
exclude_types parameters.

Includes 5 new test cases covering include, exclude, precedence,
empty results, and backward compatibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 14:42:16 -05:00
Matthias Queitsch
b5892fc723
build: allow building on older glibc versions 2026-04-11 08:21:53 +02:00
Sam Valladares
04781a95e2 feat: v2.0.4 "Deep Reference" — cognitive reasoning engine + 10 bug fixes
New features:
- deep_reference tool (#22): 8-stage cognitive reasoning pipeline with FSRS-6
  trust scoring, intent classification (FactCheck/Timeline/RootCause/Comparison/
  Synthesis), spreading activation expansion, temporal supersession, trust-weighted
  contradiction analysis, relation assessment, dream insight integration, and
  algorithmic reasoning chain generation — all without calling an LLM
- cross_reference (#23): backward-compatible alias for deep_reference
- retrieval_mode parameter on search (precise/balanced/exhaustive)
- get_batch action on memory tool (up to 20 IDs per call)
- Token budget raised from 10K to 100K on search + session_context
- Dates (createdAt/updatedAt) on all search results and session_context lines

Bug fixes (GitHub Issue #25 — all 10 resolved):
- state_transitions empty: wired record_memory_access into strengthen_batch
- chain/bridges no storage fallback: added with edge deduplication
- knowledge_edges dead schema: documented as deprecated
- insights not persisted from dream: wired save_insight after generation
- find_duplicates threshold dropped: serde alias fix
- search min_retention ignored: serde aliases for snake_case params
- intention time triggers null: removed dead trigger_at embedding
- changelog missing dreams: added get_dream_history + event integration
- phantom Related IDs: clarified message text
- fsrs_cards empty: documented as harmless dead schema

Security hardening:
- HTTP transport CORS: permissive() → localhost-only
- Auth token panic guard: &token[..8] → safe min(8) slice
- UTF-8 boundary fix: floor_char_boundary on content truncation
- All unwrap() removed from HTTP transport (unwrap_or_else fallback)
- Dream memory_count capped at 500 (prevents O(N²) hang)
- Dormant state threshold aligned (0.3 → 0.4)

Stats: 23 tools, 758 tests, 0 failures, 0 warnings, 0 unwraps in production

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 16:15:26 -05:00
Sam Valladares
816b577f69 feat: add MCP Streamable HTTP transport with Bearer auth
Adds a second transport layer alongside stdio — Streamable HTTP on port
3928. Enables Claude.ai, remote clients, and web integrations to connect
to Vestige over HTTP with per-session McpServer instances.

- POST /mcp (JSON-RPC) + DELETE /mcp (session cleanup)
- Bearer token auth with constant-time comparison (subtle crate)
- Auto-generated UUID v4 token persisted with 0o600 permissions
- Per-session McpServer instances with 30-min idle reaper
- 100 max sessions, 50 concurrency limit, 256KB body limit
- --http-port flag + VESTIGE_HTTP_PORT / VESTIGE_HTTP_BIND env vars
- Module exports moved from binary to lib.rs for reusability
- vestige CLI gains `serve` subcommand via shared lib

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 10:51:41 -06:00
Sam Valladares
c6090dc2ba fix: v2.0.1 release — fix broken installs, CI, security, and docs
Critical fixes:
- npm postinstall.js: BINARY_VERSION '1.1.3' → '2.0.1' (every install was 404ing)
- npm package name: corrected error messages to 'vestige-mcp-server'
- README: npm install command pointed to wrong package
- MSRV: bumped from 1.85 to 1.91 (uses floor_char_boundary from 1.91)
- CI: removed stale 'develop' branch from test.yml triggers

Security hardening:
- CSP: restricted connect-src from wildcard 'ws: wss:' to localhost-only
- Added X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy headers
- Added frame-ancestors 'none', base-uri 'self', form-action 'self' to CSP
- Capped retention_distribution endpoint from 10k to 1k nodes
- Added debug logging for WebSocket connections without Origin header

Maintenance:
- All clippy warnings fixed (58 total: redundant closures, collapsible ifs, no-op casts)
- All versions harmonized to 2.0.1 across Cargo.toml and package.json
- CLAUDE.md updated to match v2.0.1 (21 tools, 29 modules, 1238 tests)
- docs/CLAUDE-SETUP.md updated deprecated function names
- License corrected to AGPL-3.0-only in root package.json

1,238 tests passing, 0 clippy warnings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 20:20:14 -06:00
Sam Valladares
c2d28f3433 feat: Vestige v2.0.0 "Cognitive Leap" — 3D dashboard, HyDE search, WebSocket events
The biggest release in Vestige history. Complete visual and cognitive overhaul.

Dashboard:
- SvelteKit 2 + Three.js 3D neural visualization at localhost:3927/dashboard
- 7 interactive pages: Graph, Memories, Timeline, Feed, Explore, Intentions, Stats
- WebSocket event bus with 16 event types, real-time 3D animations
- Bloom post-processing, GPU instanced rendering, force-directed layout
- Dream visualization mode, FSRS retention curves, command palette (Cmd+K)
- Keyboard shortcuts, responsive mobile layout, PWA installable
- Single binary deployment via include_dir! (22MB)

Engine:
- HyDE query expansion (intent classification + 3-5 semantic variants + centroid)
- fastembed 5.11 with optional Nomic v2 MoE + Qwen3 reranker + Metal GPU
- Emotional memory module (#29)
- Criterion benchmark suite

Backend:
- Axum WebSocket at /ws with heartbeat + event broadcast
- 7 new REST endpoints for cognitive operations
- Event emission from MCP tools via shared broadcast channel
- CORS for SvelteKit dev mode

Distribution:
- GitHub issue templates (bug report, feature request)
- CHANGELOG with comprehensive v2.0 release notes
- README updated with dashboard docs, architecture diagram, comparison table

734 tests passing, zero warnings, 22MB release binary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 03:07:25 -06:00
Sam Valladares
5b90a73055 feat: Vestige v1.9.1 AUTONOMIC — self-regulating memory with graph visualization
Retention Target System: auto-GC low-retention memories during consolidation
(VESTIGE_RETENTION_TARGET env var, default 0.8). Auto-Promote: memories
accessed 3+ times in 24h get frequency-dependent potentiation. Waking SWR
Tagging: promoted memories get preferential 70/30 dream replay. Improved
Consolidation Scheduler: triggers on 6h staleness or 2h active use.

New tools: memory_health (retention dashboard with distribution buckets,
trend tracking, recommendations) and memory_graph (subgraph export with
Fruchterman-Reingold force-directed layout, up to 200 nodes).

Dream connections now persist to database via save_connection(), enabling
memory_graph traversal. Schema Migration V8 adds waking_tag, utility_score,
times_retrieved/useful columns and retention_snapshots table. 21 MCP tools.

v1.9.1 fixes: ConnectionRecord export, UTF-8 safe truncation, link_type
normalization, utility_score clamping, only-new-connections persistence,
70/30 split capacity fill, nonexistent center_id error handling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 02:02:06 -06:00
Sam Valladares
c29023dd80 feat: Vestige v1.7.0 — 18 tools, automation triggers, SQLite perf
Tool consolidation: 23 → 18 tools
- ingest merged into smart_ingest (single + batch mode)
- session_checkpoint merged into smart_ingest batch (items param)
- promote_memory/demote_memory merged into memory(action=promote/demote)
- health_check/stats merged into system_status

Automation triggers in system_status:
- lastDreamTimestamp, savesSinceLastDream, lastBackupTimestamp,
  lastConsolidationTimestamp — enables Claude to conditionally
  trigger dream/backup/gc/find_duplicates at session start
- Migration v6: dream_history table (dreams were in-memory only)
- DreamHistoryRecord struct + save/query methods
- Dream persistence in dream.rs (non-fatal on failure)

SQLite performance:
- PRAGMA mmap_size = 256MB (2-5x read speedup)
- PRAGMA journal_size_limit = 64MB (prevents WAL bloat)
- PRAGMA optimize = 0x10002 (fresh query planner stats on connect)
- FTS5 segment merge during consolidation (20-40% keyword boost)
- PRAGMA optimize during consolidation cycle

1,152 tests passing, 0 failures, release build clean.
2026-02-20 21:59:52 -06:00
Sam Valladares
495a88331f feat: Vestige v1.6.0 — 6x storage reduction, neural reranking, instant startup
Four internal optimizations for dramatically better performance:

1. F16 vector quantization (ScalarKind::F16 in USearch) — 2x storage savings
2. Matryoshka 256-dim truncation (768→256) — 3x embedding storage savings
3. Convex Combination fusion (0.3 keyword / 0.7 semantic) replacing RRF
4. Cross-encoder reranker (Jina Reranker v1 Turbo via fastembed TextRerank)

Combined: 6x vector storage reduction, ~20% better retrieval quality.
Cross-encoder loads in background — server starts instantly.
Old 768-dim embeddings auto-migrated on load.

614 tests pass, zero warnings.
2026-02-19 01:09:39 -06:00
Sam Valladares
927f41c3e4 feat: Vestige v1.5.0 — Cognitive Engine, memory dreaming, graph exploration, predictive retrieval
28-module CognitiveEngine with full neuroscience pipeline on every tool call.
FSRS-6 now fully automatic: periodic consolidation (6h timer + inline every
100 tool calls), real retrievability formula, episodic-to-semantic auto-merge,
cross-memory reinforcement, Park et al. triple retrieval scoring, ACT-R
base-level activation, personalized w20 optimization.

New tools (19 → 23):
- dream: memory consolidation via replay, discovers hidden connections
- explore_connections: graph traversal (chain, associations, bridges)
- predict: proactive retrieval based on context and activity patterns
- restore: memory restore from JSON backups

All existing tools upgraded with cognitive pre/post processing pipelines.
33 files changed, ~4,100 lines added.
2026-02-18 23:34:15 -06:00
Sam Valladares
04a3062328 feat: Vestige v1.3.0 — importance scoring, session checkpoints, duplicate detection
3 new MCP tools (16 → 19 total):
- importance_score: 4-channel neuroscience importance scoring (novelty/arousal/reward/attention)
- session_checkpoint: batch smart_ingest up to 20 items with PE Gating
- find_duplicates: cosine similarity clustering with union-find for dedup

CLI: vestige ingest command for memory ingestion via command line
Core: made get_node_embedding public, added get_all_embeddings for dedup scanning

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 05:02:09 -06:00
Sam Valladares
34f5e8d52a feat: Vestige v1.2.0 — dashboard, temporal tools, maintenance tools, detail levels
Add web dashboard (axum) on port 3927 with memory browser, search, and
system stats. New MCP tools: memory_timeline, memory_changelog,
health_check, consolidate, stats, backup, export, gc. Search now supports
detail_level (brief/summary/full) to control token usage. Add backup_to()
and get_recent_state_transitions() to storage layer. Bump to v1.2.0.
2026-02-12 04:33:05 -06:00
Sam Valladares
a92fb2b6ed release: v1.1.3 — security hardening, edition 2024, dependency updates
Security:
- Fix RUSTSEC-2026-0007 (bytes integer overflow)
- Restrict SQLite database file permissions to 0600 on Unix
- Add 100KB size limit to intention descriptions (DoS prevention)
- Redact JSON-RPC payloads from debug logs (data leakage prevention)
- Update SECURITY.md with encryption docs and supported versions

Modernization:
- Upgrade Rust edition 2021 → 2024, MSRV 1.75 → 1.85
- Upgrade actions/checkout@v4 → v5, codecov/codecov-action@v3 → v5
- Update all dependencies to latest compatible versions
- Fix edition 2024 match ergonomics in compression.rs

Clippy fixes:
- Rename from_str → parse_name to avoid shadowing FromStr trait
- Replace .max().min() with .clamp()
- Replace sort_by with sort_by_key

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 03:19:07 -06:00
Sam Valladares
6a5c3771fb feat: P0/P1 fixes — backup, export, gc, performance, auto-consolidation, encryption
P0 fixes:
- Add `vestige backup <path>` — full DB copy with WAL checkpoint flush
- Add `vestige export --format json|jsonl [--tags] [--since] <path>` —
  paginated memory export with tag/date filtering
- Add `vestige gc --min-retention 0.1 [--max-age-days] [--dry-run] [--yes]`
  — bulk cleanup of stale memories with safety prompts
- Fix apply_decay() scaling: batched pagination (500 rows/batch) with
  explicit transactions instead of loading all nodes into memory
- Fix hidden MCP resources: memory://insights and memory://consolidation-log
  now listed in resources/list (were implemented but undiscoverable)

P1 fixes:
- Add auto-consolidation on server startup: FSRS-6 decay runs in background
  after 2s delay, only if last consolidation was >6 hours ago
- Add encryption at rest via SQLCipher feature flag: use --features encryption
  with VESTIGE_ENCRYPTION_KEY env var (bundled-sqlite and encryption are
  mutually exclusive)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 03:02:32 -06:00
Sam Valladares
a680fa7d2f fix: dedup on ingest, Intel Mac CI, npm versions, remove dead TS package
- Route ingest tool through smart_ingest (Prediction Error Gating) to
  prevent duplicate memories when content is similar to existing entries
- Fix Intel Mac release build: use macos-13 runner for x86_64-apple-darwin
  (macos-latest is now ARM64, causing silent cross-compile failures)
- Sync npm package version to 1.1.2 (was 1.0.0 in package.json, 1.1.0
  in postinstall.js BINARY_VERSION)
- Add vestige-restore to npm makeExecutable list
- Remove abandoned packages/core/ TypeScript package (pre-Rust implementation
  referencing FSRS-5, chromadb, ollama — 32K lines of dead code)
- Sync workspace Cargo.toml version to 1.1.2

Closes #5

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 02:57:03 -06:00
Sam Valladares
bfa91474a6 fix: resolve UTF-8 string slicing bugs and feature flag issues
- Fix silent errors in stdio.rs: clients now receive fallback error
  responses instead of hanging when JSON serialization fails
- Fix UTF-8 panics in keyword.rs: use char-aware slicing instead of
  byte offsets for query sanitization and term highlighting
- Fix UTF-8 panics in prospective_memory.rs: replace hard-coded byte
  offsets with char-aware slicing for natural language parsing
- Fix UTF-8 panics in git.rs: convert byte positions to char positions
  before slicing commit messages
- Fix feature flag bug in vestige-mcp: add proper [features] section
  to forward embeddings and vector-search features from vestige-core,
  enabling the #[cfg(feature = "embeddings")] initialization code

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 01:14:59 -06:00
Sam Valladares
78810f2f5c chore: update Cargo.lock for vendored-openssl
Adds openssl-src v300.5.4+3.5.4 to lockfile.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 02:20:23 -06:00
Sam Valladares
086cc2fa2c chore: bump version to 1.1.0
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 01:44:30 -06:00
Sam Valladares
8bb6500985 feat(v1.1): consolidate 29 tools → 8 unified tools + CLI
Tool Consolidation:
- search: merges recall, semantic_search, hybrid_search
- memory: merges get_knowledge, delete_knowledge, get_memory_state
- codebase: merges remember_pattern, remember_decision, get_codebase_context
- intention: merges all 5 intention tools into action-based API

New CLI Binary:
- vestige stats [--tagging] [--states]
- vestige health
- vestige consolidate
- vestige restore <file>

Documentation:
- Verify all neuroscience claims against codebase
- Fix Memory States table: "Retention" → "Accessibility" with formula
- Clarify Spreading Activation: embedding similarity vs full network module
- Update Synaptic Tagging: clarify 9h/2h implementation vs biology
- Add comprehensive FAQ with 30+ questions
- Add storage modes: global, per-project, multi-Claude household
- Add CLAUDE.md setup instructions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 01:31:58 -06:00
Sam Valladares
bbd1c15b4a Add Prediction Error Gating and smart_ingest tool (26 tools total)
Implements neuroscience-inspired memory gating based on prediction error:
- New smart_ingest MCP tool that auto-decides CREATE/UPDATE/SUPERSEDE
- PredictionErrorGate evaluates semantic similarity vs existing memories
- Automatically supersedes demoted memories with similar new content
- Reinforces near-identical memories instead of creating duplicates
- Adds promote_memory/demote_memory/request_feedback tools

Thresholds:
- >0.92 similarity = Reinforce existing
- >0.75 similarity = Update/Merge
- <0.75 similarity = Create new
- Demoted + similar = Auto-supersede

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 13:30:03 -06:00
Sam Valladares
f9c60eb5a7 Initial commit: Vestige v1.0.0 - Cognitive memory MCP server
FSRS-6 spaced repetition, spreading activation, synaptic tagging,
hippocampal indexing, and 130 years of memory research.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 01:31:03 -06:00