apunkt/vestige
1
0
Fork 0
mirror of https://github.com/samvallad33/vestige.git synced 2026-04-25 00:36:22 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

feat: v2.0.4 "Deep Reference" — cognitive reasoning engine + 10 bug fixes

Browse source 
New features:
- deep_reference tool (#22): 8-stage cognitive reasoning pipeline with FSRS-6
  trust scoring, intent classification (FactCheck/Timeline/RootCause/Comparison/
  Synthesis), spreading activation expansion, temporal supersession, trust-weighted
  contradiction analysis, relation assessment, dream insight integration, and
  algorithmic reasoning chain generation — all without calling an LLM
- cross_reference (#23): backward-compatible alias for deep_reference
- retrieval_mode parameter on search (precise/balanced/exhaustive)
- get_batch action on memory tool (up to 20 IDs per call)
- Token budget raised from 10K to 100K on search + session_context
- Dates (createdAt/updatedAt) on all search results and session_context lines

Bug fixes (GitHub Issue #25 — all 10 resolved):
- state_transitions empty: wired record_memory_access into strengthen_batch
- chain/bridges no storage fallback: added with edge deduplication
- knowledge_edges dead schema: documented as deprecated
- insights not persisted from dream: wired save_insight after generation
- find_duplicates threshold dropped: serde alias fix
- search min_retention ignored: serde aliases for snake_case params
- intention time triggers null: removed dead trigger_at embedding
- changelog missing dreams: added get_dream_history + event integration
- phantom Related IDs: clarified message text
- fsrs_cards empty: documented as harmless dead schema

Security hardening:
- HTTP transport CORS: permissive() → localhost-only
- Auth token panic guard: &token[..8] → safe min(8) slice
- UTF-8 boundary fix: floor_char_boundary on content truncation
- All unwrap() removed from HTTP transport (unwrap_or_else fallback)
- Dream memory_count capped at 500 (prevents O(N²) hang)
- Dormant state threshold aligned (0.3 → 0.4)

Stats: 23 tools, 758 tests, 0 failures, 0 warnings, 0 unwraps in production

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sam Valladares 2026-04-09 16:15:01 -05:00
parent 61091e06b9
commit 04781a95e2
28 changed files with 1797 additions and 102 deletions
Download patch file Download diff file Expand all files Collapse all files

32
CHANGELOG.md
View file

@ -5,6 +5,38 @@ All notable changes to Vestige will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [2.0.4] - 2026-04-09 — "Deep Reference"
Context windows hit 1M tokens. Memory matters more than ever. This release removes artificial limits, adds contradiction detection, and hardens security.
### Added
#### cross_reference Tool (NEW — Tool #22)
- **Connect the dots across memories.** Given a query or claim, searches broadly, detects agreements and contradictions between memories, identifies superseded/outdated information, and returns a confidence-scored synthesis.
- Pairwise contradiction detection using negation pairs + correction signals, gated on shared topic words to prevent false positives.
- Timeline analysis (newest-first), confidence scoring (agreements boost, contradictions penalize, recency bonus).
#### retrieval_mode Parameter (search tool)
- `precise` — top results only, no spreading activation or competition. Fast, token-efficient.
- `balanced` — full 7-stage cognitive pipeline (default, no behavior change).
- `exhaustive` — 5x overfetch, deep graph traversal, no competition suppression. Maximum recall.
#### get_batch Action (memory tool)
- `memory({ action: "get_batch", ids: ["id1", "id2", ...] })` — retrieve up to 20 full memory nodes in one call.
### Changed
- **Token budget raised: 10K → 100K** on search and session_context tools.
- **HTTP transport CORS**: `permissive()` → localhost-only origin restriction.
- **Auth token display**: Guarded against panic on short tokens.
- **Dormant state threshold**: Aligned search (0.3 → 0.4) with memory tool for consistent state classification.
- **cross_reference false positive prevention**: Requires 2+ shared words before checking negation signals.
### Stats
- 23 MCP tools, 758 tests passing, 0 failures
- Full codebase audit: 3 parallel agents, all issues resolved
---
## [2.0.0] - 2026-02-22 — "Cognitive Leap" ## [2.0.0] - 2026-02-22 — "Cognitive Leap"
The biggest release in Vestige history. A complete visual and cognitive overhaul. The biggest release in Vestige history. A complete visual and cognitive overhaul.

46
CLAUDE.md
View file

@ -1,6 +1,8 @@
# Vestige v2.0.1 — Cognitive Memory System # Vestige v2.0.4 — Cognitive Memory & Reasoning System
Vestige is your long-term memory. 29 stateful cognitive modules implement real neuroscience: FSRS-6 spaced repetition, synaptic tagging, prediction error gating, hippocampal indexing, spreading activation, reconsolidation, and dual-strength memory theory. **Use it automatically. Use it aggressively.** Vestige is your long-term memory AND reasoning engine. 29 stateful cognitive modules implement real neuroscience: FSRS-6 spaced repetition, synaptic tagging, prediction error gating, hippocampal indexing, spreading activation, reconsolidation, and dual-strength memory theory. **Use it automatically. Use it aggressively.**
**NEW: `deep_reference` — call this for ALL factual questions.** It doesn't just retrieve — it REASONS across memories with FSRS-6 trust scoring, intent classification, contradiction analysis, and generates a pre-built reasoning chain. Read the `reasoning` field FIRST.
--- ---
@ -28,14 +30,14 @@ Say "Remembering..." then retrieve context before answering.
--- ---
## Complete Tool Reference (21 Tools) ## Complete Tool Reference (23 Tools)
### session_context — One-Call Initialization ### session_context — One-Call Initialization
``` ```
session_context({ session_context({
queries: ["user preferences", "project context"], // search queries queries: ["user preferences", "project context"], // search queries
context: { codebase: "project-name", topics: ["svelte", "rust"], file: "src/main.rs" }, context: { codebase: "project-name", topics: ["svelte", "rust"], file: "src/main.rs" },
token_budget: 2000, // 100-10000, controls response size token_budget: 2000, // 100-100000, controls response size
include_status: true, // system health include_status: true, // system health
include_intentions: true, // triggered reminders include_intentions: true, // triggered reminders
include_predictions: true // proactive memory predictions include_predictions: true // proactive memory predictions
@ -74,10 +76,13 @@ search({
min_similarity: 0.5, // minimum cosine similarity min_similarity: 0.5, // minimum cosine similarity
detail_level: "summary", // brief|summary|full detail_level: "summary", // brief|summary|full
context_topics: ["rust", "debugging"], // boost topic-matching memories context_topics: ["rust", "debugging"], // boost topic-matching memories
token_budget: 3000 // 100-10000, truncate to fit token_budget: 3000, // 100-100000, truncate to fit
retrieval_mode: "balanced" // precise|balanced|exhaustive (v2.1)
}) })
``` ```
Pipeline: Overfetch (3x, BM25+semantic) → Rerank (cross-encoder) → Temporal boost → Accessibility filter (FSRS-6) → Context match (Tulving 1973) → Competition (Anderson 1994) → Spreading activation. **Every search strengthens the memories it finds (Testing Effect).** Retrieval modes: `precise` (fast, no activation/competition), `balanced` (default 7-stage pipeline), `exhaustive` (5x overfetch, deep graph traversal, no competition suppression).
Pipeline: Overfetch → Rerank (cross-encoder) → Temporal boost → Accessibility filter (FSRS-6) → Context match (Tulving 1973) → Competition (Anderson 1994) → Spreading activation. **Every search strengthens the memories it finds (Testing Effect).**
### memory — Read, Edit, Delete, Promote, Demote ### memory — Read, Edit, Delete, Promote, Demote
``` ```
@ -87,8 +92,10 @@ memory({ action: "delete", id: "uuid" })
memory({ action: "promote", id: "uuid", reason: "was helpful" }) // +0.20 retrieval, +0.10 retention, 1.5x stability memory({ action: "promote", id: "uuid", reason: "was helpful" }) // +0.20 retrieval, +0.10 retention, 1.5x stability
memory({ action: "demote", id: "uuid", reason: "was wrong" }) // -0.30 retrieval, -0.15 retention, 0.5x stability memory({ action: "demote", id: "uuid", reason: "was wrong" }) // -0.30 retrieval, -0.15 retention, 0.5x stability
memory({ action: "state", id: "uuid" }) // Active/Dormant/Silent/Unavailable + accessibility score memory({ action: "state", id: "uuid" }) // Active/Dormant/Silent/Unavailable + accessibility score
memory({ action: "get_batch", ids: ["uuid1", "uuid2", "uuid3"] }) // retrieve up to 20 full memories at once (v2.1)
``` ```
Promote/demote does NOT delete — it adjusts ranking. Demoted memories rank lower; alternatives surface instead. Promote/demote does NOT delete — it adjusts ranking. Demoted memories rank lower; alternatives surface instead.
`get_batch` is designed for batch retrieval of expandable overflow IDs from search/session_context.
### codebase — Code Patterns & Architectural Decisions ### codebase — Code Patterns & Architectural Decisions
``` ```
@ -185,6 +192,27 @@ memory_graph({ center_id: "uuid", depth: 3, max_nodes: 100 })
``` ```
Returns nodes with force-directed positions + edges with weights. Returns nodes with force-directed positions + edges with weights.
### deep_reference — Cognitive Reasoning Engine (v2.0.4) ★ USE THIS FOR ALL FACTUAL QUESTIONS
```
deep_reference({ query: "What port does the dev server use?" })
deep_reference({ query: "Should I use prefix caching with vLLM?", depth: 30 })
```
**THE killer tool.** 8-stage cognitive reasoning pipeline:
1. Broad retrieval + cross-encoder reranking
2. Spreading activation expansion (finds connected memories search misses)
3. FSRS-6 trust scoring (retention × stability × reps ÷ lapses)
4. Intent classification (FactCheck / Timeline / RootCause / Comparison / Synthesis)
5. Temporal supersession (newer high-trust replaces older)
6. Trust-weighted contradiction analysis (only flags conflicts between strong memories)
7. Relation assessment (Supports / Contradicts / Supersedes / Irrelevant per pair)
8. **Template reasoning chain** — pre-built natural language reasoning the AI validates
Parameters: `query` (required), `depth` (5-50, default 20).
Returns: `intent`, `reasoning` (THE KEY FIELD — read this first), `recommended` (highest-trust answer), `evidence` (trust-sorted), `contradictions`, `superseded`, `evolution`, `related_insights`, `confidence`.
`cross_reference` is a backward-compatible alias that calls `deep_reference`.
### Maintenance Tools ### Maintenance Tools
``` ```
system_status() // health + stats + warnings + recommendations system_status() // health + stats + warnings + recommendations
@ -306,8 +334,8 @@ Memory is retrieval. Searching strengthens memory. Search liberally, save aggres
## Development ## Development
- **Crate:** `vestige-mcp` v2.0.1, Rust 2024 edition, MSRV 1.91 - **Crate:** `vestige-mcp` v2.0.4, Rust 2024 edition, MSRV 1.91
- **Tests:** 1,238 (352 unit + 192 E2E + cognitive + journey + extreme), zero warnings - **Tests:** 758 (406 mcp + 352 core), zero warnings
- **Build:** `cargo build --release -p vestige-mcp` (features: `embeddings` + `vector-search`) - **Build:** `cargo build --release -p vestige-mcp` (features: `embeddings` + `vector-search`)
- **Build (no embeddings):** `cargo build --release -p vestige-mcp --no-default-features` - **Build (no embeddings):** `cargo build --release -p vestige-mcp --no-default-features`
- **Bench:** `cargo bench -p vestige-core` - **Bench:** `cargo bench -p vestige-core`
@ -317,4 +345,4 @@ Memory is retrieval. Searching strengthens memory. Search liberally, save aggres
- **Vector index:** USearch HNSW (20x faster than FAISS) - **Vector index:** USearch HNSW (20x faster than FAISS)
- **Binaries:** `vestige-mcp` (MCP server), `vestige` (CLI), `vestige-restore` - **Binaries:** `vestige-mcp` (MCP server), `vestige` (CLI), `vestige-restore`
- **Dashboard:** SvelteKit 2 + Svelte 5 + Three.js + Tailwind 4, embedded at `/dashboard` - **Dashboard:** SvelteKit 2 + Svelte 5 + Three.js + Tailwind 4, embedded at `/dashboard`
- **Env vars:** `VESTIGE_DASHBOARD_PORT` (default 3927), `VESTIGE_CONSOLIDATION_INTERVAL_HOURS` (default 6), `RUST_LOG` - **Env vars:** `VESTIGE_DASHBOARD_PORT` (default 3927), `VESTIGE_HTTP_PORT` (default 3928), `VESTIGE_HTTP_BIND` (default 127.0.0.1), `VESTIGE_AUTH_TOKEN` (auto-generated), `VESTIGE_CONSOLIDATION_INTERVAL_HOURS` (default 6), `RUST_LOG`

6
CONTRIBUTING.md
View file

@ -26,7 +26,7 @@ vestige/
### Prerequisites ### Prerequisites
- **Rust** (1.85+ stable): [rustup.rs](https://rustup.rs) - **Rust** (1.91+ stable): [rustup.rs](https://rustup.rs)
- **Node.js** (v22+): [nodejs.org](https://nodejs.org) - **Node.js** (v22+): [nodejs.org](https://nodejs.org)
- **pnpm** (v9+): `npm install -g pnpm` - **pnpm** (v9+): `npm install -g pnpm`
@ -56,7 +56,7 @@ VESTIGE_TEST_MOCK_EMBEDDINGS=1 cargo test --workspace
## Running Tests ## Running Tests
```bash ```bash
# All tests (734 total) # All tests (746+ total)
VESTIGE_TEST_MOCK_EMBEDDINGS=1 cargo test --workspace VESTIGE_TEST_MOCK_EMBEDDINGS=1 cargo test --workspace
# Core library tests only (352 tests) # Core library tests only (352 tests)
@ -137,7 +137,7 @@ The MCP server and dashboard. Key modules:
|--------|---------| |--------|---------|
| `server.rs` | MCP JSON-RPC server (rmcp 0.14) | | `server.rs` | MCP JSON-RPC server (rmcp 0.14) |
| `cognitive.rs` | CognitiveEngine — 29 stateful modules | | `cognitive.rs` | CognitiveEngine — 29 stateful modules |
| `tools/` | One file per MCP tool (21 tools) | | `tools/` | One file per MCP tool (23 tools) |
| `dashboard/` | Axum HTTP + WebSocket + event bus | | `dashboard/` | Axum HTTP + WebSocket + event bus |
### apps/dashboard ### apps/dashboard

127
Cargo.lock generated
View file

@ -2073,6 +2073,15 @@ dependencies = [
"web-time", "web-time",
] ]
[[package]]
name = "indoc"
version = "2.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "79cf5c93f93228cf8efb3ba362535fb11199ac548a09ce117c9b1adc3030d706"
dependencies = [
"rustversion",
]
[[package]] [[package]]
name = "inotify" name = "inotify"
version = "0.11.0" version = "0.11.0"
@ -2467,6 +2476,15 @@ dependencies = [
"stable_deref_trait", "stable_deref_trait",
] ]
[[package]]
name = "memoffset"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a"
dependencies = [
"autocfg",
]
[[package]] [[package]]
name = "metal" name = "metal"
version = "0.29.0" version = "0.29.0"
@ -3188,6 +3206,69 @@ dependencies = [
"num-traits", "num-traits",
] ]
[[package]]
name = "pyo3"
version = "0.24.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5203598f366b11a02b13aa20cab591229ff0a89fd121a308a5df751d5fc9219"
dependencies = [
"cfg-if",
"indoc",
"libc",
"memoffset",
"once_cell",
"portable-atomic",
"pyo3-build-config",
"pyo3-ffi",
"pyo3-macros",
"unindent",
]
[[package]]
name = "pyo3-build-config"
version = "0.24.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "99636d423fa2ca130fa5acde3059308006d46f98caac629418e53f7ebb1e9999"
dependencies = [
"once_cell",
"target-lexicon",
]
[[package]]
name = "pyo3-ffi"
version = "0.24.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78f9cf92ba9c409279bc3305b5409d90db2d2c22392d443a87df3a1adad59e33"
dependencies = [
"libc",
"pyo3-build-config",
]
[[package]]
name = "pyo3-macros"
version = "0.24.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b999cb1a6ce21f9a6b147dcf1be9ffedf02e0043aec74dc390f3007047cecd9"
dependencies = [
"proc-macro2",
"pyo3-macros-backend",
"quote",
"syn",
]
[[package]]
name = "pyo3-macros-backend"
version = "0.24.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "822ece1c7e1012745607d5cf0bcb2874769f0f7cb34c4cde03b9358eb9ef911a"
dependencies = [
"heck",
"proc-macro2",
"pyo3-build-config",
"quote",
"syn",
]
[[package]] [[package]]
name = "qoi" name = "qoi"
version = "0.4.1" version = "0.4.1"
@ -3908,6 +3989,12 @@ dependencies = [
"libc", "libc",
] ]
[[package]]
name = "target-lexicon"
version = "0.13.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "adb6935a6f5c20170eeceb1a3835a49e12e19d792f6dd344ccc76a985ca5a6ca"
[[package]] [[package]]
name = "tempfile" name = "tempfile"
version = "3.25.0" version = "3.25.0"
@ -4355,6 +4442,12 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39ec24b3121d976906ece63c9daad25b85969647682eee313cb5779fdd69e14e" checksum = "39ec24b3121d976906ece63c9daad25b85969647682eee313cb5779fdd69e14e"
[[package]]
name = "unindent"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3"
[[package]] [[package]]
name = "untrusted" name = "untrusted"
version = "0.9.0" version = "0.9.0"
@ -4492,9 +4585,33 @@ version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "vestige-agent"
version = "0.1.0"
dependencies = [
"chrono",
"rand",
"serde",
"serde_json",
"tempfile",
"thiserror 2.0.18",
"tracing",
"uuid",
"xxhash-rust",
]
[[package]]
name = "vestige-agent-py"
version = "0.1.0"
dependencies = [
"pyo3",
"serde_json",
"vestige-agent",
]
[[package]] [[package]]
name = "vestige-core" name = "vestige-core"
version = "2.0.2" version = "2.0.4"
dependencies = [ dependencies = [
"chrono", "chrono",
"criterion", "criterion",
@ -4529,7 +4646,7 @@ dependencies = [
[[package]] [[package]]
name = "vestige-mcp" name = "vestige-mcp"
version = "2.0.2" version = "2.0.4"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"axum", "axum",
@ -5118,6 +5235,12 @@ version = "0.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9edde0db4769d2dc68579893f2306b26c6ecfbe0ef499b013d731b7b9247e0b9" checksum = "9edde0db4769d2dc68579893f2306b26c6ecfbe0ef499b013d731b7b9247e0b9"
[[package]]
name = "xxhash-rust"
version = "0.8.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fdd20c5420375476fbd4394763288da7eb0cc0b8c11deed431a91562af7335d3"
[[package]] [[package]]
name = "y4m" name = "y4m"
version = "0.8.0" version = "0.8.0"

4
Cargo.toml
View file

@ -3,6 +3,8 @@ resolver = "2"
members = [ members = [
"crates/vestige-core", "crates/vestige-core",
"crates/vestige-mcp", "crates/vestige-mcp",
"crates/vestige-agent",
"crates/vestige-agent-py",
"tests/e2e", "tests/e2e",
] ]
exclude = [ exclude = [
@ -10,7 +12,7 @@ exclude = [
] ]
[workspace.package] [workspace.package]
version = "2.0.1" version = "2.0.4"
edition = "2024" edition = "2024"
license = "AGPL-3.0-only" license = "AGPL-3.0-only"
repository = "https://github.com/samvallad33/vestige" repository = "https://github.com/samvallad33/vestige"

14
README.md
View file

@ -6,7 +6,7 @@
[![GitHub stars](https://img.shields.io/github/stars/samvallad33/vestige?style=social)](https://github.com/samvallad33/vestige) [![GitHub stars](https://img.shields.io/github/stars/samvallad33/vestige?style=social)](https://github.com/samvallad33/vestige)
[![Release](https://img.shields.io/github/v/release/samvallad33/vestige)](https://github.com/samvallad33/vestige/releases/latest) [![Release](https://img.shields.io/github/v/release/samvallad33/vestige)](https://github.com/samvallad33/vestige/releases/latest)
[![Tests](https://img.shields.io/badge/tests-1238%20passing-brightgreen)](https://github.com/samvallad33/vestige/actions) [![Tests](https://img.shields.io/badge/tests-758%20passing-brightgreen)](https://github.com/samvallad33/vestige/actions)
[![License](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE) [![License](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE)
[![MCP Compatible](https://img.shields.io/badge/MCP-compatible-green)](https://modelcontextprotocol.io) [![MCP Compatible](https://img.shields.io/badge/MCP-compatible-green)](https://modelcontextprotocol.io)
@ -14,7 +14,7 @@
Built on 130 years of memory research — FSRS-6 spaced repetition, prediction error gating, synaptic tagging, spreading activation, memory dreaming — all running in a single Rust binary with a 3D neural visualization dashboard. 100% local. Zero cloud. Built on 130 years of memory research — FSRS-6 spaced repetition, prediction error gating, synaptic tagging, spreading activation, memory dreaming — all running in a single Rust binary with a 3D neural visualization dashboard. 100% local. Zero cloud.
[Quick Start](#quick-start) | [Dashboard](#-3d-memory-dashboard) | [How It Works](#-the-cognitive-science-stack) | [Tools](#-21-mcp-tools) | [Docs](docs/) [Quick Start](#quick-start) | [Dashboard](#-3d-memory-dashboard) | [How It Works](#-the-cognitive-science-stack) | [Tools](#-23-mcp-tools) | [Docs](docs/)
</div> </div>
@ -132,7 +132,7 @@ The dashboard runs automatically at `http://localhost:3927/dashboard` when the M
│ 15 REST endpoints · WS event broadcast │ │ 15 REST endpoints · WS event broadcast │
├─────────────────────────────────────────────────────┤ ├─────────────────────────────────────────────────────┤
│ MCP Server (stdio JSON-RPC) │ │ MCP Server (stdio JSON-RPC) │
│ 21 tools · 29 cognitive modules │ │ 23 tools · 29 cognitive modules │
├─────────────────────────────────────────────────────┤ ├─────────────────────────────────────────────────────┤
│ Cognitive Engine │ │ Cognitive Engine │
│ ┌──────────┐ ┌──────────┐ ┌───────────────┐ │ │ ┌──────────┐ ┌──────────┐ ┌───────────────┐ │
@ -196,7 +196,7 @@ This isn't a key-value store with an embedding model bolted on. Vestige implemen
--- ---
## 🛠 21 MCP Tools ## 🛠 23 MCP Tools
### Context Packets ### Context Packets
| Tool | What It Does | | Tool | What It Does |
@ -241,6 +241,12 @@ This isn't a key-value store with an embedding model bolted on. Vestige implemen
| `backup` / `export` / `gc` | Database backup, JSON export, garbage collection | | `backup` / `export` / `gc` | Database backup, JSON export, garbage collection |
| `restore` | Restore from JSON backup | | `restore` | Restore from JSON backup |
### Deep Reference (v2.0.4)
| Tool | What It Does |
|------|-------------|
| `deep_reference` | **Cognitive reasoning across memories.** 8-stage pipeline: FSRS-6 trust scoring, intent classification, spreading activation, temporal supersession, contradiction analysis, relation assessment, dream insight integration, and algorithmic reasoning chain generation. Returns trust-scored evidence with a pre-built reasoning scaffold. |
| `cross_reference` | Backward-compatible alias for `deep_reference`. |
--- ---
## Make Your AI Use Vestige Automatically ## Make Your AI Use Vestige Automatically

4
SECURITY.md
View file

@ -4,8 +4,8 @@
| Version | Supported | | Version | Supported |
| ------- | ------------------ | | ------- | ------------------ |
| 1.1.x | :white_check_mark: | | 2.0.x | :white_check_mark: |
| 1.0.x | :x: | | 1.x | :x: |
## Reporting a Vulnerability ## Reporting a Vulnerability

2
crates/vestige-core/Cargo.toml
View file

@ -1,6 +1,6 @@
[package] [package]
name = "vestige-core" name = "vestige-core"
version = "2.0.3" version = "2.0.4"
edition = "2024" edition = "2024"
rust-version = "1.91" rust-version = "1.91"
authors = ["Vestige Team"] authors = ["Vestige Team"]

2
crates/vestige-core/src/advanced/mod.rs
View file

@ -29,7 +29,7 @@ pub mod speculative;
// Re-exports for convenient access // Re-exports for convenient access
pub use adaptive_embedding::{AdaptiveEmbedder, ContentType, EmbeddingStrategy, Language}; pub use adaptive_embedding::{AdaptiveEmbedder, ContentType, EmbeddingStrategy, Language};
pub use chains::{ChainStep, ConnectionType, MemoryChainBuilder, MemoryPath, ReasoningChain}; pub use chains::{ChainStep, Connection, ConnectionType, MemoryChainBuilder, MemoryNode, MemoryPath, ReasoningChain};
pub use compression::{CompressedMemory, CompressionConfig, CompressionStats, MemoryCompressor}; pub use compression::{CompressedMemory, CompressionConfig, CompressionStats, MemoryCompressor};
pub use cross_project::{ pub use cross_project::{
ApplicableKnowledge, CrossProjectLearner, ProjectContext, UniversalPattern, ApplicableKnowledge, CrossProjectLearner, ProjectContext, UniversalPattern,

5
crates/vestige-core/src/storage/migrations.rs
View file

@ -315,7 +315,10 @@ const MIGRATION_V4_UP: &str = r#"
-- TEMPORAL KNOWLEDGE GRAPH (Like Zep's Graphiti) -- TEMPORAL KNOWLEDGE GRAPH (Like Zep's Graphiti)
-- ============================================================================ -- ============================================================================
-- Knowledge edges for temporal reasoning -- DEPRECATED (v2.1.0): knowledge_edges is unused. All graph edges use
-- memory_connections (migration V3). This table was designed for bi-temporal
-- edge support but was never wired. Retained for schema compatibility with
-- existing databases. Do NOT add queries against this table.
CREATE TABLE IF NOT EXISTS knowledge_edges ( CREATE TABLE IF NOT EXISTS knowledge_edges (
id TEXT PRIMARY KEY, id TEXT PRIMARY KEY,
source_id TEXT NOT NULL, source_id TEXT NOT NULL,

57
crates/vestige-core/src/storage/sqlite.rs
View file

@ -227,6 +227,7 @@ impl Storage {
.lock() .lock()
.map_err(|_| StorageError::Init("Vector index lock poisoned".to_string()))?; .map_err(|_| StorageError::Init("Vector index lock poisoned".to_string()))?;
let mut load_failures = 0u32;
for (node_id, embedding_bytes) in embeddings { for (node_id, embedding_bytes) in embeddings {
if let Some(embedding) = Embedding::from_bytes(&embedding_bytes) { if let Some(embedding) = Embedding::from_bytes(&embedding_bytes) {
// Handle Matryoshka migration: old 768-dim → truncate to 256-dim // Handle Matryoshka migration: old 768-dim → truncate to 256-dim
@ -236,10 +237,14 @@ impl Storage {
embedding.vector embedding.vector
}; };
if let Err(e) = index.add(&node_id, &vector) { if let Err(e) = index.add(&node_id, &vector) {
load_failures += 1;
tracing::warn!("Failed to load embedding for {}: {}", node_id, e); tracing::warn!("Failed to load embedding for {}: {}", node_id, e);
} }
} }
} }
if load_failures > 0 {
tracing::error!(count = load_failures, "Vector index: {} embeddings failed to load", load_failures);
}
Ok(()) Ok(())
} }
@ -399,7 +404,11 @@ impl Storage {
superseded_id: None, superseded_id: None,
similarity: None, similarity: None,
prediction_error: Some(prediction_error), prediction_error: Some(prediction_error),
reason: format!("Created new memory: {:?}. Related: {:?}", reason, related_memory_ids), reason: if related_memory_ids.is_empty() {
format!("Created new memory: {:?}", reason)
} else {
format!("Created new memory: {:?}. Semantically similar (not linked): {:?}", reason, related_memory_ids)
},
}) })
} }
GateDecision::Update { target_id, similarity, update_type, prediction_error } => { GateDecision::Update { target_id, similarity, update_type, prediction_error } => {
@ -667,7 +676,13 @@ impl Storage {
/// Convert a row to KnowledgeNode /// Convert a row to KnowledgeNode
fn row_to_node(row: &rusqlite::Row) -> rusqlite::Result<KnowledgeNode> { fn row_to_node(row: &rusqlite::Row) -> rusqlite::Result<KnowledgeNode> {
let tags_json: String = row.get("tags")?; let tags_json: String = row.get("tags")?;
let tags: Vec<String> = serde_json::from_str(&tags_json).unwrap_or_default(); let tags: Vec<String> = match serde_json::from_str(&tags_json) {
Ok(t) => t,
Err(e) => {
tracing::warn!(raw = %tags_json, "Failed to deserialize tags JSON, using empty: {}", e);
Vec::new()
}
};
let created_at: String = row.get("created_at")?; let created_at: String = row.get("created_at")?;
let updated_at: String = row.get("updated_at")?; let updated_at: String = row.get("updated_at")?;
@ -955,6 +970,8 @@ impl Storage {
pub fn strengthen_batch_on_access(&self, ids: &[&str]) -> Result<()> { pub fn strengthen_batch_on_access(&self, ids: &[&str]) -> Result<()> {
for id in ids { for id in ids {
self.strengthen_on_access(id)?; self.strengthen_on_access(id)?;
// Also record access in memory_states for audit trail (Bug #1 fix)
let _ = self.record_memory_access(id);
} }
Ok(()) Ok(())
} }
@ -3223,6 +3240,42 @@ impl Storage {
})) }))
} }
/// Get dream history (most recent first)
pub fn get_dream_history(&self, limit: i32) -> Result<Vec<DreamHistoryRecord>> {
let reader = self.reader.lock()
.map_err(|_| StorageError::Init("Reader lock poisoned".into()))?;
let mut stmt = reader.prepare(
"SELECT dreamed_at, duration_ms, memories_replayed, connections_found,
insights_generated, memories_strengthened, memories_compressed,
phase_nrem1_ms, phase_nrem3_ms, phase_rem_ms, phase_integration_ms,
summaries_generated, emotional_memories_processed, creative_connections_found
FROM dream_history ORDER BY dreamed_at DESC LIMIT ?1"
)?;
let records = stmt.query_map(params![limit], |row| {
let dreamed_at_str: String = row.get(0)?;
let dreamed_at = DateTime::parse_from_rfc3339(&dreamed_at_str)
.map(|dt| dt.with_timezone(&Utc))
.unwrap_or_else(|_| Utc::now());
Ok(DreamHistoryRecord {
dreamed_at,
duration_ms: row.get(1)?,
memories_replayed: row.get(2)?,
connections_found: row.get(3)?,
insights_generated: row.get(4)?,
memories_strengthened: row.get(5)?,
memories_compressed: row.get(6)?,
phase_nrem1_ms: row.get(7)?,
phase_nrem3_ms: row.get(8)?,
phase_rem_ms: row.get(9)?,
phase_integration_ms: row.get(10)?,
summaries_generated: row.get(11)?,
emotional_memories_processed: row.get(12)?,
creative_connections_found: row.get(13)?,
})
})?.collect::<std::result::Result<Vec<_>, _>>()?;
Ok(records)
}
/// Count memories created since a given timestamp /// Count memories created since a given timestamp
pub fn count_memories_since(&self, since: DateTime<Utc>) -> Result<i64> { pub fn count_memories_since(&self, since: DateTime<Utc>) -> Result<i64> {
let reader = self.reader.lock() let reader = self.reader.lock()

4
crates/vestige-mcp/Cargo.toml
View file

@ -1,6 +1,6 @@
[package] [package]
name = "vestige-mcp" name = "vestige-mcp"
version = "2.0.3" version = "2.0.4"
edition = "2024" edition = "2024"
description = "Cognitive memory MCP server for Claude - FSRS-6, spreading activation, synaptic tagging, 3D dashboard, and 130 years of memory research" description = "Cognitive memory MCP server for Claude - FSRS-6, spreading activation, synaptic tagging, 3D dashboard, and 130 years of memory research"
authors = ["samvallad33"] authors = ["samvallad33"]
@ -32,7 +32,7 @@ path = "src/bin/cli.rs"
# ============================================================================ # ============================================================================
# Includes: FSRS-6, spreading activation, synaptic tagging, hippocampal indexing, # Includes: FSRS-6, spreading activation, synaptic tagging, hippocampal indexing,
# memory states, context memory, importance signals, dreams, and more # memory states, context memory, importance signals, dreams, and more
vestige-core = { version = "2.0.3", path = "../vestige-core", default-features = false, features = ["bundled-sqlite"] } vestige-core = { version = "2.0.4", path = "../vestige-core", default-features = false, features = ["bundled-sqlite"] }
# ============================================================================ # ============================================================================
# MCP Server Dependencies # MCP Server Dependencies

2
crates/vestige-mcp/src/main.rs
View file

@ -307,7 +307,7 @@ async fn main() {
Ok(token) => { Ok(token) => {
let bind = std::env::var("VESTIGE_HTTP_BIND").unwrap_or_else(|_| "127.0.0.1".to_string()); let bind = std::env::var("VESTIGE_HTTP_BIND").unwrap_or_else(|_| "127.0.0.1".to_string());
eprintln!("Vestige HTTP transport: http://{}:{}/mcp", bind, http_port); eprintln!("Vestige HTTP transport: http://{}:{}/mcp", bind, http_port);
eprintln!("Auth token: {}...", &token[..8]); eprintln!("Auth token: {}...", &token[..token.len().min(8)]);
tokio::spawn(async move { tokio::spawn(async move {
if let Err(e) = protocol::http::start_http_transport( if let Err(e) = protocol::http::start_http_transport(
http_storage, http_storage,

21
crates/vestige-mcp/src/protocol/http.rs
View file

@ -108,7 +108,20 @@ pub async fn start_http_transport(
ServiceBuilder::new() ServiceBuilder::new()
.layer(DefaultBodyLimit::max(MAX_BODY_SIZE)) .layer(DefaultBodyLimit::max(MAX_BODY_SIZE))
.layer(ConcurrencyLimitLayer::new(CONCURRENCY_LIMIT)) .layer(ConcurrencyLimitLayer::new(CONCURRENCY_LIMIT))
.layer(CorsLayer::permissive()), .layer(
CorsLayer::new()
.allow_origin(
[
format!("http://127.0.0.1:{}", port),
format!("http://localhost:{}", port),
]
.into_iter()
.filter_map(|s| s.parse().ok())
.collect::<Vec<_>>(),
)
.allow_methods([axum::http::Method::POST, axum::http::Method::DELETE, axum::http::Method::OPTIONS])
.allow_headers([axum::http::header::CONTENT_TYPE, axum::http::header::AUTHORIZATION])
),
) )
.with_state(state); .with_state(state);
@ -229,13 +242,13 @@ async fn post_mcp(
match response { match response {
Some(resp) => { Some(resp) => {
let mut resp_headers = HeaderMap::new(); let mut resp_headers = HeaderMap::new();
resp_headers.insert("mcp-session-id", session_id.parse().unwrap()); resp_headers.insert("mcp-session-id", session_id.parse().unwrap_or_else(|_| axum::http::HeaderValue::from_static("invalid")));
(StatusCode::OK, resp_headers, Json(resp)).into_response() (StatusCode::OK, resp_headers, Json(resp)).into_response()
} }
None => { None => {
// Notifications return 202 // Notifications return 202
let mut resp_headers = HeaderMap::new(); let mut resp_headers = HeaderMap::new();
resp_headers.insert("mcp-session-id", session_id.parse().unwrap()); resp_headers.insert("mcp-session-id", session_id.parse().unwrap_or_else(|_| axum::http::HeaderValue::from_static("invalid")));
(StatusCode::ACCEPTED, resp_headers).into_response() (StatusCode::ACCEPTED, resp_headers).into_response()
} }
} }
@ -275,7 +288,7 @@ async fn post_mcp(
}; };
let mut resp_headers = HeaderMap::new(); let mut resp_headers = HeaderMap::new();
resp_headers.insert("mcp-session-id", session_id.parse().unwrap()); resp_headers.insert("mcp-session-id", session_id.parse().unwrap_or_else(|_| axum::http::HeaderValue::from_static("invalid")));
match response { match response {
Some(resp) => (StatusCode::OK, resp_headers, Json(resp)).into_response(), Some(resp) => (StatusCode::OK, resp_headers, Json(resp)).into_response(),

6
crates/vestige-mcp/src/resources/memory.rs
View file

@ -95,7 +95,7 @@ async fn read_recent(storage: &Arc<Storage>, limit: i32) -> Result<String, Strin
serde_json::json!({ serde_json::json!({
"id": n.id, "id": n.id,
"summary": if n.content.len() > 200 { "summary": if n.content.len() > 200 {
format!("{}...", &n.content[..200]) format!("{}...", &n.content[..n.content.floor_char_boundary(200)])
} else { } else {
n.content.clone() n.content.clone()
}, },
@ -139,7 +139,7 @@ async fn read_decaying(storage: &Arc<Storage>) -> Result<String, String> {
serde_json::json!({ serde_json::json!({
"id": n.id, "id": n.id,
"summary": if n.content.len() > 200 { "summary": if n.content.len() > 200 {
format!("{}...", &n.content[..200]) format!("{}...", &n.content[..n.content.floor_char_boundary(200)])
} else { } else {
n.content.clone() n.content.clone()
}, },
@ -180,7 +180,7 @@ async fn read_due(storage: &Arc<Storage>) -> Result<String, String> {
serde_json::json!({ serde_json::json!({
"id": n.id, "id": n.id,
"summary": if n.content.len() > 200 { "summary": if n.content.len() > 200 {
format!("{}...", &n.content[..200]) format!("{}...", &n.content[..n.content.floor_char_boundary(200)])
} else { } else {
n.content.clone() n.content.clone()
}, },

24
crates/vestige-mcp/src/server.rs
View file

@ -159,7 +159,7 @@ impl McpServer {
/// Handle tools/list request /// Handle tools/list request
async fn handle_tools_list(&self) -> Result<serde_json::Value, JsonRpcError> { async fn handle_tools_list(&self) -> Result<serde_json::Value, JsonRpcError> {
// v1.8: 19 tools. Deprecated tools still work via redirects in handle_tools_call. // v2.0.4: 23 tools. Deprecated tools still work via redirects in handle_tools_call.
let tools = vec![ let tools = vec![
// ================================================================ // ================================================================
// UNIFIED TOOLS (v1.1+) // UNIFIED TOOLS (v1.1+)
@ -293,6 +293,19 @@ impl McpServer {
description: Some("Subgraph export for visualization. Input: center_id or query, depth (1-3), max_nodes. Returns nodes with force-directed layout positions and edges with weights. Powers memory graph visualization.".to_string()), description: Some("Subgraph export for visualization. Input: center_id or query, depth (1-3), max_nodes. Returns nodes with force-directed layout positions and edges with weights. Powers memory graph visualization.".to_string()),
input_schema: tools::graph::schema(), input_schema: tools::graph::schema(),
}, },
// ================================================================
// DEEP REFERENCE (v2.0.4+) — replaces cross_reference
// ================================================================
ToolDescription {
name: "deep_reference".to_string(),
description: Some("Deep cognitive reasoning across memories. Combines FSRS-6 trust scoring, spreading activation, temporal supersession, dream insights, and contradiction analysis to build a complete understanding of a topic. Returns trust-scored evidence, fact evolution timeline, and a recommended answer. Use this when accuracy matters.".to_string()),
input_schema: tools::cross_reference::schema(),
},
ToolDescription {
name: "cross_reference".to_string(),
description: Some("Alias for deep_reference. Connect the dots across memories with cognitive reasoning.".to_string()),
input_schema: tools::cross_reference::schema(),
},
]; ];
let result = ListToolsResult { tools }; let result = ListToolsResult { tools };
@ -644,6 +657,7 @@ impl McpServer {
// ================================================================ // ================================================================
"memory_health" => tools::health::execute(&self.storage, request.arguments).await, "memory_health" => tools::health::execute(&self.storage, request.arguments).await,
"memory_graph" => tools::graph::execute(&self.storage, request.arguments).await, "memory_graph" => tools::graph::execute(&self.storage, request.arguments).await,
"deep_reference" | "cross_reference" => tools::cross_reference::execute(&self.storage, &self.cognitive, request.arguments).await,
name => { name => {
return Err(JsonRpcError::method_not_found_with_message(&format!( return Err(JsonRpcError::method_not_found_with_message(&format!(
@ -1187,8 +1201,8 @@ mod tests {
let result = response.result.unwrap(); let result = response.result.unwrap();
let tools = result["tools"].as_array().unwrap(); let tools = result["tools"].as_array().unwrap();
// v1.9: 21 tools (4 unified + 1 core + 2 temporal + 5 maintenance + 2 auto-save + 3 cognitive + 1 restore + 1 session_context + 2 autonomic) // v2.0.4: 23 tools (4 unified + 1 core + 2 temporal + 5 maintenance + 2 auto-save + 3 cognitive + 1 restore + 1 session_context + 2 autonomic + 1 deep_reference + 1 cross_reference alias)
assert_eq!(tools.len(), 21, "Expected exactly 21 tools in v1.9+"); assert_eq!(tools.len(), 23, "Expected exactly 23 tools in v2.0.4+");
let tool_names: Vec<&str> = tools let tool_names: Vec<&str> = tools
.iter() .iter()
@ -1239,6 +1253,10 @@ mod tests {
// Autonomic tools (v1.9) // Autonomic tools (v1.9)
assert!(tool_names.contains(&"memory_health")); assert!(tool_names.contains(&"memory_health"));
assert!(tool_names.contains(&"memory_graph")); assert!(tool_names.contains(&"memory_graph"));
// Deep reference + cross_reference alias (v2.0.4)
assert!(tool_names.contains(&"deep_reference"));
assert!(tool_names.contains(&"cross_reference"));
} }
#[tokio::test] #[tokio::test]

19
crates/vestige-mcp/src/tools/changelog.rs
View file

@ -140,6 +140,11 @@ fn execute_system_wide(
.get_recent_state_transitions(limit) .get_recent_state_transitions(limit)
.map_err(|e| e.to_string())?; .map_err(|e| e.to_string())?;
// Get dream history (Bug #9 fix — dreams were invisible to audit trail)
let dreams = storage
.get_dream_history(limit)
.unwrap_or_default();
// Build unified event list // Build unified event list
let mut events: Vec<(DateTime<Utc>, Value)> = Vec::new(); let mut events: Vec<(DateTime<Utc>, Value)> = Vec::new();
@ -174,6 +179,20 @@ fn execute_system_wide(
)); ));
} }
for d in &dreams {
events.push((
d.dreamed_at,
serde_json::json!({
"type": "dream",
"timestamp": d.dreamed_at.to_rfc3339(),
"durationMs": d.duration_ms,
"memoriesReplayed": d.memories_replayed,
"connectionFound": d.connections_found,
"insightsGenerated": d.insights_generated,
}),
));
}
// Sort by timestamp descending // Sort by timestamp descending
events.sort_by(|a, b| b.0.cmp(&a.0)); events.sort_by(|a, b| b.0.cmp(&a.0));

809
crates/vestige-mcp/src/tools/cross_reference.rs Normal file
View file

@ -0,0 +1,809 @@
//! Deep Reference Tool (v2.0.4)
//!
//! Cognitive reasoning engine across memories. Combines:
//! 1. Broad retrieval (hybrid search + reranking)
//! 2. Spreading activation expansion (connected memories)
//! 3. FSRS-6 trust scoring (retention, stability, reps, lapses)
//! 4. Temporal supersession (newer = current truth)
//! 5. Contradiction analysis (trust-weighted)
//! 6. Dream insight integration (persisted insights)
//! 7. Structured synthesis (recommended answer + evidence)
//!
//! Research grounding: MAGMA (multi-graph), Kumiho (AGM belief revision),
//! InfMem (System-2 memory control), D-Mem (dual-process retrieval).
//!
//! Replaces cross_reference with full cognitive reasoning. cross_reference
//! is kept as a backward-compatible alias.
use chrono::Utc;
use serde::Deserialize;
use serde_json::Value;
use std::sync::Arc;
use tokio::sync::Mutex;
use crate::cognitive::CognitiveEngine;
use vestige_core::Storage;
/// Input schema for deep_reference / cross_reference tool
pub fn schema() -> Value {
serde_json::json!({
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The question, claim, or topic to reason about across all memories"
},
"depth": {
"type": "integer",
"description": "How many memories to analyze (default: 20, max: 50). Higher = more thorough.",
"default": 20,
"minimum": 5,
"maximum": 50
}
},
"required": ["query"]
})
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
struct DeepRefArgs {
query: String,
depth: Option<i32>,
}
// ============================================================================
// FSRS-6 Trust Score
// ============================================================================
/// Compute trust score from FSRS-6 memory state.
/// Higher = more trustworthy (frequently accessed, high retention, stable, few lapses).
fn compute_trust(retention: f64, stability: f64, reps: i32, lapses: i32) -> f64 {
let retention_factor = retention * 0.4;
let stability_factor = (stability / 30.0).min(1.0) * 0.2;
let reps_factor = (reps as f64 / 10.0).min(1.0) * 0.2;
let lapses_penalty = (1.0 - (lapses as f64 / 5.0)).max(0.0) * 0.2;
(retention_factor + stability_factor + reps_factor + lapses_penalty).clamp(0.0, 1.0)
}
// ============================================================================
// SYSTEM 1: Intent Classification (MAGMA-inspired query routing)
// ============================================================================
#[derive(Debug, Clone, PartialEq)]
enum QueryIntent {
FactCheck, // "Is X true?" → find support/contradiction evidence
Timeline, // "When did X happen?" → temporal ordering + pattern detection
RootCause, // "Why did X happen?" → causal chain backward
Comparison, // "How does X differ from Y?" → diff two memory clusters
Synthesis, // Default: "What do I know about X?" → cluster + best per cluster
}
fn classify_intent(query: &str) -> QueryIntent {
let q = query.to_lowercase();
let patterns: &[(QueryIntent, &[&str])] = &[
(QueryIntent::RootCause, &["why did", "root cause", "what caused", "because of", "reason for", "why is", "why was"]),
(QueryIntent::Timeline, &["when did", "timeline", "history of", "over time", "how has", "evolution of", "sequence of"]),
(QueryIntent::Comparison, &["differ", "compare", "versus", " vs ", "difference between", "changed from"]),
(QueryIntent::FactCheck, &["is it true", "did i", "was there", "verify", "confirm", "is this correct", "should i use", "should we"]),
];
for (intent, keywords) in patterns {
if keywords.iter().any(|kw| q.contains(kw)) {
return intent.clone();
}
}
QueryIntent::Synthesis
}
// ============================================================================
// SYSTEM 2: Relation Assessment (embedding similarity + sentiment + temporal)
// ============================================================================
#[derive(Debug, Clone)]
enum Relation {
Supports,
Contradicts,
Supersedes,
Irrelevant,
}
#[derive(Debug, Clone)]
#[allow(dead_code)]
struct RelationAssessment {
relation: Relation,
confidence: f64,
reasoning: String,
}
/// Assess the relationship between two memories using embedding similarity,
/// correction signals, temporal ordering, and trust comparison.
/// No LLM needed — pure algorithmic assessment.
fn assess_relation(a_content: &str, b_content: &str, a_trust: f64, b_trust: f64,
a_date: chrono::DateTime<Utc>, b_date: chrono::DateTime<Utc>,
topic_sim: f32) -> RelationAssessment {
// Irrelevant: different topics
if topic_sim < 0.15 {
return RelationAssessment {
relation: Relation::Irrelevant,
confidence: 1.0 - topic_sim as f64,
reasoning: format!("Different topics (similarity {:.2})", topic_sim),
};
}
let time_delta_days = (b_date - a_date).num_days().abs();
let trust_diff = b_trust - a_trust;
let has_correction = appears_contradictory(a_content, b_content);
// Supersession: same topic + newer + higher trust
if topic_sim > 0.4 && time_delta_days > 0 && trust_diff > 0.05 && !has_correction {
let (newer, older) = if b_date > a_date { ("B", "A") } else { ("A", "B") };
return RelationAssessment {
relation: Relation::Supersedes,
confidence: topic_sim as f64 * (0.5 + trust_diff.min(0.5)),
reasoning: format!("{} supersedes {} (newer by {}d, trust +{:.0}%)",
newer, older, time_delta_days, trust_diff * 100.0),
};
}
// Contradiction: same topic + correction signals detected
if has_correction && topic_sim > 0.15 {
return RelationAssessment {
relation: Relation::Contradicts,
confidence: topic_sim as f64 * 0.8,
reasoning: format!("Contradiction detected (similarity {:.2}, correction signals present)", topic_sim),
};
}
// Support: same topic + no contradiction
if topic_sim > 0.3 {
return RelationAssessment {
relation: Relation::Supports,
confidence: topic_sim as f64,
reasoning: format!("Topically aligned (similarity {:.2}), consistent stance", topic_sim),
};
}
RelationAssessment {
relation: Relation::Irrelevant,
confidence: 0.3,
reasoning: "Weak relationship".to_string(),
}
}
// ============================================================================
// SYSTEM 3: Template Reasoning Chain Generator (no LLM needed)
// ============================================================================
/// Generate a natural language reasoning chain from structured evidence.
/// The AI reads this and validates/extends it — System 1 prepares, System 2 refines.
fn generate_reasoning_chain(
query: &str,
intent: &QueryIntent,
primary: &ScoredMemory,
relations: &[(String, f64, RelationAssessment)], // (preview, trust, relation)
confidence: f64,
) -> String {
let mut chain = String::new();
// Intent-specific opening
match intent {
QueryIntent::FactCheck => {
chain.push_str(&format!(
"FACT CHECK: \"{}\"\n\n", query
));
}
QueryIntent::Timeline => {
chain.push_str(&format!(
"TIMELINE: \"{}\"\n\n", query
));
}
QueryIntent::RootCause => {
chain.push_str(&format!(
"ROOT CAUSE ANALYSIS: \"{}\"\n\n", query
));
}
QueryIntent::Comparison => {
chain.push_str(&format!(
"COMPARISON: \"{}\"\n\n", query
));
}
QueryIntent::Synthesis => {
chain.push_str(&format!(
"SYNTHESIS: \"{}\"\n\n", query
));
}
}
// Primary finding
chain.push_str(&format!(
"PRIMARY FINDING (trust {:.0}%, {}): {}\n",
primary.trust * 100.0,
primary.updated_at.format("%b %d, %Y"),
primary.content.chars().take(150).collect::<String>(),
));
// Superseded memories
let superseded: Vec<_> = relations.iter()
.filter(|(_, _, r)| matches!(r.relation, Relation::Supersedes))
.collect();
for (preview, trust, rel) in &superseded {
chain.push_str(&format!(
" SUPERSEDES (trust {:.0}%): \"{}\" — {}\n",
trust * 100.0,
preview.chars().take(80).collect::<String>(),
rel.reasoning,
));
}
// Supporting evidence
let supporting: Vec<_> = relations.iter()
.filter(|(_, _, r)| matches!(r.relation, Relation::Supports))
.collect();
if !supporting.is_empty() {
chain.push_str(&format!("\nSUPPORTED BY {} MEMOR{}:\n",
supporting.len(),
if supporting.len() == 1 { "Y" } else { "IES" },
));
for (preview, trust, _) in supporting.iter().take(5) {
chain.push_str(&format!(
" + (trust {:.0}%): \"{}\"\n",
trust * 100.0,
preview.chars().take(80).collect::<String>(),
));
}
}
// Contradicting evidence
let contradicting: Vec<_> = relations.iter()
.filter(|(_, _, r)| matches!(r.relation, Relation::Contradicts))
.collect();
if !contradicting.is_empty() {
chain.push_str(&format!("\nCONTRADICTING EVIDENCE ({}):\n", contradicting.len()));
for (preview, trust, rel) in contradicting.iter().take(3) {
chain.push_str(&format!(
" ! (trust {:.0}%): \"{}\" — {}\n",
trust * 100.0,
preview.chars().take(80).collect::<String>(),
rel.reasoning,
));
}
}
// Confidence summary
chain.push_str(&format!("\nOVERALL CONFIDENCE: {:.0}%\n", confidence * 100.0));
chain
}
// ============================================================================
// Contradiction Detection (enhanced with relation assessment)
// ============================================================================
const NEGATION_PAIRS: &[(&str, &str)] = &[
("don't", "do"), ("never", "always"), ("avoid", "use"),
("wrong", "right"), ("incorrect", "correct"),
("deprecated", "recommended"), ("outdated", "current"),
("removed", "added"), ("disabled", "enabled"),
("not ", ""), ("no longer", ""),
];
const CORRECTION_SIGNALS: &[&str] = &[
"actually", "correction", "update:", "updated:", "fixed",
"was wrong", "changed to", "now uses", "replaced by",
"superseded", "no longer", "instead of", "switched to", "migrated to",
];
fn appears_contradictory(a: &str, b: &str) -> bool {
let a_lower = a.to_lowercase();
let b_lower = b.to_lowercase();
let a_words: std::collections::HashSet<&str> = a_lower.split_whitespace().filter(|w| w.len() > 3).collect();
let b_words: std::collections::HashSet<&str> = b_lower.split_whitespace().filter(|w| w.len() > 3).collect();
let shared_words = a_words.intersection(&b_words).count();
if shared_words < 2 { return false; }
for (neg, _) in NEGATION_PAIRS {
if (a_lower.contains(neg) && !b_lower.contains(neg))
|| (b_lower.contains(neg) && !a_lower.contains(neg))
{ return true; }
}
for signal in CORRECTION_SIGNALS {
if a_lower.contains(signal) || b_lower.contains(signal) { return true; }
}
false
}
fn topic_overlap(a: &str, b: &str) -> f32 {
let a_lower = a.to_lowercase();
let b_lower = b.to_lowercase();
let a_words: std::collections::HashSet<&str> = a_lower.split_whitespace().filter(|w| w.len() > 3).collect();
let b_words: std::collections::HashSet<&str> = b_lower.split_whitespace().filter(|w| w.len() > 3).collect();
if a_words.is_empty() || b_words.is_empty() { return 0.0; }
let intersection = a_words.intersection(&b_words).count();
let union = a_words.union(&b_words).count();
if union == 0 { 0.0 } else { intersection as f32 / union as f32 }
}
// ============================================================================
// Scored Memory (used across pipeline stages)
// ============================================================================
#[allow(dead_code)]
struct ScoredMemory {
id: String,
content: String,
tags: Vec<String>,
trust: f64,
updated_at: chrono::DateTime<Utc>,
created_at: chrono::DateTime<Utc>,
retention: f64,
combined_score: f32,
}
// ============================================================================
// Main Execute — 8-Stage Pipeline
// ============================================================================
pub async fn execute(
storage: &Arc<Storage>,
cognitive: &Arc<Mutex<CognitiveEngine>>,
args: Option<Value>,
) -> Result<Value, String> {
let args: DeepRefArgs = match args {
Some(v) => serde_json::from_value(v).map_err(|e| format!("Invalid arguments: {}", e))?,
None => return Err("Missing arguments".to_string()),
};
if args.query.trim().is_empty() {
return Err("Query cannot be empty".to_string());
}
let depth = args.depth.unwrap_or(20).clamp(5, 50) as usize;
// ====================================================================
// STAGE 0: Intent Classification (MAGMA-inspired query routing)
// ====================================================================
let intent = classify_intent(&args.query);
// ====================================================================
// STAGE 1: Broad Retrieval + Reranking
// ====================================================================
let results = storage
.hybrid_search(&args.query, depth as i32, 0.3, 0.7)
.map_err(|e| e.to_string())?;
if results.is_empty() {
return Ok(serde_json::json!({
"query": args.query,
"status": "no_memories",
"confidence": 0.0,
"guidance": "No memories found. Use smart_ingest to add memories.",
"memoriesAnalyzed": 0,
}));
}
let mut ranked = results;
if let Ok(mut cog) = cognitive.try_lock() {
let candidates: Vec<_> = ranked.iter().map(|r| (r.clone(), r.node.content.clone())).collect();
if let Ok(reranked) = cog.reranker.rerank(&args.query, candidates, Some(depth)) {
ranked = reranked.into_iter().map(|rr| rr.item).collect();
}
}
// ====================================================================
// STAGE 2: Spreading Activation Expansion
// ====================================================================
let mut activation_expanded = 0usize;
let existing_ids: std::collections::HashSet<String> = ranked.iter().map(|r| r.node.id.clone()).collect();
if let Ok(mut cog) = cognitive.try_lock() {
let mut expanded_ids = Vec::new();
for r in ranked.iter().take(3) {
let activated = cog.activation_network.activate(&r.node.id, 1.0);
for a in activated.iter().take(3) {
if !existing_ids.contains(&a.memory_id) && !expanded_ids.contains(&a.memory_id) {
expanded_ids.push(a.memory_id.clone());
}
}
}
// Fetch expanded memories from storage
for id in &expanded_ids {
if let Ok(Some(node)) = storage.get_node(id) {
// Create a minimal SearchResult-like entry
ranked.push(vestige_core::SearchResult {
node,
combined_score: 0.3, // lower score since these are expanded, not direct matches
keyword_score: None,
semantic_score: None,
match_type: vestige_core::MatchType::Semantic,
});
activation_expanded += 1;
}
}
}
// ====================================================================
// STAGE 3: FSRS-6 Trust Scoring
// ====================================================================
let scored: Vec<ScoredMemory> = ranked.iter().map(|r| {
let trust = compute_trust(
r.node.retention_strength,
r.node.stability,
r.node.reps,
r.node.lapses,
);
ScoredMemory {
id: r.node.id.clone(),
content: r.node.content.clone(),
tags: r.node.tags.clone(),
trust,
updated_at: r.node.updated_at,
created_at: r.node.created_at,
retention: r.node.retention_strength,
combined_score: r.combined_score,
}
}).collect();
// ====================================================================
// STAGE 4: Temporal Supersession
// ====================================================================
let mut superseded: Vec<Value> = Vec::new();
let mut superseded_ids: std::collections::HashSet<String> = std::collections::HashSet::new();
// Sort by date descending for supersession
let mut by_date = scored.iter().collect::<Vec<_>>();
by_date.sort_by(|a, b| b.updated_at.cmp(&a.updated_at));
for i in 0..by_date.len() {
for j in (i + 1)..by_date.len() {
let newer = by_date[i];
let older = by_date[j];
let overlap = topic_overlap(&newer.content, &older.content);
if overlap > 0.3 && newer.trust > older.trust && !superseded_ids.contains(&older.id) {
superseded_ids.insert(older.id.clone());
superseded.push(serde_json::json!({
"id": older.id,
"preview": older.content.chars().take(150).collect::<String>(),
"trust": (older.trust * 100.0).round() / 100.0,
"date": older.updated_at.to_rfc3339(),
"superseded_by": newer.id,
}));
}
}
}
// ====================================================================
// STAGE 5: Trust-Weighted Contradiction Analysis
// ====================================================================
let mut contradictions: Vec<Value> = Vec::new();
for i in 0..scored.len() {
for j in (i + 1)..scored.len() {
let a = &scored[i];
let b = &scored[j];
let overlap = topic_overlap(&a.content, &b.content);
if overlap < 0.15 { continue; }
let is_contradiction = appears_contradictory(&a.content, &b.content);
if !is_contradiction { continue; }
// Only flag as real contradiction if BOTH have decent trust
let min_trust = a.trust.min(b.trust);
if min_trust < 0.3 { continue; } // Low-trust memory isn't worth flagging
let (stronger, weaker) = if a.trust >= b.trust { (a, b) } else { (b, a) };
contradictions.push(serde_json::json!({
"stronger": {
"id": stronger.id,
"preview": stronger.content.chars().take(150).collect::<String>(),
"trust": (stronger.trust * 100.0).round() / 100.0,
"date": stronger.updated_at.to_rfc3339(),
},
"weaker": {
"id": weaker.id,
"preview": weaker.content.chars().take(150).collect::<String>(),
"trust": (weaker.trust * 100.0).round() / 100.0,
"date": weaker.updated_at.to_rfc3339(),
},
"topic_overlap": overlap,
}));
}
}
// ====================================================================
// STAGE 6: Dream Insight Integration
// ====================================================================
let mut related_insights: Vec<Value> = Vec::new();
if let Ok(insights) = storage.get_insights(20) {
let memory_ids: std::collections::HashSet<&str> = scored.iter().map(|s| s.id.as_str()).collect();
for insight in insights {
let overlaps = insight.source_memories.iter()
.any(|src_id| memory_ids.contains(src_id.as_str()));
if overlaps {
related_insights.push(serde_json::json!({
"insight": insight.insight,
"type": insight.insight_type,
"confidence": insight.confidence,
"source_memories": insight.source_memories,
}));
}
}
}
// ====================================================================
// STAGE 7: Relation Assessment (per-pair, using trust + temporal + similarity)
// ====================================================================
let mut pair_relations: Vec<(String, f64, RelationAssessment)> = Vec::new();
if let Some(primary) = scored.iter()
.filter(|s| !superseded_ids.contains(&s.id))
.max_by(|a, b| a.trust.partial_cmp(&b.trust).unwrap_or(std::cmp::Ordering::Equal))
{
for other in scored.iter().filter(|s| s.id != primary.id).take(15) {
let sim = topic_overlap(&primary.content, &other.content);
let rel = assess_relation(
&primary.content, &other.content,
primary.trust, other.trust,
primary.updated_at, other.updated_at,
sim,
);
if !matches!(rel.relation, Relation::Irrelevant) {
pair_relations.push((
other.content.chars().take(100).collect(),
other.trust,
rel,
));
}
}
}
// ====================================================================
// STAGE 8: Synthesis + Reasoning Chain Generation
// ====================================================================
// Find the recommended answer: highest trust, not superseded, most recent
let recommended = scored.iter()
.filter(|s| !superseded_ids.contains(&s.id))
.max_by(|a, b| {
// Primary: trust. Secondary: date.
a.trust.partial_cmp(&b.trust)
.unwrap_or(std::cmp::Ordering::Equal)
.then_with(|| a.updated_at.cmp(&b.updated_at))
});
// Build evidence list (top memories by trust, not superseded)
let mut non_superseded: Vec<&ScoredMemory> = scored.iter()
.filter(|s| !superseded_ids.contains(&s.id))
.collect();
non_superseded.sort_by(|a, b| b.trust.partial_cmp(&a.trust).unwrap_or(std::cmp::Ordering::Equal));
let evidence: Vec<Value> = non_superseded.iter()
.take(10)
.enumerate()
.map(|(i, s)| serde_json::json!({
"id": s.id,
"preview": s.content.chars().take(200).collect::<String>(),
"trust": (s.trust * 100.0).round() / 100.0,
"date": s.updated_at.to_rfc3339(),
"role": if i == 0 { "primary" } else { "supporting" },
}))
.collect();
// Build evolution timeline
let mut evolution: Vec<Value> = by_date.iter().rev()
.map(|s| serde_json::json!({
"date": s.updated_at.format("%b %d, %Y").to_string(),
"preview": s.content.chars().take(100).collect::<String>(),
"trust": (s.trust * 100.0).round() / 100.0,
}))
.collect();
evolution.truncate(15); // cap timeline length
// Confidence scoring
let base_confidence = recommended.map(|r| r.trust).unwrap_or(0.0);
let agreement_boost = (evidence.len() as f64 * 0.03).min(0.2);
let contradiction_penalty = contradictions.len() as f64 * 0.1;
let confidence = (base_confidence + agreement_boost - contradiction_penalty).clamp(0.0, 1.0);
let status = if contradictions.is_empty() && confidence > 0.7 {
"resolved"
} else if !contradictions.is_empty() {
"contradictions_found"
} else if scored.is_empty() {
"no_evidence"
} else {
"partial_evidence"
};
let guidance = if let Some(rec) = recommended {
if contradictions.is_empty() {
format!(
"High confidence ({:.0}%). Recommended memory (trust {:.0}%, {}) is the most reliable source.",
confidence * 100.0, rec.trust * 100.0, rec.updated_at.format("%b %d, %Y")
)
} else {
format!(
"WARNING: {} contradiction(s) detected. Recommended memory has trust {:.0}% but conflicts exist. Review contradictions below.",
contradictions.len(), rec.trust * 100.0
)
}
} else {
"No strong evidence found. Verify with external sources.".to_string()
};
// Auto-strengthen accessed memories (Testing Effect)
let ids: Vec<&str> = scored.iter().map(|s| s.id.as_str()).collect();
let _ = storage.strengthen_batch_on_access(&ids);
// Generate reasoning chain (the key differentiator — no LLM needed)
let reasoning_chain = if let Some(rec) = recommended {
generate_reasoning_chain(&args.query, &intent, rec, &pair_relations, confidence)
} else {
"No strong evidence found for reasoning.".to_string()
};
// Build response
let mut response = serde_json::json!({
"query": args.query,
"intent": format!("{:?}", intent),
"status": status,
"confidence": (confidence * 100.0).round() / 100.0,
"reasoning": reasoning_chain,
"guidance": guidance,
"memoriesAnalyzed": scored.len(),
"activationExpanded": activation_expanded,
});
if let Some(rec) = recommended {
response["recommended"] = serde_json::json!({
"answer_preview": rec.content.chars().take(300).collect::<String>(),
"memory_id": rec.id,
"trust_score": (rec.trust * 100.0).round() / 100.0,
"date": rec.updated_at.to_rfc3339(),
});
}
if !evidence.is_empty() { response["evidence"] = serde_json::json!(evidence); }
if !contradictions.is_empty() { response["contradictions"] = serde_json::json!(contradictions); }
if !superseded.is_empty() { response["superseded"] = serde_json::json!(superseded); }
if !evolution.is_empty() { response["evolution"] = serde_json::json!(evolution); }
if !related_insights.is_empty() { response["related_insights"] = serde_json::json!(related_insights); }
Ok(response)
}
// ============================================================================
// TESTS
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_schema_structure() {
let s = schema();
assert!(s["properties"]["query"].is_object());
assert!(s["properties"]["depth"].is_object());
assert_eq!(s["required"], serde_json::json!(["query"]));
}
#[test]
fn test_trust_score_high() {
// High retention, high stability, many reps, no lapses → high trust
let trust = compute_trust(0.95, 60.0, 20, 0);
assert!(trust > 0.8, "Expected >0.8, got {}", trust);
}
#[test]
fn test_trust_score_low() {
// Low retention, low stability, few reps, many lapses → low trust
let trust = compute_trust(0.2, 1.0, 1, 10);
assert!(trust < 0.3, "Expected <0.3, got {}", trust);
}
#[test]
fn test_trust_score_medium() {
// Medium everything
let trust = compute_trust(0.6, 15.0, 5, 2);
assert!(trust > 0.4 && trust < 0.7, "Expected 0.4-0.7, got {}", trust);
}
#[test]
fn test_trust_score_clamped() {
// Even extreme values stay in [0, 1]
assert!(compute_trust(1.0, 1000.0, 100, 0) <= 1.0);
assert!(compute_trust(0.0, 0.0, 0, 100) >= 0.0);
}
#[test]
fn test_contradiction_requires_shared_words() {
assert!(!appears_contradictory("not sure about weather", "Rust is fast"));
}
#[test]
fn test_contradiction_with_shared_context() {
assert!(appears_contradictory(
"Don't use FAISS for vector search in production",
"Use FAISS for vector search in production always"
));
}
#[test]
fn test_topic_overlap_similar() {
let overlap = topic_overlap("Vestige uses USearch for vector search", "Vestige vector search powered by USearch HNSW");
assert!(overlap > 0.3);
}
#[test]
fn test_topic_overlap_different() {
let overlap = topic_overlap("The weather is sunny today", "Rust compile times improving");
assert!(overlap < 0.15);
}
#[test]
fn test_depth_clamped() {
let s = schema();
assert_eq!(s["properties"]["depth"]["minimum"], 5);
assert_eq!(s["properties"]["depth"]["maximum"], 50);
}
// === Intent Classification Tests ===
#[test]
fn test_intent_fact_check() {
assert_eq!(classify_intent("Is it true that Vestige uses USearch?"), QueryIntent::FactCheck);
assert_eq!(classify_intent("Did I switch to port 3002?"), QueryIntent::FactCheck);
assert_eq!(classify_intent("Should I use prefix caching?"), QueryIntent::FactCheck);
}
#[test]
fn test_intent_timeline() {
assert_eq!(classify_intent("When did the port change happen?"), QueryIntent::Timeline);
assert_eq!(classify_intent("How has the AIMO3 score evolved over time?"), QueryIntent::Timeline);
}
#[test]
fn test_intent_root_cause() {
assert_eq!(classify_intent("Why did the build fail?"), QueryIntent::RootCause);
assert_eq!(classify_intent("What caused the score regression?"), QueryIntent::RootCause);
}
#[test]
fn test_intent_comparison() {
assert_eq!(classify_intent("How does USearch differ from FAISS?"), QueryIntent::Comparison);
assert_eq!(classify_intent("Compare FSRS versus SM-2"), QueryIntent::Comparison);
}
#[test]
fn test_intent_synthesis_default() {
assert_eq!(classify_intent("Tell me about Sam's projects"), QueryIntent::Synthesis);
assert_eq!(classify_intent("What is Vestige?"), QueryIntent::Synthesis);
}
// === Relation Assessment Tests ===
#[test]
fn test_relation_irrelevant() {
let rel = assess_relation("Rust is fast", "The weather is nice", 0.8, 0.8,
Utc::now(), Utc::now(), 0.05);
assert!(matches!(rel.relation, Relation::Irrelevant));
}
#[test]
fn test_relation_supports() {
let rel = assess_relation(
"Vestige uses USearch for vector search",
"USearch provides fast HNSW indexing for Vestige",
0.8, 0.7, Utc::now(), Utc::now(), 0.6);
assert!(matches!(rel.relation, Relation::Supports));
}
#[test]
fn test_relation_contradicts() {
let rel = assess_relation(
"Don't use FAISS for vector search in production anymore",
"Use FAISS for vector search in production always",
0.8, 0.5, Utc::now(), Utc::now(), 0.7);
assert!(matches!(rel.relation, Relation::Contradicts));
}
}

1
crates/vestige-mcp/src/tools/dedup.rs
View file

@ -45,6 +45,7 @@ pub fn schema() -> Value {
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")] #[serde(rename_all = "camelCase")]
struct DedupArgs { struct DedupArgs {
#[serde(alias = "similarity_threshold")]
similarity_threshold: Option<f64>, similarity_threshold: Option<f64>,
limit: Option<usize>, limit: Option<usize>,
tags: Option<Vec<String>>, tags: Option<Vec<String>>,

80
crates/vestige-mcp/src/tools/dream.rs
View file

@ -6,7 +6,7 @@ use tokio::sync::Mutex;
use chrono::Utc; use chrono::Utc;
use crate::cognitive::CognitiveEngine; use crate::cognitive::CognitiveEngine;
use vestige_core::{DreamHistoryRecord, LinkType, Storage}; use vestige_core::{DreamHistoryRecord, InsightRecord, LinkType, Storage};
pub fn schema() -> serde_json::Value { pub fn schema() -> serde_json::Value {
serde_json::json!({ serde_json::json!({
@ -30,7 +30,8 @@ pub async fn execute(
.as_ref() .as_ref()
.and_then(|a| a.get("memory_count")) .and_then(|a| a.get("memory_count"))
.and_then(|v| v.as_u64()) .and_then(|v| v.as_u64())
.unwrap_or(50) as usize; .unwrap_or(50)
.min(500) as usize; // Cap at 500 to prevent O(N^2) hang
// v1.9.0: Waking SWR tagging — preferential replay of tagged memories (70/30 split) // v1.9.0: Waking SWR tagging — preferential replay of tagged memories (70/30 split)
let tagged_nodes = storage.get_waking_tagged_memories(memory_count as i32) let tagged_nodes = storage.get_waking_tagged_memories(memory_count as i32)
@ -94,8 +95,28 @@ pub async fn execute(
let all_connections = cog.dreamer.get_connections(); let all_connections = cog.dreamer.get_connections();
drop(cog); drop(cog);
// v2.1.0: Persist dream insights to database (Bug #4 fix)
let mut insights_persisted = 0u64;
for insight in &insights {
let record = InsightRecord {
id: insight.id.clone(),
insight: insight.insight.clone(),
source_memories: insight.source_memories.clone(),
confidence: insight.confidence,
novelty_score: insight.novelty_score,
insight_type: format!("{:?}", insight.insight_type),
generated_at: insight.generated_at,
tags: insight.tags.clone(),
feedback: None,
applied_count: 0,
};
if storage.save_insight(&record).is_ok() {
insights_persisted += 1;
}
}
// v1.9.0: Persist only NEW connections from this dream (skip accumulated ones) // v1.9.0: Persist only NEW connections from this dream (skip accumulated ones)
let new_connections = &all_connections[pre_dream_count..]; let new_connections = all_connections.get(pre_dream_count..).unwrap_or(&[]);
let mut connections_persisted = 0u64; let mut connections_persisted = 0u64;
{ {
let now = Utc::now(); let now = Utc::now();
@ -197,9 +218,11 @@ pub async fn execute(
"novelty_score": i.novelty_score, "novelty_score": i.novelty_score,
})).collect::<Vec<_>>(), })).collect::<Vec<_>>(),
"connectionsPersisted": connections_persisted, "connectionsPersisted": connections_persisted,
"insightsPersisted": insights_persisted,
"stats": { "stats": {
"new_connections_found": dream_result.new_connections_found, "new_connections_found": dream_result.new_connections_found,
"connections_persisted": connections_persisted, "connections_persisted": connections_persisted,
"insights_persisted": insights_persisted,
"memories_strengthened": dream_result.memories_strengthened, "memories_strengthened": dream_result.memories_strengthened,
"memories_compressed": dream_result.memories_compressed, "memories_compressed": dream_result.memories_compressed,
"insights_generated": dream_result.insights_generated.len(), "insights_generated": dream_result.insights_generated.len(),
@ -532,4 +555,55 @@ mod tests {
persisted persisted
); );
} }
#[tokio::test]
async fn test_dream_persists_insights() {
let (storage, _dir) = test_storage().await;
// Create diverse tagged memories to encourage insight generation
let topics = [
("Rust borrow checker prevents data races", vec!["rust", "safety"]),
("Rust ownership model ensures memory safety", vec!["rust", "safety"]),
("Cargo manages Rust project dependencies", vec!["rust", "cargo"]),
("Cargo.toml defines project configuration", vec!["rust", "cargo"]),
("Unit tests use the #[test] attribute", vec!["rust", "testing"]),
("Integration tests live in the tests directory", vec!["rust", "testing"]),
("Clippy catches common Rust mistakes", vec!["rust", "tooling"]),
("Rustfmt automatically formats code", vec!["rust", "tooling"]),
];
for (content, tags) in &topics {
storage.ingest(vestige_core::IngestInput {
content: content.to_string(),
node_type: "fact".to_string(),
source: None, sentiment_score: 0.0, sentiment_magnitude: 0.0,
tags: tags.iter().map(|t| t.to_string()).collect(),
valid_from: None, valid_until: None,
}).unwrap();
}
let result = execute(&storage, &test_cognitive(), None).await.unwrap();
assert_eq!(result["status"], "dreamed");
let response_insights = result["insights"].as_array().unwrap();
let persisted_count = result["insightsPersisted"].as_u64().unwrap_or(0);
// If insights were generated, they should be persisted
if !response_insights.is_empty() {
assert!(persisted_count > 0, "Generated insights should be persisted to database");
let stored = storage.get_insights(100).unwrap();
assert_eq!(
stored.len(), persisted_count as usize,
"All {} persisted insights should be retrievable", persisted_count
);
// Verify insight fields
for insight in &stored {
assert!(!insight.id.is_empty(), "Insight ID should not be empty");
assert!(!insight.insight.is_empty(), "Insight text should not be empty");
assert!(insight.confidence >= 0.0 && insight.confidence <= 1.0);
assert!(insight.novelty_score >= 0.0);
assert!(insight.feedback.is_none(), "Fresh insight should have no feedback");
assert_eq!(insight.applied_count, 0);
}
}
}
} }

184
crates/vestige-mcp/src/tools/explore.rs
View file

@ -5,6 +5,7 @@ use std::sync::Arc;
use tokio::sync::Mutex; use tokio::sync::Mutex;
use crate::cognitive::CognitiveEngine; use crate::cognitive::CognitiveEngine;
use vestige_core::advanced::{Connection, ConnectionType, MemoryChainBuilder, MemoryNode};
use vestige_core::Storage; use vestige_core::Storage;
pub fn schema() -> serde_json::Value { pub fn schema() -> serde_json::Value {
@ -50,12 +51,24 @@ pub async fn execute(
match action { match action {
"chain" => { "chain" => {
let to_id = to.ok_or("'to' is required for chain action")?; let to_id = to.ok_or("'to' is required for chain action")?;
match cog.chain_builder.build_chain(from, to_id) { let chain_result = cog.chain_builder.build_chain(from, to_id);
let from_owned = from.to_string();
let to_owned = to_id.to_string();
drop(cog); // release lock before potential storage fallback
let chain_opt = if chain_result.is_some() {
chain_result
} else {
// Storage fallback: build temporary chain from persisted connections
build_chain_from_storage(storage, &from_owned, &to_owned)
};
match chain_opt {
Some(chain) => { Some(chain) => {
Ok(serde_json::json!({ Ok(serde_json::json!({
"action": "chain", "action": "chain",
"from": from, "from": from_owned,
"to": to_id, "to": to_owned,
"steps": chain.steps.iter().map(|s| serde_json::json!({ "steps": chain.steps.iter().map(|s| serde_json::json!({
"memory_id": s.memory_id, "memory_id": s.memory_id,
"memory_preview": s.memory_preview, "memory_preview": s.memory_preview,
@ -70,8 +83,8 @@ pub async fn execute(
None => { None => {
Ok(serde_json::json!({ Ok(serde_json::json!({
"action": "chain", "action": "chain",
"from": from, "from": from_owned,
"to": to_id, "to": to_owned,
"steps": [], "steps": [],
"message": "No chain found between these memories" "message": "No chain found between these memories"
})) }))
@ -82,6 +95,8 @@ pub async fn execute(
let activation_assocs = cog.activation_network.get_associations(from); let activation_assocs = cog.activation_network.get_associations(from);
let hippocampal_assocs = cog.hippocampal_index.get_associations(from, 2) let hippocampal_assocs = cog.hippocampal_index.get_associations(from, 2)
.unwrap_or_default(); .unwrap_or_default();
let from_owned = from.to_string();
drop(cog); // release lock consistently (matches chain/bridges pattern)
let mut all_associations: Vec<serde_json::Value> = Vec::new(); let mut all_associations: Vec<serde_json::Value> = Vec::new();
@ -106,10 +121,9 @@ pub async fn execute(
// Fallback: if in-memory modules are empty, query storage directly // Fallback: if in-memory modules are empty, query storage directly
if all_associations.is_empty() { if all_associations.is_empty() {
drop(cog); // release cognitive lock before storage call if let Ok(connections) = storage.get_connections_for_memory(&from_owned) {
if let Ok(connections) = storage.get_connections_for_memory(from) {
for conn in connections.iter().take(limit) { for conn in connections.iter().take(limit) {
let other_id = if conn.source_id == from { let other_id = if conn.source_id == from_owned {
&conn.target_id &conn.target_id
} else { } else {
&conn.source_id &conn.source_id
@ -126,7 +140,7 @@ pub async fn execute(
Ok(serde_json::json!({ Ok(serde_json::json!({
"action": "associations", "action": "associations",
"from": from, "from": from_owned,
"associations": all_associations, "associations": all_associations,
"count": all_associations.len(), "count": all_associations.len(),
})) }))
@ -134,11 +148,23 @@ pub async fn execute(
"bridges" => { "bridges" => {
let to_id = to.ok_or("'to' is required for bridges action")?; let to_id = to.ok_or("'to' is required for bridges action")?;
let bridges = cog.chain_builder.find_bridge_memories(from, to_id); let bridges = cog.chain_builder.find_bridge_memories(from, to_id);
let limited: Vec<_> = bridges.iter().take(limit).collect(); let from_owned = from.to_string();
let to_owned = to_id.to_string();
drop(cog); // release lock before potential storage fallback
let final_bridges = if !bridges.is_empty() {
bridges
} else {
// Storage fallback: build temporary graph and find bridges
let temp_builder = build_temp_chain_builder(storage, &from_owned, &to_owned);
temp_builder.find_bridge_memories(&from_owned, &to_owned)
};
let limited: Vec<_> = final_bridges.iter().take(limit).collect();
Ok(serde_json::json!({ Ok(serde_json::json!({
"action": "bridges", "action": "bridges",
"from": from, "from": from_owned,
"to": to_id, "to": to_owned,
"bridges": limited, "bridges": limited,
"count": limited.len(), "count": limited.len(),
})) }))
@ -147,6 +173,73 @@ pub async fn execute(
} }
} }
/// Build a temporary MemoryChainBuilder from persisted connections for fallback queries.
fn build_temp_chain_builder(storage: &Arc<Storage>, from_id: &str, to_id: &str) -> MemoryChainBuilder {
let mut builder = MemoryChainBuilder::new();
// Load connections involving either endpoint
let mut all_conns = Vec::new();
if let Ok(conns) = storage.get_connections_for_memory(from_id) {
all_conns.extend(conns);
}
if let Ok(conns) = storage.get_connections_for_memory(to_id) {
all_conns.extend(conns);
}
// Deduplicate edges and load referenced memory nodes
let mut seen_edges = std::collections::HashSet::new();
all_conns.retain(|c| seen_edges.insert((c.source_id.clone(), c.target_id.clone())));
let mut seen_ids = std::collections::HashSet::new();
for conn in &all_conns {
for id in [&conn.source_id, &conn.target_id] {
if seen_ids.insert(id.clone()) {
if let Ok(Some(node)) = storage.get_node(id) {
builder.add_memory(MemoryNode {
id: node.id.clone(),
content_preview: node.content.chars().take(100).collect(),
tags: node.tags.clone(),
connections: vec![],
});
}
}
}
}
// Add edges
for conn in &all_conns {
builder.add_connection(Connection {
from_id: conn.source_id.clone(),
to_id: conn.target_id.clone(),
connection_type: link_type_to_connection_type(&conn.link_type),
strength: conn.strength,
created_at: conn.created_at,
});
}
builder
}
/// Build a chain from storage when in-memory chain_builder is empty.
fn build_chain_from_storage(
storage: &Arc<Storage>,
from_id: &str,
to_id: &str,
) -> Option<vestige_core::advanced::ReasoningChain> {
let builder = build_temp_chain_builder(storage, from_id, to_id);
builder.build_chain(from_id, to_id)
}
/// Convert storage link_type string to ConnectionType enum.
fn link_type_to_connection_type(link_type: &str) -> ConnectionType {
match link_type {
"temporal" => ConnectionType::TemporalProximity,
"causal" => ConnectionType::Causal,
"part_of" => ConnectionType::PartOf,
_ => ConnectionType::SemanticSimilarity,
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@ -351,4 +444,71 @@ mod tests {
assert_eq!(associations[0]["source"], "persistent_graph"); assert_eq!(associations[0]["source"], "persistent_graph");
assert_eq!(associations[0]["memory_id"], id2); assert_eq!(associations[0]["memory_id"], id2);
} }
#[tokio::test]
async fn test_chain_storage_fallback() {
let (storage, _dir) = test_storage().await;
// Create 3 memories: A -> B -> C
let make = |content: &str| vestige_core::IngestInput {
content: content.to_string(), node_type: "fact".to_string(),
source: None, sentiment_score: 0.0, sentiment_magnitude: 0.0,
tags: vec!["test".to_string()], valid_from: None, valid_until: None,
};
let id_a = storage.ingest(make("Memory A about databases")).unwrap().id;
let id_b = storage.ingest(make("Memory B about indexes")).unwrap().id;
let id_c = storage.ingest(make("Memory C about performance")).unwrap().id;
// Save connections A->B and B->C to storage
let now = chrono::Utc::now();
for (src, tgt) in [(&id_a, &id_b), (&id_b, &id_c)] {
storage.save_connection(&vestige_core::ConnectionRecord {
source_id: src.clone(), target_id: tgt.clone(),
strength: 0.9, link_type: "semantic".to_string(),
created_at: now, last_activated: now, activation_count: 1,
}).unwrap();
}
// Execute chain with empty cognitive engine — should fall back to storage
let args = serde_json::json!({ "action": "chain", "from": id_a, "to": id_c });
let result = execute(&storage, &test_cognitive(), Some(args)).await;
assert!(result.is_ok());
let value = result.unwrap();
assert_eq!(value["action"], "chain");
let steps = value["steps"].as_array().unwrap();
assert!(!steps.is_empty(), "Chain should find path A->B->C via storage fallback");
}
#[tokio::test]
async fn test_bridges_storage_fallback() {
let (storage, _dir) = test_storage().await;
// Create 3 memories: A -> B -> C (B is the bridge)
let make = |content: &str| vestige_core::IngestInput {
content: content.to_string(), node_type: "fact".to_string(),
source: None, sentiment_score: 0.0, sentiment_magnitude: 0.0,
tags: vec!["test".to_string()], valid_from: None, valid_until: None,
};
let id_a = storage.ingest(make("Bridge test memory A")).unwrap().id;
let id_b = storage.ingest(make("Bridge test memory B")).unwrap().id;
let id_c = storage.ingest(make("Bridge test memory C")).unwrap().id;
let now = chrono::Utc::now();
for (src, tgt) in [(&id_a, &id_b), (&id_b, &id_c)] {
storage.save_connection(&vestige_core::ConnectionRecord {
source_id: src.clone(), target_id: tgt.clone(),
strength: 0.9, link_type: "semantic".to_string(),
created_at: now, last_activated: now, activation_count: 1,
}).unwrap();
}
// Execute bridges with empty cognitive engine
let args = serde_json::json!({ "action": "bridges", "from": id_a, "to": id_c });
let result = execute(&storage, &test_cognitive(), Some(args)).await;
assert!(result.is_ok());
let value = result.unwrap();
assert_eq!(value["action"], "bridges");
let bridges = value["bridges"].as_array().unwrap();
assert!(!bridges.is_empty(), "Should find B as bridge between A and C via storage fallback");
}
} }

101
crates/vestige-mcp/src/tools/memory_unified.rs
View file

@ -43,12 +43,17 @@ pub fn schema() -> Value {
"properties": { "properties": {
"action": { "action": {
"type": "string", "type": "string",
"enum": ["get", "delete", "state", "promote", "demote", "edit"], "enum": ["get", "get_batch", "delete", "state", "promote", "demote", "edit"],
"description": "Action to perform: 'get' retrieves full memory node, 'delete' removes memory, 'state' returns accessibility state, 'promote' increases retrieval strength (thumbs up), 'demote' decreases retrieval strength (thumbs down), 'edit' updates content in-place (preserves FSRS state)" "description": "Action to perform: 'get' retrieves full memory node, 'get_batch' retrieves multiple memories by IDs (use 'ids' array), 'delete' removes memory, 'state' returns accessibility state, 'promote' increases retrieval strength (thumbs up), 'demote' decreases retrieval strength (thumbs down), 'edit' updates content in-place (preserves FSRS state)"
}, },
"id": { "id": {
"type": "string", "type": "string",
"description": "The ID of the memory node" "description": "The ID of the memory node (for single-memory actions)"
},
"ids": {
"type": "array",
"items": { "type": "string" },
"description": "Array of memory IDs (for get_batch action). Max 20 IDs per call."
}, },
"reason": { "reason": {
"type": "string", "type": "string",
@ -59,7 +64,7 @@ pub fn schema() -> Value {
"description": "New content for edit action. Replaces existing content, regenerates embedding, preserves FSRS state." "description": "New content for edit action. Replaces existing content, regenerates embedding, preserves FSRS state."
} }
}, },
"required": ["action", "id"] "required": ["action"]
}) })
} }
@ -67,7 +72,8 @@ pub fn schema() -> Value {
#[serde(rename_all = "camelCase")] #[serde(rename_all = "camelCase")]
struct MemoryArgs { struct MemoryArgs {
action: String, action: String,
id: String, id: Option<String>,
ids: Option<Vec<String>>,
reason: Option<String>, reason: Option<String>,
content: Option<String>, content: Option<String>,
} }
@ -83,18 +89,34 @@ pub async fn execute(
None => return Err("Missing arguments".to_string()), None => return Err("Missing arguments".to_string()),
}; };
// Validate UUID format // get_batch uses 'ids' array, all other actions use 'id'
uuid::Uuid::parse_str(&args.id).map_err(|_| "Invalid memory ID format".to_string())?; if args.action == "get_batch" {
let ids = args.ids.ok_or("get_batch requires 'ids' array")?;
if ids.is_empty() {
return Err("ids array cannot be empty".to_string());
}
if ids.len() > 20 {
return Err("get_batch supports max 20 IDs per call".to_string());
}
for id in &ids {
uuid::Uuid::parse_str(id).map_err(|_| format!("Invalid memory ID format: {}", id))?;
}
return execute_get_batch(storage, &ids).await;
}
// All other actions require 'id'
let id = args.id.ok_or("This action requires 'id' parameter")?;
uuid::Uuid::parse_str(&id).map_err(|_| "Invalid memory ID format".to_string())?;
match args.action.as_str() { match args.action.as_str() {
"get" => execute_get(storage, &args.id).await, "get" => execute_get(storage, &id).await,
"delete" => execute_delete(storage, &args.id).await, "delete" => execute_delete(storage, &id).await,
"state" => execute_state(storage, &args.id).await, "state" => execute_state(storage, &id).await,
"promote" => execute_promote(storage, cognitive, &args.id, args.reason).await, "promote" => execute_promote(storage, cognitive, &id, args.reason).await,
"demote" => execute_demote(storage, cognitive, &args.id, args.reason).await, "demote" => execute_demote(storage, cognitive, &id, args.reason).await,
"edit" => execute_edit(storage, &args.id, args.content).await, "edit" => execute_edit(storage, &id, args.content).await,
_ => Err(format!( _ => Err(format!(
"Invalid action '{}'. Must be one of: get, delete, state, promote, demote, edit", "Invalid action '{}'. Must be one of: get, get_batch, delete, state, promote, demote, edit",
args.action args.action
)), )),
} }
@ -140,6 +162,49 @@ async fn execute_get(storage: &Arc<Storage>, id: &str) -> Result<Value, String>
} }
} }
/// Get multiple full memory nodes by ID (batch retrieval for expandable IDs)
async fn execute_get_batch(storage: &Arc<Storage>, ids: &[String]) -> Result<Value, String> {
let mut results = Vec::with_capacity(ids.len());
let mut found_count = 0;
for id in ids {
match storage.get_node(id) {
Ok(Some(n)) => {
found_count += 1;
results.push(serde_json::json!({
"id": n.id,
"content": n.content,
"nodeType": n.node_type,
"createdAt": n.created_at.to_rfc3339(),
"updatedAt": n.updated_at.to_rfc3339(),
"tags": n.tags,
"retentionStrength": n.retention_strength,
"source": n.source,
}));
}
Ok(None) => {
results.push(serde_json::json!({
"id": id,
"found": false,
}));
}
Err(e) => {
results.push(serde_json::json!({
"id": id,
"error": e.to_string(),
}));
}
}
}
Ok(serde_json::json!({
"action": "get_batch",
"requested": ids.len(),
"found": found_count,
"results": results,
}))
}
/// Delete a memory and return success status /// Delete a memory and return success status
async fn execute_delete(storage: &Arc<Storage>, id: &str) -> Result<Value, String> { async fn execute_delete(storage: &Arc<Storage>, id: &str) -> Result<Value, String> {
let deleted = storage.delete_node(id).map_err(|e| e.to_string())?; let deleted = storage.delete_node(id).map_err(|e| e.to_string())?;
@ -388,10 +453,12 @@ mod tests {
assert!(schema["properties"]["action"].is_object()); assert!(schema["properties"]["action"].is_object());
assert!(schema["properties"]["id"].is_object()); assert!(schema["properties"]["id"].is_object());
assert!(schema["properties"]["reason"].is_object()); assert!(schema["properties"]["reason"].is_object());
assert_eq!(schema["required"], serde_json::json!(["action", "id"])); assert_eq!(schema["required"], serde_json::json!(["action"]));
// Verify all 6 actions are in enum assert!(schema["properties"]["ids"].is_object()); // get_batch support
// Verify all 7 actions are in enum
let actions = schema["properties"]["action"]["enum"].as_array().unwrap(); let actions = schema["properties"]["action"]["enum"].as_array().unwrap();
assert_eq!(actions.len(), 6); assert_eq!(actions.len(), 7);
assert!(actions.contains(&serde_json::json!("get_batch")));
assert!(actions.contains(&serde_json::json!("edit"))); assert!(actions.contains(&serde_json::json!("edit")));
assert!(actions.contains(&serde_json::json!("promote"))); assert!(actions.contains(&serde_json::json!("promote")));
assert!(actions.contains(&serde_json::json!("demote"))); assert!(actions.contains(&serde_json::json!("demote")));

3
crates/vestige-mcp/src/tools/mod.rs
View file

@ -37,6 +37,9 @@ pub mod session_context;
pub mod health; pub mod health;
pub mod graph; pub mod graph;
// v2.1: Cross-reference (connect the dots)
pub mod cross_reference;
// Deprecated/internal tools — not advertised in the public MCP tools/list, // Deprecated/internal tools — not advertised in the public MCP tools/list,
// but some functions are actively dispatched for backwards compatibility // but some functions are actively dispatched for backwards compatibility
// and internal cognitive operations. #[allow(dead_code)] suppresses warnings // and internal cognitive operations. #[allow(dead_code)] suppresses warnings

98
crates/vestige-mcp/src/tools/search_unified.rs
View file

@ -68,9 +68,15 @@ pub fn schema() -> Value {
}, },
"token_budget": { "token_budget": {
"type": "integer", "type": "integer",
"description": "Max tokens for response. Server truncates content to fit budget. Use memory(action='get') for full content of specific IDs.", "description": "Max tokens for response. Server truncates content to fit budget. Use memory(action='get') for full content of specific IDs. With 1M context models, budgets up to 100K are practical.",
"minimum": 100, "minimum": 100,
"maximum": 10000 "maximum": 100000
},
"retrieval_mode": {
"type": "string",
"description": "precise: top results only (fast, token-efficient, skips activation/competition). balanced: full 7-stage cognitive pipeline (default). exhaustive: maximum recall with 5x overfetch, deep graph traversal, no competition suppression.",
"enum": ["precise", "balanced", "exhaustive"],
"default": "balanced"
} }
}, },
"required": ["query"] "required": ["query"]
@ -82,13 +88,18 @@ pub fn schema() -> Value {
struct SearchArgs { struct SearchArgs {
query: String, query: String,
limit: Option<i32>, limit: Option<i32>,
#[serde(alias = "min_retention")]
min_retention: Option<f64>, min_retention: Option<f64>,
#[serde(alias = "min_similarity")]
min_similarity: Option<f32>, min_similarity: Option<f32>,
#[serde(alias = "detail_level")] #[serde(alias = "detail_level")]
detail_level: Option<String>, detail_level: Option<String>,
#[serde(alias = "context_topics")]
context_topics: Option<Vec<String>>, context_topics: Option<Vec<String>>,
#[serde(alias = "token_budget")] #[serde(alias = "token_budget")]
token_budget: Option<i32>, token_budget: Option<i32>,
#[serde(alias = "retrieval_mode")]
retrieval_mode: Option<String>,
} }
/// Execute unified search with 7-stage cognitive pipeline. /// Execute unified search with 7-stage cognitive pipeline.
@ -135,14 +146,32 @@ pub async fn execute(
let min_retention = args.min_retention.unwrap_or(0.0).clamp(0.0, 1.0); let min_retention = args.min_retention.unwrap_or(0.0).clamp(0.0, 1.0);
let min_similarity = args.min_similarity.unwrap_or(0.5).clamp(0.0, 1.0); let min_similarity = args.min_similarity.unwrap_or(0.5).clamp(0.0, 1.0);
// Validate retrieval_mode
let retrieval_mode = match args.retrieval_mode.as_deref() {
Some("precise") => "precise",
Some("exhaustive") => "exhaustive",
Some("balanced") | None => "balanced",
Some(invalid) => {
return Err(format!(
"Invalid retrieval_mode '{}'. Must be 'precise', 'balanced', or 'exhaustive'.",
invalid
));
}
};
// Favor semantic search — research shows 0.3/0.7 outperforms equal weights // Favor semantic search — research shows 0.3/0.7 outperforms equal weights
let keyword_weight = 0.3_f32; let keyword_weight = 0.3_f32;
let semantic_weight = 0.7_f32; let semantic_weight = 0.7_f32;
// ==================================================================== // ====================================================================
// STAGE 1: Hybrid search with 3x over-fetch for reranking pool // STAGE 1: Hybrid search with Nx over-fetch for reranking pool
// ==================================================================== // ====================================================================
let overfetch_limit = (limit * 3).min(100); // Cap at 100 to avoid excessive DB load let overfetch_multiplier = match retrieval_mode {
"precise" => 1, // No overfetch — return exactly what's asked
"exhaustive" => 5, // Deep overfetch for maximum recall
_ => 3, // Balanced default
};
let overfetch_limit = (limit * overfetch_multiplier).min(100); // Cap at 100 to avoid excessive DB load
let results = storage let results = storage
.hybrid_search(&args.query, overfetch_limit, keyword_weight, semantic_weight) .hybrid_search(&args.query, overfetch_limit, keyword_weight, semantic_weight)
@ -215,7 +244,7 @@ pub async fn execute(
// Determine state from retention strength // Determine state from retention strength
lifecycle.state = if result.node.retention_strength > 0.7 { lifecycle.state = if result.node.retention_strength > 0.7 {
MemoryState::Active MemoryState::Active
} else if result.node.retention_strength > 0.3 { } else if result.node.retention_strength > 0.4 {
MemoryState::Dormant MemoryState::Dormant
} else if result.node.retention_strength > 0.1 { } else if result.node.retention_strength > 0.1 {
MemoryState::Silent MemoryState::Silent
@ -275,9 +304,11 @@ pub async fn execute(
// ==================================================================== // ====================================================================
// STAGE 5B: Retrieval competition (Anderson et al. 1994) // STAGE 5B: Retrieval competition (Anderson et al. 1994)
// Skipped in precise mode (no need) and exhaustive mode (want all results)
// ==================================================================== // ====================================================================
let mut suppressed_count = 0_usize; let mut suppressed_count = 0_usize;
if filtered_results.len() > 1 if retrieval_mode == "balanced"
&& filtered_results.len() > 1
&& let Ok(mut cog) = cognitive.try_lock() && let Ok(mut cog) = cognitive.try_lock()
{ {
let candidates: Vec<CompetitionCandidate> = filtered_results let candidates: Vec<CompetitionCandidate> = filtered_results
@ -321,21 +352,31 @@ pub async fn execute(
// ==================================================================== // ====================================================================
// STAGE 6: Spreading activation (find associated memories) // STAGE 6: Spreading activation (find associated memories)
// Skipped in precise mode. Deeper (5 results) in exhaustive mode.
// ==================================================================== // ====================================================================
let associations: Vec<Value> = if let Ok(mut cog) = cognitive.try_lock() { let activation_take = match retrieval_mode {
if let Some(first) = filtered_results.first() { "precise" => 0, // Skip entirely
let activated = cog.activation_network.activate(&first.node.id, 1.0); "exhaustive" => 5, // Deeper graph traversal
activated _ => 3, // Balanced default
.iter() };
.take(3) let associations: Vec<Value> = if activation_take > 0 {
.map(|a| { if let Ok(mut cog) = cognitive.try_lock() {
serde_json::json!({ if let Some(first) = filtered_results.first() {
"memoryId": a.memory_id, let activated = cog.activation_network.activate(&first.node.id, 1.0);
"activation": a.activation, activated
"distance": a.distance, .iter()
.take(activation_take)
.map(|a| {
serde_json::json!({
"memoryId": a.memory_id,
"activation": a.activation,
"distance": a.distance,
})
}) })
}) .collect()
.collect() } else {
vec![]
}
} else { } else {
vec![] vec![]
} }
@ -401,7 +442,7 @@ pub async fn execute(
let mut budget_expandable: Vec<String> = Vec::new(); let mut budget_expandable: Vec<String> = Vec::new();
let mut budget_tokens_used: Option<usize> = None; let mut budget_tokens_used: Option<usize> = None;
if let Some(budget) = args.token_budget { if let Some(budget) = args.token_budget {
let budget = budget.clamp(100, 10000) as usize; let budget = budget.clamp(100, 100000) as usize;
let budget_chars = budget * 4; let budget_chars = budget * 4;
let mut used = 0; let mut used = 0;
let mut budgeted = Vec::new(); let mut budgeted = Vec::new();
@ -428,11 +469,17 @@ pub async fn execute(
let mut response = serde_json::json!({ let mut response = serde_json::json!({
"query": args.query, "query": args.query,
"method": "hybrid+cognitive", "method": "hybrid+cognitive",
"retrievalMode": retrieval_mode,
"detailLevel": detail_level, "detailLevel": detail_level,
"total": formatted.len(), "total": formatted.len(),
"results": formatted, "results": formatted,
}); });
// Helpful hint when no results found
if formatted.is_empty() {
response["hint"] = serde_json::json!("No memories found. Use smart_ingest to add memories, or try a broader query.");
}
// Include associations if any were found // Include associations if any were found
if !associations.is_empty() { if !associations.is_empty() {
response["associations"] = serde_json::json!(associations); response["associations"] = serde_json::json!(associations);
@ -499,7 +546,7 @@ fn format_search_result(r: &vestige_core::SearchResult, detail_level: &str) -> V
"validUntil": r.node.valid_until.map(|dt| dt.to_rfc3339()), "validUntil": r.node.valid_until.map(|dt| dt.to_rfc3339()),
"matchType": format!("{:?}", r.match_type), "matchType": format!("{:?}", r.match_type),
}), }),
// "summary" (default) — backwards compatible // "summary" (default) — includes dates so AI never has to guess when a memory is from
_ => serde_json::json!({ _ => serde_json::json!({
"id": r.node.id, "id": r.node.id,
"content": r.node.content, "content": r.node.content,
@ -509,6 +556,8 @@ fn format_search_result(r: &vestige_core::SearchResult, detail_level: &str) -> V
"nodeType": r.node.node_type, "nodeType": r.node.node_type,
"tags": r.node.tags, "tags": r.node.tags,
"retentionStrength": r.node.retention_strength, "retentionStrength": r.node.retention_strength,
"createdAt": r.node.created_at.to_rfc3339(),
"updatedAt": r.node.updated_at.to_rfc3339(),
}), }),
} }
} }
@ -1004,10 +1053,11 @@ mod tests {
let results = value["results"].as_array().unwrap(); let results = value["results"].as_array().unwrap();
if !results.is_empty() { if !results.is_empty() {
let first = &results[0]; let first = &results[0];
// Summary should have content but not timestamps // Summary should have content AND timestamps (v2.1: dates always visible)
assert!(first["content"].is_string()); assert!(first["content"].is_string());
assert!(first["id"].is_string()); assert!(first["id"].is_string());
assert!(first.get("createdAt").is_none() || first["createdAt"].is_null()); assert!(first["createdAt"].is_string(), "summary must include createdAt");
assert!(first["updatedAt"].is_string(), "summary must include updatedAt");
} }
} }
@ -1106,6 +1156,6 @@ mod tests {
let tb = &schema_value["properties"]["token_budget"]; let tb = &schema_value["properties"]["token_budget"];
assert!(tb.is_object()); assert!(tb.is_object());
assert_eq!(tb["minimum"], 100); assert_eq!(tb["minimum"], 100);
assert_eq!(tb["maximum"], 10000); assert_eq!(tb["maximum"], 100000);
} }
} }

11
crates/vestige-mcp/src/tools/session_context.rs
View file

@ -27,10 +27,10 @@ pub fn schema() -> Value {
}, },
"token_budget": { "token_budget": {
"type": "integer", "type": "integer",
"description": "Max tokens for response (default: 1000). Server truncates content to fit budget.", "description": "Max tokens for response (default: 1000). Server truncates content to fit budget. With 1M context models, budgets up to 100K are practical.",
"default": 1000, "default": 1000,
"minimum": 100, "minimum": 100,
"maximum": 10000 "maximum": 100000
}, },
"context": { "context": {
"type": "object", "type": "object",
@ -105,7 +105,7 @@ pub async fn execute(
None => SessionContextArgs::default(), None => SessionContextArgs::default(),
}; };
let token_budget = args.token_budget.unwrap_or(1000).clamp(100, 10000) as usize; let token_budget = args.token_budget.unwrap_or(1000).clamp(100, 100000) as usize;
let budget_chars = token_budget * 4; let budget_chars = token_budget * 4;
let include_status = args.include_status.unwrap_or(true); let include_status = args.include_status.unwrap_or(true);
let include_intentions = args.include_intentions.unwrap_or(true); let include_intentions = args.include_intentions.unwrap_or(true);
@ -132,7 +132,8 @@ pub async fn execute(
continue; continue;
} }
let summary = first_sentence(&r.node.content); let summary = first_sentence(&r.node.content);
let line = format!("- {}", summary); let date_str = r.node.updated_at.format("%b %d, %Y").to_string();
let line = format!("- ({}) {}", date_str, summary);
let line_len = line.len() + 1; // +1 for newline let line_len = line.len() + 1; // +1 for newline
if char_count + line_len > budget_chars { if char_count + line_len > budget_chars {
@ -510,7 +511,7 @@ mod tests {
let s = schema(); let s = schema();
let tb = &s["properties"]["token_budget"]; let tb = &s["properties"]["token_budget"];
assert_eq!(tb["minimum"], 100); assert_eq!(tb["minimum"], 100);
assert_eq!(tb["maximum"], 10000); assert_eq!(tb["maximum"], 100000);
assert_eq!(tb["default"], 1000); assert_eq!(tb["default"], 1000);
} }

2
docs/integrations/windsurf.md
View file

@ -115,7 +115,7 @@ It remembers.
## Important: Tool Limit ## Important: Tool Limit
Windsurf has a **hard cap of 100 tools** across all MCP servers. Vestige uses 19 tools, leaving plenty of room for other servers. Windsurf has a **hard cap of 100 tools** across all MCP servers. Vestige uses 23 tools, leaving plenty of room for other servers.
--- ---

2
docs/integrations/xcode.md
View file

@ -51,7 +51,7 @@ Quit Xcode completely (Cmd+Q) and reopen your project.
### 4. Verify ### 4. Verify
Type `/context` in the Agent panel. You should see `vestige` listed with 19 tools. Type `/context` in the Agent panel. You should see `vestige` listed with 23 tools.
--- ---

233
docs/launch/reddit-cross-reference.md Normal file
View file

@ -0,0 +1,233 @@
# Reddit Launch Posts — cross_reference Tool
## Post 1: r/ClaudeAI (Primary)
**Title:** `I built a tool that catches when Claude's memories contradict each other before it answers. It's been wrong 0 times since.`
---
I've been building Vestige — an MCP memory server that gives Claude persistent memory across sessions. FSRS-6 spaced repetition (the Anki algorithm), 29 neuroscience-inspired modules, single Rust binary. 1,111 memories stored. It works.
But last week it almost cost me hours of debugging.
Claude confidently told me my AIMO3 competition notebook should use `--enable-prefix-caching` with vLLM. I trusted it. The notebook crashed. Scored 0/50. Burned a daily submission.
The problem? I had TWO memories:
- **January**: "prefix caching crashes with our vLLM build"
- **March**: "prefix caching works with the new animsamuelk wheels"
Claude found both. Picked the wrong one. Gave me a confident wrong answer based on the January memory. The March memory was correct — but Claude had no way to know they conflicted.
So I built `cross_reference`.
Now before answering any factual question, Claude calls:
```
cross_reference({ query: "should I use --enable-prefix-caching with vLLM?" })
```
And gets back:
```json
{
"status": "contradictions_found",
"confidence": 0.35,
"guidance": "WARNING: 1 contradiction detected across 12 memories.
The newer memory is likely more accurate.",
"contradictions": [
{
"newer": {
"date": "2026-03-18",
"preview": "Switched to animsamuelk wheels which support --enable-prefix-caching..."
},
"older": {
"date": "2026-01-15",
"preview": "prefix caching crashed with our samvalladares vLLM build..."
},
"recommendation": "Trust the newer memory. Consider demoting the older one."
}
],
"timeline": [ /* 12 memories sorted newest-first */ ]
}
```
Claude sees the contradiction BEFORE answering. Trusts the March memory. Gets it right.
**I've been using it for 2 days. It's caught contradictions I didn't even know I had.** Old project decisions that got reversed. Config values that changed. Library versions that got upgraded. All sitting in memory, waiting to give me wrong answers.
### How it works under the hood:
1. **Broad retrieval** — pulls up to 50 memories related to the query
2. **Cross-encoder reranking** — filters for quality (Jina Reranker v1 Turbo)
3. **Pairwise contradiction detection** — every memory pair checked for negation patterns ("don't" vs "do", "deprecated" vs "recommended") and correction signals ("now uses", "switched to", "replaced by")
4. **Topic gating** — only compares memories that share significant words (prevents false positives between unrelated memories)
5. **Confidence scoring** — agreements boost confidence, contradictions tank it, recency helps
6. **Timeline** — everything sorted newest-first so Claude sees the evolution
7. **Superseded list** — explicitly identifies which old memories should be demoted
### The bigger picture:
Context windows are now 1M tokens (Claude Opus 4.6, GPT-5.4). But bigger context doesn't fix this problem. The "Lost in the Middle" research shows accuracy drops from 92% to 78% at 1M tokens. More context = more chances for contradictions to slip through.
Memory systems need to be SMARTER, not just bigger. That's what Vestige does — 29 cognitive modules implementing real neuroscience:
- **FSRS-6** — memories naturally decay when unused, strengthen when accessed (21 parameters trained on 700M+ reviews)
- **Prediction Error Gating** — only stores what's genuinely new (no duplicates)
- **Dream consolidation** — replays memories to discover hidden connections (yes, like sleep)
- **Spreading activation** — search for "auth bug" and find the related JWT update from last week
- **cross_reference** — the new tool that catches contradictions before they become wrong answers
### Stats:
- 22 MCP tools
- 746 tests, 0 failures
- Zero `unsafe` code
- Clean security audit (0 findings — AgentAudit verified)
- Single 22MB Rust binary — no Docker, no PostgreSQL, no cloud
- Works with Claude Code, Cursor, VS Code, Xcode 26.3, JetBrains, Windsurf
### Install (30 seconds):
```bash
# macOS Apple Silicon
curl -L https://github.com/samvallad33/vestige/releases/latest/download/vestige-mcp-aarch64-apple-darwin.tar.gz | tar -xz
sudo mv vestige-mcp /usr/local/bin/
claude mcp add vestige vestige-mcp -s user
```
Then add to your CLAUDE.md:
```
Before answering factual questions, call cross_reference({ query: "the topic" })
to verify your memories don't contradict each other.
```
That's it. Your AI now fact-checks itself.
**GitHub:** https://github.com/samvallad33/vestige
I searched every memory MCP server out there — Mem0 (47K stars), Hindsight, Zep, Letta, OMEGA, Solitaire, MuninnDB. Some detect contradictions at ingest time (when you save). Nobody else gives the AI an on-demand tool to verify its own memories before answering, with confidence scoring and guidance on which memory to trust.
Happy to answer questions about the neuroscience, the architecture, or how to set it up.
---
## Post 2: r/LocalLLaMA (Cross-post, 2h later)
**Title:** `Your AI has 1000 memories. Some contradict each other. It doesn't know. I built the fix — 100% local, single Rust binary, zero cloud.`
---
The problem nobody talks about with AI memory:
You use a memory system. Over months, you accumulate 1000+ memories. Your project config changed 3 times. A library got deprecated. A decision got reversed. All those old memories are still there.
When your AI searches memory, it finds 5 results. Two of them disagree. The AI picks one — maybe the wrong one — and gives you a confident answer based on outdated information.
I built `cross_reference` to fix this. It's tool #22 in Vestige, my cognitive memory MCP server.
```
cross_reference({ query: "what database does the project use?" })
```
Returns:
```json
{
"status": "contradictions_found",
"confidence": 0.35,
"contradictions": [{
"newer": { "date": "2026-03", "preview": "Migrated to PostgreSQL..." },
"older": { "date": "2026-01", "preview": "Using SQLite for the backend..." }
}],
"guidance": "WARNING: Trust the newer memory."
}
```
The AI sees the conflict. Picks the right one. Every time.
**What makes this different from RAG/vector search:**
| | RAG | Vestige |
|---|---|---|
| Storage | Store everything | Prediction Error Gating — only stores what's new |
| Retrieval | Nearest neighbor | 7-stage cognitive pipeline with reranking |
| Contradictions | Returns both, hopes for the best | **Detects and flags before answering** |
| Decay | Nothing expires | FSRS-6 — memories fade naturally |
| Duplicates | Manual dedup | Self-healing via semantic comparison |
**The stack:**
- Rust 2024 edition. Single 22MB binary. No Python, no Docker, no cloud.
- FSRS-6 spaced repetition (21 parameters, 700M+ reviews)
- 29 cognitive modules (spreading activation, synaptic tagging, dream consolidation, prediction error gating)
- 3D neural dashboard at localhost:3927 (Three.js, real-time WebSocket)
- MCP server — works with any AI that speaks MCP
**100% local. Your data never leaves your machine.**
```bash
curl -L https://github.com/samvallad33/vestige/releases/latest/download/vestige-mcp-aarch64-apple-darwin.tar.gz | tar -xz
sudo mv vestige-mcp /usr/local/bin/
claude mcp add vestige vestige-mcp -s user
```
746 tests. Zero unsafe code. Clean security audit. AGPL-3.0.
GitHub: https://github.com/samvallad33/vestige
---
## Post 3: r/rust (Optional, technical audience)
**Title:** `I built a 22MB Rust binary that gives AI agents a brain — FSRS-6, 29 cognitive modules, 3D dashboard, and a new contradiction detection tool. 746 tests, zero unsafe.`
---
Vestige is a cognitive memory engine for AI agents. MCP server (stdio JSON-RPC + HTTP transport), SQLite WAL backend, USearch HNSW vector search, Nomic Embed v1.5 via fastembed (local ONNX, no API).
The latest addition: `cross_reference` — pairwise contradiction detection across memories at retrieval time. The AI calls it before answering factual questions to verify its memories don't disagree.
**Why Rust:**
- Single static binary (22MB with embedded SvelteKit dashboard via `include_dir!`)
- No runtime, no GC pauses during real-time search
- `tokio::sync::Mutex` for the cognitive engine, `std::sync::Mutex` for SQLite reader/writer split
- Zero `unsafe` blocks in the entire codebase
- `cargo test` runs 746 tests in 11 seconds
**Architecture:**
```
Axum 0.8 (dashboard + HTTP transport)
↕ WebSocket event bus (tokio::broadcast, 1024 capacity)
MCP Server (stdio JSON-RPC)
→ 22 tools dispatched via match on tool name
→ Arc<Storage> + Arc<Mutex<CognitiveEngine>>
SQLite WAL + FTS5 + USearch HNSW
→ fastembed 5.11 (Nomic Embed v1.5, 768D → 256D Matryoshka)
→ Jina Reranker v1 Turbo (cross-encoder, 38M params)
```
**Key crates:** rusqlite 0.38, axum 0.8, tokio 1, fastembed 5.11, usearch 2, chrono 0.4, serde 1, uuid 1, include_dir 0.7
**What I learned building it:**
- `OnceLock<Result<Mutex<TextEmbedding>, String>>` for lazy model initialization — the embedding model downloads ~130MB on first run, `OnceLock` ensures it only happens once and caches the error if it fails
- `floor_char_boundary()` saved me from a UTF-8 panic (content truncation with multi-byte chars)
- SQLite `PRAGMA journal_mode = WAL` + `synchronous = NORMAL` + `mmap_size = 268435456` gives surprisingly good concurrent read performance
- `try_lock()` pattern for cognitive features in search — if the lock is held (by dream consolidation), search falls back to simpler scoring instead of blocking
Clean security audit. Parameterized SQL everywhere. CSP headers on the dashboard. Constant-time auth comparison (`subtle::ConstantTimeEq`). File permissions 0o600/0o700.
GitHub: https://github.com/samvallad33/vestige
AGPL-3.0 | 746 tests | 79K+ LOC
---
## Posting Strategy
| Subreddit | When | Expected Engagement |
|-----------|------|-------------------|
| r/ClaudeAI | First — 12-14 UTC (US morning) | High — direct audience for MCP tools |
| r/LocalLLaMA | 2h after r/ClaudeAI | High — local-first angle resonates here |
| r/rust | Same day, evening UTC | Medium — technical deep dive audience |
| r/MachineLearning | Next day if first two do well | Lower but prestigious |
**Title formula that works on Reddit:** Personal story + specific problem + "nobody else has this"
**DO NOT do:** Product-name-first titles, marketing speak, "introducing X", "check out my project"
**DO:** Lead with the PAIN ("my AI gave me wrong answers"), show the FIX (the JSON output), then reveal the tool.