Make Vestige a durable, local, semantically-searchable retrieval layer over an
external system of record (GitHub Issues first), citing back to the canonical
record. Unlike a live ticket-system MCP proxy, Vestige keeps a durable embedded
index: searchable offline, joinable with the rest of memory, temporally
versioned, and re-syncable idempotently with no duplication.
Phases 1-2 of #57 plus a GitHub reference connector and source-aware search:
- Source envelope on KnowledgeNode/IngestInput (source_system, source_id,
source_url, source_updated_at, content_hash, synced_at, source_project,
source_type, source_author). Migration V17: nullable columns (additive),
partial UNIQUE index on (source_system, source_id), connector_cursors table.
- Idempotent sync primitives in vestige-core: upsert_by_source (content-hash
change detection), connector cursor checkpoints, reconcile_source_tombstones
(invalidate-don't-delete via bitemporal valid_until).
- Connector contract + run_sync driver + GitHub Issues connector behind the
optional `connectors` feature (on by default in vestige-mcp, off in the core
library default so non-connector consumers link no HTTP client).
- source_sync MCP tool ({"repo": "owner/name"}); token from GITHUB_TOKEN env
only. Search results gain a sourceRecord citation for connector memories.
Adversarial review fixes: GitHub `since` Z-form (the `+00:00` offset corrupted
the cursor server-side), un-tombstone clears superseded_by too, cursor never
advances past a failing record, Link next-url host-pinned (token-leak guard),
records_seen counts new records only.
Verified: cargo check/test/clippy -D warnings green across the workspace
(default and connectors features); 483 core tests pass. Version bump to 2.1.27
and tag deferred to release.
Refs #57
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rebased on v2.1.25 merge/supersede and bumped the post-release metadata to v2.1.26 so this branch does not roll versions backward.
Adds local vestige.toml defaults, output profiles, and MCP response precedence for search, timeline, codebase context, and session context.
Verified:
- cargo metadata --format-version 1 --locked --no-deps
- cargo test -p vestige-core config --no-fail-fast
- cargo test -p vestige-mcp config --no-fail-fast
Claude Code v2.1.91+ honors the per-tool annotation
`_meta["anthropic/maxResultSizeChars"]` (up to 500_000) to override
its 50K default truncation of `CallToolResult`. Without it, large
Vestige payloads are silently truncated and spilled to disk, forcing
the parent agent to chunk-read them.
Empirically observed truncation under realistic default parameters
(measured on v1.3.0 against ~3,300 memories; v2.x tool surface
preserves the same names + payload shapes):
search(detail_level="full", limit=20) -> 134,824 chars -> truncated
search(detail_level="summary", limit=10) -> 71,318 chars -> truncated
memory_timeline(limit=30) -> 83,626 chars -> truncated
This patch:
1. Adds `meta: Option<serde_json::Value>` to `ToolDescription` with
`#[serde(rename = "_meta")]` so the wire shape matches the MCP
spec. Backwards-compatible (the field is optional +
`skip_serializing_if`; older MCP clients ignore unknown JSON keys
per the spec).
2. Derives `Default` on `ToolDescription` so existing call sites can
adopt the new field via struct-update syntax
(`..Default::default()`) without restating it.
3. Annotates the four high-payload tools per measurement-driven
discipline; the other 21 tools deliberately do NOT carry the
annotation (cargo-cult prevention — a generous cap on every tool
dilutes the signal and trains future maintainers that the value
is arbitrary):
- search -> 300_000 (2.2x headroom over observed peak)
- memory_timeline -> 200_000 (2.4x headroom over observed peak)
- memory -> 100_000 (single-record bounded)
- codebase -> 100_000 (future-growth bounded)
Tools that COULD plausibly grow into the annotated set with future
workload (`deep_reference`, `cross_reference`, `memory_graph`,
`explore_connections`, `session_context`) are left unannotated
until empirical measurement shows truncation under realistic use.
4. Adds three regression tests in `server::tests`:
- test_high_payload_tools_have_max_result_size_annotation:
pins each cap value + asserts <= 500K Anthropic ceiling
- test_other_tools_do_not_carry_max_result_size_annotation:
cargo-cult prevention; dynamically iterates `tools/list` and
asserts every tool NOT in the discipline-prescribed set lacks
the annotation (robust to new tools being added by future PRs)
- test_meta_wire_shape_uses_underscore_meta_field:
pins the serde rename to `_meta` (the spec'd wire name) so a
refactor of `ToolDescription` cannot silently drop the rename
All 22 `server::tests` pass on v2.1.22 base (19 pre-existing + 3 new).
Full lib test suite: 379/380 pass; the 1 unrelated failure
(`tools::maintenance::tests::test_portable_export_writes_archive_to_storage_exports_dir`)
is a pre-existing Windows path-separator assertion bug in
`tools/maintenance.rs:823` (`path.ends_with("exports/portable-test.json")`
fails on Windows where the path uses `\`) — unaffected by this PR.
References:
- Anthropic CC v2.1.91 release notes (April 2026): "Added MCP tool
result persistence override via _meta['anthropic/maxResultSizeChars']
annotation (up to 500K), allowing larger results like DB schemas
to pass through without truncation"
- claude-agent-sdk-python v0.1.55 #756: forward bookkeeping
establishing the on-Tool-definition (not on-CallToolResult)
semantics for this annotation
Co-authored-by: Peter Lauzon <inbijiburu@protonmail.com>
CI / Release Build (aarch64-apple-darwin) (push) Blocked by required conditions
CI / Release Build (x86_64-unknown-linux-gnu) (push) Blocked by required conditions
CI / Release Build (x86_64-apple-darwin) (push) Blocked by required conditions
Test Suite / Unit Tests (push) Waiting to run
Test Suite / MCP E2E Tests (push) Waiting to run
Test Suite / User Journey Tests (push) Blocked by required conditions
Test Suite / Dashboard Build (push) Waiting to run
Test Suite / Code Coverage (push) Waiting to run
* feat(v2.0.9): Autopilot — backend event-subscriber routes 6 live events into cognitive hooks
The single architectural change that flips 14 dormant cognitive primitives
into active ones. Before this commit, Vestige's 20-event WebSocket bus had
zero backend subscribers — every emitted event flowed to the dashboard
animation layer and terminated. Cognitive modules with fully-built trigger
methods (synaptic_tagging.trigger_prp, predictive_memory.record_*,
activation_network.activate, prospective_memory.check_triggers, the 6h
auto-consolidation dreamer path) were never actually called from the bus.
New module `crates/vestige-mcp/src/autopilot.rs` spawns two tokio tasks at
startup:
1. Event subscriber — consumes the broadcast::Receiver, routes:
- MemoryCreated → synaptic_tagging.trigger_prp(CrossReference)
+ predictive_memory.record_memory_access(id, preview, tags)
- SearchPerformed → predictive_memory.record_query(q, [])
+ record_memory_access on top 10 result_ids
- MemoryPromoted → activation_network.activate(id, 0.3) spread
- MemorySuppressed → emit Rac1CascadeSwept (was declared-never-emitted)
- ImportanceScored (composite > 0.85 AND memory_id present)
→ storage.promote_memory + re-emit MemoryPromoted
- Heartbeat (memory_count > 700, 6h cooldown)
→ spawned find_duplicates sweep (rate-limited)
The loop holds the CognitiveEngine mutex only per-handler, never across
an await, so MCP tool dispatch is never starved.
2. Prospective poller — 60s tokio::interval calls
prospective_memory.check_triggers(Context { timestamp: now, .. }).
Matched intentions are logged at info! level today; v2.5 "Autonomic"
upgrades this to MCP sampling/createMessage for agent-side notifications.
ImportanceScored event gained optional `memory_id: Option<String>` field
(#[serde(default)], backward-compatible) so auto-promote has the id to
target. Both existing emit sites (server.rs tool dispatch, dashboard
handlers::score_importance) pass None because they score arbitrary content,
not stored memories — matches current semantics.
docs/VESTIGE_STATE_AND_PLAN.md §15 POST-v2.0.8 ADDENDUM records the full
three-agent audit that produced this architecture (2026-SOTA research,
active-vs-passive module audit, competitor landscape), the v2.0.9/v2.5/v2.6
ship order, and the one-line thesis: "the bottleneck was one missing
event-subscriber task; wiring it flips Vestige from memory library to
cognitive agent that acts on the host LLM."
Verified:
- cargo check --workspace clean
- cargo clippy --workspace -- -D warnings clean (let-chain on Rust 1.91+)
- cargo test -p vestige-mcp --lib 356/356 passing, 0 failed
* fix(autopilot): supervisor + dedup race + opt-out env var
Three blockers from the 5-agent v2.0.9 audit, all in autopilot.rs.
1. Supervisor loops around both tokio tasks (event subscriber + prospective
poller). Previously, if a cognitive hook panicked on a single bad memory,
the spawned task died permanently and silently — every future event lost.
Now the outer supervisor catches JoinError::is_panic(), logs the panic
with full error detail, sleeps 5s, and respawns the inner task. Turns
a permanent silent failure into a transient hiccup.
2. DedupSweepState struct replaces the bare Option<Instant> timestamp. It
tracks the in-flight JoinHandle so the next Heartbeat skips spawning a
second sweep while the first is still running. Previously, the cooldown
timestamp was set BEFORE spawning the async sweep, which allowed two
concurrent find_duplicates scans on 100k+ memory DBs where the sweep
could exceed the 6h cooldown window. is_running() drops finished handles
so a long-dead sweep doesn't block the next legitimate tick.
3. VESTIGE_AUTOPILOT_ENABLED=0 opt-out. v2.0.8 users updating in place
can preserve the passive-library contract by setting the env var to
any of {0, false, no, off}. Any other value (unset, 1, true, etc.)
enables the default v2.0.9 Autopilot behavior. spawn() early-returns
with an info! log before any task is spawned.
Audit breakdown:
- Agent 1 (internals): NO-GO → fixed (1, 2)
- Agent 2 (backward compat): NO-GO → fixed (3)
- Agent 3 (orphan cleanup): GO clean
- Agent 4 (runtime safety): GO clean
- Agent 5 (release prep): GO, procedural note logged
Verification:
- cargo check -p vestige-mcp: clean
- cargo test -p vestige-mcp --lib: 373 passed, 0 failed
- cargo clippy -p vestige-mcp --lib --bins -- -D warnings: clean
* chore(release): v2.0.9 "Autopilot"
Bump workspace + vestige-core + vestige-mcp + apps/dashboard to 2.0.9.
CHANGELOG [2.0.9] entry + README hero banner rewrite to "Autopilot".
Scope (two commits on top of v2.0.8):
- 0e9b260: 3,091 LOC orphan-code cleanup
- fe7a68c: Autopilot backend event-subscriber
- HEAD (this branch): supervisor + dedup race + opt-out env var hardening
Pure backend release — tool count unchanged (24), schema unchanged,
JSON-RPC shape unchanged, CLI flags unchanged. Only visible behavior
change is the Autopilot task running in the background, which is
VESTIGE_AUTOPILOT_ENABLED=0-gated.
Test gate: 1,223 passing / 0 failed (workspace, no-fail-fast).
Clippy: clean on vestige-mcp lib + bins with -D warnings.
Audit: 5 parallel agents (internals, backward compat, orphan cleanup,
runtime safety, release prep) — all GO after hardening commit.
server.rs:212 was documenting v2.0.4's state (23 tools) but the
handle_tools_list test two functions below asserts 24 tools since
v2.0.5 (when `suppress` landed). Align the comment with the assertion
and point at the assertion as the source of truth so future tool
additions can't introduce comment drift without also tripping a test
failure.
The MCP `instructions` field is injected into every connecting client's
system prompt on Initialize — Claude Code, Cursor, Zed, Windsurf, and
anyone else running Vestige. That reach demands a default that serves
every user, not the project maintainer's personal workflow.
Default ("minimal", 3 sentences) tells the client how to use Vestige
and how to respond to explicit feedback signals. It is safe for every
audience: competitive coders, hobbyists saving recipes, Rails devs
saving bug fixes, enterprise deployments under system-prompt review.
The full composition mandate — Composing / Never-composed /
Recommendation shape + FSRS-trust blocking phrase + origin case study
— is load-bearing for decision-adjacent work but misfires on trivial
retrievals ("what's my favorite color"). Opt in on your own machine
with `VESTIGE_SYSTEM_PROMPT_MODE=full`; four hundred strangers do not
inherit one maintainer's trauma scar on every session.
Extracted into build_instructions() so the branch is a single env-var
check on Initialize, not a compile-time switch. main.rs --help output
advertises the new variable alongside VESTIGE_DASHBOARD_ENABLED.
First AI memory system to model forgetting as a neuroscience-grounded
PROCESS rather than passive decay. Adds the `suppress` MCP tool (#24),
Rac1 cascade worker, migration V10, and dashboard forgetting indicators.
Based on:
- Anderson, Hanslmayr & Quaegebeur (2025), Nat Rev Neurosci — right
lateral PFC as the domain-general inhibitory controller; SIF
compounds with each stopping attempt.
- Cervantes-Sandoval et al. (2020), Front Cell Neurosci PMC7477079 —
Rac1 GTPase as the active synaptic destabilization mechanism.
What's new:
* `suppress` MCP tool — each call compounds `suppression_count` and
subtracts a `0.15 × count` penalty (saturating at 80%) from
retrieval scores during hybrid search. Distinct from delete
(removes) and demote (one-shot).
* Rac1 cascade worker — background sweep piggybacks the 6h
consolidation loop, walks `memory_connections` edges from
recently-suppressed seeds, applies attenuated FSRS decay to
co-activated neighbors. You don't just forget Jake — you fade
the café, the roommate, the birthday.
* 24h labile window — reversible via `suppress({id, reverse: true})`
within 24 hours. Matches Nader reconsolidation semantics.
* Migration V10 — additive-only (`suppression_count`, `suppressed_at`
+ partial indices). All v2.0.x DBs upgrade seamlessly on first launch.
* Dashboard: `ForgettingIndicator.svelte` pulses when suppressions
are active. 3D graph nodes dim to 20% opacity when suppressed.
New WebSocket events: `MemorySuppressed`, `MemoryUnsuppressed`,
`Rac1CascadeSwept`. Heartbeat carries `suppressed_count`.
* Search pipeline: SIF penalty inserted into the accessibility stage
so it stacks on top of passive FSRS decay.
* Tool count bumped 23 → 24. Cognitive modules 29 → 30.
Memories persist — they are INHIBITED, not erased. `memory.get(id)`
returns full content through any number of suppressions. The 24h
labile window is a grace period for regret.
Also fixes issue #31 (dashboard graph view buggy) as a companion UI
bug discovered during the v2.0.5 audit cycle:
* Root cause: node glow `SpriteMaterial` had no `map`, so
`THREE.Sprite` rendered as a solid-coloured 1×1 plane. Additive
blending + `UnrealBloomPass(0.8, 0.4, 0.85)` amplified the square
edges into hard-edged glowing cubes.
* Fix: shared 128×128 radial-gradient `CanvasTexture` singleton used
as the sprite map. Retuned bloom to `(0.55, 0.6, 0.2)`. Halved fog
density (0.008 → 0.0035). Edges bumped from dark navy `0x4a4a7a`
to brand violet `0x8b5cf6` with higher opacity. Added explicit
`scene.background` and a 2000-point starfield for depth.
* 21 regression tests added in `ui-fixes.test.ts` locking every
invariant in (shared texture singleton, depthWrite:false, scale
×6, bloom magic numbers via source regex, starfield presence).
Tests: 1,284 Rust (+47) + 171 Vitest (+21) = 1,455 total, 0 failed
Clippy: clean across all targets, zero warnings
Release binary: 22.6MB, `cargo build --release -p vestige-mcp` green
Versions: workspace aligned at 2.0.5 across all 6 crates/packages
Closes#31
Handle 'vestige/memory://' and 'vestige/codebase://' URIs by stripping the
provider prefix before scheme matching. This fixes compatibility with
MCP clients like OpenCode that prepend the provider name to resource URIs.
Fixes#19
Adds a second transport layer alongside stdio — Streamable HTTP on port
3928. Enables Claude.ai, remote clients, and web integrations to connect
to Vestige over HTTP with per-session McpServer instances.
- POST /mcp (JSON-RPC) + DELETE /mcp (session cleanup)
- Bearer token auth with constant-time comparison (subtle crate)
- Auto-generated UUID v4 token persisted with 0o600 permissions
- Per-session McpServer instances with 30-min idle reaper
- 100 max sessions, 50 concurrency limit, 256KB body limit
- --http-port flag + VESTIGE_HTTP_PORT / VESTIGE_HTTP_BIND env vars
- Module exports moved from binary to lib.rs for reusability
- vestige CLI gains `serve` subcommand via shared lib
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
3 new MCP tools (16 → 19 total):
- importance_score: 4-channel neuroscience importance scoring (novelty/arousal/reward/attention)
- session_checkpoint: batch smart_ingest up to 20 items with PE Gating
- find_duplicates: cosine similarity clustering with union-find for dedup
CLI: vestige ingest command for memory ingestion via command line
Core: made get_node_embedding public, added get_all_embeddings for dedup scanning
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add web dashboard (axum) on port 3927 with memory browser, search, and
system stats. New MCP tools: memory_timeline, memory_changelog,
health_check, consolidate, stats, backup, export, gc. Search now supports
detail_level (brief/summary/full) to control token usage. Add backup_to()
and get_recent_state_transitions() to storage layer. Bump to v1.2.0.
P0 fixes:
- Add `vestige backup <path>` — full DB copy with WAL checkpoint flush
- Add `vestige export --format json|jsonl [--tags] [--since] <path>` —
paginated memory export with tag/date filtering
- Add `vestige gc --min-retention 0.1 [--max-age-days] [--dry-run] [--yes]`
— bulk cleanup of stale memories with safety prompts
- Fix apply_decay() scaling: batched pagination (500 rows/batch) with
explicit transactions instead of loading all nodes into memory
- Fix hidden MCP resources: memory://insights and memory://consolidation-log
now listed in resources/list (were implemented but undiscoverable)
P1 fixes:
- Add auto-consolidation on server startup: FSRS-6 decay runs in background
after 2s delay, only if last consolidation was >6 hours ago
- Add encryption at rest via SQLCipher feature flag: use --features encryption
with VESTIGE_ENCRYPTION_KEY env var (bundled-sqlite and encryption are
mutually exclusive)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add #![allow(dead_code)] to deprecated tool modules (kept for
backwards compatibility but not exposed in MCP tool list)
- Mark unused functions with #[allow(dead_code)] annotations
- Fix unused variable warnings (prefix with _)
- Apply clippy auto-fixes for redundant closures and derives
- Fix test to account for protocol version negotiation
- Reorganize tools/mod.rs to clarify active vs deprecated tools
Security review: LOW RISK - no critical vulnerabilities found
Dead code review: deprecated tools properly annotated
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Claude Desktop requests protocol version 2025-06-18 but Vestige was
responding with 2025-11-25, causing Claude Desktop to disconnect.
Now the server negotiates: if the client requests an older version,
use it. This maintains backward compatibility with older clients.
Fixes: Claude Desktop "Server transport closed unexpectedly" error
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Users were seeing 30 tools in /mcp which was overwhelming. Now only 8
essential tools are listed:
1. search - unified hybrid search
2. memory - get/delete/state operations
3. codebase - patterns and decisions
4. intention - prospective memory
5. ingest - add memories
6. smart_ingest - intelligent ingestion
7. promote_memory - thumbs up
8. demote_memory - thumbs down
Deprecated tools (recall, semantic_search, etc.) still work internally
for backward compatibility but are no longer listed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
FSRS-6 spaced repetition, spreading activation, synaptic tagging,
hippocampal indexing, and 130 years of memory research.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
