mirror of
https://github.com/samvallad33/vestige.git
synced 2026-04-25 00:36:22 +02:00
* docs: anti-hedging banner + master state-and-plan + UI roadmap
Project-level CLAUDE.md line 1 is now the anti-hedging banner referencing
the global rule at /Users/entity002/CLAUDE.md. When Sam commits to an
execution path (content cadence, shipping velocity, outreach, launch),
forecasts compute as if the commitment is already met — no discounting by
execution probability.
VESTIGE_STATE_AND_PLAN.md (1135 lines, 14 parts): single-page reference
any AI agent can read to plan the next phase without re-reading the repo.
Covers current state of every crate, the 24 MCP tools, the 19 WebSocket
event types, the 8 IDE integrations, and v2.1 through v3.0 roadmap.
UI_ROADMAP_v2.1_v2.2.md: compiled UI-gap analysis from 4 parallel research
agents (April 19). 10 critical UI gaps + 2026 bleeding-edge patterns +
unclaimed territory (menu bar widget, voice-first, AR/VR, time-scrubber).
* feat(v2.2-pulse): InsightToast + multi-process STORAGE docs
Two independent ship items landing together on the v2.2 branch ahead of
the Tuesday launch — a new UI surface that makes Vestige's cognitive
events visible in real time, and honest documentation of the multi-process
safety story that underpins the Stigmergic Swarm narrative.
**InsightToast** (apps/dashboard/src/lib/components/InsightToast.svelte,
apps/dashboard/src/lib/stores/toast.ts):
The dashboard already had a working WebSocket event stream on
ws://localhost:3927/ws that broadcast every cognitive event (dream
completions, consolidation sweeps, memory promotions/demotions, active-
forgetting suppression and Rac1 cascades, bridge discoveries). None of
that was surfaced to a user looking at anything other than the raw feed
view. InsightToast subscribes to the existing eventFeed derived store,
filters the spammy lifecycle events (Heartbeat, SearchPerformed,
RetentionDecayed, ActivationSpread, ImportanceScored, MemoryCreated),
and translates the narrative events into ephemeral toasts with a
bioluminescent colored accent matching EVENT_TYPE_COLORS.
Design notes:
- Rate-limited ConnectionDiscovered at 1.5s intervals (dreams emit many).
- Max 4 visible toasts, auto-dismiss at 4.5-7s depending on event weight.
- Click or Enter/Space to dismiss early.
- Bottom-right on desktop, top-banner on mobile.
- Reduced-motion honored via prefers-reduced-motion.
- Zero new websocket subscriptions — everything piggybacks on the
existing derived store.
Also added a "Preview Pulse" button to Settings -> Cognitive Operations
that fires a synthetic sequence of four toasts (DreamCompleted,
ConnectionDiscovered, MemorySuppressed, ConsolidationCompleted) so
the animation is demoable without waiting for real cognitive activity.
**Multi-Process Safety section in docs/STORAGE.md**:
Grounds the Stigmergic Swarm story with concrete tables of what the
current WAL + 5s busy_timeout configuration actually supports vs what
remains experimental. Key honest points:
- Shared --data-dir + ONE vestige-mcp + N clients is the shipping
pattern for multi-agent coordination.
- Two vestige-mcp processes writing the same file is experimental —
documented with the lsof + pkill recovery path.
- Roadmap lists the three items that would promote it to "supported":
advisory file lock, retry-with-jitter on SQLITE_BUSY, and a
concurrent-writer load test.
Build + typecheck:
- npm run check: 0 errors, 0 warnings across 583 files
- npm run build: clean static build, adapter-static succeeds
* feat(v2.3-terrarium): Memory Birth Ritual + event pipeline fix
v2.3 "Terrarium" headline feature. When a MemoryCreated event arrives, a
glowing orb materialises in the cosmic center (camera-relative z=-40),
gestates for ~800ms growing from a tiny spark into a full orb, then arcs
along a dynamic quadratic Bezier curve to the live position of the real
node, and on arrival hands off to the existing RainbowBurst + Shockwave +
RippleWave cascade. The target position is re-resolved every frame so
the force simulation can move the destination during flight without the
orb losing its mark.
**New primitive — EffectManager.createBirthOrb()** (effects.ts):
Accepts a camera, a color, a live target-position getter, and an
arrival callback. Owns a sprite pair (outer halo + inner bright core),
both depthTest:false with renderOrder 999/1000 so the orb is always
visible through the starfield and the graph.
- Gestation phase: easeOutCubic growth + sinusoidal pulse, halo tints
from neutral to event color as the ritual charges.
- Flight phase: QuadraticBezierCurve3 with control point at midpoint
raised on Y by 30 + 15% of orb-to-target distance (shooting-star
arc). Sampled with easeInOutQuad. Orb shrinks ~35% approaching target.
- Arrival: fires onArrive callback once, then fades out over 8 frames
while expanding slightly (energy dispersal).
- Caller's onArrive triggers the burst cascade at arrivePos (NOT the
original spawnPos — the force sim may have moved the target during
the ritual, so we re-read nodeManager.positions on arrival).
- Dispose path integrated with existing EffectManager.dispose().
**Event pipeline fix — Graph3D.processEvents()**:
Previously tracked `processedEventCount` assuming APPEND order, but
websocket.ts PREPENDS new events (index 0) and caps the array at
MAX_EVENTS. Result: only the first MemoryCreated event after page
load fired correctly; subsequent ones reprocessed the oldest entry.
Fixed to walk from index 0 until hitting the last-processed event
by reference identity — correct regardless of array direction or
eviction pressure. Events are then processed oldest-first so causes
precede effects. Found while wiring the v2.3 demo button; would have
manifested as "first orb only" in production.
**Demo trigger** (Settings -> Birth Ritual Preview):
Button that calls websocket.injectEvent() with a synthetic
MemoryCreated event, cycling through node types (fact / concept /
pattern / decision / person / place) to showcase the type-color
mapping. Downstream consumers can't distinguish synthetic from real,
so this drives the full ritual end-to-end. Intended for demo clip
recording for the Wednesday launch.
**Test coverage:**
- events.test.ts now tests the v2.3 birth ritual path: spawns 2+
sprites in the scene immediately, and fires the full arrival
cascade after driving the effects.update() loop past the ritual
duration.
- three-mock.ts extended with Vector3.addVectors, Vector3.applyQuaternion,
Color.multiplyScalar, Quaternion, QuadraticBezierCurve3, Texture,
and Object3D.quaternion/renderOrder so production code runs unaltered
in tests.
Build + typecheck:
- npm run check: 0 errors, 0 warnings across 583 files
- npm test: 251/251 pass (net +0 from v2.2)
- npm run build: clean adapter-static output
The Sanhedrin Shatter (anti-birth ritual for hallucination veto) needs
server-side event plumbing and is deferred. Ship this as the Wednesday
visual mic-drop.
* fix(v2.3): 5 FATAL bugs + 4 god-tier upgrades from post-ship audit
Post-ship audit surfaced 6 FATALs and 4 upgrades. Shipping 5 of the 6 +
all 4 upgrades. FATAL 4 (VRAM hemorrhage from un-pooled label canvases
in createTextSprite) is pre-existing, not from this session, and scoped
separately for a proper texture-pool refactor.
**FATAL 1 — Toast Silent Lobotomy** (stores/toast.ts)
Subscriber tracked events[0] only. When Svelte batched multiple events
in one update tick (swarm firing DreamCompleted + ConnectionDiscovered
within the same millisecond), every event but the newest got silently
dropped. Fixed to walk from index 0 until hitting lastSeen — same
pattern as Graph3D.processEvents. Processes oldest-first to preserve
narrative order.
**FATAL 2 — Premature Birth** (graph/nodes.ts + graph/events.ts)
Orb flight is 138 frames; materialization was 30 frames. Node popped
fully grown ~100 frames before orb arrived — cheap UI glitch instead
of a biological birth. Added `addNode(..., { isBirthRitual: true })`
option that reserves the physics slot but hides mesh/glow/label and
skips the materializing queue. New `igniteNode(id)` flips visibility
and enqueues materialization. events.ts onArrive now calls igniteNode
at the exact docking moment, so the elastic spring-up peaks on impact.
**FATAL 3 — 120Hz ProMotion Time-Bomb** (components/Graph3D.svelte)
All physics + effect counters are frame-based. On a 120Hz display every
ritual ran at 2x speed. Added a `lastTime`-based governor in animate()
that early-returns if dt < 16ms, clamping effective rate to ~60fps.
`- (dt % 16)` carry avoids long-term drift. Zero API changes; tonight's
fast fix until physics is rewritten to use dt.
**FATAL 5 — Bezier GC Panic** (graph/effects.ts birth-orb update)
Flight phase allocated a new Vector3 (control point) and a new
QuadraticBezierCurve3 every frame per orb. With 3 orbs in flight that's
360 objects/sec for the GC to collect. Rewrote as inline algebraic
evaluation — zero allocations per frame, identical curve.
**FATAL 6 — Phantom Shockwave** (graph/events.ts)
A 166ms setTimeout fired the 2nd shockwave. If the user navigated
away during that window the scene was disposed, the timer still
fired, and .add() on a dead scene threw unhandled rejection. Dropped
the setTimeout entirely; both shockwaves fire immediately in onArrive
with different scales/colors for the same layered-crash feel.
**UPGRADE 1 — Sanhedrin Shatter** (graph/effects.ts birth-orb update)
If getTargetPos() returns undefined AFTER gestation (target node was
deleted mid-ritual — Stop hook sniping a hallucination), the orb
turns blood-red, triggers a violent implosion in place, and skips
the arrival cascade. Cognitive immune system made visible.
**UPGRADE 2 — Newton's Cradle** (graph/events.ts onArrive)
On docking the target mesh's scale gets bumped 1.8×, so the elastic
materialization + force-sim springs physically recoil instead of the
orb landing silently. The graph flinches when an idea is born into it.
**UPGRADE 3 — Hover Panic** (stores/toast.ts + InsightToast.svelte)
Paused dwell timer on mouseenter/focus, resume on mouseleave/blur.
Stored remaining ms at pause so resume schedules a correctly-sized
timer. CSS pairs via `animation-play-state: paused` on the progress
bar. A toast the user is reading no longer dismisses mid-sentence.
**UPGRADE 4 — Event Horizon Guard** (components/Graph3D.svelte)
If >MAX_EVENTS (200) events arrive in one tick, lastProcessedEvent
falls off the end of the array and the walk consumes ALL 200 entries
as "fresh" — GPU meltdown from 200 simultaneous births. Detect the
overflow and drop the batch with a console.warn, advancing the
high-water mark so next frame is normal.
Build + test:
- npm run check: 0 errors, 0 warnings
- npm test: 251/251 pass
- npm run build: clean static build
* test(v2.3): full e2e + integration coverage for Pulse + Birth Ritual
Post-ship verification pass — five parallel write-agents produced 229 new
tests across vitest units, vitest integration, and Playwright browser e2e.
Net suite: 361 vitest pass (up from 251, +110) and 9/9 Playwright pass on
back-to-back runs.
**toast.test.ts (NEW, 661 lines, 42 tests)**
Silent-lobotomy batch walk proven (multi-event tick processes ALL, not
just newest, oldest-first ordering preserved). Hover-panic pause/resume
with remaining-ms math. All 9 event type translations asserted, all 11
noise types asserted silent. ConnectionDiscovered 1500ms throttle.
MAX_VISIBLE=4 eviction. clear() tears down all timers. fireDemoSequence
staggers 4 toasts at 800ms intervals. vi.useFakeTimers + vi.mock of
eventFeed; vi.resetModules in beforeEach for module-singleton isolation.
**websocket.test.ts (NEW, 247 lines, 30 tests)**
injectEvent adds to front, respects MAX_EVENTS=200 with FIFO eviction,
triggers eventFeed emissions. All 6 derived stores (isConnected,
heartbeat, memoryCount, avgRetention, suppressedCount, uptimeSeconds)
verified — defaults, post-heartbeat values, clearEvents preserves
lastHeartbeat. 13 formatUptime boundary cases (0/59/60/3599/3600/
86399/86400 seconds + negative / NaN / ±Infinity).
**effects.test.ts (EXTENDED, +501 lines, +21 tests, 51 total)**
createBirthOrb full lifecycle — sprite count (halo + core), cosmic
center via camera.quaternion, gestation phase (position lock, opacity
rise, scale easing, color tint), flight Bezier arc above linear
midpoint at t=0.5, dynamic mid-flight target redirect. onArrive fires
exactly once at frame 139. Post-arrival fade + disposal cleans scene
children. Sanhedrin Shatter: target goes undefined mid-flight →
onArrive NEVER called, implosion spawned, halo blood-red, eventual
cleanup. dispose() cleans active orbs. Multiple simultaneous orbs.
Custom gestation/flight frame opts honored. Zero-alloc invariant
smoke test (6 orbs × 150 frames, no leaks).
**nodes.test.ts (EXTENDED, +197 lines, +10 tests, 42 total)**
addNode({isBirthRitual:true}) hides mesh/glow/label immediately,
stamps birthRitualPending sentinel with correct totalFrames +
targetScale, does NOT enqueue materialization. igniteNode flips
visibility + enqueues materialization. Idempotent — second call
no-op. Non-ritual nodes unaffected. Unknown id is safe no-op.
Position stored in positions map while invisible (force sim still
sees it). removeNode + late igniteNode is safe.
**events.test.ts (EXTENDED, +268 lines, +7 tests, 55 total)**
MemoryCreated → mesh hidden immediately, 2 birth-orb sprites added,
ZERO RingGeometry meshes and ZERO Points particles at spawn. Full
ritual drive → onArrive fires, node visible + materializing, sentinel
cleared. Newton's Cradle: target mesh scale exactly 0.001 * 1.8 right
after arrival. Dual shockwave: exactly 2 Ring meshes added. Re-read
live position on arrival — force-sim motion during ritual → burst
lands at the NEW position. Sanhedrin abort path → rainbow burst,
shockwave, ripple wave are NEVER called (vi.spyOn).
**three-mock.ts (EXTENDED)**
Added Color.setRGB — production Three.js has it, the Sanhedrin-
Shatter path in effects.ts uses it. Two write-agents independently
monkey-patched the mock inline; consolidated as a 5-line mock
addition so tests stay clean.
**e2e/pulse-toast.spec.ts (NEW, 235 lines, 6 Playwright tests)**
Navigate /dashboard/settings → click Preview Pulse → assert first
toast appears within 500ms → assert >= 2 toasts visible at peak.
Click-to-dismiss removes clicked toast (matched by aria-label).
Hover survives >8s past the 5.5s dwell. Keyboard Enter dismisses
focused toast. CSS animation-play-state:paused on .toast-progress-
fill while hovered, running on mouseleave. Screenshots attached to
HTML report. Zero backend dependency (fireDemoSequence is purely
client-side).
**e2e/birth-ritual.spec.ts (NEW, 199 lines, 3 Playwright tests)**
Canvas mounts on /dashboard/graph (gracefully test.fixme if MCP
backend absent). Settings button injection + SPA route to /graph
→ screenshot timeline at t=0/500/1200/2000/2400/3000ms attached
to HTML report. pageerror + console-error listeners catch any
crash (would re-surface FATAL 6 if reintroduced). Three back-to-
back births — no errors, canvas still dispatches clicks.
Run commands:
cd apps/dashboard && npm test # 361/361 pass, ~600ms
cd apps/dashboard && npx playwright test # 9/9 pass, ~25s
Typecheck: 0 errors, 0 warnings. Build: clean adapter-static.
* fix(graph): default /api/graph to newest-memory center, add sort param
memory_timeline PR #37 exposed the same class of bug in the graph
endpoint: the dashboard Graph page (and the /api/graph endpoint it
hits) defaulted to centering on the most-connected memory, ran BFS at
depth 3, and capped the subgraph at 150 nodes. On a mature corpus this
clustered the visualization around a historical hotspot and hid freshly
ingested memories that hadn't accumulated edges yet. User-visible
symptom: TimeSlider on /graph showing "Feb 21 → Mar 1 2026" when the
database actually contains memories through today (Apr 20).
**Backend (`crates/vestige-mcp/src/dashboard/handlers.rs`):**
- `GraphParams` gains `sort: Option<String>` (accepted: "recent" |
"connected", unknown falls back to "recent").
- New internal `GraphSort` enum + case-insensitive `parse()`.
- Extracted `default_center_id(storage, sort)` so handler logic and
tests share the same branching. Recent path picks `get_all_nodes(1,
0)` (ORDER BY created_at DESC). Connected path picks
`get_most_connected_memory`, degrading gracefully to recent if the
DB has no edges yet.
- Default behaviour flipped from "connected" to "recent" — matches
user expectation of "show me my recent stuff".
**Dashboard (`apps/dashboard`):**
- `api.graph()` accepts `sort?: 'recent' | 'connected'` with JSDoc
explaining the rationale.
- `/graph/+page.svelte` passes `sort: 'recent'` when no query or
center_id is active. Query / center_id paths unchanged — they
already carry their own centering intent.
**Tests:** 6 new unit tests in `handlers::tests`:
- `graph_sort_parse_defaults_to_recent` (None, empty, garbage,
"recent", "Recent", "RECENT")
- `graph_sort_parse_accepts_connected_case_insensitive`
- `default_center_id_recent_returns_newest_node` — ingest 3 nodes,
assert newest is picked
- `default_center_id_connected_prefers_hub_over_newest` — wire a hub
node with 3 spokes, then ingest a newer "lonely" node; assert the
hub wins in Connected mode even though it's older
- `default_center_id_connected_falls_back_to_recent_when_no_edges`
— fresh DB with no connections still returns newest, not 404
- `default_center_id_returns_not_found_on_empty_db` — both modes
return 404 cleanly on empty storage
Build + test:
- cargo test -p vestige-mcp --lib handlers:: → 6/6 pass
- cargo test --workspace --lib → 830/830 pass, 0 failed
- cargo clippy -p vestige-core -p vestige-mcp --lib -- -D warnings →
clean
- npm run check → 0 errors, 0 warnings
- npm test → 361/361 pass
Binary already installed at ~/.local/bin/vestige-mcp (copied from
cargo build --release -p vestige-mcp). New Claude Desktop / Code
sessions will pick it up automatically when they respawn their MCP
subprocess. The dashboard HTTP server on port 3927 needs a manual
relaunch from a terminal with the usual pattern:
nohup bash -c 'tail -f /dev/null | \
VESTIGE_DASHBOARD_ENABLED=true ~/.local/bin/vestige-mcp' \
> /tmp/vestige-mcp.log 2>&1 & disown
* feat(v2.4): UI expansion — 8 new surfaces exposing the cognitive engine
Sam asked: "Build EVERY SINGLE MISSING UI PIECE." 10 parallel agents shipped
10 new viewports over the existing Rust backend, then 11 audit agents
line-by-line reviewed each one, extracted pure-logic helpers, fixed ~30
bugs, and shipped 549 new unit tests. Everything wired through the layout
with single-key shortcuts and a live theme toggle.
**Eight new routes**
- `/reasoning` — Reasoning Theater: Cmd+K ask palette → animated 8-stage
deep_reference pipeline + FSRS-trust-scored evidence cards +
contradiction arcs rendered as live SVG between evidence nodes
- `/duplicates` — threshold-driven cluster detector with winner selection,
Merge/Review/Dismiss actions, debounced slider
- `/dreams` — Dream Cinema: trigger dream + scrubbable 5-stage replay
(Replay → Cross-reference → Strengthen → Prune → Transfer) + insight
cards with novelty glow
- `/schedule` — FSRS Review Calendar: 6×7 grid with urgency color
bands (overdue/today/week/future), retention sparkline, expand-day list
- `/importance` — 4-channel radar (Novelty/Arousal/Reward/Attention) with
composite score + top-important list
- `/activation` — live spreading-activation view: search → SVG concentric
rings with decay animation + live-mode event feed
- `/contradictions` — 2D cosmic constellation of conflicting memories,
arcs colored by severity, tooltips with previews
- `/patterns` — cross-project pattern transfer heatmap with category
filters, top-transferred sidebar
**Three layout additions**
- `AmbientAwarenessStrip` — slim top band with retention vitals, at-risk
count, active intentions, recent dream, activity sparkline, dreaming
indicator, Sanhedrin-watch flash. Pure `$derived` over existing stores.
- `ThemeToggle` — dark/light/auto cycle with matchMedia listener,
localStorage persistence, SSR-safe, reduced-motion-aware. Rendered in
sidebar footer next to the connection dot.
- `MemoryAuditTrail` — per-memory Sources panel integrated as a
Content/Audit tab into the existing /memories expansion.
**Pure-logic helper modules extracted (for testability + reuse)**
reasoning-helpers, duplicates-helpers, dream-helpers, schedule-helpers,
audit-trail-helpers, awareness-helpers, contradiction-helpers,
activation-helpers, patterns-helpers, importance-helpers.
**Bugs fixed during audit (not exhaustive)**
- Trust-color inconsistency between EvidenceCard and the page confidence
ring (0.75 boundary split emerald vs amber)
- `new Date('garbage').toLocaleDateString()` returned literal "Invalid Date"
in 3 components — all now return em-dash or raw string
- NaN propagation in `Math.max(0, Math.min(1, NaN))` across clamps
- Off-by-one PRNG in audit-trail seeded mock (seed === UINT32_MAX yielded
rand() === 1.0 → index out of bounds)
- Duplicates dismissals keyed by array index broke on re-fetch; now keyed
by sorted cluster member IDs with stale-dismissal pruning
- Empty-cluster crash in DuplicateCluster.pickWinner
- Undefined tags crash in DuplicateCluster.safeTags
- Debounce timer leak in threshold slider (missing onDestroy cleanup)
- Schedule day-vs-hour granularity mismatch between calendar cell and
sidebar list ("today" in one, "in 1d" in the other)
- Schedule 500-memory hard cap silently truncated; bumped to 2000 + banner
- Schedule DST boundary bug in daysBetween (wall-clock math vs
startOfDay-normalized)
- Dream stage clamp now handles NaN/Infinity/floats
- Dream double-click debounce via `if (dreaming) return`
- Theme setTheme runtime validation; initTheme idempotence (listener +
style-element dedup on repeat calls)
- ContradictionArcs node radius unclamped (trust < 0 or > 1 rendered
invalid sizes); tooltip position clamp (could push off-canvas)
- ContradictionArcs $state closure capture (width/height weren't reactive
in the derived layout block)
- Activation route was MISSING from the repo — audit agent created it
with identity-based event filtering and proper RAF cleanup
- Layout: ThemeToggle was imported but never rendered — now in sidebar
footer; sidebar overflow-y-auto added for the 16-entry nav
**Tests — 549 new, 910 total passing (0 failures)**
ReasoningChain 42 | EvidenceCard 50
DuplicateCluster 64 | DreamStageReplay 19
DreamInsightCard 43 | FSRSCalendar 32
MemoryAuditTrail 45 | AmbientAwareness 60
theme (store) 31 | ContradictionArcs 43
ActivationNetwork 54 | PatternTransfer 31
ImportanceRadar 35 | + existing 361 tests still green
**Gates passed**
- `npm run check`: 0 errors, 0 warnings across 623 files
- `npm test`: 910/910 passing, 22 test files
- `npm run build`: clean adapter-static output
**Layout wiring**
- Nav array expanded 8 → 16 entries (existing 8 + 8 new routes)
- Single-key shortcuts added: R/A/D/C/P/U/X/N (no conflicts with
existing G/M/T/F/E/I/S/,)
- Cmd+K palette search works across all 16
- Mobile nav = top 5 (Graph, Reasoning, Memories, Timeline, Feed)
- AmbientAwarenessStrip mounted as first child of <main>
- ThemeToggle rendered in sidebar footer (was imported-but-unmounted)
- Theme initTheme() + teardown wired into onMount cleanup chain
Net branch delta: 47 files changed, +13,756 insertions, -6 deletions
* chore(release): v2.0.8 "Pulse"
Bundled release: Reasoning Theater wired to the 8-stage deep_reference
cognitive pipeline, Pulse InsightToast, Memory Birth Ritual (v2.3
Terrarium), 7 new dashboard surfaces (/duplicates, /dreams, /schedule,
/importance, /activation, /contradictions, /patterns), 3D graph
brightness system with auto distance-compensation + user slider, and
contradiction-detection + primary-selection hardening in the
cross_reference tool. Intel Mac (x86_64-apple-darwin) also flows through
to the release matrix from PR #43.
Added:
- POST /api/deep_reference — HTTP surface for the 8-stage pipeline
- DeepReferenceCompleted WebSocket event (primary + supporting +
contradicting memory IDs for downstream graph animation)
- /reasoning route, full UI + Cmd+K Ask palette
- 7 new dashboard surfaces exposing the cognitive engine
- Graph brightness slider + localStorage persistence + distance-based
emissive compensation so nodes don't disappear into fog at zoom-out
Fixed:
- Contradiction-detection false positives: adjacent-domain memories no
longer flagged as conflicts (NEGATION_PAIRS wildcards removed,
shared-words floor 2 → 4, topic-sim floor 0.15 → 0.55, STAGE 5
overlap floor 0.15 → 0.4)
- Primary-memory selection: unified composite 0.5 × relevance + 0.2 ×
trust + 0.3 × term_presence with hard topic-term filter, closing the
class of bug where off-topic high-trust memories won queries about
specific subjects
- Graph default-load fallback from sort=recent to sort=connected when
the newest memory is isolated, both backend and client
Changed:
- Reasoning page information hierarchy: chain renders first as hero,
confidence meter + Primary Source citation footer below
- Cargo feature split: embeddings code-only + ort-download | ort-dynamic
backends; defaults preserve identical behavior for existing consumers
- CI release-build now gates PRs too so multi-platform regressions
surface pre-merge
362 lines
18 KiB
Markdown
362 lines
18 KiB
Markdown
# 🚫 READ FIRST EVERY PROMPT — NO HEDGING
|
||
|
||
Sam's April 20 2026 correction: *"you NEVER LISTEN TO YOUR RULES, WHY ARE YOU ALWAYS BREAKING THE HEDGING/PESSIMISTIC RULE."*
|
||
|
||
When Sam commits to an execution path (content cadence, shipping velocity, outreach, launch), COMPUTE THE FORECAST AS IF THAT COMMITMENT IS MET. Do not discount by execution probability. Sam has earned the benefit of the doubt across 139 commits and 1,292 passing tests in 3 months.
|
||
|
||
**Forbidden in any response**: "has to be true", "has to convert", "realistic forecast" used to shrink, "contingent on" Sam's own execution, "not guaranteed", "may or may not", "aspiration not forecast", "temper your expectations", floors framed as reality vs. ceilings framed as fantasy.
|
||
|
||
**Required format for forecasts**: `Target X. Floor Y. Ceiling Z. Math: [line items]. Risks: [separate section]. Let's go.`
|
||
|
||
Full rule: `/Users/entity002/CLAUDE.md` (top banner) + `/Users/entity002/.claude/projects/-Users-entity002/memory/feedback_always_positive_energy.md`. Stop-hook enforcement: `/Users/entity002/.claude/hooks/synthesis-stop-validator.sh`.
|
||
|
||
---
|
||
|
||
# Vestige v2.0.4 — Cognitive Memory & Reasoning System
|
||
|
||
Vestige is your long-term memory AND reasoning engine. 29 stateful cognitive modules implement real neuroscience: FSRS-6 spaced repetition, synaptic tagging, prediction error gating, hippocampal indexing, spreading activation, reconsolidation, and dual-strength memory theory. **Use it automatically. Use it aggressively.**
|
||
|
||
**NEW: `deep_reference` — call this for ALL factual questions.** It doesn't just retrieve — it REASONS across memories with FSRS-6 trust scoring, intent classification, contradiction analysis, and generates a pre-built reasoning chain. Read the `reasoning` field FIRST.
|
||
|
||
---
|
||
|
||
## Session Start Protocol
|
||
|
||
Every conversation, before responding to the user:
|
||
|
||
```
|
||
session_context({
|
||
queries: ["user preferences", "[current project] context"],
|
||
context: { codebase: "[project]", topics: ["[current topics]"] },
|
||
token_budget: 2000
|
||
})
|
||
```
|
||
|
||
Then check `automationTriggers` from response:
|
||
- `needsDream` → call `dream` (consolidates memories, discovers hidden connections)
|
||
- `needsBackup` → call `backup`
|
||
- `needsGc` → call `gc(dry_run: true)` then review
|
||
- totalMemories > 700 → call `find_duplicates`
|
||
|
||
Say "Remembering..." then retrieve context before answering.
|
||
|
||
> **Fallback:** If `session_context` unavailable: `search` × 2 → `intention` check → `system_status` → `predict`.
|
||
|
||
---
|
||
|
||
## Complete Tool Reference (23 Tools)
|
||
|
||
### session_context — One-Call Initialization
|
||
```
|
||
session_context({
|
||
queries: ["user preferences", "project context"], // search queries
|
||
context: { codebase: "project-name", topics: ["svelte", "rust"], file: "src/main.rs" },
|
||
token_budget: 2000, // 100-100000, controls response size
|
||
include_status: true, // system health
|
||
include_intentions: true, // triggered reminders
|
||
include_predictions: true // proactive memory predictions
|
||
})
|
||
```
|
||
Returns: markdown context + `automationTriggers` + `expandable` IDs for on-demand retrieval.
|
||
|
||
### smart_ingest — Save Anything
|
||
**Single mode** — auto-decides CREATE/UPDATE/SUPERSEDE via Prediction Error Gating:
|
||
```
|
||
smart_ingest({
|
||
content: "What to remember",
|
||
tags: ["tag1", "tag2"],
|
||
node_type: "fact", // fact|concept|event|person|place|note|pattern|decision
|
||
source: "optional reference",
|
||
forceCreate: false // bypass dedup when needed
|
||
})
|
||
```
|
||
**Batch mode** — save up to 20 items in one call (session end, pre-compaction):
|
||
```
|
||
smart_ingest({
|
||
items: [
|
||
{ content: "Item 1", tags: ["session-end"], node_type: "fact" },
|
||
{ content: "Item 2", tags: ["bug-fix"], node_type: "fact" }
|
||
]
|
||
})
|
||
```
|
||
Each item runs the full cognitive pipeline: importance scoring → intent detection → synaptic tagging → hippocampal indexing → PE gating → cross-project recording.
|
||
|
||
### search — 7-Stage Cognitive Search
|
||
```
|
||
search({
|
||
query: "search query",
|
||
limit: 10, // 1-100
|
||
min_retention: 0.0, // filter by retention strength
|
||
min_similarity: 0.5, // minimum cosine similarity
|
||
detail_level: "summary", // brief|summary|full
|
||
context_topics: ["rust", "debugging"], // boost topic-matching memories
|
||
token_budget: 3000, // 100-100000, truncate to fit
|
||
retrieval_mode: "balanced" // precise|balanced|exhaustive (v2.1)
|
||
})
|
||
```
|
||
Retrieval modes: `precise` (fast, no activation/competition), `balanced` (default 7-stage pipeline), `exhaustive` (5x overfetch, deep graph traversal, no competition suppression).
|
||
|
||
Pipeline: Overfetch → Rerank (cross-encoder) → Temporal boost → Accessibility filter (FSRS-6) → Context match (Tulving 1973) → Competition (Anderson 1994) → Spreading activation. **Every search strengthens the memories it finds (Testing Effect).**
|
||
|
||
### memory — Read, Edit, Delete, Promote, Demote
|
||
```
|
||
memory({ action: "get", id: "uuid" }) // full node with all FSRS state
|
||
memory({ action: "edit", id: "uuid", content: "updated text" }) // preserves FSRS state, regenerates embedding
|
||
memory({ action: "delete", id: "uuid" })
|
||
memory({ action: "promote", id: "uuid", reason: "was helpful" }) // +0.20 retrieval, +0.10 retention, 1.5x stability
|
||
memory({ action: "demote", id: "uuid", reason: "was wrong" }) // -0.30 retrieval, -0.15 retention, 0.5x stability
|
||
memory({ action: "state", id: "uuid" }) // Active/Dormant/Silent/Unavailable + accessibility score
|
||
memory({ action: "get_batch", ids: ["uuid1", "uuid2", "uuid3"] }) // retrieve up to 20 full memories at once (v2.1)
|
||
```
|
||
Promote/demote does NOT delete — it adjusts ranking. Demoted memories rank lower; alternatives surface instead.
|
||
`get_batch` is designed for batch retrieval of expandable overflow IDs from search/session_context.
|
||
|
||
### codebase — Code Patterns & Architectural Decisions
|
||
```
|
||
codebase({ action: "remember_pattern", name: "Pattern Name",
|
||
description: "How it works and when to use it",
|
||
files: ["src/file.rs"], codebase: "project-name" })
|
||
|
||
codebase({ action: "remember_decision", decision: "What was decided",
|
||
rationale: "Why", alternatives: ["Option A", "Option B"],
|
||
files: ["src/file.rs"], codebase: "project-name" })
|
||
|
||
codebase({ action: "get_context", codebase: "project-name", limit: 10 })
|
||
// Returns: patterns, decisions, cross-project insights
|
||
```
|
||
|
||
### intention — Prospective Memory (Reminders)
|
||
```
|
||
intention({ action: "set", description: "What to do",
|
||
trigger: { type: "context", topic: "authentication" }, // fires when discussing auth
|
||
priority: "high" })
|
||
|
||
intention({ action: "set", description: "Deploy by Friday",
|
||
trigger: { type: "time", at: "2026-03-07T17:00:00Z" },
|
||
deadline: "2026-03-07T17:00:00Z" })
|
||
|
||
intention({ action: "set", description: "Check test coverage",
|
||
trigger: { type: "context", codebase: "vestige", file_pattern: "*.test.*" } })
|
||
|
||
intention({ action: "check", context: { codebase: "vestige", topics: ["testing"] } })
|
||
intention({ action: "update", id: "uuid", status: "complete" })
|
||
intention({ action: "list", filter_status: "active" })
|
||
```
|
||
|
||
### dream — Memory Consolidation
|
||
```
|
||
dream({ memory_count: 50 })
|
||
```
|
||
5-stage cycle: Replay → Cross-reference → Strengthen → Prune → Transfer. Uses Waking SWR tagging (70% tagged + 30% random for diversity). Discovers hidden connections, generates insights, persists new edges to the activation network.
|
||
|
||
### explore_connections — Graph Traversal
|
||
```
|
||
explore_connections({ action: "associations", from: "uuid", limit: 10 })
|
||
// Spreading activation from a memory — find related memories via graph traversal
|
||
|
||
explore_connections({ action: "chain", from: "uuid-A", to: "uuid-B" })
|
||
// Build reasoning path between two memories (A*-like pathfinding)
|
||
|
||
explore_connections({ action: "bridges", from: "uuid-A", to: "uuid-B" })
|
||
// Find connecting memories that bridge two concepts
|
||
```
|
||
|
||
### predict — Proactive Retrieval
|
||
```
|
||
predict({ context: { codebase: "vestige", current_file: "src/main.rs",
|
||
current_topics: ["error handling", "rust"] } })
|
||
```
|
||
Returns: predictions with confidence, suggestions, speculative retrievals, top interests. Uses SpeculativeRetriever's learned patterns from access history.
|
||
|
||
### importance_score — Should I Save This?
|
||
```
|
||
importance_score({ content: "Content to evaluate",
|
||
context_topics: ["debugging"], project: "vestige" })
|
||
```
|
||
4-channel model: novelty (0.25), arousal (0.30), reward (0.25), attention (0.20). Composite > 0.6 = save it.
|
||
|
||
### find_duplicates — Dedup Memory
|
||
```
|
||
find_duplicates({ similarity_threshold: 0.80, limit: 20, tags: ["bug-fix"] })
|
||
```
|
||
Cosine similarity clustering. Returns merge/review suggestions.
|
||
|
||
### memory_timeline — Chronological Browse
|
||
```
|
||
memory_timeline({ start: "2026-02-01", end: "2026-03-01",
|
||
node_type: "decision", tags: ["vestige"], limit: 50, detail_level: "summary" })
|
||
```
|
||
|
||
### memory_changelog — Audit Trail
|
||
```
|
||
memory_changelog({ memory_id: "uuid", limit: 20 }) // per-memory history
|
||
memory_changelog({ start: "2026-03-01", limit: 20 }) // system-wide
|
||
```
|
||
|
||
### memory_health — Retention Dashboard
|
||
```
|
||
memory_health()
|
||
```
|
||
Returns: avg retention, distribution buckets (0-20%, 20-40%, etc.), trend (improving/declining/stable), recommendation.
|
||
|
||
### memory_graph — Visualization Export
|
||
```
|
||
memory_graph({ query: "search term", depth: 2, max_nodes: 50 })
|
||
memory_graph({ center_id: "uuid", depth: 3, max_nodes: 100 })
|
||
```
|
||
Returns nodes with force-directed positions + edges with weights.
|
||
|
||
### deep_reference — Cognitive Reasoning Engine (v2.0.4) ★ USE THIS FOR ALL FACTUAL QUESTIONS
|
||
```
|
||
deep_reference({ query: "What port does the dev server use?" })
|
||
deep_reference({ query: "Should I use prefix caching with vLLM?", depth: 30 })
|
||
```
|
||
**THE killer tool.** 8-stage cognitive reasoning pipeline:
|
||
1. Broad retrieval + cross-encoder reranking
|
||
2. Spreading activation expansion (finds connected memories search misses)
|
||
3. FSRS-6 trust scoring (retention × stability × reps ÷ lapses)
|
||
4. Intent classification (FactCheck / Timeline / RootCause / Comparison / Synthesis)
|
||
5. Temporal supersession (newer high-trust replaces older)
|
||
6. Trust-weighted contradiction analysis (only flags conflicts between strong memories)
|
||
7. Relation assessment (Supports / Contradicts / Supersedes / Irrelevant per pair)
|
||
8. **Template reasoning chain** — pre-built natural language reasoning the AI validates
|
||
|
||
Parameters: `query` (required), `depth` (5-50, default 20).
|
||
|
||
Returns: `intent`, `reasoning` (THE KEY FIELD — read this first), `recommended` (highest-trust answer), `evidence` (trust-sorted), `contradictions`, `superseded`, `evolution`, `related_insights`, `confidence`.
|
||
|
||
`cross_reference` is a backward-compatible alias that calls `deep_reference`.
|
||
|
||
### Maintenance Tools
|
||
```
|
||
system_status() // health + stats + warnings + recommendations
|
||
consolidate() // FSRS-6 decay cycle + embedding generation
|
||
backup() // SQLite backup → ~/.vestige/backups/
|
||
export({ format: "json", tags: ["bug-fix"], since: "2026-01-01" })
|
||
gc({ min_retention: 0.1, dry_run: true }) // garbage collect (dry_run first!)
|
||
restore({ path: "/path/to/backup.json" })
|
||
```
|
||
|
||
---
|
||
|
||
## Mandatory Save Gates
|
||
|
||
**You MUST NOT proceed past a save gate without executing the save.**
|
||
|
||
| Gate | Trigger | Action |
|
||
|------|---------|--------|
|
||
| **BUG_FIX** | After any error is resolved | `smart_ingest({ content: "BUG FIX: [error]\nRoot cause: [why]\nSolution: [fix]\nFiles: [paths]", tags: ["bug-fix", "project"], node_type: "fact" })` |
|
||
| **DECISION** | After any architectural/design choice | `codebase({ action: "remember_decision", decision, rationale, alternatives, files, codebase })` |
|
||
| **CODE_CHANGE** | After >20 lines or new pattern | `codebase({ action: "remember_pattern", name, description, files, codebase })` |
|
||
| **SESSION_END** | Before stopping or compaction | `smart_ingest({ items: [{ content: "SESSION: [summary]", tags: ["session-end"] }] })` |
|
||
|
||
---
|
||
|
||
## Trigger Words — Auto-Save
|
||
|
||
| User Says | Action |
|
||
|-----------|--------|
|
||
| "Remember this" / "Don't forget" | `smart_ingest` immediately |
|
||
| "I always..." / "I never..." / "I prefer..." | Save as preference |
|
||
| "This is important" | `smart_ingest` + `memory(action="promote")` |
|
||
| "Remind me..." / "Next time..." | `intention({ action: "set" })` |
|
||
|
||
---
|
||
|
||
## Cognitive Architecture
|
||
|
||
### Search Pipeline (7 stages)
|
||
1. **Overfetch** — 3x results from hybrid search (0.3 BM25 + 0.7 semantic, nomic-embed-text-v1.5 768D)
|
||
2. **Rerank** — Cross-encoder rescoring (Jina Reranker v1 Turbo, 38M params)
|
||
3. **Temporal** — Recency + validity window boosting (85% relevance + 15% temporal)
|
||
4. **Accessibility** — FSRS-6 retention filter (Active ≥0.7, Dormant ≥0.4, Silent ≥0.1)
|
||
5. **Context** — Tulving 1973 encoding specificity (topic overlap → +30% boost)
|
||
6. **Competition** — Anderson 1994 retrieval-induced forgetting (winners strengthen, competitors weaken)
|
||
7. **Activation** — Spreading activation side effects + predictive model + reconsolidation marking
|
||
|
||
### Ingest Pipeline
|
||
**Pre:** 4-channel importance scoring (novelty/arousal/reward/attention) + intent detection → auto-tag
|
||
**Store:** Prediction Error Gating: similarity >0.92 → UPDATE, 0.75-0.92 → UPDATE/SUPERSEDE, <0.75 → CREATE
|
||
**Post:** Synaptic tagging (Frey & Morris 1997, 9h backward + 2h forward) + hippocampal indexing + cross-project recording
|
||
|
||
### FSRS-6 (State-of-the-Art Spaced Repetition)
|
||
- Retrievability: `R = (1 + factor × t / S)^(-w20)` — 21 trained parameters
|
||
- Dual-strength model (Bjork & Bjork 1992): storage strength (grows) + retrieval strength (decays)
|
||
- Accessibility = retention×0.5 + retrieval×0.3 + storage×0.2
|
||
- 20-30% more efficient than SM-2 (Anki)
|
||
|
||
### 29 Cognitive Modules (stateful, persist across calls)
|
||
|
||
**Neuroscience (16):**
|
||
ActivationNetwork (Collins & Loftus 1975), SynapticTaggingSystem (Frey & Morris 1997), HippocampalIndex (Teyler & Rudy 2007), ContextMatcher (Tulving 1973), AccessibilityCalculator, CompetitionManager (Anderson 1994), StateUpdateService, ImportanceSignals, NoveltySignal, ArousalSignal, RewardSignal, AttentionSignal, EmotionalMemory (Brown & Kulik 1977), PredictiveMemory, ProspectiveMemory, IntentionParser
|
||
|
||
**Advanced (11):**
|
||
ImportanceTracker, ReconsolidationManager (Nader — 5min labile window), IntentDetector (9 intent types), ActivityTracker, MemoryDreamer (5-stage consolidation), MemoryChainBuilder (A*-like), MemoryCompressor (30-day min age), CrossProjectLearner (6 pattern types), AdaptiveEmbedder, SpeculativeRetriever (6 trigger types), ConsolidationScheduler
|
||
|
||
**Search (2):** Reranker, TemporalSearcher
|
||
|
||
### Memory States
|
||
- **Active** (retention ≥ 0.7) — easily retrievable
|
||
- **Dormant** (≥ 0.4) — retrievable with effort
|
||
- **Silent** (≥ 0.1) — difficult, needs cues
|
||
- **Unavailable** (< 0.1) — needs reinforcement
|
||
|
||
### Connection Types
|
||
semantic, temporal, causal, spatial, part_of, user_defined — each with strength (0-1), activation_count, timestamps
|
||
|
||
---
|
||
|
||
## Advanced Techniques
|
||
|
||
### Cross-Project Intelligence
|
||
The CrossProjectLearner tracks patterns across ALL projects (ErrorHandling, AsyncConcurrency, Testing, Architecture, Performance, Security). When you learn a pattern in one project that works, it becomes available in all projects. Use `codebase({ action: "get_context" })` without a codebase filter to get universal patterns.
|
||
|
||
### Reconsolidation Window
|
||
After any memory is accessed (via search, get, or promote), it enters a 5-minute "labile" state where modifications are enhanced. This is the optimal time to edit memories with new context. The system handles this automatically.
|
||
|
||
### Synaptic Tagging (Retroactive Importance)
|
||
Memories encoded in the last 9 hours can be retroactively promoted when something important happens. If you fix a critical bug, not only does the fix get saved — related memories from the past 9 hours also get importance boosts. The SynapticTaggingSystem handles this automatically.
|
||
|
||
### Dream Insights
|
||
Dreams don't just consolidate — they generate new insights by cross-referencing recent memories with older knowledge. The insights can reveal: contradictions between memories, previously unseen patterns, connections across different projects. Always check dream results for `insights_generated`.
|
||
|
||
### Token Budget Strategy
|
||
Use `token_budget` on search and session_context to control response size. For quick lookups: 500. For deep context: 3000-5000. Results that don't fit go to `expandable` — retrieve them with `memory({ action: "get", id: "..." })`.
|
||
|
||
### Detail Levels
|
||
- `brief` — id/type/tags/score only (1-2 tokens per result, good for scanning)
|
||
- `summary` — 8 fields including content preview (default, balanced)
|
||
- `full` — all FSRS state, timestamps, embedding info (for debugging/analysis)
|
||
|
||
---
|
||
|
||
## Memory Hygiene
|
||
|
||
**Promote** when user confirms helpful, solution worked, info was accurate.
|
||
**Demote** when user corrects mistake, info was wrong, led to bad outcome.
|
||
**Never save:** secrets, API keys, passwords, temporary debugging state, trivial info.
|
||
|
||
---
|
||
|
||
## The One Rule
|
||
|
||
**When in doubt, save. The cost of a duplicate is near zero (Prediction Error Gating handles dedup). The cost of lost knowledge is permanent.**
|
||
|
||
Memory is retrieval. Searching strengthens memory. Search liberally, save aggressively.
|
||
|
||
---
|
||
|
||
## Development
|
||
|
||
- **Crate:** `vestige-mcp` v2.0.4, Rust 2024 edition, MSRV 1.91
|
||
- **Tests:** 758 (406 mcp + 352 core), zero warnings
|
||
- **Build:** `cargo build --release -p vestige-mcp` (features: `embeddings` + `vector-search`)
|
||
- **Build (no embeddings):** `cargo build --release -p vestige-mcp --no-default-features`
|
||
- **Bench:** `cargo bench -p vestige-core`
|
||
- **Architecture:** `McpServer` → `Arc<Storage>` + `Arc<Mutex<CognitiveEngine>>`
|
||
- **Storage:** SQLite WAL mode, `Mutex<Connection>` reader/writer split, FTS5 full-text search
|
||
- **Embeddings:** nomic-embed-text-v1.5 (768D, 8K context) via fastembed (local ONNX, no API)
|
||
- **Vector index:** USearch HNSW (20x faster than FAISS)
|
||
- **Binaries:** `vestige-mcp` (MCP server), `vestige` (CLI), `vestige-restore`
|
||
- **Dashboard:** SvelteKit 2 + Svelte 5 + Three.js + Tailwind 4, embedded at `/dashboard`
|
||
- **Env vars:** `VESTIGE_DASHBOARD_PORT` (default 3927), `VESTIGE_HTTP_PORT` (default 3928), `VESTIGE_HTTP_BIND` (default 127.0.0.1), `VESTIGE_AUTH_TOKEN` (auto-generated), `VESTIGE_CONSOLIDATION_INTERVAL_HOURS` (default 6), `RUST_LOG`
|