A second adversarial audit of shipped main (post the 29+3 earlier fixes) found
10 more real bugs in areas the first pass under-covered (FTS sanitizer,
protocol/HTTP, connectors, frontend). None regressions — all pre-existing, and
several are "twin sites" of a class fixed elsewhere but missed here.
Majors:
- FTS5: sanitize_fts5_terms leaked bare operators on doubled input ("foo AND AND
AND"), aborting MATCH with a syntax error or silently flipping AND->OR. Now
filters operator tokens (repetition-proof).
- Server crash (DoS): unbounded MCP `hours_back`/`hours_forward` panicked via
Duration::minutes in CaptureWindow; unclamped `limit` in match_context
panicked via negative Vec::with_capacity / limit*2 overflow. Both now
bounded/checked.
- Connectors: the source idempotency key omitted source_project, so two repos
(or two Redmine instances) with overlapping ids clobbered each other's memory.
Key + unique index (migration V19) now include source_project.
Minors/nits:
- list_memories now honors node_type/tag on the search path (were dropped).
- HTTP Accept honors */* and type wildcards (was hard 406 for curl et al.).
- Bearer scheme matched case-insensitively (RFC 7235).
- Dream insight time-span no longer seeds from now/-365d sentinels (fabricated a
TemporalTrend for old clusters).
- WebSocket disconnect() detaches onclose before close() so it can't resurrect
the socket via scheduleReconnect.
- Redmine thread truncation keeps the NEWEST journals (was keeping oldest, so
new activity never re-indexed).
Tests: FTS operator-leak regression added. Full workspace green (1559 tests,
clippy -D warnings clean); dashboard check + 937 tests green.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- #117 Agent Black Box: the trace recorder had zero production callers, so
agent_traces/receipts/memory_prs never populated. handle_tools_call now
records the opening mcp.call event (record_call) before dispatch and the
downstream memory events (record_result) after a successful call, under a
shared run_id. Also fixed the args capture so tracing works in pure stdio
mode (was gated on event_tx, which is None without a dashboard socket).
- #124 Recurring intentions now re-arm: mark_triggered advances a recurring
trigger's next_occurrence and returns the intention to Active (was firing
once and staying Triggered forever). Added Trigger::re_arm (recurses into
Compound).
- #137 Co-access prediction channel is now live: search_unified feeds the
retrieved set to speculative_retriever.record_usage, so co-access patterns
populate (were permanently empty). The temporal channel was already fixed
(#136); the file-context channel stays dormant by design (search has no file
context to supply).
Tests: Black Box trace population, recurring re-arm. Full workspace green
(1558 tests, clippy -D warnings clean).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A backend-wide audit (18 feature areas, adversarially verified) surfaced 31
real bugs. This fixes 29 of them; each was re-verified by an independent
adversarial pass and the whole workspace stays green (1556 tests, clippy -D
warnings clean).
Data integrity & storage:
- Migrations no longer brick the DB on a mid-migration failure: each runs in a
transaction (atomic schema change + version bump); V7 repages under WAL.
- decide_memory_pr rejects re-deciding a finalized PR (audit ledger + no
resurrection of a rejected memory).
- suppress reverse is now a true inverse (was leaving stability permanently
halved).
- update_node_content flips has_embedding=0 when the embedder isn't ready, so
the stale embedding is regenerated instead of matching old content forever.
- plan_merge validates survivor_id is a member (was an unchecked-unwrap panic).
- Redmine detail-fetch failure aborts the page (retry) instead of storing a
journal-less record and skipping it forever.
- ContentStore cache counter accounts for replaced entries; changelog no longer
loads the whole connections table; source_sync clamps max_pages.
Cognitive correctness:
- Contradiction heuristic requires a real polarity flip (was demoting correct
memories on benign "do not"/"cannot" notes).
- Hybrid search relevance is the RRF fused score, min-max normalized, so the
best match ranks first (was capped and outranked by recency/importance).
- Sentiment-boosted stability is clamped to MAX_STABILITY.
- Chain confidence uses the correct geometric-mean root; looks_like_failure no
longer over-fires on "$500"; contains_marker_word is UTF-8-boundary-safe.
- Prediction merge can't cross min_confidence from below; temporal prediction
wraps around midnight; match_context tag overlap is clamped to 1.0.
MCP / CLI:
- UTF-8 panic in the " at " intention parser fixed (boundary-safe match).
- vestige backup no longer fails on encrypted DBs (best-effort raw checkpoint).
- --ago-days uses checked Duration + subtraction (no overflow panic).
- smart_ingest honors an explicit forceCreate:false (was silently inverted).
- append_trace_event propagates the seq query error (no duplicate seq=0).
- RewardPattern::matches requires real overlap (was matching everything).
Deferred: the dead trace-recorder wiring (Black Box/Receipts/Memory-PR) is a
feature-completion task for its own change; two dormant speculative-prediction
channels with no production caller are left as-is (fixing dead code adds risk
without user benefit).
Regression tests added for the migration brick, suppress inverse, backfill
bound, contradiction false-positive, and forceCreate handling.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* patch(backfill-safety): VESTIGE_BACKFILL_AUTOFIRE gate (default OFF) + bounded promote_memory_backfill
Off-by-default env gate around step-8.5 auto-fire in run_consolidation (decouples
backfill from consolidation cadence). New promote_memory_backfill caps stability at
MIN(stability*1.5, stability+365.0) (the bound retroactive_backfill.rs:300 already
computes but discarded); both backfill entry points use it. Fixes the false 'capped'
comment. Cloud-sync excluded at build (--no-default-features). Pending upstream PR to
samvallad33/vestige. omega-backfill-safety-v2.2.0 off tag v2.2.0 (3bcd4667).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* release: v2.2.1 — Windows embeddings fix + backfill safety + first-run guide
Board-clearing patch release.
Fixes:
- #101 Windows embeddings: release.yml already restores vector-search on the
x86_64-pc-windows-msvc target (merged in #102); this release rebuilds the
Windows binary so users actually get working embeddings.
- #103 Retroactive Salience Backfill safety (from community PR #104, adjusted):
* promote_memory_backfill bounds the stability multiply to
MIN(stability*1.5, stability+365.0) on both auto-fire and manual paths.
* VESTIGE_BACKFILL_AUTOFIRE gate — default ON (preserves the shipped/documented
v2.2.0 behavior), disable with 0/false/off/no. Env value is trimmed.
* Corrected the false "capped" comment and the promote_memory_backfill doc.
* Added 3 tests: +365 cap binds, *1.5 multiply below crossover, gate parsing.
Docs:
- #83 First-Run: new docs/GETTING-STARTED.md + README pointer.
- Consolidated roadmap issues #82,#84-#92 into docs/ROADMAP.md (Tracked Issues).
- Documented VESTIGE_BACKFILL_AUTOFIRE in docs/CONFIGURATION.md.
- CHANGELOG v2.2.1 entry.
Version bumped to 2.2.1 across all manifests + Cargo.lock + dashboard build.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Peter Lauzon <inbijiburu@protonmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Second swarm pass (complete every-line sweep), verified against real code
(38/101 confirmed real; 63 false positives excluded). This commit lands the
main-compatible subset:
- redmine SSRF guard: rewrite host check to use host_str()+std::net::IpAddr
instead of the `url` crate (url is not a direct dep of vestige-core on main;
the previous form only compiled on the feature branch). Same protection:
blocks localhost + loopback/private/link-local/unspecified IPs.
- bin/restore: guard wrapper[0] index (empty backup array would panic)
- bin/cli: char-boundary-safe node.id truncation (2 byte-slice panics);
XML-escape the model/home strings before launchd plist substitution
- prospective_memory: Duration::try_hours/try_days (panic on out-of-range
user config); case-insensitive " at " split now uses the lowercased index
so the contains() check and the split agree
core 477/0, mcp builds, clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tool Consolidation v2.2.0 (34->12 advertised surface: recall/memory/codebase/
intention/smart_ingest/source_sync/memory_status/maintain/dedup/graph/
session_start/suppress) onto the retroactive-salience-backfill base.
Conflict resolution:
- server.rs: took #99's canonical 12-tool surface, then re-added
as a 13th ADVERTISED tool (ToolDescription + dispatch arm + test). Backfill
is the flagship 'Memory with hindsight' (Cai 2024 Nature) cognitive primitive
— a distinct capability, not a maintenance op to fold into `maintain` or hide.
- Cargo.toml: kept usearch features=["fp16lib"] with the fuller MSVC-C1021
explanation comment (Windows build fix, #71).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire Step 8.5 into run_consolidation (bounded + idempotent via durable
Causal edge), share looks_like_failure/extract_entities across MCP tool +
CLI + auto-fire, add whole-word failure markers (API_TIMEOUT no longer
false-fires), --json CLI for CauseBench, and refactor the Redmine SSRF
guard to std::net (drops the dangling url-crate dep).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds docs/launch/tool-consolidation-v2.2.0.md — the single sequenced plan that
reconciles the two prior planning notes:
- Layer 1 (this PR): 34 → 12 advertised tools, safe commit order, alias policy,
preserved invariants, and the test that proves each.
- Layer 2 (follow-up): tiny always-on default surface + SessionStart/Stop hooks.
Also refreshes stale in-code comments to match the consolidated surface:
- server.rs handle_tools_list header (was "v2.1.21: 25 tools") and the
size-annotation rationale (now lists recall/memory_status/dedup/graph).
- tools/mod.rs module doc (the facade vs. granular-handler relationship).
No behavior change. Gates: cargo test --workspace, cargo clippy -D warnings,
pnpm dashboard check + build — all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tool Consolidation v2.2.0, Layer 1 commit 5/6. Advertised tools 21 → 15.
Folds consolidate + dream + gc + importance_score + backup + export + restore
into one action-dispatched tool:
action = consolidate | dream | gc | importance_score | backup | export | restore
- Thin facade forwards the same args envelope to each existing handler; the
`action` discriminator is cloned out before the envelope is moved into a
callee. No handler uses deny_unknown_fields, so per-action params validate.
- Safety defaults preserved (all handler-internal): gc dry_run=true by default,
restore path-confinement, export traversal guard. Verified by
test_maintain_actions_and_safety (gc dry-run + restore missing-path error).
- Events preserved end-to-end:
- Pre-dispatch Started events (ConsolidationStarted/DreamStarted) re-emitted
in the `maintain` arm keyed on action.
- emit_tool_event normalizes the `maintain` name to its effective action, so
the existing ConsolidationCompleted/DreamCompleted/ImportanceScored arms
fire unchanged — no duplicated emit logic.
- All 7 old names remain hidden warn!+redirect aliases (removed v2.3.0), keeping
their own pre-emits.
- Tests: count 21→15, 7 negatives, new dispatch/safety test.
Gates: cargo test --workspace, cargo clippy -D warnings — clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tool Consolidation v2.2.0, Layer 1 commit 4/6 (3b). Advertised tools 24 → 21.
Folds explore_connections + predict + memory_graph + composed_graph into one
action-dispatched tool:
action ∈ {chain, associations, bridges, predict, memory_graph,
recent, get, memory, neighbors, never_composed, bounty_mode, label}
- Transparent facade: each action forwards the same args envelope to the
existing handler, which re-reads its own discriminator/params. No underlying
arg struct uses deny_unknown_fields, so cross-fields are ignored.
- All actions read-only except `label` (the one mutator), which is logged for
audit via tracing::info!(event_id, outcome_type).
- outcome_type enum sourced from composed_graph::OUTCOME_TYPES (now pub(crate))
rather than re-listed, so the vocabulary stays single-sourced.
- All 4 old names remain hidden warn!+redirect aliases (removed v2.3.0).
- Size annotation: graph=250_000 (memory_graph layout + bounty_mode pagination),
kept in sync across loop, helper, and both annotation tests.
- Tests: count 24→21, 4 negatives, test_graph_actions_and_aliases exercising
read-only actions + aliases (incl. the memory_graph facade branch).
Note: the design draft mislabeled graph::execute as sync; it is `pub async fn`.
The per-commit build gate caught it — the facade awaits it correctly.
Gates: cargo test --workspace, cargo clippy -D warnings — clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tool Consolidation v2.2.0, Layer 1 commit 3/6 (3a). Advertised tools 27 → 24.
Folds system_status + memory_health + memory_timeline + memory_changelog
into one view-dispatched tool:
view = health (default) | retention | timeline | changelog
- Thin facade: each view forwards the same args envelope to the existing
handler. No underlying arg struct uses deny_unknown_fields, so the `view`
discriminator is ignored by each handler and per-view fields validate as
before. The cognitive lock is never held across a forwarded call.
- view='health' returns the byte-for-byte system_status shape (audit scripts
parse it), incl. schema_introspection passthrough — verified by
test_default_view_is_health asserting equality with execute_system_status.
- All 4 old names remain hidden warn!+redirect aliases (removed v2.3.0).
- Size annotation moved: memory_timeline (200k) → memory_status, kept in sync
across the real loop, expected_max_result_size(), and both annotation tests.
- Tests: count 27→24, 4 negative asserts, test_memory_status_views_and_aliases
exercising all 4 views + 4 aliases.
Gates: cargo test --workspace, cargo clippy -D warnings — clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tool Consolidation v2.2.0, Layer 1 commit 2/6. Advertised count unchanged
at 27 (pure rename).
`session_start` is the imperative-outcome name for the one-call session
initializer. `session_context` remains a hidden warn!+redirect alias
(≥1 minor release, removed v2.3.0), calling the same handler unchanged.
Tests: positive assert swapped to session_start + negative assert for the
old name. Gates: cargo test --workspace, cargo clippy -D warnings — clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
demo/README.md: the complete self-serve demo artifact — one-command run, the
seeded scenario explained, a "build your own scenario" section, the honest
boundary (won't invent a cause; can't reach a cause that was never recorded),
the Nature citation + the "field admits this is unsolved" sources, and the
recording playbook + paste-ready caption.
Writing/testing the README surfaced a real inconsistency, now fixed:
- The CLI's failure-finder used a hardcoded content-only marker subset and
ignored tags, so a "Checkout latency spiked" memory (regression tag, no crash
word in content) was never picked as the failure. The CLI now calls the SAME
public `looks_like_failure` (content + tags, full list) the backfill tool uses
— one definition, no drift.
- Extended FAILURE_MARKERS with performance/degradation failures (spiked,
latency, degraded, slow, hang, throttled, oom, 502/503/504, flaky, ...) so the
feature backfills from perf regressions, not just hard crashes.
clippy clean; 527 core + 453 mcp tests; both the main demo and the README's
custom scenario verified end-to-end.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3-model rotation audit (DeepSeek V4-Pro / Kimi K2.7 / MiniMax M3, max thinking,
each model × each of 3 sections). Claude verified every finding against code.
CONFIRMED + FIXED:
- [FTS, consensus DeepSeek+MiniMax] sanitize_fts5_or_query split on
!is_alphanumeric()+'_', but the index uses tokenize='porter ascii' which
splits on '_' and non-ASCII. So "API_TIMEOUT"/"café" became single phrases that
could NEVER match. Now splits on !is_ascii_alphanumeric() + lowercases to mirror
the tokenizer; caps token count (64) and length (64) for DoS hardening. Also
fixes the pre-existing storage.search bug (multi-word queries silently returned
nothing). 5 new tests pin it.
- [Demo honesty, consensus Kimi+DeepSeek] the contrast labeled keyword search as
"SIMILARITY SEARCH" and asserted "NONE of these is the cause" universally. Now
prints the REAL engine ("keyword (BM25)" vs "semantic (vector + BM25 hybrid)")
and claims only what's true ("ranked by RESEMBLANCE; its top hit is a lookalike").
De-hardcoded the "Service crashed:" munging to a generic label-strip.
VERIFIED FALSE POSITIVE (not changed): MiniMax "fts.id non-existent column" —
the FTS5 table is declared `fts5(id, content, tags, ...)`, the JOIN is valid.
No injection found by any model (quote-doubling + operator-stripping confirmed safe).
clippy clean; 527 core + 453 mcp tests pass; demo verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CLI surface for Retroactive Salience Backfill so the seeded demo (and anyone who
clones it) can reproduce "memory with hindsight" from a terminal:
- `ingest --ago-days N`: backdate a memory N days (plant a dated cause/history).
- `backfill [--failure-id ID] [--manual] [--lookback-days N] [--no-promote]`:
reach backward from a failure and surface+promote the causal earlier memory,
with demo-grade colored output (↩ reached back N days, 🔗 causal join: <entity>,
similarity rank, ✅ promoted).
Verified live end-to-end on a real DB: plant a 3-day-old API_TIMEOUT env-var note
+ a semantically-similar 500 distractor + a crash, run `vestige backfill`, and it
surfaces the env-var note by the shared api_timeout entity (ignoring the similar
distractor) and promotes it. clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MCP surface for memory-with-hindsight. When a failure memory exists, the
`backfill` tool reaches backward across the real store and promotes the quiet
earlier cause that a vector search structurally cannot surface (not similar to
the failure, only causally upstream via a shared entity).
- tools/backfill.rs: builds BackfillCandidates from real KnowledgeNodes (entities
from tags + heuristic env-var/path/identifier extraction), computes real cosine
similarity from stored embeddings (to PROVE the cause ranks low on similarity),
runs the core RetroactiveBackfill, and promotes surfaced causes via
storage.promote_memory. Auto-finds the latest failure, or takes failure_id;
manual=true forces; promote=false for a dry run.
- registered + dispatched in server.rs (35 tools now); tool-list test updated.
- storage: added pub set_created_at (backdate created_at) so the demo/test can
plant a dated cause.
LIVE RECEIPT: live_backfill_surfaces_root_cause_through_storage ingests a
3-day-old API_TIMEOUT env-var note + a semantically-similar 500-error distractor
+ a crash into a REAL SQLite store, runs the backfill tool, and asserts it
surfaces + promotes the env-var note by the shared API_TIMEOUT entity (the root
cause RAG misses). clippy clean; 522 core + 453 mcp tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two deeper review findings (both blockers) + doc de-staling.
C2-deep: my earlier C2 made purge/delete TRACE as memory.write, but gate_writes
did `get_node(id) -> skip on None`, and purge had already DELETEd the row — so a
destructive removal still never opened a Memory PR (it was silently skipped).
The most security-critical write type couldn't be reviewed. Fix: a missing node
is now gateable for destructive decisions — gate_writes builds the WriteContext
from the decision itself (marks `forgets`, which classify_write gates), and the
PR records the removal with node.deleted=true. Proven live: purging a node opens
a PR (kind node_decayed, deleted true); test
gate_opens_pr_for_destructive_write_after_node_deleted_c2.
PRIV: gate_writes copied the FULL node.content into the PR diff + title, so a
real secret in a gated memory would leak into the memory_prs table, the
dashboard, and any exported proof bundle — defeating the point of gating
sensitive writes. Fix: the PR now stores a truncated content PREVIEW + an FNV
content HASH, and sensitive-topic/sensitive-node-type writes are fully REDACTED
("[redacted — sensitive content; review via risk signals]"). The reviewer still
sees the risk signals (why it opened) and a hash (to correlate), never the
secret. Tests gate_redacts_sensitive_content_in_pr_priv,
content_preview_redacts_sensitive_and_truncates, content_hash_is_stable. The
committed memory_pr.json + the whole proof bundle were re-captured and contain
no secret (verified by scan); the re-shot memory-prs.png shows the redaction.
DOC: REVIEW.md commit list is now git-log-based (no stale hashes); C2-deep + PRIV
added to the findings table; PROOF.md write/PR rows updated; test count -> 1007.
Gates: 1007 lib tests pass (+7 new regressions), clippy -D warnings clean,
dashboard check + build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two more review findings — both real, both blockers — plus stale-doc cleanup.
C1: the B1 release used reverse_suppression(subject_id, labile_hours), which
REFUSES once the 24h active-forgetting labile window has passed. So a Memory PR
reviewed late could be marked "promoted" while its memory stayed suppressed.
Approving a quarantined write is an explicit reviewer decision and must release
the memory regardless of elapsed time. New SqliteMemoryStore::release_quarantine
fully clears the suppression (count→0, suppressed_at→NULL) with NO time-window
limit; the PR handler now uses it. Proven: a test backdates suppressed_at to
+100h, shows reverse_suppression refuses, and release_quarantine still releases.
C2: memory(action="purge"|"delete") returns `action` + nodeId but those labels
weren't in is_write_decision, so destructive removal bypassed the memory.write
trace and the PR gate. Added purge/purged/delete/deleted/forget/forgotten.
Proven live: purging a node now records a second memory.write event
({"decision":"purge"}) under the run.
Docs: REVIEW.md de-staled — removed the fixed 140b15f diff-stat / "3 commits"
prose (it moved with each fix), listed all commits, added C1/C2 to the findings
table, updated the test count.
Gates: 1002 lib tests pass (+3 new regressions), clippy -D warnings clean,
dashboard check + build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A full multi-agent review found 7 real issues (4 blockers). All fixed + tested.
B1 (blocker): Promoting a Memory PR did not release the quarantined memory —
the UI said "promoted" while the memory stayed suppressed/out of retrieval.
act_on_memory_pr now calls reverse_suppression(subject_id) on accept actions;
MemoryPrAction::releases_memory() encodes the rule (promote/merge/supersede
release; forget/quarantine keep it held). Proven live: PR response
subjectReleased:true, SQLite suppression_count 0.
B2 (blocker): memory promote/demote (returns `action`, not `decision`) and
codebase remember_* writes bypassed the write-trace + PR gate. extract_writes
now reads `action` too, filtered by is_write_decision (reads like get/state
excluded); is_write_tool includes `codebase`.
B3 (blocker): receipt ids collided within a run (r_<date>_<runId> +
INSERT OR REPLACE overwrote earlier receipts). IDs are now
r_<date>_<runId8>_<unique6>; build() mints the suffix, build_with_unique()
keeps tests deterministic.
B4 (blocker): proof bundle was assembled from two runs (trace.json=run_proof,
websocket-events.jsonl=run_proof2). Re-captured the whole bundle from a single
run — trace, websocket, receipt, and memory_pr all carry run_proof now.
B5: Black Box receipts panel showed global latest, not the selected run.
Added list_receipts_for_run + /api/receipts?run= ; the page uses listForRun.
B6: SENSITIVE_TOPICS substring matching false-fired (tokenizer->token,
author->auth, secretary->secret). Switched to word-boundary matching; real
phrasings (auth token, security vulnerability, api key) still gate.
B7: set_review_mode now writes atomically (temp+rename via write_atomic);
export_trace sanitizes run_id in the Content-Disposition filename; memory-prs
static routes declared before the dynamic /{id} route.
Withdrawn: the /mode-vs-/{id} route order is NOT a functional bug (axum 0.8 /
matchit prioritizes static segments) — reordered for clarity only.
Gates: 999 lib tests pass (+9 new regressions), clippy -D warnings clean,
dashboard check + build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bounded follow-up (tight acceptance criteria, no scope expansion): flip the
dream.patch producer from "quiet because no dream ran" to a recorded live event.
The dream tool's `insights` array carries no per-item id, so the recorder
extracted zero proposals and dream.patch never fired even on a real dream.
Fix: derive a stable proposal id from each insight's REAL content (its
insight_type + the source memories it consolidated). The dream genuinely ran;
this just gives each real proposal a deterministic handle. Unit-tested against
the exact dream output shape.
Proven end to end (run_dream_proof, 6 memories consolidated):
- one dream.patch event: dream:RecurringPattern:5d941c7f+a41aca72+...
- SQLite + /api/traces/:runId: dream-trace.json (14 events, last is dream.patch)
- WebSocket: dream-websocket-events.jsonl (the dream.patch TraceEvent)
- dashboard: screenshots/dream-producers.png — the row flips to "fired this run"
PROOF.md updated: dream.patch moves from CAVEAT to REAL (still not live by
default — it fires only when a dream actually runs, and the UI says so).
sanhedrin.veto remains an honest CAVEAT (optional hook, off by default).
Gates: 957 lib tests pass, clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the receipt chain impossible to doubt. Freeze the claim surface, prove
every hop, and turn the two off-by-default producers into explicit UI states.
Frozen public claim: "Vestige records real MCP memory activity into a
replayable local trace, with receipts and reviewable risky writes." We do NOT
claim Sanhedrin vetoes or dream patches are live by default.
Regression — full-spine test (server.rs): one runId must cross, byte-identical,
MCP output -> SQLite trace -> WebSocket event -> API response shape ->
MCP resource. Fails if any hop drops or rewrites the id.
Honest UI states (Black Box "Event producers" panel):
- sanhedrin.veto -> "No veto producer connected (optional Sanhedrin hook, off
by default)" instead of empty mystery.
- dream.patch -> "No dream run in this trace" unless a dream actually ran.
- contradiction.detected -> "no contradiction in this run" when none fired.
Quarantine review (not pre-write blocking): risky writes are committed then
suppressed — audit history preserved, retrieval influence suspended until
reviewed. Reworded the server notice + UI copy to say exactly that.
Receipts UI gap closed: ReceiptCard is now mounted on the Black Box page
(retrieved/suppressed/trust-floor, activation path, "Open receipt in Cinema").
Proof pack (blackbox-proof-2026-06-22/): status.json, trace.json (the
.vestige-trace.json export), receipt.json, memory_pr.json (promoted via
UI->API->SQLite), websocket-events.jsonl (live TraceEvent x6 + PR opened/
decided), screenshots (Black Box, Receipts, Memory PRs, Graph), and PROOF.md
with real/caveat/stub per feature.
Gates: 988 lib tests pass, clippy -D warnings clean, dashboard check + build
clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The portable archive is encrypted on the client before upload and decrypted
after download, so the hosted service only ever stores ciphertext — true
zero-knowledge. The passphrase (VESTIGE_CLOUD_ENCRYPTION_KEY) is independent
of the bearer sync key and never leaves the device.
- new cloud_crypto module: Argon2id KDF + XChaCha20-Poly1305 AEAD, self-
describing envelope (MAGIC|version|salt|nonce|ciphertext+tag)
- HttpPortableSyncBackend encrypts on write / decrypts on read; transparent
upgrade of legacy plaintext archives; clear error if remote is encrypted
but no passphrase is set
- sync_portable_archive_cloud takes optional encryption_key
- CLI surfaces encryption status (on/off) on sync
- 6 crypto tests (roundtrip, wrong-key, tamper detection, non-determinism,
envelope detection); E2E verified: server blob is ciphertext, passphrase
device recovers, no-passphrase device cannot decrypt
491 core tests green, clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make Vestige a durable, local, semantically-searchable retrieval layer over an
external system of record (GitHub Issues first), citing back to the canonical
record. Unlike a live ticket-system MCP proxy, Vestige keeps a durable embedded
index: searchable offline, joinable with the rest of memory, temporally
versioned, and re-syncable idempotently with no duplication.
Phases 1-2 of #57 plus a GitHub reference connector and source-aware search:
- Source envelope on KnowledgeNode/IngestInput (source_system, source_id,
source_url, source_updated_at, content_hash, synced_at, source_project,
source_type, source_author). Migration V17: nullable columns (additive),
partial UNIQUE index on (source_system, source_id), connector_cursors table.
- Idempotent sync primitives in vestige-core: upsert_by_source (content-hash
change detection), connector cursor checkpoints, reconcile_source_tombstones
(invalidate-don't-delete via bitemporal valid_until).
- Connector contract + run_sync driver + GitHub Issues connector behind the
optional `connectors` feature (on by default in vestige-mcp, off in the core
library default so non-connector consumers link no HTTP client).
- source_sync MCP tool ({"repo": "owner/name"}); token from GITHUB_TOKEN env
only. Search results gain a sourceRecord citation for connector memories.
Adversarial review fixes: GitHub `since` Z-form (the `+00:00` offset corrupted
the cursor server-side), un-tombstone clears superseded_by too, cursor never
advances past a failing record, Link next-url host-pinned (token-leak guard),
records_seen counts new records only.
Verified: cargo check/test/clippy -D warnings green across the workspace
(default and connectors features); 483 core tests pass. Version bump to 2.1.27
and tag deferred to release.
Refs #57
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rebased on v2.1.25 merge/supersede and bumped the post-release metadata to v2.1.26 so this branch does not roll versions backward.
Adds local vestige.toml defaults, output profiles, and MCP response precedence for search, timeline, codebase context, and session context.
Verified:
- cargo metadata --format-version 1 --locked --no-deps
- cargo test -p vestige-core config --no-fail-fast
- cargo test -p vestige-mcp config --no-fail-fast
* feat(mcp/system_status): add optional schema_introspection flag
Adds an optional `schema_introspection: bool` parameter to the
`system_status` MCP tool. When set to true, the response gains a
`schema` block carrying:
- `schemaVersion` (u32) — highest applied migration, mirrors
`Storage::current_schema_version` (now exposed via a typed public
method).
- `schemaVersionAppliedAt` (RFC3339, optional) — timestamp the
current schema_version row was applied.
- `tables` ([{name, rows, columns}]) — per-table row count + column
list, walked over the canonical PORTABLE_USER_DATA_TABLES set so
the surface stays stable across migrations rather than enumerating
arbitrary sqlite_master rows.
- `embeddingNullCount` (i64) — count of knowledge_nodes with NO row
in node_embeddings. Distinct from MemoryStats.nodes_with_embeddings
(which keys off the `has_embedding` flag column), so audit scripts
can detect drift between the flag and the join-based truth.
- `activeEmbeddingModel` (string, optional) + `activeEmbeddingDimensions`
(u32, optional) — mirrors the existing MemoryStats active-model
fields, included here so audits get schema_version + active model
in a single round-trip.
Motivation: external consumers (audit scripts, migration guards,
downstream binary upgrade scripts) currently must read SQLite
directly to learn the schema shape, which couples them to internals
Vestige owns and breaks on every migration. This PR closes that gap
with a first-class MCP surface.
Implementation:
- New `pub fn schema_introspection() -> Result<SchemaIntrospection>`
inherent method on `Storage` (sqlite.rs). Inherent, not on a
trait — schema-walk is SQLite-specific by nature, so this stays
out of any future MemoryStore trait extraction.
- New typed structs `SchemaIntrospection` + `TableIntrospection` in
memory/mod.rs (canonical home alongside MemoryStats), re-exported
from the crate root.
- MCP layer (maintenance.rs) parses `SystemStatusArgs`, conditionally
extends the existing response object with a `schema` key — additive,
default off, response shape unchanged when omitted.
Coupling assessment vs PR #61 (storage-trait-phase1):
This PR adds ONE new public inherent method on `Storage` plus uses
three already-existing private helpers (`current_schema_version`,
`table_exists`, `table_row_count`, `table_columns`). It does NOT
touch the existing inherent method signatures, does NOT add anything
to the prospective `MemoryStore` trait surface, and does NOT modify
any of the ~25 methods #61 lifts into the trait. PR #61 is purely
additive on the trait surface (per its description, `pub type
Storage = SqliteMemoryStore;` preserves all existing call sites);
this PR is additive on the inherent surface. Two purely-additive
changes to disjoint surfaces should rebase cleanly.
Tests:
- system_status_schema_has_schema_introspection_flag (schema
introspection: property present, type=boolean, default=false,
not required)
- system_status_without_schema_flag_omits_schema_block
(backwards-compat: unset/false → no `schema` key)
- system_status_with_schema_flag_emits_schema_block (positive case:
schema block present, schemaVersion >= 13, tables non-empty,
knowledge_nodes row count + columns sane, convenience fields
present)
- system_status_camelcase_alias (#[serde(rename_all="camelCase")] +
alias works for both snake and camel input)
- storage_schema_introspection_method (Storage-layer method tested
directly, independent of MCP)
Closes the second of two gaps surfaced in the knowledge-mgmt-sota-uplift
initiative. Companion to PR #68 (search.tag_prefix). The two PRs are
deliberately decoupled — this one carries the storage-layer surface
extension; the other is MCP-layer-only.
* fix(memory): derive Default on SchemaIntrospection to satisfy clippy
The manual `impl Default for SchemaIntrospection` tripped
`clippy::derivable_impls` under the workspace's `-D warnings` CI gate.
All fields are types with `Default` impls (`u32`, `Option<T>`, `Vec<T>`,
`i64`), so deriving is equivalent and clippy-clean. Matches the existing
style of `ConsolidationResult` further down in the same file.
Adds an optional `tag_prefix` string parameter to the `search` MCP tool.
When set, only results that carry at least one tag whose value starts
with the prefix are returned (case-sensitive, matching the existing
exact-tag semantics in memory_timeline / export / gc).
Motivation: external consumers that need "all memories tagged
`<class>:*`" (e.g. `meeting:standup`, `meeting:1-on-1`) currently have
three paths, all bad: (i) export everything and filter client-side
(heavy), (ii) enumerate the prefix space and pass exact tags as a list
(impractical for open-set tag classes), or (iii) read SQLite directly
(an anti-pattern that couples consumers to internal schema). This PR
closes that gap with a minimal, additive surface.
Implementation note: filter runs at the MCP layer, NOT in the storage
predicate. Rationale: (a) leaves crates/vestige-core/src/storage/
untouched, avoiding collision with PR #61's storage-trait extraction;
(b) `SearchResult.node.tags` is already loaded from the same JSON-array
column the brief's proposed SQL would scan, so the post-filter is
functionally equivalent; (c) post-filter applies BEFORE the reranker
so the cross-encoder does not waste cycles on memories the caller will
not receive, and BEFORE strengthen-on-access so dropped results do not
get a testing-effect boost they did not earn.
Headroom: when tag_prefix is set, the hybrid path doubles its overfetch
multiplier (capped at the existing 100 ceiling) and the concrete path
fetches 3x its normal limit, both to leave the post-filter enough pool
to still return ~limit results after thinning. The Stage 0
keyword-priority merge also re-applies the prefix filter so it cannot
re-introduce filtered-out memories.
Backwards-compat: parameter is optional, defaults to None; every existing
call shape and response shape is unchanged.
Tests:
- tags_match_prefix unit (prefix-vs-substring, case-sensitivity,
tagless-memory semantics, empty-prefix corner case)
- schema introspection (property present, type=string, not required)
- hybrid-path filter excludes non-matching tag-classes
- hybrid-path filter excludes tagless memories
- backwards-compat: no tag_prefix → behavior unchanged
- concrete-path filter (literal-query branch) honors tag_prefix
Closes a gap surfaced in the knowledge-mgmt-sota-uplift initiative
(KMSU Session 89 audit; ~3,300-memory production Vestige).
Claude Code v2.1.91+ honors the per-tool annotation
`_meta["anthropic/maxResultSizeChars"]` (up to 500_000) to override
its 50K default truncation of `CallToolResult`. Without it, large
Vestige payloads are silently truncated and spilled to disk, forcing
the parent agent to chunk-read them.
Empirically observed truncation under realistic default parameters
(measured on v1.3.0 against ~3,300 memories; v2.x tool surface
preserves the same names + payload shapes):
search(detail_level="full", limit=20) -> 134,824 chars -> truncated
search(detail_level="summary", limit=10) -> 71,318 chars -> truncated
memory_timeline(limit=30) -> 83,626 chars -> truncated
This patch:
1. Adds `meta: Option<serde_json::Value>` to `ToolDescription` with
`#[serde(rename = "_meta")]` so the wire shape matches the MCP
spec. Backwards-compatible (the field is optional +
`skip_serializing_if`; older MCP clients ignore unknown JSON keys
per the spec).
2. Derives `Default` on `ToolDescription` so existing call sites can
adopt the new field via struct-update syntax
(`..Default::default()`) without restating it.
3. Annotates the four high-payload tools per measurement-driven
discipline; the other 21 tools deliberately do NOT carry the
annotation (cargo-cult prevention — a generous cap on every tool
dilutes the signal and trains future maintainers that the value
is arbitrary):
- search -> 300_000 (2.2x headroom over observed peak)
- memory_timeline -> 200_000 (2.4x headroom over observed peak)
- memory -> 100_000 (single-record bounded)
- codebase -> 100_000 (future-growth bounded)
Tools that COULD plausibly grow into the annotated set with future
workload (`deep_reference`, `cross_reference`, `memory_graph`,
`explore_connections`, `session_context`) are left unannotated
until empirical measurement shows truncation under realistic use.
4. Adds three regression tests in `server::tests`:
- test_high_payload_tools_have_max_result_size_annotation:
pins each cap value + asserts <= 500K Anthropic ceiling
- test_other_tools_do_not_carry_max_result_size_annotation:
cargo-cult prevention; dynamically iterates `tools/list` and
asserts every tool NOT in the discipline-prescribed set lacks
the annotation (robust to new tools being added by future PRs)
- test_meta_wire_shape_uses_underscore_meta_field:
pins the serde rename to `_meta` (the spec'd wire name) so a
refactor of `ToolDescription` cannot silently drop the rename
All 22 `server::tests` pass on v2.1.22 base (19 pre-existing + 3 new).
Full lib test suite: 379/380 pass; the 1 unrelated failure
(`tools::maintenance::tests::test_portable_export_writes_archive_to_storage_exports_dir`)
is a pre-existing Windows path-separator assertion bug in
`tools/maintenance.rs:823` (`path.ends_with("exports/portable-test.json")`
fails on Windows where the path uses `\`) — unaffected by this PR.
References:
- Anthropic CC v2.1.91 release notes (April 2026): "Added MCP tool
result persistence override via _meta['anthropic/maxResultSizeChars']
annotation (up to 500K), allowing larger results like DB schemas
to pass through without truncation"
- claude-agent-sdk-python v0.1.55 #756: forward bookkeeping
establishing the on-Tool-definition (not on-CallToolResult)
semantics for this annotation
Co-authored-by: Peter Lauzon <inbijiburu@protonmail.com>
