Tool Consolidation v2.2.0 (34->12 advertised surface: recall/memory/codebase/
intention/smart_ingest/source_sync/memory_status/maintain/dedup/graph/
session_start/suppress) onto the retroactive-salience-backfill base.
Conflict resolution:
- server.rs: took #99's canonical 12-tool surface, then re-added
as a 13th ADVERTISED tool (ToolDescription + dispatch arm + test). Backfill
is the flagship 'Memory with hindsight' (Cai 2024 Nature) cognitive primitive
— a distinct capability, not a maintenance op to fold into `maintain` or hide.
- Cargo.toml: kept usearch features=["fp16lib"] with the fuller MSVC-C1021
explanation comment (Windows build fix, #71).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire Step 8.5 into run_consolidation (bounded + idempotent via durable
Causal edge), share looks_like_failure/extract_entities across MCP tool +
CLI + auto-fire, add whole-word failure markers (API_TIMEOUT no longer
false-fires), --json CLI for CauseBench, and refactor the Redmine SSRF
guard to std::net (drops the dangling url-crate dep).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds docs/launch/tool-consolidation-v2.2.0.md — the single sequenced plan that
reconciles the two prior planning notes:
- Layer 1 (this PR): 34 → 12 advertised tools, safe commit order, alias policy,
preserved invariants, and the test that proves each.
- Layer 2 (follow-up): tiny always-on default surface + SessionStart/Stop hooks.
Also refreshes stale in-code comments to match the consolidated surface:
- server.rs handle_tools_list header (was "v2.1.21: 25 tools") and the
size-annotation rationale (now lists recall/memory_status/dedup/graph).
- tools/mod.rs module doc (the facade vs. granular-handler relationship).
No behavior change. Gates: cargo test --workspace, cargo clippy -D warnings,
pnpm dashboard check + build — all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tool Consolidation v2.2.0, Layer 1 commit 5/6. Advertised tools 21 → 15.
Folds consolidate + dream + gc + importance_score + backup + export + restore
into one action-dispatched tool:
action = consolidate | dream | gc | importance_score | backup | export | restore
- Thin facade forwards the same args envelope to each existing handler; the
`action` discriminator is cloned out before the envelope is moved into a
callee. No handler uses deny_unknown_fields, so per-action params validate.
- Safety defaults preserved (all handler-internal): gc dry_run=true by default,
restore path-confinement, export traversal guard. Verified by
test_maintain_actions_and_safety (gc dry-run + restore missing-path error).
- Events preserved end-to-end:
- Pre-dispatch Started events (ConsolidationStarted/DreamStarted) re-emitted
in the `maintain` arm keyed on action.
- emit_tool_event normalizes the `maintain` name to its effective action, so
the existing ConsolidationCompleted/DreamCompleted/ImportanceScored arms
fire unchanged — no duplicated emit logic.
- All 7 old names remain hidden warn!+redirect aliases (removed v2.3.0), keeping
their own pre-emits.
- Tests: count 21→15, 7 negatives, new dispatch/safety test.
Gates: cargo test --workspace, cargo clippy -D warnings — clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tool Consolidation v2.2.0, Layer 1 commit 4/6 (3b). Advertised tools 24 → 21.
Folds explore_connections + predict + memory_graph + composed_graph into one
action-dispatched tool:
action ∈ {chain, associations, bridges, predict, memory_graph,
recent, get, memory, neighbors, never_composed, bounty_mode, label}
- Transparent facade: each action forwards the same args envelope to the
existing handler, which re-reads its own discriminator/params. No underlying
arg struct uses deny_unknown_fields, so cross-fields are ignored.
- All actions read-only except `label` (the one mutator), which is logged for
audit via tracing::info!(event_id, outcome_type).
- outcome_type enum sourced from composed_graph::OUTCOME_TYPES (now pub(crate))
rather than re-listed, so the vocabulary stays single-sourced.
- All 4 old names remain hidden warn!+redirect aliases (removed v2.3.0).
- Size annotation: graph=250_000 (memory_graph layout + bounty_mode pagination),
kept in sync across loop, helper, and both annotation tests.
- Tests: count 24→21, 4 negatives, test_graph_actions_and_aliases exercising
read-only actions + aliases (incl. the memory_graph facade branch).
Note: the design draft mislabeled graph::execute as sync; it is `pub async fn`.
The per-commit build gate caught it — the facade awaits it correctly.
Gates: cargo test --workspace, cargo clippy -D warnings — clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tool Consolidation v2.2.0, Layer 1 commit 3/6 (3a). Advertised tools 27 → 24.
Folds system_status + memory_health + memory_timeline + memory_changelog
into one view-dispatched tool:
view = health (default) | retention | timeline | changelog
- Thin facade: each view forwards the same args envelope to the existing
handler. No underlying arg struct uses deny_unknown_fields, so the `view`
discriminator is ignored by each handler and per-view fields validate as
before. The cognitive lock is never held across a forwarded call.
- view='health' returns the byte-for-byte system_status shape (audit scripts
parse it), incl. schema_introspection passthrough — verified by
test_default_view_is_health asserting equality with execute_system_status.
- All 4 old names remain hidden warn!+redirect aliases (removed v2.3.0).
- Size annotation moved: memory_timeline (200k) → memory_status, kept in sync
across the real loop, expected_max_result_size(), and both annotation tests.
- Tests: count 27→24, 4 negative asserts, test_memory_status_views_and_aliases
exercising all 4 views + 4 aliases.
Gates: cargo test --workspace, cargo clippy -D warnings — clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tool Consolidation v2.2.0, Layer 1 commit 2/6. Advertised count unchanged
at 27 (pure rename).
`session_start` is the imperative-outcome name for the one-call session
initializer. `session_context` remains a hidden warn!+redirect alias
(≥1 minor release, removed v2.3.0), calling the same handler unchanged.
Tests: positive assert swapped to session_start + negative assert for the
old name. Gates: cargo test --workspace, cargo clippy -D warnings — clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All verified against real code:
- github fetch_updated: propagate fetch_comments errors instead of
unwrap_or_default() (silent failure stored a comment-less record with a
corrupted hash AND advanced the cursor past it = permanent gap)
- relationships load_relationships: advance next_id past loaded rel-N ids so
new_id() can't collide with persisted ids
- speculative store_pending_predictions: merge instead of clear (two predict()
calls without an intervening record_usage() wiped the first batch's accounting)
- embeddings/code is_comment_only: stop treating leading '#'/'*' as comments
(was deleting #include, #define, *ptr, CSS '*', multiplication)
- importance_signals: reconcile total_count to the surviving pattern sum after
decay/prune (was inflated, driving novelty toward zero over time)
- importance on_retrieved_with_context: set context in the SAME critical section
as the event push via record_retrieval() (two-lock approach raced; last_mut()
could land on a concurrently-pushed event)
core 535/0, clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All verified against real code:
- connectors: empty list_live_ids() no longer mass-tombstones the entire source
(treat empty as "cannot enumerate", like None) — was catastrophic data loss
- redmine: SSRF guard — require http(s), reject loopback/private/link-local hosts
+ localhost (escape hatch VESTIGE_ALLOW_PRIVATE_CONNECTOR_HOSTS for local tests)
- KnowledgeEdge::is_valid now honors valid_from via was_valid_at(now) (was
ignoring it; a future-dated edge read as valid). No production callers.
- emotional_memory: flashbulb counter reconciles to the FINAL is_flashbulb
decision after the importance override (was undercounting via ==0 guard)
- chains path_to_chain: fixed off-by-one — step i now uses the INCOMING edge
connections[i-1], not the outgoing connections[i]; removed the synthetic
connection hack that masked the misalignment
- merge_supersede compose_merged_content: exact normalized dedup instead of
substring containment (was silently dropping "cat" inside "cathedral")
core 535/0, clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Multi-model audit (deepseek-v4/minimax/kimi/qwen) surfaced these; verified
against the real code and fixed the confirmed ones:
- github connector: host-pin failed OPEN when api_root was unparseable/hostless
— the bearer token would ride a Link `next` url to an attacker host. Now
fail-closed: no pinned host => drop the url. (CRITICAL: SSRF / token exfil)
- GithubConfig/RedmineConfig derived Debug leaked the token/api_key into any
{:?} log line or panic message. Replaced with manual redacting Debug impls.
- cross_project priority calc used `as u32 - i` which underflows/panics (debug)
or wraps + corrupts the sort (release). Use saturating_sub.
Verified false-positive (no change): path-traversal in get_file_context — it
only inspects the path string, never reads the file.
core: 535 passed / 0 failed, clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
demo/README.md: the complete self-serve demo artifact — one-command run, the
seeded scenario explained, a "build your own scenario" section, the honest
boundary (won't invent a cause; can't reach a cause that was never recorded),
the Nature citation + the "field admits this is unsolved" sources, and the
recording playbook + paste-ready caption.
Writing/testing the README surfaced a real inconsistency, now fixed:
- The CLI's failure-finder used a hardcoded content-only marker subset and
ignored tags, so a "Checkout latency spiked" memory (regression tag, no crash
word in content) was never picked as the failure. The CLI now calls the SAME
public `looks_like_failure` (content + tags, full list) the backfill tool uses
— one definition, no drift.
- Extended FAILURE_MARKERS with performance/degradation failures (spiked,
latency, degraded, slow, hang, throttled, oom, 502/503/504, flaky, ...) so the
feature backfills from perf regressions, not just hard crashes.
clippy clean; 527 core + 453 mcp tests; both the main demo and the README's
custom scenario verified end-to-end.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3-model rotation audit (DeepSeek V4-Pro / Kimi K2.7 / MiniMax M3, max thinking,
each model × each of 3 sections). Claude verified every finding against code.
CONFIRMED + FIXED:
- [FTS, consensus DeepSeek+MiniMax] sanitize_fts5_or_query split on
!is_alphanumeric()+'_', but the index uses tokenize='porter ascii' which
splits on '_' and non-ASCII. So "API_TIMEOUT"/"café" became single phrases that
could NEVER match. Now splits on !is_ascii_alphanumeric() + lowercases to mirror
the tokenizer; caps token count (64) and length (64) for DoS hardening. Also
fixes the pre-existing storage.search bug (multi-word queries silently returned
nothing). 5 new tests pin it.
- [Demo honesty, consensus Kimi+DeepSeek] the contrast labeled keyword search as
"SIMILARITY SEARCH" and asserted "NONE of these is the cause" universally. Now
prints the REAL engine ("keyword (BM25)" vs "semantic (vector + BM25 hybrid)")
and claims only what's true ("ranked by RESEMBLANCE; its top hit is a lookalike").
De-hardcoded the "Service crashed:" munging to a generic label-strip.
VERIFIED FALSE POSITIVE (not changed): MiniMax "fts.id non-existent column" —
the FTS5 table is declared `fts5(id, content, tags, ...)`, the JOIN is valid.
No injection found by any model (quote-doubling + operator-stripping confirmed safe).
clippy clean; 527 core + 453 mcp tests pass; demo verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CLI surface for Retroactive Salience Backfill so the seeded demo (and anyone who
clones it) can reproduce "memory with hindsight" from a terminal:
- `ingest --ago-days N`: backdate a memory N days (plant a dated cause/history).
- `backfill [--failure-id ID] [--manual] [--lookback-days N] [--no-promote]`:
reach backward from a failure and surface+promote the causal earlier memory,
with demo-grade colored output (↩ reached back N days, 🔗 causal join: <entity>,
similarity rank, ✅ promoted).
Verified live end-to-end on a real DB: plant a 3-day-old API_TIMEOUT env-var note
+ a semantically-similar 500 distractor + a crash, run `vestige backfill`, and it
surfaces the env-var note by the shared api_timeout entity (ignoring the similar
distractor) and promotes it. clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MCP surface for memory-with-hindsight. When a failure memory exists, the
`backfill` tool reaches backward across the real store and promotes the quiet
earlier cause that a vector search structurally cannot surface (not similar to
the failure, only causally upstream via a shared entity).
- tools/backfill.rs: builds BackfillCandidates from real KnowledgeNodes (entities
from tags + heuristic env-var/path/identifier extraction), computes real cosine
similarity from stored embeddings (to PROVE the cause ranks low on similarity),
runs the core RetroactiveBackfill, and promotes surfaced causes via
storage.promote_memory. Auto-finds the latest failure, or takes failure_id;
manual=true forces; promote=false for a dry run.
- registered + dispatched in server.rs (35 tools now); tool-list test updated.
- storage: added pub set_created_at (backdate created_at) so the demo/test can
plant a dated cause.
LIVE RECEIPT: live_backfill_surfaces_root_cause_through_storage ingests a
3-day-old API_TIMEOUT env-var note + a semantically-similar 500-error distractor
+ a crash into a REAL SQLite store, runs the backfill tool, and asserts it
surfaces + promotes the env-var note by the shared API_TIMEOUT entity (the root
cause RAG misses). clippy clean; 522 core + 453 mcp tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The headliner neuro-mechanism: when a salient FAILURE lands (bug/crash/
regression — the "aversive event"), reach BACKWARD in time and promote the
quiet earlier memory that caused it — the one a vector search structurally
cannot surface because it isn't *similar* to the failure, only causally upstream.
Faithful port of Zaki/Cai et al. 2024, Nature 637:145-155 ("Offline ensemble
co-reactivation links memories across days"), causally proven (hippocampal
silencing abolishes the linking). Ported faithfully:
- backward-only asymmetry (fear links retrospectively, never prospectively) —
also exactly correct for software: a root cause is always upstream in time.
- linking flows along the shared-entity overlap (same file/env-var/service),
NOT semantic similarity — that's the whole point (RAG already covers similarity).
- scoped to failure->backward-causal-backfill, not "all salience flows backward"
(mirrors the Cai aversive->neutral paradigm; honest about scope).
Trigger: auto-detect (high prediction-error + failure markers) OR manual override.
Promotion: boosts FSRS stability so the cause stops decaying and surfaces next time.
Receipt (4/4 tests): backfill_surfaces_the_cause_rag_misses proves it promotes a
sim=0.11 env-var note over a sim=0.82 distractor by the shared API_TIMEOUT entity;
backward-only (future memory never promoted); no shared entity => no fabricated
cause; non-salient doesn't fire; manual override works. clippy clean; 522 core
tests pass (no regressions).
Wires into existing primitives: prediction_error gate (salience), dreams/
consolidation (offline window), memory/strength (promotion). MCP tool + live
demo next.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two deeper review findings (both blockers) + doc de-staling.
C2-deep: my earlier C2 made purge/delete TRACE as memory.write, but gate_writes
did `get_node(id) -> skip on None`, and purge had already DELETEd the row — so a
destructive removal still never opened a Memory PR (it was silently skipped).
The most security-critical write type couldn't be reviewed. Fix: a missing node
is now gateable for destructive decisions — gate_writes builds the WriteContext
from the decision itself (marks `forgets`, which classify_write gates), and the
PR records the removal with node.deleted=true. Proven live: purging a node opens
a PR (kind node_decayed, deleted true); test
gate_opens_pr_for_destructive_write_after_node_deleted_c2.
PRIV: gate_writes copied the FULL node.content into the PR diff + title, so a
real secret in a gated memory would leak into the memory_prs table, the
dashboard, and any exported proof bundle — defeating the point of gating
sensitive writes. Fix: the PR now stores a truncated content PREVIEW + an FNV
content HASH, and sensitive-topic/sensitive-node-type writes are fully REDACTED
("[redacted — sensitive content; review via risk signals]"). The reviewer still
sees the risk signals (why it opened) and a hash (to correlate), never the
secret. Tests gate_redacts_sensitive_content_in_pr_priv,
content_preview_redacts_sensitive_and_truncates, content_hash_is_stable. The
committed memory_pr.json + the whole proof bundle were re-captured and contain
no secret (verified by scan); the re-shot memory-prs.png shows the redaction.
DOC: REVIEW.md commit list is now git-log-based (no stale hashes); C2-deep + PRIV
added to the findings table; PROOF.md write/PR rows updated; test count -> 1007.
Gates: 1007 lib tests pass (+7 new regressions), clippy -D warnings clean,
dashboard check + build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two more review findings — both real, both blockers — plus stale-doc cleanup.
C1: the B1 release used reverse_suppression(subject_id, labile_hours), which
REFUSES once the 24h active-forgetting labile window has passed. So a Memory PR
reviewed late could be marked "promoted" while its memory stayed suppressed.
Approving a quarantined write is an explicit reviewer decision and must release
the memory regardless of elapsed time. New SqliteMemoryStore::release_quarantine
fully clears the suppression (count→0, suppressed_at→NULL) with NO time-window
limit; the PR handler now uses it. Proven: a test backdates suppressed_at to
+100h, shows reverse_suppression refuses, and release_quarantine still releases.
C2: memory(action="purge"|"delete") returns `action` + nodeId but those labels
weren't in is_write_decision, so destructive removal bypassed the memory.write
trace and the PR gate. Added purge/purged/delete/deleted/forget/forgotten.
Proven live: purging a node now records a second memory.write event
({"decision":"purge"}) under the run.
Docs: REVIEW.md de-staled — removed the fixed 140b15f diff-stat / "3 commits"
prose (it moved with each fix), listed all commits, added C1/C2 to the findings
table, updated the test count.
Gates: 1002 lib tests pass (+3 new regressions), clippy -D warnings clean,
dashboard check + build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A full multi-agent review found 7 real issues (4 blockers). All fixed + tested.
B1 (blocker): Promoting a Memory PR did not release the quarantined memory —
the UI said "promoted" while the memory stayed suppressed/out of retrieval.
act_on_memory_pr now calls reverse_suppression(subject_id) on accept actions;
MemoryPrAction::releases_memory() encodes the rule (promote/merge/supersede
release; forget/quarantine keep it held). Proven live: PR response
subjectReleased:true, SQLite suppression_count 0.
B2 (blocker): memory promote/demote (returns `action`, not `decision`) and
codebase remember_* writes bypassed the write-trace + PR gate. extract_writes
now reads `action` too, filtered by is_write_decision (reads like get/state
excluded); is_write_tool includes `codebase`.
B3 (blocker): receipt ids collided within a run (r_<date>_<runId> +
INSERT OR REPLACE overwrote earlier receipts). IDs are now
r_<date>_<runId8>_<unique6>; build() mints the suffix, build_with_unique()
keeps tests deterministic.
B4 (blocker): proof bundle was assembled from two runs (trace.json=run_proof,
websocket-events.jsonl=run_proof2). Re-captured the whole bundle from a single
run — trace, websocket, receipt, and memory_pr all carry run_proof now.
B5: Black Box receipts panel showed global latest, not the selected run.
Added list_receipts_for_run + /api/receipts?run= ; the page uses listForRun.
B6: SENSITIVE_TOPICS substring matching false-fired (tokenizer->token,
author->auth, secretary->secret). Switched to word-boundary matching; real
phrasings (auth token, security vulnerability, api key) still gate.
B7: set_review_mode now writes atomically (temp+rename via write_atomic);
export_trace sanitizes run_id in the Content-Disposition filename; memory-prs
static routes declared before the dynamic /{id} route.
Withdrawn: the /mode-vs-/{id} route order is NOT a functional bug (axum 0.8 /
matchit prioritizes static segments) — reordered for clarity only.
Gates: 999 lib tests pass (+9 new regressions), clippy -D warnings clean,
dashboard check + build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bounded follow-up (tight acceptance criteria, no scope expansion): flip the
dream.patch producer from "quiet because no dream ran" to a recorded live event.
The dream tool's `insights` array carries no per-item id, so the recorder
extracted zero proposals and dream.patch never fired even on a real dream.
Fix: derive a stable proposal id from each insight's REAL content (its
insight_type + the source memories it consolidated). The dream genuinely ran;
this just gives each real proposal a deterministic handle. Unit-tested against
the exact dream output shape.
Proven end to end (run_dream_proof, 6 memories consolidated):
- one dream.patch event: dream:RecurringPattern:5d941c7f+a41aca72+...
- SQLite + /api/traces/:runId: dream-trace.json (14 events, last is dream.patch)
- WebSocket: dream-websocket-events.jsonl (the dream.patch TraceEvent)
- dashboard: screenshots/dream-producers.png — the row flips to "fired this run"
PROOF.md updated: dream.patch moves from CAVEAT to REAL (still not live by
default — it fires only when a dream actually runs, and the UI says so).
sanhedrin.veto remains an honest CAVEAT (optional hook, off by default).
Gates: 957 lib tests pass, clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the receipt chain impossible to doubt. Freeze the claim surface, prove
every hop, and turn the two off-by-default producers into explicit UI states.
Frozen public claim: "Vestige records real MCP memory activity into a
replayable local trace, with receipts and reviewable risky writes." We do NOT
claim Sanhedrin vetoes or dream patches are live by default.
Regression — full-spine test (server.rs): one runId must cross, byte-identical,
MCP output -> SQLite trace -> WebSocket event -> API response shape ->
MCP resource. Fails if any hop drops or rewrites the id.
Honest UI states (Black Box "Event producers" panel):
- sanhedrin.veto -> "No veto producer connected (optional Sanhedrin hook, off
by default)" instead of empty mystery.
- dream.patch -> "No dream run in this trace" unless a dream actually ran.
- contradiction.detected -> "no contradiction in this run" when none fired.
Quarantine review (not pre-write blocking): risky writes are committed then
suppressed — audit history preserved, retrieval influence suspended until
reviewed. Reworded the server notice + UI copy to say exactly that.
Receipts UI gap closed: ReceiptCard is now mounted on the Black Box page
(retrieved/suppressed/trust-floor, activation path, "Open receipt in Cinema").
Proof pack (blackbox-proof-2026-06-22/): status.json, trace.json (the
.vestige-trace.json export), receipt.json, memory_pr.json (promoted via
UI->API->SQLite), websocket-events.jsonl (live TraceEvent x6 + PR opened/
decided), screenshots (Black Box, Receipts, Memory PRs, Graph), and PROOF.md
with real/caveat/stub per feature.
Gates: 988 lib tests pass, clippy -D warnings clean, dashboard check + build
clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The v2.1.26 pin set `default-features = false` on usearch to drop SimSIMD's
AVX2/Haswell dispatch (the illegal-instruction crash on older x86_64 CPUs,
#71). But usearch's defaults are ["simsimd", "fp16lib"], so disabling all
defaults also dropped the unrelated `fp16lib` feature.
With both off, usearch's build.rs sets USEARCH_USE_FP16LIB=0 and
USEARCH_USE_SIMSIMD=0, selecting the bare `#else` half-precision branch in
include/usearch/index_plugins.hpp, which carries a `#warning` directive. MSVC's
cl.exe treats `#warning` as fatal error C1021, so the Windows build aborts:
index_plugins.hpp(404): fatal error C1021: invalid preprocessor command 'warning'
GCC/Clang only warn, so Linux/macOS never caught it. `/Zc:preprocessor` does
not change MSVC's behavior here.
Fix: re-enable `fp16lib` only, keeping SimSIMD disabled. fp16lib is a scalar,
self-contained fp16<->fp32 conversion library with no SIMD intrinsics, so this
sets USEARCH_USE_FP16LIB=1 (taking the non-warning branch) without
reintroducing the #71 illegal-instruction risk. Restores v2.1.25 behavior on
the half-precision path while keeping the #71 portability fix. usearch stays
pinned at =2.23.0; Cargo.lock is unchanged.
Verified on macOS (aarch64): cargo build -p vestige-core and -p vestige-mcp
green, 9/9 vector-search tests pass, clippy clean. cargo tree confirms only the
`fp16lib` usearch feature is active and `simsimd` is not. The MSVC repro is
Windows-only and cannot be exercised on this host.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The portable archive is encrypted on the client before upload and decrypted
after download, so the hosted service only ever stores ciphertext — true
zero-knowledge. The passphrase (VESTIGE_CLOUD_ENCRYPTION_KEY) is independent
of the bearer sync key and never leaves the device.
- new cloud_crypto module: Argon2id KDF + XChaCha20-Poly1305 AEAD, self-
describing envelope (MAGIC|version|salt|nonce|ciphertext+tag)
- HttpPortableSyncBackend encrypts on write / decrypts on read; transparent
upgrade of legacy plaintext archives; clear error if remote is encrypted
but no passphrase is set
- sync_portable_archive_cloud takes optional encryption_key
- CLI surfaces encryption status (on/off) on sync
- 6 crypto tests (roundtrip, wrong-key, tamper detection, non-determinism,
envelope detection); E2E verified: server blob is ciphertext, passphrase
device recovers, no-passphrase device cannot decrypt
491 core tests green, clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump all manifests 2.1.26 → 2.1.27 and date the CHANGELOG entry for the
GitHub + Redmine connector layer and source-aware search filters (#57, PR #78).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make Vestige a durable, local, semantically-searchable retrieval layer over an
external system of record (GitHub Issues first), citing back to the canonical
record. Unlike a live ticket-system MCP proxy, Vestige keeps a durable embedded
index: searchable offline, joinable with the rest of memory, temporally
versioned, and re-syncable idempotently with no duplication.
Phases 1-2 of #57 plus a GitHub reference connector and source-aware search:
- Source envelope on KnowledgeNode/IngestInput (source_system, source_id,
source_url, source_updated_at, content_hash, synced_at, source_project,
source_type, source_author). Migration V17: nullable columns (additive),
partial UNIQUE index on (source_system, source_id), connector_cursors table.
- Idempotent sync primitives in vestige-core: upsert_by_source (content-hash
change detection), connector cursor checkpoints, reconcile_source_tombstones
(invalidate-don't-delete via bitemporal valid_until).
- Connector contract + run_sync driver + GitHub Issues connector behind the
optional `connectors` feature (on by default in vestige-mcp, off in the core
library default so non-connector consumers link no HTTP client).
- source_sync MCP tool ({"repo": "owner/name"}); token from GITHUB_TOKEN env
only. Search results gain a sourceRecord citation for connector memories.
Adversarial review fixes: GitHub `since` Z-form (the `+00:00` offset corrupted
the cursor server-side), un-tombstone clears superseded_by too, cursor never
advances past a failing record, Link next-url host-pinned (token-leak guard),
records_seen counts new records only.
Verified: cargo check/test/clippy -D warnings green across the workspace
(default and connectors features); 483 core tests pass. Version bump to 2.1.27
and tag deferred to release.
Refs #57
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cargo rm async-trait. Last usage was the FastembedEmbedder impl attribute,
removed in the preceding 0001c commit; the MemoryStore side stopped using
async-trait at 0001a.
Verification: grep -rn async_trait crates/ returns zero hits. grep -rn
async-trait --include=Cargo.toml crates/ returns zero hits. Cargo.lock no
longer references the async-trait package.
Mirror of the 0001a pattern for the Embedder side.
- embedder/mod.rs: LocalEmbedder is the source trait declared with native
async-fn-in-trait. #[trait_variant::make(EmbedderSend: Send)] derives the
Send-bounded variant that backends implement. A hand-written Embedder
trait wraps each async method in BoxedEmbedderFuture<'a, T> and forwards
sync methods through a blanket impl<T: EmbedderSend> Embedder for T, so
Box<dyn Embedder> / Arc<dyn Embedder> stay dyn-safe -- trait_variant 0.1
alone does NOT produce a dyn-safe variant (RPITIT), so the hand-written
adapter is required.
- embedder/fastembed.rs: drop the #[async_trait::async_trait] attribute and
retarget the impl block to EmbedderSend. Adjust the top-level use to
bring EmbedderSend into scope (also keeps fastembed::tests' use super::*
trait lookups working).
- lib.rs: export EmbedderSend alongside the existing Embedder /
LocalEmbedder re-exports.
The async-trait Cargo dependency is dropped in a follow-up commit so the
manifest change stays visible on its own.
Verification: cargo test -p vestige-core --features embeddings,vector-search
(428) and --no-default-features (370) both green. cargo test --test
embedder_trait green (2/2 including Box<dyn Embedder> cast). cargo build
--workspace --release green. cargo clippy --workspace --features
embeddings,vector-search -- -D warnings clean. grep -rn async_trait crates/
returns zero.
Replaces #[async_trait::async_trait] on the storage trait with a
trait_variant-driven layout plus a hand-written dyn-compatible adapter.
- memory_store.rs: LocalMemoryStore is the source trait declared with
native async-fn-in-trait. #[trait_variant::make(MemoryStoreSend: Send)]
derives the Send-bounded variant that backends actually implement (the
blanket impl in 0.1.x goes variant -> source). A hand-written
MemoryStore trait wraps every method in
Pin<Box<dyn Future<Output = MemoryStoreResult<T>> + Send + 'a>> with
a BoxedStoreFuture<'a, T> alias, and a blanket impl<T: MemoryStoreSend>
MemoryStore for T adapts every Send-variant implementation. This keeps
Arc<dyn MemoryStore> dyn-safe for Phase 1 cognitive-module tests --
trait_variant 0.1 alone does NOT produce a dyn-safe variant (RPITIT),
so the hand-written adapter is required and supersedes the plan claim
that trait_variant gives dyn-compat for free.
- sqlite.rs: drop the #[async_trait::async_trait] attribute on the impl
block and retarget it to MemoryStoreSend. Two pre-existing clippy
issues that the macro had been masking are fixed in the same body
(return Ok(out) tail expression in vector_search; DomainRow tuple
alias in get_domain).
- mod.rs: export MemoryStoreSend alongside the existing LocalMemoryStore
and MemoryStore re-exports.
Verification: cargo test -p vestige-core --features embeddings,vector-search
passes (428 lib tests). All five Phase 1 integration test binaries pass
(trait_round_trip, send_bound_variant including
arc_dyn_memory_store_moves_across_tokio_tasks, cognitive_module_isolation,
embedding_model_registry, domain_column_migration). cargo test --workspace
green across every test binary. cargo build --workspace --release green.
cargo clippy --workspace --features embeddings,vector-search -- -D warnings
clean. grep -rn async_trait crates/vestige-core/src/storage/ returns
zero hits.
Supersedes plan claim in docs/plans/0001a-trait-rewrite.md about
trait_variant emitting a dyn-compatible Send variant; option (c) from
the design conversation (hand-written dyn adapter) was selected
explicitly because trait_variant 0.1.2 does not.
Introduce two trait boundaries that the rest of the stack now sits above,
landing Phase 1 of ADR 0001 (pluggable storage and network access).
Rebased onto v2.1.22 Sanhedrin from the original April work.
MemoryStore / LocalMemoryStore (crates/vestige-core/src/storage/memory_store.rs):
One trait, ~25 methods, covering CRUD, hybrid / FTS / vector search,
FSRS scheduling, graph edges, and the forthcoming domain surface.
trait_variant::make generates a Send-bound MemoryStore alias over the
base LocalMemoryStore so Arc<dyn MemoryStore> works under tokio/axum.
Storage errors map through a dedicated MemoryStoreError.
Embedder / LocalEmbedder (crates/vestige-core/src/embedder/):
Pluggable text-to-vector encoder. FastembedEmbedder wraps the existing
EmbeddingService; storage never calls fastembed directly anymore.
Embedder::signature() produces the ModelSignature consumed by the
store's embedding_model registry.
SqliteMemoryStore (crates/vestige-core/src/storage/sqlite.rs):
Storage renamed to SqliteMemoryStore; the old name lives on as a
pub type alias so Arc<Storage> consumers in vestige-mcp stay intact.
All existing inherent methods are untouched; the trait impl is
purely additive and dispatches into them. The db_path field added
by v2.1.1 portable-sync is preserved.
Migration V14 (crates/vestige-core/src/storage/migrations.rs):
Renumbered from V12 (the original April number) to V14 to slot in
cleanly after upstream's V12 (v2.1.1 sync_tombstones) and V13
(v2.1.2 purge tombstones).
- embedding_model registry table (CHECK id = 1, code enforces the
single-row invariant).
- knowledge_nodes.domains / domain_scores TEXT columns (JSON arrays
default '[]' / '{}'), domains catalogue table, supporting indexes.
Phase 4 populates these columns; Phase 1 just exposes the schema.
Consolidation and other cognitive pathways now accept a
&dyn LocalMemoryStore (sync) or Arc<dyn MemoryStore> (async) rather
than a concrete Storage.
Tests:
- trait-method unit tests colocated in sqlite.rs and migrations.rs
- embedder/fastembed.rs tests for name/dimension/hash stability
- new integration crate tests/phase_1 (added to workspace members):
trait_round_trip (8), embedding_model_registry (7),
domain_column_migration (5), cognitive_module_isolation (4),
send_bound_variant (2), embedder_trait (2).
Acceptance gate post-rebase:
- cargo build --workspace --all-targets: ok
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo test -p vestige-core --lib: 428 pass
- cargo test -p vestige-phase-1-tests: 28 pass
- cargo test -p vestige-mcp --lib: 380 pass (Storage alias preserves
every existing call site)
Co-existence with v2.1.1 portable-sync: this trait extraction is
additive. Portable-sync's tombstone migrations (V12, V13) remain
on the concrete SqliteMemoryStore; Phase 2 (Postgres) will decide
which of those surfaces graduate into the trait.
Rebased on v2.1.25 merge/supersede and bumped the post-release metadata to v2.1.26 so this branch does not roll versions backward.
Adds local vestige.toml defaults, output profiles, and MCP response precedence for search, timeline, codebase context, and session context.
Verified:
- cargo metadata --format-version 1 --locked --no-deps
- cargo test -p vestige-core config --no-fail-fast
- cargo test -p vestige-mcp config --no-fail-fast
Tolerate SQLite-native timestamps from external writers while preserving RFC3339 as the canonical write format.
Verified locally on the merge result:
- cargo test -p vestige-core test_parse_timestamp_accepts_rfc3339_and_sqlite_native --no-fail-fast
CI/Test Suite on the updated PR branch are green.
* feat(mcp/system_status): add optional schema_introspection flag
Adds an optional `schema_introspection: bool` parameter to the
`system_status` MCP tool. When set to true, the response gains a
`schema` block carrying:
- `schemaVersion` (u32) — highest applied migration, mirrors
`Storage::current_schema_version` (now exposed via a typed public
method).
- `schemaVersionAppliedAt` (RFC3339, optional) — timestamp the
current schema_version row was applied.
- `tables` ([{name, rows, columns}]) — per-table row count + column
list, walked over the canonical PORTABLE_USER_DATA_TABLES set so
the surface stays stable across migrations rather than enumerating
arbitrary sqlite_master rows.
- `embeddingNullCount` (i64) — count of knowledge_nodes with NO row
in node_embeddings. Distinct from MemoryStats.nodes_with_embeddings
(which keys off the `has_embedding` flag column), so audit scripts
can detect drift between the flag and the join-based truth.
- `activeEmbeddingModel` (string, optional) + `activeEmbeddingDimensions`
(u32, optional) — mirrors the existing MemoryStats active-model
fields, included here so audits get schema_version + active model
in a single round-trip.
Motivation: external consumers (audit scripts, migration guards,
downstream binary upgrade scripts) currently must read SQLite
directly to learn the schema shape, which couples them to internals
Vestige owns and breaks on every migration. This PR closes that gap
with a first-class MCP surface.
Implementation:
- New `pub fn schema_introspection() -> Result<SchemaIntrospection>`
inherent method on `Storage` (sqlite.rs). Inherent, not on a
trait — schema-walk is SQLite-specific by nature, so this stays
out of any future MemoryStore trait extraction.
- New typed structs `SchemaIntrospection` + `TableIntrospection` in
memory/mod.rs (canonical home alongside MemoryStats), re-exported
from the crate root.
- MCP layer (maintenance.rs) parses `SystemStatusArgs`, conditionally
extends the existing response object with a `schema` key — additive,
default off, response shape unchanged when omitted.
Coupling assessment vs PR #61 (storage-trait-phase1):
This PR adds ONE new public inherent method on `Storage` plus uses
three already-existing private helpers (`current_schema_version`,
`table_exists`, `table_row_count`, `table_columns`). It does NOT
touch the existing inherent method signatures, does NOT add anything
to the prospective `MemoryStore` trait surface, and does NOT modify
any of the ~25 methods #61 lifts into the trait. PR #61 is purely
additive on the trait surface (per its description, `pub type
Storage = SqliteMemoryStore;` preserves all existing call sites);
this PR is additive on the inherent surface. Two purely-additive
changes to disjoint surfaces should rebase cleanly.
Tests:
- system_status_schema_has_schema_introspection_flag (schema
introspection: property present, type=boolean, default=false,
not required)
- system_status_without_schema_flag_omits_schema_block
(backwards-compat: unset/false → no `schema` key)
- system_status_with_schema_flag_emits_schema_block (positive case:
schema block present, schemaVersion >= 13, tables non-empty,
knowledge_nodes row count + columns sane, convenience fields
present)
- system_status_camelcase_alias (#[serde(rename_all="camelCase")] +
alias works for both snake and camel input)
- storage_schema_introspection_method (Storage-layer method tested
directly, independent of MCP)
Closes the second of two gaps surfaced in the knowledge-mgmt-sota-uplift
initiative. Companion to PR #68 (search.tag_prefix). The two PRs are
deliberately decoupled — this one carries the storage-layer surface
extension; the other is MCP-layer-only.
* fix(memory): derive Default on SchemaIntrospection to satisfy clippy
The manual `impl Default for SchemaIntrospection` tripped
`clippy::derivable_impls` under the workspace's `-D warnings` CI gate.
All fields are types with `Default` impls (`u32`, `Option<T>`, `Vec<T>`,
`i64`), so deriving is equivalent and clippy-clean. Matches the existing
style of `ConsolidationResult` further down in the same file.
