mirror of
https://github.com/samvallad33/vestige.git
synced 2026-06-08 20:25:16 +02:00
Release v2.1.1 portable sync
This commit is contained in:
Sam Valladares
parent
738e4f7dce
commit
f3d63af12e
35 changed files with 3257 additions and 421 deletions
42
CHANGELOG.md
42
CHANGELOG.md
|
|
@ -5,9 +5,34 @@ All notable changes to Vestige will be documented in this file.
|
|||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [2.1.1] - 2026-05-01 — "Portable Sync"
|
||||
|
||||
v2.1.1 focuses on user-controlled portability: exact storage archives, merge-safe file sync, pluggable sync backends, and explicit hook opt-ins.
|
||||
|
||||
### Added
|
||||
|
||||
- **Exact portable archives** — `vestige portable-export` / `vestige portable-import` preserve raw Vestige storage rows: memory IDs, FSRS state, graph edges, suppression state, audit rows, sessions, intentions, and embedding blobs.
|
||||
- **Merge-safe imports** — `vestige portable-import --merge` can merge into non-empty databases. It applies `sync_tombstones`, keeps newer local memory rows on timestamp conflicts, preserves stable IDs, and rebuilds FTS after import.
|
||||
- **File-backed two-way sync** — `vestige sync <archive>` performs pull-merge-push through a shared portable archive. This works today with Dropbox, iCloud Drive, Syncthing, Git, network shares, and shared folders.
|
||||
- **Pluggable portable-sync backend trait** — core now exposes `PortableSyncBackend`, `FilePortableSyncBackend`, and `PortableSyncReport`, so non-file backends can reuse the same merge semantics without reimplementing conflict handling.
|
||||
- **Portable restore merge mode** — the MCP `restore` tool accepts `merge: true` for portable archives and returns inserted/updated/deleted/skipped/conflict counts.
|
||||
- **Qwen3 embedding opt-in** — build-time and runtime support for Qwen3 embeddings, with model-aware retrieval safeguards so mixed embedding models are not compared in the same vector path.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Sanhedrin, preflight, and all Vestige Claude Code hooks are optional again.** The Cognitive Sandwich installer now activates no hooks by default and leaves every preflight hook, every Stop hook, the MLX launchd service, and the 19 GB Qwen model path behind explicit `--enable-preflight`, `--enable-sanhedrin`, or `--with-launchd` flags.
|
||||
- **x86-friendly Sanhedrin path.** The verifier bridge now accepts any OpenAI-compatible chat endpoint via `VESTIGE_SANHEDRIN_ENDPOINT` and `VESTIGE_SANHEDRIN_MODEL`, so Linux and Intel Mac users can opt in without MLX or Apple Silicon.
|
||||
|
||||
### Verified
|
||||
|
||||
- `cargo test -p vestige-core portable --no-fail-fast`
|
||||
- `cargo test -p vestige-mcp portable --no-fail-fast`
|
||||
- `cargo test --workspace --no-fail-fast`
|
||||
- Installer shell/Python/JSON validation and default/preflight/Sanhedrin migration dry-runs.
|
||||
|
||||
## [2.1.0] - 2026-04-27 — "Cognitive Sandwich Goes Local"
|
||||
|
||||
The Sanhedrin Executioner — Vestige's veto layer for Claude Code responses — now runs entirely on a local MLX model (`mlx-community/Qwen3.6-35B-A3B-4bit`). Zero API cost per Claude turn, fully offline, no Anthropic round-trip on the critical path. Combined with four pre-cognitive UserPromptSubmit hooks (synthesis-preflight, cwd-state-injector, vestige-pulse-daemon, preflight-swarm), Vestige now ships a complete "Cognitive Sandwich" — Vestige memories injected before the model thinks, local Sanhedrin veto after the model speaks — installable in one command on a MacBook.
|
||||
v2.1.0 ships the Cognitive Sandwich hook harness for Claude Code, with a hotfix that makes every hook layer opt-in. The default installer stages files, removes old Vestige hook wiring, starts no MLX service, downloads no model, and makes no automatic model calls. Users can opt into preflight context, Sanhedrin verification, or the Apple Silicon MLX backend separately.
|
||||
|
||||
### Added
|
||||
|
||||
|
|
@ -21,18 +46,18 @@ The Sanhedrin Executioner — Vestige's veto layer for Claude Code responses —
|
|||
- `synthesis-stop-validator.sh` — Stop hook regex against forbidden hedging patterns.
|
||||
- `veto-detector.sh` — fast 50ms regex pre-screen against `veto`-tagged Vestige memories.
|
||||
- `synthesis-gate.sh` — legacy v1 trigger (kept for backward compat).
|
||||
- `settings.fragment.json` — JSON snippet merged into `~/.claude/settings.json` by the installer.
|
||||
- `settings.fragment.json` — empty default JSON snippet; the installer only wires hooks from the opt-in preflight/Sanhedrin fragments.
|
||||
- **Dashboard `/api/changelog` endpoint** — bounded REST event feed for recent `DreamCompleted` and `ConnectionDiscovered` events, used by the Pulse hook to inject fresh synthesis into Claude Code context.
|
||||
- **`agents/`** — `executioner.md` (legacy/fallback Haiku 4.5 path), `lateral-thinker.md`, `synthesis-composer.md`.
|
||||
- **`launchd/com.vestige.mlx-server.plist.template`** — auto-start `mlx_lm.server` with the Qwen3.6-35B-A3B-4bit model on login. Templated with `__HOME__` and `__MODEL__` placeholders.
|
||||
- **`scripts/install-sandwich.sh`** — one-command installer that stages hooks, agents, plist, jq-merges the settings fragment, and `launchctl load`s the plist. Backs up `settings.json` to `.bak.pre-sandwich`. Supports `--force`, `--no-launchd`, `--include-memory-loader`, `--src=PATH`.
|
||||
- **`scripts/check-sandwich-prereqs.sh`** — comprehensive prereq verifier (Apple Silicon, Python 3.10+, jq, uv, mlx-lm, hf, claude, vestige-mcp, model on disk, MCP HTTP up, server up, plist installed, settings wired).
|
||||
- **`scripts/install-sandwich.sh`** — one-command installer that stages hooks and agents, removes old Vestige hook wiring by default, and only wires optional layers with `--enable-preflight`, `--enable-sanhedrin`, or `--with-launchd`. Backs up `settings.json` to `.bak.pre-sandwich`. Supports `--force`, `--include-memory-loader`, and `--src=PATH`.
|
||||
- **`scripts/check-sandwich-prereqs.sh`** — default verifier confirms no Vestige Claude Code hooks are wired. Optional `--preflight` and `--sanhedrin` modes check the corresponding enabled layer.
|
||||
- **`docs/COGNITIVE_SANDWICH.md`** — architecture diagram, install guide, performance notes (82 tok/s on M3 Max), uninstall, configuration env vars.
|
||||
- **PR #48** — `VESTIGE_DATA_DIR` env-var support + tilde expansion + secure unix perms (thanks @Jelloeater) — directly addresses the ghost env-vars exposed by v2.0.9 cleanup.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Sanhedrin Executioner default backend swapped from Anthropic Haiku 4.5 → local `mlx_lm.server` + Qwen3.6-35B-A3B-4bit.** Anthropic API key no longer required for the post-cognitive layer. The `executioner.md` agent definition is retained as manual/fallback only when invoked explicitly via `Task(subagent_type='executioner')`.
|
||||
- **Sanhedrin Executioner backend can run locally or remotely when explicitly enabled.** The bridge targets an OpenAI-compatible chat endpoint, with local `mlx_lm.server` + Qwen3.6-35B-A3B-4bit available behind `--with-launchd` on Apple Silicon. Anthropic API key no longer required for the post-cognitive layer. The `executioner.md` agent definition is retained as manual/fallback only when invoked explicitly via `Task(subagent_type='executioner')`.
|
||||
- **All hooks sanitized for public release** — replaced hardcoded personal absolute paths with `$HOME` / `$VESTIGE_*` env vars; removed personal regex tokens.
|
||||
- **NPM binary installer now follows package version** — `vestige-mcp-server@2.1.0` downloads release assets from `v2.1.0` instead of a stale hardcoded binary tag, while local workspace installs skip the release-asset download before the tag exists.
|
||||
|
||||
|
|
@ -40,22 +65,23 @@ The Sanhedrin Executioner — Vestige's veto layer for Claude Code responses —
|
|||
|
||||
- `cargo test --workspace --release --no-fail-fast`: **1,229 passing, 0 failed** (366 vestige-core + 358 vestige-mcp lib + 4 vestige-mcp bin + 497 e2e + 4 doctests).
|
||||
- Sanhedrin bridge smoke checks: Python bytecode compilation passes, fail-open bridge invocation returns `yes`, and public hook settings validate as JSON.
|
||||
- v2.1.0 hotfix installer matrix: default, preflight-only, Sanhedrin-only, full sandwich, legacy-all-hooks migration, and unrelated custom hooks preservation.
|
||||
- 8-day Sandwich dogfood: **84% pass rate, 16% legitimate vetoes** caught real hallucinations.
|
||||
|
||||
### Closes
|
||||
|
||||
- #36 (Agent Hooks for Low-Effort Automatic Memory Capture) — Cognitive Sandwich is the answer.
|
||||
|
||||
### Prerequisites for the Cognitive Sandwich
|
||||
### Prerequisites for optional local MLX Sanhedrin
|
||||
|
||||
- macOS Apple Silicon (M1+) — required for MLX
|
||||
- macOS Apple Silicon (M1+) — required only for `--with-launchd` MLX autostart
|
||||
- Python 3.10+
|
||||
- ~22 GB free RAM (Qwen3.6-35B-A3B-4bit at runtime)
|
||||
- First-run model download: ~19 GB from Hugging Face (cached locally thereafter)
|
||||
|
||||
### Migration
|
||||
|
||||
None required for existing Vestige users. The Cognitive Sandwich is opt-in via `scripts/install-sandwich.sh`. The MCP server, schema, and tool surface are bit-identical to v2.0.9.
|
||||
None required for existing Vestige users. The Cognitive Sandwich is opt-in via `scripts/install-sandwich.sh`; running the default installer removes old Vestige hook wiring and leaves preflight, Sanhedrin, launchd, and the 19 GB model path disabled. The MCP server, schema, and tool surface are bit-identical to v2.0.9.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
5
Cargo.lock
generated
5
Cargo.lock
generated
|
|
@ -4531,8 +4531,9 @@ checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
|
|||
|
||||
[[package]]
|
||||
name = "vestige-core"
|
||||
version = "2.1.0"
|
||||
version = "2.1.1"
|
||||
dependencies = [
|
||||
"candle-core",
|
||||
"chrono",
|
||||
"criterion",
|
||||
"directories",
|
||||
|
|
@ -4566,7 +4567,7 @@ dependencies = [
|
|||
|
||||
[[package]]
|
||||
name = "vestige-mcp"
|
||||
version = "2.1.0"
|
||||
version = "2.1.1"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"axum",
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ exclude = [
|
|||
]
|
||||
|
||||
[workspace.package]
|
||||
version = "2.1.0"
|
||||
version = "2.1.1"
|
||||
edition = "2024"
|
||||
license = "AGPL-3.0-only"
|
||||
repository = "https://github.com/samvallad33/vestige"
|
||||
|
|
|
|||
43
README.md
43
README.md
|
|
@ -20,12 +20,21 @@ Built on 130 years of memory research — FSRS-6 spaced repetition, prediction e
|
|||
|
||||
---
|
||||
|
||||
## What's New in v2.1.1 "Portable Sync"
|
||||
|
||||
v2.1.1 focuses on the biggest post-launch ask: move memories between machines without losing cognitive state. It also adds opt-in Qwen3 embeddings for higher-recall local retrieval.
|
||||
|
||||
- **Exact portable archives.** `vestige portable-export` / `vestige portable-import` preserve IDs, FSRS state, graph edges, suppression state, audit rows, and embedding blobs for Vestige-to-Vestige device transfer.
|
||||
- **Sync-safe merge storage.** `vestige portable-import --merge` and `vestige sync <archive>` merge non-empty databases, apply delete tombstones, keep newer local memories, rebuild FTS, and push through a pluggable portable-sync backend. v2.1.1 ships the file backend for Dropbox, iCloud, Syncthing, Git, and shared folders.
|
||||
- **Qwen3 embeddings.** Build with `qwen3-embeddings`, set `VESTIGE_EMBEDDING_MODEL=qwen3-0.6b`, and run `vestige consolidate` to re-embed existing memories.
|
||||
- **Model-aware retrieval.** Vestige now avoids comparing Qwen and Nomic vectors in the same search/dedup path.
|
||||
|
||||
## What's New in v2.1.0 "Cognitive Sandwich Goes Local"
|
||||
|
||||
v2.1.0 adds an opt-in Claude Code hook harness around the existing Vestige MCP server. The MCP tool surface and database schema stay backward compatible, while the new local Sanhedrin verifier and preflight hooks can inject trusted memory context before Claude answers and check drafts against high-trust Vestige evidence before delivery.
|
||||
v2.1.0 adds an opt-in Claude Code hook harness around the existing Vestige MCP server. The MCP tool surface and database schema stay backward compatible, while preflight hooks can inject trusted memory context before Claude answers. The heavyweight Sanhedrin verifier is optional and can be enabled separately.
|
||||
|
||||
- **Local Sanhedrin Executioner.** The post-response verifier now runs through `mlx_lm.server` with `mlx-community/Qwen3.6-35B-A3B-4bit` by default, so the veto layer can run offline on Apple Silicon without Anthropic API calls.
|
||||
- **One-command Cognitive Sandwich installer.** `scripts/install-sandwich.sh` stages hooks, agents, and a launchd plist, merges the Claude Code hooks block, and prints real verification commands.
|
||||
- **Optional Sanhedrin Executioner.** The post-response verifier is off by default. Users can enable it with an OpenAI-compatible endpoint on x86/Linux/Intel Mac, or add `--with-launchd` on Apple Silicon to run the local MLX Qwen backend.
|
||||
- **One-command Cognitive Sandwich installer.** `scripts/install-sandwich.sh` stages hook files and agents by default, removes old Vestige hook wiring, and leaves all Claude Code hook layers plus the 19 GB model path opt-in.
|
||||
- **Pulse hook backed by `/api/changelog`.** Fresh dream and connection events can be injected into the next Claude Code prompt context without blocking the prompt.
|
||||
- **`VESTIGE_DATA_DIR` support.** `--data-dir` now has an env-var fallback, tilde expansion, secure directory creation, and clear precedence docs.
|
||||
- **NPM release wrapper fixed.** `vestige-mcp-server@2.1.0` now downloads binaries from the matching `v2.1.0` GitHub release tag instead of an old hardcoded release.
|
||||
|
|
@ -211,7 +220,7 @@ Vestige v2.0 ships with a real-time 3D visualization of your AI's memory. Every
|
|||
|
||||
**Tech:** SvelteKit 2 + Svelte 5 + Three.js + Tailwind CSS 4 + WebSocket
|
||||
|
||||
The dashboard runs automatically at `http://localhost:3927/dashboard` when the MCP server starts.
|
||||
Run `vestige dashboard` to open `http://localhost:3927/dashboard`, or set `VESTIGE_DASHBOARD_ENABLED=true` to start it with the MCP server.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -335,8 +344,8 @@ This isn't a key-value store with an embedding model bolted on. Vestige implemen
|
|||
| `consolidate` | Run FSRS-6 decay cycle (also auto-runs every 6 hours) |
|
||||
| `memory_timeline` | Browse chronologically, grouped by day |
|
||||
| `memory_changelog` | Audit trail of state transitions |
|
||||
| `backup` / `export` / `gc` | Database backup, JSON export, garbage collection |
|
||||
| `restore` | Restore from JSON backup |
|
||||
| `backup` / `export` / `gc` | Database backup, JSON/JSONL/portable export, garbage collection |
|
||||
| `restore` | Restore from JSON backup or portable archive |
|
||||
|
||||
### Deep Reference (v2.0.4)
|
||||
| Tool | What It Does |
|
||||
|
|
@ -382,7 +391,7 @@ At the start of every session:
|
|||
| **Language** | Rust 2024 edition (MSRV 1.91) |
|
||||
| **Codebase** | 80,000+ lines, 1,292 tests (366 core + 425 mcp + 497 e2e + 4 doctests) |
|
||||
| **Binary size** | ~20MB |
|
||||
| **Embeddings** | Nomic Embed Text v1.5 (768d → 256d Matryoshka, 8192 context) |
|
||||
| **Embeddings** | Nomic Embed Text v1.5 by default (768d -> 256d Matryoshka, 8192 context); Qwen3 0.6B optional |
|
||||
| **Vector search** | USearch HNSW (20x faster than FAISS) |
|
||||
| **Reranker** | Jina Reranker v1 Turbo (38M params, +15-20% precision) |
|
||||
| **Storage** | SQLite + FTS5 (optional SQLCipher encryption) |
|
||||
|
|
@ -395,17 +404,9 @@ At the start of every session:
|
|||
### Optional Features
|
||||
|
||||
```bash
|
||||
# Metal GPU acceleration (Apple Silicon — faster embedding inference)
|
||||
cargo build --release -p vestige-mcp --features metal
|
||||
|
||||
# Nomic Embed Text v2 MoE (475M params, 305M active, 8 experts)
|
||||
cargo build --release -p vestige-mcp --features nomic-v2
|
||||
|
||||
# Qwen3 Reranker (Candle backend, high-precision cross-encoder)
|
||||
cargo build --release -p vestige-mcp --features qwen3-reranker
|
||||
|
||||
# SQLCipher encryption
|
||||
cargo build --release -p vestige-mcp --no-default-features --features encryption,embeddings,vector-search
|
||||
# Qwen3 embeddings (Candle backend; add metal on Apple Silicon)
|
||||
cargo build --release -p vestige-mcp --features qwen3-embeddings,metal
|
||||
VESTIGE_EMBEDDING_MODEL=qwen3-0.6b vestige consolidate
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -419,6 +420,10 @@ vestige stats --states # Cognitive state breakdown
|
|||
vestige health # System health check
|
||||
vestige consolidate # Run memory maintenance
|
||||
vestige restore <file> # Restore from backup
|
||||
vestige portable-export <file> # Exact cross-device archive
|
||||
vestige portable-import <file> # Import archive into an empty database
|
||||
vestige portable-import <file> --merge # Merge archive into this database
|
||||
vestige sync <file> # Pull/merge/push via file backend
|
||||
vestige dashboard # Open 3D dashboard in browser
|
||||
```
|
||||
|
||||
|
|
@ -465,7 +470,7 @@ Cache: macOS `~/Library/Caches/com.vestige.core/fastembed` | Linux `~/.cache/ves
|
|||
<details>
|
||||
<summary>Dashboard not loading</summary>
|
||||
|
||||
The dashboard starts automatically on port 3927 when the MCP server runs. Check:
|
||||
Run `vestige dashboard` or set `VESTIGE_DASHBOARD_ENABLED=true`, then check:
|
||||
```bash
|
||||
curl http://localhost:3927/api/health
|
||||
# Should return {"status":"healthy",...}
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
[package]
|
||||
name = "vestige-core"
|
||||
version = "2.1.0"
|
||||
version = "2.1.1"
|
||||
edition = "2024"
|
||||
rust-version = "1.91"
|
||||
authors = ["Vestige Team"]
|
||||
|
|
@ -51,8 +51,11 @@ ort-dynamic = ["embeddings", "fastembed/ort-load-dynamic"]
|
|||
# Requires: fastembed with nomic-v2-moe feature
|
||||
nomic-v2 = ["embeddings", "fastembed/nomic-v2-moe"]
|
||||
|
||||
# Qwen3 Reranker (Candle backend, high-precision cross-encoder)
|
||||
qwen3-reranker = ["embeddings", "fastembed/qwen3"]
|
||||
# Qwen3 Embeddings (Candle backend, opt-in for v2.1.1 re-embedding)
|
||||
qwen3-embeddings = ["embeddings", "fastembed/qwen3", "dep:candle-core"]
|
||||
|
||||
# Backwards-compatible feature alias from the original v2.1.0 naming.
|
||||
qwen3-reranker = ["qwen3-embeddings"]
|
||||
|
||||
# Metal GPU acceleration on Apple Silicon (significantly faster inference)
|
||||
metal = ["fastembed/metal"]
|
||||
|
|
@ -96,8 +99,9 @@ notify = "8"
|
|||
# OPTIONAL: Embeddings (fastembed v5 - local ONNX inference, 2026 bleeding edge)
|
||||
# ============================================================================
|
||||
# nomic-embed-text-v1.5: 768 dimensions, 8192 token context, Matryoshka support
|
||||
# v5.11: Adds Nomic v2 MoE (nomic-v2-moe feature) + Qwen3 reranker (qwen3 feature)
|
||||
# fastembed v5 provides Nomic v2 MoE and Qwen3 feature-gated model loaders.
|
||||
fastembed = { version = "5.11", default-features = false, features = ["hf-hub-native-tls", "image-models"], optional = true }
|
||||
candle-core = { version = "0.10.2", optional = true }
|
||||
|
||||
# ============================================================================
|
||||
# OPTIONAL: Vector Search (USearch - HNSW, 20x faster than FAISS)
|
||||
|
|
|
|||
|
|
@ -7,7 +7,13 @@
|
|||
//! - **Default**: Nomic Embed Text v1.5 (ONNX, 768d → 256d Matryoshka, 8192 context)
|
||||
//! - **Optional**: Nomic Embed Text v2 MoE (Candle, 475M params, 305M active, 8 experts)
|
||||
//! Enable with `nomic-v2` feature flag + `metal` for Apple Silicon acceleration.
|
||||
//! - **Optional**: Qwen3 Embedding 0.6B (Candle, 1024d → 256d Matryoshka)
|
||||
//! Enable with `qwen3-embeddings` and `VESTIGE_EMBEDDING_MODEL=qwen3-0.6b`.
|
||||
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
use candle_core::{DType, Device};
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
use fastembed::Qwen3TextEmbedding;
|
||||
use fastembed::{EmbeddingModel, InitOptions, TextEmbedding};
|
||||
use std::sync::{Mutex, OnceLock};
|
||||
|
||||
|
|
@ -31,7 +37,82 @@ pub const BATCH_SIZE: usize = 32;
|
|||
// ============================================================================
|
||||
|
||||
/// Result type for model initialization
|
||||
static EMBEDDING_MODEL_RESULT: OnceLock<Result<Mutex<TextEmbedding>, String>> = OnceLock::new();
|
||||
static EMBEDDING_BACKEND_RESULT: OnceLock<Result<Mutex<EmbeddingBackend>, String>> =
|
||||
OnceLock::new();
|
||||
|
||||
const NOMIC_V15_MODEL_ID: &str = "nomic-ai/nomic-embed-text-v1.5";
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
const QWEN3_06B_MODEL_ID: &str = "Qwen/Qwen3-Embedding-0.6B";
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
enum EmbeddingModelSpec {
|
||||
NomicV15,
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
Qwen3Embedding06B,
|
||||
}
|
||||
|
||||
impl EmbeddingModelSpec {
|
||||
fn selected() -> Result<Self, String> {
|
||||
let requested = std::env::var("VESTIGE_EMBEDDING_MODEL")
|
||||
.ok()
|
||||
.map(|value| value.trim().to_ascii_lowercase())
|
||||
.filter(|value| !value.is_empty())
|
||||
.unwrap_or_else(|| "nomic-v1.5".to_string());
|
||||
|
||||
match requested.as_str() {
|
||||
"nomic" | "nomic-v1.5" | "nomic-embed-text-v1.5" | NOMIC_V15_MODEL_ID => {
|
||||
Ok(Self::NomicV15)
|
||||
}
|
||||
"qwen3" | "qwen3-0.6b" | "qwen3-embedding-0.6b" | "qwen/qwen3-embedding-0.6b" => {
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
{
|
||||
Ok(Self::Qwen3Embedding06B)
|
||||
}
|
||||
#[cfg(not(feature = "qwen3-embeddings"))]
|
||||
{
|
||||
Err(
|
||||
"VESTIGE_EMBEDDING_MODEL requests Qwen3, but vestige-core was not built with the qwen3-embeddings feature"
|
||||
.to_string(),
|
||||
)
|
||||
}
|
||||
}
|
||||
other => Err(format!(
|
||||
"Unsupported VESTIGE_EMBEDDING_MODEL '{}'. Expected 'nomic-v1.5' or 'qwen3-0.6b'.",
|
||||
other
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn model_name(self) -> &'static str {
|
||||
match self {
|
||||
Self::NomicV15 => NOMIC_V15_MODEL_ID,
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
Self::Qwen3Embedding06B => QWEN3_06B_MODEL_ID,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
enum EmbeddingBackend {
|
||||
NomicV15(TextEmbedding),
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
Qwen3Embedding06B(Qwen3TextEmbedding),
|
||||
}
|
||||
|
||||
impl EmbeddingBackend {
|
||||
fn model_name(&self) -> &'static str {
|
||||
match self {
|
||||
Self::NomicV15(_) => NOMIC_V15_MODEL_ID,
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
Self::Qwen3Embedding06B(_) => QWEN3_06B_MODEL_ID,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn qwen3_format_query(query: &str) -> String {
|
||||
format!(
|
||||
"Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: {query}"
|
||||
)
|
||||
}
|
||||
|
||||
/// Get the default cache directory for fastembed models.
|
||||
///
|
||||
|
|
@ -65,10 +146,10 @@ pub(crate) fn get_cache_dir() -> std::path::PathBuf {
|
|||
std::path::PathBuf::from(".fastembed_cache")
|
||||
}
|
||||
|
||||
/// Initialize the global embedding model
|
||||
/// Using nomic-embed-text-v1.5 (768d) - 8192 token context, Matryoshka support
|
||||
fn get_model() -> Result<std::sync::MutexGuard<'static, TextEmbedding>, EmbeddingError> {
|
||||
let result = EMBEDDING_MODEL_RESULT.get_or_init(|| {
|
||||
/// Initialize the global embedding backend selected by `VESTIGE_EMBEDDING_MODEL`.
|
||||
fn get_backend() -> Result<std::sync::MutexGuard<'static, EmbeddingBackend>, EmbeddingError> {
|
||||
let result = EMBEDDING_BACKEND_RESULT.get_or_init(|| {
|
||||
let spec = EmbeddingModelSpec::selected()?;
|
||||
// Get cache directory (respects FASTEMBED_CACHE_PATH env var)
|
||||
let cache_dir = get_cache_dir();
|
||||
|
||||
|
|
@ -77,31 +158,66 @@ fn get_model() -> Result<std::sync::MutexGuard<'static, TextEmbedding>, Embeddin
|
|||
tracing::warn!("Failed to create cache directory {:?}: {}", cache_dir, e);
|
||||
}
|
||||
|
||||
// nomic-embed-text-v1.5: 768 dimensions, 8192 token context
|
||||
// Matryoshka representation learning, fully open source
|
||||
let options = InitOptions::new(EmbeddingModel::NomicEmbedTextV15)
|
||||
.with_show_download_progress(true)
|
||||
.with_cache_dir(cache_dir);
|
||||
match spec {
|
||||
EmbeddingModelSpec::NomicV15 => {
|
||||
let options = InitOptions::new(EmbeddingModel::NomicEmbedTextV15)
|
||||
.with_show_download_progress(true)
|
||||
.with_cache_dir(cache_dir);
|
||||
|
||||
TextEmbedding::try_new(options)
|
||||
.map(Mutex::new)
|
||||
.map_err(|e| {
|
||||
format!(
|
||||
"Failed to initialize nomic-embed-text-v1.5 embedding model: {}. \
|
||||
Ensure ONNX runtime is available and model files can be downloaded.",
|
||||
e
|
||||
TextEmbedding::try_new(options)
|
||||
.map(EmbeddingBackend::NomicV15)
|
||||
.map(Mutex::new)
|
||||
.map_err(|e| {
|
||||
format!(
|
||||
"Failed to initialize {} embedding model: {}. \
|
||||
Ensure ONNX runtime is available and model files can be downloaded.",
|
||||
spec.model_name(),
|
||||
e
|
||||
)
|
||||
})
|
||||
}
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
EmbeddingModelSpec::Qwen3Embedding06B => {
|
||||
let device = qwen3_device();
|
||||
Qwen3TextEmbedding::from_hf(
|
||||
QWEN3_06B_MODEL_ID,
|
||||
&device,
|
||||
DType::F32,
|
||||
MAX_TEXT_LENGTH,
|
||||
)
|
||||
})
|
||||
.map(EmbeddingBackend::Qwen3Embedding06B)
|
||||
.map(Mutex::new)
|
||||
.map_err(|e| {
|
||||
format!(
|
||||
"Failed to initialize {} embedding model: {}. \
|
||||
Ensure Hugging Face model files can be downloaded.",
|
||||
spec.model_name(),
|
||||
e
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
match result {
|
||||
Ok(model) => model
|
||||
Ok(backend) => backend
|
||||
.lock()
|
||||
.map_err(|e| EmbeddingError::ModelInit(format!("Lock poisoned: {}", e))),
|
||||
Err(err) => Err(EmbeddingError::ModelInit(err.clone())),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
fn qwen3_device() -> Device {
|
||||
#[cfg(feature = "metal")]
|
||||
{
|
||||
if let Ok(device) = Device::new_metal(0) {
|
||||
return device;
|
||||
}
|
||||
}
|
||||
Device::Cpu
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// ERROR TYPES
|
||||
// ============================================================================
|
||||
|
|
@ -223,7 +339,7 @@ impl EmbeddingService {
|
|||
|
||||
/// Check if the model is ready
|
||||
pub fn is_ready(&self) -> bool {
|
||||
match get_model() {
|
||||
match get_backend() {
|
||||
Ok(_) => true,
|
||||
Err(e) => {
|
||||
tracing::warn!("Embedding model not ready: {}", e);
|
||||
|
|
@ -234,25 +350,26 @@ impl EmbeddingService {
|
|||
|
||||
/// Check if the model is ready and return the error if not
|
||||
pub fn check_ready(&self) -> Result<(), EmbeddingError> {
|
||||
get_model().map(|_| ())
|
||||
get_backend().map(|_| ())
|
||||
}
|
||||
|
||||
/// Initialize the model (downloads if necessary)
|
||||
pub fn init(&self) -> Result<(), EmbeddingError> {
|
||||
let _model = get_model()?; // Ensures model is loaded and returns any init errors
|
||||
let _model = get_backend()?; // Ensures model is loaded and returns any init errors
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Get the model name
|
||||
pub fn model_name(&self) -> &'static str {
|
||||
#[cfg(feature = "nomic-v2")]
|
||||
if let Some(Ok(backend)) = EMBEDDING_BACKEND_RESULT.get()
|
||||
&& let Ok(backend) = backend.lock()
|
||||
{
|
||||
"nomic-ai/nomic-embed-text-v2-moe"
|
||||
}
|
||||
#[cfg(not(feature = "nomic-v2"))]
|
||||
{
|
||||
"nomic-ai/nomic-embed-text-v1.5"
|
||||
return backend.model_name();
|
||||
}
|
||||
|
||||
EmbeddingModelSpec::selected()
|
||||
.unwrap_or(EmbeddingModelSpec::NomicV15)
|
||||
.model_name()
|
||||
}
|
||||
|
||||
/// Get the embedding dimensions
|
||||
|
|
@ -268,7 +385,7 @@ impl EmbeddingService {
|
|||
));
|
||||
}
|
||||
|
||||
let mut model = get_model()?;
|
||||
let mut backend = get_backend()?;
|
||||
|
||||
// Truncate if too long (char-boundary safe)
|
||||
let text = if text.len() > MAX_TEXT_LENGTH {
|
||||
|
|
@ -281,9 +398,15 @@ impl EmbeddingService {
|
|||
text
|
||||
};
|
||||
|
||||
let embeddings = model
|
||||
.embed(vec![text], None)
|
||||
.map_err(|e| EmbeddingError::EmbeddingFailed(e.to_string()))?;
|
||||
let embeddings = match &mut *backend {
|
||||
EmbeddingBackend::NomicV15(model) => model
|
||||
.embed(vec![text], None)
|
||||
.map_err(|e| EmbeddingError::EmbeddingFailed(e.to_string()))?,
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
EmbeddingBackend::Qwen3Embedding06B(model) => model
|
||||
.embed(&[text])
|
||||
.map_err(|e| EmbeddingError::EmbeddingFailed(e.to_string()))?,
|
||||
};
|
||||
|
||||
if embeddings.is_empty() {
|
||||
return Err(EmbeddingError::EmbeddingFailed(
|
||||
|
|
@ -294,13 +417,26 @@ impl EmbeddingService {
|
|||
Ok(Embedding::new(matryoshka_truncate(embeddings[0].clone())))
|
||||
}
|
||||
|
||||
/// Generate an embedding for retrieval queries.
|
||||
///
|
||||
/// Qwen3 uses instruction-formatted queries against raw document embeddings;
|
||||
/// Nomic remains symmetric and receives the query unchanged.
|
||||
pub fn embed_query(&self, query: &str) -> Result<Embedding, EmbeddingError> {
|
||||
if self.model_name().to_ascii_lowercase().contains("qwen3") {
|
||||
let formatted = qwen3_format_query(query);
|
||||
self.embed(&formatted)
|
||||
} else {
|
||||
self.embed(query)
|
||||
}
|
||||
}
|
||||
|
||||
/// Generate embeddings for multiple texts (batch processing)
|
||||
pub fn embed_batch(&self, texts: &[&str]) -> Result<Vec<Embedding>, EmbeddingError> {
|
||||
if texts.is_empty() {
|
||||
return Ok(vec![]);
|
||||
}
|
||||
|
||||
let mut model = get_model()?;
|
||||
let mut backend = get_backend()?;
|
||||
let mut all_embeddings = Vec::with_capacity(texts.len());
|
||||
|
||||
// Process in batches for efficiency
|
||||
|
|
@ -320,9 +456,15 @@ impl EmbeddingService {
|
|||
})
|
||||
.collect();
|
||||
|
||||
let embeddings = model
|
||||
.embed(truncated, None)
|
||||
.map_err(|e| EmbeddingError::EmbeddingFailed(e.to_string()))?;
|
||||
let embeddings = match &mut *backend {
|
||||
EmbeddingBackend::NomicV15(model) => model
|
||||
.embed(truncated, None)
|
||||
.map_err(|e| EmbeddingError::EmbeddingFailed(e.to_string()))?,
|
||||
#[cfg(feature = "qwen3-embeddings")]
|
||||
EmbeddingBackend::Qwen3Embedding06B(model) => model
|
||||
.embed(&truncated)
|
||||
.map_err(|e| EmbeddingError::EmbeddingFailed(e.to_string()))?,
|
||||
};
|
||||
|
||||
for emb in embeddings {
|
||||
all_embeddings.push(Embedding::new(matryoshka_truncate(emb)));
|
||||
|
|
@ -481,6 +623,14 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_qwen3_query_format() {
|
||||
let formatted = qwen3_format_query("rust memory portability");
|
||||
assert!(formatted.starts_with("Instruct:"));
|
||||
assert!(formatted.contains("retrieve relevant passages"));
|
||||
assert!(formatted.ends_with("Query: rust memory portability"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_embedding_normalize() {
|
||||
let mut emb = Embedding::new(vec![3.0, 4.0]);
|
||||
|
|
|
|||
|
|
@ -13,6 +13,7 @@ mod code;
|
|||
mod hybrid;
|
||||
mod local;
|
||||
|
||||
#[cfg(feature = "vector-search")]
|
||||
pub(crate) use local::get_cache_dir;
|
||||
pub use local::{
|
||||
BATCH_SIZE, EMBEDDING_DIMENSIONS, Embedding, EmbeddingError, EmbeddingService, MAX_TEXT_LENGTH,
|
||||
|
|
|
|||
|
|
@ -153,7 +153,8 @@ pub use fsrs::{
|
|||
// Storage layer
|
||||
pub use storage::{
|
||||
ConnectionRecord, ConsolidationHistoryRecord, DreamHistoryRecord, InsightRecord,
|
||||
IntentionRecord, Result, SmartIngestResult, StateTransitionRecord, Storage, StorageError,
|
||||
IntentionRecord, PORTABLE_ARCHIVE_FORMAT, PortableArchive, PortableImportMode,
|
||||
PortableImportReport, Result, SmartIngestResult, StateTransitionRecord, Storage, StorageError,
|
||||
};
|
||||
|
||||
// Consolidation (sleep-inspired memory processing)
|
||||
|
|
|
|||
|
|
@ -247,8 +247,14 @@ pub struct MemoryStats {
|
|||
pub newest_memory: Option<DateTime<Utc>>,
|
||||
/// Number of nodes with semantic embeddings
|
||||
pub nodes_with_embeddings: i64,
|
||||
/// Number of nodes with embeddings generated by the active embedding model family
|
||||
pub nodes_with_active_embeddings: i64,
|
||||
/// Number of nodes whose stored embeddings belong to a different model family
|
||||
pub nodes_with_mismatched_embeddings: i64,
|
||||
/// Embedding model used (if any)
|
||||
pub embedding_model: Option<String>,
|
||||
/// Embedding model family currently configured for new queries and writes
|
||||
pub active_embedding_model: Option<String>,
|
||||
}
|
||||
|
||||
impl Default for MemoryStats {
|
||||
|
|
@ -262,7 +268,10 @@ impl Default for MemoryStats {
|
|||
oldest_memory: None,
|
||||
newest_memory: None,
|
||||
nodes_with_embeddings: 0,
|
||||
nodes_with_active_embeddings: 0,
|
||||
nodes_with_mismatched_embeddings: 0,
|
||||
embedding_model: None,
|
||||
active_embedding_model: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -59,6 +59,11 @@ pub const MIGRATIONS: &[Migration] = &[
|
|||
description: "v2.0.7 Cleanup: drop dead knowledge_edges and compressed_memories tables",
|
||||
up: MIGRATION_V11_UP,
|
||||
},
|
||||
Migration {
|
||||
version: 12,
|
||||
description: "v2.1.1 Sync: tombstones for merge-capable portable storage",
|
||||
up: MIGRATION_V12_UP,
|
||||
},
|
||||
];
|
||||
|
||||
/// A database migration
|
||||
|
|
@ -681,6 +686,26 @@ DROP TABLE IF EXISTS compressed_memories;
|
|||
UPDATE schema_version SET version = 11, applied_at = datetime('now');
|
||||
"#;
|
||||
|
||||
/// V12: Merge-capable sync tombstones.
|
||||
///
|
||||
/// Portable sync needs to propagate deletions between devices. `knowledge_nodes`
|
||||
/// remains the source of truth for live memories; this table records deletes so
|
||||
/// another device can remove the same memory during a merge import.
|
||||
const MIGRATION_V12_UP: &str = r#"
|
||||
CREATE TABLE IF NOT EXISTS sync_tombstones (
|
||||
table_name TEXT NOT NULL,
|
||||
row_id TEXT NOT NULL,
|
||||
deleted_at TEXT NOT NULL,
|
||||
reason TEXT,
|
||||
PRIMARY KEY (table_name, row_id)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_sync_tombstones_deleted_at
|
||||
ON sync_tombstones(deleted_at);
|
||||
|
||||
UPDATE schema_version SET version = 12, applied_at = datetime('now');
|
||||
"#;
|
||||
|
||||
/// Get current schema version from database
|
||||
pub fn get_current_version(conn: &rusqlite::Connection) -> rusqlite::Result<u32> {
|
||||
conn.query_row(
|
||||
|
|
@ -730,17 +755,17 @@ mod tests {
|
|||
/// version after `apply_migrations` runs all migrations end-to-end, and
|
||||
/// neither of the dead tables V11 drops must exist afterwards.
|
||||
#[test]
|
||||
fn test_apply_migrations_advances_to_v11_and_drops_dead_tables() {
|
||||
fn test_apply_migrations_advances_to_v12_and_drops_dead_tables() {
|
||||
let conn = rusqlite::Connection::open_in_memory().expect("open in-memory");
|
||||
|
||||
// Pre-requisite: schema_version must be bootstrapped by V1.
|
||||
apply_migrations(&conn).expect("apply_migrations succeeds");
|
||||
|
||||
// 1. schema_version advanced to V11
|
||||
// 1. schema_version advanced to V12
|
||||
let version = get_current_version(&conn).expect("read schema_version");
|
||||
assert_eq!(
|
||||
version, 11,
|
||||
"schema_version must be 11 after all migrations"
|
||||
version, 12,
|
||||
"schema_version must be 12 after all migrations"
|
||||
);
|
||||
|
||||
// 2. knowledge_edges is gone (V11 drops it)
|
||||
|
|
@ -768,6 +793,19 @@ mod tests {
|
|||
compressed_memories_rows, 0,
|
||||
"compressed_memories table must be dropped by V11"
|
||||
);
|
||||
|
||||
// 4. sync_tombstones exists (V12 creates it)
|
||||
let sync_tombstone_rows: i64 = conn
|
||||
.query_row(
|
||||
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name='sync_tombstones'",
|
||||
[],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.expect("query sqlite_master");
|
||||
assert_eq!(
|
||||
sync_tombstone_rows, 1,
|
||||
"sync_tombstones table must be created by V12"
|
||||
);
|
||||
}
|
||||
|
||||
/// V11 must be idempotent on replay — if the tables were already dropped
|
||||
|
|
@ -789,6 +827,6 @@ mod tests {
|
|||
apply_migrations(&conn).expect("V11 replay must be idempotent");
|
||||
|
||||
let version = get_current_version(&conn).expect("read schema_version");
|
||||
assert_eq!(version, 11, "schema_version back at 11 after replay");
|
||||
assert_eq!(version, 12, "schema_version back at 12 after replay");
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -7,10 +7,16 @@
|
|||
//! - Temporal memory support
|
||||
|
||||
mod migrations;
|
||||
mod portable;
|
||||
mod sqlite;
|
||||
|
||||
pub use migrations::MIGRATIONS;
|
||||
pub use sqlite::{
|
||||
ConnectionRecord, ConsolidationHistoryRecord, DreamHistoryRecord, InsightRecord,
|
||||
IntentionRecord, Result, SmartIngestResult, StateTransitionRecord, Storage, StorageError,
|
||||
pub use portable::{
|
||||
PORTABLE_ARCHIVE_FORMAT, PortableArchive, PortableImportMode, PortableImportReport,
|
||||
PortableTable, PortableValue,
|
||||
};
|
||||
pub use sqlite::{
|
||||
ConnectionRecord, ConsolidationHistoryRecord, DreamHistoryRecord, FilePortableSyncBackend,
|
||||
InsightRecord, IntentionRecord, PortableSyncBackend, PortableSyncReport, Result,
|
||||
SmartIngestResult, StateTransitionRecord, Storage, StorageError,
|
||||
};
|
||||
|
|
|
|||
171
crates/vestige-core/src/storage/portable.rs
Normal file
171
crates/vestige-core/src/storage/portable.rs
Normal file
|
|
@ -0,0 +1,171 @@
|
|||
//! Portable archive types for exact Vestige-to-Vestige transfer.
|
||||
//!
|
||||
//! This format preserves SQLite row data instead of re-ingesting memories. It is
|
||||
//! intentionally storage-level: import can keep IDs, FSRS state, graph edges,
|
||||
//! suppression state, embeddings, and audit/history rows intact.
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use rusqlite::types::Value;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Current portable archive format identifier.
|
||||
pub const PORTABLE_ARCHIVE_FORMAT: &str = "vestige.portable.v1";
|
||||
|
||||
/// Full exact portable archive.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct PortableArchive {
|
||||
/// Stable format marker used for compatibility checks.
|
||||
pub archive_format: String,
|
||||
/// Vestige version that produced the archive.
|
||||
pub vestige_version: String,
|
||||
/// SQLite schema version of the source database.
|
||||
pub schema_version: u32,
|
||||
/// Archive creation timestamp.
|
||||
pub exported_at: DateTime<Utc>,
|
||||
/// Export mode. v1 only writes "exact".
|
||||
pub mode: String,
|
||||
/// Dumped storage tables in deterministic import order.
|
||||
pub tables: Vec<PortableTable>,
|
||||
}
|
||||
|
||||
impl PortableArchive {
|
||||
/// Count all rows across all tables.
|
||||
pub fn total_rows(&self) -> usize {
|
||||
self.tables.iter().map(|table| table.rows.len()).sum()
|
||||
}
|
||||
}
|
||||
|
||||
/// One table in a portable archive.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct PortableTable {
|
||||
/// SQLite table name.
|
||||
pub name: String,
|
||||
/// Column names in row value order.
|
||||
pub columns: Vec<String>,
|
||||
/// Raw rows. Each row has the same order as `columns`.
|
||||
pub rows: Vec<Vec<PortableValue>>,
|
||||
}
|
||||
|
||||
/// SQLite value encoded in JSON.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(tag = "type", content = "value", rename_all = "camelCase")]
|
||||
pub enum PortableValue {
|
||||
/// SQL NULL.
|
||||
Null,
|
||||
/// SQL INTEGER.
|
||||
Integer(i64),
|
||||
/// SQL REAL.
|
||||
Real(f64),
|
||||
/// SQL TEXT.
|
||||
Text(String),
|
||||
/// SQL BLOB, hex encoded.
|
||||
Blob(String),
|
||||
}
|
||||
|
||||
impl PortableValue {
|
||||
/// Convert this portable value back into a rusqlite owned value.
|
||||
pub(crate) fn to_sql_value(&self) -> Result<Value, String> {
|
||||
match self {
|
||||
Self::Null => Ok(Value::Null),
|
||||
Self::Integer(value) => Ok(Value::Integer(*value)),
|
||||
Self::Real(value) => Ok(Value::Real(*value)),
|
||||
Self::Text(value) => Ok(Value::Text(value.clone())),
|
||||
Self::Blob(value) => decode_hex(value).map(Value::Blob),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Import behavior for duplicate primary keys.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum PortableImportMode {
|
||||
/// Reject import if user data already exists, then insert rows exactly.
|
||||
EmptyOnly,
|
||||
/// Merge archive rows into an existing database.
|
||||
///
|
||||
/// This mode is intended for file-backed sync between devices. It applies
|
||||
/// tombstones, upserts row-keyed state, and appends audit/history rows.
|
||||
Merge,
|
||||
}
|
||||
|
||||
/// Summary of an exact portable import.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct PortableImportReport {
|
||||
/// Number of imported tables.
|
||||
pub tables_imported: usize,
|
||||
/// Number of imported rows.
|
||||
pub rows_imported: usize,
|
||||
/// Number of archive tables skipped because the target schema lacks them.
|
||||
pub tables_skipped: usize,
|
||||
/// Whether FTS was rebuilt after import.
|
||||
pub fts_rebuilt: bool,
|
||||
/// Number of rows inserted.
|
||||
#[serde(default)]
|
||||
pub rows_inserted: usize,
|
||||
/// Number of existing rows updated/replaced.
|
||||
#[serde(default)]
|
||||
pub rows_updated: usize,
|
||||
/// Number of rows skipped because local state was newer or unsupported.
|
||||
#[serde(default)]
|
||||
pub rows_skipped: usize,
|
||||
/// Number of local rows deleted by imported tombstones.
|
||||
#[serde(default)]
|
||||
pub rows_deleted: usize,
|
||||
/// Number of merge conflicts resolved by keeping local state.
|
||||
#[serde(default)]
|
||||
pub conflicts_kept_local: usize,
|
||||
}
|
||||
|
||||
pub(crate) fn encode_hex(bytes: &[u8]) -> String {
|
||||
const HEX: &[u8; 16] = b"0123456789abcdef";
|
||||
let mut out = String::with_capacity(bytes.len() * 2);
|
||||
for &byte in bytes {
|
||||
out.push(HEX[(byte >> 4) as usize] as char);
|
||||
out.push(HEX[(byte & 0x0f) as usize] as char);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn decode_hex(input: &str) -> Result<Vec<u8>, String> {
|
||||
if !input.len().is_multiple_of(2) {
|
||||
return Err("hex blob has odd length".to_string());
|
||||
}
|
||||
|
||||
let mut out = Vec::with_capacity(input.len() / 2);
|
||||
let bytes = input.as_bytes();
|
||||
for chunk in bytes.chunks_exact(2) {
|
||||
let high = hex_value(chunk[0])?;
|
||||
let low = hex_value(chunk[1])?;
|
||||
out.push((high << 4) | low);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn hex_value(byte: u8) -> Result<u8, String> {
|
||||
match byte {
|
||||
b'0'..=b'9' => Ok(byte - b'0'),
|
||||
b'a'..=b'f' => Ok(byte - b'a' + 10),
|
||||
b'A'..=b'F' => Ok(byte - b'A' + 10),
|
||||
_ => Err(format!("invalid hex byte: {}", byte as char)),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn hex_round_trip() {
|
||||
let bytes = vec![0, 1, 2, 15, 16, 127, 128, 255];
|
||||
let encoded = encode_hex(&bytes);
|
||||
assert_eq!(decode_hex(&encoded).unwrap(), bytes);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_invalid_hex() {
|
||||
assert!(decode_hex("f").is_err());
|
||||
assert!(decode_hex("zz").is_err());
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,6 +1,6 @@
|
|||
[package]
|
||||
name = "vestige-mcp"
|
||||
version = "2.1.0"
|
||||
version = "2.1.1"
|
||||
edition = "2024"
|
||||
description = "Cognitive memory MCP server for Claude - FSRS-6, spreading activation, synaptic tagging, 3D dashboard, and 130 years of memory research"
|
||||
authors = ["samvallad33"]
|
||||
|
|
@ -21,6 +21,9 @@ ort-download = ["embeddings", "vestige-core/ort-download"]
|
|||
# Usage: cargo build --no-default-features --features ort-dynamic,vector-search
|
||||
# Runtime: export ORT_DYLIB_PATH=$(brew --prefix onnxruntime)/lib/libonnxruntime.dylib
|
||||
ort-dynamic = ["embeddings", "vestige-core/ort-dynamic"]
|
||||
qwen3-embeddings = ["embeddings", "vestige-core/qwen3-embeddings"]
|
||||
qwen3-reranker = ["qwen3-embeddings"]
|
||||
metal = ["embeddings", "vestige-core/metal"]
|
||||
|
||||
[[bin]]
|
||||
name = "vestige-mcp"
|
||||
|
|
@ -44,7 +47,7 @@ path = "src/bin/cli.rs"
|
|||
# Only `bundled-sqlite` is always on. `embeddings` and `vector-search` are
|
||||
# toggled via vestige-mcp's own feature flags below so `--no-default-features`
|
||||
# actually works (previously hardcoded here, which silently defeated the flag).
|
||||
vestige-core = { version = "2.1.0", path = "../vestige-core", default-features = false, features = ["bundled-sqlite"] }
|
||||
vestige-core = { version = "2.1.1", path = "../vestige-core", default-features = false, features = ["bundled-sqlite"] }
|
||||
|
||||
# ============================================================================
|
||||
# MCP Server Dependencies
|
||||
|
|
|
|||
|
|
@ -8,14 +8,13 @@ use std::io::{BufWriter, Write};
|
|||
use std::path::Path;
|
||||
use std::path::PathBuf;
|
||||
use std::process::Command;
|
||||
use std::sync::Arc;
|
||||
use std::sync::{Arc, OnceLock};
|
||||
|
||||
use anyhow::Context;
|
||||
use chrono::{NaiveDate, Utc};
|
||||
use clap::{Parser, Subcommand};
|
||||
use colored::Colorize;
|
||||
use directories::ProjectDirs;
|
||||
use vestige_core::{IngestInput, Storage};
|
||||
use vestige_core::{IngestInput, PortableImportMode, Storage};
|
||||
|
||||
/// Vestige - Cognitive Memory System CLI
|
||||
#[derive(Parser)]
|
||||
|
|
@ -27,10 +26,16 @@ use vestige_core::{IngestInput, Storage};
|
|||
long_about = "Vestige is a cognitive memory system based on 130 years of memory research.\n\nIt implements FSRS-6, spreading activation, synaptic tagging, and more."
|
||||
)]
|
||||
struct Cli {
|
||||
/// Use a specific Vestige data directory for this command.
|
||||
#[arg(long, global = true, value_name = "DIR")]
|
||||
data_dir: Option<PathBuf>,
|
||||
|
||||
#[command(subcommand)]
|
||||
command: Commands,
|
||||
}
|
||||
|
||||
static CLI_DB_PATH: OnceLock<PathBuf> = OnceLock::new();
|
||||
|
||||
#[derive(Subcommand)]
|
||||
enum Commands {
|
||||
/// Show memory statistics
|
||||
|
|
@ -52,7 +57,7 @@ enum Commands {
|
|||
|
||||
/// Update Vestige binaries from the latest GitHub release
|
||||
Update {
|
||||
/// Install a specific release tag instead of latest (example: v2.1.0)
|
||||
/// Install a specific release tag instead of latest (example: v2.1.1)
|
||||
#[arg(long)]
|
||||
version: Option<String>,
|
||||
|
||||
|
|
@ -92,6 +97,27 @@ enum Commands {
|
|||
since: Option<String>,
|
||||
},
|
||||
|
||||
/// Export an exact portable archive for Vestige-to-Vestige transfer
|
||||
PortableExport {
|
||||
/// Output archive path
|
||||
output: PathBuf,
|
||||
},
|
||||
|
||||
/// Import an exact portable archive
|
||||
PortableImport {
|
||||
/// Input archive path
|
||||
input: PathBuf,
|
||||
/// Merge into the current database instead of requiring an empty target
|
||||
#[arg(long)]
|
||||
merge: bool,
|
||||
},
|
||||
|
||||
/// Two-way sync with a file-backed portable archive
|
||||
Sync {
|
||||
/// Sync archive path, often in Dropbox/iCloud/Syncthing/Git
|
||||
archive: PathBuf,
|
||||
},
|
||||
|
||||
/// Garbage collect stale memories below retention threshold
|
||||
Gc {
|
||||
/// Minimum retention strength to keep (delete below this)
|
||||
|
|
@ -150,6 +176,13 @@ enum Commands {
|
|||
fn main() -> anyhow::Result<()> {
|
||||
let cli = Cli::parse();
|
||||
|
||||
if let Some(data_dir) = cli.data_dir {
|
||||
let db_path = Storage::db_path_for_data_dir(data_dir)?;
|
||||
CLI_DB_PATH
|
||||
.set(db_path)
|
||||
.map_err(|_| anyhow::anyhow!("data directory was initialized more than once"))?;
|
||||
}
|
||||
|
||||
match cli.command {
|
||||
Commands::Stats { tagging, states } => run_stats(tagging, states),
|
||||
Commands::Health => run_health(),
|
||||
|
|
@ -167,6 +200,9 @@ fn main() -> anyhow::Result<()> {
|
|||
tags,
|
||||
since,
|
||||
} => run_export(output, format, tags, since),
|
||||
Commands::PortableExport { output } => run_portable_export(output),
|
||||
Commands::PortableImport { input, merge } => run_portable_import(input, merge),
|
||||
Commands::Sync { archive } => run_sync(archive),
|
||||
Commands::Gc {
|
||||
min_retention,
|
||||
max_age_days,
|
||||
|
|
@ -465,7 +501,7 @@ fn run_update(
|
|||
|
||||
/// Run stats command
|
||||
fn run_stats(show_tagging: bool, show_states: bool) -> anyhow::Result<()> {
|
||||
let storage = Storage::new(None)?;
|
||||
let storage = open_storage()?;
|
||||
let stats = storage.get_stats()?;
|
||||
|
||||
println!("{}", "=== Vestige Memory Statistics ===".cyan().bold());
|
||||
|
|
@ -499,8 +535,20 @@ fn run_stats(show_tagging: bool, show_states: bool) -> anyhow::Result<()> {
|
|||
stats.nodes_with_embeddings
|
||||
);
|
||||
|
||||
if let Some(model) = &stats.active_embedding_model {
|
||||
println!("{}: {}", "Active Embedding Model".white().bold(), model);
|
||||
}
|
||||
|
||||
if let Some(model) = &stats.embedding_model {
|
||||
println!("{}: {}", "Embedding Model".white().bold(), model);
|
||||
println!("{}: {}", "Stored Embedding Model".white().bold(), model);
|
||||
}
|
||||
|
||||
if stats.nodes_with_mismatched_embeddings > 0 {
|
||||
println!(
|
||||
"{}: {}",
|
||||
"Mismatched Embeddings".white().bold(),
|
||||
stats.nodes_with_mismatched_embeddings
|
||||
);
|
||||
}
|
||||
|
||||
if let Some(oldest) = stats.oldest_memory {
|
||||
|
|
@ -520,13 +568,13 @@ fn run_stats(show_tagging: bool, show_states: bool) -> anyhow::Result<()> {
|
|||
|
||||
// Embedding coverage
|
||||
let embedding_coverage = if stats.total_nodes > 0 {
|
||||
(stats.nodes_with_embeddings as f64 / stats.total_nodes as f64) * 100.0
|
||||
(stats.nodes_with_active_embeddings as f64 / stats.total_nodes as f64) * 100.0
|
||||
} else {
|
||||
0.0
|
||||
};
|
||||
println!(
|
||||
"{}: {:.1}%",
|
||||
"Embedding Coverage".white().bold(),
|
||||
"Active Embedding Coverage".white().bold(),
|
||||
embedding_coverage
|
||||
);
|
||||
|
||||
|
|
@ -651,7 +699,7 @@ fn print_distribution_bar(label: &str, count: usize, total: usize, color: &str)
|
|||
|
||||
/// Run health check
|
||||
fn run_health() -> anyhow::Result<()> {
|
||||
let storage = Storage::new(None)?;
|
||||
let storage = open_storage()?;
|
||||
let stats = storage.get_stats()?;
|
||||
|
||||
println!("{}", "=== Vestige Health Check ===".cyan().bold());
|
||||
|
|
@ -690,13 +738,13 @@ fn run_health() -> anyhow::Result<()> {
|
|||
|
||||
// Embedding coverage
|
||||
let embedding_coverage = if stats.total_nodes > 0 {
|
||||
(stats.nodes_with_embeddings as f64 / stats.total_nodes as f64) * 100.0
|
||||
(stats.nodes_with_active_embeddings as f64 / stats.total_nodes as f64) * 100.0
|
||||
} else {
|
||||
0.0
|
||||
};
|
||||
println!(
|
||||
"{}: {:.1}%",
|
||||
"Embedding Coverage".white(),
|
||||
"Active Embedding Coverage".white(),
|
||||
embedding_coverage
|
||||
);
|
||||
println!(
|
||||
|
|
@ -721,14 +769,18 @@ fn run_health() -> anyhow::Result<()> {
|
|||
warnings.push("Many memories are due for review");
|
||||
}
|
||||
|
||||
if stats.total_nodes > 0 && stats.nodes_with_embeddings == 0 {
|
||||
warnings.push("No embeddings generated - semantic search unavailable");
|
||||
if stats.total_nodes > 0 && stats.nodes_with_active_embeddings == 0 {
|
||||
warnings.push("No active-model embeddings generated - semantic search unavailable");
|
||||
}
|
||||
|
||||
if embedding_coverage < 50.0 && stats.total_nodes > 10 {
|
||||
warnings.push("Low embedding coverage - run consolidation to improve semantic search");
|
||||
}
|
||||
|
||||
if stats.nodes_with_mismatched_embeddings > 0 {
|
||||
warnings.push("Stored embeddings from another model are present - run consolidation after changing embedding models");
|
||||
}
|
||||
|
||||
if !warnings.is_empty() {
|
||||
println!();
|
||||
println!("{}", "Warnings:".yellow().bold());
|
||||
|
|
@ -749,9 +801,9 @@ fn run_health() -> anyhow::Result<()> {
|
|||
recommendations.push("Review due memories to strengthen retention.");
|
||||
}
|
||||
|
||||
if stats.nodes_with_embeddings < stats.total_nodes {
|
||||
if stats.nodes_with_active_embeddings < stats.total_nodes {
|
||||
recommendations
|
||||
.push("Run 'vestige consolidate' to generate embeddings for better semantic search.");
|
||||
.push("Run 'vestige consolidate' to generate active-model embeddings for better semantic search.");
|
||||
}
|
||||
|
||||
if stats.total_nodes > 100 && stats.average_retention < 0.7 {
|
||||
|
|
@ -788,7 +840,7 @@ fn run_consolidate() -> anyhow::Result<()> {
|
|||
println!("Running memory consolidation cycle...");
|
||||
println!();
|
||||
|
||||
let storage = Storage::new(None)?;
|
||||
let storage = open_storage()?;
|
||||
let result = storage.run_consolidation()?;
|
||||
|
||||
println!(
|
||||
|
|
@ -834,7 +886,49 @@ fn run_restore(backup_path: PathBuf) -> anyhow::Result<()> {
|
|||
println!("Loading backup from: {}", backup_path.display());
|
||||
|
||||
// Read and parse backup
|
||||
let backup_content = std::fs::read_to_string(&backup_path)?;
|
||||
let backup_bytes = std::fs::read(&backup_path)?;
|
||||
if backup_bytes.starts_with(b"SQLite format 3\0") {
|
||||
anyhow::bail!(
|
||||
"{} is a raw SQLite database backup, not a JSON restore file. Use portable-export/portable-import for cross-device transfer, or replace the database file manually while Vestige is stopped.",
|
||||
backup_path.display()
|
||||
);
|
||||
}
|
||||
let backup_content = String::from_utf8(backup_bytes)
|
||||
.with_context(|| format!("{} is not UTF-8 JSON", backup_path.display()))?;
|
||||
|
||||
if let Ok(archive) = serde_json::from_str::<vestige_core::PortableArchive>(&backup_content)
|
||||
&& archive.archive_format == vestige_core::PORTABLE_ARCHIVE_FORMAT
|
||||
{
|
||||
println!("Detected portable archive.");
|
||||
println!("{}: {}", "Format".white().bold(), archive.archive_format);
|
||||
println!("{}: {}", "Schema".white().bold(), archive.schema_version);
|
||||
println!("{}: {}", "Tables".white().bold(), archive.tables.len());
|
||||
println!("{}: {}", "Rows".white().bold(), archive.total_rows());
|
||||
println!();
|
||||
|
||||
let storage = open_storage()?;
|
||||
let report = storage.import_portable_archive(&archive, PortableImportMode::EmptyOnly)?;
|
||||
|
||||
println!(
|
||||
"{}: {}",
|
||||
"Tables imported".white().bold(),
|
||||
report.tables_imported
|
||||
);
|
||||
println!(
|
||||
"{}: {}",
|
||||
"Rows imported".white().bold(),
|
||||
report.rows_imported
|
||||
);
|
||||
println!(
|
||||
"{}: {}",
|
||||
"Tables skipped".white().bold(),
|
||||
report.tables_skipped
|
||||
);
|
||||
println!("{}: {}", "FTS rebuilt".white().bold(), report.fts_rebuilt);
|
||||
println!();
|
||||
println!("{}", "Portable restore complete.".green().bold());
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
#[derive(serde::Deserialize)]
|
||||
struct BackupWrapper {
|
||||
|
|
@ -857,16 +951,27 @@ fn run_restore(backup_path: PathBuf) -> anyhow::Result<()> {
|
|||
source: Option<String>,
|
||||
}
|
||||
|
||||
let wrapper: Vec<BackupWrapper> = serde_json::from_str(&backup_content)?;
|
||||
let recall_result: RecallResult = serde_json::from_str(&wrapper[0].text)?;
|
||||
let memories = recall_result.results;
|
||||
let memories = if let Ok(wrapper) = serde_json::from_str::<Vec<BackupWrapper>>(&backup_content)
|
||||
{
|
||||
let first = wrapper.first().context("backup wrapper is empty")?;
|
||||
let recall_result: RecallResult = serde_json::from_str(&first.text)?;
|
||||
recall_result.results
|
||||
} else if let Ok(recall_result) = serde_json::from_str::<RecallResult>(&backup_content) {
|
||||
recall_result.results
|
||||
} else if let Ok(memories) = serde_json::from_str::<Vec<MemoryBackup>>(&backup_content) {
|
||||
memories
|
||||
} else {
|
||||
anyhow::bail!(
|
||||
"Unrecognized backup format. Expected portable archive, MCP wrapper, RecallResult, or array of memories."
|
||||
);
|
||||
};
|
||||
|
||||
println!("Found {} memories to restore", memories.len());
|
||||
println!();
|
||||
|
||||
// Initialize storage
|
||||
println!("Initializing storage...");
|
||||
let storage = Storage::new(None)?;
|
||||
let storage = open_storage()?;
|
||||
|
||||
println!("Generating embeddings and ingesting memories...");
|
||||
println!();
|
||||
|
|
@ -916,8 +1021,8 @@ fn run_restore(backup_path: PathBuf) -> anyhow::Result<()> {
|
|||
println!("{}: {}", "Total Nodes".white(), stats.total_nodes);
|
||||
println!(
|
||||
"{}: {}",
|
||||
"With Embeddings".white(),
|
||||
stats.nodes_with_embeddings
|
||||
"Active Embeddings".white(),
|
||||
stats.nodes_with_active_embeddings
|
||||
);
|
||||
|
||||
Ok(())
|
||||
|
|
@ -925,9 +1030,20 @@ fn run_restore(backup_path: PathBuf) -> anyhow::Result<()> {
|
|||
|
||||
/// Get the default database path
|
||||
fn get_default_db_path() -> anyhow::Result<PathBuf> {
|
||||
let proj_dirs = ProjectDirs::from("com", "vestige", "core")
|
||||
.ok_or_else(|| anyhow::anyhow!("Could not determine project directories"))?;
|
||||
Ok(proj_dirs.data_dir().join("vestige.db"))
|
||||
if let Some(path) = CLI_DB_PATH.get() {
|
||||
Ok(path.clone())
|
||||
} else {
|
||||
Ok(Storage::default_db_path()?)
|
||||
}
|
||||
}
|
||||
|
||||
/// Open storage using the CLI-selected data directory, if one was provided.
|
||||
fn open_storage() -> anyhow::Result<Storage> {
|
||||
if let Some(path) = CLI_DB_PATH.get() {
|
||||
Ok(Storage::new(Some(path.clone()))?)
|
||||
} else {
|
||||
Ok(Storage::new(None)?)
|
||||
}
|
||||
}
|
||||
|
||||
/// Fetch all nodes from storage using pagination
|
||||
|
|
@ -963,7 +1079,7 @@ fn run_backup(output: PathBuf) -> anyhow::Result<()> {
|
|||
// Open storage to flush WAL before copying
|
||||
println!("Flushing WAL checkpoint...");
|
||||
{
|
||||
let storage = Storage::new(None)?;
|
||||
let storage = open_storage()?;
|
||||
// get_stats triggers a read so the connection is active, then drop flushes
|
||||
let _ = storage.get_stats()?;
|
||||
}
|
||||
|
|
@ -1050,7 +1166,7 @@ fn run_export(
|
|||
})
|
||||
.unwrap_or_default();
|
||||
|
||||
let storage = Storage::new(None)?;
|
||||
let storage = open_storage()?;
|
||||
let all_nodes = fetch_all_nodes(&storage)?;
|
||||
|
||||
// Apply filters
|
||||
|
|
@ -1141,6 +1257,145 @@ fn run_export(
|
|||
Ok(())
|
||||
}
|
||||
|
||||
/// Run exact portable archive export.
|
||||
fn run_portable_export(output: PathBuf) -> anyhow::Result<()> {
|
||||
println!("{}", "=== Vestige Portable Export ===".cyan().bold());
|
||||
println!();
|
||||
|
||||
if let Some(parent) = output.parent()
|
||||
&& !parent.exists()
|
||||
{
|
||||
std::fs::create_dir_all(parent)?;
|
||||
}
|
||||
|
||||
let storage = open_storage()?;
|
||||
let archive = storage.export_portable_archive_to_path(&output)?;
|
||||
|
||||
let file_size = std::fs::metadata(&output)?.len();
|
||||
let size_display = if file_size >= 1024 * 1024 {
|
||||
format!("{:.2} MB", file_size as f64 / (1024.0 * 1024.0))
|
||||
} else if file_size >= 1024 {
|
||||
format!("{:.1} KB", file_size as f64 / 1024.0)
|
||||
} else {
|
||||
format!("{} bytes", file_size)
|
||||
};
|
||||
|
||||
println!("{}: {}", "Archive".white().bold(), output.display());
|
||||
println!("{}: {}", "Format".white().bold(), archive.archive_format);
|
||||
println!("{}: {}", "Schema".white().bold(), archive.schema_version);
|
||||
println!("{}: {}", "Tables".white().bold(), archive.tables.len());
|
||||
println!("{}: {}", "Rows".white().bold(), archive.total_rows());
|
||||
println!();
|
||||
println!(
|
||||
"{}",
|
||||
format!(
|
||||
"Portable export complete: {} ({})",
|
||||
output.display(),
|
||||
size_display
|
||||
)
|
||||
.green()
|
||||
.bold()
|
||||
);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Run exact portable archive import.
|
||||
fn run_portable_import(input: PathBuf, merge: bool) -> anyhow::Result<()> {
|
||||
println!("{}", "=== Vestige Portable Import ===".cyan().bold());
|
||||
println!();
|
||||
println!("{}: {}", "Archive".white().bold(), input.display());
|
||||
let mode = if merge {
|
||||
PortableImportMode::Merge
|
||||
} else {
|
||||
PortableImportMode::EmptyOnly
|
||||
};
|
||||
println!(
|
||||
"{}",
|
||||
if merge {
|
||||
"Mode: merge into existing database".yellow()
|
||||
} else {
|
||||
"Mode: empty database only".yellow()
|
||||
}
|
||||
);
|
||||
println!();
|
||||
|
||||
let storage = open_storage()?;
|
||||
let report = storage.import_portable_archive_from_path(&input, mode)?;
|
||||
|
||||
println!(
|
||||
"{}: {}",
|
||||
"Tables imported".white().bold(),
|
||||
report.tables_imported
|
||||
);
|
||||
println!(
|
||||
"{}: {}",
|
||||
"Rows imported".white().bold(),
|
||||
report.rows_imported
|
||||
);
|
||||
println!(
|
||||
"{}: {}",
|
||||
"Tables skipped".white().bold(),
|
||||
report.tables_skipped
|
||||
);
|
||||
println!("{}: {}", "FTS rebuilt".white().bold(), report.fts_rebuilt);
|
||||
if merge {
|
||||
println!(
|
||||
"{}: {} inserted, {} updated, {} deleted, {} skipped, {} kept local",
|
||||
"Merge".white().bold(),
|
||||
report.rows_inserted,
|
||||
report.rows_updated,
|
||||
report.rows_deleted,
|
||||
report.rows_skipped,
|
||||
report.conflicts_kept_local
|
||||
);
|
||||
}
|
||||
println!();
|
||||
println!("{}", "Portable import complete.".green().bold());
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Run file-backed two-way sync.
|
||||
fn run_sync(archive: PathBuf) -> anyhow::Result<()> {
|
||||
println!("{}", "=== Vestige File Sync ===".cyan().bold());
|
||||
println!();
|
||||
println!("{}: {}", "Archive".white().bold(), archive.display());
|
||||
|
||||
let storage = open_storage()?;
|
||||
let report = storage.sync_portable_archive_file(&archive)?;
|
||||
|
||||
if let Some(pull) = &report.pull {
|
||||
println!("{}", "Pull: merged remote archive".yellow());
|
||||
println!(
|
||||
" {} inserted, {} updated, {} deleted, {} skipped, {} kept local",
|
||||
pull.rows_inserted,
|
||||
pull.rows_updated,
|
||||
pull.rows_deleted,
|
||||
pull.rows_skipped,
|
||||
pull.conflicts_kept_local
|
||||
);
|
||||
} else {
|
||||
println!(
|
||||
"{}",
|
||||
"Pull: archive does not exist yet; creating it".yellow()
|
||||
);
|
||||
}
|
||||
|
||||
println!("{}", "Push: wrote merged local state".yellow());
|
||||
println!(
|
||||
"{}",
|
||||
format!(
|
||||
"Sync complete: {} tables, {} rows",
|
||||
report.pushed_tables, report.pushed_rows
|
||||
)
|
||||
.green()
|
||||
.bold()
|
||||
);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Run garbage collection command
|
||||
fn run_gc(
|
||||
min_retention: f64,
|
||||
|
|
@ -1151,7 +1406,7 @@ fn run_gc(
|
|||
println!("{}", "=== Vestige Garbage Collection ===".cyan().bold());
|
||||
println!();
|
||||
|
||||
let storage = Storage::new(None)?;
|
||||
let storage = open_storage()?;
|
||||
let all_nodes = fetch_all_nodes(&storage)?;
|
||||
let now = Utc::now();
|
||||
|
||||
|
|
@ -1327,7 +1582,7 @@ fn run_ingest(
|
|||
valid_until: None,
|
||||
};
|
||||
|
||||
let storage = Storage::new(None)?;
|
||||
let storage = open_storage()?;
|
||||
|
||||
// Try smart_ingest (PE Gating) if available, otherwise regular ingest
|
||||
#[cfg(all(feature = "embeddings", feature = "vector-search"))]
|
||||
|
|
@ -1381,7 +1636,7 @@ fn run_dashboard(port: u16, open_browser: bool) -> anyhow::Result<()> {
|
|||
format!("http://127.0.0.1:{}", port).cyan()
|
||||
);
|
||||
|
||||
let storage = Storage::new(None)?;
|
||||
let storage = open_storage()?;
|
||||
|
||||
// Try to initialize embeddings for search support
|
||||
#[cfg(feature = "embeddings")]
|
||||
|
|
@ -1412,7 +1667,7 @@ fn run_serve(port: u16, with_dashboard: bool, dashboard_port: u16) -> anyhow::Re
|
|||
println!("{}", "=== Vestige HTTP Server ===".cyan().bold());
|
||||
println!();
|
||||
|
||||
let storage = Storage::new(None)?;
|
||||
let storage = open_storage()?;
|
||||
|
||||
#[cfg(feature = "embeddings")]
|
||||
{
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@
|
|||
|
||||
use vestige_core::neuroscience::predictive_retrieval::PredictiveMemory;
|
||||
use vestige_core::neuroscience::prospective_memory::{IntentionParser, ProspectiveMemory};
|
||||
#[cfg(feature = "vector-search")]
|
||||
use vestige_core::search::TemporalSearcher;
|
||||
use vestige_core::{
|
||||
AccessibilityCalculator,
|
||||
|
|
@ -31,9 +32,6 @@ use vestige_core::{
|
|||
MemoryDreamer,
|
||||
NoveltySignal,
|
||||
ReconsolidationManager,
|
||||
// Search modules
|
||||
Reranker,
|
||||
RerankerConfig,
|
||||
RewardSignal,
|
||||
SpeculativeRetriever,
|
||||
StateUpdateService,
|
||||
|
|
@ -41,6 +39,8 @@ use vestige_core::{
|
|||
Storage,
|
||||
SynapticTaggingSystem,
|
||||
};
|
||||
#[cfg(feature = "vector-search")]
|
||||
use vestige_core::{Reranker, RerankerConfig};
|
||||
|
||||
/// Stateful cognitive engine holding all neuroscience modules.
|
||||
///
|
||||
|
|
@ -80,7 +80,9 @@ pub struct CognitiveEngine {
|
|||
pub consolidation_scheduler: ConsolidationScheduler,
|
||||
|
||||
// -- Search --
|
||||
#[cfg(feature = "vector-search")]
|
||||
pub reranker: Reranker,
|
||||
#[cfg(feature = "vector-search")]
|
||||
pub temporal_searcher: TemporalSearcher,
|
||||
}
|
||||
|
||||
|
|
@ -161,7 +163,9 @@ impl CognitiveEngine {
|
|||
consolidation_scheduler: ConsolidationScheduler::new(),
|
||||
|
||||
// Search
|
||||
#[cfg(feature = "vector-search")]
|
||||
reranker: Reranker::new(RerankerConfig::default()),
|
||||
#[cfg(feature = "vector-search")]
|
||||
temporal_searcher: TemporalSearcher::new(),
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -467,7 +467,7 @@ async fn main() {
|
|||
}
|
||||
|
||||
// Load cross-encoder reranker in the background (downloads ~150MB on first run)
|
||||
#[cfg(feature = "embeddings")]
|
||||
#[cfg(feature = "vector-search")]
|
||||
{
|
||||
let cog_clone = Arc::clone(&cognitive);
|
||||
tokio::spawn(async move {
|
||||
|
|
|
|||
|
|
@ -513,6 +513,7 @@ pub async fn execute(
|
|||
}
|
||||
|
||||
let mut ranked = results;
|
||||
#[cfg(feature = "vector-search")]
|
||||
if let Ok(mut cog) = cognitive.try_lock() {
|
||||
let candidates: Vec<_> = ranked
|
||||
.iter()
|
||||
|
|
|
|||
|
|
@ -4,8 +4,10 @@
|
|||
//! cosine similarity on stored embeddings. Uses union-find for
|
||||
//! efficient clustering.
|
||||
|
||||
#[cfg(all(feature = "embeddings", feature = "vector-search"))]
|
||||
use serde::Deserialize;
|
||||
use serde_json::Value;
|
||||
#[cfg(all(feature = "embeddings", feature = "vector-search"))]
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
|
|
@ -43,6 +45,7 @@ pub fn schema() -> Value {
|
|||
|
||||
#[derive(Debug, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
#[cfg(all(feature = "embeddings", feature = "vector-search"))]
|
||||
struct DedupArgs {
|
||||
#[serde(alias = "similarity_threshold")]
|
||||
similarity_threshold: Option<f64>,
|
||||
|
|
@ -51,11 +54,13 @@ struct DedupArgs {
|
|||
}
|
||||
|
||||
/// Simple union-find for clustering
|
||||
#[cfg(all(feature = "embeddings", feature = "vector-search"))]
|
||||
struct UnionFind {
|
||||
parent: Vec<usize>,
|
||||
rank: Vec<usize>,
|
||||
}
|
||||
|
||||
#[cfg(all(feature = "embeddings", feature = "vector-search"))]
|
||||
impl UnionFind {
|
||||
fn new(n: usize) -> Self {
|
||||
Self {
|
||||
|
|
@ -89,21 +94,22 @@ impl UnionFind {
|
|||
}
|
||||
|
||||
pub async fn execute(storage: &Arc<Storage>, args: Option<Value>) -> Result<Value, String> {
|
||||
let args: DedupArgs = match args {
|
||||
Some(v) => serde_json::from_value(v).map_err(|e| format!("Invalid arguments: {}", e))?,
|
||||
None => DedupArgs {
|
||||
similarity_threshold: None,
|
||||
limit: None,
|
||||
tags: None,
|
||||
},
|
||||
};
|
||||
|
||||
let threshold = args.similarity_threshold.unwrap_or(0.80) as f32;
|
||||
let limit = args.limit.unwrap_or(20);
|
||||
let tag_filter = args.tags.unwrap_or_default();
|
||||
|
||||
#[cfg(all(feature = "embeddings", feature = "vector-search"))]
|
||||
{
|
||||
let args: DedupArgs = match args {
|
||||
Some(v) => {
|
||||
serde_json::from_value(v).map_err(|e| format!("Invalid arguments: {}", e))?
|
||||
}
|
||||
None => DedupArgs {
|
||||
similarity_threshold: None,
|
||||
limit: None,
|
||||
tags: None,
|
||||
},
|
||||
};
|
||||
let threshold = args.similarity_threshold.unwrap_or(0.80) as f32;
|
||||
let limit = args.limit.unwrap_or(20);
|
||||
let tag_filter = args.tags.unwrap_or_default();
|
||||
|
||||
// Load all embeddings
|
||||
let all_embeddings = storage
|
||||
.get_all_embeddings()
|
||||
|
|
@ -261,6 +267,8 @@ pub async fn execute(storage: &Arc<Storage>, args: Option<Value>) -> Result<Valu
|
|||
|
||||
#[cfg(not(all(feature = "embeddings", feature = "vector-search")))]
|
||||
{
|
||||
let _ = storage;
|
||||
let _ = args;
|
||||
Ok(serde_json::json!({
|
||||
"error": "Embeddings feature not enabled. Cannot compute similarities.",
|
||||
"clusters": []
|
||||
|
|
|
|||
|
|
@ -36,8 +36,8 @@ pub fn export_schema() -> Value {
|
|||
"properties": {
|
||||
"format": {
|
||||
"type": "string",
|
||||
"description": "Export format: 'json' (default) or 'jsonl'",
|
||||
"enum": ["json", "jsonl"],
|
||||
"description": "Export format: 'json' (default), 'jsonl', or 'portable' for exact Vestige-to-Vestige transfer",
|
||||
"enum": ["json", "jsonl", "portable"],
|
||||
"default": "json"
|
||||
},
|
||||
"tags": {
|
||||
|
|
@ -51,7 +51,7 @@ pub fn export_schema() -> Value {
|
|||
},
|
||||
"path": {
|
||||
"type": "string",
|
||||
"description": "Custom filename (not path). File is saved in ~/.vestige/exports/. Default: memories-{timestamp}.{format}"
|
||||
"description": "Custom filename (not path). File is saved in the active Vestige data directory's exports/ folder. Default: memories-{timestamp}.{format}"
|
||||
}
|
||||
}
|
||||
})
|
||||
|
|
@ -117,7 +117,7 @@ pub async fn execute_system_status(
|
|||
};
|
||||
|
||||
let embedding_coverage = if stats.total_nodes > 0 {
|
||||
(stats.nodes_with_embeddings as f64 / stats.total_nodes as f64) * 100.0
|
||||
(stats.nodes_with_active_embeddings as f64 / stats.total_nodes as f64) * 100.0
|
||||
} else {
|
||||
0.0
|
||||
};
|
||||
|
|
@ -131,12 +131,17 @@ pub async fn execute_system_status(
|
|||
if stats.nodes_due_for_review > 10 {
|
||||
warnings.push("Many memories are due for review");
|
||||
}
|
||||
if stats.total_nodes > 0 && stats.nodes_with_embeddings == 0 {
|
||||
warnings.push("No embeddings generated - semantic search unavailable");
|
||||
if stats.total_nodes > 0 && stats.nodes_with_active_embeddings == 0 {
|
||||
warnings.push("No active-model embeddings generated - semantic search unavailable");
|
||||
}
|
||||
if embedding_coverage < 50.0 && stats.total_nodes > 10 {
|
||||
warnings.push("Low embedding coverage - run consolidate to improve semantic search");
|
||||
}
|
||||
if stats.nodes_with_mismatched_embeddings > 0 {
|
||||
warnings.push(
|
||||
"Stored embeddings from another model are present - run consolidate after changing embedding models",
|
||||
);
|
||||
}
|
||||
|
||||
let mut recommendations = Vec::new();
|
||||
if status == "critical" {
|
||||
|
|
@ -146,8 +151,8 @@ pub async fn execute_system_status(
|
|||
if stats.nodes_due_for_review > 5 {
|
||||
recommendations.push("Review due memories to strengthen retention.");
|
||||
}
|
||||
if stats.nodes_with_embeddings < stats.total_nodes {
|
||||
recommendations.push("Run 'consolidate' to generate missing embeddings.");
|
||||
if stats.nodes_with_active_embeddings < stats.total_nodes {
|
||||
recommendations.push("Run 'consolidate' to generate active-model embeddings.");
|
||||
}
|
||||
if stats.total_nodes > 100 && stats.average_retention < 0.7 {
|
||||
recommendations.push("Consider running periodic consolidation.");
|
||||
|
|
@ -233,7 +238,7 @@ pub async fn execute_system_status(
|
|||
Some(dt) => storage.count_memories_since(*dt).unwrap_or(0),
|
||||
None => stats.total_nodes,
|
||||
};
|
||||
let last_backup = Storage::get_last_backup_timestamp();
|
||||
let last_backup = storage.last_backup_timestamp();
|
||||
|
||||
Ok(serde_json::json!({
|
||||
"tool": "system_status",
|
||||
|
|
@ -249,8 +254,11 @@ pub async fn execute_system_status(
|
|||
"averageStorageStrength": stats.average_storage_strength,
|
||||
"averageRetrievalStrength": stats.average_retrieval_strength,
|
||||
"withEmbeddings": stats.nodes_with_embeddings,
|
||||
"withActiveEmbeddings": stats.nodes_with_active_embeddings,
|
||||
"mismatchedEmbeddings": stats.nodes_with_mismatched_embeddings,
|
||||
"embeddingCoverage": format!("{:.1}%", embedding_coverage),
|
||||
"embeddingModel": stats.embedding_model,
|
||||
"activeEmbeddingModel": stats.active_embedding_model,
|
||||
"oldestMemory": stats.oldest_memory.map(|dt| dt.to_rfc3339()),
|
||||
"newestMemory": stats.newest_memory.map(|dt| dt.to_rfc3339()),
|
||||
// Distribution
|
||||
|
|
@ -299,13 +307,7 @@ pub async fn execute_consolidate(
|
|||
/// Backup tool
|
||||
pub async fn execute_backup(storage: &Arc<Storage>, _args: Option<Value>) -> Result<Value, String> {
|
||||
// Determine backup path
|
||||
let vestige_dir = directories::ProjectDirs::from("com", "vestige", "core")
|
||||
.ok_or("Could not determine data directory")?;
|
||||
let backup_dir = vestige_dir
|
||||
.data_dir()
|
||||
.parent()
|
||||
.unwrap_or(vestige_dir.data_dir())
|
||||
.join("backups");
|
||||
let backup_dir = storage.sidecar_dir("backups");
|
||||
|
||||
std::fs::create_dir_all(&backup_dir)
|
||||
.map_err(|e| format!("Failed to create backup directory: {}", e))?;
|
||||
|
|
@ -354,13 +356,60 @@ pub async fn execute_export(storage: &Arc<Storage>, args: Option<Value>) -> Resu
|
|||
};
|
||||
|
||||
let format = args.format.unwrap_or_else(|| "json".to_string());
|
||||
if format != "json" && format != "jsonl" {
|
||||
if format != "json" && format != "jsonl" && format != "portable" {
|
||||
return Err(format!(
|
||||
"Invalid format '{}'. Must be 'json' or 'jsonl'.",
|
||||
"Invalid format '{}'. Must be 'json', 'jsonl', or 'portable'.",
|
||||
format
|
||||
));
|
||||
}
|
||||
|
||||
if format == "portable" {
|
||||
if args.tags.as_ref().is_some_and(|tags| !tags.is_empty()) || args.since.is_some() {
|
||||
return Err(
|
||||
"Portable export is exact and does not support tags or since filters.".to_string(),
|
||||
);
|
||||
}
|
||||
|
||||
let export_dir = storage.sidecar_dir("exports");
|
||||
std::fs::create_dir_all(&export_dir)
|
||||
.map_err(|e| format!("Failed to create export directory: {}", e))?;
|
||||
|
||||
let export_path = match args.path {
|
||||
Some(ref p) => {
|
||||
let filename = std::path::Path::new(p)
|
||||
.file_name()
|
||||
.ok_or("Invalid export filename: must be a simple filename, not a path")?;
|
||||
let name_str = filename.to_str().ok_or("Invalid filename encoding")?;
|
||||
if name_str.contains("..") {
|
||||
return Err("Invalid export filename: '..' not allowed".to_string());
|
||||
}
|
||||
export_dir.join(filename)
|
||||
}
|
||||
None => {
|
||||
let timestamp = Utc::now().format("%Y%m%d-%H%M%S");
|
||||
export_dir.join(format!("vestige-portable-{}.json", timestamp))
|
||||
}
|
||||
};
|
||||
|
||||
let archive = storage
|
||||
.export_portable_archive_to_path(&export_path)
|
||||
.map_err(|e| e.to_string())?;
|
||||
let file_size = std::fs::metadata(&export_path)
|
||||
.map(|m| m.len())
|
||||
.unwrap_or(0);
|
||||
|
||||
return Ok(serde_json::json!({
|
||||
"tool": "export",
|
||||
"path": export_path.display().to_string(),
|
||||
"format": "portable",
|
||||
"archiveFormat": archive.archive_format,
|
||||
"schemaVersion": archive.schema_version,
|
||||
"tablesExported": archive.tables.len(),
|
||||
"rowsExported": archive.total_rows(),
|
||||
"sizeBytes": file_size,
|
||||
}));
|
||||
}
|
||||
|
||||
// Parse since date
|
||||
let since_date = match &args.since {
|
||||
Some(date_str) => {
|
||||
|
|
@ -412,13 +461,7 @@ pub async fn execute_export(storage: &Arc<Storage>, args: Option<Value>) -> Resu
|
|||
.collect();
|
||||
|
||||
// Determine export path — always constrained to vestige exports directory
|
||||
let vestige_dir = directories::ProjectDirs::from("com", "vestige", "core")
|
||||
.ok_or("Could not determine data directory")?;
|
||||
let export_dir = vestige_dir
|
||||
.data_dir()
|
||||
.parent()
|
||||
.unwrap_or(vestige_dir.data_dir())
|
||||
.join("exports");
|
||||
let export_dir = storage.sidecar_dir("exports");
|
||||
std::fs::create_dir_all(&export_dir)
|
||||
.map_err(|e| format!("Failed to create export directory: {}", e))?;
|
||||
|
||||
|
|
@ -729,4 +772,41 @@ mod tests {
|
|||
assert_eq!(triggers["savesSinceLastDream"], 3);
|
||||
assert!(triggers["lastDreamTimestamp"].is_null());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_portable_export_writes_archive_to_storage_exports_dir() {
|
||||
let (storage, _dir) = test_storage().await;
|
||||
storage
|
||||
.ingest(vestige_core::IngestInput {
|
||||
content: "Portable MCP export test memory".to_string(),
|
||||
node_type: "fact".to_string(),
|
||||
source: None,
|
||||
sentiment_score: 0.0,
|
||||
sentiment_magnitude: 0.0,
|
||||
tags: vec!["portable".to_string()],
|
||||
valid_from: None,
|
||||
valid_until: None,
|
||||
})
|
||||
.unwrap();
|
||||
|
||||
let result = execute_export(
|
||||
&storage,
|
||||
Some(serde_json::json!({
|
||||
"format": "portable",
|
||||
"path": "portable-test.json"
|
||||
})),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let path = result["path"].as_str().unwrap();
|
||||
assert_eq!(result["format"], "portable");
|
||||
assert!(path.ends_with("exports/portable-test.json"));
|
||||
assert!(std::path::Path::new(path).exists());
|
||||
assert_eq!(
|
||||
result["archiveFormat"],
|
||||
vestige_core::PORTABLE_ARCHIVE_FORMAT
|
||||
);
|
||||
assert!(result["rowsExported"].as_u64().unwrap() > 0);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -6,9 +6,10 @@
|
|||
|
||||
use serde::Deserialize;
|
||||
use serde_json::Value;
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
|
||||
use vestige_core::{IngestInput, Storage};
|
||||
use vestige_core::{IngestInput, PortableArchive, PortableImportMode, Storage};
|
||||
|
||||
/// Input schema for restore tool
|
||||
pub fn schema() -> Value {
|
||||
|
|
@ -18,6 +19,16 @@ pub fn schema() -> Value {
|
|||
"path": {
|
||||
"type": "string",
|
||||
"description": "Path to the backup JSON file to restore from"
|
||||
},
|
||||
"allowAnyPath": {
|
||||
"type": "boolean",
|
||||
"description": "Allow restoring from a file outside the active Vestige backups/ or exports/ directories. Only set true for trusted local files.",
|
||||
"default": false
|
||||
},
|
||||
"merge": {
|
||||
"type": "boolean",
|
||||
"description": "For portable archives, merge into the current database instead of requiring an empty target. Applies sync tombstones and keeps newer local memory rows on conflict.",
|
||||
"default": false
|
||||
}
|
||||
},
|
||||
"required": ["path"]
|
||||
|
|
@ -25,8 +36,13 @@ pub fn schema() -> Value {
|
|||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
struct RestoreArgs {
|
||||
path: String,
|
||||
#[serde(default)]
|
||||
allow_any_path: bool,
|
||||
#[serde(default)]
|
||||
merge: bool,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
|
|
@ -56,14 +72,57 @@ pub async fn execute(storage: &Arc<Storage>, args: Option<Value>) -> Result<Valu
|
|||
None => return Err("Missing arguments".to_string()),
|
||||
};
|
||||
|
||||
let path = std::path::Path::new(&args.path);
|
||||
let path = Path::new(&args.path);
|
||||
if !path.exists() {
|
||||
return Err(format!("Backup file not found: {}", args.path));
|
||||
}
|
||||
|
||||
if !args.allow_any_path {
|
||||
ensure_restore_path_allowed(storage, path)?;
|
||||
}
|
||||
|
||||
// Read and parse backup
|
||||
let backup_content =
|
||||
std::fs::read_to_string(path).map_err(|e| format!("Failed to read backup: {}", e))?;
|
||||
let backup_bytes = std::fs::read(path).map_err(|e| format!("Failed to read backup: {}", e))?;
|
||||
if backup_bytes.starts_with(b"SQLite format 3\0") {
|
||||
return Err(
|
||||
"Restore expected JSON, but this file is a raw SQLite database backup. Use portable export/import for cross-device transfer, or replace the database file manually while Vestige is stopped."
|
||||
.to_string(),
|
||||
);
|
||||
}
|
||||
let backup_content = String::from_utf8(backup_bytes)
|
||||
.map_err(|_| "Restore file is not UTF-8 JSON".to_string())?;
|
||||
|
||||
if let Ok(archive) = serde_json::from_str::<PortableArchive>(&backup_content)
|
||||
&& archive.archive_format == vestige_core::PORTABLE_ARCHIVE_FORMAT
|
||||
{
|
||||
let rows = archive.total_rows();
|
||||
let tables = archive.tables.len();
|
||||
let mode = if args.merge {
|
||||
PortableImportMode::Merge
|
||||
} else {
|
||||
PortableImportMode::EmptyOnly
|
||||
};
|
||||
let report = storage
|
||||
.import_portable_archive(&archive, mode)
|
||||
.map_err(|e| e.to_string())?;
|
||||
return Ok(serde_json::json!({
|
||||
"tool": "restore",
|
||||
"success": true,
|
||||
"mode": if args.merge { "portable-merge" } else { "portable" },
|
||||
"tables": tables,
|
||||
"rows": rows,
|
||||
"tablesImported": report.tables_imported,
|
||||
"rowsImported": report.rows_imported,
|
||||
"tablesSkipped": report.tables_skipped,
|
||||
"rowsInserted": report.rows_inserted,
|
||||
"rowsUpdated": report.rows_updated,
|
||||
"rowsDeleted": report.rows_deleted,
|
||||
"rowsSkipped": report.rows_skipped,
|
||||
"conflictsKeptLocal": report.conflicts_kept_local,
|
||||
"ftsRebuilt": report.fts_rebuilt,
|
||||
"message": format!("Imported {} rows from portable archive.", report.rows_imported),
|
||||
}));
|
||||
}
|
||||
|
||||
// Try parsing as wrapped format first (MCP response wrapper),
|
||||
// then fall back to direct RecallResult
|
||||
|
|
@ -132,6 +191,33 @@ pub async fn execute(storage: &Arc<Storage>, args: Option<Value>) -> Result<Valu
|
|||
}))
|
||||
}
|
||||
|
||||
fn ensure_restore_path_allowed(storage: &Storage, path: &Path) -> Result<(), String> {
|
||||
let canonical_path = path
|
||||
.canonicalize()
|
||||
.map_err(|e| format!("Failed to resolve restore path: {}", e))?;
|
||||
|
||||
for dir in [
|
||||
storage.sidecar_dir("exports"),
|
||||
storage.sidecar_dir("backups"),
|
||||
] {
|
||||
if !dir.exists() {
|
||||
continue;
|
||||
}
|
||||
let canonical_dir = dir
|
||||
.canonicalize()
|
||||
.map_err(|e| format!("Failed to resolve allowed restore directory: {}", e))?;
|
||||
if canonical_path.starts_with(&canonical_dir) {
|
||||
return Ok(());
|
||||
}
|
||||
}
|
||||
|
||||
Err(format!(
|
||||
"MCP restore is restricted to {} and {} by default. Pass allowAnyPath=true only for trusted local files.",
|
||||
storage.sidecar_dir("exports").display(),
|
||||
storage.sidecar_dir("backups").display()
|
||||
))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
|
@ -156,6 +242,7 @@ mod tests {
|
|||
let s = schema();
|
||||
assert_eq!(s["type"], "object");
|
||||
assert!(s["properties"]["path"].is_object());
|
||||
assert!(s["properties"]["allowAnyPath"].is_object());
|
||||
assert!(
|
||||
s["required"]
|
||||
.as_array()
|
||||
|
|
@ -193,12 +280,22 @@ mod tests {
|
|||
async fn test_malformed_json_fails() {
|
||||
let (storage, dir) = test_storage().await;
|
||||
let path = write_temp_file(&dir, "bad.json", "this is not json {{{");
|
||||
let args = serde_json::json!({ "path": path });
|
||||
let args = serde_json::json!({ "path": path, "allowAnyPath": true });
|
||||
let result = execute(&storage, Some(args)).await;
|
||||
assert!(result.is_err());
|
||||
assert!(result.unwrap_err().contains("Unrecognized backup format"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_restore_rejects_arbitrary_path_by_default() {
|
||||
let (storage, dir) = test_storage().await;
|
||||
let path = write_temp_file(&dir, "outside_exports.json", "[]");
|
||||
let args = serde_json::json!({ "path": path });
|
||||
let result = execute(&storage, Some(args)).await;
|
||||
assert!(result.is_err());
|
||||
assert!(result.unwrap_err().contains("restricted"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_restore_direct_array_format() {
|
||||
let (storage, dir) = test_storage().await;
|
||||
|
|
@ -207,7 +304,7 @@ mod tests {
|
|||
{ "content": "Memory two", "nodeType": "concept" }
|
||||
]);
|
||||
let path = write_temp_file(&dir, "backup.json", &backup.to_string());
|
||||
let args = serde_json::json!({ "path": path });
|
||||
let args = serde_json::json!({ "path": path, "allowAnyPath": true });
|
||||
let result = execute(&storage, Some(args)).await;
|
||||
assert!(result.is_ok());
|
||||
let value = result.unwrap();
|
||||
|
|
@ -229,7 +326,7 @@ mod tests {
|
|||
]
|
||||
});
|
||||
let path = write_temp_file(&dir, "recall.json", &backup.to_string());
|
||||
let args = serde_json::json!({ "path": path });
|
||||
let args = serde_json::json!({ "path": path, "allowAnyPath": true });
|
||||
let result = execute(&storage, Some(args)).await;
|
||||
assert!(result.is_ok());
|
||||
let value = result.unwrap();
|
||||
|
|
@ -237,12 +334,47 @@ mod tests {
|
|||
assert_eq!(value["total"], 3);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_restore_portable_archive_from_allowed_exports_dir() {
|
||||
let (source, _source_dir) = test_storage().await;
|
||||
let (target, _target_dir) = test_storage().await;
|
||||
|
||||
source
|
||||
.ingest(vestige_core::IngestInput {
|
||||
content: "Portable MCP restore test memory".to_string(),
|
||||
node_type: "fact".to_string(),
|
||||
source: None,
|
||||
sentiment_score: 0.0,
|
||||
sentiment_magnitude: 0.0,
|
||||
tags: vec!["portable".to_string()],
|
||||
valid_from: None,
|
||||
valid_until: None,
|
||||
})
|
||||
.unwrap();
|
||||
|
||||
let export_dir = target.sidecar_dir("exports");
|
||||
std::fs::create_dir_all(&export_dir).unwrap();
|
||||
let archive_path = export_dir.join("portable-restore.json");
|
||||
source
|
||||
.export_portable_archive_to_path(&archive_path)
|
||||
.unwrap();
|
||||
|
||||
let args = serde_json::json!({ "path": archive_path });
|
||||
let result = execute(&target, Some(args)).await.unwrap();
|
||||
|
||||
assert_eq!(result["tool"], "restore");
|
||||
assert_eq!(result["success"], true);
|
||||
assert_eq!(result["mode"], "portable");
|
||||
assert!(result["rowsImported"].as_u64().unwrap() > 0);
|
||||
assert_eq!(target.get_stats().unwrap().total_nodes, 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_restore_empty_results_array() {
|
||||
let (storage, dir) = test_storage().await;
|
||||
let backup = serde_json::json!({ "results": [] });
|
||||
let path = write_temp_file(&dir, "empty.json", &backup.to_string());
|
||||
let args = serde_json::json!({ "path": path });
|
||||
let args = serde_json::json!({ "path": path, "allowAnyPath": true });
|
||||
let result = execute(&storage, Some(args)).await;
|
||||
assert!(result.is_ok());
|
||||
let value = result.unwrap();
|
||||
|
|
@ -255,7 +387,7 @@ mod tests {
|
|||
// Empty [] parses as Vec<BackupWrapper> first, which has no items → "Empty backup file"
|
||||
let (storage, dir) = test_storage().await;
|
||||
let path = write_temp_file(&dir, "empty_arr.json", "[]");
|
||||
let args = serde_json::json!({ "path": path });
|
||||
let args = serde_json::json!({ "path": path, "allowAnyPath": true });
|
||||
let result = execute(&storage, Some(args)).await;
|
||||
assert!(result.is_err());
|
||||
assert!(result.unwrap_err().contains("Empty backup file"));
|
||||
|
|
@ -266,7 +398,7 @@ mod tests {
|
|||
let (storage, dir) = test_storage().await;
|
||||
let backup = serde_json::json!([{ "content": "No type specified" }]);
|
||||
let path = write_temp_file(&dir, "notype.json", &backup.to_string());
|
||||
let args = serde_json::json!({ "path": path });
|
||||
let args = serde_json::json!({ "path": path, "allowAnyPath": true });
|
||||
let result = execute(&storage, Some(args)).await;
|
||||
assert!(result.is_ok());
|
||||
assert_eq!(result.unwrap()["restored"], 1);
|
||||
|
|
|
|||
|
|
@ -305,7 +305,8 @@ pub async fn execute(
|
|||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
});
|
||||
|
||||
// Rerank the remaining candidates
|
||||
// Rerank the remaining candidates when the vector-search search stack is enabled.
|
||||
#[cfg(feature = "vector-search")]
|
||||
let reranked_results: Vec<vestige_core::SearchResult> = if rerank_candidates.is_empty() {
|
||||
Vec::new()
|
||||
} else if let Ok(mut cog) = cognitive.try_lock() {
|
||||
|
|
@ -330,6 +331,11 @@ pub async fn execute(
|
|||
.cloned()
|
||||
.collect()
|
||||
};
|
||||
#[cfg(not(feature = "vector-search"))]
|
||||
let reranked_results: Vec<vestige_core::SearchResult> = rerank_candidates
|
||||
.into_iter()
|
||||
.map(|(result, _)| result)
|
||||
.collect();
|
||||
|
||||
// Merge: bypass first, then reranked, trim to limit
|
||||
filtered_results = bypass_results;
|
||||
|
|
@ -340,6 +346,7 @@ pub async fn execute(
|
|||
// ====================================================================
|
||||
// STAGE 3: Temporal boosting (recency + validity windows)
|
||||
// ====================================================================
|
||||
#[cfg(feature = "vector-search")]
|
||||
if let Ok(cog) = cognitive.try_lock() {
|
||||
for result in &mut filtered_results {
|
||||
let recency = cog.temporal_searcher.recency_boost(result.node.created_at);
|
||||
|
|
|
|||
|
|
@ -223,7 +223,7 @@ pub async fn execute(
|
|||
Some(dt) => storage.count_memories_since(*dt).unwrap_or(0),
|
||||
None => stats.total_nodes,
|
||||
};
|
||||
let last_backup = Storage::get_last_backup_timestamp();
|
||||
let last_backup = storage.last_backup_timestamp();
|
||||
let now = Utc::now();
|
||||
|
||||
let needs_dream = last_dream
|
||||
|
|
@ -236,7 +236,7 @@ pub async fn execute(
|
|||
|
||||
if include_status {
|
||||
let embedding_pct = if stats.total_nodes > 0 {
|
||||
(stats.nodes_with_embeddings as f64 / stats.total_nodes as f64) * 100.0
|
||||
(stats.nodes_with_active_embeddings as f64 / stats.total_nodes as f64) * 100.0
|
||||
} else {
|
||||
0.0
|
||||
};
|
||||
|
|
|
|||
|
|
@ -315,7 +315,10 @@ async fn execute_batch(
|
|||
|
||||
let mut results = Vec::new();
|
||||
let mut created = 0u32;
|
||||
#[cfg(all(feature = "embeddings", feature = "vector-search"))]
|
||||
let mut updated = 0u32;
|
||||
#[cfg(not(all(feature = "embeddings", feature = "vector-search")))]
|
||||
let updated = 0u32;
|
||||
let mut skipped = 0u32;
|
||||
let mut errors = 0u32;
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
**Vestige's defense-in-depth safety architecture for Claude Code.**
|
||||
|
||||
The Cognitive Sandwich wraps every Claude Code response in two layers of cognitive scaffolding:
|
||||
The default Cognitive Sandwich installer only stages files and removes old v2.1.0 hook wiring. It activates no Claude Code hooks and makes no automatic model calls. Both the preflight layer and the Stop-hook layer are explicit opt-ins:
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────┐
|
||||
|
|
@ -15,32 +15,31 @@ The Cognitive Sandwich wraps every Claude Code response in two layers of cogniti
|
|||
├────────────────────────────────────────────────┤
|
||||
│ 🥩 MEAT — Claude Code reasons │
|
||||
├────────────────────────────────────────────────┤
|
||||
│ 🥪 BOTTOM BREAD — Stop hooks │
|
||||
│ • Veto-detector (fast 50ms regex pre-screen) │
|
||||
│ • Sanhedrin Executioner (LOCAL Qwen3.6-35B) │
|
||||
│ • Synthesis stop validator (hedge detector) │
|
||||
│ 🥪 OPTIONAL BOTTOM BREAD — Stop hooks │
|
||||
│ • Veto-detector / synthesis validator │
|
||||
│ • Sanhedrin Executioner verifier │
|
||||
└────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
The Sanhedrin Executioner is the headline of v2.1.0. As of v2.1.0 it runs entirely on a local MLX model (`mlx-community/Qwen3.6-35B-A3B-4bit`), replacing the v2.0.x Haiku 4.5 subagent. **Zero API cost per Claude turn, fully offline, ~5–15s verdict latency on M-series Apple Silicon.**
|
||||
Sanhedrin, preflight, and all Vestige Claude Code hooks are optional. The default installer wires none of them; it does not call Claude, start MLX, require a 19 GB model download, or require 20+ GB of RAM. Users who want preflight context can opt in with `--enable-preflight`. Users who want the post-response verifier can opt in with `--enable-sanhedrin` and point it at any OpenAI-compatible `/v1/chat/completions` endpoint. On Apple Silicon, an additional `--with-launchd` flag can auto-start the local MLX Qwen backend.
|
||||
|
||||
---
|
||||
|
||||
## How a single response flows through the Sandwich
|
||||
|
||||
1. **You type a prompt in Claude Code.**
|
||||
2. **UserPromptSubmit hooks fire in parallel** (none can block — all fail-open):
|
||||
2. **If explicitly enabled, UserPromptSubmit hooks fire in parallel** (none can block — all fail-open):
|
||||
- `load-all-memory.sh` (opt-in) — dumps every memory MD into context
|
||||
- `synthesis-preflight.sh` — POSTs your prompt to `vestige-mcp` `/api/deep_reference`, injects the trust-scored reasoning chain
|
||||
- `cwd-state-injector.sh` — captures git status, branch, open PRs/issues, modified files
|
||||
- `vestige-pulse-daemon.sh` — injects fresh Vestige dream insights from the past 20 min into the next prompt context
|
||||
- `preflight-swarm.sh` — spawns the `lateral-thinker` subagent in fresh context to surface cross-disciplinary structural parallels
|
||||
3. **Claude reads the assembled context and generates a draft.**
|
||||
4. **Stop hooks fire serially** (any can VETO with `exit 2`, forcing a rewrite):
|
||||
4. **By default, no Vestige Stop hooks are installed.** If explicitly enabled, Stop hooks fire serially (any can VETO with `exit 2`, forcing a rewrite):
|
||||
- `veto-detector.sh` — fast regex against `veto`-tagged Vestige memories (~50ms)
|
||||
- `sanhedrin.sh` → `sanhedrin-local.py` — single-shot local Qwen3.6-35B-A3B verdict
|
||||
- `sanhedrin.sh` → `sanhedrin-local.py` — optional single-shot semantic verdict
|
||||
- `synthesis-stop-validator.sh` — regex against forbidden patterns (hedging, summary-instead-of-composition)
|
||||
5. **If all 3 Stop hooks return `exit 0`, the response is delivered.**
|
||||
5. **If all enabled Stop hooks return `exit 0`, the response is delivered.**
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -73,7 +72,7 @@ False-positive guards (added v2.1.0 after dogfood):
|
|||
### One-liner
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/samvallad33/vestige/v2.1.0/scripts/install-sandwich.sh | sh
|
||||
curl -fsSL https://raw.githubusercontent.com/samvallad33/vestige/v2.1.1/scripts/install-sandwich.sh | sh
|
||||
```
|
||||
|
||||
### From a checkout
|
||||
|
|
@ -82,30 +81,65 @@ curl -fsSL https://raw.githubusercontent.com/samvallad33/vestige/v2.1.0/scripts/
|
|||
git clone https://github.com/samvallad33/vestige
|
||||
cd vestige
|
||||
./scripts/install-sandwich.sh # add --force to overwrite existing hooks
|
||||
./scripts/check-sandwich-prereqs.sh # verify everything's wired
|
||||
./scripts/check-sandwich-prereqs.sh # verify no Vestige hooks are wired by default
|
||||
```
|
||||
|
||||
The default command does not activate any Claude Code hook. It removes old v2.1.0 Vestige hook wiring from `~/.claude/settings.json` while preserving unrelated user hooks.
|
||||
|
||||
### Optional Preflight
|
||||
|
||||
Preflight is a separate opt-in layer. It includes `preflight-swarm.sh`, which uses `claude -p --model claude-haiku-4-5-20251001`; it is not wired by default.
|
||||
|
||||
```bash
|
||||
./scripts/install-sandwich.sh --enable-preflight
|
||||
./scripts/check-sandwich-prereqs.sh --preflight
|
||||
```
|
||||
|
||||
### Optional Sanhedrin
|
||||
|
||||
Sanhedrin is a separate opt-in layer.
|
||||
|
||||
```bash
|
||||
# Wire the Sanhedrin Stop hook, using the default OpenAI-compatible endpoint.
|
||||
./scripts/install-sandwich.sh --enable-sanhedrin
|
||||
|
||||
# Apple Silicon only, and only if the machine has enough memory:
|
||||
./scripts/install-sandwich.sh --enable-sanhedrin --with-launchd
|
||||
|
||||
# x86 / Linux / Intel Mac: use any OpenAI-compatible endpoint.
|
||||
./scripts/install-sandwich.sh \
|
||||
--enable-sanhedrin \
|
||||
--sanhedrin-endpoint=http://127.0.0.1:11434/v1/chat/completions \
|
||||
--sanhedrin-model=qwen2.5:14b
|
||||
```
|
||||
|
||||
### Prerequisites
|
||||
|
||||
| Tool | Install |
|
||||
|---|---|
|
||||
| macOS Apple Silicon (M1+) | required for MLX |
|
||||
| Python 3.10+ | typically preinstalled |
|
||||
| `jq` | `brew install jq` |
|
||||
| `vestige-mcp` | `cargo install vestige-mcp` |
|
||||
| Claude Code | https://claude.ai/code |
|
||||
|
||||
Optional Apple Silicon local Sanhedrin backend:
|
||||
|
||||
| Tool | Install |
|
||||
|---|---|
|
||||
| macOS Apple Silicon (M1+) | required for MLX launchd only |
|
||||
| `uv` | `brew install uv` |
|
||||
| `mlx-lm` | `uv tool install mlx-lm` |
|
||||
| `huggingface_hub[cli]` | `uv tool install 'huggingface_hub[cli]'` |
|
||||
| `vestige-mcp` | `cargo install vestige-mcp` |
|
||||
| Claude Code | https://claude.ai/code |
|
||||
| Qwen3.6-35B-A3B-4bit | `hf download mlx-community/Qwen3.6-35B-A3B-4bit` (~19 GB) |
|
||||
|
||||
### What the installer does
|
||||
|
||||
1. Verifies prereqs (warnings for missing tools, fatal only on jq/python3).
|
||||
2. Copies hooks to `~/.claude/hooks/`, agents to `~/.claude/agents/`.
|
||||
3. Renders `launchd/com.vestige.mlx-server.plist.template` with your `$HOME` and chosen model, writes to `~/Library/LaunchAgents/`.
|
||||
4. `launchctl load` the plist (auto-start mlx_lm.server with the Qwen model on boot).
|
||||
5. Backs up existing `~/.claude/settings.json` to `.bak.pre-sandwich`, then `jq`-merges the hooks block.
|
||||
3. Backs up existing `~/.claude/settings.json` to `.bak.pre-sandwich`, then removes old Vestige hook wiring from previous v2.1.0 installs.
|
||||
4. With `--enable-preflight`, merges the UserPromptSubmit hooks block.
|
||||
5. With `--enable-sanhedrin`, writes `~/.claude/hooks/vestige-sanhedrin.env` and merges a Sanhedrin-enabled Stop hooks block.
|
||||
6. With `--enable-sanhedrin --with-launchd` on Apple Silicon, renders and loads `launchd/com.vestige.mlx-server.plist.template`.
|
||||
|
||||
### Uninstall
|
||||
|
||||
|
|
@ -120,7 +154,7 @@ cp ~/.claude/settings.json.bak.pre-sandwich ~/.claude/settings.json
|
|||
|
||||
## Performance notes
|
||||
|
||||
On M3 Max 16-core (400 GB/s memory bandwidth):
|
||||
Optional local MLX backend on M3 Max 16-core (400 GB/s memory bandwidth):
|
||||
- Sanhedrin verdict: 5–15 seconds end-to-end (single deep_reference + single Qwen call)
|
||||
- mlx_lm.server token generation: ~82 tok/s
|
||||
- mlx_lm.server peak resident memory: ~19.7 GB
|
||||
|
|
@ -134,11 +168,12 @@ On M3 Max 14-core or M2/M1 Max: closer to 3–7s prompt processing, ~50–60 tok
|
|||
|
||||
| Env var | Default | Effect |
|
||||
|---|---|---|
|
||||
| `VESTIGE_SANHEDRIN_ENABLED` | `1` | Set to `0` to disable Sanhedrin Stop hook entirely |
|
||||
| `VESTIGE_SANHEDRIN_ENABLED` | `0` | Set to `1` to enable the optional Sanhedrin Stop hook |
|
||||
| `VESTIGE_SWARM_ENABLED` | `1` | Set to `0` to disable preflight lateral-thinker swarm |
|
||||
| `VESTIGE_DASHBOARD_PORT` | `3927` | Vestige MCP HTTP API port used by hooks |
|
||||
| `MLX_ENDPOINT` | `http://127.0.0.1:8080/v1/chat/completions` | OpenAI-compatible chat completions endpoint for Sanhedrin |
|
||||
| `VESTIGE_SANDWICH_MODEL` | `mlx-community/Qwen3.6-35B-A3B-4bit` | Model launchd serves and Sanhedrin requests |
|
||||
| `VESTIGE_SANHEDRIN_ENDPOINT` | `http://127.0.0.1:8080/v1/chat/completions` | OpenAI-compatible chat completions endpoint for Sanhedrin |
|
||||
| `VESTIGE_SANHEDRIN_MODEL` | `mlx-community/Qwen3.6-35B-A3B-4bit` | Model name sent to the Sanhedrin endpoint |
|
||||
| `MLX_ENDPOINT` / `VESTIGE_SANDWICH_MODEL` | legacy aliases | Backward-compatible names still read by the bridge |
|
||||
| `VESTIGE_MEMORY_DIR` | (auto) | Override per-user Claude memory dir |
|
||||
|
||||
---
|
||||
|
|
@ -151,11 +186,12 @@ Full architecture memory: search Vestige for `god-tier-plan` or `cognitive-sandw
|
|||
|
||||
---
|
||||
|
||||
## Linux / Intel Mac
|
||||
## Linux / Intel Mac / x86
|
||||
|
||||
The launchd layer is macOS-arm64-only. On Linux or Intel Mac:
|
||||
- Hooks + agents install fine with `--no-launchd`
|
||||
- The Sanhedrin Stop hook will fail-open (mlx-server unreachable → exit 0)
|
||||
- Optional: run a remote mlx_lm.server / vLLM / Ollama OpenAI-compatible endpoint and set `MLX_ENDPOINT` to its `/v1/chat/completions` URL
|
||||
The base hook harness runs on x86. The launchd MLX helper is macOS-arm64-only.
|
||||
|
||||
Future v2.2.0 will add Linux-native MLX equivalents.
|
||||
On Linux, Windows under WSL, or Intel Mac:
|
||||
- Run `scripts/install-sandwich.sh` normally to stage files and remove old Vestige hook wiring. No hooks are activated.
|
||||
- If you want Sanhedrin, run an OpenAI-compatible endpoint such as vLLM, Ollama, llama.cpp server, or a remote MLX/vLLM box.
|
||||
- Install with `--enable-sanhedrin --sanhedrin-endpoint=<url> --sanhedrin-model=<model>`.
|
||||
- If the endpoint is unreachable, Sanhedrin fails open and does not block Claude Code.
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
|
||||
## First-Run Network Requirement
|
||||
|
||||
Vestige downloads the **Nomic Embed Text v1.5** model (~130MB) from Hugging Face on first use.
|
||||
Vestige downloads the **Nomic Embed Text v1.5** model (~130MB) from Hugging Face on first use. Qwen3 embeddings are opt-in and download their own Hugging Face model when selected.
|
||||
|
||||
**All subsequent runs are fully offline.**
|
||||
|
||||
|
|
@ -25,6 +25,8 @@ Override with environment variable:
|
|||
export FASTEMBED_CACHE_PATH="/custom/path"
|
||||
```
|
||||
|
||||
Qwen3 currently uses Hugging Face Hub's Candle loader directly, so use the standard Hugging Face cache environment such as `HF_HOME` if you need to relocate that larger model cache.
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
|
@ -32,6 +34,7 @@ export FASTEMBED_CACHE_PATH="/custom/path"
|
|||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `VESTIGE_DATA_DIR` | OS per-user data directory | Storage directory fallback; overridden by `--data-dir`; database lives at `<dir>/vestige.db` |
|
||||
| `VESTIGE_EMBEDDING_MODEL` | `nomic-v1.5` | Embedding backend selector. Use `qwen3-0.6b` with a build that enables `qwen3-embeddings` |
|
||||
| `RUST_LOG` | `info` (via tracing-subscriber) | Log verbosity + per-module filtering |
|
||||
| `FASTEMBED_CACHE_PATH` | `./.fastembed_cache` | Embedding model cache location |
|
||||
| `VESTIGE_DASHBOARD_PORT` | `3927` | Dashboard HTTP + WebSocket port |
|
||||
|
|
@ -50,6 +53,7 @@ export FASTEMBED_CACHE_PATH="/custom/path"
|
|||
```bash
|
||||
vestige-mcp --data-dir /custom/path # Custom storage location
|
||||
VESTIGE_DATA_DIR=~/.vestige vestige-mcp # Env fallback storage location
|
||||
VESTIGE_DATA_DIR=./.vestige vestige stats # Point the CLI at the same custom DB
|
||||
vestige-mcp --help # Show all options
|
||||
```
|
||||
|
||||
|
|
@ -66,6 +70,10 @@ vestige stats --states # Cognitive state distribution
|
|||
vestige health # System health check
|
||||
vestige consolidate # Run memory maintenance
|
||||
vestige restore <file> # Restore from backup
|
||||
vestige portable-export <file> # Exact Vestige-to-Vestige archive
|
||||
vestige portable-import <file> # Import exact archive into an empty database
|
||||
vestige portable-import <file> --merge # Merge exact archive into this database
|
||||
vestige sync <file> # Pull/merge/push through a file backend
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -169,7 +177,7 @@ vestige update
|
|||
|
||||
**Pin to specific version:**
|
||||
```bash
|
||||
vestige update --version v2.1.0
|
||||
vestige update --version v2.1.1
|
||||
```
|
||||
|
||||
**Check your version:**
|
||||
|
|
|
|||
|
|
@ -24,6 +24,44 @@ Override precedence:
|
|||
|
||||
---
|
||||
|
||||
## Moving Memories Between Devices
|
||||
|
||||
For device-to-device migration, use a portable archive instead of the normal JSON export:
|
||||
|
||||
```bash
|
||||
# On the source machine
|
||||
vestige portable-export ~/Desktop/vestige-portable.json
|
||||
|
||||
# On the destination machine, before adding memories
|
||||
vestige portable-import ~/Desktop/vestige-portable.json
|
||||
```
|
||||
|
||||
Portable archives preserve raw Vestige storage rows: memory IDs, FSRS state, graph connections, suppression state, timestamps, audit history, and embedding blobs.
|
||||
|
||||
For one-time migration, keep the conservative empty-database import:
|
||||
|
||||
```bash
|
||||
vestige portable-import ~/Desktop/vestige-portable.json
|
||||
```
|
||||
|
||||
For cross-device sync, use merge mode or the file-backed sync command:
|
||||
|
||||
```bash
|
||||
# Merge a portable archive into an existing database.
|
||||
vestige portable-import ~/Dropbox/vestige/portable.json --merge
|
||||
|
||||
# Pull, merge, and push through a shared archive file.
|
||||
vestige sync ~/Dropbox/vestige/portable.json
|
||||
```
|
||||
|
||||
`vestige sync` uses the same pluggable portable-sync backend interface as the core library. v2.1.1 ships a file backend, which works with Dropbox, iCloud Drive, Syncthing, Git, network shares, or any folder-sync system. The merge algorithm applies delete tombstones, keeps newer local memories on timestamp conflicts, preserves stable IDs, rebuilds FTS after import, and writes the pushed archive atomically when the filesystem supports rename.
|
||||
|
||||
When using the MCP `export` tool with `format: "portable"`, Vestige writes the archive under the active data directory's `exports/` folder. The MCP `restore` tool only reads from that `exports/` or `backups/` folder by default; pass `allowAnyPath: true` only for a trusted local file you selected manually.
|
||||
|
||||
The regular `vestige export` / `vestige restore` path remains useful for human-readable backups, partial exports, and older files, but it re-ingests memory content and does not preserve every storage-level relationship.
|
||||
|
||||
---
|
||||
|
||||
## Storage Modes
|
||||
|
||||
### Option 1: Global Memory (Default)
|
||||
|
|
@ -69,6 +107,13 @@ This creates `.vestige/vestige.db` in your project root. Add `.vestige/` to `.gi
|
|||
|
||||
If both `VESTIGE_DATA_DIR` and `--data-dir` are set, the CLI flag wins. Use the env var for a machine-wide default and the CLI flag for per-client or per-project overrides.
|
||||
|
||||
The `vestige` CLI also honors `VESTIGE_DATA_DIR`, so use the same directory when inspecting or exporting a custom MCP instance:
|
||||
|
||||
```bash
|
||||
VESTIGE_DATA_DIR=./.vestige vestige stats
|
||||
VESTIGE_DATA_DIR=./.vestige vestige portable-export ./vestige-portable.json
|
||||
```
|
||||
|
||||
**Multiple Named Instances:**
|
||||
|
||||
For power users who want both global AND project memory:
|
||||
|
|
@ -132,7 +177,7 @@ Claude Code config - for "Storm":
|
|||
|
||||
## Data Safety
|
||||
|
||||
**Important:** Vestige stores data locally with no cloud sync, redundancy, or automatic backup.
|
||||
**Important:** Vestige stores data locally. v2.1.1 adds user-controlled file-backed sync through `vestige sync`, but Vestige does not run a hosted cloud service, background replication daemon, or automatic backup for you.
|
||||
|
||||
| Use Case | Risk Level | Recommendation |
|
||||
|----------|------------|----------------|
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# sanhedrin-local.py — Local Qwen3.6-35B-A3B Sanhedrin Executioner.
|
||||
# sanhedrin-local.py — OpenAI-compatible Sanhedrin Executioner bridge.
|
||||
# Drop-in replacement for the Haiku 4.5 subagent that sanhedrin.sh used to spawn.
|
||||
#
|
||||
# Reads draft from stdin, prints single-line verdict to stdout:
|
||||
|
|
@ -8,10 +8,10 @@
|
|||
#
|
||||
# Architecture:
|
||||
# stdin (draft) -> Vestige /api/deep_reference (single semantic query)
|
||||
# -> mlx_lm.server localhost:8080 (one-shot judgment)
|
||||
# -> OpenAI-compatible chat endpoint (one-shot judgment)
|
||||
# -> stdout (single-line verdict)
|
||||
#
|
||||
# Fail-open: if mlx-server unreachable, print "yes" and exit 0 (don't break
|
||||
# Fail-open: if the endpoint is unreachable, print "yes" and exit 0 (don't break
|
||||
# the Cognitive Sandwich on infra errors). The wrapping sanhedrin.sh maps
|
||||
# "yes" to exit 0, so this preserves existing fail-open semantics.
|
||||
|
||||
|
|
@ -35,7 +35,11 @@ VESTIGE_BASE_URL = (
|
|||
os.environ.get("VESTIGE_BASE_URL") or f"http://127.0.0.1:{DASHBOARD_PORT}"
|
||||
).rstrip("/")
|
||||
|
||||
MLX_ENDPOINT = os.environ.get("MLX_ENDPOINT") or "http://127.0.0.1:8080/v1/chat/completions"
|
||||
SANHEDRIN_ENDPOINT = (
|
||||
os.environ.get("VESTIGE_SANHEDRIN_ENDPOINT")
|
||||
or os.environ.get("MLX_ENDPOINT")
|
||||
or "http://127.0.0.1:8080/v1/chat/completions"
|
||||
)
|
||||
VESTIGE_ENDPOINT = (
|
||||
os.environ.get("VESTIGE_DEEP_REFERENCE_ENDPOINT")
|
||||
or f"{VESTIGE_BASE_URL}/api/deep_reference"
|
||||
|
|
@ -43,8 +47,12 @@ VESTIGE_ENDPOINT = (
|
|||
VESTIGE_HEALTH = (
|
||||
os.environ.get("VESTIGE_HEALTH_ENDPOINT") or f"{VESTIGE_BASE_URL}/api/health"
|
||||
)
|
||||
MODEL = os.environ.get("VESTIGE_SANDWICH_MODEL") or "mlx-community/Qwen3.6-35B-A3B-4bit"
|
||||
MLX_TIMEOUT = env_int("MLX_TIMEOUT", 45)
|
||||
MODEL = (
|
||||
os.environ.get("VESTIGE_SANHEDRIN_MODEL")
|
||||
or os.environ.get("VESTIGE_SANDWICH_MODEL")
|
||||
or "mlx-community/Qwen3.6-35B-A3B-4bit"
|
||||
)
|
||||
SANHEDRIN_TIMEOUT = env_int("VESTIGE_SANHEDRIN_TIMEOUT", env_int("MLX_TIMEOUT", 45))
|
||||
VESTIGE_TIMEOUT = env_int("VESTIGE_TIMEOUT", 5)
|
||||
THINK_RE = re.compile(r"<think>.*?</think>", re.DOTALL | re.IGNORECASE)
|
||||
|
||||
|
|
@ -289,7 +297,7 @@ def judge(draft: str, evidence: str) -> str:
|
|||
"\n\nOn second thought", "\n\nOh wait",
|
||||
],
|
||||
}
|
||||
resp = post_json(MLX_ENDPOINT, body, MLX_TIMEOUT)
|
||||
resp = post_json(SANHEDRIN_ENDPOINT, body, SANHEDRIN_TIMEOUT)
|
||||
if not isinstance(resp, dict):
|
||||
return ""
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -17,25 +17,37 @@
|
|||
# sanhedrin.sh (2-8s Haiku subagent, may block) →
|
||||
# synthesis-stop-validator.sh (existing regex hedge check, may block)
|
||||
#
|
||||
# Opt-in: set VESTIGE_SANHEDRIN_ENABLED=1 in parent shell.
|
||||
# Opt-in: set VESTIGE_SANHEDRIN_ENABLED=1 in parent shell, or install with
|
||||
# scripts/install-sandwich.sh --enable-sanhedrin.
|
||||
# Re-entrancy lock: VESTIGE_EXECUTIONER_ACTIVE=1 inside the subagent.
|
||||
#
|
||||
# Ship date 2026-04-20.
|
||||
|
||||
set -u
|
||||
|
||||
# === OPT-OUT GATE ===
|
||||
# Post-Cognitive Sanhedrin is ON by default as of 2026-04-21 (birthday
|
||||
# launch day). To disable, set VESTIGE_SANHEDRIN_ENABLED=0 in your
|
||||
# environment. Default-on guarantees the Cognitive Sandwich fires on
|
||||
# fresh machines, Docker containers, GUI-launched Claude Code, and
|
||||
# shells without .zshrc — any case where the Claude Code process lacks
|
||||
# a sourced profile. The re-entrancy guard (VESTIGE_EXECUTIONER_ACTIVE)
|
||||
# below still prevents fork-bombs from the subagent's own Stop hook.
|
||||
if [ "${VESTIGE_SANHEDRIN_ENABLED:-1}" = "0" ]; then
|
||||
exit 0
|
||||
# === OPT-IN GATE ===
|
||||
# Sanhedrin is heavyweight: the default local backend is a ~19 GB model and
|
||||
# needs roughly 20+ GB of free RAM. Keep it disabled unless the user explicitly
|
||||
# opts in. The installer writes this env file only for --enable-sanhedrin.
|
||||
SANHEDRIN_ENV="${VESTIGE_SANHEDRIN_ENV:-$HOME/.claude/hooks/vestige-sanhedrin.env}"
|
||||
if [ -f "$SANHEDRIN_ENV" ]; then
|
||||
set +u
|
||||
set -a
|
||||
# shellcheck disable=SC1090
|
||||
. "$SANHEDRIN_ENV" 2>/dev/null || {
|
||||
set +a
|
||||
set -u
|
||||
exit 0
|
||||
}
|
||||
set +a
|
||||
set -u
|
||||
fi
|
||||
|
||||
case "${VESTIGE_SANHEDRIN_ENABLED:-0}" in
|
||||
1|true|TRUE|yes|YES|on|ON) ;;
|
||||
*) exit 0 ;;
|
||||
esac
|
||||
|
||||
# === RE-ENTRANCY GUARD ===
|
||||
# The Executioner's own Stop hook will fire when it returns — prevent
|
||||
# recursive spawns that would fork-bomb the quota.
|
||||
|
|
@ -114,11 +126,11 @@ if [ -z "$DRAFT" ]; then
|
|||
fi
|
||||
|
||||
# === VERIFY local executioner bridge available ===
|
||||
# 2026-04-25: switched from Haiku 4.5 subagent to local Qwen3.6-35B-A3B
|
||||
# via mlx_lm.server (launchd com.vestige.mlx-server). Bridge script
|
||||
# fetches Vestige evidence via HTTP API (VESTIGE_DASHBOARD_PORT, default 3927)
|
||||
# then judges via MLX_ENDPOINT (default port 8080). Zero per-token cost, fully offline,
|
||||
# sub-second-to-15s verdict latency. Fail-open if mlx-server unreachable.
|
||||
# 2026-04-25: switched from Haiku 4.5 subagent to an OpenAI-compatible
|
||||
# local/remote endpoint. On Apple Silicon the optional launchd path starts
|
||||
# mlx_lm.server; on x86 users can point VESTIGE_SANHEDRIN_ENDPOINT at vLLM,
|
||||
# Ollama, llama.cpp, or any compatible /v1/chat/completions endpoint.
|
||||
# Fail-open if the endpoint is unreachable.
|
||||
BRIDGE="$HOME/.claude/hooks/sanhedrin-local.py"
|
||||
if [ ! -x "$BRIDGE" ] && [ ! -f "$BRIDGE" ]; then
|
||||
exit 0
|
||||
|
|
@ -191,15 +203,15 @@ case "$TRIMMED" in
|
|||
|
||||
$REASON
|
||||
|
||||
The Executioner (local Qwen3.6-35B-A3B via mlx_lm.server, fresh context,
|
||||
fed Vestige deep_reference evidence over HTTP) judged your draft and
|
||||
The Executioner (Sanhedrin endpoint, fresh context, fed Vestige
|
||||
deep_reference evidence over HTTP) judged your draft and
|
||||
found a contradiction against a high-trust memory.
|
||||
|
||||
You may NOT stop. Rewrite WITHOUT the contradicted claim. Use
|
||||
mcp__vestige__deep_reference to inspect the cited memory and cite the
|
||||
correct replacement pattern from its \`recommended\` field.
|
||||
|
||||
Local-only, zero API cost, fully offline. Bridge script:
|
||||
Bridge script:
|
||||
~/.claude/hooks/sanhedrin-local.py
|
||||
SANHEDRIN_MSG
|
||||
exit 2
|
||||
|
|
|
|||
|
|
@ -1,23 +1 @@
|
|||
{
|
||||
"hooks": {
|
||||
"UserPromptSubmit": [
|
||||
{
|
||||
"hooks": [
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/synthesis-preflight.sh", "timeout": 8 },
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/cwd-state-injector.sh", "timeout": 8 },
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/vestige-pulse-daemon.sh", "timeout": 6 },
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/preflight-swarm.sh", "timeout": 45 }
|
||||
]
|
||||
}
|
||||
],
|
||||
"Stop": [
|
||||
{
|
||||
"hooks": [
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/veto-detector.sh", "timeout": 6 },
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/sanhedrin.sh", "timeout": 70 },
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/synthesis-stop-validator.sh", "timeout": 6 }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
{}
|
||||
|
|
|
|||
14
hooks/settings.preflight.fragment.json
Normal file
14
hooks/settings.preflight.fragment.json
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
{
|
||||
"hooks": {
|
||||
"UserPromptSubmit": [
|
||||
{
|
||||
"hooks": [
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/synthesis-preflight.sh", "timeout": 8 },
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/cwd-state-injector.sh", "timeout": 8 },
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/vestige-pulse-daemon.sh", "timeout": 6 },
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/preflight-swarm.sh", "timeout": 45 }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
13
hooks/settings.sanhedrin.fragment.json
Normal file
13
hooks/settings.sanhedrin.fragment.json
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
{
|
||||
"hooks": {
|
||||
"Stop": [
|
||||
{
|
||||
"hooks": [
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/veto-detector.sh", "timeout": 6 },
|
||||
{ "type": "command", "command": "VESTIGE_SANHEDRIN_ENABLED=1 $HOME/.claude/hooks/sanhedrin.sh", "timeout": 70 },
|
||||
{ "type": "command", "command": "$HOME/.claude/hooks/synthesis-stop-validator.sh", "timeout": 6 }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "vestige",
|
||||
"version": "2.1.0",
|
||||
"version": "2.1.1",
|
||||
"private": true,
|
||||
"description": "Cognitive memory for AI - MCP server with FSRS-6 spaced repetition",
|
||||
"author": "Sam Valladares",
|
||||
|
|
|
|||
|
|
@ -5,21 +5,53 @@ set -u
|
|||
ok() { printf ' \033[1;32m[ OK ]\033[0m %s\n' "$*"; }
|
||||
warn() { printf ' \033[1;33m[WARN]\033[0m %s\n' "$*"; FAIL=1; }
|
||||
miss() { printf ' \033[1;31m[MISS]\033[0m %s\n' "$*"; FAIL=1; }
|
||||
info() { printf ' \033[1;36m[INFO]\033[0m %s\n' "$*"; }
|
||||
|
||||
FAIL=0
|
||||
CHECK_PREFLIGHT=0
|
||||
CHECK_SANHEDRIN=0
|
||||
DASHBOARD_PORT="${VESTIGE_DASHBOARD_PORT:-3927}"
|
||||
MLX_ENDPOINT="${MLX_ENDPOINT:-http://127.0.0.1:8080/v1/chat/completions}"
|
||||
MLX_ENDPOINT="${MLX_ENDPOINT%/}"
|
||||
MLX_MODELS_URL="${MLX_ENDPOINT%/chat/completions}/models"
|
||||
SANHEDRIN_ENV="${VESTIGE_SANHEDRIN_ENV:-$HOME/.claude/hooks/vestige-sanhedrin.env}"
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--preflight|--enable-preflight) CHECK_PREFLIGHT=1 ;;
|
||||
--sanhedrin|--enable-sanhedrin) CHECK_SANHEDRIN=1 ;;
|
||||
-h|--help)
|
||||
cat <<'EOF'
|
||||
Usage: scripts/check-sandwich-prereqs.sh [--preflight] [--sanhedrin]
|
||||
|
||||
Without flags, verifies that the default install has no Vestige hooks wired.
|
||||
With --preflight, checks the optional UserPromptSubmit hook layer.
|
||||
With --sanhedrin, checks the optional OpenAI-compatible verifier endpoint.
|
||||
EOF
|
||||
exit 0
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ -f "$SANHEDRIN_ENV" ]; then
|
||||
set +u
|
||||
set -a
|
||||
# shellcheck disable=SC1090
|
||||
. "$SANHEDRIN_ENV" 2>/dev/null || true
|
||||
set +a
|
||||
set -u
|
||||
fi
|
||||
|
||||
SANHEDRIN_ENDPOINT="${VESTIGE_SANHEDRIN_ENDPOINT:-${MLX_ENDPOINT:-http://127.0.0.1:8080/v1/chat/completions}}"
|
||||
SANHEDRIN_ENDPOINT="${SANHEDRIN_ENDPOINT%/}"
|
||||
SANHEDRIN_MODELS_URL="${SANHEDRIN_ENDPOINT%/chat/completions}/models"
|
||||
|
||||
echo "Vestige Cognitive Sandwich — Prereq Check"
|
||||
echo
|
||||
|
||||
# Platform
|
||||
if [ "$(uname -s)" = "Darwin" ] && [ "$(uname -m)" = "arm64" ]; then
|
||||
ok "Apple Silicon macOS ($(sw_vers -productVersion 2>/dev/null || echo darwin))"
|
||||
else
|
||||
miss "Apple Silicon Mac required (M1+). Detected $(uname -s) $(uname -m)."
|
||||
OS_NAME="$(uname -s)"
|
||||
ARCH_NAME="$(uname -m)"
|
||||
ok "Platform: $OS_NAME $ARCH_NAME"
|
||||
if [ "$OS_NAME" != "Darwin" ] || [ "$ARCH_NAME" != "arm64" ]; then
|
||||
info "Local MLX launchd is Apple Silicon-only; base hooks and endpoint-backed Sanhedrin can run on x86."
|
||||
fi
|
||||
|
||||
# Python
|
||||
|
|
@ -35,56 +67,95 @@ fi
|
|||
|
||||
# CLI tools
|
||||
command -v jq >/dev/null && ok "jq" || miss "jq missing — brew install jq"
|
||||
command -v uv >/dev/null && ok "uv" || miss "uv missing — brew install uv"
|
||||
command -v mlx_lm.server >/dev/null && ok "mlx-lm" || miss "mlx-lm — uv tool install mlx-lm"
|
||||
command -v hf >/dev/null && ok "huggingface_hub CLI" || miss "hf — uv tool install 'huggingface_hub[cli]'"
|
||||
command -v claude >/dev/null && ok "claude CLI" || miss "claude CLI — install Claude Code"
|
||||
command -v vestige-mcp >/dev/null && ok "vestige-mcp" || miss "vestige-mcp — cargo install vestige-mcp"
|
||||
if [ "$CHECK_PREFLIGHT" -eq 1 ]; then
|
||||
command -v claude >/dev/null && ok "claude CLI" || miss "claude CLI — install Claude Code"
|
||||
command -v vestige-mcp >/dev/null && ok "vestige-mcp" || miss "vestige-mcp — cargo install vestige-mcp"
|
||||
|
||||
# Model on disk — HF cache uses `models--<org>--<name>` (double-dash separators).
|
||||
MODEL="${VESTIGE_SANDWICH_MODEL:-mlx-community/Qwen3.6-35B-A3B-4bit}"
|
||||
HF_HOME_DEFAULT="${HF_HOME:-$HOME/.cache/huggingface}"
|
||||
ENC_MODEL="models--$(printf '%s' "$MODEL" | sed 's|/|--|g')"
|
||||
if [ -d "$HF_HOME_DEFAULT/hub/$ENC_MODEL" ]; then
|
||||
ok "Model cached: $MODEL"
|
||||
else
|
||||
printf ' \033[1;33m[INFO]\033[0m Model not yet downloaded — first run will fetch ~19GB\n'
|
||||
printf ' hf download %s\n' "$MODEL"
|
||||
# NOT a failure — first-run download is expected.
|
||||
fi
|
||||
|
||||
# Vestige MCP HTTP API
|
||||
if curl -fsS -m 2 "http://127.0.0.1:${DASHBOARD_PORT}/api/health" >/dev/null 2>&1; then
|
||||
ok "vestige-mcp dashboard responding on :$DASHBOARD_PORT"
|
||||
else
|
||||
warn "vestige-mcp dashboard not responding on :$DASHBOARD_PORT"
|
||||
fi
|
||||
|
||||
# OpenAI-compatible local/remote model endpoint
|
||||
if curl -fsS -m 2 "$MLX_MODELS_URL" >/dev/null 2>&1; then
|
||||
ok "model endpoint responding at $MLX_MODELS_URL"
|
||||
else
|
||||
warn "model endpoint not responding at $MLX_MODELS_URL — install + load launchd plist or set MLX_ENDPOINT"
|
||||
fi
|
||||
|
||||
# launchd plist
|
||||
if [ -f "$HOME/Library/LaunchAgents/com.vestige.mlx-server.plist" ]; then
|
||||
ok "launchd plist installed"
|
||||
else
|
||||
warn "launchd plist missing — run: install-sandwich.sh"
|
||||
# Vestige MCP HTTP API
|
||||
if curl -fsS -m 2 "http://127.0.0.1:${DASHBOARD_PORT}/api/health" >/dev/null 2>&1; then
|
||||
ok "vestige-mcp dashboard responding on :$DASHBOARD_PORT"
|
||||
else
|
||||
warn "vestige-mcp dashboard not responding on :$DASHBOARD_PORT"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Settings hook wiring
|
||||
if [ -f "$HOME/.claude/settings.json" ] && \
|
||||
jq -e '.hooks.UserPromptSubmit and .hooks.Stop' "$HOME/.claude/settings.json" >/dev/null 2>&1; then
|
||||
ok "settings.json hooks block present"
|
||||
if [ "$CHECK_PREFLIGHT" -eq 0 ] && [ "$CHECK_SANHEDRIN" -eq 0 ]; then
|
||||
if [ -f "$HOME/.claude/settings.json" ] && \
|
||||
jq -e 'any((.hooks.UserPromptSubmit[]?.hooks[]?, .hooks.Stop[]?.hooks[]?); ((.command? // "") | test("synthesis-preflight\\.sh|cwd-state-injector\\.sh|vestige-pulse-daemon\\.sh|preflight-swarm\\.sh|load-all-memory\\.sh|veto-detector\\.sh|sanhedrin\\.sh|synthesis-stop-validator\\.sh|synthesis-gate\\.sh")))' "$HOME/.claude/settings.json" >/dev/null 2>&1; then
|
||||
warn "Vestige hooks are still wired; run: install-sandwich.sh --force"
|
||||
else
|
||||
ok "no Vestige Claude Code hooks wired by default"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$CHECK_PREFLIGHT" -eq 1 ]; then
|
||||
echo
|
||||
echo "Optional Preflight"
|
||||
|
||||
if [ -f "$HOME/.claude/settings.json" ] && \
|
||||
jq -e 'any(.hooks.UserPromptSubmit[]?.hooks[]?; ((.command? // "") | contains("synthesis-preflight.sh"))) and any(.hooks.UserPromptSubmit[]?.hooks[]?; ((.command? // "") | contains("preflight-swarm.sh")))' "$HOME/.claude/settings.json" >/dev/null 2>&1; then
|
||||
ok "preflight UserPromptSubmit hooks wired"
|
||||
else
|
||||
warn "preflight hooks not wired — run: install-sandwich.sh --enable-preflight"
|
||||
fi
|
||||
|
||||
info "preflight-swarm.sh uses claude -p with Haiku when enabled; default installs do not wire it."
|
||||
fi
|
||||
|
||||
if [ "$CHECK_SANHEDRIN" -eq 1 ]; then
|
||||
echo
|
||||
echo "Optional Sanhedrin"
|
||||
|
||||
if [ -f "$SANHEDRIN_ENV" ]; then
|
||||
ok "Sanhedrin env file present"
|
||||
else
|
||||
warn "Sanhedrin env file missing — run: install-sandwich.sh --enable-sanhedrin"
|
||||
fi
|
||||
|
||||
if [ "$OS_NAME" = "Darwin" ] && [ "$ARCH_NAME" = "arm64" ]; then
|
||||
command -v uv >/dev/null && ok "uv" || warn "uv missing — brew install uv"
|
||||
command -v mlx_lm.server >/dev/null && ok "mlx-lm" || warn "mlx-lm — uv tool install mlx-lm"
|
||||
command -v hf >/dev/null && ok "huggingface_hub CLI" || warn "hf — uv tool install 'huggingface_hub[cli]'"
|
||||
|
||||
MODEL="${VESTIGE_SANHEDRIN_MODEL:-${VESTIGE_SANDWICH_MODEL:-mlx-community/Qwen3.6-35B-A3B-4bit}}"
|
||||
HF_HOME_DEFAULT="${HF_HOME:-$HOME/.cache/huggingface}"
|
||||
ENC_MODEL="models--$(printf '%s' "$MODEL" | sed 's|/|--|g')"
|
||||
if [ -d "$HF_HOME_DEFAULT/hub/$ENC_MODEL" ]; then
|
||||
ok "Model cached: $MODEL"
|
||||
else
|
||||
info "Model not cached: $MODEL (local MLX path downloads ~19GB)"
|
||||
fi
|
||||
|
||||
if [ -f "$HOME/Library/LaunchAgents/com.vestige.mlx-server.plist" ]; then
|
||||
ok "launchd plist installed"
|
||||
else
|
||||
info "launchd plist not installed; endpoint-backed Sanhedrin can still run"
|
||||
fi
|
||||
else
|
||||
info "Skipping MLX/launchd checks on $OS_NAME $ARCH_NAME"
|
||||
fi
|
||||
|
||||
if curl -fsS -m 2 "$SANHEDRIN_MODELS_URL" >/dev/null 2>&1; then
|
||||
ok "Sanhedrin model endpoint responding at $SANHEDRIN_MODELS_URL"
|
||||
else
|
||||
warn "Sanhedrin endpoint not responding at $SANHEDRIN_MODELS_URL"
|
||||
fi
|
||||
|
||||
if [ -f "$HOME/.claude/settings.json" ] && \
|
||||
jq -e '.hooks.Stop[]?.hooks[]?.command | contains("sanhedrin.sh")' "$HOME/.claude/settings.json" >/dev/null 2>&1; then
|
||||
ok "Sanhedrin Stop hook wired"
|
||||
else
|
||||
warn "Sanhedrin Stop hook not wired — run: install-sandwich.sh --enable-sanhedrin"
|
||||
fi
|
||||
else
|
||||
warn "settings.json missing hooks block — run: install-sandwich.sh"
|
||||
echo
|
||||
info "Sanhedrin is optional and not checked. Use --sanhedrin to verify an enabled endpoint."
|
||||
fi
|
||||
|
||||
echo
|
||||
if [ $FAIL -eq 0 ]; then
|
||||
echo " Ready. Cognitive Sandwich will fire on next Claude Code prompt."
|
||||
echo " Ready. Default install has no Vestige Claude Code hooks wired and makes no automatic model calls."
|
||||
exit 0
|
||||
else
|
||||
echo " Fix the items above, then re-run."
|
||||
|
|
|
|||
|
|
@ -2,28 +2,28 @@
|
|||
# install-sandwich.sh — One-command installer for the Vestige Cognitive Sandwich.
|
||||
#
|
||||
# Usage:
|
||||
# curl -fsSL https://raw.githubusercontent.com/samvallad33/vestige/v2.1.0/scripts/install-sandwich.sh | sh
|
||||
# curl -fsSL https://raw.githubusercontent.com/samvallad33/vestige/v2.1.1/scripts/install-sandwich.sh | sh
|
||||
# # or, from a checkout:
|
||||
# ./scripts/install-sandwich.sh [--force] [--no-launchd] [--include-memory-loader]
|
||||
# ./scripts/install-sandwich.sh [--force] [--enable-preflight] [--enable-sanhedrin] [--with-launchd] [--include-memory-loader]
|
||||
# ./scripts/install-sandwich.sh --enable-sanhedrin --sanhedrin-endpoint=http://127.0.0.1:11434/v1/chat/completions --sanhedrin-model=qwen2.5:14b
|
||||
#
|
||||
# What it does:
|
||||
# 1. Verifies required local tools
|
||||
# 2. Stages ~/.claude/hooks/, ~/.claude/agents/, ~/Library/LaunchAgents/
|
||||
# 2. Stages ~/.claude/hooks/ and ~/.claude/agents/
|
||||
# 3. Copies sanitized hooks + agents
|
||||
# 4. Renders launchd plist template with $HOME and chosen MODEL
|
||||
# 5. Merges hooks block into ~/.claude/settings.json (preserves existing keys)
|
||||
# 6. launchctl load com.vestige.mlx-server (auto-starts mlx_lm.server with Qwen3.6-35B-A3B)
|
||||
# 7. Prints next-steps for model download
|
||||
# 4. Removes old Vestige hook wiring from ~/.claude/settings.json by default
|
||||
# 5. Optionally enables preflight hooks and/or Sanhedrin. Only with --with-launchd on Apple Silicon,
|
||||
# auto-starts mlx_lm.server with Qwen3.6-35B-A3B
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VERSION="${VESTIGE_SANDWICH_VERSION:-v2.1.0}"
|
||||
VERSION="${VESTIGE_SANDWICH_VERSION:-v2.1.1}"
|
||||
REPO="samvallad33/vestige"
|
||||
MODEL_ID="${VESTIGE_SANDWICH_MODEL:-mlx-community/Qwen3.6-35B-A3B-4bit}"
|
||||
MODEL_ID="${VESTIGE_SANHEDRIN_MODEL:-${VESTIGE_SANDWICH_MODEL:-mlx-community/Qwen3.6-35B-A3B-4bit}}"
|
||||
DASHBOARD_PORT="${VESTIGE_DASHBOARD_PORT:-3927}"
|
||||
MLX_ENDPOINT="${MLX_ENDPOINT:-http://127.0.0.1:8080/v1/chat/completions}"
|
||||
MLX_ENDPOINT="${MLX_ENDPOINT%/}"
|
||||
MLX_MODELS_URL="${MLX_ENDPOINT%/chat/completions}/models"
|
||||
SANHEDRIN_ENDPOINT="${VESTIGE_SANHEDRIN_ENDPOINT:-${MLX_ENDPOINT:-http://127.0.0.1:8080/v1/chat/completions}}"
|
||||
SANHEDRIN_ENDPOINT="${SANHEDRIN_ENDPOINT%/}"
|
||||
SANHEDRIN_MODELS_URL="${SANHEDRIN_ENDPOINT%/chat/completions}/models"
|
||||
|
||||
HOOKS_DIR="$HOME/.claude/hooks"
|
||||
AGENTS_DIR="$HOME/.claude/agents"
|
||||
|
|
@ -31,45 +31,70 @@ LAUNCHD_DIR="$HOME/Library/LaunchAgents"
|
|||
SETTINGS="$HOME/.claude/settings.json"
|
||||
|
||||
FORCE=0
|
||||
NO_LAUNCHD=0
|
||||
ENABLE_PREFLIGHT=0
|
||||
ENABLE_SANHEDRIN=0
|
||||
WITH_LAUNCHD=0
|
||||
INCLUDE_MEMORY_LOADER=0
|
||||
SRC=""
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--force) FORCE=1 ;;
|
||||
--no-launchd) NO_LAUNCHD=1 ;;
|
||||
--enable-preflight) ENABLE_PREFLIGHT=1 ;;
|
||||
--enable-sandwich) ENABLE_PREFLIGHT=1; ENABLE_SANHEDRIN=1 ;;
|
||||
--enable-sanhedrin) ENABLE_SANHEDRIN=1 ;;
|
||||
--with-launchd) WITH_LAUNCHD=1 ;;
|
||||
--no-launchd) WITH_LAUNCHD=0 ;;
|
||||
--include-memory-loader) INCLUDE_MEMORY_LOADER=1 ;;
|
||||
--sanhedrin-endpoint=*|--endpoint=*)
|
||||
SANHEDRIN_ENDPOINT="${arg#*=}"
|
||||
SANHEDRIN_ENDPOINT="${SANHEDRIN_ENDPOINT%/}"
|
||||
SANHEDRIN_MODELS_URL="${SANHEDRIN_ENDPOINT%/chat/completions}/models"
|
||||
;;
|
||||
--sanhedrin-model=*|--model=*)
|
||||
MODEL_ID="${arg#*=}"
|
||||
;;
|
||||
--src=*) SRC="${arg#--src=}" ;;
|
||||
-h|--help)
|
||||
sed -n '2,20p' "$0"
|
||||
sed -n '2,24p' "$0"
|
||||
exit 0
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ "$WITH_LAUNCHD" -eq 1 ] && [ "$ENABLE_SANHEDRIN" -eq 0 ]; then
|
||||
ENABLE_SANHEDRIN=1
|
||||
fi
|
||||
|
||||
say() { printf '\033[1;36m[sandwich]\033[0m %s\n' "$*"; }
|
||||
warn() { printf '\033[1;33m[sandwich]\033[0m %s\n' "$*" >&2; }
|
||||
die() { printf '\033[1;31m[sandwich]\033[0m %s\n' "$*" >&2; exit 1; }
|
||||
|
||||
# --- Platform check (honors --no-launchd for Linux/Intel users) ---
|
||||
if [ "$(uname -s)" != "Darwin" ]; then
|
||||
if [ "$NO_LAUNCHD" -eq 0 ]; then
|
||||
die "macOS required for the launchd auto-start of mlx_lm.server. Re-run with --no-launchd to install hooks only and run mlx_lm.server manually."
|
||||
fi
|
||||
warn "Non-Darwin platform — installing hooks/agents only (no launchd). Run an OpenAI-compatible model endpoint and set MLX_ENDPOINT if it is not $MLX_ENDPOINT."
|
||||
elif [ "$(uname -m)" != "arm64" ]; then
|
||||
warn "Apple Silicon recommended (M1+). Detected $(uname -m). The local Qwen3.6 model requires arm64 + Metal."
|
||||
# --- Platform check ---
|
||||
OS_NAME="$(uname -s)"
|
||||
ARCH_NAME="$(uname -m)"
|
||||
say "platform: $OS_NAME $ARCH_NAME"
|
||||
if [ "$ENABLE_SANHEDRIN" -eq 1 ] && [ "$WITH_LAUNCHD" -eq 0 ]; then
|
||||
say "Sanhedrin enabled without launchd; using OpenAI-compatible endpoint: $SANHEDRIN_ENDPOINT"
|
||||
fi
|
||||
if [ "$WITH_LAUNCHD" -eq 1 ] && { [ "$OS_NAME" != "Darwin" ] || [ "$ARCH_NAME" != "arm64" ]; }; then
|
||||
warn "--with-launchd is Apple Silicon only; skipping local MLX autostart on $OS_NAME $ARCH_NAME"
|
||||
warn "Sanhedrin can still run on x86 via --sanhedrin-endpoint or VESTIGE_SANHEDRIN_ENDPOINT."
|
||||
WITH_LAUNCHD=0
|
||||
fi
|
||||
|
||||
# --- Prereqs (warnings only, install proceeds) ---
|
||||
command -v jq >/dev/null || die "jq required: brew install jq"
|
||||
command -v python3 >/dev/null || die "python3 required (3.10+)"
|
||||
command -v claude >/dev/null || warn "'claude' CLI not found — install Claude Code first."
|
||||
command -v vestige-mcp >/dev/null || warn "'vestige-mcp' not found — install with: cargo install vestige-mcp"
|
||||
command -v uv >/dev/null || warn "'uv' not found — install with: brew install uv"
|
||||
command -v mlx_lm.server >/dev/null || warn "mlx-lm not installed — run: uv tool install mlx-lm"
|
||||
command -v hf >/dev/null || warn "'hf' not found — run: uv tool install 'huggingface_hub[cli]'"
|
||||
if [ "$ENABLE_PREFLIGHT" -eq 1 ]; then
|
||||
command -v claude >/dev/null || warn "'claude' CLI not found — preflight-swarm.sh will fail open."
|
||||
command -v vestige-mcp >/dev/null || warn "'vestige-mcp' not found — Vestige preflight hooks will fail open."
|
||||
fi
|
||||
if [ "$WITH_LAUNCHD" -eq 1 ]; then
|
||||
command -v uv >/dev/null || warn "'uv' not found — install with: brew install uv"
|
||||
command -v mlx_lm.server >/dev/null || warn "mlx-lm not installed — run: uv tool install mlx-lm"
|
||||
command -v hf >/dev/null || warn "'hf' not found — run: uv tool install 'huggingface_hub[cli]'"
|
||||
fi
|
||||
|
||||
# --- Resolve source: local checkout or release tarball ---
|
||||
if [ -n "$SRC" ]; then
|
||||
|
|
@ -89,7 +114,21 @@ fi
|
|||
[ -d "$SCRIPT_DIR/hooks" ] || die "hooks/ not found in $SCRIPT_DIR — wrong source?"
|
||||
|
||||
# --- Stage directories ---
|
||||
mkdir -p "$HOOKS_DIR" "$AGENTS_DIR" "$LAUNCHD_DIR"
|
||||
mkdir -p "$HOOKS_DIR" "$AGENTS_DIR"
|
||||
if [ "$WITH_LAUNCHD" -eq 1 ]; then
|
||||
mkdir -p "$LAUNCHD_DIR"
|
||||
fi
|
||||
|
||||
# v2.1.0 originally installed the MLX server as part of the default path.
|
||||
# Default reinstalls now retire that job; users can restore it with --with-launchd.
|
||||
if [ "$WITH_LAUNCHD" -eq 0 ] && [ "$OS_NAME" = "Darwin" ]; then
|
||||
LEGACY_PLIST="$LAUNCHD_DIR/com.vestige.mlx-server.plist"
|
||||
if [ -f "$LEGACY_PLIST" ]; then
|
||||
launchctl unload "$LEGACY_PLIST" 2>/dev/null || true
|
||||
rm -f "$LEGACY_PLIST"
|
||||
say "removed old Sanhedrin launchd job (use --with-launchd to opt back in)"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- Copy hooks ---
|
||||
copied=0; skipped=0
|
||||
|
|
@ -121,8 +160,25 @@ for f in "$SCRIPT_DIR/agents"/*.md; do
|
|||
done
|
||||
say "agents installed to $AGENTS_DIR"
|
||||
|
||||
# --- Render launchd plist (macOS only) ---
|
||||
if [ "$NO_LAUNCHD" -eq 0 ]; then
|
||||
# --- Persist optional Sanhedrin env ---
|
||||
quote_env() {
|
||||
printf "'%s'" "$(printf '%s' "$1" | sed "s/'/'\\\\''/g")"
|
||||
}
|
||||
|
||||
if [ "$ENABLE_SANHEDRIN" -eq 1 ]; then
|
||||
SANHEDRIN_ENV="$HOOKS_DIR/vestige-sanhedrin.env"
|
||||
{
|
||||
printf 'VESTIGE_SANHEDRIN_ENABLED=1\n'
|
||||
printf 'VESTIGE_SANHEDRIN_ENDPOINT=%s\n' "$(quote_env "$SANHEDRIN_ENDPOINT")"
|
||||
printf 'VESTIGE_SANHEDRIN_MODEL=%s\n' "$(quote_env "$MODEL_ID")"
|
||||
printf 'VESTIGE_DASHBOARD_PORT=%s\n' "$(quote_env "$DASHBOARD_PORT")"
|
||||
} > "$SANHEDRIN_ENV"
|
||||
chmod 0600 "$SANHEDRIN_ENV"
|
||||
say "Sanhedrin opt-in config written to $SANHEDRIN_ENV"
|
||||
fi
|
||||
|
||||
# --- Render launchd plist (Apple Silicon opt-in only) ---
|
||||
if [ "$WITH_LAUNCHD" -eq 1 ]; then
|
||||
PLIST="$LAUNCHD_DIR/com.vestige.mlx-server.plist"
|
||||
TEMPLATE="$SCRIPT_DIR/launchd/com.vestige.mlx-server.plist.template"
|
||||
[ -f "$TEMPLATE" ] || die "launchd template missing: $TEMPLATE"
|
||||
|
|
@ -140,31 +196,80 @@ else
|
|||
cp "$SETTINGS" "$HOME/.claude/settings.json.bak.pre-sandwich"
|
||||
fi
|
||||
TMP_MERGE="$(mktemp)"
|
||||
jq -s '.[0] * .[1]' "$SETTINGS" "$SCRIPT_DIR/hooks/settings.fragment.json" > "$TMP_MERGE"
|
||||
PREFLIGHT_FRAGMENT="$SCRIPT_DIR/hooks/settings.fragment.json"
|
||||
SANHEDRIN_FRAGMENT="$SCRIPT_DIR/hooks/settings.fragment.json"
|
||||
if [ "$ENABLE_PREFLIGHT" -eq 1 ]; then
|
||||
PREFLIGHT_FRAGMENT="$SCRIPT_DIR/hooks/settings.preflight.fragment.json"
|
||||
fi
|
||||
if [ "$ENABLE_SANHEDRIN" -eq 1 ]; then
|
||||
SANHEDRIN_FRAGMENT="$SCRIPT_DIR/hooks/settings.sanhedrin.fragment.json"
|
||||
fi
|
||||
jq -s '
|
||||
def is_vestige_hook:
|
||||
(.command? // "") as $cmd
|
||||
| [
|
||||
"synthesis-preflight.sh",
|
||||
"cwd-state-injector.sh",
|
||||
"vestige-pulse-daemon.sh",
|
||||
"preflight-swarm.sh",
|
||||
"load-all-memory.sh",
|
||||
"veto-detector.sh",
|
||||
"sanhedrin.sh",
|
||||
"synthesis-stop-validator.sh",
|
||||
"synthesis-gate.sh"
|
||||
] | any(. as $needle | $cmd | contains($needle));
|
||||
|
||||
def scrub_vestige_hooks:
|
||||
.hooks.UserPromptSubmit = (
|
||||
(.hooks.UserPromptSubmit // [])
|
||||
| map(.hooks = ((.hooks // []) | map(select((is_vestige_hook | not)))))
|
||||
| map(select(((.hooks // []) | length) > 0))
|
||||
)
|
||||
| if ((.hooks.UserPromptSubmit // []) | length) == 0 then del(.hooks.UserPromptSubmit) else . end
|
||||
| .hooks.Stop = (
|
||||
(.hooks.Stop // [])
|
||||
| map(.hooks = ((.hooks // []) | map(select((is_vestige_hook | not)))))
|
||||
| map(select(((.hooks // []) | length) > 0))
|
||||
)
|
||||
| if ((.hooks.Stop // []) | length) == 0 then del(.hooks.Stop) else . end
|
||||
| if ((.hooks // {}) | length) == 0 then del(.hooks) else . end;
|
||||
|
||||
(.[0] | scrub_vestige_hooks) * .[1] * .[2]
|
||||
' "$SETTINGS" "$PREFLIGHT_FRAGMENT" "$SANHEDRIN_FRAGMENT" > "$TMP_MERGE"
|
||||
mv "$TMP_MERGE" "$SETTINGS"
|
||||
say "merged hooks block into $SETTINGS (backup at .bak.pre-sandwich)"
|
||||
if [ "$ENABLE_PREFLIGHT" -eq 1 ] || [ "$ENABLE_SANHEDRIN" -eq 1 ]; then
|
||||
enabled_layers=""
|
||||
[ "$ENABLE_PREFLIGHT" -eq 1 ] && enabled_layers="${enabled_layers} preflight"
|
||||
[ "$ENABLE_SANHEDRIN" -eq 1 ] && enabled_layers="${enabled_layers} sanhedrin"
|
||||
say "merged optional hook layer(s) into $SETTINGS:${enabled_layers} (backup at .bak.pre-sandwich)"
|
||||
else
|
||||
say "removed Vestige hook wiring from $SETTINGS; default install activates no Claude Code hooks (backup at .bak.pre-sandwich)"
|
||||
fi
|
||||
|
||||
# --- Next steps ---
|
||||
cat <<EOF
|
||||
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ Cognitive Sandwich installed. │
|
||||
│ Cognitive Sandwich files installed. No hooks enabled by default. │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
|
||||
Next steps:
|
||||
1. Download the local model (~19 GB, one-time):
|
||||
hf download $MODEL_ID
|
||||
2. Restart Claude Code so it picks up the new hooks.
|
||||
3. Verify the install:
|
||||
1. Restart Claude Code if you enabled optional hooks.
|
||||
Default installs activate no Vestige Claude Code hooks and make no model calls.
|
||||
2. Verify the install:
|
||||
vestige health # if vestige CLI installed
|
||||
curl http://127.0.0.1:$DASHBOARD_PORT/api/health
|
||||
curl $MLX_MODELS_URL
|
||||
4. Try a prompt — the Sanhedrin Stop hook will fire and judge
|
||||
Claude's draft against your Vestige memory before delivery.
|
||||
scripts/check-sandwich-prereqs.sh # from a checkout
|
||||
3. Optional hook layers:
|
||||
./scripts/install-sandwich.sh --enable-preflight
|
||||
./scripts/install-sandwich.sh --enable-sanhedrin --sanhedrin-endpoint=$SANHEDRIN_ENDPOINT --sanhedrin-model=$MODEL_ID
|
||||
On Apple Silicon with >20 GB free RAM, add --with-launchd to auto-start
|
||||
the local MLX Qwen server. On x86, point --sanhedrin-endpoint at vLLM,
|
||||
Ollama, llama.cpp, or another OpenAI-compatible /v1/chat/completions URL.
|
||||
|
||||
To uninstall:
|
||||
launchctl unload $LAUNCHD_DIR/com.vestige.mlx-server.plist
|
||||
rm $LAUNCHD_DIR/com.vestige.mlx-server.plist
|
||||
launchctl unload $LAUNCHD_DIR/com.vestige.mlx-server.plist 2>/dev/null || true
|
||||
rm -f $LAUNCHD_DIR/com.vestige.mlx-server.plist
|
||||
cp $HOME/.claude/settings.json.bak.pre-sandwich $HOME/.claude/settings.json
|
||||
|
||||
EOF
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue