fix(consolidation): add VESTIGE_AUTO_CONSOLIDATE_MERGE opt-out + honor protected pins (#142)

The background consolidation cycle's auto_dedup_consolidation pass silently
concat-merged near-duplicate memories (cosine >= 0.85) into [MERGED] blocks and
hard-deleted the originals, with no opt-out and while ignoring `dedup protect`
pins — so a pinned memory could be absorbed/deleted unattended.

- Gate the pass behind VESTIGE_AUTO_CONSOLIDATE_MERGE (default on; set
  0/false/off/no to disable), parsed like the sibling VESTIGE_BACKFILL_AUTOFIRE,
  at the function top so every consolidation caller is covered consistently.
- Exclude protected (pinned) nodes from merge clusters at both loops, so a
  protected node is never an anchor nor a member — never absorbed, never deleted.
  protected_node_ids() is fetched once before the reader-lock window (lock-safe).
- Tests: default-unchanged regression, flag-off suppression, full protection
  matrix, liveness, and an explicit #142 regression guard.
- Docs: CONFIGURATION.md env-var row + CHANGELOG entry.

Default behavior is unchanged. TOCTOU: the protected set is snapshotted once per
pass (consistent with the function's existing point-in-time idiom). Out of scope
(follow-up): a reversible audit trail for merges that still happen when enabled,
and the [MERGED] concat-bloat itself.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Michael Kurras 2026-07-08 12:21:13 +02:00
parent 0e5c71b4ca
commit 5324c9690b
3 changed files with 431 additions and 3 deletions

View file

@ -46,6 +46,7 @@ Qwen3 currently uses Hugging Face Hub's Candle loader directly, so use the stand
| `VESTIGE_DASHBOARD_ENABLED` | `false` | Set `true` or `1` to enable the web dashboard |
| `VESTIGE_CONSOLIDATION_INTERVAL_HOURS` | `6` | FSRS-6 decay cycle cadence |
| `VESTIGE_BACKFILL_AUTOFIRE` | `on` | Retroactive Salience Backfill auto-fire during consolidation. On by default; set `0`/`false`/`off`/`no` to disable. The manual `backfill` tool + CLI stay available either way. When on, promotion is bounded (`stability = MIN(stability * 1.5, stability + 365)`) |
| `VESTIGE_AUTO_CONSOLIDATE_MERGE` | `on` | Auto concat-merge of near-duplicate memories during consolidation (keeps the strongest, folds the rest in as `[MERGED]` blocks, deletes the originals). On by default; set `0`/`false`/`off`/`no` to disable. Protected (`dedup protect`) memories are never absorbed or deleted by this pass, on or off. |
> **Storage location precedence:** `--data-dir <path>` wins over `VESTIGE_DATA_DIR`; if neither is set, Vestige uses your OS's per-user data directory: `~/Library/Application Support/com.vestige.core/` on macOS, `~/.local/share/vestige/core/` on Linux, `%APPDATA%\vestige\core\` on Windows. Custom paths are directories, are created if missing, expand a leading `~`, and store the database at `<dir>/vestige.db`.