Sam Valladares 9bdb9a5668 benchmarks: add Silent Rotation, a fleet-convergence memory benchmark ... Three agents fix one failing test concurrently. The fix needs a signing key id that is randomized per trial from a 50-key keyring and appears in no file the agents can read. It exists only in the memory layer. Seven arms: a no-memory control, dense cosine RAG, Vestige, mem0, supermemory, hindsight, Zep/Graphiti. The measured result is about the first retrieval, not the final score. Across 5 models and 22 trials, a query-based arm's FIRST memory call surfaced the correct key 6 times out of 87. Vestige, which takes no query argument, did it 56 out of 56. Every downstream number follows from that. Ships the raw evidence rather than a summary: all agent transcripts, the harness, and tests/bm25_baseline.py, which reproduces the ranking table offline with no API key. Two arms were contaminated by a flush bug in this harness that served them a previous trial's facts; both were fixed and re-run, and the contaminated cells are withheld rather than published. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>		2026-07-21 21:24:03 +07:00
..
silent-rotation	benchmarks: add Silent Rotation, a fleet-convergence memory benchmark	2026-07-21 21:24:03 +07:00