apunkt/vestige
1
0
Fork 0
mirror of https://github.com/samvallad33/vestige.git synced 2026-07-26 23:51:02 +02:00
Code Issues Projects Releases Packages Wiki Activity

docs: correct the stale model, trial and transcript counts

Browse source 
EVIDENCE.md opened with "Five models. Twenty-two trials. Two hundred and ten
agent transcripts." while its own headline table used the six-model recount
(anarchy 21/25, rag 12/23, sync 0/23), its model list named six models, and
README.md said 25 trials across 6 models with 246 transcripts. A reader
checking the work hit the contradiction on the first page.

Counted from results/: 25 trial directories excluding WITHHELD-contaminated,
246 agent transcripts, and 6 distinct models read from the transcript model
field (deepseek-v4-flash, glm-5.2, gpt-5.6-sol, kimi-k2.7-code, minimax-m3,
kimi-k3).

Also corrects "invariant across five models" in section 5, whose own per-model
recount lists six and sums to the 65 first-call figure, and the 5-models title
in SHOW-HN.md.

No result, number or claim changes. Counts only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Sam Valladares 2026-07-25 16:49:45 +08:00
parent 752b44d835
commit f7606e65b1
2 changed files with 3 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

4
benchmarks/silent-rotation/EVIDENCE.md
View file

@ -3,7 +3,7 @@
Companion to `FINDING.md`. Each row states a claim and the file that proves it. All transcripts are in
`results/`. Nothing below is summarized from memory; every number was read out of the JSON.
Five models. Twenty-two trials. Two hundred and ten agent transcripts.
Six models. Twenty-five trials. Two hundred and forty-six agent transcripts.
---
@ -247,7 +247,7 @@ that is 15/15 GLM 5.2, 15/15 GPT-5.6 Sol, 14/14 DeepSeek V4 Flash, 12/12 Kimi K3
3/3 MiniMax M3. Zero empty results and zero errors.
So the honest cross-model statement is not "this works everywhere." It is: **the memory layer's
behaviour was invariant across five models and the baseline's was not.**
behaviour was invariant across six models and the baseline's was not.**
---

2
benchmarks/silent-rotation/SHOW-HN.md
View file

@ -2,7 +2,7 @@
## Titles, ranked
**1. Show HN: One misled agent voids a three-agent fleet (memory benchmark, 5 models)**
**1. Show HN: One misled agent voids a three-agent fleet (memory benchmark, 6 models)**
Leads with the finding, not the product. States a concrete falsifiable claim. 79 chars.
2. Show HN: I benchmarked 6 agent-memory systems and a plain cosine baseline beat most of them