From f7606e65b11aad0dcf6bda682236caa63eed9358 Mon Sep 17 00:00:00 2001 From: Sam Valladares Date: Sat, 25 Jul 2026 16:49:45 +0800 Subject: [PATCH] docs: correct the stale model, trial and transcript counts EVIDENCE.md opened with "Five models. Twenty-two trials. Two hundred and ten agent transcripts." while its own headline table used the six-model recount (anarchy 21/25, rag 12/23, sync 0/23), its model list named six models, and README.md said 25 trials across 6 models with 246 transcripts. A reader checking the work hit the contradiction on the first page. Counted from results/: 25 trial directories excluding WITHHELD-contaminated, 246 agent transcripts, and 6 distinct models read from the transcript model field (deepseek-v4-flash, glm-5.2, gpt-5.6-sol, kimi-k2.7-code, minimax-m3, kimi-k3). Also corrects "invariant across five models" in section 5, whose own per-model recount lists six and sums to the 65 first-call figure, and the 5-models title in SHOW-HN.md. No result, number or claim changes. Counts only. Co-Authored-By: Claude Opus 5 --- benchmarks/silent-rotation/EVIDENCE.md | 4 ++-- benchmarks/silent-rotation/SHOW-HN.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/benchmarks/silent-rotation/EVIDENCE.md b/benchmarks/silent-rotation/EVIDENCE.md index 3222ace..204aa97 100644 --- a/benchmarks/silent-rotation/EVIDENCE.md +++ b/benchmarks/silent-rotation/EVIDENCE.md @@ -3,7 +3,7 @@ Companion to `FINDING.md`. Each row states a claim and the file that proves it. All transcripts are in `results/`. Nothing below is summarized from memory; every number was read out of the JSON. -Five models. Twenty-two trials. Two hundred and ten agent transcripts. +Six models. Twenty-five trials. Two hundred and forty-six agent transcripts. --- @@ -247,7 +247,7 @@ that is 15/15 GLM 5.2, 15/15 GPT-5.6 Sol, 14/14 DeepSeek V4 Flash, 12/12 Kimi K3 3/3 MiniMax M3. Zero empty results and zero errors. So the honest cross-model statement is not "this works everywhere." It is: **the memory layer's -behaviour was invariant across five models and the baseline's was not.** +behaviour was invariant across six models and the baseline's was not.** --- diff --git a/benchmarks/silent-rotation/SHOW-HN.md b/benchmarks/silent-rotation/SHOW-HN.md index f0d59f5..ed99fd0 100644 --- a/benchmarks/silent-rotation/SHOW-HN.md +++ b/benchmarks/silent-rotation/SHOW-HN.md @@ -2,7 +2,7 @@ ## Titles, ranked -**1. Show HN: One misled agent voids a three-agent fleet (memory benchmark, 5 models)** +**1. Show HN: One misled agent voids a three-agent fleet (memory benchmark, 6 models)** Leads with the finding, not the product. States a concrete falsifiable claim. 79 chars. 2. Show HN: I benchmarked 6 agent-memory systems and a plain cosine baseline beat most of them