apunkt/vestige
1
0
Fork 0
mirror of https://github.com/samvallad33/vestige.git synced 2026-07-24 23:41:01 +02:00
Code Issues Projects Releases Packages Wiki Activity

Remove the last two CauseBench remnants

Browse source 
docs/ROADMAP.md pointed at benchmarks/causebench/ as a harness pattern to follow.
That directory no longer exists; the pattern to follow is benchmarks/silent-rotation/.

assets/causebench-recall-at-1.svg charted the withdrawn recall@1 numbers. Nothing
references it any more, so it was a public image of a result that is no longer
claimed. Removed.
This commit is contained in:
Sam Valladares 2026-07-23 00:10:29 +08:00
parent e0749a0f25
commit 469b4e6f8b
1 changed files with 0 additions and 46 deletions
Download patch file Download diff file Expand all files Collapse all files

46
assets/causebench-recall-at-1.svg
View file

@ -1,46 +0,0 @@
<svg viewBox="0 0 680 430" xmlns="http://www.w3.org/2000/svg" width="680" height="430" role="img" aria-label="CauseBench root-cause recall at 1: Vestige scores 60% on synthetic and 50% on real; Dense vector, Hybrid-RRF, BM25, TF-IDF and Recency all score 0%." font-family="-apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif">
<title>CauseBench root-cause recall@1 — Vestige 60% / 50%, every resemblance retriever 0%</title>
<text x="24" y="34" font-size="17" font-weight="600" fill="#1d9e75">Root-cause recall@1 — can the retriever rank the true cause #1?</text>
<text x="24" y="56" font-size="13" fill="#888780">CauseBench · a cause that shares an entity with the failure but none of its words</text>
<line x1="330" y1="78" x2="330" y2="308" stroke="#88878055" stroke-width="1"/>
<line x1="450" y1="78" x2="450" y2="308" stroke="#88878055" stroke-width="1"/>
<line x1="570" y1="78" x2="570" y2="308" stroke="#88878055" stroke-width="1"/>
<line x1="210" y1="78" x2="210" y2="308" stroke="#888780" stroke-width="1.5"/>
<text x="210" y="330" font-size="11" fill="#888780" text-anchor="middle">0%</text>
<text x="330" y="330" font-size="11" fill="#888780" text-anchor="middle">20%</text>
<text x="450" y="330" font-size="11" fill="#888780" text-anchor="middle">40%</text>
<text x="570" y="330" font-size="11" fill="#888780" text-anchor="middle">60%</text>
<text x="196" y="99" font-size="13" font-weight="600" fill="#0f6e56" text-anchor="end">Vestige</text>
<rect x="210" y="88" width="216" height="17" rx="4" fill="#1d9e75"/>
<rect x="210" y="109" width="180" height="17" rx="4" fill="#5dcaa5"/>
<text x="434" y="101" font-size="12" font-weight="600" fill="#0f6e56">60% synthetic</text>
<text x="398" y="122" font-size="12" fill="#0f6e56">50% real</text>
<text x="196" y="158" font-size="13" fill="#888780" text-anchor="end">Dense vector (nomic)</text>
<circle cx="212" cy="153" r="2.5" fill="#888780"/>
<text x="224" y="157" font-size="12" fill="#888780">0%</text>
<text x="196" y="191" font-size="13" fill="#888780" text-anchor="end">Hybrid-RRF (BM25 + dense)</text>
<circle cx="212" cy="186" r="2.5" fill="#888780"/>
<text x="224" y="190" font-size="12" fill="#888780">0%</text>
<text x="196" y="224" font-size="13" fill="#888780" text-anchor="end">BM25</text>
<circle cx="212" cy="219" r="2.5" fill="#888780"/>
<text x="224" y="223" font-size="12" fill="#888780">0%</text>
<text x="196" y="257" font-size="13" fill="#888780" text-anchor="end">TF-IDF</text>
<circle cx="212" cy="252" r="2.5" fill="#888780"/>
<text x="224" y="256" font-size="12" fill="#888780">0%</text>
<text x="196" y="290" font-size="13" fill="#888780" text-anchor="end">Recency</text>
<circle cx="212" cy="285" r="2.5" fill="#888780"/>
<text x="224" y="289" font-size="12" fill="#888780">0%</text>
<line x1="24" y1="356" x2="656" y2="356" stroke="#88878040" stroke-width="1"/>
<text x="24" y="380" font-size="12" fill="#888780">Every resemblance retriever ranks the true cause #1 exactly 0% of the time.</text>
<text x="24" y="400" font-size="11" fill="#888780">Vestige also lands the cause in the top 5 in 100% of cases. Source: benchmarks/causebench/RESULTS.md · reproduce with 3 commands.</text>
</svg>
Side by side

Before

Width:  |  Height:  |  Size: 3.2 KiB