bench(engine): append every run to a results log (ts + git_sha)

The harness's JSON lines were vanishing into terminal scrollback — the
numbers that justify decisions (the 6.9GB merge crash, the prefilter
recall pair) had no system of record, and the v7 baselines are needed
for before/after comparison across the Lance 9 bump. Every run now
appends its record, stamped with a unix ts and the current git sha, to
--out <path> / OMNIGRAPH_BENCH_RESULTS / benches/results.jsonl
(gitignored — host-specific, self-describing via host+params).
This commit is contained in:
aaltshuler 2026-07-05 18:34:06 +03:00 committed by Andrew Altshuler
parent 54dbf2f11c
commit 80db05f16e
3 changed files with 59 additions and 2 deletions

View file

@ -114,8 +114,12 @@ The CLI system tests (`system_local.rs`) spawn the workspace-built `omnigraph` a
merge-all-changed --rows 20000 --dims 256` (also `nearest-prefilter`;
`--baseline` re-runs the identical workload minus the measured op so the
peak-RSS delta isolates it; `--memory-cap-mb` applies `RLIMIT_AS` — enforced
on Linux, best-effort on macOS). Add new scenarios here rather than new
bench targets; keep the JSON-lines/no-assertions contract.
on Linux, best-effort on macOS). Every run appends its record (with `ts` +
`git_sha`) to a results log — `--out <path>`, else `OMNIGRAPH_BENCH_RESULTS`,
else `crates/omnigraph/benches/results.jsonl` (gitignored; host-specific) —
so baselines survive across sessions and substrate bumps. Add new scenarios
here rather than new bench targets; keep the JSON-lines/no-assertions
contract.
- Add `benches/` per crate when you ship a perf-driven change, and include the motivating workload with the optimization.
## Coverage tooling — what's missing