mirror of
https://github.com/samvallad33/vestige.git
synced 2026-07-04 22:02:14 +02:00
proof(blackbox): dream.patch proven live with a real dream run
Bounded follow-up (tight acceptance criteria, no scope expansion): flip the dream.patch producer from "quiet because no dream ran" to a recorded live event. The dream tool's `insights` array carries no per-item id, so the recorder extracted zero proposals and dream.patch never fired even on a real dream. Fix: derive a stable proposal id from each insight's REAL content (its insight_type + the source memories it consolidated). The dream genuinely ran; this just gives each real proposal a deterministic handle. Unit-tested against the exact dream output shape. Proven end to end (run_dream_proof, 6 memories consolidated): - one dream.patch event: dream:RecurringPattern:5d941c7f+a41aca72+... - SQLite + /api/traces/:runId: dream-trace.json (14 events, last is dream.patch) - WebSocket: dream-websocket-events.jsonl (the dream.patch TraceEvent) - dashboard: screenshots/dream-producers.png — the row flips to "fired this run" PROOF.md updated: dream.patch moves from CAVEAT to REAL (still not live by default — it fires only when a dream actually runs, and the UI says so). sanhedrin.veto remains an honest CAVEAT (optional hook, off by default). Gates: 957 lib tests pass, clippy -D warnings clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
b89beeeb63
commit
140b15f59f
6 changed files with 92 additions and 3 deletions
|
|
@ -52,13 +52,31 @@ deterministic regression test `test_full_spine_one_runid_crosses_every_hop`
|
||||||
| WebSocket broadcast | **REAL** | proven by `websocket-events.jsonl` + a unit test |
|
| WebSocket broadcast | **REAL** | proven by `websocket-events.jsonl` + a unit test |
|
||||||
| `vestige://trace/{runId}` resource | **REAL** | proven by the full-spine test |
|
| `vestige://trace/{runId}` resource | **REAL** | proven by the full-spine test |
|
||||||
| `sanhedrin.veto` trace | **CAVEAT** | extraction code is real + unit-tested, but the Sanhedrin verifier is an optional hook, **off by default** — no producer is connected, and the UI says exactly that |
|
| `sanhedrin.veto` trace | **CAVEAT** | extraction code is real + unit-tested, but the Sanhedrin verifier is an optional hook, **off by default** — no producer is connected, and the UI says exactly that |
|
||||||
| `dream.patch` trace | **CAVEAT** | extraction is real; fires only when a dream run actually executes — the UI says "No dream run in this trace" otherwise |
|
| `dream.patch` trace | **REAL** (proven 2026-06-22) | a real `dream` run over 6 memories produced one `dream.patch` event under `run_dream_proof` — see `dream-trace.json` (last event), `dream-websocket-events.jsonl`, and `screenshots/dream-producers.png` where the row flips to "fired this run". The UI still shows "No dream run in this trace" for runs where no dream executed. |
|
||||||
| Graph-pulse "Open receipt in Cinema" | **REAL (deep-link)** | navigates the graph centered on the receipt's primary memory; MemoryCinema itself is unchanged |
|
| Graph-pulse "Open receipt in Cinema" | **REAL (deep-link)** | navigates the graph centered on the receipt's primary memory; MemoryCinema itself is unchanged |
|
||||||
|
|
||||||
No feature is stubbed. The two CAVEATs are real plumbing whose upstream
|
No feature is stubbed. The two CAVEATs are real plumbing whose upstream
|
||||||
producer is intentionally off by default — surfaced as explicit UI states, not
|
producer is intentionally off by default — surfaced as explicit UI states, not
|
||||||
empty mystery.
|
empty mystery.
|
||||||
|
|
||||||
|
## dream.patch — proven with a real dream run (2026-06-22)
|
||||||
|
|
||||||
|
Bounded follow-up: a single real `dream` consolidation flipped the `dream.patch`
|
||||||
|
producer from "quiet" to a recorded live event, same runId, every hop.
|
||||||
|
|
||||||
|
- 6 related memories seeded under `run_dream_proof`, then one `dream` call.
|
||||||
|
- The dream produced one consolidation insight → one `dream.patch` event:
|
||||||
|
`dream:RecurringPattern:5d941c7f+a41aca72+b029fe53+6167f2c3+1117dd4e+e0782442`
|
||||||
|
(the real insight type + the six source memories it bridged).
|
||||||
|
- SQLite: `dream-trace.json` (14 events, last is `dream.patch`).
|
||||||
|
- API: `/api/traces/run_dream_proof/export` → `dream-trace.json`.
|
||||||
|
- WebSocket: `dream-websocket-events.jsonl` (the `dream.patch` TraceEvent).
|
||||||
|
- Dashboard: `screenshots/black-box-dream.png` + `screenshots/dream-producers.png`
|
||||||
|
(the producers row shows **dream.patch · fired this run**).
|
||||||
|
|
||||||
|
`dream.patch` is real but not live-by-default: it fires only when a dream
|
||||||
|
actually runs. The UI says so for runs where it didn't.
|
||||||
|
|
||||||
## Reproduce
|
## Reproduce
|
||||||
|
|
||||||
1. `VESTIGE_DATA_DIR=<tmp> VESTIGE_DASHBOARD_ENABLED=true vestige-mcp` (stdio).
|
1. `VESTIGE_DATA_DIR=<tmp> VESTIGE_DASHBOARD_ENABLED=true vestige-mcp` (stdio).
|
||||||
|
|
|
||||||
1
blackbox-proof-2026-06-22/dream-trace.json
Normal file
1
blackbox-proof-2026-06-22/dream-trace.json
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
{"events":[{"argsHash":"ec88652d34b9d5b6","at":1782168489775,"runId":"run_dream_proof","tool":"smart_ingest","type":"mcp.call"},{"at":1782168489889,"diff":{"decision":"create"},"id":"b029fe53-5f78-4c49-8100-62e6243a19ae","runId":"run_dream_proof","source":"agent","type":"memory.write"},{"argsHash":"10134f9d053e1139","at":1782168490987,"runId":"run_dream_proof","tool":"smart_ingest","type":"mcp.call"},{"at":1782168491091,"diff":{"decision":"create"},"id":"6167f2c3-c567-4ec4-b63d-d0b6660173fc","runId":"run_dream_proof","source":"agent","type":"memory.write"},{"argsHash":"a687fb99ed887e10","at":1782168492200,"runId":"run_dream_proof","tool":"smart_ingest","type":"mcp.call"},{"at":1782168492297,"diff":{"decision":"create"},"id":"e0782442-0ce0-4a54-94db-4814ae392bbd","runId":"run_dream_proof","source":"agent","type":"memory.write"},{"argsHash":"a672de9e54d138f9","at":1782168493413,"runId":"run_dream_proof","tool":"smart_ingest","type":"mcp.call"},{"at":1782168493496,"diff":{"decision":"create"},"id":"a41aca72-7758-4f07-8da0-2e16469efc81","runId":"run_dream_proof","source":"agent","type":"memory.write"},{"argsHash":"d5878d7adad2cfe1","at":1782168494628,"runId":"run_dream_proof","tool":"smart_ingest","type":"mcp.call"},{"at":1782168494724,"diff":{"decision":"create"},"id":"5d941c7f-c75c-4c89-acc1-af1b95d3a4de","runId":"run_dream_proof","source":"agent","type":"memory.write"},{"argsHash":"14c334181d37393d","at":1782168495841,"runId":"run_dream_proof","tool":"smart_ingest","type":"mcp.call"},{"at":1782168495922,"diff":{"decision":"create"},"id":"1117dd4e-11f7-434d-b0b7-2b5bcbd841d4","runId":"run_dream_proof","source":"agent","type":"memory.write"},{"argsHash":"1071da7bf3583db3","at":1782168511374,"runId":"run_dream_proof","tool":"dream","type":"mcp.call"},{"at":1782168511375,"proposalIds":["dream:RecurringPattern:5d941c7f+a41aca72+b029fe53+6167f2c3+1117dd4e+e0782442"],"runId":"run_dream_proof","type":"dream.patch"}],"exportedAt":"2026-06-22T22:50:09.588334+00:00","format":"vestige-trace","runId":"run_dream_proof","summary":{"eventCount":14,"firstTool":"smart_ingest","lastAt":1782168511375,"retrievedCount":0,"startedAt":1782168489775,"suppressedCount":0,"vetoCount":0,"writeCount":6},"version":1}
|
||||||
5
blackbox-proof-2026-06-22/dream-websocket-events.jsonl
Normal file
5
blackbox-proof-2026-06-22/dream-websocket-events.jsonl
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
{"data": {"timestamp": "2026-06-22T22:48:29.454773+00:00", "version": "2.1.27"}, "type": "Connected"}
|
||||||
|
{"type": "TraceEvent", "data": {"run_id": "run_dream_proof", "seq": 12, "event": {"type": "mcp.call", "runId": "run_dream_proof", "tool": "dream", "argsHash": "1071da7bf3583db3", "at": 1782168511374}, "timestamp": "2026-06-22T22:48:31.374680Z"}}
|
||||||
|
{"type": "DreamStarted", "data": {"memory_count": 6, "timestamp": "2026-06-22T22:48:31.374852Z"}}
|
||||||
|
{"type": "DreamCompleted", "data": {"memories_replayed": 6, "connections_found": 0, "insights_generated": 1, "duration_ms": 0, "timestamp": "2026-06-22T22:48:31.375855Z"}}
|
||||||
|
{"type": "TraceEvent", "data": {"run_id": "run_dream_proof", "seq": 13, "event": {"type": "dream.patch", "runId": "run_dream_proof", "proposalIds": ["dream:RecurringPattern:5d941c7f+a41aca72+b029fe53+6167f2c3+1117dd4e+e0782442"], "at": 1782168511375}, "timestamp": "2026-06-22T22:48:31.375973Z"}}
|
||||||
BIN
blackbox-proof-2026-06-22/screenshots/black-box-dream.png
Normal file
BIN
blackbox-proof-2026-06-22/screenshots/black-box-dream.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 556 KiB |
BIN
blackbox-proof-2026-06-22/screenshots/dream-producers.png
Normal file
BIN
blackbox-proof-2026-06-22/screenshots/dream-producers.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 58 KiB |
|
|
@ -608,13 +608,22 @@ fn extract_veto(result: &Value) -> Option<(String, Vec<String>, f64)> {
|
||||||
Some((claim, evidence_ids, confidence))
|
Some((claim, evidence_ids, confidence))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Pull dream consolidation proposal ids from a dream tool result.
|
/// Pull dream consolidation proposal ids from a dream/consolidate tool result.
|
||||||
|
///
|
||||||
|
/// Proposals are identified by an explicit `id` / proposal id when present.
|
||||||
|
/// The `dream` tool emits an `insights` array whose items carry no id (they are
|
||||||
|
/// `{insight_type, insight, source_memories, confidence, …}`), so we derive a
|
||||||
|
/// stable proposal id from each insight's real content — its type plus the
|
||||||
|
/// memories it consolidated. The dream genuinely ran; this just gives each real
|
||||||
|
/// proposal a deterministic handle for the trace.
|
||||||
fn extract_dream_proposals(result: &Value, tool: &str) -> Vec<String> {
|
fn extract_dream_proposals(result: &Value, tool: &str) -> Vec<String> {
|
||||||
if tool != "dream" && tool != "consolidate" {
|
if tool != "dream" && tool != "consolidate" {
|
||||||
return Vec::new();
|
return Vec::new();
|
||||||
}
|
}
|
||||||
let mut out = Vec::new();
|
let mut out = Vec::new();
|
||||||
for key in ["proposalIds", "proposals", "insights", "connections"] {
|
|
||||||
|
// Explicit id arrays first (consolidate / future producers).
|
||||||
|
for key in ["proposalIds", "proposals", "connections"] {
|
||||||
if let Some(arr) = result.get(key).and_then(|v| v.as_array()) {
|
if let Some(arr) = result.get(key).and_then(|v| v.as_array()) {
|
||||||
for item in arr {
|
for item in arr {
|
||||||
if let Some(id) = item
|
if let Some(id) = item
|
||||||
|
|
@ -627,6 +636,36 @@ fn extract_dream_proposals(result: &Value, tool: &str) -> Vec<String> {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Dream insights: derive a stable id from real content.
|
||||||
|
if let Some(arr) = result.get("insights").and_then(|v| v.as_array()) {
|
||||||
|
for (i, item) in arr.iter().enumerate() {
|
||||||
|
if let Some(id) = item.get("id").and_then(|v| v.as_str()) {
|
||||||
|
out.push(id.to_string());
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let kind = item
|
||||||
|
.get("insight_type")
|
||||||
|
.and_then(|v| v.as_str())
|
||||||
|
.unwrap_or("insight");
|
||||||
|
// Prefer the consolidated source memories for a meaningful handle;
|
||||||
|
// fall back to the index so every real insight is still counted.
|
||||||
|
let src = item
|
||||||
|
.get("source_memories")
|
||||||
|
.and_then(|v| v.as_array())
|
||||||
|
.map(|a| {
|
||||||
|
a.iter()
|
||||||
|
.filter_map(|m| m.as_str())
|
||||||
|
.map(|s| &s[..s.len().min(8)])
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join("+")
|
||||||
|
})
|
||||||
|
.filter(|s| !s.is_empty())
|
||||||
|
.unwrap_or_else(|| format!("idx{i}"));
|
||||||
|
out.push(format!("dream:{kind}:{src}"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
out
|
out
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -692,6 +731,32 @@ mod tests {
|
||||||
assert!(out.contains(&("s2".to_string(), SuppressReason::Contradicted)));
|
assert!(out.contains(&("s2".to_string(), SuppressReason::Contradicted)));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extract_dream_proposals_from_real_insights_shape() {
|
||||||
|
// The exact shape the `dream` tool emits — insights without an id.
|
||||||
|
let r = serde_json::json!({
|
||||||
|
"status": "dreamed",
|
||||||
|
"insights": [
|
||||||
|
{
|
||||||
|
"insight_type": "Bridge",
|
||||||
|
"insight": "These two notes describe the same subsystem.",
|
||||||
|
"source_memories": ["aaaaaaaa1111", "bbbbbbbb2222"],
|
||||||
|
"confidence": 0.8,
|
||||||
|
"novelty_score": 0.6
|
||||||
|
}
|
||||||
|
]
|
||||||
|
});
|
||||||
|
let ids = extract_dream_proposals(&r, "dream");
|
||||||
|
assert_eq!(ids.len(), 1, "one real insight -> one proposal id");
|
||||||
|
assert_eq!(ids[0], "dream:Bridge:aaaaaaaa+bbbbbbbb");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extract_dream_proposals_empty_when_not_dream_tool() {
|
||||||
|
let r = serde_json::json!({ "insights": [{ "insight_type": "x" }] });
|
||||||
|
assert!(extract_dream_proposals(&r, "search").is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn extract_writes_single_and_batch() {
|
fn extract_writes_single_and_batch() {
|
||||||
let single = serde_json::json!({ "decision": "create", "nodeId": "n1" });
|
let single = serde_json::json!({ "decision": "create", "nodeId": "n1" });
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue