apunkt/vestige
1
0
Fork 0
mirror of https://github.com/samvallad33/vestige.git synced 2026-07-26 23:51:02 +02:00
Code Issues Projects Releases Packages Wiki Activity
vestige/benchmarks/silent-rotation/results/trial-2/transcript-sync-a2.json
Sam Valladares 9bdb9a5668 benchmarks: add Silent Rotation, a fleet-convergence memory benchmark 
Three agents fix one failing test concurrently. The fix needs a signing key id
that is randomized per trial from a 50-key keyring and appears in no file the
agents can read. It exists only in the memory layer. Seven arms: a no-memory
control, dense cosine RAG, Vestige, mem0, supermemory, hindsight, Zep/Graphiti.

The measured result is about the first retrieval, not the final score. Across
5 models and 22 trials, a query-based arm's FIRST memory call surfaced the
correct key 6 times out of 87. Vestige, which takes no query argument, did it
56 out of 56. Every downstream number follows from that.

Ships the raw evidence rather than a summary: all agent transcripts, the
harness, and tests/bm25_baseline.py, which reproduces the ranking table offline
with no API key. Two arms were contaminated by a flush bug in this harness that
served them a previous trial's facts; both were fixed and re-run, and the
contaminated cells are withheld rather than published.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 21:24:03 +07:00

571 lines
No EOL
46 KiB
JSON
Raw Blame History

{
"agent_id": "a2",
"mode": "sync",
"provider": "moonshot",
"model": "kimi-k3",
"task": "make the failing end-to-end charge test pass",
"status": "fixed",
"iterations": 8,
"tokens": {
"input": 48427,
"output": 1900,
"total": 50327
},
"used_vestige_backfill": true,
"used_vestige_log": true,
"used_rag_search": false,
"memory_tool_used": "vestige_backfill",
"retrieval_ok_count": 1,
"retrieval_err_count": 0,
"retrieval_empty_count": 0,
"retrieval_errors": [],
"memory_layer_alive": true,
"final_contested_signatures": {
"identity-service/src/config.ts": [
"kid",
"k_regulus"
],
"ledger-service/src/config.ts": [
"kid",
"k_regulus"
]
},
"turns": [
{
"turn": 1,
"tokens": {
"input": 920,
"output": 89
},
"reasoning": "Let me start by reproducing the failure and understanding the repo structure. I'll run the tests and list files in parallel.",
"raw_model_output": [
{
"role": "user",
"content": "The repo is at /var/folders/tg/9hll24px5v17ys2sl78r1vb80000gn/T/fleet-sync-6o00yubr/agent-2. Task: make the failing end-to-end charge test pass."
},
{
"role": "assistant",
"content": "I'll start by reproducing the failure and exploring the repo structure.",
"tool_calls": [
{
"id": "run_tests_0",
"type": "function",
"function": {
"name": "run_tests",
"arguments": "{}"
}
},
{
"id": "list_files_1",
"type": "function",
"function": {
"name": "list_files",
"arguments": "{}"
}
}
],
"reasoning": "Let me start by reproducing the failure and understanding the repo structure. I'll run the tests and list files in parallel."
}
],
"tool_calls": [
{
"name": "run_tests",
"input": {}
},
{
"name": "list_files",
"input": {}
}
],
"tool_results": [
{
"name": "run_tests",
"input": {},
"output": "exit_code=1\n\n> meridian-platform@1.0.0 test\n> npm run test --workspace @meridian/ledger-service\n\n\n> @meridian/ledger-service@1.0.0 test\n> tsx --test test/*.test.ts\n\n\u2716 a freshly-minted token charges successfully (expects 200) (2.41425ms)\n\u2139 tests 1\n\u2139 suites 0\n\u2139 pass 0\n\u2139 fail 1\n\u2139 cancelled 0\n\u2139 skipped 0\n\u2139 todo 0\n\u2139 duration_ms 184.931417\n\n\u2716 failing tests:\n\ntest at test/charge.test.ts:1:221\n\u2716 a freshly-minted token charges successfully (expects 200) (2.41425ms)\n Error: no active signing key selected\n at signSession (/private/var/folders/tg/9hll24px5v17ys2sl78r1vb80000gn/T/fleet-sync-6o00yubr/agent-2/identity-service/src/signer.ts:31:11)\n at TestContext.<anonymous> (/private/var/folders/tg/9hll24px5v17ys2sl78r1vb80000gn/T/fleet-sync-6o00yubr/agent-2/ledger-service/test/charge.test.ts:24:19)\n at process.processTicksAndRejections (node:internal/process/task_queues:103:5)\n at async Test.run (node:internal/test_runner/test:1113:7)\n at async startSubtestAfterBootstrap (node:internal/test_runner/harness:358:3)\n\nnpm error Lifecycle script `test` failed with error:\nnpm error code 1\nnpm error path /private/var/folders/tg/9hll24px5v17ys2sl78r1vb80000gn/T/fleet-sync-6o00yubr/agent-2/ledger-service\nnpm error workspace @meridian/ledger-service@1.0.0\nnpm error location /private/var/folders/tg/9hll24px5v17ys2sl78r1vb80000gn/T/fleet-sync-6o00yubr/agent-2/ledger-service\nnpm error command failed\nnpm error command sh -c tsx --test test/*.test.ts\n"
},
{
"name": "list_files",
"input": {},
"output": ".gitignore\nREADME.md\ngateway-service/package.json\ngateway-service/src/index.ts\nidentity-service/package.json\nidentity-service/src/canonical.ts\nidentity-service/src/config.ts\nidentity-service/src/index.ts\nidentity-service/src/signer.ts\nledger-service/package.json\nledger-service/src/canonical.ts\nledger-service/src/config.ts\nledger-service/src/server.ts\nledger-service/src/verify.ts\nledger-service/test/charge.test.ts\npackage-lock.json\npackage.json\nshared/package.json\nshared/src/index.ts\nshared/src/keyring.ts\ntsconfig.json"
}
]
},
{
"turn": 2,
"tokens": {
"input": 1643,
"output": 221
},
"reasoning": "The error is \"no active signing key selected\" in identity-service/src/signer.ts:31. Let me look at the relevant files: the test, the signer, the keyring, and shared files. Let me read them all in parallel. Also, per the task instructions, I should consult shared memory via vestige_backfill to find the quiet earlier change that caused this.",
"raw_model_output": [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "read_file_2",
"type": "function",
"function": {
"name": "read_file",
"arguments": "{\"path\":\"ledger-service/test/charge.test.ts\"}"
}
},
{
"id": "read_file_3",
"type": "function",
"function": {
"name": "read_file",
"arguments": "{\"path\":\"identity-service/src/signer.ts\"}"
}
},
{
"id": "read_file_4",
"type": "function",
"function": {
"name": "read_file",
"arguments": "{\"path\":\"shared/src/keyring.ts\"}"
}
},
{
"id": "vestige_backfill_5",
"type": "function",
"function": {
"name": "vestige_backfill",
"arguments": "{}"
}
}
],
"reasoning": "The error is \"no active signing key selected\" in identity-service/src/signer.ts:31. Let me look at the relevant files: the test, the signer, the keyring, and shared files. Let me read them all in parallel. Also, per the task instructions, I should consult shared memory via vestige_backfill to find the quiet earlier change that caused this."
}
],
"tool_calls": [
{
"name": "read_file",
"input": {
"path": "ledger-service/test/charge.test.ts"
}
},
{
"name": "read_file",
"input": {
"path": "identity-service/src/signer.ts"
}
},
{
"name": "read_file",
"input": {
"path": "shared/src/keyring.ts"
}
},
{
"name": "vestige_backfill",
"input": {}
}
],
"tool_results": [
{
"name": "read_file",
"input": {
"path": "ledger-service/test/charge.test.ts"
},
"output": "import { test } from \"node:test\";\nimport assert from \"node:assert/strict\";\nimport { AddressInfo } from \"net\";\nimport { signSession } from \"@meridian/identity-service\";\nimport { forwardToLedger } from \"@meridian/gateway-service\";\nimport { createLedgerServer } from \"../src/server\";\n\n/**\n * End-to-end charge path:\n *\n * identity-service.signSession() -> mint a session token\n * gateway-service.forwardToLedger() -> package it into a ChargeRequest\n * ledger-service POST /charge -> should verify + charge, returning 200\n *\n * This is the real production flow, wired with the real modules. It asserts a\n * successful 200 charge.\n */\ntest(\"a freshly-minted token charges successfully (expects 200)\", async () => {\n const server = createLedgerServer();\n await new Promise<void>((resolve) => server.listen(0, resolve));\n const { port } = server.address() as AddressInfo;\n\n try {\n const token = signSession(\"user_42\", \"pro\");\n const charge = forwardToLedger(`Bearer ${token}`, 1999, \"usd\");\n\n const resp = await fetch(`http://127.0.0.1:${port}/charge`, {\n method: \"POST\",\n headers: { \"content-type\": \"application/json\" },\n body: JSON.stringify(charge),\n });\n\n const payload = await resp.json();\n\n assert.equal(\n resp.status,\n 200,\n `expected a 200 charge but got ${resp.status}: ${JSON.stringify(payload)}`\n );\n assert.equal(payload.ok, true);\n assert.equal(payload.chargedTo, \"user_42\");\n assert.equal(payload.amountCents, 1999);\n } finally {\n await new Promise<void>((resolve) => server.close(() => resolve()));\n }\n});\n"
},
{
"name": "read_file",
"input": {
"path": "identity-service/src/signer.ts"
},
"output": "import { createHmac } from \"crypto\";\nimport { SIGNING_ALG, SessionClaims, keyForId } from \"@meridian/shared\";\nimport { config } from \"./config\";\nimport { canonicalClaimString } from \"./canonical\";\n\n/** base64url (JWS-style, no padding). */\nfunction b64url(input: Buffer | string): string {\n return Buffer.from(input)\n .toString(\"base64\")\n .replace(/\\+/g, \"-\")\n .replace(/\\//g, \"_\")\n .replace(/=+$/, \"\");\n}\n\n/**\n * Mint a signed session token for a user.\n *\n * Token layout: `header.payload.signature`.\n * - header : { alg, typ, iss } (NB: no key id is stamped into the header)\n * - payload : the JSON claims (informational; transport may re-encode it)\n * - signature: HMAC-SHA256 over the CANONICAL claim digest, using the key the\n * issuer's config selects from the shared keyring.\n *\n * The signature deliberately does NOT cover the raw payload bytes \u2014 it covers\n * the canonical digest \u2014 so the payload can be re-serialized downstream without\n * breaking verification, as long as the verifier recomputes the same canonical\n * digest AND uses the same signing key.\n */\nexport function signSession(sub: string, plan: SessionClaims[\"plan\"]): string {\n if (!config.activeKid) {\n throw new Error(\"no active signing key selected\");\n }\n const signingKey = keyForId(config.activeKid);\n\n const now = Math.floor(Date.now() / 1000);\n const claims: SessionClaims = {\n sub,\n plan,\n iat: now,\n exp: now + config.tokenTtlSeconds,\n };\n\n const header = { alg: SIGNING_ALG, typ: \"JWT\", iss: config.issuer };\n const headerB64 = b64url(JSON.stringify(header));\n const payloadB64 = b64url(JSON.stringify(claims));\n\n const signature = b64url(\n createHmac(\"sha256\", signingKey)\n .update(canonicalClaimString(claims))\n .digest()\n );\n\n return `${headerB64}.${payloadB64}.${signature}`;\n}\n"
},
{
"name": "read_file",
"input": {
"path": "shared/src/keyring.ts"
},
"output": "/**\n * @meridian/shared \u2014 the platform signing keyring.\n *\n * The keyring holds the HMAC signing keys the platform can use to sign and\n * verify session tokens. Keys are referenced by a short key id (`kid`). Each\n * service selects which key it uses from its own runtime config; the keyring\n * itself is just the material.\n *\n * Keys are 32-byte values, hex-encoded. (Dev material, committed for the local\n * harness; injected from the KMS in production.)\n */\nexport const KEYRING: Record<string, string> = {\n // Each entry is a provisioned HMAC key. Presence in the keyring does not imply\n // a service currently selects it; selection is per-service runtime config.\n k_helios: \"b636e0849d3bcf7e5afc6bbd3a3045000ff46adfc0978236924d9085a205ad6f\",\n k_vela: \"8538190891d5627f2142e49271100ea5811c6daaa15c187fdafd59d8594f1f14\",\n k_vega: \"b897f2c8c8b79e13ea49b45aa5b51f0994bf418d6ae3ed44b1b7e131e349e52c\",\n k_perseus: \"a4d5bb9c1a67a5e5bbbd74ed059498f08584c4547089855c08f2335e9c751fd4\",\n k_aludra: \"754bcfacd9cea5368974ba14751bf8f0dc694b7575de0c9c7f20840210d1d389\",\n k_nashira: \"6e32cbae1b7605d8bcf7cb2dfffc7820934b90729642f5e28321bab96cec485e\",\n k_basalt: \"901789d8729a929ff893673ab9afdee9766eb2b591610c5c6d9ab7910abfcbd0\",\n k_olivine: \"ee4d405b825b767a63f0eda0e79c8f7764ac1150bc0fbb2c2601821495907f5e\",\n k_talc: \"c9e9c21a3a2c393f66ccd4fda0f2fa4bc117fda6948e648abca02199a51afa99\",\n k_hydra: \"29f933a5dcd40c98964d4b06f7f3572aa51b0f243f705bafd4aad7fc93f98068\",\n k_procyon: \"d6bd05f8f3e1add2083a307366df8af1d33068c9bca6b94417566b369b290e29\",\n k_regulus: \"e6af7ed64900d4a5ff3a71871646ae1cffeb1d2257f36186dfead6e2ab287690\",\n k_mensa: \"9ec8d7beb65d3653dae21698e5e625bf40eed9bcff59652679288d185c258374\",\n k_gemini: \"fcce609c20e26be44c9877b42b744aca270b2a00bca78fe39d9bb6657a323d9d\",\n k_rutile: \"6c26c1fd87432f358fd9052a46ca42b0ff8e62d732e24f07551ca15ad2d9a520\",\n k_sirius: \"b3d1e52ac66f321c6cbba4aa60b912d32abb47ee67a18bb20e9fbc392a1c942e\",\n k_wezen: \"fd537049c63221d4618ec4c5c00970a43cf0d8bedb29b5e9e23df7005b785abe\",\n k_arcturus: \"021f0afccb43cc6e6e395f9a02f7676e046bf1d09678d6a260746103d6219edb\",\n k_norma: \"70df20095d7193ecfc4ffb9f7475e59f4cb473b32933388bcd75d56ed80308e0\",\n k_borealis: \"7dcc223dc61d615241e068d80cd0ec4e276fc28a3477492b3101b0d5bda5f880\",\n k_kepler: \"48f2dba2c1d058d6435c21adab8e63bdc5d06d35866c0cf222a71f596185f72d\",\n k_rigel: \"df289864ebf47da06bc34febabe92156f84d8c662e7e88a47a7e4d413eead1b0\",\n k_indus: \"18d980bad22ffc15b7ea575d93c7345a0bdb747e9d33d48ab1ba8845e33b83f5\",\n k_atlas: \"b3868be1045cc6e1398c42e8d34c8e45f006bfdd40d492405522cc9662b588c8\",\n k_capella: \"eea9077f2f694d6749820eeadbef2a45d6973ae6724986f18ff52222b369a97a\",\n k_polaris: \"334568301a5ea1bd92620e44d58c9da8e2bc858bae7d46e338f1eab21abf7c39\",\n k_pavo: \"59ab5917167faa32c0cf98367c60223764e20dde0207770326bf6f2badee42a1\",\n k_umbriel: \"b14383b59fe2a2598e594c0bdc47d67cb724ee561219aa9f859366452e54df6c\",\n k_castor: \"e20127fd70126e5e1c877a9108c0b865aefa7ac9d744feb5d27a458561070f8f\",\n k_pyrite: \"3d4618afb20e0d5b966fda3a850e092cca15f55902ec0581787dbc9a79961ea2\",\n k_bellatrix: \"96ccfa3a1eb3f47827a2cf2eec7be00d7fcacf503cd911b4176cdd2117aff38c\",\n k_spica: \"b6b5f8a8b6fe783f58a7cc8ae195fbcd0415fe4ddf7f62f6a5b2f0f7edee6614\",\n k_schist: \"972949d649a411a9f085aa8a431dae492c9ac9a512f18acc64e34251010785c3\",\n k_feldspar: \"44235980314f42298aefde95b246ed09dbddff9df7d0d06bef6b024a9ba67640\",\n k_garnet: \"0e47b9d7da6dbb30c1fc4a6015619fc5b310a7510b7d4ae647e6f499909ce809\",\n k_altair: \"a854fc394f1c9a8fb057bbdab6d5854ec6429f5041a5ef23c21cf199926396b4\",\n k_izar: \"1b7330863b977a43ed50446c34484d4676c41587d4837bd9b22c002f57d51f06\",\n k_octans: \"c1c8a041674d98e978ca10957d0f26c00fa61848af77519c3f74e22adce927f3\",\n k_zephyr: \"6233a240bbd156ea9613d133a2cbf5faa388d0f96810a380d5eeaa6c8adedfc1\",\n k_juno: \"4dd63abef018832a9b0af70649f07415648d321bec2f5010254ccc890dc276ae\",\n k_kyanite: \"a82f0c2731a506ac1a2fe66a36e582fb59748ebcf34a3ddfe86ef55465c11892\",\n k_cobalt: \"69984fe8a08769a6686e5fc3535dcef165c29a0bbd598b4c64229142c1b35047\",\n k_mizar: \"4d235a932074c08d9d240b28936ad0bc32dc1430b05c7c2236680e5814f32d1e\",\n k_yildun: \"f4fd0dfc1f1a72f6d9d0ac65201d97e603d00a6ce5988192216db3e2a65fc04c\",\n k_lyra: \"51e4b7616765ddbe061d325dfe4a15d21d05a0b603e98c19fc4c9d25904fd13b\",\n k_deneb: \"b05c70f07e6607d24a136d1c01129e0cca73df1f76cfd4ec89ab5d0a3626c385\",\n k_draco: \"da6b04177e6bd3b0855b1237d324edf1c686cc832902ed349ae51ccdd734c7a2\",\n k_jasper: \"09af91b0f8e31c74423b3e011a45d4416406361cfdd28a8498bfeae27e3f6a59\",\n k_wren: \"2cf52299970ac98d256e52ba0c59e914f1c83a3c7ed660ae5845ba98c22fbdc3\",\n k_fomalhaut: \"d403002c1958205ed132becdbf562c9601e4d0ccba68e581b477f80f1e7dd88b\",\n};\n\n/** Resolve a key id to its raw signing material, or throw if the id is unknown. */\nexport function keyForId(kid: string): Buffer {\n const hex = KEYRING[kid];\n if (!hex) {\n throw new Error(`unknown key id: ${JSON.stringify(kid)}`);\n }\n return Buffer.from(hex, \"hex\");\n}\n"
},
{
"name": "vestige_backfill",
"input": {},
"output": "{\"causes\":[{\"age_days_before_failure\":4.0,\"backfill_score\":1.51,\"candidate_edge_persisted\":true,\"content_preview\":\"Rotation runbook, Q3: we migrated the live signer to k_regulus and let the old material age out. Every credential minted since the cutover c\",\"memory_id\":\"f8239fc7-26d4-4994-a6f5-2227bb3dda82\",\"promoted\":true,\"reason\":\"Reached back 4.0d to a quiet memory sharing 1 entity (active_key) with the failure; it ranked #7 on similarity, so semantic search would have missed it.\",\"shared_entities\":[\"active_key\"],\"similarity_rank\":7}],\"failure\":{\"content_preview\":\"PRODUCTION OUTAGE: after the externalize-secrets refactor, the identity issuer and the ledger verifier both come up with no signing key id selected, so every ch\",\"entities\":[\"production\",\"active_key\",\"outage\",\"crash\"],\"id\":\"d690fc5c-6a04-499c-a10e-07499e57f740\"},\"headline\":\"Reached back across history from the failure and surfaced 1 causal memory that semantic search would have missed.\",\"lookback_days\":365,\"note\":\"Causes are ranked by causal join (shared entities, backward in time), NOT semantic similarity. A high similarity_rank means a vector search would NOT have surfaced this \u2014 that is the point.\",\"path_ids\":[\"f8239fc7-26d4-4994-a6f5-2227bb3dda82\",\"d690fc5c-6a04-499c-a10e-07499e57f740\"],\"scanned\":8,\"tool\":\"backfill\",\"triggered\":true}\n"
}
]
},
{
"turn": 3,
"tokens": {
"input": 5520,
"output": 362
},
"reasoning": "Interesting. The backfill reveals: there was a rotation runbook Q3 \u2014 the live signer migrated to `k_regulus`, old material aged out. Every credential minted since the cutover carries... (truncated). And the failure: \"PRODUCTION OUTAGE: after the externalize-secrets refactor, the identity issuer and the ledger verifier both come up with no signing key id selected, so every ch...\"\n\nSo the fix direction: identity-service config should select `k_regulus` as the active key, and ledger verifier should verify against... let me check the config files for both services.",
"raw_model_output": [
{
"role": "assistant",
"content": "The backfill points to a key rotation: the live signer migrated to `k_regulus`, and after an \"externalize-secrets\" refactor both services lost their active key selection. Let me check the configs and verifier.",
"tool_calls": [
{
"id": "read_file_6",
"type": "function",
"function": {
"name": "read_file",
"arguments": "{\"path\":\"identity-service/src/config.ts\"}"
}
},
{
"id": "read_file_7",
"type": "function",
"function": {
"name": "read_file",
"arguments": "{\"path\":\"ledger-service/src/config.ts\"}"
}
},
{
"id": "read_file_8",
"type": "function",
"function": {
"name": "read_file",
"arguments": "{\"path\":\"ledger-service/src/verify.ts\"}"
}
},
{
"id": "read_file_9",
"type": "function",
"function": {
"name": "read_file",
"arguments": "{\"path\":\"identity-service/src/canonical.ts\"}"
}
},
{
"id": "read_file_10",
"type": "function",
"function": {
"name": "read_file",
"arguments": "{\"path\":\"ledger-service/src/canonical.ts\"}"
}
}
],
"reasoning": "Interesting. The backfill reveals: there was a rotation runbook Q3 \u2014 the live signer migrated to `k_regulus`, old material aged out. Every credential minted since the cutover carries... (truncated). And the failure: \"PRODUCTION OUTAGE: after the externalize-secrets refactor, the identity issuer and the ledger verifier both come up with no signing key id selected, so every ch...\"\n\nSo the fix direction: identity-service config should select `k_regulus` as the active key, and ledger verifier should verify against... let me check the config files for both services."
}
],
"tool_calls": [
{
"name": "read_file",
"input": {
"path": "identity-service/src/config.ts"
}
},
{
"name": "read_file",
"input": {
"path": "ledger-service/src/config.ts"
}
},
{
"name": "read_file",
"input": {
"path": "ledger-service/src/verify.ts"
}
},
{
"name": "read_file",
"input": {
"path": "identity-service/src/canonical.ts"
}
},
{
"name": "read_file",
"input": {
"path": "ledger-service/src/canonical.ts"
}
}
],
"tool_results": [
{
"name": "read_file",
"input": {
"path": "identity-service/src/config.ts"
},
"output": "/**\n * identity-service runtime configuration.\n *\n * Signing material lives in the shared keyring (`@meridian/shared`). This config\n * selects WHICH key id the issuer signs with. During the \"externalize secrets\"\n * refactor the concrete key id was pulled out of source and is expected to be\n * injected from the environment/KMS \u2014 it is left blank here.\n */\nexport const config = {\n // The key id the issuer signs session tokens with. Selected from the shared\n // keyring at runtime. Left blank by the externalize-secrets refactor; must be\n // set for the issuer to mint tokens.\n //\n // entry the platform shipped with.\n activeKid: \"\",\n\n issuer: \"identity.meridian.internal\",\n tokenTtlSeconds: 3600,\n};\n"
},
{
"name": "read_file",
"input": {
"path": "ledger-service/src/config.ts"
},
"output": "import { SIGNING_ALG } from \"@meridian/shared\";\n\n/**\n * ledger-service runtime configuration.\n *\n * The ledger verifies incoming session tokens with an HMAC key selected from the\n * shared keyring. This config selects WHICH key id it trusts. During the\n * \"externalize secrets\" refactor the concrete key id was pulled out of source\n * and is expected to be injected from the environment/KMS \u2014 it is left blank\n * here.\n */\nexport const ledgerConfig = {\n alg: SIGNING_ALG,\n\n // The key id the ledger verifies token signatures against. Selected from the\n // shared keyring at runtime. Left blank by the externalize-secrets refactor;\n // must be set for the ledger to verify tokens.\n //\n trustedKid: \"\",\n\n currency: \"usd\" as const,\n};\n"
},
{
"name": "read_file",
"input": {
"path": "ledger-service/src/verify.ts"
},
"output": "import { createHmac, timingSafeEqual } from \"crypto\";\nimport { SessionClaims, keyForId } from \"@meridian/shared\";\nimport { ledgerConfig } from \"./config\";\nimport { canonicalClaimString } from \"./canonical\";\n\nexport class VerificationError extends Error {}\n\nfunction b64url(input: Buffer | string): string {\n return Buffer.from(input)\n .toString(\"base64\")\n .replace(/\\+/g, \"-\")\n .replace(/\\//g, \"_\")\n .replace(/=+$/, \"\");\n}\n\n/**\n * Verify a session token.\n *\n * Recomputes the canonical claim digest from the decoded claims, HMACs it with\n * the key the ledger's config selects from the shared keyring, and\n * constant-time-compares it against the token signature.\n */\nexport function verifyToken(token: string): SessionClaims {\n if (!ledgerConfig.trustedKid) {\n throw new VerificationError(\"no trusted verification key selected\");\n }\n const verifyKey = keyForId(ledgerConfig.trustedKid);\n\n const parts = token.split(\".\");\n if (parts.length !== 3) {\n throw new VerificationError(\"malformed token\");\n }\n\n const [headerB64, payloadB64, signatureB64] = parts;\n\n let header: { alg?: string };\n try {\n header = JSON.parse(\n Buffer.from(headerB64, \"base64url\").toString(\"utf8\")\n ) as { alg?: string };\n } catch {\n throw new VerificationError(\"malformed token header\");\n }\n\n // Never let a caller pick the verification primitive via the token.\n if (header.alg !== ledgerConfig.alg) {\n throw new VerificationError(\"unexpected signing algorithm\");\n }\n\n let claims: SessionClaims;\n try {\n claims = JSON.parse(\n Buffer.from(payloadB64, \"base64url\").toString(\"utf8\")\n ) as SessionClaims;\n } catch {\n throw new VerificationError(\"malformed token claims\");\n }\n\n const expected = b64url(\n createHmac(\"sha256\", verifyKey)\n .update(canonicalClaimString(claims))\n .digest()\n );\n\n const actualBuffer = Buffer.from(signatureB64);\n const expectedBuffer = Buffer.from(expected);\n const signatureValid =\n actualBuffer.length === expectedBuffer.length &&\n timingSafeEqual(actualBuffer, expectedBuffer);\n\n if (!signatureValid) {\n throw new VerificationError(\"signature mismatch\");\n }\n\n if (claims.exp < Math.floor(Date.now() / 1000)) {\n throw new VerificationError(\"token expired\");\n }\n\n return claims;\n}\n"
},
{
"name": "read_file",
"input": {
"path": "identity-service/src/canonical.ts"
},
"output": "import { SessionClaims } from \"@meridian/shared\";\n\n/**\n * Canonical claim digest input.\n *\n * The signature is computed over this canonical string rather than over the\n * raw JSON payload, so that re-serialization in transit (key order, spacing)\n * never invalidates a token. Each claim is emitted as `key=value` and the\n * claims are joined with a newline.\n */\nexport function canonicalClaimString(claims: SessionClaims): string {\n const fields: Array<[keyof SessionClaims, string | number]> = [\n [\"sub\", claims.sub],\n [\"plan\", claims.plan],\n [\"iat\", claims.iat],\n [\"exp\", claims.exp],\n ];\n\n return fields.map(([k, v]) => `${k}=${v}`).join(\"\\n\");\n}\n"
},
{
"name": "read_file",
"input": {
"path": "ledger-service/src/canonical.ts"
},
"output": "import { SessionClaims } from \"@meridian/shared\";\n\n/**\n * Canonical claim digest input for ledger-service verification.\n *\n * The ledger recomputes the canonical claim string from the decoded claims and\n * HMACs it to check the token signature. Each claim is emitted as `key=value`\n * and the claims are joined with a newline.\n */\nexport function canonicalClaimString(claims: SessionClaims): string {\n const fields: Array<[keyof SessionClaims, string | number]> = [\n [\"sub\", claims.sub],\n [\"plan\", claims.plan],\n [\"iat\", claims.iat],\n [\"exp\", claims.exp],\n ];\n\n return fields.map(([k, v]) => `${k}=${v}`).join(\"\\n\");\n}\n"
}
]
},
{
"turn": 4,
"tokens": {
"input": 7065,
"output": 149
},
"reasoning": "",
"raw_model_output": [
{
"role": "assistant",
"content": "The picture is clear: both `activeKid` (identity) and `trustedKid` (ledger) were blanked by the externalize-secrets refactor. The backfilled runbook says the platform's live signer is `k_regulus`. Let me check git history to confirm the original shipped key id before setting it.",
"tool_calls": [
{
"id": "grep_11",
"type": "function",
"function": {
"name": "grep",
"arguments": "{\"pattern\":\"k_regulus\"}"
}
},
{
"id": "read_file_12",
"type": "function",
"function": {
"name": "read_file",
"arguments": "{\"path\":\"README.md\"}"
}
}
]
}
],
"tool_calls": [
{
"name": "grep",
"input": {
"pattern": "k_regulus"
}
},
{
"name": "read_file",
"input": {
"path": "README.md"
}
}
],
"tool_results": [
{
"name": "grep",
"input": {
"pattern": "k_regulus"
},
"output": "shared/src/keyring.ts:26: k_regulus: \"e6af7ed64900d4a5ff3a71871646ae1cffeb1d2257f36186dfead6e2ab287690\","
},
{
"name": "read_file",
"input": {
"path": "README.md"
},
"output": "# Meridian Platform\n\nA minimal three-service ledger platform. Requests enter at the gateway, carry a\nsigned session token minted by identity, and terminate at the ledger, which\nverifies the token before posting the charge.\n\n```\n client\n \u2502 Authorization: Bearer <token>\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 identity-service \u2502 \u2502 gateway-service \u2502 \u2502 ledger-service \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 signSession() \u2502 \u2502 forwardToLedger() \u2502 \u2502 POST /charge \u2502\n\u2502 \u2192 mints a signed \u2502\u2500\u2500\u25b6 \u2502 \u2192 copies the raw \u2502\u2500\u2500\u25b6 \u2502 \u2192 verifyToken() \u2502\n\u2502 session token \u2502 \u2502 bearer token \u2502 \u2502 \u2192 post the charge \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 \u25b2\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500 @meridian/shared: SessionClaims, ChargeRequest \u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n## Services\n\n### `identity-service/`\nIssues and signs session tokens (`signer.ts`). The signature is an HMAC over a\ncanonical claim digest built in `canonical.ts`.\n\n### `gateway-service/`\nThe stateless edge. Applies per-plan rate limits and forwards the raw\nAuthorization bearer token downstream. It never parses or verifies tokens.\n\n### `ledger-service/`\nReceives a `ChargeRequest`, verifies the embedded token (`verify.ts`, using its\nown `canonical.ts`), resolves the user, and posts the charge. On any internal\nfailure it returns a single opaque `500`.\n\n### `shared/`\nThe cross-service contract: `SessionClaims`, `ChargeRequest`, and the error\nshapes. Every service depends on this package.\n\n## Running\n\n```sh\nnpm install\nnpm test\n```\n\n`npm test` runs the ledger-service end-to-end suite, which exercises the full\n`identity \u2192 gateway \u2192 ledger` charge path with the real modules wired together.\n\n## Token integrity contract\n\n- identity-service mints `header.payload.signature` tokens.\n- The signature covers a **canonical claim digest**, not the raw payload bytes,\n so the payload may be re-serialized in transit without invalidating the token.\n- Both the signer and the verifier independently recompute that canonical digest\n from the claims. They must produce byte-identical canonical strings.\n```\n"
}
]
},
{
"turn": 5,
"tokens": {
"input": 7820,
"output": 356
},
"reasoning": "The backfill memory says: \"Rotation runbook, Q3: we migrated the live signer to k_regulus and let the old material age out. Every credential minted since the cutover c...\" (truncated). So the current live signing key is `k_regulus`. Both configs were blanked by the externalize-secrets refactor.\n\nThe fix direction: set `identity-service/src/config.ts` `activeKid: \"k_regulus\"` and `ledger-service/src/config.ts` `trustedKid: \"k_regulus\"`. Both must match \u2014 signer and verifier must use the same key.\n\nLet me publish this direction to the shared memory so peers converge, then apply the fix.",
"raw_model_output": [
{
"role": "assistant",
"content": "The root cause is confirmed: the externalize-secrets refactor blanked both key selections, and the rotation runbook says the live signer key is `k_regulus`. Signer and verifier must select the same key. Let me log the direction and apply the fix.",
"tool_calls": [
{
"id": "vestige_log_13",
"type": "function",
"function": {
"name": "vestige_log",
"arguments": "{\"finding\":\"ROOT CAUSE of failing e2e charge test (\\\"no active signing key selected\\\"): the externalize-secrets refactor blanked identity-service config.activeKid and ledger-service ledgerConfig.trustedKid. Per the Q3 rotation runbook, the platform's live signing key is k_regulus. FIX DIRECTION: set identity-service/src/config.ts activeKid = \\\"k_regulus\\\" AND ledger-service/src/config.ts trustedKid = \\\"k_regulus\\\" (both must match; do NOT change keyring, canonical.ts, signer, or verifier logic).\"}"
}
}
],
"reasoning": "The backfill memory says: \"Rotation runbook, Q3: we migrated the live signer to k_regulus and let the old material age out. Every credential minted since the cutover c...\" (truncated). So the current live signing key is `k_regulus`. Both configs were blanked by the externalize-secrets refactor.\n\nThe fix direction: set `identity-service/src/config.ts` `activeKid: \"k_regulus\"` and `ledger-service/src/config.ts` `trustedKid: \"k_regulus\"`. Both must match \u2014 signer and verifier must use the same key.\n\nLet me publish this direction to the shared memory so peers converge, then apply the fix."
}
],
"tool_calls": [
{
"name": "vestige_log",
"input": {
"finding": "ROOT CAUSE of failing e2e charge test (\"no active signing key selected\"): the externalize-secrets refactor blanked identity-service config.activeKid and ledger-service ledgerConfig.trustedKid. Per the Q3 rotation runbook, the platform's live signing key is k_regulus. FIX DIRECTION: set identity-service/src/config.ts activeKid = \"k_regulus\" AND ledger-service/src/config.ts trustedKid = \"k_regulus\" (both must match; do NOT change keyring, canonical.ts, signer, or verifier logic)."
}
}
],
"tool_results": [
{
"name": "vestige_log",
"input": {
"finding": "ROOT CAUSE of failing e2e charge test (\"no active signing key selected\"): the externalize-secrets refactor blanked identity-service config.activeKid and ledger-service ledgerConfig.trustedKid. Per the Q3 rotation runbook, the platform's live signing key is k_regulus. FIX DIRECTION: set identity-service/src/config.ts activeKid = \"k_regulus\" AND ledger-service/src/config.ts trustedKid = \"k_regulus\" (both must match; do NOT change keyring, canonical.ts, signer, or verifier logic)."
},
"output": "logged to shared memory (other agents can now recall this)."
}
]
},
{
"turn": 6,
"tokens": {
"input": 8076,
"output": 450
},
"reasoning": "",
"raw_model_output": [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "write_file_14",
"type": "function",
"function": {
"name": "write_file",
"arguments": "{\"content\":\"/**\\n * identity-service runtime configuration.\\n *\\n * Signing material lives in the shared keyring (`@meridian/shared`). This config\\n * selects WHICH key id the issuer signs with. During the \\\"externalize secrets\\\"\\n * refactor the concrete key id was pulled out of source and is expected to be\\n * injected from the environment/KMS \u2014 it is left blank here.\\n */\\nexport const config = {\\n // The key id the issuer signs session tokens with. Selected from the shared\\n // keyring at runtime. Per the Q3 rotation runbook, the platform's live signer\\n // was migrated to k_regulus; the ledger verifies against the same key.\\n activeKid: \\\"k_regulus\\\",\\n\\n issuer: \\\"identity.meridian.internal\\\",\\n tokenTtlSeconds: 3600,\\n};\\n\",\"path\":\"identity-service/src/config.ts\"}"
}
},
{
"id": "write_file_15",
"type": "function",
"function": {
"name": "write_file",
"arguments": "{\"content\":\"import { SIGNING_ALG } from \\\"@meridian/shared\\\";\\n\\n/**\\n * ledger-service runtime configuration.\\n *\\n * The ledger verifies incoming session tokens with an HMAC key selected from the\\n * shared keyring. This config selects WHICH key id it trusts. During the\\n * \\\"externalize secrets\\\" refactor the concrete key id was pulled out of source\\n * and is expected to be injected from the environment/KMS \u2014 it is left blank\\n * here.\\n */\\nexport const ledgerConfig = {\\n alg: SIGNING_ALG,\\n\\n // The key id the ledger verifies token signatures against. Must match the\\n // issuer's active key: per the Q3 rotation runbook the platform's live signer\\n // is k_regulus.\\n trustedKid: \\\"k_regulus\\\",\\n\\n currency: \\\"usd\\\" as const,\\n};\\n\",\"path\":\"ledger-service/src/config.ts\"}"
}
}
]
}
],
"tool_calls": [
{
"name": "write_file",
"input": {
"content": "/**\n * identity-service runtime configuration.\n *\n * Signing material lives in the shared keyring (`@meridian/shared`). This config\n * selects WHICH key id the issuer signs with. During the \"externalize secrets\"\n * refactor the concrete key id was pulled out of source and is expected to be\n * injected from the environment/KMS \u2014 it is left blank here.\n */\nexport const config = {\n // The key id the issuer signs session tokens with. Selected from the shared\n // keyring at runtime. Per the Q3 rotation runbook, the platform's live signer\n // was migrated to k_regulus; the ledger verifies against the same key.\n activeKid: \"k_regulus\",\n\n issuer: \"identity.meridian.internal\",\n tokenTtlSeconds: 3600,\n};\n",
"path": "identity-service/src/config.ts"
}
},
{
"name": "write_file",
"input": {
"content": "import { SIGNING_ALG } from \"@meridian/shared\";\n\n/**\n * ledger-service runtime configuration.\n *\n * The ledger verifies incoming session tokens with an HMAC key selected from the\n * shared keyring. This config selects WHICH key id it trusts. During the\n * \"externalize secrets\" refactor the concrete key id was pulled out of source\n * and is expected to be injected from the environment/KMS \u2014 it is left blank\n * here.\n */\nexport const ledgerConfig = {\n alg: SIGNING_ALG,\n\n // The key id the ledger verifies token signatures against. Must match the\n // issuer's active key: per the Q3 rotation runbook the platform's live signer\n // is k_regulus.\n trustedKid: \"k_regulus\",\n\n currency: \"usd\" as const,\n};\n",
"path": "ledger-service/src/config.ts"
}
}
],
"tool_results": [
{
"name": "write_file",
"input": {
"content": "/**\n * identity-service runtime configuration.\n *\n * Signing material lives in the shared keyring (`@meridian/shared`). This config\n * selects WHICH key id the issuer signs with. During the \"externalize secrets\"\n * refactor the concrete key id was pulled out of source and is expected to be\n * injected from the environment/KMS \u2014 it is left blank here.\n */\nexport const config = {\n // The key id the issuer signs session tokens with. Selected from the shared\n // keyring at runtime. Per the Q3 rotation runbook, the platform's live signer\n // was migrated to k_regulus; the ledger verifies against the same key.\n activeKid: \"k_regulus\",\n\n issuer: \"identity.meridian.internal\",\n tokenTtlSeconds: 3600,\n};\n",
"path": "identity-service/src/config.ts"
},
"output": "wrote 710 bytes to identity-service/src/config.ts"
},
{
"name": "write_file",
"input": {
"content": "import { SIGNING_ALG } from \"@meridian/shared\";\n\n/**\n * ledger-service runtime configuration.\n *\n * The ledger verifies incoming session tokens with an HMAC key selected from the\n * shared keyring. This config selects WHICH key id it trusts. During the\n * \"externalize secrets\" refactor the concrete key id was pulled out of source\n * and is expected to be injected from the environment/KMS \u2014 it is left blank\n * here.\n */\nexport const ledgerConfig = {\n alg: SIGNING_ALG,\n\n // The key id the ledger verifies token signatures against. Must match the\n // issuer's active key: per the Q3 rotation runbook the platform's live signer\n // is k_regulus.\n trustedKid: \"k_regulus\",\n\n currency: \"usd\" as const,\n};\n",
"path": "ledger-service/src/config.ts"
},
"output": "wrote 710 bytes to ledger-service/src/config.ts"
}
]
},
{
"turn": 7,
"tokens": {
"input": 8595,
"output": 37
},
"reasoning": "",
"raw_model_output": [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "run_tests_16",
"type": "function",
"function": {
"name": "run_tests",
"arguments": "{}"
}
}
]
}
],
"tool_calls": [
{
"name": "run_tests",
"input": {}
}
],
"tool_results": [
{
"name": "run_tests",
"input": {},
"output": "exit_code=0\n\n> meridian-platform@1.0.0 test\n> npm run test --workspace @meridian/ledger-service\n\n\n> @meridian/ledger-service@1.0.0 test\n> tsx --test test/*.test.ts\n\n\u2714 a freshly-minted token charges successfully (expects 200) (21.208625ms)\n\u2139 tests 1\n\u2139 suites 0\n\u2139 pass 1\n\u2139 fail 0\n\u2139 cancelled 0\n\u2139 skipped 0\n\u2139 todo 0\n\u2139 duration_ms 185.052042\n\n"
}
]
},
{
"turn": 8,
"tokens": {
"input": 8788,
"output": 236
},
"reasoning": "",
"raw_model_output": [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "finish_17",
"type": "function",
"function": {
"name": "finish",
"arguments": "{\"fixed\":true,\"summary\":\"Root cause: the \\\"externalize-secrets\\\" refactor blanked the signing-key selection in both identity-service (config.activeKid = \\\"\\\") and ledger-service (ledgerConfig.trustedKid = \\\"\\\"), so signSession threw \\\"no active signing key selected\\\" and verification had no trusted key. A retroactive backfill surfaced the Q3 rotation runbook identifying k_regulus as the platform's live signing key. Fix: set activeKid = \\\"k_regulus\\\" in identity-service/src/config.ts and trustedKid = \\\"k_regulus\\\" in ledger-service/src/config.ts (signer and verifier must select the same key). No changes to keyring material, canonical digest, signer, or verifier logic. The end-to-end charge test now passes (exit 0). Direction published to shared fleet memory via vestige_log so peers converge on the same change.\"}"
}
}
]
}
],
"tool_calls": [
{
"name": "finish",
"input": {
"fixed": true,
"summary": "Root cause: the \"externalize-secrets\" refactor blanked the signing-key selection in both identity-service (config.activeKid = \"\") and ledger-service (ledgerConfig.trustedKid = \"\"), so signSession threw \"no active signing key selected\" and verification had no trusted key. A retroactive backfill surfaced the Q3 rotation runbook identifying k_regulus as the platform's live signing key. Fix: set activeKid = \"k_regulus\" in identity-service/src/config.ts and trustedKid = \"k_regulus\" in ledger-service/src/config.ts (signer and verifier must select the same key). No changes to keyring material, canonical digest, signer, or verifier logic. The end-to-end charge test now passes (exit 0). Direction published to shared fleet memory via vestige_log so peers converge on the same change."
}
}
],
"tool_results": [
{
"name": "finish",
"input": {
"fixed": true,
"summary": "Root cause: the \"externalize-secrets\" refactor blanked the signing-key selection in both identity-service (config.activeKid = \"\") and ledger-service (ledgerConfig.trustedKid = \"\"), so signSession threw \"no active signing key selected\" and verification had no trusted key. A retroactive backfill surfaced the Q3 rotation runbook identifying k_regulus as the platform's live signing key. Fix: set activeKid = \"k_regulus\" in identity-service/src/config.ts and trustedKid = \"k_regulus\" in ledger-service/src/config.ts (signer and verifier must select the same key). No changes to keyring material, canonical digest, signer, or verifier logic. The end-to-end charge test now passes (exit 0). Direction published to shared fleet memory via vestige_log so peers converge on the same change."
},
"output": "acknowledged"
}
]
}
]
}
Reference in a new issue View git blame Copy permalink