From 0cc635bd5e2c8dcfa0cd7c60b05573d5f3407c0a Mon Sep 17 00:00:00 2001 From: QA Runner Date: Mon, 20 Jul 2026 21:22:01 -0700 Subject: [PATCH] =?UTF-8?q?docs(e2e):=20mark=20the=20model-selection=20gap?= =?UTF-8?q?=20fixed=20=E2=80=94=20specs=20pick=20a=20Claude=20model=20when?= =?UTF-8?q?=20the=20key=20is=20set?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The "Known gap" paragraph predicted the 4 LLM specs would fail with the secret set because they selected the fork-only "Demo (no key needed)" model. That is now fixed on the e2e suite branch (#220): the helper selects "Claude Sonnet 4.6" and the critical-path assertion checks for a nonempty streamed assistant answer. Update the paragraph accordingly: setting ANTHROPIC_API_KEY now yields 27 passed / 0 skipped (verified locally; keyless stays 23 passed / 4 skipped). Co-Authored-By: Claude Fable 5 --- docs/e2e-ci.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/docs/e2e-ci.md b/docs/e2e-ci.md index e6e8474c..a201b8a6 100644 --- a/docs/e2e-ci.md +++ b/docs/e2e-ci.md @@ -105,16 +105,19 @@ Open the **Run Playwright** step in the Actions log: The uploaded `playwright-report` artifact shows the same per-spec statuses. -### Known gap: model selection +### Model selection (gap fixed on the e2e suite branch) -As of this branch the four specs drive the model picker to a keyless +Earlier revisions of the four specs drove the model picker to a keyless **"Demo (no key needed)"** entry that exists in the amal66 fork but **not in -this repository** (no `mike-demo` model id, no demo provider). With the secret -set they therefore unskip and then fail at model selection. Before the secret -turns them green, one of two follow-ups must land: point the specs' -`selectDemoModel` helper at a Claude model when the key is present, or upstream -the fork's keyless demo provider. The secret setup above is correct and ready -either way. +this repository** (no `mike-demo` model id, no demo provider), so setting the +secret unskipped them and they then failed at model selection. This is fixed on +the e2e suite branch (#220): the specs' `selectClaudeModel` helper picks +**Claude Sonnet 4.6** in the ModelToggle whenever the key is set, and the +critical-path response assertion checks for a nonempty streamed assistant +answer instead of the fork's canned demo reply. With that fix, setting the +`ANTHROPIC_API_KEY` secret yields **27 passed / 0 skipped** (verified locally: +keyless 23 passed / 4 skipped; with the key 27 / 0). The secret setup above is +all that is needed. ## Make it merge-blocking