mirror of
https://github.com/willchen96/mike.git
synced 2026-07-26 23:51:08 +02:00
72242ff0eb
The two doc commits were written against a lineage where the selectClaudeModel spec fix lived on a separate branch, which left three stale claims after the rebase: - e2e/llm.ts still called selectDemoModel a known gap and pointed at a docs section title that no longer exists; the specs in this very branch already use selectClaudeModel. Replace the paragraph with the current behavior. - docs/e2e-ci.md attributed the fix to "the e2e suite branch (#220)"; this branch carries that commit itself, so say so and name the commit. - The 23/4 keyless and 27/0 with-key figures were presented as bare facts ("verified locally"). Attribute them to where they were actually measured — the spec-fix commit's own verification against a full local stack — and state they have not been re-measured since the rebase onto current main. Drop the unattributed "~7 minutes" typical-run figure in favor of the structural 27-spec / 4-gated breakdown (confirmed by `npx playwright test --list`: 27 tests in 7 files). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
36 lines
2 KiB
TypeScript
36 lines
2 KiB
TypeScript
/**
|
|
* Helpers for specs that require a live LLM turn.
|
|
*
|
|
* Four specs (chat rename, chat delete, project-assistant create+submit, and
|
|
* the critical-path "ask a question" flow) create/populate a chat by sending a
|
|
* message. This codebase ships no keyless model: every entry in the model
|
|
* picker (frontend/src/app/components/assistant/ModelToggle.tsx MODELS) is an
|
|
* Anthropic/Google/OpenAI model whose availability requires a configured
|
|
* provider key — backend env var or user-stored key
|
|
* (backend/src/lib/userApiKeys.ts) — and ChatInput.handleSubmit refuses to
|
|
* send when the selected model's key is missing (ApiKeyMissingPopup). The
|
|
* backend enforces the same model set server-side
|
|
* (backend/src/lib/llm/models.ts ALL_MODELS/providerForModel). So with no key
|
|
* there is no way for these specs to send a message at all; unguarded they
|
|
* would hang to their timeout.
|
|
*
|
|
* The auto title-generation call (POST /chat/:id/generate-title) is NOT why
|
|
* the gate exists: keyless it just returns 500, and the specs already treat it
|
|
* as best-effort (`.catch(() => null)`).
|
|
*
|
|
* In CI the key is the `ANTHROPIC_API_KEY` repository secret, which
|
|
* `.github/workflows/e2e.yml` exposes both to the backend (backend/.env) and
|
|
* to the Playwright process. Guarding with
|
|
* `test.skip(!hasLlmKey, LLM_SKIP_REASON)` keeps a keyless run (a plain local
|
|
* run, or a fork PR with no secret access) green and fast on the other 23
|
|
* specs, while still running — and enforcing — the LLM specs whenever the key
|
|
* is present. Setup steps: docs/e2e-ci.md, "Enable the LLM specs".
|
|
*
|
|
* When the key IS set, the specs' selectClaudeModel helper picks "Claude
|
|
* Sonnet 4.6" in the ModelToggle (see docs/e2e-ci.md, "Model selection"), so
|
|
* the unskipped specs submit against a model this repository actually ships.
|
|
*/
|
|
export const hasLlmKey = Boolean(process.env.ANTHROPIC_API_KEY);
|
|
|
|
export const LLM_SKIP_REASON =
|
|
"requires a model key — set the ANTHROPIC_API_KEY secret to run LLM-dependent specs";
|