apunkt/mike
1
0
Fork 0
mirror of https://github.com/willchen96/mike.git synced 2026-07-26 23:51:08 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

ci(e2e): run Playwright suite on PRs as a merge gate

Browse source 
Adds .github/workflows/e2e.yml adapted to this repo's backend/ + frontend/
layout: on every pull_request into main (or upstream-main), boot MinIO +
local Supabase (loading backend/schema.sql), start the API and web dev
servers, and run the Playwright suite, uploading the HTML report/traces on
pass or fail. playwright.config.ts already disables its local webServer when
CI=true, so the job owns the stack.

docs/e2e-ci.md documents the one required secret (ANTHROPIC_API_KEY) and the
branch-protection steps that make the 'e2e / playwright' check required, so a
red run blocks the merge button.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
QA Runner 2026-07-17 13:36:49 -07:00
parent e551028726
commit 82fd540ee5
2 changed files with 260 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

178
.github/workflows/e2e.yml vendored Normal file
View file

@ -0,0 +1,178 @@
# End-to-end Playwright suite (auth, chat, projects, tabular reviews, workflows).
#
# This workflow is what makes the e2e suite a MERGE GATE: it runs on every pull
# request into a protected branch, boots a full local stack (Supabase + backend
# API + Next.js web + MinIO object storage), runs Playwright, and fails the check
# when any spec fails. To have a red run actually BLOCK a merge you must also mark
# this job "e2e / playwright" as a required status check in branch protection —
# see docs/e2e-ci.md ("Make it merge-blocking").
#
# Required repository secret for a full pass:
# ANTHROPIC_API_KEY — the critical-path / chat specs send a message and expect a
# streamed answer, which needs a live model key. Add it under
# Settings > Secrets and variables > Actions. Without it the
# chat specs fail; every other spec runs key-less.
#
# The e2e user (e2e@mike.local) is bootstrapped in-job by e2e/auth.setup.ts against
# the local Supabase admin API, so no E2E_PASSWORD secret is needed — the defaults
# baked into auth.setup.ts are the single source of truth.
name: e2e
on:
workflow_dispatch:
pull_request:
# `main` is the destination upstream; `upstream-main` is the fork's mirror of
# it, so the suite is exercised on the fork PR and stays correct once merged.
branches: [main, upstream-main]
# Don't pile up runs on rapid pushes to the same PR.
concurrency:
group: e2e-${{ github.ref }}
cancel-in-progress: true
jobs:
playwright:
timeout-minutes: 30
runs-on: ubuntu-latest
env:
# Defaults match e2e/auth.setup.ts; overriding the password here would break
# the valid-login specs, so only the base URL is pinned.
PLAYWRIGHT_BASE_URL: http://localhost:3000
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
# Three installs: the root package (Playwright + the e2e tooling), the
# backend API, and the Next.js web app. Each carries its own lockfile in
# this repo's backend/ + frontend/ layout (no root workspace).
- name: Install root (Playwright) deps
run: npm ci
- name: Install backend deps
run: npm ci --prefix backend
- name: Install frontend deps
run: npm ci --prefix frontend
# npm ci intermittently skips lightningcss's Linux native optional dep
# (npm/cli#4828); without it `next dev` can't compile Tailwind CSS and the
# web server never boots, so wait-on times out and every spec fails.
- name: Ensure lightningcss native binary
run: npm install --no-save lightningcss --prefix frontend
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium
# Object storage. Three specs upload a document (tabular-review row,
# project-folder fixture) and assert the result renders; those uploads go
# through backend/src/lib/storage.ts, an S3-compatible client that only
# ENABLES when R2_ENDPOINT_URL + R2_ACCESS_KEY_ID + R2_SECRET_ACCESS_KEY are
# set (region "auto", forcePathStyle true — MinIO-compatible as written).
# With no storage the upload endpoint 5xxs and the row never appears. A
# `docker run` (not a `services:` container) is used because the minio image
# needs the `server /data` argument the services block can't supply, and we
# must create the bucket after the server is healthy.
- name: Start MinIO (object storage)
run: |
docker run -d --name minio -p 9000:9000 \
-e MINIO_ROOT_USER=minioadmin \
-e MINIO_ROOT_PASSWORD=minioadmin \
minio/minio:RELEASE.2025-09-07T16-13-09Z server /data
for i in $(seq 1 30); do
if curl -sf http://localhost:9000/minio/health/ready >/dev/null; then
echo "MinIO ready"; break
fi
echo "waiting for minio ($i)…"; sleep 2
done
AWS_ACCESS_KEY_ID=minioadmin \
AWS_SECRET_ACCESS_KEY=minioadmin \
AWS_DEFAULT_REGION=us-east-1 \
aws --endpoint-url http://localhost:9000 s3 mb s3://mike
AWS_ACCESS_KEY_ID=minioadmin \
AWS_SECRET_ACCESS_KEY=minioadmin \
AWS_DEFAULT_REGION=us-east-1 \
aws --endpoint-url http://localhost:9000 s3 ls s3://mike
# Supabase (Auth + Postgres). This layout ships no supabase/ config dir
# (the app targets a hosted Supabase project), so we scaffold one with
# `supabase init` purely to boot the CLI's local stack, then load the
# repository's own fresh-database schema — exactly what the README tells a
# human to run on a new database (backend/schema.sql).
- name: Start local Supabase
uses: supabase/setup-cli@v1
with:
version: latest
- name: Init + start Supabase, load schema
run: |
supabase init --force --with-vscode-settings=false || supabase init --force
supabase start
DB_URL=$(supabase status -o json | jq -r '.DB_URL')
# README: "For a new Supabase database ... run backend/schema.sql". The
# schema file already includes the latest shape, so no migrations are
# replayed on a fresh CI database.
psql "$DB_URL" -v ON_ERROR_STOP=1 -f backend/schema.sql
- name: Wire env from Supabase
run: |
API_URL=$(supabase status -o json | jq -r '.API_URL')
ANON_KEY=$(supabase status -o json | jq -r '.ANON_KEY')
SERVICE_KEY=$(supabase status -o json | jq -r '.SERVICE_ROLE_KEY')
# dotenv is last-wins: start from the committed example (so any key we
# don't override keeps a sane placeholder) then append the real values.
cp backend/.env.example backend/.env
{
echo "PORT=3001"
echo "FRONTEND_URL=http://localhost:3000"
echo "SUPABASE_URL=$API_URL"
echo "SUPABASE_SECRET_KEY=$SERVICE_KEY"
# Both secrets are consumed lazily by download-token signing and API-key
# encryption; supply CI dummies well over any length floor.
echo "DOWNLOAD_SIGNING_SECRET=ci-download-signing-secret-0123456789abcdef"
echo "USER_API_KEYS_ENCRYPTION_SECRET=ci-user-api-keys-encryption-secret-0123456789abcdef"
# The suite drives many writes back-to-back (create folder/workflow/
# review, upload, update profile, login); the default per-window caps
# trip 429s that surface as flaky timeouts. e2e isn't testing
# throttling, so raise every tunable cap above one serial run's needs.
echo "RATE_LIMIT_GENERAL_MAX=100000"
echo "RATE_LIMIT_CHAT_MAX=100000"
echo "RATE_LIMIT_CHAT_CREATE_MAX=100000"
echo "RATE_LIMIT_UPLOAD_MAX=100000"
echo "RATE_LIMIT_EXPORT_MAX=100000"
echo "RATE_LIMIT_DATA_DELETE_MAX=100000"
# Point the S3-compatible storage adapter at the MinIO container above.
echo "R2_ENDPOINT_URL=http://localhost:9000"
echo "R2_ACCESS_KEY_ID=minioadmin"
echo "R2_SECRET_ACCESS_KEY=minioadmin"
echo "R2_BUCKET_NAME=mike"
echo "ANTHROPIC_API_KEY=${{ secrets.ANTHROPIC_API_KEY }}"
} >> backend/.env
# Frontend reads NEXT_PUBLIC_* at dev-server boot.
{
echo "NEXT_PUBLIC_SUPABASE_URL=$API_URL"
echo "NEXT_PUBLIC_SUPABASE_PUBLISHABLE_DEFAULT_KEY=$ANON_KEY"
echo "NEXT_PUBLIC_API_BASE_URL=http://localhost:3001"
} > frontend/.env.local
- name: Start backend API
run: npm run dev --prefix backend &
- name: Start web
run: npm run dev --prefix frontend &
- name: Wait for servers
run: npx wait-on http://localhost:3001/health http://localhost:3000 --timeout 120000
# playwright.config.ts sets webServer to undefined when CI=true, so the job
# is responsible for the servers above; Playwright just drives them.
- name: Run Playwright
run: npx playwright test
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report
path: |
playwright-report/
test-results/
retention-days: 14

82
docs/e2e-ci.md Normal file
View file

@ -0,0 +1,82 @@
# End-to-end tests in CI
The Playwright suite (`e2e/`) runs on every pull request through
`.github/workflows/e2e.yml`. This document covers the one repository secret it
needs and the **branch-protection step that turns a red run into a blocked
merge** — the workflow reports pass/fail on its own, but only branch protection
makes that check *required*.
## What the workflow does
On every `pull_request` targeting `main` (or `upstream-main`, the fork mirror),
and on manual `workflow_dispatch`, the `e2e / playwright` job:
1. installs the root (Playwright), `backend/`, and `frontend/` dependencies;
2. boots **MinIO** (S3-compatible object storage — three specs upload documents);
3. boots **local Supabase** (Auth + Postgres) via the Supabase CLI and loads
`backend/schema.sql` — the same fresh-database schema the README tells a human
to run;
4. writes `backend/.env` and `frontend/.env.local` from the live Supabase values;
5. starts the backend API (`:3001`) and the Next.js web app (`:3000`), waits for
both to be healthy;
6. runs `npx playwright test` and uploads the HTML report + traces as an artifact
(`playwright-report`) on both pass and fail.
`e2e/auth.setup.ts` bootstraps the shared test user (`e2e@mike.local`) against
the local Supabase admin API, so no login secret is needed — the credentials
baked into that file are the single source of truth.
## Required secret
| Secret | Why | Without it |
|---|---|---|
| `ANTHROPIC_API_KEY` | The critical-path / chat specs send a message and assert a **streamed** answer, which needs a live model key. | Those chat specs fail; every other spec still runs. |
Add it under **Settings → Secrets and variables → Actions → New repository
secret**. For pull requests opened from a **fork**, GitHub withholds secrets by
default — a maintainer approves the run (or re-runs from the branch) so the key
is available. Treat that approval as the point where the chat specs become
enforceable for external contributions.
## Make it merge-blocking
The workflow failing is not enough on its own — GitHub will still allow the merge
unless the check is **required**. Enable branch protection once you have seen the
suite go green a few times (it is environment-sensitive by nature):
1. **Settings → Branches → Add branch protection rule** (or edit the rule for
`main`).
2. Enable **Require status checks to pass before merging**.
3. Enable **Require branches to be up to date before merging**.
4. In the checks search box add **`e2e / playwright`** (the job appears in the
list after it has run at least once on a PR).
5. Recommended alongside it: the unit/build check `backend` and the `license/cla`
check.
6. Save. From now on a red e2e run blocks the **Merge** button.
Equivalent via the GitHub CLI (repo admin token required):
```bash
gh api -X PUT repos/OWNER/REPO/branches/main/protection \
-H "Accept: application/vnd.github+json" \
-f 'required_status_checks[strict]=true' \
-f 'required_status_checks[contexts][]=e2e / playwright' \
-f 'enforce_admins=true' \
-f 'required_pull_request_reviews[required_approving_review_count]=1' \
-f 'restrictions='
```
## Running the suite locally
Locally, `playwright.config.ts` starts the backend and web dev servers for you
(`webServer` is only disabled when `CI=true`), so a full local stack plus:
```bash
npm ci
npx playwright install --with-deps chromium
npm run test:e2e # or test:e2e:ui / test:e2e:headed
```
`e2e/auth.setup.ts` reads `SUPABASE_URL` / `SUPABASE_SECRET_KEY` from the
environment or `backend/.env`, so a running local Supabase + a populated
`backend/.env` is all the setup needs.