diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.yml new file mode 100644 index 00000000..c03fda47 --- /dev/null +++ b/.github/workflows/e2e.yml @@ -0,0 +1,178 @@ +# End-to-end Playwright suite (auth, chat, projects, tabular reviews, workflows). +# +# This workflow is what makes the e2e suite a MERGE GATE: it runs on every pull +# request into a protected branch, boots a full local stack (Supabase + backend +# API + Next.js web + MinIO object storage), runs Playwright, and fails the check +# when any spec fails. To have a red run actually BLOCK a merge you must also mark +# this job "e2e / playwright" as a required status check in branch protection — +# see docs/e2e-ci.md ("Make it merge-blocking"). +# +# Required repository secret for a full pass: +# ANTHROPIC_API_KEY — the critical-path / chat specs send a message and expect a +# streamed answer, which needs a live model key. Add it under +# Settings > Secrets and variables > Actions. Without it the +# chat specs fail; every other spec runs key-less. +# +# The e2e user (e2e@mike.local) is bootstrapped in-job by e2e/auth.setup.ts against +# the local Supabase admin API, so no E2E_PASSWORD secret is needed — the defaults +# baked into auth.setup.ts are the single source of truth. +name: e2e + +on: + workflow_dispatch: + pull_request: + # `main` is the destination upstream; `upstream-main` is the fork's mirror of + # it, so the suite is exercised on the fork PR and stays correct once merged. + branches: [main, upstream-main] + +# Don't pile up runs on rapid pushes to the same PR. +concurrency: + group: e2e-${{ github.ref }} + cancel-in-progress: true + +jobs: + playwright: + timeout-minutes: 30 + runs-on: ubuntu-latest + env: + # Defaults match e2e/auth.setup.ts; overriding the password here would break + # the valid-login specs, so only the base URL is pinned. + PLAYWRIGHT_BASE_URL: http://localhost:3000 + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-node@v4 + with: + node-version: 22 + cache: npm + + # Three installs: the root package (Playwright + the e2e tooling), the + # backend API, and the Next.js web app. Each carries its own lockfile in + # this repo's backend/ + frontend/ layout (no root workspace). + - name: Install root (Playwright) deps + run: npm ci + - name: Install backend deps + run: npm ci --prefix backend + - name: Install frontend deps + run: npm ci --prefix frontend + + # npm ci intermittently skips lightningcss's Linux native optional dep + # (npm/cli#4828); without it `next dev` can't compile Tailwind CSS and the + # web server never boots, so wait-on times out and every spec fails. + - name: Ensure lightningcss native binary + run: npm install --no-save lightningcss --prefix frontend + + - name: Install Playwright browsers + run: npx playwright install --with-deps chromium + + # Object storage. Three specs upload a document (tabular-review row, + # project-folder fixture) and assert the result renders; those uploads go + # through backend/src/lib/storage.ts, an S3-compatible client that only + # ENABLES when R2_ENDPOINT_URL + R2_ACCESS_KEY_ID + R2_SECRET_ACCESS_KEY are + # set (region "auto", forcePathStyle true — MinIO-compatible as written). + # With no storage the upload endpoint 5xxs and the row never appears. A + # `docker run` (not a `services:` container) is used because the minio image + # needs the `server /data` argument the services block can't supply, and we + # must create the bucket after the server is healthy. + - name: Start MinIO (object storage) + run: | + docker run -d --name minio -p 9000:9000 \ + -e MINIO_ROOT_USER=minioadmin \ + -e MINIO_ROOT_PASSWORD=minioadmin \ + minio/minio:RELEASE.2025-09-07T16-13-09Z server /data + for i in $(seq 1 30); do + if curl -sf http://localhost:9000/minio/health/ready >/dev/null; then + echo "MinIO ready"; break + fi + echo "waiting for minio ($i)…"; sleep 2 + done + AWS_ACCESS_KEY_ID=minioadmin \ + AWS_SECRET_ACCESS_KEY=minioadmin \ + AWS_DEFAULT_REGION=us-east-1 \ + aws --endpoint-url http://localhost:9000 s3 mb s3://mike + AWS_ACCESS_KEY_ID=minioadmin \ + AWS_SECRET_ACCESS_KEY=minioadmin \ + AWS_DEFAULT_REGION=us-east-1 \ + aws --endpoint-url http://localhost:9000 s3 ls s3://mike + + # Supabase (Auth + Postgres). This layout ships no supabase/ config dir + # (the app targets a hosted Supabase project), so we scaffold one with + # `supabase init` purely to boot the CLI's local stack, then load the + # repository's own fresh-database schema — exactly what the README tells a + # human to run on a new database (backend/schema.sql). + - name: Start local Supabase + uses: supabase/setup-cli@v1 + with: + version: latest + - name: Init + start Supabase, load schema + run: | + supabase init --force --with-vscode-settings=false || supabase init --force + supabase start + DB_URL=$(supabase status -o json | jq -r '.DB_URL') + # README: "For a new Supabase database ... run backend/schema.sql". The + # schema file already includes the latest shape, so no migrations are + # replayed on a fresh CI database. + psql "$DB_URL" -v ON_ERROR_STOP=1 -f backend/schema.sql + + - name: Wire env from Supabase + run: | + API_URL=$(supabase status -o json | jq -r '.API_URL') + ANON_KEY=$(supabase status -o json | jq -r '.ANON_KEY') + SERVICE_KEY=$(supabase status -o json | jq -r '.SERVICE_ROLE_KEY') + # dotenv is last-wins: start from the committed example (so any key we + # don't override keeps a sane placeholder) then append the real values. + cp backend/.env.example backend/.env + { + echo "PORT=3001" + echo "FRONTEND_URL=http://localhost:3000" + echo "SUPABASE_URL=$API_URL" + echo "SUPABASE_SECRET_KEY=$SERVICE_KEY" + # Both secrets are consumed lazily by download-token signing and API-key + # encryption; supply CI dummies well over any length floor. + echo "DOWNLOAD_SIGNING_SECRET=ci-download-signing-secret-0123456789abcdef" + echo "USER_API_KEYS_ENCRYPTION_SECRET=ci-user-api-keys-encryption-secret-0123456789abcdef" + # The suite drives many writes back-to-back (create folder/workflow/ + # review, upload, update profile, login); the default per-window caps + # trip 429s that surface as flaky timeouts. e2e isn't testing + # throttling, so raise every tunable cap above one serial run's needs. + echo "RATE_LIMIT_GENERAL_MAX=100000" + echo "RATE_LIMIT_CHAT_MAX=100000" + echo "RATE_LIMIT_CHAT_CREATE_MAX=100000" + echo "RATE_LIMIT_UPLOAD_MAX=100000" + echo "RATE_LIMIT_EXPORT_MAX=100000" + echo "RATE_LIMIT_DATA_DELETE_MAX=100000" + # Point the S3-compatible storage adapter at the MinIO container above. + echo "R2_ENDPOINT_URL=http://localhost:9000" + echo "R2_ACCESS_KEY_ID=minioadmin" + echo "R2_SECRET_ACCESS_KEY=minioadmin" + echo "R2_BUCKET_NAME=mike" + echo "ANTHROPIC_API_KEY=${{ secrets.ANTHROPIC_API_KEY }}" + } >> backend/.env + # Frontend reads NEXT_PUBLIC_* at dev-server boot. + { + echo "NEXT_PUBLIC_SUPABASE_URL=$API_URL" + echo "NEXT_PUBLIC_SUPABASE_PUBLISHABLE_DEFAULT_KEY=$ANON_KEY" + echo "NEXT_PUBLIC_API_BASE_URL=http://localhost:3001" + } > frontend/.env.local + + - name: Start backend API + run: npm run dev --prefix backend & + - name: Start web + run: npm run dev --prefix frontend & + + - name: Wait for servers + run: npx wait-on http://localhost:3001/health http://localhost:3000 --timeout 120000 + + # playwright.config.ts sets webServer to undefined when CI=true, so the job + # is responsible for the servers above; Playwright just drives them. + - name: Run Playwright + run: npx playwright test + + - uses: actions/upload-artifact@v4 + if: ${{ !cancelled() }} + with: + name: playwright-report + path: | + playwright-report/ + test-results/ + retention-days: 14 diff --git a/docs/e2e-ci.md b/docs/e2e-ci.md new file mode 100644 index 00000000..8435df46 --- /dev/null +++ b/docs/e2e-ci.md @@ -0,0 +1,82 @@ +# End-to-end tests in CI + +The Playwright suite (`e2e/`) runs on every pull request through +`.github/workflows/e2e.yml`. This document covers the one repository secret it +needs and the **branch-protection step that turns a red run into a blocked +merge** — the workflow reports pass/fail on its own, but only branch protection +makes that check *required*. + +## What the workflow does + +On every `pull_request` targeting `main` (or `upstream-main`, the fork mirror), +and on manual `workflow_dispatch`, the `e2e / playwright` job: + +1. installs the root (Playwright), `backend/`, and `frontend/` dependencies; +2. boots **MinIO** (S3-compatible object storage — three specs upload documents); +3. boots **local Supabase** (Auth + Postgres) via the Supabase CLI and loads + `backend/schema.sql` — the same fresh-database schema the README tells a human + to run; +4. writes `backend/.env` and `frontend/.env.local` from the live Supabase values; +5. starts the backend API (`:3001`) and the Next.js web app (`:3000`), waits for + both to be healthy; +6. runs `npx playwright test` and uploads the HTML report + traces as an artifact + (`playwright-report`) on both pass and fail. + +`e2e/auth.setup.ts` bootstraps the shared test user (`e2e@mike.local`) against +the local Supabase admin API, so no login secret is needed — the credentials +baked into that file are the single source of truth. + +## Required secret + +| Secret | Why | Without it | +|---|---|---| +| `ANTHROPIC_API_KEY` | The critical-path / chat specs send a message and assert a **streamed** answer, which needs a live model key. | Those chat specs fail; every other spec still runs. | + +Add it under **Settings → Secrets and variables → Actions → New repository +secret**. For pull requests opened from a **fork**, GitHub withholds secrets by +default — a maintainer approves the run (or re-runs from the branch) so the key +is available. Treat that approval as the point where the chat specs become +enforceable for external contributions. + +## Make it merge-blocking + +The workflow failing is not enough on its own — GitHub will still allow the merge +unless the check is **required**. Enable branch protection once you have seen the +suite go green a few times (it is environment-sensitive by nature): + +1. **Settings → Branches → Add branch protection rule** (or edit the rule for + `main`). +2. Enable **Require status checks to pass before merging**. +3. Enable **Require branches to be up to date before merging**. +4. In the checks search box add **`e2e / playwright`** (the job appears in the + list after it has run at least once on a PR). +5. Recommended alongside it: the unit/build check `backend` and the `license/cla` + check. +6. Save. From now on a red e2e run blocks the **Merge** button. + +Equivalent via the GitHub CLI (repo admin token required): + +```bash +gh api -X PUT repos/OWNER/REPO/branches/main/protection \ + -H "Accept: application/vnd.github+json" \ + -f 'required_status_checks[strict]=true' \ + -f 'required_status_checks[contexts][]=e2e / playwright' \ + -f 'enforce_admins=true' \ + -f 'required_pull_request_reviews[required_approving_review_count]=1' \ + -f 'restrictions=' +``` + +## Running the suite locally + +Locally, `playwright.config.ts` starts the backend and web dev servers for you +(`webServer` is only disabled when `CI=true`), so a full local stack plus: + +```bash +npm ci +npx playwright install --with-deps chromium +npm run test:e2e # or test:e2e:ui / test:e2e:headed +``` + +`e2e/auth.setup.ts` reads `SUPABASE_URL` / `SUPABASE_SECRET_KEY` from the +environment or `backend/.env`, so a running local Supabase + a populated +`backend/.env` is all the setup needs.