SurfSense/.cursor/skills/playwright-testing/infrastructure-ci-cd/parallel-sharding.md

Sharding and Parallel Execution

Table of Contents

1. CLI Commands
2. Patterns
3. Decision Guide
4. Anti-Patterns
5. Troubleshooting

When to use: Speeding up test suites by running tests concurrently on one machine (workers) or splitting across multiple CI jobs (sharding).

CLI Commands

# Parallelism within one machine
npx playwright test --workers=4
npx playwright test --workers=50%

# Splitting across CI jobs
npx playwright test --shard=1/4
npx playwright test --shard=2/4

# Merging shard outputs
npx playwright merge-reports ./blob-report
npx playwright merge-reports --reporter=html,json ./blob-report

# Override config for single run
npx playwright test --fully-parallel

Patterns

Worker Configuration

Use when: Controlling concurrent test execution on a single machine.

// playwright.config.ts
import { defineConfig } from "@playwright/test";

export default defineConfig({
  // Tests WITHIN a file also run in parallel
  fullyParallel: true,

  // Worker count options:
  // - undefined: auto-detect (half CPU cores)
  // - number: fixed count
  // - string: percentage of cores
  workers: process.env.CI ? "50%" : undefined,
});

fullyParallel behavior:

Setting	Files parallel	Tests in file parallel
fullyParallel: false (default)	Yes	No (serial)
fullyParallel: true	Yes	Yes

Serial execution for specific files:

// tests/checkout-flow.spec.ts
import { test, expect } from "@playwright/test";

test.describe.configure({ mode: "serial" });

test("add items to cart", async ({ page }) => {
  // ...
});

test("complete payment", async ({ page }) => {
  // ...
});

Sharding Across CI Machines

Use when: Suite exceeds 5 minutes even with maximum workers.

# Job 1            Job 2            Job 3            Job 4
--shard=1/4        --shard=2/4      --shard=3/4      --shard=4/4

Config for sharded runs:

// playwright.config.ts
import { defineConfig } from "@playwright/test";

export default defineConfig({
  fullyParallel: true,
  workers: process.env.CI ? "50%" : undefined,

  reporter: process.env.CI
    ? [["blob"], ["github"]]
    : [["html", { open: "on-failure" }]],
});

Merging Shard Reports

Use when: Combining blob reports from multiple shards into a unified report.

# Merge all blobs into HTML
npx playwright merge-reports --reporter=html ./all-blob-reports

# Multiple formats
npx playwright merge-reports --reporter=html,json,junit ./all-blob-reports

# Custom output location
PLAYWRIGHT_HTML_REPORT=merged-report npx playwright merge-reports --reporter=html ./all-blob-reports

GitHub Actions merge job:

merge-reports:
  if: ${{ !cancelled() }}
  needs: test
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - run: npm ci

    - uses: actions/download-artifact@v4
      with:
        path: all-blob-reports
        pattern: blob-report-*
        merge-multiple: true

    - run: npx playwright merge-reports --reporter=html ./all-blob-reports

    - uses: actions/upload-artifact@v4
      with:
        name: playwright-report
        path: playwright-report/
        retention-days: 14

Worker-Scoped Fixtures

Use when: Expensive resources (DB connections, auth tokens) should be created once per worker, not per test.

// fixtures.ts
import { test as base } from "@playwright/test";

type WorkerFixtures = {
  dbClient: DatabaseClient;
  apiToken: string;
};

export const test = base.extend<{}, WorkerFixtures>({
  dbClient: [
    async ({}, use) => {
      const client = await DatabaseClient.connect(process.env.DB_URL!);
      await use(client);
      await client.disconnect();
    },
    { scope: "worker" },
  ],

  apiToken: [
    async ({}, use, workerInfo) => {
      const res = await fetch(`${process.env.API_URL}/auth`, {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({
          user: `test-user-${workerInfo.workerIndex}`,
          password: process.env.TEST_PASSWORD,
        }),
      });
      const { token } = await res.json();
      await use(token);
    },
    { scope: "worker" },
  ],
});

export { expect } from "@playwright/test";

Test Isolation for Parallelism

Use when: Preparing tests to run without interference.

Each test must create its own state. No test should depend on or modify shared state.

// BAD: Shared user causes race conditions
test("edit settings", async ({ page }) => {
  await page.goto("/users/test-user/settings");
  await page.getByLabel("Email").fill("new@example.com");
  await page.getByRole("button", { name: "Save" }).click();
});

// GOOD: Unique user per test
test("edit settings", async ({ page, request }) => {
  const res = await request.post("/api/users", {
    data: { name: `user-${Date.now()}`, email: `${Date.now()}@test.com` },
  });
  const user = await res.json();

  await page.goto(`/users/${user.id}/settings`);
  await page.getByLabel("Email").fill("updated@example.com");
  await page.getByRole("button", { name: "Save" }).click();
  await expect(page.getByLabel("Email")).toHaveValue("updated@example.com");

  await request.delete(`/api/users/${user.id}`);
});

Using testInfo for unique identifiers:

import { test, expect } from "@playwright/test";

test("submit order", async ({ page }, testInfo) => {
  const orderId = `order-${testInfo.workerIndex}-${Date.now()}`;
  await page.goto(`/orders/new?ref=${orderId}`);
  // ...
});

Dynamic Shard Count

Use when: Automatically adjusting shards based on test count.

# .github/workflows/playwright.yml
jobs:
  calculate-shards:
    runs-on: ubuntu-latest
    outputs:
      shard-count: ${{ steps.calc.outputs.count }}
      shard-matrix: ${{ steps.calc.outputs.matrix }}
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - id: calc
        run: |
          TEST_COUNT=$(npx playwright test --list --reporter=json 2>/dev/null | node -e "
            const data = require('fs').readFileSync('/dev/stdin', 'utf8');
            const parsed = JSON.parse(data);
            console.log(parsed.suites?.reduce((acc, s) => acc + (s.specs?.length || 0), 0) || 0);
          ")
          # 1 shard per 20 tests, min 1, max 8
          SHARDS=$(( (TEST_COUNT + 19) / 20 ))
          SHARDS=$(( SHARDS > 8 ? 8 : SHARDS ))
          SHARDS=$(( SHARDS < 1 ? 1 : SHARDS ))
          MATRIX="["
          for i in $(seq 1 $SHARDS); do
            [ $i -gt 1 ] && MATRIX+=","
            MATRIX+="\"$i/$SHARDS\""
          done
          MATRIX+="]"
          echo "count=$SHARDS" >> $GITHUB_OUTPUT
          echo "matrix=$MATRIX" >> $GITHUB_OUTPUT

  test:
    needs: calculate-shards
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: ${{ fromJson(needs.calculate-shards.outputs.shard-matrix) }}
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test --shard=${{ matrix.shard }}

Decision Guide

Scenario	Workers	Shards	Reason
< 50 tests, < 5 min	Auto (default)	None	No optimization needed
50-200 tests, 5-15 min	'50%' in CI	2-4	Balance speed and cost
200+ tests, > 15 min	'50%' in CI	4-8	Keep feedback under 10 min
Flaky due to resource contention	Reduce to 2	Keep	Less CPU/memory pressure
Tests modify shared database	1 or isolate	Useful	Sharding splits files; workers run them
CI has limited resources	1 or '25%'	More	Compensate with more machines

Aspect	Workers (in-process)	Shards (across machines)
What it splits	Tests across CPU cores	Test files across CI jobs
Controlled by	Config or --workers CLI	--shard=X/Y CLI flag
Shares memory	Yes	No
Report merging	Not needed	Required (merge-reports)
Cost	Free (same machine)	More CI minutes

Anti-Patterns

Anti-Pattern	Problem	Solution
fullyParallel: false without reason	Tests in files run serially	Set fullyParallel: true unless tests need serial
workers: 1 in CI "for safety"	Negates parallelism	Fix isolation issues; use workers: '50%'
Hardcoded shared user account	Race conditions in parallel runs	Each test creates unique data
Sharding without blob reporter	Each shard produces separate HTML report	Configure reporter: [['blob']] for CI
Sharding with 3 tests	Setup overhead exceeds time saved	Only shard when suite > 5 minutes
test.describe.serial() everywhere	Kills parallelism, creates dependencies	Use only when tests genuinely need prior state
Workers > CPU cores	Context switching overhead	Use '50%' or auto-detect
Missing fail-fast: false in CI matrix	One shard failure cancels others	Always set fail-fast: false for sharded strategies

Troubleshooting

Tests pass solo but fail together

• Shared state. Make test data unique:

  test("create item", async ({ request }, ti) => {
    await request.post("/api/items", {
      data: { name: `Item-${ti.workerIndex}-${Date.now()}` },
    });
  });
  

"No tests found" in some shards

• Too many shards. Never exceed file count:

  npx playwright test --shard=1/10   # ok if 10 files
  npx playwright test --shard=1/20   # too many, some shards empty
  

Merged report missing results

• Blob reports collide. Use unique names:

  # Each shard
  - uses: actions/upload-artifact@v4
    with:
      name: blob-report-${{ strategy.job-index }}
      path: blob-report/
  # Merge step
  - uses: actions/download-artifact@v4
    with:
      pattern: blob-report-*
      merge-multiple: true
      path: all-blob-reports
  

Worker-scoped fixture not working

• Missing { scope: 'worker' }. Fix:

  export const test = base.extend({
    resource: [
      async ({}, use) => {
        const r = await Resource.create();
        await use(r);
        await r.destroy();
      },
      { scope: "worker" },
    ],
  });
  

More workers = Slower

• Too many workers thrash. Limit in CI:

  export default defineConfig({
    workers: process.env.CI ? 2 : undefined,
  });