The free hosted runners (windows-latest worst) are content-process unstable
under a heavy headless interaction sequence: clicks/moves cascade into
context-destroyed / selector-timeout / eval-CSP, even across 3 retries, even on
linux-arm64. That's an environment limit, not a binary defect (the binaries
drive 20/20 locally and the stable legs pass).
So: SMOKE (launch + http page + UA + webdriver + DOM read) runs on all 5 legs —
the firefox-8/juggler catcher, robust everywhere. FULL (+ mouse/keyboard/canvas/
navsurface, the firefox-2 class) runs only on linux-x86_64; the interaction code
is platform-identical JS (omni.ja), so one reliable full run covers every
platform, and win interaction stays covered by local pre-release testing.
An adversarial audit of the pipeline found the drive gate only did goto+evaluate,
so several historically-shipped breakages would still pass it green:
- firefox-2 (jugglerSendMouseEvent missing) — no mouse/keyboard was tested
- issue #20 (cross-origin iframe content_frame() None) — no iframe was tested
- canvas non-determinism (stealth seed) and headless navigator tells
ci_drive_gate.py now clicks a button, moves the mouse, types into an input,
reaches into an iframe, checks an identical canvas draw is byte-stable, and
checks navigator.languages/plugins — all offline (data: URLs), GPU-free, no
proxy. Validated against the real build.
Pipeline hardening from the same audit:
- Windows: stop swallowing `mach package` failure and never fall back to the
dev tree dist/bin (that masked the firefox-7/8 packaging bugs)
- macOS: plutil -lint Info.plist + required-key checks (a malformed plist ships
fine through a headless drive but Finder calls the .app "damaged")
- publish: assert all 5 archives present + fail_on_unmatched_files (no silent
partial release if a build leg drops out)
gh release download 404s ("release not found") on a draft tag when the token
is contents:read — GitHub only shows drafts to tokens with push access. The
workflow still only reads assets; the scope bump is purely for draft visibility.
The old gate ran firefox --headless --screenshot, which renders fine even
when the juggler automation layer is missing from the package — so a binary
Playwright can't actually drive (firefox-8) passed and shipped broken.
Replace it with a real drive gate: a 5-leg matrix that launches each binary
over the juggler pipe on its native runner, loads a page, and round-trips JS
(also asserts navigator.webdriver stays hidden). Headless and no screenshot,
so it stays GPU-free on the hosted runners and needs no proxy or secrets.
Same logic is reusable standalone via verify-assets.yml to drive-test an
existing release's assets without a rebuild.
release.yml builds linux-x64/arm64 + win-x64 (cross) on free Linux runners and
macos-arm64/x64 on native Mac runners; packages per the wrapper contract
(juggler-gated so binaries are Playwright-drivable, issue-#14 symlink-safe via
cp -aL), validate_release.py gate, ad-hoc macOS codesign, DRAFT publish.
constants.py: arm64 + darwin ARCHIVE_NAME + BINARY_ENTRY_REL (Firefox.app).
download.py: macOS post-extract xattr quarantine strip.
BINARY_VERSION unchanged (firefox-8); the juggler-fixed firefox-9 is a separate
release cut + pin bump.
Unit sentinels (run in CI) assert the rules a real WebRTC profile must meet:
host candidate is mDNS .local (never a raw LAN IP), the synthetic srflx carries
the egress IP with a genuine nICEr priority (rejecting the old local_pref
0xFFFF) and a foundation distinct from the host one, and CreepJS's resolver
returns the egress (and reads a host-only SDP as blocked).
e2e tests launch the binary and check the live gather. "Behind a proxy" is
reproduced without any external proxy: an in-process SOCKS5 server relays TCP
CONNECT but refuses UDP ASSOCIATE (a TCP-only residential proxy), and the
egress IP is injected as an RFC 5737 TEST-NET address.
test_not_blocked_behind_tcp_only_socks guards the gather-fails-behind-proxy bug.
webrtc-e2e.yml runs the e2e on demand (needs a binary that carries the fixes).
firefox.exe --version on Windows prints the version string but may
return non-zero exit code (sub-process fork quirk). The previous check
treated that as a launch failure, producing a false-positive failure
across the whole matrix while the binary actually launched cleanly.
Switch to matching the printed output instead, so we only fail when the
binary really can't start.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Triggered by issue #22 (firefox-7 SxS mismatch reported on Win11 26200
by jannusdorfer-create). Verifies the shipped binary launches cleanly
on every Windows runner GitHub offers: windows-2022, windows-2025,
windows-latest, across Python 3.11 / 3.12 / 3.13.
Each cell does the reporter's exact flow: fresh checkout, pip install
from source, python -m invisible_playwright fetch, then runs the
InvisiblePlaywright(seed=9128) snippet.
If all cells pass the bug is environment-specific to the reporter
(corporate edition, EDR, GPO). If any cell fails the same way we ship
a sidecar mozglue.manifest in the next release.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
