B6: pin every third-party action in the build/publish path to an immutable
commit SHA (a retagged actions/checkout or action-gh-release would otherwise
inject code into the binary users download). The other workflows (tests, webrtc,
launch-matrix) handle no secrets, so they're left on tags.
B4: the playwright pin lived in two workflow files with no shared source. Move
it to scripts/playwright_pin.txt that both read, so they can't drift. The drive
gate already ENFORCES playwright<->juggler compatibility (an incompatible pin
fails the launch/drive and nothing publishes); the file is the single bump point
when the juggler is re-synced.
The 138 @pytest.mark.e2e tests were doubly inactive: deselected by addopts AND
skipped without a cached binary — and 3 of the 6 per-file firefox_binary
fixtures silently ignored INVPW_BINARY_PATH, so they'd test whatever was cached
even when you pointed the suite elsewhere (a false-confidence trap).
- Centralize firefox_binary into conftest.py (env INVPW_BINARY_PATH → cache →
skip); delete the 6 duplicates. Unify test_webrtc_realness onto the same env.
- scripts/run_e2e.py: one command that runs ALL e2e against a chosen binary,
with reruns so an under-load interaction flake (dblclick/hover pass 3/3 in
isolation) self-heals while a real break fails every attempt. The webrtc e2e
fake a TCP-only SOCKS locally, so the suite is offline. This is the MANDATORY
pre-release browser gate (local — hosted runners are too interaction-flaky).
- Running the suite against firefox-9 surfaced a real gap: `invisible_playwright
fetch --force` was unrecognized (the subparser took no args) though the e2e
test + docstring expect it. Implement it: drop the cached version dir, refetch.
- Add pytest-rerunfailures + playwright to the dev extras.
Baseline against firefox-9: 136 passed, 1 skipped (linux_only on win host),
1 was the --force gap now fixed.
The free hosted runners (windows-latest worst) are content-process unstable
under a heavy headless interaction sequence: clicks/moves cascade into
context-destroyed / selector-timeout / eval-CSP, even across 3 retries, even on
linux-arm64. That's an environment limit, not a binary defect (the binaries
drive 20/20 locally and the stable legs pass).
So: SMOKE (launch + http page + UA + webdriver + DOM read) runs on all 5 legs —
the firefox-8/juggler catcher, robust everywhere. FULL (+ mouse/keyboard/canvas/
navsurface, the firefox-2 class) runs only on linux-x86_64; the interaction code
is platform-identical JS (omni.ja), so one reliable full run covers every
platform, and win interaction stays covered by local pre-release testing.
windows-latest headless kept flaking on special-scheme pages: data: URLs get
re-normalized (re-nav), about:blank + redundant goto destroys the context, and
both can carry a CSP that blocks eval(). Serve the test page over a real
http://127.0.0.1 instead — none of those quirks, and it adds real-navigation
coverage (await a response). Evaluates stay arrow-functions (no eval), listeners
are inline-script (no on* attrs), and transient context-destroyed/detached/timeout
gets up to 2 retries. A genuinely broken binary fails all 3 attempts.
windows-latest failed both attempts with "Page.evaluate: call to eval() blocked
by CSP". Bare-string evaluates (page.evaluate("navigator.userAgent")) make
Playwright fall back to eval(), which a page CSP blocks. Pass arrow functions
instead (called via callFunction, never eval'd), and wire the click listener
with addEventListener instead of an inline onclick (also CSP-sensitive). Both
are Playwright best practice and platform-agnostic.
linux+macOS drive went green but windows-latest kept throwing "execution
context destroyed by navigation" at a wandering evaluate (passed 20/20 win-local,
no browser crash logged). Root cause: the unencoded data: URL gets re-normalized
(re-navigated to its percent-encoded form) by Firefox; the slower win runner
races that re-nav against the evaluates. about:blank is canonical and never
re-navigates, so the DOM is now built there via innerHTML. Also add one logged
retry on transient context-destroyed/detached (a broken binary fails both).
Re-running the enriched gate on the real binaries exposed a ~1-in-5 flake: the
iframe probe (nested data: / srcdoc) re-navigates, so Juggler's frame id changes
mid-check ("execution context destroyed" / "Frame was detached"). set_content
isn't an option either — this build rejects its document.write ("operation is
insecure").
Drop the iframe from the gate: a same-origin srcdoc iframe is a weak proxy for
the cross-origin issue #20 anyway. The page is now a plain subframe-free
goto(data:), and the mouse/keyboard/canvas-determinism/navigator-surface checks
(the firefox-2 class + stealth smoke) stay. 20/20 clean locally. The faithful
#20 sentinel (tests/test_cross_origin_iframe.py, two localhost origins) should
be wired as its own e2e gate job.
An adversarial audit of the pipeline found the drive gate only did goto+evaluate,
so several historically-shipped breakages would still pass it green:
- firefox-2 (jugglerSendMouseEvent missing) — no mouse/keyboard was tested
- issue #20 (cross-origin iframe content_frame() None) — no iframe was tested
- canvas non-determinism (stealth seed) and headless navigator tells
ci_drive_gate.py now clicks a button, moves the mouse, types into an input,
reaches into an iframe, checks an identical canvas draw is byte-stable, and
checks navigator.languages/plugins — all offline (data: URLs), GPU-free, no
proxy. Validated against the real build.
Pipeline hardening from the same audit:
- Windows: stop swallowing `mach package` failure and never fall back to the
dev tree dist/bin (that masked the firefox-7/8 packaging bugs)
- macOS: plutil -lint Info.plist + required-key checks (a malformed plist ships
fine through a headless drive but Finder calls the .app "damaged")
- publish: assert all 5 archives present + fail_on_unmatched_files (no silent
partial release if a build leg drops out)
The old gate ran firefox --headless --screenshot, which renders fine even
when the juggler automation layer is missing from the package — so a binary
Playwright can't actually drive (firefox-8) passed and shipped broken.
Replace it with a real drive gate: a 5-leg matrix that launches each binary
over the juggler pipe on its native runner, loads a page, and round-trips JS
(also asserts navigator.webdriver stays hidden). Headless and no screenshot,
so it stays GPU-free on the hosted runners and needs no proxy or secrets.
Same logic is reusable standalone via verify-assets.yml to drive-test an
existing release's assets without a rebuild.
invisible-playwright: a patched Firefox 150.0.1 for browser-fingerprint
stealth, shipped as a Playwright-compatible Python wrapper.
* Sync + async InvisiblePlaywright launcher (firefox_user_prefs, virtual
desktop on Windows, SOCKS5 auth via patched nsProtocolProxyService)
* fpforge: Bayesian fingerprint sampler over GPU / audio / fonts /
screen / ~400 other navigator fields
* WebRTC stealth: srflx address swap, synthetic srflx fallback,
private-LAN host candidates. No real public IP leak via STUN.
* GPU sandbox fix for FF150 alt-desktop regression
* Bezier-curve mouse motion baked into Juggler
Targets Windows x86_64 + Linux x86_64. Binary fetched on first run from
GitHub Release "firefox-1".
