Re-running the enriched gate on the real binaries exposed a ~1-in-5 flake: the
iframe probe (nested data: / srcdoc) re-navigates, so Juggler's frame id changes
mid-check ("execution context destroyed" / "Frame was detached"). set_content
isn't an option either — this build rejects its document.write ("operation is
insecure").
Drop the iframe from the gate: a same-origin srcdoc iframe is a weak proxy for
the cross-origin issue #20 anyway. The page is now a plain subframe-free
goto(data:), and the mouse/keyboard/canvas-determinism/navigator-surface checks
(the firefox-2 class + stealth smoke) stay. 20/20 clean locally. The faithful
#20 sentinel (tests/test_cross_origin_iframe.py, two localhost origins) should
be wired as its own e2e gate job.
An adversarial audit of the pipeline found the drive gate only did goto+evaluate,
so several historically-shipped breakages would still pass it green:
- firefox-2 (jugglerSendMouseEvent missing) — no mouse/keyboard was tested
- issue #20 (cross-origin iframe content_frame() None) — no iframe was tested
- canvas non-determinism (stealth seed) and headless navigator tells
ci_drive_gate.py now clicks a button, moves the mouse, types into an input,
reaches into an iframe, checks an identical canvas draw is byte-stable, and
checks navigator.languages/plugins — all offline (data: URLs), GPU-free, no
proxy. Validated against the real build.
Pipeline hardening from the same audit:
- Windows: stop swallowing `mach package` failure and never fall back to the
dev tree dist/bin (that masked the firefox-7/8 packaging bugs)
- macOS: plutil -lint Info.plist + required-key checks (a malformed plist ships
fine through a headless drive but Finder calls the .app "damaged")
- publish: assert all 5 archives present + fail_on_unmatched_files (no silent
partial release if a build leg drops out)
The old gate ran firefox --headless --screenshot, which renders fine even
when the juggler automation layer is missing from the package — so a binary
Playwright can't actually drive (firefox-8) passed and shipped broken.
Replace it with a real drive gate: a 5-leg matrix that launches each binary
over the juggler pipe on its native runner, loads a page, and round-trips JS
(also asserts navigator.webdriver stays hidden). Headless and no screenshot,
so it stays GPU-free on the hosted runners and needs no proxy or secrets.
Same logic is reusable standalone via verify-assets.yml to drive-test an
existing release's assets without a rebuild.
