B6: pin every third-party action in the build/publish path to an immutable
commit SHA (a retagged actions/checkout or action-gh-release would otherwise
inject code into the binary users download). The other workflows (tests, webrtc,
launch-matrix) handle no secrets, so they're left on tags.
B4: the playwright pin lived in two workflow files with no shared source. Move
it to scripts/playwright_pin.txt that both read, so they can't drift. The drive
gate already ENFORCES playwright<->juggler compatibility (an incompatible pin
fails the launch/drive and nothing publishes); the file is the single bump point
when the juggler is re-synced.
The free hosted runners (windows-latest worst) are content-process unstable
under a heavy headless interaction sequence: clicks/moves cascade into
context-destroyed / selector-timeout / eval-CSP, even across 3 retries, even on
linux-arm64. That's an environment limit, not a binary defect (the binaries
drive 20/20 locally and the stable legs pass).
So: SMOKE (launch + http page + UA + webdriver + DOM read) runs on all 5 legs —
the firefox-8/juggler catcher, robust everywhere. FULL (+ mouse/keyboard/canvas/
navsurface, the firefox-2 class) runs only on linux-x86_64; the interaction code
is platform-identical JS (omni.ja), so one reliable full run covers every
platform, and win interaction stays covered by local pre-release testing.
An adversarial audit of the pipeline found the drive gate only did goto+evaluate,
so several historically-shipped breakages would still pass it green:
- firefox-2 (jugglerSendMouseEvent missing) — no mouse/keyboard was tested
- issue #20 (cross-origin iframe content_frame() None) — no iframe was tested
- canvas non-determinism (stealth seed) and headless navigator tells
ci_drive_gate.py now clicks a button, moves the mouse, types into an input,
reaches into an iframe, checks an identical canvas draw is byte-stable, and
checks navigator.languages/plugins — all offline (data: URLs), GPU-free, no
proxy. Validated against the real build.
Pipeline hardening from the same audit:
- Windows: stop swallowing `mach package` failure and never fall back to the
dev tree dist/bin (that masked the firefox-7/8 packaging bugs)
- macOS: plutil -lint Info.plist + required-key checks (a malformed plist ships
fine through a headless drive but Finder calls the .app "damaged")
- publish: assert all 5 archives present + fail_on_unmatched_files (no silent
partial release if a build leg drops out)
The old gate ran firefox --headless --screenshot, which renders fine even
when the juggler automation layer is missing from the package — so a binary
Playwright can't actually drive (firefox-8) passed and shipped broken.
Replace it with a real drive gate: a 5-leg matrix that launches each binary
over the juggler pipe on its native runner, loads a page, and round-trips JS
(also asserts navigator.webdriver stays hidden). Headless and no screenshot,
so it stays GPU-free on the hosted runners and needs no proxy or secrets.
Same logic is reusable standalone via verify-assets.yml to drive-test an
existing release's assets without a rebuild.
release.yml builds linux-x64/arm64 + win-x64 (cross) on free Linux runners and
macos-arm64/x64 on native Mac runners; packages per the wrapper contract
(juggler-gated so binaries are Playwright-drivable, issue-#14 symlink-safe via
cp -aL), validate_release.py gate, ad-hoc macOS codesign, DRAFT publish.
constants.py: arm64 + darwin ARCHIVE_NAME + BINARY_ENTRY_REL (Firefox.app).
download.py: macOS post-extract xattr quarantine strip.
BINARY_VERSION unchanged (firefox-8); the juggler-fixed firefox-9 is a separate
release cut + pin bump.
