Commit graph

27 commits

Author SHA1 Message Date
Valerio
d5d58ab612 ci(release): don't let prerelease tags clobber :latest / Homebrew / stable
A v*-rc* tag triggered the same release pipeline as a stable tag, which
would publish a normal release, overwrite ghcr.io/0xmassi/webclaw:latest,
and repoint the Homebrew formula at the rc — shipping a prerelease to every
stable Docker/brew user. Guard all three on the SemVer prerelease hyphen:

- release: add --prerelease for hyphenated tags
- docker: still push :${tag} (testable rc image) but only move :latest for stable
- homebrew: skip entirely for prereleases

Lets us cut rc tags (e.g. to validate the new musl build job) without
touching stable users.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 14:45:49 +02:00
Valerio
77fe3b52e2 ci(release): add musl static Linux builds (glibc-independent)
The gnu Linux binaries are glibc-floored (2.35 after #74), so they still
won't run on Amazon Linux 2023 / RHEL 9 (glibc 2.34), Alpine, or anything
older. Add fully static musl builds that run on ANY Linux regardless of
glibc.

Adds x86_64-unknown-linux-musl and aarch64-unknown-linux-musl to the build
matrix, built with cargo-zigbuild (zig as the C/C++ cross-compiler for
BoringSSL). Build scripts (bindgen) run as the glibc host so libclang loads,
and the linked output is fully static. A native Alpine build can't do this —
its static build scripts can't dlopen libclang.

musl assets ship ALONGSIDE the gnu ones (gnu stays default; musl is the
runs-anywhere fallback). The release job globs *.tar.gz, so the new assets
are checksummed + uploaded automatically; the docker/homebrew jobs enumerate
gnu targets explicitly and are unaffected.

Validated in Docker: cargo-zigbuild produced a fully static aarch64-musl
webclaw-mcp (ldd: not a dynamic executable) that answered an MCP handshake on
Alpine, Debian 11 (glibc 2.31), Debian 12, Amazon Linux 2023 (2.34), and
Ubuntu 24.04 — everywhere, including where the gnu builds fail.

Closes #73

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 13:49:40 +02:00
Valerio
a9567aa661 ci(release): build Linux binaries on ubuntu-22.04 for older-glibc support
Linux release binaries were built on ubuntu-latest (now Ubuntu 24.04,
glibc 2.39), so they required GLIBC_2.38 and failed to start on older LTS
distros (Debian 12 = 2.36, Ubuntu 22.04 = 2.35) with:

  libc.so.6: version `GLIBC_2.38' not found (required by webclaw-mcp)

glibc is forward- but not backward-compatible, so the build host's glibc
sets the floor. Pin both Linux targets to ubuntu-22.04 (glibc 2.35); the
aarch64 cross toolchain tracks the runner distro, so this lowers both.

Note: 2.35 still won't cover Amazon Linux 2023 / RHEL 9 (2.34) — full
coverage needs a musl static build. Tracked in #73.

Refs #73

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 12:33:05 +02:00
Valerio
8015de7db5 ci(release): build the Docker image in one multi-platform pass
The per-arch build + 'imagetools create' combine failed at the manifest
step with 'v0.6.9-arm64: not found' — buildx's default provenance/SBOM
attestations turn each per-arch tag into an index, and assembling them
races GHCR's read-after-write. Replace it with a single
'docker buildx build --platform linux/amd64,linux/arm64 --push'
(attestations off) so one manifest list is pushed atomically. Dockerfile.ci
now selects binaries by TARGETARCH. Adds a workflow_dispatch path to
re-publish an existing tag's image without rebuilding binaries or bumping
the version.
2026-06-10 15:54:28 +02:00
Valerio
2773474984 chore: release v0.6.9
Publish the multi-arch Docker image with Buildx instead of the legacy
docker driver, whose GHCR push intermittently failed with 'unknown
blob'. The manifest list is now assembled registry-side with
`imagetools create`. This also unblocks the Homebrew formula update,
which depends on the Docker job. No library or CLI behavior changes.
2026-06-10 15:30:39 +02:00
Valerio
d0d7b835f2 docs(readme): update banner to new webclaw branding 2026-06-09 18:53:14 +02:00
Valerio
8fe8bcb479 chore(ci): bump actions/checkout and artifact actions to v5
GitHub flagged checkout@v4 / upload-artifact@v4 / download-artifact@v4
as Node.js 20 actions, force-migrated to Node 24 on 2026-06-02. Bump
all nine references to v5 ahead of the deadline. The artifact steps are
v5-compatible: upload uses a unique matrix-target name and the download
step flattens subdirectories with find afterward.
2026-05-21 15:11:29 +02:00
Valerio
be8bcfebd9
fix: harden resource limits, path safety, and WASM build (#46)
Security audit follow-up across the workspace:

- webclaw-core: keep the crate WASM-safe. quickjs/rquickjs is now a
  cfg(not(wasm32)) target dependency and the extraction entry point uses
  a direct call on wasm instead of spawning a thread, so it builds and
  runs on wasm32 with or without default features.
- webclaw-core: bound the structured-data scrubber recursion (depth cap)
  so deeply nested attacker JSON-LD / __NEXT_DATA__ cannot exhaust the
  stack.
- webclaw-fetch: stream the response body with a running ceiling so a
  small highly compressed payload cannot inflate to gigabytes in memory;
  redact user:pass@ from proxy URLs before they reach error strings.
- webclaw-cli: contain output filenames inside the chosen directory
  (reject .. / absolute, drop traversal path segments), run --webhook
  URLs through the public-URL SSRF guard, clamp --watch-interval to >=1s,
  and make research slug truncation char-safe.
- webclaw-mcp: char-safe slug truncation (no multibyte slice panic).
- setup.sh / deploy/hetzner.sh: replace eval on read input with
  printf -v, and mask auth key / API token in console output.
- CI: enforce the wasm32 build invariant for webclaw-core.

Tests added for every behavioral change. Bump to 0.6.3 + CHANGELOG.
2026-05-19 17:03:52 +02:00
Valerio
0ea189c5b2 fix(ci): pass repository to release cli
Some checks failed
CI / Test (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Docs (push) Has been cancelled
2026-05-12 12:28:14 +02:00
Valerio
dbf9ce08a6 fix(ci): scope release workflow token permissions 2026-05-12 12:00:47 +02:00
SURYANSH MISHRA
513b0e493e ci: add Windows release artifacts
Closes #34
2026-05-05 11:38:30 +02:00
Valerio
0e6c7cdc97
Add GitHub Sponsors username to FUNDING.yml
Some checks failed
CI / Test (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Docs (push) Has been cancelled
Updated funding model with GitHub Sponsors username.
2026-04-27 13:18:22 +02:00
Valerio
ccdb6d364b fix(ci): release workflow must include webclaw-server
Some checks are pending
CI / Test (push) Waiting to run
CI / Lint (push) Waiting to run
CI / Docs (push) Waiting to run
v0.4.0 shipped tarballs without the new webclaw-server binary because
the release workflow predates that binary and was hardcoded for two:

- Package step used `cp ... 2>/dev/null || true`, so a missing binary
  was silently skipped instead of failing the job.
- Docker job's download step copied only webclaw + webclaw-mcp into
  the build context, so Dockerfile.ci's COPY webclaw-server step then
  died with 'file not found'.
- Homebrew formula's install block only covered the same two, so brew
  users would have gotten a release with a missing binary.

Three changes:

1. Package step now explicitly copies all three binaries and drops the
   swallow-all-errors pattern. If a future binary gets renamed or
   removed this step screams instead of silently publishing half a
   release.
2. Docker Download step copies webclaw-server alongside the other
   binaries into the build context.
3. Homebrew formula installs webclaw-server too.

v0.4.0 tag + GitHub Release will be deleted and re-pushed on top of
this commit so the canonical v0.4.0 artifacts are complete. No users
affected — download count was 0 on every broken asset.
2026-04-22 12:44:14 +02:00
Valerio
b219fc3648 fix(ci): update all 4 Homebrew checksums after Docker build completes
Previous approach used mislav/bump-homebrew-formula-action which only
updated macOS arm64 SHA. Now downloads all 4 tarballs after Docker
finishes, computes SHAs, and writes the complete formula.

Fixes #12 (brew install checksum mismatch on Linux)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 19:02:27 +02:00
Valerio
b4800e681c ci: fix aarch64 cross-compilation for BoringSSL (boring-sys2)
boring-sys2 builds BoringSSL from C source via cmake. For aarch64 cross-
compilation, we need g++, cmake, and CC/CXX env vars pointing to the
cross-compiler. Also removed stale reqwest_unstable RUSTFLAG.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 18:39:43 +02:00
Valerio
31f35fd895 ci: fix ambiguous reqwest version in dependency sync
Core has reqwest 0.12 (direct) and 0.13 (via webclaw-tls patch).
Disambiguate with version specs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 11:52:35 +02:00
Valerio
4f0c59ac7f ci: replace stale primp check with webclaw-tls dependency sync
Replaces the weekly primp compatibility check (which fails since primp
was removed in v0.3.0) with an automated dependency sync workflow.
Triggered by webclaw-tls pushes via repository_dispatch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 11:39:55 +02:00
Valerio
77e93441c0 fix(ci): add QEMU for arm64 apt-get in Docker build
Plain docker build --platform linux/arm64 on amd64 runner needs QEMU
to execute RUN commands. QEMU is only needed for apt-get (seconds),
not for Rust compilation (the binaries are pre-built).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 20:51:51 +01:00
Valerio
8cf021a00b fix(ci): single Docker job with plain docker build + manifest
buildx creates manifest lists per-platform which can't be nested.
Use plain docker build for each arch then docker manifest create
to combine them. Single job, no matrix, no QEMU.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 20:45:05 +01:00
Valerio
ef120f6ec7 fix(ci): fix Docker binary path extraction from release tarball
Tarball extracts to webclaw-vX.Y.Z-target/ directory, not flat.
Use direct cp instead of find.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 20:39:14 +01:00
Valerio
48a3c45b36 ci: use pre-built binaries for Docker instead of QEMU cross-compilation
QEMU arm64 Rust builds took 60+ min and timed out in CI. Now the Docker
job downloads the pre-built release binaries and packages them directly.

- Dockerfile.ci: slim image for CI (downloads pre-built binaries)
- Dockerfile: full source build for local dev (unchanged build stage)
- Both use ubuntu:24.04 (GLIBC 2.39 matches CI build environment)
- Multi-arch manifest combines amd64 + arm64 images

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 20:32:50 +01:00
Valerio
dfcddd1973 ci: build multi-platform Docker images (amd64 + arm64)
Image only had linux/amd64, failing on Apple Silicon Macs with
"no matching manifest for linux/arm64/v8". Added QEMU + buildx
multi-platform support.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 19:18:49 +01:00
Valerio
8b82ad12d0 ci: add weekly primp compatibility check
Runs every Monday — updates primp to latest, tries to build.
If patches are out of sync the build fails with a clear error
pointing to primp's Cargo.toml for the new patch list.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:45:45 +01:00
Valerio
b039b99858 ci: add Docker image build to release workflow
Pushes ghcr.io/0xmassi/webclaw:latest and :vX.Y.Z on every tagged
release. Uses BuildKit cache for fast rebuilds.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 16:48:44 +01:00
Valerio
bdcd0592dc fix: use macos-latest for release builds 2026-03-24 10:43:36 +01:00
Valerio
d2887cace8 feat: GitHub Release workflow + Homebrew tap + install badges
- .github/workflows/release.yml: builds prebuilt binaries for
  macOS (arm64, x86_64) and Linux (x86_64, aarch64) on tag push.
  Creates GitHub Release with tarballs + SHA256SUMS.
  Auto-updates Homebrew formula via bump-homebrew-formula-action.

- README: added GitHub download count + npm install count badges.
  Install section now lists: Homebrew, prebuilt binaries, cargo
  install --git, Docker, Docker Compose.

- Homebrew tap created at github.com/0xMassi/homebrew-webclaw
  with Formula/webclaw.rb (installs webclaw + webclaw-mcp binaries).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 10:41:37 +01:00
Valerio
c99ec684fa Initial release: webclaw v0.1.0 — web content extraction for LLMs
CLI + MCP server for extracting clean, structured content from any URL.
6 Rust crates, 10 MCP tools, TLS fingerprinting, 5 output formats.

MIT Licensed | https://webclaw.io
2026-03-23 18:31:11 +01:00