apunkt/webclaw
1
0
Fork 0
mirror of https://github.com/0xMassi/webclaw.git synced 2026-06-10 22:45:13 +02:00
Code Issues Projects Releases Packages Wiki Activity
webclaw/Dockerfile.ci

36 lines
1.7 KiB
Text
Raw Permalink Normal View History

ci: use pre-built binaries for Docker instead of QEMU cross-compilation QEMU arm64 Rust builds took 60+ min and timed out in CI. Now the Docker job downloads the pre-built release binaries and packages them directly. - Dockerfile.ci: slim image for CI (downloads pre-built binaries) - Dockerfile: full source build for local dev (unchanged build stage) - Both use ubuntu:24.04 (GLIBC 2.39 matches CI build environment) - Multi-arch manifest combines amd64 + arm64 images Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 20:32:50 +01:00
# Slim runtime image — uses pre-built binaries from the release.
# The full Dockerfile (multi-stage Rust build) is for local development.
# CI uses this to avoid 60+ min QEMU cross-compilation.
FROM ubuntu:24.04
feat(core): endpoints module for API surface extraction from HTML and JS (#47) * feat(core): endpoints module — extract API surface from HTML + JS bundles * fix(docker): source CA bundle from distroless instead of apt (fixes arm64 release build) * fix(test): serialize env-mutating CloudClient tests to stop flaky CI * feat(core): filter endpoint-extractor noise (invalid hosts, schema domains, bare paths)
2026-05-19 19:05:16 +02:00
# CA bundle copied from a reliable multi-arch image instead of apt-installing
# from ports.ubuntu.com — Canonical's arm64 ports mirror is unreachable from
# CI runners and breaks the multi-arch release build. No build-time network.
COPY --from=gcr.io/distroless/static-debian12 /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
ci: use pre-built binaries for Docker instead of QEMU cross-compilation QEMU arm64 Rust builds took 60+ min and timed out in CI. Now the Docker job downloads the pre-built release binaries and packages them directly. - Dockerfile.ci: slim image for CI (downloads pre-built binaries) - Dockerfile: full source build for local dev (unchanged build stage) - Both use ubuntu:24.04 (GLIBC 2.39 matches CI build environment) - Multi-arch manifest combines amd64 + arm64 images Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 20:32:50 +01:00
ci(release): build the Docker image in one multi-platform pass The per-arch build + 'imagetools create' combine failed at the manifest step with 'v0.6.9-arm64: not found' — buildx's default provenance/SBOM attestations turn each per-arch tag into an index, and assembling them races GHCR's read-after-write. Replace it with a single 'docker buildx build --platform linux/amd64,linux/arm64 --push' (attestations off) so one manifest list is pushed atomically. Dockerfile.ci now selects binaries by TARGETARCH. Adds a workflow_dispatch path to re-publish an existing tag's image without rebuilding binaries or bumping the version.
2026-06-10 15:54:28 +02:00
# TARGETARCH (amd64 / arm64) is provided automatically by buildx for each
# target platform, so one multi-platform build copies the matching binaries.
# The release workflow stages them in binaries-amd64 / binaries-arm64.
ARG TARGETARCH
COPY binaries-${TARGETARCH}/webclaw /usr/local/bin/webclaw
COPY binaries-${TARGETARCH}/webclaw-mcp /usr/local/bin/webclaw-mcp
COPY binaries-${TARGETARCH}/webclaw-server /usr/local/bin/webclaw-server
feat(server): add OSS webclaw-server REST API binary (closes #29) Self-hosters hitting docs/self-hosting were promised three binaries but the OSS Docker image only shipped two. webclaw-server lived in the closed-source hosted-platform repo, which couldn't be opened. This adds a minimal axum REST API in the OSS repo so self-hosting actually works without pretending to ship the cloud platform. Crate at crates/webclaw-server/. Stateless, no database, no job queue, single binary. Endpoints: GET /health, POST /v1/{scrape, crawl, map, batch, extract, summarize, diff, brand}. JSON shapes mirror api.webclaw.io for the endpoints OSS can support, so swapping between self-hosted and hosted is a base-URL change. Auth: optional bearer token via WEBCLAW_API_KEY / --api-key. Comparison is constant-time (subtle::ConstantTimeEq). Open mode (no key) is allowed and binds 127.0.0.1 by default; the Docker image flips WEBCLAW_HOST=0.0.0.0 so the container is reachable out of the box. Hard caps to keep naive callers from OOMing the process: crawl capped at 500 pages synchronously, batch capped at 100 URLs / 20 concurrent. For unbounded crawls or anti-bot bypass the docs point users at the hosted API. Dockerfile + Dockerfile.ci updated to copy webclaw-server into /usr/local/bin and EXPOSE 3000. Workspace version bumped to 0.4.0 (new public binary).
2026-04-22 12:25:11 +02:00
# Default REST API port when running `webclaw-server` inside the container.
EXPOSE 3000
# Container default: bind all interfaces so `-p 3000:3000` works. The
# binary itself defaults to 127.0.0.1; flipping here keeps the CLI safe on
# a laptop but makes the container reachable out of the box.
ENV WEBCLAW_HOST=0.0.0.0
ci: use pre-built binaries for Docker instead of QEMU cross-compilation QEMU arm64 Rust builds took 60+ min and timed out in CI. Now the Docker job downloads the pre-built release binaries and packages them directly. - Dockerfile.ci: slim image for CI (downloads pre-built binaries) - Dockerfile: full source build for local dev (unchanged build stage) - Both use ubuntu:24.04 (GLIBC 2.39 matches CI build environment) - Multi-arch manifest combines amd64 + arm64 images Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 20:32:50 +01:00
fix(docker): entrypoint shim so child images with custom CMD work (#28) v0.3.13 switched ENTRYPOINT to ["webclaw"] to make `docker run IMAGE https://example.com` work. That broke a different use case: downstream Dockerfiles that `FROM ghcr.io/0xmassi/webclaw` and set their own CMD ["./setup.sh"] — the child's ./setup.sh becomes arg to webclaw, which tries to fetch it as a URL and fails: fetch error: request failed: error sending request for uri (https://./setup.sh): client error (Connect) Both Dockerfile and Dockerfile.ci now use docker-entrypoint.sh which: - forwards flags (-*) and URLs (http://, https://) to `webclaw` - exec's anything else directly Test matrix (all pass locally): docker run IMAGE https://example.com → webclaw scrape ok docker run IMAGE --help → webclaw --help ok docker run IMAGE → default CMD, --help docker run IMAGE bash → bash runs FROM IMAGE + CMD ["./setup.sh"] → setup.sh runs, webclaw available Default CMD is ["webclaw", "--help"] so bare `docker run IMAGE` still prints help. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 15:57:47 +02:00
# Entrypoint shim: forwards webclaw args/URL to the binary, but exec's other
# commands directly so this image can be used as a FROM base with custom CMD.
ci(release): build the Docker image in one multi-platform pass The per-arch build + 'imagetools create' combine failed at the manifest step with 'v0.6.9-arm64: not found' — buildx's default provenance/SBOM attestations turn each per-arch tag into an index, and assembling them races GHCR's read-after-write. Replace it with a single 'docker buildx build --platform linux/amd64,linux/arm64 --push' (attestations off) so one manifest list is pushed atomically. Dockerfile.ci now selects binaries by TARGETARCH. Adds a workflow_dispatch path to re-publish an existing tag's image without rebuilding binaries or bumping the version.
2026-06-10 15:54:28 +02:00
# `--chmod` sets the bit at copy time so the build needs no in-container `RUN`
# (and thus no QEMU emulation for the arm64 platform).
COPY --chmod=755 docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
fix(docker): entrypoint shim so child images with custom CMD work (#28) v0.3.13 switched ENTRYPOINT to ["webclaw"] to make `docker run IMAGE https://example.com` work. That broke a different use case: downstream Dockerfiles that `FROM ghcr.io/0xmassi/webclaw` and set their own CMD ["./setup.sh"] — the child's ./setup.sh becomes arg to webclaw, which tries to fetch it as a URL and fails: fetch error: request failed: error sending request for uri (https://./setup.sh): client error (Connect) Both Dockerfile and Dockerfile.ci now use docker-entrypoint.sh which: - forwards flags (-*) and URLs (http://, https://) to `webclaw` - exec's anything else directly Test matrix (all pass locally): docker run IMAGE https://example.com → webclaw scrape ok docker run IMAGE --help → webclaw --help ok docker run IMAGE → default CMD, --help docker run IMAGE bash → bash runs FROM IMAGE + CMD ["./setup.sh"] → setup.sh runs, webclaw available Default CMD is ["webclaw", "--help"] so bare `docker run IMAGE` still prints help. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 15:57:47 +02:00
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["webclaw", "--help"]
Reference in a new issue Copy permalink