apunkt/webclaw
1
0
Fork 0
mirror of https://github.com/0xMassi/webclaw.git synced 2026-06-21 02:28:27 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat(server): add OSS webclaw-server REST API binary (closes #29)

Browse source 
Self-hosters hitting docs/self-hosting were promised three binaries
but the OSS Docker image only shipped two. webclaw-server lived in
the closed-source hosted-platform repo, which couldn't be opened. This
adds a minimal axum REST API in the OSS repo so self-hosting actually
works without pretending to ship the cloud platform.

Crate at crates/webclaw-server/. Stateless, no database, no job queue,
single binary. Endpoints: GET /health, POST /v1/{scrape, crawl, map,
batch, extract, summarize, diff, brand}. JSON shapes mirror
api.webclaw.io for the endpoints OSS can support, so swapping between
self-hosted and hosted is a base-URL change.

Auth: optional bearer token via WEBCLAW_API_KEY / --api-key. Comparison
is constant-time (subtle::ConstantTimeEq). Open mode (no key) is
allowed and binds 127.0.0.1 by default; the Docker image flips
WEBCLAW_HOST=0.0.0.0 so the container is reachable out of the box.

Hard caps to keep naive callers from OOMing the process: crawl capped
at 500 pages synchronously, batch capped at 100 URLs / 20 concurrent.
For unbounded crawls or anti-bot bypass the docs point users at the
hosted API.

Dockerfile + Dockerfile.ci updated to copy webclaw-server into
/usr/local/bin and EXPOSE 3000. Workspace version bumped to 0.4.0
(new public binary).
This commit is contained in:
Valerio 2026-04-22 12:25:11 +02:00
parent b4bfff120e
commit 2ba682adf3
20 changed files with 1116 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

9
Dockerfile.ci
View file

@ -12,6 +12,15 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
ARG BINARY_DIR
COPY ${BINARY_DIR}/webclaw /usr/local/bin/webclaw
COPY ${BINARY_DIR}/webclaw-mcp /usr/local/bin/webclaw-mcp
COPY ${BINARY_DIR}/webclaw-server /usr/local/bin/webclaw-server
# Default REST API port when running `webclaw-server` inside the container.
EXPOSE 3000
# Container default: bind all interfaces so `-p 3000:3000` works. The
# binary itself defaults to 127.0.0.1; flipping here keeps the CLI safe on
# a laptop but makes the container reachable out of the box.
ENV WEBCLAW_HOST=0.0.0.0
# Entrypoint shim: forwards webclaw args/URL to the binary, but exec's other
# commands directly so this image can be used as a FROM base with custom CMD.