mirror of
https://github.com/0xMassi/webclaw.git
synced 2026-06-22 02:38:06 +02:00
2ba682adf3
Self-hosters hitting docs/self-hosting were promised three binaries
but the OSS Docker image only shipped two. webclaw-server lived in
the closed-source hosted-platform repo, which couldn't be opened. This
adds a minimal axum REST API in the OSS repo so self-hosting actually
works without pretending to ship the cloud platform.
Crate at crates/webclaw-server/. Stateless, no database, no job queue,
single binary. Endpoints: GET /health, POST /v1/{scrape, crawl, map,
batch, extract, summarize, diff, brand}. JSON shapes mirror
api.webclaw.io for the endpoints OSS can support, so swapping between
self-hosted and hosted is a base-URL change.
Auth: optional bearer token via WEBCLAW_API_KEY / --api-key. Comparison
is constant-time (subtle::ConstantTimeEq). Open mode (no key) is
allowed and binds 127.0.0.1 by default; the Docker image flips
WEBCLAW_HOST=0.0.0.0 so the container is reachable out of the box.
Hard caps to keep naive callers from OOMing the process: crawl capped
at 500 pages synchronously, batch capped at 100 URLs / 20 concurrent.
For unbounded crawls or anti-bot bypass the docs point users at the
hosted API.
Dockerfile + Dockerfile.ci updated to copy webclaw-server into
/usr/local/bin and EXPOSE 3000. Workspace version bumped to 0.4.0
(new public binary).
18 lines
578 B
Rust
18 lines
578 B
Rust
//! HTTP route handlers.
|
|
//!
|
|
//! The OSS server exposes a deliberately small surface that mirrors the
|
|
//! hosted-API JSON shapes where the underlying capability exists in the
|
|
//! OSS crates. Endpoints that depend on private infrastructure
|
|
//! (anti-bot bypass with stealth Chrome, JS rendering at scale,
|
|
//! per-user auth, billing, async job queues, agent loops) are
|
|
//! intentionally not implemented here. Use api.webclaw.io for those.
|
|
|
|
pub mod batch;
|
|
pub mod brand;
|
|
pub mod crawl;
|
|
pub mod diff;
|
|
pub mod extract;
|
|
pub mod health;
|
|
pub mod map;
|
|
pub mod scrape;
|
|
pub mod summarize;
|