webclaw/crates
Valerio 06f151c560 feat(search): standalone web search via Serper.dev (bring-your-own-key)
Rescued from the stale perf/audit-fixes branch and ported cleanly onto
current main. OSS surfaces can now search without the hosted webclaw API
when the caller supplies their own Serper.dev key (free at serper.dev).

- webclaw-fetch::search() — calls Serper.dev directly (plain wreq client;
  a JSON API needs no fingerprinting) and, with scrape=true, fetches +
  extracts the top result pages concurrently (bounded) via the caller's
  FetchClient. parse_serper_organic() is pure and unit-tested.
- MCP `search` tool: local-first — uses SERPER_API_KEY when set, else
  falls back to the hosted webclaw API. Adds country/lang/scrape params.
- OSS REST server: POST /v1/search, gated on SERPER_API_KEY (501 when
  unset, with a setup hint). Adds ApiError::NotImplemented.
- CLI: `webclaw search <query> [--serper-key|SERPER_API_KEY] [--num]
  [--country] [--lang] [--scrape] [--format]`.

No new dependencies (reuses futures-util already in the tree). Original
work by the prior author on perf/audit-fixes; this re-applies only the
search slice onto main.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 15:10:58 +02:00
..
webclaw-cli feat(search): standalone web search via Serper.dev (bring-your-own-key) 2026-06-17 15:10:58 +02:00
webclaw-core fix: harden LLM providers, UTF-8 handling, and webhook/batch reliability 2026-06-09 21:10:15 +02:00
webclaw-fetch feat(search): standalone web search via Serper.dev (bring-your-own-key) 2026-06-17 15:10:58 +02:00
webclaw-llm feat(llm): add Gemini provider and fix stale Anthropic default model 2026-06-16 15:52:37 +02:00
webclaw-mcp feat(search): standalone web search via Serper.dev (bring-your-own-key) 2026-06-17 15:10:58 +02:00
webclaw-pdf Initial release: webclaw v0.1.0 — web content extraction for LLMs 2026-03-23 18:31:11 +01:00
webclaw-server feat(search): standalone web search via Serper.dev (bring-your-own-key) 2026-06-17 15:10:58 +02:00