webclaw/crates
djimenez18 17d805fb6d fix: add danger_accept_invalid_certs to plain TLS fallback clients
The primp-patched rustls fork rejects valid certificates with
`InvalidMessage(UnknownCertificateExtension)` on the impersonated
client, and `InvalidCertificate(UnknownIssuer)` on the plain fallback.

This causes ALL HTTPS requests to fail on Windows (native + WSL),
Linux, and Docker — only HTTP works.

Root cause: the forked rustls at `github.com/deedy5/primp` cannot
parse certain certificate extensions that standard rustls handles.
The plain fallback clients also use primp::Client (which goes through
the same broken fork), so they fail with UnknownIssuer since the
cert chain validation rejects the root CA.

Fix: add `.danger_accept_invalid_certs(true)` to all 4 plain fallback
client builders in `webclaw-fetch/src/client.rs`. These fallback
clients only run when the impersonated TLS client has already failed,
so relaxing cert validation here is acceptable as a workaround.

The proper long-term fix is to update the primp rustls fork to handle
modern certificate extensions correctly.

Tested: HTTPS now works on example.com, apartments.com, and real
apartment property websites that previously returned
"fetch error: request failed: error sending request for url".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 21:14:00 -05:00
..
webclaw-cli feat: v0.2.0 — DOCX/XLSX/CSV extraction, HTML format, multi-URL watch, batch LLM 2026-03-26 15:28:23 +01:00
webclaw-core feat: v0.1.4 — QuickJS integration for inline JavaScript data extraction 2026-03-26 10:28:16 +01:00
webclaw-fetch fix: add danger_accept_invalid_certs to plain TLS fallback clients 2026-03-28 21:14:00 -05:00
webclaw-llm Initial release: webclaw v0.1.0 — web content extraction for LLMs 2026-03-23 18:31:11 +01:00
webclaw-mcp feat: v0.1.3 — crawl streaming, resume/cancel, MCP proxy support 2026-03-25 21:38:28 +01:00
webclaw-pdf Initial release: webclaw v0.1.0 — web content extraction for LLMs 2026-03-23 18:31:11 +01:00