feat: replace primp with webclaw-tls, bump to v0.3.0

Replace primp dependency with our own TLS fingerprinting stack
(webclaw-tls). Perfect Chrome 146 JA4 + Akamai hash match.

- Remove primp entirely (zero references remaining)
- webclaw-fetch now uses webclaw-http from github.com/0xMassi/webclaw-tls
- Native + Mozilla root CAs (fixes HTTPS on cross-signed cert chains)
- Skip unknown certificate extensions (SCT tolerance)
- 99% bypass rate on 102 sites (was ~85% with primp)
- Fixes #5 (HTTPS broken — example.com and similar sites now work)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Valerio 2026-03-29 16:40:10 +02:00
parent 77e93441c0
commit f13cb83c73
8 changed files with 204 additions and 599 deletions

View file

@ -1,15 +1,14 @@
/// webclaw-fetch: HTTP client layer with browser TLS fingerprint impersonation.
/// Uses Impit under the hood to make requests that look like real
/// browsers at the TLS, HTTP/2, and header levels.
/// Automatically detects PDF responses and delegates to webclaw-pdf.
//! webclaw-fetch: HTTP client layer with browser TLS fingerprint impersonation.
//! Uses webclaw-http for browser-grade TLS + HTTP/2 fingerprinting.
//! Automatically detects PDF responses and delegates to webclaw-pdf.
pub mod browser;
pub mod client;
pub mod crawler;
pub mod document;
pub mod error;
pub mod linkedin;
pub(crate) mod linkedin;
pub mod proxy;
pub mod reddit;
pub(crate) mod reddit;
pub mod sitemap;
pub use browser::BrowserProfile;