mirror of
https://github.com/0xMassi/webclaw.git
synced 2026-04-25 00:06:21 +02:00
feat: replace custom TLS stack with wreq (BoringSSL), bump v0.3.3
Migrated webclaw-fetch from webclaw-tls (patched rustls/h2/hyper/reqwest) to wreq by @0x676e67. wreq uses BoringSSL for TLS and the http2 crate for HTTP/2 fingerprinting — battle-tested with 60+ browser profiles. This removes all 5 [patch.crates-io] entries that consumers previously needed. Browser profiles (Chrome 145, Firefox 135, Safari 18, Edge 145) are now built directly on wreq's Emulation API with correct TLS options, HTTP/2 SETTINGS ordering, pseudo-header order, and header wire order. 84% pass rate across 1000 real sites. 384 unit tests green. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
0d0da265ab
commit
aaf51eddef
10 changed files with 754 additions and 343 deletions
|
|
@ -315,7 +315,7 @@ Noise removal webclaw ██████████████████
|
|||
webclaw/
|
||||
crates/
|
||||
webclaw-core Pure extraction engine. Zero network deps. WASM-safe.
|
||||
webclaw-fetch HTTP client + TLS fingerprinting. Crawler. Batch ops.
|
||||
webclaw-fetch HTTP client + TLS fingerprinting (wreq/BoringSSL). Crawler. Batch ops.
|
||||
webclaw-llm LLM provider chain (Ollama -> OpenAI -> Anthropic)
|
||||
webclaw-pdf PDF text extraction
|
||||
webclaw-mcp MCP server (10 tools for AI agents)
|
||||
|
|
@ -391,6 +391,10 @@ We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
|
|||
- [Good first issues](https://github.com/0xMassi/webclaw/issues?q=label%3A%22good+first+issue%22)
|
||||
- [Architecture docs](CONTRIBUTING.md#architecture)
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
TLS and HTTP/2 browser fingerprinting is powered by [wreq](https://github.com/0x676e67/wreq) and [http2](https://github.com/0x676e67/http2) by [@0x676e67](https://github.com/0x676e67), who pioneered browser-grade HTTP/2 fingerprinting in Rust.
|
||||
|
||||
## License
|
||||
|
||||
[MIT](LICENSE) — use it however you want.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue