Valerio 050b2ef463 feat: add allow_subdomains and allow_external_links to CrawlConfig ... Crawls are same-origin by default. Enable allow_subdomains to follow sibling/child subdomains (blog.example.com from example.com), or allow_external_links for full cross-origin crawling. Root domain extraction uses a heuristic that handles two-part TLDs (co.uk, com.au). Includes 5 unit tests for root_domain(). Bump to 0.3.12. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>		2026-04-14 19:33:06 +02:00
..
src	feat: add allow_subdomains and allow_external_links to CrawlConfig	2026-04-14 19:33:06 +02:00
Cargo.toml	Initial release: webclaw v0.1.0 — web content extraction for LLMs	2026-03-23 18:31:11 +01:00