DESKTOP-RTLN3BA\$punk c600a2920b feat(crawler): harden web crawler and agent tooling for real-world research tasks ... Crawler engine: escalate thin JS-shell pages past static fetch, repair currency-lossy extractions, emit categorized link records with anchor provenance, and decode percent-encoded mailto:/tel: contacts; site crawls reuse the connector ladder via Scrapling's spider engine with URL pattern filters. Agent layer: read_run gains char_offset paging, search_run gains match excerpts, new export_run turns stored runs into CSV workspace docs; reddit search fair-shares the item budget across queries and dedupes cross-query hits. Subagent prompts and routing teach crawl-after-search, full-run coverage before summarizing, and executing own-tool next steps instead of returning partial.		2026-07-05 03:51:16 -07:00
..
__init__.py	feat(03a): implement Scrapling-only web crawler and remove Firecrawl integration	2026-06-29 20:37:56 -07:00
test_captcha.py	feat(native-connector): added google maps places & reviews scrapers	2026-07-02 21:58:24 -07:00
test_connector.py	feat(crawler): harden web crawler and agent tooling for real-world research tasks	2026-07-05 03:51:16 -07:00
test_connector_links.py	feat(web-crawler): add site_crawler spider + url_policy over the fetch engine	2026-07-03 10:50:41 +02:00
test_site_crawler.py	feat(web-crawler): add site_crawler spider + url_policy over the fetch engine	2026-07-03 10:50:41 +02:00
test_stealth.py	feat(proxy): refactor proxy configuration and add DataImpulse provider	2026-07-04 17:18:40 -07:00
test_url_policy.py	feat(crawler): harden web crawler and agent tooling for real-world research tasks	2026-07-05 03:51:16 -07:00