SurfSense/surfsense_backend/tests/unit/proprietary/web_crawler
CREDO23 d5d673384f Merge upstream/ci_mvp (google maps scrapers) into feature-ci_phase-4-7
Resolve conflicts against the new native google-maps actor + repo-wide
ruff-format pass:
- Keep legacy webcrawler KB indexer + its test deleted (modify/delete).
- test_validators: keep WEBCRAWLER case removed (validator gone).
- test_fetch_resilience: keep platforms.youtube import path (our reorg).
- Relocate google_maps actor + tests scrapers/ -> platforms/ to match the
  reorg convention (youtube already there); rewrite imports + fixture paths.
- Add missing __init__.py across the capabilities/ test subtree so duplicate
  test basenames get unique module paths under importlib mode.

Note: google_maps fixture-backed tests error on ci_mvp too (fixtures/*.json
never committed upstream) - pre-existing, out of scope here.
2026-07-03 12:37:12 +02:00
..
__init__.py feat(03a): implement Scrapling-only web crawler and remove Firecrawl integration 2026-06-29 20:37:56 -07:00
test_captcha.py feat(native-connector): added google maps places & reviews scrapers 2026-07-02 21:58:24 -07:00
test_connector.py feat(native-connector): added google maps places & reviews scrapers 2026-07-02 21:58:24 -07:00
test_connector_links.py feat(web-crawler): add site_crawler spider + url_policy over the fetch engine 2026-07-03 10:50:41 +02:00
test_site_crawler.py feat(web-crawler): add site_crawler spider + url_policy over the fetch engine 2026-07-03 10:50:41 +02:00
test_stealth.py feat: completed init mvp of phase 3 2026-06-30 03:02:40 -07:00
test_url_policy.py feat(web-crawler): add site_crawler spider + url_policy over the fetch engine 2026-07-03 10:50:41 +02:00