Commit graph

4 commits

Author SHA1 Message Date
CREDO23
d5d673384f Merge upstream/ci_mvp (google maps scrapers) into feature-ci_phase-4-7
Resolve conflicts against the new native google-maps actor + repo-wide
ruff-format pass:
- Keep legacy webcrawler KB indexer + its test deleted (modify/delete).
- test_validators: keep WEBCRAWLER case removed (validator gone).
- test_fetch_resilience: keep platforms.youtube import path (our reorg).
- Relocate google_maps actor + tests scrapers/ -> platforms/ to match the
  reorg convention (youtube already there); rewrite imports + fixture paths.
- Add missing __init__.py across the capabilities/ test subtree so duplicate
  test basenames get unique module paths under importlib mode.

Note: google_maps fixture-backed tests error on ci_mvp too (fixtures/*.json
never committed upstream) - pre-existing, out of scope here.
2026-07-03 12:37:12 +02:00
CREDO23
817ec0e9f4 refactor(youtube): remove legacy KB ingestion; use actor get_youtube_video_id 2026-07-03 12:00:22 +02:00
CREDO23
3312a442f3 refactor(proprietary): move scrapers/ -> platforms/ (one subpackage per platform) 2026-07-03 11:01:58 +02:00
DESKTOP-RTLN3BA\$punk
5c36cd3071 feat(03a): implement Scrapling-only web crawler and remove Firecrawl integration
- Standardized the web crawler to use Scrapling exclusively, removing Firecrawl entirely.
- Updated the crawler's location to `app/proprietary/web_crawler/connector.py` under a non-Apache-2 license boundary.
- Refactored the `WebCrawlerConnector` to eliminate the Firecrawl API key dependency, simplifying the interface for crawling URLs.
- Adjusted related components to accommodate the new structure and ensure successful crawl outcomes are properly handled.
- Updated documentation to reflect these changes and the new implementation status.
2026-06-29 20:37:56 -07:00