- Standardized the web crawler to use Scrapling exclusively, removing Firecrawl entirely.
- Updated the crawler's location to `app/proprietary/web_crawler/connector.py` under a non-Apache-2 license boundary.
- Refactored the `WebCrawlerConnector` to eliminate the Firecrawl API key dependency, simplifying the interface for crawling URLs.
- Adjusted related components to accommodate the new structure and ensure successful crawl outcomes are properly handled.
- Updated documentation to reflect these changes and the new implementation status.
Add a top status banner (SHIPPED, branch/PR, phase tip commit) and an
Implementation record (deviations, carve-outs as-shipped, re-runnable
verification gates, deploy caveats) to 02-rename-backend.md.
Add a top status banner (SHIPPED, branch/PR, phase tip commit) and an
Implementation record (deviations, carve-outs as-shipped, re-runnable
verification gates with last results) to 01-rename-db.md.
