mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-07-06 22:12:12 +02:00
5c36cd3071
- Standardized the web crawler to use Scrapling exclusively, removing Firecrawl entirely. - Updated the crawler's location to `app/proprietary/web_crawler/connector.py` under a non-Apache-2 license boundary. - Refactored the `WebCrawlerConnector` to eliminate the Firecrawl API key dependency, simplifying the interface for crawling URLs. - Adjusted related components to accommodate the new structure and ensure successful crawl outcomes are properly handled. - Updated documentation to reflect these changes and the new implementation status. |
||
|---|---|---|
| .. | ||
| alembic | ||
| app | ||
| scripts | ||
| tests | ||
| .dockerignore | ||
| .env.example | ||
| .gitignore | ||
| .python-version | ||
| alembic.ini | ||
| celery_worker.py | ||
| Dockerfile | ||
| main.py | ||
| pyproject.toml | ||
| uv.lock | ||