mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-06-14 20:55:15 +02:00
- Replaced Playwright with Scrapling's fetchers in the web crawling and YouTube processing modules for improved performance and flexibility. - Updated proxy configuration to support dynamic proxy selection via environment variables. - Enhanced logging to track performance metrics during web scraping operations. - Refactored related modules to utilize the new proxy utilities and streamline the scraping process. |
||
|---|---|---|
| .. | ||
| celery_tasks | ||
| chat | ||
| connector_indexers | ||
| document_processors | ||
| __init__.py | ||
| composio_indexer.py | ||