mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-07-06 22:12:12 +02:00
62260125f7
Add CustomProxyProvider (single endpoint or rotating pool via Scrapling ProxyRotator), registered as 'custom' alongside anonymous_proxies and selectable via PROXY_PROVIDER. Adds is_pool_backed to the ProxyProvider ABC + a zero-arg package helper. The web crawler does a bounded one-per-tier is_proxy_error rotation-retry gated on is_pool_backed() (single-endpoint providers no-op). Config/.env.example gain CUSTOM_PROXY_URL(S). Zero-arg getter contract unchanged for all consumers. Documents the proprietary boundary test (generic proxy infra stays Apache-2). Tests: provider, registry, crawler rotation (16). Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|---|---|---|
| .. | ||
| proxy | ||
| async_retry.py | ||
| blocknote_to_markdown.py | ||
| chat_comments.py | ||
| connector_naming.py | ||
| content_utils.py | ||
| document_converters.py | ||
| document_versioning.py | ||
| file_extensions.py | ||
| google_credentials.py | ||
| indexing_locks.py | ||
| notion_utils.py | ||
| oauth_security.py | ||
| pat.py | ||
| perf.py | ||
| periodic_scheduler.py | ||
| proxy_config.py | ||
| rbac.py | ||
| refresh_tokens.py | ||
| signed_image_urls.py | ||
| user_message_multimodal.py | ||
| validators.py | ||
| webcrawler_utils.py | ||