apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-10 22:32:16 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat(proxy): refactor proxy configuration and add DataImpulse provider

Browse source 
- Updated proxy configuration in `.env.example` files to use `PROXY_URL` and `PROXY_URLS` instead of `CUSTOM_PROXY_URL` and `CUSTOM_PROXY_URLS`.
- Introduced `DataImpulseProvider` for proxy management, replacing the deprecated `AnonymousProxiesProvider`.
- Enhanced documentation to reflect changes in proxy setup and usage.
- Adjusted related code in the proxy registry and configuration files to support the new provider structure.
This commit is contained in:
DESKTOP-RTLN3BA\$punk 2026-07-04 17:18:40 -07:00
parent 203215adc0
commit 701f888b9e
19 changed files with 303 additions and 193 deletions
Download patch file Download diff file Expand all files Collapse all files

32
surfsense_backend/.env.example
View file

@ -338,23 +338,21 @@ TURNSTILE_SECRET_KEY=
# Proxy provider selection. Selects a ProxyProvider implementation registered in
# app/utils/proxy/registry.py. Default: "anonymous_proxies". Add new vendors there.
# PROXY_PROVIDER=anonymous_proxies
# app/utils/proxy/registry.py. Default: "custom". Options: "custom", "dataimpulse".
# PROXY_PROVIDER=custom
# Residential Proxy Configuration (anonymous-proxies.net)
# Used for web crawling, link previews, and YouTube transcript fetching to avoid IP bans.
# Consumed by the "anonymous_proxies" provider. Leave commented out to disable proxying.
# RESIDENTIAL_PROXY_USERNAME=your_proxy_username
# RESIDENTIAL_PROXY_PASSWORD=your_proxy_password
# RESIDENTIAL_PROXY_HOSTNAME=rotating.dnsproxifier.com:31230
# RESIDENTIAL_PROXY_LOCATION=
# RESIDENTIAL_PROXY_TYPE=1
# Custom (BYO) proxy provider. Activate with PROXY_PROVIDER=custom.
# Provide a single endpoint and/or a comma-separated pool; a pool of >1 rotates
# client-side (cyclic). Each value is a full http://user:pass@host:port URL.
# CUSTOM_PROXY_URL=http://user:pass@host:port
# CUSTOM_PROXY_URLS=http://user:pass@host1:port,http://user:pass@host2:port
# Proxy endpoint(s), shared across providers — PROXY_PROVIDER picks the behavior,
# not a different env name. Used for web crawling, link previews, and YouTube
# transcript fetching to avoid IP bans. Leave unset to disable proxying.
#
# PROXY_URL: a single full http://user:pass@host:port endpoint (used by every
# provider). For "dataimpulse", country is a "__cr.<country>" username suffix the
# provider parses for geoip-match, e.g.:
# PROXY_URL=http://your_token__cr.us:your_password@gw.dataimpulse.com:823
# PROXY_URLS: a comma-separated pool the "custom" provider rotates client-side
# (cyclic); server-side-rotating gateways ignore it.
# PROXY_URL=http://user:pass@host:port
# PROXY_URLS=http://user:pass@host1:port,http://user:pass@host2:port
# =====================================================================
# Captcha solving (Phase 3d) — LAST-resort bypass tier via captchatools.
@ -380,7 +378,7 @@ TURNSTILE_SECRET_KEY=
# Stealth hardening (Phase 3e, Slice A) — runtime/config-level levers on the
# stealth browser tier. Consumed by app/proprietary/web_crawler/stealth.py.
# Defaults add no crawl-speed regression and preserve today's behavior.
# Match browser locale/timezone to RESIDENTIAL_PROXY_LOCATION (no exit-IP lookup).
# Match browser locale/timezone to the proxy provider's exit region (no exit-IP lookup).
# CRAWL_GEOIP_MATCH_ENABLED=FALSE
# WebRTC respects the proxy (anti local-IP leak). Cheap + safe.
# CRAWL_BLOCK_WEBRTC=TRUE