feat: completed init mvp of phase 3

This commit is contained in:
DESKTOP-RTLN3BA\$punk 2026-06-30 03:02:40 -07:00
parent bdacea8b6e
commit 15fd0b08d6
20 changed files with 2107 additions and 69 deletions

View file

@ -369,6 +369,22 @@ TURNSTILE_SECRET_KEY=
# CAPTCHA_V3_MIN_SCORE=0.7
# CAPTCHA_V3_ACTION=verify
# =====================================================================
# Stealth hardening (Phase 3e, Slice A) — runtime/config-level levers on the
# stealth browser tier. Consumed by app/proprietary/web_crawler/stealth.py.
# Defaults add no crawl-speed regression and preserve today's behavior.
# Match browser locale/timezone to RESIDENTIAL_PROXY_LOCATION (no exit-IP lookup).
# CRAWL_GEOIP_MATCH_ENABLED=FALSE
# WebRTC respects the proxy (anti local-IP leak). Cheap + safe.
# CRAWL_BLOCK_WEBRTC=TRUE
# Random canvas noise (an unstable canvas hash is itself a tell) — opt-in only.
# CRAWL_HIDE_CANVAS=FALSE
# Send a Google referer so the first hit looks like organic arrival.
# CRAWL_GOOGLE_SEARCH_REFERER=TRUE
# Route DNS via Cloudflare DoH (anti DNS-leak). Adds a DNS round-trip => off by
# default to avoid any speed regression; enable for leak-safety-first setups.
# CRAWL_DNS_OVER_HTTPS=FALSE
# File Parser Service
ETL_SERVICE=UNSTRUCTURED or LLAMACLOUD or DOCLING
UNSTRUCTURED_API_KEY=Tpu3P0U8iy