Add Puppeteer fallback for Cloudflare-protected sites

Browse source

When HTTP requests are blocked with 403 (e.g., B&H Photo's Cloudflare
protection), the scraper now automatically retries using a headless
Chrome browser via Puppeteer. Also updated Dockerfile to include
Chromium dependencies for container deployment.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

...

This commit is contained in:

clucraft 2026-01-21 20:55:49 -05:00

parent 7c8ab0721b

commit c96861fefb

4 changed files with 1415 additions and 36 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

1294

backend/package-lock.json generated

View file

File diff suppressed because it is too large Load diff