apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-18 23:11:12 +02:00
Code Issues Projects Releases Packages Wiki Activity
7205 commits 7 branches 273 tags 437 MiB
Commit graph

9 commits

Author SHA1 Message Date
Anish Sarkar
3956045221 feat(native-connector): classify reddit scrape urls 2026-07-04 17:27:44 +05:30
Anish Sarkar
eaa8239e36 feat(native-connector): add reddit scraper orchestrator 2026-07-04 17:27:38 +05:30
Anish Sarkar
dd43ab8505 feat(native-connector): add reddit scraper schemas 2026-07-04 17:27:33 +05:30
Anish Sarkar
0745f433ab feat(native-connector): add reddit json parsers 2026-07-04 17:27:22 +05:30
Anish Sarkar
7151465e12 feat(native-connector): add anonymous reddit fetch layer 2026-07-04 17:27:13 +05:30
Anish Sarkar
1115d647cb feat(native-connector): expose reddit scraper public API 2026-07-04 17:26:58 +05:30
Anish Sarkar
54eab56b25 docs(native-connector): document anonymous reddit scraper design 2026-07-04 17:26:12 +05:30
DESKTOP-RTLN3BA\$punk
7185079bd6 feat(native-connector): added google maps places & reviews scrapers 2026-07-02 21:58:24 -07:00
DESKTOP-RTLN3BA\$punk
0445c646c6 refactor(native-connector): relocate youtube scraper under app/proprietary + parallelize playlists 
Move app/scrapers -> app/proprietary/scrapers/youtube to sit alongside the existing proprietary web_crawler/platforms namespace, updating all external imports (routes, tests, e2e script, README). Internal imports were relative so are unchanged.

Also parallelize playlist per-video resolution: page video ids sequentially, then resolve the heavy watch-page fetches concurrently via fan_out (~150 videos ~70s, down from a few minutes). Items stream in completion order; sort by the order field for playlist order.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-02 02:34:04 -07:00