apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-26 23:51:14 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat(reddit-scraper): add support for community-only subreddit scraping without search queries

Browse source
This commit is contained in:
Anish Sarkar 2026-07-25 02:55:27 +05:30
parent 2379a6a23f
commit 8d1e2a5134
1 changed files with 12 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

12
surfsense_backend/app/proprietary/platforms/reddit/scraper.py
View file

@ -407,6 +407,18 @@ async def iter_reddit(
yield item
return
# Community-only: a bare subreddit name with no urls/searches. The capability
# schema promises "with no search_queries, its listing is scraped"; without
# this the searches loop below builds zero jobs and yields nothing. Reuse the
# same flow the URL path dispatches for /r/<sub>. (searches present => the
# name stays a search scope, handled below.)
if not input_model.searches and input_model.searchCommunityName:
async for item in _subreddit_flow(
input_model.searchCommunityName, input_model=input_model
):
yield item
return
# Fair-share the item budget across queries: with a shared cap, the
# first-finishing (often broadest/noisiest) search would fill the whole
# collector limit and starve the precise queries.