apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-20 23:21:06 +02:00
Code Issues Projects Releases Packages Wiki Activity
7704 commits 7 branches 273 tags 437 MiB
Commit graph

4 commits

Author SHA1 Message Date
DESKTOP-RTLN3BA\$punk
2cbe6fc111 feat: bumped version to 0.0.33 2026-07-17 17:23:21 -07:00
DESKTOP-RTLN3BA\$punk
8a5d37a4db refactor: implement cross-country proxy rotation for Reddit and TikTok scrapers, enhancing resilience against IP blocks and improving search query handling 2026-07-17 16:30:26 -07:00
DESKTOP-RTLN3BA\$punk
1fd58752a3 feat: update environment variables and enhance scraping capabilities 
- Adjusted Google Maps and YouTube micro pricing in the .env.example file for better cost management.
- Introduced new environment variables for captcha solving and stealth browser hardening to improve scraping resilience.
- Removed outdated smoke test for scraper API endpoints to streamline testing.
- Enhanced anonymous chat agent's system prompt to clarify capabilities and suggest account creation for advanced features.
- Updated Reddit fetch logic to prioritize new session handling and improve resilience against IP-related issues.
- Added compacting functionality for scraper results to optimize data handling and presentation.
- Improved workspace and document management tools with clearer descriptions and enhanced functionality.
- Introduced new UI components for agent setup guidance in the web application.
 2026-07-06 20:27:36 -07:00
DESKTOP-RTLN3BA\$punk
ff2e5f390f feat(reddit): implement Reddit scraping subagent and associated capabilities 
- Added a new `reddit` subagent to scrape structured data from Reddit posts, comments, and users.
- Introduced `reddit.scrape` capability for fetching data using URLs and search queries.
- Implemented tools for scraping and parsing Reddit data, including handling pagination and rate limits.
- Created input/output models for the Reddit scraper to define request and response structures.
- Added documentation for the new Reddit scraping functionality and its usage.
- Integrated the Reddit subagent into the existing multi-agent chat framework.
 2026-07-04 17:31:11 -07:00
Renamed from surfsense_backend/app/proprietary/scrapers/reddit/fetch.py (Browse further)