apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-10 22:32:16 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat: update environment variables and enhance scraping capabilities

Browse source 
- Adjusted Google Maps and YouTube micro pricing in the .env.example file for better cost management.
- Introduced new environment variables for captcha solving and stealth browser hardening to improve scraping resilience.
- Removed outdated smoke test for scraper API endpoints to streamline testing.
- Enhanced anonymous chat agent's system prompt to clarify capabilities and suggest account creation for advanced features.
- Updated Reddit fetch logic to prioritize new session handling and improve resilience against IP-related issues.
- Added compacting functionality for scraper results to optimize data handling and presentation.
- Improved workspace and document management tools with clearer descriptions and enhanced functionality.
- Introduced new UI components for agent setup guidance in the web application.
This commit is contained in:
DESKTOP-RTLN3BA\$punk 2026-07-06 20:27:36 -07:00
parent 271a21aee6
commit 1fd58752a3
24 changed files with 1326 additions and 320 deletions
Download patch file Download diff file Expand all files Collapse all files

10
surfsense_backend/tests/unit/platforms/reddit/test_fetch_resilience.py
View file

@ -95,8 +95,8 @@ def _no_sleep(monkeypatch) -> None:
async def test_warms_then_returns_json():
# old.reddit is tried first and mints loid -> a single warm call.
holder = _FakeHolder([_FakeSession(200, old_loid=True)])
# shreddit is tried first and mints loid -> a single warm call.
holder = _FakeHolder([_FakeSession(200, shreddit_loid=True)])
token = _current_session.set(holder)
try:
result = await fetch_json("r/python/hot")
@ -107,9 +107,9 @@ async def test_warms_then_returns_json():
assert holder.session.warm_calls == 1 # warmed exactly once
async def test_warm_falls_back_to_shreddit():
# old.reddit doesn't mint loid, shreddit does -> still warms on the same IP.
holder = _FakeHolder([_FakeSession(200, shreddit_loid=True, old_loid=False)])
async def test_warm_falls_back_to_old_reddit():
# shreddit doesn't mint loid, old.reddit does -> still warms on the same IP.
holder = _FakeHolder([_FakeSession(200, shreddit_loid=False, old_loid=True)])
token = _current_session.set(holder)
try:
result = await fetch_json("r/python/hot")