apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-10 22:32:16 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat(billing): meter platform scrapers per item; consolidate web scraping onto web.crawl

Browse source 
Add per-item, per-platform billing for the platform-native connectors (Reddit, Google Search, Google Maps places/reviews, YouTube videos/comments) through the capability gate/charge seam. Rates are config-driven with a shared wallet-credit module (wallet_credit) and a dedicated PlatformScrapeCreditService; agent and REST capability runs now record cost_micros. Google Maps scrape dual-meters places and attached reviews.

Remove the main-agent scrape_webpage tool now that the web.crawl capability covers single-page (maxCrawlDepth=0) and site crawling. The main agent now reaches crawling via task(web_crawler, ...). Update prompts, tool catalog, receipts, skills, proprietary docs, and tests; drop the obsolete chat-turn crawl fold path.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
DESKTOP-RTLN3BA\$punk 2026-07-05 17:08:01 -07:00
parent b8285a0b72
commit 80927a2872
48 changed files with 724 additions and 766 deletions
Download patch file Download diff file Expand all files Collapse all files

2
surfsense_backend/tests/unit/agents/multi_agent_chat/test_prompt_resources.py
View file

@ -45,7 +45,7 @@ def test_every_subagent_has_description_md(name: str):
[
"core_behavior.md",
"routing.md",
"tools/scrape_webpage/description.md",
"tools/task/description.md",
],
)
def test_main_agent_prompt_fragments_resolve(filename: str):