feat(billing): meter platform scrapers per item; consolidate web scraping onto web.crawl

Add per-item, per-platform billing for the platform-native connectors (Reddit, Google Search, Google Maps places/reviews, YouTube videos/comments) through the capability gate/charge seam. Rates are config-driven with a shared wallet-credit module (wallet_credit) and a dedicated PlatformScrapeCreditService; agent and REST capability runs now record cost_micros. Google Maps scrape dual-meters places and attached reviews.

Remove the main-agent scrape_webpage tool now that the web.crawl capability covers single-page (maxCrawlDepth=0) and site crawling. The main agent now reaches crawling via task(web_crawler, ...). Update prompts, tool catalog, receipts, skills, proprietary docs, and tests; drop the obsolete chat-turn crawl fold path.

Co-authored-by: Cursor <cursoragent@cursor.com>

surfsense_backend/tests/unit/agents/multi_agent_chat/test_prompt_resources.py

@@ -45,7 +45,7 @@ def test_every_subagent_has_description_md(name: str):
     [
         "core_behavior.md",
         "routing.md",
-        "tools/scrape_webpage/description.md",
+        "tools/task/description.md",
     ],
 )
 def test_main_agent_prompt_fragments_resolve(filename: str):

surfsense_backend/tests/unit/agents/new_chat/test_default_permissions_layering.py

@@ -9,7 +9,7 @@ earliest -> latest:
 3. (future) user-defined rules from the Agent Permissions UI
 
 Without #1 every read-only built-in (``ls``, ``read_file``, ``grep``,
-``glob``, ``scrape_webpage`` …) defaulted to ``ask`` because
+``glob`` …) defaulted to ``ask`` because
 ``permissions.evaluate`` returns ``ask`` when no rule matches. That
 caused two production-painful behaviors:
 
@@ -58,7 +58,6 @@ class TestReadOnlyToolsAllowed:
             "read_file",
             "grep",
             "glob",
-            "scrape_webpage",
             "get_connected_accounts",
             "write_todos",
             "task",