From f967cebe92fc2dbbe04dc41176091819d2f0d6cb Mon Sep 17 00:00:00 2001 From: CREDO23 Date: Thu, 2 Jul 2026 20:18:53 +0200 Subject: [PATCH] docs(scraping-agent): surface youtube in description and prompt --- .../subagents/builtins/scraping/description.md | 4 ++-- .../subagents/builtins/scraping/system_prompt.md | 3 +++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/surfsense_backend/app/agents/chat/multi_agent_chat/subagents/builtins/scraping/description.md b/surfsense_backend/app/agents/chat/multi_agent_chat/subagents/builtins/scraping/description.md index f6b2f8ada..1b0e087f6 100644 --- a/surfsense_backend/app/agents/chat/multi_agent_chat/subagents/builtins/scraping/description.md +++ b/surfsense_backend/app/agents/chat/multi_agent_chat/subagents/builtins/scraping/description.md @@ -1,2 +1,2 @@ -Scraping specialist: fetches live public web pages, searches the web to find pages, and compares fresh data against earlier findings in this chat. -Use whenever the task needs current data pulled from the open web rather than the workspace's own documents or connectors. Triggers include "scrape", "crawl", "fetch this URL/page", "search the web", "look up / find online", "check the price/stock/listing", "monitor this page", and "what changed since last time" — plus recurring versions ("check daily", "tell me weekly what changed"), which it can also schedule as an ongoing watch. +Scraping specialist: fetches live public web pages, searches the web to find pages, pulls structured data from YouTube (videos, channels, playlists, search, and comments), and compares fresh data against earlier findings in this chat. +Use whenever the task needs current data pulled from the open web or YouTube rather than the workspace's own documents or connectors. Triggers include "scrape", "crawl", "fetch this URL/page", "search the web", "look up / find online", "check the price/stock/listing", "monitor this page", "what changed since last time" — plus YouTube-specific asks like "get this YouTube video/channel/playlist", "find videos about X on YouTube", "get the comments on this video", "what are people saying about this video" — and recurring versions ("check daily", "tell me weekly what changed"), which it can also schedule as an ongoing watch. diff --git a/surfsense_backend/app/agents/chat/multi_agent_chat/subagents/builtins/scraping/system_prompt.md b/surfsense_backend/app/agents/chat/multi_agent_chat/subagents/builtins/scraping/system_prompt.md index 7f0d6841e..d4b0309a4 100644 --- a/surfsense_backend/app/agents/chat/multi_agent_chat/subagents/builtins/scraping/system_prompt.md +++ b/surfsense_backend/app/agents/chat/multi_agent_chat/subagents/builtins/scraping/system_prompt.md @@ -8,6 +8,8 @@ Answer the delegated question from live evidence gathered with your data verbs, - `web_discover` - `web_scrape` +- `youtube_scrape` +- `youtube_comments` - `start_watch` - `stop_watch` - `refresh_watch` @@ -16,6 +18,7 @@ Answer the delegated question from live evidence gathered with your data verbs, - Named URLs: `web_scrape` them directly. Otherwise `web_discover` first, then `web_scrape` the most relevant hits. - Read several pages in one batched `web_scrape` call rather than many single-URL calls. +- YouTube: use `youtube_scrape` for videos/channels/playlists (pass `urls`) or to find videos (pass `search_queries`); use `youtube_comments` to pull a video's comments. Prefer these over `web_scrape` for youtube.com links. - "What changed" / monitoring: scrape the current values, compare against the prior values in this conversation's earlier tool results, and report concrete deltas (added, removed, old -> new). - Recurring intent ("check daily", "tell me weekly what changed"): answer now, then `start_watch` with a self-contained question, a cron cadence, and an IANA timezone. Use `stop_watch` / `refresh_watch` to end or immediately re-run an existing watch.