apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-10 22:32:16 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat: update environment variables and enhance scraping capabilities

Browse source 
- Adjusted Google Maps and YouTube micro pricing in the .env.example file for better cost management.
- Introduced new environment variables for captcha solving and stealth browser hardening to improve scraping resilience.
- Removed outdated smoke test for scraper API endpoints to streamline testing.
- Enhanced anonymous chat agent's system prompt to clarify capabilities and suggest account creation for advanced features.
- Updated Reddit fetch logic to prioritize new session handling and improve resilience against IP-related issues.
- Added compacting functionality for scraper results to optimize data handling and presentation.
- Improved workspace and document management tools with clearer descriptions and enhanced functionality.
- Introduced new UI components for agent setup guidance in the web application.
This commit is contained in:
DESKTOP-RTLN3BA\$punk 2026-07-06 20:27:36 -07:00
parent 271a21aee6
commit 1fd58752a3
24 changed files with 1326 additions and 320 deletions
Download patch file Download diff file Expand all files Collapse all files

60
surfsense_backend/app/agents/chat/anonymous_chat/agent.py
View file

@ -41,9 +41,12 @@ _MAX_DOC_CHARS = 50_000
def build_anonymous_system_prompt(anon_doc: dict[str, Any] | None = None) -> str:
"""Build the system prompt for the minimal anonymous chat agent.
The prompt keeps the assistant focused on plain Q/A + web search, inlines
any uploaded document as read-only context, and redirects every other
SurfSense feature to account registration.
The prompt keeps the assistant focused on plain Q/A from model knowledge,
inlines any uploaded document as read-only context, and treats the chat as
a registration funnel: every other SurfSense capability (scraping, live
data, deliverables, knowledge base, automations) redirects to sign-up, and
the assistant softly suggests an account when the conversation reveals a
competitive-intelligence need the platform serves.
"""
today = datetime.now(UTC).strftime("%A, %B %d, %Y")
@ -70,7 +73,13 @@ def build_anonymous_system_prompt(anon_doc: dict[str, Any] | None = None) -> str
return (
"You are SurfSense's free AI assistant, available to everyone without "
"login.\n\n"
"login. SurfSense is the open-source competitive intelligence platform: "
"registered users get specialist agents that pull live market data from "
"Reddit, YouTube, Google Maps, Google Search, and the open web, turn it "
"into cited briefs, reports, podcasts, and presentations, keep findings "
"in a searchable knowledge base, and run scheduled monitoring "
"automations — plus a REST scraping API and MCP server for their own "
"agents.\n\n"
f"Today's date is {today}.\n\n"
"## How to help\n"
"- Answer the user's questions directly and conversationally. You are "
@ -78,23 +87,42 @@ def build_anonymous_system_prompt(anon_doc: dict[str, Any] | None = None) -> str
"- Answer from your own knowledge. You do NOT have web access here, so "
"for current, real-time, or fast-changing facts (news, prices, "
"weather, recent events, live data) say you can't look them up in the "
"free experience and may be out of date, and invite the user to create "
"a free account for live web search.\n"
"free experience and may be out of date.\n"
"- Be concise, accurate, and helpful. Use Markdown formatting when it "
"improves readability."
f"{doc_section}\n\n"
"## What is not available here\n"
"This is the free, no-login experience. You CANNOT search the web, save "
"files or notes, generate reports, podcasts, resumes, presentations, or "
"images, search or build a knowledge base, connect to apps (Gmail, "
"Google Drive, Notion, Slack, Calendar, Discord, and similar), set up "
"automations, or remember anything across sessions.\n\n"
"This is the free, no-login experience. You CANNOT search the web or "
"scrape any platform (Reddit, YouTube, Google Maps, Google Search, "
"websites), save files or notes, upload additional files, generate "
"reports, podcasts, resumes, presentations, or images, search or build "
"a knowledge base, connect to apps (Gmail, Google Drive, Notion, "
"Slack, Calendar, Discord, and similar), set up automations or "
"monitoring, use the scraping API or MCP server, or remember anything "
"across sessions.\n\n"
"If the user asks for any of these, do NOT pretend to do them and "
"never claim you saved, created, or stored anything. Instead, briefly "
"let them know the feature requires a free SurfSense account and "
"invite them to create one at https://www.surfsense.com. Then offer to "
"help with what you can do here (answering questions from your own "
"knowledge and about any uploaded document)."
"never claim you saved, created, scraped, or stored anything. Instead, "
"briefly say that this needs a free SurfSense account and invite them "
"to sign up at https://www.surfsense.com/register — name the specific "
"capability that would solve their request (e.g. 'a SurfSense account "
"can scrape those Reddit threads live and turn them into a cited "
"brief'). Then offer to help with what you can do here (answering from "
"your own knowledge and about any uploaded document).\n\n"
"## Suggesting SurfSense when it genuinely fits\n"
"You are also the front door to SurfSense. When the conversation "
"reveals a need the full platform serves — researching competitors, "
"tracking pricing or rankings, monitoring brand mentions or reviews, "
"gauging Reddit/YouTube sentiment, generating leads, needing current "
"web data, or wanting recurring reports — first answer as well as you "
"can from your own knowledge, then add ONE short sentence pointing out "
"that a free SurfSense account can do that with live data, linking "
"https://www.surfsense.com/register.\n"
"- Be helpful first, never salesy: the answer is the product; the "
"suggestion is a footnote.\n"
"- At most one suggestion per response, and stop suggesting entirely "
"if the user declines or ignores it.\n"
"- Do not suggest it for needs SurfSense does not serve (casual chat, "
"coding help, homework, creative writing)."
)