mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-07-10 22:32:16 +02:00
feat: update environment variables and enhance scraping capabilities
- Adjusted Google Maps and YouTube micro pricing in the .env.example file for better cost management. - Introduced new environment variables for captcha solving and stealth browser hardening to improve scraping resilience. - Removed outdated smoke test for scraper API endpoints to streamline testing. - Enhanced anonymous chat agent's system prompt to clarify capabilities and suggest account creation for advanced features. - Updated Reddit fetch logic to prioritize new session handling and improve resilience against IP-related issues. - Added compacting functionality for scraper results to optimize data handling and presentation. - Improved workspace and document management tools with clearer descriptions and enhanced functionality. - Introduced new UI components for agent setup guidance in the web application.
This commit is contained in:
parent
271a21aee6
commit
1fd58752a3
24 changed files with 1326 additions and 320 deletions
|
|
@ -35,13 +35,14 @@ Native connectors are SurfSense's own scraper APIs — built into the platform,
|
|||
/>
|
||||
</Cards>
|
||||
|
||||
## Three ways to use them
|
||||
## Four ways to use them
|
||||
|
||||
Every scraper is available through the same three doors:
|
||||
Every scraper is available through the same four doors:
|
||||
|
||||
1. **In chat** — the AI agent uses these scrapers as tools automatically. Ask "what is r/selfhosted saying about SurfSense?" and the agent runs the Reddit scraper for you.
|
||||
2. **API Playground** — open **API Playground** in your workspace sidebar, pick a scraper, fill in the form, and run it interactively. Great for exploring what a scraper returns before writing code.
|
||||
3. **REST API** — call the scrapers from your own code. Each one is a single `POST`:
|
||||
3. **MCP server** — hand every scraper to Claude Code, Codex, OpenCode, Cursor, or any MCP client as native tools. See the [MCP server guide](/docs/how-to/mcp-server).
|
||||
4. **REST API** — call the scrapers from your own code. Each one is a single `POST`:
|
||||
|
||||
```bash
|
||||
POST /api/v1/workspaces/{workspace_id}/scrapers/{platform}/{verb}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue