Commit graph

24 commits

Author SHA1 Message Date
DESKTOP-RTLN3BA\$punk
dbedf0cfa5 feat(backend): integrate PostHog analytics for enhanced observability
- Added PostHog configuration options to .env.example files for both Docker and Surfsense backend.
- Introduced PostHog dependency in pyproject.toml.
- Implemented analytics middleware to capture various events across the application, including user authentication, automation runs, and API requests.
- Enhanced existing routes and services to emit analytics events, providing insights into user interactions and system performance.
- Ensured graceful shutdown of analytics clients in worker processes and application lifecycles.
2026-07-22 22:16:28 -07:00
Rohan Verma
08d431454f
Merge pull request #1617 from CREDO23/feature-okf
[Feat] Serve the knowledge base in Open Knowledge Format (OKF)
2026-07-21 15:39:59 -07:00
CREDO23
a01b292886 request OKF concept via content negotiation 2026-07-21 19:58:53 +02:00
CREDO23
edf38edc3f support per-request headers 2026-07-21 19:58:52 +02:00
CREDO23
ca317c4686 Merge remote-tracking branch 'upstream/dev' into feature-walmart-scraper
# Conflicts:
#	docker/.env.example
#	surfsense_backend/app/capabilities/core/billing.py
#	surfsense_backend/app/capabilities/core/types.py
#	surfsense_backend/app/config/__init__.py
#	surfsense_mcp/mcp_server/features/scrapers/__init__.py
#	surfsense_web/content/docs/connectors/index.mdx
#	surfsense_web/content/docs/connectors/native/index.mdx
#	surfsense_web/content/docs/how-to/mcp-server.mdx
2026-07-21 02:42:41 +02:00
CREDO23
a3f8a7e741 feat(walmart): register routes and MCP tools 2026-07-19 08:19:22 +02:00
CREDO23
91aa265afb Merge remote-tracking branch 'upstream/dev' into feature-indeed-jobs-scraper
# Conflicts:
#	README.es.md
#	README.hi.md
#	README.md
#	README.pt-BR.md
#	README.zh-CN.md
#	surfsense_backend/tests/unit/capabilities/google_maps/test_registry.py
#	surfsense_backend/tests/unit/capabilities/reddit/test_registry.py
#	surfsense_backend/tests/unit/capabilities/youtube/test_registry.py
#	surfsense_mcp/mcp_server/features/scrapers/__init__.py
#	surfsense_web/content/docs/connectors/index.mdx
#	surfsense_web/content/docs/connectors/native/index.mdx
#	surfsense_web/content/docs/connectors/native/meta.json
#	surfsense_web/content/docs/how-to/mcp-server.mdx
#	surfsense_web/lib/connectors-marketing/index.ts
#	surfsense_web/lib/playground/catalog.ts
2026-07-18 22:11:30 +02:00
CREDO23
65c757fbc6 feat: mention indeed in MCP server instructions 2026-07-15 18:54:49 +02:00
CREDO23
e88ff8cef7 test: expect indeed MCP tool in selfcheck 2026-07-15 18:54:49 +02:00
CREDO23
b89fe1a4f0 feat: register indeed MCP tool 2026-07-15 18:54:49 +02:00
CREDO23
457c073c44 feat: add indeed MCP scraper tool 2026-07-15 18:54:49 +02:00
Anish Sarkar
8d7ea2b269 feat(selfcheck): register Amazon scraper tool in self-check configuration 2026-07-15 10:15:27 +05:30
Anish Sarkar
964ad2e083 feat(amazon): implement Amazon product scraper tool and register scraping functionality 2026-07-15 10:15:19 +05:30
DESKTOP-RTLN3BA\$punk
2b018c4474 refactor: streamline TikTok and Instagram scraping logic by removing search_queries and enhancing documentation for clarity 2026-07-13 17:11:25 -07:00
DESKTOP-RTLN3BA\$punk
1131da5ed7 feat: bumped version to 0.0.32 2026-07-13 16:29:39 -07:00
Anish Sarkar
e38ca19b18 Merge remote-tracking branch 'upstream/dev' into feat/instagram-scraper 2026-07-11 04:31:11 +05:30
Anish Sarkar
ca7d23d33a refactor(mcp): align Instagram scraper tool with capability changes 2026-07-11 03:25:23 +05:30
Anish Sarkar
8fc86b7fa5 refactor(mcp): align Instagram scraper tool with new pipeline 2026-07-11 02:20:58 +05:30
CREDO23
6a0ff1495f fix(tiktok-mcp): flag that search_queries returns no videos in the scrape tool 2026-07-10 17:40:04 +02:00
CREDO23
6b15d610d9 feat(tiktok): surface comments, user_search, trending across all product surfaces
Brings the three newer verbs to parity with tiktok.scrape everywhere it was
wired: MCP tools (+ selfcheck manifest), the chat subagent prompt/description,
the playground catalog, the native docs page, and the SEO marketing page. Also
adds live e2e stages for comments, user search, and trending. Docs/FAQ now
state the real contract: profile metadata is reliable while its video list can
be withheld, and keyword video search is walled (use user search for accounts).
2026-07-09 19:09:00 +02:00
Anish Sarkar
c5e3fd4006 feat(mcp): add Instagram scrape/comments/details tools 2026-07-09 16:01:55 +05:30
CREDO23
5f9472b122 feat(tiktok): advertise tiktok across agent prompts, MCP instructions, docs 2026-07-08 20:30:38 +02:00
CREDO23
ed1c3a1f3d feat(tiktok): expose tiktok.scrape on MCP, chat subagent, playground 2026-07-08 20:20:20 +02:00
CREDO23
116291a3b6 refactor(mcp): flatten to mcp_server package, drop src layout
Rename the import package surfsense_mcp -> mcp_server and remove the
src/ layer so the project mirrors the backend's shape (project folder
!= package name, e.g. surfsense_backend/app). Kills the redundant
surfsense_mcp/src/surfsense_mcp nesting.

Distribution name and console command (surfsense-mcp) are unchanged;
only python -m and internal import paths move to mcp_server. Dockerfile
CMD updated; no PYTHONPATH added since the editable install already
makes the package importable.
2026-07-07 20:24:53 +02:00