feat(agents): consolidate connectors under mcp_discovery; route web search through google_search

MCP consolidation:
- Route all MCP-capable connectors (Slack, Jira, Linear, ClickUp, Airtable,
  Notion, Confluence, interim Gmail/Calendar, custom MCP) through a single
  `mcp_discovery` subagent. Drive/OneDrive/Dropbox stay native to enrich the KB.
- Deprecate Discord/Teams/Luma: no viable official MCP server.

Google-only web search:
- Remove the main-agent `web_search` tool and the SearXNG platform service;
  all public web search now flows through the `google_search` subagent via task().
- Deprecate the Tavily/SearXNG/Linkup/Baidu search connectors (HTTP 410 on
  create, "Deprecated" badge); guide heavy users to the custom MCP connector.
- Remove web search from anonymous chat (pure Q&A).
- Tear SearXNG out of docker compose + install scripts; drop tavily-python
  and linkup-sdk deps and their config/env vars.

Fix:
- metrics._package_version() now swallows any metadata lookup failure. A
  malformed editable-install distribution with no `Version` field raised
  KeyError deep in importlib.metadata, and since it runs on every
  record_subagent_invoke_duration call it was crashing every task()
  delegation. Verified end-to-end against live GPT-5.4.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
DESKTOP-RTLN3BA\$punk 2026-07-04 21:06:04 -07:00
parent ff2e5f390f
commit ab747e7a49
206 changed files with 1704 additions and 7223 deletions

View file

@ -1,7 +1,7 @@
# =============================================================================
# SurfSense — Dependencies only (no backend / frontend / Celery images)
# =============================================================================
# Postgres, Redis, SearXNG, pgAdmin, Zero — run API + Next + Celery on the host.
# Postgres, Redis, pgAdmin, Zero — run API + Next + Celery on the host.
# Celery is not Dockerized here: use `uv run` from surfsense_backend/ (no extra
# backend image build just for workers).
#
@ -9,7 +9,7 @@
# docker compose -f docker/docker-compose.deps-only.yml up -d
#
# Compose variable substitution uses `docker/.env` (copy from .env.example).
# Bind mounts use ./postgresql.conf and ./searxng in this directory.
# Bind mounts use ./postgresql.conf in this directory.
#
# Local Celery (from surfsense_backend/, after Redis is up):
# uv run celery -A celery_worker.celery_app worker --loglevel=info --concurrency=1 --pool=solo --queues=surfsense,surfsense.connectors
@ -17,7 +17,6 @@
#
# Host setup:
# - Backend .env: DATABASE_URL=postgresql+asyncpg://postgres:postgres@localhost:5432/surfsense
# - Backend .env: SEARXNG_DEFAULT_HOST=http://localhost:${SEARXNG_PORT:-8888}
# - Backend .env: CELERY_BROKER_URL / REDIS_APP_URL → redis://localhost:6379/0
# - Web .env: NEXT_PUBLIC_ZERO_CACHE_URL=http://localhost:${ZERO_CACHE_PORT:-4848}
#
@ -80,20 +79,6 @@ services:
timeout: 5s
retries: 5
searxng:
image: searxng/searxng:2026.3.13-3c1f68c59
ports:
- "${SEARXNG_PORT:-8888}:8080"
volumes:
- ./searxng:/etc/searxng
environment:
- SEARXNG_SECRET=${SEARXNG_SECRET:-surfsense-searxng-secret}
healthcheck:
test: ["CMD", "wget", "--spider", "-q", "http://localhost:8080/healthz"]
interval: 10s
timeout: 5s
retries: 5
# NOTE: zero-cache requires the `zero_publication` Postgres publication to
# exist before it starts. In this deps-only stack there is no backend
# container to run migrations, so you must run `uv run alembic upgrade head`