mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-06-02 19:55:18 +02:00
feat(gateway): introduce GATEWAY_TELEGRAM_INTAKE_MODE for Telegram integration
This commit is contained in:
parent
7ff0120fc9
commit
c958fe5bc6
5 changed files with 53 additions and 14 deletions
|
|
@ -39,6 +39,39 @@ Complete all the [setup steps](/docs), including:
|
|||
|
||||
The backend is the core of SurfSense. Follow these steps to set it up:
|
||||
|
||||
### Optional: Telegram External Chat Surface
|
||||
|
||||
SurfSense can expose the same canonical chat agent through Telegram. The `external_chat_*` tables store adapter identity, delivery configuration, and durable inbox rows. The actual chat thread and messages remain in `new_chat_threads` and `new_chat_messages`, and all chat-message sources are eligible for Zero replication so a future SurfSense UI layer can render external chat surfaces. The web app initially shows pairing, health, revoke, and resume controls under **User Settings > Messaging Channels**.
|
||||
|
||||
Add these variables to `surfsense_backend/.env` when enabling the Telegram surface:
|
||||
|
||||
```bash
|
||||
TELEGRAM_SHARED_BOT_TOKEN=123456:bot-token-from-botfather
|
||||
TELEGRAM_SHARED_BOT_USERNAME=your_bot_username
|
||||
TELEGRAM_WEBHOOK_SECRET=generate-a-long-random-secret
|
||||
GATEWAY_BASE_URL=https://api.example.com
|
||||
GATEWAY_TELEGRAM_INTAKE_MODE=webhook
|
||||
```
|
||||
|
||||
`GATEWAY_TELEGRAM_INTAKE_MODE` must be `webhook`, `longpoll`, or `disabled`. Use `webhook` for production/SaaS deployments, `longpoll` only for single-replica self-host installs that cannot expose a public HTTPS webhook, and `disabled` to skip Telegram intake. `TELEGRAM_WEBHOOK_SECRET` must use only `A-Z`, `a-z`, `0-9`, `_`, and `-` characters. `REDIS_APP_URL` is reused for external chat rate limits and per-thread locks. The webhook URL shape is:
|
||||
|
||||
```text
|
||||
${GATEWAY_BASE_URL}/api/v1/gateway/webhooks/telegram/{account_id}
|
||||
```
|
||||
|
||||
After deploying the backend, register the webhook:
|
||||
|
||||
```bash
|
||||
cd surfsense_backend
|
||||
uv run python scripts/register_webhook.py
|
||||
```
|
||||
|
||||
Keep the FastAPI backend, Celery worker, and Celery beat running. Telegram webhooks write inbound updates into `external_chat_inbound_events`. The FastAPI process owns external chat inbox processing and runs the same SurfSense agent used by web UI chats, then replies back to Telegram. Celery remains maintenance-only for external chat reconciliation, health checks, and retention sweeps. There is no separate gateway service, `SERVICE_ROLE=gateway` process, or Celery agent-processing path.
|
||||
|
||||
For self-hosted BYO Telegram bots without a public HTTPS URL, set `GATEWAY_TELEGRAM_INTAKE_MODE=longpoll`. The FastAPI process starts one lifespan long-poll supervisor per non-system Telegram account and writes updates into the same durable inbox. The FastAPI inbox worker then processes those rows in-process through the canonical `new_chat_*` surface. This fallback is intended for single-replica self-hosted installs. For SaaS-style multi-replica deployments, prefer public webhooks and keep `GATEWAY_TELEGRAM_INTAKE_MODE=webhook` so API replicas skip BYO polling entirely. Telegram does not allow `get_updates()` while a webhook is active, so delete any existing webhook for a BYO bot before relying on long polling.
|
||||
|
||||
When upgrading from an older gateway-runner deployment, apply the rewritten migration 144 external chat schema, deploy the new backend, worker, and beat images, then stop the old `gateway` service. Wait about 30 seconds for any old Telegram `getUpdates` long-poll request to release its advisory lock before starting the new API process. Register each webhook again with the account-id URL above and the per-account `webhook_secret`. If you roll back before using the migration in production, restore the old image and downgrade the schema first.
|
||||
|
||||
### 1. Environment Configuration
|
||||
|
||||
First, create and configure your environment variables by copying the example file:
|
||||
|
|
@ -350,7 +383,7 @@ redis-cli ping
|
|||
|
||||
### 6. Start Celery Worker
|
||||
|
||||
In a new terminal window, start the Celery worker to handle background tasks:
|
||||
In a new terminal window, start the Celery worker to handle background tasks. For external chat surfaces, Celery only runs maintenance tasks; agent turns run inside the FastAPI process.
|
||||
|
||||
**If using uv:**
|
||||
|
||||
|
|
@ -358,9 +391,9 @@ In a new terminal window, start the Celery worker to handle background tasks:
|
|||
# Make sure you're in the surfsense_backend directory
|
||||
cd surfsense_backend
|
||||
|
||||
# Start Celery worker (consume both default and connectors queues)
|
||||
# Start Celery worker (consume default, connectors, and external chat maintenance queues)
|
||||
DEFAULT_Q="${CELERY_TASK_DEFAULT_QUEUE:-surfsense}"
|
||||
uv run celery -A celery_worker.celery_app worker --loglevel=info --concurrency=1 --pool=solo --queues="${DEFAULT_Q},${DEFAULT_Q}.connectors"
|
||||
uv run celery -A celery_worker.celery_app worker --loglevel=info --concurrency=1 --pool=solo --queues="${DEFAULT_Q},${DEFAULT_Q}.connectors,${DEFAULT_Q}.gateway"
|
||||
```
|
||||
|
||||
**If using pip/venv:**
|
||||
|
|
@ -374,9 +407,9 @@ source .venv/bin/activate # Linux/macOS
|
|||
# OR
|
||||
.venv\Scripts\activate # Windows
|
||||
|
||||
# Start Celery worker (consume both default and connectors queues)
|
||||
# Start Celery worker (consume default, connectors, and external chat maintenance queues)
|
||||
DEFAULT_Q="${CELERY_TASK_DEFAULT_QUEUE:-surfsense}"
|
||||
celery -A celery_worker.celery_app worker --loglevel=info --concurrency=1 --pool=solo --queues="${DEFAULT_Q},${DEFAULT_Q}.connectors"
|
||||
celery -A celery_worker.celery_app worker --loglevel=info --concurrency=1 --pool=solo --queues="${DEFAULT_Q},${DEFAULT_Q}.connectors,${DEFAULT_Q}.gateway"
|
||||
```
|
||||
|
||||
**Optional: Start Flower for monitoring Celery tasks:**
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue