apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-08 22:22:17 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat: docs and ui tweaks

Browse source
This commit is contained in:
DESKTOP-RTLN3BA\$punk 2026-07-06 19:26:35 -07:00
parent 64f2b4a6eb
commit 271a21aee6
103 changed files with 2161 additions and 2625 deletions
Download patch file Download diff file Expand all files Collapse all files

39
surfsense_web/content/docs/docker-installation/dev-compose.mdx
View file

@ -1,39 +0,0 @@
---
title: Docker Compose Development
description: Building SurfSense from source using docker-compose.dev.yml
---
If you're contributing to SurfSense and want to build from source, use `docker-compose.dev.yml` instead:
```bash
cd SurfSense/docker
docker compose -f docker-compose.dev.yml up --build
```
This file builds the backend and frontend from your local source code (instead
of pulling prebuilt images) and includes pgAdmin for database inspection at
[http://localhost:5050](http://localhost:5050). It intentionally keeps raw
frontend, backend, and zero-cache ports published for debugging. Use the
production `docker-compose.yml` for the default Caddy single-origin setup.
## Dev-Only Environment Variables
The following `.env` variables are **only used by the dev compose file** (they have no effect on the production `docker-compose.yml`):
| Variable | Description | Default |
|----------|-------------|---------|
| `PGADMIN_PORT` | pgAdmin web UI port | `5050` |
| `PGADMIN_DEFAULT_EMAIL` | pgAdmin login email | `admin@surfsense.com` |
| `PGADMIN_DEFAULT_PASSWORD` | pgAdmin login password | `surfsense` |
| `REDIS_PORT` | Exposed Redis port (internal-only in prod) | `6379` |
| `AUTH_TYPE` | Runtime auth mode | `LOCAL` |
| `ETL_SERVICE` | Runtime document parsing service | `DOCLING` |
| `DEPLOYMENT_MODE` | Runtime deployment mode | `self-hosted` |
| `ZERO_CACHE_PORT` | Exposed zero-cache port for debugging | `4848` |
In the production compose file, the frontend reads `AUTH_TYPE`, `ETL_SERVICE`,
and `DEPLOYMENT_MODE` at request time. Browser API and Zero traffic are
same-origin relative through bundled Caddy.
Production Docker exposes only the bundled Caddy proxy by default; dev compose
keeps direct service ports so contributors can inspect and restart individual
services without going through the proxy.

419
surfsense_web/content/docs/docker-installation/docker-compose.mdx
View file

@ -1,419 +0,0 @@
---
title: Docker Compose
description: Manual Docker Compose setup for SurfSense
---
## Setup
```bash
git clone https://github.com/MODSetter/SurfSense.git
cd SurfSense/docker
cp .env.example .env
# Edit .env, at minimum set SECRET_KEY
docker compose up -d
```
After starting, access SurfSense at:
- **SurfSense**: [http://localhost:3929](http://localhost:3929)
- **Backend API**: [http://localhost:3929/api/v1](http://localhost:3929/api/v1)
- **Zero sync**: `ws://localhost:3929/zero`
---
## Configuration
All configuration lives in a single `docker/.env` file (or `surfsense/.env` if you used the install script). Copy `.env.example` to `.env` and edit the values you need.
### Required
| Variable | Description |
|----------|-------------|
| `SECRET_KEY` | JWT secret key. Generate with: `openssl rand -base64 32`. Auto-generated by the install script. |
### Core Settings
| Variable | Description | Default |
|----------|-------------|---------|
| `SURFSENSE_VERSION` | Image tag to deploy. Use `latest`, a clean version (e.g. `0.0.14`), or a specific build (e.g. `0.0.14.1`) | `latest` |
| `SURFSENSE_VARIANT` | Backend image variant. Leave empty for CPU, set `cuda` for CUDA 12.8, or `cuda126` for CUDA 12.6. | *(empty)* |
| `AUTH_TYPE` | Authentication method: `LOCAL` (email/password) or `GOOGLE` (OAuth) | `LOCAL` |
| `ETL_SERVICE` | Document parsing: `DOCLING` (local), `UNSTRUCTURED`, or `LLAMACLOUD` | `DOCLING` |
| `EMBEDDING_MODEL` | Embedding model for vector search | `sentence-transformers/all-MiniLM-L6-v2` |
| `TTS_SERVICE` | Text-to-speech provider for podcasts | `local/kokoro` |
| `STT_SERVICE` | Speech-to-text provider for audio files | `local/base` |
| `REGISTRATION_ENABLED` | Allow new user registrations | `TRUE` |
### Image Variants
SurfSense publishes CPU and CUDA backend image variants. The frontend image is not variant-specific.
| Backend tag | Use case | `SURFSENSE_VARIANT` |
|-------------|----------|---------------------|
| `:latest` | CPU-only default | *(empty)* |
| `:latest-cuda` | NVIDIA CUDA 12.8 backend image | `cuda` |
| `:latest-cuda126` | NVIDIA CUDA 12.6 backend image for older driver stacks | `cuda126` |
All backend variants are published for `linux/amd64` and `linux/arm64`. CUDA on `linux/arm64` is best-effort.
<Callout type="info">
GPU acceleration needs two settings: `SURFSENSE_VARIANT` selects the CUDA image, and `COMPOSE_FILE` enables the GPU device overlay. The host must have the NVIDIA Container Toolkit installed.
</Callout>
### NVIDIA GPU Acceleration
For most NVIDIA systems, add these values to `.env` to use the CUDA 12.8 image:
```dotenv
SURFSENSE_VARIANT=cuda
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu.yml
SURFSENSE_GPU_COUNT=1
```
Use `SURFSENSE_VARIANT=cuda126` for older NVIDIA driver stacks or older GPUs that need the CUDA 12.6 fallback image.
On Windows, use `;` instead of `:` in `COMPOSE_FILE` inside `.env`:
```dotenv
COMPOSE_FILE=docker-compose.yml;docker-compose.gpu.yml
```
To switch variants later, edit `SURFSENSE_VARIANT` and `COMPOSE_FILE` in `.env`, then run:
```bash
docker compose pull
docker compose up -d --wait
```
### Automatic Updates
Manual Docker Compose installs do not start Watchtower automatically. To enable external automatic updates, run Watchtower separately:
```bash
docker run -d --name watchtower \
--restart unless-stopped \
-v /var/run/docker.sock:/var/run/docker.sock \
nickfedor/watchtower \
--label-enable \
--interval 86400
```
SurfSense containers are labeled for Watchtower, so `--label-enable` limits updates to the SurfSense services.
### Public URL and Ports
| Variable | Description | Default |
|----------|-------------|---------|
| `SURFSENSE_PUBLIC_URL` | Public origin used by the frontend, backend OAuth callbacks, and Zero browser URL | `http://localhost:3929` |
| `SURFSENSE_SITE_ADDRESS` | Caddy site address. `:80` means local plain HTTP; a hostname enables automatic HTTPS | `:80` |
| `LISTEN_HTTP_PORT` | Host port mapped to Caddy's HTTP listener | `3929` |
| `LISTEN_HTTPS_PORT` | Host port mapped to Caddy's HTTPS listener for domain mode | `443` |
SurfSense includes Caddy by default. The `frontend`, `backend`, and
`zero-cache` containers are internal-only in the production compose file; the
browser reaches them through Caddy path routing.
### Custom Domain / Automatic HTTPS
For a real domain, point DNS at the Docker host and set:
```dotenv
SURFSENSE_SITE_ADDRESS=surf.example.com
LISTEN_HTTP_PORT=80
LISTEN_HTTPS_PORT=443
CERT_EMAIL=you@example.com
SURFSENSE_PUBLIC_URL=https://surf.example.com
```
Caddy will issue and renew Let's Encrypt certificates automatically. Ports 80
and 443 must be reachable from the internet for the default HTTP-01 challenge.
| Variable | Description |
|----------|-------------|
| `CERT_EMAIL` | Optional ACME contact email |
| `CERT_ACME_CA` | ACME directory URL; use Let's Encrypt staging when testing cert issuance |
| `CERT_ACME_DNS` | DNS-01 challenge config; requires the custom Caddy build |
| `TRUSTED_PROXIES` | CIDR ranges trusted for forwarded client IP headers |
| `SURFSENSE_MAX_BODY_SIZE` | Upload limit enforced at the proxy |
### Bring Your Own Proxy
If you already run nginx, Traefik, Cloudflare Tunnel, or another ingress, you
can comment out the `proxy` service and route traffic to the internal services
with the same path contract:
| Public path | Upstream |
|-------------|----------|
| `/auth/*` | `backend:8000` |
| `/api/v1/*` | `backend:8000` |
| `/zero/*` | `zero-cache:4848` |
| `/*` | `frontend:3000` |
Alternative proxies must preserve WebSocket upgrades for `/zero`, avoid
buffering streaming responses, allow long-running requests, and support large
uploads. For DNS-01 or wildcard certificates with Caddy, build
`docker/proxy/Dockerfile` and set `CERT_ACME_DNS` for your DNS provider.
### Zero-cache (Real-Time Sync)
Defaults work out of the box. Change `ZERO_ADMIN_PASSWORD` for security in production.
| Variable | Description | Default |
|----------|-------------|---------|
| `ZERO_ADMIN_PASSWORD` | Password for the zero-cache admin UI and `/statz` endpoint | `surfsense-zero-admin` |
| `ZERO_UPSTREAM_DB` | PostgreSQL connection URL for replication (must be a direct connection, not via pgbouncer) | *(built from DB_* vars)* |
| `ZERO_CVR_DB` | PostgreSQL connection URL for client view records | *(built from DB_* vars)* |
| `ZERO_CHANGE_DB` | PostgreSQL connection URL for replication log entries | *(built from DB_* vars)* |
| `ZERO_APP_PUBLICATIONS` | PostgreSQL publication restricting which tables are replicated (created by migration 116, verified by the `migrations` service before `zero-cache` starts) | `zero_publication` |
| `ZERO_NUM_SYNC_WORKERS` | Number of view-sync worker processes. Must be ≤ connection pool sizes | `4` |
| `ZERO_UPSTREAM_MAX_CONNS` | Max connections to upstream PostgreSQL for mutations | `20` |
| `ZERO_CVR_MAX_CONNS` | Max connections to the CVR database | `30` |
### Database
Defaults work out of the box. Change for security in production.
| Variable | Description | Default |
|----------|-------------|---------|
| `DB_USER` | PostgreSQL username | `surfsense` |
| `DB_PASSWORD` | PostgreSQL password | `surfsense` |
| `DB_NAME` | PostgreSQL database name | `surfsense` |
| `DB_HOST` | PostgreSQL host | `db` |
| `DB_PORT` | PostgreSQL port | `5432` |
| `DB_SSLMODE` | SSL mode: `disable`, `require`, `verify-ca`, `verify-full` | `disable` |
| `DATABASE_URL` | Full connection URL override. Use for managed databases (RDS, Supabase, etc.) | *(built from above)* |
### Authentication
| Variable | Description |
|----------|-------------|
| `GOOGLE_OAUTH_CLIENT_ID` | Google OAuth client ID (required if `AUTH_TYPE=GOOGLE`) |
| `GOOGLE_OAUTH_CLIENT_SECRET` | Google OAuth client secret (required if `AUTH_TYPE=GOOGLE`) |
Create credentials at the [Google Cloud Console](https://console.cloud.google.com/apis/credentials).
### External API Keys
| Variable | Description |
|----------|-------------|
| `UNSTRUCTURED_API_KEY` | [Unstructured.io](https://unstructured.io/) API key (required if `ETL_SERVICE=UNSTRUCTURED`) |
| `LLAMA_CLOUD_API_KEY` | [LlamaCloud](https://cloud.llamaindex.ai/) API key (required if `ETL_SERVICE=LLAMACLOUD`) |
### Connector OAuth Keys
Uncomment the connectors you want to use. Redirect URIs follow the single-origin
pattern `${SURFSENSE_PUBLIC_URL}/api/v1/auth/<connector>/connector/callback`.
For local Docker defaults, that means
`http://localhost:3929/api/v1/auth/<connector>/connector/callback`.
| Connector | Variables |
|-----------|-----------|
| Google Drive / Gmail / Calendar | `GOOGLE_DRIVE_REDIRECT_URI`, `GOOGLE_GMAIL_REDIRECT_URI`, `GOOGLE_CALENDAR_REDIRECT_URI` |
| Notion | `NOTION_CLIENT_ID`, `NOTION_CLIENT_SECRET`, `NOTION_REDIRECT_URI` |
| Slack | `SLACK_CLIENT_ID`, `SLACK_CLIENT_SECRET`, `SLACK_REDIRECT_URI` |
| Discord | `DISCORD_CLIENT_ID`, `DISCORD_CLIENT_SECRET`, `DISCORD_BOT_TOKEN`, `DISCORD_REDIRECT_URI` |
| Atlassian (Jira & Confluence) | `ATLASSIAN_CLIENT_ID`, `ATLASSIAN_CLIENT_SECRET`, `JIRA_REDIRECT_URI`, `CONFLUENCE_REDIRECT_URI` |
| Linear | `LINEAR_CLIENT_ID`, `LINEAR_CLIENT_SECRET`, `LINEAR_REDIRECT_URI` |
| ClickUp | `CLICKUP_CLIENT_ID`, `CLICKUP_CLIENT_SECRET`, `CLICKUP_REDIRECT_URI` |
| Airtable | `AIRTABLE_CLIENT_ID`, `AIRTABLE_CLIENT_SECRET`, `AIRTABLE_REDIRECT_URI` |
| Microsoft (Teams & OneDrive) | `MICROSOFT_CLIENT_ID`, `MICROSOFT_CLIENT_SECRET`, `TEAMS_REDIRECT_URI`, `ONEDRIVE_REDIRECT_URI` |
| Dropbox | `DROPBOX_APP_KEY`, `DROPBOX_APP_SECRET`, `DROPBOX_REDIRECT_URI` |
### Messaging Channels
Configure these in the same `docker/.env` file when you want users to chat with
SurfSense from external apps. See [Messaging Channels](/docs/messaging-channels)
for full setup.
| Channel | Variables |
|---------|-----------|
| Telegram | `TELEGRAM_SHARED_BOT_TOKEN`, `TELEGRAM_SHARED_BOT_USERNAME`, `TELEGRAM_WEBHOOK_SECRET`, `GATEWAY_BASE_URL`, `GATEWAY_TELEGRAM_INTAKE_MODE` |
| WhatsApp | `GATEWAY_WHATSAPP_INTAKE_MODE`, `WHATSAPP_SHARED_BUSINESS_TOKEN`, `WHATSAPP_SHARED_PHONE_NUMBER_ID`, `WHATSAPP_SHARED_DISPLAY_PHONE_NUMBER`, `WHATSAPP_SHARED_WABA_ID`, `WHATSAPP_WEBHOOK_VERIFY_TOKEN`, `WHATSAPP_WEBHOOK_APP_SECRET` |
| Slack | `SLACK_CLIENT_ID`, `SLACK_CLIENT_SECRET`, `GATEWAY_SLACK_ENABLED`, `GATEWAY_SLACK_SIGNING_SECRET`, `GATEWAY_SLACK_REDIRECT_URI` |
| Discord | `DISCORD_CLIENT_ID`, `DISCORD_CLIENT_SECRET`, `DISCORD_BOT_TOKEN`, `GATEWAY_DISCORD_ENABLED`, `GATEWAY_DISCORD_REDIRECT_URI` |
### Observability (optional)
| Variable | Description |
|----------|-------------|
| `LANGSMITH_TRACING` | Enable LangSmith tracing (`true` / `false`) |
| `LANGSMITH_ENDPOINT` | LangSmith API endpoint |
| `LANGSMITH_API_KEY` | LangSmith API key |
| `LANGSMITH_PROJECT` | LangSmith project name |
### Advanced (optional)
| Variable | Description | Default |
|----------|-------------|---------|
| `SCHEDULE_CHECKER_INTERVAL` | How often to check for scheduled connector tasks (e.g. `5m`, `1h`) | `5m` |
| `RERANKERS_ENABLED` | Enable document reranking for improved search | `FALSE` |
| `RERANKERS_MODEL_NAME` | Reranker model name (e.g. `ms-marco-MiniLM-L-12-v2`) | |
| `RERANKERS_MODEL_TYPE` | Reranker model type (e.g. `flashrank`) | |
| `PAGES_LIMIT` | Max pages per user for ETL services | unlimited |
---
## Docker Services
| Service | Description |
|---------|-------------|
| `proxy` | Caddy reverse proxy; the only public ingress in production Docker |
| `db` | PostgreSQL with pgvector extension |
| `migrations` | Short-lived: runs `alembic upgrade head` and verifies `zero_publication`, then exits |
| `redis` | Message broker for Celery |
| `searxng` | Local privacy-respecting search backend |
| `backend` | FastAPI application server |
| `celery_worker` | Background task processing (document indexing, etc.) |
| `celery_beat` | Periodic task scheduler (connector sync) |
| `zero-cache` | Rocicorp Zero real-time sync (replicates Postgres to clients) |
| `frontend` | Next.js web application, internal behind Caddy |
All services start automatically with `docker compose up -d`.
### How startup ordering works
Schema migrations run as a dedicated `migrations` service that exits 0 on
success and non-zero on failure. Every other backend-image service gates on
it via `condition: service_completed_successfully`:
```text
db (healthy) ──▶ migrations (alembic upgrade head + verify zero_publication)
│
├── exit 0 ─▶ backend ──▶ frontend
│ celery_worker
│ celery_beat
│ zero-cache ──▶ frontend
│
└── exit ≠ 0 ─▶ compose halts the rest of the stack
```
This guarantees `zero-cache` only starts after `zero_publication` exists in
Postgres. Before this design, a silent migration failure would leave
`zero-cache` crash-looping with `Unknown or invalid publications. Specified:
[zero_publication]. Found: []`.
### Readiness vs liveness
The backend exposes two endpoints:
- `GET /health`: lightweight liveness probe (always returns 200 if the
process is up).
- `GET /ready`: readiness probe that confirms `zero_publication` exists.
Returns 503 if not. The compose `backend.healthcheck` uses `/ready` so the
container only reports `healthy` once the schema is actually usable by
zero-cache.
You can also monitor startup progress with `docker compose ps` (look for
`(health: starting)` → `(healthy)`). The install script polls these states
automatically and times out after 5 minutes if the stack does not converge.
---
## Useful Commands
```bash
# View logs (all services)
docker compose logs -f
# View logs for a specific service
docker compose logs -f backend
# Stop all services
docker compose down
# Restart a specific service
docker compose restart backend
# Stop and remove all containers + volumes (destructive!)
docker compose down -v
```
---
## Troubleshooting
- **Port already in use**: Change `LISTEN_HTTP_PORT` in `.env` and restart. In domain mode, use ports `80` and `443` so Caddy can complete certificate issuance.
- **Permission errors on Linux**: You may need to prefix `docker` commands with `sudo`.
- **Real-time updates not working**: Open DevTools → Console and check for WebSocket errors. In production Docker the expected URL is `${SURFSENSE_PUBLIC_URL}/zero`.
- **Line ending issues on Windows**: Run `git config --global core.autocrlf true` before cloning.
### Migration service exited non-zero
The `migrations` service exits non-zero in two cases:
1. `alembic upgrade head` failed (timeout or SQL error).
2. `alembic` succeeded but `zero_publication` is still missing from
`pg_publication`.
Inspect the logs and the alembic state:
```bash
docker compose logs migrations
docker compose exec db psql -U surfsense -d surfsense \
-c 'SELECT * FROM alembic_version;'
docker compose exec db psql -U surfsense -d surfsense \
-c 'SELECT pubname FROM pg_publication;'
```
The default migration timeout is 900 seconds. Slow disks (Windows / WSL2)
may need more. Set `MIGRATION_TIMEOUT` in `.env` to increase it.
### Zero-cache stuck on `Unknown or invalid publications`
Symptom (in `docker compose logs zero-cache`):
```text
Error: Unknown or invalid publications. Specified: [zero_publication]. Found: []
```
This means `zero-cache` started before `zero_publication` was created or the
publication does not match SurfSense's canonical Zero shape. With the current
compose files this should be impossible: the `migrations` service blocks
`zero-cache` from starting and verifies the publication before exiting
successfully. If you see it, your stack predates the fix or you brought up
`zero-cache` manually with `docker compose up zero-cache` before the migrations
service ran.
Recovery:
```bash
docker compose down
docker volume rm surfsense-zero-cache # wipe half-built SQLite replica
docker compose up -d # migrations runs first, then zero-cache
```
### Zero-cache crashes with `_zero.tableMetadata` errors
This indicates a half-initialized SQLite replica left behind by a previous
crash. Zero's own event triggers and `ZERO_AUTO_RESET` handle schema and
replication halts automatically. If the local SQLite replica is wedged, run the
recovery one-liner above to wipe `surfsense-zero-cache`; zero-cache will
re-sync from Postgres on the next start.
### Ensuring `wal_level = logical`
Logical replication is required by zero-cache. The bundled
`docker/postgresql.conf` sets `wal_level = logical` automatically. If you
swap in your own config or use a managed Postgres, confirm with:
```bash
docker compose exec db psql -U surfsense -d surfsense \
-c "SHOW wal_level;"
```
### Using `docker-compose.deps-only.yml`
`docker-compose.deps-only.yml` runs only the dependencies (Postgres, Redis,
SearXNG, zero-cache) on Docker while the backend and frontend run on the
host. Because there is no backend container in this stack, there is no
`migrations` service either, and you must run alembic on the host **before**
bringing the stack up:
```bash
cd surfsense_backend
uv run alembic upgrade head
cd ../docker
docker compose -f docker-compose.deps-only.yml up -d
```
If you skip the alembic step, `zero-cache` will crash-loop with `Unknown or
invalid publications. Specified: [zero_publication]`.

205
surfsense_web/content/docs/docker-installation/index.mdx
View file

@ -1,36 +1,181 @@
---
title: Docker Installation
description: Deploy SurfSense using Docker
description: Run SurfSense with Docker in minutes
---
import { Card, Cards } from 'fumadocs-ui/components/card';
Docker is the recommended way to run SurfSense. Everything — database, backend, frontend, background workers, real-time sync, and a reverse proxy — comes pre-configured.
Choose your preferred Docker deployment method below.
**Prerequisites:** [Docker Desktop](https://www.docker.com/products/docker-desktop/) (or Docker Engine with Compose) must be installed and running.
<Cards>
<Card
title="One-Line Install Script"
description="One-command installation of SurfSense using Docker"
href="/docs/docker-installation/install-script"
/>
<Card
title="Docker Compose"
description="Manual Docker Compose setup for SurfSense"
href="/docs/docker-installation/docker-compose"
/>
<Card
title="Updating"
description="How to update your SurfSense Docker deployment"
href="/docs/docker-installation/updating"
/>
<Card
title="Docker Compose Development"
description="Building SurfSense from source using docker-compose.dev.yml"
href="/docs/docker-installation/dev-compose"
/>
<Card
title="Migrate from the All-in-One Container"
description="Migrate your data from the legacy all-in-one Docker image"
href="/docs/docker-installation/migrate-from-allinone"
/>
</Cards>
## Option 1: One-Line Install Script (Recommended)
**Linux/macOS:**
```bash
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash
```
**Windows (PowerShell):**
```powershell
irm https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.ps1 | iex
```
The script creates a `./surfsense/` directory with the compose files and a `.env`, generates a secret key, starts all services, and waits until they're healthy. It also:
- Detects NVIDIA GPUs and asks whether to use GPU acceleration, picking the compatible backend image automatically.
- Asks whether to enable automatic daily updates via [Watchtower](https://github.com/nicholas-fedor/watchtower) (updates can download several GB in the background — pass `--no-watchtower` to skip).
- Detects a legacy all-in-one installation and migrates its data automatically (see [Updating](/docs/docker-installation/updating#migrating-from-the-all-in-one-container)).
## Option 2: Manual Docker Compose
```bash
git clone https://github.com/MODSetter/SurfSense.git
cd SurfSense/docker
cp .env.example .env
# Edit .env — at minimum set SECRET_KEY (generate with: openssl rand -base64 32)
docker compose up -d
```
## Access SurfSense
After starting, everything is served through the bundled Caddy reverse proxy on a single origin:
- **SurfSense**: [http://localhost:3929](http://localhost:3929)
- **Backend API**: [http://localhost:3929/api/v1](http://localhost:3929/api/v1)
- **Zero sync**: `ws://localhost:3929/zero`
Sign up with email/password (local auth is the default) and you're in.
## Configuration
All configuration lives in a single `.env` file — `surfsense/.env` if you used the install script, `docker/.env` if you cloned manually. The bundled `.env.example` documents every option inline with comments and working defaults; edit the values you need and restart:
```bash
docker compose up -d
```
The defaults give you local email/password auth, local document parsing with Docling (no API keys), and local TTS/STT. Common things you might change, all documented in the file:
- **Authentication** — switch to Google OAuth login.
- **Document parsing** — switch to Unstructured or LlamaCloud (both need API keys).
- **Connector credentials** — OAuth apps for [external connectors](/docs/connectors/external) that need them on self-hosted deployments.
- **Messaging channels** — Telegram, WhatsApp, Slack, and Discord bots (see [Messaging Channels](/docs/messaging-channels)).
### NVIDIA GPU Acceleration
Add these values to `.env` to use the CUDA backend image (the host needs the NVIDIA Container Toolkit):
```dotenv
SURFSENSE_VARIANT=cuda
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu.yml
SURFSENSE_GPU_COUNT=1
```
Use `SURFSENSE_VARIANT=cuda126` for older NVIDIA driver stacks. On Windows, use `;` instead of `:` in `COMPOSE_FILE`. Then apply:
```bash
docker compose pull && docker compose up -d --wait
```
### Custom Domain / Automatic HTTPS
Point DNS at your Docker host and set in `.env`:
```dotenv
SURFSENSE_SITE_ADDRESS=surf.example.com
LISTEN_HTTP_PORT=80
LISTEN_HTTPS_PORT=443
CERT_EMAIL=you@example.com
SURFSENSE_PUBLIC_URL=https://surf.example.com
```
Then `docker compose up -d --wait`. Caddy issues and renews Let's Encrypt certificates automatically — ports 80 and 443 must be reachable from the internet.
### Bring Your Own Proxy
If you already run nginx, Traefik, Cloudflare Tunnel, or another ingress, comment out the `proxy` service and route traffic to the internal services with the same path contract:
| Public path | Upstream |
|-------------|----------|
| `/auth/*` | `backend:8000` |
| `/api/v1/*` | `backend:8000` |
| `/zero/*` | `zero-cache:4848` |
| `/*` | `frontend:3000` |
Your proxy must preserve WebSocket upgrades for `/zero`, avoid buffering streaming responses, allow long-running requests, and support large uploads.
## What's Running
| Service | Description |
|---------|-------------|
| `proxy` | Caddy reverse proxy — the only public ingress |
| `db` | PostgreSQL with pgvector |
| `migrations` | Short-lived: runs database migrations, then exits |
| `redis` | Message broker for background tasks |
| `backend` | FastAPI application server |
| `celery_worker` | Background task processing (document indexing, etc.) |
| `celery_beat` | Periodic task scheduler |
| `zero-cache` | Real-time sync (replicates Postgres to browsers) |
| `frontend` | Next.js web application |
Migrations run first on every startup; everything else waits for them to succeed, so `docker compose up -d` after an update is always safe. Monitor startup with `docker compose ps` — services go from `(health: starting)` to `(healthy)`.
## Useful Commands
```bash
# View logs (all services, or one)
docker compose logs -f
docker compose logs -f backend
# Restart a service
docker compose restart backend
# Stop all services
docker compose down
# Stop and remove containers + volumes (destroys your data!)
docker compose down -v
```
To update SurfSense, see [Updating](/docs/docker-installation/updating).
## Building from Source (Contributors)
If you're contributing to SurfSense and want to build the images from your local checkout, use the dev compose file:
```bash
cd SurfSense/docker
docker compose -f docker-compose.dev.yml up --build
```
It builds the backend and frontend from source, publishes raw service ports for debugging, and includes pgAdmin at [http://localhost:5050](http://localhost:5050). There's also a `docker-compose.deps-only.yml` that runs just the dependencies (Postgres, Redis, zero-cache) in Docker while you run the backend and frontend natively — see [Manual Installation](/docs/manual-installation#alternative-let-docker-manage-all-dependencies).
## Troubleshooting
- **Port already in use**: Change `LISTEN_HTTP_PORT` in `.env` and restart. In domain mode, ports 80/443 are required for certificate issuance.
- **Permission errors on Linux**: Prefix `docker` commands with `sudo`, or add your user to the `docker` group.
- **Real-time updates not working**: Open DevTools → Console and check for WebSocket errors. The expected URL is `${SURFSENSE_PUBLIC_URL}/zero`.
- **Line ending issues on Windows**: Run `git config --global core.autocrlf true` before cloning.
### Migration service exited non-zero
The stack halts if migrations fail. Inspect what happened:
```bash
docker compose logs migrations
```
Slow disks (Windows / WSL2) may hit the default 900-second migration timeout — set `MIGRATION_TIMEOUT` in `.env` to increase it.
### Zero-cache stuck on `Unknown or invalid publications`
This means zero-cache started before migrations created its publication — usually only possible if you started services individually. Recover with:
```bash
docker compose down
docker volume rm surfsense-zero-cache # wipe the half-built replica
docker compose up -d # migrations run first, then zero-cache
```
The same recovery fixes `_zero.tableMetadata` crashes (a half-initialized replica left behind by a previous crash).

100
surfsense_web/content/docs/docker-installation/install-script.mdx
View file

@ -1,100 +0,0 @@
---
title: One-Line Install Script
description: One-command installation of SurfSense using Docker
---
Downloads the compose files, generates a `SECRET_KEY`, starts all services with `docker compose up -d --wait`, and starts [Watchtower](https://github.com/nicholas-fedor/watchtower) as an external updater for automatic daily updates.
**Prerequisites:** [Docker Desktop](https://www.docker.com/products/docker-desktop/) must be installed and running.
### For Linux/macOS users:
```bash
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash
```
### For Windows users (PowerShell):
```powershell
irm https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.ps1 | iex
```
This creates a `./surfsense/` directory with `docker-compose.yml`, `docker-compose.gpu.yml`, and `.env`, then runs `docker compose up -d --wait`.
If an NVIDIA GPU and NVIDIA Container Toolkit are detected, the installer asks whether to use GPU acceleration and chooses the compatible backend image automatically. Non-interactive installs default to CPU unless you pass an explicit flag.
Interactive installs also ask whether to enable automatic daily updates with Watchtower, noting that updates may download several GB in the background.
### GPU options
Linux/macOS:
```bash
# CUDA 12.8
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash -s -- --variant=cuda
# CUDA 12.6 fallback for older driver stacks
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash -s -- --variant=cuda126
# Reserve all available GPUs
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash -s -- --gpu --gpu-count=all
```
PowerShell:
```powershell
# Save the script locally first when passing PowerShell parameters.
.\install.ps1 -Variant cuda
.\install.ps1 -Variant cuda126 -GpuCount all
```
The installer writes the same `.env` settings you would configure manually: `SURFSENSE_VARIANT` selects the backend image and `COMPOSE_FILE` enables the GPU overlay.
To skip Watchtower (e.g. in production where you manage updates yourself, or to avoid large background image downloads):
```bash
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash -s -- --no-watchtower
```
To customise the check interval (default 24h), use `--watchtower-interval=SECONDS`.
Manual updates use the same compose state stored in `.env`, so GPU overlays and variants are preserved:
```bash
cd surfsense
docker compose pull
docker compose up -d --wait
```
If Watchtower is enabled, it preserves the running image variant tag automatically. Because SurfSense images are large, use `--no-watchtower` when you prefer to manage update timing yourself.
---
## Access SurfSense
After starting, access SurfSense at:
- **SurfSense**: [http://localhost:3929](http://localhost:3929)
- **Backend API**: [http://localhost:3929/api/v1](http://localhost:3929/api/v1)
- **Zero sync**: `ws://localhost:3929/zero`
The installer uses the bundled Caddy reverse proxy by default. The backend and
zero-cache containers are not published on separate host ports in the production
stack.
For a custom domain, edit `surfsense/.env` after installation:
```dotenv
SURFSENSE_SITE_ADDRESS=surf.example.com
LISTEN_HTTP_PORT=80
LISTEN_HTTPS_PORT=443
CERT_EMAIL=you@example.com
SURFSENSE_PUBLIC_URL=https://surf.example.com
```
Then run:
```bash
cd surfsense
docker compose up -d --wait
```

2
surfsense_web/content/docs/docker-installation/meta.json
View file

@ -1,6 +1,6 @@
{
"title": "Docker Installation",
"pages": ["install-script", "docker-compose", "updating", "dev-compose", "migrate-from-allinone"],
"pages": ["updating"],
"icon": "Container",
"defaultOpen": false
}

114
surfsense_web/content/docs/docker-installation/migrate-from-allinone.mdx
View file

@ -1,114 +0,0 @@
---
title: Migrate from the All-in-One Container
description: How to migrate your data from the legacy surfsense all-in-one Docker image to the current multi-container setup
---
The original SurfSense all-in-one image (`ghcr.io/modsetter/surfsense:latest`, run via `docker-compose.quickstart.yml`) stored all data — PostgreSQL, Redis, and configuration — in a single Docker volume named `surfsense-data`. The current setup uses separate named volumes and has upgraded PostgreSQL from **version 14 to 17**.
Because PostgreSQL data files are not compatible between major versions, a **logical dump and restore** is required. This is a one-time migration.
<Callout type="warn">
This guide only applies to users who ran the legacy `docker-compose.quickstart.yml` (the all-in-one `surfsense` container). If you were already using `docker/docker-compose.yml`, you do not need to migrate.
</Callout>
---
## Option A — One command (recommended)
`install.sh` detects the legacy `surfsense-data` volume and handles the full migration automatically — no separate migration script needed. Just run the same install command you would use for a fresh install:
```bash
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash
```
**What it does automatically:**
1. Downloads all SurfSense files (including `migrate-database.sh`) into `./surfsense/`
2. Detects the `surfsense-data` volume and enters migration mode
3. Stops the old all-in-one container if it is still running
4. Starts a temporary PostgreSQL 14 container and dumps your database
5. Recovers your `SECRET_KEY` from the old volume
6. Starts PostgreSQL 17, restores the dump, runs a smoke test
7. Starts all services
Your original `surfsense-data` volume is **never deleted** — you remove it manually after verifying.
### After it completes
1. Open [http://localhost:3000](http://localhost:3000) and confirm your data is intact.
2. Once satisfied, remove the old volume (irreversible):
```bash
docker volume rm surfsense-data
```
3. Delete the dump file once you no longer need it as a backup:
```bash
rm ./surfsense_migration_backup.sql
```
### If the migration fails mid-way
The dump file is saved to `./surfsense_migration_backup.sql` as a checkpoint. Simply re-run `install.sh` — it will detect the existing dump and skip straight to the restore step without re-extracting.
---
## Option B — Manual migration script (custom credentials)
If you launched the old all-in-one container with custom database credentials (`POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB` environment variables), the automatic path will use wrong credentials. Run `migrate-database.sh` manually first:
```bash
# 1. Extract data with your custom credentials
bash ./surfsense/scripts/migrate-database.sh --db-user myuser --db-password mypass --db-name mydb
# 2. Install and restore (detects the dump automatically)
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash
```
Or download and run if you haven't run `install.sh` yet:
```bash
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/migrate-database.sh -o migrate-database.sh
bash migrate-database.sh --db-user myuser --db-password mypass --db-name mydb
```
### Migration script options
| Flag | Description | Default |
|------|-------------|---------|
| `--db-user USER` | Old PostgreSQL username | `surfsense` |
| `--db-password PASS` | Old PostgreSQL password | `surfsense` |
| `--db-name NAME` | Old PostgreSQL database | `surfsense` |
| `--yes` / `-y` | Skip confirmation prompts (used automatically by `install.sh`) | — |
---
## Troubleshooting
### `install.sh` runs normally with a blank database (no migration happened)
The legacy volume was not detected. Confirm it exists:
```bash
docker volume ls | grep surfsense-data
```
If it doesn't appear, the old container may have used a different volume name. Check with:
```bash
docker volume ls | grep -i surfsense
```
### Extraction fails with permission errors
The script detects the UID of the data files and runs the temporary PG14 container as that user. If you see permission errors in `./surfsense-migration.log`, run `migrate-database.sh` manually and check the log for details.
### Cannot find `/data/.secret_key`
The all-in-one entrypoint always writes the key to `/data/.secret_key` unless you explicitly set `SECRET_KEY=` as an environment variable. If the key is missing, the migration script auto-generates a new one (with a warning). You can update it manually in `./surfsense/.env` afterwards. Note that a new key invalidates all existing browser sessions — users will need to log in again.
### Restore errors after re-running `install.sh`
If `surfsense-postgres` volume already exists from a previous partial run, remove it before retrying:
```bash
docker volume rm surfsense-postgres
```

54
surfsense_web/content/docs/docker-installation/updating.mdx
View file

@ -3,11 +3,20 @@ title: Updating
description: How to update your SurfSense Docker deployment
---
## Watchtower Daemon (recommended)
## Manual Update
Auto-updates every 24 hours. If you used the [install script](/docs/docker-installation/install-script), Watchtower is already running. No extra setup needed.
```bash
cd surfsense # or SurfSense/docker if you cloned manually
docker compose pull && docker compose up -d
```
For [manual Docker Compose](/docs/docker-installation/docker-compose) installs, start Watchtower separately:
Database migrations are applied automatically on every startup. GPU overlays and image variants set in `.env` are preserved.
## Automatic Updates with Watchtower
Auto-updates every 24 hours. If you used the [install script](/docs/docker-installation) and enabled updates, Watchtower is already running — no extra setup needed.
For manual Docker Compose installs, start Watchtower separately:
```bash
docker run -d --name watchtower \
@ -18,7 +27,9 @@ docker run -d --name watchtower \
--interval 86400
```
## Watchtower One-Time Update
SurfSense containers are labeled for Watchtower, so `--label-enable` limits updates to the SurfSense services.
For a one-time update instead of a daemon:
```bash
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
@ -30,21 +41,32 @@ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
Use `nickfedor/watchtower`. The original `containrrr/watchtower` is no longer maintained and may fail with newer Docker versions.
</Callout>
## Manual Update
```bash
cd surfsense # or SurfSense/docker if you cloned manually
docker compose pull && docker compose up -d
```
Database migrations are applied automatically on every startup.
---
## Migrating from the All-in-One Container
<Callout type="warn">
If you were previously using `docker-compose.quickstart.yml` (the legacy all-in-one `surfsense` container), your data lives in a `surfsense-data` volume and requires a **one-time migration** before switching to the current setup. PostgreSQL has been upgraded from version 14 to 17, so a simple volume swap will not work.
If you previously ran the legacy all-in-one image (`ghcr.io/modsetter/surfsense:latest` via `docker-compose.quickstart.yml`), your data lives in a single `surfsense-data` volume and PostgreSQL has since been upgraded from version 14 to 17 — so a simple volume swap won't work. A one-time dump and restore is required, and the install script does it for you.
See the full step-by-step guide: [Migrate from the All-in-One Container](/docs/docker-installation/migrate-from-allinone).
Just run the normal install command:
```bash
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash
```
It detects the legacy `surfsense-data` volume, stops the old container, dumps your database with a temporary PostgreSQL 14 container, recovers your `SECRET_KEY`, restores everything into PostgreSQL 17, and starts the new stack. Your original volume is **never deleted** — the dump is also saved to `./surfsense_migration_backup.sql` as a checkpoint, so if anything fails mid-way you can simply re-run the script.
After it completes:
1. Open SurfSense and confirm your data is intact.
2. Remove the old volume (irreversible): `docker volume rm surfsense-data`
3. Delete the dump file once you no longer need it: `rm ./surfsense_migration_backup.sql`
<Callout type="info">
If your old container used custom database credentials (`POSTGRES_USER` / `POSTGRES_PASSWORD` / `POSTGRES_DB`), run the migration script manually first with your credentials, then run the installer:
```bash
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/migrate-database.sh -o migrate-database.sh
bash migrate-database.sh --db-user myuser --db-password mypass --db-name mydb
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash
```
</Callout>