SurfSense/surfsense_web/content/docs/docker-installation/index.mdx

216 lines
9.4 KiB
Text

---
title: Docker Installation
description: Run SurfSense with Docker in minutes
---
Docker is the recommended way to run SurfSense. Everything — database, backend, frontend, background workers, real-time sync, and a reverse proxy — comes pre-configured.
**Prerequisites:** [Docker Desktop](https://www.docker.com/products/docker-desktop/) (or Docker Engine with Compose) must be installed and running.
## Option 1: One-Line Install Script (Recommended)
**Linux/macOS:**
```bash
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash
```
**Windows (PowerShell):**
```powershell
irm https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.ps1 | iex
```
The script creates a `./surfsense/` directory with the compose files and a `.env`, generates a secret key, starts all services, and waits until they're healthy. It also:
- Detects NVIDIA GPUs and asks whether to use GPU acceleration, picking the compatible backend image automatically.
- Asks whether to enable automatic daily updates via [Watchtower](https://github.com/nicholas-fedor/watchtower) (updates can download several GB in the background — pass `--no-watchtower` to skip).
- Detects a legacy all-in-one installation and migrates its data automatically (see [Updating](/docs/docker-installation/updating#migrating-from-the-all-in-one-container)).
## Option 2: Manual Docker Compose
```bash
git clone https://github.com/MODSetter/SurfSense.git
cd SurfSense/docker
cp .env.example .env
# Edit .env — at minimum set SECRET_KEY (generate with: openssl rand -base64 32)
docker compose up -d
```
## Access SurfSense
After starting, everything is served through the bundled Caddy reverse proxy on a single origin:
- **SurfSense**: [http://localhost:3929](http://localhost:3929)
- **Backend API**: [http://localhost:3929/api/v1](http://localhost:3929/api/v1)
- **Zero sync**: `ws://localhost:3929/zero`
Sign up with email/password (local auth is the default) and you're in.
## Configuration
All configuration lives in a single `.env` file — `surfsense/.env` if you used the install script, `docker/.env` if you cloned manually. The bundled `.env.example` documents every option inline with comments and working defaults; edit the values you need and restart:
```bash
docker compose up -d
```
The defaults give you local email/password auth, local document parsing with Docling (no API keys), and local TTS/STT. Common things you might change, all documented in the file:
- **Authentication** — switch to Google OAuth login.
- **Document parsing** — switch to Unstructured or LlamaCloud (both need API keys).
- **Embedding endpoint** — set `EMBEDDING_BASE_URL` for Chonkie/LiteLLM embedding models, or `OLLAMA_EMBEDDING_BASE_URL` as an Ollama-specific fallback.
- **Connector credentials** — OAuth apps for [external connectors](/docs/connectors/external) that need them on self-hosted deployments.
- **Messaging channels** — Telegram, WhatsApp, Slack, and Discord bots (see [Messaging Channels](/docs/messaging-channels)).
### NVIDIA GPU Acceleration
Add these values to `.env` to use the CUDA backend image (the host needs the NVIDIA Container Toolkit):
```dotenv
SURFSENSE_VARIANT=cuda
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu.yml
SURFSENSE_GPU_COUNT=1
```
Use `SURFSENSE_VARIANT=cuda126` for older NVIDIA driver stacks. On Windows, use `;` instead of `:` in `COMPOSE_FILE`. Then apply:
```bash
docker compose pull && docker compose up -d --wait
```
### Custom Domain / Automatic HTTPS
Point DNS at your Docker host and set in `.env`:
```dotenv
SURFSENSE_SITE_ADDRESS=surf.example.com
LISTEN_HTTP_PORT=80
LISTEN_HTTPS_PORT=443
CERT_EMAIL=you@example.com
SURFSENSE_PUBLIC_URL=https://surf.example.com
```
Then `docker compose up -d --wait`. Caddy issues and renews Let's Encrypt certificates automatically — ports 80 and 443 must be reachable from the internet.
### Bring Your Own Proxy
If you already run nginx, Traefik, Cloudflare Tunnel, or another ingress, comment out the `proxy` service and route traffic to the internal services with the same path contract:
| Public path | Upstream |
|-------------|----------|
| `/auth/*` | `backend:8000` |
| `/api/v1/*` | `backend:8000` |
| `/zero/*` | `zero-cache:4848` |
| `/*` | `frontend:3000` |
Your proxy must preserve WebSocket upgrades for `/zero`, avoid buffering streaming responses, allow long-running requests, and support large uploads.
## What's Running
| Service | Description |
|---------|-------------|
| `proxy` | Caddy reverse proxy — the only public ingress |
| `db` | PostgreSQL with pgvector |
| `migrations` | Short-lived: runs database migrations, then exits |
| `redis` | Message broker for background tasks |
| `backend` | FastAPI application server |
| `celery_worker` | Background task processing (document indexing, etc.) |
| `celery_beat` | Periodic task scheduler |
| `zero-cache` | Real-time sync (replicates Postgres to browsers) |
| `frontend` | Next.js web application |
Migrations run first on every startup; everything else waits for them to succeed, so `docker compose up -d` after an update is always safe. Monitor startup with `docker compose ps` — services go from `(health: starting)` to `(healthy)`.
## Useful Commands
```bash
# View logs (all services, or one)
docker compose logs -f
docker compose logs -f backend
# Restart a service
docker compose restart backend
# Stop all services
docker compose down
# Stop and remove containers + volumes (destroys your data!)
docker compose down -v
```
To update SurfSense, see [Updating](/docs/docker-installation/updating).
## Building from Source (Contributors)
If you're contributing to SurfSense and want to build the images from your local checkout, use the dev compose file. Unlike the production setup above, there's no bundled Caddy proxy — each service publishes its own port directly, and you need **three** separate `.env` files instead of one.
**Requirements:** Docker with the `buildx` plugin (needed for the multi-stage `Dockerfile`; a plain `docker build` without BuildKit fails). Check with `docker buildx version` — if missing, install `docker-buildx` (or `docker-buildx-plugin` on some distros).
```bash
git clone https://github.com/<your-fork>/SurfSense.git
cd SurfSense
cp docker/.env.example docker/.env
cp surfsense_backend/.env.example surfsense_backend/.env
cp surfsense_web/.env.example surfsense_web/.env
```
`docker/.env` only configures Docker Compose variable substitution — it is **not** passed into the containers. The backend and frontend read their own `.env` files instead (wired via `env_file:` in `docker-compose.dev.yml`).
Edit before starting:
- **`SECRET_KEY`** in `surfsense_backend/.env` — required, generate with `openssl rand -base64 32`. Note this is separate from `SECRET_KEY` in `docker/.env`; the dev compose file does not forward one to the other.
Everything else in the three example files (auth type, Stripe, connector OAuth credentials, messaging bots, proxy settings, etc.) is optional — only needed if you're testing that specific integration.
```bash
cd docker
docker compose -f docker-compose.dev.yml up --build
```
| Service | Port | Notes |
|---------|------|-------|
| `frontend` | `localhost:3000` | Next.js dev server |
| `backend` | `localhost:8000` | FastAPI, `/ready` for healthcheck |
| `pgadmin` | `localhost:5050` | Postgres GUI |
| `zero-cache` | `localhost:4848` (ws) | Real-time sync |
| `celery_worker` | — | Background task processing, no exposed port |
| `celery_beat` | — | Periodic task scheduler, no exposed port |
| `otel-lgtm` | `localhost:3001` | Grafana + Loki + Tempo + Mimir bundle, for tracing/metrics |
Observability (`otel-lgtm`) isn't needed for regular dev work — it's the heaviest non-essential service, so skip it if you're not debugging traces/metrics:
```bash
docker compose -f docker-compose.dev.yml up --build db redis zero-cache backend celery_worker celery_beat frontend
```
There's also a `docker-compose.deps-only.yml` that runs just the dependencies (Postgres, Redis, zero-cache) in Docker while you run the backend and frontend natively — see [Manual Installation](/docs/manual-installation#alternative-let-docker-manage-all-dependencies).
## Troubleshooting
- **Port already in use**: Change `LISTEN_HTTP_PORT` in `.env` and restart. In domain mode, ports 80/443 are required for certificate issuance.
- **Permission errors on Linux**: Prefix `docker` commands with `sudo`, or add your user to the `docker` group.
- **Real-time updates not working**: Open DevTools → Console and check for WebSocket errors. The expected URL is `${SURFSENSE_PUBLIC_URL}/zero`.
- **Line ending issues on Windows**: Run `git config --global core.autocrlf true` before cloning.
### Migration service exited non-zero
The stack halts if migrations fail. Inspect what happened:
```bash
docker compose logs migrations
```
Slow disks (Windows / WSL2) may hit the default 900-second migration timeout — set `MIGRATION_TIMEOUT` in `.env` to increase it.
### Zero-cache stuck on `Unknown or invalid publications`
This means zero-cache started before migrations created its publication — usually only possible if you started services individually. Recover with:
```bash
docker compose down
docker volume rm surfsense-zero-cache # wipe the half-built replica
docker compose up -d # migrations run first, then zero-cache
```
The same recovery fixes `_zero.tableMetadata` crashes (a half-initialized replica left behind by a previous crash).