From 6d9540a1e81558e1d590401c8de57e2156095ccf Mon Sep 17 00:00:00 2001 From: Anish Sarkar <104695310+AnishSarkar22@users.noreply.github.com> Date: Mon, 15 Jun 2026 11:04:31 +0530 Subject: [PATCH] docs(docker): document single-origin proxy deployment --- .../docs/docker-installation/dev-compose.mdx | 9 ++- .../docker-installation/docker-compose.mdx | 73 ++++++++++++++----- .../docker-installation/install-script.mdx | 28 ++++++- .../content/docs/how-to/zero-sync.mdx | 12 ++- .../docs/messaging-channels/docker.mdx | 18 +++-- 5 files changed, 106 insertions(+), 34 deletions(-) diff --git a/surfsense_web/content/docs/docker-installation/dev-compose.mdx b/surfsense_web/content/docs/docker-installation/dev-compose.mdx index 599e9beb2..0bfe75aa2 100644 --- a/surfsense_web/content/docs/docker-installation/dev-compose.mdx +++ b/surfsense_web/content/docs/docker-installation/dev-compose.mdx @@ -10,7 +10,11 @@ cd SurfSense/docker docker compose -f docker-compose.dev.yml up --build ``` -This file builds the backend and frontend from your local source code (instead of pulling prebuilt images) and includes pgAdmin for database inspection at [http://localhost:5050](http://localhost:5050). Use the production `docker-compose.yml` for all other cases. +This file builds the backend and frontend from your local source code (instead +of pulling prebuilt images) and includes pgAdmin for database inspection at +[http://localhost:5050](http://localhost:5050). It intentionally keeps raw +frontend, backend, and zero-cache ports published for debugging. Use the +production `docker-compose.yml` for the default Caddy single-origin setup. ## Dev-Only Environment Variables @@ -28,3 +32,6 @@ The following `.env` variables are **only used by the dev compose file** (they h | `NEXT_PUBLIC_DEPLOYMENT_MODE` | Frontend build arg for deployment mode | `self-hosted` | In the production compose file, the `NEXT_PUBLIC_*` frontend variables are automatically derived from `AUTH_TYPE`, `ETL_SERVICE`, and the port settings. In the dev compose file, they are passed as build args since the frontend is built from source. +Production Docker exposes only the bundled Caddy proxy by default; dev compose +keeps direct service ports so contributors can inspect and restart individual +services without going through the proxy. diff --git a/surfsense_web/content/docs/docker-installation/docker-compose.mdx b/surfsense_web/content/docs/docker-installation/docker-compose.mdx index 60b5e67b6..bf71c077b 100644 --- a/surfsense_web/content/docs/docker-installation/docker-compose.mdx +++ b/surfsense_web/content/docs/docker-installation/docker-compose.mdx @@ -15,9 +15,9 @@ docker compose up -d After starting, access SurfSense at: -- **Frontend**: [http://localhost:3929](http://localhost:3929) -- **Backend API**: [http://localhost:8929](http://localhost:8929) -- **API Docs**: [http://localhost:8929/docs](http://localhost:8929/docs) +- **SurfSense**: [http://localhost:3929](http://localhost:3929) +- **Backend API**: [http://localhost:3929/api/v1](http://localhost:3929/api/v1) +- **Zero sync**: `ws://localhost:3929/zero` --- ## Configuration @@ -99,24 +99,59 @@ docker run -d --name watchtower \ SurfSense containers are labeled for Watchtower, so `--label-enable` limits updates to the SurfSense services. -### Ports +### Public URL and Ports | Variable | Description | Default | |----------|-------------|---------| -| `FRONTEND_PORT` | Frontend service port | `3929` | -| `BACKEND_PORT` | Backend API service port | `8929` | -| `ZERO_CACHE_PORT` | Zero-cache real-time sync port | `5929` | +| `SURFSENSE_PUBLIC_URL` | Public origin used by the frontend, backend OAuth callbacks, and Zero browser URL | `http://localhost:3929` | +| `SURFSENSE_SITE_ADDRESS` | Caddy site address. `:80` means local plain HTTP; a hostname enables automatic HTTPS | `:80` | +| `LISTEN_HTTP_PORT` | Host port mapped to Caddy's HTTP listener | `3929` | +| `LISTEN_HTTPS_PORT` | Host port mapped to Caddy's HTTPS listener for domain mode | `443` | -### Custom Domain / Reverse Proxy +SurfSense includes Caddy by default. The `frontend`, `backend`, and +`zero-cache` containers are internal-only in the production compose file; the +browser reaches them through Caddy path routing. -Only set these if serving SurfSense on a real domain via a reverse proxy (Caddy, Nginx, Cloudflare Tunnel, etc.). Leave commented out for standard localhost deployments. +### Custom Domain / Automatic HTTPS + +For a real domain, point DNS at the Docker host and set: + +```dotenv +SURFSENSE_SITE_ADDRESS=surf.example.com +LISTEN_HTTP_PORT=80 +LISTEN_HTTPS_PORT=443 +CERT_EMAIL=you@example.com +SURFSENSE_PUBLIC_URL=https://surf.example.com +``` + +Caddy will issue and renew Let's Encrypt certificates automatically. Ports 80 +and 443 must be reachable from the internet for the default HTTP-01 challenge. | Variable | Description | |----------|-------------| -| `NEXT_FRONTEND_URL` | Public frontend URL (e.g. `https://app.yourdomain.com`) | -| `BACKEND_URL` | Public backend URL for OAuth callbacks (e.g. `https://api.yourdomain.com`) | -| `NEXT_PUBLIC_FASTAPI_BACKEND_URL` | Backend URL used by the frontend (e.g. `https://api.yourdomain.com`) | -| `NEXT_PUBLIC_ZERO_CACHE_URL` | Zero-cache URL used by the frontend (e.g. `https://zero.yourdomain.com`) | +| `CERT_EMAIL` | Optional ACME contact email | +| `CERT_ACME_CA` | ACME directory URL; use Let's Encrypt staging when testing cert issuance | +| `CERT_ACME_DNS` | DNS-01 challenge config; requires the custom Caddy build | +| `TRUSTED_PROXIES` | CIDR ranges trusted for forwarded client IP headers | +| `SURFSENSE_MAX_BODY_SIZE` | Upload limit enforced at the proxy | + +### Bring Your Own Proxy + +If you already run nginx, Traefik, Cloudflare Tunnel, or another ingress, you +can comment out the `proxy` service and route traffic to the internal services +with the same path contract: + +| Public path | Upstream | +|-------------|----------| +| `/auth/*` | `backend:8000` | +| `/api/v1/*` | `backend:8000` | +| `/zero/*` | `zero-cache:4848` | +| `/*` | `frontend:3000` | + +Alternative proxies must preserve WebSocket upgrades for `/zero`, avoid +buffering streaming responses, allow long-running requests, and support large +uploads. For DNS-01 or wildcard certificates with Caddy, build +`docker/proxy/Dockerfile` and set `CERT_ACME_DNS` for your DNS provider. ### Zero-cache (Real-Time Sync) @@ -165,7 +200,10 @@ Create credentials at the [Google Cloud Console](https://console.cloud.google.co ### Connector OAuth Keys -Uncomment the connectors you want to use. Redirect URIs follow the pattern `http://localhost:8000/api/v1/auth//connector/callback`. +Uncomment the connectors you want to use. Redirect URIs follow the single-origin +pattern `${SURFSENSE_PUBLIC_URL}/api/v1/auth//connector/callback`. +For local Docker defaults, that means +`http://localhost:3929/api/v1/auth//connector/callback`. | Connector | Variables | |-----------|-----------| @@ -218,6 +256,7 @@ for full setup. | Service | Description | |---------|-------------| +| `proxy` | Caddy reverse proxy; the only public ingress in production Docker | | `db` | PostgreSQL with pgvector extension | | `migrations` | Short-lived: runs `alembic upgrade head` and verifies `zero_publication`, then exits | | `redis` | Message broker for Celery | @@ -226,7 +265,7 @@ for full setup. | `celery_worker` | Background task processing (document indexing, etc.) | | `celery_beat` | Periodic task scheduler (connector sync) | | `zero-cache` | Rocicorp Zero real-time sync (replicates Postgres to clients) | -| `frontend` | Next.js web application | +| `frontend` | Next.js web application, internal behind Caddy | All services start automatically with `docker compose up -d`. @@ -292,9 +331,9 @@ docker compose down -v ## Troubleshooting -- **Ports already in use**: Change the relevant `*_PORT` variable in `.env` and restart. +- **Port already in use**: Change `LISTEN_HTTP_PORT` in `.env` and restart. In domain mode, use ports `80` and `443` so Caddy can complete certificate issuance. - **Permission errors on Linux**: You may need to prefix `docker` commands with `sudo`. -- **Real-time updates not working**: Open DevTools → Console and check for WebSocket errors. Verify `NEXT_PUBLIC_ZERO_CACHE_URL` matches the running zero-cache address. +- **Real-time updates not working**: Open DevTools → Console and check for WebSocket errors. In production Docker the expected URL is `${SURFSENSE_PUBLIC_URL}/zero`. - **Line ending issues on Windows**: Run `git config --global core.autocrlf true` before cloning. ### Migration service exited non-zero diff --git a/surfsense_web/content/docs/docker-installation/install-script.mdx b/surfsense_web/content/docs/docker-installation/install-script.mdx index 9f8acf9e5..fb7e6b5b6 100644 --- a/surfsense_web/content/docs/docker-installation/install-script.mdx +++ b/surfsense_web/content/docs/docker-installation/install-script.mdx @@ -74,7 +74,27 @@ If Watchtower is enabled, it preserves the running image variant tag automatical After starting, access SurfSense at: -- **Frontend**: [http://localhost:3929](http://localhost:3929) -- **Backend API**: [http://localhost:8929](http://localhost:8929) -- **API Docs**: [http://localhost:8929/docs](http://localhost:8929/docs) -- **Zero-cache**: [http://localhost:5929](http://localhost:5929) +- **SurfSense**: [http://localhost:3929](http://localhost:3929) +- **Backend API**: [http://localhost:3929/api/v1](http://localhost:3929/api/v1) +- **Zero sync**: `ws://localhost:3929/zero` + +The installer uses the bundled Caddy reverse proxy by default. The backend and +zero-cache containers are not published on separate host ports in the production +stack. + +For a custom domain, edit `surfsense/.env` after installation: + +```dotenv +SURFSENSE_SITE_ADDRESS=surf.example.com +LISTEN_HTTP_PORT=80 +LISTEN_HTTPS_PORT=443 +CERT_EMAIL=you@example.com +SURFSENSE_PUBLIC_URL=https://surf.example.com +``` + +Then run: + +```bash +cd surfsense +docker compose up -d --wait +``` diff --git a/surfsense_web/content/docs/how-to/zero-sync.mdx b/surfsense_web/content/docs/how-to/zero-sync.mdx index 7007e6637..22ba851b4 100644 --- a/surfsense_web/content/docs/how-to/zero-sync.mdx +++ b/surfsense_web/content/docs/how-to/zero-sync.mdx @@ -32,10 +32,10 @@ zero-cache is included in the Docker Compose setup. The key environment variable | Variable | Description | Default | |----------|-------------|---------| -| `ZERO_CACHE_PORT` | Port for the zero-cache service | `5929` (prod) / `4848` (dev) | +| `SURFSENSE_PUBLIC_URL` | Public SurfSense origin used by the browser | `http://localhost:3929` | | `ZERO_ADMIN_PASSWORD` | Password for the zero-cache admin UI and `/statz` endpoint | `surfsense-zero-admin` | | `ZERO_UPSTREAM_DB` | PostgreSQL connection URL for replication | Built from `DB_*` vars | -| `NEXT_PUBLIC_ZERO_CACHE_URL` | URL the frontend uses to connect to zero-cache | `http://localhost:` | +| `NEXT_PUBLIC_ZERO_CACHE_URL` | URL the frontend uses to connect to zero-cache | `${SURFSENSE_PUBLIC_URL}/zero` | | `ZERO_APP_PUBLICATIONS` | PostgreSQL publication restricting which tables are replicated | `zero_publication` | | `ZERO_NUM_SYNC_WORKERS` | Number of view-sync worker processes. Must be ≤ `ZERO_UPSTREAM_MAX_CONNS` and ≤ `ZERO_CVR_MAX_CONNS` | `4` | | `ZERO_UPSTREAM_MAX_CONNS` | Max connections to upstream PostgreSQL for mutations | `20` | @@ -71,7 +71,11 @@ For the full manual setup walkthrough, see the [Manual Installation guide](/docs ### Custom Domain / Reverse Proxy -When deploying behind a reverse proxy, set `NEXT_PUBLIC_ZERO_CACHE_URL` to your public zero-cache URL (e.g., `https://zero.yourdomain.com`). The zero-cache service must be accessible via WebSocket from the browser. +The production Docker stack includes Caddy by default. Zero is exposed under the +same public origin as the app at `${SURFSENSE_PUBLIC_URL}/zero`, for example +`https://surf.example.com/zero`. Zero accepts this single path-component base +URL, so Caddy forwards `/zero/*` to the internal `zero-cache:4848` service +without stripping the prefix. ### Database Requirements @@ -110,7 +114,7 @@ Zero syncs the following tables for real-time features: - **zero-cache not starting**: Check `docker compose logs zero-cache`. Ensure PostgreSQL has `wal_level=logical` (configured in `postgresql.conf`). - **"Insufficient upstream connections" error**: zero-cache defaults `ZERO_NUM_SYNC_WORKERS` to the number of CPU cores, which can exceed connection pool limits on high-core machines. Lower `ZERO_NUM_SYNC_WORKERS` or raise `ZERO_UPSTREAM_MAX_CONNS` / `ZERO_CVR_MAX_CONNS` in your `.env`. -- **Frontend not syncing**: Open DevTools → Console and check for WebSocket connection errors. Verify `NEXT_PUBLIC_ZERO_CACHE_URL` matches the running zero-cache address. +- **Frontend not syncing**: Open DevTools → Console and check for WebSocket connection errors. In production Docker, verify `NEXT_PUBLIC_ZERO_CACHE_URL` resolves to `${SURFSENSE_PUBLIC_URL}/zero`. - **Stale data after restart**: zero-cache rebuilds its SQLite replica from PostgreSQL on startup. This may take a moment for large databases. ## Learn More diff --git a/surfsense_web/content/docs/messaging-channels/docker.mdx b/surfsense_web/content/docs/messaging-channels/docker.mdx index 3a4d4177f..63acfa0a8 100644 --- a/surfsense_web/content/docs/messaging-channels/docker.mdx +++ b/surfsense_web/content/docs/messaging-channels/docker.mdx @@ -15,17 +15,19 @@ wired by Compose. ## Public URLs For localhost-only testing, the defaults are enough for the SurfSense UI, but -public webhooks from Telegram, WhatsApp, and Slack require a public HTTPS backend -URL. Use your deployed backend URL or a tunnel such as Cloudflare Tunnel or -ngrok. +public webhooks from Telegram, WhatsApp, and Slack require a public HTTPS +SurfSense URL. Use your deployed domain or a tunnel such as Cloudflare Tunnel +or ngrok. -When using a custom domain or tunnel, set: +When using a custom domain or tunnel with the bundled Caddy proxy, set: ```bash -BACKEND_URL=https://api.example.com -GATEWAY_BASE_URL=https://api.example.com -NEXT_FRONTEND_URL=https://app.example.com -NEXT_PUBLIC_FASTAPI_BACKEND_URL=https://api.example.com +SURFSENSE_PUBLIC_URL=https://surf.example.com +SURFSENSE_SITE_ADDRESS=surf.example.com +LISTEN_HTTP_PORT=80 +LISTEN_HTTPS_PORT=443 +CERT_EMAIL=you@example.com +GATEWAY_BASE_URL=https://surf.example.com ``` ## Environment Variables