diff --git a/surfsense_web/app/sitemap.ts b/surfsense_web/app/sitemap.ts index 4470f53bf..f1f0bad72 100644 --- a/surfsense_web/app/sitemap.ts +++ b/surfsense_web/app/sitemap.ts @@ -212,6 +212,12 @@ export default function sitemap(): MetadataRoute.Sitemap { priority: 0.8, }, // How-to documentation + { + url: "https://www.surfsense.com/docs/how-to/zero-sync", + lastModified, + changeFrequency: "daily", + priority: 0.8, + }, { url: "https://www.surfsense.com/docs/how-to/realtime-collaboration", lastModified, diff --git a/surfsense_web/content/docs/docker-installation/dev-compose.mdx b/surfsense_web/content/docs/docker-installation/dev-compose.mdx index 5c800e450..599e9beb2 100644 --- a/surfsense_web/content/docs/docker-installation/dev-compose.mdx +++ b/surfsense_web/content/docs/docker-installation/dev-compose.mdx @@ -24,6 +24,7 @@ The following `.env` variables are **only used by the dev compose file** (they h | `REDIS_PORT` | Exposed Redis port (internal-only in prod) | `6379` | | `NEXT_PUBLIC_FASTAPI_BACKEND_AUTH_TYPE` | Frontend build arg for auth type | `LOCAL` | | `NEXT_PUBLIC_ETL_SERVICE` | Frontend build arg for ETL service | `DOCLING` | +| `NEXT_PUBLIC_ZERO_CACHE_URL` | Frontend build arg for Zero-cache URL | `http://localhost:4848` | | `NEXT_PUBLIC_DEPLOYMENT_MODE` | Frontend build arg for deployment mode | `self-hosted` | In the production compose file, the `NEXT_PUBLIC_*` frontend variables are automatically derived from `AUTH_TYPE`, `ETL_SERVICE`, and the port settings. In the dev compose file, they are passed as build args since the frontend is built from source. diff --git a/surfsense_web/content/docs/docker-installation/docker-compose.mdx b/surfsense_web/content/docs/docker-installation/docker-compose.mdx index bf0ca762f..1560d3759 100644 --- a/surfsense_web/content/docs/docker-installation/docker-compose.mdx +++ b/surfsense_web/content/docs/docker-installation/docker-compose.mdx @@ -48,6 +48,7 @@ All configuration lives in a single `docker/.env` file (or `surfsense/.env` if y |----------|-------------|---------| | `FRONTEND_PORT` | Frontend service port | `3929` | | `BACKEND_PORT` | Backend API service port | `8929` | +| `ZERO_CACHE_PORT` | Zero-cache real-time sync port | `5929` | ### Custom Domain / Reverse Proxy @@ -58,6 +59,18 @@ Only set these if serving SurfSense on a real domain via a reverse proxy (Caddy, | `NEXT_FRONTEND_URL` | Public frontend URL (e.g. `https://app.yourdomain.com`) | | `BACKEND_URL` | Public backend URL for OAuth callbacks (e.g. `https://api.yourdomain.com`) | | `NEXT_PUBLIC_FASTAPI_BACKEND_URL` | Backend URL used by the frontend (e.g. `https://api.yourdomain.com`) | +| `NEXT_PUBLIC_ZERO_CACHE_URL` | Zero-cache URL used by the frontend (e.g. `https://zero.yourdomain.com`) | + +### Zero-cache (Real-Time Sync) + +Defaults work out of the box. Change `ZERO_ADMIN_PASSWORD` for security in production. + +| Variable | Description | Default | +|----------|-------------|---------| +| `ZERO_ADMIN_PASSWORD` | Password for the zero-cache admin UI and `/statz` endpoint | `surfsense-zero-admin` | +| `ZERO_UPSTREAM_DB` | PostgreSQL connection URL for replication (must be a direct connection, not via pgbouncer) | *(built from DB_* vars)* | +| `ZERO_CVR_DB` | PostgreSQL connection URL for client view records | *(built from DB_* vars)* | +| `ZERO_CHANGE_DB` | PostgreSQL connection URL for replication log entries | *(built from DB_* vars)* | ### Database @@ -136,6 +149,7 @@ Uncomment the connectors you want to use. Redirect URIs follow the pattern `http | `backend` | FastAPI application server | | `celery_worker` | Background task processing (document indexing, etc.) | | `celery_beat` | Periodic task scheduler (connector sync) | +| `zero-cache` | Rocicorp Zero real-time sync (replicates Postgres to clients) | | `frontend` | Next.js web application | All services start automatically with `docker compose up -d`. @@ -169,4 +183,6 @@ docker compose down -v - **Ports already in use**: Change the relevant `*_PORT` variable in `.env` and restart. - **Permission errors on Linux**: You may need to prefix `docker` commands with `sudo`. +- **Zero-cache not starting**: Check `docker compose logs zero-cache`. Ensure PostgreSQL has `wal_level=logical` (configured automatically by the bundled `postgresql.conf`). +- **Real-time updates not working**: Open DevTools → Console and check for WebSocket errors. Verify `NEXT_PUBLIC_ZERO_CACHE_URL` matches the running zero-cache address. - **Line ending issues on Windows**: Run `git config --global core.autocrlf true` before cloning. diff --git a/surfsense_web/content/docs/docker-installation/install-script.mdx b/surfsense_web/content/docs/docker-installation/install-script.mdx index f3f932c4c..50ccc7288 100644 --- a/surfsense_web/content/docs/docker-installation/install-script.mdx +++ b/surfsense_web/content/docs/docker-installation/install-script.mdx @@ -38,3 +38,4 @@ After starting, access SurfSense at: - **Frontend**: [http://localhost:3929](http://localhost:3929) - **Backend API**: [http://localhost:8929](http://localhost:8929) - **API Docs**: [http://localhost:8929/docs](http://localhost:8929/docs) +- **Zero-cache**: [http://localhost:5929](http://localhost:5929) diff --git a/surfsense_web/content/docs/how-to/index.mdx b/surfsense_web/content/docs/how-to/index.mdx index a6571036d..be8b6f8cd 100644 --- a/surfsense_web/content/docs/how-to/index.mdx +++ b/surfsense_web/content/docs/how-to/index.mdx @@ -8,6 +8,11 @@ import { Card, Cards } from 'fumadocs-ui/components/card'; Practical guides to help you get the most out of SurfSense. + ` | + +### Manual / Local Development + +If running the frontend outside Docker (e.g., `pnpm dev`), you need: + +1. A running zero-cache instance pointing at your PostgreSQL database +2. `NEXT_PUBLIC_ZERO_CACHE_URL` set in your `.env` file (default: `http://localhost:4848`) + +### Custom Domain / Reverse Proxy + +When deploying behind a reverse proxy, set `NEXT_PUBLIC_ZERO_CACHE_URL` to your public zero-cache URL (e.g., `https://zero.yourdomain.com`). The zero-cache service must be accessible via WebSocket from the browser. + +## Synced Tables + +Zero syncs the following tables for real-time features: + +| Table | Used By | +|-------|---------| +| `notifications` | Inbox (comments, document processing, connector status) | +| `documents` | Document list, processing status indicators | +| `search_source_connectors` | Connector status, indexing progress | +| `new_chat_messages` | Live chat message sync for shared chats | +| `chat_comments` | Real-time comment threads on AI responses | +| `chat_session_state` | Collaboration indicators (who is typing) | + +## Troubleshooting + +- **zero-cache not starting**: Check `docker compose logs zero-cache`. Ensure PostgreSQL has `wal_level=logical` (configured in `postgresql.conf`). +- **Frontend not syncing**: Open DevTools → Console and check for WebSocket connection errors. Verify `NEXT_PUBLIC_ZERO_CACHE_URL` matches the running zero-cache address. +- **Stale data after restart**: zero-cache rebuilds its SQLite replica from PostgreSQL on startup. This may take a moment for large databases. + +## Learn More + +- [Rocicorp Zero Documentation](https://zero.rocicorp.dev/docs) +- [Zero Schema Reference](https://zero.rocicorp.dev/docs/schema) +- [Zero Deployment Guide](https://zero.rocicorp.dev/docs/deployment) diff --git a/surfsense_web/content/docs/manual-installation.mdx b/surfsense_web/content/docs/manual-installation.mdx index 77dc6e45b..1577b8d8b 100644 --- a/surfsense_web/content/docs/manual-installation.mdx +++ b/surfsense_web/content/docs/manual-installation.mdx @@ -408,6 +408,7 @@ Edit the `.env` file and set: | NEXT_PUBLIC_FASTAPI_BACKEND_URL | Backend URL (e.g., `http://localhost:8000`) | | NEXT_PUBLIC_FASTAPI_BACKEND_AUTH_TYPE | Same value as set in backend AUTH_TYPE i.e `GOOGLE` for OAuth with Google, `LOCAL` for email/password authentication | | NEXT_PUBLIC_ETL_SERVICE | Document parsing service (should match backend ETL_SERVICE): `UNSTRUCTURED`, `LLAMACLOUD`, or `DOCLING` - affects supported file formats in upload interface | +| NEXT_PUBLIC_ZERO_CACHE_URL | URL for Zero-cache real-time sync service (e.g., `http://localhost:4848`) | ### 2. Install Dependencies