docs: add Zero sync documentation and update existing docs

- Create how-to/zero-sync.mdx setup guide
- Update docker-compose.mdx with zero-cache service, env vars, troubleshooting
- Update dev-compose.mdx with Zero build arg
- Update install-script.mdx with zero-cache URL
- Update manual-installation.mdx with Zero frontend env var
- Add Zero sync page to how-to index, meta.json, and sitemap
This commit is contained in:
CREDO23 2026-03-23 18:35:39 +02:00
parent f29a3edcab
commit 6ad5ead320
8 changed files with 105 additions and 1 deletions

View file

@ -24,6 +24,7 @@ The following `.env` variables are **only used by the dev compose file** (they h
| `REDIS_PORT` | Exposed Redis port (internal-only in prod) | `6379` |
| `NEXT_PUBLIC_FASTAPI_BACKEND_AUTH_TYPE` | Frontend build arg for auth type | `LOCAL` |
| `NEXT_PUBLIC_ETL_SERVICE` | Frontend build arg for ETL service | `DOCLING` |
| `NEXT_PUBLIC_ZERO_CACHE_URL` | Frontend build arg for Zero-cache URL | `http://localhost:4848` |
| `NEXT_PUBLIC_DEPLOYMENT_MODE` | Frontend build arg for deployment mode | `self-hosted` |
In the production compose file, the `NEXT_PUBLIC_*` frontend variables are automatically derived from `AUTH_TYPE`, `ETL_SERVICE`, and the port settings. In the dev compose file, they are passed as build args since the frontend is built from source.

View file

@ -48,6 +48,7 @@ All configuration lives in a single `docker/.env` file (or `surfsense/.env` if y
|----------|-------------|---------|
| `FRONTEND_PORT` | Frontend service port | `3929` |
| `BACKEND_PORT` | Backend API service port | `8929` |
| `ZERO_CACHE_PORT` | Zero-cache real-time sync port | `5929` |
### Custom Domain / Reverse Proxy
@ -58,6 +59,18 @@ Only set these if serving SurfSense on a real domain via a reverse proxy (Caddy,
| `NEXT_FRONTEND_URL` | Public frontend URL (e.g. `https://app.yourdomain.com`) |
| `BACKEND_URL` | Public backend URL for OAuth callbacks (e.g. `https://api.yourdomain.com`) |
| `NEXT_PUBLIC_FASTAPI_BACKEND_URL` | Backend URL used by the frontend (e.g. `https://api.yourdomain.com`) |
| `NEXT_PUBLIC_ZERO_CACHE_URL` | Zero-cache URL used by the frontend (e.g. `https://zero.yourdomain.com`) |
### Zero-cache (Real-Time Sync)
Defaults work out of the box. Change `ZERO_ADMIN_PASSWORD` for security in production.
| Variable | Description | Default |
|----------|-------------|---------|
| `ZERO_ADMIN_PASSWORD` | Password for the zero-cache admin UI and `/statz` endpoint | `surfsense-zero-admin` |
| `ZERO_UPSTREAM_DB` | PostgreSQL connection URL for replication (must be a direct connection, not via pgbouncer) | *(built from DB_* vars)* |
| `ZERO_CVR_DB` | PostgreSQL connection URL for client view records | *(built from DB_* vars)* |
| `ZERO_CHANGE_DB` | PostgreSQL connection URL for replication log entries | *(built from DB_* vars)* |
### Database
@ -136,6 +149,7 @@ Uncomment the connectors you want to use. Redirect URIs follow the pattern `http
| `backend` | FastAPI application server |
| `celery_worker` | Background task processing (document indexing, etc.) |
| `celery_beat` | Periodic task scheduler (connector sync) |
| `zero-cache` | Rocicorp Zero real-time sync (replicates Postgres to clients) |
| `frontend` | Next.js web application |
All services start automatically with `docker compose up -d`.
@ -169,4 +183,6 @@ docker compose down -v
- **Ports already in use**: Change the relevant `*_PORT` variable in `.env` and restart.
- **Permission errors on Linux**: You may need to prefix `docker` commands with `sudo`.
- **Zero-cache not starting**: Check `docker compose logs zero-cache`. Ensure PostgreSQL has `wal_level=logical` (configured automatically by the bundled `postgresql.conf`).
- **Real-time updates not working**: Open DevTools → Console and check for WebSocket errors. Verify `NEXT_PUBLIC_ZERO_CACHE_URL` matches the running zero-cache address.
- **Line ending issues on Windows**: Run `git config --global core.autocrlf true` before cloning.

View file

@ -38,3 +38,4 @@ After starting, access SurfSense at:
- **Frontend**: [http://localhost:3929](http://localhost:3929)
- **Backend API**: [http://localhost:8929](http://localhost:8929)
- **API Docs**: [http://localhost:8929/docs](http://localhost:8929/docs)
- **Zero-cache**: [http://localhost:5929](http://localhost:5929)

View file

@ -8,6 +8,11 @@ import { Card, Cards } from 'fumadocs-ui/components/card';
Practical guides to help you get the most out of SurfSense.
<Cards>
<Card
title="Real-Time Sync with Zero"
description="How SurfSense uses Rocicorp Zero for instant real-time data synchronization"
href="/docs/how-to/zero-sync"
/>
<Card
title="Realtime Collaboration"
description="Invite teammates, share chats, and collaborate in realtime"

View file

@ -1,6 +1,6 @@
{
"title": "How to",
"pages": ["realtime-collaboration", "web-search"],
"pages": ["zero-sync", "realtime-collaboration", "web-search"],
"icon": "Compass",
"defaultOpen": false
}

View file

@ -0,0 +1,74 @@
---
title: Real-Time Sync with Zero
description: How SurfSense uses Rocicorp Zero for instant real-time data synchronization
---
# Real-Time Sync with Zero
SurfSense uses [Rocicorp Zero](https://zero.rocicorp.dev/) for real-time data synchronization. Zero continuously replicates data from PostgreSQL to a local cache on each client, enabling instant UI updates for notifications, documents, connectors, chat messages, and comments.
## How It Works
Zero runs a **zero-cache** server that sits between PostgreSQL and the browser:
1. **zero-cache** replicates data from PostgreSQL into a local SQLite replica using logical replication
2. The browser connects to zero-cache via WebSocket and syncs relevant data locally
3. When data changes in PostgreSQL (e.g., a new notification), zero-cache pushes the update to all connected clients instantly
4. Queries run against local data first for instant results, then update when server data arrives
## Architecture
| Component | Role |
|-----------|------|
| **PostgreSQL** | Source of truth (with `wal_level=logical`) |
| **zero-cache** | Replicates Postgres → SQLite, serves client sync via WebSocket |
| **Browser** | Stores synced data locally, runs queries against local cache |
## Configuration
### Docker Deployment
zero-cache is included in the Docker Compose setup. The key environment variables are:
| Variable | Description | Default |
|----------|-------------|---------|
| `ZERO_CACHE_PORT` | Port for the zero-cache service | `5929` (prod) / `4848` (dev) |
| `ZERO_ADMIN_PASSWORD` | Password for the zero-cache admin UI and `/statz` endpoint | `surfsense-zero-admin` |
| `ZERO_UPSTREAM_DB` | PostgreSQL connection URL for replication | Built from `DB_*` vars |
| `NEXT_PUBLIC_ZERO_CACHE_URL` | URL the frontend uses to connect to zero-cache | `http://localhost:<ZERO_CACHE_PORT>` |
### Manual / Local Development
If running the frontend outside Docker (e.g., `pnpm dev`), you need:
1. A running zero-cache instance pointing at your PostgreSQL database
2. `NEXT_PUBLIC_ZERO_CACHE_URL` set in your `.env` file (default: `http://localhost:4848`)
### Custom Domain / Reverse Proxy
When deploying behind a reverse proxy, set `NEXT_PUBLIC_ZERO_CACHE_URL` to your public zero-cache URL (e.g., `https://zero.yourdomain.com`). The zero-cache service must be accessible via WebSocket from the browser.
## Synced Tables
Zero syncs the following tables for real-time features:
| Table | Used By |
|-------|---------|
| `notifications` | Inbox (comments, document processing, connector status) |
| `documents` | Document list, processing status indicators |
| `search_source_connectors` | Connector status, indexing progress |
| `new_chat_messages` | Live chat message sync for shared chats |
| `chat_comments` | Real-time comment threads on AI responses |
| `chat_session_state` | Collaboration indicators (who is typing) |
## Troubleshooting
- **zero-cache not starting**: Check `docker compose logs zero-cache`. Ensure PostgreSQL has `wal_level=logical` (configured in `postgresql.conf`).
- **Frontend not syncing**: Open DevTools → Console and check for WebSocket connection errors. Verify `NEXT_PUBLIC_ZERO_CACHE_URL` matches the running zero-cache address.
- **Stale data after restart**: zero-cache rebuilds its SQLite replica from PostgreSQL on startup. This may take a moment for large databases.
## Learn More
- [Rocicorp Zero Documentation](https://zero.rocicorp.dev/docs)
- [Zero Schema Reference](https://zero.rocicorp.dev/docs/schema)
- [Zero Deployment Guide](https://zero.rocicorp.dev/docs/deployment)

View file

@ -408,6 +408,7 @@ Edit the `.env` file and set:
| NEXT_PUBLIC_FASTAPI_BACKEND_URL | Backend URL (e.g., `http://localhost:8000`) |
| NEXT_PUBLIC_FASTAPI_BACKEND_AUTH_TYPE | Same value as set in backend AUTH_TYPE i.e `GOOGLE` for OAuth with Google, `LOCAL` for email/password authentication |
| NEXT_PUBLIC_ETL_SERVICE | Document parsing service (should match backend ETL_SERVICE): `UNSTRUCTURED`, `LLAMACLOUD`, or `DOCLING` - affects supported file formats in upload interface |
| NEXT_PUBLIC_ZERO_CACHE_URL | URL for Zero-cache real-time sync service (e.g., `http://localhost:4848`) |
### 2. Install Dependencies