--- title: Electric SQL description: Setting up Electric SQL for real-time data synchronization in SurfSense --- # Electric SQL [Electric SQL](https://electric-sql.com/) enables real-time data synchronization in SurfSense, providing instant updates for notifications, document indexing status, and connector sync progress without manual refresh. The frontend uses [PGlite](https://pglite.dev/) (a lightweight PostgreSQL in the browser) to maintain a local database that syncs with the backend via Electric SQL. ## What Does Electric SQL Do? When you index documents or receive notifications, Electric SQL pushes updates to your browser in real-time. The data flows like this: 1. Backend writes data to PostgreSQL 2. Electric SQL detects changes and streams them to the frontend 3. PGlite (running in your browser) receives and stores the data locally in IndexedDB 4. Your UI updates instantly without refreshing This means: - **Notifications appear instantly** - No need to refresh the page - **Document indexing progress updates live** - Watch your documents get processed - **Connector status syncs automatically** - See when connectors finish syncing - **Offline support** - PGlite caches data locally, so previously loaded data remains accessible ## Docker Setup ### All-in-One Quickstart The simplest way to run SurfSense with Electric SQL is using the all-in-one Docker image. This bundles everything into a single container: - PostgreSQL + pgvector (vector database) - Redis (task queue) - Electric SQL (real-time sync) - Backend API - Frontend ```bash docker run -d \ -p 3000:3000 \ -p 8000:8000 \ -p 5133:5133 \ -v surfsense-data:/data \ --name surfsense \ ghcr.io/modsetter/surfsense:latest ``` **With custom Electric SQL credentials:** ```bash docker run -d \ -p 3000:3000 \ -p 8000:8000 \ -p 5133:5133 \ -v surfsense-data:/data \ -e ELECTRIC_DB_USER=your_electric_user \ -e ELECTRIC_DB_PASSWORD=your_electric_password \ --name surfsense \ ghcr.io/modsetter/surfsense:latest ``` Access SurfSense at `http://localhost:3000`. Electric SQL is automatically configured and running on port 5133. ### Docker Compose For more control over individual services, use Docker Compose. **Quickstart (all-in-one image):** ```bash docker compose -f docker-compose.quickstart.yml up -d ``` **Standard setup (separate services):** The `docker-compose.yml` includes the Electric SQL service configuration: ```yaml electric: image: electricsql/electric:latest ports: - "${ELECTRIC_PORT:-5133}:3000" environment: - DATABASE_URL=${ELECTRIC_DATABASE_URL:-postgresql://${ELECTRIC_DB_USER:-electric}:${ELECTRIC_DB_PASSWORD:-electric_password}@${POSTGRES_HOST:-db}:${POSTGRES_PORT:-5432}/${POSTGRES_DB:-surfsense}?sslmode=disable} - ELECTRIC_INSECURE=true - ELECTRIC_WRITE_TO_PG_MODE=direct restart: unless-stopped healthcheck: test: ["CMD", "curl", "-f", "http://localhost:3000/v1/health"] interval: 10s timeout: 5s retries: 5 ``` No additional configuration is required - Electric SQL is pre-configured to work with the Docker PostgreSQL instance. ## Manual Setup Follow the steps below based on your PostgreSQL setup. ### Step 1: Configure Environment Variables Add the following variables to your root `.env` file: ```bash ELECTRIC_PORT=5133 POSTGRES_DB=surfsense POSTGRES_HOST=db ELECTRIC_DB_USER=electric ELECTRIC_DB_PASSWORD=electric_password ``` > **Note:** Set `POSTGRES_HOST=db` for Docker PostgreSQL, or `POSTGRES_HOST=host.docker.internal` for local PostgreSQL. --- ### Option A: Using Docker PostgreSQL If you're using the Docker-managed PostgreSQL instance, follow these steps: **1. Start PostgreSQL and Electric SQL:** ```bash docker-compose up -d db electric ``` **2. Run database migration:** ```bash cd surfsense_backend uv run alembic upgrade head ``` **3. Start the backend:** ```bash uv run main.py ``` Electric SQL is now configured and connected to your Docker PostgreSQL database. --- ### Option B: Using Local PostgreSQL If you're using a local PostgreSQL installation, follow these steps: **1. Enable logical replication in PostgreSQL:** Locate your `postgresql.conf` file and add the following settings: ```ini # Enable logical replication (required for Electric SQL) wal_level = logical max_replication_slots = 10 max_wal_senders = 10 ``` After saving the changes, restart your PostgreSQL server for the configuration to take effect. **2. Update environment variable:** In your root `.env` file, set: ```bash POSTGRES_HOST=host.docker.internal ``` **3. Start Electric SQL:** ```bash docker-compose up -d electric ``` **4. Run database migration:** ```bash cd surfsense_backend uv run alembic upgrade head ``` **5. Start the backend:** ```bash uv run main.py ``` Electric SQL is now configured and connected to your local PostgreSQL database. ## Environment Variables Reference | Variable | Location | Description | Default | |----------|----------|-------------|---------| | `ELECTRIC_PORT` | docker-compose | Port to expose Electric SQL | `5133` | | `ELECTRIC_DB_USER` | Backend | Database user for Electric | `electric` | | `ELECTRIC_DB_PASSWORD` | Backend | Database password for Electric | `electric_password` | | `NEXT_PUBLIC_ELECTRIC_URL` | Frontend | Electric SQL server URL (PGlite connects to this) | `http://localhost:5133` | ## Verify Setup To verify Electric SQL is running correctly: ```bash curl http://localhost:5133/v1/health ``` You should receive a healthy response. ## Troubleshooting ### Electric SQL Server Not Starting **Check PostgreSQL settings:** - Ensure `wal_level = logical` is set - Verify the Electric user has replication permissions - Check database connectivity from Electric container ### Real-time Updates Not Working 1. Open browser DevTools → Console 2. Look for errors containing `[Electric]` 3. Check Network tab for WebSocket connections to the Electric URL ### Connection Refused Errors - Verify Electric SQL server is running: `docker ps | grep electric` - Check the `NEXT_PUBLIC_ELECTRIC_URL` matches your Electric server address - For Docker setups, ensure the frontend can reach the Electric container ### Data Not Syncing - Check Electric SQL logs: `docker logs electric` - Verify PostgreSQL replication is working - Ensure the Electric user has proper table permissions ### PGlite/IndexedDB Issues If data appears stale or corrupted in the browser: 1. Open browser DevTools → Application → IndexedDB 2. Delete databases starting with `surfsense-` 3. Refresh the page - PGlite will recreate the local database and resync