SurfSense/surfsense_web/content/docs/manual-installation.mdx

---
title: Manual Installation
description: Set up SurfSense from source, component by component
icon: Wrench
---

This guide sets up SurfSense without Docker (except for zero-cache, which is simplest to run as a container). Choose this path if you want to contribute to SurfSense or need full control over each component. If you just want to run SurfSense, the [Docker installation](/docs/docker-installation) is much faster.

## What You'll Need

- **Python 3.12+** — backend runtime
- **Node.js 20+** and **pnpm** — frontend runtime
- **PostgreSQL 14+** with the **pgvector** extension — database
- **Redis** — message broker for background tasks
- **Docker** — to run zero-cache (the real-time sync server)
- **Git** — to clone the repository
- **uv** — Python package manager ([install instructions](https://docs.astral.sh/uv/getting-started/installation/))

Clone the repository first:

```bash
git clone https://github.com/MODSetter/SurfSense.git
cd SurfSense
```

## Decisions to Make Up Front

**Authentication.** SurfSense supports local email/password auth (the default, no setup needed) or Google OAuth login. For Google login you need an OAuth client from the [Google Cloud Console](https://console.cloud.google.com/apis/credentials) — the [Google connectors guide](/docs/connectors/external/google) walks through creating one.

**Document parsing (ETL).** SurfSense converts uploaded files with one of three services:

- **Docling** (default) — runs locally, no API key, privacy-friendly. Supports PDF, Office docs, images, HTML, CSV.
- **Unstructured** — needs an API key from [Unstructured Platform](https://platform.unstructured.io/). Supports 34+ formats.
- **LlamaCloud** — needs an API key from [LlamaCloud](https://cloud.llamaindex.ai/). Supports 50+ formats.

You only need one. Docling is the easiest way to start.

## Backend Setup

### 1. Configure the Environment

Copy the example environment file:

**Linux/macOS:**

```bash
cd surfsense_backend
cp .env.example .env
```

**Windows (PowerShell):**

```powershell
cd surfsense_backend
Copy-Item -Path .env.example -Destination .env
```

`.env.example` is the source of truth for configuration — every variable is documented inline with comments and sensible defaults. At minimum, set your PostgreSQL connection string and a JWT secret key (generate one with `openssl rand -base64 32`). Everything else — auth type, ETL service, embeddings, TTS/STT, connector credentials — is optional and explained in the file itself.

### 2. Install Dependencies

```bash
# From surfsense_backend/
uv sync
```

### 3. Configure PostgreSQL for Zero Sync

SurfSense uses [Rocicorp Zero](https://zero.rocicorp.dev/) for real-time updates (notifications, document status, chat comments, indexing progress). Zero replicates data from PostgreSQL via **logical replication**, which requires a one-time PostgreSQL configuration change.

Edit your `postgresql.conf` (typical locations: `/etc/postgresql/<version>/main/postgresql.conf` on Linux, `/usr/local/var/postgres/postgresql.conf` on macOS via Homebrew, `C:\Program Files\PostgreSQL\<version>\data\postgresql.conf` on Windows) and set:

```ini
wal_level = logical
max_replication_slots = 10
max_wal_senders = 10
```

Then restart PostgreSQL:

**Linux:**

```bash
sudo systemctl restart postgresql
```

**macOS (Homebrew):**

```bash
brew services restart postgresql
```

**Windows (PowerShell, replace `17` with your major version):**

```powershell
Restart-Service postgresql-x64-17
```

Verify the change:

```bash
psql -U postgres -d surfsense -c "SHOW wal_level;"
# Should return: logical
```

**Managed databases (RDS, Supabase, Cloud SQL, etc.):** enable logical replication via your provider's parameter group (e.g. `rds.logical_replication=1` on RDS) and grant your database user the `REPLICATION` privilege:

```sql
ALTER USER surfsense WITH REPLICATION;
GRANT CREATE ON DATABASE surfsense TO surfsense;
```

### 4. Run Database Migrations

This creates the schema **and** the `zero_publication` that zero-cache needs to start. Skipping this step will cause zero-cache to crash-loop with `Unknown or invalid publications. Specified: [zero_publication]`.

```bash
# From surfsense_backend/
uv run alembic upgrade head
```

Verify the publication was created:

```bash
psql -U postgres -d surfsense -c "SELECT pubname FROM pg_publication;"
# Should include: zero_publication
```

### 5. Start Redis

**Linux:**

```bash
sudo systemctl start redis
# or run directly
redis-server
```

**macOS (Homebrew):**

```bash
brew services start redis
```

**Windows — run Redis in Docker (easiest):**

```powershell
docker run -d --name redis -p 6379:6379 redis:latest
```

Verify it's running:

```bash
redis-cli ping
# Should return: PONG
```

### 6. Start the Celery Worker

In a new terminal, start the worker that handles background tasks (document indexing, connector syncs):

**Linux/macOS:**

```bash
cd surfsense_backend
DEFAULT_Q="${CELERY_TASK_DEFAULT_QUEUE:-surfsense}"
uv run celery -A celery_worker.celery_app worker --loglevel=info --concurrency=1 --pool=solo --queues="${DEFAULT_Q},${DEFAULT_Q}.connectors,${DEFAULT_Q}.gateway"
```

**Windows (PowerShell):**

```powershell
cd surfsense_backend
uv run celery -A celery_worker.celery_app worker --loglevel=info --concurrency=1 --pool=solo --queues="surfsense,surfsense.connectors,surfsense.gateway"
```

Optionally, run [Flower](https://flower.readthedocs.io/) in another terminal to monitor tasks at [http://localhost:5555](http://localhost:5555):

```bash
uv run celery -A celery_worker.celery_app flower --port=5555
```

### 7. Start Celery Beat (Scheduler)

In another terminal, start the scheduler that triggers periodic tasks (like scheduled connector syncs). Without it, scheduled tasks won't run.

```bash
cd surfsense_backend
uv run celery -A celery_worker.celery_app beat --loglevel=info
```

### 8. Run the Backend

```bash
# From surfsense_backend/
uv run main.py

# Or with hot reloading for development
uv run main.py --reload
```

You should see the server running on `http://localhost:8000`.

## Zero-Cache Setup

**zero-cache** is the Rocicorp Zero server that sits between PostgreSQL and the browser. It streams real-time updates to all connected clients via WebSocket. Without it, the UI sits on stale data. For an overview of how Zero works, see the [Real-Time Sync with Zero](/docs/how-to/zero-sync) guide.

### 1. Run Zero-Cache via Docker

**Linux/macOS:**

```bash
docker run -d --name surfsense-zero-cache \
  -p 4848:4848 \
  --add-host=host.docker.internal:host-gateway \
  -e ZERO_UPSTREAM_DB="postgresql://postgres:postgres@host.docker.internal:5432/surfsense?sslmode=disable" \
  -e ZERO_CVR_DB="postgresql://postgres:postgres@host.docker.internal:5432/surfsense?sslmode=disable" \
  -e ZERO_CHANGE_DB="postgresql://postgres:postgres@host.docker.internal:5432/surfsense?sslmode=disable" \
  -e ZERO_REPLICA_FILE=/data/zero.db \
  -e ZERO_ADMIN_PASSWORD=surfsense-zero-admin \
  -e ZERO_APP_PUBLICATIONS=zero_publication \
  -e ZERO_NUM_SYNC_WORKERS=4 \
  -e ZERO_UPSTREAM_MAX_CONNS=20 \
  -e ZERO_CVR_MAX_CONNS=30 \
  -e ZERO_QUERY_URL="http://host.docker.internal:3000/api/zero/query" \
  -e ZERO_MUTATE_URL="http://host.docker.internal:3000/api/zero/mutate" \
  -e ZERO_QUERY_FORWARD_COOKIES=true \
  -v surfsense-zero-cache:/data \
  rocicorp/zero:1.6.0
```

**Windows (PowerShell):**

```powershell
docker run -d --name surfsense-zero-cache `
  -p 4848:4848 `
  --add-host=host.docker.internal:host-gateway `
  -e ZERO_UPSTREAM_DB="postgresql://postgres:postgres@host.docker.internal:5432/surfsense?sslmode=disable" `
  -e ZERO_CVR_DB="postgresql://postgres:postgres@host.docker.internal:5432/surfsense?sslmode=disable" `
  -e ZERO_CHANGE_DB="postgresql://postgres:postgres@host.docker.internal:5432/surfsense?sslmode=disable" `
  -e ZERO_REPLICA_FILE=/data/zero.db `
  -e ZERO_ADMIN_PASSWORD=surfsense-zero-admin `
  -e ZERO_APP_PUBLICATIONS=zero_publication `
  -e ZERO_NUM_SYNC_WORKERS=4 `
  -e ZERO_UPSTREAM_MAX_CONNS=20 `
  -e ZERO_CVR_MAX_CONNS=30 `
  -e ZERO_QUERY_URL="http://host.docker.internal:3000/api/zero/query" `
  -e ZERO_MUTATE_URL="http://host.docker.internal:3000/api/zero/mutate" `
  -e ZERO_QUERY_FORWARD_COOKIES=true `
  -v surfsense-zero-cache:/data `
  rocicorp/zero:1.6.0
```

**Adjustments for your setup:**

- Replace `postgres:postgres` in the connection URLs with your actual database user and password.
- On Linux without Docker Desktop, `host.docker.internal` may not resolve. Either keep the `--add-host=host.docker.internal:host-gateway` flag (Docker 20.10+) or replace `host.docker.internal` with your host's IP / `--network=host` + `localhost`.
- For production / custom domains, set `ZERO_QUERY_URL` and `ZERO_MUTATE_URL` to your public frontend URL (e.g. `https://app.yourdomain.com/api/zero/query`).

### 2. Verify Zero-Cache

```bash
curl http://localhost:4848/keepalive
# Should return HTTP 200

# Tail logs to confirm initial replication completed without errors
docker logs -f surfsense-zero-cache
```

### Alternative: Let Docker Manage All Dependencies

If you'd rather have Docker manage Postgres, Redis, and zero-cache together (while still running the backend and frontend natively), the repository ships a deps-only compose file. **Run alembic migrations on the host first** so `zero_publication` exists before zero-cache starts:

```bash
cd surfsense_backend
uv run alembic upgrade head
cd ../docker
docker compose -f docker-compose.deps-only.yml up -d
```

The deps-only stack exposes zero-cache on port `4848`. Point the frontend at it by setting the zero-cache URL in `surfsense_web/.env` (see the comments in `surfsense_web/.env.example`).

## Frontend Setup

### 1. Configure the Environment

**Linux/macOS:**

```bash
cd surfsense_web
cp .env.example .env
```

**Windows (PowerShell):**

```powershell
cd surfsense_web
Copy-Item -Path .env.example -Destination .env
```

As with the backend, `.env.example` documents every option inline. Make sure the auth type and ETL service match what you configured for the backend, and that the backend URL points at `http://localhost:8000`.

### 2. Install Dependencies and Run

```bash
# Install pnpm if you don't have it
npm install -g pnpm

pnpm install
pnpm run dev
```

The frontend should now be running at [http://localhost:3000](http://localhost:3000).

## Browser Extension (Optional)

The SurfSense browser extension saves any webpage — including those behind authentication — straight into your knowledge base.

```bash
cd surfsense_browser_extension
cp .env.example .env   # set the backend URL, documented inline

pnpm install
pnpm build                     # Chrome (default)
pnpm build --target=firefox    # or Firefox
pnpm build --target=edge       # or Edge
```

Load the built extension in your browser's developer mode and configure it with your SurfSense API key. See the [Plasmo build docs](https://docs.plasmo.com/framework/workflows/build#with-a-specific-target) for details.

## Verify Your Installation

1. Open [http://localhost:3000](http://localhost:3000) and sign in.
2. Create a workspace and upload a document.
3. Watch the upload status update live without refreshing — this confirms zero-cache is wired up correctly.
4. Chat with your uploaded content.

## Troubleshooting

- **Database connection issues**: Verify PostgreSQL is running and pgvector is installed.
- **Redis connection issues**: `redis-cli ping` should return `PONG`. Check the Redis URL in your backend `.env`.
- **Celery worker issues**: Make sure the worker is running in a separate terminal and check its logs.
- **File upload failures**: Validate your ETL service API key, or use Docling which needs none.
- **Real-time updates not working / stale UI**: Verify zero-cache is running (`curl http://localhost:4848/keepalive` returns 200). Open browser DevTools → Console and look for WebSocket errors. Confirm the zero-cache URL in `surfsense_web/.env` matches the running zero-cache address.
- **Zero-cache stuck on `Unknown or invalid publications. Specified: [zero_publication]`**: You skipped `uv run alembic upgrade head`. Run it from `surfsense_backend/`, then `docker restart surfsense-zero-cache`.
- **Zero-cache crashes with `_zero.tableMetadata` errors**: A previous run left a half-built SQLite replica behind. Start fresh: `docker rm -f surfsense-zero-cache && docker volume rm surfsense-zero-cache`, then re-run the command from [Zero-Cache Setup](#zero-cache-setup).
- **`wal_level` is not `logical`**: zero-cache requires logical replication. Set it in `postgresql.conf`, restart PostgreSQL, and verify with `SHOW wal_level;`.
- **Backend `/ready` returns 503**: The readiness probe verifies `zero_publication` exists. Run `uv run alembic upgrade head` to create it.

## Next Steps

- Set up [connectors](/docs/connectors) to bring in your tools and services.
- Connect [local models](/docs/local-models) like Ollama or LM Studio.
- For production, put a reverse proxy in front, add SSL, and set up database backups.