mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-04-25 16:56:22 +02:00
feat: add database migration script and update installation instructions for legacy all-in-one users
This commit is contained in:
parent
91c30db3e4
commit
f051c19aca
5 changed files with 684 additions and 1 deletions
226
surfsense_web/content/docs/how-to/migrate-from-allinone.mdx
Normal file
226
surfsense_web/content/docs/how-to/migrate-from-allinone.mdx
Normal file
|
|
@ -0,0 +1,226 @@
|
|||
---
|
||||
title: Migrate from the All-in-One Container
|
||||
description: How to migrate your data from the legacy surfsense all-in-one Docker image to the current multi-container setup
|
||||
---
|
||||
|
||||
The original SurfSense all-in-one image (`ghcr.io/modsetter/surfsense:latest`, run via `docker-compose.quickstart.yml`) stored all data — PostgreSQL, Redis, and configuration — in a single Docker volume named `surfsense-data`. The current setup uses separate named volumes and has upgraded PostgreSQL from **version 14 to 17**.
|
||||
|
||||
Because PostgreSQL data files are not compatible between major versions, a **logical dump and restore** is required. This is a one-time migration.
|
||||
|
||||
<Callout type="warn">
|
||||
This guide only applies to users who ran the legacy `docker-compose.quickstart.yml` (the all-in-one `surfsense` container). If you were already using `docker/docker-compose.yml`, you do not need to migrate.
|
||||
</Callout>
|
||||
|
||||
<Callout type="info">
|
||||
If you try to run `install.sh` while the old `surfsense-data` volume exists, the script will detect it and stop with instructions to migrate first.
|
||||
</Callout>
|
||||
|
||||
---
|
||||
|
||||
## Option A — Migration Script (recommended)
|
||||
|
||||
A single script handles the entire process automatically: it dumps your PostgreSQL 14 data, recovers your `SECRET_KEY`, sets up the new stack, and restores into PostgreSQL 17.
|
||||
|
||||
**Prerequisites:** Docker running, ~500 MB free disk space, internet access.
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/migrate-database.sh | bash
|
||||
```
|
||||
|
||||
Or download and inspect it first (recommended):
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/migrate-database.sh -o migrate-database.sh
|
||||
# Review the script, then run:
|
||||
bash migrate-database.sh
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--db-user USER` | Old PostgreSQL username | `surfsense` |
|
||||
| `--db-password PASS` | Old PostgreSQL password | `surfsense` |
|
||||
| `--db-name NAME` | Old PostgreSQL database | `surfsense` |
|
||||
| `--install-dir DIR` | New installation directory | `./surfsense` |
|
||||
| `--yes` / `-y` | Skip confirmation prompts | — |
|
||||
|
||||
If you customised the database credentials in your old all-in-one container, pass them explicitly:
|
||||
|
||||
```bash
|
||||
bash migrate-database.sh --db-user myuser --db-password mypass --db-name mydb
|
||||
```
|
||||
|
||||
### What the script does
|
||||
|
||||
1. Checks prerequisites and confirms the `surfsense-data` volume exists
|
||||
2. Starts a temporary `postgres:14` container against the old data
|
||||
3. Runs `pg_dump` and validates the dump file (size + header check)
|
||||
4. Recovers your `SECRET_KEY` from the old volume (or prompts if not found)
|
||||
5. Downloads the new compose files into `./surfsense/` (skips if already present)
|
||||
6. Writes the recovered `SECRET_KEY` into `./surfsense/.env`
|
||||
7. Starts the new `db` service (PostgreSQL 17), waits for readiness
|
||||
8. Restores the dump with `psql` and runs a smoke test
|
||||
9. Starts all remaining services
|
||||
|
||||
The original `surfsense-data` volume is **never deleted** — you remove it manually after verifying the migration.
|
||||
|
||||
### After the script completes
|
||||
|
||||
1. Open [http://localhost:3000](http://localhost:3000) and confirm your data is intact.
|
||||
2. Once satisfied, remove the old volume:
|
||||
```bash
|
||||
docker volume rm surfsense-data
|
||||
```
|
||||
3. Delete the backup dump once you no longer need it:
|
||||
```bash
|
||||
rm ./surfsense_migration_backup.sql
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Option B — Manual Steps
|
||||
|
||||
Use these steps if the migration script doesn't work on your platform (e.g. Windows without WSL2), or if you want full control over each step.
|
||||
|
||||
### Before you start
|
||||
|
||||
- Confirm the old volume exists: `docker volume ls | grep surfsense-data`
|
||||
- Have ~500 MB free disk space for the SQL dump.
|
||||
|
||||
### Step 1 — Start a temporary PostgreSQL 14 container
|
||||
|
||||
```bash
|
||||
docker run -d --name surfsense-pg14-temp \
|
||||
-v surfsense-data:/data \
|
||||
-e PGDATA=/data/postgres \
|
||||
-e POSTGRES_USER=surfsense \
|
||||
-e POSTGRES_PASSWORD=surfsense \
|
||||
-e POSTGRES_DB=surfsense \
|
||||
postgres:14
|
||||
```
|
||||
|
||||
Wait ~10 seconds, then confirm it is healthy:
|
||||
|
||||
```bash
|
||||
docker exec surfsense-pg14-temp pg_isready -U surfsense
|
||||
```
|
||||
|
||||
### Step 2 — Dump the database
|
||||
|
||||
```bash
|
||||
docker exec -e PGPASSWORD=surfsense surfsense-pg14-temp \
|
||||
pg_dump -U surfsense surfsense > surfsense_backup.sql
|
||||
```
|
||||
|
||||
Verify the dump is valid:
|
||||
|
||||
```bash
|
||||
wc -l surfsense_backup.sql
|
||||
grep "PostgreSQL database dump" surfsense_backup.sql
|
||||
```
|
||||
|
||||
### Step 3 — Recover your SECRET\_KEY
|
||||
|
||||
```bash
|
||||
docker run --rm -v surfsense-data:/data alpine cat /data/.secret_key
|
||||
```
|
||||
|
||||
Copy the printed value for the next step.
|
||||
|
||||
### Step 4 — Set up the new stack
|
||||
|
||||
```bash
|
||||
git clone https://github.com/MODSetter/SurfSense.git
|
||||
cd SurfSense/docker
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Set `SECRET_KEY` in `.env` to the value recovered above.
|
||||
|
||||
### Step 5 — Start PostgreSQL 17
|
||||
|
||||
```bash
|
||||
docker compose up -d db
|
||||
```
|
||||
|
||||
Wait until ready:
|
||||
|
||||
```bash
|
||||
docker compose exec db pg_isready -U surfsense
|
||||
```
|
||||
|
||||
### Step 6 — Restore the database
|
||||
|
||||
```bash
|
||||
docker compose exec -T db \
|
||||
psql -U surfsense -d surfsense < surfsense_backup.sql
|
||||
```
|
||||
|
||||
Harmless notices like `ERROR: role "surfsense" already exists` are expected.
|
||||
|
||||
### Step 7 — Start all services
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### Step 8 — Clean up
|
||||
|
||||
After verifying everything works:
|
||||
|
||||
```bash
|
||||
# Remove temporary PG14 container
|
||||
docker stop surfsense-pg14-temp && docker rm surfsense-pg14-temp
|
||||
|
||||
# Remove old volume (irreversible — only after confirming migration success)
|
||||
docker volume rm surfsense-data
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Script exits with "surfsense-postgres already exists"
|
||||
|
||||
A previous migration attempt partially completed. Remove the incomplete volume and retry:
|
||||
|
||||
```bash
|
||||
docker volume rm surfsense-postgres
|
||||
bash migrate-database.sh
|
||||
```
|
||||
|
||||
### PostgreSQL 14 container fails to start
|
||||
|
||||
Check the container logs:
|
||||
|
||||
```bash
|
||||
docker logs surfsense-pg14-temp
|
||||
```
|
||||
|
||||
If you see permission errors, the data directory may need ownership correction. Run:
|
||||
|
||||
```bash
|
||||
docker exec surfsense-pg14-temp chown -R postgres:postgres /data/postgres
|
||||
```
|
||||
|
||||
Then restart the container.
|
||||
|
||||
### Empty or corrupt dump file
|
||||
|
||||
If `surfsense_backup.sql` is smaller than expected, run the dump command again with verbose output:
|
||||
|
||||
```bash
|
||||
docker exec -e PGPASSWORD=surfsense surfsense-pg14-temp \
|
||||
pg_dump -U surfsense surfsense -v 2>&1 | head -40
|
||||
```
|
||||
|
||||
### Cannot find `/data/.secret_key`
|
||||
|
||||
If the all-in-one was launched with `SECRET_KEY` set explicitly as an environment variable, the key was never written to the volume. Set the same value manually in `docker/.env`. If it is lost, generate a new one:
|
||||
|
||||
```bash
|
||||
openssl rand -base64 32
|
||||
```
|
||||
|
||||
Note: a new key invalidates all existing browser sessions — users will need to log in again.
|
||||
Loading…
Add table
Add a link
Reference in a new issue