From 9c633c172b346980eacd5173e5aa66eea767836a Mon Sep 17 00:00:00 2001
From: Jan De Landtsheer <jan@delandtsheer.eu>
Date: Wed, 22 Apr 2026 10:28:59 +0200
Subject: [PATCH] Added postgres admin added amends to the postgres
 backend/phase2

---
 .gitignore                                  |   3 +
 docs/plans/0002-phase-2-postgres-backend.md |   3 +-
 docs/plans/local-dev-postgres-setup.md      | 153 ++++++++++++++++++++
 3 files changed, 158 insertions(+), 1 deletion(-)
 create mode 100644 docs/plans/local-dev-postgres-setup.md

diff --git a/.gitignore b/.gitignore
index 4236e68..41e352a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -137,3 +137,6 @@ apps/dashboard/node_modules/
 # =============================================================================
 fastembed-rs/
 .mcp.json
+
+.claude/
+.codebase-memory/
diff --git a/docs/plans/0002-phase-2-postgres-backend.md b/docs/plans/0002-phase-2-postgres-backend.md
index a372e27..3fe28f2 100644
--- a/docs/plans/0002-phase-2-postgres-backend.md
+++ b/docs/plans/0002-phase-2-postgres-backend.md
@@ -2,7 +2,7 @@
 
 **Status**: Draft
 **Depends on**: Phase 1 (MemoryStore + Embedder traits, embedding_model registry, domain columns)
-**Related**: docs/adr/0001-pluggable-storage-and-network-access.md (Phase 2), docs/prd/001-getting-centralized-vestige.md
+**Related**: docs/adr/0001-pluggable-storage-and-network-access.md (Phase 2), docs/prd/001-getting-centralized-vestige.md, docs/plans/local-dev-postgres-setup.md (local cluster provisioning)
 
 ---
 
@@ -85,6 +85,7 @@ postgres-backend = ["dep:sqlx", "dep:pgvector", "dep:tokio-stream", "dep:futures
 - The `pgvector` extension installed in the target database (our migration issues `CREATE EXTENSION IF NOT EXISTS vector`).
 - `sqlx-cli@0.8` installed on the developer machine for `cargo sqlx prepare --workspace` and `cargo sqlx migrate add` (not a build-time requirement once `.sqlx/` is committed).
 - Docker or Podman reachable by the test harness for `testcontainers-modules::postgres` to spin up `pgvector/pgvector:pg16`.
+- A local Postgres cluster for `sqlx prepare`, manual migration work, and `vestige migrate --to postgres` smoke runs. The recipe for standing one up on Arch/CachyOS (install, initdb, role + db, pgvector, connection string at `~/.vestige_pg_pw`) lives in `docs/plans/local-dev-postgres-setup.md`. Postgres 18 from the Arch repo satisfies the "16 or newer" requirement above. Phase 2 work assumes `DATABASE_URL` points at that cluster once migrations are applied.
 
 ### Assumed Rust toolchain
 
diff --git a/docs/plans/local-dev-postgres-setup.md b/docs/plans/local-dev-postgres-setup.md
new file mode 100644
index 0000000..6250a55
--- /dev/null
+++ b/docs/plans/local-dev-postgres-setup.md
@@ -0,0 +1,153 @@
+# Local Dev Postgres Setup (Arch / CachyOS)
+
+**Status**: Applied on this machine on 2026-04-21
+**Related**: docs/plans/0002-phase-2-postgres-backend.md, docs/adr/0001-pluggable-storage-and-network-access.md
+
+Purpose: capture the minimum, repeatable steps to stand up a Postgres 18 instance on a local Arch/CachyOS box for Phase 2 (`PgMemoryStore`) development, `sqlx prepare`, and manual migration testing. This is a single-operator dev recipe, not a production runbook.
+
+---
+
+## Current state on this machine
+
+- Package: `postgresql` 18.3-2 (pacman). Pulls `postgresql-libs`, `libxslt`.
+- Service: `postgresql.service`, enabled + active.
+- Listens on: `127.0.0.1:5432` and `[::1]:5432` only (default `listen_addresses = 'localhost'`).
+- Data dir: `/var/lib/postgres/data`, owner `postgres:postgres`.
+- Auth (`pg_hba.conf`, Arch defaults): `peer` for local socket, `scram-sha-256` for host 127.0.0.1/::1.
+
+### Database + role
+
+- Database: `vestige`, UTF8, owner `vestige`.
+- Role: `vestige` with `LOGIN CREATEDB` (no superuser, no replication, no cross-db).
+- Schema `public` re-owned to `vestige`, plus default privileges so any future tables / sequences / functions in `public` are fully owned and granted to `vestige`.
+
+Net effect: the `vestige` role can create, alter, drop, and grant freely inside the `vestige` database -- enough for `sqlx::migrate!`, ad-hoc schema work, and the full Phase 2 `MemoryStore` surface. It cannot create extensions (see Phase 2 followups below) and cannot touch other databases.
+
+### Connection
+
+```
+postgresql://vestige:<password>@127.0.0.1:5432/vestige
+```
+
+Password lives at `~/.vestige_pg_pw`, mode 600, owned by the dev user (no sudo needed to read it). Read with:
+
+```sh
+cat ~/.vestige_pg_pw
+```
+
+Recommended dev shell export (keep this OUT of the repo; use `.env` + gitignore or a shell rc):
+
+```sh
+export DATABASE_URL="postgresql://vestige:$(cat ~/.vestige_pg_pw)@127.0.0.1:5432/vestige"
+```
+
+---
+
+## Reproduce from scratch
+
+On a fresh Arch / CachyOS box with passwordless sudo:
+
+```sh
+# 1. Install
+sudo pacman -S --noconfirm postgresql
+
+# 2. Initialize the cluster (UTF8, scram-sha-256 for host, peer for local)
+sudo -iu postgres initdb \
+  --locale=C.UTF-8 --encoding=UTF8 \
+  -D /var/lib/postgres/data \
+  --auth-host=scram-sha-256 --auth-local=peer
+
+# 3. Start + enable
+sudo systemctl enable --now postgresql
+
+# 4. Generate a password and stash it in the dev user's home (mode 600)
+VESTIGE_PW=$(python3 -c 'import secrets,string; a=string.ascii_letters+string.digits; print("".join(secrets.choice(a) for _ in range(32)))')
+umask 077
+printf '%s' "$VESTIGE_PW" > ~/.vestige_pg_pw
+chmod 600 ~/.vestige_pg_pw
+
+# 5. Create role + database + grants
+sudo -u postgres psql -v ON_ERROR_STOP=1 <<SQL
+CREATE ROLE vestige WITH LOGIN CREATEDB PASSWORD '${VESTIGE_PW}';
+CREATE DATABASE vestige OWNER vestige ENCODING 'UTF8';
+GRANT ALL PRIVILEGES ON DATABASE vestige TO vestige;
+SQL
+
+sudo -u postgres psql -d vestige -v ON_ERROR_STOP=1 <<'SQL'
+GRANT ALL ON SCHEMA public TO vestige;
+ALTER SCHEMA public OWNER TO vestige;
+ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO vestige;
+ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO vestige;
+ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON FUNCTIONS TO vestige;
+SQL
+
+# 6. Smoke test
+PGPASSWORD="$VESTIGE_PW" psql -h 127.0.0.1 -U vestige -d vestige \
+  -c 'SELECT current_user, current_database(), version();'
+unset VESTIGE_PW
+```
+
+---
+
+## Phase 2 followups (before PgMemoryStore works)
+
+The cluster above is bare Postgres. Phase 2 needs `pgvector`:
+
+```sh
+# Install the extension package
+sudo pacman -S --noconfirm pgvector
+
+# Enable it in the vestige database (must run as postgres; vestige is not superuser)
+sudo -u postgres psql -d vestige -c 'CREATE EXTENSION IF NOT EXISTS vector;'
+```
+
+Verify:
+
+```sh
+PGPASSWORD="$(cat ~/.vestige_pg_pw)" psql -h 127.0.0.1 -U vestige -d vestige \
+  -c "SELECT extname, extversion FROM pg_extension WHERE extname = 'vector';"
+```
+
+Notes:
+
+- `pgvector` must be available on the server before `sqlx::migrate!` runs, or the Phase 2 migration that declares typed `Vector` columns will fail.
+- Testcontainer-based Phase 2 integration tests use `pgvector/pgvector:pg16` and are independent of this local cluster. This local cluster is for `sqlx prepare`, `cargo run -- migrate --to postgres`, and manual poking.
+- `sqlx prepare` needs `DATABASE_URL` pointed at this cluster with `vestige` migrations already applied. Run from `crates/vestige-core/`.
+
+---
+
+## Password rotation
+
+```sh
+NEW_PW=$(python3 -c 'import secrets,string; a=string.ascii_letters+string.digits; print("".join(secrets.choice(a) for _ in range(32)))')
+umask 077
+printf '%s' "$NEW_PW" > ~/.vestige_pg_pw
+chmod 600 ~/.vestige_pg_pw
+sudo -u postgres psql -v ON_ERROR_STOP=1 \
+  -c "ALTER ROLE vestige WITH PASSWORD '${NEW_PW}';"
+unset NEW_PW
+```
+
+Then re-export `DATABASE_URL` in any live shells.
+
+---
+
+## Teardown
+
+Destroys the cluster and all data in it:
+
+```sh
+sudo systemctl disable --now postgresql
+sudo pacman -Rns postgresql postgresql-libs
+sudo rm -rf /var/lib/postgres
+rm -f ~/.vestige_pg_pw
+```
+
+---
+
+## Out of scope for this doc
+
+- TLS, client-cert auth, non-localhost access. Phase 3 exposes the Vestige HTTP API over the network, not Postgres directly.
+- Backups, PITR, WAL archiving. For dev data: `pg_dump -h 127.0.0.1 -U vestige vestige > vestige.sql`.
+- Replication, PgBouncer, tuned `postgresql.conf`. Defaults are fine for Phase 2 development.
+- Making this the canonical Vestige backend. By default Vestige still uses SQLite; this cluster exists so the `postgres-backend` feature can be built and tested locally.