Added postgres admin

added amends to the postgres backend/phase2
This commit is contained in:
Jan De Landtsheer 2026-04-22 10:28:59 +02:00
parent 0d273c5641
commit 9c633c172b
No known key found for this signature in database
GPG key ID: 95CD37F0C226040B
3 changed files with 158 additions and 1 deletions

3
.gitignore vendored
View file

@ -137,3 +137,6 @@ apps/dashboard/node_modules/
# =============================================================================
fastembed-rs/
.mcp.json
.claude/
.codebase-memory/

View file

@ -2,7 +2,7 @@
**Status**: Draft
**Depends on**: Phase 1 (MemoryStore + Embedder traits, embedding_model registry, domain columns)
**Related**: docs/adr/0001-pluggable-storage-and-network-access.md (Phase 2), docs/prd/001-getting-centralized-vestige.md
**Related**: docs/adr/0001-pluggable-storage-and-network-access.md (Phase 2), docs/prd/001-getting-centralized-vestige.md, docs/plans/local-dev-postgres-setup.md (local cluster provisioning)
---
@ -85,6 +85,7 @@ postgres-backend = ["dep:sqlx", "dep:pgvector", "dep:tokio-stream", "dep:futures
- The `pgvector` extension installed in the target database (our migration issues `CREATE EXTENSION IF NOT EXISTS vector`).
- `sqlx-cli@0.8` installed on the developer machine for `cargo sqlx prepare --workspace` and `cargo sqlx migrate add` (not a build-time requirement once `.sqlx/` is committed).
- Docker or Podman reachable by the test harness for `testcontainers-modules::postgres` to spin up `pgvector/pgvector:pg16`.
- A local Postgres cluster for `sqlx prepare`, manual migration work, and `vestige migrate --to postgres` smoke runs. The recipe for standing one up on Arch/CachyOS (install, initdb, role + db, pgvector, connection string at `~/.vestige_pg_pw`) lives in `docs/plans/local-dev-postgres-setup.md`. Postgres 18 from the Arch repo satisfies the "16 or newer" requirement above. Phase 2 work assumes `DATABASE_URL` points at that cluster once migrations are applied.
### Assumed Rust toolchain

View file

@ -0,0 +1,153 @@
# Local Dev Postgres Setup (Arch / CachyOS)
**Status**: Applied on this machine on 2026-04-21
**Related**: docs/plans/0002-phase-2-postgres-backend.md, docs/adr/0001-pluggable-storage-and-network-access.md
Purpose: capture the minimum, repeatable steps to stand up a Postgres 18 instance on a local Arch/CachyOS box for Phase 2 (`PgMemoryStore`) development, `sqlx prepare`, and manual migration testing. This is a single-operator dev recipe, not a production runbook.
---
## Current state on this machine
- Package: `postgresql` 18.3-2 (pacman). Pulls `postgresql-libs`, `libxslt`.
- Service: `postgresql.service`, enabled + active.
- Listens on: `127.0.0.1:5432` and `[::1]:5432` only (default `listen_addresses = 'localhost'`).
- Data dir: `/var/lib/postgres/data`, owner `postgres:postgres`.
- Auth (`pg_hba.conf`, Arch defaults): `peer` for local socket, `scram-sha-256` for host 127.0.0.1/::1.
### Database + role
- Database: `vestige`, UTF8, owner `vestige`.
- Role: `vestige` with `LOGIN CREATEDB` (no superuser, no replication, no cross-db).
- Schema `public` re-owned to `vestige`, plus default privileges so any future tables / sequences / functions in `public` are fully owned and granted to `vestige`.
Net effect: the `vestige` role can create, alter, drop, and grant freely inside the `vestige` database -- enough for `sqlx::migrate!`, ad-hoc schema work, and the full Phase 2 `MemoryStore` surface. It cannot create extensions (see Phase 2 followups below) and cannot touch other databases.
### Connection
```
postgresql://vestige:<password>@127.0.0.1:5432/vestige
```
Password lives at `~/.vestige_pg_pw`, mode 600, owned by the dev user (no sudo needed to read it). Read with:
```sh
cat ~/.vestige_pg_pw
```
Recommended dev shell export (keep this OUT of the repo; use `.env` + gitignore or a shell rc):
```sh
export DATABASE_URL="postgresql://vestige:$(cat ~/.vestige_pg_pw)@127.0.0.1:5432/vestige"
```
---
## Reproduce from scratch
On a fresh Arch / CachyOS box with passwordless sudo:
```sh
# 1. Install
sudo pacman -S --noconfirm postgresql
# 2. Initialize the cluster (UTF8, scram-sha-256 for host, peer for local)
sudo -iu postgres initdb \
--locale=C.UTF-8 --encoding=UTF8 \
-D /var/lib/postgres/data \
--auth-host=scram-sha-256 --auth-local=peer
# 3. Start + enable
sudo systemctl enable --now postgresql
# 4. Generate a password and stash it in the dev user's home (mode 600)
VESTIGE_PW=$(python3 -c 'import secrets,string; a=string.ascii_letters+string.digits; print("".join(secrets.choice(a) for _ in range(32)))')
umask 077
printf '%s' "$VESTIGE_PW" > ~/.vestige_pg_pw
chmod 600 ~/.vestige_pg_pw
# 5. Create role + database + grants
sudo -u postgres psql -v ON_ERROR_STOP=1 <<SQL
CREATE ROLE vestige WITH LOGIN CREATEDB PASSWORD '${VESTIGE_PW}';
CREATE DATABASE vestige OWNER vestige ENCODING 'UTF8';
GRANT ALL PRIVILEGES ON DATABASE vestige TO vestige;
SQL
sudo -u postgres psql -d vestige -v ON_ERROR_STOP=1 <<'SQL'
GRANT ALL ON SCHEMA public TO vestige;
ALTER SCHEMA public OWNER TO vestige;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO vestige;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO vestige;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON FUNCTIONS TO vestige;
SQL
# 6. Smoke test
PGPASSWORD="$VESTIGE_PW" psql -h 127.0.0.1 -U vestige -d vestige \
-c 'SELECT current_user, current_database(), version();'
unset VESTIGE_PW
```
---
## Phase 2 followups (before PgMemoryStore works)
The cluster above is bare Postgres. Phase 2 needs `pgvector`:
```sh
# Install the extension package
sudo pacman -S --noconfirm pgvector
# Enable it in the vestige database (must run as postgres; vestige is not superuser)
sudo -u postgres psql -d vestige -c 'CREATE EXTENSION IF NOT EXISTS vector;'
```
Verify:
```sh
PGPASSWORD="$(cat ~/.vestige_pg_pw)" psql -h 127.0.0.1 -U vestige -d vestige \
-c "SELECT extname, extversion FROM pg_extension WHERE extname = 'vector';"
```
Notes:
- `pgvector` must be available on the server before `sqlx::migrate!` runs, or the Phase 2 migration that declares typed `Vector` columns will fail.
- Testcontainer-based Phase 2 integration tests use `pgvector/pgvector:pg16` and are independent of this local cluster. This local cluster is for `sqlx prepare`, `cargo run -- migrate --to postgres`, and manual poking.
- `sqlx prepare` needs `DATABASE_URL` pointed at this cluster with `vestige` migrations already applied. Run from `crates/vestige-core/`.
---
## Password rotation
```sh
NEW_PW=$(python3 -c 'import secrets,string; a=string.ascii_letters+string.digits; print("".join(secrets.choice(a) for _ in range(32)))')
umask 077
printf '%s' "$NEW_PW" > ~/.vestige_pg_pw
chmod 600 ~/.vestige_pg_pw
sudo -u postgres psql -v ON_ERROR_STOP=1 \
-c "ALTER ROLE vestige WITH PASSWORD '${NEW_PW}';"
unset NEW_PW
```
Then re-export `DATABASE_URL` in any live shells.
---
## Teardown
Destroys the cluster and all data in it:
```sh
sudo systemctl disable --now postgresql
sudo pacman -Rns postgresql postgresql-libs
sudo rm -rf /var/lib/postgres
rm -f ~/.vestige_pg_pw
```
---
## Out of scope for this doc
- TLS, client-cert auth, non-localhost access. Phase 3 exposes the Vestige HTTP API over the network, not Postgres directly.
- Backups, PITR, WAL archiving. For dev data: `pg_dump -h 127.0.0.1 -U vestige vestige > vestige.sql`.
- Replication, PgBouncer, tuned `postgresql.conf`. Defaults are fine for Phase 2 development.
- Making this the canonical Vestige backend. By default Vestige still uses SQLite; this cluster exists so the `postgres-backend` feature can be built and tested locally.