diff --git a/docs/deployment/scaling.mdx b/docs/deployment/scaling.mdx index 2fd3acd..b544539 100644 --- a/docs/deployment/scaling.mdx +++ b/docs/deployment/scaling.mdx @@ -7,6 +7,10 @@ By default, the Dograh API container runs a single uvicorn worker. For productio This page covers how the multi-worker setup works, how to choose a worker count at install time, and how to change it on a running stack. + +Multi-worker support requires **Dograh v1.29.0 or newer**. Earlier releases used `uvicorn --workers` and ship a different `setup_remote.sh` / `start_services_docker.sh` / `nginx.conf` layout — the steps below will not work on them. If your stack is older, [update first](/deployment/update) and then come back to this page. + + ## How it works The API container starts `FASTAPI_WORKERS` separate uvicorn processes, each bound to its own port (`8000`, `8001`, `8002`, …). nginx exposes a single upstream `dograh_api` that includes all worker ports and routes new requests to whichever worker currently has the **fewest active connections**. diff --git a/docs/deployment/update.mdx b/docs/deployment/update.mdx index bfa431d..aefb14e 100644 --- a/docs/deployment/update.mdx +++ b/docs/deployment/update.mdx @@ -5,9 +5,11 @@ description: "Update your self-hosted Dograh stack to a newer image version" This guide covers updating a Dograh stack you've already deployed with [Docker](/deployment/docker) or a [custom domain](/deployment/custom-domain). You run commands from the same directory that contains your `docker-compose.yaml` (this is the `dograh/` directory if you used `setup_remote.sh`). - -This guide assumes you deployed in **prebuilt mode** — i.e. your stack pulls official `dograh-api` / `dograh-ui` images from a registry. If you deployed in **build mode** (a `docker-compose.override.yaml` exists in your `dograh/` directory), the update flow is different — see [Updating a source build](#updating-a-source-build) at the bottom. - +There are three update flows depending on how you deployed: + +- **Remote, prebuilt mode** (most users) — use [`update_remote.sh`](#remote-prebuilt-mode-recommended) below. +- **Local Docker** — pull images and restart; see [Local deployment](#local-deployment). +- **Remote, build mode** (you have a `docker-compose.override.yaml`) — update via git; see [Updating a source build](#updating-a-source-build). ## Find an image version @@ -20,77 +22,59 @@ Each release is published under two kinds of tags: | Tag style | Example | When to use | |-----------|---------|-------------| -| **Release tag** | `v0.8.2` | Stable, recommended for production | +| **Release tag** | `v1.29.0` | Stable, recommended for production | | **Git commit SHA** | `a1b2c3d` | Bleeding edge — any commit merged to `main` | | `latest` | `latest` | Tracks the most recent release tag | -Always update **`dograh-api`** and **`dograh-ui`** to the **same tag**. The two images are built from the same commit and the UI expects API responses in a matching shape — mixing versions will break the app. +Always update **`dograh-api`** and **`dograh-ui`** to the **same tag**. The two images are built from the same commit and the UI expects API responses in a matching shape — mixing versions will break the app. `update_remote.sh` handles this for you automatically. -## Option A: Update to the latest release +## Remote, prebuilt mode (recommended) -If your `docker-compose.yaml` uses `:latest` (the default), just pull and restart: +`update_remote.sh` is the supported path for updating a stack created with `setup_remote.sh`. In one shot it: - -```bash Local deployment -docker compose down -docker compose up --pull always -``` -```bash Remote deployment -cd dograh -sudo docker compose --profile remote down -sudo docker compose --profile remote up --pull always -``` - +- Asks for a target version (defaults to the latest release tag on GitHub). +- Pulls `docker-compose.yaml` at that version and pins both `api` and `ui` images to it. +- Regenerates `nginx.conf` and `turnserver.conf` from the upstream templates, so newer features (like [multi-worker scaling](/deployment/scaling)) are wired up correctly without manual editing. +- Reads your existing `.env` and appends any new required keys with safe defaults — your `OSS_JWT_SECRET`, `TURN_SECRET`, and other values are never touched. +- Backs up every file it changes with a `.bak.` suffix. -`--pull always` forces Docker to fetch the latest `:latest` from the registry instead of reusing your cached image. - -## Option B: Pin a specific tag - -To update (or roll back) to a specific release or commit, edit `docker-compose.yaml` and change the `image:` lines for both `api` and `ui` services to the same tag. - -Open the file: +From your install directory: ```bash -nano docker-compose.yaml +cd dograh +curl -o update_remote.sh https://raw.githubusercontent.com/dograh-hq/dograh/main/scripts/update_remote.sh +bash update_remote.sh ``` -Find these two lines: +You'll be prompted for the target version. Non-interactive callers can set it via environment variable and skip the confirmation prompt: -```yaml - api: - image: ${REGISTRY:-dograhai}/dograh-api:latest - ui: - image: ${REGISTRY:-dograhai}/dograh-ui:latest +```bash +TARGET_VERSION=v1.29.0 DOGRAH_UPDATE_YES=1 bash update_remote.sh ``` -Replace `:latest` with your chosen tag on **both** services — for example: +After the script finishes, apply the update by recreating the stack: -```yaml - api: - image: ${REGISTRY:-dograhai}/dograh-api:v0.8.2 - ui: - image: ${REGISTRY:-dograhai}/dograh-ui:v0.8.2 +```bash +sudo docker compose --profile remote down +sudo docker compose --profile remote up -d --pull always ``` -You can use either registry. Leave `REGISTRY` unset for Docker Hub (`dograhai`), or export `REGISTRY=ghcr.io/dograh-hq` to pull from GitHub Container Registry. +The script overwrites `docker-compose.yaml`, `nginx.conf`, and `turnserver.conf` from upstream templates. If you've made local edits to any of these (extra environment variables, custom ports, modified nginx routes), check the `.bak.` files after the update and re-apply your edits. -Then bring the stack down and back up: +## Local deployment - -```bash Local deployment +For local Docker installs (the [Quick Start](/deployment/docker#quick-start) flow or `setup_local.sh`), there are no host-side config files to refresh — pull new images and restart: + +```bash docker compose down docker compose up --pull always ``` -```bash Remote deployment -cd dograh -sudo docker compose --profile remote down -sudo docker compose --profile remote up --pull always -``` - + +To pin a specific version instead of `latest`, edit `docker-compose.yaml` and change both `image:` lines for `api` and `ui` to the same tag (e.g. `:v1.29.0`), then run the commands above. ## Verify the update @@ -105,12 +89,24 @@ You should see the API and UI both running the tag you pinned. Hit the health endpoint to confirm the API is responding: ```bash -curl http://localhost:8000/api/v1/health +curl -k https://YOUR_SERVER_IP/api/v1/health # remote +curl http://localhost:8000/api/v1/health # local ``` ## Roll back -If something breaks, roll back by pinning the previous tag using the same process in **Option B** and restarting. Your Postgres data volume persists across `down`/`up` cycles, so agents and call history are preserved. +`update_remote.sh` saves backups of every file it touched. To roll back, restore them and recreate the stack — the exact commands (including the timestamp it used) are printed at the end of the script's output. The generic form: + +```bash +cd dograh +for f in docker-compose.yaml nginx.conf turnserver.conf .env; do + [[ -f "$f.bak." ]] && cp "$f.bak." "$f" +done +sudo docker compose --profile remote down +sudo docker compose --profile remote up -d +``` + +Your Postgres data volume persists across `down`/`up` cycles, so agents and call history are preserved. Rolling back across a database migration is not always safe — if the newer release ran a schema migration, downgrading may leave the DB in a state the older API doesn't understand. If in doubt, [open an issue](https://github.com/dograh-hq/dograh/issues) before rolling back. @@ -118,21 +114,29 @@ Rolling back across a database migration is not always safe — if the newer rel ## Updating a source build -If you deployed in **build mode** (you'll have a `docker-compose.override.yaml` in your `dograh/` directory), there are no image tags to pull — you update by pulling new source and rebuilding: +If you deployed in **build mode** (you'll have a `docker-compose.override.yaml` in your install directory), `update_remote.sh` deliberately refuses to run — you already have the full repo locally and update via git: ```bash cd dograh +git fetch + +# Track latest main: git pull +# Or pin to a specific release: +git checkout v1.29.0 + +# Pick up pipecat and other submodule bumps +git submodule update --init --recursive + +# Rebuild and restart sudo docker compose --profile remote build sudo docker compose --profile remote up -d ``` -To roll back, check out an earlier commit or tag and rebuild: + +If you update the `pipecat` submodule, you **must** run `git submodule update --init --recursive` before rebuilding, or the Docker build will not pick up `pipecat` changes. + -```bash -git checkout -sudo docker compose --profile remote build -sudo docker compose --profile remote up -d -``` +If you maintain a fork with local customizations on top of upstream, merging conflicts in `docker-compose.yaml`, `nginx.conf`, `turnserver.conf`, or `setup_remote.sh` is up to you — resolve them as you would any other git merge. Leave `OSS_JWT_SECRET` and `TURN_SECRET` in `.env` unchanged across updates to preserve sessions and WebRTC auth. The same migration warning above applies: rolling back across a schema change can leave the DB in a state the older API can't read. diff --git a/scripts/update_remote.sh b/scripts/update_remote.sh new file mode 100755 index 0000000..0044c1f --- /dev/null +++ b/scripts/update_remote.sh @@ -0,0 +1,394 @@ +#!/bin/bash +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +REPO="dograh-hq/dograh" +TIMESTAMP=$(date +%Y%m%d-%H%M%S) + +echo -e "${BLUE}" +echo "╔══════════════════════════════════════════════════════════════╗" +echo "║ Dograh Remote Update ║" +echo "║ Refresh host-side configs and pin api/ui image versions ║" +echo "╚══════════════════════════════════════════════════════════════╝" +echo -e "${NC}" + +# Refuse outside an install — nothing to update if these aren't here. +if [[ ! -f docker-compose.yaml ]]; then + echo -e "${RED}Error: docker-compose.yaml not found in $(pwd)${NC}" + echo -e "${RED}Run this script from your Dograh install directory${NC}" + echo -e "${RED}(the 'dograh/' folder created by setup_remote.sh).${NC}" + exit 1 +fi + +if [[ ! -f .env ]]; then + echo -e "${RED}Error: .env not found in $(pwd)${NC}" + echo -e "${RED}This script updates an existing install — there is nothing here to update.${NC}" + echo -e "${RED}For a fresh install, see https://docs.dograh.com/deployment/docker${NC}" + exit 1 +fi + +# Build-mode installs update via git, not via this script. The presence of an +# override file is the definitive marker (created by setup_remote.sh in build +# mode and not in prebuilt mode). +if [[ -f docker-compose.override.yaml ]]; then + echo -e "${YELLOW}Build-mode install detected (docker-compose.override.yaml present).${NC}" + echo "" + echo -e "${YELLOW}This script is for prebuilt installs only. For build mode, update via git:${NC}" + echo "" + echo -e " ${BLUE}git fetch${NC}" + echo -e " ${BLUE}git checkout # or: git pull${NC}" + echo -e " ${BLUE}git submodule update --init --recursive${NC}" + echo -e " ${BLUE}sudo docker compose --profile remote build${NC}" + echo -e " ${BLUE}sudo docker compose --profile remote up -d${NC}" + echo "" + echo -e "${YELLOW}See https://docs.dograh.com/deployment/update#updating-a-source-build${NC}" + exit 1 +fi + +############################################################################### +### Discover existing config from .env +############################################################################### + +# Save anything the caller exported before we overwrite from .env. +_caller_FASTAPI_WORKERS="$FASTAPI_WORKERS" +_caller_TARGET_VERSION="$TARGET_VERSION" + +set -a +# shellcheck disable=SC1091 +. ./.env +set +a + +# SERVER_IP isn't a literal key in .env — derive it from BACKEND_API_ENDPOINT. +if [[ -z "$SERVER_IP" ]]; then + if [[ -n "$BACKEND_API_ENDPOINT" ]]; then + SERVER_IP="${BACKEND_API_ENDPOINT#https://}" + SERVER_IP="${SERVER_IP#http://}" + fi +fi + +if [[ -z "$SERVER_IP" ]]; then + echo -e "${RED}Error: could not determine SERVER_IP from .env${NC}" + echo -e "${RED}Expected BACKEND_API_ENDPOINT=https:// in .env${NC}" + exit 1 +fi + +if [[ -z "$TURN_SECRET" ]]; then + echo -e "${RED}Error: TURN_SECRET not found in .env${NC}" + exit 1 +fi + +# Reapply caller overrides on top of sourced .env so e.g. FASTAPI_WORKERS=8 ./update_remote.sh works. +[[ -n "$_caller_FASTAPI_WORKERS" ]] && FASTAPI_WORKERS="$_caller_FASTAPI_WORKERS" +[[ -n "$_caller_TARGET_VERSION" ]] && TARGET_VERSION="$_caller_TARGET_VERSION" + +############################################################################### +### Determine target version +############################################################################### + +if [[ -z "$TARGET_VERSION" ]]; then + echo -e "${BLUE}Fetching latest release tag from GitHub...${NC}" + LATEST_TAG=$(curl -fsSL "https://api.github.com/repos/$REPO/releases/latest" 2>/dev/null \ + | grep -E '"tag_name":' | head -1 \ + | sed -E 's/.*"tag_name":[[:space:]]*"([^"]+)".*/\1/' || true) + + if [[ -z "$LATEST_TAG" ]]; then + echo -e "${YELLOW}Could not auto-discover latest tag — defaulting to 'main'.${NC}" + LATEST_TAG="main" + fi + + if [[ -t 0 ]]; then + echo "" + echo -e "${YELLOW}Target version (release tag like v1.29.0, or 'main' for bleeding edge):${NC}" + read -p "[$LATEST_TAG]: " TARGET_VERSION + TARGET_VERSION="${TARGET_VERSION:-$LATEST_TAG}" + else + TARGET_VERSION="$LATEST_TAG" + fi +fi + +# "latest" isn't a real ref on GitHub — treat it as "latest release". +if [[ "$TARGET_VERSION" == "latest" ]]; then + TARGET_VERSION=$(curl -fsSL "https://api.github.com/repos/$REPO/releases/latest" 2>/dev/null \ + | grep -E '"tag_name":' | head -1 \ + | sed -E 's/.*"tag_name":[[:space:]]*"([^"]+)".*/\1/' || true) + if [[ -z "$TARGET_VERSION" ]]; then + echo -e "${RED}Error: could not resolve 'latest' to a release tag${NC}" + exit 1 + fi +fi + +# Validate the tag/branch actually exists by HEAD-ing the compose file at that ref. +RAW_BASE="https://raw.githubusercontent.com/$REPO/$TARGET_VERSION" +echo -e "${BLUE}Validating target version: $TARGET_VERSION...${NC}" +if ! curl -fsI "$RAW_BASE/docker-compose.yaml" >/dev/null 2>&1; then + echo -e "${RED}Error: docker-compose.yaml not found at $TARGET_VERSION${NC}" + echo -e "${RED}Check the tag exists at: https://github.com/$REPO/releases${NC}" + exit 1 +fi +echo -e "${GREEN}✓ Target version is valid${NC}" + +############################################################################### +### Reconcile required keys that may be missing on older installs +############################################################################### + +if [[ -z "$FASTAPI_WORKERS" ]]; then + if [[ -t 0 ]]; then + echo "" + echo -e "${YELLOW}FASTAPI_WORKERS not set in .env. Number of uvicorn workers nginx will load-balance:${NC}" + read -p "[4]: " FASTAPI_WORKERS + FASTAPI_WORKERS="${FASTAPI_WORKERS:-4}" + else + FASTAPI_WORKERS="4" + fi +fi + +if ! [[ "$FASTAPI_WORKERS" =~ ^[1-9][0-9]*$ ]]; then + echo -e "${RED}Error: FASTAPI_WORKERS must be a positive integer (got: $FASTAPI_WORKERS)${NC}" + exit 1 +fi + +############################################################################### +### Summary + confirmation +############################################################################### + +echo "" +echo -e "${GREEN}Update plan:${NC}" +echo -e " Server IP: ${BLUE}$SERVER_IP${NC}" +echo -e " Target version: ${BLUE}$TARGET_VERSION${NC}" +echo -e " FastAPI workers: ${BLUE}$FASTAPI_WORKERS${NC} (ports 8000..$((8000 + FASTAPI_WORKERS - 1)))" +echo "" +echo -e "${YELLOW}Files that will be replaced (backups saved with suffix .bak.$TIMESTAMP):${NC}" +echo " - docker-compose.yaml (pulled from GitHub at $TARGET_VERSION)" +echo " - nginx.conf (regenerated from this script's template)" +echo " - turnserver.conf (regenerated from this script's template)" +echo " - .env (existing values preserved; missing keys appended)" +echo "" +echo -e "${YELLOW}Any local customizations to these files will be overwritten — check the backup${NC}" +echo -e "${YELLOW}files if you need to re-apply edits afterwards.${NC}" +echo "" + +if [[ -t 0 && "$DOGRAH_UPDATE_YES" != "1" ]]; then + read -p "Proceed? [y/N]: " confirm + if ! [[ "$confirm" =~ ^[Yy] ]]; then + echo -e "${RED}Aborted.${NC}" + exit 1 + fi +fi + +############################################################################### +### Step 1 — backups +############################################################################### + +echo "" +echo -e "${BLUE}[1/5] Backing up existing files...${NC}" +for f in docker-compose.yaml nginx.conf turnserver.conf .env; do + if [[ -f "$f" ]]; then + cp -p "$f" "$f.bak.$TIMESTAMP" + echo -e " ${GREEN}✓ $f → $f.bak.$TIMESTAMP${NC}" + fi +done + +############################################################################### +### Step 2 — docker-compose.yaml (download + pin image tags) +############################################################################### + +echo -e "${BLUE}[2/5] Downloading docker-compose.yaml at $TARGET_VERSION...${NC}" +curl -fsSL -o docker-compose.yaml "$RAW_BASE/docker-compose.yaml" + +# Pin api/ui image tags for release tags (v*). Leave :latest alone if we're +# tracking a branch like 'main' so up --pull always still grabs the newest build. +if [[ "$TARGET_VERSION" =~ ^v ]]; then + sed -i.tmp -E "s#(dograh-(api|ui)):latest#\1:$TARGET_VERSION#g" docker-compose.yaml + rm -f docker-compose.yaml.tmp + echo -e "${GREEN}✓ docker-compose.yaml updated; images pinned to $TARGET_VERSION${NC}" +else + echo -e "${GREEN}✓ docker-compose.yaml updated (image tags left at :latest)${NC}" +fi + +############################################################################### +### Step 3 — nginx.conf (regenerate from embedded template) +############################################################################### + +echo -e "${BLUE}[3/5] Regenerating nginx.conf...${NC}" +{ + echo "# Backend API workers — one uvicorn process per port, balanced by least_conn." + echo "# Generated by update_remote.sh; regenerate to change worker count." + echo "upstream dograh_api {" + echo " least_conn;" + for ((i=0; i nginx.conf + +sed -i.tmp "s/SERVER_IP_PLACEHOLDER/$SERVER_IP/g" nginx.conf && rm -f nginx.conf.tmp +echo -e "${GREEN}✓ nginx.conf regenerated${NC}" + +############################################################################### +### Step 4 — turnserver.conf (regenerate from embedded template) +############################################################################### + +echo -e "${BLUE}[4/5] Regenerating turnserver.conf...${NC}" +cat > turnserver.conf << TURN_EOF +# Coturn TURN Server - Docker Configuration +# Auto-generated by update_remote.sh + +# Listener ports +listening-port=3478 +tls-listening-port=5349 + +# Relay port range +min-port=49152 +max-port=49200 + +# Network - external IP for NAT traversal +external-ip=$SERVER_IP + +# Realm +realm=dograh.com + +# Authentication (TURN REST API with time-limited credentials) +use-auth-secret +static-auth-secret=$TURN_SECRET + +# Security +fingerprint +no-cli +no-multicast-peers + +# Logging +log-file=stdout +TURN_EOF +echo -e "${GREEN}✓ turnserver.conf regenerated${NC}" + +############################################################################### +### Step 5 — reconcile .env (append missing keys; never overwrite existing) +############################################################################### + +echo -e "${BLUE}[5/5] Reconciling .env...${NC}" +if ! grep -q "^FASTAPI_WORKERS=" .env; then + { + echo "" + echo "# Number of uvicorn worker processes; nginx load-balances across them" + echo "# (ports 8000..$((8000 + FASTAPI_WORKERS - 1))) with least_conn." + echo "FASTAPI_WORKERS=$FASTAPI_WORKERS" + } >> .env + echo -e "${GREEN}✓ Added FASTAPI_WORKERS=$FASTAPI_WORKERS to .env${NC}" +else + echo -e "${GREEN}✓ .env already has FASTAPI_WORKERS — left unchanged${NC}" +fi + +############################################################################### +### Done — print restart + rollback instructions +############################################################################### + +echo "" +echo -e "${GREEN}╔══════════════════════════════════════════════════════════════╗${NC}" +echo -e "${GREEN}║ Update Prepared! ║${NC}" +echo -e "${GREEN}╚══════════════════════════════════════════════════════════════╝${NC}" +echo "" +echo -e "Backups: ${BLUE}*.bak.$TIMESTAMP${NC}" +echo "" +echo -e "${YELLOW}To apply, recreate the stack:${NC}" +echo "" +echo -e " ${BLUE}sudo docker compose --profile remote down${NC}" +echo -e " ${BLUE}sudo docker compose --profile remote up -d --pull always${NC}" +echo "" +echo -e "${YELLOW}To roll back, restore the backups and recreate:${NC}" +echo "" +echo -e " ${BLUE}for f in docker-compose.yaml nginx.conf turnserver.conf .env; do${NC}" +echo -e " ${BLUE} [[ -f \"\$f.bak.$TIMESTAMP\" ]] && cp \"\$f.bak.$TIMESTAMP\" \"\$f\"${NC}" +echo -e " ${BLUE}done${NC}" +echo -e " ${BLUE}sudo docker compose --profile remote down && sudo docker compose --profile remote up -d${NC}" +echo ""