apunkt/dograh
1
0
Fork 0
mirror of https://github.com/dograh-hq/dograh.git synced 2026-06-07 07:55:16 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat: an option to setup remote server with docker compose build (#280)

Browse source 
* feat: remote setup with docker build option

* chore: update documentation

* chore: make script run in non tty

* chore: add warning about slow build

* chore: add more documentation

* feat: add FASTAPI_WORKERS parameter

* feat: add scaling docs

* feat: add update script

* fix: fix semver options in update_remote.sh
This commit is contained in:
Abhishek 2026-05-13 17:22:14 +05:30 • committed by GitHub
parent b670004725
commit 59619e9eaa
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 1086 additions and 145 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docker-compose.yaml
View file

@ -107,8 +107,10 @@ services:
MINIO_BUCKET: "voice-audio"
MINIO_SECURE: "false"
# FastAPI workers count
FASTAPI_WORKERS: 1
# Number of uvicorn worker processes (each is its own process bound to a
# distinct port starting at 8000). nginx load-balances across them with
# least_conn — see setup_remote.sh.
FASTAPI_WORKERS: "${FASTAPI_WORKERS:-1}"
# Langfuse — credentials can be set here or per-organization via the UI
# at /settings. Tracing is automatically active when credentials are

99
docs/deployment/docker.mdx
View file

@ -6,7 +6,7 @@ description: "Deploy Dograh AI using Docker for local development and remote ser
Dograh AI can be deployed using Docker in two main configurations. Choose the option that best fits your needs:
- **Option 1**: For local development and testing on your own machine
- **Option 2**: For remote server deployment with HTTPS (using IP address)
- **Option 2**: For remote server deployment with HTTPS (using IP address). If you also have a custom domain, you can first deploy Dograh stack on your server using steps in this document and then proceed to the [Custom Domain](deployment/custom-domain) section.
## Option 1: Local Docker Deployment
@ -52,7 +52,7 @@ The Quick Start above relies on direct peer-to-peer WebRTC between your browser
- The call connects but no audio flows in either direction
- The browser console reports `iceConnectionState: failed`
- You are testing from a phone or another device on your LAN against the laptop running Docker
- You are testing from a browser on your smartphone or another device on your LAN against the laptop running Docker
- A VPN, corporate firewall, or strict NAT sits between the browser and Docker
For these cases, use the alternate local setup script which configures a coturn TURN server alongside the rest of the stack:
@ -90,17 +90,17 @@ Watch the video tutorial below for a step-by-step walkthrough of deploying Dogra
allowFullScreen
></iframe>
Deploy Dograh AI on a remote server to make it accessible from anywhere using your server's IP address. This setup includes HTTPS support via nginx reverse proxy with self-signed certificates. We need to serve the application over HTTPS, since modern browsers only allow microphone permissions for websites being served over HTTPS.
Deploy Dograh AI on a remote server to make it accessible from anywhere using your server's IP address. This setup includes HTTPS support via nginx reverse proxy with self-signed certificates. **We need to serve the application over HTTPS, since modern browsers only allow microphone permissions for websites being served over HTTPS**.
**We highly recommend you set up the platform on a fresh server, so that there are less chances of confliciting dependencies, and ports from other applications.**
<Warning>We highly recommend you set up the platform on a fresh server, so that there are less chances of confliciting dependencies, and ports from other applications.</Warning>
### Prerequisites
- A server with Docker and Docker Compose installed. It should have minimum of 8 GB RAM and 4 vCPUs.
- Public IP address for your server
- Public IP address for your server. You can also access the server using a local IP address in your VPC as long as its reachable from your browser.
- TCP Ports 80, 443, 3478, 5349 and UDP Ports 3478, 5349 and 49152:49200 reachable from Internet (Port 80 and 443 to access the UI and rest of the ports for WebRTC Signaling)
<Note>Please refer to your server hosting provider's documentaion on how you can open these ports in the firewall of the server.</Note>
<Note>IMPORTANT: Please double check the ports on your firewall. Please refer to your server hosting provider's documentaion on how you can open these ports in the firewall of the server.</Note>
### Quick Setup
@ -113,12 +113,15 @@ curl -o setup_remote.sh https://raw.githubusercontent.com/dograh-hq/dograh/main/
The script will prompt you for:
- Your server's public IP address
- A password for the TURN server (optional, press Enter for default)
- Deployment mode — press Enter for **prebuilt** (pulls official images, the recommended default) or pick **build** to compile images from source. Use **build** when you maintain a fork or want to deploy local customizations — see [Building from source](#building-from-source) below.
- Number of FastAPI workers (uvicorn processes nginx will load-balance) — press Enter for the default of `4`, or pick a value that suits your server's CPU count. See [Scaling](/deployment/scaling) for sizing guidance and how to change this on a running stack.
It will automatically:
- Download the docker-compose.yaml file
- Get the source — `docker-compose.yaml` only (prebuilt mode), or clone the full repo (build mode)
- Create and configure nginx.conf with your IP address
- Generate SSL certificates
- Create an environment file with TURN server configuration
- Write a `docker-compose.override.yaml` with build directives (build mode only)
### Start the Application
@ -126,12 +129,20 @@ It will automatically:
Please ensure that Docker Compose is installed on your machine before proceeding further. You can check whether its installed by running `docker compose version` command. If its not installed, please install it by following your server provider documentation.
</Note>
After the setup script completes, start Dograh:
After the setup script completes, start Dograh. The script prints the exact command to run at the end — it differs slightly between modes:
```bash
<CodeGroup>
```bash Prebuilt mode
cd dograh
sudo docker compose --profile remote up --pull always
```
```bash Build mode
cd dograh
sudo docker compose --profile remote up -d --build
```
</CodeGroup>
First boot in build mode takes several minutes — Docker has to build both the API and UI images before the stack comes up.
### Access Your Application
@ -162,12 +173,80 @@ The setup script creates the following files in the `dograh/` directory:
| File | Purpose |
|------|---------|
| `docker-compose.yaml` | Main Docker Compose configuration |
| `docker-compose.override.yaml` | Build directives for `api` and `ui` (**build mode only**) |
| `turnserver.conf` | Configuration for TURN server |
| `nginx.conf` | nginx reverse proxy configuration with your IP |
| `generate_certificate.sh` | Script to regenerate SSL certificates |
| `certs/local.crt` | Self-signed SSL certificate |
| `certs/local.key` | SSL private key |
| `.env` | Environment variables for TURN server |
| `.env` | Environment variables (TURN secret, JWT secret, FastAPI worker count) |
### Building from source
If you maintain a fork or want to ship local code changes without waiting for an official image release, pick **build** when the setup script prompts for deployment mode.
In this mode the script:
1. Clones the repo into `dograh/` (default: `dograh-hq/dograh@main`; the script will ask for an `owner/name` and `branch` so a fork can point at its own).
2. If you run the script from inside an existing dograh checkout, offers to use that checkout instead of cloning.
3. Generates a `docker-compose.override.yaml` next to `docker-compose.yaml` that swaps the `image:` directives for local `build:` directives. Docker Compose auto-loads the override file — you don't need to pass any `-f` flags.
#### Updating after a code change
Whenever the codebase changes — a `git pull`, a local edit, or a submodule bump — rebuild the `api` and `ui` images and recreate the running containers so they pick up the new code.
**TL;DR (rebuilds both services and restarts in one shot):**
```bash
cd dograh
sudo docker compose --profile remote up -d --build
```
`--build` rebuilds any service whose `build:` context has changed (both `api` and `ui` are wired through the `docker-compose.override.yaml` written by `setup_remote.sh`), and `up -d` recreates the affected containers so they run off the new image.
**Step-by-step** — use this when you're pulling fresh code from a remote, or want to be explicit:
```bash
cd dograh
# 1. Pull the latest source.
git pull
git submodule update --init --recursive # picks up pipecat submodule bumps
# 2. Rebuild both images.
sudo docker compose --profile remote build api ui
# 3. Recreate the running containers so they use the new images.
sudo docker compose --profile remote up -d
```
<Warning>If you update `pipecat` submodule, you must do `git submodule update --init --recursive`, or the Docker build step will not pick up `pipecat` changes</Warning>
**Rebuild a single service** — faster when you only changed one side:
```bash
sudo docker compose --profile remote build api # or: build ui
sudo docker compose --profile remote up -d api # or: up -d ui
```
**Force a clean rebuild** — needed when a base image changed, a `pip install` was added, or you suspect a stale layer:
```bash
sudo docker compose --profile remote build --no-cache api ui
sudo docker compose --profile remote up -d
```
**Verify the containers were actually recreated** — check the `CREATED` column shows a recent timestamp, not minutes/hours ago:
```bash
sudo docker compose --profile remote ps
```
To revert to pulling official images, delete `docker-compose.override.yaml` and start the stack with `--pull always` as in the [prebuilt flow](#start-the-application).
<Note>
Build mode is for **fork maintainers and self-hosters who want to deploy customized images** — not for active development on the code itself. For day-to-day contributor work (fast reload, IDE/LSP integration), use the [contributor setup](/contribution/setup) instead.
</Note>
### Next Steps
Now that you are able to create and test a voice agent, you can setup a custom domain and setup SSL using letsencrypt. Checkout [Custom Domain](custom-domain) for instructions on how to do that.

159
docs/deployment/scaling.mdx Normal file
View file

@ -0,0 +1,159 @@
---
title: "Scaling"
description: "Run multiple FastAPI worker processes behind nginx for higher throughput"
---
By default, the Dograh API container runs a single uvicorn worker. For production traffic — especially with many concurrent voice calls (long-lived WebSockets) — you'll want multiple workers. Dograh ships with built-in support for this: nginx load-balances across N independent uvicorn processes using a `least_conn` strategy.
This page covers how the multi-worker setup works, how to choose a worker count at install time, and how to change it on a running stack.
<Warning>
Multi-worker support requires **Dograh v1.29.0 or newer**. Earlier releases used `uvicorn --workers` and ship a different `setup_remote.sh` / `start_services_docker.sh` / `nginx.conf` layout — the steps below will not work on them. If your stack is older, [update first](/deployment/update) and then come back to this page.
</Warning>
## How it works
The API container starts `FASTAPI_WORKERS` separate uvicorn processes, each bound to its own port (`8000`, `8001`, `8002`, …). nginx exposes a single upstream `dograh_api` that includes all worker ports and routes new requests to whichever worker currently has the **fewest active connections**.
```
┌───────────────────────────────────┐
│ api container │
│ uvicorn worker 0 → :8000 │
browser ──► nginx ──► │ uvicorn worker 1 → :8001 │
(443) (least_conn) uvicorn worker 2 → :8002 │
│ uvicorn worker 3 → :8003 │
└───────────────────────────────────┘
```
<Note>
This is intentionally **not** `uvicorn --workers N` (the built-in pre-fork mode). With pre-fork, the Linux kernel distributes new TCP connections across workers via `accept()` — fine for short HTTP requests, but long-lived WebSockets stick to whichever worker first accepted them. A handful of unlucky workers end up handling most of the streaming traffic while the others idle. Routing at the nginx layer with `least_conn` knows the actual per-worker connection count and distributes WebSockets evenly.
</Note>
The `ari_manager` and `campaign_orchestrator` processes inside the API container stay as **singletons** regardless of `FASTAPI_WORKERS` — they coordinate global state (Asterisk channels, campaign scheduling) and should not be duplicated. ARQ background workers are controlled separately via `ARQ_WORKERS`.
## Choosing a worker count
A safe starting point is **one worker per available vCPU**, capped at 8 unless you've profiled your workload. The [Remote Server Deployment prerequisites](/deployment/docker#prerequisites) ask for a minimum of 4 vCPUs, so:
| vCPUs | Suggested `FASTAPI_WORKERS` |
|-------|-----------------------------|
| 4 | 4 |
| 8 | 6–8 |
| 16+ | profile first |
Each worker holds its own Python process and memory — budget roughly **300–500 MB RAM per worker** in addition to the postgres/redis/minio overhead. If you're near the 8 GB RAM minimum and see OOMs, drop the worker count before adding more.
## Setting the worker count at install time
`setup_remote.sh` prompts for the worker count alongside the other configuration:
```
Number of FastAPI workers (uvicorn processes nginx will load-balance):
[4]:
```
Press Enter for the default (`4`) or enter a different positive integer. Non-interactive callers (cloud-init, CI, Terraform) can set the value via environment variable instead:
```bash
SERVER_IP=... TURN_SECRET=... FASTAPI_WORKERS=8 ./setup_remote.sh
```
The script wires the value into two places:
- **`.env`** — sets `FASTAPI_WORKERS=N`, which `docker-compose.yaml` substitutes into the API container's environment.
- **`nginx.conf`** — generates an `upstream dograh_api` block with one `server api:800X` entry per worker.
Both must agree, which is why the script generates them together.
## Changing the worker count on a running stack
Once Dograh is running, increasing or decreasing the worker count is a two-file edit plus a restart. You'll touch:
1. **`.env`** — controls how many uvicorn processes the API container spawns.
2. **`nginx.conf`** — controls which worker ports nginx forwards to.
<Warning>
Both files must stay in sync. If `.env` says `FASTAPI_WORKERS=8` but `nginx.conf` only lists 4 upstream servers, half your workers will be idle. If `nginx.conf` lists more upstreams than there are workers, those upstreams will throw connection errors and trip the `proxy_next_upstream` fallback.
</Warning>
### Steps
All commands run from your `dograh/` directory (the one with `docker-compose.yaml`).
**1. Edit `.env`** and change the `FASTAPI_WORKERS` line:
```bash
# Before
FASTAPI_WORKERS=4
# After
FASTAPI_WORKERS=8
```
**2. Edit `nginx.conf`** and update the `upstream dograh_api` block so it has exactly one `server api:800X` line per worker, with ports starting at `8000`:
```nginx
upstream dograh_api {
least_conn;
server api:8000 max_fails=3 fail_timeout=10s;
server api:8001 max_fails=3 fail_timeout=10s;
server api:8002 max_fails=3 fail_timeout=10s;
server api:8003 max_fails=3 fail_timeout=10s;
server api:8004 max_fails=3 fail_timeout=10s; # ← new
server api:8005 max_fails=3 fail_timeout=10s; # ← new
server api:8006 max_fails=3 fail_timeout=10s; # ← new
server api:8007 max_fails=3 fail_timeout=10s; # ← new
keepalive 32;
}
```
To **scale down**, remove the trailing `server` lines so the list matches the new `FASTAPI_WORKERS` value.
**3. Recreate the affected containers.** The simplest path — brief downtime, no surprises:
```bash
sudo docker compose --profile remote down
sudo docker compose --profile remote up -d
```
If you want to avoid downtime and your stack is healthy, you can recreate only the `api` and `nginx` containers:
```bash
sudo docker compose --profile remote up -d --force-recreate api nginx
```
`--force-recreate` ensures the api container picks up the new `FASTAPI_WORKERS` value and nginx re-reads the updated `nginx.conf` (which is mounted read-only from disk).
**4. Verify.** Confirm the right number of uvicorn processes are running. The API image is slim and doesn't include `ps`, so use Docker's host-side view instead:
```bash
sudo docker compose --profile remote top api | grep uvicorn
```
You should see one line per worker. To confirm the bound ports, check the startup logs — each worker logs an `Uvicorn running on http://0.0.0.0:800X` line on boot:
```bash
sudo docker compose --profile remote logs api | grep "Uvicorn running"
```
Then hit the API through nginx to confirm requests still flow:
```bash
curl -k https://YOUR_SERVER_IP/api/v1/health
```
### Why not just re-run `setup_remote.sh`?
`setup_remote.sh` refuses to overwrite an existing install by design — re-running it would regenerate `OSS_JWT_SECRET` (logging everyone out), reset the TURN shared secret (breaking WebRTC auth on connected clients), and regenerate SSL certificates. The two-file edit above is the supported way to change worker count after install.
If you genuinely want a clean reinstall, see the `DOGRAH_FORCE_OVERWRITE=1` escape hatch documented in the script.
## What this does not scale
Multi-worker mode scales the HTTP/WebSocket API surface. It does **not** scale:
- **ARQ background workers** — controlled by `ARQ_WORKERS` (defaults to 1). Increase this in the API container's environment if your background job queue backs up.
- **`ari_manager` / `campaign_orchestrator`** — singletons by design; they don't benefit from extra processes.
- **Postgres, Redis, MinIO** — each runs as a single container in the stack. For production-scale Postgres you'd run a managed service and point `DATABASE_URL` at it; the same applies to Redis and S3-compatible storage.
For multi-machine horizontal scaling (separate API containers across hosts), see the [Custom Domain](/deployment/custom-domain) guide for the load-balancer-in-front-of-multiple-hosts pattern — it's the same idea as the in-container `least_conn` upstream, just one layer higher.

134
docs/deployment/update.mdx
View file

@ -5,6 +5,12 @@ description: "Update your self-hosted Dograh stack to a newer image version"
This guide covers updating a Dograh stack you've already deployed with [Docker](/deployment/docker) or a [custom domain](/deployment/custom-domain). You run commands from the same directory that contains your `docker-compose.yaml` (this is the `dograh/` directory if you used `setup_remote.sh`).
There are three update flows depending on how you deployed:
- **Remote, prebuilt mode** (most users) — use [`update_remote.sh`](#remote-prebuilt-mode-recommended) below.
- **Local Docker** — pull images and restart; see [Local deployment](#local-deployment).
- **Remote, build mode** (you have a `docker-compose.override.yaml`) — update via git; see [Updating a source build](#updating-a-source-build).
## Find an image version
Dograh publishes two images — `dograh-api` and `dograh-ui` — to both container registries:
@ -12,81 +18,64 @@ Dograh publishes two images — `dograh-api` and `dograh-ui` — to both contain
- **GitHub Container Registry** — [github.com/orgs/dograh-hq/packages](https://github.com/orgs/dograh-hq/packages)
- **Docker Hub** — [hub.docker.com/u/dograhai](https://hub.docker.com/u/dograhai)
Each release is published under two kinds of tags:
Each release is published under two kinds of tags. Note the formats differ between GitHub releases and the Docker image tags — `update_remote.sh` understands both and normalizes for you.
| Tag style | Example | When to use |
|-----------|---------|-------------|
| **Release tag** | `v0.8.2` | Stable, recommended for production |
| **Git commit SHA** | `a1b2c3d` | Bleeding edge — any commit merged to `main` |
| `latest` | `latest` | Tracks the most recent release tag |
| Where | Tag format | Example | When to use |
|-------|-----------|---------|-------------|
| GitHub release tag | `dograh-vX.Y.Z` | `dograh-v1.28.0` | What you see at [github.com/dograh-hq/dograh/releases](https://github.com/dograh-hq/dograh/releases) |
| Docker image tag (semver) | `X.Y.Z` | `1.28.0` | Stable, recommended for production |
| Docker image tag (SHA) | short SHA | `a1b2c3d` | Bleeding edge — any commit merged to `main` |
| Docker image tag (`latest`) | `latest` | `latest` | Tracks the most recent release tag |
<Warning>
Always update **`dograh-api`** and **`dograh-ui`** to the **same tag**. The two images are built from the same commit and the UI expects API responses in a matching shape — mixing versions will break the app.
Always update **`dograh-api`** and **`dograh-ui`** to the **same tag**. The two images are built from the same commit and the UI expects API responses in a matching shape — mixing versions will break the app. `update_remote.sh` handles this for you automatically.
</Warning>
## Option A: Update to the latest release
## Remote, prebuilt mode (recommended)
If your `docker-compose.yaml` uses `:latest` (the default), just pull and restart:
`update_remote.sh` is the supported path for updating a stack created with `setup_remote.sh`. In one shot it:
<CodeGroup>
```bash Local deployment
docker compose down
docker compose up --pull always
```
```bash Remote deployment
cd dograh
sudo docker compose --profile remote down
sudo docker compose --profile remote up --pull always
```
</CodeGroup>
- Asks for a target version (defaults to the latest release tag on GitHub).
- Pulls `docker-compose.yaml` at that version and pins both `api` and `ui` images to it.
- Regenerates `nginx.conf` and `turnserver.conf` from the upstream templates, so newer features (like [multi-worker scaling](/deployment/scaling)) are wired up correctly without manual editing.
- Reads your existing `.env` and appends any new required keys with safe defaults — your `OSS_JWT_SECRET`, `TURN_SECRET`, and other values are never touched.
- Backs up every file it changes with a `.bak.<timestamp>` suffix.
`--pull always` forces Docker to fetch the latest `:latest` from the registry instead of reusing your cached image.
## Option B: Pin a specific tag
To update (or roll back) to a specific release or commit, edit `docker-compose.yaml` and change the `image:` lines for both `api` and `ui` services to the same tag.
Open the file:
From your install directory:
```bash
nano docker-compose.yaml
cd dograh
curl -o update_remote.sh https://raw.githubusercontent.com/dograh-hq/dograh/main/scripts/update_remote.sh
bash update_remote.sh
```
Find these two lines:
You'll be prompted for the target version, defaulting to the most recent release. Accepted forms: bare semver (`1.28.0`), v-prefixed (`v1.28.0`), the full GitHub tag (`dograh-v1.28.0`), or `main` for bleeding edge — the script normalizes them. Non-interactive callers can set it via environment variable and skip the confirmation prompt:
```yaml
api:
image: ${REGISTRY:-dograhai}/dograh-api:latest
ui:
image: ${REGISTRY:-dograhai}/dograh-ui:latest
```bash
TARGET_VERSION=1.28.0 DOGRAH_UPDATE_YES=1 bash update_remote.sh
```
Replace `:latest` with your chosen tag on **both** services — for example:
After the script finishes, apply the update by recreating the stack:
```yaml
api:
image: ${REGISTRY:-dograhai}/dograh-api:v0.8.2
ui:
image: ${REGISTRY:-dograhai}/dograh-ui:v0.8.2
```bash
sudo docker compose --profile remote down
sudo docker compose --profile remote up -d --pull always
```
<Note>
You can use either registry. Leave `REGISTRY` unset for Docker Hub (`dograhai`), or export `REGISTRY=ghcr.io/dograh-hq` to pull from GitHub Container Registry.
The script overwrites `docker-compose.yaml`, `nginx.conf`, and `turnserver.conf` from upstream templates. If you've made local edits to any of these (extra environment variables, custom ports, modified nginx routes), check the `.bak.<timestamp>` files after the update and re-apply your edits.
</Note>
Then bring the stack down and back up:
## Local deployment
<CodeGroup>
```bash Local deployment
For local Docker installs (the [Quick Start](/deployment/docker#quick-start) flow or `setup_local.sh`), there are no host-side config files to refresh — pull new images and restart:
```bash
docker compose down
docker compose up --pull always
```
```bash Remote deployment
cd dograh
sudo docker compose --profile remote down
sudo docker compose --profile remote up --pull always
```
</CodeGroup>
To pin a specific version instead of `latest`, edit `docker-compose.yaml` and change both `image:` lines for `api` and `ui` to the same tag (e.g. `:1.28.0` — Docker image tags use bare semver, no `v` prefix), then run the commands above.
## Verify the update
@ -101,13 +90,54 @@ You should see the API and UI both running the tag you pinned.
Hit the health endpoint to confirm the API is responding:
```bash
curl http://localhost:8000/api/v1/health
curl -k https://YOUR_SERVER_IP/api/v1/health # remote
curl http://localhost:8000/api/v1/health # local
```
## Roll back
If something breaks, roll back by pinning the previous tag using the same process in **Option B** and restarting. Your Postgres data volume persists across `down`/`up` cycles, so agents and call history are preserved.
`update_remote.sh` saves backups of every file it touched. To roll back, restore them and recreate the stack — the exact commands (including the timestamp it used) are printed at the end of the script's output. The generic form:
```bash
cd dograh
for f in docker-compose.yaml nginx.conf turnserver.conf .env; do
[[ -f "$f.bak.<timestamp>" ]] && cp "$f.bak.<timestamp>" "$f"
done
sudo docker compose --profile remote down
sudo docker compose --profile remote up -d
```
Your Postgres data volume persists across `down`/`up` cycles, so agents and call history are preserved.
<Warning>
Rolling back across a database migration is not always safe — if the newer release ran a schema migration, downgrading may leave the DB in a state the older API doesn't understand. If in doubt, [open an issue](https://github.com/dograh-hq/dograh/issues) before rolling back.
</Warning>
## Updating a source build
If you deployed in **build mode** (you'll have a `docker-compose.override.yaml` in your install directory), `update_remote.sh` deliberately refuses to run — you already have the full repo locally and update via git:
```bash
cd dograh
git fetch
# Track latest main:
git pull
# Or pin to a specific release (git tag format is dograh-vX.Y.Z):
git checkout dograh-v1.28.0
# Pick up pipecat and other submodule bumps
git submodule update --init --recursive
# Rebuild and restart
sudo docker compose --profile remote build
sudo docker compose --profile remote up -d
```
<Warning>
If you update the `pipecat` submodule, you **must** run `git submodule update --init --recursive` before rebuilding, or the Docker build will not pick up `pipecat` changes.
</Warning>
If you maintain a fork with local customizations on top of upstream, merging conflicts in `docker-compose.yaml`, `nginx.conf`, `turnserver.conf`, or `setup_remote.sh` is up to you — resolve them as you would any other git merge. Leave `OSS_JWT_SECRET` and `TURN_SECRET` in `.env` unchanged across updates to preserve sessions and WebRTC auth.
The same migration warning above applies: rolling back across a schema change can leave the DB in a state the older API can't read.

1
docs/docs.json
View file

@ -145,6 +145,7 @@
"deployment/introduction",
"deployment/docker",
"deployment/custom-domain",
"deployment/scaling",
"deployment/update",
"deployment/heroku"
]

5
docs/getting-started/index.mdx
View file

@ -1,11 +1,10 @@
---
title: "Introduction"
description: "Open-source alternative to Vapi - build voice AI agents with full control & transparency"
---
## About Dograh
**Dograh is an open-source alternative to Vapi** - Dograh helps you build voice AI agents with an easy drag-and-drop workflow builder. Unlike proprietary solutions like Vapi, Dograh gives you:
**Dograh is an open-source alternative to platforms like Vapi and Retell** - Dograh helps you build voice AI agents with an easy drag-and-drop workflow builder. Unlike proprietary solutions like Vapi, Dograh gives you:
- **100% open source** - no vendor lock-in, full transparency
- **Self-hostable** - deploy anywhere, own your infrastructure
- **Complete control** - every line of code is open and customizable
@ -25,7 +24,7 @@ Watch the following video to learn about Dograh’s capabilities.
## Setting up
Get the platform up and running using Docker with a single command on your local computer. If you are looking to deploy the platform on a server, please check the [Deployment](deployment/introduction) section.
Get the platform up and running using Docker with a single command on your local computer. If you are looking to deploy the platform on a server different than your local machine, please check the [Deployment](deployment/introduction) section.
<Note>We collect anonymous usage data to improve the product. You can opt out by setting the `ENABLE_TELEMETRY` to `false` in the below command.</Note>

74
docs/integrations/telephony/agent-stream.mdx
View file

@ -1,19 +1,16 @@
---
title: "Agent Stream"
description: "Stream audio to a Dograh agent over a WebSocket using inline provider credentials"
description: "Stream audio to a Dograh agent over a WebSocket"
---
## Overview
Agent Stream is a WebSocket endpoint that lets an external caller drive a Dograh agent run by passing everything inline in the query string — including the telephony provider's credentials. Unlike the standard inbound webhook flow, this path does **not** require a stored telephony configuration in your Dograh organization. You bring the credentials with you when you connect.
Agent Stream is a WebSocket endpoint that lets a telephony provider point its media stream at a single URL and drive a Dograh agent run. The agent UUID in the URL selects the agent; provider-specific identifiers in the query string (for Cloudonix: `Domain`) tell Dograh which stored telephony configuration to use for that call. The bearer token and other credentials are never passed in the URL — they live in the stored configuration and are used by Dograh to validate the session and to issue provider API calls (hangup, transfer) during the call.
This is useful when:
- You manage your telephony tenant outside Dograh and only want Dograh as the agent runtime
- You're integrating Dograh into a SIP gateway or in-house dialer
- You need a single endpoint that doesn't need per-tenant DB rows for every caller
The endpoint authenticates with a Dograh **API key** (so the connection itself is authorized against your organization), and routes the audio through the named provider's serializer.
- You're integrating Dograh into a SIP gateway or in-house dialer that already speaks a supported provider's streaming protocol
- You want one stable endpoint per agent rather than wiring up an inbound webhook per phone number
<Warning>
Agent Stream currently supports the **Cloudonix** provider only. Other providers
@ -26,16 +23,15 @@ The endpoint authenticates with a Dograh **API key** (so the connection itself i
## Endpoint
```
wss://app.dograh.com/api/v1/agent-stream/{workflow_uuid}
wss://app.dograh.com/api/v1/agent-stream/{agent_uuid}
```
`{workflow_uuid}` is the agent's stable UUID (see [Get the Agent UUID](#get-the-agent-uuid) below). On self-hosted deployments, replace `app.dograh.com` with your backend host.
`{agent_uuid}` is the agent's stable UUID (see [Get the Agent UUID](#get-the-agent-uuid) below). On self-hosted deployments, replace `app.dograh.com` with your backend host.
## Prerequisites
- A Dograh agent (workflow) — published or in draft is fine
- A Dograh API key with access to the organization that owns the agent
- Provider credentials (for Cloudonix: bearer token + domain ID)
- A Cloudonix telephony configuration in your Dograh organization whose `domain_id` matches the `Domain` you pass on the URL. Dograh uses the bearer token from this configuration to validate the call session and to issue provider API calls (hangup, transfer).
## Get the Agent UUID
@ -53,47 +49,25 @@ The Agent UUID is the workflow's stable identifier — it doesn't change when ve
2. Scroll to the **Agent UUID** section (also linked in the right-side nav)
3. Click the UUID code block, or use the **Copy UUID** button
## Create a Dograh API key
Agent Stream auth uses your Dograh API key — not your provider's bearer token. The provider credentials go in separate query params (see [URL parameters](#url-parameters) below).
1. From the dashboard, navigate to **API Keys**
2. Click **Create API key**, give it a name, and copy the generated key (it starts with `dg_`)
3. Store it securely — Dograh shows the full key only once at creation time
For programmatic management, see the [API Keys reference](/api-reference/api-keys).
<Note>
API keys are scoped to an organization. The agent referenced by the URL must
belong to the same organization as the key, otherwise the connection is
rejected with WebSocket close code `1008`.
</Note>
## Connect to the WebSocket
### URL parameters
| Param | Required | Description |
| --- | --- | --- |
| `api_key` | Yes | Your Dograh API key (`dg_...`). Authorizes the connection. |
| `provider` | Yes | Provider name. Currently only `cloudonix` is supported. |
| `session` | Yes (cloudonix) | Cloudonix domain bearer token used to drive the call (hangup, etc.). |
| `AccountSid` | Yes (cloudonix) | Cloudonix domain ID. |
| `CallSid` | Yes (cloudonix) | Cloudonix call SID for this session. |
| `callId` | No | SIP-side call identifier; persisted on the workflow run for record-keeping. |
| `Domain` | Yes (cloudonix) | Cloudonix domain ID. Dograh uses this to look up the matching stored telephony configuration and retrieve the bearer token used for provider API calls. |
| `callId` / `CallSid` | No | Call identifier from your side; persisted on the workflow run's `gathered_context` as `call_id`. The Cloudonix call SID used for streaming is taken from the `start` event payload, not this param. |
| `from` | No | Caller phone number, persisted on the workflow run as `caller_number`. |
| `to` | No | Called phone number, persisted on the workflow run as `called_number`. |
### Cloudonix example
```
wss://app.dograh.com/api/v1/agent-stream/{workflow_uuid}
?api_key=dg_xxxxxxxxxxxxxxxx
&provider=cloudonix
&session={CLOUDONIX_BEARER_TOKEN}
&AccountSid={CLOUDONIX_DOMAIN_ID}
&CallSid={CALL_SID}
&callId={SIP_CALL_ID}
wss://app.dograh.com/api/v1/agent-stream/{agent_uuid}
?provider=cloudonix
&Domain={CLOUDONIX_DOMAIN_ID}
&callId={CALL_ID}
&from=+15555550100
&to=+15555550199
```
@ -104,23 +78,22 @@ Use this URL inside the CXML `<Stream>` your Cloudonix Voice Application returns
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Connect>
<Stream url="wss://app.dograh.com/api/v1/agent-stream/{workflow_uuid}?api_key=dg_...&provider=cloudonix&session=...&AccountSid=...&CallSid=...&callId=...&from=...&to=..."/>
<Stream url="wss://app.dograh.com/api/v1/agent-stream/{agent_uuid}?provider=cloudonix&Domain=...&callId=...&from=...&to=..."/>
</Connect>
<Pause length="40"/>
</Response>
```
The first two messages on the socket should be Cloudonix's standard `connected` and `start` events (Twilio-compatible framing). Dograh extracts `streamSid` and `callSid` from the `start` event and begins streaming audio.
The first two messages on the socket should be Cloudonix's standard `connected` and `start` events (Twilio-compatible framing). Dograh extracts `streamSid` and `callSid` from the `start` event payload, validates the session against Cloudonix using the bearer token from the stored telephony configuration matched by `Domain`, and then begins streaming audio.
## Workflow run lifecycle
When the WebSocket is accepted, Dograh:
1. Resolves your API key to a user and organization
2. Looks up the workflow by `workflow_uuid`, scoped to that organization
3. Runs a quota check
4. Creates a new `WorkflowRun` (`call_type=inbound`, `mode=cloudonix`, name `WR-AGS-XXXXXXXX`) with the `from`/`to`/`callId`/`AccountSid` fields stamped on `initial_context`
5. Transitions the run to `running` and starts the agent pipeline
1. Looks up the workflow by `agent_uuid`
2. Runs a quota check against the workflow's owning user
3. Creates a new `WorkflowRun` (`call_type=inbound`, `mode=cloudonix`, name `WR-AGS-XXXXXXXX`) with the `from`/`to` numbers stamped on `initial_context`, the `callId`/`CallSid` stored as `call_id` on `gathered_context`, and `Domain` recorded under the run's `inbound_webhook` log
4. Transitions the run to `running` and starts the agent pipeline
The run is visible under the agent's **Runs** tab as soon as it's minted, just like an inbound or outbound call.
@ -128,12 +101,11 @@ The run is visible under the agent's **Runs** tab as soon as it's minted, just l
| Code | Reason |
| --- | --- |
| `1008` | Auth or routing failure — invalid API key, unknown provider, workflow not found in your organization, or quota exceeded |
| `1008` | Routing failure — unknown provider, workflow not found, or quota exceeded |
| `1011` | Server-side failure or unsupported provider for Agent Stream |
| `4400` | Provider-level handshake error — for cloudonix, missing `session`/`AccountSid` or malformed `connected`/`start` events |
| `4400` | Provider-level handshake error — for cloudonix, missing `Domain`, no matching telephony configuration, missing bearer token on the configuration, malformed `connected`/`start` events, or session validation failed against Cloudonix |
## Security notes
- Treat the URL as a secret — both the Dograh API key and the provider bearer token sit in the query string. Store and transmit it only over TLS, and avoid logging the raw URL in places where access is broader than your operations team.
- Rotate the Dograh API key from the dashboard if you suspect it has leaked.
- The Dograh API key authorizes the connection against your organization. The provider bearer token is only used by Dograh to drive provider-side actions (e.g. hangup) — it is not stored on the workflow run.
- Treat the URL as a secret — the agent UUID itself authorizes the connection. Store and transmit it only over TLS, and avoid logging the raw URL in places where access is broader than your operations team.
- No bearer tokens or provider secrets are passed in the URL. Provider credentials live in the stored telephony configuration (matched by `Domain` for Cloudonix) and are used server-side by Dograh to validate the session and issue provider API calls.

294
scripts/setup_remote.sh
View file

@ -44,30 +44,208 @@ if [[ -z "$TURN_SECRET" ]]; then
echo -e "${BLUE}Generated random TURN secret${NC}"
fi
# Deployment mode. Skip prompt if DEPLOY_MODE is already set. Non-interactive
# callers (cloud-init, CI, terraform) without a TTY default to "prebuilt" so
# existing automation keeps working without changes - explicitly set
# DEPLOY_MODE=build to opt into source builds from a non-interactive context.
if [[ -z "$DEPLOY_MODE" ]]; then
if [[ -t 0 ]]; then
echo ""
echo -e "${YELLOW}Deployment mode:${NC}"
echo " 1) prebuilt - pull official dograh images (recommended, fastest)"
echo " 2) build - build images from source (for forks or local customizations)"
read -p "Choose [1]: " mode_choice
mode_choice="${mode_choice:-1}"
case "$mode_choice" in
1|prebuilt) DEPLOY_MODE="prebuilt" ;;
2|build) DEPLOY_MODE="build" ;;
*) echo -e "${RED}Error: invalid choice '$mode_choice'${NC}"; exit 1 ;;
esac
else
DEPLOY_MODE="prebuilt"
fi
fi
# Build mode needs source code - either use existing repo or clone fresh.
# Same TTY rule: prompt interactively, otherwise pick sensible defaults so
# automation that sets DEPLOY_MODE=build doesn't need to spell everything out.
if [[ "$DEPLOY_MODE" == "build" ]]; then
if [[ -z "$REPO_SOURCE" ]]; then
if [[ -d ".git" ]] && [[ -f "docker-compose.yaml" ]]; then
if [[ -t 0 ]]; then
echo ""
echo -e "${YELLOW}Detected a git repo with docker-compose.yaml in $(pwd).${NC}"
read -p "Build from this repo? [Y/n]: " use_existing
use_existing="${use_existing:-Y}"
if [[ "$use_existing" =~ ^[Yy] ]]; then
REPO_SOURCE="existing"
else
REPO_SOURCE="clone"
fi
else
REPO_SOURCE="existing"
fi
else
REPO_SOURCE="clone"
fi
fi
if [[ "$REPO_SOURCE" == "clone" ]]; then
if [[ -z "$FORK_REPO" ]]; then
if [[ -t 0 ]]; then
echo ""
echo -e "${YELLOW}GitHub repo to clone (format: owner/name):${NC}"
read -p "[dograh-hq/dograh]: " FORK_REPO
FORK_REPO="${FORK_REPO:-dograh-hq/dograh}"
else
FORK_REPO="dograh-hq/dograh"
fi
fi
if [[ -z "$BRANCH" ]]; then
if [[ -t 0 ]]; then
echo -e "${YELLOW}Branch:${NC}"
read -p "[main]: " BRANCH
BRANCH="${BRANCH:-main}"
else
BRANCH="main"
fi
fi
fi
fi
# Telemetry opt-out (default: true)
ENABLE_TELEMETRY="${ENABLE_TELEMETRY:-true}"
echo ""
echo -e "${GREEN}Configuration:${NC}"
echo -e " Server IP: ${BLUE}$SERVER_IP${NC}"
echo -e " TURN Secret: ${BLUE}********${NC}"
echo ""
# Create project directory and download compose file (skip when
# DOGRAH_SKIP_DOWNLOAD=1 — e.g. e2e tests that already have a cloned repo).
if [[ "$DOGRAH_SKIP_DOWNLOAD" != "1" ]]; then
mkdir -p dograh 2>/dev/null || true
cd dograh
echo -e "${BLUE}[1/5] Downloading docker-compose.yaml...${NC}"
curl -sS -o docker-compose.yaml https://raw.githubusercontent.com/dograh-hq/dograh/main/docker-compose.yaml
echo -e "${GREEN}✓ docker-compose.yaml downloaded${NC}"
else
echo -e "${BLUE}[1/5] Using docker-compose.yaml in current directory${NC}"
# Number of uvicorn worker processes. Each runs as its own process on a
# distinct port (8000, 8001, ...) and nginx balances across them with
# least_conn. Better than uvicorn --workers for long-lived WebSocket
# connections, which would otherwise stick to whichever worker accepted them.
if [[ -z "$FASTAPI_WORKERS" ]]; then
if [[ -t 0 ]]; then
echo ""
echo -e "${YELLOW}Number of FastAPI workers (uvicorn processes nginx will load-balance):${NC}"
read -p "[4]: " FASTAPI_WORKERS
FASTAPI_WORKERS="${FASTAPI_WORKERS:-4}"
else
FASTAPI_WORKERS="4"
fi
fi
echo -e "${BLUE}[2/5] Creating nginx.conf...${NC}"
cat > nginx.conf << 'NGINX_EOF'
if ! [[ "$FASTAPI_WORKERS" =~ ^[1-9][0-9]*$ ]]; then
echo -e "${RED}Error: FASTAPI_WORKERS must be a positive integer (got: $FASTAPI_WORKERS)${NC}"
exit 1
fi
# Where setup artifacts (.env, certs, nginx.conf, etc.) will land. Build mode
# with an existing repo writes them next to docker-compose.yaml in cwd;
# everything else writes into a fresh dograh/ subdirectory.
if [[ "$DEPLOY_MODE" == "build" && "$REPO_SOURCE" == "existing" ]]; then
TARGET_DIR="."
else
TARGET_DIR="dograh"
fi
# Refuse to overwrite an existing install - re-running this script would
# regenerate OSS_JWT_SECRET (invalidating logged-in sessions), reset the
# TURN secret (breaking WebRTC auth), and overwrite nginx.conf customizations.
# Set DOGRAH_FORCE_OVERWRITE=1 to bypass; DOGRAH_SKIP_DOWNLOAD=1 (used by e2e)
# also bypasses since those flows manage state themselves.
if [[ "$DOGRAH_FORCE_OVERWRITE" != "1" && "$DOGRAH_SKIP_DOWNLOAD" != "1" ]]; then
if [[ -f "$TARGET_DIR/.env" ]]; then
if [[ "$TARGET_DIR" == "." ]]; then
existing_path="$(pwd)/.env"
else
existing_path="$(pwd)/$TARGET_DIR/.env"
fi
echo ""
echo -e "${YELLOW}Detected an existing Dograh install:${NC}"
echo -e " ${YELLOW}$existing_path${NC}"
echo ""
echo -e "${RED}Refusing to continue - re-running setup would:${NC}"
echo -e "${RED} - overwrite .env (invalidates sessions, breaks TURN auth)${NC}"
echo -e "${RED} - regenerate SSL certificates${NC}"
echo -e "${RED} - reset nginx.conf and turnserver.conf customizations${NC}"
echo ""
echo -e "${BLUE}To upgrade an existing install, follow:${NC}"
echo -e " ${BLUE}https://docs.dograh.com/deployment/update${NC}"
echo ""
echo -e "${BLUE}To wipe state and reinstall from scratch, re-run with:${NC}"
echo -e " ${BLUE}DOGRAH_FORCE_OVERWRITE=1 <same command>${NC}"
echo ""
exit 1
fi
fi
# Total step count depends on mode (build adds the override-file step)
if [[ "$DEPLOY_MODE" == "build" ]]; then
TOTAL=7
else
TOTAL=6
fi
echo ""
echo -e "${GREEN}Configuration:${NC}"
echo -e " Server IP: ${BLUE}$SERVER_IP${NC}"
echo -e " TURN Secret: ${BLUE}********${NC}"
echo -e " Deploy mode: ${BLUE}$DEPLOY_MODE${NC}"
echo -e " FastAPI workers: ${BLUE}$FASTAPI_WORKERS${NC} (ports 8000..$((8000 + FASTAPI_WORKERS - 1)))"
if [[ "$DEPLOY_MODE" == "build" ]]; then
if [[ "$REPO_SOURCE" == "clone" ]]; then
echo -e " Source: ${BLUE}clone $FORK_REPO@$BRANCH${NC}"
else
echo -e " Source: ${BLUE}existing repo at $(pwd)${NC}"
fi
fi
echo ""
# Step 1: get the source - either the standalone compose file (prebuilt mode)
# or the full repo (build mode). Skip the download/clone when
# DOGRAH_SKIP_DOWNLOAD=1 (e.g. e2e tests that already have everything in place).
if [[ "$DEPLOY_MODE" == "build" ]]; then
if [[ "$DOGRAH_SKIP_DOWNLOAD" == "1" ]]; then
echo -e "${BLUE}[1/$TOTAL] Using existing repo in current directory${NC}"
elif [[ "$REPO_SOURCE" == "clone" ]]; then
if [[ -e "dograh" ]]; then
echo -e "${RED}Error: 'dograh' directory already exists. Remove it or re-run with REPO_SOURCE=existing from inside it.${NC}"
exit 1
fi
echo -e "${BLUE}[1/$TOTAL] Cloning $FORK_REPO (branch: $BRANCH)...${NC}"
git clone --branch "$BRANCH" --recurse-submodules "https://github.com/$FORK_REPO.git" dograh
cd dograh
echo -e "${GREEN}✓ Repo cloned${NC}"
else
echo -e "${BLUE}[1/$TOTAL] Using existing repo at $(pwd)${NC}"
fi
else
if [[ "$DOGRAH_SKIP_DOWNLOAD" != "1" ]]; then
mkdir -p dograh 2>/dev/null || true
cd dograh
echo -e "${BLUE}[1/$TOTAL] Downloading docker-compose.yaml...${NC}"
curl -sS -o docker-compose.yaml https://raw.githubusercontent.com/dograh-hq/dograh/main/docker-compose.yaml
echo -e "${GREEN}✓ docker-compose.yaml downloaded${NC}"
else
echo -e "${BLUE}[1/$TOTAL] Using docker-compose.yaml in current directory${NC}"
fi
fi
echo -e "${BLUE}[2/$TOTAL] Creating nginx.conf...${NC}"
# Build the upstream block first (needs shell interpolation for the server
# lines), then append the static server blocks via a quoted heredoc. The
# SERVER_IP_PLACEHOLDER gets replaced by sed below.
{
echo "# Backend API workers — one uvicorn process per port, balanced by least_conn."
echo "# Generated by setup_remote.sh; regenerate to change worker count."
echo "upstream dograh_api {"
echo " least_conn;"
for ((i=0; i<FASTAPI_WORKERS; i++)); do
port=$((8000 + i))
echo " server api:$port max_fails=3 fail_timeout=10s;"
done
echo " keepalive 32;"
echo "}"
echo ""
cat << 'NGINX_EOF'
server {
listen 80;
server_name SERVER_IP_PLACEHOLDER;
@ -87,11 +265,15 @@ server {
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
# Backend API and WebSockets — bypass the UI, go straight to api:8000
# Backend API and WebSockets - bypass the UI, go straight to the
# api workers via the least_conn upstream defined above.
location /api/v1/ {
proxy_pass http://api:8000;
proxy_pass http://dograh_api;
proxy_http_version 1.1;
# Retry on a dead/restarting worker
proxy_next_upstream error timeout http_502 http_503 http_504;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
@ -145,12 +327,13 @@ server {
}
}
NGINX_EOF
} > nginx.conf
# Replace placeholder with actual IP
sed -i.bak "s/SERVER_IP_PLACEHOLDER/$SERVER_IP/g" nginx.conf && rm -f nginx.conf.bak
echo -e "${GREEN}✓ nginx.conf created${NC}"
echo -e "${BLUE}[3/5] Creating SSL certificate generation script...${NC}"
echo -e "${BLUE}[3/$TOTAL] Creating SSL certificate generation script...${NC}"
cat > generate_certificate.sh << CERT_EOF
#!/bin/bash
mkdir -p certs
@ -163,11 +346,11 @@ CERT_EOF
chmod +x generate_certificate.sh
echo -e "${GREEN}✓ generate_certificate.sh created${NC}"
echo -e "${BLUE}[4/5] Generating SSL certificates...${NC}"
echo -e "${BLUE}[4/$TOTAL] Generating SSL certificates...${NC}"
./generate_certificate.sh
echo -e "${GREEN}✓ SSL certificates generated${NC}"
echo -e "${BLUE}[5/6] Creating TURN server configuration...${NC}"
echo -e "${BLUE}[5/$TOTAL] Creating TURN server configuration...${NC}"
cat > turnserver.conf << TURN_EOF
# Coturn TURN Server - Docker Configuration
# Auto-generated by setup_remote.sh
@ -200,7 +383,7 @@ log-file=stdout
TURN_EOF
echo -e "${GREEN}✓ turnserver.conf created${NC}"
echo -e "${BLUE}[6/6] Creating environment file...${NC}"
echo -e "${BLUE}[6/$TOTAL] Creating environment file...${NC}"
OSS_JWT_SECRET=$(openssl rand -hex 32)
cat > .env << ENV_EOF
@ -222,9 +405,43 @@ OSS_JWT_SECRET=$OSS_JWT_SECRET
# Telemetry (set to false to disable)
ENABLE_TELEMETRY=$ENABLE_TELEMETRY
# Number of uvicorn worker processes; nginx load-balances across them
# (ports 8000..$((8000 + FASTAPI_WORKERS - 1))) with least_conn.
# Must match the upstream block in nginx.conf — re-run setup_remote.sh
# (with DOGRAH_FORCE_OVERWRITE=1) to change.
FASTAPI_WORKERS=$FASTAPI_WORKERS
ENV_EOF
echo -e "${GREEN}✓ .env file created${NC}"
# In build mode, write the override file that swaps prebuilt images for
# local builds. Compose auto-loads docker-compose.override.yaml, so no -f flag
# is needed at runtime.
if [[ "$DEPLOY_MODE" == "build" ]]; then
echo -e "${BLUE}[7/$TOTAL] Creating docker-compose.override.yaml...${NC}"
cat > docker-compose.override.yaml << 'OVERRIDE_EOF'
# Auto-generated by setup_remote.sh (build mode).
# Overrides docker-compose.yaml to build api and ui images from local source
# instead of pulling them from a registry. Remove this file to revert to
# pulling prebuilt images.
services:
api:
build:
context: .
dockerfile: api/Dockerfile
image: dograh-local/dograh-api:local
pull_policy: never
ui:
build:
context: .
dockerfile: ui/Dockerfile
image: dograh-local/dograh-ui:local
pull_policy: never
OVERRIDE_EOF
echo -e "${GREEN}✓ docker-compose.override.yaml created${NC}"
fi
echo ""
echo -e "${GREEN}╔══════════════════════════════════════════════════════════════╗${NC}"
echo -e "${GREEN}║ Setup Complete! ║${NC}"
@ -232,6 +449,9 @@ echo -e "${GREEN}╚════════════════════
echo ""
echo -e "Files created in ${BLUE}$(pwd)${NC}:"
echo " - docker-compose.yaml"
if [[ "$DEPLOY_MODE" == "build" ]]; then
echo " - docker-compose.override.yaml (build directives)"
fi
echo " - nginx.conf"
echo " - turnserver.conf"
echo " - generate_certificate.sh"
@ -241,7 +461,29 @@ echo " - .env"
echo ""
echo -e "${YELLOW}To start Dograh, run:${NC}"
echo ""
echo -e " ${BLUE}sudo docker compose --profile remote up --pull always${NC}"
# The script's own cd into dograh/ doesn't persist to the user's shell, so
# remind them to cd themselves — except when they're already there (build mode
# with REPO_SOURCE=existing, which writes into cwd).
if [[ "$DEPLOY_MODE" != "build" || "$REPO_SOURCE" != "existing" ]]; then
echo -e " ${BLUE}cd $(pwd)${NC}"
fi
if [[ "$DEPLOY_MODE" == "build" ]]; then
echo -e " ${BLUE}sudo docker compose --profile remote up -d --build${NC}"
echo ""
echo -e "${YELLOW}A docker-compose.override.yaml has been created alongside${NC}"
echo -e "${YELLOW}docker-compose.yaml. Compose auto-loads it, so no -f flag is${NC}"
echo -e "${YELLOW}needed — it swaps the prebuilt images for local builds.${NC}"
echo ""
echo -e "${YELLOW}The first build can take several minutes${NC}"
echo -e "${YELLOW}(downloading base images, installing dependencies).${NC}"
echo -e "${YELLOW}If you know how to speed this up, we would love a pull request.${NC}"
echo ""
echo -e "${YELLOW}To rebuild after editing api/ or ui/ code:${NC}"
echo ""
echo -e " ${BLUE}sudo docker compose --profile remote build && sudo docker compose --profile remote up -d${NC}"
else
echo -e " ${BLUE}sudo docker compose --profile remote up --pull always${NC}"
fi
echo ""
echo -e "${YELLOW}Your application will be available at:${NC}"
echo ""

11
scripts/start_services_docker.sh
View file

@ -61,7 +61,16 @@ start() {
start ari_manager python -m api.services.telephony.ari_manager
start campaign_orchestrator python -m api.services.campaign.campaign_orchestrator
start uvicorn uvicorn api.app:app --host 0.0.0.0 --port "$UVICORN_BASE_PORT" --workers "$FASTAPI_WORKERS"
# Spawn FASTAPI_WORKERS independent uvicorn processes on consecutive ports
# starting at UVICORN_BASE_PORT. nginx upstream (configured in setup_remote.sh)
# balances across them with least_conn — better than uvicorn --workers for
# long-lived WebSocket connections, which would otherwise stick to whichever
# worker accepted them first.
for ((i=0; i<FASTAPI_WORKERS; i++)); do
port=$((UVICORN_BASE_PORT + i))
start "uvicorn$i" uvicorn api.app:app --host 0.0.0.0 --port "$port" --workers 1
done
for ((i=1; i<=ARQ_WORKERS; i++)); do
start "arq$i" python -m arq api.tasks.arq.WorkerSettings --custom-log-dict api.tasks.arq.LOG_CONFIG

448
scripts/update_remote.sh Executable file
View file

@ -0,0 +1,448 @@
#!/bin/bash
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
REPO="dograh-hq/dograh"
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
echo -e "${BLUE}"
echo "╔══════════════════════════════════════════════════════════════╗"
echo "║ Dograh Remote Update ║"
echo "║ Refresh host-side configs and pin api/ui image versions ║"
echo "╚══════════════════════════════════════════════════════════════╝"
echo -e "${NC}"
# Refuse outside an install — nothing to update if these aren't here.
if [[ ! -f docker-compose.yaml ]]; then
echo -e "${RED}Error: docker-compose.yaml not found in $(pwd)${NC}"
echo -e "${RED}Run this script from your Dograh install directory${NC}"
echo -e "${RED}(the 'dograh/' folder created by setup_remote.sh).${NC}"
exit 1
fi
if [[ ! -f .env ]]; then
echo -e "${RED}Error: .env not found in $(pwd)${NC}"
echo -e "${RED}This script updates an existing install — there is nothing here to update.${NC}"
echo -e "${RED}For a fresh install, see https://docs.dograh.com/deployment/docker${NC}"
exit 1
fi
# Build-mode installs update via git, not via this script. The presence of an
# override file is the definitive marker (created by setup_remote.sh in build
# mode and not in prebuilt mode).
if [[ -f docker-compose.override.yaml ]]; then
echo -e "${YELLOW}Build-mode install detected (docker-compose.override.yaml present).${NC}"
echo ""
echo -e "${YELLOW}This script is for prebuilt installs only. For build mode, update via git:${NC}"
echo ""
echo -e " ${BLUE}git fetch${NC}"
echo -e " ${BLUE}git checkout <tag> # or: git pull${NC}"
echo -e " ${BLUE}git submodule update --init --recursive${NC}"
echo -e " ${BLUE}sudo docker compose --profile remote build${NC}"
echo -e " ${BLUE}sudo docker compose --profile remote up -d${NC}"
echo ""
echo -e "${YELLOW}See https://docs.dograh.com/deployment/update#updating-a-source-build${NC}"
exit 1
fi
###############################################################################
### Discover existing config from .env
###############################################################################
# Save anything the caller exported before we overwrite from .env.
_caller_FASTAPI_WORKERS="$FASTAPI_WORKERS"
_caller_TARGET_VERSION="$TARGET_VERSION"
set -a
# shellcheck disable=SC1091
. ./.env
set +a
# SERVER_IP isn't a literal key in .env — derive it from BACKEND_API_ENDPOINT.
if [[ -z "$SERVER_IP" ]]; then
if [[ -n "$BACKEND_API_ENDPOINT" ]]; then
SERVER_IP="${BACKEND_API_ENDPOINT#https://}"
SERVER_IP="${SERVER_IP#http://}"
fi
fi
if [[ -z "$SERVER_IP" ]]; then
echo -e "${RED}Error: could not determine SERVER_IP from .env${NC}"
echo -e "${RED}Expected BACKEND_API_ENDPOINT=https://<ip> in .env${NC}"
exit 1
fi
if [[ -z "$TURN_SECRET" ]]; then
echo -e "${RED}Error: TURN_SECRET not found in .env${NC}"
exit 1
fi
# Reapply caller overrides on top of sourced .env so e.g. FASTAPI_WORKERS=8 ./update_remote.sh works.
[[ -n "$_caller_FASTAPI_WORKERS" ]] && FASTAPI_WORKERS="$_caller_FASTAPI_WORKERS"
[[ -n "$_caller_TARGET_VERSION" ]] && TARGET_VERSION="$_caller_TARGET_VERSION"
###############################################################################
### Determine target version
###############################################################################
if [[ -z "$TARGET_VERSION" ]]; then
echo -e "${BLUE}Fetching latest release tag from GitHub...${NC}"
LATEST_TAG=$(curl -fsSL "https://api.github.com/repos/$REPO/releases/latest" 2>/dev/null \
| grep -E '"tag_name":' | head -1 \
| sed -E 's/.*"tag_name":[[:space:]]*"([^"]+)".*/\1/' || true)
if [[ -z "$LATEST_TAG" ]]; then
echo -e "${YELLOW}Could not auto-discover latest tag — defaulting to 'main'.${NC}"
LATEST_TAG="main"
fi
if [[ -t 0 ]]; then
echo ""
echo -e "${YELLOW}Target version. Accepted forms: bare semver (1.28.0), v-prefixed (v1.28.0),${NC}"
echo -e "${YELLOW}full git tag (dograh-v1.28.0), or 'main' for bleeding edge.${NC}"
read -p "[$LATEST_TAG]: " TARGET_VERSION
TARGET_VERSION="${TARGET_VERSION:-$LATEST_TAG}"
else
TARGET_VERSION="$LATEST_TAG"
fi
fi
# "latest" isn't a real ref on GitHub — treat it as "latest release".
if [[ "$TARGET_VERSION" == "latest" ]]; then
TARGET_VERSION=$(curl -fsSL "https://api.github.com/repos/$REPO/releases/latest" 2>/dev/null \
| grep -E '"tag_name":' | head -1 \
| sed -E 's/.*"tag_name":[[:space:]]*"([^"]+)".*/\1/' || true)
if [[ -z "$TARGET_VERSION" ]]; then
echo -e "${RED}Error: could not resolve 'latest' to a release tag${NC}"
exit 1
fi
fi
# GitHub release tags use a 'dograh-v' prefix (e.g. dograh-v1.28.0); Docker
# image tags on Docker Hub drop both the prefix and the 'v' (e.g. ':1.28.0').
# Users commonly type shortcuts like '1.28.0' or 'v1.28.0' — try all reasonable
# variants so the script accepts any of those forms.
TRY_TAGS=("$TARGET_VERSION")
case "$TARGET_VERSION" in
main|HEAD)
;; # branch refs — leave as-is
dograh-*)
;; # already in the full tag form
v*)
TRY_TAGS+=("dograh-$TARGET_VERSION")
;;
*)
TRY_TAGS+=("dograh-v$TARGET_VERSION" "v$TARGET_VERSION" "dograh-$TARGET_VERSION")
;;
esac
echo -e "${BLUE}Validating target version: $TARGET_VERSION...${NC}"
RESOLVED_TAG=""
for tag in "${TRY_TAGS[@]}"; do
if curl -fsI "https://raw.githubusercontent.com/$REPO/$tag/docker-compose.yaml" >/dev/null 2>&1; then
RESOLVED_TAG="$tag"
break
fi
done
if [[ -z "$RESOLVED_TAG" ]]; then
echo -e "${RED}Error: could not find a git tag matching '$TARGET_VERSION'${NC}"
echo -e "${RED}Tried: ${TRY_TAGS[*]}${NC}"
echo -e "${RED}See available releases at: https://github.com/$REPO/releases${NC}"
exit 1
fi
if [[ "$RESOLVED_TAG" != "$TARGET_VERSION" ]]; then
echo -e "${GREEN}✓ Resolved '$TARGET_VERSION' to git tag '$RESOLVED_TAG'${NC}"
fi
TARGET_VERSION="$RESOLVED_TAG"
RAW_BASE="https://raw.githubusercontent.com/$REPO/$TARGET_VERSION"
# Derive the Docker image tag from the git tag. Tags on Docker Hub use bare
# semver — strip the 'dograh-' prefix and the leading 'v'.
IMAGE_TAG=""
case "$TARGET_VERSION" in
dograh-v*) IMAGE_TAG="${TARGET_VERSION#dograh-v}" ;;
v*) IMAGE_TAG="${TARGET_VERSION#v}" ;;
main|HEAD) IMAGE_TAG="" ;;
*) [[ "$TARGET_VERSION" =~ ^[0-9] ]] && IMAGE_TAG="$TARGET_VERSION" ;;
esac
# Verify the image tag actually exists on Docker Hub. If not (e.g. CI hasn't
# published yet), fall back to ':latest' rather than pinning to a missing tag.
if [[ -n "$IMAGE_TAG" ]]; then
if curl -fsI "https://hub.docker.com/v2/repositories/dograhai/dograh-api/tags/$IMAGE_TAG/" >/dev/null 2>&1; then
echo -e "${GREEN}✓ Image tag :$IMAGE_TAG found on Docker Hub${NC}"
else
echo -e "${YELLOW}Warning: image tag :$IMAGE_TAG not found on Docker Hub — leaving images at :latest${NC}"
IMAGE_TAG=""
fi
fi
###############################################################################
### Reconcile required keys that may be missing on older installs
###############################################################################
if [[ -z "$FASTAPI_WORKERS" ]]; then
if [[ -t 0 ]]; then
echo ""
echo -e "${YELLOW}FASTAPI_WORKERS not set in .env. Number of uvicorn workers nginx will load-balance:${NC}"
read -p "[4]: " FASTAPI_WORKERS
FASTAPI_WORKERS="${FASTAPI_WORKERS:-4}"
else
FASTAPI_WORKERS="4"
fi
fi
if ! [[ "$FASTAPI_WORKERS" =~ ^[1-9][0-9]*$ ]]; then
echo -e "${RED}Error: FASTAPI_WORKERS must be a positive integer (got: $FASTAPI_WORKERS)${NC}"
exit 1
fi
###############################################################################
### Summary + confirmation
###############################################################################
echo ""
echo -e "${GREEN}Update plan:${NC}"
echo -e " Server IP: ${BLUE}$SERVER_IP${NC}"
echo -e " Target version: ${BLUE}$TARGET_VERSION${NC}"
echo -e " FastAPI workers: ${BLUE}$FASTAPI_WORKERS${NC} (ports 8000..$((8000 + FASTAPI_WORKERS - 1)))"
echo ""
echo -e "${YELLOW}Files that will be replaced (backups saved with suffix .bak.$TIMESTAMP):${NC}"
echo " - docker-compose.yaml (pulled from GitHub at $TARGET_VERSION)"
echo " - nginx.conf (regenerated from this script's template)"
echo " - turnserver.conf (regenerated from this script's template)"
echo " - .env (existing values preserved; missing keys appended)"
echo ""
echo -e "${YELLOW}Any local customizations to these files will be overwritten — check the backup${NC}"
echo -e "${YELLOW}files if you need to re-apply edits afterwards.${NC}"
echo ""
if [[ -t 0 && "$DOGRAH_UPDATE_YES" != "1" ]]; then
read -p "Proceed? [y/N]: " confirm
if ! [[ "$confirm" =~ ^[Yy] ]]; then
echo -e "${RED}Aborted.${NC}"
exit 1
fi
fi
###############################################################################
### Step 1 — backups
###############################################################################
echo ""
echo -e "${BLUE}[1/5] Backing up existing files...${NC}"
for f in docker-compose.yaml nginx.conf turnserver.conf .env; do
if [[ -f "$f" ]]; then
cp -p "$f" "$f.bak.$TIMESTAMP"
echo -e " ${GREEN}✓ $f → $f.bak.$TIMESTAMP${NC}"
fi
done
###############################################################################
### Step 2 — docker-compose.yaml (download + pin image tags)
###############################################################################
echo -e "${BLUE}[2/5] Downloading docker-compose.yaml at $TARGET_VERSION...${NC}"
curl -fsSL -o docker-compose.yaml "$RAW_BASE/docker-compose.yaml"
# Pin api/ui image tags when we resolved one. For branch refs (main) IMAGE_TAG
# is empty, so the images stay at ':latest' and `up --pull always` grabs the
# newest build of that branch.
if [[ -n "$IMAGE_TAG" ]]; then
sed -i.tmp -E "s#(dograh-(api|ui)):latest#\1:$IMAGE_TAG#g" docker-compose.yaml
rm -f docker-compose.yaml.tmp
echo -e "${GREEN}✓ docker-compose.yaml updated; images pinned to :$IMAGE_TAG${NC}"
else
echo -e "${GREEN}✓ docker-compose.yaml updated (image tags left at :latest)${NC}"
fi
###############################################################################
### Step 3 — nginx.conf (regenerate from embedded template)
###############################################################################
echo -e "${BLUE}[3/5] Regenerating nginx.conf...${NC}"
{
echo "# Backend API workers — one uvicorn process per port, balanced by least_conn."
echo "# Generated by update_remote.sh; regenerate to change worker count."
echo "upstream dograh_api {"
echo " least_conn;"
for ((i=0; i<FASTAPI_WORKERS; i++)); do
port=$((8000 + i))
echo " server api:$port max_fails=3 fail_timeout=10s;"
done
echo " keepalive 32;"
echo "}"
echo ""
cat << 'NGINX_EOF'
server {
listen 80;
server_name SERVER_IP_PLACEHOLDER;
# Redirect all HTTP to HTTPS
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
server_name SERVER_IP_PLACEHOLDER;
ssl_certificate /etc/nginx/certs/local.crt;
ssl_certificate_key /etc/nginx/certs/local.key;
# Basic TLS settings
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
# Backend API and WebSockets - bypass the UI, go straight to the
# api workers via the least_conn upstream defined above.
location /api/v1/ {
proxy_pass http://dograh_api;
proxy_http_version 1.1;
# Retry on a dead/restarting worker
proxy_next_upstream error timeout http_502 http_503 http_504;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
# Long-lived WebSockets (audio streaming, signaling)
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
# Don't buffer streamed responses
proxy_buffering off;
client_max_body_size 100M;
}
location / {
proxy_pass http://ui:3010;
proxy_http_version 1.1;
# Important for WebSockets / hot reload etc.
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
# Rewrite localhost MinIO URLs in API responses to use current domain
sub_filter 'http://localhost:9000/voice-audio/' 'https://$host/voice-audio/';
sub_filter_once off;
sub_filter_types application/json text/html;
}
location /voice-audio/ {
proxy_pass http://minio:9000/voice-audio/;
proxy_http_version 1.1;
# Headers for file downloads from MinIO
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
# Allow large file downloads
proxy_buffering off;
client_max_body_size 100M;
}
}
NGINX_EOF
} > nginx.conf
sed -i.tmp "s/SERVER_IP_PLACEHOLDER/$SERVER_IP/g" nginx.conf && rm -f nginx.conf.tmp
echo -e "${GREEN}✓ nginx.conf regenerated${NC}"
###############################################################################
### Step 4 — turnserver.conf (regenerate from embedded template)
###############################################################################
echo -e "${BLUE}[4/5] Regenerating turnserver.conf...${NC}"
cat > turnserver.conf << TURN_EOF
# Coturn TURN Server - Docker Configuration
# Auto-generated by update_remote.sh
# Listener ports
listening-port=3478
tls-listening-port=5349
# Relay port range
min-port=49152
max-port=49200
# Network - external IP for NAT traversal
external-ip=$SERVER_IP
# Realm
realm=dograh.com
# Authentication (TURN REST API with time-limited credentials)
use-auth-secret
static-auth-secret=$TURN_SECRET
# Security
fingerprint
no-cli
no-multicast-peers
# Logging
log-file=stdout
TURN_EOF
echo -e "${GREEN}✓ turnserver.conf regenerated${NC}"
###############################################################################
### Step 5 — reconcile .env (append missing keys; never overwrite existing)
###############################################################################
echo -e "${BLUE}[5/5] Reconciling .env...${NC}"
if ! grep -q "^FASTAPI_WORKERS=" .env; then
{
echo ""
echo "# Number of uvicorn worker processes; nginx load-balances across them"
echo "# (ports 8000..$((8000 + FASTAPI_WORKERS - 1))) with least_conn."
echo "FASTAPI_WORKERS=$FASTAPI_WORKERS"
} >> .env
echo -e "${GREEN}✓ Added FASTAPI_WORKERS=$FASTAPI_WORKERS to .env${NC}"
else
echo -e "${GREEN}✓ .env already has FASTAPI_WORKERS — left unchanged${NC}"
fi
###############################################################################
### Done — print restart + rollback instructions
###############################################################################
echo ""
echo -e "${GREEN}╔══════════════════════════════════════════════════════════════╗${NC}"
echo -e "${GREEN}║ Update Prepared! ║${NC}"
echo -e "${GREEN}╚══════════════════════════════════════════════════════════════╝${NC}"
echo ""
echo -e "Backups: ${BLUE}*.bak.$TIMESTAMP${NC}"
echo ""
echo -e "${YELLOW}To apply, recreate the stack:${NC}"
echo ""
echo -e " ${BLUE}sudo docker compose --profile remote down${NC}"
echo -e " ${BLUE}sudo docker compose --profile remote up -d --pull always${NC}"
echo ""
echo -e "${YELLOW}To roll back, restore the backups and recreate:${NC}"
echo ""
echo -e " ${BLUE}for f in docker-compose.yaml nginx.conf turnserver.conf .env; do${NC}"
echo -e " ${BLUE} [[ -f \"\$f.bak.$TIMESTAMP\" ]] && cp \"\$f.bak.$TIMESTAMP\" \"\$f\"${NC}"
echo -e " ${BLUE}done${NC}"
echo -e " ${BLUE}sudo docker compose --profile remote down && sudo docker compose --profile remote up -d${NC}"
echo ""