From ae2efefa5962144acae8f8ba3db0036e2171fdbc Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Thu, 14 May 2026 15:01:11 +0530 Subject: [PATCH] chore: add powershell version for setup_local --- docs/deployment/docker.mdx | 8 +- docs/deployment/update.mdx | 2 +- scripts/AGENTS.md | 56 +++++++ scripts/CLAUDE.md | 31 +--- scripts/setup_local.ps1 | 296 +++++++++++++++++++++++++++++++++++++ 5 files changed, 361 insertions(+), 32 deletions(-) create mode 100644 scripts/AGENTS.md create mode 100644 scripts/setup_local.ps1 diff --git a/docs/deployment/docker.mdx b/docs/deployment/docker.mdx index f235d5c..d284803 100644 --- a/docs/deployment/docker.mdx +++ b/docs/deployment/docker.mdx @@ -57,9 +57,15 @@ The Quick Start above relies on direct peer-to-peer WebRTC between your browser For these cases, use the alternate local setup script which configures a coturn TURN server alongside the rest of the stack: -```bash + +```bash macOS/Linux curl -o setup_local.sh https://raw.githubusercontent.com/dograh-hq/dograh/main/scripts/setup_local.sh && chmod +x setup_local.sh && ./setup_local.sh ``` +```powershell Windows +Invoke-WebRequest -OutFile setup_local.ps1 https://raw.githubusercontent.com/dograh-hq/dograh/main/scripts/setup_local.ps1 +.\setup_local.ps1 +``` + The script will prompt you for: - Whether to enable coturn (answer `y`) diff --git a/docs/deployment/update.mdx b/docs/deployment/update.mdx index bbea9a0..cba9f2a 100644 --- a/docs/deployment/update.mdx +++ b/docs/deployment/update.mdx @@ -68,7 +68,7 @@ The script overwrites `docker-compose.yaml`, `nginx.conf`, and `turnserver.conf` ## Local deployment -For local Docker installs (the [Quick Start](/deployment/docker#quick-start) flow or `setup_local.sh`), there are no host-side config files to refresh — pull new images and restart: +For local Docker installs (the [Quick Start](/deployment/docker#quick-start) flow or `setup_local.sh` / `setup_local.ps1`), there are no host-side config files to refresh — pull new images and restart: ```bash docker compose down diff --git a/scripts/AGENTS.md b/scripts/AGENTS.md new file mode 100644 index 0000000..0ffa116 --- /dev/null +++ b/scripts/AGENTS.md @@ -0,0 +1,56 @@ +# scripts/ + +## Bash ↔ PowerShell parity — keep them in sync + +Most contributor-facing scripts ship as a `.sh` + `.ps1` pair so macOS/Linux and Windows users get the same workflow. **When you edit one, edit the other in the same change.** Env-var names, defaults, flags, and behavior should match — if `start_services_dev.sh` reads `HEALTH_MAX_ATTEMPTS`, so should `start_services_dev.ps1`. + +Current pairs: + +- `setup_fork.{sh,ps1}` — contributor bootstrap (git remotes, submodule, venv, env files) +- `setup_requirements.{sh,ps1}` — Python + pipecat dependency install +- `start_services_dev.{sh,ps1}` — local backend launcher (auto-reload + health-check wait) +- `stop_services.{sh,ps1}` +- `makemigrate.{sh,ps1}` / `migrate.{sh,ps1}` — Alembic helpers +- `setup_local.{sh,ps1}` — OSS local Docker-compose setup (optional coturn/TURN) + +Bash-only (deployment / CI / OSS-user setup — not intended for Windows contributors): + +- `start_services.sh` — VM production +- `start_services_docker.sh` — Docker image CMD +- `rolling_update.sh` — zero-downtime VM redeploy +- `setup_remote.sh` — OSS remote Docker-compose setup +- `format.sh` / `lint.sh` / `pre_commit.sh` +- `generate_sdk.sh` / `release_sdks.sh` / `dump_docs_openapi.py` + +## Deployment Memory — current OSS Docker state + +This directory now has a shared deployment model for OSS Docker installs. If you touch any of the scripts below, assume they are coupled and review them together: + +- `scripts/lib/setup_common.sh` is the shared deployment helper library. It is sourced by `setup_local.sh`, `setup_remote.sh`, `update_remote.sh`, `setup_custom_domain.sh`, `run_dograh_init.sh`, and repo-root `remote_up.sh`. +- `setup_common.sh` must stay safe to source. It should not set shell options like `set -u` for callers. +- `.env` is the single operator-owned source of truth for remote deployment settings. Remote/runtime config should derive from it, not the other way around. +- Canonical remote keys in `.env`: `ENVIRONMENT`, `SERVER_IP`, `PUBLIC_HOST`, `PUBLIC_BASE_URL`, `BACKEND_API_ENDPOINT`, `MINIO_PUBLIC_ENDPOINT`, `TURN_HOST`, `TURN_SECRET`, `FASTAPI_WORKERS`, `OSS_JWT_SECRET`. +- `remote_up.sh` is the supported remote startup entrypoint. It runs preflight via `dograh_prepare_remote_install`, runs `docker compose config -q`, then starts the stack. +- `docker-compose.yaml` uses a one-shot `dograh-init` service for profiles `remote` and `local-turn`. +- `dograh-init` executes `scripts/run_dograh_init.sh`, which renders nginx/coturn runtime config into named volumes consumed by `nginx` and `coturn`. +- Remote nginx/coturn config is runtime-generated. Host-managed `nginx.conf` / `turnserver.conf` are legacy only; update flow may back them up and delete them, but current installs should not depend on them. +- `setup_remote.sh` writes `.env`, downloads the deployment helper bundle, generates self-signed certs, validates the init-based config, and tells operators to start via `./remote_up.sh` or `./remote_up.sh --build`. +- `update_remote.sh` is the migration/upgrade path for prebuilt remote installs. It refreshes `docker-compose.yaml`, `remote_up.sh`, `scripts/run_dograh_init.sh`, `scripts/lib/setup_common.sh`, and `deploy/templates/*`, backs up touched files, removes legacy host `nginx.conf` / `turnserver.conf`, and revalidates the init-based path. +- `setup_custom_domain.sh` is certificate/domain glue only. It must not own nginx config. It updates canonical public URL keys in `.env`, copies Let's Encrypt certs into `certs/`, installs renewal hook, and restarts through `./remote_up.sh`. +- `setup_local.{sh,ps1}` has an interactive `Enable coturn? [y/N]` prompt unless `ENABLE_COTURN` is preset. If coturn is enabled, it downloads the minimal helper bundle needed for `local-turn` (`setup_common.sh`, `run_dograh_init.sh`, templates) and relies on `dograh-init` to render coturn config. +- `setup_local.{sh,ps1}` must remain safe under unset env vars; use `${VAR:-}` guards in Bash and null/empty checks in PowerShell for optional inputs like `ENABLE_COTURN`, `TURN_HOST`, `TURN_SECRET`, `DOGRAH_SKIP_DOWNLOAD`. +- `run_dograh_init.sh` is an executable entrypoint, not a library. Compose runs it directly. If it ever gets refactored, keep the distinction between sourced helper logic (`lib/`) and executable entrypoints. +- `dograh_prepare_remote_install` in `setup_common.sh` currently does three things: sync canonical `.env` keys, reject legacy compose layouts that do not use `dograh-init`, and preflight the init render in a temp directory. +- `dograh_uses_init_compose_layout` / `dograh_require_init_compose_layout` are the guardrails for old installs. If a remote install still bind-mounts host `nginx.conf` / `turnserver.conf`, the intended fix path is `./update_remote.sh`. +- Templates live under `deploy/templates/`. `nginx.remote.conf.template` contains the static shape and `dograh_render_remote_nginx_conf` expands the multi-worker upstream block dynamically. `turnserver.remote.conf.template` is also rendered from env. +- If you rename/move any of these deployment files, update all of: bootstrap curl URLs inside scripts, helper-bundle download paths in `setup_common.sh`, backup lists in `update_remote.sh`, docs under `docs/deployment/`, and any existence checks in `setup_local.{sh,ps1}` / `setup_custom_domain.sh`. + +## The three "start" scripts — pick the right one + +| Script | Where it runs | Key behavior | +| -------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------- | +| `start_services_dev.sh` | Local dev shell | `uvicorn --reload`, exits after launching, restart by re-running, single arq worker, waits for `/api/v1/health` before exiting. | +| `start_services.sh` | VM production | Multi-port uvicorn behind nginx, `sudo nginx -t && systemctl reload`, writes `run/active_band` for `rolling_update.sh`. | +| `start_services_docker.sh` | Docker image `CMD` | PID 1: traps SIGTERM, uvicorn `--workers $FASTAPI_WORKERS`, `wait -n` so a dying child tears the container down. | + +If you find yourself adding nginx/sudo logic to the dev script, or `--reload` to the production/Docker scripts, stop — you probably want a different file. diff --git a/scripts/CLAUDE.md b/scripts/CLAUDE.md index 1c3e942..43c994c 100644 --- a/scripts/CLAUDE.md +++ b/scripts/CLAUDE.md @@ -1,30 +1 @@ -# scripts/ - -## Bash ↔ PowerShell parity — keep them in sync - -Most contributor-facing scripts ship as a `.sh` + `.ps1` pair so macOS/Linux and Windows users get the same workflow. **When you edit one, edit the other in the same change.** Env-var names, defaults, flags, and behavior should match — if `start_services_dev.sh` reads `HEALTH_MAX_ATTEMPTS`, so should `start_services_dev.ps1`. - -Current pairs: -- `setup_fork.{sh,ps1}` — contributor bootstrap (git remotes, submodule, venv, env files) -- `setup_requirements.{sh,ps1}` — Python + pipecat dependency install -- `start_services_dev.{sh,ps1}` — local backend launcher (auto-reload + health-check wait) -- `stop_services.{sh,ps1}` -- `makemigrate.{sh,ps1}` / `migrate.{sh,ps1}` — Alembic helpers - -Bash-only (deployment / CI / OSS-user setup — not intended for Windows contributors): -- `start_services.sh` — VM production -- `start_services_docker.sh` — Docker image CMD -- `rolling_update.sh` — zero-downtime VM redeploy -- `setup_local.sh` / `setup_remote.sh` — OSS Docker-compose setup -- `format.sh` / `lint.sh` / `pre_commit.sh` -- `generate_sdk.sh` / `release_sdks.sh` / `dump_docs_openapi.py` - -## The three "start" scripts — pick the right one - -| Script | Where it runs | Key behavior | -| ----------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------- | -| `start_services_dev.sh` | Local dev shell | `uvicorn --reload`, exits after launching, restart by re-running, single arq worker, waits for `/api/v1/health` before exiting. | -| `start_services.sh` | VM production | Multi-port uvicorn behind nginx, `sudo nginx -t && systemctl reload`, writes `run/active_band` for `rolling_update.sh`. | -| `start_services_docker.sh` | Docker image `CMD` | PID 1: traps SIGTERM, uvicorn `--workers $FASTAPI_WORKERS`, `wait -n` so a dying child tears the container down. | - -If you find yourself adding nginx/sudo logic to the dev script, or `--reload` to the production/Docker scripts, stop — you probably want a different file. +@AGENTS.md diff --git a/scripts/setup_local.ps1 b/scripts/setup_local.ps1 new file mode 100644 index 0000000..95b4c42 --- /dev/null +++ b/scripts/setup_local.ps1 @@ -0,0 +1,296 @@ +#!/usr/bin/env pwsh + +$ErrorActionPreference = 'Stop' + +function Write-Info([string]$Message) { + Write-Host $Message -ForegroundColor Blue +} + +function Write-Success([string]$Message) { + Write-Host $Message -ForegroundColor Green +} + +function Write-Warn([string]$Message) { + Write-Host $Message -ForegroundColor Yellow +} + +function Fail([string]$Message) { + Write-Host "Error: $Message" -ForegroundColor Red + exit 1 +} + +function Test-IsEnabled([string]$Value) { + return $Value -eq 'true' +} + +function New-HexSecret([int]$ByteCount) { + $buffer = [byte[]]::new($ByteCount) + $rng = [System.Security.Cryptography.RandomNumberGenerator]::Create() + try { + $rng.GetBytes($buffer) + } finally { + $rng.Dispose() + } + return ($buffer | ForEach-Object { $_.ToString('x2') }) -join '' +} + +function Read-SecretValue([string]$Prompt) { + $readHostCommand = Get-Command Read-Host + if ($readHostCommand.Parameters.ContainsKey('MaskInput')) { + return Read-Host $Prompt -MaskInput + } + + $secureValue = Read-Host $Prompt -AsSecureString + $bstr = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($secureValue) + try { + return [System.Runtime.InteropServices.Marshal]::PtrToStringBSTR($bstr) + } finally { + [System.Runtime.InteropServices.Marshal]::ZeroFreeBSTR($bstr) + } +} + +function Get-DefaultLanIPv4 { + try { + $routes = Get-NetRoute -AddressFamily IPv4 -DestinationPrefix '0.0.0.0/0' -ErrorAction Stop | + Sort-Object -Property RouteMetric, InterfaceMetric + + foreach ($route in $routes) { + $candidate = Get-NetIPAddress -AddressFamily IPv4 -InterfaceIndex $route.InterfaceIndex -ErrorAction Stop | + Where-Object { + $_.IPAddress -ne '127.0.0.1' -and + -not $_.IPAddress.StartsWith('169.254.') + } | + Select-Object -First 1 -ExpandProperty IPAddress + + if ($candidate) { + return $candidate + } + } + } catch { + # Fall back to generic interface enumeration below. + } + + try { + $interfaces = [System.Net.NetworkInformation.NetworkInterface]::GetAllNetworkInterfaces() | + Where-Object { + $_.OperationalStatus -eq [System.Net.NetworkInformation.OperationalStatus]::Up -and + $_.NetworkInterfaceType -ne [System.Net.NetworkInformation.NetworkInterfaceType]::Loopback + } + + foreach ($iface in $interfaces) { + foreach ($unicast in $iface.GetIPProperties().UnicastAddresses) { + if ($unicast.Address.AddressFamily -ne [System.Net.Sockets.AddressFamily]::InterNetwork) { + continue + } + + $candidate = $unicast.Address.IPAddressToString + if ($candidate -and $candidate -ne '127.0.0.1' -and -not $candidate.StartsWith('169.254.')) { + return $candidate + } + } + } + } catch { + return $null + } + + return $null +} + +function Download-File([string]$Url, [string]$Destination) { + $parent = Split-Path -Parent $Destination + if ($parent) { + New-Item -ItemType Directory -Path $parent -Force | Out-Null + } + + $params = @{ + Uri = $Url + OutFile = $Destination + ErrorAction = 'Stop' + } + + $invokeWebRequest = Get-Command Invoke-WebRequest + if ($invokeWebRequest.Parameters.ContainsKey('UseBasicParsing')) { + $params.UseBasicParsing = $true + } + + Invoke-WebRequest @params +} + +function Download-BundleFileForRef([string]$Destination, [string]$RemotePath, [string]$Ref) { + $rawBase = "https://raw.githubusercontent.com/dograh-hq/dograh/$Ref" + $fallbackBase = 'https://raw.githubusercontent.com/dograh-hq/dograh/main' + + try { + Download-File "$rawBase/$RemotePath" $Destination + } catch { + if ($Ref -eq 'main') { + throw + } + + Write-Warn "Warning: '$RemotePath' not found at '$Ref' - falling back to main" + Download-File "$fallbackBase/$RemotePath" $Destination + } +} + +function Download-InitSupportBundle([string]$ProjectDir, [string]$Ref) { + Download-BundleFileForRef (Join-Path $ProjectDir 'scripts/lib/setup_common.sh') 'scripts/lib/setup_common.sh' $Ref + Download-BundleFileForRef (Join-Path $ProjectDir 'scripts/run_dograh_init.sh') 'scripts/run_dograh_init.sh' $Ref + Download-BundleFileForRef (Join-Path $ProjectDir 'deploy/templates/nginx.remote.conf.template') 'deploy/templates/nginx.remote.conf.template' $Ref + Download-BundleFileForRef (Join-Path $ProjectDir 'deploy/templates/turnserver.remote.conf.template') 'deploy/templates/turnserver.remote.conf.template' $Ref +} + +function Assert-PathExists([string]$Path, [string]$Message) { + if (-not (Test-Path $Path)) { + Fail $Message + } +} + +Write-Info '' +Write-Info '╔══════════════════════════════════════════════════════════════╗' +Write-Info '║ Dograh Local Setup ║' +Write-Info '║ Local docker deployment, optional TURN server ║' +Write-Info '╚══════════════════════════════════════════════════════════════╝' +Write-Info '' + +if ([string]::IsNullOrEmpty($env:ENABLE_COTURN)) { + Write-Warn 'Enable coturn (TURN server) for WebRTC NAT traversal? [y/N]:' + $enableCoturnInput = Read-Host '>' + if ($enableCoturnInput -match '^[Yy]') { + $EnableCoturn = 'true' + } else { + $EnableCoturn = 'false' + } +} else { + $EnableCoturn = $env:ENABLE_COTURN +} + +$UseCoturn = Test-IsEnabled $EnableCoturn +$TurnHost = $env:TURN_HOST +$TurnSecret = $env:TURN_SECRET + +if ($UseCoturn) { + $defaultTurnHost = Get-DefaultLanIPv4 + if ([string]::IsNullOrEmpty($defaultTurnHost)) { + $defaultTurnHost = '127.0.0.1' + } + + if ([string]::IsNullOrEmpty($TurnHost)) { + Write-Warn 'Enter the host browsers AND the API container will use to reach TURN' + Write-Warn "(press Enter for $defaultTurnHost):" + $TurnHost = Read-Host '>' + } + if ([string]::IsNullOrEmpty($TurnHost)) { + $TurnHost = $defaultTurnHost + } + + if ($TurnHost -notmatch '^[A-Za-z0-9.-]+$') { + Fail 'TURN host must be an IP address or hostname' + } + + if ([string]::IsNullOrEmpty($TurnSecret)) { + Write-Warn 'Enter a shared secret for the TURN server (press Enter to generate a random one):' + $TurnSecret = Read-SecretValue '>' + Write-Host '' + } + + if ([string]::IsNullOrEmpty($TurnSecret)) { + $TurnSecret = New-HexSecret 32 + Write-Info 'Generated random TURN secret' + } +} + +$EnableTelemetry = if ([string]::IsNullOrEmpty($env:ENABLE_TELEMETRY)) { 'true' } else { $env:ENABLE_TELEMETRY } +$Registry = if ([string]::IsNullOrEmpty($env:REGISTRY)) { 'ghcr.io/dograh-hq' } else { $env:REGISTRY } + +Write-Host '' +Write-Success 'Configuration:' +Write-Host " Coturn: $EnableCoturn" -ForegroundColor Blue +if ($UseCoturn) { + Write-Host " TURN Host: $TurnHost" -ForegroundColor Blue + Write-Host ' TURN Secret: ********' -ForegroundColor Blue +} +Write-Host " Telemetry: $EnableTelemetry" -ForegroundColor Blue +Write-Host " Registry: $Registry" -ForegroundColor Blue +Write-Host '' + +$TotalSteps = 2 +$CurrentDir = (Get-Location).Path + +if ($env:DOGRAH_SKIP_DOWNLOAD -ne '1') { + if ($UseCoturn) { + Write-Info "[1/$TotalSteps] Downloading docker-compose.yaml and TURN helper bundle..." + } else { + Write-Info "[1/$TotalSteps] Downloading docker-compose.yaml..." + } + + Download-File 'https://raw.githubusercontent.com/dograh-hq/dograh/main/docker-compose.yaml' (Join-Path $CurrentDir 'docker-compose.yaml') + if ($UseCoturn) { + Download-InitSupportBundle $CurrentDir 'main' + } + + Write-Success '✓ Deployment files downloaded' +} else { + Write-Info "[1/$TotalSteps] Using docker-compose.yaml in current directory" +} + +if ($UseCoturn) { + Assert-PathExists 'scripts/run_dograh_init.sh' 'scripts/run_dograh_init.sh not found. Re-run setup_local.ps1 without DOGRAH_SKIP_DOWNLOAD=1, or use a full repo checkout.' + Assert-PathExists 'scripts/lib/setup_common.sh' 'scripts/lib/setup_common.sh not found. Re-run setup_local.ps1 without DOGRAH_SKIP_DOWNLOAD=1, or use a full repo checkout.' + Assert-PathExists 'deploy/templates/turnserver.remote.conf.template' 'deploy/templates/turnserver.remote.conf.template not found. Re-run setup_local.ps1 without DOGRAH_SKIP_DOWNLOAD=1, or use a full repo checkout.' +} + +Write-Info "[2/$TotalSteps] Creating environment file..." +$ossJwtSecret = New-HexSecret 32 + +$envLines = @( + '# Container registry for Dograh images' + "REGISTRY=$Registry" + '' + '# JWT secret for OSS authentication' + "OSS_JWT_SECRET=$ossJwtSecret" + '' + '# Telemetry (set to false to disable)' + "ENABLE_TELEMETRY=$EnableTelemetry" +) + +if ($UseCoturn) { + $envLines += @( + '' + '# TURN Server Configuration (time-limited credentials via TURN REST API)' + "TURN_HOST=$TurnHost" + "TURN_SECRET=$TurnSecret" + ) +} + +$envContent = ($envLines -join [Environment]::NewLine) + [Environment]::NewLine +[System.IO.File]::WriteAllText((Join-Path $CurrentDir '.env'), $envContent, [System.Text.UTF8Encoding]::new($false)) +Write-Success '✓ .env file created' + +Write-Host '' +Write-Success '╔══════════════════════════════════════════════════════════════╗' +Write-Success '║ Setup Complete! ║' +Write-Success '╚══════════════════════════════════════════════════════════════╝' +Write-Host '' +Write-Host "Files created in $CurrentDir:" -ForegroundColor Blue +Write-Host ' - docker-compose.yaml' +Write-Host ' - .env' +if ($UseCoturn) { + Write-Host ' - scripts/run_dograh_init.sh' + Write-Host ' - scripts/lib/setup_common.sh' + Write-Host ' - deploy/templates/' +} +Write-Host '' +if ($UseCoturn) { + Write-Warn 'To start Dograh with TURN, run:' + Write-Host '' + Write-Host ' docker compose --profile local-turn up --pull always' -ForegroundColor Blue +} else { + Write-Warn 'To start Dograh, run:' + Write-Host '' + Write-Host ' docker compose up --pull always' -ForegroundColor Blue +} +Write-Host '' +Write-Warn 'Your application will be available at:' +Write-Host '' +Write-Host ' http://localhost:3010' -ForegroundColor Blue +Write-Host ''