* feat: add ElevenLabs realtime STT provider support (#512)
Wire ElevenLabs scribe_v2_realtime into the STT registry and pipeline factory so BYOK transcribers can use the same provider already supported for TTS.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: address ElevenLabs STT review feedback for language, commits, and host
Pass custom language codes through instead of defaulting to English, use ElevenLabs VAD commit strategy because Dograh VAD runs downstream of STT, and document hostname-only realtime base_url handling.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: preserve ElevenLabs STT endpoint port in realtime host parsing
Use urlparse netloc instead of hostname so validated BYOK/proxy base URLs keep non-default ports when Pipecat builds the websocket endpoint.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: preserve ElevenLabs STT proxy path prefix and remove duplicate tests
Include URL path segments in realtime host normalization for BYOK proxies and delete shadowed pytest definitions.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: allow custom ElevenLabs model input
* fix: normalize ElevenLabs websocket URLs
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* feat(auth): gate OSS signup behind ENABLE_SIGNUP flag
## Problem
The `POST /api/v1/auth/signup` endpoint is unconditionally exposed on
every OSS install. Operators running an invite-only deployment (private
customer instances, staging environments, internal-only tenants) have
no way to disable public account creation without patching the codebase.
The UI also shows the "Sign up" link on `/auth/login` regardless of
whether signup is available, so a locked-down deployment leaves broken
navigation on the login page.
## Fix
Introduce a single `ENABLE_SIGNUP` env var (default `true` — no behavior
change for existing installs) that controls signup end-to-end:
- **Backend** — `api/constants.ENABLE_SIGNUP` is read at module load.
The signup handler returns 403 when it's false. Also exposed on
`GET /api/v1/health` as `signup_enabled: bool` so the UI can mirror
the operator's choice at runtime instead of at bundle-build time.
- **UI** — `getSignupEnabled()` in `lib/auth/config.ts` proxies the
health field, `/api/config/auth` surfaces it to the browser, the
login page conditionally renders the "Sign up" link via a one-shot
`fetch("/api/config/auth")` in `useEffect`, and the middleware
redirects `/auth/signup` → `/auth/login` when disabled (fires before
Next.js can serve the statically-prerendered signup page).
- **Helm** — `config.enableSignup` (default `true`) is rendered into
the ConfigMap as `ENABLE_SIGNUP` so operators can flip it via
`--set config.enableSignup=false` at install/upgrade time.
Fallbacks default to `signupEnabled: true` in every layer so a fresh
install "just works" and matches the backend default.
* address review: rollout on ConfigMap change, cache TTL, no signup-link flash
Four review points on #514:
**P1 — ConfigMap Change Skips Rollout** (`configmap.yaml`). `helm upgrade
--set config.enableSignup=false` updated the ConfigMap but did NOT roll
the api pods, so running processes kept the ENABLE_SIGNUP env from
startup and continued serving the old signup behavior — including
divergence between replicas mid-upgrade.
Fix: add the standard `checksum/config` pod-template annotation on the
four backend Deployments that `envFrom` the ConfigMap (`web`,
`arq-worker`, `ari-manager`, `campaign-orchestrator`). Verified with
`helm template`: all four Deployments share the same checksum on any
given render, and flipping `config.enableSignup` changes the checksum
uniformly so kubectl sees a pod-template diff and rolls all four.
**P1 — Signup Flag Stays Cached (server)** (`ui/src/lib/auth/config.ts`).
Module-scoped cache had no TTL. `revalidate: 300` was passed on the
underlying `fetch()` but the in-memory short-circuit above ran first, so
the value never refreshed until the UI pod restarted.
Fix: add `AUTH_CONFIG_TTL_MS = 5 * 60 * 1000` (matching the fetch
revalidate hint) so the module cache and the Next fetch cache stay in
sync. Backend flag flips propagate within 5 minutes without a pod
restart.
**P1 — Middleware Redirect Uses Stale State** (`ui/src/middleware.ts`).
Same shape as above — a separate module cache with no expiry could keep
redirecting `/auth/signup → /auth/login` after signup was re-enabled, or
keep serving the statically-prerendered signup page after lockdown.
Fix: same `SERVER_CONFIG_TTL_MS = 5 * 60 * 1000` TTL on the middleware
cache.
**P2 — Signup link flash on login page** (`ui/src/app/auth/login/page.tsx`).
Initial `signupEnabled` state was `null`, so `{signupEnabled && ...}`
hid the link on first paint and it popped in after the fetch resolved
— a CLS on every login-page load on stock installs where signup is
enabled.
Fix: initialise the state to `true` (matches the backend default). The
fetch still overrides to `false` when the operator has actually
disabled signup, so the lockdown UI behavior is unchanged; only the
happy-path flash is gone.
* simplify signup flag: drop TTL caches and middleware redirect
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* resolve signup flag server-side to avoid signup link flicker
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: prabhat pankaj <prabhatiitbhu@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Expose MiniMax-M3 in the MiniMax model suggestions now that the provider integration supports MiniMax chat models.
Co-authored-by: octo-patch <266937838+octo-patch@users.noreply.github.com>
* fix(auth): allow invited org members to start workflow runs
Users invited to an org could not start workflows belonging to that org
because the authorization check compared actor.selected_organization_id
directly against workflow.organization_id. An invited user's selected
org correctly reflects the invited org, but if the Stack Auth token
resolves to a different org id than expected the strict equality fails.
Per api/AGENTS.md: "Whenever you read or write an organization-scoped
field, you must filter or validate by organization_id." The correct
policy is org membership, not selected-org identity.
- Add is_user_member_of_organization() to OrganizationClient; queries
the organization_users association table directly (no lazy-load risk).
- Replace the identity check in authorize_workflow_run_start() with a
membership lookup. Deny when actor_user.id is not in the org's member
set; error_code stays workflow_not_found to avoid leaking existence.
- Update test: rename rejects_actor_from_another_org to
rejects_actor_not_a_member (reflects actual policy), add positive test
allows_invited_member that seeds membership and asserts has_quota=True.
Closes#491
* fix(auth): skip membership check for personal workflows (organization_id=None)
When workflow.organization_id is None (personal or legacy workflow with no
org), the membership lookup was still called, producing a SQL IS NULL
comparison that matched nothing and denied the run.
Guard the check so it only runs when the workflow is org-scoped.
Adds a regression test confirming that an actor with a known id can start a
personal workflow without triggering is_user_member_of_organization.
* fix(auth): fail closed on workflow membership lookup errors
---------
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* feat(tts): add xAI as a Voice (TTS) provider
pipecat already ships an xAI TTS service (XAITTSService, WebSocket
streaming) but dograh never wired it into the service configuration, so
xAI could not be selected as a Voice provider in the cascading pipeline.
Wire it through:
- registry: ServiceProviders.XAI + XAITTSConfiguration (voices
eve/ara/leo/rex/sal, language, computed model) registered in TTSConfig
- service_factory: build XAITTSService in create_tts_service
- check_validity: api-key validation hook
- tests for the factory + docs
The Voice provider dropdown is schema-driven, so xAI appears with no UI
changes.
* fix(tts): validate xAI API key and drop misleading auto-language hint
Addresses review feedback on the xAI Voice provider:
- check_validity: replace the no-op xAI key check with real validation
against xAI's OpenAI-compatible API (models.list on https://api.x.ai/v1),
so a bad BYOK key is caught at configuration time instead of at call time.
- registry: remove the "auto" language hint from the field description.
pipecat's Language enum has no "auto" member, so the factory fell back to
English silently; the description no longer advertises detection we don't do.
- tests: cover xAI key validation (registered, accepts valid, rejects bad).
* fix(tts): validate xAI key against the TTS voices endpoint
xAI supports endpoint-scoped API keys, so a key scoped to Text-to-Speech
may lack the /v1/models ACL and would be wrongly rejected by the previous
models.list() check. Validate against GET /v1/tts/voices instead — the
scope the key actually needs for TTS — treating 401/403 as an invalid key
and connection errors as a clean, actionable message.
* fix: harden xAI TTS integration
---------
Co-authored-by: Sabiha Khan <sabihak89@gmail.com>
Co-authored-by: Sabiha Khan <87858386+chewwbaka@users.noreply.github.com>
Two call sites dropped the MPS billing-v2 protocol, so orgs on model
config v2 got 400 "Service Key uses billing v2" from MPS:
- text_chat_runner built PipecatEngine without embeddings_provider
(extracted but never passed), so knowledge-base retrieval fell through
to the plain OpenAI-compatible client instead of DograhEmbeddingService
and sent no correlation_id/mps_billing_version metadata. Also pass
endpoint/api_version for Azure BYOK parity with run_pipeline.
- node_summary built its QA LLM without the run's correlation id, unlike
its sibling call in qa/analysis.py.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: gate OSS email/password auth endpoints outside local auth mode
The /auth signup/login/me routes were mounted unconditionally, so the
SaaS deployment accepted unauthenticated signups that created oss_*
provider-id users (and auto-provisioned MPS service keys) bypassing
Stack Auth entirely.
Gate them with a router-level dependency that 404s when AUTH_PROVIDER
is not "local", rather than conditionally mounting the router, so the
OpenAPI spec and the clients generated from it stay identical across
deployment modes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: keep current user route available in stack auth
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat: add Helm chart for Kubernetes deployment
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Replace bundled Bitnami subcharts with in-chart manifests on official images
The Bitnami catalog removed all versioned image tags from docker.io/bitnami in
Aug 2025 (old images frozen in bitnamilegacy, maintained catalog now behind a
Broadcom subscription), so the bundled postgresql/redis/minio subcharts no
longer pull. Replace them with plain in-chart manifests built on official
upstream images, keeping the internal/all-in-one path fully self-contained and
free of third-party chart packaging that can disappear:
- internal-postgres.yaml: pgvector/pgvector:pg17 — upstream Postgres plus the
`vector` extension the migrations require. POSTGRES_USER=dograh is the initdb
superuser, so CREATE EXTENSION vector succeeds.
- internal-redis.yaml: redis:7.4-alpine, password-protected, AOF persistence.
- internal-minio.yaml: minio/minio, root creds shared with the app via a single
secret (can't drift); the app auto-creates its bucket.
Service/secret names are unchanged (<rel>-postgresql, <rel>-redisinternal-master,
<rel>-minio) so the app wiring is untouched. Dep passwords are generated once and
persisted across upgrades via lookup. Drop the Chart.yaml dependencies,
Chart.lock, and the `helm dependency` step; the internal manifests gate on the
mode toggles (database.mode=internal, etc.).
Also fixes surfaced by smoke-testing on a live EKS cluster:
- Dockerfile: ship the per-service run_*.sh entrypoints the chart invokes.
- migrate-job: run as a post-install/pre-upgrade hook (the bundled Postgres does
not exist during pre-install) with a wait-for-postgres init container.
- backend env: declare POSTGRES_PASSWORD/REDIS_PASSWORD before the DATABASE_URL/
REDIS_URL that interpolate them (Kubernetes only expands back-references).
- worker liveness probes: pgrep isn't in the slim runtime image; check
/proc/1/cmdline instead (each worker execs its process as PID 1).
- UI: set HOSTNAME=0.0.0.0 so Next.js standalone doesn't bind to the k8s-injected
pod name (which maps to the pod IP only, breaking port-forward/loopback).
Verified end-to-end on EKS 1.36: all pods Ready, migrations applied (pgvector
extension + 27 tables), UI login page and web API served via port-forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(webhooks): durable retrying delivery for final webhooks
Final webhook nodes were fired inline with a single best-effort httpx POST
(run_integrations._execute_webhook_node). On a transient error the failure was
swallowed at three levels, so ARQ never retried and the final call report was
permanently lost -- leaving downstream receivers stuck (e.g. a CRM showing a
call as still "in conversation").
Replace the one-shot POST with a durable, idempotent delivery pipeline modelled
on the campaign retry pattern (persisted row + scheduled_for + bounded attempts):
- New webhook_deliveries table (WebhookDeliveryModel) is the source of truth.
Payload is rendered once and frozen so retries are deterministic; secrets are
not stored -- the credential is referenced by uuid and re-resolved at send time.
- run_integrations now persists a delivery row and enqueues deliver_webhook with
a deterministic ARQ job id instead of sending inline.
- deliver_webhook (new ARQ task) sends the request and:
* 2xx -> succeeded
* transient -> retry with capped exponential backoff (RequestError /
5xx / 408 / 425 / 429), up to max_attempts then dead_letter
* permanent 4xx -> dead_letter immediately (no pointless looping)
It is idempotent: a non-pending delivery is a no-op, so a duplicate enqueue or
sweeper re-injection can't double-send.
- sweep_webhook_deliveries cron (every 5 min) re-enqueues overdue pending
deliveries so nothing is lost to a worker restart / Redis flush.
- Stable X-Dograh-Delivery-Id / Workflow-Run-Id / Attempt headers let receivers
dedupe retried deliveries.
- enqueue_job now forwards ARQ job options (_job_id, _defer_by); failures log
repr(e) so empty-message errors like ConnectTimeout are diagnosable.
Config via DEFAULT_WEBHOOK_DELIVERY_CONFIG (env-overridable): max_attempts=5,
base_delay=30s, max_delay=600s, timeout=30s.
Tests cover payload rendering, persist+enqueue, success, transient retry,
retryable 5xx, permanent 4xx dead-letter, attempt exhaustion, and idempotency.
Migration verified to apply/rollback against Postgres; table/enum/indexes confirmed.
* fix(webhooks): atomic claim, safe success-recording, sweep paging, migration cleanup
Address review feedback on the webhook delivery pipeline:
- deliver_webhook now atomically claims a delivery (conditional UPDATE that
leases scheduled_for) before sending, so concurrent ARQ executions can't
double-send (the prior status=='pending' read was non-atomic).
- Recording success is moved out of the dead-letter try-block: if the receiver
accepted the payload (2xx) but the success DB-write fails, the row is left
pending for the sweeper to reconcile instead of being dead-lettered.
- The sweep keyset-paginates by id so a backlog over the page size is fully
drained, and logs the true re-enqueued total.
- Migration downgrade drops the enum via op.execute(DROP TYPE IF EXISTS ...)
instead of the deprecated op.get_bind().
* fix(webhooks): idempotent delivery creation and drop secret custom headers
Address the remaining review feedback:
- Add a (workflow_run_id, webhook_node_id) unique constraint and make
create_webhook_delivery a get-or-create returning (delivery, created). A
retried run_integrations now reuses the existing row instead of creating and
sending a duplicate final webhook; only a freshly-created row is enqueued.
- Stop persisting secret-looking custom headers (Authorization, X-API-Key,
Cookie, ...) in plaintext on the delivery row: they are dropped with a warning
pointing at the credential store (which is re-resolved securely at send time).
Non-secret custom headers are unaffected.
* fix(webhooks): harden idempotency key, secret-header match, sweep reclaim id
Address follow-up review feedback:
- webhook_node_id is now NOT NULL so a NULL can't slip past the
(workflow_run_id, webhook_node_id) unique constraint and create duplicates.
- Secret-header filtering matches normalized markers (auth/token/secret/cookie/
api-key/...) instead of an exact name list, catching variants like
X-Custom-Auth-Token while leaving benign headers (e.g. X-Idempotency-Key).
- The sweeper re-enqueues with a reclaim-specific job id (the lease timestamp)
so reconciling a delivered-but-unrecorded row isn't deduped against the
original attempt's already-completed ARQ job. The atomic claim still ensures
at most one send.
* fix(webhooks): scope delivery rows to workflow org
---------
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* chore: drain active calls before rolling updates
* Use provisional VAD interruption strategy
* feat: wire provisional VAD configuration
* chore: refactor user turn strategies
* chore: bump pipecat
* fix: reject misrouted smallwebrtc runs on the telephony websocket
A smallwebrtc (browser/WebRTC) workflow run is established through the WebRTC
signaling endpoint, not the PSTN telephony websocket. When such a run reached
_handle_telephony_websocket it read no "provider" from initial_context and
closed with an opaque "Provider type not found". Detect smallwebrtc runs and
close with a clear reason pointing to the signaling endpoint, without setting
the run to running or invoking a telephony provider. Also store the provider on
smallwebrtc runs at creation so they are self-describing, and make the generic
no-provider close reason include the run id and mode.
Closes#433
* fix: merge workflow run initial context defaults
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* feat(twilio): add Answering Machine Detection (AMD) support via telephony config
Closes#339
* chore: regenerate OpenAPI spec to fix drift-check
The openapi.json snapshot had drifted from the FastAPI app definition
because main gained new organization endpoints (billing, credits,
context) after this branch was created. Regenerate it with
'python -m scripts.dump_docs_openapi' to bring it back in sync.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add provider-level AMD hooks
* fix: handle db error while persisting amd result
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Sabiha Khan <sabihak89@gmail.com>
Co-authored-by: Sabiha Khan <87858386+chewwbaka@users.noreply.github.com>
