* fix(quota): fail closed when quota verification errors (#331)
Quota enforcement fell open on unexpected errors: the outer `except` in
`authorize_workflow_run_start` returned `has_quota=True`, so a degraded
database or a config-resolution bug let a billable run start unverified.
Billing and abuse protection are control-plane functions, so this is the
wrong default under exactly the degraded conditions that matter.
- Fail closed by default: the outer handler now returns
`has_quota=False` / `quota_check_failed`, reusing the existing message.
- Add `QUOTA_FAIL_MODE=closed|open` (default `closed`) so OSS self-hosters
can explicitly opt back into availability; the open path logs loudly.
- Narrow the try-scope so `get_user_by_id` / `get_workflow_run` DB read
failures surface as their specific `user_not_found` /
`workflow_run_not_found` codes instead of the generic handler.
- Tests cover the config-resolution and DB-read failure paths (denied,
not `has_quota=True`) and the `QUOTA_FAIL_MODE=open` escape hatch.
Fixes#331
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(quota): route DB read failures through the fail-mode policy gate
Review (greptile) flagged that the narrowed get_user_by_id / get_workflow_run
catches returned user_not_found / workflow_run_not_found before the outer
QUOTA_FAIL_MODE handler ran, so QUOTA_FAIL_MODE=open never applied to a DB
failure -- the exact "degraded database" case the escape hatch documents.
Revert the two narrowed catches so DB read exceptions fall through to the
single outer policy gate: closed -> quota_check_failed, open -> allow. The
None checks still return the specific not_found codes for genuinely missing
rows; an exception is a "cannot verify" condition, not a definitive absence.
Add a regression test asserting QUOTA_FAIL_MODE=open allows a run when a DB
read throws, and update the two DB-error tests to expect quota_check_failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(quota): scope the fail-mode comment to credit-verification failures (#331)
Review (cubic) flagged the outer-handler comment as overclaiming: it said the
handler is the single gate for "all cannot-verify errors", but the earlier
workflow-load and org-membership catches always deny with workflow_not_found
regardless of QUOTA_FAIL_MODE. That distinction is intentional (those are
authorization/existence gates, not credit verification), so scope the comment
accordingly. Comment-only, no behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(quota): fail open only when MPS is unreachable
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* feat: add ElevenLabs realtime STT provider support (#512)
Wire ElevenLabs scribe_v2_realtime into the STT registry and pipeline factory so BYOK transcribers can use the same provider already supported for TTS.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: address ElevenLabs STT review feedback for language, commits, and host
Pass custom language codes through instead of defaulting to English, use ElevenLabs VAD commit strategy because Dograh VAD runs downstream of STT, and document hostname-only realtime base_url handling.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: preserve ElevenLabs STT endpoint port in realtime host parsing
Use urlparse netloc instead of hostname so validated BYOK/proxy base URLs keep non-default ports when Pipecat builds the websocket endpoint.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: preserve ElevenLabs STT proxy path prefix and remove duplicate tests
Include URL path segments in realtime host normalization for BYOK proxies and delete shadowed pytest definitions.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: allow custom ElevenLabs model input
* fix: normalize ElevenLabs websocket URLs
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* feat(auth): gate OSS signup behind ENABLE_SIGNUP flag
## Problem
The `POST /api/v1/auth/signup` endpoint is unconditionally exposed on
every OSS install. Operators running an invite-only deployment (private
customer instances, staging environments, internal-only tenants) have
no way to disable public account creation without patching the codebase.
The UI also shows the "Sign up" link on `/auth/login` regardless of
whether signup is available, so a locked-down deployment leaves broken
navigation on the login page.
## Fix
Introduce a single `ENABLE_SIGNUP` env var (default `true` — no behavior
change for existing installs) that controls signup end-to-end:
- **Backend** — `api/constants.ENABLE_SIGNUP` is read at module load.
The signup handler returns 403 when it's false. Also exposed on
`GET /api/v1/health` as `signup_enabled: bool` so the UI can mirror
the operator's choice at runtime instead of at bundle-build time.
- **UI** — `getSignupEnabled()` in `lib/auth/config.ts` proxies the
health field, `/api/config/auth` surfaces it to the browser, the
login page conditionally renders the "Sign up" link via a one-shot
`fetch("/api/config/auth")` in `useEffect`, and the middleware
redirects `/auth/signup` → `/auth/login` when disabled (fires before
Next.js can serve the statically-prerendered signup page).
- **Helm** — `config.enableSignup` (default `true`) is rendered into
the ConfigMap as `ENABLE_SIGNUP` so operators can flip it via
`--set config.enableSignup=false` at install/upgrade time.
Fallbacks default to `signupEnabled: true` in every layer so a fresh
install "just works" and matches the backend default.
* address review: rollout on ConfigMap change, cache TTL, no signup-link flash
Four review points on #514:
**P1 — ConfigMap Change Skips Rollout** (`configmap.yaml`). `helm upgrade
--set config.enableSignup=false` updated the ConfigMap but did NOT roll
the api pods, so running processes kept the ENABLE_SIGNUP env from
startup and continued serving the old signup behavior — including
divergence between replicas mid-upgrade.
Fix: add the standard `checksum/config` pod-template annotation on the
four backend Deployments that `envFrom` the ConfigMap (`web`,
`arq-worker`, `ari-manager`, `campaign-orchestrator`). Verified with
`helm template`: all four Deployments share the same checksum on any
given render, and flipping `config.enableSignup` changes the checksum
uniformly so kubectl sees a pod-template diff and rolls all four.
**P1 — Signup Flag Stays Cached (server)** (`ui/src/lib/auth/config.ts`).
Module-scoped cache had no TTL. `revalidate: 300` was passed on the
underlying `fetch()` but the in-memory short-circuit above ran first, so
the value never refreshed until the UI pod restarted.
Fix: add `AUTH_CONFIG_TTL_MS = 5 * 60 * 1000` (matching the fetch
revalidate hint) so the module cache and the Next fetch cache stay in
sync. Backend flag flips propagate within 5 minutes without a pod
restart.
**P1 — Middleware Redirect Uses Stale State** (`ui/src/middleware.ts`).
Same shape as above — a separate module cache with no expiry could keep
redirecting `/auth/signup → /auth/login` after signup was re-enabled, or
keep serving the statically-prerendered signup page after lockdown.
Fix: same `SERVER_CONFIG_TTL_MS = 5 * 60 * 1000` TTL on the middleware
cache.
**P2 — Signup link flash on login page** (`ui/src/app/auth/login/page.tsx`).
Initial `signupEnabled` state was `null`, so `{signupEnabled && ...}`
hid the link on first paint and it popped in after the fetch resolved
— a CLS on every login-page load on stock installs where signup is
enabled.
Fix: initialise the state to `true` (matches the backend default). The
fetch still overrides to `false` when the operator has actually
disabled signup, so the lockdown UI behavior is unchanged; only the
happy-path flash is gone.
* simplify signup flag: drop TTL caches and middleware redirect
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* resolve signup flag server-side to avoid signup link flicker
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: prabhat pankaj <prabhatiitbhu@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(auth): allow invited org members to start workflow runs
Users invited to an org could not start workflows belonging to that org
because the authorization check compared actor.selected_organization_id
directly against workflow.organization_id. An invited user's selected
org correctly reflects the invited org, but if the Stack Auth token
resolves to a different org id than expected the strict equality fails.
Per api/AGENTS.md: "Whenever you read or write an organization-scoped
field, you must filter or validate by organization_id." The correct
policy is org membership, not selected-org identity.
- Add is_user_member_of_organization() to OrganizationClient; queries
the organization_users association table directly (no lazy-load risk).
- Replace the identity check in authorize_workflow_run_start() with a
membership lookup. Deny when actor_user.id is not in the org's member
set; error_code stays workflow_not_found to avoid leaking existence.
- Update test: rename rejects_actor_from_another_org to
rejects_actor_not_a_member (reflects actual policy), add positive test
allows_invited_member that seeds membership and asserts has_quota=True.
Closes#491
* fix(auth): skip membership check for personal workflows (organization_id=None)
When workflow.organization_id is None (personal or legacy workflow with no
org), the membership lookup was still called, producing a SQL IS NULL
comparison that matched nothing and denied the run.
Guard the check so it only runs when the workflow is org-scoped.
Adds a regression test confirming that an actor with a known id can start a
personal workflow without triggering is_user_member_of_organization.
* fix(auth): fail closed on workflow membership lookup errors
---------
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* feat(tts): add xAI as a Voice (TTS) provider
pipecat already ships an xAI TTS service (XAITTSService, WebSocket
streaming) but dograh never wired it into the service configuration, so
xAI could not be selected as a Voice provider in the cascading pipeline.
Wire it through:
- registry: ServiceProviders.XAI + XAITTSConfiguration (voices
eve/ara/leo/rex/sal, language, computed model) registered in TTSConfig
- service_factory: build XAITTSService in create_tts_service
- check_validity: api-key validation hook
- tests for the factory + docs
The Voice provider dropdown is schema-driven, so xAI appears with no UI
changes.
* fix(tts): validate xAI API key and drop misleading auto-language hint
Addresses review feedback on the xAI Voice provider:
- check_validity: replace the no-op xAI key check with real validation
against xAI's OpenAI-compatible API (models.list on https://api.x.ai/v1),
so a bad BYOK key is caught at configuration time instead of at call time.
- registry: remove the "auto" language hint from the field description.
pipecat's Language enum has no "auto" member, so the factory fell back to
English silently; the description no longer advertises detection we don't do.
- tests: cover xAI key validation (registered, accepts valid, rejects bad).
* fix(tts): validate xAI key against the TTS voices endpoint
xAI supports endpoint-scoped API keys, so a key scoped to Text-to-Speech
may lack the /v1/models ACL and would be wrongly rejected by the previous
models.list() check. Validate against GET /v1/tts/voices instead — the
scope the key actually needs for TTS — treating 401/403 as an invalid key
and connection errors as a clean, actionable message.
* fix: harden xAI TTS integration
---------
Co-authored-by: Sabiha Khan <sabihak89@gmail.com>
Co-authored-by: Sabiha Khan <87858386+chewwbaka@users.noreply.github.com>
* fix: gate OSS email/password auth endpoints outside local auth mode
The /auth signup/login/me routes were mounted unconditionally, so the
SaaS deployment accepted unauthenticated signups that created oss_*
provider-id users (and auto-provisioned MPS service keys) bypassing
Stack Auth entirely.
Gate them with a router-level dependency that 404s when AUTH_PROVIDER
is not "local", rather than conditionally mounting the router, so the
OpenAPI spec and the clients generated from it stay identical across
deployment modes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: keep current user route available in stack auth
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(webhooks): durable retrying delivery for final webhooks
Final webhook nodes were fired inline with a single best-effort httpx POST
(run_integrations._execute_webhook_node). On a transient error the failure was
swallowed at three levels, so ARQ never retried and the final call report was
permanently lost -- leaving downstream receivers stuck (e.g. a CRM showing a
call as still "in conversation").
Replace the one-shot POST with a durable, idempotent delivery pipeline modelled
on the campaign retry pattern (persisted row + scheduled_for + bounded attempts):
- New webhook_deliveries table (WebhookDeliveryModel) is the source of truth.
Payload is rendered once and frozen so retries are deterministic; secrets are
not stored -- the credential is referenced by uuid and re-resolved at send time.
- run_integrations now persists a delivery row and enqueues deliver_webhook with
a deterministic ARQ job id instead of sending inline.
- deliver_webhook (new ARQ task) sends the request and:
* 2xx -> succeeded
* transient -> retry with capped exponential backoff (RequestError /
5xx / 408 / 425 / 429), up to max_attempts then dead_letter
* permanent 4xx -> dead_letter immediately (no pointless looping)
It is idempotent: a non-pending delivery is a no-op, so a duplicate enqueue or
sweeper re-injection can't double-send.
- sweep_webhook_deliveries cron (every 5 min) re-enqueues overdue pending
deliveries so nothing is lost to a worker restart / Redis flush.
- Stable X-Dograh-Delivery-Id / Workflow-Run-Id / Attempt headers let receivers
dedupe retried deliveries.
- enqueue_job now forwards ARQ job options (_job_id, _defer_by); failures log
repr(e) so empty-message errors like ConnectTimeout are diagnosable.
Config via DEFAULT_WEBHOOK_DELIVERY_CONFIG (env-overridable): max_attempts=5,
base_delay=30s, max_delay=600s, timeout=30s.
Tests cover payload rendering, persist+enqueue, success, transient retry,
retryable 5xx, permanent 4xx dead-letter, attempt exhaustion, and idempotency.
Migration verified to apply/rollback against Postgres; table/enum/indexes confirmed.
* fix(webhooks): atomic claim, safe success-recording, sweep paging, migration cleanup
Address review feedback on the webhook delivery pipeline:
- deliver_webhook now atomically claims a delivery (conditional UPDATE that
leases scheduled_for) before sending, so concurrent ARQ executions can't
double-send (the prior status=='pending' read was non-atomic).
- Recording success is moved out of the dead-letter try-block: if the receiver
accepted the payload (2xx) but the success DB-write fails, the row is left
pending for the sweeper to reconcile instead of being dead-lettered.
- The sweep keyset-paginates by id so a backlog over the page size is fully
drained, and logs the true re-enqueued total.
- Migration downgrade drops the enum via op.execute(DROP TYPE IF EXISTS ...)
instead of the deprecated op.get_bind().
* fix(webhooks): idempotent delivery creation and drop secret custom headers
Address the remaining review feedback:
- Add a (workflow_run_id, webhook_node_id) unique constraint and make
create_webhook_delivery a get-or-create returning (delivery, created). A
retried run_integrations now reuses the existing row instead of creating and
sending a duplicate final webhook; only a freshly-created row is enqueued.
- Stop persisting secret-looking custom headers (Authorization, X-API-Key,
Cookie, ...) in plaintext on the delivery row: they are dropped with a warning
pointing at the credential store (which is re-resolved securely at send time).
Non-secret custom headers are unaffected.
* fix(webhooks): harden idempotency key, secret-header match, sweep reclaim id
Address follow-up review feedback:
- webhook_node_id is now NOT NULL so a NULL can't slip past the
(workflow_run_id, webhook_node_id) unique constraint and create duplicates.
- Secret-header filtering matches normalized markers (auth/token/secret/cookie/
api-key/...) instead of an exact name list, catching variants like
X-Custom-Auth-Token while leaving benign headers (e.g. X-Idempotency-Key).
- The sweeper re-enqueues with a reclaim-specific job id (the lease timestamp)
so reconciling a delivered-but-unrecorded row isn't deduped against the
original attempt's already-completed ARQ job. The atomic claim still ensures
at most one send.
* fix(webhooks): scope delivery rows to workflow org
---------
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* chore: drain active calls before rolling updates
* Use provisional VAD interruption strategy
* feat: wire provisional VAD configuration
* chore: refactor user turn strategies
* chore: bump pipecat
* fix: reject misrouted smallwebrtc runs on the telephony websocket
A smallwebrtc (browser/WebRTC) workflow run is established through the WebRTC
signaling endpoint, not the PSTN telephony websocket. When such a run reached
_handle_telephony_websocket it read no "provider" from initial_context and
closed with an opaque "Provider type not found". Detect smallwebrtc runs and
close with a clear reason pointing to the signaling endpoint, without setting
the run to running or invoking a telephony provider. Also store the provider on
smallwebrtc runs at creation so they are self-describing, and make the generic
no-provider close reason include the run id and mode.
Closes#433
* fix: merge workflow run initial context defaults
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* feat(twilio): add Answering Machine Detection (AMD) support via telephony config
Closes#339
* chore: regenerate OpenAPI spec to fix drift-check
The openapi.json snapshot had drifted from the FastAPI app definition
because main gained new organization endpoints (billing, credits,
context) after this branch was created. Regenerate it with
'python -m scripts.dump_docs_openapi' to bring it back in sync.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add provider-level AMD hooks
* fix: handle db error while persisting amd result
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Sabiha Khan <sabihak89@gmail.com>
Co-authored-by: Sabiha Khan <87858386+chewwbaka@users.noreply.github.com>
* fix: disable duplicate trigger nodes in workflow builder
AddNodePanel: disable trigger buttons and show tooltip when a trigger
already exists on the canvas, using bySpecName to identify trigger-
category specs from the live node list.
useWorkflowState: preflight in saveWorkflow rejects saves with multiple
trigger nodes via a sonner toast before the network request is made.
text_chat_session_service: include the original exception message in
TextChatSessionExecutionError so the HTTP 500 detail surfaces the root
cause without DB inspection.
Closes#378
* style: format test_text_chat_session_service.py with ruff
* chore: retrigger CI checks
* fix(workflow): enforce node instance constraints
---------
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* fix(qa): tolerate non-dict JSON from QA LLM instead of crashing
parse_llm_json is explicitly designed to return a list when the model emits a
top-level JSON array (it has a dedicated test for that). The QA analyzers then
call parsed.get("tags", ...) directly on the result. When parsed is a list,
that raises AttributeError, which is NOT caught by the surrounding
except (json.JSONDecodeError, ValueError) — so a single stray array response
from the QA model crashed the entire QA analysis run instead of degrading to
empty results.
The live variable-extraction path already guards this exact case with an
isinstance(..., dict) check; mirror it in both QA analysis call sites
(_run_qa_analysis per-node and _run_whole_call_qa_analysis fallback) so a
non-dict parse result coerces to {} and the run produces empty defaults.
Adds a regression test that drives the whole-call analyzer with an array
response and asserts empty results rather than a crash.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(qa): log non-object QA JSON responses
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
validate_trigger_paths used seen_paths.get(trigger_path) and treated a None
result as "path not seen yet". But None is also what node.get("id") returns
for a node without an id, so when the first trigger node sharing a path had no
id, it was stored as None and every later node with the same path was silently
accepted as unique — duplicate trigger paths slipped through validation.
Use a membership test (trigger_path not in seen_paths) so "first occurrence"
and "node_id happens to be None" are no longer conflated. Behavior is
unchanged for nodes that have ids.
Adds a regression test that fails before and passes after.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix: add language field to CartesiaTTSConfiguration and pass to TTS service
Closes#432
* chore: regenerate OpenAPI spec to fix drift-check
The openapi.json snapshot had drifted from the FastAPI app definition
because main gained new organization endpoints (billing, credits,
context) after this branch was created. Regenerate it with
'python -m scripts.dump_docs_openapi' to bring it back in sync.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: clarify Cartesia language schema
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
