* feat: add ElevenLabs realtime STT provider support (#512)
Wire ElevenLabs scribe_v2_realtime into the STT registry and pipeline factory so BYOK transcribers can use the same provider already supported for TTS.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: address ElevenLabs STT review feedback for language, commits, and host
Pass custom language codes through instead of defaulting to English, use ElevenLabs VAD commit strategy because Dograh VAD runs downstream of STT, and document hostname-only realtime base_url handling.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: preserve ElevenLabs STT endpoint port in realtime host parsing
Use urlparse netloc instead of hostname so validated BYOK/proxy base URLs keep non-default ports when Pipecat builds the websocket endpoint.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: preserve ElevenLabs STT proxy path prefix and remove duplicate tests
Include URL path segments in realtime host normalization for BYOK proxies and delete shadowed pytest definitions.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: allow custom ElevenLabs model input
* fix: normalize ElevenLabs websocket URLs
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
Add model_options to SmallestAITTSConfiguration's voice field so the UI
renders the correct voice list per model — 15 standard voices for
lightning_v3.1 and 6 premium voices (meher, rhea, aviraj, cressida,
willow, maverick) for lightning_v3.1_pro. All 21 voice IDs verified
against the Smallest AI API. The frontend's existing model_options
machinery already handles dropdown filtering and auto-reset on model
change, so no UI changes are needed.
* feat: add Azure AI multi-provider support (TTS, STT, Embeddings, Realtime)
Enables Azure AI services across all model layers so users with Azure
credits can consolidate billing on a single provider.
- Voice (TTS): AzureSpeechTTSConfiguration via azure_speech provider
- Transcriber (STT): AzureSpeechSTTConfiguration via azure_speech provider
- Embedding: AzureOpenAIEmbeddingsConfiguration via azure provider
- Realtime: AzureRealtimeLLMConfiguration via azure_realtime provider
New files:
- api/services/pipecat/realtime/azure_realtime.py
- api/services/gen_ai/embedding/azure_openai_service.py
- api/tests/test_azure_speech_service_factory.py
The UI picks up all four providers automatically from the schema —
no frontend changes required.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: add validation for URL and params
---------
Co-authored-by: Vishal Dhateria <vishal@finela.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* fix: support object and array parameters in custom HTTP tools
* feat(ui): expose object and array types in the custom tool parameter editor
* fix: error handling and schema generation
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
* Add Sarvam LLM provider, update Sarvam STT models, expose usage_info on run detail.
Depends on pipecat PR dograh-hq/pipecat#43 for STT string language support.
Submodule bump will follow after that merges.
* test: cover Sarvam STT language mapping; link Sarvam docs
---------
Co-authored-by: Sabiha Khan <sabihak89@gmail.com>
