Commit graph

103 commits

Author SHA1 Message Date
Vishal Dhateria
7ba95c0fbe
feat: add Azure AI multi-provider support (TTS, STT, Embeddings, Realtime) (#381)
* feat: add Azure AI multi-provider support (TTS, STT, Embeddings, Realtime)

Enables Azure AI services across all model layers so users with Azure
credits can consolidate billing on a single provider.

- Voice (TTS): AzureSpeechTTSConfiguration via azure_speech provider
- Transcriber (STT): AzureSpeechSTTConfiguration via azure_speech provider
- Embedding: AzureOpenAIEmbeddingsConfiguration via azure provider
- Realtime: AzureRealtimeLLMConfiguration via azure_realtime provider

New files:
- api/services/pipecat/realtime/azure_realtime.py
- api/services/gen_ai/embedding/azure_openai_service.py
- api/tests/test_azure_speech_service_factory.py

The UI picks up all four providers automatically from the schema —
no frontend changes required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: add validation for URL and params

---------

Co-authored-by: Vishal Dhateria <vishal@finela.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-06-02 12:50:00 +05:30
Matt Van Horn
dd85c4a1b4
fix: support object and array parameters in custom HTTP tools (#373)
* fix: support object and array parameters in custom HTTP tools

* feat(ui): expose object and array types in the custom tool parameter editor

* fix: error handling and schema generation

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-06-02 11:35:38 +05:30
Abhishek Kumar
fcb7004c7a feat: create tools using MCP 2026-05-31 16:50:44 +05:30
Abhishek Kumar
c586d02d5d feat: abort immediately on max call duration exceed 2026-05-31 13:21:37 +05:30
Abhishek Kumar
78ba62e185 feat: banner if API is not reachable 2026-05-31 13:05:22 +05:30
Abhishek
5ef3be92b5
chore: update pipecat to 1.3.0 (#379)
* chore: rename PipelineTask to PipelineWorker

* fix: fix tests

* chore: update pipecat submodule

* fix: fix anyio same task cancellation scope
2026-05-29 16:19:42 +05:30
Abhishek
3892b58486
feat: add ultravox realtime and fix signature issue in telephony (#345)
* feat: add ultravox realtime and fix signature issue in telephony

- Add UltraVox realtime
- Fix signature issue on telephony

* fix: fix regression for wss_backend_endpoint
2026-05-23 12:51:55 +05:30
Abhishek Kumar
ad2fa07058 feat: add google stt and tts. add folders to organize agents 2026-05-22 14:36:50 +05:30
Abhishek
d97d1d72cd
feat: add chat based testing for voice agent (#308)
* feat: add backend foundations

* feat: add text chat UI

* chore: simplify the reload behaviour

* fix: fix upgrade banner to be triggered after package upload

* feat: simplify TesterPanel design

* chore: fix formatting and generate client

* chore: fix tracing for text chat mode

* fix: fix revert and edit CTA

* refactor: refactor TesterPanel into smaller components

* feat: enable runtime transition of nodes

* fix: fix review comments
2026-05-21 15:20:02 +05:30
Leoy
5762095edf
feat(mcp): add search_docs tool over docs corpus (closes #295) (#316)
* feat(mcp): add search_docs tool over Mintlify docs corpus

Closes #295. The docs at https://docs.dograh.com promise "Search the
Dograh docs for how to configure a TURN server" as an MCP example
prompt, but no search_docs tool exists in the MCP server — agents can
list workspace resources but cannot search the documentation.

This adds a dependency-free, in-process keyword search over the
`docs/` tree shipped into the API image (`COPY ./docs ./docs`):

- New `api/mcp_server/tools/docs_search.py` — async `search_docs(query,
  limit=10)` with weighted scoring (path > title > body), a 25-result
  hard cap, snippet extraction around the first term hit, and graceful
  empty-list degradation when docs aren't on disk. `DOGRAH_DOCS_PATH`
  env var overrides location discovery for non-Docker layouts.

- Registered in `api/mcp_server/server.py` alongside the other tools,
  keeping the existing list-alphabetical convention.

- `api/tests/test_mcp_docs_search.py` — 18 unit tests covering the
  pure helpers (tokenizer, frontmatter stripping, title extraction,
  scoring weights, URL building) and end-to-end ranking, limit
  clamping, empty-corpus degradation, and input-validation errors.
  Mocks `authenticate_mcp_request` to avoid the DB dependency,
  mirroring `test_mcp_save_workflow.py`.

Implementation notes:
- The docs corpus is ~100 files / ~140k LoC, so a per-call scan runs
  well under 50 ms; avoiding a vector index / embedding backend keeps
  the tool zero-dependency and works for fully offline self-hosted
  deployments.
- Authentication is required for consistency with the other MCP tools
  (and to route through the existing rate-limit middleware), even
  though docs are not org-scoped data.
- Title/path matches deliberately outweigh body matches so a page
  whose subject IS the query term outranks one that merely mentions
  it incidentally.

* feat: improve docs search

---------

Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-05-20 18:20:35 +05:30
Mohamed-Mamdouh
5f28c1b2a9
feat: add Tuner Integration to Dograh (#311)
* Add tuner integration

* bump pipecat version

* chore: update pipecat submodule to match upstream and use tuner-pipecat-sdk 0.2.0

Update pipecat submodule from 0.0.109.dev23 to 13e98d0d9 (the exact commit
upstream dograh-hq/dograh uses after v1.30.1). This installs pipecat-ai as
1.1.0.post277 via setuptools_scm, satisfying tuner-pipecat-sdk 0.2.0's
pipecat-ai>=1.0.0 requirement.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* wire tuner

* feat: refactor integrations into self contained packages

* chore: simplify ensure_public_access_token

* fix: remove NodeSpec and make DTOs the source of truth

* feat: send relevant signal to mcp using to_mcp_dict

* fix: fix tests

* cleanup: remove nango integrations

* feat: add agents.md for integrations

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-05-20 14:37:33 +05:30
Paulo Busato Favarato
75839f9de5
feat(mcp): generic MCP tool source with per-node function filtering (#301)
* feat(mcp): generic MCP tool source with per-node function filtering

Adds a Model Context Protocol tool category: connect a customer MCP
server and expose its tools to the agent, with optional per-node
allow-listing of individual MCP functions.

- ToolCategory.MCP enum + alembic migration
- MCP definition validator and collision-safe function-name namespacing
- McpToolSession wrapper: graceful-degrade, per-call open/close lifecycle
- CustomToolManager MCP branch (schemas + proxy handlers)
- Per-node mcp_tool_filters threaded through DTO/graph/engine
- Best-effort discovered_tools catalog cache + POST /tools/{uuid}/mcp/refresh
- UI: MCP create/edit config, tabbed ToolSelector with per-node toggles

* feat: refactor for code standardisation and documentation

---------

Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-05-19 16:10:00 +05:30
Abhishek
2381a803ad
feat: add openai realtime models (#298)
* feat: add openai realtime models

* chore: bump pipecat

* fix: resample telephony audio for openai realtime

* fix: sampling rate fix for openai realtime

* chore: clean up dead code
2026-05-16 18:05:23 +05:30
Abhishek Kumar
b728cc4922 chore: fix ARI documentation 2026-05-13 21:53:18 +05:30
Sabiha Khan
a1902829db
fix: prior pre-pr drift check failures (#276)
* fix: prior pre-pr drift check failures

* docs: update api reference openapi json
2026-05-12 14:17:40 +05:30
Sabiha Khan
4a6752e62b
feat(telephony/telnyx): add call transfer via conference bridge (#274)
Conference-based transfer for Telnyx with a two-step flow:
- transfer_call dials the destination with a per-transfer webhook URL.
- On call.answered, the webhook seeds a conference with the destination's
  live call_control_id and publishes DESTINATION_ANSWERED.
- TelnyxConferenceStrategy joins the caller into the conference on
  pipeline teardown (EndTaskReason.TRANSFER_CALL).
- On post-answer destination hangup, the webhook hangs up the caller —
  Telnyx's create_conference doesn't accept end_conference_on_exit on
  the seed leg, so we tear down the bridge ourselves.

TransferContext gains optional workflow_run_id (for webhook→provider
resolution in multi-config orgs) and conference_id (set on answer,
rd by the strategy).

Also fixes the transfer tool's provider lookup to go through
get_telephony_provider_for_run instead of the deprecated org-default
shim, which was returning the wrong provider in multi-config orgs.
2026-05-12 13:44:39 +05:30
Abhishek Kumar
4afe426f12 chore: fix tests 2026-05-11 17:21:02 +05:30
Abhishek Kumar
45a81c88e0 chore: add more fixtures 2026-05-08 16:28:09 +05:30
Abhishek Kumar
5a358d4d29 feat: add workflow graph constraints fixtures 2026-05-08 16:02:51 +05:30
Sabiha Khan
d2a119c38a
feat: add headless mode, redesign floating widget, refactor lifecycle callbacks (#268)
* feat: add headless widget for deployment

* feat: call callbacks at the right time

* feat: add onCallConnected & onCallDisconnected callback

* feat: add a button with text for floating widget

* feat: add headless widget for deployment

* feat: call callbacks at the right time

* feat: add onCallConnected & onCallDisconnected callback

* feat: add a button with text for floating widget

* docs: web widget

* fix: format issue in pre-pr drift check

* fix: fix CD to rely on pipecat dev dependey

* chore: update message

---------

Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-05-07 12:23:41 +05:30
Abhishek
d4b6afb020
feat: add logs in campaigns for failure or pausing (#265)
* feat: add logs in campaigns on failure

* chore: bump pipecat

* chore: update format.sh

* chore: fix github workflow

* fix: fix formatting errors
2026-05-05 19:23:50 +05:30
Abhishek
0e12c41fc7
chore: bump pipecat version and fix tests (#263)
* chore: bump pipecat version and fix tests

* chore: add github workflow to run tests

* fix: install reqirements.dev.txt in test script

* fix: fix api-test action

* feat: add integration test

* test: add integration tests

* test: add test for function call mute strategy
2026-05-04 21:35:37 +05:30
Abhishek Kumar
0d7b225f76 chore: updatae documentation for api trigger 2026-05-04 12:46:11 +05:30
Abhishek Kumar
4171ad7a54 feat: add test mode for API trigger 2026-04-25 16:30:26 +05:30
Abhishek Kumar
16587fd7e1 chore: fix duplicate workflow issue 2026-04-24 13:17:04 +05:30
Abhishek
00a1a22b74
feat: refactor node spec and add mcp tools (#244)
* refactor: carve out extraction panel

* refactor: create spec versions for node types

* refactor: create a GenericNode and remove custom nodes

* feat: add python and typescript sdk

* add dograh sdk

* fix: fetch draft workflow definition over published one

* fix: fix routes of SDKs to use code gen

* chore: remove doclink dependency to reduce image size

* chore: format files

* chore: bump pipecat

* feat: let mcp fetch archived workflows on demand

* chore: fix tests

* feat: add sdk documentation

* chore: change banner and add badge
2026-04-21 07:56:16 +05:30
Abhishek Kumar
e31b38122e fix: fix interruption handling for Gemini Live
1. Fixes #236
2. Fix run_inference for variable extraction for Gemini Live
2026-04-15 19:29:07 +05:30
Abhishek Kumar
7fab959e26 feat: add redial option in campaigns 2026-04-13 23:25:43 +05:30
Abhishek Kumar
74dbafb055 feat: allow recordings in tool transitions 2026-04-10 16:18:01 +05:30
Abhishek Kumar
65c76ca7ff feat: allow uploading recording as part of node transition 2026-04-10 11:54:00 +05:30
Abhishek Kumar
8b3dc02722 fix: render prompt template for variable extraction 2026-04-09 15:20:56 +05:30
Abhishek Kumar
87c8c5e2c8 feat: add full document mode in knowledge base 2026-04-09 13:49:20 +05:30
Abhishek Kumar
1f5229e2df chore: update prompt for pre-recorded audio generation 2026-04-08 22:23:14 +05:30
Abhishek
38d1d928b7
feat: agent versioning and model configurations override (#227)
* feat: add tests and migrations

* feat: workflow versioning among published and draft

* feat: add a new settings page to simplify workflow detail page

* fix: fix tsclient generation
2026-04-08 19:20:31 +05:30
Abhishek Kumar
6a473a9db5 fix: set context before update settings for live 2026-04-07 18:46:17 +05:30
Abhishek
ec2f322486
feat: add pre call fetch configuration (#222)
* feat: add pre call fetch configuration

* docs: add NEW tags for pages about new features

---------

Co-authored-by: Sabiha Khan <sabihak89@gmail.com>
2026-04-06 12:30:37 +05:30
Abhishek Kumar
56763a4527 feat: enable context summarization 2026-04-03 13:39:02 +05:30
Abhishek Kumar
501d06c00d feat: add Assembly AI STT 2026-04-03 07:10:37 +05:30
Abhishek Kumar
96c90376c3 feat: add default initial context variables 2026-04-02 14:32:43 +05:30
Abhishek Kumar
f368fe5134 feat: set calculator as custom tool on demand 2026-04-02 14:07:03 +05:30
Abhishek
87e72d5f6f
feat: add gemini live and speaches integration (#220)
* feat: add speaches models

* feat: add gemini realtime and speaches integration

- Add gemini realtime support
- Add speaches support for locally hosted LLMs

* chore: bump pipecat

* feat: add language option

* fix: add skip aggregator types to tts settings

* fix: make API key optional for realtime
2026-03-31 21:42:03 +05:30
Sabiha Khan
e0c3d6c3bf
fix: skip updating gathered_context when the extracted variables is not a dict (#219) 2026-03-31 21:41:49 +05:30
Sabiha Khan
2588956c56
chore: increase timeout for pending variable extraction tasks (#208) 2026-03-25 19:51:07 +05:30
Abhishek Kumar
2fa4191d9b feat: allow recording audio in workflow builder 2026-03-25 15:01:39 +05:30
Abhishek Kumar
ac0731a374 feat: add support for self hosted llm models 2026-03-24 17:50:45 +05:30
Sabiha Khan
330e4a05f2
fix: pass system_instruction to one shot llm inferences to avoid syst… (#203)
* fix: pass system_instruction to one shot llm inferences to avoid system instruction from llm _settings

* chore: upgrade pipecat submodule
2026-03-24 11:25:24 +05:30
Sabiha Khan
70730616ec
fix: incorrect system instruction in llm inference of variable extrac… (#201)
* fix: incorrect system instruction in llm inference of variable extraction

* chore: bump pipecat submodule version
2026-03-24 09:52:02 +05:30
Abhishek Kumar
e859b92be8 Merge branch 'main' of https://github.com/dograh-hq/dograh 2026-03-23 11:36:42 +05:30
Abhishek Kumar
affb39e57f feat: custom telemetry configuration 2026-03-23 11:36:39 +05:30
Sabiha Khan
d42c52dc87
fix: await pending variable extraction tasks before pipeline finishes (#198) 2026-03-23 10:11:17 +05:30