dograh/api/schemas/tool.py
Rushil f4c5954a39
feat: add tool test panel for HTTP API tools (#547)
* feat: add tool test panel for HTTP API tools

Lets developers run a saved HTTP API tool against its real endpoint
from the tool detail page, without needing a live call. Reuses the
production execute_http_tool path so test behavior matches call-time
behavior.

- New POST /tools/{tool_uuid}/test route
- Test panel with per-parameter typed inputs and auto-detected
  context variable inputs (from preset parameter templates)
- Validate parameter name uniqueness on save, matching the existing
  transferParameters check
- Fix stale FunctionCallsFromLLMInfoFrame import causing test
  collection failures against pipecat-ai 1.5.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: revert local-venv pipecat drift fix, apply ruff import formatting

test_custom_tools.py and test_unregistered_function_call.py were edited
locally to drop FunctionCallsFromLLMInfoFrame after hitting an
ImportError — that error was from a stale local pipecat-ai package, not
a real drift. CI's pipecat build emits this frame and the test asserted
on it, so removing it broke test_llm_calls_custom_tool_handler and its
unregistered-call counterpart. Reverted both files to match main.

Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to
clear the drift-check job (split the aliased import into its own
`from ... import (...)` block, wrapped a long monkeypatch.setattr call).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec

pytest's default discovery matches any top-level test_* name in a test
module, including imported functions — importing the route handler as
`test_tool as test_tool_route` still matched the pattern, so pytest
tried to run it as a test and failed injecting fixtures for tool_uuid/
request/user. Renamed the alias to call_test_tool_route.

Also regenerated docs/api-reference/openapi.json for the new
POST /tools/{tool_uuid}/test route and its two schemas (couldn't run
the dump script locally — pipecat-ai version mismatch documented
separately — so hand-built the diff to exactly match FastAPI's
get_openapi() output format, verified against neighboring routes).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address cubic review findings on test panel

- Resolve dotted context-variable keys into nested objects before
  posting the test request. render_template's get_nested_value walks
  nested dicts, so a flat key like "runtime_configuration.realtime_model"
  never matched — templates referencing nested context always resolved
  to empty.
- Restrict isHttpApiTool to an explicit category equality check instead
  of inferring it from exclusions. native/integration tools are
  currently disabled in the create-tool UI so this wasn't reachable
  today, but the exclusion list silently goes stale as new categories
  are added.
- Stop showing a green success badge for non-2xx responses.
  execute_http_tool returns status: "success" for any HTTP exchange
  that completes, regardless of status code — only transport-level
  errors (timeout, connection failure) get status: "error". The test
  panel now checks status_code is in the 2xx range before treating the
  call as a success.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address second round of cubic/greptile findings

- Guard setNestedValue against prototype-pollution keys (__proto__,
  constructor, prototype) in the dotted context-var path before
  traversing.
- Normalize status to "error" in the test route when the upstream
  status_code is >= 400. execute_http_tool only distinguishes
  transport-level failures (timeout, connection error) from
  "success" — a completed 4xx/5xx exchange still came back as
  "success" from the executor.
- Seed testArgValues defaults for number/boolean parameters via a
  useEffect keyed on the parameters array, so a required number or
  boolean field isn't silently omitted from the test request if the
  user never touches its input.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: only seed test-arg defaults for required number/boolean params

Seeding optional number/boolean parameters silently changed the test
request — an optional boolean flag the tester never touched was sent
as true, which can flip upstream behavior unintentionally. Restrict
seeding to required parameters.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: match whitespace and fallback-filter syntax in context var detection

extractContextVars required an exact {{initial_context.foo}} with no
whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN
(and render_template) accepts {{ initial_context.foo }} and
{{initial_context.foo | fallback:value}}. A preset parameter saved with
either of those forms resolved fine in production but showed no input
in the test panel, so testing always sent it empty context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tool-test): add hint and request_* fields to ToolTestResponse

Extends ToolTestResponse with hint, request_method, request_url,
request_body, and request_params so the frontend can surface what was
actually sent and a human-readable hint about why a test call failed.

* feat(tool-test): add status-code hints and request_method/url/body/params to test_tool()

* style: ruff-format test_mcp_tool_route.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tool-test): wire hint banner and request block into result panel

Backend has returned hint/request_method/request_url/request_body/
request_params since d45ea851/60aaf31d but the frontend never
displayed them. Extends ToolTestResult with the new fields and renders
an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx)
plus a Request block above the response showing exactly what was sent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(tool-test): include resolved preset params in request_body/params

Found via live GET/POST testing: the Request preview showed only the
model-provided arguments, not what execute_http_tool actually sends.
execute_http_tool merges resolved_arguments = {**arguments,
**preset_arguments} before building the outbound body/params — preset
params (e.g. {{initial_context.metadata.channel}}) are invisible to
the model but still go out on the wire. The preview now mirrors that
merge via the same _resolve_preset_parameters helper, so a dev sees
exactly what was sent, not just what the model provided.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tool-test): add generateSampleValue helper for sample-fill button

* feat(tool-test): add Fill sample values button for arguments and context vars

Moved generateSampleValue out of page.tsx into a sibling helpers module:
Next.js's typed-route checker rejects extra named exports on a page.tsx
file (tsc error TS2344 on .next/types), so the helper and its test import
now live in testPanelHelpers.ts instead.

* feat(tool-test): add JSON edit modal state and handlers

* feat(tool-test): collapsed preview + edit modal for object/array test parameters

* fix(tool-test): validate JSON on modal open, not just on edit

Opening the JSON edit modal on an untouched object/array param (no value
yet in testArgValues) loaded an empty draft with jsonEditError hardcoded
to null, so Save was enabled despite invalid JSON and silently no-op'd on
click. Now runs the same JSON.parse check used by the live textarea
validation when the modal opens.

* chore: regenerate openapi.json for ToolTestResponse hint/request_* fields

drift-check on PR #547 was failing because the earlier hint/request_method/
request_url/request_body/request_params fields added to ToolTestResponse
were never reflected in the dumped spec. Regenerated via
scripts.dump_docs_openapi.

* fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner

httpx raises a TypeError when a query param value is a dict/list, which
was silently caught and surfaced as a generic tool-execution error —
this is what actually broke test requests, not just the "[object
Object]" display. JSON-stringify object/array arguments before they
become query params, in both the live execute_http_tool() path and the
test route's request_params display shaping.

Also adds an unsaved-changes warning banner above Test Tool, shown
when the live form state diverges from the last-saved HTTP API config,
since Test Tool always runs the saved config.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(tool-test): keep empty request body in preview; normalize headers in snapshot

- POST/PUT/PATCH with no arguments now shows `{}` in the request body
  preview instead of null — matches what execute_http_tool actually sends
  over the wire (json={})
- buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a
  deduped key→value map before serializing, matching the shape saved to
  the backend; duplicate header keys no longer cause a false unsaved-
  changes warning

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(tool-test): update assertion for empty POST body preview

test_tool_test_no_arguments_leaves_body_and_params_none expected
request_body=None for a POST with no arguments. The fix to preserve {}
in the preview makes request_body={} the correct assertion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tools): refine HTTP tool testing

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30

522 lines
18 KiB
Python

"""Pydantic schemas for reusable Dograh tools.
These models are the single contract for tool creation/update across the
REST API, generated SDKs, and the MCP authoring surface. Field descriptions
are human/API-facing; ``llm_hint`` JSON schema extras are guidance for LLMs
when the same schema is surfaced through MCP or SDK authoring flows.
"""
from __future__ import annotations
from datetime import datetime
from typing import Annotated, Any, Dict, List, Literal, Optional, Union
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
from api.enums import ToolCategory
DEFAULT_MCP_TIMEOUT_SECS = 30
DEFAULT_MCP_SSE_READ_TIMEOUT_SECS = 300
ToolParameterType = Literal["string", "number", "boolean", "object", "array"]
HttpMethod = Literal["GET", "POST", "PUT", "PATCH", "DELETE"]
ToolCategoryValue = Literal[
"http_api",
"end_call",
"transfer_call",
"calculator",
"native",
"integration",
"mcp",
]
def _llm_hint(text: str) -> dict[str, str]:
return {"llm_hint": text}
class ToolParameter(BaseModel):
"""A parameter that the tool accepts from the model at call time."""
name: str = Field(
description="Parameter name used as a key in the tool request body.",
json_schema_extra=_llm_hint(
"Use a stable snake_case name the agent can naturally fill."
),
)
type: ToolParameterType = Field(
description="JSON type for the parameter value.",
json_schema_extra=_llm_hint(
"Allowed values are string, number, boolean, object, and array."
),
)
description: str = Field(
description="Description shown to the model for this parameter.",
json_schema_extra=_llm_hint(
"Write this as an instruction to the agent: what value to provide and when."
),
)
required: bool = Field(
default=True,
description="Whether this parameter is required when the tool is called.",
)
class PresetToolParameter(BaseModel):
"""A parameter injected by Dograh at runtime."""
name: str = Field(description="Parameter name used as a key in the request body.")
type: ToolParameterType = Field(
description="JSON type for the resolved value.",
json_schema_extra=_llm_hint(
"Allowed values are string, number, boolean, object, and array."
),
)
value_template: str = Field(
description="Fixed value or template, e.g. {{initial_context.phone_number}}.",
json_schema_extra=_llm_hint(
"Use {{initial_context.*}} for call-start context and "
"{{gathered_context.*}} for values extracted during the call."
),
)
required: bool = Field(
default=True,
description="Whether the parameter must resolve to a non-empty value.",
)
class HttpApiConfig(BaseModel):
"""Configuration for HTTP API tools."""
method: HttpMethod = Field(
description="HTTP method to use for the request.",
json_schema_extra=_llm_hint("Use one of GET, POST, PUT, PATCH, DELETE."),
)
url: str = Field(
description="Target HTTP or HTTPS URL.",
json_schema_extra=_llm_hint(
"Use the final endpoint URL. Authentication belongs in credential_uuid, "
"not embedded in the URL."
),
)
headers: Optional[Dict[str, str]] = Field(
default=None,
description="Static headers to include with every request.",
json_schema_extra=_llm_hint(
"Do not place secrets here. Store secrets in the UI credential manager "
"and reference them with credential_uuid."
),
)
credential_uuid: Optional[str] = Field(
default=None,
description="Reference to an external credential for request authentication.",
json_schema_extra=_llm_hint(
"Use a credential_uuid returned by list_credentials. The MCP flow does "
"not create credential secrets."
),
)
parameters: Optional[List[ToolParameter]] = Field(
default=None,
description="Parameters the model must provide when calling this tool.",
)
preset_parameters: Optional[List[PresetToolParameter]] = Field(
default=None,
description=(
"Parameters injected by Dograh from fixed values or workflow context "
"templates."
),
)
timeout_ms: Optional[int] = Field(
default=5000,
ge=1,
description="Request timeout in milliseconds.",
)
customMessage: Optional[str] = Field(
default=None, description="Custom message to play after tool execution."
)
customMessageType: Optional[Literal["text", "audio"]] = Field(
default=None, description="Type of custom message."
)
customMessageRecordingId: Optional[str] = Field(
default=None, description="Recording ID for an audio custom message."
)
@field_validator("method", mode="before")
@classmethod
def validate_method(cls, v: Any) -> str:
if not isinstance(v, str):
raise ValueError("method must be one of GET, POST, PUT, PATCH, DELETE")
method = v.upper()
if method not in {"GET", "POST", "PUT", "PATCH", "DELETE"}:
raise ValueError("method must be one of GET, POST, PUT, PATCH, DELETE")
return method
class EndCallConfig(BaseModel):
"""Configuration for End Call tools."""
messageType: Literal["none", "custom", "audio"] = Field(
default="none", description="Type of goodbye message."
)
customMessage: Optional[str] = Field(
default=None, description="Custom message to play before ending the call."
)
audioRecordingId: Optional[str] = Field(
default=None, description="Recording ID for audio goodbye message."
)
endCallReason: bool = Field(
default=False,
description=(
"When enabled, the model must provide a reason for ending the call. "
"The reason is set as call disposition and added to call tags."
),
)
endCallReasonDescription: Optional[str] = Field(
default=None,
description=(
"Description shown to the model for the reason parameter. Used only "
"when endCallReason is enabled."
),
)
class HttpTransferResolverConfig(BaseModel):
"""HTTP endpoint used to resolve transfer destination at call time."""
type: Literal["http"] = Field(default="http", description="Resolver type.")
url: str = Field(description="HTTP or HTTPS endpoint for transfer resolution.")
headers: Optional[Dict[str, str]] = Field(
default=None,
description="Static headers to include with every resolver request.",
)
credential_uuid: Optional[str] = Field(
default=None,
description="Reference to an external credential for resolver authentication.",
)
timeout_ms: int = Field(
default=3000,
ge=500,
le=5000,
description="Resolver request timeout in milliseconds.",
)
wait_message: Optional[str] = Field(
default=None,
description="Optional short message played while Dograh resolves routing.",
)
parameters: Optional[List[ToolParameter]] = Field(
default=None,
description="Parameters the model may provide when calling this transfer tool.",
)
preset_parameters: Optional[List[PresetToolParameter]] = Field(
default=None,
description=(
"Parameters injected by Dograh from fixed values or workflow context "
"templates."
),
)
@field_validator("url")
@classmethod
def validate_url(cls, v: str) -> str:
if not isinstance(v, str) or not v.startswith(("http://", "https://")):
raise ValueError("config.resolver.url must be an http(s) URL")
return v
class TransferCallConfig(BaseModel):
"""Configuration for Transfer Call tools."""
destination_source: Literal["static", "dynamic"] = Field(
default="static",
description="Whether transfer destination is static/template or resolved by HTTP.",
)
destination: str = Field(
default="",
description=(
"Phone number, SIP endpoint, or template to transfer the call to, e.g. "
"+1234567890, PJSIP/1234, or {{initial_context.transfer_destination}}."
),
)
messageType: Literal["none", "custom", "audio"] = Field(
default="none", description="Type of message to play before transfer."
)
customMessage: Optional[str] = Field(
default=None, description="Custom message to play before transferring."
)
audioRecordingId: Optional[str] = Field(
default=None, description="Recording ID for audio message before transfer."
)
timeout: int = Field(
default=30,
ge=5,
le=120,
description="Maximum seconds to wait for the destination to answer.",
)
parameters: Optional[List[ToolParameter]] = Field(
default=None,
description=(
"Parameters the model may provide when calling this transfer tool, "
"for example state, department, or transfer reason."
),
)
resolver: Optional[HttpTransferResolverConfig] = Field(
default=None,
description="Optional resolver that determines transfer routing at call time.",
)
@model_validator(mode="after")
def validate_destination_source_config(self):
if self.destination_source == "dynamic" and self.resolver is None:
raise ValueError(
"config.resolver is required when destination_source is dynamic"
)
return self
class McpToolConfig(BaseModel):
"""Configuration for a customer MCP server tool definition."""
transport: Literal["streamable_http"] = Field(
default="streamable_http",
description="MCP transport protocol.",
)
url: str = Field(
description="MCP server URL. Must use http:// or https://.",
json_schema_extra=_llm_hint("Use the server's streamable HTTP MCP endpoint."),
)
credential_uuid: Optional[str] = Field(
default=None,
description="Reference to an external credential for MCP server auth.",
json_schema_extra=_llm_hint(
"Use a credential_uuid returned by list_credentials. Credentials are "
"created by the user in the UI."
),
)
tools_filter: list[str] = Field(
default_factory=list,
description="Allowlist of MCP tool names to expose. Empty exposes all tools.",
json_schema_extra=_llm_hint(
"Use exact MCP tool names from the remote server catalog when you need "
"to restrict the exposed tools."
),
)
timeout_secs: int = Field(
default=DEFAULT_MCP_TIMEOUT_SECS,
ge=0,
description="Connection timeout in seconds.",
)
sse_read_timeout_secs: int = Field(
default=DEFAULT_MCP_SSE_READ_TIMEOUT_SECS,
ge=0,
description="SSE read timeout in seconds.",
)
discovered_tools: list[dict[str, Any]] = Field(
default_factory=list,
description=(
"Server-managed cache of the MCP server's tool catalog "
"[{name, description}]. Populated best-effort by the backend."
),
json_schema_extra=_llm_hint("Do not author this field; the server fills it."),
)
@field_validator("url")
@classmethod
def validate_url(cls, v: str) -> str:
if not isinstance(v, str) or not v.startswith(("http://", "https://")):
raise ValueError("config.url must be an http(s) URL")
return v
@field_validator("tools_filter")
@classmethod
def validate_tools_filter(cls, v: list[str]) -> list[str]:
if not all(isinstance(tool_name, str) for tool_name in v):
raise ValueError("config.tools_filter must be a list of strings")
return v
class HttpApiToolDefinition(BaseModel):
"""Tool definition for HTTP API tools."""
schema_version: int = Field(default=1, description="Schema version.")
type: Literal["http_api"] = Field(description="Tool type.")
config: HttpApiConfig = Field(description="HTTP API configuration.")
class EndCallToolDefinition(BaseModel):
"""Tool definition for End Call tools."""
schema_version: int = Field(default=1, description="Schema version.")
type: Literal["end_call"] = Field(description="Tool type.")
config: EndCallConfig = Field(description="End Call configuration.")
class TransferCallToolDefinition(BaseModel):
"""Tool definition for Transfer Call tools."""
schema_version: int = Field(default=1, description="Schema version.")
type: Literal["transfer_call"] = Field(description="Tool type.")
config: TransferCallConfig = Field(description="Transfer Call configuration.")
class CalculatorToolDefinition(BaseModel):
"""Tool definition for Calculator tools."""
schema_version: int = Field(default=1, description="Schema version.")
type: Literal["calculator"] = Field(description="Tool type.")
class McpToolDefinition(BaseModel):
"""Persisted MCP tool definition."""
schema_version: int = Field(default=1, description="Schema version.")
type: Literal["mcp"] = Field(description="Tool type.")
config: McpToolConfig = Field(description="MCP server configuration.")
ToolDefinition = Annotated[
Union[
HttpApiToolDefinition,
EndCallToolDefinition,
TransferCallToolDefinition,
CalculatorToolDefinition,
McpToolDefinition,
],
Field(discriminator="type"),
]
class CreateToolRequest(BaseModel):
"""Request schema for creating a reusable tool."""
name: str = Field(
max_length=255,
description="Display name for the tool.",
json_schema_extra=_llm_hint(
"Use a concise action-oriented name; this influences the function "
"name shown to the agent."
),
)
description: Optional[str] = Field(
default=None,
description="Description shown to the agent when deciding whether to call it.",
json_schema_extra=_llm_hint(
"State exactly when the agent should call the tool and what result it gets."
),
)
category: ToolCategoryValue = Field(
default=ToolCategory.HTTP_API.value,
description="Tool category. Must match definition.type.",
)
icon: Optional[str] = Field(
default="globe", max_length=50, description="Lucide icon identifier."
)
icon_color: Optional[str] = Field(
default="#3B82F6", max_length=7, description="Hex color for the tool icon."
)
definition: ToolDefinition = Field(description="Typed tool definition.")
@model_validator(mode="before")
@classmethod
def default_category_from_definition(cls, data: Any) -> Any:
if not isinstance(data, dict):
return data
if data.get("category"):
return data
definition = data.get("definition")
if isinstance(definition, dict) and definition.get("type"):
return {**data, "category": definition["type"]}
return data
@field_validator("category")
@classmethod
def validate_category(cls, v: str) -> str:
valid_categories = [c.value for c in ToolCategory]
if v not in valid_categories:
raise ValueError(
f"Invalid category '{v}'. Must be one of: {', '.join(valid_categories)}"
)
return v
@model_validator(mode="after")
def validate_category_matches_definition(self) -> "CreateToolRequest":
definition_type = self.definition.type
if self.category != definition_type:
raise ValueError(
f"category '{self.category}' must match definition.type "
f"'{definition_type}'"
)
return self
class UpdateToolRequest(BaseModel):
"""Request schema for updating a reusable tool."""
name: Optional[str] = Field(default=None, max_length=255)
description: Optional[str] = None
icon: Optional[str] = Field(default=None, max_length=50)
icon_color: Optional[str] = Field(default=None, max_length=7)
definition: Optional[ToolDefinition] = None
status: Optional[str] = None
class CreatedByResponse(BaseModel):
"""Response schema for the user who created a tool."""
id: int
provider_id: str
class ToolResponse(BaseModel):
"""Response schema for a reusable tool."""
id: int
tool_uuid: str
name: str
description: Optional[str]
category: str
icon: Optional[str]
icon_color: Optional[str]
status: str
definition: Dict[str, Any]
created_at: datetime
updated_at: Optional[datetime]
created_by: Optional[CreatedByResponse] = None
model_config = ConfigDict(from_attributes=True)
class McpRefreshResponse(BaseModel):
"""Result of re-discovering an MCP server's tool catalog."""
tool_uuid: str
discovered_tools: list = Field(default_factory=list)
error: Optional[str] = None
class ToolTestRequest(BaseModel):
"""Request body for testing an HTTP API tool outside a live call."""
llm_params: Dict[str, Any] = Field(
default_factory=dict,
description="Values for parameters normally supplied by the model.",
)
preset_params: Dict[str, Any] = Field(
default_factory=dict,
description="Resolved values for parameters normally supplied from presets.",
)
class ToolTestResponse(BaseModel):
"""Result of testing an HTTP API tool."""
status: str
status_code: Optional[int] = None
data: Optional[Any] = None
error: Optional[str] = None
hint: Optional[str] = None
request_method: str
request_url: str
request_headers: Dict[str, str] = Field(default_factory=dict)
request_body: Optional[Dict[str, Any]] = None
request_params: Optional[Dict[str, Any]] = None
duration_ms: int