mirror of
https://github.com/dograh-hq/dograh.git
synced 2026-07-19 11:41:04 +02:00
* feat: add tool test panel for HTTP API tools
Lets developers run a saved HTTP API tool against its real endpoint
from the tool detail page, without needing a live call. Reuses the
production execute_http_tool path so test behavior matches call-time
behavior.
- New POST /tools/{tool_uuid}/test route
- Test panel with per-parameter typed inputs and auto-detected
context variable inputs (from preset parameter templates)
- Validate parameter name uniqueness on save, matching the existing
transferParameters check
- Fix stale FunctionCallsFromLLMInfoFrame import causing test
collection failures against pipecat-ai 1.5.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: revert local-venv pipecat drift fix, apply ruff import formatting
test_custom_tools.py and test_unregistered_function_call.py were edited
locally to drop FunctionCallsFromLLMInfoFrame after hitting an
ImportError — that error was from a stale local pipecat-ai package, not
a real drift. CI's pipecat build emits this frame and the test asserted
on it, so removing it broke test_llm_calls_custom_tool_handler and its
unregistered-call counterpart. Reverted both files to match main.
Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to
clear the drift-check job (split the aliased import into its own
`from ... import (...)` block, wrapped a long monkeypatch.setattr call).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec
pytest's default discovery matches any top-level test_* name in a test
module, including imported functions — importing the route handler as
`test_tool as test_tool_route` still matched the pattern, so pytest
tried to run it as a test and failed injecting fixtures for tool_uuid/
request/user. Renamed the alias to call_test_tool_route.
Also regenerated docs/api-reference/openapi.json for the new
POST /tools/{tool_uuid}/test route and its two schemas (couldn't run
the dump script locally — pipecat-ai version mismatch documented
separately — so hand-built the diff to exactly match FastAPI's
get_openapi() output format, verified against neighboring routes).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address cubic review findings on test panel
- Resolve dotted context-variable keys into nested objects before
posting the test request. render_template's get_nested_value walks
nested dicts, so a flat key like "runtime_configuration.realtime_model"
never matched — templates referencing nested context always resolved
to empty.
- Restrict isHttpApiTool to an explicit category equality check instead
of inferring it from exclusions. native/integration tools are
currently disabled in the create-tool UI so this wasn't reachable
today, but the exclusion list silently goes stale as new categories
are added.
- Stop showing a green success badge for non-2xx responses.
execute_http_tool returns status: "success" for any HTTP exchange
that completes, regardless of status code — only transport-level
errors (timeout, connection failure) get status: "error". The test
panel now checks status_code is in the 2xx range before treating the
call as a success.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address second round of cubic/greptile findings
- Guard setNestedValue against prototype-pollution keys (__proto__,
constructor, prototype) in the dotted context-var path before
traversing.
- Normalize status to "error" in the test route when the upstream
status_code is >= 400. execute_http_tool only distinguishes
transport-level failures (timeout, connection error) from
"success" — a completed 4xx/5xx exchange still came back as
"success" from the executor.
- Seed testArgValues defaults for number/boolean parameters via a
useEffect keyed on the parameters array, so a required number or
boolean field isn't silently omitted from the test request if the
user never touches its input.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: only seed test-arg defaults for required number/boolean params
Seeding optional number/boolean parameters silently changed the test
request — an optional boolean flag the tester never touched was sent
as true, which can flip upstream behavior unintentionally. Restrict
seeding to required parameters.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: match whitespace and fallback-filter syntax in context var detection
extractContextVars required an exact {{initial_context.foo}} with no
whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN
(and render_template) accepts {{ initial_context.foo }} and
{{initial_context.foo | fallback:value}}. A preset parameter saved with
either of those forms resolved fine in production but showed no input
in the test panel, so testing always sent it empty context.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add hint and request_* fields to ToolTestResponse
Extends ToolTestResponse with hint, request_method, request_url,
request_body, and request_params so the frontend can surface what was
actually sent and a human-readable hint about why a test call failed.
* feat(tool-test): add status-code hints and request_method/url/body/params to test_tool()
* style: ruff-format test_mcp_tool_route.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): wire hint banner and request block into result panel
Backend has returned hint/request_method/request_url/request_body/
request_params since d45ea851/60aaf31d but the frontend never
displayed them. Extends ToolTestResult with the new fields and renders
an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx)
plus a Request block above the response showing exactly what was sent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(tool-test): include resolved preset params in request_body/params
Found via live GET/POST testing: the Request preview showed only the
model-provided arguments, not what execute_http_tool actually sends.
execute_http_tool merges resolved_arguments = {**arguments,
**preset_arguments} before building the outbound body/params — preset
params (e.g. {{initial_context.metadata.channel}}) are invisible to
the model but still go out on the wire. The preview now mirrors that
merge via the same _resolve_preset_parameters helper, so a dev sees
exactly what was sent, not just what the model provided.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add generateSampleValue helper for sample-fill button
* feat(tool-test): add Fill sample values button for arguments and context vars
Moved generateSampleValue out of page.tsx into a sibling helpers module:
Next.js's typed-route checker rejects extra named exports on a page.tsx
file (tsc error TS2344 on .next/types), so the helper and its test import
now live in testPanelHelpers.ts instead.
* feat(tool-test): add JSON edit modal state and handlers
* feat(tool-test): collapsed preview + edit modal for object/array test parameters
* fix(tool-test): validate JSON on modal open, not just on edit
Opening the JSON edit modal on an untouched object/array param (no value
yet in testArgValues) loaded an empty draft with jsonEditError hardcoded
to null, so Save was enabled despite invalid JSON and silently no-op'd on
click. Now runs the same JSON.parse check used by the live textarea
validation when the modal opens.
* chore: regenerate openapi.json for ToolTestResponse hint/request_* fields
drift-check on PR #547 was failing because the earlier hint/request_method/
request_url/request_body/request_params fields added to ToolTestResponse
were never reflected in the dumped spec. Regenerated via
scripts.dump_docs_openapi.
* fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner
httpx raises a TypeError when a query param value is a dict/list, which
was silently caught and surfaced as a generic tool-execution error —
this is what actually broke test requests, not just the "[object
Object]" display. JSON-stringify object/array arguments before they
become query params, in both the live execute_http_tool() path and the
test route's request_params display shaping.
Also adds an unsaved-changes warning banner above Test Tool, shown
when the live form state diverges from the last-saved HTTP API config,
since Test Tool always runs the saved config.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(tool-test): keep empty request body in preview; normalize headers in snapshot
- POST/PUT/PATCH with no arguments now shows `{}` in the request body
preview instead of null — matches what execute_http_tool actually sends
over the wire (json={})
- buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a
deduped key→value map before serializing, matching the shape saved to
the backend; duplicate header keys no longer cause a false unsaved-
changes warning
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(tool-test): update assertion for empty POST body preview
test_tool_test_no_arguments_leaves_body_and_params_none expected
request_body=None for a POST with no arguments. The fix to preserve {}
in the preview makes request_body={} the correct assertion.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tools): refine HTTP tool testing
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
386 lines
13 KiB
Python
386 lines
13 KiB
Python
"""Custom tool execution for user-defined HTTP API tools."""
|
|
|
|
import json
|
|
import re
|
|
from typing import Any, Dict, Optional
|
|
|
|
import httpx
|
|
from loguru import logger
|
|
|
|
from api.db import db_client
|
|
from api.services.configuration.masking import mask_key
|
|
from api.utils.credential_auth import build_auth_header
|
|
from api.utils.template_renderer import render_template
|
|
|
|
# Map tool parameter types to JSON schema types
|
|
TYPE_MAP = {
|
|
"string": "string",
|
|
"number": "number",
|
|
"boolean": "boolean",
|
|
"object": "object",
|
|
"array": "array",
|
|
}
|
|
|
|
|
|
def serialize_query_params(arguments: Dict[str, Any]) -> Dict[str, Any]:
|
|
"""JSON-stringify dict/list values so they're safe to pass as query params.
|
|
|
|
httpx (and query strings in general) only support primitive param values.
|
|
Object/array-typed tool arguments must be serialized before going out as
|
|
GET/DELETE query params, otherwise httpx raises a TypeError.
|
|
"""
|
|
return {
|
|
k: json.dumps(v) if isinstance(v, (dict, list)) else v
|
|
for k, v in arguments.items()
|
|
}
|
|
|
|
|
|
def tool_to_function_schema(tool: Any) -> Dict[str, Any]:
|
|
"""Convert a ToolModel to an LLM function schema.
|
|
|
|
Args:
|
|
tool: ToolModel instance with name, description, and definition
|
|
|
|
Returns:
|
|
Function schema dict compatible with OpenAI/Anthropic function calling
|
|
"""
|
|
definition = tool.definition or {}
|
|
config = definition.get("config", {})
|
|
parameters = config.get("parameters", []) or []
|
|
if (
|
|
definition.get("type") == "transfer_call"
|
|
and config.get("destination_source", "static") != "dynamic"
|
|
):
|
|
parameters = []
|
|
elif (
|
|
definition.get("type") == "transfer_call"
|
|
and config.get("destination_source", "static") == "dynamic"
|
|
):
|
|
resolver = config.get("resolver")
|
|
if isinstance(resolver, dict):
|
|
parameters = resolver.get("parameters", []) or []
|
|
else:
|
|
parameters = []
|
|
|
|
# Build properties and required list from parameters
|
|
properties = {}
|
|
required = []
|
|
|
|
for param in parameters:
|
|
param_name = param.get("name", "")
|
|
param_type = param.get("type", "string")
|
|
param_desc = param.get("description", "")
|
|
param_required = param.get("required", True)
|
|
|
|
if not param_name:
|
|
continue
|
|
|
|
schema_type = TYPE_MAP.get(param_type, "string")
|
|
if schema_type == "object":
|
|
properties[param_name] = {
|
|
"type": "object",
|
|
"additionalProperties": True,
|
|
"description": param_desc,
|
|
}
|
|
elif schema_type == "array":
|
|
properties[param_name] = {
|
|
"type": "array",
|
|
"items": {},
|
|
"description": param_desc,
|
|
}
|
|
else:
|
|
properties[param_name] = {
|
|
"type": schema_type,
|
|
"description": param_desc,
|
|
}
|
|
|
|
if param_required:
|
|
required.append(param_name)
|
|
|
|
# If this is an end_call tool with endCallReason enabled, add a required 'reason' parameter
|
|
if definition.get("type") == "end_call" and config.get("endCallReason", False):
|
|
default_description = (
|
|
"The reason for ending the call (e.g., 'voicemail_detected', "
|
|
"'issue_resolved', 'customer_requested')"
|
|
)
|
|
properties["reason"] = {
|
|
"type": "string",
|
|
"description": config.get("endCallReasonDescription")
|
|
or default_description,
|
|
}
|
|
required.append("reason")
|
|
|
|
# Sanitize tool name for function name (lowercase, underscores only)
|
|
function_name = re.sub(r"[^a-z0-9_]", "_", tool.name.lower())
|
|
# Remove consecutive underscores and trim
|
|
function_name = re.sub(r"_+", "_", function_name).strip("_")
|
|
|
|
return {
|
|
"type": "function",
|
|
"function": {
|
|
"name": function_name,
|
|
"description": tool.description or f"Execute {tool.name} tool",
|
|
"parameters": {
|
|
"type": "object",
|
|
"properties": properties,
|
|
"required": required,
|
|
},
|
|
},
|
|
"_tool_uuid": tool.tool_uuid,
|
|
}
|
|
|
|
|
|
def _coerce_parameter_value(value: Any, param_type: str) -> Any:
|
|
"""Coerce a rendered preset parameter into the configured JSON type."""
|
|
|
|
if value is None:
|
|
return None
|
|
|
|
if param_type == "string":
|
|
if isinstance(value, str):
|
|
return value
|
|
if isinstance(value, (dict, list)):
|
|
return json.dumps(value)
|
|
return str(value)
|
|
|
|
if param_type == "number":
|
|
if isinstance(value, (int, float)) and not isinstance(value, bool):
|
|
return value
|
|
|
|
rendered = str(value).strip()
|
|
if rendered == "":
|
|
return None
|
|
|
|
if re.fullmatch(r"[-+]?\d+", rendered):
|
|
return int(rendered)
|
|
|
|
return float(rendered)
|
|
|
|
if param_type == "boolean":
|
|
if isinstance(value, bool):
|
|
return value
|
|
|
|
if isinstance(value, (int, float)):
|
|
return bool(value)
|
|
|
|
rendered = str(value).strip().lower()
|
|
if rendered in {"true", "1", "yes", "y", "on"}:
|
|
return True
|
|
if rendered in {"false", "0", "no", "n", "off"}:
|
|
return False
|
|
|
|
raise ValueError(f"Cannot convert '{value}' to boolean")
|
|
|
|
if param_type == "object":
|
|
if isinstance(value, str):
|
|
try:
|
|
value = json.loads(value)
|
|
except json.JSONDecodeError as exc:
|
|
raise ValueError(f"Cannot convert '{value}' to object") from exc
|
|
if isinstance(value, dict):
|
|
return value
|
|
raise ValueError(f"Cannot convert '{value}' to object")
|
|
|
|
if param_type == "array":
|
|
if isinstance(value, str):
|
|
try:
|
|
value = json.loads(value)
|
|
except json.JSONDecodeError as exc:
|
|
raise ValueError(f"Cannot convert '{value}' to array") from exc
|
|
if isinstance(value, list):
|
|
return value
|
|
raise ValueError(f"Cannot convert '{value}' to array")
|
|
|
|
return value
|
|
|
|
|
|
def _resolve_preset_parameters(
|
|
config: Dict[str, Any],
|
|
call_context_vars: Optional[Dict[str, Any]],
|
|
gathered_context_vars: Optional[Dict[str, Any]],
|
|
) -> Dict[str, Any]:
|
|
"""Resolve fixed/template-backed parameters before executing the HTTP request."""
|
|
|
|
preset_parameters = config.get("preset_parameters", []) or []
|
|
if not preset_parameters:
|
|
return {}
|
|
|
|
initial_context = dict(call_context_vars or {})
|
|
render_context: Dict[str, Any] = {
|
|
**initial_context,
|
|
"initial_context": initial_context,
|
|
"gathered_context": dict(gathered_context_vars or {}),
|
|
}
|
|
|
|
resolved: Dict[str, Any] = {}
|
|
for param in preset_parameters:
|
|
param_name = (param.get("name") or "").strip()
|
|
if not param_name:
|
|
continue
|
|
|
|
rendered = render_template(param.get("value_template", ""), render_context)
|
|
if rendered in (None, ""):
|
|
if param.get("required", True):
|
|
raise ValueError(
|
|
f"Preset parameter '{param_name}' resolved to an empty value"
|
|
)
|
|
continue
|
|
|
|
resolved[param_name] = _coerce_parameter_value(
|
|
rendered, param.get("type", "string")
|
|
)
|
|
|
|
return resolved
|
|
|
|
|
|
async def execute_http_tool(
|
|
tool: Any,
|
|
arguments: Dict[str, Any],
|
|
call_context_vars: Optional[Dict[str, Any]] = None,
|
|
gathered_context_vars: Optional[Dict[str, Any]] = None,
|
|
preset_params: Optional[Dict[str, Any]] = None,
|
|
organization_id: Optional[int] = None,
|
|
include_request_headers: bool = False,
|
|
) -> Dict[str, Any]:
|
|
"""Execute an HTTP API tool.
|
|
|
|
Args:
|
|
tool: ToolModel instance
|
|
arguments: Arguments passed by the LLM (parameter name -> value)
|
|
call_context_vars: Initial context variables available at runtime
|
|
gathered_context_vars: Variables extracted during the conversation
|
|
preset_params: Pre-resolved preset parameter values. Used by the test
|
|
endpoint; live calls omit this so configured templates are resolved.
|
|
organization_id: Organization ID for credential lookup
|
|
include_request_headers: Include a client-safe header preview in the result.
|
|
Headers supplied by a stored credential are masked.
|
|
|
|
Returns:
|
|
Result dict with response data or error
|
|
"""
|
|
definition = tool.definition or {}
|
|
config = definition.get("config", {})
|
|
|
|
# Get HTTP method and URL
|
|
method = config.get("method", "POST").upper()
|
|
url = config.get("url", "")
|
|
|
|
# Get headers from config
|
|
headers = dict(config.get("headers", {}) or {})
|
|
|
|
# Add auth header if credential is configured. Keep track of which headers
|
|
# came from the credential so only those values are masked in test previews.
|
|
credential_headers: Dict[str, str] = {}
|
|
credential_uuid = config.get("credential_uuid")
|
|
if credential_uuid and organization_id:
|
|
try:
|
|
credential = await db_client.get_credential_by_uuid(
|
|
credential_uuid, organization_id
|
|
)
|
|
if credential:
|
|
credential_headers = build_auth_header(credential)
|
|
headers.update(credential_headers)
|
|
logger.debug(f"Applied credential '{credential.name}' to tool request")
|
|
else:
|
|
logger.warning(
|
|
f"Credential {credential_uuid} not found for tool '{tool.name}'"
|
|
)
|
|
except Exception as e:
|
|
logger.error(f"Failed to fetch credential for tool '{tool.name}': {e}")
|
|
|
|
request_headers: Dict[str, str] = {}
|
|
if include_request_headers:
|
|
request_headers = {str(name): str(value) for name, value in headers.items()}
|
|
for header_name, header_value in credential_headers.items():
|
|
request_headers[header_name] = mask_key(str(header_value))
|
|
|
|
def build_result(result: Dict[str, Any]) -> Dict[str, Any]:
|
|
if include_request_headers:
|
|
return {**result, "request_headers": request_headers}
|
|
return result
|
|
|
|
# Get timeout
|
|
timeout_ms = config.get("timeout_ms", 5000)
|
|
timeout_seconds = timeout_ms / 1000
|
|
|
|
if preset_params is None:
|
|
try:
|
|
preset_arguments = _resolve_preset_parameters(
|
|
config, call_context_vars, gathered_context_vars
|
|
)
|
|
except ValueError as e:
|
|
logger.error(f"Custom tool '{tool.name}' preset parameter error: {e}")
|
|
return build_result({"status": "error", "error": str(e)})
|
|
else:
|
|
preset_arguments = dict(preset_params)
|
|
|
|
resolved_arguments = {**(arguments or {}), **preset_arguments}
|
|
|
|
# Build request: JSON body for POST/PUT/PATCH, query params for GET/DELETE
|
|
body = None
|
|
params = None
|
|
if method in ("POST", "PUT", "PATCH"):
|
|
body = resolved_arguments
|
|
elif method in ("GET", "DELETE") and resolved_arguments:
|
|
params = serialize_query_params(resolved_arguments)
|
|
|
|
logger.info(
|
|
f"Executing custom tool '{tool.name}' ({tool.tool_uuid}): {method} {url}"
|
|
)
|
|
if preset_arguments:
|
|
logger.debug(
|
|
f"Resolved preset parameters for '{tool.name}': {list(preset_arguments.keys())}"
|
|
)
|
|
logger.debug(f"Request body: {body}, params: {params}")
|
|
|
|
try:
|
|
async with httpx.AsyncClient(timeout=timeout_seconds) as client:
|
|
response = await client.request(
|
|
method=method,
|
|
url=url,
|
|
headers=headers,
|
|
json=body,
|
|
params=params,
|
|
)
|
|
|
|
# Try to parse JSON response
|
|
try:
|
|
response_data = response.json()
|
|
except Exception:
|
|
response_data = {"raw_response": response.text}
|
|
|
|
result = {
|
|
"status": "success",
|
|
"status_code": response.status_code,
|
|
"data": response_data,
|
|
}
|
|
|
|
logger.debug(
|
|
f"Custom tool '{tool.name}' completed with status {response.status_code}"
|
|
)
|
|
return build_result(result)
|
|
|
|
except httpx.TimeoutException:
|
|
logger.error(f"Custom tool '{tool.name}' timed out after {timeout_seconds}s")
|
|
return build_result(
|
|
{
|
|
"status": "error",
|
|
"error": f"Request timed out after {timeout_seconds} seconds",
|
|
}
|
|
)
|
|
except httpx.RequestError as e:
|
|
logger.error(f"Custom tool '{tool.name}' request failed: {e}")
|
|
return build_result(
|
|
{
|
|
"status": "error",
|
|
"error": f"Request failed: {str(e)}",
|
|
}
|
|
)
|
|
except Exception as e:
|
|
logger.error(f"Custom tool '{tool.name}' execution failed: {e}")
|
|
return build_result(
|
|
{
|
|
"status": "error",
|
|
"error": f"Tool execution failed: {str(e)}",
|
|
}
|
|
)
|