dograh/api/services/workflow/tools/custom_tool.py

387 lines
13 KiB
Python
Raw Permalink Normal View History

"""Custom tool execution for user-defined HTTP API tools."""
import json
import re
from typing import Any, Dict, Optional
import httpx
from loguru import logger
from api.db import db_client
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
from api.services.configuration.masking import mask_key
from api.utils.credential_auth import build_auth_header
from api.utils.template_renderer import render_template
# Map tool parameter types to JSON schema types
TYPE_MAP = {
"string": "string",
"number": "number",
"boolean": "boolean",
"object": "object",
"array": "array",
}
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
def serialize_query_params(arguments: Dict[str, Any]) -> Dict[str, Any]:
"""JSON-stringify dict/list values so they're safe to pass as query params.
httpx (and query strings in general) only support primitive param values.
Object/array-typed tool arguments must be serialized before going out as
GET/DELETE query params, otherwise httpx raises a TypeError.
"""
return {
k: json.dumps(v) if isinstance(v, (dict, list)) else v
for k, v in arguments.items()
}
def tool_to_function_schema(tool: Any) -> Dict[str, Any]:
"""Convert a ToolModel to an LLM function schema.
Args:
tool: ToolModel instance with name, description, and definition
Returns:
Function schema dict compatible with OpenAI/Anthropic function calling
"""
definition = tool.definition or {}
config = definition.get("config", {})
parameters = config.get("parameters", []) or []
if (
definition.get("type") == "transfer_call"
and config.get("destination_source", "static") != "dynamic"
):
parameters = []
elif (
definition.get("type") == "transfer_call"
and config.get("destination_source", "static") == "dynamic"
):
resolver = config.get("resolver")
if isinstance(resolver, dict):
parameters = resolver.get("parameters", []) or []
else:
parameters = []
# Build properties and required list from parameters
properties = {}
required = []
for param in parameters:
param_name = param.get("name", "")
param_type = param.get("type", "string")
param_desc = param.get("description", "")
param_required = param.get("required", True)
if not param_name:
continue
schema_type = TYPE_MAP.get(param_type, "string")
if schema_type == "object":
properties[param_name] = {
"type": "object",
"additionalProperties": True,
"description": param_desc,
}
elif schema_type == "array":
properties[param_name] = {
"type": "array",
"items": {},
"description": param_desc,
}
else:
properties[param_name] = {
"type": schema_type,
"description": param_desc,
}
if param_required:
required.append(param_name)
# If this is an end_call tool with endCallReason enabled, add a required 'reason' parameter
if definition.get("type") == "end_call" and config.get("endCallReason", False):
default_description = (
"The reason for ending the call (e.g., 'voicemail_detected', "
"'issue_resolved', 'customer_requested')"
)
properties["reason"] = {
"type": "string",
"description": config.get("endCallReasonDescription")
or default_description,
}
required.append("reason")
# Sanitize tool name for function name (lowercase, underscores only)
function_name = re.sub(r"[^a-z0-9_]", "_", tool.name.lower())
# Remove consecutive underscores and trim
function_name = re.sub(r"_+", "_", function_name).strip("_")
return {
"type": "function",
"function": {
"name": function_name,
"description": tool.description or f"Execute {tool.name} tool",
"parameters": {
"type": "object",
"properties": properties,
"required": required,
},
},
"_tool_uuid": tool.tool_uuid,
}
def _coerce_parameter_value(value: Any, param_type: str) -> Any:
"""Coerce a rendered preset parameter into the configured JSON type."""
if value is None:
return None
if param_type == "string":
if isinstance(value, str):
return value
if isinstance(value, (dict, list)):
return json.dumps(value)
return str(value)
if param_type == "number":
if isinstance(value, (int, float)) and not isinstance(value, bool):
return value
rendered = str(value).strip()
if rendered == "":
return None
if re.fullmatch(r"[-+]?\d+", rendered):
return int(rendered)
return float(rendered)
if param_type == "boolean":
if isinstance(value, bool):
return value
if isinstance(value, (int, float)):
return bool(value)
rendered = str(value).strip().lower()
if rendered in {"true", "1", "yes", "y", "on"}:
return True
if rendered in {"false", "0", "no", "n", "off"}:
return False
raise ValueError(f"Cannot convert '{value}' to boolean")
if param_type == "object":
if isinstance(value, str):
try:
value = json.loads(value)
except json.JSONDecodeError as exc:
raise ValueError(f"Cannot convert '{value}' to object") from exc
if isinstance(value, dict):
return value
raise ValueError(f"Cannot convert '{value}' to object")
if param_type == "array":
if isinstance(value, str):
try:
value = json.loads(value)
except json.JSONDecodeError as exc:
raise ValueError(f"Cannot convert '{value}' to array") from exc
if isinstance(value, list):
return value
raise ValueError(f"Cannot convert '{value}' to array")
return value
def _resolve_preset_parameters(
config: Dict[str, Any],
call_context_vars: Optional[Dict[str, Any]],
gathered_context_vars: Optional[Dict[str, Any]],
) -> Dict[str, Any]:
"""Resolve fixed/template-backed parameters before executing the HTTP request."""
preset_parameters = config.get("preset_parameters", []) or []
if not preset_parameters:
return {}
initial_context = dict(call_context_vars or {})
render_context: Dict[str, Any] = {
**initial_context,
"initial_context": initial_context,
"gathered_context": dict(gathered_context_vars or {}),
}
resolved: Dict[str, Any] = {}
for param in preset_parameters:
param_name = (param.get("name") or "").strip()
if not param_name:
continue
rendered = render_template(param.get("value_template", ""), render_context)
if rendered in (None, ""):
if param.get("required", True):
raise ValueError(
f"Preset parameter '{param_name}' resolved to an empty value"
)
continue
resolved[param_name] = _coerce_parameter_value(
rendered, param.get("type", "string")
)
return resolved
async def execute_http_tool(
tool: Any,
arguments: Dict[str, Any],
call_context_vars: Optional[Dict[str, Any]] = None,
gathered_context_vars: Optional[Dict[str, Any]] = None,
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
preset_params: Optional[Dict[str, Any]] = None,
organization_id: Optional[int] = None,
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
include_request_headers: bool = False,
) -> Dict[str, Any]:
"""Execute an HTTP API tool.
Args:
tool: ToolModel instance
arguments: Arguments passed by the LLM (parameter name -> value)
call_context_vars: Initial context variables available at runtime
gathered_context_vars: Variables extracted during the conversation
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
preset_params: Pre-resolved preset parameter values. Used by the test
endpoint; live calls omit this so configured templates are resolved.
organization_id: Organization ID for credential lookup
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
include_request_headers: Include a client-safe header preview in the result.
Headers supplied by a stored credential are masked.
Returns:
Result dict with response data or error
"""
definition = tool.definition or {}
config = definition.get("config", {})
# Get HTTP method and URL
method = config.get("method", "POST").upper()
url = config.get("url", "")
# Get headers from config
headers = dict(config.get("headers", {}) or {})
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
# Add auth header if credential is configured. Keep track of which headers
# came from the credential so only those values are masked in test previews.
credential_headers: Dict[str, str] = {}
credential_uuid = config.get("credential_uuid")
if credential_uuid and organization_id:
try:
credential = await db_client.get_credential_by_uuid(
credential_uuid, organization_id
)
if credential:
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
credential_headers = build_auth_header(credential)
headers.update(credential_headers)
logger.debug(f"Applied credential '{credential.name}' to tool request")
else:
logger.warning(
f"Credential {credential_uuid} not found for tool '{tool.name}'"
)
except Exception as e:
logger.error(f"Failed to fetch credential for tool '{tool.name}': {e}")
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
request_headers: Dict[str, str] = {}
if include_request_headers:
request_headers = {str(name): str(value) for name, value in headers.items()}
for header_name, header_value in credential_headers.items():
request_headers[header_name] = mask_key(str(header_value))
def build_result(result: Dict[str, Any]) -> Dict[str, Any]:
if include_request_headers:
return {**result, "request_headers": request_headers}
return result
# Get timeout
timeout_ms = config.get("timeout_ms", 5000)
timeout_seconds = timeout_ms / 1000
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
if preset_params is None:
try:
preset_arguments = _resolve_preset_parameters(
config, call_context_vars, gathered_context_vars
)
except ValueError as e:
logger.error(f"Custom tool '{tool.name}' preset parameter error: {e}")
return build_result({"status": "error", "error": str(e)})
else:
preset_arguments = dict(preset_params)
resolved_arguments = {**(arguments or {}), **preset_arguments}
# Build request: JSON body for POST/PUT/PATCH, query params for GET/DELETE
body = None
params = None
if method in ("POST", "PUT", "PATCH"):
body = resolved_arguments
elif method in ("GET", "DELETE") and resolved_arguments:
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
params = serialize_query_params(resolved_arguments)
logger.info(
f"Executing custom tool '{tool.name}' ({tool.tool_uuid}): {method} {url}"
)
if preset_arguments:
logger.debug(
f"Resolved preset parameters for '{tool.name}': {list(preset_arguments.keys())}"
)
logger.debug(f"Request body: {body}, params: {params}")
try:
async with httpx.AsyncClient(timeout=timeout_seconds) as client:
response = await client.request(
method=method,
url=url,
headers=headers,
json=body,
params=params,
)
# Try to parse JSON response
try:
response_data = response.json()
except Exception:
response_data = {"raw_response": response.text}
result = {
"status": "success",
"status_code": response.status_code,
"data": response_data,
}
logger.debug(
f"Custom tool '{tool.name}' completed with status {response.status_code}"
)
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
return build_result(result)
except httpx.TimeoutException:
logger.error(f"Custom tool '{tool.name}' timed out after {timeout_seconds}s")
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
return build_result(
{
"status": "error",
"error": f"Request timed out after {timeout_seconds} seconds",
}
)
except httpx.RequestError as e:
logger.error(f"Custom tool '{tool.name}' request failed: {e}")
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
return build_result(
{
"status": "error",
"error": f"Request failed: {str(e)}",
}
)
except Exception as e:
logger.error(f"Custom tool '{tool.name}' execution failed: {e}")
feat: add tool test panel for HTTP API tools (#547) * feat: add tool test panel for HTTP API tools Lets developers run a saved HTTP API tool against its real endpoint from the tool detail page, without needing a live call. Reuses the production execute_http_tool path so test behavior matches call-time behavior. - New POST /tools/{tool_uuid}/test route - Test panel with per-parameter typed inputs and auto-detected context variable inputs (from preset parameter templates) - Validate parameter name uniqueness on save, matching the existing transferParameters check - Fix stale FunctionCallsFromLLMInfoFrame import causing test collection failures against pipecat-ai 1.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: revert local-venv pipecat drift fix, apply ruff import formatting test_custom_tools.py and test_unregistered_function_call.py were edited locally to drop FunctionCallsFromLLMInfoFrame after hitting an ImportError — that error was from a stale local pipecat-ai package, not a real drift. CI's pipecat build emits this frame and the test asserted on it, so removing it broke test_llm_calls_custom_tool_handler and its unregistered-call counterpart. Reverted both files to match main. Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to clear the drift-check job (split the aliased import into its own `from ... import (...)` block, wrapped a long monkeypatch.setattr call). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec pytest's default discovery matches any top-level test_* name in a test module, including imported functions — importing the route handler as `test_tool as test_tool_route` still matched the pattern, so pytest tried to run it as a test and failed injecting fixtures for tool_uuid/ request/user. Renamed the alias to call_test_tool_route. Also regenerated docs/api-reference/openapi.json for the new POST /tools/{tool_uuid}/test route and its two schemas (couldn't run the dump script locally — pipecat-ai version mismatch documented separately — so hand-built the diff to exactly match FastAPI's get_openapi() output format, verified against neighboring routes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address cubic review findings on test panel - Resolve dotted context-variable keys into nested objects before posting the test request. render_template's get_nested_value walks nested dicts, so a flat key like "runtime_configuration.realtime_model" never matched — templates referencing nested context always resolved to empty. - Restrict isHttpApiTool to an explicit category equality check instead of inferring it from exclusions. native/integration tools are currently disabled in the create-tool UI so this wasn't reachable today, but the exclusion list silently goes stale as new categories are added. - Stop showing a green success badge for non-2xx responses. execute_http_tool returns status: "success" for any HTTP exchange that completes, regardless of status code — only transport-level errors (timeout, connection failure) get status: "error". The test panel now checks status_code is in the 2xx range before treating the call as a success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address second round of cubic/greptile findings - Guard setNestedValue against prototype-pollution keys (__proto__, constructor, prototype) in the dotted context-var path before traversing. - Normalize status to "error" in the test route when the upstream status_code is >= 400. execute_http_tool only distinguishes transport-level failures (timeout, connection error) from "success" — a completed 4xx/5xx exchange still came back as "success" from the executor. - Seed testArgValues defaults for number/boolean parameters via a useEffect keyed on the parameters array, so a required number or boolean field isn't silently omitted from the test request if the user never touches its input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: only seed test-arg defaults for required number/boolean params Seeding optional number/boolean parameters silently changed the test request — an optional boolean flag the tester never touched was sent as true, which can flip upstream behavior unintentionally. Restrict seeding to required parameters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: match whitespace and fallback-filter syntax in context var detection extractContextVars required an exact {{initial_context.foo}} with no whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN (and render_template) accepts {{ initial_context.foo }} and {{initial_context.foo | fallback:value}}. A preset parameter saved with either of those forms resolved fine in production but showed no input in the test panel, so testing always sent it empty context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add hint and request_* fields to ToolTestResponse Extends ToolTestResponse with hint, request_method, request_url, request_body, and request_params so the frontend can surface what was actually sent and a human-readable hint about why a test call failed. * feat(tool-test): add status-code hints and request_method/url/body/params to test_tool() * style: ruff-format test_mcp_tool_route.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): wire hint banner and request block into result panel Backend has returned hint/request_method/request_url/request_body/ request_params since d45ea851/60aaf31d but the frontend never displayed them. Extends ToolTestResult with the new fields and renders an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx) plus a Request block above the response showing exactly what was sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tool-test): include resolved preset params in request_body/params Found via live GET/POST testing: the Request preview showed only the model-provided arguments, not what execute_http_tool actually sends. execute_http_tool merges resolved_arguments = {**arguments, **preset_arguments} before building the outbound body/params — preset params (e.g. {{initial_context.metadata.channel}}) are invisible to the model but still go out on the wire. The preview now mirrors that merge via the same _resolve_preset_parameters helper, so a dev sees exactly what was sent, not just what the model provided. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tool-test): add generateSampleValue helper for sample-fill button * feat(tool-test): add Fill sample values button for arguments and context vars Moved generateSampleValue out of page.tsx into a sibling helpers module: Next.js's typed-route checker rejects extra named exports on a page.tsx file (tsc error TS2344 on .next/types), so the helper and its test import now live in testPanelHelpers.ts instead. * feat(tool-test): add JSON edit modal state and handlers * feat(tool-test): collapsed preview + edit modal for object/array test parameters * fix(tool-test): validate JSON on modal open, not just on edit Opening the JSON edit modal on an untouched object/array param (no value yet in testArgValues) loaded an empty draft with jsonEditError hardcoded to null, so Save was enabled despite invalid JSON and silently no-op'd on click. Now runs the same JSON.parse check used by the live textarea validation when the modal opens. * chore: regenerate openapi.json for ToolTestResponse hint/request_* fields drift-check on PR #547 was failing because the earlier hint/request_method/ request_url/request_body/request_params fields added to ToolTestResponse were never reflected in the dumped spec. Regenerated via scripts.dump_docs_openapi. * fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner httpx raises a TypeError when a query param value is a dict/list, which was silently caught and surfaced as a generic tool-execution error — this is what actually broke test requests, not just the "[object Object]" display. JSON-stringify object/array arguments before they become query params, in both the live execute_http_tool() path and the test route's request_params display shaping. Also adds an unsaved-changes warning banner above Test Tool, shown when the live form state diverges from the last-saved HTTP API config, since Test Tool always runs the saved config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-test): keep empty request body in preview; normalize headers in snapshot - POST/PUT/PATCH with no arguments now shows `{}` in the request body preview instead of null — matches what execute_http_tool actually sends over the wire (json={}) - buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a deduped key→value map before serializing, matching the shape saved to the backend; duplicate header keys no longer cause a false unsaved- changes warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(tool-test): update assertion for empty POST body preview test_tool_test_no_arguments_leaves_body_and_params_none expected request_body=None for a POST with no arguments. The fix to preserve {} in the preview makes request_body={} the correct assertion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tools): refine HTTP tool testing --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
return build_result(
{
"status": "error",
"error": f"Tool execution failed: {str(e)}",
}
)