2026-05-19 07:40:00 -03:00
|
|
|
"""Route-level tests for the MCP tool definition schema.
|
|
|
|
|
|
|
|
|
|
These tests exercise the Pydantic request models (CreateToolRequest /
|
|
|
|
|
UpdateToolRequest) to catch schema gaps at the route/request-model layer —
|
|
|
|
|
the layer where the pre-fix defect lived (HTTP 422 on every MCP tool
|
|
|
|
|
creation attempt).
|
|
|
|
|
|
|
|
|
|
Test coverage:
|
|
|
|
|
- CreateToolRequest validates a valid MCP definition (was 422 before Part A).
|
|
|
|
|
- UpdateToolRequest validates a valid MCP definition.
|
|
|
|
|
- Invalid MCP bodies are rejected (ftp:// url, missing url).
|
|
|
|
|
- Round-trip: validated definition dict passes through validate_mcp_definition
|
|
|
|
|
unchanged, proving the request schema and call-time validator agree.
|
|
|
|
|
- Full HTTP round-trip via the ASGI test client (POST /api/v1/tools/).
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
from __future__ import annotations
|
|
|
|
|
|
2026-05-31 16:50:44 +05:30
|
|
|
from unittest.mock import AsyncMock, MagicMock
|
|
|
|
|
|
2026-05-19 07:40:00 -03:00
|
|
|
import pytest
|
2026-05-31 16:50:44 +05:30
|
|
|
from fastapi import HTTPException
|
2026-05-19 07:40:00 -03:00
|
|
|
from pydantic import ValidationError
|
|
|
|
|
|
2026-05-31 16:50:44 +05:30
|
|
|
from api.routes.tool import (
|
|
|
|
|
CreateToolRequest,
|
|
|
|
|
McpToolConfig,
|
|
|
|
|
McpToolDefinition,
|
feat: add tool test panel for HTTP API tools (#547)
* feat: add tool test panel for HTTP API tools
Lets developers run a saved HTTP API tool against its real endpoint
from the tool detail page, without needing a live call. Reuses the
production execute_http_tool path so test behavior matches call-time
behavior.
- New POST /tools/{tool_uuid}/test route
- Test panel with per-parameter typed inputs and auto-detected
context variable inputs (from preset parameter templates)
- Validate parameter name uniqueness on save, matching the existing
transferParameters check
- Fix stale FunctionCallsFromLLMInfoFrame import causing test
collection failures against pipecat-ai 1.5.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: revert local-venv pipecat drift fix, apply ruff import formatting
test_custom_tools.py and test_unregistered_function_call.py were edited
locally to drop FunctionCallsFromLLMInfoFrame after hitting an
ImportError — that error was from a stale local pipecat-ai package, not
a real drift. CI's pipecat build emits this frame and the test asserted
on it, so removing it broke test_llm_calls_custom_tool_handler and its
unregistered-call counterpart. Reverted both files to match main.
Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to
clear the drift-check job (split the aliased import into its own
`from ... import (...)` block, wrapped a long monkeypatch.setattr call).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec
pytest's default discovery matches any top-level test_* name in a test
module, including imported functions — importing the route handler as
`test_tool as test_tool_route` still matched the pattern, so pytest
tried to run it as a test and failed injecting fixtures for tool_uuid/
request/user. Renamed the alias to call_test_tool_route.
Also regenerated docs/api-reference/openapi.json for the new
POST /tools/{tool_uuid}/test route and its two schemas (couldn't run
the dump script locally — pipecat-ai version mismatch documented
separately — so hand-built the diff to exactly match FastAPI's
get_openapi() output format, verified against neighboring routes).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address cubic review findings on test panel
- Resolve dotted context-variable keys into nested objects before
posting the test request. render_template's get_nested_value walks
nested dicts, so a flat key like "runtime_configuration.realtime_model"
never matched — templates referencing nested context always resolved
to empty.
- Restrict isHttpApiTool to an explicit category equality check instead
of inferring it from exclusions. native/integration tools are
currently disabled in the create-tool UI so this wasn't reachable
today, but the exclusion list silently goes stale as new categories
are added.
- Stop showing a green success badge for non-2xx responses.
execute_http_tool returns status: "success" for any HTTP exchange
that completes, regardless of status code — only transport-level
errors (timeout, connection failure) get status: "error". The test
panel now checks status_code is in the 2xx range before treating the
call as a success.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address second round of cubic/greptile findings
- Guard setNestedValue against prototype-pollution keys (__proto__,
constructor, prototype) in the dotted context-var path before
traversing.
- Normalize status to "error" in the test route when the upstream
status_code is >= 400. execute_http_tool only distinguishes
transport-level failures (timeout, connection error) from
"success" — a completed 4xx/5xx exchange still came back as
"success" from the executor.
- Seed testArgValues defaults for number/boolean parameters via a
useEffect keyed on the parameters array, so a required number or
boolean field isn't silently omitted from the test request if the
user never touches its input.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: only seed test-arg defaults for required number/boolean params
Seeding optional number/boolean parameters silently changed the test
request — an optional boolean flag the tester never touched was sent
as true, which can flip upstream behavior unintentionally. Restrict
seeding to required parameters.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: match whitespace and fallback-filter syntax in context var detection
extractContextVars required an exact {{initial_context.foo}} with no
whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN
(and render_template) accepts {{ initial_context.foo }} and
{{initial_context.foo | fallback:value}}. A preset parameter saved with
either of those forms resolved fine in production but showed no input
in the test panel, so testing always sent it empty context.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add hint and request_* fields to ToolTestResponse
Extends ToolTestResponse with hint, request_method, request_url,
request_body, and request_params so the frontend can surface what was
actually sent and a human-readable hint about why a test call failed.
* feat(tool-test): add status-code hints and request_method/url/body/params to test_tool()
* style: ruff-format test_mcp_tool_route.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): wire hint banner and request block into result panel
Backend has returned hint/request_method/request_url/request_body/
request_params since d45ea851/60aaf31d but the frontend never
displayed them. Extends ToolTestResult with the new fields and renders
an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx)
plus a Request block above the response showing exactly what was sent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(tool-test): include resolved preset params in request_body/params
Found via live GET/POST testing: the Request preview showed only the
model-provided arguments, not what execute_http_tool actually sends.
execute_http_tool merges resolved_arguments = {**arguments,
**preset_arguments} before building the outbound body/params — preset
params (e.g. {{initial_context.metadata.channel}}) are invisible to
the model but still go out on the wire. The preview now mirrors that
merge via the same _resolve_preset_parameters helper, so a dev sees
exactly what was sent, not just what the model provided.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add generateSampleValue helper for sample-fill button
* feat(tool-test): add Fill sample values button for arguments and context vars
Moved generateSampleValue out of page.tsx into a sibling helpers module:
Next.js's typed-route checker rejects extra named exports on a page.tsx
file (tsc error TS2344 on .next/types), so the helper and its test import
now live in testPanelHelpers.ts instead.
* feat(tool-test): add JSON edit modal state and handlers
* feat(tool-test): collapsed preview + edit modal for object/array test parameters
* fix(tool-test): validate JSON on modal open, not just on edit
Opening the JSON edit modal on an untouched object/array param (no value
yet in testArgValues) loaded an empty draft with jsonEditError hardcoded
to null, so Save was enabled despite invalid JSON and silently no-op'd on
click. Now runs the same JSON.parse check used by the live textarea
validation when the modal opens.
* chore: regenerate openapi.json for ToolTestResponse hint/request_* fields
drift-check on PR #547 was failing because the earlier hint/request_method/
request_url/request_body/request_params fields added to ToolTestResponse
were never reflected in the dumped spec. Regenerated via
scripts.dump_docs_openapi.
* fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner
httpx raises a TypeError when a query param value is a dict/list, which
was silently caught and surfaced as a generic tool-execution error —
this is what actually broke test requests, not just the "[object
Object]" display. JSON-stringify object/array arguments before they
become query params, in both the live execute_http_tool() path and the
test route's request_params display shaping.
Also adds an unsaved-changes warning banner above Test Tool, shown
when the live form state diverges from the last-saved HTTP API config,
since Test Tool always runs the saved config.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(tool-test): keep empty request body in preview; normalize headers in snapshot
- POST/PUT/PATCH with no arguments now shows `{}` in the request body
preview instead of null — matches what execute_http_tool actually sends
over the wire (json={})
- buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a
deduped key→value map before serializing, matching the shape saved to
the backend; duplicate header keys no longer cause a false unsaved-
changes warning
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(tool-test): update assertion for empty POST body preview
test_tool_test_no_arguments_leaves_body_and_params_none expected
request_body=None for a POST with no arguments. The fix to preserve {}
in the preview makes request_body={} the correct assertion.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tools): refine HTTP tool testing
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
|
|
|
ToolTestRequest,
|
|
|
|
|
ToolTestResponse,
|
2026-05-31 16:50:44 +05:30
|
|
|
UpdateToolRequest,
|
|
|
|
|
_populate_discovered_tools,
|
|
|
|
|
refresh_mcp_tools,
|
feat: add tool test panel for HTTP API tools (#547)
* feat: add tool test panel for HTTP API tools
Lets developers run a saved HTTP API tool against its real endpoint
from the tool detail page, without needing a live call. Reuses the
production execute_http_tool path so test behavior matches call-time
behavior.
- New POST /tools/{tool_uuid}/test route
- Test panel with per-parameter typed inputs and auto-detected
context variable inputs (from preset parameter templates)
- Validate parameter name uniqueness on save, matching the existing
transferParameters check
- Fix stale FunctionCallsFromLLMInfoFrame import causing test
collection failures against pipecat-ai 1.5.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: revert local-venv pipecat drift fix, apply ruff import formatting
test_custom_tools.py and test_unregistered_function_call.py were edited
locally to drop FunctionCallsFromLLMInfoFrame after hitting an
ImportError — that error was from a stale local pipecat-ai package, not
a real drift. CI's pipecat build emits this frame and the test asserted
on it, so removing it broke test_llm_calls_custom_tool_handler and its
unregistered-call counterpart. Reverted both files to match main.
Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to
clear the drift-check job (split the aliased import into its own
`from ... import (...)` block, wrapped a long monkeypatch.setattr call).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec
pytest's default discovery matches any top-level test_* name in a test
module, including imported functions — importing the route handler as
`test_tool as test_tool_route` still matched the pattern, so pytest
tried to run it as a test and failed injecting fixtures for tool_uuid/
request/user. Renamed the alias to call_test_tool_route.
Also regenerated docs/api-reference/openapi.json for the new
POST /tools/{tool_uuid}/test route and its two schemas (couldn't run
the dump script locally — pipecat-ai version mismatch documented
separately — so hand-built the diff to exactly match FastAPI's
get_openapi() output format, verified against neighboring routes).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address cubic review findings on test panel
- Resolve dotted context-variable keys into nested objects before
posting the test request. render_template's get_nested_value walks
nested dicts, so a flat key like "runtime_configuration.realtime_model"
never matched — templates referencing nested context always resolved
to empty.
- Restrict isHttpApiTool to an explicit category equality check instead
of inferring it from exclusions. native/integration tools are
currently disabled in the create-tool UI so this wasn't reachable
today, but the exclusion list silently goes stale as new categories
are added.
- Stop showing a green success badge for non-2xx responses.
execute_http_tool returns status: "success" for any HTTP exchange
that completes, regardless of status code — only transport-level
errors (timeout, connection failure) get status: "error". The test
panel now checks status_code is in the 2xx range before treating the
call as a success.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address second round of cubic/greptile findings
- Guard setNestedValue against prototype-pollution keys (__proto__,
constructor, prototype) in the dotted context-var path before
traversing.
- Normalize status to "error" in the test route when the upstream
status_code is >= 400. execute_http_tool only distinguishes
transport-level failures (timeout, connection error) from
"success" — a completed 4xx/5xx exchange still came back as
"success" from the executor.
- Seed testArgValues defaults for number/boolean parameters via a
useEffect keyed on the parameters array, so a required number or
boolean field isn't silently omitted from the test request if the
user never touches its input.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: only seed test-arg defaults for required number/boolean params
Seeding optional number/boolean parameters silently changed the test
request — an optional boolean flag the tester never touched was sent
as true, which can flip upstream behavior unintentionally. Restrict
seeding to required parameters.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: match whitespace and fallback-filter syntax in context var detection
extractContextVars required an exact {{initial_context.foo}} with no
whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN
(and render_template) accepts {{ initial_context.foo }} and
{{initial_context.foo | fallback:value}}. A preset parameter saved with
either of those forms resolved fine in production but showed no input
in the test panel, so testing always sent it empty context.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add hint and request_* fields to ToolTestResponse
Extends ToolTestResponse with hint, request_method, request_url,
request_body, and request_params so the frontend can surface what was
actually sent and a human-readable hint about why a test call failed.
* feat(tool-test): add status-code hints and request_method/url/body/params to test_tool()
* style: ruff-format test_mcp_tool_route.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): wire hint banner and request block into result panel
Backend has returned hint/request_method/request_url/request_body/
request_params since d45ea851/60aaf31d but the frontend never
displayed them. Extends ToolTestResult with the new fields and renders
an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx)
plus a Request block above the response showing exactly what was sent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(tool-test): include resolved preset params in request_body/params
Found via live GET/POST testing: the Request preview showed only the
model-provided arguments, not what execute_http_tool actually sends.
execute_http_tool merges resolved_arguments = {**arguments,
**preset_arguments} before building the outbound body/params — preset
params (e.g. {{initial_context.metadata.channel}}) are invisible to
the model but still go out on the wire. The preview now mirrors that
merge via the same _resolve_preset_parameters helper, so a dev sees
exactly what was sent, not just what the model provided.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add generateSampleValue helper for sample-fill button
* feat(tool-test): add Fill sample values button for arguments and context vars
Moved generateSampleValue out of page.tsx into a sibling helpers module:
Next.js's typed-route checker rejects extra named exports on a page.tsx
file (tsc error TS2344 on .next/types), so the helper and its test import
now live in testPanelHelpers.ts instead.
* feat(tool-test): add JSON edit modal state and handlers
* feat(tool-test): collapsed preview + edit modal for object/array test parameters
* fix(tool-test): validate JSON on modal open, not just on edit
Opening the JSON edit modal on an untouched object/array param (no value
yet in testArgValues) loaded an empty draft with jsonEditError hardcoded
to null, so Save was enabled despite invalid JSON and silently no-op'd on
click. Now runs the same JSON.parse check used by the live textarea
validation when the modal opens.
* chore: regenerate openapi.json for ToolTestResponse hint/request_* fields
drift-check on PR #547 was failing because the earlier hint/request_method/
request_url/request_body/request_params fields added to ToolTestResponse
were never reflected in the dumped spec. Regenerated via
scripts.dump_docs_openapi.
* fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner
httpx raises a TypeError when a query param value is a dict/list, which
was silently caught and surfaced as a generic tool-execution error —
this is what actually broke test requests, not just the "[object
Object]" display. JSON-stringify object/array arguments before they
become query params, in both the live execute_http_tool() path and the
test route's request_params display shaping.
Also adds an unsaved-changes warning banner above Test Tool, shown
when the live form state diverges from the last-saved HTTP API config,
since Test Tool always runs the saved config.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(tool-test): keep empty request body in preview; normalize headers in snapshot
- POST/PUT/PATCH with no arguments now shows `{}` in the request body
preview instead of null — matches what execute_http_tool actually sends
over the wire (json={})
- buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a
deduped key→value map before serializing, matching the shape saved to
the backend; duplicate header keys no longer cause a false unsaved-
changes warning
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(tool-test): update assertion for empty POST body preview
test_tool_test_no_arguments_leaves_body_and_params_none expected
request_body=None for a POST with no arguments. The fix to preserve {}
in the preview makes request_body={} the correct assertion.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tools): refine HTTP tool testing
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
|
|
|
router,
|
|
|
|
|
)
|
|
|
|
|
from api.routes.tool import (
|
|
|
|
|
test_tool as call_test_tool_route,
|
2026-05-31 16:50:44 +05:30
|
|
|
)
|
2026-05-19 07:40:00 -03:00
|
|
|
from api.services.workflow.tools.mcp_tool import (
|
|
|
|
|
validate_mcp_definition,
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# ── Canonical valid MCP request body ─────────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
VALID_MCP_DEFINITION = {
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {
|
|
|
|
|
"transport": "streamable_http",
|
|
|
|
|
"url": "https://x/mcp",
|
|
|
|
|
"credential_uuid": None,
|
|
|
|
|
"tools_filter": [],
|
|
|
|
|
},
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# ── Part A regression: CreateToolRequest / UpdateToolRequest validation ───────
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_create_tool_request_accepts_mcp_definition():
|
|
|
|
|
"""CreateToolRequest must accept an MCP definition (was HTTP 422 before fix)."""
|
|
|
|
|
req = CreateToolRequest(
|
|
|
|
|
name="My MCP Tool",
|
|
|
|
|
description="Integration via MCP",
|
|
|
|
|
category="mcp",
|
|
|
|
|
definition=VALID_MCP_DEFINITION,
|
|
|
|
|
)
|
|
|
|
|
assert isinstance(req.definition, McpToolDefinition)
|
|
|
|
|
assert req.definition.type == "mcp"
|
|
|
|
|
assert req.definition.config.url == "https://x/mcp"
|
|
|
|
|
assert req.definition.config.transport == "streamable_http"
|
|
|
|
|
assert req.definition.config.credential_uuid is None
|
|
|
|
|
assert req.definition.config.tools_filter == []
|
|
|
|
|
assert req.definition.config.timeout_secs == 30
|
|
|
|
|
assert req.definition.config.sse_read_timeout_secs == 300
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_update_tool_request_accepts_mcp_definition():
|
|
|
|
|
"""UpdateToolRequest must also accept an MCP definition."""
|
|
|
|
|
req = UpdateToolRequest(
|
|
|
|
|
name="Updated MCP Tool",
|
|
|
|
|
definition=VALID_MCP_DEFINITION,
|
|
|
|
|
)
|
|
|
|
|
assert isinstance(req.definition, McpToolDefinition)
|
|
|
|
|
assert req.definition.type == "mcp"
|
|
|
|
|
assert req.definition.config.url == "https://x/mcp"
|
|
|
|
|
|
|
|
|
|
|
2026-06-01 23:05:38 -07:00
|
|
|
def test_update_tool_request_accepts_http_api_complex_parameter_types():
|
|
|
|
|
"""HTTP API tools may accept structured JSON parameters."""
|
|
|
|
|
req = UpdateToolRequest(
|
|
|
|
|
name="Check Availability New Multi",
|
|
|
|
|
description="Check Availability when asked for it.",
|
|
|
|
|
definition={
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "http_api",
|
|
|
|
|
"config": {
|
|
|
|
|
"method": "POST",
|
|
|
|
|
"url": "https://automation.dograh.com/webhook/example",
|
|
|
|
|
"parameters": [
|
|
|
|
|
{
|
|
|
|
|
"name": "params",
|
|
|
|
|
"type": "object",
|
|
|
|
|
"description": (
|
|
|
|
|
"An object containing the name and datetime in ISO format"
|
|
|
|
|
),
|
|
|
|
|
"required": True,
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"name": "slots",
|
|
|
|
|
"type": "array",
|
|
|
|
|
"description": "Candidate availability slots.",
|
|
|
|
|
"required": False,
|
|
|
|
|
},
|
|
|
|
|
],
|
|
|
|
|
"preset_parameters": [
|
|
|
|
|
{
|
|
|
|
|
"name": "phone_number",
|
|
|
|
|
"type": "string",
|
|
|
|
|
"value_template": "{{initial_context.phone_number}}",
|
|
|
|
|
"required": True,
|
|
|
|
|
}
|
|
|
|
|
],
|
|
|
|
|
"timeout_ms": 5000,
|
|
|
|
|
"customMessageType": "text",
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert req.definition.type == "http_api"
|
|
|
|
|
parameters = req.definition.config.parameters
|
|
|
|
|
assert parameters[0].type == "object"
|
|
|
|
|
assert parameters[1].type == "array"
|
|
|
|
|
|
|
|
|
|
|
2026-05-19 07:40:00 -03:00
|
|
|
def test_create_tool_request_accepts_mcp_with_all_fields():
|
|
|
|
|
"""All optional MCP config fields are accepted and preserved."""
|
|
|
|
|
req = CreateToolRequest(
|
|
|
|
|
name="Full MCP Tool",
|
|
|
|
|
category="mcp",
|
|
|
|
|
definition={
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {
|
|
|
|
|
"transport": "streamable_http",
|
|
|
|
|
"url": "https://acme.example.com/mcp",
|
|
|
|
|
"credential_uuid": "cred-abc-123",
|
|
|
|
|
"tools_filter": ["lookup_patient", "schedule_appointment"],
|
|
|
|
|
"timeout_secs": 60,
|
|
|
|
|
"sse_read_timeout_secs": 600,
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
cfg = req.definition.config # type: ignore[union-attr]
|
|
|
|
|
assert cfg.url == "https://acme.example.com/mcp"
|
|
|
|
|
assert cfg.credential_uuid == "cred-abc-123"
|
|
|
|
|
assert cfg.tools_filter == ["lookup_patient", "schedule_appointment"]
|
|
|
|
|
assert cfg.timeout_secs == 60
|
|
|
|
|
assert cfg.sse_read_timeout_secs == 600
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# ── Invalid bodies are rejected ───────────────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
"definition",
|
|
|
|
|
[
|
|
|
|
|
# ftp:// URL — rejected by McpToolConfig.validate_url
|
|
|
|
|
{
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {"transport": "streamable_http", "url": "ftp://x/mcp"},
|
|
|
|
|
},
|
|
|
|
|
# Empty url — rejected by McpToolConfig.validate_url
|
|
|
|
|
{
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {"transport": "streamable_http", "url": ""},
|
|
|
|
|
},
|
|
|
|
|
# Missing url — rejected by McpToolConfig (required field)
|
|
|
|
|
{
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {"transport": "streamable_http"},
|
|
|
|
|
},
|
|
|
|
|
# Unsupported transport — rejected because Literal["streamable_http"] constraint
|
|
|
|
|
{
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {"url": "https://x/mcp", "transport": "stdio"},
|
|
|
|
|
},
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
def test_create_tool_request_rejects_invalid_mcp_definition(definition):
|
|
|
|
|
"""Invalid MCP definitions must raise ValidationError."""
|
|
|
|
|
with pytest.raises(ValidationError):
|
|
|
|
|
CreateToolRequest(
|
|
|
|
|
name="Bad MCP Tool",
|
|
|
|
|
category="mcp",
|
|
|
|
|
definition=definition,
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# ── Round-trip compatibility: request schema ↔ validate_mcp_definition ───────
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_mcp_definition_round_trips_through_validate_mcp_definition():
|
|
|
|
|
"""The dict produced by CreateToolRequest.definition.model_dump() must be
|
|
|
|
|
accepted by validate_mcp_definition without raising, and the result must
|
|
|
|
|
contain the expected fields. This proves the request-layer schema and the
|
|
|
|
|
call-time validator agree on the stored config shape."""
|
|
|
|
|
req = CreateToolRequest(
|
|
|
|
|
name="Round-Trip MCP Tool",
|
|
|
|
|
category="mcp",
|
|
|
|
|
definition={
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {
|
|
|
|
|
"transport": "streamable_http",
|
|
|
|
|
"url": "https://roundtrip.example.com/mcp",
|
|
|
|
|
"credential_uuid": "cred-rt-456",
|
|
|
|
|
"tools_filter": ["ping"],
|
|
|
|
|
"timeout_secs": 45,
|
|
|
|
|
"sse_read_timeout_secs": 400,
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# Simulate what the route does: persist definition as a plain dict
|
|
|
|
|
persisted = req.definition.model_dump() # type: ignore[union-attr]
|
|
|
|
|
|
|
|
|
|
# validate_mcp_definition must accept the persisted shape without raising
|
|
|
|
|
normalized = validate_mcp_definition(persisted)
|
|
|
|
|
|
|
|
|
|
assert normalized["url"] == "https://roundtrip.example.com/mcp"
|
|
|
|
|
assert normalized["transport"] == "streamable_http"
|
|
|
|
|
assert normalized["credential_uuid"] == "cred-rt-456"
|
|
|
|
|
assert normalized["tools_filter"] == ["ping"]
|
|
|
|
|
assert normalized["timeout_secs"] == 45
|
|
|
|
|
assert normalized["sse_read_timeout_secs"] == 400
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_mcp_definition_round_trip_defaults():
|
|
|
|
|
"""Round-trip with minimal body: defaults fill in correctly and
|
|
|
|
|
validate_mcp_definition agrees on them."""
|
|
|
|
|
req = CreateToolRequest(
|
|
|
|
|
name="Minimal MCP Tool",
|
|
|
|
|
category="mcp",
|
|
|
|
|
definition=VALID_MCP_DEFINITION,
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
persisted = req.definition.model_dump() # type: ignore[union-attr]
|
|
|
|
|
normalized = validate_mcp_definition(persisted)
|
|
|
|
|
|
|
|
|
|
assert normalized["transport"] == "streamable_http"
|
|
|
|
|
assert normalized["tools_filter"] == []
|
|
|
|
|
assert normalized["timeout_secs"] == 30
|
|
|
|
|
assert normalized["sse_read_timeout_secs"] == 300
|
|
|
|
|
assert normalized["credential_uuid"] is None
|
|
|
|
|
# Part B: auth_header / auth_scheme must NOT be present in the normalized
|
|
|
|
|
# config dict (they were dead config removed in the fix)
|
|
|
|
|
assert "auth_header" not in normalized
|
|
|
|
|
assert "auth_scheme" not in normalized
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# ── Full HTTP round-trip via ASGI test client ─────────────────────────────────
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_post_tool_mcp_returns_200(test_client_factory, db_session):
|
|
|
|
|
"""POST /api/v1/tools/ with an MCP definition must return HTTP 200 and
|
|
|
|
|
persist the definition with type='mcp'. Before Part A this always
|
|
|
|
|
returned 422."""
|
|
|
|
|
# Create a user and an organization, then link them so the route's
|
|
|
|
|
# selected_organization_id check passes.
|
|
|
|
|
user, _ = await db_session.get_or_create_user_by_provider_id("mcp_route_test_user")
|
|
|
|
|
org, _ = await db_session.get_or_create_organization_by_provider_id(
|
|
|
|
|
"mcp_route_test_org", user.id
|
|
|
|
|
)
|
|
|
|
|
await db_session.update_user_selected_organization(user.id, org.id)
|
|
|
|
|
# Reload the user so selected_organization_id is populated on the object.
|
|
|
|
|
user = await db_session.get_user_by_id(user.id)
|
|
|
|
|
|
|
|
|
|
async with test_client_factory(user) as client:
|
|
|
|
|
response = await client.post(
|
|
|
|
|
"/api/v1/tools/",
|
|
|
|
|
json={
|
|
|
|
|
"name": "HTTP Round-Trip MCP Tool",
|
|
|
|
|
"description": "Testing the full route",
|
|
|
|
|
"category": "mcp",
|
|
|
|
|
"definition": {
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {
|
|
|
|
|
"transport": "streamable_http",
|
|
|
|
|
"url": "https://roundtrip.example.com/mcp",
|
|
|
|
|
"credential_uuid": None,
|
|
|
|
|
"tools_filter": [],
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert response.status_code == 200, (
|
|
|
|
|
f"Expected 200, got {response.status_code}: {response.text}"
|
|
|
|
|
)
|
|
|
|
|
body = response.json()
|
|
|
|
|
assert body["definition"]["type"] == "mcp"
|
|
|
|
|
assert body["definition"]["config"]["url"] == "https://roundtrip.example.com/mcp"
|
|
|
|
|
assert body["category"] == "mcp"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_post_tool_mcp_invalid_url_returns_422(test_client_factory, db_session):
|
|
|
|
|
"""POST /api/v1/tools/ with an ftp:// URL must return HTTP 422."""
|
|
|
|
|
user, _ = await db_session.get_or_create_user_by_provider_id(
|
|
|
|
|
"mcp_route_test_user_422"
|
|
|
|
|
)
|
|
|
|
|
org, _ = await db_session.get_or_create_organization_by_provider_id(
|
|
|
|
|
"mcp_route_test_org_422", user.id
|
|
|
|
|
)
|
|
|
|
|
await db_session.update_user_selected_organization(user.id, org.id)
|
|
|
|
|
user = await db_session.get_user_by_id(user.id)
|
|
|
|
|
|
|
|
|
|
async with test_client_factory(user) as client:
|
|
|
|
|
response = await client.post(
|
|
|
|
|
"/api/v1/tools/",
|
|
|
|
|
json={
|
|
|
|
|
"name": "Bad MCP Tool",
|
|
|
|
|
"category": "mcp",
|
|
|
|
|
"definition": {
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {
|
|
|
|
|
"transport": "streamable_http",
|
|
|
|
|
"url": "ftp://invalid.example.com/mcp",
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert response.status_code == 422
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# ── Task 6: discovered_tools field and _populate_discovered_tools helper ──────
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_mcp_config_accepts_discovered_tools():
|
|
|
|
|
cfg = McpToolConfig(
|
|
|
|
|
url="https://x/mcp",
|
|
|
|
|
discovered_tools=[{"name": "echo", "description": "Echo"}],
|
|
|
|
|
)
|
|
|
|
|
assert cfg.discovered_tools == [{"name": "echo", "description": "Echo"}]
|
|
|
|
|
# Defaults to [] when omitted
|
|
|
|
|
assert McpToolConfig(url="https://x/mcp").discovered_tools == []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_populate_discovered_tools_overwrites_cache(monkeypatch):
|
2026-05-31 16:50:44 +05:30
|
|
|
import api.services.tool_management as tool_svc
|
2026-05-19 07:40:00 -03:00
|
|
|
|
|
|
|
|
monkeypatch.setattr(
|
2026-05-31 16:50:44 +05:30
|
|
|
tool_svc,
|
2026-05-19 07:40:00 -03:00
|
|
|
"discover_mcp_tools",
|
|
|
|
|
AsyncMock(return_value=[{"name": "echo", "description": "Echo"}]),
|
|
|
|
|
)
|
|
|
|
|
definition = {
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {
|
|
|
|
|
"url": "https://x/mcp",
|
|
|
|
|
"tools_filter": [],
|
|
|
|
|
"discovered_tools": [{"name": "stale", "description": "old"}],
|
|
|
|
|
},
|
|
|
|
|
}
|
|
|
|
|
out = await _populate_discovered_tools(definition, organization_id=1)
|
|
|
|
|
assert out["config"]["discovered_tools"] == [
|
|
|
|
|
{"name": "echo", "description": "Echo"}
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_populate_discovered_tools_non_mcp_is_noop():
|
|
|
|
|
definition = {"schema_version": 1, "type": "http_api", "config": {}}
|
|
|
|
|
out = await _populate_discovered_tools(definition, organization_id=1)
|
|
|
|
|
assert out == definition # untouched
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_populate_discovered_tools_server_down_sets_empty(monkeypatch):
|
2026-05-31 16:50:44 +05:30
|
|
|
import api.services.tool_management as tool_svc
|
2026-05-19 07:40:00 -03:00
|
|
|
|
|
|
|
|
monkeypatch.setattr(
|
2026-05-31 16:50:44 +05:30
|
|
|
tool_svc,
|
2026-05-19 07:40:00 -03:00
|
|
|
"discover_mcp_tools",
|
|
|
|
|
AsyncMock(side_effect=RuntimeError("connection refused")),
|
|
|
|
|
)
|
|
|
|
|
definition = {
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {"url": "https://x/mcp", "tools_filter": []},
|
|
|
|
|
}
|
|
|
|
|
out = await _populate_discovered_tools(definition, organization_id=1)
|
|
|
|
|
assert out["config"]["discovered_tools"] == []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# ── Task 7: POST /{tool_uuid}/mcp/refresh ─────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _fake_user(org_id=1):
|
|
|
|
|
u = MagicMock()
|
|
|
|
|
u.selected_organization_id = org_id
|
|
|
|
|
u.id = 1
|
|
|
|
|
u.provider_id = "p1"
|
|
|
|
|
return u
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _mcp_tool_model(org_id=1):
|
|
|
|
|
t = MagicMock()
|
|
|
|
|
t.tool_uuid = "tu-mcp"
|
|
|
|
|
t.name = "Mock MCP"
|
|
|
|
|
t.category = "mcp"
|
|
|
|
|
t.definition = {
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "mcp",
|
|
|
|
|
"config": {"url": "https://x/mcp", "tools_filter": []},
|
|
|
|
|
}
|
|
|
|
|
return t
|
|
|
|
|
|
|
|
|
|
|
feat: add tool test panel for HTTP API tools (#547)
* feat: add tool test panel for HTTP API tools
Lets developers run a saved HTTP API tool against its real endpoint
from the tool detail page, without needing a live call. Reuses the
production execute_http_tool path so test behavior matches call-time
behavior.
- New POST /tools/{tool_uuid}/test route
- Test panel with per-parameter typed inputs and auto-detected
context variable inputs (from preset parameter templates)
- Validate parameter name uniqueness on save, matching the existing
transferParameters check
- Fix stale FunctionCallsFromLLMInfoFrame import causing test
collection failures against pipecat-ai 1.5.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: revert local-venv pipecat drift fix, apply ruff import formatting
test_custom_tools.py and test_unregistered_function_call.py were edited
locally to drop FunctionCallsFromLLMInfoFrame after hitting an
ImportError — that error was from a stale local pipecat-ai package, not
a real drift. CI's pipecat build emits this frame and the test asserted
on it, so removing it broke test_llm_calls_custom_tool_handler and its
unregistered-call counterpart. Reverted both files to match main.
Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to
clear the drift-check job (split the aliased import into its own
`from ... import (...)` block, wrapped a long monkeypatch.setattr call).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec
pytest's default discovery matches any top-level test_* name in a test
module, including imported functions — importing the route handler as
`test_tool as test_tool_route` still matched the pattern, so pytest
tried to run it as a test and failed injecting fixtures for tool_uuid/
request/user. Renamed the alias to call_test_tool_route.
Also regenerated docs/api-reference/openapi.json for the new
POST /tools/{tool_uuid}/test route and its two schemas (couldn't run
the dump script locally — pipecat-ai version mismatch documented
separately — so hand-built the diff to exactly match FastAPI's
get_openapi() output format, verified against neighboring routes).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address cubic review findings on test panel
- Resolve dotted context-variable keys into nested objects before
posting the test request. render_template's get_nested_value walks
nested dicts, so a flat key like "runtime_configuration.realtime_model"
never matched — templates referencing nested context always resolved
to empty.
- Restrict isHttpApiTool to an explicit category equality check instead
of inferring it from exclusions. native/integration tools are
currently disabled in the create-tool UI so this wasn't reachable
today, but the exclusion list silently goes stale as new categories
are added.
- Stop showing a green success badge for non-2xx responses.
execute_http_tool returns status: "success" for any HTTP exchange
that completes, regardless of status code — only transport-level
errors (timeout, connection failure) get status: "error". The test
panel now checks status_code is in the 2xx range before treating the
call as a success.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address second round of cubic/greptile findings
- Guard setNestedValue against prototype-pollution keys (__proto__,
constructor, prototype) in the dotted context-var path before
traversing.
- Normalize status to "error" in the test route when the upstream
status_code is >= 400. execute_http_tool only distinguishes
transport-level failures (timeout, connection error) from
"success" — a completed 4xx/5xx exchange still came back as
"success" from the executor.
- Seed testArgValues defaults for number/boolean parameters via a
useEffect keyed on the parameters array, so a required number or
boolean field isn't silently omitted from the test request if the
user never touches its input.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: only seed test-arg defaults for required number/boolean params
Seeding optional number/boolean parameters silently changed the test
request — an optional boolean flag the tester never touched was sent
as true, which can flip upstream behavior unintentionally. Restrict
seeding to required parameters.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: match whitespace and fallback-filter syntax in context var detection
extractContextVars required an exact {{initial_context.foo}} with no
whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN
(and render_template) accepts {{ initial_context.foo }} and
{{initial_context.foo | fallback:value}}. A preset parameter saved with
either of those forms resolved fine in production but showed no input
in the test panel, so testing always sent it empty context.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add hint and request_* fields to ToolTestResponse
Extends ToolTestResponse with hint, request_method, request_url,
request_body, and request_params so the frontend can surface what was
actually sent and a human-readable hint about why a test call failed.
* feat(tool-test): add status-code hints and request_method/url/body/params to test_tool()
* style: ruff-format test_mcp_tool_route.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): wire hint banner and request block into result panel
Backend has returned hint/request_method/request_url/request_body/
request_params since d45ea851/60aaf31d but the frontend never
displayed them. Extends ToolTestResult with the new fields and renders
an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx)
plus a Request block above the response showing exactly what was sent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(tool-test): include resolved preset params in request_body/params
Found via live GET/POST testing: the Request preview showed only the
model-provided arguments, not what execute_http_tool actually sends.
execute_http_tool merges resolved_arguments = {**arguments,
**preset_arguments} before building the outbound body/params — preset
params (e.g. {{initial_context.metadata.channel}}) are invisible to
the model but still go out on the wire. The preview now mirrors that
merge via the same _resolve_preset_parameters helper, so a dev sees
exactly what was sent, not just what the model provided.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add generateSampleValue helper for sample-fill button
* feat(tool-test): add Fill sample values button for arguments and context vars
Moved generateSampleValue out of page.tsx into a sibling helpers module:
Next.js's typed-route checker rejects extra named exports on a page.tsx
file (tsc error TS2344 on .next/types), so the helper and its test import
now live in testPanelHelpers.ts instead.
* feat(tool-test): add JSON edit modal state and handlers
* feat(tool-test): collapsed preview + edit modal for object/array test parameters
* fix(tool-test): validate JSON on modal open, not just on edit
Opening the JSON edit modal on an untouched object/array param (no value
yet in testArgValues) loaded an empty draft with jsonEditError hardcoded
to null, so Save was enabled despite invalid JSON and silently no-op'd on
click. Now runs the same JSON.parse check used by the live textarea
validation when the modal opens.
* chore: regenerate openapi.json for ToolTestResponse hint/request_* fields
drift-check on PR #547 was failing because the earlier hint/request_method/
request_url/request_body/request_params fields added to ToolTestResponse
were never reflected in the dumped spec. Regenerated via
scripts.dump_docs_openapi.
* fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner
httpx raises a TypeError when a query param value is a dict/list, which
was silently caught and surfaced as a generic tool-execution error —
this is what actually broke test requests, not just the "[object
Object]" display. JSON-stringify object/array arguments before they
become query params, in both the live execute_http_tool() path and the
test route's request_params display shaping.
Also adds an unsaved-changes warning banner above Test Tool, shown
when the live form state diverges from the last-saved HTTP API config,
since Test Tool always runs the saved config.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(tool-test): keep empty request body in preview; normalize headers in snapshot
- POST/PUT/PATCH with no arguments now shows `{}` in the request body
preview instead of null — matches what execute_http_tool actually sends
over the wire (json={})
- buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a
deduped key→value map before serializing, matching the shape saved to
the backend; duplicate header keys no longer cause a false unsaved-
changes warning
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(tool-test): update assertion for empty POST body preview
test_tool_test_no_arguments_leaves_body_and_params_none expected
request_body=None for a POST with no arguments. The fix to preserve {}
in the preview makes request_body={} the correct assertion.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tools): refine HTTP tool testing
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
|
|
|
def _http_tool_model(method="GET"):
|
|
|
|
|
t = MagicMock()
|
|
|
|
|
t.tool_uuid = "tu-http"
|
|
|
|
|
t.name = "Mock HTTP"
|
|
|
|
|
t.category = "http_api"
|
|
|
|
|
t.definition = {
|
|
|
|
|
"schema_version": 1,
|
|
|
|
|
"type": "http_api",
|
|
|
|
|
"config": {"method": method, "url": "https://example.com/search"},
|
|
|
|
|
}
|
|
|
|
|
return t
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_tool_executes_http_api_tool_with_llm_and_preset_params(monkeypatch):
|
|
|
|
|
import api.routes.tool as tool_route
|
|
|
|
|
|
|
|
|
|
tool = _http_tool_model()
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route.db_client, "get_tool_by_uuid", AsyncMock(return_value=tool)
|
|
|
|
|
)
|
|
|
|
|
executor = AsyncMock(
|
|
|
|
|
return_value={
|
|
|
|
|
"status": "success",
|
|
|
|
|
"status_code": 200,
|
|
|
|
|
"data": {"ok": True},
|
|
|
|
|
}
|
|
|
|
|
)
|
|
|
|
|
monkeypatch.setattr(tool_route, "execute_http_tool", executor)
|
|
|
|
|
|
|
|
|
|
resp = await call_test_tool_route(
|
|
|
|
|
"tu-http",
|
|
|
|
|
request=ToolTestRequest(
|
|
|
|
|
llm_params={"query": "cart"},
|
|
|
|
|
preset_params={
|
|
|
|
|
"customer_id": "c_123",
|
|
|
|
|
"sentiment": "cooperative",
|
|
|
|
|
},
|
|
|
|
|
),
|
|
|
|
|
user=_fake_user(),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert resp.status == "success"
|
|
|
|
|
assert resp.status_code == 200
|
|
|
|
|
assert resp.data == {"ok": True}
|
|
|
|
|
assert resp.error is None
|
|
|
|
|
executor.assert_awaited_once_with(
|
|
|
|
|
tool,
|
|
|
|
|
{"query": "cart"},
|
|
|
|
|
preset_params={
|
|
|
|
|
"customer_id": "c_123",
|
|
|
|
|
"sentiment": "cooperative",
|
|
|
|
|
},
|
|
|
|
|
organization_id=1,
|
|
|
|
|
include_request_headers=True,
|
|
|
|
|
)
|
|
|
|
|
assert resp.hint is None
|
|
|
|
|
assert resp.request_method == "GET"
|
|
|
|
|
assert resp.request_url == "https://example.com/search"
|
|
|
|
|
assert resp.request_headers == {}
|
|
|
|
|
assert resp.request_body is None
|
|
|
|
|
assert resp.request_params == {
|
|
|
|
|
"query": "cart",
|
|
|
|
|
"customer_id": "c_123",
|
|
|
|
|
"sentiment": "cooperative",
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_tool_test_sets_request_body_for_post_method(monkeypatch):
|
|
|
|
|
import api.routes.tool as tool_route
|
|
|
|
|
|
|
|
|
|
tool = _http_tool_model(method="POST")
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route.db_client, "get_tool_by_uuid", AsyncMock(return_value=tool)
|
|
|
|
|
)
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route,
|
|
|
|
|
"execute_http_tool",
|
|
|
|
|
AsyncMock(
|
|
|
|
|
return_value={"status": "success", "status_code": 200, "data": {"id": 1}}
|
|
|
|
|
),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
resp = await call_test_tool_route(
|
|
|
|
|
"tu-http",
|
|
|
|
|
request=ToolTestRequest(llm_params={"name": "Ada"}),
|
|
|
|
|
user=_fake_user(),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert resp.request_method == "POST"
|
|
|
|
|
assert resp.request_body == {"name": "Ada"}
|
|
|
|
|
assert resp.request_params is None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_tool_test_returns_masked_effective_request_headers(monkeypatch):
|
|
|
|
|
import api.routes.tool as tool_route
|
|
|
|
|
|
|
|
|
|
tool = _http_tool_model(method="POST")
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route.db_client, "get_tool_by_uuid", AsyncMock(return_value=tool)
|
|
|
|
|
)
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route,
|
|
|
|
|
"execute_http_tool",
|
|
|
|
|
AsyncMock(
|
|
|
|
|
return_value={
|
|
|
|
|
"status": "success",
|
|
|
|
|
"status_code": 200,
|
|
|
|
|
"data": {"ok": True},
|
|
|
|
|
"request_headers": {
|
|
|
|
|
"X-Tenant": "acme",
|
|
|
|
|
"Authorization": "****************oken",
|
|
|
|
|
},
|
|
|
|
|
}
|
|
|
|
|
),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
resp = await call_test_tool_route(
|
|
|
|
|
"tu-http", request=ToolTestRequest(), user=_fake_user()
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert resp.request_headers == {
|
|
|
|
|
"X-Tenant": "acme",
|
|
|
|
|
"Authorization": "****************oken",
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_tool_test_request_body_includes_resolved_preset_parameters(
|
|
|
|
|
monkeypatch,
|
|
|
|
|
):
|
|
|
|
|
"""The Request preview includes direct preset values alongside LLM values."""
|
|
|
|
|
import api.routes.tool as tool_route
|
|
|
|
|
|
|
|
|
|
tool = _http_tool_model(method="POST")
|
|
|
|
|
tool.definition["config"]["preset_parameters"] = [
|
|
|
|
|
{
|
|
|
|
|
"name": "source",
|
|
|
|
|
"type": "string",
|
|
|
|
|
"value_template": "{{initial_context.metadata.channel}}",
|
|
|
|
|
"required": True,
|
|
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route.db_client, "get_tool_by_uuid", AsyncMock(return_value=tool)
|
|
|
|
|
)
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route,
|
|
|
|
|
"execute_http_tool",
|
|
|
|
|
AsyncMock(
|
|
|
|
|
return_value={"status": "success", "status_code": 200, "data": {"ok": True}}
|
|
|
|
|
),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
resp = await call_test_tool_route(
|
|
|
|
|
"tu-http",
|
|
|
|
|
request=ToolTestRequest(
|
|
|
|
|
llm_params={"name": "Ada"},
|
|
|
|
|
preset_params={"source": "web_widget"},
|
|
|
|
|
),
|
|
|
|
|
user=_fake_user(),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert resp.request_body == {"name": "Ada", "source": "web_widget"}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_tool_test_no_arguments_post_shows_empty_body(monkeypatch):
|
|
|
|
|
import api.routes.tool as tool_route
|
|
|
|
|
|
|
|
|
|
tool = _http_tool_model(method="POST")
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route.db_client, "get_tool_by_uuid", AsyncMock(return_value=tool)
|
|
|
|
|
)
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route,
|
|
|
|
|
"execute_http_tool",
|
|
|
|
|
AsyncMock(return_value={"status": "success", "status_code": 200, "data": None}),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
resp = await call_test_tool_route(
|
|
|
|
|
"tu-http", request=ToolTestRequest(), user=_fake_user()
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# POST with no arguments sends json={} over the wire; preview must show {}
|
|
|
|
|
# so callers can distinguish an absent body from an empty one.
|
|
|
|
|
assert resp.request_body == {}
|
|
|
|
|
assert resp.request_params is None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
"status_code,expected_snippet",
|
|
|
|
|
[
|
|
|
|
|
(400, "HTTP 400 Bad Request"),
|
|
|
|
|
(401, "HTTP 401 Unauthorized"),
|
|
|
|
|
(403, "HTTP 403 Forbidden"),
|
|
|
|
|
(404, "HTTP 404 Not Found"),
|
|
|
|
|
(405, "HTTP 405 Method Not Allowed"),
|
|
|
|
|
(408, "HTTP 408 Request Timeout"),
|
|
|
|
|
(409, "HTTP 409 Conflict"),
|
|
|
|
|
(415, "HTTP 415 Unsupported Media Type"),
|
|
|
|
|
(422, "HTTP 422 Unprocessable Entity"),
|
|
|
|
|
(429, "HTTP 429 Too Many Requests"),
|
|
|
|
|
(500, "HTTP 500"),
|
|
|
|
|
(503, "HTTP 503"),
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
async def test_tool_test_hint_for_status_code(
|
|
|
|
|
monkeypatch, status_code, expected_snippet
|
|
|
|
|
):
|
|
|
|
|
import api.routes.tool as tool_route
|
|
|
|
|
|
|
|
|
|
tool = _http_tool_model(method="POST")
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route.db_client, "get_tool_by_uuid", AsyncMock(return_value=tool)
|
|
|
|
|
)
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route,
|
|
|
|
|
"execute_http_tool",
|
|
|
|
|
AsyncMock(
|
|
|
|
|
return_value={
|
|
|
|
|
"status": "error",
|
|
|
|
|
"status_code": status_code,
|
|
|
|
|
"error": "boom",
|
|
|
|
|
}
|
|
|
|
|
),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
resp = await call_test_tool_route(
|
|
|
|
|
"tu-http", request=ToolTestRequest(llm_params={"a": 1}), user=_fake_user()
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert resp.hint is not None
|
|
|
|
|
assert resp.hint.startswith(expected_snippet)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_tool_test_no_hint_on_success(monkeypatch):
|
|
|
|
|
import api.routes.tool as tool_route
|
|
|
|
|
|
|
|
|
|
tool = _http_tool_model(method="GET")
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route.db_client, "get_tool_by_uuid", AsyncMock(return_value=tool)
|
|
|
|
|
)
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route,
|
|
|
|
|
"execute_http_tool",
|
|
|
|
|
AsyncMock(return_value={"status": "success", "status_code": 200, "data": {}}),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
resp = await call_test_tool_route(
|
|
|
|
|
"tu-http", request=ToolTestRequest(), user=_fake_user()
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert resp.hint is None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_tool_test_no_hint_for_uncovered_status_code(monkeypatch):
|
|
|
|
|
import api.routes.tool as tool_route
|
|
|
|
|
|
|
|
|
|
tool = _http_tool_model(method="GET")
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route.db_client, "get_tool_by_uuid", AsyncMock(return_value=tool)
|
|
|
|
|
)
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route,
|
|
|
|
|
"execute_http_tool",
|
|
|
|
|
AsyncMock(
|
|
|
|
|
return_value={"status": "error", "status_code": 418, "error": "teapot"}
|
|
|
|
|
),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
resp = await call_test_tool_route(
|
|
|
|
|
"tu-http", request=ToolTestRequest(), user=_fake_user()
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert resp.hint is None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_tool_test_route_is_registered():
|
|
|
|
|
assert any(
|
|
|
|
|
route.path == "/tools/{tool_uuid}/test" and "POST" in route.methods
|
|
|
|
|
for route in router.routes
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_tool_rejects_non_http_api_tool(monkeypatch):
|
|
|
|
|
import api.routes.tool as tool_route
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
tool_route.db_client,
|
|
|
|
|
"get_tool_by_uuid",
|
|
|
|
|
AsyncMock(return_value=_mcp_tool_model()),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
with pytest.raises(HTTPException) as ei:
|
|
|
|
|
await call_test_tool_route(
|
|
|
|
|
"tu-mcp", request=ToolTestRequest(), user=_fake_user()
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert ei.value.status_code == 400
|
|
|
|
|
|
|
|
|
|
|
2026-05-19 07:40:00 -03:00
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_refresh_success(monkeypatch):
|
2026-05-31 16:50:44 +05:30
|
|
|
import api.services.tool_management as tool_svc
|
2026-05-19 07:40:00 -03:00
|
|
|
|
|
|
|
|
tool = _mcp_tool_model()
|
|
|
|
|
monkeypatch.setattr(
|
2026-05-31 16:50:44 +05:30
|
|
|
tool_svc.db_client, "get_tool_by_uuid", AsyncMock(return_value=tool)
|
2026-05-19 07:40:00 -03:00
|
|
|
)
|
|
|
|
|
monkeypatch.setattr(
|
2026-05-31 16:50:44 +05:30
|
|
|
tool_svc.db_client,
|
2026-05-19 07:40:00 -03:00
|
|
|
"update_tool",
|
|
|
|
|
AsyncMock(return_value=tool),
|
|
|
|
|
)
|
|
|
|
|
monkeypatch.setattr(
|
2026-05-31 16:50:44 +05:30
|
|
|
tool_svc,
|
2026-05-19 07:40:00 -03:00
|
|
|
"discover_mcp_tools",
|
|
|
|
|
AsyncMock(return_value=[{"name": "echo", "description": "Echo"}]),
|
|
|
|
|
)
|
|
|
|
|
resp = await refresh_mcp_tools("tu-mcp", user=_fake_user())
|
|
|
|
|
assert resp.discovered_tools == [{"name": "echo", "description": "Echo"}]
|
|
|
|
|
assert resp.error is None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_refresh_server_down_returns_200_with_error(monkeypatch):
|
2026-05-31 16:50:44 +05:30
|
|
|
import api.services.tool_management as tool_svc
|
2026-05-19 07:40:00 -03:00
|
|
|
|
|
|
|
|
tool = _mcp_tool_model()
|
|
|
|
|
monkeypatch.setattr(
|
2026-05-31 16:50:44 +05:30
|
|
|
tool_svc.db_client, "get_tool_by_uuid", AsyncMock(return_value=tool)
|
2026-05-19 07:40:00 -03:00
|
|
|
)
|
2026-05-31 16:50:44 +05:30
|
|
|
monkeypatch.setattr(tool_svc.db_client, "update_tool", AsyncMock(return_value=tool))
|
|
|
|
|
monkeypatch.setattr(tool_svc, "discover_mcp_tools", AsyncMock(return_value=[]))
|
2026-05-19 07:40:00 -03:00
|
|
|
resp = await refresh_mcp_tools("tu-mcp", user=_fake_user())
|
|
|
|
|
assert resp.discovered_tools == []
|
|
|
|
|
assert resp.error # non-empty human-readable message
|
|
|
|
|
# update_tool should NOT be called when discovery returns empty
|
2026-05-31 16:50:44 +05:30
|
|
|
tool_svc.db_client.update_tool.assert_not_called()
|
2026-05-19 07:40:00 -03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_refresh_non_mcp_is_400(monkeypatch):
|
2026-05-31 16:50:44 +05:30
|
|
|
import api.services.tool_management as tool_svc
|
2026-05-19 07:40:00 -03:00
|
|
|
|
|
|
|
|
tool = _mcp_tool_model()
|
|
|
|
|
tool.category = "http_api"
|
|
|
|
|
monkeypatch.setattr(
|
2026-05-31 16:50:44 +05:30
|
|
|
tool_svc.db_client, "get_tool_by_uuid", AsyncMock(return_value=tool)
|
2026-05-19 07:40:00 -03:00
|
|
|
)
|
|
|
|
|
with pytest.raises(HTTPException) as ei:
|
|
|
|
|
await refresh_mcp_tools("tu-mcp", user=_fake_user())
|
|
|
|
|
assert ei.value.status_code == 400
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_refresh_not_found_is_404(monkeypatch):
|
2026-05-31 16:50:44 +05:30
|
|
|
import api.services.tool_management as tool_svc
|
2026-05-19 07:40:00 -03:00
|
|
|
|
|
|
|
|
monkeypatch.setattr(
|
2026-05-31 16:50:44 +05:30
|
|
|
tool_svc.db_client, "get_tool_by_uuid", AsyncMock(return_value=None)
|
2026-05-19 07:40:00 -03:00
|
|
|
)
|
|
|
|
|
with pytest.raises(HTTPException) as ei:
|
|
|
|
|
await refresh_mcp_tools("nope", user=_fake_user())
|
|
|
|
|
assert ei.value.status_code == 404
|
feat: add tool test panel for HTTP API tools (#547)
* feat: add tool test panel for HTTP API tools
Lets developers run a saved HTTP API tool against its real endpoint
from the tool detail page, without needing a live call. Reuses the
production execute_http_tool path so test behavior matches call-time
behavior.
- New POST /tools/{tool_uuid}/test route
- Test panel with per-parameter typed inputs and auto-detected
context variable inputs (from preset parameter templates)
- Validate parameter name uniqueness on save, matching the existing
transferParameters check
- Fix stale FunctionCallsFromLLMInfoFrame import causing test
collection failures against pipecat-ai 1.5.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: revert local-venv pipecat drift fix, apply ruff import formatting
test_custom_tools.py and test_unregistered_function_call.py were edited
locally to drop FunctionCallsFromLLMInfoFrame after hitting an
ImportError — that error was from a stale local pipecat-ai package, not
a real drift. CI's pipecat build emits this frame and the test asserted
on it, so removing it broke test_llm_calls_custom_tool_handler and its
unregistered-call counterpart. Reverted both files to match main.
Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to
clear the drift-check job (split the aliased import into its own
`from ... import (...)` block, wrapped a long monkeypatch.setattr call).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec
pytest's default discovery matches any top-level test_* name in a test
module, including imported functions — importing the route handler as
`test_tool as test_tool_route` still matched the pattern, so pytest
tried to run it as a test and failed injecting fixtures for tool_uuid/
request/user. Renamed the alias to call_test_tool_route.
Also regenerated docs/api-reference/openapi.json for the new
POST /tools/{tool_uuid}/test route and its two schemas (couldn't run
the dump script locally — pipecat-ai version mismatch documented
separately — so hand-built the diff to exactly match FastAPI's
get_openapi() output format, verified against neighboring routes).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address cubic review findings on test panel
- Resolve dotted context-variable keys into nested objects before
posting the test request. render_template's get_nested_value walks
nested dicts, so a flat key like "runtime_configuration.realtime_model"
never matched — templates referencing nested context always resolved
to empty.
- Restrict isHttpApiTool to an explicit category equality check instead
of inferring it from exclusions. native/integration tools are
currently disabled in the create-tool UI so this wasn't reachable
today, but the exclusion list silently goes stale as new categories
are added.
- Stop showing a green success badge for non-2xx responses.
execute_http_tool returns status: "success" for any HTTP exchange
that completes, regardless of status code — only transport-level
errors (timeout, connection failure) get status: "error". The test
panel now checks status_code is in the 2xx range before treating the
call as a success.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address second round of cubic/greptile findings
- Guard setNestedValue against prototype-pollution keys (__proto__,
constructor, prototype) in the dotted context-var path before
traversing.
- Normalize status to "error" in the test route when the upstream
status_code is >= 400. execute_http_tool only distinguishes
transport-level failures (timeout, connection error) from
"success" — a completed 4xx/5xx exchange still came back as
"success" from the executor.
- Seed testArgValues defaults for number/boolean parameters via a
useEffect keyed on the parameters array, so a required number or
boolean field isn't silently omitted from the test request if the
user never touches its input.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: only seed test-arg defaults for required number/boolean params
Seeding optional number/boolean parameters silently changed the test
request — an optional boolean flag the tester never touched was sent
as true, which can flip upstream behavior unintentionally. Restrict
seeding to required parameters.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: match whitespace and fallback-filter syntax in context var detection
extractContextVars required an exact {{initial_context.foo}} with no
whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN
(and render_template) accepts {{ initial_context.foo }} and
{{initial_context.foo | fallback:value}}. A preset parameter saved with
either of those forms resolved fine in production but showed no input
in the test panel, so testing always sent it empty context.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add hint and request_* fields to ToolTestResponse
Extends ToolTestResponse with hint, request_method, request_url,
request_body, and request_params so the frontend can surface what was
actually sent and a human-readable hint about why a test call failed.
* feat(tool-test): add status-code hints and request_method/url/body/params to test_tool()
* style: ruff-format test_mcp_tool_route.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): wire hint banner and request block into result panel
Backend has returned hint/request_method/request_url/request_body/
request_params since d45ea851/60aaf31d but the frontend never
displayed them. Extends ToolTestResult with the new fields and renders
an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx)
plus a Request block above the response showing exactly what was sent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(tool-test): include resolved preset params in request_body/params
Found via live GET/POST testing: the Request preview showed only the
model-provided arguments, not what execute_http_tool actually sends.
execute_http_tool merges resolved_arguments = {**arguments,
**preset_arguments} before building the outbound body/params — preset
params (e.g. {{initial_context.metadata.channel}}) are invisible to
the model but still go out on the wire. The preview now mirrors that
merge via the same _resolve_preset_parameters helper, so a dev sees
exactly what was sent, not just what the model provided.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add generateSampleValue helper for sample-fill button
* feat(tool-test): add Fill sample values button for arguments and context vars
Moved generateSampleValue out of page.tsx into a sibling helpers module:
Next.js's typed-route checker rejects extra named exports on a page.tsx
file (tsc error TS2344 on .next/types), so the helper and its test import
now live in testPanelHelpers.ts instead.
* feat(tool-test): add JSON edit modal state and handlers
* feat(tool-test): collapsed preview + edit modal for object/array test parameters
* fix(tool-test): validate JSON on modal open, not just on edit
Opening the JSON edit modal on an untouched object/array param (no value
yet in testArgValues) loaded an empty draft with jsonEditError hardcoded
to null, so Save was enabled despite invalid JSON and silently no-op'd on
click. Now runs the same JSON.parse check used by the live textarea
validation when the modal opens.
* chore: regenerate openapi.json for ToolTestResponse hint/request_* fields
drift-check on PR #547 was failing because the earlier hint/request_method/
request_url/request_body/request_params fields added to ToolTestResponse
were never reflected in the dumped spec. Regenerated via
scripts.dump_docs_openapi.
* fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner
httpx raises a TypeError when a query param value is a dict/list, which
was silently caught and surfaced as a generic tool-execution error —
this is what actually broke test requests, not just the "[object
Object]" display. JSON-stringify object/array arguments before they
become query params, in both the live execute_http_tool() path and the
test route's request_params display shaping.
Also adds an unsaved-changes warning banner above Test Tool, shown
when the live form state diverges from the last-saved HTTP API config,
since Test Tool always runs the saved config.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(tool-test): keep empty request body in preview; normalize headers in snapshot
- POST/PUT/PATCH with no arguments now shows `{}` in the request body
preview instead of null — matches what execute_http_tool actually sends
over the wire (json={})
- buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a
deduped key→value map before serializing, matching the shape saved to
the backend; duplicate header keys no longer cause a false unsaved-
changes warning
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(tool-test): update assertion for empty POST body preview
test_tool_test_no_arguments_leaves_body_and_params_none expected
request_body=None for a POST with no arguments. The fix to preserve {}
in the preview makes request_body={} the correct assertion.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tools): refine HTTP tool testing
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-07-18 16:00:50 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_tool_test_response_has_hint_and_request_fields():
|
|
|
|
|
"""ToolTestResponse must carry hint + request_method/url/body/params
|
|
|
|
|
so the frontend can show what was sent and why it may have failed."""
|
|
|
|
|
resp = ToolTestResponse(
|
|
|
|
|
status="error",
|
|
|
|
|
status_code=405,
|
|
|
|
|
data=None,
|
|
|
|
|
error="Method Not Allowed",
|
|
|
|
|
duration_ms=12,
|
|
|
|
|
hint="HTTP 405 Method Not Allowed — the endpoint rejected the configured method (POST).",
|
|
|
|
|
request_method="POST",
|
|
|
|
|
request_url="https://example.com/thing",
|
|
|
|
|
request_headers={"Authorization": "********oken"},
|
|
|
|
|
request_body={"a": 1},
|
|
|
|
|
request_params=None,
|
|
|
|
|
)
|
|
|
|
|
assert resp.hint.startswith("HTTP 405")
|
|
|
|
|
assert resp.request_method == "POST"
|
|
|
|
|
assert resp.request_url == "https://example.com/thing"
|
|
|
|
|
assert resp.request_headers == {"Authorization": "********oken"}
|
|
|
|
|
assert resp.request_body == {"a": 1}
|
|
|
|
|
assert resp.request_params is None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_tool_test_response_request_fields_default_to_none_or_required():
|
|
|
|
|
"""hint/request_body/request_params are optional; request_method/url are required."""
|
|
|
|
|
resp = ToolTestResponse(
|
|
|
|
|
status="success",
|
|
|
|
|
duration_ms=5,
|
|
|
|
|
request_method="GET",
|
|
|
|
|
request_url="https://example.com/thing",
|
|
|
|
|
)
|
|
|
|
|
assert resp.hint is None
|
|
|
|
|
assert resp.request_headers == {}
|
|
|
|
|
assert resp.request_body is None
|
|
|
|
|
assert resp.request_params is None
|