mirror of
https://github.com/dograh-hq/dograh.git
synced 2026-07-19 11:41:04 +02:00
* feat: add tool test panel for HTTP API tools
Lets developers run a saved HTTP API tool against its real endpoint
from the tool detail page, without needing a live call. Reuses the
production execute_http_tool path so test behavior matches call-time
behavior.
- New POST /tools/{tool_uuid}/test route
- Test panel with per-parameter typed inputs and auto-detected
context variable inputs (from preset parameter templates)
- Validate parameter name uniqueness on save, matching the existing
transferParameters check
- Fix stale FunctionCallsFromLLMInfoFrame import causing test
collection failures against pipecat-ai 1.5.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: revert local-venv pipecat drift fix, apply ruff import formatting
test_custom_tools.py and test_unregistered_function_call.py were edited
locally to drop FunctionCallsFromLLMInfoFrame after hitting an
ImportError — that error was from a stale local pipecat-ai package, not
a real drift. CI's pipecat build emits this frame and the test asserted
on it, so removing it broke test_llm_calls_custom_tool_handler and its
unregistered-call counterpart. Reverted both files to match main.
Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to
clear the drift-check job (split the aliased import into its own
`from ... import (...)` block, wrapped a long monkeypatch.setattr call).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec
pytest's default discovery matches any top-level test_* name in a test
module, including imported functions — importing the route handler as
`test_tool as test_tool_route` still matched the pattern, so pytest
tried to run it as a test and failed injecting fixtures for tool_uuid/
request/user. Renamed the alias to call_test_tool_route.
Also regenerated docs/api-reference/openapi.json for the new
POST /tools/{tool_uuid}/test route and its two schemas (couldn't run
the dump script locally — pipecat-ai version mismatch documented
separately — so hand-built the diff to exactly match FastAPI's
get_openapi() output format, verified against neighboring routes).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address cubic review findings on test panel
- Resolve dotted context-variable keys into nested objects before
posting the test request. render_template's get_nested_value walks
nested dicts, so a flat key like "runtime_configuration.realtime_model"
never matched — templates referencing nested context always resolved
to empty.
- Restrict isHttpApiTool to an explicit category equality check instead
of inferring it from exclusions. native/integration tools are
currently disabled in the create-tool UI so this wasn't reachable
today, but the exclusion list silently goes stale as new categories
are added.
- Stop showing a green success badge for non-2xx responses.
execute_http_tool returns status: "success" for any HTTP exchange
that completes, regardless of status code — only transport-level
errors (timeout, connection failure) get status: "error". The test
panel now checks status_code is in the 2xx range before treating the
call as a success.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address second round of cubic/greptile findings
- Guard setNestedValue against prototype-pollution keys (__proto__,
constructor, prototype) in the dotted context-var path before
traversing.
- Normalize status to "error" in the test route when the upstream
status_code is >= 400. execute_http_tool only distinguishes
transport-level failures (timeout, connection error) from
"success" — a completed 4xx/5xx exchange still came back as
"success" from the executor.
- Seed testArgValues defaults for number/boolean parameters via a
useEffect keyed on the parameters array, so a required number or
boolean field isn't silently omitted from the test request if the
user never touches its input.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: only seed test-arg defaults for required number/boolean params
Seeding optional number/boolean parameters silently changed the test
request — an optional boolean flag the tester never touched was sent
as true, which can flip upstream behavior unintentionally. Restrict
seeding to required parameters.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: match whitespace and fallback-filter syntax in context var detection
extractContextVars required an exact {{initial_context.foo}} with no
whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN
(and render_template) accepts {{ initial_context.foo }} and
{{initial_context.foo | fallback:value}}. A preset parameter saved with
either of those forms resolved fine in production but showed no input
in the test panel, so testing always sent it empty context.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add hint and request_* fields to ToolTestResponse
Extends ToolTestResponse with hint, request_method, request_url,
request_body, and request_params so the frontend can surface what was
actually sent and a human-readable hint about why a test call failed.
* feat(tool-test): add status-code hints and request_method/url/body/params to test_tool()
* style: ruff-format test_mcp_tool_route.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): wire hint banner and request block into result panel
Backend has returned hint/request_method/request_url/request_body/
request_params since d45ea851/60aaf31d but the frontend never
displayed them. Extends ToolTestResult with the new fields and renders
an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx)
plus a Request block above the response showing exactly what was sent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(tool-test): include resolved preset params in request_body/params
Found via live GET/POST testing: the Request preview showed only the
model-provided arguments, not what execute_http_tool actually sends.
execute_http_tool merges resolved_arguments = {**arguments,
**preset_arguments} before building the outbound body/params — preset
params (e.g. {{initial_context.metadata.channel}}) are invisible to
the model but still go out on the wire. The preview now mirrors that
merge via the same _resolve_preset_parameters helper, so a dev sees
exactly what was sent, not just what the model provided.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tool-test): add generateSampleValue helper for sample-fill button
* feat(tool-test): add Fill sample values button for arguments and context vars
Moved generateSampleValue out of page.tsx into a sibling helpers module:
Next.js's typed-route checker rejects extra named exports on a page.tsx
file (tsc error TS2344 on .next/types), so the helper and its test import
now live in testPanelHelpers.ts instead.
* feat(tool-test): add JSON edit modal state and handlers
* feat(tool-test): collapsed preview + edit modal for object/array test parameters
* fix(tool-test): validate JSON on modal open, not just on edit
Opening the JSON edit modal on an untouched object/array param (no value
yet in testArgValues) loaded an empty draft with jsonEditError hardcoded
to null, so Save was enabled despite invalid JSON and silently no-op'd on
click. Now runs the same JSON.parse check used by the live textarea
validation when the modal opens.
* chore: regenerate openapi.json for ToolTestResponse hint/request_* fields
drift-check on PR #547 was failing because the earlier hint/request_method/
request_url/request_body/request_params fields added to ToolTestResponse
were never reflected in the dumped spec. Regenerated via
scripts.dump_docs_openapi.
* fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner
httpx raises a TypeError when a query param value is a dict/list, which
was silently caught and surfaced as a generic tool-execution error —
this is what actually broke test requests, not just the "[object
Object]" display. JSON-stringify object/array arguments before they
become query params, in both the live execute_http_tool() path and the
test route's request_params display shaping.
Also adds an unsaved-changes warning banner above Test Tool, shown
when the live form state diverges from the last-saved HTTP API config,
since Test Tool always runs the saved config.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(tool-test): keep empty request body in preview; normalize headers in snapshot
- POST/PUT/PATCH with no arguments now shows `{}` in the request body
preview instead of null — matches what execute_http_tool actually sends
over the wire (json={})
- buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a
deduped key→value map before serializing, matching the shape saved to
the backend; duplicate header keys no longer cause a false unsaved-
changes warning
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(tool-test): update assertion for empty POST body preview
test_tool_test_no_arguments_leaves_body_and_params_none expected
request_body=None for a POST with no arguments. The fix to preserve {}
in the preview makes request_body={} the correct assertion.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tools): refine HTTP tool testing
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
457 lines
14 KiB
Python
457 lines
14 KiB
Python
"""API routes for managing tools."""
|
|
|
|
import time
|
|
from typing import List, Optional
|
|
|
|
from fastapi import APIRouter, Depends, HTTPException
|
|
|
|
from api.db import db_client
|
|
from api.db.models import UserModel
|
|
from api.enums import ToolCategory, ToolStatus
|
|
from api.schemas.tool import (
|
|
CalculatorToolDefinition,
|
|
CreatedByResponse,
|
|
CreateToolRequest,
|
|
EndCallConfig,
|
|
EndCallToolDefinition,
|
|
HttpApiConfig,
|
|
HttpApiToolDefinition,
|
|
McpRefreshResponse,
|
|
McpToolConfig,
|
|
McpToolDefinition,
|
|
PresetToolParameter,
|
|
ToolDefinition,
|
|
ToolParameter,
|
|
ToolResponse,
|
|
ToolTestRequest,
|
|
ToolTestResponse,
|
|
TransferCallConfig,
|
|
TransferCallToolDefinition,
|
|
UpdateToolRequest,
|
|
)
|
|
from api.sdk_expose import sdk_expose
|
|
from api.services.auth.depends import get_user
|
|
from api.services.tool_management import (
|
|
ToolManagementError,
|
|
build_tool_response,
|
|
create_tool_for_user,
|
|
refresh_mcp_tool_for_user,
|
|
validate_tool_credential_references,
|
|
)
|
|
from api.services.tool_management import (
|
|
populate_discovered_tools as _populate_discovered_tools,
|
|
)
|
|
from api.services.workflow.tools.custom_tool import (
|
|
execute_http_tool,
|
|
serialize_query_params,
|
|
)
|
|
|
|
router = APIRouter(prefix="/tools")
|
|
|
|
__all__ = [
|
|
"CalculatorToolDefinition",
|
|
"CreateToolRequest",
|
|
"CreatedByResponse",
|
|
"EndCallConfig",
|
|
"EndCallToolDefinition",
|
|
"HttpApiConfig",
|
|
"HttpApiToolDefinition",
|
|
"McpRefreshResponse",
|
|
"McpToolConfig",
|
|
"McpToolDefinition",
|
|
"PresetToolParameter",
|
|
"ToolDefinition",
|
|
"ToolParameter",
|
|
"ToolResponse",
|
|
"ToolTestRequest",
|
|
"ToolTestResponse",
|
|
"TransferCallConfig",
|
|
"TransferCallToolDefinition",
|
|
"UpdateToolRequest",
|
|
"_populate_discovered_tools",
|
|
]
|
|
|
|
|
|
def validate_category(category: str) -> None:
|
|
"""Validate that the category is valid."""
|
|
valid_categories = [c.value for c in ToolCategory]
|
|
if category not in valid_categories:
|
|
raise HTTPException(
|
|
status_code=400,
|
|
detail=f"Invalid category '{category}'. Must be one of: {', '.join(valid_categories)}",
|
|
)
|
|
|
|
|
|
def validate_status(status: str) -> None:
|
|
"""Validate that the status is valid. Supports comma-separated values."""
|
|
valid_statuses = [s.value for s in ToolStatus]
|
|
status_list = [s.strip() for s in status.split(",")]
|
|
for s in status_list:
|
|
if s not in valid_statuses:
|
|
raise HTTPException(
|
|
status_code=400,
|
|
detail=f"Invalid status '{s}'. Must be one of: {', '.join(valid_statuses)}",
|
|
)
|
|
|
|
|
|
@router.get(
|
|
"/",
|
|
**sdk_expose(
|
|
method="list_tools",
|
|
description="List tools available to the authenticated organization.",
|
|
),
|
|
)
|
|
async def list_tools(
|
|
status: Optional[str] = None,
|
|
category: Optional[str] = None,
|
|
user: UserModel = Depends(get_user),
|
|
) -> List[ToolResponse]:
|
|
"""
|
|
List all tools for the user's organization.
|
|
|
|
Args:
|
|
status: Optional filter by status (active, archived, draft)
|
|
category: Optional filter by category (http_api, native, integration)
|
|
|
|
Returns:
|
|
List of tools
|
|
"""
|
|
if not user.selected_organization_id:
|
|
raise HTTPException(
|
|
status_code=400, detail="No organization selected for the user"
|
|
)
|
|
|
|
if status:
|
|
validate_status(status)
|
|
if category:
|
|
validate_category(category)
|
|
|
|
tools = await db_client.get_tools_for_organization(
|
|
user.selected_organization_id,
|
|
status=status,
|
|
category=category,
|
|
)
|
|
|
|
return [build_tool_response(tool) for tool in tools]
|
|
|
|
|
|
@router.post(
|
|
"/",
|
|
**sdk_expose(
|
|
method="create_tool",
|
|
description="Create a reusable tool for the authenticated organization.",
|
|
),
|
|
)
|
|
async def create_tool(
|
|
request: CreateToolRequest,
|
|
user: UserModel = Depends(get_user),
|
|
) -> ToolResponse:
|
|
"""
|
|
Create a new tool.
|
|
|
|
Args:
|
|
request: The tool creation request
|
|
|
|
Returns:
|
|
The created tool
|
|
"""
|
|
try:
|
|
return await create_tool_for_user(request, user, source="api")
|
|
except ToolManagementError as e:
|
|
raise HTTPException(status_code=e.status_code, detail=e.message) from e
|
|
|
|
|
|
@router.get("/{tool_uuid}")
|
|
async def get_tool(
|
|
tool_uuid: str,
|
|
user: UserModel = Depends(get_user),
|
|
) -> ToolResponse:
|
|
"""
|
|
Get a specific tool by UUID.
|
|
|
|
Args:
|
|
tool_uuid: The UUID of the tool
|
|
|
|
Returns:
|
|
The tool
|
|
"""
|
|
if not user.selected_organization_id:
|
|
raise HTTPException(
|
|
status_code=400, detail="No organization selected for the user"
|
|
)
|
|
|
|
tool = await db_client.get_tool_by_uuid(
|
|
tool_uuid, user.selected_organization_id, include_archived=True
|
|
)
|
|
|
|
if not tool:
|
|
raise HTTPException(status_code=404, detail="Tool not found")
|
|
|
|
return build_tool_response(tool, include_created_by=True)
|
|
|
|
|
|
@router.post("/{tool_uuid}/mcp/refresh")
|
|
async def refresh_mcp_tools(
|
|
tool_uuid: str,
|
|
user: UserModel = Depends(get_user),
|
|
) -> McpRefreshResponse:
|
|
"""Re-discover an MCP tool's server catalog and overwrite the cached
|
|
``definition.config.discovered_tools``. Server down → 200 with error
|
|
(cache not overwritten on transient failure)."""
|
|
try:
|
|
return await refresh_mcp_tool_for_user(tool_uuid, user)
|
|
except ToolManagementError as e:
|
|
raise HTTPException(status_code=e.status_code, detail=e.message) from e
|
|
|
|
|
|
@router.post("/{tool_uuid}/test")
|
|
async def test_tool(
|
|
tool_uuid: str,
|
|
request: ToolTestRequest,
|
|
user: UserModel = Depends(get_user),
|
|
) -> ToolTestResponse:
|
|
"""Execute an HTTP API tool with sample LLM and preset parameters."""
|
|
if not user.selected_organization_id:
|
|
raise HTTPException(
|
|
status_code=400, detail="No organization selected for the user"
|
|
)
|
|
|
|
tool = await db_client.get_tool_by_uuid(
|
|
tool_uuid, user.selected_organization_id, include_archived=True
|
|
)
|
|
|
|
if not tool:
|
|
raise HTTPException(status_code=404, detail="Tool not found")
|
|
|
|
if tool.category != ToolCategory.HTTP_API.value:
|
|
raise HTTPException(status_code=400, detail="Only HTTP API tools can be tested")
|
|
|
|
tool_config = (
|
|
tool.definition.get("config", {}) if isinstance(tool.definition, dict) else {}
|
|
)
|
|
configured_method = tool_config.get("method", "?")
|
|
configured_url = tool_config.get("url", "?")
|
|
|
|
started_at = time.perf_counter()
|
|
result = await execute_http_tool(
|
|
tool,
|
|
request.llm_params,
|
|
preset_params=request.preset_params,
|
|
organization_id=user.selected_organization_id,
|
|
include_request_headers=True,
|
|
)
|
|
duration_ms = max(0, round((time.perf_counter() - started_at) * 1000))
|
|
|
|
status = result.get("status", "error")
|
|
status_code = result.get("status_code")
|
|
if status_code is not None and status_code >= 400:
|
|
status = "error"
|
|
|
|
hint = _hint_for_status_code(status_code, configured_method)
|
|
|
|
# Preset values take precedence over model-supplied values, matching live
|
|
# execution after configured preset templates have been resolved.
|
|
resolved_arguments = {**request.llm_params, **request.preset_params}
|
|
|
|
# Mirror execute_http_tool's own branch: POST/PUT/PATCH send the
|
|
# resolved arguments as a JSON body; GET/DELETE send them as query
|
|
# params. Never both.
|
|
request_body = None
|
|
request_params = None
|
|
if configured_method in ("POST", "PUT", "PATCH"):
|
|
request_body = resolved_arguments # keep {} so preview matches wire request
|
|
elif resolved_arguments:
|
|
request_params = serialize_query_params(resolved_arguments)
|
|
|
|
return ToolTestResponse(
|
|
status=status,
|
|
status_code=status_code,
|
|
data=result.get("data"),
|
|
error=result.get("error"),
|
|
duration_ms=duration_ms,
|
|
hint=hint,
|
|
request_method=configured_method,
|
|
request_url=configured_url,
|
|
request_headers=result.get("request_headers", {}),
|
|
request_body=request_body,
|
|
request_params=request_params,
|
|
)
|
|
|
|
|
|
def _hint_for_status_code(
|
|
status_code: Optional[int], configured_method: str
|
|
) -> Optional[str]:
|
|
"""Human-readable explanation for a status code a misconfigured tool
|
|
is likely to hit. Returns None for 2xx and any code not covered."""
|
|
if status_code == 400:
|
|
return (
|
|
"HTTP 400 Bad Request — the server rejected the request payload. "
|
|
"Verify the arguments/body match what this endpoint expects."
|
|
)
|
|
if status_code == 401:
|
|
return (
|
|
"HTTP 401 Unauthorized — the request wasn't authenticated. Check "
|
|
"the credential configured on the Authentication tab is present "
|
|
"and valid."
|
|
)
|
|
if status_code == 403:
|
|
return (
|
|
"HTTP 403 Forbidden — authenticated, but the configured "
|
|
"credential doesn't have permission for this endpoint/action."
|
|
)
|
|
if status_code == 404:
|
|
return (
|
|
f"HTTP 404 Not Found — verify the endpoint URL is correct and "
|
|
f"that {configured_method} is a valid method for it."
|
|
)
|
|
if status_code == 405:
|
|
return (
|
|
f"HTTP 405 Method Not Allowed — the endpoint rejected the "
|
|
f"configured method ({configured_method}). Verify the API expects "
|
|
f"{configured_method} for this URL."
|
|
)
|
|
if status_code == 408:
|
|
return (
|
|
"HTTP 408 Request Timeout — the endpoint didn't respond in time. "
|
|
"Check the endpoint is reachable, or increase Timeout (ms) if it's "
|
|
"just slow."
|
|
)
|
|
if status_code == 409:
|
|
return (
|
|
"HTTP 409 Conflict — the endpoint rejected the request due to a "
|
|
"conflicting resource state (e.g. duplicate create). Not "
|
|
"necessarily a configuration problem."
|
|
)
|
|
if status_code == 415:
|
|
return (
|
|
"HTTP 415 Unsupported Media Type — check the Content-Type header "
|
|
"matches the format this endpoint expects for the body."
|
|
)
|
|
if status_code == 422:
|
|
return (
|
|
"HTTP 422 Unprocessable Entity — the request was well-formed but "
|
|
"the payload's structure or field types don't match what this "
|
|
"endpoint expects. Compare your arguments against the API's "
|
|
"documented schema."
|
|
)
|
|
if status_code == 429:
|
|
return (
|
|
"HTTP 429 Too Many Requests — the endpoint is rate-limiting. Wait "
|
|
"and retry; not a configuration problem."
|
|
)
|
|
if status_code is not None and 500 <= status_code < 600:
|
|
return (
|
|
f"HTTP {status_code} — the endpoint itself errored. This is "
|
|
"likely an issue on the API's side, not your tool configuration."
|
|
)
|
|
return None
|
|
|
|
|
|
@router.put("/{tool_uuid}")
|
|
async def update_tool(
|
|
tool_uuid: str,
|
|
request: UpdateToolRequest,
|
|
user: UserModel = Depends(get_user),
|
|
) -> ToolResponse:
|
|
"""
|
|
Update a tool.
|
|
|
|
Args:
|
|
tool_uuid: The UUID of the tool to update
|
|
request: The update request
|
|
|
|
Returns:
|
|
The updated tool
|
|
"""
|
|
if not user.selected_organization_id:
|
|
raise HTTPException(
|
|
status_code=400, detail="No organization selected for the user"
|
|
)
|
|
|
|
if request.status:
|
|
validate_status(request.status)
|
|
|
|
definition = None
|
|
if request.definition:
|
|
definition = request.definition.model_dump()
|
|
try:
|
|
await validate_tool_credential_references(
|
|
definition,
|
|
organization_id=user.selected_organization_id,
|
|
)
|
|
definition = await _populate_discovered_tools(
|
|
definition,
|
|
organization_id=user.selected_organization_id,
|
|
)
|
|
except ToolManagementError as e:
|
|
raise HTTPException(status_code=e.status_code, detail=e.message) from e
|
|
|
|
tool = await db_client.update_tool(
|
|
tool_uuid=tool_uuid,
|
|
organization_id=user.selected_organization_id,
|
|
name=request.name,
|
|
description=request.description,
|
|
definition=definition,
|
|
icon=request.icon,
|
|
icon_color=request.icon_color,
|
|
status=request.status,
|
|
)
|
|
|
|
if not tool:
|
|
raise HTTPException(status_code=404, detail="Tool not found")
|
|
|
|
return build_tool_response(tool, include_created_by=True)
|
|
|
|
|
|
@router.delete("/{tool_uuid}")
|
|
async def delete_tool(
|
|
tool_uuid: str,
|
|
user: UserModel = Depends(get_user),
|
|
) -> dict:
|
|
"""
|
|
Archive (soft delete) a tool.
|
|
|
|
Args:
|
|
tool_uuid: The UUID of the tool to delete
|
|
|
|
Returns:
|
|
Success message
|
|
"""
|
|
if not user.selected_organization_id:
|
|
raise HTTPException(
|
|
status_code=400, detail="No organization selected for the user"
|
|
)
|
|
|
|
deleted = await db_client.archive_tool(tool_uuid, user.selected_organization_id)
|
|
|
|
if not deleted:
|
|
raise HTTPException(status_code=404, detail="Tool not found")
|
|
|
|
return {"status": "archived", "tool_uuid": tool_uuid}
|
|
|
|
|
|
@router.post("/{tool_uuid}/unarchive")
|
|
async def unarchive_tool(
|
|
tool_uuid: str,
|
|
user: UserModel = Depends(get_user),
|
|
) -> ToolResponse:
|
|
"""
|
|
Unarchive a tool (restore from archived state).
|
|
|
|
Args:
|
|
tool_uuid: The UUID of the tool to unarchive
|
|
|
|
Returns:
|
|
The unarchived tool
|
|
"""
|
|
if not user.selected_organization_id:
|
|
raise HTTPException(
|
|
status_code=400, detail="No organization selected for the user"
|
|
)
|
|
|
|
tool = await db_client.unarchive_tool(tool_uuid, user.selected_organization_id)
|
|
|
|
if not tool:
|
|
raise HTTPException(status_code=404, detail="Tool not found")
|
|
|
|
return build_tool_response(tool)
|