feat: add tool test panel for HTTP API tools (#547)

* feat: add tool test panel for HTTP API tools

Lets developers run a saved HTTP API tool against its real endpoint
from the tool detail page, without needing a live call. Reuses the
production execute_http_tool path so test behavior matches call-time
behavior.

- New POST /tools/{tool_uuid}/test route
- Test panel with per-parameter typed inputs and auto-detected
  context variable inputs (from preset parameter templates)
- Validate parameter name uniqueness on save, matching the existing
  transferParameters check
- Fix stale FunctionCallsFromLLMInfoFrame import causing test
  collection failures against pipecat-ai 1.5.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: revert local-venv pipecat drift fix, apply ruff import formatting

test_custom_tools.py and test_unregistered_function_call.py were edited
locally to drop FunctionCallsFromLLMInfoFrame after hitting an
ImportError — that error was from a stale local pipecat-ai package, not
a real drift. CI's pipecat build emits this frame and the test asserted
on it, so removing it broke test_llm_calls_custom_tool_handler and its
unregistered-call counterpart. Reverted both files to match main.

Also applied ruff's import-sort/format fix to test_mcp_tool_route.py to
clear the drift-check job (split the aliased import into its own
`from ... import (...)` block, wrapped a long monkeypatch.setattr call).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: avoid pytest collecting test_tool import as a test, sync OpenAPI spec

pytest's default discovery matches any top-level test_* name in a test
module, including imported functions — importing the route handler as
`test_tool as test_tool_route` still matched the pattern, so pytest
tried to run it as a test and failed injecting fixtures for tool_uuid/
request/user. Renamed the alias to call_test_tool_route.

Also regenerated docs/api-reference/openapi.json for the new
POST /tools/{tool_uuid}/test route and its two schemas (couldn't run
the dump script locally — pipecat-ai version mismatch documented
separately — so hand-built the diff to exactly match FastAPI's
get_openapi() output format, verified against neighboring routes).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address cubic review findings on test panel

- Resolve dotted context-variable keys into nested objects before
  posting the test request. render_template's get_nested_value walks
  nested dicts, so a flat key like "runtime_configuration.realtime_model"
  never matched — templates referencing nested context always resolved
  to empty.
- Restrict isHttpApiTool to an explicit category equality check instead
  of inferring it from exclusions. native/integration tools are
  currently disabled in the create-tool UI so this wasn't reachable
  today, but the exclusion list silently goes stale as new categories
  are added.
- Stop showing a green success badge for non-2xx responses.
  execute_http_tool returns status: "success" for any HTTP exchange
  that completes, regardless of status code — only transport-level
  errors (timeout, connection failure) get status: "error". The test
  panel now checks status_code is in the 2xx range before treating the
  call as a success.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address second round of cubic/greptile findings

- Guard setNestedValue against prototype-pollution keys (__proto__,
  constructor, prototype) in the dotted context-var path before
  traversing.
- Normalize status to "error" in the test route when the upstream
  status_code is >= 400. execute_http_tool only distinguishes
  transport-level failures (timeout, connection error) from
  "success" — a completed 4xx/5xx exchange still came back as
  "success" from the executor.
- Seed testArgValues defaults for number/boolean parameters via a
  useEffect keyed on the parameters array, so a required number or
  boolean field isn't silently omitted from the test request if the
  user never touches its input.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: only seed test-arg defaults for required number/boolean params

Seeding optional number/boolean parameters silently changed the test
request — an optional boolean flag the tester never touched was sent
as true, which can flip upstream behavior unintentionally. Restrict
seeding to required parameters.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: match whitespace and fallback-filter syntax in context var detection

extractContextVars required an exact {{initial_context.foo}} with no
whitespace and no filter suffix, but the backend's TEMPLATE_VAR_PATTERN
(and render_template) accepts {{ initial_context.foo }} and
{{initial_context.foo | fallback:value}}. A preset parameter saved with
either of those forms resolved fine in production but showed no input
in the test panel, so testing always sent it empty context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tool-test): add hint and request_* fields to ToolTestResponse

Extends ToolTestResponse with hint, request_method, request_url,
request_body, and request_params so the frontend can surface what was
actually sent and a human-readable hint about why a test call failed.

* feat(tool-test): add status-code hints and request_method/url/body/params to test_tool()

* style: ruff-format test_mcp_tool_route.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tool-test): wire hint banner and request block into result panel

Backend has returned hint/request_method/request_url/request_body/
request_params since d45ea851/60aaf31d but the frontend never
displayed them. Extends ToolTestResult with the new fields and renders
an amber hint banner (for 400/401/403/404/405/408/409/415/422/429/5xx)
plus a Request block above the response showing exactly what was sent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(tool-test): include resolved preset params in request_body/params

Found via live GET/POST testing: the Request preview showed only the
model-provided arguments, not what execute_http_tool actually sends.
execute_http_tool merges resolved_arguments = {**arguments,
**preset_arguments} before building the outbound body/params — preset
params (e.g. {{initial_context.metadata.channel}}) are invisible to
the model but still go out on the wire. The preview now mirrors that
merge via the same _resolve_preset_parameters helper, so a dev sees
exactly what was sent, not just what the model provided.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tool-test): add generateSampleValue helper for sample-fill button

* feat(tool-test): add Fill sample values button for arguments and context vars

Moved generateSampleValue out of page.tsx into a sibling helpers module:
Next.js's typed-route checker rejects extra named exports on a page.tsx
file (tsc error TS2344 on .next/types), so the helper and its test import
now live in testPanelHelpers.ts instead.

* feat(tool-test): add JSON edit modal state and handlers

* feat(tool-test): collapsed preview + edit modal for object/array test parameters

* fix(tool-test): validate JSON on modal open, not just on edit

Opening the JSON edit modal on an untouched object/array param (no value
yet in testArgValues) loaded an empty draft with jsonEditError hardcoded
to null, so Save was enabled despite invalid JSON and silently no-op'd on
click. Now runs the same JSON.parse check used by the live textarea
validation when the modal opens.

* chore: regenerate openapi.json for ToolTestResponse hint/request_* fields

drift-check on PR #547 was failing because the earlier hint/request_method/
request_url/request_body/request_params fields added to ToolTestResponse
were never reflected in the dumped spec. Regenerated via
scripts.dump_docs_openapi.

* fix(tool-test): serialize object/array args for GET/DELETE query params, add unsaved-changes banner

httpx raises a TypeError when a query param value is a dict/list, which
was silently caught and surfaced as a generic tool-execution error —
this is what actually broke test requests, not just the "[object
Object]" display. JSON-stringify object/array arguments before they
become query params, in both the live execute_http_tool() path and the
test route's request_params display shaping.

Also adds an unsaved-changes warning banner above Test Tool, shown
when the live form state diverges from the last-saved HTTP API config,
since Test Tool always runs the saved config.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(tool-test): keep empty request body in preview; normalize headers in snapshot

- POST/PUT/PATCH with no arguments now shows `{}` in the request body
  preview instead of null — matches what execute_http_tool actually sends
  over the wire (json={})
- buildHttpToolTestSnapshot normalizes headers from KeyValueItem[] to a
  deduped key→value map before serializing, matching the shape saved to
  the backend; duplicate header keys no longer cause a false unsaved-
  changes warning

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(tool-test): update assertion for empty POST body preview

test_tool_test_no_arguments_leaves_body_and_params_none expected
request_body=None for a POST with no arguments. The fix to preserve {}
in the preview makes request_body={} the correct assertion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tools): refine HTTP tool testing

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
This commit is contained in:
Rushil 2026-07-18 16:00:50 +05:30 committed by GitHub
parent 104dc56706
commit f4c5954a39
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
15 changed files with 1757 additions and 67 deletions

View file

@ -0,0 +1,540 @@
"use client";
import { AlertTriangle, Loader2, Pencil } from "lucide-react";
import { useEffect, useState } from "react";
import { testToolApiV1ToolsToolUuidTestPost } from "@/client/sdk.gen";
import type { ToolTestResponse } from "@/client/types.gen";
import type { HttpMethod, PresetToolParameter, ToolParameter } from "@/components/http";
import { Button } from "@/components/ui/button";
import {
Dialog,
DialogContent,
DialogDescription,
DialogHeader,
DialogTitle,
} from "@/components/ui/dialog";
import { Input } from "@/components/ui/input";
import { Label } from "@/components/ui/label";
import { detailFromError } from "@/lib/apiError";
import { useAuth } from "@/lib/auth";
import {
generateSampleValue,
isUnsafeHttpMethod,
parseTestParameterValues,
} from "./helpers";
type HttpToolTestDialogProps = {
open: boolean;
onOpenChange: (open: boolean) => void;
toolUuid: string;
httpMethod: HttpMethod;
url: string;
parameters: ToolParameter[];
presetParameters: PresetToolParameter[];
};
type TestParameter = ToolParameter | PresetToolParameter;
type ParameterSource = "llm" | "preset";
type JsonEditTarget = {
source: ParameterSource;
name: string;
};
type ParameterFieldsProps = {
idPrefix: string;
parameters: TestParameter[];
values: Record<string, string>;
onValueChange: (name: string, value: string) => void;
onEditJson: (name: string) => void;
};
function defaultInputValue(parameter: TestParameter): string {
if (
"valueTemplate" in parameter &&
parameter.valueTemplate &&
!parameter.valueTemplate.includes("{{")
) {
return parameter.valueTemplate;
}
switch (parameter.type) {
case "number":
return "0";
case "boolean":
return "true";
case "object":
return "{}";
case "array":
return "[]";
default:
return "";
}
}
function seedMissingValues(
previous: Record<string, string>,
parameters: TestParameter[]
): Record<string, string> {
const next = { ...previous };
let changed = false;
for (const parameter of parameters) {
if (!parameter.name || parameter.name in next) continue;
next[parameter.name] = defaultInputValue(parameter);
changed = true;
}
return changed ? next : previous;
}
function ParameterFields({
idPrefix,
parameters,
values,
onValueChange,
onEditJson,
}: ParameterFieldsProps) {
if (parameters.length === 0) {
return <p className="text-sm text-muted-foreground">No parameters configured.</p>;
}
return (
<div className="space-y-4">
{parameters.map((parameter) => {
const inputId = `${idPrefix}-${parameter.name}`;
const value = values[parameter.name] ?? defaultInputValue(parameter);
return (
<div key={parameter.name} className="space-y-1.5">
<div className="flex items-center gap-2">
<Label htmlFor={inputId} className="text-sm font-mono">
{parameter.name}
</Label>
<span className="rounded bg-muted px-1.5 py-0.5 text-xs text-muted-foreground">
{parameter.type}
</span>
{parameter.required && <span className="text-xs text-destructive">required</span>}
</div>
{"description" in parameter && parameter.description && (
<p className="text-xs text-muted-foreground">{parameter.description}</p>
)}
{"valueTemplate" in parameter && parameter.valueTemplate && (
<p className="break-all text-xs text-muted-foreground">
Configured preset: <code>{parameter.valueTemplate}</code>
</p>
)}
{parameter.type === "boolean" ? (
<select
id={inputId}
value={value}
onChange={(event) => onValueChange(parameter.name, event.target.value)}
className="flex h-9 w-full rounded-md border border-input bg-background px-3 py-1 text-sm shadow-sm"
>
<option value="true">true</option>
<option value="false">false</option>
</select>
) : parameter.type === "number" ? (
<Input
id={inputId}
type="number"
value={value}
onChange={(event) => onValueChange(parameter.name, event.target.value)}
/>
) : parameter.type === "object" || parameter.type === "array" ? (
<div className="flex items-center gap-2">
<button
type="button"
onClick={() => onEditJson(parameter.name)}
className="h-9 flex-1 truncate rounded-md border border-input bg-background px-3 py-1 text-left font-mono text-sm shadow-sm hover:bg-accent"
>
{!value || value === (parameter.type === "array" ? "[]" : "{}") ? (
<span className="text-muted-foreground">Empty</span>
) : (
value
)}
</button>
<Button
type="button"
variant="outline"
size="icon"
onClick={() => onEditJson(parameter.name)}
aria-label={`Edit ${parameter.name}`}
>
<Pencil className="h-4 w-4" />
</Button>
</div>
) : (
<Input
id={inputId}
value={value}
onChange={(event) => onValueChange(parameter.name, event.target.value)}
placeholder={`Enter ${parameter.name}`}
/>
)}
</div>
);
})}
</div>
);
}
export function HttpToolTestDialog({
open,
onOpenChange,
toolUuid,
httpMethod,
url,
parameters,
presetParameters,
}: HttpToolTestDialogProps) {
const { getAccessToken } = useAuth();
const [llmParamValues, setLlmParamValues] = useState<Record<string, string>>({});
const [presetParamValues, setPresetParamValues] = useState<Record<string, string>>({});
const [result, setResult] = useState<ToolTestResponse | null>(null);
const [isTesting, setIsTesting] = useState(false);
const [testError, setTestError] = useState<string | null>(null);
const [jsonEditTarget, setJsonEditTarget] = useState<JsonEditTarget | null>(null);
const [jsonEditDraft, setJsonEditDraft] = useState("");
const [jsonEditError, setJsonEditError] = useState<string | null>(null);
useEffect(() => {
setLlmParamValues((previous) => seedMissingValues(previous, parameters));
}, [parameters]);
useEffect(() => {
setPresetParamValues((previous) => seedMissingValues(previous, presetParameters));
}, [presetParameters]);
const handleFillSampleValues = () => {
setLlmParamValues((previous) => {
const next = { ...previous };
for (const parameter of parameters) {
next[parameter.name] = generateSampleValue(parameter.type);
}
return next;
});
setPresetParamValues((previous) => {
const next = { ...previous };
for (const parameter of presetParameters) {
next[parameter.name] = generateSampleValue(parameter.type);
}
return next;
});
};
const closeJsonEditDialog = () => {
setJsonEditTarget(null);
setJsonEditDraft("");
setJsonEditError(null);
};
const openJsonEditDialog = (source: ParameterSource, parameterName: string) => {
const values = source === "llm" ? llmParamValues : presetParamValues;
const current = values[parameterName] ?? "";
let draft = current;
try {
draft = JSON.stringify(JSON.parse(current), null, 2);
} catch {
// Keep incomplete or invalid JSON as-is. Validation is triggered
// only when the user explicitly chooses Save or Format JSON.
}
setJsonEditTarget({ source, name: parameterName });
setJsonEditDraft(draft);
setJsonEditError(null);
};
const handleJsonEditDraftChange = (value: string) => {
setJsonEditDraft(value);
setJsonEditError(null);
};
const handleFormatJson = () => {
try {
const parsed = JSON.parse(jsonEditDraft);
setJsonEditDraft(JSON.stringify(parsed, null, 2));
setJsonEditError(null);
} catch (caughtError) {
setJsonEditError(caughtError instanceof Error ? caughtError.message : "Invalid JSON");
}
};
const handleSaveJsonEdit = () => {
if (jsonEditTarget === null) return;
const target = jsonEditTarget;
try {
const parsed = JSON.parse(jsonEditDraft);
const setValues = target.source === "llm" ? setLlmParamValues : setPresetParamValues;
setValues((previous) => ({ ...previous, [target.name]: JSON.stringify(parsed) }));
closeJsonEditDialog();
} catch (caughtError) {
setJsonEditError(caughtError instanceof Error ? caughtError.message : "Invalid JSON");
}
};
const handleTestTool = async () => {
try {
setIsTesting(true);
setTestError(null);
setResult(null);
const llmParams = parseTestParameterValues(parameters, llmParamValues);
const presetParams = parseTestParameterValues(presetParameters, presetParamValues);
const accessToken = await getAccessToken();
const response = await testToolApiV1ToolsToolUuidTestPost({
path: { tool_uuid: toolUuid },
headers: { Authorization: `Bearer ${accessToken}` },
body: {
llm_params: llmParams,
preset_params: presetParams,
},
});
if (response.error) {
setTestError(detailFromError(response.error, "Failed to test tool"));
return;
}
if (response.data) setResult(response.data);
} catch (caughtError) {
setTestError(caughtError instanceof Error ? caughtError.message : "Failed to test tool");
} finally {
setIsTesting(false);
}
};
const isSuccess =
result?.status === "success" &&
(result.status_code == null || (result.status_code >= 200 && result.status_code < 300));
const resultBadgeLabel = isSuccess ? "success" : result?.status === "success" ? "failed" : "error";
return (
<>
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className="max-h-[90vh] max-w-3xl grid-rows-[auto_minmax(0,1fr)]">
<DialogHeader>
<DialogTitle>Test Tool</DialogTitle>
<DialogDescription>
Run the saved configuration against the real endpoint.
</DialogDescription>
</DialogHeader>
<div className="space-y-6 overflow-y-auto pr-1">
<div className="rounded-lg border bg-muted/40 p-4">
<p className="mb-2 text-xs font-medium uppercase tracking-wide text-muted-foreground">
Request
</p>
<div className="flex items-start gap-3">
<span className="rounded bg-foreground px-2 py-1 font-mono text-xs font-semibold text-background">
{httpMethod}
</span>
<code className="break-all pt-0.5 text-sm">{url}</code>
</div>
</div>
{isUnsafeHttpMethod(httpMethod) && (
<div
role="alert"
className="flex gap-3 rounded-lg border border-amber-300 bg-amber-50 p-4 text-amber-900 dark:border-amber-800 dark:bg-amber-950/40 dark:text-amber-200"
>
<AlertTriangle className="mt-0.5 h-5 w-5 shrink-0" />
<div className="space-y-1">
<p className="text-sm font-medium">This performs a real external request</p>
<p className="text-sm">
Testing sends an actual {httpMethod} request to this endpoint. Any configured
credential is used for the request, and the operation may modify external data.
</p>
</div>
</div>
)}
<div className="space-y-3">
<div className="flex items-center justify-between">
<div>
<p className="text-sm font-medium">Parameters</p>
<p className="text-xs text-muted-foreground">
Supply the values that would normally come from the model and configured presets.
</p>
</div>
{(parameters.length > 0 || presetParameters.length > 0) && (
<Button
type="button"
variant="outline"
size="sm"
onClick={handleFillSampleValues}
>
Fill sample values
</Button>
)}
</div>
</div>
<div className="space-y-3 border-t pt-4">
<div>
<p className="text-sm font-medium">LLM Parameters</p>
<p className="text-xs text-muted-foreground">
Values the model would provide at call time.
</p>
</div>
<ParameterFields
idPrefix="llm-param"
parameters={parameters}
values={llmParamValues}
onValueChange={(name, value) =>
setLlmParamValues((previous) => ({ ...previous, [name]: value }))
}
onEditJson={(name) => openJsonEditDialog("llm", name)}
/>
</div>
<div className="space-y-3 border-t pt-4">
<div>
<p className="text-sm font-medium">Preset Parameters</p>
<p className="text-xs text-muted-foreground">
Resolved values that Dograh would normally derive from each configured preset.
</p>
</div>
<ParameterFields
idPrefix="preset-param"
parameters={presetParameters}
values={presetParamValues}
onValueChange={(name, value) =>
setPresetParamValues((previous) => ({ ...previous, [name]: value }))
}
onEditJson={(name) => openJsonEditDialog("preset", name)}
/>
</div>
<div className="flex justify-end">
<Button onClick={handleTestTool} disabled={isTesting}>
{isTesting ? (
<>
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
Testing...
</>
) : (
"Test Tool"
)}
</Button>
</div>
{testError && (
<div className="rounded-lg border border-destructive/20 bg-destructive/10 p-3 text-sm text-destructive">
{testError}
</div>
)}
{result && (
<div className="space-y-3 border-t pt-4">
{(result.request_method || result.request_url) && (
<div className="space-y-1 overflow-auto rounded-lg bg-muted p-3 font-mono text-xs">
<p className="font-medium text-foreground">
{result.request_method} {result.request_url}
</p>
{result.request_headers && Object.keys(result.request_headers).length > 0 && (
<pre className="whitespace-pre-wrap">
Headers: {JSON.stringify(result.request_headers, null, 2)}
</pre>
)}
{result.request_body != null && (
<pre className="whitespace-pre-wrap">
Body: {JSON.stringify(result.request_body, null, 2)}
</pre>
)}
{result.request_params != null && (
<pre className="whitespace-pre-wrap">
Query:{" "}
{Object.entries(result.request_params)
.map(([key, value]) => `${key}=${value}`)
.join(" ")}
</pre>
)}
</div>
)}
<div className="flex items-center gap-3">
<span
className={`inline-flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium ${
isSuccess
? "bg-green-100 text-green-800 dark:bg-green-900/30 dark:text-green-400"
: "bg-red-100 text-red-800 dark:bg-red-900/30 dark:text-red-400"
}`}
>
<span>{isSuccess ? "✓" : "✗"}</span>
{resultBadgeLabel}
</span>
{result.status_code != null && (
<span className="text-sm text-muted-foreground">HTTP {result.status_code}</span>
)}
{result.duration_ms !== undefined && (
<span className="rounded bg-muted px-2 py-0.5 font-mono text-xs text-muted-foreground">
{result.duration_ms}ms
</span>
)}
</div>
{result.hint && (
<div className="rounded border border-amber-200 bg-amber-100 p-3 text-sm text-amber-900 dark:border-amber-900/40 dark:bg-amber-900/20 dark:text-amber-400">
{result.hint}
</div>
)}
{result.error && (
<div className="rounded border border-destructive/20 bg-destructive/10 p-3 text-sm text-destructive">
{result.error}
</div>
)}
{result.data != null && (
<div className="max-h-80 overflow-auto rounded-lg bg-muted p-4 font-mono text-sm">
<pre>{JSON.stringify(result.data, null, 2)}</pre>
</div>
)}
</div>
)}
</div>
</DialogContent>
</Dialog>
<Dialog
open={jsonEditTarget !== null}
onOpenChange={(isOpen) => {
if (!isOpen) closeJsonEditDialog();
}}
>
<DialogContent className="max-w-2xl">
<DialogHeader>
<DialogTitle>Edit {jsonEditTarget?.name}</DialogTitle>
<DialogDescription>
Edit the JSON value sent for this parameter when testing.
</DialogDescription>
</DialogHeader>
<textarea
value={jsonEditDraft}
onChange={(event) => handleJsonEditDraftChange(event.target.value)}
rows={12}
className="w-full resize-y rounded-md border border-input bg-background px-3 py-2 font-mono text-sm shadow-sm"
spellCheck={false}
/>
{jsonEditError && <p className="text-sm text-destructive">{jsonEditError}</p>}
<div className="flex justify-end gap-2">
<Button
type="button"
variant="outline"
onClick={handleFormatJson}
disabled={jsonEditDraft.length === 0}
>
Format JSON
</Button>
<Button type="button" variant="outline" onClick={closeJsonEditDialog}>
Cancel
</Button>
<Button type="button" onClick={handleSaveJsonEdit}>
Save
</Button>
</div>
</DialogContent>
</Dialog>
</>
);
}

View file

@ -0,0 +1,120 @@
import { describe, expect, it } from "vitest";
import {
buildHttpToolTestSnapshot,
generateSampleValue,
type HttpToolTestSnapshotFields,
isUnsafeHttpMethod,
parseTestParameterValues,
} from "./helpers";
describe("generateSampleValue", () => {
it.each([
["string", "sample_text"],
["number", "5"],
["boolean", "true"],
["array", "[]"],
["object", "{}"],
] as const)("returns %s sample input", (type, expected) => {
expect(generateSampleValue(type)).toBe(expected);
});
});
describe("isUnsafeHttpMethod", () => {
it("treats GET as safe", () => {
expect(isUnsafeHttpMethod("GET")).toBe(false);
});
it.each(["POST", "PUT", "PATCH", "DELETE"] as const)("treats %s as unsafe", (method) => {
expect(isUnsafeHttpMethod(method)).toBe(true);
});
});
describe("parseTestParameterValues", () => {
it("converts input strings to configured parameter types", () => {
expect(
parseTestParameterValues(
[
{ name: "query", type: "string", required: true },
{ name: "limit", type: "number", required: true },
{ name: "enabled", type: "boolean", required: true },
{ name: "filters", type: "object", required: true },
{ name: "tags", type: "array", required: true },
],
{
query: "cart",
limit: "5",
enabled: "false",
filters: '{"status":"open"}',
tags: '["new"]',
}
)
).toEqual({
query: "cart",
limit: 5,
enabled: false,
filters: { status: "open" },
tags: ["new"],
});
});
it("reports the parameter containing invalid JSON", () => {
expect(() =>
parseTestParameterValues(
[{ name: "filters", type: "object", required: true }],
{ filters: "{" }
)
).toThrow("filters: invalid JSON");
});
it("rejects missing required values", () => {
expect(() =>
parseTestParameterValues(
[{ name: "customer_id", type: "string", required: true }],
{ customer_id: "" }
)
).toThrow("customer_id: value is required");
});
});
describe("buildHttpToolTestSnapshot", () => {
const base: HttpToolTestSnapshotFields = {
name: "Search API",
description: "Search for records",
httpMethod: "GET",
url: "https://api.example.com",
credentialUuid: "",
headers: [],
parameters: [],
presetParameters: [],
timeoutMs: 5000,
customMessage: "",
customMessageType: "text",
customMessageRecordingId: "",
};
it("produces identical output for identical fields", () => {
expect(buildHttpToolTestSnapshot(base)).toBe(buildHttpToolTestSnapshot({ ...base }));
});
it("changes when a saved HTTP field changes", () => {
expect(buildHttpToolTestSnapshot(base)).not.toBe(
buildHttpToolTestSnapshot({ ...base, url: "https://api.example.com/v2" })
);
expect(buildHttpToolTestSnapshot(base)).not.toBe(
buildHttpToolTestSnapshot({
...base,
parameters: [{ name: "q", type: "string", description: "", required: true }],
})
);
expect(buildHttpToolTestSnapshot(base)).not.toBe(
buildHttpToolTestSnapshot({ ...base, name: "Updated Search API" })
);
expect(buildHttpToolTestSnapshot(base)).not.toBe(
buildHttpToolTestSnapshot({ ...base, description: "Updated description" })
);
expect(buildHttpToolTestSnapshot(base)).not.toBe(
buildHttpToolTestSnapshot({ ...base, customMessage: "Done" })
);
});
});

View file

@ -0,0 +1,80 @@
import type { HttpMethod, KeyValueItem, ParameterType, PresetToolParameter, ToolParameter } from "@/components/http";
const TYPE_SAMPLE_VALUES: Record<ParameterType, string> = {
string: "sample_text",
number: "5",
boolean: "true",
array: "[]",
object: "{}",
};
/**
* Type-based sample value used to pre-fill test-dialog inputs. Parameters do
* not currently include a schema, so objects and arrays can only be seeded
* with valid empty JSON containers.
*/
export function generateSampleValue(type: ParameterType): string {
return TYPE_SAMPLE_VALUES[type];
}
/** Whether testing this method may change state on the external service. */
export function isUnsafeHttpMethod(method: HttpMethod): boolean {
return method !== "GET";
}
export function parseTestParameterValues(
parameters: Array<Pick<ToolParameter, "name" | "type" | "required">>,
values: Record<string, string>
): Record<string, unknown> {
const parsedValues: Record<string, unknown> = {};
for (const parameter of parameters) {
const rawValue = values[parameter.name];
if (rawValue === undefined || rawValue === "") {
if (parameter.required) throw new Error(`${parameter.name}: value is required`);
continue;
}
if (parameter.type === "number") {
parsedValues[parameter.name] = Number(rawValue);
} else if (parameter.type === "boolean") {
parsedValues[parameter.name] = rawValue === "true";
} else if (parameter.type === "object" || parameter.type === "array") {
try {
parsedValues[parameter.name] = JSON.parse(rawValue);
} catch {
throw new Error(`${parameter.name}: invalid JSON`);
}
} else {
parsedValues[parameter.name] = rawValue;
}
}
return parsedValues;
}
export type HttpToolTestSnapshotFields = {
name: string;
description: string;
httpMethod: HttpMethod;
url: string;
credentialUuid: string;
headers: KeyValueItem[];
parameters: ToolParameter[];
presetParameters: PresetToolParameter[];
timeoutMs: number;
customMessage: string;
customMessageType: "text" | "audio";
customMessageRecordingId: string;
};
/**
* Canonical string for HTTP API fields that affect the saved configuration
* used by Test Tool.
*/
export function buildHttpToolTestSnapshot(fields: HttpToolTestSnapshotFields): string {
const normalizedHeaders = Object.fromEntries(
fields.headers.filter((header) => header.key).map((header) => [header.key, header.value])
);
return JSON.stringify({ ...fields, headers: normalizedHeaders });
}

View file

@ -0,0 +1,8 @@
export {
buildHttpToolTestSnapshot,
generateSampleValue,
type HttpToolTestSnapshotFields,
isUnsafeHttpMethod,
parseTestParameterValues,
} from "./helpers";
export { HttpToolTestDialog } from "./HttpToolTestDialog";

View file

@ -1,4 +1,5 @@
export { BuiltinToolConfig, type BuiltinToolConfigProps } from "./BuiltinToolConfig";
export { EndCallToolConfig, type EndCallToolConfigProps } from "./EndCallToolConfig";
export { buildHttpToolTestSnapshot, HttpToolTestDialog } from "./http-tool-test";
export { HttpApiToolConfig, type HttpApiToolConfigProps } from "./HttpApiToolConfig";
export { TransferCallToolConfig, type TransferCallToolConfigProps } from "./TransferCallToolConfig";

View file

@ -1,6 +1,6 @@
"use client";
import { ArrowLeft, Code, ExternalLink, Loader2, Save } from "lucide-react";
import { ArrowLeft, Code, ExternalLink, FlaskConical, Loader2, Save } from "lucide-react";
import { useParams, useRouter } from "next/navigation";
import { useCallback, useEffect, useState } from "react";
@ -38,6 +38,7 @@ import { Input } from "@/components/ui/input";
import { Label } from "@/components/ui/label";
import { Skeleton } from "@/components/ui/skeleton";
import { Textarea } from "@/components/ui/textarea";
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip";
import { TOOL_DOCUMENTATION_URLS } from "@/constants/documentation";
import { detailFromError } from "@/lib/apiError";
import { useAuth } from "@/lib/auth";
@ -54,7 +55,14 @@ import {
type ToolCategory,
type TransferDestinationSource,
} from "../config";
import { BuiltinToolConfig, EndCallToolConfig, HttpApiToolConfig, TransferCallToolConfig } from "./components";
import {
buildHttpToolTestSnapshot,
BuiltinToolConfig,
EndCallToolConfig,
HttpApiToolConfig,
HttpToolTestDialog,
TransferCallToolConfig,
} from "./components";
function normalizeParameterType(value: string | null | undefined): ParameterType {
switch (value) {
@ -84,6 +92,8 @@ export default function ToolDetailPage() {
const [error, setError] = useState<string | null>(null);
const [saveSuccess, setSaveSuccess] = useState(false);
const [showCodeDialog, setShowCodeDialog] = useState(false);
const [showTestDialog, setShowTestDialog] = useState(false);
const [savedHttpTestSnapshot, setSavedHttpTestSnapshot] = useState<string | null>(null);
// Common form state
const [name, setName] = useState("");
@ -264,52 +274,73 @@ export default function ToolDetailPage() {
// Populate HTTP API specific fields
const config = tool.definition?.config as HttpApiToolDefinition["config"] | undefined;
if (config) {
setHttpMethod((config.method as HttpMethod) || "POST");
setUrl(config.url || "");
setCredentialUuid(config.credential_uuid || "");
setTimeoutMs(config.timeout_ms || 5000);
setCustomMessage(config.customMessage || "");
setCustomMessageType(config.customMessageType || "text");
setCustomMessageRecordingId(config.customMessageRecordingId || "");
const loadedHttpMethod = (config.method as HttpMethod) || "POST";
const loadedUrl = config.url || "";
const loadedCredentialUuid = config.credential_uuid || "";
const loadedTimeoutMs = config.timeout_ms || 5000;
const loadedCustomMessage = config.customMessage || "";
const loadedCustomMessageType = config.customMessageType || "text";
const loadedCustomMessageRecordingId = config.customMessageRecordingId || "";
setHttpMethod(loadedHttpMethod);
setUrl(loadedUrl);
setCredentialUuid(loadedCredentialUuid);
setTimeoutMs(loadedTimeoutMs);
setCustomMessage(loadedCustomMessage);
setCustomMessageType(loadedCustomMessageType);
setCustomMessageRecordingId(loadedCustomMessageRecordingId);
// Convert headers object to array
if (config.headers) {
setHeaders(
Object.entries(config.headers).map(([key, value]) => ({
key,
value: value as string,
}))
);
} else {
setHeaders([]);
}
const loadedHeaders = config.headers
? Object.entries(config.headers).map(([key, value]) => ({
key,
value: value as string,
}))
: [];
setHeaders(loadedHeaders);
// Load parameters
let loadedParameters: ToolParameter[] = [];
if (config.parameters && Array.isArray(config.parameters)) {
setParameters(
config.parameters.map((p) => ({
name: p.name || "",
type: normalizeParameterType(p.type),
description: p.description || "",
required: p.required ?? true,
}))
);
loadedParameters = config.parameters.map((p) => ({
name: p.name || "",
type: normalizeParameterType(p.type),
description: p.description || "",
required: p.required ?? true,
}));
setParameters(loadedParameters);
} else {
setParameters([]);
}
let loadedPresetParameters: PresetToolParameter[] = [];
if (config.preset_parameters && Array.isArray(config.preset_parameters)) {
setPresetParameters(
config.preset_parameters.map((p) => ({
name: p.name || "",
type: normalizeParameterType(p.type),
valueTemplate: p.value_template || "",
required: p.required ?? true,
}))
);
loadedPresetParameters = config.preset_parameters.map((p) => ({
name: p.name || "",
type: normalizeParameterType(p.type),
valueTemplate: p.value_template || "",
required: p.required ?? true,
}));
setPresetParameters(loadedPresetParameters);
} else {
setPresetParameters([]);
}
setSavedHttpTestSnapshot(
buildHttpToolTestSnapshot({
name: tool.name,
description: tool.description || "",
httpMethod: loadedHttpMethod,
url: loadedUrl,
credentialUuid: loadedCredentialUuid,
headers: loadedHeaders,
parameters: loadedParameters,
presetParameters: loadedPresetParameters,
timeoutMs: loadedTimeoutMs,
customMessage: loadedCustomMessage,
customMessageType: loadedCustomMessageType,
customMessageRecordingId: loadedCustomMessageRecordingId,
})
);
}
}
};
@ -406,6 +437,11 @@ export default function ToolDetailPage() {
setError("All parameters must have a name");
return;
}
const paramNames = parameters.map((p) => p.name.trim()).filter(Boolean);
if (new Set(paramNames).size !== paramNames.length) {
setError("Parameter names must be unique");
return;
}
const invalidPresetParams = presetParameters.filter(
(p) => !p.name.trim() || !p.valueTemplate.trim()
@ -580,6 +616,24 @@ export default function ToolDetailPage() {
setTool(response.data);
setSaveSuccess(true);
setTimeout(() => setSaveSuccess(false), 3000);
if (tool.category === "http_api") {
setSavedHttpTestSnapshot(
buildHttpToolTestSnapshot({
name,
description,
httpMethod,
url,
credentialUuid,
headers,
parameters,
presetParameters,
timeoutMs,
customMessage,
customMessageType,
customMessageRecordingId,
})
);
}
}
} catch (err) {
setError("Failed to save tool");
@ -681,6 +735,24 @@ const data = await response.json();`;
const isTransferCallTool = tool.category === "transfer_call";
const isBuiltinTool = tool.category === "calculator";
const isMcpTool = tool.category === "mcp";
const isHttpApiTool = tool.category === "http_api";
const hasUnsavedHttpChanges =
isHttpApiTool &&
(savedHttpTestSnapshot === null ||
buildHttpToolTestSnapshot({
name,
description,
httpMethod,
url,
credentialUuid,
headers,
parameters,
presetParameters,
timeoutMs,
customMessage,
customMessageType,
customMessageRecordingId,
}) !== savedHttpTestSnapshot);
const categoryConfig = getCategoryConfig(tool.category as ToolCategory);
return (
@ -716,7 +788,7 @@ const data = await response.json();`;
</div>
</div>
<div className="flex items-center gap-2">
{!isEndCallTool && !isTransferCallTool && !isBuiltinTool && !isMcpTool && (
{isHttpApiTool && (
<Button
variant="outline"
onClick={() => setShowCodeDialog(true)}
@ -903,6 +975,18 @@ const data = await response.json();`;
/>
)}
{isHttpApiTool && (
<HttpToolTestDialog
open={showTestDialog}
onOpenChange={setShowTestDialog}
toolUuid={toolUuid}
httpMethod={httpMethod}
url={url}
parameters={parameters}
presetParameters={presetParameters}
/>
)}
{error && (
<div className="mt-4 p-4 bg-destructive/10 border border-destructive/20 rounded-lg text-destructive">
{error}
@ -915,7 +999,34 @@ const data = await response.json();`;
</div>
)}
<div className="flex justify-end mt-6">
<div className="flex justify-end gap-2 mt-6">
{isHttpApiTool && (
hasUnsavedHttpChanges ? (
<Tooltip>
<TooltipTrigger asChild>
<span className="inline-flex" tabIndex={0}>
<Button type="button" variant="outline" disabled>
<FlaskConical className="w-4 h-4 mr-2" />
Test Tool
</Button>
</span>
</TooltipTrigger>
<TooltipContent side="top">
Save the tool before testing.
</TooltipContent>
</Tooltip>
) : (
<Button
type="button"
variant="outline"
onClick={() => setShowTestDialog(true)}
disabled={isSaving}
>
<FlaskConical className="w-4 h-4 mr-2" />
Test Tool
</Button>
)
)}
<Button onClick={handleSave} disabled={isSaving}>
{isSaving ? (
<>
@ -947,6 +1058,7 @@ const data = await response.json();`;
</div>
</DialogContent>
</Dialog>
</div>
);
}

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View file

@ -5938,6 +5938,88 @@ export type ToolResponse = {
created_by?: CreatedByResponse | null;
};
/**
* ToolTestRequest
*
* Request body for testing an HTTP API tool outside a live call.
*/
export type ToolTestRequest = {
/**
* Llm Params
*
* Values for parameters normally supplied by the model.
*/
llm_params?: {
[key: string]: unknown;
};
/**
* Preset Params
*
* Resolved values for parameters normally supplied from presets.
*/
preset_params?: {
[key: string]: unknown;
};
};
/**
* ToolTestResponse
*
* Result of testing an HTTP API tool.
*/
export type ToolTestResponse = {
/**
* Status
*/
status: string;
/**
* Status Code
*/
status_code?: number | null;
/**
* Data
*/
data?: unknown | null;
/**
* Error
*/
error?: string | null;
/**
* Hint
*/
hint?: string | null;
/**
* Request Method
*/
request_method: string;
/**
* Request Url
*/
request_url: string;
/**
* Request Headers
*/
request_headers?: {
[key: string]: string;
};
/**
* Request Body
*/
request_body?: {
[key: string]: unknown;
} | null;
/**
* Request Params
*/
request_params?: {
[key: string]: unknown;
} | null;
/**
* Duration Ms
*/
duration_ms: number;
};
/**
* TransferCallConfig
*
@ -10664,6 +10746,50 @@ export type RefreshMcpToolsApiV1ToolsToolUuidMcpRefreshPostResponses = {
export type RefreshMcpToolsApiV1ToolsToolUuidMcpRefreshPostResponse = RefreshMcpToolsApiV1ToolsToolUuidMcpRefreshPostResponses[keyof RefreshMcpToolsApiV1ToolsToolUuidMcpRefreshPostResponses];
export type TestToolApiV1ToolsToolUuidTestPostData = {
body: ToolTestRequest;
headers?: {
/**
* Authorization
*/
authorization?: string | null;
/**
* X-Api-Key
*/
'X-API-Key'?: string | null;
};
path: {
/**
* Tool Uuid
*/
tool_uuid: string;
};
query?: never;
url: '/api/v1/tools/{tool_uuid}/test';
};
export type TestToolApiV1ToolsToolUuidTestPostErrors = {
/**
* Not found
*/
404: unknown;
/**
* Validation Error
*/
422: HttpValidationError;
};
export type TestToolApiV1ToolsToolUuidTestPostError = TestToolApiV1ToolsToolUuidTestPostErrors[keyof TestToolApiV1ToolsToolUuidTestPostErrors];
export type TestToolApiV1ToolsToolUuidTestPostResponses = {
/**
* Successful Response
*/
200: ToolTestResponse;
};
export type TestToolApiV1ToolsToolUuidTestPostResponse = TestToolApiV1ToolsToolUuidTestPostResponses[keyof TestToolApiV1ToolsToolUuidTestPostResponses];
export type UnarchiveToolApiV1ToolsToolUuidUnarchivePostData = {
body?: never;
headers?: {