dograh/api/mcp_server/instructions.py
Leoy 5762095edf
feat(mcp): add search_docs tool over docs corpus (closes #295) (#316)
* feat(mcp): add search_docs tool over Mintlify docs corpus

Closes #295. The docs at https://docs.dograh.com promise "Search the
Dograh docs for how to configure a TURN server" as an MCP example
prompt, but no search_docs tool exists in the MCP server — agents can
list workspace resources but cannot search the documentation.

This adds a dependency-free, in-process keyword search over the
`docs/` tree shipped into the API image (`COPY ./docs ./docs`):

- New `api/mcp_server/tools/docs_search.py` — async `search_docs(query,
  limit=10)` with weighted scoring (path > title > body), a 25-result
  hard cap, snippet extraction around the first term hit, and graceful
  empty-list degradation when docs aren't on disk. `DOGRAH_DOCS_PATH`
  env var overrides location discovery for non-Docker layouts.

- Registered in `api/mcp_server/server.py` alongside the other tools,
  keeping the existing list-alphabetical convention.

- `api/tests/test_mcp_docs_search.py` — 18 unit tests covering the
  pure helpers (tokenizer, frontmatter stripping, title extraction,
  scoring weights, URL building) and end-to-end ranking, limit
  clamping, empty-corpus degradation, and input-validation errors.
  Mocks `authenticate_mcp_request` to avoid the DB dependency,
  mirroring `test_mcp_save_workflow.py`.

Implementation notes:
- The docs corpus is ~100 files / ~140k LoC, so a per-call scan runs
  well under 50 ms; avoiding a vector index / embedding backend keeps
  the tool zero-dependency and works for fully offline self-hosted
  deployments.
- Authentication is required for consistency with the other MCP tools
  (and to route through the existing rate-limit middleware), even
  though docs are not org-scoped data.
- Title/path matches deliberately outweigh body matches so a page
  whose subject IS the query term outranks one that merely mentions
  it incidentally.

* feat: improve docs search

---------

Co-authored-by: Abhishek Kumar <abhishek@a6k.me>
2026-05-20 18:20:35 +05:30

100 lines
6.3 KiB
Python

"""Top-level orchestration guide surfaced to every MCP session.
Sent to the client via `FastMCP(instructions=...)` — the client bakes
this into its system prompt, so every LLM session sees it before the
first tool call. Prefer procedural orchestration here (call order, error
handling, hard constraints). Design-level per-field guidance belongs in
each `PropertySpec.llm_hint`; it flows out through `get_node_type` and
doesn't need to be repeated here.
Extend based on real LLM failures — every bullet below ideally maps to a
mistake the system has seen at least once.
"""
DOGRAH_MCP_INSTRUCTIONS = """\
You build and edit Dograh voice-AI workflows by emitting TypeScript that uses the `@dograh/sdk` package. Workflows are stored as JSON; this server projects them to TypeScript for editing and parses them back on save.
## Call order
### Reading documentation
1. `search_docs(query)` — use first for keyword or acronym lookup when the user is asking how Dograh works or how to configure something.
2. `read_doc(path)` — fetch the full page once one result looks likely. Prefer this over reasoning from search summaries alone.
3. `list_docs(path=None, depth=1)` — use when the user wants to browse a topic area or when search terms are too vague. Returned section paths feed back into `list_docs`; returned page paths feed into `read_doc`.
### Editing an existing workflow
1. `list_workflows` — locate the target workflow.
2. `get_workflow_code(workflow_id)` — fetch the current source.
3. (optional) `list_node_types` / `get_node_type(name)` — consult before adding or editing a node type whose fields aren't already visible in the current code.
4. Mutate the code in place. Preserve existing nodes, edges, and variable names unless the task requires removing or renaming them.
5. `save_workflow(workflow_id, code)` — persist as a new draft. The published version is untouched.
### Creating a new workflow
1. Create a simple 1-node workflow with only `startCall`. The user can iteratively add complexity by editing it.
2. `list_node_types` / `get_node_type(name)` — consult to learn the fields available on the node types you intend to use.
3. Author SDK TypeScript from scratch. The `new Workflow({ name: "..." })` call is required — `name` becomes the workflow's display name.
4. `create_workflow(code)` — persists a new workflow as version 1 (published). Returns the new `workflow_id`. For subsequent edits use `save_workflow(workflow_id, code)` (which writes a draft).
## Allowed source shape
The parser is AST-only and rejects anything outside this grammar. At the top level, only three statement forms are accepted:
import ... from "..."; // any import
const <var> = <initializer>; // bindings (see below)
wf.edge(<src>, <tgt>, { label, condition }); // bare edge calls
`<initializer>` is one of:
new Workflow({ name: "..." })
wf.addTyped(<factory>({ ...fields }) [, { position: [x, y] }])
wf.add({ type: "<nodeType>", ...fields [, position: [x, y]] })
No functions, arrow fns, loops, conditionals, ternaries, spreads, destructuring, template interpolation, `export`, or `.map`/`.forEach`.
Data-position values must be plain literals (strings, numbers, booleans, null, arrays/objects of same). A single `new Workflow(...)` per file — the `name` you pass there is the workflow's display name and is applied on save (renames propagate immediately; definition changes go to draft).
## Adding edges — explicit syntax
wf.edge(source, target, { label: "...", condition: "..." });
Rules:
- `source` and `target` are the **bare variable identifiers** bound by `wf.addTyped(...)` / `wf.add(...)` — not strings, not `.id`, not inline factories. Both must be declared earlier in the file.
- `label` is a short tag (≤4 words) shown in call logs to identify the branch: `"qualified"`, `"wrap up"`, `"retry"`.
- `condition` is a full natural-language predicate the runtime evaluates against the live conversation: `"caller confirmed interest in a demo"`, not `"interested"`. Condition clarity determines routing accuracy.
- Both fields are required and must be non-empty strings.
- Edges are directional; emit one `wf.edge(...)` per outgoing branch.
- Place all edges after all node bindings; group by source node.
Example:
const greet = wf.addTyped(startCall({ name: "Greet", prompt: "Hi!" }));
const done = wf.addTyped(endCall({ name: "Done", prompt: "Bye." }));
wf.edge(greet, done, {
label: "wrap up",
condition: "user acknowledged the greeting and is ready to end"
});
## Iterating on errors
`save_workflow` and `create_workflow` return one of:
- `parse_error` — Disallowed construct (see grammar above) or malformed TypeScript.
- `validation_error` — Node data failed spec validation (unknown field, missing required, wrong type, bad `options` value).
- `graph_validation` — Structural rule broken (missing startCall, unreachable node, edge to/from wrong node type).
- `missing_name` — (`create_workflow` only) `new Workflow({ name })` is absent or empty.
- `bridge_error` — Internal; retry once, then surface to the user.
Every error carries `line` and `column`. Fix at that location and resubmit the **complete source** — this tool does not accept patches.
## Field conventions
- `data.name` is the canonical identifier. Pick a descriptive name (`"Qualify Budget"`, not `"Node1"`) — the generated code uses it as the variable name and call logs reference it.
- Reference fields take UUIDs, not human names:
- `tool_refs`, `document_refs` → from `list_tools`, `list_documents`
- `credential_ref` → from `list_credentials`
- `recording_ref` → from `list_recordings`
- `mention_textarea` fields (prompts, greetings, etc.) accept `{{template_variables}}` — values resolved at runtime from `pre_call_fetch`, caller context, or earlier extraction passes.
## Style
- Prefer `wf.addTyped(factory({ ... }))` over `wf.add({ type, ... })`.
- Only include fields whose values differ from the spec default — the parser re-applies defaults on save, so extras are noise.
- Omit `position`; the server reconciles positions against the previous saved workflow and lays out new nodes automatically.
- Add nodes in call-flow order (start → intermediate → end) so the generated code reads top-to-bottom, with all edges after all nodes.
"""