diff --git a/apps/docs/docs/add_tools.md b/apps/docs/docs/add_tools.md new file mode 100644 index 00000000..a1e151da --- /dev/null +++ b/apps/docs/docs/add_tools.md @@ -0,0 +1,19 @@ +## Add tools to agents +Copilot can help you add tools to agents. + +### Instruct copilot to add tools +![Add Tool](img/add-tool.png) + +![Example Tool](img/example-tool.png) + +### Inspect tools and agents +Note how copilot not only creates the tool definitions for you, but also updates the relevant agent instructions to use the tool and connects the tool to the agent. + +![Inspect Agent Tools](img/inspect-agent-tools.png) +![Inspect Agent Tool Connections](img/inspect-agent-tool-connections.png) + +### Debug tool calls in the playground +When agents call tools during a chat in the playground, the tool call parameters and response are available for debugging real-time. For testing purposes, the platform can produce mock tool responses in the playground, without integrating actual tools. + +![Mock Tool Responses](img/mock-tool-responses.png) +![Debug Tool Calls](img/debug-tool-calls.png) \ No newline at end of file diff --git a/apps/docs/docs/agents.md b/apps/docs/docs/agents.md index c8712439..e6e7f9f0 100644 --- a/apps/docs/docs/agents.md +++ b/apps/docs/docs/agents.md @@ -14,10 +14,26 @@ The description conveys the agent's role in the multi-agent system. Writing a go Agent instructions are the backbone of an agent, defining its behavior. RowBoat Studio's copilot produces a good framework for agent instructions, involving Role, Steps to Follow, Scope and Guidelines. Since agents are powered by LLMs, general best practices while writing prompts apply. ### Examples -Coming soon. +The agent uses examples as a reference for behavior in different scenarios. While there are no prescribed formats to provide examples in, examples should include what the user might say, what the agent should respond with as well as indications of any tool calls to be made. + +### Prompts +Prompts attached to an agent will be used by the agent in addition to instructions. + +### RAG +Data sources added to an agent will be used as knowledge, retrieved using embedding match in a typical RAG fashion. Advanced configurations allow for setting number of matches, etc. RAG is currently implemented as a predefined tool call which the agent will use when it determines that it needs to retrieve knowledge. This behavior can be further fine-tuned by specifying corresponding instructions or prompts. + +### Tools +Tools attached to an agent will be put out as tool calls. The behavior of when to invoke tools can be fine-tuned by specifying corresponding instructions or prompts. Adding examples to agents can also be useful in controlling tool call behavior. + +### Connected Agents +In the agent graph, connected agents refer to children of an agent. An agent can choose to transfer control of the conversation to one of its children, by using internal tool calls (need not be configured separately). Similar to tools, the behavior of when to transfer the chat to a child agent can be fine-tuned by specifying corresponding instructions, examples and prompts. ### Model -Coming soon. +RowBoat currently supports OpenAI LLMs. Agents can be configured to use any of the OpenAI LLMs. -### Control -Coming soon. \ No newline at end of file +### Conversation control after turn +This setting specifies different options for control of conversation after the current agent has put out a user-facing response (i.e., completed the turn). Currently available options are: + +1. Retain control for the next turn of conversation (most common and default setting) +2. Give up control to the parent agent (used when the agent has narrow scope such as answering a FAQ) +3. Give up control to the agent designated as Start agent \ No newline at end of file diff --git a/apps/docs/docs/building_in_studio.md b/apps/docs/docs/building_in_studio.md deleted file mode 100644 index 8566aaed..00000000 --- a/apps/docs/docs/building_in_studio.md +++ /dev/null @@ -1,81 +0,0 @@ -# Building Assistants in Studio -This is a guide to building your first assistant on RowBoat Studio, with examples.
- -Prerequisite: - -1. **Open Source Users:** Complete the [open-source installation steps](/oss_installation/) to set up RowBoat Studio. -2. **Hosted App Users:** Sign in to [https://app.rowboatlabs.com/](https://app.rowboatlabs.com/) - -## Create the set of initial agents -Copilot can set up agents for you from scratch. - -### Instruct copilot -First, tell it about the initial set of agents that make up your assistant. - -![Create Initial Agents](img/copilot-create.png) - -Using copilot to create your initial set of agents helps you leverage best practices in formatting agent instructions and connecting agents to each other as a graph, all of which have been baked into copilot. - -### Inspect the agents -Once you apply changes, inspect the agents to see how copilot has built them. Specifically, note the Instructions, Examples and Connected Agents in each agent. - -![Agent Config](img/agent-config.png) - -Also notice that copilot would likely have created a "Hub" agent that is "connected" to other agents. - -![Hub Agent Config](img/hub-config.png) - -### Make changes if needed -Tweak the instructions and examples manually if needed. - -![Edit Agent Manually](img/edit-agent-manually.png) - -## Try an example chat in the playground - -### Chat with the assistant - -The playground is intended to test out the assistant as you build it. The User and Assistant messages represent the conversation that your end-user will have if your assistant is deployed in production. The playground also has debug elements which show the flow of control between different agents in your system, as well as which agent finally responded to the user. - -![Try Chat](img/try-chat.png) - -### Ask copilot questions -You can ask copilot clarifications about the chat, such as why the agents responded a certain way or why an agent was invoked. - -![Copilot Clarifications](img/copilot-clarifications.png) - -## Add tools to agents -Copilot can help you add tools to agents. - -#### Instruct copilot to add tools -![Add Tool](img/add-tool.png) - -![Example Tool](img/example-tool.png) - -### Inspect tools and agents -Note how copilot not only creates the tool definitions for you, but also updates the relevant agent instructions to use the tool and connects the tool to the agent. - -![Inspect Agent Tools](img/inspect-agent-tools.png) -![Inspect Agent Tool Connections](img/inspect-agent-tool-connections.png) - -### Debug tool calls in the playground -When agents call tools during a chat in the playground, the tool call parameters and response are available for debugging real-time. For testing purposes, the platform can produce mock tool responses in the playground, without integrating actual tools. - -![Mock Tool Responses](img/mock-tool-responses.png) -![Debug Tool Calls](img/debug-tool-calls.png) - -## Update agent behavior - -Copilot can help you update agent behavior. It is also aware of the current chat in the playground so you can make references to the current chat while instructing copilot to update agents. - -![Update Agent Behavior](img/update-agent-with-copilot.png) - - -Playground: -![Test Updated Agent](img/test-updated-agent.png) - -## Simulate real-world user scenarios -Create a test-bench of real-world scenarios in the simulator. -![Scenarios](img/scenarios.png) - -Run the scenarios as simulated chats betweeen a user (role-played) and the assistant, in the playground. -![Simulation](img/simulate.png) \ No newline at end of file diff --git a/apps/docs/docs/create_agents.md b/apps/docs/docs/create_agents.md new file mode 100644 index 00000000..c9aa40cd --- /dev/null +++ b/apps/docs/docs/create_agents.md @@ -0,0 +1,23 @@ +## Create the set of initial agents +Copilot can set up agents for you from scratch. + +### Instruct copilot +First, tell it about the initial set of agents that make up your assistant. + +![Create Initial Agents](img/copilot-create.png) + +Using copilot to create your initial set of agents helps you leverage best practices in formatting agent instructions and connecting agents to each other as a graph, all of which have been baked into copilot. + +### Inspect the agents +Once you apply changes, inspect the agents to see how copilot has built them. Specifically, note the Instructions, Examples and Connected Agents in each agent. + +![Agent Config](img/agent-config.png) + +Also notice that copilot would likely have created a "Hub" agent that is "connected" to other agents. + +![Hub Agent Config](img/hub-config.png) + +### Make changes if needed +Tweak the instructions and examples manually if needed. + +![Edit Agent Manually](img/edit-agent-manually.png) \ No newline at end of file diff --git a/apps/docs/docs/graph.md b/apps/docs/docs/graph.md index 4452882f..c5d47e95 100644 --- a/apps/docs/docs/graph.md +++ b/apps/docs/docs/graph.md @@ -19,5 +19,5 @@ ## Pipelines -- RowBoat also has the concept of pipelines, which are agents invoked sequentially after an agent in the graph has produced a user-facing response. +- RowBoat also has the concept of pipelines - specialized agents invoked sequentially after an agent in the graph has produced a user-facing response. - E.g. a pipeline with a post processing agent and a guardrail agent will ensure that every response is post processed and guardrailed for appropriateness before presenting it to the user. diff --git a/apps/docs/docs/img/project-page.png b/apps/docs/docs/img/project-page.png index 8c4cf5a6..61ce2c9e 100644 Binary files a/apps/docs/docs/img/project-page.png and b/apps/docs/docs/img/project-page.png differ diff --git a/apps/docs/docs/img/sys-msg.png b/apps/docs/docs/img/sys-msg.png new file mode 100644 index 00000000..2a2abfba Binary files /dev/null and b/apps/docs/docs/img/sys-msg.png differ diff --git a/apps/docs/docs/playground.md b/apps/docs/docs/playground.md new file mode 100644 index 00000000..a18c6411 --- /dev/null +++ b/apps/docs/docs/playground.md @@ -0,0 +1,15 @@ +## Try an example chat in the playground + +### Chat with the assistant + +The playground is intended to test out the assistant as you build it. The User and Assistant messages represent the conversation that your end-user will have if your assistant is deployed in production. The playground also has debug elements which show the flow of control between different agents in your system, as well as which agent finally responded to the user. + +![Try Chat](img/try-chat.png) + +In the playground, you can also set initial context at start of chat, that will be passed to all agents. This is typically used for providing user identity information such as user ID, login email, etc. +![Use System Message](img/sys-msg.png) + +### Ask copilot questions +You can ask copilot clarifications about the chat, such as why the agents responded a certain way or why an agent was invoked. + +![Copilot Clarifications](img/copilot-clarifications.png) \ No newline at end of file diff --git a/apps/docs/docs/prompts.md b/apps/docs/docs/prompts.md index fcc39fe6..2552cc49 100644 --- a/apps/docs/docs/prompts.md +++ b/apps/docs/docs/prompts.md @@ -1 +1,5 @@ -Coming soon. \ No newline at end of file +# Prompts + +- Prompts are reusable pieces of agent instructions in Studio. +- Prompts can be defined once and reused across multiple agents. +- Common examples of prompts are style prompts which indicate brand voice and structured output prompts which specify a format for the agent to provide its output in (e.g. ReAct) \ No newline at end of file diff --git a/apps/docs/docs/simulate.md b/apps/docs/docs/simulate.md new file mode 100644 index 00000000..e488d0d5 --- /dev/null +++ b/apps/docs/docs/simulate.md @@ -0,0 +1,6 @@ +## Simulate real-world user scenarios +Create a test-bench of real-world scenarios in the simulator. +![Scenarios](img/scenarios.png) + +Run the scenarios as simulated chats betweeen a user (role-played) and the assistant, in the playground. +![Simulation](img/simulate.png) \ No newline at end of file diff --git a/apps/docs/docs/studio_overview.md b/apps/docs/docs/studio_overview.md new file mode 100644 index 00000000..89ac4e85 --- /dev/null +++ b/apps/docs/docs/studio_overview.md @@ -0,0 +1,7 @@ +# Building Assistants in Studio +This is a guide to building your first assistant on RowBoat Studio, with examples.
+ +Prerequisite: + +1. **Open Source Users:** Complete the [open-source installation steps](/oss_installation/) to set up RowBoat Studio. +2. **Hosted App Users:** Sign in to [https://app.rowboatlabs.com/](https://app.rowboatlabs.com/) \ No newline at end of file diff --git a/apps/docs/docs/testing.md b/apps/docs/docs/testing.md index 98b16491..86df88be 100644 --- a/apps/docs/docs/testing.md +++ b/apps/docs/docs/testing.md @@ -7,7 +7,7 @@ - To use our hosted app, the URL for Studio is [https://app.rowboatlabs.com](https://app.rowboatlabs.com/)
-2. Once in Studio, create a new blank project or use one of the example templates: +2. Once in Studio, create a new blank project or browse through one of the example projects: ![Create Project](img/project-page.png)
diff --git a/apps/docs/docs/tools.md b/apps/docs/docs/tools.md index fcc39fe6..c5f17128 100644 --- a/apps/docs/docs/tools.md +++ b/apps/docs/docs/tools.md @@ -1 +1,6 @@ -Coming soon. \ No newline at end of file +# Tools +- Tools are used to carry out specific tasks such as fetching or updating information. +- Tools can be defined once in RowBoat Studio and reused across different agents. +- RowBoat uses OpenAI style tools with name, description and parameters. +- For the purposes of quick testing in the Playground, RowBoat Studio can mock tool responses based on tool descriptions. +- Developers can easily connect tools to APIs by configuring a Webhook URL in Studio, to which all tool calls will be routed. \ No newline at end of file diff --git a/apps/docs/docs/update_agents.md b/apps/docs/docs/update_agents.md new file mode 100644 index 00000000..28b24482 --- /dev/null +++ b/apps/docs/docs/update_agents.md @@ -0,0 +1,8 @@ +## Update agent behavior + +Copilot can help you update agent behavior. It is also aware of the current chat in the playground so you can make references to the current chat while instructing copilot to update agents. + +![Update Agent Behavior](img/update-agent-with-copilot.png) + +Playground: +![Test Updated Agent](img/test-updated-agent.png) \ No newline at end of file diff --git a/apps/docs/mkdocs.yml b/apps/docs/mkdocs.yml index 99b74e1d..147fd033 100644 --- a/apps/docs/mkdocs.yml +++ b/apps/docs/mkdocs.yml @@ -13,8 +13,15 @@ nav: - Option 2 - Hosted App: hosted_setup.md - Testing Your Setup: testing.md - - Building With RowBoat: - - Building in Studio: building_in_studio.md + - Building in Studio: + - Overview: studio_overview.md + - Create agents: create_agents.md + - Test chats in the playground: playground.md + - Add tools: add_tools.md + - Update agents: update_agents.md + - Simulate scenarios: simulate.md + + - API & SDK: - Using the API: using_the_api.md - Using the SDK: using_the_sdk.md diff --git a/apps/rowboat/app/actions.ts b/apps/rowboat/app/actions.ts index 6e6f9998..fef6f4ab 100644 --- a/apps/rowboat/app/actions.ts +++ b/apps/rowboat/app/actions.ts @@ -579,6 +579,11 @@ export async function getCopilotResponse( const test = { name: 'test', description: 'test', + parameters: { + type: 'object', + properties: {}, + required: [], + }, } as z.infer; // iterate over each field in part.content.config_changes // and test if the final object schema is valid @@ -1049,11 +1054,12 @@ export async function updateWebhookUrl(projectId: string, url: string) { export async function executeClientTool( toolCall: z.infer['tool_calls'][number], + messages: z.infer[], projectId: string, ): Promise { await projectAuthCheck(projectId); - const result = await callClientToolWebhook(toolCall, projectId); + const result = await callClientToolWebhook(toolCall, messages, projectId); return result; } diff --git a/apps/rowboat/app/api/widget/v1/chats/[chatId]/turn/route.ts b/apps/rowboat/app/api/widget/v1/chats/[chatId]/turn/route.ts index ee6f1633..99abed42 100644 --- a/apps/rowboat/app/api/widget/v1/chats/[chatId]/turn/route.ts +++ b/apps/rowboat/app/api/widget/v1/chats/[chatId]/turn/route.ts @@ -118,7 +118,11 @@ export async function POST( const toolCallResults = await Promise.all(lastMessage.tool_calls.map(async toolCall => { console.log('executing tool call', toolCall); try { - return await callClientToolWebhook(toolCall, session.projectId); + return await callClientToolWebhook( + toolCall, + [...messages, ...unsavedMessages], + session.projectId, + ); } catch (error) { console.error(`Error executing tool call ${toolCall.id}:`, error); return { error: "Tool execution failed" }; diff --git a/apps/rowboat/app/lib/utils.ts b/apps/rowboat/app/lib/utils.ts index cfa2efaa..e754d488 100644 --- a/apps/rowboat/app/lib/utils.ts +++ b/apps/rowboat/app/lib/utils.ts @@ -7,6 +7,7 @@ import crypto from "crypto"; export async function callClientToolWebhook( toolCall: z.infer['tool_calls'][number], + messages: z.infer[], projectId: string, ): Promise { const project = await projectsCollection.findOne({ @@ -23,6 +24,7 @@ export async function callClientToolWebhook( // prepare request body const content = JSON.stringify({ toolCall, + messages, } as z.infer); const requestId = crypto.randomUUID(); const bodyHash = crypto diff --git a/apps/rowboat/app/projects/[projectId]/config/app.tsx b/apps/rowboat/app/projects/[projectId]/config/app.tsx index 7b16df9a..dc619449 100644 --- a/apps/rowboat/app/projects/[projectId]/config/app.tsx +++ b/apps/rowboat/app/projects/[projectId]/config/app.tsx @@ -328,12 +328,8 @@ export function SecretSection({ return

- The project secret is used for: + The project secret is used for signing tool-call requests sent to your webhook

-
    -
  • Signing tool-call requests sent to your webhook
    • -
  • Signing user-data sent through the chat widget
    • -
@@ -398,10 +394,7 @@ export function WebhookUrlSection({ function validate(url: string) { try { - const parsedUrl = new URL(url); - if (parsedUrl.protocol !== 'https:') { - return { valid: false, errorMessage: 'URL must use HTTPS' }; - } + new URL(url); return { valid: true }; } catch { return { valid: false, errorMessage: 'Please enter a valid URL' }; @@ -410,7 +403,7 @@ export function WebhookUrlSection({ return

- Tool calls issued through the chat widget will be posted to this URL. + In workflow editor, tool calls will be posted to this URL, unless they are mocked.

@@ -587,7 +580,7 @@ export default function App({ - + {/* */} diff --git a/apps/rowboat/app/projects/[projectId]/playground/messages.tsx b/apps/rowboat/app/projects/[projectId]/playground/messages.tsx index 13035bbe..41f8f118 100644 --- a/apps/rowboat/app/projects/[projectId]/playground/messages.tsx +++ b/apps/rowboat/app/projects/[projectId]/playground/messages.tsx @@ -443,6 +443,7 @@ function ClientToolCall({ try { response = await executeClientTool( toolCall, + messages, projectId, ); } catch (e) {