Update docs and readme

README.md

@@ -1,11 +1,33 @@
-# RowBoat
+[![ui](/assets/banner.png)](https://www.rowboatlabs.com/)
-[![RowBoat Logo](/assets/rb-logo.png)](https://www.rowboatlabs.com/)
 
-RowBoat is the fastest way to build production-ready multi-agent systems with OpenAI's Agents SDK.
+<h2 align="center">The AI-assisted agent builder</h2>
+<h5 align="center">
+
+[Quickstart](#quick-start) | [Docs](https://docs.rowboatlabs.com/) | [Website](https://www.rowboatlabs.com/) |  [Discord](https://discord.gg/jHhUKkKHn8) 
+
+</h5>
+
+A Cursor-like, AI-assisted, no-code IDE for building production-ready multi-agents. 
+
+- ✨ Start from a simple prompt to create fully functional agents with the Copilot  
+- 🧪 Test them in AI-simulated scenarios  
+- 🌐 Connect MCP servers and tools  
+- 📞 Interact through the Python SDK, a web widget, or a Twilio phone number  
+- ♻️ Continuously refine your agents by providing feedback to the Copilot  
+
+
+
+Built on OpenAI's Agents SDK, **Rowboat is the fastest way to build multi-agents!**
+
+
+
+![ui](/assets/ui_revamp_screenshot.png)
+
+# Quick start
 
 ## Prerequisites
 
-Before running RowBoat, ensure you have:
+Before running Rowboat, ensure you have:
 
 1. **Docker Desktop**
    - [Download Docker Desktop](https://www.docker.com/products/docker-desktop)
@@ -20,9 +42,9 @@ Before running RowBoat, ensure you have:
      brew install mongodb-community@8.0
      brew services start mongodb-community@8.0
      ```
-   - Other platforms: Refer to the MongoDB documentation for details.
+   - Other platforms: Refer to the [MongoDB documentation](https://www.mongodb.com/docs/manual/installation/) for details.
 
-## Quickstart
+## Setup Rowboat
 
 1. **Clone the Repository**
    ```bash
@@ -50,27 +72,95 @@ Before running RowBoat, ensure you have:
 4. **Access the App**
    - Visit [http://localhost:3000](http://localhost:3000).
 
-## Enable RAG
 
-RowBoat supports RAG capabilities to enhance responses with your custom knowledge base. To enable RAG, you'll need:
+Refer to [Docs](https://docs.rowboatlabs.com/) to learn how to start building agents with Rowboat.
 
-1. **Qdrant Vector Database**
+# Advanced 
-   - **Option 1**: Use [Qdrant Cloud](https://cloud.qdrant.io/)
+
-     - Create an account and cluster
+## 1. Tool Use
-     - Note your cluster URL and API key
+
-   - **Option 2**: Run Qdrant locally with Docker:
+You can add your tools / APIs to Rowboat through (a) connecting MCP servers, or (b) connecting a webhook. 
-     ```bash
+
-     docker run -p 6333:6333 qdrant/qdrant
+### 1.1 MCP Servers
-     ```
+
+You can intergrate any MCP server in Settings -> Tools -> MCP Server. The Tools on the servers will show up inside Rowboats Tools section.
+
+### 1.2 Webhook
+
+You can point Rowboat to any webhook in Settings -> Tools -> Webhook. 
+
+Rowboat also includes a built-in webhook service that allows you to implement custom tool functions easily. To use this feature:
+
+1. **Generate Signing Secret**
+   Generate a secret for securing webhook requests:
+   ```bash
+   openssl rand -hex 32
+   ```
 
 2. **Update Environment Variables**
    ```ini
-   USE_RAG=true
+   SIGNING_SECRET=<your-generated-secret>
-   QDRANT_URL=<your-qdrant-url>  # e.g., http://localhost:6333 for local
-   QDRANT_API_KEY=<your-api-key>  # Only needed for Qdrant Cloud
    ```
 
-3. **Initialize Qdrant Collections**
+3. **Implement Your Functions**
+   Add your custom functions to `apps/tools_webhook/function_map.py`:
+   ```python
+   def get_weather(location: str, units: str = "metric"):
+       """Return weather data for the given location."""
+       # Your implementation here
+       return {"temperature": 20, "conditions": "sunny"}
+
+   def check_inventory(product_id: str):
+       """Check inventory levels for a product."""
+       # Your implementation here
+       return {"in_stock": 42, "warehouse": "NYC"}
+
+   # Add your functions to the map
+   FUNCTIONS_MAP = {
+       "get_weather": get_weather,
+       "check_inventory": check_inventory
+   }
+   ```
+
+4. **Start the Tools Webhook Service**
+   ```bash
+   docker compose --profile tools_webhook up -d
+   ```
+
+5. **Register Tools in Rowboat**
+   - Navigate to your project config at `/projects/<PROJECT_ID>/config`
+   - Ensure that the webhook URL is set to: `http://tools_webhook:3005/tool_call`
+   - Tools will automatically be forwarded to your webhook implementation
+
+The webhook service handles all the security and parameter validation, allowing you to focus on implementing your tool logic.
+
+## 2. Retrieve Augmented Generation (RAG)
+
+Rowboat supports adding text directly, document uploads or scraping URLs to enhance the responses with your custom knowledge base. Rowboat uses Qdrant as the vector DB. 
+
+### 2.1 Setup Qdrant
+To enable RAG you need to first setup Qdrant.
+
+1. Option1: Run Qdrant locally
+    - Run Qdrant docker
+    ```bash
+     docker run -p 6333:6333 qdrant/qdrant
+     ```
+    - Update environment variables
+   ```ini
+   USE_RAG=true
+   QDRANT_URL=http://localhost:6333
+   QDRANT_API_KEY=<your-api-key>  # Only needed for Qdrant Cloud
+   ```
+2. Option2: Use [Qdrant Cloud](https://cloud.qdrant.io/)
+    - Note your cluster URL and API key
+    - Update environment variables
+   ```ini
+   USE_RAG=true
+   QDRANT_URL=<your-qdrant-cloud-url>
+   QDRANT_API_KEY=<your-api-key>
+   ```
+3. Initialize Qdrant Collections
    ```bash
    docker compose --profile setup_qdrant up setup_qdrant
    ```
@@ -79,39 +169,42 @@ RowBoat supports RAG capabilities to enhance responses with your custom knowledg
    ```bash
    docker compose --profile delete_qdrant up delete_qdrant
    ```
+### 2.2 Adding Knowledge Base for RAG
 
-### RAG Features
+You can add a knowledge corpus to Rowboat by directly adding text information, uploading supported files or by pointing Rowboat to URLs for scraping.
 
-RowBoat supports two types of knowledge base ingestion:
+#### (a) Create Text for Knowledge
 
-#### URL Scraping
+Setting up Qdrant automatically enables the RAG by adding text information inside Rowboat's RAG menu.
 
-Enable web page scraping to build your knowledge base:
+#### (b) Scrape URLs for Knowledge
 
-1. **Get Firecrawl API Key**
+Rowboat supports scraping urls using Firecrawl. To setup scraping:
+
+1. Get Firecrawl API Key
    - Sign up at [Firecrawl](https://firecrawl.co)
    - Generate an API key
 
-2. **Update Environment Variables**
+2. Update Environment Variables
    ```ini
    USE_RAG_SCRAPING=true
    FIRECRAWL_API_KEY=<your-firecrawl-api-key>
    ```
 
-3. **Start the URLs Worker**
+3. Start the URLs Worker
    ```bash
    docker compose --profile rag_urls_worker up -d
    ```
 
-#### File Uploads
+#### (c) Upload Files for Knowledge 
 
-Enable file upload support (PDF, DOCX, TXT) for your knowledge base:
+Rowboat supports file uploads (PDF, DOCX, TXT) for your knowledge base. It uses Google's Gemini LLM to convert the documents to Markdown before indexing:
 
-1. **Prerequisites**
+1. Prerequisites
    - An AWS S3 bucket for file storage
    - Google Cloud API key with Generative Language (Gemini) API enabled (for enhanced document parsing)
 
-2. **Configure AWS S3**
+2. Configure AWS S3
    - Create an S3 bucket
    - Add the following CORS configuration to your bucket:
      ```json
@@ -158,7 +251,7 @@ Enable file upload support (PDF, DOCX, TXT) for your knowledge base:
      }
      ```
 
-3. **Update Environment Variables**
+3. Update Environment Variables
    ```ini
    USE_RAG_UPLOADS=true
    AWS_ACCESS_KEY_ID=<your-aws-access-key>
@@ -168,63 +261,16 @@ Enable file upload support (PDF, DOCX, TXT) for your knowledge base:
    GOOGLE_API_KEY=<your-google-api-key>
    ```
 
-4. **Start the Files Worker**
+4. Start the Files Worker
    ```bash
    docker compose --profile rag_files_worker up -d
    ```
 
-After enabling RAG and starting the required workers, you can manage your knowledge base through the RowBoat UI at `/projects/<PROJECT_ID>/sources`.
+After enabling RAG and starting the required workers, you can manage your knowledge base through the Rowboat UI at `/projects/<PROJECT_ID>/sources`.
 
-## Enable Tools Webhook
+## 3. Chat Widget
 
-RowBoat includes a built-in webhook service that allows you to implement custom tool functions. To use this feature:
+Rowboat provides an embeddable chat widget that you can add to any website. To enable and use the chat widget:
-
-1. **Generate Signing Secret**
-   Generate a secret for securing webhook requests:
-   ```bash
-   openssl rand -hex 32
-   ```
-
-2. **Update Environment Variables**
-   ```ini
-   SIGNING_SECRET=<your-generated-secret>
-   ```
-
-3. **Implement Your Functions**
-   Add your custom functions to `apps/tools_webhook/function_map.py`:
-   ```python
-   def get_weather(location: str, units: str = "metric"):
-       """Return weather data for the given location."""
-       # Your implementation here
-       return {"temperature": 20, "conditions": "sunny"}
-
-   def check_inventory(product_id: str):
-       """Check inventory levels for a product."""
-       # Your implementation here
-       return {"in_stock": 42, "warehouse": "NYC"}
-
-   # Add your functions to the map
-   FUNCTIONS_MAP = {
-       "get_weather": get_weather,
-       "check_inventory": check_inventory
-   }
-   ```
-
-4. **Start the Tools Webhook Service**
-   ```bash
-   docker compose --profile tools_webhook up -d
-   ```
-
-5. **Register Tools in RowBoat**
-   - Navigate to your project config at `/projects/<PROJECT_ID>/config`
-   - Ensure that the webhook URL is set to: `http://tools_webhook:3005/tool_call`
-   - Tools will automatically be forwarded to your webhook implementation
-
-The webhook service handles all the security and parameter validation, allowing you to focus on implementing your tool logic.
-
-## Enable Chat Widget
-
-RowBoat provides an embeddable chat widget that you can add to any website. To enable and use the chat widget:
 
 1. **Generate JWT Secret**
    Generate a secret for securing chat widget sessions:
@@ -246,42 +292,12 @@ RowBoat provides an embeddable chat widget that you can add to any website. To e
 4. **Add Widget to Your Website**
    You can find the chat-widget embed code under `/projects/<PROJECT_ID>/config`
 
-After setup, the chat widget will appear on your website and connect to your RowBoat project.
+After setup, the chat widget will appear on your website and connect to your Rowboat project.
 
-## Enable Authentication
 
-By default, RowBoat runs without authentication. To enable user authentication using Auth0:
+### 4. Interact with Rowboat API
 
-1. **Auth0 Setup**
+There are two ways to interact with Rowboat's API:
-   - **Create an Auth0 Account**: Sign up at [Auth0](https://auth0.com).
-   - **Create a New Application**: Choose "Regular Web Application", select "Next.js" as the application type, and name it "RowBoat".
-   - **Configure Application**:
-     - **Allowed Callback URLs**: In the Auth0 Dashboard, go to your "RowBoat" application settings and set `http://localhost:3000/api/auth/callback` as an Allowed Callback URL.
-   - **Get Credentials**: Collect the following from your Auth0 application settings:
-     - **Domain**: Copy your Auth0 domain (ensure you append `https://` to the Domain that the Auth0 dashboard shows you)
-     - **Client ID**: Your application's unique identifier
-     - **Client Secret**: Your application's secret key
-   - **Generate secret**: Generate a session encryption secret in your terminal and note the output for later:
-     ```bash
-     openssl rand -hex 32
-     ```
-
-2. **Update Environment Variables**
-   Add the following to your `.env` file:
-   ```ini
-   USE_AUTH=true
-   AUTH0_SECRET=your-generated-secret               # Generated using openssl command
-   AUTH0_BASE_URL=http://localhost:3000             # Your application's base URL
-   AUTH0_ISSUER_BASE_URL=https://example.auth0.com  # Your Auth0 domain (ensure it is prefixed with https://)
-   AUTH0_CLIENT_ID=your-client-id
-   AUTH0_CLIENT_SECRET=your-client-secret
-   ```
-   
-After enabling authentication, users will need to sign in to access the application.
-
-## Interact with RowBoat API
-
-There are two ways to interact with RowBoat's API:
 
 1. **Option 1: Python SDK**
 
@@ -351,18 +367,35 @@ There are two ways to interact with RowBoat's API:
    }
    ```
 
-## Troubleshooting
+### 5. Authentication
 
-1. **MongoDB Connection Issues**
+By default, Rowboat runs without authentication. To enable user authentication using Auth0:
-   - Ensure local MongoDB service is running: `brew services list`
-   - Verify connection string and network connectivity.
 
-2. **Container Start-up Issues**
+1. **Auth0 Setup**
-   - Remove all containers: `docker-compose down`
+   - **Create an Auth0 Account**: Sign up at [Auth0](https://auth0.com).
-   - Rebuild: `docker-compose up --build`
+   - **Create a New Application**: Choose "Regular Web Application", select "Next.js" as the application type, and name it "Rowboat".
+   - **Configure Application**:
+     - **Allowed Callback URLs**: In the Auth0 Dashboard, go to your "Rowboat" application settings and set `http://localhost:3000/api/auth/callback` as an Allowed Callback URL.
+   - **Get Credentials**: Collect the following from your Auth0 application settings:
+     - **Domain**: Copy your Auth0 domain (ensure you append `https://` to the Domain that the Auth0 dashboard shows you)
+     - **Client ID**: Your application's unique identifier
+     - **Client Secret**: Your application's secret key
+   - **Generate secret**: Generate a session encryption secret in your terminal and note the output for later:
+     ```bash
+     openssl rand -hex 32
+     ```
 
-3. **Sign-in Button Not Appearing**
+2. **Update Environment Variables**
-   - If the sign-in button does not appear in the UI, ensure the Auth0 domain in your `.env` file is prefixed with `https://`.
+   Add the following to your `.env` file:
+   ```ini
+   USE_AUTH=true
+   AUTH0_SECRET=your-generated-secret               # Generated using openssl command
+   AUTH0_BASE_URL=http://localhost:3000             # Your application's base URL
+   AUTH0_ISSUER_BASE_URL=https://example.auth0.com  # Your Auth0 domain (ensure it is prefixed with https://)
+   AUTH0_CLIENT_ID=your-client-id
+   AUTH0_CLIENT_SECRET=your-client-secret
+   ```
+   
+After enabling authentication, users will need to sign in to access the application.
 
-## Attribution
+   
-Our agents framework is built on top of [OpenAI Swarm](https://github.com/openai/swarm) with custom enhancements and improvements. Check the [NOTICE](https://github.com/rowboatlabs/rowboat/blob/main/apps/agents/NOTICE.md) for attribution and license.

apps/docs/docs/agents.md

@@ -3,7 +3,7 @@
 ## Overview
 - Agents carry out a specific part of the conversation and / or perform tasks like orchestrating between other agents, triggering internal processes and fetching information.
 - Agents carry out tasks through tools provided to them.
-- Agents are connected to each other in a Directed Acyclic Graph (DAG). Hence, every agent has a parent agent and children agents, to which they can pass control of the conversation to. 
+- Agents can be connected to other agents through a mention in the agent's instruction.
 
 ## Agent Configurations
 
@@ -11,10 +11,10 @@
 The description conveys the agent's role in the multi-agent system. Writing a good description is important for other agents to know when to pass control of the conversation to an agent.
 
 ### Instructions
-Agent instructions are the backbone of an agent, defining its behavior. RowBoat Studio's copilot produces a good framework for agent instructions, involving Role, Steps to Follow, Scope and Guidelines. Since agents are powered by LLMs, general best practices while writing prompts apply. 
+Agent instructions are the backbone of an agent, defining its behavior. RowBoat Studio's copilot produces a good framework for agent instructions, involving Role, Steps to Follow, Scope and Guidelines. Since agents are powered by LLMs, general best practices while writing prompts apply.
 
 ### Examples
-The agent uses examples as a reference for behavior in different scenarios. While there are no prescribed formats to provide examples in, examples should include what the user might say, what the agent should respond with as well as indications of any tool calls to be made. 
+The agent uses examples as a reference for behavior in different scenarios. While there are no prescribed formats to provide examples in, examples should include what the user might say, what the agent should respond with as well as indications of any tool calls to be made.
 
 ### Prompts
 Prompts attached to an agent will be used by the agent in addition to instructions.
@@ -23,7 +23,7 @@ Prompts attached to an agent will be used by the agent in addition to instructio
 Data sources added to an agent will be used as knowledge, retrieved using embedding match in a typical RAG fashion. Advanced configurations allow for setting number of matches, etc. RAG is currently implemented as a predefined tool call which the agent will use when it determines that it needs to retrieve knowledge. This behavior can be further fine-tuned by specifying corresponding instructions or prompts.
 
 ### Tools
-Tools attached to an agent will be put out as tool calls. The behavior of when to invoke tools can be fine-tuned by specifying corresponding instructions or prompts. Adding examples to agents can also be useful in controlling tool call behavior. 
+Tools attached to an agent will be put out as tool calls. The behavior of when to invoke tools can be fine-tuned by specifying corresponding instructions or prompts. Adding examples to agents can also be useful in controlling tool call behavior.
 
 ### Connected Agents
 In the agent graph, connected agents refer to children of an agent. An agent can choose to transfer control of the conversation to one of its children, by using internal tool calls (need not be configured separately). Similar to tools, the behavior of when to transfer the chat to a child agent can be fine-tuned by specifying corresponding instructions, examples and prompts.

apps/docs/docs/index.md

@@ -1,6 +1,6 @@
 # Welcome to Rowboat
 
-Rowboat is an open-source project that helps you build, test, and deploy multi-agent AI customer support assistants. 
+Rowboat is an open-source Cursor like IDE that helps you build, test, and deploy multi-agent AI systems.
 
 **Note:** These docs are intended for both developers who would like to self-host our [open-source code](https://github.com/rowboatlabs/rowboat/) as well as users of our [hosted (managed) app](https://app.rowboatlabs.com/).
 
@@ -12,18 +12,18 @@ Rowboat is an open-source project that helps you build, test, and deploy multi-a
 ## What is RowBoat?
 **RowBoat is a state-of-art platform to build multi-agent AI systems in a visual interface, with the help of a copilot.**
 
-RowBoat enables you to build, manage and deploy user-facing assistants. An assistant is made up of multiple agents, each having access to a set of tools and working together to interact with the user as a single assistant. 
+RowBoat enables you to build, manage and deploy user-facing assistants. An assistant is made up of multiple agents, each having access to a set of tools and working together to interact with the user as a single assistant.
 
 For example, you can build a *credit card assistant*, where each agent handles a workflow such as *outstanding payments*, *balance inquiries* and *transaction disputes*. You can equip agents with tools to carry out tasks such as *fetching payment options*, *checking outstanding balance* and *updating user information*. The assistant would help your end-users their credit card-related needs without having to talk to a human agent on your end.
 
 ## How RowBoat works
 
 ### RowBoat Studio
-RowBoat Studio lets you create human-quality customer support assistants in minutes, using a visual interface and plain language. Here are key components that you will work with:
+RowBoat Studio lets you create AI agents in minutes, using a visual interface and plain language. Here are key components that you will work with:
 
 | Component  | Description | Highlights |
 |------------|-------------|------------|
-| Agent     | Handles a specific part of the conversation and<br>performs tasks using tools, based on instructions |• Configurable using plain language instructions<br>• Orchestrate between agents connected as a graph (DAG)<br>• Can access tools and knowledge sources (RAG)|
+| Agent     | Handles a specific part of the conversation and<br>performs tasks using tools, based on instructions |• Configurable using plain language instructions<br>• Orchestrate between agents connected as a graph<br>• Can access tools and knowledge sources (RAG)|
 | Playground | Interactive environment to test assistants<br>conversationally as you build them |• Real-time testing and debugging<br>• Inspect parameters and results of tool calls in-line<br>• Converse with individual agents or the entire assistant|
 | Copilot    | AI-powered concierge that creates and<br>updates agents and tools on your behalf |• Context-aware of all components including playground<br>• Improves agents based on conversations and feedback <br>• Understands your requests in plain language|
 | Simulator  | Simulates real-world user interactions<br>with your assistant |• Maintain and run a test-bench of different scenarios<br>• Mock tool responses for quick testing<br>• Reproduce your end-user's experience comprehensively|
@@ -48,10 +48,10 @@ RowBoat Studio lets you create human-quality customer support assistants in minu
 2. Alternatively, **export** your assistant as a JSON artifact from RowBoat Studio and use it to power your custom implementations.
 
 ## Why RowBoat?
-Increase CSAT. Reduce costs. Improve CX.
+Accelerate your path to production-ready multi-agent systems.
 
 1. **Build** complex assistants using plain language and a visual interface
-2. **Simplify** collaboration across teams, to build powerful customer experiences
+2. **Integrate** tools and MCP servers in minutes
 3. **Expedite** your multi-agent AI roadmap using battle-tested tooling
 
 ## Getting started

apps/docs/docs/oss_installation.md

@@ -2,7 +2,7 @@
 
 - This is the developers guide to self-hosting the open-source version of RowBoat. To get started with the hosted app, please see [Using the Hosted App](/hosted_setup)
 - Please see our [Introduction](/) page before referring to this guide.
-- For direct installation steps, please head to the README of RowBoat's Github repo: [@rowboatlabs/rowboat](https://github.com/rowboatlabs/rowboat/). This page provides more context about the installation process and the different components involved. 
+- For direct installation steps, please head to the README of RowBoat's Github repo: [@rowboatlabs/rowboat](https://github.com/rowboatlabs/rowboat/). This page provides more context about the installation process and the different components involved.
 
 ## Overview
 
@@ -14,20 +14,19 @@ RowBoat's codebase has three main components:
 | **Copilot** | Python framework powering the copilot in RowBoat Studio |
 | **RowBoat** | Frontend and backend services to power RowBoat Studio and Chat APIs |
 
-These components are structured as separate services, each containerized with Docker. Running `docker-compose up --build` enables you to use the Studio in your browser, as well as stands up the APIs and SDK. 
+These components are structured as separate services, each containerized with Docker. Running `docker-compose up --build` enables you to use the Studio in your browser, as well as stands up the APIs and SDK.
 
 ## Prerequisites
-All of these prerequistes have open-source or free versions.
+All of these prerequisites have open-source or free versions.
 
 | Prerequisite | Description |
 |--------------|---------------|
 | **Docker** | Bundles and builds all services |
 | **OpenAI API Key** | Agents and Copilot services are powered by OpenAI LLMs |
 | **MongoDB** | Stores workflow versions, chats and RAG embeddings |
-| **Auth0 Account** | Handles user authentication and identity management for Studio |
 
 Refer to our [Github Readme for Prerequisites](https://github.com/rowboatlabs/rowboat/?tab=readme-ov-file#prerequisites) to set up prerequisites.
 
 ## Setting up
 
-Refer to our [Github Readme for Local Development](https://github.com/rowboatlabs/rowboat/?tab=readme-ov-file#local-development-setup) to set up Studio, Chat API and SDK via `docker-compose`. 
+Refer to our [Github Readme for Local Development](https://github.com/rowboatlabs/rowboat/?tab=readme-ov-file#local-development-setup) to set up Studio, Chat API and SDK via `docker-compose`.