apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-04 22:02:16 +02:00
Code Issues Projects Releases Packages Wiki Activity

refactor: integrate TodoListMiddleware and update related components

Browse source 
- Removed the write_todos tool as it is now included by default through TodoListMiddleware in the deep agent.
- Updated the system prompt and documentation to reflect the integration of TodoListMiddleware, clarifying its capabilities for managing planning and todo lists.
- Enhanced the chat handling logic to extract todos directly from the deep agent's command output, ensuring seamless user experience.
- Refactored UI components to align with the new data structure and improve rendering of todo items, including updates to the Plan and TodoItem components.
- Cleaned up code for better maintainability and readability, following recent refactoring efforts.
This commit is contained in:
Anish Sarkar 2025-12-27 15:18:34 +05:30
parent 8a3ab3dfac
commit c28a90fc29
10 changed files with 172 additions and 419 deletions
Download patch file Download diff file Expand all files Collapse all files

4
surfsense_backend/app/agents/new_chat/chat_deepagent.py
View file

@ -50,6 +50,9 @@ def create_surfsense_deep_agent(
- display_image: Display images in chat
- scrape_webpage: Extract content from webpages
The agent also includes TodoListMiddleware by default (via create_deep_agent) which provides:
- write_todos: Create and update planning/todo lists for complex tasks
The system prompt can be configured via agent_config:
- Custom system instructions (or use defaults)
- Citation toggle (enable/disable citation requirements)
@ -138,6 +141,7 @@ def create_surfsense_deep_agent(
system_prompt = build_surfsense_system_prompt()
# Create the deep agent with system prompt and checkpointer
# Note: TodoListMiddleware (write_todos) is included by default in create_deep_agent
agent = create_deep_agent(
model=llm,
tools=tools,

85
surfsense_backend/app/agents/new_chat/system_prompt.py
View file

@ -111,64 +111,19 @@ You have access to the following tools:
* Don't show every image - just the most relevant 1-3 images that enhance understanding.
6. write_todos: Create and update a planning/todo list to break down complex tasks.
- Use this tool when you need to plan your approach to a complex task.
- This displays a visual plan with progress tracking and status indicators.
- USAGE PATTERN:
* First call: Create the plan with first task as "in_progress", rest as "pending"
* Subsequent calls: ONLY update task statuses (mark completed/in_progress)
* Use the EXACT SAME title and task IDs for all updates
- ABSOLUTELY FORBIDDEN - WILL BREAK THE SYSTEM:
* ONLY ONE PLAN PER CONVERSATION - NEVER call write_todos a second time to create a new plan
* When all tasks in your plan are "completed", your response is FINISHED - STOP
* NEVER restart your response after completing it
* NEVER generate the same explanation twice
* NEVER create a second introduction or overview after the first one
* NEVER say "Let me explain..." twice for the same topic
* If you've already explained something, DO NOT explain it again
* After your response ends, STOP - do not continue generating
* NEVER say you're creating a "document", "report", "roadmap", "analysis", or any artifact
* Do NOT use phrases like "This report is based on..." or "Based on my research..."
* Just answer the question directly - do not roleplay producing a deliverable
- CORRECT BEHAVIOR:
* Call write_todos to update statuses as you progress
* Each section of your response appears EXACTLY ONCE
* When you finish explaining all tasks, your response is COMPLETE
* Do NOT generate additional content after concluding
- CONTENT QUALITY:
* Provide thorough, detailed explanations for each task
* The restriction is on DUPLICATING content, not on depth or detail
* Each task deserves a complete, comprehensive explanation
* Be as detailed as needed - just don't repeat yourself
- IMPORTANT: Use this tool when the user asks you to create a plan, break down a task, or explain something in structured steps.
- This tool creates a visual plan with progress tracking that the user can see in the UI.
- When to use:
* Breaking down a complex multi-step task (3-5 tasks recommended)
* Showing the user what steps you'll take to solve their problem
* Creating an implementation roadmap
* User asks to "create a plan" or "break down" a task
* User asks for "steps" to do something
* User asks you to "explain" something in sections
* Any multi-step task that would benefit from structured planning
- Args:
- todos: List of todo items, each with:
* id: Unique identifier (KEEP SAME IDs across updates)
* content: Description of the task (KEEP SAME content across updates)
* status: "pending", "in_progress", or "completed"
* description: Optional subtask/detail text shown when the item is expanded (e.g., "Analyzing document structure and key concepts")
- title: Title for the plan (MUST BE IDENTICAL across all updates)
- description: Optional context description
- Returns: A visual plan card with progress bar and status indicators
- CORRECT PATTERN:
1. Create plan with task 1 as "in_progress"
2. Explain task 1 content in detail
3. Update plan: task 1 "completed", task 2 "in_progress"
4. Explain task 2 content (NEW content, not repeating task 1)
5. Continue until all tasks are "completed"
6. When all tasks are "completed", your response is FINISHED
7. STOP IMMEDIATELY - do NOT create another plan or continue generating
8. ONE PLAN ONLY - never call write_todos again after completing all tasks
* content: Description of the task (required)
* status: "pending", "in_progress", or "completed" (required)
- The tool automatically adds IDs and formats the output for the UI.
- Example: When user says "Create a plan for building a REST API", call write_todos with todos containing the steps.
</tools>
<tool_call_examples>
- User: "Fetch all my notes and what's in them?"
@ -227,21 +182,13 @@ You have access to the following tools:
- Call: `display_image(src="https://example.com/nn-diagram.png", alt="Neural Network Diagram", title="Neural Network Architecture")`
- Then provide your explanation, referencing the displayed image
- User: "Help me implement a user authentication system"
- Step 1: Create plan with task 1 in_progress:
`write_todos(title="Auth Plan", todos=[{"id": "1", "content": "Design database schema", "status": "in_progress"}, {"id": "2", "content": "Set up password hashing", "status": "pending"}, {"id": "3", "content": "Create endpoints", "status": "pending"}])`
- Step 2: Provide DETAILED explanation of database schema design
- Step 3: Update plan (task 1 done, task 2 in_progress):
`write_todos(title="Auth Plan", todos=[{"id": "1", "content": "Design database schema", "status": "completed"}, {"id": "2", "content": "Set up password hashing", "status": "in_progress"}, {"id": "3", "content": "Create endpoints", "status": "pending"}])`
- Step 4: Provide DETAILED explanation of password hashing (NEW content only)
- Step 5: Update plan, explain endpoints in detail
- Step 6: Mark all complete, END response - DO NOT restart or regenerate
- FORBIDDEN: Do not go back and explain schema again after step 2
- User: "Create a plan for building a user authentication system"
- Call: `write_todos(todos=[{"content": "Design database schema for users and sessions", "status": "in_progress"}, {"content": "Implement registration and login endpoints", "status": "pending"}, {"content": "Add password reset functionality", "status": "pending"}])`
- Then explain each step in detail as you work through them
- User: "How should I approach refactoring this large codebase?"
- Create plan, explain each step with thorough detail, update statuses as you go
- Each explanation is comprehensive but appears ONLY ONCE
- When finished with all tasks, STOP - do not continue generating
- User: "Break down how to build a REST API into steps"
- Call: `write_todos(todos=[{"content": "Design API endpoints and data models", "status": "in_progress"}, {"content": "Set up server framework and routing", "status": "pending"}, {"content": "Implement CRUD operations", "status": "pending"}, {"content": "Add authentication and error handling", "status": "pending"}])`
- Then provide detailed explanations for each step
</tool_call_examples>
"""

9
surfsense_backend/app/agents/new_chat/tools/registry.py
View file

@ -48,7 +48,6 @@ from .knowledge_base import create_search_knowledge_base_tool
from .link_preview import create_link_preview_tool
from .podcast import create_generate_podcast_tool
from .scrape_webpage import create_scrape_webpage_tool
from .write_todos import create_write_todos_tool
# =============================================================================
# Tool Definition
@ -126,13 +125,7 @@ BUILTIN_TOOLS: list[ToolDefinition] = [
),
requires=[], # firecrawl_api_key is optional
),
# Planning/Todo tool - creates visual todo lists
ToolDefinition(
name="write_todos",
description="Create a planning/todo list to break down complex tasks",
factory=lambda deps: create_write_todos_tool(),
requires=[],
),
# Note: write_todos is now provided by TodoListMiddleware from deepagents
# =========================================================================
# ADD YOUR CUSTOM TOOLS BELOW
# =========================================================================

99
surfsense_backend/app/agents/new_chat/tools/write_todos.py
View file

@ -1,99 +0,0 @@
"""
Write todos tool for the SurfSense agent.
This module provides a tool for creating and displaying a planning/todo list
in the chat UI. It helps the agent break down complex tasks into steps.
"""
from typing import Any
from langchain_core.tools import tool
def create_write_todos_tool():
"""
Factory function to create the write_todos tool.
Returns:
A configured tool function for writing todos/plans.
"""
@tool
async def write_todos(
todos: list[dict[str, Any]],
title: str = "Planning Approach",
description: str | None = None,
) -> dict[str, Any]:
"""
Create a planning/todo list to break down a complex task.
Use this tool when you need to plan your approach to a complex task
or show the user a step-by-step breakdown of what you'll do.
This displays a visual plan with:
- Progress tracking (X of Y complete)
- Status indicators (pending, in progress, completed, cancelled)
- Expandable details for each step
Args:
todos: List of todo items. Each item should have:
- id: Unique identifier for the todo
- content: Description of the task
- status: One of "pending", "in_progress", "completed", "cancelled"
- description: Optional subtask/detail text shown when the item is expanded
title: Title for the plan (default: "Planning Approach")
description: Optional description providing context
Returns:
A dictionary containing the plan data for the UI to render.
Example:
write_todos(
title="Implementation Plan",
description="Steps to add the new feature",
todos=[
{"id": "1", "content": "Analyze requirements", "status": "completed", "description": "Reviewed all user stories and acceptance criteria"},
{"id": "2", "content": "Design solution", "status": "in_progress", "description": "Creating component architecture and data flow diagrams"},
{"id": "3", "content": "Write code", "status": "pending"},
{"id": "4", "content": "Add tests", "status": "pending", "description": "Unit tests and integration tests for all new components"},
]
)
"""
# Generate a unique plan ID
import uuid
plan_id = f"plan-{uuid.uuid4().hex[:8]}"
# Transform todos to the expected format for the UI
formatted_todos = []
for i, todo in enumerate(todos):
todo_id = todo.get("id", f"todo-{i}")
content = todo.get("content", "")
status = todo.get("status", "pending")
todo_description = todo.get("description")
# Validate status
valid_statuses = ["pending", "in_progress", "completed", "cancelled"]
if status not in valid_statuses:
status = "pending"
todo_item = {
"id": todo_id,
"label": content,
"status": status,
}
# Only include description if provided
if todo_description:
todo_item["description"] = todo_description
formatted_todos.append(todo_item)
return {
"id": plan_id,
"title": title,
"description": description,
"todos": formatted_todos,
}
return write_todos

37
surfsense_backend/app/tasks/chat/stream_new_chat.py
View file

@ -69,6 +69,30 @@ def format_mentioned_documents_as_context(documents: list[Document]) -> str:
return "\n".join(context_parts)
def extract_todos_from_deepagents(command_output) -> dict:
"""
Extract todos from deepagents' TodoListMiddleware Command output.
deepagents returns a Command object with:
- Command.update['todos'] = [{'content': '...', 'status': '...'}]
Returns the todos directly (no transformation needed - UI matches deepagents format).
"""
todos_data = []
if hasattr(command_output, "update"):
# It's a Command object from deepagents
update = command_output.update
todos_data = update.get("todos", [])
elif isinstance(command_output, dict):
# Already a dict - check if it has todos directly or in update
if "todos" in command_output:
todos_data = command_output.get("todos", [])
elif "update" in command_output and isinstance(command_output["update"], dict):
todos_data = command_output["update"].get("todos", [])
return {"todos": todos_data}
async def stream_new_chat(
user_query: str,
search_space_id: int,
@ -557,9 +581,11 @@ async def stream_new_chat(
tool_name = event.get("name", "unknown_tool")
raw_output = event.get("data", {}).get("output", "")
# Extract content from ToolMessage if needed
# LangGraph may return a ToolMessage object instead of raw dict
if hasattr(raw_output, "content"):
# Handle deepagents' write_todos Command object specially
if tool_name == "write_todos" and hasattr(raw_output, "update"):
# deepagents returns a Command object - extract todos directly
tool_output = extract_todos_from_deepagents(raw_output)
elif hasattr(raw_output, "content"):
# It's a ToolMessage object - extract the content
content = raw_output.content
# If content is a string that looks like JSON, try to parse it
@ -721,12 +747,10 @@ async def stream_new_chat(
elif tool_name == "write_todos":
# Build completion items for planning
if isinstance(tool_output, dict):
plan_title = tool_output.get("title", "Plan")
todos = tool_output.get("todos", [])
todo_count = len(todos) if isinstance(todos, list) else 0
completed_items = [
*last_active_step_items,
f"Plan: {plan_title[:50]}{'...' if len(plan_title) > 50 else ''}",
f"Tasks: {todo_count} steps defined",
]
else:
@ -883,11 +907,10 @@ async def stream_new_chat(
)
# Send terminal message with plan info
if isinstance(tool_output, dict):
title = tool_output.get("title", "Plan")
todos = tool_output.get("todos", [])
todo_count = len(todos) if isinstance(todos, list) else 0
yield streaming_service.format_terminal_info(
f"Plan created: {title} ({todo_count} tasks)",
f"Plan created ({todo_count} tasks)",
"success",
)
else: