- Accept folder_id and folder_name as indexing parameters
- Hide date range for Google Drive connectors
- Create wrapper function to avoid circular imports
- Trigger Google Drive indexing Celery task
- Full folder scan on first index
- Delta sync using change tracking for subsequent indexes
- Process files in parallel batches
- Handle file additions, modifications, and deletions
- Store change tracking token for efficient re-indexing
- OAuth initialization and callback handling
- Folder and file browsing with parent_id support
- Validate credentials and handle token refresh
- Return folder contents with metadata for UI tree view
- Get start page token for change tracking baseline
- Fetch incremental changes using Google Drive Changes API
- Categorize changes into added, modified, and removed files
- Enable efficient re-indexing of only changed content
- List folder contents with full pagination support
- Query root folder or specific parent folder
- Return both folders and files with metadata (size, icons, links)
- Filter out shortcuts and trashed items
- Download files from Google Drive to temporary location
- Export Google Workspace files as PDF
- Delegate content extraction to existing process_file_in_background
- Reuse Surfsense's ETL services (Unstructured, LlamaCloud, Docling)
- Detect Google Workspace files (Docs, Sheets, Slides)
- Map to PDF export format to preserve rich content (images, formatting)
- Identify files to skip (shortcuts, unsupported types)
- Build and manage Google Drive service with credentials
- List files with query support and pagination
- Download binary files and export Google Workspace files as PDF
- Handle HTTP errors gracefully
- Removed the write_todos tool as it is now included by default through TodoListMiddleware in the deep agent.
- Updated the system prompt and documentation to reflect the integration of TodoListMiddleware, clarifying its capabilities for managing planning and todo lists.
- Enhanced the chat handling logic to extract todos directly from the deep agent's command output, ensuring seamless user experience.
- Refactored UI components to align with the new data structure and improve rendering of todo items, including updates to the Plan and TodoItem components.
- Cleaned up code for better maintainability and readability, following recent refactoring efforts.
- Updated the write_todos tool to include an optional description field for todo items, improving task detail management.
- Enhanced the system prompt with clearer guidelines on using the write_todos tool, including refined usage patterns and examples for various user scenarios.
- Improved UI components to support the new description feature, ensuring better visibility of task details during planning and execution.
- Streamlined the code for better readability and maintainability, aligning with recent refactoring efforts.
- Refactored the write_todos tool to enhance argument and result schemas using Zod for better validation and type safety.
- Updated the WriteTodosToolUI to streamline the rendering logic and improve loading states, ensuring a smoother user experience.
- Enhanced the Plan and TodoItem components to better handle streaming states and display progress, providing clearer feedback during task management.
- Cleaned up code formatting and structure for improved readability and maintainability.
- Enhanced the system prompt with detailed criteria and examples for using the write_todos tool, including valid trigger words and usage patterns.
- Clarified restrictions on when to use the tool, emphasizing the need for multi-phase tasks and explicit user requests.
- Provided specific use cases for different user types to guide appropriate usage of the tool, ensuring better task management and user experience.
- Added a new write_todos tool to facilitate the creation and management of planning lists within the chat interface.
- Updated system prompt with detailed instructions on using the write_todos tool, including usage patterns and restrictions.
- Enhanced the chat message handling to support the new tool, ensuring proper integration and user experience.
- Implemented UI components for displaying and interacting with the planning lists, including progress tracking and status indicators.
- Updated system prompt to clarify usage of the display_image tool, emphasizing valid URL requirements and prohibiting user-uploaded image displays.
- Introduced write_todos tool for creating and updating planning lists, with detailed usage instructions and examples.
- Enhanced message handling to persist attachments and mentioned documents, ensuring they survive page reloads.
- Improved UI components to integrate the new write_todos tool and ensure consistent user experience across chat interactions.
- Updated WebCrawlerConnector to prioritize Firecrawl API for crawling if an API key is provided, falling back to Chromium if Firecrawl fails.
- Improved error handling to log failures from both Firecrawl and Chromium.
- Enhanced link preview tool to use a random User-Agent for better compatibility with web servers.
- Passed Firecrawl API key to the stream_new_chat function for improved configuration management.
- Introduced DocumentMentionPicker component to enhance document selection experience in the chat interface.
- Updated InlineMentionEditor and Composer components to utilize the new DocumentMentionPicker.
- Removed the deprecated DocumentsDataTable component to streamline the codebase and improve maintainability.
- Enhanced type safety and validation in document handling logic.
- Added a fallback mechanism using headless Chromium to fetch page content when standard HTTP requests fail.
- Introduced utility functions for unescaping HTML entities and converting relative URLs to absolute.
- Updated HTTP request headers to mimic a browser for better compatibility with web servers.
- Improved error handling and logging for better debugging and user feedback.
- Made various properties in Zod schemas nullable for better type safety and flexibility in handling optional data.
- Updated system prompt to clarify usage of the display_image tool, emphasizing URL requirements and restrictions on user-uploaded images.
- Enhanced the streaming chat process to provide more context about user attachments and documents during analysis.
- Implemented state resets when switching between chats to prevent stale data and race conditions.
- Added new components for displaying image previews and document attachments in the chat interface.
- Improved attachment processing to support image data URLs for persistent display after uploads.
- Migrated data from the old 'chats' table to 'new_chat_threads' and 'new_chat_messages'.
- Dropped the 'chats' table and removed the 'chattype' enum as part of the migration process.
- Updated the migration script to truncate thread titles to 500 characters to comply with database constraints.
- Adjusted the downgrade function to reflect changes in the migration process.
- Updated the new chat routes to include handling for mentioned document IDs, allowing users to reference specific documents in their chat.
- Modified the NewChatRequest schema to accommodate optional document IDs.
- Implemented document mention formatting in the chat streaming service for improved context.
- Enhanced the frontend to manage document mentions, including a new atom for state management and UI updates for document selection.
- Refactored the DocumentsDataTable component for better integration with the new mention functionality.
- Enhanced the new chat agent module to allow for configurable tools, enabling users to customize their experience with various functionalities.
- Removed outdated tools including display image, knowledge base search, link preview, podcast generation, and web scraping, streamlining the codebase.
- Updated the system prompt and agent factory to reflect these changes, ensuring a more cohesive and efficient architecture.
