- Added methods to retrieve the starting page token and list changes in Google Drive, enabling delta sync capabilities.
- Updated Composio service to handle file download directory configuration.
- Modified indexing tasks to support delta sync, improving efficiency by processing only changed files.
- Adjusted date handling in connector tasks to allow optional start and end dates.
- Improved error handling and logging throughout the Composio indexing process.
- Introduced new enum values for Composio connectors: COMPOSIO_GOOGLE_DRIVE_CONNECTOR, COMPOSIO_GMAIL_CONNECTOR, and COMPOSIO_GOOGLE_CALENDAR_CONNECTOR.
- Updated database migration to add these new enum values to the relevant types.
- Refactored Composio integration logic to handle specific connector types, improving the management of connected accounts and indexing processes.
- Enhanced frontend components to support the new Composio connector types, including updated UI elements and connector configuration handling.
- Improved backend services to manage Composio connected accounts more effectively, including deletion and indexing tasks.
- Added logic to refresh connector and notification attributes after indexing to ensure up-to-date information.
- Enhanced periodic sync configuration to disable the option when no folders or files are selected for Google Drive, providing user feedback through a message.
- Updated the connector edit view to reflect the new disabled state for periodic sync based on selected items.
- Implemented validation in the connector dialog to prevent enabling periodic sync without selected items, improving user experience.
- Updated the Google Drive indexing functionality to include indexing options such as max files per folder, incremental sync, and inclusion of subfolders.
- Modified the API to accept a new 'indexing_options' parameter in the request body.
- Enhanced the UI to allow users to configure these options when selecting folders and files for indexing.
- Updated related components and tasks to support the new indexing options, ensuring a more flexible and efficient indexing process.
- Added session refresh for notifications to prevent stale data after rollbacks in multiple document processing tasks.
- Wrapped notification update logic in try-except blocks to handle potential failures gracefully and log errors without crashing the process.
- Improved error handling for notification updates in various document processing functions, enhancing overall robustness.
- Introduced a new DocumentProcessingNotificationHandler to manage notifications for document processing stages.
- Updated existing notification methods to include detailed progress updates for various stages (queued, parsing, chunking, embedding, storing, completed, failed).
- Refactored NotificationService to support the new document processing notification type and metadata schema.
- Updated multiple document processing tasks to create and manage notifications throughout the processing lifecycle.
- Adjusted UI components to reflect changes in notification types and improve user experience during document uploads and processing.
- Added setup_electric_replication function to handle Electric SQL replication for the notifications table during app startup.
- Updated alembic migration script to remove direct SQL commands for replication, now managed in app/db.py.
- Refactored indexing functions in search_source_connectors_routes to support new start_date and end_date parameters for improved flexibility.
- Enhanced Google Gmail indexing task to utilize new date parameters, ensuring better control over indexing periods.
- Add structured request body with folders and files arrays
- Support individual file indexing alongside folder indexing
- Remove deprecated folder_ids/folder_names query params
- Update UI to allow selecting both folders and files
- Enhanced the new chat agent module to allow for configurable tools, enabling users to customize their experience with various functionalities.
- Removed outdated tools including display image, knowledge base search, link preview, podcast generation, and web scraping, streamlining the codebase.
- Updated the system prompt and agent factory to reflect these changes, ensuring a more cohesive and efficient architecture.
- Added streaming service support to the Q&A agent for real-time token streaming.
- Updated `answer_question` method to stream responses token-by-token to the frontend.
- Modified `handle_qna_workflow` to handle both custom and values streaming modes.
- Enhanced state management to include streaming service for improved user experience.
- Updated the document saving endpoint to trigger reindexing after saving.
- Introduced a new Celery task for reindexing documents.
- Refactored the editor page to reflect the changes in the API endpoint and method.
