- Added methods to retrieve the starting page token and list changes in Google Drive, enabling delta sync capabilities.
- Updated Composio service to handle file download directory configuration.
- Modified indexing tasks to support delta sync, improving efficiency by processing only changed files.
- Adjusted date handling in connector tasks to allow optional start and end dates.
- Improved error handling and logging throughout the Composio indexing process.
- Added a method to find existing notifications by mention ID to prevent duplicate notifications.
- Updated notify_new_mention to check for existing notifications before creating a new one, ensuring idempotency.
- Implemented error handling for race conditions during notification creation, allowing retrieval of existing notifications in case of duplicate key errors.
- Improved documentation for clarity on the use of mention_id for idempotency.
- Enhanced the handling of file content from Composio, supporting both binary and text files with appropriate processing methods.
- Introduced robust error logging and handling for file content extraction, ensuring better visibility into issues during processing.
- Updated the indexing logic to accommodate new content processing methods, improving overall reliability and user feedback on errors.
- Added temporary file handling for binary files to facilitate text extraction using the ETL service.
- Added a new endpoint to list folders and files in a user's Composio Google Drive, supporting hierarchical structure.
- Implemented UI components for selecting specific folders and files to index, improving user control over indexing options.
- Introduced indexing options for maximum files per folder and inclusion of subfolders, allowing for customizable indexing behavior.
- Enhanced error handling and logging for Composio Drive operations, ensuring better visibility into issues during file retrieval and indexing.
- Updated the Composio configuration component to reflect new selection capabilities and indexing options.
- Updated the list_gmail_messages method to support pagination with page tokens, allowing for more efficient message retrieval.
- Modified the return structure to include next_page_token and result_size_estimate for better client-side handling.
- Improved error handling and logging throughout the Gmail indexing process, ensuring better visibility into failures.
- Implemented batch processing for Gmail messages, committing changes incrementally to prevent data loss.
- Ensured consistent timestamp updates for connectors, even when no documents are indexed, to maintain accurate UI states.
- Refactored the indexing logic to streamline message processing and enhance overall performance.
- Introduced new enum values for Composio connectors: COMPOSIO_GOOGLE_DRIVE_CONNECTOR, COMPOSIO_GMAIL_CONNECTOR, and COMPOSIO_GOOGLE_CALENDAR_CONNECTOR.
- Updated database migration to add these new enum values to the relevant types.
- Refactored Composio integration logic to handle specific connector types, improving the management of connected accounts and indexing processes.
- Enhanced frontend components to support the new Composio connector types, including updated UI elements and connector configuration handling.
- Improved backend services to manage Composio connected accounts more effectively, including deletion and indexing tasks.
- Added author_avatar_url and author_email to comment creation and reply functions for improved metadata.
- Updated MentionNotificationHandler to include author details in notifications.
- Enhanced NotificationPopup to display author's avatar with initials fallback for mentions.
- Updated notification types to accommodate new author fields.
- Enhanced error handling in the indexing process to differentiate between actual failures and cases where no new documents are processed.
- Updated notification messages to reflect the status accurately, including a message for when no new items are synced.
- Standardized return values across various indexer tasks to return `None` on success, simplifying logging and error management.
- Introduced a new DocumentProcessingNotificationHandler to manage notifications for document processing stages.
- Updated existing notification methods to include detailed progress updates for various stages (queued, parsing, chunking, embedding, storing, completed, failed).
- Refactored NotificationService to support the new document processing notification type and metadata schema.
- Updated multiple document processing tasks to create and manage notifications throughout the processing lifecycle.
- Adjusted UI components to reflect changes in notification types and improve user experience during document uploads and processing.
- Removed the deprecated create_connector_indexed_notification method from NotificationService.
- Introduced new notification types and schemas in notification.types.ts to standardize notification handling.
- Updated useNotifications hook to utilize the new notification type definitions.
- Added notifications table to the database schema with replication support for Electric SQL.
- Developed NotificationService to manage indexing notifications, including creation, updates, and status tracking.
- Introduced NotificationButton and NotificationPopup components for displaying notifications in the UI.
- Enhanced useNotifications hook for real-time notification syncing using PGlite live queries.
- Updated package dependencies for Electric SQL and improved error handling in notification processes.
- Added Electric SQL service to docker-compose for real-time data synchronization.
- Introduced PostgreSQL configuration for logical replication and performance tuning.
- Created scripts for initializing Electric SQL user and electrifying tables.
- Implemented notification model and service in the backend.
- Developed ElectricProvider and useNotifications hook in the frontend for managing notifications.
- Updated environment variables and package dependencies for Electric SQL integration.
