- Enhanced error handling in the indexing process to differentiate between actual failures and cases where no new documents are processed.
- Updated notification messages to reflect the status accurately, including a message for when no new items are synced.
- Standardized return values across various indexer tasks to return `None` on success, simplifying logging and error management.
- Added session refresh for notifications to prevent stale data after rollbacks in multiple document processing tasks.
- Wrapped notification update logic in try-except blocks to handle potential failures gracefully and log errors without crashing the process.
- Improved error handling for notification updates in various document processing functions, enhancing overall robustness.
- Introduced new notifications API routes for marking notifications as read and marking all as read, with automatic syncing via Electric SQL.
- Updated the frontend hook to utilize the new API for marking notifications as read, enhancing the user experience with real-time updates.
- Included necessary response models for API interactions.
- Introduced a new DocumentProcessingNotificationHandler to manage notifications for document processing stages.
- Updated existing notification methods to include detailed progress updates for various stages (queued, parsing, chunking, embedding, storing, completed, failed).
- Refactored NotificationService to support the new document processing notification type and metadata schema.
- Updated multiple document processing tasks to create and manage notifications throughout the processing lifecycle.
- Adjusted UI components to reflect changes in notification types and improve user experience during document uploads and processing.
- Refactored multiple indexing functions in search_source_connectors_routes to utilize a new helper function for handling notifications.
- Removed redundant error handling and logging from individual indexing tasks, streamlining the code.
- Added setup_electric_replication function to handle Electric SQL replication for the notifications table during app startup.
- Updated alembic migration script to remove direct SQL commands for replication, now managed in app/db.py.
- Refactored indexing functions in search_source_connectors_routes to support new start_date and end_date parameters for improved flexibility.
- Enhanced Google Gmail indexing task to utilize new date parameters, ensuring better control over indexing periods.
- Added notifications table to the database schema with replication support for Electric SQL.
- Developed NotificationService to manage indexing notifications, including creation, updates, and status tracking.
- Introduced NotificationButton and NotificationPopup components for displaying notifications in the UI.
- Enhanced useNotifications hook for real-time notification syncing using PGlite live queries.
- Updated package dependencies for Electric SQL and improved error handling in notification processes.
- Updated date handling in indexing functions to permit future dates for Google Calendar and Luma connectors.
- Enhanced UI components to support future date selection, including a new button for selecting the next 30 days.
- Adjusted documentation and descriptions to clarify date range options for users.
- Added ClickUp OAuth authentication flow with new environment variables for client ID, client secret, and redirect URI.
- Introduced ClickUpHistoryConnector to manage OAuth-based authentication and token refresh for ClickUp API access.
- Created ClickUp connector routes for OAuth flow, including authorization and callback handling.
- Updated indexing logic to utilize the new ClickUpHistoryConnector, supporting both OAuth and legacy API token methods.
- Enhanced frontend components to reflect the new ClickUp integration and removed legacy API token forms.
Backend:
- Add connectorId to OAuth redirect URLs in all 10 connector routes
- Enables frontend to identify the specific connector created
Frontend:
- Update OAuth success handler to use connectorId for finding new connector
- Set connectorId in URL when transitioning to configure view
- Add connectorId support in URL sync effect for page refresh
- Consolidate handleAddAccountOAuth into handleConnectOAuth
- Update indexing config view to show connector type and display name
