- Implemented per-thread HTTP transport for concurrent downloads to ensure thread safety.
- Refactored `download_file` and `download_file_to_disk` methods to utilize blocking calls on separate threads, improving performance during file operations.
- Added logging to track the start and end of download and export processes, providing better visibility into execution time.
- Updated unit tests to verify parallel execution of download and export operations, ensuring efficiency in handling multiple requests.
- Refactored Confluence and Jira indexers to utilize the shared IndexingPipelineService for improved document processing.
- Updated the `_build_connector_doc` function in both indexers to create ConnectorDocument instances with enhanced metadata and fallback summaries.
- Modified the `index_confluence_pages` and `index_jira_issues` functions to return a tuple of (indexed_count, skipped_count, warning_or_error_message) for better error handling and reporting.
- Added unit tests for both indexers to validate the new parallel processing logic and ensure correct document creation and indexing behavior.
- Added a reentrant lock to ensure thread-safe access to the tokenizer and embedding model, preventing runtime errors during concurrent operations.
- Updated the `truncate_for_embedding` and `embed_text` functions to utilize the lock, ensuring safe execution in multi-threaded environments.
- Enhanced the `embed_texts` function to maintain thread safety while processing multiple texts for embedding.
- Refactored Linear and Notion indexers to utilize the shared IndexingPipelineService for improved document deduplication, summarization, chunking, and embedding with bounded parallel indexing.
- Updated the `_build_connector_doc` function in both indexers to create ConnectorDocument instances with enhanced metadata and fallback summaries.
- Modified the `index_linear_issues` and `index_notion_pages` functions to return a tuple of (indexed_count, skipped_count, warning_or_error_message) for better error handling and reporting.
- Added unit tests for both indexers to validate the new parallel processing logic and ensure correct document creation and indexing behavior.
- Introduced `download_file_to_disk` method to stream files directly to disk in chunks, reducing memory usage during downloads.
- Updated `download_and_extract_content` function to utilize the new streaming download method for binary files, enhancing efficiency in handling large files.
- Improved error handling for download operations, providing clearer feedback on failures.
- Introduced `index_google_drive_selected_files` function to enable indexing of multiple user-selected files in parallel, improving efficiency.
- Refactored existing indexing logic to handle batch processing, including error handling for individual file failures.
- Added unit tests for the new batch indexing functionality, ensuring robustness and proper error collection during the indexing process.
- Introduced an asyncio lock to the GoogleDriveClient to ensure thread-safe access to the service instance.
- Refactored the get_service method to utilize the lock, preventing concurrent attempts to create the service and improving stability in multi-threaded environments.
- Added `_download_files_parallel` function to enable concurrent downloading of files from Google Drive, improving efficiency in document processing.
- Introduced `_download_and_index` function to handle the parallel downloading and indexing phases, streamlining the overall workflow.
- Updated `_index_full_scan` and `_index_with_delta_sync` methods to utilize the new parallel downloading functionality, enhancing performance.
- Added unit tests to validate the new parallel downloading and indexing logic, ensuring robustness and error handling during document processing.
- Added performance logging to the `index_batch_parallel` method, capturing metrics for document indexing duration and concurrency.
- Introduced timing measurements for both the overall indexing process and the parallel document gathering phase, improving observability of the indexing workflow.
- Updated logging statements to provide detailed insights into the number of documents processed, indexed, and failed during the indexing operation.
- Refactored Google Calendar and Gmail indexers to utilize the new `index_batch_parallel` method for concurrent document indexing, enhancing performance.
- Updated the indexing logic to replace serial processing with parallel execution, allowing for improved efficiency in handling multiple documents.
- Adjusted logging and error handling to accommodate the new parallel processing approach, ensuring robust operation during indexing.
- Enhanced unit tests to validate the functionality of the parallel indexing method and its integration with existing workflows.
- Added `index_batch_parallel` method to enable concurrent indexing of documents with bounded concurrency, improving performance and efficiency.
- Refactored existing indexing logic to utilize `asyncio.to_thread` for non-blocking execution of embedding and chunking functions.
- Introduced unit tests to validate the functionality of the new parallel indexing method, ensuring robustness and error handling during document processing.
- Introduced helper functions `_is_date_only` and `_build_time_body` to streamline the construction of event start and end times for all-day and timed events.
- Refactored the `create_update_calendar_event_tool` to utilize the new helper functions, improving code readability and maintainability.
- Updated the Google Calendar sync service to ensure proper handling of calendar IDs with a default fallback to "primary".
- Modified the ApprovalCard component to simplify the construction of event update arguments, enhancing clarity and reducing redundancy.
- Modified the `_should_skip_file` function to prevent skipping of documents with a FAILED status, ensuring they are reprocessed even if their content remains unchanged.
- Added a new integration test to verify that FAILED documents are not skipped during the indexing process.
- Added `download_and_extract_content` function to extract content from Google Drive files as markdown.
- Updated Google Drive indexer to utilize the new content extraction method.
- Implemented document migration logic to update legacy Composio document types to their native Google types.
- Introduced identifier hashing for stable document identification.
- Improved file pre-filtering to handle unchanged and rename-only files efficiently.
Resolve 8 conflicts:
- Accept upstream deletion of 3 composio_*_connector.py (unified Google connectors)
- Accept our deletion of ElectricProvider.tsx, use-connectors-electric.ts,
use-messages-electric.ts (replaced by Zero equivalents)
- Keep both new deps in package.json (@rocicorp/zero + @slate-serializers/html)
- Regenerate pnpm-lock.yaml
- Fix timestamp conversion: String(epochMs) → new Date(epochMs).toISOString()
in use-messages-sync, use-comments-sync, use-documents, use-inbox.
Without this, date comparisons (isEdited, cutoff filters) would fail.
- Fix updated_at: undefined → null in use-inbox to match InboxItem type
- Fix ZeroProvider: skip Zero connection for unauthenticated users
- Clean 30+ stale "Electric SQL" comments in backend Python code
- Refactored token refresh logic in ConfluenceHistoryConnector and JiraHistoryConnector to use lazy imports, avoiding circular dependencies.
- Enhanced the ComposerAction component to manage tool availability based on connected types, adding support for Jira and Confluence tools.
- Updated tool icon management to include Jira and Confluence, improving the user interface for tool interactions.
- Added support for grouping tools with connector icons, improving organization and user interaction.
- Implemented logic to toggle tool groups based on their enabled/disabled state, enhancing user experience.
- Updated the display of enabled tools count to reflect the new grouping structure.
- Introduced a new constant for connector tool icon paths to streamline icon management across components.
- Added a new tool action for updating Gmail drafts in the backend agent, expanding functionality.
- Introduced DedupHITLToolCallsMiddleware to prevent duplicate HITL tool calls within a single LLM response, ensuring only the first occurrence of each tool call is retained.
- Updated the create_surfsense_deep_agent function to include the new middleware, enhancing the efficiency of tool interactions.
- Added a new middleware file for better organization and maintainability of the codebase.
- Added a guideline to ensure that each tool (Gmail, Google Calendar, Google Drive, Linear, Notion) is called only once per user request.
- Updated documentation to clarify that the system will automatically select the most relevant match when multiple items share the same title or subject, enhancing user experience and preventing redundant calls.
- Improved the update_gmail_draft tool to allow users to review and edit draft content before applying changes.
- Added logic to generate draft body content based on user requests and conversation context.
- Implemented fetching of existing draft body to facilitate user edits in the approval card.
- Updated UI components to support displaying and editing existing draft content, enhancing user experience.
- Introduced a new tool to update existing Gmail drafts, allowing users to modify draft content, recipients, and subject lines.
- Updated the Gmail tools registry to include the new update_gmail_draft tool.
- Enhanced the GmailKBSyncService to support draft ID handling during synchronization.
- Added UI components for the update draft functionality in the web application, improving user interaction with Gmail drafts.
- Updated the GmailAccount class to extract email from the connector name when formatted with " - ".
- Added new tool actions for Gmail and Google Calendar, including creating drafts, sending emails, and managing calendar events, improving integration and user functionality.
- Implemented logic to persist authentication expiration status for connectors when insufficient permissions are detected, improving error handling and user experience.
- Updated messages to guide users to re-authenticate in connector settings for Gmail, Google Calendar, and Google Drive tools.
- Added InsufficientPermissionsResult type and corresponding UI components to display permission-related messages consistently across Gmail and Google Calendar tools.
- Added `sync_after_create` methods in `LinearKBSyncService` and `NotionKBSyncService` to handle synchronization of newly created issues and pages with the knowledge base.
- Enhanced the `create_issue.py` and `create_page.py` tools to provide user feedback on the success of the knowledge base update, indicating whether the content has been synced or will be added in the next scheduled sync.
- Improved error handling during synchronization to log failures and manage duplicate document scenarios effectively.
- Implemented checks for expired authentication in the Notion page creation and deletion tools, returning appropriate error messages for re-authentication.
- Updated the Notion tool metadata service to track account health and persist authentication status, improving error handling and user feedback during operations.
- Added a new GoogleDriveKBSyncService to handle synchronization of newly created Google Drive files with the knowledge base.
- Enhanced the create_file.py tool to include feedback on the success of the knowledge base update, informing users if their file has been added or will be synced later.
- Updated the Google Drive tool metadata service to include parent folder information for improved file organization.
- Modified the UI components to support selection of parent folders during file creation, enhancing user experience and file management.
