- Introduced `download_file_to_disk` method to stream files directly to disk in chunks, reducing memory usage during downloads.
- Updated `download_and_extract_content` function to utilize the new streaming download method for binary files, enhancing efficiency in handling large files.
- Improved error handling for download operations, providing clearer feedback on failures.
- Introduced `index_google_drive_selected_files` function to enable indexing of multiple user-selected files in parallel, improving efficiency.
- Refactored existing indexing logic to handle batch processing, including error handling for individual file failures.
- Added unit tests for the new batch indexing functionality, ensuring robustness and proper error collection during the indexing process.
- Introduced an asyncio lock to the GoogleDriveClient to ensure thread-safe access to the service instance.
- Refactored the get_service method to utilize the lock, preventing concurrent attempts to create the service and improving stability in multi-threaded environments.
- Added `_download_files_parallel` function to enable concurrent downloading of files from Google Drive, improving efficiency in document processing.
- Introduced `_download_and_index` function to handle the parallel downloading and indexing phases, streamlining the overall workflow.
- Updated `_index_full_scan` and `_index_with_delta_sync` methods to utilize the new parallel downloading functionality, enhancing performance.
- Added unit tests to validate the new parallel downloading and indexing logic, ensuring robustness and error handling during document processing.
- Added performance logging to the `index_batch_parallel` method, capturing metrics for document indexing duration and concurrency.
- Introduced timing measurements for both the overall indexing process and the parallel document gathering phase, improving observability of the indexing workflow.
- Updated logging statements to provide detailed insights into the number of documents processed, indexed, and failed during the indexing operation.
- Refactored Google Calendar and Gmail indexers to utilize the new `index_batch_parallel` method for concurrent document indexing, enhancing performance.
- Updated the indexing logic to replace serial processing with parallel execution, allowing for improved efficiency in handling multiple documents.
- Adjusted logging and error handling to accommodate the new parallel processing approach, ensuring robust operation during indexing.
- Enhanced unit tests to validate the functionality of the parallel indexing method and its integration with existing workflows.
- Added `index_batch_parallel` method to enable concurrent indexing of documents with bounded concurrency, improving performance and efficiency.
- Refactored existing indexing logic to utilize `asyncio.to_thread` for non-blocking execution of embedding and chunking functions.
- Introduced unit tests to validate the functionality of the new parallel indexing method, ensuring robustness and error handling during document processing.
- Introduced helper functions `_is_date_only` and `_build_time_body` to streamline the construction of event start and end times for all-day and timed events.
- Refactored the `create_update_calendar_event_tool` to utilize the new helper functions, improving code readability and maintainability.
- Updated the Google Calendar sync service to ensure proper handling of calendar IDs with a default fallback to "primary".
- Modified the ApprovalCard component to simplify the construction of event update arguments, enhancing clarity and reducing redundancy.
- Modified the `_should_skip_file` function to prevent skipping of documents with a FAILED status, ensuring they are reprocessed even if their content remains unchanged.
- Added a new integration test to verify that FAILED documents are not skipped during the indexing process.
- Introduced integration tests for Calendar, Drive, and Gmail indexers to ensure proper document creation and migration.
- Added tests for batch indexing functionality to validate the processing of multiple documents.
- Implemented tests for legacy document migration to verify updates to document types and hashes.
- Enhanced test coverage for the IndexingPipelineService to ensure robust functionality across various document types.
- Added `download_and_extract_content` function to extract content from Google Drive files as markdown.
- Updated Google Drive indexer to utilize the new content extraction method.
- Implemented document migration logic to update legacy Composio document types to their native Google types.
- Introduced identifier hashing for stable document identification.
- Improved file pre-filtering to handle unchanged and rename-only files efficiently.
Resolve 8 conflicts:
- Accept upstream deletion of 3 composio_*_connector.py (unified Google connectors)
- Accept our deletion of ElectricProvider.tsx, use-connectors-electric.ts,
use-messages-electric.ts (replaced by Zero equivalents)
- Keep both new deps in package.json (@rocicorp/zero + @slate-serializers/html)
- Regenerate pnpm-lock.yaml
Check information_schema before creating table to avoid
DuplicateTableError when table already exists from model metadata.
Original op.create_table preserved exactly, only skipped if exists.
- Fix timestamp conversion: String(epochMs) → new Date(epochMs).toISOString()
in use-messages-sync, use-comments-sync, use-documents, use-inbox.
Without this, date comparisons (isEdited, cutoff filters) would fail.
- Fix updated_at: undefined → null in use-inbox to match InboxItem type
- Fix ZeroProvider: skip Zero connection for unauthenticated users
- Clean 30+ stale "Electric SQL" comments in backend Python code
Remove all Electric SQL client code, Docker service, env vars, CI build
args, install scripts, and documentation. Feature hooks that depend on
Electric are intentionally left in place to be rewritten with Rocicorp
Zero in subsequent commits.
Deleted:
- lib/electric/ (client.ts, context.ts, auth.ts, baseline.ts)
- ElectricProvider.tsx
- docker/scripts/init-electric-user.sh
- content/docs/how-to/electric-sql.mdx
Cleaned:
- package.json (4 @electric-sql/* deps)
- app/layout.tsx, UserDropdown.tsx, LayoutDataProvider.tsx
- docker-compose.yml, docker-compose.dev.yml
- Dockerfile, docker-entrypoint.js
- .env.example (frontend, docker, backend)
- CI workflows, install scripts, docs
- Refactored token refresh logic in ConfluenceHistoryConnector and JiraHistoryConnector to use lazy imports, avoiding circular dependencies.
- Enhanced the ComposerAction component to manage tool availability based on connected types, adding support for Jira and Confluence tools.
- Updated tool icon management to include Jira and Confluence, improving the user interface for tool interactions.
- Added support for grouping tools with connector icons, improving organization and user interaction.
- Implemented logic to toggle tool groups based on their enabled/disabled state, enhancing user experience.
- Updated the display of enabled tools count to reflect the new grouping structure.
- Introduced a new constant for connector tool icon paths to streamline icon management across components.
- Added a new tool action for updating Gmail drafts in the backend agent, expanding functionality.
- Introduced DedupHITLToolCallsMiddleware to prevent duplicate HITL tool calls within a single LLM response, ensuring only the first occurrence of each tool call is retained.
- Updated the create_surfsense_deep_agent function to include the new middleware, enhancing the efficiency of tool interactions.
- Added a new middleware file for better organization and maintainability of the codebase.
- Added a guideline to ensure that each tool (Gmail, Google Calendar, Google Drive, Linear, Notion) is called only once per user request.
- Updated documentation to clarify that the system will automatically select the most relevant match when multiple items share the same title or subject, enhancing user experience and preventing redundant calls.
- Improved the update_gmail_draft tool to allow users to review and edit draft content before applying changes.
- Added logic to generate draft body content based on user requests and conversation context.
- Implemented fetching of existing draft body to facilitate user edits in the approval card.
- Updated UI components to support displaying and editing existing draft content, enhancing user experience.
- Introduced a new tool to update existing Gmail drafts, allowing users to modify draft content, recipients, and subject lines.
- Updated the Gmail tools registry to include the new update_gmail_draft tool.
- Enhanced the GmailKBSyncService to support draft ID handling during synchronization.
- Added UI components for the update draft functionality in the web application, improving user interaction with Gmail drafts.
- Updated the GmailAccount class to extract email from the connector name when formatted with " - ".
- Added new tool actions for Gmail and Google Calendar, including creating drafts, sending emails, and managing calendar events, improving integration and user functionality.
- Implemented logic to persist authentication expiration status for connectors when insufficient permissions are detected, improving error handling and user experience.
- Updated messages to guide users to re-authenticate in connector settings for Gmail, Google Calendar, and Google Drive tools.
- Added InsufficientPermissionsResult type and corresponding UI components to display permission-related messages consistently across Gmail and Google Calendar tools.
