SurfSense/surfsense_backend/app/tasks/connector_indexers
Anish Sarkar c016962064 feat: implement parallel file downloading and indexing in Google Drive indexer
- Added `_download_files_parallel` function to enable concurrent downloading of files from Google Drive, improving efficiency in document processing.
- Introduced `_download_and_index` function to handle the parallel downloading and indexing phases, streamlining the overall workflow.
- Updated `_index_full_scan` and `_index_with_delta_sync` methods to utilize the new parallel downloading functionality, enhancing performance.
- Added unit tests to validate the new parallel downloading and indexing logic, ensuring robustness and error handling during document processing.
2026-03-26 23:53:26 +05:30
..
__init__.py refactor: remove unused COMPOSIO_CONNECTOR migration and linting 2026-01-22 16:43:08 -08:00
airtable_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
base.py chore: ran linting 2026-03-17 04:40:46 +05:30
bookstack_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
clickup_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
confluence_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
discord_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
elasticsearch_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
github_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
google_calendar_indexer.py feat: implement parallel indexing for Google Calendar and Gmail connectors 2026-03-26 19:34:04 +05:30
google_drive_indexer.py feat: implement parallel file downloading and indexing in Google Drive indexer 2026-03-26 23:53:26 +05:30
google_gmail_indexer.py feat: implement parallel indexing for Google Calendar and Gmail connectors 2026-03-26 19:34:04 +05:30
jira_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
linear_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
luma_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
notion_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
obsidian_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
slack_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
teams_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00
webcrawler_indexer.py fix: critical timestamp parsing and audit fixes 2026-03-23 19:49:28 +02:00