mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-05-05 22:02:39 +02:00
c016962064
- Added `_download_files_parallel` function to enable concurrent downloading of files from Google Drive, improving efficiency in document processing. - Introduced `_download_and_index` function to handle the parallel downloading and indexing phases, streamlining the overall workflow. - Updated `_index_full_scan` and `_index_with_delta_sync` methods to utilize the new parallel downloading functionality, enhancing performance. - Added unit tests to validate the new parallel downloading and indexing logic, ensuring robustness and error handling during document processing. |
||
|---|---|---|
| .. | ||
| agents | ||
| config | ||
| connectors | ||
| indexing_pipeline | ||
| prompts | ||
| retriever | ||
| routes | ||
| schemas | ||
| services | ||
| tasks | ||
| templates | ||
| utils | ||
| __init__.py | ||
| app.py | ||
| celery_app.py | ||
| db.py | ||
| users.py | ||