mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-06-16 21:05:20 +02:00
e5cb6bfacf
- Added `index_batch_parallel` method to enable concurrent indexing of documents with bounded concurrency, improving performance and efficiency. - Refactored existing indexing logic to utilize `asyncio.to_thread` for non-blocking execution of embedding and chunking functions. - Introduced unit tests to validate the functionality of the new parallel indexing method, ensuring robustness and error handling during document processing. |
||
|---|---|---|
| .. | ||
| agents | ||
| config | ||
| connectors | ||
| indexing_pipeline | ||
| prompts | ||
| retriever | ||
| routes | ||
| schemas | ||
| services | ||
| tasks | ||
| templates | ||
| utils | ||
| __init__.py | ||
| app.py | ||
| celery_app.py | ||
| db.py | ||
| users.py | ||