SurfSense/surfsense_backend/app/etl_pipeline
DESKTOP-RTLN3BA\$punk 656e061f84
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
feat: add processing mode support for document uploads and ETL pipeline, improded error handling ux
- Introduced a `ProcessingMode` enum to differentiate between basic and premium processing modes.
- Updated `EtlRequest` to include a `processing_mode` field, defaulting to basic.
- Enhanced ETL pipeline services to utilize the selected processing mode for Azure Document Intelligence and LlamaCloud parsing.
- Modified various routes and services to handle processing mode, affecting document upload and indexing tasks.
- Improved error handling and logging to include processing mode details.
- Added tests to validate processing mode functionality and its impact on ETL operations.
2026-04-14 21:26:00 -07:00
..
parsers feat: add processing mode support for document uploads and ETL pipeline, improded error handling ux 2026-04-14 21:26:00 -07:00
__init__.py feat: implement ETL pipeline with file classification and extraction services 2026-04-05 17:25:25 +05:30
constants.py feat: implement ETL pipeline with file classification and extraction services 2026-04-05 17:25:25 +05:30
etl_document.py feat: add processing mode support for document uploads and ETL pipeline, improded error handling ux 2026-04-14 21:26:00 -07:00
etl_pipeline_service.py feat: add processing mode support for document uploads and ETL pipeline, improded error handling ux 2026-04-14 21:26:00 -07:00
exceptions.py refactor: implement file type classification for supported extensions across Dropbox, Google Drive, and OneDrive connectors, enhancing file handling and error management 2026-04-06 22:03:47 +05:30
file_classifier.py Route uploaded images to vision LLM with document-parser fallback 2026-04-09 14:33:33 +02:00