apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-04-25 00:36:31 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

feat: add processing mode support for document uploads and ETL pipeline, improded error handling ux
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Details
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Details
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Details
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Details
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Details
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Details
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
Details

Browse source 
- Introduced a `ProcessingMode` enum to differentiate between basic and premium processing modes.
- Updated `EtlRequest` to include a `processing_mode` field, defaulting to basic.
- Enhanced ETL pipeline services to utilize the selected processing mode for Azure Document Intelligence and LlamaCloud parsing.
- Modified various routes and services to handle processing mode, affecting document upload and indexing tasks.
- Improved error handling and logging to include processing mode details.
- Added tests to validate processing mode functionality and its impact on ETL operations.
This commit is contained in:
DESKTOP-RTLN3BA\$punk 2026-04-14 21:26:00 -07:00
parent b659f41bab
commit 656e061f84
104 changed files with 1900 additions and 909 deletions
Download patch file Download diff file Expand all files Collapse all files

19
surfsense_web/components/layout/ui/sidebar/DocumentsSidebar.tsx
View file

@ -167,7 +167,6 @@ export function DocumentsSidebar({
setFolderWatchOpen(true);
}, [setWatchInitialFolder, setFolderWatchOpen]);
const refreshWatchedIds = useCallback(async () => {
if (!electronAPI?.getWatchedFolders) return;
const api = electronAPI;
@ -675,10 +674,10 @@ export function DocumentsSidebar({
function collectSubtreeDocs(parentId: number): DocumentNodeDoc[] {
const directDocs = (treeDocuments ?? []).filter(
(d) =>
d.folderId === parentId &&
d.status?.state !== "pending" &&
d.status?.state !== "processing" &&
d.status?.state !== "failed"
d.folderId === parentId &&
d.status?.state !== "pending" &&
d.status?.state !== "processing" &&
d.status?.state !== "failed"
);
const childFolders = foldersByParent[String(parentId)] ?? [];
const descendantDocs = childFolders.flatMap((cf) => collectSubtreeDocs(cf.id));
@ -1123,16 +1122,14 @@ export function DocumentsSidebar({
<AlertDialogHeader>
<AlertDialogTitle>Enable AI File Sorting?</AlertDialogTitle>
<AlertDialogDescription>
All documents in this search space will be organized into folders by
connector type, date, and AI-generated categories. New documents will
also be sorted automatically. You can disable this at any time.
All documents in this search space will be organized into folders by connector type,
date, and AI-generated categories. New documents will also be sorted automatically.
You can disable this at any time.
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>Cancel</AlertDialogCancel>
<AlertDialogAction onClick={handleConfirmEnableAiSort}>
Enable
</AlertDialogAction>
<AlertDialogAction onClick={handleConfirmEnableAiSort}>Enable</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>