mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-04-29 02:46:25 +02:00
feat: add processing mode support for document uploads and ETL pipeline, improded error handling ux
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
- Introduced a `ProcessingMode` enum to differentiate between basic and premium processing modes. - Updated `EtlRequest` to include a `processing_mode` field, defaulting to basic. - Enhanced ETL pipeline services to utilize the selected processing mode for Azure Document Intelligence and LlamaCloud parsing. - Modified various routes and services to handle processing mode, affecting document upload and indexing tasks. - Improved error handling and logging to include processing mode details. - Added tests to validate processing mode functionality and its impact on ETL operations.
This commit is contained in:
parent
b659f41bab
commit
656e061f84
104 changed files with 1900 additions and 909 deletions
|
|
@ -145,18 +145,14 @@ export function GeneralSettingsManager({ searchSpaceId }: GeneralSettingsManager
|
|||
<form onSubmit={onSubmit} className="space-y-6">
|
||||
<div className="flex flex-col gap-6">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="search-space-name">
|
||||
{t("general_name_label")}
|
||||
</Label>
|
||||
<Label htmlFor="search-space-name">{t("general_name_label")}</Label>
|
||||
<Input
|
||||
id="search-space-name"
|
||||
placeholder={t("general_name_placeholder")}
|
||||
value={name}
|
||||
onChange={(e) => setName(e.target.value)}
|
||||
/>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
{t("general_name_description")}
|
||||
</p>
|
||||
<p className="text-xs text-muted-foreground">{t("general_name_description")}</p>
|
||||
</div>
|
||||
|
||||
<div className="space-y-2">
|
||||
|
|
@ -170,9 +166,7 @@ export function GeneralSettingsManager({ searchSpaceId }: GeneralSettingsManager
|
|||
value={description}
|
||||
onChange={(e) => setDescription(e.target.value)}
|
||||
/>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
{t("general_description_description")}
|
||||
</p>
|
||||
<p className="text-xs text-muted-foreground">{t("general_description_description")}</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
|
@ -189,28 +183,28 @@ export function GeneralSettingsManager({ searchSpaceId }: GeneralSettingsManager
|
|||
</div>
|
||||
</form>
|
||||
|
||||
<div className="border-t pt-6 flex flex-col gap-3 md:flex-row md:items-center md:justify-between">
|
||||
<div className="space-y-1">
|
||||
<Label>Export knowledge base</Label>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Download all documents in this search space as a ZIP of markdown files.
|
||||
</p>
|
||||
<div className="border-t pt-6 flex flex-col gap-3 md:flex-row md:items-center md:justify-between">
|
||||
<div className="space-y-1">
|
||||
<Label>Export knowledge base</Label>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Download all documents in this search space as a ZIP of markdown files.
|
||||
</p>
|
||||
</div>
|
||||
<Button
|
||||
type="button"
|
||||
variant="secondary"
|
||||
size="sm"
|
||||
disabled={isExporting}
|
||||
onClick={handleExportKB}
|
||||
className="relative w-fit shrink-0"
|
||||
>
|
||||
<span className={isExporting ? "opacity-0" : ""}>
|
||||
<FolderArchive className="h-3 w-3 opacity-60" />
|
||||
</span>
|
||||
<span className={isExporting ? "opacity-0" : ""}>Export</span>
|
||||
{isExporting && <Spinner size="sm" className="absolute" />}
|
||||
</Button>
|
||||
</div>
|
||||
<Button
|
||||
type="button"
|
||||
variant="secondary"
|
||||
size="sm"
|
||||
disabled={isExporting}
|
||||
onClick={handleExportKB}
|
||||
className="relative w-fit shrink-0"
|
||||
>
|
||||
<span className={isExporting ? "opacity-0" : ""}>
|
||||
<FolderArchive className="h-3 w-3 opacity-60" />
|
||||
</span>
|
||||
<span className={isExporting ? "opacity-0" : ""}>Export</span>
|
||||
{isExporting && <Spinner size="sm" className="absolute" />}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue