Commit graph

1421 commits

Author SHA1 Message Date
DESKTOP-RTLN3BA\$punk
ff4e0f9b62 feat: no login experience and prem tokens
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
2026-04-15 17:02:00 -07:00
DESKTOP-RTLN3BA\$punk
656e061f84 feat: add processing mode support for document uploads and ETL pipeline, improded error handling ux
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
- Introduced a `ProcessingMode` enum to differentiate between basic and premium processing modes.
- Updated `EtlRequest` to include a `processing_mode` field, defaulting to basic.
- Enhanced ETL pipeline services to utilize the selected processing mode for Azure Document Intelligence and LlamaCloud parsing.
- Modified various routes and services to handle processing mode, affecting document upload and indexing tasks.
- Improved error handling and logging to include processing mode details.
- Added tests to validate processing mode functionality and its impact on ETL operations.
2026-04-14 21:26:00 -07:00
Anish Sarkar
9fc0976d5e chore: ran linting 2026-04-14 21:52:26 +05:30
Anish Sarkar
f01ddf3f0a feat: implement token usage recording in chat routes and enhance title generation handling 2026-04-14 20:56:07 +05:30
Anish Sarkar
8fd7664f8f Merge remote-tracking branch 'upstream/dev' into feat/token-calculation 2026-04-14 15:49:39 +05:30
Anish Sarkar
f21bdc0668 feat: enhance LLM response handling and token usage tracking in chat services and UI components 2026-04-14 15:29:02 +05:30
Anish Sarkar
5af6005163 feat: improve token usage tracking and response handling in chat routes and services 2026-04-14 14:28:31 +05:30
DESKTOP-RTLN3BA\$punk
4bee367d4a feat: added ai file sorting 2026-04-14 01:43:30 -07:00
Anish Sarkar
3cfe53fb7f feat: implement token usage tracking for LLM calls with new accumulator and callback 2026-04-14 13:40:32 +05:30
Anish Sarkar
917f35eb33 feat: add token_usage table and relationships for tracking LLM token consumption 2026-04-14 13:26:53 +05:30
DESKTOP-RTLN3BA\$punk
5d3142332b Merge commit 'e1e4bb4706' into dev_mod 2026-04-13 20:35:04 -07:00
DESKTOP-RTLN3BA\$punk
b5301fa438 feat: enhance sandbox functionality with threading support and file download capabilities 2026-04-13 16:41:46 -07:00
Anish Sarkar
136901276a Merge remote-tracking branch 'upstream/dev' into fix/sensitive-actions 2026-04-14 04:10:00 +05:30
Anish Sarkar
3f68962772 feat: implement cache invalidation for MCP connectors upon deletion in search source connectors 2026-04-14 04:09:30 +05:30
CREDO23
9396ee9c85 fix: delete broken sandbox before creating replacement to prevent orphan accumulation 2026-04-13 21:57:06 +02:00
CREDO23
facfb1f7e7 disable file sync to sandbox, LLM handles data extraction itself 2026-04-13 20:59:31 +02:00
CREDO23
5ea2732bf2 make execute_code a Python tool: auto-wrap, print() nudge on empty output 2026-04-13 20:53:17 +02:00
CREDO23
5719a794a0 refine execute_code prompts: tool description + XML workflow guidance 2026-04-13 19:51:34 +02:00
Anish Sarkar
ec27807644 refactor: improve document processing error handling and UI state management 2026-04-13 22:22:50 +05:30
CREDO23
bb41c09eef clean execute_code tool description, remove sandbox internals 2026-04-13 18:03:22 +02:00
CREDO23
c9e3b1a08a fix KB search skipping when system message follows user message 2026-04-13 18:03:16 +02:00
Anish Sarkar
b3a8364fbd feat: add MCP Tool Trust routes for managing trusted tools 2026-04-13 20:18:12 +05:30
Anish Sarkar
8d8ba6cbe8 refactor: replace interrupt calls with request_approval utility in Linear and Notion tools
Updated the create, delete, and update functions in Linear and Notion tools to utilize the new request_approval utility for handling user approvals. This change improves code consistency and simplifies decision handling by directly merging parameters from the approval response.
2026-04-13 20:17:36 +05:30
Anish Sarkar
4875fd9211 refactor: replace interrupt calls with request_approval utility in Jira and OneDrive tools
Updated the create, delete, and update functions in Jira and OneDrive tools to utilize the new request_approval utility for handling user approvals. This change improves code consistency and simplifies decision handling by directly merging parameters from the approval response.
2026-04-13 20:17:06 +05:30
Anish Sarkar
2f59fc9c72 refactor: replace interrupt calls with request_approval utility in Google Calendar and Drive tools
Updated the create, delete, and update functions in Google Calendar and Google Drive tools to utilize the new request_approval utility for handling user approvals. This change enhances code consistency and simplifies decision handling by directly merging parameters from the approval response.
2026-04-13 20:16:09 +05:30
Anish Sarkar
85baaacd0a refactor: replace interrupt calls with request_approval utility in Gmail tools
Updated the create, send, update, and trash functions in Gmail tools to utilize the new request_approval utility for handling user approvals. This change improves code consistency and simplifies decision handling by directly merging parameters from the approval response.
2026-04-13 20:15:31 +05:30
Anish Sarkar
3eb448ec8d refactor: replace interrupt calls with unified request_approval utility in Confluence and Dropbox tools
Updated the create, delete, and update functions in Confluence and Dropbox tools to utilize the new request_approval utility for handling user approvals. This change enhances code consistency and simplifies decision handling by merging parameters directly from the approval response.
2026-04-13 20:14:50 +05:30
Anish Sarkar
82c7d4a2ab refactor: enhance deduplication logic for HITL tool calls
Updated the deduplication mechanism in the DedupHITLToolCallsMiddleware to utilize a comprehensive list of native HITL tools. The deduplication keys are now dynamically populated from both hardcoded values and metadata from StructuredTool instances. Additionally, integrated HITL approval into MCP tool creation, ensuring all tools are gated by user approval, with the ability to bypass for trusted tools.
2026-04-13 20:14:12 +05:30
Anish Sarkar
0c4fd30cce feat: add unified HITL approval utility for sensitive tool actions
This new module provides a `request_approval()` function that streamlines the process of requesting user approval for sensitive actions, including decision parsing and parameter merging. It enhances the interaction with tools by allowing for user modifications and handling trusted tools seamlessly.
2026-04-13 20:12:57 +05:30
CREDO23
823e3ebb1d pass thread_id to filesystem middleware 2026-04-13 16:23:53 +02:00
CREDO23
a7e70020b1 add execute_code tool with sandbox integration 2026-04-13 16:23:47 +02:00
CREDO23
9e8ea1fd1c harden sandbox: security params, file sync, path traversal fix 2026-04-13 16:23:40 +02:00
CREDO23
c30cc08771 Merge upstream/dev into feat/kb-export-and-folder-upload 2026-04-11 10:28:40 +02:00
Rohan Verma
7ec2973223
Merge pull request #1214 from AnishSarkar22/fix/azure-document-intelligence
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
fix: update Azure Document Intelligence to use `prebuilt-layout`
2026-04-10 14:37:55 -07:00
Anish Sarkar
bb5b90e5bd fix: update Azure Document Intelligence parser to use prebuilt-layout model 2026-04-10 20:40:54 +05:30
CREDO23
a95bf58c8f Make Vision LLM opt-in for uploads and connectors 2026-04-10 16:45:51 +02:00
Anish Sarkar
c6730c5551 feat: implement download functionality for different OS in SidebarUserProfile and DownloadButton components, enhance user experience with localized download messages 2026-04-10 19:53:13 +05:30
Anish Sarkar
0925f89b7c fix: guard display_name.split() against whitespace-only IndexError (recurseml feedback) 2026-04-10 15:29:56 +05:30
Anish Sarkar
8c9440998a chore: ran linting 2026-04-10 14:20:11 +05:30
Anish Sarkar
8d7b5bfe36 feat: enhance memory management protocols to enforce structured headings, utilize user first names in entries, and improve clarity in memory update rules 2026-04-10 04:51:56 +05:30
Anish Sarkar
b8e1c9801b feat: update memory extraction and management protocols to enforce structured bullet formats, utilize user first names, and enhance validation for team and user memory entries 2026-04-10 04:36:51 +05:30
Anish Sarkar
c06b82647d feat: refine team memory extraction decision policy to prioritize durable context and clarify criteria for updates, enhancing memory relevance and management 2026-04-10 02:02:14 +05:30
Anish Sarkar
a0883d2ab6 feat: implement team memory extraction and validation mechanisms, enhancing memory management by enforcing scope restrictions and improving memory persistence for shared threads 2026-04-10 01:54:00 +05:30
CREDO23
0aefcbd504 Remove vision LLM from desktop folder watcher 2026-04-09 22:06:06 +02:00
Anish Sarkar
33626d4f91 chore: ran linting 2026-04-10 00:31:43 +05:30
Anish Sarkar
84fc72e596 feat: implement background memory extraction and editing capabilities for user and team memory management, enhancing long-term memory persistence and user interaction 2026-04-10 00:21:55 +05:30
Anish Sarkar
cd72fa9a48 refactor: remove memory extraction functionality and update memory management protocols to ensure immediate updates for user and team interactions 2026-04-09 23:30:17 +05:30
CREDO23
4ccdd80e26 Harden vision LLM fallback, folder upload validation, and export memory 2026-04-09 16:14:53 +02:00
CREDO23
e164fe0612 Fix misleading log when vision LLM fails vs not provided 2026-04-09 15:29:39 +02:00
CREDO23
ff2a9c77f9 Pass vision_llm in legacy process_file_in_background path 2026-04-09 15:28:21 +02:00