Commit graph

1682 commits

Author SHA1 Message Date
DESKTOP-RTLN3BA\$punk
ff4e0f9b62 feat: no login experience and prem tokens
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
2026-04-15 17:02:00 -07:00
DESKTOP-RTLN3BA\$punk
656e061f84 feat: add processing mode support for document uploads and ETL pipeline, improded error handling ux
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
- Introduced a `ProcessingMode` enum to differentiate between basic and premium processing modes.
- Updated `EtlRequest` to include a `processing_mode` field, defaulting to basic.
- Enhanced ETL pipeline services to utilize the selected processing mode for Azure Document Intelligence and LlamaCloud parsing.
- Modified various routes and services to handle processing mode, affecting document upload and indexing tasks.
- Improved error handling and logging to include processing mode details.
- Added tests to validate processing mode functionality and its impact on ETL operations.
2026-04-14 21:26:00 -07:00
Anish Sarkar
9fc0976d5e chore: ran linting 2026-04-14 21:52:26 +05:30
Anish Sarkar
f01ddf3f0a feat: implement token usage recording in chat routes and enhance title generation handling 2026-04-14 20:56:07 +05:30
Anish Sarkar
8fd7664f8f Merge remote-tracking branch 'upstream/dev' into feat/token-calculation 2026-04-14 15:49:39 +05:30
Anish Sarkar
1f9840d4a4 feat: update alembic migration number 2026-04-14 15:49:33 +05:30
Anish Sarkar
f21bdc0668 feat: enhance LLM response handling and token usage tracking in chat services and UI components 2026-04-14 15:29:02 +05:30
Anish Sarkar
5af6005163 feat: improve token usage tracking and response handling in chat routes and services 2026-04-14 14:28:31 +05:30
DESKTOP-RTLN3BA\$punk
4bee367d4a feat: added ai file sorting 2026-04-14 01:43:30 -07:00
Anish Sarkar
3cfe53fb7f feat: implement token usage tracking for LLM calls with new accumulator and callback 2026-04-14 13:40:32 +05:30
Anish Sarkar
917f35eb33 feat: add token_usage table and relationships for tracking LLM token consumption 2026-04-14 13:26:53 +05:30
DESKTOP-RTLN3BA\$punk
5d3142332b Merge commit 'e1e4bb4706' into dev_mod 2026-04-13 20:35:04 -07:00
DESKTOP-RTLN3BA\$punk
b5301fa438 feat: enhance sandbox functionality with threading support and file download capabilities 2026-04-13 16:41:46 -07:00
Anish Sarkar
136901276a Merge remote-tracking branch 'upstream/dev' into fix/sensitive-actions 2026-04-14 04:10:00 +05:30
Anish Sarkar
3f68962772 feat: implement cache invalidation for MCP connectors upon deletion in search source connectors 2026-04-14 04:09:30 +05:30
CREDO23
9396ee9c85 fix: delete broken sandbox before creating replacement to prevent orphan accumulation 2026-04-13 21:57:06 +02:00
CREDO23
facfb1f7e7 disable file sync to sandbox, LLM handles data extraction itself 2026-04-13 20:59:31 +02:00
CREDO23
5ea2732bf2 make execute_code a Python tool: auto-wrap, print() nudge on empty output 2026-04-13 20:53:17 +02:00
CREDO23
5719a794a0 refine execute_code prompts: tool description + XML workflow guidance 2026-04-13 19:51:34 +02:00
Anish Sarkar
ec27807644 refactor: improve document processing error handling and UI state management 2026-04-13 22:22:50 +05:30
CREDO23
a71338e368 rewrite snapshot script: use run_commands, dotenv, idempotent delete 2026-04-13 18:29:17 +02:00
CREDO23
c4a5ac4144 delete existing snapshot before recreating 2026-04-13 18:11:40 +02:00
CREDO23
4bd9ff15eb add /documents symlink to sandbox snapshot 2026-04-13 18:06:45 +02:00
CREDO23
bb41c09eef clean execute_code tool description, remove sandbox internals 2026-04-13 18:03:22 +02:00
CREDO23
c9e3b1a08a fix KB search skipping when system message follows user message 2026-04-13 18:03:16 +02:00
Anish Sarkar
b3a8364fbd feat: add MCP Tool Trust routes for managing trusted tools 2026-04-13 20:18:12 +05:30
Anish Sarkar
8d8ba6cbe8 refactor: replace interrupt calls with request_approval utility in Linear and Notion tools
Updated the create, delete, and update functions in Linear and Notion tools to utilize the new request_approval utility for handling user approvals. This change improves code consistency and simplifies decision handling by directly merging parameters from the approval response.
2026-04-13 20:17:36 +05:30
Anish Sarkar
4875fd9211 refactor: replace interrupt calls with request_approval utility in Jira and OneDrive tools
Updated the create, delete, and update functions in Jira and OneDrive tools to utilize the new request_approval utility for handling user approvals. This change improves code consistency and simplifies decision handling by directly merging parameters from the approval response.
2026-04-13 20:17:06 +05:30
Anish Sarkar
2f59fc9c72 refactor: replace interrupt calls with request_approval utility in Google Calendar and Drive tools
Updated the create, delete, and update functions in Google Calendar and Google Drive tools to utilize the new request_approval utility for handling user approvals. This change enhances code consistency and simplifies decision handling by directly merging parameters from the approval response.
2026-04-13 20:16:09 +05:30
Anish Sarkar
85baaacd0a refactor: replace interrupt calls with request_approval utility in Gmail tools
Updated the create, send, update, and trash functions in Gmail tools to utilize the new request_approval utility for handling user approvals. This change improves code consistency and simplifies decision handling by directly merging parameters from the approval response.
2026-04-13 20:15:31 +05:30
Anish Sarkar
3eb448ec8d refactor: replace interrupt calls with unified request_approval utility in Confluence and Dropbox tools
Updated the create, delete, and update functions in Confluence and Dropbox tools to utilize the new request_approval utility for handling user approvals. This change enhances code consistency and simplifies decision handling by merging parameters directly from the approval response.
2026-04-13 20:14:50 +05:30
Anish Sarkar
82c7d4a2ab refactor: enhance deduplication logic for HITL tool calls
Updated the deduplication mechanism in the DedupHITLToolCallsMiddleware to utilize a comprehensive list of native HITL tools. The deduplication keys are now dynamically populated from both hardcoded values and metadata from StructuredTool instances. Additionally, integrated HITL approval into MCP tool creation, ensuring all tools are gated by user approval, with the ability to bypass for trusted tools.
2026-04-13 20:14:12 +05:30
Anish Sarkar
0c4fd30cce feat: add unified HITL approval utility for sensitive tool actions
This new module provides a `request_approval()` function that streamlines the process of requesting user approval for sensitive actions, including decision parsing and parameter merging. It enhances the interaction with tools by allowing for user modifications and handling trusted tools seamlessly.
2026-04-13 20:12:57 +05:30
CREDO23
c72b737a81 add snapshot creation script for surfsense-sandbox 2026-04-13 16:30:27 +02:00
CREDO23
1e74c87aa1 add Daytona sandbox env vars to .env.example 2026-04-13 16:23:58 +02:00
CREDO23
823e3ebb1d pass thread_id to filesystem middleware 2026-04-13 16:23:53 +02:00
CREDO23
a7e70020b1 add execute_code tool with sandbox integration 2026-04-13 16:23:47 +02:00
CREDO23
9e8ea1fd1c harden sandbox: security params, file sync, path traversal fix 2026-04-13 16:23:40 +02:00
CREDO23
ec3bd2f5f4 fix: renumber vision_llm migration from 121 to 123 to resolve conflict with upstream 2026-04-11 11:02:44 +02:00
CREDO23
c30cc08771 Merge upstream/dev into feat/kb-export-and-folder-upload 2026-04-11 10:28:40 +02:00
Rohan Verma
7ec2973223
Merge pull request #1214 from AnishSarkar22/fix/azure-document-intelligence
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
fix: update Azure Document Intelligence to use `prebuilt-layout`
2026-04-10 14:37:55 -07:00
CREDO23
f556446d07 Fix test mocks for vision_llm kwarg 2026-04-10 18:20:49 +02:00
Anish Sarkar
bb5b90e5bd fix: update Azure Document Intelligence parser to use prebuilt-layout model 2026-04-10 20:40:54 +05:30
CREDO23
a95bf58c8f Make Vision LLM opt-in for uploads and connectors 2026-04-10 16:45:51 +02:00
Anish Sarkar
c6730c5551 feat: implement download functionality for different OS in SidebarUserProfile and DownloadButton components, enhance user experience with localized download messages 2026-04-10 19:53:13 +05:30
Anish Sarkar
0925f89b7c fix: guard display_name.split() against whitespace-only IndexError (recurseml feedback) 2026-04-10 15:29:56 +05:30
Anish Sarkar
8c9440998a chore: ran linting 2026-04-10 14:20:11 +05:30
Anish Sarkar
a5bf3af4f8 feat: migrate legacy memory tables to markdown format and drop old structures 2026-04-10 14:19:20 +05:30
Anish Sarkar
8d7b5bfe36 feat: enhance memory management protocols to enforce structured headings, utilize user first names in entries, and improve clarity in memory update rules 2026-04-10 04:51:56 +05:30
Anish Sarkar
b8e1c9801b feat: update memory extraction and management protocols to enforce structured bullet formats, utilize user first names, and enhance validation for team and user memory entries 2026-04-10 04:36:51 +05:30