Commit graph

417 commits

Author SHA1 Message Date
Anish Sarkar
e8cf677b25 refactor: update title generation logic to improve user experience by generating titles in parallel with assistant responses 2026-03-09 01:28:53 +05:30
Anish Sarkar
6d00b0debf Merge remote-tracking branch 'upstream/dev' into refactor/upload-document-adapter-class 2026-03-01 22:35:17 +05:30
DESKTOP-RTLN3BA\$punk
ecb0a25cc8 feat: enhance memory management and session handling in database operations
- Introduced a shielded async session context manager to ensure safe session closure during cancellations.
- Updated various database operations to utilize the new shielded session, preventing orphaned connections.
- Added environment variables to optimize glibc memory management, improving overall application performance.
- Implemented a function to trim the native heap, allowing for better memory reclamation on Linux systems.
2026-02-28 23:59:28 -08:00
DESKTOP-RTLN3BA\$punk
dd3da2bc36 refactor: improve session management and cleanup in chat streaming
- Added proper session closure to prevent connection leaks during streaming.
- Implemented a fresh session for cleanup tasks to ensure data integrity.
- Enhanced error handling during session operations to improve robustness.
- Removed unnecessary session parameters from function signatures for clarity.
2026-02-28 23:17:11 -08:00
DESKTOP-RTLN3BA\$punk
d959a6a6c8 feat: optimize document upload process and enhance memory management
- Increased maximum file upload limit from 10 to 50 to improve user experience.
- Implemented batch processing for document uploads to avoid proxy timeouts, splitting files into manageable chunks.
- Enhanced garbage collection in chat streaming functions to prevent memory leaks and improve performance.
- Added memory delta tracking in system snapshots for better monitoring of resource usage.
- Updated LLM router and service configurations to prevent unbounded internal accumulation and improve efficiency.
2026-02-28 17:22:34 -08:00
DESKTOP-RTLN3BA\$punk
f4b2ab0899 feat: enhance caching mechanisms to prevent memory leaks
- Improved in-memory rate limiting by evicting timestamps outside the current window and cleaning up empty keys.
- Updated LLM router service to cache context profiles and avoid redundant computations.
- Introduced cache eviction logic for MCP tools and sandbox instances to manage memory usage effectively.
- Added garbage collection triggers in chat streaming functions to reclaim resources promptly.
2026-02-27 17:56:00 -08:00
DESKTOP-RTLN3BA\$punk
664c43ca13 feat: add performance logging middleware and enhance performance tracking across services
- Introduced RequestPerfMiddleware to log request performance metrics, including slow request thresholds.
- Updated various services and retrievers to utilize the new performance logging utility for better tracking of execution times.
- Enhanced existing methods with detailed performance logs for operations such as embedding, searching, and indexing.
- Removed deprecated logging setup in stream_new_chat and replaced it with the new performance logger.
2026-02-27 16:32:30 -08:00
Anish Sarkar
23a98d802c refactor: implement UploadDocumentAdapter for file indexing and reindexing 2026-02-28 01:38:32 +05:30
DESKTOP-RTLN3BA\$punk
e9892c8fe9 feat: added configable summary calculation and various improvements
- Replaced direct embedding calls with a utility function across various components to streamline embedding logic.
- Added enable_summary flag to several models and routes to control summary generation behavior.
2026-02-26 18:24:57 -08:00
DESKTOP-RTLN3BA\$punk
23f553ef84 Merge branch 'dev' of https://github.com/MODSetter/SurfSense into dev 2026-02-26 13:01:24 -08:00
DESKTOP-RTLN3BA\$punk
aabc24f82c feat: enhance performance logging and caching in various components
- Introduced slow callback logging in FastAPI to identify blocking calls.
- Added performance logging for agent creation and tool loading processes.
- Implemented caching for MCP tools to reduce redundant server calls.
- Enhanced sandbox management with in-process caching for improved efficiency.
- Refactored several functions for better readability and performance tracking.
- Updated tests to ensure proper functionality of new features and optimizations.
2026-02-26 13:00:31 -08:00
Anish Sarkar
9ccee054a5 chore: ran linting 2026-02-26 03:05:20 +05:30
Anish Sarkar
f59a70f7a5 Merge remote-tracking branch 'upstream/dev' into feat/document-test 2026-02-26 02:22:10 +05:30
Rohan Verma
2e99f1e853
Merge pull request #837 from CREDO23/test-document-creation
[Refactor] Core document creation / indexing pipeline with unit and e2e tests
2026-02-25 12:27:41 -08:00
DESKTOP-RTLN3BA\$punk
54dd6f904c feat: implement local file persistence for Daytona sandboxes
- Added functionality to persist sandbox files locally before deletion.
- Introduced methods for retrieving and deleting locally stored sandbox files.
- Updated routes to handle local file downloads and background deletion of sandboxes.
- Enhanced streaming tasks to support sandbox file management.
2026-02-25 12:25:59 -08:00
Anish Sarkar
380c1c3877 fix: Refactor document ID usage in file processing to improve clarity 2026-02-26 01:28:09 +05:30
CREDO23
ab31bba3ad merge upstream/dev 2026-02-25 20:52:05 +02:00
CREDO23
4293910e8e plug file upload into indexing pipeline adapter and add integration tests 2026-02-25 20:20:52 +02:00
DESKTOP-RTLN3BA\$punk
d198c8b89d chore; linting 2026-02-25 01:50:28 -08:00
DESKTOP-RTLN3BA\$punk
d570cae3c6 feat: added file handling for daytona sandboxes
- Added _TimeoutAwareSandbox class to handle per-command timeouts in DaytonaSandbox.
- Updated _find_or_create function to manage sandbox states and restart stopped/archived sandboxes.
- Enhanced get_or_create_sandbox to return the new sandbox class.
- Introduced file download functionality in the frontend, allowing users to download generated files from the sandbox.
- Updated system prompt to include guidelines for sharing generated files.
2026-02-25 01:36:30 -08:00
DESKTOP-RTLN3BA\$punk
2869d3a9c8 Merge commit 'd0ee8b12b6' into codesandbox_integration 2026-02-24 16:38:09 -08:00
DESKTOP-RTLN3BA\$punk
c007f0e056 feat: unut codesandbox integration 2026-02-24 16:36:11 -08:00
Rohan Verma
d0ee8b12b6
Merge pull request #834 from CREDO23/google-drive-hitl
[Feature] Add human in the loop for google drive sensitive actions (create:docs,sheets & delete)
2026-02-24 16:34:25 -08:00
CREDO23
e89b68520a rename trash_google_drive_file to delete_google_drive_file 2026-02-24 13:18:39 +02:00
CREDO23
33bcf19b62 stream full output for google drive HITL tools 2026-02-23 16:13:18 +02:00
DESKTOP-RTLN3BA\$punk
0fb6caa01f fix: improve error handling for AI responding state clearance in chat streaming 2026-02-21 20:05:17 -08:00
DESKTOP-RTLN3BA\$punk
08c75127f1 feat: update migration to skip blocknode conversion and update transaction handling in chat routes 2026-02-20 22:43:25 -08:00
Rohan Verma
f15833fec6
Merge pull request #826 from AnishSarkar22/fix/report-artifact
fix: improve report artifact & enhance revision handling
2026-02-19 19:10:21 -08:00
Rohan Verma
9aef655566
Merge pull request #825 from CREDO23/sur-169-feat-implement-human-in-the-loop-for-linear-sensitive
[Feat] Add human in the loop for linear sensitive actions
2026-02-19 19:09:50 -08:00
Anish Sarkar
e059ee4193 chore: ran linting 2026-02-19 19:34:25 +05:30
Anish Sarkar
4afdfb580d feat: enhance chat streaming by incorporating recent report context for improved LLM decision-making 2026-02-19 19:28:03 +05:30
Anish Sarkar
69dfe943b7 refactor: enhance report generation tool with improved source strategies and revision handling 2026-02-19 15:33:32 +05:30
Anish Sarkar
f3ec48fb00 feat: enhance chat event streaming by tracking active tool depth to prevent inner-tool LLM token leakage 2026-02-19 13:11:18 +05:30
CREDO23
1d6c1a2287 feat(linear): disable linear tools when no connector, handle in streaming 2026-02-18 18:49:03 +02:00
CREDO23
7d1bd1fab4 Implement KB sync after Notion page updates with block ID verification
- Add NotionKBSyncService for immediate KB updates after page changes
- Implement block ID verification to ensure content freshness
- Refactor duplicate block processing logic to shared utils
- Add user-friendly status messages
- Include debug logging for troubleshooting
2026-02-17 20:30:12 +02:00
Anish Sarkar
8b497da130 feat: add source_markdown column to documents and implement migration logic for existing records using a pure-Python BlockNote JSON to Markdown converter 2026-02-17 11:34:11 +05:30
DESKTOP-RTLN3BA\$punk
1849b451a5 feat: update Celery worker configuration and improve connector deletion process
- Added support for multiple queues in Celery worker configuration.
- Modified connector deletion to handle documents inline instead of using a background task.
- Updated response messages for document creation and connector deletion to reflect new processing status.
- Removed the obsolete connector deletion Celery task file.
2026-02-16 00:07:23 -08:00
Anish Sarkar
793c19556d chore: ran linting 2026-02-16 03:54:19 +05:30
Anish Sarkar
bff52c71ef feat: improve report generation functionality to chat streaming service 2026-02-16 03:53:33 +05:30
CREDO23
79d1f7d603 pass full notion tool output in stream 2026-02-11 17:38:42 +02:00
CREDO23
8f81c9859d deduplicate backend streaming loop into shared _stream_agent_events helper 2026-02-10 19:47:10 +02:00
CREDO23
39ee4742d2 add stream_resume_chat and POST /threads/{id}/resume endpoint 2026-02-10 15:43:52 +02:00
CREDO23
9751918e41 add interrupt detection, SSE event, and resume schemas 2026-02-10 15:35:42 +02:00
DESKTOP-RTLN3BA\$punk
db652116d6 chore: linting 2026-02-09 16:49:11 -08:00
DESKTOP-RTLN3BA\$punk
c979609041 feat: simplified document upload handling
- Introduced a new endpoint for batch document status retrieval, allowing users to check the status of multiple documents in a search space.
- Enhanced the document upload process to return duplicate document IDs and improved response structure.
- Updated schemas to include new response models for document status.
- Removed unused attachment processing code from chat routes and UI components to streamline functionality.
2026-02-09 16:46:54 -08:00
Rohan Verma
3f0c9c35f7
Merge pull request #799 from CREDO23/sur-152-impr-split-private-and-shared-memory
[Feat] Split private vs shared chat memory and add team prompt/attribution
2026-02-09 15:03:54 -08:00
Rohan Verma
26fd61fcbb
Merge pull request #796 from AnishSarkar22/feat/sur-149-batch-index
impr: batch index for messaging connectors & some fixes
2026-02-09 15:00:16 -08:00
DESKTOP-RTLN3BA\$punk
2b75c4e6d3 chore: linting 2026-02-09 14:14:46 -08:00
DESKTOP-RTLN3BA\$punk
17b7348f61 feat: fixed and improved search and background task management. 2026-02-09 14:03:56 -08:00
Anish Sarkar
20ab128b05 feat: implement batch indexing for Microsoft Teams messages to improve efficiency and conversational context 2026-02-09 14:31:22 +05:30