Commit graph

466 commits

Author SHA1 Message Date
Anish Sarkar
5c598e8588 Merge remote-tracking branch 'upstream/dev' into feat/human-in-the-loop 2026-03-22 15:45:45 +05:30
DESKTOP-RTLN3BA\$punk
eb8cfd296c feat: add public routes for video presentations and audio streaming 2026-03-21 23:29:23 -07:00
DESKTOP-RTLN3BA\$punk
b28f135a96 feat: init video presentation agent 2026-03-21 22:13:41 -07:00
Anish Sarkar
e37e6d2d18 chore: ran linting 2026-03-21 13:21:19 +05:30
Anish Sarkar
de8841fb86 chore: ran linting 2026-03-21 13:20:13 +05:30
Anish Sarkar
e71eae26fc feat: initial files for jira and confluence HITL tool 2026-03-21 12:16:44 +05:30
Anish Sarkar
283b4194cc feat: enhance permission handling and user feedback for Gmail and Google Calendar tools
- Implemented logic to persist authentication expiration status for connectors when insufficient permissions are detected, improving error handling and user experience.
- Updated messages to guide users to re-authenticate in connector settings for Gmail, Google Calendar, and Google Drive tools.
- Added InsufficientPermissionsResult type and corresponding UI components to display permission-related messages consistently across Gmail and Google Calendar tools.
2026-03-20 19:36:00 +05:30
Anish Sarkar
510f9150cb feat: improve error handling for Google Drive authentication
- Enhanced error handling in both Composio and Google Drive folder listing functions to check for expired authentication and mark connectors as 'auth_expired' when necessary.
- Added logging for failed persistence of authentication status and raised appropriate HTTP exceptions to prompt users for re-authentication.
- Streamlined the error checking process to include various authentication failure scenarios, improving overall robustness of the integration.
2026-03-20 14:48:31 +05:30
Anish Sarkar
df0d9bb2b5 feat: improve Composio Drive folder listing and authentication handling
- Simplified the folder and file listing process in the Composio Drive integration by utilizing the GoogleDriveClient for fetching contents.
- Enhanced error handling for authentication issues, marking connectors as 'auth_expired' when necessary and prompting users to re-authenticate.
- Updated UI components to display authentication status and provide re-authentication options, improving user experience during folder selection.
2026-03-20 14:48:19 +05:30
Anish Sarkar
a27c10a5f5 feat: update Google Drive connector UI and authentication handling 2026-03-20 13:39:28 +05:30
Anish Sarkar
8e7cda31c5 feat: update Google indexing functions to track skipped messages
- Modified the indexing functions for Google Calendar and Gmail to return the count of skipped messages alongside indexed messages, enhancing performance tracking.
- Updated related tests to accommodate the new return values, ensuring comprehensive coverage of the indexing process.
- Improved error handling to maintain consistency in returned values across different indexing functions.
2026-03-19 20:56:40 +05:30
Anish Sarkar
e9485ab2df feat: update Google Drive indexing to include skipped file tracking 2026-03-19 20:27:50 +05:30
Anish Sarkar
eac4cb6075 feat: enhance Google Drive indexing to track skipped files
- Updated the indexing function to return the count of skipped files alongside indexed files, improving tracking of indexing performance.
- Added logic to accumulate skipped file counts during the indexing process, providing better insights into potential issues.
- Enhanced notification updates to include skipped file counts, ensuring comprehensive progress reporting for users.
2026-03-19 20:27:36 +05:30
Anish Sarkar
83d9c49a50 feat: enhance Composio connector handling and status management
- Updated the .env.example file to include a note about disabling "Mask Connected Account Secrets" for Google indexing.
- Implemented a wait_for_connection method in ComposioService to ensure connected accounts reach ACTIVE status after authentication.
- Added logic in composio_callback and composio_reauth_callback to wait for Composio to finish processing tokens, improving reliability in token handling.
- Enhanced logging to provide clearer warnings when connection timeouts occur, ensuring better visibility into potential issues.
2026-03-19 19:24:55 +05:30
Anish Sarkar
2390bd7d26 feat: enhance Google Drive authentication error handling
- Improved error handling for Google Drive indexing and listing operations to manage authentication failures more effectively.
- Added logic to mark connectors as 'auth_expired' when a 401 error or invalid credentials are detected, prompting users to re-authenticate.
- Updated error messages to provide clearer guidance on authentication issues, ensuring a better user experience.
2026-03-19 18:24:41 +05:30
Anish Sarkar
53e555f10c feat: implement Composio re-authentication endpoints and refresh logic
- Added new endpoints for re-authenticating Composio connectors, allowing users to refresh their authentication when expired.
- Introduced a method in ComposioService to handle the refresh of connected accounts.
- Updated the frontend to support re-authentication flows, including success notifications and redirection after re-authentication.
- Enhanced error handling for re-authentication processes to improve user experience.
2026-03-19 18:08:23 +05:30
Anish Sarkar
c9deae940c feat: implement re-authentication flows for Google Drive, Gmail, and Calendar connectors
- Added re-authentication endpoints for Google Drive, Gmail, and Calendar connectors to handle expired authentication.
- Enhanced the UI to prompt users for re-authentication when their credentials are expired.
- Updated backend logic to mark connectors as 'auth_expired' and manage re-authentication requests effectively.
- Improved error handling for authentication failures across Google connectors.
2026-03-19 17:51:59 +05:30
Anish Sarkar
83152e8e7e refactor: unify all 3 google Composio and non-Composio connector types and pipelines keeping same credential adapters 2026-03-19 05:08:21 +05:30
Anish Sarkar
7ba5e9c662 feat: implement re-authentication flow for connectors and enhance auth expiration handling
- Added a mechanism to mark connectors as 'auth_expired' in the database, allowing the frontend to prompt users for re-authentication.
- Updated Linear and Notion connector routes to handle token refresh failures by invoking the new expiration flagging function.
- Enhanced UI components to display re-authentication options when a connector's authentication status is expired.
2026-03-19 01:20:48 +05:30
Anish Sarkar
8baba0693d feat: ensure unique connector names for MCP connectors 2026-03-18 16:09:35 +05:30
Anish Sarkar
c343368456 refactor: remove query params for connector popup and improve connector callback URLs 2026-03-18 16:08:30 +05:30
Anish Sarkar
df872e261e feat: implement re-authentication flow for Linear and Notion HITL connectors and improve their HITL flow and error states
- Added re-authentication endpoints for Linear and Notion connectors to handle expired authentication.
- Enhanced error handling in issue creation, deletion, and update tools to return appropriate authentication error messages.
- Updated UI components to display authentication status and guide users on re-authentication steps.
- Improved account health checks to ensure valid tokens are used for operations.
2026-03-18 14:10:11 +05:30
Anish Sarkar
c1fe95939f refactor: remove legacy SearXNG API references and web search settings
- Deleted the migration script for adding web search columns to the database.
- Removed web search related fields from the SearchSpace model and schemas.
- Eliminated web search health check endpoint and associated service logic.
- Updated frontend components to remove web search settings management.
- Cleaned up references to SearXNG API in various modules, transitioning to a platform service approach.
2026-03-15 03:49:37 +05:30
Anish Sarkar
d40c6bf743 feat: integrate SearXNG web search service with platform capabilities
- Added SearXNG service configuration to Docker setup, including environment variables and health checks.
- Introduced new settings management for web search in the frontend, allowing users to enable/disable and configure search engines and language preferences.
- Updated backend to support web search functionality, including database schema changes and service integration.
- Implemented health check endpoint for the web search service and integrated it into the application.
- Removed legacy SearXNG API connector references in favor of the new platform service approach.
2026-03-14 20:25:25 +05:30
DESKTOP-RTLN3BA\$punk
d61e29e74b feat: fixed connectors dialog navigation, Implement bulk document deletion and improve search space filtering
- Added bulk delete functionality for documents in DocumentsTableShell and DocumentsSidebar.
- Enhanced search space retrieval to exclude spaces marked for deletion in read_search_spaces.
- Updated connector dialog to synchronize URL parameters when opened externally.
- Improved layout behavior to handle search space deletion and redirection more effectively.
2026-03-11 15:09:10 -07:00
DESKTOP-RTLN3BA\$punk
d8a05ae4d5 feat: refactor agent tools management and add UI integration
- Added endpoint to list agent tools with metadata, excluding hidden tools.
- Updated NewChatRequest and RegenerateRequest schemas to include disabled tools.
- Integrated disabled tools management in the NewChatPage and Composer components.
- Improved tool instructions and visibility in the system prompt.
- Refactored tool registration to support hidden tools and default enabled states.
- Enhanced document chunk creation to handle strict zip behavior.
- Cleaned up imports and formatting across various files for consistency.
2026-03-10 17:36:26 -07:00
CREDO23
cf8f70da2b fix auth bypass on picker endpoint, async safety, and picker error handling
- Add check_permission to drive-picker-token endpoint (IDOR fix)
- Use get_composio_service singleton + asyncio.to_thread to avoid blocking the event loop
- Sanitize error detail in 500 response to prevent internal info leakage
- Dispose picker on unmount to prevent orphaned overlay
- Surface error state on Google Picker Action.ERROR instead of silently closing
2026-03-10 23:21:35 +02:00
CREDO23
1e2c54eea6 add unified drive-picker-token endpoint and GOOGLE_PICKER_API_KEY config 2026-03-10 20:06:45 +02:00
DESKTOP-RTLN3BA\$punk
403097646d feat: implement batch unread counts for notifications to reduce API calls and improve performance 2026-03-10 01:26:37 -07:00
DESKTOP-RTLN3BA\$punk
45cc527d02 feat: update report generation and export capabilities to support multiple formats (PDF, DOCX, HTML, LaTeX, EPUB, ODT, plain text) across documentation and backend 2026-03-09 18:41:21 -07:00
DESKTOP-RTLN3BA\$punk
11b84a3fb4 feat: expand report export functionality to support multiple formats (PDF, DOCX, HTML, EPUB, etc.) and enhance UI for format selection 2026-03-09 18:31:14 -07:00
DESKTOP-RTLN3BA\$punk
c6fc4edbc2 feat: add YouTube video and playlist support in document collection with enhanced URL handling 2026-03-09 16:07:54 -07:00
Anish Sarkar
a11c95e30f feat: add last_login column to user table and update user login tracking 2026-03-08 18:24:29 +05:30
Anish Sarkar
f8b0e946ce chore: ran linting 2026-03-07 04:46:48 +05:30
Anish Sarkar
1a688c7161 feat: enhance notifications system by introducing category-based filtering for comments and status, improving user experience in the inbox and API interactions 2026-03-06 19:35:35 +05:30
Anish Sarkar
bd783cc2d0 feat: add filtering options for notifications by 'unread' and 'errors', enhancing user experience in the notifications list 2026-03-06 18:32:28 +05:30
Anish Sarkar
d03f938fcd feat: implement source type filtering in notifications API and UI, enhancing user experience by allowing users to filter notifications by connector and document types in the status tab 2026-03-06 17:25:07 +05:30
Anish Sarkar
dc6c18b3f6 refactor: add sorting functionality to document retrieval and enhance DocumentsSidebar for improved search and pagination handling 2026-03-06 11:22:33 +05:30
DESKTOP-RTLN3BA\$punk
ecb0a25cc8 feat: enhance memory management and session handling in database operations
- Introduced a shielded async session context manager to ensure safe session closure during cancellations.
- Updated various database operations to utilize the new shielded session, preventing orphaned connections.
- Added environment variables to optimize glibc memory management, improving overall application performance.
- Implemented a function to trim the native heap, allowing for better memory reclamation on Linux systems.
2026-02-28 23:59:28 -08:00
DESKTOP-RTLN3BA\$punk
dd3da2bc36 refactor: improve session management and cleanup in chat streaming
- Added proper session closure to prevent connection leaks during streaming.
- Implemented a fresh session for cleanup tasks to ensure data integrity.
- Enhanced error handling during session operations to improve robustness.
- Removed unnecessary session parameters from function signatures for clarity.
2026-02-28 23:17:11 -08:00
DESKTOP-RTLN3BA\$punk
d959a6a6c8 feat: optimize document upload process and enhance memory management
- Increased maximum file upload limit from 10 to 50 to improve user experience.
- Implemented batch processing for document uploads to avoid proxy timeouts, splitting files into manageable chunks.
- Enhanced garbage collection in chat streaming functions to prevent memory leaks and improve performance.
- Added memory delta tracking in system snapshots for better monitoring of resource usage.
- Updated LLM router and service configurations to prevent unbounded internal accumulation and improve efficiency.
2026-02-28 17:22:34 -08:00
DESKTOP-RTLN3BA\$punk
0e723a5b8b feat: perf optimizations
- improved search_knowledgebase_tool
- Added new endpoint to batch-fetch comments for multiple messages, reducing the number of API calls.
- Introduced CommentBatchRequest and CommentBatchResponse schemas for handling batch requests and responses.
- Updated chat_comments_service to validate message existence and permissions before fetching comments.
- Enhanced frontend with useBatchCommentsPreload hook to optimize comment loading for assistant messages.
2026-02-27 17:19:25 -08:00
DESKTOP-RTLN3BA\$punk
6f4bf11a32 Merge branch 'dev' of https://github.com/MODSetter/SurfSense into dev 2026-02-26 18:25:05 -08:00
DESKTOP-RTLN3BA\$punk
e9892c8fe9 feat: added configable summary calculation and various improvements
- Replaced direct embedding calls with a utility function across various components to streamline embedding logic.
- Added enable_summary flag to several models and routes to control summary generation behavior.
2026-02-26 18:24:57 -08:00
Rohan Verma
2f08dc9cf4
Merge pull request #839 from AnishSarkar22/feat/document-test
fix: enhanced document upload, page limit, upload limit tests
2026-02-26 13:48:45 -08:00
DESKTOP-RTLN3BA\$punk
23f553ef84 Merge branch 'dev' of https://github.com/MODSetter/SurfSense into dev 2026-02-26 13:01:24 -08:00
DESKTOP-RTLN3BA\$punk
aabc24f82c feat: enhance performance logging and caching in various components
- Introduced slow callback logging in FastAPI to identify blocking calls.
- Added performance logging for agent creation and tool loading processes.
- Implemented caching for MCP tools to reduce redundant server calls.
- Enhanced sandbox management with in-process caching for improved efficiency.
- Refactored several functions for better readability and performance tracking.
- Updated tests to ensure proper functionality of new features and optimizations.
2026-02-26 13:00:31 -08:00
Anish Sarkar
3393e435f9 feat: implement task dispatcher for document processing
- Introduced a TaskDispatcher abstraction to decouple the upload endpoint from Celery, allowing for easier testing with synchronous implementations.
- Updated the create_documents_file_upload function to utilize the new dispatcher for task management.
- Removed direct Celery task imports from the upload function, enhancing modularity.
- Added integration tests for document upload, including page limit enforcement and file size restrictions.
2026-02-26 23:55:47 +05:30
Anish Sarkar
9ccee054a5 chore: ran linting 2026-02-26 03:05:20 +05:30
Anish Sarkar
f59a70f7a5 Merge remote-tracking branch 'upstream/dev' into feat/document-test 2026-02-26 02:22:10 +05:30