Commit graph

272 commits

Author SHA1 Message Date
Anish Sarkar
3f4e1a7dfd refactor: remove frontend of scrape_webpage tool 2026-03-24 18:55:06 +05:30
Anish Sarkar
a009cae62a refactor: remove link_preview tool and associated components to streamline agent functionality 2026-03-24 17:15:29 +05:30
Anish Sarkar
6c507989d2 refactor: remove display_image tool and update related components to streamline image handling 2026-03-24 16:28:11 +05:30
Anish Sarkar
5c598e8588 Merge remote-tracking branch 'upstream/dev' into feat/human-in-the-loop 2026-03-22 15:45:45 +05:30
DESKTOP-RTLN3BA\$punk
eb8cfd296c feat: add public routes for video presentations and audio streaming 2026-03-21 23:29:23 -07:00
Anish Sarkar
2bc6a0c3bc chore: ran linting 2026-03-22 00:43:53 +05:30
Anish Sarkar
68f1a7c5ce refactor: deduplicate issue type names in JiraToolMetadataService 2026-03-21 21:02:52 +05:30
Anish Sarkar
de8841fb86 chore: ran linting 2026-03-21 13:20:13 +05:30
Anish Sarkar
e71eae26fc feat: initial files for jira and confluence HITL tool 2026-03-21 12:16:44 +05:30
Anish Sarkar
9a20db7fc4 feat: add created_by_id to document creation in various sync services 2026-03-21 11:41:59 +05:30
Anish Sarkar
744ad1fa79 feat: enforce single invocation for tool actions across various services
- Added a guideline to ensure that each tool (Gmail, Google Calendar, Google Drive, Linear, Notion) is called only once per user request.
- Updated documentation to clarify that the system will automatically select the most relevant match when multiple items share the same title or subject, enhancing user experience and preventing redundant calls.
2026-03-21 03:07:13 +05:30
Anish Sarkar
cb6b687933 feat: enhance Gmail draft update functionality
- Improved the update_gmail_draft tool to allow users to review and edit draft content before applying changes.
- Added logic to generate draft body content based on user requests and conversation context.
- Implemented fetching of existing draft body to facilitate user edits in the approval card.
- Updated UI components to support displaying and editing existing draft content, enhancing user experience.
2026-03-21 00:30:18 +05:30
Anish Sarkar
85462675a0 feat: add update functionality for Gmail drafts
- Introduced a new tool to update existing Gmail drafts, allowing users to modify draft content, recipients, and subject lines.
- Updated the Gmail tools registry to include the new update_gmail_draft tool.
- Enhanced the GmailKBSyncService to support draft ID handling during synchronization.
- Added UI components for the update draft functionality in the web application, improving user interaction with Gmail drafts.
2026-03-20 23:50:27 +05:30
Anish Sarkar
bd2d633546 feat: enhance Gmail account handling and expand chat tool capabilities
- Updated the GmailAccount class to extract email from the connector name when formatted with " - ".
- Added new tool actions for Gmail and Google Calendar, including creating drafts, sending emails, and managing calendar events, improving integration and user functionality.
2026-03-20 20:18:03 +05:30
Anish Sarkar
f4c0c8c945 feat: add initial logic for HITL for gmail and google calendar connectors 2026-03-20 18:47:06 +05:30
Anish Sarkar
8850fac722 feat: implement knowledge base synchronization for Linear and Notion issue creation
- Added `sync_after_create` methods in `LinearKBSyncService` and `NotionKBSyncService` to handle synchronization of newly created issues and pages with the knowledge base.
- Enhanced the `create_issue.py` and `create_page.py` tools to provide user feedback on the success of the knowledge base update, indicating whether the content has been synced or will be added in the next scheduled sync.
- Improved error handling during synchronization to log failures and manage duplicate document scenarios effectively.
2026-03-20 17:50:36 +05:30
Anish Sarkar
f4da05d0ea refactor: streamline authentication expiration handling in Notion tool metadata service 2026-03-20 16:29:50 +05:30
Anish Sarkar
10af5d3f3e feat: add authentication expiration handling for Notion tools
- Implemented checks for expired authentication in the Notion page creation and deletion tools, returning appropriate error messages for re-authentication.
- Updated the Notion tool metadata service to track account health and persist authentication status, improving error handling and user feedback during operations.
2026-03-20 16:12:18 +05:30
Anish Sarkar
3d6ff39bf4 feat: implement Google Drive knowledge base synchronization after file creation
- Added a new GoogleDriveKBSyncService to handle synchronization of newly created Google Drive files with the knowledge base.
- Enhanced the create_file.py tool to include feedback on the success of the knowledge base update, informing users if their file has been added or will be synced later.
- Updated the Google Drive tool metadata service to include parent folder information for improved file organization.
- Modified the UI components to support selection of parent folders during file creation, enhancing user experience and file management.
2026-03-20 15:41:08 +05:30
Anish Sarkar
75f0975674 feat: enhance Google Drive account authentication handling
- Added checks for expired authentication in Google Drive file creation and deletion tools, returning appropriate error messages for re-authentication.
- Updated the Google Drive tool metadata service to track account health and persist authentication status.
- Enhanced UI components to display authentication errors and differentiate between valid and expired accounts, improving user experience during file operations.
2026-03-20 12:34:30 +05:30
Anish Sarkar
d21593ee71 feat: unify handling of native and legacy document types for Google connectors
- Introduced a mapping of native Google document types to their legacy Composio equivalents, ensuring seamless search and indexing for both types.
- Updated relevant components to utilize the new mapping, enhancing the consistency of document type handling across the application.
- Improved search functionality to transparently include legacy types, maintaining accessibility for older documents until re-indexed.
2026-03-20 03:41:32 +05:30
Anish Sarkar
83d9c49a50 feat: enhance Composio connector handling and status management
- Updated the .env.example file to include a note about disabling "Mask Connected Account Secrets" for Google indexing.
- Implemented a wait_for_connection method in ComposioService to ensure connected accounts reach ACTIVE status after authentication.
- Added logic in composio_callback and composio_reauth_callback to wait for Composio to finish processing tokens, improving reliability in token handling.
- Enhanced logging to provide clearer warnings when connection timeouts occur, ensuring better visibility into potential issues.
2026-03-19 19:24:55 +05:30
Anish Sarkar
53e555f10c feat: implement Composio re-authentication endpoints and refresh logic
- Added new endpoints for re-authenticating Composio connectors, allowing users to refresh their authentication when expired.
- Introduced a method in ComposioService to handle the refresh of connected accounts.
- Updated the frontend to support re-authentication flows, including success notifications and redirection after re-authentication.
- Enhanced error handling for re-authentication processes to improve user experience.
2026-03-19 18:08:23 +05:30
Anish Sarkar
83152e8e7e refactor: unify all 3 google Composio and non-Composio connector types and pipelines keeping same credential adapters 2026-03-19 05:08:21 +05:30
Anish Sarkar
7ba5e9c662 feat: implement re-authentication flow for connectors and enhance auth expiration handling
- Added a mechanism to mark connectors as 'auth_expired' in the database, allowing the frontend to prompt users for re-authentication.
- Updated Linear and Notion connector routes to handle token refresh failures by invoking the new expiration flagging function.
- Enhanced UI components to display re-authentication options when a connector's authentication status is expired.
2026-03-19 01:20:48 +05:30
Anish Sarkar
df872e261e feat: implement re-authentication flow for Linear and Notion HITL connectors and improve their HITL flow and error states
- Added re-authentication endpoints for Linear and Notion connectors to handle expired authentication.
- Enhanced error handling in issue creation, deletion, and update tools to return appropriate authentication error messages.
- Updated UI components to display authentication status and guide users on re-authentication steps.
- Improved account health checks to ensure valid tokens are used for operations.
2026-03-18 14:10:11 +05:30
Anish Sarkar
ac0f2fa2eb chore: ran linting 2026-03-17 04:40:46 +05:30
Anish Sarkar
60d12b0a70 Merge remote-tracking branch 'upstream/dev' into feat/web-search 2026-03-16 19:34:38 +05:30
Rohan Verma
ee07ed8168
Merge pull request #883 from octo-patch/feat/complete-minimax-provider-support
feat: add MiniMax LLM provider support
2026-03-15 00:45:16 -07:00
DESKTOP-RTLN3BA\$punk
2b33dfe728 refactor: update safe_set_chunks function to be asynchronous and modify all connector and document processor files to use the new async implementation 2026-03-15 00:44:27 -07:00
Anish Sarkar
c1fe95939f refactor: remove legacy SearXNG API references and web search settings
- Deleted the migration script for adding web search columns to the database.
- Removed web search related fields from the SearchSpace model and schemas.
- Eliminated web search health check endpoint and associated service logic.
- Updated frontend components to remove web search settings management.
- Cleaned up references to SearXNG API in various modules, transitioning to a platform service approach.
2026-03-15 03:49:37 +05:30
Anish Sarkar
d9e312ddaa refactor: simplify SearXNG service configuration in Docker setup
- Consolidated volume mappings for SearXNG to use a single directory.
- Removed unnecessary port mappings and legacy data volume definitions.
- Updated web search service documentation to clarify Redis usage and circuit breaker implementation, eliminating Redis dependency for circuit breaker logic.
2026-03-14 20:34:34 +05:30
Anish Sarkar
d40c6bf743 feat: integrate SearXNG web search service with platform capabilities
- Added SearXNG service configuration to Docker setup, including environment variables and health checks.
- Introduced new settings management for web search in the frontend, allowing users to enable/disable and configure search engines and language preferences.
- Updated backend to support web search functionality, including database schema changes and service integration.
- Implemented health check endpoint for the web search service and integrated it into the application.
- Removed legacy SearXNG API connector references in favor of the new platform service approach.
2026-03-14 20:25:25 +05:30
PR Bot
760aa38225 feat: complete MiniMax LLM provider integration
Add full MiniMax provider support across the entire stack:

Backend:
- Add MINIMAX to LiteLLMProvider enum in db.py
- Add MINIMAX mapping to all provider_map dicts in llm_service.py,
  llm_router_service.py, and llm_config.py
- Add Alembic migration (rev 106) for PostgreSQL enum
- Add MiniMax M2.5 example in global_llm_config.example.yaml

Frontend:
- Add MiniMax to LLM_PROVIDERS enum with apiBase
- Add MiniMax-M2.5 and MiniMax-M2.5-highspeed to LLM_MODELS
- Add MINIMAX to Zod validation schema
- Add MiniMax SVG icon and wire up in provider-icons

Docs:
- Add MiniMax setup guide in chinese-llm-setup.md

MiniMax uses an OpenAI-compatible API (https://api.minimax.io/v1)
with models supporting up to 204K context window.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 07:27:47 +08:00
DESKTOP-RTLN3BA\$punk
eec4db4a3b feat: Enhance LLM configuration and routing with model profile attachment
- Added `_attach_model_profile` function to attach model context metadata to `ChatLiteLLM`.
- Updated `create_chat_litellm_from_config` and `create_chat_litellm_from_agent_config` to utilize the new profile attachment.
- Improved context profile caching in `llm_router_service.py` to include both minimum and maximum input tokens, along with token model names for better context management.
- Introduced new methods for token counting and context trimming based on model profiles.
2026-03-10 18:18:59 -07:00
CREDO23
89ed845dcd merge upstream/dev into improve-ux-connectors
Resolve 3 conflicts:
- connector-popup.tsx: keep Picker modal coordination (pickerOpen state)
- google-drive-config.tsx: keep Google Picker, discard upstream folder tree
- composio-drive-config.tsx: accept upstream collapsible folder tree UX
2026-03-10 23:40:04 +02:00
Rohan Verma
547077e5b9
Merge pull request #865 from CREDO23/sur-182-fix-ux-experience-for-composio-google-drive-connector
[Perf] Batch embedding, non-blocking search, chunks index & Google Drive UX fix
2026-03-10 12:52:16 -07:00
CREDO23
1e2c54eea6 add unified drive-picker-token endpoint and GOOGLE_PICKER_API_KEY config 2026-03-10 20:06:45 +02:00
DESKTOP-RTLN3BA\$punk
403097646d feat: implement batch unread counts for notifications to reduce API calls and improve performance 2026-03-10 01:26:37 -07:00
CREDO23
a49a4db6d6 perf: use asyncio.to_thread for embed_text in FastAPI paths 2026-03-09 16:33:24 +02:00
DESKTOP-RTLN3BA\$punk
40a091f8cc feat: enhance knowledge base search and document retrieval
- Introduced a mechanism to identify degenerate queries that lack meaningful search signals, improving search accuracy.
- Implemented a fallback method for browsing recent documents when queries are degenerate, ensuring relevant results are returned.
- Added limits on the number of chunks fetched per document to optimize performance and prevent excessive data loading.
- Updated the ConnectorService to allow for reusable query embeddings, enhancing efficiency in search operations.
- Enhanced LLM router service to support context window fallbacks, improving robustness during context window limitations.
2026-02-28 19:40:24 -08:00
DESKTOP-RTLN3BA\$punk
d959a6a6c8 feat: optimize document upload process and enhance memory management
- Increased maximum file upload limit from 10 to 50 to improve user experience.
- Implemented batch processing for document uploads to avoid proxy timeouts, splitting files into manageable chunks.
- Enhanced garbage collection in chat streaming functions to prevent memory leaks and improve performance.
- Added memory delta tracking in system snapshots for better monitoring of resource usage.
- Updated LLM router and service configurations to prevent unbounded internal accumulation and improve efficiency.
2026-02-28 17:22:34 -08:00
DESKTOP-RTLN3BA\$punk
f4b2ab0899 feat: enhance caching mechanisms to prevent memory leaks
- Improved in-memory rate limiting by evicting timestamps outside the current window and cleaning up empty keys.
- Updated LLM router service to cache context profiles and avoid redundant computations.
- Introduced cache eviction logic for MCP tools and sandbox instances to manage memory usage effectively.
- Added garbage collection triggers in chat streaming functions to reclaim resources promptly.
2026-02-27 17:56:00 -08:00
DESKTOP-RTLN3BA\$punk
0e723a5b8b feat: perf optimizations
- improved search_knowledgebase_tool
- Added new endpoint to batch-fetch comments for multiple messages, reducing the number of API calls.
- Introduced CommentBatchRequest and CommentBatchResponse schemas for handling batch requests and responses.
- Updated chat_comments_service to validate message existence and permissions before fetching comments.
- Enhanced frontend with useBatchCommentsPreload hook to optimize comment loading for assistant messages.
2026-02-27 17:19:25 -08:00
DESKTOP-RTLN3BA\$punk
664c43ca13 feat: add performance logging middleware and enhance performance tracking across services
- Introduced RequestPerfMiddleware to log request performance metrics, including slow request thresholds.
- Updated various services and retrievers to utilize the new performance logging utility for better tracking of execution times.
- Enhanced existing methods with detailed performance logs for operations such as embedding, searching, and indexing.
- Removed deprecated logging setup in stream_new_chat and replaced it with the new performance logger.
2026-02-27 16:32:30 -08:00
DESKTOP-RTLN3BA\$punk
1e4b8d3e89 feat: enhance document formatting and context management for LLM tools
- Introduced dynamic character budget calculation for document formatting based on model's context window.
- Updated `format_documents_for_context` to respect character limits and improve output quality.
- Added `max_input_tokens` parameter to various functions to facilitate context-aware processing.
- Enhanced error handling for context overflow in LLM router service.
2026-02-26 20:47:19 -08:00
DESKTOP-RTLN3BA\$punk
a4dc84d1ab feat: add should_summarize parameter to task dispatchers
- Introduced should_summarize parameter in TaskDispatcher and CeleryTaskDispatcher to control summary generation.
- Updated InlineTaskDispatcher to support the new parameter for document processing.
2026-02-26 19:12:37 -08:00
DESKTOP-RTLN3BA\$punk
6f4bf11a32 Merge branch 'dev' of https://github.com/MODSetter/SurfSense into dev 2026-02-26 18:25:05 -08:00
DESKTOP-RTLN3BA\$punk
e9892c8fe9 feat: added configable summary calculation and various improvements
- Replaced direct embedding calls with a utility function across various components to streamline embedding logic.
- Added enable_summary flag to several models and routes to control summary generation behavior.
2026-02-26 18:24:57 -08:00
Anish Sarkar
3393e435f9 feat: implement task dispatcher for document processing
- Introduced a TaskDispatcher abstraction to decouple the upload endpoint from Celery, allowing for easier testing with synchronous implementations.
- Updated the create_documents_file_upload function to utilize the new dispatcher for task management.
- Removed direct Celery task imports from the upload function, enhancing modularity.
- Added integration tests for document upload, including page limit enforcement and file size restrictions.
2026-02-26 23:55:47 +05:30