Commit graph

1141 commits

Author SHA1 Message Date
Anish Sarkar
8e7cda31c5 feat: update Google indexing functions to track skipped messages
- Modified the indexing functions for Google Calendar and Gmail to return the count of skipped messages alongside indexed messages, enhancing performance tracking.
- Updated related tests to accommodate the new return values, ensuring comprehensive coverage of the indexing process.
- Improved error handling to maintain consistency in returned values across different indexing functions.
2026-03-19 20:56:40 +05:30
Anish Sarkar
e9485ab2df feat: update Google Drive indexing to include skipped file tracking 2026-03-19 20:27:50 +05:30
Anish Sarkar
eac4cb6075 feat: enhance Google Drive indexing to track skipped files
- Updated the indexing function to return the count of skipped files alongside indexed files, improving tracking of indexing performance.
- Added logic to accumulate skipped file counts during the indexing process, providing better insights into potential issues.
- Enhanced notification updates to include skipped file counts, ensuring comprehensive progress reporting for users.
2026-03-19 20:27:36 +05:30
Anish Sarkar
83d9c49a50 feat: enhance Composio connector handling and status management
- Updated the .env.example file to include a note about disabling "Mask Connected Account Secrets" for Google indexing.
- Implemented a wait_for_connection method in ComposioService to ensure connected accounts reach ACTIVE status after authentication.
- Added logic in composio_callback and composio_reauth_callback to wait for Composio to finish processing tokens, improving reliability in token handling.
- Enhanced logging to provide clearer warnings when connection timeouts occur, ensuring better visibility into potential issues.
2026-03-19 19:24:55 +05:30
Anish Sarkar
2390bd7d26 feat: enhance Google Drive authentication error handling
- Improved error handling for Google Drive indexing and listing operations to manage authentication failures more effectively.
- Added logic to mark connectors as 'auth_expired' when a 401 error or invalid credentials are detected, prompting users to re-authenticate.
- Updated error messages to provide clearer guidance on authentication issues, ensuring a better user experience.
2026-03-19 18:24:41 +05:30
Anish Sarkar
53e555f10c feat: implement Composio re-authentication endpoints and refresh logic
- Added new endpoints for re-authenticating Composio connectors, allowing users to refresh their authentication when expired.
- Introduced a method in ComposioService to handle the refresh of connected accounts.
- Updated the frontend to support re-authentication flows, including success notifications and redirection after re-authentication.
- Enhanced error handling for re-authentication processes to improve user experience.
2026-03-19 18:08:23 +05:30
Anish Sarkar
90481b9462 feat: Added logic to encrypt the access token, refresh token, and client secret if the configuration specifies token encryption for native Gmail connector 2026-03-19 17:57:51 +05:30
Anish Sarkar
c9deae940c feat: implement re-authentication flows for Google Drive, Gmail, and Calendar connectors
- Added re-authentication endpoints for Google Drive, Gmail, and Calendar connectors to handle expired authentication.
- Enhanced the UI to prompt users for re-authentication when their credentials are expired.
- Updated backend logic to mark connectors as 'auth_expired' and manage re-authentication requests effectively.
- Improved error handling for authentication failures across Google connectors.
2026-03-19 17:51:59 +05:30
Anish Sarkar
83152e8e7e refactor: unify all 3 google Composio and non-Composio connector types and pipelines keeping same credential adapters 2026-03-19 05:08:21 +05:30
Anish Sarkar
7ba5e9c662 feat: implement re-authentication flow for connectors and enhance auth expiration handling
- Added a mechanism to mark connectors as 'auth_expired' in the database, allowing the frontend to prompt users for re-authentication.
- Updated Linear and Notion connector routes to handle token refresh failures by invoking the new expiration flagging function.
- Enhanced UI components to display re-authentication options when a connector's authentication status is expired.
2026-03-19 01:20:48 +05:30
Anish Sarkar
8baba0693d feat: ensure unique connector names for MCP connectors 2026-03-18 16:09:35 +05:30
Anish Sarkar
c343368456 refactor: remove query params for connector popup and improve connector callback URLs 2026-03-18 16:08:30 +05:30
Anish Sarkar
df872e261e feat: implement re-authentication flow for Linear and Notion HITL connectors and improve their HITL flow and error states
- Added re-authentication endpoints for Linear and Notion connectors to handle expired authentication.
- Enhanced error handling in issue creation, deletion, and update tools to return appropriate authentication error messages.
- Updated UI components to display authentication status and guide users on re-authentication steps.
- Improved account health checks to ensure valid tokens are used for operations.
2026-03-18 14:10:11 +05:30
Anish Sarkar
804106af11 refactor: clarify usage instructions for Google Drive file HITL creation tool 2026-03-18 03:36:02 +05:30
Anish Sarkar
0b8bee0076 feat: integrate notion-markdown for markdown conversion in NotionHistoryConnector
- Added notion-markdown dependency to pyproject.toml.
- Refactored _markdown_to_blocks method to utilize notion-markdown for converting markdown content to Notion blocks.
- Updated create-notion-page component to replace Loader2Icon with Spinner for improved loading indication.
2026-03-17 23:42:26 +05:30
Anish Sarkar
20c444f83c refactor: edit HITL prompts to be more expressive 2026-03-17 21:16:37 +05:30
Anish Sarkar
4474b73f29 feat: remove WIP status HITL notion, linear and google drive tools 2026-03-17 20:50:14 +05:30
Anish Sarkar
ac0f2fa2eb chore: ran linting 2026-03-17 04:40:46 +05:30
Anish Sarkar
993c8539e8 feat: update UI components to utilize new main panel styling
- Replaced background styles with `bg-main-panel` in various components for consistent theming.
- Enhanced the `Header`, `LayoutShell`, and `Thread` components to improve visual coherence.
- Adjusted tool management UI to reflect new design standards, ensuring a unified user experience.
2026-03-17 01:09:15 +05:30
Anish Sarkar
60d12b0a70 Merge remote-tracking branch 'upstream/dev' into feat/web-search 2026-03-16 19:34:38 +05:30
Tim Ren
0269900c60 fix: use asyncio.to_thread for embedding calls in search endpoints
Wrap synchronous embedding_model.embed() calls with asyncio.to_thread()
in both vector_search and hybrid_search methods. This prevents blocking
the asyncio event loop during embedding computation, improving server
responsiveness under concurrent load.

Fixes #794

Signed-off-by: Tim Ren <137012659+xr843@users.noreply.github.com>
2026-03-15 16:21:19 +08:00
Rohan Verma
ee07ed8168
Merge pull request #883 from octo-patch/feat/complete-minimax-provider-support
feat: add MiniMax LLM provider support
2026-03-15 00:45:16 -07:00
DESKTOP-RTLN3BA\$punk
2b33dfe728 refactor: update safe_set_chunks function to be asynchronous and modify all connector and document processor files to use the new async implementation 2026-03-15 00:44:27 -07:00
Anish Sarkar
b22fe012d5 feat: enhance search_web functionality and clarify connector usage
- modify the `web_search` tool that integrates multiple live search connectors (Tavily, Linkup, Baidu) alongside the SearXNG platform for real-time information retrieval.
- Updated comments to reflect the separation of local knowledge base searches from web searches.
- Improved handling of connector types and search logic to ensure accurate routing of queries to the appropriate tools.
- Refactored existing code to streamline the search process and enhance performance.
2026-03-15 04:45:39 +05:30
Anish Sarkar
c1fe95939f refactor: remove legacy SearXNG API references and web search settings
- Deleted the migration script for adding web search columns to the database.
- Removed web search related fields from the SearchSpace model and schemas.
- Eliminated web search health check endpoint and associated service logic.
- Updated frontend components to remove web search settings management.
- Cleaned up references to SearXNG API in various modules, transitioning to a platform service approach.
2026-03-15 03:49:37 +05:30
Anish Sarkar
d9e312ddaa refactor: simplify SearXNG service configuration in Docker setup
- Consolidated volume mappings for SearXNG to use a single directory.
- Removed unnecessary port mappings and legacy data volume definitions.
- Updated web search service documentation to clarify Redis usage and circuit breaker implementation, eliminating Redis dependency for circuit breaker logic.
2026-03-14 20:34:34 +05:30
Anish Sarkar
d40c6bf743 feat: integrate SearXNG web search service with platform capabilities
- Added SearXNG service configuration to Docker setup, including environment variables and health checks.
- Introduced new settings management for web search in the frontend, allowing users to enable/disable and configure search engines and language preferences.
- Updated backend to support web search functionality, including database schema changes and service integration.
- Implemented health check endpoint for the web search service and integrated it into the application.
- Removed legacy SearXNG API connector references in favor of the new platform service approach.
2026-03-14 20:25:25 +05:30
PR Bot
760aa38225 feat: complete MiniMax LLM provider integration
Add full MiniMax provider support across the entire stack:

Backend:
- Add MINIMAX to LiteLLMProvider enum in db.py
- Add MINIMAX mapping to all provider_map dicts in llm_service.py,
  llm_router_service.py, and llm_config.py
- Add Alembic migration (rev 106) for PostgreSQL enum
- Add MiniMax M2.5 example in global_llm_config.example.yaml

Frontend:
- Add MiniMax to LLM_PROVIDERS enum with apiBase
- Add MiniMax-M2.5 and MiniMax-M2.5-highspeed to LLM_MODELS
- Add MINIMAX to Zod validation schema
- Add MiniMax SVG icon and wire up in provider-icons

Docs:
- Add MiniMax setup guide in chinese-llm-setup.md

MiniMax uses an OpenAI-compatible API (https://api.minimax.io/v1)
with models supporting up to 204K context window.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 07:27:47 +08:00
DESKTOP-RTLN3BA\$punk
d61e29e74b feat: fixed connectors dialog navigation, Implement bulk document deletion and improve search space filtering
- Added bulk delete functionality for documents in DocumentsTableShell and DocumentsSidebar.
- Enhanced search space retrieval to exclude spaces marked for deletion in read_search_spaces.
- Updated connector dialog to synchronize URL parameters when opened externally.
- Improved layout behavior to handle search space deletion and redirection more effectively.
2026-03-11 15:09:10 -07:00
DESKTOP-RTLN3BA\$punk
eec4db4a3b feat: Enhance LLM configuration and routing with model profile attachment
- Added `_attach_model_profile` function to attach model context metadata to `ChatLiteLLM`.
- Updated `create_chat_litellm_from_config` and `create_chat_litellm_from_agent_config` to utilize the new profile attachment.
- Improved context profile caching in `llm_router_service.py` to include both minimum and maximum input tokens, along with token model names for better context management.
- Introduced new methods for token counting and context trimming based on model profiles.
2026-03-10 18:18:59 -07:00
DESKTOP-RTLN3BA\$punk
d8a05ae4d5 feat: refactor agent tools management and add UI integration
- Added endpoint to list agent tools with metadata, excluding hidden tools.
- Updated NewChatRequest and RegenerateRequest schemas to include disabled tools.
- Integrated disabled tools management in the NewChatPage and Composer components.
- Improved tool instructions and visibility in the system prompt.
- Refactored tool registration to support hidden tools and default enabled states.
- Enhanced document chunk creation to handle strict zip behavior.
- Cleaned up imports and formatting across various files for consistency.
2026-03-10 17:36:26 -07:00
CREDO23
89ed845dcd merge upstream/dev into improve-ux-connectors
Resolve 3 conflicts:
- connector-popup.tsx: keep Picker modal coordination (pickerOpen state)
- google-drive-config.tsx: keep Google Picker, discard upstream folder tree
- composio-drive-config.tsx: accept upstream collapsible folder tree UX
2026-03-10 23:40:04 +02:00
CREDO23
cf8f70da2b fix auth bypass on picker endpoint, async safety, and picker error handling
- Add check_permission to drive-picker-token endpoint (IDOR fix)
- Use get_composio_service singleton + asyncio.to_thread to avoid blocking the event loop
- Sanitize error detail in 500 response to prevent internal info leakage
- Dispose picker on unmount to prevent orphaned overlay
- Surface error state on Google Picker Action.ERROR instead of silently closing
2026-03-10 23:21:35 +02:00
Rohan Verma
547077e5b9
Merge pull request #865 from CREDO23/sur-182-fix-ux-experience-for-composio-google-drive-connector
[Perf] Batch embedding, non-blocking search, chunks index & Google Drive UX fix
2026-03-10 12:52:16 -07:00
CREDO23
1e2c54eea6 add unified drive-picker-token endpoint and GOOGLE_PICKER_API_KEY config 2026-03-10 20:06:45 +02:00
CREDO23
0bb1b730dd fix Gmail HTML rendering: unwrap layout tables and use native img 2026-03-10 18:57:49 +02:00
DESKTOP-RTLN3BA\$punk
403097646d feat: implement batch unread counts for notifications to reduce API calls and improve performance 2026-03-10 01:26:37 -07:00
DESKTOP-RTLN3BA\$punk
45cc527d02 feat: update report generation and export capabilities to support multiple formats (PDF, DOCX, HTML, LaTeX, EPUB, ODT, plain text) across documentation and backend 2026-03-09 18:41:21 -07:00
DESKTOP-RTLN3BA\$punk
11b84a3fb4 feat: expand report export functionality to support multiple formats (PDF, DOCX, HTML, EPUB, etc.) and enhance UI for format selection 2026-03-09 18:31:14 -07:00
DESKTOP-RTLN3BA\$punk
c6fc4edbc2 feat: add YouTube video and playlist support in document collection with enhanced URL handling 2026-03-09 16:07:54 -07:00
CREDO23
fffef4cb5e perf: add missing index on chunks.document_id for faster search retrieval 2026-03-09 21:16:53 +02:00
CREDO23
6eabfe2396 perf: conditional batch embedding — batch for API, sequential for local 2026-03-09 19:12:43 +02:00
CREDO23
a49a4db6d6 perf: use asyncio.to_thread for embed_text in FastAPI paths 2026-03-09 16:33:24 +02:00
CREDO23
c4f2e9a3a5 feat: use batch embedding in create_document_chunks 2026-03-09 16:21:14 +02:00
CREDO23
929445afd9 feat: use batch embedding in IndexingPipelineService.index 2026-03-09 16:13:44 +02:00
CREDO23
cb4b155b9d feat: re-export embed_texts from document_embedder 2026-03-09 15:54:02 +02:00
CREDO23
15aeec1fcb feat: add embed_texts batch embedding utility 2026-03-09 15:53:40 +02:00
Anish Sarkar
e8cf677b25 refactor: update title generation logic to improve user experience by generating titles in parallel with assistant responses 2026-03-09 01:28:53 +05:30
Anish Sarkar
a11c95e30f feat: add last_login column to user table and update user login tracking 2026-03-08 18:24:29 +05:30
Anish Sarkar
f8b0e946ce chore: ran linting 2026-03-07 04:46:48 +05:30