Commit graph

1465 commits

Author SHA1 Message Date
CREDO23
5f4f7780d1 Fix create_prompt is_public bug, conditional version bump, and add Jotai prompts atoms
- Pass is_public from request body in create_prompt route
- Only bump version on content field changes (name, prompt, mode),
  not on is_public toggle
- Add prompts query and mutation atoms (atomWithQuery/atomWithMutation)
  with TanStack Query caching, replacing manual useEffect fetches
- Update PromptPicker, PromptsContent, and CommunityPromptsContent
  to consume shared atoms instead of local state
2026-03-31 18:34:10 +02:00
CREDO23
11387268a7 Seed default prompts on registration and for existing users 2026-03-31 18:12:09 +02:00
CREDO23
0c975a6f80 Simplify prompts to seed-based CRUD with version tracking 2026-03-31 18:05:42 +02:00
CREDO23
80d096db32 Add system prompt merge logic, upsert override, and reset endpoints 2026-03-31 16:47:51 +02:00
CREDO23
329e979d48 Remove icon from prompts, update schemas for system prompt overrides 2026-03-31 16:38:35 +02:00
CREDO23
16f4d007e4 Add system_prompt_slug column and migration for prompt overrides 2026-03-31 16:27:04 +02:00
CREDO23
2024e9b1ed Add system prompt defaults constant for copy-on-write overrides 2026-03-31 16:23:11 +02:00
CREDO23
24fd19a394 add public prompts and copy endpoints 2026-03-30 19:36:54 +02:00
CREDO23
f14f09cbe6 add is_public to prompts model, schema, and migration 2026-03-30 19:33:16 +02:00
CREDO23
3363c71921 fix: renumber prompts migration to 111 to follow upstream OneDrive migration 2026-03-29 10:43:49 +02:00
CREDO23
ab3c636bcd Merge remote-tracking branch 'upstream/dev' into electon-desktop 2026-03-29 10:41:05 +02:00
Anish Sarkar
5a3eece397 Merge remote-tracking branch 'upstream/dev' into feat/onedrive-connector 2026-03-29 11:55:06 +05:30
CREDO23
726fec7871 fix: renumber prompts migration to 110 to follow upstream folders migration 2026-03-29 08:13:47 +02:00
CREDO23
b8a1d1f594 Merge remote-tracking branch 'upstream/dev' into electon-desktop 2026-03-29 03:10:51 +02:00
DESKTOP-RTLN3BA\$punk
2cc2d339e6 feat: made agent file sytem optimized 2026-03-28 16:39:46 -07:00
Anish Sarkar
5d467cdfc2 feat: enhance OneDrive file creation tool by adding parent folder selection and handling expired accounts 2026-03-29 05:08:04 +05:30
Anish Sarkar
c325f53941 feat: enhance delete OneDrive file tool with improved error handling and user feedback 2026-03-29 05:04:47 +05:30
Anish Sarkar
fd87e38dac feat: implement OneDrive knowledge base synchronization after file creation to enhance document management 2026-03-29 05:03:05 +05:30
Anish Sarkar
8035eb9749 feat: enhance OneDrive file creation by converting markdown to DOCX format and updating client to handle byte content 2026-03-29 05:02:08 +05:30
Anish Sarkar
65bd83b167 feat: add ONEDRIVE_FILE to connector mappings and descriptions for enhanced OneDrive support 2026-03-29 04:32:46 +05:30
CREDO23
a6ccb7a875 rename quick-ask-actions to prompts across backend and frontend 2026-03-29 00:07:08 +02:00
Anish Sarkar
c8767272ec feat: enhance OneDrive folder management by adding mimeType handling and integrating DriveFolderTree component for improved UI 2026-03-29 03:29:31 +05:30
CREDO23
041401aefc add custom quick-ask actions: model, migration, schemas, CRUD routes 2026-03-28 23:49:50 +02:00
Anish Sarkar
ea218b7be6 feat: implement OneDrive picker integration and enhance connector functionality with new API endpoints and UI updates 2026-03-28 23:57:46 +05:30
Anish Sarkar
028c88be72 feat: add integration and unit tests for OneDrive indexing pipeline and parallel downloads 2026-03-28 16:39:47 +05:30
Anish Sarkar
7004e764a9 chore: refactor Microsoft OAuth configuration to unify client ID and secret for Teams and OneDrive in environment files and related code 2026-03-28 16:37:23 +05:30
Anish Sarkar
bb894ee158 chore: update environment variable names for Microsoft OAuth integration in Docker and SurfSense backend 2026-03-28 16:35:36 +05:30
Anish Sarkar
5f0a4d1a0f feat: add OneDrive file creation and deletion tools with connector checks 2026-03-28 14:34:30 +05:30
Anish Sarkar
5bddde60cb feat: implement Microsoft OneDrive connector with OAuth support and indexing capabilities 2026-03-28 14:31:25 +05:30
Anish Sarkar
b5ef7afb1c feat: add multi-format document export functionality to editor routes and UI components
- Implemented a new export endpoint in the backend to support exporting documents in various formats (PDF, DOCX, HTML, LaTeX, EPUB, ODT, plain text).
- Enhanced DocumentNode and FolderTreeView components to include export options in context and dropdown menus.
- Created shared ExportMenuItems component for consistent export options across the application.
- Integrated loading indicators for export actions to improve user experience.
2026-03-28 02:58:38 +05:30
Anish Sarkar
17091edb77 Merge remote-tracking branch 'upstream/dev' into refactor/indexing-pipelines 2026-03-27 22:36:34 +05:30
Anish Sarkar
6d4eb32345 fix: update export format for Google Docs to use correct MIME type 2026-03-27 22:20:32 +05:30
Anish Sarkar
489e48644f fix: revert native excel parsing 2026-03-27 22:15:24 +05:30
Anish Sarkar
dff8a1df37 feat: add descendant checking for folder filtering in Google Drive changes 2026-03-27 22:00:31 +05:30
Anish Sarkar
3da0ffd683 feat: add native Excel parsing and improve Google Drive content extraction
- Introduced a new utility for parsing .xlsx files into markdown format, enhancing the ability to process Excel documents natively.
- Updated the Google Drive content extractor to utilize the new Excel parsing functionality, allowing for better handling of spreadsheet files.
- Enhanced file type detection and export logic to support various document formats, improving overall content extraction accuracy.
- Added unit tests to ensure the correctness of the new Excel parsing feature and its integration with existing content extraction workflows.
2026-03-27 21:47:14 +05:30
Anish Sarkar
4e0749f907 fix: update file skipping logic for failed documents in Google Drive indexer
- Modified the `_should_skip_file` function to skip previously failed documents during processing, improving error handling.
- Updated the corresponding test to reflect the new behavior, ensuring that failed documents are correctly identified and skipped during automatic sync.
2026-03-27 20:01:08 +05:30
Anish Sarkar
00934ff462 feat: enhance Google Drive client with improved logging and thread-safe operations
- Added logging to track the start and end of file download and export processes, improving visibility into execution time.
- Implemented per-thread HTTP transport for concurrent downloads and exports, ensuring thread safety.
- Refactored download and export methods to utilize resolved credentials, enhancing functionality.
- Updated unit tests to validate the new threading and logging features, ensuring robust parallel execution.
2026-03-27 19:25:45 +05:30
Anish Sarkar
d2a4b238d7 feat: enhance Google Drive client with thread-safe download and export methods
- Implemented per-thread HTTP transport for concurrent downloads to ensure thread safety.
- Refactored `download_file` and `download_file_to_disk` methods to utilize blocking calls on separate threads, improving performance during file operations.
- Added logging to track the start and end of download and export processes, providing better visibility into execution time.
- Updated unit tests to verify parallel execution of download and export operations, ensuring efficiency in handling multiple requests.
2026-03-27 19:25:03 +05:30
Anish Sarkar
0bc1c766ff feat: migrate Confluence and Jira indexers to unified parallel pipeline
- Refactored Confluence and Jira indexers to utilize the shared IndexingPipelineService for improved document processing.
- Updated the `_build_connector_doc` function in both indexers to create ConnectorDocument instances with enhanced metadata and fallback summaries.
- Modified the `index_confluence_pages` and `index_jira_issues` functions to return a tuple of (indexed_count, skipped_count, warning_or_error_message) for better error handling and reporting.
- Added unit tests for both indexers to validate the new parallel processing logic and ensure correct document creation and indexing behavior.
2026-03-27 16:02:09 +05:30
DESKTOP-RTLN3BA\$punk
685ad0c02d feat: add folder management features including creation, deletion, and organization of documents within folders 2026-03-27 01:39:15 -07:00
Anish Sarkar
683a4c17dd feat: implement thread-safe embedding access in document converters
- Added a reentrant lock to ensure thread-safe access to the tokenizer and embedding model, preventing runtime errors during concurrent operations.
- Updated the `truncate_for_embedding` and `embed_text` functions to utilize the lock, ensuring safe execution in multi-threaded environments.
- Enhanced the `embed_texts` function to maintain thread safety while processing multiple texts for embedding.
2026-03-27 11:31:00 +05:30
Anish Sarkar
db6dd058dd feat: migrate Linear and Notion indexers to unified parallel pipeline
- Refactored Linear and Notion indexers to utilize the shared IndexingPipelineService for improved document deduplication, summarization, chunking, and embedding with bounded parallel indexing.
- Updated the `_build_connector_doc` function in both indexers to create ConnectorDocument instances with enhanced metadata and fallback summaries.
- Modified the `index_linear_issues` and `index_notion_pages` functions to return a tuple of (indexed_count, skipped_count, warning_or_error_message) for better error handling and reporting.
- Added unit tests for both indexers to validate the new parallel processing logic and ensure correct document creation and indexing behavior.
2026-03-27 11:19:32 +05:30
Anish Sarkar
da6bbcfe39 feat: add file streaming download functionality to Google Drive client
- Introduced `download_file_to_disk` method to stream files directly to disk in chunks, reducing memory usage during downloads.
- Updated `download_and_extract_content` function to utilize the new streaming download method for binary files, enhancing efficiency in handling large files.
- Improved error handling for download operations, providing clearer feedback on failures.
2026-03-27 08:54:06 +05:30
Anish Sarkar
7c7f8b216c feat: implement batch indexing for selected Google Drive files
- Introduced `index_google_drive_selected_files` function to enable indexing of multiple user-selected files in parallel, improving efficiency.
- Refactored existing indexing logic to handle batch processing, including error handling for individual file failures.
- Added unit tests for the new batch indexing functionality, ensuring robustness and proper error collection during the indexing process.
2026-03-27 00:17:07 +05:30
Anish Sarkar
2f30e48e90 feat: implement async service locking in Google Drive client
- Introduced an asyncio lock to the GoogleDriveClient to ensure thread-safe access to the service instance.
- Refactored the get_service method to utilize the lock, preventing concurrent attempts to create the service and improving stability in multi-threaded environments.
2026-03-27 00:06:21 +05:30
Anish Sarkar
c016962064 feat: implement parallel file downloading and indexing in Google Drive indexer
- Added `_download_files_parallel` function to enable concurrent downloading of files from Google Drive, improving efficiency in document processing.
- Introduced `_download_and_index` function to handle the parallel downloading and indexing phases, streamlining the overall workflow.
- Updated `_index_full_scan` and `_index_with_delta_sync` methods to utilize the new parallel downloading functionality, enhancing performance.
- Added unit tests to validate the new parallel downloading and indexing logic, ensuring robustness and error handling during document processing.
2026-03-26 23:53:26 +05:30
Anish Sarkar
bd6e335cb3 feat: enhance performance logging in indexing pipeline
- Added performance logging to the `index_batch_parallel` method, capturing metrics for document indexing duration and concurrency.
- Introduced timing measurements for both the overall indexing process and the parallel document gathering phase, improving observability of the indexing workflow.
- Updated logging statements to provide detailed insights into the number of documents processed, indexed, and failed during the indexing operation.
2026-03-26 23:10:49 +05:30
Anish Sarkar
4fd776e7ef feat: implement parallel indexing for Google Calendar and Gmail connectors
- Refactored Google Calendar and Gmail indexers to utilize the new `index_batch_parallel` method for concurrent document indexing, enhancing performance.
- Updated the indexing logic to replace serial processing with parallel execution, allowing for improved efficiency in handling multiple documents.
- Adjusted logging and error handling to accommodate the new parallel processing approach, ensuring robust operation during indexing.
- Enhanced unit tests to validate the functionality of the parallel indexing method and its integration with existing workflows.
2026-03-26 19:34:04 +05:30
Anish Sarkar
e5cb6bfacf feat: implement parallel document indexing in IndexingPipelineService
- Added `index_batch_parallel` method to enable concurrent indexing of documents with bounded concurrency, improving performance and efficiency.
- Refactored existing indexing logic to utilize `asyncio.to_thread` for non-blocking execution of embedding and chunking functions.
- Introduced unit tests to validate the functionality of the new parallel indexing method, ensuring robustness and error handling during document processing.
2026-03-26 19:33:49 +05:30
Anish Sarkar
bbd5ee8a19 feat: enhance Google Calendar event update functionality
- Introduced helper functions `_is_date_only` and `_build_time_body` to streamline the construction of event start and end times for all-day and timed events.
- Refactored the `create_update_calendar_event_tool` to utilize the new helper functions, improving code readability and maintainability.
- Updated the Google Calendar sync service to ensure proper handling of calendar IDs with a default fallback to "primary".
- Modified the ApprovalCard component to simplify the construction of event update arguments, enhancing clarity and reducing redundancy.
2026-03-25 20:35:23 +05:30