Commit graph

1471 commits

Author SHA1 Message Date
CREDO23
440762fb07 Merge upstream/dev 2026-03-31 20:21:12 +02:00
CREDO23
8f7dce3bbd Fix PromptMode enum to match PostgreSQL type 2026-03-31 20:12:04 +02:00
CREDO23
950be8e17d Fix review issues: idempotent migration, toggle guard, picker states 2026-03-31 19:27:32 +02:00
CREDO23
aecb58e22b Fix review issues: authz, race conditions, UX safety, and consolidate migrations 2026-03-31 19:09:05 +02:00
CREDO23
5f4f7780d1 Fix create_prompt is_public bug, conditional version bump, and add Jotai prompts atoms
- Pass is_public from request body in create_prompt route
- Only bump version on content field changes (name, prompt, mode),
  not on is_public toggle
- Add prompts query and mutation atoms (atomWithQuery/atomWithMutation)
  with TanStack Query caching, replacing manual useEffect fetches
- Update PromptPicker, PromptsContent, and CommunityPromptsContent
  to consume shared atoms instead of local state
2026-03-31 18:34:10 +02:00
CREDO23
11387268a7 Seed default prompts on registration and for existing users 2026-03-31 18:12:09 +02:00
CREDO23
0c975a6f80 Simplify prompts to seed-based CRUD with version tracking 2026-03-31 18:05:42 +02:00
CREDO23
80d096db32 Add system prompt merge logic, upsert override, and reset endpoints 2026-03-31 16:47:51 +02:00
CREDO23
329e979d48 Remove icon from prompts, update schemas for system prompt overrides 2026-03-31 16:38:35 +02:00
CREDO23
16f4d007e4 Add system_prompt_slug column and migration for prompt overrides 2026-03-31 16:27:04 +02:00
CREDO23
2024e9b1ed Add system prompt defaults constant for copy-on-write overrides 2026-03-31 16:23:11 +02:00
CREDO23
24fd19a394 add public prompts and copy endpoints 2026-03-30 19:36:54 +02:00
CREDO23
f14f09cbe6 add is_public to prompts model, schema, and migration 2026-03-30 19:33:16 +02:00
Anish Sarkar
04691d572b chore: ran linting 2026-03-30 01:50:41 +05:30
Anish Sarkar
74826b3714 feat: enhance web search tool integration with citation management and UI enhancements 2026-03-30 01:38:36 +05:30
CREDO23
3363c71921 fix: renumber prompts migration to 111 to follow upstream OneDrive migration 2026-03-29 10:43:49 +02:00
CREDO23
ab3c636bcd Merge remote-tracking branch 'upstream/dev' into electon-desktop 2026-03-29 10:41:05 +02:00
Anish Sarkar
5a3eece397 Merge remote-tracking branch 'upstream/dev' into feat/onedrive-connector 2026-03-29 11:55:06 +05:30
CREDO23
726fec7871 fix: renumber prompts migration to 110 to follow upstream folders migration 2026-03-29 08:13:47 +02:00
CREDO23
b8a1d1f594 Merge remote-tracking branch 'upstream/dev' into electon-desktop 2026-03-29 03:10:51 +02:00
DESKTOP-RTLN3BA\$punk
2cc2d339e6 feat: made agent file sytem optimized 2026-03-28 16:39:46 -07:00
Anish Sarkar
5d467cdfc2 feat: enhance OneDrive file creation tool by adding parent folder selection and handling expired accounts 2026-03-29 05:08:04 +05:30
Anish Sarkar
c325f53941 feat: enhance delete OneDrive file tool with improved error handling and user feedback 2026-03-29 05:04:47 +05:30
Anish Sarkar
fd87e38dac feat: implement OneDrive knowledge base synchronization after file creation to enhance document management 2026-03-29 05:03:05 +05:30
Anish Sarkar
8035eb9749 feat: enhance OneDrive file creation by converting markdown to DOCX format and updating client to handle byte content 2026-03-29 05:02:08 +05:30
Anish Sarkar
65bd83b167 feat: add ONEDRIVE_FILE to connector mappings and descriptions for enhanced OneDrive support 2026-03-29 04:32:46 +05:30
CREDO23
a6ccb7a875 rename quick-ask-actions to prompts across backend and frontend 2026-03-29 00:07:08 +02:00
Anish Sarkar
c8767272ec feat: enhance OneDrive folder management by adding mimeType handling and integrating DriveFolderTree component for improved UI 2026-03-29 03:29:31 +05:30
CREDO23
041401aefc add custom quick-ask actions: model, migration, schemas, CRUD routes 2026-03-28 23:49:50 +02:00
Anish Sarkar
ea218b7be6 feat: implement OneDrive picker integration and enhance connector functionality with new API endpoints and UI updates 2026-03-28 23:57:46 +05:30
Anish Sarkar
028c88be72 feat: add integration and unit tests for OneDrive indexing pipeline and parallel downloads 2026-03-28 16:39:47 +05:30
Anish Sarkar
7004e764a9 chore: refactor Microsoft OAuth configuration to unify client ID and secret for Teams and OneDrive in environment files and related code 2026-03-28 16:37:23 +05:30
Anish Sarkar
bb894ee158 chore: update environment variable names for Microsoft OAuth integration in Docker and SurfSense backend 2026-03-28 16:35:36 +05:30
Anish Sarkar
5f0a4d1a0f feat: add OneDrive file creation and deletion tools with connector checks 2026-03-28 14:34:30 +05:30
Anish Sarkar
5bddde60cb feat: implement Microsoft OneDrive connector with OAuth support and indexing capabilities 2026-03-28 14:31:25 +05:30
Anish Sarkar
b5ef7afb1c feat: add multi-format document export functionality to editor routes and UI components
- Implemented a new export endpoint in the backend to support exporting documents in various formats (PDF, DOCX, HTML, LaTeX, EPUB, ODT, plain text).
- Enhanced DocumentNode and FolderTreeView components to include export options in context and dropdown menus.
- Created shared ExportMenuItems component for consistent export options across the application.
- Integrated loading indicators for export actions to improve user experience.
2026-03-28 02:58:38 +05:30
Anish Sarkar
17091edb77 Merge remote-tracking branch 'upstream/dev' into refactor/indexing-pipelines 2026-03-27 22:36:34 +05:30
Anish Sarkar
6d4eb32345 fix: update export format for Google Docs to use correct MIME type 2026-03-27 22:20:32 +05:30
Anish Sarkar
489e48644f fix: revert native excel parsing 2026-03-27 22:15:24 +05:30
Anish Sarkar
dff8a1df37 feat: add descendant checking for folder filtering in Google Drive changes 2026-03-27 22:00:31 +05:30
Anish Sarkar
3da0ffd683 feat: add native Excel parsing and improve Google Drive content extraction
- Introduced a new utility for parsing .xlsx files into markdown format, enhancing the ability to process Excel documents natively.
- Updated the Google Drive content extractor to utilize the new Excel parsing functionality, allowing for better handling of spreadsheet files.
- Enhanced file type detection and export logic to support various document formats, improving overall content extraction accuracy.
- Added unit tests to ensure the correctness of the new Excel parsing feature and its integration with existing content extraction workflows.
2026-03-27 21:47:14 +05:30
Anish Sarkar
4e0749f907 fix: update file skipping logic for failed documents in Google Drive indexer
- Modified the `_should_skip_file` function to skip previously failed documents during processing, improving error handling.
- Updated the corresponding test to reflect the new behavior, ensuring that failed documents are correctly identified and skipped during automatic sync.
2026-03-27 20:01:08 +05:30
Anish Sarkar
00934ff462 feat: enhance Google Drive client with improved logging and thread-safe operations
- Added logging to track the start and end of file download and export processes, improving visibility into execution time.
- Implemented per-thread HTTP transport for concurrent downloads and exports, ensuring thread safety.
- Refactored download and export methods to utilize resolved credentials, enhancing functionality.
- Updated unit tests to validate the new threading and logging features, ensuring robust parallel execution.
2026-03-27 19:25:45 +05:30
Anish Sarkar
d2a4b238d7 feat: enhance Google Drive client with thread-safe download and export methods
- Implemented per-thread HTTP transport for concurrent downloads to ensure thread safety.
- Refactored `download_file` and `download_file_to_disk` methods to utilize blocking calls on separate threads, improving performance during file operations.
- Added logging to track the start and end of download and export processes, providing better visibility into execution time.
- Updated unit tests to verify parallel execution of download and export operations, ensuring efficiency in handling multiple requests.
2026-03-27 19:25:03 +05:30
Anish Sarkar
0bc1c766ff feat: migrate Confluence and Jira indexers to unified parallel pipeline
- Refactored Confluence and Jira indexers to utilize the shared IndexingPipelineService for improved document processing.
- Updated the `_build_connector_doc` function in both indexers to create ConnectorDocument instances with enhanced metadata and fallback summaries.
- Modified the `index_confluence_pages` and `index_jira_issues` functions to return a tuple of (indexed_count, skipped_count, warning_or_error_message) for better error handling and reporting.
- Added unit tests for both indexers to validate the new parallel processing logic and ensure correct document creation and indexing behavior.
2026-03-27 16:02:09 +05:30
DESKTOP-RTLN3BA\$punk
685ad0c02d feat: add folder management features including creation, deletion, and organization of documents within folders 2026-03-27 01:39:15 -07:00
Anish Sarkar
683a4c17dd feat: implement thread-safe embedding access in document converters
- Added a reentrant lock to ensure thread-safe access to the tokenizer and embedding model, preventing runtime errors during concurrent operations.
- Updated the `truncate_for_embedding` and `embed_text` functions to utilize the lock, ensuring safe execution in multi-threaded environments.
- Enhanced the `embed_texts` function to maintain thread safety while processing multiple texts for embedding.
2026-03-27 11:31:00 +05:30
Anish Sarkar
db6dd058dd feat: migrate Linear and Notion indexers to unified parallel pipeline
- Refactored Linear and Notion indexers to utilize the shared IndexingPipelineService for improved document deduplication, summarization, chunking, and embedding with bounded parallel indexing.
- Updated the `_build_connector_doc` function in both indexers to create ConnectorDocument instances with enhanced metadata and fallback summaries.
- Modified the `index_linear_issues` and `index_notion_pages` functions to return a tuple of (indexed_count, skipped_count, warning_or_error_message) for better error handling and reporting.
- Added unit tests for both indexers to validate the new parallel processing logic and ensure correct document creation and indexing behavior.
2026-03-27 11:19:32 +05:30
Anish Sarkar
da6bbcfe39 feat: add file streaming download functionality to Google Drive client
- Introduced `download_file_to_disk` method to stream files directly to disk in chunks, reducing memory usage during downloads.
- Updated `download_and_extract_content` function to utilize the new streaming download method for binary files, enhancing efficiency in handling large files.
- Improved error handling for download operations, providing clearer feedback on failures.
2026-03-27 08:54:06 +05:30
Anish Sarkar
7c7f8b216c feat: implement batch indexing for selected Google Drive files
- Introduced `index_google_drive_selected_files` function to enable indexing of multiple user-selected files in parallel, improving efficiency.
- Refactored existing indexing logic to handle batch processing, including error handling for individual file failures.
- Added unit tests for the new batch indexing functionality, ensuring robustness and proper error collection during the indexing process.
2026-03-27 00:17:07 +05:30