Commit graph

98 commits

Author SHA1 Message Date
Anish Sarkar
5c598e8588 Merge remote-tracking branch 'upstream/dev' into feat/human-in-the-loop 2026-03-22 15:45:45 +05:30
DESKTOP-RTLN3BA\$punk
d90b6d35ce feat: enhance video presentation agent with parallel theme assignment and watermarking 2026-03-21 23:02:09 -07:00
DESKTOP-RTLN3BA\$punk
b28f135a96 feat: init video presentation agent 2026-03-21 22:13:41 -07:00
Anish Sarkar
83152e8e7e refactor: unify all 3 google Composio and non-Composio connector types and pipelines keeping same credential adapters 2026-03-19 05:08:21 +05:30
DESKTOP-RTLN3BA\$punk
403097646d feat: implement batch unread counts for notifications to reduce API calls and improve performance 2026-03-10 01:26:37 -07:00
Anish Sarkar
6d00b0debf Merge remote-tracking branch 'upstream/dev' into refactor/upload-document-adapter-class 2026-03-01 22:35:17 +05:30
DESKTOP-RTLN3BA\$punk
d959a6a6c8 feat: optimize document upload process and enhance memory management
- Increased maximum file upload limit from 10 to 50 to improve user experience.
- Implemented batch processing for document uploads to avoid proxy timeouts, splitting files into manageable chunks.
- Enhanced garbage collection in chat streaming functions to prevent memory leaks and improve performance.
- Added memory delta tracking in system snapshots for better monitoring of resource usage.
- Updated LLM router and service configurations to prevent unbounded internal accumulation and improve efficiency.
2026-02-28 17:22:34 -08:00
Anish Sarkar
23a98d802c refactor: implement UploadDocumentAdapter for file indexing and reindexing 2026-02-28 01:38:32 +05:30
DESKTOP-RTLN3BA\$punk
e9892c8fe9 feat: added configable summary calculation and various improvements
- Replaced direct embedding calls with a utility function across various components to streamline embedding logic.
- Added enable_summary flag to several models and routes to control summary generation behavior.
2026-02-26 18:24:57 -08:00
Anish Sarkar
8b497da130 feat: add source_markdown column to documents and implement migration logic for existing records using a pure-Python BlockNote JSON to Markdown converter 2026-02-17 11:34:11 +05:30
DESKTOP-RTLN3BA\$punk
1849b451a5 feat: update Celery worker configuration and improve connector deletion process
- Added support for multiple queues in Celery worker configuration.
- Modified connector deletion to handle documents inline instead of using a background task.
- Updated response messages for document creation and connector deletion to reflect new processing status.
- Removed the obsolete connector deletion Celery task file.
2026-02-16 00:07:23 -08:00
DESKTOP-RTLN3BA\$punk
2b75c4e6d3 chore: linting 2026-02-09 14:14:46 -08:00
DESKTOP-RTLN3BA\$punk
17b7348f61 feat: fixed and improved search and background task management. 2026-02-09 14:03:56 -08:00
Anish Sarkar
76e7ddee2f chore: ran all linting 2026-02-06 18:22:19 +05:30
Anish Sarkar
2470fb70a6 fix: update error messages for task interruptions and refine document upload file size limits in English and Chinese translations 2026-02-06 18:13:11 +05:30
Anish Sarkar
e3faf4cc5e feat: enhance document upload handling by managing duplicates and updating statuses for existing documents 2026-02-06 18:12:46 +05:30
Anish Sarkar
72205ce11b feat: implement Redis heartbeat mechanism for document processing tasks and enhance stale notification cleanup 2026-02-06 18:09:05 +05:30
Anish Sarkar
0fdd194d92 Merge remote-tracking branch 'upstream/dev' into fix/documents 2026-02-06 12:13:26 +05:30
DESKTOP-RTLN3BA\$punk
1511c26ef5 feat: add residential proxy configuration for web crawling and YouTube transcript fetching 2026-02-05 20:44:13 -08:00
Anish Sarkar
aa66928154 chore: ran linting 2026-02-06 05:35:15 +05:30
Anish Sarkar
00a617ef17 feat: enhance stale notification cleanup task to mark associated documents as failed 2026-02-06 05:31:45 +05:30
Anish Sarkar
ed2fc5c636 feat: enhance document upload process with two-phase indexing and real-time status updates 2026-02-06 05:15:47 +05:30
Manoj Aggarwal
e6c0fabd0a Merge branch 'dev' into bugs_prod 2026-02-05 10:53:16 -08:00
Anish Sarkar
103baa8b7a Merge remote-tracking branch 'upstream/dev' into fix/documents 2026-02-04 03:04:52 +05:30
Anish Sarkar
30c6f42102 feat: streamline Composio connector logic by removing redundant checks and enhancing email retrieval for user accounts 2026-02-04 03:03:40 +05:30
Rohan Verma
d0673cecf6
Merge pull request #765 from AnishSarkar22/fix/documents
feat: Add document ownership & deletion of documents
2026-02-02 14:50:18 -08:00
DESKTOP-RTLN3BA\$punk
4df40f8cea feat: add CELERY_TASK_DEFAULT_QUEUE environment variable for task isolation 2026-02-02 13:17:12 -08:00
Anish Sarkar
f730df7c9d chore: ran linting 2026-02-03 01:43:38 +05:30
Manoj Aggarwal
48e646607b Fix google calendar and notion erros 2026-02-02 12:07:53 -08:00
DESKTOP-RTLN3BA\$punk
6f92eac3da try(hotpatch): add autoscaling command 2026-02-02 11:36:54 -08:00
Anish Sarkar
bf08982029 feat: add connector_id to documents for source tracking and implement connector deletion task 2026-02-02 16:23:26 +05:30
DESKTOP-RTLN3BA\$punk
8fb5a7fb8f chore: linting 2026-02-01 21:17:24 -08:00
DESKTOP-RTLN3BA\$punk
8301e0169c Merge branch 'dev' of https://github.com/MODSetter/SurfSense into dev 2026-02-01 18:02:27 -08:00
DESKTOP-RTLN3BA\$punk
d476e18c54 feat: implement page limit exceeded notifications and enhance handling in the notification system 2026-02-01 18:02:17 -08:00
Anish Sarkar
05d1d6ac04 feat: implement Redis heartbeat tracking for connector indexing tasks and update stale notification cleanup logic 2026-02-02 00:18:47 +05:30
Anish Sarkar
085653d3e3 chore: ran frontend and backend linting 2026-02-01 22:54:25 +05:30
Anish Sarkar
024a683b4f feat: add heartbeat callback support for long-running indexing tasks and implement stale notification cleanup task 2026-02-01 02:17:06 +05:30
CREDO23
949ec949f6 style(backend): run ruff format on 10 files 2026-01-28 22:20:02 +02:00
CREDO23
b20fbaca4b fix: skip webcrawler indexing gracefully when no URLs configured 2026-01-28 17:54:46 +02:00
CREDO23
0c8d1f3fef refactor(backend): two-phase synchronous cloning 2026-01-28 00:17:29 +02:00
CREDO23
87c7d92672 feat: add podcast status tracking 2026-01-27 17:51:36 +02:00
CREDO23
a7145b2c63 merge: upstream/dev with migration renumbering 2026-01-27 11:22:26 +02:00
CREDO23
7017a14107 feat: pass thread_id through podcast generation chain 2026-01-26 15:56:34 +02:00
CREDO23
272e675669 add clone celery task and update route 2026-01-26 15:03:35 +02:00
Anish Sarkar
29382070aa feat: enhance Composio connector functionality with Google Drive delta sync support
- Added methods to retrieve the starting page token and list changes in Google Drive, enabling delta sync capabilities.
- Updated Composio service to handle file download directory configuration.
- Modified indexing tasks to support delta sync, improving efficiency by processing only changed files.
- Adjusted date handling in connector tasks to allow optional start and end dates.
- Improved error handling and logging throughout the Composio indexing process.
2026-01-23 18:37:09 +05:30
Anish Sarkar
fae52345f8 Merge remote-tracking branch 'upstream/dev' into feat/composio 2026-01-23 14:35:17 +05:30
Manoj Aggarwal
f0760c14e9 Merge dev into feature/obsidian - resolved conflicts keeping both Obsidian and Composio connectors 2026-01-22 11:43:18 -08:00
Anish Sarkar
be5715cfeb feat: add Composio connector types and enhance integration
- Introduced new enum values for Composio connectors: COMPOSIO_GOOGLE_DRIVE_CONNECTOR, COMPOSIO_GMAIL_CONNECTOR, and COMPOSIO_GOOGLE_CALENDAR_CONNECTOR.
- Updated database migration to add these new enum values to the relevant types.
- Refactored Composio integration logic to handle specific connector types, improving the management of connected accounts and indexing processes.
- Enhanced frontend components to support the new Composio connector types, including updated UI elements and connector configuration handling.
- Improved backend services to manage Composio connected accounts more effectively, including deletion and indexing tasks.
2026-01-22 22:33:28 +05:30
DESKTOP-RTLN3BA\$punk
8c625d4237 feat: composio connector 2026-01-21 22:57:58 -08:00
Manoj Aggarwal
8a83424be5 Add support for obsidian to surfsense 2026-01-21 15:21:06 -08:00