Commit graph

103 commits

Author SHA1 Message Date
Anish Sarkar
1f12151e03 feat: implement Dropbox API client and folder management for enhanced file indexing 2026-03-30 22:17:50 +05:30
Anish Sarkar
5bddde60cb feat: implement Microsoft OneDrive connector with OAuth support and indexing capabilities 2026-03-28 14:31:25 +05:30
DESKTOP-RTLN3BA\$punk
685ad0c02d feat: add folder management features including creation, deletion, and organization of documents within folders 2026-03-27 01:39:15 -07:00
CREDO23
5d8a62a4a6 merge upstream/dev into feat/migrate-electric-to-zero
Resolve 8 conflicts:
- Accept upstream deletion of 3 composio_*_connector.py (unified Google connectors)
- Accept our deletion of ElectricProvider.tsx, use-connectors-electric.ts,
  use-messages-electric.ts (replaced by Zero equivalents)
- Keep both new deps in package.json (@rocicorp/zero + @slate-serializers/html)
- Regenerate pnpm-lock.yaml
2026-03-24 17:40:34 +02:00
CREDO23
cf21eaacfc fix: critical timestamp parsing and audit fixes
- Fix timestamp conversion: String(epochMs) → new Date(epochMs).toISOString()
  in use-messages-sync, use-comments-sync, use-documents, use-inbox.
  Without this, date comparisons (isEdited, cutoff filters) would fail.
- Fix updated_at: undefined → null in use-inbox to match InboxItem type
- Fix ZeroProvider: skip Zero connection for unauthenticated users
- Clean 30+ stale "Electric SQL" comments in backend Python code
2026-03-23 19:49:28 +02:00
Anish Sarkar
5c598e8588 Merge remote-tracking branch 'upstream/dev' into feat/human-in-the-loop 2026-03-22 15:45:45 +05:30
DESKTOP-RTLN3BA\$punk
d90b6d35ce feat: enhance video presentation agent with parallel theme assignment and watermarking 2026-03-21 23:02:09 -07:00
DESKTOP-RTLN3BA\$punk
b28f135a96 feat: init video presentation agent 2026-03-21 22:13:41 -07:00
Anish Sarkar
83152e8e7e refactor: unify all 3 google Composio and non-Composio connector types and pipelines keeping same credential adapters 2026-03-19 05:08:21 +05:30
DESKTOP-RTLN3BA\$punk
403097646d feat: implement batch unread counts for notifications to reduce API calls and improve performance 2026-03-10 01:26:37 -07:00
Anish Sarkar
6d00b0debf Merge remote-tracking branch 'upstream/dev' into refactor/upload-document-adapter-class 2026-03-01 22:35:17 +05:30
DESKTOP-RTLN3BA\$punk
d959a6a6c8 feat: optimize document upload process and enhance memory management
- Increased maximum file upload limit from 10 to 50 to improve user experience.
- Implemented batch processing for document uploads to avoid proxy timeouts, splitting files into manageable chunks.
- Enhanced garbage collection in chat streaming functions to prevent memory leaks and improve performance.
- Added memory delta tracking in system snapshots for better monitoring of resource usage.
- Updated LLM router and service configurations to prevent unbounded internal accumulation and improve efficiency.
2026-02-28 17:22:34 -08:00
Anish Sarkar
23a98d802c refactor: implement UploadDocumentAdapter for file indexing and reindexing 2026-02-28 01:38:32 +05:30
DESKTOP-RTLN3BA\$punk
e9892c8fe9 feat: added configable summary calculation and various improvements
- Replaced direct embedding calls with a utility function across various components to streamline embedding logic.
- Added enable_summary flag to several models and routes to control summary generation behavior.
2026-02-26 18:24:57 -08:00
Anish Sarkar
8b497da130 feat: add source_markdown column to documents and implement migration logic for existing records using a pure-Python BlockNote JSON to Markdown converter 2026-02-17 11:34:11 +05:30
DESKTOP-RTLN3BA\$punk
1849b451a5 feat: update Celery worker configuration and improve connector deletion process
- Added support for multiple queues in Celery worker configuration.
- Modified connector deletion to handle documents inline instead of using a background task.
- Updated response messages for document creation and connector deletion to reflect new processing status.
- Removed the obsolete connector deletion Celery task file.
2026-02-16 00:07:23 -08:00
DESKTOP-RTLN3BA\$punk
2b75c4e6d3 chore: linting 2026-02-09 14:14:46 -08:00
DESKTOP-RTLN3BA\$punk
17b7348f61 feat: fixed and improved search and background task management. 2026-02-09 14:03:56 -08:00
Anish Sarkar
76e7ddee2f chore: ran all linting 2026-02-06 18:22:19 +05:30
Anish Sarkar
2470fb70a6 fix: update error messages for task interruptions and refine document upload file size limits in English and Chinese translations 2026-02-06 18:13:11 +05:30
Anish Sarkar
e3faf4cc5e feat: enhance document upload handling by managing duplicates and updating statuses for existing documents 2026-02-06 18:12:46 +05:30
Anish Sarkar
72205ce11b feat: implement Redis heartbeat mechanism for document processing tasks and enhance stale notification cleanup 2026-02-06 18:09:05 +05:30
Anish Sarkar
0fdd194d92 Merge remote-tracking branch 'upstream/dev' into fix/documents 2026-02-06 12:13:26 +05:30
DESKTOP-RTLN3BA\$punk
1511c26ef5 feat: add residential proxy configuration for web crawling and YouTube transcript fetching 2026-02-05 20:44:13 -08:00
Anish Sarkar
aa66928154 chore: ran linting 2026-02-06 05:35:15 +05:30
Anish Sarkar
00a617ef17 feat: enhance stale notification cleanup task to mark associated documents as failed 2026-02-06 05:31:45 +05:30
Anish Sarkar
ed2fc5c636 feat: enhance document upload process with two-phase indexing and real-time status updates 2026-02-06 05:15:47 +05:30
Manoj Aggarwal
e6c0fabd0a Merge branch 'dev' into bugs_prod 2026-02-05 10:53:16 -08:00
Anish Sarkar
103baa8b7a Merge remote-tracking branch 'upstream/dev' into fix/documents 2026-02-04 03:04:52 +05:30
Anish Sarkar
30c6f42102 feat: streamline Composio connector logic by removing redundant checks and enhancing email retrieval for user accounts 2026-02-04 03:03:40 +05:30
Rohan Verma
d0673cecf6
Merge pull request #765 from AnishSarkar22/fix/documents
feat: Add document ownership & deletion of documents
2026-02-02 14:50:18 -08:00
DESKTOP-RTLN3BA\$punk
4df40f8cea feat: add CELERY_TASK_DEFAULT_QUEUE environment variable for task isolation 2026-02-02 13:17:12 -08:00
Anish Sarkar
f730df7c9d chore: ran linting 2026-02-03 01:43:38 +05:30
Manoj Aggarwal
48e646607b Fix google calendar and notion erros 2026-02-02 12:07:53 -08:00
DESKTOP-RTLN3BA\$punk
6f92eac3da try(hotpatch): add autoscaling command 2026-02-02 11:36:54 -08:00
Anish Sarkar
bf08982029 feat: add connector_id to documents for source tracking and implement connector deletion task 2026-02-02 16:23:26 +05:30
DESKTOP-RTLN3BA\$punk
8fb5a7fb8f chore: linting 2026-02-01 21:17:24 -08:00
DESKTOP-RTLN3BA\$punk
8301e0169c Merge branch 'dev' of https://github.com/MODSetter/SurfSense into dev 2026-02-01 18:02:27 -08:00
DESKTOP-RTLN3BA\$punk
d476e18c54 feat: implement page limit exceeded notifications and enhance handling in the notification system 2026-02-01 18:02:17 -08:00
Anish Sarkar
05d1d6ac04 feat: implement Redis heartbeat tracking for connector indexing tasks and update stale notification cleanup logic 2026-02-02 00:18:47 +05:30
Anish Sarkar
085653d3e3 chore: ran frontend and backend linting 2026-02-01 22:54:25 +05:30
Anish Sarkar
024a683b4f feat: add heartbeat callback support for long-running indexing tasks and implement stale notification cleanup task 2026-02-01 02:17:06 +05:30
CREDO23
949ec949f6 style(backend): run ruff format on 10 files 2026-01-28 22:20:02 +02:00
CREDO23
b20fbaca4b fix: skip webcrawler indexing gracefully when no URLs configured 2026-01-28 17:54:46 +02:00
CREDO23
0c8d1f3fef refactor(backend): two-phase synchronous cloning 2026-01-28 00:17:29 +02:00
CREDO23
87c7d92672 feat: add podcast status tracking 2026-01-27 17:51:36 +02:00
CREDO23
a7145b2c63 merge: upstream/dev with migration renumbering 2026-01-27 11:22:26 +02:00
CREDO23
7017a14107 feat: pass thread_id through podcast generation chain 2026-01-26 15:56:34 +02:00
CREDO23
272e675669 add clone celery task and update route 2026-01-26 15:03:35 +02:00
Anish Sarkar
29382070aa feat: enhance Composio connector functionality with Google Drive delta sync support
- Added methods to retrieve the starting page token and list changes in Google Drive, enabling delta sync capabilities.
- Updated Composio service to handle file download directory configuration.
- Modified indexing tasks to support delta sync, improving efficiency by processing only changed files.
- Adjusted date handling in connector tasks to allow optional start and end dates.
- Improved error handling and logging throughout the Composio indexing process.
2026-01-23 18:37:09 +05:30