apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-04-26 01:06:23 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
4976 commits 3 branches 209 tags 381 MiB
Commit graph

115 commits

Author SHA1 Message Date
DESKTOP-RTLN3BA\$punk
ff4e0f9b62 feat: no login experience and prem tokens
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Details
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Details
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Details
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Details
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Details
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Details
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
Details
2026-04-15 17:02:00 -07:00
DESKTOP-RTLN3BA\$punk
656e061f84 feat: add processing mode support for document uploads and ETL pipeline, improded error handling ux
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Details
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Details
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Details
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Details
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Details
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Details
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
Details 
- Introduced a `ProcessingMode` enum to differentiate between basic and premium processing modes.
- Updated `EtlRequest` to include a `processing_mode` field, defaulting to basic.
- Enhanced ETL pipeline services to utilize the selected processing mode for Azure Document Intelligence and LlamaCloud parsing.
- Modified various routes and services to handle processing mode, affecting document upload and indexing tasks.
- Improved error handling and logging to include processing mode details.
- Added tests to validate processing mode functionality and its impact on ETL operations.
 2026-04-14 21:26:00 -07:00
DESKTOP-RTLN3BA\$punk
4bee367d4a feat: added ai file sorting 2026-04-14 01:43:30 -07:00
CREDO23
a95bf58c8f Make Vision LLM opt-in for uploads and connectors 2026-04-10 16:45:51 +02:00
Anish Sarkar
5f5954e932 feat: implement upload-based folder indexing and synchronization features 2026-04-08 15:46:52 +05:30
Anish Sarkar
746c730b2e chore: ran linting 2026-04-03 13:14:40 +05:30
Anish Sarkar
4ca0787ab7 feat: add version history functionality to document management, enabling users to view and manage document versions through UI updates and backend integration 2026-04-03 10:42:21 +05:30
Anish Sarkar
1fa8e1cc83 feat: refactor folder indexing to support batch processing of multiple files, enhancing performance and error handling 2026-04-03 10:02:36 +05:30
Anish Sarkar
fe7fcaae5d feat: update folder deletion process to queue document deletions first and handle folder cleanup in Celery task 2026-04-03 04:16:19 +05:30
Anish Sarkar
22ee5c99cc refactor: remove Local Folder connector and related tasks, implement new folder indexing endpoints 2026-04-02 22:21:31 +05:30
Anish Sarkar
96a58d0d30 feat: implement local folder indexing and document versioning capabilities 2026-04-02 11:11:57 +05:30
DESKTOP-RTLN3BA\$punk
a9fd45844d feat: integrate Stripe for page purchases and reconciliation tasks 2026-03-31 18:39:45 -07:00
Anish Sarkar
1f12151e03 feat: implement Dropbox API client and folder management for enhanced file indexing 2026-03-30 22:17:50 +05:30
Anish Sarkar
5bddde60cb feat: implement Microsoft OneDrive connector with OAuth support and indexing capabilities 2026-03-28 14:31:25 +05:30
DESKTOP-RTLN3BA\$punk
685ad0c02d feat: add folder management features including creation, deletion, and organization of documents within folders 2026-03-27 01:39:15 -07:00
CREDO23
5d8a62a4a6 merge upstream/dev into feat/migrate-electric-to-zero 
Resolve 8 conflicts:
- Accept upstream deletion of 3 composio_*_connector.py (unified Google connectors)
- Accept our deletion of ElectricProvider.tsx, use-connectors-electric.ts,
  use-messages-electric.ts (replaced by Zero equivalents)
- Keep both new deps in package.json (@rocicorp/zero + @slate-serializers/html)
- Regenerate pnpm-lock.yaml
 2026-03-24 17:40:34 +02:00
CREDO23
cf21eaacfc fix: critical timestamp parsing and audit fixes 
- Fix timestamp conversion: String(epochMs) → new Date(epochMs).toISOString()
  in use-messages-sync, use-comments-sync, use-documents, use-inbox.
  Without this, date comparisons (isEdited, cutoff filters) would fail.
- Fix updated_at: undefined → null in use-inbox to match InboxItem type
- Fix ZeroProvider: skip Zero connection for unauthenticated users
- Clean 30+ stale "Electric SQL" comments in backend Python code
 2026-03-23 19:49:28 +02:00
Anish Sarkar
5c598e8588 Merge remote-tracking branch 'upstream/dev' into feat/human-in-the-loop 2026-03-22 15:45:45 +05:30
DESKTOP-RTLN3BA\$punk
d90b6d35ce feat: enhance video presentation agent with parallel theme assignment and watermarking 2026-03-21 23:02:09 -07:00
DESKTOP-RTLN3BA\$punk
b28f135a96 feat: init video presentation agent 2026-03-21 22:13:41 -07:00
Anish Sarkar
83152e8e7e refactor: unify all 3 google Composio and non-Composio connector types and pipelines keeping same credential adapters 2026-03-19 05:08:21 +05:30
DESKTOP-RTLN3BA\$punk
403097646d feat: implement batch unread counts for notifications to reduce API calls and improve performance 2026-03-10 01:26:37 -07:00
Anish Sarkar
6d00b0debf Merge remote-tracking branch 'upstream/dev' into refactor/upload-document-adapter-class 2026-03-01 22:35:17 +05:30
DESKTOP-RTLN3BA\$punk
d959a6a6c8 feat: optimize document upload process and enhance memory management 
- Increased maximum file upload limit from 10 to 50 to improve user experience.
- Implemented batch processing for document uploads to avoid proxy timeouts, splitting files into manageable chunks.
- Enhanced garbage collection in chat streaming functions to prevent memory leaks and improve performance.
- Added memory delta tracking in system snapshots for better monitoring of resource usage.
- Updated LLM router and service configurations to prevent unbounded internal accumulation and improve efficiency.
 2026-02-28 17:22:34 -08:00
Anish Sarkar
23a98d802c refactor: implement UploadDocumentAdapter for file indexing and reindexing 2026-02-28 01:38:32 +05:30
DESKTOP-RTLN3BA\$punk
e9892c8fe9 feat: added configable summary calculation and various improvements 
- Replaced direct embedding calls with a utility function across various components to streamline embedding logic.
- Added enable_summary flag to several models and routes to control summary generation behavior.
 2026-02-26 18:24:57 -08:00
Anish Sarkar
8b497da130 feat: add source_markdown column to documents and implement migration logic for existing records using a pure-Python BlockNote JSON to Markdown converter 2026-02-17 11:34:11 +05:30
DESKTOP-RTLN3BA\$punk
1849b451a5 feat: update Celery worker configuration and improve connector deletion process 
- Added support for multiple queues in Celery worker configuration.
- Modified connector deletion to handle documents inline instead of using a background task.
- Updated response messages for document creation and connector deletion to reflect new processing status.
- Removed the obsolete connector deletion Celery task file.
 2026-02-16 00:07:23 -08:00
DESKTOP-RTLN3BA\$punk
2b75c4e6d3 chore: linting 2026-02-09 14:14:46 -08:00
DESKTOP-RTLN3BA\$punk
17b7348f61 feat: fixed and improved search and background task management. 2026-02-09 14:03:56 -08:00
Anish Sarkar
76e7ddee2f chore: ran all linting 2026-02-06 18:22:19 +05:30
Anish Sarkar
2470fb70a6 fix: update error messages for task interruptions and refine document upload file size limits in English and Chinese translations 2026-02-06 18:13:11 +05:30
Anish Sarkar
e3faf4cc5e feat: enhance document upload handling by managing duplicates and updating statuses for existing documents 2026-02-06 18:12:46 +05:30
Anish Sarkar
72205ce11b feat: implement Redis heartbeat mechanism for document processing tasks and enhance stale notification cleanup 2026-02-06 18:09:05 +05:30
Anish Sarkar
0fdd194d92 Merge remote-tracking branch 'upstream/dev' into fix/documents 2026-02-06 12:13:26 +05:30
DESKTOP-RTLN3BA\$punk
1511c26ef5 feat: add residential proxy configuration for web crawling and YouTube transcript fetching 2026-02-05 20:44:13 -08:00
Anish Sarkar
aa66928154 chore: ran linting 2026-02-06 05:35:15 +05:30
Anish Sarkar
00a617ef17 feat: enhance stale notification cleanup task to mark associated documents as failed 2026-02-06 05:31:45 +05:30
Anish Sarkar
ed2fc5c636 feat: enhance document upload process with two-phase indexing and real-time status updates 2026-02-06 05:15:47 +05:30
Manoj Aggarwal
e6c0fabd0a Merge branch 'dev' into bugs_prod 2026-02-05 10:53:16 -08:00
Anish Sarkar
103baa8b7a Merge remote-tracking branch 'upstream/dev' into fix/documents 2026-02-04 03:04:52 +05:30
Anish Sarkar
30c6f42102 feat: streamline Composio connector logic by removing redundant checks and enhancing email retrieval for user accounts 2026-02-04 03:03:40 +05:30
Rohan Verma
d0673cecf6
Merge pull request #765 from AnishSarkar22/fix/documents 
feat: Add document ownership & deletion of documents
 2026-02-02 14:50:18 -08:00
DESKTOP-RTLN3BA\$punk
4df40f8cea feat: add CELERY_TASK_DEFAULT_QUEUE environment variable for task isolation 2026-02-02 13:17:12 -08:00
Anish Sarkar
f730df7c9d chore: ran linting 2026-02-03 01:43:38 +05:30
Manoj Aggarwal
48e646607b Fix google calendar and notion erros 2026-02-02 12:07:53 -08:00
DESKTOP-RTLN3BA\$punk
6f92eac3da try(hotpatch): add autoscaling command 2026-02-02 11:36:54 -08:00
Anish Sarkar
bf08982029 feat: add connector_id to documents for source tracking and implement connector deletion task 2026-02-02 16:23:26 +05:30
DESKTOP-RTLN3BA\$punk
8fb5a7fb8f chore: linting 2026-02-01 21:17:24 -08:00
DESKTOP-RTLN3BA\$punk
8301e0169c Merge branch 'dev' of https://github.com/MODSetter/SurfSense into dev 2026-02-01 18:02:27 -08:00