Commit graph

61 commits

Author SHA1 Message Date
Anish Sarkar
83d9c49a50 feat: enhance Composio connector handling and status management
- Updated the .env.example file to include a note about disabling "Mask Connected Account Secrets" for Google indexing.
- Implemented a wait_for_connection method in ComposioService to ensure connected accounts reach ACTIVE status after authentication.
- Added logic in composio_callback and composio_reauth_callback to wait for Composio to finish processing tokens, improving reliability in token handling.
- Enhanced logging to provide clearer warnings when connection timeouts occur, ensuring better visibility into potential issues.
2026-03-19 19:24:55 +05:30
Anish Sarkar
cc841a5af4 chore: update SearXNG engine settings and example configuration 2026-03-14 21:27:09 +05:30
Anish Sarkar
4376f74127 chore: update SearXNG service configuration in Docker and documentation
- Updated SearXNG image version to 2026.3.13-3c1f68c59 in the Docker Compose file.
- Added port mapping for SearXNG service to allow access on a configurable port.
- Enhanced the .env.example file with comments for SearXNG configuration and usage instructions.
2026-03-14 21:03:25 +05:30
CREDO23
1e2c54eea6 add unified drive-picker-token endpoint and GOOGLE_PICKER_API_KEY config 2026-03-10 20:06:45 +02:00
Anish Sarkar
87711ee381 chore: clean up .env.example and pyproject.toml
- Removed commented-out testing configuration from .env.example to streamline the file.
- Updated markers in pyproject.toml to remove the e2e test marker, clarifying the purpose of the remaining markers.
2026-02-26 23:56:01 +05:30
Anish Sarkar
f59a70f7a5 Merge remote-tracking branch 'upstream/dev' into feat/document-test 2026-02-26 02:22:10 +05:30
DESKTOP-RTLN3BA\$punk
54dd6f904c feat: implement local file persistence for Daytona sandboxes
- Added functionality to persist sandbox files locally before deletion.
- Introduced methods for retrieving and deleting locally stored sandbox files.
- Updated routes to handle local file downloads and background deletion of sandboxes.
- Enhanced streaming tasks to support sandbox file management.
2026-02-25 12:25:59 -08:00
Anish Sarkar
93c0af475b Merge remote-tracking branch 'upstream/dev' into feat/document-test 2026-02-25 20:25:06 +05:30
Anish Sarkar
e1ac8bfc76 feat: Add testing environment variables to surfsense_backend/.env.example and update documentation 2026-02-25 19:52:25 +05:30
DESKTOP-RTLN3BA\$punk
e574a7c38b feat: update .env.example for Daytona sandbox integration
- Removed Uvicorn server configuration options.
- Added new agent-specific configuration for Daytona sandbox, including API key and URL settings.
2026-02-25 01:54:24 -08:00
DESKTOP-RTLN3BA\$punk
17b7348f61 feat: fixed and improved search and background task management. 2026-02-09 14:03:56 -08:00
DESKTOP-RTLN3BA\$punk
1511c26ef5 feat: add residential proxy configuration for web crawling and YouTube transcript fetching 2026-02-05 20:44:13 -08:00
CREDO23
1690ce1891 Document token lifetime env vars in .env.example 2026-02-05 18:59:15 +02:00
DESKTOP-RTLN3BA\$punk
4df40f8cea feat: add CELERY_TASK_DEFAULT_QUEUE environment variable for task isolation 2026-02-02 13:17:12 -08:00
DESKTOP-RTLN3BA\$punk
8c625d4237 feat: composio connector 2026-01-21 22:57:58 -08:00
DESKTOP-RTLN3BA\$punk
4595fd3d1f docs: updated docs 2026-01-20 02:03:05 -08:00
Manoj Aggarwal
fa35b71522 Add teams connector similar to slack 2026-01-09 13:20:30 -08:00
Manoj Aggarwal
786fd63e5b
Revert "Add Microsoft Teams Connector" 2026-01-09 12:33:26 -08:00
Rohan Verma
8a2715e549
Merge pull request #679 from manojag115/feat/ms-teams-connector
Add Microsoft Teams Connector
2026-01-08 20:27:51 -08:00
DESKTOP-RTLN3BA\$punk
c9a9d29c54 docs: update README 2026-01-07 19:43:31 -08:00
DESKTOP-RTLN3BA\$punk
48fc70a08b chore: cleanup 2026-01-07 19:07:06 -08:00
Manoj Aggarwal
73a9dccefc Add teams connector similar to slack 2026-01-07 15:15:49 -08:00
Anish Sarkar
86af9b5a21 feat: implement ClickUp OAuth integration and connector support
- Added ClickUp OAuth authentication flow with new environment variables for client ID, client secret, and redirect URI.
- Introduced ClickUpHistoryConnector to manage OAuth-based authentication and token refresh for ClickUp API access.
- Created ClickUp connector routes for OAuth flow, including authorization and callback handling.
- Updated indexing logic to utilize the new ClickUpHistoryConnector, supporting both OAuth and legacy API token methods.
- Enhanced frontend components to reflect the new ClickUp integration and removed legacy API token forms.
2026-01-07 15:15:25 +05:30
Anish Sarkar
982b9ceb76 feat: implement Jira OAuth integration and connector routes
- Added support for Jira OAuth with new environment variables for client ID, client secret, and redirect URI.
- Implemented Jira connector routes for OAuth flow, including authorization and callback handling.
- Enhanced Jira connector to support both OAuth 2.0 and legacy API token authentication methods.
- Updated Jira indexing logic to utilize OAuth credentials with auto-refresh capabilities.
- Removed outdated Jira UI components and adjusted frontend logic to reflect the new integration.
2026-01-06 01:01:04 +05:30
Anish Sarkar
df23813f1c feat: add Discord OAuth integration and connector routes
- Introduced Discord OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Discord connector routes for OAuth flow, including authorization and callback handling.
- Enhanced Discord connector to support both OAuth-based authentication and legacy bot token usage.
- Updated Discord indexing logic to utilize OAuth credentials with auto-refresh capabilities.
- Removed outdated Discord UI components and adjusted frontend logic to reflect the new integration.
2026-01-05 14:21:39 +05:30
Anish Sarkar
0fe94bfcf3 feat: add Slack OAuth integration and connector routes
- Introduced Slack OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Slack connector routes for OAuth flow, including authorization and callback handling.
- Updated configuration to support both new OAuth format and legacy token handling.
- Enhanced the Slack indexer to decrypt tokens when necessary, ensuring compatibility with existing encrypted credentials.
- Removed outdated Slack connector UI components and adjusted frontend logic to reflect the new integration.
2026-01-04 02:30:00 +05:30
Anish Sarkar
b81af397c0 feat: add Linear OAuth integration and connector routes
- Introduced Linear OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Linear connector routes for OAuth flow, including authorization and callback handling.
- Updated existing components to accommodate Linear integration, including validation changes and connector configuration.
- Enhanced the Linear indexer to utilize OAuth access tokens instead of API keys.
- Adjusted UI components to reflect the new Linear connector without requiring special configuration.
2026-01-02 21:24:28 +05:30
Anish Sarkar
c5b184d475 feat: add Notion OAuth integration and connector routes
- Introduced Notion OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Notion connector routes for OAuth flow, including authorization and callback handling.
- Updated existing components to accommodate Notion integration, including validation changes and connector configuration.
- Enhanced the Notion indexer to utilize OAuth access tokens instead of integration tokens.
- Adjusted UI components to reflect the new Notion connector without requiring special configuration.
2026-01-02 20:07:14 +05:30
DESKTOP-RTLN3BA\$punk
8f37170217 chore: update documentation and environment configuration
- Added GOOGLE_DRIVE_REDIRECT_URI to .env.example for Google Drive connector OAuth.
- Updated docker-installation.mdx and manual-installation.mdx to include GOOGLE_DRIVE_REDIRECT_URI in installation instructions.
2026-01-01 22:50:29 -08:00
CREDO23
c3054809ee chore: update env example 2025-12-29 20:39:36 +02:00
DESKTOP-RTLN3BA\$punk
3f2b915cd0 feat: fixed OAUTH issues
- Added BACKEND_URL to .env.example for optional OAuth callback configuration when behind a reverse proxy with HTTPS.
- Updated fastapi-users dependency to version 15.0.3 in pyproject.toml.
- Updated argon2-cffi and email-validator versions in uv.lock.
- Enhanced Google OAuth integration in app.py to handle secure contexts for CSRF cookies.
- Updated GoogleLoginButton component to include credentials in fetch requests.
- Updated documentation to reflect the new BACKEND_URL variable in both docker and manual installation guides.
2025-12-19 14:19:30 -08:00
DESKTOP-RTLN3BA\$punk
3207fbf9ea feat(fix): add PAGES_LIMIT configuration for ETL services 2025-12-11 00:29:56 -08:00
DESKTOP-RTLN3BA\$punk
0987440893 chore: update configuration for rerankers
- Added RERANKERS_ENABLED option to control reranking functionality.
- Updated rerank_documents function to handle cases when reranking is disabled.
- Enhanced documentation for environment variables related to rerankers in installation guides.
2025-10-29 23:23:08 -07:00
DESKTOP-RTLN3BA\$punk
71e4860495 chore: updated chonkie and temp fix for azure embeddings registry
- TODO: Raise PR in upstream for fix in next version
2025-10-29 14:42:05 -07:00
DESKTOP-RTLN3BA\$punk
aed8163aee feat: added celery beat to docker 2025-10-23 00:56:28 -07:00
MSI\ModSetter
a90767a478 feat: added meta-scheduler for running periodic tasks 2025-10-22 19:49:30 -07:00
Anish Sarkar
19ed0becce feat: implement registration toggle in backend and handle disabled state in frontend 2025-10-20 15:54:52 +05:30
DESKTOP-RTLN3BA\$punk
c80bbfa867 feat: added celery and removed background_tasks for MQ's
- removed pre commit hooks
- updated docker setup
- updated github docker actions
- updated docs
2025-10-20 00:30:00 -07:00
Nabhan
cf0e265107 refactor: integrate local STT with existing upload flow
- Simplify STT_SERVICE config to local/MODEL_SIZE format
- Remove separate STT routes, integrate with document upload
- Add local STT support to audio file processing pipeline
- Remove React component, use existing upload interface
- Support both local Faster-Whisper and external STT services
- Tested with real speech: 99% accuracy, 2.87s processing
2025-10-12 10:50:55 +05:00
Nabhan
dad79674c8 Added local Speech-to-Text (STT) support using Faster-Whisper 2025-10-11 23:56:12 +05:00
CREDO23
27b914f822 Add airtable connector auth flow routes 2025-08-26 13:56:31 +02:00
CREDO23
464d4e3891 update .env.example 2025-08-15 09:11:14 +02:00
DESKTOP-RTLN3BA\$punk
1b29310ae7 feat: Added Local TTS (Kokoro TTS) Support 2025-08-13 17:25:34 -07:00
CREDO23
edf46e4de1 update seach source connector schema 2025-08-03 12:16:40 +02:00
Abdullah 3li
aa00822169 feat: Add Docling support as ETL_SERVICE option
- Added DOCLING as third ETL_SERVICE option (alongside UNSTRUCTURED/LLAMACLOUD)
- Implemented add_received_file_document_using_docling function
- Added Docling processing logic in documents_routes.py
- Enhanced chunking with configurable overlap support
- Added comprehensive document processing service
- Supports both CPU and GPU processing with user selection

Addresses #161 - Add Docling Support as an ETL_SERVICE
Follows same pattern as LlamaCloud integration (PR #123)
2025-07-20 11:42:55 +03:00
Muhamad Aji Wibisono
5a39920e40 chore: removed duplicate .env var 2025-06-10 23:39:13 +07:00
Muhamad Aji Wibisono
d31b9589fb chore: organized env example style 2025-06-10 23:26:28 +07:00
Muhamad Aji Wibisono
03919c88c6 fix: config loading into run 2025-06-10 23:23:11 +07:00
Muhamad Aji Wibisono
505d539fdf refactor: moved uvicorn env loading to ./app/config/uvicorn.py 2025-06-10 23:23:11 +07:00
Muhamad Aji Wibisono
587f9f4c09 feat: added richer uvicorn configuration in .env 2025-06-10 23:23:11 +07:00