Commit graph

557 commits

Author SHA1 Message Date
Anish Sarkar
c7c5caf559 chore: ran both frontend and backend linting 2026-01-06 15:49:31 +05:30
Anish Sarkar
6ea6e752f6 refactor: update OAuth scopes and improve connector UI messages
- Removed unnecessary OAuth scopes for Confluence and Jira connectors to streamline authentication requirements.
- Updated the Confluence and Jira configuration components to enhance user experience by clarifying connection update instructions and improving styling for displayed URLs.
2026-01-06 15:44:43 +05:30
Anish Sarkar
3dc04f906d Merge remote-tracking branch 'upstream/dev' into feat/atlassian-oauth 2026-01-06 15:05:14 +05:30
Anish Sarkar
c7fa640594 feat: enhance ConfluenceHistoryConnector to support legacy API token authentication
- Added support for legacy API token authentication alongside OAuth 2.0 in ConfluenceHistoryConnector.
- Implemented logic to handle both authentication methods, ensuring backward compatibility.
- Refactored token management to accommodate legacy credentials and updated API request handling accordingly.
- Enhanced error handling for credential validation and improved logging for better traceability.
2026-01-06 15:04:58 +05:30
Anish Sarkar
0f5bf93f68 feat: implement JiraHistoryConnector for OAuth and legacy authentication
- Introduced JiraHistoryConnector to handle OAuth-based authentication and automatic token refresh for Jira API access.
- Refactored Jira indexing logic to utilize the new connector, simplifying credential management and enhancing token refresh capabilities.
- Removed legacy token handling code from the Jira indexer, streamlining the integration process.
- Ensured compatibility with both OAuth 2.0 and legacy API token methods for improved flexibility.
2026-01-06 14:36:51 +05:30
Anish Sarkar
5d363b8a60 feat: implement Confluence OAuth integration and connector routes
- Added support for Confluence OAuth with new environment variables for client ID, client secret, and redirect URI.
- Implemented Confluence connector routes for OAuth flow, including authorization and callback handling.
- Enhanced Confluence connector to support both OAuth 2.0 and legacy API token authentication methods.
- Updated Confluence indexing logic to utilize OAuth credentials with auto-refresh capabilities.
- Removed outdated Confluence UI components and adjusted frontend logic to reflect the new integration.
2026-01-06 13:20:22 +05:30
DESKTOP-RTLN3BA\$punk
aac0432023 refactor: update Discord message indexing logic
- Enhanced the indexing process for Discord messages to treat each message as an individual document, improving metadata handling and content management.
- Replaced the announcement banner component and related state management with a more streamlined approach, removing unnecessary files and simplifying the dashboard layout.
- Updated logging messages for clarity and accuracy regarding processed messages.
2026-01-05 22:18:25 -08:00
Anish Sarkar
bf8c3bfcf7 feat: add Atlassian OAuth support for Jira and Confluence
- Introduced a shared schema for Atlassian OAuth 2.0 credentials, accommodating both Jira and Confluence.
- Updated Jira connector routes to utilize the new AtlassianAuthCredentialsBase for handling OAuth tokens.
- Enhanced configuration to include new environment variables for Jira OAuth integration.
- Refactored token handling in Jira indexing logic to support the new shared credential structure.
2026-01-06 01:27:29 +05:30
Anish Sarkar
982b9ceb76 feat: implement Jira OAuth integration and connector routes
- Added support for Jira OAuth with new environment variables for client ID, client secret, and redirect URI.
- Implemented Jira connector routes for OAuth flow, including authorization and callback handling.
- Enhanced Jira connector to support both OAuth 2.0 and legacy API token authentication methods.
- Updated Jira indexing logic to utilize OAuth credentials with auto-refresh capabilities.
- Removed outdated Jira UI components and adjusted frontend logic to reflect the new integration.
2026-01-06 01:01:04 +05:30
Anish Sarkar
f236110a08 Revert "feat: implement Jira OAuth integration and connector routes"
This reverts commit bfed9a31f8.
2026-01-06 00:09:08 +05:30
Anish Sarkar
bfed9a31f8 feat: implement Jira OAuth integration and connector routes
- Added support for Jira OAuth with new environment variables for client ID, client secret, and redirect URI.
- Implemented Jira connector routes for OAuth flow, including authorization and callback handling.
- Enhanced Jira connector to support both OAuth 2.0 and legacy API token authentication.
- Updated Jira indexing logic to utilize OAuth credentials with auto-refresh capabilities.
- Removed outdated Jira UI components and adjusted frontend logic to reflect the new integration.
2026-01-05 23:59:16 +05:30
Anish Sarkar
df23813f1c feat: add Discord OAuth integration and connector routes
- Introduced Discord OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Discord connector routes for OAuth flow, including authorization and callback handling.
- Enhanced Discord connector to support both OAuth-based authentication and legacy bot token usage.
- Updated Discord indexing logic to utilize OAuth credentials with auto-refresh capabilities.
- Removed outdated Discord UI components and adjusted frontend logic to reflect the new integration.
2026-01-05 14:21:39 +05:30
Anish Sarkar
1862732913 refactor: improve error handling for Slack token refresh logic
- Updated SlackHistory class to enforce the use of session and connector_id for token refresh, raising a ValueError for legacy token usage.
- Simplified conditional checks for client initialization in SlackHistory.
- Cleaned up unnecessary comments and whitespace in the codebase.
2026-01-04 02:47:50 +05:30
Anish Sarkar
81e4a4ada0 feat: database driven refresh tokens for slack oauth connector 2026-01-04 02:38:19 +05:30
Anish Sarkar
0fe94bfcf3 feat: add Slack OAuth integration and connector routes
- Introduced Slack OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Slack connector routes for OAuth flow, including authorization and callback handling.
- Updated configuration to support both new OAuth format and legacy token handling.
- Enhanced the Slack indexer to decrypt tokens when necessary, ensuring compatibility with existing encrypted credentials.
- Removed outdated Slack connector UI components and adjusted frontend logic to reflect the new integration.
2026-01-04 02:30:00 +05:30
Anish Sarkar
431ea44b56 feat: enhance periodic indexing configuration with detailed validation and UI options
- Updated the SearchSourceConnectorBase class to include detailed documentation on supported periodic indexing frequencies.
- Added "Every 5 minutes" option to the frequency selection in multiple connector forms (BookStack, ClickUp, Confluence, Discord, Elasticsearch, Github, Jira, Luma, Slack) to improve user experience and flexibility in scheduling.
2026-01-03 23:06:09 +05:30
Anish Sarkar
4f77d171d8 feat: implement auto-refresh capability for Linear and Notion connectors similar to google oauth based ones
- Enhanced LinearConnector and NotionHistoryConnector classes to support automatic token refresh, improving reliability in accessing APIs.
- Updated initialization to require session and connector ID, allowing for dynamic credential management.
- Introduced new credential schemas for Linear and Notion, encapsulating access and refresh tokens with expiration handling.
- Refactored indexers to utilize the new connector structure, ensuring seamless integration with the updated authentication flow.
- Improved error handling and logging during token refresh processes for better debugging and user feedback.
2026-01-03 04:24:00 +05:30
Anish Sarkar
1e30bc6484 docs: clarify token parameter descriptions in Linear and Notion connectors
- Updated documentation for the LinearConnector and NotionHistoryConnector classes to specify that the access token can also be an API key or integration token, enhancing clarity for users regarding token usage.
2026-01-03 03:48:46 +05:30
Anish Sarkar
ed995b0341 refactor: update token decryption handling for connectors
- Enhanced token decryption logic in Airtable, Google Drive, Linear, and Notion indexers to only attempt decryption when tokens are explicitly marked as encrypted.
- Added error handling for missing SECRET_KEY when tokens are marked as encrypted, improving robustness and clarity in error reporting.
- Updated comments to clarify the handling of plaintext tokens when encryption is not indicated.
2026-01-03 03:43:40 +05:30
Anish Sarkar
645e849d93 chore: ran both frontend and backend linting 2026-01-03 00:18:17 +05:30
Anish Sarkar
45489423d1 feat: implement token encryption and state management for OAuth connectors
- Added encryption for sensitive tokens (access token, refresh token, client secret) in Google Calendar, Google Drive, Gmail, Linear, and Notion connectors to enhance security.
- Introduced OAuthStateManager for secure state parameter generation and validation, improving the integrity of OAuth flows.
- Updated callback routes to handle state validation and error management, ensuring robust handling of authorization processes.
- Enhanced indexers to support decryption of tokens for backward compatibility, maintaining functionality with existing encrypted credentials.
- Improved validation for date parameters in connector routes to ensure proper input handling.
2026-01-02 23:46:03 +05:30
Anish Sarkar
ec7599362d feat: enhance OAuth callback handling for Airtable, Linear, and Notion connectors if user deny access
- Updated callback routes to handle optional error parameters for OAuth flows, improving user experience during authorization failures.
- Implemented error logging and redirection logic to provide feedback when access is denied or errors occur.
- Added validation for required parameters to ensure proper handling of authorization codes and state parameters.
- Enhanced documentation in the callback functions to clarify the purpose of each parameter.
2026-01-02 21:44:50 +05:30
Anish Sarkar
b81af397c0 feat: add Linear OAuth integration and connector routes
- Introduced Linear OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Linear connector routes for OAuth flow, including authorization and callback handling.
- Updated existing components to accommodate Linear integration, including validation changes and connector configuration.
- Enhanced the Linear indexer to utilize OAuth access tokens instead of API keys.
- Adjusted UI components to reflect the new Linear connector without requiring special configuration.
2026-01-02 21:24:28 +05:30
Anish Sarkar
c5b184d475 feat: add Notion OAuth integration and connector routes
- Introduced Notion OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Notion connector routes for OAuth flow, including authorization and callback handling.
- Updated existing components to accommodate Notion integration, including validation changes and connector configuration.
- Enhanced the Notion indexer to utilize OAuth access tokens instead of integration tokens.
- Adjusted UI components to reflect the new Notion connector without requiring special configuration.
2026-01-02 20:07:14 +05:30
Anish Sarkar
2b01120c2b refactor: disable write_todos functionality across chat and UI components
- Commented out the write_todos tracking and messaging logic in the stream_new_chat.py file.
- Disabled the import and usage of WriteTodosToolUI in the new-chat page component.
- Updated related logic in the active connectors tab to remove indexing state handling for write_todos.
- These changes are part of a temporary disablement of the write_todos feature for further evaluation.
2026-01-02 18:22:38 +05:30
DESKTOP-RTLN3BA\$punk
9029f5c621 feat: implement redirect-based OAuth authorization for Google login
- Added a new endpoint `/auth/google/authorize-redirect` to handle OAuth authorization via server-side redirect, addressing CSRF cookie issues in Firefox/Safari.
- Updated the `GoogleLoginButton` component to use the new redirect endpoint instead of the previous JSON-based authorization method.
- Enhanced CSRF cookie handling by explicitly setting the cookie domain and ensuring compatibility with cross-origin requests.
2026-01-02 00:25:02 -08:00
DESKTOP-RTLN3BA\$punk
2832d57bda chore: linting 2026-01-01 22:56:37 -08:00
DESKTOP-RTLN3BA\$punk
eaa21c7150 feat: version bump to v0.0.10 2026-01-01 22:53:22 -08:00
DESKTOP-RTLN3BA\$punk
8f37170217 chore: update documentation and environment configuration
- Added GOOGLE_DRIVE_REDIRECT_URI to .env.example for Google Drive connector OAuth.
- Updated docker-installation.mdx and manual-installation.mdx to include GOOGLE_DRIVE_REDIRECT_URI in installation instructions.
2026-01-01 22:50:29 -08:00
Anish Sarkar
fda682c9e7 Merge remote-tracking branch 'upstream/dev' into fix/connector 2026-01-01 02:03:02 +05:30
CREDO23
9c78726b6b feat: add file selection to Google Drive connector
- Add structured request body with folders and files arrays
- Support individual file indexing alongside folder indexing
- Remove deprecated folder_ids/folder_names query params
- Update UI to allow selecting both folders and files
2025-12-31 14:15:07 +02:00
Anish Sarkar
d21e800740 Merge remote-tracking branch 'upstream/dev' into fix/connector 2025-12-31 17:26:10 +05:30
Anish Sarkar
163df8fda7 feat: Enhance connector management UI with improved loading states, add document count display for connectors, and implement indexing progress indicators for better user feedback. 2025-12-31 17:21:26 +05:30
Anish Sarkar
ddfbb9509b feat: Implement connector editing functionality in the popup, including Google Drive folder selection, and enhance connector management with improved state handling and UI updates. 2025-12-31 02:00:11 +05:30
DESKTOP-RTLN3BA\$punk
476c764611 feat: fix Circleback connector and update related enums 2025-12-30 12:13:18 -08:00
DESKTOP-RTLN3BA\$punk
c19d300c9d feat: added circleback connector 2025-12-30 09:00:59 -08:00
Anish Sarkar
2898192ac4 feat: Update redirect URLs in connector routes to include success parameters and improve indexing configuration handling in the connector popup. 2025-12-30 19:49:28 +05:30
Rohan Verma
23870042f3
Merge pull request #640 from CREDO23/google-drive-connector
[Feat] Add Google drive connector
2025-12-29 20:51:34 -08:00
CREDO23
c3054809ee chore: update env example 2025-12-29 20:39:36 +02:00
CREDO23
7618662e70 refactor: rename GOOGLE_DRIVE_CONNECTOR to GOOGLE_DRIVE_FILE document type 2025-12-29 20:38:26 +02:00
CREDO23
16bc991b13 feat: add Google Drive connector to knowledge base search 2025-12-29 18:13:27 +02:00
DESKTOP-RTLN3BA\$punk
35904ba0c8 Merge branch 'dev' of https://github.com/MODSetter/SurfSense into dev 2025-12-28 15:54:10 -08:00
DESKTOP-RTLN3BA\$punk
46eb63fa11 hotpatch: remove consitent 2 sec interval api call to logs summary endpoint 2025-12-28 15:52:58 -08:00
Anish Sarkar
7a852f0ea5 feat: Enhance todo status normalization and update schema for improved task management 2025-12-29 01:08:18 +05:30
CREDO23
27beac4f62 fix: Google Drive folder handling and connector page updates 2025-12-28 19:57:10 +02:00
CREDO23
c5c61a2c6b Merge branch 'dev' into google-drive-connector
Merge in dev
2025-12-28 19:00:09 +02:00
CREDO23
acf47e3b0c refactor(connectors): remove verbose docstrings and obvious comments
- Simplify module docstrings (remove meta-commentary about 'small focused modules')
- Remove redundant inline comments (e.g., 'Log task start', 'Get connector from database')
- Trim verbose function docstrings to essential information only
- Remove over-explanatory comments that restate what code does
- Keep necessary documentation, remove noise for better readability
2025-12-28 18:53:13 +02:00
CREDO23
506a9297a9 fix(connectors): track delta sync tokens per folder for Google Drive
- Store tokens in folder_tokens dict instead of single global token
- Each folder now tracks its own sync state independently
- Fixes issue where indexing folder 2 incorrectly used delta sync after folder 1 was indexed
- First-time indexing now correctly uses full scan for each new folder
2025-12-28 18:32:59 +02:00
CREDO23
a5935bc677 feat(connectors): add connector parameter to file processor for source tracking
- Add optional 'connector' parameter with 'type' and 'metadata' fields
- Create helper function _update_document_from_connector
- Use document_metadata column (not metadata) for JSON field
- Merge metadata with existing using dict spread operator
- Google Drive documents now marked as GOOGLE_DRIVE_CONNECTOR
- Backward compatible - no changes to existing logic
- Simple and clean implementation
2025-12-28 18:01:39 +02:00
CREDO23
8da58be9e0 fix(connectors): refresh document from DB before updating type
- Query document from database to ensure it's attached to session
- Prevents detached instance errors after process_file_in_background commits
- Properly updates document_type and metadata with session management
2025-12-28 17:21:44 +02:00