Commit graph

82 commits

Author SHA1 Message Date
Rohan Verma
383592ce63
Merge pull request #680 from AnishSarkar22/fix/index-future-date
feat: Index future dates for calendar based connectors & bug fixes
2026-01-10 14:00:29 -08:00
Manoj Aggarwal
8b735a492a lint 2026-01-09 13:53:09 -08:00
Manoj Aggarwal
62d0d8b6db ruff lint 2026-01-09 13:38:49 -08:00
Manoj Aggarwal
18035b3728 Add MS Teams connector 2026-01-09 13:20:47 -08:00
Manoj Aggarwal
fa35b71522 Add teams connector similar to slack 2026-01-09 13:20:30 -08:00
Manoj Aggarwal
786fd63e5b
Revert "Add Microsoft Teams Connector" 2026-01-09 12:33:26 -08:00
Anish Sarkar
e21bc8086a Merge remote-tracking branch 'upstream/dev' into fix/index-future-date 2026-01-09 13:24:38 +05:30
Anish Sarkar
4aeb05e2e5 feat: allow future dates for Google Calendar and Luma connectors
- Updated date handling in indexing functions to permit future dates for Google Calendar and Luma connectors.
- Enhanced UI components to support future date selection, including a new button for selecting the next 30 days.
- Adjusted documentation and descriptions to clarify date range options for users.
2026-01-09 13:20:12 +05:30
Manoj Aggarwal
ba7e4f0ceb Add MS Teams connector 2026-01-08 17:13:19 -08:00
Manoj Aggarwal
73a9dccefc Add teams connector similar to slack 2026-01-07 15:15:49 -08:00
Anish Sarkar
86af9b5a21 feat: implement ClickUp OAuth integration and connector support
- Added ClickUp OAuth authentication flow with new environment variables for client ID, client secret, and redirect URI.
- Introduced ClickUpHistoryConnector to manage OAuth-based authentication and token refresh for ClickUp API access.
- Created ClickUp connector routes for OAuth flow, including authorization and callback handling.
- Updated indexing logic to utilize the new ClickUpHistoryConnector, supporting both OAuth and legacy API token methods.
- Enhanced frontend components to reflect the new ClickUp integration and removed legacy API token forms.
2026-01-07 15:15:25 +05:30
Anish Sarkar
f2724ea162 feat: enhance Airtable integration with OAuth support and date validation
- Introduced AirtableHistoryConnector to manage OAuth-based authentication and token refresh for Airtable API access.
- Added date string validation in AirtableConnector to ensure valid date inputs before processing.
- Updated indexing logic to utilize the new AirtableHistoryConnector, improving credential management and token handling.
2026-01-07 03:00:56 +05:30
Anish Sarkar
c7c5caf559 chore: ran both frontend and backend linting 2026-01-06 15:49:31 +05:30
Anish Sarkar
3dc04f906d Merge remote-tracking branch 'upstream/dev' into feat/atlassian-oauth 2026-01-06 15:05:14 +05:30
Anish Sarkar
0f5bf93f68 feat: implement JiraHistoryConnector for OAuth and legacy authentication
- Introduced JiraHistoryConnector to handle OAuth-based authentication and automatic token refresh for Jira API access.
- Refactored Jira indexing logic to utilize the new connector, simplifying credential management and enhancing token refresh capabilities.
- Removed legacy token handling code from the Jira indexer, streamlining the integration process.
- Ensured compatibility with both OAuth 2.0 and legacy API token methods for improved flexibility.
2026-01-06 14:36:51 +05:30
Anish Sarkar
5d363b8a60 feat: implement Confluence OAuth integration and connector routes
- Added support for Confluence OAuth with new environment variables for client ID, client secret, and redirect URI.
- Implemented Confluence connector routes for OAuth flow, including authorization and callback handling.
- Enhanced Confluence connector to support both OAuth 2.0 and legacy API token authentication methods.
- Updated Confluence indexing logic to utilize OAuth credentials with auto-refresh capabilities.
- Removed outdated Confluence UI components and adjusted frontend logic to reflect the new integration.
2026-01-06 13:20:22 +05:30
DESKTOP-RTLN3BA\$punk
aac0432023 refactor: update Discord message indexing logic
- Enhanced the indexing process for Discord messages to treat each message as an individual document, improving metadata handling and content management.
- Replaced the announcement banner component and related state management with a more streamlined approach, removing unnecessary files and simplifying the dashboard layout.
- Updated logging messages for clarity and accuracy regarding processed messages.
2026-01-05 22:18:25 -08:00
Anish Sarkar
bf8c3bfcf7 feat: add Atlassian OAuth support for Jira and Confluence
- Introduced a shared schema for Atlassian OAuth 2.0 credentials, accommodating both Jira and Confluence.
- Updated Jira connector routes to utilize the new AtlassianAuthCredentialsBase for handling OAuth tokens.
- Enhanced configuration to include new environment variables for Jira OAuth integration.
- Refactored token handling in Jira indexing logic to support the new shared credential structure.
2026-01-06 01:27:29 +05:30
Anish Sarkar
982b9ceb76 feat: implement Jira OAuth integration and connector routes
- Added support for Jira OAuth with new environment variables for client ID, client secret, and redirect URI.
- Implemented Jira connector routes for OAuth flow, including authorization and callback handling.
- Enhanced Jira connector to support both OAuth 2.0 and legacy API token authentication methods.
- Updated Jira indexing logic to utilize OAuth credentials with auto-refresh capabilities.
- Removed outdated Jira UI components and adjusted frontend logic to reflect the new integration.
2026-01-06 01:01:04 +05:30
Anish Sarkar
f236110a08 Revert "feat: implement Jira OAuth integration and connector routes"
This reverts commit bfed9a31f8.
2026-01-06 00:09:08 +05:30
Anish Sarkar
bfed9a31f8 feat: implement Jira OAuth integration and connector routes
- Added support for Jira OAuth with new environment variables for client ID, client secret, and redirect URI.
- Implemented Jira connector routes for OAuth flow, including authorization and callback handling.
- Enhanced Jira connector to support both OAuth 2.0 and legacy API token authentication.
- Updated Jira indexing logic to utilize OAuth credentials with auto-refresh capabilities.
- Removed outdated Jira UI components and adjusted frontend logic to reflect the new integration.
2026-01-05 23:59:16 +05:30
Anish Sarkar
df23813f1c feat: add Discord OAuth integration and connector routes
- Introduced Discord OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Discord connector routes for OAuth flow, including authorization and callback handling.
- Enhanced Discord connector to support both OAuth-based authentication and legacy bot token usage.
- Updated Discord indexing logic to utilize OAuth credentials with auto-refresh capabilities.
- Removed outdated Discord UI components and adjusted frontend logic to reflect the new integration.
2026-01-05 14:21:39 +05:30
Anish Sarkar
1862732913 refactor: improve error handling for Slack token refresh logic
- Updated SlackHistory class to enforce the use of session and connector_id for token refresh, raising a ValueError for legacy token usage.
- Simplified conditional checks for client initialization in SlackHistory.
- Cleaned up unnecessary comments and whitespace in the codebase.
2026-01-04 02:47:50 +05:30
Anish Sarkar
81e4a4ada0 feat: database driven refresh tokens for slack oauth connector 2026-01-04 02:38:19 +05:30
Anish Sarkar
0fe94bfcf3 feat: add Slack OAuth integration and connector routes
- Introduced Slack OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Slack connector routes for OAuth flow, including authorization and callback handling.
- Updated configuration to support both new OAuth format and legacy token handling.
- Enhanced the Slack indexer to decrypt tokens when necessary, ensuring compatibility with existing encrypted credentials.
- Removed outdated Slack connector UI components and adjusted frontend logic to reflect the new integration.
2026-01-04 02:30:00 +05:30
Anish Sarkar
4f77d171d8 feat: implement auto-refresh capability for Linear and Notion connectors similar to google oauth based ones
- Enhanced LinearConnector and NotionHistoryConnector classes to support automatic token refresh, improving reliability in accessing APIs.
- Updated initialization to require session and connector ID, allowing for dynamic credential management.
- Introduced new credential schemas for Linear and Notion, encapsulating access and refresh tokens with expiration handling.
- Refactored indexers to utilize the new connector structure, ensuring seamless integration with the updated authentication flow.
- Improved error handling and logging during token refresh processes for better debugging and user feedback.
2026-01-03 04:24:00 +05:30
Anish Sarkar
ed995b0341 refactor: update token decryption handling for connectors
- Enhanced token decryption logic in Airtable, Google Drive, Linear, and Notion indexers to only attempt decryption when tokens are explicitly marked as encrypted.
- Added error handling for missing SECRET_KEY when tokens are marked as encrypted, improving robustness and clarity in error reporting.
- Updated comments to clarify the handling of plaintext tokens when encryption is not indicated.
2026-01-03 03:43:40 +05:30
Anish Sarkar
645e849d93 chore: ran both frontend and backend linting 2026-01-03 00:18:17 +05:30
Anish Sarkar
45489423d1 feat: implement token encryption and state management for OAuth connectors
- Added encryption for sensitive tokens (access token, refresh token, client secret) in Google Calendar, Google Drive, Gmail, Linear, and Notion connectors to enhance security.
- Introduced OAuthStateManager for secure state parameter generation and validation, improving the integrity of OAuth flows.
- Updated callback routes to handle state validation and error management, ensuring robust handling of authorization processes.
- Enhanced indexers to support decryption of tokens for backward compatibility, maintaining functionality with existing encrypted credentials.
- Improved validation for date parameters in connector routes to ensure proper input handling.
2026-01-02 23:46:03 +05:30
Anish Sarkar
b81af397c0 feat: add Linear OAuth integration and connector routes
- Introduced Linear OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Linear connector routes for OAuth flow, including authorization and callback handling.
- Updated existing components to accommodate Linear integration, including validation changes and connector configuration.
- Enhanced the Linear indexer to utilize OAuth access tokens instead of API keys.
- Adjusted UI components to reflect the new Linear connector without requiring special configuration.
2026-01-02 21:24:28 +05:30
Anish Sarkar
c5b184d475 feat: add Notion OAuth integration and connector routes
- Introduced Notion OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Notion connector routes for OAuth flow, including authorization and callback handling.
- Updated existing components to accommodate Notion integration, including validation changes and connector configuration.
- Enhanced the Notion indexer to utilize OAuth access tokens instead of integration tokens.
- Adjusted UI components to reflect the new Notion connector without requiring special configuration.
2026-01-02 20:07:14 +05:30
DESKTOP-RTLN3BA\$punk
2832d57bda chore: linting 2026-01-01 22:56:37 -08:00
CREDO23
9c78726b6b feat: add file selection to Google Drive connector
- Add structured request body with folders and files arrays
- Support individual file indexing alongside folder indexing
- Remove deprecated folder_ids/folder_names query params
- Update UI to allow selecting both folders and files
2025-12-31 14:15:07 +02:00
DESKTOP-RTLN3BA\$punk
c19d300c9d feat: added circleback connector 2025-12-30 09:00:59 -08:00
CREDO23
7618662e70 refactor: rename GOOGLE_DRIVE_CONNECTOR to GOOGLE_DRIVE_FILE document type 2025-12-29 20:38:26 +02:00
CREDO23
acf47e3b0c refactor(connectors): remove verbose docstrings and obvious comments
- Simplify module docstrings (remove meta-commentary about 'small focused modules')
- Remove redundant inline comments (e.g., 'Log task start', 'Get connector from database')
- Trim verbose function docstrings to essential information only
- Remove over-explanatory comments that restate what code does
- Keep necessary documentation, remove noise for better readability
2025-12-28 18:53:13 +02:00
CREDO23
506a9297a9 fix(connectors): track delta sync tokens per folder for Google Drive
- Store tokens in folder_tokens dict instead of single global token
- Each folder now tracks its own sync state independently
- Fixes issue where indexing folder 2 incorrectly used delta sync after folder 1 was indexed
- First-time indexing now correctly uses full scan for each new folder
2025-12-28 18:32:59 +02:00
CREDO23
a5935bc677 feat(connectors): add connector parameter to file processor for source tracking
- Add optional 'connector' parameter with 'type' and 'metadata' fields
- Create helper function _update_document_from_connector
- Use document_metadata column (not metadata) for JSON field
- Merge metadata with existing using dict spread operator
- Google Drive documents now marked as GOOGLE_DRIVE_CONNECTOR
- Backward compatible - no changes to existing logic
- Simple and clean implementation
2025-12-28 18:01:39 +02:00
CREDO23
8da58be9e0 fix(connectors): refresh document from DB before updating type
- Query document from database to ensure it's attached to session
- Prevents detached instance errors after process_file_in_background commits
- Properly updates document_type and metadata with session management
2025-12-28 17:21:44 +02:00
CREDO23
b2b891e4d7 fix(connectors): properly commit Google Drive document type changes
- Return file metadata from content_extractor for indexer to use
- Update document type and metadata in indexer after processing
- Explicitly commit changes to database
- Ensures documents are properly marked as GOOGLE_DRIVE_CONNECTOR type
2025-12-28 17:15:29 +02:00
CREDO23
7b8900d51f feat(indexer): export Google Drive indexer function 2025-12-28 15:55:46 +02:00
CREDO23
1696c7056a feat(indexer): add Google Drive folder indexing with delta sync
- Full folder scan on first index
- Delta sync using change tracking for subsequent indexes
- Process files in parallel batches
- Handle file additions, modifications, and deletions
- Store change tracking token for efficient re-indexing
2025-12-28 15:55:25 +02:00
CREDO23
c6cb754aac refactor: update the webcrawler index to compare hashes without metadata 2025-12-17 18:44:58 +02:00
DESKTOP-RTLN3BA\$punk
8c9aa68faa feat: update document tracking to use 'updated_at' timestamp instead of 'last_edited_at' 2025-12-12 01:32:14 -08:00
Differ
500bc60d02 fix: add input validation, retry limit, code formatting, and exclude i18n from secret detection 2025-12-05 09:58:49 +08:00
Differ
6b1b8d0f2e feat: add BookStack connector for wiki documentation indexing 2025-12-04 14:08:44 +08:00
DESKTOP-RTLN3BA\$punk
ab6ea7e0ab feat(UI): reorganized connectors 2025-11-26 13:44:38 -08:00
DESKTOP-RTLN3BA\$punk
8f30cfd69a chore(lint): ruff checks 2025-11-26 13:22:31 -08:00
samkul-swe
121e2f0c0e Renaming resources 2025-11-22 19:19:00 -08:00
samkul-swe
896e410e2a Webcrawler connector draft 2025-11-21 23:27:21 -08:00