- Added encryption for sensitive tokens (access token, refresh token, client secret) in Google Calendar, Google Drive, Gmail, Linear, and Notion connectors to enhance security.
- Introduced OAuthStateManager for secure state parameter generation and validation, improving the integrity of OAuth flows.
- Updated callback routes to handle state validation and error management, ensuring robust handling of authorization processes.
- Enhanced indexers to support decryption of tokens for backward compatibility, maintaining functionality with existing encrypted credentials.
- Improved validation for date parameters in connector routes to ensure proper input handling.
- Updated callback routes to handle optional error parameters for OAuth flows, improving user experience during authorization failures.
- Implemented error logging and redirection logic to provide feedback when access is denied or errors occur.
- Added validation for required parameters to ensure proper handling of authorization codes and state parameters.
- Enhanced documentation in the callback functions to clarify the purpose of each parameter.
- Introduced Linear OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Linear connector routes for OAuth flow, including authorization and callback handling.
- Updated existing components to accommodate Linear integration, including validation changes and connector configuration.
- Enhanced the Linear indexer to utilize OAuth access tokens instead of API keys.
- Adjusted UI components to reflect the new Linear connector without requiring special configuration.
- Introduced Notion OAuth support with new environment variables for client ID, client secret, and redirect URI.
- Implemented Notion connector routes for OAuth flow, including authorization and callback handling.
- Updated existing components to accommodate Notion integration, including validation changes and connector configuration.
- Enhanced the Notion indexer to utilize OAuth access tokens instead of integration tokens.
- Adjusted UI components to reflect the new Notion connector without requiring special configuration.
- Commented out the write_todos tracking and messaging logic in the stream_new_chat.py file.
- Disabled the import and usage of WriteTodosToolUI in the new-chat page component.
- Updated related logic in the active connectors tab to remove indexing state handling for write_todos.
- These changes are part of a temporary disablement of the write_todos feature for further evaluation.
- Added a new endpoint `/auth/google/authorize-redirect` to handle OAuth authorization via server-side redirect, addressing CSRF cookie issues in Firefox/Safari.
- Updated the `GoogleLoginButton` component to use the new redirect endpoint instead of the previous JSON-based authorization method.
- Enhanced CSRF cookie handling by explicitly setting the cookie domain and ensuring compatibility with cross-origin requests.
- Add structured request body with folders and files arrays
- Support individual file indexing alongside folder indexing
- Remove deprecated folder_ids/folder_names query params
- Update UI to allow selecting both folders and files
- Simplify module docstrings (remove meta-commentary about 'small focused modules')
- Remove redundant inline comments (e.g., 'Log task start', 'Get connector from database')
- Trim verbose function docstrings to essential information only
- Remove over-explanatory comments that restate what code does
- Keep necessary documentation, remove noise for better readability
- Store tokens in folder_tokens dict instead of single global token
- Each folder now tracks its own sync state independently
- Fixes issue where indexing folder 2 incorrectly used delta sync after folder 1 was indexed
- First-time indexing now correctly uses full scan for each new folder
- Add optional 'connector' parameter with 'type' and 'metadata' fields
- Create helper function _update_document_from_connector
- Use document_metadata column (not metadata) for JSON field
- Merge metadata with existing using dict spread operator
- Google Drive documents now marked as GOOGLE_DRIVE_CONNECTOR
- Backward compatible - no changes to existing logic
- Simple and clean implementation
- Query document from database to ensure it's attached to session
- Prevents detached instance errors after process_file_in_background commits
- Properly updates document_type and metadata with session management
- Return file metadata from content_extractor for indexer to use
- Update document type and metadata in indexer after processing
- Explicitly commit changes to database
- Ensures documents are properly marked as GOOGLE_DRIVE_CONNECTOR type
- Change document_type from file type (PDF, DOCX) to GOOGLE_DRIVE_CONNECTOR
- Store original file type in metadata for reference
- Add Google Drive specific metadata (file_id, mime_type, source)
- Include export format info for Google Workspace files
- Enables proper source tracking and bulk management
- Accept comma-separated folder_ids and folder_names
- Loop through each folder and index sequentially
- Collect total indexed count and errors
- Update timestamp only on full success
- Accept folder_id and folder_name as indexing parameters
- Hide date range for Google Drive connectors
- Create wrapper function to avoid circular imports
- Trigger Google Drive indexing Celery task
- Full folder scan on first index
- Delta sync using change tracking for subsequent indexes
- Process files in parallel batches
- Handle file additions, modifications, and deletions
- Store change tracking token for efficient re-indexing
- OAuth initialization and callback handling
- Folder and file browsing with parent_id support
- Validate credentials and handle token refresh
- Return folder contents with metadata for UI tree view
- Get start page token for change tracking baseline
- Fetch incremental changes using Google Drive Changes API
- Categorize changes into added, modified, and removed files
- Enable efficient re-indexing of only changed content
- List folder contents with full pagination support
- Query root folder or specific parent folder
- Return both folders and files with metadata (size, icons, links)
- Filter out shortcuts and trashed items
- Download files from Google Drive to temporary location
- Export Google Workspace files as PDF
- Delegate content extraction to existing process_file_in_background
- Reuse Surfsense's ETL services (Unstructured, LlamaCloud, Docling)
- Detect Google Workspace files (Docs, Sheets, Slides)
- Map to PDF export format to preserve rich content (images, formatting)
- Identify files to skip (shortcuts, unsupported types)
- Build and manage Google Drive service with credentials
- List files with query support and pagination
- Download binary files and export Google Workspace files as PDF
- Handle HTTP errors gracefully
