Anish Sarkar
e7beeb2a36
refactor: unify file skipping logic across Dropbox, Google Drive, and OneDrive connectors by replacing classification checks with a centralized service-based approach, enhancing maintainability and consistency in file handling
2026-04-07 02:19:31 +05:30
Anish Sarkar
f03bf05aaa
refactor: enhance Google Drive indexer to support file extension filtering, improving file handling and error reporting
2026-04-06 22:34:49 +05:30
Anish Sarkar
0fb92b7c56
refactor: streamline file skipping logic in Dropbox indexer by removing redundant checks, improving code clarity
2026-04-06 22:17:50 +05:30
Anish Sarkar
63a75052ca
Merge remote-tracking branch 'upstream/dev' into feat/unified-etl-pipeline
2026-04-06 22:04:51 +05:30
Anish Sarkar
dc7047f64d
refactor: implement file type classification for supported extensions across Dropbox, Google Drive, and OneDrive connectors, enhancing file handling and error management
2026-04-06 22:03:47 +05:30
Anish Sarkar
47f4be08d9
refactor: remove allowed_formats from DocumentConverter initialization in DoclingService to allow acceptance of all supported formats
2026-04-06 19:31:42 +05:30
Anish Sarkar
caca491774
test: add unit tests for Dropbox integration, covering delta sync methods, file type filtering, and re-authentication behavior
2026-04-06 18:36:48 +05:30
Anish Sarkar
b5a15b7681
feat: implement cursor-based delta sync for Dropbox integration, enhancing file indexing efficiency and preserving folder cursors during re-authentication
2026-04-06 18:36:29 +05:30
Rohan Verma
74bf3df880
Merge pull request #1130 from CREDO23/feat/vision-autocomplete
...
[Feat] Vision-based autocomplete with KB grounding
2026-04-05 13:24:55 -07:00
Anish Sarkar
f8913adaa3
test: add unit tests for content extraction from cloud connectors and ETL pipeline functionality
2026-04-05 17:46:04 +05:30
Anish Sarkar
87af012a60
refactor: streamline file processing by integrating ETL pipeline for all file types and removing redundant functions
2026-04-05 17:45:18 +05:30
Anish Sarkar
8224360afa
refactor: unify file parsing logic across Dropbox, Google Drive, and OneDrive using the ETL pipeline
2026-04-05 17:30:29 +05:30
Anish Sarkar
1248363ca9
refactor: consolidate document processing logic and remove unused files and ETL strategies
2026-04-05 17:29:24 +05:30
Anish Sarkar
f40de6b695
feat: add parsers for Docling, LlamaCloud, and Unstructured to ETL pipeline
2026-04-05 17:27:24 +05:30
Anish Sarkar
2824410be2
feat: add plaintext parser to ETL pipeline for reading text files
2026-04-05 17:26:42 +05:30
Anish Sarkar
35582c9389
feat: add direct_convert module to ETL pipeline for file conversion
2026-04-05 17:26:29 +05:30
Anish Sarkar
02fc6f1d16
feat: add audio transcription functionality to ETL pipeline
2026-04-05 17:26:03 +05:30
Anish Sarkar
5d22349dc1
feat: implement ETL pipeline with file classification and extraction services
2026-04-05 17:25:25 +05:30
CREDO23
a180bf5576
fix merge conflicts
2026-04-04 09:39:20 +02:00
CREDO23
d7315e7f27
Merge upstream/dev into feat/vision-autocomplete
2026-04-04 09:15:13 +02:00
CREDO23
18103417bb
fix: harden autocomplete endpoint security and error handling
2026-04-04 08:57:09 +02:00
CREDO23
46e8134b23
catch non-vision model errors with actionable user message
2026-04-04 08:40:55 +02:00
Anish Sarkar
82d4d3e272
chore: ran linting
2026-04-04 03:37:33 +05:30
Anish Sarkar
e814540727
refactor: move PKCE pair generatio for airtable
...
- Removed the `generate_pkce_pair` function from `airtable_add_connector_route.py` and relocated it to `oauth_security.py` for better organization.
- Updated imports in `airtable_add_connector_route.py` to reflect the new location of the PKCE generation function.
2026-04-04 03:36:54 +05:30
Anish Sarkar
8e6b1c77ea
feat: implement PKCE support in native Google OAuth flows
...
- Added `generate_code_verifier` function to create a PKCE code verifier for enhanced security.
- Updated Google Calendar, Drive, and Gmail connector routes to utilize the PKCE code verifier during OAuth authorization.
- Modified state management to include the code verifier for secure state generation and validation.
2026-04-04 03:35:34 +05:30
Anish Sarkar
a2b3541046
chore: ran linting
2026-04-04 03:11:56 +05:30
Anish Sarkar
0d2acc665d
Merge remote-tracking branch 'upstream/dev' into feat/page-limit-connectors
2026-04-04 03:08:27 +05:30
Anish Sarkar
ce40da80ea
feat: implement page limit estimation and enforcement in file based connector indexers
...
- Added a static method `estimate_pages_from_metadata` to `PageLimitService` for estimating page counts based on file metadata.
- Integrated page limit checks in Google Drive, Dropbox, and OneDrive indexers to prevent exceeding user quotas during file indexing.
- Updated relevant indexing methods to utilize the new page estimation logic and enforce limits accordingly.
- Enhanced tests for page limit functionality, ensuring accurate estimation and enforcement across different file types.
2026-04-04 02:51:28 +05:30
CREDO23
960b8fc012
add app context and KB grounding to autocomplete suggestions
2026-04-03 21:34:01 +02:00
CREDO23
c5aa869adb
switch to on-demand permission requests and improve suggestion UX
2026-04-03 19:57:48 +02:00
CREDO23
aeb3f13f91
replace text-based autocomplete with vision-based endpoint
2026-04-03 18:17:50 +02:00
CREDO23
482238e5d4
add vision LLM role for screenshot analysis
2026-04-03 17:40:27 +02:00
Anish Sarkar
9c0af6569d
feat: implement page limit checks in local folder indexing to manage user page usage
2026-04-03 19:13:25 +05:30
Anish Sarkar
edda5b98cb
chore: ran linting
2026-04-03 17:38:29 +05:30
Anish Sarkar
b759bb36a9
feat: add direct conversion support for CSV, TSV, and HTML files in local folder indexing
2026-04-03 17:36:48 +05:30
Anish Sarkar
746c730b2e
chore: ran linting
2026-04-03 13:14:40 +05:30
Anish Sarkar
3833084dad
feat: changed the revision number of folder alembic migration
2026-04-03 11:45:53 +05:30
Anish Sarkar
62b44889d1
Merge remote-tracking branch 'upstream/dev' into feat/local-folder-sync
2026-04-03 11:42:43 +05:30
Anish Sarkar
525fae5abf
feat: trigger document reindexing after restoring a document version to ensure content is up-to-date
2026-04-03 11:19:54 +05:30
Anish Sarkar
4ca0787ab7
feat: add version history functionality to document management, enabling users to view and manage document versions through UI updates and backend integration
2026-04-03 10:42:21 +05:30
Anish Sarkar
2b9d79d44c
feat: add integration tests for batch processing of local folder indexing, covering multiple file scenarios and error handling
2026-04-03 10:04:14 +05:30
Anish Sarkar
1fa8e1cc83
feat: refactor folder indexing to support batch processing of multiple files, enhancing performance and error handling
2026-04-03 10:02:36 +05:30
Anish Sarkar
e2ba509314
feat: enhance error handling in local folder indexing by adding rollback and refresh on IntegrityError
2026-04-03 09:29:59 +05:30
DESKTOP-RTLN3BA\$punk
6b06d3abb1
chore: optimize zero publication column migration process
...
- Updated migration instructions to emphasize the importance of stopping zero-cache before and after running the migration.
- Added a function to terminate blocked PIDs that could interfere with the migration.
- Set a lock timeout to prevent deadlocks during the migration process.
2026-04-02 20:38:10 -07:00
DESKTOP-RTLN3BA\$punk
62e698d8aa
refactor: streamline document upload limits and enhance handling of mentioned documents
...
- Updated maximum file size limit to 500 MB per file.
- Removed restrictions on the number of files per upload and total upload size.
- Enhanced handling of user-mentioning documents in the knowledge base search middleware.
- Improved document reading and processing logic to accommodate new features and optimizations.
2026-04-02 19:39:10 -07:00
Anish Sarkar
fe7fcaae5d
feat: update folder deletion process to queue document deletions first and handle folder cleanup in Celery task
2026-04-03 04:16:19 +05:30
Anish Sarkar
44e39792da
feat: assign folder_id to documents before indexing to ensure correct folder visibility during processing
2026-04-03 04:14:28 +05:30
Anish Sarkar
bd21c2842d
feat: enhance document upload and folder synchronization UI with improved processing state indicators and responsive design adjustments
2026-04-03 04:14:09 +05:30
Anish Sarkar
f0a7c7134a
feat: move endpoint above to retrieve watched folders based on search space ID
2026-04-03 00:09:17 +05:30
CREDO23
3e68d4aa3e
refactor: extract autocomplete service and fix tooltip screen-edge positioning
2026-04-02 20:38:09 +02:00
