Commit graph

3377 commits

Author SHA1 Message Date
DESKTOP-RTLN3BA\$punk
37a0bd4533 feat: update contact information and meeting links across components
- Changed meeting link from Calendly to Cal.com for scheduling.
- Updated email contact from eric@surfsense.com to rohan@surfsense.com in multiple components.
- Revised text in the contact form to reflect the new scheduling options.
2026-02-27 15:31:31 -08:00
Rohan Verma
3ca401cb2c
Merge pull request #840 from MODSetter/dev
feat: Google Drive HITL tools, team management UI, Daytona sandboxes, indexing pipeline hardening & test infrastructure
2026-02-26 20:49:50 -08:00
DESKTOP-RTLN3BA\$punk
1e4b8d3e89 feat: enhance document formatting and context management for LLM tools
- Introduced dynamic character budget calculation for document formatting based on model's context window.
- Updated `format_documents_for_context` to respect character limits and improve output quality.
- Added `max_input_tokens` parameter to various functions to facilitate context-aware processing.
- Enhanced error handling for context overflow in LLM router service.
2026-02-26 20:47:19 -08:00
DESKTOP-RTLN3BA\$punk
a4dc84d1ab feat: add should_summarize parameter to task dispatchers
- Introduced should_summarize parameter in TaskDispatcher and CeleryTaskDispatcher to control summary generation.
- Updated InlineTaskDispatcher to support the new parameter for document processing.
2026-02-26 19:12:37 -08:00
DESKTOP-RTLN3BA\$punk
6f4bf11a32 Merge branch 'dev' of https://github.com/MODSetter/SurfSense into dev 2026-02-26 18:25:05 -08:00
DESKTOP-RTLN3BA\$punk
e9892c8fe9 feat: added configable summary calculation and various improvements
- Replaced direct embedding calls with a utility function across various components to streamline embedding logic.
- Added enable_summary flag to several models and routes to control summary generation behavior.
2026-02-26 18:24:57 -08:00
Rohan Verma
2f08dc9cf4
Merge pull request #839 from AnishSarkar22/feat/document-test
fix: enhanced document upload, page limit, upload limit tests
2026-02-26 13:48:45 -08:00
Anish Sarkar
2468cc2a4a feat: add SidebarSeparator component and integrate into documentation layout
- Introduced SidebarSeparator component for improved sidebar organization.
- Updated layout configuration to utilize the new SidebarSeparator in the documentation layout.
2026-02-27 02:38:52 +05:30
Anish Sarkar
394a37576c refactor: update layout configuration and enhance documentation layout 2026-02-27 02:36:18 +05:30
DESKTOP-RTLN3BA\$punk
dc33a4a68f Merge commit 'af3810b7c4' into dev 2026-02-26 13:05:02 -08:00
DESKTOP-RTLN3BA\$punk
23f553ef84 Merge branch 'dev' of https://github.com/MODSetter/SurfSense into dev 2026-02-26 13:01:24 -08:00
DESKTOP-RTLN3BA\$punk
aabc24f82c feat: enhance performance logging and caching in various components
- Introduced slow callback logging in FastAPI to identify blocking calls.
- Added performance logging for agent creation and tool loading processes.
- Implemented caching for MCP tools to reduce redundant server calls.
- Enhanced sandbox management with in-process caching for improved efficiency.
- Refactored several functions for better readability and performance tracking.
- Updated tests to ensure proper functionality of new features and optimizations.
2026-02-26 13:00:31 -08:00
Anish Sarkar
20ebb0703f chore: add icons to documentation metadata for improved visual representation 2026-02-27 02:15:21 +05:30
Anish Sarkar
78dcce3e06 refactor: update testing documentation for clarity and structure 2026-02-27 02:07:14 +05:30
Anish Sarkar
836d5293df refactor: remove unused TestStatusPolling class from document upload integration tests 2026-02-27 01:52:35 +05:30
Anish Sarkar
fd032f3709 refactor: simplify and clarify documentation in document upload integration tests 2026-02-27 01:48:25 +05:30
Anish Sarkar
7c09958ddc refactor: enhance document upload integration tests for API contract validation 2026-02-27 01:24:20 +05:30
Anish Sarkar
1068ea25a7 refactor: standardize test database configuration across test files 2026-02-27 00:45:51 +05:30
Anish Sarkar
f09b5b0ea4 refactor: replace hardcoded embedding dimension with dynamic configuration
- Updated the embedding dimension in test configurations to use the value from the application config, enhancing maintainability and consistency across tests.
2026-02-27 00:17:39 +05:30
Anish Sarkar
223c2de0d2 refactor: update database connection handling in test configurations 2026-02-27 00:05:21 +05:30
Anish Sarkar
87711ee381 chore: clean up .env.example and pyproject.toml
- Removed commented-out testing configuration from .env.example to streamline the file.
- Updated markers in pyproject.toml to remove the e2e test marker, clarifying the purpose of the remaining markers.
2026-02-26 23:56:01 +05:30
Anish Sarkar
3393e435f9 feat: implement task dispatcher for document processing
- Introduced a TaskDispatcher abstraction to decouple the upload endpoint from Celery, allowing for easier testing with synchronous implementations.
- Updated the create_documents_file_upload function to utilize the new dispatcher for task management.
- Removed direct Celery task imports from the upload function, enhancing modularity.
- Added integration tests for document upload, including page limit enforcement and file size restrictions.
2026-02-26 23:55:47 +05:30
Rohan Verma
30617c6e54
Merge pull request #836 from AnishSarkar22/feat/document-test
feat: add document upload E2E tests
2026-02-25 14:22:07 -08:00
Anish Sarkar
bf60a5049f feat: add end-to-end test for document searchability after upload
- Introduced a new test class to verify that uploaded documents appear in search results once their status is ready.
- Implemented assertions to ensure the uploaded document's ID is present in the search response.
2026-02-26 03:33:37 +05:30
Anish Sarkar
9ccee054a5 chore: ran linting 2026-02-26 03:05:20 +05:30
Anish Sarkar
7332be956e refactor: reorganize test fixtures and helpers for improved clarity and maintainability 2026-02-26 03:04:56 +05:30
Anish Sarkar
f59a70f7a5 Merge remote-tracking branch 'upstream/dev' into feat/document-test 2026-02-26 02:22:10 +05:30
Rohan Verma
2e99f1e853
Merge pull request #837 from CREDO23/test-document-creation
[Refactor] Core document creation / indexing pipeline with unit and e2e tests
2026-02-25 12:27:41 -08:00
DESKTOP-RTLN3BA\$punk
54dd6f904c feat: implement local file persistence for Daytona sandboxes
- Added functionality to persist sandbox files locally before deletion.
- Introduced methods for retrieving and deleting locally stored sandbox files.
- Updated routes to handle local file downloads and background deletion of sandboxes.
- Enhanced streaming tasks to support sandbox file management.
2026-02-25 12:25:59 -08:00
Anish Sarkar
d6e442b466 fix: Improve document upload tests by adding assertions for document IDs and handling transient transport errors 2026-02-26 01:50:57 +05:30
Anish Sarkar
25df3dff64 chore: ran linting 2026-02-26 01:29:25 +05:30
Anish Sarkar
380c1c3877 fix: Refactor document ID usage in file processing to improve clarity 2026-02-26 01:28:09 +05:30
Anish Sarkar
a57ab02900 feat: Implement file upload limits and page limit enforcement in backend
- Added constants for maximum files per upload, per-file size, and total upload size.
- Enhanced document upload route to validate file counts and sizes, returning appropriate HTTP errors.
- Introduced end-to-end tests for upload limits and page limit enforcement, ensuring correct behavior under various scenarios.
- Updated test helpers to support notification retrieval for page limit exceeded scenarios.
2026-02-26 01:25:34 +05:30
CREDO23
c50d661d7d fix wrong status key in adapter error reporting 2026-02-25 21:00:55 +02:00
CREDO23
7a0b8838d1 chore: cleanup 2026-02-25 20:58:46 +02:00
CREDO23
ab31bba3ad merge upstream/dev 2026-02-25 20:52:05 +02:00
CREDO23
d0fdd3224a fix metadata keys casing and set content_needs_reindexing in adapter 2026-02-25 20:39:18 +02:00
CREDO23
0de74f4bf7 add docstrings to all indexing pipeline tests 2026-02-25 20:30:31 +02:00
CREDO23
4293910e8e plug file upload into indexing pipeline adapter and add integration tests 2026-02-25 20:20:52 +02:00
CREDO23
cad400be1b add file upload adapter and make index() return refreshed document 2026-02-25 19:56:59 +02:00
CREDO23
86ecb82c6e fix: tighten indexing pipeline exception handling and logging 2026-02-25 17:44:35 +02:00
CREDO23
5be58b78ad simplify indexing pipeline DB error handling 2026-02-25 16:59:09 +02:00
Anish Sarkar
93c0af475b Merge remote-tracking branch 'upstream/dev' into feat/document-test 2026-02-25 20:25:06 +05:30
Anish Sarkar
853750e6e9 chore: ran linting 2026-02-25 20:08:20 +05:30
Anish Sarkar
a7df26d451 refactor: remove redundant status assertion 2026-02-25 20:07:34 +05:30
CREDO23
66d7d3da8a fix bugs in indexing pipeline exception handling 2026-02-25 16:27:12 +02:00
Anish Sarkar
e1ac8bfc76 feat: Add testing environment variables to surfsense_backend/.env.example and update documentation 2026-02-25 19:52:25 +05:30
Anish Sarkar
c564e5f768 feat: Enhance test document purging by implementing direct database access 2026-02-25 19:51:29 +05:30
CREDO23
b6c25628c8 add structured logging to indexing pipeline 2026-02-25 16:04:35 +02:00
CREDO23
610080bfef extract persistence helpers into document_persistence.py 2026-02-25 15:30:25 +02:00