Commit graph

3352 commits

Author SHA1 Message Date
Anish Sarkar
9ccee054a5 chore: ran linting 2026-02-26 03:05:20 +05:30
Anish Sarkar
7332be956e refactor: reorganize test fixtures and helpers for improved clarity and maintainability 2026-02-26 03:04:56 +05:30
Anish Sarkar
f59a70f7a5 Merge remote-tracking branch 'upstream/dev' into feat/document-test 2026-02-26 02:22:10 +05:30
Rohan Verma
2e99f1e853
Merge pull request #837 from CREDO23/test-document-creation
[Refactor] Core document creation / indexing pipeline with unit and e2e tests
2026-02-25 12:27:41 -08:00
DESKTOP-RTLN3BA\$punk
54dd6f904c feat: implement local file persistence for Daytona sandboxes
- Added functionality to persist sandbox files locally before deletion.
- Introduced methods for retrieving and deleting locally stored sandbox files.
- Updated routes to handle local file downloads and background deletion of sandboxes.
- Enhanced streaming tasks to support sandbox file management.
2026-02-25 12:25:59 -08:00
Anish Sarkar
d6e442b466 fix: Improve document upload tests by adding assertions for document IDs and handling transient transport errors 2026-02-26 01:50:57 +05:30
Anish Sarkar
25df3dff64 chore: ran linting 2026-02-26 01:29:25 +05:30
Anish Sarkar
380c1c3877 fix: Refactor document ID usage in file processing to improve clarity 2026-02-26 01:28:09 +05:30
Anish Sarkar
a57ab02900 feat: Implement file upload limits and page limit enforcement in backend
- Added constants for maximum files per upload, per-file size, and total upload size.
- Enhanced document upload route to validate file counts and sizes, returning appropriate HTTP errors.
- Introduced end-to-end tests for upload limits and page limit enforcement, ensuring correct behavior under various scenarios.
- Updated test helpers to support notification retrieval for page limit exceeded scenarios.
2026-02-26 01:25:34 +05:30
CREDO23
c50d661d7d fix wrong status key in adapter error reporting 2026-02-25 21:00:55 +02:00
CREDO23
7a0b8838d1 chore: cleanup 2026-02-25 20:58:46 +02:00
CREDO23
ab31bba3ad merge upstream/dev 2026-02-25 20:52:05 +02:00
CREDO23
d0fdd3224a fix metadata keys casing and set content_needs_reindexing in adapter 2026-02-25 20:39:18 +02:00
CREDO23
0de74f4bf7 add docstrings to all indexing pipeline tests 2026-02-25 20:30:31 +02:00
CREDO23
4293910e8e plug file upload into indexing pipeline adapter and add integration tests 2026-02-25 20:20:52 +02:00
CREDO23
cad400be1b add file upload adapter and make index() return refreshed document 2026-02-25 19:56:59 +02:00
CREDO23
86ecb82c6e fix: tighten indexing pipeline exception handling and logging 2026-02-25 17:44:35 +02:00
CREDO23
5be58b78ad simplify indexing pipeline DB error handling 2026-02-25 16:59:09 +02:00
Anish Sarkar
93c0af475b Merge remote-tracking branch 'upstream/dev' into feat/document-test 2026-02-25 20:25:06 +05:30
Anish Sarkar
853750e6e9 chore: ran linting 2026-02-25 20:08:20 +05:30
Anish Sarkar
a7df26d451 refactor: remove redundant status assertion 2026-02-25 20:07:34 +05:30
CREDO23
66d7d3da8a fix bugs in indexing pipeline exception handling 2026-02-25 16:27:12 +02:00
Anish Sarkar
e1ac8bfc76 feat: Add testing environment variables to surfsense_backend/.env.example and update documentation 2026-02-25 19:52:25 +05:30
Anish Sarkar
c564e5f768 feat: Enhance test document purging by implementing direct database access 2026-02-25 19:51:29 +05:30
CREDO23
b6c25628c8 add structured logging to indexing pipeline 2026-02-25 16:04:35 +02:00
CREDO23
610080bfef extract persistence helpers into document_persistence.py 2026-02-25 15:30:25 +02:00
Anish Sarkar
c3273af20b feat: Add fixture to purge all test search space documents before a new session 2026-02-25 18:59:23 +05:30
CREDO23
0aeb888be0 add structured error handling to indexing pipeline 2026-02-25 15:26:04 +02:00
Anish Sarkar
2fef971fa9 chore: Update pytest duration reporting in pyproject.toml and refine testing documentation 2026-02-25 18:36:55 +05:30
Anish Sarkar
f52090391a feat: Add testing documentation and update meta.json 2026-02-25 17:54:03 +05:30
Anish Sarkar
03d8788241 chore: Enhance pytest configuration and add custom markers in pyproject.toml 2026-02-25 17:51:33 +05:30
Anish Sarkar
d350f47ee7 chore: Update pytest configuration in pyproject.toml to enhance test output
- Added command-line options to pytest configuration for verbose output, short tracebacks, and duration reporting of the slowest tests.
2026-02-25 17:34:49 +05:30
Anish Sarkar
4ff712578d refactor: Enhance test utilities for document upload by integrating search space handling
- Updated test fixtures to include search space ID retrieval for improved document upload tests.
- Refactored authentication and document upload functions to accept search space ID as a parameter.
- Removed hardcoded search space ID references to streamline test configurations.
2026-02-25 17:29:09 +05:30
CREDO23
ca870cf660 add fallback document sumary 2026-02-25 13:47:36 +02:00
Anish Sarkar
41eb68663a feat: Add end-to-end tests for document upload pipeline and shared test utilities
- Introduced new test files for end-to-end testing of document uploads, including support for .txt, .md, and .pdf formats.
- Created shared fixtures and helper functions for authentication, document management, and cleanup.
- Added sample documents for testing purposes.
- Established a conftest.py file to provide reusable fixtures across test modules.
2026-02-25 16:39:45 +05:30
CREDO23
36d1fba75f fix: isolate per-document errors in prepare_for_indexing 2026-02-25 13:00:34 +02:00
Anish Sarkar
b7447b26f9 Refactor dependency management in pyproject.toml and uv.lock to include testing libraries and update pytest configuration 2026-02-25 16:06:16 +05:30
Anish Sarkar
d576e93f2a Update dependencies in pyproject.toml and uv.lock for testing and package management 2026-02-25 15:36:29 +05:30
CREDO23
e6b7ce7345 fix: handle IntegrityError in prepare_for_indexing and add within-batch content dedup test 2026-02-25 12:03:00 +02:00
Rohan Verma
80be9f26c4
Merge pull request #835 from MODSetter/codesandbox_integration
feat: codesandbox(daytona) integration
2026-02-25 01:59:20 -08:00
DESKTOP-RTLN3BA\$punk
96595d3ff7 refactor: remove security notes from system prompt
- Eliminated the security notes section from the code execution system prompt to streamline the instructions and focus on user guidance.
2026-02-25 01:56:16 -08:00
DESKTOP-RTLN3BA\$punk
e574a7c38b feat: update .env.example for Daytona sandbox integration
- Removed Uvicorn server configuration options.
- Added new agent-specific configuration for Daytona sandbox, including API key and URL settings.
2026-02-25 01:54:24 -08:00
DESKTOP-RTLN3BA\$punk
d198c8b89d chore; linting 2026-02-25 01:50:28 -08:00
DESKTOP-RTLN3BA\$punk
70686a1eb2 feat: enhance code execution system prompt
- Added detailed instructions for prioritizing code execution over text responses in specific scenarios, such as data visualization and file generation.
- Included examples to illustrate correct and incorrect approaches to handling user requests involving code execution.
- Emphasized the importance of printing file paths for user downloads and clarified the handling of sandbox-generated files.
2026-02-25 01:48:54 -08:00
DESKTOP-RTLN3BA\$punk
d570cae3c6 feat: added file handling for daytona sandboxes
- Added _TimeoutAwareSandbox class to handle per-command timeouts in DaytonaSandbox.
- Updated _find_or_create function to manage sandbox states and restart stopped/archived sandboxes.
- Enhanced get_or_create_sandbox to return the new sandbox class.
- Introduced file download functionality in the frontend, allowing users to download generated files from the sandbox.
- Updated system prompt to include guidelines for sharing generated files.
2026-02-25 01:36:30 -08:00
CREDO23
1b4ed35de3 fix: correct test fixtures and add missing summarizer tests 2026-02-25 11:15:48 +02:00
CREDO23
c5ae62140d fix: rescue stuck documents with unchanged content on next indexing run 2026-02-25 11:13:25 +02:00
DESKTOP-RTLN3BA\$punk
a6563f396a chore: moved to daytona due to lack of windows support in microsandbox 2026-02-25 00:38:27 -08:00
DESKTOP-RTLN3BA\$punk
421bb29466 chore: update microsandbox command to bind to all interfaces and add compatibility shims for missing types 2026-02-24 23:53:03 -08:00
DESKTOP-RTLN3BA\$punk
8095cec37e refactor: enable microsandbox service in docker-compose with updated installation commands and healthcheck 2026-02-24 23:26:31 -08:00