Commit graph

299 commits

Author SHA1 Message Date
DESKTOP-RTLN3BA\$punk
c80bbfa867 feat: added celery and removed background_tasks for MQ's
- removed pre commit hooks
- updated docker setup
- updated github docker actions
- updated docs
2025-10-20 00:30:00 -07:00
DESKTOP-RTLN3BA\$punk
5b957ec21c feat: bumped version to v0.0.8 2025-10-16 22:44:12 -07:00
Anish Sarkar
bbb2abfc02 fix: ran formatter as per coderrabbitai 2025-10-17 02:44:44 +05:30
Anish Sarkar
0ff1b586a2 feat: update Elasticsearch integration and logging
- revised Elasticsearch connector enum revision IDs
- added `TaskLoggingService` to elasticsearch_indexer
- integrated Elasticsearch into prompts.py as requested
2025-10-17 02:21:56 +05:30
Anish Sarkar
8e1e81ebae Merge remote-tracking branch 'upstream/main' into feature/elasticsearch-connector 2025-10-17 01:02:29 +05:30
Differ
a479fea588 fix: Baidu search UI rendering and clean debug code
- Fix: Add BAIDU_SEARCH_API to direct render sources
- Fix: Remove API key exposure in debug logs
- Resolves 404 error when clicking Baidu search results
2025-10-16 22:45:35 +08:00
Anish Sarkar
82438c7396 refactor: streamline Elasticsearch indexing by removing unused services and integrating document chunking, also added documentation 2025-10-16 17:48:28 +05:30
Differ
beaf8f89a6 feat: add Baidu AI Search integration
- Add BAIDU_SEARCH_API connector type to support Chinese web search
- Implement search_baidu() method in connector_service.py
- Add frontend configuration page for Baidu Search API
- Create Alembic migration for new enum values
- Add validation rules and agent integration
- Support configurable model, search source, and deep search options
- Update .gitignore to exclude .env.local and other env files

Addresses integration with Chinese search ecosystem for better local market support.
Baidu AI Search provides intelligent search with automatic summarization.
2025-10-16 19:30:42 +08:00
Anish Sarkar
3c0f5e42f3 updated enum revision id and uv.lock 2025-10-16 16:52:27 +05:30
Anish Sarkar
929035f802 Merge remote-tracking branch 'upstream/main' into feature/elasticsearch-connector 2025-10-16 16:24:37 +05:30
DESKTOP-RTLN3BA\$punk
70b547c9c9 chore: updated docs & refactored sst_service.py 2025-10-15 14:31:38 -07:00
Rohan Verma
5ed9aa2b0b
Merge pull request #387 from nabthebest135/local-stt
local STT implementation with Faster-Whisper
2025-10-15 14:08:09 -07:00
DESKTOP-RTLN3BA\$punk
97f06899e4 Merge branch 'dev' of https://github.com/MODSetter/SurfSense into dev 2025-10-14 21:12:17 -07:00
DESKTOP-RTLN3BA\$punk
c99cd710ea feat: add unique identifier hash for documents to prevent duplicates across various connectors 2025-10-14 21:11:19 -07:00
MSI\ModSetter
031dc714eb chore: fixed validators 2025-10-14 17:03:38 -07:00
DESKTOP-RTLN3BA\$punk
31982cea9a chore: removed content trunking for better UI 2025-10-14 14:19:48 -07:00
DESKTOP-RTLN3BA\$punk
ba5bb91a7b chore: removed chinese comments to pass ruff checks and updated migration nos 2025-10-13 20:07:32 -07:00
Rohan Verma
c99469bfdf
Merge pull request #391 from unitagain/feature/chinese-llm-support
feat: add Chinese LLM providers support with auto-fill API Base URL
2025-10-13 19:52:21 -07:00
DESKTOP-RTLN3BA\$punk
acd900c41d ruff fix 2025-10-13 14:01:55 -07:00
DESKTOP-RTLN3BA\$punk
576dc19260 refactor: updated SearxNG connector migration and fixed validation for SearxNG host URL 2025-10-13 13:57:58 -07:00
Rohan Verma
08661e686c
Merge pull request #392 from Aki-07/feature/add-searxng-connector
Add Searxng connector
2025-10-13 13:41:37 -07:00
Nabhan
9b72ec65b5 fix: address code review feedback for STT implementation
- Add header to local STT transcription for consistency
- Add empty text validation for external STT path
- Refactor external STT to eliminate duplication in atranscription calls
- Ensure both local and external paths have consistent error handling
2025-10-13 14:26:36 +05:00
DESKTOP-RTLN3BA\$punk
a3f50ebc4d feat: added missed migration 2025-10-12 20:15:27 -07:00
Tarun
120d60465e refactor: streamline language instruction handling across prompts 2025-10-12 23:40:46 +05:30
Aki-07
ff08daccb1 fix: restore Tavily search and tighten SearxNG config 2025-10-12 21:29:12 +05:30
Aki-07
ec3e4888d8 feat(back): surface SearxNG results in researcher workflow 2025-10-12 20:44:14 +05:30
Aki-07
020c5ed04e feat(back): add SearxNG search integration 2025-10-12 20:44:00 +05:30
Aki-07
d2ffd12480 feat(back): add SearxNG connector type 2025-10-12 20:43:45 +05:30
Differ
917cf4f398 feat: add Chinese LLM providers support with auto-fill API Base URL
- Add support for DeepSeek, Qwen (Alibaba), Kimi (Moonshot), and GLM (Zhipu)
- Implement auto-fill API Base URL when selecting Chinese LLM providers
- Add smart validation and warnings for missing API endpoints
- Fix session state management in task logging service
- Add comprehensive Chinese setup documentation
- Add database migration for new LLM provider enums

Closes #383
2025-10-12 19:10:46 +08:00
Tarun
045537aa79 feat: add language support across configurations and prompts 2025-10-12 13:13:42 +05:30
Nabhan
15ba2b86f6 fix: add defensive dictionary access and error handling for local STT
- Use .get() for safe dictionary access instead of direct key access
- Add explicit try-catch for local STT transcription failures
- Validate transcription result is not empty
- Provide clear error messages for corrupted audio files
- Match error handling pattern with external STT service
2025-10-12 11:14:12 +05:00
Nabhan
504399ad01 refactor: eliminate duplicated STT service condition check
- Compute stt_service_type once and reuse
- Follow DRY principles
- Improve code maintainability
2025-10-12 11:13:30 +05:00
Nabhan
cf0e265107 refactor: integrate local STT with existing upload flow
- Simplify STT_SERVICE config to local/MODEL_SIZE format
- Remove separate STT routes, integrate with document upload
- Add local STT support to audio file processing pipeline
- Remove React component, use existing upload interface
- Support both local Faster-Whisper and external STT services
- Tested with real speech: 99% accuracy, 2.87s processing
2025-10-12 10:50:55 +05:00
Anish Sarkar
72e8d98f40 feat: enhance Elasticsearch connector to handle missing index configuration 2025-10-12 10:10:19 +05:30
Anish Sarkar
c3d249682f feat: update Elasticsearch connector handling and display name 2025-10-12 09:56:52 +05:30
Anish Sarkar
55d752e3c8 feat: added elasticsearch connector 2025-10-12 09:39:04 +05:30
Nabhan
bd6b198e20 feat: implement local STT with Faster-Whisper
- Add STT service with CPU-optimized Faster-Whisper
- Add API endpoints for transcription and model management
- Add React audio recorder component
- Support multiple Whisper models (tiny to large-v3)
- Include error handling for corrupted/invalid files
- Tested with real speech audio (99% accuracy)
- No external API dependencies, fully offline
2025-10-12 01:11:04 +05:00
Nabhan
dad79674c8 Added local Speech-to-Text (STT) support using Faster-Whisper 2025-10-11 23:56:12 +05:00
DESKTOP-RTLN3BA\$punk
633ea3ac0f feat: moved LLMConfigs from User to SearchSpaces
- RBAC soon??
- Updated various services and routes to handle search space-specific LLM preferences.
- Modified frontend components to pass search space ID for LLM configuration management.
- Removed onboarding page and settings page as part of the refactor.
2025-10-10 00:50:29 -07:00
Rohan Verma
a1b1db3895
Merge pull request #372 from iamsyg/add-inference-params-1
feat: add interface params
2025-10-09 00:34:25 -07:00
DESKTOP-RTLN3BA\$punk
b5c1157468 fix: removed checks that were not checked
- fuck this hacktoberfest
2025-10-08 21:45:21 -07:00
Rohan Verma
6d53b0c8f7
Merge pull request #370 from vaishcodescape/aditya/features
fix:chat-routes backend security vulnerability fixed
2025-10-08 21:28:39 -07:00
DESKTOP-RTLN3BA\$punk
aea09a5dad feat: Moved searchconnectors association from user to searchspace
- Need to move llm configs to searchspace
2025-10-08 21:13:01 -07:00
Swayam
46e921846b fix: PR changes 2025-10-08 15:11:48 +05:30
Aditya Vaish
20ac726a37 cleanup issue resolved 2025-10-08 12:22:28 +05:30
Aditya Vaish
3d52a5e9d6 attribute error fixed 2025-10-08 12:13:26 +05:30
Aditya Vaish
c1975091ce minor changes added 2025-10-08 12:06:49 +05:30
Aditya Vaish
a13a0594b8 fix trim mode values before validating 2025-10-08 01:07:32 +05:30
Aditya Vaish
d1f21a8dc6 pending issues resolved 2025-10-08 00:29:09 +05:30
Swayam
b7c68277f6 feat: add interface params 2025-10-07 04:59:01 +05:30