Commit graph

112 commits

Author SHA1 Message Date
CREDO23
91d947ff79 refactor(embedding-cache): rename index cache to embedding cache
The cached payload is the indexing pipeline's embeddings (markdown is
chunked then embedded), so "embedding cache" names the expensive output
directly and removes the "index" ambiguity (DB index vs vector index vs
indexing phase). Renames the service, settings, eligibility, eviction
task, metrics, config flags (INDEX_CACHE_* -> EMBEDDING_CACHE_*), object
prefix, and the table (index_cache_embedding_sets -> embedding_cache_sets)
with its constraint and indexes. Migration 161 renamed accordingly.
2026-06-12 17:00:01 +02:00
CREDO23
daccd304ee feat(index-cache): add settings, eligibility, and config flags 2026-06-12 16:48:10 +02:00
CREDO23
5c4eec26cc feat(config): add ETL_CACHE_* settings 2026-06-12 11:23:50 +02:00
DESKTOP-RTLN3BA\$punk
a7407502d3 feat(refactor): refactor payment system to implement unified credit wallet.
- Updated environment variables and - configurations for credit purchases via Stripe, replacing legacy page pack system.
- Introduced auto-reload feature for credit top-ups and modified database models to track credit transactions.
- Updated notification system to handle insufficient credits and auto-reload failures.
- Adjusted API routes and schemas to reflect changes in credit management.
2026-06-10 16:49:03 -07:00
DESKTOP-RTLN3BA\$punk
ce952d2ad1 chore: linting 2026-06-09 00:42:26 -07:00
DESKTOP-RTLN3BA\$punk
640ef5f15d feat(proxy): integrate Scrapling for enhanced web scraping capabilities
- Replaced Playwright with Scrapling's fetchers in the web crawling and YouTube processing modules for improved performance and flexibility.
- Updated proxy configuration to support dynamic proxy selection via environment variables.
- Enhanced logging to track performance metrics during web scraping operations.
- Refactored related modules to utilize the new proxy utilities and streamline the scraping process.
2026-06-09 00:15:10 -07:00
DESKTOP-RTLN3BA\$punk
41a93ca8fb refactor(env): streamline Redis configuration and remove deprecated variables
- Consolidated Redis configuration by introducing a single `REDIS_URL` variable for Celery broker, result backend, and app cache.
- Removed deprecated variables related to Firecrawl and Stripe token limits from `.env.example` files.
- Updated documentation to reflect changes in environment variable usage for improved clarity and maintainability.
2026-06-08 14:37:44 -07:00
DESKTOP-RTLN3BA\$punk
c2beaf1e5a refactor(config): centralize configuration management across modules
- Replaced environment variable usage with a centralized configuration system in multiple modules, including `celery_app`, `agent_cache_store`, `sandbox`, `file_storage`, and `connector_service`.
- Enhanced maintainability and readability by sourcing configuration values from the `config` module instead of directly from environment variables.
- Updated relevant settings to ensure consistent access to configuration values across the application.
2026-06-08 13:50:16 -07:00
DESKTOP-RTLN3BA\$punk
ef7a20a5d0 feat(gateway): implement global messaging gateway toggle
- Added a global switch `GATEWAY_ENABLED` to control the activation of all messaging gateway channels (Telegram, WhatsApp, Slack, Discord).
- Updated relevant routes and workers to check the `GATEWAY_ENABLED` flag, returning 404 for HTTP routes when disabled.
- Enhanced documentation in the `.env.example` file to reflect the new configuration option.
2026-06-08 13:24:29 -07:00
CREDO23
305a8fe7e6 refactor(config): remove dead MULTI_AGENT_CHAT_ENABLED flag (bucket B5)
Multi-agent is now the only chat path (B1-B4), so the flag is dead config.
Drop it from app/config and .env.example. Suite green (2710 passed).
2026-06-04 13:44:02 +02:00
Anish Sarkar
2e64d5d3de Merge remote-tracking branch 'upstream/dev' into feat/whatsapp-gateway-integration 2026-06-02 01:22:32 +05:30
Anish Sarkar
799a83239f feat(gateway): add Slack and Telegram gateway configuration and enablement checks 2026-06-01 23:58:28 +05:30
octo-patch
722b5fefcd feat: upgrade MiniMax default model to M3
- Add MiniMax-M3 to the model selection list (set as the new default)
- Add MiniMax-M2.7 and MiniMax-M2.7-highspeed as alternatives
- Remove deprecated MiniMax-M2.5 / M2.5-highspeed entries
- Update example config and Chinese setup docs to reference M3 (512K context)
2026-06-02 01:48:11 +08:00
Anish Sarkar
68da295b5d feat(gateway): add Discord gateway configuration 2026-06-01 20:58:31 +05:30
Anish Sarkar
5b71685dad feat(gateway): add Slack gateway configuration 2026-06-01 12:36:07 +05:30
Anish Sarkar
9143cb6ad1 feat(gateway): update WhatsApp bridge configuration and expose port 9929 2026-05-29 11:39:11 +05:30
Anish Sarkar
a6b2882275 feat(gateway): add WhatsApp gateway configuration 2026-05-29 10:18:11 +05:30
Anish Sarkar
c958fe5bc6 feat(gateway): introduce GATEWAY_TELEGRAM_INTAKE_MODE for Telegram integration 2026-05-28 05:02:07 +05:30
Anish Sarkar
08bf3cc023 refactor(gateway): run inbox and BYO polling from FastAPI lifespan 2026-05-28 04:38:00 +05:30
Anish Sarkar
ae3ce91465 feat(gateway): add configuration and metrics hooks 2026-05-27 23:37:26 +05:30
CREDO23
71dead0406 perf(kb-planner): route internal planner calls to dedicated small/fast LLM
Adds an optional planner LLM role wired through KnowledgePriorityMiddleware
so KB query rewriting, date extraction, and recency classification run on a
cheap model (e.g. gpt-4o-mini, Haiku, Azure nano) instead of the user's
chat LLM. Operators opt in by setting is_planner: true on exactly one
global config; without it, behavior is unchanged.
2026-05-20 11:42:52 +02:00
Anish Sarkar
275e2c9e83 chore: fix linting 2026-05-12 04:00:04 +05:30
Anish Sarkar
efff7ab2a2 chore: enhance Dockerfile and config to support conditional static ffmpeg import 2026-05-11 04:51:19 +05:30
CREDO23
5119915f4f Merge upstream/dev into feature/multi-agent 2026-05-05 01:44:46 +02:00
DESKTOP-RTLN3BA\$punk
47b2994ec7 feat: fixed vision/image provider specific errors and fixed podcast/video streaming
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
2026-05-02 19:18:53 -07:00
DESKTOP-RTLN3BA\$punk
ae9d36d77f feat: unified credits and its cost calculations 2026-05-02 14:34:23 -07:00
Anish Sarkar
cd25175b84 chore: ran linting 2026-05-02 03:36:13 +05:30
Anish Sarkar
c229b4356a feat(config): stamp Auto (Fastest) ranking metadata on YAML configs 2026-05-01 23:38:21 +05:30
Anish Sarkar
680a1c1c38 refactor(openrouter): remove virtual openrouter/free auto-select entry 2026-05-01 18:16:47 +05:30
Anish Sarkar
925c33abd1 chore(config): deprecate billing_tier / anonymous_enabled, split anon flags 2026-05-01 17:42:44 +05:30
CREDO23
c077939522 Gate chat on MULTI_AGENT_CHAT_ENABLED and route to the multi-agent graph when on. 2026-04-30 16:43:31 +02:00
CREDO23
dcae196eea Revert "feat(chat): add multi-agent mode routing scaffold and telemetry."
This reverts commit 7b9a218d62.
2026-04-29 16:21:57 +02:00
CREDO23
7b9a218d62 feat(chat): add multi-agent mode routing scaffold and telemetry. 2026-04-28 15:35:14 +02:00
Anish Sarkar
1eadecee23 feat(new-chat): integrate filesystem flow into agent pipeline 2026-04-23 15:45:33 +05:30
DESKTOP-RTLN3BA\$punk
0e4285095c fix: llamaclud v2 impl 2026-04-16 01:15:47 -07:00
DESKTOP-RTLN3BA\$punk
4a51ccdc2c cloud: added openrouter integration with global configs 2026-04-15 23:46:29 -07:00
DESKTOP-RTLN3BA\$punk
ff4e0f9b62 feat: no login experience and prem tokens
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
2026-04-15 17:02:00 -07:00
Anish Sarkar
6038f6dfc0 Merge remote-tracking branch 'upstream/dev' into feat/azure-ocr 2026-04-08 05:00:32 +05:30
Anish Sarkar
20fa93f0ba refactor: make Azure Document Intelligence an internal LLAMACLOUD accelerator instead of a standalone ETL service 2026-04-08 03:26:24 +05:30
CREDO23
26bffbcc47 Add dynamic vision model list from OpenRouter with combobox selector 2026-04-07 23:39:52 +02:00
CREDO23
36b8a84b0b Add vision LLM config examples to global_llm_config.example.yaml 2026-04-07 21:55:58 +02:00
Anish Sarkar
1fa8d1220b feat: add support for Azure Document Intelligence in ETL pipeline 2026-04-08 00:59:12 +05:30
CREDO23
7448f27ee0 Add vision LLM config loading and router initialization to Config 2026-04-07 19:02:18 +02:00
Anish Sarkar
000c2d9b5b style: simplify LLM model terminology in UI 2026-04-02 10:11:35 +05:30
DESKTOP-RTLN3BA\$punk
a9fd45844d feat: integrate Stripe for page purchases and reconciliation tasks 2026-03-31 18:39:45 -07:00
Anish Sarkar
54828522f8 feat: add Dropbox integration support with configuration and connector updates 2026-03-30 22:16:41 +05:30
Anish Sarkar
7004e764a9 chore: refactor Microsoft OAuth configuration to unify client ID and secret for Teams and OneDrive in environment files and related code 2026-03-28 16:37:23 +05:30
Anish Sarkar
5bddde60cb feat: implement Microsoft OneDrive connector with OAuth support and indexing capabilities 2026-03-28 14:31:25 +05:30
Anish Sarkar
60d12b0a70 Merge remote-tracking branch 'upstream/dev' into feat/web-search 2026-03-16 19:34:38 +05:30
Anish Sarkar
d40c6bf743 feat: integrate SearXNG web search service with platform capabilities
- Added SearXNG service configuration to Docker setup, including environment variables and health checks.
- Introduced new settings management for web search in the frontend, allowing users to enable/disable and configure search engines and language preferences.
- Updated backend to support web search functionality, including database schema changes and service integration.
- Implemented health check endpoint for the web search service and integrated it into the application.
- Removed legacy SearXNG API connector references in favor of the new platform service approach.
2026-03-14 20:25:25 +05:30