Commit graph

2324 commits

Author SHA1 Message Date
CREDO23
4e4f7f34fa feat(index-cache): add TTL/size eviction task and daily schedule 2026-06-12 16:48:18 +02:00
CREDO23
019aa7bf76 feat(index-cache): serve chunk embeddings from cache during indexing 2026-06-12 16:48:18 +02:00
CREDO23
e8938c119b feat(index-cache): add recall/remember service 2026-06-12 16:48:10 +02:00
CREDO23
4d6378e031 feat(observability): add index cache hit/miss and eviction metrics 2026-06-12 16:48:10 +02:00
CREDO23
daccd304ee feat(index-cache): add settings, eligibility, and config flags 2026-06-12 16:48:10 +02:00
CREDO23
ad6da7c6af feat(index-cache): add embedding blob store sharing the cache backend 2026-06-12 16:48:01 +02:00
CREDO23
f541114544 feat(index-cache): add cached embedding set table and repository 2026-06-12 16:48:01 +02:00
CREDO23
59fa4c38c3 feat(index-cache): add pickle-free blob serialization 2026-06-12 16:48:01 +02:00
CREDO23
cf208365b4 feat(index-cache): add embedding set value objects 2026-06-12 16:48:01 +02:00
CREDO23
0fb1d3d37b feat(etl-cache): route all file-based sources through the parse cache
Every file ingestion path (Dropbox, Google Drive / Composio Drive, OneDrive,
local folder, Obsidian, and the legacy upload handlers) now parses via the
extract_with_cache facade instead of calling EtlPipelineService.extract
directly, so identical bytes are deduplicated globally regardless of source.
vision_llm is passed through, keeping the existing cacheability gate intact.
2026-06-12 14:47:25 +02:00
CREDO23
0808fbcdee feat(etl-cache): emit hit/miss and eviction metrics 2026-06-12 11:57:03 +02:00
CREDO23
9efe24879d feat(observability): add etl cache lookup and eviction metrics 2026-06-12 11:57:03 +02:00
CREDO23
ce1e90386f refactor(etl-cache): extract pure cacheability gate 2026-06-12 11:50:51 +02:00
CREDO23
0dc2ccc003 feat(tasks): route extraction through etl cache 2026-06-12 11:23:50 +02:00
CREDO23
1c05980ffb feat(celery): schedule etl cache eviction 2026-06-12 11:23:50 +02:00
CREDO23
9f29a885b1 feat(db): register CachedParse model 2026-06-12 11:23:50 +02:00
CREDO23
5c4eec26cc feat(config): add ETL_CACHE_* settings 2026-06-12 11:23:50 +02:00
CREDO23
324ba141a6 feat(etl-cache): add eviction task and public API 2026-06-12 11:23:40 +02:00
CREDO23
7ad39fd995 feat(etl-cache): add eviction policy 2026-06-12 11:23:40 +02:00
CREDO23
758da06c4f feat(etl-cache): add extract_with_cache 2026-06-12 11:23:40 +02:00
CREDO23
41dea96af4 feat(etl-cache): add EtlCacheService 2026-06-12 11:23:40 +02:00
CREDO23
87fdb37fa3 feat(etl-cache): expose storage layer 2026-06-12 11:23:40 +02:00
CREDO23
a6f2457c7c feat(etl-cache): add MarkdownCacheStore for cache blobs 2026-06-12 11:22:57 +02:00
CREDO23
217d040e9e feat(etl-cache): resolve cache blob storage backend 2026-06-12 11:22:57 +02:00
CREDO23
d9b1b491e9 feat(etl-cache): add cache blob object-key builder 2026-06-12 11:22:57 +02:00
CREDO23
8d3238bcd1 feat(etl-cache): expose cache persistence layer 2026-06-12 11:22:57 +02:00
CREDO23
ea10127979 feat(etl-cache): add CachedParseRepository data access 2026-06-12 11:22:57 +02:00
CREDO23
c624235780 feat(etl-cache): add CachedParse table model 2026-06-12 11:22:48 +02:00
CREDO23
205a63b9bc feat(etl-cache): add EtlCacheSettings resolved from config 2026-06-12 11:22:48 +02:00
CREDO23
b84debd999 feat(etl-cache): expose cache schema value objects 2026-06-12 11:22:48 +02:00
CREDO23
3c9ea0011d feat(etl-cache): add EvictionCandidate value object 2026-06-12 11:22:48 +02:00
CREDO23
24f824b597 feat(etl-cache): add ParseKey cache identity value object 2026-06-12 11:22:48 +02:00
Rohan Verma
4c28ba5295
Merge pull request #1487 from CREDO23/improvement-podcast-graph
[Feat] Podcast: Backend-owned language offering for the brief form
2026-06-12 00:58:02 -07:00
CREDO23
1ee38fc9ab feat(podcast): GET /podcasts/languages 2026-06-12 07:38:38 +02:00
CREDO23
fe4d69f478 feat(podcast): LanguageOptions read model 2026-06-12 07:38:38 +02:00
CREDO23
f3d253ae77 feat(podcast): export LanguageOffering 2026-06-12 07:38:38 +02:00
CREDO23
c8ee74b123 feat(podcast): offerable languages on the voice catalog 2026-06-12 07:38:38 +02:00
CREDO23
87be162d78 feat(podcast): curated common languages data 2026-06-12 07:38:38 +02:00
Anish Sarkar
ced1bb85ed feat(model-connections): implement bulk model update endpoint and related schema changes 2026-06-12 09:43:56 +05:30
DESKTOP-RTLN3BA\$punk
c855be8ccd fix(auto_reload): update task to use a lambda for user_id in async call 2026-06-11 16:51:18 -07:00
Anish Sarkar
8e8cf96faa feat(error-handling): implement LLM error adaptation and classification for chat streaming
- Introduced LLMErrorCategory and adapt_llm_exception to normalize LLM exceptions.
- Updated llm_retryable_message and llm_permanent_message to utilize the new adaptation logic.
- Enhanced classify_stream_exception to classify provider errors and return user-friendly messages.
- Added tests for error classification and adaptation to ensure robustness.
- Updated frontend error handling to display appropriate messages based on new classifications.
2026-06-12 05:03:14 +05:30
DESKTOP-RTLN3BA\$punk
cff721aa42 feat(migration): evolve podcast lifecycle by detaching from zero_publication and updating column handling 2026-06-11 16:17:14 -07:00
DESKTOP-RTLN3BA\$punk
05190da0a9 chore: linting 2026-06-11 15:31:43 -07:00
Anish Sarkar
203ef78346 Merge remote-tracking branch 'upstream/dev' into feat/unified-model-connections 2026-06-12 04:00:25 +05:30
CREDO23
7b30a76856 fix(gitignore): anchor data/ rule; track podcast voice catalogs 2026-06-12 00:06:37 +02:00
Anish Sarkar
aba95e4faf feat(database): enhance podcast lifecycle management by adding temporary unpublishing during migration 2026-06-12 03:35:49 +05:30
Anish Sarkar
908790e40f Merge remote-tracking branch 'upstream/dev' into feat/unified-model-connections 2026-06-12 03:15:28 +05:30
CREDO23
41f4a58663 Merge remote-tracking branch 'upstream/dev' into improvement-podcast-graph
# Conflicts:
#	surfsense_backend/app/tasks/celery_tasks/podcast_tasks.py
2026-06-11 23:14:49 +02:00
Anish Sarkar
3dd54230e7 fix(chat): normalize provider-safe message history 2026-06-12 02:17:37 +05:30
Anish Sarkar
5d5d574550 refactor(model-connections): move backend model connections to provider capabilities 2026-06-12 02:17:22 +05:30