CREDO23
c6e71c851c
feat(chunks): add explicit position column with backfill migration
...
Chunk ids stop reflecting document order once incremental re-indexing keeps
unchanged rows across edits. Backfill preserves the historical id ordering
so behavior is identical on day one.
2026-06-12 18:52:45 +02:00
CREDO23
91d947ff79
refactor(embedding-cache): rename index cache to embedding cache
...
The cached payload is the indexing pipeline's embeddings (markdown is
chunked then embedded), so "embedding cache" names the expensive output
directly and removes the "index" ambiguity (DB index vs vector index vs
indexing phase). Renames the service, settings, eligibility, eviction
task, metrics, config flags (INDEX_CACHE_* -> EMBEDDING_CACHE_*), object
prefix, and the table (index_cache_embedding_sets -> embedding_cache_sets)
with its constraint and indexes. Migration 161 renamed accordingly.
2026-06-12 17:00:01 +02:00
CREDO23
4e4f7f34fa
feat(index-cache): add TTL/size eviction task and daily schedule
2026-06-12 16:48:18 +02:00
CREDO23
019aa7bf76
feat(index-cache): serve chunk embeddings from cache during indexing
2026-06-12 16:48:18 +02:00
CREDO23
e8938c119b
feat(index-cache): add recall/remember service
2026-06-12 16:48:10 +02:00
CREDO23
4d6378e031
feat(observability): add index cache hit/miss and eviction metrics
2026-06-12 16:48:10 +02:00
CREDO23
daccd304ee
feat(index-cache): add settings, eligibility, and config flags
2026-06-12 16:48:10 +02:00
CREDO23
ad6da7c6af
feat(index-cache): add embedding blob store sharing the cache backend
2026-06-12 16:48:01 +02:00
CREDO23
f541114544
feat(index-cache): add cached embedding set table and repository
2026-06-12 16:48:01 +02:00
CREDO23
59fa4c38c3
feat(index-cache): add pickle-free blob serialization
2026-06-12 16:48:01 +02:00
CREDO23
cf208365b4
feat(index-cache): add embedding set value objects
2026-06-12 16:48:01 +02:00
CREDO23
0fb1d3d37b
feat(etl-cache): route all file-based sources through the parse cache
...
Every file ingestion path (Dropbox, Google Drive / Composio Drive, OneDrive,
local folder, Obsidian, and the legacy upload handlers) now parses via the
extract_with_cache facade instead of calling EtlPipelineService.extract
directly, so identical bytes are deduplicated globally regardless of source.
vision_llm is passed through, keeping the existing cacheability gate intact.
2026-06-12 14:47:25 +02:00
CREDO23
0808fbcdee
feat(etl-cache): emit hit/miss and eviction metrics
2026-06-12 11:57:03 +02:00
CREDO23
9efe24879d
feat(observability): add etl cache lookup and eviction metrics
2026-06-12 11:57:03 +02:00
CREDO23
ce1e90386f
refactor(etl-cache): extract pure cacheability gate
2026-06-12 11:50:51 +02:00
CREDO23
0dc2ccc003
feat(tasks): route extraction through etl cache
2026-06-12 11:23:50 +02:00
CREDO23
1c05980ffb
feat(celery): schedule etl cache eviction
2026-06-12 11:23:50 +02:00
CREDO23
9f29a885b1
feat(db): register CachedParse model
2026-06-12 11:23:50 +02:00
CREDO23
5c4eec26cc
feat(config): add ETL_CACHE_* settings
2026-06-12 11:23:50 +02:00
CREDO23
324ba141a6
feat(etl-cache): add eviction task and public API
2026-06-12 11:23:40 +02:00
CREDO23
7ad39fd995
feat(etl-cache): add eviction policy
2026-06-12 11:23:40 +02:00
CREDO23
758da06c4f
feat(etl-cache): add extract_with_cache
2026-06-12 11:23:40 +02:00
CREDO23
41dea96af4
feat(etl-cache): add EtlCacheService
2026-06-12 11:23:40 +02:00
CREDO23
87fdb37fa3
feat(etl-cache): expose storage layer
2026-06-12 11:23:40 +02:00
CREDO23
a6f2457c7c
feat(etl-cache): add MarkdownCacheStore for cache blobs
2026-06-12 11:22:57 +02:00
CREDO23
217d040e9e
feat(etl-cache): resolve cache blob storage backend
2026-06-12 11:22:57 +02:00
CREDO23
d9b1b491e9
feat(etl-cache): add cache blob object-key builder
2026-06-12 11:22:57 +02:00
CREDO23
8d3238bcd1
feat(etl-cache): expose cache persistence layer
2026-06-12 11:22:57 +02:00
CREDO23
ea10127979
feat(etl-cache): add CachedParseRepository data access
2026-06-12 11:22:57 +02:00
CREDO23
c624235780
feat(etl-cache): add CachedParse table model
2026-06-12 11:22:48 +02:00
CREDO23
205a63b9bc
feat(etl-cache): add EtlCacheSettings resolved from config
2026-06-12 11:22:48 +02:00
CREDO23
b84debd999
feat(etl-cache): expose cache schema value objects
2026-06-12 11:22:48 +02:00
CREDO23
3c9ea0011d
feat(etl-cache): add EvictionCandidate value object
2026-06-12 11:22:48 +02:00
CREDO23
24f824b597
feat(etl-cache): add ParseKey cache identity value object
2026-06-12 11:22:48 +02:00
Rohan Verma
4c28ba5295
Merge pull request #1487 from CREDO23/improvement-podcast-graph
...
[Feat] Podcast: Backend-owned language offering for the brief form
2026-06-12 00:58:02 -07:00
CREDO23
1ee38fc9ab
feat(podcast): GET /podcasts/languages
2026-06-12 07:38:38 +02:00
CREDO23
fe4d69f478
feat(podcast): LanguageOptions read model
2026-06-12 07:38:38 +02:00
CREDO23
f3d253ae77
feat(podcast): export LanguageOffering
2026-06-12 07:38:38 +02:00
CREDO23
c8ee74b123
feat(podcast): offerable languages on the voice catalog
2026-06-12 07:38:38 +02:00
CREDO23
87be162d78
feat(podcast): curated common languages data
2026-06-12 07:38:38 +02:00
Anish Sarkar
ced1bb85ed
feat(model-connections): implement bulk model update endpoint and related schema changes
2026-06-12 09:43:56 +05:30
DESKTOP-RTLN3BA\$punk
c855be8ccd
fix(auto_reload): update task to use a lambda for user_id in async call
2026-06-11 16:51:18 -07:00
Anish Sarkar
8e8cf96faa
feat(error-handling): implement LLM error adaptation and classification for chat streaming
...
- Introduced LLMErrorCategory and adapt_llm_exception to normalize LLM exceptions.
- Updated llm_retryable_message and llm_permanent_message to utilize the new adaptation logic.
- Enhanced classify_stream_exception to classify provider errors and return user-friendly messages.
- Added tests for error classification and adaptation to ensure robustness.
- Updated frontend error handling to display appropriate messages based on new classifications.
2026-06-12 05:03:14 +05:30
DESKTOP-RTLN3BA\$punk
cff721aa42
feat(migration): evolve podcast lifecycle by detaching from zero_publication and updating column handling
2026-06-11 16:17:14 -07:00
DESKTOP-RTLN3BA\$punk
05190da0a9
chore: linting
2026-06-11 15:31:43 -07:00
Anish Sarkar
203ef78346
Merge remote-tracking branch 'upstream/dev' into feat/unified-model-connections
2026-06-12 04:00:25 +05:30
CREDO23
7b30a76856
fix(gitignore): anchor data/ rule; track podcast voice catalogs
2026-06-12 00:06:37 +02:00
Anish Sarkar
aba95e4faf
feat(database): enhance podcast lifecycle management by adding temporary unpublishing during migration
2026-06-12 03:35:49 +05:30
Anish Sarkar
908790e40f
Merge remote-tracking branch 'upstream/dev' into feat/unified-model-connections
2026-06-12 03:15:28 +05:30
CREDO23
41f4a58663
Merge remote-tracking branch 'upstream/dev' into improvement-podcast-graph
...
# Conflicts:
# surfsense_backend/app/tasks/celery_tasks/podcast_tasks.py
2026-06-11 23:14:49 +02:00
