Commit graph

492 commits

Author SHA1 Message Date
CREDO23
e195fb77c5 test format_title helper 2026-06-17 15:06:05 +02:00
CREDO23
aee0f1ef7d add persist_scratch_index unit tests 2026-06-17 14:59:24 +02:00
CREDO23
a8a1f01945 update index batch parallel tests 2026-06-17 14:59:24 +02:00
DESKTOP-RTLN3BA\$punk
0fe650fd8e Merge commit '7ce409c580' into dev 2026-06-16 22:48:14 -07:00
Dmitry Maranik
81fc467187 test(connectors): regression tests for cross-search-space index authorization
Two integration tests pinning the connector index endpoint's authorization:

- cross-space index (attacker owns space B, connector lives in victim's
  space A, request passes search_space_id=B) is rejected with 404 at the
  search-space reconciliation, before the permission check (which would
  otherwise pass for the attacker's own space).
- same-space index authorizes check_permission against the connector's
  own search space, not the caller-supplied query param.

Mirrors the existing tests/integration harness (direct handler calls with
the savepoint-rolled-back db_session; check_permission patched so the test
needs no real RBAC wiring).
2026-06-16 16:18:40 -07:00
CREDO23
7a415b61ea test: align QuotaInsufficientError fixtures with balance_micros API
Billable calls now raise quota errors with balance_micros instead of
used_micros/limit_micros; update mocks so CI passes on main.
2026-06-16 23:56:11 +02:00
CREDO23
fd96c930bf test(podcasts): cover seconds duration and legacy minute specs 2026-06-16 23:38:28 +02:00
CREDO23
f997b6464e test(podcasts): update renderer test for second-based duration 2026-06-16 23:38:28 +02:00
CREDO23
cb70b64a70 test(podcasts): update unit fixtures for second-based duration 2026-06-16 23:38:28 +02:00
CREDO23
38991c7db8 test(podcasts): update integration fixtures for second-based duration 2026-06-16 23:38:28 +02:00
CREDO23
1048d0afc3 test(podcasts): cover public stream missing-object 404 2026-06-16 20:09:08 +02:00
CREDO23
810ded2dde test(podcasts): cover in-flight 409 and missing-object 404 2026-06-16 20:09:08 +02:00
CREDO23
86a8833fb4 test(podcasts): add exists to fake storage backend 2026-06-16 20:09:08 +02:00
Rohan Verma
69bdcf5946
Merge pull request #1491 from AnishSarkar22/feat/unified-model-connections
feat: Fix model attribution for prefix-stripped token usage callbacks
2026-06-14 17:50:48 -07:00
CREDO23
32a6e54ce6 Merge remote-tracking branch 'upstream/dev' into features/documents-injestion-layered-cached 2026-06-14 11:30:33 +02:00
Anish Sarkar
d9a4f14f99 feat(token-tracking): enhance model metadata reconciliation by adding bare model name handling 2026-06-14 12:18:22 +05:30
Anish Sarkar
c7409c8995 chore: ran linting 2026-06-13 21:59:35 +05:30
Anish Sarkar
ab5423d2d2 Merge remote-tracking branch 'upstream/dev' into feat/unified-model-connections 2026-06-13 19:04:49 +05:30
Anish Sarkar
e104193ddf refactor(provider-configuration): standardize provider parameter naming across various modules and improve quota error handling in tests 2026-06-13 14:23:32 +05:30
Anish Sarkar
4a6a282a46 feat(runtime-cooldown): implement Redis-based shared cooldown management for model selection 2026-06-13 13:53:01 +05:30
Anish Sarkar
bd4a04f2e7 feat(database-migrations): add migration to remove legacy model config tables and remove stale model connection code 2026-06-13 12:45:43 +05:30
CREDO23
dcebfc4756 Merge remote-tracking branch 'upstream/dev' into features/documents-injestion-layered-cached 2026-06-12 19:35:34 +02:00
CREDO23
311570b4f0 test(indexing): cover the edit path and make integration caches hermetic
Real-DB tests assert unchanged chunk rows survive edits, only new text is
embedded, removed rows are deleted with positions compacted, and the kill
switch restores full-replace. An autouse fixture disables the ETL/embedding
caches so a developer's .env can't leak cache hits into unrelated tests.
2026-06-12 18:53:21 +02:00
CREDO23
f82dedf712 feat(indexing): add pure chunk reconciler for content-addressed diffs
Greedy multiset match on chunk text decides which rows keep their embeddings,
which texts need embedding, and which rows are deleted. No DB, no embeddings;
fully unit-tested (reuse, head insert, middle edit, deletion, duplicates,
reorder, full rewrite).
2026-06-12 18:52:46 +02:00
CREDO23
412493ae08 test(embedding-cache): add integration tests for service, repository, and store
Covers the public cache surface against real Postgres and a real local file
backend (no mocks): recall miss, remember->recall vector/text/order round-trip,
the dimension-mismatch refusal, the repository SQL behind eviction and dedup
(size sum, coldest ordering, TTL cutoff, duplicate-key no-op, reuse counter),
and the blob store save/load round-trip and delete.
2026-06-12 17:33:21 +02:00
CREDO23
91d947ff79 refactor(embedding-cache): rename index cache to embedding cache
The cached payload is the indexing pipeline's embeddings (markdown is
chunked then embedded), so "embedding cache" names the expensive output
directly and removes the "index" ambiguity (DB index vs vector index vs
indexing phase). Renames the service, settings, eligibility, eviction
task, metrics, config flags (INDEX_CACHE_* -> EMBEDDING_CACHE_*), object
prefix, and the table (index_cache_embedding_sets -> embedding_cache_sets)
with its constraint and indexes. Migration 161 renamed accordingly.
2026-06-12 17:00:01 +02:00
CREDO23
8cf578d965 test(index-cache): add unit tests and repoint embed/chunk patch targets 2026-06-12 16:48:18 +02:00
CREDO23
99cf212c31 test: fix auth-mode mismatch and stale QuotaInsufficientError kwargs
Pin AUTH_TYPE=LOCAL (and REGISTRATION_ENABLED=TRUE) in the test bootstrap so
the email/password auth routers mount during integration tests regardless of a
developer's .env=GOOGLE; without this the upload tests 404 on registration.
Also update three tests to the current QuotaInsufficientError signature
(balance_micros) after used_micros/limit_micros were removed.
2026-06-12 12:19:49 +02:00
CREDO23
d5e0280097 test(etl-cache): cover two-phase eviction task on real infra 2026-06-12 11:54:36 +02:00
CREDO23
1460173dad test(etl-cache): cover extract_with_cache end-to-end 2026-06-12 11:50:57 +02:00
CREDO23
c49a0f1233 test(etl-cache): cover store, service, and repository on real infra 2026-06-12 11:50:57 +02:00
CREDO23
3dec3231d0 test(etl-cache): cover over-budget eviction selection 2026-06-12 11:50:52 +02:00
CREDO23
a3e7047c35 test(etl-cache): cover cacheability gate rules 2026-06-12 11:50:52 +02:00
CREDO23
dddacbe762 test(etl-cache): cover content-addressing dedup and key shape 2026-06-12 11:50:52 +02:00
Rohan Verma
4c28ba5295
Merge pull request #1487 from CREDO23/improvement-podcast-graph
[Feat] Podcast: Backend-owned language offering for the brief form
2026-06-12 00:58:02 -07:00
CREDO23
0c7e5dee8b test(podcast): align quota error kwargs with wallet refactor 2026-06-12 07:38:38 +02:00
CREDO23
402ae6befe test(podcast): languages endpoint 2026-06-12 07:38:38 +02:00
CREDO23
a19b7dd8e0 test(podcast): offerable languages catalog rules 2026-06-12 07:38:38 +02:00
Anish Sarkar
8e8cf96faa feat(error-handling): implement LLM error adaptation and classification for chat streaming
- Introduced LLMErrorCategory and adapt_llm_exception to normalize LLM exceptions.
- Updated llm_retryable_message and llm_permanent_message to utilize the new adaptation logic.
- Enhanced classify_stream_exception to classify provider errors and return user-friendly messages.
- Added tests for error classification and adaptation to ensure robustness.
- Updated frontend error handling to display appropriate messages based on new classifications.
2026-06-12 05:03:14 +05:30
DESKTOP-RTLN3BA\$punk
05190da0a9 chore: linting 2026-06-11 15:31:43 -07:00
Anish Sarkar
908790e40f Merge remote-tracking branch 'upstream/dev' into feat/unified-model-connections 2026-06-12 03:15:28 +05:30
CREDO23
41f4a58663 Merge remote-tracking branch 'upstream/dev' into improvement-podcast-graph
# Conflicts:
#	surfsense_backend/app/tasks/celery_tasks/podcast_tasks.py
2026-06-11 23:14:49 +02:00
Anish Sarkar
3dd54230e7 fix(chat): normalize provider-safe message history 2026-06-12 02:17:37 +05:30
Anish Sarkar
831ad23c6c fix(chat): harden image generation model routing 2026-06-11 18:22:45 +05:30
Anish Sarkar
c28c4f5785 feat(chat): route models by provider capabilities 2026-06-11 18:22:23 +05:30
Anish Sarkar
8f20a32571 refactor(model-connections): consolidate provider capability handling 2026-06-11 18:21:07 +05:30
Anish Sarkar
c6a25cc1fe refactor(model-connections): streamline global model config persistence 2026-06-11 18:20:53 +05:30
CREDO23
ca9b157676 fix(podcasts): keep legacy episodes readable and guard regenerate 2026-06-11 12:43:07 +02:00
CREDO23
aa7f14d94f feat(podcasts): add revert-regeneration and surface cancel on the live card 2026-06-11 12:31:42 +02:00
CREDO23
f0fc660d70 feat(podcasts): constrain monologue briefs to a single speaker 2026-06-11 11:56:57 +02:00