Commit graph

557 commits

Author SHA1 Message Date
CREDO23
0c7e5dee8b test(podcast): align quota error kwargs with wallet refactor 2026-06-12 07:38:38 +02:00
CREDO23
402ae6befe test(podcast): languages endpoint 2026-06-12 07:38:38 +02:00
CREDO23
a19b7dd8e0 test(podcast): offerable languages catalog rules 2026-06-12 07:38:38 +02:00
Anish Sarkar
8e8cf96faa feat(error-handling): implement LLM error adaptation and classification for chat streaming
- Introduced LLMErrorCategory and adapt_llm_exception to normalize LLM exceptions.
- Updated llm_retryable_message and llm_permanent_message to utilize the new adaptation logic.
- Enhanced classify_stream_exception to classify provider errors and return user-friendly messages.
- Added tests for error classification and adaptation to ensure robustness.
- Updated frontend error handling to display appropriate messages based on new classifications.
2026-06-12 05:03:14 +05:30
DESKTOP-RTLN3BA\$punk
05190da0a9 chore: linting 2026-06-11 15:31:43 -07:00
Anish Sarkar
908790e40f Merge remote-tracking branch 'upstream/dev' into feat/unified-model-connections 2026-06-12 03:15:28 +05:30
CREDO23
41f4a58663 Merge remote-tracking branch 'upstream/dev' into improvement-podcast-graph
# Conflicts:
#	surfsense_backend/app/tasks/celery_tasks/podcast_tasks.py
2026-06-11 23:14:49 +02:00
Anish Sarkar
3dd54230e7 fix(chat): normalize provider-safe message history 2026-06-12 02:17:37 +05:30
Anish Sarkar
831ad23c6c fix(chat): harden image generation model routing 2026-06-11 18:22:45 +05:30
Anish Sarkar
c28c4f5785 feat(chat): route models by provider capabilities 2026-06-11 18:22:23 +05:30
Anish Sarkar
8f20a32571 refactor(model-connections): consolidate provider capability handling 2026-06-11 18:21:07 +05:30
Anish Sarkar
c6a25cc1fe refactor(model-connections): streamline global model config persistence 2026-06-11 18:20:53 +05:30
CREDO23
ca9b157676 fix(podcasts): keep legacy episodes readable and guard regenerate 2026-06-11 12:43:07 +02:00
CREDO23
aa7f14d94f feat(podcasts): add revert-regeneration and surface cancel on the live card 2026-06-11 12:31:42 +02:00
CREDO23
f0fc660d70 feat(podcasts): constrain monologue briefs to a single speaker 2026-06-11 11:56:57 +02:00
CREDO23
eb56acc407 refactor(podcasts): regenerate via brief gate, render brief inline in chat 2026-06-11 11:45:17 +02:00
CREDO23
11a6b178a0 refactor(podcasts): drop transcript gate, add regenerate-from-ready and voice previews 2026-06-11 10:42:13 +02:00
CREDO23
c84525897b test(podcasts): relocate stateful tests to integration
Move the lifecycle service, Celery task bodies, and mark_failed coverage out of
DB-faking unit tests and into integration tests against a real Postgres, faking
only true externals (broker, object store, TTS, ffmpeg, billing, LLM). Add HTTP
slices for cancel, voices, scoping, and public-chat streaming. The unit tier is
now fake-free pure logic with no session doubles.
2026-06-11 06:27:00 +02:00
DESKTOP-RTLN3BA\$punk
a7407502d3 feat(refactor): refactor payment system to implement unified credit wallet.
- Updated environment variables and - configurations for credit purchases via Stripe, replacing legacy page pack system.
- Introduced auto-reload feature for credit top-ups and modified database models to track credit transactions.
- Updated notification system to handle insufficient credits and auto-reload failures.
- Adjusted API routes and schemas to reflect changes in credit management.
2026-06-10 16:49:03 -07:00
CREDO23
8f38737ad9 test(podcasts): retarget celery and observability tests to new tasks 2026-06-10 21:45:04 +02:00
CREDO23
aa7aa81c16 refactor(podcasts): drop language detection from brief 2026-06-10 20:51:38 +02:00
CREDO23
15e44616f3 test(podcasts): cover drafting billing gate 2026-06-10 18:44:26 +02:00
CREDO23
0bed4a0d38 test(podcasts): cover failure recording 2026-06-10 18:44:25 +02:00
CREDO23
0c7987cd9e test(podcasts): cover api read model 2026-06-10 18:44:25 +02:00
CREDO23
fa7ab8a06d test(podcasts): cover renderer validation 2026-06-10 18:44:25 +02:00
CREDO23
36c201f9e2 test(podcasts): cover structured json parsing 2026-06-10 18:44:25 +02:00
CREDO23
0c92ee963e test(podcasts): cover voice catalog 2026-06-10 18:44:25 +02:00
CREDO23
e926990d8e test(podcasts): cover language and voice resolution 2026-06-10 18:44:25 +02:00
CREDO23
aaa9f01087 test(podcasts): cover brief and transcript contracts 2026-06-10 18:44:25 +02:00
CREDO23
9d8e4e4f9d test(podcasts): cover lifecycle state machine 2026-06-10 18:44:25 +02:00
CREDO23
f61e8af8c0 test(podcasts): add shared test fixtures 2026-06-10 18:44:25 +02:00
Anish Sarkar
adb857925b feat(models): add model connection persistence 2026-06-10 21:47:23 +05:30
CREDO23
59c1cf14c7 test(indexers): cover mark_connector_documents_failed behavior 2026-06-10 00:11:00 +02:00
CREDO23
77544ab768 test(google-drive): assert stuck pending/processing docs retry 2026-06-10 00:11:00 +02:00
CREDO23
9f76daec8f test(indexers): update download mock return shape 2026-06-09 23:39:25 +02:00
CREDO23
bdd3728c5b test(dropbox): update download failure return shape 2026-06-09 23:39:25 +02:00
CREDO23
b5aa41beb6 test(onedrive): update download failure return shape 2026-06-09 23:39:25 +02:00
CREDO23
5f59ad3ad3 test(google-drive): update download failure return shape 2026-06-09 23:39:25 +02:00
DESKTOP-RTLN3BA\$punk
ce952d2ad1 chore: linting 2026-06-09 00:42:26 -07:00
CREDO23
53a3920a82 fix(e2e): load .env after harness env defaults 2026-06-05 19:24:26 +02:00
CREDO23
8bdfd00a15 Merge upstream/dev 2026-06-05 19:18:12 +02:00
CREDO23
52ff304d64 fix(e2e): delegate connector work via task in fake LLM 2026-06-05 18:49:57 +02:00
CREDO23
bfadde93b7 fix(e2e): call .unique() when minting test token
The User mapper eager-loads the oauth_accounts collection via joined load
under AUTH_TYPE=GOOGLE, so the mint endpoint's query must call .unique()
before scalar_one_or_none() to avoid InvalidRequestError (500).
2026-06-05 18:17:11 +02:00
CREDO23
88fe213176 refactor(agents): extract subagent-invocation contract to subagents/shared
The knowledge_base subagent imported subagent_invoke_config + EXCLUDED_STATE_KEYS
from main_agent's checkpointed_subagent_middleware -- a subagent reaching into
main-agent internals. Both symbols (plus the recursion-limit constant they need)
are a subagent-invocation contract shared by the orchestrator's task middleware
and any nested-invoking subagent. Move them to subagents/shared/invocation.py;
config.py keeps the HITL resume side-channel and constants.py keeps the
main-agent tuning knobs. All consumers (task_tool, kb tool, tests) repointed.
2026-06-05 14:18:44 +02:00
CREDO23
0081b627e9 refactor(agents): move kb_persistence middleware into main_agent (owner)
The KB-persistence impl lived in shared/middleware/ but no subagent uses it --
consumers are the main_agent builder and the boundary event loop. Colocate with
its owner using the folder-per-middleware shape; __init__ re-exports the public
surface. Tests that reached module internals now alias the .middleware submodule.

  main_agent/middleware/kb_persistence.py -> kb_persistence/builder.py
  shared/middleware/kb_persistence.py     -> kb_persistence/middleware.py
2026-06-05 14:11:55 +02:00
CREDO23
a7a642fedc refactor(agents): move busy_mutex middleware into main_agent (owner)
The busy-mutex impl (BusyMutexMiddleware + cancel/turn-lifecycle primitives)
lived in shared/middleware/ but no subagent uses it -- consumers are the
main_agent builder and the boundary (turn lifecycle). Colocate with its owner
using the folder-per-middleware shape; __init__ re-exports the public surface so
boundary import sites only change package path:

  main_agent/middleware/busy_mutex.py    -> busy_mutex/builder.py
  shared/middleware/busy_mutex.py        -> busy_mutex/middleware.py
2026-06-05 14:08:45 +02:00
CREDO23
84b775c0ac refactor(agents): unify permissions into one vertical-slice package
Per-file verification of the slice-3 candidates showed receipts/ and
date_filters.py are shared contracts (consumed by shared/state + shared
middleware + subagents), so they correctly stay put.

permissions was the real misfit: the rule *model* lived at shared/permissions.py
while its enforcement lived at shared/middleware/permissions/. Unify them into a
single self-contained subsystem:

  shared/permissions.py                 -> shared/permissions/model.py
  shared/middleware/permissions/{deny,ask,middleware}
                                        -> shared/permissions/{deny,ask,middleware}

The package __init__ re-exports the model API + build_permission_mw, so the 32
external model consumers keep importing `from ...shared.permissions import Rule`
unchanged; only the 8 internal files redirect to `.model` (cycle-safe, model
loaded before middleware).
2026-06-05 13:29:48 +02:00
CREDO23
f2a61bc0ef refactor(agents): consolidate chat runtime infra under chat/runtime
Move the lower-level runtime/infra modules out of multi_agent_chat/shared/
(they were never used by subagents, so they failed the shared-by-all-siblings
rule) and unify them with the already-relocated checkpointer:

  agents/runtime/                      -> agents/chat/runtime/
  mac/shared/errors.py                 -> chat/runtime/errors.py
  mac/shared/llm_config.py             -> chat/runtime/llm_config.py
  mac/shared/prompt_caching.py         -> chat/runtime/prompt_caching.py
  mac/shared/mention_resolver.py       -> chat/runtime/mention_resolver.py
  mac/shared/path_resolver.py          -> chat/runtime/path_resolver.py

These sit below the agent packages: the boundary + agent factory + shared
middleware depend on them, and they import no agent code (acyclic).
2026-06-05 13:19:24 +02:00
CREDO23
24b62a63b4 refactor(agents): introduce chat/ category; dissolve top-level agents/shared
Recursive shared-folder rule: a shared/ must be shared by ALL siblings at its
level. The kernel (context, compaction, retry_after, web_search) was shared by
only 2 of the agents -- anonymous_chat + multi_agent_chat -- never by podcaster
or video_presentation. Those 2 are the "chat" category, so their shared code
belongs in that category's shared/, not the top-level one.

  app/agents/anonymous_chat/   -> app/agents/chat/anonymous_chat/
  app/agents/multi_agent_chat/ -> app/agents/chat/multi_agent_chat/
  app/agents/shared/           -> app/agents/chat/shared/   (anon<->mac kernel)

Top-level app/agents/shared/ is gone: nothing was shared across all three
categories (chat / podcaster / video_presentation).

~289 import sites rewritten (app.agents.{anonymous_chat,multi_agent_chat,shared}
-> app.agents.chat.*); all moves are git renames (history preserved).
app/agents/ now: chat/, podcaster/, video_presentation/, runtime/.
2026-06-05 12:54:02 +02:00
CREDO23
d59bb2b5aa refactor(agents): evict mac-only tools/middleware from shared kernel
These were never shared with anonymous_chat (nor podcaster/video_presentation)
-- only multi_agent_chat (subagents/main agent) and the boundary use them:

  shared/tools/mcp/             -> multi_agent_chat/shared/tools/mcp/
  shared/tools/hitl.py          -> multi_agent_chat/shared/tools/hitl.py
  shared/tools/catalog.py       -> multi_agent_chat/shared/tools/catalog.py
  shared/middleware/dedup_tool_calls.py
                                -> multi_agent_chat/shared/middleware/dedup_tool_calls.py

app/agents/shared/ now holds only the genuine anon<->mac kernel:
context, middleware/{compaction,retry_after}, tools/web_search.
2026-06-05 12:50:46 +02:00