Commit graph

54 commits

Author SHA1 Message Date
CREDO23
98b6977c68 permissions/ask: gate 'approve_always' palette entry on MCP-ness
Only MCP tools have a persistence target for 'approve_always' (the
connector's trusted-tools list); for native tools the decision lives
only in the in-memory runtime ruleset. Reflect that in the wire palette
so the FE can stay a pure renderer of allowed_decisions instead of
peeking at context.mcp_connector_id to decide whether to show the
'Always Allow' button.

The backend still accepts an 'approve_always' reply for any tool kind
(in-memory promotion is harmless), it just doesn't advertise it when
there's nowhere to persist.
2026-05-15 14:54:16 +02:00
CREDO23
c8b756ae8f hitl/wire: rename 'always' decision-type to 'approve_always'
Renames the SurfSense HITL extension decision-type from "always" to
"approve_always" so it sits in the same verb-first family as "approve",
"reject", and "edit". The Python constant is now SURFSENSE_DECISION_APPROVE_ALWAYS;
the wire value, the permission-domain decision_type, and the FE union members
all match (no wire/internal mismatch).

Both the multi_agent_chat permission middleware and the legacy new_chat one
accept the new wire value; the FE types.ts union is updated accordingly.

The "context.always" payload key is intentionally left untouched - it's the
patterns-to-promote field, semantically distinct from the decision type.
2026-05-15 14:47:32 +02:00
CREDO23
6671c91841 multi_agent_chat/permissions: persist 'always' decisions to trusted-tools list
Until now an "Always Allow" reply only updated the in-memory runtime
ruleset, evaporating after the session ended. Persist it to the
existing connector.config['trusted_tools'] list so the next session's
fetch_user_allowlist_rulesets picks it up and the user is never asked
again for the same (connector, tool) pair.

- TrustedToolSaver + make_trusted_tool_saver(user_id) in
  user_tool_allowlist: opens its own session via async_session_maker
  per call, logs and swallows failures (in-memory promotion is the
  canonical "always" path, durable persistence is opportunistic).

- PermissionMiddleware._process is now pure: returns
  (state_update, list[_AlwaysPromotion]). aafter_model awaits the
  saver for each promotion; after_model discards them. Promotions are
  only emitted for tools whose metadata exposes mcp_connector_id, so
  native tools and KB FS ops are correctly skipped.

- main_agent factory builds the saver once per turn and stashes it in
  dependencies["trusted_tool_saver"]; pack_subagent and the KB
  middleware stack forward it through build_permission_mw.

- Renamed pm._process(state, None) call sites in two existing tests to
  pm.after_model(state, None) so they exercise the public hook
  contract instead of the now-tuple-returning private method.
2026-05-15 14:07:08 +02:00
CREDO23
a97d1548a6 multi_agent_chat/permissions: surface MCP tool metadata into ask interrupts
The FE permission card needs mcp_connector_id, mcp_server, and
tool_description in the interrupt context to render "Always Allow"
against the right connected account. Thread the tool through the
ask pipeline:

- pack_subagent → build_permission_mw(tools=...) → PermissionMiddleware
  (tools_by_name) → request_permission_decision(tool=...) →
  build_permission_ask_payload(tool=...) projects card fields out of
  BaseTool.

- mcp_tool.py: stdio path now stashes mcp_connector_id in metadata for
  parity with the HTTP path.
2026-05-15 11:28:06 +02:00
CREDO23
ef1152b80e multi_agent_chat/permissions: layer user allow-list into subagent compile 2026-05-14 21:57:38 +02:00
CREDO23
d45dfbfbd6 multi_agent_chat: pack_subagent owns per-subagent PermissionMiddleware via Ruleset 2026-05-14 20:09:29 +02:00
CREDO23
0723702320 multi_agent_chat: real-graph regressions for unified HITL paths + format pass 2026-05-14 17:41:24 +02:00
CREDO23
a36b15b834 multi_agent_chat/middleware: tighten parallel-keying test with heterogeneous bundles and per-slice assertions 2026-05-14 10:11:51 +02:00
CREDO23
d69d2cc1fc multi_agent_chat/middleware: tighten heterogeneous slice arithmetic to (2,3) bundles 2026-05-14 10:05:04 +02:00
CREDO23
668b89927b multi_agent_chat/middleware: real-graph regression test for partial-pause parallel routing 2026-05-14 09:47:24 +02:00
CREDO23
8e10f38f32 multi_agent_chat/middleware: real-graph regression test for all-reject parallel routing 2026-05-14 09:36:03 +02:00
CREDO23
ca57b2106e multi_agent_chat/middleware: real-graph regression test for heterogeneous parallel decisions 2026-05-14 09:26:08 +02:00
CREDO23
0fd87ccb7f chat/stream_resume: key Command(resume=...) by Interrupt.id for parallel HITL 2026-05-13 20:59:57 +02:00
CREDO23
1001f56206 multi_agent_chat/middleware: parallel task tests and full bridge coverage 2026-05-13 19:57:57 +02:00
CREDO23
6fb011c95c multi_agent_chat/middleware: real-graph regression tests for interrupt stamping 2026-05-13 19:57:09 +02:00
CREDO23
fc2c5b6445 multi_agent_chat/middleware: per-call thread_id, tcid-keyed resume, decisions slicer 2026-05-13 19:56:51 +02:00
CREDO23
246dae40a8 Merge upstream/dev into feature/multi-agent 2026-05-12 21:23:37 +02:00
CREDO23
d843468256 multi_agent_chat/subagents: dict-keyed middleware_stack + always-on KB 2026-05-12 18:04:54 +02:00
DESKTOP-RTLN3BA\$punk
c8374e6c5b feat: improved document, folder mentions rendering
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
2026-05-09 22:15:51 -07:00
CREDO23
2ab6b1c757 Merge upstream/dev into feature/multi-agent. 2026-05-09 23:00:56 +02:00
CREDO23
78f4747382 refactor(chat): stream agent events via stream_output and remove parity v2 flag 2026-05-07 19:40:10 +02:00
DESKTOP-RTLN3BA\$punk
4e174f17f2 chore: linting 2026-05-05 17:08:34 -07:00
CREDO23
a3c3db2a18 test(multi-agent): pin first-wins assumption on pending subagent interrupts 2026-05-05 22:57:35 +02:00
CREDO23
73272ce348 refactor(multi-agent): switch compile graph to new orchestrator and drop deepagent_stack 2026-05-05 20:55:38 +02:00
CREDO23
9a4ee5d16b chore: trim narrative comments and docstrings 2026-05-05 18:27:46 +02:00
CREDO23
1745d7dccf feat(middleware): scope model fallback to provider/network errors only 2026-05-05 18:04:47 +02:00
CREDO23
f695298d30 feat(multi-agent): wire model fallback and retry into subagent middleware 2026-05-05 17:39:27 +02:00
CREDO23
fa6f3015a9 test(hitl): regression net for bundle decision shape preservation 2026-05-05 17:19:39 +02:00
CREDO23
0c55635603 test(hitl): regression net for fail-loud guard on missing resume value 2026-05-05 17:15:28 +02:00
CREDO23
681895d28d test(hitl): regression net for end-to-end resume bridge dispatch 2026-05-05 17:13:13 +02:00
CREDO23
102f77ab7f test(hitl): regression net for surfsense_resume_value pop-on-read 2026-05-05 17:04:19 +02:00
Rohan Verma
ce6d9233bc
Merge pull request #1335 from AnishSarkar22/fix/memory-extraction
refactor(memory): streamline memory extraction
2026-05-04 17:26:27 -07:00
CREDO23
5119915f4f Merge upstream/dev into feature/multi-agent 2026-05-05 01:44:46 +02:00
CREDO23
277bd50f37 Harden HITL for multi-step tasks: bypass internal MCP gate, full-args dedup, and decision-envelope normalization. 2026-05-04 19:25:27 +02:00
Anish Sarkar
b981b51ab1 Merge remote-tracking branch 'upstream/dev' into fix/memory-extraction 2026-05-04 12:03:44 +05:30
DESKTOP-RTLN3BA\$punk
a34f1fb25c feat: implement agent caches and fix invalid prompt cache configs
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
- Added a new function `_warm_agent_jit_caches` to pre-warm agent caches at startup, reducing cold invocation costs.
- Updated the `SurfSenseContextSchema` to include per-invocation fields for better state management during agent execution.
- Introduced caching mechanisms in various tools to ensure fresh database sessions are used, improving performance and reliability.
- Enhanced middleware to support new context features and improve error handling during connector and document type discovery.
2026-05-03 06:03:40 -07:00
DESKTOP-RTLN3BA\$punk
c938d39277 feat: moved most things behind correct feature flag 2026-05-02 23:10:48 -07:00
Anish Sarkar
9975e085aa refactor(memory): streamline memory extraction by utilizing extract_text_content utility 2026-05-02 16:10:30 +05:30
Rohan Verma
451a98936e
Merge pull request #1332 from AnishSarkar22/feat/model-pinnning-mode
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
feat: Auto-pin quality scoring, OpenRouter tier refactor and live usage sidebar
2026-05-01 15:57:19 -07:00
Anish Sarkar
25ccc959cf feat(busy_mutex): enhance thread lock management to prevent stale middleware interference 2026-05-02 01:35:30 +05:30
DESKTOP-RTLN3BA\$punk
e57c3a7d0c feat: prompt caching
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
- Updated `litellm` dependency version from `1.83.4` to `1.83.7`.
- Adjusted `aiohttp` version from `3.13.5` to `3.13.4` in the lock file.
- Implemented `apply_litellm_prompt_caching` in `chat_deepagent.py` to improve prompt caching.
- Added model name resolution logic in `chat_deepagent.py` to ensure correct provider-variant dispatch.
- Enhanced `llm_config.py` to configure prompt caching for various LLM providers.
- Updated tests to verify correct model name forwarding and prompt caching behavior.
2026-05-01 05:10:53 -07:00
CREDO23
5d3b8b9ca9 Merge remote-tracking branch 'upstream/dev' into feature/multi-agent 2026-05-01 00:05:20 +02:00
Anish Sarkar
af66fbf106 refactor(chat): implement turn cancellation and status management in new chat routes for improved user experience and error handling 2026-05-01 01:47:52 +05:30
CREDO23
bf9b606a61 Wire Linear and Slack specialists and prompt routing. 2026-04-29 20:51:06 +02:00
DESKTOP-RTLN3BA\$punk
c110f5b955 feat: improved agent streaming 2026-04-29 07:20:31 -07:00
DESKTOP-RTLN3BA\$punk
f9b5367754 chore: cleaned comments slop 2026-04-28 23:52:37 -07:00
DESKTOP-RTLN3BA\$punk
ca9bbee06d chore: linting 2026-04-28 21:37:51 -07:00
DESKTOP-RTLN3BA\$punk
b9a66cb417 feat: various UI fixes, prompt optimizations, and allowing duplicate docs
- Updated `content_hash` in the `Document` model to remove global uniqueness, allowing identical content across different paths.
- Enhanced `_create_document` function to handle path uniqueness and prevent session-poisoning from `IntegrityError`.
- Added detailed comments for clarity on the changes and their implications.
- Introduced new citation handling in the editor for improved user experience with citation jumps.
- Updated package dependencies in the frontend for better functionality.
2026-04-28 21:30:53 -07:00
DESKTOP-RTLN3BA\$punk
31a372bb84 feat: updated agent harness 2026-04-28 09:22:19 -07:00
DESKTOP-RTLN3BA\$punk
05ca4c0b9f feat: updated file management for main agent 2026-04-28 04:32:52 -07:00