Vertical-slice colocation: all main-agent code should live under
main_agent/ instead of being split across a parallel middleware/main_agent
tree. Move multi_agent_chat/middleware/main_agent/ -> main_agent/middleware/
and its assembler middleware/stack.py -> main_agent/middleware/stack.py, so
the main-agent slice is self-contained (graph, runtime, system_prompt, tools,
middleware).
Genuinely cross-slice middleware (middleware/shared/, middleware/subagent/)
stays under multi_agent_chat/middleware/ for a later slice; the moved builders
now reference it via absolute imports.
Pure move + import rewrite (git-tracked renames). Verified: full unit suite
green (2430 passed, 1 skipped), including test_import_all and the
checkpointed-subagent middleware suite.
- skills/ (builtin SKILL.md assets) has zero Python importers; it is read by
filesystem path only. Moved the dir and restored
skills_backends._default_builtin_root() to the clean
parent.parent / "skills" / "builtin" form (undoing the transient path from 5c).
- plugin_loader.py -> shared (frozen chat_deepagent uses it -> re-export shim).
- plugins/ package -> shared (year_substituter rewired to shared.plugin_loader;
docstring entry-point example updated to the shared dotted path). No shim
needed (only a test imported it). Plugin discovery is via importlib entry
points (group "surfsense.plugins"), not dotted-path import, and nothing is
registered in pyproject, so the move does not affect runtime discovery.
Relocate the entire new_chat/tools/ package (62 files incl. registry, hitl, MCP
cluster, and all connector subpackages: gmail/slack/discord/teams/drive/etc.)
to the shared kernel. The package turned out to be a clean cohesive cluster:
its only references to non-tools new_chat modules were comments, and its
middleware deps were already flipped to shared in slice 5c.
Flip 33 live importers (multi-agent, flows, routes, services, anonymous_agent,
tests). Re-export shims remain for the frozen single-agent stack: a package
__init__ mirroring the public surface (new_chat.__init__ imports it) plus
invalid_tool + registry submodule shims (chat_deepagent imports those).
Resolves slice 5c's two transient back-edges: shared/middleware/action_log
(TYPE_CHECKING ToolDefinition) and tool_call_repair (local INVALID_TOOL_NAME)
now point at app.agents.shared.tools.
Relocate the entire new_chat/middleware/ package to the shared kernel as one
cohesive unit (it is live shared infrastructure: the multi-agent stack wraps
nearly every middleware via multi_agent_chat/middleware/main_agent/*, and
anonymous_agent consumes it too). Flip 69 live importers across both the
package-path and submodule-path forms.
Shims left for the frozen single-agent stack: a package __init__ re-export plus
submodule shims for permission, skills_backends, and scoped_model_fallback
(the three imported via submodule path by chat_deepagent/subagents).
Cycle break: importing shared.middleware previously reached back into
new_chat.tools at module load, which dragged in new_chat.__init__ ->
chat_deepagent -> the middleware shim -> half-initialized shared.middleware.
Made action_log's ToolDefinition import TYPE_CHECKING-only and
tool_call_repair's INVALID_TOOL_NAME import function-local. These tools-package
back-edges fully resolve in slice 6.
Asset note: skills_backends._default_builtin_root now walks to
app/agents/new_chat/skills/builtin (the skills/ tree migrates in slice 7).
Relocate the permission evaluator (wildcard matcher + rule evaluation) to the
shared kernel and flip 43 non-frozen importers. A re-export shim remains at
new_chat/permissions.py for the frozen single-agent stack (chat_deepagent and
subagents/{config,providers/linear,providers/slack}); it will be removed when
that stack is retired.
Relocate three leaf filesystem-cluster modules to the shared kernel and flip
all 38 importers. No re-export shims needed (no frozen single-agent importer).
This also resolves the pre-existing shared->new_chat back-edge from
shared/receipt_command.py onto filesystem_state.
filesystem_backends is intentionally deferred to slice 5: it depends on
new_chat middleware (kb_postgres_backend, multi_root_local_folder_backend)
that have not yet moved, so relocating it now would create a shared->new_chat edge.
Promote the filesystem mode contracts (FilesystemMode, FilesystemSelection,
ClientPlatform, LocalFilesystemMount) out of `new_chat` into the cross-agent
`app/agents/shared` kernel.
Pure leaf consumed across the whole multi-agent filesystem middleware/tool tree,
the chat flows/monolith, routes and tests. git mv (content unchanged) + flipped
all ~48 importers. A re-export shim remains at new_chat/filesystem_selection.py
only for the not-yet-retired single-agent (chat_deepagent).
Also updated the stream parity test's annotation normalizer to strip the new
app.agents.shared.filesystem_selection. prefix (the dataclasses' __module__
changed with the move), keeping monolith<->flows signature parity intact.
Behavior-preserving: only import paths change. 1326 tests green.
Promote the agent feature-flag resolver (AgentFeatureFlags / get_flags) out of
`new_chat` into the cross-agent `app/agents/shared` kernel.
feature_flags is a pure leaf consumed across the multi-agent middleware stack,
the chat routes, and tests. Moved it via git mv (content unchanged) and flipped
all 37 importers to app.agents.shared.feature_flags. A thin re-export shim
remains at new_chat/feature_flags.py only for the not-yet-retired single-agent
(chat_deepagent); it goes away with the single-agent deletion.
Behavior-preserving: only import paths change. 1243 tests green.
Continue promoting the shared agent toolkit out of `new_chat` into the
cross-agent `app/agents/shared` kernel.
- state_reducers.py: clean move (no single-agent importer); all 7 importers
flipped to app.agents.shared.state_reducers.
- context.py: moved to app.agents.shared.context; flipped the multi-agent,
app, automations, chat-flows and monolith importers. A thin re-export shim
remains at new_chat/context.py because the not-yet-retired single-agent
(chat_deepagent) and the new_chat package __init__ still import it; the shim
goes away with the single-agent deletion.
- Updated the stream parity test's annotation normalizer to strip the new
app.agents.shared.context. prefix (SurfSenseContextSchema.__module__ changed
with the move), keeping monolith<->flows signature parity intact.
Behavior-preserving: definitions unchanged; only import paths move. 1219 tests green.
First slice of promoting the shared agent toolkit out of the misnamed
`new_chat` package into the cross-agent `app/agents/shared` kernel.
`errors.py` is a leaf module (no intra-package deps) consumed by the
multi-agent chat, the chat streaming flows/monolith, and tests — i.e. it is
shared infrastructure, not single-agent code. Moved it verbatim to
`app.agents.shared.errors` and flipped all 12 importers. No re-export shim
remains since zero importers needed it.
Behavior-preserving: identical class/enum definitions; only the import path
changes. 1208 agent + chat-task tests green.
- Added new environment variables for controlling task execution limits, including `SURFSENSE_SUBAGENT_INVOKE_TIMEOUT_SECONDS`, `SURFSENSE_TASK_BATCH_CONCURRENCY`, and `SURFSENSE_TASK_BATCH_MAX_SIZE`.
- Updated documentation to reflect new batch processing capabilities for `task` calls, allowing for concurrent execution of multiple subagent tasks.
- Improved error handling and receipt generation for deliverables, ensuring consistent feedback on task status.
- Refactored middleware to incorporate search space ID for better task management.
Adds an optional planner LLM role wired through KnowledgePriorityMiddleware
so KB query rewriting, date extraction, and recency classification run on a
cheap model (e.g. gpt-4o-mini, Haiku, Azure nano) instead of the user's
chat LLM. Operators opt in by setting is_planner: true on exactly one
global config; without it, behavior is unchanged.
Only MCP tools have a persistence target for 'approve_always' (the
connector's trusted-tools list); for native tools the decision lives
only in the in-memory runtime ruleset. Reflect that in the wire palette
so the FE can stay a pure renderer of allowed_decisions instead of
peeking at context.mcp_connector_id to decide whether to show the
'Always Allow' button.
The backend still accepts an 'approve_always' reply for any tool kind
(in-memory promotion is harmless), it just doesn't advertise it when
there's nowhere to persist.
Renames the SurfSense HITL extension decision-type from "always" to
"approve_always" so it sits in the same verb-first family as "approve",
"reject", and "edit". The Python constant is now SURFSENSE_DECISION_APPROVE_ALWAYS;
the wire value, the permission-domain decision_type, and the FE union members
all match (no wire/internal mismatch).
Both the multi_agent_chat permission middleware and the legacy new_chat one
accept the new wire value; the FE types.ts union is updated accordingly.
The "context.always" payload key is intentionally left untouched - it's the
patterns-to-promote field, semantically distinct from the decision type.
Until now an "Always Allow" reply only updated the in-memory runtime
ruleset, evaporating after the session ended. Persist it to the
existing connector.config['trusted_tools'] list so the next session's
fetch_user_allowlist_rulesets picks it up and the user is never asked
again for the same (connector, tool) pair.
- TrustedToolSaver + make_trusted_tool_saver(user_id) in
user_tool_allowlist: opens its own session via async_session_maker
per call, logs and swallows failures (in-memory promotion is the
canonical "always" path, durable persistence is opportunistic).
- PermissionMiddleware._process is now pure: returns
(state_update, list[_AlwaysPromotion]). aafter_model awaits the
saver for each promotion; after_model discards them. Promotions are
only emitted for tools whose metadata exposes mcp_connector_id, so
native tools and KB FS ops are correctly skipped.
- main_agent factory builds the saver once per turn and stashes it in
dependencies["trusted_tool_saver"]; pack_subagent and the KB
middleware stack forward it through build_permission_mw.
- Renamed pm._process(state, None) call sites in two existing tests to
pm.after_model(state, None) so they exercise the public hook
contract instead of the now-tuple-returning private method.
The FE permission card needs mcp_connector_id, mcp_server, and
tool_description in the interrupt context to render "Always Allow"
against the right connected account. Thread the tool through the
ask pipeline:
- pack_subagent → build_permission_mw(tools=...) → PermissionMiddleware
(tools_by_name) → request_permission_decision(tool=...) →
build_permission_ask_payload(tool=...) projects card fields out of
BaseTool.
- mcp_tool.py: stdio path now stashes mcp_connector_id in metadata for
parity with the HTTP path.
