app/agents/shared/middleware/permission.py was an older, monolithic
PermissionMiddleware superseded by the modular permissions/ package under
multi_agent_chat/shared/middleware/ (core + evaluation + ask/ + factory).
Production wires only the package (main_agent stack + every subagent
builder); the kernel file was reachable only through the shared barrel
re-export (itself unused) and two tests pinned to its dead internals
(_raise_interrupt, _normalize_permission_decision, old after_model shape).
- delete app/agents/shared/middleware/permission.py
- drop PermissionMiddleware from the shared middleware barrel
- delete test_permission_middleware.py (covered the dead impl only; live
behavior is covered by tests/.../middleware/shared/permissions/*)
- test_desktop_safety_rules.py: keep the ruleset-level regression tests,
drop the dead import + TestPermissionMiddlewareIntegration class
knowledge_search, memory_injection and scoped_model_fallback no longer
belong in the cross-agent kernel (app/agents/shared/middleware): they are
consumed only inside multi_agent_chat. Relocate each impl next to the
builder that uses it:
- knowledge_search.py -> multi_agent_chat/shared/middleware/ (genuinely
shared: its _render_priority_message feeds kb_context_projection, used by
both the main agent and the KB subagent)
- memory_injection.py -> multi_agent_chat/shared/middleware/ (beside its
memory.py builder)
- scoped_model_fallback.py -> multi_agent_chat/shared/middleware/resilience/
(beside fallback.py/bundle.py)
Impls moved verbatim (git rename). Builders/consumers now import the local
sibling; main_agent knowledge_priority imports the new shared path; shared
middleware barrel trimmed.
Tests: repoint imports; convert the knowledge_search monkeypatch targets
from brittle dotted-string form to object-based patching (monkeypatch.setattr
on the imported module), which is robust to import ordering. No behavior
change.
Each main-agent-only middleware now lives in its own folder under
main_agent/middleware/<concept>/ with builder.py (flag-gated construction)
+ middleware.py (the impl), re-exported via __init__.py. This kills the
cross-folder hop into agents/shared/middleware and keeps each middleware's
two responsibilities (build vs behavior) as colocated siblings.
Moved (impl from shared/middleware, builder from main_agent/middleware):
action_log, anonymous_document, context_editing, doom_loop, knowledge_tree,
noop_injection, otel_span, tool_call_repair.
Impls moved verbatim (git rename, no body edits) so behavior is unchanged.
Builders now import from the local .middleware sibling. stack.py import
paths updated for the 3 renamed folders; shared middleware barrel trimmed;
tests repointed (imports + patch targets).
DedupHITLToolCallsMiddleware is only wired by the main_agent stack, but
its module also exports dedup-key resolvers consumed by the shared MCP
tool layer. Splitting keeps the resolvers (dedup_key_full_args,
wrap_dedup_key_by_arg_name, DedupResolver) in shared and moves the
middleware class verbatim into main_agent/middleware/dedup_hitl.py
(merged with its builder), eliminating the shared->main_agent dependency
that a flat move would create. No behavior change.
file_intent (FileIntentMiddleware) and flatten_system
(FlattenSystemMessageMiddleware) were only ever instantiated in the
single-agent chat_deepagent stack, which was removed in 14bbea085. They
have no production consumer in multi_agent_chat. Delete both modules and
their unit tests.
Also drop the vestigial KnowledgeBaseSearchMiddleware alias (= the live
KnowledgePriorityMiddleware); its tests now target the real class so the
behavior coverage is preserved. Trim the three barrel/__all__ entries and
strip the now-dead class names from comments.
The concrete filesystem backends are consumed only by the MAC filesystem
layer (tools, path-resolution middleware, the resolver, skills backend) and
tests -- no external app code. Group them next to the filesystem middleware
they serve:
- filesystem_backends.py -> filesystem/backends/resolver.py
- middleware/kb_postgres_backend.py -> filesystem/backends/kb_postgres.py
- middleware/local_folder_backend.py -> filesystem/backends/local_folder.py
- middleware/multi_root_local_folder_backend.py -> .../multi_root_local_folder.py
- document_xml.py -> filesystem/backends/document_xml.py
Repoint all 21 importers. No behavior change; import-all + filesystem
backend/path-resolution/knowledge-search unit tests stay green (478).
permissions.py (authorization Rule/Ruleset model) is consumed across all
MAC subagents + the permissions middleware, with a single external
consumer (user_tool_allowlist service) -> move to
multi_agent_chat/shared/permissions.py and repoint all 42 sites.
deliverable_wait.py (wait_for_deliverable) is used only by the podcast and
video_presentation deliverable tools -> colocate into
subagents/builtins/deliverables/.
No behavior change; import-all + permission/allowlist/deliverable unit
tests stay green.
receipt.py (Receipt model + make_receipt) and receipt_command.py
(with_receipt Command helper) are a tight pair used only by MAC subagent
tools, the graph state, and the kb_persistence middleware -- no external
code imports them (the streaming tool_end handler only references them in a
docstring). Move both into a dedicated receipts/ package
(receipts/receipt.py + receipts/command.py) and repoint importers.
No behavior change; import-all + receipt/deliverable unit tests stay green.
filesystem_state.py (the multi-agent graph state) and state_reducers.py
(its merge reducers) are consumed only by multi_agent_chat (filesystem
tools/middleware, kb projection, and the MAC-only shared middleware) plus
two unit tests -- no external app code. Relocate them into a dedicated
multi_agent_chat/shared/state/ package (filesystem_state.py + reducers.py)
and repoint every importer.
No behavior change; import-all + the full unit/middleware + unit/agents
suites (1066 tests) stay green.
shared/tools/knowledge_base.py had exactly one production consumer: the
report deliverable, which imported it via `from .knowledge_base import ...`
-- a sibling path that did not exist, so the report KB-search path would
raise ImportError at runtime.
Move the module next to report.py (subagents/builtins/deliverables/tools/)
which makes that relative import valid, and move its only dependency
(shared/utils.py date helpers) to multi_agent_chat/shared/date_filters.py,
shared between the KB tool and the knowledge_search middleware.
Drop the now-unused knowledge-base re-exports from the shared/tools barrel
and repoint the integration tests. import-all + error-contract stay green.
Move modules out of agents/shared/ that are consumed by a single package
(main_agent), placing each next to its only consumer instead of in a
"shared" grab-bag:
- agent_cache.py -> main_agent/runtime/agent_cache_store.py
- connector_searchable_types.py -> main_agent/runtime/
- plugin_loader.py + plugins/ -> main_agent/plugins/
- skills/ + skills_backends.py -> main_agent/skills/
- tools/invalid_tool.py -> main_agent/tools/
Drop the skills_backends re-export from the shared middleware barrel and
repoint all consumers + tests. No behavior change; import-all,
error-contract, and the moved tests stay green.
Replace the connector-coupled BUILTIN_TOOLS registry with a pure-data
catalog so shared/tools no longer imports any connector module, making the
connector packages independently deletable.
- add shared/tools/catalog.py (ToolMetadata + TOOL_CATALOG, 41 tools, no imports)
- point GET /agent/tools (the only live consumer) at the catalog
- relocate ToolDefinition into action_log middleware (its sole consumer);
drop the inert tool_definitions wiring (no tool defines reverse)
- delete shared/tools/registry.py: connector imports, dead factories,
dead get_connector_gated_tools, and BUILTIN_TOOLS
- drop stale dedup-propagation test (path removed in C1) + refresh docstrings
import-all guardrail + agents unit suite green (987 passed).
After the main agent moved to its own build_main_agent_tools, nothing calls
the shared registry's builders. Delete the dead functions (build_tools,
build_tools_async, get_tool_by_name, get_all_tool_names,
get_default_enabled_tools) plus the now-orphaned load_mcp_tools import and the
stale __init__ re-exports.
BUILTIN_TOOLS, ToolDefinition, and get_connector_gated_tools are retained:
the catalog is still consumed for tool *metadata* (action_log revert/dedup
resolvers and the /agent/tools listing). Also drop stale references to the
deleted chat_deepagent.py within the agents module.
Verified: full unit suite green (2431 passed, 1 skipped); lints clean.
The single-agent-era filesystem middleware (app/agents/shared/middleware/
filesystem.py, ~2000 lines) was never instantiated in production, yet three
unit suites validated it — an illusory guardrail while the live decomposed
middleware (multi_agent_chat/middleware/shared/filesystem) was unguarded.
Close the gap before reorganizing the agents module:
- Add 14 integration tests driving live B's tools in desktop mode (real
on-disk effects) and cloud mode (in-state staging, namespace policy).
- Port all high-value dead-twin assertions onto the live path: cloud rm/rmdir
staging + guard rails, KBPostgresBackend delete-view filter, mode-scoped
system prompt, cwd/relative/namespace resolution, multi-root mount
normalization.
- Delete dead twin filesystem.py, drop its __init__ re-export, and retire its
3 dead-twin tests.
Verified: test_import_all + middleware unit + FS integration all green.
After deleting app/agents/new_chat/, several shared-kernel comments still cited
new_chat paths/cycles. Update the two lazy-import comments in middleware to state
the real reason (tools.registry <-> shared.middleware cycle), and repoint dangling
``new_chat/tools/hitl.py`` / ``chat_deepagent`` doc references to their shared
locations. Comment-only; suite unaffected.
- skills/ (builtin SKILL.md assets) has zero Python importers; it is read by
filesystem path only. Moved the dir and restored
skills_backends._default_builtin_root() to the clean
parent.parent / "skills" / "builtin" form (undoing the transient path from 5c).
- plugin_loader.py -> shared (frozen chat_deepagent uses it -> re-export shim).
- plugins/ package -> shared (year_substituter rewired to shared.plugin_loader;
docstring entry-point example updated to the shared dotted path). No shim
needed (only a test imported it). Plugin discovery is via importlib entry
points (group "surfsense.plugins"), not dotted-path import, and nothing is
registered in pyproject, so the move does not affect runtime discovery.
Relocate the entire new_chat/tools/ package (62 files incl. registry, hitl, MCP
cluster, and all connector subpackages: gmail/slack/discord/teams/drive/etc.)
to the shared kernel. The package turned out to be a clean cohesive cluster:
its only references to non-tools new_chat modules were comments, and its
middleware deps were already flipped to shared in slice 5c.
Flip 33 live importers (multi-agent, flows, routes, services, anonymous_agent,
tests). Re-export shims remain for the frozen single-agent stack: a package
__init__ mirroring the public surface (new_chat.__init__ imports it) plus
invalid_tool + registry submodule shims (chat_deepagent imports those).
Resolves slice 5c's two transient back-edges: shared/middleware/action_log
(TYPE_CHECKING ToolDefinition) and tool_call_repair (local INVALID_TOOL_NAME)
now point at app.agents.shared.tools.
Relocate the entire new_chat/middleware/ package to the shared kernel as one
cohesive unit (it is live shared infrastructure: the multi-agent stack wraps
nearly every middleware via multi_agent_chat/middleware/main_agent/*, and
anonymous_agent consumes it too). Flip 69 live importers across both the
package-path and submodule-path forms.
Shims left for the frozen single-agent stack: a package __init__ re-export plus
submodule shims for permission, skills_backends, and scoped_model_fallback
(the three imported via submodule path by chat_deepagent/subagents).
Cycle break: importing shared.middleware previously reached back into
new_chat.tools at module load, which dragged in new_chat.__init__ ->
chat_deepagent -> the middleware shim -> half-initialized shared.middleware.
Made action_log's ToolDefinition import TYPE_CHECKING-only and
tool_call_repair's INVALID_TOOL_NAME import function-local. These tools-package
back-edges fully resolve in slice 6.
Asset note: skills_backends._default_builtin_root now walks to
app/agents/new_chat/skills/builtin (the skills/ tree migrates in slice 7).
