Greedy multiset match on chunk text decides which rows keep their embeddings,
which texts need embedding, and which rows are deleted. No DB, no embeddings;
fully unit-tested (reuse, head insert, middle edit, deletion, duplicates,
reorder, full rewrite).
The cached payload is the indexing pipeline's embeddings (markdown is
chunked then embedded), so "embedding cache" names the expensive output
directly and removes the "index" ambiguity (DB index vs vector index vs
indexing phase). Renames the service, settings, eligibility, eviction
task, metrics, config flags (INDEX_CACHE_* -> EMBEDDING_CACHE_*), object
prefix, and the table (index_cache_embedding_sets -> embedding_cache_sets)
with its constraint and indexes. Migration 161 renamed accordingly.
Pin AUTH_TYPE=LOCAL (and REGISTRATION_ENABLED=TRUE) in the test bootstrap so
the email/password auth routers mount during integration tests regardless of a
developer's .env=GOOGLE; without this the upload tests 404 on registration.
Also update three tests to the current QuotaInsufficientError signature
(balance_micros) after used_micros/limit_micros were removed.
Move the lifecycle service, Celery task bodies, and mark_failed coverage out of
DB-faking unit tests and into integration tests against a real Postgres, faking
only true externals (broker, object store, TTS, ffmpeg, billing, LLM). Add HTTP
slices for cancel, voices, scoping, and public-chat streaming. The unit tier is
now fake-free pure logic with no session doubles.
- Updated environment variables and - configurations for credit purchases via Stripe, replacing legacy page pack system.
- Introduced auto-reload feature for credit top-ups and modified database models to track credit transactions.
- Updated notification system to handle insufficient credits and auto-reload failures.
- Adjusted API routes and schemas to reflect changes in credit management.
The knowledge_base subagent imported subagent_invoke_config + EXCLUDED_STATE_KEYS
from main_agent's checkpointed_subagent_middleware -- a subagent reaching into
main-agent internals. Both symbols (plus the recursion-limit constant they need)
are a subagent-invocation contract shared by the orchestrator's task middleware
and any nested-invoking subagent. Move them to subagents/shared/invocation.py;
config.py keeps the HITL resume side-channel and constants.py keeps the
main-agent tuning knobs. All consumers (task_tool, kb tool, tests) repointed.
The KB-persistence impl lived in shared/middleware/ but no subagent uses it --
consumers are the main_agent builder and the boundary event loop. Colocate with
its owner using the folder-per-middleware shape; __init__ re-exports the public
surface. Tests that reached module internals now alias the .middleware submodule.
main_agent/middleware/kb_persistence.py -> kb_persistence/builder.py
shared/middleware/kb_persistence.py -> kb_persistence/middleware.py
The busy-mutex impl (BusyMutexMiddleware + cancel/turn-lifecycle primitives)
lived in shared/middleware/ but no subagent uses it -- consumers are the
main_agent builder and the boundary (turn lifecycle). Colocate with its owner
using the folder-per-middleware shape; __init__ re-exports the public surface so
boundary import sites only change package path:
main_agent/middleware/busy_mutex.py -> busy_mutex/builder.py
shared/middleware/busy_mutex.py -> busy_mutex/middleware.py
Per-file verification of the slice-3 candidates showed receipts/ and
date_filters.py are shared contracts (consumed by shared/state + shared
middleware + subagents), so they correctly stay put.
permissions was the real misfit: the rule *model* lived at shared/permissions.py
while its enforcement lived at shared/middleware/permissions/. Unify them into a
single self-contained subsystem:
shared/permissions.py -> shared/permissions/model.py
shared/middleware/permissions/{deny,ask,middleware}
-> shared/permissions/{deny,ask,middleware}
The package __init__ re-exports the model API + build_permission_mw, so the 32
external model consumers keep importing `from ...shared.permissions import Rule`
unchanged; only the 8 internal files redirect to `.model` (cycle-safe, model
loaded before middleware).
Move the lower-level runtime/infra modules out of multi_agent_chat/shared/
(they were never used by subagents, so they failed the shared-by-all-siblings
rule) and unify them with the already-relocated checkpointer:
agents/runtime/ -> agents/chat/runtime/
mac/shared/errors.py -> chat/runtime/errors.py
mac/shared/llm_config.py -> chat/runtime/llm_config.py
mac/shared/prompt_caching.py -> chat/runtime/prompt_caching.py
mac/shared/mention_resolver.py -> chat/runtime/mention_resolver.py
mac/shared/path_resolver.py -> chat/runtime/path_resolver.py
These sit below the agent packages: the boundary + agent factory + shared
middleware depend on them, and they import no agent code (acyclic).
Recursive shared-folder rule: a shared/ must be shared by ALL siblings at its
level. The kernel (context, compaction, retry_after, web_search) was shared by
only 2 of the agents -- anonymous_chat + multi_agent_chat -- never by podcaster
or video_presentation. Those 2 are the "chat" category, so their shared code
belongs in that category's shared/, not the top-level one.
app/agents/anonymous_chat/ -> app/agents/chat/anonymous_chat/
app/agents/multi_agent_chat/ -> app/agents/chat/multi_agent_chat/
app/agents/shared/ -> app/agents/chat/shared/ (anon<->mac kernel)
Top-level app/agents/shared/ is gone: nothing was shared across all three
categories (chat / podcaster / video_presentation).
~289 import sites rewritten (app.agents.{anonymous_chat,multi_agent_chat,shared}
-> app.agents.chat.*); all moves are git renames (history preserved).
app/agents/ now: chat/, podcaster/, video_presentation/, runtime/.
These were never shared with anonymous_chat (nor podcaster/video_presentation)
-- only multi_agent_chat (subagents/main agent) and the boundary use them:
shared/tools/mcp/ -> multi_agent_chat/shared/tools/mcp/
shared/tools/hitl.py -> multi_agent_chat/shared/tools/hitl.py
shared/tools/catalog.py -> multi_agent_chat/shared/tools/catalog.py
shared/middleware/dedup_tool_calls.py
-> multi_agent_chat/shared/middleware/dedup_tool_calls.py
app/agents/shared/ now holds only the genuine anon<->mac kernel:
context, middleware/{compaction,retry_after}, tools/web_search.
Neither module is imported by any sibling agent package, so neither belongs in
the cross-agent shared kernel:
- checkpointer.py -> app/agents/runtime/checkpointer.py
LangGraph Postgres checkpoint saver. It's cross-agent *runtime infra* wired by
the boundary (app lifespan + anonymous_chat & multi_agent_chat flows), not
agent code. New app/agents/runtime/ layer holds boundary-wired agent infra.
- shared/system_prompt.py + shared/prompts/ -> app/prompts/
The legacy single-agent prompt composer. The live agents don't use it
(main_agent has its own system_prompt/ builder; anonymous_chat builds inline);
its only consumer is new_llm_config_routes for displaying default instructions.
Moved to the existing non-agent prompt domain:
system_prompt.py -> app/prompts/default_system_instructions.py
prompts/ -> app/prompts/system_prompt_composer/
app/agents/shared/ now contains only genuinely cross-agent code: context,
middleware/{compaction,retry_after,dedup_tool_calls}, tools/.
NOTE: get_default_system_instructions() (LLM-config UI) composes from the legacy
library, which differs from what the live agents actually run -- pre-existing
latent staleness, not changed here.
app/agents/shared/ is a sibling of anonymous_chat/podcaster/multi_agent_chat/
video_presentation, so it should only hold code shared across 2+ of those
agents. In practice podcaster and video_presentation import nothing from it,
and anonymous_chat needs only context + compaction + retry_after + web_search.
Everything else was multi_agent_chat-only (the boundary just passes through).
Move the multi_agent_chat-only cluster into multi_agent_chat/shared/ (files
moved verbatim via git rename; ~116 import sites rewritten):
errors, feature_flags, filesystem_selection, path_resolver, prompt_caching,
sandbox, llm_config, mention_resolver
middleware/busy_mutex, middleware/kb_persistence
busy_mutex/llm_config/mention_resolver are boundary-only but import the moved
modules, so they were folded in to avoid a backwards shared -> multi_agent_chat
dependency. main_agent builders now import the impls directly; the shared
middleware barrel keeps only the genuinely-shared compaction + retry_after.
Also delete the dead leftover shared/plugins and shared/skills dirs (live
copies already live under main_agent/).
Remaining in app/agents/shared/: context, system_prompt(+prompts), checkpointer,
middleware/{compaction,retry_after,dedup_tool_calls}, tools/. checkpointer and
system_prompt are boundary-only infra pending a dedicated home decision.
app/agents/shared/middleware/permission.py was an older, monolithic
PermissionMiddleware superseded by the modular permissions/ package under
multi_agent_chat/shared/middleware/ (core + evaluation + ask/ + factory).
Production wires only the package (main_agent stack + every subagent
builder); the kernel file was reachable only through the shared barrel
re-export (itself unused) and two tests pinned to its dead internals
(_raise_interrupt, _normalize_permission_decision, old after_model shape).
- delete app/agents/shared/middleware/permission.py
- drop PermissionMiddleware from the shared middleware barrel
- delete test_permission_middleware.py (covered the dead impl only; live
behavior is covered by tests/.../middleware/shared/permissions/*)
- test_desktop_safety_rules.py: keep the ruleset-level regression tests,
drop the dead import + TestPermissionMiddlewareIntegration class
knowledge_search, memory_injection and scoped_model_fallback no longer
belong in the cross-agent kernel (app/agents/shared/middleware): they are
consumed only inside multi_agent_chat. Relocate each impl next to the
builder that uses it:
- knowledge_search.py -> multi_agent_chat/shared/middleware/ (genuinely
shared: its _render_priority_message feeds kb_context_projection, used by
both the main agent and the KB subagent)
- memory_injection.py -> multi_agent_chat/shared/middleware/ (beside its
memory.py builder)
- scoped_model_fallback.py -> multi_agent_chat/shared/middleware/resilience/
(beside fallback.py/bundle.py)
Impls moved verbatim (git rename). Builders/consumers now import the local
sibling; main_agent knowledge_priority imports the new shared path; shared
middleware barrel trimmed.
Tests: repoint imports; convert the knowledge_search monkeypatch targets
from brittle dotted-string form to object-based patching (monkeypatch.setattr
on the imported module), which is robust to import ordering. No behavior
change.
Each main-agent-only middleware now lives in its own folder under
main_agent/middleware/<concept>/ with builder.py (flag-gated construction)
+ middleware.py (the impl), re-exported via __init__.py. This kills the
cross-folder hop into agents/shared/middleware and keeps each middleware's
two responsibilities (build vs behavior) as colocated siblings.
Moved (impl from shared/middleware, builder from main_agent/middleware):
action_log, anonymous_document, context_editing, doom_loop, knowledge_tree,
noop_injection, otel_span, tool_call_repair.
Impls moved verbatim (git rename, no body edits) so behavior is unchanged.
Builders now import from the local .middleware sibling. stack.py import
paths updated for the 3 renamed folders; shared middleware barrel trimmed;
tests repointed (imports + patch targets).
DedupHITLToolCallsMiddleware is only wired by the main_agent stack, but
its module also exports dedup-key resolvers consumed by the shared MCP
tool layer. Splitting keeps the resolvers (dedup_key_full_args,
wrap_dedup_key_by_arg_name, DedupResolver) in shared and moves the
middleware class verbatim into main_agent/middleware/dedup_hitl.py
(merged with its builder), eliminating the shared->main_agent dependency
that a flat move would create. No behavior change.
file_intent (FileIntentMiddleware) and flatten_system
(FlattenSystemMessageMiddleware) were only ever instantiated in the
single-agent chat_deepagent stack, which was removed in 14bbea085. They
have no production consumer in multi_agent_chat. Delete both modules and
their unit tests.
Also drop the vestigial KnowledgeBaseSearchMiddleware alias (= the live
KnowledgePriorityMiddleware); its tests now target the real class so the
behavior coverage is preserved. Trim the three barrel/__all__ entries and
strip the now-dead class names from comments.
The concrete filesystem backends are consumed only by the MAC filesystem
layer (tools, path-resolution middleware, the resolver, skills backend) and
tests -- no external app code. Group them next to the filesystem middleware
they serve:
- filesystem_backends.py -> filesystem/backends/resolver.py
- middleware/kb_postgres_backend.py -> filesystem/backends/kb_postgres.py
- middleware/local_folder_backend.py -> filesystem/backends/local_folder.py
- middleware/multi_root_local_folder_backend.py -> .../multi_root_local_folder.py
- document_xml.py -> filesystem/backends/document_xml.py
Repoint all 21 importers. No behavior change; import-all + filesystem
backend/path-resolution/knowledge-search unit tests stay green (478).
permissions.py (authorization Rule/Ruleset model) is consumed across all
MAC subagents + the permissions middleware, with a single external
consumer (user_tool_allowlist service) -> move to
multi_agent_chat/shared/permissions.py and repoint all 42 sites.
deliverable_wait.py (wait_for_deliverable) is used only by the podcast and
video_presentation deliverable tools -> colocate into
subagents/builtins/deliverables/.
No behavior change; import-all + permission/allowlist/deliverable unit
tests stay green.
filesystem_state.py (the multi-agent graph state) and state_reducers.py
(its merge reducers) are consumed only by multi_agent_chat (filesystem
tools/middleware, kb projection, and the MAC-only shared middleware) plus
two unit tests -- no external app code. Relocate them into a dedicated
multi_agent_chat/shared/state/ package (filesystem_state.py + reducers.py)
and repoint every importer.
No behavior change; import-all + the full unit/middleware + unit/agents
suites (1066 tests) stay green.
Move modules out of agents/shared/ that are consumed by a single package
(main_agent), placing each next to its only consumer instead of in a
"shared" grab-bag:
- agent_cache.py -> main_agent/runtime/agent_cache_store.py
- connector_searchable_types.py -> main_agent/runtime/
- plugin_loader.py + plugins/ -> main_agent/plugins/
- skills/ + skills_backends.py -> main_agent/skills/
- tools/invalid_tool.py -> main_agent/tools/
Drop the skills_backends re-export from the shared middleware barrel and
repoint all consumers + tests. No behavior change; import-all,
error-contract, and the moved tests stay green.
The three MCP siblings (mcp_client/mcp_tool/mcp_tools_cache) served one
objective but sat loose at the top of shared/tools. Grouped them into an
mcp/ package and dropped the redundant prefix: client.py, tool.py, cache.py.
Updated all importers (routes, mcp_tools subagent, e2e fake patch targets,
unit test) to the new paths.
