- Introduced lazy knowledge base retrieval mode, allowing the main agent to fetch KB content on demand via the `search_knowledge_base` tool, improving performance by skipping expensive pre-injection processes.
- Added cross-thread caching capability, enabling reuse of compiled graphs across different user chats, reducing latency for returning users.
- Updated middleware to support new lazy loading and caching features, ensuring efficient resource utilization and improved response times.
- Enhanced logging for performance tracking during knowledge retrieval and agent interactions.
- Integrated performance logging in `OtelSpanMiddleware` to track model call durations even when OTel is disabled.
- Added detailed performance metrics in `KnowledgePriorityMiddleware` for database operations and embedding processes, improving visibility into query performance.
- Utilized `get_perf_logger` for consistent logging across middleware components.
- Replaced Playwright with Scrapling's fetchers in the web crawling and YouTube processing modules for improved performance and flexibility.
- Updated proxy configuration to support dynamic proxy selection via environment variables.
- Enhanced logging to track performance metrics during web scraping operations.
- Refactored related modules to utilize the new proxy utilities and streamline the scraping process.
- Consolidated Redis configuration by introducing a single `REDIS_URL` variable for Celery broker, result backend, and app cache.
- Removed deprecated variables related to Firecrawl and Stripe token limits from `.env.example` files.
- Updated documentation to reflect changes in environment variable usage for improved clarity and maintainability.
- Replaced environment variable usage with a centralized configuration system in multiple modules, including `celery_app`, `agent_cache_store`, `sandbox`, `file_storage`, and `connector_service`.
- Enhanced maintainability and readability by sourcing configuration values from the `config` module instead of directly from environment variables.
- Updated relevant settings to ensure consistent access to configuration values across the application.
- Added a global switch `GATEWAY_ENABLED` to control the activation of all messaging gateway channels (Telegram, WhatsApp, Slack, Discord).
- Updated relevant routes and workers to check the `GATEWAY_ENABLED` flag, returning 404 for HTTP routes when disabled.
- Enhanced documentation in the `.env.example` file to reflect the new configuration option.
- Added a new utility function `isLlmOnboardingComplete` to determine if the onboarding process is complete based on the agent LLM ID and the presence of global configurations.
- Updated the onboarding logic in the `OnboardPage` and `DashboardClientLayout` components to utilize the new utility function for improved readability and maintainability.
- Updated the backend URL initialization to use a function that retrieves the URL from environment variables, enhancing configurability for different environments.
- Added support for HF_TOKEN as a secret in the Docker build workflow to enhance security during model embedding retrieval.
- Updated the Dockerfile to utilize the HF_TOKEN secret for fetching embeddings, ensuring sensitive information is handled securely.
- Added support for version tagging with 'v*' and 'beta-v*' patterns in the Docker build workflow.
- Updated the tag_release job to identify release tags and calculate the next Docker version accordingly.
- Improved error handling for version validation and adjusted conditions for build and release processes.
- Added output for commit SHA in the Docker build workflow to facilitate tagging.
- Introduced a new verification step to ensure both architectures are present for each required variant before proceeding with the release.
- Updated the finalize_release job to tag the correct commit after successful builds and verifications.
- Introduced the ZERO_AUTO_RESET environment variable to enable automatic reset of the SQLite replica in case of replication halts.
- Updated Docker Compose files to include ZERO_AUTO_RESET in service configurations.
- Enhanced documentation to clarify the purpose and usage of the new variable.
- Introduced a new docker-compose.gpu.yml file to define GPU resource reservations for backend services.
- Updated .env.example to include GPU-related environment variables and usage instructions.
- Enhanced install.ps1 and install.sh scripts to support GPU variant selection and validation for GPU count.
- Improved error handling and user feedback for invalid GPU configurations.
- Added support for multiple image variants (CPU, CUDA 12.8, CUDA 12.6) in the Docker build workflow.
- Updated Docker Compose configuration to utilize the new SURFSENSE_VARIANT environment variable for selecting image variants.
- Enhanced documentation to clarify usage of backend image variants and their corresponding environment variable settings.
- Introduced a new optional dependency group for CUDA 12.6 in pyproject.toml.
- Updated conflict resolution to include CUDA 12.6 alongside existing configurations.
- Added source configurations for PyTorch and torchvision for CUDA 12.6.
The User mapper eager-loads the oauth_accounts collection via joined load
under AUTH_TYPE=GOOGLE, so the mint endpoint's query must call .unique()
before scalar_one_or_none() to avoid InvalidRequestError (500).
- Updated Dockerfile to conditionally install PyTorch with CPU or CUDA support based on build arguments.
- Added optional dependencies for CPU and CUDA versions of PyTorch in pyproject.toml.
- Configured uv.lock to manage dependencies and conflicts between CPU and CUDA installations.
Recursive pass over the agents module to make docstrings and inline
comments concise and intent-oriented: drop narration that just restates
the code, condense verbose module/function docstrings, and keep only the
non-obvious "why" notes. No functional code changed.
When app.notifications is the import entry point (e.g. Celery loading
app.notifications.service before any ORM code), app.db re-entered the
half-initialized app.notifications.persistence at its model-registration import
and failed with "cannot import name 'Notification' ... partially initialized".
Import app.db at the top of app/notifications/__init__.py so db fully initializes
(including its own Notification registration) before we re-import from
.persistence. Pre-existing issue (reproduces on the base commit); surfaced by the
celery worker startup.
The knowledge_base subagent imported subagent_invoke_config + EXCLUDED_STATE_KEYS
from main_agent's checkpointed_subagent_middleware -- a subagent reaching into
main-agent internals. Both symbols (plus the recursion-limit constant they need)
are a subagent-invocation contract shared by the orchestrator's task middleware
and any nested-invoking subagent. Move them to subagents/shared/invocation.py;
config.py keeps the HITL resume side-channel and constants.py keeps the
main-agent tuning knobs. All consumers (task_tool, kb tool, tests) repointed.
build_credentials/get_token_encryption are Google-OAuth helpers used by both the
Gmail and Calendar connector tools. They lived inside gmail/tools/_helpers.py,
forcing calendar -> gmail coupling. Move them to a neutral connector-level module
(connectors/google_auth.py); gmail/_helpers.py re-exports them under the legacy
private names so existing gmail tools are untouched, and calendar now imports the
shared module directly.
The KB-persistence impl lived in shared/middleware/ but no subagent uses it --
consumers are the main_agent builder and the boundary event loop. Colocate with
its owner using the folder-per-middleware shape; __init__ re-exports the public
surface. Tests that reached module internals now alias the .middleware submodule.
main_agent/middleware/kb_persistence.py -> kb_persistence/builder.py
shared/middleware/kb_persistence.py -> kb_persistence/middleware.py
The busy-mutex impl (BusyMutexMiddleware + cancel/turn-lifecycle primitives)
lived in shared/middleware/ but no subagent uses it -- consumers are the
main_agent builder and the boundary (turn lifecycle). Colocate with its owner
using the folder-per-middleware shape; __init__ re-exports the public surface so
boundary import sites only change package path:
main_agent/middleware/busy_mutex.py -> busy_mutex/builder.py
shared/middleware/busy_mutex.py -> busy_mutex/middleware.py
memory (builder) + memory_injection (impl) lived in shared/middleware/ but are
consumed only by main_agent (no subagent, no shared plumbing). Colocate with
their owner using the folder-per-middleware shape:
shared/middleware/memory.py -> main_agent/middleware/memory/builder.py
shared/middleware/memory_injection.py -> main_agent/middleware/memory/middleware.py
Re-decide subagent_stack placement using the primary-built-for lens rather
than consumer-only: it assembles the middleware stack threaded into every
subagent, so its domain is subagents -- even though main_agent is its sole
caller (analogous to subagents/registry.py, also invoked from main_agent).
Since no subagent *sibling* imports it, it does not belong in subagents/shared/
but at the subagents/ package root:
main_agent/middleware/subagent_stack.py
-> subagents/middleware_stack.py
subagents/shared/middleware/ held build_subagent_middleware_stack, but no
subagent package imports it -- its only caller is main_agent/middleware/stack.py
(the generic pack_subagent builder merely consumes the resulting dict at
runtime). It is main_agent's policy for which middleware to thread into
subagents, so it belongs with its caller:
subagents/shared/middleware/middleware_stack.py
-> main_agent/middleware/subagent_stack.py
subagents/shared/ now holds only genuinely subagent-shared code (md_file_reader,
snippets, spec, subagent_builder, hitl).
Per-file verification of the slice-3 candidates showed receipts/ and
date_filters.py are shared contracts (consumed by shared/state + shared
middleware + subagents), so they correctly stay put.
permissions was the real misfit: the rule *model* lived at shared/permissions.py
while its enforcement lived at shared/middleware/permissions/. Unify them into a
single self-contained subsystem:
shared/permissions.py -> shared/permissions/model.py
shared/middleware/permissions/{deny,ask,middleware}
-> shared/permissions/{deny,ask,middleware}
The package __init__ re-exports the model API + build_permission_mw, so the 32
external model consumers keep importing `from ...shared.permissions import Rule`
unchanged; only the 8 internal files redirect to `.model` (cycle-safe, model
loaded before middleware).
Move the lower-level runtime/infra modules out of multi_agent_chat/shared/
(they were never used by subagents, so they failed the shared-by-all-siblings
rule) and unify them with the already-relocated checkpointer:
agents/runtime/ -> agents/chat/runtime/
mac/shared/errors.py -> chat/runtime/errors.py
mac/shared/llm_config.py -> chat/runtime/llm_config.py
mac/shared/prompt_caching.py -> chat/runtime/prompt_caching.py
mac/shared/mention_resolver.py -> chat/runtime/mention_resolver.py
mac/shared/path_resolver.py -> chat/runtime/path_resolver.py
These sit below the agent packages: the boundary + agent factory + shared
middleware depend on them, and they import no agent code (acyclic).
shared/sandbox.py was used only by the filesystem middleware/tools (and the
boundary) -- never by main_agent or subagents as shared code. Move it next to
its only agent-side consumer:
multi_agent_chat/shared/sandbox.py
-> multi_agent_chat/shared/middleware/filesystem/sandbox.py
Recursive shared-folder rule: a shared/ must be shared by ALL siblings at its
level. The kernel (context, compaction, retry_after, web_search) was shared by
only 2 of the agents -- anonymous_chat + multi_agent_chat -- never by podcaster
or video_presentation. Those 2 are the "chat" category, so their shared code
belongs in that category's shared/, not the top-level one.
app/agents/anonymous_chat/ -> app/agents/chat/anonymous_chat/
app/agents/multi_agent_chat/ -> app/agents/chat/multi_agent_chat/
app/agents/shared/ -> app/agents/chat/shared/ (anon<->mac kernel)
Top-level app/agents/shared/ is gone: nothing was shared across all three
categories (chat / podcaster / video_presentation).
~289 import sites rewritten (app.agents.{anonymous_chat,multi_agent_chat,shared}
-> app.agents.chat.*); all moves are git renames (history preserved).
app/agents/ now: chat/, podcaster/, video_presentation/, runtime/.
