- Deleted the `search_surfsense_docs` tool and its associated files, streamlining the agent's toolset.
- Updated various components and prompts to remove references to the now-removed tool, ensuring consistency across the codebase.
- Adjusted documentation to direct users to the SurfSense documentation link for product-related queries instead.
- Added support for @-mentions in agent tasks, allowing users to reference documents, folders, and connectors directly in their queries.
- Updated `run_agent_task` to resolve mentions and include them in the context passed to the agent.
- Introduced new parameters in `AgentTaskActionParams` for handling mentioned document and connector IDs.
- Refactored the automation edit and new components to utilize the new `AutomationBuilderForm` for a more streamlined user experience.
- Removed deprecated JSON forms to simplify the automation creation process.
The shared AsyncPostgresSaver caches DB connections in a module-level
pool. Cached connections are bound to the asyncio loop that opened
them, but `run_async_celery_task` discards the loop on each task's
exit — so after the first task the pool holds connections pointing
to a dead loop, and the next automation hangs 30s before failing
with `PoolTimeout: couldn't get a connection after 30.00 sec`.
Swap agent_task to `InMemorySaver`; automation runs only need state
within one Celery task, so nothing is lost. Site-local TODO tracks
the proper future fix (dispose the checkpointer pool around each
Celery task, mirroring `_dispose_shared_db_engine`).
Top-level tests that span multiple submodules:
- test_stores.py (7): the trigger + action registry contracts — register
round-trip, unknown type → None (not raise), duplicate registration
rejected, defensive snapshot from all_*.
- test_definition_types.py (2): params_schema property on both
ActionDefinition and TriggerDefinition reflects the Pydantic model.
- test_persistence_enums.py (3): exact string values + member sets of
AutomationStatus / RunStatus / TriggerType — the postgres-mirrored
contract that breaks stored rows if drifted.
- test_import_registrations.py (2): the bundled agent_task action and
schedule trigger self-register on package import (canary for the
side-effect import chain).
conftest.py adds isolated_action_registry / isolated_trigger_registry
fixtures: snapshot + restore of the module-level _REGISTRY dicts so
tests that add their own definitions don't leak across the suite.
14 tests, pure unit.
auto_decide.build_auto_decisions (3): produces one decision per
action_request entry, defaults to one decision for legacy scalar
interrupts, and skips malformed interrupts silently so a misbehaving
tool can't take down the whole agent_task step.
finalize.extract_final_assistant_message (4): string-content AIMessage
returned verbatim, list-of-parts content concatenated (skipping
non-text parts like tool_use), walks back past trailing ToolMessages
to find the last AIMessage, and returns None when no extractable text
is present (so callers can branch on silence vs. empty).
7 tests, pure unit.
render.py (4): variable substitution, StrictUndefined raises on missing
keys, evaluate_predicate coerces to bool, render_value walks dicts/lists
and renders string leaves.
filters.py (4): slugify produces URL-safe output, date formats datetime
with strftime, date(None) → "" so templates can write
{{ inputs.last_fired_at | date }} on first run, date(str) passes through.
environment.py (4): the sandbox boundary — disallowed Jinja built-ins
(e.g. pprint) raise, and the finalize hook coerces non-string outputs
to predictable wire shapes (datetime → ISO, None → "", dict → JSON).
context.py (1): build_run_context exposes {run, inputs, steps} with the
exact shape every plan template body relies on.
13 tests total, all pure unit.
execute_step (6 tests): happy path, when=falsy → skipped, unknown action
→ ActionNotFound failure, retry budget exhaustion (attempts = 1 +
max_retries), retry recovery, and template-rendering of step params
against the run context.
with_retries (3 tests): first-try success returns attempts=1, recovery
returns the actual attempt that produced the result, and exhaustion
re-raises the last exception with the handler called 1 + max_retries
times.
All tests use backoff="none" to keep wall-clock time zero; timeout
testing is intentionally skipped (would need >= 1s per the int contract,
and exhaustion already locks that any Exception triggers retry).
Cover the input-validation contract dispatch_run relies on:
- no declared schema → inputs pass through unchanged (regression site
that previously stripped runtime keys like fired_at / last_fired_at
and broke Jinja templates).
- declared schema, valid inputs → passthrough validated.
- declared schema, invalid inputs → DispatchError (uniform exception
type, not raw jsonschema.ValidationError).
Plus the DispatchError exception identity (Exception subclass, message
preserved, isinstance-friendly for the dispatch layer's consumers).
4 tests, pure unit.
Cover the cron + IANA timezone + UTC normalization contract for the
schedule trigger: next-match strictly-after, DST offset shift across
spring-forward, malformed cron / unknown timezone rejection, and the
ScheduleTriggerParams Pydantic gate that surfaces InvalidCronError as
ValidationError at the API boundary.
8 tests, pure unit (no DB, no mocks).
Closes the create loop in chat: the agent describes user intent → the
drafter sub-LLM produces an AutomationCreate JSON → this card surfaces
a structured preview → approve persists; reject cancels. Edits flow
through chat refinement (re-call with a refined intent), not in-card,
so the card stays simple and the multi-turn checkpointer carries the
context.
Tool UI (components/tool-ui/automation/):
- create-automation.tsx — entry dispatcher + ApprovalCard chrome
(pending/processing/complete/rejected via useHitlPhase) + SavedCard
(links to the detail page) + InvalidCard (lists drafter validation
issues) + ErrorCard (verbatim message). Rejection result is hidden
because the approval card itself shows the rejected phase inline.
- automation-draft-preview.tsx — structured preview body: name +
description + goal, triggers (humanised cron + tz + static-input
keys), plan steps (step_id → action), and a collapsible raw JSON
for power users.
Wiring:
- components/tool-ui/index.ts — re-export.
- features/chat-messages/timeline/tool-registry/registry.ts —
register create_automation → CreateAutomationToolUI (dynamic import,
same pattern as other connector tools).
- contracts/enums/toolIcons.tsx — Workflow icon + "Create automation"
display name so fallback chrome (and timeline headers) are honest.
Shared util:
- lib/automations/describe-cron.ts — lifted from the route slice's
lib/ folder since both the dashboard slice and the new approval card
now render schedule descriptions. Slice imports updated; the now-
empty slice lib/ folder is gone.
Backend prompt fragments:
- main_agent/system_prompt/.../create_automation/description.md and
the tool's docstring no longer promise in-card edits. They make the
refinement path explicit: if the user wants changes after seeing the
draft, they reply in chat and the agent calls the tool again with a
refined intent.
v1 deliberately excludes:
- In-card edit form / right-side edit panel — defer until we see real
demand. The chat refinement loop covers the common case.
- approve_always / persistent allow rules — automations are a single
artifact, not a repeated mutation, so the "trust this kind of call"
affordance doesn't apply.
Backend already defined automations:create/read/update/delete/execute and
seeded them on Owner/Editor/Viewer roles, but the Settings → Roles UI was
missing the metadata to render them properly.
- backend: add PERMISSION_DESCRIPTIONS entries for the 5 automations perms so
the role editor stops falling back to "Permission for automations:create".
- frontend: add automations to CATEGORY_CONFIG (Workflow icon, slotted between
podcasts and connectors) so the role editor groups them as a real section.
- frontend: extend the three ROLE_PRESETS — Editor and Contributor get
create/read/update/execute (mirroring backend Editor); Viewer gets read.
Prep work for the automations frontend; canPerform/usePermissionGate already
handle the runtime gating, so no new hook is needed.
Single tool exposed to the main agent. The main agent passes a natural-language
`intent`; a focused drafter sub-LLM turns it into a full AutomationCreate JSON;
that JSON is surfaced via request_approval (action_type "automation_create") so
the user can edit/approve it on a frontend card; on approval the tool persists
via AutomationService. Three phases, one tool call.
Scope split:
- main agent sees only `intent: str` (no schema knowledge leaks into the calling
graph) — prompt fragments scoped accordingly.
- drafter sub-LLM owns the schema + few-shot intent→JSON examples — lives in
the generating graph's prompt (tools/automation/prompt.py).
Files:
- main_agent/tools/automation/{create.py, prompt.py, __init__.py}: new tool
+ drafter system prompt with two few-shot intent→JSON examples.
- system_prompt/prompts/tools/create_automation/{description.md, example.md}:
intent-only guidance for the main agent.
- main_agent/tools/index.py: add create_automation to the main-agent allowlist.
- new_chat/tools/registry.py: deferred-import factory to break the
multi_agent_chat ↔ registry cycle; one ToolDefinition entry.
- Added new environment variables for controlling task execution limits, including `SURFSENSE_SUBAGENT_INVOKE_TIMEOUT_SECONDS`, `SURFSENSE_TASK_BATCH_CONCURRENCY`, and `SURFSENSE_TASK_BATCH_MAX_SIZE`.
- Updated documentation to reflect new batch processing capabilities for `task` calls, allowing for concurrent execution of multiple subagent tasks.
- Improved error handling and receipt generation for deliverables, ensuring consistent feedback on task status.
- Refactored middleware to incorporate search space ID for better task management.
Manual-as-a-standalone-trigger conflates "user clicks Run now" with the
trigger model and forces ad-hoc input plumbing on the caller. Remove the
unreachable surface so the tree reflects reality (schedule is the only
v1 trigger).
- Unregister `manual`: drop import from triggers/__init__.py
- Delete `app/automations/triggers/manual/`
- Drop `RunService.dispatch_manual` (RunService is now read-only)
- Drop `POST /automations/{id}/run` and `RunDispatched` schema
- Keep `TriggerType.MANUAL` Python + PG enum value (reserved, documented)
to avoid an Alembic round-trip when Run-now is redesigned
