Commit graph

1980 commits

Author SHA1 Message Date
CREDO23
30fff9e52f refactor(automations): move agent_task to builtin and restructure dispatch 2026-05-29 18:13:09 +02:00
CREDO23
f356e304e8 feat(automations): wire trigger selectors and beat into celery 2026-05-29 17:49:12 +02:00
CREDO23
b1642993ba feat(automations): add schedule trigger source, selector and inputs 2026-05-29 17:49:12 +02:00
CREDO23
f293aa6bdf refactor(automations): move schedule trigger into builtin package 2026-05-29 17:49:05 +02:00
CREDO23
acd673023a feat(automations): add event trigger source, selector and registration 2026-05-29 17:48:48 +02:00
CREDO23
4ba637ea44 feat(automations): add event trigger match and inputs 2026-05-29 17:48:48 +02:00
CREDO23
3ba18c7750 feat(automations): add event trigger filter grammar 2026-05-29 17:48:48 +02:00
CREDO23
f09e302d4f feat(automations): add event trigger params 2026-05-29 17:48:48 +02:00
CREDO23
6fa2e52361 feat(automations): add shared start_run dispatch helper 2026-05-29 17:48:39 +02:00
CREDO23
9247a2337f feat(automations): add EVENT to TriggerType enum 2026-05-29 17:48:39 +02:00
CREDO23
d6dfe53d62 feat(event_bus): add in-process domain event bus
A standalone, domain-agnostic pub/sub seam: an EventBus that owns its
subscriber registry and streams Event values from producers to listeners
in process. Boundary-crossing (Celery/DB/workers) is left to subscribers,
keeping the bus single-responsibility. Includes the immutable Event value
object and full unit coverage.
2026-05-29 15:26:12 +02:00
DESKTOP-RTLN3BA\$punk
40ca9e6ed2 refactor: remove search_surfsense_docs tool and related references
- Deleted the `search_surfsense_docs` tool and its associated files, streamlining the agent's toolset.
- Updated various components and prompts to remove references to the now-removed tool, ensuring consistency across the codebase.
- Adjusted documentation to direct users to the SurfSense documentation link for product-related queries instead.
2026-05-28 22:35:14 -07:00
DESKTOP-RTLN3BA\$punk
d013617bf6 feat(automations): added UI and improved mentions
- Added support for @-mentions in agent tasks, allowing users to reference documents, folders, and connectors directly in their queries.
- Updated `run_agent_task` to resolve mentions and include them in the context passed to the agent.
- Introduced new parameters in `AgentTaskActionParams` for handling mentioned document and connector IDs.
- Refactored the automation edit and new components to utilize the new `AutomationBuilderForm` for a more streamlined user experience.
- Removed deprecated JSON forms to simplify the automation creation process.
2026-05-28 21:26:32 -07:00
DESKTOP-RTLN3BA\$punk
94e834134f chore: linting 2026-05-28 19:21:29 -07:00
Rohan Verma
4dda02c06c
Merge pull request #1443 from CREDO23/feature-automations
[Feat] Automation V1 — Scheduled Agent Tasks, Created via Chat (HITL) or JSON
2026-05-28 12:41:41 -07:00
CREDO23
958bf9f95a fix(automations/agent_task): use in-memory checkpointer to avoid Celery PoolTimeout
The shared AsyncPostgresSaver caches DB connections in a module-level
pool. Cached connections are bound to the asyncio loop that opened
them, but `run_async_celery_task` discards the loop on each task's
exit — so after the first task the pool holds connections pointing
to a dead loop, and the next automation hangs 30s before failing
with `PoolTimeout: couldn't get a connection after 30.00 sec`.

Swap agent_task to `InMemorySaver`; automation runs only need state
within one Celery task, so nothing is lost. Site-local TODO tracks
the proper future fix (dispose the checkpointer pool around each
Celery task, mirroring `_dispose_shared_db_engine`).
2026-05-28 21:10:24 +02:00
CREDO23
91962ba879 fix automation run inputs, hitl routing, and detail UI polish 2026-05-28 02:48:47 +02:00
CREDO23
2e572d7818 feat(web): create_automation HITL approval card in chat
Closes the create loop in chat: the agent describes user intent → the
drafter sub-LLM produces an AutomationCreate JSON → this card surfaces
a structured preview → approve persists; reject cancels. Edits flow
through chat refinement (re-call with a refined intent), not in-card,
so the card stays simple and the multi-turn checkpointer carries the
context.

Tool UI (components/tool-ui/automation/):
- create-automation.tsx — entry dispatcher + ApprovalCard chrome
  (pending/processing/complete/rejected via useHitlPhase) + SavedCard
  (links to the detail page) + InvalidCard (lists drafter validation
  issues) + ErrorCard (verbatim message). Rejection result is hidden
  because the approval card itself shows the rejected phase inline.
- automation-draft-preview.tsx — structured preview body: name +
  description + goal, triggers (humanised cron + tz + static-input
  keys), plan steps (step_id → action), and a collapsible raw JSON
  for power users.

Wiring:
- components/tool-ui/index.ts — re-export.
- features/chat-messages/timeline/tool-registry/registry.ts —
  register create_automation → CreateAutomationToolUI (dynamic import,
  same pattern as other connector tools).
- contracts/enums/toolIcons.tsx — Workflow icon + "Create automation"
  display name so fallback chrome (and timeline headers) are honest.

Shared util:
- lib/automations/describe-cron.ts — lifted from the route slice's
  lib/ folder since both the dashboard slice and the new approval card
  now render schedule descriptions. Slice imports updated; the now-
  empty slice lib/ folder is gone.

Backend prompt fragments:
- main_agent/system_prompt/.../create_automation/description.md and
  the tool's docstring no longer promise in-card edits. They make the
  refinement path explicit: if the user wants changes after seeing the
  draft, they reply in chat and the agent calls the tool again with a
  refined intent.

v1 deliberately excludes:
- In-card edit form / right-side edit panel — defer until we see real
  demand. The chat refinement loop covers the common case.
- approve_always / persistent allow rules — automations are a single
  artifact, not a repeated mutation, so the "trust this kind of call"
  affordance doesn't apply.
2026-05-28 01:32:04 +02:00
CREDO23
79f0218360 rbac: surface automations permissions in the UI
Backend already defined automations:create/read/update/delete/execute and
seeded them on Owner/Editor/Viewer roles, but the Settings → Roles UI was
missing the metadata to render them properly.

- backend: add PERMISSION_DESCRIPTIONS entries for the 5 automations perms so
  the role editor stops falling back to "Permission for automations:create".
- frontend: add automations to CATEGORY_CONFIG (Workflow icon, slotted between
  podcasts and connectors) so the role editor groups them as a real section.
- frontend: extend the three ROLE_PRESETS — Editor and Contributor get
  create/read/update/execute (mirroring backend Editor); Viewer gets read.

Prep work for the automations frontend; canPerform/usePermissionGate already
handle the runtime gating, so no new hook is needed.
2026-05-28 00:30:40 +02:00
CREDO23
2b7d91aa03 feat(automations): add create_automation HITL tool (NL → draft → approve → save)
Single tool exposed to the main agent. The main agent passes a natural-language
`intent`; a focused drafter sub-LLM turns it into a full AutomationCreate JSON;
that JSON is surfaced via request_approval (action_type "automation_create") so
the user can edit/approve it on a frontend card; on approval the tool persists
via AutomationService. Three phases, one tool call.

Scope split:
- main agent sees only `intent: str` (no schema knowledge leaks into the calling
  graph) — prompt fragments scoped accordingly.
- drafter sub-LLM owns the schema + few-shot intent→JSON examples — lives in
  the generating graph's prompt (tools/automation/prompt.py).

Files:
- main_agent/tools/automation/{create.py, prompt.py, __init__.py}: new tool
  + drafter system prompt with two few-shot intent→JSON examples.
- system_prompt/prompts/tools/create_automation/{description.md, example.md}:
  intent-only guidance for the main agent.
- main_agent/tools/index.py: add create_automation to the main-agent allowlist.
- new_chat/tools/registry.py: deferred-import factory to break the
  multi_agent_chat ↔ registry cycle; one ToolDefinition entry.
2026-05-28 00:12:02 +02:00
DESKTOP-RTLN3BA\$punk
9d6e9b7e2d feat: enhance task management and timeout configurations in multi-agent chat
- Added new environment variables for controlling task execution limits, including `SURFSENSE_SUBAGENT_INVOKE_TIMEOUT_SECONDS`, `SURFSENSE_TASK_BATCH_CONCURRENCY`, and `SURFSENSE_TASK_BATCH_MAX_SIZE`.
- Updated documentation to reflect new batch processing capabilities for `task` calls, allowing for concurrent execution of multiple subagent tasks.
- Improved error handling and receipt generation for deliverables, ensuring consistent feedback on task status.
- Refactored middleware to incorporate search space ID for better task management.
2026-05-27 14:58:10 -07:00
CREDO23
c0232fdcfe refactor(automations): park manual trigger pending Run-now redesign
Manual-as-a-standalone-trigger conflates "user clicks Run now" with the
trigger model and forces ad-hoc input plumbing on the caller. Remove the
unreachable surface so the tree reflects reality (schedule is the only
v1 trigger).

- Unregister `manual`: drop import from triggers/__init__.py
- Delete `app/automations/triggers/manual/`
- Drop `RunService.dispatch_manual` (RunService is now read-only)
- Drop `POST /automations/{id}/run` and `RunDispatched` schema
- Keep `TriggerType.MANUAL` Python + PG enum value (reserved, documented)
  to avoid an Alembic round-trip when Run-now is redesigned
2026-05-27 22:29:51 +02:00
CREDO23
8fb65d7188 fix(automations): use enum values not names for postgres enum columns 2026-05-27 21:53:07 +02:00
CREDO23
27ab367a13 feat(automations): static_inputs on triggers + vertical-slice api/services 2026-05-27 21:21:43 +02:00
CREDO23
84d99f19a2 automations(api): API request/response schemas 2026-05-27 19:10:20 +02:00
CREDO23
dd6bc30f98 move automations api into vertical slice with service layer 2026-05-27 18:56:16 +02:00
CREDO23
d84240a630 add schedule tick task and beat entry 2026-05-27 17:56:07 +02:00
CREDO23
3b1d7c4389 add cron-based schedule trigger 2026-05-27 17:56:02 +02:00
CREDO23
f08b316441 add next_fire_at to automation_triggers and croniter dep 2026-05-27 17:55:58 +02:00
CREDO23
861b91004d refactor(automations): extract dispatch_run; move manual adapter under triggers/manual/dispatch.py 2026-05-27 17:20:23 +02:00
CREDO23
8c32455818 refactor(automations): vertical-slice actions and triggers by domain 2026-05-27 17:07:20 +02:00
CREDO23
ce45e11009 feat(automations): wire agent_task to multi_agent_chat with auto-approve loop 2026-05-27 17:02:44 +02:00
CREDO23
7ec3468113 refactor(automations): bind action handlers via ActionContext factory 2026-05-27 16:29:32 +02:00
CREDO23
cfbe2a7fe0 feat(automations): expose POST /automations/{id}/run 2026-05-27 15:30:45 +02:00
CREDO23
3bb02d8889 feat(automations): add manual dispatch service 2026-05-27 15:30:41 +02:00
CREDO23
1366c8a711 feat(rbac): add automations permission family 2026-05-27 15:30:34 +02:00
CREDO23
b26bf0bbcf feat(automation): register automation run celery task 2026-05-27 15:02:36 +02:00
CREDO23
273b98f350 feat(automation): expose runtime package surface 2026-05-27 15:02:36 +02:00
CREDO23
d3cda12191 feat(automation): add automation run executor 2026-05-27 15:02:36 +02:00
CREDO23
0a329e5a69 feat(automation): add per-step execution 2026-05-27 15:02:36 +02:00
CREDO23
f71a02db2f feat(automation): add automation run repository 2026-05-27 15:02:36 +02:00
CREDO23
924a82c0b1 feat(automation): add retry policy helper 2026-05-27 15:02:36 +02:00
CREDO23
8b87d179e9 feat(automation): add recursive render_value to templating 2026-05-27 15:02:36 +02:00
CREDO23
cb42b3a84f feat(automation): add template run context builder 2026-05-27 14:23:18 +02:00
CREDO23
de6da1b775 feat(automation): add template render and predicate evaluation 2026-05-27 14:23:17 +02:00
CREDO23
8345e79f6d feat(automation): add sandboxed template environment 2026-05-27 14:23:17 +02:00
CREDO23
08e94ac5ca feat(automation): add custom template filters 2026-05-27 14:23:17 +02:00
CREDO23
b4e5bf95a4 feat(automation): add template filter and test allowlist 2026-05-27 14:23:17 +02:00
CREDO23
99fd1a1338 feat(automation): register agent_task action and schedule/manual triggers 2026-05-27 13:58:57 +02:00
CREDO23
56b3e1bfc4 refactor(automation): drop Block suffix from definition components 2026-05-27 13:48:41 +02:00