Commit graph

6313 commits

Author SHA1 Message Date
CREDO23
6fa2e52361 feat(automations): add shared start_run dispatch helper 2026-05-29 17:48:39 +02:00
CREDO23
356400ae2a feat(automations): migrate automation_trigger_type enum to add event 2026-05-29 17:48:39 +02:00
CREDO23
9247a2337f feat(automations): add EVENT to TriggerType enum 2026-05-29 17:48:39 +02:00
CREDO23
d6dfe53d62 feat(event_bus): add in-process domain event bus
A standalone, domain-agnostic pub/sub seam: an EventBus that owns its
subscriber registry and streams Event values from producers to listeners
in process. Boundary-crossing (Celery/DB/workers) is left to subscribers,
keeping the bus single-responsibility. Includes the immutable Event value
object and full unit coverage.
2026-05-29 15:26:12 +02:00
DESKTOP-RTLN3BA\$punk
43c66008e4 fix(llm-role-manager): synchronize local state with preferences updates
- Added useEffect to sync local assignments state with preferences when they change, ensuring the UI reflects the latest data.
- Updated state initialization to use null instead of empty strings for clarity in role assignments.
- Adjusted role assignment handling to correctly manage "unassigned" values and preserve Auto mode configuration during updates.
2026-05-29 03:30:22 -07:00
DESKTOP-RTLN3BA\$punk
409fec94c3 feat(automations): implement model eligibility checks for automation creation
- Added model eligibility checks to ensure automations can only use billable models (premium or BYOK).
- Introduced new API endpoint to report model eligibility status for search spaces.
- Updated frontend components to display eligibility alerts and disable creation options when models are not billable.
- Enhanced automation creation forms to reflect model eligibility, preventing users from submitting invalid configurations.
- Implemented server-side logic to capture and preserve model preferences across automation edits, ensuring consistent behavior during execution.
2026-05-29 03:13:46 -07:00
Anish Sarkar
389a51d494 feat(gateway): enhance WhatsApp bridge with pairing timeout and health check integration 2026-05-29 13:37:45 +05:30
Anish Sarkar
9143cb6ad1 feat(gateway): update WhatsApp bridge configuration and expose port 9929 2026-05-29 11:39:11 +05:30
Rohan Verma
5d90fbe99f
Merge pull request #1445 from MODSetter/dev_mod
feat: added basic UI for automations and removed surfsense docs in chat related code.
2026-05-28 22:41:07 -07:00
DESKTOP-RTLN3BA\$punk
40ca9e6ed2 refactor: remove search_surfsense_docs tool and related references
- Deleted the `search_surfsense_docs` tool and its associated files, streamlining the agent's toolset.
- Updated various components and prompts to remove references to the now-removed tool, ensuring consistency across the codebase.
- Adjusted documentation to direct users to the SurfSense documentation link for product-related queries instead.
2026-05-28 22:35:14 -07:00
Anish Sarkar
bba33b5947 feat(web): add WhatsApp messaging channel controls 2026-05-29 10:21:37 +05:30
Anish Sarkar
185759de1f feat(gateway): register multi-platform gateway routes 2026-05-29 10:20:43 +05:30
Anish Sarkar
51bf2a8361 feat(gateway): wire WhatsApp bridge runtime 2026-05-29 10:20:25 +05:30
Anish Sarkar
76a594ac60 feat(gateway): add self-hosted WhatsApp bridge service 2026-05-29 10:20:07 +05:30
Anish Sarkar
63f9fe61b5 feat(gateway): expose Baileys pairing endpoints 2026-05-29 10:19:29 +05:30
Anish Sarkar
3b529a3ab2 feat(gateway): add Baileys WhatsApp adapter 2026-05-29 10:19:13 +05:30
Anish Sarkar
dbd9966219 feat(gateway): add WhatsApp Cloud webhook intake 2026-05-29 10:18:58 +05:30
Anish Sarkar
3d9620275b feat(gateway): handle WhatsApp Cloud conversations 2026-05-29 10:18:45 +05:30
Anish Sarkar
daa123832e feat(gateway): add WhatsApp Cloud adapter 2026-05-29 10:18:28 +05:30
Anish Sarkar
a6b2882275 feat(gateway): add WhatsApp gateway configuration 2026-05-29 10:18:11 +05:30
Anish Sarkar
5048b0fd7c refactor(gateway): route inbound events through platform bundles 2026-05-29 10:17:57 +05:30
Anish Sarkar
e953be5e99 refactor(gateway): abstract platform command handling 2026-05-29 10:17:41 +05:30
Anish Sarkar
f6eb955676 refactor(gateway): share outbound text splitting 2026-05-29 10:17:28 +05:30
DESKTOP-RTLN3BA\$punk
9b9e6828c7 refactor(automations): enhance mention handling in task parameters
- Updated the `mentionParams` function to separate document and connector mentions, improving clarity and organization of the output.
- Modified the `mentionsFromParams` function to correctly handle and categorize mentions from parameters, ensuring connectors are processed separately.
- Adjusted documentation comments for better understanding of the changes in mention handling.
2026-05-28 21:44:22 -07:00
DESKTOP-RTLN3BA\$punk
6b76f8c138 refactor(automations): update icons and button labels in empty state and header components
- Replaced the FileJson icon with SquarePen in both AutomationsEmptyState and AutomationsHeader components.
- Updated button label from "Create via JSON" to "Create manually" for clarity in the automation creation process.
2026-05-28 21:29:24 -07:00
DESKTOP-RTLN3BA\$punk
d013617bf6 feat(automations): added UI and improved mentions
- Added support for @-mentions in agent tasks, allowing users to reference documents, folders, and connectors directly in their queries.
- Updated `run_agent_task` to resolve mentions and include them in the context passed to the agent.
- Introduced new parameters in `AgentTaskActionParams` for handling mentioned document and connector IDs.
- Refactored the automation edit and new components to utilize the new `AutomationBuilderForm` for a more streamlined user experience.
- Removed deprecated JSON forms to simplify the automation creation process.
2026-05-28 21:26:32 -07:00
DESKTOP-RTLN3BA\$punk
c601a9b102 fix: biome errs 2026-05-28 19:22:54 -07:00
DESKTOP-RTLN3BA\$punk
94e834134f chore: linting 2026-05-28 19:21:29 -07:00
Rohan Verma
4dda02c06c
Merge pull request #1443 from CREDO23/feature-automations
[Feat] Automation V1 — Scheduled Agent Tasks, Created via Chat (HITL) or JSON
2026-05-28 12:41:41 -07:00
CREDO23
958bf9f95a fix(automations/agent_task): use in-memory checkpointer to avoid Celery PoolTimeout
The shared AsyncPostgresSaver caches DB connections in a module-level
pool. Cached connections are bound to the asyncio loop that opened
them, but `run_async_celery_task` discards the loop on each task's
exit — so after the first task the pool holds connections pointing
to a dead loop, and the next automation hangs 30s before failing
with `PoolTimeout: couldn't get a connection after 30.00 sec`.

Swap agent_task to `InMemorySaver`; automation runs only need state
within one Celery task, so nothing is lost. Site-local TODO tracks
the proper future fix (dispose the checkpointer pool around each
Celery task, mirroring `_dispose_shared_db_engine`).
2026-05-28 21:10:24 +02:00
CREDO23
353755fd73 test(automations): cross-cutting registries, enums, side-effects + shared fixtures
Top-level tests that span multiple submodules:

- test_stores.py (7): the trigger + action registry contracts — register
  round-trip, unknown type → None (not raise), duplicate registration
  rejected, defensive snapshot from all_*.
- test_definition_types.py (2): params_schema property on both
  ActionDefinition and TriggerDefinition reflects the Pydantic model.
- test_persistence_enums.py (3): exact string values + member sets of
  AutomationStatus / RunStatus / TriggerType — the postgres-mirrored
  contract that breaks stored rows if drifted.
- test_import_registrations.py (2): the bundled agent_task action and
  schedule trigger self-register on package import (canary for the
  side-effect import chain).

conftest.py adds isolated_action_registry / isolated_trigger_registry
fixtures: snapshot + restore of the module-level _REGISTRY dicts so
tests that add their own definitions don't leak across the suite.

14 tests, pure unit.
2026-05-28 19:03:55 +02:00
CREDO23
822940b09e test(automations/schemas): lock definition + api validation gates
definition/ (29 tests): the envelope (defaults, extra=forbid, empty
plan/name rejection), Inputs schema-alias roundtrip (Python schema_ ↔
wire schema), PlanStep numeric bounds + addressing-field constraints,
Execution production defaults stability (10-min timeout, 2 retries,
exponential backoff, drop_if_running) + closed-set Literal gates,
Metadata's exceptional extra="allow" contract, and TriggerSpec type
requirement.

api/ (9 tests): AutomationCreate/Update cascade-validate into the
nested definition, reject unknown payload fields, enforce name length;
TriggerCreate exposes safe defaults (enabled=True, params={},
static_inputs={}) and rejects unknown TriggerType strings at the
boundary.

All pure unit, no DB.
2026-05-28 19:03:42 +02:00
CREDO23
acbeb60a43 test(automations/actions): lock agent_task helpers (auto_decide + finalize)
auto_decide.build_auto_decisions (3): produces one decision per
action_request entry, defaults to one decision for legacy scalar
interrupts, and skips malformed interrupts silently so a misbehaving
tool can't take down the whole agent_task step.

finalize.extract_final_assistant_message (4): string-content AIMessage
returned verbatim, list-of-parts content concatenated (skipping
non-text parts like tool_use), walks back past trailing ToolMessages
to find the last AIMessage, and returns None when no extractable text
is present (so callers can branch on silence vs. empty).

7 tests, pure unit.
2026-05-28 19:03:29 +02:00
CREDO23
db4eef651f test(automations/templating): lock render, filters, environment, context
render.py (4): variable substitution, StrictUndefined raises on missing
keys, evaluate_predicate coerces to bool, render_value walks dicts/lists
and renders string leaves.

filters.py (4): slugify produces URL-safe output, date formats datetime
with strftime, date(None) → "" so templates can write
{{ inputs.last_fired_at | date }} on first run, date(str) passes through.

environment.py (4): the sandbox boundary — disallowed Jinja built-ins
(e.g. pprint) raise, and the finalize hook coerces non-string outputs
to predictable wire shapes (datetime → ISO, None → "", dict → JSON).

context.py (1): build_run_context exposes {run, inputs, steps} with the
exact shape every plan template body relies on.

13 tests total, all pure unit.
2026-05-28 19:03:22 +02:00
CREDO23
49af95b652 test(automations/runtime): lock execute_step + with_retries
execute_step (6 tests): happy path, when=falsy → skipped, unknown action
→ ActionNotFound failure, retry budget exhaustion (attempts = 1 +
max_retries), retry recovery, and template-rendering of step params
against the run context.

with_retries (3 tests): first-try success returns attempts=1, recovery
returns the actual attempt that produced the result, and exhaustion
re-raises the last exception with the handler called 1 + max_retries
times.

All tests use backoff="none" to keep wall-clock time zero; timeout
testing is intentionally skipped (would need >= 1s per the int contract,
and exhaustion already locks that any Exception triggers retry).
2026-05-28 19:03:08 +02:00
CREDO23
18b4800e49 test(automations/dispatch): lock _validate_inputs + DispatchError
Cover the input-validation contract dispatch_run relies on:

- no declared schema → inputs pass through unchanged (regression site
  that previously stripped runtime keys like fired_at / last_fired_at
  and broke Jinja templates).
- declared schema, valid inputs → passthrough validated.
- declared schema, invalid inputs → DispatchError (uniform exception
  type, not raw jsonschema.ValidationError).

Plus the DispatchError exception identity (Exception subclass, message
preserved, isinstance-friendly for the dispatch layer's consumers).

4 tests, pure unit.
2026-05-28 19:03:00 +02:00
CREDO23
2a76f43387 test(automations/triggers): lock schedule cron + params
Cover the cron + IANA timezone + UTC normalization contract for the
schedule trigger: next-match strictly-after, DST offset shift across
spring-forward, malformed cron / unknown timezone rejection, and the
ScheduleTriggerParams Pydantic gate that surfaces InvalidCronError as
ValidationError at the API boundary.

8 tests, pure unit (no DB, no mocks).
2026-05-28 19:02:52 +02:00
CREDO23
4f202e1fa3 feat(web): inline edit on trigger cards 2026-05-28 17:58:00 +02:00
CREDO23
fa0cdb9760 feat(web): unified json viewer/editor + edit existing automation 2026-05-28 16:07:54 +02:00
CREDO23
2d8d42bd9c refactor(web): polish automations detail view 2026-05-28 15:40:18 +02:00
CREDO23
b90bed2dbd chore: drop local design plan 2026-05-28 15:38:57 +02:00
Anish Sarkar
3faaa25af6 refactor(database): update default source values to 'surfsense' for chat threads and messages 2026-05-28 13:11:05 +05:30
CREDO23
91962ba879 fix automation run inputs, hitl routing, and detail UI polish 2026-05-28 02:48:47 +02:00
CREDO23
ed8d56aa16 feat(web): create automation via raw JSON 2026-05-28 01:44:13 +02:00
CREDO23
4625bd937e feat(web): run history section on automations detail page
Recent runs card under triggers. Each row expands lazily to fetch the
full run (step results, output, artifacts, error). 20-row cap for now;
real pagination lands if usage demands it.
2026-05-28 01:35:48 +02:00
Anish Sarkar
c958fe5bc6 feat(gateway): introduce GATEWAY_TELEGRAM_INTAKE_MODE for Telegram integration 2026-05-28 05:02:07 +05:30
CREDO23
2e572d7818 feat(web): create_automation HITL approval card in chat
Closes the create loop in chat: the agent describes user intent → the
drafter sub-LLM produces an AutomationCreate JSON → this card surfaces
a structured preview → approve persists; reject cancels. Edits flow
through chat refinement (re-call with a refined intent), not in-card,
so the card stays simple and the multi-turn checkpointer carries the
context.

Tool UI (components/tool-ui/automation/):
- create-automation.tsx — entry dispatcher + ApprovalCard chrome
  (pending/processing/complete/rejected via useHitlPhase) + SavedCard
  (links to the detail page) + InvalidCard (lists drafter validation
  issues) + ErrorCard (verbatim message). Rejection result is hidden
  because the approval card itself shows the rejected phase inline.
- automation-draft-preview.tsx — structured preview body: name +
  description + goal, triggers (humanised cron + tz + static-input
  keys), plan steps (step_id → action), and a collapsible raw JSON
  for power users.

Wiring:
- components/tool-ui/index.ts — re-export.
- features/chat-messages/timeline/tool-registry/registry.ts —
  register create_automation → CreateAutomationToolUI (dynamic import,
  same pattern as other connector tools).
- contracts/enums/toolIcons.tsx — Workflow icon + "Create automation"
  display name so fallback chrome (and timeline headers) are honest.

Shared util:
- lib/automations/describe-cron.ts — lifted from the route slice's
  lib/ folder since both the dashboard slice and the new approval card
  now render schedule descriptions. Slice imports updated; the now-
  empty slice lib/ folder is gone.

Backend prompt fragments:
- main_agent/system_prompt/.../create_automation/description.md and
  the tool's docstring no longer promise in-card edits. They make the
  refinement path explicit: if the user wants changes after seeing the
  draft, they reply in chat and the agent calls the tool again with a
  refined intent.

v1 deliberately excludes:
- In-card edit form / right-side edit panel — defer until we see real
  demand. The chat refinement loop covers the common case.
- approve_always / persistent allow rules — automations are a single
  artifact, not a repeated mutation, so the "trust this kind of call"
  affordance doesn't apply.
2026-05-28 01:32:04 +02:00
CREDO23
c0a9ea368f feat(web): automations detail page (definition viewer + trigger manager)
Vertical slice at /dashboard/[id]/automations/[automation_id]. Branches
in the orchestrator are: perms loading → skeleton, no-access → access
denied panel, bad id → not-found, fetch loading → skeleton, fetch
error → not-found, loaded → header + definition + triggers.

Route:
- page.tsx — server boundary; extracts both ids.
- automation-detail-content.tsx — client orchestrator.

Header:
- automation-detail-header.tsx — back link, name, status badge,
  description, pause/resume + delete actions. Delete navigates back to
  the list via a new onDeleted hook on DeleteAutomationDialog so the
  list page (where the row just vanishes) stays unaffected.
- automation-not-found.tsx — 404/403/NaN-id panel. We don't
  distinguish missing vs. forbidden in the UI.

Definition (read-only in v1):
- automation-definition-section.tsx — wrapper Card; renders goal +
  tags + execution defaults + inputs schema (if present) + plan.
- plan-step-card.tsx — one step (when, output_as, retries, timeout,
  params JSON).
- execution-summary.tsx — timeout / max_retries / backoff /
  concurrency + on_failure step count.
- inputs-schema-preview.tsx — formatted JSON of inputs.schema; only
  rendered when the definition declares inputs.

Triggers:
- automation-triggers-section.tsx — wrapper Card, "Add via chat" CTA
  (creation is intent-driven, same philosophy as automations).
- trigger-card.tsx — schedule + timezone + cron, last/next fire
  hints, static_inputs JSON, enable Switch and remove button.
- delete-trigger-dialog.tsx — confirm + mutation atom.

Shared:
- lib/describe-cron.ts — moved out of automation-triggers-summary.tsx
  so both list and detail can describe schedules consistently
  (daily/weekdays/weekly/monthly/hourly, raw cron fallback).

Loading:
- automation-detail-loading.tsx — same shell as the loaded view so the
  layout doesn't jump on data arrival.

RBAC: each interactive surface is independently gated
(canUpdate/canDelete/canCreate) so the orchestrator stays thin and the
component tree is self-documenting about what each action requires.

Out of scope (later PRs):
- Editing definition / trigger params (raw-JSON path) — PR5
- Run history — PR6
2026-05-28 01:21:54 +02:00
Anish Sarkar
7ff0120fc9 feat(gateway): add worker and webhook metrics 2026-05-28 04:44:28 +05:30
CREDO23
bc3c2fd515 fix(web): hide header Create CTA on the automations empty state
The empty-state card already hosts the primary "Create via chat" CTA;
keeping the header button on the same screen showed two identical
buttons. Adds an optional ``showCreateCta`` prop to AutomationsHeader
(default true) and turns it off only in the empty branch so the card
stays the focal point.
2026-05-28 01:14:10 +02:00