apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-05-29 19:35:20 +02:00
Code Issues Projects Releases Packages Wiki Activity
6090 commits 3 branches 232 tags 400 MiB
Commit graph

350 commits

Author SHA1 Message Date
CREDO23
fe32cd35ed refactor(automation): rename trigger config column to params 2026-05-27 13:29:18 +02:00
CREDO23
35117a952d refactor(automation): drop agent_session_id from AutomationRun 
A run can contain zero, one, or N agent_task steps. A single
agent_session_id at the run level holds at most one of them, so the
column is the wrong shape for the data.

Per-step session ids (LangGraph thread/checkpoint reference for an
agent_task step) live inside step_results[i] alongside the rest of
the per-step bag (status, timings, output). Each agent step records
its own; non-agent steps record nothing. Run-level "primary session"
is a UI concern, not a schema concern.

Trade-off: trace -> run reverse lookup is now a JSONB query, not an
index hit. Usually traversal goes run -> trace; if the reverse
becomes hot we add a GIN index on step_results or a generated
column — both additive.

Changes:
- AutomationRun: drop the agent_session_id column; module docstring
  notes where per-step session ids now live.
- Migration 144: drop the column from the CREATE TABLE; downgrade
  unchanged.

Safe to edit migration 144 in place (vs. add 145 with ALTER ... DROP):
this branch has not shipped and the table has never existed in any
deployed database.
 2026-05-27 11:41:32 +02:00
CREDO23
d9183464d9 feat(automation): add Alembic migration for the three automation tables 
Migration 144 -> 143. Matches the SQLAlchemy models added in commit 7
and the v1 data model in automation-design-plan.md §9.

Up:
  - CREATE TYPE automation_status / automation_trigger_type /
    automation_run_status (PostgreSQL ENUMs created first because the
    tables reference them).
  - CREATE TABLE automations with FK to searchspaces (CASCADE) and
    user (SET NULL); five indexes matching the SQLAlchemy model.
  - CREATE TABLE automation_triggers with FK to automations
    (CASCADE); four indexes.
  - CREATE TABLE automation_runs with FK to automations (CASCADE) and
    automation_triggers (SET NULL — null trigger_id == manual via UI);
    four indexes.

Down: drops every index, table, and ENUM in reverse-dependency order
so the migration is reversible without ON DELETE side effects.

Verified: `alembic history` resolves 143 -> 144 (head) cleanly.

domain_events (Phase 3) and mcp_connections / mcp_tools (Phase 4) ship
in their own migrations when the consuming feature lands; this
migration only covers the three v1 tables.
 2026-05-26 22:44:33 +02:00
Anish Sarkar
af1d2fa430 Merge remote-tracking branch 'upstream/dev' into fix/zero-cache-stale-replica-1355 2026-05-16 19:30:09 +05:30
Anish Sarkar
c43bfdb1d9 chore(migration): update migration files to enforce new publication mutation pattern 2026-05-12 17:31:49 +05:30
Anish Sarkar
d923d34e38 feat(migration): add migration 143 to force zero-cache resync after Zero upgrade 2026-05-12 16:43:50 +05:30
Anish Sarkar
2c8828f60c fix: ensure idempotency in alembic migrations by checking for existing columns and indexes before creation 2026-05-10 22:45:26 +05:30
Anish Sarkar
319923fb40 fix: add checks for existing tables and indexes before creating them in alembic migrations for idempotency 2026-05-10 22:40:29 +05:30
Anish Sarkar
292b4d70ac chore: enhance E2E tests workflow by adding caching for Next.js build and updating test command 2026-05-10 22:21:06 +05:30
DESKTOP-RTLN3BA\$punk
19b6e0a025 feat: moved chat persistance to Server Side 2026-05-04 03:06:15 -07:00
DESKTOP-RTLN3BA\$punk
ae9d36d77f feat: unified credits and its cost calculations 2026-05-02 14:34:23 -07:00
Anish Sarkar
cd25175b84 chore: ran linting 2026-05-02 03:36:13 +05:30
Anish Sarkar
e9d964514b feat(alembic): add user table to zero_publication for selective replication of usage metrics 2026-05-02 03:31:03 +05:30
Anish Sarkar
421a4d7d08 refactor(auto_model_pin): simplify thread-level pinning by removing unused fields and indexes 2026-05-01 19:32:42 +05:30
DESKTOP-RTLN3BA\$punk
c644f02d05 chore: linting
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
 2026-04-30 18:42:38 -07:00
Anish Sarkar
2a01711bc9 feat(chat): expand error handling for chat operations by introducing a passthrough code set, improving response management and user feedback 2026-04-30 16:20:14 +05:30
Anish Sarkar
1ce122cc99 feat(database): change alembic number and add idempotency 2026-04-30 16:05:58 +05:30
Anish Sarkar
ba2a08b295 Merge remote-tracking branch 'upstream/dev' into feat/split-auto-free-premium 2026-04-30 15:45:11 +05:30
DESKTOP-RTLN3BA\$punk
c110f5b955 feat: improved agent streaming 2026-04-29 07:20:31 -07:00
Anish Sarkar
57db198919 feat(chat): add thread-level auto model pinning fields 2026-04-29 19:14:56 +05:30
DESKTOP-RTLN3BA\$punk
f9b5367754 chore: cleaned comments slop 2026-04-28 23:52:37 -07:00
DESKTOP-RTLN3BA\$punk
ca9bbee06d chore: linting 2026-04-28 21:37:51 -07:00
DESKTOP-RTLN3BA\$punk
b9a66cb417 feat: various UI fixes, prompt optimizations, and allowing duplicate docs 
- Updated `content_hash` in the `Document` model to remove global uniqueness, allowing identical content across different paths.
- Enhanced `_create_document` function to handle path uniqueness and prevent session-poisoning from `IntegrityError`.
- Added detailed comments for clarity on the changes and their implications.
- Introduced new citation handling in the editor for improved user experience with citation jumps.
- Updated package dependencies in the frontend for better functionality.
 2026-04-28 21:30:53 -07:00
DESKTOP-RTLN3BA\$punk
e6433f78c4 Merge commit '61f4d05cd1' into dev_mod 2026-04-28 09:25:41 -07:00
DESKTOP-RTLN3BA\$punk
31a372bb84 feat: updated agent harness 2026-04-28 09:22:19 -07:00
CREDO23
2d962f6dd2 Merge upstream/dev 2026-04-27 22:44:40 +02:00
CREDO23
dfa6c0423d Tidy alembic migration version scripts 2026-04-24 18:49:22 +02:00
Anish Sarkar
3eb4d55ef5 chore: ran linting 2026-04-22 06:40:39 +05:30
Anish Sarkar
9fded124fe feat: add IF NOT EXISTS clause to unique index creation for Obsidian connectors to prevent errors during upgrades 2026-04-21 04:37:45 +05:30
Anish Sarkar
54ce2666f5 feat: implement cross-device deduplication for Obsidian connectors using vault fingerprinting and enhance connector management 2026-04-21 04:21:33 +05:30
Anish Sarkar
2d90ed0fec feat: deactivate legacy Obsidian connectors and implement partial unique index for improved upsert handling 2026-04-21 03:18:44 +05:30
Anish Sarkar
e8fc1069bc feat: implement Obsidian plugin ingestion routes and indexing service 2026-04-20 04:03:19 +05:30
DESKTOP-RTLN3BA\$punk
7a389e7a25 fix: alembic migration nos 2026-04-18 16:05:18 -07:00
Anish Sarkar
2e026549be chore: update alembic migration numbers 2026-04-18 16:22:20 +05:30
Rohan Verma
2b2453e015
Merge pull request #1240 from AnishSarkar22/feat/resume-builder
Some checks failed
Build and Push Docker Images / tag_release (push) Has been cancelled
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Has been cancelled
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Has been cancelled
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Has been cancelled
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Has been cancelled
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Has been cancelled
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Has been cancelled
feat: resume builder
 2026-04-17 13:41:32 -07:00
DESKTOP-RTLN3BA\$punk
ff4e0f9b62 feat: no login experience and prem tokens
Some checks are pending
Build and Push Docker Images / tag_release (push) Waiting to run
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web) (push) Blocked by required conditions
 2026-04-15 17:02:00 -07:00
Anish Sarkar
586199ae17 feat: add content_type column to reports and seed default prompt for resume generation 2026-04-15 20:40:40 +05:30
Anish Sarkar
9fc0976d5e chore: ran linting 2026-04-14 21:52:26 +05:30
Anish Sarkar
8fd7664f8f Merge remote-tracking branch 'upstream/dev' into feat/token-calculation 2026-04-14 15:49:39 +05:30
Anish Sarkar
1f9840d4a4 feat: update alembic migration number 2026-04-14 15:49:33 +05:30
DESKTOP-RTLN3BA\$punk
4bee367d4a feat: added ai file sorting 2026-04-14 01:43:30 -07:00
Anish Sarkar
917f35eb33 feat: add token_usage table and relationships for tracking LLM token consumption 2026-04-14 13:26:53 +05:30
CREDO23
ec3bd2f5f4 fix: renumber vision_llm migration from 121 to 123 to resolve conflict with upstream 2026-04-11 11:02:44 +02:00
CREDO23
c30cc08771 Merge upstream/dev into feat/kb-export-and-folder-upload 2026-04-11 10:28:40 +02:00
CREDO23
a95bf58c8f Make Vision LLM opt-in for uploads and connectors 2026-04-10 16:45:51 +02:00
Anish Sarkar
8c9440998a chore: ran linting 2026-04-10 14:20:11 +05:30
Anish Sarkar
a5bf3af4f8 feat: migrate legacy memory tables to markdown format and drop old structures 2026-04-10 14:19:20 +05:30
Anish Sarkar
f38ea77940 chore: ran linting 2026-04-09 18:10:34 +05:30
Anish Sarkar
b8091114b5 feat: drop legacy memory tables and update memory management tools to streamline user and shared memory handling 2026-04-09 18:05:30 +05:30
Anish Sarkar
ba7e288879 feat: add memory_md columns to user and searchspaces tables 2026-04-08 23:21:24 +05:30