Commit graph

1273 commits

Author SHA1 Message Date
CREDO23
c6cb754aac refactor: update the webcrawler index to compare hashes without metadata 2025-12-17 18:44:58 +02:00
CREDO23
4cfeffb38a refactor: update the webcrawler connector formater 2025-12-17 18:42:37 +02:00
DESKTOP-RTLN3BA\$punk
c768730b8c feat: fixed issues of note management
Issues Fixed

- Missing pagination fields in API response schemas (page, page_size, has_more)
- NOTE enum missing from frontend Zod schema
- Missing fields in DocumentRead response construction (content_hash, updated_at)
- BlockNote slash menu clipped by overflow-hidden CSS
- Sidebar click conflicts - hidden action buttons intercepting clicks
- Rewrote All Notes sidebar - replaced fragile custom portal with shadcn Sheet
- Missing translation keys for new UI strings
- Missing NOTE retrieval logic in researcher agent
- Added search to All Notes sidebar
- Removed frontend logging - was causing toasters on every page refresh
- Added backend logging to document reindex Celery task
2025-12-17 00:09:43 -08:00
Anish Sarkar
4cef8cc93f feat: enhance validation and formatting in editor and sidebar components 2025-12-16 21:27:31 +05:30
Anish Sarkar
7bd638668a Merge remote-tracking branch 'upstream/main' into feature/note-management
updating
2025-12-16 20:28:35 +05:30
Anish Sarkar
e8e393a0d5 fix: fixed linting for some files 2025-12-16 20:25:50 +05:30
Anish Sarkar
e1e813702a feat: enhance note creation and editing experience
- Added dynamic title extraction from the first block of BlockNote documents for NOTE type.
- Updated editor routes to support new note creation with a BlockNote editor.
- Implemented unsaved changes dialog for better user experience when navigating away from the editor.
- Refactored BlockNoteEditor to ensure the first block is a heading when creating new notes.
- Removed the old note creation page in favor of the new streamlined editor experience.
2025-12-16 13:57:06 +05:30
Anish Sarkar
8eceb7a5cb feat: add note management functionality with BlockNote support
- Introduced a new ENUM value 'NOTE' for document types in the database.
- Implemented backend routes for creating, listing, and deleting notes.
- Added a new frontend page for creating notes with a BlockNote editor.
- Updated sidebar to include recent notes and an option to add new notes.
- Enhanced API service for notes with validation and request/response schemas.
- Updated translations to support new note-related terms.
2025-12-16 12:28:30 +05:30
DESKTOP-RTLN3BA\$punk
fd1f3eaf60 chore: update langchain
- Updated langchain-unstructured to version 1.0.0 and langchain to 1.2.0.
- Updated litellm to version 1.80.10 and grpcio versions based on Python version.
- Updated fastuuid to version 0.14.0 and langgraph to 1.0.5.
- Updated openai to version 2.12.0 and langgraph-sdk to 0.3.0.
- Added uuid-utils package version 0.12.0.
- Refactored imports in utils.py, chats_routes.py, and query_service.py to use langchain_core.messages.
2025-12-15 21:25:44 -08:00
DESKTOP-RTLN3BA\$punk
fea1837186 roadmap(1.3): Update citation prompt to use new whole document structure
- Modified the document extraction and citation formatting to accommodate a new structure that includes a `chunks` list for each document.
- Enhanced the citation format to reference `chunk_id` instead of `source_id`, ensuring accurate citations in the UI.
- Updated various components, including the connector service and reranker service, to handle the new document format and maintain compatibility with existing functionalities.
- Improved documentation and comments to reflect changes in the data structure and citation requirements.
2025-12-14 22:07:31 -08:00
DESKTOP-RTLN3BA\$punk
b46349ae3a chore: Update dependencies in pyproject.toml and uv.lock
- Removed outdated dependencies: langchain-community, langchain-unstructured, langgraph, langchain-litellm, and litellm.
- Added updated versions of langchain-community (0.3.31), langchain-unstructured (0.1.6), langgraph (1.0.1), langchain-litellm (0.3.5), and litellm (1.77.5).
- Updated langchain (0.3.27 to 0.3.80) and langchain-community (0.3.25 to 0.3.31) in uv.lock.
- Updated langgraph (0.4.8 to 1.0.1) and langchain-litellm (0.2.3 to 0.3.5) in uv.lock.
- Updated xxhash version from 3.5.0 to 3.6.0 in uv.lock.
2025-12-14 00:52:01 -08:00
CREDO23
cc03900d63 fix alembic version 46 2025-12-13 22:04:53 +02:00
CREDO23
84100380c4 fix migration version 46 | upgrade 2025-12-13 21:49:45 +02:00
CREDO23
181df4237e fix migration version 45 | upgrade 2025-12-13 21:48:54 +02:00
CREDO23
88eb557161 fix migration version 43 | upgrade 2025-12-13 21:48:13 +02:00
CREDO23
85f4068735 fix migration version 40 | upgrade 2025-12-13 21:47:20 +02:00
CREDO23
c0b414fefa fix migration version 29 2025-12-13 21:46:39 +02:00
CREDO23
abc1981e6c fix migration version 8 2025-12-13 21:45:45 +02:00
Rohan Verma
1252817ac4
Merge pull request #570 from waychan23/main
fix: Web Pages connector not working due to lack of dependencies
2025-12-12 12:23:05 -08:00
DESKTOP-RTLN3BA\$punk
08fb488995 feat(roadmap-1.2): removed SearchMode enum and combined hybrid search 2025-12-12 04:17:30 -08:00
DESKTOP-RTLN3BA\$punk
919c323ef3 feat: implement time-based filtering for document retrieval using 'updated_at' timestamp 2025-12-12 02:42:20 -08:00
DESKTOP-RTLN3BA\$punk
8c9aa68faa feat: update document tracking to use 'updated_at' timestamp instead of 'last_edited_at' 2025-12-12 01:32:14 -08:00
waychan23
ce2fb5d012 fix: (for docker installation) Web Pages connector not working due to lack of dependencies 2025-12-12 15:08:01 +08:00
DESKTOP-RTLN3BA\$punk
1cbe479af4 chore: ruff format 2025-12-11 00:31:58 -08:00
DESKTOP-RTLN3BA\$punk
3207fbf9ea feat(fix): add PAGES_LIMIT configuration for ETL services 2025-12-11 00:29:56 -08:00
subbareddyalamur
24c6683471
Add boto3 dependency to pyproject.toml
This is required to add AWS Bedrock as LLM Provider
2025-12-10 10:15:52 +05:30
DESKTOP-RTLN3BA\$punk
3c5efee252 fix: add dos2unix to Dockerfile to handle line endings for entrypoint script 2025-12-07 02:01:09 -08:00
DESKTOP-RTLN3BA\$punk
bdffb17505 fix: backend container fails with missing entrypoint when built locally 2025-12-06 21:31:32 -08:00
Rohan Verma
e1e14c7ab6
Merge pull request #534 from ankitpasayat/fix/retriever-typo
fix(backend): rename retriver directory to retriever
2025-12-06 17:54:18 -08:00
Ankit Pasayat
b30f53bf0b fix(backend): rename retriver directory to retriever
Fixes typo in directory name and updates all import paths:
- Renamed surfsense_backend/app/retriver/ to surfsense_backend/app/retriever/
- Updated imports in db.py
- Updated imports in connector_service.py
2025-12-06 22:21:30 +05:30
Differ
71d7856565 fix: update migration id from 39 to 44 after syncing with upstream 2025-12-06 09:16:24 +08:00
Differ
e238fab638 Merge remote-tracking branch 'upstream/main' into feat/bookstack-connector 2025-12-06 09:15:02 +08:00
Rohan Verma
601489bec2
Merge pull request #525 from CREDO23/refactor/jira-connector-migrate-tot-rest-v3
[Refactor] Jira Connector | Migrate to atlassian rest api v3
2025-12-05 12:14:46 -08:00
DESKTOP-RTLN3BA\$punk
c97887a63d fix: implement real-time streaming for responses
- Added streaming service support to the Q&A agent for real-time token streaming.
- Updated `answer_question` method to stream responses token-by-token to the frontend.
- Modified `handle_qna_workflow` to handle both custom and values streaming modes.
- Enhanced state management to include streaming service for improved user experience.
2025-12-05 00:14:36 -08:00
Differ
0457e9f735 feat: add alembic migration for BOOKSTACK_CONNECTOR enum 2025-12-05 15:38:38 +08:00
Differ
500bc60d02 fix: add input validation, retry limit, code formatting, and exclude i18n from secret detection 2025-12-05 09:58:49 +08:00
CREDO23
803f792a9d clean up 2025-12-04 12:55:19 +02:00
CREDO23
521cea3ef0 update query parmas for get issues by date range method 2025-12-04 12:53:18 +02:00
Differ
6b1b8d0f2e feat: add BookStack connector for wiki documentation indexing 2025-12-04 14:08:44 +08:00
WayChan
3c423436cc fix: retrieve wrong field for content in saving extension document. 2025-12-04 00:31:50 +00:00
CREDO23
107f013ff9 jira-connector: update get_issues_by_date_range method 2025-12-04 01:21:46 +02:00
CREDO23
abf017eabb jira-connector: update get_issues_by_date_range method 2025-12-04 00:48:54 +02:00
CREDO23
4df6b09db9 jira-connector: update get all issues method 2025-12-04 00:42:10 +02:00
CREDO23
875924e5fd jira-connector: update make_api_request to accespt POST with payload 2025-12-04 00:38:13 +02:00
WayChan
081080233a fix: saving document from browser extension fails due to missing and mismatch fields of backend data models 2025-12-03 15:32:32 +00:00
DESKTOP-RTLN3BA\$punk
ea94c778c9 refactor: streamline document actions
- Simplified the RowActions component by removing unused imports and replacing the dropdown menu with tooltip buttons for editing, viewing metadata, and deleting documents.
- Updated the JsonMetadataViewer component to support controlled mode for better integration with the RowActions component.
- Adjusted CSS variables for destructive actions in globals.css for improved visual consistency.
2025-11-30 15:06:48 -08:00
DESKTOP-RTLN3BA\$punk
d0c7be7eca feat: update document editor routes to include search space context
- Modified API endpoints to include search space ID in the URL for fetching and saving documents.
- Added permission checks for reading and updating documents based on search space.
2025-11-30 14:27:27 -08:00
Anish Sarkar
f92112a79a fix: code cleanup 2025-11-30 11:37:50 +05:30
Anish Sarkar
5e53207edc refactor: update alembic migration revision ID and added some defaults for blocknote in file_processors.py file 2025-11-30 04:57:07 +05:30
Anish Sarkar
4e6b4cc57d fixed: cleaned up code and added import in init 2025-11-30 04:25:27 +05:30