- Modified the document extraction and citation formatting to accommodate a new structure that includes a `chunks` list for each document.
- Enhanced the citation format to reference `chunk_id` instead of `source_id`, ensuring accurate citations in the UI.
- Updated various components, including the connector service and reranker service, to handle the new document format and maintain compatibility with existing functionalities.
- Improved documentation and comments to reflect changes in the data structure and citation requirements.
- Removed outdated dependencies: langchain-community, langchain-unstructured, langgraph, langchain-litellm, and litellm.
- Added updated versions of langchain-community (0.3.31), langchain-unstructured (0.1.6), langgraph (1.0.1), langchain-litellm (0.3.5), and litellm (1.77.5).
- Updated langchain (0.3.27 to 0.3.80) and langchain-community (0.3.25 to 0.3.31) in uv.lock.
- Updated langgraph (0.4.8 to 1.0.1) and langchain-litellm (0.2.3 to 0.3.5) in uv.lock.
- Updated xxhash version from 3.5.0 to 3.6.0 in uv.lock.
Fixes typo in directory name and updates all import paths:
- Renamed surfsense_backend/app/retriver/ to surfsense_backend/app/retriever/
- Updated imports in db.py
- Updated imports in connector_service.py
- Added streaming service support to the Q&A agent for real-time token streaming.
- Updated `answer_question` method to stream responses token-by-token to the frontend.
- Modified `handle_qna_workflow` to handle both custom and values streaming modes.
- Enhanced state management to include streaming service for improved user experience.
- Simplified the RowActions component by removing unused imports and replacing the dropdown menu with tooltip buttons for editing, viewing metadata, and deleting documents.
- Updated the JsonMetadataViewer component to support controlled mode for better integration with the RowActions component.
- Adjusted CSS variables for destructive actions in globals.css for improved visual consistency.
- Modified API endpoints to include search space ID in the URL for fetching and saving documents.
- Added permission checks for reading and updating documents based on search space.
- Updated the document saving endpoint to trigger reindexing after saving.
- Introduced a new Celery task for reindexing documents.
- Refactored the editor page to reflect the changes in the API endpoint and method.
- Deleted UserSearchSpacePreference class and its relationships from SearchSpace and User models.
- Cleaned up unused code to streamline database interactions.
-Introduce granular permissions for documents, chats, podcasts, and logs.
- Update routes to enforce permission checks for creating, reading, updating, and deleting resources. - Refactor user and search space interactions to align with RBAC model, removing ownership checks in favor of permission validation.
