apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-04-25 00:36:31 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

feat(removed): sub_section_writer

Browse source 
- Its bad and not needed.
This commit is contained in:
DESKTOP-RTLN3BA\$punk 2025-10-27 20:30:10 -07:00
parent 5ac6ebf199
commit 81ddc81026
57 changed files with 2213 additions and 4023 deletions
Download patch file Download diff file Expand all files Collapse all files

118
surfsense_backend/README.md
View file

@ -1,118 +0,0 @@
# Surf Backend
## Technology Stack Overview
This application is a modern AI-powered search and knowledge management platform built with the following technology stack:
### Core Framework and Environment
- **Python 3.12+**: The application requires Python 3.12 or newer
- **FastAPI**: Modern, fast web framework for building APIs with Python
- **Uvicorn**: ASGI server implementation, running the FastAPI application
- **PostgreSQL with pgvector**: Database with vector search capabilities for similarity searches
- **SQLAlchemy**: SQL toolkit and ORM (Object-Relational Mapping) for database interactions
- **FastAPI Users**: Authentication and user management with JWT and OAuth support
### Key Features and Components
#### Authentication and User Management
- JWT-based authentication
- OAuth integration (Google)
- User registration, login, and password reset flows
#### Search and Retrieval System
- **Hybrid Search**: Combines vector similarity and full-text search for optimal results using Reciprocal Rank Fusion (RRF)
- **Vector Embeddings**: Document and text embeddings for semantic search
- **pgvector**: PostgreSQL extension for efficient vector similarity operations
- **Chonkie**: Advanced document chunking and embedding library
- Uses `AutoEmbeddings` for flexible embedding model selection
- `LateChunker` for optimized document chunking based on embedding model's max sequence length
#### AI and NLP Capabilities
- **LangChain**: Framework for developing AI-powered applications
- Used for document processing, research, and response generation
- Integration with various LLM models through LiteLLM
- Document conversion utilities for standardized processing
- **GPT Integration**: Integration with LLM models through LiteLLM
- Multiple LLM configurations for different use cases:
- Fast LLM: Quick responses (default: gpt-4o-mini)
- Smart LLM: More comprehensive analysis (default: gpt-4o-mini)
- Strategic LLM: Complex reasoning (default: gpt-4o-mini)
- Long Context LLM: For processing large documents (default: gemini-2.0-flash-thinking)
- **Rerankers with FlashRank**: Advanced result ranking for improved search relevance
- Configurable reranking models (default: ms-marco-MiniLM-L-12-v2)
- Supports multiple reranking backends (FlashRank, Cohere, etc.)
- Improves search result quality by reordering based on semantic relevance
- **GPT-Researcher**: Advanced research capabilities
- Multiple research modes (GENERAL, DEEP, DEEPER)
- Customizable report formats with proper citations
- Streaming research results for real-time updates
#### External Integrations
- **Slack Connector**: Integration with Slack for data retrieval and notifications
- **Notion Connector**: Integration with Notion for document retrieval
- **Search APIs**: Integration with Tavily and Serper API for web search
- **Firecrawl**: Web crawling and data extraction capabilities
#### Data Processing
- **Unstructured**: Tools for processing unstructured data
- **Markdownify**: Converting HTML to Markdown
- **Playwright**: Web automation and scraping capabilities
#### Main Modules
- **Search Spaces**: Isolated search environments for different contexts or projects
- **Documents**: Storage and retrieval of various document types
- **Chunks**: Document fragments for more precise retrieval
- **Chats**: Conversation management with different depth levels (GENERAL, DEEP)
- **Podcasts**: Audio content management with generation capabilities
- **Search Source Connectors**: Integration with various data sources
### Development Tools
- **Poetry**: Python dependency management (indicated by pyproject.toml)
- **CORS support**: Cross-Origin Resource Sharing enabled for API access
- **Environment Variables**: Configuration through .env files
## Database Schema
The application uses a relational database with the following main entities:
- Users: Authentication and user management
- SearchSpaces: Isolated search environments owned by users
- Documents: Various document types with content and embeddings
- Chunks: Smaller pieces of documents for granular retrieval
- Chats: Conversation tracking with different depth levels
- Podcasts: Audio content with generation capabilities
- SearchSourceConnectors: External data source integrations
## API Endpoints
The API is structured with the following main route groups:
- `/auth/*`: Authentication endpoints (JWT, OAuth)
- `/users/*`: User management
- `/api/v1/search-spaces/*`: Search space management
- `/api/v1/documents/*`: Document management
- `/api/v1/podcasts/*`: Podcast functionality
- `/api/v1/chats/*`: Chat and conversation endpoints
- `/api/v1/search-source-connectors/*`: External data source management
## Deployment
The application is configured to run with Uvicorn and can be deployed with:
```
python main.py
```
This will start the server on all interfaces (0.0.0.0) with info-level logging.
## Requirements
See pyproject.toml for detailed dependency information. Key dependencies include:
- asyncpg: Asynchronous PostgreSQL client
- chonkie: Document chunking and embedding library
- fastapi and related packages
- fastapi-users: Authentication and user management
- firecrawl-py: Web crawling capabilities
- langchain components for AI workflows
- litellm: LLM model integration
- pgvector: Vector similarity search in PostgreSQL
- rerankers with FlashRank: Advanced result ranking
- Various AI and NLP libraries
- Integration clients for Slack, Notion, etc.

11
surfsense_backend/app/agents/researcher/configuration.py
View file

@ -15,27 +15,16 @@ class SearchMode(Enum):
DOCUMENTS = "DOCUMENTS"
class ResearchMode(Enum):
"""Enum defining the type of research mode."""
QNA = "QNA"
REPORT_GENERAL = "REPORT_GENERAL"
REPORT_DEEP = "REPORT_DEEP"
REPORT_DEEPER = "REPORT_DEEPER"
@dataclass(kw_only=True)
class Configuration:
"""The configuration for the agent."""
# Input parameters provided at invocation
user_query: str
num_sections: int
connectors_to_search: list[str]
user_id: str
search_space_id: int
search_mode: SearchMode
research_mode: ResearchMode
document_ids_to_add_in_context: list[int]
language: str | None = None

55
surfsense_backend/app/agents/researcher/graph.py
View file

@ -1,34 +1,23 @@
from typing import Any, TypedDict
from langgraph.graph import StateGraph
from .configuration import Configuration, ResearchMode
from .configuration import Configuration
from .nodes import (
generate_further_questions,
handle_qna_workflow,
process_sections,
reformulate_user_query,
write_answer_outline,
)
from .state import State
# Define what keys are in our state dict
class GraphState(TypedDict):
# Intermediate data produced during workflow
answer_outline: Any | None
# Final output
final_written_report: str | None
def build_graph():
"""
Build and return the LangGraph workflow.
This function constructs the researcher agent graph with conditional routing
based on research_mode - QNA mode uses a direct Q&A workflow while other modes
use the full report generation pipeline. Both paths generate follow-up questions
at the end using the reranked documents from the sub-agents.
This function constructs the researcher agent graph for Q&A workflow.
The workflow follows a simple path:
1. Reformulate user query based on chat history
2. Handle QNA workflow (fetch documents and generate answer)
3. Generate follow-up questions
Returns:
A compiled LangGraph workflow
@ -39,40 +28,12 @@ def build_graph():
# Add nodes to the graph
workflow.add_node("reformulate_user_query", reformulate_user_query)
workflow.add_node("handle_qna_workflow", handle_qna_workflow)
workflow.add_node("write_answer_outline", write_answer_outline)
workflow.add_node("process_sections", process_sections)
workflow.add_node("generate_further_questions", generate_further_questions)
# Define the edges
# Define the edges - simple linear flow for QNA
workflow.add_edge("__start__", "reformulate_user_query")
# Add conditional edges from reformulate_user_query based on research mode
def route_after_reformulate(state: State, config) -> str:
"""Route based on research_mode after reformulating the query."""
configuration = Configuration.from_runnable_config(config)
if configuration.research_mode == ResearchMode.QNA.value:
return "handle_qna_workflow"
else:
return "write_answer_outline"
workflow.add_conditional_edges(
"reformulate_user_query",
route_after_reformulate,
{
"handle_qna_workflow": "handle_qna_workflow",
"write_answer_outline": "write_answer_outline",
},
)
# QNA workflow path: handle_qna_workflow -> generate_further_questions -> __end__
workflow.add_edge("reformulate_user_query", "handle_qna_workflow")
workflow.add_edge("handle_qna_workflow", "generate_further_questions")
# Report generation workflow path: write_answer_outline -> process_sections -> generate_further_questions -> __end__
workflow.add_edge("write_answer_outline", "process_sections")
workflow.add_edge("process_sections", "generate_further_questions")
# Both paths end after generating further questions
workflow.add_edge("generate_further_questions", "__end__")
# Compile the workflow into an executable graph

595
surfsense_backend/app/agents/researcher/nodes.py
View file

@ -1,4 +1,3 @@
import asyncio
import json
import logging
import traceback
@ -17,15 +16,10 @@ from app.services.connector_service import ConnectorService
from app.services.query_service import QueryService
from .configuration import Configuration, SearchMode
from .prompts import (
get_answer_outline_system_prompt,
get_further_questions_system_prompt,
)
from .prompts import get_further_questions_system_prompt
from .qna_agent.graph import graph as qna_agent_graph
from .state import State
from .sub_section_writer.configuration import SubSectionType
from .sub_section_writer.graph import graph as sub_section_writer_graph
from .utils import AnswerOutline, get_connector_emoji, get_connector_friendly_name
from .utils import get_connector_emoji, get_connector_friendly_name
def extract_sources_from_documents(
@ -519,156 +513,6 @@ async def fetch_documents_by_ids(
return [], []
async def write_answer_outline(
state: State, config: RunnableConfig, writer: StreamWriter
) -> dict[str, Any]:
"""
Create a structured answer outline based on the user query.
This node takes the user query and number of sections from the configuration and uses
an LLM to generate a comprehensive outline with logical sections and research questions
for each section.
Returns:
Dict containing the answer outline in the "answer_outline" key for state update.
"""
from app.services.llm_service import get_user_strategic_llm
streaming_service = state.streaming_service
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
"🔍 Generating answer outline..."
)
}
)
# Get configuration from runnable config
configuration = Configuration.from_runnable_config(config)
reformulated_query = state.reformulated_query
user_query = configuration.user_query
num_sections = configuration.num_sections
user_id = configuration.user_id
search_space_id = configuration.search_space_id
language = configuration.language # Get language from configuration
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
f'🤔 Planning research approach for: "{user_query[:100]}..."'
)
}
)
# Get user's strategic LLM
llm = await get_user_strategic_llm(state.db_session, user_id, search_space_id)
if not llm:
error_message = f"No strategic LLM configured for user {user_id} in search space {search_space_id}"
writer({"yield_value": streaming_service.format_error(error_message)})
raise RuntimeError(error_message)
# Create the human message content
human_message_content = f"""
Now Please create an answer outline for the following query:
User Query: {reformulated_query}
Number of Sections: {num_sections}
Remember to format your response as valid JSON exactly matching this structure:
{{
"answer_outline": [
{{
"section_id": 0,
"section_title": "Section Title",
"questions": [
"Question 1 to research for this section",
"Question 2 to research for this section"
]
}}
]
}}
Your output MUST be valid JSON in exactly this format. Do not include any other text or explanation.
"""
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
"📝 Designing structured outline with AI..."
)
}
)
# Create messages for the LLM
messages = [
SystemMessage(content=get_answer_outline_system_prompt(language=language)),
HumanMessage(content=human_message_content),
]
# Call the LLM directly without using structured output
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
"⚙️ Processing answer structure..."
)
}
)
response = await llm.ainvoke(messages)
# Parse the JSON response manually
try:
# Extract JSON content from the response
content = response.content
# Find the JSON in the content (handle case where LLM might add additional text)
json_start = content.find("{")
json_end = content.rfind("}") + 1
if json_start >= 0 and json_end > json_start:
json_str = content[json_start:json_end]
# Parse the JSON string
parsed_data = json.loads(json_str)
# Convert to Pydantic model
answer_outline = AnswerOutline(**parsed_data)
total_questions = sum(
len(section.questions) for section in answer_outline.answer_outline
)
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
f"✅ Successfully generated outline with {len(answer_outline.answer_outline)} sections and {total_questions} research questions!"
)
}
)
print(
f"Successfully generated answer outline with {len(answer_outline.answer_outline)} sections"
)
# Return state update
return {"answer_outline": answer_outline}
else:
# If JSON structure not found, raise a clear error
error_message = (
f"Could not find valid JSON in LLM response. Raw response: {content}"
)
writer({"yield_value": streaming_service.format_error(error_message)})
raise ValueError(error_message)
except (json.JSONDecodeError, ValueError) as e:
# Log the error and re-raise it
error_message = f"Error parsing LLM response: {e!s}"
writer({"yield_value": streaming_service.format_error(error_message)})
print(f"Error parsing LLM response: {e!s}")
print(f"Raw response: {response.content}")
raise
async def fetch_relevant_documents(
research_questions: list[str],
user_id: str,
@ -1453,439 +1297,6 @@ async def fetch_relevant_documents(
return deduplicated_docs
async def process_sections(
state: State, config: RunnableConfig, writer: StreamWriter
) -> dict[str, Any]:
"""
Process all sections in parallel and combine the results.
This node takes the answer outline from the previous step, fetches relevant documents
for all questions across all sections once, and then processes each section in parallel
using the sub_section_writer graph with the shared document pool.
Returns:
Dict containing the final written report in the "final_written_report" key.
"""
# Get configuration and answer outline from state
configuration = Configuration.from_runnable_config(config)
answer_outline = state.answer_outline
streaming_service = state.streaming_service
# Initialize a dictionary to track content for all sections
# This is used to maintain section content while streaming multiple sections
section_contents = {}
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
"🚀 Starting to process research sections..."
)
}
)
print(f"Processing sections from outline: {answer_outline is not None}")
if not answer_outline:
error_message = "No answer outline was provided. Cannot generate report."
writer({"yield_value": streaming_service.format_error(error_message)})
return {
"final_written_report": "No answer outline was provided. Cannot generate final report."
}
# Collect all questions from all sections
all_questions = []
for section in answer_outline.answer_outline:
all_questions.extend(section.questions)
print(f"Collected {len(all_questions)} questions from all sections")
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
f"🧩 Found {len(all_questions)} research questions across {len(answer_outline.answer_outline)} sections"
)
}
)
# Fetch relevant documents once for all questions
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
"🔍 Searching for relevant information across all connectors..."
)
}
)
if configuration.num_sections == 1:
top_k = 10
elif configuration.num_sections == 3:
top_k = 20
elif configuration.num_sections == 6:
top_k = 30
else:
top_k = 10
relevant_documents = []
user_selected_documents = []
user_selected_sources = []
try:
# First, fetch user-selected documents if any
if configuration.document_ids_to_add_in_context:
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
f"📋 Including {len(configuration.document_ids_to_add_in_context)} user-selected documents..."
)
}
)
(
user_selected_sources,
user_selected_documents,
) = await fetch_documents_by_ids(
document_ids=configuration.document_ids_to_add_in_context,
user_id=configuration.user_id,
db_session=state.db_session,
)
if user_selected_documents:
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
f"✅ Successfully added {len(user_selected_documents)} user-selected documents to context"
)
}
)
# Create connector service using state db_session
connector_service = ConnectorService(
state.db_session, user_id=configuration.user_id
)
await connector_service.initialize_counter()
relevant_documents = await fetch_relevant_documents(
research_questions=all_questions,
user_id=configuration.user_id,
search_space_id=configuration.search_space_id,
db_session=state.db_session,
connectors_to_search=configuration.connectors_to_search,
writer=writer,
state=state,
top_k=top_k,
connector_service=connector_service,
search_mode=configuration.search_mode,
user_selected_sources=user_selected_sources,
)
except Exception as e:
error_message = f"Error fetching relevant documents: {e!s}"
print(error_message)
writer({"yield_value": streaming_service.format_error(error_message)})
# Log the error and continue with an empty list of documents
# This allows the process to continue, but the report might lack information
relevant_documents = []
# Combine user-selected documents with connector-fetched documents
all_documents = user_selected_documents + relevant_documents
print(f"Fetched {len(relevant_documents)} relevant documents for all sections")
print(
f"Added {len(user_selected_documents)} user-selected documents for all sections"
)
print(f"Total documents for sections: {len(all_documents)}")
# Extract and stream sources from all_documents
if all_documents:
sources_to_stream = extract_sources_from_documents(all_documents)
writer(
{"yield_value": streaming_service.format_sources_delta(sources_to_stream)}
)
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
f"✨ Starting to draft {len(answer_outline.answer_outline)} sections using {len(all_documents)} total document chunks ({len(user_selected_documents)} user-selected + {len(relevant_documents)} connector-found)"
)
}
)
# Create tasks to process each section in parallel with the same document set
section_tasks = []
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
"⚙️ Creating processing tasks for each section..."
)
}
)
for i, section in enumerate(answer_outline.answer_outline):
if i == 0:
sub_section_type = SubSectionType.START
elif i == len(answer_outline.answer_outline) - 1:
sub_section_type = SubSectionType.END
else:
sub_section_type = SubSectionType.MIDDLE
# Initialize the section_contents entry for this section
section_contents[i] = {
"title": section.section_title,
"content": "",
"index": i,
}
section_tasks.append(
process_section_with_documents(
section_id=i,
section_title=section.section_title,
section_questions=section.questions,
user_query=configuration.user_query,
user_id=configuration.user_id,
search_space_id=configuration.search_space_id,
relevant_documents=all_documents, # Use combined documents
state=state,
writer=writer,
sub_section_type=sub_section_type,
section_contents=section_contents,
)
)
# Run all section processing tasks in parallel
print(f"Running {len(section_tasks)} section processing tasks in parallel")
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
f"⏳ Processing {len(section_tasks)} sections simultaneously..."
)
}
)
section_results = await asyncio.gather(*section_tasks, return_exceptions=True)
# Handle any exceptions in the results
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
"🧵 Combining section results into final report..."
)
}
)
processed_results = []
for i, result in enumerate(section_results):
if isinstance(result, Exception):
section_title = answer_outline.answer_outline[i].section_title
error_message = f"Error processing section '{section_title}': {result!s}"
print(error_message)
writer({"yield_value": streaming_service.format_error(error_message)})
processed_results.append(error_message)
else:
processed_results.append(result)
# Combine the results into a final report with section titles
final_report = []
for _i, (section, content) in enumerate(
zip(answer_outline.answer_outline, processed_results, strict=False)
):
# Skip adding the section header since the content already contains the title
final_report.append(content)
final_report.append("\n")
# Stream each section with its title
writer(
{
"yield_value": state.streaming_service.format_text_chunk(
f"# {section.section_title}\n\n{content}"
)
}
)
# Join all sections with newlines
final_written_report = "\n".join(final_report)
print(f"Generated final report with {len(final_report)} parts")
writer(
{
"yield_value": streaming_service.format_terminal_info_delta(
"🎉 Final research report generated successfully!"
)
}
)
# Use the shared documents for further question generation
# Since all sections used the same document pool, we can use it directly
return {
"final_written_report": final_written_report,
"reranked_documents": all_documents,
}
async def process_section_with_documents(
section_id: int,
section_title: str,
section_questions: list[str],
user_id: str,
search_space_id: int,
relevant_documents: list[dict[str, Any]],
user_query: str,
state: State = None,
writer: StreamWriter = None,
sub_section_type: SubSectionType = SubSectionType.MIDDLE,
section_contents: dict[int, dict[str, Any]] | None = None,
) -> str:
"""
Process a single section using pre-fetched documents.
Args:
section_id: The ID of the section
section_title: The title of the section
section_questions: List of research questions for this section
user_id: The user ID
search_space_id: The search space ID
relevant_documents: Pre-fetched documents to use for this section
state: The current state
writer: StreamWriter for sending progress updates
sub_section_type: The type of section (start, middle, end)
section_contents: Dictionary to track content across multiple sections
Returns:
The written section content
"""
try:
# Use the provided documents
documents_to_use = relevant_documents
# Send status update via streaming if available
if state and state.streaming_service and writer:
writer(
{
"yield_value": state.streaming_service.format_terminal_info_delta(
f'📝 Writing section: "{section_title}" with {len(section_questions)} research questions'
)
}
)
# Fallback if no documents found
if not documents_to_use:
print(f"No relevant documents found for section: {section_title}")
if state and state.streaming_service and writer:
writer(
{
"yield_value": state.streaming_service.format_terminal_info_delta(
f'📝 Writing section "{section_title}" using general knowledge (no specific sources found)'
)
}
)
documents_to_use = [
{"content": f"No specific information was found for: {question}"}
for question in section_questions
]
# Call the sub_section_writer graph with the appropriate config
config = {
"configurable": {
"sub_section_title": section_title,
"sub_section_questions": section_questions,
"sub_section_type": sub_section_type,
"user_query": user_query,
"relevant_documents": documents_to_use,
"user_id": user_id,
"search_space_id": search_space_id,
}
}
# Create the initial state with db_session and chat_history
sub_state = {"db_session": state.db_session, "chat_history": state.chat_history}
# Invoke the sub-section writer graph with streaming
print(f"Invoking sub_section_writer for: {section_title}")
if state and state.streaming_service and writer:
writer(
{
"yield_value": state.streaming_service.format_terminal_info_delta(
f'🧠 Analyzing information and drafting content for section: "{section_title}"'
)
}
)
# Variables to track streaming state
complete_content = "" # Tracks the complete content received so far
async for _chunk_type, chunk in sub_section_writer_graph.astream(
sub_state, config, stream_mode=["values"]
):
if "final_answer" in chunk:
new_content = chunk["final_answer"]
if new_content and new_content != complete_content:
# Extract only the new content (delta)
delta = new_content[len(complete_content) :]
# Update what we've processed so far
complete_content = new_content
# Only stream if there's actual new content
if delta and state and state.streaming_service and writer:
# Update terminal with real-time progress indicator
writer(
{
"yield_value": state.streaming_service.format_terminal_info_delta(
f"✍️ Writing section {section_id + 1}... ({len(complete_content.split())} words)"
)
}
)
# Update section_contents with just the new delta
section_contents[section_id]["content"] += delta
# Build UI-friendly content for all sections
complete_answer = []
for i in range(len(section_contents)):
if i in section_contents and section_contents[i]["content"]:
# Add section header
complete_answer.append(
f"# {section_contents[i]['title']}"
)
complete_answer.append("") # Empty line after title
# Add section content
content_lines = section_contents[i]["content"].split(
"\n"
)
complete_answer.extend(content_lines)
complete_answer.append("") # Empty line after content
# Set default if no content was received
if not complete_content:
complete_content = "No content was generated for this section."
section_contents[section_id]["content"] = complete_content
# Final terminal update
if state and state.streaming_service and writer:
writer(
{
"yield_value": state.streaming_service.format_terminal_info_delta(
f'✅ Completed section: "{section_title}"'
)
}
)
return complete_content
except Exception as e:
print(f"Error processing section '{section_title}': {e!s}")
# Send error update via streaming if available
if state and state.streaming_service and writer:
writer(
{
"yield_value": state.streaming_service.format_error(
f'Error processing section "{section_title}": {e!s}'
)
}
)
return f"Error processing section: {section_title}. Details: {e!s}"
async def reformulate_user_query(
state: State, config: RunnableConfig, writer: StreamWriter
) -> dict[str, Any]:
@ -2133,7 +1544,7 @@ async def generate_further_questions(
"""
Generate contextually relevant follow-up questions based on chat history and available documents.
This node takes the chat history and reranked documents from sub-agents (qna_agent or sub_section_writer)
This node takes the chat history and reranked documents from the QNA agent
and uses an LLM to generate follow-up questions that would naturally extend the conversation
and provide additional value to the user.

95
surfsense_backend/app/agents/researcher/prompts.py
View file

@ -2,105 +2,12 @@ import datetime
def _build_language_instruction(language: str | None = None):
"""Build language instruction for prompts."""
if language:
return f"\n\nIMPORTANT: Please respond in {language} language. All your responses, explanations, and analysis should be written in {language}."
return ""
def get_answer_outline_system_prompt(language: str | None = None) -> str:
language_instruction = _build_language_instruction(language)
return f"""
Today's date: {datetime.datetime.now().strftime("%Y-%m-%d")}
{language_instruction}
<answer_outline_system>
You are an expert research assistant specializing in structuring information. Your task is to create a detailed and logical research outline based on the user's query. This outline will serve as the blueprint for generating a comprehensive research report.
<input>
- user_query (string): The main question or topic the user wants researched. This guides the entire outline creation process.
- num_sections (integer): The target number of distinct sections the final research report should have. This helps control the granularity and structure of the outline.
</input>
<output_format>
A JSON object with the following structure:
{{
"answer_outline": [
{{
"section_id": 0,
"section_title": "Section Title",
"questions": [
"Question 1 to research for this section",
"Question 2 to research for this section"
]
}}
]
}}
</output_format>
<instructions>
1. **Deconstruct the `user_query`:** Identify the key concepts, entities, and the core information requested by the user.
2. **Determine Section Themes:** Based on the analysis and the requested `num_sections`, divide the topic into distinct, logical themes or sub-topics. Each theme will become a section. Ensure these themes collectively address the `user_query` comprehensively.
3. **Develop Sections:** For *each* of the `num_sections`:
* **Assign `section_id`:** Start with 0 and increment sequentially for each section.
* **Craft `section_title`:** Write a concise, descriptive title that clearly defines the scope and focus of the section's theme.
* **Formulate Research `questions`:** Generate 2 to 5 specific, targeted research questions for this section. These questions must:
* Directly relate to the `section_title` and explore its key aspects.
* Be answerable through focused research (e.g., searching documents, databases, or knowledge bases).
* Be distinct from each other and from questions in other sections. Avoid redundancy.
* Collectively guide the gathering of information needed to fully address the section's theme.
4. **Ensure Logical Flow:** Arrange the sections in a coherent and intuitive sequence. Consider structures like:
* General background -> Specific details -> Analysis/Comparison -> Applications/Implications
* Problem definition -> Proposed solutions -> Evaluation -> Conclusion
* Chronological progression
5. **Verify Completeness and Cohesion:** Review the entire outline (`section_titles` and `questions`) to confirm that:
* All sections together provide a complete and well-structured answer to the original `user_query`.
* There are no significant overlaps or gaps in coverage between sections.
6. **Adhere Strictly to Output Format:** Ensure the final output is a valid JSON object matching the specified structure exactly, including correct field names (`answer_outline`, `section_id`, `section_title`, `questions`) and data types.
</instructions>
<examples>
User Query: "What are the health benefits of meditation?"
Number of Sections: 3
{{
"answer_outline": [
{{
"section_id": 0,
"section_title": "Physical Health Benefits of Meditation",
"questions": [
"What physiological changes occur in the body during meditation?",
"How does regular meditation affect blood pressure and heart health?",
"What impact does meditation have on inflammation and immune function?",
"Can meditation help with pain management, and if so, how?"
]
}},
{{
"section_id": 1,
"section_title": "Mental Health Benefits of Meditation",
"questions": [
"How does meditation affect stress and anxiety levels?",
"What changes in brain structure or function have been observed in meditation practitioners?",
"Can meditation help with depression and mood disorders?",
"What is the relationship between meditation and cognitive function?"
]
}},
{{
"section_id": 2,
"section_title": "Best Meditation Practices for Maximum Benefits",
"questions": [
"What are the most effective meditation techniques for beginners?",
"How long and how frequently should one meditate to see benefits?",
"Are there specific meditation approaches best suited for particular health goals?",
"What common obstacles prevent people from experiencing meditation benefits?"
]
}}
]
}}
</examples>
</answer_outline_system>
"""
def get_further_questions_system_prompt():
return f"""
Today's date: {datetime.datetime.now().strftime("%Y-%m-%d")}

2
surfsense_backend/app/agents/researcher/state.py
View file

@ -28,8 +28,6 @@ class State:
chat_history: list[Any] | None = field(default_factory=list)
reformulated_query: str | None = field(default=None)
# Using field to explicitly mark as part of state
answer_outline: Any | None = field(default=None)
further_questions: Any | None = field(default=None)
# Temporary field to hold reranked documents from sub-agents for further question generation

8
surfsense_backend/app/agents/researcher/sub_section_writer/__init__.py
View file

@ -1,8 +0,0 @@
"""New LangGraph Agent.
This module defines a custom graph.
"""
from .graph import graph
__all__ = ["graph"]

40
surfsense_backend/app/agents/researcher/sub_section_writer/configuration.py
View file

@ -1,40 +0,0 @@
"""Define the configurable parameters for the agent."""
from __future__ import annotations
from dataclasses import dataclass, fields
from enum import Enum
from typing import Any
from langchain_core.runnables import RunnableConfig
class SubSectionType(Enum):
"""Enum defining the type of sub-section."""
START = "START"
MIDDLE = "MIDDLE"
END = "END"
@dataclass(kw_only=True)
class Configuration:
"""The configuration for the agent."""
# Input parameters provided at invocation
sub_section_title: str
sub_section_questions: list[str]
sub_section_type: SubSectionType
user_query: str
relevant_documents: list[Any] # Documents provided directly to the agent
user_id: str
search_space_id: int
@classmethod
def from_runnable_config(
cls, config: RunnableConfig | None = None
) -> Configuration:
"""Create a Configuration instance from a RunnableConfig object."""
configurable = (config.get("configurable") or {}) if config else {}
_fields = {f.name for f in fields(cls) if f.init}
return cls(**{k: v for k, v in configurable.items() if k in _fields})

21
surfsense_backend/app/agents/researcher/sub_section_writer/graph.py
View file

@ -1,21 +0,0 @@
from langgraph.graph import StateGraph
from .configuration import Configuration
from .nodes import rerank_documents, write_sub_section
from .state import State
# Define a new graph
workflow = StateGraph(State, config_schema=Configuration)
# Add the nodes to the graph
workflow.add_node("rerank_documents", rerank_documents)
workflow.add_node("write_sub_section", write_sub_section)
# Connect the nodes
workflow.add_edge("__start__", "rerank_documents")
workflow.add_edge("rerank_documents", "write_sub_section")
workflow.add_edge("write_sub_section", "__end__")
# Compile the workflow into an executable graph
graph = workflow.compile()
graph.name = "Sub Section Writer" # This defines the custom name in LangSmith

241
surfsense_backend/app/agents/researcher/sub_section_writer/nodes.py
View file

@ -1,241 +0,0 @@
from typing import Any
from langchain_core.messages import HumanMessage, SystemMessage
from langchain_core.runnables import RunnableConfig
from app.services.reranker_service import RerankerService
from ..utils import (
calculate_token_count,
format_documents_section,
langchain_chat_history_to_str,
optimize_documents_for_token_limit,
)
from .configuration import Configuration, SubSectionType
from .prompts import get_citation_system_prompt, get_no_documents_system_prompt
from .state import State
async def rerank_documents(state: State, config: RunnableConfig) -> dict[str, Any]:
"""
Rerank the documents based on relevance to the sub-section title.
This node takes the relevant documents provided in the configuration,
reranks them using the reranker service based on the sub-section title,
and updates the state with the reranked documents.
Returns:
Dict containing the reranked documents.
"""
# Get configuration and relevant documents
configuration = Configuration.from_runnable_config(config)
documents = configuration.relevant_documents
sub_section_questions = configuration.sub_section_questions
# If no documents were provided, return empty list
if not documents or len(documents) == 0:
return {"reranked_documents": []}
# Get reranker service from app config
reranker_service = RerankerService.get_reranker_instance()
# Use documents as is if no reranker service is available
reranked_docs = documents
if reranker_service:
try:
# Use the sub-section questions for reranking context
# rerank_query = "\n".join(sub_section_questions)
# rerank_query = configuration.user_query
rerank_query = (
configuration.user_query + "\n" + "\n".join(sub_section_questions)
)
# Convert documents to format expected by reranker if needed
reranker_input_docs = [
{
"chunk_id": doc.get("chunk_id", f"chunk_{i}"),
"content": doc.get("content", ""),
"score": doc.get("score", 0.0),
"document": {
"id": doc.get("document", {}).get("id", ""),
"title": doc.get("document", {}).get("title", ""),
"document_type": doc.get("document", {}).get(
"document_type", ""
),
"metadata": doc.get("document", {}).get("metadata", {}),
},
}
for i, doc in enumerate(documents)
]
# Rerank documents using the section title
reranked_docs = reranker_service.rerank_documents(
rerank_query, reranker_input_docs
)
# Sort by score in descending order
reranked_docs.sort(key=lambda x: x.get("score", 0), reverse=True)
print(
f"Reranked {len(reranked_docs)} documents for section: {configuration.sub_section_title}"
)
except Exception as e:
print(f"Error during reranking: {e!s}")
# Use original docs if reranking fails
return {"reranked_documents": reranked_docs}
async def write_sub_section(state: State, config: RunnableConfig) -> dict[str, Any]:
"""
Write the sub-section using the provided documents.
This node takes the relevant documents provided in the configuration and uses
an LLM to generate a comprehensive answer to the sub-section title with
proper citations. The citations follow [citation:source_id] format using source IDs from the
documents. If no documents are provided, it will use chat history to generate
content.
Returns:
Dict containing the final answer in the "final_answer" key.
"""
from app.services.llm_service import get_user_fast_llm
# Get configuration and relevant documents from configuration
configuration = Configuration.from_runnable_config(config)
documents = state.reranked_documents
user_id = configuration.user_id
search_space_id = configuration.search_space_id
# Get user's fast LLM
llm = await get_user_fast_llm(state.db_session, user_id, search_space_id)
if not llm:
error_message = f"No fast LLM configured for user {user_id} in search space {search_space_id}"
print(error_message)
raise RuntimeError(error_message)
# Extract configuration data
section_title = configuration.sub_section_title
sub_section_questions = configuration.sub_section_questions
user_query = configuration.user_query
sub_section_type = configuration.sub_section_type
# Format the questions as bullet points for clarity
questions_text = "\n".join([f"- {question}" for question in sub_section_questions])
# Provide context based on the subsection type
section_position_context_map = {
SubSectionType.START: "This is the INTRODUCTION section.",
SubSectionType.MIDDLE: "This is a MIDDLE section. Ensure this content flows naturally from previous sections and into subsequent ones. This could be any middle section in the document, so maintain coherence with the overall structure while addressing the specific topic of this section. Do not provide any conclusions in this section, as conclusions should only appear in the final section.",
SubSectionType.END: "This is the CONCLUSION section. Focus on summarizing key points, providing closure.",
}
section_position_context = section_position_context_map.get(sub_section_type, "")
# Determine if we have documents and optimize for token limits
has_documents_initially = documents and len(documents) > 0
chat_history_str = langchain_chat_history_to_str(state.chat_history)
if has_documents_initially:
# Create base message template for token calculation (without documents)
base_human_message_template = f"""
Now user's query is:
<user_query>
{user_query}
</user_query>
The sub-section title is:
<sub_section_title>
{section_title}
</sub_section_title>
<section_position>
{section_position_context}
</section_position>
<guiding_questions>
{questions_text}
</guiding_questions>
Please write content for this sub-section using the provided source material and cite all information appropriately.
"""
# Use initial system prompt for token calculation
initial_system_prompt = get_citation_system_prompt(chat_history_str)
base_messages = [
SystemMessage(content=initial_system_prompt),
HumanMessage(content=base_human_message_template),
]
# Optimize documents to fit within token limits
optimized_documents, has_optimized_documents = (
optimize_documents_for_token_limit(documents, base_messages, llm.model)
)
# Update state based on optimization result
documents = optimized_documents
has_documents = has_optimized_documents
else:
has_documents = False
# Choose system prompt based on final document availability
system_prompt = (
get_citation_system_prompt(chat_history_str)
if has_documents
else get_no_documents_system_prompt(chat_history_str)
)
# Generate documents section
documents_text = (
format_documents_section(documents, "Source material") if has_documents else ""
)
# Create final human message content
instruction_text = (
"Please write content for this sub-section using the provided source material and cite all information appropriately."
if has_documents
else "Please write content for this sub-section based on our conversation history and your general knowledge."
)
human_message_content = f"""
{documents_text}
Now user's query is:
<user_query>
{user_query}
</user_query>
The sub-section title is:
<sub_section_title>
{section_title}
</sub_section_title>
<section_position>
{section_position_context}
</section_position>
<guiding_questions>
{questions_text}
</guiding_questions>
{instruction_text}
"""
# Create final messages for the LLM
messages_with_chat_history = [
SystemMessage(content=system_prompt),
HumanMessage(content=human_message_content),
]
# Log final token count
total_tokens = calculate_token_count(messages_with_chat_history, llm.model)
print(f"Final token count: {total_tokens}")
# Call the LLM and get the response
response = await llm.ainvoke(messages_with_chat_history)
final_answer = response.content
return {"final_answer": final_answer}

239
surfsense_backend/app/agents/researcher/sub_section_writer/prompts.py
View file

@ -1,239 +0,0 @@
import datetime
from ..prompts import _build_language_instruction
def get_citation_system_prompt(
chat_history: str | None = None, language: str | None = None
):
chat_history_section = (
f"""
<chat_history>
{chat_history if chat_history else "NO CHAT HISTORY PROVIDED"}
</chat_history>
"""
if chat_history is not None
else """
<chat_history>
NO CHAT HISTORY PROVIDED
</chat_history>
"""
)
# Add language instruction if specified
language_instruction = _build_language_instruction(language)
return f"""
Today's date: {datetime.datetime.now().strftime("%Y-%m-%d")}
You are SurfSense, an advanced AI research assistant that synthesizes information from multiple knowledge sources to provide comprehensive, well-cited answers to user queries.{language_instruction}
{chat_history_section}
<knowledge_sources>
- EXTENSION: "Web content saved via SurfSense browser extension" (personal browsing history)
- CRAWLED_URL: "Webpages indexed by SurfSense web crawler" (personally selected websites)
- FILE: "User-uploaded documents (PDFs, Word, etc.)" (personal files)
- SLACK_CONNECTOR: "Slack conversations and shared content" (personal workspace communications)
- NOTION_CONNECTOR: "Notion workspace pages and databases" (personal knowledge management)
- YOUTUBE_VIDEO: "YouTube video transcripts and metadata" (personally saved videos)
- GITHUB_CONNECTOR: "GitHub repository content and issues" (personal repositories and interactions)
- ELASTICSEARCH_CONNECTOR: "Elasticsearch documents and indices (indexed content from your ES connector)" (personal search index)
- LINEAR_CONNECTOR: "Linear project issues and discussions" (personal project management)
- JIRA_CONNECTOR: "Jira project issues, tickets, and comments" (personal project tracking)
- CONFLUENCE_CONNECTOR: "Confluence pages and comments" (personal project documentation)
- CLICKUP_CONNECTOR: "ClickUp tasks and project data" (personal task management)
- GOOGLE_CALENDAR_CONNECTOR: "Google Calendar events, meetings, and schedules" (personal calendar and time management)
- GOOGLE_GMAIL_CONNECTOR: "Google Gmail emails and conversations" (personal emails and communications)
- DISCORD_CONNECTOR: "Discord server messages and channels" (personal community interactions)
- AIRTABLE_CONNECTOR: "Airtable records, tables, and database content" (personal data management and organization)
- TAVILY_API: "Tavily search API results" (personalized search results)
- LINKUP_API: "Linkup search API results" (personalized search results)
- LUMA_CONNECTOR: "Luma events"
</knowledge_sources>
<instructions>
1. Review the chat history to understand the conversation context and any previous topics discussed.
2. Carefully analyze all provided documents in the <document> section's.
3. Extract relevant information that addresses the user's query.
4. Synthesize a comprehensive, personalized answer using information from the user's personal knowledge sources.
5. For EVERY piece of information you include from the documents, add a citation in the format [citation:knowledge_source_id] where knowledge_source_id is the source_id from the document's metadata.
6. Make sure ALL factual statements from the documents have proper citations.
7. If multiple documents support the same point, include all relevant citations [citation:source_id1], [citation:source_id2].
8. Present information in a logical, coherent flow that reflects the user's personal context.
9. Use your own words to connect ideas, but cite ALL information from the documents.
10. If documents contain conflicting information, acknowledge this and present both perspectives with appropriate citations.
11. Do not make up or include information not found in the provided documents.
12. Use the chat history to maintain conversation continuity and refer to previous discussions when relevant.
13. CRITICAL: You MUST use the exact source_id value from each document's metadata for citations. Do not create your own citation numbers.
14. CRITICAL: Every citation MUST be in the format [citation:knowledge_source_id] where knowledge_source_id is the exact source_id value.
15. CRITICAL: Never modify or change the source_id - always use the original values exactly as provided in the metadata.
16. CRITICAL: Do not return citations as clickable links.
17. CRITICAL: Never format citations as markdown links like "([citation:5](https://example.com))". Always use plain square brackets only.
18. CRITICAL: Citations must ONLY appear as [citation:source_id] or [citation:source_id1], [citation:source_id2] format - never with parentheses, hyperlinks, or other formatting.
19. CRITICAL: Never make up source IDs. Only use source_id values that are explicitly provided in the document metadata.
20. CRITICAL: If you are unsure about a source_id, do not include a citation rather than guessing or making one up.
21. CRITICAL: Focus only on answering the user's query. Any guiding questions provided are for your thinking process only and should not be mentioned in your response.
22. CRITICAL: Ensure your response aligns with the provided sub-section title and section position.
23. CRITICAL: Remember that all knowledge sources contain personal information - provide answers that reflect this personal context.
</instructions>
<format>
- Write in clear, professional language suitable for academic or technical audiences
- Tailor your response to the user's personal context based on their knowledge sources
- Organize your response with appropriate paragraphs, headings, and structure
- Every fact from the documents must have a citation in the format [citation:knowledge_source_id] where knowledge_source_id is the EXACT source_id from the document's metadata
- Citations should appear at the end of the sentence containing the information they support
- Multiple citations should be separated by commas: [citation:source_id1], [citation:source_id2], [citation:source_id3]
- No need to return references section. Just citations in answer.
- NEVER create your own citation format - use the exact source_id values from the documents in the [citation:source_id] format.
- NEVER format citations as clickable links or as markdown links like "([citation:5](https://example.com))". Always use plain square brackets only.
- NEVER make up source IDs if you are unsure about the source_id. It is better to omit the citation than to guess.
- NEVER include or mention the guiding questions in your response. They are only to help guide your thinking.
- ALWAYS focus on answering the user's query directly from the information in the documents.
- ALWAYS provide personalized answers that reflect the user's own knowledge and context.
</format>
<input_example>
<documents>
<document>
<metadata>
<source_id>1</source_id>
<source_type>EXTENSION</source_type>
</metadata>
<content>
The Great Barrier Reef is the world's largest coral reef system, stretching over 2,300 kilometers along the coast of Queensland, Australia. It comprises over 2,900 individual reefs and 900 islands.
</content>
</document>
<document>
<metadata>
<source_id>13</source_id>
<source_type>YOUTUBE_VIDEO</source_type>
</metadata>
<content>
Climate change poses a significant threat to coral reefs worldwide. Rising ocean temperatures have led to mass coral bleaching events in the Great Barrier Reef in 2016, 2017, and 2020.
</content>
</document>
<document>
<metadata>
<source_id>21</source_id>
<source_type>CRAWLED_URL</source_type>
</metadata>
<content>
The Great Barrier Reef was designated a UNESCO World Heritage Site in 1981 due to its outstanding universal value and biological diversity. It is home to over 1,500 species of fish and 400 types of coral.
</content>
</document>
</documents>
</input_example>
<output_example>
Based on your saved browser content and videos, the Great Barrier Reef is the world's largest coral reef system, stretching over 2,300 kilometers along the coast of Queensland, Australia [citation:1]. From your browsing history, you've looked into its designation as a UNESCO World Heritage Site in 1981 due to its outstanding universal value and biological diversity [citation:21]. The reef is home to over 1,500 species of fish and 400 types of coral [citation:21]. According to a YouTube video you've watched, climate change poses a significant threat to coral reefs worldwide, with rising ocean temperatures leading to mass coral bleaching events in the Great Barrier Reef in 2016, 2017, and 2020 [citation:13]. The reef system comprises over 2,900 individual reefs and 900 islands [citation:1], making it an ecological treasure that requires protection from multiple threats [citation:1], [citation:13].
</output_example>
<incorrect_citation_formats>
DO NOT use any of these incorrect citation formats:
- Using parentheses and markdown links: ([citation:1](https://github.com/MODSetter/SurfSense))
- Using parentheses around brackets: ([citation:1])
- Using hyperlinked text: [link to source 1](https://example.com)
- Using footnote style: ... reef system¹
- Making up source IDs when source_id is unknown
- Using old IEEE format: [1], [2], [3]
- Using source types instead of IDs: [citation:EXTENSION] instead of [citation:1]
</incorrect_citation_formats>
ONLY use the format [citation:source_id] or multiple citations [citation:source_id1], [citation:source_id2], [citation:source_id3]
Note that the citations use the exact source_id values (1, 13, and 21) from the document metadata. Citations appear at the end of sentences and maintain the new citation format.
<user_query_instructions>
When you see a user query like:
<user_query>
Give all linear issues.
</user_query>
Focus exclusively on answering this query using information from the provided documents, which contain the user's personal knowledge and data.
If guiding questions are provided in a <guiding_questions> section, use them only to guide your thinking process. Do not mention or list these questions in your response.
Make sure your response:
1. Considers the chat history for context and conversation continuity
2. Directly answers the user's query with personalized information from their own knowledge sources
3. Fits the provided sub-section title and section position
4. Uses proper citations for all information from documents
5. Is well-structured and professional in tone
6. Acknowledges the personal nature of the information being provided
</user_query_instructions>
"""
def get_no_documents_system_prompt(
chat_history: str | None = None, language: str | None = None
):
chat_history_section = (
f"""
<chat_history>
{chat_history if chat_history else "NO CHAT HISTORY PROVIDED"}
</chat_history>
"""
if chat_history is not None
else """
<chat_history>
NO CHAT HISTORY PROVIDED
</chat_history>
"""
)
# Add language instruction if specified
language_instruction = _build_language_instruction(language)
return f"""
Today's date: {datetime.datetime.now().strftime("%Y-%m-%d")}
You are SurfSense, an advanced AI research assistant that helps users create well-structured content for their documents and research.{language_instruction}
{chat_history_section}
<context>
You are writing content for a specific sub-section of a document. No specific documents from the user's personal knowledge base are available, so you should create content based on:
1. The conversation history and context
2. Your general knowledge and expertise
3. The specific sub-section requirements provided
4. Understanding of the user's needs based on our conversation
</context>
<instructions>
1. Write comprehensive, well-structured content for the specified sub-section
2. Draw upon the conversation history to understand the user's context and needs
3. Use your general knowledge to provide accurate, detailed information
4. Ensure the content fits the sub-section title and position in the document
5. Follow the section positioning guidelines (introduction, middle, or conclusion)
6. Structure the content logically with appropriate flow and transitions
7. Write in a professional, academic tone suitable for research documents
8. Acknowledge when you're drawing from general knowledge rather than personal sources
9. If the content would benefit from personalized information, gently mention that adding relevant sources to SurfSense could enhance the content
10. Ensure the content addresses the guiding questions without explicitly mentioning them
11. Create content that flows naturally and maintains coherence with the overall document structure
</instructions>
<format>
- Write in clear, professional language suitable for academic or research documents
- Organize content with appropriate paragraphs and logical structure
- No citations are needed since you're using general knowledge
- Follow the specified section type (START/MIDDLE/END) guidelines
- Ensure content flows naturally and maintains document coherence
- Be comprehensive and detailed while staying focused on the sub-section topic
- When appropriate, mention that adding relevant sources to SurfSense could provide more personalized and cited content
</format>
<section_guidelines>
- START (Introduction): Provide context, background, and introduce key concepts
- MIDDLE: Develop main points, provide detailed analysis, ensure smooth transitions
- END (Conclusion): Summarize key points, provide closure, synthesize main insights
</section_guidelines>
<user_query_instructions>
When writing content for a sub-section without access to personal documents:
1. Review the chat history to understand conversation context and maintain continuity
2. Create the most comprehensive and useful content possible using general knowledge
3. Ensure the content fits the sub-section title and document position
4. Draw upon conversation history for context about the user's needs
5. Write in a professional, research-appropriate tone
6. Address the guiding questions through natural content flow without explicitly listing them
7. Suggest how adding relevant sources to SurfSense could enhance future content when appropriate
</user_query_instructions>
"""

26
surfsense_backend/app/agents/researcher/sub_section_writer/state.py
View file

@ -1,26 +0,0 @@
"""Define the state structures for the agent."""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
from sqlalchemy.ext.asyncio import AsyncSession
@dataclass
class State:
"""Defines the dynamic state for the agent during execution.
This state tracks the database session and the outputs generated by the agent's nodes.
See: https://langchain-ai.github.io/langgraph/concepts/low_level/#state
for more information.
"""
# Runtime context
db_session: AsyncSession
chat_history: list[Any] | None = field(default_factory=list)
# OUTPUT: Populated by agent nodes
reranked_documents: list[Any] | None = None
final_answer: str | None = None

19
surfsense_backend/app/agents/researcher/utils.py
View file

@ -3,25 +3,6 @@ from typing import Any, NamedTuple
from langchain.schema import AIMessage, HumanMessage, SystemMessage
from langchain_core.messages import BaseMessage
from litellm import get_model_info, token_counter
from pydantic import BaseModel, Field
class Section(BaseModel):
"""A section in the answer outline."""
section_id: int = Field(..., description="The zero-based index of the section")
section_title: str = Field(..., description="The title of the section")
questions: list[str] = Field(
..., description="Questions to research for this section"
)
class AnswerOutline(BaseModel):
"""The complete answer outline with all sections."""
answer_outline: list[Section] = Field(
..., description="List of sections in the answer outline"
)
class DocumentTokenInfo(NamedTuple):

3
surfsense_backend/app/db.py
View file

@ -76,9 +76,6 @@ class SearchSourceConnectorType(str, Enum):
class ChatType(str, Enum):
QNA = "QNA"
REPORT_GENERAL = "REPORT_GENERAL"
REPORT_DEEP = "REPORT_DEEP"
REPORT_DEEPER = "REPORT_DEEPER"
class LiteLLMProvider(str, Enum):

12
surfsense_backend/app/tasks/stream_connector_search_results.py
View file

@ -38,16 +38,6 @@ async def stream_connector_search_results(
"""
streaming_service = StreamingService()
if research_mode == "REPORT_GENERAL":
num_sections = 1
elif research_mode == "REPORT_DEEP":
num_sections = 3
elif research_mode == "REPORT_DEEPER":
num_sections = 6
else:
# Default fallback
num_sections = 1
# Convert UUID to string if needed
user_id_str = str(user_id) if isinstance(user_id, UUID) else user_id
@ -60,12 +50,10 @@ async def stream_connector_search_results(
config = {
"configurable": {
"user_query": user_query,
"num_sections": num_sections,
"connectors_to_search": selected_connectors,
"user_id": user_id_str,
"search_space_id": search_space_id,
"search_mode": search_mode,
"research_mode": research_mode,
"document_ids_to_add_in_context": document_ids_to_add_in_context,
"language": language, # Add language to the configuration
}

2
surfsense_backend/app/utils/validators.py
View file

@ -201,7 +201,7 @@ def validate_research_mode(research_mode: Any) -> str:
if not normalized_mode:
raise HTTPException(status_code=400, detail="research_mode cannot be empty")
valid_modes = ["REPORT_GENERAL", "REPORT_DEEP", "REPORT_DEEPER", "QNA"]
valid_modes = ["QNA"]
if normalized_mode not in valid_modes:
raise HTTPException(
status_code=400,

96
surfsense_web/README.md
View file

@ -1,96 +0,0 @@
# Next.js Token Handler Component
This project includes a reusable client component for Next.js that handles token storage from URL parameters.
## TokenHandler Component
The `TokenHandler` component is designed to:
1. Extract a token from URL parameters
2. Store the token in localStorage
3. Redirect the user to a specified path
### Usage
```tsx
import TokenHandler from '@/components/TokenHandler';
export default function AuthCallbackPage() {
return (
<div>
<h1>Authentication Callback</h1>
<TokenHandler
redirectPath="/dashboard"
tokenParamName="token"
storageKey="auth_token"
/>
</div>
);
}
```
### Props
The component accepts the following props:
- `redirectPath` (optional): Path to redirect after storing token (default: '/')
- `tokenParamName` (optional): Name of the URL parameter containing the token (default: 'token')
- `storageKey` (optional): Key to use when storing in localStorage (default: 'auth_token')
### Example URL
After authentication, redirect users to:
```
https://your-domain.com/auth/callback?token=your-auth-token
```
## Implementation Details
- Uses Next.js's `useSearchParams` hook to access URL parameters
- Uses `useRouter` for client-side navigation after token storage
- Includes error handling for localStorage operations
- Displays a loading message while processing
## Security Considerations
- This implementation assumes the token is passed securely
- Consider using HTTPS to prevent token interception
- For enhanced security, consider using HTTP-only cookies instead of localStorage
- The token in the URL might be visible in browser history and server logs
This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app).
## Getting Started
First, run the development server:
```bash
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
```
Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.
You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file.
This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel.
## Learn More
To learn more about Next.js, take a look at the following resources:
- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.
You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome!
## Deploy on Vercel
The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.
Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details.

14
surfsense_web/app/(home)/login/GoogleLoginButton.tsx
View file

@ -6,8 +6,8 @@ import { Logo } from "@/components/Logo";
import { AmbientBackground } from "./AmbientBackground";
export function GoogleLoginButton() {
const t = useTranslations('auth');
const t = useTranslations("auth");
const handleGoogleLogin = () => {
// Redirect to Google OAuth authorization URL
fetch(`${process.env.NEXT_PUBLIC_FASTAPI_BACKEND_URL}/auth/google/authorize`)
@ -34,7 +34,7 @@ export function GoogleLoginButton() {
<div className="mx-auto flex h-screen max-w-lg flex-col items-center justify-center">
<Logo className="rounded-md" />
<h1 className="my-8 text-xl font-bold text-neutral-800 dark:text-neutral-100 md:text-4xl">
{t('welcome_back')}
{t("welcome_back")}
</h1>
<motion.div
@ -68,14 +68,14 @@ export function GoogleLoginButton() {
</svg>
<div className="ml-1">
<p className="text-sm font-medium">
{t('cloud_dev_notice')}{" "}
{t("cloud_dev_notice")}{" "}
<a
href="/docs"
className="text-blue-600 underline dark:text-blue-400 hover:text-blue-800 dark:hover:text-blue-300"
>
{t('docs')}
{t("docs")}
</a>{" "}
{t('cloud_dev_self_hosted')}
{t("cloud_dev_self_hosted")}
</p>
</div>
</motion.div>
@ -94,7 +94,7 @@ export function GoogleLoginButton() {
<div className="absolute -bottom-px -right-px h-4 w-4 rounded-br-lg border-b-2 border-r-2 border-blue-500 bg-transparent transition-all duration-200 group-hover/btn:-bottom-2 group-hover/btn:-right-2"></div>
</div>
<IconBrandGoogleFilled className="h-5 w-5 text-neutral-700 dark:text-neutral-200" />
<span className="text-base font-medium">{t('continue_with_google')}</span>
<span className="text-base font-medium">{t("continue_with_google")}</span>
</motion.button>
</div>
</div>

142
surfsense_web/app/(home)/login/LocalLoginForm.tsx
View file

@ -3,14 +3,14 @@ import { Eye, EyeOff } from "lucide-react";
import { AnimatePresence, motion } from "motion/react";
import Link from "next/link";
import { useRouter } from "next/navigation";
import { useTranslations } from "next-intl";
import { useEffect, useState } from "react";
import { toast } from "sonner";
import { useTranslations } from "next-intl";
import { getAuthErrorDetails, isNetworkError, shouldRetry } from "@/lib/auth-errors";
export function LocalLoginForm() {
const t = useTranslations('auth');
const tCommon = useTranslations('common');
const t = useTranslations("auth");
const tCommon = useTranslations("common");
const [username, setUsername] = useState("");
const [password, setPassword] = useState("");
const [showPassword, setShowPassword] = useState(false);
@ -32,7 +32,7 @@ export function LocalLoginForm() {
setErrorTitle(null);
// Show loading toast
const loadingToast = toast.loading(tCommon('loading'));
const loadingToast = toast.loading(tCommon("loading"));
try {
// Create form data for the API request
@ -59,7 +59,7 @@ export function LocalLoginForm() {
}
// Success toast
toast.success(t('login_success'), {
toast.success(t("login_success"), {
id: loadingToast,
description: "Redirecting to dashboard...",
duration: 2000,
@ -170,84 +170,84 @@ export function LocalLoginForm() {
</div>
</motion.div>
)}
</AnimatePresence>
</AnimatePresence>
<div>
<label
htmlFor="email"
className="block text-sm font-medium text-gray-700 dark:text-gray-300"
>
{t('email')}
</label>
<input
id="email"
type="email"
required
value={username}
onChange={(e) => setUsername(e.target.value)}
className={`mt-1 block w-full rounded-md border px-3 py-2 shadow-sm focus:outline-none focus:ring-2 focus:ring-offset-2 dark:bg-gray-800 dark:text-white transition-colors ${
error
? "border-red-300 focus:border-red-500 focus:ring-red-500 dark:border-red-700"
: "border-gray-300 focus:border-blue-500 focus:ring-blue-500 dark:border-gray-700"
}`}
disabled={isLoading}
/>
</div>
<div>
<label
htmlFor="password"
className="block text-sm font-medium text-gray-700 dark:text-gray-300"
>
{t('password')}
</label>
<div className="relative">
<div>
<label
htmlFor="email"
className="block text-sm font-medium text-gray-700 dark:text-gray-300"
>
{t("email")}
</label>
<input
id="password"
type={showPassword ? "text" : "password"}
id="email"
type="email"
required
value={password}
onChange={(e) => setPassword(e.target.value)}
className={`mt-1 block w-full rounded-md border pr-10 px-3 py-2 shadow-sm focus:outline-none focus:ring-2 focus:ring-offset-2 dark:bg-gray-800 dark:text-white transition-colors ${
value={username}
onChange={(e) => setUsername(e.target.value)}
className={`mt-1 block w-full rounded-md border px-3 py-2 shadow-sm focus:outline-none focus:ring-2 focus:ring-offset-2 dark:bg-gray-800 dark:text-white transition-colors ${
error
? "border-red-300 focus:border-red-500 focus:ring-red-500 dark:border-red-700"
: "border-gray-300 focus:border-blue-500 focus:ring-blue-500 dark:border-gray-700"
}`}
disabled={isLoading}
/>
<button
type="button"
onClick={() => setShowPassword((prev) => !prev)}
className="absolute inset-y-0 right-0 flex items-center pr-3 mt-1 text-gray-500 hover:text-gray-700 dark:text-gray-400 dark:hover:text-gray-200"
aria-label={showPassword ? t('hide_password') : t('show_password')}
>
{showPassword ? <EyeOff className="h-4 w-4" /> : <Eye className="h-4 w-4" />}
</button>
</div>
</div>
<button
type="submit"
disabled={isLoading}
className="w-full rounded-md bg-blue-600 px-4 py-2 text-white shadow-sm hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 transition-colors"
>
{isLoading ? tCommon('loading') : t('sign_in')}
</button>
</form>
{authType === "LOCAL" && (
<div className="mt-4 text-center text-sm">
<p className="text-gray-600 dark:text-gray-400">
{t('dont_have_account')}{" "}
<Link
href="/register"
className="font-medium text-blue-600 hover:text-blue-500 dark:text-blue-400"
<div>
<label
htmlFor="password"
className="block text-sm font-medium text-gray-700 dark:text-gray-300"
>
{t('sign_up')}
</Link>
</p>
</div>
)}
{t("password")}
</label>
<div className="relative">
<input
id="password"
type={showPassword ? "text" : "password"}
required
value={password}
onChange={(e) => setPassword(e.target.value)}
className={`mt-1 block w-full rounded-md border pr-10 px-3 py-2 shadow-sm focus:outline-none focus:ring-2 focus:ring-offset-2 dark:bg-gray-800 dark:text-white transition-colors ${
error
? "border-red-300 focus:border-red-500 focus:ring-red-500 dark:border-red-700"
: "border-gray-300 focus:border-blue-500 focus:ring-blue-500 dark:border-gray-700"
}`}
disabled={isLoading}
/>
<button
type="button"
onClick={() => setShowPassword((prev) => !prev)}
className="absolute inset-y-0 right-0 flex items-center pr-3 mt-1 text-gray-500 hover:text-gray-700 dark:text-gray-400 dark:hover:text-gray-200"
aria-label={showPassword ? t("hide_password") : t("show_password")}
>
{showPassword ? <EyeOff className="h-4 w-4" /> : <Eye className="h-4 w-4" />}
</button>
</div>
</div>
<button
type="submit"
disabled={isLoading}
className="w-full rounded-md bg-blue-600 px-4 py-2 text-white shadow-sm hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 transition-colors"
>
{isLoading ? tCommon("loading") : t("sign_in")}
</button>
</form>
{authType === "LOCAL" && (
<div className="mt-4 text-center text-sm">
<p className="text-gray-600 dark:text-gray-400">
{t("dont_have_account")}{" "}
<Link
href="/register"
className="font-medium text-blue-600 hover:text-blue-500 dark:text-blue-400"
>
{t("sign_up")}
</Link>
</p>
</div>
)}
</div>
);
}

16
surfsense_web/app/(home)/login/page.tsx
View file

@ -3,9 +3,9 @@
import { Loader2 } from "lucide-react";
import { AnimatePresence, motion } from "motion/react";
import { useSearchParams } from "next/navigation";
import { useTranslations } from "next-intl";
import { Suspense, useEffect, useState } from "react";
import { toast } from "sonner";
import { useTranslations } from "next-intl";
import { Logo } from "@/components/Logo";
import { getAuthErrorDetails, shouldRetry } from "@/lib/auth-errors";
import { AmbientBackground } from "./AmbientBackground";
@ -13,8 +13,8 @@ import { GoogleLoginButton } from "./GoogleLoginButton";
import { LocalLoginForm } from "./LocalLoginForm";
function LoginContent() {
const t = useTranslations('auth');
const tCommon = useTranslations('common');
const t = useTranslations("auth");
const tCommon = useTranslations("common");
const [authType, setAuthType] = useState<string | null>(null);
const [isLoading, setIsLoading] = useState(true);
const [urlError, setUrlError] = useState<{ title: string; message: string } | null>(null);
@ -29,15 +29,15 @@ function LoginContent() {
// Show registration success message
if (registered === "true") {
toast.success(t('register_success'), {
description: t('login_subtitle'),
toast.success(t("register_success"), {
description: t("login_subtitle"),
duration: 5000,
});
}
// Show logout confirmation
if (logout === "true") {
toast.success(tCommon('success'), {
toast.success(tCommon("success"), {
description: "You have been securely logged out",
duration: 3000,
});
@ -96,7 +96,7 @@ function LoginContent() {
<Logo className="rounded-md" />
<div className="mt-8 flex items-center space-x-2">
<Loader2 className="h-5 w-5 animate-spin text-muted-foreground" />
<span className="text-muted-foreground">{tCommon('loading')}</span>
<span className="text-muted-foreground">{tCommon("loading")}</span>
</div>
</div>
</div>
@ -113,7 +113,7 @@ function LoginContent() {
<div className="mx-auto flex h-screen max-w-lg flex-col items-center justify-center">
<Logo className="rounded-md" />
<h1 className="my-8 text-xl font-bold text-neutral-800 dark:text-neutral-100 md:text-4xl">
{t('sign_in')}
{t("sign_in")}
</h1>
{/* URL Error Display */}

36
surfsense_web/app/(home)/register/page.tsx
View file

@ -3,16 +3,16 @@
import { AnimatePresence, motion } from "motion/react";
import Link from "next/link";
import { useRouter } from "next/navigation";
import { useTranslations } from "next-intl";
import { useEffect, useState } from "react";
import { toast } from "sonner";
import { useTranslations } from "next-intl";
import { Logo } from "@/components/Logo";
import { getAuthErrorDetails, isNetworkError, shouldRetry } from "@/lib/auth-errors";
import { AmbientBackground } from "../login/AmbientBackground";
export default function RegisterPage() {
const t = useTranslations('auth');
const tCommon = useTranslations('common');
const t = useTranslations("auth");
const tCommon = useTranslations("common");
const [email, setEmail] = useState("");
const [password, setPassword] = useState("");
const [confirmPassword, setConfirmPassword] = useState("");
@ -34,10 +34,10 @@ export default function RegisterPage() {
// Form validation
if (password !== confirmPassword) {
setError(t('passwords_no_match'));
setErrorTitle(t('password_mismatch'));
toast.error(t('password_mismatch'), {
description: t('passwords_no_match_desc'),
setError(t("passwords_no_match"));
setErrorTitle(t("password_mismatch"));
toast.error(t("password_mismatch"), {
description: t("passwords_no_match_desc"),
duration: 4000,
});
return;
@ -48,7 +48,7 @@ export default function RegisterPage() {
setErrorTitle(null);
// Show loading toast
const loadingToast = toast.loading(t('creating_account'));
const loadingToast = toast.loading(t("creating_account"));
try {
const response = await fetch(`${process.env.NEXT_PUBLIC_FASTAPI_BACKEND_URL}/auth/register`, {
@ -86,9 +86,9 @@ export default function RegisterPage() {
}
// Success toast
toast.success(t('register_success'), {
toast.success(t("register_success"), {
id: loadingToast,
description: t('redirecting_login'),
description: t("redirecting_login"),
duration: 2000,
});
@ -123,7 +123,7 @@ export default function RegisterPage() {
// Add retry action if the error is retryable
if (shouldRetry(errorCode)) {
toastOptions.action = {
label: tCommon('retry'),
label: tCommon("retry"),
onClick: () => handleSubmit(e),
};
}
@ -140,7 +140,7 @@ export default function RegisterPage() {
<div className="mx-auto flex h-screen max-w-lg flex-col items-center justify-center">
<Logo className="rounded-md" />
<h1 className="my-8 text-xl font-bold text-neutral-800 dark:text-neutral-100 md:text-4xl">
{t('create_account')}
{t("create_account")}
</h1>
<div className="w-full max-w-md">
@ -212,7 +212,7 @@ export default function RegisterPage() {
htmlFor="email"
className="block text-sm font-medium text-gray-700 dark:text-gray-300"
>
{t('email')}
{t("email")}
</label>
<input
id="email"
@ -234,7 +234,7 @@ export default function RegisterPage() {
htmlFor="password"
className="block text-sm font-medium text-gray-700 dark:text-gray-300"
>
{t('password')}
{t("password")}
</label>
<input
id="password"
@ -256,7 +256,7 @@ export default function RegisterPage() {
htmlFor="confirmPassword"
className="block text-sm font-medium text-gray-700 dark:text-gray-300"
>
{t('confirm_password')}
{t("confirm_password")}
</label>
<input
id="confirmPassword"
@ -278,18 +278,18 @@ export default function RegisterPage() {
disabled={isLoading}
className="w-full rounded-md bg-blue-600 px-4 py-2 text-white shadow-sm hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 transition-colors"
>
{isLoading ? t('creating_account_btn') : t('register')}
{isLoading ? t("creating_account_btn") : t("register")}
</button>
</form>
<div className="mt-4 text-center text-sm">
<p className="text-gray-600 dark:text-gray-400">
{t('already_have_account')}{" "}
{t("already_have_account")}{" "}
<Link
href="/login"
className="font-medium text-blue-600 hover:text-blue-500 dark:text-blue-400"
>
{t('sign_in')}
{t("sign_in")}
</Link>
</p>
</div>

24
surfsense_web/app/dashboard/[search_space_id]/client-layout.tsx
View file

@ -2,17 +2,17 @@
import { Loader2 } from "lucide-react";
import { usePathname, useRouter } from "next/navigation";
import type React from "react";
import { useEffect, useState, useMemo } from "react";
import { useTranslations } from "next-intl";
import type React from "react";
import { useEffect, useMemo, useState } from "react";
import { DashboardBreadcrumb } from "@/components/dashboard-breadcrumb";
import { LanguageSwitcher } from "@/components/LanguageSwitcher";
import { AppSidebarProvider } from "@/components/sidebar/AppSidebarProvider";
import { ThemeTogglerComponent } from "@/components/theme/theme-toggle";
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card";
import { Separator } from "@/components/ui/separator";
import { SidebarInset, SidebarProvider, SidebarTrigger } from "@/components/ui/sidebar";
import { useLLMPreferences } from "@/hooks/use-llm-configs";
import { LanguageSwitcher } from "@/components/LanguageSwitcher";
export function DashboardClientLayout({
children,
@ -25,7 +25,7 @@ export function DashboardClientLayout({
navSecondary: any[];
navMain: any[];
}) {
const t = useTranslations('dashboard');
const t = useTranslations("dashboard");
const router = useRouter();
const pathname = usePathname();
const searchSpaceIdNum = Number(searchSpaceId);
@ -37,14 +37,14 @@ export function DashboardClientLayout({
const isOnboardingPage = pathname?.includes("/onboard");
// Translate navigation items
const tNavMenu = useTranslations('nav_menu');
const tNavMenu = useTranslations("nav_menu");
const translatedNavMain = useMemo(() => {
return navMain.map((item) => ({
...item,
title: tNavMenu(item.title.toLowerCase().replace(/ /g, '_')),
title: tNavMenu(item.title.toLowerCase().replace(/ /g, "_")),
items: item.items?.map((subItem: any) => ({
...subItem,
title: tNavMenu(subItem.title.toLowerCase().replace(/ /g, '_')),
title: tNavMenu(subItem.title.toLowerCase().replace(/ /g, "_")),
})),
}));
}, [navMain, tNavMenu]);
@ -52,7 +52,7 @@ export function DashboardClientLayout({
const translatedNavSecondary = useMemo(() => {
return navSecondary.map((item) => ({
...item,
title: item.title === 'All Search Spaces' ? tNavMenu('all_search_spaces') : item.title,
title: item.title === "All Search Spaces" ? tNavMenu("all_search_spaces") : item.title,
}));
}, [navSecondary, tNavMenu]);
@ -98,8 +98,8 @@ export function DashboardClientLayout({
<div className="flex flex-col items-center justify-center min-h-screen space-y-4">
<Card className="w-[350px] bg-background/60 backdrop-blur-sm">
<CardHeader className="pb-2">
<CardTitle className="text-xl font-medium">{t('loading_config')}</CardTitle>
<CardDescription>{t('checking_llm_prefs')}</CardDescription>
<CardTitle className="text-xl font-medium">{t("loading_config")}</CardTitle>
<CardDescription>{t("checking_llm_prefs")}</CardDescription>
</CardHeader>
<CardContent className="flex justify-center py-6">
<Loader2 className="h-12 w-12 text-primary animate-spin" />
@ -116,9 +116,9 @@ export function DashboardClientLayout({
<Card className="w-[400px] bg-background/60 backdrop-blur-sm border-destructive/20">
<CardHeader className="pb-2">
<CardTitle className="text-xl font-medium text-destructive">
{t('config_error')}
{t("config_error")}
</CardTitle>
<CardDescription>{t('failed_load_llm_config')}</CardDescription>
<CardDescription>{t("failed_load_llm_config")}</CardDescription>
</CardHeader>
<CardContent>
<p className="text-sm text-muted-foreground">{error}</p>

100
surfsense_web/app/dashboard/[search_space_id]/connectors/(manage)/page.tsx
View file

@ -12,9 +12,9 @@ import {
} from "lucide-react";
import { motion } from "motion/react";
import { useParams, useRouter } from "next/navigation";
import { useTranslations } from "next-intl";
import { useEffect, useState } from "react";
import { toast } from "sonner";
import { useTranslations } from "next-intl";
import {
AlertDialog,
AlertDialogAction,
@ -63,12 +63,12 @@ import { useSearchSourceConnectors } from "@/hooks/use-search-source-connectors"
import { cn } from "@/lib/utils";
export default function ConnectorsPage() {
const t = useTranslations('connectors');
const tCommon = useTranslations('common');
const t = useTranslations("connectors");
const tCommon = useTranslations("common");
// Helper function to format date with time
const formatDateTime = (dateString: string | null): string => {
if (!dateString) return t('never');
if (!dateString) return t("never");
const date = new Date(dateString);
return new Intl.DateTimeFormat("en-US", {
@ -107,7 +107,7 @@ export default function ConnectorsPage() {
useEffect(() => {
if (error) {
toast.error(t('failed_load'));
toast.error(t("failed_load"));
console.error("Error fetching connectors:", error);
}
}, [error, t]);
@ -118,10 +118,10 @@ export default function ConnectorsPage() {
try {
await deleteConnector(connectorToDelete);
toast.success(t('delete_success'));
toast.success(t("delete_success"));
} catch (error) {
console.error("Error deleting connector:", error);
toast.error(t('delete_failed'));
toast.error(t("delete_failed"));
} finally {
setConnectorToDelete(null);
}
@ -145,10 +145,10 @@ export default function ConnectorsPage() {
const endDateStr = endDate ? format(endDate, "yyyy-MM-dd") : undefined;
await indexConnector(selectedConnectorForIndexing, searchSpaceId, startDateStr, endDateStr);
toast.success(t('indexing_started'));
toast.success(t("indexing_started"));
} catch (error) {
console.error("Error indexing connector content:", error);
toast.error(error instanceof Error ? error.message : t('indexing_failed'));
toast.error(error instanceof Error ? error.message : t("indexing_failed"));
} finally {
setIndexingConnectorId(null);
setSelectedConnectorForIndexing(null);
@ -162,10 +162,10 @@ export default function ConnectorsPage() {
setIndexingConnectorId(connectorId);
try {
await indexConnector(connectorId, searchSpaceId);
toast.success(t('indexing_started'));
toast.success(t("indexing_started"));
} catch (error) {
console.error("Error indexing connector content:", error);
toast.error(error instanceof Error ? error.message : t('indexing_failed'));
toast.error(error instanceof Error ? error.message : t("indexing_failed"));
} finally {
setIndexingConnectorId(null);
}
@ -258,21 +258,19 @@ export default function ConnectorsPage() {
className="mb-8 flex items-center justify-between"
>
<div>
<h1 className="text-3xl font-bold tracking-tight">{t('title')}</h1>
<p className="text-muted-foreground mt-2">
{t('subtitle')}
</p>
<h1 className="text-3xl font-bold tracking-tight">{t("title")}</h1>
<p className="text-muted-foreground mt-2">{t("subtitle")}</p>
</div>
<Button onClick={() => router.push(`/dashboard/${searchSpaceId}/connectors/add`)}>
<Plus className="mr-2 h-4 w-4" />
{t('add_connector')}
{t("add_connector")}
</Button>
</motion.div>
<Card>
<CardHeader className="pb-3">
<CardTitle>{t('your_connectors')}</CardTitle>
<CardDescription>{t('view_manage')}</CardDescription>
<CardTitle>{t("your_connectors")}</CardTitle>
<CardDescription>{t("view_manage")}</CardDescription>
</CardHeader>
<CardContent>
{isLoading ? (
@ -284,13 +282,11 @@ export default function ConnectorsPage() {
</div>
) : connectors.length === 0 ? (
<div className="text-center py-12">
<h3 className="text-lg font-medium mb-2">{t('no_connectors')}</h3>
<p className="text-muted-foreground mb-6">
{t('no_connectors_desc')}
</p>
<h3 className="text-lg font-medium mb-2">{t("no_connectors")}</h3>
<p className="text-muted-foreground mb-6">{t("no_connectors_desc")}</p>
<Button onClick={() => router.push(`/dashboard/${searchSpaceId}/connectors/add`)}>
<Plus className="mr-2 h-4 w-4" />
{t('add_first')}
{t("add_first")}
</Button>
</div>
) : (
@ -298,11 +294,11 @@ export default function ConnectorsPage() {
<Table>
<TableHeader>
<TableRow>
<TableHead>{t('name')}</TableHead>
<TableHead>{t('type')}</TableHead>
<TableHead>{t('last_indexed')}</TableHead>
<TableHead>{t('periodic')}</TableHead>
<TableHead className="text-right">{t('actions')}</TableHead>
<TableHead>{t("name")}</TableHead>
<TableHead>{t("type")}</TableHead>
<TableHead>{t("last_indexed")}</TableHead>
<TableHead>{t("periodic")}</TableHead>
<TableHead className="text-right">{t("actions")}</TableHead>
</TableRow>
</TableHeader>
<TableBody>
@ -313,7 +309,7 @@ export default function ConnectorsPage() {
<TableCell>
{connector.is_indexable
? formatDateTime(connector.last_indexed_at)
: t('not_indexable')}
: t("not_indexable")}
</TableCell>
<TableCell>
{connector.is_indexable ? (
@ -368,11 +364,11 @@ export default function ConnectorsPage() {
) : (
<CalendarIcon className="h-4 w-4" />
)}
<span className="sr-only">{t('index_date_range')}</span>
<span className="sr-only">{t("index_date_range")}</span>
</Button>
</TooltipTrigger>
<TooltipContent>
<p>{t('index_date_range')}</p>
<p>{t("index_date_range")}</p>
</TooltipContent>
</Tooltip>
</TooltipProvider>
@ -390,11 +386,11 @@ export default function ConnectorsPage() {
) : (
<RefreshCw className="h-4 w-4" />
)}
<span className="sr-only">{t('quick_index')}</span>
<span className="sr-only">{t("quick_index")}</span>
</Button>
</TooltipTrigger>
<TooltipContent>
<p>{t('quick_index_auto')}</p>
<p>{t("quick_index_auto")}</p>
</TooltipContent>
</Tooltip>
</TooltipProvider>
@ -429,7 +425,7 @@ export default function ConnectorsPage() {
}
>
<Edit className="h-4 w-4" />
<span className="sr-only">{tCommon('edit')}</span>
<span className="sr-only">{tCommon("edit")}</span>
</Button>
<AlertDialog>
<AlertDialogTrigger asChild>
@ -440,25 +436,25 @@ export default function ConnectorsPage() {
onClick={() => setConnectorToDelete(connector.id)}
>
<Trash2 className="h-4 w-4" />
<span className="sr-only">{tCommon('delete')}</span>
<span className="sr-only">{tCommon("delete")}</span>
</Button>
</AlertDialogTrigger>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>{t('delete_connector')}</AlertDialogTitle>
<AlertDialogTitle>{t("delete_connector")}</AlertDialogTitle>
<AlertDialogDescription>
{t('delete_confirm')}
{t("delete_confirm")}
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel onClick={() => setConnectorToDelete(null)}>
{tCommon('cancel')}
{tCommon("cancel")}
</AlertDialogCancel>
<AlertDialogAction
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
onClick={handleDeleteConnector}
>
{tCommon('delete')}
{tCommon("delete")}
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
@ -478,15 +474,13 @@ export default function ConnectorsPage() {
<Dialog open={datePickerOpen} onOpenChange={setDatePickerOpen}>
<DialogContent className="sm:max-w-[500px]">
<DialogHeader>
<DialogTitle>{t('select_date_range')}</DialogTitle>
<DialogDescription>
{t('select_date_range_desc')}
</DialogDescription>
<DialogTitle>{t("select_date_range")}</DialogTitle>
<DialogDescription>{t("select_date_range_desc")}</DialogDescription>
</DialogHeader>
<div className="grid gap-4 py-4">
<div className="grid grid-cols-2 gap-4">
<div className="space-y-2">
<Label htmlFor="start-date">{t('start_date')}</Label>
<Label htmlFor="start-date">{t("start_date")}</Label>
<Popover>
<PopoverTrigger asChild>
<Button
@ -498,7 +492,7 @@ export default function ConnectorsPage() {
)}
>
<CalendarIcon className="mr-2 h-4 w-4" />
{startDate ? format(startDate, "PPP") : t('pick_date')}
{startDate ? format(startDate, "PPP") : t("pick_date")}
</Button>
</PopoverTrigger>
<PopoverContent className="w-auto p-0" align="start">
@ -512,7 +506,7 @@ export default function ConnectorsPage() {
</Popover>
</div>
<div className="space-y-2">
<Label htmlFor="end-date">{t('end_date')}</Label>
<Label htmlFor="end-date">{t("end_date")}</Label>
<Popover>
<PopoverTrigger asChild>
<Button
@ -524,7 +518,7 @@ export default function ConnectorsPage() {
)}
>
<CalendarIcon className="mr-2 h-4 w-4" />
{endDate ? format(endDate, "PPP") : t('pick_date')}
{endDate ? format(endDate, "PPP") : t("pick_date")}
</Button>
</PopoverTrigger>
<PopoverContent className="w-auto p-0" align="start">
@ -542,7 +536,7 @@ export default function ConnectorsPage() {
setEndDate(undefined);
}}
>
{t('clear_dates')}
{t("clear_dates")}
</Button>
<Button
variant="outline"
@ -554,7 +548,7 @@ export default function ConnectorsPage() {
setEndDate(today);
}}
>
{t('last_30_days')}
{t("last_30_days")}
</Button>
<Button
variant="outline"
@ -566,7 +560,7 @@ export default function ConnectorsPage() {
setEndDate(today);
}}
>
{t('last_year')}
{t("last_year")}
</Button>
</div>
</div>
@ -580,9 +574,9 @@ export default function ConnectorsPage() {
setEndDate(undefined);
}}
>
{tCommon('cancel')}
{tCommon("cancel")}
</Button>
<Button onClick={handleIndexConnector}>{t('start_indexing')}</Button>
<Button onClick={handleIndexConnector}>{t("start_indexing")}</Button>
</DialogFooter>
</DialogContent>
</Dialog>

24
surfsense_web/app/dashboard/[search_space_id]/connectors/add/page.tsx
View file

@ -9,8 +9,8 @@ import {
import { AnimatePresence, motion, type Variants } from "motion/react";
import Link from "next/link";
import { useParams } from "next/navigation";
import { useState } from "react";
import { useTranslations } from "next-intl";
import { useState } from "react";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
import { Card, CardContent, CardFooter, CardHeader } from "@/components/ui/card";
@ -239,7 +239,7 @@ const cardVariants: Variants = {
};
export default function ConnectorsPage() {
const t = useTranslations('add_connector');
const t = useTranslations("add_connector");
const params = useParams();
const searchSpaceId = params.search_space_id as string;
const [expandedCategories, setExpandedCategories] = useState<string[]>([
@ -268,11 +268,9 @@ export default function ConnectorsPage() {
className="mb-12 text-center"
>
<h1 className="text-4xl font-bold tracking-tight bg-gradient-to-r from-indigo-500 to-purple-500 bg-clip-text text-transparent">
{t('title')}
{t("title")}
</h1>
<p className="text-muted-foreground mt-3 text-lg max-w-2xl mx-auto">
{t('subtitle')}
</p>
<p className="text-muted-foreground mt-3 text-lg max-w-2xl mx-auto">{t("subtitle")}</p>
</motion.div>
<motion.div
@ -343,7 +341,7 @@ export default function ConnectorsPage() {
variant="outline"
className="text-xs bg-amber-100 dark:bg-amber-950 text-amber-800 dark:text-amber-300 border-amber-200 dark:border-amber-800"
>
{t('coming_soon')}
{t("coming_soon")}
</Badge>
)}
{connector.status === "connected" && (
@ -351,7 +349,7 @@ export default function ConnectorsPage() {
variant="outline"
className="text-xs bg-green-100 dark:bg-green-950 text-green-800 dark:text-green-300 border-green-200 dark:border-green-800"
>
{t('connected')}
{t("connected")}
</Badge>
)}
</div>
@ -359,7 +357,9 @@ export default function ConnectorsPage() {
</CardHeader>
<CardContent className="pb-4">
<p className="text-sm text-muted-foreground">{t(connector.description)}</p>
<p className="text-sm text-muted-foreground">
{t(connector.description)}
</p>
</CardContent>
<CardFooter className="mt-auto pt-2">
@ -369,7 +369,7 @@ export default function ConnectorsPage() {
className="w-full"
>
<Button variant="default" className="w-full group">
<span>{t('connect')}</span>
<span>{t("connect")}</span>
<motion.div
className="ml-1"
initial={{ x: 0 }}
@ -387,7 +387,7 @@ export default function ConnectorsPage() {
)}
{connector.status === "coming-soon" && (
<Button variant="outline" disabled className="w-full opacity-70">
{t('coming_soon')}
{t("coming_soon")}
</Button>
)}
{connector.status === "connected" && (
@ -395,7 +395,7 @@ export default function ConnectorsPage() {
variant="outline"
className="w-full border-green-500 text-green-600 hover:bg-green-50 dark:hover:bg-green-950"
>
{t('manage')}
{t("manage")}
</Button>
)}
</CardFooter>

8
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsFilters.tsx
View file

@ -2,8 +2,8 @@
import { CircleAlert, CircleX, Columns3, Filter, ListFilter, Trash } from "lucide-react";
import { AnimatePresence, motion, type Variants } from "motion/react";
import React, { useMemo, useRef } from "react";
import { useTranslations } from "next-intl";
import React, { useMemo, useRef } from "react";
import {
AlertDialog,
AlertDialogAction,
@ -56,7 +56,7 @@ export function DocumentsFilters({
columnVisibility: ColumnVisibility;
onToggleColumn: (id: keyof ColumnVisibility, checked: boolean) => void;
}) {
const t = useTranslations('documents');
const t = useTranslations("documents");
const id = React.useId();
const inputRef = useRef<HTMLInputElement>(null);
@ -92,9 +92,9 @@ export function DocumentsFilters({
className="peer min-w-60 ps-9"
value={searchValue}
onChange={(e) => onSearch(e.target.value)}
placeholder={t('filter_placeholder')}
placeholder={t("filter_placeholder")}
type="text"
aria-label={t('filter_placeholder')}
aria-label={t("filter_placeholder")}
/>
<motion.div
className="pointer-events-none absolute inset-y-0 start-0 flex items-center justify-center ps-3 text-muted-foreground/80 peer-disabled:opacity-50"

22
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsTableShell.tsx
View file

@ -2,8 +2,8 @@
import { ChevronDown, ChevronUp, FileX } from "lucide-react";
import { motion } from "motion/react";
import React from "react";
import { useTranslations } from "next-intl";
import React from "react";
import { DocumentViewer } from "@/components/document-viewer";
import { Button } from "@/components/ui/button";
import { Checkbox } from "@/components/ui/checkbox";
@ -67,7 +67,7 @@ export function DocumentsTableShell({
sortDesc: boolean;
onSortChange: (key: SortKey) => void;
}) {
const t = useTranslations('documents');
const t = useTranslations("documents");
const sorted = React.useMemo(
() => sortDocuments(documents, sortKey, sortDesc),
[documents, sortKey, sortDesc]
@ -103,15 +103,15 @@ export function DocumentsTableShell({
<div className="flex h-[400px] w-full items-center justify-center">
<div className="flex flex-col items-center gap-2">
<div className="h-8 w-8 animate-spin rounded-full border-b-2 border-primary"></div>
<p className="text-sm text-muted-foreground">{t('loading')}</p>
<p className="text-sm text-muted-foreground">{t("loading")}</p>
</div>
</div>
) : error ? (
<div className="flex h-[400px] w-full items-center justify-center">
<div className="flex flex-col items-center gap-2">
<p className="text-sm text-destructive">{t('error_loading')}</p>
<p className="text-sm text-destructive">{t("error_loading")}</p>
<Button variant="outline" size="sm" onClick={() => onRefresh()} className="mt-2">
{t('retry')}
{t("retry")}
</Button>
</div>
</div>
@ -119,7 +119,7 @@ export function DocumentsTableShell({
<div className="flex h-[400px] w-full items-center justify-center">
<div className="flex flex-col items-center gap-2">
<FileX className="h-8 w-8 text-muted-foreground" />
<p className="text-sm text-muted-foreground">{t('no_documents')}</p>
<p className="text-sm text-muted-foreground">{t("no_documents")}</p>
</div>
</div>
) : (
@ -142,7 +142,7 @@ export function DocumentsTableShell({
className="flex h-full w-full cursor-pointer select-none items-center justify-between gap-2"
onClick={() => onSortHeader("title")}
>
{t('title')}
{t("title")}
{sortKey === "title" ? (
sortDesc ? (
<ChevronDown className="shrink-0 opacity-60" size={16} />
@ -160,7 +160,7 @@ export function DocumentsTableShell({
className="flex h-full w-full cursor-pointer select-none items-center justify-between gap-2"
onClick={() => onSortHeader("document_type")}
>
{t('type')}
{t("type")}
{sortKey === "document_type" ? (
sortDesc ? (
<ChevronDown className="shrink-0 opacity-60" size={16} />
@ -172,7 +172,7 @@ export function DocumentsTableShell({
</TableHead>
)}
{columnVisibility.content && (
<TableHead style={{ width: 300 }}>{t('content_summary')}</TableHead>
<TableHead style={{ width: 300 }}>{t("content_summary")}</TableHead>
)}
{columnVisibility.created_at && (
<TableHead style={{ width: 120 }}>
@ -266,7 +266,7 @@ export function DocumentsTableShell({
content={doc.content}
trigger={
<Button variant="ghost" size="sm" className="w-fit text-xs">
{t('view_full')}
{t("view_full")}
</Button>
}
/>
@ -337,7 +337,7 @@ export function DocumentsTableShell({
size="sm"
className="w-fit text-xs p-0 h-auto"
>
{t('view_full')}
{t("view_full")}
</Button>
}
/>

4
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/PaginationControls.tsx
View file

@ -39,7 +39,7 @@ export function PaginationControls({
canNext: boolean;
id: string;
}) {
const t = useTranslations('documents');
const t = useTranslations("documents");
const start = total === 0 ? 0 : pageIndex * pageSize + 1;
const end = Math.min((pageIndex + 1) * pageSize, total);
@ -52,7 +52,7 @@ export function PaginationControls({
transition={{ type: "spring", stiffness: 300, damping: 30 }}
>
<Label htmlFor={id} className="max-sm:sr-only">
{t('rows_per_page')}
{t("rows_per_page")}
</Label>
<Select value={String(pageSize)} onValueChange={(v) => onPageSizeChange(Number(v))}>
<SelectTrigger id={id} className="w-fit whitespace-nowrap">

12
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/page.tsx
View file

@ -2,9 +2,9 @@
import { motion } from "motion/react";
import { useParams } from "next/navigation";
import { useTranslations } from "next-intl";
import { useCallback, useEffect, useId, useMemo, useState } from "react";
import { toast } from "sonner";
import { useTranslations } from "next-intl";
import { useDocuments } from "@/hooks/use-documents";
@ -23,7 +23,7 @@ function useDebounced<T>(value: T, delay = 250) {
}
export default function DocumentsTable() {
const t = useTranslations('documents');
const t = useTranslations("documents");
const id = useId();
const params = useParams();
const searchSpaceId = Number(params.search_space_id);
@ -122,21 +122,21 @@ export default function DocumentsTable() {
const onBulkDelete = async () => {
if (selectedIds.size === 0) {
toast.error(t('no_rows_selected'));
toast.error(t("no_rows_selected"));
return;
}
try {
const results = await Promise.all(Array.from(selectedIds).map((id) => deleteDocument?.(id)));
const okCount = results.filter((r) => r === true).length;
if (okCount === selectedIds.size)
toast.success(t('delete_success_count', { count: okCount }));
else toast.error(t('delete_partial_failed'));
toast.success(t("delete_success_count", { count: okCount }));
else toast.error(t("delete_partial_failed"));
// Refetch the current page with appropriate method
await refreshCurrentView();
setSelectedIds(new Set());
} catch (e) {
console.error(e);
toast.error(t('delete_error'));
toast.error(t("delete_error"));
}
};

50
surfsense_web/app/dashboard/[search_space_id]/documents/upload/page.tsx
View file

@ -3,10 +3,10 @@
import { CheckCircle2, FileType, Info, Tag, Upload, X } from "lucide-react";
import { AnimatePresence, motion } from "motion/react";
import { useParams, useRouter } from "next/navigation";
import { useTranslations } from "next-intl";
import { useCallback, useState } from "react";
import { useDropzone } from "react-dropzone";
import { toast } from "sonner";
import { useTranslations } from "next-intl";
import { Alert, AlertDescription } from "@/components/ui/alert";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
@ -40,7 +40,7 @@ function GridPattern() {
}
export default function FileUploader() {
const t = useTranslations('upload_documents');
const t = useTranslations("upload_documents");
const params = useParams();
const search_space_id = params.search_space_id as string;
@ -276,16 +276,16 @@ export default function FileUploader() {
await response.json();
toast(t('upload_initiated'), {
description: t('upload_initiated_desc'),
toast(t("upload_initiated"), {
description: t("upload_initiated_desc"),
});
router.push(`/dashboard/${search_space_id}/documents`);
} catch (error: any) {
setIsUploading(false);
setUploadProgress(0);
toast(t('upload_error'), {
description: `${t('upload_error_desc')}: ${error.message}`,
toast(t("upload_error"), {
description: `${t("upload_error_desc")}: ${error.message}`,
});
}
};
@ -332,18 +332,14 @@ export default function FileUploader() {
<CardHeader>
<CardTitle className="flex items-center gap-2">
<Upload className="h-5 w-5" />
{t('title')}
{t("title")}
</CardTitle>
<CardDescription>
{t('subtitle')}
</CardDescription>
<CardDescription>{t("subtitle")}</CardDescription>
</CardHeader>
<CardContent>
<Alert>
<Info className="h-4 w-4" />
<AlertDescription>
{t('file_size_limit')}
</AlertDescription>
<AlertDescription>{t("file_size_limit")}</AlertDescription>
</Alert>
</CardContent>
</Card>
@ -371,7 +367,7 @@ export default function FileUploader() {
className="flex flex-col items-center gap-4"
>
<Upload className="h-12 w-12 text-primary" />
<p className="text-lg font-medium text-primary">{t('drop_files')}</p>
<p className="text-lg font-medium text-primary">{t("drop_files")}</p>
</motion.div>
) : (
<motion.div
@ -381,8 +377,8 @@ export default function FileUploader() {
>
<Upload className="h-12 w-12 text-muted-foreground" />
<div className="text-center">
<p className="text-lg font-medium">{t('drag_drop')}</p>
<p className="text-sm text-muted-foreground mt-1">{t('or_browse')}</p>
<p className="text-lg font-medium">{t("drag_drop")}</p>
<p className="text-sm text-muted-foreground mt-1">{t("or_browse")}</p>
</div>
</motion.div>
)}
@ -400,7 +396,7 @@ export default function FileUploader() {
if (input) input.click();
}}
>
{t('browse_files')}
{t("browse_files")}
</Button>
</div>
</div>
@ -422,9 +418,9 @@ export default function FileUploader() {
<CardHeader>
<div className="flex items-center justify-between">
<div>
<CardTitle>{t('selected_files', { count: files.length })}</CardTitle>
<CardTitle>{t("selected_files", { count: files.length })}</CardTitle>
<CardDescription>
{t('total_size')}: {formatFileSize(getTotalFileSize())}
{t("total_size")}: {formatFileSize(getTotalFileSize())}
</CardDescription>
</div>
<Button
@ -433,7 +429,7 @@ export default function FileUploader() {
onClick={() => setFiles([])}
disabled={isUploading}
>
{t('clear_all')}
{t("clear_all")}
</Button>
</div>
</CardHeader>
@ -490,7 +486,7 @@ export default function FileUploader() {
<Separator />
<div className="space-y-2">
<div className="flex items-center justify-between text-sm">
<span>{t('uploading_files')}</span>
<span>{t("uploading_files")}</span>
<span>{Math.round(uploadProgress)}%</span>
</div>
<Progress value={uploadProgress} className="h-2" />
@ -521,7 +517,7 @@ export default function FileUploader() {
>
<Upload className="h-5 w-5" />
</motion.div>
<span>{t('uploading')}</span>
<span>{t("uploading")}</span>
</motion.div>
) : (
<motion.div
@ -530,9 +526,7 @@ export default function FileUploader() {
whileTap={{ scale: 0.98 }}
>
<CheckCircle2 className="h-5 w-5" />
<span>
{t('upload_button', { count: files.length })}
</span>
<span>{t("upload_button", { count: files.length })}</span>
</motion.div>
)}
</Button>
@ -549,11 +543,9 @@ export default function FileUploader() {
<CardHeader>
<CardTitle className="flex items-center gap-2">
<Tag className="h-5 w-5" />
{t('supported_file_types')}
{t("supported_file_types")}
</CardTitle>
<CardDescription>
{t('file_types_desc')}
</CardDescription>
<CardDescription>{t("file_types_desc")}</CardDescription>
</CardHeader>
<CardContent>
<div className="flex flex-wrap gap-2">

58
surfsense_web/app/dashboard/[search_space_id]/documents/webpage/page.tsx
View file

@ -3,9 +3,9 @@
import { type Tag, TagInput } from "emblor";
import { Globe, Loader2 } from "lucide-react";
import { useParams, useRouter } from "next/navigation";
import { useTranslations } from "next-intl";
import { useState } from "react";
import { toast } from "sonner";
import { useTranslations } from "next-intl";
import { Button } from "@/components/ui/button";
import {
Card,
@ -21,7 +21,7 @@ import { Label } from "@/components/ui/label";
const urlRegex = /^(https?:\/\/)?([\da-z.-]+)\.([a-z.]{2,6})([/\w .-]*)*\/?$/;
export default function WebpageCrawler() {
const t = useTranslations('add_webpage');
const t = useTranslations("add_webpage");
const params = useParams();
const router = useRouter();
const search_space_id = params.search_space_id as string;
@ -40,14 +40,14 @@ export default function WebpageCrawler() {
const handleSubmit = async () => {
// Validate that we have at least one URL
if (urlTags.length === 0) {
setError(t('error_no_url'));
setError(t("error_no_url"));
return;
}
// Validate all URLs
const invalidUrls = urlTags.filter((tag) => !isValidUrl(tag.text));
if (invalidUrls.length > 0) {
setError(t('error_invalid_urls', { urls: invalidUrls.map((tag) => tag.text).join(", ") }));
setError(t("error_invalid_urls", { urls: invalidUrls.map((tag) => tag.text).join(", ") }));
return;
}
@ -55,8 +55,8 @@ export default function WebpageCrawler() {
setIsSubmitting(true);
try {
toast(t('crawling_toast'), {
description: t('crawling_toast_desc'),
toast(t("crawling_toast"), {
description: t("crawling_toast_desc"),
});
// Extract URLs from tags
@ -85,16 +85,16 @@ export default function WebpageCrawler() {
await response.json();
toast(t('success_toast'), {
description: t('success_toast_desc'),
toast(t("success_toast"), {
description: t("success_toast_desc"),
});
// Redirect to documents page
router.push(`/dashboard/${search_space_id}/documents`);
} catch (error: any) {
setError(error.message || t('error_generic'));
toast(t('error_toast'), {
description: `${t('error_toast_desc')}: ${error.message}`,
setError(error.message || t("error_generic"));
toast(t("error_toast"), {
description: `${t("error_toast_desc")}: ${error.message}`,
});
} finally {
setIsSubmitting(false);
@ -105,16 +105,16 @@ export default function WebpageCrawler() {
const handleAddTag = (text: string) => {
// Basic URL validation
if (!isValidUrl(text)) {
toast(t('invalid_url_toast'), {
description: t('invalid_url_toast_desc'),
toast(t("invalid_url_toast"), {
description: t("invalid_url_toast_desc"),
});
return;
}
// Check for duplicates
if (urlTags.some((tag) => tag.text === text)) {
toast(t('duplicate_url_toast'), {
description: t('duplicate_url_toast_desc'),
toast(t("duplicate_url_toast"), {
description: t("duplicate_url_toast_desc"),
});
return;
}
@ -134,19 +134,19 @@ export default function WebpageCrawler() {
<CardHeader>
<CardTitle className="flex items-center gap-2">
<Globe className="h-5 w-5" />
{t('title')}
{t("title")}
</CardTitle>
<CardDescription>{t('subtitle')}</CardDescription>
<CardDescription>{t("subtitle")}</CardDescription>
</CardHeader>
<CardContent>
<div className="space-y-4">
<div className="space-y-2">
<Label htmlFor="url-input">{t('label')}</Label>
<Label htmlFor="url-input">{t("label")}</Label>
<TagInput
id="url-input"
tags={urlTags}
setTags={setUrlTags}
placeholder={t('placeholder')}
placeholder={t("placeholder")}
onAddTag={handleAddTag}
styleClasses={{
inlineTagsContainer:
@ -161,20 +161,18 @@ export default function WebpageCrawler() {
activeTagIndex={activeTagIndex}
setActiveTagIndex={setActiveTagIndex}
/>
<p className="text-xs text-muted-foreground mt-1">
{t('hint')}
</p>
<p className="text-xs text-muted-foreground mt-1">{t("hint")}</p>
</div>
{error && <div className="text-sm text-red-500 mt-2">{error}</div>}
<div className="bg-muted/50 rounded-lg p-4 text-sm">
<h4 className="font-medium mb-2">{t('tips_title')}</h4>
<h4 className="font-medium mb-2">{t("tips_title")}</h4>
<ul className="list-disc pl-5 space-y-1 text-muted-foreground">
<li>{t('tip_1')}</li>
<li>{t('tip_2')}</li>
<li>{t('tip_3')}</li>
<li>{t('tip_4')}</li>
<li>{t("tip_1")}</li>
<li>{t("tip_2")}</li>
<li>{t("tip_3")}</li>
<li>{t("tip_4")}</li>
</ul>
</div>
</div>
@ -184,16 +182,16 @@ export default function WebpageCrawler() {
variant="outline"
onClick={() => router.push(`/dashboard/${search_space_id}/documents`)}
>
{t('cancel')}
{t("cancel")}
</Button>
<Button onClick={handleSubmit} disabled={isSubmitting || urlTags.length === 0}>
{isSubmitting ? (
<>
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
{t('submitting')}
{t("submitting")}
</>
) : (
t('submit')
t("submit")
)}
</Button>
</CardFooter>

62
surfsense_web/app/dashboard/[search_space_id]/documents/youtube/page.tsx
View file

@ -5,9 +5,9 @@ import { type Tag, TagInput } from "emblor";
import { Loader2 } from "lucide-react";
import { motion, type Variants } from "motion/react";
import { useParams, useRouter } from "next/navigation";
import { useTranslations } from "next-intl";
import { useState } from "react";
import { toast } from "sonner";
import { useTranslations } from "next-intl";
import { Button } from "@/components/ui/button";
import {
Card,
@ -24,7 +24,7 @@ const youtubeRegex =
/^(https:\/\/)?(www\.)?(youtube\.com\/watch\?v=|youtu\.be\/)([a-zA-Z0-9_-]{11})$/;
export default function YouTubeVideoAdder() {
const t = useTranslations('add_youtube');
const t = useTranslations("add_youtube");
const params = useParams();
const router = useRouter();
const search_space_id = params.search_space_id as string;
@ -49,14 +49,14 @@ export default function YouTubeVideoAdder() {
const handleSubmit = async () => {
// Validate that we have at least one video URL
if (videoTags.length === 0) {
setError(t('error_no_video'));
setError(t("error_no_video"));
return;
}
// Validate all URLs
const invalidUrls = videoTags.filter((tag) => !isValidYoutubeUrl(tag.text));
if (invalidUrls.length > 0) {
setError(t('error_invalid_urls', { urls: invalidUrls.map((tag) => tag.text).join(", ") }));
setError(t("error_invalid_urls", { urls: invalidUrls.map((tag) => tag.text).join(", ") }));
return;
}
@ -64,8 +64,8 @@ export default function YouTubeVideoAdder() {
setIsSubmitting(true);
try {
toast(t('processing_toast'), {
description: t('processing_toast_desc'),
toast(t("processing_toast"), {
description: t("processing_toast_desc"),
});
// Extract URLs from tags
@ -94,16 +94,16 @@ export default function YouTubeVideoAdder() {
await response.json();
toast(t('success_toast'), {
description: t('success_toast_desc'),
toast(t("success_toast"), {
description: t("success_toast_desc"),
});
// Redirect to documents page
router.push(`/dashboard/${search_space_id}/documents`);
} catch (error: any) {
setError(error.message || t('error_generic'));
toast(t('error_toast'), {
description: `${t('error_toast_desc')}: ${error.message}`,
setError(error.message || t("error_generic"));
toast(t("error_toast"), {
description: `${t("error_toast_desc")}: ${error.message}`,
});
} finally {
setIsSubmitting(false);
@ -114,16 +114,16 @@ export default function YouTubeVideoAdder() {
const handleAddTag = (text: string) => {
// Basic URL validation
if (!isValidYoutubeUrl(text)) {
toast(t('invalid_url_toast'), {
description: t('invalid_url_toast_desc'),
toast(t("invalid_url_toast"), {
description: t("invalid_url_toast_desc"),
});
return;
}
// Check for duplicates
if (videoTags.some((tag) => tag.text === text)) {
toast(t('duplicate_url_toast'), {
description: t('duplicate_url_toast_desc'),
toast(t("duplicate_url_toast"), {
description: t("duplicate_url_toast_desc"),
});
return;
}
@ -169,11 +169,9 @@ export default function YouTubeVideoAdder() {
<CardHeader>
<CardTitle className="flex items-center gap-2">
<IconBrandYoutube className="h-5 w-5" />
{t('title')}
{t("title")}
</CardTitle>
<CardDescription>
{t('subtitle')}
</CardDescription>
<CardDescription>{t("subtitle")}</CardDescription>
</CardHeader>
</motion.div>
@ -181,12 +179,12 @@ export default function YouTubeVideoAdder() {
<CardContent>
<div className="space-y-4">
<div className="space-y-2">
<Label htmlFor="video-input">{t('label')}</Label>
<Label htmlFor="video-input">{t("label")}</Label>
<TagInput
id="video-input"
tags={videoTags}
setTags={setVideoTags}
placeholder={t('placeholder')}
placeholder={t("placeholder")}
onAddTag={handleAddTag}
styleClasses={{
inlineTagsContainer:
@ -201,9 +199,7 @@ export default function YouTubeVideoAdder() {
activeTagIndex={activeTagIndex}
setActiveTagIndex={setActiveTagIndex}
/>
<p className="text-xs text-muted-foreground mt-1">
{t('hint')}
</p>
<p className="text-xs text-muted-foreground mt-1">{t("hint")}</p>
</div>
{error && (
@ -218,18 +214,18 @@ export default function YouTubeVideoAdder() {
)}
<motion.div variants={itemVariants} className="bg-muted/50 rounded-lg p-4 text-sm">
<h4 className="font-medium mb-2">{t('tips_title')}</h4>
<h4 className="font-medium mb-2">{t("tips_title")}</h4>
<ul className="list-disc pl-5 space-y-1 text-muted-foreground">
<li>{t('tip_1')}</li>
<li>{t('tip_2')}</li>
<li>{t('tip_3')}</li>
<li>{t('tip_4')}</li>
<li>{t("tip_1")}</li>
<li>{t("tip_2")}</li>
<li>{t("tip_3")}</li>
<li>{t("tip_4")}</li>
</ul>
</motion.div>
{videoTags.length > 0 && (
<motion.div variants={itemVariants} className="mt-4 space-y-2">
<h4 className="font-medium">{t('preview')}:</h4>
<h4 className="font-medium">{t("preview")}:</h4>
<div className="grid grid-cols-1 gap-3">
{videoTags.map((tag, index) => {
const videoId = extractVideoId(tag.text);
@ -265,7 +261,7 @@ export default function YouTubeVideoAdder() {
variant="outline"
onClick={() => router.push(`/dashboard/${search_space_id}/documents`)}
>
{t('cancel')}
{t("cancel")}
</Button>
<Button
onClick={handleSubmit}
@ -275,7 +271,7 @@ export default function YouTubeVideoAdder() {
{isSubmitting ? (
<>
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
{t('processing')}
{t("processing")}
</>
) : (
<>
@ -287,7 +283,7 @@ export default function YouTubeVideoAdder() {
>
<IconBrandYoutube className="h-4 w-4" />
</motion.span>
{t('submit')}
{t("submit")}
</>
)}
<motion.div

122
surfsense_web/app/dashboard/[search_space_id]/logs/(manage)/page.tsx
View file

@ -43,9 +43,9 @@ import {
} from "lucide-react";
import { AnimatePresence, motion, type Variants } from "motion/react";
import { useParams } from "next/navigation";
import { useTranslations } from "next-intl";
import React, { useContext, useId, useMemo, useRef, useState } from "react";
import { toast } from "sonner";
import { useTranslations } from "next-intl";
import { JsonMetadataViewer } from "@/components/json-metadata-viewer";
import {
AlertDialog,
@ -196,7 +196,7 @@ const createColumns = (t: (key: string) => string): ColumnDef<Log>[] => [
enableHiding: false,
},
{
header: t('level'),
header: t("level"),
accessorKey: "level",
cell: ({ row }) => {
const level = row.getValue("level") as LogLevel;
@ -220,7 +220,7 @@ const createColumns = (t: (key: string) => string): ColumnDef<Log>[] => [
size: 120,
},
{
header: t('status'),
header: t("status"),
accessorKey: "status",
cell: ({ row }) => {
const status = row.getValue("status") as LogStatus;
@ -246,7 +246,7 @@ const createColumns = (t: (key: string) => string): ColumnDef<Log>[] => [
size: 140,
},
{
header: t('source'),
header: t("source"),
accessorKey: "source",
cell: ({ row }) => {
const source = row.getValue("source") as string;
@ -257,14 +257,14 @@ const createColumns = (t: (key: string) => string): ColumnDef<Log>[] => [
transition={{ type: "spring", stiffness: 300 }}
>
<Terminal size={14} className="text-muted-foreground" />
<span className="text-sm font-mono">{source || t('system')}</span>
<span className="text-sm font-mono">{source || t("system")}</span>
</motion.div>
);
},
size: 150,
},
{
header: t('message'),
header: t("message"),
accessorKey: "message",
cell: ({ row }) => {
const message = row.getValue("message") as string;
@ -297,7 +297,7 @@ const createColumns = (t: (key: string) => string): ColumnDef<Log>[] => [
size: 400,
},
{
header: t('created_at'),
header: t("created_at"),
accessorKey: "created_at",
cell: ({ row }) => {
const date = new Date(row.getValue("created_at"));
@ -312,7 +312,7 @@ const createColumns = (t: (key: string) => string): ColumnDef<Log>[] => [
},
{
id: "actions",
header: () => <span className="sr-only">{t('actions')}</span>,
header: () => <span className="sr-only">{t("actions")}</span>,
cell: ({ row }) => <LogRowActions row={row} t={t} />,
size: 60,
enableHiding: false,
@ -329,7 +329,7 @@ const LogsContext = React.createContext<{
} | null>(null);
export default function LogsManagePage() {
const t = useTranslations('logs');
const t = useTranslations("logs");
const id = useId();
const params = useParams();
const searchSpaceId = Number(params.search_space_id);
@ -462,12 +462,12 @@ export default function LogsManagePage() {
transition={{ delay: 0.1 }}
>
<div>
<h2 className="text-2xl font-bold tracking-tight">{t('title')}</h2>
<p className="text-muted-foreground">{t('subtitle')}</p>
<h2 className="text-2xl font-bold tracking-tight">{t("title")}</h2>
<p className="text-muted-foreground">{t("subtitle")}</p>
</div>
<Button onClick={handleRefresh} variant="outline" size="sm">
<RefreshCw className="w-4 h-4 mr-2" />
{t('refresh')}
{t("refresh")}
</Button>
</motion.div>
@ -492,7 +492,7 @@ export default function LogsManagePage() {
<AlertDialogTrigger asChild>
<Button variant="outline">
<Trash className="-ms-1 me-2 opacity-60" size={16} strokeWidth={2} />
{t('delete_selected')}
{t("delete_selected")}
<span className="-me-1 ms-3 inline-flex h-5 max-h-full items-center rounded border border-border bg-background px-1 font-[inherit] text-[0.625rem] font-medium text-muted-foreground/70">
{table.getSelectedRowModel().rows.length}
</span>
@ -504,15 +504,15 @@ export default function LogsManagePage() {
<CircleAlert className="opacity-80" size={16} strokeWidth={2} />
</div>
<AlertDialogHeader>
<AlertDialogTitle>{t('confirm_title')}</AlertDialogTitle>
<AlertDialogDescription>
{t('confirm_delete_desc', { count: table.getSelectedRowModel().rows.length })}
</AlertDialogDescription>
<AlertDialogTitle>{t("confirm_title")}</AlertDialogTitle>
<AlertDialogDescription>
{t("confirm_delete_desc", { count: table.getSelectedRowModel().rows.length })}
</AlertDialogDescription>
</AlertDialogHeader>
</div>
<AlertDialogFooter>
<AlertDialogCancel>{t('cancel')}</AlertDialogCancel>
<AlertDialogAction onClick={handleDeleteRows}>{t('delete')}</AlertDialogAction>
<AlertDialogCancel>{t("cancel")}</AlertDialogCancel>
<AlertDialogAction onClick={handleDeleteRows}>{t("delete")}</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
@ -546,7 +546,7 @@ function LogsSummaryDashboard({
error: string | null;
onRefresh: () => void;
}) {
const t = useTranslations('logs');
const t = useTranslations("logs");
if (loading) {
return (
<motion.div
@ -574,9 +574,9 @@ function LogsSummaryDashboard({
<CardContent className="flex items-center justify-center h-32">
<div className="flex flex-col items-center gap-2">
<AlertCircle className="h-8 w-8 text-destructive" />
<p className="text-sm text-destructive">{t('failed_load_summary')}</p>
<p className="text-sm text-destructive">{t("failed_load_summary")}</p>
<Button variant="outline" size="sm" onClick={onRefresh}>
{t('retry')}
{t("retry")}
</Button>
</div>
</CardContent>
@ -595,12 +595,14 @@ function LogsSummaryDashboard({
<motion.div variants={fadeInScale}>
<Card>
<CardHeader className="flex flex-row items-center justify-between space-y-0 pb-2">
<CardTitle className="text-sm font-medium">{t('total_logs')}</CardTitle>
<CardTitle className="text-sm font-medium">{t("total_logs")}</CardTitle>
<Activity className="h-4 w-4 text-muted-foreground" />
</CardHeader>
<CardContent>
<div className="text-2xl font-bold">{summary.total_logs}</div>
<p className="text-xs text-muted-foreground">{t('last_hours', { hours: summary.time_window_hours })}</p>
<p className="text-xs text-muted-foreground">
{t("last_hours", { hours: summary.time_window_hours })}
</p>
</CardContent>
</Card>
</motion.div>
@ -609,14 +611,14 @@ function LogsSummaryDashboard({
<motion.div variants={fadeInScale}>
<Card>
<CardHeader className="flex flex-row items-center justify-between space-y-0 pb-2">
<CardTitle className="text-sm font-medium">{t('active_tasks')}</CardTitle>
<CardTitle className="text-sm font-medium">{t("active_tasks")}</CardTitle>
<Clock className="h-4 w-4 text-blue-600" />
</CardHeader>
<CardContent>
<div className="text-2xl font-bold text-blue-600">
{summary.active_tasks?.length || 0}
</div>
<p className="text-xs text-muted-foreground">{t('currently_running')}</p>
<p className="text-xs text-muted-foreground">{t("currently_running")}</p>
</CardContent>
</Card>
</motion.div>
@ -625,7 +627,7 @@ function LogsSummaryDashboard({
<motion.div variants={fadeInScale}>
<Card>
<CardHeader className="flex flex-row items-center justify-between space-y-0 pb-2">
<CardTitle className="text-sm font-medium">{t('success_rate')}</CardTitle>
<CardTitle className="text-sm font-medium">{t("success_rate")}</CardTitle>
<CheckCircle2 className="h-4 w-4 text-green-600" />
</CardHeader>
<CardContent>
@ -636,7 +638,7 @@ function LogsSummaryDashboard({
%
</div>
<p className="text-xs text-muted-foreground">
{summary.by_status?.SUCCESS || 0} {t('successful')}
{summary.by_status?.SUCCESS || 0} {t("successful")}
</p>
</CardContent>
</Card>
@ -646,14 +648,14 @@ function LogsSummaryDashboard({
<motion.div variants={fadeInScale}>
<Card>
<CardHeader className="flex flex-row items-center justify-between space-y-0 pb-2">
<CardTitle className="text-sm font-medium">{t('recent_failures')}</CardTitle>
<CardTitle className="text-sm font-medium">{t("recent_failures")}</CardTitle>
<AlertCircle className="h-4 w-4 text-red-600" />
</CardHeader>
<CardContent>
<div className="text-2xl font-bold text-red-600">
{summary.recent_failures?.length || 0}
</div>
<p className="text-xs text-muted-foreground">{t('need_attention')}</p>
<p className="text-xs text-muted-foreground">{t("need_attention")}</p>
</CardContent>
</Card>
</motion.div>
@ -675,7 +677,7 @@ function LogsFilters({
inputRef: React.RefObject<HTMLInputElement | null>;
id: string;
}) {
const t = useTranslations('logs');
const t = useTranslations("logs");
return (
<motion.div
className="flex flex-wrap items-center justify-between gap-3"
@ -694,7 +696,7 @@ function LogsFilters({
)}
value={(table.getColumn("message")?.getFilterValue() ?? "") as string}
onChange={(e) => table.getColumn("message")?.setFilterValue(e.target.value)}
placeholder={t('filter_by_message')}
placeholder={t("filter_by_message")}
type="text"
/>
<div className="pointer-events-none absolute inset-y-0 start-0 flex items-center justify-center ps-3 text-muted-foreground/80">
@ -717,7 +719,7 @@ function LogsFilters({
{/* Level Filter */}
<FilterDropdown
title={t('level')}
title={t("level")}
column={table.getColumn("level")}
options={uniqueLevels}
id={`${id}-level`}
@ -726,7 +728,7 @@ function LogsFilters({
{/* Status Filter */}
<FilterDropdown
title={t('status')}
title={t("status")}
column={table.getColumn("status")}
options={uniqueStatuses}
id={`${id}-status`}
@ -738,11 +740,11 @@ function LogsFilters({
<DropdownMenuTrigger asChild>
<Button variant="outline">
<Columns3 className="-ms-1 me-2 opacity-60" size={16} strokeWidth={2} />
{t('view')}
{t("view")}
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuLabel>{t('toggle_columns')}</DropdownMenuLabel>
<DropdownMenuLabel>{t("toggle_columns")}</DropdownMenuLabel>
{table
.getAllColumns()
.filter((column: any) => column.getCanHide())
@ -814,7 +816,9 @@ function FilterDropdown({
</PopoverTrigger>
<PopoverContent className="min-w-36 p-3" align="start">
<div className="space-y-3">
<div className="text-xs font-medium text-muted-foreground">{t('filter_by')} {title}</div>
<div className="text-xs font-medium text-muted-foreground">
{t("filter_by")} {title}
</div>
<div className="space-y-2">
{options.map((value, i) => (
<div key={value} className="flex items-center gap-2">
@ -897,12 +901,12 @@ function LogsTable({
initial={{ opacity: 0, y: 20 }}
animate={{ opacity: 1, y: 0 }}
>
<div className="flex h-[400px] w-full items-center justify-center">
<div className="flex flex-col items-center gap-2">
<Terminal className="h-8 w-8 text-muted-foreground" />
<p className="text-sm text-muted-foreground">{t('no_logs')}</p>
<div className="flex h-[400px] w-full items-center justify-center">
<div className="flex flex-col items-center gap-2">
<Terminal className="h-8 w-8 text-muted-foreground" />
<p className="text-sm text-muted-foreground">{t("no_logs")}</p>
</div>
</div>
</div>
</motion.div>
);
}
@ -995,13 +999,13 @@ function LogsTable({
})}
</motion.tr>
))
) : (
<TableRow>
<TableCell colSpan={columns.length} className="h-24 text-center">
{t('no_logs')}
</TableCell>
</TableRow>
)}
) : (
<TableRow>
<TableCell colSpan={columns.length} className="h-24 text-center">
{t("no_logs")}
</TableCell>
</TableRow>
)}
</AnimatePresence>
</TableBody>
</Table>
@ -1023,7 +1027,7 @@ function LogsPagination({ table, id, t }: { table: any; id: string; t: (key: str
animate={{ opacity: 1, x: 0 }}
>
<Label htmlFor={id} className="max-sm:sr-only">
{t('rows_per_page')}
{t("rows_per_page")}
</Label>
<Select
value={table.getState().pagination.pageSize.toString()}
@ -1122,11 +1126,11 @@ function LogRowActions({ row, t }: { row: Row<Log>; t: (key: string) => string }
setIsDeleting(true);
try {
await deleteLog(log.id);
toast.success(t('log_deleted_success'));
toast.success(t("log_deleted_success"));
await refreshLogs();
} catch (error) {
console.error("Error deleting log:", error);
toast.error(t('log_deleted_error'));
toast.error(t("log_deleted_error"));
} finally {
setIsDeleting(false);
setIsOpen(false);
@ -1147,7 +1151,7 @@ function LogRowActions({ row, t }: { row: Row<Log>; t: (key: string) => string }
metadata={log.log_metadata}
trigger={
<DropdownMenuItem onSelect={(e) => e.preventDefault()}>
{t('view_metadata')}
{t("view_metadata")}
</DropdownMenuItem>
}
/>
@ -1161,20 +1165,18 @@ function LogRowActions({ row, t }: { row: Row<Log>; t: (key: string) => string }
setIsOpen(true);
}}
>
{t('delete')}
{t("delete")}
</DropdownMenuItem>
</AlertDialogTrigger>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>{t('confirm_delete_log_title')}</AlertDialogTitle>
<AlertDialogDescription>
{t('confirm_delete_log_desc')}
</AlertDialogDescription>
<AlertDialogTitle>{t("confirm_delete_log_title")}</AlertDialogTitle>
<AlertDialogDescription>{t("confirm_delete_log_desc")}</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>{t('cancel')}</AlertDialogCancel>
<AlertDialogCancel>{t("cancel")}</AlertDialogCancel>
<AlertDialogAction onClick={handleDelete} disabled={isDeleting}>
{isDeleting ? t('deleting') : t('delete')}
{isDeleting ? t("deleting") : t("delete")}
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>

32
surfsense_web/app/dashboard/[search_space_id]/onboard/page.tsx
View file

@ -3,8 +3,8 @@
import { ArrowLeft, ArrowRight, Bot, CheckCircle, Sparkles } from "lucide-react";
import { AnimatePresence, motion } from "motion/react";
import { useParams, useRouter } from "next/navigation";
import { useEffect, useState } from "react";
import { useTranslations } from "next-intl";
import { useEffect, useState } from "react";
import { Logo } from "@/components/Logo";
import { AddProviderStep } from "@/components/onboard/add-provider-step";
import { AssignRolesStep } from "@/components/onboard/assign-roles-step";
@ -17,7 +17,7 @@ import { useLLMConfigs, useLLMPreferences } from "@/hooks/use-llm-configs";
const TOTAL_STEPS = 3;
const OnboardPage = () => {
const t = useTranslations('onboard');
const t = useTranslations("onboard");
const router = useRouter();
const params = useParams();
const searchSpaceId = Number(params.search_space_id);
@ -69,12 +69,12 @@ const OnboardPage = () => {
const progress = (currentStep / TOTAL_STEPS) * 100;
const stepTitles = [t('add_llm_provider'), t('assign_llm_roles'), t('setup_complete')];
const stepTitles = [t("add_llm_provider"), t("assign_llm_roles"), t("setup_complete")];
const stepDescriptions = [
t('configure_first_provider'),
t('assign_specific_roles'),
t('all_set'),
t("configure_first_provider"),
t("assign_specific_roles"),
t("all_set"),
];
const canProceedToStep2 = !configsLoading && llmConfigs.length > 0;
@ -106,7 +106,7 @@ const OnboardPage = () => {
<Card className="w-[350px] bg-background/60 backdrop-blur-sm">
<CardContent className="flex flex-col items-center justify-center py-12">
<Bot className="h-12 w-12 text-primary animate-pulse mb-4" />
<p className="text-sm text-muted-foreground">{t('loading_config')}</p>
<p className="text-sm text-muted-foreground">{t("loading_config")}</p>
</CardContent>
</Card>
</div>
@ -125,11 +125,9 @@ const OnboardPage = () => {
<div className="text-center mb-8">
<div className="flex items-center justify-center mb-4">
<Logo className="w-12 h-12 mr-3 rounded-full" />
<h1 className="text-3xl font-bold">{t('welcome_title')}</h1>
<h1 className="text-3xl font-bold">{t("welcome_title")}</h1>
</div>
<p className="text-muted-foreground text-lg">
{t('welcome_subtitle')}
</p>
<p className="text-muted-foreground text-lg">{t("welcome_subtitle")}</p>
</div>
{/* Progress */}
@ -137,9 +135,11 @@ const OnboardPage = () => {
<CardContent className="pt-6">
<div className="flex items-center justify-between mb-4">
<div className="text-sm font-medium">
{t('step_of', { current: currentStep, total: TOTAL_STEPS })}
{t("step_of", { current: currentStep, total: TOTAL_STEPS })}
</div>
<div className="text-sm text-muted-foreground">
{t("percent_complete", { percent: Math.round(progress) })}
</div>
<div className="text-sm text-muted-foreground">{t('percent_complete', { percent: Math.round(progress) })}</div>
</div>
<Progress value={progress} className="mb-4" />
<div className="grid grid-cols-3 gap-4">
@ -227,7 +227,7 @@ const OnboardPage = () => {
className="flex items-center gap-2"
>
<ArrowLeft className="w-4 h-4" />
{t('previous')}
{t("previous")}
</Button>
<div className="flex gap-2">
@ -240,14 +240,14 @@ const OnboardPage = () => {
}
className="flex items-center gap-2"
>
{t('next')}
{t("next")}
<ArrowRight className="w-4 h-4" />
</Button>
)}
{currentStep === TOTAL_STEPS && (
<Button onClick={handleComplete} className="flex items-center gap-2">
{t('complete_setup')}
{t("complete_setup")}
<CheckCircle className="w-4 h-4" />
</Button>
)}

21
surfsense_web/app/dashboard/[search_space_id]/researcher/[[...chat_id]]/page.tsx
View file

@ -3,7 +3,6 @@
import { type CreateMessage, type Message, useChat } from "@ai-sdk/react";
import { useParams, useRouter } from "next/navigation";
import { useEffect, useMemo } from "react";
import type { ResearchMode } from "@/components/chat";
import ChatInterface from "@/components/chat/ChatInterface";
import { useChatAPI, useChatState } from "@/hooks/use-chat";
import type { Document } from "@/hooks/use-documents";
@ -22,7 +21,6 @@ export default function ResearcherPage() {
searchMode,
setSearchMode,
researchMode,
setResearchMode,
selectedConnectors,
setSelectedConnectors,
selectedDocuments,
@ -52,7 +50,7 @@ export default function ResearcherPage() {
selectedDocuments: Document[];
selectedConnectors: string[];
searchMode: "DOCUMENTS" | "CHUNKS";
researchMode: ResearchMode;
researchMode: "QNA"; // Always QNA mode
}
const getChatStateStorageKey = (searchSpaceId: string, chatId: string) =>
@ -132,17 +130,10 @@ export default function ResearcherPage() {
setSelectedDocuments(restoredState.selectedDocuments);
setSelectedConnectors(restoredState.selectedConnectors);
setSearchMode(restoredState.searchMode);
setResearchMode(restoredState.researchMode);
// researchMode is always "QNA", no need to restore
}
}
}, [
chatIdParam,
search_space_id,
setSelectedDocuments,
setSelectedConnectors,
setSearchMode,
setResearchMode,
]);
}, [chatIdParam, search_space_id, setSelectedDocuments, setSelectedConnectors, setSearchMode]);
const loadChatData = async (chatId: string) => {
try {
@ -150,9 +141,7 @@ export default function ResearcherPage() {
if (!chatData) return;
// Update configuration from chat data
if (chatData.type) {
setResearchMode(chatData.type as ResearchMode);
}
// researchMode is always "QNA", no need to set from chat data
if (chatData.initial_connectors && Array.isArray(chatData.initial_connectors)) {
setSelectedConnectors(chatData.initial_connectors);
@ -209,8 +198,6 @@ export default function ResearcherPage() {
selectedConnectors={selectedConnectors}
searchMode={searchMode}
onSearchModeChange={setSearchMode}
researchMode={researchMode}
onResearchModeChange={setResearchMode}
/>
);
}

58
surfsense_web/app/dashboard/page.tsx
View file

@ -5,9 +5,9 @@ import { motion, type Variants } from "motion/react";
import Image from "next/image";
import Link from "next/link";
import { useRouter } from "next/navigation";
import { useTranslations } from "next-intl";
import { useEffect, useState } from "react";
import { toast } from "sonner";
import { useTranslations } from "next-intl";
import { Logo } from "@/components/Logo";
import { ThemeTogglerComponent } from "@/components/theme/theme-toggle";
import { UserDropdown } from "@/components/UserDropdown";
@ -62,7 +62,7 @@ const formatDate = (dateString: string): string => {
* Loading screen component with animation
*/
const LoadingScreen = () => {
const t = useTranslations('dashboard');
const t = useTranslations("dashboard");
return (
<div className="flex flex-col items-center justify-center min-h-[60vh] space-y-4">
<motion.div
@ -72,8 +72,8 @@ const LoadingScreen = () => {
>
<Card className="w-[350px] bg-background/60 backdrop-blur-sm">
<CardHeader className="pb-2">
<CardTitle className="text-xl font-medium">{t('loading')}</CardTitle>
<CardDescription>{t('fetching_spaces')}</CardDescription>
<CardTitle className="text-xl font-medium">{t("loading")}</CardTitle>
<CardDescription>{t("fetching_spaces")}</CardDescription>
</CardHeader>
<CardContent className="flex justify-center py-6">
<motion.div
@ -84,7 +84,7 @@ const LoadingScreen = () => {
</motion.div>
</CardContent>
<CardFooter className="border-t pt-4 text-sm text-muted-foreground">
{t('may_take_moment')}
{t("may_take_moment")}
</CardFooter>
</Card>
</motion.div>
@ -96,7 +96,7 @@ const LoadingScreen = () => {
* Error screen component with animation
*/
const ErrorScreen = ({ message }: { message: string }) => {
const t = useTranslations('dashboard');
const t = useTranslations("dashboard");
const router = useRouter();
return (
@ -110,22 +110,22 @@ const ErrorScreen = ({ message }: { message: string }) => {
<CardHeader className="pb-2">
<div className="flex items-center gap-2">
<AlertCircle className="h-5 w-5 text-destructive" />
<CardTitle className="text-xl font-medium">{t('error')}</CardTitle>
<CardTitle className="text-xl font-medium">{t("error")}</CardTitle>
</div>
<CardDescription>{t('something_wrong')}</CardDescription>
<CardDescription>{t("something_wrong")}</CardDescription>
</CardHeader>
<CardContent>
<Alert variant="destructive" className="bg-destructive/10 border-destructive/30">
<AlertCircle className="h-4 w-4" />
<AlertTitle>{t('error_details')}</AlertTitle>
<AlertTitle>{t("error_details")}</AlertTitle>
<AlertDescription className="mt-2">{message}</AlertDescription>
</Alert>
</CardContent>
<CardFooter className="flex justify-end gap-2 border-t pt-4">
<Button variant="outline" onClick={() => router.refresh()}>
{t('try_again')}
{t("try_again")}
</Button>
<Button onClick={() => router.push("/")}>{t('go_home')}</Button>
<Button onClick={() => router.push("/")}>{t("go_home")}</Button>
</CardFooter>
</Card>
</motion.div>
@ -134,9 +134,9 @@ const ErrorScreen = ({ message }: { message: string }) => {
};
const DashboardPage = () => {
const t = useTranslations('dashboard');
const tCommon = useTranslations('common');
const t = useTranslations("dashboard");
const tCommon = useTranslations("common");
// Animation variants
const containerVariants: Variants = {
hidden: { opacity: 0 },
@ -245,8 +245,8 @@ const DashboardPage = () => {
<div className="flex flex-row space-x-4">
<Logo className="w-10 h-10 rounded-md" />
<div className="flex flex-col space-y-2">
<h1 className="text-4xl font-bold">{t('surfsense_dashboard')}</h1>
<p className="text-muted-foreground">{t('welcome_message')}</p>
<h1 className="text-4xl font-bold">{t("surfsense_dashboard")}</h1>
<p className="text-muted-foreground">{t("welcome_message")}</p>
</div>
</div>
<div className="flex items-center space-x-3">
@ -257,12 +257,12 @@ const DashboardPage = () => {
<div className="flex flex-col space-y-6 mt-6">
<div className="flex justify-between items-center">
<h2 className="text-2xl font-semibold">{t('your_search_spaces')}</h2>
<h2 className="text-2xl font-semibold">{t("your_search_spaces")}</h2>
<motion.div whileHover={{ scale: 1.05 }} whileTap={{ scale: 0.95 }}>
<Link href="/dashboard/searchspaces">
<Button className="h-10">
<Plus className="mr-2 h-4 w-4" />
{t('create_search_space')}
{t("create_search_space")}
</Button>
</Link>
</motion.div>
@ -318,18 +318,18 @@ const DashboardPage = () => {
</AlertDialogTrigger>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>{t('delete_search_space')}</AlertDialogTitle>
<AlertDialogTitle>{t("delete_search_space")}</AlertDialogTitle>
<AlertDialogDescription>
{t('delete_space_confirm', { name: space.name })}
{t("delete_space_confirm", { name: space.name })}
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>{tCommon('cancel')}</AlertDialogCancel>
<AlertDialogCancel>{tCommon("cancel")}</AlertDialogCancel>
<AlertDialogAction
onClick={() => handleDeleteSearchSpace(space.id)}
className="bg-destructive hover:bg-destructive/90"
>
{tCommon('delete')}
{tCommon("delete")}
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
@ -347,7 +347,9 @@ const DashboardPage = () => {
</div>
<div className="mt-4 flex justify-between text-xs text-muted-foreground">
{/* <span>{space.title}</span> */}
<span>{t('created')} {formatDate(space.created_at)}</span>
<span>
{t("created")} {formatDate(space.created_at)}
</span>
</div>
</div>
</Link>
@ -364,14 +366,12 @@ const DashboardPage = () => {
<div className="rounded-full bg-muted/50 p-4 mb-4">
<Search className="h-8 w-8 text-muted-foreground" />
</div>
<h3 className="text-lg font-medium mb-2">{t('no_spaces_found')}</h3>
<p className="text-muted-foreground mb-6">
{t('create_first_space')}
</p>
<h3 className="text-lg font-medium mb-2">{t("no_spaces_found")}</h3>
<p className="text-muted-foreground mb-6">{t("create_first_space")}</p>
<Link href="/dashboard/searchspaces">
<Button>
<Plus className="mr-2 h-4 w-4" />
{t('create_search_space')}
{t("create_search_space")}
</Button>
</Link>
</motion.div>
@ -392,7 +392,7 @@ const DashboardPage = () => {
<Link href="/dashboard/searchspaces" className="flex h-full">
<div className="flex flex-col items-center justify-center h-full w-full rounded-xl border border-dashed bg-muted/10 hover:border-primary/50 transition-colors">
<Plus className="h-10 w-10 mb-3 text-muted-foreground" />
<span className="text-sm font-medium">{t('add_new_search_space')}</span>
<span className="text-sm font-medium">{t("add_new_search_space")}</span>
</div>
</Link>
</Tilt>

4
surfsense_web/app/layout.tsx
View file

@ -2,11 +2,11 @@ import type { Metadata } from "next";
import "./globals.css";
import { RootProvider } from "fumadocs-ui/provider";
import { Roboto } from "next/font/google";
import { I18nProvider } from "@/components/providers/I18nProvider";
import { ThemeProvider } from "@/components/theme/theme-provider";
import { Toaster } from "@/components/ui/sonner";
import { cn } from "@/lib/utils";
import { LocaleProvider } from "@/contexts/LocaleContext";
import { I18nProvider } from "@/components/providers/I18nProvider";
import { cn } from "@/lib/utils";
const roboto = Roboto({
subsets: ["latin"],

85
surfsense_web/components/LanguageSwitcher.tsx
View file

@ -1,14 +1,14 @@
'use client';
"use client";
import {useLocaleContext} from '@/contexts/LocaleContext';
import { Globe } from "lucide-react";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import {Globe} from 'lucide-react';
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { useLocaleContext } from "@/contexts/LocaleContext";
/**
* Language switcher component
@ -16,41 +16,40 @@ import {Globe} from 'lucide-react';
* Persists preference in localStorage
*/
export function LanguageSwitcher() {
const {locale, setLocale} = useLocaleContext();
const { locale, setLocale } = useLocaleContext();
// Supported languages configuration
const languages = [
{code: 'en' as const, name: 'English', flag: '🇺🇸'},
{code: 'zh' as const, name: '简体中文', flag: '🇨🇳'},
];
// Supported languages configuration
const languages = [
{ code: "en" as const, name: "English", flag: "🇺🇸" },
{ code: "zh" as const, name: "简体中文", flag: "🇨🇳" },
];
/**
* Handle language change
* Updates locale in context and localStorage
*/
const handleLanguageChange = (newLocale: string) => {
setLocale(newLocale as 'en' | 'zh');
};
/**
* Handle language change
* Updates locale in context and localStorage
*/
const handleLanguageChange = (newLocale: string) => {
setLocale(newLocale as "en" | "zh");
};
return (
<Select value={locale} onValueChange={handleLanguageChange}>
<SelectTrigger className="w-[160px]">
<Globe className="mr-2 h-4 w-4" />
<SelectValue>
{languages.find(lang => lang.code === locale)?.name || 'English'}
</SelectValue>
</SelectTrigger>
<SelectContent>
{languages.map((language) => (
<SelectItem key={language.code} value={language.code}>
<span className="flex items-center gap-2">
<span>{language.flag}</span>
<span>{language.name}</span>
</span>
</SelectItem>
))}
</SelectContent>
</Select>
);
return (
<Select value={locale} onValueChange={handleLanguageChange}>
<SelectTrigger className="w-[160px]">
<Globe className="mr-2 h-4 w-4" />
<SelectValue>
{languages.find((lang) => lang.code === locale)?.name || "English"}
</SelectValue>
</SelectTrigger>
<SelectContent>
{languages.map((language) => (
<SelectItem key={language.code} value={language.code}>
<span className="flex items-center gap-2">
<span>{language.flag}</span>
<span>{language.name}</span>
</span>
</SelectItem>
))}
</SelectContent>
</Select>
);
}

83
surfsense_web/components/chat/ChatInputGroup.tsx
View file

@ -4,7 +4,6 @@ import { ChatInput } from "@llamaindex/chat-ui";
import { Brain, Check, FolderOpen, Zap } from "lucide-react";
import { useParams } from "next/navigation";
import React, { Suspense, useCallback, useState } from "react";
import type { ResearchMode } from "@/components/chat";
import { ConnectorButton as ConnectorButtonComponent } from "@/components/chat/ConnectorComponents";
import { DocumentsDataTable } from "@/components/chat/DocumentsDataTable";
import { Badge } from "@/components/ui/badge";
@ -243,74 +242,6 @@ const SearchModeSelector = React.memo(
SearchModeSelector.displayName = "SearchModeSelector";
const ResearchModeSelector = React.memo(
({
researchMode,
onResearchModeChange,
}: {
researchMode?: ResearchMode;
onResearchModeChange?: (mode: ResearchMode) => void;
}) => {
const handleValueChange = React.useCallback(
(value: string) => {
onResearchModeChange?.(value as ResearchMode);
},
[onResearchModeChange]
);
// Memoize mode options to prevent recreation
const modeOptions = React.useMemo(
() => [
{ value: "QNA", label: "Q&A", shortLabel: "Q&A" },
{
value: "REPORT_GENERAL",
label: "General Report",
shortLabel: "General",
},
{
value: "REPORT_DEEP",
label: "Deep Report",
shortLabel: "Deep",
},
{
value: "REPORT_DEEPER",
label: "Deeper Report",
shortLabel: "Deeper",
},
],
[]
);
return (
<div className="flex items-center gap-1 sm:gap-2">
<span className="text-xs text-muted-foreground hidden sm:block">Mode:</span>
<Select value={researchMode} onValueChange={handleValueChange}>
<SelectTrigger className="w-auto min-w-[80px] sm:min-w-[120px] h-8 text-xs border-border bg-background hover:bg-muted/50 transition-colors duration-200 focus:ring-2 focus:ring-primary/20">
<SelectValue placeholder="Mode" className="text-xs" />
</SelectTrigger>
<SelectContent align="end" className="min-w-[140px]">
<div className="px-2 py-1.5 text-xs font-medium text-muted-foreground border-b bg-muted/30">
Research Mode
</div>
{modeOptions.map((option) => (
<SelectItem
key={option.value}
value={option.value}
className="px-3 py-2 cursor-pointer hover:bg-accent/50 focus:bg-accent"
>
<span className="hidden sm:inline">{option.label}</span>
<span className="sm:hidden">{option.shortLabel}</span>
</SelectItem>
))}
</SelectContent>
</Select>
</div>
);
}
);
ResearchModeSelector.displayName = "ResearchModeSelector";
const LLMSelector = React.memo(() => {
const { search_space_id } = useParams();
const searchSpaceId = Number(search_space_id);
@ -473,8 +404,6 @@ const CustomChatInputOptions = React.memo(
selectedConnectors,
searchMode,
onSearchModeChange,
researchMode,
onResearchModeChange,
}: {
onDocumentSelectionChange?: (documents: Document[]) => void;
selectedDocuments?: Document[];
@ -482,8 +411,6 @@ const CustomChatInputOptions = React.memo(
selectedConnectors?: string[];
searchMode?: "DOCUMENTS" | "CHUNKS";
onSearchModeChange?: (mode: "DOCUMENTS" | "CHUNKS") => void;
researchMode?: ResearchMode;
onResearchModeChange?: (mode: ResearchMode) => void;
}) => {
// Memoize the loading fallback to prevent recreation
const loadingFallback = React.useMemo(
@ -506,10 +433,6 @@ const CustomChatInputOptions = React.memo(
/>
</Suspense>
<SearchModeSelector searchMode={searchMode} onSearchModeChange={onSearchModeChange} />
<ResearchModeSelector
researchMode={researchMode}
onResearchModeChange={onResearchModeChange}
/>
<LLMSelector />
</div>
);
@ -526,8 +449,6 @@ export const ChatInputUI = React.memo(
selectedConnectors,
searchMode,
onSearchModeChange,
researchMode,
onResearchModeChange,
}: {
onDocumentSelectionChange?: (documents: Document[]) => void;
selectedDocuments?: Document[];
@ -535,8 +456,6 @@ export const ChatInputUI = React.memo(
selectedConnectors?: string[];
searchMode?: "DOCUMENTS" | "CHUNKS";
onSearchModeChange?: (mode: "DOCUMENTS" | "CHUNKS") => void;
researchMode?: ResearchMode;
onResearchModeChange?: (mode: ResearchMode) => void;
}) => {
return (
<ChatInput>
@ -551,8 +470,6 @@ export const ChatInputUI = React.memo(
selectedConnectors={selectedConnectors}
searchMode={searchMode}
onSearchModeChange={onSearchModeChange}
researchMode={researchMode}
onResearchModeChange={onResearchModeChange}
/>
</ChatInput>
);

7
surfsense_web/components/chat/ChatInterface.tsx
View file

@ -1,7 +1,6 @@
"use client";
import { type ChatHandler, ChatSection as LlamaIndexChatSection } from "@llamaindex/chat-ui";
import type { ResearchMode } from "@/components/chat";
import { ChatInputUI } from "@/components/chat/ChatInputGroup";
import { ChatMessagesUI } from "@/components/chat/ChatMessages";
import type { Document } from "@/hooks/use-documents";
@ -14,8 +13,6 @@ interface ChatInterfaceProps {
selectedConnectors?: string[];
searchMode?: "DOCUMENTS" | "CHUNKS";
onSearchModeChange?: (mode: "DOCUMENTS" | "CHUNKS") => void;
researchMode?: ResearchMode;
onResearchModeChange?: (mode: ResearchMode) => void;
}
export default function ChatInterface({
@ -26,8 +23,6 @@ export default function ChatInterface({
selectedConnectors = [],
searchMode,
onSearchModeChange,
researchMode,
onResearchModeChange,
}: ChatInterfaceProps) {
return (
<LlamaIndexChatSection handler={handler} className="flex h-full">
@ -41,8 +36,6 @@ export default function ChatInterface({
selectedConnectors={selectedConnectors}
searchMode={searchMode}
onSearchModeChange={onSearchModeChange}
researchMode={researchMode}
onResearchModeChange={onResearchModeChange}
/>
</div>
</div>

121
surfsense_web/components/chat/ConnectorComponents.tsx
View file

@ -1,35 +1,8 @@
import { ChevronDown, FileText, MessageCircle, Plus } from "lucide-react";
import { ChevronDown, Plus } from "lucide-react";
import type React from "react";
import { Button } from "@/components/ui/button";
import { getConnectorIcon } from "@/contracts/enums/connectorIcons";
import type { Connector, ResearchMode } from "./types";
export const researcherOptions: {
value: ResearchMode;
label: string;
icon: React.ReactNode;
}[] = [
{
value: "QNA",
label: "Q/A",
icon: getConnectorIcon("GENERAL"),
},
{
value: "REPORT_GENERAL",
label: "General",
icon: getConnectorIcon("GENERAL"),
},
{
value: "REPORT_DEEP",
label: "Deep",
icon: getConnectorIcon("DEEP"),
},
{
value: "REPORT_DEEPER",
label: "Deeper",
icon: getConnectorIcon("DEEPER"),
},
];
import type { Connector } from "./types";
/**
* Displays a small icon for a connector type
@ -134,93 +107,3 @@ export const ConnectorButton = ({
</Button>
);
};
// New component for Research Mode Control with Q/A and Report toggle
type ResearchModeControlProps = {
value: ResearchMode;
onChange: (value: ResearchMode) => void;
};
export const ResearchModeControl = ({ value, onChange }: ResearchModeControlProps) => {
// Determine if we're in Q/A mode or Report mode
const isQnaMode = value === "QNA";
const isReportMode = value.startsWith("REPORT_");
// Get the current report sub-mode
const getCurrentReportMode = () => {
if (!isReportMode) return "GENERAL";
return value.replace("REPORT_", "") as "GENERAL" | "DEEP" | "DEEPER";
};
const reportSubOptions = [
{ value: "GENERAL", label: "General", icon: getConnectorIcon("GENERAL") },
{ value: "DEEP", label: "Deep", icon: getConnectorIcon("DEEP") },
{ value: "DEEPER", label: "Deeper", icon: getConnectorIcon("DEEPER") },
];
const handleModeToggle = (mode: "QNA" | "REPORT") => {
if (mode === "QNA") {
onChange("QNA");
} else {
// Default to GENERAL for Report mode
onChange("REPORT_GENERAL");
}
};
const handleReportSubModeChange = (subMode: string) => {
onChange(`REPORT_${subMode}` as ResearchMode);
};
return (
<div className="flex items-center gap-2">
{/* Main Q/A vs Report Toggle */}
<div className="flex h-8 rounded-md border border-border overflow-hidden">
<Button
className={`flex h-full items-center gap-1 px-3 text-xs font-medium transition-colors whitespace-nowrap ${
isQnaMode
? "bg-primary text-primary-foreground"
: "hover:bg-muted text-muted-foreground hover:text-foreground"
}`}
onClick={() => handleModeToggle("QNA")}
aria-pressed={isQnaMode}
>
<MessageCircle className="h-3 w-3" />
<span>Q/A</span>
</Button>
<Button
className={`flex h-full items-center gap-1 px-3 text-xs font-medium transition-colors whitespace-nowrap ${
isReportMode
? "bg-primary text-primary-foreground"
: "hover:bg-muted text-muted-foreground hover:text-foreground"
}`}
onClick={() => handleModeToggle("REPORT")}
aria-pressed={isReportMode}
>
<FileText className="h-3 w-3" />
<span>Report</span>
</Button>
</div>
{/* Report Sub-options (only show when in Report mode) */}
{isReportMode && (
<div className="flex h-8 rounded-md border border-border overflow-hidden">
{reportSubOptions.map((option) => (
<Button
key={option.value}
className={`flex h-full items-center gap-1 px-2 text-xs font-medium transition-colors whitespace-nowrap ${
getCurrentReportMode() === option.value
? "bg-primary text-primary-foreground"
: "hover:bg-muted text-muted-foreground hover:text-foreground"
}`}
onClick={() => handleReportSubModeChange(option.value)}
aria-pressed={getCurrentReportMode() === option.value}
>
{option.icon}
<span>{option.label}</span>
</Button>
))}
</div>
)}
</div>
);
};

2
surfsense_web/components/chat/types.ts
View file

@ -47,4 +47,4 @@ export interface ToolInvocationUIPart {
toolInvocation: ToolInvocation;
}
export type ResearchMode = "QNA" | "REPORT_GENERAL" | "REPORT_DEEP" | "REPORT_DEEPER";
export type ResearchMode = "QNA";

51
surfsense_web/components/dashboard-breadcrumb.tsx
View file

@ -1,8 +1,8 @@
"use client";
import { usePathname } from "next/navigation";
import React from "react";
import { useTranslations } from "next-intl";
import React from "react";
import {
Breadcrumb,
BreadcrumbItem,
@ -18,7 +18,7 @@ interface BreadcrumbItemInterface {
}
export function DashboardBreadcrumb() {
const t = useTranslations('breadcrumb');
const t = useTranslations("breadcrumb");
const pathname = usePathname();
// Parse the pathname to create breadcrumb items
@ -27,11 +27,14 @@ export function DashboardBreadcrumb() {
const breadcrumbs: BreadcrumbItemInterface[] = [];
// Always start with Dashboard
breadcrumbs.push({ label: t('dashboard'), href: "/dashboard" });
breadcrumbs.push({ label: t("dashboard"), href: "/dashboard" });
// Handle search space
if (segments[0] === "dashboard" && segments[1]) {
breadcrumbs.push({ label: `${t('search_space')} ${segments[1]}`, href: `/dashboard/${segments[1]}` });
breadcrumbs.push({
label: `${t("search_space")} ${segments[1]}`,
href: `/dashboard/${segments[1]}`,
});
// Handle specific sections
if (segments[2]) {
@ -40,13 +43,13 @@ export function DashboardBreadcrumb() {
// Map section names to more readable labels
const sectionLabels: Record<string, string> = {
researcher: t('researcher'),
documents: t('documents'),
connectors: t('connectors'),
podcasts: t('podcasts'),
logs: t('logs'),
chats: t('chats'),
settings: t('settings'),
researcher: t("researcher"),
documents: t("documents"),
connectors: t("connectors"),
podcasts: t("podcasts"),
logs: t("logs"),
chats: t("chats"),
settings: t("settings"),
};
sectionLabel = sectionLabels[section] || sectionLabel;
@ -59,14 +62,14 @@ export function DashboardBreadcrumb() {
// Handle documents sub-sections
if (section === "documents") {
const documentLabels: Record<string, string> = {
upload: t('upload_documents'),
youtube: t('add_youtube'),
webpage: t('add_webpages'),
upload: t("upload_documents"),
youtube: t("add_youtube"),
webpage: t("add_webpages"),
};
const documentLabel = documentLabels[subSection] || subSectionLabel;
breadcrumbs.push({
label: t('documents'),
label: t("documents"),
href: `/dashboard/${segments[1]}/documents`,
});
breadcrumbs.push({ label: documentLabel });
@ -108,13 +111,13 @@ export function DashboardBreadcrumb() {
}
const connectorLabels: Record<string, string> = {
add: t('add_connector'),
manage: t('manage_connectors'),
add: t("add_connector"),
manage: t("manage_connectors"),
};
const connectorLabel = connectorLabels[subSection] || subSectionLabel;
breadcrumbs.push({
label: t('connectors'),
label: t("connectors"),
href: `/dashboard/${segments[1]}/connectors`,
});
breadcrumbs.push({ label: connectorLabel });
@ -123,12 +126,12 @@ export function DashboardBreadcrumb() {
// Handle other sub-sections
const subSectionLabels: Record<string, string> = {
upload: t('upload_documents'),
youtube: t('add_youtube'),
webpage: t('add_webpages'),
add: t('add_connector'),
edit: t('edit_connector'),
manage: t('manage'),
upload: t("upload_documents"),
youtube: t("add_youtube"),
webpage: t("add_webpages"),
add: t("add_connector"),
edit: t("edit_connector"),
manage: t("manage"),
};
subSectionLabel = subSectionLabels[subSection] || subSectionLabel;

344
surfsense_web/components/homepage/footer-new.tsx
View file

@ -1,188 +1,180 @@
import {
IconBrandDiscord,
IconBrandGithub,
IconBrandLinkedin,
IconBrandTwitter,
} from "@tabler/icons-react";
import Image from "next/image";
import Link from "next/link";
import React from "react";
import {
IconBrandTwitter,
IconBrandLinkedin,
IconBrandGithub,
IconBrandDiscord,
} from "@tabler/icons-react";
import { Logo } from "@/components/Logo";
export function FooterNew() {
const pages = [
// {
// title: "All Products",
// href: "#",
// },
// {
// title: "Studio",
// href: "#",
// },
// {
// title: "Clients",
// href: "#",
// },
{
title: "Pricing",
href: "/pricing",
},
{
title: "Docs",
href: "/docs",
},
// {
// title: "Blog",
// href: "#",
// },
];
const pages = [
// {
// title: "All Products",
// href: "#",
// },
// {
// title: "Studio",
// href: "#",
// },
// {
// title: "Clients",
// href: "#",
// },
{
title: "Pricing",
href: "/pricing",
},
{
title: "Docs",
href: "/docs",
},
// {
// title: "Blog",
// href: "#",
// },
];
const socials = [
{
title: "Twitter",
href: "https://x.com/mod_setter",
icon: IconBrandTwitter,
},
{
title: "LinkedIn",
href: "https://www.linkedin.com/in/rohan-verma-sde/",
icon: IconBrandLinkedin,
},
{
title: "GitHub",
href: "https://github.com/MODSetter",
icon: IconBrandGithub,
},
{
title: "Discord",
href: "https://discord.gg/ejRNvftDp9",
icon: IconBrandDiscord,
},
];
const legals = [
{
title: "Privacy Policy",
href: "/privacy",
},
{
title: "Terms of Service",
href: "/terms",
},
// {
// title: "Cookie Policy",
// href: "#",
// },
];
const socials = [
{
title: "Twitter",
href: "https://x.com/mod_setter",
icon: IconBrandTwitter,
},
{
title: "LinkedIn",
href: "https://www.linkedin.com/in/rohan-verma-sde/",
icon: IconBrandLinkedin,
},
{
title: "GitHub",
href: "https://github.com/MODSetter",
icon: IconBrandGithub,
},
{
title: "Discord",
href: "https://discord.gg/ejRNvftDp9",
icon: IconBrandDiscord,
},
];
const legals = [
{
title: "Privacy Policy",
href: "/privacy",
},
{
title: "Terms of Service",
href: "/terms",
},
// {
// title: "Cookie Policy",
// href: "#",
// },
];
const signups = [
{
title: "Sign In",
href: "/login",
},
// {
// title: "Login",
// href: "#",
// },
// {
// title: "Forgot Password",
// href: "#",
// },
];
return (
<div className="border-t border-neutral-100 dark:border-white/[0.1] px-8 py-20 bg-white dark:bg-neutral-950 w-full relative overflow-hidden">
<div className="max-w-7xl mx-auto text-sm text-neutral-500 flex sm:flex-row flex-col justify-between items-start md:px-8">
<div>
<div className="mr-0 md:mr-4 md:flex mb-4">
<Logo className="h-6 w-6 rounded-md mr-2" />
<span className="dark:text-white/90 text-gray-800 text-lg font-bold">SurfSense</span>
</div>
const signups = [
{
title: "Sign In",
href: "/login",
},
// {
// title: "Login",
// href: "#",
// },
// {
// title: "Forgot Password",
// href: "#",
// },
];
return (
<div className="border-t border-neutral-100 dark:border-white/[0.1] px-8 py-20 bg-white dark:bg-neutral-950 w-full relative overflow-hidden">
<div className="max-w-7xl mx-auto text-sm text-neutral-500 flex sm:flex-row flex-col justify-between items-start md:px-8">
<div>
<div className="mr-0 md:mr-4 md:flex mb-4">
<Logo className="h-6 w-6 rounded-md mr-2" />
<span className="dark:text-white/90 text-gray-800 text-lg font-bold">SurfSense</span>
</div>
<div className="mt-2 ml-2">
&copy; SurfSense 2025. All rights reserved.
</div>
</div>
<div className="grid grid-cols-2 lg:grid-cols-4 gap-10 items-start mt-10 sm:mt-0 md:mt-0">
<div className="flex justify-center space-y-4 flex-col w-full">
<p className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 font-bold">
Pages
</p>
<ul className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 list-none space-y-4">
{pages.map((page, idx) => (
<li key={"pages" + idx} className="list-none">
<Link
className="transition-colors hover:text-text-neutral-800 "
href={page.href}
>
{page.title}
</Link>
</li>
))}
</ul>
</div>
<div className="mt-2 ml-2">&copy; SurfSense 2025. All rights reserved.</div>
</div>
<div className="grid grid-cols-2 lg:grid-cols-4 gap-10 items-start mt-10 sm:mt-0 md:mt-0">
<div className="flex justify-center space-y-4 flex-col w-full">
<p className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 font-bold">
Pages
</p>
<ul className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 list-none space-y-4">
{pages.map((page, idx) => (
<li key={"pages" + idx} className="list-none">
<Link className="transition-colors hover:text-text-neutral-800 " href={page.href}>
{page.title}
</Link>
</li>
))}
</ul>
</div>
<div className="flex justify-center space-y-4 flex-col">
<p className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 font-bold">
Socials
</p>
<ul className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 list-none space-y-4">
{socials.map((social, idx) => {
const Icon = social.icon;
return (
<li key={"social" + idx} className="list-none">
<Link
className="transition-colors hover:text-text-neutral-800 flex items-center gap-2"
href={social.href}
target="_blank"
rel="noopener noreferrer"
>
<Icon className="h-5 w-5" />
{social.title}
</Link>
</li>
);
})}
</ul>
</div>
<div className="flex justify-center space-y-4 flex-col">
<p className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 font-bold">
Socials
</p>
<ul className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 list-none space-y-4">
{socials.map((social, idx) => {
const Icon = social.icon;
return (
<li key={"social" + idx} className="list-none">
<Link
className="transition-colors hover:text-text-neutral-800 flex items-center gap-2"
href={social.href}
target="_blank"
rel="noopener noreferrer"
>
<Icon className="h-5 w-5" />
{social.title}
</Link>
</li>
);
})}
</ul>
</div>
<div className="flex justify-center space-y-4 flex-col">
<p className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 font-bold">
Legal
</p>
<ul className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 list-none space-y-4">
{legals.map((legal, idx) => (
<li key={"legal" + idx} className="list-none">
<Link
className="transition-colors hover:text-text-neutral-800 "
href={legal.href}
>
{legal.title}
</Link>
</li>
))}
</ul>
</div>
<div className="flex justify-center space-y-4 flex-col">
<p className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 font-bold">
Register
</p>
<ul className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 list-none space-y-4">
{signups.map((auth, idx) => (
<li key={"auth" + idx} className="list-none">
<Link
className="transition-colors hover:text-text-neutral-800 "
href={auth.href}
>
{auth.title}
</Link>
</li>
))}
</ul>
</div>
</div>
</div>
<p className="text-center mt-20 text-5xl md:text-9xl lg:text-[12rem] xl:text-[13rem] font-bold bg-clip-text text-transparent bg-gradient-to-b from-neutral-50 dark:from-neutral-950 to-neutral-200 dark:to-neutral-800 inset-x-0">
SurfSense
</p>
</div>
);
<div className="flex justify-center space-y-4 flex-col">
<p className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 font-bold">
Legal
</p>
<ul className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 list-none space-y-4">
{legals.map((legal, idx) => (
<li key={"legal" + idx} className="list-none">
<Link
className="transition-colors hover:text-text-neutral-800 "
href={legal.href}
>
{legal.title}
</Link>
</li>
))}
</ul>
</div>
<div className="flex justify-center space-y-4 flex-col">
<p className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 font-bold">
Register
</p>
<ul className="transition-colors hover:text-text-neutral-800 text-neutral-600 dark:text-neutral-300 list-none space-y-4">
{signups.map((auth, idx) => (
<li key={"auth" + idx} className="list-none">
<Link className="transition-colors hover:text-text-neutral-800 " href={auth.href}>
{auth.title}
</Link>
</li>
))}
</ul>
</div>
</div>
</div>
<p className="text-center mt-20 text-5xl md:text-9xl lg:text-[12rem] xl:text-[13rem] font-bold bg-clip-text text-transparent bg-gradient-to-b from-neutral-50 dark:from-neutral-950 to-neutral-200 dark:to-neutral-800 inset-x-0">
SurfSense
</p>
</div>
);
}

64
surfsense_web/components/onboard/add-provider-step.tsx
View file

@ -2,9 +2,9 @@
import { AlertCircle, Bot, Plus, Trash2 } from "lucide-react";
import { motion } from "motion/react";
import { useTranslations } from "next-intl";
import { useState } from "react";
import { toast } from "sonner";
import { useTranslations } from "next-intl";
import { Alert, AlertDescription } from "@/components/ui/alert";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
@ -35,7 +35,7 @@ export function AddProviderStep({
onConfigCreated,
onConfigDeleted,
}: AddProviderStepProps) {
const t = useTranslations('onboard');
const t = useTranslations("onboard");
const { llmConfigs, createLLMConfig, deleteLLMConfig } = useLLMConfigs(searchSpaceId);
const [isAddingNew, setIsAddingNew] = useState(false);
const [formData, setFormData] = useState<CreateLLMConfig>({
@ -95,15 +95,13 @@ export function AddProviderStep({
{/* Info Alert */}
<Alert>
<AlertCircle className="h-4 w-4" />
<AlertDescription>
{t('add_provider_instruction')}
</AlertDescription>
<AlertDescription>{t("add_provider_instruction")}</AlertDescription>
</Alert>
{/* Existing Configurations */}
{llmConfigs.length > 0 && (
<div className="space-y-4">
<h3 className="text-lg font-semibold">{t('your_llm_configs')}</h3>
<h3 className="text-lg font-semibold">{t("your_llm_configs")}</h3>
<div className="grid gap-4">
{llmConfigs.map((config) => (
<motion.div
@ -122,9 +120,9 @@ export function AddProviderStep({
<Badge variant="secondary">{config.provider}</Badge>
</div>
<p className="text-sm text-muted-foreground">
{t('model')}: {config.model_name}
{config.language && ` • ${t('language')}: ${config.language}`}
{config.api_base && ` • ${t('base')}: ${config.api_base}`}
{t("model")}: {config.model_name}
{config.language && ` • ${t("language")}: ${config.language}`}
{config.api_base && ` • ${t("base")}: ${config.api_base}`}
</p>
</div>
<Button
@ -154,32 +152,28 @@ export function AddProviderStep({
<Card className="border-dashed border-2 hover:border-primary/50 transition-colors">
<CardContent className="flex flex-col items-center justify-center py-12">
<Plus className="w-12 h-12 text-muted-foreground mb-4" />
<h3 className="text-lg font-semibold mb-2">{t('add_provider_title')}</h3>
<p className="text-muted-foreground text-center mb-4">
{t('add_provider_subtitle')}
</p>
<h3 className="text-lg font-semibold mb-2">{t("add_provider_title")}</h3>
<p className="text-muted-foreground text-center mb-4">{t("add_provider_subtitle")}</p>
<Button onClick={() => setIsAddingNew(true)}>
<Plus className="w-4 h-4 mr-2" />
{t('add_provider_button')}
{t("add_provider_button")}
</Button>
</CardContent>
</Card>
) : (
<Card>
<CardHeader>
<CardTitle>{t('add_new_llm_provider')}</CardTitle>
<CardDescription>
{t('configure_new_provider')}
</CardDescription>
<CardTitle>{t("add_new_llm_provider")}</CardTitle>
<CardDescription>{t("configure_new_provider")}</CardDescription>
</CardHeader>
<CardContent>
<form onSubmit={handleSubmit} className="space-y-4">
<div className="grid grid-cols-1 md:grid-cols-3 gap-4">
<div className="space-y-2">
<Label htmlFor="name">{t('config_name_required')}</Label>
<Label htmlFor="name">{t("config_name_required")}</Label>
<Input
id="name"
placeholder={t('config_name_placeholder')}
placeholder={t("config_name_placeholder")}
value={formData.name}
onChange={(e) => handleInputChange("name", e.target.value)}
required
@ -187,13 +181,13 @@ export function AddProviderStep({
</div>
<div className="space-y-2">
<Label htmlFor="provider">{t('provider_required')}</Label>
<Label htmlFor="provider">{t("provider_required")}</Label>
<Select
value={formData.provider}
onValueChange={(value) => handleInputChange("provider", value)}
>
<SelectTrigger>
<SelectValue placeholder={t('provider_placeholder')} />
<SelectValue placeholder={t("provider_placeholder")} />
</SelectTrigger>
<SelectContent>
{LLM_PROVIDERS.map((provider) => (
@ -207,13 +201,13 @@ export function AddProviderStep({
{/* language */}
<div className="space-y-2">
<Label htmlFor="language">{t('language_optional')}</Label>
<Label htmlFor="language">{t("language_optional")}</Label>
<Select
value={formData.language || "English"}
onValueChange={(value) => handleInputChange("language", value)}
>
<SelectTrigger>
<SelectValue placeholder={t('language_placeholder')} />
<SelectValue placeholder={t("language_placeholder")} />
</SelectTrigger>
<SelectContent>
{LANGUAGES.map((language) => (
@ -228,10 +222,10 @@ export function AddProviderStep({
{formData.provider === "CUSTOM" && (
<div className="space-y-2">
<Label htmlFor="custom_provider">{t('custom_provider_name')}</Label>
<Label htmlFor="custom_provider">{t("custom_provider_name")}</Label>
<Input
id="custom_provider"
placeholder={t('custom_provider_placeholder')}
placeholder={t("custom_provider_placeholder")}
value={formData.custom_provider}
onChange={(e) => handleInputChange("custom_provider", e.target.value)}
required
@ -240,27 +234,27 @@ export function AddProviderStep({
)}
<div className="space-y-2">
<Label htmlFor="model_name">{t('model_name_required')}</Label>
<Label htmlFor="model_name">{t("model_name_required")}</Label>
<Input
id="model_name"
placeholder={selectedProvider?.example || t('model_name_placeholder')}
placeholder={selectedProvider?.example || t("model_name_placeholder")}
value={formData.model_name}
onChange={(e) => handleInputChange("model_name", e.target.value)}
required
/>
{selectedProvider && (
<p className="text-xs text-muted-foreground">
{t('examples')}: {selectedProvider.example}
{t("examples")}: {selectedProvider.example}
</p>
)}
</div>
<div className="space-y-2">
<Label htmlFor="api_key">{t('api_key_required')}</Label>
<Label htmlFor="api_key">{t("api_key_required")}</Label>
<Input
id="api_key"
type="password"
placeholder={t('api_key_placeholder')}
placeholder={t("api_key_placeholder")}
value={formData.api_key}
onChange={(e) => handleInputChange("api_key", e.target.value)}
required
@ -268,10 +262,10 @@ export function AddProviderStep({
</div>
<div className="space-y-2">
<Label htmlFor="api_base">{t('api_base_optional')}</Label>
<Label htmlFor="api_base">{t("api_base_optional")}</Label>
<Input
id="api_base"
placeholder={t('api_base_placeholder')}
placeholder={t("api_base_placeholder")}
value={formData.api_base}
onChange={(e) => handleInputChange("api_base", e.target.value)}
/>
@ -287,7 +281,7 @@ export function AddProviderStep({
<div className="flex gap-2 pt-4">
<Button type="submit" disabled={isSubmitting}>
{isSubmitting ? t('adding') : t('add_provider')}
{isSubmitting ? t("adding") : t("add_provider")}
</Button>
<Button
type="button"
@ -295,7 +289,7 @@ export function AddProviderStep({
onClick={() => setIsAddingNew(false)}
disabled={isSubmitting}
>
{t('cancel')}
{t("cancel")}
</Button>
</div>
</form>

50
surfsense_web/components/onboard/assign-roles-step.tsx
View file

@ -2,8 +2,8 @@
import { AlertCircle, Bot, Brain, CheckCircle, Zap } from "lucide-react";
import { motion } from "motion/react";
import { useEffect, useState } from "react";
import { useTranslations } from "next-intl";
import { useEffect, useState } from "react";
import { Alert, AlertDescription } from "@/components/ui/alert";
import { Badge } from "@/components/ui/badge";
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card";
@ -23,31 +23,31 @@ interface AssignRolesStepProps {
}
export function AssignRolesStep({ searchSpaceId, onPreferencesUpdated }: AssignRolesStepProps) {
const t = useTranslations('onboard');
const t = useTranslations("onboard");
const { llmConfigs } = useLLMConfigs(searchSpaceId);
const { preferences, updatePreferences } = useLLMPreferences(searchSpaceId);
const ROLE_DESCRIPTIONS = {
long_context: {
icon: Brain,
title: t('long_context_llm_title'),
description: t('long_context_llm_desc'),
title: t("long_context_llm_title"),
description: t("long_context_llm_desc"),
color: "bg-blue-100 text-blue-800 border-blue-200",
examples: t('long_context_llm_examples'),
examples: t("long_context_llm_examples"),
},
fast: {
icon: Zap,
title: t('fast_llm_title'),
description: t('fast_llm_desc'),
title: t("fast_llm_title"),
description: t("fast_llm_desc"),
color: "bg-green-100 text-green-800 border-green-200",
examples: t('fast_llm_examples'),
examples: t("fast_llm_examples"),
},
strategic: {
icon: Bot,
title: t('strategic_llm_title'),
description: t('strategic_llm_desc'),
title: t("strategic_llm_title"),
description: t("strategic_llm_desc"),
color: "bg-purple-100 text-purple-800 border-purple-200",
examples: t('strategic_llm_examples'),
examples: t("strategic_llm_examples"),
},
};
@ -111,10 +111,8 @@ export function AssignRolesStep({ searchSpaceId, onPreferencesUpdated }: AssignR
return (
<div className="flex flex-col items-center justify-center py-12">
<AlertCircle className="w-16 h-16 text-muted-foreground mb-4" />
<h3 className="text-lg font-semibold mb-2">{t('no_llm_configs_found')}</h3>
<p className="text-muted-foreground text-center">
{t('add_provider_before_roles')}
</p>
<h3 className="text-lg font-semibold mb-2">{t("no_llm_configs_found")}</h3>
<p className="text-muted-foreground text-center">{t("add_provider_before_roles")}</p>
</div>
);
}
@ -124,9 +122,7 @@ export function AssignRolesStep({ searchSpaceId, onPreferencesUpdated }: AssignR
{/* Info Alert */}
<Alert>
<AlertCircle className="h-4 w-4" />
<AlertDescription>
{t('assign_roles_instruction')}
</AlertDescription>
<AlertDescription>{t("assign_roles_instruction")}</AlertDescription>
</Alert>
{/* Role Assignment Cards */}
@ -162,17 +158,17 @@ export function AssignRolesStep({ searchSpaceId, onPreferencesUpdated }: AssignR
</CardHeader>
<CardContent className="space-y-4">
<div className="text-sm text-muted-foreground">
<strong>{t('use_cases')}:</strong> {role.examples}
<strong>{t("use_cases")}:</strong> {role.examples}
</div>
<div className="space-y-2">
<Label className="text-sm font-medium">{t('assign_llm_config')}:</Label>
<Label className="text-sm font-medium">{t("assign_llm_config")}:</Label>
<Select
value={currentAssignment?.toString() || ""}
onValueChange={(value) => handleRoleAssignment(`${key}_llm_id`, value)}
>
<SelectTrigger>
<SelectValue placeholder={t('select_llm_config')} />
<SelectValue placeholder={t("select_llm_config")} />
</SelectTrigger>
<SelectContent>
{llmConfigs
@ -196,12 +192,12 @@ export function AssignRolesStep({ searchSpaceId, onPreferencesUpdated }: AssignR
<div className="mt-3 p-3 bg-muted/50 rounded-lg">
<div className="flex items-center gap-2 text-sm">
<Bot className="w-4 h-4" />
<span className="font-medium">{t('assigned')}:</span>
<span className="font-medium">{t("assigned")}:</span>
<Badge variant="secondary">{assignedConfig.provider}</Badge>
<span>{assignedConfig.name}</span>
</div>
<div className="text-xs text-muted-foreground mt-1">
{t('model')}: {assignedConfig.model_name}
{t("model")}: {assignedConfig.model_name}
</div>
</div>
)}
@ -217,7 +213,7 @@ export function AssignRolesStep({ searchSpaceId, onPreferencesUpdated }: AssignR
<div className="flex justify-center pt-4">
<div className="flex items-center gap-2 px-4 py-2 bg-green-50 text-green-700 rounded-lg border border-green-200">
<CheckCircle className="w-4 h-4" />
<span className="text-sm font-medium">{t('all_roles_assigned_saved')}</span>
<span className="text-sm font-medium">{t("all_roles_assigned_saved")}</span>
</div>
</div>
)}
@ -225,7 +221,7 @@ export function AssignRolesStep({ searchSpaceId, onPreferencesUpdated }: AssignR
{/* Progress Indicator */}
<div className="flex justify-center">
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<span>{t('progress')}:</span>
<span>{t("progress")}:</span>
<div className="flex gap-1">
{Object.keys(ROLE_DESCRIPTIONS).map((key, _index) => (
<div
@ -239,9 +235,9 @@ export function AssignRolesStep({ searchSpaceId, onPreferencesUpdated }: AssignR
))}
</div>
<span>
{t('roles_assigned', {
{t("roles_assigned", {
assigned: Object.values(assignments).filter(Boolean).length,
total: Object.keys(ROLE_DESCRIPTIONS).length
total: Object.keys(ROLE_DESCRIPTIONS).length,
})}
</span>
</div>

19
surfsense_web/components/providers/I18nProvider.tsx
View file

@ -1,19 +1,18 @@
'use client';
"use client";
import { NextIntlClientProvider } from 'next-intl';
import { useLocaleContext } from '@/contexts/LocaleContext';
import { NextIntlClientProvider } from "next-intl";
import { useLocaleContext } from "@/contexts/LocaleContext";
/**
* I18n Provider component
* Wraps NextIntlClientProvider with dynamic locale and messages from LocaleContext
*/
export function I18nProvider({ children }: { children: React.ReactNode }) {
const { locale, messages } = useLocaleContext();
const { locale, messages } = useLocaleContext();
return (
<NextIntlClientProvider messages={messages} locale={locale}>
{children}
</NextIntlClientProvider>
);
return (
<NextIntlClientProvider messages={messages} locale={locale}>
{children}
</NextIntlClientProvider>
);
}

40
surfsense_web/components/sidebar/AppSidebarProvider.tsx
View file

@ -1,8 +1,8 @@
"use client";
import { Trash2 } from "lucide-react";
import { useCallback, useEffect, useMemo, useState } from "react";
import { useTranslations } from "next-intl";
import { useCallback, useEffect, useMemo, useState } from "react";
import { AppSidebar } from "@/components/sidebar/app-sidebar";
import { Button } from "@/components/ui/button";
import {
@ -56,8 +56,8 @@ export function AppSidebarProvider({
navSecondary,
navMain,
}: AppSidebarProviderProps) {
const t = useTranslations('dashboard');
const tCommon = useTranslations('common');
const t = useTranslations("dashboard");
const tCommon = useTranslations("common");
const [recentChats, setRecentChats] = useState<
{
name: string;
@ -199,14 +199,14 @@ export function AppSidebarProvider({
if (chatError) {
return [
{
name: t('error_loading_chats'),
name: t("error_loading_chats"),
url: "#",
icon: "AlertCircle",
id: 0,
search_space_id: Number(searchSpaceId),
actions: [
{
name: tCommon('retry'),
name: tCommon("retry"),
icon: "RefreshCw",
onClick: retryFetch,
},
@ -218,7 +218,7 @@ export function AppSidebarProvider({
if (!isLoadingChats && recentChats.length === 0) {
return [
{
name: t('no_recent_chats'),
name: t("no_recent_chats"),
url: "#",
icon: "MessageCircleMore",
id: 0,
@ -243,14 +243,22 @@ export function AppSidebarProvider({
title:
searchSpace?.name ||
(isLoadingSearchSpace
? tCommon('loading')
? tCommon("loading")
: searchSpaceError
? t('error_loading_space')
: t('unknown_search_space')),
? t("error_loading_space")
: t("unknown_search_space")),
};
}
return updated;
}, [navSecondary, isClient, searchSpace?.name, isLoadingSearchSpace, searchSpaceError, t, tCommon]);
}, [
navSecondary,
isClient,
searchSpace?.name,
isLoadingSearchSpace,
searchSpaceError,
t,
tCommon,
]);
// Show loading state if not client-side
if (!isClient) {
@ -267,11 +275,11 @@ export function AppSidebarProvider({
<DialogHeader>
<DialogTitle className="flex items-center gap-2">
<Trash2 className="h-5 w-5 text-destructive" />
<span>{t('delete_chat')}</span>
<span>{t("delete_chat")}</span>
</DialogTitle>
<DialogDescription>
{t('delete_chat_confirm')}{" "}
<span className="font-medium">{chatToDelete?.name}</span>? {t('action_cannot_undone')}
{t("delete_chat_confirm")} <span className="font-medium">{chatToDelete?.name}</span>?{" "}
{t("action_cannot_undone")}
</DialogDescription>
</DialogHeader>
<DialogFooter className="flex gap-2 sm:justify-end">
@ -280,7 +288,7 @@ export function AppSidebarProvider({
onClick={() => setShowDeleteDialog(false)}
disabled={isDeleting}
>
{tCommon('cancel')}
{tCommon("cancel")}
</Button>
<Button
variant="destructive"
@ -291,12 +299,12 @@ export function AppSidebarProvider({
{isDeleting ? (
<>
<span className="h-4 w-4 animate-spin rounded-full border-2 border-current border-t-transparent" />
{t('deleting')}
{t("deleting")}
</>
) : (
<>
<Trash2 className="h-4 w-4" />
{tCommon('delete')}
{tCommon("delete")}
</>
)}
</Button>

38
surfsense_web/components/sidebar/nav-main.tsx
View file

@ -1,8 +1,8 @@
"use client";
import { ChevronRight, type LucideIcon } from "lucide-react";
import { useMemo } from "react";
import { useTranslations } from "next-intl";
import { useMemo } from "react";
import { Collapsible, CollapsibleContent, CollapsibleTrigger } from "@/components/ui/collapsible";
import {
@ -29,36 +29,36 @@ interface NavItem {
}
export function NavMain({ items }: { items: NavItem[] }) {
const t = useTranslations('nav_menu');
const t = useTranslations("nav_menu");
// Translation function that handles both exact matches and fallback to original
const translateTitle = (title: string): string => {
const titleMap: Record<string, string> = {
'Researcher': 'researcher',
'Manage LLMs': 'manage_llms',
'Documents': 'documents',
'Upload Documents': 'upload_documents',
'Add Webpages': 'add_webpages',
'Add Youtube Videos': 'add_youtube',
'Manage Documents': 'manage_documents',
'Connectors': 'connectors',
'Add Connector': 'add_connector',
'Manage Connectors': 'manage_connectors',
'Podcasts': 'podcasts',
'Logs': 'logs',
'Platform': 'platform',
Researcher: "researcher",
"Manage LLMs": "manage_llms",
Documents: "documents",
"Upload Documents": "upload_documents",
"Add Webpages": "add_webpages",
"Add Youtube Videos": "add_youtube",
"Manage Documents": "manage_documents",
Connectors: "connectors",
"Add Connector": "add_connector",
"Manage Connectors": "manage_connectors",
Podcasts: "podcasts",
Logs: "logs",
Platform: "platform",
};
const key = titleMap[title];
return key ? t(key) : title;
};
// Memoize items to prevent unnecessary re-renders
const memoizedItems = useMemo(() => items, [items]);
return (
<SidebarGroup>
<SidebarGroupLabel>{translateTitle('Platform')}</SidebarGroupLabel>
<SidebarGroupLabel>{translateTitle("Platform")}</SidebarGroupLabel>
<SidebarMenu>
{memoizedItems.map((item, index) => {
const translatedTitle = translateTitle(item.title);

12
surfsense_web/components/sidebar/nav-projects.tsx
View file

@ -11,8 +11,8 @@ import {
Trash2,
} from "lucide-react";
import { useRouter } from "next/navigation";
import { useCallback, useMemo, useState } from "react";
import { useTranslations } from "next-intl";
import { useCallback, useMemo, useState } from "react";
import {
DropdownMenu,
DropdownMenuContent,
@ -57,7 +57,7 @@ interface ChatItem {
}
export function NavProjects({ chats }: { chats: ChatItem[] }) {
const t = useTranslations('sidebar');
const t = useTranslations("sidebar");
const { isMobile } = useSidebar();
const router = useRouter();
const [searchQuery, setSearchQuery] = useState("");
@ -147,13 +147,13 @@ export function NavProjects({ chats }: { chats: ChatItem[] }) {
return (
<SidebarGroup className="group-data-[collapsible=icon]:hidden">
<SidebarGroupLabel>{t('recent_chats')}</SidebarGroupLabel>
<SidebarGroupLabel>{t("recent_chats")}</SidebarGroupLabel>
{/* Search Input */}
{showSearch && (
<div className="px-2 pb-2">
<SidebarInput
placeholder={t('search_chats')}
placeholder={t("search_chats")}
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
className="h-8"
@ -170,7 +170,7 @@ export function NavProjects({ chats }: { chats: ChatItem[] }) {
<SidebarMenuItem>
<SidebarMenuButton disabled className="text-muted-foreground">
<Search className="h-4 w-4" />
<span>{searchQuery ? t('no_chats_found') : t('no_recent_chats')}</span>
<span>{searchQuery ? t("no_chats_found") : t("no_recent_chats")}</span>
</SidebarMenuButton>
</SidebarMenuItem>
)}
@ -180,7 +180,7 @@ export function NavProjects({ chats }: { chats: ChatItem[] }) {
<SidebarMenuItem>
<SidebarMenuButton onClick={() => router.push(`/dashboard/${searchSpaceId}/chats`)}>
<MoreHorizontal />
<span>{t('view_all_chats')}</span>
<span>{t("view_all_chats")}</span>
</SidebarMenuButton>
</SidebarMenuItem>
)}

8
surfsense_web/components/sidebar/nav-secondary.tsx
View file

@ -1,9 +1,9 @@
"use client";
import type { LucideIcon } from "lucide-react";
import { useTranslations } from "next-intl";
import type * as React from "react";
import { useMemo } from "react";
import { useTranslations } from "next-intl";
import {
SidebarGroup,
@ -25,14 +25,14 @@ export function NavSecondary({
}: {
items: NavSecondaryItem[];
} & React.ComponentPropsWithoutRef<typeof SidebarGroup>) {
const t = useTranslations('sidebar');
const t = useTranslations("sidebar");
// Memoize items to prevent unnecessary re-renders
const memoizedItems = useMemo(() => items, [items]);
return (
<SidebarGroup {...props}>
<SidebarGroupLabel>{t('search_space')}</SidebarGroupLabel>
<SidebarGroupLabel>{t("search_space")}</SidebarGroupLabel>
<SidebarMenu>
{memoizedItems.map((item, index) => (
<SidebarMenuItem key={`${item.title}-${index}`}>

102
surfsense_web/contexts/LocaleContext.tsx
View file

@ -1,70 +1,70 @@
'use client';
"use client";
import React, { createContext, useContext, useState, useEffect } from 'react';
import enMessages from '../messages/en.json';
import zhMessages from '../messages/zh.json';
import type React from "react";
import { createContext, useContext, useEffect, useState } from "react";
import enMessages from "../messages/en.json";
import zhMessages from "../messages/zh.json";
type Locale = 'en' | 'zh';
type Locale = "en" | "zh";
interface LocaleContextType {
locale: Locale;
messages: typeof enMessages;
setLocale: (locale: Locale) => void;
locale: Locale;
messages: typeof enMessages;
setLocale: (locale: Locale) => void;
}
const LocaleContext = createContext<LocaleContextType | undefined>(undefined);
const LOCALE_STORAGE_KEY = 'surfsense-locale';
const LOCALE_STORAGE_KEY = "surfsense-locale";
export function LocaleProvider({ children }: { children: React.ReactNode }) {
// Always start with 'en' to avoid hydration mismatch
// Then sync with localStorage after mount
const [locale, setLocaleState] = useState<Locale>('en');
const [mounted, setMounted] = useState(false);
// Always start with 'en' to avoid hydration mismatch
// Then sync with localStorage after mount
const [locale, setLocaleState] = useState<Locale>("en");
const [mounted, setMounted] = useState(false);
// Get messages based on current locale
const messages = locale === 'zh' ? zhMessages : enMessages;
// Get messages based on current locale
const messages = locale === "zh" ? zhMessages : enMessages;
// Load locale from localStorage after component mounts (client-side only)
useEffect(() => {
setMounted(true);
if (typeof window !== 'undefined') {
const stored = localStorage.getItem(LOCALE_STORAGE_KEY);
if (stored === 'zh') {
setLocaleState('zh');
}
}
}, []);
// Load locale from localStorage after component mounts (client-side only)
useEffect(() => {
setMounted(true);
if (typeof window !== "undefined") {
const stored = localStorage.getItem(LOCALE_STORAGE_KEY);
if (stored === "zh") {
setLocaleState("zh");
}
}
}, []);
// Update locale and persist to localStorage
const setLocale = (newLocale: Locale) => {
setLocaleState(newLocale);
if (typeof window !== 'undefined') {
localStorage.setItem(LOCALE_STORAGE_KEY, newLocale);
// Update HTML lang attribute
document.documentElement.lang = newLocale;
}
};
// Update locale and persist to localStorage
const setLocale = (newLocale: Locale) => {
setLocaleState(newLocale);
if (typeof window !== "undefined") {
localStorage.setItem(LOCALE_STORAGE_KEY, newLocale);
// Update HTML lang attribute
document.documentElement.lang = newLocale;
}
};
// Set HTML lang attribute when locale changes
useEffect(() => {
if (typeof window !== 'undefined' && mounted) {
document.documentElement.lang = locale;
}
}, [locale, mounted]);
// Set HTML lang attribute when locale changes
useEffect(() => {
if (typeof window !== "undefined" && mounted) {
document.documentElement.lang = locale;
}
}, [locale, mounted]);
return (
<LocaleContext.Provider value={{ locale, messages, setLocale }}>
{children}
</LocaleContext.Provider>
);
return (
<LocaleContext.Provider value={{ locale, messages, setLocale }}>
{children}
</LocaleContext.Provider>
);
}
export function useLocaleContext() {
const context = useContext(LocaleContext);
if (context === undefined) {
throw new Error('useLocaleContext must be used within a LocaleProvider');
}
return context;
const context = useContext(LocaleContext);
if (context === undefined) {
throw new Error("useLocaleContext must be used within a LocaleProvider");
}
return context;
}

27
surfsense_web/i18n/request.ts
View file

@ -1,22 +1,21 @@
import {getRequestConfig} from 'next-intl/server';
import {routing} from './routing';
import { getRequestConfig } from "next-intl/server";
import { routing } from "./routing";
/**
* Configuration for internationalization request handling
* This function is called for each request to determine the locale and load translations
*/
export default getRequestConfig(async ({requestLocale}) => {
// This typically corresponds to the `[locale]` segment
let locale = await requestLocale;
export default getRequestConfig(async ({ requestLocale }) => {
// This typically corresponds to the `[locale]` segment
let locale = await requestLocale;
// Ensure that the incoming `locale` is valid
if (!locale || !routing.locales.includes(locale as any)) {
locale = routing.defaultLocale;
}
// Ensure that the incoming `locale` is valid
if (!locale || !routing.locales.includes(locale as any)) {
locale = routing.defaultLocale;
}
return {
locale,
messages: (await import(`../messages/${locale}.json`)).default
};
return {
locale,
messages: (await import(`../messages/${locale}.json`)).default,
};
});

22
surfsense_web/i18n/routing.ts
View file

@ -1,24 +1,22 @@
import {defineRouting} from 'next-intl/routing';
import {createNavigation} from 'next-intl/navigation';
import { createNavigation } from "next-intl/navigation";
import { defineRouting } from "next-intl/routing";
/**
* Internationalization routing configuration
* Defines supported locales and routing behavior for the application
*/
export const routing = defineRouting({
// A list of all locales that are supported
locales: ['en', 'zh'],
// A list of all locales that are supported
locales: ["en", "zh"],
// Used when no locale matches
defaultLocale: 'en',
// Used when no locale matches
defaultLocale: "en",
// The `localePrefix` setting controls whether the locale is included in the pathname
// 'as-needed': Only add locale prefix when not using the default locale
localePrefix: 'as-needed'
// The `localePrefix` setting controls whether the locale is included in the pathname
// 'as-needed': Only add locale prefix when not using the default locale
localePrefix: "as-needed",
});
// Lightweight wrappers around Next.js' navigation APIs
// that will consider the routing configuration
export const {Link, redirect, usePathname, useRouter, getPathname} =
createNavigation(routing);
export const { Link, redirect, usePathname, useRouter, getPathname } = createNavigation(routing);

1379
surfsense_web/messages/en.json
View file

File diff suppressed because it is too large Load diff

1379
surfsense_web/messages/zh.json
View file

File diff suppressed because it is too large Load diff

7
surfsense_web/middleware.ts
View file

@ -2,11 +2,10 @@
// Server-side i18n routing would require restructuring entire app directory to app/[locale]/...
// which is too invasive for this project
import { NextResponse } from 'next/server';
import type { NextRequest } from 'next/server';
import type { NextRequest } from "next/server";
import { NextResponse } from "next/server";
// Empty middleware - just pass through all requests
export function middleware(request: NextRequest) {
return NextResponse.next();
return NextResponse.next();
}

4
surfsense_web/next.config.ts
View file

@ -1,9 +1,9 @@
import { createMDX } from "fumadocs-mdx/next";
import type { NextConfig } from "next";
import createNextIntlPlugin from 'next-intl/plugin';
import createNextIntlPlugin from "next-intl/plugin";
// Create the next-intl plugin
const withNextIntl = createNextIntlPlugin('./i18n/request.ts');
const withNextIntl = createNextIntlPlugin("./i18n/request.ts");
const nextConfig: NextConfig = {
output: "standalone",