Merge remote-tracking branch 'upstream/dev' into fix/chatpage-ux

README.zh-CN.md

@@ -16,7 +16,7 @@
 
 # SurfSense
 
-虽然像 NotebookLM 和 Perplexity 这样的工具在对任何主题/查询进行研究时令人印象深刻且非常有效，但 SurfSense 通过与您的个人知识库集成，将这一能力提升到了新的高度。它是一个高度可定制的 AI 研究助手，可以连接外部数据源，如搜索引擎（SearxNG、Tavily、LinkUp）、Slack、Linear、Jira、ClickUp、Confluence、Gmail、Notion、YouTube、GitHub、Discord、Airtable、Google Calendar、Luma、Elasticsearch 等，未来还会支持更多。
+虽然像 NotebookLM 和 Perplexity 这样的工具在对任何主题/查询进行研究时令人印象深刻且非常有效，但 SurfSense 通过与您的个人知识库集成，将这一能力提升到了新的高度。它是一个高度可定制的 AI 研究助手，可以连接外部数据源，如搜索引擎（SearxNG、Tavily、LinkUp）、Slack、Linear、Jira、ClickUp、Confluence、BookStack、Gmail、Notion、YouTube、GitHub、Discord、Airtable、Google Calendar、Luma、Elasticsearch 等，未来还会支持更多。
 
 <div align="center">
 <a href="https://trendshift.io/repositories/13606" target="_blank"><img src="https://trendshift.io/api/badge/repositories/13606" alt="MODSetter%2FSurfSense | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
@@ -25,8 +25,7 @@
 
 # 视频演示
 
-
+https://github.com/user-attachments/assets/42a29ea1-d4d8-4213-9c69-972b5b806d58
-https://github.com/user-attachments/assets/d9221908-e0de-4b2f-ac3a-691cf4b202da
 
 
 ## 播客示例
@@ -71,6 +70,27 @@ https://github.com/user-attachments/assets/a0a16566-6967-4374-ac51-9b3e07fbecd7
 - 支持本地 TTS 提供商（Kokoro TTS）
 - 支持多个 TTS 提供商（OpenAI、Azure、Google Vertex AI）
 
+### 🤖 **深度代理架构**
+
+#### 内置代理工具
+| 工具 | 描述 |
+|------|------|
+| **search_knowledge_base** | 使用语义+全文混合搜索、日期过滤和连接器特定查询搜索您的个人知识库 |
+| **generate_podcast** | 从聊天对话或知识库内容生成音频播客 |
+| **link_preview** | 获取 URL 的 Open Graph 元数据以显示预览卡片 |
+| **display_image** | 在聊天中显示带有元数据和来源归属的图像 |
+| **scrape_webpage** | 从网页中提取完整内容用于分析和总结（支持 Firecrawl 或本地 Chromium/Trafilatura） |
+
+#### 可扩展工具注册表
+贡献者可以通过注册表模式轻松添加新工具：
+1. 在 `surfsense_backend/app/agents/new_chat/tools/` 中创建工具工厂函数
+2. 在 `registry.py` 的 `BUILTIN_TOOLS` 列表中注册
+
+#### 可配置的系统提示词
+- 通过 LLM 配置自定义系统指令
+- 按配置切换引用开关
+- 通过 LiteLLM 集成支持 100+ 种 LLM
+
 ### 📊 **先进的 RAG 技术**
 - 支持 100+ 种大语言模型
 - 支持 6000+ 种嵌入模型
@@ -86,6 +106,7 @@ https://github.com/user-attachments/assets/a0a16566-6967-4374-ac51-9b3e07fbecd7
 - Jira
 - ClickUp
 - Confluence
+- BookStack
 - Notion
 - Gmail
 - YouTube 视频
@@ -214,32 +235,6 @@ Docker 和手动安装指南都包含适用于 Windows、macOS 和 Linux 的详
   - LlamaIndex API 密钥（增强解析，支持 50+ 种格式）
 - 其他根据用例需要的 API 密钥
 
-## 截图
-
-**研究助手** 
-
-![updated_researcher](https://github.com/user-attachments/assets/e22c5d86-f511-4c72-8c50-feba0c1561b4)
-
-**搜索空间** 
-
-![search_spaces](https://github.com/user-attachments/assets/e254c38c-f937-44b6-9e9d-770db583d099)
-
-**管理文档** 
-![documents](https://github.com/user-attachments/assets/7001e306-eb06-4009-89c6-8fadfdc3fc4d)
-
-**播客助手** 
-![podcasts](https://github.com/user-attachments/assets/6cb82ffd-9e14-4172-bc79-67faf34c4c1c)
-
-
-**对话助手** 
-
-![git_chat](https://github.com/user-attachments/assets/bb352d52-1c6d-4020-926b-722d0b98b491)
-
-**浏览器扩展**
-
-![ext1](https://github.com/user-attachments/assets/1f042b7a-6349-422b-94fb-d40d0df16c40)
-
-![ext2](https://github.com/user-attachments/assets/a9b9f1aa-2677-404d-b0a0-c1b2dddf24a7)
 
 
 ## 技术栈
@@ -257,11 +252,13 @@ Docker 和手动安装指南都包含适用于 Windows、macOS 和 Linux 的详
 
 -  **FastAPI Users**：使用 JWT 和 OAuth 支持的身份验证和用户管理
 
--  **LangGraph**：用于开发 AI 代理的框架
+-  **深度代理**：基于 LangGraph 构建的自定义代理框架，用于推理和行动的 AI 代理，支持可配置工具
-  
+
+-  **LangGraph**：用于开发具有对话持久性的有状态 AI 代理的框架
+
 -  **LangChain**：用于开发 AI 驱动应用程序的框架
 
--  **LLM 集成**：通过 LiteLLM 与大语言模型集成
+-  **LiteLLM**：通用 LLM 集成，支持 100+ 种模型（OpenAI、Anthropic、Ollama 等）
 
 -  **Rerankers**：先进的结果排序，提高搜索相关性
 
@@ -285,33 +282,19 @@ Docker 和手动安装指南都包含适用于 Windows、macOS 和 Linux 的详
 ---
  ### **前端**
 
--  **Next.js 15.2.3**：React 框架，具有应用路由器、服务器组件、自动代码拆分和优化渲染功能
+-  **Next.js**：React 框架，具有应用路由器、服务器组件、自动代码拆分和优化渲染功能
 
--  **React 19.0.0**：用于构建用户界面的 JavaScript 库
+-  **React**：用于构建用户界面的 JavaScript 库
 
 -  **TypeScript**：JavaScript 的静态类型检查，提升代码质量和开发体验
 
 - **Vercel AI SDK Kit UI Stream Protocol**：创建可扩展的聊天 UI
 
--  **Tailwind CSS 4.x**：实用优先的 CSS 框架，用于构建自定义 UI 设计
+-  **Tailwind CSS**：实用优先的 CSS 框架，用于构建自定义 UI 设计
 
 -  **Shadcn**：无头组件库
 
--  **Lucide React**：作为 React 组件实现的图标集
+-  **Motion（Framer Motion）**：React 动画库
-
--  **Framer Motion**：React 动画库
-
--  **Sonner**：Toast 通知库
-
--  **Geist**：Vercel 的字体系列
-
--  **React Hook Form**：表单状态管理和验证
-
--  **Zod**：TypeScript 优先的模式验证，带静态类型推断
-
--  **@hookform/resolvers**：用于在 React Hook Form 中使用验证库的解析器
-
--  **@tanstack/react-table**：用于构建强大表格和数据网格的无头 UI
 
 
  ### **DevOps**
@@ -332,6 +315,25 @@ Docker 和手动安装指南都包含适用于 Windows、macOS 和 Linux 的详
 非常欢迎贡献！贡献可以小到一个 ⭐，甚至是发现和创建问题。
 后端的微调总是受欢迎的。
 
+### 添加新的代理工具
+
+想要为 SurfSense 代理添加新工具？非常简单：
+
+1. 在 `surfsense_backend/app/agents/new_chat/tools/my_tool.py` 中创建您的工具文件
+2. 在 `registry.py` 中注册：
+
+```python
+ToolDefinition(
+    name="my_tool",
+    description="What my tool does",
+    factory=lambda deps: create_my_tool(
+        search_space_id=deps["search_space_id"],
+        db_session=deps["db_session"],
+    ),
+    requires=["search_space_id", "db_session"],
+),
+```
+
 有关详细的贡献指南，请参阅我们的 [CONTRIBUTING.md](CONTRIBUTING.md) 文件。
 
 ## Star 历史

surfsense_web/app/(home)/login/GoogleLoginButton.tsx

@@ -3,12 +3,16 @@ import { IconBrandGoogleFilled } from "@tabler/icons-react";
 import { motion } from "motion/react";
 import { useTranslations } from "next-intl";
 import { Logo } from "@/components/Logo";
+import { trackLoginAttempt, trackLoginFailure } from "@/lib/posthog/events";
 import { AmbientBackground } from "./AmbientBackground";
 
 export function GoogleLoginButton() {
 	const t = useTranslations("auth");
 
 	const handleGoogleLogin = () => {
+		// Track Google login attempt
+		trackLoginAttempt("google");
+
 		// Redirect to Google OAuth authorization URL
 		// credentials: 'include' is required to accept the CSRF cookie from cross-origin response
 		fetch(`${process.env.NEXT_PUBLIC_FASTAPI_BACKEND_URL}/auth/google/authorize`, {
@@ -24,10 +28,12 @@ export function GoogleLoginButton() {
 				if (data.authorization_url) {
 					window.location.href = data.authorization_url;
 				} else {
+					trackLoginFailure("google", "No authorization URL received");
 					console.error("No authorization URL received");
 				}
 			})
 			.catch((error) => {
+				trackLoginFailure("google", error?.message || "Unknown error");
 				console.error("Error during Google login:", error);
 			});
 	};

surfsense_web/app/(home)/login/LocalLoginForm.tsx

@@ -10,6 +10,7 @@ import { toast } from "sonner";
 import { loginMutationAtom } from "@/atoms/auth/auth-mutation.atoms";
 import { getAuthErrorDetails, isNetworkError, shouldRetry } from "@/lib/auth-errors";
 import { ValidationError } from "@/lib/error";
+import { trackLoginAttempt, trackLoginFailure, trackLoginSuccess } from "@/lib/posthog/events";
 
 export function LocalLoginForm() {
 	const t = useTranslations("auth");
@@ -37,6 +38,9 @@ export function LocalLoginForm() {
 		e.preventDefault();
 		setError({ title: null, message: null }); // Clear any previous errors
 
+		// Track login attempt
+		trackLoginAttempt("local");
+
 		// Show loading toast
 		const loadingToast = toast.loading(tCommon("loading"));
 
@@ -47,6 +51,9 @@ export function LocalLoginForm() {
 				grant_type: "password",
 			});
 
+			// Track successful login
+			trackLoginSuccess("local");
+
 			// Success toast
 			toast.success(t("login_success"), {
 				id: loadingToast,
@@ -60,6 +67,7 @@ export function LocalLoginForm() {
 			}, 500);
 		} catch (err) {
 			if (err instanceof ValidationError) {
+				trackLoginFailure("local", err.message);
 				setError({ title: err.name, message: err.message });
 				toast.error(err.name, {
 					id: loadingToast,
@@ -78,6 +86,9 @@ export function LocalLoginForm() {
 				errorCode = "NETWORK_ERROR";
 			}
 
+			// Track login failure
+			trackLoginFailure("local", errorCode);
+
 			// Get detailed error information from auth-errors utility
 			const errorDetails = getAuthErrorDetails(errorCode);
 

surfsense_web/app/(home)/register/page.tsx

@@ -11,6 +11,11 @@ import { registerMutationAtom } from "@/atoms/auth/auth-mutation.atoms";
 import { Logo } from "@/components/Logo";
 import { getAuthErrorDetails, isNetworkError, shouldRetry } from "@/lib/auth-errors";
 import { AppError, ValidationError } from "@/lib/error";
+import {
+	trackRegistrationAttempt,
+	trackRegistrationFailure,
+	trackRegistrationSuccess,
+} from "@/lib/posthog/events";
 import { AmbientBackground } from "../login/AmbientBackground";
 
 export default function RegisterPage() {
@@ -52,6 +57,9 @@ export default function RegisterPage() {
 
 		setError({ title: null, message: null }); // Clear any previous errors
 
+		// Track registration attempt
+		trackRegistrationAttempt();
+
 		// Show loading toast
 		const loadingToast = toast.loading(t("creating_account"));
 
@@ -64,6 +72,9 @@ export default function RegisterPage() {
 				is_verified: false,
 			});
 
+			// Track successful registration
+			trackRegistrationSuccess();
+
 			// Success toast
 			toast.success(t("register_success"), {
 				id: loadingToast,
@@ -81,6 +92,7 @@ export default function RegisterPage() {
 					case 403: {
 						const friendlyMessage =
 							"Registrations are currently closed. If you need access, contact your administrator.";
+						trackRegistrationFailure("Registration disabled");
 						setError({ title: "Registration is disabled", message: friendlyMessage });
 						toast.error("Registration is disabled", {
 							id: loadingToast,
@@ -94,6 +106,7 @@ export default function RegisterPage() {
 				}
 
 				if (err instanceof ValidationError) {
+					trackRegistrationFailure(err.message);
 					setError({ title: err.name, message: err.message });
 					toast.error(err.name, {
 						id: loadingToast,
@@ -113,6 +126,9 @@ export default function RegisterPage() {
 				errorCode = "NETWORK_ERROR";
 			}
 
+			// Track registration failure
+			trackRegistrationFailure(errorCode);
+
 			// Get detailed error information from auth-errors utility
 			const errorDetails = getAuthErrorDetails(errorCode);
 

surfsense_web/app/dashboard/[search_space_id]/new-chat/[[...chat_id]]/page.tsx

@@ -44,6 +44,12 @@ import {
 	getThreadMessages,
 	type MessageRecord,
 } from "@/lib/chat/thread-persistence";
+import {
+	trackChatCreated,
+	trackChatError,
+	trackChatMessageSent,
+	trackChatResponseReceived,
+} from "@/lib/posthog/events";
 
 /**
  * Extract thinking steps from message content
@@ -378,6 +384,10 @@ export default function NewChatPage() {
 					const newThread = await createThread(searchSpaceId, "New Chat");
 					currentThreadId = newThread.id;
 					setThreadId(currentThreadId);
+
+					// Track chat creation
+					trackChatCreated(searchSpaceId, currentThreadId);
+
 					isNewThread = true;
 					// Update URL silently using browser API (not router.replace) to avoid
 					// interrupting the ongoing fetch/streaming with React navigation
@@ -405,6 +415,13 @@ export default function NewChatPage() {
 			};
 			setMessages((prev) => [...prev, userMessage]);
 
+			// Track message sent
+			trackChatMessageSent(searchSpaceId, currentThreadId, {
+				hasAttachments: messageAttachments.length > 0,
+				hasMentionedDocuments: mentionedDocumentIds.length > 0,
+				messageLength: userQuery.length,
+			});
+
 			// Store mentioned documents with this message for display
 			if (mentionedDocuments.length > 0) {
 				const docsInfo: MentionedDocumentInfo[] = mentionedDocuments.map((doc) => ({
@@ -752,6 +769,9 @@ export default function NewChatPage() {
 						role: "assistant",
 						content: finalContent,
 					}).catch((err) => console.error("Failed to persist assistant message:", err));
+
+					// Track successful response
+					trackChatResponseReceived(searchSpaceId, currentThreadId);
 				}
 			} catch (error) {
 				if (error instanceof Error && error.name === "AbortError") {
@@ -773,6 +793,14 @@ export default function NewChatPage() {
 					return;
 				}
 				console.error("[NewChatPage] Chat error:", error);
+
+				// Track chat error
+				trackChatError(
+					searchSpaceId,
+					currentThreadId,
+					error instanceof Error ? error.message : "Unknown error"
+				);
+
 				toast.error("Failed to get response. Please try again.");
 				// Update assistant message with error
 				setMessages((prev) =>

surfsense_web/app/dashboard/[search_space_id]/settings/page.tsx

@@ -13,11 +13,12 @@ import {
 } from "lucide-react";
 import { AnimatePresence, motion } from "motion/react";
 import { useParams, useRouter } from "next/navigation";
-import { useCallback, useState } from "react";
+import { useCallback, useEffect, useState } from "react";
 import { LLMRoleManager } from "@/components/settings/llm-role-manager";
 import { ModelConfigManager } from "@/components/settings/model-config-manager";
 import { PromptConfigManager } from "@/components/settings/prompt-config-manager";
 import { Button } from "@/components/ui/button";
+import { trackSettingsViewed } from "@/lib/posthog/events";
 import { cn } from "@/lib/utils";
 
 interface SettingsNavItem {
@@ -271,6 +272,11 @@ export default function SettingsPage() {
 	const [activeSection, setActiveSection] = useState("models");
 	const [isSidebarOpen, setIsSidebarOpen] = useState(false);
 
+	// Track settings section view
+	useEffect(() => {
+		trackSettingsViewed(searchSpaceId, activeSection);
+	}, [searchSpaceId, activeSection]);
+
 	const handleBackToApp = useCallback(() => {
 		router.push(`/dashboard/${searchSpaceId}/new-chat`);
 	}, [router, searchSpaceId]);

surfsense_web/app/dashboard/[search_space_id]/sources/add/page.tsx

@@ -9,6 +9,7 @@ import { ConnectorsTab } from "@/components/sources/ConnectorsTab";
 import { DocumentUploadTab } from "@/components/sources/DocumentUploadTab";
 import { YouTubeTab } from "@/components/sources/YouTubeTab";
 import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
+import { trackSourcesTabViewed } from "@/lib/posthog/events";
 
 export default function AddSourcesPage() {
 	const params = useParams();
@@ -30,9 +31,16 @@ export default function AddSourcesPage() {
 			router.push(`/dashboard/${search_space_id}/connectors/add/webcrawler-connector`);
 		} else {
 			setActiveTab(value);
+			// Track tab view
+			trackSourcesTabViewed(Number(search_space_id), value);
 		}
 	};
 
+	// Track initial tab view
+	useEffect(() => {
+		trackSourcesTabViewed(Number(search_space_id), activeTab);
+	}, []);
+
 	return (
 		<div className="container mx-auto py-8 px-4 min-h-[calc(100vh-64px)]">
 			<motion.div

surfsense_web/app/dashboard/searchspaces/page.tsx

@@ -5,6 +5,7 @@ import { motion } from "motion/react";
 import { useRouter } from "next/navigation";
 import { createSearchSpaceMutationAtom } from "@/atoms/search-spaces/search-space-mutation.atoms";
 import { SearchSpaceForm } from "@/components/search-space-form";
+import { trackSearchSpaceCreated } from "@/lib/posthog/events";
 
 export default function SearchSpacesPage() {
 	const router = useRouter();
@@ -16,6 +17,9 @@ export default function SearchSpacesPage() {
 			description: data.description || "",
 		});
 
+		// Track search space creation
+		trackSearchSpaceCreated(result.id, data.name);
+
 		// Redirect to the newly created search space's onboarding
 		router.push(`/dashboard/${result.id}/onboard`);
 

surfsense_web/app/layout.tsx

@@ -4,6 +4,7 @@ import { GoogleAnalytics } from "@next/third-parties/google";
 import { RootProvider } from "fumadocs-ui/provider/next";
 import { Roboto } from "next/font/google";
 import { I18nProvider } from "@/components/providers/I18nProvider";
+import { PostHogProvider } from "@/components/providers/PostHogProvider";
 import { ThemeProvider } from "@/components/theme/theme-provider";
 import { Toaster } from "@/components/ui/sonner";
 import { LocaleProvider } from "@/contexts/LocaleContext";
@@ -93,21 +94,23 @@ export default function RootLayout({
 		<html lang="en" suppressHydrationWarning>
 			<GoogleAnalytics gaId="G-T4CHE7W3TE" />
 			<body className={cn(roboto.className, "bg-white dark:bg-black antialiased h-full w-full ")}>
-				<LocaleProvider>
+				<PostHogProvider>
-					<I18nProvider>
+					<LocaleProvider>
-						<ThemeProvider
+						<I18nProvider>
-							attribute="class"
+							<ThemeProvider
-							enableSystem
+								attribute="class"
-							disableTransitionOnChange
+								enableSystem
-							defaultTheme="light"
+								disableTransitionOnChange
-						>
+								defaultTheme="light"
-							<RootProvider>
+							>
-								<ReactQueryClientProvider>{children}</ReactQueryClientProvider>
+								<RootProvider>
-								<Toaster />
+									<ReactQueryClientProvider>{children}</ReactQueryClientProvider>
-							</RootProvider>
+									<Toaster />
-						</ThemeProvider>
+								</RootProvider>
-					</I18nProvider>
+							</ThemeProvider>
-				</LocaleProvider>
+						</I18nProvider>
+					</LocaleProvider>
+				</PostHogProvider>
 			</body>
 		</html>
 	);

surfsense_web/components/providers/PostHogProvider.tsx

@@ -0,0 +1,15 @@
+"use client";
+
+import { PostHogProvider as PHProvider } from "@posthog/react";
+import posthog from "posthog-js";
+import type { ReactNode } from "react";
+
+interface PostHogProviderProps {
+	children: ReactNode;
+}
+
+export function PostHogProvider({ children }: PostHogProviderProps) {
+	// posthog-js is already initialized in instrumentation-client.ts
+	// We just need to wrap the app with the PostHogProvider for hook access
+	return <PHProvider client={posthog}>{children}</PHProvider>;
+}

surfsense_web/components/sources/DocumentUploadTab.tsx

@@ -9,13 +9,17 @@ import { useCallback, useState } from "react";
 import { useDropzone } from "react-dropzone";
 import { toast } from "sonner";
 import { uploadDocumentMutationAtom } from "@/atoms/documents/document-mutation.atoms";
-
 import { Alert, AlertDescription } from "@/components/ui/alert";
 import { Badge } from "@/components/ui/badge";
 import { Button } from "@/components/ui/button";
 import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card";
 import { Progress } from "@/components/ui/progress";
 import { Separator } from "@/components/ui/separator";
+import {
+	trackDocumentUploadFailure,
+	trackDocumentUploadStarted,
+	trackDocumentUploadSuccess,
+} from "@/lib/posthog/events";
 import { GridPattern } from "./GridPattern";
 
 interface DocumentUploadTabProps {
@@ -154,6 +158,10 @@ export function DocumentUploadTab({ searchSpaceId }: DocumentUploadTabProps) {
 	const handleUpload = async () => {
 		setUploadProgress(0);
 
+		// Track upload started
+		const totalSize = files.reduce((sum, file) => sum + file.size, 0);
+		trackDocumentUploadStarted(Number(searchSpaceId), files.length, totalSize);
+
 		// Create a progress interval to simulate progress
 		const progressInterval = setInterval(() => {
 			setUploadProgress((prev) => {
@@ -172,6 +180,10 @@ export function DocumentUploadTab({ searchSpaceId }: DocumentUploadTabProps) {
 				onSuccess: () => {
 					clearInterval(progressInterval);
 					setUploadProgress(100);
+
+					// Track upload success
+					trackDocumentUploadSuccess(Number(searchSpaceId), files.length);
+
 					toast(t("upload_initiated"), {
 						description: t("upload_initiated_desc"),
 					});
@@ -180,6 +192,10 @@ export function DocumentUploadTab({ searchSpaceId }: DocumentUploadTabProps) {
 				onError: (error: any) => {
 					clearInterval(progressInterval);
 					setUploadProgress(0);
+
+					// Track upload failure
+					trackDocumentUploadFailure(Number(searchSpaceId), error.message || "Upload failed");
+
 					toast(t("upload_error"), {
 						description: `${t("upload_error_desc")}: ${error.message || "Upload failed"}`,
 					});

surfsense_web/instrumentation-client.ts

@@ -0,0 +1,16 @@
+import posthog from "posthog-js";
+
+if (process.env.NEXT_PUBLIC_POSTHOG_KEY) {
+	posthog.init(process.env.NEXT_PUBLIC_POSTHOG_KEY, {
+		// Use reverse proxy to bypass ad blockers
+		api_host: "/ingest",
+		// Required for toolbar and other UI features to work correctly
+		ui_host: "https://us.posthog.com",
+		defaults: "2025-11-30",
+		// Disable automatic pageview capture, as we capture manually with PostHogProvider
+		// This ensures proper pageview tracking with Next.js client-side navigation
+		capture_pageview: "history_change",
+		// Enable session recording
+		capture_pageleave: true,
+	});
+}

surfsense_web/lib/posthog/events.ts

@@ -0,0 +1,291 @@
+import posthog from "posthog-js";
+
+/**
+ * PostHog Analytics Event Definitions
+ *
+ * This file defines all custom analytics events tracked in SurfSense.
+ * Events follow a consistent naming convention: category_action
+ *
+ * Categories:
+ * - auth: Authentication events
+ * - search_space: Search space management
+ * - document: Document management
+ * - chat: Chat and messaging
+ * - connector: External connector events
+ * - contact: Contact form events
+ * - settings: Settings changes
+ */
+
+// ============================================
+// AUTH EVENTS
+// ============================================
+
+export function trackLoginAttempt(method: "local" | "google") {
+	posthog.capture("auth_login_attempt", {
+		method,
+	});
+}
+
+export function trackLoginSuccess(method: "local" | "google") {
+	posthog.capture("auth_login_success", {
+		method,
+	});
+}
+
+export function trackLoginFailure(method: "local" | "google", error?: string) {
+	posthog.capture("auth_login_failure", {
+		method,
+		error,
+	});
+}
+
+export function trackRegistrationAttempt() {
+	posthog.capture("auth_registration_attempt");
+}
+
+export function trackRegistrationSuccess() {
+	posthog.capture("auth_registration_success");
+}
+
+export function trackRegistrationFailure(error?: string) {
+	posthog.capture("auth_registration_failure", {
+		error,
+	});
+}
+
+export function trackLogout() {
+	posthog.capture("auth_logout");
+}
+
+// ============================================
+// SEARCH SPACE EVENTS
+// ============================================
+
+export function trackSearchSpaceCreated(searchSpaceId: number, name: string) {
+	posthog.capture("search_space_created", {
+		search_space_id: searchSpaceId,
+		name,
+	});
+}
+
+export function trackSearchSpaceDeleted(searchSpaceId: number) {
+	posthog.capture("search_space_deleted", {
+		search_space_id: searchSpaceId,
+	});
+}
+
+export function trackSearchSpaceViewed(searchSpaceId: number) {
+	posthog.capture("search_space_viewed", {
+		search_space_id: searchSpaceId,
+	});
+}
+
+// ============================================
+// CHAT EVENTS
+// ============================================
+
+export function trackChatCreated(searchSpaceId: number, chatId: number) {
+	posthog.capture("chat_created", {
+		search_space_id: searchSpaceId,
+		chat_id: chatId,
+	});
+}
+
+export function trackChatMessageSent(
+	searchSpaceId: number,
+	chatId: number,
+	options?: {
+		hasAttachments?: boolean;
+		hasMentionedDocuments?: boolean;
+		messageLength?: number;
+	}
+) {
+	posthog.capture("chat_message_sent", {
+		search_space_id: searchSpaceId,
+		chat_id: chatId,
+		has_attachments: options?.hasAttachments ?? false,
+		has_mentioned_documents: options?.hasMentionedDocuments ?? false,
+		message_length: options?.messageLength,
+	});
+}
+
+export function trackChatResponseReceived(searchSpaceId: number, chatId: number) {
+	posthog.capture("chat_response_received", {
+		search_space_id: searchSpaceId,
+		chat_id: chatId,
+	});
+}
+
+export function trackChatError(searchSpaceId: number, chatId: number, error?: string) {
+	posthog.capture("chat_error", {
+		search_space_id: searchSpaceId,
+		chat_id: chatId,
+		error,
+	});
+}
+
+// ============================================
+// DOCUMENT EVENTS
+// ============================================
+
+export function trackDocumentUploadStarted(
+	searchSpaceId: number,
+	fileCount: number,
+	totalSizeBytes: number
+) {
+	posthog.capture("document_upload_started", {
+		search_space_id: searchSpaceId,
+		file_count: fileCount,
+		total_size_bytes: totalSizeBytes,
+	});
+}
+
+export function trackDocumentUploadSuccess(searchSpaceId: number, fileCount: number) {
+	posthog.capture("document_upload_success", {
+		search_space_id: searchSpaceId,
+		file_count: fileCount,
+	});
+}
+
+export function trackDocumentUploadFailure(searchSpaceId: number, error?: string) {
+	posthog.capture("document_upload_failure", {
+		search_space_id: searchSpaceId,
+		error,
+	});
+}
+
+export function trackDocumentDeleted(searchSpaceId: number, documentId: number) {
+	posthog.capture("document_deleted", {
+		search_space_id: searchSpaceId,
+		document_id: documentId,
+	});
+}
+
+export function trackDocumentBulkDeleted(searchSpaceId: number, count: number) {
+	posthog.capture("document_bulk_deleted", {
+		search_space_id: searchSpaceId,
+		count,
+	});
+}
+
+export function trackYouTubeImport(searchSpaceId: number, url: string) {
+	posthog.capture("youtube_import_started", {
+		search_space_id: searchSpaceId,
+		url,
+	});
+}
+
+// ============================================
+// CONNECTOR EVENTS
+// ============================================
+
+export function trackConnectorSetupStarted(searchSpaceId: number, connectorType: string) {
+	posthog.capture("connector_setup_started", {
+		search_space_id: searchSpaceId,
+		connector_type: connectorType,
+	});
+}
+
+export function trackConnectorSetupSuccess(
+	searchSpaceId: number,
+	connectorType: string,
+	connectorId: number
+) {
+	posthog.capture("connector_setup_success", {
+		search_space_id: searchSpaceId,
+		connector_type: connectorType,
+		connector_id: connectorId,
+	});
+}
+
+export function trackConnectorSetupFailure(
+	searchSpaceId: number,
+	connectorType: string,
+	error?: string
+) {
+	posthog.capture("connector_setup_failure", {
+		search_space_id: searchSpaceId,
+		connector_type: connectorType,
+		error,
+	});
+}
+
+export function trackConnectorDeleted(
+	searchSpaceId: number,
+	connectorType: string,
+	connectorId: number
+) {
+	posthog.capture("connector_deleted", {
+		search_space_id: searchSpaceId,
+		connector_type: connectorType,
+		connector_id: connectorId,
+	});
+}
+
+export function trackConnectorSynced(
+	searchSpaceId: number,
+	connectorType: string,
+	connectorId: number
+) {
+	posthog.capture("connector_synced", {
+		search_space_id: searchSpaceId,
+		connector_type: connectorType,
+		connector_id: connectorId,
+	});
+}
+
+// ============================================
+// SETTINGS EVENTS
+// ============================================
+
+export function trackSettingsViewed(searchSpaceId: number, section: string) {
+	posthog.capture("settings_viewed", {
+		search_space_id: searchSpaceId,
+		section,
+	});
+}
+
+export function trackSettingsUpdated(searchSpaceId: number, section: string, setting: string) {
+	posthog.capture("settings_updated", {
+		search_space_id: searchSpaceId,
+		section,
+		setting,
+	});
+}
+
+// ============================================
+// FEATURE USAGE EVENTS
+// ============================================
+
+export function trackPodcastGenerated(searchSpaceId: number, chatId: number) {
+	posthog.capture("podcast_generated", {
+		search_space_id: searchSpaceId,
+		chat_id: chatId,
+	});
+}
+
+export function trackSourcesTabViewed(searchSpaceId: number, tab: string) {
+	posthog.capture("sources_tab_viewed", {
+		search_space_id: searchSpaceId,
+		tab,
+	});
+}
+
+// ============================================
+// USER IDENTIFICATION
+// ============================================
+
+/**
+ * Identify a user for PostHog analytics
+ * Call this after successful authentication
+ */
+export function identifyUser(userId: string, properties?: Record<string, unknown>) {
+	posthog.identify(userId, properties);
+}
+
+/**
+ * Reset user identity (call on logout)
+ */
+export function resetUser() {
+	posthog.reset();
+}

surfsense_web/lib/posthog/server.ts

@@ -0,0 +1,17 @@
+import { PostHog } from "posthog-node";
+
+export default function PostHogClient() {
+	if (!process.env.NEXT_PUBLIC_POSTHOG_KEY) {
+		throw new Error("NEXT_PUBLIC_POSTHOG_KEY is not set");
+	}
+
+	const posthogClient = new PostHog(process.env.NEXT_PUBLIC_POSTHOG_KEY, {
+		host: process.env.NEXT_PUBLIC_POSTHOG_HOST,
+		// Because server-side functions in Next.js can be short-lived,
+		// we set flushAt to 1 and flushInterval to 0 to ensure events are sent immediately
+		flushAt: 1,
+		flushInterval: 0,
+	});
+
+	return posthogClient;
+}

surfsense_web/next.config.ts

@@ -31,6 +31,27 @@ const nextConfig: NextConfig = {
 		}
 		return config;
 	},
+
+	// PostHog reverse proxy configuration
+	// This helps bypass ad blockers by routing requests through your domain
+	async rewrites() {
+		return [
+			{
+				source: "/ingest/static/:path*",
+				destination: "https://us-assets.i.posthog.com/static/:path*",
+			},
+			{
+				source: "/ingest/:path*",
+				destination: "https://us.i.posthog.com/:path*",
+			},
+			{
+				source: "/ingest/decide",
+				destination: "https://us.i.posthog.com/decide",
+			},
+		];
+	},
+	// Required for PostHog reverse proxy to work correctly
+	skipTrailingSlashRedirect: true,
 };
 
 // Wrap the config with MDX and next-intl plugins

surfsense_web/package.json

@@ -32,6 +32,7 @@
 		"@hookform/resolvers": "^5.2.2",
 		"@next/third-parties": "^16.1.0",
 		"@number-flow/react": "^0.5.10",
+		"@posthog/react": "^1.5.2",
 		"@radix-ui/react-accordion": "^1.2.11",
 		"@radix-ui/react-alert-dialog": "^1.1.14",
 		"@radix-ui/react-avatar": "^1.1.10",
@@ -81,6 +82,8 @@
 		"next-themes": "^0.4.6",
 		"pg": "^8.16.3",
 		"postgres": "^3.4.7",
+		"posthog-js": "^1.310.1",
+		"posthog-node": "^5.18.0",
 		"react": "^19.2.3",
 		"react-day-picker": "^9.8.1",
 		"react-dom": "^19.2.3",

surfsense_web/pnpm-lock.yaml

@@ -41,6 +41,9 @@ importers:
       '@number-flow/react':
         specifier: ^0.5.10
         version: 0.5.10(react-dom@19.2.3(react@19.2.3))(react@19.2.3)
+      '@posthog/react':
+        specifier: ^1.5.2
+        version: 1.5.2(@types/react@19.2.7)(posthog-js@1.310.1)(react@19.2.3)
       '@radix-ui/react-accordion':
         specifier: ^1.2.11
         version: 1.2.12(@types/react-dom@19.2.3(@types/react@19.2.7))(@types/react@19.2.7)(react-dom@19.2.3(react@19.2.3))(react@19.2.3)
@@ -188,6 +191,12 @@ importers:
       postgres:
         specifier: ^3.4.7
         version: 3.4.7
+      posthog-js:
+        specifier: ^1.310.1
+        version: 1.310.1
+      posthog-node:
+        specifier: ^5.18.0
+        version: 5.18.0
       react:
         specifier: ^19.2.3
         version: 19.2.3
@@ -1492,6 +1501,19 @@ packages:
     resolution: {integrity: sha512-dfUnCxiN9H4ap84DvD2ubjw+3vUNpstxa0TneY/Paat8a3R4uQZDLSvWjmznAY/DoahqTHl9V46HF/Zs3F29pg==}
     engines: {node: '>= 10.0.0'}
 
+  '@posthog/core@1.9.0':
+    resolution: {integrity: sha512-j7KSWxJTUtNyKynLt/p0hfip/3I46dWU2dk+pt7dKRoz2l5CYueHuHK4EO7Wlgno5yo1HO4sc4s30MXMTICHJw==}
+
+  '@posthog/react@1.5.2':
+    resolution: {integrity: sha512-KHdXbV1yba7Y2l8BVmwXlySWxqKVLNQ5ZiVvWOf7r3Eo7GIFxCM4CaNK/z83kKWn8KTskmKy7AGF6Hl6INWK3g==}
+    peerDependencies:
+      '@types/react': '>=16.8.0'
+      posthog-js: '>=1.257.2'
+      react: '>=16.8.0'
+    peerDependenciesMeta:
+      '@types/react':
+        optional: true
+
   '@radix-ui/number@1.1.1':
     resolution: {integrity: sha512-MkKCwxlXTgz6CFoJx3pCwn07GKp36+aZyu/u2Ln2VrA5DcdyCZkASEDBTd8x5whTQQL5CiYf4prXKLcgQdv29g==}
 
@@ -3273,6 +3295,9 @@ packages:
   confbox@0.1.8:
     resolution: {integrity: sha512-RMtmw0iFkeR4YV+fUOSucriAQNb9g8zFR52MWCtl+cCZOFRNL6zeB395vPzFhEjjn4fMxXudmELnl/KF/WrK6w==}
 
+  core-js@3.47.0:
+    resolution: {integrity: sha512-c3Q2VVkGAUyupsjRnaNX6u8Dq2vAdzm9iuPj5FW0fRxzlxgq9Q39MDq10IvmQSpLgHQNyQzQmOo6bgGHmH3NNg==}
+
   cose-base@1.0.3:
     resolution: {integrity: sha512-s9whTXInMSgAp/NVXVNuVxVKzGH2qck3aQlVHxDCdAEPgtMKwc4Wq6/QKhgdEdgbLSi9rBTAcPoRa6JpiG4ksg==}
 
@@ -3933,6 +3958,9 @@ packages:
       picomatch:
         optional: true
 
+  fflate@0.4.8:
+    resolution: {integrity: sha512-FJqqoDBR00Mdj9ppamLa/Y7vxm+PRmNWA67N846RvsoYVMKB4q3y/de5PA7gUmRMYK/8CMz2GDZQmCRN1wBcWA==}
+
   file-entry-cache@8.0.0:
     resolution: {integrity: sha512-XXTUwCvisa5oacNGRP9SfNtYBNAMi+RPwBFmblZEF7N7swHYQS6/Zfk7SRwx4D5j3CH211YNRco1DEMNVfZCnQ==}
     engines: {node: '>=16.0.0'}
@@ -5215,6 +5243,16 @@ packages:
     resolution: {integrity: sha512-Jtc2612XINuBjIl/QTWsV5UvE8UHuNblcO3vVADSrKsrc6RqGX6lOW1cEo3CM2v0XG4Nat8nI+YM7/f26VxXLw==}
     engines: {node: '>=12'}
 
+  posthog-js@1.310.1:
+    resolution: {integrity: sha512-UkR6zzlWNtqHDXHJl2Yk062DOmZyVKTPL5mX4j4V+u3RiYbMHJe47+PpMMUsvK1R2e1r/m9uSlHaJMJRzyUjGg==}
+
+  posthog-node@5.18.0:
+    resolution: {integrity: sha512-SLBEs+sCThxzTGSSDEe97nZHuFFYh6DupObR1yQdvQND3CJh0ogZ0Sa1Vb+Tbrnf0cWbfBC9XNkm44yhaWf3aA==}
+    engines: {node: '>=20'}
+
+  preact@10.28.1:
+    resolution: {integrity: sha512-u1/ixq/lVQI0CakKNvLDEcW5zfCjUQfZdK9qqWuIJtsezuyG6pk9TWj75GMuI/EzRSZB/VAE43sNWWZfiy8psw==}
+
   prelude-ls@1.2.1:
     resolution: {integrity: sha512-vkcDPrRZo1QZLbn5RLGPpg/WmIQ65qoWWhcGKf/b5eplkkarX0m9z8ppCat4mlOqUsWpyNuYgO3VRyrYHSzX5g==}
     engines: {node: '>= 0.8.0'}
@@ -6090,6 +6128,9 @@ packages:
   web-namespaces@2.0.1:
     resolution: {integrity: sha512-bKr1DkiNa2krS7qxNtdrtHAmzuYGFQLiQ13TsorsdT6ULTkPLKuu5+GsFpDlg6JFjUTwX2DyhMPG2be8uPrqsQ==}
 
+  web-vitals@4.2.4:
+    resolution: {integrity: sha512-r4DIlprAGwJ7YM11VZp4R884m0Vmgr6EAKe3P+kO0PPj3Unqyvv59rczf6UiGcb9Z8QxZVcqKNwv/g0WNdWwsw==}
+
   webidl-conversions@7.0.0:
     resolution: {integrity: sha512-VwddBukDzu71offAQR975unBIGqfKZpM+8ZX6ySk8nYhVoo5CYaZyzt3YBvYtRtO+aoGlqxPg/B87NGVZ/fu6g==}
     engines: {node: '>=12'}
@@ -7264,6 +7305,17 @@ snapshots:
       '@parcel/watcher-win32-ia32': 2.5.1
       '@parcel/watcher-win32-x64': 2.5.1
 
+  '@posthog/core@1.9.0':
+    dependencies:
+      cross-spawn: 7.0.6
+
+  '@posthog/react@1.5.2(@types/react@19.2.7)(posthog-js@1.310.1)(react@19.2.3)':
+    dependencies:
+      posthog-js: 1.310.1
+      react: 19.2.3
+    optionalDependencies:
+      '@types/react': 19.2.7
+
   '@radix-ui/number@1.1.1': {}
 
   '@radix-ui/primitive@1.0.0':
@@ -9110,6 +9162,8 @@ snapshots:
 
   confbox@0.1.8: {}
 
+  core-js@3.47.0: {}
+
   cose-base@1.0.3:
     dependencies:
       layout-base: 1.0.2
@@ -9939,6 +9993,8 @@ snapshots:
     optionalDependencies:
       picomatch: 4.0.3
 
+  fflate@0.4.8: {}
+
   file-entry-cache@8.0.0:
     dependencies:
       flat-cache: 4.0.1
@@ -11662,6 +11718,20 @@ snapshots:
 
   postgres@3.4.7: {}
 
+  posthog-js@1.310.1:
+    dependencies:
+      '@posthog/core': 1.9.0
+      core-js: 3.47.0
+      fflate: 0.4.8
+      preact: 10.28.1
+      web-vitals: 4.2.4
+
+  posthog-node@5.18.0:
+    dependencies:
+      '@posthog/core': 1.9.0
+
+  preact@10.28.1: {}
+
   prelude-ls@1.2.1: {}
 
   prismjs@1.27.0: {}
@@ -12767,6 +12837,8 @@ snapshots:
 
   web-namespaces@2.0.1: {}
 
+  web-vitals@4.2.4: {}
+
   webidl-conversions@7.0.0: {}
 
   whatwg-encoding@3.1.1: