diff --git a/README.zh-CN.md b/README.zh-CN.md index cd64467c8..def665fa9 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -16,7 +16,7 @@ # SurfSense -虽然像 NotebookLM 和 Perplexity 这样的工具在对任何主题/查询进行研究时令人印象深刻且非常有效，但 SurfSense 通过与您的个人知识库集成，将这一能力提升到了新的高度。它是一个高度可定制的 AI 研究助手，可以连接外部数据源，如搜索引擎（SearxNG、Tavily、LinkUp）、Slack、Linear、Jira、ClickUp、Confluence、Gmail、Notion、YouTube、GitHub、Discord、Airtable、Google Calendar、Luma、Elasticsearch 等，未来还会支持更多。 +虽然像 NotebookLM 和 Perplexity 这样的工具在对任何主题/查询进行研究时令人印象深刻且非常有效，但 SurfSense 通过与您的个人知识库集成，将这一能力提升到了新的高度。它是一个高度可定制的 AI 研究助手，可以连接外部数据源，如搜索引擎（SearxNG、Tavily、LinkUp）、Slack、Linear、Jira、ClickUp、Confluence、BookStack、Gmail、Notion、YouTube、GitHub、Discord、Airtable、Google Calendar、Luma、Elasticsearch 等，未来还会支持更多。
MODSetter%2FSurfSense | Trendshift @@ -25,8 +25,7 @@ # 视频演示 - -https://github.com/user-attachments/assets/d9221908-e0de-4b2f-ac3a-691cf4b202da +https://github.com/user-attachments/assets/42a29ea1-d4d8-4213-9c69-972b5b806d58 ## 播客示例 @@ -71,6 +70,27 @@ https://github.com/user-attachments/assets/a0a16566-6967-4374-ac51-9b3e07fbecd7 - 支持本地 TTS 提供商（Kokoro TTS） - 支持多个 TTS 提供商（OpenAI、Azure、Google Vertex AI） +### 🤖 **深度代理架构** + +#### 内置代理工具 +| 工具 | 描述 | +|------|------| +| **search_knowledge_base** | 使用语义+全文混合搜索、日期过滤和连接器特定查询搜索您的个人知识库 | +| **generate_podcast** | 从聊天对话或知识库内容生成音频播客 | +| **link_preview** | 获取 URL 的 Open Graph 元数据以显示预览卡片 | +| **display_image** | 在聊天中显示带有元数据和来源归属的图像 | +| **scrape_webpage** | 从网页中提取完整内容用于分析和总结（支持 Firecrawl 或本地 Chromium/Trafilatura） | + +#### 可扩展工具注册表 +贡献者可以通过注册表模式轻松添加新工具： +1. 在 `surfsense_backend/app/agents/new_chat/tools/` 中创建工具工厂函数 +2. 在 `registry.py` 的 `BUILTIN_TOOLS` 列表中注册 + +#### 可配置的系统提示词 +- 通过 LLM 配置自定义系统指令 +- 按配置切换引用开关 +- 通过 LiteLLM 集成支持 100+ 种 LLM + ### 📊 **先进的 RAG 技术** - 支持 100+ 种大语言模型 - 支持 6000+ 种嵌入模型 @@ -86,6 +106,7 @@ https://github.com/user-attachments/assets/a0a16566-6967-4374-ac51-9b3e07fbecd7 - Jira - ClickUp - Confluence +- BookStack - Notion - Gmail - YouTube 视频 @@ -214,32 +235,6 @@ Docker 和手动安装指南都包含适用于 Windows、macOS 和 Linux 的详 - LlamaIndex API 密钥（增强解析，支持 50+ 种格式） - 其他根据用例需要的 API 密钥 -## 截图 - -**研究助手** - -![updated_researcher](https://github.com/user-attachments/assets/e22c5d86-f511-4c72-8c50-feba0c1561b4) - -**搜索空间** - -![search_spaces](https://github.com/user-attachments/assets/e254c38c-f937-44b6-9e9d-770db583d099) - -**管理文档** -![documents](https://github.com/user-attachments/assets/7001e306-eb06-4009-89c6-8fadfdc3fc4d) - -**播客助手** -![podcasts](https://github.com/user-attachments/assets/6cb82ffd-9e14-4172-bc79-67faf34c4c1c) - - -**对话助手** - -![git_chat](https://github.com/user-attachments/assets/bb352d52-1c6d-4020-926b-722d0b98b491) - -**浏览器扩展** - -![ext1](https://github.com/user-attachments/assets/1f042b7a-6349-422b-94fb-d40d0df16c40) - -![ext2](https://github.com/user-attachments/assets/a9b9f1aa-2677-404d-b0a0-c1b2dddf24a7) ## 技术栈 @@ -257,11 +252,13 @@ Docker 和手动安装指南都包含适用于 Windows、macOS 和 Linux 的详 - **FastAPI Users**：使用 JWT 和 OAuth 支持的身份验证和用户管理 -- **LangGraph**：用于开发 AI 代理的框架 - +- **深度代理**：基于 LangGraph 构建的自定义代理框架，用于推理和行动的 AI 代理，支持可配置工具 + +- **LangGraph**：用于开发具有对话持久性的有状态 AI 代理的框架 + - **LangChain**：用于开发 AI 驱动应用程序的框架 -- **LLM 集成**：通过 LiteLLM 与大语言模型集成 +- **LiteLLM**：通用 LLM 集成，支持 100+ 种模型（OpenAI、Anthropic、Ollama 等） - **Rerankers**：先进的结果排序，提高搜索相关性 @@ -285,33 +282,19 @@ Docker 和手动安装指南都包含适用于 Windows、macOS 和 Linux 的详 --- ### **前端** -- **Next.js 15.2.3**：React 框架，具有应用路由器、服务器组件、自动代码拆分和优化渲染功能 +- **Next.js**：React 框架，具有应用路由器、服务器组件、自动代码拆分和优化渲染功能 -- **React 19.0.0**：用于构建用户界面的 JavaScript 库 +- **React**：用于构建用户界面的 JavaScript 库 - **TypeScript**：JavaScript 的静态类型检查，提升代码质量和开发体验 - **Vercel AI SDK Kit UI Stream Protocol**：创建可扩展的聊天 UI -- **Tailwind CSS 4.x**：实用优先的 CSS 框架，用于构建自定义 UI 设计 +- **Tailwind CSS**：实用优先的 CSS 框架，用于构建自定义 UI 设计 - **Shadcn**：无头组件库 -- **Lucide React**：作为 React 组件实现的图标集 - -- **Framer Motion**：React 动画库 - -- **Sonner**：Toast 通知库 - -- **Geist**：Vercel 的字体系列 - -- **React Hook Form**：表单状态管理和验证 - -- **Zod**：TypeScript 优先的模式验证，带静态类型推断 - -- **@hookform/resolvers**：用于在 React Hook Form 中使用验证库的解析器 - -- **@tanstack/react-table**：用于构建强大表格和数据网格的无头 UI +- **Motion（Framer Motion）**：React 动画库 ### **DevOps** @@ -332,6 +315,25 @@ Docker 和手动安装指南都包含适用于 Windows、macOS 和 Linux 的详 非常欢迎贡献！贡献可以小到一个 ⭐，甚至是发现和创建问题。 后端的微调总是受欢迎的。 +### 添加新的代理工具 + +想要为 SurfSense 代理添加新工具？非常简单： + +1. 在 `surfsense_backend/app/agents/new_chat/tools/my_tool.py` 中创建您的工具文件 +2. 在 `registry.py` 中注册： + +```python +ToolDefinition( + name="my_tool", + description="What my tool does", + factory=lambda deps: create_my_tool( + search_space_id=deps["search_space_id"], + db_session=deps["db_session"], + ), + requires=["search_space_id", "db_session"], +), +``` + 有关详细的贡献指南，请参阅我们的 [CONTRIBUTING.md](CONTRIBUTING.md) 文件。 ## Star 历史 diff --git a/surfsense_web/app/(home)/login/GoogleLoginButton.tsx b/surfsense_web/app/(home)/login/GoogleLoginButton.tsx index 6eef58aed..d4d9d4b4a 100644 --- a/surfsense_web/app/(home)/login/GoogleLoginButton.tsx +++ b/surfsense_web/app/(home)/login/GoogleLoginButton.tsx @@ -3,12 +3,16 @@ import { IconBrandGoogleFilled } from "@tabler/icons-react"; import { motion } from "motion/react"; import { useTranslations } from "next-intl"; import { Logo } from "@/components/Logo"; +import { trackLoginAttempt, trackLoginFailure } from "@/lib/posthog/events"; import { AmbientBackground } from "./AmbientBackground"; export function GoogleLoginButton() { const t = useTranslations("auth"); const handleGoogleLogin = () => { + // Track Google login attempt + trackLoginAttempt("google"); + // Redirect to Google OAuth authorization URL // credentials: 'include' is required to accept the CSRF cookie from cross-origin response fetch(`${process.env.NEXT_PUBLIC_FASTAPI_BACKEND_URL}/auth/google/authorize`, { @@ -24,10 +28,12 @@ export function GoogleLoginButton() { if (data.authorization_url) { window.location.href = data.authorization_url; } else { + trackLoginFailure("google", "No authorization URL received"); console.error("No authorization URL received"); } }) .catch((error) => { + trackLoginFailure("google", error?.message || "Unknown error"); console.error("Error during Google login:", error); }); }; diff --git a/surfsense_web/app/(home)/login/LocalLoginForm.tsx b/surfsense_web/app/(home)/login/LocalLoginForm.tsx index 0157c9faf..44e9b27c2 100644 --- a/surfsense_web/app/(home)/login/LocalLoginForm.tsx +++ b/surfsense_web/app/(home)/login/LocalLoginForm.tsx @@ -10,6 +10,7 @@ import { toast } from "sonner"; import { loginMutationAtom } from "@/atoms/auth/auth-mutation.atoms"; import { getAuthErrorDetails, isNetworkError, shouldRetry } from "@/lib/auth-errors"; import { ValidationError } from "@/lib/error"; +import { trackLoginAttempt, trackLoginFailure, trackLoginSuccess } from "@/lib/posthog/events"; export function LocalLoginForm() { const t = useTranslations("auth"); @@ -37,6 +38,9 @@ export function LocalLoginForm() { e.preventDefault(); setError({ title: null, message: null }); // Clear any previous errors + // Track login attempt + trackLoginAttempt("local"); + // Show loading toast const loadingToast = toast.loading(tCommon("loading")); @@ -47,6 +51,9 @@ export function LocalLoginForm() { grant_type: "password", }); + // Track successful login + trackLoginSuccess("local"); + // Success toast toast.success(t("login_success"), { id: loadingToast, @@ -60,6 +67,7 @@ export function LocalLoginForm() { }, 500); } catch (err) { if (err instanceof ValidationError) { + trackLoginFailure("local", err.message); setError({ title: err.name, message: err.message }); toast.error(err.name, { id: loadingToast, @@ -78,6 +86,9 @@ export function LocalLoginForm() { errorCode = "NETWORK_ERROR"; } + // Track login failure + trackLoginFailure("local", errorCode); + // Get detailed error information from auth-errors utility const errorDetails = getAuthErrorDetails(errorCode); diff --git a/surfsense_web/app/(home)/register/page.tsx b/surfsense_web/app/(home)/register/page.tsx index c535832be..4a8dce546 100644 --- a/surfsense_web/app/(home)/register/page.tsx +++ b/surfsense_web/app/(home)/register/page.tsx @@ -11,6 +11,11 @@ import { registerMutationAtom } from "@/atoms/auth/auth-mutation.atoms"; import { Logo } from "@/components/Logo"; import { getAuthErrorDetails, isNetworkError, shouldRetry } from "@/lib/auth-errors"; import { AppError, ValidationError } from "@/lib/error"; +import { + trackRegistrationAttempt, + trackRegistrationFailure, + trackRegistrationSuccess, +} from "@/lib/posthog/events"; import { AmbientBackground } from "../login/AmbientBackground"; export default function RegisterPage() { @@ -52,6 +57,9 @@ export default function RegisterPage() { setError({ title: null, message: null }); // Clear any previous errors + // Track registration attempt + trackRegistrationAttempt(); + // Show loading toast const loadingToast = toast.loading(t("creating_account")); @@ -64,6 +72,9 @@ export default function RegisterPage() { is_verified: false, }); + // Track successful registration + trackRegistrationSuccess(); + // Success toast toast.success(t("register_success"), { id: loadingToast, @@ -81,6 +92,7 @@ export default function RegisterPage() { case 403: { const friendlyMessage = "Registrations are currently closed. If you need access, contact your administrator."; + trackRegistrationFailure("Registration disabled"); setError({ title: "Registration is disabled", message: friendlyMessage }); toast.error("Registration is disabled", { id: loadingToast, @@ -94,6 +106,7 @@ export default function RegisterPage() { } if (err instanceof ValidationError) { + trackRegistrationFailure(err.message); setError({ title: err.name, message: err.message }); toast.error(err.name, { id: loadingToast, @@ -113,6 +126,9 @@ export default function RegisterPage() { errorCode = "NETWORK_ERROR"; } + // Track registration failure + trackRegistrationFailure(errorCode); + // Get detailed error information from auth-errors utility const errorDetails = getAuthErrorDetails(errorCode); diff --git a/surfsense_web/app/dashboard/[search_space_id]/new-chat/[[...chat_id]]/page.tsx b/surfsense_web/app/dashboard/[search_space_id]/new-chat/[[...chat_id]]/page.tsx index 5831a0069..1a6d173f3 100644 --- a/surfsense_web/app/dashboard/[search_space_id]/new-chat/[[...chat_id]]/page.tsx +++ b/surfsense_web/app/dashboard/[search_space_id]/new-chat/[[...chat_id]]/page.tsx @@ -44,6 +44,12 @@ import { getThreadMessages, type MessageRecord, } from "@/lib/chat/thread-persistence"; +import { + trackChatCreated, + trackChatError, + trackChatMessageSent, + trackChatResponseReceived, +} from "@/lib/posthog/events"; /** * Extract thinking steps from message content @@ -378,6 +384,10 @@ export default function NewChatPage() { const newThread = await createThread(searchSpaceId, "New Chat"); currentThreadId = newThread.id; setThreadId(currentThreadId); + + // Track chat creation + trackChatCreated(searchSpaceId, currentThreadId); + isNewThread = true; // Update URL silently using browser API (not router.replace) to avoid // interrupting the ongoing fetch/streaming with React navigation @@ -405,6 +415,13 @@ export default function NewChatPage() { }; setMessages((prev) => [...prev, userMessage]); + // Track message sent + trackChatMessageSent(searchSpaceId, currentThreadId, { + hasAttachments: messageAttachments.length > 0, + hasMentionedDocuments: mentionedDocumentIds.length > 0, + messageLength: userQuery.length, + }); + // Store mentioned documents with this message for display if (mentionedDocuments.length > 0) { const docsInfo: MentionedDocumentInfo[] = mentionedDocuments.map((doc) => ({ @@ -752,6 +769,9 @@ export default function NewChatPage() { role: "assistant", content: finalContent, }).catch((err) => console.error("Failed to persist assistant message:", err)); + + // Track successful response + trackChatResponseReceived(searchSpaceId, currentThreadId); } } catch (error) { if (error instanceof Error && error.name === "AbortError") { @@ -773,6 +793,14 @@ export default function NewChatPage() { return; } console.error("[NewChatPage] Chat error:", error); + + // Track chat error + trackChatError( + searchSpaceId, + currentThreadId, + error instanceof Error ? error.message : "Unknown error" + ); + toast.error("Failed to get response. Please try again."); // Update assistant message with error setMessages((prev) => diff --git a/surfsense_web/app/dashboard/[search_space_id]/settings/page.tsx b/surfsense_web/app/dashboard/[search_space_id]/settings/page.tsx index ad96402a4..48efcd922 100644 --- a/surfsense_web/app/dashboard/[search_space_id]/settings/page.tsx +++ b/surfsense_web/app/dashboard/[search_space_id]/settings/page.tsx @@ -13,11 +13,12 @@ import { } from "lucide-react"; import { AnimatePresence, motion } from "motion/react"; import { useParams, useRouter } from "next/navigation"; -import { useCallback, useState } from "react"; +import { useCallback, useEffect, useState } from "react"; import { LLMRoleManager } from "@/components/settings/llm-role-manager"; import { ModelConfigManager } from "@/components/settings/model-config-manager"; import { PromptConfigManager } from "@/components/settings/prompt-config-manager"; import { Button } from "@/components/ui/button"; +import { trackSettingsViewed } from "@/lib/posthog/events"; import { cn } from "@/lib/utils"; interface SettingsNavItem { @@ -271,6 +272,11 @@ export default function SettingsPage() { const [activeSection, setActiveSection] = useState("models"); const [isSidebarOpen, setIsSidebarOpen] = useState(false); + // Track settings section view + useEffect(() => { + trackSettingsViewed(searchSpaceId, activeSection); + }, [searchSpaceId, activeSection]); + const handleBackToApp = useCallback(() => { router.push(`/dashboard/${searchSpaceId}/new-chat`); }, [router, searchSpaceId]); diff --git a/surfsense_web/app/dashboard/[search_space_id]/sources/add/page.tsx b/surfsense_web/app/dashboard/[search_space_id]/sources/add/page.tsx index 3c9b57f98..e0729e29b 100644 --- a/surfsense_web/app/dashboard/[search_space_id]/sources/add/page.tsx +++ b/surfsense_web/app/dashboard/[search_space_id]/sources/add/page.tsx @@ -9,6 +9,7 @@ import { ConnectorsTab } from "@/components/sources/ConnectorsTab"; import { DocumentUploadTab } from "@/components/sources/DocumentUploadTab"; import { YouTubeTab } from "@/components/sources/YouTubeTab"; import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs"; +import { trackSourcesTabViewed } from "@/lib/posthog/events"; export default function AddSourcesPage() { const params = useParams(); @@ -30,9 +31,16 @@ export default function AddSourcesPage() { router.push(`/dashboard/${search_space_id}/connectors/add/webcrawler-connector`); } else { setActiveTab(value); + // Track tab view + trackSourcesTabViewed(Number(search_space_id), value); } }; + // Track initial tab view + useEffect(() => { + trackSourcesTabViewed(Number(search_space_id), activeTab); + }, []); + return (
- - - - - {children} - - - - - + + + + + + {children} + + + + + + ); diff --git a/surfsense_web/components/providers/PostHogProvider.tsx b/surfsense_web/components/providers/PostHogProvider.tsx new file mode 100644 index 000000000..6f62afce7 --- /dev/null +++ b/surfsense_web/components/providers/PostHogProvider.tsx @@ -0,0 +1,15 @@ +"use client"; + +import { PostHogProvider as PHProvider } from "@posthog/react"; +import posthog from "posthog-js"; +import type { ReactNode } from "react"; + +interface PostHogProviderProps { + children: ReactNode; +} + +export function PostHogProvider({ children }: PostHogProviderProps) { + // posthog-js is already initialized in instrumentation-client.ts + // We just need to wrap the app with the PostHogProvider for hook access + return {children}; +} diff --git a/surfsense_web/components/sources/DocumentUploadTab.tsx b/surfsense_web/components/sources/DocumentUploadTab.tsx index 7ad82d124..168db35cc 100644 --- a/surfsense_web/components/sources/DocumentUploadTab.tsx +++ b/surfsense_web/components/sources/DocumentUploadTab.tsx @@ -9,13 +9,17 @@ import { useCallback, useState } from "react"; import { useDropzone } from "react-dropzone"; import { toast } from "sonner"; import { uploadDocumentMutationAtom } from "@/atoms/documents/document-mutation.atoms"; - import { Alert, AlertDescription } from "@/components/ui/alert"; import { Badge } from "@/components/ui/badge"; import { Button } from "@/components/ui/button"; import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"; import { Progress } from "@/components/ui/progress"; import { Separator } from "@/components/ui/separator"; +import { + trackDocumentUploadFailure, + trackDocumentUploadStarted, + trackDocumentUploadSuccess, +} from "@/lib/posthog/events"; import { GridPattern } from "./GridPattern"; interface DocumentUploadTabProps { @@ -154,6 +158,10 @@ export function DocumentUploadTab({ searchSpaceId }: DocumentUploadTabProps) { const handleUpload = async () => { setUploadProgress(0); + // Track upload started + const totalSize = files.reduce((sum, file) => sum + file.size, 0); + trackDocumentUploadStarted(Number(searchSpaceId), files.length, totalSize); + // Create a progress interval to simulate progress const progressInterval = setInterval(() => { setUploadProgress((prev) => { @@ -172,6 +180,10 @@ export function DocumentUploadTab({ searchSpaceId }: DocumentUploadTabProps) { onSuccess: () => { clearInterval(progressInterval); setUploadProgress(100); + + // Track upload success + trackDocumentUploadSuccess(Number(searchSpaceId), files.length); + toast(t("upload_initiated"), { description: t("upload_initiated_desc"), }); @@ -180,6 +192,10 @@ export function DocumentUploadTab({ searchSpaceId }: DocumentUploadTabProps) { onError: (error: any) => { clearInterval(progressInterval); setUploadProgress(0); + + // Track upload failure + trackDocumentUploadFailure(Number(searchSpaceId), error.message || "Upload failed"); + toast(t("upload_error"), { description: `${t("upload_error_desc")}: ${error.message || "Upload failed"}`, }); diff --git a/surfsense_web/instrumentation-client.ts b/surfsense_web/instrumentation-client.ts new file mode 100644 index 000000000..15f989bb4 --- /dev/null +++ b/surfsense_web/instrumentation-client.ts @@ -0,0 +1,16 @@ +import posthog from "posthog-js"; + +if (process.env.NEXT_PUBLIC_POSTHOG_KEY) { + posthog.init(process.env.NEXT_PUBLIC_POSTHOG_KEY, { + // Use reverse proxy to bypass ad blockers + api_host: "/ingest", + // Required for toolbar and other UI features to work correctly + ui_host: "https://us.posthog.com", + defaults: "2025-11-30", + // Disable automatic pageview capture, as we capture manually with PostHogProvider + // This ensures proper pageview tracking with Next.js client-side navigation + capture_pageview: "history_change", + // Enable session recording + capture_pageleave: true, + }); +} diff --git a/surfsense_web/lib/posthog/events.ts b/surfsense_web/lib/posthog/events.ts new file mode 100644 index 000000000..fae713f80 --- /dev/null +++ b/surfsense_web/lib/posthog/events.ts @@ -0,0 +1,291 @@ +import posthog from "posthog-js"; + +/** + * PostHog Analytics Event Definitions + * + * This file defines all custom analytics events tracked in SurfSense. + * Events follow a consistent naming convention: category_action + * + * Categories: + * - auth: Authentication events + * - search_space: Search space management + * - document: Document management + * - chat: Chat and messaging + * - connector: External connector events + * - contact: Contact form events + * - settings: Settings changes + */ + +// ============================================ +// AUTH EVENTS +// ============================================ + +export function trackLoginAttempt(method: "local" | "google") { + posthog.capture("auth_login_attempt", { + method, + }); +} + +export function trackLoginSuccess(method: "local" | "google") { + posthog.capture("auth_login_success", { + method, + }); +} + +export function trackLoginFailure(method: "local" | "google", error?: string) { + posthog.capture("auth_login_failure", { + method, + error, + }); +} + +export function trackRegistrationAttempt() { + posthog.capture("auth_registration_attempt"); +} + +export function trackRegistrationSuccess() { + posthog.capture("auth_registration_success"); +} + +export function trackRegistrationFailure(error?: string) { + posthog.capture("auth_registration_failure", { + error, + }); +} + +export function trackLogout() { + posthog.capture("auth_logout"); +} + +// ============================================ +// SEARCH SPACE EVENTS +// ============================================ + +export function trackSearchSpaceCreated(searchSpaceId: number, name: string) { + posthog.capture("search_space_created", { + search_space_id: searchSpaceId, + name, + }); +} + +export function trackSearchSpaceDeleted(searchSpaceId: number) { + posthog.capture("search_space_deleted", { + search_space_id: searchSpaceId, + }); +} + +export function trackSearchSpaceViewed(searchSpaceId: number) { + posthog.capture("search_space_viewed", { + search_space_id: searchSpaceId, + }); +} + +// ============================================ +// CHAT EVENTS +// ============================================ + +export function trackChatCreated(searchSpaceId: number, chatId: number) { + posthog.capture("chat_created", { + search_space_id: searchSpaceId, + chat_id: chatId, + }); +} + +export function trackChatMessageSent( + searchSpaceId: number, + chatId: number, + options?: { + hasAttachments?: boolean; + hasMentionedDocuments?: boolean; + messageLength?: number; + } +) { + posthog.capture("chat_message_sent", { + search_space_id: searchSpaceId, + chat_id: chatId, + has_attachments: options?.hasAttachments ?? false, + has_mentioned_documents: options?.hasMentionedDocuments ?? false, + message_length: options?.messageLength, + }); +} + +export function trackChatResponseReceived(searchSpaceId: number, chatId: number) { + posthog.capture("chat_response_received", { + search_space_id: searchSpaceId, + chat_id: chatId, + }); +} + +export function trackChatError(searchSpaceId: number, chatId: number, error?: string) { + posthog.capture("chat_error", { + search_space_id: searchSpaceId, + chat_id: chatId, + error, + }); +} + +// ============================================ +// DOCUMENT EVENTS +// ============================================ + +export function trackDocumentUploadStarted( + searchSpaceId: number, + fileCount: number, + totalSizeBytes: number +) { + posthog.capture("document_upload_started", { + search_space_id: searchSpaceId, + file_count: fileCount, + total_size_bytes: totalSizeBytes, + }); +} + +export function trackDocumentUploadSuccess(searchSpaceId: number, fileCount: number) { + posthog.capture("document_upload_success", { + search_space_id: searchSpaceId, + file_count: fileCount, + }); +} + +export function trackDocumentUploadFailure(searchSpaceId: number, error?: string) { + posthog.capture("document_upload_failure", { + search_space_id: searchSpaceId, + error, + }); +} + +export function trackDocumentDeleted(searchSpaceId: number, documentId: number) { + posthog.capture("document_deleted", { + search_space_id: searchSpaceId, + document_id: documentId, + }); +} + +export function trackDocumentBulkDeleted(searchSpaceId: number, count: number) { + posthog.capture("document_bulk_deleted", { + search_space_id: searchSpaceId, + count, + }); +} + +export function trackYouTubeImport(searchSpaceId: number, url: string) { + posthog.capture("youtube_import_started", { + search_space_id: searchSpaceId, + url, + }); +} + +// ============================================ +// CONNECTOR EVENTS +// ============================================ + +export function trackConnectorSetupStarted(searchSpaceId: number, connectorType: string) { + posthog.capture("connector_setup_started", { + search_space_id: searchSpaceId, + connector_type: connectorType, + }); +} + +export function trackConnectorSetupSuccess( + searchSpaceId: number, + connectorType: string, + connectorId: number +) { + posthog.capture("connector_setup_success", { + search_space_id: searchSpaceId, + connector_type: connectorType, + connector_id: connectorId, + }); +} + +export function trackConnectorSetupFailure( + searchSpaceId: number, + connectorType: string, + error?: string +) { + posthog.capture("connector_setup_failure", { + search_space_id: searchSpaceId, + connector_type: connectorType, + error, + }); +} + +export function trackConnectorDeleted( + searchSpaceId: number, + connectorType: string, + connectorId: number +) { + posthog.capture("connector_deleted", { + search_space_id: searchSpaceId, + connector_type: connectorType, + connector_id: connectorId, + }); +} + +export function trackConnectorSynced( + searchSpaceId: number, + connectorType: string, + connectorId: number +) { + posthog.capture("connector_synced", { + search_space_id: searchSpaceId, + connector_type: connectorType, + connector_id: connectorId, + }); +} + +// ============================================ +// SETTINGS EVENTS +// ============================================ + +export function trackSettingsViewed(searchSpaceId: number, section: string) { + posthog.capture("settings_viewed", { + search_space_id: searchSpaceId, + section, + }); +} + +export function trackSettingsUpdated(searchSpaceId: number, section: string, setting: string) { + posthog.capture("settings_updated", { + search_space_id: searchSpaceId, + section, + setting, + }); +} + +// ============================================ +// FEATURE USAGE EVENTS +// ============================================ + +export function trackPodcastGenerated(searchSpaceId: number, chatId: number) { + posthog.capture("podcast_generated", { + search_space_id: searchSpaceId, + chat_id: chatId, + }); +} + +export function trackSourcesTabViewed(searchSpaceId: number, tab: string) { + posthog.capture("sources_tab_viewed", { + search_space_id: searchSpaceId, + tab, + }); +} + +// ============================================ +// USER IDENTIFICATION +// ============================================ + +/** + * Identify a user for PostHog analytics + * Call this after successful authentication + */ +export function identifyUser(userId: string, properties?: Record) { + posthog.identify(userId, properties); +} + +/** + * Reset user identity (call on logout) + */ +export function resetUser() { + posthog.reset(); +} diff --git a/surfsense_web/lib/posthog/server.ts b/surfsense_web/lib/posthog/server.ts new file mode 100644 index 000000000..5bfd79ca7 --- /dev/null +++ b/surfsense_web/lib/posthog/server.ts @@ -0,0 +1,17 @@ +import { PostHog } from "posthog-node"; + +export default function PostHogClient() { + if (!process.env.NEXT_PUBLIC_POSTHOG_KEY) { + throw new Error("NEXT_PUBLIC_POSTHOG_KEY is not set"); + } + + const posthogClient = new PostHog(process.env.NEXT_PUBLIC_POSTHOG_KEY, { + host: process.env.NEXT_PUBLIC_POSTHOG_HOST, + // Because server-side functions in Next.js can be short-lived, + // we set flushAt to 1 and flushInterval to 0 to ensure events are sent immediately + flushAt: 1, + flushInterval: 0, + }); + + return posthogClient; +} diff --git a/surfsense_web/next.config.ts b/surfsense_web/next.config.ts index 1f232b1e1..f7491b4d3 100644 --- a/surfsense_web/next.config.ts +++ b/surfsense_web/next.config.ts @@ -31,6 +31,27 @@ const nextConfig: NextConfig = { } return config; }, + + // PostHog reverse proxy configuration + // This helps bypass ad blockers by routing requests through your domain + async rewrites() { + return [ + { + source: "/ingest/static/:path*", + destination: "https://us-assets.i.posthog.com/static/:path*", + }, + { + source: "/ingest/:path*", + destination: "https://us.i.posthog.com/:path*", + }, + { + source: "/ingest/decide", + destination: "https://us.i.posthog.com/decide", + }, + ]; + }, + // Required for PostHog reverse proxy to work correctly + skipTrailingSlashRedirect: true, }; // Wrap the config with MDX and next-intl plugins diff --git a/surfsense_web/package.json b/surfsense_web/package.json index ed4c762a3..5d602c2ab 100644 --- a/surfsense_web/package.json +++ b/surfsense_web/package.json @@ -32,6 +32,7 @@ "@hookform/resolvers": "^5.2.2", "@next/third-parties": "^16.1.0", "@number-flow/react": "^0.5.10", + "@posthog/react": "^1.5.2", "@radix-ui/react-accordion": "^1.2.11", "@radix-ui/react-alert-dialog": "^1.1.14", "@radix-ui/react-avatar": "^1.1.10", @@ -81,6 +82,8 @@ "next-themes": "^0.4.6", "pg": "^8.16.3", "postgres": "^3.4.7", + "posthog-js": "^1.310.1", + "posthog-node": "^5.18.0", "react": "^19.2.3", "react-day-picker": "^9.8.1", "react-dom": "^19.2.3", diff --git a/surfsense_web/pnpm-lock.yaml b/surfsense_web/pnpm-lock.yaml index 22a251e12..a94b63c0d 100644 --- a/surfsense_web/pnpm-lock.yaml +++ b/surfsense_web/pnpm-lock.yaml @@ -41,6 +41,9 @@ importers: '@number-flow/react': specifier: ^0.5.10 version: 0.5.10(react-dom@19.2.3(react@19.2.3))(react@19.2.3) + '@posthog/react': + specifier: ^1.5.2 + version: 1.5.2(@types/react@19.2.7)(posthog-js@1.310.1)(react@19.2.3) '@radix-ui/react-accordion': specifier: ^1.2.11 version: 1.2.12(@types/react-dom@19.2.3(@types/react@19.2.7))(@types/react@19.2.7)(react-dom@19.2.3(react@19.2.3))(react@19.2.3) @@ -188,6 +191,12 @@ importers: postgres: specifier: ^3.4.7 version: 3.4.7 + posthog-js: + specifier: ^1.310.1 + version: 1.310.1 + posthog-node: + specifier: ^5.18.0 + version: 5.18.0 react: specifier: ^19.2.3 version: 19.2.3 @@ -1492,6 +1501,19 @@ packages: resolution: {integrity: sha512-dfUnCxiN9H4ap84DvD2ubjw+3vUNpstxa0TneY/Paat8a3R4uQZDLSvWjmznAY/DoahqTHl9V46HF/Zs3F29pg==} engines: {node: '>= 10.0.0'} + '@posthog/core@1.9.0': + resolution: {integrity: sha512-j7KSWxJTUtNyKynLt/p0hfip/3I46dWU2dk+pt7dKRoz2l5CYueHuHK4EO7Wlgno5yo1HO4sc4s30MXMTICHJw==} + + '@posthog/react@1.5.2': + resolution: {integrity: sha512-KHdXbV1yba7Y2l8BVmwXlySWxqKVLNQ5ZiVvWOf7r3Eo7GIFxCM4CaNK/z83kKWn8KTskmKy7AGF6Hl6INWK3g==} + peerDependencies: + '@types/react': '>=16.8.0' + posthog-js: '>=1.257.2' + react: '>=16.8.0' + peerDependenciesMeta: + '@types/react': + optional: true + '@radix-ui/number@1.1.1': resolution: {integrity: sha512-MkKCwxlXTgz6CFoJx3pCwn07GKp36+aZyu/u2Ln2VrA5DcdyCZkASEDBTd8x5whTQQL5CiYf4prXKLcgQdv29g==} @@ -3273,6 +3295,9 @@ packages: confbox@0.1.8: resolution: {integrity: sha512-RMtmw0iFkeR4YV+fUOSucriAQNb9g8zFR52MWCtl+cCZOFRNL6zeB395vPzFhEjjn4fMxXudmELnl/KF/WrK6w==} + core-js@3.47.0: + resolution: {integrity: sha512-c3Q2VVkGAUyupsjRnaNX6u8Dq2vAdzm9iuPj5FW0fRxzlxgq9Q39MDq10IvmQSpLgHQNyQzQmOo6bgGHmH3NNg==} + cose-base@1.0.3: resolution: {integrity: sha512-s9whTXInMSgAp/NVXVNuVxVKzGH2qck3aQlVHxDCdAEPgtMKwc4Wq6/QKhgdEdgbLSi9rBTAcPoRa6JpiG4ksg==} @@ -3933,6 +3958,9 @@ packages: picomatch: optional: true + fflate@0.4.8: + resolution: {integrity: sha512-FJqqoDBR00Mdj9ppamLa/Y7vxm+PRmNWA67N846RvsoYVMKB4q3y/de5PA7gUmRMYK/8CMz2GDZQmCRN1wBcWA==} + file-entry-cache@8.0.0: resolution: {integrity: sha512-XXTUwCvisa5oacNGRP9SfNtYBNAMi+RPwBFmblZEF7N7swHYQS6/Zfk7SRwx4D5j3CH211YNRco1DEMNVfZCnQ==} engines: {node: '>=16.0.0'} @@ -5215,6 +5243,16 @@ packages: resolution: {integrity: sha512-Jtc2612XINuBjIl/QTWsV5UvE8UHuNblcO3vVADSrKsrc6RqGX6lOW1cEo3CM2v0XG4Nat8nI+YM7/f26VxXLw==} engines: {node: '>=12'} + posthog-js@1.310.1: + resolution: {integrity: sha512-UkR6zzlWNtqHDXHJl2Yk062DOmZyVKTPL5mX4j4V+u3RiYbMHJe47+PpMMUsvK1R2e1r/m9uSlHaJMJRzyUjGg==} + + posthog-node@5.18.0: + resolution: {integrity: sha512-SLBEs+sCThxzTGSSDEe97nZHuFFYh6DupObR1yQdvQND3CJh0ogZ0Sa1Vb+Tbrnf0cWbfBC9XNkm44yhaWf3aA==} + engines: {node: '>=20'} + + preact@10.28.1: + resolution: {integrity: sha512-u1/ixq/lVQI0CakKNvLDEcW5zfCjUQfZdK9qqWuIJtsezuyG6pk9TWj75GMuI/EzRSZB/VAE43sNWWZfiy8psw==} + prelude-ls@1.2.1: resolution: {integrity: sha512-vkcDPrRZo1QZLbn5RLGPpg/WmIQ65qoWWhcGKf/b5eplkkarX0m9z8ppCat4mlOqUsWpyNuYgO3VRyrYHSzX5g==} engines: {node: '>= 0.8.0'} @@ -6090,6 +6128,9 @@ packages: web-namespaces@2.0.1: resolution: {integrity: sha512-bKr1DkiNa2krS7qxNtdrtHAmzuYGFQLiQ13TsorsdT6ULTkPLKuu5+GsFpDlg6JFjUTwX2DyhMPG2be8uPrqsQ==} + web-vitals@4.2.4: + resolution: {integrity: sha512-r4DIlprAGwJ7YM11VZp4R884m0Vmgr6EAKe3P+kO0PPj3Unqyvv59rczf6UiGcb9Z8QxZVcqKNwv/g0WNdWwsw==} + webidl-conversions@7.0.0: resolution: {integrity: sha512-VwddBukDzu71offAQR975unBIGqfKZpM+8ZX6ySk8nYhVoo5CYaZyzt3YBvYtRtO+aoGlqxPg/B87NGVZ/fu6g==} engines: {node: '>=12'} @@ -7264,6 +7305,17 @@ snapshots: '@parcel/watcher-win32-ia32': 2.5.1 '@parcel/watcher-win32-x64': 2.5.1 + '@posthog/core@1.9.0': + dependencies: + cross-spawn: 7.0.6 + + '@posthog/react@1.5.2(@types/react@19.2.7)(posthog-js@1.310.1)(react@19.2.3)': + dependencies: + posthog-js: 1.310.1 + react: 19.2.3 + optionalDependencies: + '@types/react': 19.2.7 + '@radix-ui/number@1.1.1': {} '@radix-ui/primitive@1.0.0': @@ -9110,6 +9162,8 @@ snapshots: confbox@0.1.8: {} + core-js@3.47.0: {} + cose-base@1.0.3: dependencies: layout-base: 1.0.2 @@ -9939,6 +9993,8 @@ snapshots: optionalDependencies: picomatch: 4.0.3 + fflate@0.4.8: {} + file-entry-cache@8.0.0: dependencies: flat-cache: 4.0.1 @@ -11662,6 +11718,20 @@ snapshots: postgres@3.4.7: {} + posthog-js@1.310.1: + dependencies: + '@posthog/core': 1.9.0 + core-js: 3.47.0 + fflate: 0.4.8 + preact: 10.28.1 + web-vitals: 4.2.4 + + posthog-node@5.18.0: + dependencies: + '@posthog/core': 1.9.0 + + preact@10.28.1: {} + prelude-ls@1.2.1: {} prismjs@1.27.0: {} @@ -12767,6 +12837,8 @@ snapshots: web-namespaces@2.0.1: {} + web-vitals@4.2.4: {} + webidl-conversions@7.0.0: {} whatwg-encoding@3.1.1: