From 9b0776596d28c6984583261a9b7fac4ec9ad3289 Mon Sep 17 00:00:00 2001 From: Gagan Date: Mon, 13 Jul 2026 21:02:20 +0530 Subject: [PATCH 1/8] =?UTF-8?q?feat(x):=20resident=20app=20=E2=80=94=20lau?= =?UTF-8?q?nch=20at=20login=20+=20menu=20bar=20tray=20(Granola=20parity=20?= =?UTF-8?q?phase=201)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Register as an OS login item on first packaged run (default on); the OS registry stays the source of truth afterward, with a toggle in Settings > Appearance > System - Menu bar tray with template icon (derived from the app glyph, embedded as base64): Open Rowboat / Start-Stop meeting notes / Quit - App keeps running with no windows while the tray exists; launches at login start hidden (menu bar only), Dock/tray reveals the window - Tray menu reflects live recording state via meeting:setRecordingState; tray-initiated toggles survive window-loading races through a pending command drained on renderer mount (same pattern as deep links) - GRANOLA_PARITY.md: research doc — how Granola works, current-state audit, gap analysis, phased parity plan --- apps/x/GRANOLA_PARITY.md | 153 ++++++++++++++++++ apps/x/apps/main/src/ipc.ts | 27 ++++ apps/x/apps/main/src/main.ts | 101 +++++++++++- apps/x/apps/main/src/tray.ts | 107 ++++++++++++ apps/x/apps/renderer/src/App.tsx | 20 +++ .../src/components/settings-dialog.tsx | 37 +++++ .../packages/core/src/config/app_settings.ts | 36 +++++ apps/x/packages/shared/src/ipc.ts | 40 +++++ 8 files changed, 513 insertions(+), 8 deletions(-) create mode 100644 apps/x/GRANOLA_PARITY.md create mode 100644 apps/x/apps/main/src/tray.ts create mode 100644 apps/x/packages/core/src/config/app_settings.ts diff --git a/apps/x/GRANOLA_PARITY.md b/apps/x/GRANOLA_PARITY.md new file mode 100644 index 00000000..53af3bd9 --- /dev/null +++ b/apps/x/GRANOLA_PARITY.md @@ -0,0 +1,153 @@ +# Granola Parity — Research & Gap Analysis + +Goal: make Rowboat's meeting feature work exactly like Granola (granola.ai). We copy Granola's behavior — no new invention. This doc is the source of truth: how Granola works, what Rowboat has today (with file:line pointers), the gaps, and the parity plan. + +Research basis: Granola's official docs (docs.granola.ai), granola.ai blog/security/jobs pages, third-party reviews and reverse-engineering writeups, plus a full audit of this codebase. Claims that are inferred (not directly documented by Granola) are marked **[inference]**. + +--- + +## Part 1 — How Granola works + +Granola is an Electron app (confirmed by their own job postings: "Granola is an Electron app with deep OS integrations", React/TypeScript UI + native OS helpers). No bot ever joins the call — it captures audio locally on the device, so nothing is visible to other meeting participants. + +### 1.1 Always-running app (launch at login) + +- The product model is an **always-running, menu-bar-resident app**. Granola must be running for detection/notifications to work ("You must have Granola open... for transcription to run" — their troubleshooting docs). +- A literal "open at login" toggle is **not publicly documented**. **[inference]** A clone should register as a login item (Electron `app.setLoginItemSettings`), default on, with a setting to turn it off — the entire detection value prop collapses if the app isn't running. +- The app keeps running when the window closes (menu bar presence remains). + +### 1.2 Menu bar icon + +- Granola "sits in your Mac's menu bar". Documented affordance: **click the menu bar icon to start recording / open the app**. +- Recording status is NOT primarily shown in the menu bar. The documented indicators are: + - Inside the note: "green dancing bars" at the bottom while capturing. + - When another app has focus: a **floating always-on-top "live meeting" pill** on the right side of the screen. +- Exact dropdown contents (upcoming meetings list, icon state change while recording) are not documented. The "Coming up" list lives on the home screen, not the tray. + +### 1.3 Meeting detection — two signals, never auto-record + +Granola detects meetings via **calendar events + microphone-in-use detection**, and it **never records without a user action** ("Granola only starts transcribing when you open a note for that meeting"). + +**Signal A — Calendar:** +- Google / Microsoft calendar sync. Pre-creates an auto-titled note per event with 2+ attendees; filters declined events, OOO, Focus Time. +- **In-app popup (Granola-drawn, not macOS Notification Center) 1 minute before** any scheduled call with 2+ attendees. One click on it **opens the meeting URL AND starts transcription** at once. +- **Armed auto-start:** if you have the upcoming meeting's note open before it starts, recording starts automatically at the scheduled time. + +**Signal B — Mic-in-use (ad-hoc calls, incl. browser meetings):** +- "Granola notices when your microphone is in use and offers to start taking notes." +- It's app-aware: notification title is "**Huddle detected**" (Slack), "**Call detected**" (FaceTime/WhatsApp), "**Meeting detected**" (anything else, including browsers). Buttons: "Take Notes". Ad-hoc popups have a dashed left border; calendar ones a solid colored bar. +- If the ad-hoc call starts within **15 minutes** of a scheduled event, the popup adopts that event's name (merged). +- **[inference]** Mechanism: poll CoreAudio `kAudioDevicePropertyDeviceIsRunningSomewhere` on input devices for "mic in use", plus enumerate running processes against a known meeting-app list to get the app-specific title. There's no public macOS API for "process X is using the mic", so it's a heuristic combo. Browser meetings degrade to the generic "Meeting detected" title; calendar linkage supplies identity. +- Notifications are configurable in Settings (off entirely, or per-application). +- Related preference: "**Reposition Granola for meetings**" — window auto-repositions/resizes alongside the call when a meeting starts (toggleable, default on). + +### 1.4 Audio capture + +- Capture = **default microphone + system audio output**, at the OS level. Works with any app or browser (Chrome/Safari/Firefox/Edge) with no extension. +- Cannot isolate per-app audio — it's the combined system stream; uses the OS default sound devices. +- The two streams stay separate through transcription → transcript UI shows **grey bubbles (left) = system audio ("Them"), green bubbles (right) = your mic ("Me")**. No true live diarization on desktop — just Me/Them. +- **macOS permissions: exactly two** — Microphone, and Screen & System Audio Recording (macOS bundles system-audio capture under screen recording). No Accessibility permission. +- **[inference]** System audio mechanism: ScreenCaptureKit audio loopback (macOS 13+ baseline, they require 13+ / recommend 14.2+), possibly CoreAudio process taps on 14.4+. No virtual audio driver (setup has no driver install step). +- **Audio is never stored** — streamed to the ASR provider in real time and discarded. + +### 1.5 Transcription + +- Cloud, real-time streaming ASR. Subprocessors named on their security page: **Deepgram and AssemblyAI** for ASR; **OpenAI and Anthropic** for note enhancement. +- Live transcript accrues during the call, hidden behind a waveform-icon toggle in the note. + +### 1.6 Meeting end & post-meeting flow + +- **Auto-stop conditions** (documented): (a) call-end heuristic — transcript inactivity + whether the call software is still in use + scheduled end time for calendar-linked meetings; (b) **15 minutes of silence**; (c) computer sleeps; (d) manual stop. Note: "on macOS, meeting auto-end detection requires admin rights" — without it, manual stop only. +- **On stop, enhancement runs automatically** — no click needed. Enhanced notes are ready in seconds (~30s max reported). +- A "**notes ready**" notification fires when enhancement completes — that's the re-entry point back into the app if you switched away. The note itself was already open (recording ran inside it), so the enhanced version replaces the raw view in place. There's no documented "force-focus the window" behavior; the notification does the redirecting. +- Extras: auto-drafted follow-up emails (toggleable), pre-meeting briefs. + +### 1.7 Notes UI + +**During the meeting:** a plain notepad — the user types rough bullets; transcription runs invisibly. Green dancing bars at the bottom = capturing. Waveform button (next to the per-note "Ask anything" chat bar) opens the live transcript side-by-side. Mid-meeting you can ask the chat to catch you up. + +**After the meeting (enhanced notes):** +- Enhancement merges exactly three inputs: **transcript + your raw notes + calendar metadata**, through a template. +- Signature affordance: **your words render black, AI-generated text grey**. +- Per-line provenance: magnifying-glass icon on a line reveals where it came from in the transcript. +- Templates: built-ins (1:1, standup, sales discovery) + custom; re-enhance with a different template (✨), regenerate (🔁), edit raw notes and re-enhance. +- Chat: per-note "Ask anything" bar, global chat (⌘J), cross-meeting chat over folders; edit-by-asking. +- Home screen: "**Coming up**" strip (next ~5 meetings) + reverse-chronological past notes list; folders; recurring meetings grouped by recurring-event ID + title; shareable links. + +--- + +## Part 2 — What Rowboat has today (audit of main) + +Two separate feature families exist in `apps/x`; only the second is Granola-adjacent: +- **"Calls" (video mode)** — live voice/video chat *with the Rowboat AI* (`VIDEO_MODE.md`). Not meeting capture. +- **"Meetings"** — real meeting capture → transcript → AI notes. Working pipeline, but **calendar-triggered and manual-click only**. + +### What EXISTS (and is solid) + +| Area | Status | Where | +|---|---|---| +| Mic + system-audio capture, 2-channel | ✅ | `apps/renderer/src/hooks/useMeetingTranscription.ts:282-299` — mic `getUserMedia` (ch 0) + `getDisplayMedia({audio})` loopback (ch 1), merged to 16 kHz PCM (`:424-485`); loopback auto-approved in `apps/main/src/main.ts:216-226` | +| Realtime ASR | ✅ | Deepgram realtime WS, `nova-3`, multichannel + diarize (`useMeetingTranscription.ts:9-21`); proxy or raw key (`:253-280`) | +| Speaker labels | ✅ | ch 0 → "You", ch 1 → diarized `Speaker N` (`:335-347`) | +| Transcript storage | ✅ | Markdown + frontmatter + fenced transcript block, `knowledge/Meetings/rowboat//.md` (`:93-136`, `:487-510`) | +| AI meeting notes on stop | ✅ | `packages/core/src/knowledge/summarize_meeting.ts` — LLM summary, attendee-name resolution from calendar; orchestrated in `App.tsx:5601-5661`; notes prepended above transcript | +| Auto-stop heuristics | ✅ (partial) | silence RMS backstop, calendar-end gating, system-track ended/muted — `useMeetingTranscription.ts:378-549` | +| Calendar sync | ✅ | Google OAuth via `googleapis`, `packages/core/src/knowledge/sync_calendar.ts`, per-event JSON in `~/.rowboat/calendar_sync/` | +| Pre-meeting notification | ✅ (system toast) | `packages/core/src/knowledge/notify_calendar_meetings.ts` — polls every 30s, notifies ~1 min before, deep-links `rowboat://action?type=join-and-take-meeting-notes` | +| Meetings screen | ✅ | `apps/renderer/src/components/meetings-view.tsx` — "Coming up" (+Join / Take-notes buttons, inline prep) + past-notes table | +| Transcript rendering | ✅ | `apps/renderer/src/extensions/transcript-block.tsx` (TipTap, colored speakers, collapsible) | +| Meeting prep briefs | ✅ | `meeting_prep_scheduler.ts`, `meeting_prep_brief.ts` (Granola has this too) | +| Permissions flows | ✅ | mic `ipc.ts:2090-2106`; screen recording check/open-settings `ipc.ts:2003-2018`; Info.plist strings `forge.config.cjs:201-204`; entitlements OK | + +### What is MISSING (the entire "ambient" layer) + +| # | Gap | Detail | +|---|---|---| +| 1 | **Launch at login** | Zero `setLoginItemSettings` / auto-launch code anywhere. | +| 2 | **Menu bar (tray) icon** | Zero `Tray` usage. No background presence: on macOS closing the window leaves only the Dock; no way to start recording without the full window. | +| 3 | **Meeting detection** | No mic-in-use detection, no process detection (Zoom/Teams/Slack/FaceTime), no browser awareness. Only calendar (time-based). Ad-hoc calls are invisible. | +| 4 | **Auto/one-click start UX** | Capture starts only via explicit button click or clicking the calendar toast. No "Meeting detected — Take notes?" popup, no armed auto-start when a note is open at meeting start time. | +| 5 | **Headless capture** | Capture depends on the renderer window holding a live `getDisplayMedia` stream. No native audio helper → can't capture while the app is "in the background" the way Granola does. | +| 6 | **Meeting-end redirect** | On stop, the note refreshes in place, but there's **no "notes ready" notification** and no bring-the-user-back moment if they're in another app. | +| 7 | **Notes UI polish (Granola signatures)** | No black-vs-grey authorship distinction (we replace, they merge raw notes + transcript), no during-meeting notepad-first flow (we show the transcript note), no per-line provenance, no templates/re-enhance, no floating "recording" pill. | + +### Honest assessment + +Rowboat has built the *second half* of Granola well — what happens once you're recording, and after the meeting ends. It has essentially none of the *first half* — noticing a meeting is happening and quietly being there without the user opening the app. That first half is exactly items 1–5 above, and it's where all the work is. + +--- + +## Part 3 — Parity plan (copy Granola, no new stuff) + +Ordered so each phase is shippable and testable on its own. + +### Phase 1 — Resident app: login item + tray +- `app.setLoginItemSettings({ openAtLogin: true })`, default on, toggle in Settings. When launched at login: no window, tray only **[inference — Granola undocumented, but implied]**. +- Electron `Tray` with template icon; menu: "Start recording", "Open Rowboat", recording status line, Quit. Keep app alive on window close (already macOS default; add tray so it's reachable). +- Acceptance: reboot → icon in menu bar, no window; click tray → start an ad-hoc meeting note. + +### Phase 2 — Meeting detection + "Take notes?" popup +- Native signal (small Swift helper or node addon, polled from main): mic-in-use via CoreAudio `kAudioDevicePropertyDeviceIsRunningSomewhere` + running-process match against a known list (zoom.us, Teams, Slack, FaceTime, browsers…). +- Granola-style app-drawn popup (small always-on-top BrowserWindow, like the existing video popout): "Huddle detected / Call detected / Meeting detected — [Take Notes]". Merge with a calendar event if within 15 min. Never auto-record. +- Upgrade the existing 1-min-before calendar toast to the same popup style; one click = open meeting URL + start capture (deep-link plumbing already exists in `deeplink.ts`). +- Armed auto-start: meeting note open before start time → auto-start at start time. +- Per-app notification settings. +- Acceptance: start a Meet call in Chrome with no calendar event → popup appears; click → recording. + +### Phase 3 — Meeting end → notes ready redirect +- Keep existing auto-stop heuristics; add the missing "call software no longer active" signal from the Phase-2 process watcher; keep 15-min silence backstop (ours is stricter — align to Granola's 15 min). +- On summary completion: fire a **"Your meeting notes are ready" notification**; clicking focuses the app on the note (deep link exists). This is the "redirect when we cut the call" the feature needs. +- Acceptance: leave the call → recording stops on its own → notification within ~30s → click lands on finished notes. + +### Phase 4 — Notes UI parity +- During meeting: notepad-first note (user types; transcript behind a waveform toggle); green capture indicator in-note; floating "live meeting" pill when app unfocused (reuse the video-popout window machinery). +- After meeting: enhancement merges **raw notes + transcript + calendar event** (today we only use the transcript); render user text black / AI text grey; ✨ re-enhance + 🔁 regenerate; keep transcript below as today. +- Home: Meetings view already ≈ Granola's home (Coming up + past list) — minor polish only. + +### Phase 5 (only if needed for true parity) — Headless capture +- Native mic + ScreenCaptureKit system-audio capture in main/helper process so recording doesn't require the renderer window. Biggest lift; Phases 1–4 deliver the Granola experience with the window opening on record start, which is acceptable Granola-like behavior (their note opens on start too). + +### Key implementation notes +- Permissions stay exactly two (mic + screen-recording) — we already handle both. +- Deepgram nova-3 multichannel already matches Granola's Me/Them model — no ASR change needed. +- Reuse: popup ← video popout window pattern; deep links ← `deeplink.ts`; detection loop ← same main-process init pattern as `notify_calendar_meetings.ts`. diff --git a/apps/x/apps/main/src/ipc.ts b/apps/x/apps/main/src/ipc.ts index ef530ff0..746b80f3 100644 --- a/apps/x/apps/main/src/ipc.ts +++ b/apps/x/apps/main/src/ipc.ts @@ -66,6 +66,8 @@ import { rankSlackHomeMessages } from '@x/core/dist/knowledge/sources/rank_slack import { syncSlackKnowledgeSources, triggerSync as triggerSlackKnowledgeSync, getSlackKnowledgeSyncStatus } from '@x/core/dist/knowledge/sources/sync_slack.js'; import { isOnboardingComplete, markOnboardingComplete } from '@x/core/dist/config/note_creation_config.js'; import { loadNotificationSettings, saveNotificationSettings } from '@x/core/dist/config/notification_config.js'; +import { saveAppSettings } from '@x/core/dist/config/app_settings.js'; +import { consumePendingToggleMeetingNotes, setTrayRecordingState } from './tray.js'; import * as composioHandler from './composio-handler.js'; import * as appsIndexer from '@x/core/dist/apps/indexer.js'; import * as appsServer from '@x/core/dist/apps/server.js'; @@ -839,6 +841,31 @@ export function setupIpcHandlers() { 'app:consumePendingDeepLink': async () => { return { url: consumePendingDeepLink() }; }, + 'app:consumePendingTrayCommand': async () => { + return { toggleMeetingNotes: consumePendingToggleMeetingNotes() }; + }, + 'app:getLoginItemSettings': async () => { + // Dev builds never register a login item (it would point at the dev + // Electron binary), so report off. + if (!app.isPackaged) return { openAtLogin: false }; + return { openAtLogin: app.getLoginItemSettings().openAtLogin }; + }, + 'app:setLoginItemSettings': async (_event, args) => { + if (app.isPackaged) { + app.setLoginItemSettings({ + openAtLogin: args.openAtLogin, + ...(process.platform === 'win32' ? { args: ['--hidden'] } : {}), + }); + // The user has expressed an explicit choice — never re-apply the + // first-run default over it. + saveAppSettings({ loginItemRegistered: true }); + } + return { success: true as const }; + }, + 'meeting:setRecordingState': async (_event, args) => { + setTrayRecordingState(args.recording); + return { success: true as const }; + }, 'analytics:bootstrap': async () => { return { installationId: getInstallationId(), diff --git a/apps/x/apps/main/src/main.ts b/apps/x/apps/main/src/main.ts index 1e1d3874..650c6cb9 100644 --- a/apps/x/apps/main/src/main.ts +++ b/apps/x/apps/main/src/main.ts @@ -1,5 +1,6 @@ import { app, BrowserWindow, desktopCapturer, protocol, net, shell, session, safeStorage, type Session } from "electron"; import path from "node:path"; +import os from "node:os"; import { setupIpcHandlers, startRunsWatcher, startSessionsWatcher, startTurnEventsWatcher, markSessionsIndexReady, @@ -65,6 +66,8 @@ import { } from "./deeplink.js"; import { disconnectGoogleIfScopesStale } from "./oauth-handler.js"; import { startModelsDevRefresh } from "@x/core/dist/models/models-dev.js"; +import { loadAppSettings, saveAppSettings } from "@x/core/dist/config/app_settings.js"; +import { createAppTray, hasTray, markPendingToggleMeetingNotes } from "./tray.js"; // Captured as early as possible so it reflects actual process start. Used to // gate grace-eligible notifications (e.g. the burst of background-task @@ -258,7 +261,46 @@ function setupZoomShortcuts(win: BrowserWindow) { }); } -function createWindow() { +// Resident-app plumbing (Granola-style): the main window may exist hidden +// (launched at login) or not at all (closed on macOS) while the app keeps +// running from the tray. showApp() is the single "bring the app up" path +// used by the tray, the Dock, and pending tray commands. +let mainWindow: BrowserWindow | null = null; + +function showApp(): void { + if (mainWindow && !mainWindow.isDestroyed()) { + if (mainWindow.isMinimized()) mainWindow.restore(); + if (!mainWindow.isVisible()) mainWindow.maximize(); + mainWindow.show(); + mainWindow.focus(); + return; + } + createWindow(); +} + +/** + * Was this process launched by the OS at login (rather than by the user)? + * Used to start with the window hidden so login launches are invisible. + * + * - Windows: our login item registers with an explicit --hidden arg. + * - macOS: wasOpenedAtLogin when available. On macOS 13+ (SMAppService) + * Electron doesn't reliably populate it (electron#37244), so fall back to + * a heuristic: a packaged launch while registered as a login item within + * two minutes of boot is treated as a login launch. + */ +function wasLaunchedAtLogin(): boolean { + if (process.argv.includes("--hidden")) return true; + if (process.platform !== "darwin" || !app.isPackaged) return false; + try { + const settings = app.getLoginItemSettings(); + if (settings.wasOpenedAtLogin) return true; + return settings.openAtLogin && os.uptime() < 120; + } catch { + return false; + } +} + +function createWindow(options: { startHidden?: boolean } = {}) { const win = new BrowserWindow({ width: 1280, height: 800, @@ -284,11 +326,18 @@ function createWindow() { configureSessionPermissions(session.defaultSession); configureSessionPermissions(session.fromPartition(BROWSER_PARTITION)); + mainWindow = win; setMainWindowForDeepLinks(win); - win.on("closed", () => setMainWindowForDeepLinks(null)); + win.on("closed", () => { + if (mainWindow === win) mainWindow = null; + setMainWindowForDeepLinks(null); + }); - // Show window when content is ready to prevent blank screen + // Show window when content is ready to prevent blank screen. + // Launched-at-login starts stay hidden: the app is reachable from the + // tray/Dock, and showApp() maximizes on first reveal. win.once("ready-to-show", () => { + if (options.startHidden) return; win.maximize(); win.show(); }); @@ -419,7 +468,41 @@ app.whenReady().then(async () => { console.error('[Apps] Failed to start:', error); }); - createWindow(); + // Resident app (Granola-style): register as an OS login item once, on the + // first packaged run. After that the OS registry is the source of truth — + // the Settings toggle writes it directly, and disabling the login item in + // System Settings sticks because we never re-register on boot. + if (app.isPackaged && !loadAppSettings().loginItemRegistered) { + try { + app.setLoginItemSettings({ + openAtLogin: true, + ...(process.platform === "win32" ? { args: ["--hidden"] } : {}), + }); + saveAppSettings({ loginItemRegistered: true }); + } catch (error) { + console.error("[LoginItem] Failed to register login item:", error); + } + } + + createWindow({ startHidden: wasLaunchedAtLogin() }); + + // Menu bar icon: open the app / start-stop meeting notes without the + // window. If the renderer isn't ready to receive the toggle (window closed + // or still loading), park it as a pending command the renderer drains on + // mount — same pull pattern as pending deep links. + createAppTray({ + openApp: showApp, + toggleMeetingNotes: () => { + const hadWindow = mainWindow !== null && !mainWindow.isDestroyed(); + showApp(); + const win = mainWindow; + if (!hadWindow || !win || win.webContents.isLoading()) { + markPendingToggleMeetingNotes(); + return; + } + win.webContents.send("app:toggleMeetingNotes", null); + }, + }); // Start workspace watcher as a main-process service // Watcher runs independently and catches ALL filesystem changes: @@ -547,14 +630,16 @@ app.whenReady().then(async () => { initChromeSync(); app.on("activate", () => { - if (BrowserWindow.getAllWindows().length === 0) { - createWindow(); - } + // Reveal the hidden/closed main window (login launches start hidden). + showApp(); }); }); app.on("window-all-closed", () => { - if (process.platform !== "darwin") { + // Resident app: with a tray present, keep running with no windows so + // meeting detection/notifications stay alive (Granola-style). Without a + // tray (creation failed), fall back to the platform-default quit. + if (process.platform !== "darwin" && !hasTray()) { app.quit(); } }); diff --git a/apps/x/apps/main/src/tray.ts b/apps/x/apps/main/src/tray.ts new file mode 100644 index 00000000..6882ec9b --- /dev/null +++ b/apps/x/apps/main/src/tray.ts @@ -0,0 +1,107 @@ +import { app, Menu, Tray, nativeImage } from "electron"; + +/** + * Menu bar / system tray presence (Granola-style resident app). + * + * The icon is the app glyph pre-rendered as a macOS "template" image + * (pure black + alpha, derived from icons/icon.png: alpha = pixel + * luminance, so the white sail becomes an opaque black shape and the + * black rounded square becomes transparent). Embedded as base64 so the + * tray never depends on asset paths that differ between dev and + * packaged layouts. Template rendering makes macOS tint it correctly + * in light/dark menu bars and while highlighted. + */ +const TRAY_ICON_16 = + "iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAABD0lEQVR4nKSSz+pBQRTH585vfgtrllYiSkpJHkAegLJQlvIE9soD2MvSE3gCG57ARiwoe2Kl/Lm+J2d0XJd7y7c+dc898z1zzswYFV6OwAU3+zNIf9LglwwyWqKgBKogBpZAa+XfKukKIqABxmABJqAPcnat8Zi1aLUNOiAl8hfeYMaxazw7kzkNRqAoOpFjHcCc45sWZpqzxtVpxqPIac6TpmDP/5UtQMkkz1YGCZAHA89YDndnv1+u8R+c1buGoMVFtiADTjYpb+HMsSNy9N3llinusfnpC/OQSBuwAwWO7Xko88VkDzarHo+owvHLpkEFSHXQBCtR9G3RJ9HBxsHazxymgFzn+iWM+sFMugMAAP//whjxNQAAAAZJREFUAwCzrTaLm3OP9AAAAABJRU5ErkJggg=="; +const TRAY_ICON_32 = + "iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAACfUlEQVR4nMSXO2gUURSGz8yuYuGjsBHFBza+UGwULUxjoY0iWPnA0lZQURDsFC1E7Owt7JQ0SZGQLk2SNoQEQhICCXlUCXk/ZvOf3HM2Z2/uZDM7m90fPnbm3jn3P/e5M0VqsorUZBWocYqogQnEhj1VzwTUsOTBOgJOg1WwYYMiyqdITDe9RK6CW+A2uAYugOPgBhiXuO3kal2E1pg5BO6BJ+A+uEK7OzcJpuVaR6amBArG+Bx4BZ6LqZUOdSI+I+SmoEJZEoilMTa+CN6Cl+CEMUrkuUgS5d91+e33OpApAQ06Cj6CN+CY1G14pqryPMt1b6jhagloI2z+EPwCl4xxgXabqnQXFCW+R8oTaxDvw5z1DbSL+Zpp2D4bpcTz7zAYMomVVaxizkP+FzwydYfJ9Uh3QskYWXGZrolOiamY/7QE1Pwk6AaXwTzoA7OghdyhQtJY7PVWzcnU/fPKK8xC5mdBB1gGv6UHY/IMr/rH4D25gyUtCS3n4efDaJ1Seuvf8xC/BhPgv1dnTz1+7qskogvSrhsu4wPqM/hCgeEPJUC0s99VBbkveWXa2A/wzktC45fILdxJL7kKM1+JV7cZCNRFyIYfwICXqA7/HzGPA22Ue5KmElWXjtYceCrXuiVXwDOp27OBPNJpaAVT5HaVTsVPcv98qb0nqs/7ABvw4XQTXJf7UfBCEkyqBeeV9u6UXHObvIsW9xOcNwFd2WfAHWnvO+iinUV5oAlo/ANyx3Yb+EQpez6keryS8QgMkjvp7oIFStnzIeX5LtAtyK9h3OOWrOZ5E9C/5PPk3gdnspoT5Z8Ce2xnNtegeqgmc1YjP82CavrHadMT2AIAAP//JKuLxQAAAAZJREFUAwAJ75pCeqjxVQAAAABJRU5ErkJggg=="; + +interface TrayActions { + openApp: () => void; + toggleMeetingNotes: () => void; +} + +let tray: Tray | null = null; +let actions: TrayActions | null = null; +let recording = false; + +// Tray commands issued while the renderer wasn't ready to receive them +// (window closed or still loading). Drained by the renderer on mount via +// app:consumePendingTrayCommand — same pull pattern as pending deep links. +let pendingToggleMeetingNotes = false; + +export function markPendingToggleMeetingNotes(): void { + pendingToggleMeetingNotes = true; +} + +export function consumePendingToggleMeetingNotes(): boolean { + const value = pendingToggleMeetingNotes; + pendingToggleMeetingNotes = false; + return value; +} + +function buildTrayIcon() { + const icon = nativeImage.createEmpty(); + icon.addRepresentation({ + scaleFactor: 1, + buffer: Buffer.from(TRAY_ICON_16, "base64"), + }); + icon.addRepresentation({ + scaleFactor: 2, + buffer: Buffer.from(TRAY_ICON_32, "base64"), + }); + icon.setTemplateImage(true); + return icon; +} + +export function createAppTray(trayActions: TrayActions): void { + if (tray) return; + actions = trayActions; + + try { + tray = new Tray(buildTrayIcon()); + } catch (error) { + // Tray support can be missing (some Linux environments). The app just + // behaves as before: no resident presence. + console.error("[Tray] Failed to create tray:", error); + return; + } + + rebuildMenu(); + + // macOS opens the context menu on any click. On Windows/Linux a plain + // left-click should open the app; the menu stays on right-click. + if (process.platform !== "darwin") { + tray.on("click", () => actions?.openApp()); + } +} + +export function hasTray(): boolean { + return tray !== null; +} + +export function setTrayRecordingState(isRecording: boolean): void { + if (recording === isRecording) return; + recording = isRecording; + rebuildMenu(); +} + +function rebuildMenu(): void { + if (!tray) return; + const menu = Menu.buildFromTemplate([ + { label: "Open Rowboat", click: () => actions?.openApp() }, + recording + ? { + label: "Stop recording and generate notes", + click: () => actions?.toggleMeetingNotes(), + } + : { + label: "Start meeting notes", + click: () => actions?.toggleMeetingNotes(), + }, + { type: "separator" }, + { label: "Quit Rowboat", click: () => app.quit() }, + ]); + tray.setContextMenu(menu); + tray.setToolTip(recording ? "Rowboat — recording meeting" : "Rowboat"); +} diff --git a/apps/x/apps/renderer/src/App.tsx b/apps/x/apps/renderer/src/App.tsx index b322d278..5906b390 100644 --- a/apps/x/apps/renderer/src/App.tsx +++ b/apps/x/apps/renderer/src/App.tsx @@ -1101,6 +1101,14 @@ function App() { handleToggleMeetingRef.current?.() }) + // Keep the tray menu in sync with meeting capture ("Start meeting notes" + // vs "Stop recording & generate notes"). + useEffect(() => { + void window.ipc + .invoke('meeting:setRecordingState', { recording: meetingTranscription.state === 'recording' }) + .catch(() => { /* tray may be unavailable */ }) + }, [meetingTranscription.state]) + // Check if voice is available on mount and when OAuth state changes const refreshVoiceAvailability = useCallback(() => { Promise.all([ @@ -4678,6 +4686,18 @@ function App() { return () => observer.disconnect() }, []) + // Tray menu "Start/Stop meeting notes": same toggle as the Meetings header + // button. Also drains a toggle parked while the window was closed/loading + // (mirrors the pending deep-link pull above). + useEffect(() => { + void window.ipc.invoke('app:consumePendingTrayCommand', null).then(({ toggleMeetingNotes }) => { + if (toggleMeetingNotes) handleToggleMeetingRef.current?.() + }) + return window.ipc.on('app:toggleMeetingNotes', () => { + handleToggleMeetingRef.current?.() + }) + }, []) + // Triggered by main when the user clicks a calendar-meeting notification. // Reuses the same flow as the in-app "Join meeting & take notes" button. // When `openMeeting` is true, also opens the meeting URL in the system browser. diff --git a/apps/x/apps/renderer/src/components/settings-dialog.tsx b/apps/x/apps/renderer/src/components/settings-dialog.tsx index 4b56eb51..a8ba8416 100644 --- a/apps/x/apps/renderer/src/components/settings-dialog.tsx +++ b/apps/x/apps/renderer/src/components/settings-dialog.tsx @@ -226,11 +226,48 @@ function ThemeOption({ ) } +function LaunchAtLoginSetting() { + const [openAtLogin, setOpenAtLogin] = useState(false) + const [loaded, setLoaded] = useState(false) + + useEffect(() => { + window.ipc.invoke("app:getLoginItemSettings", null) + .then(({ openAtLogin }) => setOpenAtLogin(openAtLogin)) + .catch(() => { /* dev builds report off */ }) + .finally(() => setLoaded(true)) + }, []) + + const handleToggle = async (next: boolean) => { + setOpenAtLogin(next) + try { + await window.ipc.invoke("app:setLoginItemSettings", { openAtLogin: next }) + } catch { + setOpenAtLogin(!next) + } + } + + return ( +
+
+
Start Rowboat when you log in
+
+ Keeps Rowboat in your menu bar so meeting notes and notifications work without opening the app +
+
+ +
+ ) +} + function AppearanceSettings() { const { theme, setTheme, chatPanePlacement, setChatPanePlacement, chatPaneSize, setChatPaneSize } = useTheme() return (
+
+

System

+ +

Theme

diff --git a/apps/x/packages/core/src/config/app_settings.ts b/apps/x/packages/core/src/config/app_settings.ts new file mode 100644 index 00000000..f487dbb2 --- /dev/null +++ b/apps/x/packages/core/src/config/app_settings.ts @@ -0,0 +1,36 @@ +import fs from 'fs'; +import path from 'path'; +import { WorkDir } from './config.js'; + +const APP_SETTINGS_PATH = path.join(WorkDir, 'config', 'app_settings.json'); + +export interface AppSettings { + /** + * Set once the app has registered itself as an OS login item (first run + * of a packaged build, or the user touching the Settings toggle). After + * this, the OS login-item registry is the source of truth — the app never + * re-registers on boot, so disabling it in System Settings sticks. + */ + loginItemRegistered?: boolean; +} + +export function loadAppSettings(): AppSettings { + try { + if (fs.existsSync(APP_SETTINGS_PATH)) { + const parsed = JSON.parse(fs.readFileSync(APP_SETTINGS_PATH, 'utf-8')); + if (parsed && typeof parsed === 'object') return parsed as AppSettings; + } + } catch (error) { + console.error('[AppSettings] Error loading app settings:', error); + } + return {}; +} + +export function saveAppSettings(patch: Partial): void { + const merged = { ...loadAppSettings(), ...patch }; + const dir = path.dirname(APP_SETTINGS_PATH); + if (!fs.existsSync(dir)) { + fs.mkdirSync(dir, { recursive: true }); + } + fs.writeFileSync(APP_SETTINGS_PATH, JSON.stringify(merged, null, 2)); +} diff --git a/apps/x/packages/shared/src/ipc.ts b/apps/x/packages/shared/src/ipc.ts index 17872e88..fbea4288 100644 --- a/apps/x/packages/shared/src/ipc.ts +++ b/apps/x/packages/shared/src/ipc.ts @@ -743,6 +743,46 @@ const ipcSchemas = { url: z.string().nullable(), }), }, + // Tray commands issued before the renderer was ready (mirrors the pending + // deep-link pull above): the renderer drains this once on mount. + 'app:consumePendingTrayCommand': { + req: z.null(), + res: z.object({ + toggleMeetingNotes: z.boolean(), + }), + }, + // Main → renderer: tray menu "Start/Stop meeting notes" — runs the same + // toggle flow as the Meetings header button. + 'app:toggleMeetingNotes': { + req: z.null(), + res: z.null(), + }, + // Launch-at-login (resident app). The OS login-item registry is the source + // of truth; these read/write it directly rather than a config file. + 'app:getLoginItemSettings': { + req: z.null(), + res: z.object({ + openAtLogin: z.boolean(), + }), + }, + 'app:setLoginItemSettings': { + req: z.object({ + openAtLogin: z.boolean(), + }), + res: z.object({ + success: z.literal(true), + }), + }, + // Renderer → main: meeting capture state, so the tray menu/tooltip can + // reflect an active recording. + 'meeting:setRecordingState': { + req: z.object({ + recording: z.boolean(), + }), + res: z.object({ + success: z.literal(true), + }), + }, 'granola:getConfig': { req: z.null(), res: z.object({ From 66865b4f6b93833ce3ed44919e44bd64ca99c2e6 Mon Sep 17 00:00:00 2001 From: Gagan Date: Mon, 13 Jul 2026 21:50:39 +0530 Subject: [PATCH 2/8] feat(x): ambient meeting detection + take-notes popup (Granola parity phase 2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - mic-monitor Swift helper: polls CoreAudio for mic-in-use, with owning PIDs on macOS 14.4+ (process-object API) so calls attribute to the app that actually holds the mic (Chrome vs Zoom vs Slack); no TCC permission needed. Compiled best-effort by bundle.mjs; exits when stdin closes - core meetings/detector: 5s mic debounce, PID→process attribution with running-app fallback, Granola labels (Huddle/Call/Meeting detected), calendar merge (event start −15min through end), once per mic session (30s idle reset), suppressed while Rowboat itself captures (recording or assistant voice call), gated by a meeting_detection notification category (Settings > Notifications) - popup: lean Granola-style bar (title + platform + Take notes pill, floating ×) in a transparent non-activating panel at screen-saver level with fullscreen-auxiliary behavior — floats over fullscreen meetings on the cursor's display; activation policy restored immediately so the Dock icon survives and Take Notes can foreground Rowboat - Take Notes routes into the existing take-meeting-notes flow with a synthetic or matched calendar event (source: detected); never records without a click - tray icon bumped to 18pt --- apps/x/apps/main/bundle.mjs | 20 + apps/x/apps/main/native/mic-monitor.swift | 123 ++++++ apps/x/apps/main/src/ipc.ts | 28 ++ apps/x/apps/main/src/main.ts | 42 +- apps/x/apps/main/src/meeting-popup.ts | 156 +++++++ apps/x/apps/main/src/tray.ts | 16 +- apps/x/apps/renderer/src/App.tsx | 11 +- .../src/components/meeting-detected-popup.tsx | 88 ++++ .../src/components/settings-dialog.tsx | 7 +- apps/x/apps/renderer/src/main.tsx | 8 + apps/x/packages/core/src/meetings/detector.ts | 406 ++++++++++++++++++ apps/x/packages/shared/src/ipc.ts | 45 ++ .../shared/src/notification-settings.ts | 4 + 13 files changed, 942 insertions(+), 12 deletions(-) create mode 100644 apps/x/apps/main/native/mic-monitor.swift create mode 100644 apps/x/apps/main/src/meeting-popup.ts create mode 100644 apps/x/apps/renderer/src/components/meeting-detected-popup.tsx create mode 100644 apps/x/packages/core/src/meetings/detector.ts diff --git a/apps/x/apps/main/bundle.mjs b/apps/x/apps/main/bundle.mjs index 548e037c..696084f3 100644 --- a/apps/x/apps/main/bundle.mjs +++ b/apps/x/apps/main/bundle.mjs @@ -91,6 +91,26 @@ if (!fs.existsSync(stagedBinary)) { } console.log('✅ node-pty staged in .package/node_modules'); +// Compile the mic-monitor helper (ambient meeting detection) on macOS. +// Best-effort: without swiftc — or on other platforms — the app still works, +// ad-hoc meeting detection just stays off (main checks the binary exists). +if (process.platform === 'darwin') { + const swiftSrc = path.join(here, 'native', 'mic-monitor.swift'); + const helperOut = path.join(here, '.package', 'dist', 'mic-monitor'); + const upToDate = fs.existsSync(helperOut) && + fs.statSync(helperOut).mtimeMs >= fs.statSync(swiftSrc).mtimeMs; + if (upToDate) { + console.log('✅ mic-monitor helper up to date'); + } else { + try { + execSync(`swiftc -O "${swiftSrc}" -o "${helperOut}"`, { stdio: 'inherit' }); + console.log('✅ mic-monitor helper compiled'); + } catch { + console.warn('⚠️ mic-monitor helper not built (swiftc unavailable?) — meeting detection disabled'); + } + } +} + // Bundle the vendored agent-slack CLI into a single self-contained script next // to main.cjs. It runs as a child process (process.execPath with // ELECTRON_RUN_AS_NODE=1), so it must exist as a real file on disk — it can't diff --git a/apps/x/apps/main/native/mic-monitor.swift b/apps/x/apps/main/native/mic-monitor.swift new file mode 100644 index 00000000..bb923485 --- /dev/null +++ b/apps/x/apps/main/native/mic-monitor.swift @@ -0,0 +1,123 @@ +// mic-monitor: prints a JSON line whenever the set of processes using the +// microphone changes. +// +// This is the ambient meeting-detection signal (Granola-style): when another +// app (Zoom, Meet in a browser, Slack huddle, FaceTime…) opens the +// microphone, we report it. No audio is captured, so this requires no +// microphone permission (TCC) — it's device state, not content. +// +// Two signals, best available wins: +// - macOS 14.4+: per-process audio objects (kAudioHardwarePropertyProcessObjectList +// + kAudioProcessPropertyIsRunningInput) give the PIDs that own the mic, +// so the consumer can attribute the call to Chrome vs Zoom vs Slack. +// - Fallback: kAudioDevicePropertyDeviceIsRunningSomewhere on the default +// input device — mic in use by *someone*, no attribution. +// +// Protocol: one JSON object per line on stdout, emitted on every state +// change (and once at startup): {"micInUse":true|false,"pids":[123,456]} +// ("pids" is empty when attribution is unavailable.) +// The process exits when stdin closes, so it can never outlive the app. +// +// Compiled by apps/main/bundle.mjs (best-effort, macOS only) to +// .package/dist/mic-monitor. + +import Foundation +import CoreAudio + +func defaultInputDeviceID() -> AudioDeviceID? { + var deviceID = AudioDeviceID(0) + var size = UInt32(MemoryLayout.size) + var addr = AudioObjectPropertyAddress( + mSelector: kAudioHardwarePropertyDefaultInputDevice, + mScope: kAudioObjectPropertyScopeGlobal, + mElement: kAudioObjectPropertyElementMain) + let status = AudioObjectGetPropertyData( + AudioObjectID(kAudioObjectSystemObject), &addr, 0, nil, &size, &deviceID) + guard status == noErr, deviceID != kAudioObjectUnknown else { return nil } + return deviceID +} + +func isRunningSomewhere(_ device: AudioDeviceID) -> Bool { + var running = UInt32(0) + var size = UInt32(MemoryLayout.size) + var addr = AudioObjectPropertyAddress( + mSelector: kAudioDevicePropertyDeviceIsRunningSomewhere, + mScope: kAudioObjectPropertyScopeGlobal, + mElement: kAudioObjectPropertyElementMain) + let status = AudioObjectGetPropertyData(device, &addr, 0, nil, &size, &running) + return status == noErr && running != 0 +} + +func audioProcessObjectIDs() -> [AudioObjectID] { + var addr = AudioObjectPropertyAddress( + mSelector: kAudioHardwarePropertyProcessObjectList, + mScope: kAudioObjectPropertyScopeGlobal, + mElement: kAudioObjectPropertyElementMain) + var size = UInt32(0) + let sysID = AudioObjectID(kAudioObjectSystemObject) + guard AudioObjectGetPropertyDataSize(sysID, &addr, 0, nil, &size) == noErr, size > 0 else { + return [] + } + var ids = [AudioObjectID](repeating: 0, count: Int(size) / MemoryLayout.size) + guard AudioObjectGetPropertyData(sysID, &addr, 0, nil, &size, &ids) == noErr else { return [] } + return ids +} + +func processIsRunningInput(_ object: AudioObjectID) -> Bool { + var addr = AudioObjectPropertyAddress( + mSelector: kAudioProcessPropertyIsRunningInput, + mScope: kAudioObjectPropertyScopeGlobal, + mElement: kAudioObjectPropertyElementMain) + var value = UInt32(0) + var size = UInt32(MemoryLayout.size) + let status = AudioObjectGetPropertyData(object, &addr, 0, nil, &size, &value) + return status == noErr && value != 0 +} + +func processPID(_ object: AudioObjectID) -> Int32? { + var addr = AudioObjectPropertyAddress( + mSelector: kAudioProcessPropertyPID, + mScope: kAudioObjectPropertyScopeGlobal, + mElement: kAudioObjectPropertyElementMain) + var pid = pid_t(-1) + var size = UInt32(MemoryLayout.size) + guard AudioObjectGetPropertyData(object, &addr, 0, nil, &size, &pid) == noErr, pid >= 0 else { + return nil + } + return Int32(pid) +} + +/// PIDs of processes currently capturing from any input device (macOS 14.4+; +/// returns [] where unsupported and the device-level fallback takes over). +func micOwningPIDs() -> [Int32] { + var pids: [Int32] = [] + for object in audioProcessObjectIDs() where processIsRunningInput(object) { + if let pid = processPID(object) { pids.append(pid) } + } + return pids.sorted() +} + +// Exit when the parent closes our stdin (app quit/crash) — never orphan. +Thread { + while readLine(strippingNewline: false) != nil {} + exit(0) +}.start() + +setbuf(stdout, nil) + +var lastInUse: Bool? = nil +var lastPids: [Int32] = [] +while true { + let pids = micOwningPIDs() + // Re-resolve the default device every poll: the user can switch input + // devices (AirPods in/out) mid-session. + let deviceInUse = defaultInputDeviceID().map(isRunningSomewhere) ?? false + let inUse = deviceInUse || !pids.isEmpty + if inUse != lastInUse || pids != lastPids { + lastInUse = inUse + lastPids = pids + let pidList = pids.map(String.init).joined(separator: ",") + print("{\"micInUse\":\(inUse),\"pids\":[\(pidList)]}") + } + Thread.sleep(forTimeInterval: 1.0) +} diff --git a/apps/x/apps/main/src/ipc.ts b/apps/x/apps/main/src/ipc.ts index 746b80f3..85f6b047 100644 --- a/apps/x/apps/main/src/ipc.ts +++ b/apps/x/apps/main/src/ipc.ts @@ -67,7 +67,18 @@ import { syncSlackKnowledgeSources, triggerSync as triggerSlackKnowledgeSync, ge import { isOnboardingComplete, markOnboardingComplete } from '@x/core/dist/config/note_creation_config.js'; import { loadNotificationSettings, saveNotificationSettings } from '@x/core/dist/config/notification_config.js'; import { saveAppSettings } from '@x/core/dist/config/app_settings.js'; +import { setSelfCaptureActive } from '@x/core/dist/meetings/detector.js'; import { consumePendingToggleMeetingNotes, setTrayRecordingState } from './tray.js'; +import { closeMeetingPopup, getMeetingPopupPayload, handleMeetingPopupAction } from './meeting-popup.js'; + +// Ambient meeting detection must ignore Rowboat's own mic use: meeting +// capture and assistant voice/video calls both hold the mic. Either being +// active suppresses "Meeting detected" prompts. +let meetingRecordingActive = false; +let voiceCallActive = false; +function updateSelfCaptureState() { + setSelfCaptureActive(meetingRecordingActive || voiceCallActive); +} import * as composioHandler from './composio-handler.js'; import * as appsIndexer from '@x/core/dist/apps/indexer.js'; import * as appsServer from '@x/core/dist/apps/server.js'; @@ -864,8 +875,25 @@ export function setupIpcHandlers() { }, 'meeting:setRecordingState': async (_event, args) => { setTrayRecordingState(args.recording); + meetingRecordingActive = args.recording; + updateSelfCaptureState(); + // Recording started through another path — a lingering "Take Notes?" + // popup is stale now. + if (args.recording) closeMeetingPopup(); return { success: true as const }; }, + 'voice:setCallActive': async (_event, args) => { + voiceCallActive = args.active; + updateSelfCaptureState(); + return { success: true as const }; + }, + 'meetingDetect:getPayload': async () => { + return { payload: getMeetingPopupPayload() }; + }, + 'meetingDetect:action': async (_event, args) => { + handleMeetingPopupAction(args.action); + return {}; + }, 'analytics:bootstrap': async () => { return { installationId: getInstallationId(), diff --git a/apps/x/apps/main/src/main.ts b/apps/x/apps/main/src/main.ts index 650c6cb9..cd4c18ba 100644 --- a/apps/x/apps/main/src/main.ts +++ b/apps/x/apps/main/src/main.ts @@ -67,7 +67,9 @@ import { import { disconnectGoogleIfScopesStale } from "./oauth-handler.js"; import { startModelsDevRefresh } from "@x/core/dist/models/models-dev.js"; import { loadAppSettings, saveAppSettings } from "@x/core/dist/config/app_settings.js"; -import { createAppTray, hasTray, markPendingToggleMeetingNotes } from "./tray.js"; +import { init as initMeetingDetection } from "@x/core/dist/meetings/detector.js"; +import { createAppTray, hasTray, isRecordingActive, markPendingToggleMeetingNotes } from "./tray.js"; +import { initMeetingPopup, showMeetingPopup } from "./meeting-popup.js"; // Captured as early as possible so it reflects actual process start. Used to // gate grace-eligible notifications (e.g. the burst of background-task @@ -273,9 +275,12 @@ function showApp(): void { if (!mainWindow.isVisible()) mainWindow.maximize(); mainWindow.show(); mainWindow.focus(); - return; + } else { + createWindow(); } - createWindow(); + // The user is usually in another app (a meeting!) when this runs — a plain + // focus() won't take the foreground from it. + app.focus({ steal: true }); } /** @@ -504,6 +509,37 @@ app.whenReady().then(async () => { }, }); + // Ambient meeting detection (Granola-style): the mic-monitor helper + + // running-app scan produce "Meeting detected" events; the popup asks + // before anything records. Clicking "Take Notes" routes into the same + // renderer flow as the calendar notification. + initMeetingPopup({ + onTakeNotes: (meeting) => { + showApp(); + // The user may have started recording between popup and click — + // sending the take-notes flow then would toggle it OFF. + if (isRecordingActive()) return; + const payload = { + event: meeting.calendarEvent ?? { summary: meeting.noteTitle }, + openMeeting: false, + source: "detected", + }; + const win = mainWindow; + if (!win || win.isDestroyed()) return; + if (win.webContents.isLoading()) { + win.webContents.once("did-finish-load", () => { + if (!win.isDestroyed()) win.webContents.send("app:takeMeetingNotes", payload); + }); + return; + } + win.webContents.send("app:takeMeetingNotes", payload); + }, + }); + initMeetingDetection({ + helperPath: path.join(__dirname, "mic-monitor"), + onDetected: (meeting) => showMeetingPopup(meeting), + }); + // Start workspace watcher as a main-process service // Watcher runs independently and catches ALL filesystem changes: // - Changes made via IPC handlers (workspace:writeFile, etc.) diff --git a/apps/x/apps/main/src/meeting-popup.ts b/apps/x/apps/main/src/meeting-popup.ts new file mode 100644 index 00000000..7284578b --- /dev/null +++ b/apps/x/apps/main/src/meeting-popup.ts @@ -0,0 +1,156 @@ +import { app, BrowserWindow, screen } from "electron"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import type { DetectedMeeting } from "@x/core/dist/meetings/detector.js"; + +/** + * The "Meeting detected — Take Notes?" popup: a small frameless panel in the + * top-left corner of the display the user is on. A macOS panel (NSPanel) at + * screen-saver level with fullscreen-auxiliary behavior, so it floats over + * whatever the user is looking at — including a fullscreen Meet/Zoom — and + * appears without stealing focus. Auto-dismisses after a while and never + * records anything by itself: clicking "Take Notes" hands off to the main + * window's existing take-meeting-notes flow. + */ + +// Lean bar + margins for the overhanging × and the CSS drop shadow. +const POPUP_WIDTH = 448; +const POPUP_HEIGHT = 96; +const AUTO_DISMISS_MS = 45_000; + +// Display names, Granola-style ("Chrome", not "Google Chrome"). +const SHORT_APP_NAMES: Record = { + "Google Chrome": "Chrome", + "Microsoft Edge": "Edge", + "Brave Browser": "Brave", + "Microsoft Teams": "Teams", +}; + +export interface MeetingPopupPayload { + title: string; + message: string; + hasCalendarEvent: boolean; +} + +let popupWin: BrowserWindow | null = null; +let currentPayload: MeetingPopupPayload | null = null; +let currentMeeting: DetectedMeeting | null = null; +let dismissTimer: NodeJS.Timeout | null = null; +let onTakeNotes: ((meeting: DetectedMeeting) => void) | null = null; + +/** Main registers the take-notes handoff once at startup. */ +export function initMeetingPopup(handlers: { onTakeNotes: (meeting: DetectedMeeting) => void }): void { + onTakeNotes = handlers.onTakeNotes; +} + +export function getMeetingPopupPayload(): MeetingPopupPayload | null { + return currentPayload; +} + +export function handleMeetingPopupAction(action: "take-notes" | "dismiss"): void { + const meeting = currentMeeting; + closeMeetingPopup(); + if (action === "take-notes" && meeting) { + onTakeNotes?.(meeting); + } +} + +export function closeMeetingPopup(): void { + if (dismissTimer) { + clearTimeout(dismissTimer); + dismissTimer = null; + } + currentPayload = null; + currentMeeting = null; + if (popupWin && !popupWin.isDestroyed()) popupWin.destroy(); + popupWin = null; +} + +export function showMeetingPopup(meeting: DetectedMeeting): void { + const eventSummary = + typeof meeting.calendarEvent?.summary === "string" + ? (meeting.calendarEvent.summary as string).trim() + : ""; + const appLabel = SHORT_APP_NAMES[meeting.appName] ?? meeting.appName; + const payload: MeetingPopupPayload = { + // Lean two-liner: "Meeting detected" / "Chrome" — or the event name + // over the platform when a calendar event is happening now. + title: eventSummary ? eventSummary : meeting.title, + message: appLabel, + hasCalendarEvent: Boolean(meeting.calendarEvent), + }; + + // Replace any popup that's still up (stale detection loses to fresh). + closeMeetingPopup(); + currentPayload = payload; + currentMeeting = meeting; + + // Top-left of the display the user is actually on (cursor display), not + // necessarily the primary one. + const display = screen.getDisplayNearestPoint(screen.getCursorScreenPoint()); + const workArea = display.workArea; + const popupDir = path.dirname(fileURLToPath(import.meta.url)); + const preloadPath = app.isPackaged + ? path.join(popupDir, "../preload/dist/preload.js") + : path.join(popupDir, "../../../preload/dist/preload.js"); + + const win = new BrowserWindow({ + width: POPUP_WIDTH, + height: POPUP_HEIGHT, + x: workArea.x + 24, + y: workArea.y + 24, + // NSPanel (macOS): non-activating, and — unlike a regular window with + // visibleOnFullScreen — can float over fullscreen Spaces without + // turning Rowboat into an "agent" app that loses its Dock icon. + ...(process.platform === "darwin" ? { type: "panel" as const } : {}), + frame: false, + resizable: false, + alwaysOnTop: true, + skipTaskbar: true, + show: false, + // Transparent window: the rounded card + overhanging × render inside; + // the shadow is CSS (native shadows artifact on transparent windows). + transparent: true, + hasShadow: false, + webPreferences: { + nodeIntegration: false, + contextIsolation: true, + sandbox: true, + preload: preloadPath, + }, + }); + // Screen-saver level + fullscreen-auxiliary: visible over fullscreen + // meeting apps on every workspace — the whole point of the popup is to be + // seen while the user is IN the meeting, wherever that is. + win.setAlwaysOnTop(true, "screen-saver"); + win.setVisibleOnAllWorkspaces(true, { visibleOnFullScreen: true }); + // visibleOnFullScreen flips the app's activation policy to "accessory": + // the Dock icon disappears and the app can no longer take foreground + // focus — clicking "Take Notes" then hands focus to whatever is next in + // the window stack instead of Rowboat. Restore the policy immediately: + // the popup keeps its fullscreen-auxiliary collection behavior (it's a + // non-activating panel, same trick as Zoom's floating controls), while + // Rowboat stays a regular app. + if (process.platform === "darwin") { + void app.dock?.show(); + } + win.webContents.once("did-finish-load", () => { + if (win.isDestroyed()) return; + win.webContents.send("meetingDetect:payload", payload); + // showInactive: the user is in a meeting app right now — appearing + // must not steal focus from it. + win.showInactive(); + }); + win.on("closed", () => { + if (popupWin === win) popupWin = null; + }); + popupWin = win; + + if (app.isPackaged) { + win.loadURL("app://-/index.html#meeting-detected"); + } else { + win.loadURL("http://localhost:5173/#meeting-detected"); + } + + dismissTimer = setTimeout(() => closeMeetingPopup(), AUTO_DISMISS_MS); +} diff --git a/apps/x/apps/main/src/tray.ts b/apps/x/apps/main/src/tray.ts index 6882ec9b..44a65791 100644 --- a/apps/x/apps/main/src/tray.ts +++ b/apps/x/apps/main/src/tray.ts @@ -11,10 +11,10 @@ import { app, Menu, Tray, nativeImage } from "electron"; * packaged layouts. Template rendering makes macOS tint it correctly * in light/dark menu bars and while highlighted. */ -const TRAY_ICON_16 = - "iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAABD0lEQVR4nKSSz+pBQRTH585vfgtrllYiSkpJHkAegLJQlvIE9soD2MvSE3gCG57ARiwoe2Kl/Lm+J2d0XJd7y7c+dc898z1zzswYFV6OwAU3+zNIf9LglwwyWqKgBKogBpZAa+XfKukKIqABxmABJqAPcnat8Zi1aLUNOiAl8hfeYMaxazw7kzkNRqAoOpFjHcCc45sWZpqzxtVpxqPIac6TpmDP/5UtQMkkz1YGCZAHA89YDndnv1+u8R+c1buGoMVFtiADTjYpb+HMsSNy9N3llinusfnpC/OQSBuwAwWO7Xko88VkDzarHo+owvHLpkEFSHXQBCtR9G3RJ9HBxsHazxymgFzn+iWM+sFMugMAAP//whjxNQAAAAZJREFUAwCzrTaLm3OP9AAAAABJRU5ErkJggg=="; -const TRAY_ICON_32 = - "iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAACfUlEQVR4nMSXO2gUURSGz8yuYuGjsBHFBza+UGwULUxjoY0iWPnA0lZQURDsFC1E7Owt7JQ0SZGQLk2SNoQEQhICCXlUCXk/ZvOf3HM2Z2/uZDM7m90fPnbm3jn3P/e5M0VqsorUZBWocYqogQnEhj1VzwTUsOTBOgJOg1WwYYMiyqdITDe9RK6CW+A2uAYugOPgBhiXuO3kal2E1pg5BO6BJ+A+uEK7OzcJpuVaR6amBArG+Bx4BZ6LqZUOdSI+I+SmoEJZEoilMTa+CN6Cl+CEMUrkuUgS5d91+e33OpApAQ06Cj6CN+CY1G14pqryPMt1b6jhagloI2z+EPwCl4xxgXabqnQXFCW+R8oTaxDvw5z1DbSL+Zpp2D4bpcTz7zAYMomVVaxizkP+FzwydYfJ9Uh3QskYWXGZrolOiamY/7QE1Pwk6AaXwTzoA7OghdyhQtJY7PVWzcnU/fPKK8xC5mdBB1gGv6UHY/IMr/rH4D25gyUtCS3n4efDaJ1Seuvf8xC/BhPgv1dnTz1+7qskogvSrhsu4wPqM/hCgeEPJUC0s99VBbkveWXa2A/wzktC45fILdxJL7kKM1+JV7cZCNRFyIYfwICXqA7/HzGPA22Ue5KmElWXjtYceCrXuiVXwDOp27OBPNJpaAVT5HaVTsVPcv98qb0nqs/7ABvw4XQTXJf7UfBCEkyqBeeV9u6UXHObvIsW9xOcNwFd2WfAHWnvO+iinUV5oAlo/ANyx3Yb+EQpez6keryS8QgMkjvp7oIFStnzIeX5LtAtyK9h3OOWrOZ5E9C/5PPk3gdnspoT5Z8Ce2xnNtegeqgmc1YjP82CavrHadMT2AIAAP//JKuLxQAAAAZJREFUAwAJ75pCeqjxVQAAAABJRU5ErkJggg=="; +const TRAY_ICON_18 = + "iVBORw0KGgoAAAANSUhEUgAAABIAAAASCAYAAABWzo5XAAABRElEQVR4nKySMS9EQRSF7773REEUlFpRiEg0Oj1RqTQqCiLRKNV+gkLvF5CoJRqJPyBEqyMURJB9zzl5Z3bvTuZtdjdu8mVm7sw9c+fOzWw0a4HMO3Ib3DJRaV0NK5QrqHLBM2AB/IAvOgrrn0EJ2pqvgg2N82AarIBX7jcJtSQyCXbALliKzryDJ83LIiEQnrAFTsBcOOxg3CN4C4FFQmQCnINN+b/BmPZDvTi/0T597SwS4dtvwTK4VPrj1mu5zl9oXQWBMDKTfXAPruSfBcfgwOqi03j5A1h0vo5QJ0W3Dr9GOwN7eiYzPASn/owXSgmEBmTfsLhT4NnqHvqMg2IrozkvewHXyvpIIj2xqYyajJncWf2bPuvGjFL76+ADbFu3WW0YIdaHvcYfWrOoLt4GeVquC3+t228jCfmzVb/b/sX+AAAA//+fzjrgAAAABklEQVQDAHsTRGNUkus5AAAAAElFTkSuQmCC"; +const TRAY_ICON_36 = + "iVBORw0KGgoAAAANSUhEUgAAACQAAAAkCAYAAADhAJiYAAAC3klEQVR4nOyYu4sUQRDGa3fm8DgxUxB8ICqKqBj4QBOFMxMDRQUfgS/0D1A0MRQMTA1MxBcngpFmBiocgokgCopo4PnCSE193e74FV11XTtMz+zs9sLB3Qc/drenp/ubrqruuUtpmimlaaYZb6gBmvK9DbKiDoNUU+bIxEClEoqvRIxkOYbBcrAJ7ALzwATlFiVWyDQULYE1F2wE28E2sBYsJr8I58Fjc18UQw2hLYOOgJ1gHxgFSwru+SsmXsnvjjzq1ZA1wgOuBMfBQXJhUbUF7c8akraJWIZ4yVsyEIfhjBgZket6TRPa5mkmbd/AZ9PWk6GGmXAhuABOgTnSPikmmoH79F7+/Rr8IV+BtQ3xJFq2HJpLYkqNJNS5EkXbiVYb65kZt1XXkIZoAbhKLmGtkdRMFNrXMmOA+4zn2qdUtTGqma1gDKwAv2XgtOD+MkOaV5w/XAS/ijqmXZjZA+6QT9ph00erLDGTFpnitkn5fCRmbBpUGtLYHgU3pe0teAKegnVgL7kqI+lrjwlrKh+ue1Si0NPwICfBFXAX3CaXiP9MP16V3eSqbXOFKb32BawmF/ZCNQvaeDDOlWVgvRgbFzOJgSd5QO5YuEyd+5OaUnRzHBMzwTM0lIRD5FcjofBpnZDPIzZ1jnz12Qdk8ZGxBnykgv2nyhDLnthl0oOVjb0AG8itlOYnPxg/4C1wjALJrCp7/agykjfF/b+CI9QZNq2ww+BH1UBNiiPNkYfgHflQtuT7NfCe/KoHFfMFTZN6PthBblU4bLwq+ymwEeYVa4WsXspnW8Y/C75TSSJbDeIlf1Q+eUe/T24PK01kq9gv+fyAb8Aq8IncK+xPudZVkcQKmY6zhfzBeYBc/nQVKlXskJ0Wc4fAc6oRKlWMkOkKLCUXphPgBvmqqz1Yv9K95SL4AK73aiaWIRYfDYvInVO1w0QDMKTqywwr5k5dq5pCGsTf9n1p9h9WVfoPAAD//0eu+ckAAAAGSURBVAMAjdCu7L3gD4wAAAAASUVORK5CYII="; interface TrayActions { openApp: () => void; @@ -44,11 +44,11 @@ function buildTrayIcon() { const icon = nativeImage.createEmpty(); icon.addRepresentation({ scaleFactor: 1, - buffer: Buffer.from(TRAY_ICON_16, "base64"), + buffer: Buffer.from(TRAY_ICON_18, "base64"), }); icon.addRepresentation({ scaleFactor: 2, - buffer: Buffer.from(TRAY_ICON_32, "base64"), + buffer: Buffer.from(TRAY_ICON_36, "base64"), }); icon.setTemplateImage(true); return icon; @@ -80,6 +80,10 @@ export function hasTray(): boolean { return tray !== null; } +export function isRecordingActive(): boolean { + return recording; +} + export function setTrayRecordingState(isRecording: boolean): void { if (recording === isRecording) return; recording = isRecording; diff --git a/apps/x/apps/renderer/src/App.tsx b/apps/x/apps/renderer/src/App.tsx index 5906b390..65a9c934 100644 --- a/apps/x/apps/renderer/src/App.tsx +++ b/apps/x/apps/renderer/src/App.tsx @@ -1083,6 +1083,13 @@ function App() { // floating popout; camera on → full-screen call; camera off → popout // (mascot pill). Handlers live below the voice/submit plumbing they drive. const video = useVideoMode() + // Assistant calls hold the mic — tell main so ambient meeting detection + // doesn't mistake our own capture for an external meeting. + useEffect(() => { + void window.ipc + .invoke('voice:setCallActive', { active: video.state !== 'idle' }) + .catch(() => { /* detection may be unavailable */ }) + }, [video.state]) const [inCall, setInCall] = useState(false) const inCallRef = useRef(false) // User explicitly shrank the full-screen call to the floating pill. @@ -4702,7 +4709,7 @@ function App() { // Reuses the same flow as the in-app "Join meeting & take notes" button. // When `openMeeting` is true, also opens the meeting URL in the system browser. useEffect(() => { - return window.ipc.on('app:takeMeetingNotes', ({ event, openMeeting }) => { + return window.ipc.on('app:takeMeetingNotes', ({ event, openMeeting, source }) => { const e = event as { summary?: string start?: { dateTime?: string; date?: string; timeZone?: string } @@ -4726,7 +4733,7 @@ function App() { location: e.location, htmlLink: e.htmlLink, conferenceLink, - source: 'calendar-sync', + source: source ?? 'calendar-sync', } window.dispatchEvent(new Event('calendar-block:join-meeting')) }) diff --git a/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx b/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx new file mode 100644 index 00000000..533f2cb3 --- /dev/null +++ b/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx @@ -0,0 +1,88 @@ +import { useEffect, useState } from 'react' +import { X } from 'lucide-react' + +type PopupPayload = { + title: string + message: string + hasCalendarEvent: boolean +} + +const dragRegion = { WebkitAppRegion: 'drag' } as React.CSSProperties +const noDragRegion = { WebkitAppRegion: 'no-drag' } as React.CSSProperties + +// Rowboat sail glyph (black on transparent, same asset as the tray icon) — +// rendered on a white chip inside the "Take notes" pill. +const SAIL_ICON = + 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAACfUlEQVR4nMSXO2gUURSGz8yuYuGjsBHFBza+UGwULUxjoY0iWPnA0lZQURDsFC1E7Owt7JQ0SZGQLk2SNoQEQhICCXlUCXk/ZvOf3HM2Z2/uZDM7m90fPnbm3jn3P/e5M0VqsorUZBWocYqogQnEhj1VzwTUsOTBOgJOg1WwYYMiyqdITDe9RK6CW+A2uAYugOPgBhiXuO3kal2E1pg5BO6BJ+A+uEK7OzcJpuVaR6amBArG+Bx4BZ6LqZUOdSI+I+SmoEJZEoilMTa+CN6Cl+CEMUrkuUgS5d91+e33OpApAQ06Cj6CN+CY1G14pqryPMt1b6jhagloI2z+EPwCl4xxgXabqnQXFCW+R8oTaxDvw5z1DbSL+Zpp2D4bpcTz7zAYMomVVaxizkP+FzwydYfJ9Uh3QskYWXGZrolOiamY/7QE1Pwk6AaXwTzoA7OghdyhQtJY7PVWzcnU/fPKK8xC5mdBB1gGv6UHY/IMr/rH4D25gyUtCS3n4efDaJ1Seuvf8xC/BhPgv1dnTz1+7qskogvSrhsu4wPqM/hCgeEPJUC0s99VBbkveWXa2A/wzktC45fILdxJL7kKM1+JV7cZCNRFyIYfwICXqA7/HzGPA22Ue5KmElWXjtYceCrXuiVXwDOp27OBPNJpaAVT5HaVTsVPcv98qb0nqs/7ABvw4XQTXJf7UfBCEkyqBeeV9u6UXHObvIsW9xOcNwFd2WfAHWnvO+iinUV5oAlo/ANyx3Yb+EQpez6keryS8QgMkjvp7oIFStnzIeX5LtAtyK9h3OOWrOZ5E9C/5PPk3gdnspoT5Z8Ce2xnNtegeqgmc1YjP82CavrHadMT2AIAAP//JKuLxQAAAAZJREFUAwAJ75pCeqjxVQAAAABJRU5ErkJggg==' + +/** + * Content of the "Meeting detected" always-on-top popup panel (see + * `showMeetingPopup` in the main process). Granola-style consent prompt — + * a lean horizontal bar: dashed rule (ad-hoc) or solid accent (calendar + * match), "Meeting detected" + platform, and a "Take notes" pill. The × + * floats on the card's top-left corner; the window itself is transparent. + * Detection never records — the user has to click "Take notes". + */ +export function MeetingDetectedPopup() { + const [payload, setPayload] = useState(null) + + useEffect(() => { + const cleanup = window.ipc.on('meetingDetect:payload', (next) => setPayload(next)) + // The main process pushes on did-finish-load, but that can race this + // listener's registration — fetch explicitly too. + window.ipc + .invoke('meetingDetect:getPayload', null) + .then(({ payload: cached }) => { + if (cached) setPayload(cached) + }) + .catch(() => {}) + return cleanup + }, []) + + const act = (action: 'take-notes' | 'dismiss') => { + void window.ipc.invoke('meetingDetect:action', { action }).catch(() => {}) + } + + return ( +

+ {/* Close — floats over the card's top-left corner */} + + + {/* Card */} +
+
+
+
+ {payload?.title ?? 'Meeting detected'} +
+
+ {payload?.message ?? ''} +
+
+ +
+
+ ) +} diff --git a/apps/x/apps/renderer/src/components/settings-dialog.tsx b/apps/x/apps/renderer/src/components/settings-dialog.tsx index a8ba8416..ce223cbd 100644 --- a/apps/x/apps/renderer/src/components/settings-dialog.tsx +++ b/apps/x/apps/renderer/src/components/settings-dialog.tsx @@ -2149,7 +2149,7 @@ function CodeModeSettings({ dialogOpen }: { dialogOpen: boolean }) { // --- Notification Settings --- -type NotificationCategoryKey = "chat_completion" | "new_email" | "agent_permission" | "background_task" +type NotificationCategoryKey = "chat_completion" | "new_email" | "agent_permission" | "background_task" | "meeting_detection" const NOTIFICATION_CATEGORIES: { key: NotificationCategoryKey; label: string; description: string }[] = [ { @@ -2172,6 +2172,11 @@ const NOTIFICATION_CATEGORIES: { key: NotificationCategoryKey; label: string; de label: "Background agents", description: "When a background agent you've set up has something to surface. Click to open it on the background tasks page.", }, + { + key: "meeting_detection", + label: "Meeting detection", + description: "A popup offering to take notes when Rowboat notices you're in a call or meeting. Nothing records until you accept.", + }, ] function NotificationSettings({ dialogOpen }: { dialogOpen: boolean }) { diff --git a/apps/x/apps/renderer/src/main.tsx b/apps/x/apps/renderer/src/main.tsx index ddd4c27e..28895675 100644 --- a/apps/x/apps/renderer/src/main.tsx +++ b/apps/x/apps/renderer/src/main.tsx @@ -7,6 +7,7 @@ import type { CaptureResult } from 'posthog-js' import { ThemeProvider } from '@/contexts/theme-context' import { configureAnalyticsContext } from './lib/analytics' import { VideoPopout } from '@/components/video-popout' +import { MeetingDetectedPopup } from '@/components/meeting-detected-popup' // Fetch the stable installation ID from main so renderer + main share one // PostHog distinct_id. Falls back to PostHog's auto-generated anonymous ID @@ -66,6 +67,13 @@ if (window.location.hash === '#video-popout') { , ) +} else if (window.location.hash === '#meeting-detected') { + // "Meeting detected — Take Notes?" popup window; same pattern. + createRoot(document.getElementById('root')!).render( + + + , + ) } else { bootstrap() } diff --git a/apps/x/packages/core/src/meetings/detector.ts b/apps/x/packages/core/src/meetings/detector.ts new file mode 100644 index 00000000..d3da04e3 --- /dev/null +++ b/apps/x/packages/core/src/meetings/detector.ts @@ -0,0 +1,406 @@ +import { spawn, execFile } from "node:child_process"; +import { existsSync } from "node:fs"; +import fs from "node:fs/promises"; +import path from "node:path"; +import { WorkDir } from "../config/config.js"; +import { isNotificationCategoryEnabled } from "../config/notification_config.js"; + +/** + * Ambient meeting detection (Granola-style). + * + * Signal: the mic-monitor helper (apps/main/native/mic-monitor.swift) reports + * when ANY process starts using the microphone — with the owning PIDs on + * macOS 14.4+, so the call is attributed to the app that actually holds the + * mic (Chrome vs Zoom vs Slack), not just whatever meeting app happens to be + * running. When the mic has been in use continuously for a few seconds — and + * it isn't Rowboat's own capture — we label the popup ("Huddle detected" for + * Slack, "Call detected" for FaceTime/WhatsApp, "Meeting detected" + * otherwise), merge with a calendar event whose window covers now + * (start − 15 min through end), and emit a DetectedMeeting. + * + * Never auto-records: the consumer (main process) shows a "Take Notes?" + * popup and waits for a click. + * + * Fires at most once per continuous mic-in-use session; the session resets + * after the mic has been idle for MIC_SESSION_RESET_MS. + */ + +const POLL_INTERVAL_MS = 2_000; +// Mic must be in use continuously this long before we prompt — filters out +// Siri, dictation bursts, and app mic-permission probes. +const MIC_DEBOUNCE_MS = 5_000; +// Mic idle this long ends the "session" and re-arms the prompt. +const MIC_SESSION_RESET_MS = 30_000; +// A calendar event merges with a detected call from 15 min before its start +// (Granola merges ad-hoc calls within 15 minutes of a scheduled event). +const CALENDAR_MERGE_LEAD_MS = 15 * 60_000; +const CALENDAR_SYNC_DIR = path.join(WorkDir, "calendar_sync"); +const HELPER_MAX_RESTARTS = 3; + +export type DetectedMeetingKind = "huddle" | "call" | "meeting"; + +export interface DetectedMeeting { + kind: DetectedMeetingKind; + /** Popup title, Granola wording: "Huddle detected" / "Call detected" / "Meeting detected". */ + title: string; + /** Human label of the app that likely owns the call, e.g. "Slack", "Zoom", "Google Chrome". */ + appName: string; + /** Suggested note title when no calendar event matched, e.g. "Slack huddle". */ + noteTitle: string; + /** Raw calendar event JSON when a nearby event matched. */ + calendarEvent?: Record; +} + +// Matched against process names by case-insensitive prefix (Electron/Chromium +// apps capture through helper processes: "Google Chrome Helper (Renderer)", +// "Slack Helper"…). When mic-owner PIDs are available this is exact +// attribution; otherwise it's a running-app heuristic ordered by confidence: +// dedicated conferencing apps first, always-running chat apps after, browsers +// as the generic fallback. +const MEETING_APPS: Array<{ proc: string; app: string; kind: DetectedMeetingKind }> = [ + { proc: "zoom.us", app: "Zoom", kind: "meeting" }, + { proc: "MSTeams", app: "Microsoft Teams", kind: "meeting" }, + { proc: "Microsoft Teams", app: "Microsoft Teams", kind: "meeting" }, + { proc: "Webex", app: "Webex", kind: "meeting" }, + { proc: "FaceTime", app: "FaceTime", kind: "call" }, + { proc: "WhatsApp", app: "WhatsApp", kind: "call" }, + { proc: "Slack", app: "Slack", kind: "huddle" }, + { proc: "Discord", app: "Discord", kind: "call" }, +]; + +const BROWSERS = [ + "Google Chrome", + "Safari", + "Arc", + "Brave Browser", + "Microsoft Edge", + "Firefox", + "Dia", + "Comet", +]; + +const KIND_TITLES: Record = { + huddle: "Huddle detected", + call: "Call detected", + meeting: "Meeting detected", +}; + +interface DetectorOptions { + /** Absolute path to the compiled mic-monitor helper binary. */ + helperPath: string; + onDetected: (meeting: DetectedMeeting) => void; +} + +let started = false; +let selfCaptureActive = false; +let micInUse = false; +// PIDs currently capturing from the mic (macOS 14.4+; empty = unknown). +let micPids: number[] = []; +let micInUseSince: number | null = null; +let micIdleSince: number | null = null; +let sessionNotified = false; +let helperRestarts = 0; + +/** + * Rowboat's own capture (meeting recording, assistant voice/video call) also + * flips the mic-in-use signal — the consumer reports it here so we never + * prompt about our own audio. + */ +export function setSelfCaptureActive(active: boolean): void { + selfCaptureActive = active; + if (active) { + // Whatever mic session is in flight is ours — don't prompt when the + // user stops and the mic lingers. + sessionNotified = true; + } +} + +export function init(options: DetectorOptions): void { + if (started) return; + if (process.platform !== "darwin") return; + if (!existsSync(options.helperPath)) { + console.warn( + `[MeetingDetect] mic-monitor helper not found at ${options.helperPath} — ambient detection disabled`, + ); + return; + } + started = true; + console.log("[MeetingDetect] starting ambient meeting detection"); + + spawnHelper(options.helperPath); + setInterval(() => { + try { + tick(options.onDetected); + } catch (err) { + console.error("[MeetingDetect] tick failed:", err); + } + }, POLL_INTERVAL_MS); +} + +function spawnHelper(helperPath: string): void { + let child: ReturnType; + try { + child = spawn(helperPath, [], { stdio: ["pipe", "pipe", "ignore"] }); + } catch (err) { + console.error("[MeetingDetect] failed to spawn mic-monitor:", err); + return; + } + + let buffer = ""; + child.stdout?.setEncoding("utf8"); + child.stdout?.on("data", (chunk: string) => { + buffer += chunk; + let idx: number; + while ((idx = buffer.indexOf("\n")) >= 0) { + const line = buffer.slice(0, idx).trim(); + buffer = buffer.slice(idx + 1); + if (!line) continue; + try { + const parsed = JSON.parse(line); + if (typeof parsed?.micInUse === "boolean") { + micInUse = parsed.micInUse; + micPids = Array.isArray(parsed.pids) + ? parsed.pids.filter((p: unknown): p is number => typeof p === "number") + : []; + } + } catch { + // Ignore malformed lines. + } + } + }); + + child.on("exit", (code) => { + micInUse = false; + micPids = []; + if (helperRestarts < HELPER_MAX_RESTARTS) { + helperRestarts += 1; + const delay = 5_000 * helperRestarts; + console.warn( + `[MeetingDetect] mic-monitor exited (code ${code}); restart ${helperRestarts}/${HELPER_MAX_RESTARTS} in ${delay / 1000}s`, + ); + setTimeout(() => spawnHelper(helperPath), delay); + } else { + console.error("[MeetingDetect] mic-monitor kept exiting — ambient detection disabled"); + } + }); + child.on("error", (err) => { + console.error("[MeetingDetect] mic-monitor error:", err); + }); +} + +function tick(onDetected: DetectorOptions["onDetected"]): void { + const now = Date.now(); + + if (!micInUse) { + micInUseSince = null; + if (micIdleSince === null) micIdleSince = now; + if (sessionNotified && now - micIdleSince >= MIC_SESSION_RESET_MS) { + sessionNotified = false; + } + return; + } + + micIdleSince = null; + if (micInUseSince === null) micInUseSince = now; + if (sessionNotified) return; + if (selfCaptureActive) { + // Mark the session as handled so a call that started as ours doesn't + // prompt the moment our capture stops while the meeting app holds on. + sessionNotified = true; + return; + } + if (now - micInUseSince < MIC_DEBOUNCE_MS) return; + + // Claim the session before the async work — a slow ps/calendar scan must + // not let a second tick double-fire. + sessionNotified = true; + void detect(onDetected); +} + +async function detect(onDetected: DetectorOptions["onDetected"]): Promise { + if (!isNotificationCategoryEnabled("meeting_detection")) return; + + const source = await findLikelyMeetingApp(); + if (!source) { + // Mic in use but nothing call-capable is running (dictation, voice + // memo, unknown recorder) — stay quiet to avoid noise. + return; + } + + const calendarEvent = await findNearbyCalendarEvent(); + const eventSummary = + typeof calendarEvent?.summary === "string" ? calendarEvent.summary.trim() : ""; + + const meeting: DetectedMeeting = { + kind: source.kind, + title: KIND_TITLES[source.kind], + appName: source.app, + noteTitle: eventSummary || defaultNoteTitle(source), + ...(calendarEvent ? { calendarEvent } : {}), + }; + + console.log( + `[MeetingDetect] ${meeting.title} (app: ${meeting.appName}` + + (eventSummary ? `, calendar: "${eventSummary}")` : ")"), + ); + onDetected(meeting); +} + +function defaultNoteTitle(source: { app: string; kind: DetectedMeetingKind }): string { + if (source.kind === "huddle") return `${source.app} huddle`; + if (source.kind === "call") return `${source.app} call`; + return BROWSERS.includes(source.app) ? "Meeting" : `${source.app} meeting`; +} + +/** Match one process name to a platform, by case-insensitive prefix. */ +function matchProcessName(name: string): { app: string; kind: DetectedMeetingKind } | null { + const lower = name.toLowerCase(); + // Safari captures through WebKit's out-of-process media stack. + if (lower.startsWith("com.apple.webkit")) return { app: "Safari", kind: "meeting" }; + // Firefox media capture lives in plugin-container. + if (lower.startsWith("plugin-container")) return { app: "Firefox", kind: "meeting" }; + for (const candidate of MEETING_APPS) { + if (lower.startsWith(candidate.proc.toLowerCase())) { + return { app: candidate.app, kind: candidate.kind }; + } + } + for (const browser of BROWSERS) { + if (lower.startsWith(browser.toLowerCase())) return { app: browser, kind: "meeting" }; + } + return null; +} + +/** + * Attribute the call to an app. Exact when mic-owner PIDs are known: resolve + * their process names and match those (a Meet call in Chrome attributes to + * Chrome even while Slack/Zoom idle in the background). If owners are known + * but none is call-capable (voice memo, dictation, screen recorder), that's + * NOT a meeting — stay quiet. Only without PID info (pre-14.4 macOS) fall + * back to the running-app heuristic. + */ +async function findLikelyMeetingApp(): Promise<{ app: string; kind: DetectedMeetingKind } | null> { + if (micPids.length > 0) { + const owners = await processNamesForPids(micPids); + if (owners.length > 0) { + // Prefer dedicated apps over browsers when several own the mic. + const matches = owners + .map(matchProcessName) + .filter((m): m is NonNullable => m !== null); + const dedicated = matches.find((m) => !BROWSERS.includes(m.app)); + return dedicated ?? matches[0] ?? null; + } + } + + // No attribution available — running-app heuristic in table order + // (dedicated conferencing apps beat always-running chat apps beat browsers). + const names = await runningProcessNames(); + if (!names) return null; + const hasPrefix = (prefix: string) => { + const lower = prefix.toLowerCase(); + for (const name of names) { + if (name.toLowerCase().startsWith(lower)) return true; + } + return false; + }; + for (const candidate of MEETING_APPS) { + if (hasPrefix(candidate.proc)) return { app: candidate.app, kind: candidate.kind }; + } + for (const browser of BROWSERS) { + if (hasPrefix(browser)) return { app: browser, kind: "meeting" }; + } + return null; +} + +function processNamesForPids(pids: number[]): Promise { + return new Promise((resolve) => { + execFile( + "ps", + ["-o", "comm=", "-p", pids.join(",")], + { maxBuffer: 1024 * 1024 }, + (err, stdout) => { + if (err) { + resolve([]); + return; + } + const names: string[] = []; + for (const line of stdout.split("\n")) { + const trimmed = line.trim(); + if (!trimmed) continue; + // GUI apps list as full executable paths — use the basename. + names.push(path.basename(trimmed)); + } + resolve(names); + }, + ); + }); +} + +function runningProcessNames(): Promise | null> { + return new Promise((resolve) => { + execFile("ps", ["-axo", "comm="], { maxBuffer: 4 * 1024 * 1024 }, (err, stdout) => { + if (err) { + resolve(null); + return; + } + const names = new Set(); + for (const line of stdout.split("\n")) { + const trimmed = line.trim(); + if (!trimmed) continue; + // GUI apps list as full executable paths — match the basename. + names.add(path.basename(trimmed)); + } + resolve(names); + }); + }); +} + +interface CalendarEvent { + status?: string; + summary?: string; + start?: { dateTime?: string }; + end?: { dateTime?: string }; + attendees?: Array<{ self?: boolean; responseStatus?: string }>; + [key: string]: unknown; +} + +/** + * The calendar event (if any) whose window covers "now": from 15 minutes + * before its start through its end. Skips all-day, cancelled, and + * self-declined events. Ties go to the event that started most recently. + */ +async function findNearbyCalendarEvent(): Promise { + let files: string[]; + try { + files = await fs.readdir(CALENDAR_SYNC_DIR); + } catch { + return null; + } + + const now = Date.now(); + let best: { event: CalendarEvent; startMs: number } | null = null; + + for (const name of files) { + if (!name.endsWith(".json") || name.startsWith("sync_state")) continue; + let event: CalendarEvent; + try { + event = JSON.parse(await fs.readFile(path.join(CALENDAR_SYNC_DIR, name), "utf-8")); + } catch { + continue; + } + if (event.status === "cancelled") continue; + const startStr = event.start?.dateTime; + if (!startStr) continue; // all-day + const self = event.attendees?.find((a) => a.self); + if (self?.responseStatus === "declined") continue; + + const startMs = Date.parse(startStr); + if (!Number.isFinite(startMs)) continue; + const endStr = event.end?.dateTime; + const endMs = endStr ? Date.parse(endStr) : startMs + 60 * 60_000; + + if (now < startMs - CALENDAR_MERGE_LEAD_MS || now > endMs) continue; + if (!best || startMs > best.startMs) best = { event, startMs }; + } + + return best?.event ?? null; +} diff --git a/apps/x/packages/shared/src/ipc.ts b/apps/x/packages/shared/src/ipc.ts index fbea4288..c6452399 100644 --- a/apps/x/packages/shared/src/ipc.ts +++ b/apps/x/packages/shared/src/ipc.ts @@ -734,6 +734,9 @@ const ipcSchemas = { // When true, the renderer should also open the meeting URL (Zoom/Meet/etc.) // in addition to triggering the take-notes flow. openMeeting: z.boolean().optional(), + // Origin recorded in the note frontmatter: 'calendar-sync' (default) for + // notification/deep-link starts, 'detected' for ambient meeting detection. + source: z.string().optional(), }), res: z.null(), }, @@ -783,6 +786,48 @@ const ipcSchemas = { success: z.literal(true), }), }, + // Renderer → main: assistant voice/video call holds the mic — suppresses + // ambient meeting detection (it would otherwise see our own capture). + 'voice:setCallActive': { + req: z.object({ + active: z.boolean(), + }), + res: z.object({ + success: z.literal(true), + }), + }, + // --- Ambient meeting detection popup (own always-on-top window) --- + // Main → popup: the detection to display. + 'meetingDetect:payload': { + req: z.object({ + title: z.string(), + message: z.string(), + // Calendar-linked detections render a solid accent bar; ad-hoc ones a + // dashed one (Granola's affordance). + hasCalendarEvent: z.boolean(), + }), + res: z.null(), + }, + // Popup → main: fetch the payload (the push can race listener registration). + 'meetingDetect:getPayload': { + req: z.null(), + res: z.object({ + payload: z + .object({ + title: z.string(), + message: z.string(), + hasCalendarEvent: z.boolean(), + }) + .nullable(), + }), + }, + // Popup → main: user clicked a button. + 'meetingDetect:action': { + req: z.object({ + action: z.enum(['take-notes', 'dismiss']), + }), + res: z.object({}), + }, 'granola:getConfig': { req: z.null(), res: z.object({ diff --git a/apps/x/packages/shared/src/notification-settings.ts b/apps/x/packages/shared/src/notification-settings.ts index 33608fb6..3a245bf2 100644 --- a/apps/x/packages/shared/src/notification-settings.ts +++ b/apps/x/packages/shared/src/notification-settings.ts @@ -7,12 +7,14 @@ import { z } from 'zod'; * - new_email: a new email arrived during incremental Gmail sync * - agent_permission: an agent is requesting permission to run a tool * - background_task: a background task agent pinged via the notify-user tool + * - meeting_detection: popup when Rowboat detects you're in a call/meeting */ export const NotificationCategorySchema = z.enum([ 'chat_completion', 'new_email', 'agent_permission', 'background_task', + 'meeting_detection', ]); export const NotificationCategoriesSchema = z.object({ @@ -20,6 +22,7 @@ export const NotificationCategoriesSchema = z.object({ new_email: z.boolean(), agent_permission: z.boolean(), background_task: z.boolean(), + meeting_detection: z.boolean(), }); export const NotificationSettingsSchema = z.object({ @@ -32,6 +35,7 @@ export const DEFAULT_NOTIFICATION_SETTINGS: NotificationSettings = { new_email: true, agent_permission: true, background_task: true, + meeting_detection: true, }, }; From 95f232cca9b9376289a1f98ff947ff46f8582902 Mon Sep 17 00:00:00 2001 From: Gagan Date: Tue, 14 Jul 2026 00:37:10 +0530 Subject: [PATCH 3/8] feat(x): auto-stop on call end + redirect to finished notes (Granola parity phase 3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - mic-monitor now resolves each mic owner's bundle ID (CoreAudio) and executable path (libproc) natively — the main process never shells out during capture (child-process spawns fail with EBADF while Electron holds a capture session; this bit the ps-based first cut) - detector matches owners by bundle-ID prefix (com.google.chrome, us.zoom, com.tinyspeck.slackmacgap, …) with executable-name fallback, and recognizes Rowboat's own capture (com.rowboat / dev electron) - call-end detection: once a meeting app has been seen on the mic during a recording, its absence for 3s means the call ended → auto-stop → summarize — same flow as a manual stop. Arms only when a meeting app was actually observed, so in-person recordings never auto-stop - post-summary redirect: Rowboat foregrounds itself on the finished note (app:focusMainWindow now steals focus); the new meeting_notes_ready notification stays as a background-only fallback with a toggle in Settings > Notifications --- apps/x/apps/main/native/mic-monitor.swift | 89 +++++-- apps/x/apps/main/src/ipc.ts | 17 ++ apps/x/apps/main/src/main.ts | 7 + apps/x/apps/renderer/src/App.tsx | 18 ++ .../src/components/settings-dialog.tsx | 7 +- apps/x/packages/core/src/meetings/detector.ts | 245 +++++++++++++----- apps/x/packages/shared/src/ipc.ts | 17 ++ .../shared/src/notification-settings.ts | 4 + 8 files changed, 320 insertions(+), 84 deletions(-) diff --git a/apps/x/apps/main/native/mic-monitor.swift b/apps/x/apps/main/native/mic-monitor.swift index bb923485..239de7ee 100644 --- a/apps/x/apps/main/native/mic-monitor.swift +++ b/apps/x/apps/main/native/mic-monitor.swift @@ -8,14 +8,18 @@ // // Two signals, best available wins: // - macOS 14.4+: per-process audio objects (kAudioHardwarePropertyProcessObjectList -// + kAudioProcessPropertyIsRunningInput) give the PIDs that own the mic, -// so the consumer can attribute the call to Chrome vs Zoom vs Slack. +// + kAudioProcessPropertyIsRunningInput) give the processes that own the +// mic. Each owner is resolved to its bundle ID (CoreAudio) and executable +// path (libproc) HERE, natively — the consumer must not need to shell out +// (child-process spawns from Electron's main process fail with EBADF +// while capture is active). // - Fallback: kAudioDevicePropertyDeviceIsRunningSomewhere on the default // input device — mic in use by *someone*, no attribution. // // Protocol: one JSON object per line on stdout, emitted on every state -// change (and once at startup): {"micInUse":true|false,"pids":[123,456]} -// ("pids" is empty when attribution is unavailable.) +// change (and once at startup): +// {"micInUse":true,"owners":[{"pid":123,"bundleId":"com.google.Chrome.helper","path":"/Applications/..."}]} +// ("owners" is empty when attribution is unavailable.) // The process exits when stdin closes, so it can never outlive the app. // // Compiled by apps/main/bundle.mjs (best-effort, macOS only) to @@ -23,6 +27,7 @@ import Foundation import CoreAudio +import Darwin func defaultInputDeviceID() -> AudioDeviceID? { var deviceID = AudioDeviceID(0) @@ -87,14 +92,64 @@ func processPID(_ object: AudioObjectID) -> Int32? { return Int32(pid) } -/// PIDs of processes currently capturing from any input device (macOS 14.4+; +func processBundleID(_ object: AudioObjectID) -> String? { + var addr = AudioObjectPropertyAddress( + mSelector: kAudioProcessPropertyBundleID, + mScope: kAudioObjectPropertyScopeGlobal, + mElement: kAudioObjectPropertyElementMain) + var value: Unmanaged? = nil + var size = UInt32(MemoryLayout?>.size) + guard AudioObjectGetPropertyData(object, &addr, 0, nil, &size, &value) == noErr, + let cf = value else { return nil } + let str = cf.takeRetainedValue() as String + return str.isEmpty ? nil : str +} + +func processPath(_ pid: Int32) -> String? { + var buffer = [CChar](repeating: 0, count: 4096) + let length = proc_pidpath(pid, &buffer, UInt32(buffer.count)) + guard length > 0 else { return nil } + return String(cString: buffer) +} + +struct MicOwner: Equatable { + let pid: Int32 + let bundleId: String + let path: String +} + +/// Processes currently capturing from any input device (macOS 14.4+; /// returns [] where unsupported and the device-level fallback takes over). -func micOwningPIDs() -> [Int32] { - var pids: [Int32] = [] +func micOwners() -> [MicOwner] { + var owners: [MicOwner] = [] for object in audioProcessObjectIDs() where processIsRunningInput(object) { - if let pid = processPID(object) { pids.append(pid) } + guard let pid = processPID(object) else { continue } + owners.append(MicOwner( + pid: pid, + bundleId: processBundleID(object) ?? "", + path: processPath(pid) ?? "")) } - return pids.sorted() + return owners.sorted { $0.pid < $1.pid } +} + +func jsonEscape(_ s: String) -> String { + var out = "" + for ch in s.unicodeScalars { + switch ch { + case "\"": out += "\\\"" + case "\\": out += "\\\\" + case "\n": out += "\\n" + case "\r": out += "\\r" + case "\t": out += "\\t" + default: + if ch.value < 0x20 { + out += String(format: "\\u%04x", ch.value) + } else { + out.unicodeScalars.append(ch) + } + } + } + return out } // Exit when the parent closes our stdin (app quit/crash) — never orphan. @@ -106,18 +161,20 @@ Thread { setbuf(stdout, nil) var lastInUse: Bool? = nil -var lastPids: [Int32] = [] +var lastOwners: [MicOwner] = [] while true { - let pids = micOwningPIDs() + let owners = micOwners() // Re-resolve the default device every poll: the user can switch input // devices (AirPods in/out) mid-session. let deviceInUse = defaultInputDeviceID().map(isRunningSomewhere) ?? false - let inUse = deviceInUse || !pids.isEmpty - if inUse != lastInUse || pids != lastPids { + let inUse = deviceInUse || !owners.isEmpty + if inUse != lastInUse || owners != lastOwners { lastInUse = inUse - lastPids = pids - let pidList = pids.map(String.init).joined(separator: ",") - print("{\"micInUse\":\(inUse),\"pids\":[\(pidList)]}") + lastOwners = owners + let ownerJson = owners.map { o in + "{\"pid\":\(o.pid),\"bundleId\":\"\(jsonEscape(o.bundleId))\",\"path\":\"\(jsonEscape(o.path))\"}" + }.joined(separator: ",") + print("{\"micInUse\":\(inUse),\"owners\":[\(ownerJson)]}") } Thread.sleep(forTimeInterval: 1.0) } diff --git a/apps/x/apps/main/src/ipc.ts b/apps/x/apps/main/src/ipc.ts index 85f6b047..ae258b49 100644 --- a/apps/x/apps/main/src/ipc.ts +++ b/apps/x/apps/main/src/ipc.ts @@ -68,6 +68,7 @@ import { isOnboardingComplete, markOnboardingComplete } from '@x/core/dist/confi import { loadNotificationSettings, saveNotificationSettings } from '@x/core/dist/config/notification_config.js'; import { saveAppSettings } from '@x/core/dist/config/app_settings.js'; import { setSelfCaptureActive } from '@x/core/dist/meetings/detector.js'; +import { notifyIfEnabled } from '@x/core/dist/application/notification/notifier.js'; import { consumePendingToggleMeetingNotes, setTrayRecordingState } from './tray.js'; import { closeMeetingPopup, getMeetingPopupPayload, handleMeetingPopupAction } from './meeting-popup.js'; @@ -887,6 +888,19 @@ export function setupIpcHandlers() { updateSelfCaptureState(); return { success: true as const }; }, + 'meeting:notifyNotesReady': async (_event, args) => { + // Granola-style re-entry point: the note refreshed in place, but the + // user has usually switched back to the meeting app — the notification + // brings them back. Suppressed while the app is focused. + void notifyIfEnabled('meeting_notes_ready', { + title: 'Meeting notes ready', + message: `Your notes for "${args.title}" are ready.`, + link: `rowboat://open?type=file&path=${encodeURIComponent(args.notePath)}`, + actionLabel: 'Open notes', + onlyWhenBackground: true, + }); + return { success: true as const }; + }, 'meetingDetect:getPayload': async () => { return { payload: getMeetingPopupPayload() }; }, @@ -2246,6 +2260,9 @@ export function setupIpcHandlers() { if (main.isMinimized()) main.restore(); main.show(); main.focus(); + // The user is typically in another app (e.g. just left a meeting) — + // a plain focus() won't take the foreground from it. + app.focus({ steal: true }); } return {}; }, diff --git a/apps/x/apps/main/src/main.ts b/apps/x/apps/main/src/main.ts index cd4c18ba..1ec99a0a 100644 --- a/apps/x/apps/main/src/main.ts +++ b/apps/x/apps/main/src/main.ts @@ -538,6 +538,13 @@ app.whenReady().then(async () => { initMeetingDetection({ helperPath: path.join(__dirname, "mic-monitor"), onDetected: (meeting) => showMeetingPopup(meeting), + // Call ended while recording (meeting app released the mic) — the + // renderer stops capture and generates notes, same as a manual stop. + onExternalCallEnded: () => { + const win = mainWindow; + if (!win || win.isDestroyed() || win.webContents.isLoading()) return; + win.webContents.send("meeting:externalCallEnded", null); + }, }); // Start workspace watcher as a main-process service diff --git a/apps/x/apps/renderer/src/App.tsx b/apps/x/apps/renderer/src/App.tsx index 65a9c934..99791ba9 100644 --- a/apps/x/apps/renderer/src/App.tsx +++ b/apps/x/apps/renderer/src/App.tsx @@ -1116,6 +1116,16 @@ function App() { .catch(() => { /* tray may be unavailable */ }) }, [meetingTranscription.state]) + // Main detected the meeting app released the mic (call ended) — stop and + // generate notes, exactly like a manual stop. Listener only exists while + // recording, so a stale signal can never toggle a new recording ON. + useEffect(() => { + if (meetingTranscription.state !== 'recording') return + return window.ipc.on('meeting:externalCallEnded', () => { + handleToggleMeetingRef.current?.() + }) + }, [meetingTranscription.state]) + // Check if voice is available on mount and when OAuth state changes const refreshVoiceAvailability = useCallback(() => { Promise.all([ @@ -5664,6 +5674,14 @@ function App() { }) // Refresh the file view await handleVoiceNoteCreated(notePath) + // Notes are done — bring Rowboat to the foreground on the + // finished note (the post-call "redirect"). The notification + // below is background-only, so it only fires if the focus + // grab didn't take. + void window.ipc.invoke('app:focusMainWindow', null).catch(() => {}) + void window.ipc + .invoke('meeting:notifyNotesReady', { notePath, title: noteTitle }) + .catch(() => { /* notification is best-effort */ }) } } } catch (err) { diff --git a/apps/x/apps/renderer/src/components/settings-dialog.tsx b/apps/x/apps/renderer/src/components/settings-dialog.tsx index ce223cbd..f4daa12c 100644 --- a/apps/x/apps/renderer/src/components/settings-dialog.tsx +++ b/apps/x/apps/renderer/src/components/settings-dialog.tsx @@ -2149,7 +2149,7 @@ function CodeModeSettings({ dialogOpen }: { dialogOpen: boolean }) { // --- Notification Settings --- -type NotificationCategoryKey = "chat_completion" | "new_email" | "agent_permission" | "background_task" | "meeting_detection" +type NotificationCategoryKey = "chat_completion" | "new_email" | "agent_permission" | "background_task" | "meeting_detection" | "meeting_notes_ready" const NOTIFICATION_CATEGORIES: { key: NotificationCategoryKey; label: string; description: string }[] = [ { @@ -2177,6 +2177,11 @@ const NOTIFICATION_CATEGORIES: { key: NotificationCategoryKey; label: string; de label: "Meeting detection", description: "A popup offering to take notes when Rowboat notices you're in a call or meeting. Nothing records until you accept.", }, + { + key: "meeting_notes_ready", + label: "Meeting notes ready", + description: "When your meeting notes finish generating after a call. Click to open the note. Only shown while the app is in the background.", + }, ] function NotificationSettings({ dialogOpen }: { dialogOpen: boolean }) { diff --git a/apps/x/packages/core/src/meetings/detector.ts b/apps/x/packages/core/src/meetings/detector.ts index d3da04e3..36a82b1f 100644 --- a/apps/x/packages/core/src/meetings/detector.ts +++ b/apps/x/packages/core/src/meetings/detector.ts @@ -9,20 +9,26 @@ import { isNotificationCategoryEnabled } from "../config/notification_config.js" * Ambient meeting detection (Granola-style). * * Signal: the mic-monitor helper (apps/main/native/mic-monitor.swift) reports - * when ANY process starts using the microphone — with the owning PIDs on - * macOS 14.4+, so the call is attributed to the app that actually holds the - * mic (Chrome vs Zoom vs Slack), not just whatever meeting app happens to be - * running. When the mic has been in use continuously for a few seconds — and - * it isn't Rowboat's own capture — we label the popup ("Huddle detected" for - * Slack, "Call detected" for FaceTime/WhatsApp, "Meeting detected" - * otherwise), merge with a calendar event whose window covers now - * (start − 15 min through end), and emit a DetectedMeeting. + * when ANY process starts using the microphone — with the owning processes' + * bundle IDs and executable paths on macOS 14.4+, resolved natively in the + * helper. That matters twice over: attribution is exact (a Meet call in + * Chrome attributes to Chrome even while Slack/Zoom idle in the background), + * and the main process never has to shell out while capture is active + * (child-process spawns from Electron's main fail with EBADF during capture). * - * Never auto-records: the consumer (main process) shows a "Take Notes?" - * popup and waits for a click. + * When the mic has been in use continuously for a few seconds — and it isn't + * Rowboat's own capture — we label the popup ("Huddle detected" for Slack, + * "Call detected" for FaceTime/WhatsApp, "Meeting detected" otherwise), + * merge with a calendar event whose window covers now (start − 15 min + * through end), and emit a DetectedMeeting. Never auto-records: the consumer + * (main process) shows a "Take Notes?" popup and waits for a click. * * Fires at most once per continuous mic-in-use session; the session resets * after the mic has been idle for MIC_SESSION_RESET_MS. + * + * While Rowboat records, the same owner list powers call-end detection: once + * a meeting app has been seen on the mic, its absence for CALL_END_GRACE_MS + * means the call ended. */ const POLL_INTERVAL_MS = 2_000; @@ -34,6 +40,10 @@ const MIC_SESSION_RESET_MS = 30_000; // A calendar event merges with a detected call from 15 min before its start // (Granola merges ad-hoc calls within 15 minutes of a scheduled event). const CALENDAR_MERGE_LEAD_MS = 15 * 60_000; +// While recording, the meeting app must be off the mic this long before we +// call the meeting over. Kept short so notes arrive right after hang-up; +// only guards against sub-poll device churn, not full reconnects. +const CALL_END_GRACE_MS = 3_000; const CALENDAR_SYNC_DIR = path.join(WorkDir, "calendar_sync"); const HELPER_MAX_RESTARTS = 3; @@ -51,12 +61,47 @@ export interface DetectedMeeting { calendarEvent?: Record; } -// Matched against process names by case-insensitive prefix (Electron/Chromium -// apps capture through helper processes: "Google Chrome Helper (Renderer)", -// "Slack Helper"…). When mic-owner PIDs are available this is exact -// attribution; otherwise it's a running-app heuristic ordered by confidence: -// dedicated conferencing apps first, always-running chat apps after, browsers -// as the generic fallback. +interface PlatformMatch { + app: string; + kind: DetectedMeetingKind; +} + +// Bundle-ID prefixes (lowercased) → platform. The primary matcher: exact and +// stable, covers helper processes too (com.google.Chrome.helper etc.). +const BUNDLE_MATCHERS: Array<{ prefix: string; app: string; kind: DetectedMeetingKind }> = [ + { prefix: "us.zoom", app: "Zoom", kind: "meeting" }, + { prefix: "com.microsoft.teams", app: "Microsoft Teams", kind: "meeting" }, + { prefix: "com.cisco.webex", app: "Webex", kind: "meeting" }, + { prefix: "com.webex", app: "Webex", kind: "meeting" }, + { prefix: "com.tinyspeck.slackmacgap", app: "Slack", kind: "huddle" }, + { prefix: "com.apple.facetime", app: "FaceTime", kind: "call" }, + { prefix: "net.whatsapp", app: "WhatsApp", kind: "call" }, + { prefix: "com.hnc.discord", app: "Discord", kind: "call" }, + // Browsers — generic "Meeting detected". + { prefix: "com.google.chrome", app: "Google Chrome", kind: "meeting" }, + { prefix: "com.apple.safari", app: "Safari", kind: "meeting" }, + { prefix: "com.apple.webkit", app: "Safari", kind: "meeting" }, + { prefix: "org.mozilla.firefox", app: "Firefox", kind: "meeting" }, + { prefix: "com.brave.browser", app: "Brave Browser", kind: "meeting" }, + { prefix: "com.microsoft.edgemac", app: "Microsoft Edge", kind: "meeting" }, + { prefix: "company.thebrowser", app: "Arc", kind: "meeting" }, +]; + +// Rowboat's own capture (and the dev Electron shell) — never a "meeting". +const SELF_BUNDLE_PREFIXES = ["com.rowboat", "com.github.electron"]; + +const BROWSER_APPS = new Set([ + "Google Chrome", + "Safari", + "Firefox", + "Brave Browser", + "Microsoft Edge", + "Arc", +]); + +// Process-name fallbacks (case-insensitive prefix on the executable +// basename): used when an owner has no bundle ID, and by the pre-14.4 +// running-app heuristic. Ordered by confidence for the heuristic. const MEETING_APPS: Array<{ proc: string; app: string; kind: DetectedMeetingKind }> = [ { proc: "zoom.us", app: "Zoom", kind: "meeting" }, { proc: "MSTeams", app: "Microsoft Teams", kind: "meeting" }, @@ -85,21 +130,38 @@ const KIND_TITLES: Record = { meeting: "Meeting detected", }; +interface MicOwner { + pid: number; + bundleId: string; + path: string; +} + interface DetectorOptions { /** Absolute path to the compiled mic-monitor helper binary. */ helperPath: string; onDetected: (meeting: DetectedMeeting) => void; + /** + * Fired once per recording session when the meeting app that was on the + * mic has released it for CALL_END_GRACE_MS while Rowboat is still + * capturing — i.e. the call ended. Needs per-process attribution + * (macOS 14.4+); silently unavailable otherwise. + */ + onExternalCallEnded?: () => void; } let started = false; let selfCaptureActive = false; let micInUse = false; -// PIDs currently capturing from the mic (macOS 14.4+; empty = unknown). -let micPids: number[] = []; +// Processes currently capturing from the mic (macOS 14.4+; empty = unknown). +let micOwners: MicOwner[] = []; let micInUseSince: number | null = null; let micIdleSince: number | null = null; let sessionNotified = false; let helperRestarts = 0; +// Call-end tracking for the current self-capture session. +let externalAppSeen = false; +let externalAbsentSince: number | null = null; +let callEndFired = false; /** * Rowboat's own capture (meeting recording, assistant voice/video call) also @@ -108,6 +170,10 @@ let helperRestarts = 0; */ export function setSelfCaptureActive(active: boolean): void { selfCaptureActive = active; + // Fresh capture session — re-arm call-end tracking. + externalAppSeen = false; + externalAbsentSince = null; + callEndFired = false; if (active) { // Whatever mic session is in flight is ours — don't prompt when the // user stops and the mic lingers. @@ -131,6 +197,7 @@ export function init(options: DetectorOptions): void { setInterval(() => { try { tick(options.onDetected); + checkExternalCallEnd(options.onExternalCallEnded); } catch (err) { console.error("[MeetingDetect] tick failed:", err); } @@ -159,8 +226,13 @@ function spawnHelper(helperPath: string): void { const parsed = JSON.parse(line); if (typeof parsed?.micInUse === "boolean") { micInUse = parsed.micInUse; - micPids = Array.isArray(parsed.pids) - ? parsed.pids.filter((p: unknown): p is number => typeof p === "number") + micOwners = Array.isArray(parsed.owners) + ? parsed.owners.filter( + (o: unknown): o is MicOwner => + typeof (o as MicOwner)?.pid === "number" && + typeof (o as MicOwner)?.bundleId === "string" && + typeof (o as MicOwner)?.path === "string", + ) : []; } } catch { @@ -171,7 +243,7 @@ function spawnHelper(helperPath: string): void { child.on("exit", (code) => { micInUse = false; - micPids = []; + micOwners = []; if (helperRestarts < HELPER_MAX_RESTARTS) { helperRestarts += 1; const delay = 5_000 * helperRestarts; @@ -211,18 +283,66 @@ function tick(onDetected: DetectorOptions["onDetected"]): void { } if (now - micInUseSince < MIC_DEBOUNCE_MS) return; - // Claim the session before the async work — a slow ps/calendar scan must + // Claim the session before the async work — a slow calendar scan must // not let a second tick double-fire. sessionNotified = true; void detect(onDetected); } +/** + * While Rowboat is recording, watch whether any known meeting app/browser + * still owns the mic. Once one has been seen, its absence for + * CALL_END_GRACE_MS means the call ended — fire once per session. Fully + * synchronous: reads the helper-provided owner list, spawns nothing. + */ +function checkExternalCallEnd(onExternalCallEnded?: () => void): void { + if (!onExternalCallEnded) return; + if (!selfCaptureActive || callEndFired) return; + // No attribution data (pre-14.4 macOS) — feature unavailable. + if (micOwners.length === 0) return; + + const now = Date.now(); + const externalOnMic = micOwners.some((owner) => { + const match = matchOwner(owner); + return match !== null && match !== "self"; + }); + if (externalOnMic) { + if (!externalAppSeen) { + console.log( + `[MeetingDetect] call-end watch armed — mic owners: ${describeOwners(micOwners)}`, + ); + } + externalAppSeen = true; + externalAbsentSince = null; + return; + } + if (!externalAppSeen) return; + if (externalAbsentSince === null) { + externalAbsentSince = now; + console.log( + `[MeetingDetect] meeting app off the mic — remaining owners: ${describeOwners(micOwners)}`, + ); + return; + } + if (now - externalAbsentSince >= CALL_END_GRACE_MS) { + callEndFired = true; + console.log("[MeetingDetect] meeting app released the mic — call likely ended"); + onExternalCallEnded(); + } +} + +function describeOwners(owners: MicOwner[]): string { + return owners + .map((o) => o.bundleId || path.basename(o.path) || `pid ${o.pid}`) + .join(", "); +} + async function detect(onDetected: DetectorOptions["onDetected"]): Promise { if (!isNotificationCategoryEnabled("meeting_detection")) return; const source = await findLikelyMeetingApp(); if (!source) { - // Mic in use but nothing call-capable is running (dictation, voice + // Mic in use but nothing call-capable owns it (dictation, voice // memo, unknown recorder) — stay quiet to avoid noise. return; } @@ -246,14 +366,33 @@ async function detect(onDetected: DetectorOptions["onDetected"]): Promise onDetected(meeting); } -function defaultNoteTitle(source: { app: string; kind: DetectedMeetingKind }): string { +function defaultNoteTitle(source: PlatformMatch): string { if (source.kind === "huddle") return `${source.app} huddle`; if (source.kind === "call") return `${source.app} call`; - return BROWSERS.includes(source.app) ? "Meeting" : `${source.app} meeting`; + return BROWSER_APPS.has(source.app) || BROWSERS.includes(source.app) + ? "Meeting" + : `${source.app} meeting`; +} + +/** Match a mic owner to a platform (or to Rowboat itself). */ +function matchOwner(owner: MicOwner): PlatformMatch | "self" | null { + const bundle = owner.bundleId.toLowerCase(); + if (bundle) { + for (const prefix of SELF_BUNDLE_PREFIXES) { + if (bundle.startsWith(prefix)) return "self"; + } + for (const matcher of BUNDLE_MATCHERS) { + if (bundle.startsWith(matcher.prefix)) { + return { app: matcher.app, kind: matcher.kind }; + } + } + } + const basename = path.basename(owner.path); + return basename ? matchProcessName(basename) : null; } /** Match one process name to a platform, by case-insensitive prefix. */ -function matchProcessName(name: string): { app: string; kind: DetectedMeetingKind } | null { +function matchProcessName(name: string): PlatformMatch | null { const lower = name.toLowerCase(); // Safari captures through WebKit's out-of-process media stack. if (lower.startsWith("com.apple.webkit")) return { app: "Safari", kind: "meeting" }; @@ -271,24 +410,20 @@ function matchProcessName(name: string): { app: string; kind: DetectedMeetingKin } /** - * Attribute the call to an app. Exact when mic-owner PIDs are known: resolve - * their process names and match those (a Meet call in Chrome attributes to - * Chrome even while Slack/Zoom idle in the background). If owners are known - * but none is call-capable (voice memo, dictation, screen recorder), that's - * NOT a meeting — stay quiet. Only without PID info (pre-14.4 macOS) fall - * back to the running-app heuristic. + * Attribute the call to an app. Exact when mic owners are known (bundle IDs + * from the helper, no child processes involved). Owners known but none + * call-capable (voice memo, dictation, screen recorder) → NOT a meeting, + * stay quiet. Only without attribution (pre-14.4 macOS) fall back to the + * running-app heuristic. */ -async function findLikelyMeetingApp(): Promise<{ app: string; kind: DetectedMeetingKind } | null> { - if (micPids.length > 0) { - const owners = await processNamesForPids(micPids); - if (owners.length > 0) { - // Prefer dedicated apps over browsers when several own the mic. - const matches = owners - .map(matchProcessName) - .filter((m): m is NonNullable => m !== null); - const dedicated = matches.find((m) => !BROWSERS.includes(m.app)); - return dedicated ?? matches[0] ?? null; - } +async function findLikelyMeetingApp(): Promise { + if (micOwners.length > 0) { + const matches = micOwners + .map(matchOwner) + .filter((m): m is PlatformMatch => m !== null && m !== "self"); + // Prefer dedicated apps over browsers when several own the mic. + const dedicated = matches.find((m) => !BROWSER_APPS.has(m.app)); + return dedicated ?? matches[0] ?? null; } // No attribution available — running-app heuristic in table order @@ -311,30 +446,6 @@ async function findLikelyMeetingApp(): Promise<{ app: string; kind: DetectedMeet return null; } -function processNamesForPids(pids: number[]): Promise { - return new Promise((resolve) => { - execFile( - "ps", - ["-o", "comm=", "-p", pids.join(",")], - { maxBuffer: 1024 * 1024 }, - (err, stdout) => { - if (err) { - resolve([]); - return; - } - const names: string[] = []; - for (const line of stdout.split("\n")) { - const trimmed = line.trim(); - if (!trimmed) continue; - // GUI apps list as full executable paths — use the basename. - names.push(path.basename(trimmed)); - } - resolve(names); - }, - ); - }); -} - function runningProcessNames(): Promise | null> { return new Promise((resolve) => { execFile("ps", ["-axo", "comm="], { maxBuffer: 4 * 1024 * 1024 }, (err, stdout) => { diff --git a/apps/x/packages/shared/src/ipc.ts b/apps/x/packages/shared/src/ipc.ts index c6452399..5d7c8a31 100644 --- a/apps/x/packages/shared/src/ipc.ts +++ b/apps/x/packages/shared/src/ipc.ts @@ -786,6 +786,23 @@ const ipcSchemas = { success: z.literal(true), }), }, + // Renderer → main: meeting notes finished generating — fire the "notes + // ready" notification (background only; click opens the note). + 'meeting:notifyNotesReady': { + req: z.object({ + notePath: z.string(), + title: z.string(), + }), + res: z.object({ + success: z.literal(true), + }), + }, + // Main → renderer: the meeting app released the microphone while a + // recording was running — the call likely ended, auto-stop and summarize. + 'meeting:externalCallEnded': { + req: z.null(), + res: z.null(), + }, // Renderer → main: assistant voice/video call holds the mic — suppresses // ambient meeting detection (it would otherwise see our own capture). 'voice:setCallActive': { diff --git a/apps/x/packages/shared/src/notification-settings.ts b/apps/x/packages/shared/src/notification-settings.ts index 3a245bf2..1f31d194 100644 --- a/apps/x/packages/shared/src/notification-settings.ts +++ b/apps/x/packages/shared/src/notification-settings.ts @@ -8,6 +8,7 @@ import { z } from 'zod'; * - agent_permission: an agent is requesting permission to run a tool * - background_task: a background task agent pinged via the notify-user tool * - meeting_detection: popup when Rowboat detects you're in a call/meeting + * - meeting_notes_ready: meeting notes finished generating after a call */ export const NotificationCategorySchema = z.enum([ 'chat_completion', @@ -15,6 +16,7 @@ export const NotificationCategorySchema = z.enum([ 'agent_permission', 'background_task', 'meeting_detection', + 'meeting_notes_ready', ]); export const NotificationCategoriesSchema = z.object({ @@ -23,6 +25,7 @@ export const NotificationCategoriesSchema = z.object({ agent_permission: z.boolean(), background_task: z.boolean(), meeting_detection: z.boolean(), + meeting_notes_ready: z.boolean(), }); export const NotificationSettingsSchema = z.object({ @@ -36,6 +39,7 @@ export const DEFAULT_NOTIFICATION_SETTINGS: NotificationSettings = { agent_permission: true, background_task: true, meeting_detection: true, + meeting_notes_ready: true, }, }; From 3419cf8ef0e4ad948f5142780f773528af1a3bf8 Mon Sep 17 00:00:00 2001 From: Gagan Date: Tue, 14 Jul 2026 00:51:11 +0530 Subject: [PATCH 4/8] fix(x): don't overwrite same-titled meeting notes within a day MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Note filenames derive from the meeting title, so every ad-hoc detected meeting (titled "Meeting") — and any recurring calendar event repeating its summary — resolved to the same file and each recording clobbered the previous one's notes. Suffix with the start timestamp when the path already exists. --- .../renderer/src/hooks/useMeetingTranscription.ts | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/apps/x/apps/renderer/src/hooks/useMeetingTranscription.ts b/apps/x/apps/renderer/src/hooks/useMeetingTranscription.ts index 75b4d91f..b47146ef 100644 --- a/apps/x/apps/renderer/src/hooks/useMeetingTranscription.ts +++ b/apps/x/apps/renderer/src/hooks/useMeetingTranscription.ts @@ -493,7 +493,16 @@ export function useMeetingTranscription(onAutoStop?: () => void) { const filename = calendarEvent?.summary ? calendarEvent.summary.replace(/[\\/*?:"<>|]/g, '').replace(/\s+/g, '_').substring(0, 100).trim() : `meeting-${timestamp}`; - const notePath = `knowledge/Meetings/rowboat/${dateFolder}/${filename}.md`; + let notePath = `knowledge/Meetings/rowboat/${dateFolder}/${filename}.md`; + // Title-derived names collide within a day — every ad-hoc detection is + // titled "Meeting", and recurring calendar events repeat their summary. + // Never overwrite an earlier meeting's note: suffix with the timestamp. + if (calendarEvent?.summary) { + try { + const { exists } = await window.ipc.invoke('workspace:exists', { path: notePath }); + if (exists) notePath = `knowledge/Meetings/rowboat/${dateFolder}/${filename}-${timestamp}.md`; + } catch { /* fall through with the unsuffixed path */ } + } notePathRef.current = notePath; calendarEventRef.current = calendarEvent; From f2d6bbbff3373d0d7b5eea92a673e681586611a5 Mon Sep 17 00:00:00 2001 From: Gagan Date: Tue, 14 Jul 2026 01:06:46 +0530 Subject: [PATCH 5/8] style(x): refine meeting-detected popup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - leaner: 400×84 window, 56px card, type stepped down (14/13px) - close button only appears on hover - Rowboat sail sits directly on the button background (no chip box), slightly larger - dropped the dashed/solid left rule; symmetric margins and padding - browser preview mode (vite only, no Electron): renders sample data at the real window size on a backdrop — #meeting-detected, plus ?variant=calendar for the calendar-linked look --- apps/x/apps/main/src/meeting-popup.ts | 4 +- .../src/components/meeting-detected-popup.tsx | 59 +++++++++++++------ 2 files changed, 43 insertions(+), 20 deletions(-) diff --git a/apps/x/apps/main/src/meeting-popup.ts b/apps/x/apps/main/src/meeting-popup.ts index 7284578b..0b36fe8a 100644 --- a/apps/x/apps/main/src/meeting-popup.ts +++ b/apps/x/apps/main/src/meeting-popup.ts @@ -14,8 +14,8 @@ import type { DetectedMeeting } from "@x/core/dist/meetings/detector.js"; */ // Lean bar + margins for the overhanging × and the CSS drop shadow. -const POPUP_WIDTH = 448; -const POPUP_HEIGHT = 96; +const POPUP_WIDTH = 400; +const POPUP_HEIGHT = 84; const AUTO_DISMISS_MS = 45_000; // Display names, Granola-style ("Chrome", not "Google Chrome"). diff --git a/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx b/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx index 533f2cb3..75f555de 100644 --- a/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx +++ b/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx @@ -23,10 +23,24 @@ const SAIL_ICON = * floats on the card's top-left corner; the window itself is transparent. * Detection never records — the user has to click "Take notes". */ +// Opened directly in a browser (vite only, no Electron) there is no IPC +// bridge — render sample data so the popup can be styled with plain HMR. +// `?variant=calendar` previews the calendar-linked look. +const isPreview = typeof window.ipc === 'undefined' + export function MeetingDetectedPopup() { const [payload, setPayload] = useState(null) useEffect(() => { + if (isPreview) { + const calendar = new URLSearchParams(window.location.search).get('variant') === 'calendar' + setPayload( + calendar + ? { title: 'Weekly design sync', message: 'Chrome', hasCalendarEvent: true } + : { title: 'Meeting detected', message: 'Chrome', hasCalendarEvent: false }, + ) + return + } const cleanup = window.ipc.on('meetingDetect:payload', (next) => setPayload(next)) // The main process pushes on did-finish-load, but that can race this // listener's registration — fetch explicitly too. @@ -40,15 +54,24 @@ export function MeetingDetectedPopup() { }, []) const act = (action: 'take-notes' | 'dismiss') => { + if (isPreview) { + console.log(`[preview] action: ${action}`) + return + } void window.ipc.invoke('meetingDetect:action', { action }).catch(() => {}) } - return ( -
- {/* Close — floats over the card's top-left corner */} + // In the browser preview, reproduce the real popup window's exact size + // (448×96) on a desktop-ish backdrop; in Electron the window IS that size. + const popup = ( +
+ {/* Close — floats over the card's top-left corner, revealed on hover */} {/* Card */} -
-
+
-
+
{payload?.title ?? 'Meeting detected'}
-
+
{payload?.message ?? ''}
) + + if (!isPreview) return popup + return ( +
+ {popup} +
+ ) } From 9ee332e21bda88211e7812dd868627b3b49c3c36 Mon Sep 17 00:00:00 2001 From: Gagan Date: Tue, 14 Jul 2026 01:27:32 +0530 Subject: [PATCH 6/8] style(x): tray recording waveform + popup polish - animated sine-wave indicator (braille dot bars, two per cell, 300ms frames) beside the tray icon while a meeting records - popup: hover-anywhere reveals the close button (the drag region was swallowing mouse events), leaner card (48px, 76px window), sits 44px below the menu bar --- apps/x/apps/main/src/meeting-popup.ts | 5 +- apps/x/apps/main/src/tray.ts | 56 +++++++++++++++++++ .../src/components/meeting-detected-popup.tsx | 16 ++---- 3 files changed, 65 insertions(+), 12 deletions(-) diff --git a/apps/x/apps/main/src/meeting-popup.ts b/apps/x/apps/main/src/meeting-popup.ts index 0b36fe8a..972873fa 100644 --- a/apps/x/apps/main/src/meeting-popup.ts +++ b/apps/x/apps/main/src/meeting-popup.ts @@ -15,7 +15,7 @@ import type { DetectedMeeting } from "@x/core/dist/meetings/detector.js"; // Lean bar + margins for the overhanging × and the CSS drop shadow. const POPUP_WIDTH = 400; -const POPUP_HEIGHT = 84; +const POPUP_HEIGHT = 76; const AUTO_DISMISS_MS = 45_000; // Display names, Granola-style ("Chrome", not "Google Chrome"). @@ -98,7 +98,8 @@ export function showMeetingPopup(meeting: DetectedMeeting): void { width: POPUP_WIDTH, height: POPUP_HEIGHT, x: workArea.x + 24, - y: workArea.y + 24, + // Sit a bit clear of the menu bar rather than hugging it. + y: workArea.y + 44, // NSPanel (macOS): non-activating, and — unlike a regular window with // visibleOnFullScreen — can float over fullscreen Spaces without // turning Rowboat into an "agent" app that loses its Dock icon. diff --git a/apps/x/apps/main/src/tray.ts b/apps/x/apps/main/src/tray.ts index 44a65791..2d55081d 100644 --- a/apps/x/apps/main/src/tray.ts +++ b/apps/x/apps/main/src/tray.ts @@ -88,6 +88,62 @@ export function setTrayRecordingState(isRecording: boolean): void { if (recording === isRecording) return; recording = isRecording; rebuildMenu(); + if (isRecording) startWaveAnimation(); + else stopWaveAnimation(); +} + +// --- Recording indicator: animated mini-waveform beside the tray icon --- +// macOS renders tray titles to the right of the icon. Braille cells give +// 1-dot-wide bars (two bars per character, four height steps each) — a slim +// waveform, an unmissable "Rowboat is capturing this meeting" signal. + +const WAVE_FRAME_MS = 300; +const WAVE_BAR_COUNT = 5; +// Dot bits for a bar of height 1–4 (index 0–3), built bottom-up. Two bars +// per braille cell (left column: dots 7,3,2,1 — right column: dots 8,6,5,4) +// keeps the columns tightly packed; the sine wave keeps every bar ≥1 dot so +// no column ever reads as missing. +const WAVE_LEFT_BITS = [0x40, 0x44, 0x46, 0x47]; +const WAVE_RIGHT_BITS = [0x80, 0xa0, 0xb0, 0xb8]; +// Radians per bar / per frame: together they make the crest travel smoothly +// leftward across the five bars. +const WAVE_SPATIAL_STEP = 1.1; +const WAVE_PHASE_STEP = 0.9; + +let waveTimer: NodeJS.Timeout | null = null; +let wavePhase = 0; + +function waveString(phase: number): string { + const levels: number[] = []; + for (let i = 0; i < WAVE_BAR_COUNT; i++) { + const level = Math.round(1.5 + 1.5 * Math.sin(phase + i * WAVE_SPATIAL_STEP)); + levels.push(Math.min(3, Math.max(0, level))); + } + let out = ""; + for (let i = 0; i < levels.length; i += 2) { + const left = WAVE_LEFT_BITS[levels[i]]; + const right = levels[i + 1] !== undefined ? WAVE_RIGHT_BITS[levels[i + 1]] : 0; + out += String.fromCharCode(0x2800 + left + right); + } + return out; +} + +function startWaveAnimation(): void { + if (!tray || process.platform !== "darwin") return; + stopWaveAnimation(); + waveTimer = setInterval(() => { + if (!tray) return; + wavePhase += WAVE_PHASE_STEP; + tray.setTitle(` ${waveString(wavePhase)}`, { fontType: "monospaced" }); + }, WAVE_FRAME_MS); +} + +function stopWaveAnimation(): void { + if (waveTimer) { + clearInterval(waveTimer); + waveTimer = null; + } + if (tray && process.platform === "darwin") tray.setTitle(""); } function rebuildMenu(): void { diff --git a/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx b/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx index 75f555de..c0798823 100644 --- a/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx +++ b/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx @@ -7,9 +7,6 @@ type PopupPayload = { hasCalendarEvent: boolean } -const dragRegion = { WebkitAppRegion: 'drag' } as React.CSSProperties -const noDragRegion = { WebkitAppRegion: 'no-drag' } as React.CSSProperties - // Rowboat sail glyph (black on transparent, same asset as the tray icon) — // rendered on a white chip inside the "Take notes" pill. const SAIL_ICON = @@ -63,23 +60,23 @@ export function MeetingDetectedPopup() { // In the browser preview, reproduce the real popup window's exact size // (448×96) on a desktop-ish backdrop; in Electron the window IS that size. + // No drag region: draggable areas swallow mouse events, which would keep + // the hover-revealed × from ever showing while over the card body. const popup = (
{/* Close — floats over the card's top-left corner, revealed on hover */} {/* Card */} -
+
{payload?.title ?? 'Meeting detected'} @@ -90,8 +87,7 @@ export function MeetingDetectedPopup() {
- {/* Card */} -
-
-
- {payload?.title ?? 'Meeting detected'} -
-
- {payload?.message ?? ''} -
+
+
+ {payload?.title ?? 'Meeting detected'} +
+
+ {payload?.message ?? ''}
-
+
) From 827bd4df5e13ffeeabb071f5137d73d115b7de23 Mon Sep 17 00:00:00 2001 From: Gagan Date: Tue, 14 Jul 2026 13:42:03 +0530 Subject: [PATCH 8/8] feat(x): popup countdown line, arrival chime, faster call-end MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 2px progress line drains toward auto-dismiss (now 30s); hovering the popup freezes both the line and the dismissal. The renderer owns the countdown; main keeps only a 3-min crash-safety fallback - soft synthesized arrival chime (WebAudio — no asset, no spawn) - close button in the top-left corner, hover-revealed - call-end grace 3s → 1s with a 1s detector poll: notes flow starts within ~2s of hang-up --- apps/x/apps/main/src/meeting-popup.ts | 7 +- .../src/components/meeting-detected-popup.tsx | 69 +++++++++++++++++-- apps/x/packages/core/src/meetings/detector.ts | 6 +- 3 files changed, 72 insertions(+), 10 deletions(-) diff --git a/apps/x/apps/main/src/meeting-popup.ts b/apps/x/apps/main/src/meeting-popup.ts index 8d33fec1..5c5f7d53 100644 --- a/apps/x/apps/main/src/meeting-popup.ts +++ b/apps/x/apps/main/src/meeting-popup.ts @@ -18,7 +18,10 @@ import type { DetectedMeeting } from "@x/core/dist/meetings/detector.js"; // `transparent: true` reliably and paint a grey backing slab instead. const POPUP_WIDTH = 376; const POPUP_HEIGHT = 48; -const AUTO_DISMISS_MS = 45_000; +// The popup renderer owns the real 45s countdown (it pauses on hover and +// draws the progress line); this is only a crash-safety net so a wedged +// renderer can't leave the popup on screen forever. +const FALLBACK_DISMISS_MS = 180_000; // Display names, Granola-style ("Chrome", not "Google Chrome"). const SHORT_APP_NAMES: Record = { @@ -153,5 +156,5 @@ export function showMeetingPopup(meeting: DetectedMeeting): void { win.loadURL("http://localhost:5173/#meeting-detected"); } - dismissTimer = setTimeout(() => closeMeetingPopup(), AUTO_DISMISS_MS); + dismissTimer = setTimeout(() => closeMeetingPopup(), FALLBACK_DISMISS_MS); } diff --git a/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx b/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx index eca03a43..3333a17c 100644 --- a/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx +++ b/apps/x/apps/renderer/src/components/meeting-detected-popup.tsx @@ -1,6 +1,11 @@ -import { useEffect, useState } from 'react' +import { useCallback, useEffect, useRef, useState } from 'react' import { X } from 'lucide-react' +// How long the popup stays up before dismissing itself. The countdown (and +// its progress line) pauses while the pointer is over the popup — main only +// keeps a much longer crash-safety fallback. +const AUTO_DISMISS_MS = 30_000 + type PopupPayload = { title: string message: string @@ -50,13 +55,57 @@ export function MeetingDetectedPopup() { return cleanup }, []) - const act = (action: 'take-notes' | 'dismiss') => { + const act = useCallback((action: 'take-notes' | 'dismiss') => { if (isPreview) { console.log(`[preview] action: ${action}`) return } void window.ipc.invoke('meetingDetect:action', { action }).catch(() => {}) - } + }, []) + + // Subtle arrival chime, synthesized (no asset, no process spawn): a soft + // sine ding gliding up a fourth, ~0.35s, low gain. Once per popup. + const chimedRef = useRef(false) + useEffect(() => { + if (!payload || chimedRef.current || isPreview) return + chimedRef.current = true + try { + const ctx = new AudioContext() + const osc = ctx.createOscillator() + const gain = ctx.createGain() + osc.type = 'sine' + osc.frequency.setValueAtTime(880, ctx.currentTime) + osc.frequency.exponentialRampToValueAtTime(1174.66, ctx.currentTime + 0.08) + gain.gain.setValueAtTime(0.0001, ctx.currentTime) + gain.gain.exponentialRampToValueAtTime(0.06, ctx.currentTime + 0.02) + gain.gain.exponentialRampToValueAtTime(0.0001, ctx.currentTime + 0.35) + osc.connect(gain) + gain.connect(ctx.destination) + osc.start() + osc.stop(ctx.currentTime + 0.4) + osc.onended = () => void ctx.close() + } catch { /* audio is best-effort */ } + }, [payload]) + + // Auto-dismiss countdown, rendered as the progress line at the bottom. + // Hovering pauses it (elapsed only accrues while the pointer is away). + const [remainingFrac, setRemainingFrac] = useState(1) + const hoveredRef = useRef(false) + useEffect(() => { + const TICK_MS = 100 + let elapsed = 0 + const timer = setInterval(() => { + if (hoveredRef.current) return + elapsed += TICK_MS + const frac = Math.max(0, 1 - elapsed / AUTO_DISMISS_MS) + setRemainingFrac(frac) + if (frac <= 0) { + clearInterval(timer) + act('dismiss') + } + }, TICK_MS) + return () => clearInterval(timer) + }, [act]) // The Electron window IS the card (376×48, card background, native macOS // rounded corners + shadow — no transparency, which panels don't honor). @@ -69,14 +118,16 @@ export function MeetingDetectedPopup() { isPreview ? 'rounded-xl shadow-[0_8px_28px_rgba(0,0,0,0.45)]' : 'h-screen w-screen' }`} style={isPreview ? { width: 376, height: 48 } : undefined} + onMouseEnter={() => { hoveredRef.current = true }} + onMouseLeave={() => { hoveredRef.current = false }} > - {/* Close — slides in over the left edge on hover */} + {/* Close — top-left corner, revealed on hover */}
@@ -96,6 +147,12 @@ export function MeetingDetectedPopup() { Take notes + + {/* Time-left line: drains toward dismissal, frozen while hovered */} +
) diff --git a/apps/x/packages/core/src/meetings/detector.ts b/apps/x/packages/core/src/meetings/detector.ts index 36a82b1f..079801c4 100644 --- a/apps/x/packages/core/src/meetings/detector.ts +++ b/apps/x/packages/core/src/meetings/detector.ts @@ -31,7 +31,9 @@ import { isNotificationCategoryEnabled } from "../config/notification_config.js" * means the call ended. */ -const POLL_INTERVAL_MS = 2_000; +// 1s so call-end lands within ~2s of hang-up; all tick work is in-memory +// (the helper pushes owner updates, nothing is spawned). +const POLL_INTERVAL_MS = 1_000; // Mic must be in use continuously this long before we prompt — filters out // Siri, dictation bursts, and app mic-permission probes. const MIC_DEBOUNCE_MS = 5_000; @@ -43,7 +45,7 @@ const CALENDAR_MERGE_LEAD_MS = 15 * 60_000; // While recording, the meeting app must be off the mic this long before we // call the meeting over. Kept short so notes arrive right after hang-up; // only guards against sub-poll device churn, not full reconnects. -const CALL_END_GRACE_MS = 3_000; +const CALL_END_GRACE_MS = 1_000; const CALENDAR_SYNC_DIR = path.join(WorkDir, "calendar_sync"); const HELPER_MAX_RESTARTS = 3;