mirror of
https://github.com/rowboatlabs/rowboat.git
synced 2026-07-21 21:31:12 +02:00
Merge pull request #748 from rowboatlabs/feat/granola-parity
Granola-parity meeting capture: resident app, ambient detection, auto-stop + notes redirect
This commit is contained in:
commit
f68f546223
16 changed files with 1831 additions and 12 deletions
153
apps/x/GRANOLA_PARITY.md
Normal file
153
apps/x/GRANOLA_PARITY.md
Normal file
|
|
@ -0,0 +1,153 @@
|
|||
# Granola Parity — Research & Gap Analysis
|
||||
|
||||
Goal: make Rowboat's meeting feature work exactly like Granola (granola.ai). We copy Granola's behavior — no new invention. This doc is the source of truth: how Granola works, what Rowboat has today (with file:line pointers), the gaps, and the parity plan.
|
||||
|
||||
Research basis: Granola's official docs (docs.granola.ai), granola.ai blog/security/jobs pages, third-party reviews and reverse-engineering writeups, plus a full audit of this codebase. Claims that are inferred (not directly documented by Granola) are marked **[inference]**.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — How Granola works
|
||||
|
||||
Granola is an Electron app (confirmed by their own job postings: "Granola is an Electron app with deep OS integrations", React/TypeScript UI + native OS helpers). No bot ever joins the call — it captures audio locally on the device, so nothing is visible to other meeting participants.
|
||||
|
||||
### 1.1 Always-running app (launch at login)
|
||||
|
||||
- The product model is an **always-running, menu-bar-resident app**. Granola must be running for detection/notifications to work ("You must have Granola open... for transcription to run" — their troubleshooting docs).
|
||||
- A literal "open at login" toggle is **not publicly documented**. **[inference]** A clone should register as a login item (Electron `app.setLoginItemSettings`), default on, with a setting to turn it off — the entire detection value prop collapses if the app isn't running.
|
||||
- The app keeps running when the window closes (menu bar presence remains).
|
||||
|
||||
### 1.2 Menu bar icon
|
||||
|
||||
- Granola "sits in your Mac's menu bar". Documented affordance: **click the menu bar icon to start recording / open the app**.
|
||||
- Recording status is NOT primarily shown in the menu bar. The documented indicators are:
|
||||
- Inside the note: "green dancing bars" at the bottom while capturing.
|
||||
- When another app has focus: a **floating always-on-top "live meeting" pill** on the right side of the screen.
|
||||
- Exact dropdown contents (upcoming meetings list, icon state change while recording) are not documented. The "Coming up" list lives on the home screen, not the tray.
|
||||
|
||||
### 1.3 Meeting detection — two signals, never auto-record
|
||||
|
||||
Granola detects meetings via **calendar events + microphone-in-use detection**, and it **never records without a user action** ("Granola only starts transcribing when you open a note for that meeting").
|
||||
|
||||
**Signal A — Calendar:**
|
||||
- Google / Microsoft calendar sync. Pre-creates an auto-titled note per event with 2+ attendees; filters declined events, OOO, Focus Time.
|
||||
- **In-app popup (Granola-drawn, not macOS Notification Center) 1 minute before** any scheduled call with 2+ attendees. One click on it **opens the meeting URL AND starts transcription** at once.
|
||||
- **Armed auto-start:** if you have the upcoming meeting's note open before it starts, recording starts automatically at the scheduled time.
|
||||
|
||||
**Signal B — Mic-in-use (ad-hoc calls, incl. browser meetings):**
|
||||
- "Granola notices when your microphone is in use and offers to start taking notes."
|
||||
- It's app-aware: notification title is "**Huddle detected**" (Slack), "**Call detected**" (FaceTime/WhatsApp), "**Meeting detected**" (anything else, including browsers). Buttons: "Take Notes". Ad-hoc popups have a dashed left border; calendar ones a solid colored bar.
|
||||
- If the ad-hoc call starts within **15 minutes** of a scheduled event, the popup adopts that event's name (merged).
|
||||
- **[inference]** Mechanism: poll CoreAudio `kAudioDevicePropertyDeviceIsRunningSomewhere` on input devices for "mic in use", plus enumerate running processes against a known meeting-app list to get the app-specific title. There's no public macOS API for "process X is using the mic", so it's a heuristic combo. Browser meetings degrade to the generic "Meeting detected" title; calendar linkage supplies identity.
|
||||
- Notifications are configurable in Settings (off entirely, or per-application).
|
||||
- Related preference: "**Reposition Granola for meetings**" — window auto-repositions/resizes alongside the call when a meeting starts (toggleable, default on).
|
||||
|
||||
### 1.4 Audio capture
|
||||
|
||||
- Capture = **default microphone + system audio output**, at the OS level. Works with any app or browser (Chrome/Safari/Firefox/Edge) with no extension.
|
||||
- Cannot isolate per-app audio — it's the combined system stream; uses the OS default sound devices.
|
||||
- The two streams stay separate through transcription → transcript UI shows **grey bubbles (left) = system audio ("Them"), green bubbles (right) = your mic ("Me")**. No true live diarization on desktop — just Me/Them.
|
||||
- **macOS permissions: exactly two** — Microphone, and Screen & System Audio Recording (macOS bundles system-audio capture under screen recording). No Accessibility permission.
|
||||
- **[inference]** System audio mechanism: ScreenCaptureKit audio loopback (macOS 13+ baseline, they require 13+ / recommend 14.2+), possibly CoreAudio process taps on 14.4+. No virtual audio driver (setup has no driver install step).
|
||||
- **Audio is never stored** — streamed to the ASR provider in real time and discarded.
|
||||
|
||||
### 1.5 Transcription
|
||||
|
||||
- Cloud, real-time streaming ASR. Subprocessors named on their security page: **Deepgram and AssemblyAI** for ASR; **OpenAI and Anthropic** for note enhancement.
|
||||
- Live transcript accrues during the call, hidden behind a waveform-icon toggle in the note.
|
||||
|
||||
### 1.6 Meeting end & post-meeting flow
|
||||
|
||||
- **Auto-stop conditions** (documented): (a) call-end heuristic — transcript inactivity + whether the call software is still in use + scheduled end time for calendar-linked meetings; (b) **15 minutes of silence**; (c) computer sleeps; (d) manual stop. Note: "on macOS, meeting auto-end detection requires admin rights" — without it, manual stop only.
|
||||
- **On stop, enhancement runs automatically** — no click needed. Enhanced notes are ready in seconds (~30s max reported).
|
||||
- A "**notes ready**" notification fires when enhancement completes — that's the re-entry point back into the app if you switched away. The note itself was already open (recording ran inside it), so the enhanced version replaces the raw view in place. There's no documented "force-focus the window" behavior; the notification does the redirecting.
|
||||
- Extras: auto-drafted follow-up emails (toggleable), pre-meeting briefs.
|
||||
|
||||
### 1.7 Notes UI
|
||||
|
||||
**During the meeting:** a plain notepad — the user types rough bullets; transcription runs invisibly. Green dancing bars at the bottom = capturing. Waveform button (next to the per-note "Ask anything" chat bar) opens the live transcript side-by-side. Mid-meeting you can ask the chat to catch you up.
|
||||
|
||||
**After the meeting (enhanced notes):**
|
||||
- Enhancement merges exactly three inputs: **transcript + your raw notes + calendar metadata**, through a template.
|
||||
- Signature affordance: **your words render black, AI-generated text grey**.
|
||||
- Per-line provenance: magnifying-glass icon on a line reveals where it came from in the transcript.
|
||||
- Templates: built-ins (1:1, standup, sales discovery) + custom; re-enhance with a different template (✨), regenerate (🔁), edit raw notes and re-enhance.
|
||||
- Chat: per-note "Ask anything" bar, global chat (⌘J), cross-meeting chat over folders; edit-by-asking.
|
||||
- Home screen: "**Coming up**" strip (next ~5 meetings) + reverse-chronological past notes list; folders; recurring meetings grouped by recurring-event ID + title; shareable links.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — What Rowboat has today (audit of main)
|
||||
|
||||
Two separate feature families exist in `apps/x`; only the second is Granola-adjacent:
|
||||
- **"Calls" (video mode)** — live voice/video chat *with the Rowboat AI* (`VIDEO_MODE.md`). Not meeting capture.
|
||||
- **"Meetings"** — real meeting capture → transcript → AI notes. Working pipeline, but **calendar-triggered and manual-click only**.
|
||||
|
||||
### What EXISTS (and is solid)
|
||||
|
||||
| Area | Status | Where |
|
||||
|---|---|---|
|
||||
| Mic + system-audio capture, 2-channel | ✅ | `apps/renderer/src/hooks/useMeetingTranscription.ts:282-299` — mic `getUserMedia` (ch 0) + `getDisplayMedia({audio})` loopback (ch 1), merged to 16 kHz PCM (`:424-485`); loopback auto-approved in `apps/main/src/main.ts:216-226` |
|
||||
| Realtime ASR | ✅ | Deepgram realtime WS, `nova-3`, multichannel + diarize (`useMeetingTranscription.ts:9-21`); proxy or raw key (`:253-280`) |
|
||||
| Speaker labels | ✅ | ch 0 → "You", ch 1 → diarized `Speaker N` (`:335-347`) |
|
||||
| Transcript storage | ✅ | Markdown + frontmatter + fenced transcript block, `knowledge/Meetings/rowboat/<date>/<name>.md` (`:93-136`, `:487-510`) |
|
||||
| AI meeting notes on stop | ✅ | `packages/core/src/knowledge/summarize_meeting.ts` — LLM summary, attendee-name resolution from calendar; orchestrated in `App.tsx:5601-5661`; notes prepended above transcript |
|
||||
| Auto-stop heuristics | ✅ (partial) | silence RMS backstop, calendar-end gating, system-track ended/muted — `useMeetingTranscription.ts:378-549` |
|
||||
| Calendar sync | ✅ | Google OAuth via `googleapis`, `packages/core/src/knowledge/sync_calendar.ts`, per-event JSON in `~/.rowboat/calendar_sync/` |
|
||||
| Pre-meeting notification | ✅ (system toast) | `packages/core/src/knowledge/notify_calendar_meetings.ts` — polls every 30s, notifies ~1 min before, deep-links `rowboat://action?type=join-and-take-meeting-notes` |
|
||||
| Meetings screen | ✅ | `apps/renderer/src/components/meetings-view.tsx` — "Coming up" (+Join / Take-notes buttons, inline prep) + past-notes table |
|
||||
| Transcript rendering | ✅ | `apps/renderer/src/extensions/transcript-block.tsx` (TipTap, colored speakers, collapsible) |
|
||||
| Meeting prep briefs | ✅ | `meeting_prep_scheduler.ts`, `meeting_prep_brief.ts` (Granola has this too) |
|
||||
| Permissions flows | ✅ | mic `ipc.ts:2090-2106`; screen recording check/open-settings `ipc.ts:2003-2018`; Info.plist strings `forge.config.cjs:201-204`; entitlements OK |
|
||||
|
||||
### What is MISSING (the entire "ambient" layer)
|
||||
|
||||
| # | Gap | Detail |
|
||||
|---|---|---|
|
||||
| 1 | **Launch at login** | Zero `setLoginItemSettings` / auto-launch code anywhere. |
|
||||
| 2 | **Menu bar (tray) icon** | Zero `Tray` usage. No background presence: on macOS closing the window leaves only the Dock; no way to start recording without the full window. |
|
||||
| 3 | **Meeting detection** | No mic-in-use detection, no process detection (Zoom/Teams/Slack/FaceTime), no browser awareness. Only calendar (time-based). Ad-hoc calls are invisible. |
|
||||
| 4 | **Auto/one-click start UX** | Capture starts only via explicit button click or clicking the calendar toast. No "Meeting detected — Take notes?" popup, no armed auto-start when a note is open at meeting start time. |
|
||||
| 5 | **Headless capture** | Capture depends on the renderer window holding a live `getDisplayMedia` stream. No native audio helper → can't capture while the app is "in the background" the way Granola does. |
|
||||
| 6 | **Meeting-end redirect** | On stop, the note refreshes in place, but there's **no "notes ready" notification** and no bring-the-user-back moment if they're in another app. |
|
||||
| 7 | **Notes UI polish (Granola signatures)** | No black-vs-grey authorship distinction (we replace, they merge raw notes + transcript), no during-meeting notepad-first flow (we show the transcript note), no per-line provenance, no templates/re-enhance, no floating "recording" pill. |
|
||||
|
||||
### Honest assessment
|
||||
|
||||
Rowboat has built the *second half* of Granola well — what happens once you're recording, and after the meeting ends. It has essentially none of the *first half* — noticing a meeting is happening and quietly being there without the user opening the app. That first half is exactly items 1–5 above, and it's where all the work is.
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — Parity plan (copy Granola, no new stuff)
|
||||
|
||||
Ordered so each phase is shippable and testable on its own.
|
||||
|
||||
### Phase 1 — Resident app: login item + tray
|
||||
- `app.setLoginItemSettings({ openAtLogin: true })`, default on, toggle in Settings. When launched at login: no window, tray only **[inference — Granola undocumented, but implied]**.
|
||||
- Electron `Tray` with template icon; menu: "Start recording", "Open Rowboat", recording status line, Quit. Keep app alive on window close (already macOS default; add tray so it's reachable).
|
||||
- Acceptance: reboot → icon in menu bar, no window; click tray → start an ad-hoc meeting note.
|
||||
|
||||
### Phase 2 — Meeting detection + "Take notes?" popup
|
||||
- Native signal (small Swift helper or node addon, polled from main): mic-in-use via CoreAudio `kAudioDevicePropertyDeviceIsRunningSomewhere` + running-process match against a known list (zoom.us, Teams, Slack, FaceTime, browsers…).
|
||||
- Granola-style app-drawn popup (small always-on-top BrowserWindow, like the existing video popout): "Huddle detected / Call detected / Meeting detected — [Take Notes]". Merge with a calendar event if within 15 min. Never auto-record.
|
||||
- Upgrade the existing 1-min-before calendar toast to the same popup style; one click = open meeting URL + start capture (deep-link plumbing already exists in `deeplink.ts`).
|
||||
- Armed auto-start: meeting note open before start time → auto-start at start time.
|
||||
- Per-app notification settings.
|
||||
- Acceptance: start a Meet call in Chrome with no calendar event → popup appears; click → recording.
|
||||
|
||||
### Phase 3 — Meeting end → notes ready redirect
|
||||
- Keep existing auto-stop heuristics; add the missing "call software no longer active" signal from the Phase-2 process watcher; keep 15-min silence backstop (ours is stricter — align to Granola's 15 min).
|
||||
- On summary completion: fire a **"Your meeting notes are ready" notification**; clicking focuses the app on the note (deep link exists). This is the "redirect when we cut the call" the feature needs.
|
||||
- Acceptance: leave the call → recording stops on its own → notification within ~30s → click lands on finished notes.
|
||||
|
||||
### Phase 4 — Notes UI parity
|
||||
- During meeting: notepad-first note (user types; transcript behind a waveform toggle); green capture indicator in-note; floating "live meeting" pill when app unfocused (reuse the video-popout window machinery).
|
||||
- After meeting: enhancement merges **raw notes + transcript + calendar event** (today we only use the transcript); render user text black / AI text grey; ✨ re-enhance + 🔁 regenerate; keep transcript below as today.
|
||||
- Home: Meetings view already ≈ Granola's home (Coming up + past list) — minor polish only.
|
||||
|
||||
### Phase 5 (only if needed for true parity) — Headless capture
|
||||
- Native mic + ScreenCaptureKit system-audio capture in main/helper process so recording doesn't require the renderer window. Biggest lift; Phases 1–4 deliver the Granola experience with the window opening on record start, which is acceptable Granola-like behavior (their note opens on start too).
|
||||
|
||||
### Key implementation notes
|
||||
- Permissions stay exactly two (mic + screen-recording) — we already handle both.
|
||||
- Deepgram nova-3 multichannel already matches Granola's Me/Them model — no ASR change needed.
|
||||
- Reuse: popup ← video popout window pattern; deep links ← `deeplink.ts`; detection loop ← same main-process init pattern as `notify_calendar_meetings.ts`.
|
||||
|
|
@ -91,6 +91,26 @@ if (!fs.existsSync(stagedBinary)) {
|
|||
}
|
||||
console.log('✅ node-pty staged in .package/node_modules');
|
||||
|
||||
// Compile the mic-monitor helper (ambient meeting detection) on macOS.
|
||||
// Best-effort: without swiftc — or on other platforms — the app still works,
|
||||
// ad-hoc meeting detection just stays off (main checks the binary exists).
|
||||
if (process.platform === 'darwin') {
|
||||
const swiftSrc = path.join(here, 'native', 'mic-monitor.swift');
|
||||
const helperOut = path.join(here, '.package', 'dist', 'mic-monitor');
|
||||
const upToDate = fs.existsSync(helperOut) &&
|
||||
fs.statSync(helperOut).mtimeMs >= fs.statSync(swiftSrc).mtimeMs;
|
||||
if (upToDate) {
|
||||
console.log('✅ mic-monitor helper up to date');
|
||||
} else {
|
||||
try {
|
||||
execSync(`swiftc -O "${swiftSrc}" -o "${helperOut}"`, { stdio: 'inherit' });
|
||||
console.log('✅ mic-monitor helper compiled');
|
||||
} catch {
|
||||
console.warn('⚠️ mic-monitor helper not built (swiftc unavailable?) — meeting detection disabled');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Bundle the vendored agent-slack CLI into a single self-contained script next
|
||||
// to main.cjs. It runs as a child process (process.execPath with
|
||||
// ELECTRON_RUN_AS_NODE=1), so it must exist as a real file on disk — it can't
|
||||
|
|
|
|||
180
apps/x/apps/main/native/mic-monitor.swift
Normal file
180
apps/x/apps/main/native/mic-monitor.swift
Normal file
|
|
@ -0,0 +1,180 @@
|
|||
// mic-monitor: prints a JSON line whenever the set of processes using the
|
||||
// microphone changes.
|
||||
//
|
||||
// This is the ambient meeting-detection signal (Granola-style): when another
|
||||
// app (Zoom, Meet in a browser, Slack huddle, FaceTime…) opens the
|
||||
// microphone, we report it. No audio is captured, so this requires no
|
||||
// microphone permission (TCC) — it's device state, not content.
|
||||
//
|
||||
// Two signals, best available wins:
|
||||
// - macOS 14.4+: per-process audio objects (kAudioHardwarePropertyProcessObjectList
|
||||
// + kAudioProcessPropertyIsRunningInput) give the processes that own the
|
||||
// mic. Each owner is resolved to its bundle ID (CoreAudio) and executable
|
||||
// path (libproc) HERE, natively — the consumer must not need to shell out
|
||||
// (child-process spawns from Electron's main process fail with EBADF
|
||||
// while capture is active).
|
||||
// - Fallback: kAudioDevicePropertyDeviceIsRunningSomewhere on the default
|
||||
// input device — mic in use by *someone*, no attribution.
|
||||
//
|
||||
// Protocol: one JSON object per line on stdout, emitted on every state
|
||||
// change (and once at startup):
|
||||
// {"micInUse":true,"owners":[{"pid":123,"bundleId":"com.google.Chrome.helper","path":"/Applications/..."}]}
|
||||
// ("owners" is empty when attribution is unavailable.)
|
||||
// The process exits when stdin closes, so it can never outlive the app.
|
||||
//
|
||||
// Compiled by apps/main/bundle.mjs (best-effort, macOS only) to
|
||||
// .package/dist/mic-monitor.
|
||||
|
||||
import Foundation
|
||||
import CoreAudio
|
||||
import Darwin
|
||||
|
||||
func defaultInputDeviceID() -> AudioDeviceID? {
|
||||
var deviceID = AudioDeviceID(0)
|
||||
var size = UInt32(MemoryLayout<AudioDeviceID>.size)
|
||||
var addr = AudioObjectPropertyAddress(
|
||||
mSelector: kAudioHardwarePropertyDefaultInputDevice,
|
||||
mScope: kAudioObjectPropertyScopeGlobal,
|
||||
mElement: kAudioObjectPropertyElementMain)
|
||||
let status = AudioObjectGetPropertyData(
|
||||
AudioObjectID(kAudioObjectSystemObject), &addr, 0, nil, &size, &deviceID)
|
||||
guard status == noErr, deviceID != kAudioObjectUnknown else { return nil }
|
||||
return deviceID
|
||||
}
|
||||
|
||||
func isRunningSomewhere(_ device: AudioDeviceID) -> Bool {
|
||||
var running = UInt32(0)
|
||||
var size = UInt32(MemoryLayout<UInt32>.size)
|
||||
var addr = AudioObjectPropertyAddress(
|
||||
mSelector: kAudioDevicePropertyDeviceIsRunningSomewhere,
|
||||
mScope: kAudioObjectPropertyScopeGlobal,
|
||||
mElement: kAudioObjectPropertyElementMain)
|
||||
let status = AudioObjectGetPropertyData(device, &addr, 0, nil, &size, &running)
|
||||
return status == noErr && running != 0
|
||||
}
|
||||
|
||||
func audioProcessObjectIDs() -> [AudioObjectID] {
|
||||
var addr = AudioObjectPropertyAddress(
|
||||
mSelector: kAudioHardwarePropertyProcessObjectList,
|
||||
mScope: kAudioObjectPropertyScopeGlobal,
|
||||
mElement: kAudioObjectPropertyElementMain)
|
||||
var size = UInt32(0)
|
||||
let sysID = AudioObjectID(kAudioObjectSystemObject)
|
||||
guard AudioObjectGetPropertyDataSize(sysID, &addr, 0, nil, &size) == noErr, size > 0 else {
|
||||
return []
|
||||
}
|
||||
var ids = [AudioObjectID](repeating: 0, count: Int(size) / MemoryLayout<AudioObjectID>.size)
|
||||
guard AudioObjectGetPropertyData(sysID, &addr, 0, nil, &size, &ids) == noErr else { return [] }
|
||||
return ids
|
||||
}
|
||||
|
||||
func processIsRunningInput(_ object: AudioObjectID) -> Bool {
|
||||
var addr = AudioObjectPropertyAddress(
|
||||
mSelector: kAudioProcessPropertyIsRunningInput,
|
||||
mScope: kAudioObjectPropertyScopeGlobal,
|
||||
mElement: kAudioObjectPropertyElementMain)
|
||||
var value = UInt32(0)
|
||||
var size = UInt32(MemoryLayout<UInt32>.size)
|
||||
let status = AudioObjectGetPropertyData(object, &addr, 0, nil, &size, &value)
|
||||
return status == noErr && value != 0
|
||||
}
|
||||
|
||||
func processPID(_ object: AudioObjectID) -> Int32? {
|
||||
var addr = AudioObjectPropertyAddress(
|
||||
mSelector: kAudioProcessPropertyPID,
|
||||
mScope: kAudioObjectPropertyScopeGlobal,
|
||||
mElement: kAudioObjectPropertyElementMain)
|
||||
var pid = pid_t(-1)
|
||||
var size = UInt32(MemoryLayout<pid_t>.size)
|
||||
guard AudioObjectGetPropertyData(object, &addr, 0, nil, &size, &pid) == noErr, pid >= 0 else {
|
||||
return nil
|
||||
}
|
||||
return Int32(pid)
|
||||
}
|
||||
|
||||
func processBundleID(_ object: AudioObjectID) -> String? {
|
||||
var addr = AudioObjectPropertyAddress(
|
||||
mSelector: kAudioProcessPropertyBundleID,
|
||||
mScope: kAudioObjectPropertyScopeGlobal,
|
||||
mElement: kAudioObjectPropertyElementMain)
|
||||
var value: Unmanaged<CFString>? = nil
|
||||
var size = UInt32(MemoryLayout<Unmanaged<CFString>?>.size)
|
||||
guard AudioObjectGetPropertyData(object, &addr, 0, nil, &size, &value) == noErr,
|
||||
let cf = value else { return nil }
|
||||
let str = cf.takeRetainedValue() as String
|
||||
return str.isEmpty ? nil : str
|
||||
}
|
||||
|
||||
func processPath(_ pid: Int32) -> String? {
|
||||
var buffer = [CChar](repeating: 0, count: 4096)
|
||||
let length = proc_pidpath(pid, &buffer, UInt32(buffer.count))
|
||||
guard length > 0 else { return nil }
|
||||
return String(cString: buffer)
|
||||
}
|
||||
|
||||
struct MicOwner: Equatable {
|
||||
let pid: Int32
|
||||
let bundleId: String
|
||||
let path: String
|
||||
}
|
||||
|
||||
/// Processes currently capturing from any input device (macOS 14.4+;
|
||||
/// returns [] where unsupported and the device-level fallback takes over).
|
||||
func micOwners() -> [MicOwner] {
|
||||
var owners: [MicOwner] = []
|
||||
for object in audioProcessObjectIDs() where processIsRunningInput(object) {
|
||||
guard let pid = processPID(object) else { continue }
|
||||
owners.append(MicOwner(
|
||||
pid: pid,
|
||||
bundleId: processBundleID(object) ?? "",
|
||||
path: processPath(pid) ?? ""))
|
||||
}
|
||||
return owners.sorted { $0.pid < $1.pid }
|
||||
}
|
||||
|
||||
func jsonEscape(_ s: String) -> String {
|
||||
var out = ""
|
||||
for ch in s.unicodeScalars {
|
||||
switch ch {
|
||||
case "\"": out += "\\\""
|
||||
case "\\": out += "\\\\"
|
||||
case "\n": out += "\\n"
|
||||
case "\r": out += "\\r"
|
||||
case "\t": out += "\\t"
|
||||
default:
|
||||
if ch.value < 0x20 {
|
||||
out += String(format: "\\u%04x", ch.value)
|
||||
} else {
|
||||
out.unicodeScalars.append(ch)
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Exit when the parent closes our stdin (app quit/crash) — never orphan.
|
||||
Thread {
|
||||
while readLine(strippingNewline: false) != nil {}
|
||||
exit(0)
|
||||
}.start()
|
||||
|
||||
setbuf(stdout, nil)
|
||||
|
||||
var lastInUse: Bool? = nil
|
||||
var lastOwners: [MicOwner] = []
|
||||
while true {
|
||||
let owners = micOwners()
|
||||
// Re-resolve the default device every poll: the user can switch input
|
||||
// devices (AirPods in/out) mid-session.
|
||||
let deviceInUse = defaultInputDeviceID().map(isRunningSomewhere) ?? false
|
||||
let inUse = deviceInUse || !owners.isEmpty
|
||||
if inUse != lastInUse || owners != lastOwners {
|
||||
lastInUse = inUse
|
||||
lastOwners = owners
|
||||
let ownerJson = owners.map { o in
|
||||
"{\"pid\":\(o.pid),\"bundleId\":\"\(jsonEscape(o.bundleId))\",\"path\":\"\(jsonEscape(o.path))\"}"
|
||||
}.joined(separator: ",")
|
||||
print("{\"micInUse\":\(inUse),\"owners\":[\(ownerJson)]}")
|
||||
}
|
||||
Thread.sleep(forTimeInterval: 1.0)
|
||||
}
|
||||
|
|
@ -66,6 +66,20 @@ import { rankSlackHomeMessages } from '@x/core/dist/knowledge/sources/rank_slack
|
|||
import { syncSlackKnowledgeSources, triggerSync as triggerSlackKnowledgeSync, getSlackKnowledgeSyncStatus } from '@x/core/dist/knowledge/sources/sync_slack.js';
|
||||
import { isOnboardingComplete, markOnboardingComplete } from '@x/core/dist/config/note_creation_config.js';
|
||||
import { loadNotificationSettings, saveNotificationSettings } from '@x/core/dist/config/notification_config.js';
|
||||
import { saveAppSettings } from '@x/core/dist/config/app_settings.js';
|
||||
import { setSelfCaptureActive } from '@x/core/dist/meetings/detector.js';
|
||||
import { notifyIfEnabled } from '@x/core/dist/application/notification/notifier.js';
|
||||
import { consumePendingToggleMeetingNotes, setTrayRecordingState } from './tray.js';
|
||||
import { closeMeetingPopup, getMeetingPopupPayload, handleMeetingPopupAction } from './meeting-popup.js';
|
||||
|
||||
// Ambient meeting detection must ignore Rowboat's own mic use: meeting
|
||||
// capture and assistant voice/video calls both hold the mic. Either being
|
||||
// active suppresses "Meeting detected" prompts.
|
||||
let meetingRecordingActive = false;
|
||||
let voiceCallActive = false;
|
||||
function updateSelfCaptureState() {
|
||||
setSelfCaptureActive(meetingRecordingActive || voiceCallActive);
|
||||
}
|
||||
import * as composioHandler from './composio-handler.js';
|
||||
import * as appsIndexer from '@x/core/dist/apps/indexer.js';
|
||||
import * as appsServer from '@x/core/dist/apps/server.js';
|
||||
|
|
@ -842,6 +856,61 @@ export function setupIpcHandlers() {
|
|||
'app:consumePendingDeepLink': async () => {
|
||||
return { url: consumePendingDeepLink() };
|
||||
},
|
||||
'app:consumePendingTrayCommand': async () => {
|
||||
return { toggleMeetingNotes: consumePendingToggleMeetingNotes() };
|
||||
},
|
||||
'app:getLoginItemSettings': async () => {
|
||||
// Dev builds never register a login item (it would point at the dev
|
||||
// Electron binary), so report off.
|
||||
if (!app.isPackaged) return { openAtLogin: false };
|
||||
return { openAtLogin: app.getLoginItemSettings().openAtLogin };
|
||||
},
|
||||
'app:setLoginItemSettings': async (_event, args) => {
|
||||
if (app.isPackaged) {
|
||||
app.setLoginItemSettings({
|
||||
openAtLogin: args.openAtLogin,
|
||||
...(process.platform === 'win32' ? { args: ['--hidden'] } : {}),
|
||||
});
|
||||
// The user has expressed an explicit choice — never re-apply the
|
||||
// first-run default over it.
|
||||
saveAppSettings({ loginItemRegistered: true });
|
||||
}
|
||||
return { success: true as const };
|
||||
},
|
||||
'meeting:setRecordingState': async (_event, args) => {
|
||||
setTrayRecordingState(args.recording);
|
||||
meetingRecordingActive = args.recording;
|
||||
updateSelfCaptureState();
|
||||
// Recording started through another path — a lingering "Take Notes?"
|
||||
// popup is stale now.
|
||||
if (args.recording) closeMeetingPopup();
|
||||
return { success: true as const };
|
||||
},
|
||||
'voice:setCallActive': async (_event, args) => {
|
||||
voiceCallActive = args.active;
|
||||
updateSelfCaptureState();
|
||||
return { success: true as const };
|
||||
},
|
||||
'meeting:notifyNotesReady': async (_event, args) => {
|
||||
// Granola-style re-entry point: the note refreshed in place, but the
|
||||
// user has usually switched back to the meeting app — the notification
|
||||
// brings them back. Suppressed while the app is focused.
|
||||
void notifyIfEnabled('meeting_notes_ready', {
|
||||
title: 'Meeting notes ready',
|
||||
message: `Your notes for "${args.title}" are ready.`,
|
||||
link: `rowboat://open?type=file&path=${encodeURIComponent(args.notePath)}`,
|
||||
actionLabel: 'Open notes',
|
||||
onlyWhenBackground: true,
|
||||
});
|
||||
return { success: true as const };
|
||||
},
|
||||
'meetingDetect:getPayload': async () => {
|
||||
return { payload: getMeetingPopupPayload() };
|
||||
},
|
||||
'meetingDetect:action': async (_event, args) => {
|
||||
handleMeetingPopupAction(args.action);
|
||||
return {};
|
||||
},
|
||||
'analytics:bootstrap': async () => {
|
||||
return {
|
||||
installationId: getInstallationId(),
|
||||
|
|
@ -2221,6 +2290,9 @@ export function setupIpcHandlers() {
|
|||
if (main.isMinimized()) main.restore();
|
||||
main.show();
|
||||
main.focus();
|
||||
// The user is typically in another app (e.g. just left a meeting) —
|
||||
// a plain focus() won't take the foreground from it.
|
||||
app.focus({ steal: true });
|
||||
}
|
||||
return {};
|
||||
},
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
import { app, BrowserWindow, desktopCapturer, dialog, protocol, net, shell, session, safeStorage, type Session } from "electron";
|
||||
import path from "node:path";
|
||||
import os from "node:os";
|
||||
import {
|
||||
setupIpcHandlers,
|
||||
startRunsWatcher, startSessionsWatcher, startTurnEventsWatcher, markSessionsIndexReady,
|
||||
|
|
@ -64,6 +65,10 @@ import {
|
|||
} from "./deeplink.js";
|
||||
import { disconnectGoogleIfScopesStale } from "./oauth-handler.js";
|
||||
import { startModelsDevRefresh } from "@x/core/dist/models/models-dev.js";
|
||||
import { loadAppSettings, saveAppSettings } from "@x/core/dist/config/app_settings.js";
|
||||
import { init as initMeetingDetection } from "@x/core/dist/meetings/detector.js";
|
||||
import { createAppTray, hasTray, isRecordingActive, markPendingToggleMeetingNotes } from "./tray.js";
|
||||
import { initMeetingPopup, showMeetingPopup } from "./meeting-popup.js";
|
||||
|
||||
// Captured as early as possible so it reflects actual process start. Used to
|
||||
// gate grace-eligible notifications (e.g. the burst of background-task
|
||||
|
|
@ -275,7 +280,49 @@ function setupZoomShortcuts(win: BrowserWindow) {
|
|||
});
|
||||
}
|
||||
|
||||
function createWindow() {
|
||||
// Resident-app plumbing (Granola-style): the main window may exist hidden
|
||||
// (launched at login) or not at all (closed on macOS) while the app keeps
|
||||
// running from the tray. showApp() is the single "bring the app up" path
|
||||
// used by the tray, the Dock, and pending tray commands.
|
||||
let mainWindow: BrowserWindow | null = null;
|
||||
|
||||
function showApp(): void {
|
||||
if (mainWindow && !mainWindow.isDestroyed()) {
|
||||
if (mainWindow.isMinimized()) mainWindow.restore();
|
||||
if (!mainWindow.isVisible()) mainWindow.maximize();
|
||||
mainWindow.show();
|
||||
mainWindow.focus();
|
||||
} else {
|
||||
createWindow();
|
||||
}
|
||||
// The user is usually in another app (a meeting!) when this runs — a plain
|
||||
// focus() won't take the foreground from it.
|
||||
app.focus({ steal: true });
|
||||
}
|
||||
|
||||
/**
|
||||
* Was this process launched by the OS at login (rather than by the user)?
|
||||
* Used to start with the window hidden so login launches are invisible.
|
||||
*
|
||||
* - Windows: our login item registers with an explicit --hidden arg.
|
||||
* - macOS: wasOpenedAtLogin when available. On macOS 13+ (SMAppService)
|
||||
* Electron doesn't reliably populate it (electron#37244), so fall back to
|
||||
* a heuristic: a packaged launch while registered as a login item within
|
||||
* two minutes of boot is treated as a login launch.
|
||||
*/
|
||||
function wasLaunchedAtLogin(): boolean {
|
||||
if (process.argv.includes("--hidden")) return true;
|
||||
if (process.platform !== "darwin" || !app.isPackaged) return false;
|
||||
try {
|
||||
const settings = app.getLoginItemSettings();
|
||||
if (settings.wasOpenedAtLogin) return true;
|
||||
return settings.openAtLogin && os.uptime() < 120;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function createWindow(options: { startHidden?: boolean } = {}) {
|
||||
const win = new BrowserWindow({
|
||||
width: 1280,
|
||||
height: 800,
|
||||
|
|
@ -301,11 +348,18 @@ function createWindow() {
|
|||
configureSessionPermissions(session.defaultSession);
|
||||
configureSessionPermissions(session.fromPartition(BROWSER_PARTITION));
|
||||
|
||||
mainWindow = win;
|
||||
setMainWindowForDeepLinks(win);
|
||||
win.on("closed", () => setMainWindowForDeepLinks(null));
|
||||
win.on("closed", () => {
|
||||
if (mainWindow === win) mainWindow = null;
|
||||
setMainWindowForDeepLinks(null);
|
||||
});
|
||||
|
||||
// Show window when content is ready to prevent blank screen
|
||||
// Show window when content is ready to prevent blank screen.
|
||||
// Launched-at-login starts stay hidden: the app is reachable from the
|
||||
// tray/Dock, and showApp() maximizes on first reveal.
|
||||
win.once("ready-to-show", () => {
|
||||
if (options.startHidden) return;
|
||||
win.maximize();
|
||||
win.show();
|
||||
});
|
||||
|
|
@ -436,7 +490,79 @@ app.whenReady().then(async () => {
|
|||
console.error('[Apps] Failed to start:', error);
|
||||
});
|
||||
|
||||
createWindow();
|
||||
// Resident app (Granola-style): register as an OS login item once, on the
|
||||
// first packaged run. After that the OS registry is the source of truth —
|
||||
// the Settings toggle writes it directly, and disabling the login item in
|
||||
// System Settings sticks because we never re-register on boot.
|
||||
if (app.isPackaged && !loadAppSettings().loginItemRegistered) {
|
||||
try {
|
||||
app.setLoginItemSettings({
|
||||
openAtLogin: true,
|
||||
...(process.platform === "win32" ? { args: ["--hidden"] } : {}),
|
||||
});
|
||||
saveAppSettings({ loginItemRegistered: true });
|
||||
} catch (error) {
|
||||
console.error("[LoginItem] Failed to register login item:", error);
|
||||
}
|
||||
}
|
||||
|
||||
createWindow({ startHidden: wasLaunchedAtLogin() });
|
||||
|
||||
// Menu bar icon: open the app / start-stop meeting notes without the
|
||||
// window. If the renderer isn't ready to receive the toggle (window closed
|
||||
// or still loading), park it as a pending command the renderer drains on
|
||||
// mount — same pull pattern as pending deep links.
|
||||
createAppTray({
|
||||
openApp: showApp,
|
||||
toggleMeetingNotes: () => {
|
||||
const hadWindow = mainWindow !== null && !mainWindow.isDestroyed();
|
||||
showApp();
|
||||
const win = mainWindow;
|
||||
if (!hadWindow || !win || win.webContents.isLoading()) {
|
||||
markPendingToggleMeetingNotes();
|
||||
return;
|
||||
}
|
||||
win.webContents.send("app:toggleMeetingNotes", null);
|
||||
},
|
||||
});
|
||||
|
||||
// Ambient meeting detection (Granola-style): the mic-monitor helper +
|
||||
// running-app scan produce "Meeting detected" events; the popup asks
|
||||
// before anything records. Clicking "Take Notes" routes into the same
|
||||
// renderer flow as the calendar notification.
|
||||
initMeetingPopup({
|
||||
onTakeNotes: (meeting) => {
|
||||
showApp();
|
||||
// The user may have started recording between popup and click —
|
||||
// sending the take-notes flow then would toggle it OFF.
|
||||
if (isRecordingActive()) return;
|
||||
const payload = {
|
||||
event: meeting.calendarEvent ?? { summary: meeting.noteTitle },
|
||||
openMeeting: false,
|
||||
source: "detected",
|
||||
};
|
||||
const win = mainWindow;
|
||||
if (!win || win.isDestroyed()) return;
|
||||
if (win.webContents.isLoading()) {
|
||||
win.webContents.once("did-finish-load", () => {
|
||||
if (!win.isDestroyed()) win.webContents.send("app:takeMeetingNotes", payload);
|
||||
});
|
||||
return;
|
||||
}
|
||||
win.webContents.send("app:takeMeetingNotes", payload);
|
||||
},
|
||||
});
|
||||
initMeetingDetection({
|
||||
helperPath: path.join(__dirname, "mic-monitor"),
|
||||
onDetected: (meeting) => showMeetingPopup(meeting),
|
||||
// Call ended while recording (meeting app released the mic) — the
|
||||
// renderer stops capture and generates notes, same as a manual stop.
|
||||
onExternalCallEnded: () => {
|
||||
const win = mainWindow;
|
||||
if (!win || win.isDestroyed() || win.webContents.isLoading()) return;
|
||||
win.webContents.send("meeting:externalCallEnded", null);
|
||||
},
|
||||
});
|
||||
|
||||
// Start workspace watcher as a main-process service
|
||||
// Watcher runs independently and catches ALL filesystem changes:
|
||||
|
|
@ -561,14 +687,16 @@ app.whenReady().then(async () => {
|
|||
initChromeSync();
|
||||
|
||||
app.on("activate", () => {
|
||||
if (BrowserWindow.getAllWindows().length === 0) {
|
||||
createWindow();
|
||||
}
|
||||
// Reveal the hidden/closed main window (login launches start hidden).
|
||||
showApp();
|
||||
});
|
||||
});
|
||||
|
||||
app.on("window-all-closed", () => {
|
||||
if (process.platform !== "darwin") {
|
||||
// Resident app: with a tray present, keep running with no windows so
|
||||
// meeting detection/notifications stay alive (Granola-style). Without a
|
||||
// tray (creation failed), fall back to the platform-default quit.
|
||||
if (process.platform !== "darwin" && !hasTray()) {
|
||||
app.quit();
|
||||
}
|
||||
});
|
||||
|
|
|
|||
160
apps/x/apps/main/src/meeting-popup.ts
Normal file
160
apps/x/apps/main/src/meeting-popup.ts
Normal file
|
|
@ -0,0 +1,160 @@
|
|||
import { app, BrowserWindow, screen } from "electron";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import type { DetectedMeeting } from "@x/core/dist/meetings/detector.js";
|
||||
|
||||
/**
|
||||
* The "Meeting detected — Take Notes?" popup: a small frameless panel in the
|
||||
* top-left corner of the display the user is on. A macOS panel (NSPanel) at
|
||||
* screen-saver level with fullscreen-auxiliary behavior, so it floats over
|
||||
* whatever the user is looking at — including a fullscreen Meet/Zoom — and
|
||||
* appears without stealing focus. Auto-dismisses after a while and never
|
||||
* records anything by itself: clicking "Take Notes" hands off to the main
|
||||
* window's existing take-meeting-notes flow.
|
||||
*/
|
||||
|
||||
// The window IS the card: exact card dimensions, card background, native
|
||||
// rounded corners + shadow. No transparency — macOS panels don't honor
|
||||
// `transparent: true` reliably and paint a grey backing slab instead.
|
||||
const POPUP_WIDTH = 376;
|
||||
const POPUP_HEIGHT = 48;
|
||||
// The popup renderer owns the real 45s countdown (it pauses on hover and
|
||||
// draws the progress line); this is only a crash-safety net so a wedged
|
||||
// renderer can't leave the popup on screen forever.
|
||||
const FALLBACK_DISMISS_MS = 180_000;
|
||||
|
||||
// Display names, Granola-style ("Chrome", not "Google Chrome").
|
||||
const SHORT_APP_NAMES: Record<string, string> = {
|
||||
"Google Chrome": "Chrome",
|
||||
"Microsoft Edge": "Edge",
|
||||
"Brave Browser": "Brave",
|
||||
"Microsoft Teams": "Teams",
|
||||
};
|
||||
|
||||
export interface MeetingPopupPayload {
|
||||
title: string;
|
||||
message: string;
|
||||
hasCalendarEvent: boolean;
|
||||
}
|
||||
|
||||
let popupWin: BrowserWindow | null = null;
|
||||
let currentPayload: MeetingPopupPayload | null = null;
|
||||
let currentMeeting: DetectedMeeting | null = null;
|
||||
let dismissTimer: NodeJS.Timeout | null = null;
|
||||
let onTakeNotes: ((meeting: DetectedMeeting) => void) | null = null;
|
||||
|
||||
/** Main registers the take-notes handoff once at startup. */
|
||||
export function initMeetingPopup(handlers: { onTakeNotes: (meeting: DetectedMeeting) => void }): void {
|
||||
onTakeNotes = handlers.onTakeNotes;
|
||||
}
|
||||
|
||||
export function getMeetingPopupPayload(): MeetingPopupPayload | null {
|
||||
return currentPayload;
|
||||
}
|
||||
|
||||
export function handleMeetingPopupAction(action: "take-notes" | "dismiss"): void {
|
||||
const meeting = currentMeeting;
|
||||
closeMeetingPopup();
|
||||
if (action === "take-notes" && meeting) {
|
||||
onTakeNotes?.(meeting);
|
||||
}
|
||||
}
|
||||
|
||||
export function closeMeetingPopup(): void {
|
||||
if (dismissTimer) {
|
||||
clearTimeout(dismissTimer);
|
||||
dismissTimer = null;
|
||||
}
|
||||
currentPayload = null;
|
||||
currentMeeting = null;
|
||||
if (popupWin && !popupWin.isDestroyed()) popupWin.destroy();
|
||||
popupWin = null;
|
||||
}
|
||||
|
||||
export function showMeetingPopup(meeting: DetectedMeeting): void {
|
||||
const eventSummary =
|
||||
typeof meeting.calendarEvent?.summary === "string"
|
||||
? (meeting.calendarEvent.summary as string).trim()
|
||||
: "";
|
||||
const appLabel = SHORT_APP_NAMES[meeting.appName] ?? meeting.appName;
|
||||
const payload: MeetingPopupPayload = {
|
||||
// Lean two-liner: "Meeting detected" / "Chrome" — or the event name
|
||||
// over the platform when a calendar event is happening now.
|
||||
title: eventSummary ? eventSummary : meeting.title,
|
||||
message: appLabel,
|
||||
hasCalendarEvent: Boolean(meeting.calendarEvent),
|
||||
};
|
||||
|
||||
// Replace any popup that's still up (stale detection loses to fresh).
|
||||
closeMeetingPopup();
|
||||
currentPayload = payload;
|
||||
currentMeeting = meeting;
|
||||
|
||||
// Top-left of the display the user is actually on (cursor display), not
|
||||
// necessarily the primary one.
|
||||
const display = screen.getDisplayNearestPoint(screen.getCursorScreenPoint());
|
||||
const workArea = display.workArea;
|
||||
const popupDir = path.dirname(fileURLToPath(import.meta.url));
|
||||
const preloadPath = app.isPackaged
|
||||
? path.join(popupDir, "../preload/dist/preload.js")
|
||||
: path.join(popupDir, "../../../preload/dist/preload.js");
|
||||
|
||||
const win = new BrowserWindow({
|
||||
width: POPUP_WIDTH,
|
||||
height: POPUP_HEIGHT,
|
||||
x: workArea.x + 24,
|
||||
// Sit a bit clear of the menu bar rather than hugging it.
|
||||
y: workArea.y + 44,
|
||||
// NSPanel (macOS): non-activating, and — unlike a regular window with
|
||||
// visibleOnFullScreen — can float over fullscreen Spaces without
|
||||
// turning Rowboat into an "agent" app that loses its Dock icon.
|
||||
...(process.platform === "darwin" ? { type: "panel" as const } : {}),
|
||||
frame: false,
|
||||
resizable: false,
|
||||
alwaysOnTop: true,
|
||||
skipTaskbar: true,
|
||||
show: false,
|
||||
backgroundColor: "#1d1d1d",
|
||||
hasShadow: true,
|
||||
webPreferences: {
|
||||
nodeIntegration: false,
|
||||
contextIsolation: true,
|
||||
sandbox: true,
|
||||
preload: preloadPath,
|
||||
},
|
||||
});
|
||||
// Screen-saver level + fullscreen-auxiliary: visible over fullscreen
|
||||
// meeting apps on every workspace — the whole point of the popup is to be
|
||||
// seen while the user is IN the meeting, wherever that is.
|
||||
win.setAlwaysOnTop(true, "screen-saver");
|
||||
win.setVisibleOnAllWorkspaces(true, { visibleOnFullScreen: true });
|
||||
// visibleOnFullScreen flips the app's activation policy to "accessory":
|
||||
// the Dock icon disappears and the app can no longer take foreground
|
||||
// focus — clicking "Take Notes" then hands focus to whatever is next in
|
||||
// the window stack instead of Rowboat. Restore the policy immediately:
|
||||
// the popup keeps its fullscreen-auxiliary collection behavior (it's a
|
||||
// non-activating panel, same trick as Zoom's floating controls), while
|
||||
// Rowboat stays a regular app.
|
||||
if (process.platform === "darwin") {
|
||||
void app.dock?.show();
|
||||
}
|
||||
win.webContents.once("did-finish-load", () => {
|
||||
if (win.isDestroyed()) return;
|
||||
win.webContents.send("meetingDetect:payload", payload);
|
||||
// showInactive: the user is in a meeting app right now — appearing
|
||||
// must not steal focus from it.
|
||||
win.showInactive();
|
||||
});
|
||||
win.on("closed", () => {
|
||||
if (popupWin === win) popupWin = null;
|
||||
});
|
||||
popupWin = win;
|
||||
|
||||
if (app.isPackaged) {
|
||||
win.loadURL("app://-/index.html#meeting-detected");
|
||||
} else {
|
||||
win.loadURL("http://localhost:5173/#meeting-detected");
|
||||
}
|
||||
|
||||
dismissTimer = setTimeout(() => closeMeetingPopup(), FALLBACK_DISMISS_MS);
|
||||
}
|
||||
167
apps/x/apps/main/src/tray.ts
Normal file
167
apps/x/apps/main/src/tray.ts
Normal file
|
|
@ -0,0 +1,167 @@
|
|||
import { app, Menu, Tray, nativeImage } from "electron";
|
||||
|
||||
/**
|
||||
* Menu bar / system tray presence (Granola-style resident app).
|
||||
*
|
||||
* The icon is the app glyph pre-rendered as a macOS "template" image
|
||||
* (pure black + alpha, derived from icons/icon.png: alpha = pixel
|
||||
* luminance, so the white sail becomes an opaque black shape and the
|
||||
* black rounded square becomes transparent). Embedded as base64 so the
|
||||
* tray never depends on asset paths that differ between dev and
|
||||
* packaged layouts. Template rendering makes macOS tint it correctly
|
||||
* in light/dark menu bars and while highlighted.
|
||||
*/
|
||||
const TRAY_ICON_18 =
|
||||
"iVBORw0KGgoAAAANSUhEUgAAABIAAAASCAYAAABWzo5XAAABRElEQVR4nKySMS9EQRSF7773REEUlFpRiEg0Oj1RqTQqCiLRKNV+gkLvF5CoJRqJPyBEqyMURJB9zzl5Z3bvTuZtdjdu8mVm7sw9c+fOzWw0a4HMO3Ib3DJRaV0NK5QrqHLBM2AB/IAvOgrrn0EJ2pqvgg2N82AarIBX7jcJtSQyCXbALliKzryDJ83LIiEQnrAFTsBcOOxg3CN4C4FFQmQCnINN+b/BmPZDvTi/0T597SwS4dtvwTK4VPrj1mu5zl9oXQWBMDKTfXAPruSfBcfgwOqi03j5A1h0vo5QJ0W3Dr9GOwN7eiYzPASn/owXSgmEBmTfsLhT4NnqHvqMg2IrozkvewHXyvpIIj2xqYyajJncWf2bPuvGjFL76+ADbFu3WW0YIdaHvcYfWrOoLt4GeVquC3+t228jCfmzVb/b/sX+AAAA//+fzjrgAAAABklEQVQDAHsTRGNUkus5AAAAAElFTkSuQmCC";
|
||||
const TRAY_ICON_36 =
|
||||
"iVBORw0KGgoAAAANSUhEUgAAACQAAAAkCAYAAADhAJiYAAAC3klEQVR4nOyYu4sUQRDGa3fm8DgxUxB8ICqKqBj4QBOFMxMDRQUfgS/0D1A0MRQMTA1MxBcngpFmBiocgokgCopo4PnCSE193e74FV11XTtMz+zs9sLB3Qc/drenp/ubrqruuUtpmimlaaYZb6gBmvK9DbKiDoNUU+bIxEClEoqvRIxkOYbBcrAJ7ALzwATlFiVWyDQULYE1F2wE28E2sBYsJr8I58Fjc18UQw2hLYOOgJ1gHxgFSwru+SsmXsnvjjzq1ZA1wgOuBMfBQXJhUbUF7c8akraJWIZ4yVsyEIfhjBgZket6TRPa5mkmbd/AZ9PWk6GGmXAhuABOgTnSPikmmoH79F7+/Rr8IV+BtQ3xJFq2HJpLYkqNJNS5EkXbiVYb65kZt1XXkIZoAbhKLmGtkdRMFNrXMmOA+4zn2qdUtTGqma1gDKwAv2XgtOD+MkOaV5w/XAS/ijqmXZjZA+6QT9ph00erLDGTFpnitkn5fCRmbBpUGtLYHgU3pe0teAKegnVgL7kqI+lrjwlrKh+ue1Si0NPwICfBFXAX3CaXiP9MP16V3eSqbXOFKb32BawmF/ZCNQvaeDDOlWVgvRgbFzOJgSd5QO5YuEyd+5OaUnRzHBMzwTM0lIRD5FcjofBpnZDPIzZ1jnz12Qdk8ZGxBnykgv2nyhDLnthl0oOVjb0AG8itlOYnPxg/4C1wjALJrCp7/agykjfF/b+CI9QZNq2ww+BH1UBNiiPNkYfgHflQtuT7NfCe/KoHFfMFTZN6PthBblU4bLwq+ymwEeYVa4WsXspnW8Y/C75TSSJbDeIlf1Q+eUe/T24PK01kq9gv+fyAb8Aq8IncK+xPudZVkcQKmY6zhfzBeYBc/nQVKlXskJ0Wc4fAc6oRKlWMkOkKLCUXphPgBvmqqz1Yv9K95SL4AK73aiaWIRYfDYvInVO1w0QDMKTqywwr5k5dq5pCGsTf9n1p9h9WVfoPAAD//0eu+ckAAAAGSURBVAMAjdCu7L3gD4wAAAAASUVORK5CYII=";
|
||||
|
||||
interface TrayActions {
|
||||
openApp: () => void;
|
||||
toggleMeetingNotes: () => void;
|
||||
}
|
||||
|
||||
let tray: Tray | null = null;
|
||||
let actions: TrayActions | null = null;
|
||||
let recording = false;
|
||||
|
||||
// Tray commands issued while the renderer wasn't ready to receive them
|
||||
// (window closed or still loading). Drained by the renderer on mount via
|
||||
// app:consumePendingTrayCommand — same pull pattern as pending deep links.
|
||||
let pendingToggleMeetingNotes = false;
|
||||
|
||||
export function markPendingToggleMeetingNotes(): void {
|
||||
pendingToggleMeetingNotes = true;
|
||||
}
|
||||
|
||||
export function consumePendingToggleMeetingNotes(): boolean {
|
||||
const value = pendingToggleMeetingNotes;
|
||||
pendingToggleMeetingNotes = false;
|
||||
return value;
|
||||
}
|
||||
|
||||
function buildTrayIcon() {
|
||||
const icon = nativeImage.createEmpty();
|
||||
icon.addRepresentation({
|
||||
scaleFactor: 1,
|
||||
buffer: Buffer.from(TRAY_ICON_18, "base64"),
|
||||
});
|
||||
icon.addRepresentation({
|
||||
scaleFactor: 2,
|
||||
buffer: Buffer.from(TRAY_ICON_36, "base64"),
|
||||
});
|
||||
icon.setTemplateImage(true);
|
||||
return icon;
|
||||
}
|
||||
|
||||
export function createAppTray(trayActions: TrayActions): void {
|
||||
if (tray) return;
|
||||
actions = trayActions;
|
||||
|
||||
try {
|
||||
tray = new Tray(buildTrayIcon());
|
||||
} catch (error) {
|
||||
// Tray support can be missing (some Linux environments). The app just
|
||||
// behaves as before: no resident presence.
|
||||
console.error("[Tray] Failed to create tray:", error);
|
||||
return;
|
||||
}
|
||||
|
||||
rebuildMenu();
|
||||
|
||||
// macOS opens the context menu on any click. On Windows/Linux a plain
|
||||
// left-click should open the app; the menu stays on right-click.
|
||||
if (process.platform !== "darwin") {
|
||||
tray.on("click", () => actions?.openApp());
|
||||
}
|
||||
}
|
||||
|
||||
export function hasTray(): boolean {
|
||||
return tray !== null;
|
||||
}
|
||||
|
||||
export function isRecordingActive(): boolean {
|
||||
return recording;
|
||||
}
|
||||
|
||||
export function setTrayRecordingState(isRecording: boolean): void {
|
||||
if (recording === isRecording) return;
|
||||
recording = isRecording;
|
||||
rebuildMenu();
|
||||
if (isRecording) startWaveAnimation();
|
||||
else stopWaveAnimation();
|
||||
}
|
||||
|
||||
// --- Recording indicator: animated mini-waveform beside the tray icon ---
|
||||
// macOS renders tray titles to the right of the icon. Braille cells give
|
||||
// 1-dot-wide bars (two bars per character, four height steps each) — a slim
|
||||
// waveform, an unmissable "Rowboat is capturing this meeting" signal.
|
||||
|
||||
const WAVE_FRAME_MS = 300;
|
||||
const WAVE_BAR_COUNT = 5;
|
||||
// Dot bits for a bar of height 1–4 (index 0–3), built bottom-up. Two bars
|
||||
// per braille cell (left column: dots 7,3,2,1 — right column: dots 8,6,5,4)
|
||||
// keeps the columns tightly packed; the sine wave keeps every bar ≥1 dot so
|
||||
// no column ever reads as missing.
|
||||
const WAVE_LEFT_BITS = [0x40, 0x44, 0x46, 0x47];
|
||||
const WAVE_RIGHT_BITS = [0x80, 0xa0, 0xb0, 0xb8];
|
||||
// Radians per bar / per frame: together they make the crest travel smoothly
|
||||
// leftward across the five bars.
|
||||
const WAVE_SPATIAL_STEP = 1.1;
|
||||
const WAVE_PHASE_STEP = 0.9;
|
||||
|
||||
let waveTimer: NodeJS.Timeout | null = null;
|
||||
let wavePhase = 0;
|
||||
|
||||
function waveString(phase: number): string {
|
||||
const levels: number[] = [];
|
||||
for (let i = 0; i < WAVE_BAR_COUNT; i++) {
|
||||
const level = Math.round(1.5 + 1.5 * Math.sin(phase + i * WAVE_SPATIAL_STEP));
|
||||
levels.push(Math.min(3, Math.max(0, level)));
|
||||
}
|
||||
let out = "";
|
||||
for (let i = 0; i < levels.length; i += 2) {
|
||||
const left = WAVE_LEFT_BITS[levels[i]];
|
||||
const right = levels[i + 1] !== undefined ? WAVE_RIGHT_BITS[levels[i + 1]] : 0;
|
||||
out += String.fromCharCode(0x2800 + left + right);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function startWaveAnimation(): void {
|
||||
if (!tray || process.platform !== "darwin") return;
|
||||
stopWaveAnimation();
|
||||
waveTimer = setInterval(() => {
|
||||
if (!tray) return;
|
||||
wavePhase += WAVE_PHASE_STEP;
|
||||
tray.setTitle(` ${waveString(wavePhase)}`, { fontType: "monospaced" });
|
||||
}, WAVE_FRAME_MS);
|
||||
}
|
||||
|
||||
function stopWaveAnimation(): void {
|
||||
if (waveTimer) {
|
||||
clearInterval(waveTimer);
|
||||
waveTimer = null;
|
||||
}
|
||||
if (tray && process.platform === "darwin") tray.setTitle("");
|
||||
}
|
||||
|
||||
function rebuildMenu(): void {
|
||||
if (!tray) return;
|
||||
const menu = Menu.buildFromTemplate([
|
||||
{ label: "Open Rowboat", click: () => actions?.openApp() },
|
||||
recording
|
||||
? {
|
||||
label: "Stop recording and generate notes",
|
||||
click: () => actions?.toggleMeetingNotes(),
|
||||
}
|
||||
: {
|
||||
label: "Start meeting notes",
|
||||
click: () => actions?.toggleMeetingNotes(),
|
||||
},
|
||||
{ type: "separator" },
|
||||
{ label: "Quit Rowboat", click: () => app.quit() },
|
||||
]);
|
||||
tray.setContextMenu(menu);
|
||||
tray.setToolTip(recording ? "Rowboat — recording meeting" : "Rowboat");
|
||||
}
|
||||
|
|
@ -1083,6 +1083,13 @@ function App() {
|
|||
// floating popout; camera on → full-screen call; camera off → popout
|
||||
// (mascot pill). Handlers live below the voice/submit plumbing they drive.
|
||||
const video = useVideoMode()
|
||||
// Assistant calls hold the mic — tell main so ambient meeting detection
|
||||
// doesn't mistake our own capture for an external meeting.
|
||||
useEffect(() => {
|
||||
void window.ipc
|
||||
.invoke('voice:setCallActive', { active: video.state !== 'idle' })
|
||||
.catch(() => { /* detection may be unavailable */ })
|
||||
}, [video.state])
|
||||
const [inCall, setInCall] = useState(false)
|
||||
const inCallRef = useRef(false)
|
||||
// User explicitly shrank the full-screen call to the floating pill.
|
||||
|
|
@ -1101,6 +1108,24 @@ function App() {
|
|||
handleToggleMeetingRef.current?.()
|
||||
})
|
||||
|
||||
// Keep the tray menu in sync with meeting capture ("Start meeting notes"
|
||||
// vs "Stop recording & generate notes").
|
||||
useEffect(() => {
|
||||
void window.ipc
|
||||
.invoke('meeting:setRecordingState', { recording: meetingTranscription.state === 'recording' })
|
||||
.catch(() => { /* tray may be unavailable */ })
|
||||
}, [meetingTranscription.state])
|
||||
|
||||
// Main detected the meeting app released the mic (call ended) — stop and
|
||||
// generate notes, exactly like a manual stop. Listener only exists while
|
||||
// recording, so a stale signal can never toggle a new recording ON.
|
||||
useEffect(() => {
|
||||
if (meetingTranscription.state !== 'recording') return
|
||||
return window.ipc.on('meeting:externalCallEnded', () => {
|
||||
handleToggleMeetingRef.current?.()
|
||||
})
|
||||
}, [meetingTranscription.state])
|
||||
|
||||
// Check if voice is available on mount and when OAuth state changes
|
||||
const refreshVoiceAvailability = useCallback(() => {
|
||||
Promise.all([
|
||||
|
|
@ -4678,11 +4703,23 @@ function App() {
|
|||
return () => observer.disconnect()
|
||||
}, [])
|
||||
|
||||
// Tray menu "Start/Stop meeting notes": same toggle as the Meetings header
|
||||
// button. Also drains a toggle parked while the window was closed/loading
|
||||
// (mirrors the pending deep-link pull above).
|
||||
useEffect(() => {
|
||||
void window.ipc.invoke('app:consumePendingTrayCommand', null).then(({ toggleMeetingNotes }) => {
|
||||
if (toggleMeetingNotes) handleToggleMeetingRef.current?.()
|
||||
})
|
||||
return window.ipc.on('app:toggleMeetingNotes', () => {
|
||||
handleToggleMeetingRef.current?.()
|
||||
})
|
||||
}, [])
|
||||
|
||||
// Triggered by main when the user clicks a calendar-meeting notification.
|
||||
// Reuses the same flow as the in-app "Join meeting & take notes" button.
|
||||
// When `openMeeting` is true, also opens the meeting URL in the system browser.
|
||||
useEffect(() => {
|
||||
return window.ipc.on('app:takeMeetingNotes', ({ event, openMeeting }) => {
|
||||
return window.ipc.on('app:takeMeetingNotes', ({ event, openMeeting, source }) => {
|
||||
const e = event as {
|
||||
summary?: string
|
||||
start?: { dateTime?: string; date?: string; timeZone?: string }
|
||||
|
|
@ -4706,7 +4743,7 @@ function App() {
|
|||
location: e.location,
|
||||
htmlLink: e.htmlLink,
|
||||
conferenceLink,
|
||||
source: 'calendar-sync',
|
||||
source: source ?? 'calendar-sync',
|
||||
}
|
||||
window.dispatchEvent(new Event('calendar-block:join-meeting'))
|
||||
})
|
||||
|
|
@ -5637,6 +5674,14 @@ function App() {
|
|||
})
|
||||
// Refresh the file view
|
||||
await handleVoiceNoteCreated(notePath)
|
||||
// Notes are done — bring Rowboat to the foreground on the
|
||||
// finished note (the post-call "redirect"). The notification
|
||||
// below is background-only, so it only fires if the focus
|
||||
// grab didn't take.
|
||||
void window.ipc.invoke('app:focusMainWindow', null).catch(() => {})
|
||||
void window.ipc
|
||||
.invoke('meeting:notifyNotesReady', { notePath, title: noteTitle })
|
||||
.catch(() => { /* notification is best-effort */ })
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
|
|
|
|||
165
apps/x/apps/renderer/src/components/meeting-detected-popup.tsx
Normal file
165
apps/x/apps/renderer/src/components/meeting-detected-popup.tsx
Normal file
|
|
@ -0,0 +1,165 @@
|
|||
import { useCallback, useEffect, useRef, useState } from 'react'
|
||||
import { X } from 'lucide-react'
|
||||
|
||||
// How long the popup stays up before dismissing itself. The countdown (and
|
||||
// its progress line) pauses while the pointer is over the popup — main only
|
||||
// keeps a much longer crash-safety fallback.
|
||||
const AUTO_DISMISS_MS = 30_000
|
||||
|
||||
type PopupPayload = {
|
||||
title: string
|
||||
message: string
|
||||
hasCalendarEvent: boolean
|
||||
}
|
||||
|
||||
// Rowboat sail glyph (black on transparent, same asset as the tray icon) —
|
||||
// rendered on a white chip inside the "Take notes" pill.
|
||||
const SAIL_ICON =
|
||||
'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAACfUlEQVR4nMSXO2gUURSGz8yuYuGjsBHFBza+UGwULUxjoY0iWPnA0lZQURDsFC1E7Owt7JQ0SZGQLk2SNoQEQhICCXlUCXk/ZvOf3HM2Z2/uZDM7m90fPnbm3jn3P/e5M0VqsorUZBWocYqogQnEhj1VzwTUsOTBOgJOg1WwYYMiyqdITDe9RK6CW+A2uAYugOPgBhiXuO3kal2E1pg5BO6BJ+A+uEK7OzcJpuVaR6amBArG+Bx4BZ6LqZUOdSI+I+SmoEJZEoilMTa+CN6Cl+CEMUrkuUgS5d91+e33OpApAQ06Cj6CN+CY1G14pqryPMt1b6jhagloI2z+EPwCl4xxgXabqnQXFCW+R8oTaxDvw5z1DbSL+Zpp2D4bpcTz7zAYMomVVaxizkP+FzwydYfJ9Uh3QskYWXGZrolOiamY/7QE1Pwk6AaXwTzoA7OghdyhQtJY7PVWzcnU/fPKK8xC5mdBB1gGv6UHY/IMr/rH4D25gyUtCS3n4efDaJ1Seuvf8xC/BhPgv1dnTz1+7qskogvSrhsu4wPqM/hCgeEPJUC0s99VBbkveWXa2A/wzktC45fILdxJL7kKM1+JV7cZCNRFyIYfwICXqA7/HzGPA22Ue5KmElWXjtYceCrXuiVXwDOp27OBPNJpaAVT5HaVTsVPcv98qb0nqs/7ABvw4XQTXJf7UfBCEkyqBeeV9u6UXHObvIsW9xOcNwFd2WfAHWnvO+iinUV5oAlo/ANyx3Yb+EQpez6keryS8QgMkjvp7oIFStnzIeX5LtAtyK9h3OOWrOZ5E9C/5PPk3gdnspoT5Z8Ce2xnNtegeqgmc1YjP82CavrHadMT2AIAAP//JKuLxQAAAAZJREFUAwAJ75pCeqjxVQAAAABJRU5ErkJggg=='
|
||||
|
||||
/**
|
||||
* Content of the "Meeting detected" always-on-top popup panel (see
|
||||
* `showMeetingPopup` in the main process). Granola-style consent prompt —
|
||||
* a lean horizontal bar: dashed rule (ad-hoc) or solid accent (calendar
|
||||
* match), "Meeting detected" + platform, and a "Take notes" pill. The ×
|
||||
* floats on the card's top-left corner; the window itself is transparent.
|
||||
* Detection never records — the user has to click "Take notes".
|
||||
*/
|
||||
// Opened directly in a browser (vite only, no Electron) there is no IPC
|
||||
// bridge — render sample data so the popup can be styled with plain HMR.
|
||||
// `?variant=calendar` previews the calendar-linked look.
|
||||
const isPreview = typeof window.ipc === 'undefined'
|
||||
|
||||
export function MeetingDetectedPopup() {
|
||||
const [payload, setPayload] = useState<PopupPayload | null>(null)
|
||||
|
||||
useEffect(() => {
|
||||
if (isPreview) {
|
||||
const calendar = new URLSearchParams(window.location.search).get('variant') === 'calendar'
|
||||
setPayload(
|
||||
calendar
|
||||
? { title: 'Weekly design sync', message: 'Chrome', hasCalendarEvent: true }
|
||||
: { title: 'Meeting detected', message: 'Chrome', hasCalendarEvent: false },
|
||||
)
|
||||
return
|
||||
}
|
||||
const cleanup = window.ipc.on('meetingDetect:payload', (next) => setPayload(next))
|
||||
// The main process pushes on did-finish-load, but that can race this
|
||||
// listener's registration — fetch explicitly too.
|
||||
window.ipc
|
||||
.invoke('meetingDetect:getPayload', null)
|
||||
.then(({ payload: cached }) => {
|
||||
if (cached) setPayload(cached)
|
||||
})
|
||||
.catch(() => {})
|
||||
return cleanup
|
||||
}, [])
|
||||
|
||||
const act = useCallback((action: 'take-notes' | 'dismiss') => {
|
||||
if (isPreview) {
|
||||
console.log(`[preview] action: ${action}`)
|
||||
return
|
||||
}
|
||||
void window.ipc.invoke('meetingDetect:action', { action }).catch(() => {})
|
||||
}, [])
|
||||
|
||||
// Subtle arrival chime, synthesized (no asset, no process spawn): a soft
|
||||
// sine ding gliding up a fourth, ~0.35s, low gain. Once per popup.
|
||||
const chimedRef = useRef(false)
|
||||
useEffect(() => {
|
||||
if (!payload || chimedRef.current || isPreview) return
|
||||
chimedRef.current = true
|
||||
try {
|
||||
const ctx = new AudioContext()
|
||||
const osc = ctx.createOscillator()
|
||||
const gain = ctx.createGain()
|
||||
osc.type = 'sine'
|
||||
osc.frequency.setValueAtTime(880, ctx.currentTime)
|
||||
osc.frequency.exponentialRampToValueAtTime(1174.66, ctx.currentTime + 0.08)
|
||||
gain.gain.setValueAtTime(0.0001, ctx.currentTime)
|
||||
gain.gain.exponentialRampToValueAtTime(0.06, ctx.currentTime + 0.02)
|
||||
gain.gain.exponentialRampToValueAtTime(0.0001, ctx.currentTime + 0.35)
|
||||
osc.connect(gain)
|
||||
gain.connect(ctx.destination)
|
||||
osc.start()
|
||||
osc.stop(ctx.currentTime + 0.4)
|
||||
osc.onended = () => void ctx.close()
|
||||
} catch { /* audio is best-effort */ }
|
||||
}, [payload])
|
||||
|
||||
// Auto-dismiss countdown, rendered as the progress line at the bottom.
|
||||
// Hovering pauses it (elapsed only accrues while the pointer is away).
|
||||
const [remainingFrac, setRemainingFrac] = useState(1)
|
||||
const hoveredRef = useRef(false)
|
||||
useEffect(() => {
|
||||
const TICK_MS = 100
|
||||
let elapsed = 0
|
||||
const timer = setInterval(() => {
|
||||
if (hoveredRef.current) return
|
||||
elapsed += TICK_MS
|
||||
const frac = Math.max(0, 1 - elapsed / AUTO_DISMISS_MS)
|
||||
setRemainingFrac(frac)
|
||||
if (frac <= 0) {
|
||||
clearInterval(timer)
|
||||
act('dismiss')
|
||||
}
|
||||
}, TICK_MS)
|
||||
return () => clearInterval(timer)
|
||||
}, [act])
|
||||
|
||||
// The Electron window IS the card (376×48, card background, native macOS
|
||||
// rounded corners + shadow — no transparency, which panels don't honor).
|
||||
// The preview emulates that frame with its own rounding/shadow.
|
||||
// No drag region: draggable areas swallow mouse events, which would keep
|
||||
// the hover-revealed × from ever showing while over the card body.
|
||||
const popup = (
|
||||
<div
|
||||
className={`group relative flex items-center gap-3 pl-4 pr-2 bg-[#1d1d1d] overflow-hidden ${
|
||||
isPreview ? 'rounded-xl shadow-[0_8px_28px_rgba(0,0,0,0.45)]' : 'h-screen w-screen'
|
||||
}`}
|
||||
style={isPreview ? { width: 376, height: 48 } : undefined}
|
||||
onMouseEnter={() => { hoveredRef.current = true }}
|
||||
onMouseLeave={() => { hoveredRef.current = false }}
|
||||
>
|
||||
{/* Close — top-left corner, revealed on hover */}
|
||||
<button
|
||||
onClick={() => act('dismiss')}
|
||||
className="absolute left-1 top-1 z-10 flex size-5 items-center justify-center rounded-full bg-neutral-800 border border-neutral-600 text-neutral-200 shadow-md hover:bg-neutral-700 opacity-0 group-hover:opacity-100 transition-opacity"
|
||||
aria-label="Dismiss"
|
||||
>
|
||||
<X className="size-3" strokeWidth={2.5} />
|
||||
</button>
|
||||
|
||||
<div className="flex-1 min-w-0 transition-[padding] group-hover:pl-6">
|
||||
<div className="text-sm font-semibold text-white leading-tight truncate">
|
||||
{payload?.title ?? 'Meeting detected'}
|
||||
</div>
|
||||
<div className="text-[13px] text-neutral-400 leading-tight truncate mt-0.5">
|
||||
{payload?.message ?? ''}
|
||||
</div>
|
||||
</div>
|
||||
<button
|
||||
onClick={() => act('take-notes')}
|
||||
className="flex h-8.5 shrink-0 items-center gap-1.5 rounded-lg bg-neutral-800/90 border border-neutral-700 pl-1.5 pr-2.5 hover:bg-neutral-700 transition-colors"
|
||||
>
|
||||
<span className="flex size-6 items-center justify-center">
|
||||
<img src={SAIL_ICON} alt="" className="size-4.5 invert" />
|
||||
</span>
|
||||
<span className="text-[13px] font-semibold text-white">Take notes</span>
|
||||
</button>
|
||||
|
||||
{/* Time-left line: drains toward dismissal, frozen while hovered */}
|
||||
<div
|
||||
className="absolute bottom-0 left-0 h-[2px] bg-white/25 transition-[width] duration-100 ease-linear"
|
||||
style={{ width: `${remainingFrac * 100}%` }}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
|
||||
if (!isPreview) return popup
|
||||
return (
|
||||
<div className="min-h-screen w-full bg-gradient-to-br from-sky-200 via-indigo-200 to-rose-200 p-6">
|
||||
{popup}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
|
@ -227,11 +227,48 @@ function ThemeOption({
|
|||
)
|
||||
}
|
||||
|
||||
function LaunchAtLoginSetting() {
|
||||
const [openAtLogin, setOpenAtLogin] = useState(false)
|
||||
const [loaded, setLoaded] = useState(false)
|
||||
|
||||
useEffect(() => {
|
||||
window.ipc.invoke("app:getLoginItemSettings", null)
|
||||
.then(({ openAtLogin }) => setOpenAtLogin(openAtLogin))
|
||||
.catch(() => { /* dev builds report off */ })
|
||||
.finally(() => setLoaded(true))
|
||||
}, [])
|
||||
|
||||
const handleToggle = async (next: boolean) => {
|
||||
setOpenAtLogin(next)
|
||||
try {
|
||||
await window.ipc.invoke("app:setLoginItemSettings", { openAtLogin: next })
|
||||
} catch {
|
||||
setOpenAtLogin(!next)
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex items-center justify-between gap-4 rounded-md border px-3 py-2.5">
|
||||
<div className="min-w-0">
|
||||
<div className="text-sm font-medium">Start Rowboat when you log in</div>
|
||||
<div className="text-xs text-muted-foreground mt-0.5">
|
||||
Keeps Rowboat in your menu bar so meeting notes and notifications work without opening the app
|
||||
</div>
|
||||
</div>
|
||||
<Switch checked={openAtLogin} onCheckedChange={handleToggle} disabled={!loaded} />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
function AppearanceSettings() {
|
||||
const { theme, setTheme, chatPanePlacement, setChatPanePlacement, chatPaneSize, setChatPaneSize } = useTheme()
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
<div>
|
||||
<h4 className="text-sm font-medium mb-3">System</h4>
|
||||
<LaunchAtLoginSetting />
|
||||
</div>
|
||||
<div>
|
||||
<h4 className="text-sm font-medium mb-3">Theme</h4>
|
||||
<p className="text-xs text-muted-foreground mb-4">
|
||||
|
|
@ -2308,7 +2345,7 @@ function CodeModeSettings({ dialogOpen }: { dialogOpen: boolean }) {
|
|||
|
||||
// --- Notification Settings ---
|
||||
|
||||
type NotificationCategoryKey = "chat_completion" | "new_email" | "agent_permission" | "background_task"
|
||||
type NotificationCategoryKey = "chat_completion" | "new_email" | "agent_permission" | "background_task" | "meeting_detection" | "meeting_notes_ready"
|
||||
|
||||
const NOTIFICATION_CATEGORIES: { key: NotificationCategoryKey; label: string; description: string }[] = [
|
||||
{
|
||||
|
|
@ -2331,6 +2368,16 @@ const NOTIFICATION_CATEGORIES: { key: NotificationCategoryKey; label: string; de
|
|||
label: "Background agents",
|
||||
description: "When a background agent you've set up has something to surface. Click to open it on the background tasks page.",
|
||||
},
|
||||
{
|
||||
key: "meeting_detection",
|
||||
label: "Meeting detection",
|
||||
description: "A popup offering to take notes when Rowboat notices you're in a call or meeting. Nothing records until you accept.",
|
||||
},
|
||||
{
|
||||
key: "meeting_notes_ready",
|
||||
label: "Meeting notes ready",
|
||||
description: "When your meeting notes finish generating after a call. Click to open the note. Only shown while the app is in the background.",
|
||||
},
|
||||
]
|
||||
|
||||
function NotificationSettings({ dialogOpen }: { dialogOpen: boolean }) {
|
||||
|
|
|
|||
|
|
@ -493,7 +493,16 @@ export function useMeetingTranscription(onAutoStop?: () => void) {
|
|||
const filename = calendarEvent?.summary
|
||||
? calendarEvent.summary.replace(/[\\/*?:"<>|]/g, '').replace(/\s+/g, '_').substring(0, 100).trim()
|
||||
: `meeting-${timestamp}`;
|
||||
const notePath = `knowledge/Meetings/rowboat/${dateFolder}/${filename}.md`;
|
||||
let notePath = `knowledge/Meetings/rowboat/${dateFolder}/${filename}.md`;
|
||||
// Title-derived names collide within a day — every ad-hoc detection is
|
||||
// titled "Meeting", and recurring calendar events repeat their summary.
|
||||
// Never overwrite an earlier meeting's note: suffix with the timestamp.
|
||||
if (calendarEvent?.summary) {
|
||||
try {
|
||||
const { exists } = await window.ipc.invoke('workspace:exists', { path: notePath });
|
||||
if (exists) notePath = `knowledge/Meetings/rowboat/${dateFolder}/${filename}-${timestamp}.md`;
|
||||
} catch { /* fall through with the unsuffixed path */ }
|
||||
}
|
||||
notePathRef.current = notePath;
|
||||
calendarEventRef.current = calendarEvent;
|
||||
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ import type { CaptureResult } from 'posthog-js'
|
|||
import { ThemeProvider } from '@/contexts/theme-context'
|
||||
import { configureAnalyticsContext } from './lib/analytics'
|
||||
import { VideoPopout } from '@/components/video-popout'
|
||||
import { MeetingDetectedPopup } from '@/components/meeting-detected-popup'
|
||||
|
||||
// Fetch the stable installation ID from main so renderer + main share one
|
||||
// PostHog distinct_id. Falls back to PostHog's auto-generated anonymous ID
|
||||
|
|
@ -66,6 +67,13 @@ if (window.location.hash === '#video-popout') {
|
|||
<VideoPopout />
|
||||
</StrictMode>,
|
||||
)
|
||||
} else if (window.location.hash === '#meeting-detected') {
|
||||
// "Meeting detected — Take Notes?" popup window; same pattern.
|
||||
createRoot(document.getElementById('root')!).render(
|
||||
<StrictMode>
|
||||
<MeetingDetectedPopup />
|
||||
</StrictMode>,
|
||||
)
|
||||
} else {
|
||||
bootstrap()
|
||||
}
|
||||
|
|
|
|||
36
apps/x/packages/core/src/config/app_settings.ts
Normal file
36
apps/x/packages/core/src/config/app_settings.ts
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
import { WorkDir } from './config.js';
|
||||
|
||||
const APP_SETTINGS_PATH = path.join(WorkDir, 'config', 'app_settings.json');
|
||||
|
||||
export interface AppSettings {
|
||||
/**
|
||||
* Set once the app has registered itself as an OS login item (first run
|
||||
* of a packaged build, or the user touching the Settings toggle). After
|
||||
* this, the OS login-item registry is the source of truth — the app never
|
||||
* re-registers on boot, so disabling it in System Settings sticks.
|
||||
*/
|
||||
loginItemRegistered?: boolean;
|
||||
}
|
||||
|
||||
export function loadAppSettings(): AppSettings {
|
||||
try {
|
||||
if (fs.existsSync(APP_SETTINGS_PATH)) {
|
||||
const parsed = JSON.parse(fs.readFileSync(APP_SETTINGS_PATH, 'utf-8'));
|
||||
if (parsed && typeof parsed === 'object') return parsed as AppSettings;
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[AppSettings] Error loading app settings:', error);
|
||||
}
|
||||
return {};
|
||||
}
|
||||
|
||||
export function saveAppSettings(patch: Partial<AppSettings>): void {
|
||||
const merged = { ...loadAppSettings(), ...patch };
|
||||
const dir = path.dirname(APP_SETTINGS_PATH);
|
||||
if (!fs.existsSync(dir)) {
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
}
|
||||
fs.writeFileSync(APP_SETTINGS_PATH, JSON.stringify(merged, null, 2));
|
||||
}
|
||||
519
apps/x/packages/core/src/meetings/detector.ts
Normal file
519
apps/x/packages/core/src/meetings/detector.ts
Normal file
|
|
@ -0,0 +1,519 @@
|
|||
import { spawn, execFile } from "node:child_process";
|
||||
import { existsSync } from "node:fs";
|
||||
import fs from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { WorkDir } from "../config/config.js";
|
||||
import { isNotificationCategoryEnabled } from "../config/notification_config.js";
|
||||
|
||||
/**
|
||||
* Ambient meeting detection (Granola-style).
|
||||
*
|
||||
* Signal: the mic-monitor helper (apps/main/native/mic-monitor.swift) reports
|
||||
* when ANY process starts using the microphone — with the owning processes'
|
||||
* bundle IDs and executable paths on macOS 14.4+, resolved natively in the
|
||||
* helper. That matters twice over: attribution is exact (a Meet call in
|
||||
* Chrome attributes to Chrome even while Slack/Zoom idle in the background),
|
||||
* and the main process never has to shell out while capture is active
|
||||
* (child-process spawns from Electron's main fail with EBADF during capture).
|
||||
*
|
||||
* When the mic has been in use continuously for a few seconds — and it isn't
|
||||
* Rowboat's own capture — we label the popup ("Huddle detected" for Slack,
|
||||
* "Call detected" for FaceTime/WhatsApp, "Meeting detected" otherwise),
|
||||
* merge with a calendar event whose window covers now (start − 15 min
|
||||
* through end), and emit a DetectedMeeting. Never auto-records: the consumer
|
||||
* (main process) shows a "Take Notes?" popup and waits for a click.
|
||||
*
|
||||
* Fires at most once per continuous mic-in-use session; the session resets
|
||||
* after the mic has been idle for MIC_SESSION_RESET_MS.
|
||||
*
|
||||
* While Rowboat records, the same owner list powers call-end detection: once
|
||||
* a meeting app has been seen on the mic, its absence for CALL_END_GRACE_MS
|
||||
* means the call ended.
|
||||
*/
|
||||
|
||||
// 1s so call-end lands within ~2s of hang-up; all tick work is in-memory
|
||||
// (the helper pushes owner updates, nothing is spawned).
|
||||
const POLL_INTERVAL_MS = 1_000;
|
||||
// Mic must be in use continuously this long before we prompt — filters out
|
||||
// Siri, dictation bursts, and app mic-permission probes.
|
||||
const MIC_DEBOUNCE_MS = 5_000;
|
||||
// Mic idle this long ends the "session" and re-arms the prompt.
|
||||
const MIC_SESSION_RESET_MS = 30_000;
|
||||
// A calendar event merges with a detected call from 15 min before its start
|
||||
// (Granola merges ad-hoc calls within 15 minutes of a scheduled event).
|
||||
const CALENDAR_MERGE_LEAD_MS = 15 * 60_000;
|
||||
// While recording, the meeting app must be off the mic this long before we
|
||||
// call the meeting over. Kept short so notes arrive right after hang-up;
|
||||
// only guards against sub-poll device churn, not full reconnects.
|
||||
const CALL_END_GRACE_MS = 1_000;
|
||||
const CALENDAR_SYNC_DIR = path.join(WorkDir, "calendar_sync");
|
||||
const HELPER_MAX_RESTARTS = 3;
|
||||
|
||||
export type DetectedMeetingKind = "huddle" | "call" | "meeting";
|
||||
|
||||
export interface DetectedMeeting {
|
||||
kind: DetectedMeetingKind;
|
||||
/** Popup title, Granola wording: "Huddle detected" / "Call detected" / "Meeting detected". */
|
||||
title: string;
|
||||
/** Human label of the app that likely owns the call, e.g. "Slack", "Zoom", "Google Chrome". */
|
||||
appName: string;
|
||||
/** Suggested note title when no calendar event matched, e.g. "Slack huddle". */
|
||||
noteTitle: string;
|
||||
/** Raw calendar event JSON when a nearby event matched. */
|
||||
calendarEvent?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
interface PlatformMatch {
|
||||
app: string;
|
||||
kind: DetectedMeetingKind;
|
||||
}
|
||||
|
||||
// Bundle-ID prefixes (lowercased) → platform. The primary matcher: exact and
|
||||
// stable, covers helper processes too (com.google.Chrome.helper etc.).
|
||||
const BUNDLE_MATCHERS: Array<{ prefix: string; app: string; kind: DetectedMeetingKind }> = [
|
||||
{ prefix: "us.zoom", app: "Zoom", kind: "meeting" },
|
||||
{ prefix: "com.microsoft.teams", app: "Microsoft Teams", kind: "meeting" },
|
||||
{ prefix: "com.cisco.webex", app: "Webex", kind: "meeting" },
|
||||
{ prefix: "com.webex", app: "Webex", kind: "meeting" },
|
||||
{ prefix: "com.tinyspeck.slackmacgap", app: "Slack", kind: "huddle" },
|
||||
{ prefix: "com.apple.facetime", app: "FaceTime", kind: "call" },
|
||||
{ prefix: "net.whatsapp", app: "WhatsApp", kind: "call" },
|
||||
{ prefix: "com.hnc.discord", app: "Discord", kind: "call" },
|
||||
// Browsers — generic "Meeting detected".
|
||||
{ prefix: "com.google.chrome", app: "Google Chrome", kind: "meeting" },
|
||||
{ prefix: "com.apple.safari", app: "Safari", kind: "meeting" },
|
||||
{ prefix: "com.apple.webkit", app: "Safari", kind: "meeting" },
|
||||
{ prefix: "org.mozilla.firefox", app: "Firefox", kind: "meeting" },
|
||||
{ prefix: "com.brave.browser", app: "Brave Browser", kind: "meeting" },
|
||||
{ prefix: "com.microsoft.edgemac", app: "Microsoft Edge", kind: "meeting" },
|
||||
{ prefix: "company.thebrowser", app: "Arc", kind: "meeting" },
|
||||
];
|
||||
|
||||
// Rowboat's own capture (and the dev Electron shell) — never a "meeting".
|
||||
const SELF_BUNDLE_PREFIXES = ["com.rowboat", "com.github.electron"];
|
||||
|
||||
const BROWSER_APPS = new Set([
|
||||
"Google Chrome",
|
||||
"Safari",
|
||||
"Firefox",
|
||||
"Brave Browser",
|
||||
"Microsoft Edge",
|
||||
"Arc",
|
||||
]);
|
||||
|
||||
// Process-name fallbacks (case-insensitive prefix on the executable
|
||||
// basename): used when an owner has no bundle ID, and by the pre-14.4
|
||||
// running-app heuristic. Ordered by confidence for the heuristic.
|
||||
const MEETING_APPS: Array<{ proc: string; app: string; kind: DetectedMeetingKind }> = [
|
||||
{ proc: "zoom.us", app: "Zoom", kind: "meeting" },
|
||||
{ proc: "MSTeams", app: "Microsoft Teams", kind: "meeting" },
|
||||
{ proc: "Microsoft Teams", app: "Microsoft Teams", kind: "meeting" },
|
||||
{ proc: "Webex", app: "Webex", kind: "meeting" },
|
||||
{ proc: "FaceTime", app: "FaceTime", kind: "call" },
|
||||
{ proc: "WhatsApp", app: "WhatsApp", kind: "call" },
|
||||
{ proc: "Slack", app: "Slack", kind: "huddle" },
|
||||
{ proc: "Discord", app: "Discord", kind: "call" },
|
||||
];
|
||||
|
||||
const BROWSERS = [
|
||||
"Google Chrome",
|
||||
"Safari",
|
||||
"Arc",
|
||||
"Brave Browser",
|
||||
"Microsoft Edge",
|
||||
"Firefox",
|
||||
"Dia",
|
||||
"Comet",
|
||||
];
|
||||
|
||||
const KIND_TITLES: Record<DetectedMeetingKind, string> = {
|
||||
huddle: "Huddle detected",
|
||||
call: "Call detected",
|
||||
meeting: "Meeting detected",
|
||||
};
|
||||
|
||||
interface MicOwner {
|
||||
pid: number;
|
||||
bundleId: string;
|
||||
path: string;
|
||||
}
|
||||
|
||||
interface DetectorOptions {
|
||||
/** Absolute path to the compiled mic-monitor helper binary. */
|
||||
helperPath: string;
|
||||
onDetected: (meeting: DetectedMeeting) => void;
|
||||
/**
|
||||
* Fired once per recording session when the meeting app that was on the
|
||||
* mic has released it for CALL_END_GRACE_MS while Rowboat is still
|
||||
* capturing — i.e. the call ended. Needs per-process attribution
|
||||
* (macOS 14.4+); silently unavailable otherwise.
|
||||
*/
|
||||
onExternalCallEnded?: () => void;
|
||||
}
|
||||
|
||||
let started = false;
|
||||
let selfCaptureActive = false;
|
||||
let micInUse = false;
|
||||
// Processes currently capturing from the mic (macOS 14.4+; empty = unknown).
|
||||
let micOwners: MicOwner[] = [];
|
||||
let micInUseSince: number | null = null;
|
||||
let micIdleSince: number | null = null;
|
||||
let sessionNotified = false;
|
||||
let helperRestarts = 0;
|
||||
// Call-end tracking for the current self-capture session.
|
||||
let externalAppSeen = false;
|
||||
let externalAbsentSince: number | null = null;
|
||||
let callEndFired = false;
|
||||
|
||||
/**
|
||||
* Rowboat's own capture (meeting recording, assistant voice/video call) also
|
||||
* flips the mic-in-use signal — the consumer reports it here so we never
|
||||
* prompt about our own audio.
|
||||
*/
|
||||
export function setSelfCaptureActive(active: boolean): void {
|
||||
selfCaptureActive = active;
|
||||
// Fresh capture session — re-arm call-end tracking.
|
||||
externalAppSeen = false;
|
||||
externalAbsentSince = null;
|
||||
callEndFired = false;
|
||||
if (active) {
|
||||
// Whatever mic session is in flight is ours — don't prompt when the
|
||||
// user stops and the mic lingers.
|
||||
sessionNotified = true;
|
||||
}
|
||||
}
|
||||
|
||||
export function init(options: DetectorOptions): void {
|
||||
if (started) return;
|
||||
if (process.platform !== "darwin") return;
|
||||
if (!existsSync(options.helperPath)) {
|
||||
console.warn(
|
||||
`[MeetingDetect] mic-monitor helper not found at ${options.helperPath} — ambient detection disabled`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
started = true;
|
||||
console.log("[MeetingDetect] starting ambient meeting detection");
|
||||
|
||||
spawnHelper(options.helperPath);
|
||||
setInterval(() => {
|
||||
try {
|
||||
tick(options.onDetected);
|
||||
checkExternalCallEnd(options.onExternalCallEnded);
|
||||
} catch (err) {
|
||||
console.error("[MeetingDetect] tick failed:", err);
|
||||
}
|
||||
}, POLL_INTERVAL_MS);
|
||||
}
|
||||
|
||||
function spawnHelper(helperPath: string): void {
|
||||
let child: ReturnType<typeof spawn>;
|
||||
try {
|
||||
child = spawn(helperPath, [], { stdio: ["pipe", "pipe", "ignore"] });
|
||||
} catch (err) {
|
||||
console.error("[MeetingDetect] failed to spawn mic-monitor:", err);
|
||||
return;
|
||||
}
|
||||
|
||||
let buffer = "";
|
||||
child.stdout?.setEncoding("utf8");
|
||||
child.stdout?.on("data", (chunk: string) => {
|
||||
buffer += chunk;
|
||||
let idx: number;
|
||||
while ((idx = buffer.indexOf("\n")) >= 0) {
|
||||
const line = buffer.slice(0, idx).trim();
|
||||
buffer = buffer.slice(idx + 1);
|
||||
if (!line) continue;
|
||||
try {
|
||||
const parsed = JSON.parse(line);
|
||||
if (typeof parsed?.micInUse === "boolean") {
|
||||
micInUse = parsed.micInUse;
|
||||
micOwners = Array.isArray(parsed.owners)
|
||||
? parsed.owners.filter(
|
||||
(o: unknown): o is MicOwner =>
|
||||
typeof (o as MicOwner)?.pid === "number" &&
|
||||
typeof (o as MicOwner)?.bundleId === "string" &&
|
||||
typeof (o as MicOwner)?.path === "string",
|
||||
)
|
||||
: [];
|
||||
}
|
||||
} catch {
|
||||
// Ignore malformed lines.
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
child.on("exit", (code) => {
|
||||
micInUse = false;
|
||||
micOwners = [];
|
||||
if (helperRestarts < HELPER_MAX_RESTARTS) {
|
||||
helperRestarts += 1;
|
||||
const delay = 5_000 * helperRestarts;
|
||||
console.warn(
|
||||
`[MeetingDetect] mic-monitor exited (code ${code}); restart ${helperRestarts}/${HELPER_MAX_RESTARTS} in ${delay / 1000}s`,
|
||||
);
|
||||
setTimeout(() => spawnHelper(helperPath), delay);
|
||||
} else {
|
||||
console.error("[MeetingDetect] mic-monitor kept exiting — ambient detection disabled");
|
||||
}
|
||||
});
|
||||
child.on("error", (err) => {
|
||||
console.error("[MeetingDetect] mic-monitor error:", err);
|
||||
});
|
||||
}
|
||||
|
||||
function tick(onDetected: DetectorOptions["onDetected"]): void {
|
||||
const now = Date.now();
|
||||
|
||||
if (!micInUse) {
|
||||
micInUseSince = null;
|
||||
if (micIdleSince === null) micIdleSince = now;
|
||||
if (sessionNotified && now - micIdleSince >= MIC_SESSION_RESET_MS) {
|
||||
sessionNotified = false;
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
micIdleSince = null;
|
||||
if (micInUseSince === null) micInUseSince = now;
|
||||
if (sessionNotified) return;
|
||||
if (selfCaptureActive) {
|
||||
// Mark the session as handled so a call that started as ours doesn't
|
||||
// prompt the moment our capture stops while the meeting app holds on.
|
||||
sessionNotified = true;
|
||||
return;
|
||||
}
|
||||
if (now - micInUseSince < MIC_DEBOUNCE_MS) return;
|
||||
|
||||
// Claim the session before the async work — a slow calendar scan must
|
||||
// not let a second tick double-fire.
|
||||
sessionNotified = true;
|
||||
void detect(onDetected);
|
||||
}
|
||||
|
||||
/**
|
||||
* While Rowboat is recording, watch whether any known meeting app/browser
|
||||
* still owns the mic. Once one has been seen, its absence for
|
||||
* CALL_END_GRACE_MS means the call ended — fire once per session. Fully
|
||||
* synchronous: reads the helper-provided owner list, spawns nothing.
|
||||
*/
|
||||
function checkExternalCallEnd(onExternalCallEnded?: () => void): void {
|
||||
if (!onExternalCallEnded) return;
|
||||
if (!selfCaptureActive || callEndFired) return;
|
||||
// No attribution data (pre-14.4 macOS) — feature unavailable.
|
||||
if (micOwners.length === 0) return;
|
||||
|
||||
const now = Date.now();
|
||||
const externalOnMic = micOwners.some((owner) => {
|
||||
const match = matchOwner(owner);
|
||||
return match !== null && match !== "self";
|
||||
});
|
||||
if (externalOnMic) {
|
||||
if (!externalAppSeen) {
|
||||
console.log(
|
||||
`[MeetingDetect] call-end watch armed — mic owners: ${describeOwners(micOwners)}`,
|
||||
);
|
||||
}
|
||||
externalAppSeen = true;
|
||||
externalAbsentSince = null;
|
||||
return;
|
||||
}
|
||||
if (!externalAppSeen) return;
|
||||
if (externalAbsentSince === null) {
|
||||
externalAbsentSince = now;
|
||||
console.log(
|
||||
`[MeetingDetect] meeting app off the mic — remaining owners: ${describeOwners(micOwners)}`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (now - externalAbsentSince >= CALL_END_GRACE_MS) {
|
||||
callEndFired = true;
|
||||
console.log("[MeetingDetect] meeting app released the mic — call likely ended");
|
||||
onExternalCallEnded();
|
||||
}
|
||||
}
|
||||
|
||||
function describeOwners(owners: MicOwner[]): string {
|
||||
return owners
|
||||
.map((o) => o.bundleId || path.basename(o.path) || `pid ${o.pid}`)
|
||||
.join(", ");
|
||||
}
|
||||
|
||||
async function detect(onDetected: DetectorOptions["onDetected"]): Promise<void> {
|
||||
if (!isNotificationCategoryEnabled("meeting_detection")) return;
|
||||
|
||||
const source = await findLikelyMeetingApp();
|
||||
if (!source) {
|
||||
// Mic in use but nothing call-capable owns it (dictation, voice
|
||||
// memo, unknown recorder) — stay quiet to avoid noise.
|
||||
return;
|
||||
}
|
||||
|
||||
const calendarEvent = await findNearbyCalendarEvent();
|
||||
const eventSummary =
|
||||
typeof calendarEvent?.summary === "string" ? calendarEvent.summary.trim() : "";
|
||||
|
||||
const meeting: DetectedMeeting = {
|
||||
kind: source.kind,
|
||||
title: KIND_TITLES[source.kind],
|
||||
appName: source.app,
|
||||
noteTitle: eventSummary || defaultNoteTitle(source),
|
||||
...(calendarEvent ? { calendarEvent } : {}),
|
||||
};
|
||||
|
||||
console.log(
|
||||
`[MeetingDetect] ${meeting.title} (app: ${meeting.appName}` +
|
||||
(eventSummary ? `, calendar: "${eventSummary}")` : ")"),
|
||||
);
|
||||
onDetected(meeting);
|
||||
}
|
||||
|
||||
function defaultNoteTitle(source: PlatformMatch): string {
|
||||
if (source.kind === "huddle") return `${source.app} huddle`;
|
||||
if (source.kind === "call") return `${source.app} call`;
|
||||
return BROWSER_APPS.has(source.app) || BROWSERS.includes(source.app)
|
||||
? "Meeting"
|
||||
: `${source.app} meeting`;
|
||||
}
|
||||
|
||||
/** Match a mic owner to a platform (or to Rowboat itself). */
|
||||
function matchOwner(owner: MicOwner): PlatformMatch | "self" | null {
|
||||
const bundle = owner.bundleId.toLowerCase();
|
||||
if (bundle) {
|
||||
for (const prefix of SELF_BUNDLE_PREFIXES) {
|
||||
if (bundle.startsWith(prefix)) return "self";
|
||||
}
|
||||
for (const matcher of BUNDLE_MATCHERS) {
|
||||
if (bundle.startsWith(matcher.prefix)) {
|
||||
return { app: matcher.app, kind: matcher.kind };
|
||||
}
|
||||
}
|
||||
}
|
||||
const basename = path.basename(owner.path);
|
||||
return basename ? matchProcessName(basename) : null;
|
||||
}
|
||||
|
||||
/** Match one process name to a platform, by case-insensitive prefix. */
|
||||
function matchProcessName(name: string): PlatformMatch | null {
|
||||
const lower = name.toLowerCase();
|
||||
// Safari captures through WebKit's out-of-process media stack.
|
||||
if (lower.startsWith("com.apple.webkit")) return { app: "Safari", kind: "meeting" };
|
||||
// Firefox media capture lives in plugin-container.
|
||||
if (lower.startsWith("plugin-container")) return { app: "Firefox", kind: "meeting" };
|
||||
for (const candidate of MEETING_APPS) {
|
||||
if (lower.startsWith(candidate.proc.toLowerCase())) {
|
||||
return { app: candidate.app, kind: candidate.kind };
|
||||
}
|
||||
}
|
||||
for (const browser of BROWSERS) {
|
||||
if (lower.startsWith(browser.toLowerCase())) return { app: browser, kind: "meeting" };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Attribute the call to an app. Exact when mic owners are known (bundle IDs
|
||||
* from the helper, no child processes involved). Owners known but none
|
||||
* call-capable (voice memo, dictation, screen recorder) → NOT a meeting,
|
||||
* stay quiet. Only without attribution (pre-14.4 macOS) fall back to the
|
||||
* running-app heuristic.
|
||||
*/
|
||||
async function findLikelyMeetingApp(): Promise<PlatformMatch | null> {
|
||||
if (micOwners.length > 0) {
|
||||
const matches = micOwners
|
||||
.map(matchOwner)
|
||||
.filter((m): m is PlatformMatch => m !== null && m !== "self");
|
||||
// Prefer dedicated apps over browsers when several own the mic.
|
||||
const dedicated = matches.find((m) => !BROWSER_APPS.has(m.app));
|
||||
return dedicated ?? matches[0] ?? null;
|
||||
}
|
||||
|
||||
// No attribution available — running-app heuristic in table order
|
||||
// (dedicated conferencing apps beat always-running chat apps beat browsers).
|
||||
const names = await runningProcessNames();
|
||||
if (!names) return null;
|
||||
const hasPrefix = (prefix: string) => {
|
||||
const lower = prefix.toLowerCase();
|
||||
for (const name of names) {
|
||||
if (name.toLowerCase().startsWith(lower)) return true;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
for (const candidate of MEETING_APPS) {
|
||||
if (hasPrefix(candidate.proc)) return { app: candidate.app, kind: candidate.kind };
|
||||
}
|
||||
for (const browser of BROWSERS) {
|
||||
if (hasPrefix(browser)) return { app: browser, kind: "meeting" };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function runningProcessNames(): Promise<Set<string> | null> {
|
||||
return new Promise((resolve) => {
|
||||
execFile("ps", ["-axo", "comm="], { maxBuffer: 4 * 1024 * 1024 }, (err, stdout) => {
|
||||
if (err) {
|
||||
resolve(null);
|
||||
return;
|
||||
}
|
||||
const names = new Set<string>();
|
||||
for (const line of stdout.split("\n")) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) continue;
|
||||
// GUI apps list as full executable paths — match the basename.
|
||||
names.add(path.basename(trimmed));
|
||||
}
|
||||
resolve(names);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
interface CalendarEvent {
|
||||
status?: string;
|
||||
summary?: string;
|
||||
start?: { dateTime?: string };
|
||||
end?: { dateTime?: string };
|
||||
attendees?: Array<{ self?: boolean; responseStatus?: string }>;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
/**
|
||||
* The calendar event (if any) whose window covers "now": from 15 minutes
|
||||
* before its start through its end. Skips all-day, cancelled, and
|
||||
* self-declined events. Ties go to the event that started most recently.
|
||||
*/
|
||||
async function findNearbyCalendarEvent(): Promise<CalendarEvent | null> {
|
||||
let files: string[];
|
||||
try {
|
||||
files = await fs.readdir(CALENDAR_SYNC_DIR);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
let best: { event: CalendarEvent; startMs: number } | null = null;
|
||||
|
||||
for (const name of files) {
|
||||
if (!name.endsWith(".json") || name.startsWith("sync_state")) continue;
|
||||
let event: CalendarEvent;
|
||||
try {
|
||||
event = JSON.parse(await fs.readFile(path.join(CALENDAR_SYNC_DIR, name), "utf-8"));
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (event.status === "cancelled") continue;
|
||||
const startStr = event.start?.dateTime;
|
||||
if (!startStr) continue; // all-day
|
||||
const self = event.attendees?.find((a) => a.self);
|
||||
if (self?.responseStatus === "declined") continue;
|
||||
|
||||
const startMs = Date.parse(startStr);
|
||||
if (!Number.isFinite(startMs)) continue;
|
||||
const endStr = event.end?.dateTime;
|
||||
const endMs = endStr ? Date.parse(endStr) : startMs + 60 * 60_000;
|
||||
|
||||
if (now < startMs - CALENDAR_MERGE_LEAD_MS || now > endMs) continue;
|
||||
if (!best || startMs > best.startMs) best = { event, startMs };
|
||||
}
|
||||
|
||||
return best?.event ?? null;
|
||||
}
|
||||
|
|
@ -786,6 +786,9 @@ const ipcSchemas = {
|
|||
// When true, the renderer should also open the meeting URL (Zoom/Meet/etc.)
|
||||
// in addition to triggering the take-notes flow.
|
||||
openMeeting: z.boolean().optional(),
|
||||
// Origin recorded in the note frontmatter: 'calendar-sync' (default) for
|
||||
// notification/deep-link starts, 'detected' for ambient meeting detection.
|
||||
source: z.string().optional(),
|
||||
}),
|
||||
res: z.null(),
|
||||
},
|
||||
|
|
@ -795,6 +798,105 @@ const ipcSchemas = {
|
|||
url: z.string().nullable(),
|
||||
}),
|
||||
},
|
||||
// Tray commands issued before the renderer was ready (mirrors the pending
|
||||
// deep-link pull above): the renderer drains this once on mount.
|
||||
'app:consumePendingTrayCommand': {
|
||||
req: z.null(),
|
||||
res: z.object({
|
||||
toggleMeetingNotes: z.boolean(),
|
||||
}),
|
||||
},
|
||||
// Main → renderer: tray menu "Start/Stop meeting notes" — runs the same
|
||||
// toggle flow as the Meetings header button.
|
||||
'app:toggleMeetingNotes': {
|
||||
req: z.null(),
|
||||
res: z.null(),
|
||||
},
|
||||
// Launch-at-login (resident app). The OS login-item registry is the source
|
||||
// of truth; these read/write it directly rather than a config file.
|
||||
'app:getLoginItemSettings': {
|
||||
req: z.null(),
|
||||
res: z.object({
|
||||
openAtLogin: z.boolean(),
|
||||
}),
|
||||
},
|
||||
'app:setLoginItemSettings': {
|
||||
req: z.object({
|
||||
openAtLogin: z.boolean(),
|
||||
}),
|
||||
res: z.object({
|
||||
success: z.literal(true),
|
||||
}),
|
||||
},
|
||||
// Renderer → main: meeting capture state, so the tray menu/tooltip can
|
||||
// reflect an active recording.
|
||||
'meeting:setRecordingState': {
|
||||
req: z.object({
|
||||
recording: z.boolean(),
|
||||
}),
|
||||
res: z.object({
|
||||
success: z.literal(true),
|
||||
}),
|
||||
},
|
||||
// Renderer → main: meeting notes finished generating — fire the "notes
|
||||
// ready" notification (background only; click opens the note).
|
||||
'meeting:notifyNotesReady': {
|
||||
req: z.object({
|
||||
notePath: z.string(),
|
||||
title: z.string(),
|
||||
}),
|
||||
res: z.object({
|
||||
success: z.literal(true),
|
||||
}),
|
||||
},
|
||||
// Main → renderer: the meeting app released the microphone while a
|
||||
// recording was running — the call likely ended, auto-stop and summarize.
|
||||
'meeting:externalCallEnded': {
|
||||
req: z.null(),
|
||||
res: z.null(),
|
||||
},
|
||||
// Renderer → main: assistant voice/video call holds the mic — suppresses
|
||||
// ambient meeting detection (it would otherwise see our own capture).
|
||||
'voice:setCallActive': {
|
||||
req: z.object({
|
||||
active: z.boolean(),
|
||||
}),
|
||||
res: z.object({
|
||||
success: z.literal(true),
|
||||
}),
|
||||
},
|
||||
// --- Ambient meeting detection popup (own always-on-top window) ---
|
||||
// Main → popup: the detection to display.
|
||||
'meetingDetect:payload': {
|
||||
req: z.object({
|
||||
title: z.string(),
|
||||
message: z.string(),
|
||||
// Calendar-linked detections render a solid accent bar; ad-hoc ones a
|
||||
// dashed one (Granola's affordance).
|
||||
hasCalendarEvent: z.boolean(),
|
||||
}),
|
||||
res: z.null(),
|
||||
},
|
||||
// Popup → main: fetch the payload (the push can race listener registration).
|
||||
'meetingDetect:getPayload': {
|
||||
req: z.null(),
|
||||
res: z.object({
|
||||
payload: z
|
||||
.object({
|
||||
title: z.string(),
|
||||
message: z.string(),
|
||||
hasCalendarEvent: z.boolean(),
|
||||
})
|
||||
.nullable(),
|
||||
}),
|
||||
},
|
||||
// Popup → main: user clicked a button.
|
||||
'meetingDetect:action': {
|
||||
req: z.object({
|
||||
action: z.enum(['take-notes', 'dismiss']),
|
||||
}),
|
||||
res: z.object({}),
|
||||
},
|
||||
'granola:getConfig': {
|
||||
req: z.null(),
|
||||
res: z.object({
|
||||
|
|
|
|||
|
|
@ -7,12 +7,16 @@ import { z } from 'zod';
|
|||
* - new_email: a new email arrived during incremental Gmail sync
|
||||
* - agent_permission: an agent is requesting permission to run a tool
|
||||
* - background_task: a background task agent pinged via the notify-user tool
|
||||
* - meeting_detection: popup when Rowboat detects you're in a call/meeting
|
||||
* - meeting_notes_ready: meeting notes finished generating after a call
|
||||
*/
|
||||
export const NotificationCategorySchema = z.enum([
|
||||
'chat_completion',
|
||||
'new_email',
|
||||
'agent_permission',
|
||||
'background_task',
|
||||
'meeting_detection',
|
||||
'meeting_notes_ready',
|
||||
]);
|
||||
|
||||
export const NotificationCategoriesSchema = z.object({
|
||||
|
|
@ -20,6 +24,8 @@ export const NotificationCategoriesSchema = z.object({
|
|||
new_email: z.boolean(),
|
||||
agent_permission: z.boolean(),
|
||||
background_task: z.boolean(),
|
||||
meeting_detection: z.boolean(),
|
||||
meeting_notes_ready: z.boolean(),
|
||||
});
|
||||
|
||||
export const NotificationSettingsSchema = z.object({
|
||||
|
|
@ -32,6 +38,8 @@ export const DEFAULT_NOTIFICATION_SETTINGS: NotificationSettings = {
|
|||
new_email: true,
|
||||
agent_permission: true,
|
||||
background_task: true,
|
||||
meeting_detection: true,
|
||||
meeting_notes_ready: true,
|
||||
},
|
||||
};
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue