diff --git a/README.md b/README.md index 8d46398b..9e002cd1 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,7 @@ You can set up background agents that run on events like new email or on schedul

Built-in Browser

-Rowboat includes a browser that lets you and assistant collaborate on web tasks. Because its isolated from your main browser, you can log in only to the accounts that want the assistant to access. +Rowboat includes a browser that lets you and assistant collaborate on web tasks. Because it's isolated from your main browser, you can log in only to the accounts that want the assistant to access. Browser screenshot @@ -110,7 +110,7 @@ Code mode lets you spin up parallel coding agents with Claude Code or Codex, and

Apps

-You can bulild your own work surfaces inside Rowboat — they get acess to all the tools and integrations, and you can share them with other people. +You can build your own work surfaces inside Rowboat — they get access to all the tools and integrations, and you can share them with other people. Apps screenshot diff --git a/apps/x/ANALYTICS.md b/apps/x/ANALYTICS.md index e933ab9a..6c8dec08 100644 --- a/apps/x/ANALYTICS.md +++ b/apps/x/ANALYTICS.md @@ -96,6 +96,14 @@ All in `apps/renderer/src/lib/analytics.ts`: - `search_executed` — `{ types: string[] }` - `note_exported` — `{ format }` +### Client auto-update funnel + +The desktop client's own updates — distinct from the in-app apps feature, which owns `app_updated`: + +- `update_prompted` — renderer (`apps/renderer/src/lib/analytics.ts`): the "Update available" card was shown for a staged update +- `update_restarted` — main (`apps/main/src/updater.ts`), `{ from, to? }`: the user clicked restart-to-update (`to` may be missing when the update feed doesn't report the release name) +- `update_failed` — main (`apps/main/src/updater.ts`), `{ message }`: the auto-updater errored (includes network errors for now) +- `client_updated` — main (`apps/main/src/ipc.ts`), `{ from, to }`: first launch on a newer version (fires once per update, whatever the restart path; downgrades restamp silently and don't fire) ### `view_opened` — feature-importance funnel One event per view the user lands on, fired centrally from the `currentViewState` effect in `apps/renderer/src/App.tsx`. `view` is one of: `chat`, `file`, `graph`, `task`, `suggested-topics`, `meetings`, `live-notes`, `email`, `workspace`, `knowledge-view`, `chat-history`, `home`, `code`, `bg-tasks`, `apps`. Keyed on the view *type*, so switching files or threads inside a view doesn't re-fire. @@ -173,7 +181,8 @@ All renderer events live in `apps/renderer/src/lib/analytics.ts` (typed wrappers - `email_send_failed` — send returned an error or threw (`components/email-view.tsx`) - `meeting_summarize_failed` — post-recording notes generation threw (`App.tsx`) -- `bg_agent_run_failed` / `bg_agent_run_completed` — `{ trigger: 'manual' | 'cron' | 'window' | 'event' }` **(core)** — every background-agent run settles as exactly one of these (`packages/core/src/background-tasks/runner.ts`), giving a failure *rate* across all trigger sources, not just manual clicks +- `bg_agent_run_failed` — `{ trigger: 'manual' | 'cron' | 'window' | 'event', error: string }` **(core)** — emitted when a background-agent run fails; `error` contains the normalized failure message +- `bg_agent_run_completed` — `{ trigger: 'manual' | 'cron' | 'window' | 'event' }` **(core)** — emitted when a background-agent run succeeds; together these events give a failure *rate* across all trigger sources, not just manual clicks **Misc**: diff --git a/apps/x/apps/main/bundle.mjs b/apps/x/apps/main/bundle.mjs index 696084f3..cca7fdc4 100644 --- a/apps/x/apps/main/bundle.mjs +++ b/apps/x/apps/main/bundle.mjs @@ -91,6 +91,17 @@ if (!fs.existsSync(stagedBinary)) { } console.log('✅ node-pty staged in .package/node_modules'); +// electron-chrome-extensions injects a preload script into browser tabs to +// implement the chrome.* extension APIs. It resolves that file at runtime: +// via require.resolve when node_modules is present (dev), falling back to a +// file next to the running bundle (packaged app, where node_modules is +// gone). Stage it next to main.cjs for the packaged case. +const crxPreloadSrc = fs.realpathSync( + path.join(here, 'node_modules', 'electron-chrome-extensions', 'dist', 'chrome-extension-api.preload.js'), +); +fs.copyFileSync(crxPreloadSrc, path.join(here, '.package', 'dist', 'chrome-extension-api.preload.js')); +console.log('✅ electron-chrome-extensions preload staged'); + // Compile the mic-monitor helper (ambient meeting detection) on macOS. // Best-effort: without swiftc — or on other platforms — the app still works, // ad-hoc meeting detection just stays off (main checks the binary exists). diff --git a/apps/x/apps/main/forge.config.cjs b/apps/x/apps/main/forge.config.cjs index 0cbcd4ae..581206a8 100644 --- a/apps/x/apps/main/forge.config.cjs +++ b/apps/x/apps/main/forge.config.cjs @@ -250,6 +250,16 @@ module.exports = { name: `Rowboat-win32-${arch}`, setupExe: `Rowboat-win32-${arch}-${pkg.version}-setup.exe`, setupIcon: path.join(__dirname, 'icons/icon.ico'), + // The animation is Squirrel's ONLY install UI — without this + // users stare at Squirrel's unbranded default mid-install. + loadingGif: path.join(__dirname, 'icons/install-loading.gif'), + // Add/Remove Programs icon. Must be a remote URL (Squirrel + // limitation); defaults to the Atom feather otherwise. + iconUrl: 'https://raw.githubusercontent.com/rowboatlabs/rowboat/main/apps/x/apps/main/icons/icon.ico', + // Skip the machine-wide MSI deployment stub — it lands on the + // GitHub release page next to setup.exe and users grab the + // wrong one (it neither launches the app nor auto-updates). + noMsi: true, }) }, { diff --git a/apps/x/apps/main/icons/gen-install-loading.sh b/apps/x/apps/main/icons/gen-install-loading.sh new file mode 100644 index 00000000..9b807fad --- /dev/null +++ b/apps/x/apps/main/icons/gen-install-loading.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +# One-off dev tool, run manually (requires ImageMagick) — NOT part of any +# build step. Its output, install-loading.gif, is committed and wired into +# forge.config.cjs as Squirrel.Windows' `loadingGif` (the installer's only +# UI). Re-run only if the icon or branding changes. +# +# Generates the Squirrel install animation: icon + title + operation label. +# Timeline approximates a typical 20-25s install, then holds at "Almost done" +# for the slow-machine tail. Frame every 0.5s. No progress bar — Squirrel gives +# no real progress signal, so a bar would just be a fake timeline. +set -euo pipefail + +ICON=$(dirname "$0")/icon.png +# DejaVu Sans by name on systems that have it installed; set FONT to a +# DejaVuSans.ttf path on systems that don't (e.g. Windows/macOS). +FONT="${FONT:-DejaVu-Sans}" +OUT_DIR=$(mktemp -d) +mkdir -p "$OUT_DIR" +rm -f "$OUT_DIR"/frame-*.png + +FRAMES=110 # 55s total at 0.5s/frame + +for i in $(seq 0 $((FRAMES - 1))); do + label=$(awk -v i="$i" 'BEGIN { + t = i * 0.5 + if (t < 14) print "Installing"; else if (t < 21) print "Creating shortcuts"; else print "Almost done" + }') + + # Animated trailing dots (ellipsis), cycling every 2s + ndots=$((i % 4)) + dots="" + for _ in $(seq 1 $ndots); do dots=". $dots"; done + + magick -size 480x320 xc:'#252525' \ + \( "$ICON" -resize 84x84 \) -gravity center -geometry +0-72 -composite \ + -gravity center -font "$FONT" -pointsize 21 -fill '#e8e8e8' -annotate +0+8 'Installing Rowboat' \ + -gravity center -font "$FONT" -pointsize 14 -fill '#9a9a9a' -annotate +0+78 "$label" \ + -gravity center -font "$FONT" -pointsize 14 -fill '#6f6f6f' -annotate +0+100 "$dots" \ + "$OUT_DIR/frame-$(printf '%03d' "$i").png" +done + +magick -delay 50 -loop 0 "$OUT_DIR"/frame-*.png -layers Optimize "$(dirname "$0")/install-loading.gif" +echo "frames: $FRAMES" +ls -la "$(dirname "$0")/install-loading.gif" diff --git a/apps/x/apps/main/icons/install-loading.gif b/apps/x/apps/main/icons/install-loading.gif new file mode 100644 index 00000000..f673439e Binary files /dev/null and b/apps/x/apps/main/icons/install-loading.gif differ diff --git a/apps/x/apps/main/package.json b/apps/x/apps/main/package.json index 2577b0c1..2a6c5018 100644 --- a/apps/x/apps/main/package.json +++ b/apps/x/apps/main/package.json @@ -19,13 +19,13 @@ "@x/shared": "workspace:*", "agent-slack": "0.9.3", "chokidar": "^4.0.3", + "electron-chrome-extensions": "^4.9.0", "electron-squirrel-startup": "^1.0.1", "html-to-docx": "^1.8.0", "mammoth": "^1.11.0", "node-pty": "^1.1.0", "papaparse": "^5.5.3", "pdf-parse": "^2.4.5", - "update-electron-app": "^3.1.2", "xlsx": "^0.18.5", "zod": "^4.2.1" }, diff --git a/apps/x/apps/main/src/auth-server.ts b/apps/x/apps/main/src/auth-server.ts index 5c46ca3f..b1e93d67 100644 --- a/apps/x/apps/main/src/auth-server.ts +++ b/apps/x/apps/main/src/auth-server.ts @@ -20,9 +20,46 @@ export interface AuthServerResult { port: number; } +interface CallbackHandlingOpts { + callbackPath: string; + /** Invoked when the provider redirects back with an `error` param. */ + onError?: (error: string) => void; + /** + * Gatekeeper run BEFORE the error/callback handling. Return a message to + * reject the request with a polite close-this-tab error page — without + * invoking onCallback/onError. Lets a caller drop stale callbacks (e.g. the + * browser tab of a cancelled sign-in attempt carrying an old `state`) + * without disturbing the live flow. + */ + validateCallback?: (url: URL) => string | null; +} + +function renderErrorPage(res: import('http').ServerResponse, message: string): void { + res.writeHead(200, { 'Content-Type': 'text/html' }); + res.end(` + + + + OAuth Error + + + +

Authorization Failed

+

${escapeHtml(message)}

+

You can close this window.

+ + + + `); +} + function tryBindPort( port: number, - onCallback: (callbackUrl: URL) => void | Promise + onCallback: (callbackUrl: URL) => void | Promise, + opts: CallbackHandlingOpts, ): Promise { return new Promise((resolve, reject) => { const server = createServer((req, res) => { @@ -34,29 +71,24 @@ function tryBindPort( const url = new URL(req.url, `http://localhost:${port}`); - if (url.pathname === OAUTH_CALLBACK_PATH) { + if (url.pathname === opts.callbackPath) { + // Gatekeeper first: stale/foreign requests must not reach onError or + // onCallback (a stale tab's redirect must never settle a live flow). + const rejection = opts.validateCallback?.(url) ?? null; + if (rejection) { + renderErrorPage(res, rejection); + return; + } + const error = url.searchParams.get('error'); if (error) { - res.writeHead(200, { 'Content-Type': 'text/html' }); - res.end(` - - - - OAuth Error - - - -

Authorization Failed

-

Error: ${escapeHtml(error)}

-

You can close this window.

- - - - `); + // Surface the provider error (e.g. access_denied when the user + // cancels consent) so the caller can settle its flow instead of + // waiting for the timeout. Callers that don't opt in keep the old + // behaviour: the error page renders and the flow times out. + opts.onError?.(error); + renderErrorPage(res, `Error: ${error}`); return; } @@ -117,18 +149,29 @@ function tryBindPort( * through `port + PORT_RANGE_SIZE - 1` and binds the first available, handling * both EADDRINUSE and EACCES (the latter is common on Windows when * Hyper-V/WSL2 reserve the port). + * + * `callbackPath` overrides the served path for providers whose registered + * redirect URI differs (ChatGPT/Codex uses /auth/callback). `onError` is + * invoked when the provider redirects back with an `error` param (e.g. + * access_denied), letting the caller settle instead of waiting for timeout. + * `validateCallback` runs before both — see CallbackHandlingOpts. */ export async function createAuthServer( port: number = DEFAULT_PORT, onCallback: (callbackUrl: URL) => void | Promise, - opts: { fallback?: boolean } = {}, + opts: { fallback?: boolean } & Partial = {}, ): Promise { const fallback = opts.fallback === true; + const handlingOpts: CallbackHandlingOpts = { + callbackPath: opts.callbackPath ?? OAUTH_CALLBACK_PATH, + onError: opts.onError, + validateCallback: opts.validateCallback, + }; const limit = fallback ? port + PORT_RANGE_SIZE - 1 : port; for (let p = port; p <= limit; p++) { try { - return await tryBindPort(p, onCallback); + return await tryBindPort(p, onCallback, handlingOpts); } catch (err) { const code = (err as NodeJS.ErrnoException).code; if (fallback && (code === 'EADDRINUSE' || code === 'EACCES') && p < limit) { diff --git a/apps/x/apps/main/src/browser/extensions.ts b/apps/x/apps/main/src/browser/extensions.ts new file mode 100644 index 00000000..a521c86c --- /dev/null +++ b/apps/x/apps/main/src/browser/extensions.ts @@ -0,0 +1,129 @@ +import path from 'node:path'; +import fs from 'node:fs/promises'; +import { session, type BrowserWindow, type Session, type WebContents } from 'electron'; +import { ElectronChromeExtensions } from 'electron-chrome-extensions'; +import { WorkDir } from '@x/core/dist/config/config.js'; +import { browserViewManager } from './view.js'; + +/** + * Chrome extension support for the embedded browser, via + * electron-chrome-extensions (GPL-3.0 / Patron dual-licensed — see the + * library's LICENSE.md before shipping this in a release build). + * + * Extensions are loaded unpacked from ~/.rowboat/extensions// — either + * a directory containing manifest.json directly, or (Chrome Web Store + * unpacked layout) a directory whose single versioned subdirectory contains + * it. There is no install UI; drop a folder there and restart the app. + * + * Known limits: + * - The browser session's client-hint spoofing uses Electron's webRequest + * API, which prevents extensions' chrome.webRequest listeners from firing + * (Electron allows one consumer per session). Blockers that rely on + * webRequest (uBlock MV2) won't block; content-script-based extensions + * (Dark Reader, password managers) work. + * - declarativeNetRequest is not implemented by Electron. + */ + +const EXTENSIONS_DIR = path.join(WorkDir, 'extensions'); + +let extensions: ElectronChromeExtensions | null = null; + +/** + * Called once at startup (before any browser use). Binds the extension + * system to the browser session when the BrowserViewManager creates it, and + * mirrors the manager's tab lifecycle into chrome.tabs. + */ +export function setupBrowserExtensions(): void { + browserViewManager.on('session-created', (browserSession: Session) => { + initExtensions(browserSession); + }); + browserViewManager.on('tab-created', (wc: WebContents, win: BrowserWindow) => { + try { + extensions?.addTab(wc, win); + } catch (error) { + console.error('[Extensions] addTab failed:', error); + } + }); + browserViewManager.on('tab-selected', (wc: WebContents) => { + try { + extensions?.selectTab(wc); + } catch (error) { + console.error('[Extensions] selectTab failed:', error); + } + }); + + // Serves crx:// (extension action icons) to the app renderer, which hosts + // the element on the default session; the element's + // partition attribute routes state queries to the browser session. + ElectronChromeExtensions.handleCRXProtocol(session.defaultSession); +} + +function initExtensions(browserSession: Session): void { + if (extensions) return; + + extensions = new ElectronChromeExtensions({ + license: 'GPL-3.0', + session: browserSession, + async createTab(details) { + const result = await browserViewManager.newTab(details.url); + if (!result.ok || !result.tabId) { + throw new Error(result.error ?? 'Failed to create tab'); + } + const wc = browserViewManager.getTabWebContents(result.tabId); + const win = browserViewManager.getWindow(); + if (!wc || !win) throw new Error('Browser window is not available'); + return [wc, win]; + }, + selectTab(wc) { + const tabId = browserViewManager.getTabIdForWebContents(wc); + if (tabId) browserViewManager.switchTab(tabId); + }, + removeTab(wc) { + const tabId = browserViewManager.getTabIdForWebContents(wc); + if (tabId) browserViewManager.closeTab(tabId); + }, + }); + + void loadUnpackedExtensions(browserSession); +} + +async function resolveExtensionRoot(dir: string): Promise { + const hasManifest = async (candidate: string) => + fs.access(path.join(candidate, 'manifest.json')).then(() => true, () => false); + + if (await hasManifest(dir)) return dir; + + // Chrome Web Store unpacked layout: //manifest.json. Pick the + // lexically-latest version directory. + const entries = await fs.readdir(dir, { withFileTypes: true }).catch(() => []); + const versionDirs = entries + .filter((entry) => entry.isDirectory()) + .map((entry) => entry.name) + .sort() + .reverse(); + for (const name of versionDirs) { + const candidate = path.join(dir, name); + if (await hasManifest(candidate)) return candidate; + } + return null; +} + +async function loadUnpackedExtensions(browserSession: Session): Promise { + await fs.mkdir(EXTENSIONS_DIR, { recursive: true }); + const entries = await fs.readdir(EXTENSIONS_DIR, { withFileTypes: true }).catch(() => []); + + for (const entry of entries) { + if (!entry.isDirectory()) continue; + const root = await resolveExtensionRoot(path.join(EXTENSIONS_DIR, entry.name)); + if (!root) { + console.warn(`[Extensions] No manifest.json under ${path.join(EXTENSIONS_DIR, entry.name)} — skipped`); + continue; + } + try { + const extension = await browserSession.extensions.loadExtension(root); + console.log(`[Extensions] Loaded ${extension.name}@${extension.version} (${root})`); + } catch (error) { + console.error(`[Extensions] Failed to load ${root}:`, error); + } + } +} diff --git a/apps/x/apps/main/src/browser/ipc.ts b/apps/x/apps/main/src/browser/ipc.ts index 1d1bd3b0..4bdafe9b 100644 --- a/apps/x/apps/main/src/browser/ipc.ts +++ b/apps/x/apps/main/src/browser/ipc.ts @@ -1,6 +1,6 @@ import { BrowserWindow } from 'electron'; import { ipc } from '@x/shared'; -import { browserViewManager, type BrowserState, type HttpAuthRequest } from './view.js'; +import { browserViewManager, type BrowserState, type DisplayMediaRequest, type HttpAuthRequest } from './view.js'; type IPCChannels = ipc.IPCChannels; @@ -21,6 +21,7 @@ type BrowserHandlers = { 'browser:reload': InvokeHandler<'browser:reload'>; 'browser:getState': InvokeHandler<'browser:getState'>; 'browser:httpAuthResponse': InvokeHandler<'browser:httpAuthResponse'>; + 'browser:displayMediaResponse': InvokeHandler<'browser:displayMediaResponse'>; }; /** @@ -66,6 +67,9 @@ export const browserIpcHandlers: BrowserHandlers = { 'browser:httpAuthResponse': async (_event, args) => { return browserViewManager.respondToHttpAuth(args); }, + 'browser:displayMediaResponse': async (_event, args) => { + return browserViewManager.respondToDisplayMedia(args); + }, }; /** @@ -96,4 +100,12 @@ export function setupBrowserEventForwarding(): void { browserViewManager.on('http-auth-resolved', (requestId: string) => { broadcast('browser:httpAuthResolved', { requestId }); }); + + browserViewManager.on('display-media-request', (request: DisplayMediaRequest) => { + broadcast('browser:displayMediaRequest', request); + }); + + browserViewManager.on('display-media-resolved', (requestId: string) => { + broadcast('browser:displayMediaResolved', { requestId }); + }); } diff --git a/apps/x/apps/main/src/browser/view.ts b/apps/x/apps/main/src/browser/view.ts index 034aa06d..11a3a5bc 100644 --- a/apps/x/apps/main/src/browser/view.ts +++ b/apps/x/apps/main/src/browser/view.ts @@ -1,11 +1,12 @@ import { randomUUID } from 'node:crypto'; import { EventEmitter } from 'node:events'; -import { BrowserWindow, WebContentsView, session, shell, type Session, type WebContents } from 'electron'; +import { BrowserWindow, WebContentsView, desktopCapturer, session, shell, type Session, type WebContents } from 'electron'; import type { BrowserPageElement, BrowserPageSnapshot, BrowserState, BrowserTabState, + DisplayMediaRequest, HttpAuthRequest, } from '@x/shared/dist/browser-control.js'; import { normalizeNavigationTarget } from './navigation.js'; @@ -21,7 +22,7 @@ import { type RawBrowserPageSnapshot, } from './page-scripts.js'; -export type { BrowserPageSnapshot, BrowserState, BrowserTabState, HttpAuthRequest }; +export type { BrowserPageSnapshot, BrowserState, BrowserTabState, DisplayMediaRequest, HttpAuthRequest }; /** * Embedded browser pane implementation. @@ -64,6 +65,8 @@ const SPOOF_UA = buildChromeUserAgent(); const HOME_URL = 'https://www.google.com'; const NAVIGATION_TIMEOUT_MS = 10000; const HTTP_AUTH_TIMEOUT_MS = 120000; +const DISPLAY_MEDIA_TIMEOUT_MS = 120000; +const DISPLAY_MEDIA_THUMBNAIL_SIZE = { width: 320, height: 180 }; const POST_ACTION_IDLE_MS = 400; const POST_ACTION_MAX_ELEMENTS = 25; const POST_ACTION_MAX_TEXT_LENGTH = 4000; @@ -96,6 +99,14 @@ type PendingHttpAuth = { webContents: WebContents; }; +type PendingDisplayMedia = { + callback: (streams: Electron.Streams) => void; + timer: NodeJS.Timeout; + // The picker answers with a source id; the native callback needs the full + // DesktopCapturerSource, so keep the gathered sources until resolution. + sources: Map; +}; + const EMPTY_STATE: BrowserState = { activeTabId: null, tabs: [], @@ -138,6 +149,7 @@ export class BrowserViewManager extends EventEmitter { private bounds: BrowserBounds = { x: 0, y: 0, width: 0, height: 0 }; private snapshotCache = new Map(); private pendingHttpAuth = new Map(); + private pendingDisplayMedia = new Map(); // Child windows created by page window.open() (OAuth/SSO popups). Tracked so // they can be closed when the host window goes away — otherwise an orphaned // popup keeps BrowserWindow.getAllWindows() non-empty and, on macOS, blocks @@ -188,6 +200,9 @@ export class BrowserViewManager extends EventEmitter { for (const requestId of [...this.pendingHttpAuth.keys()]) { this.finishHttpAuth(requestId); } + for (const requestId of [...this.pendingDisplayMedia.keys()]) { + this.finishDisplayMedia(requestId); + } // Close any OAuth/SSO popups so they don't outlive the app window. for (const popup of popups) { if (!popup.isDestroyed()) popup.close(); @@ -244,10 +259,108 @@ export class BrowserViewManager extends EventEmitter { callback({ requestHeaders }); }); + // getDisplayMedia() from pages (e.g. Google Meet "Present now"). When the + // browser pane is on screen, forward a source picker to the renderer and + // resolve with the user's choice. Registered here (not in main.ts's + // configureSessionPermissions) so the picker plumbing lives with the rest + // of the pane's request/response wiring; the app's own session keeps its + // auto-approve loopback handler for meeting transcription. + browserSession.setDisplayMediaRequestHandler((_request, callback) => { + this.handleDisplayMediaRequest(callback).catch(() => callback({})); + }); + this.browserSession = browserSession; + // Synchronous on purpose: the extension system (browser/extensions.ts) + // must bind to the session — registering its tab-API preload — before the + // first tab's WebContentsView is constructed right after this returns. + this.emit('session-created', browserSession); return browserSession; } + /** + * Resolve a page's getDisplayMedia() request. With the pane visible, gather + * shareable sources and ask the renderer to show a picker (denied after a + * timeout if unanswered). With the pane hidden, deny outright: nobody can + * answer a picker, and auto-granting would hand a live screen capture to an + * arbitrary page without consent. `visible` is also false whenever the + * renderer hides the view behind a blocking overlay — including this + * picker's own dialog — so a page re-requesting mid-picker lands here and + * must be denied, not silently granted. + */ + private async handleDisplayMediaRequest(callback: (streams: Electron.Streams) => void): Promise { + if (!this.visible || !this.window) { + callback({}); + return; + } + + const sources = await desktopCapturer.getSources({ + types: ['screen', 'window'], + thumbnailSize: DISPLAY_MEDIA_THUMBNAIL_SIZE, + fetchWindowIcons: true, + }); + if (sources.length === 0) { + callback({}); + return; + } + + const requestId = randomUUID(); + const timer = setTimeout(() => { + this.finishDisplayMedia(requestId); + }, DISPLAY_MEDIA_TIMEOUT_MS); + this.pendingDisplayMedia.set(requestId, { + callback, + timer, + sources: new Map(sources.map((source) => [source.id, source])), + }); + + const request: DisplayMediaRequest = { + requestId, + sources: sources.map((source) => ({ + id: source.id, + name: source.name, + kind: source.id.startsWith('screen:') ? 'screen' as const : 'window' as const, + thumbnailDataUrl: source.thumbnail.isEmpty() ? '' : source.thumbnail.toDataURL(), + // appIcon is typed non-null but is absent for screen sources. + appIconDataUrl: source.appIcon && !source.appIcon.isEmpty() ? source.appIcon.toDataURL() : null, + })), + }; + this.emit('display-media-request', request); + } + + /** + * Resolve a pending display-media request. `sourceId === undefined` (or an + * id no longer in the gathered set) denies it. Always notifies the renderer + * so a picker it may still be showing (e.g. after a timeout) is pruned. + */ + private finishDisplayMedia(requestId: string, sourceId?: string, audio?: boolean): boolean { + const pending = this.pendingDisplayMedia.get(requestId); + if (!pending) return false; + this.pendingDisplayMedia.delete(requestId); + clearTimeout(pending.timer); + const source = sourceId != null ? pending.sources.get(sourceId) : undefined; + try { + if (!source) { + pending.callback({}); + } else if (audio) { + pending.callback({ video: source, audio: 'loopback' }); + } else { + pending.callback({ video: source }); + } + } catch { + // The requesting webContents may already be destroyed. + } + this.emit('display-media-resolved', requestId); + return true; + } + + respondToDisplayMedia(input: { + requestId: string; + sourceId?: string; + audio?: boolean; + }): { ok: boolean } { + return { ok: this.finishDisplayMedia(input.requestId, input.sourceId, input.audio) }; + } + private emitState(): void { this.emit('state-updated', this.snapshotState()); } @@ -560,6 +673,10 @@ export class BrowserViewManager extends EventEmitter { this.invalidateSnapshot(tabId); this.syncAttachedView(); this.emitState(); + // Register with the extension system (chrome.tabs) before the initial + // load below so extensions observe the tab's first navigation. + this.emit('tab-created', tab.view.webContents, this.window); + this.emit('tab-selected', tab.view.webContents); const targetUrl = initialUrl === 'about:blank' @@ -733,11 +850,13 @@ export class BrowserViewManager extends EventEmitter { } switchTab(tabId: string): { ok: boolean } { - if (!this.tabs.has(tabId)) return { ok: false }; + const tab = this.tabs.get(tabId); + if (!tab) return { ok: false }; if (this.activeTabId === tabId) return { ok: true }; this.activeTabId = tabId; this.syncAttachedView(); this.emitState(); + this.emit('tab-selected', tab.view.webContents); return { ok: true }; } @@ -1048,6 +1167,24 @@ export class BrowserViewManager extends EventEmitter { return this.snapshotState(); } + // Accessors for the extension system's chrome.tabs bridge + // (browser/extensions.ts), which speaks in WebContents while the manager + // speaks in tab ids. + getWindow(): BrowserWindow | null { + return this.window; + } + + getTabWebContents(tabId: string): WebContents | null { + return this.getTab(tabId)?.view.webContents ?? null; + } + + getTabIdForWebContents(wc: WebContents): string | null { + for (const tab of this.tabs.values()) { + if (tab.view.webContents === wc) return tab.id; + } + return null; + } + private snapshotState(): BrowserState { if (this.tabOrder.length === 0) return { ...EMPTY_STATE }; return { diff --git a/apps/x/apps/main/src/chatgpt-signin.ts b/apps/x/apps/main/src/chatgpt-signin.ts new file mode 100644 index 00000000..81e789d6 --- /dev/null +++ b/apps/x/apps/main/src/chatgpt-signin.ts @@ -0,0 +1,247 @@ +import { shell } from 'electron'; +import type { Server } from 'http'; +import { createAuthServer } from './auth-server.js'; +import * as oauthClient from '@x/core/dist/auth/oauth-client.js'; +import { exchangeChatGPTCode, getChatGPTStatus } from '@x/core/dist/auth/chatgpt-auth.js'; +import { + CHATGPT_AUTHORIZE_URL, + CHATGPT_CALLBACK_PATH, + CHATGPT_CALLBACK_PORT, + CHATGPT_CLIENT_ID, + CHATGPT_EXTRA_AUTHORIZE_PARAMS, + CHATGPT_REDIRECT_URI, + CHATGPT_SCOPES, +} from '@x/core/dist/auth/chatgpt-constants.js'; + +// Interactive "Sign in with ChatGPT" flow (OAuth 2.0 + PKCE, Codex CLI client +// — see chatgpt-constants.ts). Orchestration only: PKCE/state generation and +// all token-endpoint traffic + storage live in core; this module owns the +// system browser, the loopback callback server on 127.0.0.1:1455, and flow +// lifecycle. The port is FIXED — the redirect URI is pre-registered at OpenAI +// for the Codex client id, so there is no scan-to-next-port fallback. + +export type ChatGPTSignInResult = { + signedIn: boolean; + email?: string; + accountId?: string; + /** True when the attempt was cancelled (Cancel button or superseded). */ + cancelled?: boolean; + error?: string; +}; + +/** Generous, mirrors the Google flow's abandoned-flow cleanup ceiling. */ +const SIGN_IN_TIMEOUT_MS = 5 * 60 * 1000; + +type ActiveAttempt = { + promise: Promise; + /** + * Settle the attempt with a cancelled outcome. Resolves once the loopback + * server has fully closed (listener + keep-alive connections), so a + * follow-up attempt can rebind 1455 immediately. + */ + cancel: (reason: string) => Promise; +}; + +let activeAttempt: ActiveAttempt | null = null; + +/** + * Start a sign-in attempt. If one is already pending it is stale by + * definition — the user is clicking Sign In again precisely because no + * browser flow is visibly in progress (e.g. they closed the tab and hit + * Cancel) — so cancel it and start FRESH (new PKCE verifier/state, new + * loopback server, new browser window). Awaiting the cancel preserves the + * one-server-at-a-time invariant: 1455 is fully released before rebinding. + */ +export async function signInWithChatGPT(): Promise { + if (activeAttempt) { + const stale = activeAttempt; + activeAttempt = null; + console.log('[ChatGPTAuth] Cancelling stale sign-in attempt before starting a new one'); + await stale.cancel('Superseded by a new sign-in attempt.'); + } + + const attempt = startAttempt(); + activeAttempt = attempt; + void attempt.promise.finally(() => { + if (activeAttempt === attempt) activeAttempt = null; + }); + return attempt.promise; +} + +/** + * Abort the pending attempt (renderer Cancel button): stops the loopback + * server, clears pending state, settles the in-flight signIn promise with a + * cancelled outcome. No-op when nothing is pending. Never touches stored + * tokens — after cancel, chatgpt:getStatus reports signed-out (unless an + * earlier sign-in already completed). + */ +export async function cancelChatGPTSignIn(): Promise { + const attempt = activeAttempt; + if (!attempt) return; + activeAttempt = null; + await attempt.cancel('Sign-in cancelled.'); +} + +/** + * One sign-in attempt. The returned promise always RESOLVES (never rejects), + * and every exit path — success, denial, timeout, port busy, exchange + * failure, cancellation — tears down the loopback server and the timeout + * exactly once via the settle-once `finish`. + */ +function startAttempt(): ActiveAttempt { + let settle!: (result: ChatGPTSignInResult) => void; + const promise = new Promise((resolve) => { + settle = resolve; + }); + + let settled = false; + let server: Server | null = null; + let timeoutHandle: NodeJS.Timeout | null = null; + let serverClosed: Promise | null = null; + + // Close the listening socket AND any keep-alive connections (the browser + // holds one open after the callback response) so 1455 frees immediately. + const closeServer = (): Promise => { + if (serverClosed) return serverClosed; + const s = server; + server = null; + serverClosed = !s + ? Promise.resolve() + : new Promise((resolve) => { + s.close(() => resolve()); + s.closeAllConnections(); + }); + return serverClosed; + }; + + const finish = (result: ChatGPTSignInResult): Promise => { + if (settled) return serverClosed ?? Promise.resolve(); + settled = true; + if (timeoutHandle) clearTimeout(timeoutHandle); + const closed = closeServer(); + if (!result.signedIn) { + console.log(`[ChatGPTAuth] Sign-in did not complete: ${result.error ?? 'unknown'}`); + } + settle(result); + return closed; + }; + + const cancel = (reason: string): Promise => + finish({ signedIn: false, cancelled: true, error: reason }); + + void run(); + return { promise, cancel }; + + async function run(): Promise { + console.log('[ChatGPTAuth] Starting sign-in flow...'); + try { + const { verifier, challenge } = await oauthClient.generatePKCE(); + const state = oauthClient.generateState(); + if (settled) return; // cancelled while generating PKCE — nothing bound yet + + // Guard against duplicate callbacks (browser may send multiple requests). + let callbackHandled = false; + const onCallback = async (callbackUrl: URL) => { + if (settled || callbackHandled) return; + callbackHandled = true; + try { + // State already verified by validateCallback below. + const code = callbackUrl.searchParams.get('code'); + if (!code) { + void finish({ signedIn: false, error: 'Sign-in failed: callback is missing the authorization code.' }); + return; + } + // Exchange + persistence live in core (never log token values). + await exchangeChatGPTCode(code, verifier); + const status = await getChatGPTStatus(); + console.log('[ChatGPTAuth] Sign-in complete'); + void finish({ ...status }); + } catch (error) { + console.error('[ChatGPTAuth] Token exchange failed:', error); + void finish({ + signedIn: false, + error: error instanceof Error ? error.message : 'Token exchange failed', + }); + } + }; + + // Bind the loopback server FIRST so a busy port fails fast, before any + // browser tab opens. Fixed port — createAuthServer's no-fallback error + // message tells the user to free the port. + let boundServer: Server; + try { + ({ server: boundServer } = await createAuthServer(CHATGPT_CALLBACK_PORT, onCallback, { + callbackPath: CHATGPT_CALLBACK_PATH, + onError: (error) => { + void finish({ + signedIn: false, + error: error === 'access_denied' + ? 'Sign-in was cancelled in the browser.' + : `Sign-in failed: ${error}`, + }); + }, + // Stale callbacks — a tab left over from an earlier, cancelled + // attempt carries that attempt's `state` — get a polite + // close-this-tab page and never reach onError/onCallback, so they + // can neither complete sign-in nor settle the live attempt. + validateCallback: (url) => { + if (settled) { + return 'This sign-in attempt is no longer active. Close this tab and try again from Rowboat.'; + } + if (url.searchParams.get('state') !== state) { + return 'This sign-in link has expired. Close this tab and try again from Rowboat.'; + } + return null; + }, + })); + } catch (error) { + void finish({ + signedIn: false, + error: error instanceof Error ? error.message : 'Failed to start the sign-in callback server', + }); + return; + } + + if (settled) { + // Cancelled while the bind was in flight — release the port we just + // grabbed (finish() already ran with no server to close). + boundServer.closeAllConnections(); + boundServer.close(); + return; + } + server = boundServer; + + timeoutHandle = setTimeout(() => { + void finish({ signedIn: false, error: 'Sign-in timed out. Please try again.' }); + }, SIGN_IN_TIMEOUT_MS); + + const authUrl = new URL(CHATGPT_AUTHORIZE_URL); + authUrl.search = new URLSearchParams({ + response_type: 'code', + client_id: CHATGPT_CLIENT_ID, + redirect_uri: CHATGPT_REDIRECT_URI, + scope: CHATGPT_SCOPES.join(' '), + code_challenge: challenge, + code_challenge_method: 'S256', + state, + ...CHATGPT_EXTRA_AUTHORIZE_PARAMS, + }).toString(); + + try { + // System browser: shares the user's existing ChatGPT session cookies. + await shell.openExternal(authUrl.toString()); + } catch (error) { + void finish({ + signedIn: false, + error: error instanceof Error ? `Failed to open browser: ${error.message}` : 'Failed to open browser', + }); + } + } catch (error) { + console.error('[ChatGPTAuth] Sign-in flow error:', error); + void finish({ + signedIn: false, + error: error instanceof Error ? error.message : 'Sign-in failed', + }); + } + } +} diff --git a/apps/x/apps/main/src/ipc.ts b/apps/x/apps/main/src/ipc.ts index 99bf361c..cd8c3551 100644 --- a/apps/x/apps/main/src/ipc.ts +++ b/apps/x/apps/main/src/ipc.ts @@ -40,6 +40,9 @@ import { isSignedIn } from '@x/core/dist/account/account.js'; import { listGatewayModels } from '@x/core/dist/models/gateway.js'; import type { IModelConfigRepo } from '@x/core/dist/models/repo.js'; import type { IOAuthRepo } from '@x/core/dist/auth/repo.js'; +import { getChatGPTStatus, signOutChatGPT } from '@x/core/dist/auth/chatgpt-auth.js'; +import { listCodexModels } from '@x/core/dist/models/codex.js'; +import { signInWithChatGPT, cancelChatGPTSignIn } from './chatgpt-signin.js'; import { IGranolaConfigRepo } from '@x/core/dist/knowledge/granola/repo.js'; import { ICodeModeConfigRepo } from '@x/core/dist/code-mode/repo.js'; import { CodePermissionRegistry } from '@x/core/dist/code-mode/acp/permission-registry.js'; @@ -66,6 +69,7 @@ import { rankSlackHomeMessages } from '@x/core/dist/knowledge/sources/rank_slack import { syncSlackKnowledgeSources, triggerSync as triggerSlackKnowledgeSync, getSlackKnowledgeSyncStatus } from '@x/core/dist/knowledge/sources/sync_slack.js'; import { isOnboardingComplete, markOnboardingComplete } from '@x/core/dist/config/note_creation_config.js'; import { loadNotificationSettings, saveNotificationSettings } from '@x/core/dist/config/notification_config.js'; +import { loadTurnLimitsSettings, saveTurnLimitsSettings } from '@x/core/dist/config/turn_limits.js'; import { saveAppSettings } from '@x/core/dist/config/app_settings.js'; import { setSelfCaptureActive } from '@x/core/dist/meetings/detector.js'; import { notifyIfEnabled } from '@x/core/dist/application/notification/notifier.js'; @@ -85,6 +89,8 @@ import * as appsIndexer from '@x/core/dist/apps/indexer.js'; import * as appsServer from '@x/core/dist/apps/server.js'; import * as appsAgents from '@x/core/dist/apps/agents.js'; import { capture } from '@x/core/dist/analytics/posthog.js'; +import { recordAppVersion, isVersionUpgrade } from '@x/core/dist/config/app_version.js'; +import { getUpdaterStatus, checkForUpdates, quitAndInstallUpdate } from './updater.js'; import * as githubAuth from '@x/core/dist/apps/github-auth.js'; import * as appsStars from '@x/core/dist/apps/stars.js'; import * as appsInstaller from '@x/core/dist/apps/installer.js'; @@ -108,6 +114,7 @@ import { invalidateKnowledgeIndex } from '@x/core/dist/knowledge/knowledge_index import { versionHistory, voice } from '@x/core'; import { classifySchedule, processRowboatInstruction } from '@x/core/dist/knowledge/inline_tasks.js'; import { getBillingInfo } from '@x/core/dist/billing/billing.js'; +import { claimReferralCode, getCreditsState, maybeActivateCredit, subscribeCreditActivations } from '@x/core/dist/billing/credits.js'; import { summarizeMeeting } from '@x/core/dist/knowledge/summarize_meeting.js'; import { getAccessToken } from '@x/core/dist/auth/tokens.js'; import { getRowboatConfig } from '@x/core/dist/config/rowboat.js'; @@ -657,6 +664,11 @@ export function emitOAuthEvent(event: { provider: string; success: boolean; erro // prompt, so any OAuth state change must rebuild it. invalidateCopilotInstructionsCache(); broadcastToWindows('oauth:didConnect', event); + // Google connect (both BYOK and rowboat-mode paths funnel through here) is + // the "connected Gmail" first-time reward. + if (event.provider === 'google' && event.success) { + void maybeActivateCredit('first_gmail_connected'); + } } async function requireCodeSession(sessionId: string): Promise { @@ -833,6 +845,10 @@ export function stopServicesWatcher(): void { // Handler Implementations // ============================================================================ +// app:consumeUpdateInfo returns `updatedFrom` at most once per app run, so a +// renderer reload doesn't re-show the "updated to vX" card. +let updateNoticeConsumed = false; + /** * Register all IPC handlers * Add new handlers here as you add channels to IPCChannels @@ -841,6 +857,10 @@ export function setupIpcHandlers() { // Forward knowledge commit events to renderer for panel refresh versionHistory.onCommit(() => emitKnowledgeCommitEvent()); + // Relay backend-confirmed credit grants (first-time-action rewards) to all + // windows so the UI can update balances and celebrate. + subscribeCreditActivations((event) => broadcastToWindows('credits:didActivate', event)); + // Pre-warm the Gmail contact indices so the first compose-box keystroke is instant. // - warmContactIndex(): synchronous local-snapshot fallback (instant, narrow coverage). // - warmSentContacts(): kicks off a background Gmail API sync of the SENT label @@ -856,6 +876,28 @@ export function setupIpcHandlers() { 'app:consumePendingDeepLink': async () => { return { url: consumePendingDeepLink() }; }, + 'app:consumeUpdateInfo': async () => { + const version = app.getVersion(); + if (updateNoticeConsumed) return { version, updatedFrom: null }; + updateNoticeConsumed = true; + const changedFrom = recordAppVersion(version); + // Downgrades still restamp (so the next upgrade reports correctly) but + // don't toast "Updated to vX" or count as a client update. + const updatedFrom = changedFrom && isVersionUpgrade(changedFrom, version) ? changedFrom : null; + // 'app_updated' is taken by the in-app apps feature; this is the client itself. + if (updatedFrom) capture('client_updated', { from: updatedFrom, to: version }); + return { version, updatedFrom }; + }, + 'updater:getStatus': async () => { + return getUpdaterStatus(); + }, + 'updater:check': async () => { + return checkForUpdates(); + }, + 'updater:quitAndInstall': async () => { + quitAndInstallUpdate(); + return {}; + }, 'app:consumePendingTrayCommand': async () => { return { toggleMeetingNotes: consumePendingToggleMeetingNotes() }; }, @@ -961,7 +1003,11 @@ export function setupIpcHandlers() { return {}; }, 'gmail:sendReply': async (_event, args) => { - return sendThreadReply(args); + const result = await sendThreadReply(args); + if (!result.error) { + void maybeActivateCredit('first_email_sent'); + } + return result; }, 'gmail:saveDraft': async (_event, args) => { return saveThreadDraft(args); @@ -1204,10 +1250,21 @@ export function setupIpcHandlers() { } }, 'models:list': async () => { - if (await isSignedIn()) { - return await listGatewayModels(); + const base = (await isSignedIn()) + ? await listGatewayModels() + : await listOnboardingModels(); + // ChatGPT-subscription (codex) models are additive and independent of + // Rowboat sign-in; their failure must never break the main list. + try { + const chatgpt = await getChatGPTStatus(); + if (chatgpt.signedIn) { + const codex = await listCodexModels(); + return { providers: [...base.providers, ...codex.providers] }; + } + } catch (error) { + console.warn('[Codex] Listing subscription models failed:', error); } - return await listOnboardingModels(); + return base; }, 'models:test': async (_event, args) => { return await testModelConnection(args.provider, args.model); @@ -1257,6 +1314,32 @@ export function setupIpcHandlers() { const config = await repo.getClientFacingConfig(); return { config }; }, + 'chatgpt:getStatus': async () => { + return await getChatGPTStatus(); + }, + 'chatgpt:signIn': async () => { + const result = await signInWithChatGPT(); + if (result.signedIn) { + // Model lists gate on sign-in state (composer picker, models:list) — + // push the change so they refresh without polling. + broadcastToWindows('chatgpt:statusChanged', { signedIn: true }); + } + return result; + }, + 'chatgpt:cancelSignIn': async () => { + await cancelChatGPTSignIn(); + return { success: true }; + }, + 'chatgpt:signOut': async () => { + try { + await signOutChatGPT(); + broadcastToWindows('chatgpt:statusChanged', { signedIn: false }); + return { success: true }; + } catch (error) { + console.error('[ChatGPTAuth] Sign-out failed:', error); + return { success: false }; + } + }, 'account:getRowboat': async () => { const signedIn = await isSignedIn(); if (!signedIn) { @@ -1732,6 +1815,13 @@ export function setupIpcHandlers() { lastAppsFingerprint = fingerprint; invalidateCopilotInstructionsCache(); } + // The copilot builds apps by writing the folder directly — apps:create is + // never on that path — so the first-app reward triggers off observed + // state instead: a valid non-installed app means the user built one. + // Cheap on repeat polls (maybeActivateCredit short-circuits once claimed). + if (apps.some((a) => a.kind === 'local' && a.status === 'ok')) { + void maybeActivateCredit('first_app_built'); + } return { serverRunning: status.running, ...(status.error ? { serverError: status.error } : {}), @@ -1751,6 +1841,7 @@ export function setupIpcHandlers() { 'apps:create': async (_event, args) => { const app = await appsIndexer.createApp(args); capture('app_created', { folder: app.folder }); + void maybeActivateCredit('first_app_built'); return { app }; }, 'apps:delete': async (_event, args) => { @@ -2124,6 +2215,9 @@ export function setupIpcHandlers() { }, 'meeting:summarize': async (_event, args) => { const notes = await summarizeMeeting(args.transcript, args.meetingStartTime, args.calendarEventJson); + if (notes && notes.trim()) { + void maybeActivateCredit('first_meeting_note'); + } return { notes }; }, 'meeting-prep:resolve': async (_event, args) => { @@ -2415,6 +2509,7 @@ export function setupIpcHandlers() { ...(args.model ? { model: args.model } : {}), ...(args.provider ? { provider: args.provider } : {}), }); + void maybeActivateCredit('first_bg_agent'); return { success: true, slug }; } catch (err) { return { success: false, error: err instanceof Error ? err.message : String(err) }; @@ -2451,6 +2546,13 @@ export function setupIpcHandlers() { 'billing:getInfo': async () => { return await getBillingInfo(); }, + // First-time-action credit rewards + 'credits:getState': async () => { + return await getCreditsState(); + }, + 'referral:claim': async (_event, args) => { + return await claimReferralCode(args.code); + }, 'notifications:getSettings': async () => { return loadNotificationSettings(); }, @@ -2458,6 +2560,13 @@ export function setupIpcHandlers() { saveNotificationSettings(args); return { success: true }; }, + 'turnLimits:getSettings': async () => { + return await loadTurnLimitsSettings(); + }, + 'turnLimits:setSettings': async (_event, args) => { + await saveTurnLimitsSettings(args); + return { success: true }; + }, // Embedded browser handlers (WebContentsView + navigation) ...browserIpcHandlers, }); diff --git a/apps/x/apps/main/src/main.ts b/apps/x/apps/main/src/main.ts index c0deeec4..fc1bd2d7 100644 --- a/apps/x/apps/main/src/main.ts +++ b/apps/x/apps/main/src/main.ts @@ -18,7 +18,7 @@ import { import { disposeAllTerminals } from "./terminal.js"; import { fileURLToPath, pathToFileURL } from "node:url"; import { dirname } from "node:path"; -import { updateElectronApp, UpdateSourceType } from "update-electron-app"; +import { initUpdater } from "./updater.js"; import { init as initGmailSync } from "@x/core/dist/knowledge/sync_gmail.js"; import { init as initCalendarSync } from "@x/core/dist/knowledge/sync_calendar.js"; import { init as initFirefliesSync } from "@x/core/dist/knowledge/sync_fireflies.js"; @@ -40,6 +40,7 @@ import { startSkillsWatcher, stopSkillsWatcher } from "@x/core/dist/runtime/asse import { init as initAppsServer, shutdown as shutdownAppsServer } from "@x/core/dist/apps/server.js"; import { registerAppsHostApi } from "@x/core/dist/apps/host-api.js"; import { setTokenCipher as setGithubTokenCipher } from "@x/core/dist/apps/github-auth.js"; +import { setTokenCipher as setChatGPTTokenCipher } from "@x/core/dist/auth/chatgpt-auth.js"; import { shutdown as shutdownAnalytics } from "@x/core/dist/analytics/posthog.js"; import { identifyIfSignedIn } from "@x/core/dist/analytics/identify.js"; import { migrateRuns } from "@x/core/dist/migrations/runs/migrate.js"; @@ -56,6 +57,7 @@ import type { CodeModeManager } from "@x/core/dist/code-mode/acp/manager.js"; import type { ISessions } from "@x/core/dist/runtime/sessions/index.js"; import { browserViewManager, BROWSER_PARTITION } from "./browser/view.js"; import { setupBrowserEventForwarding } from "./browser/ipc.js"; +import { setupBrowserExtensions } from "./browser/extensions.js"; import { ElectronBrowserControlService } from "./browser/control-service.js"; import { ElectronNotificationService } from "./notification/electron-notification-service.js"; import { @@ -227,52 +229,35 @@ protocol.registerSchemesAsPrivileged([ const ALLOWED_SESSION_PERMISSIONS = new Set(["media", "display-capture", "clipboard-read", "clipboard-sanitized-write"]); -// On Linux, Chromium's loopback capture records the default sink's monitor -// through the PulseAudio layer, at the monitor source's own volume. Desktop -// tools sometimes leave that volume near zero — it's invisible plumbing that -// doesn't affect what the user hears — which turns the whole capture into -// digital silence with no error anywhere. Raise it back to 100% before -// capture starts (raise only, so a deliberate >100% boost is left alone). -// Best-effort: no pactl or no Pulse layer just skips. -async function ensureLinuxMonitorVolume(): Promise { - const execFileP = promisify(execFile); - try { - const { stdout: sinkOut } = await execFileP("pactl", ["get-default-sink"], { timeout: 3000 }); - const monitor = `${sinkOut.trim()}.monitor`; - const { stdout: volOut } = await execFileP("pactl", ["get-source-volume", monitor], { timeout: 3000 }); - const percents = [...volOut.matchAll(/(\d+)%/g)].map((m) => Number(m[1])); - if (percents.length === 0 || Math.min(...percents) >= 100) return; - await execFileP("pactl", ["set-source-volume", monitor, "100%"], { timeout: 3000 }); - console.log(`[meeting] Raised ${monitor} volume from ${Math.min(...percents)}% to 100% for system-audio capture`); - } catch { - // pactl missing or non-Pulse audio stack — nothing to fix here. - } -} +// Granted to the embedded browser partition on top of the base set. +// `notifications` lets sites (WhatsApp Web, Gmail, Slack, ...) show native OS +// notifications via the HTML5 Notification API — Electron renders these +// through the system notification center once the permission resolves to +// granted. Background Web Push is still unavailable (Electron has no FCM), +// so notifications only fire while the site is loaded in a tab. The app's +// own renderer keeps the base set; it notifies through the main-process +// notification service instead. +const BROWSER_EXTRA_PERMISSIONS = ["notifications"] as const; + +function configureSessionPermissions(targetSession: Session, extraPermissions: readonly string[] = []): void { + const allowed = new Set([...ALLOWED_SESSION_PERMISSIONS, ...extraPermissions]); -function configureSessionPermissions(targetSession: Session): void { targetSession.setPermissionCheckHandler((_webContents, permission) => { - return ALLOWED_SESSION_PERMISSIONS.has(permission); + return allowed.has(permission); }); targetSession.setPermissionRequestHandler((_webContents, permission, callback) => { - callback(ALLOWED_SESSION_PERMISSIONS.has(permission)); + callback(allowed.has(permission)); }); +} - // Auto-approve display media requests and route system audio as loopback. - // Electron requires a video source in the callback even if we only want audio. - // We pass the first available screen source; the renderer discards the video track. - targetSession.setDisplayMediaRequestHandler(async (request, callback) => { - // On Linux, enumerating screens via desktopCapturer goes through the - // Wayland screencast portal, which can block on a system dialog or hang - // outright. Requests that want audio (meeting transcription — the only - // audio consumer; it discards the video track) don't need a real screen, - // so answer with the requesting frame as the mandatory video source and - // Chromium's PulseAudio loopback for the audio. - if (process.platform === 'linux' && request.audioRequested && request.frame) { - await ensureLinuxMonitorVolume(); - callback({ video: request.frame, audio: 'loopback' }); - return; - } +// Auto-approve display media requests and route system audio as loopback. +// Electron requires a video source in the callback even if we only want audio. +// We pass the first available screen source; the renderer discards the video track. +// App session only — the embedded browser partition registers its own handler +// (a user-facing source picker) in BrowserViewManager. +function configureAppDisplayMediaHandler(targetSession: Session): void { + targetSession.setDisplayMediaRequestHandler(async (_request, callback) => { const sources = await desktopCapturer.getSources({ types: ['screen'] }); if (sources.length === 0) { callback({}); @@ -380,7 +365,8 @@ function createWindow(options: { startHidden?: boolean } = {}) { }); configureSessionPermissions(session.defaultSession); - configureSessionPermissions(session.fromPartition(BROWSER_PARTITION)); + configureAppDisplayMediaHandler(session.defaultSession); + configureSessionPermissions(session.fromPartition(BROWSER_PARTITION), BROWSER_EXTRA_PERMISSIONS); mainWindow = win; setMainWindowForDeepLinks(win); @@ -465,16 +451,9 @@ app.whenReady().then(async () => { // serves workspace files via app://workspace/ for media previews. registerAppProtocol(); - // Initialize auto-updater (only in production) - if (app.isPackaged) { - updateElectronApp({ - updateSource: { - type: UpdateSourceType.ElectronPublicUpdateService, - repo: "rowboatlabs/rowboat", - }, - notifyUser: true, // Shows native dialog when update is available - }); - } + // Initialize auto-updater (no-ops in dev). Update state is pushed to the + // renderer (updater:status), which owns the restart prompt — see updater.ts. + initUpdater(); // The agent-slack CLI ships bundled with the app (.package/dist/agent-slack.cjs) // and is resolved per call by the shared executor in @x/core. Availability is @@ -505,6 +484,7 @@ app.whenReady().then(async () => { setupIpcHandlers(); setupBrowserEventForwarding(); + setupBrowserExtensions(); // Start the Rowboat Apps server (per-app origins on 127.0.0.1:3210) BEFORE // the window and the long service-init chain below. The Apps view is @@ -520,6 +500,12 @@ app.whenReady().then(async () => { encrypt: (plain) => safeStorage.encryptString(plain).toString('base64'), decrypt: (encrypted) => safeStorage.decryptString(Buffer.from(encrypted, 'base64')), }); + // ChatGPT subscription tokens at rest: same keychain-backed cipher. + setChatGPTTokenCipher({ + isAvailable: () => safeStorage.isEncryptionAvailable(), + encrypt: (plain) => safeStorage.encryptString(plain).toString('base64'), + decrypt: (encrypted) => safeStorage.decryptString(Buffer.from(encrypted, 'base64')), + }); initAppsServer().catch((error) => { console.error('[Apps] Failed to start:', error); }); diff --git a/apps/x/apps/main/src/updater.ts b/apps/x/apps/main/src/updater.ts new file mode 100644 index 00000000..53369548 --- /dev/null +++ b/apps/x/apps/main/src/updater.ts @@ -0,0 +1,164 @@ +import { app, autoUpdater, net, nativeImage, BrowserWindow } from "electron"; +import { capture } from "@x/core/dist/analytics/posthog.js"; +import type { ipc } from "@x/shared"; + +export type UpdaterStatus = ipc.IPCChannels["updater:status"]["req"]; + +const REPO = "rowboatlabs/rowboat"; +const CHECK_INTERVAL_MS = 10 * 60 * 1000; + +let status: UpdaterStatus = { state: "disabled", version: "", reason: "dev" }; + +function setStatus(next: Omit): void { + status = { version: status.version, ...next }; + for (const win of BrowserWindow.getAllWindows()) { + if (!win.isDestroyed() && win.webContents) { + win.webContents.send("updater:status", status); + } + } +} + +export function getUpdaterStatus(): UpdaterStatus { + return status; +} + +// 32x32 green dot with a white ring (scratchpad-generated PNG). Windows' +// counterpart of the macOS dock badge: overlays the taskbar icon while an +// update is staged. Cleared implicitly — installing quits the process. +const WIN_BADGE_DATA_URL = + "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAABPUlEQVR42s1XOwoCMRC12CvkAhb2HmMvYS/kCgveQU9gYS17AKu1EKwXsdDCwtQWtk9GkiUbkv2RkAw8WLLJzEvmk8kMwCwmpixiAHIAHEAhweUYC0Ugk0Yq9Esl52a+CJAygfEi5NrJBOg4S5vm6+eOgzhh+zr+Qd805pCyyzUu4wsAta7l+X1j89hjeVljfl5ZQf9oDs01pJY6BxFgpnHapcuoC7TGQoINIdA6dn7bjTauQGst7ugkwH0Z7yDBXQQyPdqnHPtAdwg9Ra27pyDyZVzBCExuI9AUGYpk3wRIp1GsWgSY/rcr1aaCdBrCdAK5XmR8G1cwilWuE2j8T1UtFAHSbcaBIlCEiP6ebCiSIhDdBdGDMHoaRi9ESZTi6JdR9Os4iYYkiZYselOaRFuexMMkmadZEo/TYPgB7Se8LkyPD5UAAAAASUVORK5CYII="; + +function showReadyBadge(): void { + if (process.platform === "darwin") { + // The window may be closed for days on macOS (app keeps running) — the + // dock badge is the only surface that says "an update is waiting". + app.dock?.setBadge("1"); + } else if (process.platform === "win32") { + const badge = nativeImage.createFromDataURL(WIN_BADGE_DATA_URL); + for (const win of BrowserWindow.getAllWindows()) { + if (!win.isDestroyed()) win.setOverlayIcon(badge, "Update ready — restart to install"); + } + } +} + +/** + * Initialize auto-update, driving Electron's autoUpdater (Squirrel) against + * update.electronjs.org directly. Events are forwarded to the renderer + * (updater:status), which shows a "Restart to update" card once the update + * is staged. By then Squirrel has already installed it — the card only asks + * for the restart, Chrome-style. Must be called after app ready (it is — + * from whenReady() in main.ts). + */ +export function initUpdater(): void { + const version = app.getVersion(); + + if (!app.isPackaged) { + status = { state: "disabled", version, reason: "dev" }; + return; + } + if (process.platform === "linux") { + // Electron's autoUpdater doesn't support Linux (deb/zip installs). + status = { state: "unsupported", version, reason: "platform" }; + return; + } + if (process.platform === "darwin" && !app.isInApplicationsFolder()) { + // Squirrel.Mac swaps the .app bundle in place, which fails outside + // /Applications (DMG mount, ~/Downloads). Don't wire the updater — + // Settings > Help tells the user to move the app. + status = { state: "unsupported", version, reason: "not-in-applications" }; + return; + } + + status = { state: "idle", version }; + + autoUpdater.on("checking-for-update", () => { + setStatus({ state: "checking", lastCheckedAt: status.lastCheckedAt }); + }); + autoUpdater.on("update-available", () => { + setStatus({ state: "downloading" }); + }); + autoUpdater.on("update-not-available", () => { + setStatus({ state: "idle", lastCheckedAt: Date.now() }); + }); + autoUpdater.on("update-downloaded", (_event, releaseNotes, releaseName) => { + // macOS (Squirrel.Mac fed by update.electronjs.org) supplies both the + // release name and the GitHub release body; Squirrel.Windows only the + // name. When notes are missing the card shows a static fallback line. + setStatus({ + state: "ready", + newVersion: releaseName || undefined, + releaseNotes: releaseNotes || undefined, + }); + showReadyBadge(); + // Squirrel.Windows never carries notes, and Squirrel.Mac's copy is a + // snapshot from download time — stale when the release body is edited + // after publish (update.electronjs.org caching widens that window). + // Refresh from the GitHub API; on failure the snapshot (or the card's + // static fallback line) stands. + void backfillReleaseNotes(releaseName || undefined); + }); + autoUpdater.on("error", (err) => { + setStatus({ state: "error", error: err.message, lastCheckedAt: status.lastCheckedAt }); + capture("update_failed", { message: err.message }); + }); + + // update.electronjs.org serves both Squirrel dialects from one URL: + // Squirrel.Mac GETs it as-is (204 = up to date, JSON = update; that legacy + // format is serverType "default"), Squirrel.Windows appends /RELEASES. + autoUpdater.setFeedURL({ + url: `https://update.electronjs.org/${REPO}/${process.platform}-${process.arch}/${version}`, + serverType: "default", + }); + // Check now and every 10 minutes, through the same guard as the manual + // check: a tick is a no-op while a check/download is in flight or an + // update is already staged. + checkForUpdates(); + setInterval(checkForUpdates, CHECK_INTERVAL_MS); +} + +/** + * Replace the staged update's release notes with the current GitHub release + * body. releaseName is "0.7.7" from Squirrel.Windows and "v0.7.7" from + * Squirrel.Mac — normalize to the tag form. + */ +async function backfillReleaseNotes(releaseName: string | undefined): Promise { + if (!releaseName) return; + try { + const tag = `v${releaseName.replace(/^v/, "")}`; + const res = await net.fetch(`https://api.github.com/repos/${REPO}/releases/tags/${tag}`, { + headers: { Accept: "application/vnd.github+json", "User-Agent": "Rowboat" }, + }); + if (!res.ok) return; + const { body } = (await res.json()) as { body?: string }; + const notes = body?.trim(); + // Re-check the state: the fetch raced user actions (quitAndInstall). + if (notes && status.state === "ready" && status.newVersion === releaseName) { + setStatus({ ...status, releaseNotes: notes }); + } + } catch { + // Offline or rate-limited — the Squirrel snapshot / fallback line stands. + } +} + +/** + * Manual "Check for updates". Only meaningful when idle or errored; + * checking/downloading are already in flight and ready is already staged. + * Returns the snapshot after initiating. + */ +export function checkForUpdates(): UpdaterStatus { + if (status.state === "idle" || status.state === "error") { + try { + autoUpdater.checkForUpdates(); + } catch (err) { + const error = err instanceof Error ? err : new Error(String(err)); + setStatus({ state: "error", error: error.message, lastCheckedAt: status.lastCheckedAt }); + capture("update_failed", { message: error.message }); + } + } + return status; +} + +export function quitAndInstallUpdate(): void { + capture("update_restarted", { from: status.version, to: status.newVersion }); + autoUpdater.quitAndInstall(); +} diff --git a/apps/x/apps/preload/package.json b/apps/x/apps/preload/package.json index 910d241e..056f14c3 100644 --- a/apps/x/apps/preload/package.json +++ b/apps/x/apps/preload/package.json @@ -7,7 +7,8 @@ "build": "rm -rf dist && tsc && esbuild dist/preload.js --bundle --platform=node --format=cjs --external:electron --outfile=dist/preload.bundle.js && mv dist/preload.bundle.js dist/preload.js" }, "dependencies": { - "@x/shared": "workspace:*" + "@x/shared": "workspace:*", + "electron-chrome-extensions": "^4.9.0" }, "devDependencies": { "electron": "^39.2.7", diff --git a/apps/x/apps/preload/src/preload.ts b/apps/x/apps/preload/src/preload.ts index bc69d4bb..44b33381 100644 --- a/apps/x/apps/preload/src/preload.ts +++ b/apps/x/apps/preload/src/preload.ts @@ -1,6 +1,19 @@ import { contextBridge, ipcRenderer, webFrame, webUtils } from 'electron'; +import { injectBrowserAction } from 'electron-chrome-extensions/browser-action'; import { ipc as ipcShared } from '@x/shared'; +// Expose the custom element (extension action icons + +// popups for the embedded browser pane). App documents only — this preload +// is attached solely to the app window, but guard against it ever being +// reused for remote content. +if (location.protocol === 'app:' || location.origin === 'http://localhost:5173') { + try { + injectBrowserAction(); + } catch (error) { + console.error('[preload] injectBrowserAction failed:', error); + } +} + type InvokeChannels = ipcShared.InvokeChannels; type IPCChannels = ipcShared.IPCChannels; type SendChannels = ipcShared.SendChannels; diff --git a/apps/x/apps/renderer/src/App.css b/apps/x/apps/renderer/src/App.css index 6a7b644a..899aa433 100644 --- a/apps/x/apps/renderer/src/App.css +++ b/apps/x/apps/renderer/src/App.css @@ -1445,3 +1445,10 @@ transform: translateX(-120%); } } + +/* Issue #749: BiDi. unicode-bidi:plaintext resolves direction PER BLOCK from + its first strong character, so a message mixing English + RTL paragraphs + aligns each block correctly. text-align:start follows the resolved dir. */ +.bidi-auto :is(p,h1,h2,h3,h4,h5,h6,li,blockquote,td,th){unicode-bidi:plaintext;text-align:start;} +/* Code is inherently LTR — keep it stable inside RTL messages. */ +.bidi-auto :is(pre,code,kbd,samp){direction:ltr;unicode-bidi:isolate;text-align:left;} diff --git a/apps/x/apps/renderer/src/App.tsx b/apps/x/apps/renderer/src/App.tsx index 3ee42e8c..58b78d14 100644 --- a/apps/x/apps/renderer/src/App.tsx +++ b/apps/x/apps/renderer/src/App.tsx @@ -79,7 +79,9 @@ import { Tooltip, TooltipContent, TooltipProvider, TooltipTrigger } from "@/comp import { Dialog, DialogContent, DialogHeader, DialogTitle, DialogDescription, DialogFooter } from "@/components/ui/dialog" import { Button } from "@/components/ui/button" import { Toaster } from "@/components/ui/sonner" +import { UpdateCard } from "@/components/update-card" import { BillingErrorDialog } from "@/components/billing-error-dialog" +import { CreditCelebration } from "@/components/credit-celebration" import { matchBillingError, type BillingErrorMatch } from "@/lib/billing-error" import { dispatchCreditExhausted, dispatchCreditReplenished } from "@/lib/credit-status" import { ensureMarkdownExtension, normalizeWikiPath, splitWikiFragment, stripKnowledgePrefix, toKnowledgePath, wikiLabel } from '@/lib/wiki-links' @@ -3019,6 +3021,13 @@ function App() { // via the agent resolver; keep them session-sticky where possible so the // provider prefix cache survives across turns. const reasoningEffort = reasoningEffortByTabRef.current.get(submitTabId) + // The runtime defaults omitted maxModelCalls to the global limit; the + // chat-specific override is the UI's job to pass explicitly. A failed + // settings read just falls back to the global limit. + const chatMaxModelCalls = await window.ipc + .invoke('turnLimits:getSettings', null) + .then((settings) => settings.chatMaxModelCalls) + .catch(() => undefined) const sendConfig = { agent: { agentId, @@ -3037,6 +3046,7 @@ function App() { }, autoPermission: (permissionMode ?? 'manual') === 'auto', ...(reasoningEffort ? { reasoningEffort } : {}), + ...(chatMaxModelCalls !== undefined ? { maxModelCalls: chatMaxModelCalls } : {}), } const userMessageContextFor = (middlePane: Awaited>) => ({ currentDateTime: new Date().toISOString(), @@ -4280,6 +4290,9 @@ function App() { setEmailInitialThreadId(threadId) setEmailThreadIdVersion((v) => v + 1) } + // Same reason as in navigateToView: a stale assistant-driven search must + // not repopulate the search box when the user re-enters the email view. + setEmailInitialSearchQuery(null) ensureEmailFileTab() }, [ensureEmailFileTab]) @@ -4454,6 +4467,10 @@ function App() { if (view.searchQuery) { setEmailInitialSearchQuery(view.searchQuery) setEmailSearchQueryVersion((v) => v + 1) + } else { + // Otherwise a past assistant-driven search would be re-applied on + // every re-entry, even after the user cleared the search box. + setEmailInitialSearchQuery(null) } ensureEmailFileTab() return @@ -4703,6 +4720,23 @@ function App() { return window.ipc.on('app:openUrl', ({ url }) => handle(url)) }, []) + // "Updated to vX.Y.Z" card on the first launch after an update. Main + // compares its persisted version stamp against the running version and + // hands out `updatedFrom` exactly once, so reloads don't re-show this. + useEffect(() => { + void window.ipc.invoke('app:consumeUpdateInfo', null).then(({ version, updatedFrom }) => { + if (!updatedFrom) return + toast(`Updated to v${version}`, { + description: `Rowboat was updated from v${updatedFrom}.`, + action: { + label: "What's new", + onClick: () => window.open(`https://github.com/rowboatlabs/rowboat/releases/tag/v${version}`, '_blank'), + }, + duration: 10000, + }) + }) + }, []) + // Report the UI theme to the apps server (spec §7.1): apps read it from // GET /_rowboat/app and get live changes via the SSE theme event. useEffect(() => { @@ -6266,6 +6300,7 @@ function App() { onOpenBgTasks={() => { setBgTaskInitialSlug(null); setBgTaskSlugVersion((v) => v + 1); openBgTasksView() }} onOpenAgent={(slug) => { setBgTaskInitialSlug(slug); setBgTaskSlugVersion((v) => v + 1); openBgTasksView() }} onOpenApps={openAppsView} + onOpenApp={(folder) => { setAppInitialId(folder); setAppIdVersion((v) => v + 1); openAppsView() }} recentRuns={runs} onOpenRun={(rid) => void navigateToView({ type: 'chat', runId: rid })} onRenameRun={(rid, title) => { @@ -7186,6 +7221,8 @@ function App() { /> + +