diff --git a/apps/x/CODE_MODE_ENGINES_PLAN.md b/apps/x/CODE_MODE_ENGINES_PLAN.md new file mode 100644 index 00000000..ce7a4958 --- /dev/null +++ b/apps/x/CODE_MODE_ENGINES_PLAN.md @@ -0,0 +1,271 @@ +# Code Mode — Managed Engine Provisioning Plan + +Branch: `feat/code-mode-managed-engines` (off `dev` @ `8ce24ebb`) + +## 1. Problem & Goal + +Code mode runs two coding agents — **Claude Code** and **Codex** — by spawning their +ACP adapters, which in turn spawn a heavy **native engine binary** (~205 MB claude, +~194 MB codex). We need code mode to work in **packaged releases with ~99% reliability +for both agents**, without shipping a ~400 MB installer. + +### Current state (HEAD = revert of #614) +- Packaged builds **do not stage** the ACP adapters, and `forge.config.cjs` + `ignore: /^\/node_modules\//` strips them. At runtime `agents.ts` resolves the + adapter via `require.resolve(...)` then spawns it — which **throws + `Cannot find module '@agentclientprotocol/...'`**. +- **Net: packaged code mode is broken in every release.** It only works in `dev` + because pnpm symlinks exist. There is no 400 MB bloat today — but no function either. + +### Why the two prior approaches are insufficient +- **Bundle engines (A):** +~400 MB per installer (one claude + one codex native binary + per OS/arch). Works offline, but installer is huge. +- **Drive from user's local install (B)** — what #614 settled on and was reverted: + requires the user to have **both** CLIs installed **and** logged in, correct version, + on the right PATH. Depends on the user's machine → **structurally cannot hit 99%** + (version skew, GUI-launch PATH stripping, missing installs). This is why #614 was + reverted. + +## 2. Chosen Architecture — Managed Engine Provisioning (validated against Conductor) + +Split the problem into **engine** vs **auth**, treat them differently — exactly what +Conductor (conductor.build) does: + +1. **Engine = owned by the app.** We provision **version-pinned** engine binaries into + **app-support** (`~/.rowboat/engines///`), download-on-demand on first + use, sha256-verified, symlink/path-pinned. Not the user's global npm, not their PATH. + → no version skew, no PATH quirks → this is what delivers the 99%. +2. **Auth = reused from the user.** The engines read existing credentials + (`~/.claude` API key / Pro / Max, `~/.codex` auth.json). No second login. `status.ts` + already inspects these. + +### Empirical proof from Conductor on this machine +- DMG is **123 MB** — far smaller than the ~400 MB of engines → engines are **not** in + the installer; they're downloaded after install. +- Layout observed at `~/Library/Application Support/com.conductor.app/`: + ``` + agent-binaries/claude/2.1.170/claude (222 MB, single Mach-O arm64) + agent-binaries/codex/0.138.0/codex (205 MB, single Mach-O arm64) + bin/claude -> agent-binaries/claude/2.1.170/claude (symlink to active version) + bin/codex -> agent-binaries/codex/0.138.0/codex + agent-binaries/.meta/claude-2.1.170.json = {sha256, size, downloaded_at_unix_ms, ...} + ``` +- So: versioned dirs + stable symlink + sha256 `.meta` ledger + download. We mirror this. + +## 3. Concrete facts that make this clean (verified) + +### Adapters honor an external engine via env var (no code change in adapters needed) +- **Claude** — `@agentclientprotocol/claude-agent-acp@0.39.0`, + `dist/acp-agent.js:39`: `if (process.env.CLAUDE_CODE_EXECUTABLE) return it;` + and line 1552 `pathToClaudeCodeExecutable: process.env.CLAUDE_CODE_EXECUTABLE ?? ...`. + If unset and no bundled native dep → it throws "set CLAUDE_CODE_EXECUTABLE". +- **Codex** — `@agentclientprotocol/codex-acp@0.0.44`, + `dist/index.js:20900`: `const codexPath = process.env["CODEX_PATH"] ?? "codex";` + → `spawn(codexPath, ["app-server"])`. +- **Implication:** provisioning + setting these two env vars is the *entire* engine story. + +### Our engine packages are already single self-contained binaries +- `@anthropic-ai/claude-agent-sdk-darwin-arm64@0.3.156` → contains one `claude` + executable (+ LICENSE/README). +- `@openai/codex@0.128.0-darwin-arm64` → `vendor//codex/codex` native binary + **plus a bundled `rg` (ripgrep) at `vendor//path/rg`** — see Risk R1. + +### Pinned versions + platform package names (read from the installed adapter trees) +- **Claude:** adapter `claude-agent-acp@0.39.0` → engine `@anthropic-ai/claude-agent-sdk@0.3.156`. + Platform optional deps `@anthropic-ai/claude-agent-sdk-@0.3.156`: + `darwin-arm64`, `darwin-x64`, `linux-x64`, `linux-arm64`, `linux-x64-musl`, + `linux-arm64-musl`, `win32-x64`, `win32-arm64`. +- **Codex:** adapter `codex-acp@0.0.44` → `@openai/codex@^0.128.0` (pnpm-**patched**, see R2). + Platform deps aliased `@openai/codex-` → `npm:@openai/codex@0.128.0-`: + `darwin-arm64`, `darwin-x64`, `linux-x64`, `linux-arm64`, `win32-x64`, `win32-arm64`. + +## 4. Distribution source — npm platform packages, adapter-pinned (DECIDED) + +We fetch the **per-platform engine packages from the npm registry at the exact versions +our ACP adapters depend on**, extract the native binary, and provision it into +`~/.rowboat/engines/...`. No self-host, no curl installer, no fallback. + +- **Tarball URL:** `https://registry.npmjs.org//-/-.tgz` + - claude: `@anthropic-ai/claude-agent-sdk-@0.3.156` → extract the `claude` binary. + - codex: `@openai/codex@0.128.0-` → extract `vendor//codex/codex` + **and keep `vendor//path/rg`** (see R1). +- **Why adapter-pinned npm (not curl installer, not release bucket, not self-host):** + - The binary is the **exact version our adapter was built/tested against** → ACP + handshake guaranteed → the key to ~99% reliability. + - npm registry is highly available, immutable per-version, and the packument provides + `dist.integrity` (sha512) + `dist.shasum` (sha1) → integrity verification with **zero + infra**. + - The official `curl | bash` installer is for end-users: it installs globally to + `~/.local/bin` and **auto-updates in the background** — exactly what we must avoid. We + want an isolated, pinned, app-managed copy that never moves under the adapter. + (Conductor likewise fetches the raw binary rather than running the installer.) +- **Versions are read from our lockfile at build time** and embedded in + `engine-manifest.json`, so the manifest can never drift from the adapters we ship. + +### Reference: how this relates to Conductor / official installers +- Conductor self-hosts the same kind of native binaries on `storage.conductor.build` + (raw/`.gz`/`.zst` + `{url,gzipUrl,zstdUrl,sha256}` manifest), pairing the adapters with + recent **standalone CLI** versions (claude 2.1.x, codex 0.138). That proves recent + versions work — but we deliberately pin to the **adapter's own** engine version for + determinism. +- Official Claude releases also publish a GPG-signed `manifest.json` (SHA256/platform) at + `downloads.claude.ai/claude-code-releases//`. We don't use it now (npm is simpler + and adapter-matched), but it's the upgrade path if we ever want the latest CLI line. +- If we later want control/availability/compression, we can mirror these npm tarballs to + our own bucket — **runtime stays identical, only manifest URLs change.** + +### Locked decisions +- **Source: npm platform packages, adapter-pinned versions (user).** +- **No self-host (user).** +- **Always provision; no fallback** to a user's pre-installed `claude`/`codex` (user). + Reverted path B is fully retired. +- **Codex pnpm patch — IRRELEVANT (user):** it targets the JS launcher; we point + `CODEX_PATH` at the native binary, so it does not apply. (Risk R2 removed.) + +## 5. Design + +### 5.1 Build-time: generate an engine manifest + stage adapter JS + +**(a) Engine manifest (`engine-manifest.json`, embedded in the app).** +A build script reads the installed adapter dependency trees and emits, per agent: +```jsonc +{ + "claude": { + "version": "0.3.156", + "platforms": { + "darwin-arm64": { "pkg": "@anthropic-ai/claude-agent-sdk-darwin-arm64", + "tarball": "https://registry.npmjs.org/.../-/...-0.3.156.tgz", + "integrity": "sha512-...", + "binRelPath": "claude" }, + "...": {} + } + }, + "codex": { + "version": "0.128.0", + "platforms": { + "darwin-arm64": { "pkg": "@openai/codex-darwin-arm64", + "tarball": "https://registry.npmjs.org/...", + "integrity": "sha512-...", + "binRelPath": "vendor/aarch64-apple-darwin/codex/codex", + "extraPaths": ["vendor/aarch64-apple-darwin/path/rg"] } + } + } +} +``` +- Versions + tarball URLs + integrity are pulled from the lockfile / npm packument so the + manifest is always in sync with the adapters we ship. (Generating it at build time + sidesteps hardcoding fragile package names.) +- `binRelPath` / `extraPaths` capture where the executable (and codex's `rg`) live inside + each tarball. + +**(b) Stage the ACP adapter JS into the package (the part of #614 we KEEP).** +The adapters themselves (tiny, ~15 MB total incl. their non-native JS deps) must exist on +disk in packaged builds so `agents.ts` can resolve + spawn them. In `forge.config.cjs` +`generateAssets`, stage the two adapters and their **non-native** production dependency +closure into `.package/acp/node_modules` (npm-style nested layout), and **exempt +`.package` from the `node_modules` ignore rule**. We **drop** #614's native-engine staging +entirely — engines come from provisioning, not the bundle. + +### 5.2 Runtime: engine provisioner (new module in `packages/core`) + +New file: `packages/core/src/code-mode/acp/engine-provisioner.ts`. + +``` +ensureEngine(agent): Promise<{ executablePath: string }> + 1. Read manifest entry for (agent, currentPlatform). Error clearly if unsupported. + 2. dir = ~/.rowboat/engines/// + 3. If dir exists AND .meta/-.json sha256 matches → return binPath. + 4. Else acquire a cross-process lock (avoid double download), then: + a. Download tarball to a temp file, streaming, with progress events. + b. Verify integrity (sha512/sha256 from manifest). + c. Extract into a temp dir, then atomic rename into / (tar gzip). + d. chmod +x the binary (and codex's rg) on unix. + e. Write .meta/-.json {sha256, size, downloaded_at_unix_ms}. + 5. Return absolute path to the engine executable (binRelPath joined to dir). +``` +- **Progress + cancellation** surfaced over IPC for the first-run "Downloading engine…" UI. +- **Offline / failure** → typed error with a clear, actionable message (not a hang). +- **Resumability / atomicity:** download to temp, verify, then rename; never leave a + half-extracted version dir that passes the existence check. + +### 5.3 Wire provisioning into launch + +In `agents.ts` `getAgentLaunchSpec()` (currently sets only `CLAUDE_CODE_EXECUTABLE`): +- Make it (or its caller in `client.ts` / `manager.ts`) `await ensureEngine(agent)` first, + then set: + - claude → `env.CLAUDE_CODE_EXECUTABLE = ` + - codex → `env.CODEX_PATH = ` (and ensure its `rg` sibling resolves; + keep the vendor dir layout intact so codex finds `../path/rg`). +- Keep `ELECTRON_RUN_AS_NODE=1` for the adapter spawn (unchanged). +- The provisioner replaces both the bundled-engine dependency *and* the reverted + "resolve user's local install" path. (Optionally: if a healthy provisioned engine is + absent but a compatible local install exists, we *may* fall back to it — but the default + and reliable path is the provisioned engine. Decide in review; default = provisioned only.) + +### 5.4 Status & UX (`status.ts` + renderer) +- `checkCodeModeAgentStatus()` becomes: engine = provisioned? (instead of "installed on + PATH?"); auth = existing credentials present? (unchanged logic). +- First-run flow: user picks agent → if engine missing, show "Downloading engine + (~200 MB), one time…" with progress; on completion, proceed. Subsequent uses: instant. +- Clear error states: download failed / offline / unsupported platform / auth missing. + +## 6. File-by-file change plan + +| File | Change | +|---|---| +| `apps/main/forge.config.cjs` | Stage adapter JS closure into `.package/acp/node_modules`; exempt `.package` from node_modules ignore; generate + copy `engine-manifest.json`. (No native engines staged.) | +| `apps/main/scripts/gen-engine-manifest.mjs` *(new)* | Build-time: read lockfile/packuments → emit `engine-manifest.json` (versions, tarball URLs, integrity, bin paths). | +| `packages/core/src/code-mode/acp/engine-provisioner.ts` *(new)* | `ensureEngine()` — download, verify, extract, lock, progress, `.meta` ledger. | +| `packages/core/src/code-mode/acp/agents.ts` | Resolve adapter from staged `.package/acp` first (fallback to node_modules in dev); `await ensureEngine`; set `CLAUDE_CODE_EXECUTABLE`/`CODEX_PATH` to provisioned binaries. | +| `packages/core/src/code-mode/acp/client.ts` / `manager.ts` | Await provisioning before spawn; surface provisioning progress/errors; keep startup deadline. | +| `packages/core/src/code-mode/status.ts` | Engine status = provisioned (not PATH); keep auth checks. | +| `apps/main/src/ipc.ts` + preload + renderer | IPC for provisioning progress + first-run download UI + error states. | +| CI (optional) | Smoke: packaged app boots each adapter, provisions a (small fake) engine, sets env var, answers ACP `initialize`; offline → clear error not a hang. | + +## 7. Edge cases & risks + +- **R1 — Codex needs `rg`.** The codex platform package bundles ripgrep at + `vendor//path/rg`. We must extract/keep the vendor layout so codex finds it + (don't extract the bare binary). Verify codex `app-server` boots from the provisioned dir. +- **R2 — Codex pnpm patch. RESOLVED (irrelevant).** The patch targets the JS launcher; we + point `CODEX_PATH` at the native binary, so it does not apply. +- **R3 — Platform/arch matrix.** Manifest must cover darwin x64/arm64, linux x64/arm64 + (+musl for claude), win32 x64/arm64. Windows engine is `claude.exe`/`codex.exe`. +- **R4 — Integrity & supply chain.** Always verify the manifest integrity hash before + chmod/exec. Treat a hash mismatch as a hard failure. +- **R5 — Disk + upgrades.** Versioned dirs accumulate. Add a cleanup that keeps only the + active pinned version per agent. +- **R6 — First-run network.** Required once per agent; cached forever after. Must be a + clear, cancellable UX, never a silent hang (reuse the #614 startup-deadline lesson). +- **R7 — code signing / Gatekeeper (macOS).** Downloaded native binaries aren't covered by + our app's signature. Verify they run under Gatekeeper (they're already + signed/notarized by Anthropic/OpenAI; quarantine attr may need clearing). Conductor runs + them fine from app-support — confirm we do too. +- **R8 — `engine-manifest` staleness.** Manifest must regenerate whenever the adapter/engine + versions change; tie generation into the build so it can't drift. + +## 8. Phasing + +1. **P1 — Packaging fix (unblocks dev→packaged parity):** stage adapter JS into `.package`, + resolver checks staged path first. Verify packaged code mode can at least *spawn* the + adapter. (Independent of provisioning; the part of #614 worth keeping.) +2. **P2 — Provisioner (core):** `ensureEngine()` + manifest + wire env vars. Verify both + engines provision + boot from `~/.rowboat/engines` on macOS arm64. +3. **P3 — UX + status:** first-run download UI, status panel, error states. +4. **P4 — Cross-platform + CI smoke:** matrix manifest, mac/linux/win smoke. +5. **P5 — Polish:** version cleanup, cancellation, offline messaging, optional local-install + fallback. + +## 9. Decisions & remaining questions + +### Resolved (locked) +1. **Source = npm platform packages, adapter-pinned versions** (not self-host, not curl + installer, not release bucket). +2. **Always provision; no local-install fallback.** +3. **Codex pnpm patch irrelevant.** + +### Still open (can decide during implementation) +- **Provision timing:** on first code-mode *use* (lazy) vs first app launch (eager, + background download). Plan assumes **lazy + cached**. +- **Version-dir cleanup:** keep only the active pinned version per agent (R5). +- **Verify R1** (codex `rg`) and **R7** (Gatekeeper on downloaded macOS binaries) during P2. diff --git a/apps/x/apps/main/forge.config.cjs b/apps/x/apps/main/forge.config.cjs index 4cca4194..5e33bb49 100644 --- a/apps/x/apps/main/forge.config.cjs +++ b/apps/x/apps/main/forge.config.cjs @@ -5,6 +5,92 @@ const path = require('path'); const pkg = require('./package.json'); +// Stage the ACP coding-adapters (@agentclientprotocol/*-acp) and their full +// production dependency closure into the packaged app. +// +// Why this is needed: code mode spawns each adapter as a SEPARATE `node ` +// process and locates it at runtime via require.resolve — so it must ship as a real +// on-disk file. esbuild can't inline it (dynamic resolve + spawn target), and Forge +// strips the workspace node_modules (see `ignore` below). Without this, packaged +// builds throw `Cannot find module '@agentclientprotocol/...'`. +// +// Why we reconstruct a nested tree instead of copying node_modules: pnpm's store is a +// symlink farm that legitimately holds multiple versions of the same package (e.g. +// @agentclientprotocol/sdk 0.21 for claude vs 0.22 for codex). We rebuild an npm-style +// nested node_modules — dereferencing symlinks and nesting on version conflict — which +// resolves correctly regardless of pnpm layout. +// +// What we DON'T bundle: the agents' native engines (claude / codex, ~200 MB each, shipped +// as platform-specific packages). Those are PROVISIONED on demand into +// ~/.rowboat/engines/// and the adapters are pointed at them via +// CLAUDE_CODE_EXECUTABLE / CODEX_PATH (see packages/core/src/code-mode/acp/). Skipping +// them keeps each OS installer ~400 MB smaller while code mode stays fully functional. +function stageAcpAdapters(mainDir, destNodeModules) { + const fs = require('fs'); + const ADAPTERS = [ + '@agentclientprotocol/claude-agent-acp', + '@agentclientprotocol/codex-acp', + ]; + + // The native engines, shipped as platform packages. Provisioned on demand instead + // (see comment above), so they're excluded from staging. + const isNativeEngine = (key) => + /^@anthropic-ai\/claude-agent-sdk-(win32|darwin|linux)/.test(key) || // native claude + /^@openai\/codex-(win32|darwin|linux)/.test(key); // native codex + + // Resolve a dependency's real directory by walking node_modules the way Node does, + // looking for the package DIRECTORY. We deliberately do NOT use + // require.resolve(`${key}/package.json`): that throws for packages whose `exports` + // map doesn't expose package.json (e.g. @anthropic-ai/claude-agent-sdk), which would + // silently drop them and their subtrees. realpathSync dereferences pnpm's symlinks. + // Returns null for deps not installed for this OS (platform-optional binaries). + const realDirOf = (key, fromDir) => { + let dir = fromDir; + for (;;) { + const cand = path.join(dir, 'node_modules', ...key.split('/')); + if (fs.existsSync(path.join(cand, 'package.json'))) return fs.realpathSync(cand); + const parent = path.dirname(dir); + if (parent === dir) return null; + dir = parent; + } + }; + + let copied = 0; + const skippedEngines = new Set(); + const install = (srcDir, key, destNM, chain) => { + const destDir = path.join(destNM, ...key.split('/')); + if (fs.existsSync(destDir)) return; // already placed at this exact location + if (chain.has(srcDir)) return; // dependency cycle — resolves to ancestor copy + fs.mkdirSync(path.dirname(destDir), { recursive: true }); + fs.cpSync(srcDir, destDir, { + recursive: true, + dereference: true, + filter: (s) => path.basename(s) !== 'node_modules', // deps handled by recursion + }); + copied++; + const pj = JSON.parse(fs.readFileSync(path.join(srcDir, 'package.json'), 'utf8')); + const deps = { ...pj.dependencies, ...pj.optionalDependencies }; + const nextChain = new Set(chain).add(srcDir); + for (const depKey of Object.keys(deps)) { + if (isNativeEngine(depKey)) { skippedEngines.add(depKey); continue; } + const depDir = realDirOf(depKey, srcDir); + if (depDir) install(depDir, depKey, path.join(destDir, 'node_modules'), nextChain); + } + }; + + for (const key of ADAPTERS) { + const srcDir = realDirOf(key, mainDir); + if (!srcDir) { + throw new Error(`ACP adapter '${key}' is not installed in ${mainDir} — run pnpm install`); + } + install(srcDir, key, destNodeModules, new Set()); + } + if (skippedEngines.size) { + console.log(` (skipped native engines — provisioned on demand: ${[...skippedEngines].join(', ')})`); + } + return copied; +} + module.exports = { packagerConfig: { executableName: 'rowboat', @@ -29,19 +115,23 @@ module.exports = { appleIdPassword: process.env.APPLE_PASSWORD, teamId: process.env.APPLE_TEAM_ID }, - // Since we bundle everything with esbuild, we don't need node_modules at all. - // These settings prevent Forge's dependency walker (flora-colossus) from trying - // to analyze/copy node_modules, which fails with pnpm's symlinked workspaces. - // Regexes are ANCHORED to the app root: .package/node_modules (where - // bundle.mjs stages the native node-pty module) must survive packaging. + // Since we bundle the main process with esbuild, we don't need the workspace + // node_modules. These settings prevent Forge's dependency walker (flora-colossus) + // from trying to analyze/copy node_modules, which fails with pnpm's symlinked + // workspaces. prune: false, - ignore: [ - /^\/src\//, - /^\/node_modules\//, - /.gitignore/, - /bundle\.mjs/, - /tsconfig.json/, - ], + // Strip the workspace src/node_modules (paths are ANCHORED to the app root), BUT + // always keep everything under `.package/` — that's our staged output: the + // bundled main process, the ACP adapters + their dependency closure (staged by + // the generateAssets hook), and the native node-pty module (staged into + // .package/node_modules by bundle.mjs). Without the `.package` exemption the + // node_modules rule would strip those and code mode / the embedded terminal + // would break in packaged builds. + ignore: (p) => { + if (p === '/.package' || p.startsWith('/.package/')) return false; + return [/^\/src\//, /^\/node_modules\//, /\.gitignore/, /bundle\.mjs/, /tsconfig\.json/] + .some((re) => re.test(p)); + }, }, makers: [ { @@ -195,6 +285,15 @@ module.exports = { fs.mkdirSync(rendererDest, { recursive: true }); fs.cpSync(rendererSrc, rendererDest, { recursive: true }); + // Stage the ACP coding-adapters (+ their JS dependency closure, minus native + // engines) into .package/acp/node_modules. They are spawned as separate node + // processes at runtime and Forge strips the workspace node_modules, so they + // must be copied in explicitly. See stageAcpAdapters() above for the why. + console.log('Staging ACP adapters...'); + const acpDest = path.join(packageDir, 'acp', 'node_modules'); + const staged = stageAcpAdapters(__dirname, acpDest); + console.log(`✅ Staged ${staged} ACP adapter packages into .package/acp/node_modules`); + console.log('✅ All assets staged in .package/'); }, } diff --git a/apps/x/apps/main/src/ipc.ts b/apps/x/apps/main/src/ipc.ts index e59b1994..71e9a66e 100644 --- a/apps/x/apps/main/src/ipc.ts +++ b/apps/x/apps/main/src/ipc.ts @@ -34,6 +34,7 @@ import { IGranolaConfigRepo } from '@x/core/dist/knowledge/granola/repo.js'; import { ICodeModeConfigRepo } from '@x/core/dist/code-mode/repo.js'; import { CodePermissionRegistry } from '@x/core/dist/code-mode/acp/permission-registry.js'; import { checkCodeModeAgentStatus } from '@x/core/dist/code-mode/status.js'; +import { ensureEngine } from '@x/core/dist/code-mode/acp/engine-provisioner.js'; import type { ICodeProjectsRepo } from '@x/core/dist/code-mode/projects/repo.js'; import type { ICodeSessionsRepo } from '@x/core/dist/code-mode/sessions/repo.js'; import { CodeSessionService } from '@x/core/dist/code-mode/sessions/service.js'; @@ -714,6 +715,26 @@ export function setupIpcHandlers() { 'codeMode:checkAgentStatus': async () => { return await checkCodeModeAgentStatus(); }, + 'codeMode:provisionEngine': async (_event, args) => { + // Download + install the agent's engine, streaming progress back to the + // requesting window so Settings can show a live bar. 'check' is instant — skip it. + try { + await ensureEngine(args.agent, { + onProgress: (p) => { + if (p.phase === 'check') return; + _event.sender.send('codeMode:engineProgress', { + agent: args.agent, + phase: p.phase, + receivedBytes: p.receivedBytes, + totalBytes: p.totalBytes, + }); + }, + }); + return { success: true }; + } catch (e) { + return { success: false, error: e instanceof Error ? e.message : String(e) }; + } + }, 'codeProject:add': async (_event, args) => { const repo = container.resolve('codeProjectsRepo'); const project = await repo.add(args.path); diff --git a/apps/x/apps/renderer/src/components/code/new-session-dialog.tsx b/apps/x/apps/renderer/src/components/code/new-session-dialog.tsx index 80895be2..a47cc8d1 100644 --- a/apps/x/apps/renderer/src/components/code/new-session-dialog.tsx +++ b/apps/x/apps/renderer/src/components/code/new-session-dialog.tsx @@ -178,7 +178,7 @@ export function NewSessionDialog({ >
{a === 'claude' ? 'Claude Code' : 'Codex'}
- {ready ? 'Ready' : agentStatus?.[a]?.installed ? 'Not signed in' : 'Not installed'} + {ready ? 'Ready' : agentStatus?.[a]?.installed ? 'Not signed in' : 'Enable in Settings'}
) diff --git a/apps/x/apps/renderer/src/components/settings-dialog.tsx b/apps/x/apps/renderer/src/components/settings-dialog.tsx index 691b4da2..1319abb9 100644 --- a/apps/x/apps/renderer/src/components/settings-dialog.tsx +++ b/apps/x/apps/renderer/src/components/settings-dialog.tsx @@ -1759,52 +1759,122 @@ function NoteTaggingSettings({ dialogOpen }: { dialogOpen: boolean }) { type AgentStatus = { installed: boolean; signedIn: boolean } type CodeModeAgentStatus = { claude: AgentStatus; codex: AgentStatus } +// Engine provisioning runs in the main process and keeps going even if the Settings +// dialog is closed. Track its state at MODULE level (not in the row component, which +// unmounts on close) so reopening Settings still shows the live % instead of the Enable +// button. A single persistent listener on the progress channel feeds this store. +type ProvState = { pct: number | null; error?: string } +const provStore: Record = {} +// Agents we provisioned this session — used to show "Ready" immediately on success +// without waiting for the async status refresh to round-trip (which caused the row to +// briefly flash the Enable button again). +const enabledOptimistic = new Set() +const provListeners = new Set<() => void>() +let provChannelHooked = false + +function notifyProv() { provListeners.forEach((l) => l()) } + +function startProvisioning(agent: 'claude' | 'codex', onDone: () => void | Promise): void { + if (provStore[agent] && !provStore[agent]!.error) return // already in flight + provStore[agent] = { pct: null } + notifyProv() + if (!provChannelHooked) { + provChannelHooked = true + window.ipc.on('codeMode:engineProgress', (p) => { + const cur = provStore[p.agent] + if (!cur) return + const pct = p.totalBytes ? Math.floor(((p.receivedBytes ?? 0) / p.totalBytes) * 100) : cur.pct + provStore[p.agent] = { pct } + notifyProv() + }) + } + window.ipc.invoke('codeMode:provisionEngine', { agent }) + .then((res) => { + if (res.success) { + // Mark installed optimistically so the row shows "Ready" the instant the flag + // clears — don't depend on the async status refresh (which re-renders the parent + // separately and left a window showing the Enable button). loadStatus still runs + // in the background to sync the real status. + enabledOptimistic.add(agent) + provStore[agent] = undefined + void onDone() + } else { + provStore[agent] = { pct: null, error: res.error ?? 'Failed to enable' } + } + }) + .catch((e) => { provStore[agent] = { pct: null, error: e instanceof Error ? e.message : 'Failed to enable' } }) + .finally(notifyProv) +} + +function useProvisioning(agent: string): ProvState | undefined { + const [, force] = useState(0) + useEffect(() => { + const l = () => force((n) => n + 1) + provListeners.add(l) + return () => { provListeners.delete(l) } + }, []) + return provStore[agent] +} + function AgentStatusRow({ name, - installLink, + agent, signInCommand, status, + onProvisioned, }: { name: string - installLink: string + agent: 'claude' | 'codex' signInCommand: string status: AgentStatus | null + onProvisioned: () => void }) { - const ready = status?.installed && status?.signedIn - const needsSignInOnly = status?.installed && !status?.signedIn + const prov = useProvisioning(agent) + const provisioning = prov !== undefined && prov.error === undefined + const error = prov?.error ?? null + const enable = useCallback(() => startProvisioning(agent, onProvisioned), [agent, onProvisioned]) + + // Treat a just-enabled engine as installed even before the status refresh lands. + const installed = (status?.installed ?? false) || enabledOptimistic.has(agent) + const ready = installed && status?.signedIn return (
{name}
- - {status?.installed ? : } - Installed + + {installed ? : } + {installed ? 'Engine ready' : 'Not enabled'} {status?.signedIn ? : } Signed in
+ {error &&
{error}
}
- {ready ? ( + {provisioning ? ( + + + {prov?.pct != null ? `${prov.pct}%` : null} + + ) : ready ? ( Ready - ) : needsSignInOnly ? ( + ) : !installed ? ( + + ) : ( Run {signInCommand} - ) : ( - - Install & sign in - )}
) @@ -1907,6 +1977,14 @@ function CodeModeSettings({ dialogOpen }: { dialogOpen: boolean }) { Requires an active Claude Code subscription or a ChatGPT/Codex subscription. You can have one or both.

+

+ For each agent you want to use, you must have it{' '} + installed and logged in on this machine: click{' '} + Enable below to download its engine, and sign in by + running claude login{' '} + or codex login{' '} + in your terminal. Code mode uses that saved login. +

@@ -1924,15 +2002,17 @@ function CodeModeSettings({ dialogOpen }: { dialogOpen: boolean }) {
@@ -1984,8 +2064,8 @@ function CodeModeSettings({ dialogOpen }: { dialogOpen: boolean }) {
- Neither Claude Code nor Codex is ready. Install at least one and sign in with a subscription - account, then click Re-check. + Neither Claude Code nor Codex is ready. Click Enable above to download an engine, sign in with a + subscription account, then click Re-check.
)} diff --git a/apps/x/packages/core/scripts/gen-engine-manifest.mjs b/apps/x/packages/core/scripts/gen-engine-manifest.mjs new file mode 100644 index 00000000..8a363b69 --- /dev/null +++ b/apps/x/packages/core/scripts/gen-engine-manifest.mjs @@ -0,0 +1,111 @@ +// Build-time generator for the code-mode engine manifest. +// +// Code mode provisions each agent's native engine on demand by downloading the +// per-platform npm package AT THE EXACT VERSION OUR ACP ADAPTER DEPENDS ON, so the +// adapter <-> engine handshake is always in lockstep. This script reads those pinned +// versions from the installed adapter package.json files, queries the npm registry for +// each platform package's tarball URL + integrity, and writes them to +// `src/code-mode/acp/engine-manifest.ts` (committed; regenerate on a version bump). +// +// Usage: node scripts/gen-engine-manifest.mjs (run from packages/core) +// +// Why a committed .ts (not a fetched-at-build .json): it needs no network at app build +// time, works offline in dev, is reviewable in PRs, and esbuild inlines it into the +// packaged main bundle. + +import { createRequire } from 'module'; +import { writeFileSync } from 'fs'; +import { fileURLToPath } from 'url'; +import * as path from 'path'; + +const require = createRequire(import.meta.url); +const REGISTRY = 'https://registry.npmjs.org'; + +// Platform keys we publish for each agent. These mirror the optionalDependencies of the +// engine packages. claude ships musl variants; codex does not. +const CLAUDE_PLATFORMS = [ + 'darwin-arm64', 'darwin-x64', + 'linux-x64', 'linux-arm64', 'linux-x64-musl', 'linux-arm64-musl', + 'win32-x64', 'win32-arm64', +]; +const CODEX_PLATFORMS = [ + 'darwin-arm64', 'darwin-x64', + 'linux-x64', 'linux-arm64', + 'win32-x64', 'win32-arm64', +]; + +// Read a pinned dependency version from an adapter's package.json, stripping any range +// prefix (^, ~). createRequire resolves the adapter from the installed node_modules. +function pinnedDep(adapterPkg, depName) { + const pj = require(`${adapterPkg}/package.json`); + const spec = (pj.dependencies || {})[depName] || (pj.optionalDependencies || {})[depName]; + if (!spec) throw new Error(`${adapterPkg} has no dependency on ${depName}`); + return spec.replace(/^[\^~]/, ''); +} + +// Fetch a single version's manifest from the registry and return its dist coordinates. +async function distFor(pkg, version) { + const url = `${REGISTRY}/${pkg}/${version}`; + const res = await fetch(url); + if (!res.ok) { + if (res.status === 404) return null; // platform not published for this version + throw new Error(`registry ${res.status} for ${pkg}@${version}`); + } + const doc = await res.json(); + return { tarball: doc.dist.tarball, integrity: doc.dist.integrity }; +} + +// Build one agent's manifest entry: { version, platforms: { : {tarball, integrity} } }. +// pkgFor(platform) -> { pkg, version } gives the registry coordinates for each platform. +async function buildAgent(version, platforms, pkgFor) { + const out = { version, platforms: {} }; + for (const key of platforms) { + const { pkg, version: pv } = pkgFor(key); + const dist = await distFor(pkg, pv); + if (!dist) { + console.warn(` ! ${pkg}@${pv} not found on registry — skipping ${key}`); + continue; + } + out.platforms[key] = { pkg, pkgVersion: pv, ...dist }; + console.log(` ✓ ${key}: ${pkg}@${pv}`); + } + return out; +} + +async function main() { + // Pinned engine versions, read straight from the adapters we ship. + const claudeVersion = pinnedDep('@agentclientprotocol/claude-agent-acp', '@anthropic-ai/claude-agent-sdk'); + const codexVersion = pinnedDep('@agentclientprotocol/codex-acp', '@openai/codex'); + console.log(`claude engine: @anthropic-ai/claude-agent-sdk@${claudeVersion}`); + console.log(`codex engine: @openai/codex@${codexVersion}`); + + const manifest = { + claude: await buildAgent(claudeVersion, CLAUDE_PLATFORMS, (key) => ({ + pkg: `@anthropic-ai/claude-agent-sdk-${key}`, + version: claudeVersion, + })), + codex: await buildAgent(codexVersion, CODEX_PLATFORMS, (key) => ({ + // codex publishes platform binaries as VERSIONS of @openai/codex + // (e.g. 0.128.0-darwin-arm64), not as separate package names. + pkg: '@openai/codex', + version: `${codexVersion}-${key}`, + })), + }; + + const outPath = path.join( + path.dirname(fileURLToPath(import.meta.url)), + '..', 'src', 'code-mode', 'acp', 'engine-manifest.ts', + ); + const banner = + '// AUTO-GENERATED by packages/core/scripts/gen-engine-manifest.mjs — do not edit by hand.\n' + + '// Regenerate after bumping the @agentclientprotocol/*-acp adapter (engine) versions.\n' + + '// Maps each agent + platform to the npm tarball + integrity of its native engine.\n\n'; + const body = `export const ENGINE_MANIFEST = ${JSON.stringify(manifest, null, 4)} as const;\n`; + writeFileSync(outPath, banner + body); + console.log(`\nWrote ${outPath}`); +} + +main().catch((e) => { + console.error(e); + process.exit(1); +}); diff --git a/apps/x/packages/core/src/code-mode/acp/agents.ts b/apps/x/packages/core/src/code-mode/acp/agents.ts index da06d8ea..9085ddc4 100644 --- a/apps/x/packages/core/src/code-mode/acp/agents.ts +++ b/apps/x/packages/core/src/code-mode/acp/agents.ts @@ -1,7 +1,9 @@ import { createRequire } from 'module'; import * as path from 'path'; +import { fileURLToPath } from 'url'; import type { CodingAgent } from './types.js'; -import { resolveClaudeExecutable } from './claude-exec.js'; +import { getProvisionedEnginePath } from './engine-provisioner.js'; +import { loginShellPath } from './shell-env.js'; const require = createRequire(import.meta.url); @@ -20,13 +22,36 @@ export interface AgentLaunchSpec { env: NodeJS.ProcessEnv; } +// Locate an adapter's package.json. In packaged builds Electron Forge strips the +// workspace node_modules, so the adapters (+ their dependency closure) are staged +// next to the bundle at `.package/acp/node_modules` by the generateAssets hook (see +// apps/main/forge.config.cjs). In dev they resolve normally via the pnpm symlink. +// Try the staged location first, then fall back to ordinary resolution. +function resolveAdapterPkgJson(pkg: string): string { + // The main process is esbuild-bundled to `.package/dist/main.cjs`, so the staged + // adapters live one level up at `.package/acp`. (import.meta.url is rewritten to + // the bundle path by bundle.mjs, so this holds in both dev and packaged builds.) + const stagedRoot = path.join(path.dirname(fileURLToPath(import.meta.url)), '..', 'acp'); + for (const opts of [{ paths: [stagedRoot] }, undefined]) { + try { + return require.resolve(`${pkg}/package.json`, opts); + } catch { + // not here — try the next resolution strategy + } + } + throw new Error( + `ACP adapter '${pkg}' not found — expected it staged at ` + + `${path.join(stagedRoot, 'node_modules', pkg)} (packaged build) or resolvable ` + + `from node_modules (dev).`, + ); +} + // Resolve the adapter's executable ENTRY (its `bin`, not its library `main`) to an -// absolute path so we can spawn it directly with `node `. createRequire lets -// us resolve workspace/pnpm-installed packages from this module's location. +// absolute path so we can spawn it directly with `node `. function resolveAdapterEntry(pkg: string): string { - const pkgJsonPath = require.resolve(`${pkg}/package.json`); + const pkgJsonPath = resolveAdapterPkgJson(pkg); const pkgDir = path.dirname(pkgJsonPath); - const pkgJson = require(`${pkg}/package.json`) as { bin?: string | Record }; + const pkgJson = require(pkgJsonPath) as { bin?: string | Record }; const bin = pkgJson.bin; const rel = typeof bin === 'string' ? bin : bin ? Object.values(bin)[0] : undefined; if (!rel) { @@ -39,14 +64,31 @@ export function getAgentLaunchSpec(agent: CodingAgent): AgentLaunchSpec { const entry = resolveAdapterEntry(ADAPTER_PACKAGE[agent]); const env: NodeJS.ProcessEnv = { ...process.env }; - // Point the Claude adapter at the real claude executable. On Windows this is - // mandatory (Node can't spawn the .cmd shim — EINVAL); on macOS/Linux it's a - // PATH safety net for GUI launches. Resolver is a no-op when claude isn't found, - // leaving the adapter to do its own lookup. (Codex relies on PATH for now — wire - // an equivalent when we add Codex support.) - if (agent === 'claude' && !env.CLAUDE_CODE_EXECUTABLE) { - const exe = resolveClaudeExecutable(); - if (exe) env.CLAUDE_CODE_EXECUTABLE = exe; + // Graft the user's login-shell PATH onto the engine's env. GUI (Finder) launches + // inherit launchd's stripped PATH, so tools the engine spawns — git, gh, rg, bash — + // would otherwise fail with "command not found" even though they work from a + // terminal. No-op on Windows / when the probe fails. + const shellPath = loginShellPath(); + if (shellPath && shellPath !== env.PATH) { + const dirs = [...shellPath.split(path.delimiter), ...(env.PATH ?? '').split(path.delimiter)]; + env.PATH = [...new Set(dirs.filter(Boolean))].join(path.delimiter); + } + + // Point the adapter at the engine the user already enabled in Settings. We do NOT + // download here — getProvisionedEnginePath throws a clear "enable it in Settings" + // error if the engine isn't present, so code mode never triggers a surprise + // mid-chat download. The version is locked to what the adapter was built against, so + // the ACP handshake is always compatible. The adapters honor these env vars + // (claude: CLAUDE_CODE_EXECUTABLE, codex: CODEX_PATH). + const executablePath = getProvisionedEnginePath(agent); + if (agent === 'claude') { + env.CLAUDE_CODE_EXECUTABLE = executablePath; + // Make the claude-agent-sdk log the exact spawn command + claude's stderr to + // ~/.claude/debug/sdk-*.txt, so a failed/hung launch has a diagnosable trail + // instead of a silently dropped connection. + env.DEBUG_CLAUDE_AGENT_SDK = '1'; + } else { + env.CODEX_PATH = executablePath; } // We spawn the adapter with process.execPath. Inside Electron's main process diff --git a/apps/x/packages/core/src/code-mode/acp/client.ts b/apps/x/packages/core/src/code-mode/acp/client.ts index 5c2bd1ba..477c994b 100644 --- a/apps/x/packages/core/src/code-mode/acp/client.ts +++ b/apps/x/packages/core/src/code-mode/acp/client.ts @@ -27,6 +27,16 @@ export interface AcpClientOptions { onEvent: (event: CodeRunEvent) => void; } +// Deadline for the startup phases (initialize / session create+load). A healthy cold +// start — adapter boot, engine spawn, SDK handshake, MCP connects — takes seconds; only +// a wedged engine takes this long. Without a deadline that failure mode is an infinite +// "(pending...)" with zero feedback. Prompts are intentionally NOT time-limited: turns +// legitimately run for many minutes and may wait on user permission asks. Overridable +// via ROWBOAT_ACP_STARTUP_TIMEOUT_MS (CI smoke test; escape hatch for MCP-heavy setups). +const STARTUP_TIMEOUT_MS = Number(process.env.ROWBOAT_ACP_STARTUP_TIMEOUT_MS) > 0 + ? Number(process.env.ROWBOAT_ACP_STARTUP_TIMEOUT_MS) + : 60_000; + // Map a raw ACP session/update notification onto our small CodeRunEvent union. function toEvent(update: SessionUpdate): CodeRunEvent { switch (update.sessionUpdate) { @@ -130,19 +140,40 @@ export class AcpClient { this.connection = new ClientSideConnection(() => client, stream); try { - const init = await this.connection.initialize({ + const init = await this.withStartupTimeout(this.connection.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: { fs: { readTextFile: true, writeTextFile: true } }, - }); + })); this.loadSession_ = init.agentCapabilities?.loadSession === true; } catch (e) { throw this.enrich(e, 'initialize'); } } + // Race a startup-phase request against the deadline so a wedged engine fails with a + // clear, enriched error instead of leaving the turn pending forever. Callers dispose + // the client on failure, which kills the spawned adapter. + private async withStartupTimeout(work: Promise): Promise { + let timer: ReturnType | undefined; + const timeout = new Promise((_, reject) => { + timer = setTimeout(() => { + reject(new Error( + `timed out after ${STARTUP_TIMEOUT_MS / 1000}s — the ${this.agent} engine failed to ` + + `complete startup (it may be wedged or misconfigured)`, + )); + }, STARTUP_TIMEOUT_MS); + timer.unref?.(); + }); + try { + return await Promise.race([work, timeout]); + } finally { + if (timer) clearTimeout(timer); + } + } + async newSession(): Promise { try { - const res = await this.conn().newSession({ cwd: this.cwd, mcpServers: [] }); + const res = await this.withStartupTimeout(this.conn().newSession({ cwd: this.cwd, mcpServers: [] })); return res.sessionId; } catch (e) { throw this.enrich(e, 'newSession'); @@ -151,7 +182,7 @@ export class AcpClient { async loadSession(sessionId: string): Promise { try { - await this.conn().loadSession({ sessionId, cwd: this.cwd, mcpServers: [] }); + await this.withStartupTimeout(this.conn().loadSession({ sessionId, cwd: this.cwd, mcpServers: [] })); } catch (e) { throw this.enrich(e, 'loadSession'); } diff --git a/apps/x/packages/core/src/code-mode/acp/engine-manifest.ts b/apps/x/packages/core/src/code-mode/acp/engine-manifest.ts new file mode 100644 index 00000000..8d2f042d --- /dev/null +++ b/apps/x/packages/core/src/code-mode/acp/engine-manifest.ts @@ -0,0 +1,100 @@ +// AUTO-GENERATED by packages/core/scripts/gen-engine-manifest.mjs — do not edit by hand. +// Regenerate after bumping the @agentclientprotocol/*-acp adapter (engine) versions. +// Maps each agent + platform to the npm tarball + integrity of its native engine. + +export const ENGINE_MANIFEST = { + "claude": { + "version": "0.3.156", + "platforms": { + "darwin-arm64": { + "pkg": "@anthropic-ai/claude-agent-sdk-darwin-arm64", + "pkgVersion": "0.3.156", + "tarball": "https://registry.npmjs.org/@anthropic-ai/claude-agent-sdk-darwin-arm64/-/claude-agent-sdk-darwin-arm64-0.3.156.tgz", + "integrity": "sha512-IkjcS9dqAUlD4Nb62L9AZtmAXCa+FV4ul8lIlyXXUprh3nlecbKsWOXVd/GORrzAhMmynJaX4+iV1JiutFKXUA==" + }, + "darwin-x64": { + "pkg": "@anthropic-ai/claude-agent-sdk-darwin-x64", + "pkgVersion": "0.3.156", + "tarball": "https://registry.npmjs.org/@anthropic-ai/claude-agent-sdk-darwin-x64/-/claude-agent-sdk-darwin-x64-0.3.156.tgz", + "integrity": "sha512-6PKi5fPmGRuzXu+Em/iwLmPG3mqg0hl92wcTU8fmChqyNtxhxsjCw7LTbdFqp/05o5NeZVVV4k3p7YUv5IFD6g==" + }, + "linux-x64": { + "pkg": "@anthropic-ai/claude-agent-sdk-linux-x64", + "pkgVersion": "0.3.156", + "tarball": "https://registry.npmjs.org/@anthropic-ai/claude-agent-sdk-linux-x64/-/claude-agent-sdk-linux-x64-0.3.156.tgz", + "integrity": "sha512-ymhrdlbWoYvTACUdaGdhrEv+ZMfwXLsf0BRLkr/IvY5aqybP7URzWmmZGOtDQpqkT/8xu/UCGqUYH3woJwUxfg==" + }, + "linux-arm64": { + "pkg": "@anthropic-ai/claude-agent-sdk-linux-arm64", + "pkgVersion": "0.3.156", + "tarball": "https://registry.npmjs.org/@anthropic-ai/claude-agent-sdk-linux-arm64/-/claude-agent-sdk-linux-arm64-0.3.156.tgz", + "integrity": "sha512-H0Nfd41iw5isto9uQI1FlVSZ0eaDttr8rBpJMR25oK/mj3egMO5EmZ6aAxeeUYSLn2mSU50HA5VNxlGUE118TQ==" + }, + "linux-x64-musl": { + "pkg": "@anthropic-ai/claude-agent-sdk-linux-x64-musl", + "pkgVersion": "0.3.156", + "tarball": "https://registry.npmjs.org/@anthropic-ai/claude-agent-sdk-linux-x64-musl/-/claude-agent-sdk-linux-x64-musl-0.3.156.tgz", + "integrity": "sha512-/Q6WUizI6a+hqZZ6ElwRU0PEuFhOoN4v6CuU35HHbiZ/7uaocGht4A8ZIgK1Fw6wOGtZzGLbc00CA1OU1Zg8EA==" + }, + "linux-arm64-musl": { + "pkg": "@anthropic-ai/claude-agent-sdk-linux-arm64-musl", + "pkgVersion": "0.3.156", + "tarball": "https://registry.npmjs.org/@anthropic-ai/claude-agent-sdk-linux-arm64-musl/-/claude-agent-sdk-linux-arm64-musl-0.3.156.tgz", + "integrity": "sha512-R7KEVjxkR4rYgIQoHGBzwPdUJYxRTO8I4vHjRbMLH1eW4FS7BJvVs7ogfKR/NnHFBvMVqtC+l6jHLQv8bobUiw==" + }, + "win32-x64": { + "pkg": "@anthropic-ai/claude-agent-sdk-win32-x64", + "pkgVersion": "0.3.156", + "tarball": "https://registry.npmjs.org/@anthropic-ai/claude-agent-sdk-win32-x64/-/claude-agent-sdk-win32-x64-0.3.156.tgz", + "integrity": "sha512-/PofeTWoiKgnWNSNk0wG4SsRn22GGLmnLhg2R94WcNhCRFOyOTmiZcYH2DBlWZBIRVTZDsSfa/Pl1DyPvYCGKw==" + }, + "win32-arm64": { + "pkg": "@anthropic-ai/claude-agent-sdk-win32-arm64", + "pkgVersion": "0.3.156", + "tarball": "https://registry.npmjs.org/@anthropic-ai/claude-agent-sdk-win32-arm64/-/claude-agent-sdk-win32-arm64-0.3.156.tgz", + "integrity": "sha512-5sAeNObQQrMy4NF9HwxewrMnU7mVxZDHh+/MfJVQSz0GSTvXQ6gOuRH8helMlfspoU6VOdekPxVLRooX/3foEw==" + } + } + }, + "codex": { + "version": "0.128.0", + "platforms": { + "darwin-arm64": { + "pkg": "@openai/codex", + "pkgVersion": "0.128.0-darwin-arm64", + "tarball": "https://registry.npmjs.org/@openai/codex/-/codex-0.128.0-darwin-arm64.tgz", + "integrity": "sha512-w+6zohfHx/kHBdles/CyFKaY57u9I3nK8QI9+NrdwMliKA0b7xn13yblRNkMpe09j6vL1oAWoxYsMOQ/vjBGug==" + }, + "darwin-x64": { + "pkg": "@openai/codex", + "pkgVersion": "0.128.0-darwin-x64", + "tarball": "https://registry.npmjs.org/@openai/codex/-/codex-0.128.0-darwin-x64.tgz", + "integrity": "sha512-SDbn6fO22Puy8xmMIbZi4f2znMrUEPwABApke4mo+4ihaauwuVjeqzXvW5SPJz5ty/bG11/mSupQgReT7T8BBw==" + }, + "linux-x64": { + "pkg": "@openai/codex", + "pkgVersion": "0.128.0-linux-x64", + "tarball": "https://registry.npmjs.org/@openai/codex/-/codex-0.128.0-linux-x64.tgz", + "integrity": "sha512-2lnSPA05CRRuKAzFW8BCmmNCSieDcToLwfC2ALLbBYilGLgzhRibjlDglK9F1BkEzfohSSWJu4PBbRu/aG60lQ==" + }, + "linux-arm64": { + "pkg": "@openai/codex", + "pkgVersion": "0.128.0-linux-arm64", + "tarball": "https://registry.npmjs.org/@openai/codex/-/codex-0.128.0-linux-arm64.tgz", + "integrity": "sha512-+SvH73H60qvCXFuQGP/EsmR//s1hHMBR22PvJkXvM/hdnTIGucx+JqRUjAWdmmQ1IU6j3kgwVvdLW/6ICB+M6w==" + }, + "win32-x64": { + "pkg": "@openai/codex", + "pkgVersion": "0.128.0-win32-x64", + "tarball": "https://registry.npmjs.org/@openai/codex/-/codex-0.128.0-win32-x64.tgz", + "integrity": "sha512-k3jmUAFrzkUtvjGTXvSKjQqJLLlzjxp/VoHJDYedgmXUn6j70HxK38IwapzmnYfiBiTuzETvGwjXHzZgzKjhoQ==" + }, + "win32-arm64": { + "pkg": "@openai/codex", + "pkgVersion": "0.128.0-win32-arm64", + "tarball": "https://registry.npmjs.org/@openai/codex/-/codex-0.128.0-win32-arm64.tgz", + "integrity": "sha512-ECJvsqmYFdA9pn42xxK3Odp/G16AjmBW0BglX8L0PwPjqbstbmlew9bfHf7xvL+SNfNl4NmyotW0+RNo1phgaA==" + } + } + } +} as const; diff --git a/apps/x/packages/core/src/code-mode/acp/engine-provisioner.ts b/apps/x/packages/core/src/code-mode/acp/engine-provisioner.ts new file mode 100644 index 00000000..60443116 --- /dev/null +++ b/apps/x/packages/core/src/code-mode/acp/engine-provisioner.ts @@ -0,0 +1,284 @@ +// Code-mode engine provisioner. +// +// Code mode drives Claude Code / Codex through their ACP adapters, which spawn a heavy +// native engine binary (~200 MB each). We do NOT bundle those engines into the installer +// (that would add ~400 MB). Instead we provision them on demand: the first time an agent +// is used we download the per-platform npm package AT THE EXACT VERSION OUR ADAPTER WAS +// BUILT AGAINST (see engine-manifest.ts), verify its integrity, and extract it into +// ~/.rowboat/engines///. Subsequent runs reuse the cached copy. +// +// The adapters are then pointed at the provisioned binary via CLAUDE_CODE_EXECUTABLE / +// CODEX_PATH (see agents.ts). This keeps the installer small while making code mode work +// out of the box, with no dependency on the user having a global claude/codex install. + +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import * as crypto from 'crypto'; +import { spawnSync } from 'child_process'; +import { Readable } from 'stream'; +import { pipeline } from 'stream/promises'; +import { ENGINE_MANIFEST } from './engine-manifest.js'; +import type { CodingAgent } from './types.js'; + +export const ENGINES_ROOT = path.join(os.homedir(), '.rowboat', 'engines'); + +interface PlatformEntry { + pkg: string; + pkgVersion: string; + tarball: string; + integrity: string; +} + +export interface EngineProgress { + phase: 'check' | 'download' | 'verify' | 'extract' | 'done'; + /** Bytes received so far (download phase). */ + receivedBytes?: number; + /** Total bytes, when the server reports content-length. */ + totalBytes?: number; +} + +export interface EnsureEngineOptions { + onProgress?: (p: EngineProgress) => void; + signal?: AbortSignal; +} + +export interface ProvisionedEngine { + executablePath: string; + version: string; +} + +// Map this process's platform/arch (+ libc on linux) to a manifest platform key for the +// given agent. Returns null when no engine is published for this platform. +function platformKey(agent: CodingAgent): string | null { + const arch = process.arch === 'arm64' ? 'arm64' : process.arch === 'x64' ? 'x64' : null; + if (!arch) return null; + const plats = ENGINE_MANIFEST[agent].platforms as Record; + const candidates: string[] = []; + if (process.platform === 'darwin') { + candidates.push(`darwin-${arch}`); + } else if (process.platform === 'win32') { + candidates.push(`win32-${arch}`); + } else if (process.platform === 'linux') { + // Prefer a musl build on musl systems (Alpine); fall back to the glibc build. + if (isMuslLibc()) candidates.push(`linux-${arch}-musl`); + candidates.push(`linux-${arch}`); + } + return candidates.find((c) => c in plats) ?? null; +} + +// glibc builds expose a glibcVersionRuntime in the process report header; musl (Alpine) +// does not. Same heuristic Node's native-addon loaders use. +function isMuslLibc(): boolean { + try { + const report = (process as unknown as { report?: { getReport?: () => unknown } }).report?.getReport?.(); + const header = (report as { header?: Record } | undefined)?.header; + return !(header && 'glibcVersionRuntime' in header); + } catch { + return false; + } +} + +// Locate the engine executable inside an extracted package root. We extract the whole npm +// package (so codex's bundled ripgrep travels with it), then find the binary. +function locateExecutable(agent: CodingAgent, root: string): string | null { + if (agent === 'claude') { + for (const name of ['claude', 'claude.exe']) { + const p = path.join(root, name); + if (fs.existsSync(p)) return p; + } + return null; + } + // codex: vendor//codex/codex[.exe] + const vendor = path.join(root, 'vendor'); + if (!fs.existsSync(vendor)) return null; + for (const triple of fs.readdirSync(vendor)) { + for (const name of ['codex', 'codex.exe']) { + const p = path.join(vendor, triple, 'codex', name); + if (fs.existsSync(p)) return p; + } + } + return null; +} + +// True when this OS/arch has a published engine for `agent` — i.e. we can provision it. +// (Used for status: code mode no longer requires a user-installed CLI.) +export function isEngineSupported(agent: CodingAgent): boolean { + return platformKey(agent) !== null; +} + +// True when the pinned engine for `agent` is already downloaded and intact locally. +export function isEngineProvisioned(agent: CodingAgent): boolean { + const version = ENGINE_MANIFEST[agent].version; + const versionDir = path.join(ENGINES_ROOT, agent, version); + const metaPath = path.join(ENGINES_ROOT, agent, '.meta', `${agent}-${version}.json`); + return locateExecutable(agent, versionDir) !== null && fs.existsSync(metaPath); +} + +const AGENT_LABEL: Record = { claude: 'Claude Code', codex: 'Codex' }; + +// Return the provisioned engine's executable path, or throw a clear, user-facing error. +// The chat/run path uses this — we deliberately do NOT download here: the engine must be +// enabled up front in Settings → Code Mode, so the user never eats a surprise ~200 MB +// download mid-conversation. ensureEngine() (the downloading path) is driven only by the +// Settings "Enable" action. +export function getProvisionedEnginePath(agent: CodingAgent): string { + const version = ENGINE_MANIFEST[agent].version; + const exe = locateExecutable(agent, path.join(ENGINES_ROOT, agent, version)); + if (!exe) { + throw new Error( + `${AGENT_LABEL[agent]} isn't enabled yet. Open Settings → Code Mode and click Enable to download it.`, + ); + } + return exe; +} + +// Remove every provisioned version of `agent` except `keepVersion`, plus its stale +// .meta entries. Called after a successful install so old engines don't pile up across +// version bumps. Best-effort — never throws (cleanup must not fail a good install). +function pruneOldVersions(agent: CodingAgent, keepVersion: string): void { + const agentRoot = path.join(ENGINES_ROOT, agent); + try { + for (const name of fs.readdirSync(agentRoot)) { + // Keep the active version, the meta dir, and any in-flight temp dirs. + if (name === keepVersion || name === '.meta' || name.startsWith('.tmp-')) continue; + const full = path.join(agentRoot, name); + try { + if (fs.statSync(full).isDirectory()) fs.rmSync(full, { recursive: true, force: true }); + } catch { /* ignore a single stubborn entry */ } + } + const metaDir = path.join(agentRoot, '.meta'); + if (fs.existsSync(metaDir)) { + for (const f of fs.readdirSync(metaDir)) { + if (f !== `${agent}-${keepVersion}.json`) { + try { fs.rmSync(path.join(metaDir, f), { force: true }); } catch { /* ignore */ } + } + } + } + } catch { /* agentRoot unreadable — nothing to prune */ } +} + +async function downloadTo(url: string, dest: string, opts: EnsureEngineOptions): Promise { + opts.onProgress?.({ phase: 'download', receivedBytes: 0 }); + const res = await fetch(url, { signal: opts.signal }); + if (!res.ok || !res.body) { + throw new Error(`Code mode: engine download failed (HTTP ${res.status}) — ${url}`); + } + const total = Number(res.headers.get('content-length')) || undefined; + let received = 0; + const body = Readable.fromWeb(res.body as Parameters[0]); + body.on('data', (chunk: Buffer) => { + received += chunk.length; + opts.onProgress?.({ phase: 'download', receivedBytes: received, totalBytes: total }); + }); + await pipeline(body, fs.createWriteStream(dest)); +} + +// Verify the tarball against the npm Subresource Integrity string ("sha512-"). +function verifyIntegrity(file: string, integrity: string): void { + const dash = integrity.indexOf('-'); + const algo = integrity.slice(0, dash); + const expected = integrity.slice(dash + 1); + const actual = crypto.createHash(algo).update(fs.readFileSync(file)).digest('base64'); + if (actual !== expected) { + throw new Error(`Code mode: engine integrity check failed (${algo}) — download may be corrupt.`); + } +} + +// Extract an npm tarball, stripping its leading `package/` component so the package +// contents land directly in destDir. Uses the system tar (bsdtar on macOS/Windows 10+, +// GNU tar on Linux) — all support -xzf and --strip-components. +function extractTarball(tarPath: string, destDir: string): void { + const r = spawnSync('tar', ['-xzf', tarPath, '-C', destDir, '--strip-components=1'], { stdio: 'pipe' }); + if (r.status !== 0) { + const err = r.stderr?.toString().trim() || r.error?.message || `tar exited ${r.status}`; + throw new Error(`Code mode: failed to extract engine — ${err}`); + } +} + +// Mark the engine (and codex's bundled ripgrep) executable on unix. +function makeExecutable(agent: CodingAgent, root: string, exe: string): void { + fs.chmodSync(exe, 0o755); + if (agent === 'codex') { + const vendor = path.join(root, 'vendor'); + for (const triple of fs.existsSync(vendor) ? fs.readdirSync(vendor) : []) { + const rg = path.join(vendor, triple, 'path', 'rg'); + if (fs.existsSync(rg)) fs.chmodSync(rg, 0o755); + } + } +} + +/** + * Ensure the pinned engine for `agent` is provisioned locally, downloading it on first + * use. Returns the absolute path to the engine executable. Idempotent and cached. + */ +export async function ensureEngine(agent: CodingAgent, opts: EnsureEngineOptions = {}): Promise { + const entry = ENGINE_MANIFEST[agent]; + const version = entry.version; + const key = platformKey(agent); + if (!key) { + throw new Error(`Code mode: no ${agent} engine is available for ${process.platform}/${process.arch}.`); + } + const plat = (entry.platforms as Record)[key]; + + const agentRoot = path.join(ENGINES_ROOT, agent); + const versionDir = path.join(agentRoot, version); + const metaDir = path.join(agentRoot, '.meta'); + const metaPath = path.join(metaDir, `${agent}-${version}.json`); + + opts.onProgress?.({ phase: 'check' }); + // Fast path: already provisioned and intact. + const existing = locateExecutable(agent, versionDir); + if (existing && fs.existsSync(metaPath)) { + opts.onProgress?.({ phase: 'done' }); + return { executablePath: existing, version }; + } + + // Download to a unique temp dir, verify, extract, then swap into place. Concurrent + // callers each use their own temp dir; the final rename is idempotent (same content). + fs.mkdirSync(agentRoot, { recursive: true }); + const tmpRoot = fs.mkdtempSync(path.join(agentRoot, `.tmp-${version}-`)); + try { + const tarPath = path.join(tmpRoot, 'engine.tgz'); + await downloadTo(plat.tarball, tarPath, opts); + + opts.onProgress?.({ phase: 'verify' }); + verifyIntegrity(tarPath, plat.integrity); + + opts.onProgress?.({ phase: 'extract' }); + const extractDir = path.join(tmpRoot, 'pkg'); + fs.mkdirSync(extractDir); + extractTarball(tarPath, extractDir); + + const exe = locateExecutable(agent, extractDir); + if (!exe) { + throw new Error(`Code mode: ${agent} engine binary not found in the downloaded package.`); + } + if (process.platform !== 'win32') makeExecutable(agent, extractDir, exe); + + // Swap the freshly extracted package into the versioned location. + if (fs.existsSync(versionDir)) fs.rmSync(versionDir, { recursive: true, force: true }); + fs.renameSync(extractDir, versionDir); + + const finalExe = locateExecutable(agent, versionDir); + if (!finalExe) { + throw new Error(`Code mode: ${agent} engine binary missing after install.`); + } + fs.mkdirSync(metaDir, { recursive: true }); + fs.writeFileSync(metaPath, JSON.stringify({ + version, + platform: key, + integrity: plat.integrity, + binRelPath: path.relative(versionDir, finalExe), + }, null, 2)); + + // A new version is in place — remove superseded versions so old engines + // (~200 MB each) don't accumulate after a bump. Best-effort. + pruneOldVersions(agent, version); + + opts.onProgress?.({ phase: 'done' }); + return { executablePath: finalExe, version }; + } finally { + fs.rmSync(tmpRoot, { recursive: true, force: true }); + } +} diff --git a/apps/x/packages/core/src/code-mode/acp/manager.ts b/apps/x/packages/core/src/code-mode/acp/manager.ts index 033bf994..0e12d9ed 100644 --- a/apps/x/packages/core/src/code-mode/acp/manager.ts +++ b/apps/x/packages/core/src/code-mode/acp/manager.ts @@ -174,13 +174,19 @@ export class CodeModeManager { broker, onEvent: suppressReplay ? () => {} : onEvent, }); - await client.start(); - - const sessionId = await this.openSession(runId, agent, cwd, client); - if (suppressReplay) client.setHandlers(broker, onEvent); - const run: ActiveRun = { client, sessionId, agent, cwd, inflight: 0 }; - this.runs.set(runId, run); - return run; + // Dispose the client if startup fails (e.g. the startup-timeout fires) so the + // spawned adapter process doesn't leak. + try { + await client.start(); + const sessionId = await this.openSession(runId, agent, cwd, client); + if (suppressReplay) client.setHandlers(broker, onEvent); + const run: ActiveRun = { client, sessionId, agent, cwd, inflight: 0 }; + this.runs.set(runId, run); + return run; + } catch (e) { + client.dispose(); + throw e; + } } // Resume the persisted session for this chat when possible; else start a new one diff --git a/apps/x/packages/core/src/code-mode/acp/shell-env.ts b/apps/x/packages/core/src/code-mode/acp/shell-env.ts new file mode 100644 index 00000000..fb8b8edf --- /dev/null +++ b/apps/x/packages/core/src/code-mode/acp/shell-env.ts @@ -0,0 +1,40 @@ +import { execSync } from 'child_process'; +import * as path from 'path'; + +let cached: string | null = null; + +// The user's login-shell PATH (macOS/Linux; undefined on Windows or probe failure). +// GUI-launched Electron apps inherit launchd's stripped PATH (/usr/bin:/bin:...), so +// anything the engines spawn off process.env.PATH misses Homebrew/nvm/npm-global tools. +// The provisioned engine itself runs from an absolute path, but claude/codex spawn +// git, gh, rg, bash, etc. themselves — without this they fail with "command not found" +// on a Finder launch even though they work from a terminal. +export function loginShellPath(): string | undefined { + if (process.platform === 'win32') return undefined; + if (cached !== null) return cached || undefined; + + // Prefer the user's own shell when it's POSIX-flavored, so its login profile + // (~/.zprofile for zsh — macOS default — ~/.profile for bash/sh) is the one that + // builds the PATH. fish et al. are skipped: their `echo $PATH` is space-joined. + const userShell = process.env.SHELL; + const shellOk = userShell && ['sh', 'bash', 'zsh', 'dash', 'ksh'].includes(path.basename(userShell)); + const shells = [...new Set([...(shellOk ? [userShell] : []), '/bin/sh'])]; + + for (const shell of shells) { + try { + const out = execSync(`${shell} -lc 'echo $PATH'`, { timeout: 5000, encoding: 'utf-8' }); + // Profile scripts may echo their own lines; our `echo $PATH` runs last, + // so take the last non-empty line and sanity-check it looks like a PATH. + const lines = out.split('\n').map((l) => l.trim()).filter(Boolean); + const last = lines[lines.length - 1]; + if (last && last.includes('/')) { + cached = last; + return last; + } + } catch { + // probe failed — try the next shell + } + } + cached = ''; // remember the failure so we don't re-pay the probe every spawn + return undefined; +} diff --git a/apps/x/packages/core/src/code-mode/status.ts b/apps/x/packages/core/src/code-mode/status.ts index a78b23f4..40ab0c1e 100644 --- a/apps/x/packages/core/src/code-mode/status.ts +++ b/apps/x/packages/core/src/code-mode/status.ts @@ -3,8 +3,8 @@ import { promisify } from 'util'; import os from 'os'; import path from 'path'; import fs from 'fs/promises'; -import { existsSync } from 'fs'; import { CodeModeAgentStatus } from './types.js'; +import { isEngineProvisioned } from './acp/engine-provisioner.js'; const execAsync = promisify(exec); @@ -40,30 +40,6 @@ export function commonInstallPaths(binary: string): string[] { ].map(dir => path.join(dir, binary)); } -async function probeShell(binary: string): Promise { - try { - if (process.platform === 'win32') { - const { stdout } = await execAsync(`where ${binary}`, { timeout: 5000 }); - return stdout.trim().length > 0; - } - // Login shell so ~/.zprofile / ~/.bashrc PATH additions are visible — - // essential for Homebrew, nvm, asdf, volta installs on macOS GUI launches. - const { stdout } = await execAsync(`/bin/sh -lc 'command -v ${binary}'`, { timeout: 5000 }); - return stdout.trim().length > 0; - } catch { - return false; - } -} - -async function isInstalled(binary: string): Promise { - if (await probeShell(binary)) return true; - // Fallback: scan well-known install locations directly. - for (const candidate of commonInstallPaths(binary)) { - if (existsSync(candidate)) return true; - } - return false; -} - function decodeJwtPayload(token: string): Record | null { try { const parts = token.split('.'); @@ -186,14 +162,15 @@ async function checkCodexSignedIn(): Promise { export { decodeJwtPayload }; export async function checkCodeModeAgentStatus(): Promise { - const [claudeInstalled, codexInstalled, claudeSignedIn, codexSignedIn] = await Promise.all([ - isInstalled('claude'), - isInstalled('codex'), + const [claudeSignedIn, codexSignedIn] = await Promise.all([ checkClaudeSignedIn(), checkCodexSignedIn(), ]); + // `installed` means the engine is provisioned (downloaded) locally — the user has + // clicked Enable in Settings → Code Mode. We no longer look for a global claude/codex + // CLI on PATH; code mode runs our own pinned engine from ~/.rowboat/engines. return { - claude: { installed: claudeInstalled, signedIn: claudeSignedIn }, - codex: { installed: codexInstalled, signedIn: codexSignedIn }, + claude: { installed: isEngineProvisioned('claude'), signedIn: claudeSignedIn }, + codex: { installed: isEngineProvisioned('codex'), signedIn: codexSignedIn }, }; } diff --git a/apps/x/packages/shared/src/ipc.ts b/apps/x/packages/shared/src/ipc.ts index 8ad606bb..d0ca382a 100644 --- a/apps/x/packages/shared/src/ipc.ts +++ b/apps/x/packages/shared/src/ipc.ts @@ -481,6 +481,22 @@ const ipcSchemas = { codex: z.object({ installed: z.boolean(), signedIn: z.boolean() }), }), }, + // Download + install an agent's native engine (the Settings "Enable" action). + // Streams progress over the 'codeMode:engineProgress' push channel while it runs. + 'codeMode:provisionEngine': { + req: z.object({ agent: z.enum(['claude', 'codex']) }), + res: z.object({ success: z.boolean(), error: z.string().optional() }), + }, + // Push (main -> renderer): engine provisioning progress for the Settings UI. + 'codeMode:engineProgress': { + req: z.object({ + agent: z.enum(['claude', 'codex']), + phase: z.enum(['download', 'verify', 'extract', 'done']), + receivedBytes: z.number().optional(), + totalBytes: z.number().optional(), + }), + res: z.null(), + }, // ========================================================================== // Code section: project registry + coding sessions // ========================================================================== diff --git a/apps/x/pnpm-workspace.yaml b/apps/x/pnpm-workspace.yaml index b4bc70d9..3f3b314a 100644 --- a/apps/x/pnpm-workspace.yaml +++ b/apps/x/pnpm-workspace.yaml @@ -3,13 +3,14 @@ packages: - packages/* allowBuilds: - core-js: set this to true or false - electron: set this to true or false - electron-winstaller: set this to true or false - esbuild: set this to true or false - fs-xattr: set this to true or false - macos-alias: set this to true or false - protobufjs: set this to true or false + core-js: true + electron: true + electron-winstaller: true + esbuild: true + fs-xattr: true + macos-alias: true + node-pty: true + protobufjs: true catalog: vitest: 4.1.7