rowboat/apps/x/PLAN-MEDIA.md

# Media File Rendering — Implementation Plan

## Goal
Render PDFs, images, and videos in the knowledge view, alongside the existing HTML viewer.

## Foundation
All three file types are served through a single custom `app://` protocol registered in the Electron main process. This avoids `file://` (elevated privileges in Electron) and supports byte-range streaming (needed for video seeking).

---

## Phase 1 — Register `app://` custom protocol

### What
Main-process protocol handler that serves any workspace-relative file with path traversal guard.

### Work
1. In `apps/main/src/main.ts`, before `app.whenReady()`:
   - `protocol.registerSchemesAsPrivileged` for `app` with `{ standard: true, secure: true, supportFetchAPI: true, stream: true }`
2. After `app.whenReady()`:
   - `protocol.handle('app', ...)` — parses URL, decodes path, resolves against workspace root using the existing `resolveWorkspacePath`, returns `net.fetch(pathToFileURL(absPath))`
   - On invalid/outside-workspace path: return 403

### Test ✅
- DevTools: `fetch('app://local/knowledge/<known-file>').then(r => r.status)` → 200
- DevTools: `fetch('app://local/../../etc/passwd')` → 403
- Verify response includes correct mime type for known extensions

---

## Phase 2 — Image renderer

### What
`ImageFileViewer` component renders supported images via `<img src="app://local/<path>">`.

### Work
1. New file `apps/renderer/src/components/image-file-viewer.tsx`
2. Detects: `.png .jpg .jpeg .webp .gif .svg .avif .bmp .ico`
3. Centered, object-contain layout, dark/light bg
4. HEIC/HEIF (or any other unsupported decode failure) → fall back to "Open in system" via `shell:openPath`
5. Loading state and error state mirroring HtmlFileViewer
6. Wire into `App.tsx` render switch before the `<pre>` fallback

### Test ✅
- Open `.png`, `.jpg`, `.svg`, `.webp` files → renders correctly
- Open a `.heic` file (or rename a non-image to `.png` to force decode failure) → shows fallback
- Switch between image files rapidly → no stale image, no flicker

---

## Phase 3 — Video renderer

### What
`VideoFileViewer` component using native `<video>` tag.

### Work
1. New file `apps/renderer/src/components/video-file-viewer.tsx`
2. Detects: `.mp4 .mov .webm .m4v`
3. `<video controls src="app://local/<path>" />` — full width, max height
4. Loading and error states
5. Wire into `App.tsx` render switch

### Test ✅
- Open a 50MB+ MP4 → starts playing, scrubber works, can seek to middle without re-downloading
- Verify byte-range requests in DevTools Network tab (Range header on requests)
- Switch away mid-playback → video stops cleanly, no leaked audio

---

## Phase 4 — PDF renderer

### What
`PdfFileViewer` using Chromium's built-in PDFium via `<iframe src="app://local/<path>">`.

### Work
1. New file `apps/renderer/src/components/pdf-file-viewer.tsx`
2. Detects: `.pdf`
3. `<iframe src="app://local/<path>" className="w-full h-full" />` — Chromium auto-detects PDF mime type and uses PDFium plugin
4. Confirm `webPreferences.plugins: true` is set on the BrowserWindow (required for PDFium to activate)
5. Wire into `App.tsx` render switch

### Test ✅
- Open multi-page PDF → renders with native zoom/scroll/print toolbar
- Open password-protected PDF → Chromium prompts for password (built-in)
- Switch between PDFs → no stale content

---

## Out of scope for this branch
- DOCX rendering (requires `docx-preview` library — separate PR)
- Split-pane resizable layout
- Audio (the existing `AudioFileCard` is reused if needed)
- Annotation/highlighting on PDFs (would require `pdfjs-dist`)
- Iframe persistence cache for HTML viewer (separate optimization)