From 08a4077d4311ff89f26834bb24cd1905de7ae998 Mon Sep 17 00:00:00 2001 From: gagan Date: Thu, 2 Jul 2026 00:35:35 +0530 Subject: [PATCH 01/24] Meeting prep: attendee notes + proactive 6h-ahead prep briefs (#636) * feat(meetings): inline meeting prep under next meeting Resolve a meeting's attendees against the knowledge base and render each attendee's existing person.md inline beneath the next upcoming meeting in the Meetings view. Deterministic (email-exact match, then unambiguous name/alias), no LLM in the hot path. - core: resolveMeetingPrep() + meeting-prep:resolve IPC - renderer: notes-first rows with expandable person.md; unmatched attendees collapse into a 'no notes yet' row with a Create note action that hands off to the Copilot - prep re-resolves when a People note changes * feat(meetings): per-meeting prep toggle Add a Prep toggle to every eligible meeting row, not just the next one. The next meeting still auto-expands; other meetings resolve their attendee notes lazily when their toggle is opened. All-day and solo events get no toggle. * feat(meetings): org matching, cached index, and field-bleed fix #2 Matching: resolve the meeting's external companies from attendee email domains (and matched people's Organization field), excluding the user's own domain, and surface those org notes in a Companies section. Normalize display names (strip '(via ...)' suffixes) before name/alias matching. #3 Perf: meeting prep reads a cached knowledge index instead of rescanning the whole knowledge dir on every resolve; invalidated whenever a file under knowledge/ changes (wired to the workspace watcher). Fix: extractField in the knowledge index consumed newlines after the label, so an empty field (e.g. **Role:**) bled the next line's value. Restrict it to spaces/tabs so empty fields resolve to undefined. Also strip the folder prefix from link targets for display (Organizations/Rowboat Labs -> Rowboat Labs). * feat(meetings): proactive prep notes generated 6h ahead Generate a meeting prep note ~6h before each meeting and surface its brief in the prep card. - Generator (meeting_prep_brief.ts): assembles roster + 'last time' recap (extracts the prior instance's Action items) + agenda, ordered by meeting type (recurring -> recap first, one-off -> agenda first), then one model call for a 'what matters' brief. Writes knowledge/Meetings/prep/-.md with eventId/recurringEventId frontmatter. Reuses the summarizeMeeting LLM path (configured model, useCase: meeting_prep). - Scheduler (meeting_prep_scheduler.ts): calendar-aware tick (5m), generates prep within the 6h lead window, state file dedupes, self-heals on changes. - Card: shows the brief (bulleted, compact) above a People list whose rows link out to each person's note instead of rendering it inline. Generated prep notes are kept out of the past-notes table. * chore(meetings): poll prep scheduler every 15m instead of 5m --- PLAN.md | 111 +++++++ apps/x/apps/main/src/ipc.ts | 26 ++ apps/x/apps/main/src/main.ts | 4 + apps/x/apps/renderer/src/App.tsx | 14 + .../renderer/src/components/meetings-view.tsx | 271 +++++++++++++++- .../x/packages/core/src/analytics/use_case.ts | 2 +- .../core/src/knowledge/knowledge_index.ts | 34 +- .../core/src/knowledge/meeting_prep.ts | 230 +++++++++++++ .../core/src/knowledge/meeting_prep_brief.ts | 307 ++++++++++++++++++ .../src/knowledge/meeting_prep_scheduler.ts | 158 +++++++++ apps/x/packages/shared/src/ipc.ts | 41 +++ 11 files changed, 1187 insertions(+), 11 deletions(-) create mode 100644 PLAN.md create mode 100644 apps/x/packages/core/src/knowledge/meeting_prep.ts create mode 100644 apps/x/packages/core/src/knowledge/meeting_prep_brief.ts create mode 100644 apps/x/packages/core/src/knowledge/meeting_prep_scheduler.ts diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 00000000..05d6cd1b --- /dev/null +++ b/PLAN.md @@ -0,0 +1,111 @@ +# Meeting Prep — "Next up" prep card + +## Goal + +When a meeting is imminent, surface who will be there by pulling up each +attendee's existing `person.md` from the knowledge base and showing it — no +click, no LLM in the hot path. An ambient card that's just *ready* when you +look. + +This is a deliberately light, deterministic surface. It does **not** replace +the two existing meeting-prep paths: +- the chat skill (`assistant/skills/meeting-prep/`) — LLM brief on demand +- the pre-built automation (`pre_built/meeting-prep.md`) — LLM brief files + +It complements them with an instant, no-LLM "who's in this meeting" view. + +## Decisions (settled) + +| Axis | Choice | +|------|--------| +| Surface | **Ambient card**, no notification. Pinned at top of Meetings view. | +| Activation | Next eligible meeting is **≤ 30 min away**. Outside that → no card. | +| Depth | **Just show `person.md`** — render the note, no LLM synthesis. | +| Who | **Everyone incl. teammates.** Skip self / all-day / declined / cancelled. | +| Sparse meetings | **Notes-first**: attendees with notes shown prominently; the rest collapse into "N others — no notes yet" → expand to create. | +| No-note action | **Offer to create** a note for that attendee. | + +## Resolution rules + +For each attendee (excluding `self`): +1. **Email exact match** (case-insensitive) against `KnowledgeIndex.people[].email` — reliable. +2. Else **name/alias match** on `displayName` → `people[].name` / `people[].aliases`, + only when it's a **single unambiguous hit**. +3. Matched → read `person.md`, return `{ name, role, org, path, markdown }`. +4. Unmatched → returned as a "no note" attendee (name + email). + +Org notes (`Organizations/{Org}.md`) are **deferred** — `person.md` already +carries Org/Role inline, enough for v1. + +## Architecture + +| Piece | Location | Notes | +|-------|----------|-------| +| Attendee → note resolver | **new IPC** `meeting-prep:resolve` in `apps/main/src/ipc.ts` | Input: attendee list (`{email?, displayName?, self?}[]`). Builds/uses `buildKnowledgeIndex()`, resolves per rules above, reads matched notes, returns structured payload. Deterministic + fast. | +| Shared types | `packages/shared/src/` | `MeetingPrepAttendee`, `MeetingPrepResult` request/response shapes + IPC channel typing. | +| "Next up" card UI | **new component** rendered above `UpcomingEvents` in `meetings-view.tsx` | Picks next event ≤30 min out, calls IPC, renders notes-first + collapsed others. Reuses existing markdown rendering. | +| Create-note action | reuse the note-creation agent (only LLM touch, user-initiated) | v1: open a prefilled Copilot chat to create the note (visible/steerable) rather than firing silently. | + +No new background loop — the renderer already polls `calendar_sync/` every +minute, so the card stays current for free. + +## Build slices + +1. **Resolver IPC + shared types** — `meeting-prep:resolve`, wired through + preload. Unit-testable in isolation. ← first slice +2. **"Next up" card UI** — component + 30-min gating + notes-first / collapsed + rendering, reading from the IPC. +3. **Create-note action** — prefilled Copilot chat for an unmatched attendee. + +## Out of scope (fast-follow candidates) + +- Notification variant (extend `notify_calendar_meetings.ts` tick). +- Org-note surfacing. +- Configurable lead time (hardcode 30 min for v1). +- LLM-synthesized brief button on the card. + +## Phase 2 — Proactive, meeting-aware prep (6h ahead) + +Shift from a static contact viewer to meeting-aware prep, generated ahead of time. + +### Decisions +- **Trigger:** a calendar-driven tick (clone of `notify_calendar_meetings.ts`), NOT the + cron/bg-task engine — fire time is `start − 6h`, which varies per event, so a + calendar scan + state file is the right fit (self-healing on reschedule/cancel). +- **Storage:** write a real note → `knowledge/Meetings/prep/-.md` (openable/editable). +- **Notify:** none — silent background generation; appears in the Meetings view. +- **Adaptive by meeting type:** + - Recurring (`recurringEventId` + prior notes found) → lead with **"Last time" recap** + (prior instance's `## Action items` / summary), then roster. + - One-off / new → lead with **agenda** (event `description`), then roster + company. +- **Roster:** always list every attendee; ones with a note link to it, ones without + get **Create note**. +- **AI brief:** the only LLM step. Reuses the `summarizeMeeting` path exactly — + `generateText` against the user's configured model (`getMeetingNotesModel` / + `getDefaultModelAndProvider`), tagged `useCase: 'meeting_prep'`. Deterministic + parts (roster, recap, agenda) are assembled in code; the model only writes the + "what matters" summary. + +### Build slices +- **A. Core generator** (`meeting_prep_brief.ts`): assemble context (roster + recap + + agenda, meeting-type aware) → `generatePrepBrief()` (one model call) → render + + `writePrepNote()` to `knowledge/Meetings/prep/`. Frontmatter stamps `eventId` / + `recurringEventId` / `meetingDate` / `generatedAt`. +- **B. Scheduler** (`meeting_prep_scheduler.ts`): tick (~few min), scan `calendar_sync/`, + for events starting within 6h, not ended, ≥1 non-self attendee, not already prepped + (state file keyed by eventId) → generate + write. `init()` wired in `main.ts` beside + `initCalendarNotifications()`. +- **C. Card integration:** the inline prep card prefers the generated prep note when it + exists (instant), else falls back to the live deterministic resolve. + +### Open detail / enabler +- Notes don't store an event id today, so recap linking is a title/date heuristic. + Stamp `eventId`/`recurringEventId` into captured notes' frontmatter going forward so + future links are exact. + +## Verification + +- Resolver: feed known attendee emails → correct note match; unknown → no-note. +- Card: appears only inside the 30-min window; notes-first ordering; collapse + toggles; create action opens prefilled chat. +- `npm run deps && npm run lint` clean. diff --git a/apps/x/apps/main/src/ipc.ts b/apps/x/apps/main/src/ipc.ts index a1f6c4a6..9665172a 100644 --- a/apps/x/apps/main/src/ipc.ts +++ b/apps/x/apps/main/src/ipc.ts @@ -62,6 +62,9 @@ import { IAgentScheduleRepo } from '@x/core/dist/agent-schedule/repo.js'; import { IAgentScheduleStateRepo } from '@x/core/dist/agent-schedule/state-repo.js'; import { triggerRun as triggerAgentScheduleRun } from '@x/core/dist/agent-schedule/runner.js'; import { search } from '@x/core/dist/search/search.js'; +import { resolveMeetingPrep } from '@x/core/dist/knowledge/meeting_prep.js'; +import { readPrepNoteForEvent } from '@x/core/dist/knowledge/meeting_prep_brief.js'; +import { invalidateKnowledgeIndex } from '@x/core/dist/knowledge/knowledge_index.js'; import { versionHistory, voice } from '@x/core'; import { classifySchedule, processRowboatInstruction } from '@x/core/dist/knowledge/inline_tasks.js'; import { getBillingInfo } from '@x/core/dist/billing/billing.js'; @@ -507,7 +510,25 @@ function queueChange(relPath: string): void { /** * Handle workspace change event from core watcher */ +function touchesKnowledge(event: z.infer): boolean { + const hit = (p: string | undefined) => typeof p === 'string' && p.startsWith('knowledge/'); + switch (event.type) { + case 'created': + case 'changed': + case 'deleted': + return hit(event.path); + case 'moved': + return hit(event.from) || hit(event.to); + case 'bulkChanged': + return !event.paths || event.paths.some(hit); + default: + return false; + } +} + function handleWorkspaceChange(event: z.infer): void { + // Any knowledge-base change drops the cached index so the next read rebuilds. + if (touchesKnowledge(event)) invalidateKnowledgeIndex(); // Debounce 'changed' events, emit others immediately if (event.type === 'changed' && event.path) { queueChange(event.path); @@ -1544,6 +1565,11 @@ export function setupIpcHandlers() { const notes = await summarizeMeeting(args.transcript, args.meetingStartTime, args.calendarEventJson); return { notes }; }, + 'meeting-prep:resolve': async (_event, args) => { + const result = await resolveMeetingPrep(args.attendees); + const prepNote = args.eventId ? await readPrepNoteForEvent(args.eventId) : null; + return { ...result, prepNote }; + }, 'inline-task:classifySchedule': async (_event, args) => { const schedule = await classifySchedule(args.instruction); return { schedule }; diff --git a/apps/x/apps/main/src/main.ts b/apps/x/apps/main/src/main.ts index 450d0f9e..e93d8ac5 100644 --- a/apps/x/apps/main/src/main.ts +++ b/apps/x/apps/main/src/main.ts @@ -27,6 +27,7 @@ import { init as initInlineTasks } from "@x/core/dist/knowledge/inline_tasks.js" import { init as initAgentRunner } from "@x/core/dist/agent-schedule/runner.js"; import { init as initAgentNotes } from "@x/core/dist/knowledge/agent_notes.js"; import { init as initCalendarNotifications } from "@x/core/dist/knowledge/notify_calendar_meetings.js"; +import { init as initMeetingPrep } from "@x/core/dist/knowledge/meeting_prep_scheduler.js"; import { init as initLiveNoteScheduler } from "@x/core/dist/knowledge/live-note/scheduler.js"; import { init as initEventProcessor, registerConsumer } from "@x/core/dist/events/init.js"; import { liveNoteEventConsumer } from "@x/core/dist/knowledge/live-note/event-consumer.js"; @@ -450,6 +451,9 @@ app.whenReady().then(async () => { // start calendar meeting notification service (fires 1-minute warnings) initCalendarNotifications(); + // start meeting prep scheduler (generates prep notes ~6h before a meeting) + void initMeetingPrep(); + // start chrome extension sync server initChromeSync(); diff --git a/apps/x/apps/renderer/src/App.tsx b/apps/x/apps/renderer/src/App.tsx index cf1d3d8f..9f635bde 100644 --- a/apps/x/apps/renderer/src/App.tsx +++ b/apps/x/apps/renderer/src/App.tsx @@ -5238,6 +5238,20 @@ function App() { return () => window.removeEventListener('email-block:draft-with-assistant', handler) }, []) + // Meeting prep: create a person note for an unmatched attendee via Copilot. + useEffect(() => { + const handler = () => { + const pending = window.__pendingMeetingPrepCreate + if (pending) { + setPresetMessage(pending.prompt) + setIsChatSidebarOpen(true) + window.__pendingMeetingPrepCreate = undefined + } + } + window.addEventListener('meeting-prep:create-note', handler) + return () => window.removeEventListener('meeting-prep:create-note', handler) + }, []) + const resolveWikiFilePath = useCallback((wikiPath: string) => { const normalized = normalizeWikiPath(wikiPath) const { path: basePath } = splitWikiFragment(normalized) diff --git a/apps/x/apps/renderer/src/components/meetings-view.tsx b/apps/x/apps/renderer/src/components/meetings-view.tsx index 2ad04082..49090ce9 100644 --- a/apps/x/apps/renderer/src/components/meetings-view.tsx +++ b/apps/x/apps/renderer/src/components/meetings-view.tsx @@ -1,6 +1,7 @@ import { useCallback, useEffect, useMemo, useRef, useState } from 'react' import { createPortal } from 'react-dom' -import { Calendar, ChevronDown, Clock, ExternalLink, Loader2, MapPin, Mic, Square, UserRound, UsersRound, Video, X } from 'lucide-react' +import { Calendar, ChevronDown, ChevronRight, Clock, ExternalLink, FileText, Loader2, MapPin, Mic, Sparkles, Square, UserPlus, UserRound, UsersRound, Video, X } from 'lucide-react' +import { Streamdown } from 'streamdown' import { Button } from '@/components/ui/button' import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover' @@ -14,6 +15,39 @@ const MEETINGS_ROOT = 'knowledge/Meetings' const CALENDAR_DIR = 'calendar_sync' const UPCOMING_MAX_DAYS = 4 // today + next 3 +declare global { + interface Window { + __pendingMeetingPrepCreate?: { prompt: string } + } +} + +// Mirrors the `meeting-prep:resolve` IPC response shape. +type PrepNote = { + path: string + name: string + role?: string + organization?: string + markdown: string +} +type PrepAttendee = { + label: string + email?: string + displayName?: string + note: PrepNote | null +} +type PrepOrg = { + path: string + name: string + markdown: string +} +type PrepResult = { + attendees: PrepAttendee[] + organizations: PrepOrg[] + prepNote: { path: string; brief: string } | null + matchedCount: number + unmatchedCount: number +} + type MeetingNoteRow = { path: string name: string @@ -358,7 +392,178 @@ function parseDescriptionParts(value: string): DescriptionPart[] { }).filter((part) => part.text.length > 0) } -function UpcomingEvents() { +// Hand the unmatched attendee off to the Copilot to research + create a note. +function requestCreateNote(attendee: PrepAttendee, meetingSummary: string) { + const who = attendee.displayName || attendee.label + const email = attendee.email ? ` <${attendee.email}>` : '' + window.__pendingMeetingPrepCreate = { + prompt: `Create a person note in my knowledge base for ${who}${email}. They're attending my "${meetingSummary}" meeting. Pull together what you know about them from my emails, past meetings, and calendar.`, + } + window.dispatchEvent(new Event('meeting-prep:create-note')) +} + +// One note row (used for both people and organizations): a clickable row that +// navigates to the note. The markdown is NOT rendered inline in the card. +function PrepNoteRow({ title, subtitle, path, onOpenNote }: { + title: string + subtitle?: string + path: string + onOpenNote: (path: string) => void +}) { + return ( + + ) +} + +function PrepAttendeeNote({ attendee, onOpenNote }: { attendee: PrepAttendee; onOpenNote: (path: string) => void }) { + const note = attendee.note + if (!note) return null + const subtitle = [note.role, note.organization].filter(Boolean).join(' · ') + return +} + +function PrepUnmatchedSection({ attendees, meetingSummary }: { attendees: PrepAttendee[]; meetingSummary: string }) { + const [open, setOpen] = useState(false) + if (attendees.length === 0) return null + + return ( +
+ + {open ? ( +
+ {attendees.map((att, idx) => ( +
+ {att.label} + +
+ ))} +
+ ) : null} +
+ ) +} + +// Inline prep for a single event: resolves the attendees against the knowledge +// base and renders their notes directly beneath the event row. Re-resolves when +// a person note changes (e.g. after "Create note") so it stays fresh. +function InlineMeetingPrep({ event, onOpenNote }: { event: UpcomingEvent; onOpenNote: (path: string) => void }) { + const [prep, setPrep] = useState(null) + const [refreshTick, setRefreshTick] = useState(0) + + useEffect(() => { + let cancelled = false + const run = async () => { + try { + const attendees = event.attendees.map((a) => ({ email: a.email, displayName: a.displayName, self: a.self })) + const result = await window.ipc.invoke('meeting-prep:resolve', { attendees, eventId: event.id }) + if (!cancelled) setPrep(result) + } catch (err) { + console.error('Meeting prep failed:', err) + if (!cancelled) setPrep(null) + } + } + void run() + return () => { + cancelled = true + } + }, [event.id, refreshTick]) + + // Refresh when a People note is created/changed so newly-created notes appear. + useEffect(() => { + const isPeoplePath = (p: string | undefined) => + typeof p === 'string' && p.startsWith('knowledge/People/') + const cleanup = window.ipc.on('workspace:didChange', (e) => { + switch (e.type) { + case 'created': + case 'changed': + case 'deleted': + if (isPeoplePath(e.path)) setRefreshTick((t) => t + 1) + break + case 'moved': + if (isPeoplePath(e.from) || isPeoplePath(e.to)) setRefreshTick((t) => t + 1) + break + case 'bulkChanged': + if (!e.paths || e.paths.some(isPeoplePath)) setRefreshTick((t) => t + 1) + break + } + }) + return cleanup + }, []) + + if (!prep || prep.attendees.length === 0) return null + + const matched = prep.attendees.filter((a) => a.note) + const unmatched = prep.attendees.filter((a) => !a.note) + + return ( +
+ {prep.prepNote && prep.prepNote.brief ? ( +
+ + {prep.prepNote.brief} + + +
+ ) : null} +
+ + People +
+ {matched.map((att, idx) => ( + + ))} + + {prep.organizations.length > 0 ? ( + <> +
+ + {prep.organizations.length === 1 ? 'Company' : 'Companies'} + +
+ {prep.organizations.map((org) => ( + + ))} + + ) : null} +
+ ) +} + +function UpcomingEvents({ onOpenNote }: { onOpenNote: (path: string) => void }) { const [events, setEvents] = useState([]) const [loading, setLoading] = useState(true) const [error, setError] = useState(null) @@ -487,6 +692,20 @@ function UpcomingEvents() { return selectVisibleDays(window) }, [events]) + // The next meeting that's worth prepping for — soonest timed event with at + // least one other attendee that hasn't ended. Its row gets inline prep. + // `events` is sorted (all-day first, then by start), so `find` returns it. + const prepEventId = useMemo(() => { + const nowMs = Date.now() + const candidate = events.find((ev) => { + if (ev.isAllDay) return false + if (ev.attendees.every((a) => a.self)) return false + const endMs = ev.end ? ev.end.getTime() : ev.start.getTime() + 30 * 60 * 1000 + return endMs > nowMs + }) + return candidate?.id ?? null + }, [events]) + const totalVisible = visibleDays.reduce((s, d) => s + d.events.length, 0) const now = new Date() const todayKey = localDateKey(now) @@ -532,6 +751,8 @@ function UpcomingEvents() { key={day.dateKey} day={day} isToday={day.dateKey === todayKey} + prepEventId={prepEventId} + onOpenNote={onOpenNote} /> ))} @@ -542,7 +763,7 @@ function UpcomingEvents() { ) } -function UpcomingDayCard({ day, isToday }: { day: DayGroup; isToday: boolean }) { +function UpcomingDayCard({ day, isToday, prepEventId, onOpenNote }: { day: DayGroup; isToday: boolean; prepEventId: string | null; onOpenNote: (path: string) => void }) { const dayNum = day.date.getDate() const month = day.date.toLocaleDateString([], { month: 'short' }) const weekday = day.date.toLocaleDateString([], { weekday: 'short' }) @@ -573,7 +794,13 @@ function UpcomingDayCard({ day, isToday }: { day: DayGroup; isToday: boolean }) ) : ( day.events.map((ev, idx) => ( - + )) )} @@ -588,14 +815,20 @@ function NowBadge() { ) } -function UpcomingEventItem({ event, isLast }: { event: UpcomingEvent; isLast: boolean }) { +function UpcomingEventItem({ event, isLast, isPrepTarget, onOpenNote }: { event: UpcomingEvent; isLast: boolean; isPrepTarget: boolean; onOpenNote: (path: string) => void }) { const [open, setOpen] = useState(false) + // The next meeting auto-expands its prep; any other meeting with attendees + // can be expanded on demand via the Prep toggle (resolves lazily on open). + const prepEligible = !event.isAllDay && event.attendees.some((a) => !a.self) + const [prepOpen, setPrepOpen] = useState(isPrepTarget) + const showPrep = prepEligible && prepOpen const isNow = isEventNow(event) const platform = meetingPlatformLabel(event.conferenceLink) const subtitle = platform ?? event.location const titleAndLocation = event.location ? `${event.summary} · ${event.location}` : event.summary return ( +
@@ -625,7 +858,24 @@ function UpcomingEventItem({ event, isLast }: { event: UpcomingEvent; isLast: bo ) : null} -
+
+ {prepEligible ? ( + + ) : null} {event.conferenceLink ? ( triggerMeetingCapture(event, true)} @@ -647,6 +897,8 @@ function UpcomingEventItem({ event, isLast }: { event: UpcomingEvent; isLast: bo setOpen(false)} /> + {showPrep ? : null} +
) } @@ -917,6 +1169,9 @@ export function MeetingsView({ onOpenNote, onTakeMeetingNotes, meetingState, mee const rows = entries .filter((entry) => entry.kind === 'file' && entry.name.endsWith('.md')) + // Generated prep notes live under Meetings/prep/ — they're upcoming + // prep, not past meeting notes, so keep them out of this table. + .filter((entry) => !entry.path.startsWith(`${MEETINGS_ROOT}/prep/`)) .map((entry) => { const relative = entry.path.slice(`${MEETINGS_ROOT}/`.length) const parts = relative.split('/') @@ -1017,7 +1272,7 @@ export function MeetingsView({ onOpenNote, onTakeMeetingNotes, meetingState, mee

- +
{loading ? (
diff --git a/apps/x/packages/core/src/analytics/use_case.ts b/apps/x/packages/core/src/analytics/use_case.ts index a865a7cf..031a53fe 100644 --- a/apps/x/packages/core/src/analytics/use_case.ts +++ b/apps/x/packages/core/src/analytics/use_case.ts @@ -1,6 +1,6 @@ import { AsyncLocalStorage } from 'node:async_hooks'; -export type UseCase = 'copilot_chat' | 'live_note_agent' | 'background_task_agent' | 'meeting_note' | 'knowledge_sync' | 'code_session'; +export type UseCase = 'copilot_chat' | 'live_note_agent' | 'background_task_agent' | 'meeting_note' | 'meeting_prep' | 'knowledge_sync' | 'code_session'; export interface UseCaseContext { useCase: UseCase; diff --git a/apps/x/packages/core/src/knowledge/knowledge_index.ts b/apps/x/packages/core/src/knowledge/knowledge_index.ts index 2df46ca3..0d2ef435 100644 --- a/apps/x/packages/core/src/knowledge/knowledge_index.ts +++ b/apps/x/packages/core/src/knowledge/knowledge_index.ts @@ -73,8 +73,10 @@ export interface KnowledgeIndex { * Looks for patterns like **Field:** value or **Field:** [[Link]] */ function extractField(content: string, fieldName: string): string | undefined { - // Match **Field:** value (handles [[links]] and plain text) - const pattern = new RegExp(`\\*\\*${fieldName}:\\*\\*\\s*(.+?)(?:\\n|$)`, 'i'); + // Match **Field:** value (handles [[links]] and plain text). Only consume + // spaces/tabs after the label — NOT newlines — so an empty field returns + // undefined instead of bleeding the next line's value into it. + const pattern = new RegExp(`\\*\\*${fieldName}:\\*\\*[ \\t]*(.+?)(?:\\r?\\n|$)`, 'i'); const match = content.match(pattern); if (match) { let value = match[1].trim(); @@ -275,6 +277,34 @@ export function buildKnowledgeIndex(): KnowledgeIndex { return index; } +// --------------------------------------------------------------------------- +// Cached access +// --------------------------------------------------------------------------- +// buildKnowledgeIndex() does a synchronous recursive scan + read of the whole +// knowledge dir, so callers that run often (e.g. meeting prep) should use the +// cached accessor and rely on invalidateKnowledgeIndex() being called whenever +// a knowledge file changes (wired to the workspace watcher in the main process). + +let cachedIndex: KnowledgeIndex | null = null; + +/** + * Return the knowledge index, building it once and reusing it until invalidated. + */ +export function getKnowledgeIndex(): KnowledgeIndex { + if (!cachedIndex) { + cachedIndex = buildKnowledgeIndex(); + } + return cachedIndex; +} + +/** + * Drop the cached index so the next getKnowledgeIndex() rebuilds it. Call this + * whenever a file under knowledge/ is created, changed, moved, or deleted. + */ +export function invalidateKnowledgeIndex(): void { + cachedIndex = null; +} + /** * Format the index as a string for inclusion in agent prompts */ diff --git a/apps/x/packages/core/src/knowledge/meeting_prep.ts b/apps/x/packages/core/src/knowledge/meeting_prep.ts new file mode 100644 index 00000000..34e0afb6 --- /dev/null +++ b/apps/x/packages/core/src/knowledge/meeting_prep.ts @@ -0,0 +1,230 @@ +import fs from 'node:fs/promises'; +import path from 'node:path'; +import { WorkDir } from '../config/config.js'; +import { getKnowledgeIndex } from './knowledge_index.js'; + +const KNOWLEDGE_DIR = path.join(WorkDir, 'knowledge'); + +/** + * A calendar attendee as it arrives from the renderer (sourced from the + * Google Calendar event's `attendees[]`). + */ +export interface MeetingPrepAttendee { + email?: string; + displayName?: string; + self?: boolean; +} + +/** + * The note we resolved for a matched attendee. `markdown` is the full note + * body so the renderer can render it as-is — no LLM, no summarisation. + */ +export interface MeetingPrepNote { + /** Workspace-relative path, e.g. "knowledge/People/Sarah Chen.md". */ + path: string; + name: string; + role?: string; + organization?: string; + markdown: string; +} + +/** + * One attendee after resolution. `note` is set when we found a person note, + * `null` otherwise (the "no note yet" case the UI offers to create). + */ +export interface MeetingPrepResolved { + /** Best display label — the note name, else displayName, else email. */ + label: string; + email?: string; + displayName?: string; + note: MeetingPrepNote | null; +} + +/** An organization note relevant to the meeting (resolved from attendee domains). */ +export interface MeetingPrepOrg { + path: string; + name: string; + markdown: string; +} + +export interface MeetingPrepResult { + /** Resolved attendees, matched ones first (notes-first ordering). */ + attendees: MeetingPrepResolved[]; + /** Distinct external organizations in the meeting that have a note. */ + organizations: MeetingPrepOrg[]; + /** How many have a note vs. not — convenience for the UI header. */ + matchedCount: number; + unmatchedCount: number; +} + +function norm(value: string | undefined): string { + return (value ?? '').trim().toLowerCase(); +} + +/** Normalize a display name for matching: drop parenthetical suffixes like + * "(via Google Calendar)" / "(Guest)" that calendars sometimes append. */ +function normName(value: string | undefined): string { + return norm((value ?? '').replace(/\s*\([^)]*\)\s*$/g, '')); +} + +/** Lowercased domain part of an email, or '' if there isn't one. */ +function domainOf(email: string | undefined): string { + const e = norm(email); + const at = e.lastIndexOf('@'); + return at >= 0 ? e.slice(at + 1) : ''; +} + +/** Tidy a field value for display: strip a leading knowledge folder segment so + * a link target like "Organizations/Rowboat Labs" reads as "Rowboat Labs". */ +function displayRef(value: string | undefined): string | undefined { + if (!value) return undefined; + const m = value.match(/^(?:People|Organizations|Projects|Topics)\/(.+)$/i); + return (m ? m[1] : value).trim() || undefined; +} + +/** + * Resolve a meeting's attendees against the knowledge base, returning each + * attendee's existing person note (or null). Deterministic: email-exact match + * first, then an unambiguous name/alias match on the display name. + */ +export async function resolveMeetingPrep( + attendees: MeetingPrepAttendee[], +): Promise { + const index = getKnowledgeIndex(); + + // email -> person (first wins; emails are effectively unique). + const byEmail = new Map(); + // normalized name/alias -> people that carry it (for ambiguity checks). + const byName = new Map(); + + for (const person of index.people) { + const email = norm(person.email); + if (email && !byEmail.has(email)) byEmail.set(email, person); + + for (const key of [person.name, ...person.aliases]) { + const nk = normName(key); + if (!nk) continue; + const list = byName.get(nk) ?? []; + list.push(person); + byName.set(nk, list); + } + } + + // domain -> organization, and name/alias -> organization, for resolving the + // companies in the meeting from attendee email domains. + const orgByDomain = new Map(); + const orgByName = new Map(); + for (const org of index.organizations) { + const d = norm(org.domain); + if (d && !orgByDomain.has(d)) orgByDomain.set(d, org); + for (const key of [org.name, ...org.aliases]) { + const nk = norm(key); + if (nk && !orgByName.has(nk)) orgByName.set(nk, org); + } + } + + // The user's own domain (from the self attendee) is "internal" — we never + // surface the user's own company as meeting context. + const selfDomain = domainOf(attendees.find((a) => a.self)?.email); + + // Cache note reads so a person listed under multiple keys is read once. + const noteCache = new Map(); + const readNote = async (person: (typeof index.people)[number]): Promise => { + if (noteCache.has(person.file)) return noteCache.get(person.file)!; + let note: MeetingPrepNote | null = null; + try { + const markdown = await fs.readFile(path.join(KNOWLEDGE_DIR, person.file), 'utf-8'); + note = { + path: path.posix.join('knowledge', person.file.split(path.sep).join('/')), + name: person.name, + role: displayRef(person.role), + organization: displayRef(person.organization), + markdown, + }; + } catch { + // Indexed file vanished between index build and read — treat as no note. + note = null; + } + noteCache.set(person.file, note); + return note; + }; + + const resolved: MeetingPrepResolved[] = []; + const seenFiles = new Set(); + // Distinct external orgs to surface, keyed by note file. + const orgEntries = new Map(); + + for (const attendee of attendees) { + if (attendee.self) continue; + + const email = norm(attendee.email); + const displayName = normName(attendee.displayName); + const domain = domainOf(attendee.email); + + let person = email ? byEmail.get(email) : undefined; + if (!person && displayName) { + const candidates = byName.get(displayName); + // Only a single, unambiguous hit counts — never guess between two + // people who happen to share a name. + if (candidates && candidates.length === 1) person = candidates[0]; + } + + // Resolve the attendee's company — but only for external domains, so an + // internal standup doesn't surface the user's own org note. Prefer a + // domain match; fall back to the matched person's Organization field. + if (domain && domain !== selfDomain) { + const org = + orgByDomain.get(domain) ?? + (person?.organization ? orgByName.get(norm(person.organization)) : undefined); + if (org) orgEntries.set(org.file, org); + } + + const label = + person?.name || + attendee.displayName?.trim() || + attendee.email?.trim() || + 'Unknown'; + + const note = person ? await readNote(person) : null; + // Dedupe: the same person can appear once even if the calendar lists + // them twice (e.g. organizer + attendee). + if (note && seenFiles.has(note.path)) continue; + if (note) seenFiles.add(note.path); + + resolved.push({ + label, + email: attendee.email?.trim() || undefined, + displayName: attendee.displayName?.trim() || undefined, + note, + }); + } + + // Notes-first ordering; stable within each group. + resolved.sort((a, b) => { + if (Boolean(a.note) === Boolean(b.note)) return 0; + return a.note ? -1 : 1; + }); + + // Read the resolved org notes (skip any whose file vanished). + const organizations: MeetingPrepOrg[] = []; + for (const org of orgEntries.values()) { + try { + const markdown = await fs.readFile(path.join(KNOWLEDGE_DIR, org.file), 'utf-8'); + organizations.push({ + path: path.posix.join('knowledge', org.file.split(path.sep).join('/')), + name: org.name, + markdown, + }); + } catch { + // Indexed file vanished between index build and read — skip it. + } + } + + const matchedCount = resolved.filter((a) => a.note).length; + return { + attendees: resolved, + organizations, + matchedCount, + unmatchedCount: resolved.length - matchedCount, + }; +} diff --git a/apps/x/packages/core/src/knowledge/meeting_prep_brief.ts b/apps/x/packages/core/src/knowledge/meeting_prep_brief.ts new file mode 100644 index 00000000..45cb26e0 --- /dev/null +++ b/apps/x/packages/core/src/knowledge/meeting_prep_brief.ts @@ -0,0 +1,307 @@ +import fs from 'node:fs/promises'; +import path from 'node:path'; +import { generateText } from 'ai'; +import { WorkDir } from '../config/config.js'; +import { createProvider } from '../models/models.js'; +import { getDefaultModelAndProvider, getMeetingNotesModel, resolveProviderConfig } from '../models/defaults.js'; +import { captureLlmUsage } from '../analytics/usage.js'; +import { withUseCase } from '../analytics/use_case.js'; +import { parseFrontmatter } from '../application/lib/parse-frontmatter.js'; +import { resolveMeetingPrep, type MeetingPrepResult } from './meeting_prep.js'; + +const MEETINGS_DIR = path.join(WorkDir, 'knowledge', 'Meetings'); +const PREP_DIR = path.join(MEETINGS_DIR, 'prep'); + +/** The bits of a Google Calendar event we use for prep. */ +interface CalendarEvent { + id?: string; + summary?: string; + description?: string; + status?: string; + recurringEventId?: string; + start?: { dateTime?: string; date?: string }; + attendees?: Array<{ email?: string; displayName?: string; self?: boolean; responseStatus?: string }>; +} + +export interface PrepNoteResult { + /** Workspace-relative path of the written note. */ + path: string; +} + +function norm(s: string | undefined): string { + return (s ?? '').trim().toLowerCase(); +} + +function slugify(s: string): string { + return (s || 'meeting') + .toLowerCase() + .replace(/[^a-z0-9]+/g, '-') + .replace(/^-+|-+$/g, '') + .slice(0, 60) || 'meeting'; +} + +/** Local YYYY-MM-DD for the event's start. */ +function eventDateKey(event: CalendarEvent): string { + const iso = event.start?.dateTime ?? event.start?.date ?? ''; + const d = iso ? new Date(iso) : null; + if (!d || Number.isNaN(d.getTime())) return 'undated'; + const y = d.getFullYear(); + const m = String(d.getMonth() + 1).padStart(2, '0'); + const day = String(d.getDate()).padStart(2, '0'); + return `${y}-${m}-${day}`; +} + +/** Pull a "## Heading" section's body (until the next "## " or end). */ +function extractSection(markdown: string, heading: string): string { + const headRe = new RegExp(`^##\\s+${heading}\\s*$`, 'i'); + const out: string[] = []; + let capturing = false; + for (const line of markdown.split('\n')) { + if (capturing) { + if (/^##\s/.test(line)) break; + out.push(line); + } else if (headRe.test(line)) { + capturing = true; + } + } + return out.join('\n').trim(); +} + +interface PriorNote { + file: string; // workspace-relative + title: string; + date: string; + actionItems: string; + body: string; +} + +/** + * Find the most recent prior meeting note for this series. We match by title + * resemblance to the event summary (notes don't yet store an event id), and + * only consider notes dated before the meeting. + */ +async function findLastMeetingNote(event: CalendarEvent): Promise { + const summaryNorm = norm(event.summary); + if (!summaryNorm) return null; + const meetingDate = eventDateKey(event); + + let entries: string[] = []; + try { + entries = (await fs.readdir(MEETINGS_DIR, { recursive: true })) + .filter((p) => p.endsWith('.md')); + } catch { + return null; + } + + const candidates: PriorNote[] = []; + for (const rel of entries) { + // Skip our own generated prep notes. + if (rel.startsWith('prep/') || rel.startsWith(`prep${path.sep}`)) continue; + let raw: string; + try { + raw = await fs.readFile(path.join(MEETINGS_DIR, rel), 'utf-8'); + } catch { + continue; + } + const { frontmatter, content } = parseFrontmatter(raw); + const fm = (frontmatter ?? {}) as Record; + const title = String(fm.title ?? (content.match(/^#\s+(.+)$/m)?.[1] ?? '')).trim(); + const date = String(fm.date ?? '').trim(); + const titleNorm = norm(title); + // Series match: the event summary appears in the note title (e.g. + // "standup" within "Eng Standup — 2026-06-18"). + if (!titleNorm || !titleNorm.includes(summaryNorm)) continue; + // Only prior instances. + if (date && meetingDate !== 'undated' && date >= meetingDate) continue; + candidates.push({ + file: path.posix.join('knowledge', 'Meetings', rel.split(path.sep).join('/')), + title, + date, + actionItems: extractSection(content, 'Action items'), + body: content, + }); + } + + if (candidates.length === 0) return null; + // Most recent by date (notes without a date sort last). + candidates.sort((a, b) => (b.date || '').localeCompare(a.date || '')); + return candidates[0]; +} + +/** True when this looks like a recurring meeting we have history for. */ +function isRecurring(event: CalendarEvent, prior: PriorNote | null): boolean { + return Boolean(event.recurringEventId) && prior !== null; +} + +/** Assemble the deterministic prep context for an event. */ +async function assembleContext(event: CalendarEvent): Promise<{ + roster: MeetingPrepResult; + prior: PriorNote | null; + recurring: boolean; + agenda: string; +}> { + const attendees = (event.attendees ?? []).map((a) => ({ + email: a.email, + displayName: a.displayName, + self: a.self, + })); + const roster = await resolveMeetingPrep(attendees); + const prior = await findLastMeetingNote(event); + return { + roster, + prior, + recurring: isRecurring(event, prior), + agenda: (event.description ?? '').trim(), + }; +} + +const BRIEF_SYSTEM = `You write a short, concrete "what matters for this meeting" brief. +Rules: +- Use ONLY the context provided. Never invent facts, names, or commitments. +- 3-5 bullet points, one line each. No preamble, no headings, no sign-off. +- Lead with what the user should focus on or decide. Reference open items and + prior commitments by name where the context supplies them. +- If the context is thin, say so in one line rather than padding.`; + +/** Generate the "what matters" brief via the user's configured model. */ +async function generateBrief(event: CalendarEvent, ctx: Awaited>): Promise { + const parts: string[] = [`Meeting: ${event.summary || '(untitled)'}`]; + if (ctx.agenda) parts.push(`Agenda:\n${ctx.agenda}`); + if (ctx.prior?.actionItems) parts.push(`Action items from last time (${ctx.prior.date || 'prior'}):\n${ctx.prior.actionItems}`); + const attendeeLines = ctx.roster.attendees.map((a) => { + if (!a.note) return `- ${a.label} (no note)`; + const sub = [a.note.role, a.note.organization].filter(Boolean).join(', '); + return `- ${a.note.name}${sub ? ` — ${sub}` : ''}`; + }); + if (attendeeLines.length) parts.push(`Attendees:\n${attendeeLines.join('\n')}`); + + const modelId = await getMeetingNotesModel(); + const { provider: providerName } = await getDefaultModelAndProvider(); + const providerConfig = await resolveProviderConfig(providerName); + const model = createProvider(providerConfig).languageModel(modelId); + + const result = await withUseCase({ useCase: 'meeting_prep' }, () => generateText({ + model, + system: BRIEF_SYSTEM, + prompt: parts.join('\n\n'), + })); + captureLlmUsage({ useCase: 'meeting_prep', model: modelId, provider: providerName, usage: result.usage }); + return result.text.trim(); +} + +/** Render the prep note's markdown body (brief is optional). */ +function renderPrepNote(event: CalendarEvent, ctx: Awaited>, brief: string, generatedAt: string): string { + const fm = [ + '---', + 'source: meeting-prep', + `title: "Prep: ${(event.summary || 'Meeting').replace(/"/g, "'")}"`, + `meetingDate: "${eventDateKey(event)}"`, + event.id ? `eventId: "${event.id}"` : null, + event.recurringEventId ? `recurringEventId: "${event.recurringEventId}"` : null, + `generatedAt: "${generatedAt}"`, + '---', + '', + ].filter((l) => l !== null).join('\n'); + + const lines: string[] = [`# Prep: ${event.summary || 'Meeting'}`, '']; + + if (brief) { + lines.push('## What matters', '', brief, ''); + } + + // Adaptive ordering: recurring → recap first; new → agenda first. + const recapBlock = ctx.prior + ? ['## Last time', '', ctx.prior.actionItems + ? ctx.prior.actionItems + : `See [[${ctx.prior.title}]].`, ''] + : []; + const agendaBlock = ctx.agenda ? ['## Agenda', '', ctx.agenda, ''] : []; + if (ctx.recurring) { + lines.push(...recapBlock, ...agendaBlock); + } else { + lines.push(...agendaBlock, ...recapBlock); + } + + // Roster — every attendee, linking to their note when we have one. + lines.push('## Who’s coming', ''); + for (const a of ctx.roster.attendees) { + if (a.note) { + const sub = [a.note.role, a.note.organization].filter(Boolean).join(', '); + lines.push(`- [[${a.note.name}]]${sub ? ` — ${sub}` : ''}`); + } else { + lines.push(`- ${a.label} _(no note yet)_`); + } + } + lines.push(''); + + if (ctx.roster.organizations.length > 0) { + lines.push('## Companies', ''); + for (const org of ctx.roster.organizations) lines.push(`- [[${org.name}]]`); + lines.push(''); + } + + return fm + lines.join('\n').trimEnd() + '\n'; +} + +/** + * Generate and write the prep note for a calendar event. Returns the note path, + * or null when there's nothing to prep (no other attendees). The AI brief is + * best-effort — if no model is configured the note is still written with the + * deterministic sections. + */ +export async function generateAndWritePrep(eventJson: string): Promise { + const event = JSON.parse(eventJson) as CalendarEvent; + if (event.status === 'cancelled') return null; + if (!(event.attendees ?? []).some((a) => !a.self)) return null; // nobody else + + const ctx = await assembleContext(event); + if (ctx.roster.attendees.length === 0) return null; + + let brief = ''; + try { + brief = await generateBrief(event, ctx); + } catch (err) { + console.error('[MeetingPrep] brief generation failed:', err); + } + + const generatedAt = new Date().toISOString(); + const body = renderPrepNote(event, ctx, brief, generatedAt); + + await fs.mkdir(PREP_DIR, { recursive: true }); + const fileName = `${slugify(event.summary || 'meeting')}-${eventDateKey(event)}.md`; + const absPath = path.join(PREP_DIR, fileName); + await fs.writeFile(absPath, body, 'utf-8'); + + return { path: path.posix.join('knowledge', 'Meetings', 'prep', fileName) }; +} + +/** + * Find the pre-generated prep note for a calendar event (matched by the + * `eventId` stamped in frontmatter) and return its path + "What matters" brief. + * Returns null when no prep has been generated yet. + */ +export async function readPrepNoteForEvent(eventId: string): Promise<{ path: string; brief: string } | null> { + if (!eventId) return null; + let files: string[]; + try { + files = (await fs.readdir(PREP_DIR)).filter((f) => f.endsWith('.md')); + } catch { + return null; + } + for (const f of files) { + let raw: string; + try { + raw = await fs.readFile(path.join(PREP_DIR, f), 'utf-8'); + } catch { + continue; + } + const { frontmatter, content } = parseFrontmatter(raw); + const fm = (frontmatter ?? {}) as Record; + if (String(fm.eventId ?? '') !== eventId) continue; + return { + path: path.posix.join('knowledge', 'Meetings', 'prep', f), + brief: extractSection(content, 'What matters'), + }; + } + return null; +} diff --git a/apps/x/packages/core/src/knowledge/meeting_prep_scheduler.ts b/apps/x/packages/core/src/knowledge/meeting_prep_scheduler.ts new file mode 100644 index 00000000..b9b31ba3 --- /dev/null +++ b/apps/x/packages/core/src/knowledge/meeting_prep_scheduler.ts @@ -0,0 +1,158 @@ +import path from "node:path"; +import fs from "node:fs/promises"; +import type { Dirent } from "node:fs"; +import { WorkDir } from "../config/config.js"; +import { generateAndWritePrep } from "./meeting_prep_brief.js"; + +// Generate prep up to 6h before a meeting. We tick every 5 minutes and scan the +// synced calendar — a calendar-aware loop fits "N hours before each meeting" +// better than a fixed cron, and re-reading the calendar each tick means moves +// and cancellations are picked up automatically. +const TICK_INTERVAL_MS = 15 * 60_000; +const PREP_LEAD_MS = 6 * 60 * 60_000; +// Drop state entries older than 24h so the file doesn't grow forever. +const STATE_TTL_MS = 24 * 60 * 60 * 1000; + +const CALENDAR_SYNC_DIR = path.join(WorkDir, "calendar_sync"); +const STATE_FILE = path.join(WorkDir, "meeting_prep_state.json"); + +interface PrepState { + preppedEventIds: Record; +} + +interface CalendarEvent { + id?: string; + summary?: string; + status?: string; + start?: { dateTime?: string; date?: string }; + attendees?: Array<{ self?: boolean; responseStatus?: string }>; +} + +async function loadState(): Promise { + try { + const raw = await fs.readFile(STATE_FILE, "utf-8"); + const parsed = JSON.parse(raw); + if (parsed && typeof parsed === "object" && parsed.preppedEventIds) { + return parsed as PrepState; + } + } catch { + // No state file yet, or corrupt — start fresh. + } + return { preppedEventIds: {} }; +} + +async function saveState(state: PrepState): Promise { + // Write to a sibling tmp file then rename so a mid-write crash can't leave + // the JSON corrupt. + const tmp = `${STATE_FILE}.tmp`; + await fs.writeFile(tmp, JSON.stringify(state, null, 2), "utf-8"); + await fs.rename(tmp, STATE_FILE); +} + +function gcState(state: PrepState): PrepState { + const cutoff = Date.now() - STATE_TTL_MS; + const fresh: PrepState["preppedEventIds"] = {}; + for (const [id, entry] of Object.entries(state.preppedEventIds)) { + const ts = Date.parse(entry.preppedAt); + if (Number.isFinite(ts) && ts >= cutoff) fresh[id] = entry; + } + return { preppedEventIds: fresh }; +} + +function isAllDay(event: CalendarEvent): boolean { + return !event.start?.dateTime; +} + +function isDeclinedBySelf(event: CalendarEvent): boolean { + return event.attendees?.find((a) => a.self)?.responseStatus === "declined"; +} + +async function tick(state: PrepState): Promise<{ state: PrepState; dirty: boolean }> { + let entries: Dirent[]; + try { + entries = await fs.readdir(CALENDAR_SYNC_DIR, { withFileTypes: true }); + } catch { + return { state, dirty: false }; + } + + const now = Date.now(); + let dirty = false; + + for (const entry of entries) { + if (!entry.isFile() || !entry.name.endsWith(".json")) continue; + if (entry.name.startsWith("sync_state")) continue; + + const eventId = entry.name.replace(/\.json$/, ""); + if (state.preppedEventIds[eventId]) continue; + + let raw: string; + try { + raw = await fs.readFile(path.join(CALENDAR_SYNC_DIR, entry.name), "utf-8"); + } catch { + continue; + } + let event: CalendarEvent; + try { + event = JSON.parse(raw); + } catch { + continue; + } + + if (event.status === "cancelled") continue; + if (isAllDay(event)) continue; + if (isDeclinedBySelf(event)) continue; + if (!(event.attendees ?? []).some((a) => !a.self)) continue; // nobody else + + const startStr = event.start?.dateTime; + if (!startStr) continue; + const startMs = Date.parse(startStr); + if (!Number.isFinite(startMs)) continue; + + const msUntilStart = startMs - now; + if (msUntilStart > PREP_LEAD_MS) continue; // too far out + if (msUntilStart <= 0) continue; // already started — too late to pre-generate + + try { + const result = await generateAndWritePrep(raw); + if (result) { + console.log(`[MeetingPrep] generated prep for "${event.summary ?? eventId}" → ${result.path}`); + } + } catch (err) { + console.error(`[MeetingPrep] prep generation failed for ${eventId}:`, err); + continue; // leave unmarked so we retry next tick + } + + state.preppedEventIds[eventId] = { + preppedAt: new Date().toISOString(), + startTime: startStr, + }; + dirty = true; + } + + return { state, dirty }; +} + +export async function init(): Promise { + console.log("[MeetingPrep] starting meeting prep scheduler"); + console.log(`[MeetingPrep] tick every ${TICK_INTERVAL_MS / 60_000}m, lead ${PREP_LEAD_MS / 3_600_000}h`); + + let state = gcState(await loadState()); + + while (true) { + try { + const result = await tick(state); + state = result.state; + if (result.dirty) { + state = gcState(state); + try { + await saveState(state); + } catch (err) { + console.error("[MeetingPrep] failed to save state:", err); + } + } + } catch (err) { + console.error("[MeetingPrep] tick failed:", err); + } + await new Promise((resolve) => setTimeout(resolve, TICK_INTERVAL_MS)); + } +} diff --git a/apps/x/packages/shared/src/ipc.ts b/apps/x/packages/shared/src/ipc.ts index 25ad6a74..067adb4f 100644 --- a/apps/x/packages/shared/src/ipc.ts +++ b/apps/x/packages/shared/src/ipc.ts @@ -1248,6 +1248,47 @@ const ipcSchemas = { notes: z.string(), }), }, + // Resolve a meeting's attendees against the knowledge base — returns each + // attendee's existing person note (or null). Deterministic, no LLM; powers + // the ambient "Next up" prep card. + 'meeting-prep:resolve': { + req: z.object({ + attendees: z.array(z.object({ + email: z.string().optional(), + displayName: z.string().optional(), + self: z.boolean().optional(), + })), + // When provided, the response includes any pre-generated prep note for + // this calendar event (matched by the eventId stamped in frontmatter). + eventId: z.string().optional(), + }), + res: z.object({ + attendees: z.array(z.object({ + label: z.string(), + email: z.string().optional(), + displayName: z.string().optional(), + note: z.object({ + path: z.string(), + name: z.string(), + role: z.string().optional(), + organization: z.string().optional(), + markdown: z.string(), + }).nullable(), + })), + organizations: z.array(z.object({ + path: z.string(), + name: z.string(), + markdown: z.string(), + })), + // The pre-generated prep note (brief + path), if one exists for eventId. + prepNote: z.object({ + path: z.string(), + brief: z.string(), + }).nullable(), + matchedCount: z.number().int().nonnegative(), + unmatchedCount: z.number().int().nonnegative(), + }), + }, // Inline task schedule classification 'export:note': { req: z.object({ From 4800a86fbb0ca85aaeab4050336e1ad207ed9666 Mon Sep 17 00:00:00 2001 From: gagan Date: Thu, 2 Jul 2026 00:44:28 +0530 Subject: [PATCH 02/24] chore: remove meeting-prep PLAN.md from repo (keep local-only) (#652) --- .gitignore | 3 ++ PLAN.md | 111 ----------------------------------------------------- 2 files changed, 3 insertions(+), 111 deletions(-) delete mode 100644 PLAN.md diff --git a/.gitignore b/.gitignore index 086ea0b5..c9eee447 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,6 @@ data/ .venv/ .claude/ + +# Local-only meeting-prep planning doc +/PLAN.md diff --git a/PLAN.md b/PLAN.md deleted file mode 100644 index 05d6cd1b..00000000 --- a/PLAN.md +++ /dev/null @@ -1,111 +0,0 @@ -# Meeting Prep — "Next up" prep card - -## Goal - -When a meeting is imminent, surface who will be there by pulling up each -attendee's existing `person.md` from the knowledge base and showing it — no -click, no LLM in the hot path. An ambient card that's just *ready* when you -look. - -This is a deliberately light, deterministic surface. It does **not** replace -the two existing meeting-prep paths: -- the chat skill (`assistant/skills/meeting-prep/`) — LLM brief on demand -- the pre-built automation (`pre_built/meeting-prep.md`) — LLM brief files - -It complements them with an instant, no-LLM "who's in this meeting" view. - -## Decisions (settled) - -| Axis | Choice | -|------|--------| -| Surface | **Ambient card**, no notification. Pinned at top of Meetings view. | -| Activation | Next eligible meeting is **≤ 30 min away**. Outside that → no card. | -| Depth | **Just show `person.md`** — render the note, no LLM synthesis. | -| Who | **Everyone incl. teammates.** Skip self / all-day / declined / cancelled. | -| Sparse meetings | **Notes-first**: attendees with notes shown prominently; the rest collapse into "N others — no notes yet" → expand to create. | -| No-note action | **Offer to create** a note for that attendee. | - -## Resolution rules - -For each attendee (excluding `self`): -1. **Email exact match** (case-insensitive) against `KnowledgeIndex.people[].email` — reliable. -2. Else **name/alias match** on `displayName` → `people[].name` / `people[].aliases`, - only when it's a **single unambiguous hit**. -3. Matched → read `person.md`, return `{ name, role, org, path, markdown }`. -4. Unmatched → returned as a "no note" attendee (name + email). - -Org notes (`Organizations/{Org}.md`) are **deferred** — `person.md` already -carries Org/Role inline, enough for v1. - -## Architecture - -| Piece | Location | Notes | -|-------|----------|-------| -| Attendee → note resolver | **new IPC** `meeting-prep:resolve` in `apps/main/src/ipc.ts` | Input: attendee list (`{email?, displayName?, self?}[]`). Builds/uses `buildKnowledgeIndex()`, resolves per rules above, reads matched notes, returns structured payload. Deterministic + fast. | -| Shared types | `packages/shared/src/` | `MeetingPrepAttendee`, `MeetingPrepResult` request/response shapes + IPC channel typing. | -| "Next up" card UI | **new component** rendered above `UpcomingEvents` in `meetings-view.tsx` | Picks next event ≤30 min out, calls IPC, renders notes-first + collapsed others. Reuses existing markdown rendering. | -| Create-note action | reuse the note-creation agent (only LLM touch, user-initiated) | v1: open a prefilled Copilot chat to create the note (visible/steerable) rather than firing silently. | - -No new background loop — the renderer already polls `calendar_sync/` every -minute, so the card stays current for free. - -## Build slices - -1. **Resolver IPC + shared types** — `meeting-prep:resolve`, wired through - preload. Unit-testable in isolation. ← first slice -2. **"Next up" card UI** — component + 30-min gating + notes-first / collapsed - rendering, reading from the IPC. -3. **Create-note action** — prefilled Copilot chat for an unmatched attendee. - -## Out of scope (fast-follow candidates) - -- Notification variant (extend `notify_calendar_meetings.ts` tick). -- Org-note surfacing. -- Configurable lead time (hardcode 30 min for v1). -- LLM-synthesized brief button on the card. - -## Phase 2 — Proactive, meeting-aware prep (6h ahead) - -Shift from a static contact viewer to meeting-aware prep, generated ahead of time. - -### Decisions -- **Trigger:** a calendar-driven tick (clone of `notify_calendar_meetings.ts`), NOT the - cron/bg-task engine — fire time is `start − 6h`, which varies per event, so a - calendar scan + state file is the right fit (self-healing on reschedule/cancel). -- **Storage:** write a real note → `knowledge/Meetings/prep/-.md` (openable/editable). -- **Notify:** none — silent background generation; appears in the Meetings view. -- **Adaptive by meeting type:** - - Recurring (`recurringEventId` + prior notes found) → lead with **"Last time" recap** - (prior instance's `## Action items` / summary), then roster. - - One-off / new → lead with **agenda** (event `description`), then roster + company. -- **Roster:** always list every attendee; ones with a note link to it, ones without - get **Create note**. -- **AI brief:** the only LLM step. Reuses the `summarizeMeeting` path exactly — - `generateText` against the user's configured model (`getMeetingNotesModel` / - `getDefaultModelAndProvider`), tagged `useCase: 'meeting_prep'`. Deterministic - parts (roster, recap, agenda) are assembled in code; the model only writes the - "what matters" summary. - -### Build slices -- **A. Core generator** (`meeting_prep_brief.ts`): assemble context (roster + recap + - agenda, meeting-type aware) → `generatePrepBrief()` (one model call) → render + - `writePrepNote()` to `knowledge/Meetings/prep/`. Frontmatter stamps `eventId` / - `recurringEventId` / `meetingDate` / `generatedAt`. -- **B. Scheduler** (`meeting_prep_scheduler.ts`): tick (~few min), scan `calendar_sync/`, - for events starting within 6h, not ended, ≥1 non-self attendee, not already prepped - (state file keyed by eventId) → generate + write. `init()` wired in `main.ts` beside - `initCalendarNotifications()`. -- **C. Card integration:** the inline prep card prefers the generated prep note when it - exists (instant), else falls back to the live deterministic resolve. - -### Open detail / enabler -- Notes don't store an event id today, so recap linking is a title/date heuristic. - Stamp `eventId`/`recurringEventId` into captured notes' frontmatter going forward so - future links are exact. - -## Verification - -- Resolver: feed known attendee emails → correct note match; unknown → no-note. -- Card: appears only inside the 30-min window; notes-first ordering; collapse - toggles; create action opens prefilled chat. -- `npm run deps && npm run lint` clean. From 1825202e6567d1c48ee14a902d0680b8c3e6fcf3 Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 07:45:53 +0530 Subject: [PATCH 03/24] docs(x): turn runtime + session layer design specs Turn runtime spec amended after review: context references with a context resolver, recovery that re-issues interrupted model calls and continues after interrupted sync tools, turn_failed.code with model-call-limit, default maxModelCalls 20. New session layer spec: append-only session JSONL, turn-file-first write ordering, in-memory write-through index with startup scan, reject-while-busy sendMessage, dedicated ask-human endpoint, deferred v2 shapes for queueing/steering/grants/compaction. Co-Authored-By: Claude Fable 5 --- apps/x/packages/core/docs/session-design.md | 561 +++++ .../packages/core/docs/turn-runtime-design.md | 1830 +++++++++++++++++ 2 files changed, 2391 insertions(+) create mode 100644 apps/x/packages/core/docs/session-design.md create mode 100644 apps/x/packages/core/docs/turn-runtime-design.md diff --git a/apps/x/packages/core/docs/session-design.md b/apps/x/packages/core/docs/session-design.md new file mode 100644 index 00000000..2694f9e9 --- /dev/null +++ b/apps/x/packages/core/docs/session-design.md @@ -0,0 +1,561 @@ +# Session Layer Technical Specification + +Status: design complete for the session layer v1. No implementation exists +yet. + +This document specifies the session layer that sits above the turn runtime +defined in `turn-runtime-design.md`. That document is assumed context; this +one does not restate turn semantics. + +## 1. Goals + +The session layer must: + +1. Own conversations composed of ordered turns. +2. Persist each session as one append-only JSONL file with the same + validation discipline as turn files. +3. Enforce one active turn per session. +4. Assemble each turn's context as a reference to the previous turn. +5. Maintain an in-memory index for listing, sorting, and filtering sessions, + updated write-through and rebuilt by scanning at startup. +6. Route external inputs — permission decisions, ask-human answers, async + tool results — to the correct turn through dedicated APIs. +7. Forward live turn events to the renderer over IPC. +8. Provide headless standalone turns outside any session. + +## 2. Non-goals (v1) + +- Queued user messages. The committed future shape is in section 12.1. +- Steering / mid-turn message injection (section 12.4). +- Session-scoped permission grants ("always allow for this chat"); every + applicable tool call prompts in v1 (section 12.2). +- Context compaction; a viable mechanism sketch is recorded in section 12.3. + V1 behavior on context overflow is the turn-level model failure. +- LLM auto-titling (section 12.6). +- A persisted index cache; startup always scans (section 12.5). +- Cross-process coordination. A single main process is enforced. +- Data migration from the current runs system. Old conversations are not + converted; the old code path remains readable until it is deleted. +- Session list pagination. The index is in-memory and shipped whole. + +## 3. Terminology + +A **session** is a durable, ordered chain of turns plus presentation +metadata (title). Conversation content lives exclusively in turn files; the +session file stores turn references with denormalized metadata. + +The **index** is an in-memory projection over all session files, used for +the session list UI. It is never a source of truth. + +A **standalone turn** is a turn with `sessionId: null`, created outside any +session by headless callers. Standalone turns do not appear in the index. + +## 4. Storage design + +### 4.1 File location + +Session files live under: + +```text +WorkDir/storage/sessions/YYYY/MM/DD/.jsonl +``` + +Session IDs come from the existing +`IMonotonicallyIncreasingIdGenerator`, and the repository derives the +date-partitioned path from the ID exactly as the turn repository does, +including format validation and path-traversal rejection. + +### 4.2 File rules + +Identical discipline to turn files: + +- The first line is always `session_created` with `schemaVersion: 1`. +- Every event contains `sessionId` and an ISO timestamp `ts`. +- Physical line order is authoritative. +- Reads validate every line strictly; any malformed line makes the session + corrupt; no truncation, repair, or skipping. +- Unknown schema versions and unknown event types fail loudly. Future + additive event types (queueing, grants, compaction) arrive as a schema + version bump; the reducer will accept old and new versions and write the + newest. +- Appends are awaited but not explicitly `fsync`ed. + +### 4.3 Repository contract + +```ts +interface ISessionRepo { + create(event: SessionCreated): Promise; + read(sessionId: string): Promise; + append(sessionId: string, events: SessionEvent[]): Promise; + withLock(sessionId: string, fn: () => Promise): Promise; + listSessionIds(): Promise; + delete(sessionId: string): Promise; +} +``` + +- `create` fails if the file exists. +- `listSessionIds` enumerates the partition directories for the startup + scan. +- `delete` removes the session file only. Turn files referenced by the + session are left in place as harmless orphans (see section 9, deletion). +- `withLock` is in-process per-session exclusion, mirroring the turn repo. + +## 5. Event schemas + +All session event schemas live in `@x/shared` alongside the turn schemas. + +```ts +interface BaseSessionEvent { + sessionId: string; + ts: string; +} + +interface SessionCreated extends BaseSessionEvent { + type: "session_created"; + schemaVersion: 1; + title?: string; +} + +interface SessionTurnAppended extends BaseSessionEvent { + type: "turn_appended"; + turnId: string; + sessionSeq: number; // 1-based position of the turn in the session + agentId: string; + model: ModelDescriptor; // resolved provider/model for the turn +} + +interface SessionTitleChanged extends BaseSessionEvent { + type: "title_changed"; + title: string; +} + +type SessionEvent = + | SessionCreated + | SessionTurnAppended + | SessionTitleChanged; +``` + +`turn_appended` deliberately denormalizes `agentId` and `model` from the +turn so the index can fold from session files without opening turn files. +The turn file remains authoritative for the turn's actual configuration. + +The session file never mirrors turn outcomes. Turn lifecycle facts live only +in turn files; deriving "is this session busy/suspended/failed" reads the +latest turn (section 8). + +## 6. Session reducer + +`@x/shared` owns one pure reducer shared by core and renderer: + +```ts +function reduceSession(events: SessionEvent[]): SessionState; + +interface SessionState { + definition: SessionCreated; + title?: string; + turns: Array<{ + turnId: string; + sessionSeq: number; + agentId: string; + model: ModelDescriptor; + ts: string; + }>; + latestTurnId?: string; + createdAt: string; // definition.ts + updatedAt: string; // ts of the last event +} +``` + +Invariants (violations throw, as with the turn reducer): + +- `session_created` is present, first, and unique. +- All event `sessionId` values match. +- `sessionSeq` is strictly increasing starting at 1, with no gaps. +- `turnId` values are unique. +- Unsupported schema versions and unknown event types fail loudly. + +## 7. Write ordering and consistency + +Per user message, the session layer performs, in order: + +1. `turnRuntime.createTurn(...)` — the turn file is created. +2. `sessionRepo.append(turn_appended)` — the session references the turn. +3. `advanceTurn(...)` — execution begins. + +Rules: + +- A crash between steps 1 and 2 leaves an orphan turn file: unreferenced, + never advanced, and benign. Turns are only ever found by reference, so an + orphan is invisible. V1 does not garbage-collect orphans. +- The reverse order is forbidden: a `turn_appended` referencing a turn file + that was never created would be a dangling reference, which is corruption. +- Step 2 precedes step 3 so that a turn that is executing is always already + referenced by its session. +- Session-file appends happen under the session lock; turn-file appends are + the turn runtime's concern. + +## 8. In-memory index + +### 8.1 Shape + +```ts +interface SessionIndexEntry { + sessionId: string; + title?: string; + createdAt: string; + updatedAt: string; + turnCount: number; + lastAgentId?: string; + lastModel?: ModelDescriptor; + latestTurnId?: string; + latestTurnStatus: + | "none" // session has no turns yet + | "completed" + | "failed" + | "cancelled" + | "suspended" // durable suspension: pending permissions/async tools + | "idle"; // non-terminal, not suspended: interrupted by a crash +} +``` + +`latestTurnStatus` is derived from the latest turn's reduced state, using +the same derivation everywhere: terminal event kind if present, else +`suspended` if a suspension with outstanding work is the resting state, else +`idle`. Whether a turn is *actively processing right now* is not in the +index; it is ephemeral bus state (`turn-processing-start/end`), per the turn +specification. + +### 8.2 Startup scan + +1. `listSessionIds()`. +2. For each session: read and reduce the session file, producing the entry's + session-derived fields. +3. For each session with turns: read and reduce the latest turn file only, + producing `latestTurnStatus`. +4. Publish the completed index to the renderer. + +A corrupt session file or corrupt latest-turn file does not abort startup: +the entry is surfaced in an errored state (identifiable in the UI, excluded +from normal interaction) and the scan continues. + +### 8.3 Maintenance + +- Every session mutation updates the entry in the same code path that + appends to the session file (write-through), then publishes a + `session-index-changed` event on the application bus. +- When an `advanceTurn` outcome settles, the session layer updates + `latestTurnStatus` and publishes `session-index-changed`. +- There is no filesystem watcher. Out-of-band edits to session or turn files + while the app runs are unsupported; offline changes are reconciled by the + next startup scan. +- The main process enforces single-instance via + `app.requestSingleInstanceLock()`; all locking in both layers is + in-process. + +## 9. Sessions API + +```ts +interface SendMessageConfig { + agent: RequestedAgent; // agent id + optional model override + autoPermission?: boolean; // default false + maxModelCalls?: number; // default per turn spec +} + +interface ISessions { + createSession(input?: { title?: string }): Promise; + listSessions(): SessionIndexEntry[]; + getSession(sessionId: string): Promise; + getTurn(turnId: string): Promise; // passthrough to turn runtime + + sendMessage( + sessionId: string, + input: UserMessage, + config: SendMessageConfig, + ): Promise<{ turnId: string }>; + + respondToPermission( + turnId: string, + toolCallId: string, + decision: "allow" | "deny", + metadata?: JsonValue, + ): Promise; + + respondToAskHuman( + turnId: string, + toolCallId: string, + answer: string, + ): Promise; + + deliverAsyncToolResult( + turnId: string, + toolCallId: string, + result: ToolResultData, + ): Promise; + + stopTurn(turnId: string, reason?: string): Promise; + resumeTurn(sessionId: string): Promise; + + setTitle(sessionId: string, title: string): Promise; + deleteSession(sessionId: string): Promise; +} +``` + +### 9.1 sendMessage + +Under the per-session lock: + +1. Read and reduce the session. +2. If the session has turns, read and reduce the latest turn. If it is + non-terminal — running, suspended, or idle — reject with a typed + `TurnNotSettledError`. There is no implicit queueing, steering, or + cancel-and-replace, and no implicit routing to a pending ask-human. +3. Build context: `[]` (inline, empty) for the first turn, else + `{ previousTurnId: latestTurnId }`. +4. Create the turn: config from `SendMessageConfig` lands on the turn + (`humanAvailable: true` always, for session turns). Sessions store no + configuration; every turn is self-describing. +5. Append `turn_appended` with the next `sessionSeq` and denormalized + agent/model. +6. If the session has no title, append `title_changed` derived from the + truncated first user message. +7. Start `advanceTurn` in the background; consume its events (section 10). +8. Return `{ turnId }` immediately. The renderer follows progress through + events, not the return value. + +Continuation after a failed or exhausted (`code: "model-call-limit"`) turn +is just `sendMessage`: failed turns are terminal, and the new turn's context +reference includes the failed turn's structurally complete transcript. + +### 9.2 External inputs + +`respondToPermission`, `respondToAskHuman`, and `deliverAsyncToolResult` +each translate to one `advanceTurn(turnId, input)` call with the +corresponding `TurnExternalInput`. `respondToAskHuman` is the dedicated +endpoint for the `ask-human` tool — a thin wrapper over +`async_tool_result` — and is deliberately separate from `sendMessage`. +Validation (unknown call, already-resolved call, terminal turn) is the turn +runtime's job; the session layer passes its errors through. + +### 9.3 stopTurn and resumeTurn + +- `stopTurn` cancels via the turn runtime: aborting the active invocation's + signal if one is running, else advancing a suspended turn with a `cancel` + input. +- `resumeTurn` re-enters the latest turn with no input — the turn spec's + recovery entry point — for turns left `idle` by a crash. There is no + automatic resume sweep at startup: recovery re-issues interrupted model + calls, so resumption must be an explicit user action. Suspended turns need + no resumption; they advance when their inputs arrive. + +### 9.4 Deletion + +`deleteSession` removes the session file and the index entry, and publishes +`session-index-changed`. Referenced turn files are retained as orphans: +turns are only discoverable by reference, so orphaned files are inert. +Deleting an entity's file is not a violation of append-only discipline, +which governs mutation of live logs, not their removal. + +## 10. Event forwarding and live UI + +1. For every `advanceTurn` it initiates, the session layer consumes + `TurnExecution.events` and forwards each `TurnStreamEvent` — tagged with + `sessionId` — through the application bus to renderer windows over one + IPC channel (`sessions:events`). +2. When `outcome` settles, the session layer updates the index entry and + publishes `session-index-changed`. +3. The renderer follows the turn spec's historical/live pattern: fetch + turns via `getTurn`, run the shared `reduceTurn` per turn, compose the + session timeline turn-by-turn (each turn renders its input and its own + activity; the referenced prefix is never re-rendered from context), + append live durable events and re-reduce, and keep text/reasoning deltas + in an ephemeral overlay cleared by canonical responses. +4. Pending approvals and ask-human prompts render from the suspended turn's + reduced state, so they survive restarts without any session-layer + bookkeeping. + +## 11. Headless standalone turns + +A helper covers the non-session callers (background tasks, live notes, +knowledge pipelines, scheduled agents): + +```ts +function runHeadlessTurn(input: { + agent: RequestedAgent; + context?: ConversationMessage[]; // inline; defaults to [] + input: UserMessage; + maxModelCalls?: number; + signal?: AbortSignal; +}): Promise; +``` + +- `sessionId: null`, `autoPermission: true`, `humanAvailable: false`. +- Creates the turn, advances to the first settled outcome, and returns it. +- Standalone turns never appear in the index; callers keep their own turn + IDs if they need history. + +## 12. Deferred designs with committed shapes + +These are not implemented in v1. Their shapes are recorded so v1 decisions +stay compatible; each arrives as a session schema version bump. + +### 12.1 Queued messages + +```ts +{ type: "message_queued", queueId, message, ts } +{ type: "queued_message_replaced", queueId, message, ts } // edit +{ type: "queued_message_removed", queueId, ts } // cancel +// promotion: turn_appended gains consumedQueueIds?: string[] +``` + +The reducer derives the pending queue by supersession. Promotion rules +(collapse-into-one-turn vs FIFO, behavior after failed turns) are decided +together with steering. + +### 12.2 Session permission grants + +```ts +{ type: "permission_grant_added", grantId, toolId, ts } +{ type: "permission_grant_removed", grantId, ts } +``` + +The injected `IPermissionChecker` consults a session-keyed grants view +before answering `required: true`. V1 grants would be blanket per-toolId; +argument-pattern matchers are a separate, security-sensitive project. + +### 12.3 Compaction (mechanism sketch) + +Compaction requires zero turn-schema change. A session-level compaction +event records `{ compactionId, summary, firstKeptTurnId }`; the next turn +after a compaction uses **inline** context (summary message + kept +transcript), which restarts the reference chain and bounds resolution depth +by construction. Trigger policy and summarizer design are unspecified. + +### 12.4 Steering + +Rides the queue events with a different promotion rule, and additionally +requires a turn-level `steer` external input (a turn schema bump) with an +injection boundary after tool-batch completion. Recorded here so the session +design does not foreclose it. + +### 12.5 Persisted index cache + +If the startup scan ever gets slow, a single cache file keyed by file +mtimes can be added. It is a rebuildable cache, never a source of truth: +missing, stale, or invalid means rebuild from session files. + +### 12.6 Auto-titling + +An LLM-generated title replacing the truncated-first-message default, +appended as an ordinary `title_changed` event. + +## 13. Required test scenarios + +All tests use the in-memory/mocked turn runtime and repo fakes. + +### 13.1 Reducer + +- Valid event sequences reduce to expected state. +- Every invariant violation throws: missing/duplicate `session_created`, + mismatched sessionId, non-monotonic or gapped `sessionSeq`, duplicate + turnIds, unknown type/version. +- Title folding: default, explicit changes, last-wins. + +### 13.2 Repository + +- Date-partitioned paths, ID validation, create-if-absent. +- Strict line validation on read; corrupt files rejected whole. +- `listSessionIds` enumeration across partitions. +- Deletion removes only the session file. + +### 13.3 sendMessage + +- First turn: inline `[]` context, `sessionSeq: 1`, default title appended. +- Subsequent turns: context references the latest turn; seq increments. +- Rejection with `TurnNotSettledError` when the latest turn is running, + suspended, or idle — and success after it settles. +- Continuation after failed and model-call-limit turns. +- Concurrent sendMessage calls serialize under the session lock; exactly + one wins, the other rejects. +- Config lands on the turn; the session file stores only denormalized + agent/model on `turn_appended`. + +### 13.4 Ordering and crash simulation + +- Simulated crash between `createTurn` and `turn_appended`: orphan turn + file, session unchanged, retry produces a fresh turn. +- `turn_appended` is present before the first advance begins. + +### 13.5 External inputs + +- Permission decision, ask-human answer, and async result each advance the + correct turn with the correct input type. +- Turn-runtime rejections (unknown call, terminal turn) pass through. +- `sendMessage` never routes to ask-human. + +### 13.6 Index + +- Startup fold matches write-through state for the same history. +- Latest-turn status derivation for every status value. +- Corrupt session file yields an errored entry without aborting the scan. +- Mutations publish `session-index-changed`; deletion removes the entry. + +### 13.7 Event forwarding + +- Forwarded events are tagged with sessionId and arrive in order. +- Outcome settlement updates `latestTurnStatus`. + +### 13.8 Headless + +- Standalone turns: `sessionId: null`, auto permission, human unavailable, + absent from the index. + +## 14. Suggested module layout + +```text +apps/x/packages/shared/src/sessions.ts # event schemas, reducer, index types + +apps/x/packages/core/src/sessions/ + sessions.ts # ISessions implementation + repo.ts # ISessionRepo contract + fs-repo.ts # filesystem implementation + index.ts # in-memory index + startup scan + headless.ts # runHeadlessTurn +``` + +Awilix registration mirrors the turn runtime: singleton scope, PROXY +constructor injection, no container resolution from inside the classes. + +## 15. Integration sequence + +The rollout is staged as commits on one branch (squash-merge acceptable); +old and new stacks coexist briefly but never share state, and no data is +migrated: + +1. `@x/shared`: turn + session schemas and reducers. +2. Turn runtime + fs turn repo (unit tests only, wired to nothing). +3. Session layer + index (unit tests only). +4. Bridges: agent resolver, context resolver, tool runner, permission + checker/classifier — adapted from the `new-runtime` reference + implementation where applicable. +5. IPC (`sessions:*`) + renderer swap: Copilot chat UI moves to the + sessions API. +6. Headless callers move to `runHeadlessTurn`. +7. Delete the old runs runtime. + +## 16. Implementation acceptance criteria + +The session layer is implementation-complete only when: + +1. Session event schemas and `reduceSession` live in `@x/shared` and are + consumed unchanged by core and renderer. +2. Session files follow the partitioned append-only JSONL layout with + strict validation. +3. Turn-file-first write ordering is enforced; orphan turns are benign. +4. `sendMessage` rejects non-terminal latest turns with a typed error; no + implicit queueing, steering, or ask-human routing exists. +5. Ask-human answers flow only through the dedicated endpoint. +6. The index is write-through with a startup scan, no watcher, and no + persisted cache; single-instance is enforced. +7. Deletion removes only the session file. +8. Headless callers run standalone turns and appear nowhere in the index. +9. All required test scenarios pass with mocked dependencies. diff --git a/apps/x/packages/core/docs/turn-runtime-design.md b/apps/x/packages/core/docs/turn-runtime-design.md new file mode 100644 index 00000000..2daaf2bb --- /dev/null +++ b/apps/x/packages/core/docs/turn-runtime-design.md @@ -0,0 +1,1830 @@ +# Turn Runtime Technical Specification + +Status: design complete for the turn layer, amended after review for session +integration: context references, revised crash-recovery semantics, and a +distinguishable model-call-limit outcome. No implementation exists yet. The +companion session layer is specified in `session-design.md`. + +This document specifies a new turn-oriented agent loop for `@x/core`. It is +intended to replace the behavioral responsibilities of the current run runtime +eventually, but migration and replacement are explicitly outside the initial +implementation scope. + +The design was developed independently of sessions. Session storage, +conversation ordering, queued user messages, context compaction, and selection +of the next active turn will be designed separately. + +## 1. Goals + +The turn runtime must: + +1. Execute one complete agent turn from one user input. +2. Repeatedly call a model and process tool calls until the model produces a + final response, the turn suspends, or the turn reaches a terminal outcome. +3. Persist every durable fact to one append-only JSONL file per turn. +4. Support turns that do not belong to a session. +5. Support suspension for human permission decisions and externally executed + asynchronous tools. +6. Recover deterministically from the durable log after a process restart. +7. Keep all runtime dependencies explicit and injectable. +8. Keep the core loop small and focused on flow control. +9. Expose normalized live events without coupling turn correctness to event + consumption. +10. Make historical turns reconstructable for the UI and other consumers. + +## 2. Non-goals + +The first implementation will not include: + +- Session storage or session scheduling. +- Queued user messages. +- Enforcement of the session-level one-active-turn rule. +- Context pruning or compaction. +- Agent-as-tool behavior. An agent tool handler may implement this separately, + but the parent turn sees it only as a normal sync or async tool. +- Inline, caller-supplied agents. +- Mid-turn agent, model, prompt, or tool-set switching. +- Tool argument validation beyond what the model/tool integration already + provides. +- External-input idempotency keys or duplicate-delivery handling. +- Automatic model-call retries. +- Cross-process locking. +- Explicit `fsync` after every append. +- Automatic repair of corrupt JSONL files. +- Stream backpressure or bounded event buffering. +- A shared UI timeline selector. +- Migration from the current run system. + +## 3. Core terminology + +### 3.1 Turn + +A turn starts with one intentional user message and contains all agent work +caused by that message: + +- Model requests and responses. +- Permission checks and decisions. +- Sync tool executions. +- Async tool requests, progress, and results. +- Repeated model calls after tool results. +- Final completion, failure, cancellation, or suspension. + +Permission decisions and async tool results do not create new turns. They +advance the existing turn. + +### 3.2 Session + +A session is an optional higher-level conversation and scheduling concept. A +turn has `sessionId: string | null`, but turn execution never reads session +state. + +The session layer will eventually own: + +- Turn ordering. +- One-active-turn enforcement. +- Queued user messages. +- Context construction from prior turns. +- Context pruning and compaction. +- Session-wide permission grants. + +### 3.3 Agent + +An agent is a reusable execution preset consisting primarily of: + +- A system prompt. +- A model/provider selection. +- A set of tools. + +The current codebase follows this model across Copilot, background-task, +live-note, knowledge-sync, and user-defined agents. + +An agent is resolved once when a turn is created. The resulting execution +snapshot is immutable for the lifetime of the turn. + +## 4. Architectural principles + +### 4.1 The JSONL log is the source of truth + +There is one JSONL file per turn. Every line contains one validated JSON event. +State is always reconstructed by replaying the file from the beginning. + +There are no mutable summary files, checkpoints, or storage sidecars. + +### 4.2 Events describe facts, not current process liveness + +The durable log must never claim that a turn is currently running. A process +can crash at any time and make such a claim false. + +Durable events may state that: + +- A model request was prepared and requested. +- A tool invocation was requested. +- A result was received. +- A turn suspended. +- A turn completed, failed, or was cancelled. + +Whether an `advanceTurn` invocation is currently active is ephemeral +application state communicated through the application bus. + +### 4.3 The turn runtime has no hidden dependencies + +`TurnRuntime` is a class used as an immutable dependency container. It holds no +mutable per-turn execution state. All active turn state is reconstructed from +the repository inside each invocation. + +Awilix constructs the singleton using PROXY/destructured-object injection. +`TurnRuntime` must never resolve dependencies from the Awilix container itself. + +### 4.4 Durable barriers precede side effects + +When a durable event represents intent to perform an external side effect, the +event is successfully appended before the side effect begins. + +Examples: + +- Append `model_call_requested` before calling the model. +- Append `tool_invocation_requested` before invoking a sync tool or exposing an + async tool request. + +This does not provide exactly-once execution. It provides conservative recovery +semantics after ambiguous interruptions. + +### 4.5 Execution order and model-facing order are separate + +Tool calls may complete in any order. The next model request must always contain +tool results in the original order emitted by the model. + +The initial implementation may execute sync tools sequentially for simplicity. +Async tools naturally complete independently. No behavior may rely on physical +completion order. + +## 5. Storage design + +### 5.1 File location + +Turn files live under: + +```text +WorkDir/storage/turns/YYYY/MM/DD/.jsonl +``` + +The existing time-based `IMonotonicallyIncreasingIdGenerator` produces IDs such +as: + +```text +2025-11-11T04-36-29Z-0001234-000 +``` + +The deterministic path is therefore: + +```text +WorkDir/storage/turns/2025/11/11/2025-11-11T04-36-29Z-0001234-000.jsonl +``` + +The repository validates the ID format before deriving the path. It extracts +the UTC `YYYY-MM-DD` prefix and rejects malformed or path-like values. + +### 5.2 File rules + +- The first line is always `turn_created`. +- The first line contains `schemaVersion: 1`. +- Every event contains `turnId` and an ISO timestamp `ts`. +- Physical line order is authoritative event order. +- There are no generic event IDs or sequence numbers. +- Domain identifiers such as `toolCallId` and `modelCallIndex` remain explicit. +- Every event must be JSON-serializable. +- `undefined`, `BigInt`, functions, cyclic data, class instances, and raw + `Error` objects are rejected. +- Reads validate every line strictly. +- Any malformed line, including a malformed final line, makes the turn corrupt. +- The repository does not truncate, repair, or skip malformed lines. +- Unknown schema versions and unknown event types fail loudly. +- Appends are awaited but are not explicitly flushed with `fsync` initially. + +### 5.3 Repository contract + +```ts +interface ITurnRepo { + create(event: TurnCreated): Promise; + read(turnId: string): Promise; + append(turnId: string, events: TurnEvent[]): Promise; + withLock(turnId: string, fn: () => Promise): Promise; +} +``` + +Rules: + +- `create` fails if the target file already exists. +- `append` validates events before writing. +- `read` validates every line and verifies event `turnId` values. +- `withLock` provides in-process per-turn exclusion. +- Cross-process coordination is out of scope. +- Whether lock contention waits or reports busy is an implementation detail for + the first version. +- Listing, deletion, session lookup, and presentation metadata are not required + by the loop-facing repository contract. + +## 6. Agent resolution + +### 6.1 Caller input + +`createTurn` accepts a registered agent ID and an optional atomic model +override: + +```ts +type JsonPrimitive = string | number | boolean | null; +type JsonValue = + | JsonPrimitive + | JsonValue[] + | { [key: string]: JsonValue }; + +interface ModelDescriptor { + provider: string; + model: string; +} + +interface RequestedAgent { + agentId: string; + overrides?: { + model?: ModelDescriptor; + }; +} +``` + +Provider and model are overridden together. Independent partial provider/model +overrides are not supported. + +Inline resolved agents are not supported initially because there is no current +application use case. + +### 6.2 AgentResolver responsibilities + +`IAgentResolver` absorbs the current agent-assembly responsibilities: + +- Built-in agent selection currently handled by `loadAgent`. +- User-defined agent loading from `IAgentsRepo`. +- Dynamic Copilot, background-task, live-note, and knowledge-agent builders. +- Agent-specific system-prompt augmentation. +- Agent Notes, work-directory context, and similar prompt assembly. +- Model override, agent configuration, and application-default precedence. +- Tool attachment resolution. +- Tool availability filtering. +- Creation of immutable serializable tool descriptors. + +```ts +interface ResolvedAgent { + agentId: string; + systemPrompt: string; + model: ModelDescriptor; + tools: ToolDescriptor[]; +} + +interface IAgentResolver { + resolve(agent: RequestedAgent): Promise; +} +``` + +The resolved system prompt is the final byte-for-byte string used by model +requests. The turn loop never appends additional instructions. + +Model resolution precedence is: + +1. `createTurn` model override. +2. Model/provider configured on the agent. +3. Application default model/provider. + +### 6.3 Tool identity + +Model-facing names and runtime implementation identities are distinct: + +```ts +interface ToolDescriptor { + toolId: string; + name: string; + description: string; + inputSchema: JsonValue; + execution: "sync" | "async"; + requiresHuman: boolean; +} +``` + +- `toolId` is the stable registry lookup identity. +- `name` is the name advertised to the model. +- The distinction supports aliases and MCP-backed tools. +- The descriptor is fully snapshotted at turn creation. +- `advanceTurn` may not silently add, remove, or modify tools. + +### 6.4 Requested and resolved agent snapshots + +`turn_created` stores both caller intent and resolved output: + +```ts +agent: { + requested: RequestedAgent; + resolved: ResolvedAgent; +} +``` + +This makes defaults, aliases, and overrides visible during debugging. + +If agent resolution fails, `createTurn` rejects without creating a turn file. + +### 6.5 Model and tool implementation materialization + +Agent resolution and live runtime materialization are separate responsibilities, +but all dependencies are constructor-injected: + +- `IAgentResolver` creates the immutable serializable snapshot during + `createTurn`. +- `IModelRegistry` resolves the persisted model descriptor to a live AI SDK + model during `advanceTurn`. +- `IToolRegistry` resolves persisted tool descriptors to live implementations + during `advanceTurn`. + +This lets an old turn resume from its snapshot even if the registered agent +definition later changes. + +### 6.6 Context references and materialization + +Session turns do not inline their conversation prefix. Context is either an +inline message array or a reference to the immediately preceding turn: + +```ts +type TurnContext = + | { previousTurnId: string } + | ConversationMessage[]; +``` + +- `{ previousTurnId }` references an immutable prior turn. The materialized + context is that turn's closed transcript: its own materialized context, + followed by its input and every message it produced, including synthetic + closure results for failed, cancelled, or interrupted work. +- Inline arrays are used by standalone turns and by callers that assemble + context themselves (for example after a future compaction). + +Materialization is performed by an injected resolver: + +```ts +interface IContextResolver { + resolve(context: TurnContext): Promise; +} +``` + +Rules: + +- Resolution happens at the start of every `advanceTurn` invocation, from + durable state only. Normal execution and crash recovery therefore share one + code path. +- Resolution traverses references recursively until an inline context + terminates the chain. +- Turns with any terminal status participate in resolution. Their transcripts + are structurally complete because unresolved work receives synthetic + results before a terminal event is appended. +- A reference to a missing or corrupt turn file is an infrastructure error. + It rejects the execution and does not append `turn_failed`. +- The reducer treats `context` as opaque data and never resolves references. + +## 7. Turn creation schema + +The authoritative initial event is: + +```ts +interface TurnCreated { + type: "turn_created"; + schemaVersion: 1; + turnId: string; + ts: string; + sessionId: string | null; + + agent: { + requested: RequestedAgent; + resolved: ResolvedAgent; + }; + + context: TurnContext; + input: UserMessage; + + config: { + autoPermission: boolean; + humanAvailable: boolean; + maxModelCalls: number; + }; +} +``` + +Rules: + +- `sessionId` is explicitly nullable. +- `context` is either an inline message array or a reference to the previous + turn (section 6.6). +- Inline context contains prior user, assistant, and tool messages only. +- System-role messages in inline context are rejected. +- A context reference is resolved by the injected context resolver during + `advanceTurn`, never at creation and never by the reducer. +- The resolved agent owns the single authoritative system prompt. +- `input` is the user message that defines this turn boundary. +- `autoPermission` defaults to `false` before persistence. +- `humanAvailable` is required explicitly. +- `maxModelCalls` defaults to `20` before persistence. +- Persisted values are fully resolved and immutable. + +The capability is named `humanAvailable`, not `headless`. `headless` describes +one deployment mode, while this flag states the exact runtime capability used +by permission fallback and human-dependent tools. + +The canonical system prompt is written once in `turn_created`. It may be +deliberately duplicated inside each `model_call_requested.request` because that +event records the exact model input for auditing. + +## 8. Shared event schemas + +All durable serializable event schemas live in `@x/shared`. The repository, +runtime, live stream implementation, and filesystem behavior live in `@x/core`. + +### 8.1 Base event + +```ts +interface BaseTurnEvent { + turnId: string; + ts: string; +} +``` + +### 8.2 Usage + +Usage means token usage only. Monetary pricing and cost calculation are out of +scope. + +```ts +interface TurnUsage { + inputTokens?: number; + outputTokens?: number; + totalTokens?: number; + reasoningTokens?: number; + cachedInputTokens?: number; +} +``` + +Usage is stored per completed model call and aggregated across the turn. + +### 8.3 Model request events + +Every AI SDK `streamText` invocation performs exactly one model step. Tool +execution is controlled manually by the turn loop. + +```ts +interface ModelRequest { + systemPrompt: string; + contextRef?: { previousTurnId: string }; + messages: ConversationMessage[]; + tools: ToolDescriptor[]; + parameters: Record; +} + +interface ModelCallRequested extends BaseTurnEvent { + type: "model_call_requested"; + modelCallIndex: number; + request: ModelRequest; +} +``` + +`modelCallIndex` starts at `0` and increments monotonically for each primary +agent model call in the turn. + +`request` records the exact model input with one substitution: when the +turn's context is a reference, the resolved prefix is not re-inlined. +`contextRef` repeats the reference and `messages` contains only current-turn +messages — the turn input, this turn's prior assistant responses, and ordered +tool results — captured byte-for-byte as assembled immediately before +provider invocation. When the turn's context is inline, `contextRef` is +absent and `messages` begins with that inline context. The full request is +deterministically materializable because referenced turns are immutable; a +`materializeRequest(turnId, modelCallIndex)` debug utility composes resolver +output with the persisted messages. + +Within-turn duplication is deliberate: each request restates the current +turn's accumulated messages even though they also exist as earlier events in +the same file. The persisted `messages` are the ground truth of what the loop +actually assembled and sent — including any assembly bugs — which a derived +reconstruction would mask. + +`request` excludes credentials, auth headers, functions, model objects, and +transport objects. + +The name `requested` is intentional. The event proves durable intent, not that +the provider definitely received the request. + +### 8.4 Provider step events + +Normalized AI SDK step events may be persisted for debugging: + +```ts +interface ModelStepEvent extends BaseTurnEvent { + type: "model_step_event"; + modelCallIndex: number; + event: DurableLlmStepStreamEvent; +} +``` + +The wrapper may contain normalized events such as: + +- Text start/end. +- Reasoning start/end. +- Completed tool calls. +- Finish-step metadata and usage. +- Normalized provider errors. + +Raw text and reasoning delta events are not durable. Partial tool-argument +fragments are also not required; completed parsed tool calls are durable. + +There is one `TurnEvent` union. Events are not permanently classified as +"state" or "diagnostic" because reducer usage will evolve. The reducer decides +which known events currently affect derived state. + +### 8.5 Completed and failed model calls + +```ts +interface ModelCallCompleted extends BaseTurnEvent { + type: "model_call_completed"; + modelCallIndex: number; + message: AssistantMessage; + finishReason: string; + usage: TurnUsage; + providerMetadata?: JsonValue; +} + +interface ModelCallFailed extends BaseTurnEvent { + type: "model_call_failed"; + modelCallIndex: number; + error: string; +} +``` + +`model_call_completed` is the canonical completed model response even when its +content duplicates provider step events. + +Any successfully completed assistant response without tool calls completes the +turn, including responses whose finish reason is `length` or `content-filter`. +Provider and stream failures fail the turn. + +Only the primary model calls directly controlled by the turn loop are recorded +as model calls. Internal model calls hidden inside the permission classifier or +tool implementations belong to those dependencies and are not turn-loop model +calls. + +## 9. Permission model + +### 9.1 Dependencies + +An injected permission checker determines whether a tool call needs permission: + +```ts +interface PermissionCheckAllowed { + required: false; +} + +interface PermissionCheckRequired { + required: true; + request: JsonValue; +} + +interface IPermissionChecker { + check(input: PermissionCheckInput): Promise< + PermissionCheckAllowed | PermissionCheckRequired + >; +} +``` + +Tool-specific policy, command analysis, filesystem boundaries, and allowlists +remain outside the loop. + +When automatic permission is enabled, the injected classifier handles all +permission-required calls from one model response in one batch: + +```ts +interface PermissionClassification { + toolCallId: string; + decision: "allow" | "deny" | "defer"; + reason: string; +} + +interface IPermissionClassifier { + classify( + requests: PermissionClassificationInput[], + signal: AbortSignal, + ): Promise; +} +``` + +The classifier's internal implementation and internal model calls are opaque to +the turn loop. + +### 9.2 Permission events + +```ts +interface ToolPermissionRequired extends BaseTurnEvent { + type: "tool_permission_required"; + toolCallId: string; + toolName: string; + request: JsonValue; + checkerError?: string; +} + +interface ToolPermissionClassified extends BaseTurnEvent { + type: "tool_permission_classified"; + toolCallId: string; + decision: "allow" | "deny" | "defer"; + reason: string; +} + +interface ToolPermissionClassificationFailed extends BaseTurnEvent { + type: "tool_permission_classification_failed"; + toolCallIds: string[]; + error: string; +} + +interface ToolPermissionResolved extends BaseTurnEvent { + type: "tool_permission_resolved"; + toolCallId: string; + decision: "allow" | "deny"; + source: "classifier" | "human" | "human_unavailable"; + reason?: string; + metadata?: JsonValue; +} +``` + +Only `tool_permission_resolved` is an effective execution decision. + +### 9.3 Permission behavior matrix + +`autoPermission` and `humanAvailable` are orthogonal: + +| autoPermission | humanAvailable | Behavior | +| --- | --- | --- | +| false | true | Ask human and suspend. | +| false | false | Deny and continue with an error tool result. | +| true | true | Classify; deferred calls ask human. | +| true | false | Classify; deferred calls are denied. | + +Classifier decisions behave as follows: + +- `allow`: resolve allow and execute/dispatch immediately. +- `deny`: resolve deny and create an error tool result. +- `defer`: ask a human if available; otherwise deny. +- Classifier failure or omitted decision: record failure and treat as `defer`. + +If the permission checker itself throws, the loop fails closed: + +- Record the checker error as a permission-required event. +- Ask a human if available. +- Otherwise create a denied error result. +- Never execute automatically. + +### 9.4 Partial decisions + +Permission resolution is per tool call, not a batch barrier. + +When one permission decision arrives: + +- An allowed sync tool executes immediately. +- An allowed async tool is exposed immediately. +- A denied tool receives an immediate error result. +- Other unresolved permission requests remain pending. +- The turn suspends again if any external inputs remain outstanding. + +Permission responses are accepted one at a time. The caller is never required +to batch external inputs. + +### 9.5 Permission scopes + +The turn consumes only the effective allow/deny decision for the current tool +call. Session-wide or global approval grants are persisted by the caller before +advancing the turn. Scope may be retained as audit metadata, but the loop does +not interpret or apply it. + +## 10. Tool model + +### 10.1 Runtime tools + +Sync and async execution are immutable tool metadata: + +```ts +interface ToolExecutionContext { + signal: AbortSignal; + reportProgress(progress: JsonValue): Promise; +} + +interface SyncRuntimeTool { + descriptor: ToolDescriptor & { execution: "sync" }; + execute( + input: JsonValue, + context: ToolExecutionContext, + ): Promise; +} + +interface AsyncRuntimeTool { + descriptor: ToolDescriptor & { execution: "async" }; +} + +type RuntimeTool = SyncRuntimeTool | AsyncRuntimeTool; +``` + +An async tool has no in-process executor. Its request is exposed externally and +its progress/result is later supplied through `advanceTurn`. + +`ask-human` is an ordinary async tool. Other externally resolved async tools may +be added without changing the loop. + +### 10.2 Human-dependent tools + +`requiresHuman` is generic metadata, not a hard-coded `ask-human` name check. + +If `requiresHuman` is true and `humanAvailable` is false: + +- Append the tool invocation request. +- Append an immediate runtime error result such as "Human input is unavailable + for this turn." +- Continue the tool batch. + +`humanAvailable` affects human-dependent tools and permission fallback. It does +not disable other async tools. + +### 10.3 Tool events + +```ts +interface ToolInvocationRequested extends BaseTurnEvent { + type: "tool_invocation_requested"; + toolCallId: string; + toolId: string; + toolName: string; + execution: "sync" | "async"; + input: JsonValue; +} + +interface ToolProgress extends BaseTurnEvent { + type: "tool_progress"; + toolCallId: string; + source: "sync" | "async"; + progress: JsonValue; +} + +interface ToolResultData { + output: JsonValue; + isError: boolean; + metadata?: JsonValue; +} + +interface ToolResult extends BaseTurnEvent { + type: "tool_result"; + toolCallId: string; + toolName: string; + source: "sync" | "async" | "runtime"; + result: ToolResultData; +} +``` + +`runtime` results cover: + +- Permission denial. +- Unknown tools. +- Invalid or unusable model tool calls. +- Human unavailability. +- Cancellation. +- Interrupted sync execution. + +A tool result may exist without an invocation event when execution is rejected +before dispatch, such as permission denial or unknown tool handling. + +### 10.4 Progress + +Any sync or async tool may report progress. + +- Sync tools call and await `reportProgress`. +- The loop appends `tool_progress` before resolving that callback. +- Async progress arrives as one external `advanceTurn` input. +- Progress is durable and informational. +- Progress does not satisfy the pending tool call. +- Progress after a terminal tool result is rejected. +- Backend flow control may ignore progress while UI reducers retain it. + +No progress write queue or non-blocking batching is included initially. + +### 10.5 Tool scheduling and ordering + +For one completed assistant response: + +1. Identify all model-produced tool calls and their source order. +2. Determine permission requirements. +3. Apply automatic permission decisions when enabled. +4. Advance each tool independently as its permission is resolved. +5. Execute allowed sync tools using a simple sequential policy initially. +6. Expose allowed async tool requests. +7. Suspend when any permissions or async results remain outstanding. +8. Once all calls have terminal results, build the next model request with + results in original model-call order. + +The model is not called while any tool in the current batch lacks a terminal +result. + +## 11. Suspension and external inputs + +### 11.1 Suspension event + +```ts +interface TurnSuspended extends BaseTurnEvent { + type: "turn_suspended"; + pendingPermissions: Array<{ + toolCallId: string; + toolName: string; + request: JsonValue; + }>; + pendingAsyncTools: Array<{ + toolCallId: string; + toolId: string; + toolName: string; + input: JsonValue; + }>; + usage: TurnUsage; +} +``` + +Before an invocation returns suspended, it appends a full snapshot of currently +pending external work. + +Calling `advanceTurn` with no input when the turn is already suspended returns +the current suspended outcome without appending a duplicate event. + +After a valid external input, if work remains pending, a new suspension snapshot +is appended before returning. + +### 11.2 One input per invocation + +```ts +type TurnExternalInput = + | { + type: "permission_decision"; + toolCallId: string; + decision: "allow" | "deny"; + metadata?: JsonValue; + } + | { + type: "async_tool_progress"; + toolCallId: string; + progress: JsonValue; + } + | { + type: "async_tool_result"; + toolCallId: string; + result: ToolResultData; + } + | { + type: "cancel"; + reason?: string; + }; +``` + +Each `advanceTurn` accepts at most one input. It validates that input against the +current durable pending state, persists the resulting semantic event, advances +as far as possible, and settles again. + +Inputs targeting calls that are no longer pending are rejected. + +No caller-generated input IDs or duplicate-delivery semantics are included. + +### 11.3 Partial async results + +Async progress and results may arrive independently and in any order. + +- Persist each input immediately. +- Keep the turn suspended while any permission or async result remains pending. +- Do not initiate the next model call until every original tool call has a + terminal result. + +## 12. Terminal events + +```ts +interface TurnCompleted extends BaseTurnEvent { + type: "turn_completed"; + output: AssistantMessage; + finishReason: string; + usage: TurnUsage; +} + +interface TurnFailed extends BaseTurnEvent { + type: "turn_failed"; + error: string; + code?: string; + usage: TurnUsage; +} + +interface TurnCancelled extends BaseTurnEvent { + type: "turn_cancelled"; + reason?: string; + usage: TurnUsage; +} +``` + +The completed event deliberately duplicates the final assistant message, +finish reason, and aggregate usage so the final line is a self-contained +completion summary. + +Errors begin as a single descriptive string. Structured details may be added +later when they are naturally available upstream; the loop does not invent +structure by parsing arbitrary errors. + +`code` is an optional machine-readable discriminator for failures that +callers must tell apart. This specification defines `"model-call-limit"` +(section 20); other codes may be added when a caller concretely needs them. + +Terminal states are immutable: + +- Events may not be appended after completion, failure, or cancellation. +- Re-advancing a terminal turn returns its existing outcome. +- Retrying a failed turn means creating a new turn with a new ID. + +## 13. Turn event union and live deltas + +```ts +type TurnEvent = + | TurnCreated + | ModelCallRequested + | ModelStepEvent + | ModelCallCompleted + | ModelCallFailed + | ToolPermissionRequired + | ToolPermissionClassified + | ToolPermissionClassificationFailed + | ToolPermissionResolved + | ToolInvocationRequested + | ToolProgress + | ToolResult + | TurnSuspended + | TurnCompleted + | TurnFailed + | TurnCancelled; +``` + +Ephemeral model deltas are stream-only: + +```ts +interface TextDelta { + type: "text_delta"; + turnId: string; + modelCallIndex: number; + delta: string; +} + +interface ReasoningDelta { + type: "reasoning_delta"; + turnId: string; + modelCallIndex: number; + delta: string; +} + +type TurnStreamEvent = TurnEvent | TextDelta | ReasoningDelta; +``` + +All durable provider step events are appended before the loop reads the next +provider stream event. This intentionally applies simple filesystem +backpressure. Text and reasoning deltas bypass storage. + +## 14. Derived turn state + +### 14.1 One canonical reducer + +`@x/shared` owns one pure reducer used by core and renderer: + +```ts +function reduceTurn(events: TurnEvent[]): TurnState; +``` + +Properties: + +- Pure and deterministic. +- Performs no I/O. +- Performs no expensive external work. +- Replays the full bounded turn log from the beginning. +- Known events that do not currently affect state may be ignored. +- Invalid event transitions throw a corruption/state error. +- There is no incremental reducer API initially. + +Consumers may append a newly received durable event to their local event list +and call `reduceTurn` again. + +### 14.2 State shape + +```ts +interface TurnState { + definition: TurnCreated; + modelCalls: ModelCallState[]; + toolCalls: ToolCallState[]; + suspension?: TurnSuspended; + terminal?: TurnCompleted | TurnFailed | TurnCancelled; + usage: TurnUsage; +} + +interface ModelCallState { + index: number; + request: ModelRequest; + stepEvents: DurableLlmStepStreamEvent[]; + response?: AssistantMessage; + finishReason?: string; + usage?: TurnUsage; + error?: string; +} + +interface ToolCallState { + modelCallIndex: number; + order: number; + toolCallId: string; + toolId?: string; + toolName: string; + input: JsonValue; + execution?: "sync" | "async"; + + permission?: { + required: ToolPermissionRequired; + classification?: ToolPermissionClassified; + resolved?: ToolPermissionResolved; + }; + + invocation?: ToolInvocationRequested; + progress: ToolProgress[]; + result?: ToolResult; +} +``` + +The exact implementation may refine these field names, but the state must +retain enough information for both execution and historical presentation. + +### 14.3 No durable or derived running status + +`TurnState` does not contain a single status field. + +Consumers derive what they need from: + +- `suspension`. +- `terminal`. +- Outstanding model and tool records. +- Ephemeral application bus activity. + +This avoids encoding process liveness in durable or replayed state. + +### 14.4 No transient streaming buffer + +`TurnState` contains only durable projections. It does not contain partial text +or reasoning buffers. The renderer owns those ephemeral buffers while handling +live deltas and discards them when a canonical model response arrives. + +### 14.5 No shared timeline selector initially + +The renderer may interpret `modelCalls` and `toolCalls` directly. A shared +semantic timeline selector can be added later if duplicated presentation logic +becomes a concrete problem. + +### 14.6 Historical read API + +```ts +interface Turn { + turnId: string; + events: TurnEvent[]; +} + +interface ITurnRuntime { + getTurn(turnId: string): Promise; +} +``` + +`getTurn` is strictly read-only: + +- It reads and validates the JSONL file. +- It returns the validated event list. +- It never resumes work. +- It never appends recovery records. +- Callers use the shared reducer to obtain `TurnState`. + +Only `advanceTurn` may reconcile or advance execution. + +Prior session context remains inside `state.definition.context`. Historical UI +for one turn should normally show the triggering input and activity produced by +that turn, not repeat the prior context. Session presentation will handle +composition later. + +## 15. TurnRuntime API and dependency injection + +### 15.1 Constructor + +```ts +interface TurnRuntimeDependencies { + turnRepo: ITurnRepo; + idGenerator: IMonotonicallyIncreasingIdGenerator; + clock: IClock; + agentResolver: IAgentResolver; + modelRegistry: IModelRegistry; + toolRegistry: IToolRegistry; + contextResolver: IContextResolver; + permissionChecker: IPermissionChecker; + permissionClassifier: IPermissionClassifier; + bus: IBus; +} + +class TurnRuntime implements ITurnRuntime { + constructor({ + turnRepo, + idGenerator, + clock, + agentResolver, + modelRegistry, + toolRegistry, + contextResolver, + permissionChecker, + permissionClassifier, + bus, + }: TurnRuntimeDependencies); +} +``` + +The injected clock exposes a simple deterministic timestamp source for unit +tests. The existing ID generator remains responsible for turn IDs. + +Awilix registration is singleton-scoped: + +```ts +turnRepo: asClass(FSTurnRepo).singleton(), +turnRuntime: asClass(TurnRuntime).singleton(), +``` + +Model, tool, permission, clock, and bus dependencies are also explicitly +registered and injected. + +### 15.2 Public API + +```ts +interface CreateTurnInput { + agent: RequestedAgent; + sessionId?: string | null; + context: TurnContext; + input: UserMessage; + config: { + autoPermission?: boolean; + humanAvailable: boolean; + maxModelCalls?: number; + }; +} + +interface ITurnRuntime { + createTurn(input: CreateTurnInput): Promise; + + advanceTurn( + turnId: string, + input?: TurnExternalInput, + options?: { signal?: AbortSignal }, + ): TurnExecution; + + getTurn(turnId: string): Promise; +} +``` + +Creation and execution remain separate operations. A turn may exist before its +first model call. + +### 15.3 Runtime dependency validation + +Before advancing, the runtime resolves live model and tool implementations from +the persisted descriptors and validates compatibility. + +Missing or mismatched runtime dependencies: + +- Reject the execution. +- Do not append `turn_failed`. +- Leave the turn unchanged so the caller can fix its environment and retry. + +Provider failures after actual execution begins are modeled turn failures. + +## 16. Hot TurnExecution stream + +### 16.1 Shape + +```ts +interface TurnExecution { + events: AsyncIterable; + outcome: Promise; +} +``` + +`advanceTurn` returns immediately with a hot execution object. Execution starts +independently of event consumption. + +### 16.2 Rationale + +A plain async generator would make execution pull-driven: + +- Never consuming it would mean execution never starts. +- Slow consumers would pause the loop at every yield. +- Abandoning a partially consumed iterator could leave execution paused and a + lock held indefinitely. + +The hot stream prevents turn correctness and lock release from depending on an +observer. + +### 16.3 Initial stream semantics + +- One event consumer is assumed. +- The caller may fan events out through its own event emitter or bus bridge. +- Events produced by that `advanceTurn` invocation only are emitted. +- Historical events are not replayed. +- Durable events enter the stream only after successful persistence. +- Text and reasoning deltas enter without persistence. +- The stream uses a simple in-memory queue. +- There is no backpressure or queue bound initially. +- A slow or absent consumer may cause memory growth. +- If the consumer closes, subsequent events are dropped. +- Closing or abandoning event consumption does not cancel turn execution. +- Cancellation is explicit through `AbortSignal` or a cancel input. +- `outcome` may be awaited independently of `events`. + +### 16.4 Outcome + +```ts +type TurnOutcome = + | { + status: "completed"; + output: AssistantMessage; + finishReason: string; + usage: TurnUsage; + } + | { + status: "suspended"; + pendingPermissions: TurnSuspended["pendingPermissions"]; + pendingAsyncTools: TurnSuspended["pendingAsyncTools"]; + usage: TurnUsage; + } + | { + status: "failed"; + error: string; + usage: TurnUsage; + } + | { + status: "cancelled"; + reason?: string; + usage: TurnUsage; + }; +``` + +### 16.5 Infrastructure errors + +If trustworthy turn state cannot be established: + +- Event iteration terminates by throwing the infrastructure error. +- `outcome` rejects with the same error. +- Already persisted events remain in the file. +- No synthetic `turn_failed` is appended unless the failure is explicitly a + modeled turn failure. + +Infrastructure examples: + +- Corrupt JSONL. +- Repository failure. +- Lock failure. +- Missing or mismatched injected runtime dependency. + +## 17. Ephemeral application lifecycle events + +`TurnRuntime` publishes process-lifecycle events through the injected bus: + +```ts +interface TurnProcessingStart { + type: "turn-processing-start"; + turnId: string; +} + +interface TurnProcessingEnd { + type: "turn-processing-end"; + turnId: string; +} +``` + +These events are never written to `TurnRepo` and never replayed. + +The UI may maintain an in-memory set of active turn IDs. If the process crashes, +that set disappears, which accurately reflects that no execution is known to be +active. + +Lifecycle publication is observational and must not alter durable turn +semantics. Live durable events and deltas are consumed from `TurnExecution` by +the application and may be forwarded over the existing bus/IPC bridge. + +## 18. Main execution algorithm + +At a high level, `advanceTurn` performs: + +```text +1. Start hot execution task. +2. Acquire per-turn lock. +3. Publish turn-processing-start. +4. Read and validate JSONL. +5. Reduce events to TurnState. +6. Materialize context through the injected context resolver. +7. Validate terminal state, input, and runtime dependencies. +8. Apply the optional single external input. +9. Repeatedly advance deterministic work: + a. Recover from the current durable boundary. + b. Resolve permissions. + c. Execute eligible sync tools. + d. Expose eligible async tools. + e. If external work remains, append turn_suspended and settle. + f. If a tool batch is complete, build ordered tool-result messages. + g. If no model-call budget remains, fail. + h. Append model_call_requested with exact input. + i. Call streamText for one step. + j. Persist normalized non-delta events and stream deltas. + k. Append model_call_completed or model_call_failed. + l. Complete when the response has no tool calls. +10. Publish turn-processing-end in finalization. +11. Release the lock. +12. Resolve/reject outcome and close/error event stream. +``` + +The loop does not read session queues, accept new user messages, switch agents, +or transform context. + +## 19. Context and model requests + +Context selection before turn creation belongs to the caller/session layer. +The turn receives either inline messages or a reference to the previous turn +(section 6.6). Within a turn: + +- The initial context — inline or resolved from the reference — is immutable. +- The system prompt is immutable. +- The model and tools are immutable. +- The loop appends current-turn assistant messages and ordered tool results. +- No pruning or compaction occurs. +- Context overflow is a model failure. + +Every model call persists the current-turn portion of its request +byte-for-byte, even when it duplicates earlier events in the same file +(section 8.3). The referenced prefix is deterministic and is not re-inlined. +Within-turn storage size is not treated as a constraint. + +## 20. Model-call limit + +`maxModelCalls` limits primary agent model calls, not permission-classifier or +tool-internal model calls. + +When the final allowed model call returns tool calls: + +1. Resolve permissions. +2. Execute/dispatch and collect the complete tool batch. +3. Do not make a disallowed next model call. +4. Append `turn_failed` with `code: "model-call-limit"`. + +If the final allowed call returns a response without tool calls, complete +normally. + +Limit exhaustion is a distinguishable outcome, not a generic failure. The +transcript is structurally complete — every tool call has a result — so a +caller such as the session layer can offer continuation by creating a new +turn whose context references the exhausted turn. + +## 21. Failure semantics + +### 21.1 Model failures + +- No automatic retry in the turn loop. +- The injected model/provider may perform internal transport retries. +- Append `model_call_failed`. +- Append `turn_failed`. +- Return a failed outcome. + +These rules apply to failures observed during live execution. Closing a model +call that was interrupted by a process crash is a recovery action (section +23) and does not fail the turn. + +### 21.2 Tool failures + +Tool failures do not fail the turn: + +- Sync tool throws: append an error tool result. +- Async tool reports failure: append an error tool result. +- Permission denial: append an error tool result. +- Human unavailable: append an error tool result. + +Once every call has a result, send all results to the model in source order. + +### 21.3 Unknown tools and unusable calls + +Unknown tools and unusable model tool calls become runtime error results so the +model can observe and potentially correct its behavior. They do not fail the +turn directly. + +### 21.4 Storage and configuration failures + +Repository failures, corrupt data, and incompatible live dependencies reject +the execution without pretending the agent turn itself failed. + +## 22. Cancellation + +Cancellation enters through: + +- An `AbortSignal` for an active invocation. +- A `{ type: "cancel" }` external input for a suspended turn. + +Rules: + +- Propagate the signal to the model, permission classifier, and sync tools. +- Create synthetic cancellation results for unresolved tool calls from a + completed assistant response. +- Append `model_call_failed` when an active model call is cancelled. +- Append `turn_cancelled`. +- Never initiate another model call. +- Reject late permission decisions, progress, and results after cancellation. + +Sync tools are cooperatively cancellable. A tool that ignores the signal may not +settle immediately. The runtime cannot guarantee rollback of external side +effects. + +## 23. Recovery model + +Provider streams are not resumable. Sync tool side effects are not assumed +idempotent. Recovery never repeats side-effecting work; side-effect-free +model calls are re-issued rather than failing the turn. + +Calling `advanceTurn(turnId)` with no external input is the recovery entry point. + +| Last durable condition | Recovery behavior | +| --- | --- | +| `turn_created` only | Safely initiate the first model call. | +| `model_call_requested` without completion/failure | Append `model_call_failed` (interrupted) to close the call, then re-issue the request as a new model call with the next index. The re-issue counts against `maxModelCalls`. No `turn_failed`. | +| Completed model response before permission/tool processing | Safely continue processing. | +| Permission required/classification partially processed | Continue unresolved permission work safely. | +| Permission resolved allow before invocation | Safely invoke the tool. | +| Sync `tool_invocation_requested` without result | Append an indeterminate error tool result and continue the turn; never re-execute. No `turn_failed`. | +| Async `tool_invocation_requested` without result | Remain suspended awaiting external result. | +| Some async results present | Preserve them and await remaining inputs. | +| All tool results present before next model request | Safely initiate the next model call. | +| Terminal event present | Return existing outcome without writing or executing. | +| Corrupt JSONL | Reject as infrastructure error. | + +The synthetic interrupted sync result should communicate: + +```text +Tool execution was interrupted; its outcome is unknown and it was not retried. +``` + +This preserves a structurally complete transcript for later session context +without claiming that the side effect did or did not happen. The model sees +the indeterminate result on the next model call and can decide whether to +re-run the tool, verify its effect, or proceed — consistent with section +21.2's rule that tool-level problems are conversational, not terminal. + +Async invocation requests act as durable external requests. Exactly-once +delivery is not guaranteed. External systems that perform side effects should +use `toolCallId` as a correlation/deduplication key if they need that guarantee, +but the first turn implementation does not enforce it. + +## 24. Reducer invariants + +`reduceTurn` must reject impossible histories, including: + +- Missing or non-first `turn_created`. +- Multiple `turn_created` events. +- Mismatched turn IDs. +- Unsupported schema version. +- Model-call indices that are reused or out of order. +- Concurrent unresolved primary model requests. +- Model completion/failure without a matching request. +- Duplicate model completion/failure. +- Duplicate tool-call IDs within one model response. +- Permission records targeting unknown tool calls. +- Conflicting effective permission decisions. +- Tool progress after a terminal tool result. +- Duplicate terminal tool results. +- Async external results for sync tools. +- Sync execution results for async tools unless explicitly represented as + runtime-generated errors. +- Starting the next model call before all prior tool calls have results. +- Model-facing tool-result ordering that differs from original call order. +- Completion while tool calls remain unresolved. +- Terminal turn events while unresolved calls lack synthetic terminal results. +- Multiple terminal turn events. +- Any event after a terminal turn event. +- Mutation of immutable turn definition data. +- `model_call_requested.contextRef` values inconsistent with the turn + definition's context. + +The reducer validates `context` structurally but treats it as opaque; it +never resolves references (section 6.6). + +## 25. Historical and live UI behavior + +### 25.1 Historical load + +```text +1. UI requests a turn by ID. +2. TurnRuntime.getTurn reads and validates the JSONL file. +3. Core returns { turnId, events }. +4. Renderer calls reduceTurn(events). +5. Renderer renders messages, tools, progress, permissions, usage, and terminal + information from TurnState. +``` + +### 25.2 Live updates + +```text +1. TurnRuntime publishes turn-processing-start. +2. The application consumes TurnExecution.events. +3. The application forwards events through the existing bus/IPC bridge. +4. The renderer appends each durable TurnEvent to its local event list. +5. The renderer reruns reduceTurn(events). +6. Text/reasoning deltas update separate ephemeral renderer buffers. +7. A canonical completed model response replaces/clears partial display state. +8. TurnRuntime publishes turn-processing-end. +``` + +The UI derives current activity from ephemeral bus events. It does not infer +that a turn is running from the JSONL log. + +## 26. Required unit-test scenarios + +All dependencies are injectable, so these tests must use fakes/mocks and avoid +real providers, tools, clocks, filesystems where unnecessary, and permission +classifiers. + +### 26.1 Plain model response + +Expected sequence: + +```text +turn_created +model_call_requested +model_step_event(s) +model_call_completed +turn_completed +``` + +Assertions: + +- Exact model request is persisted. +- Deltas are streamed but absent from JSONL. +- Completed response includes finish reason and usage. +- Terminal event duplicates final output and aggregate usage. +- Bus processing events are emitted but not persisted. + +### 26.2 Mixed sync and async tools + +Model order: + +```text +sync-A, async-B, sync-C, async-D +``` + +Assertions: + +- Sync tools execute and store results. +- Async invocation requests are exposed and turn suspends. +- Async results can arrive in reverse order. +- The next model request contains results in original source order. +- Physical event order does not change model-facing order. + +### 26.3 Partial human permission decisions + +Assertions: + +- All required permissions are recorded. +- One approval advances only its tool. +- Approved sync tools execute immediately. +- Approved async tools become pending immediately. +- Denied calls receive runtime error results. +- Remaining permissions and async tools coexist in one suspension snapshot. +- Async results may arrive while other permissions remain unresolved. +- No next model call occurs until all original calls have terminal results. + +### 26.4 Automatic permission classification + +Test `allow`, `deny`, and `defer` in one classifier batch. + +Assertions: + +- Classifier decisions and effective decisions are distinct records. +- Allow executes. +- Deny creates an error result without invocation. +- Defer asks a human when available. +- Defer denies when no human is available. +- Classifier failure and missing decisions normalize to defer. +- Manual mode never calls the classifier. + +### 26.5 Cancellation + +Test cancellation: + +- While suspended. +- During a model call. +- During a sync tool. + +Assertions: + +- Signals are propagated. +- Unresolved calls receive synthetic cancellation results. +- Turn becomes terminal cancelled. +- No subsequent model request occurs. +- Late external inputs are rejected. + +### 26.6 Failures + +Assertions: + +- Provider failure appends model and turn failures. +- Sync throw becomes an error tool result and does not fail the turn. +- Async failure becomes an error tool result. +- Permission-checker failure fails closed. +- Repository, corruption, and dependency errors reject stream and outcome. +- Infrastructure errors do not append a false `turn_failed` event. + +### 26.7 Process crash and recovery + +Construct logs ending at every recovery boundary in the recovery table. + +Assertions: + +- Safe boundaries resume. +- Unmatched model requests are closed as interrupted and re-issued; the + re-issue counts against the model-call budget and no `turn_failed` is + appended. +- Unmatched sync invocation requests get indeterminate error results and the + turn continues. +- Unmatched async requests remain pending. +- Completed tool batches proceed to the next model call. +- Terminal turns perform no writes or side effects. + +### 26.8 Historical and live reconstruction + +Assertions: + +- `getTurn` is read-only. +- Full replay reconstructs messages, model calls, tools, progress, permissions, + usage, suspension, and terminal data. +- Appending a durable live event and rerunning the reducer matches full replay. +- Ephemeral deltas are absent after reload. +- Final completed messages remain available after reload. +- No running state is inferred from durable events. + +### 26.9 Model-call limit + +With `maxModelCalls = 10`: + +- A tool-free response on call 10 completes normally. +- Tool calls from call 10 are fully processed. +- Call 11 is never made. +- After the call-10 tool batch completes, the turn fails with a limit error. +- The limit failure carries `code: "model-call-limit"`. + +## 27. Additional unit-test coverage + +Beyond the nine end-to-end loop scenarios, implementation should include focused +tests for: + +### 27.1 Repository + +- Deterministic date-partitioned paths. +- ID path-traversal rejection. +- Create-if-absent behavior. +- Append ordering. +- Read/write schema validation. +- Mismatched turn IDs. +- Empty files. +- Malformed first, middle, and final lines. +- Unsupported schema versions. +- In-process locking. + +### 27.2 Reducer + +- Every valid event transition. +- Every corruption invariant. +- Usage aggregation. +- Original tool-call ordering. +- Suspension replacement/invalidation after new inputs. +- Terminal immutability. +- Multiple model iterations. +- Progress retention. +- Requested versus resolved agent snapshots. + +### 27.3 Hot stream + +- Execution starts without event consumption. +- Outcome resolves without draining events. +- Events retain order. +- Durable events appear only after repository append. +- Closing the consumer drops future events without cancelling execution. +- Producer completion closes events. +- Infrastructure failure errors events and rejects outcome with the same error. +- Cancellation remains explicit. + +### 27.4 Agent resolution + +- Override precedence. +- Application-default fallback. +- Final system prompt snapshot. +- Tool availability filtering. +- Tool aliases and stable `toolId` resolution. +- Resolution failure creates no file. +- Agent edits after creation do not alter the turn. +- Runtime dependency mismatch leaves the turn unchanged. + +### 27.5 Context resolution + +- Inline context passes through unchanged. +- A single reference materializes the referenced turn's closed transcript. +- A chain of references materializes recursively down to the inline base. +- Referenced failed/cancelled/exhausted turns contribute their synthetic + closure results. +- A reference to a missing or corrupt turn file is an infrastructure error + and appends nothing. +- `materializeRequest` output equals resolver output plus persisted + current-turn request messages. +- The reducer never resolves references. + +## 28. Suggested module layout + +This is a suggested organization, not a locked implementation requirement: + +```text +apps/x/packages/shared/src/turns.ts + +apps/x/packages/core/src/turns/ + runtime.ts + reducer.ts # if implementation is re-exported through shared, + # locate pure code to avoid dependency cycles + repo.ts + fs-repo.ts + stream.ts + agent-resolver.ts + context-resolver.ts + model-registry.ts + tool-registry.ts + permission.ts + index.ts +``` + +The final reducer location must permit both core and renderer to use exactly the +same pure implementation. It may therefore live entirely in `@x/shared` despite +being used heavily by `@x/core`. + +## 29. Deferred design work + +The following topics must be handled in later specifications rather than +implicitly added to the turn loop: + +### 29.1 Sessions + +The session layer is specified in `session-design.md`. It covers the session +JSONL schema, turn references and ordering, one-active-turn enforcement, +context assembly through turn references, the in-memory index, session UI +projections, and startup reconciliation. Queued user messages, steering, +session-level permission grants, and compaction remain deferred there with +committed future shapes. + +### 29.2 Agent-as-tool + +The main loop treats agent tools as ordinary sync or async tools. A dedicated +tool handler may create and execute child turns, forward progress, and return a +parent tool result. No subflow or parent-turn concept is added to this initial +turn schema. + +### 29.3 Reliability enhancements + +- External-input idempotency. +- Exactly-once async dispatch acknowledgements. +- Tool retry-safety metadata. +- Automatic retry policies. +- Cross-process locking. +- `fsync` durability. +- Torn-write repair. +- Event-stream bounds and backpressure. +- Large-blob sidecars, if ever required. + +### 29.4 API refinements + +- Inline agent support if a real use case appears. +- Explicit tool-argument validation. +- Shared semantic timeline selectors. +- Public query projections beyond `getTurn`. +- Turn listing and deletion APIs. + +## 30. Implementation acceptance criteria + +The turn layer is implementation-complete only when: + +1. All durable schemas are defined and exported from `@x/shared`. +2. Filesystem storage follows the deterministic partitioned JSONL layout. +3. `TurnRuntime` uses explicit PROXY constructor injection. +4. Agent resolution produces one immutable snapshot. +5. Each model invocation is exactly one AI SDK model step. +6. Sync, async, permission, suspension, cancellation, and recovery behavior + matches this specification. +7. The pure reducer is shared by core and renderer. +8. No durable running state exists. +9. The hot stream is independent of consumer progress. +10. Every required scenario and focused test group passes using mocked + dependencies. +11. No session, agent-as-tool, compaction, or migration behavior leaks into the + turn loop. From d821c419fe1e61f5aa35244b771966433b1c4aef Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 07:46:07 +0530 Subject: [PATCH 04/24] feat(x/shared): turn + session event schemas and pure reducers (stage 1) Durable contracts for the new turn/session runtime, shared by core and renderer: - turns.ts: zod schemas for all 16 durable turn events, TurnContext (previousTurnId ref | inline messages), ModelRequest with contextRef and current-turn-only messages, ephemeral delta types, reduceTurn enforcing every spec invariant, and pure derivations (deriveTurnStatus, turnTranscript, outstanding work). - sessions.ts: session_created/turn_appended/title_changed schemas, reduceSession, and the SessionIndexEntry projection. - vitest wiring for packages/shared (config, build-excluded tests); 92 tests covering happy paths, recovery-shaped histories, and one named test per corruption invariant. Co-Authored-By: Claude Fable 5 --- apps/x/packages/shared/package.json | 9 +- apps/x/packages/shared/src/sessions.test.ts | 201 +++ apps/x/packages/shared/src/sessions.ts | 186 +++ apps/x/packages/shared/src/turns.test.ts | 1337 +++++++++++++++++++ apps/x/packages/shared/src/turns.ts | 958 +++++++++++++ apps/x/packages/shared/tsconfig.build.json | 7 + apps/x/packages/shared/vitest.config.ts | 11 + apps/x/pnpm-lock.yaml | 4 + 8 files changed, 2711 insertions(+), 2 deletions(-) create mode 100644 apps/x/packages/shared/src/sessions.test.ts create mode 100644 apps/x/packages/shared/src/sessions.ts create mode 100644 apps/x/packages/shared/src/turns.test.ts create mode 100644 apps/x/packages/shared/src/turns.ts create mode 100644 apps/x/packages/shared/tsconfig.build.json create mode 100644 apps/x/packages/shared/vitest.config.ts diff --git a/apps/x/packages/shared/package.json b/apps/x/packages/shared/package.json index 0bab918a..5a1a9c45 100644 --- a/apps/x/packages/shared/package.json +++ b/apps/x/packages/shared/package.json @@ -5,10 +5,15 @@ "main": "./dist/index.js", "types": "./dist/index.d.ts", "scripts": { - "build": "rm -rf dist && tsc", - "dev": "tsc -w" + "build": "rm -rf dist && tsc -p tsconfig.build.json", + "dev": "tsc -w", + "test": "vitest run", + "test:watch": "vitest" }, "dependencies": { "zod": "^4.2.1" + }, + "devDependencies": { + "vitest": "catalog:" } } \ No newline at end of file diff --git a/apps/x/packages/shared/src/sessions.test.ts b/apps/x/packages/shared/src/sessions.test.ts new file mode 100644 index 00000000..2676a2b7 --- /dev/null +++ b/apps/x/packages/shared/src/sessions.test.ts @@ -0,0 +1,201 @@ +import { describe, expect, it } from "vitest"; +import { z } from "zod"; +import { + SessionCorruptionError, + SessionCreated, + SessionEvent, + SessionTurnAppended, + reduceSession, + sessionIndexEntry, +} from "./sessions.js"; + +type SEvent = z.infer; + +const SESSION_ID = "2026-07-02T09-00-00Z-0000042-000"; +const MODEL = { provider: "openai", model: "gpt-test" }; + +function sessionCreated( + overrides: Partial> = {}, +): z.infer { + return { + type: "session_created", + schemaVersion: 1, + sessionId: SESSION_ID, + ts: "2026-07-02T09:00:00Z", + ...overrides, + }; +} + +function turnAppended( + sessionSeq: number, + turnId: string, + overrides: Partial> = {}, +): z.infer { + return { + type: "turn_appended", + sessionId: SESSION_ID, + ts: `2026-07-02T09:0${sessionSeq}:00Z`, + turnId, + sessionSeq, + agentId: "copilot", + model: MODEL, + ...overrides, + }; +} + +function titleChanged(title: string, ts = "2026-07-02T09:05:00Z") { + return { + type: "title_changed" as const, + sessionId: SESSION_ID, + ts, + title, + }; +} + +function expectCorruption(events: SEvent[], match: string | RegExp): void { + expect(() => reduceSession(events)).toThrowError(SessionCorruptionError); + expect(() => reduceSession(events)).toThrowError(match); +} + +describe("schemas", () => { + it("session events round-trip through the SessionEvent schema", () => { + const events: SEvent[] = [ + sessionCreated({ title: "Hello" }), + turnAppended(1, "turn-1"), + titleChanged("Renamed"), + ]; + for (const event of events) { + expect(SessionEvent.parse(event)).toEqual(event); + } + }); + + it("rejects non-positive sessionSeq at the schema level", () => { + expect(() => SessionEvent.parse(turnAppended(0, "turn-0"))).toThrowError(); + }); +}); + +describe("reduceSession", () => { + it("reduces a created-only log", () => { + const state = reduceSession([sessionCreated()]); + expect(state.definition.sessionId).toBe(SESSION_ID); + expect(state.turns).toEqual([]); + expect(state.latestTurnId).toBeUndefined(); + expect(state.title).toBeUndefined(); + expect(state.createdAt).toBe("2026-07-02T09:00:00Z"); + expect(state.updatedAt).toBe("2026-07-02T09:00:00Z"); + }); + + it("folds turns in order with denormalized metadata", () => { + const state = reduceSession([ + sessionCreated(), + turnAppended(1, "turn-1"), + turnAppended(2, "turn-2", { agentId: "researcher", model: { provider: "anthropic", model: "claude-x" } }), + ]); + expect(state.turns).toHaveLength(2); + expect(state.turns[0]).toMatchObject({ turnId: "turn-1", sessionSeq: 1, agentId: "copilot" }); + expect(state.turns[1]).toMatchObject({ + turnId: "turn-2", + sessionSeq: 2, + agentId: "researcher", + model: { provider: "anthropic", model: "claude-x" }, + }); + expect(state.latestTurnId).toBe("turn-2"); + expect(state.updatedAt).toBe("2026-07-02T09:02:00Z"); + }); + + it("folds titles: creation default then last change wins", () => { + const withDefault = reduceSession([sessionCreated({ title: "First message…" })]); + expect(withDefault.title).toBe("First message…"); + + const renamed = reduceSession([ + sessionCreated({ title: "First message…" }), + titleChanged("Better title"), + titleChanged("Best title", "2026-07-02T09:06:00Z"), + ]); + expect(renamed.title).toBe("Best title"); + expect(renamed.updatedAt).toBe("2026-07-02T09:06:00Z"); + }); + + it("rejects an empty log", () => { + expectCorruption([], /empty/); + }); + + it("rejects a log not starting with session_created", () => { + expectCorruption([turnAppended(1, "turn-1")], /first event must be session_created/); + }); + + it("rejects duplicate session_created", () => { + expectCorruption([sessionCreated(), sessionCreated()], /duplicate session_created/); + }); + + it("rejects mismatched session ids", () => { + expectCorruption( + [sessionCreated(), { ...turnAppended(1, "turn-1"), sessionId: "other" }], + /does not match session/, + ); + }); + + it("rejects unsupported schema versions", () => { + const bad = { ...sessionCreated(), schemaVersion: 2 } as unknown as SEvent; + expectCorruption([bad], /unsupported session schema version/); + }); + + it("rejects unknown event types", () => { + const bad = { type: "wat", sessionId: SESSION_ID, ts: "t" } as unknown as SEvent; + expectCorruption([sessionCreated(), bad], /unknown session event type/); + }); + + it("rejects a first turn not at sessionSeq 1", () => { + expectCorruption([sessionCreated(), turnAppended(2, "turn-2")], /out of order; expected 1/); + }); + + it("rejects gapped sessionSeq", () => { + expectCorruption( + [sessionCreated(), turnAppended(1, "turn-1"), turnAppended(3, "turn-3")], + /out of order; expected 2/, + ); + }); + + it("rejects duplicate sessionSeq", () => { + expectCorruption( + [sessionCreated(), turnAppended(1, "turn-1"), turnAppended(1, "turn-1b")], + /out of order; expected 2/, + ); + }); + + it("rejects duplicate turn ids", () => { + expectCorruption( + [sessionCreated(), turnAppended(1, "turn-1"), turnAppended(2, "turn-1")], + /duplicate turnId/, + ); + }); +}); + +describe("sessionIndexEntry", () => { + it("projects an entry with last-turn metadata and the provided status", () => { + const state = reduceSession([ + sessionCreated({ title: "T" }), + turnAppended(1, "turn-1"), + turnAppended(2, "turn-2", { agentId: "researcher" }), + ]); + expect(sessionIndexEntry(state, "suspended")).toEqual({ + sessionId: SESSION_ID, + title: "T", + createdAt: "2026-07-02T09:00:00Z", + updatedAt: "2026-07-02T09:02:00Z", + turnCount: 2, + lastAgentId: "researcher", + lastModel: MODEL, + latestTurnId: "turn-2", + latestTurnStatus: "suspended", + }); + }); + + it("projects an empty session with status none", () => { + const state = reduceSession([sessionCreated()]); + const entry = sessionIndexEntry(state, "none"); + expect(entry.turnCount).toBe(0); + expect(entry.lastAgentId).toBeUndefined(); + expect(entry.latestTurnStatus).toBe("none"); + }); +}); diff --git a/apps/x/packages/shared/src/sessions.ts b/apps/x/packages/shared/src/sessions.ts new file mode 100644 index 00000000..52cfd11b --- /dev/null +++ b/apps/x/packages/shared/src/sessions.ts @@ -0,0 +1,186 @@ +import { z } from "zod"; +import { ModelDescriptor, type TurnStatus } from "./turns.js"; + +// Durable session contract for the session layer (see +// packages/core/docs/session-design.md). A session is an append-only chain +// of turn references plus presentation metadata; conversation content lives +// exclusively in turn files. Pure module shared by core and renderer. + +// --------------------------------------------------------------------------- +// Durable events +// --------------------------------------------------------------------------- + +export const SessionCreated = z.object({ + type: z.literal("session_created"), + schemaVersion: z.literal(1), + sessionId: z.string(), + ts: z.string(), + title: z.string().optional(), +}); + +// agentId/model are denormalized from the turn so the session index can fold +// without opening turn files; the turn file stays authoritative. +export const SessionTurnAppended = z.object({ + type: z.literal("turn_appended"), + sessionId: z.string(), + ts: z.string(), + turnId: z.string(), + sessionSeq: z.number().int().positive(), + agentId: z.string(), + model: ModelDescriptor, +}); + +export const SessionTitleChanged = z.object({ + type: z.literal("title_changed"), + sessionId: z.string(), + ts: z.string(), + title: z.string(), +}); + +export const SessionEvent = z.discriminatedUnion("type", [ + SessionCreated, + SessionTurnAppended, + SessionTitleChanged, +]); + +// --------------------------------------------------------------------------- +// Derived session state +// --------------------------------------------------------------------------- + +export class SessionCorruptionError extends Error { + constructor(message: string) { + super(message); + this.name = "SessionCorruptionError"; + } +} + +export interface SessionTurnRef { + turnId: string; + sessionSeq: number; + agentId: string; + model: z.infer; + ts: string; +} + +export interface SessionState { + definition: z.infer; + title?: string; + turns: SessionTurnRef[]; + latestTurnId?: string; + createdAt: string; + updatedAt: string; +} + +function fail(message: string): never { + throw new SessionCorruptionError(message); +} + +export function reduceSession( + events: Array>, +): SessionState { + if (events.length === 0) { + fail("session log is empty"); + } + const [first, ...rest] = events; + if (first.type !== "session_created") { + fail(`first event must be session_created, got ${first.type}`); + } + if (first.schemaVersion !== 1) { + fail(`unsupported session schema version: ${String(first.schemaVersion)}`); + } + + const state: SessionState = { + definition: first, + title: first.title, + turns: [], + createdAt: first.ts, + updatedAt: first.ts, + }; + + for (const event of rest) { + if (event.sessionId !== first.sessionId) { + fail( + `event sessionId ${event.sessionId} does not match session ${first.sessionId}`, + ); + } + switch (event.type) { + case "session_created": + fail("duplicate session_created event"); + break; + case "turn_appended": { + const expectedSeq = state.turns.length + 1; + if (event.sessionSeq !== expectedSeq) { + fail( + `sessionSeq ${event.sessionSeq} out of order; expected ${expectedSeq}`, + ); + } + if (state.turns.some((t) => t.turnId === event.turnId)) { + fail(`duplicate turnId in session: ${event.turnId}`); + } + state.turns.push({ + turnId: event.turnId, + sessionSeq: event.sessionSeq, + agentId: event.agentId, + model: event.model, + ts: event.ts, + }); + state.latestTurnId = event.turnId; + break; + } + case "title_changed": + state.title = event.title; + break; + default: { + const unknown: never = event; + fail( + `unknown session event type: ${(unknown as { type: string }).type}`, + ); + } + } + state.updatedAt = event.ts; + } + + return state; +} + +// --------------------------------------------------------------------------- +// Session index (in-memory projection; never a source of truth) +// --------------------------------------------------------------------------- + +// "none" = the session has no turns yet. The remaining values are the latest +// turn's derived status (deriveTurnStatus in turns.ts). +export type SessionLatestTurnStatus = "none" | TurnStatus; + +export interface SessionIndexEntry { + sessionId: string; + title?: string; + createdAt: string; + updatedAt: string; + turnCount: number; + lastAgentId?: string; + lastModel?: z.infer; + latestTurnId?: string; + latestTurnStatus: SessionLatestTurnStatus; + // Set when the session (or its latest turn) failed to load/validate; the + // entry is surfaced in an errored state instead of aborting the startup + // scan. + error?: string; +} + +export function sessionIndexEntry( + state: SessionState, + latestTurnStatus: SessionLatestTurnStatus, +): SessionIndexEntry { + const lastTurn = state.turns[state.turns.length - 1]; + return { + sessionId: state.definition.sessionId, + title: state.title, + createdAt: state.createdAt, + updatedAt: state.updatedAt, + turnCount: state.turns.length, + lastAgentId: lastTurn?.agentId, + lastModel: lastTurn?.model, + latestTurnId: state.latestTurnId, + latestTurnStatus, + }; +} diff --git a/apps/x/packages/shared/src/turns.test.ts b/apps/x/packages/shared/src/turns.test.ts new file mode 100644 index 00000000..bcdae400 --- /dev/null +++ b/apps/x/packages/shared/src/turns.test.ts @@ -0,0 +1,1337 @@ +import { describe, expect, it } from "vitest"; +import { z } from "zod"; +import { + type JsonValue, + ModelCallCompleted, + ModelCallFailed, + ModelCallRequested, + ModelStepEvent, + MODEL_CALL_LIMIT_ERROR_CODE, + ToolDescriptor, + ToolInvocationRequested, + ToolPermissionClassificationFailed, + ToolPermissionClassified, + ToolPermissionRequired, + ToolPermissionResolved, + ToolProgress, + ToolResult, + TurnCancelled, + TurnCompleted, + TurnCorruptionError, + TurnCreated, + TurnEvent, + TurnFailed, + TurnSuspended, + deriveTurnStatus, + outstandingAsyncTools, + outstandingPermissions, + reduceTurn, + turnTranscript, +} from "./turns.js"; + +type TEvent = z.infer; + +const TURN_ID = "2026-07-02T10-00-00Z-0000001-000"; +const PREV_TURN_ID = "2026-07-02T09-00-00Z-0000001-000"; +const TS = "2026-07-02T10:00:00Z"; + +const echoTool: z.infer = { + toolId: "tool.echo", + name: "echo", + description: "Echo tool", + inputSchema: {}, + execution: "sync", + requiresHuman: false, +}; + +const fetchTool: z.infer = { + toolId: "tool.fetch", + name: "fetch", + description: "Async fetch tool", + inputSchema: {}, + execution: "async", + requiresHuman: false, +}; + +function user(text: string) { + return { role: "user" as const, content: text }; +} + +function assistantText(text: string) { + return { role: "assistant" as const, content: text }; +} + +function toolCallPart(id: string, name: string, args: JsonValue = {}) { + return { + type: "tool-call" as const, + toolCallId: id, + toolName: name, + arguments: args, + }; +} + +function assistantCalls(...parts: Array>) { + return { role: "assistant" as const, content: parts }; +} + +function toolMsg(id: string, name: string, content = "ok") { + return { role: "tool" as const, content, toolCallId: id, toolName: name }; +} + +function created( + overrides: Partial> = {}, +): z.infer { + return { + type: "turn_created", + schemaVersion: 1, + turnId: TURN_ID, + ts: TS, + sessionId: null, + agent: { + requested: { agentId: "copilot" }, + resolved: { + agentId: "copilot", + systemPrompt: "SYS", + model: { provider: "openai", model: "gpt-test" }, + tools: [echoTool, fetchTool], + }, + }, + context: [], + input: user("hello"), + config: { + autoPermission: false, + humanAvailable: true, + maxModelCalls: 20, + }, + ...overrides, + }; +} + +function requested( + index: number, + messages: z.infer["request"]["messages"], + requestOverrides: Partial["request"]> = {}, +): z.infer { + return { + type: "model_call_requested", + turnId: TURN_ID, + ts: TS, + modelCallIndex: index, + request: { + systemPrompt: "SYS", + messages, + tools: [echoTool, fetchTool], + parameters: {}, + ...requestOverrides, + }, + }; +} + +function completed( + index: number, + message: z.infer["message"], + usage: z.infer["usage"] = { + inputTokens: 10, + outputTokens: 5, + totalTokens: 15, + }, +): z.infer { + return { + type: "model_call_completed", + turnId: TURN_ID, + ts: TS, + modelCallIndex: index, + message, + finishReason: "stop", + usage, + }; +} + +function callFailed( + index: number, + error = "provider exploded", +): z.infer { + return { + type: "model_call_failed", + turnId: TURN_ID, + ts: TS, + modelCallIndex: index, + error, + }; +} + +function stepEvent( + index: number, + event: z.infer["event"] = { + type: "text_end", + text: "chunk", + }, +): z.infer { + return { + type: "model_step_event", + turnId: TURN_ID, + ts: TS, + modelCallIndex: index, + event, + }; +} + +function permRequired( + id: string, + name: string, + checkerError?: string, +): z.infer { + return { + type: "tool_permission_required", + turnId: TURN_ID, + ts: TS, + toolCallId: id, + toolName: name, + request: { tool: name }, + ...(checkerError === undefined ? {} : { checkerError }), + }; +} + +function permClassified( + id: string, + decision: z.infer["decision"], +): z.infer { + return { + type: "tool_permission_classified", + turnId: TURN_ID, + ts: TS, + toolCallId: id, + decision, + reason: "because", + }; +} + +function permClassificationFailed( + ids: string[], +): z.infer { + return { + type: "tool_permission_classification_failed", + turnId: TURN_ID, + ts: TS, + toolCallIds: ids, + error: "classifier timed out", + }; +} + +function permResolved( + id: string, + decision: z.infer["decision"], + source: z.infer["source"] = "human", +): z.infer { + return { + type: "tool_permission_resolved", + turnId: TURN_ID, + ts: TS, + toolCallId: id, + decision, + source, + }; +} + +function invocation( + id: string, + tool: z.infer = echoTool, + overrides: Partial> = {}, +): z.infer { + return { + type: "tool_invocation_requested", + turnId: TURN_ID, + ts: TS, + toolCallId: id, + toolId: tool.toolId, + toolName: tool.name, + execution: tool.execution, + input: {}, + ...overrides, + }; +} + +function progress( + id: string, + source: z.infer["source"] = "sync", +): z.infer { + return { + type: "tool_progress", + turnId: TURN_ID, + ts: TS, + toolCallId: id, + source, + progress: { pct: 50 }, + }; +} + +function result( + id: string, + name: string, + source: z.infer["source"] = "sync", + output: JsonValue = "ok", + isError = false, +): z.infer { + return { + type: "tool_result", + turnId: TURN_ID, + ts: TS, + toolCallId: id, + toolName: name, + source, + result: { output, isError }, + }; +} + +function suspendedEv( + perms: Array<{ id: string; name: string }>, + asyncs: Array<{ id: string; tool: z.infer }>, +): z.infer { + return { + type: "turn_suspended", + turnId: TURN_ID, + ts: TS, + pendingPermissions: perms.map((p) => ({ + toolCallId: p.id, + toolName: p.name, + request: { tool: p.name }, + })), + pendingAsyncTools: asyncs.map((a) => ({ + toolCallId: a.id, + toolId: a.tool.toolId, + toolName: a.tool.name, + input: {}, + })), + usage: {}, + }; +} + +function turnCompletedEv( + output: z.infer["output"] = assistantText("done"), +): z.infer { + return { + type: "turn_completed", + turnId: TURN_ID, + ts: TS, + output, + finishReason: "stop", + usage: {}, + }; +} + +function turnFailedEv( + error = "it broke", + code?: string, +): z.infer { + return { + type: "turn_failed", + turnId: TURN_ID, + ts: TS, + error, + ...(code === undefined ? {} : { code }), + usage: {}, + }; +} + +function turnCancelledEv(reason?: string): z.infer { + return { + type: "turn_cancelled", + turnId: TURN_ID, + ts: TS, + ...(reason === undefined ? {} : { reason }), + usage: {}, + }; +} + +// A complete happy-path sequence with one sync tool round trip. +function syncToolSequence(): TEvent[] { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + return [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + permResolved("tc1", "allow"), + invocation("tc1"), + result("tc1", "echo"), + requested(1, [user("hello"), call0, toolMsg("tc1", "echo")]), + completed(1, assistantText("done")), + turnCompletedEv(), + ]; +} + +function expectCorruption(events: TEvent[], match: string | RegExp): void { + expect(() => reduceTurn(events)).toThrowError(TurnCorruptionError); + expect(() => reduceTurn(events)).toThrowError(match); +} + +describe("schemas", () => { + it("every builder output round-trips through the TurnEvent schema", () => { + for (const event of syncToolSequence()) { + expect(TurnEvent.parse(event)).toEqual(event); + } + }); + + it("rejects system-role messages in inline context", () => { + const bad = created({ + context: [ + { role: "system", content: "sneaky" }, + ] as unknown as z.infer["context"], + }); + expect(() => TurnCreated.parse(bad)).toThrowError(); + }); +}); + +describe("plain completion", () => { + it("reduces a created-only log to an idle empty state", () => { + const state = reduceTurn([created()]); + expect(state.definition.turnId).toBe(TURN_ID); + expect(state.modelCalls).toEqual([]); + expect(state.toolCalls).toEqual([]); + expect(state.terminal).toBeUndefined(); + expect(deriveTurnStatus(state)).toBe("idle"); + }); + + it("reduces a plain model response to a completed turn", () => { + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + stepEvent(0), + completed(0, assistantText("done")), + turnCompletedEv(), + ]); + expect(state.modelCalls).toHaveLength(1); + expect(state.modelCalls[0].response).toEqual(assistantText("done")); + expect(state.modelCalls[0].finishReason).toBe("stop"); + expect(state.modelCalls[0].stepEvents).toHaveLength(1); + expect(state.terminal?.type).toBe("turn_completed"); + expect(deriveTurnStatus(state)).toBe("completed"); + }); + + it("aggregates usage across model calls", () => { + const state = reduceTurn(syncToolSequence()); + expect(state.usage).toEqual({ + inputTokens: 20, + outputTokens: 10, + totalTokens: 30, + }); + }); + + it("leaves usage fields undefined when never reported", () => { + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, assistantText("a"), { inputTokens: 5 }), + ]); + expect(state.usage).toEqual({ inputTokens: 5 }); + expect(state.usage.cachedInputTokens).toBeUndefined(); + }); +}); + +describe("tool execution", () => { + it("extracts tool calls with order, batch index, and descriptor identity", () => { + const call0 = assistantCalls( + toolCallPart("tc1", "echo", { text: "a" }), + toolCallPart("a1", "fetch", { url: "u" }), + ); + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + ]); + expect(state.toolCalls).toHaveLength(2); + expect(state.toolCalls[0]).toMatchObject({ + toolCallId: "tc1", + toolName: "echo", + toolId: "tool.echo", + execution: "sync", + modelCallIndex: 0, + order: 0, + input: { text: "a" }, + }); + expect(state.toolCalls[1]).toMatchObject({ + toolCallId: "a1", + execution: "async", + order: 1, + }); + }); + + it("leaves identity undefined for tools missing from the agent snapshot", () => { + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, assistantCalls(toolCallPart("x1", "unknown-tool"))), + result("x1", "unknown-tool", "runtime", "No such tool", true), + ]); + expect(state.toolCalls[0].toolId).toBeUndefined(); + expect(state.toolCalls[0].execution).toBeUndefined(); + expect(state.toolCalls[0].result?.result.isError).toBe(true); + }); + + it("reduces a full sync tool round trip", () => { + const state = reduceTurn(syncToolSequence()); + const tc = state.toolCalls[0]; + expect(tc.permission?.required).toBeDefined(); + expect(tc.permission?.resolved?.decision).toBe("allow"); + expect(tc.invocation).toBeDefined(); + expect(tc.result?.source).toBe("sync"); + expect(state.terminal?.type).toBe("turn_completed"); + }); + + it("records classifier provenance separately from the effective decision", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + const state = reduceTurn([ + created({ config: { autoPermission: true, humanAvailable: true, maxModelCalls: 20 } }), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + permClassified("tc1", "allow"), + permResolved("tc1", "allow", "classifier"), + invocation("tc1"), + result("tc1", "echo"), + ]); + const permission = state.toolCalls[0].permission; + expect(permission?.classification?.decision).toBe("allow"); + expect(permission?.resolved?.source).toBe("classifier"); + }); + + it("accepts classification failure as audit and continues to human resolution", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + permClassificationFailed(["tc1"]), + permResolved("tc1", "deny", "human"), + result("tc1", "echo", "runtime", "Permission denied", true), + ]); + expect(state.toolCalls[0].result?.result.isError).toBe(true); + }); + + it("accumulates tool progress", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + progress("tc1"), + progress("tc1"), + result("tc1", "echo"), + ]); + expect(state.toolCalls[0].progress).toHaveLength(2); + }); + + it("accepts denial results without invocation (runtime source)", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + permResolved("tc1", "deny"), + result("tc1", "echo", "runtime", "Permission denied", true), + ]); + expect(state.toolCalls[0].invocation).toBeUndefined(); + expect(state.toolCalls[0].result?.source).toBe("runtime"); + }); +}); + +describe("context references", () => { + it("accepts a referenced context with matching contextRef on requests", () => { + const state = reduceTurn([ + created({ context: { previousTurnId: PREV_TURN_ID } }), + requested(0, [user("hello")], { + contextRef: { previousTurnId: PREV_TURN_ID }, + }), + completed(0, assistantText("done")), + turnCompletedEv(), + ]); + expect(deriveTurnStatus(state)).toBe("completed"); + }); + + it("rejects a request missing the contextRef when context is a reference", () => { + expectCorruption( + [ + created({ context: { previousTurnId: PREV_TURN_ID } }), + requested(0, [user("hello")]), + ], + /contextRef inconsistent/, + ); + }); + + it("rejects a request whose contextRef targets the wrong turn", () => { + expectCorruption( + [ + created({ context: { previousTurnId: PREV_TURN_ID } }), + requested(0, [user("hello")], { + contextRef: { previousTurnId: "some-other-turn" }, + }), + ], + /contextRef inconsistent/, + ); + }); + + it("rejects a contextRef when the turn context is inline", () => { + expectCorruption( + [ + created(), + requested(0, [user("hello")], { + contextRef: { previousTurnId: PREV_TURN_ID }, + }), + ], + /contextRef but turn context is inline/, + ); + }); + + it("ignores tool messages inside an inline context prefix when checking order", () => { + const context = [ + user("earlier"), + assistantCalls(toolCallPart("old1", "echo")), + toolMsg("old1", "echo"), + ]; + const state = reduceTurn([ + created({ context }), + requested(0, [...context, user("hello")]), + completed(0, assistantText("done")), + turnCompletedEv(), + ]); + expect(deriveTurnStatus(state)).toBe("completed"); + }); +}); + +describe("suspension", () => { + it("accepts a snapshot matching pending permissions and async tools", () => { + const call0 = assistantCalls( + toolCallPart("tc1", "echo"), + toolCallPart("a1", "fetch"), + ); + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + invocation("a1", fetchTool), + suspendedEv([{ id: "tc1", name: "echo" }], [{ id: "a1", tool: fetchTool }]), + ]); + expect(state.suspension?.pendingPermissions).toHaveLength(1); + expect(state.suspension?.pendingAsyncTools).toHaveLength(1); + expect(deriveTurnStatus(state)).toBe("suspended"); + }); + + it("replaces the snapshot as inputs arrive, one at a time", () => { + const call0 = assistantCalls( + toolCallPart("tc1", "echo"), + toolCallPart("a1", "fetch"), + ); + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + invocation("a1", fetchTool), + suspendedEv([{ id: "tc1", name: "echo" }], [{ id: "a1", tool: fetchTool }]), + result("a1", "fetch", "async", { data: 1 }), + suspendedEv([{ id: "tc1", name: "echo" }], []), + ]); + expect(state.suspension?.pendingAsyncTools).toHaveLength(0); + expect(outstandingPermissions(state)).toHaveLength(1); + expect(outstandingAsyncTools(state)).toHaveLength(0); + }); + + it("supports async results arriving in any order before the next model call", () => { + const call0 = assistantCalls( + toolCallPart("a1", "fetch"), + toolCallPart("a2", "fetch"), + ); + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("a1", fetchTool), + invocation("a2", fetchTool), + suspendedEv([], [{ id: "a1", tool: fetchTool }, { id: "a2", tool: fetchTool }]), + result("a2", "fetch", "async", "second"), + suspendedEv([], [{ id: "a1", tool: fetchTool }]), + result("a1", "fetch", "async", "first"), + requested(1, [ + user("hello"), + call0, + toolMsg("a1", "fetch", "first"), + toolMsg("a2", "fetch", "second"), + ]), + completed(1, assistantText("done")), + turnCompletedEv(), + ]); + expect(deriveTurnStatus(state)).toBe("completed"); + }); + + it("rejects a snapshot claiming already-resolved work", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + permResolved("tc1", "allow"), + invocation("tc1"), + result("tc1", "echo"), + suspendedEv([{ id: "tc1", name: "echo" }], []), + ], + /suspension without pending external work/, + ); + }); + + it("rejects a snapshot omitting pending work", () => { + const call0 = assistantCalls( + toolCallPart("tc1", "echo"), + toolCallPart("a1", "fetch"), + ); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + invocation("a1", fetchTool), + suspendedEv([{ id: "tc1", name: "echo" }], []), + ], + /suspension snapshot inconsistent/, + ); + }); + + it("rejects suspension while a model call is unsettled", () => { + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + suspendedEv([{ id: "tc1", name: "echo" }], []), + ], + /suspension while a model call is unsettled/, + ); + }); +}); + +describe("recovery-shaped histories", () => { + it("accepts an interrupted model call closed and re-issued (§23 fix)", () => { + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + callFailed(0, "interrupted by process restart"), + requested(1, [user("hello")]), + completed(1, assistantText("done")), + turnCompletedEv(), + ]); + expect(state.modelCalls[0].error).toMatch(/interrupted/); + expect(state.modelCalls[1].response).toEqual(assistantText("done")); + expect(deriveTurnStatus(state)).toBe("completed"); + }); + + it("re-issued model calls count against maxModelCalls", () => { + expectCorruption( + [ + created({ + config: { autoPermission: false, humanAvailable: true, maxModelCalls: 1 }, + }), + requested(0, [user("hello")]), + callFailed(0, "interrupted"), + requested(1, [user("hello")]), + ], + /exceeds maxModelCalls/, + ); + }); + + it("accepts an interrupted sync tool closed with an indeterminate runtime result, turn continuing (§23 fix)", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + result( + "tc1", + "echo", + "runtime", + "Tool execution was interrupted; its outcome is unknown and it was not retried.", + true, + ), + requested(1, [user("hello"), call0, toolMsg("tc1", "echo", "…")]), + completed(1, assistantText("done")), + turnCompletedEv(), + ]); + expect(state.toolCalls[0].result?.source).toBe("runtime"); + expect(state.terminal?.type).toBe("turn_completed"); + }); + + it("accepts cancellation with synthetic runtime results for unresolved calls", () => { + const call0 = assistantCalls(toolCallPart("a1", "fetch")); + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("a1", fetchTool), + suspendedEv([], [{ id: "a1", tool: fetchTool }]), + result("a1", "fetch", "runtime", "Cancelled", true), + turnCancelledEv("user stop"), + ]); + expect(state.terminal?.type).toBe("turn_cancelled"); + expect(deriveTurnStatus(state)).toBe("cancelled"); + }); + + it("accepts a live model failure closing the turn", () => { + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + callFailed(0), + turnFailedEv("provider exploded"), + ]); + expect(deriveTurnStatus(state)).toBe("failed"); + }); + + it("accepts model-call-limit exhaustion with a machine-readable code", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + const state = reduceTurn([ + created({ + config: { autoPermission: false, humanAvailable: true, maxModelCalls: 1 }, + }), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + result("tc1", "echo"), + turnFailedEv("model call limit reached", MODEL_CALL_LIMIT_ERROR_CODE), + ]); + expect(state.terminal?.type).toBe("turn_failed"); + expect( + state.terminal?.type === "turn_failed" ? state.terminal.code : undefined, + ).toBe(MODEL_CALL_LIMIT_ERROR_CODE); + }); +}); + +describe("invariants", () => { + it("rejects an empty log", () => { + expectCorruption([], /empty/); + }); + + it("rejects a log not starting with turn_created", () => { + expectCorruption([requested(0, [user("hello")])], /first event must be turn_created/); + }); + + it("rejects duplicate turn_created", () => { + expectCorruption([created(), created()], /duplicate turn_created/); + }); + + it("rejects mismatched turn ids", () => { + expectCorruption( + [created(), { ...requested(0, [user("hello")]), turnId: "other" }], + /does not match turn/, + ); + }); + + it("rejects unsupported schema versions", () => { + const bad = { + ...created(), + schemaVersion: 2, + } as unknown as TEvent; + expectCorruption([bad], /unsupported turn schema version/); + }); + + it("rejects unknown event types", () => { + const bad = { + type: "wat", + turnId: TURN_ID, + ts: TS, + } as unknown as TEvent; + expectCorruption([created(), bad], /unknown turn event type/); + }); + + it("rejects out-of-order model call indices", () => { + expectCorruption( + [created(), requested(1, [user("hello")])], + /out of order/, + ); + }); + + it("rejects a reused model call index", () => { + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, assistantText("a")), + requested(0, [user("hello")]), + ], + /out of order/, + ); + }); + + it("rejects concurrent unresolved model requests", () => { + expectCorruption( + [created(), requested(0, [user("hello")]), requested(1, [user("hello")])], + /concurrent unresolved model call requests/, + ); + }); + + it("rejects completion without a matching request", () => { + expectCorruption( + [created(), completed(0, assistantText("a"))], + /without matching request/, + ); + }); + + it("rejects failure without a matching request", () => { + expectCorruption([created(), callFailed(0)], /without matching request/); + }); + + it("rejects duplicate model call completion", () => { + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, assistantText("a")), + completed(0, assistantText("b")), + ], + /duplicate settlement/, + ); + }); + + it("rejects completion after failure of the same call", () => { + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + callFailed(0), + completed(0, assistantText("a")), + ], + /duplicate settlement/, + ); + }); + + it("rejects the next model call while tool calls are unresolved", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + requested(1, [user("hello"), call0]), + ], + /while tool calls are unresolved/, + ); + }); + + it("rejects a model call past the budget", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created({ + config: { autoPermission: false, humanAvailable: true, maxModelCalls: 1 }, + }), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + result("tc1", "echo"), + requested(1, [user("hello"), call0, toolMsg("tc1", "echo")]), + ], + /exceeds maxModelCalls/, + ); + }); + + it("rejects duplicate tool call ids within one response", () => { + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed( + 0, + assistantCalls(toolCallPart("tc1", "echo"), toolCallPart("tc1", "echo")), + ), + ], + /duplicate tool call id/, + ); + }); + + it("rejects duplicate tool call ids across responses", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + result("tc1", "echo"), + requested(1, [user("hello"), call0, toolMsg("tc1", "echo")]), + completed(1, assistantCalls(toolCallPart("tc1", "echo"))), + ], + /duplicate tool call id/, + ); + }); + + it("rejects tool-result ordering that differs from original call order", () => { + const call0 = assistantCalls( + toolCallPart("tc1", "echo"), + toolCallPart("tc2", "echo"), + ); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + result("tc1", "echo"), + invocation("tc2"), + result("tc2", "echo"), + requested(1, [ + user("hello"), + call0, + toolMsg("tc2", "echo"), + toolMsg("tc1", "echo"), + ]), + ], + /tool-result order differs/, + ); + }); + + it("rejects permission records targeting unknown tool calls", () => { + expectCorruption( + [created(), requested(0, [user("hello")]), completed(0, assistantText("a")), permRequired("ghost", "echo")], + /unknown tool call/, + ); + }); + + it("rejects duplicate permission requirements", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + permRequired("tc1", "echo"), + ], + /duplicate permission requirement/, + ); + }); + + it("rejects classification without a requirement", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permClassified("tc1", "allow"), + ], + /classification without permission requirement/, + ); + }); + + it("rejects resolution without a requirement", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permResolved("tc1", "allow"), + ], + /resolution without requirement/, + ); + }); + + it("rejects conflicting effective permission decisions", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + permResolved("tc1", "allow"), + permResolved("tc1", "deny"), + ], + /conflicting permission decisions/, + ); + }); + + it("rejects invocation while permission is pending", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + invocation("tc1"), + ], + /invocation without permission allowance/, + ); + }); + + it("rejects invocation of a denied tool call", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + permResolved("tc1", "deny"), + invocation("tc1"), + ], + /invocation without permission allowance/, + ); + }); + + it("rejects duplicate invocations", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + invocation("tc1"), + ], + /duplicate tool invocation/, + ); + }); + + it("rejects invocations whose execution mode contradicts the snapshot", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1", echoTool, { execution: "async" }), + ], + /execution mismatch/, + ); + }); + + it("rejects progress without invocation", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [created(), requested(0, [user("hello")]), completed(0, call0), progress("tc1")], + /progress without invocation/, + ); + }); + + it("rejects progress after a terminal tool result", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + result("tc1", "echo"), + progress("tc1"), + ], + /progress after terminal result/, + ); + }); + + it("rejects duplicate tool results", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + result("tc1", "echo"), + result("tc1", "echo"), + ], + /duplicate tool result/, + ); + }); + + it("rejects sync-sourced results without invocation", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [created(), requested(0, [user("hello")]), completed(0, call0), result("tc1", "echo", "sync")], + /sync tool result without invocation/, + ); + }); + + it("rejects async results for sync tools", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + result("tc1", "echo", "async"), + ], + /result source mismatch/, + ); + }); + + it("rejects step events without a matching open call", () => { + expectCorruption([created(), stepEvent(0)], /without matching model call request/); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, assistantText("a")), + stepEvent(0), + ], + /after model call 0 settled/, + ); + }); + + it("rejects completion while tool calls remain unresolved", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [created(), requested(0, [user("hello")]), completed(0, call0), turnCompletedEv()], + /completion while tool calls lack terminal results/, + ); + }); + + it("rejects completion when the final response has tool calls", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + result("tc1", "echo"), + turnCompletedEv(), + ], + /final response has tool calls/, + ); + }); + + it("rejects completion without any completed model response", () => { + expectCorruption([created(), turnCompletedEv()], /without a completed model response/); + }); + + it("rejects terminal failure while tool calls remain unresolved", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + expectCorruption( + [created(), requested(0, [user("hello")]), completed(0, call0), turnFailedEv()], + /failure while tool calls lack terminal results/, + ); + }); + + it("rejects terminal events while a model call is unsettled", () => { + expectCorruption( + [created(), requested(0, [user("hello")]), turnCancelledEv()], + /cancellation while a model call is unsettled/, + ); + }); + + it("rejects any event after a terminal event", () => { + const base = [ + created(), + requested(0, [user("hello")]), + completed(0, assistantText("a")), + ]; + for (const terminal of [turnCompletedEv(), turnFailedEv(), turnCancelledEv()]) { + expectCorruption( + [...base, terminal, stepEvent(0)], + /event after terminal turn event/, + ); + } + }); + + it("rejects multiple terminal events", () => { + expectCorruption( + [ + created(), + requested(0, [user("hello")]), + completed(0, assistantText("a")), + turnCompletedEv(), + turnFailedEv(), + ], + /event after terminal turn event/, + ); + }); +}); + +describe("derivations", () => { + it("derives suspended for pending permissions and idle otherwise", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + const pending = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + permRequired("tc1", "echo"), + suspendedEv([{ id: "tc1", name: "echo" }], []), + ]); + expect(deriveTurnStatus(pending)).toBe("suspended"); + + const idle = reduceTurn([created(), requested(0, [user("hello")]), completed(0, call0)]); + expect(deriveTurnStatus(idle)).toBe("idle"); + }); + + it("builds the turn transcript in source order with serialized results", () => { + const call0 = assistantCalls( + toolCallPart("tc1", "echo"), + toolCallPart("tc2", "echo"), + ); + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + invocation("tc1"), + invocation("tc2"), + result("tc2", "echo", "sync", { n: 2 }), + result("tc1", "echo", "sync", "plain text"), + requested(1, [ + user("hello"), + call0, + toolMsg("tc1", "echo", "plain text"), + toolMsg("tc2", "echo", '{"n":2}'), + ]), + completed(1, assistantText("done")), + turnCompletedEv(), + ]); + expect(turnTranscript(state)).toEqual([ + user("hello"), + call0, + { role: "tool", content: "plain text", toolCallId: "tc1", toolName: "echo" }, + { role: "tool", content: '{"n":2}', toolCallId: "tc2", toolName: "echo" }, + assistantText("done"), + ]); + }); + + it("omits failed model calls from the transcript", () => { + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + callFailed(0, "interrupted"), + requested(1, [user("hello")]), + completed(1, assistantText("done")), + turnCompletedEv(), + ]); + expect(turnTranscript(state)).toEqual([user("hello"), assistantText("done")]); + }); + + it("transcript excludes the context prefix", () => { + const state = reduceTurn([ + created({ context: { previousTurnId: PREV_TURN_ID } }), + requested(0, [user("hello")], { + contextRef: { previousTurnId: PREV_TURN_ID }, + }), + completed(0, assistantText("done")), + turnCompletedEv(), + ]); + expect(turnTranscript(state)[0]).toEqual(user("hello")); + expect(turnTranscript(state)).toHaveLength(2); + }); + + it("transcript throws on unresolved tool calls", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + const state = reduceTurn([ + created(), + requested(0, [user("hello")]), + completed(0, call0), + ]); + expect(() => turnTranscript(state)).toThrowError(/unresolved/); + }); +}); diff --git a/apps/x/packages/shared/src/turns.ts b/apps/x/packages/shared/src/turns.ts new file mode 100644 index 00000000..52e39c23 --- /dev/null +++ b/apps/x/packages/shared/src/turns.ts @@ -0,0 +1,958 @@ +import { z } from "zod"; +import { + AssistantMessage, + ToolCallPart, + ToolMessage, + UserMessage, +} from "./message.js"; + +// Durable turn contract for the turn runtime (see +// packages/core/docs/turn-runtime-design.md). This module is the +// cross-boundary source of truth shared by core and renderer: event schemas, +// the pure reducer, and pure derivations over the reduced state. It must +// stay free of I/O and node-only imports so the renderer can consume it +// directly. + +export type JsonValue = z.infer>; + +export const DEFAULT_MAX_MODEL_CALLS = 20; +export const MODEL_CALL_LIMIT_ERROR_CODE = "model-call-limit"; + +// --------------------------------------------------------------------------- +// Agent snapshot +// --------------------------------------------------------------------------- + +export const ModelDescriptor = z.object({ + provider: z.string(), + model: z.string(), +}); + +export const RequestedAgent = z.object({ + agentId: z.string(), + overrides: z + .object({ + model: ModelDescriptor.optional(), + }) + .optional(), +}); + +export const ToolDescriptor = z.object({ + toolId: z.string(), + name: z.string(), + description: z.string(), + inputSchema: z.json(), + execution: z.enum(["sync", "async"]), + requiresHuman: z.boolean(), +}); + +export const ResolvedAgent = z.object({ + agentId: z.string(), + systemPrompt: z.string(), + model: ModelDescriptor, + tools: z.array(ToolDescriptor), +}); + +// --------------------------------------------------------------------------- +// Context +// --------------------------------------------------------------------------- + +// Context excludes system-role messages: the resolved agent owns the single +// authoritative system prompt. +export const ConversationMessage = z.discriminatedUnion("role", [ + UserMessage, + AssistantMessage, + ToolMessage, +]); + +// A reference points at the immediately preceding turn; its materialized +// value is that turn's transcript (context + input + produced messages). +// Inline arrays are used by standalone turns and by callers that assemble +// context themselves. Resolution is a runtime concern; the reducer treats +// context as opaque. +export const TurnContextRef = z.object({ + previousTurnId: z.string(), +}); + +export const TurnContext = z.union([ + TurnContextRef, + z.array(ConversationMessage), +]); + +// --------------------------------------------------------------------------- +// Usage +// --------------------------------------------------------------------------- + +export const TurnUsage = z.object({ + inputTokens: z.number().optional(), + outputTokens: z.number().optional(), + totalTokens: z.number().optional(), + reasoningTokens: z.number().optional(), + cachedInputTokens: z.number().optional(), +}); + +// --------------------------------------------------------------------------- +// Durable events +// --------------------------------------------------------------------------- + +export const TurnCreated = z.object({ + type: z.literal("turn_created"), + schemaVersion: z.literal(1), + turnId: z.string(), + ts: z.string(), + sessionId: z.string().nullable(), + agent: z.object({ + requested: RequestedAgent, + resolved: ResolvedAgent, + }), + context: TurnContext, + input: UserMessage, + config: z.object({ + autoPermission: z.boolean(), + humanAvailable: z.boolean(), + maxModelCalls: z.number().int().positive(), + }), +}); + +// `messages` contains current-turn messages only; when the turn's context is +// a reference, `contextRef` repeats it and the resolved prefix is implicit +// (deterministically materializable because referenced turns are immutable). +// When context is inline, `contextRef` is absent and `messages` begins with +// the inline context. +export const ModelRequest = z.object({ + systemPrompt: z.string(), + contextRef: TurnContextRef.optional(), + messages: z.array(ConversationMessage), + tools: z.array(ToolDescriptor), + parameters: z.record(z.string(), z.json()), +}); + +export const ModelCallRequested = z.object({ + type: z.literal("model_call_requested"), + turnId: z.string(), + ts: z.string(), + modelCallIndex: z.number().int().nonnegative(), + request: ModelRequest, +}); + +// Normalized provider step events kept for debugging. Raw text/reasoning +// deltas are stream-only and never appear here. +export const DurableLlmStepStreamEvent = z.discriminatedUnion("type", [ + z.object({ type: z.literal("text_start") }), + z.object({ type: z.literal("text_end"), text: z.string() }), + z.object({ type: z.literal("reasoning_start") }), + z.object({ type: z.literal("reasoning_end"), text: z.string() }), + z.object({ type: z.literal("tool_call"), toolCall: ToolCallPart }), + z.object({ + type: z.literal("finish_step"), + finishReason: z.string(), + usage: TurnUsage.optional(), + providerMetadata: z.json().optional(), + }), + z.object({ type: z.literal("provider_error"), error: z.string() }), +]); + +export const ModelStepEvent = z.object({ + type: z.literal("model_step_event"), + turnId: z.string(), + ts: z.string(), + modelCallIndex: z.number().int().nonnegative(), + event: DurableLlmStepStreamEvent, +}); + +export const ModelCallCompleted = z.object({ + type: z.literal("model_call_completed"), + turnId: z.string(), + ts: z.string(), + modelCallIndex: z.number().int().nonnegative(), + message: AssistantMessage, + finishReason: z.string(), + usage: TurnUsage, + providerMetadata: z.json().optional(), +}); + +export const ModelCallFailed = z.object({ + type: z.literal("model_call_failed"), + turnId: z.string(), + ts: z.string(), + modelCallIndex: z.number().int().nonnegative(), + error: z.string(), +}); + +export const ToolPermissionRequired = z.object({ + type: z.literal("tool_permission_required"), + turnId: z.string(), + ts: z.string(), + toolCallId: z.string(), + toolName: z.string(), + request: z.json(), + checkerError: z.string().optional(), +}); + +export const ToolPermissionClassified = z.object({ + type: z.literal("tool_permission_classified"), + turnId: z.string(), + ts: z.string(), + toolCallId: z.string(), + decision: z.enum(["allow", "deny", "defer"]), + reason: z.string(), +}); + +export const ToolPermissionClassificationFailed = z.object({ + type: z.literal("tool_permission_classification_failed"), + turnId: z.string(), + ts: z.string(), + toolCallIds: z.array(z.string()), + error: z.string(), +}); + +// The only effective execution decision; classifier records are provenance. +export const ToolPermissionResolved = z.object({ + type: z.literal("tool_permission_resolved"), + turnId: z.string(), + ts: z.string(), + toolCallId: z.string(), + decision: z.enum(["allow", "deny"]), + source: z.enum(["classifier", "human", "human_unavailable"]), + reason: z.string().optional(), + metadata: z.json().optional(), +}); + +export const ToolInvocationRequested = z.object({ + type: z.literal("tool_invocation_requested"), + turnId: z.string(), + ts: z.string(), + toolCallId: z.string(), + toolId: z.string(), + toolName: z.string(), + execution: z.enum(["sync", "async"]), + input: z.json(), +}); + +export const ToolProgress = z.object({ + type: z.literal("tool_progress"), + turnId: z.string(), + ts: z.string(), + toolCallId: z.string(), + source: z.enum(["sync", "async"]), + progress: z.json(), +}); + +export const ToolResultData = z.object({ + output: z.json(), + isError: z.boolean(), + metadata: z.json().optional(), +}); + +// `runtime` results cover permission denial, unknown tools, unusable calls, +// human unavailability, cancellation, and interrupted sync execution. A +// result may exist without an invocation when execution was rejected before +// dispatch. +export const ToolResult = z.object({ + type: z.literal("tool_result"), + turnId: z.string(), + ts: z.string(), + toolCallId: z.string(), + toolName: z.string(), + source: z.enum(["sync", "async", "runtime"]), + result: ToolResultData, +}); + +export const TurnSuspended = z.object({ + type: z.literal("turn_suspended"), + turnId: z.string(), + ts: z.string(), + pendingPermissions: z.array( + z.object({ + toolCallId: z.string(), + toolName: z.string(), + request: z.json(), + }), + ), + pendingAsyncTools: z.array( + z.object({ + toolCallId: z.string(), + toolId: z.string(), + toolName: z.string(), + input: z.json(), + }), + ), + usage: TurnUsage, +}); + +export const TurnCompleted = z.object({ + type: z.literal("turn_completed"), + turnId: z.string(), + ts: z.string(), + output: AssistantMessage, + finishReason: z.string(), + usage: TurnUsage, +}); + +export const TurnFailed = z.object({ + type: z.literal("turn_failed"), + turnId: z.string(), + ts: z.string(), + error: z.string(), + // Machine-readable discriminator for failures callers must tell apart; + // MODEL_CALL_LIMIT_ERROR_CODE is defined by the spec. + code: z.string().optional(), + usage: TurnUsage, +}); + +export const TurnCancelled = z.object({ + type: z.literal("turn_cancelled"), + turnId: z.string(), + ts: z.string(), + reason: z.string().optional(), + usage: TurnUsage, +}); + +export const TurnEvent = z.discriminatedUnion("type", [ + TurnCreated, + ModelCallRequested, + ModelStepEvent, + ModelCallCompleted, + ModelCallFailed, + ToolPermissionRequired, + ToolPermissionClassified, + ToolPermissionClassificationFailed, + ToolPermissionResolved, + ToolInvocationRequested, + ToolProgress, + ToolResult, + TurnSuspended, + TurnCompleted, + TurnFailed, + TurnCancelled, +]); + +// --------------------------------------------------------------------------- +// Ephemeral stream-only deltas (never persisted) +// --------------------------------------------------------------------------- + +export type TextDelta = { + type: "text_delta"; + turnId: string; + modelCallIndex: number; + delta: string; +}; + +export type ReasoningDelta = { + type: "reasoning_delta"; + turnId: string; + modelCallIndex: number; + delta: string; +}; + +export type TurnStreamEvent = + | z.infer + | TextDelta + | ReasoningDelta; + +// --------------------------------------------------------------------------- +// Derived turn state +// --------------------------------------------------------------------------- + +export class TurnCorruptionError extends Error { + constructor(message: string) { + super(message); + this.name = "TurnCorruptionError"; + } +} + +export interface ModelCallState { + index: number; + request: z.infer; + stepEvents: Array>; + response?: z.infer; + finishReason?: string; + usage?: z.infer; + providerMetadata?: JsonValue; + error?: string; +} + +export interface ToolCallState { + modelCallIndex: number; + order: number; + toolCallId: string; + toolName: string; + input: unknown; + toolId?: string; + execution?: "sync" | "async"; + permission?: { + required: z.infer; + classification?: z.infer; + resolved?: z.infer; + }; + invocation?: z.infer; + progress: Array>; + result?: z.infer; +} + +export interface TurnState { + definition: z.infer; + modelCalls: ModelCallState[]; + toolCalls: ToolCallState[]; + suspension?: z.infer; + terminal?: + | z.infer + | z.infer + | z.infer; + usage: z.infer; +} + +function fail(message: string): never { + throw new TurnCorruptionError(message); +} + +function isContextRef( + context: z.infer, +): context is z.infer { + return !Array.isArray(context); +} + +function findToolCall(state: TurnState, toolCallId: string): ToolCallState { + const toolCall = state.toolCalls.find( + (tc) => tc.toolCallId === toolCallId, + ); + if (!toolCall) { + fail(`event targets unknown tool call: ${toolCallId}`); + } + return toolCall; +} + +function openModelCall(state: TurnState): ModelCallState | undefined { + const last = state.modelCalls[state.modelCalls.length - 1]; + if (last && last.response === undefined && last.error === undefined) { + return last; + } + return undefined; +} + +function batchToolCalls(state: TurnState, modelCallIndex: number): ToolCallState[] { + return state.toolCalls + .filter((tc) => tc.modelCallIndex === modelCallIndex) + .sort((a, b) => a.order - b.order); +} + +function addUsage( + total: z.infer, + usage: z.infer, +): void { + const keys = [ + "inputTokens", + "outputTokens", + "totalTokens", + "reasoningTokens", + "cachedInputTokens", + ] as const; + for (const key of keys) { + const value = usage[key]; + if (value !== undefined) { + total[key] = (total[key] ?? 0) + value; + } + } +} + +function applyModelCallRequested( + state: TurnState, + event: z.infer, +): void { + const expectedIndex = state.modelCalls.length; + if (event.modelCallIndex !== expectedIndex) { + fail( + `model call index ${event.modelCallIndex} out of order; expected ${expectedIndex}`, + ); + } + if (expectedIndex >= state.definition.config.maxModelCalls) { + fail( + `model call index ${event.modelCallIndex} exceeds maxModelCalls ${state.definition.config.maxModelCalls}`, + ); + } + if (openModelCall(state)) { + fail("concurrent unresolved model call requests"); + } + const unresolved = state.toolCalls.filter((tc) => !tc.result); + if (unresolved.length > 0) { + fail( + `model call requested while tool calls are unresolved: ${unresolved + .map((tc) => tc.toolCallId) + .join(", ")}`, + ); + } + + const context = state.definition.context; + if (isContextRef(context)) { + if (event.request.contextRef?.previousTurnId !== context.previousTurnId) { + fail("model request contextRef inconsistent with turn context"); + } + } else if (event.request.contextRef !== undefined) { + fail("model request has contextRef but turn context is inline"); + } + + // Model-facing tool-result order must match original call order: the tool + // messages in the request (past any inline context prefix) must be the + // concatenated batches of prior completed calls, in source order. + const prefixLength = isContextRef(context) ? 0 : context.length; + const toolMessageIds = event.request.messages + .slice(prefixLength) + .filter((m) => m.role === "tool") + .map((m) => m.toolCallId); + const expectedIds = state.modelCalls + .filter((call) => call.response !== undefined) + .flatMap((call) => + batchToolCalls(state, call.index).map((tc) => tc.toolCallId), + ); + if ( + toolMessageIds.length !== expectedIds.length || + toolMessageIds.some((id, i) => id !== expectedIds[i]) + ) { + fail("model request tool-result order differs from original call order"); + } + + state.modelCalls.push({ + index: event.modelCallIndex, + request: event.request, + stepEvents: [], + }); +} + +function applyModelStepEvent( + state: TurnState, + event: z.infer, +): void { + const call = state.modelCalls[event.modelCallIndex]; + if (!call) { + fail(`step event without matching model call request: ${event.modelCallIndex}`); + } + if (call.response !== undefined || call.error !== undefined) { + fail(`step event after model call ${event.modelCallIndex} settled`); + } + call.stepEvents.push(event.event); +} + +function applyModelCallCompleted( + state: TurnState, + event: z.infer, +): void { + const call = state.modelCalls[event.modelCallIndex]; + if (!call) { + fail(`model call completion without matching request: ${event.modelCallIndex}`); + } + if (call.response !== undefined || call.error !== undefined) { + fail(`duplicate settlement for model call ${event.modelCallIndex}`); + } + call.response = event.message; + call.finishReason = event.finishReason; + call.usage = event.usage; + call.providerMetadata = event.providerMetadata; + addUsage(state.usage, event.usage); + + const parts = Array.isArray(event.message.content) + ? event.message.content + : []; + let order = 0; + for (const part of parts) { + if (part.type !== "tool-call") { + continue; + } + if (state.toolCalls.some((tc) => tc.toolCallId === part.toolCallId)) { + fail(`duplicate tool call id: ${part.toolCallId}`); + } + const descriptor = state.definition.agent.resolved.tools.find( + (tool) => tool.name === part.toolName, + ); + state.toolCalls.push({ + modelCallIndex: event.modelCallIndex, + order: order++, + toolCallId: part.toolCallId, + toolName: part.toolName, + input: part.arguments, + toolId: descriptor?.toolId, + execution: descriptor?.execution, + progress: [], + }); + } +} + +function applyModelCallFailed( + state: TurnState, + event: z.infer, +): void { + const call = state.modelCalls[event.modelCallIndex]; + if (!call) { + fail(`model call failure without matching request: ${event.modelCallIndex}`); + } + if (call.response !== undefined || call.error !== undefined) { + fail(`duplicate settlement for model call ${event.modelCallIndex}`); + } + call.error = event.error; +} + +function applyToolPermissionRequired( + state: TurnState, + event: z.infer, +): void { + const toolCall = findToolCall(state, event.toolCallId); + if (toolCall.result) { + fail(`permission requirement after tool result: ${event.toolCallId}`); + } + if (toolCall.invocation) { + fail(`permission requirement after invocation: ${event.toolCallId}`); + } + if (toolCall.permission) { + fail(`duplicate permission requirement: ${event.toolCallId}`); + } + toolCall.permission = { required: event }; +} + +function applyToolPermissionClassified( + state: TurnState, + event: z.infer, +): void { + const toolCall = findToolCall(state, event.toolCallId); + if (!toolCall.permission) { + fail(`classification without permission requirement: ${event.toolCallId}`); + } + if (toolCall.permission.resolved) { + fail(`classification after permission resolution: ${event.toolCallId}`); + } + if (toolCall.permission.classification) { + fail(`duplicate permission classification: ${event.toolCallId}`); + } + toolCall.permission.classification = event; +} + +function applyToolPermissionClassificationFailed( + state: TurnState, + event: z.infer, +): void { + for (const toolCallId of event.toolCallIds) { + const toolCall = findToolCall(state, toolCallId); + if (!toolCall.permission) { + fail(`classification failure without permission requirement: ${toolCallId}`); + } + if (toolCall.permission.resolved) { + fail(`classification failure after permission resolution: ${toolCallId}`); + } + } + // Recorded for the audit trail; does not currently affect derived state. +} + +function applyToolPermissionResolved( + state: TurnState, + event: z.infer, +): void { + const toolCall = findToolCall(state, event.toolCallId); + if (!toolCall.permission) { + fail(`permission resolution without requirement: ${event.toolCallId}`); + } + if (toolCall.permission.resolved) { + fail(`conflicting permission decisions: ${event.toolCallId}`); + } + if (toolCall.result) { + fail(`permission resolution after tool result: ${event.toolCallId}`); + } + toolCall.permission.resolved = event; +} + +function applyToolInvocationRequested( + state: TurnState, + event: z.infer, +): void { + const toolCall = findToolCall(state, event.toolCallId); + if (toolCall.invocation) { + fail(`duplicate tool invocation: ${event.toolCallId}`); + } + if (toolCall.result) { + fail(`tool invocation after result: ${event.toolCallId}`); + } + if (toolCall.toolName !== event.toolName) { + fail(`tool invocation name mismatch: ${event.toolCallId}`); + } + if (toolCall.permission && toolCall.permission.resolved?.decision !== "allow") { + fail(`tool invocation without permission allowance: ${event.toolCallId}`); + } + if (toolCall.execution !== undefined && toolCall.execution !== event.execution) { + fail(`tool invocation execution mismatch: ${event.toolCallId}`); + } + if (toolCall.toolId !== undefined && toolCall.toolId !== event.toolId) { + fail(`tool invocation toolId mismatch: ${event.toolCallId}`); + } + toolCall.execution = event.execution; + toolCall.toolId = event.toolId; + toolCall.invocation = event; +} + +function applyToolProgress( + state: TurnState, + event: z.infer, +): void { + const toolCall = findToolCall(state, event.toolCallId); + if (!toolCall.invocation) { + fail(`tool progress without invocation: ${event.toolCallId}`); + } + if (toolCall.result) { + fail(`tool progress after terminal result: ${event.toolCallId}`); + } + if (event.source !== toolCall.execution) { + fail(`tool progress source mismatch: ${event.toolCallId}`); + } + toolCall.progress.push(event); +} + +function applyToolResult( + state: TurnState, + event: z.infer, +): void { + const toolCall = findToolCall(state, event.toolCallId); + if (toolCall.result) { + fail(`duplicate tool result: ${event.toolCallId}`); + } + if (toolCall.toolName !== event.toolName) { + fail(`tool result name mismatch: ${event.toolCallId}`); + } + if (event.source !== "runtime") { + if (!toolCall.invocation) { + fail(`${event.source} tool result without invocation: ${event.toolCallId}`); + } + if (event.source !== toolCall.execution) { + fail(`tool result source mismatch: ${event.toolCallId}`); + } + } + toolCall.result = event; +} + +function sortedIds(ids: string[]): string { + return [...ids].sort().join(","); +} + +function applyTurnSuspended( + state: TurnState, + event: z.infer, +): void { + if (openModelCall(state)) { + fail("suspension while a model call is unsettled"); + } + const expectedPermissions = outstandingPermissions(state).map( + (tc) => tc.toolCallId, + ); + const expectedAsync = outstandingAsyncTools(state).map((tc) => tc.toolCallId); + if (expectedPermissions.length + expectedAsync.length === 0) { + fail("suspension without pending external work"); + } + const claimedPermissions = event.pendingPermissions.map((p) => p.toolCallId); + const claimedAsync = event.pendingAsyncTools.map((p) => p.toolCallId); + if ( + sortedIds(claimedPermissions) !== sortedIds(expectedPermissions) || + sortedIds(claimedAsync) !== sortedIds(expectedAsync) + ) { + fail("suspension snapshot inconsistent with pending state"); + } + state.suspension = event; +} + +function assertTerminalPreconditions(state: TurnState, kind: string): void { + if (openModelCall(state)) { + fail(`${kind} while a model call is unsettled`); + } + const unresolved = state.toolCalls.filter((tc) => !tc.result); + if (unresolved.length > 0) { + fail( + `${kind} while tool calls lack terminal results: ${unresolved + .map((tc) => tc.toolCallId) + .join(", ")}`, + ); + } +} + +function applyTurnCompleted( + state: TurnState, + event: z.infer, +): void { + assertTerminalPreconditions(state, "completion"); + const last = state.modelCalls[state.modelCalls.length - 1]; + if (!last || last.response === undefined) { + fail("completion without a completed model response"); + } + if (batchToolCalls(state, last.index).length > 0) { + fail("completion while the final response has tool calls"); + } + state.terminal = event; +} + +export function reduceTurn( + events: Array>, +): TurnState { + if (events.length === 0) { + fail("turn log is empty"); + } + const [first, ...rest] = events; + if (first.type !== "turn_created") { + fail(`first event must be turn_created, got ${first.type}`); + } + if (first.schemaVersion !== 1) { + fail(`unsupported turn schema version: ${String(first.schemaVersion)}`); + } + + const state: TurnState = { + definition: first, + modelCalls: [], + toolCalls: [], + usage: {}, + }; + + for (const event of rest) { + if (event.turnId !== first.turnId) { + fail( + `event turnId ${event.turnId} does not match turn ${first.turnId}`, + ); + } + if (state.terminal) { + fail(`event after terminal turn event: ${event.type}`); + } + switch (event.type) { + case "turn_created": + fail("duplicate turn_created event"); + break; + case "model_call_requested": + applyModelCallRequested(state, event); + break; + case "model_step_event": + applyModelStepEvent(state, event); + break; + case "model_call_completed": + applyModelCallCompleted(state, event); + break; + case "model_call_failed": + applyModelCallFailed(state, event); + break; + case "tool_permission_required": + applyToolPermissionRequired(state, event); + break; + case "tool_permission_classified": + applyToolPermissionClassified(state, event); + break; + case "tool_permission_classification_failed": + applyToolPermissionClassificationFailed(state, event); + break; + case "tool_permission_resolved": + applyToolPermissionResolved(state, event); + break; + case "tool_invocation_requested": + applyToolInvocationRequested(state, event); + break; + case "tool_progress": + applyToolProgress(state, event); + break; + case "tool_result": + applyToolResult(state, event); + break; + case "turn_suspended": + applyTurnSuspended(state, event); + break; + case "turn_completed": + applyTurnCompleted(state, event); + break; + case "turn_failed": + assertTerminalPreconditions(state, "failure"); + state.terminal = event; + break; + case "turn_cancelled": + assertTerminalPreconditions(state, "cancellation"); + state.terminal = event; + break; + default: { + const unknown: never = event; + fail(`unknown turn event type: ${(unknown as { type: string }).type}`); + } + } + } + + return state; +} + +// --------------------------------------------------------------------------- +// Pure derivations over TurnState +// --------------------------------------------------------------------------- + +export function outstandingPermissions(state: TurnState): ToolCallState[] { + return state.toolCalls.filter( + (tc) => tc.permission && !tc.permission.resolved && !tc.result, + ); +} + +export function outstandingAsyncTools(state: TurnState): ToolCallState[] { + return state.toolCalls.filter( + (tc) => tc.invocation && tc.execution === "async" && !tc.result, + ); +} + +export type TurnStatus = + | "completed" + | "failed" + | "cancelled" + | "suspended" + | "idle"; + +// No durable running status exists by design: an "idle" turn is simply +// non-terminal with no outstanding external work. Whether it is actively +// being advanced right now is ephemeral bus state. +export function deriveTurnStatus(state: TurnState): TurnStatus { + if (state.terminal) { + switch (state.terminal.type) { + case "turn_completed": + return "completed"; + case "turn_failed": + return "failed"; + case "turn_cancelled": + return "cancelled"; + } + } + if ( + outstandingPermissions(state).length > 0 || + outstandingAsyncTools(state).length > 0 + ) { + return "suspended"; + } + return "idle"; +} + +function toolResultContent(result: z.infer): string { + const output = result.result.output; + return typeof output === "string" ? output : JSON.stringify(output); +} + +// The messages this turn contributed to the conversation: its input plus, per +// completed model call, the assistant response and its tool results in source +// order. Failed model calls contribute nothing. The turn's context prefix is +// NOT included; materializing the full conversation is the context resolver's +// job (core). Requires every tool call to have a terminal result, which holds +// for all terminal turns. +export function turnTranscript( + state: TurnState, +): Array> { + const messages: Array> = [ + state.definition.input, + ]; + for (const call of state.modelCalls) { + if (call.response === undefined) { + continue; + } + messages.push(call.response); + for (const toolCall of batchToolCalls(state, call.index)) { + if (!toolCall.result) { + throw new Error( + `turnTranscript requires terminal tool results; ${toolCall.toolCallId} is unresolved`, + ); + } + messages.push({ + role: "tool", + content: toolResultContent(toolCall.result), + toolCallId: toolCall.toolCallId, + toolName: toolCall.toolName, + }); + } + } + return messages; +} diff --git a/apps/x/packages/shared/tsconfig.build.json b/apps/x/packages/shared/tsconfig.build.json new file mode 100644 index 00000000..4838eeca --- /dev/null +++ b/apps/x/packages/shared/tsconfig.build.json @@ -0,0 +1,7 @@ +{ + "extends": "./tsconfig.json", + "exclude": [ + "src/**/*.test.ts", + "src/**/*.spec.ts" + ] +} diff --git a/apps/x/packages/shared/vitest.config.ts b/apps/x/packages/shared/vitest.config.ts new file mode 100644 index 00000000..4064b801 --- /dev/null +++ b/apps/x/packages/shared/vitest.config.ts @@ -0,0 +1,11 @@ +import { defineConfig } from "vitest/config"; + +export default defineConfig({ + test: { + environment: "node", + include: ["src/**/*.test.ts", "src/**/*.spec.ts"], + globals: false, + clearMocks: true, + restoreMocks: true, + }, +}); diff --git a/apps/x/pnpm-lock.yaml b/apps/x/pnpm-lock.yaml index 5ad372b2..edd7e7b1 100644 --- a/apps/x/pnpm-lock.yaml +++ b/apps/x/pnpm-lock.yaml @@ -545,6 +545,10 @@ importers: zod: specifier: ^4.2.1 version: 4.2.1 + devDependencies: + vitest: + specifier: 'catalog:' + version: 4.1.7(@opentelemetry/api@1.9.0)(@types/node@25.0.3)(vite@7.3.0(@types/node@25.0.3)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2)) packages: From cda17c2d40300e929b12ade0ab6b0b3a9d765048 Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 09:32:27 +0530 Subject: [PATCH 05/24] feat(x/core): turn runtime with event-sourced execution loop (stage 2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The TurnRuntime per turn-runtime-design.md: constructor-injected, container-ignorant, no per-turn state — every advanceTurn reconstructs from the JSONL log, so normal execution and crash recovery share one path. - runtime.ts: createTurn/advanceTurn/getTurn; durable-barrier appends before side effects; sequential sync tools with progress; async tool exposure + durable suspension; permission pipeline (checker → optional classifier batch → human/deny fallback, fail-closed on checker errors); cancellation with synthetic results; recovery that re-issues interrupted model calls and continues after interrupted sync tools; model-call-limit exhaustion with machine-readable code. - fs-repo.ts: date-partitioned append-only JSONL, strict line validation, path-traversal rejection, per-turn in-process locking. - stream.ts: hot execution stream — buffers before the consumer attaches, outcome independent of consumption, close-drops-events. - context-resolver.ts: chain-walking materialization of context references with cycle detection. - Seam interfaces for stage-4 bridges: model/tool registries, permission checker/classifier, agent resolver, lifecycle bus, clock. - shared reducer: track classification failures durably so failed classifications are never re-run. 67 new tests: all nine §26 end-to-end scenarios (mocked deps, recovery boundaries seeded directly) plus repo/stream/resolver suites. Co-Authored-By: Claude Fable 5 --- .../packages/core/src/turns/agent-resolver.ts | 14 + apps/x/packages/core/src/turns/api.ts | 112 ++ apps/x/packages/core/src/turns/bus.ts | 18 + apps/x/packages/core/src/turns/clock.ts | 11 + .../core/src/turns/context-resolver.test.ts | 257 +++ .../core/src/turns/context-resolver.ts | 55 + .../x/packages/core/src/turns/fs-repo.test.ts | 205 +++ apps/x/packages/core/src/turns/fs-repo.ts | 128 ++ .../core/src/turns/in-memory-turn-repo.ts | 62 + apps/x/packages/core/src/turns/index.ts | 14 + apps/x/packages/core/src/turns/keyed-mutex.ts | 21 + .../packages/core/src/turns/model-registry.ts | 46 + apps/x/packages/core/src/turns/permission.ts | 52 + apps/x/packages/core/src/turns/repo.ts | 15 + .../x/packages/core/src/turns/runtime.test.ts | 1495 +++++++++++++++++ apps/x/packages/core/src/turns/runtime.ts | 928 ++++++++++ apps/x/packages/core/src/turns/stream.test.ts | 89 + apps/x/packages/core/src/turns/stream.ts | 94 ++ .../packages/core/src/turns/tool-registry.ts | 35 + apps/x/packages/shared/src/turns.test.ts | 1 + apps/x/packages/shared/src/turns.ts | 5 +- 21 files changed, 3656 insertions(+), 1 deletion(-) create mode 100644 apps/x/packages/core/src/turns/agent-resolver.ts create mode 100644 apps/x/packages/core/src/turns/api.ts create mode 100644 apps/x/packages/core/src/turns/bus.ts create mode 100644 apps/x/packages/core/src/turns/clock.ts create mode 100644 apps/x/packages/core/src/turns/context-resolver.test.ts create mode 100644 apps/x/packages/core/src/turns/context-resolver.ts create mode 100644 apps/x/packages/core/src/turns/fs-repo.test.ts create mode 100644 apps/x/packages/core/src/turns/fs-repo.ts create mode 100644 apps/x/packages/core/src/turns/in-memory-turn-repo.ts create mode 100644 apps/x/packages/core/src/turns/index.ts create mode 100644 apps/x/packages/core/src/turns/keyed-mutex.ts create mode 100644 apps/x/packages/core/src/turns/model-registry.ts create mode 100644 apps/x/packages/core/src/turns/permission.ts create mode 100644 apps/x/packages/core/src/turns/repo.ts create mode 100644 apps/x/packages/core/src/turns/runtime.test.ts create mode 100644 apps/x/packages/core/src/turns/runtime.ts create mode 100644 apps/x/packages/core/src/turns/stream.test.ts create mode 100644 apps/x/packages/core/src/turns/stream.ts create mode 100644 apps/x/packages/core/src/turns/tool-registry.ts diff --git a/apps/x/packages/core/src/turns/agent-resolver.ts b/apps/x/packages/core/src/turns/agent-resolver.ts new file mode 100644 index 00000000..5ea1760d --- /dev/null +++ b/apps/x/packages/core/src/turns/agent-resolver.ts @@ -0,0 +1,14 @@ +import type { z } from "zod"; +import type { RequestedAgent, ResolvedAgent } from "@x/shared/dist/turns.js"; + +// Absorbs agent assembly: built-in/dynamic agent selection, user-defined +// agent loading, system-prompt augmentation, model precedence +// (override > agent > application default), tool attachment and filtering. +// The result is the immutable execution snapshot persisted in turn_created; +// the resolved system prompt is final byte-for-byte. Resolution failure +// rejects createTurn without creating a turn file. +export interface IAgentResolver { + resolve( + agent: z.infer, + ): Promise>; +} diff --git a/apps/x/packages/core/src/turns/api.ts b/apps/x/packages/core/src/turns/api.ts new file mode 100644 index 00000000..55a60e2c --- /dev/null +++ b/apps/x/packages/core/src/turns/api.ts @@ -0,0 +1,112 @@ +import type { z } from "zod"; +import type { AssistantMessage, UserMessage } from "@x/shared/dist/message.js"; +import type { + JsonValue, + RequestedAgent, + ToolResultData, + TurnContext, + TurnEvent, + TurnStreamEvent, + TurnSuspended, + TurnUsage, +} from "@x/shared/dist/turns.js"; + +export interface CreateTurnInput { + agent: z.infer; + sessionId?: string | null; + context: z.infer; + input: z.infer; + config: { + autoPermission?: boolean; + humanAvailable: boolean; + maxModelCalls?: number; + }; +} + +// Exactly one external input per advanceTurn invocation. +export type TurnExternalInput = + | { + type: "permission_decision"; + toolCallId: string; + decision: "allow" | "deny"; + metadata?: JsonValue; + } + | { + type: "async_tool_progress"; + toolCallId: string; + progress: JsonValue; + } + | { + type: "async_tool_result"; + toolCallId: string; + result: z.infer; + } + | { + type: "cancel"; + reason?: string; + }; + +export type TurnOutcome = + | { + status: "completed"; + output: z.infer; + finishReason: string; + usage: z.infer; + } + | { + status: "suspended"; + pendingPermissions: z.infer["pendingPermissions"]; + pendingAsyncTools: z.infer["pendingAsyncTools"]; + usage: z.infer; + } + | { + status: "failed"; + error: string; + // Mirrors turn_failed.code (e.g. MODEL_CALL_LIMIT_ERROR_CODE). + code?: string; + usage: z.infer; + } + | { + status: "cancelled"; + reason?: string; + usage: z.infer; + }; + +export interface TurnExecution { + events: AsyncIterable; + outcome: Promise; +} + +export interface Turn { + turnId: string; + events: Array>; +} + +export interface ITurnRuntime { + createTurn(input: CreateTurnInput): Promise; + advanceTurn( + turnId: string, + input?: TurnExternalInput, + options?: { signal?: AbortSignal }, + ): TurnExecution; + getTurn(turnId: string): Promise; +} + +// An external input that does not match the turn's current durable pending +// state. Rejects the execution; nothing is appended. +export class TurnInputError extends Error { + constructor(message: string) { + super(message); + this.name = "TurnInputError"; + } +} + +// A live runtime dependency is missing or incompatible with the persisted +// snapshot. Rejects the execution; the turn is left unchanged so the caller +// can fix its environment and retry. +export class TurnDependencyError extends Error { + constructor(message: string) { + super(message); + this.name = "TurnDependencyError"; + } +} diff --git a/apps/x/packages/core/src/turns/bus.ts b/apps/x/packages/core/src/turns/bus.ts new file mode 100644 index 00000000..c998e6a1 --- /dev/null +++ b/apps/x/packages/core/src/turns/bus.ts @@ -0,0 +1,18 @@ +// Ephemeral process-lifecycle events (turn-runtime-design.md §17). Never +// persisted, never replayed; if the process crashes the information +// disappears, which accurately reflects that no execution is known active. +export interface TurnProcessingStart { + type: "turn-processing-start"; + turnId: string; +} + +export interface TurnProcessingEnd { + type: "turn-processing-end"; + turnId: string; +} + +export type TurnLifecycleEvent = TurnProcessingStart | TurnProcessingEnd; + +export interface ITurnLifecycleBus { + publish(event: TurnLifecycleEvent): void; +} diff --git a/apps/x/packages/core/src/turns/clock.ts b/apps/x/packages/core/src/turns/clock.ts new file mode 100644 index 00000000..a88a3122 --- /dev/null +++ b/apps/x/packages/core/src/turns/clock.ts @@ -0,0 +1,11 @@ +// Deterministic timestamp seam. No IClock existed in the codebase before the +// turn runtime; production uses SystemClock, tests inject fixed clocks. +export interface IClock { + now(): string; // ISO-8601 timestamp +} + +export class SystemClock implements IClock { + now(): string { + return new Date().toISOString(); + } +} diff --git a/apps/x/packages/core/src/turns/context-resolver.test.ts b/apps/x/packages/core/src/turns/context-resolver.test.ts new file mode 100644 index 00000000..39911786 --- /dev/null +++ b/apps/x/packages/core/src/turns/context-resolver.test.ts @@ -0,0 +1,257 @@ +import { describe, expect, it } from "vitest"; +import type { z } from "zod"; +import { + MODEL_CALL_LIMIT_ERROR_CODE, + type TurnContext, + TurnCorruptionError, + type TurnEvent, +} from "@x/shared/dist/turns.js"; +import { TurnRepoContextResolver } from "./context-resolver.js"; +import { InMemoryTurnRepo } from "./in-memory-turn-repo.js"; + +type TEvent = z.infer; + +function user(text: string) { + return { role: "user" as const, content: text }; +} + +function assistant(text: string) { + return { role: "assistant" as const, content: text }; +} + +// A minimal completed turn: input → one model call → text response. +function completedTurnLog( + turnId: string, + context: z.infer, + inputText: string, + responseText: string, +): TEvent[] { + const ts = "2026-07-02T10:00:00Z"; + return [ + { + type: "turn_created", + schemaVersion: 1, + turnId, + ts, + sessionId: "sess-1", + agent: { + requested: { agentId: "copilot" }, + resolved: { + agentId: "copilot", + systemPrompt: "SYS", + model: { provider: "fake", model: "m" }, + tools: [], + }, + }, + context, + input: user(inputText), + config: { + autoPermission: false, + humanAvailable: true, + maxModelCalls: 20, + }, + }, + { + type: "model_call_requested", + turnId, + ts, + modelCallIndex: 0, + request: { + systemPrompt: "SYS", + ...(Array.isArray(context) ? {} : { contextRef: context }), + messages: Array.isArray(context) + ? [...context, user(inputText)] + : [user(inputText)], + tools: [], + parameters: {}, + }, + }, + { + type: "model_call_completed", + turnId, + ts, + modelCallIndex: 0, + message: assistant(responseText), + finishReason: "stop", + usage: {}, + }, + { + type: "turn_completed", + turnId, + ts, + output: assistant(responseText), + finishReason: "stop", + usage: {}, + }, + ]; +} + +// A turn that failed at the model-call limit after one tool round trip; its +// transcript is structurally complete including the synthetic closure. +function limitFailedTurnLog(turnId: string): TEvent[] { + const ts = "2026-07-02T10:00:00Z"; + const echo = { + toolId: "tool.echo", + name: "echo", + description: "Echo", + inputSchema: {}, + execution: "sync" as const, + requiresHuman: false, + }; + const call = { + role: "assistant" as const, + content: [ + { + type: "tool-call" as const, + toolCallId: "tc1", + toolName: "echo", + arguments: {}, + }, + ], + }; + return [ + { + type: "turn_created", + schemaVersion: 1, + turnId, + ts, + sessionId: "sess-1", + agent: { + requested: { agentId: "copilot" }, + resolved: { + agentId: "copilot", + systemPrompt: "SYS", + model: { provider: "fake", model: "m" }, + tools: [echo], + }, + }, + context: [], + input: user("do it"), + config: { + autoPermission: false, + humanAvailable: true, + maxModelCalls: 1, + }, + }, + { + type: "model_call_requested", + turnId, + ts, + modelCallIndex: 0, + request: { + systemPrompt: "SYS", + messages: [user("do it")], + tools: [echo], + parameters: {}, + }, + }, + { + type: "model_call_completed", + turnId, + ts, + modelCallIndex: 0, + message: call, + finishReason: "tool-calls", + usage: {}, + }, + { + type: "tool_invocation_requested", + turnId, + ts, + toolCallId: "tc1", + toolId: "tool.echo", + toolName: "echo", + execution: "sync", + input: {}, + }, + { + type: "tool_result", + turnId, + ts, + toolCallId: "tc1", + toolName: "echo", + source: "sync", + result: { output: "echoed", isError: false }, + }, + { + type: "turn_failed", + turnId, + ts, + error: "Model call limit of 1 reached before the turn completed.", + code: MODEL_CALL_LIMIT_ERROR_CODE, + usage: {}, + }, + ]; +} + +const T1 = "2026-07-02T10-00-00Z-0000001-000"; +const T2 = "2026-07-02T10-00-00Z-0000002-000"; +const T3 = "2026-07-02T10-00-00Z-0000003-000"; + +describe("TurnRepoContextResolver", () => { + it("passes inline context through unchanged", async () => { + const repo = new InMemoryTurnRepo(); + const resolver = new TurnRepoContextResolver({ turnRepo: repo }); + const inline = [user("a"), assistant("b")]; + expect(await resolver.resolve(inline)).toEqual(inline); + }); + + it("resolves a single reference to the referenced turn's transcript", async () => { + const repo = new InMemoryTurnRepo(); + repo.seed(completedTurnLog(T1, [], "first question", "first answer")); + const resolver = new TurnRepoContextResolver({ turnRepo: repo }); + expect(await resolver.resolve({ previousTurnId: T1 })).toEqual([ + user("first question"), + assistant("first answer"), + ]); + }); + + it("resolves a chain of references down to the inline base", async () => { + const repo = new InMemoryTurnRepo(); + repo.seed(completedTurnLog(T1, [user("preamble")], "q1", "a1")); + repo.seed(completedTurnLog(T2, { previousTurnId: T1 }, "q2", "a2")); + repo.seed(completedTurnLog(T3, { previousTurnId: T2 }, "q3", "a3")); + const resolver = new TurnRepoContextResolver({ turnRepo: repo }); + expect(await resolver.resolve({ previousTurnId: T3 })).toEqual([ + user("preamble"), + user("q1"), + assistant("a1"), + user("q2"), + assistant("a2"), + user("q3"), + assistant("a3"), + ]); + }); + + it("includes failed turns' transcripts with synthetic closures", async () => { + const repo = new InMemoryTurnRepo(); + repo.seed(limitFailedTurnLog(T1)); + const resolver = new TurnRepoContextResolver({ turnRepo: repo }); + const resolved = await resolver.resolve({ previousTurnId: T1 }); + expect(resolved.map((m) => m.role)).toEqual(["user", "assistant", "tool"]); + expect(resolved[2]).toMatchObject({ + role: "tool", + toolCallId: "tc1", + content: "echoed", + }); + }); + + it("rejects references to missing turns as infrastructure errors", async () => { + const repo = new InMemoryTurnRepo(); + const resolver = new TurnRepoContextResolver({ turnRepo: repo }); + await expect( + resolver.resolve({ previousTurnId: T1 }), + ).rejects.toThrowError(/turn not found/); + }); + + it("rejects cyclic reference chains as corruption", async () => { + const repo = new InMemoryTurnRepo(); + // Two turns referencing each other (only constructable by corruption). + repo.seed(completedTurnLog(T1, { previousTurnId: T2 }, "q1", "a1")); + repo.seed(completedTurnLog(T2, { previousTurnId: T1 }, "q2", "a2")); + const resolver = new TurnRepoContextResolver({ turnRepo: repo }); + await expect( + resolver.resolve({ previousTurnId: T1 }), + ).rejects.toThrowError(TurnCorruptionError); + }); +}); diff --git a/apps/x/packages/core/src/turns/context-resolver.ts b/apps/x/packages/core/src/turns/context-resolver.ts new file mode 100644 index 00000000..0dce3181 --- /dev/null +++ b/apps/x/packages/core/src/turns/context-resolver.ts @@ -0,0 +1,55 @@ +import type { z } from "zod"; +import { + type ConversationMessage, + type TurnContext, + TurnCorruptionError, + reduceTurn, + turnTranscript, +} from "@x/shared/dist/turns.js"; +import type { ITurnRepo } from "./repo.js"; + +// Materializes a turn's context (turn-runtime-design.md §6.6). Inline +// contexts pass through; references resolve to the referenced turn's full +// transcript by walking the chain down to its inline base. Resolution always +// reads durable state, so normal execution and crash recovery share one +// path. A missing or corrupt referenced turn is an infrastructure error. +export interface IContextResolver { + resolve( + context: z.infer, + ): Promise>>; +} + +export class TurnRepoContextResolver implements IContextResolver { + private readonly turnRepo: ITurnRepo; + + constructor({ turnRepo }: { turnRepo: ITurnRepo }) { + this.turnRepo = turnRepo; + } + + async resolve( + context: z.infer, + ): Promise>> { + // Walk the reference chain back to the inline base, then concatenate + // transcripts oldest-first. Iterative to bound stack depth; a visited + // set catches cyclic (corrupt) chains. + const segments: Array>> = []; + const visited = new Set(); + let current = context; + while (!Array.isArray(current)) { + const turnId = current.previousTurnId; + if (visited.has(turnId)) { + throw new TurnCorruptionError( + `cyclic context reference chain at turn ${turnId}`, + ); + } + visited.add(turnId); + const events = await this.turnRepo.read(turnId); + const state = reduceTurn(events); + segments.push(turnTranscript(state)); + current = state.definition.context; + } + segments.push(current); + segments.reverse(); + return segments.flat(); + } +} diff --git a/apps/x/packages/core/src/turns/fs-repo.test.ts b/apps/x/packages/core/src/turns/fs-repo.test.ts new file mode 100644 index 00000000..8e44b5b1 --- /dev/null +++ b/apps/x/packages/core/src/turns/fs-repo.test.ts @@ -0,0 +1,205 @@ +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import type { z } from "zod"; +import { + TurnCorruptionError, + TurnCreated, + TurnEvent, +} from "@x/shared/dist/turns.js"; +import { FSTurnRepo } from "./fs-repo.js"; + +const TURN_ID = "2026-07-02T10-00-00Z-0000001-000"; + +function created(turnId = TURN_ID): z.infer { + return { + type: "turn_created", + schemaVersion: 1, + turnId, + ts: "2026-07-02T10:00:00Z", + sessionId: null, + agent: { + requested: { agentId: "copilot" }, + resolved: { + agentId: "copilot", + systemPrompt: "SYS", + model: { provider: "fake", model: "m" }, + tools: [], + }, + }, + context: [], + input: { role: "user", content: "hello" }, + config: { autoPermission: false, humanAvailable: true, maxModelCalls: 20 }, + }; +} + +function requested(turnId = TURN_ID): z.infer { + return { + type: "model_call_requested", + turnId, + ts: "2026-07-02T10:00:01Z", + modelCallIndex: 0, + request: { + systemPrompt: "SYS", + messages: [{ role: "user", content: "hello" }], + tools: [], + parameters: {}, + }, + }; +} + +function failed(turnId = TURN_ID): z.infer { + return { + type: "model_call_failed", + turnId, + ts: "2026-07-02T10:00:02Z", + modelCallIndex: 0, + error: "boom", + }; +} + +describe("FSTurnRepo", () => { + let root: string; + let repo: FSTurnRepo; + + beforeEach(async () => { + root = await fs.mkdtemp(path.join(os.tmpdir(), "turn-repo-")); + repo = new FSTurnRepo({ turnsRootDir: root }); + }); + + afterEach(async () => { + await fs.rm(root, { recursive: true, force: true }); + }); + + it("writes to a deterministic date-partitioned path", async () => { + await repo.create(created()); + const file = path.join(root, "2026", "07", "02", `${TURN_ID}.jsonl`); + const raw = await fs.readFile(file, "utf8"); + expect(raw.endsWith("\n")).toBe(true); + expect(JSON.parse(raw.trim()).type).toBe("turn_created"); + }); + + it("rejects malformed and path-like turn ids", async () => { + for (const bad of [ + "../../../etc/passwd", + "2026-07-02T10/evil", + "2026-07-02T10-00-00Z-0000001-000.jsonl", + "not-a-turn-id", + "", + ]) { + await expect(repo.read(bad)).rejects.toThrowError(/invalid turn id/); + } + }); + + it("create fails if the turn already exists", async () => { + await repo.create(created()); + await expect(repo.create(created())).rejects.toThrowError(); + }); + + it("appends preserve order and read validates every line", async () => { + await repo.create(created()); + await repo.append(TURN_ID, [requested()]); + await repo.append(TURN_ID, [failed()]); + const events = await repo.read(TURN_ID); + expect(events.map((e) => e.type)).toEqual([ + "turn_created", + "model_call_requested", + "model_call_failed", + ]); + }); + + it("append validates events and turn id match before writing", async () => { + await repo.create(created()); + await expect( + repo.append(TURN_ID, [requested("2026-07-02T10-00-00Z-0000002-000")]), + ).rejects.toThrowError(/does not match/); + await expect( + repo.append(TURN_ID, [{ type: "wat" } as unknown as z.infer]), + ).rejects.toThrowError(); + // Nothing was written by the failed appends. + expect((await repo.read(TURN_ID)).length).toBe(1); + }); + + it("append never creates a missing turn file", async () => { + await expect(repo.append(TURN_ID, [requested()])).rejects.toThrowError( + /turn not found/, + ); + }); + + it("reading a missing turn reports not found", async () => { + await expect(repo.read(TURN_ID)).rejects.toThrowError(/turn not found/); + }); + + async function writeRaw(content: string): Promise { + const file = path.join(root, "2026", "07", "02", `${TURN_ID}.jsonl`); + await fs.mkdir(path.dirname(file), { recursive: true }); + await fs.writeFile(file, content); + } + + it("rejects an empty file as corrupt", async () => { + await writeRaw(""); + await expect(repo.read(TURN_ID)).rejects.toThrowError(TurnCorruptionError); + }); + + it("rejects malformed first, middle, and final lines", async () => { + const good = JSON.stringify(created()); + const req = JSON.stringify(requested()); + for (const content of [ + `not json\n${req}\n`, + `${good}\nnot json\n${req}\n`, + `${good}\n{"type":"wat"}\n`, + ]) { + await writeRaw(content); + await expect(repo.read(TURN_ID)).rejects.toThrowError( + TurnCorruptionError, + ); + } + }); + + it("rejects a torn final line (no trailing newline)", async () => { + const good = JSON.stringify(created()); + const torn = JSON.stringify(requested()).slice(0, 20); + await writeRaw(`${good}\n${torn}`); + await expect(repo.read(TURN_ID)).rejects.toThrowError( + /does not end with a complete line/, + ); + }); + + it("rejects unsupported schema versions on read", async () => { + const v2 = { ...created(), schemaVersion: 2 }; + await writeRaw(`${JSON.stringify(v2)}\n`); + await expect(repo.read(TURN_ID)).rejects.toThrowError(TurnCorruptionError); + }); + + it("rejects events whose turnId does not match the file", async () => { + const other = created("2026-07-02T10-00-00Z-0000002-000"); + await writeRaw(`${JSON.stringify(other)}\n`); + await expect(repo.read(TURN_ID)).rejects.toThrowError(/does not match file/); + }); + + it("withLock serializes work per turn", async () => { + const order: string[] = []; + await Promise.all([ + repo.withLock(TURN_ID, async () => { + order.push("a-start"); + await new Promise((r) => setTimeout(r, 20)); + order.push("a-end"); + }), + repo.withLock(TURN_ID, async () => { + order.push("b-start"); + order.push("b-end"); + }), + ]); + expect(order).toEqual(["a-start", "a-end", "b-start", "b-end"]); + }); + + it("withLock releases after failures", async () => { + await expect( + repo.withLock(TURN_ID, async () => { + throw new Error("first fails"); + }), + ).rejects.toThrowError("first fails"); + await expect(repo.withLock(TURN_ID, async () => "ok")).resolves.toBe("ok"); + }); +}); diff --git a/apps/x/packages/core/src/turns/fs-repo.ts b/apps/x/packages/core/src/turns/fs-repo.ts new file mode 100644 index 00000000..57d19ca8 --- /dev/null +++ b/apps/x/packages/core/src/turns/fs-repo.ts @@ -0,0 +1,128 @@ +import fs from "node:fs/promises"; +import path from "node:path"; +import { z } from "zod"; +import { + TurnCorruptionError, + TurnCreated, + TurnEvent, +} from "@x/shared/dist/turns.js"; +import { KeyedMutex } from "./keyed-mutex.js"; +import type { ITurnRepo } from "./repo.js"; + +// Turn IDs come from IMonotonicallyIncreasingIdGenerator and look like +// 2025-11-11T04-36-29Z-0001234-000. The repo validates the format before +// deriving a path and rejects anything path-like. +const TURN_ID_PATTERN = /^(\d{4})-(\d{2})-(\d{2})T[A-Za-z0-9-]+$/; + +export class FSTurnRepo implements ITurnRepo { + private readonly rootDir: string; + private readonly mutex = new KeyedMutex(); + + constructor({ turnsRootDir }: { turnsRootDir: string }) { + this.rootDir = turnsRootDir; + } + + private filePath(turnId: string): string { + const match = TURN_ID_PATTERN.exec(turnId); + if (!match) { + throw new Error(`invalid turn id: ${turnId}`); + } + const [, year, month, day] = match; + return path.join(this.rootDir, year, month, day, `${turnId}.jsonl`); + } + + private serialize( + turnId: string, + events: Array>, + ): string { + let payload = ""; + for (const event of events) { + const parsed = TurnEvent.parse(event); + if (parsed.turnId !== turnId) { + throw new Error( + `event turnId ${parsed.turnId} does not match ${turnId}`, + ); + } + payload += `${JSON.stringify(parsed)}\n`; + } + return payload; + } + + async create(event: z.infer): Promise { + const parsed = TurnCreated.parse(event); + const file = this.filePath(parsed.turnId); + await fs.mkdir(path.dirname(file), { recursive: true }); + // "wx" fails if the file already exists. + await fs.writeFile(file, this.serialize(parsed.turnId, [parsed]), { + flag: "wx", + }); + } + + async read(turnId: string): Promise>> { + const file = this.filePath(turnId); + let raw: string; + try { + raw = await fs.readFile(file, "utf8"); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") { + throw new Error(`turn not found: ${turnId}`); + } + throw error; + } + if (raw.length === 0) { + throw new TurnCorruptionError(`turn file is empty: ${turnId}`); + } + const lines = raw.split("\n"); + // A well-formed file ends with a newline, leaving one trailing empty + // segment. Anything else (including a torn final line) is corrupt. + const trailing = lines.pop(); + if (trailing !== "") { + throw new TurnCorruptionError( + `turn file does not end with a complete line: ${turnId}`, + ); + } + const events: Array> = []; + for (const [index, line] of lines.entries()) { + let parsed: z.infer; + try { + parsed = TurnEvent.parse(JSON.parse(line)); + } catch (error) { + throw new TurnCorruptionError( + `malformed turn event at ${turnId}:${index + 1}: ${String( + error instanceof Error ? error.message : error, + )}`, + ); + } + if (parsed.turnId !== turnId) { + throw new TurnCorruptionError( + `event turnId ${parsed.turnId} does not match file ${turnId}`, + ); + } + events.push(parsed); + } + return events; + } + + async append( + turnId: string, + events: Array>, + ): Promise { + if (events.length === 0) { + return; + } + const payload = this.serialize(turnId, events); + const file = this.filePath(turnId); + // Appends must never create a file: a turn exists only via create(). + // All writers hold the per-turn lock, so check-then-append is safe. + try { + await fs.access(file); + } catch { + throw new Error(`turn not found: ${turnId}`); + } + await fs.appendFile(file, payload); + } + + async withLock(turnId: string, fn: () => Promise): Promise { + return this.mutex.run(turnId, fn); + } +} diff --git a/apps/x/packages/core/src/turns/in-memory-turn-repo.ts b/apps/x/packages/core/src/turns/in-memory-turn-repo.ts new file mode 100644 index 00000000..3dfd176d --- /dev/null +++ b/apps/x/packages/core/src/turns/in-memory-turn-repo.ts @@ -0,0 +1,62 @@ +import { z } from "zod"; +import { TurnCreated, TurnEvent } from "@x/shared/dist/turns.js"; +import { KeyedMutex } from "./keyed-mutex.js"; +import type { ITurnRepo } from "./repo.js"; + +// Test fake mirroring FSTurnRepo semantics (create-if-absent, validate on +// write and read boundaries, per-turn locking) without touching disk. +export class InMemoryTurnRepo implements ITurnRepo { + private files = new Map>>(); + private mutex = new KeyedMutex(); + + async create(event: z.infer): Promise { + const parsed = TurnCreated.parse(event); + if (this.files.has(parsed.turnId)) { + throw new Error(`turn already exists: ${parsed.turnId}`); + } + this.files.set(parsed.turnId, [structuredClone(parsed)]); + } + + async read(turnId: string): Promise>> { + const events = this.files.get(turnId); + if (!events) { + throw new Error(`turn not found: ${turnId}`); + } + return structuredClone(events); + } + + async append( + turnId: string, + events: Array>, + ): Promise { + const file = this.files.get(turnId); + if (!file) { + throw new Error(`turn not found: ${turnId}`); + } + for (const event of events) { + const parsed = TurnEvent.parse(event); + if (parsed.turnId !== turnId) { + throw new Error( + `event turnId ${parsed.turnId} does not match ${turnId}`, + ); + } + file.push(structuredClone(parsed)); + } + } + + async withLock(turnId: string, fn: () => Promise): Promise { + return this.mutex.run(turnId, fn); + } + + // Test helper: seed a raw event log (validated) for recovery scenarios. + seed(events: Array>): void { + if (events.length === 0) { + throw new Error("cannot seed an empty log"); + } + const turnId = events[0].turnId; + this.files.set( + turnId, + events.map((e) => structuredClone(TurnEvent.parse(e))), + ); + } +} diff --git a/apps/x/packages/core/src/turns/index.ts b/apps/x/packages/core/src/turns/index.ts new file mode 100644 index 00000000..e7cbccf3 --- /dev/null +++ b/apps/x/packages/core/src/turns/index.ts @@ -0,0 +1,14 @@ +export * from "./api.js"; +export * from "./agent-resolver.js"; +export * from "./bus.js"; +export * from "./clock.js"; +export * from "./context-resolver.js"; +export * from "./fs-repo.js"; +export * from "./in-memory-turn-repo.js"; +export * from "./keyed-mutex.js"; +export * from "./model-registry.js"; +export * from "./permission.js"; +export * from "./repo.js"; +export * from "./runtime.js"; +export * from "./stream.js"; +export * from "./tool-registry.js"; diff --git a/apps/x/packages/core/src/turns/keyed-mutex.ts b/apps/x/packages/core/src/turns/keyed-mutex.ts new file mode 100644 index 00000000..e39c2404 --- /dev/null +++ b/apps/x/packages/core/src/turns/keyed-mutex.ts @@ -0,0 +1,21 @@ +// In-process per-key exclusion. Cross-process coordination is explicitly out +// of scope for the turn runtime (single Electron main process). +export class KeyedMutex { + private tails = new Map>(); + + async run(key: string, fn: () => Promise): Promise { + const prev = this.tails.get(key) ?? Promise.resolve(); + const next = prev.then( + () => fn(), + () => fn(), + ); + this.tails.set(key, next); + try { + return await next; + } finally { + if (this.tails.get(key) === next) { + this.tails.delete(key); + } + } + } +} diff --git a/apps/x/packages/core/src/turns/model-registry.ts b/apps/x/packages/core/src/turns/model-registry.ts new file mode 100644 index 00000000..d87aa097 --- /dev/null +++ b/apps/x/packages/core/src/turns/model-registry.ts @@ -0,0 +1,46 @@ +import type { z } from "zod"; +import type { AssistantMessage } from "@x/shared/dist/message.js"; +import type { + ConversationMessage, + DurableLlmStepStreamEvent, + JsonValue, + ModelDescriptor, + ToolDescriptor, + TurnUsage, +} from "@x/shared/dist/turns.js"; + +// One stream() call performs exactly one model step; the turn loop drives +// multi-step behavior. The stream yields normalized events and must end with +// exactly one "completed" event, or throw (a throw is a model failure; an +// abort-triggered throw is cancellation). +export type LlmStreamEvent = + | { type: "text_delta"; delta: string } + | { type: "reasoning_delta"; delta: string } + | { type: "step_event"; event: z.infer } + | { + type: "completed"; + message: z.infer; + finishReason: string; + usage: z.infer; + providerMetadata?: JsonValue; + }; + +export interface ModelStreamRequest { + systemPrompt: string; + messages: Array>; + tools: Array>; + parameters: Record; + signal: AbortSignal; +} + +export interface ResolvedModel { + descriptor: z.infer; + stream(request: ModelStreamRequest): AsyncIterable; +} + +// Resolves the persisted model descriptor to a live model during advanceTurn. +// A rejection is an infrastructure error: the execution is rejected and the +// turn is left unchanged. +export interface IModelRegistry { + resolve(descriptor: z.infer): Promise; +} diff --git a/apps/x/packages/core/src/turns/permission.ts b/apps/x/packages/core/src/turns/permission.ts new file mode 100644 index 00000000..7f386623 --- /dev/null +++ b/apps/x/packages/core/src/turns/permission.ts @@ -0,0 +1,52 @@ +import type { JsonValue } from "@x/shared/dist/turns.js"; + +export interface PermissionCheckInput { + turnId: string; + toolCallId: string; + toolId: string; + toolName: string; + input: unknown; +} + +export interface PermissionCheckAllowed { + required: false; +} + +export interface PermissionCheckRequired { + required: true; + // Presentation payload persisted on tool_permission_required and shown to + // the human/classifier. + request: JsonValue; +} + +// Tool-specific policy (command analysis, filesystem boundaries, allowlists) +// lives behind this seam, outside the loop. A thrown error fails closed: the +// call is recorded as permission-required and never executes automatically. +export interface IPermissionChecker { + check( + input: PermissionCheckInput, + ): Promise; +} + +export interface PermissionClassificationInput { + toolCallId: string; + toolName: string; + input: unknown; + request: JsonValue; +} + +export interface PermissionClassification { + toolCallId: string; + decision: "allow" | "deny" | "defer"; + reason: string; +} + +// Handles all permission-required calls from one model response in one batch +// when automatic permission is enabled. Internal model calls are opaque to +// the turn loop. Failures and omitted decisions normalize to defer. +export interface IPermissionClassifier { + classify( + requests: PermissionClassificationInput[], + signal: AbortSignal, + ): Promise; +} diff --git a/apps/x/packages/core/src/turns/repo.ts b/apps/x/packages/core/src/turns/repo.ts new file mode 100644 index 00000000..6824f529 --- /dev/null +++ b/apps/x/packages/core/src/turns/repo.ts @@ -0,0 +1,15 @@ +import type { z } from "zod"; +import type { TurnCreated, TurnEvent } from "@x/shared/dist/turns.js"; + +// Loop-facing repository contract. Listing, deletion, session lookup, and +// presentation metadata are deliberately not part of it. +export interface ITurnRepo { + // Fails if the turn already exists. + create(event: z.infer): Promise; + // Validates every line strictly; corrupt files are rejected whole. + read(turnId: string): Promise>>; + // Validates events before writing. + append(turnId: string, events: Array>): Promise; + // In-process per-turn exclusion. + withLock(turnId: string, fn: () => Promise): Promise; +} diff --git a/apps/x/packages/core/src/turns/runtime.test.ts b/apps/x/packages/core/src/turns/runtime.test.ts new file mode 100644 index 00000000..b9019b60 --- /dev/null +++ b/apps/x/packages/core/src/turns/runtime.test.ts @@ -0,0 +1,1495 @@ +import { describe, expect, it } from "vitest"; +import type { z } from "zod"; +import { + MODEL_CALL_LIMIT_ERROR_CODE, + type JsonValue, + type ResolvedAgent, + type ToolDescriptor, + type TurnEvent, + type TurnStreamEvent, +} from "@x/shared/dist/turns.js"; +import type { IAgentResolver } from "./agent-resolver.js"; +import type { TurnExecution, TurnOutcome } from "./api.js"; +import { TurnDependencyError, TurnInputError } from "./api.js"; +import type { TurnLifecycleEvent } from "./bus.js"; +import { TurnRepoContextResolver } from "./context-resolver.js"; +import { InMemoryTurnRepo } from "./in-memory-turn-repo.js"; +import type { + IModelRegistry, + LlmStreamEvent, + ModelStreamRequest, + ResolvedModel, +} from "./model-registry.js"; +import type { + IPermissionChecker, + IPermissionClassifier, + PermissionCheckInput, + PermissionClassification, + PermissionClassificationInput, +} from "./permission.js"; +import { TurnRuntime } from "./runtime.js"; +import type { + IToolRegistry, + RuntimeTool, + SyncRuntimeTool, + ToolExecutionContext, +} from "./tool-registry.js"; + +// --------------------------------------------------------------------------- +// Fixtures +// --------------------------------------------------------------------------- + +const TS = "2026-07-02T10:00:00Z"; + +const echoDescriptor: z.infer = { + toolId: "tool.echo", + name: "echo", + description: "Echo tool", + inputSchema: {}, + execution: "sync", + requiresHuman: false, +}; + +const fetchDescriptor: z.infer = { + toolId: "tool.fetch", + name: "fetch", + description: "Async fetch tool", + inputSchema: {}, + execution: "async", + requiresHuman: false, +}; + +const askHumanDescriptor: z.infer = { + toolId: "tool.ask-human", + name: "ask-human", + description: "Ask the human", + inputSchema: {}, + execution: "async", + requiresHuman: true, +}; + +const defaultAgent: z.infer = { + agentId: "copilot", + systemPrompt: "SYS", + model: { provider: "fake", model: "m" }, + tools: [echoDescriptor, fetchDescriptor, askHumanDescriptor], +}; + +function user(text: string) { + return { role: "user" as const, content: text }; +} + +function assistantText(text: string) { + return { role: "assistant" as const, content: text }; +} + +function toolCallPart(id: string, name: string, args: JsonValue = {}) { + return { + type: "tool-call" as const, + toolCallId: id, + toolName: name, + arguments: args, + }; +} + +function assistantCalls(...parts: Array>) { + return { role: "assistant" as const, content: parts }; +} + +function completedResp( + message: Extract["message"], +): LlmStreamEvent { + return { + type: "completed", + message, + finishReason: "stop", + usage: { inputTokens: 10, outputTokens: 5, totalTokens: 15 }, + }; +} + +type ScriptedCall = ( + request: ModelStreamRequest, +) => AsyncGenerator; + +function respond(...events: LlmStreamEvent[]): ScriptedCall { + return async function* () { + yield* events; + }; +} + +function failCall(error: string): ScriptedCall { + // eslint-disable-next-line require-yield + return async function* () { + throw new Error(error); + }; +} + +function hangUntilAbort(onStarted?: () => void): ScriptedCall { + // eslint-disable-next-line require-yield + return async function* (request) { + onStarted?.(); + await new Promise((resolve) => { + if (request.signal.aborted) { + resolve(); + } else { + request.signal.addEventListener("abort", () => resolve(), { + once: true, + }); + } + }); + throw new Error("aborted"); + }; +} + +class FakeModelRegistry implements IModelRegistry { + requests: ModelStreamRequest[] = []; + private next = 0; + + constructor(private readonly calls: ScriptedCall[]) {} + + async resolve( + descriptor: ResolvedModel["descriptor"], + ): Promise { + return { + descriptor, + stream: (request) => { + this.requests.push(request); + const call = this.calls[this.next++]; + if (!call) { + throw new Error("no scripted model call remaining"); + } + return call(request); + }, + }; + } +} + +function syncTool( + descriptor: z.infer, + execute: SyncRuntimeTool["execute"], +): SyncRuntimeTool { + return { + descriptor: descriptor as SyncRuntimeTool["descriptor"], + execute, + }; +} + +const defaultTools: RuntimeTool[] = [ + syncTool(echoDescriptor, async (input) => ({ + output: { echoed: (input ?? null) as JsonValue }, + isError: false, + })), + { descriptor: fetchDescriptor as { execution: "async" } & typeof fetchDescriptor }, + { descriptor: askHumanDescriptor as { execution: "async" } & typeof askHumanDescriptor }, +]; + +class FakeToolRegistry implements IToolRegistry { + constructor(private readonly tools: RuntimeTool[]) {} + + async resolve( + descriptor: z.infer, + ): Promise { + const tool = this.tools.find( + (t) => t.descriptor.toolId === descriptor.toolId, + ); + if (!tool) { + throw new TurnDependencyError(`no live tool for ${descriptor.toolId}`); + } + return tool; + } +} + +type CheckerRule = "allow" | { request: JsonValue } | "throw"; + +class FakePermissionChecker implements IPermissionChecker { + calls: PermissionCheckInput[] = []; + + constructor(private readonly rules: Record = {}) {} + + async check(input: PermissionCheckInput) { + this.calls.push(input); + const rule = this.rules[input.toolName] ?? "allow"; + if (rule === "allow") { + return { required: false as const }; + } + if (rule === "throw") { + throw new Error("checker exploded"); + } + return { required: true as const, request: rule.request }; + } +} + +class FakePermissionClassifier implements IPermissionClassifier { + batches: PermissionClassificationInput[][] = []; + + constructor( + private readonly impl?: ( + requests: PermissionClassificationInput[], + ) => PermissionClassification[], + private readonly throws?: string, + ) {} + + async classify( + requests: PermissionClassificationInput[], + ): Promise { + this.batches.push(requests); + if (this.throws) { + throw new Error(this.throws); + } + if (!this.impl) { + throw new Error("classifier must not be called in this test"); + } + return this.impl(requests); + } +} + +class FakeAgentResolver implements IAgentResolver { + constructor( + private readonly agent: z.infer, + private readonly error?: string, + ) {} + + async resolve() { + if (this.error) { + throw new Error(this.error); + } + return this.agent; + } +} + +class FakeBus { + events: TurnLifecycleEvent[] = []; + publish(event: TurnLifecycleEvent): void { + this.events.push(event); + } +} + +class FakeIdGen { + private n = 0; + async next(): Promise { + this.n += 1; + return `2026-07-02T10-00-00Z-${String(this.n).padStart(7, "0")}-000`; + } +} + +class FakeClock { + now(): string { + return TS; + } +} + +function makeRuntime(opts: { + models?: ScriptedCall[]; + tools?: RuntimeTool[]; + checker?: FakePermissionChecker; + classifier?: FakePermissionClassifier; + agent?: z.infer; + agentError?: string; + repo?: InMemoryTurnRepo; +} = {}) { + const repo = opts.repo ?? new InMemoryTurnRepo(); + const models = new FakeModelRegistry(opts.models ?? []); + const checker = opts.checker ?? new FakePermissionChecker(); + const classifier = opts.classifier ?? new FakePermissionClassifier(); + const bus = new FakeBus(); + const runtime = new TurnRuntime({ + turnRepo: repo, + idGenerator: new FakeIdGen(), + clock: new FakeClock(), + agentResolver: new FakeAgentResolver(opts.agent ?? defaultAgent, opts.agentError), + modelRegistry: models, + toolRegistry: new FakeToolRegistry(opts.tools ?? defaultTools), + contextResolver: new TurnRepoContextResolver({ turnRepo: repo }), + permissionChecker: checker, + permissionClassifier: classifier, + bus, + }); + return { runtime, repo, models, checker, classifier, bus }; +} + +async function newTurn( + runtime: TurnRuntime, + overrides: Partial[0]> = {}, +): Promise { + return runtime.createTurn({ + agent: { agentId: "copilot" }, + context: [], + input: user("hello"), + config: { humanAvailable: true }, + ...overrides, + }); +} + +async function settle(execution: TurnExecution): Promise<{ + outcome?: TurnOutcome; + error?: unknown; + events: TurnStreamEvent[]; +}> { + const events: TurnStreamEvent[] = []; + const consumer = (async () => { + try { + for await (const event of execution.events) { + events.push(event); + } + } catch { + // Infrastructure failures also reject the outcome; tests assert there. + } + })(); + let outcome: TurnOutcome | undefined; + let error: unknown; + try { + outcome = await execution.outcome; + } catch (err) { + error = err; + } + await consumer; + return { outcome, error, events }; +} + +async function advanceAndSettle( + runtime: TurnRuntime, + turnId: string, + input?: Parameters[1], + options?: Parameters[2], +) { + return settle(runtime.advanceTurn(turnId, input, options)); +} + +function typesOf(events: Array<{ type: string }>): string[] { + return events.map((e) => e.type); +} + +async function persisted( + repo: InMemoryTurnRepo, + turnId: string, +): Promise>> { + return repo.read(turnId); +} + +// --------------------------------------------------------------------------- +// §26.1 Plain model response +// --------------------------------------------------------------------------- + +describe("plain model response (26.1)", () => { + it("runs one model step to completion with exact persisted request", async () => { + const { runtime, repo, bus } = makeRuntime({ + models: [ + respond( + { type: "text_delta", delta: "do" }, + { type: "step_event", event: { type: "text_end", text: "done" } }, + { type: "text_delta", delta: "ne" }, + completedResp(assistantText("done")), + ), + ], + }); + const turnId = await newTurn(runtime); + const { outcome, events } = await advanceAndSettle(runtime, turnId); + + expect(outcome).toMatchObject({ + status: "completed", + output: assistantText("done"), + finishReason: "stop", + }); + + const log = await persisted(repo, turnId); + expect(typesOf(log)).toEqual([ + "turn_created", + "model_call_requested", + "model_step_event", + "model_call_completed", + "turn_completed", + ]); + const request = log[1]; + expect(request).toMatchObject({ + modelCallIndex: 0, + request: { + systemPrompt: "SYS", + messages: [user("hello")], + }, + }); + // Deltas are streamed but never persisted. + expect(events.filter((e) => e.type === "text_delta")).toHaveLength(2); + expect(typesOf(log)).not.toContain("text_delta"); + // Terminal event duplicates output and aggregate usage. + const terminal = log[log.length - 1]; + expect(terminal).toMatchObject({ + type: "turn_completed", + output: assistantText("done"), + usage: { inputTokens: 10, outputTokens: 5, totalTokens: 15 }, + }); + // Lifecycle bus events are emitted but not persisted. + expect(bus.events.map((e) => e.type)).toEqual([ + "turn-processing-start", + "turn-processing-end", + ]); + }); + + it("streams durable events only after persistence, matching the file", async () => { + const { runtime, repo } = makeRuntime({ + models: [respond(completedResp(assistantText("done")))], + }); + const turnId = await newTurn(runtime); + const { events } = await advanceAndSettle(runtime, turnId); + const durable = events.filter( + (e) => e.type !== "text_delta" && e.type !== "reasoning_delta", + ); + const log = await persisted(repo, turnId); + expect(durable).toEqual(log.slice(1)); + }); + + it("outcome resolves without consuming the event stream", async () => { + const { runtime } = makeRuntime({ + models: [respond(completedResp(assistantText("done")))], + }); + const turnId = await newTurn(runtime); + const execution = runtime.advanceTurn(turnId); + const outcome = await execution.outcome; + expect(outcome.status).toBe("completed"); + }); +}); + +// --------------------------------------------------------------------------- +// §26.2 Mixed sync and async tools +// --------------------------------------------------------------------------- + +describe("mixed sync and async tools (26.2)", () => { + it("executes sync tools, suspends on async, and preserves source order in the next request", async () => { + const batch = assistantCalls( + toolCallPart("A", "echo", { i: 1 }), + toolCallPart("B", "fetch", { i: 2 }), + toolCallPart("C", "echo", { i: 3 }), + toolCallPart("D", "fetch", { i: 4 }), + ); + const { runtime, repo, models } = makeRuntime({ + models: [ + respond(completedResp(batch)), + respond(completedResp(assistantText("done"))), + ], + }); + const turnId = await newTurn(runtime); + + const first = await advanceAndSettle(runtime, turnId); + expect(first.outcome).toMatchObject({ + status: "suspended", + pendingAsyncTools: [ + expect.objectContaining({ toolCallId: "B" }), + expect.objectContaining({ toolCallId: "D" }), + ], + }); + + // Async results arrive in reverse order. + const second = await advanceAndSettle(runtime, turnId, { + type: "async_tool_result", + toolCallId: "D", + result: { output: "d-result", isError: false }, + }); + expect(second.outcome?.status).toBe("suspended"); + + const third = await advanceAndSettle(runtime, turnId, { + type: "async_tool_result", + toolCallId: "B", + result: { output: "b-result", isError: false }, + }); + expect(third.outcome?.status).toBe("completed"); + + // The next model request carries results in original source order + // even though physical completion order was A, C, D, B. + const log = await persisted(repo, turnId); + const secondRequest = log.find( + (e) => e.type === "model_call_requested" && e.modelCallIndex === 1, + ); + expect(secondRequest).toBeDefined(); + const toolMessages = + secondRequest?.type === "model_call_requested" + ? secondRequest.request.messages.filter((m) => m.role === "tool") + : []; + expect(toolMessages.map((m) => (m.role === "tool" ? m.toolCallId : ""))).toEqual( + ["A", "B", "C", "D"], + ); + // The live model call saw the same ordering. + expect( + models.requests[1].messages + .filter((m) => m.role === "tool") + .map((m) => (m.role === "tool" ? m.toolCallId : "")), + ).toEqual(["A", "B", "C", "D"]); + }); + + it("rejects async results for calls that are not pending", async () => { + const { runtime } = makeRuntime({ + models: [respond(completedResp(assistantText("done")))], + }); + const turnId = await newTurn(runtime); + await advanceAndSettle(runtime, turnId); + const { error } = await advanceAndSettle(runtime, turnId, { + type: "async_tool_result", + toolCallId: "ghost", + result: { output: "x", isError: false }, + }); + expect(error).toBeInstanceOf(TurnInputError); + }); +}); + +// --------------------------------------------------------------------------- +// §26.3 Partial human permission decisions +// --------------------------------------------------------------------------- + +describe("partial human permission decisions (26.3)", () => { + async function setup() { + const batch = assistantCalls( + toolCallPart("P1", "echo"), + toolCallPart("P2", "echo"), + toolCallPart("F1", "fetch"), + ); + const fixture = makeRuntime({ + models: [ + respond(completedResp(batch)), + respond(completedResp(assistantText("done"))), + ], + checker: new FakePermissionChecker({ + echo: { request: { kind: "command" } }, + }), + }); + const turnId = await newTurn(fixture.runtime); + const first = await advanceAndSettle(fixture.runtime, turnId); + return { ...fixture, turnId, first }; + } + + it("records all required permissions and exposes allowed async tools in one suspension", async () => { + const { first } = await setup(); + expect(first.outcome).toMatchObject({ + status: "suspended", + pendingPermissions: [ + expect.objectContaining({ toolCallId: "P1" }), + expect.objectContaining({ toolCallId: "P2" }), + ], + pendingAsyncTools: [expect.objectContaining({ toolCallId: "F1" })], + }); + }); + + it("one approval advances only its tool; denial yields an error result; no model call until all settle", async () => { + const { runtime, repo, turnId } = await setup(); + + const afterAllow = await advanceAndSettle(runtime, turnId, { + type: "permission_decision", + toolCallId: "P1", + decision: "allow", + }); + expect(afterAllow.outcome).toMatchObject({ + status: "suspended", + pendingPermissions: [expect.objectContaining({ toolCallId: "P2" })], + }); + let log = await persisted(repo, turnId); + expect( + log.some( + (e) => e.type === "tool_result" && e.toolCallId === "P1" && e.source === "sync", + ), + ).toBe(true); + + // Async result may arrive while a permission is still unresolved. + const afterAsync = await advanceAndSettle(runtime, turnId, { + type: "async_tool_result", + toolCallId: "F1", + result: { output: "fetched", isError: false }, + }); + expect(afterAsync.outcome?.status).toBe("suspended"); + + const afterDeny = await advanceAndSettle(runtime, turnId, { + type: "permission_decision", + toolCallId: "P2", + decision: "deny", + }); + expect(afterDeny.outcome?.status).toBe("completed"); + + log = await persisted(repo, turnId); + const denial = log.find( + (e) => e.type === "tool_result" && e.toolCallId === "P2", + ); + expect(denial).toMatchObject({ + source: "runtime", + result: { isError: true }, + }); + // Exactly two model calls: the batch, then the follow-up. + expect(log.filter((e) => e.type === "model_call_requested")).toHaveLength(2); + }); + + it("rejects decisions for permissions that are not pending", async () => { + const { runtime, turnId } = await setup(); + await advanceAndSettle(runtime, turnId, { + type: "permission_decision", + toolCallId: "P1", + decision: "allow", + }); + const { error } = await advanceAndSettle(runtime, turnId, { + type: "permission_decision", + toolCallId: "P1", + decision: "deny", + }); + expect(error).toBeInstanceOf(TurnInputError); + }); +}); + +// --------------------------------------------------------------------------- +// §26.4 Automatic permission classification +// --------------------------------------------------------------------------- + +describe("automatic permission classification (26.4)", () => { + const batch = assistantCalls( + toolCallPart("CA", "echo"), + toolCallPart("CD", "echo"), + toolCallPart("CF", "echo"), + ); + const checkerRules = { echo: { request: { kind: "command" as const } } }; + const classifierImpl = (requests: PermissionClassificationInput[]) => + requests.map((r): PermissionClassification => { + const decision = + r.toolCallId === "CA" ? "allow" : r.toolCallId === "CD" ? "deny" : "defer"; + return { toolCallId: r.toolCallId, decision, reason: `because ${decision}` }; + }); + + it("handles allow, deny, and defer in one batch with a human available", async () => { + const { runtime, repo, classifier } = makeRuntime({ + models: [respond(completedResp(batch))], + checker: new FakePermissionChecker(checkerRules), + classifier: new FakePermissionClassifier(classifierImpl), + }); + const turnId = await newTurn(runtime, { + config: { humanAvailable: true, autoPermission: true }, + }); + const { outcome } = await advanceAndSettle(runtime, turnId); + + // Deferred call asks the human. + expect(outcome).toMatchObject({ + status: "suspended", + pendingPermissions: [expect.objectContaining({ toolCallId: "CF" })], + }); + expect(classifier.batches).toHaveLength(1); + expect(classifier.batches[0].map((r) => r.toolCallId)).toEqual([ + "CA", + "CD", + "CF", + ]); + + const log = await persisted(repo, turnId); + // Classifier provenance and effective decisions are distinct records. + expect( + log.filter((e) => e.type === "tool_permission_classified"), + ).toHaveLength(3); + const resolved = log.filter((e) => e.type === "tool_permission_resolved"); + expect(resolved).toEqual([ + expect.objectContaining({ toolCallId: "CA", decision: "allow", source: "classifier" }), + expect.objectContaining({ toolCallId: "CD", decision: "deny", source: "classifier" }), + ]); + // Allow executed; deny got an error result without invocation. + expect( + log.some((e) => e.type === "tool_result" && e.toolCallId === "CA" && e.source === "sync"), + ).toBe(true); + expect( + log.some( + (e) => e.type === "tool_invocation_requested" && e.toolCallId === "CD", + ), + ).toBe(false); + }); + + it("denies deferred calls when no human is available", async () => { + const { runtime, repo } = makeRuntime({ + models: [ + respond(completedResp(batch)), + respond(completedResp(assistantText("done"))), + ], + checker: new FakePermissionChecker(checkerRules), + classifier: new FakePermissionClassifier(classifierImpl), + }); + const turnId = await newTurn(runtime, { + config: { humanAvailable: false, autoPermission: true }, + }); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome?.status).toBe("completed"); + const log = await persisted(repo, turnId); + expect( + log.find( + (e) => e.type === "tool_permission_resolved" && e.toolCallId === "CF", + ), + ).toMatchObject({ decision: "deny", source: "human_unavailable" }); + }); + + it("classifier failure records the failure and defers to the human", async () => { + const { runtime, repo } = makeRuntime({ + models: [respond(completedResp(batch))], + checker: new FakePermissionChecker(checkerRules), + classifier: new FakePermissionClassifier(undefined, "classifier exploded"), + }); + const turnId = await newTurn(runtime, { + config: { humanAvailable: true, autoPermission: true }, + }); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome).toMatchObject({ status: "suspended" }); + expect( + outcome?.status === "suspended" + ? outcome.pendingPermissions.map((p) => p.toolCallId) + : [], + ).toEqual(["CA", "CD", "CF"]); + const log = await persisted(repo, turnId); + expect( + log.find((e) => e.type === "tool_permission_classification_failed"), + ).toMatchObject({ toolCallIds: ["CA", "CD", "CF"] }); + // A later advance does not re-classify failed calls. + const again = await advanceAndSettle(runtime, turnId); + expect(again.outcome?.status).toBe("suspended"); + }); + + it("missing decisions are recorded as per-call classification failures", async () => { + const { runtime, repo } = makeRuntime({ + models: [respond(completedResp(batch))], + checker: new FakePermissionChecker(checkerRules), + classifier: new FakePermissionClassifier((requests) => + requests + .filter((r) => r.toolCallId !== "CF") + .map((r) => ({ + toolCallId: r.toolCallId, + decision: "allow", + reason: "ok", + })), + ), + }); + const turnId = await newTurn(runtime, { + config: { humanAvailable: true, autoPermission: true }, + }); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome).toMatchObject({ + status: "suspended", + pendingPermissions: [expect.objectContaining({ toolCallId: "CF" })], + }); + const log = await persisted(repo, turnId); + expect( + log.find((e) => e.type === "tool_permission_classification_failed"), + ).toMatchObject({ toolCallIds: ["CF"] }); + }); + + it("manual mode never calls the classifier", async () => { + const classifier = new FakePermissionClassifier(); // throws if called + const { runtime } = makeRuntime({ + models: [respond(completedResp(batch))], + checker: new FakePermissionChecker(checkerRules), + classifier, + }); + const turnId = await newTurn(runtime, { + config: { humanAvailable: true, autoPermission: false }, + }); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome?.status).toBe("suspended"); + expect(classifier.batches).toHaveLength(0); + }); + + it("checker failure fails closed: recorded, routed to human, never auto-executed", async () => { + const { runtime, repo, classifier } = makeRuntime({ + models: [respond(completedResp(assistantCalls(toolCallPart("X", "echo"))))], + checker: new FakePermissionChecker({ echo: "throw" }), + classifier: new FakePermissionClassifier(() => [ + { toolCallId: "X", decision: "allow", reason: "should not matter" }, + ]), + }); + const turnId = await newTurn(runtime, { + config: { humanAvailable: true, autoPermission: true }, + }); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome).toMatchObject({ + status: "suspended", + pendingPermissions: [expect.objectContaining({ toolCallId: "X" })], + }); + // The classifier is bypassed for checker-error calls. + expect(classifier.batches).toHaveLength(0); + const log = await persisted(repo, turnId); + expect( + log.find((e) => e.type === "tool_permission_required"), + ).toMatchObject({ checkerError: "checker exploded" }); + expect(log.some((e) => e.type === "tool_invocation_requested")).toBe(false); + }); + + it("checker failure with no human denies without executing", async () => { + const { runtime, repo } = makeRuntime({ + models: [ + respond(completedResp(assistantCalls(toolCallPart("X", "echo")))), + respond(completedResp(assistantText("done"))), + ], + checker: new FakePermissionChecker({ echo: "throw" }), + }); + const turnId = await newTurn(runtime, { + config: { humanAvailable: false, autoPermission: false }, + }); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome?.status).toBe("completed"); + const log = await persisted(repo, turnId); + expect( + log.find((e) => e.type === "tool_permission_resolved"), + ).toMatchObject({ decision: "deny", source: "human_unavailable" }); + expect(log.some((e) => e.type === "tool_invocation_requested")).toBe(false); + }); +}); + +// --------------------------------------------------------------------------- +// Human-dependent tools +// --------------------------------------------------------------------------- + +describe("human-dependent tools", () => { + it("ask-human suspends as a pending async tool when a human is available", async () => { + const { runtime } = makeRuntime({ + models: [ + respond(completedResp(assistantCalls(toolCallPart("H", "ask-human")))), + respond(completedResp(assistantText("done"))), + ], + }); + const turnId = await newTurn(runtime); + const first = await advanceAndSettle(runtime, turnId); + expect(first.outcome).toMatchObject({ + status: "suspended", + pendingAsyncTools: [expect.objectContaining({ toolCallId: "H" })], + }); + const second = await advanceAndSettle(runtime, turnId, { + type: "async_tool_result", + toolCallId: "H", + result: { output: "42", isError: false }, + }); + expect(second.outcome?.status).toBe("completed"); + }); + + it("requiresHuman tools get an immediate error result when no human is available", async () => { + const { runtime, repo } = makeRuntime({ + models: [ + respond(completedResp(assistantCalls(toolCallPart("H", "ask-human")))), + respond(completedResp(assistantText("done"))), + ], + }); + const turnId = await newTurn(runtime, { + config: { humanAvailable: false }, + }); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome?.status).toBe("completed"); + const log = await persisted(repo, turnId); + expect( + log.some((e) => e.type === "tool_invocation_requested" && e.toolCallId === "H"), + ).toBe(true); + expect(log.find((e) => e.type === "tool_result" && e.toolCallId === "H")).toMatchObject({ + source: "runtime", + result: { isError: true }, + }); + }); +}); + +// --------------------------------------------------------------------------- +// §26.5 Cancellation +// --------------------------------------------------------------------------- + +describe("cancellation (26.5)", () => { + it("cancels a suspended turn via the cancel input", async () => { + const { runtime, repo } = makeRuntime({ + models: [ + respond(completedResp(assistantCalls(toolCallPart("B", "fetch")))), + ], + }); + const turnId = await newTurn(runtime); + await advanceAndSettle(runtime, turnId); + + const { outcome } = await advanceAndSettle(runtime, turnId, { + type: "cancel", + reason: "user stop", + }); + expect(outcome).toMatchObject({ status: "cancelled", reason: "user stop" }); + + const log = await persisted(repo, turnId); + expect(log.find((e) => e.type === "tool_result" && e.toolCallId === "B")).toMatchObject({ + source: "runtime", + result: { isError: true }, + }); + expect(log[log.length - 1]).toMatchObject({ + type: "turn_cancelled", + reason: "user stop", + }); + + // Late external inputs are rejected. + const late = await advanceAndSettle(runtime, turnId, { + type: "async_tool_result", + toolCallId: "B", + result: { output: "late", isError: false }, + }); + expect(late.error).toBeInstanceOf(TurnInputError); + }); + + it("cancels during a model call via the abort signal", async () => { + const controller = new AbortController(); + let abortNow: (() => void) | undefined; + const started = new Promise((resolve) => { + abortNow = () => resolve(); + }); + const { runtime, repo } = makeRuntime({ + models: [hangUntilAbort(() => abortNow?.())], + }); + const turnId = await newTurn(runtime); + const execution = runtime.advanceTurn(turnId, undefined, { + signal: controller.signal, + }); + await started; + controller.abort(); + const { outcome } = await settle(execution); + expect(outcome?.status).toBe("cancelled"); + const log = await persisted(repo, turnId); + expect(typesOf(log)).toEqual([ + "turn_created", + "model_call_requested", + "model_call_failed", + "turn_cancelled", + ]); + }); + + it("cancels during a sync tool with a synthetic result", async () => { + const controller = new AbortController(); + const tools: RuntimeTool[] = [ + syncTool(echoDescriptor, async (_input, ctx: ToolExecutionContext) => { + controller.abort(); + if (!ctx.signal.aborted) { + await new Promise((resolve) => { + ctx.signal.addEventListener("abort", () => resolve(), { + once: true, + }); + }); + } + throw new Error("aborted mid-tool"); + }), + { descriptor: fetchDescriptor as { execution: "async" } & typeof fetchDescriptor }, + { descriptor: askHumanDescriptor as { execution: "async" } & typeof askHumanDescriptor }, + ]; + const { runtime, repo } = makeRuntime({ + models: [respond(completedResp(assistantCalls(toolCallPart("S", "echo"))))], + tools, + }); + const turnId = await newTurn(runtime); + const { outcome } = await advanceAndSettle(runtime, turnId, undefined, { + signal: controller.signal, + }); + expect(outcome?.status).toBe("cancelled"); + const log = await persisted(repo, turnId); + expect(log.find((e) => e.type === "tool_result" && e.toolCallId === "S")).toMatchObject({ + source: "runtime", + result: { isError: true }, + }); + expect(log[log.length - 1].type).toBe("turn_cancelled"); + }); +}); + +// --------------------------------------------------------------------------- +// §26.6 Failures +// --------------------------------------------------------------------------- + +describe("failures (26.6)", () => { + it("provider failure appends model and turn failures", async () => { + const { runtime, repo } = makeRuntime({ models: [failCall("boom")] }); + const turnId = await newTurn(runtime); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome).toMatchObject({ status: "failed", error: "boom" }); + const log = await persisted(repo, turnId); + expect(typesOf(log)).toEqual([ + "turn_created", + "model_call_requested", + "model_call_failed", + "turn_failed", + ]); + }); + + it("a sync tool throw becomes an error result and the turn continues", async () => { + const tools: RuntimeTool[] = [ + syncTool(echoDescriptor, async () => { + throw new Error("tool exploded"); + }), + { descriptor: fetchDescriptor as { execution: "async" } & typeof fetchDescriptor }, + { descriptor: askHumanDescriptor as { execution: "async" } & typeof askHumanDescriptor }, + ]; + const { runtime, repo } = makeRuntime({ + models: [ + respond(completedResp(assistantCalls(toolCallPart("T", "echo")))), + respond(completedResp(assistantText("recovered"))), + ], + tools, + }); + const turnId = await newTurn(runtime); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome).toMatchObject({ + status: "completed", + output: assistantText("recovered"), + }); + const log = await persisted(repo, turnId); + expect(log.find((e) => e.type === "tool_result")).toMatchObject({ + source: "sync", + result: { output: "tool exploded", isError: true }, + }); + }); + + it("an async failure result becomes an error tool result and the turn continues", async () => { + const { runtime } = makeRuntime({ + models: [ + respond(completedResp(assistantCalls(toolCallPart("B", "fetch")))), + respond(completedResp(assistantText("done"))), + ], + }); + const turnId = await newTurn(runtime); + await advanceAndSettle(runtime, turnId); + const { outcome } = await advanceAndSettle(runtime, turnId, { + type: "async_tool_result", + toolCallId: "B", + result: { output: "network unreachable", isError: true }, + }); + expect(outcome?.status).toBe("completed"); + }); + + it("unknown tools become runtime error results, not turn failures", async () => { + const { runtime, repo } = makeRuntime({ + models: [ + respond(completedResp(assistantCalls(toolCallPart("U", "no-such-tool")))), + respond(completedResp(assistantText("done"))), + ], + }); + const turnId = await newTurn(runtime); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome?.status).toBe("completed"); + const log = await persisted(repo, turnId); + expect(log.find((e) => e.type === "tool_result" && e.toolCallId === "U")).toMatchObject({ + source: "runtime", + result: { isError: true }, + }); + }); + + it("repository errors reject stream and outcome without a false turn_failed", async () => { + const { runtime } = makeRuntime(); + const execution = runtime.advanceTurn("2026-07-02T10-00-00Z-9999999-000"); + await expect(execution.outcome).rejects.toThrowError(/turn not found/); + await expect( + (async () => { + for await (const event of execution.events) { + void event; // drain + } + })(), + ).rejects.toThrowError(/turn not found/); + }); + + it("missing live tools reject the execution and leave the turn unchanged", async () => { + const { runtime, repo } = makeRuntime({ + models: [respond(completedResp(assistantText("done")))], + tools: [], // registry knows nothing + }); + const turnId = await newTurn(runtime); + const { error } = await advanceAndSettle(runtime, turnId); + expect(error).toBeInstanceOf(TurnDependencyError); + expect((await persisted(repo, turnId)).length).toBe(1); // only turn_created + }); + + it("agent resolution failure rejects createTurn without creating a file", async () => { + const { runtime, repo } = makeRuntime({ agentError: "no such agent" }); + await expect(newTurn(runtime)).rejects.toThrowError("no such agent"); + await expect( + repo.read("2026-07-02T10-00-00Z-0000001-000"), + ).rejects.toThrowError(/turn not found/); + }); +}); + +// --------------------------------------------------------------------------- +// §26.7 Crash recovery +// --------------------------------------------------------------------------- + +describe("crash recovery (26.7)", () => { + const SEED_ID = "2026-07-02T10-00-00Z-0000001-000"; + + function seedCreated(config?: { + autoPermission: boolean; + humanAvailable: boolean; + maxModelCalls: number; + }): z.infer { + return { + type: "turn_created", + schemaVersion: 1, + turnId: SEED_ID, + ts: TS, + sessionId: null, + agent: { requested: { agentId: "copilot" }, resolved: defaultAgent }, + context: [], + input: user("hello"), + config: config ?? { + autoPermission: false, + humanAvailable: true, + maxModelCalls: 20, + }, + }; + } + + function seedRequested(index: number, messages: Array | ReturnType | { role: "assistant"; content: string } | { role: "tool"; content: string; toolCallId: string; toolName: string }>): z.infer { + return { + type: "model_call_requested", + turnId: SEED_ID, + ts: TS, + modelCallIndex: index, + request: { + systemPrompt: "SYS", + messages, + tools: defaultAgent.tools, + parameters: {}, + }, + }; + } + + function seedCompleted(index: number, message: Parameters[0]): z.infer { + return { + type: "model_call_completed", + turnId: SEED_ID, + ts: TS, + modelCallIndex: index, + message, + finishReason: "stop", + usage: {}, + }; + } + + it("created-only log initiates the first model call", async () => { + const repo = new InMemoryTurnRepo(); + repo.seed([seedCreated()]); + const { runtime } = makeRuntime({ + repo, + models: [respond(completedResp(assistantText("done")))], + }); + const { outcome } = await advanceAndSettle(runtime, SEED_ID); + expect(outcome?.status).toBe("completed"); + }); + + it("an unmatched model request is closed as interrupted and re-issued", async () => { + const repo = new InMemoryTurnRepo(); + repo.seed([seedCreated(), seedRequested(0, [user("hello")])]); + const { runtime, models } = makeRuntime({ + repo, + models: [respond(completedResp(assistantText("done")))], + }); + const { outcome } = await advanceAndSettle(runtime, SEED_ID); + expect(outcome?.status).toBe("completed"); + const log = await persisted(repo, SEED_ID); + expect(typesOf(log)).toEqual([ + "turn_created", + "model_call_requested", + "model_call_failed", + "model_call_requested", + "model_call_completed", + "turn_completed", + ]); + expect(log[2]).toMatchObject({ error: expect.stringMatching(/interrupted/) }); + expect(models.requests).toHaveLength(1); // only the re-issued call hit the provider + }); + + it("the re-issued call counts against the model-call budget", async () => { + const repo = new InMemoryTurnRepo(); + repo.seed([ + seedCreated({ + autoPermission: false, + humanAvailable: true, + maxModelCalls: 1, + }), + seedRequested(0, [user("hello")]), + ]); + const { runtime } = makeRuntime({ repo }); + const { outcome } = await advanceAndSettle(runtime, SEED_ID); + expect(outcome).toMatchObject({ + status: "failed", + code: MODEL_CALL_LIMIT_ERROR_CODE, + }); + }); + + it("an unmatched sync invocation gets an indeterminate result and the turn continues", async () => { + const repo = new InMemoryTurnRepo(); + const batch = assistantCalls(toolCallPart("S", "echo")); + repo.seed([ + seedCreated(), + seedRequested(0, [user("hello")]), + seedCompleted(0, batch), + { + type: "tool_invocation_requested", + turnId: SEED_ID, + ts: TS, + toolCallId: "S", + toolId: "tool.echo", + toolName: "echo", + execution: "sync", + input: {}, + }, + ]); + const { runtime } = makeRuntime({ + repo, + models: [respond(completedResp(assistantText("done")))], + }); + const { outcome } = await advanceAndSettle(runtime, SEED_ID); + expect(outcome?.status).toBe("completed"); + const log = await persisted(repo, SEED_ID); + expect(log.find((e) => e.type === "tool_result" && e.toolCallId === "S")).toMatchObject({ + source: "runtime", + result: { + output: expect.stringMatching(/interrupted; its outcome is unknown/), + isError: true, + }, + }); + // No turn_failed anywhere: the turn completed. + expect(typesOf(log)).not.toContain("turn_failed"); + }); + + it("an unmatched async invocation remains suspended, appending the missing snapshot", async () => { + const repo = new InMemoryTurnRepo(); + const batch = assistantCalls(toolCallPart("B", "fetch")); + repo.seed([ + seedCreated(), + seedRequested(0, [user("hello")]), + seedCompleted(0, batch), + { + type: "tool_invocation_requested", + turnId: SEED_ID, + ts: TS, + toolCallId: "B", + toolId: "tool.fetch", + toolName: "fetch", + execution: "async", + input: {}, + }, + ]); + const { runtime } = makeRuntime({ repo }); + const { outcome } = await advanceAndSettle(runtime, SEED_ID); + expect(outcome).toMatchObject({ + status: "suspended", + pendingAsyncTools: [expect.objectContaining({ toolCallId: "B" })], + }); + const log = await persisted(repo, SEED_ID); + expect(log[log.length - 1].type).toBe("turn_suspended"); + }); + + it("re-advancing an already-suspended turn appends no duplicate snapshot", async () => { + const { runtime, repo } = makeRuntime({ + models: [respond(completedResp(assistantCalls(toolCallPart("B", "fetch"))))], + }); + const turnId = await newTurn(runtime); + await advanceAndSettle(runtime, turnId); + const before = (await persisted(repo, turnId)).length; + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome?.status).toBe("suspended"); + expect((await persisted(repo, turnId)).length).toBe(before); + }); + + it("a completed tool batch proceeds to the next model call", async () => { + const repo = new InMemoryTurnRepo(); + const batch = assistantCalls(toolCallPart("S", "echo")); + repo.seed([ + seedCreated(), + seedRequested(0, [user("hello")]), + seedCompleted(0, batch), + { + type: "tool_invocation_requested", + turnId: SEED_ID, + ts: TS, + toolCallId: "S", + toolId: "tool.echo", + toolName: "echo", + execution: "sync", + input: {}, + }, + { + type: "tool_result", + turnId: SEED_ID, + ts: TS, + toolCallId: "S", + toolName: "echo", + source: "sync", + result: { output: "ok", isError: false }, + }, + ]); + const { runtime, models } = makeRuntime({ + repo, + models: [respond(completedResp(assistantText("done")))], + }); + const { outcome } = await advanceAndSettle(runtime, SEED_ID); + expect(outcome?.status).toBe("completed"); + expect( + models.requests[0].messages.filter((m) => m.role === "tool"), + ).toHaveLength(1); + }); + + it("a permission resolved allow before invocation safely invokes the tool", async () => { + const repo = new InMemoryTurnRepo(); + const batch = assistantCalls(toolCallPart("P", "echo")); + repo.seed([ + seedCreated(), + seedRequested(0, [user("hello")]), + seedCompleted(0, batch), + { + type: "tool_permission_required", + turnId: SEED_ID, + ts: TS, + toolCallId: "P", + toolName: "echo", + request: {}, + }, + { + type: "tool_permission_resolved", + turnId: SEED_ID, + ts: TS, + toolCallId: "P", + decision: "allow", + source: "human", + }, + ]); + const { runtime, repo: r } = makeRuntime({ + repo, + models: [respond(completedResp(assistantText("done")))], + }); + const { outcome } = await advanceAndSettle(runtime, SEED_ID); + expect(outcome?.status).toBe("completed"); + const log = await persisted(r, SEED_ID); + expect(log.some((e) => e.type === "tool_invocation_requested" && e.toolCallId === "P")).toBe(true); + expect(log.find((e) => e.type === "tool_result" && e.toolCallId === "P")).toMatchObject({ + source: "sync", + result: { isError: false }, + }); + }); + + it("a terminal turn returns its outcome and performs no writes", async () => { + const { runtime, repo } = makeRuntime({ + models: [respond(completedResp(assistantText("done")))], + }); + const turnId = await newTurn(runtime); + await advanceAndSettle(runtime, turnId); + const before = (await persisted(repo, turnId)).length; + + const again = await advanceAndSettle(runtime, turnId); + expect(again.outcome).toMatchObject({ + status: "completed", + output: assistantText("done"), + }); + expect((await persisted(repo, turnId)).length).toBe(before); + + const withInput = await advanceAndSettle(runtime, turnId, { + type: "cancel", + }); + expect(withInput.error).toBeInstanceOf(TurnInputError); + }); +}); + +// --------------------------------------------------------------------------- +// §26.8 Historical and live reconstruction +// --------------------------------------------------------------------------- + +describe("historical and live reconstruction (26.8)", () => { + it("getTurn is read-only and matches live durable events", async () => { + const { runtime, repo } = makeRuntime({ + models: [ + respond( + { type: "text_delta", delta: "d" }, + completedResp(assistantText("done")), + ), + ], + }); + const turnId = await newTurn(runtime); + const { events } = await advanceAndSettle(runtime, turnId); + + const before = (await persisted(repo, turnId)).length; + const turn = await runtime.getTurn(turnId); + expect((await persisted(repo, turnId)).length).toBe(before); + + const durable = events.filter( + (e) => e.type !== "text_delta" && e.type !== "reasoning_delta", + ); + expect(turn.events.slice(1)).toEqual(durable); + // Ephemeral deltas are absent after reload. + expect(turn.events.some((e) => (e.type as string) === "text_delta")).toBe(false); + }); +}); + +// --------------------------------------------------------------------------- +// §26.9 Model-call limit +// --------------------------------------------------------------------------- + +describe("model-call limit (26.9)", () => { + it("a tool-free response on the final allowed call completes normally", async () => { + const { runtime } = makeRuntime({ + models: [ + respond(completedResp(assistantCalls(toolCallPart("S", "echo")))), + respond(completedResp(assistantText("done"))), + ], + }); + const turnId = await newTurn(runtime, { + config: { humanAvailable: true, maxModelCalls: 2 }, + }); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome?.status).toBe("completed"); + }); + + it("tool calls on the final call are fully processed, then the turn fails with the limit code", async () => { + const { runtime, repo, models } = makeRuntime({ + models: [respond(completedResp(assistantCalls(toolCallPart("S", "echo"))))], + }); + const turnId = await newTurn(runtime, { + config: { humanAvailable: true, maxModelCalls: 1 }, + }); + const { outcome } = await advanceAndSettle(runtime, turnId); + expect(outcome).toMatchObject({ + status: "failed", + code: MODEL_CALL_LIMIT_ERROR_CODE, + }); + const log = await persisted(repo, turnId); + // The batch was fully processed before failing… + expect(log.find((e) => e.type === "tool_result" && e.toolCallId === "S")).toMatchObject({ + result: { isError: false }, + }); + // …and no further model call was made. + expect(models.requests).toHaveLength(1); + expect(log[log.length - 1]).toMatchObject({ + type: "turn_failed", + code: MODEL_CALL_LIMIT_ERROR_CODE, + }); + }); +}); + +// --------------------------------------------------------------------------- +// Tool progress +// --------------------------------------------------------------------------- + +describe("tool progress", () => { + it("sync progress persists before the callback resolves; async progress arrives as input", async () => { + const tools: RuntimeTool[] = [ + syncTool(echoDescriptor, async (_input, ctx) => { + await ctx.reportProgress({ pct: 50 }); + return { output: "done", isError: false }; + }), + { descriptor: fetchDescriptor as { execution: "async" } & typeof fetchDescriptor }, + { descriptor: askHumanDescriptor as { execution: "async" } & typeof askHumanDescriptor }, + ]; + const { runtime, repo } = makeRuntime({ + models: [ + respond( + completedResp( + assistantCalls(toolCallPart("S", "echo"), toolCallPart("B", "fetch")), + ), + ), + respond(completedResp(assistantText("done"))), + ], + tools, + }); + const turnId = await newTurn(runtime); + await advanceAndSettle(runtime, turnId); + + await advanceAndSettle(runtime, turnId, { + type: "async_tool_progress", + toolCallId: "B", + progress: { pct: 10 }, + }); + const { outcome } = await advanceAndSettle(runtime, turnId, { + type: "async_tool_result", + toolCallId: "B", + result: { output: "fetched", isError: false }, + }); + expect(outcome?.status).toBe("completed"); + + const log = await persisted(repo, turnId); + const progress = log.filter((e) => e.type === "tool_progress"); + expect(progress).toEqual([ + expect.objectContaining({ toolCallId: "S", source: "sync" }), + expect.objectContaining({ toolCallId: "B", source: "async" }), + ]); + }); +}); diff --git a/apps/x/packages/core/src/turns/runtime.ts b/apps/x/packages/core/src/turns/runtime.ts new file mode 100644 index 00000000..4a532a6e --- /dev/null +++ b/apps/x/packages/core/src/turns/runtime.ts @@ -0,0 +1,928 @@ +import type { z } from "zod"; +import { + DEFAULT_MAX_MODEL_CALLS, + MODEL_CALL_LIMIT_ERROR_CODE, + type JsonValue, + type ModelCallFailed, + type ModelRequest, + type ToolCallState, + type ToolInvocationRequested, + ToolResultData, + type ToolDescriptor, + type ToolPermissionRequired, + type ToolPermissionResolved, + type ToolResult, + TurnCreated, + type TurnEvent, + type TurnState, + type TurnStreamEvent, + type TurnSuspended, + outstandingAsyncTools, + outstandingPermissions, + reduceTurn, + turnTranscript, +} from "@x/shared/dist/turns.js"; +import type { IMonotonicallyIncreasingIdGenerator } from "../application/lib/id-gen.js"; +import type { IAgentResolver } from "./agent-resolver.js"; +import { + type CreateTurnInput, + type ITurnRuntime, + type Turn, + TurnDependencyError, + type TurnExecution, + type TurnExternalInput, + TurnInputError, + type TurnOutcome, +} from "./api.js"; +import type { ITurnLifecycleBus } from "./bus.js"; +import type { IClock } from "./clock.js"; +import type { IContextResolver } from "./context-resolver.js"; +import type { IModelRegistry, LlmStreamEvent } from "./model-registry.js"; +import type { IPermissionChecker, IPermissionClassifier } from "./permission.js"; +import type { ITurnRepo } from "./repo.js"; +import { HotStream } from "./stream.js"; +import type { IToolRegistry, RuntimeTool, SyncRuntimeTool } from "./tool-registry.js"; + +type TEvent = z.infer; + +const INTERRUPTED_TOOL_MESSAGE = + "Tool execution was interrupted; its outcome is unknown and it was not retried."; + +export interface TurnRuntimeDependencies { + turnRepo: ITurnRepo; + idGenerator: IMonotonicallyIncreasingIdGenerator; + clock: IClock; + agentResolver: IAgentResolver; + modelRegistry: IModelRegistry; + toolRegistry: IToolRegistry; + contextResolver: IContextResolver; + permissionChecker: IPermissionChecker; + permissionClassifier: IPermissionClassifier; + bus: ITurnLifecycleBus; +} + +// Immutable dependency container: holds no mutable per-turn state. All active +// turn state is reconstructed from the repository inside each invocation. +export class TurnRuntime implements ITurnRuntime { + private readonly turnRepo: ITurnRepo; + private readonly idGenerator: IMonotonicallyIncreasingIdGenerator; + private readonly clock: IClock; + private readonly agentResolver: IAgentResolver; + private readonly modelRegistry: IModelRegistry; + private readonly toolRegistry: IToolRegistry; + private readonly contextResolver: IContextResolver; + private readonly permissionChecker: IPermissionChecker; + private readonly permissionClassifier: IPermissionClassifier; + private readonly bus: ITurnLifecycleBus; + + constructor({ + turnRepo, + idGenerator, + clock, + agentResolver, + modelRegistry, + toolRegistry, + contextResolver, + permissionChecker, + permissionClassifier, + bus, + }: TurnRuntimeDependencies) { + this.turnRepo = turnRepo; + this.idGenerator = idGenerator; + this.clock = clock; + this.agentResolver = agentResolver; + this.modelRegistry = modelRegistry; + this.toolRegistry = toolRegistry; + this.contextResolver = contextResolver; + this.permissionChecker = permissionChecker; + this.permissionClassifier = permissionClassifier; + this.bus = bus; + } + + async createTurn(input: CreateTurnInput): Promise { + const resolved = await this.agentResolver.resolve(input.agent); + const turnId = await this.idGenerator.next(); + const event = TurnCreated.parse({ + type: "turn_created", + schemaVersion: 1, + turnId, + ts: this.clock.now(), + sessionId: input.sessionId ?? null, + agent: { requested: input.agent, resolved }, + context: input.context, + input: input.input, + config: { + autoPermission: input.config.autoPermission ?? false, + humanAvailable: input.config.humanAvailable, + maxModelCalls: input.config.maxModelCalls ?? DEFAULT_MAX_MODEL_CALLS, + }, + }); + await this.turnRepo.create(event); + return turnId; + } + + async getTurn(turnId: string): Promise { + const events = await this.turnRepo.read(turnId); + return { turnId, events }; + } + + advanceTurn( + turnId: string, + input?: TurnExternalInput, + options?: { signal?: AbortSignal }, + ): TurnExecution { + const stream = new HotStream(); + void this.turnRepo + .withLock(turnId, () => + this.advanceLocked(turnId, input, options?.signal, stream), + ) + .then( + (outcome) => stream.end(outcome), + (error: unknown) => stream.fail(error), + ); + return { events: stream.events, outcome: stream.outcome }; + } + + private async advanceLocked( + turnId: string, + input: TurnExternalInput | undefined, + externalSignal: AbortSignal | undefined, + stream: HotStream, + ): Promise { + this.bus.publish({ type: "turn-processing-start", turnId }); + try { + return await this.advance(turnId, input, externalSignal, stream); + } finally { + this.bus.publish({ type: "turn-processing-end", turnId }); + } + } + + private async advance( + turnId: string, + input: TurnExternalInput | undefined, + externalSignal: AbortSignal | undefined, + stream: HotStream, + ): Promise { + const events = await this.turnRepo.read(turnId); + let state = reduceTurn(events); + + if (state.terminal) { + if (input) { + throw new TurnInputError( + `turn ${turnId} is terminal; input rejected`, + ); + } + return outcomeFromTerminal(state); + } + + const definition = state.definition; + + // Materialize context and live dependencies. Failures here are + // infrastructure errors: the execution rejects, the turn is unchanged. + const resolvedContext = await this.contextResolver.resolve( + definition.context, + ); + const model = await this.modelRegistry.resolve( + definition.agent.resolved.model, + ); + const toolsByName = new Map(); + for (const descriptor of definition.agent.resolved.tools) { + const tool = await this.toolRegistry.resolve(descriptor); + if ( + tool.descriptor.toolId !== descriptor.toolId || + tool.descriptor.execution !== descriptor.execution + ) { + throw new TurnDependencyError( + `resolved tool ${descriptor.toolId} does not match its persisted descriptor`, + ); + } + toolsByName.set(descriptor.name, tool); + } + + const controller = new AbortController(); + const forwardAbort = () => controller.abort(); + if (externalSignal) { + if (externalSignal.aborted) { + controller.abort(); + } else { + externalSignal.addEventListener("abort", forwardAbort, { + once: true, + }); + } + } + + let appended = false; + const append = async (...batch: TEvent[]): Promise => { + await this.turnRepo.append(turnId, batch); + events.push(...batch); + state = reduceTurn(events); + appended = true; + for (const event of batch) { + stream.push(event); + } + }; + const now = () => this.clock.now(); + // Checker "allowed" outcomes are deliberately not durable: after a + // crash the checker is simply re-consulted. + const checkerAllowed = new Set(); + let cancelReason: string | undefined; + + const cancelTurn = async (): Promise => { + const open = state.modelCalls.find( + (c) => c.response === undefined && c.error === undefined, + ); + if (open) { + await append( + modelCallFailedEvent(turnId, now(), open.index, "model call was cancelled"), + ); + } + for (const tc of state.toolCalls.filter((t) => !t.result)) { + await append( + runtimeResultEvent(turnId, now(), tc, { + output: "Tool call was cancelled before completion.", + isError: true, + }), + ); + } + await append({ + type: "turn_cancelled", + turnId, + ts: now(), + ...(cancelReason === undefined ? {} : { reason: cancelReason }), + usage: state.usage, + }); + return { + status: "cancelled", + ...(cancelReason === undefined ? {} : { reason: cancelReason }), + usage: state.usage, + }; + }; + + try { + // Apply the optional single external input against durable + // pending state. + if (input) { + switch (input.type) { + case "cancel": + cancelReason = input.reason; + return await cancelTurn(); + case "permission_decision": { + const tc = state.toolCalls.find( + (t) => t.toolCallId === input.toolCallId, + ); + if (!tc?.permission || tc.permission.resolved || tc.result) { + throw new TurnInputError( + `no pending permission for tool call ${input.toolCallId}`, + ); + } + await append({ + type: "tool_permission_resolved", + turnId, + ts: now(), + toolCallId: input.toolCallId, + decision: input.decision, + source: "human", + ...(input.metadata === undefined + ? {} + : { metadata: input.metadata }), + }); + if (input.decision === "deny") { + await append( + runtimeResultEvent(turnId, now(), tc, { + output: "Permission denied by user.", + isError: true, + }), + ); + } + break; + } + case "async_tool_progress": { + const tc = requirePendingAsync(state, input.toolCallId); + await append({ + type: "tool_progress", + turnId, + ts: now(), + toolCallId: tc.toolCallId, + source: "async", + progress: input.progress, + }); + break; + } + case "async_tool_result": { + const tc = requirePendingAsync(state, input.toolCallId); + await append({ + type: "tool_result", + turnId, + ts: now(), + toolCallId: tc.toolCallId, + toolName: tc.toolName, + source: "async", + result: input.result, + }); + break; + } + } + } + + for (;;) { + if (controller.signal.aborted) { + return await cancelTurn(); + } + + // Recovery: close a model call interrupted by a crash, then + // re-issue it as a new call (counts against the budget). + const open = state.modelCalls.find( + (c) => c.response === undefined && c.error === undefined, + ); + if (open) { + await append( + modelCallFailedEvent( + turnId, + now(), + open.index, + "model call was interrupted before a response was recorded", + ), + ); + continue; + } + + // Recovery: close sync invocations interrupted by a crash + // with an indeterminate result; the turn continues. + const interruptedSync = state.toolCalls.filter( + (tc) => tc.invocation && tc.execution === "sync" && !tc.result, + ); + if (interruptedSync.length > 0) { + for (const tc of interruptedSync) { + await append( + runtimeResultEvent(turnId, now(), tc, { + output: INTERRUPTED_TOOL_MESSAGE, + isError: true, + }), + ); + } + continue; + } + + // Permission requirements for freshly extracted tool calls. + const fresh = state.toolCalls.filter( + (tc) => + !tc.result && + !tc.invocation && + !tc.permission && + !checkerAllowed.has(tc.toolCallId), + ); + for (const tc of fresh) { + if (controller.signal.aborted) { + break; + } + const tool = toolsByName.get(tc.toolName); + if (!tool) { + await append( + runtimeResultEvent(turnId, now(), tc, { + output: `Unknown tool: ${tc.toolName}`, + isError: true, + }), + ); + continue; + } + if ( + tool.descriptor.requiresHuman && + !definition.config.humanAvailable + ) { + await append( + invocationEvent(turnId, now(), tc, tool.descriptor), + ); + await append( + runtimeResultEvent(turnId, now(), tc, { + output: "Human input is unavailable for this turn.", + isError: true, + }), + ); + continue; + } + try { + const check = await this.permissionChecker.check({ + turnId, + toolCallId: tc.toolCallId, + toolId: tool.descriptor.toolId, + toolName: tc.toolName, + input: tc.input, + }); + if (!check.required) { + checkerAllowed.add(tc.toolCallId); + } else { + await append( + permissionRequiredEvent( + turnId, + now(), + tc, + check.request, + ), + ); + } + } catch (error) { + // Checker failure fails closed: record it and route + // to a human (or denial below); never execute. + await append( + permissionRequiredEvent(turnId, now(), tc, {}, errorMessage(error)), + ); + } + } + if (controller.signal.aborted) { + return await cancelTurn(); + } + + // Automatic classification, one batch. Checker-error calls + // and previously failed classifications go straight to the + // human/deny fallback. + if (definition.config.autoPermission) { + const candidates = state.toolCalls.filter( + (tc) => + tc.permission && + !tc.permission.resolved && + !tc.permission.classification && + !tc.permission.classificationFailed && + tc.permission.required.checkerError === undefined && + !tc.result, + ); + if (candidates.length > 0) { + try { + const decisions = await this.permissionClassifier.classify( + candidates.map((tc) => ({ + toolCallId: tc.toolCallId, + toolName: tc.toolName, + input: tc.input, + request: tc.permission!.required.request, + })), + controller.signal, + ); + for (const tc of candidates) { + const decision = decisions.find( + (d) => d.toolCallId === tc.toolCallId, + ); + if (!decision) { + await append({ + type: "tool_permission_classification_failed", + turnId, + ts: now(), + toolCallIds: [tc.toolCallId], + error: "classifier returned no decision", + }); + continue; + } + await append({ + type: "tool_permission_classified", + turnId, + ts: now(), + toolCallId: tc.toolCallId, + decision: decision.decision, + reason: decision.reason, + }); + if (decision.decision === "allow") { + await append( + resolvedEvent(turnId, now(), tc.toolCallId, "allow", "classifier", decision.reason), + ); + } else if (decision.decision === "deny") { + await append( + resolvedEvent(turnId, now(), tc.toolCallId, "deny", "classifier", decision.reason), + ); + await append( + runtimeResultEvent(turnId, now(), tc, { + output: `Permission denied: ${decision.reason}`, + isError: true, + }), + ); + } + // "defer" falls through to human/deny fallback. + } + } catch (error) { + if (controller.signal.aborted) { + return await cancelTurn(); + } + await append({ + type: "tool_permission_classification_failed", + turnId, + ts: now(), + toolCallIds: candidates.map((c) => c.toolCallId), + error: errorMessage(error), + }); + } + } + } + + // No human available: deny whatever remains unresolved. + if (!definition.config.humanAvailable) { + const unresolved = state.toolCalls.filter( + (tc) => tc.permission && !tc.permission.resolved && !tc.result, + ); + for (const tc of unresolved) { + await append( + resolvedEvent(turnId, now(), tc.toolCallId, "deny", "human_unavailable"), + ); + await append( + runtimeResultEvent(turnId, now(), tc, { + output: "Permission denied: no human is available for this turn.", + isError: true, + }), + ); + } + } + + // Execute allowed sync tools sequentially; expose allowed + // async tools. Source order. + const executable = state.toolCalls.filter( + (tc) => + !tc.result && + !tc.invocation && + (checkerAllowed.has(tc.toolCallId) || + tc.permission?.resolved?.decision === "allow"), + ); + for (const tc of executable) { + if (controller.signal.aborted) { + break; + } + const tool = toolsByName.get(tc.toolName); + if (!tool) { + await append( + runtimeResultEvent(turnId, now(), tc, { + output: `Unknown tool: ${tc.toolName}`, + isError: true, + }), + ); + continue; + } + await append(invocationEvent(turnId, now(), tc, tool.descriptor)); + if (tool.descriptor.execution === "async") { + continue; + } + const syncTool = tool as SyncRuntimeTool; + try { + const result = await syncTool.execute(tc.input, { + signal: controller.signal, + reportProgress: async (progress) => { + await append({ + type: "tool_progress", + turnId, + ts: now(), + toolCallId: tc.toolCallId, + source: "sync", + progress, + }); + }, + }); + await append({ + type: "tool_result", + turnId, + ts: now(), + toolCallId: tc.toolCallId, + toolName: tc.toolName, + source: "sync", + result: ToolResultData.parse(result), + }); + } catch (error) { + if (controller.signal.aborted) { + await append( + runtimeResultEvent(turnId, now(), tc, { + output: "Tool execution was cancelled.", + isError: true, + }), + ); + break; + } + // A tool failure is conversational, not terminal. + await append({ + type: "tool_result", + turnId, + ts: now(), + toolCallId: tc.toolCallId, + toolName: tc.toolName, + source: "sync", + result: { output: errorMessage(error), isError: true }, + }); + } + } + if (controller.signal.aborted) { + return await cancelTurn(); + } + + // Suspend while external work remains outstanding. + const pendingPerms = outstandingPermissions(state); + const pendingAsync = outstandingAsyncTools(state); + if (pendingPerms.length + pendingAsync.length > 0) { + const last = events[events.length - 1]; + if (appended || last.type !== "turn_suspended") { + await append({ + type: "turn_suspended", + turnId, + ts: now(), + pendingPermissions: permissionsSnapshot(pendingPerms), + pendingAsyncTools: asyncSnapshot(pendingAsync), + usage: state.usage, + }); + } + return { + status: "suspended", + pendingPermissions: permissionsSnapshot(pendingPerms), + pendingAsyncTools: asyncSnapshot(pendingAsync), + usage: state.usage, + }; + } + + // Tool batch complete. Completion, budget, or the next call. + const lastCall = state.modelCalls[state.modelCalls.length - 1]; + if ( + lastCall?.response !== undefined && + state.toolCalls.every((tc) => tc.modelCallIndex !== lastCall.index) + ) { + const output = lastCall.response; + const finishReason = lastCall.finishReason ?? "unknown"; + await append({ + type: "turn_completed", + turnId, + ts: now(), + output, + finishReason, + usage: state.usage, + }); + return { + status: "completed", + output, + finishReason, + usage: state.usage, + }; + } + + if (state.modelCalls.length >= definition.config.maxModelCalls) { + const error = `Model call limit of ${definition.config.maxModelCalls} reached before the turn completed.`; + await append({ + type: "turn_failed", + turnId, + ts: now(), + error, + code: MODEL_CALL_LIMIT_ERROR_CODE, + usage: state.usage, + }); + return { + status: "failed", + error, + code: MODEL_CALL_LIMIT_ERROR_CODE, + usage: state.usage, + }; + } + + // One model step. The durable request barrier precedes the + // provider call; step events persist before the next provider + // stream read; deltas bypass storage. + const index = state.modelCalls.length; + const transcript = turnTranscript(state); + const isRef = !Array.isArray(definition.context); + const request: z.infer = { + systemPrompt: definition.agent.resolved.systemPrompt, + ...(isRef ? { contextRef: definition.context as { previousTurnId: string } } : {}), + messages: isRef + ? transcript + : [...(definition.context as typeof transcript), ...transcript], + tools: definition.agent.resolved.tools, + parameters: {}, + }; + await append({ + type: "model_call_requested", + turnId, + ts: now(), + modelCallIndex: index, + request, + }); + + let completion: Extract< + LlmStreamEvent, + { type: "completed" } + > | null = null; + try { + for await (const event of model.stream({ + systemPrompt: request.systemPrompt, + messages: [...resolvedContext, ...transcript], + tools: request.tools, + parameters: request.parameters, + signal: controller.signal, + })) { + switch (event.type) { + case "text_delta": + stream.push({ + type: "text_delta", + turnId, + modelCallIndex: index, + delta: event.delta, + }); + break; + case "reasoning_delta": + stream.push({ + type: "reasoning_delta", + turnId, + modelCallIndex: index, + delta: event.delta, + }); + break; + case "step_event": + await append({ + type: "model_step_event", + turnId, + ts: now(), + modelCallIndex: index, + event: event.event, + }); + break; + case "completed": + completion = event; + break; + } + } + if (!completion) { + throw new Error( + "model stream ended without a completed response", + ); + } + } catch (error) { + if (controller.signal.aborted) { + await append( + modelCallFailedEvent(turnId, now(), index, "model call was cancelled"), + ); + return await cancelTurn(); + } + const message = errorMessage(error); + await append(modelCallFailedEvent(turnId, now(), index, message)); + await append({ + type: "turn_failed", + turnId, + ts: now(), + error: message, + usage: state.usage, + }); + return { status: "failed", error: message, usage: state.usage }; + } + + await append({ + type: "model_call_completed", + turnId, + ts: now(), + modelCallIndex: index, + message: completion.message, + finishReason: completion.finishReason, + usage: completion.usage, + ...(completion.providerMetadata === undefined + ? {} + : { providerMetadata: completion.providerMetadata }), + }); + } + } finally { + if (externalSignal) { + externalSignal.removeEventListener("abort", forwardAbort); + } + } + } +} + +function errorMessage(error: unknown): string { + return error instanceof Error ? error.message : String(error); +} + +function outcomeFromTerminal(state: TurnState): TurnOutcome { + const terminal = state.terminal; + if (!terminal) { + throw new Error("turn is not terminal"); + } + switch (terminal.type) { + case "turn_completed": + return { + status: "completed", + output: terminal.output, + finishReason: terminal.finishReason, + usage: terminal.usage, + }; + case "turn_failed": + return { + status: "failed", + error: terminal.error, + ...(terminal.code === undefined ? {} : { code: terminal.code }), + usage: terminal.usage, + }; + case "turn_cancelled": + return { + status: "cancelled", + ...(terminal.reason === undefined ? {} : { reason: terminal.reason }), + usage: terminal.usage, + }; + } +} + +function requirePendingAsync(state: TurnState, toolCallId: string): ToolCallState { + const tc = state.toolCalls.find((t) => t.toolCallId === toolCallId); + if (!tc?.invocation || tc.execution !== "async" || tc.result) { + throw new TurnInputError( + `no pending async tool call ${toolCallId}`, + ); + } + return tc; +} + +function permissionsSnapshot( + calls: ToolCallState[], +): z.infer["pendingPermissions"] { + return calls.map((tc) => ({ + toolCallId: tc.toolCallId, + toolName: tc.toolName, + request: (tc.permission as NonNullable) + .required.request, + })); +} + +function asyncSnapshot( + calls: ToolCallState[], +): z.infer["pendingAsyncTools"] { + return calls.map((tc) => ({ + toolCallId: tc.toolCallId, + toolId: (tc.invocation as z.infer).toolId, + toolName: tc.toolName, + input: (tc.invocation as z.infer).input, + })); +} + +function modelCallFailedEvent( + turnId: string, + ts: string, + modelCallIndex: number, + error: string, +): z.infer { + return { type: "model_call_failed", turnId, ts, modelCallIndex, error }; +} + +function runtimeResultEvent( + turnId: string, + ts: string, + tc: ToolCallState, + result: { output: JsonValue; isError: boolean }, +): z.infer { + return { + type: "tool_result", + turnId, + ts, + toolCallId: tc.toolCallId, + toolName: tc.toolName, + source: "runtime", + result, + }; +} + +function permissionRequiredEvent( + turnId: string, + ts: string, + tc: ToolCallState, + request: JsonValue, + checkerError?: string, +): z.infer { + return { + type: "tool_permission_required", + turnId, + ts, + toolCallId: tc.toolCallId, + toolName: tc.toolName, + request, + ...(checkerError === undefined ? {} : { checkerError }), + }; +} + +function resolvedEvent( + turnId: string, + ts: string, + toolCallId: string, + decision: "allow" | "deny", + source: z.infer["source"], + reason?: string, +): z.infer { + return { + type: "tool_permission_resolved", + turnId, + ts, + toolCallId, + decision, + source, + ...(reason === undefined ? {} : { reason }), + }; +} + +function invocationEvent( + turnId: string, + ts: string, + tc: ToolCallState, + descriptor: z.infer, +): z.infer { + return { + type: "tool_invocation_requested", + turnId, + ts, + toolCallId: tc.toolCallId, + toolId: descriptor.toolId, + toolName: tc.toolName, + execution: descriptor.execution, + input: (tc.input ?? null) as JsonValue, + }; +} diff --git a/apps/x/packages/core/src/turns/stream.test.ts b/apps/x/packages/core/src/turns/stream.test.ts new file mode 100644 index 00000000..f2b35614 --- /dev/null +++ b/apps/x/packages/core/src/turns/stream.test.ts @@ -0,0 +1,89 @@ +import { describe, expect, it } from "vitest"; +import { HotStream } from "./stream.js"; + +async function collect(iterable: AsyncIterable): Promise { + const out: T[] = []; + for await (const item of iterable) { + out.push(item); + } + return out; +} + +describe("HotStream", () => { + it("buffers events pushed before a consumer attaches", async () => { + const stream = new HotStream(); + stream.push(1); + stream.push(2); + stream.end("done"); + expect(await collect(stream.events)).toEqual([1, 2]); + expect(await stream.outcome).toBe("done"); + }); + + it("outcome resolves without draining events", async () => { + const stream = new HotStream(); + stream.push(1); + stream.push(2); + stream.end("done"); + expect(await stream.outcome).toBe("done"); + }); + + it("delivers events in order across await boundaries", async () => { + const stream = new HotStream(); + const consumer = collect(stream.events); + stream.push(1); + await Promise.resolve(); + stream.push(2); + stream.push(3); + stream.end("done"); + expect(await consumer).toEqual([1, 2, 3]); + }); + + it("closing the consumer drops future events without affecting outcome", async () => { + const stream = new HotStream(); + stream.push(1); + stream.push(2); + for await (const event of stream.events) { + expect(event).toBe(1); + break; // closes the iterator + } + stream.push(3); // dropped + stream.end("done"); + expect(await collect(stream.events)).toEqual([]); + expect(await stream.outcome).toBe("done"); + }); + + it("failure drains queued events, then throws, and rejects outcome with the same error", async () => { + const stream = new HotStream(); + const boom = new Error("boom"); + stream.push(1); + stream.fail(boom); + const seen: number[] = []; + await expect( + (async () => { + for await (const event of stream.events) { + seen.push(event); + } + })(), + ).rejects.toBe(boom); + expect(seen).toEqual([1]); + await expect(stream.outcome).rejects.toBe(boom); + }); + + it("ignores pushes and settlement after completion", async () => { + const stream = new HotStream(); + stream.end("first"); + stream.push(9); + stream.end("second"); + stream.fail(new Error("late")); + expect(await stream.outcome).toBe("first"); + expect(await collect(stream.events)).toEqual([]); + }); + + it("a waiting consumer wakes on end", async () => { + const stream = new HotStream(); + const consumer = collect(stream.events); + await Promise.resolve(); + stream.end("done"); + expect(await consumer).toEqual([]); + }); +}); diff --git a/apps/x/packages/core/src/turns/stream.ts b/apps/x/packages/core/src/turns/stream.ts new file mode 100644 index 00000000..9092b454 --- /dev/null +++ b/apps/x/packages/core/src/turns/stream.ts @@ -0,0 +1,94 @@ +// Hot execution stream (turn-runtime-design.md §16). Execution starts +// independently of event consumption; events buffer in an unbounded in-memory +// queue until the single assumed consumer attaches. If the consumer closes, +// subsequent events are dropped; closing never cancels execution. On +// infrastructure failure, iteration drains already-queued events and then +// throws, and the outcome rejects with the same error. +export class HotStream { + private queue: TEvent[] = []; + private waiters: Array<() => void> = []; + private done = false; + private failure: { error: unknown } | null = null; + private consumerClosed = false; + + readonly outcome: Promise; + private resolveOutcome!: (outcome: TOutcome) => void; + private rejectOutcome!: (error: unknown) => void; + + constructor() { + this.outcome = new Promise((resolve, reject) => { + this.resolveOutcome = resolve; + this.rejectOutcome = reject; + }); + // The outcome may legitimately never be awaited (fire-and-forget + // callers); don't surface unhandled rejections for it. + this.outcome.catch(() => undefined); + } + + push(event: TEvent): void { + if (this.done || this.consumerClosed) { + return; + } + this.queue.push(event); + this.wake(); + } + + end(outcome: TOutcome): void { + if (this.done) { + return; + } + this.done = true; + this.resolveOutcome(outcome); + this.wake(); + } + + fail(error: unknown): void { + if (this.done) { + return; + } + this.done = true; + this.failure = { error }; + this.rejectOutcome(error); + this.wake(); + } + + private wake(): void { + const waiters = this.waiters; + this.waiters = []; + for (const waiter of waiters) { + waiter(); + } + } + + get events(): AsyncIterable { + return { + [Symbol.asyncIterator]: (): AsyncIterator => ({ + next: async (): Promise> => { + for (;;) { + if (this.consumerClosed) { + return { value: undefined, done: true }; + } + const event = this.queue.shift(); + if (event !== undefined) { + return { value: event, done: false }; + } + if (this.done) { + if (this.failure) { + throw this.failure.error; + } + return { value: undefined, done: true }; + } + await new Promise((resolve) => + this.waiters.push(resolve), + ); + } + }, + return: async (): Promise> => { + this.consumerClosed = true; + this.queue.length = 0; + return { value: undefined, done: true }; + }, + }), + }; + } +} diff --git a/apps/x/packages/core/src/turns/tool-registry.ts b/apps/x/packages/core/src/turns/tool-registry.ts new file mode 100644 index 00000000..82670e72 --- /dev/null +++ b/apps/x/packages/core/src/turns/tool-registry.ts @@ -0,0 +1,35 @@ +import type { z } from "zod"; +import type { + JsonValue, + ToolDescriptor, + ToolResultData, +} from "@x/shared/dist/turns.js"; + +export interface ToolExecutionContext { + signal: AbortSignal; + // The loop appends a durable tool_progress event before resolving. + reportProgress(progress: JsonValue): Promise; +} + +export interface SyncRuntimeTool { + descriptor: z.infer & { execution: "sync" }; + execute( + input: unknown, + context: ToolExecutionContext, + ): Promise>; +} + +// An async tool has no in-process executor. Its invocation is exposed +// externally and its progress/result arrives later through advanceTurn. +export interface AsyncRuntimeTool { + descriptor: z.infer & { execution: "async" }; +} + +export type RuntimeTool = SyncRuntimeTool | AsyncRuntimeTool; + +// Resolves persisted tool descriptors to live implementations during +// advanceTurn. A rejection or a descriptor mismatch is an infrastructure +// error: the execution is rejected and the turn is left unchanged. +export interface IToolRegistry { + resolve(descriptor: z.infer): Promise; +} diff --git a/apps/x/packages/shared/src/turns.test.ts b/apps/x/packages/shared/src/turns.test.ts index bcdae400..3c77faf5 100644 --- a/apps/x/packages/shared/src/turns.test.ts +++ b/apps/x/packages/shared/src/turns.test.ts @@ -506,6 +506,7 @@ describe("tool execution", () => { permResolved("tc1", "deny", "human"), result("tc1", "echo", "runtime", "Permission denied", true), ]); + expect(state.toolCalls[0].permission?.classificationFailed).toBe(true); expect(state.toolCalls[0].result?.result.isError).toBe(true); }); diff --git a/apps/x/packages/shared/src/turns.ts b/apps/x/packages/shared/src/turns.ts index 52e39c23..f8b87c5b 100644 --- a/apps/x/packages/shared/src/turns.ts +++ b/apps/x/packages/shared/src/turns.ts @@ -382,6 +382,9 @@ export interface ToolCallState { permission?: { required: z.infer; classification?: z.infer; + // Set when a tool_permission_classification_failed event named this + // call; such calls are treated as defer and never re-classified. + classificationFailed?: boolean; resolved?: z.infer; }; invocation?: z.infer; @@ -635,8 +638,8 @@ function applyToolPermissionClassificationFailed( if (toolCall.permission.resolved) { fail(`classification failure after permission resolution: ${toolCallId}`); } + toolCall.permission.classificationFailed = true; } - // Recorded for the audit trail; does not currently affect derived state. } function applyToolPermissionResolved( From 9e1d9b81ce4481af8f5b3a48c87373d5f0a9092b Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 09:46:12 +0530 Subject: [PATCH 06/24] =?UTF-8?q?refactor(x/core):=20decompose=20turn=20ad?= =?UTF-8?q?vance=20into=20=C2=A718=20phase=20methods?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit No behavior change — all 67 turn tests pass untouched. The ~620-line advance method becomes an internal TurnAdvance class holding the per-invocation context, with one named method per spec phase (applyInput, closeInterrupted*, evaluatePermissions, classifyBatch, denyUnresolvedWithoutHuman, executeAllowedTools, suspendIfPending, completeIfFinished, failIfExhausted, runModelStep, cancel). The main loop now reads as a direct transcription of the design doc's §18 algorithm; TurnRuntime itself is reduced to construction and createTurn/getTurn/advanceTurn plumbing. Co-Authored-By: Claude Fable 5 --- apps/x/packages/core/src/turns/runtime.ts | 1344 ++++++++++++--------- 1 file changed, 763 insertions(+), 581 deletions(-) diff --git a/apps/x/packages/core/src/turns/runtime.ts b/apps/x/packages/core/src/turns/runtime.ts index 4a532a6e..d5352832 100644 --- a/apps/x/packages/core/src/turns/runtime.ts +++ b/apps/x/packages/core/src/turns/runtime.ts @@ -2,16 +2,17 @@ import type { z } from "zod"; import { DEFAULT_MAX_MODEL_CALLS, MODEL_CALL_LIMIT_ERROR_CODE, + type ConversationMessage, type JsonValue, type ModelCallFailed, type ModelRequest, type ToolCallState, - type ToolInvocationRequested, - ToolResultData, type ToolDescriptor, + type ToolInvocationRequested, type ToolPermissionRequired, type ToolPermissionResolved, type ToolResult, + ToolResultData, TurnCreated, type TurnEvent, type TurnState, @@ -37,7 +38,11 @@ import { import type { ITurnLifecycleBus } from "./bus.js"; import type { IClock } from "./clock.js"; import type { IContextResolver } from "./context-resolver.js"; -import type { IModelRegistry, LlmStreamEvent } from "./model-registry.js"; +import type { + IModelRegistry, + LlmStreamEvent, + ResolvedModel, +} from "./model-registry.js"; import type { IPermissionChecker, IPermissionClassifier } from "./permission.js"; import type { ITurnRepo } from "./repo.js"; import { HotStream } from "./stream.js"; @@ -157,6 +162,10 @@ export class TurnRuntime implements ITurnRuntime { } } + // §18 steps 4–7: read, reduce, short-circuit terminal turns, materialize + // context, and validate live dependencies — all before any mutation. + // Failures here are infrastructure errors: the execution rejects and the + // turn is left unchanged. private async advance( turnId: string, input: TurnExternalInput | undefined, @@ -164,21 +173,16 @@ export class TurnRuntime implements ITurnRuntime { stream: HotStream, ): Promise { const events = await this.turnRepo.read(turnId); - let state = reduceTurn(events); + const state = reduceTurn(events); if (state.terminal) { if (input) { - throw new TurnInputError( - `turn ${turnId} is terminal; input rejected`, - ); + throw new TurnInputError(`turn ${turnId} is terminal; input rejected`); } return outcomeFromTerminal(state); } const definition = state.definition; - - // Materialize context and live dependencies. Failures here are - // infrastructure errors: the execution rejects, the turn is unchanged. const resolvedContext = await this.contextResolver.resolve( definition.context, ); @@ -211,568 +215,22 @@ export class TurnRuntime implements ITurnRuntime { } } - let appended = false; - const append = async (...batch: TEvent[]): Promise => { - await this.turnRepo.append(turnId, batch); - events.push(...batch); - state = reduceTurn(events); - appended = true; - for (const event of batch) { - stream.push(event); - } - }; - const now = () => this.clock.now(); - // Checker "allowed" outcomes are deliberately not durable: after a - // crash the checker is simply re-consulted. - const checkerAllowed = new Set(); - let cancelReason: string | undefined; - - const cancelTurn = async (): Promise => { - const open = state.modelCalls.find( - (c) => c.response === undefined && c.error === undefined, - ); - if (open) { - await append( - modelCallFailedEvent(turnId, now(), open.index, "model call was cancelled"), - ); - } - for (const tc of state.toolCalls.filter((t) => !t.result)) { - await append( - runtimeResultEvent(turnId, now(), tc, { - output: "Tool call was cancelled before completion.", - isError: true, - }), - ); - } - await append({ - type: "turn_cancelled", - turnId, - ts: now(), - ...(cancelReason === undefined ? {} : { reason: cancelReason }), - usage: state.usage, - }); - return { - status: "cancelled", - ...(cancelReason === undefined ? {} : { reason: cancelReason }), - usage: state.usage, - }; - }; - + const run = new TurnAdvance({ + turnId, + events, + state, + stream, + resolvedContext, + model, + toolsByName, + signal: controller.signal, + turnRepo: this.turnRepo, + clock: this.clock, + permissionChecker: this.permissionChecker, + permissionClassifier: this.permissionClassifier, + }); try { - // Apply the optional single external input against durable - // pending state. - if (input) { - switch (input.type) { - case "cancel": - cancelReason = input.reason; - return await cancelTurn(); - case "permission_decision": { - const tc = state.toolCalls.find( - (t) => t.toolCallId === input.toolCallId, - ); - if (!tc?.permission || tc.permission.resolved || tc.result) { - throw new TurnInputError( - `no pending permission for tool call ${input.toolCallId}`, - ); - } - await append({ - type: "tool_permission_resolved", - turnId, - ts: now(), - toolCallId: input.toolCallId, - decision: input.decision, - source: "human", - ...(input.metadata === undefined - ? {} - : { metadata: input.metadata }), - }); - if (input.decision === "deny") { - await append( - runtimeResultEvent(turnId, now(), tc, { - output: "Permission denied by user.", - isError: true, - }), - ); - } - break; - } - case "async_tool_progress": { - const tc = requirePendingAsync(state, input.toolCallId); - await append({ - type: "tool_progress", - turnId, - ts: now(), - toolCallId: tc.toolCallId, - source: "async", - progress: input.progress, - }); - break; - } - case "async_tool_result": { - const tc = requirePendingAsync(state, input.toolCallId); - await append({ - type: "tool_result", - turnId, - ts: now(), - toolCallId: tc.toolCallId, - toolName: tc.toolName, - source: "async", - result: input.result, - }); - break; - } - } - } - - for (;;) { - if (controller.signal.aborted) { - return await cancelTurn(); - } - - // Recovery: close a model call interrupted by a crash, then - // re-issue it as a new call (counts against the budget). - const open = state.modelCalls.find( - (c) => c.response === undefined && c.error === undefined, - ); - if (open) { - await append( - modelCallFailedEvent( - turnId, - now(), - open.index, - "model call was interrupted before a response was recorded", - ), - ); - continue; - } - - // Recovery: close sync invocations interrupted by a crash - // with an indeterminate result; the turn continues. - const interruptedSync = state.toolCalls.filter( - (tc) => tc.invocation && tc.execution === "sync" && !tc.result, - ); - if (interruptedSync.length > 0) { - for (const tc of interruptedSync) { - await append( - runtimeResultEvent(turnId, now(), tc, { - output: INTERRUPTED_TOOL_MESSAGE, - isError: true, - }), - ); - } - continue; - } - - // Permission requirements for freshly extracted tool calls. - const fresh = state.toolCalls.filter( - (tc) => - !tc.result && - !tc.invocation && - !tc.permission && - !checkerAllowed.has(tc.toolCallId), - ); - for (const tc of fresh) { - if (controller.signal.aborted) { - break; - } - const tool = toolsByName.get(tc.toolName); - if (!tool) { - await append( - runtimeResultEvent(turnId, now(), tc, { - output: `Unknown tool: ${tc.toolName}`, - isError: true, - }), - ); - continue; - } - if ( - tool.descriptor.requiresHuman && - !definition.config.humanAvailable - ) { - await append( - invocationEvent(turnId, now(), tc, tool.descriptor), - ); - await append( - runtimeResultEvent(turnId, now(), tc, { - output: "Human input is unavailable for this turn.", - isError: true, - }), - ); - continue; - } - try { - const check = await this.permissionChecker.check({ - turnId, - toolCallId: tc.toolCallId, - toolId: tool.descriptor.toolId, - toolName: tc.toolName, - input: tc.input, - }); - if (!check.required) { - checkerAllowed.add(tc.toolCallId); - } else { - await append( - permissionRequiredEvent( - turnId, - now(), - tc, - check.request, - ), - ); - } - } catch (error) { - // Checker failure fails closed: record it and route - // to a human (or denial below); never execute. - await append( - permissionRequiredEvent(turnId, now(), tc, {}, errorMessage(error)), - ); - } - } - if (controller.signal.aborted) { - return await cancelTurn(); - } - - // Automatic classification, one batch. Checker-error calls - // and previously failed classifications go straight to the - // human/deny fallback. - if (definition.config.autoPermission) { - const candidates = state.toolCalls.filter( - (tc) => - tc.permission && - !tc.permission.resolved && - !tc.permission.classification && - !tc.permission.classificationFailed && - tc.permission.required.checkerError === undefined && - !tc.result, - ); - if (candidates.length > 0) { - try { - const decisions = await this.permissionClassifier.classify( - candidates.map((tc) => ({ - toolCallId: tc.toolCallId, - toolName: tc.toolName, - input: tc.input, - request: tc.permission!.required.request, - })), - controller.signal, - ); - for (const tc of candidates) { - const decision = decisions.find( - (d) => d.toolCallId === tc.toolCallId, - ); - if (!decision) { - await append({ - type: "tool_permission_classification_failed", - turnId, - ts: now(), - toolCallIds: [tc.toolCallId], - error: "classifier returned no decision", - }); - continue; - } - await append({ - type: "tool_permission_classified", - turnId, - ts: now(), - toolCallId: tc.toolCallId, - decision: decision.decision, - reason: decision.reason, - }); - if (decision.decision === "allow") { - await append( - resolvedEvent(turnId, now(), tc.toolCallId, "allow", "classifier", decision.reason), - ); - } else if (decision.decision === "deny") { - await append( - resolvedEvent(turnId, now(), tc.toolCallId, "deny", "classifier", decision.reason), - ); - await append( - runtimeResultEvent(turnId, now(), tc, { - output: `Permission denied: ${decision.reason}`, - isError: true, - }), - ); - } - // "defer" falls through to human/deny fallback. - } - } catch (error) { - if (controller.signal.aborted) { - return await cancelTurn(); - } - await append({ - type: "tool_permission_classification_failed", - turnId, - ts: now(), - toolCallIds: candidates.map((c) => c.toolCallId), - error: errorMessage(error), - }); - } - } - } - - // No human available: deny whatever remains unresolved. - if (!definition.config.humanAvailable) { - const unresolved = state.toolCalls.filter( - (tc) => tc.permission && !tc.permission.resolved && !tc.result, - ); - for (const tc of unresolved) { - await append( - resolvedEvent(turnId, now(), tc.toolCallId, "deny", "human_unavailable"), - ); - await append( - runtimeResultEvent(turnId, now(), tc, { - output: "Permission denied: no human is available for this turn.", - isError: true, - }), - ); - } - } - - // Execute allowed sync tools sequentially; expose allowed - // async tools. Source order. - const executable = state.toolCalls.filter( - (tc) => - !tc.result && - !tc.invocation && - (checkerAllowed.has(tc.toolCallId) || - tc.permission?.resolved?.decision === "allow"), - ); - for (const tc of executable) { - if (controller.signal.aborted) { - break; - } - const tool = toolsByName.get(tc.toolName); - if (!tool) { - await append( - runtimeResultEvent(turnId, now(), tc, { - output: `Unknown tool: ${tc.toolName}`, - isError: true, - }), - ); - continue; - } - await append(invocationEvent(turnId, now(), tc, tool.descriptor)); - if (tool.descriptor.execution === "async") { - continue; - } - const syncTool = tool as SyncRuntimeTool; - try { - const result = await syncTool.execute(tc.input, { - signal: controller.signal, - reportProgress: async (progress) => { - await append({ - type: "tool_progress", - turnId, - ts: now(), - toolCallId: tc.toolCallId, - source: "sync", - progress, - }); - }, - }); - await append({ - type: "tool_result", - turnId, - ts: now(), - toolCallId: tc.toolCallId, - toolName: tc.toolName, - source: "sync", - result: ToolResultData.parse(result), - }); - } catch (error) { - if (controller.signal.aborted) { - await append( - runtimeResultEvent(turnId, now(), tc, { - output: "Tool execution was cancelled.", - isError: true, - }), - ); - break; - } - // A tool failure is conversational, not terminal. - await append({ - type: "tool_result", - turnId, - ts: now(), - toolCallId: tc.toolCallId, - toolName: tc.toolName, - source: "sync", - result: { output: errorMessage(error), isError: true }, - }); - } - } - if (controller.signal.aborted) { - return await cancelTurn(); - } - - // Suspend while external work remains outstanding. - const pendingPerms = outstandingPermissions(state); - const pendingAsync = outstandingAsyncTools(state); - if (pendingPerms.length + pendingAsync.length > 0) { - const last = events[events.length - 1]; - if (appended || last.type !== "turn_suspended") { - await append({ - type: "turn_suspended", - turnId, - ts: now(), - pendingPermissions: permissionsSnapshot(pendingPerms), - pendingAsyncTools: asyncSnapshot(pendingAsync), - usage: state.usage, - }); - } - return { - status: "suspended", - pendingPermissions: permissionsSnapshot(pendingPerms), - pendingAsyncTools: asyncSnapshot(pendingAsync), - usage: state.usage, - }; - } - - // Tool batch complete. Completion, budget, or the next call. - const lastCall = state.modelCalls[state.modelCalls.length - 1]; - if ( - lastCall?.response !== undefined && - state.toolCalls.every((tc) => tc.modelCallIndex !== lastCall.index) - ) { - const output = lastCall.response; - const finishReason = lastCall.finishReason ?? "unknown"; - await append({ - type: "turn_completed", - turnId, - ts: now(), - output, - finishReason, - usage: state.usage, - }); - return { - status: "completed", - output, - finishReason, - usage: state.usage, - }; - } - - if (state.modelCalls.length >= definition.config.maxModelCalls) { - const error = `Model call limit of ${definition.config.maxModelCalls} reached before the turn completed.`; - await append({ - type: "turn_failed", - turnId, - ts: now(), - error, - code: MODEL_CALL_LIMIT_ERROR_CODE, - usage: state.usage, - }); - return { - status: "failed", - error, - code: MODEL_CALL_LIMIT_ERROR_CODE, - usage: state.usage, - }; - } - - // One model step. The durable request barrier precedes the - // provider call; step events persist before the next provider - // stream read; deltas bypass storage. - const index = state.modelCalls.length; - const transcript = turnTranscript(state); - const isRef = !Array.isArray(definition.context); - const request: z.infer = { - systemPrompt: definition.agent.resolved.systemPrompt, - ...(isRef ? { contextRef: definition.context as { previousTurnId: string } } : {}), - messages: isRef - ? transcript - : [...(definition.context as typeof transcript), ...transcript], - tools: definition.agent.resolved.tools, - parameters: {}, - }; - await append({ - type: "model_call_requested", - turnId, - ts: now(), - modelCallIndex: index, - request, - }); - - let completion: Extract< - LlmStreamEvent, - { type: "completed" } - > | null = null; - try { - for await (const event of model.stream({ - systemPrompt: request.systemPrompt, - messages: [...resolvedContext, ...transcript], - tools: request.tools, - parameters: request.parameters, - signal: controller.signal, - })) { - switch (event.type) { - case "text_delta": - stream.push({ - type: "text_delta", - turnId, - modelCallIndex: index, - delta: event.delta, - }); - break; - case "reasoning_delta": - stream.push({ - type: "reasoning_delta", - turnId, - modelCallIndex: index, - delta: event.delta, - }); - break; - case "step_event": - await append({ - type: "model_step_event", - turnId, - ts: now(), - modelCallIndex: index, - event: event.event, - }); - break; - case "completed": - completion = event; - break; - } - } - if (!completion) { - throw new Error( - "model stream ended without a completed response", - ); - } - } catch (error) { - if (controller.signal.aborted) { - await append( - modelCallFailedEvent(turnId, now(), index, "model call was cancelled"), - ); - return await cancelTurn(); - } - const message = errorMessage(error); - await append(modelCallFailedEvent(turnId, now(), index, message)); - await append({ - type: "turn_failed", - turnId, - ts: now(), - error: message, - usage: state.usage, - }); - return { status: "failed", error: message, usage: state.usage }; - } - - await append({ - type: "model_call_completed", - turnId, - ts: now(), - modelCallIndex: index, - message: completion.message, - finishReason: completion.finishReason, - usage: completion.usage, - ...(completion.providerMetadata === undefined - ? {} - : { providerMetadata: completion.providerMetadata }), - }); - } + return await run.run(input); } finally { if (externalSignal) { externalSignal.removeEventListener("abort", forwardAbort); @@ -781,6 +239,740 @@ export class TurnRuntime implements ITurnRuntime { } } +// One advanceTurn invocation. Owns the per-invocation context and implements +// the §18 main loop as one method per phase. All state it acts on is derived +// from the durable log via the shared reducer after every append. +class TurnAdvance { + private readonly turnId: string; + private readonly events: TEvent[]; + private state: TurnState; + private readonly stream: HotStream; + private readonly resolvedContext: Array>; + private readonly model: ResolvedModel; + private readonly toolsByName: Map; + private readonly signal: AbortSignal; + private readonly turnRepo: ITurnRepo; + private readonly clock: IClock; + private readonly permissionChecker: IPermissionChecker; + private readonly permissionClassifier: IPermissionClassifier; + + // Checker "allowed" outcomes are deliberately not durable: after a crash + // the checker is simply re-consulted. + private readonly checkerAllowed = new Set(); + private appended = false; + private cancelReason: string | undefined; + + constructor(init: { + turnId: string; + events: TEvent[]; + state: TurnState; + stream: HotStream; + resolvedContext: Array>; + model: ResolvedModel; + toolsByName: Map; + signal: AbortSignal; + turnRepo: ITurnRepo; + clock: IClock; + permissionChecker: IPermissionChecker; + permissionClassifier: IPermissionClassifier; + }) { + this.turnId = init.turnId; + this.events = init.events; + this.state = init.state; + this.stream = init.stream; + this.resolvedContext = init.resolvedContext; + this.model = init.model; + this.toolsByName = init.toolsByName; + this.signal = init.signal; + this.turnRepo = init.turnRepo; + this.clock = init.clock; + this.permissionChecker = init.permissionChecker; + this.permissionClassifier = init.permissionClassifier; + } + + private get definition(): TurnState["definition"] { + return this.state.definition; + } + + private now(): string { + return this.clock.now(); + } + + // Durable barrier: persist, re-reduce (the reducer doubles as a runtime + // assertion that the appended history is legal), then stream. + private async append(...batch: TEvent[]): Promise { + await this.turnRepo.append(this.turnId, batch); + this.events.push(...batch); + this.state = reduceTurn(this.events); + this.appended = true; + for (const event of batch) { + this.stream.push(event); + } + } + + // §18 step 8: repeatedly advance deterministic work. Each phase either + // appends durable facts and lets the loop continue, or produces the + // invocation's outcome. + async run(input: TurnExternalInput | undefined): Promise { + if (input) { + const cancelled = await this.applyInput(input); + if (cancelled) { + return cancelled; + } + } + for (;;) { + if (this.signal.aborted) { + return this.cancel(); + } + if (await this.closeInterruptedModelCall()) { + continue; + } + if (await this.closeInterruptedSyncTools()) { + continue; + } + await this.evaluatePermissions(); + if (this.signal.aborted) { + return this.cancel(); + } + await this.classifyBatch(); + if (this.signal.aborted) { + return this.cancel(); + } + await this.denyUnresolvedWithoutHuman(); + await this.executeAllowedTools(); + if (this.signal.aborted) { + return this.cancel(); + } + const suspended = await this.suspendIfPending(); + if (suspended) { + return suspended; + } + const completed = await this.completeIfFinished(); + if (completed) { + return completed; + } + const exhausted = await this.failIfExhausted(); + if (exhausted) { + return exhausted; + } + const settled = await this.runModelStep(); + if (settled) { + return settled; + } + } + } + + // §11.2: exactly one input, validated against durable pending state. + // Returns an outcome only for cancel inputs. + private async applyInput( + input: TurnExternalInput, + ): Promise { + switch (input.type) { + case "cancel": + this.cancelReason = input.reason; + return this.cancel(); + case "permission_decision": { + const tc = this.state.toolCalls.find( + (t) => t.toolCallId === input.toolCallId, + ); + if (!tc?.permission || tc.permission.resolved || tc.result) { + throw new TurnInputError( + `no pending permission for tool call ${input.toolCallId}`, + ); + } + await this.append({ + type: "tool_permission_resolved", + turnId: this.turnId, + ts: this.now(), + toolCallId: input.toolCallId, + decision: input.decision, + source: "human", + ...(input.metadata === undefined + ? {} + : { metadata: input.metadata }), + }); + if (input.decision === "deny") { + await this.append( + runtimeResultEvent(this.turnId, this.now(), tc, { + output: "Permission denied by user.", + isError: true, + }), + ); + } + return undefined; + } + case "async_tool_progress": { + const tc = this.requirePendingAsync(input.toolCallId); + await this.append({ + type: "tool_progress", + turnId: this.turnId, + ts: this.now(), + toolCallId: tc.toolCallId, + source: "async", + progress: input.progress, + }); + return undefined; + } + case "async_tool_result": { + const tc = this.requirePendingAsync(input.toolCallId); + await this.append({ + type: "tool_result", + turnId: this.turnId, + ts: this.now(), + toolCallId: tc.toolCallId, + toolName: tc.toolName, + source: "async", + result: input.result, + }); + return undefined; + } + } + } + + private requirePendingAsync(toolCallId: string): ToolCallState { + const tc = this.state.toolCalls.find((t) => t.toolCallId === toolCallId); + if (!tc?.invocation || tc.execution !== "async" || tc.result) { + throw new TurnInputError(`no pending async tool call ${toolCallId}`); + } + return tc; + } + + // §23: a model call interrupted by a crash is closed as failed and later + // re-issued by the normal model step (counting against the budget). + private async closeInterruptedModelCall(): Promise { + const open = this.state.modelCalls.find( + (c) => c.response === undefined && c.error === undefined, + ); + if (!open) { + return false; + } + await this.append( + modelCallFailedEvent( + this.turnId, + this.now(), + open.index, + "model call was interrupted before a response was recorded", + ), + ); + return true; + } + + // §23: a sync invocation interrupted by a crash gets an indeterminate + // error result; the turn continues (tool problems are conversational). + private async closeInterruptedSyncTools(): Promise { + const interrupted = this.state.toolCalls.filter( + (tc) => tc.invocation && tc.execution === "sync" && !tc.result, + ); + if (interrupted.length === 0) { + return false; + } + for (const tc of interrupted) { + await this.append( + runtimeResultEvent(this.turnId, this.now(), tc, { + output: INTERRUPTED_TOOL_MESSAGE, + isError: true, + }), + ); + } + return true; + } + + // §9/§10: for freshly extracted tool calls — unknown tools and + // human-dependent tools settle immediately; everything else is checked. + // A checker throw fails closed (recorded, never auto-executed). + private async evaluatePermissions(): Promise { + const fresh = this.state.toolCalls.filter( + (tc) => + !tc.result && + !tc.invocation && + !tc.permission && + !this.checkerAllowed.has(tc.toolCallId), + ); + for (const tc of fresh) { + if (this.signal.aborted) { + return; + } + const tool = this.toolsByName.get(tc.toolName); + if (!tool) { + await this.append( + runtimeResultEvent(this.turnId, this.now(), tc, { + output: `Unknown tool: ${tc.toolName}`, + isError: true, + }), + ); + continue; + } + if ( + tool.descriptor.requiresHuman && + !this.definition.config.humanAvailable + ) { + await this.append( + invocationEvent(this.turnId, this.now(), tc, tool.descriptor), + ); + await this.append( + runtimeResultEvent(this.turnId, this.now(), tc, { + output: "Human input is unavailable for this turn.", + isError: true, + }), + ); + continue; + } + try { + const check = await this.permissionChecker.check({ + turnId: this.turnId, + toolCallId: tc.toolCallId, + toolId: tool.descriptor.toolId, + toolName: tc.toolName, + input: tc.input, + }); + if (!check.required) { + this.checkerAllowed.add(tc.toolCallId); + } else { + await this.append( + permissionRequiredEvent( + this.turnId, + this.now(), + tc, + check.request, + ), + ); + } + } catch (error) { + await this.append( + permissionRequiredEvent( + this.turnId, + this.now(), + tc, + {}, + errorMessage(error), + ), + ); + } + } + } + + // §9.3: one classifier batch per model response in auto mode. + // Checker-error calls and previously failed classifications skip the + // classifier and go straight to the human/deny fallback. + private async classifyBatch(): Promise { + if (!this.definition.config.autoPermission) { + return; + } + const candidates = this.state.toolCalls.filter( + (tc) => + tc.permission && + !tc.permission.resolved && + !tc.permission.classification && + !tc.permission.classificationFailed && + tc.permission.required.checkerError === undefined && + !tc.result, + ); + if (candidates.length === 0) { + return; + } + let decisions; + try { + decisions = await this.permissionClassifier.classify( + candidates.map((tc) => ({ + toolCallId: tc.toolCallId, + toolName: tc.toolName, + input: tc.input, + request: (tc.permission as NonNullable) + .required.request, + })), + this.signal, + ); + } catch (error) { + if (this.signal.aborted) { + return; + } + await this.append({ + type: "tool_permission_classification_failed", + turnId: this.turnId, + ts: this.now(), + toolCallIds: candidates.map((c) => c.toolCallId), + error: errorMessage(error), + }); + return; + } + for (const tc of candidates) { + const decision = decisions.find((d) => d.toolCallId === tc.toolCallId); + if (!decision) { + await this.append({ + type: "tool_permission_classification_failed", + turnId: this.turnId, + ts: this.now(), + toolCallIds: [tc.toolCallId], + error: "classifier returned no decision", + }); + continue; + } + await this.append({ + type: "tool_permission_classified", + turnId: this.turnId, + ts: this.now(), + toolCallId: tc.toolCallId, + decision: decision.decision, + reason: decision.reason, + }); + if (decision.decision === "allow") { + await this.append( + resolvedEvent( + this.turnId, + this.now(), + tc.toolCallId, + "allow", + "classifier", + decision.reason, + ), + ); + } else if (decision.decision === "deny") { + await this.append( + resolvedEvent( + this.turnId, + this.now(), + tc.toolCallId, + "deny", + "classifier", + decision.reason, + ), + ); + await this.append( + runtimeResultEvent(this.turnId, this.now(), tc, { + output: `Permission denied: ${decision.reason}`, + isError: true, + }), + ); + } + // "defer" falls through to the human/deny fallback. + } + } + + // §9.3 matrix, humanAvailable = false: deny whatever remains unresolved. + private async denyUnresolvedWithoutHuman(): Promise { + if (this.definition.config.humanAvailable) { + return; + } + const unresolved = this.state.toolCalls.filter( + (tc) => tc.permission && !tc.permission.resolved && !tc.result, + ); + for (const tc of unresolved) { + await this.append( + resolvedEvent( + this.turnId, + this.now(), + tc.toolCallId, + "deny", + "human_unavailable", + ), + ); + await this.append( + runtimeResultEvent(this.turnId, this.now(), tc, { + output: "Permission denied: no human is available for this turn.", + isError: true, + }), + ); + } + } + + // §10.5: execute allowed sync tools sequentially and expose allowed async + // tools, in source order. Tool failures are conversational, not terminal. + private async executeAllowedTools(): Promise { + const executable = this.state.toolCalls.filter( + (tc) => + !tc.result && + !tc.invocation && + (this.checkerAllowed.has(tc.toolCallId) || + tc.permission?.resolved?.decision === "allow"), + ); + for (const tc of executable) { + if (this.signal.aborted) { + return; + } + const tool = this.toolsByName.get(tc.toolName); + if (!tool) { + await this.append( + runtimeResultEvent(this.turnId, this.now(), tc, { + output: `Unknown tool: ${tc.toolName}`, + isError: true, + }), + ); + continue; + } + await this.append( + invocationEvent(this.turnId, this.now(), tc, tool.descriptor), + ); + if (tool.descriptor.execution === "async") { + continue; // exposed; the result arrives through advanceTurn + } + const syncTool = tool as SyncRuntimeTool; + try { + const result = await syncTool.execute(tc.input, { + signal: this.signal, + reportProgress: async (progress) => { + await this.append({ + type: "tool_progress", + turnId: this.turnId, + ts: this.now(), + toolCallId: tc.toolCallId, + source: "sync", + progress, + }); + }, + }); + await this.append({ + type: "tool_result", + turnId: this.turnId, + ts: this.now(), + toolCallId: tc.toolCallId, + toolName: tc.toolName, + source: "sync", + result: ToolResultData.parse(result), + }); + } catch (error) { + if (this.signal.aborted) { + await this.append( + runtimeResultEvent(this.turnId, this.now(), tc, { + output: "Tool execution was cancelled.", + isError: true, + }), + ); + return; + } + await this.append({ + type: "tool_result", + turnId: this.turnId, + ts: this.now(), + toolCallId: tc.toolCallId, + toolName: tc.toolName, + source: "sync", + result: { output: errorMessage(error), isError: true }, + }); + } + } + } + + // §11.1: settle suspended while external work remains. A no-input + // re-advance of an already-snapshotted suspension appends nothing. + private async suspendIfPending(): Promise { + const pendingPerms = outstandingPermissions(this.state); + const pendingAsync = outstandingAsyncTools(this.state); + if (pendingPerms.length + pendingAsync.length === 0) { + return undefined; + } + const last = this.events[this.events.length - 1]; + if (this.appended || last.type !== "turn_suspended") { + await this.append({ + type: "turn_suspended", + turnId: this.turnId, + ts: this.now(), + pendingPermissions: permissionsSnapshot(pendingPerms), + pendingAsyncTools: asyncSnapshot(pendingAsync), + usage: this.state.usage, + }); + } + return { + status: "suspended", + pendingPermissions: permissionsSnapshot(pendingPerms), + pendingAsyncTools: asyncSnapshot(pendingAsync), + usage: this.state.usage, + }; + } + + // §8.5: a completed response without tool calls completes the turn. + private async completeIfFinished(): Promise { + const lastCall = this.state.modelCalls[this.state.modelCalls.length - 1]; + if ( + lastCall?.response === undefined || + this.state.toolCalls.some((tc) => tc.modelCallIndex === lastCall.index) + ) { + return undefined; + } + const output = lastCall.response; + const finishReason = lastCall.finishReason ?? "unknown"; + await this.append({ + type: "turn_completed", + turnId: this.turnId, + ts: this.now(), + output, + finishReason, + usage: this.state.usage, + }); + return { status: "completed", output, finishReason, usage: this.state.usage }; + } + + // §20: limit exhaustion is a distinguishable outcome; the transcript is + // structurally complete, so sessions can offer continuation. + private async failIfExhausted(): Promise { + if (this.state.modelCalls.length < this.definition.config.maxModelCalls) { + return undefined; + } + const error = `Model call limit of ${this.definition.config.maxModelCalls} reached before the turn completed.`; + await this.append({ + type: "turn_failed", + turnId: this.turnId, + ts: this.now(), + error, + code: MODEL_CALL_LIMIT_ERROR_CODE, + usage: this.state.usage, + }); + return { + status: "failed", + error, + code: MODEL_CALL_LIMIT_ERROR_CODE, + usage: this.state.usage, + }; + } + + // §8.3/§18h–l: one model step. The durable request barrier precedes the + // provider call; step events persist before the next provider read; + // deltas bypass storage. Returns an outcome only on failure/cancel. + private async runModelStep(): Promise { + const index = this.state.modelCalls.length; + const transcript = turnTranscript(this.state); + const context = this.definition.context; + const isRef = !Array.isArray(context); + const request: z.infer = { + systemPrompt: this.definition.agent.resolved.systemPrompt, + ...(isRef ? { contextRef: context } : {}), + messages: isRef ? transcript : [...context, ...transcript], + tools: this.definition.agent.resolved.tools, + parameters: {}, + }; + await this.append({ + type: "model_call_requested", + turnId: this.turnId, + ts: this.now(), + modelCallIndex: index, + request, + }); + + let completion: Extract | null = + null; + try { + for await (const event of this.model.stream({ + systemPrompt: request.systemPrompt, + messages: [...this.resolvedContext, ...transcript], + tools: request.tools, + parameters: request.parameters, + signal: this.signal, + })) { + switch (event.type) { + case "text_delta": + this.stream.push({ + type: "text_delta", + turnId: this.turnId, + modelCallIndex: index, + delta: event.delta, + }); + break; + case "reasoning_delta": + this.stream.push({ + type: "reasoning_delta", + turnId: this.turnId, + modelCallIndex: index, + delta: event.delta, + }); + break; + case "step_event": + await this.append({ + type: "model_step_event", + turnId: this.turnId, + ts: this.now(), + modelCallIndex: index, + event: event.event, + }); + break; + case "completed": + completion = event; + break; + } + } + if (!completion) { + throw new Error("model stream ended without a completed response"); + } + } catch (error) { + if (this.signal.aborted) { + await this.append( + modelCallFailedEvent( + this.turnId, + this.now(), + index, + "model call was cancelled", + ), + ); + return this.cancel(); + } + const message = errorMessage(error); + await this.append( + modelCallFailedEvent(this.turnId, this.now(), index, message), + ); + await this.append({ + type: "turn_failed", + turnId: this.turnId, + ts: this.now(), + error: message, + usage: this.state.usage, + }); + return { status: "failed", error: message, usage: this.state.usage }; + } + + await this.append({ + type: "model_call_completed", + turnId: this.turnId, + ts: this.now(), + modelCallIndex: index, + message: completion.message, + finishReason: completion.finishReason, + usage: completion.usage, + ...(completion.providerMetadata === undefined + ? {} + : { providerMetadata: completion.providerMetadata }), + }); + return undefined; + } + + // §22: close any open model call, give synthetic results to unresolved + // calls, and append the terminal cancellation. + private async cancel(): Promise { + const open = this.state.modelCalls.find( + (c) => c.response === undefined && c.error === undefined, + ); + if (open) { + await this.append( + modelCallFailedEvent( + this.turnId, + this.now(), + open.index, + "model call was cancelled", + ), + ); + } + for (const tc of this.state.toolCalls.filter((t) => !t.result)) { + await this.append( + runtimeResultEvent(this.turnId, this.now(), tc, { + output: "Tool call was cancelled before completion.", + isError: true, + }), + ); + } + await this.append({ + type: "turn_cancelled", + turnId: this.turnId, + ts: this.now(), + ...(this.cancelReason === undefined + ? {} + : { reason: this.cancelReason }), + usage: this.state.usage, + }); + return { + status: "cancelled", + ...(this.cancelReason === undefined ? {} : { reason: this.cancelReason }), + usage: this.state.usage, + }; + } +} + function errorMessage(error: unknown): string { return error instanceof Error ? error.message : String(error); } @@ -814,16 +1006,6 @@ function outcomeFromTerminal(state: TurnState): TurnOutcome { } } -function requirePendingAsync(state: TurnState, toolCallId: string): ToolCallState { - const tc = state.toolCalls.find((t) => t.toolCallId === toolCallId); - if (!tc?.invocation || tc.execution !== "async" || tc.result) { - throw new TurnInputError( - `no pending async tool call ${toolCallId}`, - ); - } - return tc; -} - function permissionsSnapshot( calls: ToolCallState[], ): z.infer["pendingPermissions"] { From ccf38c6a0fd5e53e41bb74af228274fc89cf8443 Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 09:57:20 +0530 Subject: [PATCH 07/24] feat(x/core): session layer with write-through index (stage 3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SessionsImpl per session-design.md, constructor-injected and tested entirely against a mocked ITurnRuntime: - sendMessage: per-session lock, typed TurnNotSettledError while the latest turn is non-terminal (terminal statuses including failed/ cancelled allow continuation), context as { previousTurnId } ref, turn-file-first write ordering (a failed session append leaves a benign orphan turn and no advance), denormalized agent/model on turn_appended, default title from the first message. - Dedicated respondToAskHuman endpoint (async_tool_result wrapper); respondToPermission / deliverAsyncToolResult pass turn-runtime rejections through. stopTurn aborts a live advance or cancels an at-rest turn; resumeTurn re-enters idle turns (never at startup). - In-memory index: startup scan (session files + each latest turn for status; corrupt files yield errored entries without aborting), write-through updates, deletion guard against late-settle resurrection. Bus events (turn-event / index-changed) defined in @x/shared as the renderer IPC contract. - FSSessionRepo: date-partitioned append-only JSONL + list/delete; deleteSession removes the session file only. - runHeadlessTurn: standalone turns (sessionId null, auto permission, no human) for background/knowledge/scheduled callers. 29 new tests covering the session-design §13 matrix. Co-Authored-By: Claude Fable 5 --- apps/x/packages/core/src/sessions/api.ts | 79 ++ apps/x/packages/core/src/sessions/bus.ts | 7 + .../core/src/sessions/fs-repo.test.ts | 108 +++ apps/x/packages/core/src/sessions/fs-repo.ts | 149 ++++ apps/x/packages/core/src/sessions/headless.ts | 41 + .../src/sessions/in-memory-session-repo.ts | 83 ++ apps/x/packages/core/src/sessions/index.ts | 8 + apps/x/packages/core/src/sessions/repo.ts | 20 + .../core/src/sessions/session-index.ts | 25 + .../core/src/sessions/sessions.test.ts | 801 ++++++++++++++++++ apps/x/packages/core/src/sessions/sessions.ts | 418 +++++++++ apps/x/packages/shared/src/sessions.ts | 22 +- 12 files changed, 1760 insertions(+), 1 deletion(-) create mode 100644 apps/x/packages/core/src/sessions/api.ts create mode 100644 apps/x/packages/core/src/sessions/bus.ts create mode 100644 apps/x/packages/core/src/sessions/fs-repo.test.ts create mode 100644 apps/x/packages/core/src/sessions/fs-repo.ts create mode 100644 apps/x/packages/core/src/sessions/headless.ts create mode 100644 apps/x/packages/core/src/sessions/in-memory-session-repo.ts create mode 100644 apps/x/packages/core/src/sessions/index.ts create mode 100644 apps/x/packages/core/src/sessions/repo.ts create mode 100644 apps/x/packages/core/src/sessions/session-index.ts create mode 100644 apps/x/packages/core/src/sessions/sessions.test.ts create mode 100644 apps/x/packages/core/src/sessions/sessions.ts diff --git a/apps/x/packages/core/src/sessions/api.ts b/apps/x/packages/core/src/sessions/api.ts new file mode 100644 index 00000000..7f787b49 --- /dev/null +++ b/apps/x/packages/core/src/sessions/api.ts @@ -0,0 +1,79 @@ +import type { z } from "zod"; +import type { UserMessage } from "@x/shared/dist/message.js"; +import type { + SessionIndexEntry, + SessionState, +} from "@x/shared/dist/sessions.js"; +import type { + JsonValue, + RequestedAgent, + ToolResultData, +} from "@x/shared/dist/turns.js"; +import type { Turn } from "../turns/api.js"; + +// Per-message configuration; it lands on the turn (sessions store none). +export interface SendMessageConfig { + agent: z.infer; + autoPermission?: boolean; + maxModelCalls?: number; +} + +export interface ISessions { + // Startup scan: builds the in-memory index from session files (reading + // each session's latest turn for status). Must run before listSessions. + initialize(): Promise; + + createSession(input?: { title?: string }): Promise; + listSessions(): SessionIndexEntry[]; + getSession(sessionId: string): Promise; + getTurn(turnId: string): Promise; + + // Rejects with TurnNotSettledError while the latest turn is non-terminal. + // Returns as soon as the turn is created, referenced, and advancing; + // progress flows through the bus. + sendMessage( + sessionId: string, + input: z.infer, + config: SendMessageConfig, + ): Promise<{ turnId: string }>; + + // External inputs, one advanceTurn each. These settle with that + // invocation's outcome; turn-runtime input rejections pass through. + respondToPermission( + turnId: string, + toolCallId: string, + decision: "allow" | "deny", + metadata?: JsonValue, + ): Promise; + // The dedicated ask-human endpoint; sendMessage never routes here. + respondToAskHuman( + turnId: string, + toolCallId: string, + answer: string, + ): Promise; + deliverAsyncToolResult( + turnId: string, + toolCallId: string, + result: z.infer, + ): Promise; + + stopTurn(turnId: string, reason?: string): Promise; + // Recovery entry for turns left idle by a crash; runs in the background. + resumeTurn(sessionId: string): Promise; + + setTitle(sessionId: string, title: string): Promise; + deleteSession(sessionId: string): Promise; +} + +export class TurnNotSettledError extends Error { + constructor( + readonly sessionId: string, + readonly turnId: string, + readonly turnStatus: string, + ) { + super( + `session ${sessionId} has a non-terminal turn ${turnId} (${turnStatus})`, + ); + this.name = "TurnNotSettledError"; + } +} diff --git a/apps/x/packages/core/src/sessions/bus.ts b/apps/x/packages/core/src/sessions/bus.ts new file mode 100644 index 00000000..e82f6f47 --- /dev/null +++ b/apps/x/packages/core/src/sessions/bus.ts @@ -0,0 +1,7 @@ +import type { SessionBusEvent } from "@x/shared/dist/sessions.js"; + +// Ephemeral fan-out toward the renderer (bridged over IPC in the app layer). +// Publishing is fire-and-forget; nothing durable depends on delivery. +export interface ISessionBus { + publish(event: SessionBusEvent): void; +} diff --git a/apps/x/packages/core/src/sessions/fs-repo.test.ts b/apps/x/packages/core/src/sessions/fs-repo.test.ts new file mode 100644 index 00000000..dc85993d --- /dev/null +++ b/apps/x/packages/core/src/sessions/fs-repo.test.ts @@ -0,0 +1,108 @@ +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import type { z } from "zod"; +import { + SessionCorruptionError, + type SessionCreated, + type SessionEvent, +} from "@x/shared/dist/sessions.js"; +import { FSSessionRepo } from "./fs-repo.js"; + +const S1 = "2026-07-02T09-00-00Z-0000001-000"; +const S2 = "2026-07-03T09-00-00Z-0000002-000"; + +function created(sessionId = S1): z.infer { + return { + type: "session_created", + schemaVersion: 1, + sessionId, + ts: "2026-07-02T09:00:00Z", + title: "T", + }; +} + +function appended(sessionId = S1): z.infer { + return { + type: "turn_appended", + sessionId, + ts: "2026-07-02T09:01:00Z", + turnId: "turn-1", + sessionSeq: 1, + agentId: "copilot", + model: { provider: "fake", model: "m" }, + }; +} + +describe("FSSessionRepo", () => { + let root: string; + let repo: FSSessionRepo; + + beforeEach(async () => { + root = await fs.mkdtemp(path.join(os.tmpdir(), "session-repo-")); + repo = new FSSessionRepo({ sessionsRootDir: root }); + }); + + afterEach(async () => { + await fs.rm(root, { recursive: true, force: true }); + }); + + it("creates date-partitioned files and round-trips events", async () => { + await repo.create(created()); + await repo.append(S1, [appended()]); + const file = path.join(root, "2026", "07", "02", `${S1}.jsonl`); + await fs.access(file); + const events = await repo.read(S1); + expect(events.map((e) => e.type)).toEqual([ + "session_created", + "turn_appended", + ]); + }); + + it("rejects malformed and path-like session ids", async () => { + for (const bad of ["../evil", "a/b", "nope", ""]) { + await expect(repo.read(bad)).rejects.toThrowError(/invalid session id/); + } + }); + + it("create fails if the session exists; append fails if it doesn't", async () => { + await repo.create(created()); + await expect(repo.create(created())).rejects.toThrowError(); + await expect(repo.append(S2, [appended(S2)])).rejects.toThrowError( + /session not found/, + ); + }); + + it("read rejects corrupt files whole", async () => { + const file = path.join(root, "2026", "07", "02", `${S1}.jsonl`); + await fs.mkdir(path.dirname(file), { recursive: true }); + await fs.writeFile(file, `${JSON.stringify(created())}\nnot json\n`); + await expect(repo.read(S1)).rejects.toThrowError(SessionCorruptionError); + await fs.writeFile(file, `${JSON.stringify(created())}\n{"torn`); + await expect(repo.read(S1)).rejects.toThrowError( + /does not end with a complete line/, + ); + }); + + it("lists session ids across date partitions", async () => { + await repo.create(created(S1)); + await repo.create(created(S2)); + expect(await repo.listSessionIds()).toEqual([S1, S2]); + }); + + it("listing an empty root returns no ids", async () => { + const empty = new FSSessionRepo({ + sessionsRootDir: path.join(root, "missing"), + }); + expect(await empty.listSessionIds()).toEqual([]); + }); + + it("delete removes the file only; deleting a missing session throws", async () => { + await repo.create(created(S1)); + await repo.create(created(S2)); + await repo.delete(S1); + expect(await repo.listSessionIds()).toEqual([S2]); + await expect(repo.delete(S1)).rejects.toThrowError(/session not found/); + }); +}); diff --git a/apps/x/packages/core/src/sessions/fs-repo.ts b/apps/x/packages/core/src/sessions/fs-repo.ts new file mode 100644 index 00000000..63eccecd --- /dev/null +++ b/apps/x/packages/core/src/sessions/fs-repo.ts @@ -0,0 +1,149 @@ +import fs from "node:fs/promises"; +import path from "node:path"; +import { z } from "zod"; +import { + SessionCorruptionError, + SessionCreated, + SessionEvent, +} from "@x/shared/dist/sessions.js"; +import { KeyedMutex } from "../turns/keyed-mutex.js"; +import type { ISessionRepo } from "./repo.js"; + +// Session IDs come from the same generator as turn IDs. +const SESSION_ID_PATTERN = /^(\d{4})-(\d{2})-(\d{2})T[A-Za-z0-9-]+$/; + +export class FSSessionRepo implements ISessionRepo { + private readonly rootDir: string; + private readonly mutex = new KeyedMutex(); + + constructor({ sessionsRootDir }: { sessionsRootDir: string }) { + this.rootDir = sessionsRootDir; + } + + private filePath(sessionId: string): string { + const match = SESSION_ID_PATTERN.exec(sessionId); + if (!match) { + throw new Error(`invalid session id: ${sessionId}`); + } + const [, year, month, day] = match; + return path.join(this.rootDir, year, month, day, `${sessionId}.jsonl`); + } + + private serialize( + sessionId: string, + events: Array>, + ): string { + let payload = ""; + for (const event of events) { + const parsed = SessionEvent.parse(event); + if (parsed.sessionId !== sessionId) { + throw new Error( + `event sessionId ${parsed.sessionId} does not match ${sessionId}`, + ); + } + payload += `${JSON.stringify(parsed)}\n`; + } + return payload; + } + + async create(event: z.infer): Promise { + const parsed = SessionCreated.parse(event); + const file = this.filePath(parsed.sessionId); + await fs.mkdir(path.dirname(file), { recursive: true }); + await fs.writeFile(file, this.serialize(parsed.sessionId, [parsed]), { + flag: "wx", + }); + } + + async read(sessionId: string): Promise>> { + const file = this.filePath(sessionId); + let raw: string; + try { + raw = await fs.readFile(file, "utf8"); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") { + throw new Error(`session not found: ${sessionId}`); + } + throw error; + } + if (raw.length === 0) { + throw new SessionCorruptionError(`session file is empty: ${sessionId}`); + } + const lines = raw.split("\n"); + const trailing = lines.pop(); + if (trailing !== "") { + throw new SessionCorruptionError( + `session file does not end with a complete line: ${sessionId}`, + ); + } + const events: Array> = []; + for (const [index, line] of lines.entries()) { + let parsed: z.infer; + try { + parsed = SessionEvent.parse(JSON.parse(line)); + } catch (error) { + throw new SessionCorruptionError( + `malformed session event at ${sessionId}:${index + 1}: ${String( + error instanceof Error ? error.message : error, + )}`, + ); + } + if (parsed.sessionId !== sessionId) { + throw new SessionCorruptionError( + `event sessionId ${parsed.sessionId} does not match file ${sessionId}`, + ); + } + events.push(parsed); + } + return events; + } + + async append( + sessionId: string, + events: Array>, + ): Promise { + if (events.length === 0) { + return; + } + const payload = this.serialize(sessionId, events); + const file = this.filePath(sessionId); + try { + await fs.access(file); + } catch { + throw new Error(`session not found: ${sessionId}`); + } + await fs.appendFile(file, payload); + } + + async withLock(sessionId: string, fn: () => Promise): Promise { + return this.mutex.run(sessionId, fn); + } + + async listSessionIds(): Promise { + let names: string[]; + try { + names = (await fs.readdir(this.rootDir, { recursive: true })) as string[]; + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") { + return []; + } + throw error; + } + return names + .filter((name) => name.endsWith(".jsonl")) + .map((name) => path.basename(name, ".jsonl")) + .sort(); + } + + async delete(sessionId: string): Promise { + const file = this.filePath(sessionId); + try { + await fs.unlink(file); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") { + throw new Error(`session not found: ${sessionId}`); + } + throw error; + } + } +} diff --git a/apps/x/packages/core/src/sessions/headless.ts b/apps/x/packages/core/src/sessions/headless.ts new file mode 100644 index 00000000..071d7fd4 --- /dev/null +++ b/apps/x/packages/core/src/sessions/headless.ts @@ -0,0 +1,41 @@ +import type { z } from "zod"; +import type { UserMessage } from "@x/shared/dist/message.js"; +import type { + ConversationMessage, + RequestedAgent, +} from "@x/shared/dist/turns.js"; +import type { ITurnRuntime, TurnOutcome } from "../turns/api.js"; + +// Standalone turns for non-session callers (background tasks, live notes, +// knowledge pipelines, scheduled agents): sessionId null, automatic +// permissions, no human. Never appears in the session index; callers keep +// the turnId if they need history. +export async function runHeadlessTurn( + turnRuntime: ITurnRuntime, + input: { + agent: z.infer; + context?: Array>; + input: z.infer; + maxModelCalls?: number; + signal?: AbortSignal; + }, +): Promise<{ turnId: string; outcome: TurnOutcome }> { + const turnId = await turnRuntime.createTurn({ + agent: input.agent, + sessionId: null, + context: input.context ?? [], + input: input.input, + config: { + autoPermission: true, + humanAvailable: false, + ...(input.maxModelCalls === undefined + ? {} + : { maxModelCalls: input.maxModelCalls }), + }, + }); + const execution = turnRuntime.advanceTurn(turnId, undefined, { + signal: input.signal, + }); + const outcome = await execution.outcome; + return { turnId, outcome }; +} diff --git a/apps/x/packages/core/src/sessions/in-memory-session-repo.ts b/apps/x/packages/core/src/sessions/in-memory-session-repo.ts new file mode 100644 index 00000000..35c54765 --- /dev/null +++ b/apps/x/packages/core/src/sessions/in-memory-session-repo.ts @@ -0,0 +1,83 @@ +import { z } from "zod"; +import { + SessionCorruptionError, + SessionCreated, + SessionEvent, +} from "@x/shared/dist/sessions.js"; +import { KeyedMutex } from "../turns/keyed-mutex.js"; +import type { ISessionRepo } from "./repo.js"; + +// Test fake mirroring FSSessionRepo semantics without touching disk. +export class InMemorySessionRepo implements ISessionRepo { + private files = new Map>>(); + private corrupt = new Set(); + private mutex = new KeyedMutex(); + + async create(event: z.infer): Promise { + const parsed = SessionCreated.parse(event); + if (this.files.has(parsed.sessionId) || this.corrupt.has(parsed.sessionId)) { + throw new Error(`session already exists: ${parsed.sessionId}`); + } + this.files.set(parsed.sessionId, [structuredClone(parsed)]); + } + + async read(sessionId: string): Promise>> { + if (this.corrupt.has(sessionId)) { + throw new SessionCorruptionError(`session file is corrupt: ${sessionId}`); + } + const events = this.files.get(sessionId); + if (!events) { + throw new Error(`session not found: ${sessionId}`); + } + return structuredClone(events); + } + + async append( + sessionId: string, + events: Array>, + ): Promise { + const file = this.files.get(sessionId); + if (!file) { + throw new Error(`session not found: ${sessionId}`); + } + for (const event of events) { + const parsed = SessionEvent.parse(event); + if (parsed.sessionId !== sessionId) { + throw new Error( + `event sessionId ${parsed.sessionId} does not match ${sessionId}`, + ); + } + file.push(structuredClone(parsed)); + } + } + + async withLock(sessionId: string, fn: () => Promise): Promise { + return this.mutex.run(sessionId, fn); + } + + async listSessionIds(): Promise { + return [...this.files.keys(), ...this.corrupt].sort(); + } + + async delete(sessionId: string): Promise { + const existed = this.files.delete(sessionId) || this.corrupt.delete(sessionId); + if (!existed) { + throw new Error(`session not found: ${sessionId}`); + } + } + + // Test helpers + seed(events: Array>): void { + if (events.length === 0) { + throw new Error("cannot seed an empty log"); + } + this.files.set( + events[0].sessionId, + events.map((e) => structuredClone(SessionEvent.parse(e))), + ); + } + + seedCorrupt(sessionId: string): void { + this.corrupt.add(sessionId); + } +} diff --git a/apps/x/packages/core/src/sessions/index.ts b/apps/x/packages/core/src/sessions/index.ts new file mode 100644 index 00000000..df906d01 --- /dev/null +++ b/apps/x/packages/core/src/sessions/index.ts @@ -0,0 +1,8 @@ +export * from "./api.js"; +export * from "./bus.js"; +export * from "./fs-repo.js"; +export * from "./headless.js"; +export * from "./in-memory-session-repo.js"; +export * from "./repo.js"; +export * from "./session-index.js"; +export * from "./sessions.js"; diff --git a/apps/x/packages/core/src/sessions/repo.ts b/apps/x/packages/core/src/sessions/repo.ts new file mode 100644 index 00000000..eb10f4e1 --- /dev/null +++ b/apps/x/packages/core/src/sessions/repo.ts @@ -0,0 +1,20 @@ +import type { z } from "zod"; +import type { SessionCreated, SessionEvent } from "@x/shared/dist/sessions.js"; + +export interface ISessionRepo { + // Fails if the session already exists. + create(event: z.infer): Promise; + // Validates every line strictly; corrupt files are rejected whole. + read(sessionId: string): Promise>>; + // Validates events before writing. + append( + sessionId: string, + events: Array>, + ): Promise; + // In-process per-session exclusion. + withLock(sessionId: string, fn: () => Promise): Promise; + // Enumerates all session ids for the startup scan. + listSessionIds(): Promise; + // Removes the session file only; referenced turn files stay as orphans. + delete(sessionId: string): Promise; +} diff --git a/apps/x/packages/core/src/sessions/session-index.ts b/apps/x/packages/core/src/sessions/session-index.ts new file mode 100644 index 00000000..f09dcbf6 --- /dev/null +++ b/apps/x/packages/core/src/sessions/session-index.ts @@ -0,0 +1,25 @@ +import type { SessionIndexEntry } from "@x/shared/dist/sessions.js"; + +// In-memory projection over session files. Never a source of truth: rebuilt +// by the startup scan, maintained write-through by the sessions service. +export class SessionIndex { + private entries = new Map(); + + list(): SessionIndexEntry[] { + return [...this.entries.values()].sort((a, b) => + b.updatedAt.localeCompare(a.updatedAt), + ); + } + + get(sessionId: string): SessionIndexEntry | undefined { + return this.entries.get(sessionId); + } + + upsert(entry: SessionIndexEntry): void { + this.entries.set(entry.sessionId, entry); + } + + remove(sessionId: string): boolean { + return this.entries.delete(sessionId); + } +} diff --git a/apps/x/packages/core/src/sessions/sessions.test.ts b/apps/x/packages/core/src/sessions/sessions.test.ts new file mode 100644 index 00000000..f0c53fcf --- /dev/null +++ b/apps/x/packages/core/src/sessions/sessions.test.ts @@ -0,0 +1,801 @@ +import { describe, expect, it } from "vitest"; +import type { z } from "zod"; +import type { SessionBusEvent, SessionEvent } from "@x/shared/dist/sessions.js"; +import type { + ResolvedAgent, + TurnEvent, + TurnStreamEvent, +} from "@x/shared/dist/turns.js"; +import type { + CreateTurnInput, + ITurnRuntime, + Turn, + TurnExecution, + TurnExternalInput, + TurnOutcome, +} from "../turns/api.js"; +import { TurnInputError } from "../turns/api.js"; +import { HotStream } from "../turns/stream.js"; +import { TurnNotSettledError } from "./api.js"; +import { runHeadlessTurn } from "./headless.js"; +import { InMemorySessionRepo } from "./in-memory-session-repo.js"; +import type { ISessionRepo } from "./repo.js"; +import { SessionsImpl } from "./sessions.js"; + +type TEvent = z.infer; + +const TS = "2026-07-02T10:00:00Z"; +const FIXTURE_MODEL = { provider: "openai", model: "gpt-fixture" }; +const FIXTURE_AGENT: z.infer = { + agentId: "copilot", + systemPrompt: "SYS", + model: FIXTURE_MODEL, + tools: [], +}; + +function user(text: string) { + return { role: "user" as const, content: text }; +} + +function assistantText(text: string) { + return { role: "assistant" as const, content: text }; +} + +function completedOutcome(): TurnOutcome { + return { + status: "completed", + output: assistantText("ok"), + finishReason: "stop", + usage: {}, + }; +} + +function createdEvent(turnId: string, input: CreateTurnInput): TEvent { + return { + type: "turn_created", + schemaVersion: 1, + turnId, + ts: TS, + sessionId: input.sessionId ?? null, + agent: { requested: input.agent, resolved: FIXTURE_AGENT }, + context: input.context, + input: input.input, + config: { + autoPermission: input.config.autoPermission ?? false, + humanAvailable: input.config.humanAvailable, + maxModelCalls: input.config.maxModelCalls ?? 20, + }, + }; +} + +// Minimal valid event logs reducing to each turn status. +function turnLog( + turnId: string, + sessionId: string, + status: "idle" | "completed" | "failed" | "cancelled" | "suspended", +): TEvent[] { + const created: TEvent = createdEvent(turnId, { + agent: { agentId: "copilot" }, + sessionId, + context: [], + input: user("hi"), + config: { humanAvailable: true }, + }); + const requested: TEvent = { + type: "model_call_requested", + turnId, + ts: TS, + modelCallIndex: 0, + request: { systemPrompt: "SYS", messages: [user("hi")], tools: [], parameters: {} }, + }; + switch (status) { + case "idle": + return [created]; + case "completed": + return [ + created, + requested, + { + type: "model_call_completed", + turnId, + ts: TS, + modelCallIndex: 0, + message: assistantText("ok"), + finishReason: "stop", + usage: {}, + }, + { + type: "turn_completed", + turnId, + ts: TS, + output: assistantText("ok"), + finishReason: "stop", + usage: {}, + }, + ]; + case "failed": + return [ + created, + requested, + { type: "model_call_failed", turnId, ts: TS, modelCallIndex: 0, error: "boom" }, + { type: "turn_failed", turnId, ts: TS, error: "boom", usage: {} }, + ]; + case "cancelled": + return [ + created, + { type: "turn_cancelled", turnId, ts: TS, usage: {} }, + ]; + case "suspended": + return [ + created, + requested, + { + type: "model_call_completed", + turnId, + ts: TS, + modelCallIndex: 0, + message: { + role: "assistant", + content: [ + { + type: "tool-call", + toolCallId: "B", + toolName: "fetch", + arguments: {}, + }, + ], + }, + finishReason: "tool-calls", + usage: {}, + }, + { + type: "tool_invocation_requested", + turnId, + ts: TS, + toolCallId: "B", + toolId: "tool.fetch", + toolName: "fetch", + execution: "async", + input: {}, + }, + { + type: "turn_suspended", + turnId, + ts: TS, + pendingPermissions: [], + pendingAsyncTools: [ + { toolCallId: "B", toolId: "tool.fetch", toolName: "fetch", input: {} }, + ], + usage: {}, + }, + ]; + } +} + +type AdvanceScript = (call: { + turnId: string; + input?: TurnExternalInput; + signal?: AbortSignal; +}) => + | { events?: TurnStreamEvent[]; outcome: TurnOutcome } + | { error: unknown } + | { untilAbort: true }; + +class FakeTurnRuntime implements ITurnRuntime { + createTurnInputs: CreateTurnInput[] = []; + advanceCalls: Array<{ turnId: string; input?: TurnExternalInput }> = []; + logs = new Map(); + createError?: Error; + script?: AdvanceScript; + private n = 0; + + async createTurn(input: CreateTurnInput): Promise { + if (this.createError) { + throw this.createError; + } + this.createTurnInputs.push(input); + this.n += 1; + const turnId = `2026-07-02T10-00-00Z-${String(this.n).padStart(7, "0")}-000`; + this.logs.set(turnId, [createdEvent(turnId, input)]); + return turnId; + } + + advanceTurn( + turnId: string, + input?: TurnExternalInput, + options?: { signal?: AbortSignal }, + ): TurnExecution { + this.advanceCalls.push({ turnId, input }); + const stream = new HotStream(); + const result = this.script?.({ turnId, input, signal: options?.signal }) ?? { + outcome: completedOutcome(), + }; + if ("error" in result) { + stream.fail(result.error); + } else if ("untilAbort" in result) { + const signal = options?.signal; + if (!signal) { + throw new Error("untilAbort script requires a signal"); + } + const finish = () => stream.end({ status: "cancelled", usage: {} }); + if (signal.aborted) { + finish(); + } else { + signal.addEventListener("abort", finish, { once: true }); + } + } else { + for (const event of result.events ?? []) { + stream.push(event); + } + stream.end(result.outcome); + } + return { events: stream.events, outcome: stream.outcome }; + } + + async getTurn(turnId: string): Promise { + const log = this.logs.get(turnId); + if (!log) { + throw new Error(`turn not found: ${turnId}`); + } + return { turnId, events: structuredClone(log) }; + } + + setLog(turnId: string, events: TEvent[]): void { + this.logs.set(turnId, events); + } +} + +class RecordingBus { + events: SessionBusEvent[] = []; + publish(event: SessionBusEvent): void { + this.events.push(event); + } +} + +class FakeIdGen { + private n = 0; + async next(): Promise { + this.n += 1; + return `2026-07-02T09-00-00Z-${String(this.n).padStart(7, "0")}-000`; + } +} + +class FakeClock { + now(): string { + return TS; + } +} + +// Wrapper to simulate a crash between turn-file creation and session append. +class FlakySessionRepo implements ISessionRepo { + failNextAppend = false; + constructor(private readonly inner: InMemorySessionRepo) {} + create(event: Parameters[0]) { + return this.inner.create(event); + } + read(sessionId: string) { + return this.inner.read(sessionId); + } + async append( + sessionId: string, + events: Array>, + ): Promise { + if (this.failNextAppend) { + this.failNextAppend = false; + throw new Error("disk full"); + } + return this.inner.append(sessionId, events); + } + withLock(sessionId: string, fn: () => Promise): Promise { + return this.inner.withLock(sessionId, fn); + } + listSessionIds() { + return this.inner.listSessionIds(); + } + delete(sessionId: string) { + return this.inner.delete(sessionId); + } +} + +function makeSessions(opts: { repo?: ISessionRepo; fake?: FakeTurnRuntime } = {}) { + const repo = opts.repo ?? new InMemorySessionRepo(); + const fake = opts.fake ?? new FakeTurnRuntime(); + const bus = new RecordingBus(); + const sessions = new SessionsImpl({ + sessionRepo: repo, + turnRuntime: fake, + idGenerator: new FakeIdGen(), + clock: new FakeClock(), + bus, + }); + return { sessions, repo, fake, bus }; +} + +const flush = () => new Promise((resolve) => setTimeout(resolve, 0)); + +describe("createSession and listing", () => { + it("persists session_created and publishes an index entry with status none", async () => { + const { sessions, repo, bus } = makeSessions(); + const sessionId = await sessions.createSession({ title: "My chat" }); + const events = await (repo as InMemorySessionRepo).read(sessionId); + expect(events).toEqual([ + expect.objectContaining({ + type: "session_created", + schemaVersion: 1, + sessionId, + title: "My chat", + }), + ]); + expect(sessions.listSessions()).toEqual([ + expect.objectContaining({ + sessionId, + title: "My chat", + turnCount: 0, + latestTurnStatus: "none", + }), + ]); + expect(bus.events).toEqual([ + expect.objectContaining({ kind: "index-changed", sessionId }), + ]); + }); +}); + +describe("sendMessage (13.3)", () => { + it("first message: inline empty context, config on the turn, denormalized ref, default title", async () => { + const { sessions, repo, fake } = makeSessions(); + const sessionId = await sessions.createSession(); + const { turnId } = await sessions.sendMessage(sessionId, user("Fix the bug in parser"), { + agent: { agentId: "copilot", overrides: { model: { provider: "x", model: "y" } } }, + autoPermission: true, + maxModelCalls: 5, + }); + + expect(fake.createTurnInputs[0]).toEqual({ + agent: { agentId: "copilot", overrides: { model: { provider: "x", model: "y" } } }, + sessionId, + context: [], + input: user("Fix the bug in parser"), + config: { humanAvailable: true, autoPermission: true, maxModelCalls: 5 }, + }); + + const events = await (repo as InMemorySessionRepo).read(sessionId); + expect(events[1]).toEqual( + expect.objectContaining({ + type: "turn_appended", + turnId, + sessionSeq: 1, + agentId: "copilot", + model: FIXTURE_MODEL, // resolved, not requested override input + }), + ); + expect(events[2]).toEqual( + expect.objectContaining({ + type: "title_changed", + title: "Fix the bug in parser", + }), + ); + expect(fake.advanceCalls).toEqual([{ turnId, input: undefined }]); + }); + + it("subsequent messages reference the latest turn and skip the title", async () => { + const { sessions, repo, fake } = makeSessions(); + const sessionId = await sessions.createSession(); + const first = await sessions.sendMessage(sessionId, user("one"), { + agent: { agentId: "copilot" }, + }); + fake.setLog(first.turnId, turnLog(first.turnId, sessionId, "completed")); + + const second = await sessions.sendMessage(sessionId, user("two"), { + agent: { agentId: "copilot" }, + }); + expect(fake.createTurnInputs[1].context).toEqual({ + previousTurnId: first.turnId, + }); + const events = await (repo as InMemorySessionRepo).read(sessionId); + const appends = events.filter((e) => e.type === "turn_appended"); + expect(appends.map((e) => (e.type === "turn_appended" ? e.sessionSeq : 0))).toEqual([1, 2]); + expect(appends[1]).toEqual( + expect.objectContaining({ turnId: second.turnId }), + ); + expect(events.filter((e) => e.type === "title_changed")).toHaveLength(1); + }); + + it("rejects while the latest turn is idle or suspended; allows all terminal statuses", async () => { + const { sessions, fake } = makeSessions(); + const sessionId = await sessions.createSession(); + const { turnId } = await sessions.sendMessage(sessionId, user("one"), { + agent: { agentId: "copilot" }, + }); + + for (const status of ["idle", "suspended"] as const) { + fake.setLog(turnId, turnLog(turnId, sessionId, status)); + const attempt = sessions.sendMessage(sessionId, user("nope"), { + agent: { agentId: "copilot" }, + }); + await expect(attempt).rejects.toThrowError(TurnNotSettledError); + await expect(attempt).rejects.toMatchObject({ + sessionId, + turnId, + turnStatus: status, + }); + } + + let latest = turnId; + for (const status of ["completed", "failed", "cancelled"] as const) { + fake.setLog(latest, turnLog(latest, sessionId, status)); + const result = await sessions.sendMessage(sessionId, user(`after ${status}`), { + agent: { agentId: "copilot" }, + }); + latest = result.turnId; + } + }); + + it("serializes concurrent sends: exactly one wins", async () => { + const { sessions, repo } = makeSessions(); + const sessionId = await sessions.createSession(); + const results = await Promise.allSettled([ + sessions.sendMessage(sessionId, user("a"), { agent: { agentId: "copilot" } }), + sessions.sendMessage(sessionId, user("b"), { agent: { agentId: "copilot" } }), + ]); + expect(results.filter((r) => r.status === "fulfilled")).toHaveLength(1); + const rejected = results.find((r) => r.status === "rejected"); + expect((rejected as PromiseRejectedResult).reason).toBeInstanceOf( + TurnNotSettledError, + ); + const events = await (repo as InMemorySessionRepo).read(sessionId); + expect(events.filter((e) => e.type === "turn_appended")).toHaveLength(1); + }); +}); + +describe("write ordering and crash simulation (13.4)", () => { + it("a failed session append leaves a benign orphan turn and no advance; retry works", async () => { + const inner = new InMemorySessionRepo(); + const flaky = new FlakySessionRepo(inner); + const { sessions, fake } = makeSessions({ repo: flaky }); + const sessionId = await sessions.createSession(); + + flaky.failNextAppend = true; + await expect( + sessions.sendMessage(sessionId, user("one"), { agent: { agentId: "copilot" } }), + ).rejects.toThrowError("disk full"); + + // Turn file exists (orphan), session unchanged, nothing advanced. + expect(fake.logs.size).toBe(1); + expect(fake.advanceCalls).toHaveLength(0); + expect(await inner.read(sessionId)).toHaveLength(1); + + // Retry produces a fresh turn at sessionSeq 1. + const { turnId } = await sessions.sendMessage(sessionId, user("one again"), { + agent: { agentId: "copilot" }, + }); + const events = await inner.read(sessionId); + expect(events.filter((e) => e.type === "turn_appended")).toEqual([ + expect.objectContaining({ turnId, sessionSeq: 1 }), + ]); + }); + + it("a failed createTurn leaves the session untouched", async () => { + const { sessions, repo, fake } = makeSessions(); + const sessionId = await sessions.createSession(); + fake.createError = new Error("agent resolution failed"); + await expect( + sessions.sendMessage(sessionId, user("one"), { agent: { agentId: "ghost" } }), + ).rejects.toThrowError("agent resolution failed"); + expect(await (repo as InMemorySessionRepo).read(sessionId)).toHaveLength(1); + expect(fake.advanceCalls).toHaveLength(0); + }); +}); + +describe("external input routing (13.5)", () => { + async function setupSuspended() { + const fixture = makeSessions(); + const sessionId = await fixture.sessions.createSession(); + const { turnId } = await fixture.sessions.sendMessage(sessionId, user("go"), { + agent: { agentId: "copilot" }, + }); + fixture.fake.setLog(turnId, turnLog(turnId, sessionId, "suspended")); + fixture.fake.advanceCalls.length = 0; + return { ...fixture, sessionId, turnId }; + } + + it("respondToPermission advances with a permission_decision input", async () => { + const { sessions, fake, turnId } = await setupSuspended(); + await sessions.respondToPermission(turnId, "tc1", "allow", { scope: "once" }); + expect(fake.advanceCalls).toEqual([ + { + turnId, + input: { + type: "permission_decision", + toolCallId: "tc1", + decision: "allow", + metadata: { scope: "once" }, + }, + }, + ]); + }); + + it("respondToAskHuman is a dedicated async_tool_result wrapper", async () => { + const { sessions, fake, turnId } = await setupSuspended(); + await sessions.respondToAskHuman(turnId, "B", "the answer is 42"); + expect(fake.advanceCalls).toEqual([ + { + turnId, + input: { + type: "async_tool_result", + toolCallId: "B", + result: { output: "the answer is 42", isError: false }, + }, + }, + ]); + }); + + it("deliverAsyncToolResult passes the result through verbatim", async () => { + const { sessions, fake, turnId } = await setupSuspended(); + await sessions.deliverAsyncToolResult(turnId, "B", { + output: { rows: 3 }, + isError: false, + }); + expect(fake.advanceCalls[0].input).toEqual({ + type: "async_tool_result", + toolCallId: "B", + result: { output: { rows: 3 }, isError: false }, + }); + }); + + it("turn-runtime input rejections pass through", async () => { + const { sessions, fake, turnId } = await setupSuspended(); + fake.script = () => ({ error: new TurnInputError("no pending async tool call X") }); + await expect( + sessions.deliverAsyncToolResult(turnId, "X", { output: "x", isError: false }), + ).rejects.toThrowError(TurnInputError); + }); +}); + +describe("stopTurn and resumeTurn", () => { + it("aborts an actively running advance instead of issuing a cancel input", async () => { + const { sessions, fake } = makeSessions(); + fake.script = () => ({ untilAbort: true }); + const sessionId = await sessions.createSession(); + const { turnId } = await sessions.sendMessage(sessionId, user("go"), { + agent: { agentId: "copilot" }, + }); + expect(fake.advanceCalls).toHaveLength(1); + await sessions.stopTurn(turnId); + // No second advance: the running invocation's signal was aborted. + expect(fake.advanceCalls).toHaveLength(1); + }); + + it("cancels an at-rest turn through a cancel input", async () => { + const { sessions, fake } = makeSessions(); + const sessionId = await sessions.createSession(); + const { turnId } = await sessions.sendMessage(sessionId, user("go"), { + agent: { agentId: "copilot" }, + }); + await flush(); + fake.setLog(turnId, turnLog(turnId, sessionId, "suspended")); + fake.advanceCalls.length = 0; + await sessions.stopTurn(turnId, "user stop"); + expect(fake.advanceCalls).toEqual([ + { turnId, input: { type: "cancel", reason: "user stop" } }, + ]); + }); + + it("resumeTurn re-enters the latest turn with no input", async () => { + const { sessions, fake } = makeSessions(); + const sessionId = await sessions.createSession(); + const { turnId } = await sessions.sendMessage(sessionId, user("go"), { + agent: { agentId: "copilot" }, + }); + await flush(); + fake.advanceCalls.length = 0; + await sessions.resumeTurn(sessionId); + expect(fake.advanceCalls).toEqual([{ turnId, input: undefined }]); + }); + + it("resumeTurn on a session with no turns throws", async () => { + const { sessions } = makeSessions(); + const sessionId = await sessions.createSession(); + await expect(sessions.resumeTurn(sessionId)).rejects.toThrowError( + /no turns to resume/, + ); + }); +}); + +describe("event forwarding and index maintenance (13.6, 13.7)", () => { + it("forwards stream events to the bus tagged with sessionId in order", async () => { + const { sessions, fake, bus } = makeSessions(); + const streamed: TurnStreamEvent[] = [ + { type: "text_delta", turnId: "x", modelCallIndex: 0, delta: "he" }, + { type: "text_delta", turnId: "x", modelCallIndex: 0, delta: "y" }, + ]; + fake.script = () => ({ events: streamed, outcome: completedOutcome() }); + const sessionId = await sessions.createSession(); + const { turnId } = await sessions.sendMessage(sessionId, user("go"), { + agent: { agentId: "copilot" }, + }); + await flush(); + const turnEvents = bus.events.filter((e) => e.kind === "turn-event"); + expect(turnEvents).toEqual( + streamed.map((event) => ({ + kind: "turn-event", + sessionId, + turnId, + event, + })), + ); + }); + + it("outcome settlement updates the index entry's latest turn status", async () => { + const { sessions, fake, bus } = makeSessions(); + fake.script = () => ({ + outcome: { + status: "suspended", + pendingPermissions: [], + pendingAsyncTools: [ + { toolCallId: "B", toolId: "tool.fetch", toolName: "fetch", input: {} }, + ], + usage: {}, + }, + }); + const sessionId = await sessions.createSession(); + await sessions.sendMessage(sessionId, user("go"), { + agent: { agentId: "copilot" }, + }); + await flush(); + expect(sessions.listSessions()[0].latestTurnStatus).toBe("suspended"); + const last = bus.events[bus.events.length - 1]; + expect(last).toMatchObject({ + kind: "index-changed", + entry: { latestTurnStatus: "suspended" }, + }); + }); + + it("setTitle appends and updates the index preserving status", async () => { + const { sessions, repo } = makeSessions(); + const sessionId = await sessions.createSession(); + await sessions.sendMessage(sessionId, user("go"), { + agent: { agentId: "copilot" }, + }); + await flush(); + await sessions.setTitle(sessionId, "Renamed"); + const events = await (repo as InMemorySessionRepo).read(sessionId); + expect(events[events.length - 1]).toMatchObject({ + type: "title_changed", + title: "Renamed", + }); + expect(sessions.listSessions()[0]).toMatchObject({ + title: "Renamed", + latestTurnStatus: "completed", + }); + }); + + it("deleteSession removes the file and entry; turn files stay; late settles don't resurrect", async () => { + const { sessions, repo, fake, bus } = makeSessions(); + fake.script = () => ({ untilAbort: true }); + const sessionId = await sessions.createSession(); + const { turnId } = await sessions.sendMessage(sessionId, user("go"), { + agent: { agentId: "copilot" }, + }); + + await sessions.deleteSession(sessionId); + await expect( + (repo as InMemorySessionRepo).read(sessionId), + ).rejects.toThrowError(/session not found/); + expect(sessions.listSessions()).toEqual([]); + expect(bus.events[bus.events.length - 1]).toEqual({ + kind: "index-changed", + sessionId, + entry: null, + }); + // Turn file untouched (orphaned, inert). + expect(fake.logs.has(turnId)).toBe(true); + + // The still-running advance settles after deletion: no resurrection. + await sessions.stopTurn(turnId); + await flush(); + expect(sessions.listSessions()).toEqual([]); + }); +}); + +describe("startup scan (13.6)", () => { + it("builds the index from session files and each latest turn; corrupt files yield errored entries", async () => { + const repo = new InMemorySessionRepo(); + const fake = new FakeTurnRuntime(); + const s1 = "2026-07-02T09-00-00Z-0000001-000"; + const s2 = "2026-07-02T09-00-00Z-0000002-000"; + const s3 = "2026-07-02T09-00-00Z-0000003-000"; + repo.seed([ + { type: "session_created", schemaVersion: 1, sessionId: s1, ts: TS, title: "Done one" }, + { + type: "turn_appended", + sessionId: s1, + ts: TS, + turnId: "t1", + sessionSeq: 1, + agentId: "copilot", + model: FIXTURE_MODEL, + }, + ]); + fake.setLog("t1", turnLog("t1", s1, "completed")); + repo.seed([ + { type: "session_created", schemaVersion: 1, sessionId: s2, ts: TS }, + { + type: "turn_appended", + sessionId: s2, + ts: TS, + turnId: "t2", + sessionSeq: 1, + agentId: "researcher", + model: { provider: "anthropic", model: "claude-x" }, + }, + ]); + fake.setLog("t2", turnLog("t2", s2, "suspended")); + repo.seedCorrupt(s3); + + const { sessions } = makeSessions({ repo, fake }); + await sessions.initialize(); + + const entries = sessions.listSessions(); + expect(entries).toHaveLength(3); + const byId = new Map(entries.map((e) => [e.sessionId, e])); + expect(byId.get(s1)).toMatchObject({ + title: "Done one", + turnCount: 1, + lastAgentId: "copilot", + lastModel: FIXTURE_MODEL, + latestTurnStatus: "completed", + }); + expect(byId.get(s2)).toMatchObject({ + lastAgentId: "researcher", + latestTurnStatus: "suspended", + }); + expect(byId.get(s3)).toMatchObject({ + latestTurnStatus: "none", + error: expect.stringMatching(/corrupt/), + }); + }); + + it("a rebuilt index matches write-through state for the same history", async () => { + const repo = new InMemorySessionRepo(); + const fake = new FakeTurnRuntime(); + const live = makeSessions({ repo, fake }); + const sessionId = await live.sessions.createSession(); + const { turnId } = await live.sessions.sendMessage(sessionId, user("hello world"), { + agent: { agentId: "copilot" }, + }); + await flush(); + fake.setLog(turnId, turnLog(turnId, sessionId, "completed")); + + const rebuilt = makeSessions({ repo, fake }); + await rebuilt.sessions.initialize(); + expect(rebuilt.sessions.listSessions()).toEqual([ + expect.objectContaining({ + sessionId, + title: "hello world", + turnCount: 1, + latestTurnId: turnId, + latestTurnStatus: "completed", + }), + ]); + }); +}); + +describe("headless standalone turns (13.8)", () => { + it("runs a standalone turn with sessionId null, auto permission, no human", async () => { + const fake = new FakeTurnRuntime(); + const { turnId, outcome } = await runHeadlessTurn(fake, { + agent: { agentId: "background-task-agent" }, + input: user("summarize"), + maxModelCalls: 3, + }); + expect(fake.createTurnInputs[0]).toEqual({ + agent: { agentId: "background-task-agent" }, + sessionId: null, + context: [], + input: user("summarize"), + config: { autoPermission: true, humanAvailable: false, maxModelCalls: 3 }, + }); + expect(outcome.status).toBe("completed"); + expect(fake.advanceCalls).toEqual([{ turnId, input: undefined }]); + }); +}); diff --git a/apps/x/packages/core/src/sessions/sessions.ts b/apps/x/packages/core/src/sessions/sessions.ts new file mode 100644 index 00000000..c258c2eb --- /dev/null +++ b/apps/x/packages/core/src/sessions/sessions.ts @@ -0,0 +1,418 @@ +import type { z } from "zod"; +import type { UserMessage } from "@x/shared/dist/message.js"; +import { + SessionCreated, + type SessionEvent, + type SessionIndexEntry, + type SessionLatestTurnStatus, + type SessionState, + reduceSession, + sessionIndexEntry, +} from "@x/shared/dist/sessions.js"; +import { + type JsonValue, + type ModelDescriptor, + type ToolResultData, + deriveTurnStatus, + reduceTurn, +} from "@x/shared/dist/turns.js"; +import type { IMonotonicallyIncreasingIdGenerator } from "../application/lib/id-gen.js"; +import type { + ITurnRuntime, + Turn, + TurnExecution, + TurnExternalInput, + TurnOutcome, +} from "../turns/api.js"; +import type { IClock } from "../turns/clock.js"; +import { + type ISessions, + type SendMessageConfig, + TurnNotSettledError, +} from "./api.js"; +import type { ISessionBus } from "./bus.js"; +import type { ISessionRepo } from "./repo.js"; +import { SessionIndex } from "./session-index.js"; + +export interface SessionsDependencies { + sessionRepo: ISessionRepo; + turnRuntime: ITurnRuntime; + idGenerator: IMonotonicallyIncreasingIdGenerator; + clock: IClock; + bus: ISessionBus; +} + +// The session layer per session-design.md: owns conversations as ordered +// chains of turn references, enforces one active turn per session, assembles +// context as a reference to the previous turn, and maintains the in-memory +// index write-through. It never reads or writes turn-file contents beyond +// what ITurnRuntime exposes. +export class SessionsImpl implements ISessions { + private readonly sessionRepo: ISessionRepo; + private readonly turnRuntime: ITurnRuntime; + private readonly idGenerator: IMonotonicallyIncreasingIdGenerator; + private readonly clock: IClock; + private readonly bus: ISessionBus; + + private readonly index = new SessionIndex(); + // Ephemeral: executions this process started, for stopTurn's abort path. + private readonly active = new Map< + string, + { sessionId: string | null; controller: AbortController; execution: TurnExecution } + >(); + + constructor({ + sessionRepo, + turnRuntime, + idGenerator, + clock, + bus, + }: SessionsDependencies) { + this.sessionRepo = sessionRepo; + this.turnRuntime = turnRuntime; + this.idGenerator = idGenerator; + this.clock = clock; + this.bus = bus; + } + + // §8.2: scan session files, read each session's latest turn for status. + // Corrupt files yield errored entries; the scan never aborts. + async initialize(): Promise { + for (const sessionId of await this.sessionRepo.listSessionIds()) { + this.index.upsert(await this.scanSession(sessionId)); + } + } + + private async scanSession(sessionId: string): Promise { + try { + const state = reduceSession(await this.sessionRepo.read(sessionId)); + const status = await this.latestTurnStatus(state); + return sessionIndexEntry(state, status); + } catch (error) { + return { + sessionId, + createdAt: "", + updatedAt: "", + turnCount: 0, + latestTurnStatus: "none", + error: errorMessage(error), + }; + } + } + + private async latestTurnStatus( + state: SessionState, + ): Promise { + if (!state.latestTurnId) { + return "none"; + } + const turn = await this.turnRuntime.getTurn(state.latestTurnId); + return deriveTurnStatus(reduceTurn(turn.events)); + } + + async createSession(input?: { title?: string }): Promise { + const sessionId = await this.idGenerator.next(); + const event = SessionCreated.parse({ + type: "session_created", + schemaVersion: 1, + sessionId, + ts: this.clock.now(), + ...(input?.title === undefined ? {} : { title: input.title }), + }); + await this.sessionRepo.create(event); + this.publishEntry(sessionIndexEntry(reduceSession([event]), "none")); + return sessionId; + } + + listSessions(): SessionIndexEntry[] { + return this.index.list(); + } + + async getSession(sessionId: string): Promise { + return reduceSession(await this.sessionRepo.read(sessionId)); + } + + async getTurn(turnId: string): Promise { + return this.turnRuntime.getTurn(turnId); + } + + // §9.1. Write order per §7: turn file first, then turn_appended, then the + // first advance — so an orphan turn (crash between the writes) is benign + // and an executing turn is always referenced. + async sendMessage( + sessionId: string, + input: z.infer, + config: SendMessageConfig, + ): Promise<{ turnId: string }> { + return this.sessionRepo.withLock(sessionId, async () => { + const events = await this.sessionRepo.read(sessionId); + const state = reduceSession(events); + + if (state.latestTurnId) { + const status = await this.latestTurnStatus(state); + if ( + status !== "completed" && + status !== "failed" && + status !== "cancelled" + ) { + throw new TurnNotSettledError( + sessionId, + state.latestTurnId, + status, + ); + } + } + + const turnId = await this.turnRuntime.createTurn({ + agent: config.agent, + sessionId, + context: state.latestTurnId + ? { previousTurnId: state.latestTurnId } + : [], + input, + config: { + humanAvailable: true, + ...(config.autoPermission === undefined + ? {} + : { autoPermission: config.autoPermission }), + ...(config.maxModelCalls === undefined + ? {} + : { maxModelCalls: config.maxModelCalls }), + }, + }); + + const batch: Array> = [ + { + type: "turn_appended", + sessionId, + ts: this.clock.now(), + turnId, + sessionSeq: state.turns.length + 1, + agentId: config.agent.agentId, + model: await this.resolvedModelOf(turnId), + }, + ]; + if (!state.title) { + batch.push({ + type: "title_changed", + sessionId, + ts: this.clock.now(), + title: defaultTitle(input), + }); + } + await this.sessionRepo.append(sessionId, batch); + + this.publishEntry( + sessionIndexEntry(reduceSession([...events, ...batch]), "idle"), + ); + this.startTrackedAdvance(sessionId, turnId); + return { turnId }; + }); + } + + async respondToPermission( + turnId: string, + toolCallId: string, + decision: "allow" | "deny", + metadata?: JsonValue, + ): Promise { + await this.advanceWithInput(turnId, { + type: "permission_decision", + toolCallId, + decision, + ...(metadata === undefined ? {} : { metadata }), + }); + } + + async respondToAskHuman( + turnId: string, + toolCallId: string, + answer: string, + ): Promise { + await this.advanceWithInput(turnId, { + type: "async_tool_result", + toolCallId, + result: { output: answer, isError: false }, + }); + } + + async deliverAsyncToolResult( + turnId: string, + toolCallId: string, + result: z.infer, + ): Promise { + await this.advanceWithInput(turnId, { + type: "async_tool_result", + toolCallId, + result, + }); + } + + async stopTurn(turnId: string, reason?: string): Promise { + const running = this.active.get(turnId); + if (running) { + running.controller.abort(); + await running.execution.outcome.catch(() => undefined); + return; + } + await this.advanceWithInput(turnId, { + type: "cancel", + ...(reason === undefined ? {} : { reason }), + }); + } + + // Recovery entry for idle (crash-interrupted) turns. Deliberately not run + // at startup: recovery re-issues interrupted model calls, so resumption + // must be an explicit action. Runs in the background. + async resumeTurn(sessionId: string): Promise { + const state = reduceSession(await this.sessionRepo.read(sessionId)); + if (!state.latestTurnId) { + throw new Error(`session ${sessionId} has no turns to resume`); + } + this.startTrackedAdvance(sessionId, state.latestTurnId); + } + + async setTitle(sessionId: string, title: string): Promise { + await this.sessionRepo.withLock(sessionId, async () => { + const events = await this.sessionRepo.read(sessionId); + const batch: Array> = [ + { type: "title_changed", sessionId, ts: this.clock.now(), title }, + ]; + await this.sessionRepo.append(sessionId, batch); + const state = reduceSession([...events, ...batch]); + const existing = this.index.get(sessionId); + this.publishEntry( + sessionIndexEntry(state, existing?.latestTurnStatus ?? "none"), + ); + }); + } + + // §9.4: removes the session file and index entry only. Referenced turn + // files stay behind as inert orphans. + async deleteSession(sessionId: string): Promise { + await this.sessionRepo.withLock(sessionId, async () => { + await this.sessionRepo.delete(sessionId); + this.index.remove(sessionId); + this.bus.publish({ kind: "index-changed", sessionId, entry: null }); + }); + } + + // ------------------------------------------------------------------ + // Internals + // ------------------------------------------------------------------ + + private async resolvedModelOf( + turnId: string, + ): Promise> { + const turn = await this.turnRuntime.getTurn(turnId); + const created = turn.events[0]; + if (created.type !== "turn_created") { + throw new Error(`turn ${turnId} has no turn_created event`); + } + return created.agent.resolved.model; + } + + private async sessionIdOf(turnId: string): Promise { + const turn = await this.turnRuntime.getTurn(turnId); + const created = turn.events[0]; + if (created.type !== "turn_created") { + throw new Error(`turn ${turnId} has no turn_created event`); + } + return created.sessionId; + } + + private async advanceWithInput( + turnId: string, + input: TurnExternalInput, + ): Promise { + const sessionId = await this.sessionIdOf(turnId); + const execution = this.startTrackedAdvance(sessionId, turnId, input); + await execution.outcome; + } + + // Every advance this layer initiates: forward its stream to the bus + // tagged with sessionId, keep the abort controller for stopTurn, and + // update the index entry when the outcome settles. + private startTrackedAdvance( + sessionId: string | null, + turnId: string, + input?: TurnExternalInput, + ): TurnExecution { + const controller = new AbortController(); + const execution = this.turnRuntime.advanceTurn(turnId, input, { + signal: controller.signal, + }); + this.active.set(turnId, { sessionId, controller, execution }); + + void (async () => { + try { + for await (const event of execution.events) { + if (sessionId !== null) { + this.bus.publish({ + kind: "turn-event", + sessionId, + turnId, + event, + }); + } + } + } catch { + // Infrastructure failures surface through the outcome. + } + })(); + + void execution.outcome + .then((outcome) => this.onSettled(sessionId, turnId, outcome)) + .catch(() => undefined) + .finally(() => this.active.delete(turnId)); + + return execution; + } + + private onSettled( + sessionId: string | null, + turnId: string, + outcome: TurnOutcome, + ): void { + if (sessionId === null) { + return; + } + const entry = this.index.get(sessionId); + // The session may have been deleted, or a newer turn appended. + if (!entry || entry.latestTurnId !== turnId) { + return; + } + this.publishEntry({ + ...entry, + latestTurnStatus: outcome.status, + updatedAt: this.clock.now(), + }); + } + + private publishEntry(entry: SessionIndexEntry): void { + this.index.upsert(entry); + this.bus.publish({ + kind: "index-changed", + sessionId: entry.sessionId, + entry, + }); + } +} + +function errorMessage(error: unknown): string { + return error instanceof Error ? error.message : String(error); +} + +function defaultTitle(input: z.infer): string { + const text = + typeof input.content === "string" + ? input.content + : input.content + .map((part) => (part.type === "text" ? part.text : "")) + .join(" "); + const collapsed = text.trim().replace(/\s+/g, " "); + if (collapsed.length === 0) { + return "New session"; + } + return collapsed.length > 80 ? `${collapsed.slice(0, 79)}…` : collapsed; +} diff --git a/apps/x/packages/shared/src/sessions.ts b/apps/x/packages/shared/src/sessions.ts index 52cfd11b..1a52d38f 100644 --- a/apps/x/packages/shared/src/sessions.ts +++ b/apps/x/packages/shared/src/sessions.ts @@ -1,5 +1,9 @@ import { z } from "zod"; -import { ModelDescriptor, type TurnStatus } from "./turns.js"; +import { + ModelDescriptor, + type TurnStatus, + type TurnStreamEvent, +} from "./turns.js"; // Durable session contract for the session layer (see // packages/core/docs/session-design.md). A session is an append-only chain @@ -167,6 +171,22 @@ export interface SessionIndexEntry { error?: string; } +// What the renderer's single feed consumer receives over IPC: live turn +// stream events (durable events + ephemeral deltas) and index updates. +// entry: null signals deletion. +export type SessionBusEvent = + | { + kind: "turn-event"; + sessionId: string; + turnId: string; + event: TurnStreamEvent; + } + | { + kind: "index-changed"; + sessionId: string; + entry: SessionIndexEntry | null; + }; + export function sessionIndexEntry( state: SessionState, latestTurnStatus: SessionLatestTurnStatus, From 1d5782fa757ecb30e844c89b7d3094da5e537cf3 Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 10:45:35 +0530 Subject: [PATCH 08/24] feat(x/core): real bridges + DI assembly for the turn runtime (stage 4) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five bridges adapt existing app code to the runtime seams, each unit- tested against injected fakes: - RealAgentResolver: loadAgent + dynamic builders -> immutable ResolvedAgent. Model precedence override > agent > app default; tool descriptors from BuiltinTools (zod -> JSON schema) and MCP attachments (toolId schemes builtin:/mcp:server:tool); ask-human as the async requiresHuman tool. System prompt composed via composeSystemInstructions — extracted verbatim from streamAgent so old and new runtimes share one implementation — driven by the new opaque RequestedAgent.overrides.composition (session-sticky inputs preserve provider prefix caching; per-message context stays on UserMessage.userMessageContext as today). - RealModelRegistry: models.json -> live AI SDK model; one streamText step (stopWhen: stepCountIs(1)) normalized into deltas, durable step events, and the completed assistant message. - RealToolRegistry: execTool dispatch (builtins + MCP) with per-call abortRegistry bracketing and tool-output-stream -> durable progress. - RealPermissionChecker: getToolPermissionMetadata rules (command allowlist, workspace file boundaries); session grants deferred. - RealPermissionClassifier: classifyToolPermissions with conversation context now threaded through the classifier batch. Interface refinements (doc updated): ToolExecutionContext gains turnId/toolCallId; classifier takes a batch with turnId + messages; TurnRuntime dep renamed bus -> lifecycleBus and SessionsImpl bus -> sessionBus for strict-PROXY DI. Container registers the whole stack (FS repos rooted at WorkDir/storage); the legacy agentRuntime registration became lazy to survive the new import-order cycle. Co-Authored-By: Claude Fable 5 --- .../packages/core/docs/turn-runtime-design.md | 22 +- apps/x/packages/core/src/agents/runtime.ts | 162 ++++++----- apps/x/packages/core/src/di/container.ts | 54 +++- apps/x/packages/core/src/sessions/bus.ts | 21 ++ .../core/src/sessions/sessions.test.ts | 2 +- apps/x/packages/core/src/sessions/sessions.ts | 14 +- .../packages/core/src/turns/bridges/index.ts | 5 + .../turns/bridges/real-agent-resolver.test.ts | 199 ++++++++++++++ .../src/turns/bridges/real-agent-resolver.ts | 198 ++++++++++++++ .../turns/bridges/real-model-registry.test.ts | 178 ++++++++++++ .../src/turns/bridges/real-model-registry.ts | 255 ++++++++++++++++++ .../bridges/real-permission-checker.test.ts | 72 +++++ .../turns/bridges/real-permission-checker.ts | 49 ++++ .../real-permission-classifier.test.ts | 98 +++++++ .../bridges/real-permission-classifier.ts | 57 ++++ .../turns/bridges/real-tool-registry.test.ts | 198 ++++++++++++++ .../src/turns/bridges/real-tool-registry.ts | 141 ++++++++++ apps/x/packages/core/src/turns/bus.ts | 21 ++ apps/x/packages/core/src/turns/permission.ts | 13 +- .../x/packages/core/src/turns/runtime.test.ts | 19 +- apps/x/packages/core/src/turns/runtime.ts | 47 +++- .../packages/core/src/turns/tool-registry.ts | 2 + apps/x/packages/shared/src/turns.ts | 6 + 23 files changed, 1734 insertions(+), 99 deletions(-) create mode 100644 apps/x/packages/core/src/turns/bridges/index.ts create mode 100644 apps/x/packages/core/src/turns/bridges/real-agent-resolver.test.ts create mode 100644 apps/x/packages/core/src/turns/bridges/real-agent-resolver.ts create mode 100644 apps/x/packages/core/src/turns/bridges/real-model-registry.test.ts create mode 100644 apps/x/packages/core/src/turns/bridges/real-model-registry.ts create mode 100644 apps/x/packages/core/src/turns/bridges/real-permission-checker.test.ts create mode 100644 apps/x/packages/core/src/turns/bridges/real-permission-checker.ts create mode 100644 apps/x/packages/core/src/turns/bridges/real-permission-classifier.test.ts create mode 100644 apps/x/packages/core/src/turns/bridges/real-permission-classifier.ts create mode 100644 apps/x/packages/core/src/turns/bridges/real-tool-registry.test.ts create mode 100644 apps/x/packages/core/src/turns/bridges/real-tool-registry.ts diff --git a/apps/x/packages/core/docs/turn-runtime-design.md b/apps/x/packages/core/docs/turn-runtime-design.md index 2daaf2bb..904fcb02 100644 --- a/apps/x/packages/core/docs/turn-runtime-design.md +++ b/apps/x/packages/core/docs/turn-runtime-design.md @@ -247,6 +247,12 @@ interface RequestedAgent { agentId: string; overrides?: { model?: ModelDescriptor; + // Opaque composition hints interpreted by the agent resolver (e.g. + // work-dir id, voice/search/code modes). Persisted verbatim for audit. + // The resolver decides which keys affect system-prompt bytes; callers + // should keep prompt-affecting inputs session-sticky to preserve + // provider prefix caching across turns. + composition?: JsonValue; }; } ``` @@ -608,9 +614,17 @@ interface PermissionClassification { reason: string; } +interface PermissionClassificationBatch { + turnId: string; + // Conversation context: resolved context plus current-turn settled + // messages (the pending batch's tool results are not yet terminal). + messages: ConversationMessage[]; + requests: PermissionClassificationInput[]; +} + interface IPermissionClassifier { classify( - requests: PermissionClassificationInput[], + batch: PermissionClassificationBatch, signal: AbortSignal, ): Promise; } @@ -710,6 +724,8 @@ Sync and async execution are immutable tool metadata: ```ts interface ToolExecutionContext { + turnId: string; + toolCallId: string; signal: AbortSignal; reportProgress(progress: JsonValue): Promise; } @@ -1125,7 +1141,7 @@ interface TurnRuntimeDependencies { contextResolver: IContextResolver; permissionChecker: IPermissionChecker; permissionClassifier: IPermissionClassifier; - bus: IBus; + lifecycleBus: ITurnLifecycleBus; } class TurnRuntime implements ITurnRuntime { @@ -1139,7 +1155,7 @@ class TurnRuntime implements ITurnRuntime { contextResolver, permissionChecker, permissionClassifier, - bus, + lifecycleBus, }: TurnRuntimeDependencies); } ``` diff --git a/apps/x/packages/core/src/agents/runtime.ts b/apps/x/packages/core/src/agents/runtime.ts index da756b2c..178ddccc 100644 --- a/apps/x/packages/core/src/agents/runtime.ts +++ b/apps/x/packages/core/src/agents/runtime.ts @@ -114,7 +114,7 @@ function filePermissionTargets(toolName: string, args: Record): } } -async function getToolPermissionMetadata( +export async function getToolPermissionMetadata( toolCall: z.infer, underlyingTool: z.infer, sessionAllowedCommands: Set, @@ -172,7 +172,7 @@ async function getToolPermissionMetadata( }; } -function loadUserWorkDir(runId: string): string | null { +export function loadUserWorkDir(runId: string): string | null { try { const file = workDirConfigFile(runId); if (!fs.existsSync(file)) return null; @@ -185,7 +185,7 @@ function loadUserWorkDir(runId: string): string | null { } } -function loadAgentNotesContext(): string | null { +export function loadAgentNotesContext(): string | null { const sections: string[] = []; const userFile = path.join(AGENT_NOTES_DIR, 'user.md'); @@ -327,6 +327,87 @@ If Middle pane state is note, the supplied path and content are available so you If Middle pane state is browser, only the URL and page title are supplied; the page content itself is NOT included. If you need the page content to answer, use the browser tools available to you to read the page. The user may or may not be talking about this page. Only reference or act on this page when the user's message clearly relates to it, such as "this page", "this article", "what I'm looking at", "this site", or "summarize this". For unrelated questions, ignore this page entirely and answer normally. Do not mention that you can see the browser unless it is relevant to the answer.`; + +export interface ComposeSystemInstructionsInput { + instructions: string; + agentNotesContext: string | null; + userWorkDir: string | null; + voiceInput: boolean; + voiceOutput: 'summary' | 'full' | null; + searchEnabled: boolean; + codeMode: 'claude' | 'codex' | null; + codeCwd: string | null; +} + +// System-prompt assembly, extracted verbatim from streamAgent so the new turn +// runtime's agent resolver composes byte-identical prompts. Pure: callers +// load agent notes / work dir themselves. +export function composeSystemInstructions({ + instructions, + agentNotesContext, + userWorkDir, + voiceInput, + voiceOutput, + searchEnabled, + codeMode, + codeCwd, +}: ComposeSystemInstructionsInput): string { + let instructionsWithDateTime = `${instructions}\n\n${USER_CONTEXT_SYSTEM_INSTRUCTIONS}`; + if (agentNotesContext) { + instructionsWithDateTime += `\n\n${agentNotesContext}`; + } + if (userWorkDir) { + instructionsWithDateTime += `\n\n# User Work Directory +The user has chosen the following directory as their current **work directory**: + +\`${userWorkDir}\` + +Treat this as the **default location** for file operations whenever the user refers to files generically: +- "list the files", "show me what's in here", "what's the latest report" — list or look in the work directory. +- "save this", "export it", "write that to a file" — write the output into the work directory unless the user names another location. +- "open the file I was just working on", "the doc from earlier" — assume the work directory first. + +Use absolute paths rooted at this directory with the \`file-*\` tools. For example, list with \`file-list({ path: "${userWorkDir}" })\`, read text with \`file-readText\`, and write text with \`file-writeText\`. For PDFs, Office docs, images, scanned docs, and other non-text files, use \`parseFile\` or \`LLMParse\` with the absolute path; you do NOT need to copy the file into the workspace first. + +**Exceptions — these ALWAYS take precedence over the work directory default:** +1. **Knowledge base questions.** If the user asks about anything in the knowledge graph (notes, people, organizations, projects, topics) or paths starting with \`knowledge/\`, use file tools against \`knowledge/\` as documented above. Do NOT redirect those into the work directory. +2. **Explicit paths.** If the user names a different directory or gives an absolute/relative path (e.g. "in ~/Downloads", "from /tmp/foo", "the Desktop"), honor that path exactly and ignore the work-directory default for that request. +3. **Workspace-specific operations.** Anything that obviously belongs in the Rowboat workspace (config files, MCP servers, agent schedules, etc.) stays in the workspace, not the work directory. + +Do not announce the work directory unless it's relevant. Just use it.`; + } + if (voiceInput) { + instructionsWithDateTime += `\n\n# Voice Input\nThe user's message was transcribed from speech. Be aware that:\n- There may be transcription errors. Silently correct obvious ones (e.g. homophones, misheard words). If an error is genuinely ambiguous, briefly mention your interpretation (e.g. "I'm assuming you meant X").\n- Spoken messages are often long-winded. The user may ramble, repeat themselves, or correct something they said earlier in the same message. Focus on their final intent, not every word verbatim.`; + } + if (voiceOutput === 'summary') { + instructionsWithDateTime += `\n\n# Voice Output (MANDATORY — READ THIS FIRST)\nThe user has voice output enabled. THIS IS YOUR #1 PRIORITY: you MUST start your response with tags. If your response does not begin with tags, the user will hear nothing — which is a broken experience. NEVER skip this.\n\nRules:\n1. YOUR VERY FIRST OUTPUT MUST BE A TAG. No exceptions. Do not start with markdown, headings, or any other text. The literal first characters of your response must be "".\n2. Place ALL tags at the BEGINNING of your response, before any detailed content. Do NOT intersperse tags throughout the response.\n3. Wrap EACH spoken sentence in its own separate tag so it can be spoken incrementally. Do NOT wrap everything in a single block.\n4. Use voice as a TL;DR and navigation aid — do NOT read the entire response aloud.\n5. After all tags, you may include detailed written content (markdown, tables, code, etc.) that will be shown visually but not spoken.\n\n## Examples\n\nExample 1 — User asks: "what happened in my meeting with Alex yesterday?"\n\nYour meeting with Alex covered three main things: the Q2 roadmap timeline, hiring for the backend role, and the client demo next week.\nI've pulled out the key details and action items below — the demo prep notes are at the end.\n\n## Meeting with Alex — March 11\n### Roadmap\n- Agreed to push Q2 launch to April 15...\n(detailed written content continues)\n\nExample 2 — User asks: "summarize my emails"\n\nYou have five new emails since this morning.\nTwo are from your team — Jordan sent the RFC you requested and Taylor flagged a contract issue.\nThere's also a warm intro from a VC partner connecting you with someone at a prospective customer.\nI've drafted responses for three of them. The details and drafts are below.\n\n(email blocks, tables, and detailed content follow)\n\nExample 3 — User asks: "what's on my calendar today?"\n\nYou've got a pretty packed day — seven meetings starting with standup at 9.\nThe big ones are your investor call at 11, lunch with a partner from your lead VC at 12:30, and a customer call at 4.\nYour only free block for deep work is 2:30 to 4.\n\n(calendar block with full event details follows)\n\nExample 4 — User asks: "draft an email to Sam with our metrics"\n\nDone — I've drafted the email to Sam with your latest WAU and churn numbers.\nTake a look at the draft below and send it when you're ready.\n\n(email block with draft follows)\n\nREMEMBER: If you do not start with tags, the user hears silence. Always speak first, then write.`; + } else if (voiceOutput === 'full') { + instructionsWithDateTime += `\n\n# Voice Output — Full Read-Aloud (MANDATORY — READ THIS FIRST)\nThe user wants your ENTIRE response spoken aloud. THIS IS YOUR #1 PRIORITY: every single sentence must be wrapped in tags. If you write anything outside tags, the user will not hear it — which is a broken experience. NEVER skip this.\n\nRules:\n1. YOUR VERY FIRST OUTPUT MUST BE A TAG. No exceptions. The literal first characters of your response must be "".\n2. Wrap EACH sentence in its own separate tag so it can be spoken incrementally.\n3. Write your response in a natural, conversational style suitable for listening — no markdown headings, bullet points, or formatting symbols. Use plain spoken language.\n4. Structure the content as if you are speaking to the user directly. Use transitions like "first", "also", "one more thing" instead of visual formatting.\n5. EVERY sentence MUST be inside a tag. Do not leave ANY content outside tags. If it's not in a tag, the user cannot hear it.\n\n## Examples\n\nExample 1 — User asks: "what happened in my meeting with Alex yesterday?"\n\nYour meeting with Alex covered three main things.\nFirst, you discussed the Q2 roadmap timeline and agreed to push the launch to April.\nSecond, you talked about hiring for the backend role — Alex will send over two candidates by Friday.\nAnd lastly, the client demo is next week on Thursday at 2pm, and you're handling the intro slides.\n\nExample 2 — User asks: "summarize my emails"\n\nYou've got five new emails since this morning.\nTwo are from your team — Jordan sent the RFC you asked for, and Taylor flagged a contract issue that needs your sign-off.\nThere's a warm intro from a VC partner connecting you with an engineering lead at a potential customer.\nAnd someone from a prospective client wants to confirm your API tier before your call this afternoon.\nI've drafted replies for three of them — the metrics update, the intro, and the API question.\nThe only one I left for you is Taylor's contract redline, since that needs your judgment on the liability cap.\n\nExample 3 — User asks: "what's on my calendar today?"\n\nYou've got a packed day — seven meetings starting with standup at 9.\nThe highlights are your investor call at 11, lunch with a VC partner at 12:30, and a customer call at 4.\nYour only open block for deep work is 2:30 to 4, so plan accordingly.\nOh, and your 1-on-1 with your co-founder is at 5:30 — that's a walking meeting.\n\nExample 4 — User asks: "how are our metrics looking?"\n\nMetrics are looking strong this week.\nYou hit 2,573 weekly active users, which is up 12% week over week.\nThat means you've crossed the 2,500 milestone — worth calling out in your next investor update.\nChurn is down to 4.1%, improving month over month.\nThe trailing 8-week compound growth rate is about 10%.\n\nREMEMBER: Start with immediately. No preamble, no markdown before it. Speak first.`; + } + if (searchEnabled) { + instructionsWithDateTime += `\n\n# Search\nThe user has requested a search. Use the web-search tool to answer their query.`; + } + if (codeMode) { + const agentDisplay = codeMode === 'claude' ? 'Claude Code' : 'Codex'; + instructionsWithDateTime += `\n\n# Code Mode (Active) — Agent: ${agentDisplay} +The user has turned on **code mode** and the composer chip is set to **${agentDisplay}** (\`${codeMode}\`). For EVERY coding task this turn, use **${agentDisplay}**, and narrate that agent ("Using ${agentDisplay} to …"). + +The chip is the single source of truth for which agent runs: +- Do NOT carry over a different agent from earlier in this thread — even if a previous run used the other agent, use **${agentDisplay}** now. +- Do NOT switch agents based on an in-chat text request ("use codex", "switch to claude"). The agent only changes when the user toggles the chip; if they ask in chat, tell them to toggle the chip. + +**How to run coding work — call the \`code_agent_run\` tool** with: +- \`agent\`: \`${codeMode}\` (always — match the chip). +- \`cwd\`: ${codeCwd ? `\`${codeCwd}\` (always — this coding session is pinned to that directory; never use another path)` : `the absolute project/working directory (resolve it per the code-with-agents skill — a path the user named, the "# User Work Directory" block, or ask once)`}. +- \`prompt\`: a clear, self-contained coding instruction. + +The tool runs the agent on-device and streams its tool calls, file diffs, and plan into the chat; any action needing approval surfaces as an inline permission card, so you do NOT pre-confirm with an in-chat "reply yes". This chat keeps ONE persistent agent session, so follow-up coding requests automatically resume with full context — just call \`code_agent_run\` again. Do NOT shell out to \`acpx\` or \`executeCommand\` for coding, and do NOT fall back to your own file tools. + +If the user's message is clearly NOT a coding request (small talk, an unrelated question), answer directly without invoking the coding agent. Code mode signals readiness, not that every message must route through the agent.`; + } + return instructionsWithDateTime; +} + export interface IAgentRuntime { trigger(runId: string): Promise; } @@ -1423,70 +1504,17 @@ export async function* streamAgent({ loopLogger.log('running llm turn'); // stream agent response and build message const messageBuilder = new StreamStepMessageBuilder(); - let instructionsWithDateTime = `${agent.instructions}\n\n${USER_CONTEXT_SYSTEM_INSTRUCTIONS}`; - // Inject Agent Notes context for copilot - if (state.agentName === 'copilot' || state.agentName === 'rowboatx') { - const agentNotesContext = loadAgentNotesContext(); - if (agentNotesContext) { - instructionsWithDateTime += `\n\n${agentNotesContext}`; - } - const userWorkDir = loadUserWorkDir(runId); - if (userWorkDir) { - loopLogger.log('injecting user work directory', userWorkDir); - instructionsWithDateTime += `\n\n# User Work Directory -The user has chosen the following directory as their current **work directory**: - -\`${userWorkDir}\` - -Treat this as the **default location** for file operations whenever the user refers to files generically: -- "list the files", "show me what's in here", "what's the latest report" — list or look in the work directory. -- "save this", "export it", "write that to a file" — write the output into the work directory unless the user names another location. -- "open the file I was just working on", "the doc from earlier" — assume the work directory first. - -Use absolute paths rooted at this directory with the \`file-*\` tools. For example, list with \`file-list({ path: "${userWorkDir}" })\`, read text with \`file-readText\`, and write text with \`file-writeText\`. For PDFs, Office docs, images, scanned docs, and other non-text files, use \`parseFile\` or \`LLMParse\` with the absolute path; you do NOT need to copy the file into the workspace first. - -**Exceptions — these ALWAYS take precedence over the work directory default:** -1. **Knowledge base questions.** If the user asks about anything in the knowledge graph (notes, people, organizations, projects, topics) or paths starting with \`knowledge/\`, use file tools against \`knowledge/\` as documented above. Do NOT redirect those into the work directory. -2. **Explicit paths.** If the user names a different directory or gives an absolute/relative path (e.g. "in ~/Downloads", "from /tmp/foo", "the Desktop"), honor that path exactly and ignore the work-directory default for that request. -3. **Workspace-specific operations.** Anything that obviously belongs in the Rowboat workspace (config files, MCP servers, agent schedules, etc.) stays in the workspace, not the work directory. - -Do not announce the work directory unless it's relevant. Just use it.`; - } - } - if (voiceInput) { - loopLogger.log('voice input enabled, injecting voice input prompt'); - instructionsWithDateTime += `\n\n# Voice Input\nThe user's message was transcribed from speech. Be aware that:\n- There may be transcription errors. Silently correct obvious ones (e.g. homophones, misheard words). If an error is genuinely ambiguous, briefly mention your interpretation (e.g. "I'm assuming you meant X").\n- Spoken messages are often long-winded. The user may ramble, repeat themselves, or correct something they said earlier in the same message. Focus on their final intent, not every word verbatim.`; - } - if (voiceOutput === 'summary') { - loopLogger.log('voice output enabled (summary mode), injecting voice output prompt'); - instructionsWithDateTime += `\n\n# Voice Output (MANDATORY — READ THIS FIRST)\nThe user has voice output enabled. THIS IS YOUR #1 PRIORITY: you MUST start your response with tags. If your response does not begin with tags, the user will hear nothing — which is a broken experience. NEVER skip this.\n\nRules:\n1. YOUR VERY FIRST OUTPUT MUST BE A TAG. No exceptions. Do not start with markdown, headings, or any other text. The literal first characters of your response must be "".\n2. Place ALL tags at the BEGINNING of your response, before any detailed content. Do NOT intersperse tags throughout the response.\n3. Wrap EACH spoken sentence in its own separate tag so it can be spoken incrementally. Do NOT wrap everything in a single block.\n4. Use voice as a TL;DR and navigation aid — do NOT read the entire response aloud.\n5. After all tags, you may include detailed written content (markdown, tables, code, etc.) that will be shown visually but not spoken.\n\n## Examples\n\nExample 1 — User asks: "what happened in my meeting with Alex yesterday?"\n\nYour meeting with Alex covered three main things: the Q2 roadmap timeline, hiring for the backend role, and the client demo next week.\nI've pulled out the key details and action items below — the demo prep notes are at the end.\n\n## Meeting with Alex — March 11\n### Roadmap\n- Agreed to push Q2 launch to April 15...\n(detailed written content continues)\n\nExample 2 — User asks: "summarize my emails"\n\nYou have five new emails since this morning.\nTwo are from your team — Jordan sent the RFC you requested and Taylor flagged a contract issue.\nThere's also a warm intro from a VC partner connecting you with someone at a prospective customer.\nI've drafted responses for three of them. The details and drafts are below.\n\n(email blocks, tables, and detailed content follow)\n\nExample 3 — User asks: "what's on my calendar today?"\n\nYou've got a pretty packed day — seven meetings starting with standup at 9.\nThe big ones are your investor call at 11, lunch with a partner from your lead VC at 12:30, and a customer call at 4.\nYour only free block for deep work is 2:30 to 4.\n\n(calendar block with full event details follows)\n\nExample 4 — User asks: "draft an email to Sam with our metrics"\n\nDone — I've drafted the email to Sam with your latest WAU and churn numbers.\nTake a look at the draft below and send it when you're ready.\n\n(email block with draft follows)\n\nREMEMBER: If you do not start with tags, the user hears silence. Always speak first, then write.`; - } else if (voiceOutput === 'full') { - loopLogger.log('voice output enabled (full mode), injecting voice output prompt'); - instructionsWithDateTime += `\n\n# Voice Output — Full Read-Aloud (MANDATORY — READ THIS FIRST)\nThe user wants your ENTIRE response spoken aloud. THIS IS YOUR #1 PRIORITY: every single sentence must be wrapped in tags. If you write anything outside tags, the user will not hear it — which is a broken experience. NEVER skip this.\n\nRules:\n1. YOUR VERY FIRST OUTPUT MUST BE A TAG. No exceptions. The literal first characters of your response must be "".\n2. Wrap EACH sentence in its own separate tag so it can be spoken incrementally.\n3. Write your response in a natural, conversational style suitable for listening — no markdown headings, bullet points, or formatting symbols. Use plain spoken language.\n4. Structure the content as if you are speaking to the user directly. Use transitions like "first", "also", "one more thing" instead of visual formatting.\n5. EVERY sentence MUST be inside a tag. Do not leave ANY content outside tags. If it's not in a tag, the user cannot hear it.\n\n## Examples\n\nExample 1 — User asks: "what happened in my meeting with Alex yesterday?"\n\nYour meeting with Alex covered three main things.\nFirst, you discussed the Q2 roadmap timeline and agreed to push the launch to April.\nSecond, you talked about hiring for the backend role — Alex will send over two candidates by Friday.\nAnd lastly, the client demo is next week on Thursday at 2pm, and you're handling the intro slides.\n\nExample 2 — User asks: "summarize my emails"\n\nYou've got five new emails since this morning.\nTwo are from your team — Jordan sent the RFC you asked for, and Taylor flagged a contract issue that needs your sign-off.\nThere's a warm intro from a VC partner connecting you with an engineering lead at a potential customer.\nAnd someone from a prospective client wants to confirm your API tier before your call this afternoon.\nI've drafted replies for three of them — the metrics update, the intro, and the API question.\nThe only one I left for you is Taylor's contract redline, since that needs your judgment on the liability cap.\n\nExample 3 — User asks: "what's on my calendar today?"\n\nYou've got a packed day — seven meetings starting with standup at 9.\nThe highlights are your investor call at 11, lunch with a VC partner at 12:30, and a customer call at 4.\nYour only open block for deep work is 2:30 to 4, so plan accordingly.\nOh, and your 1-on-1 with your co-founder is at 5:30 — that's a walking meeting.\n\nExample 4 — User asks: "how are our metrics looking?"\n\nMetrics are looking strong this week.\nYou hit 2,573 weekly active users, which is up 12% week over week.\nThat means you've crossed the 2,500 milestone — worth calling out in your next investor update.\nChurn is down to 4.1%, improving month over month.\nThe trailing 8-week compound growth rate is about 10%.\n\nREMEMBER: Start with immediately. No preamble, no markdown before it. Speak first.`; - } - if (searchEnabled) { - loopLogger.log('search enabled, injecting search prompt'); - instructionsWithDateTime += `\n\n# Search\nThe user has requested a search. Use the web-search tool to answer their query.`; - } - if (codeMode) { - loopLogger.log('code mode enabled, injecting coding-agent context', codeMode); - const agentDisplay = codeMode === 'claude' ? 'Claude Code' : 'Codex'; - instructionsWithDateTime += `\n\n# Code Mode (Active) — Agent: ${agentDisplay} -The user has turned on **code mode** and the composer chip is set to **${agentDisplay}** (\`${codeMode}\`). For EVERY coding task this turn, use **${agentDisplay}**, and narrate that agent ("Using ${agentDisplay} to …"). - -The chip is the single source of truth for which agent runs: -- Do NOT carry over a different agent from earlier in this thread — even if a previous run used the other agent, use **${agentDisplay}** now. -- Do NOT switch agents based on an in-chat text request ("use codex", "switch to claude"). The agent only changes when the user toggles the chip; if they ask in chat, tell them to toggle the chip. - -**How to run coding work — call the \`code_agent_run\` tool** with: -- \`agent\`: \`${codeMode}\` (always — match the chip). -- \`cwd\`: ${codeCwd ? `\`${codeCwd}\` (always — this coding session is pinned to that directory; never use another path)` : `the absolute project/working directory (resolve it per the code-with-agents skill — a path the user named, the "# User Work Directory" block, or ask once)`}. -- \`prompt\`: a clear, self-contained coding instruction. - -The tool runs the agent on-device and streams its tool calls, file diffs, and plan into the chat; any action needing approval surfaces as an inline permission card, so you do NOT pre-confirm with an in-chat "reply yes". This chat keeps ONE persistent agent session, so follow-up coding requests automatically resume with full context — just call \`code_agent_run\` again. Do NOT shell out to \`acpx\` or \`executeCommand\` for coding, and do NOT fall back to your own file tools. - -If the user's message is clearly NOT a coding request (small talk, an unrelated question), answer directly without invoking the coding agent. Code mode signals readiness, not that every message must route through the agent.`; - } + const composeCopilotContext = state.agentName === 'copilot' || state.agentName === 'rowboatx'; + const instructionsWithDateTime = composeSystemInstructions({ + instructions: agent.instructions, + agentNotesContext: composeCopilotContext ? loadAgentNotesContext() : null, + userWorkDir: composeCopilotContext ? loadUserWorkDir(runId) : null, + voiceInput, + voiceOutput, + searchEnabled, + codeMode, + codeCwd, + }); let streamError: string | null = null; for await (const event of streamLlm( model, diff --git a/apps/x/packages/core/src/di/container.ts b/apps/x/packages/core/src/di/container.ts index 4bfc92ed..34388754 100644 --- a/apps/x/packages/core/src/di/container.ts +++ b/apps/x/packages/core/src/di/container.ts @@ -1,4 +1,6 @@ -import { asClass, asValue, createContainer, InjectionMode } from "awilix"; +import path from "node:path"; +import { asClass, asFunction, asValue, createContainer, InjectionMode } from "awilix"; +import { WorkDir } from "../config/config.js"; import { FSModelConfigRepo, IModelConfigRepo } from "../models/repo.js"; import { FSMcpConfigRepo, IMcpConfigRepo } from "../mcp/repo.js"; import { FSAgentsRepo, IAgentsRepo } from "../agents/repo.js"; @@ -24,6 +26,27 @@ import { CodeSessionService } from "../code-mode/sessions/service.js"; import { CodeSessionStatusTracker } from "../code-mode/sessions/status-tracker.js"; import type { IBrowserControlService } from "../application/browser-control/service.js"; import type { INotificationService } from "../application/notification/service.js"; +import { SystemClock, type IClock } from "../turns/clock.js"; +import { FSTurnRepo } from "../turns/fs-repo.js"; +import type { ITurnRepo } from "../turns/repo.js"; +import { TurnRepoContextResolver, type IContextResolver } from "../turns/context-resolver.js"; +import { EmitterTurnLifecycleBus, type ITurnLifecycleBus } from "../turns/bus.js"; +import { TurnRuntime } from "../turns/runtime.js"; +import type { ITurnRuntime } from "../turns/api.js"; +import type { IAgentResolver } from "../turns/agent-resolver.js"; +import type { IModelRegistry } from "../turns/model-registry.js"; +import type { IToolRegistry } from "../turns/tool-registry.js"; +import type { IPermissionChecker, IPermissionClassifier } from "../turns/permission.js"; +import { RealAgentResolver } from "../turns/bridges/real-agent-resolver.js"; +import { RealModelRegistry } from "../turns/bridges/real-model-registry.js"; +import { RealToolRegistry } from "../turns/bridges/real-tool-registry.js"; +import { RealPermissionChecker } from "../turns/bridges/real-permission-checker.js"; +import { RealPermissionClassifier } from "../turns/bridges/real-permission-classifier.js"; +import { FSSessionRepo } from "../sessions/fs-repo.js"; +import type { ISessionRepo } from "../sessions/repo.js"; +import { EmitterSessionBus, type ISessionBus } from "../sessions/bus.js"; +import { SessionsImpl } from "../sessions/sessions.js"; +import type { ISessions } from "../sessions/api.js"; const container = createContainer({ injectionMode: InjectionMode.PROXY, @@ -36,7 +59,15 @@ container.register({ bus: asClass(InMemoryBus).singleton(), runsLock: asClass(InMemoryRunsLock).singleton(), abortRegistry: asClass(InMemoryAbortRegistry).singleton(), - agentRuntime: asClass(AgentRuntime).singleton(), + // Lazy: agents/runtime.js participates in an import cycle with this + // module (and is now also reachable via the turn-runtime bridges), so the + // class binding may not be initialized yet when this body runs. + agentRuntime: asFunction( + (cradle) => + new AgentRuntime( + cradle as unknown as ConstructorParameters[0], + ), + ).singleton(), mcpConfigRepo: asClass(FSMcpConfigRepo).singleton(), modelConfigRepo: asClass(FSModelConfigRepo).singleton(), @@ -62,6 +93,25 @@ container.register({ codeSessionsRepo: asClass(FSCodeSessionsRepo).singleton(), codeSessionService: asClass(CodeSessionService).singleton(), codeSessionStatusTracker: asClass(CodeSessionStatusTracker).singleton(), + + // New turn/session runtime (turn-runtime-design.md / session-design.md). + // Bridges are constructed via asFunction so their optional test seams + // don't collide with strict PROXY cradle resolution. + clock: asClass(SystemClock).singleton(), + turnsRootDir: asValue(path.join(WorkDir, "storage", "turns")), + sessionsRootDir: asValue(path.join(WorkDir, "storage", "sessions")), + turnRepo: asClass(FSTurnRepo).singleton(), + contextResolver: asClass(TurnRepoContextResolver).singleton(), + lifecycleBus: asClass(EmitterTurnLifecycleBus).singleton(), + agentResolver: asFunction(() => new RealAgentResolver()).singleton(), + modelRegistry: asFunction(() => new RealModelRegistry()).singleton(), + toolRegistry: asFunction(() => new RealToolRegistry()).singleton(), + permissionChecker: asFunction(() => new RealPermissionChecker()).singleton(), + permissionClassifier: asFunction(() => new RealPermissionClassifier()).singleton(), + turnRuntime: asClass(TurnRuntime).singleton(), + sessionRepo: asClass(FSSessionRepo).singleton(), + sessionBus: asClass(EmitterSessionBus).singleton(), + sessions: asClass(SessionsImpl).singleton(), }); export default container; diff --git a/apps/x/packages/core/src/sessions/bus.ts b/apps/x/packages/core/src/sessions/bus.ts index e82f6f47..02fe25a6 100644 --- a/apps/x/packages/core/src/sessions/bus.ts +++ b/apps/x/packages/core/src/sessions/bus.ts @@ -5,3 +5,24 @@ import type { SessionBusEvent } from "@x/shared/dist/sessions.js"; export interface ISessionBus { publish(event: SessionBusEvent): void; } + +// Default in-process fan-out; the app layer subscribes and forwards over IPC. +// Listener errors are swallowed: observers must never affect sessions. +export class EmitterSessionBus implements ISessionBus { + private listeners = new Set<(event: SessionBusEvent) => void>(); + + publish(event: SessionBusEvent): void { + for (const listener of this.listeners) { + try { + listener(event); + } catch { + // observational only + } + } + } + + subscribe(listener: (event: SessionBusEvent) => void): () => void { + this.listeners.add(listener); + return () => this.listeners.delete(listener); + } +} diff --git a/apps/x/packages/core/src/sessions/sessions.test.ts b/apps/x/packages/core/src/sessions/sessions.test.ts index f0c53fcf..ad9809cd 100644 --- a/apps/x/packages/core/src/sessions/sessions.test.ts +++ b/apps/x/packages/core/src/sessions/sessions.test.ts @@ -306,7 +306,7 @@ function makeSessions(opts: { repo?: ISessionRepo; fake?: FakeTurnRuntime } = {} turnRuntime: fake, idGenerator: new FakeIdGen(), clock: new FakeClock(), - bus, + sessionBus: bus, }); return { sessions, repo, fake, bus }; } diff --git a/apps/x/packages/core/src/sessions/sessions.ts b/apps/x/packages/core/src/sessions/sessions.ts index c258c2eb..ad9d8c98 100644 --- a/apps/x/packages/core/src/sessions/sessions.ts +++ b/apps/x/packages/core/src/sessions/sessions.ts @@ -39,7 +39,7 @@ export interface SessionsDependencies { turnRuntime: ITurnRuntime; idGenerator: IMonotonicallyIncreasingIdGenerator; clock: IClock; - bus: ISessionBus; + sessionBus: ISessionBus; } // The session layer per session-design.md: owns conversations as ordered @@ -52,7 +52,7 @@ export class SessionsImpl implements ISessions { private readonly turnRuntime: ITurnRuntime; private readonly idGenerator: IMonotonicallyIncreasingIdGenerator; private readonly clock: IClock; - private readonly bus: ISessionBus; + private readonly sessionBus: ISessionBus; private readonly index = new SessionIndex(); // Ephemeral: executions this process started, for stopTurn's abort path. @@ -66,13 +66,13 @@ export class SessionsImpl implements ISessions { turnRuntime, idGenerator, clock, - bus, + sessionBus, }: SessionsDependencies) { this.sessionRepo = sessionRepo; this.turnRuntime = turnRuntime; this.idGenerator = idGenerator; this.clock = clock; - this.bus = bus; + this.sessionBus = sessionBus; } // §8.2: scan session files, read each session's latest turn for status. @@ -293,7 +293,7 @@ export class SessionsImpl implements ISessions { await this.sessionRepo.withLock(sessionId, async () => { await this.sessionRepo.delete(sessionId); this.index.remove(sessionId); - this.bus.publish({ kind: "index-changed", sessionId, entry: null }); + this.sessionBus.publish({ kind: "index-changed", sessionId, entry: null }); }); } @@ -348,7 +348,7 @@ export class SessionsImpl implements ISessions { try { for await (const event of execution.events) { if (sessionId !== null) { - this.bus.publish({ + this.sessionBus.publish({ kind: "turn-event", sessionId, turnId, @@ -391,7 +391,7 @@ export class SessionsImpl implements ISessions { private publishEntry(entry: SessionIndexEntry): void { this.index.upsert(entry); - this.bus.publish({ + this.sessionBus.publish({ kind: "index-changed", sessionId: entry.sessionId, entry, diff --git a/apps/x/packages/core/src/turns/bridges/index.ts b/apps/x/packages/core/src/turns/bridges/index.ts new file mode 100644 index 00000000..5927554c --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/index.ts @@ -0,0 +1,5 @@ +export * from "./real-agent-resolver.js"; +export * from "./real-model-registry.js"; +export * from "./real-permission-checker.js"; +export * from "./real-permission-classifier.js"; +export * from "./real-tool-registry.js"; diff --git a/apps/x/packages/core/src/turns/bridges/real-agent-resolver.test.ts b/apps/x/packages/core/src/turns/bridges/real-agent-resolver.test.ts new file mode 100644 index 00000000..974d7502 --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/real-agent-resolver.test.ts @@ -0,0 +1,199 @@ +import { describe, expect, it } from "vitest"; +import { z } from "zod"; +import type { Agent } from "@x/shared/dist/agent.js"; +import { composeSystemInstructions } from "../../agents/runtime.js"; +import type { BuiltinTools } from "../../application/lib/builtin-tools.js"; +import { RealAgentResolver } from "./real-agent-resolver.js"; + +const DEFAULTS = async () => ({ model: "gpt-default", provider: "openai" }); + +function makeAgent( + overrides: Partial> = {}, +): z.infer { + return { + name: "copilot", + instructions: "You are Copilot.", + tools: {}, + ...overrides, + }; +} + +// A minimal fake builtin catalog (the real one is heavy and irrelevant here). +const fakeBuiltins = { + "file-list": { + description: "List files", + inputSchema: z.object({ path: z.string() }), + execute: async () => null, + }, + "web-search": { + description: "Search the web", + inputSchema: z.object({ query: z.string() }), + execute: async () => null, + isAvailable: async () => false, + }, +} as unknown as typeof BuiltinTools; + +function makeResolver(agent: z.infer, deps: Partial[0]> = {}) { + return new RealAgentResolver({ + load: async () => agent, + builtins: fakeBuiltins, + defaultModel: DEFAULTS, + loadNotes: () => null, + loadWorkDir: () => null, + ...deps, + }); +} + +describe("RealAgentResolver", () => { + it("applies model precedence: override > agent config > app default", async () => { + const withNothing = makeResolver(makeAgent()); + expect((await withNothing.resolve({ agentId: "copilot" })).model).toEqual({ + provider: "openai", + model: "gpt-default", + }); + + const withAgentModel = makeResolver( + makeAgent({ provider: "anthropic", model: "claude-x" }), + ); + expect( + (await withAgentModel.resolve({ agentId: "copilot" })).model, + ).toEqual({ provider: "anthropic", model: "claude-x" }); + + expect( + ( + await withAgentModel.resolve({ + agentId: "copilot", + overrides: { model: { provider: "google", model: "gemini-x" } }, + }) + ).model, + ).toEqual({ provider: "google", model: "gemini-x" }); + }); + + it("maps builtins to descriptors with JSON schemas, filtering unavailable ones", async () => { + const resolver = makeResolver( + makeAgent({ + tools: { + "file-list": { type: "builtin", name: "file-list" }, + "web-search": { type: "builtin", name: "web-search" }, // unavailable + ghost: { type: "builtin", name: "ghost" }, // not in catalog + }, + }), + ); + const resolved = await resolver.resolve({ agentId: "copilot" }); + expect(resolved.tools).toHaveLength(1); + expect(resolved.tools[0]).toMatchObject({ + toolId: "builtin:file-list", + name: "file-list", + description: "List files", + execution: "sync", + requiresHuman: false, + }); + expect(resolved.tools[0].inputSchema).toMatchObject({ + type: "object", + properties: { path: { type: "string" } }, + }); + }); + + it("passes MCP schemas through and skips agent-as-tool attachments", async () => { + const resolver = makeResolver( + makeAgent({ + tools: { + lookup: { + type: "mcp", + name: "lookup", + description: "Look things up", + inputSchema: { type: "object", properties: { q: { type: "string" } } }, + mcpServerName: "kb", + }, + subflow: { type: "agent", name: "researcher" }, + }, + }), + ); + const resolved = await resolver.resolve({ agentId: "copilot" }); + expect(resolved.tools).toHaveLength(1); + expect(resolved.tools[0]).toMatchObject({ + toolId: "mcp:kb:lookup", + name: "lookup", + description: "Look things up", + execution: "sync", + }); + }); + + it("maps ask-human to an async human-dependent descriptor", async () => { + const resolver = makeResolver( + makeAgent({ + tools: { "ask-human": { type: "builtin", name: "ask-human" } }, + }), + ); + const resolved = await resolver.resolve({ agentId: "copilot" }); + expect(resolved.tools[0]).toMatchObject({ + toolId: "builtin:ask-human", + execution: "async", + requiresHuman: true, + }); + }); + + it("composes the system prompt byte-identically to the shared composer", async () => { + const resolver = makeResolver(makeAgent(), { + loadNotes: () => "# Agent Notes\nremember X", + loadWorkDir: (id) => (id === "sess-1" ? "/Users/me/work" : null), + }); + const resolved = await resolver.resolve({ + agentId: "copilot", + overrides: { + composition: { + workDirId: "sess-1", + searchEnabled: true, + codeMode: "claude", + }, + }, + }); + expect(resolved.systemPrompt).toBe( + composeSystemInstructions({ + instructions: "You are Copilot.", + agentNotesContext: "# Agent Notes\nremember X", + userWorkDir: "/Users/me/work", + voiceInput: false, + voiceOutput: null, + searchEnabled: true, + codeMode: "claude", + codeCwd: null, + }), + ); + }); + + it("is prompt-stable: identical composition yields identical bytes; unknown keys are ignored", async () => { + const resolver = makeResolver(makeAgent()); + const a = await resolver.resolve({ agentId: "copilot" }); + const b = await resolver.resolve({ + agentId: "copilot", + overrides: { composition: { someUnknownKey: 42 } }, + }); + expect(b.systemPrompt).toBe(a.systemPrompt); + }); + + it("does not load notes/work-dir for non-copilot agents", async () => { + let notesLoaded = 0; + const resolver = makeResolver(makeAgent({ name: "background-task-agent" }), { + loadNotes: () => { + notesLoaded += 1; + return "notes"; + }, + }); + await resolver.resolve({ agentId: "background-task-agent" }); + expect(notesLoaded).toBe(0); + }); + + it("rejects when the agent cannot be loaded, creating nothing", async () => { + const resolver = new RealAgentResolver({ + load: async () => { + throw new Error("no such agent"); + }, + builtins: fakeBuiltins, + defaultModel: DEFAULTS, + }); + await expect(resolver.resolve({ agentId: "ghost" })).rejects.toThrowError( + "no such agent", + ); + }); +}); diff --git a/apps/x/packages/core/src/turns/bridges/real-agent-resolver.ts b/apps/x/packages/core/src/turns/bridges/real-agent-resolver.ts new file mode 100644 index 00000000..d4fd1987 --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/real-agent-resolver.ts @@ -0,0 +1,198 @@ +import { z } from "zod"; +import type { Agent } from "@x/shared/dist/agent.js"; +import { + type JsonValue, + RequestedAgent, + ResolvedAgent, + type ToolDescriptor, +} from "@x/shared/dist/turns.js"; +import { + composeSystemInstructions, + loadAgent, + loadAgentNotesContext, + loadUserWorkDir, +} from "../../agents/runtime.js"; +import { BuiltinTools } from "../../application/lib/builtin-tools.js"; +import { getDefaultModelAndProvider } from "../../models/defaults.js"; +import type { IAgentResolver } from "../agent-resolver.js"; + +export const ASK_HUMAN_TOOL = "ask-human"; + +const ASK_HUMAN_DESCRIPTOR: z.infer = { + toolId: `builtin:${ASK_HUMAN_TOOL}`, + name: ASK_HUMAN_TOOL, + description: + "Ask a human before proceeding. Optionally pass `options` (an array of short button labels) when a small set of choices would help the human answer quickly.", + inputSchema: { + type: "object", + properties: { + question: { + type: "string", + description: "The question to ask the human", + }, + options: { + type: "array", + items: { type: "string" }, + description: "Optional short button labels the human can pick from", + }, + }, + required: ["question"], + additionalProperties: false, + }, + execution: "async", + requiresHuman: true, +}; + +// Recognized keys of the opaque RequestedAgent.overrides.composition value. +// Unknown keys are ignored. Prompt-affecting inputs should be session-sticky: +// every key here alters system-prompt bytes and therefore busts provider +// prefix caching when it changes between turns. +const CompositionOverrides = z.object({ + workDirId: z.string().nullable().optional(), + voiceInput: z.boolean().optional(), + voiceOutput: z.enum(["summary", "full"]).nullable().optional(), + searchEnabled: z.boolean().optional(), + codeMode: z.enum(["claude", "codex"]).nullable().optional(), + codeCwd: z.string().nullable().optional(), +}); + +export interface RealAgentResolverDeps { + load?: typeof loadAgent; + builtins?: typeof BuiltinTools; + defaultModel?: () => Promise<{ model: string; provider: string }>; + loadNotes?: () => string | null; + loadWorkDir?: (workDirId: string) => string | null; +} + +// Bridges the existing agent system (loadAgent + dynamic builders, the +// BuiltinTools catalog, MCP attachments) to the immutable ResolvedAgent +// snapshot. The composed system prompt is byte-identical to the old +// runtime's streamAgent assembly for the same inputs. +export class RealAgentResolver implements IAgentResolver { + private readonly load: typeof loadAgent; + private readonly builtins: typeof BuiltinTools; + private readonly defaultModel: () => Promise<{ model: string; provider: string }>; + private readonly loadNotes: () => string | null; + private readonly loadWorkDir: (workDirId: string) => string | null; + + constructor(deps: RealAgentResolverDeps = {}) { + this.load = deps.load ?? loadAgent; + this.builtins = deps.builtins ?? BuiltinTools; + this.defaultModel = deps.defaultModel ?? getDefaultModelAndProvider; + this.loadNotes = deps.loadNotes ?? loadAgentNotesContext; + this.loadWorkDir = deps.loadWorkDir ?? loadUserWorkDir; + } + + async resolve( + requested: z.infer, + ): Promise> { + const agent = await this.load(requested.agentId); + if (!agent) { + throw new Error(`agent not found: ${requested.agentId}`); + } + + // Model precedence: createTurn override > agent config > app default. + let model = requested.overrides?.model; + if (!model) { + const fallback = await this.defaultModel(); + model = { + provider: agent.provider ?? fallback.provider, + model: agent.model ?? fallback.model, + }; + } + + const parsed = CompositionOverrides.safeParse( + requested.overrides?.composition ?? {}, + ); + const composition = parsed.success ? parsed.data : {}; + // Agent notes and work-dir context are copilot-scoped, matching the + // old runtime's behavior. + const copilotContext = + requested.agentId === "copilot" || requested.agentId === "rowboatx"; + const systemPrompt = composeSystemInstructions({ + instructions: agent.instructions, + agentNotesContext: copilotContext ? this.loadNotes() : null, + userWorkDir: + copilotContext && composition.workDirId + ? this.loadWorkDir(composition.workDirId) + : null, + voiceInput: composition.voiceInput ?? false, + voiceOutput: composition.voiceOutput ?? null, + searchEnabled: composition.searchEnabled ?? false, + codeMode: composition.codeMode ?? null, + codeCwd: composition.codeCwd ?? null, + }); + + const tools = await this.resolveTools(agent); + return ResolvedAgent.parse({ + agentId: requested.agentId, + systemPrompt, + model, + tools, + }); + } + + private async resolveTools( + agent: z.infer, + ): Promise>> { + const tools: Array> = []; + for (const [name, attachment] of Object.entries(agent.tools ?? {})) { + if (attachment.type === "agent") { + continue; // agent-as-tool is not supported in v1 + } + if (attachment.type === "mcp") { + tools.push({ + toolId: `mcp:${attachment.mcpServerName}:${attachment.name}`, + name, + description: attachment.description, + inputSchema: + toJsonValue(attachment.inputSchema) ?? + { type: "object", properties: {} }, + execution: "sync", + requiresHuman: false, + }); + continue; + } + if (name === ASK_HUMAN_TOOL) { + tools.push(ASK_HUMAN_DESCRIPTOR); + continue; + } + const builtin = this.builtins[attachment.name]; + if (!builtin) { + continue; + } + if (builtin.isAvailable && !(await builtin.isAvailable())) { + continue; + } + tools.push({ + toolId: `builtin:${attachment.name}`, + name, + description: builtin.description, + inputSchema: toJsonSchema(builtin.inputSchema), + execution: "sync", + requiresHuman: false, + }); + } + return tools; + } +} + +function toJsonSchema(schema: unknown): JsonValue { + try { + return toJsonValue(z.toJSONSchema(schema as z.ZodType)) ?? { + type: "object", + properties: {}, + }; + } catch { + // An exotic zod schema must not break the whole turn. + return { type: "object", properties: {} }; + } +} + +function toJsonValue(value: unknown): JsonValue | undefined { + try { + return JSON.parse(JSON.stringify(value)) as JsonValue; + } catch { + return undefined; + } +} diff --git a/apps/x/packages/core/src/turns/bridges/real-model-registry.test.ts b/apps/x/packages/core/src/turns/bridges/real-model-registry.test.ts new file mode 100644 index 00000000..a8eb7a2c --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/real-model-registry.test.ts @@ -0,0 +1,178 @@ +import { describe, expect, it } from "vitest"; +import type { LanguageModel } from "ai"; +import type { LlmStreamEvent, ModelStreamRequest } from "../model-registry.js"; +import { RealModelRegistry, type StreamTextInvoker } from "./real-model-registry.js"; + +type InvokerOptions = Parameters[0]; + +function makeRegistry(parts: Array>, capture: InvokerOptions[]) { + const fakeModel = { modelId: "gpt-test" } as unknown as LanguageModel; + return new RealModelRegistry({ + resolveProvider: async () => ({ flavor: "openai" }), + createProviderImpl: (() => ({ + languageModel: () => fakeModel, + })) as never, + invoke: (options) => { + capture.push(options); + return { + fullStream: (async function* () { + yield* parts; + })(), + }; + }, + }); +} + +function request(overrides: Partial = {}): ModelStreamRequest { + return { + systemPrompt: "SYS", + messages: [{ role: "user", content: "hello" }], + tools: [ + { + toolId: "builtin:echo", + name: "echo", + description: "Echo", + inputSchema: { type: "object", properties: {} }, + execution: "sync", + requiresHuman: false, + }, + ], + parameters: {}, + signal: new AbortController().signal, + ...overrides, + }; +} + +async function collect(registry: RealModelRegistry, req: ModelStreamRequest) { + const model = await registry.resolve({ provider: "openai", model: "gpt-test" }); + const events: LlmStreamEvent[] = []; + for await (const event of model.stream(req)) { + events.push(event); + } + return events; +} + +describe("RealModelRegistry", () => { + it("normalizes one streamText step into deltas, step events, and a completed message", async () => { + const capture: InvokerOptions[] = []; + const registry = makeRegistry( + [ + { type: "start" }, + { type: "text-start" }, + { type: "text-delta", text: "Hel" }, + { type: "text-delta", text: "lo" }, + { type: "text-end" }, + { type: "tool-call", toolCallId: "tc1", toolName: "echo", input: { x: 1 } }, + { + type: "finish-step", + finishReason: "tool-calls", + usage: { inputTokens: 7, outputTokens: 3, totalTokens: 10 }, + providerMetadata: { openai: { cached: true } }, + }, + ], + capture, + ); + const events = await collect(registry, request()); + + expect(events.map((e) => e.type)).toEqual([ + "step_event", // text_start + "text_delta", + "text_delta", + "step_event", // text_end + "step_event", // tool_call + "step_event", // finish_step + "completed", + ]); + expect(events[3]).toEqual({ + type: "step_event", + event: { type: "text_end", text: "Hello" }, + }); + const completed = events[events.length - 1]; + expect(completed).toMatchObject({ + type: "completed", + finishReason: "tool-calls", + usage: { inputTokens: 7, outputTokens: 3, totalTokens: 10 }, + providerMetadata: { openai: { cached: true } }, + message: { + role: "assistant", + content: [ + { type: "text", text: "Hello" }, + { + type: "tool-call", + toolCallId: "tc1", + toolName: "echo", + arguments: { x: 1 }, + }, + ], + }, + }); + + // The invoker received the system prompt, converted messages, and tools. + expect(capture[0].system).toBe("SYS"); + expect(capture[0].messages[0]).toMatchObject({ role: "user" }); + expect(Object.keys(capture[0].tools)).toEqual(["echo"]); + }); + + it("accumulates reasoning separately and emits reasoning deltas", async () => { + const registry = makeRegistry( + [ + { type: "reasoning-start" }, + { type: "reasoning-delta", text: "thinking…" }, + { type: "reasoning-end" }, + { type: "text-start" }, + { type: "text-delta", text: "done" }, + { type: "text-end" }, + { type: "finish-step", finishReason: "stop", usage: {} }, + ], + [], + ); + const events = await collect(registry, request()); + expect(events.filter((e) => e.type === "reasoning_delta")).toHaveLength(1); + const completed = events[events.length - 1]; + expect( + completed.type === "completed" ? completed.message.content : undefined, + ).toEqual([ + { type: "reasoning", text: "thinking…" }, + { type: "text", text: "done" }, + ]); + }); + + it("throws on provider error parts (a model failure, not a completion)", async () => { + const registry = makeRegistry( + [{ type: "error", error: new Error("rate limited") }], + [], + ); + const model = await registry.resolve({ provider: "openai", model: "gpt-test" }); + await expect( + (async () => { + for await (const event of model.stream(request())) { + void event; + } + })(), + ).rejects.toThrowError("rate limited"); + }); + + it("stops promptly when the signal aborts mid-stream", async () => { + const controller = new AbortController(); + const registry = makeRegistry( + [ + { type: "text-delta", text: "a" }, + { type: "text-delta", text: "b" }, + ], + [], + ); + const model = await registry.resolve({ provider: "openai", model: "gpt-test" }); + const seen: string[] = []; + await expect( + (async () => { + for await (const event of model.stream( + request({ signal: controller.signal }), + )) { + seen.push(event.type); + controller.abort(); + } + })(), + ).rejects.toThrowError(); + expect(seen).toEqual(["text_delta"]); + }); +}); diff --git a/apps/x/packages/core/src/turns/bridges/real-model-registry.ts b/apps/x/packages/core/src/turns/bridges/real-model-registry.ts new file mode 100644 index 00000000..3ad9abf4 --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/real-model-registry.ts @@ -0,0 +1,255 @@ +import { + jsonSchema, + stepCountIs, + streamText, + tool, + type LanguageModel, + type ModelMessage, + type ToolSet, +} from "ai"; +import type { z } from "zod"; +import type { LlmProvider } from "@x/shared/dist/models.js"; +import type { AssistantContentPart } from "@x/shared/dist/message.js"; +import type { JsonValue, ModelDescriptor, TurnUsage } from "@x/shared/dist/turns.js"; +import { convertFromMessages } from "../../agents/runtime.js"; +import { resolveProviderConfig } from "../../models/defaults.js"; +import { createProvider } from "../../models/models.js"; +import type { + IModelRegistry, + LlmStreamEvent, + ModelStreamRequest, + ResolvedModel, +} from "../model-registry.js"; + +// Injectable seam over streamText so normalization is testable without a +// provider. The bridge always requests exactly one step. +export type StreamTextInvoker = (options: { + model: LanguageModel; + system: string; + messages: ModelMessage[]; + tools: ToolSet; + abortSignal: AbortSignal; +}) => { fullStream: AsyncIterable }; + +const defaultInvoker: StreamTextInvoker = (options) => + streamText({ ...options, stopWhen: stepCountIs(1) }); + +export interface RealModelRegistryDeps { + resolveProvider?: (name: string) => Promise>; + createProviderImpl?: typeof createProvider; + invoke?: StreamTextInvoker; +} + +// Bridges models.json provider configs to live AI SDK models and normalizes +// one streamText step into LlmStreamEvents. Tools are declared without +// execute: the turn loop harvests tool calls and runs them itself. +export class RealModelRegistry implements IModelRegistry { + private readonly resolveProvider: ( + name: string, + ) => Promise>; + private readonly createProviderImpl: typeof createProvider; + private readonly invoke: StreamTextInvoker; + + constructor(deps: RealModelRegistryDeps = {}) { + this.resolveProvider = deps.resolveProvider ?? resolveProviderConfig; + this.createProviderImpl = deps.createProviderImpl ?? createProvider; + this.invoke = deps.invoke ?? defaultInvoker; + } + + async resolve( + descriptor: z.infer, + ): Promise { + const providerConfig = await this.resolveProvider(descriptor.provider); + const provider = this.createProviderImpl(providerConfig); + const model = provider.languageModel(descriptor.model); + return { + descriptor, + stream: (request) => this.run(model, request), + }; + } + + private async *run( + model: LanguageModel, + request: ModelStreamRequest, + ): AsyncGenerator { + const tools: ToolSet = {}; + for (const descriptor of request.tools) { + tools[descriptor.name] = tool({ + ...(descriptor.description + ? { description: descriptor.description } + : {}), + inputSchema: jsonSchema( + (descriptor.inputSchema ?? { + type: "object", + properties: {}, + }) as Parameters[0], + ), + }); + } + + const result = this.invoke({ + model, + system: request.systemPrompt, + messages: convertFromMessages(request.messages), + tools, + abortSignal: request.signal, + }); + + const parts: Array> = []; + let textBuffer = ""; + let reasoningBuffer = ""; + let finishReason = "unknown"; + let usage: z.infer = {}; + let providerMetadata: JsonValue | undefined; + + for await (const raw of result.fullStream) { + request.signal.throwIfAborted(); + const event = raw as { + type: string; + text?: string; + toolCallId?: string; + toolName?: string; + input?: unknown; + finishReason?: string; + usage?: Record; + providerMetadata?: unknown; + error?: unknown; + }; + switch (event.type) { + case "text-start": + textBuffer = ""; + yield { type: "step_event", event: { type: "text_start" } }; + break; + case "text-delta": { + const delta = event.text ?? ""; + textBuffer += delta; + const last = parts[parts.length - 1]; + if (last?.type === "text") { + last.text += delta; + } else { + parts.push({ type: "text", text: delta }); + } + yield { type: "text_delta", delta }; + break; + } + case "text-end": + yield { + type: "step_event", + event: { type: "text_end", text: textBuffer }, + }; + break; + case "reasoning-start": + reasoningBuffer = ""; + yield { type: "step_event", event: { type: "reasoning_start" } }; + break; + case "reasoning-delta": { + const delta = event.text ?? ""; + reasoningBuffer += delta; + const last = parts[parts.length - 1]; + if (last?.type === "reasoning") { + last.text += delta; + } else { + parts.push({ type: "reasoning", text: delta }); + } + yield { type: "reasoning_delta", delta }; + break; + } + case "reasoning-end": + yield { + type: "step_event", + event: { type: "reasoning_end", text: reasoningBuffer }, + }; + break; + case "tool-call": { + const toolCall = { + type: "tool-call" as const, + toolCallId: String(event.toolCallId), + toolName: String(event.toolName), + arguments: event.input, + }; + parts.push(toolCall); + yield { type: "step_event", event: { type: "tool_call", toolCall } }; + break; + } + case "finish-step": { + finishReason = event.finishReason ?? "unknown"; + usage = mapUsage(event.usage); + providerMetadata = toJsonValue(event.providerMetadata); + yield { + type: "step_event", + event: { + type: "finish_step", + finishReason, + usage, + ...(providerMetadata === undefined + ? {} + : { providerMetadata }), + }, + }; + break; + } + case "error": + throw event.error instanceof Error + ? event.error + : new Error(formatStreamError(event.error)); + default: + break; + } + } + + yield { + type: "completed", + message: { + role: "assistant", + content: parts.length > 0 ? parts : "", + }, + finishReason, + usage, + ...(providerMetadata === undefined ? {} : { providerMetadata }), + }; + } +} + +function mapUsage( + usage: Record | undefined, +): z.infer { + const mapped: z.infer = {}; + if (!usage) { + return mapped; + } + for (const key of [ + "inputTokens", + "outputTokens", + "totalTokens", + "reasoningTokens", + "cachedInputTokens", + ] as const) { + const value = usage[key]; + if (typeof value === "number" && Number.isFinite(value)) { + mapped[key] = value; + } + } + return mapped; +} + +function toJsonValue(value: unknown): JsonValue | undefined { + if (value === undefined || value === null) { + return undefined; + } + try { + return JSON.parse(JSON.stringify(value)) as JsonValue; + } catch { + return undefined; + } +} + +function formatStreamError(error: unknown): string { + if (typeof error === "string") { + return error; + } + try { + return JSON.stringify(error); + } catch { + return String(error); + } +} diff --git a/apps/x/packages/core/src/turns/bridges/real-permission-checker.test.ts b/apps/x/packages/core/src/turns/bridges/real-permission-checker.test.ts new file mode 100644 index 00000000..2967b6ae --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/real-permission-checker.test.ts @@ -0,0 +1,72 @@ +import { describe, expect, it } from "vitest"; +import type { getToolPermissionMetadata } from "../../agents/runtime.js"; +import { RealPermissionChecker } from "./real-permission-checker.js"; + +type MetadataFn = typeof getToolPermissionMetadata; +type MetadataCall = { + toolCall: Parameters[0]; + attachment: Parameters[1]; +}; + +function makeChecker(result: Awaited> | Error) { + const calls: MetadataCall[] = []; + const checker = new RealPermissionChecker({ + getMetadata: (async (toolCall, attachment) => { + calls.push({ toolCall, attachment }); + if (result instanceof Error) { + throw result; + } + return result; + }) as MetadataFn, + }); + return { checker, calls }; +} + +const input = { + turnId: "turn-1", + toolCallId: "tc-1", + toolId: "builtin:executeCommand", + toolName: "executeCommand", + input: { command: "rm -rf /" }, +}; + +describe("RealPermissionChecker", () => { + it("gates builtins through getToolPermissionMetadata with empty session grants", async () => { + const metadata = { kind: "command" as const, commandNames: ["rm"] }; + const { checker, calls } = makeChecker(metadata); + const result = await checker.check(input); + expect(result).toEqual({ required: true, request: metadata }); + expect(calls[0].toolCall).toMatchObject({ + type: "tool-call", + toolCallId: "tc-1", + toolName: "executeCommand", + arguments: { command: "rm -rf /" }, + }); + expect(calls[0].attachment).toEqual({ + type: "builtin", + name: "executeCommand", + }); + }); + + it("returns not-required when metadata is null", async () => { + const { checker } = makeChecker(null); + expect(await checker.check(input)).toEqual({ required: false }); + }); + + it("never gates non-builtin tools", async () => { + const { checker, calls } = makeChecker(new Error("must not be called")); + expect( + await checker.check({ + ...input, + toolId: "mcp:kb:search", + toolName: "search", + }), + ).toEqual({ required: false }); + expect(calls).toHaveLength(0); + }); + + it("propagates metadata errors so the loop fails closed", async () => { + const { checker } = makeChecker(new Error("policy exploded")); + await expect(checker.check(input)).rejects.toThrowError("policy exploded"); + }); +}); diff --git a/apps/x/packages/core/src/turns/bridges/real-permission-checker.ts b/apps/x/packages/core/src/turns/bridges/real-permission-checker.ts new file mode 100644 index 00000000..50fec935 --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/real-permission-checker.ts @@ -0,0 +1,49 @@ +import type { JsonValue } from "@x/shared/dist/turns.js"; +import { getToolPermissionMetadata } from "../../agents/runtime.js"; +import type { + IPermissionChecker, + PermissionCheckAllowed, + PermissionCheckInput, + PermissionCheckRequired, +} from "../permission.js"; + +export interface RealPermissionCheckerDeps { + getMetadata?: typeof getToolPermissionMetadata; +} + +// Bridges the existing deterministic permission rules: only builtins are +// gated (executeCommand via the command allowlist, file tools via workspace +// boundaries and file-access grants). Session-scoped grants are deferred, so +// the session grant inputs are always empty. A thrown metadata error +// propagates: the turn loop fails closed on checker errors. +export class RealPermissionChecker implements IPermissionChecker { + private readonly getMetadata: typeof getToolPermissionMetadata; + + constructor(deps: RealPermissionCheckerDeps = {}) { + this.getMetadata = deps.getMetadata ?? getToolPermissionMetadata; + } + + async check( + input: PermissionCheckInput, + ): Promise { + if (!input.toolId.startsWith("builtin:")) { + return { required: false }; + } + const name = input.toolId.slice("builtin:".length); + const metadata = await this.getMetadata( + { + type: "tool-call", + toolCallId: input.toolCallId, + toolName: input.toolName, + arguments: input.input, + }, + { type: "builtin", name }, + new Set(), // session-scoped command grants: deferred + [], // session-scoped file grants: deferred + ); + if (!metadata) { + return { required: false }; + } + return { required: true, request: metadata as JsonValue }; + } +} diff --git a/apps/x/packages/core/src/turns/bridges/real-permission-classifier.test.ts b/apps/x/packages/core/src/turns/bridges/real-permission-classifier.test.ts new file mode 100644 index 00000000..23312fca --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/real-permission-classifier.test.ts @@ -0,0 +1,98 @@ +import { describe, expect, it } from "vitest"; +import type { classifyToolPermissions } from "../../security/auto-permission-classifier.js"; +import { RealPermissionClassifier } from "./real-permission-classifier.js"; + +type ClassifierFn = typeof classifyToolPermissions; +type ClassifierInput = Parameters[0]; + +function makeClassifier( + decisions: Awaited> | Error, +) { + const calls: ClassifierInput[] = []; + const classifier = new RealPermissionClassifier({ + classifier: (async (input) => { + calls.push(input); + if (decisions instanceof Error) { + throw decisions; + } + return decisions; + }) as ClassifierFn, + }); + return { classifier, calls }; +} + +const batch = { + turnId: "turn-1", + messages: [ + { role: "user" as const, content: "list my downloads" }, + { role: "assistant" as const, content: "sure" }, + ], + requests: [ + { + toolCallId: "tc-1", + toolName: "file-list", + input: { path: "/Users/me/Downloads" }, + request: { + kind: "file", + operation: "list", + paths: ["/Users/me/Downloads"], + pathPrefix: "/Users/me/Downloads", + }, + }, + ], +}; + +describe("RealPermissionClassifier", () => { + it("adapts the batch into classifyToolPermissions candidates with conversation context", async () => { + const { classifier, calls } = makeClassifier([ + { toolCallId: "tc-1", decision: "allow", reason: "user asked for it" }, + ]); + const decisions = await classifier.classify(batch); + + expect(decisions).toEqual([ + { toolCallId: "tc-1", decision: "allow", reason: "user asked for it" }, + ]); + expect(calls[0].runId).toBe("turn-1"); + expect(calls[0].useCase).toBe("copilot_chat"); + expect(calls[0].messages).toHaveLength(2); + expect(calls[0].candidates[0]).toMatchObject({ + toolCall: { + type: "tool-call", + toolCallId: "tc-1", + toolName: "file-list", + }, + permission: { kind: "file", operation: "list" }, + }); + }); + + it("returns an empty result for an empty batch without calling the LLM", async () => { + const { classifier, calls } = makeClassifier(new Error("must not be called")); + expect( + await classifier.classify({ ...batch, requests: [] }), + ).toEqual([]); + expect(calls).toHaveLength(0); + }); + + it("omitted decisions surface as missing entries (loop treats them as defer)", async () => { + const { classifier } = makeClassifier([]); + const decisions = await classifier.classify(batch); + expect(decisions).toEqual([]); + }); + + it("propagates classifier failures (loop normalizes to defer)", async () => { + const { classifier } = makeClassifier(new Error("llm unavailable")); + await expect(classifier.classify(batch)).rejects.toThrowError( + "llm unavailable", + ); + }); + + it("rejects malformed permission metadata via schema parsing", async () => { + const { classifier } = makeClassifier([]); + await expect( + classifier.classify({ + ...batch, + requests: [{ ...batch.requests[0], request: { kind: "nonsense" } }], + }), + ).rejects.toThrowError(); + }); +}); diff --git a/apps/x/packages/core/src/turns/bridges/real-permission-classifier.ts b/apps/x/packages/core/src/turns/bridges/real-permission-classifier.ts new file mode 100644 index 00000000..1d5732d6 --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/real-permission-classifier.ts @@ -0,0 +1,57 @@ +import { ToolPermissionMetadata } from "@x/shared/dist/runs.js"; +import { convertFromMessages } from "../../agents/runtime.js"; +import type { UseCase } from "../../analytics/use_case.js"; +import { classifyToolPermissions } from "../../security/auto-permission-classifier.js"; +import type { + IPermissionClassifier, + PermissionClassification, + PermissionClassificationBatch, +} from "../permission.js"; + +export interface RealPermissionClassifierDeps { + classifier?: typeof classifyToolPermissions; + useCase?: UseCase; +} + +// Bridges the existing LLM auto-permission classifier. The underlying +// classifier only ever answers allow/deny; omitted decisions surface as +// missing entries, which the turn loop records as classification failures +// and treats as defer. A parse/LLM failure throws, which the loop likewise +// normalizes to defer for the whole batch. +export class RealPermissionClassifier implements IPermissionClassifier { + private readonly classifier: typeof classifyToolPermissions; + private readonly useCase: UseCase; + + constructor(deps: RealPermissionClassifierDeps = {}) { + this.classifier = deps.classifier ?? classifyToolPermissions; + this.useCase = deps.useCase ?? "copilot_chat"; + } + + async classify( + batch: PermissionClassificationBatch, + ): Promise { + if (batch.requests.length === 0) { + return []; + } + const decisions = await this.classifier({ + runId: batch.turnId, + agentName: null, + messages: convertFromMessages(batch.messages), + candidates: batch.requests.map((request) => ({ + toolCall: { + type: "tool-call", + toolCallId: request.toolCallId, + toolName: request.toolName, + arguments: request.input, + }, + permission: ToolPermissionMetadata.parse(request.request), + })), + useCase: this.useCase, + }); + return decisions.map((decision) => ({ + toolCallId: decision.toolCallId, + decision: decision.decision, + reason: decision.reason, + })); + } +} diff --git a/apps/x/packages/core/src/turns/bridges/real-tool-registry.test.ts b/apps/x/packages/core/src/turns/bridges/real-tool-registry.test.ts new file mode 100644 index 00000000..4fb6113e --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/real-tool-registry.test.ts @@ -0,0 +1,198 @@ +import { describe, expect, it } from "vitest"; +import type { z } from "zod"; +import type { ToolDescriptor } from "@x/shared/dist/turns.js"; +import type { execTool } from "../../application/lib/exec-tool.js"; +import type { BuiltinTools } from "../../application/lib/builtin-tools.js"; +import type { IAbortRegistry } from "../../runs/abort-registry.js"; +import { TurnDependencyError } from "../api.js"; +import type { SyncRuntimeTool, ToolExecutionContext } from "../tool-registry.js"; +import { RealToolRegistry } from "./real-tool-registry.js"; + +type ExecCall = { + attachment: Parameters[0]; + input: Record; + ctx: NonNullable[2]>; +}; + +class FakeAbortRegistry implements IAbortRegistry { + calls: string[] = []; + createForRun(runId: string): AbortSignal { + this.calls.push(`create:${runId}`); + return new AbortController().signal; + } + registerProcess(): void {} + unregisterProcess(): void {} + abort(runId: string): void { + this.calls.push(`abort:${runId}`); + } + forceAbort(): void {} + isAborted(): boolean { + return false; + } + cleanup(runId: string): void { + this.calls.push(`cleanup:${runId}`); + } +} + +const fakeBuiltins = { + echo: { description: "Echo", inputSchema: {}, execute: async () => null }, +} as unknown as typeof BuiltinTools; + +function descriptor( + overrides: Partial> = {}, +): z.infer { + return { + toolId: "builtin:echo", + name: "echo", + description: "Echo", + inputSchema: {}, + execution: "sync", + requiresHuman: false, + ...overrides, + }; +} + +function makeCtx(overrides: Partial = {}): ToolExecutionContext & { + progress: unknown[]; +} { + const progress: unknown[] = []; + return { + turnId: "turn-1", + toolCallId: "tc-1", + signal: new AbortController().signal, + reportProgress: async (p) => { + progress.push(p); + }, + progress, + ...overrides, + }; +} + +function makeRegistry(execImpl: (call: ExecCall) => Promise) { + const calls: ExecCall[] = []; + const abortRegistry = new FakeAbortRegistry(); + const registry = new RealToolRegistry({ + execToolImpl: (async (attachment, input, ctx) => { + const call = { attachment, input, ctx: ctx! }; + calls.push(call); + return execImpl(call); + }) as typeof execTool, + abortRegistry, + builtins: fakeBuiltins, + }); + return { registry, calls, abortRegistry }; +} + +describe("RealToolRegistry", () => { + it("executes builtins through execTool with a turn-scoped ToolContext", async () => { + const { registry, calls, abortRegistry } = makeRegistry(async () => ({ ok: 1 })); + const tool = (await registry.resolve(descriptor())) as SyncRuntimeTool; + const ctx = makeCtx(); + const result = await tool.execute({ text: "hi" }, ctx); + + expect(result).toEqual({ output: { ok: 1 }, isError: false }); + expect(calls[0].attachment).toEqual({ type: "builtin", name: "echo" }); + expect(calls[0].input).toEqual({ text: "hi" }); + expect(calls[0].ctx).toMatchObject({ runId: "turn-1", toolCallId: "tc-1" }); + // Abort registry bracketed per call. + expect(abortRegistry.calls).toEqual(["create:turn-1", "cleanup:turn-1"]); + }); + + it("normalizes undefined results to null and serializes objects", async () => { + const { registry } = makeRegistry(async () => undefined); + const tool = (await registry.resolve(descriptor())) as SyncRuntimeTool; + expect(await tool.execute({}, makeCtx())).toEqual({ + output: null, + isError: false, + }); + }); + + it("forwards tool-output-stream publishes as progress", async () => { + const { registry } = makeRegistry(async ({ ctx }) => { + await ctx.publish({ + runId: "turn-1", + type: "tool-output-stream", + toolCallId: "tc-1", + toolName: "echo", + output: "chunk-1", + subflow: [], + }); + return "done"; + }); + const tool = (await registry.resolve(descriptor())) as SyncRuntimeTool; + const ctx = makeCtx(); + await tool.execute({}, ctx); + expect(ctx.progress).toEqual([{ kind: "tool-output", chunk: "chunk-1" }]); + }); + + it("wires the abort signal to the registry's force-kill path", async () => { + const controller = new AbortController(); + const { registry, abortRegistry } = makeRegistry(async () => { + controller.abort(); + return "late"; + }); + const tool = (await registry.resolve(descriptor())) as SyncRuntimeTool; + await tool.execute({}, makeCtx({ signal: controller.signal })); + expect(abortRegistry.calls).toEqual([ + "create:turn-1", + "abort:turn-1", + "cleanup:turn-1", + ]); + }); + + it("lets tool errors propagate (the loop converts them to error results) and still cleans up", async () => { + const { registry, abortRegistry } = makeRegistry(async () => { + throw new Error("tool exploded"); + }); + const tool = (await registry.resolve(descriptor())) as SyncRuntimeTool; + await expect(tool.execute({}, makeCtx())).rejects.toThrowError("tool exploded"); + expect(abortRegistry.calls).toEqual(["create:turn-1", "cleanup:turn-1"]); + }); + + it("resolves mcp descriptors into mcp attachments (server:tool split on first colon)", async () => { + const { registry, calls } = makeRegistry(async () => "mcp result"); + const tool = (await registry.resolve( + descriptor({ + toolId: "mcp:kb:search:advanced", + name: "search:advanced", + description: "Search KB", + inputSchema: { type: "object" }, + }), + )) as SyncRuntimeTool; + await tool.execute({ q: "x" }, makeCtx()); + expect(calls[0].attachment).toEqual({ + type: "mcp", + name: "search:advanced", + mcpServerName: "kb", + description: "Search KB", + inputSchema: { type: "object" }, + }); + }); + + it("resolves ask-human as an async tool with no executor", async () => { + const { registry } = makeRegistry(async () => null); + const tool = await registry.resolve( + descriptor({ + toolId: "builtin:ask-human", + name: "ask-human", + execution: "async", + requiresHuman: true, + }), + ); + expect("execute" in tool).toBe(false); + expect(tool.descriptor.execution).toBe("async"); + }); + + it("rejects unknown builtins and malformed toolIds as dependency errors", async () => { + const { registry } = makeRegistry(async () => null); + await expect( + registry.resolve(descriptor({ toolId: "builtin:ghost", name: "ghost" })), + ).rejects.toThrowError(TurnDependencyError); + await expect( + registry.resolve(descriptor({ toolId: "mcp:onlyserver" })), + ).rejects.toThrowError(TurnDependencyError); + await expect( + registry.resolve(descriptor({ toolId: "weird:scheme" })), + ).rejects.toThrowError(TurnDependencyError); + }); +}); diff --git a/apps/x/packages/core/src/turns/bridges/real-tool-registry.ts b/apps/x/packages/core/src/turns/bridges/real-tool-registry.ts new file mode 100644 index 00000000..697978ab --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/real-tool-registry.ts @@ -0,0 +1,141 @@ +import type { z } from "zod"; +import type { ToolAttachment } from "@x/shared/dist/agent.js"; +import type { JsonValue, ToolDescriptor } from "@x/shared/dist/turns.js"; +import { execTool } from "../../application/lib/exec-tool.js"; +import { BuiltinTools } from "../../application/lib/builtin-tools.js"; +import { + type IAbortRegistry, + InMemoryAbortRegistry, +} from "../../runs/abort-registry.js"; +import { TurnDependencyError } from "../api.js"; +import type { + IToolRegistry, + RuntimeTool, + SyncRuntimeTool, + ToolExecutionContext, +} from "../tool-registry.js"; +import { ASK_HUMAN_TOOL } from "./real-agent-resolver.js"; + +export interface RealToolRegistryDeps { + execToolImpl?: typeof execTool; + abortRegistry?: IAbortRegistry; + builtins?: typeof BuiltinTools; +} + +// Bridges persisted tool descriptors to the existing dispatch: builtins via +// the BuiltinTools catalog, MCP tools via execTool's MCP path. toolId encodes +// the attachment: "builtin:" or "mcp::". ask-human is the +// async human-dependent tool with no in-process executor. +export class RealToolRegistry implements IToolRegistry { + private readonly execToolImpl: typeof execTool; + private readonly abortRegistry: IAbortRegistry; + private readonly builtins: typeof BuiltinTools; + + constructor(deps: RealToolRegistryDeps = {}) { + this.execToolImpl = deps.execToolImpl ?? execTool; + this.abortRegistry = deps.abortRegistry ?? new InMemoryAbortRegistry(); + this.builtins = deps.builtins ?? BuiltinTools; + } + + async resolve( + descriptor: z.infer, + ): Promise { + if (descriptor.toolId === `builtin:${ASK_HUMAN_TOOL}`) { + return { + descriptor: descriptor as { execution: "async" } & z.infer< + typeof ToolDescriptor + >, + }; + } + if (descriptor.toolId.startsWith("builtin:")) { + const name = descriptor.toolId.slice("builtin:".length); + const builtin = this.builtins[name]; + if (!builtin?.execute) { + throw new TurnDependencyError( + `no live builtin tool for ${descriptor.toolId}`, + ); + } + return this.syncTool(descriptor, { type: "builtin", name }); + } + if (descriptor.toolId.startsWith("mcp:")) { + const rest = descriptor.toolId.slice("mcp:".length); + const separator = rest.indexOf(":"); + if (separator <= 0 || separator === rest.length - 1) { + throw new TurnDependencyError( + `malformed mcp toolId: ${descriptor.toolId}`, + ); + } + return this.syncTool(descriptor, { + type: "mcp", + name: rest.slice(separator + 1), + mcpServerName: rest.slice(0, separator), + description: descriptor.description, + inputSchema: descriptor.inputSchema, + }); + } + throw new TurnDependencyError( + `unrecognized toolId scheme: ${descriptor.toolId}`, + ); + } + + private syncTool( + descriptor: z.infer, + attachment: z.infer, + ): SyncRuntimeTool { + return { + descriptor: descriptor as { execution: "sync" } & z.infer< + typeof ToolDescriptor + >, + execute: async (input, ctx: ToolExecutionContext) => { + // AbortSignal is the primary kill path; the abort registry is + // the secondary force-kill for spawned child processes, + // bracketed per call and keyed by turn. + this.abortRegistry.createForRun(ctx.turnId); + const onAbort = () => this.abortRegistry.abort(ctx.turnId); + ctx.signal.addEventListener("abort", onAbort, { once: true }); + try { + const value = await this.execToolImpl( + attachment, + asArgs(input), + { + runId: ctx.turnId, + toolCallId: ctx.toolCallId, + signal: ctx.signal, + abortRegistry: this.abortRegistry, + publish: async (event) => { + if (event.type === "tool-output-stream") { + await ctx.reportProgress({ + kind: "tool-output", + chunk: event.output, + }); + } + }, + }, + ); + return { + output: toJsonValue(value === undefined ? null : value), + isError: false, + }; + } finally { + ctx.signal.removeEventListener("abort", onAbort); + this.abortRegistry.cleanup(ctx.turnId); + } + }, + }; + } +} + +function asArgs(input: unknown): Record { + return input && typeof input === "object" + ? (input as Record) + : {}; +} + +function toJsonValue(value: unknown): JsonValue { + try { + const parsed: unknown = JSON.parse(JSON.stringify(value)); + return parsed === undefined ? null : (parsed as JsonValue); + } catch { + return String(value); + } +} diff --git a/apps/x/packages/core/src/turns/bus.ts b/apps/x/packages/core/src/turns/bus.ts index c998e6a1..2a6b0fe4 100644 --- a/apps/x/packages/core/src/turns/bus.ts +++ b/apps/x/packages/core/src/turns/bus.ts @@ -16,3 +16,24 @@ export type TurnLifecycleEvent = TurnProcessingStart | TurnProcessingEnd; export interface ITurnLifecycleBus { publish(event: TurnLifecycleEvent): void; } + +// Default in-process fan-out. Observers must never affect turn semantics, so +// listener errors are swallowed. +export class EmitterTurnLifecycleBus implements ITurnLifecycleBus { + private listeners = new Set<(event: TurnLifecycleEvent) => void>(); + + publish(event: TurnLifecycleEvent): void { + for (const listener of this.listeners) { + try { + listener(event); + } catch { + // observational only + } + } + } + + subscribe(listener: (event: TurnLifecycleEvent) => void): () => void { + this.listeners.add(listener); + return () => this.listeners.delete(listener); + } +} diff --git a/apps/x/packages/core/src/turns/permission.ts b/apps/x/packages/core/src/turns/permission.ts index 7f386623..c6cc9022 100644 --- a/apps/x/packages/core/src/turns/permission.ts +++ b/apps/x/packages/core/src/turns/permission.ts @@ -1,4 +1,5 @@ -import type { JsonValue } from "@x/shared/dist/turns.js"; +import type { z } from "zod"; +import type { ConversationMessage, JsonValue } from "@x/shared/dist/turns.js"; export interface PermissionCheckInput { turnId: string; @@ -41,12 +42,20 @@ export interface PermissionClassification { reason: string; } +export interface PermissionClassificationBatch { + turnId: string; + // Conversation context for the classifier: the turn's resolved context + // plus current-turn settled messages. + messages: Array>; + requests: PermissionClassificationInput[]; +} + // Handles all permission-required calls from one model response in one batch // when automatic permission is enabled. Internal model calls are opaque to // the turn loop. Failures and omitted decisions normalize to defer. export interface IPermissionClassifier { classify( - requests: PermissionClassificationInput[], + batch: PermissionClassificationBatch, signal: AbortSignal, ): Promise; } diff --git a/apps/x/packages/core/src/turns/runtime.test.ts b/apps/x/packages/core/src/turns/runtime.test.ts index b9019b60..dc21b55d 100644 --- a/apps/x/packages/core/src/turns/runtime.test.ts +++ b/apps/x/packages/core/src/turns/runtime.test.ts @@ -25,6 +25,7 @@ import type { IPermissionClassifier, PermissionCheckInput, PermissionClassification, + PermissionClassificationBatch, PermissionClassificationInput, } from "./permission.js"; import { TurnRuntime } from "./runtime.js"; @@ -221,6 +222,7 @@ class FakePermissionChecker implements IPermissionChecker { class FakePermissionClassifier implements IPermissionClassifier { batches: PermissionClassificationInput[][] = []; + contexts: Array<{ turnId: string; messageCount: number }> = []; constructor( private readonly impl?: ( @@ -230,16 +232,20 @@ class FakePermissionClassifier implements IPermissionClassifier { ) {} async classify( - requests: PermissionClassificationInput[], + batch: PermissionClassificationBatch, ): Promise { - this.batches.push(requests); + this.batches.push(batch.requests); + this.contexts.push({ + turnId: batch.turnId, + messageCount: batch.messages.length, + }); if (this.throws) { throw new Error(this.throws); } if (!this.impl) { throw new Error("classifier must not be called in this test"); } - return this.impl(requests); + return this.impl(batch.requests); } } @@ -302,7 +308,7 @@ function makeRuntime(opts: { contextResolver: new TurnRepoContextResolver({ turnRepo: repo }), permissionChecker: checker, permissionClassifier: classifier, - bus, + lifecycleBus: bus, }); return { runtime, repo, models, checker, classifier, bus }; } @@ -668,6 +674,11 @@ describe("automatic permission classification (26.4)", () => { "CD", "CF", ]); + // Conversation context reaches the classifier: input + batch response. + expect(classifier.contexts[0]).toEqual({ + turnId, + messageCount: 2, + }); const log = await persisted(repo, turnId); // Classifier provenance and effective decisions are distinct records. diff --git a/apps/x/packages/core/src/turns/runtime.ts b/apps/x/packages/core/src/turns/runtime.ts index d5352832..f8682f1c 100644 --- a/apps/x/packages/core/src/turns/runtime.ts +++ b/apps/x/packages/core/src/turns/runtime.ts @@ -63,7 +63,7 @@ export interface TurnRuntimeDependencies { contextResolver: IContextResolver; permissionChecker: IPermissionChecker; permissionClassifier: IPermissionClassifier; - bus: ITurnLifecycleBus; + lifecycleBus: ITurnLifecycleBus; } // Immutable dependency container: holds no mutable per-turn state. All active @@ -78,7 +78,7 @@ export class TurnRuntime implements ITurnRuntime { private readonly contextResolver: IContextResolver; private readonly permissionChecker: IPermissionChecker; private readonly permissionClassifier: IPermissionClassifier; - private readonly bus: ITurnLifecycleBus; + private readonly lifecycleBus: ITurnLifecycleBus; constructor({ turnRepo, @@ -90,7 +90,7 @@ export class TurnRuntime implements ITurnRuntime { contextResolver, permissionChecker, permissionClassifier, - bus, + lifecycleBus, }: TurnRuntimeDependencies) { this.turnRepo = turnRepo; this.idGenerator = idGenerator; @@ -101,7 +101,7 @@ export class TurnRuntime implements ITurnRuntime { this.contextResolver = contextResolver; this.permissionChecker = permissionChecker; this.permissionClassifier = permissionClassifier; - this.bus = bus; + this.lifecycleBus = lifecycleBus; } async createTurn(input: CreateTurnInput): Promise { @@ -154,11 +154,11 @@ export class TurnRuntime implements ITurnRuntime { externalSignal: AbortSignal | undefined, stream: HotStream, ): Promise { - this.bus.publish({ type: "turn-processing-start", turnId }); + this.lifecycleBus.publish({ type: "turn-processing-start", turnId }); try { return await this.advance(turnId, input, externalSignal, stream); } finally { - this.bus.publish({ type: "turn-processing-end", turnId }); + this.lifecycleBus.publish({ type: "turn-processing-end", turnId }); } } @@ -551,6 +551,21 @@ class TurnAdvance { } } + // Conversation context for the classifier: resolved context, the turn + // input, and completed assistant responses (the current batch's tool + // results are not yet terminal, so tool messages are omitted). + private conversationSoFar(): Array> { + const messages: Array> = [ + this.state.definition.input, + ]; + for (const call of this.state.modelCalls) { + if (call.response !== undefined) { + messages.push(call.response); + } + } + return [...this.resolvedContext, ...messages]; + } + // §9.3: one classifier batch per model response in auto mode. // Checker-error calls and previously failed classifications skip the // classifier and go straight to the human/deny fallback. @@ -573,13 +588,17 @@ class TurnAdvance { let decisions; try { decisions = await this.permissionClassifier.classify( - candidates.map((tc) => ({ - toolCallId: tc.toolCallId, - toolName: tc.toolName, - input: tc.input, - request: (tc.permission as NonNullable) - .required.request, - })), + { + turnId: this.turnId, + messages: this.conversationSoFar(), + requests: candidates.map((tc) => ({ + toolCallId: tc.toolCallId, + toolName: tc.toolName, + input: tc.input, + request: (tc.permission as NonNullable) + .required.request, + })), + }, this.signal, ); } catch (error) { @@ -708,6 +727,8 @@ class TurnAdvance { const syncTool = tool as SyncRuntimeTool; try { const result = await syncTool.execute(tc.input, { + turnId: this.turnId, + toolCallId: tc.toolCallId, signal: this.signal, reportProgress: async (progress) => { await this.append({ diff --git a/apps/x/packages/core/src/turns/tool-registry.ts b/apps/x/packages/core/src/turns/tool-registry.ts index 82670e72..3d9c09b7 100644 --- a/apps/x/packages/core/src/turns/tool-registry.ts +++ b/apps/x/packages/core/src/turns/tool-registry.ts @@ -6,6 +6,8 @@ import type { } from "@x/shared/dist/turns.js"; export interface ToolExecutionContext { + turnId: string; + toolCallId: string; signal: AbortSignal; // The loop appends a durable tool_progress event before resolving. reportProgress(progress: JsonValue): Promise; diff --git a/apps/x/packages/shared/src/turns.ts b/apps/x/packages/shared/src/turns.ts index f8b87c5b..b93ff3ec 100644 --- a/apps/x/packages/shared/src/turns.ts +++ b/apps/x/packages/shared/src/turns.ts @@ -32,6 +32,12 @@ export const RequestedAgent = z.object({ overrides: z .object({ model: ModelDescriptor.optional(), + // Opaque composition hints interpreted by the agent resolver + // (e.g. work-dir id, voice/search/code modes). Persisted verbatim + // for audit. The resolver decides which keys affect the system + // prompt; keeping prompt-affecting inputs session-sticky is what + // preserves provider prefix caching across turns. + composition: z.json().optional(), }) .optional(), }); From df607ea510bbbf494fe94b123020890d5901e64b Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 11:10:17 +0530 Subject: [PATCH 09/24] feat(x): sessions IPC + renderer data layer with testable stores (stage 5a) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The wire and state layer for the UI cutover; App.tsx integration follows separately. - shared/ipc.ts: eleven sessions:* invoke channels + the sessions:events push feed (SessionBusEvent via z.custom, like runs:events). The generic preload bridge needs no changes. - main: sessions:* handlers as thin pass-throughs to the DI'd sessions service; startSessionsWatcher forwards the session bus to all windows; startup awaits the session-index scan before the renderer can list. - renderer architecture per review guidance — all logic in framework- agnostic, dependency-injected modules; hooks are thin useSyncExternalStore subscriptions; components will consume pre-digested view models: - client.ts: narrow SessionsClient over window.ipc (fakeable). - feed.ts: one shared sessions:events consumer with fan-out (factory for tests). - turn-view.ts: pure derivations — live overlay (deltas accumulate, canonical events clear), TurnState -> ConversationItem[], and the session chat state (permission/ask-human maps re-manufactured in the runs-era shapes so existing components render unchanged; isProcessing/isThinking contract preserved). - store.ts: SessionChatStore (seed via getSession/getTurn, shared reduceTurn over live events, prior-turn freezing, unknown-turn reconciliation, stale-load guard, action routing) + SessionListStore. - hooks/useSessionChat + useSessions: thin wrappers, deps injectable. - renderer test infra added from scratch (vitest + jsdom + testing-library); 29 tests across stores, pure views, feed, and hooks. Co-Authored-By: Claude Fable 5 --- apps/x/apps/main/src/ipc.ts | 66 ++++ apps/x/apps/main/src/main.ts | 8 +- apps/x/apps/renderer/package.json | 11 +- .../src/hooks/useSessionChat.test.tsx | 118 +++++++ .../apps/renderer/src/hooks/useSessionChat.ts | 36 +++ apps/x/apps/renderer/src/hooks/useSessions.ts | 32 ++ .../renderer/src/lib/session-chat/client.ts | 70 ++++ .../src/lib/session-chat/feed.test.ts | 52 +++ .../renderer/src/lib/session-chat/feed.ts | 42 +++ .../src/lib/session-chat/store.test.ts | 283 +++++++++++++++++ .../renderer/src/lib/session-chat/store.ts | 297 +++++++++++++++++ .../src/lib/session-chat/test-fixtures.ts | 169 ++++++++++ .../src/lib/session-chat/turn-view.test.ts | 272 ++++++++++++++++ .../src/lib/session-chat/turn-view.ts | 300 ++++++++++++++++++ apps/x/apps/renderer/src/test/setup.ts | 1 + apps/x/apps/renderer/vite.config.ts | 8 +- apps/x/packages/shared/src/ipc.ts | 80 ++++- 17 files changed, 1840 insertions(+), 5 deletions(-) create mode 100644 apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx create mode 100644 apps/x/apps/renderer/src/hooks/useSessionChat.ts create mode 100644 apps/x/apps/renderer/src/hooks/useSessions.ts create mode 100644 apps/x/apps/renderer/src/lib/session-chat/client.ts create mode 100644 apps/x/apps/renderer/src/lib/session-chat/feed.test.ts create mode 100644 apps/x/apps/renderer/src/lib/session-chat/feed.ts create mode 100644 apps/x/apps/renderer/src/lib/session-chat/store.test.ts create mode 100644 apps/x/apps/renderer/src/lib/session-chat/store.ts create mode 100644 apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts create mode 100644 apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts create mode 100644 apps/x/apps/renderer/src/lib/session-chat/turn-view.ts create mode 100644 apps/x/apps/renderer/src/test/setup.ts diff --git a/apps/x/apps/main/src/ipc.ts b/apps/x/apps/main/src/ipc.ts index 9665172a..c290486e 100644 --- a/apps/x/apps/main/src/ipc.ts +++ b/apps/x/apps/main/src/ipc.ts @@ -24,6 +24,8 @@ const execFileAsync = promisify(execFile); import { RunEvent } from '@x/shared/dist/runs.js'; import { ServiceEvent } from '@x/shared/dist/service-events.js'; +import type { SessionBusEvent } from '@x/shared/dist/sessions.js'; +import type { ISessions, EmitterSessionBus } from '@x/core/dist/sessions/index.js'; import container from '@x/core/dist/di/container.js'; import { listOnboardingModels } from '@x/core/dist/models/models-dev.js'; import { testModelConnection, listModelsForProvider, generateOneShot } from '@x/core/dist/models/models.js'; @@ -634,6 +636,25 @@ export async function startRunsWatcher(): Promise { }); } +// New runtime: session bus → renderer windows (session-design.md §10). +function emitSessionEvent(event: SessionBusEvent): void { + const windows = BrowserWindow.getAllWindows(); + for (const win of windows) { + if (!win.isDestroyed() && win.webContents) { + win.webContents.send('sessions:events', event); + } + } +} + +let sessionsWatcher: (() => void) | null = null; +export function startSessionsWatcher(): void { + if (sessionsWatcher) { + return; + } + const sessionBus = container.resolve('sessionBus'); + sessionsWatcher = sessionBus.subscribe((event) => emitSessionEvent(event)); +} + let servicesWatcher: (() => void) | null = null; export async function startServicesWatcher(): Promise { if (servicesWatcher) { @@ -841,6 +862,51 @@ export function setupIpcHandlers() { await runsCore.deleteRun(args.runId); return { success: true }; }, + // ── New runtime: sessions + turns ───────────────────────── + // Thin pass-throughs to the sessions service. sendMessage returns the + // turnId immediately; the turn advances in the background and the + // renderer reconciles via the sessions:events feed. Input-routing calls + // settle with that advance's outcome (the renderer fire-and-forgets). + 'sessions:create': async (_event, args) => { + const sessionId = await container.resolve('sessions').createSession(args); + return { sessionId }; + }, + 'sessions:list': async () => { + return { sessions: container.resolve('sessions').listSessions() }; + }, + 'sessions:get': async (_event, args) => { + return container.resolve('sessions').getSession(args.sessionId); + }, + 'sessions:getTurn': async (_event, args) => { + return container.resolve('sessions').getTurn(args.turnId); + }, + 'sessions:sendMessage': async (_event, args) => { + return container.resolve('sessions').sendMessage(args.sessionId, args.input, args.config); + }, + 'sessions:respondToPermission': async (_event, args) => { + await container.resolve('sessions').respondToPermission(args.turnId, args.toolCallId, args.decision, args.metadata); + return { success: true }; + }, + 'sessions:respondToAskHuman': async (_event, args) => { + await container.resolve('sessions').respondToAskHuman(args.turnId, args.toolCallId, args.answer); + return { success: true }; + }, + 'sessions:stopTurn': async (_event, args) => { + await container.resolve('sessions').stopTurn(args.turnId, args.reason); + return { success: true }; + }, + 'sessions:resumeTurn': async (_event, args) => { + await container.resolve('sessions').resumeTurn(args.sessionId); + return { success: true }; + }, + 'sessions:setTitle': async (_event, args) => { + await container.resolve('sessions').setTitle(args.sessionId, args.title); + return { success: true }; + }, + 'sessions:delete': async (_event, args) => { + await container.resolve('sessions').deleteSession(args.sessionId); + return { success: true }; + }, 'runs:downloadLog': async (event, args) => { const runFileName = `${args.runId}.jsonl`; if (path.basename(runFileName) !== runFileName) { diff --git a/apps/x/apps/main/src/main.ts b/apps/x/apps/main/src/main.ts index e93d8ac5..fb033413 100644 --- a/apps/x/apps/main/src/main.ts +++ b/apps/x/apps/main/src/main.ts @@ -2,7 +2,7 @@ import { app, BrowserWindow, desktopCapturer, protocol, net, shell, session, typ import path from "node:path"; import { setupIpcHandlers, - startRunsWatcher, + startRunsWatcher, startSessionsWatcher, startCodeSessionStatusWatcher, startServicesWatcher, startLiveNoteAgentWatcher, @@ -45,6 +45,7 @@ import { execFileSync } from "node:child_process"; import { init as initChromeSync } from "@x/core/dist/knowledge/chrome-extension/server/server.js"; import container, { registerBrowserControlService, registerNotificationService } from "@x/core/dist/di/container.js"; import type { CodeModeManager } from "@x/core/dist/code-mode/acp/manager.js"; +import type { ISessions } from "@x/core/dist/sessions/index.js"; import { browserViewManager, BROWSER_PARTITION } from "./browser/view.js"; import { setupBrowserEventForwarding } from "./browser/ipc.js"; import { ElectronBrowserControlService } from "./browser/control-service.js"; @@ -388,6 +389,11 @@ app.whenReady().then(async () => { // start runs watcher startRunsWatcher(); + // New runtime: build the in-memory session index (startup scan) before the + // renderer can list sessions, then forward the session bus to windows. + await container.resolve('sessions').initialize(); + startSessionsWatcher(); + // start code-session status tracker (derives working/needs-you/idle + notifications) startCodeSessionStatusWatcher(); diff --git a/apps/x/apps/renderer/package.json b/apps/x/apps/renderer/package.json index eec078d6..0a8c9f9f 100644 --- a/apps/x/apps/renderer/package.json +++ b/apps/x/apps/renderer/package.json @@ -6,7 +6,9 @@ "dev": "vite", "build": "tsc -b && vite build", "lint": "eslint .", - "preview": "vite preview" + "preview": "vite preview", + "test": "vitest run", + "test:watch": "vitest" }, "dependencies": { "@codemirror/language": "^6.12.3", @@ -83,6 +85,9 @@ }, "devDependencies": { "@eslint/js": "^9.39.1", + "@testing-library/dom": "^10.4.1", + "@testing-library/jest-dom": "^6.9.1", + "@testing-library/react": "^16.3.2", "@types/node": "^24.10.1", "@types/react": "^19.2.5", "@types/react-dom": "^19.2.3", @@ -91,9 +96,11 @@ "eslint-plugin-react-hooks": "^7.0.1", "eslint-plugin-react-refresh": "^0.4.24", "globals": "^16.5.0", + "jsdom": "^29.1.1", "tw-animate-css": "^1.4.0", "typescript": "~5.9.3", "typescript-eslint": "^8.46.4", - "vite": "^7.2.4" + "vite": "^7.2.4", + "vitest": "catalog:" } } diff --git a/apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx b/apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx new file mode 100644 index 00000000..30caf6a9 --- /dev/null +++ b/apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx @@ -0,0 +1,118 @@ +import { act, renderHook, waitFor } from '@testing-library/react' +import { describe, expect, it } from 'vitest' +import type { SessionBusEvent } from '@x/shared/src/sessions.js' +import type { SessionsClient } from '@/lib/session-chat/client' +import type { SessionFeedListener } from '@/lib/session-chat/feed' +import { + assistantText, + completed, + completedTurnLog, + created, + requested, + sessionState, + turnCompleted, + user, +} from '@/lib/session-chat/test-fixtures' +import { isChatMessage } from '@/lib/chat-conversation' +import { useSessionChat } from './useSessionChat' + +const S1 = 'sess-1' + +function makeDeps() { + const calls: Array<{ method: string; args: unknown[] }> = [] + let emit: SessionFeedListener = () => undefined + let unsubscribed = 0 + const sessions = new Map([[S1, sessionState(S1, ['turn-1'])]]) + const turns = new Map([['turn-1', completedTurnLog('turn-1', S1, 'q1', 'a1')]]) + const client: SessionsClient = { + create: async () => ({ sessionId: 'x' }), + list: async () => ({ sessions: [] }), + get: async (sessionId) => { + const state = sessions.get(sessionId) + if (!state) throw new Error('session not found') + return state + }, + getTurn: async (turnId) => ({ turnId, events: turns.get(turnId) ?? [] }), + sendMessage: async (...args) => { + calls.push({ method: 'sendMessage', args }) + return { turnId: 'turn-2' } + }, + respondToPermission: async (...args) => { + calls.push({ method: 'respondToPermission', args }) + }, + respondToAskHuman: async (...args) => { + calls.push({ method: 'respondToAskHuman', args }) + }, + stopTurn: async (...args) => { + calls.push({ method: 'stopTurn', args }) + }, + resumeTurn: async () => undefined, + setTitle: async () => undefined, + delete: async () => undefined, + } + return { + deps: { + client, + subscribeFeed: (listener: SessionFeedListener) => { + emit = listener + return () => { + unsubscribed += 1 + } + }, + }, + calls, + emit: (event: SessionBusEvent) => emit(event), + getUnsubscribed: () => unsubscribed, + } +} + +describe('useSessionChat', () => { + it('seeds from the session, follows live events, and routes actions', async () => { + const { deps, calls, emit } = makeDeps() + const { result } = renderHook(() => useSessionChat(S1, deps)) + + await waitFor(() => { + expect(result.current.latestTurnId).toBe('turn-1') + }) + expect( + result.current.chatState?.conversation.filter(isChatMessage).map((m) => m.content), + ).toEqual(['q1', 'a1']) + + // A new turn streams in over the feed. + act(() => { + emit({ kind: 'turn-event', sessionId: S1, turnId: 'turn-2', event: created('turn-2', S1, user('q2')) }) + emit({ kind: 'turn-event', sessionId: S1, turnId: 'turn-2', event: requested('turn-2', 0, [user('q2')]) }) + emit({ + kind: 'turn-event', + sessionId: S1, + turnId: 'turn-2', + event: { type: 'text_delta', turnId: 'turn-2', modelCallIndex: 0, delta: 'a2…' }, + }) + }) + expect(result.current.latestTurnId).toBe('turn-2') + expect(result.current.chatState?.currentAssistantMessage).toBe('a2…') + expect(result.current.chatState?.isProcessing).toBe(true) + + act(() => { + emit({ kind: 'turn-event', sessionId: S1, turnId: 'turn-2', event: completed('turn-2', 0, assistantText('a2')) }) + emit({ kind: 'turn-event', sessionId: S1, turnId: 'turn-2', event: turnCompleted('turn-2', 'a2') }) + }) + expect(result.current.chatState?.isProcessing).toBe(false) + + await act(async () => { + await result.current.respondToPermission('tc1', 'deny') + await result.current.stop() + }) + expect(calls).toEqual([ + { method: 'respondToPermission', args: ['turn-2', 'tc1', 'deny', undefined] }, + { method: 'stopTurn', args: ['turn-2'] }, + ]) + }) + + it('unsubscribes from the feed on unmount', async () => { + const { deps, getUnsubscribed } = makeDeps() + const { unmount } = renderHook(() => useSessionChat(S1, deps)) + unmount() + expect(getUnsubscribed()).toBe(1) + }) +}) diff --git a/apps/x/apps/renderer/src/hooks/useSessionChat.ts b/apps/x/apps/renderer/src/hooks/useSessionChat.ts new file mode 100644 index 00000000..c598330c --- /dev/null +++ b/apps/x/apps/renderer/src/hooks/useSessionChat.ts @@ -0,0 +1,36 @@ +import { useEffect, useMemo, useState, useSyncExternalStore } from 'react' +import { ipcSessionsClient } from '@/lib/session-chat/client' +import { subscribeSessionFeed } from '@/lib/session-chat/feed' +import { SessionChatStore, type SessionChatStoreDeps } from '@/lib/session-chat/store' + +const defaultDeps: SessionChatStoreDeps = { + client: ipcSessionsClient, + subscribeFeed: subscribeSessionFeed, +} + +// Thin subscription over SessionChatStore — all logic (seeding, feed events, +// reducer, overlay, action routing) lives in the store, which is unit-tested +// without React. `deps` is injectable for tests. +export function useSessionChat( + sessionId: string | null, + deps: SessionChatStoreDeps = defaultDeps, +) { + const [store] = useState(() => new SessionChatStore(deps)) + useEffect(() => () => store.dispose(), [store]) + useEffect(() => { + void store.setSession(sessionId) + }, [store, sessionId]) + const snapshot = useSyncExternalStore(store.subscribe, store.getSnapshot) + return useMemo( + () => ({ + ...snapshot, + sendMessage: store.sendMessage, + respondToPermission: store.respondToPermission, + answerAskHuman: store.answerAskHuman, + stop: store.stop, + }), + [snapshot, store], + ) +} + +export type SessionChat = ReturnType diff --git a/apps/x/apps/renderer/src/hooks/useSessions.ts b/apps/x/apps/renderer/src/hooks/useSessions.ts new file mode 100644 index 00000000..872c6a6c --- /dev/null +++ b/apps/x/apps/renderer/src/hooks/useSessions.ts @@ -0,0 +1,32 @@ +import { useEffect, useMemo, useState, useSyncExternalStore } from 'react' +import { ipcSessionsClient } from '@/lib/session-chat/client' +import { subscribeSessionFeed } from '@/lib/session-chat/feed' +import { SessionListStore, type SessionChatStoreDeps } from '@/lib/session-chat/store' + +const defaultDeps: SessionChatStoreDeps = { + client: ipcSessionsClient, + subscribeFeed: subscribeSessionFeed, +} + +// The session list (chat history sidebar): seeded from sessions:list, kept +// current by index-changed feed events. Logic lives in SessionListStore. +export function useSessions(deps: SessionChatStoreDeps = defaultDeps) { + const [store] = useState(() => new SessionListStore(deps)) + useEffect(() => { + void store.load() + return () => store.dispose() + }, [store]) + const snapshot = useSyncExternalStore(store.subscribe, store.getSnapshot) + const client = deps.client + return useMemo( + () => ({ + ...snapshot, + createSession: (input: { title?: string } = {}) => client.create(input), + deleteSession: (sessionId: string) => client.delete(sessionId), + setTitle: (sessionId: string, title: string) => client.setTitle(sessionId, title), + }), + [snapshot, client], + ) +} + +export type Sessions = ReturnType diff --git a/apps/x/apps/renderer/src/lib/session-chat/client.ts b/apps/x/apps/renderer/src/lib/session-chat/client.ts new file mode 100644 index 00000000..87df44f0 --- /dev/null +++ b/apps/x/apps/renderer/src/lib/session-chat/client.ts @@ -0,0 +1,70 @@ +import type { z } from 'zod' +import type { UserMessage } from '@x/shared/src/message.js' +import type { SessionIndexEntry, SessionState } from '@x/shared/src/sessions.js' +import type { JsonValue, RequestedAgent, TurnEvent } from '@x/shared/src/turns.js' + +// Narrow, injectable surface over the sessions IPC channels so stores are +// testable with a plain fake instead of a window.ipc stub. +export interface SendMessageConfig { + agent: z.infer + autoPermission?: boolean + maxModelCalls?: number +} + +export interface SessionsClient { + create(input: { title?: string }): Promise<{ sessionId: string }> + list(): Promise<{ sessions: SessionIndexEntry[] }> + get(sessionId: string): Promise + getTurn(turnId: string): Promise<{ turnId: string; events: Array> }> + sendMessage( + sessionId: string, + input: z.infer, + config: SendMessageConfig, + ): Promise<{ turnId: string }> + respondToPermission( + turnId: string, + toolCallId: string, + decision: 'allow' | 'deny', + metadata?: JsonValue, + ): Promise + respondToAskHuman(turnId: string, toolCallId: string, answer: string): Promise + stopTurn(turnId: string, reason?: string): Promise + resumeTurn(sessionId: string): Promise + setTitle(sessionId: string, title: string): Promise + delete(sessionId: string): Promise +} + +export const ipcSessionsClient: SessionsClient = { + create: (input) => window.ipc.invoke('sessions:create', input), + list: () => window.ipc.invoke('sessions:list', {}), + get: (sessionId) => window.ipc.invoke('sessions:get', { sessionId }), + getTurn: (turnId) => window.ipc.invoke('sessions:getTurn', { turnId }), + sendMessage: (sessionId, input, config) => + window.ipc.invoke('sessions:sendMessage', { sessionId, input, config }), + respondToPermission: async (turnId, toolCallId, decision, metadata) => { + await window.ipc.invoke('sessions:respondToPermission', { + turnId, + toolCallId, + decision, + ...(metadata === undefined ? {} : { metadata }), + }) + }, + respondToAskHuman: async (turnId, toolCallId, answer) => { + await window.ipc.invoke('sessions:respondToAskHuman', { turnId, toolCallId, answer }) + }, + stopTurn: async (turnId, reason) => { + await window.ipc.invoke('sessions:stopTurn', { + turnId, + ...(reason === undefined ? {} : { reason }), + }) + }, + resumeTurn: async (sessionId) => { + await window.ipc.invoke('sessions:resumeTurn', { sessionId }) + }, + setTitle: async (sessionId, title) => { + await window.ipc.invoke('sessions:setTitle', { sessionId, title }) + }, + delete: async (sessionId) => { + await window.ipc.invoke('sessions:delete', { sessionId }) + }, +} diff --git a/apps/x/apps/renderer/src/lib/session-chat/feed.test.ts b/apps/x/apps/renderer/src/lib/session-chat/feed.test.ts new file mode 100644 index 00000000..86d00bbd --- /dev/null +++ b/apps/x/apps/renderer/src/lib/session-chat/feed.test.ts @@ -0,0 +1,52 @@ +import { describe, expect, it } from 'vitest' +import type { SessionBusEvent } from '@x/shared/src/sessions.js' +import { createSessionFeed, type SessionFeedListener } from './feed' + +const event: SessionBusEvent = { + kind: 'index-changed', + sessionId: 's1', + entry: null, +} + +describe('createSessionFeed', () => { + it('starts the source lazily on first subscribe and fans events out', () => { + let sourceStarted = 0 + let push: SessionFeedListener = () => undefined + const feed = createSessionFeed((listener) => { + sourceStarted += 1 + push = listener + return () => undefined + }) + expect(sourceStarted).toBe(0) + + const seenA: SessionBusEvent[] = [] + const seenB: SessionBusEvent[] = [] + feed.subscribe((e) => seenA.push(e)) + feed.subscribe((e) => seenB.push(e)) + expect(sourceStarted).toBe(1) // one shared IPC listener + + push(event) + expect(seenA).toEqual([event]) + expect(seenB).toEqual([event]) + }) + + it('unsubscribes cleanly and isolates a throwing listener', () => { + let push: SessionFeedListener = () => undefined + const feed = createSessionFeed((listener) => { + push = listener + return () => undefined + }) + const seen: SessionBusEvent[] = [] + feed.subscribe(() => { + throw new Error('bad subscriber') + }) + const unsubscribe = feed.subscribe((e) => seen.push(e)) + + push(event) + expect(seen).toEqual([event]) + + unsubscribe() + push(event) + expect(seen).toEqual([event]) + }) +}) diff --git a/apps/x/apps/renderer/src/lib/session-chat/feed.ts b/apps/x/apps/renderer/src/lib/session-chat/feed.ts new file mode 100644 index 00000000..bac8a735 --- /dev/null +++ b/apps/x/apps/renderer/src/lib/session-chat/feed.ts @@ -0,0 +1,42 @@ +import type { SessionBusEvent } from '@x/shared/src/sessions.js' + +export type SessionFeedListener = (event: SessionBusEvent) => void +export type SessionFeedSource = (listener: SessionFeedListener) => () => void + +// One shared consumer of the sessions:events push channel; stores tap this +// fan-out instead of each opening their own IPC listener. Factory so tests +// can drive a fake source. +export function createSessionFeed(source: SessionFeedSource) { + const listeners = new Set() + let detach: (() => void) | null = null + + const ensureStarted = () => { + if (detach) return + detach = source((event) => { + // Copy so (un)subscribing during dispatch is safe. + for (const listener of [...listeners]) { + try { + listener(event) + } catch { + // A misbehaving subscriber must never break the feed. + } + } + }) + } + + return { + subscribe(listener: SessionFeedListener): () => void { + ensureStarted() + listeners.add(listener) + return () => { + listeners.delete(listener) + } + }, + } +} + +const appFeed = createSessionFeed((listener) => window.ipc.on('sessions:events', listener)) + +export function subscribeSessionFeed(listener: SessionFeedListener): () => void { + return appFeed.subscribe(listener) +} diff --git a/apps/x/apps/renderer/src/lib/session-chat/store.test.ts b/apps/x/apps/renderer/src/lib/session-chat/store.test.ts new file mode 100644 index 00000000..c8b95bcf --- /dev/null +++ b/apps/x/apps/renderer/src/lib/session-chat/store.test.ts @@ -0,0 +1,283 @@ +import { describe, expect, it } from 'vitest' +import type { SessionBusEvent, SessionState } from '@x/shared/src/sessions.js' +import { isChatMessage } from '@/lib/chat-conversation' +import type { SessionsClient } from './client' +import type { SessionFeedListener } from './feed' +import { SessionChatStore, SessionListStore } from './store' +import { + assistantText, + completed, + completedTurnLog, + created, + indexEntry, + requested, + sessionState, + turnCompleted, + user, + type TEvent, +} from './test-fixtures' + +const S1 = 'sess-1' + +class FakeClient implements SessionsClient { + sessions = new Map() + turns = new Map() + calls: Array<{ method: string; args: unknown[] }> = [] + // When set, get() defers until the returned resolver is invoked. + deferredGet: (() => void) | null = null + + private record(method: string, ...args: unknown[]) { + this.calls.push({ method, args }) + } + + async create(input: { title?: string }) { + this.record('create', input) + return { sessionId: 'new-session' } + } + async list() { + this.record('list') + return { sessions: [...this.sessions.keys()].map((id) => indexEntry(id)) } + } + async get(sessionId: string) { + this.record('get', sessionId) + if (this.deferredGet === null) { + const state = this.sessions.get(sessionId) + if (!state) throw new Error(`session not found: ${sessionId}`) + return state + } + return new Promise((resolve, reject) => { + this.deferredGet = () => { + const state = this.sessions.get(sessionId) + if (state) resolve(state) + else reject(new Error(`session not found: ${sessionId}`)) + } + }) + } + async getTurn(turnId: string) { + this.record('getTurn', turnId) + const events = this.turns.get(turnId) + if (!events) throw new Error(`turn not found: ${turnId}`) + return { turnId, events } + } + async sendMessage(sessionId: string, input: unknown, config: unknown) { + this.record('sendMessage', sessionId, input, config) + return { turnId: 'turn-next' } + } + async respondToPermission(...args: unknown[]) { + this.record('respondToPermission', ...args) + } + async respondToAskHuman(...args: unknown[]) { + this.record('respondToAskHuman', ...args) + } + async stopTurn(...args: unknown[]) { + this.record('stopTurn', ...args) + } + async resumeTurn(...args: unknown[]) { + this.record('resumeTurn', ...args) + } + async setTitle(...args: unknown[]) { + this.record('setTitle', ...args) + } + async delete(...args: unknown[]) { + this.record('delete', ...args) + } +} + +function makeStore() { + const client = new FakeClient() + let emit: SessionFeedListener = () => undefined + let unsubscribed = 0 + const store = new SessionChatStore({ + client, + subscribeFeed: (listener) => { + emit = listener + return () => { + unsubscribed += 1 + } + }, + }) + return { + client, + store, + emit: (event: SessionBusEvent) => emit(event), + getUnsubscribed: () => unsubscribed, + } +} + +function turnEvent( + sessionId: string, + turnId: string, + event: + | TEvent + | { type: 'text_delta' | 'reasoning_delta'; turnId: string; modelCallIndex: number; delta: string }, +): SessionBusEvent { + return { kind: 'turn-event', sessionId, turnId, event } +} + +describe('SessionChatStore', () => { + it('seeds a session: prior turns frozen, latest live, conversation composed', async () => { + const { client, store } = makeStore() + client.sessions.set(S1, sessionState(S1, ['turn-1', 'turn-2'])) + client.turns.set('turn-1', completedTurnLog('turn-1', S1, 'first?', 'first answer')) + client.turns.set('turn-2', completedTurnLog('turn-2', S1, 'second?', 'second answer')) + + await store.setSession(S1) + const snapshot = store.getSnapshot() + expect(snapshot.loading).toBe(false) + expect(snapshot.latestTurnId).toBe('turn-2') + expect( + snapshot.chatState?.conversation.filter(isChatMessage).map((m) => m.content), + ).toEqual(['first?', 'first answer', 'second?', 'second answer']) + expect(snapshot.chatState?.isProcessing).toBe(false) + }) + + it('applies live durable events through the shared reducer', async () => { + const { client, store, emit } = makeStore() + client.sessions.set(S1, sessionState(S1, [])) + await store.setSession(S1) + + emit(turnEvent(S1, 'turn-1', created('turn-1', S1, user('go')))) + emit(turnEvent(S1, 'turn-1', requested('turn-1', 0, [user('go')]))) + expect(store.getSnapshot().chatState?.isThinking).toBe(true) + + emit(turnEvent(S1, 'turn-1', completed('turn-1', 0, assistantText('done')))) + emit(turnEvent(S1, 'turn-1', turnCompleted('turn-1'))) + const snapshot = store.getSnapshot() + expect(snapshot.latestTurnId).toBe('turn-1') + expect(snapshot.chatState?.isProcessing).toBe(false) + expect( + snapshot.chatState?.conversation.filter(isChatMessage).map((m) => m.content), + ).toEqual(['go', 'done']) + }) + + it('accumulates text deltas and clears them on the canonical response', async () => { + const { client, store, emit } = makeStore() + client.sessions.set(S1, sessionState(S1, [])) + await store.setSession(S1) + emit(turnEvent(S1, 'turn-1', created('turn-1', S1, user('go')))) + emit(turnEvent(S1, 'turn-1', requested('turn-1', 0, [user('go')]))) + emit(turnEvent(S1, 'turn-1', { type: 'text_delta', turnId: 'turn-1', modelCallIndex: 0, delta: 'he' })) + emit(turnEvent(S1, 'turn-1', { type: 'text_delta', turnId: 'turn-1', modelCallIndex: 0, delta: 'y' })) + expect(store.getSnapshot().chatState?.currentAssistantMessage).toBe('hey') + + emit(turnEvent(S1, 'turn-1', completed('turn-1', 0, assistantText('hey')))) + expect(store.getSnapshot().chatState?.currentAssistantMessage).toBe('') + }) + + it('freezes the previous latest turn when a new turn starts', async () => { + const { client, store, emit } = makeStore() + client.sessions.set(S1, sessionState(S1, ['turn-1'])) + client.turns.set('turn-1', completedTurnLog('turn-1', S1, 'q1', 'a1')) + await store.setSession(S1) + + emit(turnEvent(S1, 'turn-2', created('turn-2', S1, user('q2')))) + const snapshot = store.getSnapshot() + expect(snapshot.latestTurnId).toBe('turn-2') + expect( + snapshot.chatState?.conversation.filter(isChatMessage).map((m) => m.content), + ).toEqual(['q1', 'a1', 'q2']) + }) + + it('ignores events for other sessions', async () => { + const { client, store, emit } = makeStore() + client.sessions.set(S1, sessionState(S1, [])) + await store.setSession(S1) + emit(turnEvent('other-session', 'turn-x', created('turn-x', 'other-session'))) + expect(store.getSnapshot().chatState?.conversation).toEqual([]) + }) + + it('reconciles an unknown mid-turn event by refetching the turn', async () => { + const { client, store, emit } = makeStore() + client.sessions.set(S1, sessionState(S1, [])) + await store.setSession(S1) + // The feed attached mid-turn: we never saw turn_created for turn-9. + client.turns.set('turn-9', [ + created('turn-9', S1, user('missed')), + requested('turn-9', 0, [user('missed')]), + ]) + emit(turnEvent(S1, 'turn-9', completed('turn-9', 0, assistantText('caught up')))) + await Promise.resolve() + await Promise.resolve() + expect(store.getSnapshot().latestTurnId).toBe('turn-9') + }) + + it('routes actions against the latest turn', async () => { + const { client, store, emit } = makeStore() + client.sessions.set(S1, sessionState(S1, [])) + await store.setSession(S1) + emit(turnEvent(S1, 'turn-1', created('turn-1', S1))) + + await store.sendMessage(user('next'), { agent: { agentId: 'copilot' } }) + await store.respondToPermission('tc1', 'allow') + await store.answerAskHuman('ah1', '42') + await store.stop() + + expect(client.calls.filter((c) => c.method !== 'get' && c.method !== 'getTurn')).toEqual([ + { method: 'sendMessage', args: [S1, user('next'), { agent: { agentId: 'copilot' } }] }, + { method: 'respondToPermission', args: ['turn-1', 'tc1', 'allow', undefined] }, + { method: 'respondToAskHuman', args: ['turn-1', 'ah1', '42'] }, + { method: 'stopTurn', args: ['turn-1'] }, + ]) + }) + + it('rejects sendMessage without an active session', async () => { + const { store } = makeStore() + await expect( + store.sendMessage(user('x'), { agent: { agentId: 'copilot' } }), + ).rejects.toThrowError('No active session') + }) + + it('drops stale loads after a session switch', async () => { + const { client, store } = makeStore() + client.sessions.set(S1, sessionState(S1, ['turn-1'])) + client.turns.set('turn-1', completedTurnLog('turn-1', S1, 'old', 'old answer')) + client.sessions.set('sess-2', sessionState('sess-2', [])) + + client.deferredGet = () => undefined // arm deferral + const first = store.setSession(S1) + const release = client.deferredGet + client.deferredGet = null + const second = store.setSession('sess-2') + release?.() // S1's get resolves after the switch + await Promise.all([first, second]) + + expect(store.getSnapshot().sessionId).toBe('sess-2') + expect(store.getSnapshot().chatState?.conversation).toEqual([]) + }) + + it('surfaces load errors and disposes its feed subscription', async () => { + const { store, getUnsubscribed } = makeStore() + await store.setSession('missing-session') + expect(store.getSnapshot().error).toMatch(/session not found/) + store.dispose() + expect(getUnsubscribed()).toBe(1) + }) +}) + +describe('SessionListStore', () => { + it('loads, applies index updates and deletions, and sorts by updatedAt', async () => { + const client = new FakeClient() + client.sessions.set('a', sessionState('a', [])) + client.sessions.set('b', sessionState('b', [])) + let emit: SessionFeedListener = () => undefined + const store = new SessionListStore({ + client, + subscribeFeed: (listener) => { + emit = listener + return () => undefined + }, + }) + await store.load() + expect(store.getSnapshot().sessions.map((s) => s.sessionId).sort()).toEqual(['a', 'b']) + + emit({ + kind: 'index-changed', + sessionId: 'c', + entry: indexEntry('c', { updatedAt: '2026-07-03T00:00:00Z' }), + }) + expect(store.getSnapshot().sessions[0].sessionId).toBe('c') + + emit({ kind: 'index-changed', sessionId: 'a', entry: null }) + expect(store.getSnapshot().sessions.map((s) => s.sessionId)).toEqual(['c', 'b']) + }) +}) diff --git a/apps/x/apps/renderer/src/lib/session-chat/store.ts b/apps/x/apps/renderer/src/lib/session-chat/store.ts new file mode 100644 index 00000000..8dc3b1f7 --- /dev/null +++ b/apps/x/apps/renderer/src/lib/session-chat/store.ts @@ -0,0 +1,297 @@ +import type { z } from 'zod' +import type { UserMessage } from '@x/shared/src/message.js' +import type { SessionBusEvent, SessionIndexEntry } from '@x/shared/src/sessions.js' +import { + reduceTurn, + type JsonValue, + type TurnEvent, + type TurnState, + type TurnStreamEvent, +} from '@x/shared/src/turns.js' +import type { SendMessageConfig, SessionsClient } from './client' +import type { SessionFeedListener } from './feed' +import { + applyOverlay, + buildSessionChatState, + emptyOverlay, + type LiveOverlay, + type SessionChatState, +} from './turn-view' + +type TEvent = z.infer + +export interface SessionChatSnapshot { + sessionId: string | null + chatState: SessionChatState | null + latestTurnId: string | null + loading: boolean + error: string | null +} + +export interface SessionChatStoreDeps { + client: SessionsClient + subscribeFeed: (listener: SessionFeedListener) => () => void +} + +// Framework-agnostic controller for one active session's chat. Owns all the +// logic (seeding via getSession/getTurn, applying live feed events with the +// shared reducer, the ephemeral overlay, action routing); the useSessionChat +// hook is a thin useSyncExternalStore subscription over it. +export class SessionChatStore { + private readonly client: SessionsClient + private readonly unsubscribeFeed: () => void + private readonly listeners = new Set<() => void>() + + private sessionId: string | null = null + // Settled earlier turns, reduced once and frozen. + private priorTurns: TurnState[] = [] + // The latest turn's raw event log; re-reduced on each durable event. + private latestEvents: TEvent[] | null = null + private overlay: LiveOverlay = emptyOverlay() + private loading = false + private error: string | null = null + // Guards stale async loads after a session switch. + private generation = 0 + + private snapshot: SessionChatSnapshot = { + sessionId: null, + chatState: null, + latestTurnId: null, + loading: false, + error: null, + } + + constructor(deps: SessionChatStoreDeps) { + this.client = deps.client + this.unsubscribeFeed = deps.subscribeFeed(this.onFeedEvent) + } + + dispose(): void { + this.unsubscribeFeed() + this.listeners.clear() + } + + subscribe = (onChange: () => void): (() => void) => { + this.listeners.add(onChange) + return () => { + this.listeners.delete(onChange) + } + } + + getSnapshot = (): SessionChatSnapshot => this.snapshot + + async setSession(sessionId: string | null): Promise { + if (sessionId === this.sessionId) return + this.generation += 1 + const generation = this.generation + this.sessionId = sessionId + this.priorTurns = [] + this.latestEvents = null + this.overlay = emptyOverlay() + this.error = null + this.loading = sessionId !== null + this.emit() + if (sessionId === null) return + + try { + const state = await this.client.get(sessionId) + const turns = await Promise.all( + state.turns.map((ref) => this.client.getTurn(ref.turnId)), + ) + if (generation !== this.generation) return + const reduced = turns.map((turn) => reduceTurn(turn.events)) + this.priorTurns = reduced.slice(0, -1) + this.latestEvents = turns.length > 0 ? turns[turns.length - 1].events : null + this.loading = false + this.emit() + } catch (error) { + if (generation !== this.generation) return + this.loading = false + this.error = error instanceof Error ? error.message : String(error) + this.emit() + } + } + + private onFeedEvent: SessionFeedListener = (event: SessionBusEvent) => { + if (event.kind !== 'turn-event' || event.sessionId !== this.sessionId) return + const turnEvent = event.event + if (isDurable(turnEvent)) { + if (turnEvent.type === 'turn_created') { + // A new turn started for this session: freeze the previous latest. + this.freezeLatest() + this.latestEvents = [turnEvent] + this.overlay = emptyOverlay() + } else if (this.latestEvents && this.latestEvents[0].turnId === turnEvent.turnId) { + this.latestEvents.push(turnEvent) + } else { + // An event for a turn we haven't seen (missed turn_created, e.g. the + // feed attached mid-turn): reconcile by refetching that turn. + void this.reloadTurn(event.turnId) + return + } + } + this.overlay = applyOverlay(this.overlay, turnEvent) + this.emit() + } + + private freezeLatest(): void { + if (!this.latestEvents) return + try { + this.priorTurns = [...this.priorTurns, reduceTurn(this.latestEvents)] + } catch { + // A turn we can't reduce is dropped from history rather than wedging + // the whole conversation. + } + this.latestEvents = null + } + + private async reloadTurn(turnId: string): Promise { + const generation = this.generation + try { + const turn = await this.client.getTurn(turnId) + if (generation !== this.generation) return + if (this.latestEvents && this.latestEvents[0].turnId !== turnId) { + this.freezeLatest() + } + this.latestEvents = turn.events + this.emit() + } catch { + // The next snapshot-worthy event will retry. + } + } + + // ── Actions ───────────────────────────────────────────────────────────── + + sendMessage = async ( + input: z.infer, + config: SendMessageConfig, + ): Promise<{ turnId: string }> => { + if (!this.sessionId) throw new Error('No active session') + return this.client.sendMessage(this.sessionId, input, config) + } + + respondToPermission = async ( + toolCallId: string, + decision: 'allow' | 'deny', + metadata?: JsonValue, + ): Promise => { + const turnId = this.snapshot.latestTurnId + if (!turnId) return + await this.client.respondToPermission(turnId, toolCallId, decision, metadata) + } + + answerAskHuman = async (toolCallId: string, answer: string): Promise => { + const turnId = this.snapshot.latestTurnId + if (!turnId) return + await this.client.respondToAskHuman(turnId, toolCallId, answer) + } + + stop = async (): Promise => { + const turnId = this.snapshot.latestTurnId + if (!turnId) return + await this.client.stopTurn(turnId) + } + + // ── Derivation ────────────────────────────────────────────────────────── + + private emit(): void { + this.snapshot = this.derive() + for (const listener of [...this.listeners]) { + listener() + } + } + + private derive(): SessionChatSnapshot { + let turns = this.priorTurns + let error = this.error + if (this.latestEvents) { + try { + turns = [...this.priorTurns, reduceTurn(this.latestEvents)] + } catch (reduceError) { + error = + reduceError instanceof Error ? reduceError.message : String(reduceError) + } + } + const latest = turns[turns.length - 1] + return { + sessionId: this.sessionId, + chatState: + this.sessionId !== null && !this.loading + ? buildSessionChatState(turns, this.overlay) + : null, + latestTurnId: latest?.definition.turnId ?? null, + loading: this.loading, + error, + } + } +} + +function isDurable(event: TurnStreamEvent): event is TEvent { + return event.type !== 'text_delta' && event.type !== 'reasoning_delta' +} + +// --------------------------------------------------------------------------- +// Session list store +// --------------------------------------------------------------------------- + +export interface SessionListSnapshot { + sessions: SessionIndexEntry[] + loading: boolean +} + +export class SessionListStore { + private readonly client: SessionsClient + private readonly unsubscribeFeed: () => void + private readonly listeners = new Set<() => void>() + private entries = new Map() + private loading = true + private snapshot: SessionListSnapshot = { sessions: [], loading: true } + + constructor(deps: SessionChatStoreDeps) { + this.client = deps.client + this.unsubscribeFeed = deps.subscribeFeed(this.onFeedEvent) + } + + dispose(): void { + this.unsubscribeFeed() + this.listeners.clear() + } + + subscribe = (onChange: () => void): (() => void) => { + this.listeners.add(onChange) + return () => { + this.listeners.delete(onChange) + } + } + + getSnapshot = (): SessionListSnapshot => this.snapshot + + async load(): Promise { + const { sessions } = await this.client.list() + this.entries = new Map(sessions.map((entry) => [entry.sessionId, entry])) + this.loading = false + this.emit() + } + + private onFeedEvent: SessionFeedListener = (event: SessionBusEvent) => { + if (event.kind !== 'index-changed') return + if (event.entry === null) { + this.entries.delete(event.sessionId) + } else { + this.entries.set(event.sessionId, event.entry) + } + this.emit() + } + + private emit(): void { + this.snapshot = { + sessions: [...this.entries.values()].sort((a, b) => + b.updatedAt.localeCompare(a.updatedAt), + ), + loading: this.loading, + } + for (const listener of [...this.listeners]) { + listener() + } + } +} diff --git a/apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts b/apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts new file mode 100644 index 00000000..381f8cb9 --- /dev/null +++ b/apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts @@ -0,0 +1,169 @@ +import type { z } from 'zod' +import type { SessionIndexEntry, SessionState } from '@x/shared/src/sessions.js' +import type { ResolvedAgent, TurnEvent } from '@x/shared/src/turns.js' + +// Compact turn-event builders for renderer tests. Sequences must satisfy the +// shared reducer's invariants (reduceTurn is what the store runs on them). + +export type TEvent = z.infer + +export const TS = '2026-07-02T10:00:00Z' + +export const FIXTURE_AGENT: z.infer = { + agentId: 'copilot', + systemPrompt: 'SYS', + model: { provider: 'openai', model: 'gpt-fixture' }, + tools: [], +} + +export function user(text: string) { + return { role: 'user' as const, content: text } +} + +export function assistantText(text: string) { + return { role: 'assistant' as const, content: text } +} + +export function toolCallPart(id: string, name: string, args: unknown = {}) { + return { type: 'tool-call' as const, toolCallId: id, toolName: name, arguments: args } +} + +export function created( + turnId: string, + sessionId: string, + input: ReturnType = user('hello'), +): TEvent { + return { + type: 'turn_created', + schemaVersion: 1, + turnId, + ts: TS, + sessionId, + agent: { requested: { agentId: 'copilot' }, resolved: FIXTURE_AGENT }, + context: [], + input, + config: { autoPermission: false, humanAvailable: true, maxModelCalls: 20 }, + } +} + +export function requested(turnId: string, index: number, messages: unknown[]): TEvent { + return { + type: 'model_call_requested', + turnId, + ts: TS, + modelCallIndex: index, + request: { + systemPrompt: 'SYS', + messages: messages as never, + tools: [], + parameters: {}, + }, + } +} + +export function completed( + turnId: string, + index: number, + message: { role: 'assistant'; content: unknown }, +): TEvent { + return { + type: 'model_call_completed', + turnId, + ts: TS, + modelCallIndex: index, + message: message as never, + finishReason: 'stop', + usage: {}, + } +} + +export function invocation(turnId: string, toolCallId: string, name: string): TEvent { + return { + type: 'tool_invocation_requested', + turnId, + ts: TS, + toolCallId, + toolId: `builtin:${name}`, + toolName: name, + execution: 'sync', + input: {}, + } +} + +export function toolResult( + turnId: string, + toolCallId: string, + name: string, + output: unknown = 'ok', +): TEvent { + return { + type: 'tool_result', + turnId, + ts: TS, + toolCallId, + toolName: name, + source: 'sync', + result: { output: output as never, isError: false }, + } +} + +export function turnCompleted(turnId: string, text = 'done'): TEvent { + return { + type: 'turn_completed', + turnId, + ts: TS, + output: assistantText(text), + finishReason: 'stop', + usage: {}, + } +} + +// A settled single-response turn. +export function completedTurnLog( + turnId: string, + sessionId: string, + question: string, + answer: string, +): TEvent[] { + return [ + created(turnId, sessionId, user(question)), + requested(turnId, 0, [user(question)]), + completed(turnId, 0, assistantText(answer)), + turnCompleted(turnId, answer), + ] +} + +export function indexEntry( + sessionId: string, + overrides: Partial = {}, +): SessionIndexEntry { + return { + sessionId, + title: `Session ${sessionId}`, + createdAt: TS, + updatedAt: TS, + turnCount: 1, + latestTurnStatus: 'completed', + ...overrides, + } +} + +export function sessionState( + sessionId: string, + turnIds: string[], +): SessionState { + return { + definition: { type: 'session_created', schemaVersion: 1, sessionId, ts: TS }, + title: `Session ${sessionId}`, + turns: turnIds.map((turnId, i) => ({ + turnId, + sessionSeq: i + 1, + agentId: 'copilot', + model: FIXTURE_AGENT.model, + ts: TS, + })), + latestTurnId: turnIds[turnIds.length - 1], + createdAt: TS, + updatedAt: TS, + } +} diff --git a/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts b/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts new file mode 100644 index 00000000..4acbd6e9 --- /dev/null +++ b/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts @@ -0,0 +1,272 @@ +import { describe, expect, it } from 'vitest' +import { reduceTurn } from '@x/shared/src/turns.js' +import { isChatMessage, isErrorMessage, isToolCall } from '@/lib/chat-conversation' +import { + applyOverlay, + buildSessionChatState, + buildTurnConversation, + emptyOverlay, +} from './turn-view' +import { + TS, + assistantText, + completed, + completedTurnLog, + created, + invocation, + requested, + toolCallPart, + toolResult, + turnCompleted, + user, + type TEvent, +} from './test-fixtures' + +const T1 = 'turn-1' +const S1 = 'sess-1' + +function assistantCalls(...parts: Array>) { + return { role: 'assistant' as const, content: parts } +} + +describe('applyOverlay', () => { + it('accumulates text and reasoning deltas', () => { + let overlay = emptyOverlay() + overlay = applyOverlay(overlay, { type: 'text_delta', turnId: T1, modelCallIndex: 0, delta: 'he' }) + overlay = applyOverlay(overlay, { type: 'text_delta', turnId: T1, modelCallIndex: 0, delta: 'y' }) + overlay = applyOverlay(overlay, { + type: 'reasoning_delta', + turnId: T1, + modelCallIndex: 0, + delta: 'hmm', + }) + expect(overlay.text).toBe('hey') + expect(overlay.reasoning).toBe('hmm') + }) + + it('clears text/reasoning when the canonical model response arrives', () => { + let overlay = { ...emptyOverlay(), text: 'streaming', reasoning: 'thinking' } + overlay = applyOverlay(overlay, completed(T1, 0, assistantText('final'))) + expect(overlay.text).toBe('') + expect(overlay.reasoning).toBe('') + }) + + it('accumulates tool-output progress chunks and drops them on the terminal result', () => { + let overlay = emptyOverlay() + const progress = (chunk: string): TEvent => ({ + type: 'tool_progress', + turnId: T1, + ts: TS, + toolCallId: 'tc1', + source: 'sync', + progress: { kind: 'tool-output', chunk }, + }) + overlay = applyOverlay(overlay, progress('$ ls\n')) + overlay = applyOverlay(overlay, progress('README.md\n')) + expect(overlay.toolOutput.tc1).toBe('$ ls\nREADME.md\n') + + overlay = applyOverlay(overlay, toolResult(T1, 'tc1', 'executeCommand')) + expect(overlay.toolOutput.tc1).toBeUndefined() + }) + + it('ignores non-output progress payloads', () => { + const overlay = applyOverlay(emptyOverlay(), { + type: 'tool_progress', + turnId: T1, + ts: TS, + toolCallId: 'tc1', + source: 'sync', + progress: { pct: 50 }, + }) + expect(overlay.toolOutput).toEqual({}) + }) +}) + +describe('buildTurnConversation', () => { + it('maps user input, assistant text, and settled tool calls', () => { + const call = assistantCalls(toolCallPart('tc1', 'echo', { x: 1 })) + const state = reduceTurn([ + created(T1, S1, user('run it')), + requested(T1, 0, [user('run it')]), + completed(T1, 0, call), + invocation(T1, 'tc1', 'echo'), + toolResult(T1, 'tc1', 'echo', { echoed: true }), + requested(T1, 1, [ + user('run it'), + call, + { role: 'tool', content: '{"echoed":true}', toolCallId: 'tc1', toolName: 'echo' }, + ]), + completed(T1, 1, assistantText('all done')), + turnCompleted(T1, 'all done'), + ]) + const items = buildTurnConversation(state) + expect(items.map((i) => (isToolCall(i) ? `tool:${i.name}:${i.status}` : isChatMessage(i) ? `${i.role}` : 'x'))).toEqual([ + 'user', + 'tool:echo:completed', + 'assistant', + ]) + const tool = items.find(isToolCall) + expect(tool?.result).toEqual({ echoed: true }) + expect(tool?.input).toEqual({ x: 1 }) + }) + + it('marks permission-pending tools pending and running tools running', () => { + const state = reduceTurn([ + created(T1, S1), + requested(T1, 0, [user('hello')]), + completed(T1, 0, assistantCalls(toolCallPart('p1', 'executeCommand'), toolCallPart('r1', 'echo'))), + { + type: 'tool_permission_required', + turnId: T1, + ts: TS, + toolCallId: 'p1', + toolName: 'executeCommand', + request: { kind: 'command', commandNames: ['rm'] }, + }, + invocation(T1, 'r1', 'echo'), + ]) + const items = buildTurnConversation(state).filter(isToolCall) + expect(items.map((i) => i.status)).toEqual(['pending', 'running']) + }) + + it('renders user attachments and a failed turn as an error item', () => { + const input = { + role: 'user' as const, + content: [ + { type: 'text' as const, text: 'see file' }, + { type: 'attachment' as const, path: '/tmp/a.png', filename: 'a.png', mimeType: 'image/png' }, + ], + } + const state = reduceTurn([ + created(T1, S1, input as never), + requested(T1, 0, [input]), + { type: 'model_call_failed', turnId: T1, ts: TS, modelCallIndex: 0, error: 'boom' }, + { type: 'turn_failed', turnId: T1, ts: TS, error: 'boom', usage: {} }, + ]) + const items = buildTurnConversation(state) + expect(isChatMessage(items[0]) && items[0].attachments?.[0].filename).toBe('a.png') + expect(isErrorMessage(items[1]) && items[1].message).toBe('boom') + }) +}) + +describe('buildSessionChatState', () => { + it('composes the conversation across turns and derives processing flags', () => { + const prior = reduceTurn(completedTurnLog('turn-1', S1, 'first?', 'first answer')) + const latest = reduceTurn([ + created('turn-2', S1, user('second?')), + requested('turn-2', 0, [user('second?')]), + ]) + const state = buildSessionChatState([prior, latest], emptyOverlay()) + expect( + state.conversation.filter(isChatMessage).map((m) => m.content), + ).toEqual(['first?', 'first answer', 'second?']) + expect(state.isProcessing).toBe(true) // latest turn idle = actively working + expect(state.isThinking).toBe(true) + }) + + it('is settled (not processing) when the latest turn is terminal', () => { + const turn = reduceTurn(completedTurnLog(T1, S1, 'q', 'a')) + const state = buildSessionChatState([turn], emptyOverlay()) + expect(state.isProcessing).toBe(false) + expect(state.isThinking).toBe(false) + }) + + it('exposes pending permissions as request events; waiting is not thinking', () => { + const turn = reduceTurn([ + created(T1, S1), + requested(T1, 0, [user('hello')]), + completed(T1, 0, assistantCalls(toolCallPart('p1', 'executeCommand', { command: 'rm -rf /' }))), + { + type: 'tool_permission_required', + turnId: T1, + ts: TS, + toolCallId: 'p1', + toolName: 'executeCommand', + request: { kind: 'command', commandNames: ['rm'] }, + }, + { + type: 'turn_suspended', + turnId: T1, + ts: TS, + pendingPermissions: [ + { toolCallId: 'p1', toolName: 'executeCommand', request: { kind: 'command', commandNames: ['rm'] } }, + ], + pendingAsyncTools: [], + usage: {}, + }, + ]) + const state = buildSessionChatState([turn], emptyOverlay()) + const request = state.allPermissionRequests.get('p1') + expect(request).toMatchObject({ + type: 'tool-permission-request', + toolCall: { toolCallId: 'p1', toolName: 'executeCommand' }, + permission: { kind: 'command', commandNames: ['rm'] }, + }) + expect(state.isProcessing).toBe(true) + expect(state.isThinking).toBe(false) + }) + + it('maps human decisions to responses and classifier decisions to auto-decisions', () => { + const turn = reduceTurn([ + created(T1, S1), + requested(T1, 0, [user('hello')]), + completed(T1, 0, assistantCalls(toolCallPart('h1', 'echo'), toolCallPart('c1', 'echo'))), + { type: 'tool_permission_required', turnId: T1, ts: TS, toolCallId: 'h1', toolName: 'echo', request: { kind: 'command', commandNames: ['x'] } }, + { type: 'tool_permission_required', turnId: T1, ts: TS, toolCallId: 'c1', toolName: 'echo', request: { kind: 'command', commandNames: ['y'] } }, + { type: 'tool_permission_resolved', turnId: T1, ts: TS, toolCallId: 'h1', decision: 'allow', source: 'human' }, + { type: 'tool_permission_resolved', turnId: T1, ts: TS, toolCallId: 'c1', decision: 'deny', source: 'classifier', reason: 'risky' }, + { type: 'tool_result', turnId: T1, ts: TS, toolCallId: 'c1', toolName: 'echo', source: 'runtime', result: { output: 'denied', isError: true } }, + invocation(T1, 'h1', 'echo'), + toolResult(T1, 'h1', 'echo'), + ]) + const state = buildSessionChatState([turn], emptyOverlay()) + expect(state.permissionResponses.get('h1')).toBe('approve') + expect(state.autoPermissionDecisions.get('c1')).toMatchObject({ + decision: 'deny', + reason: 'risky', + }) + }) + + it('exposes pending ask-human calls with question and options', () => { + const turn = reduceTurn([ + created(T1, S1), + requested(T1, 0, [user('hello')]), + completed(T1, 0, assistantCalls(toolCallPart('ah1', 'ask-human', { question: 'Deploy?', options: ['Yes', 'No'] }))), + { + type: 'tool_invocation_requested', + turnId: T1, + ts: TS, + toolCallId: 'ah1', + toolId: 'builtin:ask-human', + toolName: 'ask-human', + execution: 'async', + input: { question: 'Deploy?', options: ['Yes', 'No'] }, + }, + ]) + const state = buildSessionChatState([turn], emptyOverlay()) + expect(state.pendingAskHumanRequests.get('ah1')).toMatchObject({ + query: 'Deploy?', + options: ['Yes', 'No'], + }) + expect(state.isThinking).toBe(false) // waiting on the human, no shimmer + }) + + it('stitches live tool output onto the matching tool item', () => { + const turn = reduceTurn([ + created(T1, S1), + requested(T1, 0, [user('hello')]), + completed(T1, 0, assistantCalls(toolCallPart('tc1', 'executeCommand'))), + invocation(T1, 'tc1', 'executeCommand'), + ]) + const overlay = { ...emptyOverlay(), toolOutput: { tc1: 'partial output' } } + const state = buildSessionChatState([turn], overlay) + const tool = state.conversation.filter(isToolCall)[0] + expect(tool.streamingOutput).toBe('partial output') + }) + + it('surfaces streaming text as currentAssistantMessage', () => { + const turn = reduceTurn([created(T1, S1), requested(T1, 0, [user('hello')])]) + const state = buildSessionChatState([turn], { ...emptyOverlay(), text: 'typing…' }) + expect(state.currentAssistantMessage).toBe('typing…') + }) +}) diff --git a/apps/x/apps/renderer/src/lib/session-chat/turn-view.ts b/apps/x/apps/renderer/src/lib/session-chat/turn-view.ts new file mode 100644 index 00000000..fad07998 --- /dev/null +++ b/apps/x/apps/renderer/src/lib/session-chat/turn-view.ts @@ -0,0 +1,300 @@ +import type { z } from 'zod' +import type { + AskHumanRequestEvent, + ToolPermissionAutoDecisionEvent, + ToolPermissionMetadata, + ToolPermissionRequestEvent, +} from '@x/shared/src/runs.js' +import { + deriveTurnStatus, + outstandingAsyncTools, + outstandingPermissions, + type ToolCallState, + type TurnState, + type TurnStreamEvent, +} from '@x/shared/src/turns.js' +import type { + ChatMessage, + ConversationItem, + ErrorMessage, + MessageAttachment, + PermissionResponse, + ToolCall, +} from '@/lib/chat-conversation' + +// Pure derivations from reduced turn state (+ the ephemeral live overlay) to +// the view shapes the existing chat components already consume. No IPC, no +// React — everything here is unit-testable with plain state values. + +// --------------------------------------------------------------------------- +// Live overlay: ephemeral streaming buffers (turn-runtime-design.md §14.4). +// --------------------------------------------------------------------------- + +export type LiveOverlay = { + text: string + reasoning: string + toolOutput: Record +} + +export const emptyOverlay = (): LiveOverlay => ({ text: '', reasoning: '', toolOutput: {} }) + +// Accumulates deltas; canonical durable events supersede the buffers (the +// committed transcript now contains what was streaming). +export function applyOverlay(overlay: LiveOverlay, event: TurnStreamEvent): LiveOverlay { + switch (event.type) { + case 'text_delta': + return { ...overlay, text: overlay.text + event.delta } + case 'reasoning_delta': + return { ...overlay, reasoning: overlay.reasoning + event.delta } + case 'model_call_completed': + return { ...overlay, text: '', reasoning: '' } + case 'tool_progress': { + const progress = event.progress + if ( + progress && + typeof progress === 'object' && + !Array.isArray(progress) && + (progress as { kind?: unknown }).kind === 'tool-output' && + typeof (progress as { chunk?: unknown }).chunk === 'string' + ) { + const chunk = (progress as { chunk: string }).chunk + return { + ...overlay, + toolOutput: { + ...overlay.toolOutput, + [event.toolCallId]: (overlay.toolOutput[event.toolCallId] ?? '') + chunk, + }, + } + } + return overlay + } + case 'tool_result': { + if (!(event.toolCallId in overlay.toolOutput)) return overlay + const toolOutput = { ...overlay.toolOutput } + delete toolOutput[event.toolCallId] + return { ...overlay, toolOutput } + } + default: + return overlay + } +} + +// --------------------------------------------------------------------------- +// Conversation items +// --------------------------------------------------------------------------- + +type UserContent = TurnState['definition']['input']['content'] + +function extractText(content: UserContent): string { + if (typeof content === 'string') return content + return content + .map((part) => (part.type === 'text' ? part.text : '')) + .filter(Boolean) + .join('\n') +} + +function extractAttachments(content: UserContent): MessageAttachment[] | undefined { + if (typeof content === 'string') return undefined + const attachments = content.flatMap((part) => + part.type === 'attachment' + ? [ + { + path: part.path, + filename: part.filename, + mimeType: part.mimeType, + ...(part.size === undefined ? {} : { size: part.size }), + }, + ] + : [], + ) + return attachments.length > 0 ? attachments : undefined +} + +function toolStatus(tc: ToolCallState): ToolCall['status'] { + if (tc.result) return 'completed' + if (tc.permission && !tc.permission.resolved) return 'pending' + return 'running' +} + +// One turn's contribution to the conversation: the user input, then per +// completed model call its text and tool calls (with live status/results). +export function buildTurnConversation(state: TurnState): ConversationItem[] { + const items: ConversationItem[] = [] + const turnId = state.definition.turnId + let seq = 0 + const ts = () => Date.parse(state.definition.ts) + seq++ + + const userText = extractText(state.definition.input.content) + const attachments = extractAttachments(state.definition.input.content) + if (userText || attachments) { + items.push({ + id: `${turnId}:user`, + role: 'user', + content: userText, + timestamp: ts(), + ...(attachments ? { attachments } : {}), + } satisfies ChatMessage) + } + + const toolCallsById = new Map(state.toolCalls.map((tc) => [tc.toolCallId, tc])) + for (const call of state.modelCalls) { + if (call.response === undefined) continue + const content = call.response.content + const text = + typeof content === 'string' + ? content + : content + .map((part) => (part.type === 'text' ? part.text : '')) + .filter(Boolean) + .join('\n') + if (text) { + items.push({ + id: `${turnId}:a${call.index}`, + role: 'assistant', + content: text, + timestamp: ts(), + } satisfies ChatMessage) + } + if (Array.isArray(content)) { + for (const part of content) { + if (part.type !== 'tool-call') continue + const tc = toolCallsById.get(part.toolCallId) + items.push({ + id: part.toolCallId, + name: part.toolName, + input: part.arguments as ToolCall['input'], + ...(tc?.result ? { result: tc.result.result.output as ToolCall['result'] } : {}), + status: tc ? toolStatus(tc) : 'running', + timestamp: ts(), + } satisfies ToolCall) + } + } + } + + if (state.terminal?.type === 'turn_failed') { + items.push({ + id: `${turnId}:error`, + kind: 'error', + message: state.terminal.error, + timestamp: ts(), + } satisfies ErrorMessage) + } + + return items +} + +// --------------------------------------------------------------------------- +// Session chat state (superset of the ChatTabViewState fields) +// --------------------------------------------------------------------------- + +type PermMeta = z.infer + +export type SessionChatState = { + conversation: ConversationItem[] + currentAssistantMessage: string + pendingAskHumanRequests: Map> + allPermissionRequests: Map> + permissionResponses: Map + autoPermissionDecisions: Map> + // Composer blocked / Stop shown until the latest turn settles (waiting on a + // permission or ask-human still counts as processing). + isProcessing: boolean + // Actively working (non-terminal, nothing waiting on the user) — drives the + // "Thinking…" shimmer; deliberately false under permission/ask-human cards. + isThinking: boolean +} + +function toolCallPartOf(tc: ToolCallState) { + return { + type: 'tool-call' as const, + toolCallId: tc.toolCallId, + toolName: tc.toolName, + arguments: tc.input, + } +} + +// Compose the whole session's conversation: prior (settled) turns plus the +// latest turn, with the live overlay stitched onto the latest turn's items. +export function buildSessionChatState( + turns: TurnState[], + overlay: LiveOverlay, +): SessionChatState { + const conversation: ConversationItem[] = [] + for (const turn of turns) { + conversation.push(...buildTurnConversation(turn)) + } + for (let i = 0; i < conversation.length; i++) { + const item = conversation[i] + if ('name' in item && overlay.toolOutput[item.id]) { + conversation[i] = { ...item, streamingOutput: overlay.toolOutput[item.id] } + } + } + + const latest = turns[turns.length - 1] + const status = latest ? deriveTurnStatus(latest) : undefined + const latestTurnId = latest?.definition.turnId ?? '' + + const allPermissionRequests = new Map>() + const permissionResponses = new Map() + const autoPermissionDecisions = new Map< + string, + z.infer + >() + const pendingAskHumanRequests = new Map>() + + if (latest) { + for (const tc of outstandingPermissions(latest)) { + allPermissionRequests.set(tc.toolCallId, { + runId: latestTurnId, + type: 'tool-permission-request', + subflow: [], + toolCall: toolCallPartOf(tc), + permission: tc.permission?.required.request as PermMeta, + }) + } + for (const tc of latest.toolCalls) { + const resolved = tc.permission?.resolved + if (!resolved) continue + if (resolved.source === 'human') { + permissionResponses.set(tc.toolCallId, resolved.decision === 'allow' ? 'approve' : 'deny') + } else if (resolved.source === 'classifier') { + autoPermissionDecisions.set(tc.toolCallId, { + runId: latestTurnId, + type: 'tool-permission-auto-decision', + subflow: [], + toolCallId: tc.toolCallId, + toolCall: toolCallPartOf(tc), + permission: tc.permission?.required.request as PermMeta, + decision: resolved.decision, + reason: resolved.reason ?? '', + }) + } + } + for (const tc of outstandingAsyncTools(latest)) { + if (tc.toolName !== 'ask-human') continue + const input = (tc.input ?? {}) as { question?: unknown; options?: unknown } + pendingAskHumanRequests.set(tc.toolCallId, { + runId: latestTurnId, + type: 'ask-human-request', + toolCallId: tc.toolCallId, + subflow: [], + query: typeof input.question === 'string' ? input.question : '', + ...(Array.isArray(input.options) && input.options.every((o) => typeof o === 'string') + ? { options: input.options } + : {}), + }) + } + } + + const settled = status === 'completed' || status === 'failed' || status === 'cancelled' + return { + conversation, + currentAssistantMessage: overlay.text, + pendingAskHumanRequests, + allPermissionRequests, + permissionResponses, + autoPermissionDecisions, + isProcessing: latest !== undefined && !settled, + isThinking: status === 'idle', + } +} diff --git a/apps/x/apps/renderer/src/test/setup.ts b/apps/x/apps/renderer/src/test/setup.ts new file mode 100644 index 00000000..a9d0dd31 --- /dev/null +++ b/apps/x/apps/renderer/src/test/setup.ts @@ -0,0 +1 @@ +import '@testing-library/jest-dom/vitest' diff --git a/apps/x/apps/renderer/vite.config.ts b/apps/x/apps/renderer/vite.config.ts index 9bcca968..20b2b43b 100644 --- a/apps/x/apps/renderer/vite.config.ts +++ b/apps/x/apps/renderer/vite.config.ts @@ -1,5 +1,5 @@ import path from "path" -import { defineConfig } from 'vite' +import { defineConfig } from 'vitest/config' import react from '@vitejs/plugin-react' import tailwindcss from '@tailwindcss/vite' @@ -18,4 +18,10 @@ export default defineConfig({ build: { outDir: 'dist', }, + test: { + environment: 'jsdom', + setupFiles: ['./src/test/setup.ts'], + include: ['src/**/*.test.{ts,tsx}'], + css: false, + }, }) diff --git a/apps/x/packages/shared/src/ipc.ts b/apps/x/packages/shared/src/ipc.ts index 067adb4f..a174da9b 100644 --- a/apps/x/packages/shared/src/ipc.ts +++ b/apps/x/packages/shared/src/ipc.ts @@ -13,7 +13,9 @@ import { BackgroundTaskSummarySchema, TriggersSchema, } from './background-task.js'; -import { UserMessageContent } from './message.js'; +import { UserMessage, UserMessageContent } from './message.js'; +import { RequestedAgent, type TurnEvent } from './turns.js'; +import type { SessionBusEvent, SessionIndexEntry, SessionState } from './sessions.js'; import { RowboatApiConfig } from './rowboat-account.js'; import { ZListToolkitsResponse } from './composio.js'; import { BrowserStateSchema } from './browser-control.js'; @@ -375,6 +377,82 @@ const ipcSchemas = { req: z.null(), res: z.null(), }, + // ── New runtime: sessions + turns (session-design.md) ──────────────────── + // Turn-mutating calls return quickly; the renderer follows progress through + // the sessions:events feed and the shared reduceTurn reducer. + 'sessions:create': { + req: z.object({ title: z.string().optional() }), + res: z.object({ sessionId: z.string() }), + }, + 'sessions:list': { + req: z.object({}), + res: z.object({ sessions: z.array(z.custom()) }), + }, + 'sessions:get': { + req: z.object({ sessionId: z.string() }), + res: z.custom(), + }, + 'sessions:getTurn': { + // Events are strictly validated at the repository read; typed via + // z.custom to avoid re-validating potentially large logs per IPC hop. + req: z.object({ turnId: z.string() }), + res: z.custom<{ turnId: string; events: Array> }>(), + }, + 'sessions:sendMessage': { + req: z.object({ + sessionId: z.string(), + input: UserMessage, + config: z.object({ + agent: RequestedAgent, + autoPermission: z.boolean().optional(), + maxModelCalls: z.number().int().positive().optional(), + }), + }), + res: z.object({ turnId: z.string() }), + }, + 'sessions:respondToPermission': { + req: z.object({ + turnId: z.string(), + toolCallId: z.string(), + decision: z.enum(['allow', 'deny']), + metadata: z.json().optional(), + }), + res: z.object({ success: z.literal(true) }), + }, + 'sessions:respondToAskHuman': { + req: z.object({ + turnId: z.string(), + toolCallId: z.string(), + answer: z.string(), + }), + res: z.object({ success: z.literal(true) }), + }, + 'sessions:stopTurn': { + req: z.object({ + turnId: z.string(), + reason: z.string().optional(), + }), + res: z.object({ success: z.literal(true) }), + }, + 'sessions:resumeTurn': { + req: z.object({ sessionId: z.string() }), + res: z.object({ success: z.literal(true) }), + }, + 'sessions:setTitle': { + req: z.object({ sessionId: z.string(), title: z.string() }), + res: z.object({ success: z.literal(true) }), + }, + 'sessions:delete': { + req: z.object({ sessionId: z.string() }), + res: z.object({ success: z.literal(true) }), + }, + 'sessions:events': { + // Typed via z.custom so the renderer's `on` handler is typed without + // runtime validation (the broadcast path bypasses preload validation, + // like runs:events). + req: z.custom(), + res: z.null(), + }, 'services:events': { req: ServiceEvent, res: z.null(), From 63917df061a238c7e03fbc61f7e68d0a5bfca1bf Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 11:20:52 +0530 Subject: [PATCH 10/24] feat(x/renderer): cut the active chat over to the sessions runtime (stage 5b) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The active chat tab now renders from useSessionChat; background tabs keep the legacy cached view state until stage 7 retires it. - Send path: sessions:create + sessions:sendMessage with the structural UserMessage (userMessageContext carries datetime + middle-pane — the cache-safe per-message tier) and per-message turn config (model override + composition: workDirId/voice/search/codeMode, permissionMode -> autoPermission). Attachments/mentions ride as real content parts — the old 'as unknown as string' cast is gone. - Stop / permission / ask-human handlers route through the hook against the latest turn; permission scope is passed as audit metadata (session grants are a deferred v2 feature). Stop state clears when the turn settles. - Chat history lists sessions (sessions:list/delete); selecting one just switches the active session — the hook loads the conversation, so the 200-line run-log replay in loadRun is gone. - Work-dir persistence unchanged: the per-chat sidecar file is keyed by the session id, which the resolver reads via composition.workDirId. Renderer tsc/vite build clean; all 324 workspace tests pass; renderer lint identical to baseline (108 pre-existing findings, none in new code). Co-Authored-By: Claude Fable 5 --- apps/x/apps/renderer/src/App.tsx | 377 +++++++++---------------------- 1 file changed, 113 insertions(+), 264 deletions(-) diff --git a/apps/x/apps/renderer/src/App.tsx b/apps/x/apps/renderer/src/App.tsx index 9f635bde..61a5a471 100644 --- a/apps/x/apps/renderer/src/App.tsx +++ b/apps/x/apps/renderer/src/App.tsx @@ -1,7 +1,7 @@ import * as React from 'react' import { useCallback, useEffect, useLayoutEffect, useState, useRef } from 'react' import { workspace } from '@x/shared'; -import { RunEvent, ListRunsResponse } from '@x/shared/src/runs.js'; +import { RunEvent } from '@x/shared/src/runs.js'; import type { LanguageModelUsage, ToolUIPart } from 'ai'; import './App.css' import z from 'zod'; @@ -9,6 +9,7 @@ import { CheckIcon, LoaderIcon, PanelLeftIcon, ArrowLeft, ArrowRight, MessageSqu import { cn } from '@/lib/utils'; import { MarkdownEditor, type MarkdownEditorHandle } from './components/markdown-editor'; import { ChatSidebar } from './components/chat-sidebar'; +import { useSessionChat } from '@/hooks/useSessionChat'; import { ChatHeader } from './components/chat-header'; import { ChatEmptyState } from './components/chat-empty-state'; import { ChatInputWithMentions, type PermissionMode, type StagedAttachment } from './components/chat-input-with-mentions'; @@ -96,7 +97,6 @@ import { type ChatViewportAnchorState, type ChatTabViewState, type ConversationItem, - type ToolCall, createEmptyChatTabViewState, getWebSearchCardData, getAppActionCardData, @@ -126,7 +126,6 @@ import { useTheme } from '@/contexts/theme-context' type DirEntry = z.infer type RunEventType = z.infer -type ListRunsResponseType = z.infer interface TreeNode extends DirEntry { children?: TreeNode[] @@ -938,6 +937,10 @@ function App() { }, [conversation]) const [, setModelUsage] = useState(null) const [runId, setRunId] = useState(null) + // New runtime: the active session's chat data + actions. All logic lives in + // SessionChatStore (tested headlessly); the hook is a thin subscription. + // runId IS the session id in the sessions runtime. + const sessionChat = useSessionChat(runId) const runIdRef = useRef(null) const loadRunRequestIdRef = useRef(0) const [isProcessing, setIsProcessing] = useState(false) @@ -945,7 +948,7 @@ function App() { const processingRunIdsRef = useRef>(new Set()) const streamingBuffersRef = useRef>(new Map()) const [isStopping, setIsStopping] = useState(false) - const [stopClickedAt, setStopClickedAt] = useState(null) + const [, setStopClickedAt] = useState(null) const [agentId] = useState('copilot') const [presetMessage, setPresetMessage] = useState(undefined) @@ -1960,21 +1963,16 @@ function App() { // Load runs list (all pages) const loadRuns = useCallback(async () => { try { - const allRuns: RunListItem[] = [] - let cursor: string | undefined = undefined - - // Fetch all pages - do { - const result: ListRunsResponseType = await window.ipc.invoke('runs:list', { cursor }) - allRuns.push(...result.runs) - cursor = result.nextCursor - } while (cursor) - - // Filter for copilot chats only (Code-section sessions live in the Code view) - const copilotRuns = allRuns.filter((run: RunListItem) => run.agentId === 'copilot' && run.useCase !== 'code_session') - setRuns(copilotRuns) + const { sessions } = await window.ipc.invoke('sessions:list', {}) + setRuns(sessions.map((entry) => ({ + id: entry.sessionId, + title: entry.title ?? 'New chat', + createdAt: entry.createdAt, + modifiedAt: entry.updatedAt, + agentId: entry.lastAgentId ?? 'copilot', + }))) } catch (err) { - console.error('Failed to load runs:', err) + console.error('Failed to load sessions:', err) } }, []) @@ -2073,193 +2071,30 @@ function App() { } }, [backgroundTasks, loadBackgroundTasks]) - // Load a specific run and populate conversation + // Switch the active session. The useSessionChat hook loads and follows the + // conversation; this only resets composer/tab state. const loadRun = useCallback(async (id: string) => { const requestId = (loadRunRequestIdRef.current += 1) + setConversation([]) + setCurrentAssistantMessage('') + setRunId(id) + setMessage('') + setIsProcessing(false) + setIsStopping(false) + setStopClickedAt(null) + setPendingPermissionRequests(new Map()) + setPendingAskHumanRequests(new Map()) + setAllPermissionRequests(new Map()) + setPermissionResponses(new Map()) + setAutoPermissionDecisions(new Map()) try { - const run = await window.ipc.invoke('runs:fetch', { runId: id }) - if (loadRunRequestIdRef.current !== requestId) return - - // Parse the log events into conversation items - const items: ConversationItem[] = [] - const toolCallMap = new Map() - - for (const event of run.log) { - switch (event.type) { - case 'message': { - const msg = event.message - if (msg.role === 'user' || msg.role === 'assistant') { - // Extract text content from message - let textContent = '' - let msgAttachments: ChatMessage['attachments'] = undefined - if (typeof msg.content === 'string') { - textContent = msg.content - } else if (Array.isArray(msg.content)) { - const contentParts = msg.content as Array<{ - type: string - text?: string - path?: string - filename?: string - mimeType?: string - size?: number - toolCallId?: string - toolName?: string - arguments?: ToolUIPart['input'] - }> - - textContent = contentParts - .filter((part) => part.type === 'text') - .map((part) => part.text || '') - .join('') - - const attachmentParts = contentParts.filter((part) => part.type === 'attachment' && part.path) - if (attachmentParts.length > 0) { - msgAttachments = attachmentParts.map((part) => ({ - path: part.path!, - filename: part.filename || part.path!.split('/').pop() || part.path!, - mimeType: part.mimeType || 'application/octet-stream', - size: part.size, - })) - } - - // Also extract tool-call parts from assistant messages - if (msg.role === 'assistant') { - for (const part of contentParts) { - if (part.type === 'tool-call' && part.toolCallId && part.toolName) { - const toolCall: ToolCall = { - id: part.toolCallId, - name: part.toolName, - input: normalizeToolInput(part.arguments), - status: 'pending', - timestamp: event.ts ? new Date(event.ts).getTime() : Date.now(), - } - toolCallMap.set(toolCall.id, toolCall) - items.push(toolCall) - } - } - } - } - if (textContent || msgAttachments) { - items.push({ - id: event.messageId, - role: msg.role, - content: textContent, - attachments: msgAttachments, - timestamp: event.ts ? new Date(event.ts).getTime() : Date.now(), - }) - } - } - break - } - case 'tool-invocation': { - // Update existing tool call status or create new one - const existingTool = event.toolCallId ? toolCallMap.get(event.toolCallId) : null - if (existingTool) { - existingTool.input = normalizeToolInput(event.input) - existingTool.status = 'running' - } else { - const toolCall: ToolCall = { - id: event.toolCallId || `tool-${Date.now()}-${Math.random()}`, - name: event.toolName, - input: normalizeToolInput(event.input), - status: 'running', - timestamp: event.ts ? new Date(event.ts).getTime() : Date.now(), - } - toolCallMap.set(toolCall.id, toolCall) - items.push(toolCall) - } - break - } - case 'tool-result': { - const existingTool = event.toolCallId ? toolCallMap.get(event.toolCallId) : null - if (existingTool) { - existingTool.result = event.result - existingTool.status = 'completed' - } - break - } - case 'error': { - items.push({ - id: `error-${Date.now()}-${Math.random()}`, - kind: 'error', - message: event.error, - timestamp: event.ts ? new Date(event.ts).getTime() : Date.now(), - }) - break - } - case 'llm-stream-event': { - // We don't need to reconstruct streaming events for history - // Reasoning is captured in the final message - break - } - } - } - if (loadRunRequestIdRef.current !== requestId) return - - // Track permission requests and responses from history - const allPermissionRequests = new Map>() - const permResponseMap = new Map() - const autoPermissionDecisions = new Map>() - const askHumanRequests = new Map>() - const respondedAskHumanIds = new Set() - - for (const event of run.log) { - if (event.type === 'tool-permission-request') { - allPermissionRequests.set(event.toolCall.toolCallId, event) - } else if (event.type === 'tool-permission-response') { - permResponseMap.set(event.toolCallId, event.response) - } else if (event.type === 'tool-permission-auto-decision') { - autoPermissionDecisions.set(event.toolCallId, event) - } else if (event.type === 'ask-human-request') { - askHumanRequests.set(event.toolCallId, event) - } else if (event.type === 'ask-human-response') { - respondedAskHumanIds.add(event.toolCallId) - } - } - if (loadRunRequestIdRef.current !== requestId) return - - // Separate pending vs responded permission requests - const pendingPerms = new Map>() - for (const [id, req] of allPermissionRequests.entries()) { - if (!permResponseMap.has(id)) { - pendingPerms.set(id, req) - } - } - - const pendingAsks = new Map>() - for (const [id, req] of askHumanRequests.entries()) { - if (!respondedAskHumanIds.has(id)) { - pendingAsks.set(id, req) - } - } - if (loadRunRequestIdRef.current !== requestId) return - - // Set the conversation and runId - setConversation(items) - setRunId(id) - setMessage('') - // Reconcile composer state with THIS run. Loading a run while another one - // is mid-turn (e.g. binding a code session steals the single chat tab) - // must not leave isProcessing/isStopping pointing at the old run — that - // wedges the composer: stop targets the new run (a no-op) while the old - // run's processing-end arrives flagged as non-active and clears nothing. - setIsProcessing(processingRunIdsRef.current.has(id)) - setIsStopping(false) - setStopClickedAt(null) - setCurrentAssistantMessage(streamingBuffersRef.current.get(id)?.assistant ?? '') - setPendingPermissionRequests(pendingPerms) - setPendingAskHumanRequests(pendingAsks) - setAllPermissionRequests(allPermissionRequests) - setPermissionResponses(permResponseMap) - setAutoPermissionDecisions(autoPermissionDecisions) - - // Restore the run's per-chat work directory into the tab it was loaded into. + // Restore the session's per-chat work directory into the active tab. const tabId = activeChatTabIdRef.current const wd = await loadRunWorkDir(id) if (loadRunRequestIdRef.current !== requestId) return setWorkDirByTab((prev) => ({ ...prev, [tabId]: wd })) } catch (err) { - console.error('Failed to load run:', err) + console.error('Failed to load session work dir:', err) } }, [loadRunWorkDir]) @@ -2743,15 +2578,11 @@ function App() { let currentRunId = runId let isNewRun = false let newRunCreatedAt: string | null = null + const selected = selectedModelByTabRef.current.get(submitTabId) if (!currentRunId) { - const selected = selectedModelByTabRef.current.get(submitTabId) - const run = await window.ipc.invoke('runs:create', { - agentId, - ...(selected ? { model: selected.model, provider: selected.provider } : {}), - permissionMode: permissionMode ?? 'manual', - }) - currentRunId = run.id - newRunCreatedAt = run.createdAt + const createdSession = await window.ipc.invoke('sessions:create', {}) + currentRunId = createdSession.sessionId + newRunCreatedAt = new Date().toISOString() setRunId(currentRunId) analytics.chatSessionCreated(currentRunId) // Update active chat tab's runId to the new run @@ -2770,6 +2601,30 @@ function App() { let titleSource = userMessage const hasMentions = (mentions?.length ?? 0) > 0 + // Per-message turn config. Composition inputs land in the system prompt + // via the agent resolver; keep them session-sticky where possible so the + // provider prefix cache survives across turns. + const sendConfig = { + agent: { + agentId, + overrides: { + ...(selected ? { model: { provider: selected.provider, model: selected.model } } : {}), + composition: { + workDirId: currentRunId, + ...(pendingVoiceInputRef.current ? { voiceInput: true } : {}), + ...(ttsEnabledRef.current ? { voiceOutput: ttsModeRef.current } : {}), + ...(searchEnabled ? { searchEnabled: true } : {}), + ...(codeMode ? { codeMode } : {}), + }, + }, + }, + autoPermission: (permissionMode ?? 'manual') === 'auto', + } + const userMessageContextFor = (middlePane: Awaited>) => ({ + currentDateTime: new Date().toISOString(), + middlePane: middlePane ?? { kind: 'empty' as const }, + }) + if (hasAttachments || hasMentions) { type ContentPart = | { type: 'text'; text: string } @@ -2812,17 +2667,15 @@ function App() { titleSource = stagedAttachments[0]?.filename ?? mentions?.[0]?.displayName ?? mentions?.[0]?.path ?? '' } - // Shared IPC payload types can lag until package rebuilds; runtime validation still enforces schema. - const attachmentPayload = contentParts as unknown as string const middlePaneContext = await buildMiddlePaneContext() - await window.ipc.invoke('runs:createMessage', { - runId: currentRunId, - message: attachmentPayload, - voiceInput: pendingVoiceInputRef.current || undefined, - voiceOutput: ttsEnabledRef.current ? ttsModeRef.current : undefined, - searchEnabled: searchEnabled || undefined, - codeMode: codeMode || undefined, - middlePaneContext, + await window.ipc.invoke('sessions:sendMessage', { + sessionId: currentRunId, + input: { + role: 'user', + content: contentParts, + userMessageContext: userMessageContextFor(middlePaneContext), + }, + config: sendConfig, }) analytics.chatMessageSent({ voiceInput: pendingVoiceInputRef.current || undefined, @@ -2831,14 +2684,14 @@ function App() { }) } else { const middlePaneContext = await buildMiddlePaneContext() - await window.ipc.invoke('runs:createMessage', { - runId: currentRunId, - message: userMessage, - voiceInput: pendingVoiceInputRef.current || undefined, - voiceOutput: ttsEnabledRef.current ? ttsModeRef.current : undefined, - searchEnabled: searchEnabled || undefined, - codeMode: codeMode || undefined, - middlePaneContext, + await window.ipc.invoke('sessions:sendMessage', { + sessionId: currentRunId, + input: { + role: 'user', + content: userMessage, + userMessageContext: userMessageContextFor(middlePaneContext), + }, + config: sendConfig, }) analytics.chatMessageSent({ voiceInput: pendingVoiceInputRef.current || undefined, @@ -2875,20 +2728,34 @@ function App() { handlePromptSubmitRef.current?.({ text: `${name} connected successfully.`, files: [] }) }, []) + // Active tab renders from the sessions hook; background tabs keep the + // legacy cached view state until stage 7 retires it. + const chatTabStatesForRender = React.useMemo(() => { + if (!sessionChat.chatState) return chatViewStateByTab + return { + ...chatViewStateByTab, + [activeChatTabId]: { runId, ...sessionChat.chatState }, + } + }, [chatViewStateByTab, sessionChat.chatState, activeChatTabId, runId]) + + // The composer's stop state clears when the active turn settles. + useEffect(() => { + if (sessionChat.chatState && !sessionChat.chatState.isProcessing) { + setIsStopping(false) + setStopClickedAt(null) + } + }, [sessionChat.chatState]) + const handleStop = useCallback(async () => { if (!runId) return - const now = Date.now() - const isForce = isStopping && stopClickedAt !== null && (now - stopClickedAt) < 2000 - - setStopClickedAt(now) + setStopClickedAt(Date.now()) setIsStopping(true) - try { - await window.ipc.invoke('runs:stop', { runId, force: isForce }) + await sessionChat.stop() } catch (error) { - console.error('Failed to stop run:', error) + console.error('Failed to stop turn:', error) } - }, [runId, isStopping, stopClickedAt]) + }, [runId, sessionChat]) const handlePermissionResponse = useCallback(async ( toolCallId: string, @@ -2898,33 +2765,17 @@ function App() { ) => { if (!runId) return - // Optimistically update the UI immediately - setPermissionResponses(prev => { - const next = new Map(prev) - next.set(toolCallId, response) - return next - }) - setPendingPermissionRequests(prev => { - const next = new Map(prev) - next.delete(toolCallId) - return next - }) - + void subflow // subflows retired with the runs runtime try { - await window.ipc.invoke('runs:authorizePermission', { - runId, - authorization: { subflow, toolCallId, response, scope } - }) + await sessionChat.respondToPermission( + toolCallId, + response === 'approve' ? 'allow' : 'deny', + scope ? { scope } : undefined, + ) } catch (error) { console.error('Failed to authorize permission:', error) - // Revert the optimistic update on error - setPermissionResponses(prev => { - const next = new Map(prev) - next.delete(toolCallId) - return next - }) } - }, [runId]) + }, [runId, sessionChat]) // Answer a mid-run permission request from a code_agent_run coding turn. The // pending ask lives on the tool call itself, so we optimistically clear it and @@ -2948,15 +2799,13 @@ function App() { const handleAskHumanResponse = useCallback(async (toolCallId: string, subflow: string[], response: string) => { if (!runId) return + void subflow // subflows retired with the runs runtime try { - await window.ipc.invoke('runs:provideHumanInput', { - runId, - reply: { subflow, toolCallId, response } - }) + await sessionChat.answerAskHuman(toolCallId, response) } catch (error) { console.error('Failed to provide human input:', error) } - }, [runId]) + }, [runId, sessionChat]) const dismissBrowserOverlay = useCallback(() => { setIsBrowserOpen(false) @@ -6007,7 +5856,7 @@ function App() { onSelectRun={(rid) => void navigateToView({ type: 'chat', runId: rid })} onDeleteRun={async (rid) => { try { - await window.ipc.invoke('runs:delete', { runId: rid }) + await window.ipc.invoke('sessions:delete', { sessionId: rid }) await loadRuns() } catch (err) { console.error('Failed to delete run:', err) @@ -6454,11 +6303,11 @@ function App() { }} onOpenChatHistory={() => void navigateToView({ type: 'chat-history' })} onOpenFullScreen={toggleRightPaneMaximize} - conversation={conversation} - currentAssistantMessage={currentAssistantMessage} - chatTabStates={chatViewStateByTab} + conversation={sessionChat.chatState?.conversation ?? conversation} + currentAssistantMessage={sessionChat.chatState?.currentAssistantMessage ?? currentAssistantMessage} + chatTabStates={chatTabStatesForRender} viewportAnchors={chatViewportAnchorByTab} - isProcessing={isProcessing} + isProcessing={sessionChat.chatState ? sessionChat.chatState.isProcessing : isProcessing} isStopping={isStopping} onStop={handleStop} onSubmit={handlePromptSubmit} From ec5a53446507a57296f2f770ba1bb49c6f8dd00e Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 11:31:24 +0530 Subject: [PATCH 11/24] fix(x/renderer): StrictMode-safe feed lifecycle; drop obsolete runs:fetch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The chat stuck at 'Thinking…' with a completed turn on disk: the stores subscribed to the sessions:events feed in their constructors while cleanup was effect-managed, so React StrictMode's dev-mode mount -> cleanup -> mount cycle tore down the one subscription and never restored it. The renderer then loaded a mid-turn snapshot and was deaf to every event after (the backend turn completed fine). Feed attachment is now an effect-managed, idempotent connect() on both stores; hooks return its disconnect as the effect cleanup. New tests pin the bug: a StrictMode-style connect -> cleanup -> connect store test, and the hook tests now render under a StrictMode wrapper. Also removed the composer's runs:fetch model-freeze effect (the ENOENT noise): sessions carry model/permission per message, so the picker stays live for existing chats. Co-Authored-By: Claude Fable 5 --- .../components/chat-input-with-mentions.tsx | 13 ++----- .../src/hooks/useSessionChat.test.tsx | 13 +++++-- .../apps/renderer/src/hooks/useSessionChat.ts | 2 +- apps/x/apps/renderer/src/hooks/useSessions.ts | 3 +- .../src/lib/session-chat/store.test.ts | 37 +++++++++++++++++-- .../renderer/src/lib/session-chat/store.ts | 35 +++++++++++++----- 6 files changed, 74 insertions(+), 29 deletions(-) diff --git a/apps/x/apps/renderer/src/components/chat-input-with-mentions.tsx b/apps/x/apps/renderer/src/components/chat-input-with-mentions.tsx index 182ed20a..4ed34844 100644 --- a/apps/x/apps/renderer/src/components/chat-input-with-mentions.tsx +++ b/apps/x/apps/renderer/src/components/chat-input-with-mentions.tsx @@ -342,22 +342,15 @@ function ChatInputInner({ } }) - // When a run exists, freeze the dropdown to the run's resolved model+provider. + // Sessions runtime: model and permission mode are per-message turn config, + // so nothing is frozen for an existing chat — the picker stays live. useEffect(() => { if (!runId) { setLockedModel(null) setPermissionMode('auto') return } - let cancelled = false - window.ipc.invoke('runs:fetch', { runId }).then((run) => { - if (cancelled) return - if (run.provider && run.model) { - setLockedModel({ provider: run.provider, model: run.model }) - } - setPermissionMode(run.permissionMode ?? 'manual') - }).catch(() => { /* legacy run or fetch failure — leave unlocked */ }) - return () => { cancelled = true } + setLockedModel(null) }, [runId]) useEffect(() => { diff --git a/apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx b/apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx index 30caf6a9..3ec452f2 100644 --- a/apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx +++ b/apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx @@ -1,3 +1,4 @@ +import { StrictMode } from 'react' import { act, renderHook, waitFor } from '@testing-library/react' import { describe, expect, it } from 'vitest' import type { SessionBusEvent } from '@x/shared/src/sessions.js' @@ -69,7 +70,7 @@ function makeDeps() { describe('useSessionChat', () => { it('seeds from the session, follows live events, and routes actions', async () => { const { deps, calls, emit } = makeDeps() - const { result } = renderHook(() => useSessionChat(S1, deps)) + const { result } = renderHook(() => useSessionChat(S1, deps), { wrapper: StrictMode }) await waitFor(() => { expect(result.current.latestTurnId).toBe('turn-1') @@ -109,10 +110,14 @@ describe('useSessionChat', () => { ]) }) - it('unsubscribes from the feed on unmount', async () => { + it('unsubscribes from the feed on unmount (StrictMode double-mounts included)', async () => { const { deps, getUnsubscribed } = makeDeps() - const { unmount } = renderHook(() => useSessionChat(S1, deps)) + const { unmount } = renderHook(() => useSessionChat(S1, deps), { + wrapper: StrictMode, + }) unmount() - expect(getUnsubscribed()).toBe(1) + // StrictMode's simulated cleanup plus the real unmount: every subscribe + // was matched by an unsubscribe. + expect(getUnsubscribed()).toBe(2) }) }) diff --git a/apps/x/apps/renderer/src/hooks/useSessionChat.ts b/apps/x/apps/renderer/src/hooks/useSessionChat.ts index c598330c..6f8ee5ce 100644 --- a/apps/x/apps/renderer/src/hooks/useSessionChat.ts +++ b/apps/x/apps/renderer/src/hooks/useSessionChat.ts @@ -16,7 +16,7 @@ export function useSessionChat( deps: SessionChatStoreDeps = defaultDeps, ) { const [store] = useState(() => new SessionChatStore(deps)) - useEffect(() => () => store.dispose(), [store]) + useEffect(() => store.connect(), [store]) useEffect(() => { void store.setSession(sessionId) }, [store, sessionId]) diff --git a/apps/x/apps/renderer/src/hooks/useSessions.ts b/apps/x/apps/renderer/src/hooks/useSessions.ts index 872c6a6c..9c4a7213 100644 --- a/apps/x/apps/renderer/src/hooks/useSessions.ts +++ b/apps/x/apps/renderer/src/hooks/useSessions.ts @@ -13,8 +13,9 @@ const defaultDeps: SessionChatStoreDeps = { export function useSessions(deps: SessionChatStoreDeps = defaultDeps) { const [store] = useState(() => new SessionListStore(deps)) useEffect(() => { + const disconnect = store.connect() void store.load() - return () => store.dispose() + return disconnect }, [store]) const snapshot = useSyncExternalStore(store.subscribe, store.getSnapshot) const client = deps.client diff --git a/apps/x/apps/renderer/src/lib/session-chat/store.test.ts b/apps/x/apps/renderer/src/lib/session-chat/store.test.ts index c8b95bcf..96a23abb 100644 --- a/apps/x/apps/renderer/src/lib/session-chat/store.test.ts +++ b/apps/x/apps/renderer/src/lib/session-chat/store.test.ts @@ -86,20 +86,26 @@ class FakeClient implements SessionsClient { function makeStore() { const client = new FakeClient() let emit: SessionFeedListener = () => undefined + let subscribed = 0 let unsubscribed = 0 const store = new SessionChatStore({ client, subscribeFeed: (listener) => { + subscribed += 1 emit = listener return () => { unsubscribed += 1 + emit = () => undefined } }, }) + const disconnect = store.connect() return { client, store, + disconnect, emit: (event: SessionBusEvent) => emit(event), + getSubscribed: () => subscribed, getUnsubscribed: () => unsubscribed, } } @@ -245,12 +251,36 @@ describe('SessionChatStore', () => { expect(store.getSnapshot().chatState?.conversation).toEqual([]) }) - it('surfaces load errors and disposes its feed subscription', async () => { - const { store, getUnsubscribed } = makeStore() + it('surfaces load errors and disconnects its feed subscription', async () => { + const { store, disconnect, getUnsubscribed } = makeStore() await store.setSession('missing-session') expect(store.getSnapshot().error).toMatch(/session not found/) - store.dispose() + disconnect() expect(getUnsubscribed()).toBe(1) + void store + }) + + it('survives a StrictMode-style connect -> cleanup -> connect cycle', async () => { + // React StrictMode runs every effect's mount/cleanup/mount cycle in dev; + // the feed must re-attach or the store goes permanently deaf (the bug + // this test pins: a constructor-made subscription torn down by the first + // cleanup was never restored, leaving the chat stuck at "Thinking…"). + const { client, store, disconnect, emit, getSubscribed } = makeStore() + client.sessions.set(S1, sessionState(S1, [])) + disconnect() + const disconnect2 = store.connect() + expect(getSubscribed()).toBe(2) + + await store.setSession(S1) + emit(turnEvent(S1, 'turn-1', created('turn-1', S1, user('go')))) + emit(turnEvent(S1, 'turn-1', requested('turn-1', 0, [user('go')]))) + emit(turnEvent(S1, 'turn-1', completed('turn-1', 0, assistantText('done')))) + emit(turnEvent(S1, 'turn-1', turnCompleted('turn-1'))) + expect(store.getSnapshot().chatState?.isProcessing).toBe(false) + expect( + store.getSnapshot().chatState?.conversation.filter(isChatMessage).map((m) => m.content), + ).toEqual(['go', 'done']) + disconnect2() }) }) @@ -267,6 +297,7 @@ describe('SessionListStore', () => { return () => undefined }, }) + store.connect() await store.load() expect(store.getSnapshot().sessions.map((s) => s.sessionId).sort()).toEqual(['a', 'b']) diff --git a/apps/x/apps/renderer/src/lib/session-chat/store.ts b/apps/x/apps/renderer/src/lib/session-chat/store.ts index 8dc3b1f7..42408d4c 100644 --- a/apps/x/apps/renderer/src/lib/session-chat/store.ts +++ b/apps/x/apps/renderer/src/lib/session-chat/store.ts @@ -39,7 +39,8 @@ export interface SessionChatStoreDeps { // hook is a thin useSyncExternalStore subscription over it. export class SessionChatStore { private readonly client: SessionsClient - private readonly unsubscribeFeed: () => void + private readonly subscribeFeed: (listener: SessionFeedListener) => () => void + private feedDisconnect: (() => void) | null = null private readonly listeners = new Set<() => void>() private sessionId: string | null = null @@ -63,12 +64,20 @@ export class SessionChatStore { constructor(deps: SessionChatStoreDeps) { this.client = deps.client - this.unsubscribeFeed = deps.subscribeFeed(this.onFeedEvent) + this.subscribeFeed = deps.subscribeFeed } - dispose(): void { - this.unsubscribeFeed() - this.listeners.clear() + // Feed attachment is effect-managed and idempotent so React StrictMode's + // mount -> cleanup -> mount cycle re-attaches cleanly (a constructor-made + // subscription would be torn down by the first cleanup and never restored). + connect(): () => void { + if (!this.feedDisconnect) { + this.feedDisconnect = this.subscribeFeed(this.onFeedEvent) + } + return () => { + this.feedDisconnect?.() + this.feedDisconnect = null + } } subscribe = (onChange: () => void): (() => void) => { @@ -241,7 +250,8 @@ export interface SessionListSnapshot { export class SessionListStore { private readonly client: SessionsClient - private readonly unsubscribeFeed: () => void + private readonly subscribeFeed: (listener: SessionFeedListener) => () => void + private feedDisconnect: (() => void) | null = null private readonly listeners = new Set<() => void>() private entries = new Map() private loading = true @@ -249,12 +259,17 @@ export class SessionListStore { constructor(deps: SessionChatStoreDeps) { this.client = deps.client - this.unsubscribeFeed = deps.subscribeFeed(this.onFeedEvent) + this.subscribeFeed = deps.subscribeFeed } - dispose(): void { - this.unsubscribeFeed() - this.listeners.clear() + connect(): () => void { + if (!this.feedDisconnect) { + this.feedDisconnect = this.subscribeFeed(this.onFeedEvent) + } + return () => { + this.feedDisconnect?.() + this.feedDisconnect = null + } } subscribe = (onChange: () => void): (() => void) => { From 58b016ba99c8427e480efda450bde5eb68e414c8 Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 11:43:25 +0530 Subject: [PATCH 12/24] fix(x/renderer): surface permission/ask-human cards and load errors MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ChatSidebar assembles the active tab's view state from individual props, not from chatTabStates — and the four permission/ask-human maps were still fed from legacy standalone state that nothing populates under the sessions runtime. Pending permission requests (recorded correctly in the turn file) therefore never rendered, leaving the chat on the 'Thinking…' shimmer. - Feed pendingAskHumanRequests / allPermissionRequests / permissionResponses / autoPermissionDecisions from the sessions hook (legacy fallback until stage 7). - New optional isThinking prop on ChatSidebar: waiting on a permission/ask-human no longer shimmers (isProcessing still blocks the composer, per the state contract). - Session load failures are now visible: an error item renders in the chat instead of a silent blank, and the store logs load/reload failures to the console. Verified the full data path against the real on-disk session (the suspended list-downloads turn): conversation renders and the pending permission card map carries the request, under StrictMode. Co-Authored-By: Claude Fable 5 --- apps/x/apps/renderer/src/App.tsx | 18 +++++++++++++----- .../renderer/src/components/chat-sidebar.tsx | 7 ++++++- .../renderer/src/lib/session-chat/store.ts | 4 +++- 3 files changed, 22 insertions(+), 7 deletions(-) diff --git a/apps/x/apps/renderer/src/App.tsx b/apps/x/apps/renderer/src/App.tsx index 61a5a471..8b2a32f4 100644 --- a/apps/x/apps/renderer/src/App.tsx +++ b/apps/x/apps/renderer/src/App.tsx @@ -2728,6 +2728,13 @@ function App() { handlePromptSubmitRef.current?.({ text: `${name} connected successfully.`, files: [] }) }, []) + // A failed session load must be visible, not a blank chat. + const sessionLoadErrorItems = React.useMemo(() => ( + sessionChat.error + ? [{ id: 'session-load-error', kind: 'error', message: `Failed to load chat: ${sessionChat.error}`, timestamp: 0 }] + : [] + ), [sessionChat.error]) + // Active tab renders from the sessions hook; background tabs keep the // legacy cached view state until stage 7 retires it. const chatTabStatesForRender = React.useMemo(() => { @@ -6303,7 +6310,7 @@ function App() { }} onOpenChatHistory={() => void navigateToView({ type: 'chat-history' })} onOpenFullScreen={toggleRightPaneMaximize} - conversation={sessionChat.chatState?.conversation ?? conversation} + conversation={sessionChat.chatState?.conversation ?? (sessionLoadErrorItems.length > 0 ? sessionLoadErrorItems : conversation)} currentAssistantMessage={sessionChat.chatState?.currentAssistantMessage ?? currentAssistantMessage} chatTabStates={chatTabStatesForRender} viewportAnchors={chatViewportAnchorByTab} @@ -6338,10 +6345,11 @@ function App() { ? { title: activeCodeSession.session.title } : null } - pendingAskHumanRequests={pendingAskHumanRequests} - allPermissionRequests={allPermissionRequests} - permissionResponses={permissionResponses} - autoPermissionDecisions={autoPermissionDecisions} + pendingAskHumanRequests={sessionChat.chatState?.pendingAskHumanRequests ?? pendingAskHumanRequests} + allPermissionRequests={sessionChat.chatState?.allPermissionRequests ?? allPermissionRequests} + permissionResponses={sessionChat.chatState?.permissionResponses ?? permissionResponses} + autoPermissionDecisions={sessionChat.chatState?.autoPermissionDecisions ?? autoPermissionDecisions} + isThinking={sessionChat.chatState?.isThinking} onPermissionResponse={handlePermissionResponse} onAskHumanResponse={handleAskHumanResponse} isToolOpenForTab={isToolOpenForTab} diff --git a/apps/x/apps/renderer/src/components/chat-sidebar.tsx b/apps/x/apps/renderer/src/components/chat-sidebar.tsx index a9d6d9f7..976f9684 100644 --- a/apps/x/apps/renderer/src/components/chat-sidebar.tsx +++ b/apps/x/apps/renderer/src/components/chat-sidebar.tsx @@ -142,6 +142,10 @@ interface ChatSidebarProps { chatTabStates?: Record viewportAnchors?: Record isProcessing: boolean + // Actively working (sessions runtime). When provided, drives the shimmer + // instead of isProcessing so waiting on a permission/ask-human doesn't + // render a "Thinking…" under the card. + isThinking?: boolean isStopping?: boolean onStop?: () => void onSubmit: (message: PromptInputMessage, mentions?: FileMention[], attachments?: StagedAttachment[], searchEnabled?: boolean, codeMode?: 'claude' | 'codex', permissionMode?: PermissionMode) => void @@ -211,6 +215,7 @@ export function ChatSidebar({ chatTabStates = {}, viewportAnchors = {}, isProcessing, + isThinking, isStopping, onStop, onSubmit, @@ -754,7 +759,7 @@ export function ChatSidebar({ )} - {isActive && isProcessing && !tabState.currentAssistantMessage && ( + {isActive && (isThinking ?? isProcessing) && !tabState.currentAssistantMessage && ( Thinking... diff --git a/apps/x/apps/renderer/src/lib/session-chat/store.ts b/apps/x/apps/renderer/src/lib/session-chat/store.ts index 42408d4c..2bd0b4b4 100644 --- a/apps/x/apps/renderer/src/lib/session-chat/store.ts +++ b/apps/x/apps/renderer/src/lib/session-chat/store.ts @@ -115,6 +115,7 @@ export class SessionChatStore { this.emit() } catch (error) { if (generation !== this.generation) return + console.error('[session-chat] failed to load session', sessionId, error) this.loading = false this.error = error instanceof Error ? error.message : String(error) this.emit() @@ -164,8 +165,9 @@ export class SessionChatStore { } this.latestEvents = turn.events this.emit() - } catch { + } catch (error) { // The next snapshot-worthy event will retry. + console.error('[session-chat] failed to reload turn', turnId, error) } } From a58493b545da33ef4413bcdaaed406f05007b5af Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 11:51:31 +0530 Subject: [PATCH 13/24] fix(x/renderer): enable suspended-turn cards; unify active-tab state for both chat views MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two follow-ups from live testing: 1. Permission/ask-human buttons were disabled while a turn waited for input. PermissionRequest disables via isProcessing, whose old-runtime meaning was 'agent busy right now' (false while awaiting approval); the sessions meaning is 'turn not settled' (true while suspended). Cards in both chat views now take the isThinking flag — actively working disables, waiting on the user enables — while the composer and stop button stay on isProcessing. 2. The middle-pane (full-screen) chat rendered blank: App.tsx has its own conversation block whose activeChatTabState was still assembled from legacy standalone states. activeChatTabState is now the single hook-backed source of truth for the active tab; the sidebar props, the middle pane, the merged background-tab map, and the submit guard all derive from it (plus hook-driven activeIsProcessing/ activeIsThinking), so both views land on identical data. Co-Authored-By: Claude Fable 5 --- apps/x/apps/renderer/src/App.tsx | 85 ++++++++++--------- .../renderer/src/components/chat-sidebar.tsx | 4 +- 2 files changed, 48 insertions(+), 41 deletions(-) diff --git a/apps/x/apps/renderer/src/App.tsx b/apps/x/apps/renderer/src/App.tsx index 8b2a32f4..6b820f70 100644 --- a/apps/x/apps/renderer/src/App.tsx +++ b/apps/x/apps/renderer/src/App.tsx @@ -949,6 +949,18 @@ function App() { const streamingBuffersRef = useRef>(new Map()) const [isStopping, setIsStopping] = useState(false) const [, setStopClickedAt] = useState(null) + // Sessions runtime: hook-derived activity for the ACTIVE chat. + // activeIsProcessing blocks the composer until the turn settles; + // activeIsThinking ("actively working") drives shimmers and disables + // permission/ask-human buttons — false while waiting on the user. + const activeIsProcessing = sessionChat.chatState?.isProcessing ?? isProcessing + const activeIsThinking = sessionChat.chatState ? sessionChat.chatState.isThinking : isProcessing + // A failed session load must be visible, not a blank chat. + const sessionLoadErrorItems = React.useMemo(() => ( + sessionChat.error + ? [{ id: 'session-load-error', kind: 'error', message: `Failed to load chat: ${sessionChat.error}`, timestamp: 0 }] + : [] + ), [sessionChat.error]) const [agentId] = useState('copilot') const [presetMessage, setPresetMessage] = useState(undefined) @@ -2545,7 +2557,7 @@ function App() { codeMode?: 'claude' | 'codex', permissionMode?: PermissionMode, ) => { - if (isProcessing) return + if (activeIsProcessing) return const submitTabId = activeChatTabIdRef.current const { text } = message @@ -2728,23 +2740,6 @@ function App() { handlePromptSubmitRef.current?.({ text: `${name} connected successfully.`, files: [] }) }, []) - // A failed session load must be visible, not a blank chat. - const sessionLoadErrorItems = React.useMemo(() => ( - sessionChat.error - ? [{ id: 'session-load-error', kind: 'error', message: `Failed to load chat: ${sessionChat.error}`, timestamp: 0 }] - : [] - ), [sessionChat.error]) - - // Active tab renders from the sessions hook; background tabs keep the - // legacy cached view state until stage 7 retires it. - const chatTabStatesForRender = React.useMemo(() => { - if (!sessionChat.chatState) return chatViewStateByTab - return { - ...chatViewStateByTab, - [activeChatTabId]: { runId, ...sessionChat.chatState }, - } - }, [chatViewStateByTab, sessionChat.chatState, activeChatTabId, runId]) - // The composer's stop state clears when the active turn settles. useEffect(() => { if (sessionChat.chatState && !sessionChat.chatState.isProcessing) { @@ -5443,16 +5438,24 @@ function App() { return null } - const activeChatTabState = React.useMemo(() => ({ - runId, - conversation, - currentAssistantMessage, - pendingAskHumanRequests, - allPermissionRequests, - permissionResponses, - autoPermissionDecisions, - }), [ + // The active chat's view state, backed by the sessions hook (legacy + // standalone states remain only as the pre-load fallback until stage 7). + const activeChatTabState = React.useMemo(() => ( + sessionChat.chatState + ? { runId, ...sessionChat.chatState } + : { + runId, + conversation: sessionLoadErrorItems.length > 0 ? sessionLoadErrorItems : conversation, + currentAssistantMessage, + pendingAskHumanRequests, + allPermissionRequests, + permissionResponses, + autoPermissionDecisions, + } + ), [ runId, + sessionChat.chatState, + sessionLoadErrorItems, conversation, currentAssistantMessage, pendingAskHumanRequests, @@ -5465,6 +5468,10 @@ function App() { if (tabId === activeChatTabId) return activeChatTabState return chatViewStateByTab[tabId] ?? emptyChatTabState }, [activeChatTabId, activeChatTabState, chatViewStateByTab, emptyChatTabState]) + const chatTabStatesForRender = React.useMemo(() => ({ + ...chatViewStateByTab, + [activeChatTabId]: activeChatTabState, + }), [chatViewStateByTab, activeChatTabId, activeChatTabState]) const selectedTask = selectedBackgroundTask ? backgroundTasks.find(t => t.name === selectedBackgroundTask) : null @@ -6162,7 +6169,7 @@ function App() { onApproveSession={() => handlePermissionResponse(permRequest.toolCall.toolCallId, permRequest.subflow, 'approve', 'session')} onApproveAlways={() => handlePermissionResponse(permRequest.toolCall.toolCallId, permRequest.subflow, 'approve', 'always')} onDeny={() => handlePermissionResponse(permRequest.toolCall.toolCallId, permRequest.subflow, 'deny')} - isProcessing={isActive && isProcessing} + isProcessing={isActive && activeIsThinking} response={response} /> )} @@ -6180,7 +6187,7 @@ function App() { query={request.query} options={request.options} onResponse={(response) => handleAskHumanResponse(request.toolCallId, request.subflow, response)} - isProcessing={isActive && isProcessing} + isProcessing={isActive && activeIsThinking} /> ))} @@ -6192,7 +6199,7 @@ function App() { )} - {isActive && isProcessing && !tabState.currentAssistantMessage && ( + {isActive && activeIsThinking && !tabState.currentAssistantMessage && ( Thinking... @@ -6228,7 +6235,7 @@ function App() { visibleFiles={visibleKnowledgeFiles} onSubmit={handlePromptSubmit} onStop={handleStop} - isProcessing={isActive && isProcessing} + isProcessing={isActive && activeIsProcessing} isStopping={isActive && isStopping} isActive={isActive} presetMessage={isActive ? presetMessage : undefined} @@ -6310,11 +6317,11 @@ function App() { }} onOpenChatHistory={() => void navigateToView({ type: 'chat-history' })} onOpenFullScreen={toggleRightPaneMaximize} - conversation={sessionChat.chatState?.conversation ?? (sessionLoadErrorItems.length > 0 ? sessionLoadErrorItems : conversation)} - currentAssistantMessage={sessionChat.chatState?.currentAssistantMessage ?? currentAssistantMessage} + conversation={activeChatTabState.conversation} + currentAssistantMessage={activeChatTabState.currentAssistantMessage} chatTabStates={chatTabStatesForRender} viewportAnchors={chatViewportAnchorByTab} - isProcessing={sessionChat.chatState ? sessionChat.chatState.isProcessing : isProcessing} + isProcessing={activeIsProcessing} isStopping={isStopping} onStop={handleStop} onSubmit={handlePromptSubmit} @@ -6345,11 +6352,11 @@ function App() { ? { title: activeCodeSession.session.title } : null } - pendingAskHumanRequests={sessionChat.chatState?.pendingAskHumanRequests ?? pendingAskHumanRequests} - allPermissionRequests={sessionChat.chatState?.allPermissionRequests ?? allPermissionRequests} - permissionResponses={sessionChat.chatState?.permissionResponses ?? permissionResponses} - autoPermissionDecisions={sessionChat.chatState?.autoPermissionDecisions ?? autoPermissionDecisions} - isThinking={sessionChat.chatState?.isThinking} + pendingAskHumanRequests={activeChatTabState.pendingAskHumanRequests} + allPermissionRequests={activeChatTabState.allPermissionRequests} + permissionResponses={activeChatTabState.permissionResponses} + autoPermissionDecisions={activeChatTabState.autoPermissionDecisions} + isThinking={activeIsThinking} onPermissionResponse={handlePermissionResponse} onAskHumanResponse={handleAskHumanResponse} isToolOpenForTab={isToolOpenForTab} diff --git a/apps/x/apps/renderer/src/components/chat-sidebar.tsx b/apps/x/apps/renderer/src/components/chat-sidebar.tsx index 976f9684..893f9f1a 100644 --- a/apps/x/apps/renderer/src/components/chat-sidebar.tsx +++ b/apps/x/apps/renderer/src/components/chat-sidebar.tsx @@ -730,7 +730,7 @@ export function ChatSidebar({ onApproveSession={() => onPermissionResponse(permRequest.toolCall.toolCallId, permRequest.subflow, 'approve', 'session')} onApproveAlways={() => onPermissionResponse(permRequest.toolCall.toolCallId, permRequest.subflow, 'approve', 'always')} onDeny={() => onPermissionResponse(permRequest.toolCall.toolCallId, permRequest.subflow, 'deny')} - isProcessing={isActive && isProcessing} + isProcessing={isActive && (isThinking ?? isProcessing)} response={response} /> )} @@ -747,7 +747,7 @@ export function ChatSidebar({ key={request.toolCallId} query={request.query} onResponse={(response) => onAskHumanResponse(request.toolCallId, request.subflow, response)} - isProcessing={isActive && isProcessing} + isProcessing={isActive && (isThinking ?? isProcessing)} /> ))} From 3822bf53daccfd964af05158669614a7fff047da Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 12:46:23 +0530 Subject: [PATCH 14/24] feat(x): reference-based model requests + wire-form composer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit model_call_requested carried three duplications that dominated turn-file size (measured on a real 647KB two-step turn): the system prompt (28KB) and the 38-tool snapshot (40KB) repeated per call despite being byte- identical to turn_created.agent.resolved, and messages re-inlining the whole current-turn transcript (230KB re-stating a 207KB tool result). ModelRequest is now a list of references into the turn's own events — call 0: [{context}?, {input}]; call N: [{assistant: N-1}, ...that batch's toolResults in source order]; re-issue after an interruption: []. Every referenced byte exists exactly once in the file; the measured turn drops to ~285KB and each further model call costs ~200 bytes. The reducer's ordering invariant got stricter: reference lists are matched exactly against the transcript. Debuggability is now byte-for-byte at the wire level: ResolvedModel gains encodeMessages (the structural->wire conversion — user-message context weaving, attachment rendering, tool-result enveloping, i.e. convertFromMessages), and composeModelRequest rebuilds the exact provider payload (resolved system prompt + wrapped tools + materialized prefix + resolved refs, encoded). The loop transmits exactly the composer's output, so the durable file plus the composer reproduce what the model received — pinned by a property test asserting composed == sent for every call, and a 2KB size guard on request events. A bridge test demonstrates the woven wire form (no raw userMessageContext). Breaking for existing dev turn files (same schemaVersion, pre-release): wipe ~/.rowboat/storage. Co-Authored-By: Claude Fable 5 --- apps/x/.gitignore | 1 + .../src/hooks/useSessionChat.test.tsx | 2 +- .../src/lib/session-chat/store.test.ts | 8 +- .../src/lib/session-chat/test-fixtures.ts | 18 +- .../src/lib/session-chat/turn-view.test.ts | 23 +- .../packages/core/docs/turn-runtime-design.md | 56 +++-- .../core/src/sessions/sessions.test.ts | 2 +- .../turns/bridges/real-model-registry.test.ts | 42 +++- .../src/turns/bridges/real-model-registry.ts | 8 +- .../core/src/turns/compose-model-request.ts | 61 ++++++ .../core/src/turns/context-resolver.test.ts | 13 +- .../x/packages/core/src/turns/fs-repo.test.ts | 4 +- apps/x/packages/core/src/turns/index.ts | 1 + .../packages/core/src/turns/model-registry.ts | 10 +- .../x/packages/core/src/turns/runtime.test.ts | 88 +++++--- apps/x/packages/core/src/turns/runtime.ts | 52 ++++- apps/x/packages/shared/src/turns.test.ts | 204 ++++++++++-------- apps/x/packages/shared/src/turns.ts | 158 ++++++++++---- 18 files changed, 524 insertions(+), 227 deletions(-) create mode 100644 apps/x/packages/core/src/turns/compose-model-request.ts diff --git a/apps/x/.gitignore b/apps/x/.gitignore index db195fb4..2c5f63fd 100644 --- a/apps/x/.gitignore +++ b/apps/x/.gitignore @@ -1,2 +1,3 @@ node_modules/ test-fixtures/ +*.tsbuildinfo diff --git a/apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx b/apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx index 3ec452f2..b1b57124 100644 --- a/apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx +++ b/apps/x/apps/renderer/src/hooks/useSessionChat.test.tsx @@ -82,7 +82,7 @@ describe('useSessionChat', () => { // A new turn streams in over the feed. act(() => { emit({ kind: 'turn-event', sessionId: S1, turnId: 'turn-2', event: created('turn-2', S1, user('q2')) }) - emit({ kind: 'turn-event', sessionId: S1, turnId: 'turn-2', event: requested('turn-2', 0, [user('q2')]) }) + emit({ kind: 'turn-event', sessionId: S1, turnId: 'turn-2', event: requested('turn-2', 0) }) emit({ kind: 'turn-event', sessionId: S1, diff --git a/apps/x/apps/renderer/src/lib/session-chat/store.test.ts b/apps/x/apps/renderer/src/lib/session-chat/store.test.ts index 96a23abb..9c5e0e9c 100644 --- a/apps/x/apps/renderer/src/lib/session-chat/store.test.ts +++ b/apps/x/apps/renderer/src/lib/session-chat/store.test.ts @@ -143,7 +143,7 @@ describe('SessionChatStore', () => { await store.setSession(S1) emit(turnEvent(S1, 'turn-1', created('turn-1', S1, user('go')))) - emit(turnEvent(S1, 'turn-1', requested('turn-1', 0, [user('go')]))) + emit(turnEvent(S1, 'turn-1', requested('turn-1', 0))) expect(store.getSnapshot().chatState?.isThinking).toBe(true) emit(turnEvent(S1, 'turn-1', completed('turn-1', 0, assistantText('done')))) @@ -161,7 +161,7 @@ describe('SessionChatStore', () => { client.sessions.set(S1, sessionState(S1, [])) await store.setSession(S1) emit(turnEvent(S1, 'turn-1', created('turn-1', S1, user('go')))) - emit(turnEvent(S1, 'turn-1', requested('turn-1', 0, [user('go')]))) + emit(turnEvent(S1, 'turn-1', requested('turn-1', 0))) emit(turnEvent(S1, 'turn-1', { type: 'text_delta', turnId: 'turn-1', modelCallIndex: 0, delta: 'he' })) emit(turnEvent(S1, 'turn-1', { type: 'text_delta', turnId: 'turn-1', modelCallIndex: 0, delta: 'y' })) expect(store.getSnapshot().chatState?.currentAssistantMessage).toBe('hey') @@ -199,7 +199,7 @@ describe('SessionChatStore', () => { // The feed attached mid-turn: we never saw turn_created for turn-9. client.turns.set('turn-9', [ created('turn-9', S1, user('missed')), - requested('turn-9', 0, [user('missed')]), + requested('turn-9', 0), ]) emit(turnEvent(S1, 'turn-9', completed('turn-9', 0, assistantText('caught up')))) await Promise.resolve() @@ -273,7 +273,7 @@ describe('SessionChatStore', () => { await store.setSession(S1) emit(turnEvent(S1, 'turn-1', created('turn-1', S1, user('go')))) - emit(turnEvent(S1, 'turn-1', requested('turn-1', 0, [user('go')]))) + emit(turnEvent(S1, 'turn-1', requested('turn-1', 0))) emit(turnEvent(S1, 'turn-1', completed('turn-1', 0, assistantText('done')))) emit(turnEvent(S1, 'turn-1', turnCompleted('turn-1'))) expect(store.getSnapshot().chatState?.isProcessing).toBe(false) diff --git a/apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts b/apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts index 381f8cb9..d20c4bd0 100644 --- a/apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts +++ b/apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts @@ -46,16 +46,24 @@ export function created( } } -export function requested(turnId: string, index: number, messages: unknown[]): TEvent { +type RequestRef = + | { kind: 'context' } + | { kind: 'input' } + | { kind: 'assistant'; modelCallIndex: number } + | { kind: 'toolResult'; toolCallId: string } + +export function requested( + turnId: string, + index: number, + refs: RequestRef[] = [{ kind: 'input' }], +): TEvent { return { type: 'model_call_requested', turnId, ts: TS, modelCallIndex: index, request: { - systemPrompt: 'SYS', - messages: messages as never, - tools: [], + messages: refs, parameters: {}, }, } @@ -127,7 +135,7 @@ export function completedTurnLog( ): TEvent[] { return [ created(turnId, sessionId, user(question)), - requested(turnId, 0, [user(question)]), + requested(turnId, 0), completed(turnId, 0, assistantText(answer)), turnCompleted(turnId, answer), ] diff --git a/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts b/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts index 4acbd6e9..4c43c7c1 100644 --- a/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts +++ b/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts @@ -87,14 +87,13 @@ describe('buildTurnConversation', () => { const call = assistantCalls(toolCallPart('tc1', 'echo', { x: 1 })) const state = reduceTurn([ created(T1, S1, user('run it')), - requested(T1, 0, [user('run it')]), + requested(T1, 0), completed(T1, 0, call), invocation(T1, 'tc1', 'echo'), toolResult(T1, 'tc1', 'echo', { echoed: true }), requested(T1, 1, [ - user('run it'), - call, - { role: 'tool', content: '{"echoed":true}', toolCallId: 'tc1', toolName: 'echo' }, + { kind: 'assistant', modelCallIndex: 0 }, + { kind: 'toolResult', toolCallId: 'tc1' }, ]), completed(T1, 1, assistantText('all done')), turnCompleted(T1, 'all done'), @@ -113,7 +112,7 @@ describe('buildTurnConversation', () => { it('marks permission-pending tools pending and running tools running', () => { const state = reduceTurn([ created(T1, S1), - requested(T1, 0, [user('hello')]), + requested(T1, 0), completed(T1, 0, assistantCalls(toolCallPart('p1', 'executeCommand'), toolCallPart('r1', 'echo'))), { type: 'tool_permission_required', @@ -139,7 +138,7 @@ describe('buildTurnConversation', () => { } const state = reduceTurn([ created(T1, S1, input as never), - requested(T1, 0, [input]), + requested(T1, 0), { type: 'model_call_failed', turnId: T1, ts: TS, modelCallIndex: 0, error: 'boom' }, { type: 'turn_failed', turnId: T1, ts: TS, error: 'boom', usage: {} }, ]) @@ -154,7 +153,7 @@ describe('buildSessionChatState', () => { const prior = reduceTurn(completedTurnLog('turn-1', S1, 'first?', 'first answer')) const latest = reduceTurn([ created('turn-2', S1, user('second?')), - requested('turn-2', 0, [user('second?')]), + requested('turn-2', 0), ]) const state = buildSessionChatState([prior, latest], emptyOverlay()) expect( @@ -174,7 +173,7 @@ describe('buildSessionChatState', () => { it('exposes pending permissions as request events; waiting is not thinking', () => { const turn = reduceTurn([ created(T1, S1), - requested(T1, 0, [user('hello')]), + requested(T1, 0), completed(T1, 0, assistantCalls(toolCallPart('p1', 'executeCommand', { command: 'rm -rf /' }))), { type: 'tool_permission_required', @@ -209,7 +208,7 @@ describe('buildSessionChatState', () => { it('maps human decisions to responses and classifier decisions to auto-decisions', () => { const turn = reduceTurn([ created(T1, S1), - requested(T1, 0, [user('hello')]), + requested(T1, 0), completed(T1, 0, assistantCalls(toolCallPart('h1', 'echo'), toolCallPart('c1', 'echo'))), { type: 'tool_permission_required', turnId: T1, ts: TS, toolCallId: 'h1', toolName: 'echo', request: { kind: 'command', commandNames: ['x'] } }, { type: 'tool_permission_required', turnId: T1, ts: TS, toolCallId: 'c1', toolName: 'echo', request: { kind: 'command', commandNames: ['y'] } }, @@ -230,7 +229,7 @@ describe('buildSessionChatState', () => { it('exposes pending ask-human calls with question and options', () => { const turn = reduceTurn([ created(T1, S1), - requested(T1, 0, [user('hello')]), + requested(T1, 0), completed(T1, 0, assistantCalls(toolCallPart('ah1', 'ask-human', { question: 'Deploy?', options: ['Yes', 'No'] }))), { type: 'tool_invocation_requested', @@ -254,7 +253,7 @@ describe('buildSessionChatState', () => { it('stitches live tool output onto the matching tool item', () => { const turn = reduceTurn([ created(T1, S1), - requested(T1, 0, [user('hello')]), + requested(T1, 0), completed(T1, 0, assistantCalls(toolCallPart('tc1', 'executeCommand'))), invocation(T1, 'tc1', 'executeCommand'), ]) @@ -265,7 +264,7 @@ describe('buildSessionChatState', () => { }) it('surfaces streaming text as currentAssistantMessage', () => { - const turn = reduceTurn([created(T1, S1), requested(T1, 0, [user('hello')])]) + const turn = reduceTurn([created(T1, S1), requested(T1, 0)]) const state = buildSessionChatState([turn], { ...emptyOverlay(), text: 'typing…' }) expect(state.currentAssistantMessage).toBe('typing…') }) diff --git a/apps/x/packages/core/docs/turn-runtime-design.md b/apps/x/packages/core/docs/turn-runtime-design.md index 904fcb02..9a0ca542 100644 --- a/apps/x/packages/core/docs/turn-runtime-design.md +++ b/apps/x/packages/core/docs/turn-runtime-design.md @@ -479,11 +479,15 @@ Every AI SDK `streamText` invocation performs exactly one model step. Tool execution is controlled manually by the turn loop. ```ts +type ModelRequestMessageRef = + | { kind: "context" } // the inline context block + | { kind: "input" } // the turn's user message + | { kind: "assistant"; modelCallIndex: number } // → model_call_completed + | { kind: "toolResult"; toolCallId: string }; // → tool_result + interface ModelRequest { - systemPrompt: string; - contextRef?: { previousTurnId: string }; - messages: ConversationMessage[]; - tools: ToolDescriptor[]; + contextRef?: { previousTurnId: string }; // call 0 only (cross-turn prefix) + messages: ModelRequestMessageRef[]; // what is NEW since the previous call parameters: Record; } @@ -497,24 +501,26 @@ interface ModelCallRequested extends BaseTurnEvent { `modelCallIndex` starts at `0` and increments monotonically for each primary agent model call in the turn. -`request` records the exact model input with one substitution: when the -turn's context is a reference, the resolved prefix is not re-inlined. -`contextRef` repeats the reference and `messages` contains only current-turn -messages — the turn input, this turn's prior assistant responses, and ordered -tool results — captured byte-for-byte as assembled immediately before -provider invocation. When the turn's context is inline, `contextRef` is -absent and `messages` begins with that inline context. The full request is -deterministically materializable because referenced turns are immutable; a -`materializeRequest(turnId, modelCallIndex)` debug utility composes resolver -output with the persisted messages. +`request` is a list of references into the turn's own events: every +referenced byte exists exactly once in the file. A request records only what +is new since the previous model call — call 0 is `[{context}?, {input}]` +(context only when inline and nonempty; cross-turn prefixes ride +`contextRef`); call N is `[{assistant: N-1}, …that batch's toolResults in +source order]`; a re-issued call after an interruption is `[]`. The system +prompt and tool set are not repeated: they are byte-identical to +`turn_created.agent.resolved` by construction. -Within-turn duplication is deliberate: each request restates the current -turn's accumulated messages even though they also exist as earlier events in -the same file. The persisted `messages` are the ground truth of what the loop -actually assembled and sent — including any assembly bugs — which a derived -reconstruction would mask. +The exact provider payload is rebuilt by the request composer +(`composeModelRequest` in core): resolved system prompt, wrapped tool +definitions, the materialized cross-turn prefix, and every request's +references resolved against the turn's events, all passed through the model +bridge's `encodeMessages` (the deterministic structural→wire conversion — +user-message context weaving, attachment rendering, tool-result enveloping). +The loop transmits exactly the composer's output, and debugging/audit calls +the same function, so the durable file plus the composer reproduce the wire +bytes and the two views cannot diverge. -`request` excludes credentials, auth headers, functions, model objects, and +Requests exclude credentials, auth headers, functions, model objects, and transport objects. The name `requested` is intentional. The event proves durable intent, not that @@ -1526,7 +1532,11 @@ but the first turn implementation does not enforce it. - Any event after a terminal turn event. - Mutation of immutable turn definition data. - `model_call_requested.contextRef` values inconsistent with the turn - definition's context. + definition's context, or present on a non-initial model call. +- Request references that do not match the transcript: call 0 must be + `[{context}?, {input}]`; call N must reference the previous completed + call's response and its batch's tool results in source order (or nothing, + after an interrupted call). The reducer validates `context` structurally but treats it as opaque; it never resolves references (section 6.6). @@ -1755,8 +1765,8 @@ tests for: closure results. - A reference to a missing or corrupt turn file is an infrastructure error and appends nothing. -- `materializeRequest` output equals resolver output plus persisted - current-turn request messages. +- The composed request equals what the loop sent (byte-for-byte property: + durable file + composer reproduce the provider payload). - The reducer never resolves references. ## 28. Suggested module layout diff --git a/apps/x/packages/core/src/sessions/sessions.test.ts b/apps/x/packages/core/src/sessions/sessions.test.ts index ad9809cd..fccdb948 100644 --- a/apps/x/packages/core/src/sessions/sessions.test.ts +++ b/apps/x/packages/core/src/sessions/sessions.test.ts @@ -86,7 +86,7 @@ function turnLog( turnId, ts: TS, modelCallIndex: 0, - request: { systemPrompt: "SYS", messages: [user("hi")], tools: [], parameters: {} }, + request: { messages: [{ kind: 'input' as const }], parameters: {} }, }; switch (status) { case "idle": diff --git a/apps/x/packages/core/src/turns/bridges/real-model-registry.test.ts b/apps/x/packages/core/src/turns/bridges/real-model-registry.test.ts index a8eb7a2c..ea0fe990 100644 --- a/apps/x/packages/core/src/turns/bridges/real-model-registry.test.ts +++ b/apps/x/packages/core/src/turns/bridges/real-model-registry.test.ts @@ -26,7 +26,7 @@ function makeRegistry(parts: Array>, capture: InvokerOpt function request(overrides: Partial = {}): ModelStreamRequest { return { systemPrompt: "SYS", - messages: [{ role: "user", content: "hello" }], + messages: [{ role: "user", content: "hello" }] as ModelStreamRequest["messages"], tools: [ { toolId: "builtin:echo", @@ -107,9 +107,10 @@ describe("RealModelRegistry", () => { }, }); - // The invoker received the system prompt, converted messages, and tools. + // The invoker received the system prompt, the pre-encoded messages + // verbatim (encoding is the composer's job), and the wrapped tools. expect(capture[0].system).toBe("SYS"); - expect(capture[0].messages[0]).toMatchObject({ role: "user" }); + expect(capture[0].messages).toEqual([{ role: "user", content: "hello" }]); expect(Object.keys(capture[0].tools)).toEqual(["echo"]); }); @@ -152,6 +153,41 @@ describe("RealModelRegistry", () => { ).rejects.toThrowError("rate limited"); }); + it("encodeMessages produces the LLM-facing wire form (context woven, tool results enveloped)", async () => { + const registry = makeRegistry([], []); + const model = await registry.resolve({ provider: "openai", model: "gpt-test" }); + const encoded = model.encodeMessages([ + { + role: "user", + content: "list my downloads", + userMessageContext: { + currentDateTime: "2026-07-02T10:30:00Z", + middlePane: { kind: "empty" }, + }, + }, + { role: "tool", content: "[…files…]", toolCallId: "tc1", toolName: "file-list" }, + ]) as Array<{ role: string; content: unknown }>; + + // The user message is the woven wire text, not the internal structure. + expect(encoded[0].role).toBe("user"); + const userText = String(encoded[0].content); + expect(userText).toContain("2026-07-02T10:30:00Z"); + expect(userText).toContain("list my downloads"); + expect(userText).not.toContain("userMessageContext"); + + // Tool output rides the AI SDK tool-result envelope. + expect(encoded[1]).toMatchObject({ + role: "tool", + content: [ + { + type: "tool-result", + toolCallId: "tc1", + output: { type: "text", value: "[…files…]" }, + }, + ], + }); + }); + it("stops promptly when the signal aborts mid-stream", async () => { const controller = new AbortController(); const registry = makeRegistry( diff --git a/apps/x/packages/core/src/turns/bridges/real-model-registry.ts b/apps/x/packages/core/src/turns/bridges/real-model-registry.ts index 3ad9abf4..f79ef32d 100644 --- a/apps/x/packages/core/src/turns/bridges/real-model-registry.ts +++ b/apps/x/packages/core/src/turns/bridges/real-model-registry.ts @@ -64,6 +64,12 @@ export class RealModelRegistry implements IModelRegistry { const model = provider.languageModel(descriptor.model); return { descriptor, + // The structural -> wire conversion the app uses today: weaves + // userMessageContext into the user text, renders attachments, + // wraps tool output as tool-result parts. Deterministic and + // per-message, so composed requests are byte-stable. + encodeMessages: (messages) => + convertFromMessages(messages) as unknown as JsonValue[], stream: (request) => this.run(model, request), }; } @@ -90,7 +96,7 @@ export class RealModelRegistry implements IModelRegistry { const result = this.invoke({ model, system: request.systemPrompt, - messages: convertFromMessages(request.messages), + messages: request.messages as ModelMessage[], tools, abortSignal: request.signal, }); diff --git a/apps/x/packages/core/src/turns/compose-model-request.ts b/apps/x/packages/core/src/turns/compose-model-request.ts new file mode 100644 index 00000000..ec0a7ef7 --- /dev/null +++ b/apps/x/packages/core/src/turns/compose-model-request.ts @@ -0,0 +1,61 @@ +import type { z } from "zod"; +import { + type ConversationMessage, + type JsonValue, + type ToolDescriptor, + type TurnState, + requestMessagesFor, +} from "@x/shared/dist/turns.js"; +import type { IContextResolver } from "./context-resolver.js"; + +// The exact provider payload for one model call, rebuilt deterministically +// from durable state (turn-runtime-design.md §8.3): +// - systemPrompt and tools come from the resolved agent snapshot (their +// single canonical copy in turn_created), +// - messages are the cross-turn prefix plus every request's reference list +// resolved against the turn's own events, encoded to wire form. +// This is the SAME code path the loop sends through, so the debug view and +// the transmitted bytes cannot diverge. +export interface ComposedModelRequest { + systemPrompt: string; + messages: JsonValue[]; + tools: Array>; + parameters: Record; +} + +export function composeModelRequest( + state: TurnState, + modelCallIndex: number, + // The materialized cross-turn prefix (contextResolver output). Ignored + // for inline-context turns, whose context rides the {context} ref. + resolvedPrefix: Array>, + encode: (messages: Array>) => JsonValue[], +): ComposedModelRequest { + const call = state.modelCalls[modelCallIndex]; + if (!call) { + throw new Error(`no model call at index ${modelCallIndex}`); + } + const prefix = Array.isArray(state.definition.context) ? [] : resolvedPrefix; + const structural = [...prefix]; + for (let index = 0; index <= modelCallIndex; index++) { + structural.push(...requestMessagesFor(state, index)); + } + return { + systemPrompt: state.definition.agent.resolved.systemPrompt, + messages: encode(structural), + tools: state.definition.agent.resolved.tools, + parameters: call.request.parameters, + }; +} + +// Debug/materialization convenience: compose from durable state alone, +// resolving the cross-turn prefix through the context resolver. +export async function materializeModelRequest( + state: TurnState, + modelCallIndex: number, + contextResolver: IContextResolver, + encode: (messages: Array>) => JsonValue[], +): Promise { + const prefix = await contextResolver.resolve(state.definition.context); + return composeModelRequest(state, modelCallIndex, prefix, encode); +} diff --git a/apps/x/packages/core/src/turns/context-resolver.test.ts b/apps/x/packages/core/src/turns/context-resolver.test.ts index 39911786..78cb34bd 100644 --- a/apps/x/packages/core/src/turns/context-resolver.test.ts +++ b/apps/x/packages/core/src/turns/context-resolver.test.ts @@ -57,12 +57,11 @@ function completedTurnLog( ts, modelCallIndex: 0, request: { - systemPrompt: "SYS", ...(Array.isArray(context) ? {} : { contextRef: context }), - messages: Array.isArray(context) - ? [...context, user(inputText)] - : [user(inputText)], - tools: [], + messages: + Array.isArray(context) && context.length > 0 + ? [{ kind: "context" as const }, { kind: "input" as const }] + : [{ kind: "input" as const }], parameters: {}, }, }, @@ -139,9 +138,7 @@ function limitFailedTurnLog(turnId: string): TEvent[] { ts, modelCallIndex: 0, request: { - systemPrompt: "SYS", - messages: [user("do it")], - tools: [echo], + messages: [{ kind: "input" as const }], parameters: {}, }, }, diff --git a/apps/x/packages/core/src/turns/fs-repo.test.ts b/apps/x/packages/core/src/turns/fs-repo.test.ts index 8e44b5b1..b4afe769 100644 --- a/apps/x/packages/core/src/turns/fs-repo.test.ts +++ b/apps/x/packages/core/src/turns/fs-repo.test.ts @@ -41,9 +41,7 @@ function requested(turnId = TURN_ID): z.infer { ts: "2026-07-02T10:00:01Z", modelCallIndex: 0, request: { - systemPrompt: "SYS", - messages: [{ role: "user", content: "hello" }], - tools: [], + messages: [{ kind: "input" }], parameters: {}, }, }; diff --git a/apps/x/packages/core/src/turns/index.ts b/apps/x/packages/core/src/turns/index.ts index e7cbccf3..1ec99055 100644 --- a/apps/x/packages/core/src/turns/index.ts +++ b/apps/x/packages/core/src/turns/index.ts @@ -2,6 +2,7 @@ export * from "./api.js"; export * from "./agent-resolver.js"; export * from "./bus.js"; export * from "./clock.js"; +export * from "./compose-model-request.js"; export * from "./context-resolver.js"; export * from "./fs-repo.js"; export * from "./in-memory-turn-repo.js"; diff --git a/apps/x/packages/core/src/turns/model-registry.ts b/apps/x/packages/core/src/turns/model-registry.ts index d87aa097..d4c83a76 100644 --- a/apps/x/packages/core/src/turns/model-registry.ts +++ b/apps/x/packages/core/src/turns/model-registry.ts @@ -27,7 +27,10 @@ export type LlmStreamEvent = export interface ModelStreamRequest { systemPrompt: string; - messages: Array>; + // Provider wire-form messages: the output of encodeMessages. The loop + // builds these through the shared request composer, so what is sent is + // exactly what composeModelRequest reproduces from the durable log. + messages: JsonValue[]; tools: Array>; parameters: Record; signal: AbortSignal; @@ -35,6 +38,11 @@ export interface ModelStreamRequest { export interface ResolvedModel { descriptor: z.infer; + // Deterministic per-message structural -> wire conversion (e.g. weaving + // userMessageContext into the user text, tool-result enveloping). + encodeMessages( + messages: Array>, + ): JsonValue[]; stream(request: ModelStreamRequest): AsyncIterable; } diff --git a/apps/x/packages/core/src/turns/runtime.test.ts b/apps/x/packages/core/src/turns/runtime.test.ts index dc21b55d..3fad9d4b 100644 --- a/apps/x/packages/core/src/turns/runtime.test.ts +++ b/apps/x/packages/core/src/turns/runtime.test.ts @@ -3,10 +3,12 @@ import type { z } from "zod"; import { MODEL_CALL_LIMIT_ERROR_CODE, type JsonValue, + type ModelRequestMessageRef, type ResolvedAgent, type ToolDescriptor, type TurnEvent, type TurnStreamEvent, + reduceTurn, } from "@x/shared/dist/turns.js"; import type { IAgentResolver } from "./agent-resolver.js"; import type { TurnExecution, TurnOutcome } from "./api.js"; @@ -28,6 +30,7 @@ import type { PermissionClassificationBatch, PermissionClassificationInput, } from "./permission.js"; +import { composeModelRequest } from "./compose-model-request.js"; import { TurnRuntime } from "./runtime.js"; import type { IToolRegistry, @@ -153,6 +156,8 @@ class FakeModelRegistry implements IModelRegistry { ): Promise { return { descriptor, + // Identity encoding: tests assert on structural messages directly. + encodeMessages: (messages) => messages as unknown as JsonValue[], stream: (request) => { this.requests.push(request); const call = this.calls[this.next++]; @@ -365,6 +370,15 @@ function typesOf(events: Array<{ type: string }>): string[] { return events.map((e) => e.type); } +// The fake model's identity encoding passes structural messages through. +function sentMessages(request: ModelStreamRequest) { + return request.messages as Array<{ + role: string; + content?: unknown; + toolCallId?: string; + }>; +} + async function persisted( repo: InMemoryTurnRepo, turnId: string, @@ -378,7 +392,7 @@ async function persisted( describe("plain model response (26.1)", () => { it("runs one model step to completion with exact persisted request", async () => { - const { runtime, repo, bus } = makeRuntime({ + const { runtime, repo, models, bus } = makeRuntime({ models: [ respond( { type: "text_delta", delta: "do" }, @@ -408,11 +422,12 @@ describe("plain model response (26.1)", () => { const request = log[1]; expect(request).toMatchObject({ modelCallIndex: 0, - request: { - systemPrompt: "SYS", - messages: [user("hello")], - }, + request: { messages: [{ kind: "input" }] }, }); + // The model received the composed payload: resolved system prompt, + // snapshot tools, encoded messages. + expect(models.requests[0].systemPrompt).toBe("SYS"); + expect(sentMessages(models.requests[0])).toEqual([user("hello")]); // Deltas are streamed but never persisted. expect(events.filter((e) => e.type === "text_delta")).toHaveLength(2); expect(typesOf(log)).not.toContain("text_delta"); @@ -505,19 +520,39 @@ describe("mixed sync and async tools (26.2)", () => { (e) => e.type === "model_call_requested" && e.modelCallIndex === 1, ); expect(secondRequest).toBeDefined(); - const toolMessages = - secondRequest?.type === "model_call_requested" - ? secondRequest.request.messages.filter((m) => m.role === "tool") - : []; - expect(toolMessages.map((m) => (m.role === "tool" ? m.toolCallId : ""))).toEqual( - ["A", "B", "C", "D"], - ); - // The live model call saw the same ordering. expect( - models.requests[1].messages + secondRequest?.type === "model_call_requested" + ? secondRequest.request.messages + : [], + ).toEqual([ + { kind: "assistant", modelCallIndex: 0 }, + { kind: "toolResult", toolCallId: "A" }, + { kind: "toolResult", toolCallId: "B" }, + { kind: "toolResult", toolCallId: "C" }, + { kind: "toolResult", toolCallId: "D" }, + ]); + // The live model call saw the results in source order. + expect( + sentMessages(models.requests[1]) .filter((m) => m.role === "tool") - .map((m) => (m.role === "tool" ? m.toolCallId : "")), + .map((m) => m.toolCallId), ).toEqual(["A", "B", "C", "D"]); + // Byte-for-byte property: the durable file plus the shared composer + // reproduces exactly what the model received. + const state = reduceTurn(log); + for (const index of [0, 1]) { + const composed = composeModelRequest(state, index, [], (m) => m as never); + expect(composed.messages).toEqual(models.requests[index].messages); + expect(composed.systemPrompt).toBe(models.requests[index].systemPrompt); + expect(composed.tools).toEqual(models.requests[index].tools); + } + // Size guard: request events stay reference-sized — the transcript + // duplication this design removes must not creep back. + for (const event of log) { + if (event.type === "model_call_requested") { + expect(JSON.stringify(event).length).toBeLessThan(2048); + } + } }); it("rejects async results for calls that are not pending", async () => { @@ -1129,16 +1164,17 @@ describe("crash recovery (26.7)", () => { }; } - function seedRequested(index: number, messages: Array | ReturnType | { role: "assistant"; content: string } | { role: "tool"; content: string; toolCallId: string; toolName: string }>): z.infer { + function seedRequested( + index: number, + refs: z.infer[] = [{ kind: "input" }], + ): z.infer { return { type: "model_call_requested", turnId: SEED_ID, ts: TS, modelCallIndex: index, request: { - systemPrompt: "SYS", - messages, - tools: defaultAgent.tools, + messages: refs, parameters: {}, }, }; @@ -1169,7 +1205,7 @@ describe("crash recovery (26.7)", () => { it("an unmatched model request is closed as interrupted and re-issued", async () => { const repo = new InMemoryTurnRepo(); - repo.seed([seedCreated(), seedRequested(0, [user("hello")])]); + repo.seed([seedCreated(), seedRequested(0)]); const { runtime, models } = makeRuntime({ repo, models: [respond(completedResp(assistantText("done")))], @@ -1197,7 +1233,7 @@ describe("crash recovery (26.7)", () => { humanAvailable: true, maxModelCalls: 1, }), - seedRequested(0, [user("hello")]), + seedRequested(0), ]); const { runtime } = makeRuntime({ repo }); const { outcome } = await advanceAndSettle(runtime, SEED_ID); @@ -1212,7 +1248,7 @@ describe("crash recovery (26.7)", () => { const batch = assistantCalls(toolCallPart("S", "echo")); repo.seed([ seedCreated(), - seedRequested(0, [user("hello")]), + seedRequested(0), seedCompleted(0, batch), { type: "tool_invocation_requested", @@ -1248,7 +1284,7 @@ describe("crash recovery (26.7)", () => { const batch = assistantCalls(toolCallPart("B", "fetch")); repo.seed([ seedCreated(), - seedRequested(0, [user("hello")]), + seedRequested(0), seedCompleted(0, batch), { type: "tool_invocation_requested", @@ -1288,7 +1324,7 @@ describe("crash recovery (26.7)", () => { const batch = assistantCalls(toolCallPart("S", "echo")); repo.seed([ seedCreated(), - seedRequested(0, [user("hello")]), + seedRequested(0), seedCompleted(0, batch), { type: "tool_invocation_requested", @@ -1317,7 +1353,7 @@ describe("crash recovery (26.7)", () => { const { outcome } = await advanceAndSettle(runtime, SEED_ID); expect(outcome?.status).toBe("completed"); expect( - models.requests[0].messages.filter((m) => m.role === "tool"), + sentMessages(models.requests[0]).filter((m) => m.role === "tool"), ).toHaveLength(1); }); @@ -1326,7 +1362,7 @@ describe("crash recovery (26.7)", () => { const batch = assistantCalls(toolCallPart("P", "echo")); repo.seed([ seedCreated(), - seedRequested(0, [user("hello")]), + seedRequested(0), seedCompleted(0, batch), { type: "tool_permission_required", diff --git a/apps/x/packages/core/src/turns/runtime.ts b/apps/x/packages/core/src/turns/runtime.ts index f8682f1c..fd7caaf4 100644 --- a/apps/x/packages/core/src/turns/runtime.ts +++ b/apps/x/packages/core/src/turns/runtime.ts @@ -6,6 +6,7 @@ import { type JsonValue, type ModelCallFailed, type ModelRequest, + type ModelRequestMessageRef, type ToolCallState, type ToolDescriptor, type ToolInvocationRequested, @@ -21,7 +22,6 @@ import { outstandingAsyncTools, outstandingPermissions, reduceTurn, - turnTranscript, } from "@x/shared/dist/turns.js"; import type { IMonotonicallyIncreasingIdGenerator } from "../application/lib/id-gen.js"; import type { IAgentResolver } from "./agent-resolver.js"; @@ -37,6 +37,7 @@ import { } from "./api.js"; import type { ITurnLifecycleBus } from "./bus.js"; import type { IClock } from "./clock.js"; +import { composeModelRequest } from "./compose-model-request.js"; import type { IContextResolver } from "./context-resolver.js"; import type { IModelRegistry, @@ -847,17 +848,39 @@ class TurnAdvance { // §8.3/§18h–l: one model step. The durable request barrier precedes the // provider call; step events persist before the next provider read; - // deltas bypass storage. Returns an outcome only on failure/cancel. + // deltas bypass storage. The request records only REFERENCES to what is + // new since the previous call; the payload actually sent is built by the + // shared composer over the durable state, so file + composer reproduce + // the wire bytes exactly. Returns an outcome only on failure/cancel. private async runModelStep(): Promise { const index = this.state.modelCalls.length; - const transcript = turnTranscript(this.state); const context = this.definition.context; const isRef = !Array.isArray(context); + let refs: Array>; + if (index === 0) { + refs = + !isRef && context.length > 0 + ? [{ kind: "context" }, { kind: "input" }] + : [{ kind: "input" }]; + } else { + const previous = this.state.modelCalls[index - 1]; + refs = + previous.response !== undefined + ? [ + { kind: "assistant", modelCallIndex: previous.index }, + ...this.state.toolCalls + .filter((tc) => tc.modelCallIndex === previous.index) + .sort((a, b) => a.order - b.order) + .map((tc) => ({ + kind: "toolResult" as const, + toolCallId: tc.toolCallId, + })), + ] + : []; // re-issue after an interrupted call adds nothing new + } const request: z.infer = { - systemPrompt: this.definition.agent.resolved.systemPrompt, - ...(isRef ? { contextRef: context } : {}), - messages: isRef ? transcript : [...context, ...transcript], - tools: this.definition.agent.resolved.tools, + ...(isRef && index === 0 ? { contextRef: context } : {}), + messages: refs, parameters: {}, }; await this.append({ @@ -868,14 +891,21 @@ class TurnAdvance { request, }); + const composed = composeModelRequest( + this.state, + index, + this.resolvedContext, + (messages) => this.model.encodeMessages(messages), + ); + let completion: Extract | null = null; try { for await (const event of this.model.stream({ - systemPrompt: request.systemPrompt, - messages: [...this.resolvedContext, ...transcript], - tools: request.tools, - parameters: request.parameters, + systemPrompt: composed.systemPrompt, + messages: composed.messages, + tools: composed.tools, + parameters: composed.parameters, signal: this.signal, })) { switch (event.type) { diff --git a/apps/x/packages/shared/src/turns.test.ts b/apps/x/packages/shared/src/turns.test.ts index 3c77faf5..4f5f4d11 100644 --- a/apps/x/packages/shared/src/turns.test.ts +++ b/apps/x/packages/shared/src/turns.test.ts @@ -118,9 +118,7 @@ function requested( ts: TS, modelCallIndex: index, request: { - systemPrompt: "SYS", messages, - tools: [echoTool, fetchTool], parameters: {}, ...requestOverrides, }, @@ -348,13 +346,13 @@ function syncToolSequence(): TEvent[] { const call0 = assistantCalls(toolCallPart("tc1", "echo")); return [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), permResolved("tc1", "allow"), invocation("tc1"), result("tc1", "echo"), - requested(1, [user("hello"), call0, toolMsg("tc1", "echo")]), + requested(1, [{ kind: "assistant", modelCallIndex: 0 }, { kind: "toolResult", toolCallId: "tc1" }]), completed(1, assistantText("done")), turnCompletedEv(), ]; @@ -395,7 +393,7 @@ describe("plain completion", () => { it("reduces a plain model response to a completed turn", () => { const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), stepEvent(0), completed(0, assistantText("done")), turnCompletedEv(), @@ -420,7 +418,7 @@ describe("plain completion", () => { it("leaves usage fields undefined when never reported", () => { const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, assistantText("a"), { inputTokens: 5 }), ]); expect(state.usage).toEqual({ inputTokens: 5 }); @@ -436,7 +434,7 @@ describe("tool execution", () => { ); const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), ]); expect(state.toolCalls).toHaveLength(2); @@ -459,7 +457,7 @@ describe("tool execution", () => { it("leaves identity undefined for tools missing from the agent snapshot", () => { const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, assistantCalls(toolCallPart("x1", "unknown-tool"))), result("x1", "unknown-tool", "runtime", "No such tool", true), ]); @@ -482,7 +480,7 @@ describe("tool execution", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created({ config: { autoPermission: true, humanAvailable: true, maxModelCalls: 20 } }), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), permClassified("tc1", "allow"), @@ -499,7 +497,7 @@ describe("tool execution", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), permClassificationFailed(["tc1"]), @@ -514,7 +512,7 @@ describe("tool execution", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), progress("tc1"), @@ -528,7 +526,7 @@ describe("tool execution", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), permResolved("tc1", "deny"), @@ -543,7 +541,7 @@ describe("context references", () => { it("accepts a referenced context with matching contextRef on requests", () => { const state = reduceTurn([ created({ context: { previousTurnId: PREV_TURN_ID } }), - requested(0, [user("hello")], { + requested(0, [{ kind: "input" }], { contextRef: { previousTurnId: PREV_TURN_ID }, }), completed(0, assistantText("done")), @@ -556,7 +554,7 @@ describe("context references", () => { expectCorruption( [ created({ context: { previousTurnId: PREV_TURN_ID } }), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), ], /contextRef inconsistent/, ); @@ -566,7 +564,7 @@ describe("context references", () => { expectCorruption( [ created({ context: { previousTurnId: PREV_TURN_ID } }), - requested(0, [user("hello")], { + requested(0, [{ kind: "input" }], { contextRef: { previousTurnId: "some-other-turn" }, }), ], @@ -578,7 +576,7 @@ describe("context references", () => { expectCorruption( [ created(), - requested(0, [user("hello")], { + requested(0, [{ kind: "input" }], { contextRef: { previousTurnId: PREV_TURN_ID }, }), ], @@ -586,7 +584,7 @@ describe("context references", () => { ); }); - it("ignores tool messages inside an inline context prefix when checking order", () => { + it("requires a {context} ref before {input} when inline context is nonempty", () => { const context = [ user("earlier"), assistantCalls(toolCallPart("old1", "echo")), @@ -594,11 +592,38 @@ describe("context references", () => { ]; const state = reduceTurn([ created({ context }), - requested(0, [...context, user("hello")]), + requested(0, [{ kind: "context" }, { kind: "input" }]), completed(0, assistantText("done")), turnCompletedEv(), ]); expect(deriveTurnStatus(state)).toBe("completed"); + expectCorruption( + [created({ context }), requested(0, [{ kind: "input" }])], + /references do not match/, + ); + }); + + it("rejects a contextRef on a non-initial model call", () => { + expectCorruption( + [ + created({ context: { previousTurnId: PREV_TURN_ID } }), + requested(0, [{ kind: "input" }], { + contextRef: { previousTurnId: PREV_TURN_ID }, + }), + completed(0, assistantCalls(toolCallPart("tc1", "echo"))), + invocation("tc1"), + result("tc1", "echo"), + requested( + 1, + [ + { kind: "assistant", modelCallIndex: 0 }, + { kind: "toolResult", toolCallId: "tc1" }, + ], + { contextRef: { previousTurnId: PREV_TURN_ID } }, + ), + ], + /contextRef on a non-initial model call/, + ); }); }); @@ -610,7 +635,7 @@ describe("suspension", () => { ); const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), invocation("a1", fetchTool), @@ -628,7 +653,7 @@ describe("suspension", () => { ); const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), invocation("a1", fetchTool), @@ -648,7 +673,7 @@ describe("suspension", () => { ); const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("a1", fetchTool), invocation("a2", fetchTool), @@ -657,10 +682,9 @@ describe("suspension", () => { suspendedEv([], [{ id: "a1", tool: fetchTool }]), result("a1", "fetch", "async", "first"), requested(1, [ - user("hello"), - call0, - toolMsg("a1", "fetch", "first"), - toolMsg("a2", "fetch", "second"), + { kind: "assistant", modelCallIndex: 0 }, + { kind: "toolResult", toolCallId: "a1" }, + { kind: "toolResult", toolCallId: "a2" }, ]), completed(1, assistantText("done")), turnCompletedEv(), @@ -673,7 +697,7 @@ describe("suspension", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), permResolved("tc1", "allow"), @@ -693,7 +717,7 @@ describe("suspension", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), invocation("a1", fetchTool), @@ -707,7 +731,7 @@ describe("suspension", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), suspendedEv([{ id: "tc1", name: "echo" }], []), ], /suspension while a model call is unsettled/, @@ -719,9 +743,9 @@ describe("recovery-shaped histories", () => { it("accepts an interrupted model call closed and re-issued (§23 fix)", () => { const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), callFailed(0, "interrupted by process restart"), - requested(1, [user("hello")]), + requested(1, []), completed(1, assistantText("done")), turnCompletedEv(), ]); @@ -736,9 +760,9 @@ describe("recovery-shaped histories", () => { created({ config: { autoPermission: false, humanAvailable: true, maxModelCalls: 1 }, }), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), callFailed(0, "interrupted"), - requested(1, [user("hello")]), + requested(1, []), ], /exceeds maxModelCalls/, ); @@ -748,7 +772,7 @@ describe("recovery-shaped histories", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), result( @@ -758,7 +782,7 @@ describe("recovery-shaped histories", () => { "Tool execution was interrupted; its outcome is unknown and it was not retried.", true, ), - requested(1, [user("hello"), call0, toolMsg("tc1", "echo", "…")]), + requested(1, [{ kind: "assistant", modelCallIndex: 0 }, { kind: "toolResult", toolCallId: "tc1" }]), completed(1, assistantText("done")), turnCompletedEv(), ]); @@ -770,7 +794,7 @@ describe("recovery-shaped histories", () => { const call0 = assistantCalls(toolCallPart("a1", "fetch")); const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("a1", fetchTool), suspendedEv([], [{ id: "a1", tool: fetchTool }]), @@ -784,7 +808,7 @@ describe("recovery-shaped histories", () => { it("accepts a live model failure closing the turn", () => { const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), callFailed(0), turnFailedEv("provider exploded"), ]); @@ -797,7 +821,7 @@ describe("recovery-shaped histories", () => { created({ config: { autoPermission: false, humanAvailable: true, maxModelCalls: 1 }, }), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), @@ -816,7 +840,7 @@ describe("invariants", () => { }); it("rejects a log not starting with turn_created", () => { - expectCorruption([requested(0, [user("hello")])], /first event must be turn_created/); + expectCorruption([requested(0, [{ kind: "input" }])], /first event must be turn_created/); }); it("rejects duplicate turn_created", () => { @@ -825,7 +849,7 @@ describe("invariants", () => { it("rejects mismatched turn ids", () => { expectCorruption( - [created(), { ...requested(0, [user("hello")]), turnId: "other" }], + [created(), { ...requested(0, [{ kind: "input" }]), turnId: "other" }], /does not match turn/, ); }); @@ -849,7 +873,7 @@ describe("invariants", () => { it("rejects out-of-order model call indices", () => { expectCorruption( - [created(), requested(1, [user("hello")])], + [created(), requested(1, [])], /out of order/, ); }); @@ -858,9 +882,9 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, assistantText("a")), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), ], /out of order/, ); @@ -868,7 +892,7 @@ describe("invariants", () => { it("rejects concurrent unresolved model requests", () => { expectCorruption( - [created(), requested(0, [user("hello")]), requested(1, [user("hello")])], + [created(), requested(0, [{ kind: "input" }]), requested(1, [])], /concurrent unresolved model call requests/, ); }); @@ -888,7 +912,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, assistantText("a")), completed(0, assistantText("b")), ], @@ -900,7 +924,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), callFailed(0), completed(0, assistantText("a")), ], @@ -913,9 +937,9 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), - requested(1, [user("hello"), call0]), + requested(1, [{ kind: "assistant", modelCallIndex: 0 }]), ], /while tool calls are unresolved/, ); @@ -928,11 +952,11 @@ describe("invariants", () => { created({ config: { autoPermission: false, humanAvailable: true, maxModelCalls: 1 }, }), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), - requested(1, [user("hello"), call0, toolMsg("tc1", "echo")]), + requested(1, [{ kind: "assistant", modelCallIndex: 0 }, { kind: "toolResult", toolCallId: "tc1" }]), ], /exceeds maxModelCalls/, ); @@ -942,7 +966,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed( 0, assistantCalls(toolCallPart("tc1", "echo"), toolCallPart("tc1", "echo")), @@ -957,18 +981,18 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), - requested(1, [user("hello"), call0, toolMsg("tc1", "echo")]), + requested(1, [{ kind: "assistant", modelCallIndex: 0 }, { kind: "toolResult", toolCallId: "tc1" }]), completed(1, assistantCalls(toolCallPart("tc1", "echo"))), ], /duplicate tool call id/, ); }); - it("rejects tool-result ordering that differs from original call order", () => { + it("rejects request references whose tool-result order differs from the batch", () => { const call0 = assistantCalls( toolCallPart("tc1", "echo"), toolCallPart("tc2", "echo"), @@ -976,26 +1000,25 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), invocation("tc2"), result("tc2", "echo"), requested(1, [ - user("hello"), - call0, - toolMsg("tc2", "echo"), - toolMsg("tc1", "echo"), + { kind: "assistant", modelCallIndex: 0 }, + { kind: "toolResult", toolCallId: "tc2" }, + { kind: "toolResult", toolCallId: "tc1" }, ]), ], - /tool-result order differs/, + /references do not match/, ); }); it("rejects permission records targeting unknown tool calls", () => { expectCorruption( - [created(), requested(0, [user("hello")]), completed(0, assistantText("a")), permRequired("ghost", "echo")], + [created(), requested(0, [{ kind: "input" }]), completed(0, assistantText("a")), permRequired("ghost", "echo")], /unknown tool call/, ); }); @@ -1005,7 +1028,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), permRequired("tc1", "echo"), @@ -1019,7 +1042,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permClassified("tc1", "allow"), ], @@ -1032,7 +1055,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permResolved("tc1", "allow"), ], @@ -1045,7 +1068,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), permResolved("tc1", "allow"), @@ -1060,7 +1083,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), invocation("tc1"), @@ -1074,7 +1097,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), permResolved("tc1", "deny"), @@ -1089,7 +1112,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), invocation("tc1"), @@ -1103,7 +1126,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1", echoTool, { execution: "async" }), ], @@ -1114,7 +1137,7 @@ describe("invariants", () => { it("rejects progress without invocation", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); expectCorruption( - [created(), requested(0, [user("hello")]), completed(0, call0), progress("tc1")], + [created(), requested(0, [{ kind: "input" }]), completed(0, call0), progress("tc1")], /progress without invocation/, ); }); @@ -1124,7 +1147,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), @@ -1139,7 +1162,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), @@ -1152,7 +1175,7 @@ describe("invariants", () => { it("rejects sync-sourced results without invocation", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); expectCorruption( - [created(), requested(0, [user("hello")]), completed(0, call0), result("tc1", "echo", "sync")], + [created(), requested(0, [{ kind: "input" }]), completed(0, call0), result("tc1", "echo", "sync")], /sync tool result without invocation/, ); }); @@ -1162,7 +1185,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), result("tc1", "echo", "async"), @@ -1176,7 +1199,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, assistantText("a")), stepEvent(0), ], @@ -1187,7 +1210,7 @@ describe("invariants", () => { it("rejects completion while tool calls remain unresolved", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); expectCorruption( - [created(), requested(0, [user("hello")]), completed(0, call0), turnCompletedEv()], + [created(), requested(0, [{ kind: "input" }]), completed(0, call0), turnCompletedEv()], /completion while tool calls lack terminal results/, ); }); @@ -1197,7 +1220,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), @@ -1214,14 +1237,14 @@ describe("invariants", () => { it("rejects terminal failure while tool calls remain unresolved", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); expectCorruption( - [created(), requested(0, [user("hello")]), completed(0, call0), turnFailedEv()], + [created(), requested(0, [{ kind: "input" }]), completed(0, call0), turnFailedEv()], /failure while tool calls lack terminal results/, ); }); it("rejects terminal events while a model call is unsettled", () => { expectCorruption( - [created(), requested(0, [user("hello")]), turnCancelledEv()], + [created(), requested(0, [{ kind: "input" }]), turnCancelledEv()], /cancellation while a model call is unsettled/, ); }); @@ -1229,7 +1252,7 @@ describe("invariants", () => { it("rejects any event after a terminal event", () => { const base = [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, assistantText("a")), ]; for (const terminal of [turnCompletedEv(), turnFailedEv(), turnCancelledEv()]) { @@ -1244,7 +1267,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, assistantText("a")), turnCompletedEv(), turnFailedEv(), @@ -1259,14 +1282,14 @@ describe("derivations", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const pending = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), permRequired("tc1", "echo"), suspendedEv([{ id: "tc1", name: "echo" }], []), ]); expect(deriveTurnStatus(pending)).toBe("suspended"); - const idle = reduceTurn([created(), requested(0, [user("hello")]), completed(0, call0)]); + const idle = reduceTurn([created(), requested(0, [{ kind: "input" }]), completed(0, call0)]); expect(deriveTurnStatus(idle)).toBe("idle"); }); @@ -1277,17 +1300,16 @@ describe("derivations", () => { ); const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), invocation("tc1"), invocation("tc2"), result("tc2", "echo", "sync", { n: 2 }), result("tc1", "echo", "sync", "plain text"), requested(1, [ - user("hello"), - call0, - toolMsg("tc1", "echo", "plain text"), - toolMsg("tc2", "echo", '{"n":2}'), + { kind: "assistant", modelCallIndex: 0 }, + { kind: "toolResult", toolCallId: "tc1" }, + { kind: "toolResult", toolCallId: "tc2" }, ]), completed(1, assistantText("done")), turnCompletedEv(), @@ -1304,9 +1326,9 @@ describe("derivations", () => { it("omits failed model calls from the transcript", () => { const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), callFailed(0, "interrupted"), - requested(1, [user("hello")]), + requested(1, []), completed(1, assistantText("done")), turnCompletedEv(), ]); @@ -1316,7 +1338,7 @@ describe("derivations", () => { it("transcript excludes the context prefix", () => { const state = reduceTurn([ created({ context: { previousTurnId: PREV_TURN_ID } }), - requested(0, [user("hello")], { + requested(0, [{ kind: "input" }], { contextRef: { previousTurnId: PREV_TURN_ID }, }), completed(0, assistantText("done")), @@ -1330,7 +1352,7 @@ describe("derivations", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created(), - requested(0, [user("hello")]), + requested(0, [{ kind: "input" }]), completed(0, call0), ]); expect(() => turnTranscript(state)).toThrowError(/unresolved/); diff --git a/apps/x/packages/shared/src/turns.ts b/apps/x/packages/shared/src/turns.ts index b93ff3ec..3b71b0d3 100644 --- a/apps/x/packages/shared/src/turns.ts +++ b/apps/x/packages/shared/src/turns.ts @@ -119,16 +119,30 @@ export const TurnCreated = z.object({ }), }); -// `messages` contains current-turn messages only; when the turn's context is -// a reference, `contextRef` repeats it and the resolved prefix is implicit -// (deterministically materializable because referenced turns are immutable). -// When context is inline, `contextRef` is absent and `messages` begins with -// the inline context. +// A model request is a list of REFERENCES into the turn's own events; every +// referenced byte exists exactly once in the file. The request records only +// what is NEW since the previous model call: +// call 0: [{context}?, {input}] (context only when inline and nonempty; +// cross-turn prefixes ride contextRef) +// call N: [{assistant: N-1}, ...that batch's toolResults in source order] +// re-issue after an interrupted call: [] +// The system prompt and tool set are NOT repeated here — they are byte- +// identical to turn_created.agent.resolved by construction. The exact +// provider payload is rebuilt deterministically by the request composer +// (core), which is the same code path the loop sends through. +export const ModelRequestMessageRef = z.discriminatedUnion("kind", [ + z.object({ kind: z.literal("context") }), + z.object({ kind: z.literal("input") }), + z.object({ + kind: z.literal("assistant"), + modelCallIndex: z.number().int().nonnegative(), + }), + z.object({ kind: z.literal("toolResult"), toolCallId: z.string() }), +]); + export const ModelRequest = z.object({ - systemPrompt: z.string(), contextRef: TurnContextRef.optional(), - messages: z.array(ConversationMessage), - tools: z.array(ToolDescriptor), + messages: z.array(ModelRequestMessageRef), parameters: z.record(z.string(), z.json()), }); @@ -491,32 +505,44 @@ function applyModelCallRequested( } const context = state.definition.context; - if (isContextRef(context)) { - if (event.request.contextRef?.previousTurnId !== context.previousTurnId) { - fail("model request contextRef inconsistent with turn context"); + let expectedRefs: Array>; + if (event.modelCallIndex === 0) { + if (isContextRef(context)) { + if (event.request.contextRef?.previousTurnId !== context.previousTurnId) { + fail("model request contextRef inconsistent with turn context"); + } + expectedRefs = [{ kind: "input" }]; + } else { + if (event.request.contextRef !== undefined) { + fail("model request has contextRef but turn context is inline"); + } + expectedRefs = + context.length > 0 + ? [{ kind: "context" }, { kind: "input" }] + : [{ kind: "input" }]; } - } else if (event.request.contextRef !== undefined) { - fail("model request has contextRef but turn context is inline"); + } else { + if (event.request.contextRef !== undefined) { + fail("model request has contextRef on a non-initial model call"); + } + const previous = state.modelCalls[event.modelCallIndex - 1]; + expectedRefs = + previous.response !== undefined + ? [ + { kind: "assistant", modelCallIndex: previous.index }, + ...batchToolCalls(state, previous.index).map((tc) => ({ + kind: "toolResult" as const, + toolCallId: tc.toolCallId, + })), + ] + : []; // re-issue after an interrupted call adds nothing new } - - // Model-facing tool-result order must match original call order: the tool - // messages in the request (past any inline context prefix) must be the - // concatenated batches of prior completed calls, in source order. - const prefixLength = isContextRef(context) ? 0 : context.length; - const toolMessageIds = event.request.messages - .slice(prefixLength) - .filter((m) => m.role === "tool") - .map((m) => m.toolCallId); - const expectedIds = state.modelCalls - .filter((call) => call.response !== undefined) - .flatMap((call) => - batchToolCalls(state, call.index).map((tc) => tc.toolCallId), + if (JSON.stringify(event.request.messages) !== JSON.stringify(expectedRefs)) { + fail( + `model request references do not match the transcript: expected ${JSON.stringify( + expectedRefs, + )}, got ${JSON.stringify(event.request.messages)}`, ); - if ( - toolMessageIds.length !== expectedIds.length || - toolMessageIds.some((id, i) => id !== expectedIds[i]) - ) { - fail("model request tool-result order differs from original call order"); } state.modelCalls.push({ @@ -932,6 +958,69 @@ function toolResultContent(result: z.infer): string { return typeof output === "string" ? output : JSON.stringify(output); } +// The canonical model-facing tool message for a resolved tool call. +export function toolResultMessage( + toolCall: ToolCallState, +): z.infer { + if (!toolCall.result) { + throw new Error( + `tool call ${toolCall.toolCallId} has no terminal result`, + ); + } + return { + role: "tool", + content: toolResultContent(toolCall.result), + toolCallId: toolCall.toolCallId, + toolName: toolCall.toolName, + }; +} + +// Resolves one model call's request references to structural messages (the +// NEW portion that call added relative to the previous one). Concatenating +// calls 0..N yields the full current-turn conversation for call N; the +// request composer (core) prepends the resolved cross-turn prefix and +// encodes to the provider wire form. +export function requestMessagesFor( + state: TurnState, + modelCallIndex: number, +): Array> { + const call = state.modelCalls[modelCallIndex]; + if (!call) { + throw new Error(`no model call at index ${modelCallIndex}`); + } + return call.request.messages.flatMap((ref) => { + switch (ref.kind) { + case "context": { + const context = state.definition.context; + if (!Array.isArray(context)) { + throw new Error("context ref on a turn without inline context"); + } + return context; + } + case "input": + return [state.definition.input]; + case "assistant": { + const response = state.modelCalls[ref.modelCallIndex]?.response; + if (response === undefined) { + throw new Error( + `assistant ref to unsettled model call ${ref.modelCallIndex}`, + ); + } + return [response]; + } + case "toolResult": { + const toolCall = state.toolCalls.find( + (tc) => tc.toolCallId === ref.toolCallId, + ); + if (!toolCall) { + throw new Error(`toolResult ref to unknown call ${ref.toolCallId}`); + } + return [toolResultMessage(toolCall)]; + } + } + }); +} + // The messages this turn contributed to the conversation: its input plus, per // completed model call, the assistant response and its tool results in source // order. Failed model calls contribute nothing. The turn's context prefix is @@ -955,12 +1044,7 @@ export function turnTranscript( `turnTranscript requires terminal tool results; ${toolCall.toolCallId} is unresolved`, ); } - messages.push({ - role: "tool", - content: toolResultContent(toolCall.result), - toolCallId: toolCall.toolCallId, - toolName: toolCall.toolName, - }); + messages.push(toolResultMessage(toolCall)); } } return messages; From 0ab09d4d4740c0adb2d851afe15b700537d7a58c Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 12:47:09 +0530 Subject: [PATCH 15/24] chore(x): lockfile for renderer test dependencies Co-Authored-By: Claude Fable 5 --- apps/x/pnpm-lock.yaml | 510 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 504 insertions(+), 6 deletions(-) diff --git a/apps/x/pnpm-lock.yaml b/apps/x/pnpm-lock.yaml index edd7e7b1..cc6d0b32 100644 --- a/apps/x/pnpm-lock.yaml +++ b/apps/x/pnpm-lock.yaml @@ -338,7 +338,7 @@ importers: version: 3.3.0(react-dom@19.2.3(react@19.2.3))(react@19.2.3) recharts: specifier: ^3.8.0 - version: 3.8.1(@types/react@19.2.7)(react-dom@19.2.3(react@19.2.3))(react-is@16.13.1)(react@19.2.3)(redux@5.0.1) + version: 3.8.1(@types/react@19.2.7)(react-dom@19.2.3(react@19.2.3))(react-is@17.0.2)(react@19.2.3)(redux@5.0.1) remark-breaks: specifier: ^4.0.0 version: 4.0.0 @@ -373,6 +373,15 @@ importers: '@eslint/js': specifier: ^9.39.1 version: 9.39.2 + '@testing-library/dom': + specifier: ^10.4.1 + version: 10.4.1 + '@testing-library/jest-dom': + specifier: ^6.9.1 + version: 6.9.1 + '@testing-library/react': + specifier: ^16.3.2 + version: 16.3.2(@testing-library/dom@10.4.1)(@types/react-dom@19.2.3(@types/react@19.2.7))(@types/react@19.2.7)(react-dom@19.2.3(react@19.2.3))(react@19.2.3) '@types/node': specifier: ^24.10.1 version: 24.10.4 @@ -397,6 +406,9 @@ importers: globals: specifier: ^16.5.0 version: 16.5.0 + jsdom: + specifier: ^29.1.1 + version: 29.1.1 tw-animate-css: specifier: ^1.4.0 version: 1.4.0 @@ -409,6 +421,9 @@ importers: vite: specifier: ^7.2.4 version: 7.3.0(@types/node@24.10.4)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2) + vitest: + specifier: 'catalog:' + version: 4.1.7(@opentelemetry/api@1.9.0)(@types/node@24.10.4)(jsdom@29.1.1)(vite@7.3.0(@types/node@24.10.4)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2)) packages/core: dependencies: @@ -538,7 +553,7 @@ importers: version: 1.1.5 vitest: specifier: 'catalog:' - version: 4.1.7(@opentelemetry/api@1.9.0)(@types/node@25.0.3)(vite@7.3.0(@types/node@25.0.3)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2)) + version: 4.1.7(@opentelemetry/api@1.9.0)(@types/node@25.0.3)(jsdom@29.1.1)(vite@7.3.0(@types/node@25.0.3)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2)) packages/shared: dependencies: @@ -548,10 +563,13 @@ importers: devDependencies: vitest: specifier: 'catalog:' - version: 4.1.7(@opentelemetry/api@1.9.0)(@types/node@25.0.3)(vite@7.3.0(@types/node@25.0.3)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2)) + version: 4.1.7(@opentelemetry/api@1.9.0)(@types/node@25.0.3)(jsdom@29.1.1)(vite@7.3.0(@types/node@25.0.3)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2)) packages: + '@adobe/css-tools@4.5.0': + resolution: {integrity: sha512-6OzddxPio9UiWTCemp4N8cYLV2ZN1ncRnV1cVGtve7dhPOtRkleRyx32GQCYSwDYgaHU3USMm84tNsvKzRCa1Q==} + '@agentclientprotocol/claude-agent-acp@0.39.0': resolution: {integrity: sha512-+tCm5v32L0R3zE4qjZQowfO1L/zqvQ5FapmsMSIf4gawXfTf26CG5hgz99wARdo0zn20/1eP80gzx7PbZlSX9A==} hasBin: true @@ -682,6 +700,21 @@ packages: zod: optional: true + '@asamuzakjp/css-color@5.1.11': + resolution: {integrity: sha512-KVw6qIiCTUQhByfTd78h2yD1/00waTmm9uy/R7Ck/ctUyAPj+AEDLkQIdJW0T8+qGgj3j5bpNKK7Q3G+LedJWg==} + engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} + + '@asamuzakjp/dom-selector@7.1.1': + resolution: {integrity: sha512-67RZDnYRc8H/8MLDgQCDE//zoqVFwajkepHZgmXrbwybzXOEwOWGPYGmALYl9J2DOLfFPPs6kKCqmbzV895hTQ==} + engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} + + '@asamuzakjp/generational-cache@1.0.1': + resolution: {integrity: sha512-wajfB8KqzMCN2KGNFdLkReeHncd0AslUSrvHVvvYWuU8ghncRJoA50kT3zP9MVL0+9g4/67H+cdvBskj9THPzg==} + engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} + + '@asamuzakjp/nwsapi@2.3.9': + resolution: {integrity: sha512-n8GuYSrI9bF7FFZ/SjhwevlHc8xaVlb/7HmHelnc/PZXBD2ZR49NnN9sMMuDdEGPeeRQ5d0hqlSlEpgCX3Wl0Q==} + '@aws-crypto/crc32@5.2.0': resolution: {integrity: sha512-nLbCWqQNgUiwwtFsen1AdzAtvuLRsQS8rYgMuxCrdKf9kOssamGLuPwyTY9wyYblNr9+1XM8v6zoDTPPSIeANg==} engines: {node: '>=16.0.0'} @@ -941,6 +974,10 @@ packages: '@braintree/sanitize-url@7.1.1': resolution: {integrity: sha512-i1L7noDNxtFyL5DmZafWy1wRVhGehQmzZaz1HiN5e7iylJMSZR7ekOV7NsIqa5qBldlLrsKv4HbgFUVlQrz8Mw==} + '@bramus/specificity@2.4.2': + resolution: {integrity: sha512-ctxtJ/eA+t+6q2++vj5j7FYX3nRu311q1wfYH3xjlLOsczhlhxAg2FWNUXhpGvAw3BWo1xBcvOV6/YLc2r5FJw==} + hasBin: true + '@chevrotain/cst-dts-gen@12.0.0': resolution: {integrity: sha512-fSL4KXjTl7cDgf0B5Rip9Q05BOrYvkJV/RrBTE/bKDN096E4hN/ySpcBK5B24T76dlQ2i32Zc3PAE27jFnFrKg==} @@ -1062,6 +1099,42 @@ packages: peerDependencies: zod: '>=3.25.76 <5' + '@csstools/color-helpers@6.1.0': + resolution: {integrity: sha512-064IFJdjTfUqnjpCVpMOdbr8FLQBhinbZj6yRv2An2E41O/pLEXqfFRWqGq/SxlE5PEUYTlvWsG2r8MswAVvkg==} + engines: {node: '>=20.19.0'} + + '@csstools/css-calc@3.2.1': + resolution: {integrity: sha512-DtdHlgXh5ZkA43cwBcAm+huzgJiwx3ZTWVjBs94kwz2xKqSimDA3lBgCjphYgwgVUMWatSM0pDd8TILB1yrVVg==} + engines: {node: '>=20.19.0'} + peerDependencies: + '@csstools/css-parser-algorithms': ^4.0.0 + '@csstools/css-tokenizer': ^4.0.0 + + '@csstools/css-color-parser@4.1.9': + resolution: {integrity: sha512-paQcIaOO53Rk5+YrBaBjm/SgrV4INImjo2BT1DtQRYr+XeTRbeAYlS+jxXp9drqvKmtFnWRJKIalDLhZZDu42A==} + engines: {node: '>=20.19.0'} + peerDependencies: + '@csstools/css-parser-algorithms': ^4.0.0 + '@csstools/css-tokenizer': ^4.0.0 + + '@csstools/css-parser-algorithms@4.0.0': + resolution: {integrity: sha512-+B87qS7fIG3L5h3qwJ/IFbjoVoOe/bpOdh9hAjXbvx0o8ImEmUsGXN0inFOnk2ChCFgqkkGFQ+TpM5rbhkKe4w==} + engines: {node: '>=20.19.0'} + peerDependencies: + '@csstools/css-tokenizer': ^4.0.0 + + '@csstools/css-syntax-patches-for-csstree@1.1.6': + resolution: {integrity: sha512-TcJCWFbXLPpJYq6z7bfOyjWYJDiDg2/I4gyUC9pqPNqHFRIey0EB0q0L5cSnQDfWJg8Jd6VadakxdIez/3zkqQ==} + peerDependencies: + css-tree: ^3.2.1 + peerDependenciesMeta: + css-tree: + optional: true + + '@csstools/css-tokenizer@4.0.0': + resolution: {integrity: sha512-QxULHAm7cNu72w97JUNCBFODFaXpbDg+dP8b/oWFAZ2MTRppA3U00Y2L1HqaS4J6yBqxwa/Y3nMBaxVKbB/NsA==} + engines: {node: '>=20.19.0'} + '@eigenpal/docx-editor-agents@1.0.3': resolution: {integrity: sha512-Bk/J9/PBnMCOxb6w4cHQiCTuN/1C4FtZM9evC9EXXcLP13yFMdqoEqsYs+Lh3HyaRRAaCZTrkfgOZyTqqyjtwQ==} peerDependencies: @@ -1592,6 +1665,15 @@ packages: resolution: {integrity: sha512-43/qtrDUokr7LJqoF2c3+RInu/t4zfrpYdoSDfYyhg52rwLV6TnOvdG4fXm7IkSB3wErkcmJS9iEhjVtOSEjjA==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} + '@exodus/bytes@1.15.1': + resolution: {integrity: sha512-S6mL0yNB/Abt9Ei4tq8gDhcczc4S3+vQ4ra7vxnAf+YHC02srtqxKKZghx2Dq6p0e66THKwR6r8N6P95wEty7Q==} + engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} + peerDependencies: + '@noble/hashes': ^1.8.0 || ^2.0.0 + peerDependenciesMeta: + '@noble/hashes': + optional: true + '@floating-ui/core@1.7.3': resolution: {integrity: sha512-sGnvb5dmrJaKEZ+LDIpguvdX3bDlEllmv4/ClQ9awcmCZrlx5jQyyMWFM5kBI+EyNOCDDiKk8il0zeuX3Zlg/w==} @@ -3488,6 +3570,29 @@ packages: peerDependencies: vite: ^5.2.0 || ^6 || ^7 + '@testing-library/dom@10.4.1': + resolution: {integrity: sha512-o4PXJQidqJl82ckFaXUeoAW+XysPLauYI43Abki5hABd853iMhitooc6znOnczgbTYmEP6U6/y1ZyKAIsvMKGg==} + engines: {node: '>=18'} + + '@testing-library/jest-dom@6.9.1': + resolution: {integrity: sha512-zIcONa+hVtVSSep9UT3jZ5rizo2BsxgyDYU7WFD5eICBE7no3881HGeb/QkGfsJs6JTkY1aQhT7rIPC7e+0nnA==} + engines: {node: '>=14', npm: '>=6', yarn: '>=1'} + + '@testing-library/react@16.3.2': + resolution: {integrity: sha512-XU5/SytQM+ykqMnAnvB2umaJNIOsLF3PVv//1Ew4CTcpz0/BRyy/af40qqrt7SjKpDdT1saBMc42CUok5gaw+g==} + engines: {node: '>=18'} + peerDependencies: + '@testing-library/dom': ^10.0.0 + '@types/react': ^18.0.0 || ^19.0.0 + '@types/react-dom': ^18.0.0 || ^19.0.0 + react: ^18.0.0 || ^19.0.0 + react-dom: ^18.0.0 || ^19.0.0 + peerDependenciesMeta: + '@types/react': + optional: true + '@types/react-dom': + optional: true + '@tiptap/core@3.22.4': resolution: {integrity: sha512-vGIGm/HpqLg8EAAQXQ+koV+/S828OEpzocfWcPOwo1u2QUVf9dQG47Yy6JJ8zFFaJwfv4dBcOXli+7BrJwsxDQ==} peerDependencies: @@ -3682,6 +3787,9 @@ packages: '@types/appdmg@0.5.5': resolution: {integrity: sha512-G+n6DgZTZFOteITE30LnWj+HRVIGr7wMlAiLWOO02uJFWVEitaPU9JVXm9wJokkgshBawb2O1OykdcsmkkZfgg==} + '@types/aria-query@5.0.4': + resolution: {integrity: sha512-rfT93uj5s0PRL7EzccGMs3brplhcrghnDoV26NqKhCAS1hVo+WdNsPvE/yb6ilfr5hi2MEk6d5EWJTKdxg8jVw==} + '@types/babel__core@7.20.5': resolution: {integrity: sha512-qoQprZvz5wQFJwMDqeseRXWv3rqMvhgpbXFfVyWhbx9X47POIA6i/+dXefEmZKoAgOaTdaIgNSMqMIU61yRyzA==} @@ -4230,6 +4338,10 @@ packages: resolution: {integrity: sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==} engines: {node: '>=8'} + ansi-styles@5.2.0: + resolution: {integrity: sha512-Cxwpt2SfTzTtXcfOlzGEee8O+c+MmUgGrNiBcXnuWxuFJHe6a5Hz7qwhwe5OgaSYI0IJvkLqWX1ASG+cJOkEiA==} + engines: {node: '>=10'} + ansi-styles@6.2.3: resolution: {integrity: sha512-4Dj6M28JB+oAH8kFkTLUo+a2jwOFkuqb3yucU0CANcRRUbxS0cP0nZYCGjcc3BNXwRIsUVmDGgzawme7zvJHvg==} engines: {node: '>=12'} @@ -4250,6 +4362,13 @@ packages: resolution: {integrity: sha512-ik3ZgC9dY/lYVVM++OISsaYDeg1tb0VtP5uL3ouh1koGOaUMDPpbFIei4JkFimWUFPn90sbMNMXQAIVOlnYKJA==} engines: {node: '>=10'} + aria-query@5.3.0: + resolution: {integrity: sha512-b0P0sZPKtyu8HkeRAfCq0IfURZK+SuwMjY1UXGBU27wpAiTwQAIlq56IbIO+ytk/JjS1fMR14ee5WBBfKi5J6A==} + + aria-query@5.3.2: + resolution: {integrity: sha512-COROpnaoap1E2F000S62r6A60uHZnmlvomhfyT2DlTcrY1OrBKn2UhH7qn5wTC9zMvD0AY7csdPSNwKP+7WiQw==} + engines: {node: '>= 0.4'} + arrify@2.0.1: resolution: {integrity: sha512-3duEwti880xqi4eAMN8AyR4a0ByT90zoYdLlevfrvU43vb0YZwZVfxOgxWrLXXXpyugL0hNZc9G6BiB5B3nUug==} engines: {node: '>=8'} @@ -4687,10 +4806,17 @@ packages: css-select@5.2.2: resolution: {integrity: sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw==} + css-tree@3.2.1: + resolution: {integrity: sha512-X7sjQzceUhu1u7Y/ylrRZFU2FS6LRiFVp6rKLPg23y3x3c3DOKAwuXGDp+PAGjh6CSnCjYeAul8pcT8bAl+lSA==} + engines: {node: ^10 || ^12.20.0 || ^14.13.0 || >=15.0.0} + css-what@6.2.2: resolution: {integrity: sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA==} engines: {node: '>= 6'} + css.escape@1.5.1: + resolution: {integrity: sha512-YUifsXXuknHlUsmlgyY0PKzgPOr7/FjCePfHNt0jxm83wHZi44VDMQ7/fGNkjY3/jV1MC+1CmZbaHzugyeRtpg==} + csstype@3.2.3: resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==} @@ -4854,6 +4980,10 @@ packages: resolution: {integrity: sha512-0R9ikRb668HB7QDxT1vkpuUBtqc53YyAwMwGeUFKRojY/NWKvdZ+9UYtRfGmhqNbRkTSVpMbmyhXipFFv2cb/A==} engines: {node: '>= 12'} + data-urls@7.0.0: + resolution: {integrity: sha512-23XHcCF+coGYevirZceTVD7NdJOqVn+49IHyxgszm+JIiHLoB2TkmPtsYkNWT1pvRSGkc35L6NHs0yHkN2SumA==} + engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} + dayjs@1.11.19: resolution: {integrity: sha512-t5EcLVS6QPBNqM2z8fakk/NKel+Xzshgt8FFKAn+qwlD1pzZWxh0nVCrvFK7ZDb6XucZeF9z8C7CBWTRIVApAw==} @@ -4877,6 +5007,9 @@ packages: decimal.js-light@2.5.1: resolution: {integrity: sha512-qIMFpTMZmny+MMIitAB6D7iVPEorVw6YQRWkvarTkT4tBeSLLiHzcwj6q0MmYSFCiVpiqPJTJEYIrpcPzVEIvg==} + decimal.js@10.6.0: + resolution: {integrity: sha512-YpgQiITW3JXGntzdUmyUR1V812Hn8T1YVXhCu+wO3OpS4eU9l4YdD3qjyiKdV6mvV29zapkMeD390UVEf2lkUg==} + decode-named-character-reference@1.2.0: resolution: {integrity: sha512-c6fcElNV6ShtZXmsgNgFFV5tVX2PaV4g+MOAkb8eXHvn6sryJBrZa9r0zV6+dtTyoCKxtDy5tyQ5ZwQuidtd+Q==} @@ -4965,6 +5098,12 @@ packages: resolution: {integrity: sha512-FwgeAKqY2vc9eVm2V2XGg8bq25B0OQjtSDITGi9zNnvu5GbtR4WvGjM5QNld/ALB6ZbsSuHskBPK9SvPpKhsbA==} engines: {node: '>=0.10'} + dom-accessibility-api@0.5.16: + resolution: {integrity: sha512-X7BJ2yElsnOJ30pZF4uIIDfBEVgF4XEBxL9Bxhy6dnrm5hkzqmsWHGTiHqRiITNhMyFLyAiWndIJP7Z1NTteDg==} + + dom-accessibility-api@0.6.3: + resolution: {integrity: sha512-7ZgogeTnjuHbo+ct10G9Ffp0mif17idi0IyWNVA/wcwcm7NPOD/WEHVP3n7n3MhXqxoIYm8d6MuZohYWIZ4T3w==} + dom-serializer@0.2.2: resolution: {integrity: sha512-2/xPb3ORsQ42nHYiSunXkDjPLBaEj/xTwUO4B7XCZQTRk7EBtTOPaygh10YAAh2OI1Qrp6NWfpAhzswj0ydt9g==} @@ -5098,6 +5237,10 @@ packages: resolution: {integrity: sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==} engines: {node: '>=0.12'} + entities@8.0.0: + resolution: {integrity: sha512-zwfzJecQ/Uej6tusMqwAqU/6KL2XaB2VZ2Jg54Je6ahNBGNH6Ek6g3jjNCF0fG9EWQKGZNddNjU5F1ZQn/sBnA==} + engines: {node: '>=20.19.0'} + env-paths@2.2.1: resolution: {integrity: sha512-+h1lkLKhZMTYjog1VEpJNG7NZJWcuc2DDk/qsqSTRRCOXiLjeQ1d1/udrUGhqMxUgAlwKNZ0cf2uqan5GLuS2A==} engines: {node: '>=6'} @@ -5747,6 +5890,10 @@ packages: hsl-to-rgb-for-reals@1.1.1: resolution: {integrity: sha512-LgOWAkrN0rFaQpfdWBQlv/VhkOxb5AsBjk6NQVx4yEzWS923T07X0M1Y0VNko2H52HeSpZrZNNMJ0aFqsdVzQg==} + html-encoding-sniffer@6.0.0: + resolution: {integrity: sha512-CV9TW3Y3f8/wT0BRFc1/KAVQ3TUHiXmaAb6VW9vtiMFf7SLoMd1PdAc4W3KFOFETBJUb90KatHqlsZMWV+R9Gg==} + engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} + html-entities@2.6.0: resolution: {integrity: sha512-kig+rMn/QOVRvr7c86gQ8lWXq+Hkv6CbAH1hLu+RG338StTpE8Z0b44SDVaqVu7HGKf27frdmUYEs9hTUX/cLQ==} @@ -5985,6 +6132,9 @@ packages: resolution: {integrity: sha512-+Pgi+vMuUNkJyExiMBt5IlFoMyKnr5zhJ4Uspz58WOhBF5QoIZkFyNHIbBAtHwzVAgk5RtndVNsDRN61/mmDqg==} engines: {node: '>=12'} + is-potential-custom-element-name@1.0.1: + resolution: {integrity: sha512-bCYeRA2rVibKZd+s2625gGnGF/t7DSqDs4dP7CrLA1m7jKWz6pps0LpYLJN8Q64HtmPKJ1hrN3nzPNKFEKOUiQ==} + is-promise@4.0.0: resolution: {integrity: sha512-hvpoI6korhJMnej285dSg6nu1+e6uxs7zG3BYAm5byqDsgJNWwxzM6z6iZiAgQR4TJ30JmBTOwqZUw3WlyH3AQ==} @@ -6068,6 +6218,15 @@ packages: resolution: {integrity: sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==} hasBin: true + jsdom@29.1.1: + resolution: {integrity: sha512-ECi4Fi2f7BdJtUKTflYRTiaMxIB0O6zfR1fX0GXpUrf6flp8QIYn1UT20YQqdSOfk2dfkCwS8LAFoJDEppNK5Q==} + engines: {node: ^20.19.0 || ^22.13.0 || >=24.0.0} + peerDependencies: + canvas: ^3.0.0 + peerDependenciesMeta: + canvas: + optional: true + jsesc@3.1.0: resolution: {integrity: sha512-/sM3dO2FOzXjKQhJuo0Q173wf2KOo8t4I8vHy6lF9poUp7bKT0/NHE8fPX23PwfhnykfqnC2xRxOnVw5XuGIaA==} engines: {node: '>=6'} @@ -6306,6 +6465,10 @@ packages: resolution: {integrity: sha512-B5Y16Jr9LB9dHVkh6ZevG+vAbOsNOYCX+sXvFWFu7B3Iz5mijW3zdbMyhsh8ANd2mSWBYdJgnqi+mL7/LrOPYg==} engines: {node: 20 || >=22} + lru-cache@11.5.1: + resolution: {integrity: sha512-RPimw/7aMdv2oqRrxKwvZXcPfwBrn/JZ2xYcY9Hus/6LaS3VOAKVWKWgNLCFSiOm1ESXinjsDlidVU7JlnCN2A==} + engines: {node: 20 || >=22} + lru-cache@5.1.1: resolution: {integrity: sha512-KpNARQA3Iwv+jTA0utUVVbrh+Jlrr1Fv0e56GGzAFOXN7dk/FviaDW8LHmK52DlcH4WP2n6gI8vN1aesBFgo9w==} @@ -6327,6 +6490,10 @@ packages: resolution: {integrity: sha512-vtEhXh/gNjI9Yg1u4jX/0YVPMvxzHuGgCm6tC5kZyb08yjGWGnqAjGJvcXbqQR2P3MyMEFnRbpcdFS6PBcLqew==} engines: {node: '>=12'} + lz-string@1.5.0: + resolution: {integrity: sha512-h5bgJWpxJNswbU7qCrV0tIKQCaS3blPDrqKWx+QxzuzL1zGUzij9XCWLrSLsJPu5t+eWA/ycetzYAO5IOMcWAQ==} + hasBin: true + macos-alias@0.2.12: resolution: {integrity: sha512-yiLHa7cfJcGRFq4FrR4tMlpNHb4Vy4mWnpajlSSIFM5k4Lv8/7BbbDLzCAVogWNl0LlLhizRp1drXv0hK9h0Yw==} os: [darwin] @@ -6421,6 +6588,9 @@ packages: mdast-util-to-string@4.0.0: resolution: {integrity: sha512-0H44vDimn51F0YwvxSJSm0eCDOJTRlmN0R1yBh4HLj9wiV1Dn0QoXGbvFAWj2hSItVTlCmBF1hqKlIyUBVFLPg==} + mdn-data@2.27.1: + resolution: {integrity: sha512-9Yubnt3e8A0OKwxYSXyhLymGW4sCufcLG6VdiDdUGVkPhpqLxlvP5vl1983gQjJl3tqbrM731mjaZaP68AgosQ==} + mdurl@2.0.0: resolution: {integrity: sha512-Lf+9+2r+Tdp5wXDXC4PcIBjTDtq4UKjCPMQhKIuzpJNW0b96kVqSwW0bT7FhRSfmAiFYgP+SCRvdrDozfh0U5w==} @@ -6600,6 +6770,10 @@ packages: min-document@2.19.2: resolution: {integrity: sha512-8S5I8db/uZN8r9HSLFVWPdJCvYOejMcEC82VIzNUc6Zkklf/d1gg2psfE79/vyhWOj4+J8MtwmoOz3TmvaGu5A==} + min-indent@1.0.1: + resolution: {integrity: sha512-I9jwMn07Sy/IwOj3zVkVik2JTvgpaykDZEigL6Rx6N9LbMywwUSMtxET+7lVoDLLd3O3IXwJwvuuns8UB/HeAg==} + engines: {node: '>=4'} + minimatch@10.1.1: resolution: {integrity: sha512-enIvLvRAFZYXJzkCYG5RKmPfrFArdLv+R+lbQ53BmIMLIry74bjKzX6iHAm8WYamJkhSSEabrWN5D97XnKObjQ==} engines: {node: 20 || >=22} @@ -6981,6 +7155,9 @@ packages: parse5@7.3.0: resolution: {integrity: sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==} + parse5@8.0.1: + resolution: {integrity: sha512-z1e/HMG90obSGeidlli3hj7cbocou0/wa5HacvI3ASx34PecNjNQeaHNo5WIZpWofN9kgkqV1q5YvXe3F0FoPw==} + parseurl@1.3.3: resolution: {integrity: sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ==} engines: {node: '>= 0.8'} @@ -7126,6 +7303,10 @@ packages: engines: {node: '>=14'} hasBin: true + pretty-format@27.5.1: + resolution: {integrity: sha512-Qb1gy5OrP5+zDf2Bvnzdl3jsTf1qXVMazbvCoKhtKqVs4/YK4ozX4gKQJJVyNe+cajNPn0KoC0MC3FUmaHWEmQ==} + engines: {node: ^10.13.0 || ^12.13.0 || ^14.15.0 || >=15.0.0} + proc-log@2.0.1: resolution: {integrity: sha512-Kcmo2FhfDTXdcbfDH76N7uBYHINxc/8GW7UAVuVP9I+Va3uHSerrnKV6dLooga/gh7GlgzuCCr/eoldnL1muGw==} engines: {node: ^12.13.0 || ^14.15.0 || >=16.0.0} @@ -7282,6 +7463,9 @@ packages: react-is@16.13.1: resolution: {integrity: sha512-24e6ynE2H+OKt4kqsOvNd8kBpV65zoxbA4BVsEOB3ARVWQki/DHzaUoC5KuON/BiccDaCCTZBuOcfZs70kR8bQ==} + react-is@17.0.2: + resolution: {integrity: sha512-w2GsyukL62IJnlaff/nRegPQR94C/XXamvMWmSHRJ4y7Ts/4ocGRmTHvOs8PSE6pB3dWOrD/nueuU5sduBsQ4w==} + react-redux@9.2.0: resolution: {integrity: sha512-ROY9fvHhwOD9ySfrF0wmvu//bKCQ6AeZZq1nJNtbDC+kk5DuSuNX/n6YWYF/SYy7bSba4D4FSz8DJeKY/S/r+g==} peerDependencies: @@ -7377,6 +7561,10 @@ packages: resolution: {integrity: sha512-/vxpCXddiX8NGfGO/mTafwjq4aFa/71pvamip0++IQk3zG8cbCj0fifNPrjjF1XMXUne91jL9OoxmdykoEtifQ==} engines: {node: '>= 10.13.0'} + redent@3.0.0: + resolution: {integrity: sha512-6tDA8g98We0zd0GvVeMT9arEOnTw9qM03L9cJXaCjrip1OO764RDBLBfrB4cwzNGDj5OA5ioymC9GkizgWJDUg==} + engines: {node: '>=8'} + redux-thunk@3.1.0: resolution: {integrity: sha512-NW2r5T6ksUKXCabzhL9z+h206HQw/NJkcLm1GPImRQ8IzfXwRGqjVhKJGauHirT0DAuyy6hjdnMZaRoAcy0Klw==} peerDependencies: @@ -7570,6 +7758,10 @@ packages: resolution: {integrity: sha512-6R3J5M4AcbtLUdZmRv2SygeVaM7IhrLXu9BmnOGmmACak8fiUtOsYNWUS4uK7upbmHIBbLBeFeI//477BKLBzA==} engines: {node: '>=11.0.0'} + saxes@6.0.0: + resolution: {integrity: sha512-xAg7SOnEhrm5zI3puOOKyy1OMcMlIJZYNJY7xLBwSze0UjhPLnWfj2GF2EpT0jmzaJKIWKHLsaSSajf35bcYnA==} + engines: {node: '>=v12.22.7'} + scheduler@0.25.0-rc-603e6108-20241029: resolution: {integrity: sha512-pFwF6H1XrSdYYNLfOcGlM28/j8CGLu8IvdrxqhjWULe2bPcKiKW4CV+OWqR/9fT52mywx65l7ysNkjLKBda7eA==} @@ -7814,6 +8006,10 @@ packages: resolution: {integrity: sha512-7FCwGGmx8mD5xQd3RPUvnSpUXHM3BWuzjtpD4TXsfcZ9EL4azvVVUscFYwD9nx8Kh+uCBC00XBtAykoMHwTh8Q==} engines: {node: '>=0.10.0'} + strip-indent@3.0.0: + resolution: {integrity: sha512-laJTa3Jb+VQpaC6DseHhF7dXVqHTfJPCRDaEbid/drOhgitgYku/letMUqOXFoWV0zIIUbjpdH2t+tYj4bQMRQ==} + engines: {node: '>=8'} + strip-json-comments@3.1.1: resolution: {integrity: sha512-6fPc+R4ihwqP6N/aIv2f1gMH8lOVtWQHoqC4yK6oSDVVocumAsfCqjkXnqiYMhmMwS/mEHLp7Vehlt3ql6lEig==} engines: {node: '>=8'} @@ -7861,6 +8057,9 @@ packages: peerDependencies: react: ^16.11.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 + symbol-tree@3.2.4: + resolution: {integrity: sha512-9QNk5KwDF+Bvz+PyObkmSYjI5ksVUYtjW7AU22r2NKcfLJcXp96hkDWU3+XndOsUb+AQ9QhfzfCT2O+CNWT5Tw==} + tailwind-merge@3.4.0: resolution: {integrity: sha512-uSaO4gnW+b3Y2aWoWfFpX62vn2sR3skfhbjsEnaBI81WD1wBLlHZe5sWf0AqjksNdYTbGBEd0UasQMT3SNV15g==} @@ -7930,6 +8129,13 @@ packages: peerDependencies: '@tiptap/core': ^3.0.1 + tldts-core@7.4.5: + resolution: {integrity: sha512-pGrwzZDvPwKe+7NNUqAunb6rqTfynr0VOUhCMdqbu5xlvNiszsAJygRzwvpVycdzejlbpY+SWJOn+s75Og7FEA==} + + tldts@7.4.5: + resolution: {integrity: sha512-RfEzKWcq5fHUOFq7J3rl3Oz6ylKGtcHqUznzj4EcXsxLSIjJcvpbXAQtWGeJQ0xKnimR5e0Cn+cn9TssfMzm+g==} + hasBin: true + tmp-promise@3.0.3: resolution: {integrity: sha512-RwM7MoPojPxsOBYnyd2hy0bxtIlVrihNs9pj5SUvY8Zz1sQcQG2tG1hSr8PDxfgEB8RNKDhqbIlroIarSNDNsQ==} @@ -7963,9 +8169,17 @@ packages: tokenlens@1.3.1: resolution: {integrity: sha512-7oxmsS5PNCX3z+b+z07hL5vCzlgHKkCGrEQjQmWl5l+v5cUrtL7S1cuST4XThaL1XyjbTX8J5hfP0cjDJRkaLA==} + tough-cookie@6.0.1: + resolution: {integrity: sha512-LktZQb3IeoUWB9lqR5EWTHgW/VTITCXg4D21M+lvybRVdylLrRMnqaIONLVb5mav8vM19m44HIcGq4qASeu2Qw==} + engines: {node: '>=16'} + tr46@0.0.3: resolution: {integrity: sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw==} + tr46@6.0.0: + resolution: {integrity: sha512-bLVMLPtstlZ4iMQHpFHTR7GAGj2jxi8Dg0s2h2MafAE4uSWF98FC/3MomU51iQAMf8/qDUbKWf5GxuvvVcXEhw==} + engines: {node: '>=20'} + tree-kill@1.2.2: resolution: {integrity: sha512-L0Orpi8qGpRG//Nd+H90vFB+3iHnue1zSSGmNOOCh1GLJ7rUKVwV2HvijphGQS2UmhUZewS9VgvxYIdgr+fG1A==} hasBin: true @@ -8065,6 +8279,10 @@ packages: undici-types@7.16.0: resolution: {integrity: sha512-Zz+aZWSj8LE6zoxD+xrjh4VfkIG8Ya6LvYkZqtUQGJPZjYl53ypCaUwWqo7eI0x66KBGeRo+mlBEkMSeSZ38Nw==} + undici@7.28.0: + resolution: {integrity: sha512-cRZYrTDwWznlnRiPjggAGxZXanty6M8RV1ff8Wm4LWXBp7/IG8v5DnOm74DtUBp9OONpK75YlPnIjQqX0dBDtA==} + engines: {node: '>=20.18.1'} + unicode-emoji-modifier-base@1.0.0: resolution: {integrity: sha512-yLSH4py7oFH3oG/9K+XWrz1pSi3dfUrWEnInbxMfArOfc1+33BlGPQtLsOYwvdMy11AwUBetYuaRxSPqgkq+8g==} engines: {node: '>=4'} @@ -8321,6 +8539,10 @@ packages: w3c-keyname@2.2.8: resolution: {integrity: sha512-dpojBhNsCNN7T82Tm7k26A6G9ML3NkhDsnw9n/eoxSRlVBB4CEtIQ/KTCLI2Fwf3ataSXRhYFkQi3SlnFwPvPQ==} + w3c-xmlserializer@5.0.0: + resolution: {integrity: sha512-o8qghlI8NZHU1lLPrpi2+Uq7abh4GGPpYANlalzWxyWteJOCsr/P+oPBA49TOLu5FTZO4d3F9MnWJfiMo4BkmA==} + engines: {node: '>=18'} + wait-on@9.0.3: resolution: {integrity: sha512-13zBnyYvFDW1rBvWiJ6Av3ymAaq8EDQuvxZnPIw3g04UqGi4TyoIJABmfJ6zrvKo9yeFQExNkOk7idQbDJcuKA==} engines: {node: '>=20.0.0'} @@ -8346,6 +8568,10 @@ packages: webidl-conversions@3.0.1: resolution: {integrity: sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==} + webidl-conversions@8.0.1: + resolution: {integrity: sha512-BMhLD/Sw+GbJC21C/UgyaZX41nPt8bUTg+jWyDeg7e7YN4xOM05YPSIXceACnXVtqyEw/LMClUQMtMZ+PGGpqQ==} + engines: {node: '>=20'} + webpack-sources@3.3.3: resolution: {integrity: sha512-yd1RBzSGanHkitROoPFd6qsrxt+oFhg/129YzheDGqeustzX0vTZJZsSsQjVQC4yzBQ56K55XU8gaNCtIzOnTg==} engines: {node: '>=10.13.0'} @@ -8360,6 +8586,14 @@ packages: webpack-cli: optional: true + whatwg-mimetype@5.0.0: + resolution: {integrity: sha512-sXcNcHOC51uPGF0P/D4NVtrkjSU2fNsm9iog4ZvZJsL3rjoDAzXZhkm2MWt1y+PUdggKAYVoMAIYcs78wJ51Cw==} + engines: {node: '>=20'} + + whatwg-url@16.0.1: + resolution: {integrity: sha512-1to4zXBxmXHV3IiSSEInrreIlu02vUOvrhxJJH5vcxYTBDAx51cqZiKdyTxlecdKNSjj8EcxGBxNf6Vg+945gw==} + engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} + whatwg-url@5.0.0: resolution: {integrity: sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==} @@ -8427,6 +8661,10 @@ packages: resolution: {integrity: sha512-7rVi2KMfwfWFl+GpPg6m80IVMWXLRjO+PxTq7V2CDhoGak0wzYzFgUY2m4XJ47OGdXd8eLE8EmwfAmdjw7lC1g==} hasBin: true + xml-name-validator@5.0.0: + resolution: {integrity: sha512-EvGK8EJ3DhaHfbRlETOWAS5pO9MZITeauHKJyb8wyajUfQUenkIg2MvLDTZ4T/TgIcm3HU0TFBgWWboAZ30UHg==} + engines: {node: '>=18'} + xmlbuilder2@2.1.2: resolution: {integrity: sha512-PI710tmtVlQ5VmwzbRTuhmVhKnj9pM8Si+iOZCV2g2SNo3gCrpzR2Ka9wNzZtqfD+mnP+xkrqoNy0sjKZqP4Dg==} engines: {node: '>=8.0'} @@ -8439,6 +8677,9 @@ packages: resolution: {integrity: sha512-yMqGBqtXyeN1e3TGYvgNgDVZ3j84W4cwkOXQswghol6APgZWaff9lnbvN7MHYJOiXsvGPXtjTYJEiC9J2wv9Eg==} engines: {node: '>=8.0'} + xmlchars@2.2.0: + resolution: {integrity: sha512-JZnDKK8B0RCDw84FNdDAIpZK+JuJw+s7Lz8nksI7SIuU3UXJJslUthsi+uWBUYOwPFwW7W7PRLRfUKpxjtjFCw==} + xtend@4.0.2: resolution: {integrity: sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==} engines: {node: '>=0.4'} @@ -8510,6 +8751,8 @@ packages: snapshots: + '@adobe/css-tools@4.5.0': {} + '@agentclientprotocol/claude-agent-acp@0.39.0(@anthropic-ai/sdk@0.100.1(zod@4.2.1))(@modelcontextprotocol/sdk@1.25.1(hono@4.11.3)(zod@4.2.1))': dependencies: '@agentclientprotocol/sdk': 0.22.1(zod@4.2.1) @@ -8644,6 +8887,26 @@ snapshots: optionalDependencies: zod: 4.2.1 + '@asamuzakjp/css-color@5.1.11': + dependencies: + '@asamuzakjp/generational-cache': 1.0.1 + '@csstools/css-calc': 3.2.1(@csstools/css-parser-algorithms@4.0.0(@csstools/css-tokenizer@4.0.0))(@csstools/css-tokenizer@4.0.0) + '@csstools/css-color-parser': 4.1.9(@csstools/css-parser-algorithms@4.0.0(@csstools/css-tokenizer@4.0.0))(@csstools/css-tokenizer@4.0.0) + '@csstools/css-parser-algorithms': 4.0.0(@csstools/css-tokenizer@4.0.0) + '@csstools/css-tokenizer': 4.0.0 + + '@asamuzakjp/dom-selector@7.1.1': + dependencies: + '@asamuzakjp/generational-cache': 1.0.1 + '@asamuzakjp/nwsapi': 2.3.9 + bidi-js: 1.0.3 + css-tree: 3.2.1 + is-potential-custom-element-name: 1.0.1 + + '@asamuzakjp/generational-cache@1.0.1': {} + + '@asamuzakjp/nwsapi@2.3.9': {} + '@aws-crypto/crc32@5.2.0': dependencies: '@aws-crypto/util': 5.2.0 @@ -9256,6 +9519,10 @@ snapshots: '@braintree/sanitize-url@7.1.1': {} + '@bramus/specificity@2.4.2': + dependencies: + css-tree: 3.2.1 + '@chevrotain/cst-dts-gen@12.0.0': dependencies: '@chevrotain/gast': 12.0.0 @@ -9550,6 +9817,30 @@ snapshots: dependencies: zod: 4.2.1 + '@csstools/color-helpers@6.1.0': {} + + '@csstools/css-calc@3.2.1(@csstools/css-parser-algorithms@4.0.0(@csstools/css-tokenizer@4.0.0))(@csstools/css-tokenizer@4.0.0)': + dependencies: + '@csstools/css-parser-algorithms': 4.0.0(@csstools/css-tokenizer@4.0.0) + '@csstools/css-tokenizer': 4.0.0 + + '@csstools/css-color-parser@4.1.9(@csstools/css-parser-algorithms@4.0.0(@csstools/css-tokenizer@4.0.0))(@csstools/css-tokenizer@4.0.0)': + dependencies: + '@csstools/color-helpers': 6.1.0 + '@csstools/css-calc': 3.2.1(@csstools/css-parser-algorithms@4.0.0(@csstools/css-tokenizer@4.0.0))(@csstools/css-tokenizer@4.0.0) + '@csstools/css-parser-algorithms': 4.0.0(@csstools/css-tokenizer@4.0.0) + '@csstools/css-tokenizer': 4.0.0 + + '@csstools/css-parser-algorithms@4.0.0(@csstools/css-tokenizer@4.0.0)': + dependencies: + '@csstools/css-tokenizer': 4.0.0 + + '@csstools/css-syntax-patches-for-csstree@1.1.6(css-tree@3.2.1)': + optionalDependencies: + css-tree: 3.2.1 + + '@csstools/css-tokenizer@4.0.0': {} + '@eigenpal/docx-editor-agents@1.0.3(ai@5.0.117(zod@4.2.1))(react@19.2.3)': dependencies: docxtemplater: 3.68.7 @@ -10210,6 +10501,8 @@ snapshots: '@eslint/core': 0.17.0 levn: 0.4.1 + '@exodus/bytes@1.15.1': {} + '@floating-ui/core@1.7.3': dependencies: '@floating-ui/utils': 0.2.10 @@ -12345,6 +12638,36 @@ snapshots: tailwindcss: 4.1.18 vite: 7.3.0(@types/node@24.10.4)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2) + '@testing-library/dom@10.4.1': + dependencies: + '@babel/code-frame': 7.27.1 + '@babel/runtime': 7.28.6 + '@types/aria-query': 5.0.4 + aria-query: 5.3.0 + dom-accessibility-api: 0.5.16 + lz-string: 1.5.0 + picocolors: 1.1.1 + pretty-format: 27.5.1 + + '@testing-library/jest-dom@6.9.1': + dependencies: + '@adobe/css-tools': 4.5.0 + aria-query: 5.3.2 + css.escape: 1.5.1 + dom-accessibility-api: 0.6.3 + picocolors: 1.1.1 + redent: 3.0.0 + + '@testing-library/react@16.3.2(@testing-library/dom@10.4.1)(@types/react-dom@19.2.3(@types/react@19.2.7))(@types/react@19.2.7)(react-dom@19.2.3(react@19.2.3))(react@19.2.3)': + dependencies: + '@babel/runtime': 7.28.6 + '@testing-library/dom': 10.4.1 + react: 19.2.3 + react-dom: 19.2.3(react@19.2.3) + optionalDependencies: + '@types/react': 19.2.7 + '@types/react-dom': 19.2.3(@types/react@19.2.7) + '@tiptap/core@3.22.4(@tiptap/pm@3.22.4)': dependencies: '@tiptap/pm': 3.22.4 @@ -12559,6 +12882,8 @@ snapshots: '@types/node': 25.0.3 optional: true + '@types/aria-query@5.0.4': {} + '@types/babel__core@7.20.5': dependencies: '@babel/parser': 7.28.5 @@ -12996,6 +13321,14 @@ snapshots: chai: 6.2.2 tinyrainbow: 3.1.0 + '@vitest/mocker@4.1.7(vite@7.3.0(@types/node@24.10.4)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2))': + dependencies: + '@vitest/spy': 4.1.7 + estree-walker: 3.0.3 + magic-string: 0.30.21 + optionalDependencies: + vite: 7.3.0(@types/node@24.10.4)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2) + '@vitest/mocker@4.1.7(vite@7.3.0(@types/node@25.0.3)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2))': dependencies: '@vitest/spy': 4.1.7 @@ -13232,6 +13565,8 @@ snapshots: dependencies: color-convert: 2.0.1 + ansi-styles@5.2.0: {} + ansi-styles@6.2.3: {} appdmg@0.6.6: @@ -13259,6 +13594,12 @@ snapshots: dependencies: tslib: 2.8.1 + aria-query@5.3.0: + dependencies: + dequal: 2.0.3 + + aria-query@5.3.2: {} + arrify@2.0.1: {} assertion-error@2.0.1: {} @@ -13716,8 +14057,15 @@ snapshots: domutils: 3.2.2 nth-check: 2.1.1 + css-tree@3.2.1: + dependencies: + mdn-data: 2.27.1 + source-map-js: 1.2.1 + css-what@6.2.2: {} + css.escape@1.5.1: {} + csstype@3.2.3: {} cytoscape-cose-bilkent@4.1.0(cytoscape@3.33.1): @@ -13906,6 +14254,13 @@ snapshots: data-uri-to-buffer@4.0.1: {} + data-urls@7.0.0: + dependencies: + whatwg-mimetype: 5.0.0 + whatwg-url: 16.0.1 + transitivePeerDependencies: + - '@noble/hashes' + dayjs@1.11.19: {} debug@2.6.9: @@ -13918,6 +14273,8 @@ snapshots: decimal.js-light@2.5.1: {} + decimal.js@10.6.0: {} + decode-named-character-reference@1.2.0: dependencies: character-entities: 2.0.2 @@ -13996,6 +14353,10 @@ snapshots: dependencies: '@xmldom/xmldom': 0.9.10 + dom-accessibility-api@0.5.16: {} + + dom-accessibility-api@0.6.3: {} + dom-serializer@0.2.2: dependencies: domelementtype: 2.3.0 @@ -14187,6 +14548,8 @@ snapshots: entities@6.0.1: {} + entities@8.0.0: {} + env-paths@2.2.1: {} err-code@2.0.3: {} @@ -15094,6 +15457,12 @@ snapshots: hsl-to-rgb-for-reals@1.1.1: {} + html-encoding-sniffer@6.0.0: + dependencies: + '@exodus/bytes': 1.15.1 + transitivePeerDependencies: + - '@noble/hashes' + html-entities@2.6.0: {} html-to-docx@1.8.0(encoding@0.1.13): @@ -15316,6 +15685,8 @@ snapshots: is-plain-obj@4.1.0: {} + is-potential-custom-element-name@1.0.1: {} + is-promise@4.0.0: {} is-property@1.0.2: @@ -15406,6 +15777,32 @@ snapshots: dependencies: argparse: 2.0.1 + jsdom@29.1.1: + dependencies: + '@asamuzakjp/css-color': 5.1.11 + '@asamuzakjp/dom-selector': 7.1.1 + '@bramus/specificity': 2.4.2 + '@csstools/css-syntax-patches-for-csstree': 1.1.6(css-tree@3.2.1) + '@exodus/bytes': 1.15.1 + css-tree: 3.2.1 + data-urls: 7.0.0 + decimal.js: 10.6.0 + html-encoding-sniffer: 6.0.0 + is-potential-custom-element-name: 1.0.1 + lru-cache: 11.5.1 + parse5: 8.0.1 + saxes: 6.0.0 + symbol-tree: 3.2.4 + tough-cookie: 6.0.1 + undici: 7.28.0 + w3c-xmlserializer: 5.0.0 + webidl-conversions: 8.0.1 + whatwg-mimetype: 5.0.0 + whatwg-url: 16.0.1 + xml-name-validator: 5.0.0 + transitivePeerDependencies: + - '@noble/hashes' + jsesc@3.1.0: {} json-bigint@1.0.0: @@ -15633,6 +16030,8 @@ snapshots: lru-cache@11.2.4: {} + lru-cache@11.5.1: {} + lru-cache@5.1.1: dependencies: yallist: 3.1.1 @@ -15649,6 +16048,8 @@ snapshots: luxon@3.7.2: {} + lz-string@1.5.0: {} + macos-alias@0.2.12: dependencies: nan: 2.24.0 @@ -15889,6 +16290,8 @@ snapshots: dependencies: '@types/mdast': 4.0.4 + mdn-data@2.27.1: {} + mdurl@2.0.0: {} media-engine@1.0.3: {} @@ -16191,6 +16594,8 @@ snapshots: dependencies: dom-walk: 0.1.2 + min-indent@1.0.1: {} + minimatch@10.1.1: dependencies: '@isaacs/brace-expansion': 5.0.0 @@ -16562,6 +16967,10 @@ snapshots: dependencies: entities: 6.0.1 + parse5@8.0.1: + dependencies: + entities: 8.0.0 + parseurl@1.3.3: {} pascal-case@3.1.2: @@ -16693,6 +17102,12 @@ snapshots: prettier@3.8.0: {} + pretty-format@27.5.1: + dependencies: + ansi-regex: 5.0.1 + ansi-styles: 5.2.0 + react-is: 17.0.2 + proc-log@2.0.1: {} process-nextick-args@2.0.1: {} @@ -16933,6 +17348,8 @@ snapshots: react-is@16.13.1: {} + react-is@17.0.2: {} + react-redux@9.2.0(@types/react@19.2.7)(react@19.2.3)(redux@5.0.1): dependencies: '@types/use-sync-external-store': 0.0.6 @@ -17024,7 +17441,7 @@ snapshots: readdirp@4.1.2: {} - recharts@3.8.1(@types/react@19.2.7)(react-dom@19.2.3(react@19.2.3))(react-is@16.13.1)(react@19.2.3)(redux@5.0.1): + recharts@3.8.1(@types/react@19.2.7)(react-dom@19.2.3(react@19.2.3))(react-is@17.0.2)(react@19.2.3)(redux@5.0.1): dependencies: '@reduxjs/toolkit': 2.11.2(react-redux@9.2.0(@types/react@19.2.7)(react@19.2.3)(redux@5.0.1))(react@19.2.3) clsx: 2.1.1 @@ -17034,7 +17451,7 @@ snapshots: immer: 10.2.0 react: 19.2.3 react-dom: 19.2.3(react@19.2.3) - react-is: 16.13.1 + react-is: 17.0.2 react-redux: 9.2.0(@types/react@19.2.7)(react@19.2.3)(redux@5.0.1) reselect: 5.1.1 tiny-invariant: 1.3.3 @@ -17048,6 +17465,11 @@ snapshots: dependencies: resolve: 1.22.11 + redent@3.0.0: + dependencies: + indent-string: 4.0.0 + strip-indent: 3.0.0 + redux-thunk@3.1.0(redux@5.0.1): dependencies: redux: 5.0.1 @@ -17300,6 +17722,10 @@ snapshots: sax@1.6.0: {} + saxes@6.0.0: + dependencies: + xmlchars: 2.2.0 + scheduler@0.25.0-rc-603e6108-20241029: {} scheduler@0.27.0: {} @@ -17602,6 +18028,10 @@ snapshots: strip-eof@1.0.0: {} + strip-indent@3.0.0: + dependencies: + min-indent: 1.0.1 + strip-json-comments@3.1.1: {} strip-outer@1.0.1: @@ -17646,6 +18076,8 @@ snapshots: react: 19.2.3 use-sync-external-store: 1.6.0(react@19.2.3) + symbol-tree@3.2.4: {} + tailwind-merge@3.4.0: {} tailwindcss@4.1.18: {} @@ -17711,6 +18143,12 @@ snapshots: markdown-it-task-lists: 2.1.1 prosemirror-markdown: 1.13.2 + tldts-core@7.4.5: {} + + tldts@7.4.5: + dependencies: + tldts-core: 7.4.5 + tmp-promise@3.0.3: dependencies: tmp: 0.2.5 @@ -17750,8 +18188,16 @@ snapshots: '@tokenlens/helpers': 1.3.1 '@tokenlens/models': 1.3.0 + tough-cookie@6.0.1: + dependencies: + tldts: 7.4.5 + tr46@0.0.3: {} + tr46@6.0.0: + dependencies: + punycode: 2.3.1 + tree-kill@1.2.2: {} trim-lines@3.0.1: {} @@ -17830,6 +18276,8 @@ snapshots: undici-types@7.16.0: {} + undici@7.28.0: {} + unicode-emoji-modifier-base@1.0.0: {} unicode-properties@1.4.1: @@ -18043,7 +18491,36 @@ snapshots: terser: 5.46.0 yaml: 2.8.2 - vitest@4.1.7(@opentelemetry/api@1.9.0)(@types/node@25.0.3)(vite@7.3.0(@types/node@25.0.3)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2)): + vitest@4.1.7(@opentelemetry/api@1.9.0)(@types/node@24.10.4)(jsdom@29.1.1)(vite@7.3.0(@types/node@24.10.4)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2)): + dependencies: + '@vitest/expect': 4.1.7 + '@vitest/mocker': 4.1.7(vite@7.3.0(@types/node@24.10.4)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2)) + '@vitest/pretty-format': 4.1.7 + '@vitest/runner': 4.1.7 + '@vitest/snapshot': 4.1.7 + '@vitest/spy': 4.1.7 + '@vitest/utils': 4.1.7 + es-module-lexer: 2.0.0 + expect-type: 1.3.0 + magic-string: 0.30.21 + obug: 2.1.1 + pathe: 2.0.3 + picomatch: 4.0.3 + std-env: 4.1.0 + tinybench: 2.9.0 + tinyexec: 1.0.2 + tinyglobby: 0.2.15 + tinyrainbow: 3.1.0 + vite: 7.3.0(@types/node@24.10.4)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2) + why-is-node-running: 2.3.0 + optionalDependencies: + '@opentelemetry/api': 1.9.0 + '@types/node': 24.10.4 + jsdom: 29.1.1 + transitivePeerDependencies: + - msw + + vitest@4.1.7(@opentelemetry/api@1.9.0)(@types/node@25.0.3)(jsdom@29.1.1)(vite@7.3.0(@types/node@25.0.3)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2)): dependencies: '@vitest/expect': 4.1.7 '@vitest/mocker': 4.1.7(vite@7.3.0(@types/node@25.0.3)(jiti@2.6.1)(lightningcss@1.30.2)(terser@5.46.0)(yaml@2.8.2)) @@ -18068,6 +18545,7 @@ snapshots: optionalDependencies: '@opentelemetry/api': 1.9.0 '@types/node': 25.0.3 + jsdom: 29.1.1 transitivePeerDependencies: - msw @@ -18092,6 +18570,10 @@ snapshots: w3c-keyname@2.2.8: {} + w3c-xmlserializer@5.0.0: + dependencies: + xml-name-validator: 5.0.0 + wait-on@9.0.3: dependencies: axios: 1.13.2 @@ -18119,6 +18601,8 @@ snapshots: webidl-conversions@3.0.1: {} + webidl-conversions@8.0.1: {} + webpack-sources@3.3.3: {} webpack@5.104.1(esbuild@0.24.2): @@ -18153,6 +18637,16 @@ snapshots: - esbuild - uglify-js + whatwg-mimetype@5.0.0: {} + + whatwg-url@16.0.1: + dependencies: + '@exodus/bytes': 1.15.1 + tr46: 6.0.0 + webidl-conversions: 8.0.1 + transitivePeerDependencies: + - '@noble/hashes' + whatwg-url@5.0.0: dependencies: tr46: 0.0.3 @@ -18230,6 +18724,8 @@ snapshots: dependencies: sax: 1.6.0 + xml-name-validator@5.0.0: {} + xmlbuilder2@2.1.2: dependencies: '@oozcitak/dom': 1.15.5 @@ -18240,6 +18736,8 @@ snapshots: xmlbuilder@15.1.1: {} + xmlchars@2.2.0: {} + xtend@4.0.2: {} y18n@5.0.8: {} From 97169be93e40d1d03d19219d44f663939f9a0a15 Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 13:00:33 +0530 Subject: [PATCH 16/24] feat(x): compact string request refs + turn inspector CLI MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two debuggability follow-ups from reviewing real turn files: - Request refs are now compact strings — 'context' | 'input' | 'assistant:' | 'toolResult:' — so a raw JSONL line reads naturally: {"messages": ["assistant:0", "toolResult:toolu_…"]}. Same exact-match reducer invariants, via parseRequestRef. - npm run inspect-turn -- [callIndex] [--full]: prints, per model call, the EXACT provider payload (resolved system prompt, tool list, wire-form messages with user-message context woven in, and the response/failure) — rebuilt by the same composer the loop sends through. This is the missing viewer for the derived-not-duplicated request design: the file stores facts once; the inspector shows what the model actually received. Breaking for turn files written since the previous commit (dev only): wipe ~/.rowboat/storage. Co-Authored-By: Claude Fable 5 --- .../packages/core/docs/turn-runtime-design.md | 18 +- apps/x/packages/core/package.json | 3 +- .../core/src/sessions/sessions.test.ts | 2 +- .../core/src/turns/context-resolver.test.ts | 6 +- .../x/packages/core/src/turns/fs-repo.test.ts | 2 +- apps/x/packages/core/src/turns/inspect-cli.ts | 105 +++++++++++ .../x/packages/core/src/turns/runtime.test.ts | 15 +- apps/x/packages/core/src/turns/runtime.ts | 16 +- apps/x/packages/shared/src/turns.test.ts | 166 +++++++++--------- apps/x/packages/shared/src/turns.ts | 63 ++++--- 10 files changed, 261 insertions(+), 135 deletions(-) create mode 100644 apps/x/packages/core/src/turns/inspect-cli.ts diff --git a/apps/x/packages/core/docs/turn-runtime-design.md b/apps/x/packages/core/docs/turn-runtime-design.md index 9a0ca542..88290b7e 100644 --- a/apps/x/packages/core/docs/turn-runtime-design.md +++ b/apps/x/packages/core/docs/turn-runtime-design.md @@ -479,11 +479,12 @@ Every AI SDK `streamText` invocation performs exactly one model step. Tool execution is controlled manually by the turn loop. ```ts -type ModelRequestMessageRef = - | { kind: "context" } // the inline context block - | { kind: "input" } // the turn's user message - | { kind: "assistant"; modelCallIndex: number } // → model_call_completed - | { kind: "toolResult"; toolCallId: string }; // → tool_result +// Compact string refs, so raw JSONL reads naturally: +// "context" the inline context block +// "input" the turn's user message +// "assistant:" → that model_call_completed's message +// "toolResult:" → that tool_result +type ModelRequestMessageRef = string; interface ModelRequest { contextRef?: { previousTurnId: string }; // call 0 only (cross-turn prefix) @@ -503,10 +504,11 @@ agent model call in the turn. `request` is a list of references into the turn's own events: every referenced byte exists exactly once in the file. A request records only what -is new since the previous model call — call 0 is `[{context}?, {input}]` +is new since the previous model call — call 0 is `["context"?, "input"]` (context only when inline and nonempty; cross-turn prefixes ride -`contextRef`); call N is `[{assistant: N-1}, …that batch's toolResults in -source order]`; a re-issued call after an interruption is `[]`. The system +`contextRef`); call N is `["assistant:N-1", …that batch's +"toolResult:" refs in source order]`; a re-issued call after an +interruption is `[]`. The system prompt and tool set are not repeated: they are byte-identical to `turn_created.agent.resolved` by construction. diff --git a/apps/x/packages/core/package.json b/apps/x/packages/core/package.json index 08c2644d..8ba46048 100644 --- a/apps/x/packages/core/package.json +++ b/apps/x/packages/core/package.json @@ -8,7 +8,8 @@ "build": "rm -rf dist && tsc -p tsconfig.build.json", "dev": "tsc -w -p tsconfig.build.json", "test": "vitest run", - "test:watch": "vitest" + "test:watch": "vitest", + "inspect-turn": "node dist/turns/inspect-cli.js" }, "dependencies": { "@agentclientprotocol/claude-agent-acp": "^0.39.0", diff --git a/apps/x/packages/core/src/sessions/sessions.test.ts b/apps/x/packages/core/src/sessions/sessions.test.ts index fccdb948..6ff498d8 100644 --- a/apps/x/packages/core/src/sessions/sessions.test.ts +++ b/apps/x/packages/core/src/sessions/sessions.test.ts @@ -86,7 +86,7 @@ function turnLog( turnId, ts: TS, modelCallIndex: 0, - request: { messages: [{ kind: 'input' as const }], parameters: {} }, + request: { messages: ['input'], parameters: {} }, }; switch (status) { case "idle": diff --git a/apps/x/packages/core/src/turns/context-resolver.test.ts b/apps/x/packages/core/src/turns/context-resolver.test.ts index 78cb34bd..6ccfe671 100644 --- a/apps/x/packages/core/src/turns/context-resolver.test.ts +++ b/apps/x/packages/core/src/turns/context-resolver.test.ts @@ -60,8 +60,8 @@ function completedTurnLog( ...(Array.isArray(context) ? {} : { contextRef: context }), messages: Array.isArray(context) && context.length > 0 - ? [{ kind: "context" as const }, { kind: "input" as const }] - : [{ kind: "input" as const }], + ? ["context", "input"] + : ["input"], parameters: {}, }, }, @@ -138,7 +138,7 @@ function limitFailedTurnLog(turnId: string): TEvent[] { ts, modelCallIndex: 0, request: { - messages: [{ kind: "input" as const }], + messages: ["input"], parameters: {}, }, }, diff --git a/apps/x/packages/core/src/turns/fs-repo.test.ts b/apps/x/packages/core/src/turns/fs-repo.test.ts index b4afe769..583ea330 100644 --- a/apps/x/packages/core/src/turns/fs-repo.test.ts +++ b/apps/x/packages/core/src/turns/fs-repo.test.ts @@ -41,7 +41,7 @@ function requested(turnId = TURN_ID): z.infer { ts: "2026-07-02T10:00:01Z", modelCallIndex: 0, request: { - messages: [{ kind: "input" }], + messages: ["input"], parameters: {}, }, }; diff --git a/apps/x/packages/core/src/turns/inspect-cli.ts b/apps/x/packages/core/src/turns/inspect-cli.ts new file mode 100644 index 00000000..3329d99d --- /dev/null +++ b/apps/x/packages/core/src/turns/inspect-cli.ts @@ -0,0 +1,105 @@ +// Turn inspector: prints, per model call, the EXACT provider payload the +// loop sent — rebuilt from the durable file by the same composer the loop +// transmits through (compose-model-request.ts). This is where the woven +// wire-form messages (user-message context, attachments, tool-result +// envelopes) are visible; the file itself stores only structural facts and +// references. +// +// Usage: +// npm run inspect-turn -- [modelCallIndex] [--full] +// +// --full prints the entire system prompt and untruncated message contents. +import fs from "node:fs"; +import path from "node:path"; +import { + reduceTurn, + type JsonValue, + type TurnEvent, +} from "@x/shared/dist/turns.js"; +import type { z } from "zod"; +import { convertFromMessages } from "../agents/runtime.js"; +import { WorkDir } from "../config/config.js"; +import { composeModelRequest } from "./compose-model-request.js"; +import { TurnRepoContextResolver } from "./context-resolver.js"; +import { FSTurnRepo } from "./fs-repo.js"; + +function usage(): never { + console.error( + "usage: inspect-turn [modelCallIndex] [--full]", + ); + process.exit(1); +} + +function clip(text: string, full: boolean, limit = 400): string { + if (full || text.length <= limit) return text; + return `${text.slice(0, limit)}… [${text.length} chars total; pass --full]`; +} + +async function main(): Promise { + const args = process.argv.slice(2).filter((a) => a !== "--full"); + const full = process.argv.includes("--full"); + const target = args[0]; + if (!target) usage(); + const onlyIndex = args[1] !== undefined ? Number(args[1]) : undefined; + + const turnsRootDir = path.join(WorkDir, "storage", "turns"); + const repo = new FSTurnRepo({ turnsRootDir }); + const turnId = target.endsWith(".jsonl") + ? path.basename(target, ".jsonl") + : target; + + let events: Array>; + if (target.endsWith(".jsonl") && fs.existsSync(target)) { + // Direct path: parse in place (still strict via the reducer below). + events = fs + .readFileSync(target, "utf8") + .trim() + .split("\n") + .map((line) => JSON.parse(line) as z.infer); + } else { + events = await repo.read(turnId); + } + const state = reduceTurn(events); + const resolver = new TurnRepoContextResolver({ turnRepo: repo }); + const prefix = await resolver.resolve(state.definition.context); + const encode = (messages: Parameters[0]) => + convertFromMessages(messages) as unknown as JsonValue[]; + + console.log(`turn ${turnId}`); + console.log( + `agent ${state.definition.agent.resolved.agentId} model ${state.definition.agent.resolved.model.provider}/${state.definition.agent.resolved.model.model} calls ${state.modelCalls.length}`, + ); + + for (const call of state.modelCalls) { + if (onlyIndex !== undefined && call.index !== onlyIndex) continue; + const composed = composeModelRequest(state, call.index, prefix, encode); + console.log(`\n━━ model call ${call.index} ━━ (as sent to the provider)`); + console.log(`system (${composed.systemPrompt.length} chars): ${clip(composed.systemPrompt, full)}`); + console.log( + `tools (${composed.tools.length}): ${composed.tools.map((t) => t.name).join(", ")}`, + ); + console.log(`messages (${composed.messages.length}):`); + for (const message of composed.messages) { + const m = message as { role?: string; content?: unknown }; + const content = + typeof m.content === "string" + ? m.content + : JSON.stringify(m.content); + console.log(` [${m.role}] ${clip(content, full)}`); + } + if (call.error !== undefined) { + console.log(` → failed: ${call.error}`); + } else if (call.response !== undefined) { + const response = + typeof call.response.content === "string" + ? call.response.content + : JSON.stringify(call.response.content); + console.log(` → response (${call.finishReason}): ${clip(response, full)}`); + } + } +} + +main().catch((error: unknown) => { + console.error(error instanceof Error ? error.message : error); + process.exit(1); +}); diff --git a/apps/x/packages/core/src/turns/runtime.test.ts b/apps/x/packages/core/src/turns/runtime.test.ts index 3fad9d4b..1d2671ed 100644 --- a/apps/x/packages/core/src/turns/runtime.test.ts +++ b/apps/x/packages/core/src/turns/runtime.test.ts @@ -3,7 +3,6 @@ import type { z } from "zod"; import { MODEL_CALL_LIMIT_ERROR_CODE, type JsonValue, - type ModelRequestMessageRef, type ResolvedAgent, type ToolDescriptor, type TurnEvent, @@ -422,7 +421,7 @@ describe("plain model response (26.1)", () => { const request = log[1]; expect(request).toMatchObject({ modelCallIndex: 0, - request: { messages: [{ kind: "input" }] }, + request: { messages: ["input"] }, }); // The model received the composed payload: resolved system prompt, // snapshot tools, encoded messages. @@ -525,11 +524,11 @@ describe("mixed sync and async tools (26.2)", () => { ? secondRequest.request.messages : [], ).toEqual([ - { kind: "assistant", modelCallIndex: 0 }, - { kind: "toolResult", toolCallId: "A" }, - { kind: "toolResult", toolCallId: "B" }, - { kind: "toolResult", toolCallId: "C" }, - { kind: "toolResult", toolCallId: "D" }, + "assistant:0", + "toolResult:A", + "toolResult:B", + "toolResult:C", + "toolResult:D", ]); // The live model call saw the results in source order. expect( @@ -1166,7 +1165,7 @@ describe("crash recovery (26.7)", () => { function seedRequested( index: number, - refs: z.infer[] = [{ kind: "input" }], + refs: string[] = ["input"], ): z.infer { return { type: "model_call_requested", diff --git a/apps/x/packages/core/src/turns/runtime.ts b/apps/x/packages/core/src/turns/runtime.ts index fd7caaf4..d82e575a 100644 --- a/apps/x/packages/core/src/turns/runtime.ts +++ b/apps/x/packages/core/src/turns/runtime.ts @@ -6,7 +6,8 @@ import { type JsonValue, type ModelCallFailed, type ModelRequest, - type ModelRequestMessageRef, + assistantRef, + toolResultRef, type ToolCallState, type ToolDescriptor, type ToolInvocationRequested, @@ -856,25 +857,22 @@ class TurnAdvance { const index = this.state.modelCalls.length; const context = this.definition.context; const isRef = !Array.isArray(context); - let refs: Array>; + let refs: string[]; if (index === 0) { refs = !isRef && context.length > 0 - ? [{ kind: "context" }, { kind: "input" }] - : [{ kind: "input" }]; + ? ["context", "input"] + : ["input"]; } else { const previous = this.state.modelCalls[index - 1]; refs = previous.response !== undefined ? [ - { kind: "assistant", modelCallIndex: previous.index }, + assistantRef(previous.index), ...this.state.toolCalls .filter((tc) => tc.modelCallIndex === previous.index) .sort((a, b) => a.order - b.order) - .map((tc) => ({ - kind: "toolResult" as const, - toolCallId: tc.toolCallId, - })), + .map((tc) => toolResultRef(tc.toolCallId)), ] : []; // re-issue after an interrupted call adds nothing new } diff --git a/apps/x/packages/shared/src/turns.test.ts b/apps/x/packages/shared/src/turns.test.ts index 4f5f4d11..844d76b2 100644 --- a/apps/x/packages/shared/src/turns.test.ts +++ b/apps/x/packages/shared/src/turns.test.ts @@ -346,13 +346,13 @@ function syncToolSequence(): TEvent[] { const call0 = assistantCalls(toolCallPart("tc1", "echo")); return [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), permResolved("tc1", "allow"), invocation("tc1"), result("tc1", "echo"), - requested(1, [{ kind: "assistant", modelCallIndex: 0 }, { kind: "toolResult", toolCallId: "tc1" }]), + requested(1, ["assistant:0", "toolResult:tc1"]), completed(1, assistantText("done")), turnCompletedEv(), ]; @@ -393,7 +393,7 @@ describe("plain completion", () => { it("reduces a plain model response to a completed turn", () => { const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), stepEvent(0), completed(0, assistantText("done")), turnCompletedEv(), @@ -418,7 +418,7 @@ describe("plain completion", () => { it("leaves usage fields undefined when never reported", () => { const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, assistantText("a"), { inputTokens: 5 }), ]); expect(state.usage).toEqual({ inputTokens: 5 }); @@ -434,7 +434,7 @@ describe("tool execution", () => { ); const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), ]); expect(state.toolCalls).toHaveLength(2); @@ -457,7 +457,7 @@ describe("tool execution", () => { it("leaves identity undefined for tools missing from the agent snapshot", () => { const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, assistantCalls(toolCallPart("x1", "unknown-tool"))), result("x1", "unknown-tool", "runtime", "No such tool", true), ]); @@ -480,7 +480,7 @@ describe("tool execution", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created({ config: { autoPermission: true, humanAvailable: true, maxModelCalls: 20 } }), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), permClassified("tc1", "allow"), @@ -497,7 +497,7 @@ describe("tool execution", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), permClassificationFailed(["tc1"]), @@ -512,7 +512,7 @@ describe("tool execution", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), progress("tc1"), @@ -526,7 +526,7 @@ describe("tool execution", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), permResolved("tc1", "deny"), @@ -541,7 +541,7 @@ describe("context references", () => { it("accepts a referenced context with matching contextRef on requests", () => { const state = reduceTurn([ created({ context: { previousTurnId: PREV_TURN_ID } }), - requested(0, [{ kind: "input" }], { + requested(0, ["input"], { contextRef: { previousTurnId: PREV_TURN_ID }, }), completed(0, assistantText("done")), @@ -554,7 +554,7 @@ describe("context references", () => { expectCorruption( [ created({ context: { previousTurnId: PREV_TURN_ID } }), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), ], /contextRef inconsistent/, ); @@ -564,7 +564,7 @@ describe("context references", () => { expectCorruption( [ created({ context: { previousTurnId: PREV_TURN_ID } }), - requested(0, [{ kind: "input" }], { + requested(0, ["input"], { contextRef: { previousTurnId: "some-other-turn" }, }), ], @@ -576,7 +576,7 @@ describe("context references", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }], { + requested(0, ["input"], { contextRef: { previousTurnId: PREV_TURN_ID }, }), ], @@ -592,13 +592,13 @@ describe("context references", () => { ]; const state = reduceTurn([ created({ context }), - requested(0, [{ kind: "context" }, { kind: "input" }]), + requested(0, ["context", "input"]), completed(0, assistantText("done")), turnCompletedEv(), ]); expect(deriveTurnStatus(state)).toBe("completed"); expectCorruption( - [created({ context }), requested(0, [{ kind: "input" }])], + [created({ context }), requested(0, ["input"])], /references do not match/, ); }); @@ -607,7 +607,7 @@ describe("context references", () => { expectCorruption( [ created({ context: { previousTurnId: PREV_TURN_ID } }), - requested(0, [{ kind: "input" }], { + requested(0, ["input"], { contextRef: { previousTurnId: PREV_TURN_ID }, }), completed(0, assistantCalls(toolCallPart("tc1", "echo"))), @@ -616,8 +616,8 @@ describe("context references", () => { requested( 1, [ - { kind: "assistant", modelCallIndex: 0 }, - { kind: "toolResult", toolCallId: "tc1" }, + "assistant:0", + "toolResult:tc1", ], { contextRef: { previousTurnId: PREV_TURN_ID } }, ), @@ -635,7 +635,7 @@ describe("suspension", () => { ); const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), invocation("a1", fetchTool), @@ -653,7 +653,7 @@ describe("suspension", () => { ); const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), invocation("a1", fetchTool), @@ -673,7 +673,7 @@ describe("suspension", () => { ); const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("a1", fetchTool), invocation("a2", fetchTool), @@ -682,9 +682,9 @@ describe("suspension", () => { suspendedEv([], [{ id: "a1", tool: fetchTool }]), result("a1", "fetch", "async", "first"), requested(1, [ - { kind: "assistant", modelCallIndex: 0 }, - { kind: "toolResult", toolCallId: "a1" }, - { kind: "toolResult", toolCallId: "a2" }, + "assistant:0", + "toolResult:a1", + "toolResult:a2", ]), completed(1, assistantText("done")), turnCompletedEv(), @@ -697,7 +697,7 @@ describe("suspension", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), permResolved("tc1", "allow"), @@ -717,7 +717,7 @@ describe("suspension", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), invocation("a1", fetchTool), @@ -731,7 +731,7 @@ describe("suspension", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), suspendedEv([{ id: "tc1", name: "echo" }], []), ], /suspension while a model call is unsettled/, @@ -743,7 +743,7 @@ describe("recovery-shaped histories", () => { it("accepts an interrupted model call closed and re-issued (§23 fix)", () => { const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), callFailed(0, "interrupted by process restart"), requested(1, []), completed(1, assistantText("done")), @@ -760,7 +760,7 @@ describe("recovery-shaped histories", () => { created({ config: { autoPermission: false, humanAvailable: true, maxModelCalls: 1 }, }), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), callFailed(0, "interrupted"), requested(1, []), ], @@ -772,7 +772,7 @@ describe("recovery-shaped histories", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), result( @@ -782,7 +782,7 @@ describe("recovery-shaped histories", () => { "Tool execution was interrupted; its outcome is unknown and it was not retried.", true, ), - requested(1, [{ kind: "assistant", modelCallIndex: 0 }, { kind: "toolResult", toolCallId: "tc1" }]), + requested(1, ["assistant:0", "toolResult:tc1"]), completed(1, assistantText("done")), turnCompletedEv(), ]); @@ -794,7 +794,7 @@ describe("recovery-shaped histories", () => { const call0 = assistantCalls(toolCallPart("a1", "fetch")); const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("a1", fetchTool), suspendedEv([], [{ id: "a1", tool: fetchTool }]), @@ -808,7 +808,7 @@ describe("recovery-shaped histories", () => { it("accepts a live model failure closing the turn", () => { const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), callFailed(0), turnFailedEv("provider exploded"), ]); @@ -821,7 +821,7 @@ describe("recovery-shaped histories", () => { created({ config: { autoPermission: false, humanAvailable: true, maxModelCalls: 1 }, }), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), @@ -840,7 +840,7 @@ describe("invariants", () => { }); it("rejects a log not starting with turn_created", () => { - expectCorruption([requested(0, [{ kind: "input" }])], /first event must be turn_created/); + expectCorruption([requested(0, ["input"])], /first event must be turn_created/); }); it("rejects duplicate turn_created", () => { @@ -849,7 +849,7 @@ describe("invariants", () => { it("rejects mismatched turn ids", () => { expectCorruption( - [created(), { ...requested(0, [{ kind: "input" }]), turnId: "other" }], + [created(), { ...requested(0, ["input"]), turnId: "other" }], /does not match turn/, ); }); @@ -882,9 +882,9 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, assistantText("a")), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), ], /out of order/, ); @@ -892,7 +892,7 @@ describe("invariants", () => { it("rejects concurrent unresolved model requests", () => { expectCorruption( - [created(), requested(0, [{ kind: "input" }]), requested(1, [])], + [created(), requested(0, ["input"]), requested(1, [])], /concurrent unresolved model call requests/, ); }); @@ -912,7 +912,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, assistantText("a")), completed(0, assistantText("b")), ], @@ -924,7 +924,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), callFailed(0), completed(0, assistantText("a")), ], @@ -937,9 +937,9 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), - requested(1, [{ kind: "assistant", modelCallIndex: 0 }]), + requested(1, ["assistant:0"]), ], /while tool calls are unresolved/, ); @@ -952,11 +952,11 @@ describe("invariants", () => { created({ config: { autoPermission: false, humanAvailable: true, maxModelCalls: 1 }, }), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), - requested(1, [{ kind: "assistant", modelCallIndex: 0 }, { kind: "toolResult", toolCallId: "tc1" }]), + requested(1, ["assistant:0", "toolResult:tc1"]), ], /exceeds maxModelCalls/, ); @@ -966,7 +966,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed( 0, assistantCalls(toolCallPart("tc1", "echo"), toolCallPart("tc1", "echo")), @@ -981,11 +981,11 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), - requested(1, [{ kind: "assistant", modelCallIndex: 0 }, { kind: "toolResult", toolCallId: "tc1" }]), + requested(1, ["assistant:0", "toolResult:tc1"]), completed(1, assistantCalls(toolCallPart("tc1", "echo"))), ], /duplicate tool call id/, @@ -1000,16 +1000,16 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), invocation("tc2"), result("tc2", "echo"), requested(1, [ - { kind: "assistant", modelCallIndex: 0 }, - { kind: "toolResult", toolCallId: "tc2" }, - { kind: "toolResult", toolCallId: "tc1" }, + "assistant:0", + "toolResult:tc2", + "toolResult:tc1", ]), ], /references do not match/, @@ -1018,7 +1018,7 @@ describe("invariants", () => { it("rejects permission records targeting unknown tool calls", () => { expectCorruption( - [created(), requested(0, [{ kind: "input" }]), completed(0, assistantText("a")), permRequired("ghost", "echo")], + [created(), requested(0, ["input"]), completed(0, assistantText("a")), permRequired("ghost", "echo")], /unknown tool call/, ); }); @@ -1028,7 +1028,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), permRequired("tc1", "echo"), @@ -1042,7 +1042,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permClassified("tc1", "allow"), ], @@ -1055,7 +1055,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permResolved("tc1", "allow"), ], @@ -1068,7 +1068,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), permResolved("tc1", "allow"), @@ -1083,7 +1083,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), invocation("tc1"), @@ -1097,7 +1097,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), permResolved("tc1", "deny"), @@ -1112,7 +1112,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), invocation("tc1"), @@ -1126,7 +1126,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1", echoTool, { execution: "async" }), ], @@ -1137,7 +1137,7 @@ describe("invariants", () => { it("rejects progress without invocation", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); expectCorruption( - [created(), requested(0, [{ kind: "input" }]), completed(0, call0), progress("tc1")], + [created(), requested(0, ["input"]), completed(0, call0), progress("tc1")], /progress without invocation/, ); }); @@ -1147,7 +1147,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), @@ -1162,7 +1162,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), @@ -1175,7 +1175,7 @@ describe("invariants", () => { it("rejects sync-sourced results without invocation", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); expectCorruption( - [created(), requested(0, [{ kind: "input" }]), completed(0, call0), result("tc1", "echo", "sync")], + [created(), requested(0, ["input"]), completed(0, call0), result("tc1", "echo", "sync")], /sync tool result without invocation/, ); }); @@ -1185,7 +1185,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), result("tc1", "echo", "async"), @@ -1199,7 +1199,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, assistantText("a")), stepEvent(0), ], @@ -1210,7 +1210,7 @@ describe("invariants", () => { it("rejects completion while tool calls remain unresolved", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); expectCorruption( - [created(), requested(0, [{ kind: "input" }]), completed(0, call0), turnCompletedEv()], + [created(), requested(0, ["input"]), completed(0, call0), turnCompletedEv()], /completion while tool calls lack terminal results/, ); }); @@ -1220,7 +1220,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), result("tc1", "echo"), @@ -1237,14 +1237,14 @@ describe("invariants", () => { it("rejects terminal failure while tool calls remain unresolved", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); expectCorruption( - [created(), requested(0, [{ kind: "input" }]), completed(0, call0), turnFailedEv()], + [created(), requested(0, ["input"]), completed(0, call0), turnFailedEv()], /failure while tool calls lack terminal results/, ); }); it("rejects terminal events while a model call is unsettled", () => { expectCorruption( - [created(), requested(0, [{ kind: "input" }]), turnCancelledEv()], + [created(), requested(0, ["input"]), turnCancelledEv()], /cancellation while a model call is unsettled/, ); }); @@ -1252,7 +1252,7 @@ describe("invariants", () => { it("rejects any event after a terminal event", () => { const base = [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, assistantText("a")), ]; for (const terminal of [turnCompletedEv(), turnFailedEv(), turnCancelledEv()]) { @@ -1267,7 +1267,7 @@ describe("invariants", () => { expectCorruption( [ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, assistantText("a")), turnCompletedEv(), turnFailedEv(), @@ -1282,14 +1282,14 @@ describe("derivations", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const pending = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), permRequired("tc1", "echo"), suspendedEv([{ id: "tc1", name: "echo" }], []), ]); expect(deriveTurnStatus(pending)).toBe("suspended"); - const idle = reduceTurn([created(), requested(0, [{ kind: "input" }]), completed(0, call0)]); + const idle = reduceTurn([created(), requested(0, ["input"]), completed(0, call0)]); expect(deriveTurnStatus(idle)).toBe("idle"); }); @@ -1300,16 +1300,16 @@ describe("derivations", () => { ); const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), invocation("tc1"), invocation("tc2"), result("tc2", "echo", "sync", { n: 2 }), result("tc1", "echo", "sync", "plain text"), requested(1, [ - { kind: "assistant", modelCallIndex: 0 }, - { kind: "toolResult", toolCallId: "tc1" }, - { kind: "toolResult", toolCallId: "tc2" }, + "assistant:0", + "toolResult:tc1", + "toolResult:tc2", ]), completed(1, assistantText("done")), turnCompletedEv(), @@ -1326,7 +1326,7 @@ describe("derivations", () => { it("omits failed model calls from the transcript", () => { const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), callFailed(0, "interrupted"), requested(1, []), completed(1, assistantText("done")), @@ -1338,7 +1338,7 @@ describe("derivations", () => { it("transcript excludes the context prefix", () => { const state = reduceTurn([ created({ context: { previousTurnId: PREV_TURN_ID } }), - requested(0, [{ kind: "input" }], { + requested(0, ["input"], { contextRef: { previousTurnId: PREV_TURN_ID }, }), completed(0, assistantText("done")), @@ -1352,7 +1352,7 @@ describe("derivations", () => { const call0 = assistantCalls(toolCallPart("tc1", "echo")); const state = reduceTurn([ created(), - requested(0, [{ kind: "input" }]), + requested(0, ["input"]), completed(0, call0), ]); expect(() => turnTranscript(state)).toThrowError(/unresolved/); diff --git a/apps/x/packages/shared/src/turns.ts b/apps/x/packages/shared/src/turns.ts index 3b71b0d3..6d28e035 100644 --- a/apps/x/packages/shared/src/turns.ts +++ b/apps/x/packages/shared/src/turns.ts @@ -130,15 +130,39 @@ export const TurnCreated = z.object({ // identical to turn_created.agent.resolved by construction. The exact // provider payload is rebuilt deterministically by the request composer // (core), which is the same code path the loop sends through. -export const ModelRequestMessageRef = z.discriminatedUnion("kind", [ - z.object({ kind: z.literal("context") }), - z.object({ kind: z.literal("input") }), - z.object({ - kind: z.literal("assistant"), - modelCallIndex: z.number().int().nonnegative(), - }), - z.object({ kind: z.literal("toolResult"), toolCallId: z.string() }), -]); +// Compact string refs so raw JSONL reads naturally: +// "context" | "input" | "assistant:" | "toolResult:" +export const ModelRequestMessageRef = z + .string() + .regex(/^(context|input|assistant:\d+|toolResult:.+)$/); + +export type ParsedRequestRef = + | { kind: "context" } + | { kind: "input" } + | { kind: "assistant"; modelCallIndex: number } + | { kind: "toolResult"; toolCallId: string }; + +export const assistantRef = (modelCallIndex: number): string => + `assistant:${modelCallIndex}`; +export const toolResultRef = (toolCallId: string): string => + `toolResult:${toolCallId}`; + +export function parseRequestRef(ref: string): ParsedRequestRef { + if (ref === "context" || ref === "input") { + return { kind: ref }; + } + if (ref.startsWith("assistant:")) { + const modelCallIndex = Number(ref.slice("assistant:".length)); + if (!Number.isInteger(modelCallIndex) || modelCallIndex < 0) { + throw new Error(`malformed request ref: ${ref}`); + } + return { kind: "assistant", modelCallIndex }; + } + if (ref.startsWith("toolResult:")) { + return { kind: "toolResult", toolCallId: ref.slice("toolResult:".length) }; + } + throw new Error(`malformed request ref: ${ref}`); +} export const ModelRequest = z.object({ contextRef: TurnContextRef.optional(), @@ -505,21 +529,18 @@ function applyModelCallRequested( } const context = state.definition.context; - let expectedRefs: Array>; + let expectedRefs: string[]; if (event.modelCallIndex === 0) { if (isContextRef(context)) { if (event.request.contextRef?.previousTurnId !== context.previousTurnId) { fail("model request contextRef inconsistent with turn context"); } - expectedRefs = [{ kind: "input" }]; + expectedRefs = ["input"]; } else { if (event.request.contextRef !== undefined) { fail("model request has contextRef but turn context is inline"); } - expectedRefs = - context.length > 0 - ? [{ kind: "context" }, { kind: "input" }] - : [{ kind: "input" }]; + expectedRefs = context.length > 0 ? ["context", "input"] : ["input"]; } } else { if (event.request.contextRef !== undefined) { @@ -529,11 +550,10 @@ function applyModelCallRequested( expectedRefs = previous.response !== undefined ? [ - { kind: "assistant", modelCallIndex: previous.index }, - ...batchToolCalls(state, previous.index).map((tc) => ({ - kind: "toolResult" as const, - toolCallId: tc.toolCallId, - })), + assistantRef(previous.index), + ...batchToolCalls(state, previous.index).map((tc) => + toolResultRef(tc.toolCallId), + ), ] : []; // re-issue after an interrupted call adds nothing new } @@ -988,7 +1008,8 @@ export function requestMessagesFor( if (!call) { throw new Error(`no model call at index ${modelCallIndex}`); } - return call.request.messages.flatMap((ref) => { + return call.request.messages.flatMap((raw) => { + const ref = parseRequestRef(raw); switch (ref.kind) { case "context": { const context = state.definition.context; From 32e02d588c95a852f98fb7810d922879aeeab86c Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 13:17:30 +0530 Subject: [PATCH 17/24] feat(x): agent snapshot inheritance, session inspection, runtime docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Third application of the reference mechanism: session turns whose system prompt + tools are byte-identical to the context predecessor's materialized snapshot persist { agentId, model, inheritedFrom } instead of ~70KB per turn (measured: turn 2 of a session is now ~1.1KB total). Inheritance is decided at createTurn by equality against the materialized predecessor; the model stays concrete, and on materialization the inherited record's own agentId/model win — a rule the new test matrix caught as a real bug (the chain base's model was overriding a mid-session model switch). Reducer invariants: inherited snapshots must reference the context predecessor; tool identity arrives via invocation events. Test matrix per review: prompt-diff -> full snapshot, tools-diff -> full snapshot, model-switch -> inherits with concrete model, multi-hop chains materialize, standalone turns never inherit, unreadable predecessor falls back to full, cyclic inheritance is corruption, sessions denormalize the model from inherited snapshots. The inspector now handles sessions too (auto-detected): overview with per-turn status/size/input preview, --turns to cascade full turn inspection. Documented in a new repo-root AGENTS.md (storage layout, reference model, inspector usage, invariants) with a CLAUDE.md pointer. Breaking for dev turn files: wipe ~/.rowboat/storage. Co-Authored-By: Claude Fable 5 --- AGENTS.md | 74 +++++++ CLAUDE.md | 1 + .../packages/core/docs/turn-runtime-design.md | 29 +++ apps/x/packages/core/package.json | 3 +- .../core/src/sessions/sessions.test.ts | 45 +++- .../core/src/turns/compose-model-request.ts | 13 +- .../core/src/turns/context-resolver.test.ts | 76 +++++++ .../core/src/turns/context-resolver.ts | 33 +++ apps/x/packages/core/src/turns/inspect-cli.ts | 174 +++++++++++---- .../x/packages/core/src/turns/runtime.test.ts | 199 +++++++++++++++++- apps/x/packages/core/src/turns/runtime.ts | 45 +++- apps/x/packages/shared/package.json | 36 ++-- apps/x/packages/shared/src/turns.test.ts | 61 ++++++ apps/x/packages/shared/src/turns.ts | 47 ++++- 14 files changed, 759 insertions(+), 77 deletions(-) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..9a9b0a73 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,74 @@ +# AGENTS.md — Working on the Rowboat runtime + +Context for AI coding agents (and humans) working on this repo. General +codebase orientation lives in `CLAUDE.md`; this file covers the **new +turn/session runtime** in `apps/x` — its storage, its debugging tools, and +the invariants you must not break. + +## The runtime in one paragraph + +Chats are **sessions**; each user message starts a **turn**. Both are +append-only JSONL event logs under `~/.rowboat/storage/{turns,sessions}/YYYY/MM/DD/`. +All state is derived by pure reducers (`reduceTurn`, `reduceSession` in +`@x/shared/src/turns.ts` / `sessions.ts`) shared byte-for-byte between the +main process and the renderer. Design specs: +`apps/x/packages/core/docs/turn-runtime-design.md` and `session-design.md` — +read the relevant spec before changing runtime behavior. + +## Storage is reference-based — files store each fact once + +Three applications of the same mechanism: + +1. **Context**: a session turn's `context` is `{ previousTurnId }`; the + conversation prefix is materialized by walking the chain + (`TurnRepoContextResolver`). +2. **Model requests**: `model_call_requested.request.messages` is a list of + string refs into the turn's own events — `"context"`, `"input"`, + `"assistant:"`, `"toolResult:"` — recording only what + is NEW since the previous call. +3. **Agent snapshots**: when a turn's system prompt + tools are + byte-identical to its predecessor's, `turn_created.agent.resolved` is + `{ agentId, model, inheritedFrom }` instead of re-persisting ~70KB. The + model stays concrete (mid-session model switches still inherit). + +The exact provider payload is rebuilt by `composeModelRequest` +(`packages/core/src/turns/compose-model-request.ts`) — **the same code path +the loop transmits through**, so the file plus the composer reproduce the +wire bytes exactly (there is a property test asserting composed == sent). + +## Inspecting turns and sessions + +```bash +cd apps/x/packages/core + +# A whole session: title, per-turn status/size/input preview +npm run inspect -- + +# One turn: per model call, the EXACT provider payload — resolved system +# prompt, tool list, wire-form messages (user-message context woven in, +# tool-result envelopes), and the response/failure +npm run inspect -- [modelCallIndex] [--full] + +# Cascade full turn inspection across a session +npm run inspect -- --turns +``` + +`--full` prints untruncated system prompts and message contents. Turn vs +session ids are auto-detected. This is the intended way to see "what did the +model actually receive" — the raw JSONL deliberately stores structural facts +and references, never the derived wire form. + +## Invariants to respect + +- Turn/session files are **append-only**; reducers reject impossible + histories loudly (`TurnCorruptionError`). Never hand-edit files. +- Durable events are persisted **before** side effects (model calls, tool + invocations). Deltas (`text_delta`, …) are stream-only, never persisted. +- The reducers in `@x/shared` must stay pure (no I/O, no node imports) — + the renderer imports them directly. +- Every behavior change needs tests: reducers in `packages/shared`, + runtime/sessions in `packages/core`, renderer stores/views in + `apps/renderer` (all vitest; run `npm test` per package). +- Schema changes: the schema is pre-release (`schemaVersion: 1` throughout); + breaking changes are acceptable but require wiping `~/.rowboat/storage` + and a note in the commit message. diff --git a/CLAUDE.md b/CLAUDE.md index 75a0f6a5..bf34bd90 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -110,6 +110,7 @@ Long-form docs for specific features. Read the relevant file before making chang |---------|-----| | Live Notes — single `live:` frontmatter block (one objective + optional cron / windows / eventMatchCriteria) that turns a note into a self-updating artifact, panel UI, Copilot skill, prompts catalog | `apps/x/LIVE_NOTE.md` | | Analytics — PostHog event catalog, person properties, use-case taxonomy, how to add a new event | `apps/x/ANALYTICS.md` | +| Turn/session runtime — event-sourced storage, reference model, the `npm run inspect` debugger | `AGENTS.md` (repo root), `apps/x/packages/core/docs/turn-runtime-design.md`, `session-design.md` | ## Common Tasks diff --git a/apps/x/packages/core/docs/turn-runtime-design.md b/apps/x/packages/core/docs/turn-runtime-design.md index 88290b7e..091168f2 100644 --- a/apps/x/packages/core/docs/turn-runtime-design.md +++ b/apps/x/packages/core/docs/turn-runtime-design.md @@ -390,6 +390,35 @@ Rules: It rejects the execution and does not append `turn_failed`. - The reducer treats `context` as opaque data and never resolves references. +### 6.7 Agent snapshot inheritance + +Session turns whose resolved system prompt and tool set are byte-identical +to the context predecessor's materialized snapshot persist an inherited +form instead of repeating ~tens of KB per turn: + +```ts +type ResolvedAgentSnapshot = + | ResolvedAgent + | { agentId: string; model: ModelDescriptor; inheritedFrom: string }; +``` + +Rules: + +- Inheritance is decided at `createTurn` by comparing against the + predecessor's materialized snapshot; any difference in system prompt or + tools persists the full snapshot. An unreadable predecessor falls back to + the full snapshot. +- The model stays concrete: it is small, the session index denormalizes it, + and a mid-session model switch must not block inheritance. On + materialization the inherited record's own `agentId`/`model` win; only the + heavy fields come from the chain base. +- `inheritedFrom` must equal the turn's `context.previousTurnId` (reducer + invariant); materialization walks the chain with cycle detection + (`IContextResolver.resolveAgent`), exactly like context references. +- The reducer treats inherited snapshots as opaque: tool identity for + extraction arrives via `tool_invocation_requested` events instead of the + descriptor lookup. + ## 7. Turn creation schema The authoritative initial event is: diff --git a/apps/x/packages/core/package.json b/apps/x/packages/core/package.json index 8ba46048..18c32a8c 100644 --- a/apps/x/packages/core/package.json +++ b/apps/x/packages/core/package.json @@ -9,7 +9,8 @@ "dev": "tsc -w -p tsconfig.build.json", "test": "vitest run", "test:watch": "vitest", - "inspect-turn": "node dist/turns/inspect-cli.js" + "inspect-turn": "node dist/turns/inspect-cli.js", + "inspect": "node dist/turns/inspect-cli.js" }, "dependencies": { "@agentclientprotocol/claude-agent-acp": "^0.39.0", diff --git a/apps/x/packages/core/src/sessions/sessions.test.ts b/apps/x/packages/core/src/sessions/sessions.test.ts index 6ff498d8..d4efdaa1 100644 --- a/apps/x/packages/core/src/sessions/sessions.test.ts +++ b/apps/x/packages/core/src/sessions/sessions.test.ts @@ -187,6 +187,9 @@ class FakeTurnRuntime implements ITurnRuntime { logs = new Map(); createError?: Error; script?: AdvanceScript; + // When set, session turns get an inherited agent snapshot (like the real + // runtime does for identical predecessors). + inheritSnapshots = false; private n = 0; async createTurn(input: CreateTurnInput): Promise { @@ -196,7 +199,22 @@ class FakeTurnRuntime implements ITurnRuntime { this.createTurnInputs.push(input); this.n += 1; const turnId = `2026-07-02T10-00-00Z-${String(this.n).padStart(7, "0")}-000`; - this.logs.set(turnId, [createdEvent(turnId, input)]); + const created = createdEvent(turnId, input); + if ( + this.inheritSnapshots && + created.type === "turn_created" && + !Array.isArray(input.context) + ) { + created.agent = { + ...created.agent, + resolved: { + agentId: FIXTURE_AGENT.agentId, + model: FIXTURE_MODEL, + inheritedFrom: input.context.previousTurnId, + }, + }; + } + this.logs.set(turnId, [created]); return turnId; } @@ -430,6 +448,31 @@ describe("sendMessage (13.3)", () => { } }); + it("denormalizes the model correctly for inherited agent snapshots", async () => { + const { sessions, repo, fake } = makeSessions(); + fake.inheritSnapshots = true; + const sessionId = await sessions.createSession(); + const first = await sessions.sendMessage(sessionId, user("one"), { + agent: { agentId: "copilot" }, + }); + fake.setLog(first.turnId, turnLog(first.turnId, sessionId, "completed")); + + const second = await sessions.sendMessage(sessionId, user("two"), { + agent: { agentId: "copilot" }, + }); + // The second turn's created event carries an inherited snapshot; the + // session still denormalizes the concrete model onto turn_appended, + // and status derivation reduces the inherited log fine. + const created = fake.logs.get(second.turnId)?.[0]; + expect( + created?.type === "turn_created" && "inheritedFrom" in created.agent.resolved, + ).toBe(true); + const events = await (repo as InMemorySessionRepo).read(sessionId); + const appended = events.filter((e) => e.type === "turn_appended"); + expect(appended[1]).toMatchObject({ model: FIXTURE_MODEL }); + expect(sessions.listSessions()[0].lastModel).toEqual(FIXTURE_MODEL); + }); + it("serializes concurrent sends: exactly one wins", async () => { const { sessions, repo } = makeSessions(); const sessionId = await sessions.createSession(); diff --git a/apps/x/packages/core/src/turns/compose-model-request.ts b/apps/x/packages/core/src/turns/compose-model-request.ts index ec0a7ef7..9811e9c0 100644 --- a/apps/x/packages/core/src/turns/compose-model-request.ts +++ b/apps/x/packages/core/src/turns/compose-model-request.ts @@ -2,6 +2,7 @@ import type { z } from "zod"; import { type ConversationMessage, type JsonValue, + type ResolvedAgent, type ToolDescriptor, type TurnState, requestMessagesFor, @@ -29,6 +30,9 @@ export function composeModelRequest( // The materialized cross-turn prefix (contextResolver output). Ignored // for inline-context turns, whose context rides the {context} ref. resolvedPrefix: Array>, + // The materialized agent snapshot (contextResolver.resolveAgent output — + // inherited snapshots must be resolved before composing). + agent: z.infer, encode: (messages: Array>) => JsonValue[], ): ComposedModelRequest { const call = state.modelCalls[modelCallIndex]; @@ -41,9 +45,9 @@ export function composeModelRequest( structural.push(...requestMessagesFor(state, index)); } return { - systemPrompt: state.definition.agent.resolved.systemPrompt, + systemPrompt: agent.systemPrompt, messages: encode(structural), - tools: state.definition.agent.resolved.tools, + tools: agent.tools, parameters: call.request.parameters, }; } @@ -57,5 +61,8 @@ export async function materializeModelRequest( encode: (messages: Array>) => JsonValue[], ): Promise { const prefix = await contextResolver.resolve(state.definition.context); - return composeModelRequest(state, modelCallIndex, prefix, encode); + const agent = await contextResolver.resolveAgent( + state.definition.agent.resolved, + ); + return composeModelRequest(state, modelCallIndex, prefix, agent, encode); } diff --git a/apps/x/packages/core/src/turns/context-resolver.test.ts b/apps/x/packages/core/src/turns/context-resolver.test.ts index 6ccfe671..35621178 100644 --- a/apps/x/packages/core/src/turns/context-resolver.test.ts +++ b/apps/x/packages/core/src/turns/context-resolver.test.ts @@ -241,6 +241,82 @@ describe("TurnRepoContextResolver", () => { ).rejects.toThrowError(/turn not found/); }); + it("resolveAgent materializes inherited snapshots through the chain", async () => { + const repo = new InMemoryTurnRepo(); + repo.seed(completedTurnLog(T1, [], "q1", "a1")); // concrete base + const inheritedLog = completedTurnLog(T2, { previousTurnId: T1 }, "q2", "a2").map( + (event) => + event.type === "turn_created" + ? { + ...event, + agent: { + ...event.agent, + resolved: { + agentId: "copilot", + model: { provider: "fake", model: "m" }, + inheritedFrom: T1, + }, + }, + } + : event, + ); + repo.seed(inheritedLog); + const resolver = new TurnRepoContextResolver({ turnRepo: repo }); + + const concrete = await resolver.resolveAgent({ + agentId: "copilot", + systemPrompt: "SYS", + model: { provider: "fake", model: "m" }, + tools: [], + }); + expect(concrete.systemPrompt).toBe("SYS"); // passthrough + + const materialized = await resolver.resolveAgent({ + agentId: "copilot", + model: { provider: "fake", model: "m2" }, + inheritedFrom: T2, // hops T2 -> T1 + }); + expect(materialized.systemPrompt).toBe("SYS"); + expect(materialized.tools).toEqual([]); + // The inherited record's own model wins over the chain base's model. + expect(materialized.model).toEqual({ provider: "fake", model: "m2" }); + }); + + it("resolveAgent rejects cyclic inheritance as corruption", async () => { + const repo = new InMemoryTurnRepo(); + const cyclic = completedTurnLog(T1, [], "q1", "a1").map((event) => + event.type === "turn_created" + ? { + ...event, + context: { previousTurnId: T1 }, + agent: { + ...event.agent, + resolved: { + agentId: "copilot", + model: { provider: "fake", model: "m" }, + inheritedFrom: T1, + }, + }, + } + : event, + ); + // Fix the request contextRef to match the now-ref context. + const fixed = cyclic.map((event) => + event.type === "model_call_requested" + ? { ...event, request: { ...event.request, contextRef: { previousTurnId: T1 } } } + : event, + ); + repo.seed(fixed); + const resolver = new TurnRepoContextResolver({ turnRepo: repo }); + await expect( + resolver.resolveAgent({ + agentId: "copilot", + model: { provider: "fake", model: "m" }, + inheritedFrom: T1, + }), + ).rejects.toThrowError(TurnCorruptionError); + }); + it("rejects cyclic reference chains as corruption", async () => { const repo = new InMemoryTurnRepo(); // Two turns referencing each other (only constructable by corruption). diff --git a/apps/x/packages/core/src/turns/context-resolver.ts b/apps/x/packages/core/src/turns/context-resolver.ts index 0dce3181..3785cc26 100644 --- a/apps/x/packages/core/src/turns/context-resolver.ts +++ b/apps/x/packages/core/src/turns/context-resolver.ts @@ -1,8 +1,11 @@ import type { z } from "zod"; import { type ConversationMessage, + type ResolvedAgent, + type ResolvedAgentSnapshot, type TurnContext, TurnCorruptionError, + isInheritedSnapshot, reduceTurn, turnTranscript, } from "@x/shared/dist/turns.js"; @@ -17,6 +20,12 @@ export interface IContextResolver { resolve( context: z.infer, ): Promise>>; + // Materializes an inherited agent snapshot by walking inheritedFrom to + // the nearest concrete snapshot (same discipline as context references: + // deterministic, from durable state, cycle-checked). + resolveAgent( + resolved: z.infer, + ): Promise>; } export class TurnRepoContextResolver implements IContextResolver { @@ -52,4 +61,28 @@ export class TurnRepoContextResolver implements IContextResolver { segments.reverse(); return segments.flat(); } + + async resolveAgent( + resolved: z.infer, + ): Promise> { + if (!isInheritedSnapshot(resolved)) { + return resolved; + } + const visited = new Set(); + let current: z.infer = resolved; + while (isInheritedSnapshot(current)) { + const turnId = current.inheritedFrom; + if (visited.has(turnId)) { + throw new TurnCorruptionError( + `cyclic agent snapshot inheritance at turn ${turnId}`, + ); + } + visited.add(turnId); + const events = await this.turnRepo.read(turnId); + current = reduceTurn(events).definition.agent.resolved; + } + // Only the heavy fields inherit; the turn's own concrete identity + // (agentId, model — a mid-session model switch) always wins. + return { ...current, agentId: resolved.agentId, model: resolved.model }; + } } diff --git a/apps/x/packages/core/src/turns/inspect-cli.ts b/apps/x/packages/core/src/turns/inspect-cli.ts index 3329d99d..b662e407 100644 --- a/apps/x/packages/core/src/turns/inspect-cli.ts +++ b/apps/x/packages/core/src/turns/inspect-cli.ts @@ -1,31 +1,48 @@ -// Turn inspector: prints, per model call, the EXACT provider payload the -// loop sent — rebuilt from the durable file by the same composer the loop -// transmits through (compose-model-request.ts). This is where the woven -// wire-form messages (user-message context, attachments, tool-result -// envelopes) are visible; the file itself stores only structural facts and -// references. +// Runtime storage inspector. Works on turns AND sessions (auto-detected): // -// Usage: -// npm run inspect-turn -- [modelCallIndex] [--full] +// npm run inspect -- [modelCallIndex] [--full] [--turns] // -// --full prints the entire system prompt and untruncated message contents. +// Turn mode prints, per model call, the EXACT provider payload the loop sent +// — rebuilt from the durable file by the same composer the loop transmits +// through (compose-model-request.ts). This is where the woven wire-form +// messages (user-message context, attachments, tool-result envelopes) are +// visible; the file itself stores only structural facts and references. +// +// Session mode prints the session overview (title, turns, statuses, sizes); +// pass --turns to cascade full turn inspection for every turn. +// +// --full prints untruncated system prompts and message contents. import fs from "node:fs"; import path from "node:path"; import { + deriveTurnStatus, reduceTurn, type JsonValue, type TurnEvent, + type TurnState, } from "@x/shared/dist/turns.js"; +import { reduceSession } from "@x/shared/dist/sessions.js"; import type { z } from "zod"; import { convertFromMessages } from "../agents/runtime.js"; import { WorkDir } from "../config/config.js"; +import { FSSessionRepo } from "../sessions/fs-repo.js"; import { composeModelRequest } from "./compose-model-request.js"; import { TurnRepoContextResolver } from "./context-resolver.js"; import { FSTurnRepo } from "./fs-repo.js"; +const turnRepo = new FSTurnRepo({ + turnsRootDir: path.join(WorkDir, "storage", "turns"), +}); +const sessionRepo = new FSSessionRepo({ + sessionsRootDir: path.join(WorkDir, "storage", "sessions"), +}); +const resolver = new TurnRepoContextResolver({ turnRepo }); +const encode = (messages: Parameters[0]) => + convertFromMessages(messages) as unknown as JsonValue[]; + function usage(): never { console.error( - "usage: inspect-turn [modelCallIndex] [--full]", + "usage: inspect [modelCallIndex] [--full] [--turns]", ); process.exit(1); } @@ -35,46 +52,43 @@ function clip(text: string, full: boolean, limit = 400): string { return `${text.slice(0, limit)}… [${text.length} chars total; pass --full]`; } -async function main(): Promise { - const args = process.argv.slice(2).filter((a) => a !== "--full"); - const full = process.argv.includes("--full"); - const target = args[0]; - if (!target) usage(); - const onlyIndex = args[1] !== undefined ? Number(args[1]) : undefined; +function inputPreview(state: TurnState): string { + const content = state.definition.input.content; + const text = + typeof content === "string" + ? content + : content + .map((part) => (part.type === "text" ? part.text : `<${part.type}>`)) + .join(" "); + return text.replace(/\s+/g, " ").slice(0, 60); +} - const turnsRootDir = path.join(WorkDir, "storage", "turns"); - const repo = new FSTurnRepo({ turnsRootDir }); - const turnId = target.endsWith(".jsonl") - ? path.basename(target, ".jsonl") - : target; - - let events: Array>; - if (target.endsWith(".jsonl") && fs.existsSync(target)) { - // Direct path: parse in place (still strict via the reducer below). - events = fs - .readFileSync(target, "utf8") - .trim() - .split("\n") - .map((line) => JSON.parse(line) as z.infer); - } else { - events = await repo.read(turnId); - } +async function inspectTurn( + turnId: string, + events: Array>, + onlyIndex: number | undefined, + full: boolean, +): Promise { const state = reduceTurn(events); - const resolver = new TurnRepoContextResolver({ turnRepo: repo }); const prefix = await resolver.resolve(state.definition.context); - const encode = (messages: Parameters[0]) => - convertFromMessages(messages) as unknown as JsonValue[]; + const agent = await resolver.resolveAgent(state.definition.agent.resolved); - console.log(`turn ${turnId}`); + const inherited = + "inheritedFrom" in state.definition.agent.resolved + ? ` (snapshot inherited from ${state.definition.agent.resolved.inheritedFrom})` + : ""; + console.log(`turn ${turnId} status ${deriveTurnStatus(state)}`); console.log( - `agent ${state.definition.agent.resolved.agentId} model ${state.definition.agent.resolved.model.provider}/${state.definition.agent.resolved.model.model} calls ${state.modelCalls.length}`, + `agent ${agent.agentId} model ${agent.model.provider}/${agent.model.model} calls ${state.modelCalls.length}${inherited}`, ); for (const call of state.modelCalls) { if (onlyIndex !== undefined && call.index !== onlyIndex) continue; - const composed = composeModelRequest(state, call.index, prefix, encode); + const composed = composeModelRequest(state, call.index, prefix, agent, encode); console.log(`\n━━ model call ${call.index} ━━ (as sent to the provider)`); - console.log(`system (${composed.systemPrompt.length} chars): ${clip(composed.systemPrompt, full)}`); + console.log( + `system (${composed.systemPrompt.length} chars): ${clip(composed.systemPrompt, full)}`, + ); console.log( `tools (${composed.tools.length}): ${composed.tools.map((t) => t.name).join(", ")}`, ); @@ -82,9 +96,7 @@ async function main(): Promise { for (const message of composed.messages) { const m = message as { role?: string; content?: unknown }; const content = - typeof m.content === "string" - ? m.content - : JSON.stringify(m.content); + typeof m.content === "string" ? m.content : JSON.stringify(m.content); console.log(` [${m.role}] ${clip(content, full)}`); } if (call.error !== undefined) { @@ -99,6 +111,82 @@ async function main(): Promise { } } +async function inspectSession( + sessionId: string, + full: boolean, + cascade: boolean, +): Promise { + const state = reduceSession(await sessionRepo.read(sessionId)); + console.log(`session ${sessionId}`); + console.log( + `title "${state.title ?? ""}" turns ${state.turns.length} created ${state.createdAt} updated ${state.updatedAt}`, + ); + for (const ref of state.turns) { + try { + const events = await turnRepo.read(ref.turnId); + const turnState = reduceTurn(events); + const bytes = JSON.stringify(events).length; + console.log( + ` ${String(ref.sessionSeq).padStart(3)}. ${ref.turnId} ${deriveTurnStatus(turnState).padEnd(9)} ${turnState.modelCalls.length} calls ~${Math.round(bytes / 1024)}KB "${inputPreview(turnState)}"`, + ); + } catch (error) { + console.log( + ` ${String(ref.sessionSeq).padStart(3)}. ${ref.turnId} UNREADABLE: ${error instanceof Error ? error.message : error}`, + ); + } + } + if (cascade) { + for (const ref of state.turns) { + console.log(`\n════════ turn ${ref.sessionSeq} ════════`); + try { + await inspectTurn(ref.turnId, await turnRepo.read(ref.turnId), undefined, full); + } catch (error) { + console.log(`unreadable: ${error instanceof Error ? error.message : error}`); + } + } + } else { + console.log(`\n(inspect a turn: npm run inspect -- ; whole session: --turns)`); + } +} + +async function main(): Promise { + const flags = new Set(process.argv.slice(2).filter((a) => a.startsWith("--"))); + const args = process.argv.slice(2).filter((a) => !a.startsWith("--")); + const full = flags.has("--full"); + const cascade = flags.has("--turns"); + const target = args[0]; + if (!target) usage(); + const onlyIndex = args[1] !== undefined ? Number(args[1]) : undefined; + + const id = target.endsWith(".jsonl") ? path.basename(target, ".jsonl") : target; + + // Direct file path: session files live under storage/sessions. + if (target.endsWith(".jsonl") && fs.existsSync(target)) { + if (path.resolve(target).includes(`${path.sep}sessions${path.sep}`)) { + await inspectSession(id, full, cascade); + return; + } + const events = fs + .readFileSync(target, "utf8") + .trim() + .split("\n") + .map((line) => JSON.parse(line) as z.infer); + await inspectTurn(id, events, onlyIndex, full); + return; + } + + // Bare id: try turn first, then session. + try { + const events = await turnRepo.read(id); + await inspectTurn(id, events, onlyIndex, full); + } catch (error) { + if (!(error instanceof Error) || !/turn not found/.test(error.message)) { + throw error; + } + await inspectSession(id, full, cascade); + } +} + main().catch((error: unknown) => { console.error(error instanceof Error ? error.message : error); process.exit(1); diff --git a/apps/x/packages/core/src/turns/runtime.test.ts b/apps/x/packages/core/src/turns/runtime.test.ts index 1d2671ed..170ba61b 100644 --- a/apps/x/packages/core/src/turns/runtime.test.ts +++ b/apps/x/packages/core/src/turns/runtime.test.ts @@ -146,6 +146,7 @@ function hangUntilAbort(onStarted?: () => void): ScriptedCall { class FakeModelRegistry implements IModelRegistry { requests: ModelStreamRequest[] = []; + resolved: Array = []; private next = 0; constructor(private readonly calls: ScriptedCall[]) {} @@ -153,6 +154,7 @@ class FakeModelRegistry implements IModelRegistry { async resolve( descriptor: ResolvedModel["descriptor"], ): Promise { + this.resolved.push(descriptor); return { descriptor, // Identity encoding: tests assert on structural messages directly. @@ -275,7 +277,10 @@ class FakeBus { } class FakeIdGen { - private n = 0; + private n: number; + constructor(start = 0) { + this.n = start; + } async next(): Promise { this.n += 1; return `2026-07-02T10-00-00Z-${String(this.n).padStart(7, "0")}-000`; @@ -296,6 +301,7 @@ function makeRuntime(opts: { agent?: z.infer; agentError?: string; repo?: InMemoryTurnRepo; + idStart?: number; } = {}) { const repo = opts.repo ?? new InMemoryTurnRepo(); const models = new FakeModelRegistry(opts.models ?? []); @@ -304,7 +310,7 @@ function makeRuntime(opts: { const bus = new FakeBus(); const runtime = new TurnRuntime({ turnRepo: repo, - idGenerator: new FakeIdGen(), + idGenerator: new FakeIdGen(opts.idStart ?? 0), clock: new FakeClock(), agentResolver: new FakeAgentResolver(opts.agent ?? defaultAgent, opts.agentError), modelRegistry: models, @@ -540,7 +546,7 @@ describe("mixed sync and async tools (26.2)", () => { // reproduces exactly what the model received. const state = reduceTurn(log); for (const index of [0, 1]) { - const composed = composeModelRequest(state, index, [], (m) => m as never); + const composed = composeModelRequest(state, index, [], defaultAgent, (m) => m as never); expect(composed.messages).toEqual(models.requests[index].messages); expect(composed.systemPrompt).toBe(models.requests[index].systemPrompt); expect(composed.tools).toEqual(models.requests[index].tools); @@ -1420,6 +1426,193 @@ describe("crash recovery (26.7)", () => { // §26.8 Historical and live reconstruction // --------------------------------------------------------------------------- +describe("agent snapshot inheritance", () => { + it("inherits system prompt + tools from an identical context predecessor", async () => { + const { runtime, repo, models } = makeRuntime({ + models: [ + respond(completedResp(assistantText("first"))), + respond(completedResp(assistantText("second"))), + ], + }); + const first = await newTurn(runtime, { sessionId: "S" }); + await advanceAndSettle(runtime, first); + + const second = await runtime.createTurn({ + agent: { agentId: "copilot" }, + sessionId: "S", + context: { previousTurnId: first }, + input: user("again"), + config: { humanAvailable: true }, + }); + const created = (await persisted(repo, second))[0]; + expect(created.type === "turn_created" ? created.agent.resolved : null).toEqual({ + agentId: "copilot", + model: defaultAgent.model, + inheritedFrom: first, + }); + + // The inherited turn still sends the full materialized snapshot. + const { outcome } = await advanceAndSettle(runtime, second); + expect(outcome?.status).toBe("completed"); + expect(models.requests[1].systemPrompt).toBe("SYS"); + expect(models.requests[1].tools).toEqual(defaultAgent.tools); + }); + + it("a model switch still inherits the prompt and tools (model stays concrete)", async () => { + const repo = new InMemoryTurnRepo(); + const a = makeRuntime({ + repo, + models: [respond(completedResp(assistantText("first")))], + }); + const first = await newTurn(a.runtime, { sessionId: "S" }); + await advanceAndSettle(a.runtime, first); + + const switched = { + ...defaultAgent, + model: { provider: "anthropic", model: "claude-x" }, + }; + const b = makeRuntime({ + repo, + agent: switched, + idStart: 200, + models: [respond(completedResp(assistantText("second")))], + }); + const second = await b.runtime.createTurn({ + agent: { agentId: "copilot" }, + sessionId: "S", + context: { previousTurnId: first }, + input: user("again"), + config: { humanAvailable: true }, + }); + const created = (await persisted(repo, second))[0]; + expect(created.type === "turn_created" ? created.agent.resolved : null).toEqual({ + agentId: "copilot", + model: { provider: "anthropic", model: "claude-x" }, + inheritedFrom: first, + }); + const { outcome } = await advanceAndSettle(b.runtime, second); + expect(outcome?.status).toBe("completed"); + // The switched model was resolved; prompt/tools came through the chain. + expect(b.models.resolved[0]).toEqual({ provider: "anthropic", model: "claude-x" }); + expect(b.models.requests[0].systemPrompt).toBe("SYS"); + expect(b.models.requests[0].tools).toEqual(defaultAgent.tools); + }); + + it("a tools difference forces a full snapshot", async () => { + const repo = new InMemoryTurnRepo(); + const a = makeRuntime({ + repo, + models: [respond(completedResp(assistantText("first")))], + }); + const first = await newTurn(a.runtime, { sessionId: "S" }); + await advanceAndSettle(a.runtime, first); + + const fewerTools = { ...defaultAgent, tools: [echoDescriptor] }; + const b = makeRuntime({ repo, agent: fewerTools, idStart: 300 }); + const second = await b.runtime.createTurn({ + agent: { agentId: "copilot" }, + sessionId: "S", + context: { previousTurnId: first }, + input: user("again"), + config: { humanAvailable: true }, + }); + const created = (await persisted(repo, second))[0]; + expect( + created.type === "turn_created" && "tools" in created.agent.resolved + ? created.agent.resolved.tools + : null, + ).toEqual([echoDescriptor]); + }); + + it("inheritance chains across turns and materializes through multiple hops", async () => { + const { runtime, repo, models } = makeRuntime({ + models: [ + respond(completedResp(assistantText("one"))), + respond(completedResp(assistantText("two"))), + respond(completedResp(assistantText("three"))), + ], + }); + const t1 = await newTurn(runtime, { sessionId: "S" }); + await advanceAndSettle(runtime, t1); + const t2 = await runtime.createTurn({ + agent: { agentId: "copilot" }, + sessionId: "S", + context: { previousTurnId: t1 }, + input: user("two"), + config: { humanAvailable: true }, + }); + await advanceAndSettle(runtime, t2); + const t3 = await runtime.createTurn({ + agent: { agentId: "copilot" }, + sessionId: "S", + context: { previousTurnId: t2 }, + input: user("three"), + config: { humanAvailable: true }, + }); + const created3 = (await persisted(repo, t3))[0]; + // t3 inherits from t2, which itself inherits from t1 (the concrete base). + expect( + created3.type === "turn_created" && "inheritedFrom" in created3.agent.resolved + ? created3.agent.resolved.inheritedFrom + : null, + ).toBe(t2); + const { outcome } = await advanceAndSettle(runtime, t3); + expect(outcome?.status).toBe("completed"); + expect(models.requests[2].systemPrompt).toBe("SYS"); + expect(models.requests[2].tools).toEqual(defaultAgent.tools); + }); + + it("standalone (inline-context) turns always persist a full snapshot", async () => { + const { runtime, repo } = makeRuntime(); + const turnId = await newTurn(runtime); + const created = (await persisted(repo, turnId))[0]; + expect( + created.type === "turn_created" && "systemPrompt" in created.agent.resolved, + ).toBe(true); + }); + + it("falls back to a full snapshot when the predecessor is unreadable", async () => { + const { runtime, repo } = makeRuntime(); + const turnId = await runtime.createTurn({ + agent: { agentId: "copilot" }, + sessionId: "S", + context: { previousTurnId: "2026-07-02T09-00-00Z-0000404-000" }, + input: user("orphan ref"), + config: { humanAvailable: true }, + }); + const created = (await persisted(repo, turnId))[0]; + expect( + created.type === "turn_created" && "systemPrompt" in created.agent.resolved, + ).toBe(true); + }); + + it("persists a full snapshot when the resolved agent differs", async () => { + const repo = new InMemoryTurnRepo(); + const a = makeRuntime({ + repo, + models: [respond(completedResp(assistantText("first")))], + }); + const first = await newTurn(a.runtime, { sessionId: "S" }); + await advanceAndSettle(a.runtime, first); + + const changedAgent = { ...defaultAgent, systemPrompt: "SYS v2" }; + const b = makeRuntime({ repo, agent: changedAgent, idStart: 100 }); + const second = await b.runtime.createTurn({ + agent: { agentId: "copilot" }, + sessionId: "S", + context: { previousTurnId: first }, + input: user("again"), + config: { humanAvailable: true }, + }); + const created = (await persisted(repo, second))[0]; + expect( + created.type === "turn_created" && "systemPrompt" in created.agent.resolved + ? created.agent.resolved.systemPrompt + : null, + ).toBe("SYS v2"); + }); +}); + describe("historical and live reconstruction (26.8)", () => { it("getTurn is read-only and matches live durable events", async () => { const { runtime, repo } = makeRuntime({ diff --git a/apps/x/packages/core/src/turns/runtime.ts b/apps/x/packages/core/src/turns/runtime.ts index d82e575a..80f657b9 100644 --- a/apps/x/packages/core/src/turns/runtime.ts +++ b/apps/x/packages/core/src/turns/runtime.ts @@ -6,6 +6,8 @@ import { type JsonValue, type ModelCallFailed, type ModelRequest, + type ResolvedAgent, + type ResolvedAgentSnapshot, assistantRef, toolResultRef, type ToolCallState, @@ -109,13 +111,42 @@ export class TurnRuntime implements ITurnRuntime { async createTurn(input: CreateTurnInput): Promise { const resolved = await this.agentResolver.resolve(input.agent); const turnId = await this.idGenerator.next(); + // Inherit the heavy snapshot fields (system prompt + tools) when they + // are byte-identical to the context predecessor's materialized + // snapshot — the same reference mechanism as context and requests. + // The model stays concrete for the session index and so a model + // switch never blocks inheritance. + let snapshot: z.infer = resolved; + if (!Array.isArray(input.context)) { + try { + const previousEvents = await this.turnRepo.read( + input.context.previousTurnId, + ); + const previous = await this.contextResolver.resolveAgent( + reduceTurn(previousEvents).definition.agent.resolved, + ); + if ( + previous.systemPrompt === resolved.systemPrompt && + JSON.stringify(previous.tools) === JSON.stringify(resolved.tools) + ) { + snapshot = { + agentId: resolved.agentId, + model: resolved.model, + inheritedFrom: input.context.previousTurnId, + }; + } + } catch { + // Unreadable predecessor: persist the full snapshot; context + // resolution surfaces the real problem at advance time. + } + } const event = TurnCreated.parse({ type: "turn_created", schemaVersion: 1, turnId, ts: this.clock.now(), sessionId: input.sessionId ?? null, - agent: { requested: input.agent, resolved }, + agent: { requested: input.agent, resolved: snapshot }, context: input.context, input: input.input, config: { @@ -188,11 +219,12 @@ export class TurnRuntime implements ITurnRuntime { const resolvedContext = await this.contextResolver.resolve( definition.context, ); - const model = await this.modelRegistry.resolve( - definition.agent.resolved.model, + const resolvedAgent = await this.contextResolver.resolveAgent( + definition.agent.resolved, ); + const model = await this.modelRegistry.resolve(resolvedAgent.model); const toolsByName = new Map(); - for (const descriptor of definition.agent.resolved.tools) { + for (const descriptor of resolvedAgent.tools) { const tool = await this.toolRegistry.resolve(descriptor); if ( tool.descriptor.toolId !== descriptor.toolId || @@ -223,6 +255,7 @@ export class TurnRuntime implements ITurnRuntime { state, stream, resolvedContext, + resolvedAgent, model, toolsByName, signal: controller.signal, @@ -250,6 +283,7 @@ class TurnAdvance { private state: TurnState; private readonly stream: HotStream; private readonly resolvedContext: Array>; + private readonly resolvedAgent: z.infer; private readonly model: ResolvedModel; private readonly toolsByName: Map; private readonly signal: AbortSignal; @@ -270,6 +304,7 @@ class TurnAdvance { state: TurnState; stream: HotStream; resolvedContext: Array>; + resolvedAgent: z.infer; model: ResolvedModel; toolsByName: Map; signal: AbortSignal; @@ -283,6 +318,7 @@ class TurnAdvance { this.state = init.state; this.stream = init.stream; this.resolvedContext = init.resolvedContext; + this.resolvedAgent = init.resolvedAgent; this.model = init.model; this.toolsByName = init.toolsByName; this.signal = init.signal; @@ -893,6 +929,7 @@ class TurnAdvance { this.state, index, this.resolvedContext, + this.resolvedAgent, (messages) => this.model.encodeMessages(messages), ); diff --git a/apps/x/packages/shared/package.json b/apps/x/packages/shared/package.json index 5a1a9c45..31f99cdd 100644 --- a/apps/x/packages/shared/package.json +++ b/apps/x/packages/shared/package.json @@ -1,19 +1,19 @@ { - "name": "@x/shared", - "private": true, - "type": "module", - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "scripts": { - "build": "rm -rf dist && tsc -p tsconfig.build.json", - "dev": "tsc -w", - "test": "vitest run", - "test:watch": "vitest" - }, - "dependencies": { - "zod": "^4.2.1" - }, - "devDependencies": { - "vitest": "catalog:" - } -} \ No newline at end of file + "name": "@x/shared", + "private": true, + "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "scripts": { + "build": "rm -rf dist && tsc -p tsconfig.build.json", + "dev": "tsc -w", + "test": "vitest run", + "test:watch": "vitest" + }, + "dependencies": { + "zod": "^4.2.1" + }, + "devDependencies": { + "vitest": "catalog:" + } +} diff --git a/apps/x/packages/shared/src/turns.test.ts b/apps/x/packages/shared/src/turns.test.ts index 844d76b2..84a5a9c7 100644 --- a/apps/x/packages/shared/src/turns.test.ts +++ b/apps/x/packages/shared/src/turns.test.ts @@ -627,6 +627,67 @@ describe("context references", () => { }); }); +describe("inherited agent snapshots", () => { + const inherited = { + agentId: "copilot", + model: { provider: "openai", model: "gpt-test" }, + inheritedFrom: PREV_TURN_ID, + }; + + it("accepts inheritance referencing the context predecessor; identity arrives via invocation", () => { + const call0 = assistantCalls(toolCallPart("tc1", "echo")); + const state = reduceTurn([ + created({ + context: { previousTurnId: PREV_TURN_ID }, + agent: { requested: { agentId: "copilot" }, resolved: inherited }, + }), + requested(0, ["input"], { contextRef: { previousTurnId: PREV_TURN_ID } }), + completed(0, call0), + ]); + // No descriptor lookup without the concrete snapshot… + expect(state.toolCalls[0].toolId).toBeUndefined(); + expect(state.toolCalls[0].execution).toBeUndefined(); + + // …until tool_invocation_requested supplies identity. + const withInvocation = reduceTurn([ + created({ + context: { previousTurnId: PREV_TURN_ID }, + agent: { requested: { agentId: "copilot" }, resolved: inherited }, + }), + requested(0, ["input"], { contextRef: { previousTurnId: PREV_TURN_ID } }), + completed(0, call0), + invocation("tc1"), + result("tc1", "echo"), + ]); + expect(withInvocation.toolCalls[0].toolId).toBe("tool.echo"); + expect(withInvocation.toolCalls[0].execution).toBe("sync"); + }); + + it("rejects an inherited snapshot on an inline-context turn", () => { + expectCorruption( + [ + created({ + context: [], + agent: { requested: { agentId: "copilot" }, resolved: inherited }, + }), + ], + /inherited agent snapshot must reference the turn's context predecessor/, + ); + }); + + it("rejects an inherited snapshot pointing at a different turn than the context", () => { + expectCorruption( + [ + created({ + context: { previousTurnId: "some-other-turn" }, + agent: { requested: { agentId: "copilot" }, resolved: inherited }, + }), + ], + /inherited agent snapshot must reference the turn's context predecessor/, + ); + }); +}); + describe("suspension", () => { it("accepts a snapshot matching pending permissions and async tools", () => { const call0 = assistantCalls( diff --git a/apps/x/packages/shared/src/turns.ts b/apps/x/packages/shared/src/turns.ts index 6d28e035..df9a65f4 100644 --- a/apps/x/packages/shared/src/turns.ts +++ b/apps/x/packages/shared/src/turns.ts @@ -58,6 +58,30 @@ export const ResolvedAgent = z.object({ tools: z.array(ToolDescriptor), }); +// Session turns whose system prompt and tool set are byte-identical to the +// previous turn's materialized snapshot inherit it by reference instead of +// re-persisting ~tens of KB per turn (same mechanism as context references). +// The model stays concrete: it is tiny, the session index denormalizes it, +// and a mid-session model switch must not block inheritance. Written only +// when equality holds at creation; materialization walks inheritedFrom to +// the nearest concrete snapshot. +export const InheritedAgentSnapshot = z.object({ + agentId: z.string(), + model: ModelDescriptor, + inheritedFrom: z.string(), +}); + +export const ResolvedAgentSnapshot = z.union([ + ResolvedAgent, + InheritedAgentSnapshot, +]); + +export function isInheritedSnapshot( + resolved: z.infer, +): resolved is z.infer { + return "inheritedFrom" in resolved; +} + // --------------------------------------------------------------------------- // Context // --------------------------------------------------------------------------- @@ -108,7 +132,7 @@ export const TurnCreated = z.object({ sessionId: z.string().nullable(), agent: z.object({ requested: RequestedAgent, - resolved: ResolvedAgent, + resolved: ResolvedAgentSnapshot, }), context: TurnContext, input: UserMessage, @@ -614,9 +638,12 @@ function applyModelCallCompleted( if (state.toolCalls.some((tc) => tc.toolCallId === part.toolCallId)) { fail(`duplicate tool call id: ${part.toolCallId}`); } - const descriptor = state.definition.agent.resolved.tools.find( - (tool) => tool.name === part.toolName, - ); + const resolved = state.definition.agent.resolved; + // Inherited snapshots resolve outside the reducer; identity fields + // then arrive via tool_invocation_requested events. + const descriptor = isInheritedSnapshot(resolved) + ? undefined + : resolved.tools.find((tool) => tool.name === part.toolName); state.toolCalls.push({ modelCallIndex: event.modelCallIndex, order: order++, @@ -850,6 +877,18 @@ export function reduceTurn( fail(`unsupported turn schema version: ${String(first.schemaVersion)}`); } + if (isInheritedSnapshot(first.agent.resolved)) { + const context = first.context; + if ( + Array.isArray(context) || + context.previousTurnId !== first.agent.resolved.inheritedFrom + ) { + fail( + "inherited agent snapshot must reference the turn's context predecessor", + ); + } + } + const state: TurnState = { definition: first, modelCalls: [], From 0042dde3bbad405dddf52a5fd1c9bd6654ae8784 Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 13:17:43 +0530 Subject: [PATCH 18/24] test(x/renderer): string request refs in session-chat fixtures Co-Authored-By: Claude Fable 5 --- .../x/apps/renderer/src/lib/session-chat/test-fixtures.ts | 8 +------- .../apps/renderer/src/lib/session-chat/turn-view.test.ts | 4 ++-- 2 files changed, 3 insertions(+), 9 deletions(-) diff --git a/apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts b/apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts index d20c4bd0..3df22529 100644 --- a/apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts +++ b/apps/x/apps/renderer/src/lib/session-chat/test-fixtures.ts @@ -46,16 +46,10 @@ export function created( } } -type RequestRef = - | { kind: 'context' } - | { kind: 'input' } - | { kind: 'assistant'; modelCallIndex: number } - | { kind: 'toolResult'; toolCallId: string } - export function requested( turnId: string, index: number, - refs: RequestRef[] = [{ kind: 'input' }], + refs: string[] = ['input'], ): TEvent { return { type: 'model_call_requested', diff --git a/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts b/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts index 4c43c7c1..937ad9a2 100644 --- a/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts +++ b/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts @@ -92,8 +92,8 @@ describe('buildTurnConversation', () => { invocation(T1, 'tc1', 'echo'), toolResult(T1, 'tc1', 'echo', { echoed: true }), requested(T1, 1, [ - { kind: 'assistant', modelCallIndex: 0 }, - { kind: 'toolResult', toolCallId: 'tc1' }, + 'assistant:0', + 'toolResult:tc1', ]), completed(T1, 1, assistantText('all done')), turnCompleted(T1, 'all done'), From c73531ec3891d4379bd00e75a67bc38aead8c582 Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 13:37:39 +0530 Subject: [PATCH 19/24] =?UTF-8?q?feat(x):=20stage=206=20=E2=80=94=20migrat?= =?UTF-8?q?e=20headless=20callers=20to=20the=20turn=20runtime?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New src/agents/headless.ts wraps the turn runtime in the old headless calling convention: startHeadlessAgent returns the turn id immediately (callers record it in pointer files / bus events before completion) and a done promise settling with { outcome, state, summary }; runHeadlessAgent awaits it. throwOnError reproduces the old waitForRunCompletion({ throwOnError }) semantics via HeadlessRunError; summary reproduces extractAgentResponse (last assistant text); toolInputPaths replaces the run-bus tool-invocation subscriptions by reading invoked calls from durable turn state. Model overrides pair the caller's model id with the app-default provider. Unit-tested against an injected fake runtime (7 tests). Migrated all nine callers: - background-tasks/runner: handle start wrapped in withUseCase so tools (notify-user) read the use case via AsyncLocalStorage - knowledge/live-note/runner: same shape, gains withUseCase - pre_built/runner, knowledge/agent_notes: run-and-wait - knowledge/tag_notes, label_emails, build_graph: edited/created paths now come from turn state (toolInputPaths) instead of bus streaming - knowledge/inline_tasks (both sites): summary text feeds the existing marker parsing unchanged - agent-schedule/runner: fire-and-forget start with AbortSignal.timeout(TIMEOUT_MS); dropped the now-unused runsRepo/agentRuntime/idGenerator plumbing Code-mode sessions remain on the runs infrastructure (deliberate carve-out until stage 7 scoping); agents/utils.ts stays for launch-code-task's extractAgentResponse. The notify-user useCase gate already prefers ALS with a best-effort fetchRun fallback, so it works unchanged for turn ids. Co-Authored-By: Claude Fable 5 --- .../core/src/agent-schedule/runner.ts | 58 ++-- .../packages/core/src/agents/headless.test.ts | 260 ++++++++++++++++++ apps/x/packages/core/src/agents/headless.ts | 159 +++++++++++ .../core/src/background-tasks/runner.ts | 49 ++-- .../core/src/knowledge/agent_notes.ts | 12 +- .../core/src/knowledge/build_graph.ts | 58 +--- .../core/src/knowledge/inline_tasks.ts | 35 +-- .../core/src/knowledge/label_emails.ts | 45 +-- .../core/src/knowledge/live-note/runner.ts | 42 +-- .../packages/core/src/knowledge/tag_notes.ts | 45 +-- apps/x/packages/core/src/pre_built/runner.ts | 24 +- 11 files changed, 538 insertions(+), 249 deletions(-) create mode 100644 apps/x/packages/core/src/agents/headless.test.ts create mode 100644 apps/x/packages/core/src/agents/headless.ts diff --git a/apps/x/packages/core/src/agent-schedule/runner.ts b/apps/x/packages/core/src/agent-schedule/runner.ts index 44dac07d..e34c9e11 100644 --- a/apps/x/packages/core/src/agent-schedule/runner.ts +++ b/apps/x/packages/core/src/agent-schedule/runner.ts @@ -2,13 +2,10 @@ import { CronExpressionParser } from "cron-parser"; import container from "../di/container.js"; import { IAgentScheduleRepo } from "./repo.js"; import { IAgentScheduleStateRepo } from "./state-repo.js"; -import { IRunsRepo } from "../runs/repo.js"; -import { IAgentRuntime } from "../agents/runtime.js"; -import { IMonotonicallyIncreasingIdGenerator } from "../application/lib/id-gen.js"; import { AgentScheduleConfig, AgentScheduleEntry } from "@x/shared/dist/agent-schedule.js"; import { AgentScheduleState, AgentScheduleStateEntry } from "@x/shared/dist/agent-schedule-state.js"; -import { MessageEvent } from "@x/shared/dist/runs.js"; -import { createRun } from "../runs/runs.js"; +import { startHeadlessAgent } from "../agents/headless.js"; +import { withUseCase } from "../analytics/use_case.js"; import z from "zod"; const DEFAULT_STARTING_MESSAGE = "go"; @@ -147,10 +144,7 @@ function shouldRunNow( async function runAgent( agentName: string, entry: z.infer, - stateRepo: IAgentScheduleStateRepo, - runsRepo: IRunsRepo, - agentRuntime: IAgentRuntime, - idGenerator: IMonotonicallyIncreasingIdGenerator + stateRepo: IAgentScheduleStateRepo ): Promise { console.log(`[AgentRunner] Starting agent: ${agentName}`); @@ -163,31 +157,24 @@ async function runAgent( }); try { - // Create a new run via core (resolves agent + default model+provider). - const run = await createRun({ - agentId: agentName, - useCase: 'copilot_chat', - subUseCase: 'scheduled', - }); - console.log(`[AgentRunner] Created run ${run.id} for agent ${agentName}`); - - // Add the starting message as a user message + // Start a standalone turn (resolves agent + default model/provider). + // Fire-and-forget like the old trigger(): the schedule state machine + // below runs on wall-clock; completion is observed only for logging. + // The signal caps a hung turn at the same TIMEOUT_MS the state-based + // watchdog uses. const startingMessage = entry.startingMessage ?? DEFAULT_STARTING_MESSAGE; - const messageEvent: z.infer = { - runId: run.id, - type: "message", - messageId: await idGenerator.next(), - message: { - role: "user", - content: startingMessage, - }, - subflow: [], - }; - await runsRepo.appendEvents(run.id, [messageEvent]); - console.log(`[AgentRunner] Sent starting message to agent ${agentName}: "${startingMessage}"`); - - // Trigger the run - await agentRuntime.trigger(run.id); + const handle = await withUseCase( + { useCase: 'copilot_chat', subUseCase: 'scheduled' }, + () => startHeadlessAgent({ + agentId: agentName, + message: startingMessage, + signal: AbortSignal.timeout(TIMEOUT_MS), + }), + ); + console.log(`[AgentRunner] Started turn ${handle.turnId} for agent ${agentName}: "${startingMessage}"`); + void handle.done + .then((result) => console.log(`[AgentRunner] Turn ${handle.turnId} (${agentName}) settled: ${result.outcome.status}`)) + .catch((error) => console.error(`[AgentRunner] Turn ${handle.turnId} (${agentName}) errored:`, error instanceof Error ? error.message : error)); // Calculate next run time const nextRunAt = calculateNextRunAt(entry.schedule); @@ -264,9 +251,6 @@ async function checkForTimeouts( async function pollAndRun(): Promise { const scheduleRepo = container.resolve("agentScheduleRepo"); const stateRepo = container.resolve("agentScheduleStateRepo"); - const runsRepo = container.resolve("runsRepo"); - const agentRuntime = container.resolve("agentRuntime"); - const idGenerator = container.resolve("idGenerator"); // Load config and state let config: z.infer; @@ -314,7 +298,7 @@ async function pollAndRun(): Promise { if (shouldRunNow(entry, agentState)) { // Run agent (don't await - let it run in background) - runAgent(agentName, entry, stateRepo, runsRepo, agentRuntime, idGenerator).catch((error) => { + runAgent(agentName, entry, stateRepo).catch((error) => { console.error(`[AgentRunner] Unhandled error in runAgent for ${agentName}:`, error); }); } diff --git a/apps/x/packages/core/src/agents/headless.test.ts b/apps/x/packages/core/src/agents/headless.test.ts new file mode 100644 index 00000000..202bfbce --- /dev/null +++ b/apps/x/packages/core/src/agents/headless.test.ts @@ -0,0 +1,260 @@ +import { describe, expect, it } from "vitest"; +import type { z } from "zod"; +import { reduceTurn, type TurnEvent } from "@x/shared/dist/turns.js"; +import type { + CreateTurnInput, + ITurnRuntime, + TurnExecution, + TurnOutcome, +} from "../turns/api.js"; +import { + HeadlessRunError, + assistantText, + lastAssistantText, + startHeadlessAgent, + toolInputPaths, +} from "./headless.js"; + +type TEvent = z.infer; + +const TURN_ID = "2026-07-02T10-00-00Z-0000001-000"; +const TS = "2026-07-02T10:00:00Z"; + +const echoTool = { + toolId: "builtin:file-editText", + name: "file-editText", + description: "Edit", + inputSchema: {}, + execution: "sync" as const, + requiresHuman: false, +}; + +function turnLog(opts: { + responseText?: string; + toolCalls?: Array<{ id: string; name: string; input: unknown; invoked: boolean }>; + failed?: string; +}): TEvent[] { + const events: TEvent[] = [ + { + type: "turn_created", + schemaVersion: 1, + turnId: TURN_ID, + ts: TS, + sessionId: null, + agent: { + requested: { agentId: "worker" }, + resolved: { + agentId: "worker", + systemPrompt: "SYS", + model: { provider: "fake", model: "m" }, + tools: [echoTool], + }, + }, + context: [], + input: { role: "user", content: "go" }, + config: { autoPermission: true, humanAvailable: false, maxModelCalls: 20 }, + }, + { + type: "model_call_requested", + turnId: TURN_ID, + ts: TS, + modelCallIndex: 0, + request: { messages: ["input"], parameters: {} }, + }, + ]; + const toolCalls = opts.toolCalls ?? []; + if (toolCalls.length > 0) { + events.push({ + type: "model_call_completed", + turnId: TURN_ID, + ts: TS, + modelCallIndex: 0, + message: { + role: "assistant", + content: toolCalls.map((tc) => ({ + type: "tool-call" as const, + toolCallId: tc.id, + toolName: tc.name, + arguments: tc.input as never, + })), + }, + finishReason: "tool-calls", + usage: {}, + }); + for (const tc of toolCalls) { + if (!tc.invoked) continue; + events.push({ + type: "tool_invocation_requested", + turnId: TURN_ID, + ts: TS, + toolCallId: tc.id, + toolId: "builtin:" + tc.name, + toolName: tc.name, + execution: "sync", + input: tc.input as never, + }); + events.push({ + type: "tool_result", + turnId: TURN_ID, + ts: TS, + toolCallId: tc.id, + toolName: tc.name, + source: "sync", + result: { output: "ok", isError: false }, + }); + } + } + if (opts.failed) { + events.push({ + type: "model_call_failed", + turnId: TURN_ID, + ts: TS, + modelCallIndex: 0, + error: opts.failed, + }); + events.push({ type: "turn_failed", turnId: TURN_ID, ts: TS, error: opts.failed, usage: {} }); + } else if (opts.responseText !== undefined && toolCalls.length === 0) { + events.push({ + type: "model_call_completed", + turnId: TURN_ID, + ts: TS, + modelCallIndex: 0, + message: { role: "assistant", content: opts.responseText }, + finishReason: "stop", + usage: {}, + }); + events.push({ + type: "turn_completed", + turnId: TURN_ID, + ts: TS, + output: { role: "assistant", content: opts.responseText }, + finishReason: "stop", + usage: {}, + }); + } + return events; +} + +class FakeRuntime implements ITurnRuntime { + createInputs: CreateTurnInput[] = []; + constructor( + private readonly log: TEvent[], + private readonly outcome: TurnOutcome, + ) {} + + async createTurn(input: CreateTurnInput): Promise { + this.createInputs.push(input); + return TURN_ID; + } + + advanceTurn(): TurnExecution { + return { + events: (async function* () {})(), + outcome: Promise.resolve(this.outcome), + }; + } + + async getTurn() { + return { turnId: TURN_ID, events: this.log }; + } +} + +const completedOutcome: TurnOutcome = { + status: "completed", + output: { role: "assistant", content: "done" }, + finishReason: "stop", + usage: {}, +}; + +describe("headless agent helpers", () => { + it("assistantText handles string and parts content", () => { + expect(assistantText({ role: "assistant", content: "hi" })).toBe("hi"); + expect( + assistantText({ + role: "assistant", + content: [ + { type: "reasoning", text: "hmm" }, + { type: "text", text: "a" }, + { type: "text", text: "b" }, + ], + }), + ).toBe("ab"); + expect(assistantText({ role: "assistant", content: "" })).toBeNull(); + }); + + it("lastAssistantText returns the final assistant text in the transcript", () => { + const state = reduceTurn(turnLog({ responseText: "the summary" })); + expect(lastAssistantText(state)).toBe("the summary"); + expect(lastAssistantText(reduceTurn(turnLog({ failed: "boom" })))).toBeNull(); + }); + + it("toolInputPaths collects paths of invoked calls only", () => { + const state = reduceTurn( + turnLog({ + toolCalls: [ + { id: "a", name: "file-editText", input: { path: "notes/x.md" }, invoked: true }, + { id: "b", name: "file-editText", input: { path: "notes/y.md" }, invoked: true }, + { id: "c", name: "file-writeText", input: { path: "notes/z.md" }, invoked: true }, + { id: "d", name: "file-editText", input: { path: "never-ran.md" }, invoked: false }, + ], + }), + ); + expect(toolInputPaths(state, ["file-editText"])).toEqual( + new Set(["notes/x.md", "notes/y.md"]), + ); + expect(toolInputPaths(state, ["file-writeText"])).toEqual(new Set(["notes/z.md"])); + }); +}); + +describe("startHeadlessAgent", () => { + it("creates a standalone auto-permission turn and returns the id before settling", async () => { + const runtime = new FakeRuntime(turnLog({ responseText: "the summary" }), completedOutcome); + const handle = await startHeadlessAgent( + { agentId: "worker", message: "go", model: "m", provider: "fake" }, + runtime, + ); + expect(handle.turnId).toBe(TURN_ID); + expect(runtime.createInputs[0]).toMatchObject({ + agent: { agentId: "worker", overrides: { model: { provider: "fake", model: "m" } } }, + sessionId: null, + context: [], + input: { role: "user", content: "go" }, + config: { autoPermission: true, humanAvailable: false }, + }); + const result = await handle.done; + expect(result.outcome.status).toBe("completed"); + expect(result.summary).toBe("the summary"); + }); + + it("omits the model override when neither model nor provider is set", async () => { + const runtime = new FakeRuntime(turnLog({ responseText: "ok" }), completedOutcome); + await startHeadlessAgent({ agentId: "worker", message: "go" }, runtime); + expect(runtime.createInputs[0].agent.overrides).toBeUndefined(); + }); + + it("throwOnError rejects done with HeadlessRunError on failed outcomes", async () => { + const runtime = new FakeRuntime(turnLog({ failed: "provider exploded" }), { + status: "failed", + error: "provider exploded", + usage: {}, + }); + const handle = await startHeadlessAgent( + { agentId: "worker", message: "go", throwOnError: true }, + runtime, + ); + await expect(handle.done).rejects.toThrowError(HeadlessRunError); + await expect(handle.done).rejects.toThrowError("provider exploded"); + }); + + it("without throwOnError a failed outcome resolves (old wait semantics)", async () => { + const runtime = new FakeRuntime(turnLog({ failed: "boom" }), { + status: "failed", + error: "boom", + usage: {}, + }); + const handle = await startHeadlessAgent({ agentId: "worker", message: "go" }, runtime); + const result = await handle.done; + expect(result.outcome.status).toBe("failed"); + expect(result.summary).toBeNull(); + }); +}); diff --git a/apps/x/packages/core/src/agents/headless.ts b/apps/x/packages/core/src/agents/headless.ts new file mode 100644 index 00000000..54f20826 --- /dev/null +++ b/apps/x/packages/core/src/agents/headless.ts @@ -0,0 +1,159 @@ +import type { z } from "zod"; +import type { AssistantMessage } from "@x/shared/dist/message.js"; +import { + reduceTurn, + type TurnState, +} from "@x/shared/dist/turns.js"; +import container from "../di/container.js"; +import { getDefaultModelAndProvider } from "../models/defaults.js"; +import type { ITurnRuntime, TurnOutcome } from "../turns/api.js"; + +// Drop-in replacement for the old headless runs pattern +// (createRun → createMessage → waitForRunCompletion → extractAgentResponse): +// one standalone turn per invocation (sessionId null, automatic permissions, +// no human). startHeadlessAgent returns the turn id immediately (callers +// record it in pointer files / bus events before completion); `done` settles +// with the outcome, the reduced turn state, and the final assistant text. + +export class HeadlessRunError extends Error { + constructor( + message: string, + readonly turnId: string, + readonly outcome: TurnOutcome, + ) { + super(message); + this.name = "HeadlessRunError"; + } +} + +export interface HeadlessAgentOptions { + agentId: string; + message: string; + // Model id; when set without provider, the app-default provider applies. + model?: string; + provider?: string; + maxModelCalls?: number; + signal?: AbortSignal; + // Old waitForRunCompletion({ throwOnError: true }) semantics: `done` + // rejects with HeadlessRunError unless the turn completes. + throwOnError?: boolean; +} + +export interface HeadlessAgentResult { + outcome: TurnOutcome; + state: TurnState; + // Last assistant text in the transcript (old extractAgentResponse). + summary: string | null; +} + +export interface HeadlessAgentHandle { + turnId: string; + done: Promise; +} + +export function assistantText( + message: z.infer, +): string | null { + const content = message.content; + if (typeof content === "string") { + return content || null; + } + const text = content + .map((part) => (part.type === "text" ? part.text : "")) + .join(""); + return text || null; +} + +export function lastAssistantText(state: TurnState): string | null { + for (let i = state.modelCalls.length - 1; i >= 0; i--) { + const response = state.modelCalls[i].response; + if (response) { + const text = assistantText(response); + if (text) { + return text; + } + } + } + return null; +} + +// Paths passed to the given file tools across the turn — replaces the old +// pattern of subscribing to the run bus for tool-invocation events. Only +// actually-invoked calls count (denied/unknown calls never ran). +export function toolInputPaths( + state: TurnState, + toolNames: string[], +): Set { + const paths = new Set(); + for (const toolCall of state.toolCalls) { + if (!toolNames.includes(toolCall.toolName) || !toolCall.invocation) { + continue; + } + const input = toolCall.input as { path?: unknown } | null | undefined; + if (input && typeof input === "object" && typeof input.path === "string") { + paths.add(input.path); + } + } + return paths; +} + +export async function startHeadlessAgent( + options: HeadlessAgentOptions, + // Injectable for tests; defaults to the app container's runtime. + turnRuntime: ITurnRuntime = container.resolve("turnRuntime"), +): Promise { + let modelOverride: { provider: string; model: string } | undefined; + if (options.model || options.provider) { + const defaults = await getDefaultModelAndProvider(); + modelOverride = { + provider: options.provider ?? defaults.provider, + model: options.model ?? defaults.model, + }; + } + + const turnId = await turnRuntime.createTurn({ + agent: { + agentId: options.agentId, + ...(modelOverride ? { overrides: { model: modelOverride } } : {}), + }, + sessionId: null, + context: [], + input: { role: "user", content: options.message }, + config: { + autoPermission: true, + humanAvailable: false, + ...(options.maxModelCalls === undefined + ? {} + : { maxModelCalls: options.maxModelCalls }), + }, + }); + + const execution = turnRuntime.advanceTurn(turnId, undefined, { + signal: options.signal, + }); + const done = execution.outcome.then(async (outcome) => { + const state = reduceTurn((await turnRuntime.getTurn(turnId)).events); + if (options.throwOnError && outcome.status !== "completed") { + throw new HeadlessRunError( + outcome.status === "failed" + ? outcome.error + : `turn ${outcome.status}`, + turnId, + outcome, + ); + } + return { outcome, state, summary: lastAssistantText(state) }; + }); + // The handle may be used fire-and-forget; rejections surface when awaited. + done.catch(() => undefined); + return { turnId, done }; +} + +export async function runHeadlessAgent( + options: HeadlessAgentOptions, + turnRuntime?: ITurnRuntime, +): Promise { + const handle = await startHeadlessAgent(options, turnRuntime); + const result = await handle.done; + return { turnId: handle.turnId, ...result }; +} diff --git a/apps/x/packages/core/src/background-tasks/runner.ts b/apps/x/packages/core/src/background-tasks/runner.ts index dee147d8..98f56d90 100644 --- a/apps/x/packages/core/src/background-tasks/runner.ts +++ b/apps/x/packages/core/src/background-tasks/runner.ts @@ -1,9 +1,8 @@ import type { BackgroundTask, BackgroundTaskTriggerType } from '@x/shared/dist/background-task.js'; import { PrefixLogger } from '@x/shared/dist/prefix-logger.js'; import { fetchTask, patchTask, prependRunId } from './fileops.js'; -import { createRun, createMessage } from '../runs/runs.js'; import { getBackgroundTaskAgentModel } from '../models/defaults.js'; -import { extractAgentResponse, waitForRunCompletion } from '../agents/utils.js'; +import { startHeadlessAgent } from '../agents/headless.js'; import { buildTriggerBlock } from '../agents/build-trigger-block.js'; import { backgroundTaskBus } from './bus.js'; import { withUseCase } from '../analytics/use_case.js'; @@ -139,20 +138,25 @@ export async function runBackgroundTask( } const model = task.model || await getBackgroundTaskAgentModel(); - const agentRun = await createRun({ - agentId: 'background-task-agent', - model, - ...(task.provider ? { provider: task.provider } : {}), - useCase: 'background_task_agent', - // Granular trigger as analytics sub-use-case — matches live-note's - // pattern at runner.ts:149. - subUseCase: trigger, - }); + // Establish the use-case context for the whole turn so every tool the + // agent calls (notably notify-user) reads `background_task_agent` via + // getCurrentUseCase(); the AsyncLocalStorage context set here flows + // through the turn's async execution chain. + const handle = await withUseCase( + { useCase: 'background_task_agent', subUseCase: trigger }, + () => startHeadlessAgent({ + agentId: 'background-task-agent', + message: buildMessage(slug, task, trigger, context, codeProject), + model, + ...(task.provider ? { provider: task.provider } : {}), + throwOnError: true, + }), + ); - const runId = agentRun.id; - // Record this run in the task's runs.log pointer file (newest first). - // The transcript itself lives at the global $WorkDir/runs/.jsonl - // — runs.log is just an index that ties runIds to this task. + const runId = handle.turnId; + // Record this turn in the task's runs.log pointer file (newest first). + // The transcript itself lives at $WorkDir/storage/turns/YYYY/MM/DD/ + // — runs.log is just an index that ties turn ids to this task. await prependRunId(slug, runId); const startedAt = new Date().toISOString(); @@ -184,20 +188,7 @@ export async function runBackgroundTask( }); try { - // Establish the use-case context for the whole run so every tool the - // agent calls (notably notify-user) reads `background_task_agent` via - // getCurrentUseCase(). createMessage synchronously fires - // agentRuntime.trigger(), so the detached run loop — and the tool - // calls within it — inherit this AsyncLocalStorage context. (The - // runtime's own enterUseCase runs inside an async generator and - // doesn't reliably propagate to tool execution, so we set it here at - // the trigger point instead.) - await withUseCase( - { useCase: 'background_task_agent', subUseCase: trigger }, - () => createMessage(runId, buildMessage(slug, task, trigger, context, codeProject)), - ); - await waitForRunCompletion(runId, { throwOnError: true }); - const summary = await extractAgentResponse(runId); + const { summary } = await handle.done; // Success — bump cycle anchor, refresh summary, clear any prior error. await patchTask(slug, { diff --git a/apps/x/packages/core/src/knowledge/agent_notes.ts b/apps/x/packages/core/src/knowledge/agent_notes.ts index f1380c4a..8d0b0ab1 100644 --- a/apps/x/packages/core/src/knowledge/agent_notes.ts +++ b/apps/x/packages/core/src/knowledge/agent_notes.ts @@ -2,9 +2,9 @@ import fs from 'fs'; import path from 'path'; import { google } from 'googleapis'; import { WorkDir } from '../config/config.js'; -import { createRun, createMessage } from '../runs/runs.js'; +import { runHeadlessAgent } from '../agents/headless.js'; import { getKgModel } from '../models/defaults.js'; -import { getErrorDetails, waitForRunCompletion } from '../agents/utils.js'; +import { getErrorDetails } from '../agents/utils.js'; import { serviceLogger } from '../services/service_logger.js'; import { loadUserConfig, updateUserEmail } from '../config/user_config.js'; import { GoogleClientFactory } from './google-client-factory.js'; @@ -281,14 +281,12 @@ async function processAgentNotes(): Promise { const timestamp = new Date().toISOString(); const message = `Current timestamp: ${timestamp}\n\nProcess the following source material and update the Agent Notes folder accordingly.\n\n${messageParts.join('\n\n')}`; - const agentRun = await createRun({ + await runHeadlessAgent({ agentId: AGENT_ID, + message, model: await getKgModel(), - useCase: 'knowledge_sync', - subUseCase: 'agent_notes', + throwOnError: true, }); - await createMessage(agentRun.id, message); - await waitForRunCompletion(agentRun.id, { throwOnError: true }); // Mark everything as processed for (const p of emailPaths) { diff --git a/apps/x/packages/core/src/knowledge/build_graph.ts b/apps/x/packages/core/src/knowledge/build_graph.ts index ccfeea49..f0c8836a 100644 --- a/apps/x/packages/core/src/knowledge/build_graph.ts +++ b/apps/x/packages/core/src/knowledge/build_graph.ts @@ -2,9 +2,8 @@ import fs from 'fs'; import path from 'path'; import { WorkDir } from '../config/config.js'; import { getKgModel } from '../models/defaults.js'; -import { createRun, createMessage } from '../runs/runs.js'; -import { bus } from '../runs/bus.js'; -import { getErrorDetails, waitForRunCompletion } from '../agents/utils.js'; +import { runHeadlessAgent, toolInputPaths } from '../agents/headless.js'; +import { getErrorDetails } from '../agents/utils.js'; import { serviceLogger, type ServiceRunContext } from '../services/service_logger.js'; import { loadState, @@ -89,14 +88,6 @@ function hasNoiseLabels(content: string): boolean { return false; } -function extractPathFromToolInput(input: string): string | null { - try { - const parsed = JSON.parse(input) as { path?: string }; - return typeof parsed.path === 'string' ? parsed.path : null; - } catch { - return null; - } -} function ensureSuggestedTopicsFileLocation(): string { if (fs.existsSync(SUGGESTED_TOPICS_PATH)) { @@ -252,13 +243,6 @@ async function createNotesFromBatch( fs.mkdirSync(NOTES_OUTPUT_DIR, { recursive: true }); } - // Create a run for the note creation agent - const run = await createRun({ - agentId: NOTE_CREATION_AGENT, - model: await getKgModel(), - useCase: 'knowledge_sync', - subUseCase: 'build_graph', - }); const suggestedTopicsContent = readSuggestedTopicsFile(); // Build message with index and all files in the batch @@ -293,37 +277,19 @@ async function createNotesFromBatch( message += `\n\n---\n\n`; }); - const notesCreated = new Set(); - const notesModified = new Set(); - - const unsubscribe = await bus.subscribe(run.id, async (event) => { - if (event.type !== "tool-invocation") { - return; - } - if (event.toolName !== "file-writeText" && event.toolName !== "file-editText") { - return; - } - const toolPath = extractPathFromToolInput(event.input); - if (!toolPath) { - return; - } - if (event.toolName === "file-writeText") { - notesCreated.add(toolPath); - } else if (event.toolName === "file-editText") { - notesModified.add(toolPath); - } + const { turnId, state } = await runHeadlessAgent({ + agentId: NOTE_CREATION_AGENT, + message, + model: await getKgModel(), + throwOnError: true, }); - await createMessage(run.id, message); + // Created/modified paths come from the durable turn state instead of + // streaming bus subscriptions. + const notesCreated = toolInputPaths(state, ["file-writeText"]); + const notesModified = toolInputPaths(state, ["file-editText"]); - // Wait for the run to complete - try { - await waitForRunCompletion(run.id, { throwOnError: true }); - } finally { - unsubscribe(); - } - - return { runId: run.id, notesCreated, notesModified }; + return { runId: turnId, notesCreated, notesModified }; } /** diff --git a/apps/x/packages/core/src/knowledge/inline_tasks.ts b/apps/x/packages/core/src/knowledge/inline_tasks.ts index cd2f23c0..7bf92364 100644 --- a/apps/x/packages/core/src/knowledge/inline_tasks.ts +++ b/apps/x/packages/core/src/knowledge/inline_tasks.ts @@ -3,13 +3,12 @@ import path from 'path'; import { CronExpressionParser } from 'cron-parser'; import { generateText } from 'ai'; import { WorkDir } from '../config/config.js'; -import { createRun, createMessage, fetchRun } from '../runs/runs.js'; +import { runHeadlessAgent } from '../agents/headless.js'; import { getKgModel } from '../models/defaults.js'; import container from '../di/container.js'; import type { IModelConfigRepo } from '../models/repo.js'; import { createProvider } from '../models/models.js'; import { inlineTask } from '@x/shared'; -import { extractAgentResponse, waitForRunCompletion } from '../agents/utils.js'; import { captureLlmUsage } from '../analytics/usage.js'; import { withUseCase } from '../analytics/use_case.js'; @@ -470,13 +469,6 @@ async function processInlineTasks(): Promise { console.log(`[InlineTasks] Running task: "${task.instruction.slice(0, 80)}..."`); try { - const run = await createRun({ - agentId: INLINE_TASK_AGENT, - model: await getKgModel(), - useCase: 'knowledge_sync', - subUseCase: 'inline_task_run', - }); - const message = [ `Execute the following instruction from the note "${relativePath}":`, '', @@ -488,10 +480,11 @@ async function processInlineTasks(): Promise { '```', ].join('\n'); - await createMessage(run.id, message); - await waitForRunCompletion(run.id); - - const result = await extractAgentResponse(run.id); + const { summary: result } = await runHeadlessAgent({ + agentId: INLINE_TASK_AGENT, + message, + model: await getKgModel(), + }); if (result) { if (task.targetId) { // Recurring task with target region — replace content inside the region @@ -555,13 +548,6 @@ export async function processRowboatInstruction( scheduleLabel: string | null; response: string | null; }> { - const run = await createRun({ - agentId: INLINE_TASK_AGENT, - model: await getKgModel(), - useCase: 'knowledge_sync', - subUseCase: 'inline_task_run', - }); - const message = [ `Process the following @rowboat instruction from the note "${notePath}":`, '', @@ -573,10 +559,11 @@ export async function processRowboatInstruction( '```', ].join('\n'); - await createMessage(run.id, message); - await waitForRunCompletion(run.id); - - const rawResponse = await extractAgentResponse(run.id); + const { summary: rawResponse } = await runHeadlessAgent({ + agentId: INLINE_TASK_AGENT, + message, + model: await getKgModel(), + }); if (!rawResponse) { return { instruction, schedule: null, scheduleLabel: null, response: null }; } diff --git a/apps/x/packages/core/src/knowledge/label_emails.ts b/apps/x/packages/core/src/knowledge/label_emails.ts index ebb940c6..e91732cf 100644 --- a/apps/x/packages/core/src/knowledge/label_emails.ts +++ b/apps/x/packages/core/src/knowledge/label_emails.ts @@ -1,10 +1,9 @@ import fs from 'fs'; import path from 'path'; import { WorkDir } from '../config/config.js'; -import { createRun, createMessage } from '../runs/runs.js'; +import { runHeadlessAgent, toolInputPaths } from '../agents/headless.js'; import { getKgModel } from '../models/defaults.js'; -import { bus } from '../runs/bus.js'; -import { getErrorDetails, waitForRunCompletion } from '../agents/utils.js'; +import { getErrorDetails } from '../agents/utils.js'; import { serviceLogger } from '../services/service_logger.js'; import { limitEventItems } from './limit_event_items.js'; import { @@ -70,13 +69,6 @@ function getUnlabeledEmails(state: LabelingState): string[] { async function labelEmailBatch( files: { path: string; content: string }[] ): Promise<{ runId: string; filesEdited: Set }> { - const run = await createRun({ - agentId: LABELING_AGENT, - model: await getKgModel(), - useCase: 'knowledge_sync', - subUseCase: 'label_emails', - }); - let message = `Label the following ${files.length} email files by prepending YAML frontmatter.\n\n`; message += `**Important:** Use workspace-relative paths with file-editText (e.g. "gmail_sync/email.md", NOT absolute paths).\n\n`; @@ -92,33 +84,16 @@ async function labelEmailBatch( message += `\n\n---\n\n`; } - const filesEdited = new Set(); - - const unsubscribe = await bus.subscribe(run.id, async (event) => { - if (event.type !== 'tool-invocation') { - return; - } - if (event.toolName !== 'file-editText') { - return; - } - try { - const parsed = JSON.parse(event.input) as { path?: string }; - if (typeof parsed.path === 'string') { - filesEdited.add(parsed.path); - } - } catch { - // ignore parse errors - } + const { turnId, state } = await runHeadlessAgent({ + agentId: LABELING_AGENT, + message, + model: await getKgModel(), + throwOnError: true, }); - await createMessage(run.id, message); - try { - await waitForRunCompletion(run.id, { throwOnError: true }); - } finally { - unsubscribe(); - } - - return { runId: run.id, filesEdited }; + // Edited paths come from the durable turn state instead of streaming + // bus subscriptions. + return { runId: turnId, filesEdited: toolInputPaths(state, ['file-editText']) }; } /** diff --git a/apps/x/packages/core/src/knowledge/live-note/runner.ts b/apps/x/packages/core/src/knowledge/live-note/runner.ts index b263c3e5..64b1f2b8 100644 --- a/apps/x/packages/core/src/knowledge/live-note/runner.ts +++ b/apps/x/packages/core/src/knowledge/live-note/runner.ts @@ -1,8 +1,8 @@ import type { LiveNote, LiveNoteTriggerType } from '@x/shared/dist/live-note.js'; import { fetchLiveNote, patchLiveNote, readNoteBody } from './fileops.js'; -import { createRun, createMessage } from '../../runs/runs.js'; import { getLiveNoteAgentModel } from '../../models/defaults.js'; -import { extractAgentResponse, waitForRunCompletion } from '../../agents/utils.js'; +import { startHeadlessAgent } from '../../agents/headless.js'; +import { withUseCase } from '../../analytics/use_case.js'; import { buildTriggerBlock } from '../../agents/build-trigger-block.js'; import { liveNoteBus } from './bus.js'; import { PrefixLogger } from '@x/shared/dist/prefix-logger.js'; @@ -109,17 +109,20 @@ export async function runLiveNoteAgent( const bodyBefore = await readNoteBody(filePath); const model = live.model ?? await getLiveNoteAgentModel(); - const agentRun = await createRun({ - agentId: 'live-note-agent', - model, - ...(live.provider ? { provider: live.provider } : {}), - useCase: 'live_note_agent', - // Use the granular trigger as the analytics sub-use-case so - // dashboards can break down agent runs by what woke them up - // (manual / cron / window / event). Pass 1 routing emits the - // separate `routing` sub-use-case from routing.ts. - subUseCase: trigger, - }); + // The use-case context propagates to every tool the agent calls; the + // granular trigger doubles as the sub-use-case (manual / cron / + // window / event) so dashboards can break down what woke the agent. + const handle = await withUseCase( + { useCase: 'live_note_agent', subUseCase: trigger }, + () => startHeadlessAgent({ + agentId: 'live-note-agent', + message: buildMessage(filePath, live, trigger, context), + model, + ...(live.provider ? { provider: live.provider } : {}), + throwOnError: true, + }), + ); + const agentRun = { id: handle.turnId }; log.log(`${filePath} — start trigger=${trigger} runId=${agentRun.id}`); @@ -141,14 +144,11 @@ export async function runLiveNoteAgent( }); try { - await createMessage(agentRun.id, buildMessage(filePath, live, trigger, context)); - // throwOnError: surface any error event in the run's log (LLM API - // failures, tool errors, billing/credit issues) as a rejection so - // the failure branch records lastRunError. Without this the run - // can "complete" with errors silently and we'd hit the success - // branch with an empty summary, clobbering any prior lastRunError. - await waitForRunCompletion(agentRun.id, { throwOnError: true }); - const summary = await extractAgentResponse(agentRun.id); + // throwOnError: a failed/cancelled turn rejects here so the + // failure branch records lastRunError. Without this the turn + // could settle silently and we'd hit the success branch with an + // empty summary, clobbering any prior lastRunError. + const { summary } = await handle.done; const bodyAfter = await readNoteBody(filePath); const didUpdate = bodyAfter !== bodyBefore; diff --git a/apps/x/packages/core/src/knowledge/tag_notes.ts b/apps/x/packages/core/src/knowledge/tag_notes.ts index 7cc7b426..a1d36bcb 100644 --- a/apps/x/packages/core/src/knowledge/tag_notes.ts +++ b/apps/x/packages/core/src/knowledge/tag_notes.ts @@ -1,10 +1,9 @@ import fs from 'fs'; import path from 'path'; import { WorkDir } from '../config/config.js'; -import { createRun, createMessage } from '../runs/runs.js'; +import { runHeadlessAgent, toolInputPaths } from '../agents/headless.js'; import { getKgModel } from '../models/defaults.js'; -import { bus } from '../runs/bus.js'; -import { getErrorDetails, waitForRunCompletion } from '../agents/utils.js'; +import { getErrorDetails } from '../agents/utils.js'; import { serviceLogger } from '../services/service_logger.js'; import { limitEventItems } from './limit_event_items.js'; import { @@ -83,13 +82,6 @@ function getUntaggedNotes(state: NoteTaggingState): string[] { async function tagNoteBatch( files: { path: string; content: string }[] ): Promise<{ runId: string; filesEdited: Set }> { - const run = await createRun({ - agentId: NOTE_TAGGING_AGENT, - model: await getKgModel(), - useCase: 'knowledge_sync', - subUseCase: 'tag_notes', - }); - let message = `Tag the following ${files.length} knowledge notes by prepending YAML frontmatter with appropriate tags.\n\n`; message += `**Important:** Use workspace-relative paths with file-editText (e.g. "knowledge/People/Sarah Chen.md", NOT absolute paths).\n\n`; @@ -105,33 +97,16 @@ async function tagNoteBatch( message += `\n\n---\n\n`; } - const filesEdited = new Set(); - - const unsubscribe = await bus.subscribe(run.id, async (event) => { - if (event.type !== 'tool-invocation') { - return; - } - if (event.toolName !== 'file-editText') { - return; - } - try { - const parsed = JSON.parse(event.input) as { path?: string }; - if (typeof parsed.path === 'string') { - filesEdited.add(parsed.path); - } - } catch { - // ignore parse errors - } + const { turnId, state } = await runHeadlessAgent({ + agentId: NOTE_TAGGING_AGENT, + message, + model: await getKgModel(), + throwOnError: true, }); - await createMessage(run.id, message); - try { - await waitForRunCompletion(run.id, { throwOnError: true }); - } finally { - unsubscribe(); - } - - return { runId: run.id, filesEdited }; + // Edited paths come from the durable turn state instead of streaming + // bus subscriptions. + return { runId: turnId, filesEdited: toolInputPaths(state, ['file-editText']) }; } /** diff --git a/apps/x/packages/core/src/pre_built/runner.ts b/apps/x/packages/core/src/pre_built/runner.ts index 0596372f..3be9bcaf 100644 --- a/apps/x/packages/core/src/pre_built/runner.ts +++ b/apps/x/packages/core/src/pre_built/runner.ts @@ -1,9 +1,8 @@ import fs from 'fs'; import path from 'path'; import { WorkDir } from '../config/config.js'; -import { createRun, createMessage } from '../runs/runs.js'; +import { runHeadlessAgent } from '../agents/headless.js'; import { getKgModel } from '../models/defaults.js'; -import { waitForRunCompletion } from '../agents/utils.js'; import { loadConfig, loadState, @@ -38,15 +37,6 @@ async function runAgent(agentName: string): Promise { } try { - // Create a run for the agent - // The agent file is expected to be in the agents directory with the same name - const run = await createRun({ - agentId: agentName, - model: await getKgModel(), - useCase: 'knowledge_sync', - subUseCase: 'pre_built', - }); - // Build trigger message with user context const message = `Run your scheduled task. @@ -59,10 +49,14 @@ async function runAgent(agentName: string): Promise { Process new items and use the user context above to identify yourself when drafting responses.`; - await createMessage(run.id, message); - - // Wait for completion - await waitForRunCompletion(run.id); + // The agent file is expected to be in the agents directory with + // the same name. Waits for the turn to settle (errors tolerated, + // matching the old no-throwOnError wait). + await runHeadlessAgent({ + agentId: agentName, + message, + model: await getKgModel(), + }); // Update last run time setLastRunTime(agentName, new Date()); From 49e1e99d165a65e685e3337002efb0a7c9a9bf29 Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 13:53:47 +0530 Subject: [PATCH 20/24] fix(x/renderer): load headless transcripts via the turn runtime MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Background-task and live-note history views called the legacy runs:fetch for ids that are now turn ids (ENOENT after stage 6). New shared loader src/lib/agent-transcript.ts fetches sessions:getTurn first and falls back to runs:fetch so pre-migration histories stay readable (fallback dies with the runs runtime in stage 7); turn transcripts render through the same buildTurnConversation used by the chat views. Unit-tested (turnToTranscript mapping + failure surfacing). The chat-log download used runs:downloadLog with what is now a session id. Added sessions:downloadLog (concatenates the session's turn logs into one JSONL via the save dialog); the sidebar tries it first and falls back to runs:downloadLog for legacy background tabs. Remaining renderer runs:* callers are code-mode only (use-code-chat, runs:events feed) — the deliberate stage-7 carve-out. Co-Authored-By: Claude Fable 5 --- apps/x/apps/main/src/ipc.ts | 31 +++++++ .../renderer/src/components/bg-tasks-view.tsx | 70 ++++++--------- .../renderer/src/components/chat-sidebar.tsx | 9 +- .../src/components/live-note-sidebar.tsx | 20 ++--- .../renderer/src/lib/agent-transcript.test.ts | 28 ++++++ .../apps/renderer/src/lib/agent-transcript.ts | 89 +++++++++++++++++++ apps/x/packages/shared/src/ipc.ts | 8 ++ 7 files changed, 198 insertions(+), 57 deletions(-) create mode 100644 apps/x/apps/renderer/src/lib/agent-transcript.test.ts create mode 100644 apps/x/apps/renderer/src/lib/agent-transcript.ts diff --git a/apps/x/apps/main/src/ipc.ts b/apps/x/apps/main/src/ipc.ts index c290486e..89520999 100644 --- a/apps/x/apps/main/src/ipc.ts +++ b/apps/x/apps/main/src/ipc.ts @@ -907,6 +907,37 @@ export function setupIpcHandlers() { await container.resolve('sessions').deleteSession(args.sessionId); return { success: true }; }, + 'sessions:downloadLog': async (event, args) => { + // Concatenate the session's turn logs into one JSONL for debugging. + const sessions = container.resolve('sessions'); + const state = await sessions.getSession(args.sessionId); + const win = BrowserWindow.fromWebContents(event.sender); + const result = await dialog.showSaveDialog(win!, { + defaultPath: `${args.sessionId}.jsonl.log`, + filters: [ + { name: 'Chat Log', extensions: ['log'] }, + { name: 'JSONL', extensions: ['jsonl'] }, + { name: 'All Files', extensions: ['*'] }, + ], + }); + if (result.canceled || !result.filePath) { + return { success: false }; + } + try { + const lines: string[] = []; + for (const ref of state.turns) { + const turn = await sessions.getTurn(ref.turnId); + for (const turnEvent of turn.events) { + lines.push(JSON.stringify(turnEvent)); + } + } + await fs.writeFile(result.filePath, lines.join('\n') + '\n'); + return { success: true }; + } catch (err) { + const message = err instanceof Error ? err.message : 'Failed to download chat log'; + return { success: false, error: message }; + } + }, 'runs:downloadLog': async (event, args) => { const runFileName = `${args.runId}.jsonl`; if (path.basename(runFileName) !== runFileName) { diff --git a/apps/x/apps/renderer/src/components/bg-tasks-view.tsx b/apps/x/apps/renderer/src/components/bg-tasks-view.tsx index c48df79e..45e23169 100644 --- a/apps/x/apps/renderer/src/components/bg-tasks-view.tsx +++ b/apps/x/apps/renderer/src/components/bg-tasks-view.tsx @@ -6,9 +6,7 @@ import { Pencil, Check, PanelRightClose, PanelRightOpen, Sparkles, Code2, FolderOpen, LayoutTemplate, } from 'lucide-react' -import type { z } from 'zod' import type { BackgroundTask, BackgroundTaskSummary, Triggers } from '@x/shared/dist/background-task.js' -import type { Run } from '@x/shared/dist/runs.js' import { Button } from '@/components/ui/button' import { Switch } from '@/components/ui/switch' import { Input } from '@/components/ui/input' @@ -17,7 +15,7 @@ import { useBackgroundTaskAgentStatus } from '@/hooks/use-bg-task-agent-status' import { formatRelativeTime } from '@/lib/relative-time' import { toast } from '@/lib/toast' import type { ConversationItem } from '@/lib/chat-conversation' -import { runLogToConversation } from '@/lib/run-to-conversation' +import { fetchAgentRunTranscript, type AgentRunTranscript } from '@/lib/agent-transcript' import { CompactConversation } from '@/components/compact-conversation' import { RichMarkdownViewer } from '@/components/rich-markdown-viewer' import { HtmlFileViewer } from '@/components/html-file-viewer' @@ -970,10 +968,9 @@ function SetupTab({ // Runs history tab — list + drill-down transcript view // // Source of truth: `bg-tasks//runs.log` — a plain-text file with one -// runId per line (newest first). The actual transcripts live at the global -// `$WorkDir/runs/.jsonl`, so this tab fetches runIds via the bg-task -// IPC, then loads each Run through the standard `runs:fetch`. No bg-task- -// specific transcript path or schema needed. +// turn id per line (newest first). Transcripts live in the turn runtime's +// storage; this tab fetches ids via the bg-task IPC, then loads each through +// the shared agent-transcript loader (turn-first, legacy-run fallback). // --------------------------------------------------------------------------- interface RunRowSummary { @@ -984,27 +981,6 @@ interface RunRowSummary { error?: string } -// Pull the bits we want to display for a row out of a full Run's event log. -function summarizeRun(run: z.infer): RunRowSummary { - const out: RunRowSummary = { runId: run.id, createdAt: run.createdAt, trigger: run.subUseCase } - for (const event of run.log) { - if (event.type === 'error' && typeof event.error === 'string') { - out.error = event.error - } else if (event.type === 'message' && event.message?.role === 'assistant') { - const content = event.message.content - if (typeof content === 'string') { - out.summary = content - } else if (Array.isArray(content)) { - const text = content - .filter((p) => p.type === 'text') - .map((p) => ('text' in p ? p.text : '')) - .join('') - if (text) out.summary = text - } - } - } - return out -} function RunsHistoryTab({ slug, task }: { slug: string; task: BackgroundTask }) { const [rows, setRows] = useState([]) @@ -1016,19 +992,25 @@ function RunsHistoryTab({ slug, task }: { slug: string; task: BackgroundTask }) setLoading(true) try { const { runIds } = await window.ipc.invoke('bg-task:listRunIds', { slug, limit: 100 }) - // Fetch each Run in parallel via the canonical IPC. Runs whose - // jsonl no longer exists (deleted manually, never written, …) are - // dropped silently. + // Fetch transcripts in parallel (turn-first, legacy-run + // fallback). Ids whose files no longer exist keep a bare row so + // the user knows the run happened. const settled = await Promise.allSettled( - runIds.map(runId => window.ipc.invoke('runs:fetch', { runId })) + runIds.map(runId => fetchAgentRunTranscript(runId)) ) const next: RunRowSummary[] = [] for (let i = 0; i < settled.length; i++) { const r = settled[i] - if (r.status === 'fulfilled' && r.value) { - next.push(summarizeRun(r.value)) + if (r.status === 'fulfilled') { + const t = r.value + next.push({ + runId: t.id, + ...(t.createdAt === undefined ? {} : { createdAt: t.createdAt }), + ...(t.trigger === undefined ? {} : { trigger: t.trigger }), + ...(t.summary === undefined ? {} : { summary: t.summary }), + ...(t.error === undefined ? {} : { error: t.error }), + }) } else { - // Keep the row visible with just the id so the user knows it exists. next.push({ runId: runIds[i] }) } } @@ -1134,7 +1116,7 @@ function RunTranscriptView({ isInFlight: boolean onBack: () => void }) { - const [run, setRun] = useState | null>(null) + const [transcript, setTranscript] = useState(null) const [loading, setLoading] = useState(true) const [error, setError] = useState(null) @@ -1144,15 +1126,13 @@ function RunTranscriptView({ setError(null) void (async () => { try { - // Bg-task transcripts now live at the global runs/ location — - // same path resolution as every other run, no special handling. - const r = await window.ipc.invoke('runs:fetch', { runId }) + const t = await fetchAgentRunTranscript(runId) if (cancelled) return - setRun(r) + setTranscript(t) } catch (err) { if (cancelled) return setError(err instanceof Error ? err.message : String(err)) - setRun(null) + setTranscript(null) } finally { if (!cancelled) setLoading(false) } @@ -1160,8 +1140,8 @@ function RunTranscriptView({ return () => { cancelled = true } }, [runId]) - const summary = run ? summarizeRun(run) : undefined - const items: ConversationItem[] = run ? runLogToConversation(run.log) : [] + const summary = transcript ?? undefined + const items: ConversationItem[] = transcript?.items ?? [] return (
@@ -1221,10 +1201,10 @@ function RunTranscriptView({ Couldn't load transcript: {error}
)} - {run && !loading && items.length === 0 && ( + {transcript && !loading && items.length === 0 && (

No messages or tool calls recorded.

)} - {run && !loading && items.length > 0 && ( + {transcript && !loading && items.length > 0 && ( )}
diff --git a/apps/x/apps/renderer/src/components/chat-sidebar.tsx b/apps/x/apps/renderer/src/components/chat-sidebar.tsx index 893f9f1a..e24ac6de 100644 --- a/apps/x/apps/renderer/src/components/chat-sidebar.tsx +++ b/apps/x/apps/renderer/src/components/chat-sidebar.tsx @@ -378,7 +378,14 @@ export function ChatSidebar({ } try { - const result = await window.ipc.invoke('runs:downloadLog', { runId: activeRunId }) + // Session-first (new runtime); legacy runs fallback covers old + // background tabs until stage 7 removes the runs runtime. + let result: { success: boolean; error?: string } + try { + result = await window.ipc.invoke('sessions:downloadLog', { sessionId: activeRunId }) + } catch { + result = await window.ipc.invoke('runs:downloadLog', { runId: activeRunId }) + } if (result.success) { toast.success('Chat log saved') } else if (result.error) { diff --git a/apps/x/apps/renderer/src/components/live-note-sidebar.tsx b/apps/x/apps/renderer/src/components/live-note-sidebar.tsx index 1510e6c9..18654729 100644 --- a/apps/x/apps/renderer/src/components/live-note-sidebar.tsx +++ b/apps/x/apps/renderer/src/components/live-note-sidebar.tsx @@ -11,11 +11,9 @@ import { ChevronDown, ChevronRight, } from 'lucide-react' import { LiveNoteSchema, type LiveNote, type Triggers } from '@x/shared/dist/live-note.js' -import type { Run } from '@x/shared/dist/runs.js' -import type z from 'zod' import { useLiveNoteAgentStatus } from '@/hooks/use-live-note-agent-status' import { formatRelativeTime } from '@/lib/relative-time' -import { runLogToConversation } from '@/lib/run-to-conversation' +import { fetchAgentRunTranscript, type AgentRunTranscript } from '@/lib/agent-transcript' import { CompactConversation } from '@/components/compact-conversation' export type OpenLiveNotePanelDetail = { @@ -661,7 +659,7 @@ function SectionRegion({ label, children }: { label?: string; children: React.Re } function LastRunTab({ live }: { live: LiveNote }) { - const [run, setRun] = useState | null>(null) + const [transcript, setTranscript] = useState(null) const [loadingRun, setLoadingRun] = useState(false) const [fetchError, setFetchError] = useState(null) @@ -669,7 +667,7 @@ function LastRunTab({ live }: { live: LiveNote }) { useEffect(() => { if (!runId) { - setRun(null) + setTranscript(null) setFetchError(null) setLoadingRun(false) return @@ -679,13 +677,13 @@ function LastRunTab({ live }: { live: LiveNote }) { setFetchError(null) void (async () => { try { - const r = await window.ipc.invoke('runs:fetch', { runId }) + const t = await fetchAgentRunTranscript(runId) if (cancelled) return - setRun(r) + setTranscript(t) } catch (err) { if (cancelled) return setFetchError(err instanceof Error ? err.message : String(err)) - setRun(null) + setTranscript(null) } finally { if (!cancelled) setLoadingRun(false) } @@ -704,7 +702,7 @@ function LastRunTab({ live }: { live: LiveNote }) { } const isError = !!live.lastRunError - const items = run ? runLogToConversation(run.log) : [] + const items = transcript?.items ?? [] return (
@@ -751,10 +749,10 @@ function LastRunTab({ live }: { live: LiveNote }) { Couldn't load transcript: {fetchError}
)} - {run && !loadingRun && items.length === 0 && ( + {transcript && !loadingRun && items.length === 0 && (

No messages or tool calls recorded.

)} - {run && !loadingRun && items.length > 0 && ( + {transcript && !loadingRun && items.length > 0 && ( )}
diff --git a/apps/x/apps/renderer/src/lib/agent-transcript.test.ts b/apps/x/apps/renderer/src/lib/agent-transcript.test.ts new file mode 100644 index 00000000..311309f3 --- /dev/null +++ b/apps/x/apps/renderer/src/lib/agent-transcript.test.ts @@ -0,0 +1,28 @@ +import { describe, expect, it } from 'vitest' +import { turnToTranscript } from './agent-transcript' +import { completedTurnLog, created, requested, user } from './session-chat/test-fixtures' + +describe('turnToTranscript', () => { + it('maps a completed turn to id/createdAt/summary/items', () => { + const events = completedTurnLog('turn-1', 'sess-1', 'do the thing', 'all done') + const t = turnToTranscript('turn-1', events) + expect(t.id).toBe('turn-1') + expect(t.createdAt).toBeDefined() + expect(t.summary).toBe('all done') + expect(t.error).toBeUndefined() + expect(t.trigger).toBeUndefined() + expect(t.items.length).toBeGreaterThan(0) + }) + + it('surfaces turn_failed as error with no summary', () => { + const events = [ + created('turn-2', 'sess-1', user('go')), + requested('turn-2', 0), + { type: 'model_call_failed' as const, turnId: 'turn-2', ts: '2026-07-02T10:00:00Z', modelCallIndex: 0, error: 'boom' }, + { type: 'turn_failed' as const, turnId: 'turn-2', ts: '2026-07-02T10:00:00Z', error: 'boom', usage: {} }, + ] + const t = turnToTranscript('turn-2', events) + expect(t.error).toBe('boom') + expect(t.summary).toBeUndefined() + }) +}) diff --git a/apps/x/apps/renderer/src/lib/agent-transcript.ts b/apps/x/apps/renderer/src/lib/agent-transcript.ts new file mode 100644 index 00000000..71c4f75e --- /dev/null +++ b/apps/x/apps/renderer/src/lib/agent-transcript.ts @@ -0,0 +1,89 @@ +import type { z } from 'zod' +import type { Run } from '@x/shared/src/runs.js' +import { reduceTurn, type TurnEvent } from '@x/shared/src/turns.js' +import type { ConversationItem } from '@/lib/chat-conversation' +import { runLogToConversation } from '@/lib/run-to-conversation' +import { buildTurnConversation } from '@/lib/session-chat/turn-view' + +// Unified read model for a headless agent run's transcript, whether the id +// is a turn (new runtime) or a legacy run (pre-turn-runtime files under +// $WorkDir/runs/). Used by the background-tasks and live-note history views. + +export interface AgentRunTranscript { + id: string + createdAt?: string + // Legacy runs only (subUseCase); turns carry the trigger inside the + // message text instead. + trigger?: string + summary?: string + error?: string + items: ConversationItem[] +} + +export function turnToTranscript( + turnId: string, + events: Array>, +): AgentRunTranscript { + const state = reduceTurn(events) + const out: AgentRunTranscript = { + id: turnId, + createdAt: state.definition.ts, + items: buildTurnConversation(state), + } + if (state.terminal?.type === 'turn_failed') { + out.error = state.terminal.error + } + for (let i = state.modelCalls.length - 1; i >= 0; i--) { + const response = state.modelCalls[i].response + if (!response) continue + const content = response.content + const text = + typeof content === 'string' + ? content + : content.map((p) => (p.type === 'text' ? p.text : '')).join('') + if (text) { + out.summary = text + break + } + } + return out +} + +export function runToTranscript(run: z.infer): AgentRunTranscript { + const out: AgentRunTranscript = { + id: run.id, + createdAt: run.createdAt, + trigger: run.subUseCase, + items: runLogToConversation(run.log), + } + for (const event of run.log) { + if (event.type === 'error' && typeof event.error === 'string') { + out.error = event.error + } else if (event.type === 'message' && event.message?.role === 'assistant') { + const content = event.message.content + if (typeof content === 'string') { + out.summary = content + } else if (Array.isArray(content)) { + const text = content + .filter((p) => p.type === 'text') + .map((p) => ('text' in p ? p.text : '')) + .join('') + if (text) out.summary = text + } + } + } + return out +} + +// Turn-first with a legacy-run fallback so histories recorded before the +// turn-runtime migration stay readable. The fallback goes away with the +// runs runtime (stage 7). +export async function fetchAgentRunTranscript(id: string): Promise { + try { + const turn = await window.ipc.invoke('sessions:getTurn', { turnId: id }) + return turnToTranscript(id, turn.events) + } catch { + const run = await window.ipc.invoke('runs:fetch', { runId: id }) + return runToTranscript(run) + } +} diff --git a/apps/x/packages/shared/src/ipc.ts b/apps/x/packages/shared/src/ipc.ts index a174da9b..565ac47a 100644 --- a/apps/x/packages/shared/src/ipc.ts +++ b/apps/x/packages/shared/src/ipc.ts @@ -442,6 +442,14 @@ const ipcSchemas = { req: z.object({ sessionId: z.string(), title: z.string() }), res: z.object({ success: z.literal(true) }), }, + 'sessions:downloadLog': { + // Concatenates the session's turn logs into one JSONL for debugging. + req: z.object({ sessionId: z.string() }), + res: z.object({ + success: z.boolean(), + error: z.string().optional(), + }), + }, 'sessions:delete': { req: z.object({ sessionId: z.string() }), res: z.object({ success: z.literal(true) }), From fa6943e905033bc63456af7494b044aae2c9c58f Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 14:07:49 +0530 Subject: [PATCH 21/24] feat(x): llm_usage analytics for turn model calls + voice-output parity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Analytics: the new runtime emitted usage into durable events but never reported it to PostHog (only the permission classifier self-reported). New IUsageReporter seam on the turn loop — invoked once per completed model call, after the durable append, failure-isolated. The real bridge emits the same llm_usage event as the old loop; useCase/subUseCase come from the AsyncLocalStorage context the stage-6 headless runners already establish via withUseCase, defaulting to copilot_chat for UI-driven session turns (matching the old createRun default). Capturing at the loop covers ALL turns — the session bus never sees headless ones. Voice: the new renderer path rendered tags literally and never fired TTS. The live overlay now extracts completed blocks (robust to tags split across deltas) into chatState.voiceSegments; App speaks new segments via the existing useVoiceTTS when voice output is on, skipping segments streamed before a session became active. Tags are stripped from both the streaming message and persisted assistant messages, mirroring the legacy display-time strip. Tests: usage report assertion in the runtime suite; four voice tests (split-delta extraction, per-call scan reset, state stripping+segments, persisted-message stripping). Co-Authored-By: Claude Fable 5 --- apps/x/apps/renderer/src/App.tsx | 22 +++++++ .../src/lib/session-chat/turn-view.test.ts | 43 +++++++++++++ .../src/lib/session-chat/turn-view.ts | 62 ++++++++++++++++--- apps/x/packages/core/src/di/container.ts | 3 + .../src/turns/bridges/real-usage-reporter.ts | 22 +++++++ apps/x/packages/core/src/turns/index.ts | 1 + .../x/packages/core/src/turns/runtime.test.ts | 29 ++++++++- apps/x/packages/core/src/turns/runtime.ts | 20 ++++++ .../packages/core/src/turns/usage-reporter.ts | 19 ++++++ 9 files changed, 212 insertions(+), 9 deletions(-) create mode 100644 apps/x/packages/core/src/turns/bridges/real-usage-reporter.ts create mode 100644 apps/x/packages/core/src/turns/usage-reporter.ts diff --git a/apps/x/apps/renderer/src/App.tsx b/apps/x/apps/renderer/src/App.tsx index 6b820f70..5ef631e2 100644 --- a/apps/x/apps/renderer/src/App.tsx +++ b/apps/x/apps/renderer/src/App.tsx @@ -980,6 +980,28 @@ function App() { const ttsRef = useRef(tts) ttsRef.current = tts + // Speak newly completed blocks from the new runtime's live stream + // (parity with the legacy text-delta voice extraction below). The store + // accumulates completed blocks in chatState.voiceSegments; we speak only + // segments that appeared after the current session became active. + const spokenVoiceRef = useRef<{ key: string | null; count: number }>({ key: null, count: 0 }) + const voiceSegments = sessionChat.chatState?.voiceSegments + useEffect(() => { + if (!voiceSegments) return + if (spokenVoiceRef.current.key !== runId) { + // Session switch: skip anything already streamed before we arrived. + spokenVoiceRef.current = { key: runId, count: voiceSegments.length } + return + } + while (spokenVoiceRef.current.count < voiceSegments.length) { + const segment = voiceSegments[spokenVoiceRef.current.count] + spokenVoiceRef.current.count += 1 + if (ttsEnabledRef.current) { + ttsRef.current.speak(segment) + } + } + }, [voiceSegments, runId]) + const voice = useVoiceMode() const voiceRef = useRef(voice) voiceRef.current = voice diff --git a/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts b/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts index 937ad9a2..66a781ce 100644 --- a/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts +++ b/apps/x/apps/renderer/src/lib/session-chat/turn-view.test.ts @@ -82,6 +82,49 @@ describe('applyOverlay', () => { }) }) +describe('voice output', () => { + const delta = (d: string): Parameters[1] => + ({ type: 'text_delta', turnId: T1, modelCallIndex: 0, delta: d }) + + it('extracts completed blocks across split deltas', () => { + let overlay = emptyOverlay() + overlay = applyOverlay(overlay, delta('Sure! hello there and bye')) + expect(overlay.voiceSegments).toEqual(['hello there']) + overlay = applyOverlay(overlay, delta(' done')) + expect(overlay.voiceSegments).toEqual(['hello there', 'bye']) + }) + + it('keeps segments but resets the scan on model_call_completed', () => { + let overlay = emptyOverlay() + overlay = applyOverlay(overlay, delta('one')) + overlay = applyOverlay(overlay, completed(T1, 0, assistantText('one'))) + expect(overlay.voiceSegments).toEqual(['one']) + expect(overlay.text).toBe('') + overlay = applyOverlay(overlay, delta('two')) + expect(overlay.voiceSegments).toEqual(['one', 'two']) + }) + + it('strips voice tags from the streaming message and exposes segments on state', () => { + const turn = reduceTurn([created(T1, S1), requested(T1, 0)]) + let overlay = emptyOverlay() + overlay = applyOverlay(overlay, delta('Plan: speak this rest')) + const state = buildSessionChatState([turn], overlay) + expect(state.currentAssistantMessage).toBe('Plan: speak this rest') + expect(state.voiceSegments).toEqual(['speak this']) + }) + + it('strips voice tags from persisted assistant messages', () => { + const state = reduceTurn( + completedTurnLog(T1, S1, 'q', 'Sure. Here you go. Done.'), + ) + const items = buildTurnConversation(state) + const assistant = items.find((i) => isChatMessage(i) && i.role === 'assistant') + expect(isChatMessage(assistant!) && assistant.content).toBe('Sure. Here you go. Done.') + }) +}) + describe('buildTurnConversation', () => { it('maps user input, assistant text, and settled tool calls', () => { const call = assistantCalls(toolCallPart('tc1', 'echo', { x: 1 })) diff --git a/apps/x/apps/renderer/src/lib/session-chat/turn-view.ts b/apps/x/apps/renderer/src/lib/session-chat/turn-view.ts index fad07998..e4ca3cdd 100644 --- a/apps/x/apps/renderer/src/lib/session-chat/turn-view.ts +++ b/apps/x/apps/renderer/src/lib/session-chat/turn-view.ts @@ -34,20 +34,62 @@ export type LiveOverlay = { text: string reasoning: string toolOutput: Record + // Contents of completed … blocks seen while streaming, in + // order, monotonically growing for the lifetime of the overlay (i.e. one + // active turn). Consumers speak segments beyond what they've already + // spoken; the overlay reset on turn switch starts a fresh list. + voiceSegments: string[] + // Scan cursor into `text` — everything before it has been checked for + // complete voice blocks. + voiceScanIndex: number } -export const emptyOverlay = (): LiveOverlay => ({ text: '', reasoning: '', toolOutput: {} }) +export const emptyOverlay = (): LiveOverlay => ({ + text: '', + reasoning: '', + toolOutput: {}, + voiceSegments: [], + voiceScanIndex: 0, +}) + +// The model emits … around speakable text when voice output +// is enabled; tags are never shown to the user. +export function stripVoiceTags(text: string): string { + return text.replace(/<\/?voice>/g, '') +} + +const VOICE_BLOCK = /([\s\S]*?)<\/voice>/g // Accumulates deltas; canonical durable events supersede the buffers (the // committed transcript now contains what was streaming). export function applyOverlay(overlay: LiveOverlay, event: TurnStreamEvent): LiveOverlay { switch (event.type) { - case 'text_delta': - return { ...overlay, text: overlay.text + event.delta } + case 'text_delta': { + const text = overlay.text + event.delta + // Extract complete voice blocks past the scan cursor. Incomplete + // blocks (opening tag seen, closing not yet) stay unconsumed until a + // later delta completes them. + const segments: string[] = [] + let scanIndex = overlay.voiceScanIndex + VOICE_BLOCK.lastIndex = scanIndex + for (let m = VOICE_BLOCK.exec(text); m; m = VOICE_BLOCK.exec(text)) { + const content = m[1].trim() + if (content) segments.push(content) + scanIndex = m.index + m[0].length + } + return { + ...overlay, + text, + ...(segments.length > 0 + ? { voiceSegments: [...overlay.voiceSegments, ...segments] } + : {}), + voiceScanIndex: scanIndex, + } + } case 'reasoning_delta': return { ...overlay, reasoning: overlay.reasoning + event.delta } case 'model_call_completed': - return { ...overlay, text: '', reasoning: '' } + return { ...overlay, text: '', reasoning: '', voiceScanIndex: 0 } case 'tool_progress': { const progress = event.progress if ( @@ -140,13 +182,16 @@ export function buildTurnConversation(state: TurnState): ConversationItem[] { for (const call of state.modelCalls) { if (call.response === undefined) continue const content = call.response.content - const text = + // Voice tags are model-facing markup, never shown (parity with the + // legacy path's display-time strip). + const text = stripVoiceTags( typeof content === 'string' ? content : content .map((part) => (part.type === 'text' ? part.text : '')) .filter(Boolean) - .join('\n') + .join('\n'), + ) if (text) { items.push({ id: `${turnId}:a${call.index}`, @@ -192,6 +237,8 @@ type PermMeta = z.infer export type SessionChatState = { conversation: ConversationItem[] currentAssistantMessage: string + // See LiveOverlay.voiceSegments. + voiceSegments: string[] pendingAskHumanRequests: Map> allPermissionRequests: Map> permissionResponses: Map @@ -289,7 +336,8 @@ export function buildSessionChatState( const settled = status === 'completed' || status === 'failed' || status === 'cancelled' return { conversation, - currentAssistantMessage: overlay.text, + currentAssistantMessage: stripVoiceTags(overlay.text), + voiceSegments: overlay.voiceSegments, pendingAskHumanRequests, allPermissionRequests, permissionResponses, diff --git a/apps/x/packages/core/src/di/container.ts b/apps/x/packages/core/src/di/container.ts index 34388754..e9be949f 100644 --- a/apps/x/packages/core/src/di/container.ts +++ b/apps/x/packages/core/src/di/container.ts @@ -31,6 +31,8 @@ import { FSTurnRepo } from "../turns/fs-repo.js"; import type { ITurnRepo } from "../turns/repo.js"; import { TurnRepoContextResolver, type IContextResolver } from "../turns/context-resolver.js"; import { EmitterTurnLifecycleBus, type ITurnLifecycleBus } from "../turns/bus.js"; +import { RealUsageReporter } from "../turns/bridges/real-usage-reporter.js"; +import type { IUsageReporter } from "../turns/usage-reporter.js"; import { TurnRuntime } from "../turns/runtime.js"; import type { ITurnRuntime } from "../turns/api.js"; import type { IAgentResolver } from "../turns/agent-resolver.js"; @@ -103,6 +105,7 @@ container.register({ turnRepo: asClass(FSTurnRepo).singleton(), contextResolver: asClass(TurnRepoContextResolver).singleton(), lifecycleBus: asClass(EmitterTurnLifecycleBus).singleton(), + usageReporter: asClass(RealUsageReporter).singleton(), agentResolver: asFunction(() => new RealAgentResolver()).singleton(), modelRegistry: asFunction(() => new RealModelRegistry()).singleton(), toolRegistry: asFunction(() => new RealToolRegistry()).singleton(), diff --git a/apps/x/packages/core/src/turns/bridges/real-usage-reporter.ts b/apps/x/packages/core/src/turns/bridges/real-usage-reporter.ts new file mode 100644 index 00000000..99d004e5 --- /dev/null +++ b/apps/x/packages/core/src/turns/bridges/real-usage-reporter.ts @@ -0,0 +1,22 @@ +import { captureLlmUsage } from "../../analytics/usage.js"; +import { getCurrentUseCase } from "../../analytics/use_case.js"; +import type { IUsageReporter, ModelUsageReport } from "../usage-reporter.js"; + +// Reports each completed model call as the same `llm_usage` PostHog event +// the old run loop emitted. The use case comes from the AsyncLocalStorage +// context the caller established (headless runners wrap startHeadlessAgent +// in withUseCase); UI-driven session turns have no context and default to +// copilot_chat — matching the old createRun default. +export class RealUsageReporter implements IUsageReporter { + reportModelUsage(report: ModelUsageReport): void { + const context = getCurrentUseCase(); + captureLlmUsage({ + useCase: context?.useCase ?? "copilot_chat", + ...(context?.subUseCase ? { subUseCase: context.subUseCase } : {}), + agentName: report.agentId, + model: report.model.model, + provider: report.model.provider, + usage: report.usage, + }); + } +} diff --git a/apps/x/packages/core/src/turns/index.ts b/apps/x/packages/core/src/turns/index.ts index 1ec99055..77153361 100644 --- a/apps/x/packages/core/src/turns/index.ts +++ b/apps/x/packages/core/src/turns/index.ts @@ -13,3 +13,4 @@ export * from "./repo.js"; export * from "./runtime.js"; export * from "./stream.js"; export * from "./tool-registry.js"; +export * from "./usage-reporter.js"; diff --git a/apps/x/packages/core/src/turns/runtime.test.ts b/apps/x/packages/core/src/turns/runtime.test.ts index 170ba61b..335250e4 100644 --- a/apps/x/packages/core/src/turns/runtime.test.ts +++ b/apps/x/packages/core/src/turns/runtime.test.ts @@ -308,6 +308,7 @@ function makeRuntime(opts: { const checker = opts.checker ?? new FakePermissionChecker(); const classifier = opts.classifier ?? new FakePermissionClassifier(); const bus = new FakeBus(); + const usage = new FakeUsageReporter(); const runtime = new TurnRuntime({ turnRepo: repo, idGenerator: new FakeIdGen(opts.idStart ?? 0), @@ -319,8 +320,9 @@ function makeRuntime(opts: { permissionChecker: checker, permissionClassifier: classifier, lifecycleBus: bus, + usageReporter: usage, }); - return { runtime, repo, models, checker, classifier, bus }; + return { runtime, repo, models, checker, classifier, bus, usage }; } async function newTurn( @@ -371,6 +373,21 @@ async function advanceAndSettle( return settle(runtime.advanceTurn(turnId, input, options)); } +class FakeUsageReporter { + reports: Array<{ + agentId: string; + model: { provider: string; model: string }; + usage: Record; + }> = []; + reportModelUsage(report: { + agentId: string; + model: { provider: string; model: string }; + usage: Record; + }): void { + this.reports.push(report); + } +} + function typesOf(events: Array<{ type: string }>): string[] { return events.map((e) => e.type); } @@ -397,7 +414,7 @@ async function persisted( describe("plain model response (26.1)", () => { it("runs one model step to completion with exact persisted request", async () => { - const { runtime, repo, models, bus } = makeRuntime({ + const { runtime, repo, models, bus, usage } = makeRuntime({ models: [ respond( { type: "text_delta", delta: "do" }, @@ -433,6 +450,14 @@ describe("plain model response (26.1)", () => { // snapshot tools, encoded messages. expect(models.requests[0].systemPrompt).toBe("SYS"); expect(sentMessages(models.requests[0])).toEqual([user("hello")]); + // One usage report per completed model call, after the durable append. + expect(usage.reports).toEqual([ + { + agentId: "copilot", + model: defaultAgent.model, + usage: { inputTokens: 10, outputTokens: 5, totalTokens: 15 }, + }, + ]); // Deltas are streamed but never persisted. expect(events.filter((e) => e.type === "text_delta")).toHaveLength(2); expect(typesOf(log)).not.toContain("text_delta"); diff --git a/apps/x/packages/core/src/turns/runtime.ts b/apps/x/packages/core/src/turns/runtime.ts index 80f657b9..da2362a4 100644 --- a/apps/x/packages/core/src/turns/runtime.ts +++ b/apps/x/packages/core/src/turns/runtime.ts @@ -41,6 +41,7 @@ import { import type { ITurnLifecycleBus } from "./bus.js"; import type { IClock } from "./clock.js"; import { composeModelRequest } from "./compose-model-request.js"; +import type { IUsageReporter } from "./usage-reporter.js"; import type { IContextResolver } from "./context-resolver.js"; import type { IModelRegistry, @@ -68,6 +69,7 @@ export interface TurnRuntimeDependencies { permissionChecker: IPermissionChecker; permissionClassifier: IPermissionClassifier; lifecycleBus: ITurnLifecycleBus; + usageReporter: IUsageReporter; } // Immutable dependency container: holds no mutable per-turn state. All active @@ -83,6 +85,7 @@ export class TurnRuntime implements ITurnRuntime { private readonly permissionChecker: IPermissionChecker; private readonly permissionClassifier: IPermissionClassifier; private readonly lifecycleBus: ITurnLifecycleBus; + private readonly usageReporter: IUsageReporter; constructor({ turnRepo, @@ -95,6 +98,7 @@ export class TurnRuntime implements ITurnRuntime { permissionChecker, permissionClassifier, lifecycleBus, + usageReporter, }: TurnRuntimeDependencies) { this.turnRepo = turnRepo; this.idGenerator = idGenerator; @@ -106,6 +110,7 @@ export class TurnRuntime implements ITurnRuntime { this.permissionChecker = permissionChecker; this.permissionClassifier = permissionClassifier; this.lifecycleBus = lifecycleBus; + this.usageReporter = usageReporter; } async createTurn(input: CreateTurnInput): Promise { @@ -257,6 +262,7 @@ export class TurnRuntime implements ITurnRuntime { resolvedContext, resolvedAgent, model, + usageReporter: this.usageReporter, toolsByName, signal: controller.signal, turnRepo: this.turnRepo, @@ -285,6 +291,7 @@ class TurnAdvance { private readonly resolvedContext: Array>; private readonly resolvedAgent: z.infer; private readonly model: ResolvedModel; + private readonly usageReporter: IUsageReporter; private readonly toolsByName: Map; private readonly signal: AbortSignal; private readonly turnRepo: ITurnRepo; @@ -306,6 +313,7 @@ class TurnAdvance { resolvedContext: Array>; resolvedAgent: z.infer; model: ResolvedModel; + usageReporter: IUsageReporter; toolsByName: Map; signal: AbortSignal; turnRepo: ITurnRepo; @@ -320,6 +328,7 @@ class TurnAdvance { this.resolvedContext = init.resolvedContext; this.resolvedAgent = init.resolvedAgent; this.model = init.model; + this.usageReporter = init.usageReporter; this.toolsByName = init.toolsByName; this.signal = init.signal; this.turnRepo = init.turnRepo; @@ -1015,6 +1024,17 @@ class TurnAdvance { ? {} : { providerMetadata: completion.providerMetadata }), }); + // Analytics after the durable barrier; a reporter failure must never + // affect the turn. + try { + this.usageReporter.reportModelUsage({ + agentId: this.resolvedAgent.agentId, + model: this.resolvedAgent.model, + usage: completion.usage, + }); + } catch { + // best effort + } return undefined; } diff --git a/apps/x/packages/core/src/turns/usage-reporter.ts b/apps/x/packages/core/src/turns/usage-reporter.ts new file mode 100644 index 00000000..90bbd284 --- /dev/null +++ b/apps/x/packages/core/src/turns/usage-reporter.ts @@ -0,0 +1,19 @@ +import type { z } from "zod"; +import type { ModelDescriptor, TurnUsage } from "@x/shared/dist/turns.js"; + +// Analytics seam for the turn loop: invoked once per completed model call, +// after the durable model_call_completed append. Implementations must never +// throw into the loop and must not block it (fire-and-forget). +export interface ModelUsageReport { + agentId: string; + model: z.infer; + usage: z.infer; +} + +export interface IUsageReporter { + reportModelUsage(report: ModelUsageReport): void; +} + +export class NoopUsageReporter implements IUsageReporter { + reportModelUsage(): void {} +} From 627451cd943ab7df525258ac29947ca9d02c6ee5 Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 14:19:24 +0530 Subject: [PATCH 22/24] =?UTF-8?q?docs(x):=20stage=207=20resolved=20?= =?UTF-8?q?=E2=80=94=20document=20the=20code-mode=20runs=20carve-out?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Decision: defer code-mode unification; no deletions. The legacy runs runtime (runs/, AgentRuntime/streamAgent, runs:* IPC, App.tsx legacy tab state) remains solely for code-mode sessions. AGENTS.md gets the authoritative carve-out section: what stays and why, the no-new-callers rule (headless work uses agents/headless.ts), the temporary fallbacks that die with unification, and the scoped future project (rowboat-mode code prompts as composition turns; direct-mode ACP streams as a delegated turn kind). Turn-runtime design doc status updated to implemented-and-live. Co-Authored-By: Claude Fable 5 --- AGENTS.md | 27 +++++++++++++++++++ .../packages/core/docs/turn-runtime-design.md | 6 ++--- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 9a9b0a73..0b40683a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -58,6 +58,33 @@ session ids are auto-detected. This is the intended way to see "what did the model actually receive" — the raw JSONL deliberately stores structural facts and references, never the derived wire form. +## The legacy runs runtime is code-mode-only + +The old runs runtime (`packages/core/src/runs/`, `AgentRuntime`/`streamAgent` +in `packages/core/src/agents/runtime.ts`, the `runs:*` IPC channels, and the +renderer's legacy chat-tab state machine in `App.tsx`) remains in the tree +**solely for code-mode sessions** — a deliberate, documented carve-out: + +- A code session IS a run (`sessionId === runId`). Direct-mode prompts drive + an external ACP agent and hand-assemble `code-run-event`s into the run log + (`code-mode/sessions/service.ts`); rowboat-mode code tabs go through + `runs:createMessage` → the old `AgentRuntime` loop. +- Everything else — main chat, background tasks, live notes, knowledge + pipelines, scheduled agents — runs on turns/sessions. Do NOT add new + callers of `createRun`/`createMessage`/`streamAgent`; headless work uses + `packages/core/src/agents/headless.ts` (`startHeadlessAgent`/ + `runHeadlessAgent`). +- Temporary bridges that die when code-mode is unified: the renderer + transcript loader's `runs:fetch` fallback (`lib/agent-transcript.ts`), the + `runs:downloadLog` fallback in the chat sidebar, and the `notify-user` + gate's `fetchRun` fallback (`application/lib/builtin-tools.ts`). +- The remaining migration (designed but deferred): port code sessions onto + sessions/turns — rowboat-mode prompts become normal turns with + `codeMode`/`codeCwd` composition (already supported by the agent + resolver's composition overrides), direct-mode prompts become a delegated + turn kind carrying opaque code events. That project deletes `runs/` + entirely. + ## Invariants to respect - Turn/session files are **append-only**; reducers reject impossible diff --git a/apps/x/packages/core/docs/turn-runtime-design.md b/apps/x/packages/core/docs/turn-runtime-design.md index 091168f2..d199fabe 100644 --- a/apps/x/packages/core/docs/turn-runtime-design.md +++ b/apps/x/packages/core/docs/turn-runtime-design.md @@ -1,8 +1,8 @@ # Turn Runtime Technical Specification -Status: design complete for the turn layer, amended after review for session -integration: context references, revised crash-recovery semantics, and a -distinguishable model-call-limit outcome. No implementation exists yet. The +Status: implemented and live. All chat, background, and knowledge callers +run on this runtime; the legacy runs runtime remains only for code-mode +sessions (see the carve-out section in the repo-root `AGENTS.md`). The companion session layer is specified in `session-design.md`. This document specifies a new turn-oriented agent loop for `@x/core`. It is From eaa55df1610d9b5e7600c331bfd9a1aaf2686436 Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 14:23:39 +0530 Subject: [PATCH 23/24] ci(x): parent test harness + GitHub Actions workflow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit apps/x gains npm test — builds @x/shared first (the core and renderer suites import its dist), then runs the shared, core, and renderer vitest suites in dependency order; per-suite test:shared/test:core/ test:renderer scripts for targeted runs. .github/workflows/x-tests.yml runs that harness on every pull request touching apps/x (plus pushes to main and manual dispatch): pnpm 10 / Node 22, frozen lockfile, pnpm store cached against apps/x's lockfile. Co-Authored-By: Claude Fable 5 --- .github/workflows/x-tests.yml | 40 +++++++++++++++++++++++++++++++++++ apps/x/package.json | 8 +++++-- 2 files changed, 46 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/x-tests.yml diff --git a/.github/workflows/x-tests.yml b/.github/workflows/x-tests.yml new file mode 100644 index 00000000..f657d478 --- /dev/null +++ b/.github/workflows/x-tests.yml @@ -0,0 +1,40 @@ +name: apps/x tests + +on: + pull_request: + paths: + - "apps/x/**" + - ".github/workflows/x-tests.yml" + push: + branches: [main] + paths: + - "apps/x/**" + - ".github/workflows/x-tests.yml" + workflow_dispatch: + +jobs: + test: + runs-on: ubuntu-latest + defaults: + run: + working-directory: apps/x + steps: + - uses: actions/checkout@v4 + + - uses: pnpm/action-setup@v4 + with: + version: 10 + + - uses: actions/setup-node@v4 + with: + node-version: 22 + cache: pnpm + cache-dependency-path: apps/x/pnpm-lock.yaml + + - name: Install dependencies + run: pnpm install --frozen-lockfile + + # Builds @x/shared (core/renderer tests import its dist), then runs + # the shared, core, and renderer vitest suites in order. + - name: Run tests + run: npm test diff --git a/apps/x/package.json b/apps/x/package.json index 5d875653..d0931dc7 100644 --- a/apps/x/package.json +++ b/apps/x/package.json @@ -12,7 +12,11 @@ "deps": "npm run shared && npm run core && npm run preload", "main": "wait-on http://localhost:5173 && cd apps/main && npm run build && npm run start", "lint": "eslint .", - "lint:fix": "eslint . --fix" + "lint:fix": "eslint . --fix", + "test": "npm run shared && npm run test:shared && npm run test:core && npm run test:renderer", + "test:shared": "cd packages/shared && npm test", + "test:core": "cd packages/core && npm test", + "test:renderer": "cd apps/renderer && npm test" }, "devDependencies": { "@eslint/js": "^9.39.2", @@ -26,4 +30,4 @@ "typescript-eslint": "^8.50.1", "wait-on": "^9.0.3" } -} \ No newline at end of file +} From 0e316d12734f1b926ff74b4e3e389c8d09faac7d Mon Sep 17 00:00:00 2001 From: Ramnique Singh <30795890+ramnique@users.noreply.github.com> Date: Thu, 2 Jul 2026 15:18:30 +0530 Subject: [PATCH 24/24] refactor(x/core): inject headless agent dependencies --- .../core/src/agent-schedule/runner.ts | 2 +- .../packages/core/src/agents/headless-app.ts | 25 ++++ .../packages/core/src/agents/headless.test.ts | 66 +++++++-- apps/x/packages/core/src/agents/headless.ts | 139 ++++++++++-------- .../core/src/background-tasks/runner.ts | 2 +- apps/x/packages/core/src/di/container.ts | 12 ++ .../core/src/knowledge/agent_notes.ts | 2 +- .../core/src/knowledge/build_graph.ts | 2 +- .../core/src/knowledge/inline_tasks.ts | 2 +- .../core/src/knowledge/label_emails.ts | 2 +- .../core/src/knowledge/live-note/runner.ts | 2 +- .../packages/core/src/knowledge/tag_notes.ts | 2 +- .../core/src/models/default-model-resolver.ts | 11 ++ apps/x/packages/core/src/pre_built/runner.ts | 2 +- 14 files changed, 191 insertions(+), 80 deletions(-) create mode 100644 apps/x/packages/core/src/agents/headless-app.ts create mode 100644 apps/x/packages/core/src/models/default-model-resolver.ts diff --git a/apps/x/packages/core/src/agent-schedule/runner.ts b/apps/x/packages/core/src/agent-schedule/runner.ts index e34c9e11..bd818388 100644 --- a/apps/x/packages/core/src/agent-schedule/runner.ts +++ b/apps/x/packages/core/src/agent-schedule/runner.ts @@ -4,7 +4,7 @@ import { IAgentScheduleRepo } from "./repo.js"; import { IAgentScheduleStateRepo } from "./state-repo.js"; import { AgentScheduleConfig, AgentScheduleEntry } from "@x/shared/dist/agent-schedule.js"; import { AgentScheduleState, AgentScheduleStateEntry } from "@x/shared/dist/agent-schedule-state.js"; -import { startHeadlessAgent } from "../agents/headless.js"; +import { startHeadlessAgent } from "../agents/headless-app.js"; import { withUseCase } from "../analytics/use_case.js"; import z from "zod"; diff --git a/apps/x/packages/core/src/agents/headless-app.ts b/apps/x/packages/core/src/agents/headless-app.ts new file mode 100644 index 00000000..d9b78f3a --- /dev/null +++ b/apps/x/packages/core/src/agents/headless-app.ts @@ -0,0 +1,25 @@ +import container from "../di/container.js"; +import { + type HeadlessAgentHandle, + type HeadlessAgentOptions, + type HeadlessAgentResult, + type IHeadlessAgentRunner, +} from "./headless.js"; + +function runner(): IHeadlessAgentRunner { + return container.resolve("headlessAgentRunner"); +} + +export function startHeadlessAgent( + options: HeadlessAgentOptions, +): Promise { + return runner().start(options); +} + +export function runHeadlessAgent( + options: HeadlessAgentOptions, +): Promise { + return runner().run(options); +} + +export { toolInputPaths } from "./headless.js"; diff --git a/apps/x/packages/core/src/agents/headless.test.ts b/apps/x/packages/core/src/agents/headless.test.ts index 202bfbce..1f756ffd 100644 --- a/apps/x/packages/core/src/agents/headless.test.ts +++ b/apps/x/packages/core/src/agents/headless.test.ts @@ -1,4 +1,4 @@ -import { describe, expect, it } from "vitest"; +import { describe, expect, it, vi } from "vitest"; import type { z } from "zod"; import { reduceTurn, type TurnEvent } from "@x/shared/dist/turns.js"; import type { @@ -9,11 +9,12 @@ import type { } from "../turns/api.js"; import { HeadlessRunError, + HeadlessAgentRunner, assistantText, lastAssistantText, - startHeadlessAgent, toolInputPaths, } from "./headless.js"; +import type { IDefaultModelResolver } from "../models/default-model-resolver.js"; type TEvent = z.infer; @@ -166,6 +167,22 @@ const completedOutcome: TurnOutcome = { usage: {}, }; +function createRunner( + runtime: ITurnRuntime, + resolve = vi.fn(async () => ({ + provider: "default-provider", + model: "default-model", + })), +): { runner: HeadlessAgentRunner; resolve: IDefaultModelResolver["resolve"] } { + return { + runner: new HeadlessAgentRunner({ + turnRuntime: runtime, + defaultModelResolver: { resolve }, + }), + resolve, + }; +} + describe("headless agent helpers", () => { it("assistantText handles string and parts content", () => { expect(assistantText({ role: "assistant", content: "hi" })).toBe("hi"); @@ -206,14 +223,18 @@ describe("headless agent helpers", () => { }); }); -describe("startHeadlessAgent", () => { +describe("HeadlessAgentRunner", () => { it("creates a standalone auto-permission turn and returns the id before settling", async () => { const runtime = new FakeRuntime(turnLog({ responseText: "the summary" }), completedOutcome); - const handle = await startHeadlessAgent( - { agentId: "worker", message: "go", model: "m", provider: "fake" }, - runtime, - ); + const { runner, resolve } = createRunner(runtime); + const handle = await runner.start({ + agentId: "worker", + message: "go", + model: "m", + provider: "fake", + }); expect(handle.turnId).toBe(TURN_ID); + expect(resolve).not.toHaveBeenCalled(); expect(runtime.createInputs[0]).toMatchObject({ agent: { agentId: "worker", overrides: { model: { provider: "fake", model: "m" } } }, sessionId: null, @@ -228,20 +249,38 @@ describe("startHeadlessAgent", () => { it("omits the model override when neither model nor provider is set", async () => { const runtime = new FakeRuntime(turnLog({ responseText: "ok" }), completedOutcome); - await startHeadlessAgent({ agentId: "worker", message: "go" }, runtime); + const { runner, resolve } = createRunner(runtime); + await runner.start({ agentId: "worker", message: "go" }); + expect(resolve).not.toHaveBeenCalled(); expect(runtime.createInputs[0].agent.overrides).toBeUndefined(); }); + it("uses an injected default only for a partial model override", async () => { + const runtime = new FakeRuntime(turnLog({ responseText: "ok" }), completedOutcome); + const { runner, resolve } = createRunner(runtime); + await runner.start({ + agentId: "worker", + message: "go", + model: "custom-model", + }); + expect(resolve).toHaveBeenCalledOnce(); + expect(runtime.createInputs[0].agent.overrides).toEqual({ + model: { provider: "default-provider", model: "custom-model" }, + }); + }); + it("throwOnError rejects done with HeadlessRunError on failed outcomes", async () => { const runtime = new FakeRuntime(turnLog({ failed: "provider exploded" }), { status: "failed", error: "provider exploded", usage: {}, }); - const handle = await startHeadlessAgent( - { agentId: "worker", message: "go", throwOnError: true }, - runtime, - ); + const { runner } = createRunner(runtime); + const handle = await runner.start({ + agentId: "worker", + message: "go", + throwOnError: true, + }); await expect(handle.done).rejects.toThrowError(HeadlessRunError); await expect(handle.done).rejects.toThrowError("provider exploded"); }); @@ -252,7 +291,8 @@ describe("startHeadlessAgent", () => { error: "boom", usage: {}, }); - const handle = await startHeadlessAgent({ agentId: "worker", message: "go" }, runtime); + const { runner } = createRunner(runtime); + const handle = await runner.start({ agentId: "worker", message: "go" }); const result = await handle.done; expect(result.outcome.status).toBe("failed"); expect(result.summary).toBeNull(); diff --git a/apps/x/packages/core/src/agents/headless.ts b/apps/x/packages/core/src/agents/headless.ts index 54f20826..d974a5fd 100644 --- a/apps/x/packages/core/src/agents/headless.ts +++ b/apps/x/packages/core/src/agents/headless.ts @@ -4,14 +4,13 @@ import { reduceTurn, type TurnState, } from "@x/shared/dist/turns.js"; -import container from "../di/container.js"; -import { getDefaultModelAndProvider } from "../models/defaults.js"; import type { ITurnRuntime, TurnOutcome } from "../turns/api.js"; +import type { IDefaultModelResolver } from "../models/default-model-resolver.js"; // Drop-in replacement for the old headless runs pattern // (createRun → createMessage → waitForRunCompletion → extractAgentResponse): // one standalone turn per invocation (sessionId null, automatic permissions, -// no human). startHeadlessAgent returns the turn id immediately (callers +// no human). HeadlessAgentRunner.start returns the turn id immediately (callers // record it in pointer files / bus events before completion); `done` settles // with the outcome, the reduced turn state, and the final assistant text. @@ -51,6 +50,18 @@ export interface HeadlessAgentHandle { done: Promise; } +export interface HeadlessAgentRunnerDependencies { + turnRuntime: ITurnRuntime; + defaultModelResolver: IDefaultModelResolver; +} + +export interface IHeadlessAgentRunner { + start(options: HeadlessAgentOptions): Promise; + run( + options: HeadlessAgentOptions, + ): Promise; +} + export function assistantText( message: z.infer, ): string | null { @@ -97,63 +108,75 @@ export function toolInputPaths( return paths; } -export async function startHeadlessAgent( - options: HeadlessAgentOptions, - // Injectable for tests; defaults to the app container's runtime. - turnRuntime: ITurnRuntime = container.resolve("turnRuntime"), -): Promise { - let modelOverride: { provider: string; model: string } | undefined; - if (options.model || options.provider) { - const defaults = await getDefaultModelAndProvider(); - modelOverride = { - provider: options.provider ?? defaults.provider, - model: options.model ?? defaults.model, - }; +export class HeadlessAgentRunner implements IHeadlessAgentRunner { + private readonly turnRuntime: ITurnRuntime; + private readonly defaultModelResolver: IDefaultModelResolver; + + constructor({ + turnRuntime, + defaultModelResolver, + }: HeadlessAgentRunnerDependencies) { + this.turnRuntime = turnRuntime; + this.defaultModelResolver = defaultModelResolver; } - const turnId = await turnRuntime.createTurn({ - agent: { - agentId: options.agentId, - ...(modelOverride ? { overrides: { model: modelOverride } } : {}), - }, - sessionId: null, - context: [], - input: { role: "user", content: options.message }, - config: { - autoPermission: true, - humanAvailable: false, - ...(options.maxModelCalls === undefined - ? {} - : { maxModelCalls: options.maxModelCalls }), - }, - }); - - const execution = turnRuntime.advanceTurn(turnId, undefined, { - signal: options.signal, - }); - const done = execution.outcome.then(async (outcome) => { - const state = reduceTurn((await turnRuntime.getTurn(turnId)).events); - if (options.throwOnError && outcome.status !== "completed") { - throw new HeadlessRunError( - outcome.status === "failed" - ? outcome.error - : `turn ${outcome.status}`, - turnId, - outcome, - ); + async start(options: HeadlessAgentOptions): Promise { + let modelOverride: { provider: string; model: string } | undefined; + if (options.model && options.provider) { + modelOverride = { provider: options.provider, model: options.model }; + } else if (options.model || options.provider) { + const defaults = await this.defaultModelResolver.resolve(); + modelOverride = { + provider: options.provider ?? defaults.provider, + model: options.model ?? defaults.model, + }; } - return { outcome, state, summary: lastAssistantText(state) }; - }); - // The handle may be used fire-and-forget; rejections surface when awaited. - done.catch(() => undefined); - return { turnId, done }; -} -export async function runHeadlessAgent( - options: HeadlessAgentOptions, - turnRuntime?: ITurnRuntime, -): Promise { - const handle = await startHeadlessAgent(options, turnRuntime); - const result = await handle.done; - return { turnId: handle.turnId, ...result }; + const turnId = await this.turnRuntime.createTurn({ + agent: { + agentId: options.agentId, + ...(modelOverride ? { overrides: { model: modelOverride } } : {}), + }, + sessionId: null, + context: [], + input: { role: "user", content: options.message }, + config: { + autoPermission: true, + humanAvailable: false, + ...(options.maxModelCalls === undefined + ? {} + : { maxModelCalls: options.maxModelCalls }), + }, + }); + + const execution = this.turnRuntime.advanceTurn(turnId, undefined, { + signal: options.signal, + }); + const done = execution.outcome.then(async (outcome) => { + const state = reduceTurn( + (await this.turnRuntime.getTurn(turnId)).events, + ); + if (options.throwOnError && outcome.status !== "completed") { + throw new HeadlessRunError( + outcome.status === "failed" + ? outcome.error + : `turn ${outcome.status}`, + turnId, + outcome, + ); + } + return { outcome, state, summary: lastAssistantText(state) }; + }); + // The handle may be used fire-and-forget; rejections surface when awaited. + done.catch(() => undefined); + return { turnId, done }; + } + + async run( + options: HeadlessAgentOptions, + ): Promise { + const handle = await this.start(options); + const result = await handle.done; + return { turnId: handle.turnId, ...result }; + } } diff --git a/apps/x/packages/core/src/background-tasks/runner.ts b/apps/x/packages/core/src/background-tasks/runner.ts index 98f56d90..700f203a 100644 --- a/apps/x/packages/core/src/background-tasks/runner.ts +++ b/apps/x/packages/core/src/background-tasks/runner.ts @@ -2,7 +2,7 @@ import type { BackgroundTask, BackgroundTaskTriggerType } from '@x/shared/dist/b import { PrefixLogger } from '@x/shared/dist/prefix-logger.js'; import { fetchTask, patchTask, prependRunId } from './fileops.js'; import { getBackgroundTaskAgentModel } from '../models/defaults.js'; -import { startHeadlessAgent } from '../agents/headless.js'; +import { startHeadlessAgent } from '../agents/headless-app.js'; import { buildTriggerBlock } from '../agents/build-trigger-block.js'; import { backgroundTaskBus } from './bus.js'; import { withUseCase } from '../analytics/use_case.js'; diff --git a/apps/x/packages/core/src/di/container.ts b/apps/x/packages/core/src/di/container.ts index e9be949f..3455f1b7 100644 --- a/apps/x/packages/core/src/di/container.ts +++ b/apps/x/packages/core/src/di/container.ts @@ -49,6 +49,14 @@ import type { ISessionRepo } from "../sessions/repo.js"; import { EmitterSessionBus, type ISessionBus } from "../sessions/bus.js"; import { SessionsImpl } from "../sessions/sessions.js"; import type { ISessions } from "../sessions/api.js"; +import { + DefaultModelResolver, + type IDefaultModelResolver, +} from "../models/default-model-resolver.js"; +import { + HeadlessAgentRunner, + type IHeadlessAgentRunner, +} from "../agents/headless.js"; const container = createContainer({ injectionMode: InjectionMode.PROXY, @@ -115,6 +123,10 @@ container.register({ sessionRepo: asClass(FSSessionRepo).singleton(), sessionBus: asClass(EmitterSessionBus).singleton(), sessions: asClass(SessionsImpl).singleton(), + defaultModelResolver: + asClass(DefaultModelResolver).singleton(), + headlessAgentRunner: + asClass(HeadlessAgentRunner).singleton(), }); export default container; diff --git a/apps/x/packages/core/src/knowledge/agent_notes.ts b/apps/x/packages/core/src/knowledge/agent_notes.ts index 8d0b0ab1..091b4d78 100644 --- a/apps/x/packages/core/src/knowledge/agent_notes.ts +++ b/apps/x/packages/core/src/knowledge/agent_notes.ts @@ -2,7 +2,7 @@ import fs from 'fs'; import path from 'path'; import { google } from 'googleapis'; import { WorkDir } from '../config/config.js'; -import { runHeadlessAgent } from '../agents/headless.js'; +import { runHeadlessAgent } from '../agents/headless-app.js'; import { getKgModel } from '../models/defaults.js'; import { getErrorDetails } from '../agents/utils.js'; import { serviceLogger } from '../services/service_logger.js'; diff --git a/apps/x/packages/core/src/knowledge/build_graph.ts b/apps/x/packages/core/src/knowledge/build_graph.ts index f0c8836a..03dcef73 100644 --- a/apps/x/packages/core/src/knowledge/build_graph.ts +++ b/apps/x/packages/core/src/knowledge/build_graph.ts @@ -2,7 +2,7 @@ import fs from 'fs'; import path from 'path'; import { WorkDir } from '../config/config.js'; import { getKgModel } from '../models/defaults.js'; -import { runHeadlessAgent, toolInputPaths } from '../agents/headless.js'; +import { runHeadlessAgent, toolInputPaths } from '../agents/headless-app.js'; import { getErrorDetails } from '../agents/utils.js'; import { serviceLogger, type ServiceRunContext } from '../services/service_logger.js'; import { diff --git a/apps/x/packages/core/src/knowledge/inline_tasks.ts b/apps/x/packages/core/src/knowledge/inline_tasks.ts index 7bf92364..f24972ed 100644 --- a/apps/x/packages/core/src/knowledge/inline_tasks.ts +++ b/apps/x/packages/core/src/knowledge/inline_tasks.ts @@ -3,7 +3,7 @@ import path from 'path'; import { CronExpressionParser } from 'cron-parser'; import { generateText } from 'ai'; import { WorkDir } from '../config/config.js'; -import { runHeadlessAgent } from '../agents/headless.js'; +import { runHeadlessAgent } from '../agents/headless-app.js'; import { getKgModel } from '../models/defaults.js'; import container from '../di/container.js'; import type { IModelConfigRepo } from '../models/repo.js'; diff --git a/apps/x/packages/core/src/knowledge/label_emails.ts b/apps/x/packages/core/src/knowledge/label_emails.ts index e91732cf..db462d93 100644 --- a/apps/x/packages/core/src/knowledge/label_emails.ts +++ b/apps/x/packages/core/src/knowledge/label_emails.ts @@ -1,7 +1,7 @@ import fs from 'fs'; import path from 'path'; import { WorkDir } from '../config/config.js'; -import { runHeadlessAgent, toolInputPaths } from '../agents/headless.js'; +import { runHeadlessAgent, toolInputPaths } from '../agents/headless-app.js'; import { getKgModel } from '../models/defaults.js'; import { getErrorDetails } from '../agents/utils.js'; import { serviceLogger } from '../services/service_logger.js'; diff --git a/apps/x/packages/core/src/knowledge/live-note/runner.ts b/apps/x/packages/core/src/knowledge/live-note/runner.ts index 64b1f2b8..61b2b0b0 100644 --- a/apps/x/packages/core/src/knowledge/live-note/runner.ts +++ b/apps/x/packages/core/src/knowledge/live-note/runner.ts @@ -1,7 +1,7 @@ import type { LiveNote, LiveNoteTriggerType } from '@x/shared/dist/live-note.js'; import { fetchLiveNote, patchLiveNote, readNoteBody } from './fileops.js'; import { getLiveNoteAgentModel } from '../../models/defaults.js'; -import { startHeadlessAgent } from '../../agents/headless.js'; +import { startHeadlessAgent } from '../../agents/headless-app.js'; import { withUseCase } from '../../analytics/use_case.js'; import { buildTriggerBlock } from '../../agents/build-trigger-block.js'; import { liveNoteBus } from './bus.js'; diff --git a/apps/x/packages/core/src/knowledge/tag_notes.ts b/apps/x/packages/core/src/knowledge/tag_notes.ts index a1d36bcb..70d451f4 100644 --- a/apps/x/packages/core/src/knowledge/tag_notes.ts +++ b/apps/x/packages/core/src/knowledge/tag_notes.ts @@ -1,7 +1,7 @@ import fs from 'fs'; import path from 'path'; import { WorkDir } from '../config/config.js'; -import { runHeadlessAgent, toolInputPaths } from '../agents/headless.js'; +import { runHeadlessAgent, toolInputPaths } from '../agents/headless-app.js'; import { getKgModel } from '../models/defaults.js'; import { getErrorDetails } from '../agents/utils.js'; import { serviceLogger } from '../services/service_logger.js'; diff --git a/apps/x/packages/core/src/models/default-model-resolver.ts b/apps/x/packages/core/src/models/default-model-resolver.ts new file mode 100644 index 00000000..dff99247 --- /dev/null +++ b/apps/x/packages/core/src/models/default-model-resolver.ts @@ -0,0 +1,11 @@ +import { getDefaultModelAndProvider } from "./defaults.js"; + +export interface IDefaultModelResolver { + resolve(): Promise<{ model: string; provider: string }>; +} + +export class DefaultModelResolver implements IDefaultModelResolver { + resolve(): Promise<{ model: string; provider: string }> { + return getDefaultModelAndProvider(); + } +} diff --git a/apps/x/packages/core/src/pre_built/runner.ts b/apps/x/packages/core/src/pre_built/runner.ts index 3be9bcaf..19414cc3 100644 --- a/apps/x/packages/core/src/pre_built/runner.ts +++ b/apps/x/packages/core/src/pre_built/runner.ts @@ -1,7 +1,7 @@ import fs from 'fs'; import path from 'path'; import { WorkDir } from '../config/config.js'; -import { runHeadlessAgent } from '../agents/headless.js'; +import { runHeadlessAgent } from '../agents/headless-app.js'; import { getKgModel } from '../models/defaults.js'; import { loadConfig,