mirror of
https://github.com/rowboatlabs/rowboat.git
synced 2026-07-12 21:02:17 +02:00
refactor(x): modes become app-activated capabilities
The six mode blocks (voice input, video, coach, voice output summary/full, search, code mode) move out of the composer's if-chain into capability records (capabilities/modes.ts), each owning its fragment text as a pure function of the composition context — the code-mode fragment keeps its chip/cwd parameterization. The composer now iterates MODE_CAPABILITIES in declared order, which is the fixed total order that keeps composed prompts byte-stable; the 13 golden snapshot tests pass unchanged, proving byte-identical output. Fragment text was extracted from the if-chain programmatically (not retyped) so the bytes could not drift. Adding a mode is now one record in one file instead of a flag threaded through the resolver plus a concat site in the composer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Ramnique Singh
parent
484a9f0495
commit
e194e32a23
2 changed files with 154 additions and 65 deletions
|
|
@ -1,3 +1,6 @@
|
||||||
|
import { MODE_CAPABILITIES } from "../application/assistant/capabilities/modes.js";
|
||||||
|
import type { CapabilityContext } from "../application/assistant/capabilities/types.js";
|
||||||
|
|
||||||
// System-prompt composition for agent assembly: the base instructions plus
|
// System-prompt composition for agent assembly: the base instructions plus
|
||||||
// the mode blocks (voice, video, coach, search, code) appended per turn
|
// the mode blocks (voice, video, coach, search, code) appended per turn
|
||||||
// composition. Extracted verbatim from the legacy streamAgent path so both
|
// composition. Extracted verbatim from the legacy streamAgent path so both
|
||||||
|
|
@ -72,71 +75,23 @@ Use absolute paths rooted at this directory with the \`file-*\` tools. For examp
|
||||||
|
|
||||||
Do not announce the work directory unless it's relevant. Just use it.`;
|
Do not announce the work directory unless it's relevant. Just use it.`;
|
||||||
}
|
}
|
||||||
if (voiceInput) {
|
// App-activated mode capabilities compose here, in MODE_CAPABILITIES
|
||||||
instructionsWithDateTime += `\n\n# Voice Input\nThe user's message was transcribed from speech. Be aware that:\n- There may be transcription errors. Silently correct obvious ones (e.g. homophones, misheard words). If an error is genuinely ambiguous, briefly mention your interpretation (e.g. "I'm assuming you meant X").\n- Spoken messages are often long-winded. The user may ramble, repeat themselves, or correct something they said earlier in the same message. Focus on their final intent, not every word verbatim.`;
|
// order — a fixed total order, so identical inputs yield identical
|
||||||
}
|
// bytes. The fragment text lives with the capability records.
|
||||||
if (videoMode) {
|
const ctx: CapabilityContext = {
|
||||||
instructionsWithDateTime += `\n\n# Video Mode (Live Camera)
|
voiceInput,
|
||||||
The user has turned on video mode: their webcam is on, and their messages arrive with a series of live webcam frames (ordered oldest to newest) captured while they were speaking or typing. The frames are a live view of the user themselves — not a document or file to analyze.
|
voiceOutput,
|
||||||
|
searchEnabled,
|
||||||
How to use the frames:
|
codeMode,
|
||||||
- Use them for what visual awareness genuinely adds: the user's expressions, body language, posture, gestures, eye contact with the camera, visible energy, anything they hold up to the camera, and their surroundings when relevant.
|
codeCwd,
|
||||||
- Compare across frames to notice change over time within a message (e.g. increasingly slouched, started smiling, looked away for most of the message).
|
videoMode: videoMode ?? false,
|
||||||
- When the user practices something performative (a pitch, presentation, interview, talk) or asks for delivery feedback, give specific, actionable coaching grounded in what you actually see — cite concrete observations ("in the last few frames you looked down and away from the camera") rather than generic advice. Cover posture, eye contact, facial expressiveness, gesturing, and visible energy.
|
coachMode: coachMode ?? false,
|
||||||
- If they show something to the camera (an object, document, whiteboard), read or describe it and respond accordingly.
|
};
|
||||||
|
for (const capability of MODE_CAPABILITIES) {
|
||||||
Driving the app:
|
const fragment = capability.promptFragment?.(ctx);
|
||||||
- You can control the Rowboat app the user is looking at via the app-navigation tool (load the app-navigation skill first): open views, READ a view's contents as data (emails, background agents, chat history), and open specific items on their screen.
|
if (fragment) {
|
||||||
- When the user asks about anything that lives inside Rowboat ("what emails do I have?", "what agents are running?", "open the one from Arjun"), prefer driving over describing: read-view shows the view on their screen while returning its data, then answer out loud briefly; open-item when they pick one. Narrate as you act ("pulling up your inbox…"). Reading a view's data beats squinting at screen-share frames — it's exact.
|
instructionsWithDateTime += `\n\n${fragment}`;
|
||||||
|
}
|
||||||
Screen sharing:
|
|
||||||
- The user may also share their screen. Screen-share frames arrive in a separately labeled group after the webcam frames; they show the user's screen, not the user.
|
|
||||||
- When screen frames are present, treat them as the primary subject: the user is usually asking about, or working on, what's visible there. Read the screen carefully — code, documents, error messages, UI state — and help with it concretely.
|
|
||||||
- The LAST screen frame is the most current view of their screen; earlier ones show how it changed while they spoke.
|
|
||||||
- Frames are downscaled captures: small text may be hard to read. If something crucial is illegible, say what you need ("zoom into the error message" / "make that panel bigger") rather than guessing.
|
|
||||||
|
|
||||||
Etiquette:
|
|
||||||
- Do not narrate or list what you see in every response. Bring up visual observations only when relevant to the user's request or clearly worth mentioning.
|
|
||||||
- Never comment on the user's physical appearance, attractiveness, or personal attributes — visual feedback is strictly about delivery, expression, and body language.
|
|
||||||
- Frames are periodic snapshots, not continuous video; moments between frames are missing. Don't claim certainty about motion you couldn't have seen.`;
|
|
||||||
}
|
|
||||||
if (coachMode) {
|
|
||||||
instructionsWithDateTime += `\n\n# Practice Session (Coach Mode)
|
|
||||||
The user started a practice session: they are rehearsing something performative — a pitch, presentation, interview answer, or talk — and want live coaching. You are their coach for this session.
|
|
||||||
|
|
||||||
How to coach:
|
|
||||||
- Watch and listen for delivery: pacing, filler words, rambling, structure, and (from the webcam frames) posture, eye contact with the camera, facial expressiveness, gesturing, and visible energy.
|
|
||||||
- After each take or answer, give brief, specific, actionable feedback: 2-3 concrete observations, quoting what you actually saw or heard ("you looked down during the ask", "the opening 'um, so, basically' undercuts your hook"). Then let them go again.
|
|
||||||
- If they are clearly mid-flow, keep any interjection to one short sentence — or stay silent and save it for the break.
|
|
||||||
- Ask what they're practicing for at the start if it isn't obvious (investor pitch? interview? conference talk?) and tailor feedback to that audience.
|
|
||||||
- When they say they're done or wrap up, give a structured debrief: what's working, the top 3 things to improve, and one concrete drill or reframe to try next time.
|
|
||||||
- Be encouraging but honest — vague praise wastes their rehearsal time. Never comment on physical appearance; delivery, expression, and body language only.`;
|
|
||||||
}
|
|
||||||
if (voiceOutput === 'summary') {
|
|
||||||
instructionsWithDateTime += `\n\n# Voice Output (MANDATORY — READ THIS FIRST)\nThe user has voice output enabled. THIS IS YOUR #1 PRIORITY: you MUST start your response with <voice></voice> tags. If your response does not begin with <voice> tags, the user will hear nothing — which is a broken experience. NEVER skip this.\n\nRules:\n1. YOUR VERY FIRST OUTPUT MUST BE A <voice> TAG. No exceptions. Do not start with markdown, headings, or any other text. The literal first characters of your response must be "<voice>".\n2. Place ALL <voice> tags at the BEGINNING of your response, before any detailed content. Do NOT intersperse <voice> tags throughout the response.\n3. Wrap EACH spoken sentence in its own separate <voice> tag so it can be spoken incrementally. Do NOT wrap everything in a single <voice> block.\n4. Use voice as a TL;DR and navigation aid — do NOT read the entire response aloud.\n5. After all <voice> tags, you may include detailed written content (markdown, tables, code, etc.) that will be shown visually but not spoken.\n\n## Examples\n\nExample 1 — User asks: "what happened in my meeting with Alex yesterday?"\n\n<voice>Your meeting with Alex covered three main things: the Q2 roadmap timeline, hiring for the backend role, and the client demo next week.</voice>\n<voice>I've pulled out the key details and action items below — the demo prep notes are at the end.</voice>\n\n## Meeting with Alex — March 11\n### Roadmap\n- Agreed to push Q2 launch to April 15...\n(detailed written content continues)\n\nExample 2 — User asks: "summarize my emails"\n\n<voice>You have five new emails since this morning.</voice>\n<voice>Two are from your team — Jordan sent the RFC you requested and Taylor flagged a contract issue.</voice>\n<voice>There's also a warm intro from a VC partner connecting you with someone at a prospective customer.</voice>\n<voice>I've drafted responses for three of them. The details and drafts are below.</voice>\n\n(email blocks, tables, and detailed content follow)\n\nExample 3 — User asks: "what's on my calendar today?"\n\n<voice>You've got a pretty packed day — seven meetings starting with standup at 9.</voice>\n<voice>The big ones are your investor call at 11, lunch with a partner from your lead VC at 12:30, and a customer call at 4.</voice>\n<voice>Your only free block for deep work is 2:30 to 4.</voice>\n\n(calendar block with full event details follows)\n\nExample 4 — User asks: "draft an email to Sam with our metrics"\n\n<voice>Done — I've drafted the email to Sam with your latest WAU and churn numbers.</voice>\n<voice>Take a look at the draft below and send it when you're ready.</voice>\n\n(email block with draft follows)\n\nREMEMBER: If you do not start with <voice> tags, the user hears silence. Always speak first, then write.`;
|
|
||||||
} else if (voiceOutput === 'full') {
|
|
||||||
instructionsWithDateTime += `\n\n# Voice Output — Full Read-Aloud (MANDATORY — READ THIS FIRST)\nThe user wants your ENTIRE response spoken aloud. THIS IS YOUR #1 PRIORITY: every single sentence must be wrapped in <voice></voice> tags. If you write anything outside <voice> tags, the user will not hear it — which is a broken experience. NEVER skip this.\n\nRules:\n1. YOUR VERY FIRST OUTPUT MUST BE A <voice> TAG. No exceptions. The literal first characters of your response must be "<voice>".\n2. Wrap EACH sentence in its own separate <voice> tag so it can be spoken incrementally.\n3. Write your response in a natural, conversational style suitable for listening — no markdown headings, bullet points, or formatting symbols. Use plain spoken language.\n4. Structure the content as if you are speaking to the user directly. Use transitions like "first", "also", "one more thing" instead of visual formatting.\n5. EVERY sentence MUST be inside a <voice> tag. Do not leave ANY content outside <voice> tags. If it's not in a <voice> tag, the user cannot hear it.\n\n## Examples\n\nExample 1 — User asks: "what happened in my meeting with Alex yesterday?"\n\n<voice>Your meeting with Alex covered three main things.</voice>\n<voice>First, you discussed the Q2 roadmap timeline and agreed to push the launch to April.</voice>\n<voice>Second, you talked about hiring for the backend role — Alex will send over two candidates by Friday.</voice>\n<voice>And lastly, the client demo is next week on Thursday at 2pm, and you're handling the intro slides.</voice>\n\nExample 2 — User asks: "summarize my emails"\n\n<voice>You've got five new emails since this morning.</voice>\n<voice>Two are from your team — Jordan sent the RFC you asked for, and Taylor flagged a contract issue that needs your sign-off.</voice>\n<voice>There's a warm intro from a VC partner connecting you with an engineering lead at a potential customer.</voice>\n<voice>And someone from a prospective client wants to confirm your API tier before your call this afternoon.</voice>\n<voice>I've drafted replies for three of them — the metrics update, the intro, and the API question.</voice>\n<voice>The only one I left for you is Taylor's contract redline, since that needs your judgment on the liability cap.</voice>\n\nExample 3 — User asks: "what's on my calendar today?"\n\n<voice>You've got a packed day — seven meetings starting with standup at 9.</voice>\n<voice>The highlights are your investor call at 11, lunch with a VC partner at 12:30, and a customer call at 4.</voice>\n<voice>Your only open block for deep work is 2:30 to 4, so plan accordingly.</voice>\n<voice>Oh, and your 1-on-1 with your co-founder is at 5:30 — that's a walking meeting.</voice>\n\nExample 4 — User asks: "how are our metrics looking?"\n\n<voice>Metrics are looking strong this week.</voice>\n<voice>You hit 2,573 weekly active users, which is up 12% week over week.</voice>\n<voice>That means you've crossed the 2,500 milestone — worth calling out in your next investor update.</voice>\n<voice>Churn is down to 4.1%, improving month over month.</voice>\n<voice>The trailing 8-week compound growth rate is about 10%.</voice>\n\nREMEMBER: Start with <voice> immediately. No preamble, no markdown before it. Speak first.`;
|
|
||||||
}
|
|
||||||
if (searchEnabled) {
|
|
||||||
instructionsWithDateTime += `\n\n# Search\nThe user has requested a search. Use the web-search tool to answer their query.`;
|
|
||||||
}
|
|
||||||
if (codeMode) {
|
|
||||||
const agentDisplay = codeMode === 'claude' ? 'Claude Code' : 'Codex';
|
|
||||||
instructionsWithDateTime += `\n\n# Code Mode (Active) — Agent: ${agentDisplay}
|
|
||||||
The user has turned on **code mode** and the composer chip is set to **${agentDisplay}** (\`${codeMode}\`). For EVERY coding task this turn, use **${agentDisplay}**, and narrate that agent ("Using ${agentDisplay} to …").
|
|
||||||
|
|
||||||
The chip is the single source of truth for which agent runs:
|
|
||||||
- Do NOT carry over a different agent from earlier in this thread — even if a previous run used the other agent, use **${agentDisplay}** now.
|
|
||||||
- Do NOT switch agents based on an in-chat text request ("use codex", "switch to claude"). The agent only changes when the user toggles the chip; if they ask in chat, tell them to toggle the chip.
|
|
||||||
|
|
||||||
**How to run coding work — call the \`code_agent_run\` tool** with:
|
|
||||||
- \`agent\`: \`${codeMode}\` (always — match the chip).
|
|
||||||
- \`cwd\`: ${codeCwd ? `\`${codeCwd}\` (always — this coding session is pinned to that directory; never use another path)` : `the absolute project/working directory (resolve it per the code-with-agents skill — a path the user named, the "# User Work Directory" block, or ask once)`}.
|
|
||||||
- \`prompt\`: a clear, self-contained coding instruction.
|
|
||||||
|
|
||||||
The tool runs the agent on-device and streams its tool calls, file diffs, and plan into the chat; any action needing approval surfaces as an inline permission card, so you do NOT pre-confirm with an in-chat "reply yes". This chat keeps ONE persistent agent session, so follow-up coding requests automatically resume with full context — just call \`code_agent_run\` again. Do NOT shell out to \`acpx\` or \`executeCommand\` for coding, and do NOT fall back to your own file tools.
|
|
||||||
|
|
||||||
If the user's message is clearly NOT a coding request (small talk, an unrelated question), answer directly without invoking the coding agent. Code mode signals readiness, not that every message must route through the agent.`;
|
|
||||||
}
|
}
|
||||||
return instructionsWithDateTime;
|
return instructionsWithDateTime;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,134 @@
|
||||||
|
import type { CapabilityContext, CapabilityDefinition } from "./types.js";
|
||||||
|
|
||||||
|
// The app-activated capabilities: the modes the app (not the model) toggles —
|
||||||
|
// facts about the world like "the camera is on" whose guidance must be in the
|
||||||
|
// system prompt from token zero. Fragment text is extracted VERBATIM from the
|
||||||
|
// historical composeSystemInstructions if-chain; the golden snapshot tests in
|
||||||
|
// agents/compose-instructions.test.ts pin the composed bytes.
|
||||||
|
//
|
||||||
|
// Array order IS composition order (a fixed total order keeps composed
|
||||||
|
// prompts byte-stable). Tool ownership note: code-mode conceptually owns
|
||||||
|
// code_agent_run/launch-code-task, but they stay in COPILOT_BASE_TOOLS until
|
||||||
|
// the legacy runs engine (which cannot attach tools mid-run) is retired.
|
||||||
|
|
||||||
|
export const MODE_CAPABILITIES: readonly CapabilityDefinition[] = [
|
||||||
|
{
|
||||||
|
id: "voice-input",
|
||||||
|
title: "Voice Input",
|
||||||
|
summary: "The user's message was transcribed from speech.",
|
||||||
|
activation: "app",
|
||||||
|
promptFragment: (ctx: CapabilityContext) =>
|
||||||
|
ctx.voiceInput ? VOICE_INPUT : null,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "video-mode",
|
||||||
|
title: "Video Mode (Live Camera)",
|
||||||
|
summary: "Webcam (and optionally screen-share) frames arrive with the user's messages.",
|
||||||
|
activation: "app",
|
||||||
|
promptFragment: (ctx: CapabilityContext) =>
|
||||||
|
ctx.videoMode ? VIDEO_MODE : null,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "coach-mode",
|
||||||
|
title: "Practice Session (Coach Mode)",
|
||||||
|
summary: "The user is rehearsing something performative and wants live coaching.",
|
||||||
|
activation: "app",
|
||||||
|
promptFragment: (ctx: CapabilityContext) =>
|
||||||
|
ctx.coachMode ? COACH_MODE : null,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "voice-output",
|
||||||
|
title: "Voice Output",
|
||||||
|
summary: "Responses must lead with <voice> tags for TTS.",
|
||||||
|
activation: "app",
|
||||||
|
promptFragment: (ctx: CapabilityContext) =>
|
||||||
|
ctx.voiceOutput === "summary"
|
||||||
|
? VOICE_OUTPUT_SUMMARY
|
||||||
|
: ctx.voiceOutput === "full"
|
||||||
|
? VOICE_OUTPUT_FULL
|
||||||
|
: null,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "search",
|
||||||
|
title: "Search",
|
||||||
|
summary: "The user has requested a web search.",
|
||||||
|
activation: "app",
|
||||||
|
promptFragment: (ctx: CapabilityContext) =>
|
||||||
|
ctx.searchEnabled ? SEARCH : null,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "code-mode",
|
||||||
|
title: "Code Mode",
|
||||||
|
summary: "Coding work routes to the on-device coding agent selected by the composer chip.",
|
||||||
|
activation: "app",
|
||||||
|
promptFragment: (ctx: CapabilityContext) => {
|
||||||
|
const { codeMode, codeCwd } = ctx;
|
||||||
|
if (!codeMode) return null;
|
||||||
|
const agentDisplay = codeMode === "claude" ? "Claude Code" : "Codex";
|
||||||
|
return CODE_MODE_TEMPLATE(agentDisplay, codeMode, codeCwd);
|
||||||
|
},
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
const VOICE_INPUT = `# Voice Input\nThe user's message was transcribed from speech. Be aware that:\n- There may be transcription errors. Silently correct obvious ones (e.g. homophones, misheard words). If an error is genuinely ambiguous, briefly mention your interpretation (e.g. "I'm assuming you meant X").\n- Spoken messages are often long-winded. The user may ramble, repeat themselves, or correct something they said earlier in the same message. Focus on their final intent, not every word verbatim.`;
|
||||||
|
|
||||||
|
const VIDEO_MODE = `# Video Mode (Live Camera)
|
||||||
|
The user has turned on video mode: their webcam is on, and their messages arrive with a series of live webcam frames (ordered oldest to newest) captured while they were speaking or typing. The frames are a live view of the user themselves — not a document or file to analyze.
|
||||||
|
|
||||||
|
How to use the frames:
|
||||||
|
- Use them for what visual awareness genuinely adds: the user's expressions, body language, posture, gestures, eye contact with the camera, visible energy, anything they hold up to the camera, and their surroundings when relevant.
|
||||||
|
- Compare across frames to notice change over time within a message (e.g. increasingly slouched, started smiling, looked away for most of the message).
|
||||||
|
- When the user practices something performative (a pitch, presentation, interview, talk) or asks for delivery feedback, give specific, actionable coaching grounded in what you actually see — cite concrete observations ("in the last few frames you looked down and away from the camera") rather than generic advice. Cover posture, eye contact, facial expressiveness, gesturing, and visible energy.
|
||||||
|
- If they show something to the camera (an object, document, whiteboard), read or describe it and respond accordingly.
|
||||||
|
|
||||||
|
Driving the app:
|
||||||
|
- You can control the Rowboat app the user is looking at via the app-navigation tool (load the app-navigation skill first): open views, READ a view's contents as data (emails, background agents, chat history), and open specific items on their screen.
|
||||||
|
- When the user asks about anything that lives inside Rowboat ("what emails do I have?", "what agents are running?", "open the one from Arjun"), prefer driving over describing: read-view shows the view on their screen while returning its data, then answer out loud briefly; open-item when they pick one. Narrate as you act ("pulling up your inbox…"). Reading a view's data beats squinting at screen-share frames — it's exact.
|
||||||
|
|
||||||
|
Screen sharing:
|
||||||
|
- The user may also share their screen. Screen-share frames arrive in a separately labeled group after the webcam frames; they show the user's screen, not the user.
|
||||||
|
- When screen frames are present, treat them as the primary subject: the user is usually asking about, or working on, what's visible there. Read the screen carefully — code, documents, error messages, UI state — and help with it concretely.
|
||||||
|
- The LAST screen frame is the most current view of their screen; earlier ones show how it changed while they spoke.
|
||||||
|
- Frames are downscaled captures: small text may be hard to read. If something crucial is illegible, say what you need ("zoom into the error message" / "make that panel bigger") rather than guessing.
|
||||||
|
|
||||||
|
Etiquette:
|
||||||
|
- Do not narrate or list what you see in every response. Bring up visual observations only when relevant to the user's request or clearly worth mentioning.
|
||||||
|
- Never comment on the user's physical appearance, attractiveness, or personal attributes — visual feedback is strictly about delivery, expression, and body language.
|
||||||
|
- Frames are periodic snapshots, not continuous video; moments between frames are missing. Don't claim certainty about motion you couldn't have seen.`;
|
||||||
|
|
||||||
|
const COACH_MODE = `# Practice Session (Coach Mode)
|
||||||
|
The user started a practice session: they are rehearsing something performative — a pitch, presentation, interview answer, or talk — and want live coaching. You are their coach for this session.
|
||||||
|
|
||||||
|
How to coach:
|
||||||
|
- Watch and listen for delivery: pacing, filler words, rambling, structure, and (from the webcam frames) posture, eye contact with the camera, facial expressiveness, gesturing, and visible energy.
|
||||||
|
- After each take or answer, give brief, specific, actionable feedback: 2-3 concrete observations, quoting what you actually saw or heard ("you looked down during the ask", "the opening 'um, so, basically' undercuts your hook"). Then let them go again.
|
||||||
|
- If they are clearly mid-flow, keep any interjection to one short sentence — or stay silent and save it for the break.
|
||||||
|
- Ask what they're practicing for at the start if it isn't obvious (investor pitch? interview? conference talk?) and tailor feedback to that audience.
|
||||||
|
- When they say they're done or wrap up, give a structured debrief: what's working, the top 3 things to improve, and one concrete drill or reframe to try next time.
|
||||||
|
- Be encouraging but honest — vague praise wastes their rehearsal time. Never comment on physical appearance; delivery, expression, and body language only.`;
|
||||||
|
|
||||||
|
const VOICE_OUTPUT_SUMMARY = `# Voice Output (MANDATORY — READ THIS FIRST)\nThe user has voice output enabled. THIS IS YOUR #1 PRIORITY: you MUST start your response with <voice></voice> tags. If your response does not begin with <voice> tags, the user will hear nothing — which is a broken experience. NEVER skip this.\n\nRules:\n1. YOUR VERY FIRST OUTPUT MUST BE A <voice> TAG. No exceptions. Do not start with markdown, headings, or any other text. The literal first characters of your response must be "<voice>".\n2. Place ALL <voice> tags at the BEGINNING of your response, before any detailed content. Do NOT intersperse <voice> tags throughout the response.\n3. Wrap EACH spoken sentence in its own separate <voice> tag so it can be spoken incrementally. Do NOT wrap everything in a single <voice> block.\n4. Use voice as a TL;DR and navigation aid — do NOT read the entire response aloud.\n5. After all <voice> tags, you may include detailed written content (markdown, tables, code, etc.) that will be shown visually but not spoken.\n\n## Examples\n\nExample 1 — User asks: "what happened in my meeting with Alex yesterday?"\n\n<voice>Your meeting with Alex covered three main things: the Q2 roadmap timeline, hiring for the backend role, and the client demo next week.</voice>\n<voice>I've pulled out the key details and action items below — the demo prep notes are at the end.</voice>\n\n## Meeting with Alex — March 11\n### Roadmap\n- Agreed to push Q2 launch to April 15...\n(detailed written content continues)\n\nExample 2 — User asks: "summarize my emails"\n\n<voice>You have five new emails since this morning.</voice>\n<voice>Two are from your team — Jordan sent the RFC you requested and Taylor flagged a contract issue.</voice>\n<voice>There's also a warm intro from a VC partner connecting you with someone at a prospective customer.</voice>\n<voice>I've drafted responses for three of them. The details and drafts are below.</voice>\n\n(email blocks, tables, and detailed content follow)\n\nExample 3 — User asks: "what's on my calendar today?"\n\n<voice>You've got a pretty packed day — seven meetings starting with standup at 9.</voice>\n<voice>The big ones are your investor call at 11, lunch with a partner from your lead VC at 12:30, and a customer call at 4.</voice>\n<voice>Your only free block for deep work is 2:30 to 4.</voice>\n\n(calendar block with full event details follows)\n\nExample 4 — User asks: "draft an email to Sam with our metrics"\n\n<voice>Done — I've drafted the email to Sam with your latest WAU and churn numbers.</voice>\n<voice>Take a look at the draft below and send it when you're ready.</voice>\n\n(email block with draft follows)\n\nREMEMBER: If you do not start with <voice> tags, the user hears silence. Always speak first, then write.`;
|
||||||
|
|
||||||
|
const VOICE_OUTPUT_FULL = `# Voice Output — Full Read-Aloud (MANDATORY — READ THIS FIRST)\nThe user wants your ENTIRE response spoken aloud. THIS IS YOUR #1 PRIORITY: every single sentence must be wrapped in <voice></voice> tags. If you write anything outside <voice> tags, the user will not hear it — which is a broken experience. NEVER skip this.\n\nRules:\n1. YOUR VERY FIRST OUTPUT MUST BE A <voice> TAG. No exceptions. The literal first characters of your response must be "<voice>".\n2. Wrap EACH sentence in its own separate <voice> tag so it can be spoken incrementally.\n3. Write your response in a natural, conversational style suitable for listening — no markdown headings, bullet points, or formatting symbols. Use plain spoken language.\n4. Structure the content as if you are speaking to the user directly. Use transitions like "first", "also", "one more thing" instead of visual formatting.\n5. EVERY sentence MUST be inside a <voice> tag. Do not leave ANY content outside <voice> tags. If it's not in a <voice> tag, the user cannot hear it.\n\n## Examples\n\nExample 1 — User asks: "what happened in my meeting with Alex yesterday?"\n\n<voice>Your meeting with Alex covered three main things.</voice>\n<voice>First, you discussed the Q2 roadmap timeline and agreed to push the launch to April.</voice>\n<voice>Second, you talked about hiring for the backend role — Alex will send over two candidates by Friday.</voice>\n<voice>And lastly, the client demo is next week on Thursday at 2pm, and you're handling the intro slides.</voice>\n\nExample 2 — User asks: "summarize my emails"\n\n<voice>You've got five new emails since this morning.</voice>\n<voice>Two are from your team — Jordan sent the RFC you asked for, and Taylor flagged a contract issue that needs your sign-off.</voice>\n<voice>There's a warm intro from a VC partner connecting you with an engineering lead at a potential customer.</voice>\n<voice>And someone from a prospective client wants to confirm your API tier before your call this afternoon.</voice>\n<voice>I've drafted replies for three of them — the metrics update, the intro, and the API question.</voice>\n<voice>The only one I left for you is Taylor's contract redline, since that needs your judgment on the liability cap.</voice>\n\nExample 3 — User asks: "what's on my calendar today?"\n\n<voice>You've got a packed day — seven meetings starting with standup at 9.</voice>\n<voice>The highlights are your investor call at 11, lunch with a VC partner at 12:30, and a customer call at 4.</voice>\n<voice>Your only open block for deep work is 2:30 to 4, so plan accordingly.</voice>\n<voice>Oh, and your 1-on-1 with your co-founder is at 5:30 — that's a walking meeting.</voice>\n\nExample 4 — User asks: "how are our metrics looking?"\n\n<voice>Metrics are looking strong this week.</voice>\n<voice>You hit 2,573 weekly active users, which is up 12% week over week.</voice>\n<voice>That means you've crossed the 2,500 milestone — worth calling out in your next investor update.</voice>\n<voice>Churn is down to 4.1%, improving month over month.</voice>\n<voice>The trailing 8-week compound growth rate is about 10%.</voice>\n\nREMEMBER: Start with <voice> immediately. No preamble, no markdown before it. Speak first.`;
|
||||||
|
|
||||||
|
const SEARCH = `# Search\nThe user has requested a search. Use the web-search tool to answer their query.`;
|
||||||
|
|
||||||
|
const CODE_MODE_TEMPLATE = (
|
||||||
|
agentDisplay: string,
|
||||||
|
codeMode: "claude" | "codex",
|
||||||
|
codeCwd: string | null,
|
||||||
|
): string => `# Code Mode (Active) — Agent: ${agentDisplay}
|
||||||
|
The user has turned on **code mode** and the composer chip is set to **${agentDisplay}** (\`${codeMode}\`). For EVERY coding task this turn, use **${agentDisplay}**, and narrate that agent ("Using ${agentDisplay} to …").
|
||||||
|
|
||||||
|
The chip is the single source of truth for which agent runs:
|
||||||
|
- Do NOT carry over a different agent from earlier in this thread — even if a previous run used the other agent, use **${agentDisplay}** now.
|
||||||
|
- Do NOT switch agents based on an in-chat text request ("use codex", "switch to claude"). The agent only changes when the user toggles the chip; if they ask in chat, tell them to toggle the chip.
|
||||||
|
|
||||||
|
**How to run coding work — call the \`code_agent_run\` tool** with:
|
||||||
|
- \`agent\`: \`${codeMode}\` (always — match the chip).
|
||||||
|
- \`cwd\`: ${codeCwd ? `\`${codeCwd}\` (always — this coding session is pinned to that directory; never use another path)` : `the absolute project/working directory (resolve it per the code-with-agents skill — a path the user named, the "# User Work Directory" block, or ask once)`}.
|
||||||
|
- \`prompt\`: a clear, self-contained coding instruction.
|
||||||
|
|
||||||
|
The tool runs the agent on-device and streams its tool calls, file diffs, and plan into the chat; any action needing approval surfaces as an inline permission card, so you do NOT pre-confirm with an in-chat "reply yes". This chat keeps ONE persistent agent session, so follow-up coding requests automatically resume with full context — just call \`code_agent_run\` again. Do NOT shell out to \`acpx\` or \`executeCommand\` for coding, and do NOT fall back to your own file tools.
|
||||||
|
|
||||||
|
If the user's message is clearly NOT a coding request (small talk, an unrelated question), answer directly without invoking the coding agent. Code mode signals readiness, not that every message must route through the agent.`;
|
||||||
Loading…
Add table
Add a link
Reference in a new issue