mirror of
https://github.com/rowboatlabs/rowboat.git
synced 2026-07-12 21:02:17 +02:00
feat(mini-apps): copilot builder — build-mini-app skill + install/data tools
Lets the copilot build a Mini App into ~/.rowboat/apps from chat. - build-mini-app skill: intent gate (confirm when ambiguous), verify Composio wiring by actually calling tools before building, writer branch (Code Mode via code-with-agents vs Copilot), agent-backed data pipeline, install - mini-app-install tool: write an app folder (manifest + dist/index.html) - mini-app-set-data tool: atomic deterministic data.json write (agent returns content; path/write handled in code) - serve canonical bridge shim at app://miniapp/__bridge__.js
This commit is contained in:
Gagan
parent
924a93fa4a
commit
4a4dcb1fa0
6 changed files with 298 additions and 1 deletions
|
|
@ -136,6 +136,86 @@ Copilot writing a new app folder + an associated background agent; the agent
|
|||
browsing via embedded browser for social feeds; copilot verifying Composio wiring
|
||||
by actually calling tools before finalizing.
|
||||
|
||||
## 2c. Remaining work (after on-disk move)
|
||||
|
||||
Done so far: surface + runtime (Phase 1), real scoped Composio bridge (Phase 2),
|
||||
apps on disk served via `app://miniapp` (Phase 2.5).
|
||||
|
||||
**Next — Copilot builder (demo target).**
|
||||
- Copilot creates an app folder in `~/.rowboat/apps/<id>/` via the `mini-apps:seed`
|
||||
install primitive (writes `manifest.json` + `dist/index.html`).
|
||||
- Copilot must **verify wiring by actually calling Composio tools** and inspecting
|
||||
the returned data before finalizing — never speculate the shape.
|
||||
- Give generated apps the bridge contract — prefer **serving a canonical shim**
|
||||
from the protocol (e.g. `app://miniapp/__bridge__.js`) over inlining.
|
||||
|
||||
**Background-agent data pipeline (deterministic).**
|
||||
- Reuse the existing bg-tasks engine; link via `manifest.agent`.
|
||||
- The agent does NOT write files. It **returns structured data validated against
|
||||
the app's data schema**; the bg-task **runner (code) atomically writes the
|
||||
app's `data.json`** (temp→rename, keep last-good on failure). Path / atomicity
|
||||
/ validation / fallback all live in code — only the *content* is LLM-driven.
|
||||
- Social feeds (Twitter/LinkedIn/Reddit): the agent browses via the embedded
|
||||
browser, curates, returns data → runner writes `data.json`.
|
||||
|
||||
**Later (roadmap).**
|
||||
- Living apps + breakage recovery (edit by chat; surface errors → fix-with-copilot).
|
||||
- Scale/polish: context-overload windowing; app notifications; design-consistency
|
||||
/ component templates (team-deferred).
|
||||
- V2: drag an app into the main workspace.
|
||||
|
||||
## 2d. Mini App builder skill — spec
|
||||
|
||||
A Copilot skill (`build-mini-app`, in `packages/core/src/application/assistant/
|
||||
skills/`) that turns a chat request into an installed app under `~/.rowboat/
|
||||
apps/<id>/`. Copilot orchestrates; the actual code-writing is delegated by the
|
||||
chat's active engine (the chip), but the on-disk artifact is identical either way.
|
||||
|
||||
**Trigger + intent gate** (like live-note/background-task signal tiers):
|
||||
- **Strong (build directly):** "make/build/create an app · mini app · dashboard
|
||||
for …", "turn this into an app".
|
||||
- **Medium (CONFIRM FIRST):** requests that could be a one-off answer or a
|
||||
recurring app — e.g. "show me my open PRs", "track competitor launches". Ask:
|
||||
"Want this as a Mini App you can reopen, or just a one-time answer?" Build only
|
||||
on yes. (Building installs a folder + maybe a bg agent + an OAuth prompt — too
|
||||
heavy to trigger on a casual question.)
|
||||
- **Anti (do NOT build):** clearly one-off lookups/questions → just answer.
|
||||
|
||||
**Flow:**
|
||||
1. **Scope the app** — derive `id` (slug), `title`, `description`, `source`, the
|
||||
Composio `scope[]`, and whether it's **agent-backed** (scheduled data) or
|
||||
**live** (calls Composio on use).
|
||||
2. **Verify wiring BEFORE building** (mandatory, engine-agnostic) — ensure the
|
||||
toolkits are connected (prompt OAuth if not), then actually call the needed
|
||||
tools (`composio-search-tools` → `composio-execute-tool`), inspect the REAL
|
||||
returned data, and derive the **data schema from actual responses** — never
|
||||
guess the shape.
|
||||
3. **Pick the writer (branch):**
|
||||
- **Code Mode active** → create the folder + a manifest skeleton, then
|
||||
`code_agent_run` with `cwd = ~/.rowboat/apps/<id>/` to author
|
||||
`dist/index.html` against the verified schema + bridge contract; it can
|
||||
iterate/test on-device.
|
||||
- **No Code Mode** → Copilot writes `dist/index.html` itself (from the app
|
||||
template + bridge shim) and installs via `mini-apps:seed`.
|
||||
4. **Bridge contract** — generated app references the canonical shim
|
||||
(`app://miniapp/__bridge__.js`) and uses `window.rowboat`
|
||||
(`getData/onData`, `isConnected/connect`, `searchTools/callAction`).
|
||||
5. **Data pipeline (if agent-backed)** — create a background task (existing
|
||||
bg-tasks engine), set `manifest.agent` to its slug. The agent **returns
|
||||
schema-validated data**; the **runner writes `data.json` deterministically**
|
||||
(temp→rename, last-good on failure). App reads it via `onData`.
|
||||
6. **Finalize** — write `manifest.json` (incl. `scope`, `agent?`), ensure
|
||||
`dist/index.html`, install → app appears in the gallery (`mini-apps:list`).
|
||||
Confirm end-to-end (open it; data loads).
|
||||
|
||||
**Infra this needs (repo side):**
|
||||
- Serve the canonical bridge shim from the protocol: `app://miniapp/__bridge__.js`
|
||||
(move the shim out of per-app inlining into served infra).
|
||||
- bg-tasks runner: when a task is app-linked, validate its structured output
|
||||
against the app schema and write that app's `data.json` (deterministic write
|
||||
mode) instead of `index.md`.
|
||||
- A `build-mini-app` skill registered in the skills catalog.
|
||||
|
||||
## 3. Phase 1 — detailed implementation
|
||||
|
||||
UI-first. Everything hand-coded; no `~/.rowboat` storage, no IPC, no agent yet.
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ import { identifyIfSignedIn } from "@x/core/dist/analytics/identify.js";
|
|||
import { initConfigs } from "@x/core/dist/config/initConfigs.js";
|
||||
import { getAgentSlackCliStatus } from "@x/core/dist/slack/agent-slack-exec.js";
|
||||
import { resolveWorkspacePath } from "@x/core/dist/workspace/workspace.js";
|
||||
import { resolveMiniAppAsset } from "./mini-apps-handler.js";
|
||||
import { resolveMiniAppAsset, MINIAPP_BRIDGE_JS } from "./mini-apps-handler.js";
|
||||
import started from "electron-squirrel-startup";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { init as initChromeSync } from "@x/core/dist/knowledge/chrome-extension/server/server.js";
|
||||
|
|
@ -173,6 +173,12 @@ function registerAppProtocol() {
|
|||
const segments = decodeURIComponent(url.pathname).replace(/^\/+/, "").split("/");
|
||||
const id = segments.shift();
|
||||
if (!id) return new Response("Not Found", { status: 404 });
|
||||
// Canonical bridge shim shared by all apps.
|
||||
if (id === "__bridge__.js") {
|
||||
return new Response(MINIAPP_BRIDGE_JS, {
|
||||
headers: { "content-type": "text/javascript; charset=utf-8" },
|
||||
});
|
||||
}
|
||||
const absPath = resolveMiniAppAsset(id, segments.join("/"));
|
||||
if (!absPath) return new Response("Forbidden", { status: 403 });
|
||||
return net.fetch(pathToFileURL(absPath).toString());
|
||||
|
|
|
|||
|
|
@ -16,6 +16,54 @@ function appDir(id: string): string {
|
|||
return path.join(APPS_DIR, id);
|
||||
}
|
||||
|
||||
/**
|
||||
* Canonical Mini App bridge shim, served at app://miniapp/__bridge__.js. Apps
|
||||
* include it with `<script src="/__bridge__.js"></script>` and code against
|
||||
* `window.rowboat`. This is the single source of truth for the bridge protocol
|
||||
* (kept in sync with the host in components/mini-app-frame.tsx + types.ts).
|
||||
*/
|
||||
export const MINIAPP_BRIDGE_JS = `
|
||||
(function () {
|
||||
var data = null, state = null;
|
||||
var dataCbs = [], stateCbs = [];
|
||||
var pending = {}, seq = 0;
|
||||
function post(msg) { parent.postMessage(msg, '*'); }
|
||||
function rpc(method, params) {
|
||||
var id = 'r' + (++seq);
|
||||
return new Promise(function (resolve, reject) {
|
||||
pending[id] = { resolve: resolve, reject: reject };
|
||||
post({ type: 'rowboat:mini-app:rpc', id: id, method: method, params: params });
|
||||
});
|
||||
}
|
||||
window.addEventListener('message', function (e) {
|
||||
var m = e.data;
|
||||
if (!m || typeof m !== 'object') return;
|
||||
if (m.type === 'rowboat:mini-app:data') {
|
||||
data = m.data;
|
||||
dataCbs.forEach(function (cb) { try { cb(data); } catch (_) {} });
|
||||
} else if (m.type === 'rowboat:mini-app:state') {
|
||||
state = m.state;
|
||||
stateCbs.forEach(function (cb) { try { cb(state); } catch (_) {} });
|
||||
} else if (m.type === 'rowboat:mini-app:rpc-result') {
|
||||
var p = pending[m.id];
|
||||
if (p) { delete pending[m.id]; if (m.ok) p.resolve(m.result); else p.reject(new Error(m.error || 'request failed')); }
|
||||
}
|
||||
});
|
||||
window.rowboat = {
|
||||
getData: function () { return data; },
|
||||
onData: function (cb) { dataCbs.push(cb); if (data !== null) { try { cb(data); } catch (_) {} } return function () { var i = dataCbs.indexOf(cb); if (i >= 0) dataCbs.splice(i, 1); }; },
|
||||
getState: function () { return state; },
|
||||
onState: function (cb) { stateCbs.push(cb); if (state !== null) { try { cb(state); } catch (_) {} } return function () { var i = stateCbs.indexOf(cb); if (i >= 0) stateCbs.splice(i, 1); }; },
|
||||
setState: function (patch) { state = Object.assign({}, state || {}, patch); post({ type: 'rowboat:mini-app:setState', patch: patch }); stateCbs.forEach(function (cb) { try { cb(state); } catch (_) {} }); },
|
||||
callAction: function (scope, tool, args) { return rpc('callAction', { scope: scope, tool: tool, args: args }); },
|
||||
searchTools: function (scope, query) { return rpc('searchTools', { scope: scope, query: query }); },
|
||||
isConnected: function (scope) { return rpc('isConnected', { scope: scope }); },
|
||||
connect: function (scope) { return rpc('connect', { scope: scope }); },
|
||||
ready: function () { post({ type: 'rowboat:mini-app:ready' }); },
|
||||
};
|
||||
})();
|
||||
`;
|
||||
|
||||
function ensureDir(dir: string): void {
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,110 @@
|
|||
import { z } from 'zod';
|
||||
import { stringify as stringifyYaml } from 'yaml';
|
||||
import { MiniAppManifest } from '@x/shared/dist/mini-app.js';
|
||||
|
||||
const manifestSchema = stringifyYaml(z.toJSONSchema(MiniAppManifest)).trimEnd();
|
||||
|
||||
export const skill = String.raw`
|
||||
# Build a Mini App
|
||||
|
||||
A *Mini App* is a small app the user opens inside Rowboat — its own UI, powered by
|
||||
their integrations and (optionally) a background agent. Apps live on disk at
|
||||
\`~/.rowboat/apps/<id>/\` and are served at \`app://miniapp/<id>/\`:
|
||||
|
||||
\`\`\`
|
||||
~/.rowboat/apps/<id>/
|
||||
manifest.json # see schema below
|
||||
dist/index.html # the app UI (self-contained), served via app://miniapp/<id>/
|
||||
data.json # data the UI reads (produced by the app's agent; optional)
|
||||
\`\`\`
|
||||
|
||||
You do NOT hand-write these files with file tools. Use **\`mini-app-install\`** to
|
||||
write the folder, and **\`create-background-task\`** for the optional agent.
|
||||
|
||||
## 0. Should this even be an app? (intent gate)
|
||||
|
||||
- **Strong — build it:** "make/build/create an app · mini app · dashboard for …",
|
||||
"turn this into an app".
|
||||
- **Medium — CONFIRM FIRST:** the request could be a one-off answer OR a recurring
|
||||
app (e.g. "show me my open PRs", "track competitor launches"). Ask once:
|
||||
*"Want this as a Mini App you can reopen, or just a one-time answer?"* Build only
|
||||
if they say app. (Building installs a folder, maybe a background agent, and may
|
||||
prompt an OAuth connection — too heavy for a casual question.)
|
||||
- **Anti — don't build:** a clear one-off lookup/question → just answer it.
|
||||
|
||||
## 1. Scope the app
|
||||
|
||||
Decide: \`id\` (kebab slug), \`title\`, \`description\`, \`source\` (e.g. "GitHub"),
|
||||
the Composio \`scope\` (toolkits it may use), and whether it is:
|
||||
- **live** — calls Composio when the user interacts (no agent), or
|
||||
- **agent-backed** — a background agent refreshes \`data.json\` on a schedule and the
|
||||
UI just reads it (keeps tokens low; best for feeds/digests/dashboards).
|
||||
|
||||
## 2. Verify the wiring BEFORE building (required — do not speculate)
|
||||
|
||||
Load the \`composio-integration\` skill. Then for each toolkit in scope:
|
||||
1. Ensure it is connected (\`composio-connect-toolkit\` → user authorizes if needed).
|
||||
2. Actually call the tools you intend to use (\`composio-search-tools\` →
|
||||
\`composio-execute-tool\`) and **inspect the real returned JSON**.
|
||||
3. Derive the app's **data shape from those real responses** — never guess field
|
||||
names. This shape is the contract between the agent (or live calls) and the UI.
|
||||
|
||||
If a toolkit doesn't support managed OAuth2 (e.g. X/Twitter), tell the user it
|
||||
can't be connected this way and pick a different integration or a browser-based
|
||||
agent.
|
||||
|
||||
## 3. Write the UI (dist/index.html)
|
||||
|
||||
The app is a self-contained HTML document. It talks to Rowboat ONLY through the
|
||||
bridge: include the shim and code against \`window.rowboat\`:
|
||||
|
||||
\`\`\`html
|
||||
<script src="/__bridge__.js"></script>
|
||||
\`\`\`
|
||||
|
||||
\`window.rowboat\` API:
|
||||
- \`getData()\` / \`onData(cb)\` — the app's data (host serves \`data.json\`). Register
|
||||
\`onData\` then call \`ready()\`.
|
||||
- \`getState()\` / \`setState(patch)\` — per-app persistent UI state.
|
||||
- \`isConnected(scope)\` / \`connect(scope)\` — connection status / start OAuth.
|
||||
- \`searchTools(scope, query)\` → [{slug,name,description}], and
|
||||
\`callAction(scope, toolSlug, args)\` → tool result (rejects on error). Scope is
|
||||
enforced against the manifest, so only declared toolkits work.
|
||||
- \`ready()\` — call once after registering callbacks to receive initial data/state.
|
||||
|
||||
Keep it dependency-free (no remote CDNs unless truly needed; the app:// origin
|
||||
allows them but offline-safe is better). Render loading / empty / error states.
|
||||
|
||||
**Who writes this HTML:**
|
||||
- If a **coding agent is active** (user toggled the Code chip), delegate authoring
|
||||
via the \`code-with-agents\` skill — run it with \`cwd\` = the app's
|
||||
\`~/.rowboat/apps/<id>/\` folder so it can iterate and test, then install.
|
||||
- Otherwise, author the HTML yourself and install it with \`mini-app-install\`.
|
||||
|
||||
## 4. Data pipeline (agent-backed apps only)
|
||||
|
||||
Create a background task with \`create-background-task\` whose instructions:
|
||||
- fetch the data via Composio (or browse via the embedded browser for social
|
||||
feeds), and
|
||||
- call **\`mini-app-set-data\`** with \`{ appId: "<id>", data: <payload> }\`.
|
||||
|
||||
The agent only RETURNS the data; \`mini-app-set-data\` writes \`data.json\`
|
||||
atomically (deterministic path + write — the agent never touches files). Set the
|
||||
manifest's \`agent\` field to the task's slug. Give the task a sensible trigger
|
||||
(cron/window) from the background-task skill.
|
||||
|
||||
## 5. Finalize
|
||||
|
||||
Call \`mini-app-install\` with the manifest + html (+ optional seed data). Then
|
||||
confirm to the user, and ideally open it so they see it populate. For agent-backed
|
||||
apps, trigger the agent once (\`run-background-task-agent\`) so \`data.json\` exists
|
||||
immediately instead of waiting for the first scheduled run.
|
||||
|
||||
## Manifest schema (manifest.json)
|
||||
|
||||
\`\`\`yaml
|
||||
` + manifestSchema + `
|
||||
\`\`\`
|
||||
`;
|
||||
|
||||
export default skill;
|
||||
|
|
@ -16,6 +16,7 @@ import composioIntegrationSkill from "./composio-integration/skill.js";
|
|||
import liveNoteSkill from "./live-note/skill.js";
|
||||
import backgroundTaskSkill from "./background-task/skill.js";
|
||||
import notifyUserSkill from "./notify-user/skill.js";
|
||||
import buildMiniAppSkill from "./build-mini-app/skill.js";
|
||||
|
||||
const CURRENT_DIR = path.dirname(fileURLToPath(import.meta.url));
|
||||
const CATALOG_PREFIX = "src/application/assistant/skills";
|
||||
|
|
@ -102,6 +103,12 @@ const definitions: SkillDefinition[] = [
|
|||
summary: "Write code, build projects, create scripts, or fix bugs by delegating to Claude Code or Codex.",
|
||||
content: codeWithAgentsSkill,
|
||||
},
|
||||
{
|
||||
id: "build-mini-app",
|
||||
title: "Build a Mini App",
|
||||
summary: "Build a Mini App the user opens inside Rowboat — its own UI powered by their integrations and an optional background agent. Use when the user asks to make/build/create an app, mini app, or dashboard; for ambiguous 'show me X' requests, confirm whether they want an app first.",
|
||||
content: buildMiniAppSkill,
|
||||
},
|
||||
{
|
||||
id: "background-task",
|
||||
title: "Background Tasks",
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@ import { WorkDir } from "../../config/config.js";
|
|||
import { composioAccountsRepo } from "../../composio/repo.js";
|
||||
import { executeAction as executeComposioAction, isConfigured as isComposioConfigured, searchTools as searchComposioTools } from "../../composio/client.js";
|
||||
import { CURATED_TOOLKITS, CURATED_TOOLKIT_SLUGS } from "@x/shared/dist/composio.js";
|
||||
import { MiniAppManifest } from "@x/shared/dist/mini-app.js";
|
||||
import { BrowserControlInputSchema, type BrowserControlInput } from "@x/shared/dist/browser-control.js";
|
||||
import { BackgroundTaskSchema, TriggersSchema } from "@x/shared/dist/background-task.js";
|
||||
import type { CodeModeManager } from "../../code-mode/acp/manager.js";
|
||||
|
|
@ -1495,6 +1496,51 @@ export const BuiltinTools: z.infer<typeof BuiltinToolsSchema> = {
|
|||
},
|
||||
isAvailable: async () => isComposioConfigured(),
|
||||
},
|
||||
'mini-app-install': {
|
||||
description: "Install or update a Mini App on disk at ~/.rowboat/apps/<id>/. Writes manifest.json + dist/index.html (and optional initial data.json). Use this to materialize an app you built — do NOT hand-write these files with file tools. The HTML must be self-contained and include the bridge via <script src=\"/__bridge__.js\"></script>, coding against window.rowboat. After install, the app shows up in the Mini Apps gallery.",
|
||||
inputSchema: z.object({
|
||||
manifest: MiniAppManifest,
|
||||
html: z.string().describe('Full self-contained HTML for dist/index.html.'),
|
||||
data: z.unknown().optional().describe('Optional initial data.json content (only written if no data.json exists yet).'),
|
||||
}),
|
||||
execute: async ({ manifest, html, data }: { manifest: z.infer<typeof MiniAppManifest>; html: string; data?: unknown }) => {
|
||||
try {
|
||||
const m = MiniAppManifest.parse(manifest);
|
||||
const dir = path.join(WorkDir, 'apps', m.id);
|
||||
const dist = path.join(dir, 'dist');
|
||||
await fs.mkdir(dist, { recursive: true });
|
||||
await fs.writeFile(path.join(dir, 'manifest.json'), JSON.stringify(m, null, 2));
|
||||
await fs.writeFile(path.join(dist, 'index.html'), html);
|
||||
if (data !== undefined) {
|
||||
const dataPath = path.join(dir, 'data.json');
|
||||
try { await fs.access(dataPath); } catch { await fs.writeFile(dataPath, JSON.stringify(data, null, 2)); }
|
||||
}
|
||||
return { success: true, id: m.id, url: `app://miniapp/${m.id}/index.html` };
|
||||
} catch (e) {
|
||||
return { success: false, error: e instanceof Error ? e.message : String(e) };
|
||||
}
|
||||
},
|
||||
},
|
||||
'mini-app-set-data': {
|
||||
description: "Write a Mini App's data.json — the JSON its frontend reads via rowboat.getData/onData. The path is derived from appId and the write is atomic (temp→rename), so you only supply the content. This is how a background task refreshes an app's data; the agent returns the data, the write is deterministic.",
|
||||
inputSchema: z.object({
|
||||
appId: z.string().describe('The app id (folder name under ~/.rowboat/apps).'),
|
||||
data: z.unknown().describe('Full data payload to store as data.json (matching the app data schema).'),
|
||||
}),
|
||||
execute: async ({ appId, data }: { appId: string; data: unknown }) => {
|
||||
try {
|
||||
const dir = path.join(WorkDir, 'apps', appId);
|
||||
await fs.mkdir(dir, { recursive: true });
|
||||
const dataPath = path.join(dir, 'data.json');
|
||||
const tmp = `${dataPath}.tmp`;
|
||||
await fs.writeFile(tmp, JSON.stringify(data, null, 2));
|
||||
await fs.rename(tmp, dataPath);
|
||||
return { success: true, appId };
|
||||
} catch (e) {
|
||||
return { success: false, error: e instanceof Error ? e.message : String(e) };
|
||||
}
|
||||
},
|
||||
},
|
||||
'run-live-note-agent': {
|
||||
description: "Manually trigger the live-note agent to run now on a note. Equivalent to the user clicking the Run button in the live-note sidebar, but you can pass extra `context` to bias what the agent does this run — most useful for backfills (e.g. seeding a newly-made-live note from existing synced emails) or focused refreshes. Returns the action taken, summary, and the new note body.",
|
||||
inputSchema: z.object({
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue