From a5fc8c2b7d2e268dd18312e22867ccb3d08eebef Mon Sep 17 00:00:00 2001 From: CREDO23 Date: Tue, 30 Jun 2026 23:35:42 +0200 Subject: [PATCH] docs(revamp): add Access domain (chat/REST/MCP doors) --- plans/backend/revamp/02-access.md | 99 +++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 plans/backend/revamp/02-access.md diff --git a/plans/backend/revamp/02-access.md b/plans/backend/revamp/02-access.md new file mode 100644 index 000000000..326b6a055 --- /dev/null +++ b/plans/backend/revamp/02-access.md @@ -0,0 +1,99 @@ +# Domain ② — Access / Surfaces (CI pivot revamp · WIP) + +> **WIP design doc.** Part of the Phase 4 → end revamp. Sits on top of Domain ① (Capabilities). +> **Scope guardrail:** Phases 1–3 are SHIPPED and FIXED. Identity/Tenancy, API keys, the chat +> agent + its tool registry, the streaming layer, and Metering (`03c`) already exist — Access +> *reuses* them. + +## Role in the universe + +``` +FIXED: Acquisition · Metering (03c) · Identity/Tenancy · API keys · chat agent · streaming +SCOPE: Capabilities → ▶ Access ◀ → Intelligence + Timeline → Triggers +``` + +Access is the set of **doors** onto the capability registry. It contains **no business logic** — +every door is the same thin adapter. + +## Purpose + +Expose the capability registry to callers, **authenticated + metered**. One adapter shape for every +verb, on every door: + +``` +parse input → validate against verb.input_schema → authn/authz → meter-gate (03c) + → call the SAME executor → serialize verb.output_schema → return the uniform envelope +``` + +## The three doors (locked order: chat → REST → MCP) + +| Door | Who | Auth | Status | +|------|-----|------|--------| +| **Chat tools** | in-app agent (Product B delivery + interactive) | existing session + workspace | partly exists (`scrape_webpage`) | +| **REST + API keys** | external developers (Product A) | **existing API-key infra** (reuse, do not build net-new) | **public day one** | +| **MCP server** | external agents (Cursor/ChatGPT/Claude) | OAuth 2.1 **or** bearer — *chosen at implementation* | **fast-follow** after chat+REST | + +All three are **generated from the one capability registry** (Domain ①). REST being public day one +is cheap precisely because the routes are generated, not hand-written — it's a go-to-market choice, +not an engineering cost. + +## The two MCP directions (keep distinct) + +- **We *serve* MCP** — our capabilities as a remote MCP server (door #3, new). "External agents gain + the real web." Remote Streamable-HTTP `/mcp`, stateless, bounded/paginated outputs, untrusted + inputs, least-privilege (read-only verbs). Auth depth decided at implementation. +- **We *consume* MCP** — the BYO-`MCP_CONNECTOR` from old 04a: the user's *own* external MCP tools + inside our chat agent. This is the only "connector" worth keeping; the 04a routing-gap fix lands + **here** (Access/Conversation), not in Capabilities. + +## Chat ↔ slow jobs — reuse the existing background-worker pattern + +Do **not** invent a chat-async mechanism. The deliverables stack already solves it: + +- `subagents/builtins/deliverables/deliverable_wait.py` — a shared **poll-until-terminal** helper: + dispatch the Celery task, poll the row's `status` until `READY`/`FAILED` (1.5s cadence), return a + real terminal outcome. Bounded by `SURFSENSE_SUBAGENT_INVOKE_TIMEOUT_SECONDS` (default 300s) in + multi-agent mode. +- `deliverables/tools/podcast.py` — the "return now + a live card tracks progress" model for very + long work; streaming emission frames live under + `tasks/chat/streaming/handlers/tools/deliverables/...`. + +**Mapping to capabilities:** a slow verb (`web.scrape` over many URLs, `maps.search`, `maps.reviews`) +invoked from chat dispatches the **job** (Domain ① job record) and uses the `deliverable_wait` +poll-until-terminal path; the capability **job record's `status`** is what the helper polls (the +analogue of the podcast/artifact status row). Most calls finish inside the poll window → the tool +returns results inline; genuinely long ones surface a tracked card. REST/MCP doors expose the same +job via `GET /v1/jobs/{id}` (and an MCP equivalent). + +## Reused / fixed (not built here) + +- **API keys** — existing infra; keys scope to a workspace; billed to the workspace owner via `03c`. +- **Identity & Tenancy** — workspace/user/permission checks on every door. +- **Chat agent + tool registry** — capability tools are *registered into* the existing registry. +- **Streaming layer** — existing SSE/card emission for chat job progress. +- **Metering (`03c`)** — the balance gate before execute + charge after, on every door. + +## Relationship to the drafted Phase 4 + +- **04a BYO-`MCP_CONNECTOR` routing fix** → lands here (we *consume* the user's MCP tools). +- Old connector-config routes for data sources are **not** the surface anymore — the capability + REST/MCP/chat doors are. Legacy branded connectors stay only for backward-compat (separate + hygiene task). + +## Locked decisions + +1. Three doors, generated from the capability registry; order chat → REST → MCP. +2. REST is **public day one** (cheap; go-to-market choice). +3. API keys: **reuse existing infra**, billed to workspace owner. +4. MCP server is a **fast-follow**; auth depth (OAuth 2.1 vs bearer) chosen at implementation. +5. Chat ↔ slow jobs: **reuse `deliverable_wait` poll-until-terminal + live card**, polling the + capability job record's `status`. +6. "Serve MCP" (our tools out) vs "consume MCP" (BYO tools in) are distinct; the 04a fix is the + consume side. + +## Open questions (carry forward) + +- MCP auth depth (decide at implementation). +- Public REST rate-limiting / abuse posture (bounded inputs, per-key quotas) — design alongside the + public launch. +- Whether `web.discover` is metered or free (carried from Domain ①).