Domain ② — Access / Surfaces (CI pivot revamp · WIP)

WIP design doc. Part of the Phase 4 → end revamp. Sits on top of Domain ① (Capabilities). Scope guardrail: Phases 1–3 are SHIPPED and FIXED. Identity/Tenancy, API keys, the chat agent + its tool registry, the streaming layer, and Metering (03c) already exist — Access reuses them.

Role in the universe

FIXED:   Acquisition · Metering (03c) · Identity/Tenancy · API keys · chat agent · streaming
SCOPE:   Capabilities → ▶ Access ◀ → Intelligence + Timeline → Triggers

Access is the set of doors onto the capability registry. It contains no business logic — every door is the same thin adapter.

Purpose

Expose the capability registry to callers, authenticated + metered. One adapter shape for every verb, on every door:

parse input → validate against verb.input_schema → authn/authz → meter-gate (03c)
  → call the SAME executor → serialize verb.output_schema → return the uniform envelope

The three doors (locked order: chat → REST → MCP)

Door	Who	Auth	Status
Chat tools	in-app agent (Product B delivery + interactive)	existing session + workspace	partly exists (`scrape_webpage`)
REST + API keys	external developers (Product A)	existing API-key infra (reuse, do not build net-new)	public day one
MCP server	external agents (Cursor/ChatGPT/Claude)	OAuth 2.1 or bearer — chosen at implementation	fast-follow after chat+REST

All three are generated from the one capability registry (Domain ①). REST being public day one is cheap precisely because the routes are generated, not hand-written — it's a go-to-market choice, not an engineering cost.

The two MCP directions (keep distinct)

We serve MCP — our capabilities as a remote MCP server (door #3, new). "External agents gain the real web." Remote Streamable-HTTP /mcp, stateless, bounded/paginated outputs, untrusted inputs, least-privilege (read-only verbs). Auth depth decided at implementation.
We consume MCP — the BYO-MCP_CONNECTOR from old 04a: the user's own external MCP tools inside our chat agent. This is the only "connector" worth keeping; the 04a routing-gap fix lands here (Access/Conversation), not in Capabilities.

Chat ↔ slow jobs — reuse the existing background-worker pattern

Do not invent a chat-async mechanism. The deliverables stack already solves it:

subagents/builtins/deliverables/deliverable_wait.py — a shared poll-until-terminal helper: dispatch the Celery task, poll the row's status until READY/FAILED (1.5s cadence), return a real terminal outcome. Bounded by SURFSENSE_SUBAGENT_INVOKE_TIMEOUT_SECONDS (default 300s) in multi-agent mode.
deliverables/tools/podcast.py — the "return now + a live card tracks progress" model for very long work; streaming emission frames live under tasks/chat/streaming/handlers/tools/deliverables/....

Mapping to capabilities: a slow verb (web.scrape over many URLs, maps.search, maps.reviews) invoked from chat dispatches the job (Domain ① job record) and uses the deliverable_wait poll-until-terminal path; the capability job record's status is what the helper polls (the analogue of the podcast/artifact status row). Most calls finish inside the poll window → the tool returns results inline; genuinely long ones surface a tracked card. REST/MCP doors expose the same job via GET /v1/jobs/{id} (and an MCP equivalent).

Reused / fixed (not built here)

API keys — existing infra; keys scope to a workspace; billed to the workspace owner via 03c.
Identity & Tenancy — workspace/user/permission checks on every door.
Chat agent + tool registry — capability tools are registered into the existing registry.
Streaming layer — existing SSE/card emission for chat job progress.
Metering (03c) — the balance gate before execute + charge after, on every door.

Relationship to the drafted Phase 4

04a BYO-MCP_CONNECTOR routing fix → lands here (we consume the user's MCP tools).
Old connector-config routes for data sources are not the surface anymore — the capability REST/MCP/chat doors are. Legacy branded connectors stay only for backward-compat (separate hygiene task).

Locked decisions

Three doors, generated from the capability registry; order chat → REST → MCP.
REST is public day one (cheap; go-to-market choice).
API keys: reuse existing infra, billed to workspace owner.
MCP server is a fast-follow; auth depth (OAuth 2.1 vs bearer) chosen at implementation.
Chat ↔ slow jobs: reuse deliverable_wait poll-until-terminal + live card, polling the capability job record's status.
"Serve MCP" (our tools out) vs "consume MCP" (BYO tools in) are distinct; the 04a fix is the consume side.

Open questions (carry forward)

MCP auth depth (decide at implementation).
Public REST rate-limiting / abuse posture (bounded inputs, per-key quotas) — design alongside the public launch.
Whether web.discover is metered or free (carried from Domain ①).

5.2 KiB Raw Blame History Unescape Escape