docs(plans): sync phase 4-7 state; rename to scraping subagent

plans/backend/00-umbrella-plan.md

@@ -20,7 +20,7 @@ Premium, open-source, self-hostable **scraper APIs** (Bright Data / Firecrawl /
 > - **Automation = a persistent ongoing chat** that periodically re-invokes verbs. This replaces the `Triggers` subsystem; the periodic mechanism is **design-deferred** (Phase 6).
 > - **KB stays input-only**; crawled data is never indexed.
 >
-> **Phase mapping:** **04 Capabilities** · **05 Access** · **06 Ongoing-Automation** (deferred) · **07 Orchestration** (`intelligence_agent`). The subplans `05a-timeline`/`05b-intelligence`/`06-triggers` are **removed** (history in git); `04a`/`04b` are folded into `04`/`05`.
+> **Phase mapping:** **04 Capabilities** · **05 Access** · **06 Ongoing-Automation** · **07 Orchestration** (`scraping` subagent). The subplans `05a-timeline`/`05b-intelligence`/`06-triggers` are **removed** (history in git); `04a`/`04b` are folded into `04`/`05`.
 
 ## ⚠️ Architecture correction (2026-06-30) — Pipelines dropped; automations + input-only KB adopted
 
@@ -52,11 +52,11 @@ flowchart TD
   REG["Capability registry (P4): web.scrape · web.discover · maps.search/place/reviews — typed verbs, cleaned AI-ready output, bill at executor"]
   ACQ --> REG
   REG --> DOORS["Access doors (P5): REST + API key · MCP server · chat tools — generated from the registry"]
-  DOORS --> AGENT["intelligence_agent subagent (P7): intent routing · verb chains · 'what changed' from chat history"]
+  DOORS --> AGENT["scraping subagent (P7): intent routing · verb chains · 'what changed' from chat history"]
   KB[("Knowledge Base — input-only: user files/context (uploads + Drive/Dropbox/OneDrive)")] --> AGENT
   CONN["Connectors = MCP tools (BYO) + file/KB-input"] --> AGENT
   AGENT -->|one-shot| ANS["plain-language answer (nothing persists)"]
-  AGENT -.->|standing need| ONG["Ongoing-Automation (P6, DEFERRED): persistent chat re-invokes verbs; memory = chat history"]
+  AGENT -.->|standing need| ONG["Ongoing-Automation (P6): persistent chat re-invokes verbs; memory = chat history"]
   ONG -.-> REG
   APIKEY["Platform API key"] --> DOORS
 ```
@@ -168,16 +168,16 @@ The Universal WebURL Crawler is the flagship Type-1 data source (**the moat**).
 
 ### Phase 6 — Ongoing-Automation (backend) [`subplan: 06-ongoing-automation.md`]
 
-> **Canonical subplan:** [`06-ongoing-automation.md`](06-ongoing-automation.md) — **⚠️ design deferred.** The periodic mechanism is designed separately, after `04`/`05`/`07`. **Depends on `04`** (the verbs it re-invokes) and `05` (the chat surface + delivery channel).
+> **Canonical subplan:** [`06-ongoing-automation.md`](06-ongoing-automation.md) — **shipped.** A chat watch is an `Automation` bound to the current chat (`schedule` + `chat_message`); the durable checkpointer is the memory. **Depends on `04`** (the verbs it re-invokes) and `05` (the chat surface + delivery channel).
 
 - Support **"keep watching"** with no stateful storage: a **persistent, ongoing chat** where the agent periodically re-invokes scraper verbs and drops results into the session; the agent reports "what's new" by reading the chat history. Open questions (resolved together): periodicity driver, delivery channel (SSE vs Zero-published table), context-window limit, loop owner, stop/cost controls.
 
 ### Phase 7 — Orchestration (backend) [`subplan: 07-orchestration.md`]
 
-> **Canonical subplan:** [`07-orchestration.md`](07-orchestration.md) — the human-facing brain. **Build last** (atop `04`/`05`, and `06` for the ongoing mode). We plug into the shipped multi-agent runtime; we don't rebuild it.
+> **Canonical subplan:** [`07-orchestration.md`](07-orchestration.md) — the human-facing brain. **Shipped for the web verbs** (atop `04`/`05`/`06`). We plug into the shipped multi-agent runtime; we don't rebuild it.
 
-- Ship the net-new builtin **`intelligence_agent`** subagent (peer to `research`/`deliverables`): **intent routing** (one-shot vs keep-watching, one clarifying question when ambiguous), **verb composition** (`web.discover → web.scrape`, `maps.search → maps.place → maps.reviews`), and **"what changed" from the chat history** (re-invoke a verb, compare against prior tool outputs already in context — no timeline, no diff store).
-- Toolset: registry-backed capability verbs (shared with `research`) + a deferred `start_watch` handoff to `06`. Tools follow the shipped `scrape_webpage` shape (executor + door + 03c billing), **direct-return** (no `deliverable_wait`).
+- Ship the builtin **`scraping`** subagent (peer to `research`/`deliverables`): **intent routing** (one-shot vs keep-watching, block on ambiguous cadence), **verb composition** (`web.discover → web.scrape`; `maps.*` chains deferred until the Maps actor exists), and **"what changed" from the chat history** (re-invoke a verb, compare against prior tool outputs already in context — no timeline, no diff store).
+- Toolset: registry-backed capability verbs + `start_watch`/`stop_watch`/`refresh_watch` bound to the current chat (`06`). Tools follow the shipped `scrape_webpage` shape (executor + door + 03c billing), **direct-return** (no `deliverable_wait`).
 
 ### (Future) Platform API-key + public MCP
 
@@ -249,9 +249,9 @@ These are recorded for continuity but are NOT planned in this umbrella. They sta
 | 3 | `03d-captcha-solving.md` | **IMPLEMENTED** (`ci_mvp`) — `captchatools` page_action (proprietary) + Apache-2 config + per-attempt `web_crawl_captcha` billing; off by default |
 | 3 | `03f-undetectability-testing.md` | **IMPLEMENTED** (`ci_mvp`) — manual scorecard under the **proprietary boundary** at `app/proprietary/web_crawler/testbench/` (`python -m app.proprietary.web_crawler.testbench`); Suite S (stealth, shipped builder) + Suite E (extraction via real `crawl_url`) + scorecard JSON/MD baseline diff |
 | — | `00b-pipeline-diagrams.md` | end-to-end flow diagrams (companion to this umbrella) |
-| 4 | `04-capabilities.md` | **CANONICAL** · drafted — capability registry + verbs `web.scrape`/`web.discover`/`maps.*` → cleaned data; bill at executor. **← build next** |
-| 5 | `05-access.md` | **CANONICAL** · drafted — generated REST/MCP/chat doors + intent router + BYO-MCP routing fix |
-| 6 | `06-ongoing-automation.md` | **CANONICAL** · **design deferred** — chat-native "keep watching" (periodic re-invocation) |
-| 7 | `07-orchestration.md` | **CANONICAL** · drafted — `intelligence_agent` subagent; "what changed" from chat history |
+| 4 | `04-capabilities.md` | **IMPLEMENTED (web verbs)** — capability registry + `web.scrape`/`web.discover` → cleaned data, bill at executor (top_k bounds, max_length truncation, per-attempt captcha metering). `maps.*` deferred (Maps actor) |
+| 5 | `05-access.md` | **IMPLEMENTED (REST + chat doors)** — generated REST/API-key + chat tools from the registry. MCP door + BYO-MCP routing fix deferred (MCP server) |
+| 6 | `06-ongoing-automation.md` | **IMPLEMENTED** — chat-native "keep watching": `Automation` bound to the chat (`schedule` + `chat_message`), worker-safe durable checkpointer, watch tools + REST controls |
+| 7 | `07-orchestration.md` | **IMPLEMENTED (web verbs)** — `scraping` subagent; intent routing, verb composition, "what changed" from chat history |
 
 Frontend & client subplans will be added under a separate umbrella later (see "Deferred — Frontend & client phases").

plans/backend/00b-pipeline-diagrams.md

@@ -16,8 +16,8 @@ flowchart LR
   subgraph SCOPE["OUR SCOPE · Phase 04 -> 07"]
     CAP["04 Capabilities<br/>web.* · maps.*<br/>cleaned, AI-ready output"]
     ACC["05 Access<br/>REST · MCP · chat doors"]
-    ORC["07 Orchestration<br/>intelligence_agent"]
-    ONG["06 Ongoing-Automation<br/>(keep-watching · DEFERRED)"]
+    ORC["07 Orchestration<br/>scraping subagent"]
+    ONG["06 Ongoing-Automation<br/>(keep-watching)"]
   end
 
   ACQ --> CAP
@@ -77,20 +77,19 @@ sequenceDiagram
 
 ```mermaid
 flowchart TD
-  U(["User speaks in natural language"]) --> R{"Intent router<br/>(intelligence_agent prompt, 07)"}
+  U(["User speaks in natural language"]) --> R{"Intent router<br/>(scraping prompt, 07)"}
   R -->|"find / compare / what is / right now"| A["ONE-SHOT<br/>compose verbs, answer (section 3)"]
-  R -->|"watch / track / notify me / over time"| B["KEEP-WATCHING<br/>hand to Ongoing-Automation (06)"]
-  R -->|ambiguous| Q["Ask ONE question:<br/>'just once, or keep watching?'"]
+  R -->|"watch / track / notify me / over time"| B["KEEP-WATCHING<br/>start_watch -> Ongoing-Automation (06)"]
+  R -->|ambiguous cadence| Q["Block: ask for the missing cadence / timezone"]
   Q -->|once| A
   Q -->|keep watching| B
-  B -.->|DESIGN DEFERRED| X["06 mechanism TBD<br/>(periodicity · delivery · context limits)"]
 ```
 
 ## 5. "What changed" — chat history is the memory
 
 ```mermaid
 flowchart LR
-  P["Prior tool outputs<br/>(already in chat context)"] --> AG["intelligence_agent"]
+  P["Prior tool outputs<br/>(already in chat context)"] --> AG["scraping subagent"]
   NEW["Fresh verb call (04)"] --> AG
   AG --> ANS["'Here's what's new since last time'<br/>(reasoned from chat history)"]
 ```

plans/backend/06-ongoing-automation.md

@@ -1,6 +1,6 @@
 # Phase 6 — Ongoing Automation (chat-native "keep watching")
 
-> Depends on `04` (the verbs it re-invokes), `05` (the agent tools), and `07` (the `intelligence_agent`).
+> Depends on `04` (the verbs it re-invokes), `05` (the agent tools), and `07` (the `scraping` subagent).
 > Reuses the existing chat + automations machinery; adds no parallel engine.
 
 ## Objective
@@ -28,7 +28,7 @@ chat's own checkpoint is the memory.
   opened them, but Celery uses a fresh loop per task (`PoolTimeout`). Dispose the checkpointer pool per
   task in `run_async_celery_task`, mirroring `_dispose_shared_db_engine`, so a worker can use the *durable*
   checkpointer (not `InMemorySaver`) that "what changed" requires.
-- **`start_watch`** — an `intelligence_agent` tool that binds a watch to the *current* chat: it distills
+- **`start_watch`** — a `scraping` subagent tool that binds a watch to the *current* chat: it distills
   the recurring question + cadence and creates the automation (`schedule` + `chat_message(thread_id =
   current chat)`).
 - **"Is this chat watched?"** — derived: an active automation with a `chat_message` action targeting the
@@ -44,7 +44,7 @@ chat's own checkpoint is the memory.
 2. `chat_message` action: params + factory + handler (drains `stream_new_chat`); concurrency guard. **[done]**
 3. Watch service: create (bind `schedule` + `chat_message` automation to a chat) / stop (delete) /
    find-for-thread (is-watched) / run-now. **[done]**
-4. `start_watch` tool on `intelligence_agent` (+ prompt line); binds to the current chat. **[done]**
+4. `start_watch` tool on the `scraping` subagent (+ prompt line); binds to the current chat. **[done]**
 5. Controls — chat tools (`stop_watch`, `refresh_watch`) + REST (`GET /automations/watches`,
    `POST /automations/{id}/run`; stop = `DELETE /automations/{id}`). **[done]**
 

plans/backend/07-orchestration.md

@@ -1,19 +1,22 @@
 # Phase 7 — Orchestration / Conversation (the CI-expert subagent)
 
-> Build last; sits atop `04`/`05` (and `06` for the ongoing mode).
-> The multi-agent chat runtime (deepagents, subagent dispatch, streaming, citation middleware) is shipped.
-> Net-new here = one builtin subagent + its tools + its prompt. Locate code by symbol/grep.
+> **Status (2026-07-02): shipped for the web verbs.** The `scraping` subagent, its tools, prompt, and
+> router blurb are live; `web.discover`/`web.scrape` compose and the "keep watching" handoff (`06`) is
+> wired. `maps.*` composition is deferred until the Maps actor exists.
+> Sits atop `04`/`05` (and `06` for the ongoing mode). The multi-agent chat runtime (deepagents, subagent
+> dispatch, streaming, citation middleware) is shipped. Net-new here = one builtin subagent + its tools +
+> its prompt. Locate code by symbol/grep.
 
 ## Objective
 
-Ship the `intelligence_agent` — a builtin CI/scraper-expert subagent that delivers the product through
+Ship the `scraping` subagent — a builtin CI/scraper-expert that delivers the product through
 plain conversation: understand intent, compose scraper verbs (`04`), answer in plain language, and hand
 standing needs to the ongoing "keep watching" mode (`06`). It reasons over chat history to report what's
 new vs prior tool outputs already in context.
 
 ## The subagent
 
-New builtin `subagents/builtins/intelligence_agent/`, peer to `research`/`deliverables`, packed via
+Builtin `subagents/builtins/scraping/`, peer to `research`/`deliverables`, packed via
 `pack_subagent`. Files follow the existing pattern: `agent.py` (`build_subagent`), `tools/index.py`
 (`NAME · RULESET · load_tools`), `description.md` (router one-liner), `system_prompt.md` (the playbook).
 The main agent delegates CI-/scraping-flavored requests to it via `description.md`.
@@ -32,11 +35,11 @@ The main agent delegates CI-/scraping-flavored requests to it via `description.m
 
 | Tool | Wraps | Billing |
 |------|-------|---------|
-| capability verbs (`web.scrape`, `web.discover`, `maps.search`, `maps.place`, `maps.reviews`) | `04` executors (direct-return) | `03c` owner-wallet charge via `charge_capability` (same path as REST/MCP) |
-| `start_watch(intent)` *(deferred)* | hands off to `06` | per re-invocation (design with `06`) |
+| capability verbs — `web.scrape`, `web.discover` shipped; `maps.search`/`maps.place`/`maps.reviews` deferred (Maps actor) | `04` executors (direct-return) | `03c` owner-wallet charge via `charge_capability` (same path as REST/MCP) |
+| `start_watch` / `stop_watch` / `refresh_watch` | bind a watch to the current chat via `06` | per re-invocation |
 
 Verbs reach the subagent through the chat door generator `access/chat.py::build_capability_tools` (`05`),
-which turns registry verbs (`04`) into tools with owner-wallet billing. The `intelligence_agent` owns these
+which turns registry verbs (`04`) into tools with owner-wallet billing. The `scraping` subagent owns these
 CI-flavored calls; the main agent's `web_search`/`scrape_webpage` stay untouched.
 
 ## Boundaries
@@ -47,16 +50,16 @@ CI-flavored calls; the main agent's `web_search`/`scrape_webpage` stay untouched
 
 ## Work items
 
-1. Subagent scaffold: `subagents/builtins/intelligence_agent/` (`agent.py` / `tools/index.py` /
+1. **[done]** Subagent scaffold: `subagents/builtins/scraping/` (`agent.py` / `tools/index.py` /
    `description.md` / `system_prompt.md`), packed via `pack_subagent`.
-2. CI playbook prompt: intent routing (one clarifying question), verb chains, "what changed" reasoning.
-3. Chat door generator: `access/chat.py::build_capability_tools` turns registry verbs (`04`) into tools (`05`).
-4. `start_watch` seam to `06` (wired when `06` is designed).
-5. Router registration: `description.md` for main-agent delegation.
+2. **[done]** CI playbook prompt: intent routing (block on ambiguous cadence), verb chains, "what changed" reasoning.
+3. **[done]** Chat door generator: `access/chat.py::build_capability_tools` turns registry verbs (`04`) into tools (`05`).
+4. **[done]** `start_watch` / `stop_watch` / `refresh_watch` seam to `06`, bound to the current chat.
+5. **[done]** Router registration: `description.md` for main-agent delegation.
 
 ## Tests
 
-- A CI-/scraping-flavored request routes to `intelligence_agent`; a non-CI request does not.
+- A CI-/scraping-flavored request routes to `scraping`; a non-CI request does not.
 - "compare X and Y" composes verbs and answers in plain language, persisting nothing.
 - "reviews for the top 3 coffee shops near me" chains `maps.search → maps.place → maps.reviews`.
 - A follow-up re-invokes the verb and reports new items vs prior in-context outputs.
@@ -70,4 +73,4 @@ CI-flavored calls; the main agent's `web_search`/`scrape_webpage` stay untouched
 
 ## Open questions
 
-- The `start_watch` handoff shape (settled with `06`).
+- None outstanding for the web verbs. `maps.*` composition reopens when the Maps actor lands.