diff --git a/plans/backend/revamp/03-intelligence.md b/plans/backend/revamp/03-intelligence.md new file mode 100644 index 000000000..946a77044 --- /dev/null +++ b/plans/backend/revamp/03-intelligence.md @@ -0,0 +1,114 @@ +# Domain ③ — Intelligence (the decision-grounded engine) (CI pivot revamp · WIP) + +> **WIP design doc.** Part of the Phase 4 → end revamp. Pairs with `04-timeline.md` (the state it writes). +> **Scope guardrail:** Phases 1–3 SHIPPED/FIXED. This is net-new and is **not** the KB and **not** +> the automations subsystem. +> **Working name:** the standing-concern primitive is called a **Lens** (provisional — easily renamed). + +## The stateless / stateful line + +``` +STATELESS (Product A): ① Capabilities + ② Access → call → data → bill, nothing persists +STATEFUL (Product B): ③ Intelligence + ④ Timeline → the Timeline IS the state +``` + +Intelligence is the **process** that mutates state; Timeline (`04`) is the **state**. Everything +below the Access→Intelligence boundary stays pure functions. + +## Purpose + +Turn repeated capability calls into **decision-relevant structured signal**. The motto: **the agent +judges, code computes.** This replaces the old "pipeline" as the *standing concern*. + +## The primitive — `Lens` + +A saved, decision-grounded subject that accumulates structured signal over time: + +| Field | Meaning | +|-------|---------| +| `decision` | the question being tracked toward ("is this competitor pulling ahead?") | +| `capability_binding` | which verb + input feeds it (`maps.place(X)`, `web.scrape([Y])`) | +| `definition` (locked, versioned) | `{ field_schema, identity_rule, materiality }` — the agent-drafted, human-locked contract | +| `status` | `draft` → `locked`/`active` | + +One **entity per Lens for MVP** (one place / one URL). Multi-entity (`maps.search → many`) is deferred +(the Timeline model stays multi-entity-ready so it's additive). + +## Setup (once) — the agent-designed schema flow (IN MVP) + +The product must not be rigid: **we cannot author one schema that serves everyone**, so the schema is +derived from the *user's* decision by an agent and locked by the human. Conversationally (chat-first, +no UI needed): + +1. **Bind** a capability + input (the thing to track). +2. **Sample fetch** — one real capability call so the agent drafts against *actual* returned data, + not a hallucinated shape. +3. **Agent drafts the `definition`** from `decision` + the sample: + - `field_schema` — the fields that matter + types (structured-output contract). + - `materiality` — per-field rules (numeric thresholds where possible; "ask agent" otherwise). + - `identity_rule` — the stable entity key (e.g. Maps `place_id`, canonical URL). + - a reserved **`notable_signals`** escape-hatch field for the unanticipated. +4. **Human reviews & locks** (in chat: "looks good" / "add field X"). Locked ⇒ stable run-to-run. +5. **Versioned** — a locked `definition` is a snapshot; edits create a new version (mirrors how + `automations` snapshots `definition`). + +## The hot loop (per refresh) — `refresh(lens)` + +1. **Crawl** — call the bound capability (Domain ①) → raw data. +2. **Cheap pre-check** — content hash via the existing + `WebCrawlerConnector.format_to_structured_document(exclude_metadata=True)`; identical hash to the + stored `content_hash` → stamp `last_checked_at`, **stop** (no LLM cost). +3. **Fill** — agent conforms raw data to the **locked `field_schema`** via structured output; it does + **not** invent fields. Unanticipated observations go into `notable_signals`. +4. **Diff (code)** — deterministic compare of the new record vs Current state (`04`) → raw deltas. +5. **Judge — the materiality split:** + - **deterministic (code):** numeric / clear-cut rules from `materiality`, applied for free, 100% + reproducible. e.g. `rating Δ≥0.2 → material`, `review_count Δ≥10 → material`, `1¢ price wobble + → noise`, `any hours change → material`. + - **agent (only on ambiguous):** anything a rule can't decide — reworded `description`, a new + `notable_signals` entry, "does this matter *for the decision*?" → one LLM call rules + material/noise. +6. **Append** — if material: write a Change + update Current state (`04`). Else: only `last_checked_at`. + **No change → no row.** + +**Worked example (`maps.place` refresh):** +``` +rating 4.4 → 4.3 (Δ0.1) → code: < 0.2 → NOISE (no LLM) +review_count 312 → 470 → code: ≥ 10 → MATERIAL (no LLM) +hours unchanged → no delta +description reworded → code: no rule → ASK AGENT → NOISE +⇒ one Change row (review spike); one cheap LLM call; zero LLM on the rating tick. +``` + +## Where it lives / decoupling + +- New **Apache-2** package `app/intelligence/` (the schema-design agent, the hot loop, the materiality + evaluator). Calls capability `executor`s directly (not through a door). +- Exposes **`refresh(lens)`**. *Who* calls it (manual / agent / external cron / optional automation) is + the **Triggers** domain's concern — Intelligence has **no dependency** on any scheduler. + +## MVP cut vs north star + +- **MVP:** agent-designed-schema flow (conversational, sample-grounded, human-locked) · single entity + per Lens · the hot loop with content-hash pre-check + code-threshold + agent-on-ambiguous · + `refresh(lens)` fired manually / by agent / by external cron. +- **North star (deferred):** schema **auto-evolution** from recurring `notable_signals` · multi-entity + Lenses (`maps.search`) · backward-replay reconstruction · coverage-confidence · full + provenance/explainability · the resale/data-product stage. + +## Locked decisions + +1. `Lens` (provisional name) is the standing-concern primitive; replaces "pipeline". +2. Stateless (①②/Product A) vs stateful (③④/Product B) is the Access→Intelligence boundary. +3. **Agent-designed schema flow is in MVP** (not hand-authored) — sample-grounded, human-locked, versioned. +4. Single entity per Lens for MVP. +5. Materiality = deterministic numeric/clear rules in code + agent only on ambiguous. +6. Content-hash pre-check short-circuits unchanged pages before any LLM spend. +7. `app/intelligence/` Apache-2; `refresh(lens)` is trigger-agnostic. + +## Open questions (carry forward) + +- How the schema-design agent surfaces the "review & lock" step before the frontend exists (pure-chat + confirmation for MVP?). +- Versioning policy on re-lock (new version vs in-place) — lean new version. +- Where the schema-design agent itself runs (a setup capability? a chat sub-flow?).