nyx/tests/benchmark/RESULTS.md

# Nyx Benchmark Results

Current baseline (as of Auth Rule FP-Remediation Phase B5 corpus add, 2026-04-23):

| Metric                | File-level | Rule-level | CI floor |
|-----------------------|------------|------------|----------|
| Precision             | 0.946      | 0.946      | 0.861    |
| Recall                | 1.000      | 0.994      | 0.944    |
| F1                    | 0.972      | 0.970      | 0.901    |

Corpus: 305 cases across 10 languages — 267 synthetic + 28 real-CVE cases (14 vulnerable/patched pairs) + 10 auth-rule cases (3 positive + 7 negative). Scanner 0.5.0, full analysis mode. CI floors are unchanged from Phase CF-7; the Auth-B5 delta is within 1 pp and does not warrant tightening.

Machine-readable per-run data lives in `tests/benchmark/results/` (`latest.json` plus dated snapshots). This file is a narrative changelog — only the two most recent phases are kept in full detail; earlier phases are condensed into the history table at the end.

---

## Auth Rule FP Remediation — Phase B5 (2026-04-23)

### Motivation

Until B5, the `rs.auth.missing_ownership_check` rule (introduced as part of `auth_analysis`) was defended only by `cargo test --test auth_analysis_tests` integration assertions; it had **zero** entries in the benchmark corpus. So precision/recall regressions on auth fixtures wouldn't show up in the headline P/R/F1 numbers, and the Phase A1–A3 / B1–B4 fixture work in `tests/fixtures/auth_analysis/` was invisible to anyone reading `RESULTS.md`. This phase mirrors the relevant Rust auth fixtures into `tests/benchmark/corpus/rust/auth/`, adds ground-truth cases for each, and surfaces the resulting `auth` vuln-class metrics in the by-class breakdown.

### What changed

- **10 new fixtures** copied to `tests/benchmark/corpus/rust/auth/` (mirrors of the integration fixtures — keep both in sync when fixtures evolve).
- **Ground truth**: 10 new cases (`rs-auth-001` … `rs-auth-003` positive, `rs-auth-101` … `rs-auth-107` negative); `corpus_size` bumped 295 → 305.
- **Positive cases** assert `expected_rule_ids: ["rs.auth.missing_ownership_check"]` and `expected_sink_lines` pinned to the specific call line; one of them (`rs-auth-002`) is the Phase-A "true positive control" mandated by the FP-remediation plan.
- **Negative cases** assert `is_vulnerable: false` + `forbidden_rule_ids: ["rs.auth.missing_ownership_check"]` (the schema's noise-budget-zero shape), one per Phase A1/A2/A3/B2/B3/B4 fixture so each regression has a dedicated wire.
- **Regression thresholds unchanged**: floors stay at `P≥0.861 R≥0.944 F1≥0.901`.

### Auth Corpus

| Case ID      | Fixture                                      | Phase covered | Vulnerable | Why it's in the corpus |
|--------------|----------------------------------------------|---------------|------------|------------------------|
| rs-auth-001  | actix_scoped_write_missing.rs                | regression    | yes        | Original positive baseline — must keep flagging |
| rs-auth-002  | true_positive_missing_check.rs               | A control     | yes        | Phase A's positive control — must still flag after every FP fix |
| rs-auth-003  | row_ownership_no_early_exit.rs               | A2 guard      | yes        | Equality without early exit — A2 must NOT silence this |
| rs-auth-101  | hashmap_local_noise.rs                       | A1            | no         | std::collections noise — A1 receiver-type/var gate suppresses |
| rs-auth-102  | helper_scoped_params.rs                      | A1            | no         | Library helper with locally-bound HashSet — A1 suppresses |
| rs-auth-103  | row_ownership_equality.rs                    | A2            | no         | `if owner_id != user.id { return … }` covers downstream column reads |
| rs-auth-104  | self_scoped_user.rs                          | A3            | no         | `let user = require_auth(..).await?` — `user.id` is self, not a foreign id |
| rs-auth-105  | db_connection_type_inferred.rs               | B2            | no         | SSA-derived `DatabaseConnection` type drives sink classification |
| rs-auth-106  | sql_join_acl.rs                              | B3            | no         | `JOIN group_members WHERE gm.user_id = ?1` makes returned rows membership-gated |
| rs-auth-107  | transitive_helper.rs                         | B4            | no         | Helper-summary lifting recognises `validate_target(group_id, user.id)` as an auth check |

### Delta

Aggregate rule-level metrics on the 305-case corpus: **P = 0.946, R = 0.994, F1 = 0.970** (vs Phase 15's `P = 0.945, R = 0.994, F1 = 0.969` on 295 cases — auth cases all hit P=1.0 R=1.0 and net to 3 TP + 7 TN, lifting precision by 1 pp via dilution). The new `by_vuln_class` row is `auth: TP=3 FP=0 FN=0 TN=0 P=1.000 R=1.000 F1=1.000`; the seven negatives roll into the existing `safe` class. The Rust per-language line moves from **TP=22 FP=0 FN=0 TN=13** (before) to **TP=25 FP=0 FN=0 TN=20** (after).

### Notes

- Fixture mirroring is an explicit choice over a symlink: `scan_corpus_file` copies the case into a tempdir before scanning, and absolute symlinks would break that path. When the integration fixture changes, copy the new file into the corpus mirror as well.
- The negative cases are the regression wires for the FP-remediation work. Each one corresponds to a phase landed in the project memory tracker; if a future change reintroduces the FP, the matching `rs-auth-1xx` case flips to FP and the Rust precision drops below the floor.
- `actix_scoped_write_missing.rs` is the only auth fixture that overlaps the "generic_ownership_check_is_consistent_across_languages" integration test — keep both wires alive (the integration test exercises the multi-language consistency, the bench wire defends the precision number).

---

## Phase 15 — Real-CVE language-gap expansion (2026-04-23)

### Motivation

Phase 13 and Phase 14 covered 9 CVEs across Python, JavaScript, TypeScript, Go, Java, Ruby, and PHP — every Stable and Beta tier language. The Preview-tier languages (C, C++) had zero real-CVE coverage, so the memory-safety and command-injection pattern rules for those languages were defended only by synthetic micro-fixtures. Phase 15 fills that gap with 4 Preview-tier CVEs (2 C, 2 C++) and adds a second Java CVE in the Runtime.exec class (to complement the existing Commons Collections deserialization case). Rust was considered but dropped: its code-quality pattern rules (`rs.memory.*`, `rs.quality.*`) are not CVE-class, and the taint-flow sink set (sqlx / rusqlite / diesel / reqwest / `std::process::Command`) did not yield a permissive-licensed, well-documented CVE reducible to ~30 LOC with a clean patched variant. Go was considered for a second CVE but dropped: idiomatic prepared statements make Go SQLi CVEs rare, `gob` decoding is niche, and published `InsecureSkipVerify` CVEs mostly describe receiver-side TLS bypass rather than the client-side pattern the rule detects.

### What changed

- **Five additional CVE pairs** added to `tests/benchmark/cve_corpus/` (10 fixture files, vulnerable + patched per CVE). Same header convention, same minimal-reproducer discipline, same `provenance: "real_cve"` marker. First entries ever for `cve_corpus/c/` and `cve_corpus/cpp/`.
- **Ground truth**: 10 new cases; `corpus_size` bumped 285 → 295. Vulnerable fixtures assert on an `expected_rule_ids` entry (the pattern-rule that fires on the disclosed sink) plus `taint-unsanitised-flow` as an acceptable alternative. Patched fixtures assert on `forbidden_rule_ids` (the CVE's class-specific rule plus the cross-cutting taint ID) so Nyx does not refire on the fix.
- **No harness changes**: the `cve_corpus/` path resolution and `real_cve` provenance scaffolding landed in Phase 13; Phase 15 is pure fixture + ground-truth expansion.
- **Regression thresholds unchanged**: floors stay at `P≥0.861 R≥0.944 F1≥0.901`.

### Real-CVE Corpus

| CVE              | Language   | Project                      | License              | Vuln class       | Vulnerable outcome | Patched outcome |
|------------------|------------|------------------------------|----------------------|------------------|--------------------|-----------------|
| CVE-2023-48022   | Python     | Ray                          | Apache-2.0           | CMDI             | TP (rule + line)   | TN              |
| CVE-2017-18342   | Python     | PyYAML                       | MIT                  | Deserialization  | TP (rule + line)   | TN              |
| CVE-2019-14939   | JavaScript | mongo-express                | MIT                  | code_exec        | TP (rule + line)   | TN              |
| CVE-2023-26159   | TypeScript | follow-redirects             | MIT                  | SSRF             | TP (rule + line)   | TN              |
| CVE-2022-30323   | Go         | hashicorp/go-getter          | MPL-2.0              | CMDI             | TP (rule + line)   | TN              |
| CVE-2015-7501    | Java       | Apache Commons Collections   | Apache-2.0           | Deserialization  | TP (rule + line)   | TN              |
| CVE-2013-0156    | Ruby       | Ruby on Rails                | MIT                  | Deserialization  | TP (rule)          | TN              |
| CVE-2017-9841    | PHP        | PHPUnit                      | BSD-3-Clause         | code_exec        | TP (rule + line)   | TN              |
| CVE-2018-15133   | PHP        | Laravel                      | MIT                  | Deserialization  | TP (rule + line)   | TN              |
| CVE-2016-3714    | C          | ImageMagick (ImageTragick)   | ImageMagick License  | CMDI             | TP (rule + line)   | TN              |
| CVE-2019-18634   | C          | sudo (pwfeedback)            | ISC                  | memory_safety    | TP (rule + line)   | TN              |
| CVE-2019-13132   | C++        | ZeroMQ libzmq                | MPL-2.0              | memory_safety    | TP (rule + line)   | TN              |
| CVE-2022-1941    | C++        | Protocol Buffers             | BSD-3-Clause         | memory_safety    | TP (rule + line)   | TN              |
| CVE-2017-12629   | Java       | Apache Solr                  | Apache-2.0           | CMDI             | TP (rule + line)   | TN              |

New-in-Phase-15 detail:

- **CVE-2016-3714** (ImageMagick "ImageTragick" delegate RCE). Vulnerable fixture: user-controlled filename is substituted into a shell template and handed to `system()` — Nyx fires `c.cmdi.system` at the documented sink line. Patched fixture: in-process coder + basename check, no `system()` path — zero findings.
- **CVE-2019-18634** (sudo pwfeedback stack overflow). Vulnerable fixture: stdin-sourced token `strcpy`'d into a fixed on-stack feedback buffer — Nyx fires `c.memory.strcpy`. Patched fixture: a bounded `copy_bounded` helper replaces the unchecked copy — zero findings.
- **CVE-2019-13132** (ZeroMQ libzmq V2 metadata overflow). Vulnerable fixture: peer-controlled bytes `strcpy`'d into a fixed on-stack identity buffer, mirroring the ZMTP v2 decode path — Nyx fires `cpp.memory.strcpy`. Patched fixture: bounded `std::string.assign` + hard length cap — zero findings.
- **CVE-2022-1941** (Protocol Buffers C++ `ParseContext` unknown-field overflow). Vulnerable fixture: wire-declared length trusted and `strcpy`'d into a scratch buffer — Nyx fires `cpp.memory.strcpy`. Patched fixture: bounded `std::string.assign` + `MAX_LABEL` cap — zero findings.
- **CVE-2017-12629** (Apache Solr XSLT response writer RCE). Vulnerable fixture: `req.getParameter("tr") → Runtime.getRuntime().exec(new String[]{"/bin/sh","-c","xsltproc "+tr})` — Nyx fires `java.cmdi.runtime_exec` and `taint-unsanitised-flow` (source line 29 → sink line 33). Patched fixture: fixed allowlist of transformer names mapped to classpath resources, no `Runtime.exec` path — zero findings.

Per-CVE precision/recall: each vulnerable case contributes 1 TP (Java CVE-2017-12629 contributes 2 — pattern-rule + taint edge) and its patched sibling 1 TN, so per-CVE precision and recall are both 1.000 at the rule level.

### Delta

Aggregate rule-level F1 on the 295-case corpus is **0.969** (P=**0.945**, R=**0.994**), a +0.001 delta vs the Phase 14 baseline (F1=0.968, P=0.944, R=0.994). File-level F1 **0.972** (P=0.945, R=1.000). The precision win is the ten new cases contributing 10 TP + 5 TN with no spurious firings on the fixes, diluting the existing FP rate slightly.

### Notes on selection

Phase 15 followed the same criteria as Phase 13/14: publicly disclosed CVE with a stable NVD advisory URL, vulnerability class already covered by a Nyx pattern rule so the vulnerable fixture produces a concrete `expected_rule_ids` hit (not just a generic `taint-unsanitised-flow`), extractable to ~30 LOC, permissive upstream license. The C/C++ picks target the two most abundant CVE classes for those languages — unchecked-copy memory-safety bugs (`strcpy` / `sprintf`-family) and shell-injection command-execution (`system()`-family). Each picked CVE is a well-known, historically damaging bug: ImageTragick mass-exploited image-upload endpoints in 2016, sudo pwfeedback gave any local user root on default-configured Linux distros in 2019, libzmq CVE-2019-13132 was pre-auth RCE on curve-disabled sockets, protobuf CVE-2022-1941 exposed every gRPC or Envoy binary decoding untrusted bytes, and Solr CVE-2017-12629 was a flagship unauthenticated-RCE vector for the entire Lucene / Solr fleet. Fixtures are minimal reproducers of the unsafe sink pattern, with explicit disclaimers — they are not verbatim excerpts of upstream internals.

---

## Phase 14 — Real-CVE corpus expansion (2026-04-23)

### Motivation

Phase 13 seeded the real-CVE subtree with one CVE per stable-tier language (Python / JavaScript / TypeScript). Six fixtures is enough to demonstrate the mechanism but not enough to defend the Beta- and Preview-tier languages against regressions on real-world code. Phase 14 extends the subtree to cover Go, Java, Ruby, and PHP, plus a second Python CVE in a different vulnerability class (deserialization, not CMDI). The goal is the same as Phase 13: regression protection on demonstrably real disclosed bugs, not synthetic analogues.

### What changed

- **Six additional CVE pairs** added to `tests/benchmark/cve_corpus/` (12 fixture files, vulnerable + patched per CVE). Same header convention, same minimal-reproducer discipline, same `provenance: "real_cve"` marker.
- **Ground truth**: 12 new cases; `corpus_size` bumped 273 → 285. Vulnerable fixtures assert on an `expected_rule_ids` entry (the pattern-rule that fires on the disclosed sink) plus `taint-unsanitised-flow` as an acceptable alternative. Patched fixtures assert on `forbidden_rule_ids` (the CVE's class-specific rule) so Nyx does not refire on the fix.
- **No harness changes**: the `cve_corpus/` path resolution and `real_cve` provenance scaffolding landed in Phase 13; Phase 14 is pure fixture + ground-truth expansion.
- **Regression thresholds unchanged**: floors stay at `P≥0.861 R≥0.944 F1≥0.901`.

### Real-CVE Corpus

| CVE              | Language   | Project                      | License      | Vuln class       | Vulnerable outcome | Patched outcome |
|------------------|------------|------------------------------|--------------|------------------|--------------------|-----------------|
| CVE-2023-48022   | Python     | Ray                          | Apache-2.0   | CMDI             | TP (rule + line)   | TN              |
| CVE-2017-18342   | Python     | PyYAML                       | MIT          | Deserialization  | TP (rule + line)   | TN              |
| CVE-2019-14939   | JavaScript | mongo-express                | MIT          | code_exec        | TP (rule + line)   | TN              |
| CVE-2023-26159   | TypeScript | follow-redirects             | MIT          | SSRF             | TP (rule + line)   | TN              |
| CVE-2022-30323   | Go         | hashicorp/go-getter          | MPL-2.0      | CMDI             | TP (rule + line)   | TN              |
| CVE-2015-7501    | Java       | Apache Commons Collections   | Apache-2.0   | Deserialization  | TP (rule + line)   | TN              |
| CVE-2013-0156    | Ruby       | Ruby on Rails                | MIT          | Deserialization  | TP (rule)          | TN              |
| CVE-2017-9841    | PHP        | PHPUnit                      | BSD-3-Clause | code_exec        | TP (rule + line)   | TN              |
| CVE-2018-15133   | PHP        | Laravel                      | MIT          | Deserialization  | TP (rule + line)   | TN              |

New-in-Phase-14 detail:

- **CVE-2017-18342** (PyYAML `yaml.load` default loader). Vulnerable fixture: `request.get_data → yaml.load` — Nyx fires `py.deser.yaml_load` and `taint-unsanitised-flow` at the documented sink line. Patched fixture: `yaml.safe_load` — zero findings.
- **CVE-2022-30323** (hashicorp/go-getter URL → git argv injection). Vulnerable fixture: `r.URL.Query().Get("src") → exec.Command("git", "clone", url, ...)` — Nyx fires `go.cmdi.exec_command` and `taint-unsanitised-flow`. Patched fixture: scheme allowlist + in-process go-git `PlainClone` removes the `exec.Command` path entirely — zero findings.
- **CVE-2015-7501** (Apache Commons Collections `InvokerTransformer` gadget chain). Vulnerable fixture: `req.getInputStream → new ObjectInputStream(...).readObject()` — Nyx fires `java.deser.readobject` and `taint-unsanitised-flow`. Patched fixture: Jackson JSON codec replaces native Java deserialization — zero findings.
- **CVE-2013-0156** (Rails XML-params YAML tag RCE). Vulnerable fixture: `YAML.load(params[:prefs])` — Nyx fires `rb.deser.yaml_load` (no taint edge because Ruby `params[...]` is not currently labeled as a taint source; the AST pattern is what catches this class). Patched fixture: `JSON.parse` replaces `YAML.load` — zero findings.
- **CVE-2017-9841** (PHPUnit `eval-stdin.php` webshell). Vulnerable fixture: `file_get_contents('php://input') → eval(...)` — Nyx fires `php.code_exec.eval` and `taint-unsanitised-flow`. Patched fixture: SAPI guard and the eval sink removed — zero findings.
- **CVE-2018-15133** (Laravel cookie `unserialize` on leaked APP_KEY). Vulnerable fixture: `$_COOKIE['XSRF-TOKEN'] → base64_decode → unserialize` — Nyx fires `php.deser.unserialize` and `taint-unsanitised-flow`. Patched fixture: HMAC-verified JSON payload — zero findings.

Per-CVE precision/recall: each vulnerable case contributes 1 TP and its patched sibling 1 TN, so per-CVE precision and recall are both 1.000 at the rule level.

### Delta

Aggregate rule-level F1 on the 285-case corpus is **0.968** (P=**0.944**, R=**0.994**), a +0.001 delta vs the Phase 13 baseline (F1=0.967, P=0.942, R=0.994). File-level F1 **0.971** (P=0.944, R=1.000). The precision win is the twelve new cases contributing 6 TP + 6 TN with no spurious firings on the fixes, diluting the existing FP rate slightly.

### Notes on selection

Phase 14's picks followed the Phase 13 criteria: publicly disclosed CVE with a known patch, vulnerability class already covered by a Nyx pattern rule (so the vulnerable fixture produces a concrete `expected_rule_ids` hit, not just a generic `taint-unsanitised-flow`), extractable to ~30 LOC, permissive upstream license. Each added CVE is a well-known, historically damaging bug — mass-scanned webshells (PHPUnit 2017), pre-auth RCE on every Rails app (2013-0156), the original Java deserialization gadget chain (Commons Collections 2015), the go-getter fleet-wide Terraform/Packer/Nomad/Vault exposure (2022), and a textbook Laravel cookie-forgery chain (2018).

---

## Phase 13 — Real-CVE replay corpus (2026-04-23)

### Motivation

The corpus up to Phase CF-7 was 267 synthetic micro-fixtures (8–20 LOC each). A 95% F1 on toy code does not imply a 95% F1 on real applications. Phase 13 adds a small number of *real* historical CVEs — vulnerable code extracted from the patched upstream project and held under a stable expected rule — so the benchmark floor is now defended by regression protection on demonstrably real bugs, not just synthetic analogues.

### What changed

- **New subtree**: `tests/benchmark/cve_corpus/<lang>/<CVE-ID>/` with a `vulnerable.*` and a `patched.*` file per CVE. Each file carries a header comment with the CVE ID, upstream project, upstream license, and advisory link.
- **Harness**: `tests/benchmark_test.rs::scan_corpus_file` now resolves any `file` entry whose path starts with `cve_corpus/` from the `benchmark_dir` (one level above `corpus/`) instead of the synthetic-corpus root. The change is a single if-branch; all existing synthetic cases are unaffected.
- **Ground truth**: six new cases added with `provenance: "real_cve"`. Vulnerable fixtures assert on `expected_rule_ids`; patched fixtures assert on `forbidden_rule_ids` so Nyx does not *refire* on the fix.
- **Regression thresholds unchanged**: floors stay at `P≥0.861 R≥0.944 F1≥0.901`. The Phase 13 rule-level F1 delta is +0.001 against the CF-7 baseline — the repo's policy ("tighten on durable, measurable improvements") does not justify movement on a corpus-expansion phase.

### Real-CVE Corpus

| CVE              | Language   | Project         | License    | Vuln class | Vulnerable outcome | Patched outcome |
|------------------|------------|-----------------|------------|------------|--------------------|-----------------|
| CVE-2023-48022   | Python     | Ray             | Apache-2.0 | CMDI       | TP (rule + line)   | TN              |
| CVE-2019-14939   | JavaScript | mongo-express   | MIT        | code_exec  | TP (rule + line)   | TN              |
| CVE-2023-26159   | TypeScript | follow-redirects | MIT        | SSRF       | TP (rule + line)   | TN              |

- **CVE-2023-48022** (Ray job-submission RCE). Vulnerable fixture: `request.get_json → os.system` with shell concatenation — Nyx fires `py.cmdi.os_system` and the cross-cutting `taint-unsanitised-flow` at the documented sink line. Patched fixture: `shlex.split → subprocess.run(argv, shell=False)` — zero findings.
- **CVE-2019-14939** (mongo-express `/checkValid` eval RCE). Vulnerable fixture: `req.body.document → eval("(" + document + ")")` — Nyx fires `js.code_exec.eval` and `taint-unsanitised-flow`. Patched fixture: `EJSON.parse(document)` inside a try/catch — zero findings.
- **CVE-2023-26159** (follow-redirects credential-leak / SSRF surface). Vulnerable fixture: `req.query.url → axios.get(target)` — Nyx fires `taint-unsanitised-flow` (no TypeScript SSRF-specific rule ID is emitted on this sink, which matches the rest of the TS SSRF corpus). Patched fixture: allowlist check over the parsed host + fixed internal URL handed to axios — zero findings.

Per-CVE precision/recall: each vulnerable case contributes 1 TP (and its patched sibling 1 TN), so per-CVE precision and recall are both 1.000 at the rule level.

### Delta

Aggregate rule-level F1 on the new 273-case corpus is 0.967 (P=0.942, R=0.994) — a hair above the pre-Phase-13 baseline of 0.966, and materially above the rule-level precision floor (0.894). The win is concentrated in honest regression protection on real code; precision edges up because the six new cases contribute 3 TP + 3 TN and no spurious firings on the fixes.

### Notes on selection

The starter set is intentionally small (1 CVE per stable-tier language, vulnerable + patched pair per CVE). Criteria applied when choosing each CVE: publicly disclosed with a known patch, vulnerability class that Nyx's existing rules cover (CMDI / code_exec / SSRF), extractable to ~30 LOC of representative code, permissive upstream license (Apache-2.0 / MIT) so the attribution header is sufficient. Fixtures are minimal reproducers of the *unsafe sink pattern*, not verbatim excerpts of upstream internals — the goal is regression protection on the documented pattern, not re-running the original exploit end-to-end.

---

## Phase CF-7 — Demand-driven backwards analysis (2026-04-22)

### Motivation

The forward taint engine proceeds source-to-sink, spending budget on
every function the source might touch.  Its precision ceiling is fixed
by what summaries + inline re-analysis can preserve on every edge of a
flow — a single lossy edge drops the finding.  This phase adds the
opposite direction: start at each sink value and walk *reverse* SSA
edges (and cross-file callee bodies via
`GlobalSummaries.bodies_by_key`) until a source is reached, the
accumulated predicate renders the flow infeasible, or a budget is
exhausted.  Off by default; benchmark is neutral.

### Changes

1. **`src/taint/backwards.rs`** — new module with the core types:
   `DemandState` (sink-side demand: caps + validated-predicate bits +
   cross-function depth), `BackwardFlow` (the reached verdict per
   walked value), `BackwardsCtx` (minimal driver-inputs view),
   `FindingVerdict` (Confirmed / Inconclusive / Infeasible /
   BudgetExhausted), and the `analyse_sink_backwards` driver.  The
   backwards transfer handles every `SsaOp` variant — `Assign`/`Phi`
   fan out to operands, `Call` tries cross-file body expansion before
   falling back to arg-fanout, `Source`/`Const`/`Param`/`CatchParam`
   terminate.  Source recognition also consults the defining CFG
   node's `DataLabel::Source(_)` so Python-style call-sites like
   `request.args.get` are treated as source terminals.  Budgets:
   `DEFAULT_BACKWARDS_DEPTH = 2`, `BACKWARDS_VALUE_BUDGET = 1024`,
   `MAX_BACKWARDS_CALLEE_BLOCKS = 500`.
2. **Finding annotation** (`src/taint/mod.rs`): after forward taint
   and symex complete, if `analysis.engine.backwards_analysis` is on,
   the pass walks each finding's sink and writes its verdict onto
   `Finding.symbolic.cutoff_notes` via `annotate_finding`.  Placed
   after symex so its witness-style `symbolic` output survives;
   backwards layers `backwards-confirmed` / `backwards-infeasible` /
   `backwards-budget-exhausted` onto the notes vector.
3. **Confidence integration** (`src/evidence.rs`):
   `compute_taint_confidence` treats `backwards-confirmed` as a
   `+1` signal and `backwards-infeasible` as a `-3` penalty (a
   smaller-magnitude signal than the symex verdict, which reasons
   about concrete payloads).  `compute_confidence_limiters` surfaces
   infeasible/budget verdicts as user-readable strings.
4. **Switch surfaces**: new `AnalysisOptions.backwards_analysis` field
   (default `false`), CLI pair
   `--backwards-analysis / --no-backwards-analysis`, and legacy
   env-var `NYX_BACKWARDS=1`.  Same tri-state pattern as the other
   engine toggles.
5. **Docs** (`docs/advanced-analysis.md`): new "Demand-driven
   analysis" section documents the pass, how to enable it, and the
   first-cut limitations (no reverse-call-graph expansion past
   `ReachedParam`; constraint pruning uses predicate-summary bits
   only, not the full SMT backend; depth-bounded at k=2).

### Test coverage

* **Unit tests** (`src/taint/backwards.rs` — 12 tests): demand-state
  seeding, backward transfer per op (`Source`, `Const`, `Param`,
  `Assign`, `Phi`), driver end-to-end on a trivial
  Source→Assign→sink body, phi fan-out producing per-predecessor
  flows, verdict aggregation (`Confirmed` beats `Infeasible`), and
  `annotate_finding` idempotence + inconclusive no-op.
* **Integration** (`tests/backwards_analysis_tests.rs` + 4 fixtures):
  `demand_driven_reach_source` confirms a SQL-injection source is
  reached and picks up `backwards-confirmed` when the switch is on;
  `demand_driven_prove_infeasible` locks in first-cut structural
  behaviour (SMT-backed prune is a follow-up); `demand_driven_catch_new_fn`
  locks in the first-cut ReachedParam termination; `demand_driven_no_source`
  regression-guards against synthetic findings on source-free code.  A
  fifth sub-case asserts backwards OFF is a strict no-op (no
  annotations appear).

### Benchmark delta

Off-by-default posture preserves the benchmark floor byte-for-byte
(P=0.940, R=0.994, F1=0.966 rule-level; P=0.941, R=1.000, F1=0.970
file-level).  On-path precision improvements require two follow-ups:
reverse-call-graph expansion for flows that escape a function's
`ReachedParam` boundary, and full SMT integration for the infeasible
path class.  Both are tracked as CF-7 follow-up work; the
off-by-default switch lets operators opt in without disturbing CI.

---

## Phase CF-6 — Parameter-granularity points-to summaries (2026-04-22)

### Motivation

Prior to CF-6, the cross-file summary channel had no way to express
"callee mutates a shared heap object through one parameter so another
parameter's alias sees the new taint."  Container-op patterns (`push`,
`set`, …) were already captured through `param_to_container_store`, but
direct field writes — `obj.x = val` — fell outside
`classify_container_op`'s recognised-method list, so a common class of
flow (void helper that stores through a parameter) was invisible to
cross-file taint.  Whole-program points-to is out of scope for a
security scanner; a minimal parameter-granularity summary closes the
real flows at a negligible cost.

### Changes

1. **`PointsToSummary` data type** (`src/summary/points_to.rs`):
   bounded `SmallVec<[AliasEdge; 4]>` of directed `(source, target,
   kind)` edges where endpoints are `AliasPosition::Param(u32)` or
   `AliasPosition::Return` and `AliasKind` is `MayAlias` only for CF-6.
   Edge count is capped at `MAX_ALIAS_EDGES = 8`; overflow sets an
   `overflow` flag that callers honour as "any param aliases any other
   param and the return" — the conservative greatest-lower-bound over
   the alias lattice.
2. **Intra-procedural analysis** (`src/ssa/param_points_to.rs`): a
   single bounded pass over the SSA body.  For each `SsaOp::Assign`
   whose `var_name` is a dotted/indexed path, we resolve the root base
   to a formal-parameter index via `formal_param_names` (authoritative
   declaration-order map) and trace the RHS through Assign/Phi chains to
   another parameter, emitting `Param(src) → Param(dst)`.  For each
   `Terminator::Return(Some(v))` whose value traces to a parameter we
   emit `Param(i) → Return`.  Declaration-order indexing matters: SSA
   lowering skips formal params that are never read, so SSA-level
   indices and caller-side positional indices can diverge.  Trusting
   formal-order is the only way to keep the summary's edges aligned with
   the caller's `args[i]` slots.
3. **`SsaFuncSummary.points_to`** (`src/summary/ssa_summary.rs`): new
   `#[serde(default, skip_serializing_if = PointsToSummary::is_empty)]`
   field.  Legacy on-disk rows deserialise cleanly with an empty
   summary, so no engine-version bump is required.
4. **Summary application at cross-file call sites**
   (`src/taint/ssa_transfer.rs`): `resolved_points_to` is captured
   alongside the other cross-file fields before `callee_summary` is
   moved into the main taint branch.  Each `Param(src) → Param(dst)`
   edge unions caller-`args[src]`'s taint into the heap of caller-
   `args[dst]`'s points-to set *and* directly taints the dst SSA
   value — the direct channel is necessary when the caller's heap
   analysis has no allocation site for the arg (common for plain
   constructors in Python / JS / Java).  Each `Param(src) → Return`
   edge threads caller-`args[src]`'s points-to set through
   `dynamic_pts` onto the call's return value.  Overflow synthesises
   the conservative all-pairs graph.
5. **`ssa_summary_fits_arity`** (`src/summary/mod.rs`): arity filter
   extended to reject points-to entries referencing parameters past the
   key's declared arity (same guard that `param_to_return` /
   `param_to_sink` already use).  Prevents synthetic-capture
   mis-attributions from leaking into cross-file resolution.
6. **Observable-effects filter** (`src/taint/mod.rs`): summary
   filtering in `lower_all_functions` now treats a non-empty
   `PointsToSummary` as an observable effect so void helpers whose only
   signal is a parameter alias survive the "no effects, skip" filter.

### Test coverage

* **Unit tests** (`src/summary/points_to.rs`): data-structure
  invariants (dedup, overflow promotion, serde round-trip, legacy JSON
  decodes).
* **Unit tests** (`src/ssa/param_points_to.rs`): 5 structural shapes
  (field-write emits edge, return-alias emits edge, self-alias is
  dropped, out-of-range param rejected, bounded graph terminates).
* **Summary serde + arity** (`src/summary/tests.rs`): round-trip with
  points_to populated, legacy JSON deserialises with empty points_to,
  arity filter rejects out-of-range param indices.
* **Cross-file integration** (`tests/cross_file_alias_tests.rs` + 3
  fixtures): `cross_file_alias_mutating_helper` (Python void helper →
  py.cmdi finding through param alias), `cross_file_alias_returned_alias`
  (JS passthrough → shell-exec finding through return alias),
  `cross_file_alias_bounded_graph` (Python 20-edge graph → scan
  terminates under the overflow fallback).

### Benchmark

Rule-level F1 unchanged at 0.966 (P=0.940, R=0.994); file-level F1
unchanged at 0.970 (P=0.941, R=1.000).  Neutral, as expected: the
existing benchmark corpus does not exercise cross-file field-alias
flows, so CF-6's precision win is latent and will surface as the corpus
grows.  All 2173 tests pass (1687 lib + 486 integration).

---

## Phase CF-5 — Cross-file SCC joint fixed-point (2026-04-22)

### Motivation

The pass-2 orchestrator already iterates mutually-recursive SCCs to
convergence on merged summaries (`MAX_SCC_FIXPOINT_ITERS`-bounded with a
`SCC_FIXPOINT_SAFETY_CAP = 64` guard).  Post-CF-1/CF-2, those iterations
run cross-file inline re-analysis under the *current* merged summaries on
each iteration, so the summary-equality convergence predicate implicitly
covers inline convergence for monotone summaries.  What was missing was
an explicit signal distinguishing *cross-file* SCCs (where the recursion
crosses file boundaries and the inline+summary interaction is what drives
precision) from *intra-file* SCCs (where the iteration is purely about
summary fixpoint).  Without that signal, cap-hit diagnostics conflated
the two root causes and the orchestrator could not target cross-file
SCCs for specialised handling.

### Changes

1. **`scc_spans_files()` helper + `FileBatch.cross_file` flag**
   (`src/callgraph.rs`): an SCC is flagged cross-file when its nodes
   belong to more than one namespace.  `scc_file_batches_with_metadata`
   unions the flag across all SCCs contributing to each topo batch.
   `cross_file ⊆ has_mutual_recursion` by construction (a non-recursive
   cross-file chain resolves topologically and is not batched).
2. **Inline cache lifecycle hooks** (`src/taint/ssa_transfer.rs`): new
   `inline_cache_clear_epoch()` and `inline_cache_fingerprint()` helpers
   give the SCC orchestrator a concrete contract for per-iteration cache
   semantics.  The per-file cache is already reconstructed fresh inside
   `analyse_file`, so today these are no-op plumbing — kept explicit so
   any future shared-cache refactor has a pre-agreed API.
3. **Cross-file-specific cap-hit tag** (`src/commands/scan.rs`):
   `SCC_UNCONVERGED_CROSS_FILE_NOTE_PREFIX` is a strict superset of
   `SCC_UNCONVERGED_NOTE_PREFIX`; callers filtering on the base prefix
   still match, while consumers that want the narrower cross-file case
   can match on the longer constant.  `tag_unconverged_findings()`
   takes a `cross_file: bool` switch and `run_topo_batches()` threads
   the batch flag through.
4. **Observability**: cross-file SCCs emit a dedicated `debug!` log at
   iteration start; cap-hit warnings carry the `cross_file = {bool}`
   field so operators can root-cause imprecision quickly.

### Fixtures and tests

- `tests/fixtures/cross_file_scc_mutual_recursion/` (Python, 2-file
  mutual recursion with CMDI sink): transitive taint must reach the
  caller across the cycle.
- `tests/fixtures/cross_file_scc_three_way_cycle/` (Python, 3-file
  cycle): pinned iteration envelope proves the SCC fix-point loop does
  the work, not topo order.
- `tests/fixtures/cross_file_scc_recursive_with_sanitiser/` (Python,
  2-file sanitised cycle): joint convergence carries the `shlex.quote`
  sanitizer across the cycle and suppresses the caller's CMDI.
- `tests/scc_cross_file_tests.rs`: wires the three fixtures into the
  integration harness.
- Callgraph unit tests: `scc_file_batches_with_metadata_marks_cross_file`,
  `scc_file_batches_with_metadata_intra_file_scc_not_cross_file`,
  `scc_spans_files_single_node`.
- Inline-cache lifecycle unit tests
  (`inline_cache_epoch_tests` in `src/taint/ssa_transfer.rs`):
  `clear_epoch_drops_all_entries`,
  `fingerprint_is_order_independent`,
  `fingerprint_changes_when_return_caps_change`,
  `fingerprint_tracks_missing_return_taint_as_zero`.
- Tag-variant unit tests (`scc_tagging_tests` in `src/commands/scan.rs`):
  cross-file and non-cross-file variants emit the expected prefixes.

### Benchmark delta

Byte-for-byte neutral vs CF-3 (P/R/F1 unchanged at 0.940 / 0.994 /
0.966).  The corpus exercises cross-file SCCs that already converge
cleanly under the existing summary-snapshot loop, so CF-5's value is
diagnostic clarity (tighter cap-hit tag, `cross_file` metric) and an
API surface the future joint-cache refactor can hook into — not a
precision shift on today's fixtures.

### Known limitations

- `inline_cache_clear_epoch` is a semantic hook, not a shared-cache
  lifecycle: the per-file cache is already ambient-cleared at each
  iteration via `analyse_file` reconstruction.  A true cross-file
  shared cache would be a more involved refactor (rayon-safe shared
  `RefCell<InlineCache>` across SCC files, epoch-tag invalidation on
  cache miss/hit).
- Benchmark-visible precision win will require corpus fixtures that
  specifically exercise cross-file SCCs with precision-degrading
  summary approximation; the current corpus's cross-file cycles all
  converge in 0–5 iterations and land on the same answer at both the
  summary and inline path.

---

## Phase CF-3 — Abstract-domain transfer channels in summaries (2026-04-22)

### Motivation

Phase 17 abstract interpretation tracks per-SSA-value intervals, string prefix/suffix facts, and known-bit masks during pass 2 and uses them to suppress findings via `is_abstract_safe_for_sink`. None of those facts crossed function boundaries through summaries: a caller that proved `port ∈ [1024, 65535]` lost the bound the moment the value entered a cross-file callee. CF-3 records, per parameter, a bounded symbolic description of how that parameter's abstract value maps to the return, so callers can synthesise the return abstract at summary-path call sites without re-running the callee.

### Changes

1. **`AbstractTransfer` domain** (`src/abstract_interp/mod.rs`) — product of bounded per-subdomain forms: `IntervalTransfer` (`Top` | `Identity` | `Affine { add, mul }` | `Clamped { lo, hi }`), `StringTransfer` (`Unknown` | `Identity` | `LiteralPrefix(String)` capped at `MAX_LITERAL_PREFIX_LEN = 64`). Bit subdomain is intentionally not carried cross-file.
2. **Summary schema** (`src/summary/ssa_summary.rs`): new `SsaFuncSummary.abstract_transfer: Vec<(usize, AbstractTransfer)>`, serde-gated so old DBs deserialise unchanged and only propagating functions contribute bytes.
3. **Pass-1 extraction** (`src/taint/ssa_transfer.rs::derive_abstract_transfer`): structural inference. *Identity* when every return-block return value traces (through single-use `Assign` and same-param `Phi` merges, depth ≤ 8) to the same `SsaOp::Param { index }`. *Clamped / LiteralPrefix* attached when the callee's baseline `return_abstract` has a bounded interval or known prefix.
4. **Pass-2 application** (`SsaOp::Call` arm of `transfer_inst`): runs whenever the callee was resolved via SSA summary. Per-param transfers evaluate on the caller's current abstract value of the argument, joined then `meet`-ed with baseline `return_abstract` (falling back to the less restrictive side if the meet contradicts).

### Fixtures and tests

- `tests/abstract_transfer_tests.rs` (29 tests): serde round-trip, per-subdomain `apply` semantics, join widening, LCP join on shared literal prefixes, and an end-to-end pass-1 structural test.
- `tests/fixtures/cross_file_abstract_port_range/`, `tests/fixtures/cross_file_abstract_bounded_index/`: cross-file summary-path regression guards.
- `tests/fixtures/cross_file_abstract_url_prefix_lock/`: JS literal-prefix SSRF suppression (landed via follow-up below).

#### CF-3 follow-up — JS literal-prefix SSRF suppression fix (2026-04-22)

Two surgical changes downstream of CF-3:

- **`src/ssa/copy_prop.rs`** — copy-prop now skips single-use `Assign` instructions whose CFG node carries `string_prefix`. Without this, copy-prop + DCE eliminated `url = 'lit' + userInput` in pass 2's optimised SSA, rewriting `fetch(url)`'s arg to the bare param and erasing the prefix. Mirrors the existing `is_numeric_length_access` guard.
- **`src/taint/ssa_transfer.rs::transfer_abstract`** — added a `Call` arm symmetric with the Assign-with-prefix arm: when a `Call` instruction's CFG node carries `string_prefix` (e.g. `url = wrapper('lit' + x)`), seed the call result's `StringFact` with the prefix. Lets `axios.get(url)` consume the prefix lock through cross-file identity-passthrough wrappers like CF-3's `asIs`.

Single-file and cross-file `'lit' + userInput → fetch/axios.get` both now produce zero findings.

### Benchmark delta

Byte-for-byte neutral vs pre-CF-3 (P/R/F1 unchanged at 0.940 / 0.994 / 0.966). Expected: the corpus does not yet exercise call chains where an identity-passthrough cross-file callee is the only thing between a caller-side abstract bound and a downstream suppression. The precision win will materialise when broader corpora exercise cross-file integer-bound propagation.

### Known limitations

- Per-return-path decomposition is CF-4's scope. A callee whose return traces to `param_0` on one branch and `param_1` on another yields `identity_consistent = false` and falls back to the baseline-invariant form (or Top).
- Only single-use `Assign` and consistent-origin `Phi` merges are followed by the Identity tracer; richer alias reasoning is CF-6.
- `Affine` is defined in the domain but the pass-1 structural inferrer never emits it yet.

---

## Phase CF-2 — Cross-file k=1 context-sensitive inline taint (2026-04-22)

Intra-file k=1 inline analysis (Phase 11) was extended to fire on cross-file call edges too. Before CF-2 every cross-file call collapsed into the callee's worst-case `SsaFuncSummary`; CF-2 exposes call-site-specific argument taint, call-site constants, and path-predicate structure to cross-file callees.

### Key changes

- **Cross-file body fallback in `inline_analyse_callee`** (`src/taint/ssa_transfer.rs`): intra-file lookup runs first; on miss, resolves the call via `GlobalSummaries.resolve_callee` and loads the body from `transfer.cross_file_bodies`. Body-size budget, k=1 depth cap, and the `context_sensitive` config switch shared with intra-file path via `InlineCache`.
- **Origin source-span pre-fill in param seed**: populate `source_span` from the caller's CFG before origins cross into a callee body, so cross-file inline preserves caller attribution.
- **Indexed-scan parity (CF-2 follow-up)**: `CrossFileNodeMeta` extended to carry full `NodeInfo` snapshot; `rebuild_body_graph` rehydrates a proxy `Cfg` at DB load time. `build_index` now persists `ssa_bodies` rows at index-build time (prior behaviour silently wrote zero bodies). Engine-version salt bumped to `+cf3-xfile-meta`.

### Fixtures

Four cross-file fixtures under `tests/fixtures/cross_file_context_*`: `two_call_sites` (Python, primary CF-2 win), `callback` (JS, callback-as-argument via summary path), `sanitizer` (JS, regression guard that CF-2 inline doesn't add findings where the summary path strips taint), `deep_chain` (Python three-file chain). Each has in-memory and indexed-scan test variants.

### Benchmark delta

Precision **+2.9pp** vs pre-CF-2 (0.911 → 0.940); recall unchanged (0.994); F1 **+1.5pp** (0.951 → 0.966). No per-language regression; Python/Rust/TypeScript at 1.000, others ≥ 0.889. Indexed-scan parity follow-up was neutral (correctness fix, not a precision delta).

### Known limitations

- k=1 is preserved: cross-file inline will not recursively inline the next cross-file hop. CF-5 (SCC joint fixed-point) revisits this for mutually recursive cross-file SCCs.

---

## History

Earlier phases, most recent first. Metrics are rule-level unless noted.

| Date       | Phase                                  | Corpus | P      | R      | F1     | Notes |
|------------|----------------------------------------|--------|--------|--------|--------|-------|
| 2026-04-20 | Rust Weak Spot Fixes                   | 262    | 0.906  | 0.994  | 0.948  | Rust FN→0 across FILE_IO/SSRF/SQL/DESERIALIZE sink families; SHELL_ESCAPE added to Phase 10 type suppression; identity-method peeling for constructor typing; Rust rule-level P/R/F1 jumped +7.8/+21.1/+13.2pp. |
| 2026-04-20 | TypeScript Weak Spot Fixes             | 262    | 0.899  | 0.981  | 0.938  | Closed all three Phase 19 TS weak spots: encodeURIComponent→axios cap-overlap (StringFact prefix-locked SSRF suppression), Fastify framework detection from in-file imports, TSX/JSX grammar wiring. TS rule-level F1 → 1.000. |
| 2026-04-20 | Rust Honesty Expansion                 | 262    | 0.891  | 0.961  | 0.925  | Rust corpus expanded 18→31 cases with honest FNs in classes lacking Rust rules (SQL, deserialize, reqwest builder chains). Correction, not a regression. |
| 2026-04-20 | TypeScript Coverage Expansion          | 246    | 0.904  | 0.986  | 0.944  | TS corpus 0→32 cases (12 vuln classes, adversarial type-system stressors, framework/cap-overlap/interproc cases). |
| 2026-03-24 | Phase 19 — Benchmark Expansion         | 214    | 0.827  | 0.950  | 0.885  | +73 cases (+52%); C, C++, Rust added as first-class languages; interprocedural + path-pruning cases; `buffer_overflow` and `fmt_string` classes for C/C++. Thresholds reset to baseline −5pp. |
| 2026-03-22 | Phase 8.5 — Cross-file SSA validation  | 141    | 0.840  | 0.975  | 0.903  | `param_to_sink_param` field on `SsaFuncSummary`; directory-based multi-file benchmark cases; 6 new cross-file cases across PY/JS/Go (propagation, source detection, wrong-cap sanitizer) — all TP. |
| 2026-03-22 | Ruby Parity                            | 123    | 0.821  | 0.986  | 0.896  | Ruby corpus 1→21 cases across 8 vuln classes; no label rule changes. |
| 2026-03-22 | Phase 5 — SSA Lowering X-lang hardening| 103    | 0.841  | 0.983  | 0.906  | PHP closures + throw; Python try/except + raise; new exception-edge fixtures. Precision +17.0pp vs Phase 30 via confidence scoring / allowlist / type-check guards. |
| 2026-03-21 | Phase 30 — SSRF Semantic Completion    | 103    | 0.671  | 0.966  | 0.792  | New SSRF sink matchers (axios, got, undici, httpx, http.NewRequestWithContext, Net::HTTP, HTTParty, requests.*); `flask_request.*` source; Ruby added to corpus. |
| 2026-03-21 | Phase 22.5b — Constant-arg suppression | 95     | 0.654  | 0.964  | 0.779  | AST + CFG suppression of calls with all-literal args; removed buggy `!source_derived` guard in `guards.rs`. |
| 2026-03-21 | Phase 22.5                             | 95     | 0.624  | 0.964  | 0.757  | py-ssrf-001 rule-ID fix; bare `exec`/`execSync` as JS cmdi sinks; Python `Template` as XSS sink. |
| 2026-03-21 | Phase 22 baseline                      | 95     | 0.620  | 0.891  | 0.731  | First benchmarked baseline post-Phase-22 symbolic strings. |

### Recurring known limitations

- **Variable-receiver method calls** (e.g. `client.send(...)` vs `HttpClient.send(...)`): suffix-matching misses without type-qualified resolution. Partly addressed by Phase 10 type-aware callee resolution; residual cases remain where the receiver has no inferred type.
- **Import aliasing**: arbitrary import aliases (`from flask import request as r`) are not traced; only explicitly listed aliases resolve.
- **No SSRF sanitizers as function calls**: URL-parsing doesn't sanitize; allowlist checks are condition patterns, modelled via `classify_condition()` validation markers rather than call-site credits.
- **Rust structural `cfg-unguarded-sink`** still fires for SHELL_ESCAPE when a source is in scope but not flowing to the sink arg — intentional for high-risk sinks; requires plumbing `TypeFactResult` into `AnalysisContext` to suppress.
- **Rust negative-validation `contains` dominators** and **match-arm guards** are not yet modelled by `classify_condition()`.
- **DNS-rebinding / async callback flows**: out of scope for static analysis without runtime context.