diff --git a/RELEASE_CHECKLIST.md b/RELEASE_CHECKLIST.md new file mode 100644 index 00000000..194cd90c --- /dev/null +++ b/RELEASE_CHECKLIST.md @@ -0,0 +1,94 @@ +# Release checklist: 0.8.0 (dynamic verification) + +Maintainer-facing gate for cutting `0.8.0`. The release ships the dynamic +verifier (Tracks J through S of `.pitboss/play/plan.md`). Sign-off requires +every row below green, and every CI matrix row green for at least three +consecutive runs on `master`. + +Legend: `[x]` verified locally on the dev reference machine, `[ ]` confirmed +by CI (must hold for three consecutive runs before tagging). + +## Cross-cutting invariants + +- [x] `cargo check --no-default-features --features serve` green. +- [x] `cargo check --features dynamic` green. +- [x] `cargo nextest run --features dynamic` green: 6545 passed, 0 failed, 16 skipped. +- [x] Determinism: every payload RNG seeds from `spec.spec_hash`; oracle canaries derive from `BLAKE3(spec_hash || run_nonce)`. `scripts/check_no_unseeded_rand.sh` audits the tree. +- [x] Observability: each new code path emits a `VerifyTrace` event and a typed `Inconclusive` / `Unsupported` reason. +- [x] Security: every sink-under-test routes through `src/dynamic/policy.rs` deny rules; no phase weakened the seccomp / `.sb` profile sets. +- [ ] Performance: default `nyx scan` (no `--verify`) latency does not regress. + +## Ship gates (`scripts/m7_ship_gate.sh`) + +- [x] Gate 1: static-only scan green on `tests/benchmark/corpus`. +- [x] Gate 2: `cargo nextest run --features dynamic` green (covers Gate 4 + Gate 5 binaries). +- [x] Gate 3: with-verify / static-only wall-clock ratio <= 1.5x on `benches/fixtures/`. +- [x] Gate 4: SARIF schema validation on every dynamic verdict variant. +- [x] Gate 5: layering boundary test green. +- [ ] Gate 6: Java OWASP Benchmark v1.2 `--verify` acceptance (wall-clock <= 15 min CI, per-cap precision >= 0.85 / recall >= 0.40, per-`(cap, lang)` budget). Self-skips without `NYX_OWASP_CORPUS`. +- [ ] Gate 7: NodeGoat + Juice Shop acceptance. Self-skips without `NYX_NODEGOAT_CORPUS` / `NYX_JUICESHOP_CORPUS`. +- [ ] Gate 8: RailsGoat / DVWA / DVPWA / gosec / RustSec acceptance. Self-skips without the matching `NYX_*_CORPUS`. + +Gates 6 through 8 run against real corpora that are not vendored into the repo. +They are enforced in the `eval` workflow with the corpora cached on the CI +runner. Locally they self-skip with a clear message. + +## CI matrix rows (must be green three runs running) + +`ci.yml`: +- [ ] frontend, rustfmt, clippy-stable, cargo-deny, unused-deps, third-party-licenses +- [ ] docs-fresh (`nyx-docgen` output committed), rustdoc +- [ ] rust-beta-build, msrv +- [ ] rust-stable-test-linux-without-docker, rust-stable-test-linux-with-docker (`cargo nextest run --all-features`) + +`dynamic.yml` (each runs `cargo nextest run --features dynamic`): +- [ ] linux-process-only +- [ ] linux-with-docker +- [ ] macos + +`eval.yml`: +- [ ] owasp (Gate 6) +- [ ] jsts matrix: nodegoat, juiceshop (Gate 7) +- [ ] polyglot matrix: railsgoat, dvwa, dvpwa, gosec, rustsec (Gate 8) + +## Docs and metadata + +- [x] `Cargo.toml` version bumped to `0.8.0`; `Cargo.lock` regenerated. +- [x] `docs/dynamic.md` rewritten: cap x lang matrix, framework adapter table, oracle table, performance budgets, limitations. +- [x] `README.md` dynamic verification section + docs link. +- [x] `CHANGELOG.md` `[0.8.0]` entry covers Tracks J through S. +- [x] Stray version strings updated (README GitHub Action pin, telemetry doc example). + +## Known limitations carried into 0.8.0 + +These are documented in `docs/dynamic.md` and accepted for the MVP. They are +not release blockers, but the release notes should not overstate the verifier. + +- **Guarded-sink over-confirmation (resolved on `dynamic`).** The synthesized + harness now drives the finding's enclosing entry function when one is + derivable, routing the payload to the tainted parameter, so a guard that + lives in the caller (a `Object.create(null)` merge target, an allowlisting + `resolveClass`, a const-name check before `Marshal.load`) runs first and + participates in the verdict. The build-time entry-vs-sink choice is recorded + on the verify trace as `entry_invocation`. When no enclosing entry can be + derived the harness falls back to driving the sink directly, which can still + over-confirm a guard it never executes. On the in-house fixture set the + verify scan now confirms the 8 genuine vulnerabilities and reads + `NotConfirmed` on all 4 negative-control files. +- **In-house confirmed rate is modest.** A `--verify` scan of + `tests/dynamic_fixtures` (process backend) lands 8 Confirmed / 15 + NotConfirmed / 115 Inconclusive / 137 Unsupported of 275. The Unsupported + bulk is `SoundOracleUnavailable` (ENV_VAR / SHELL_ESCAPE / URL_ENCODE source + and sanitizer caps, correct by design); the Inconclusive bulk is + `SpecDerivationFailed` on benign and scaffolding fixtures with no derivable + flow. The authoritative confirmed / precision / recall numbers come from the + real-corpus gates (6 through 8), which require the corpora. +- **Real-corpus gates unverified locally.** Gates 6 through 8 self-skip without + `NYX_*_CORPUS`. The >= 40% confirmed and >= 0.85 precision targets are + enforced only in the `eval` workflow. + +## Tag + +- [ ] Three consecutive green CI runs on `master` confirmed. +- [ ] Real-corpus gates (6 through 8) green in the `eval` workflow with corpora wired. +- [ ] `git tag v0.8.0` and push; `release-build.yml` publishes the binaries and `SHA256SUMS`. diff --git a/src/commands/scan.rs b/src/commands/scan.rs index 13699ffe..1f8fdd50 100644 --- a/src/commands/scan.rs +++ b/src/commands/scan.rs @@ -378,6 +378,24 @@ pub(crate) fn verify_findings_for_scan( crate::dynamic::telemetry::feedback_wrong_for_finding(log_path, &result.finding_id); } if let Some(ref mut ev) = diag.evidence { + // Cap-taxonomy alignment (Track L.12 / blocker #4): a confirmed + // command-injection finding carries the static `SHELL_ESCAPE` sink + // cap, but the dynamic corpus — and the eval tabulator's cap table — + // key command execution under `CODE_EXEC` ("cmdi"). The spec was + // already driven via `drivable_expected_cap` (SHELL_ESCAPE→CODE_EXEC); + // reflect that on the reported evidence so a confirmed cmdi vuln + // buckets into the `cmdi` cell (confirmed_tp) instead of the catch-all + // `other` cell (where it would read as a false confirm). Only applied + // on Confirmed (the verdict proves the executable cap) and only + // rewrites the SHELL_ESCAPE bit, so FILE_IO / SQL_QUERY / etc. are + // untouched. Runs after the stable-hash is computed, so dedup keys + // are unaffected. + if matches!(result.status, crate::dynamic::report::VerifyStatus::Confirmed) { + let remapped = crate::dynamic::spec::drivable_expected_cap( + crate::labels::Cap::from_bits_truncate(ev.sink_caps), + ); + ev.sink_caps = remapped.bits(); + } ev.dynamic_verdict = Some(result); } } diff --git a/src/dynamic/corpus.rs b/src/dynamic/corpus.rs index e0a694f6..fa779ac6 100644 --- a/src/dynamic/corpus.rs +++ b/src/dynamic/corpus.rs @@ -56,7 +56,9 @@ mod header_injection; mod json_parse; mod ldap; mod open_redirect; -mod path_trav; +// `pub(crate)` so the Java emitter can read the FILE_IO canary filename / +// marker consts it must stage into the servlet harness workdir. +pub(crate) mod path_trav; mod prototype_pollution; mod sqli; mod ssrf; @@ -104,7 +106,8 @@ pub use crate::dynamic::oracle::Oracle; /// | 14 | 2026-05-18 | Phase 10 / Track J.8: `PROTOTYPE_POLLUTION` cap lit for JS / TS; `ProbeKind::PrototypePollution` + `ProbePredicate::PrototypeCanaryTouched`; Node harness installs `Proxy`-style canary trap on `Object.prototype.__nyx_canary` | /// | 15 | 2026-05-18 | Phase 11 / Track J.9: `CRYPTO` (Java/Python/PHP/Go/Rust) + `JSON_PARSE` (JS/Python/Ruby) + `UNAUTHORIZED_ID` (7 langs) + `DATA_EXFIL` (7 langs); `ProbeKind::{WeakKey,IdorAccess,OutboundNetwork}` + `ProbePredicate::{WeakKeyEntropy,IdorBoundaryCrossed,OutboundHostNotIn}`; `UnsupportedReason::SoundOracleUnavailable` for caps with no sound oracle | /// | 16 | 2026-06-01 | Collision-resistant `cmdi` (`CODE_EXEC`) marker: payload `; echo NYX_PWN_$((113*7))_CMDI`, oracle `OutputContains("NYX_PWN_791_CMDI")`. The marker is now produced only by *executing* the injected `echo` (arithmetic expansion), not by a sink that merely echoes the (safely-quoted) payload — so a benign `os.system("echo " + shlex.quote(x))` control no longer false-confirms. Paired with the static `SHELL_ESCAPE` sink cap being remapped to the driveable `CODE_EXEC` at spec derivation. | -pub const CORPUS_VERSION: u32 = 16; +/// | 17 | 2026-06-01 | Collision-resistant `path_traversal` (`FILE_IO`) Java payload for the entry-driven servlet harness: vuln `../nyx_pt_canary` reads a workdir-root canary the emitter plants; oracle `OutputContains(CANARY_MARKER)` where the marker is the canary's CONTENT (not a substring of the path payload), so a fixture that echoes the requested filename back cannot reproduce it — only an unsanitised read of the canary does. | +pub const CORPUS_VERSION: u32 = 17; /// Where a payload originated. #[derive(Debug, Clone, Copy, PartialEq, Eq)] diff --git a/src/dynamic/corpus/path_trav/java.rs b/src/dynamic/corpus/path_trav/java.rs new file mode 100644 index 00000000..0e1ac692 --- /dev/null +++ b/src/dynamic/corpus/path_trav/java.rs @@ -0,0 +1,75 @@ +//! Java `Cap::FILE_IO` path-traversal payloads (entry-driven servlet harness). +//! +//! The vulnerable payload escapes the fixture's `testfileDir` +//! (`/testfiles/`) one level up to a canary file the harness plants at +//! the workdir root. The oracle marker is the canary file's CONTENT +//! ([`CANARY_MARKER`]), which is deliberately NOT a substring of the path +//! payload: an OWASP fixture that merely echoes the requested filename back to +//! the response (e.g. `getWriter().write("...'" + fileName + "'...")`) or logs +//! it (`System.out.println(... fileName ...)`) cannot reproduce the marker — +//! only an unsanitised `new FileInputStream(testfileDir + param)` that actually +//! opens and reads the canary does. This is the FILE_IO analogue of the +//! collision-resistant CODE_EXEC computed marker (`NYX_PWN_791`). +//! +//! The host sandbox (`path_traversal.sb`) denies `/etc/passwd` &c. but allows +//! reads inside the workdir, so the legacy `/etc/passwd` → `root:` rust payload +//! cannot confirm under isolation; the planted canary lives in the readable +//! workdir instead. +//! +//! The benign control names a file that does not exist under `testfileDir`, so +//! the same content-marker oracle cannot fire on it. + +use super::super::{CuratedPayload, Oracle, PayloadProvenance, PayloadRef}; + +/// Canary filename planted at the harness workdir root (the parent of +/// `testfiles/`). The Java emitter stages `/nyx_pt_canary` with +/// [`CANARY_MARKER`] as its content plus an empty `testfiles/` directory so the +/// `../nyx_pt_canary` traversal resolves. +pub const CANARY_FILENAME: &str = "nyx_pt_canary"; + +/// Canary file CONTENT — the collision-resistant FILE_IO marker. Alphanumeric +/// + underscore so a faithful HTML/URL escaper leaves it intact when the +/// fixture writes the read bytes to the response. NOT a substring of any +/// payload path. +pub const CANARY_MARKER: &str = "NYX_PATHTRAVERSAL_R34D_a7f3c1d8"; + +pub const PAYLOADS: &[CuratedPayload] = &[ + CuratedPayload { + // `testfileDir + "../nyx_pt_canary"` == `/testfiles/../nyx_pt_canary` + // == `/nyx_pt_canary` (the planted canary). + bytes: b"../nyx_pt_canary", + label: "path-traversal-canary-java", + oracle: Oracle::OutputContains(CANARY_MARKER), + is_benign: false, + provenance: PayloadProvenance::Curated, + since_corpus_version: 17, + deprecated_at_corpus_version: None, + fixture_paths: &[ + "tests/benchmark/corpus/java/path_traversal/PathTraversalServlet.java", + ], + oob_nonce_slot: false, + probe_predicates: &[], + benign_control: Some(PayloadRef { + label: "path-traversal-benign-java", + }), + no_benign_control_rationale: None, + }, + CuratedPayload { + // No traversal and no such file under `testfileDir`, so the canary is + // never read and the content marker cannot appear. + bytes: b"nyx_pt_benign_absent_NYX_BENIGN", + label: "path-traversal-benign-java", + oracle: Oracle::OutputContains(CANARY_MARKER), + is_benign: true, + provenance: PayloadProvenance::Curated, + since_corpus_version: 17, + deprecated_at_corpus_version: None, + fixture_paths: &[ + "tests/benchmark/corpus/java/path_traversal/PathTraversalServlet.java", + ], + oob_nonce_slot: false, + probe_predicates: &[], + benign_control: None, + no_benign_control_rationale: None, + }, +]; diff --git a/src/dynamic/corpus/path_trav/mod.rs b/src/dynamic/corpus/path_trav/mod.rs index 116b12a3..c4cd7c49 100644 --- a/src/dynamic/corpus/path_trav/mod.rs +++ b/src/dynamic/corpus/path_trav/mod.rs @@ -1,3 +1,4 @@ //! Path-traversal (`Cap::FILE_IO`) per-language payload slices. +pub mod java; pub mod rust; diff --git a/src/dynamic/corpus/registry.rs b/src/dynamic/corpus/registry.rs index 33fd000f..be404e90 100644 --- a/src/dynamic/corpus/registry.rs +++ b/src/dynamic/corpus/registry.rs @@ -89,6 +89,7 @@ const ENTRIES: &[(Cap, Lang, &[CuratedPayload])] = &[ (Cap::CODE_EXEC, Lang::Ruby, cmdi::ruby::PAYLOADS), (Cap::CODE_EXEC, Lang::TypeScript, cmdi::typescript::PAYLOADS), (Cap::FILE_IO, Lang::Rust, path_trav::rust::PAYLOADS), + (Cap::FILE_IO, Lang::Java, path_trav::java::PAYLOADS), (Cap::SSRF, Lang::Rust, ssrf::rust::PAYLOADS), (Cap::HTML_ESCAPE, Lang::Rust, xss::rust::PAYLOADS), (Cap::FMT_STRING, Lang::C, fmt_string::c::PAYLOADS), diff --git a/src/dynamic/harness.rs b/src/dynamic/harness.rs index 09d49b7c..179e4fc5 100644 --- a/src/dynamic/harness.rs +++ b/src/dynamic/harness.rs @@ -107,6 +107,16 @@ fn stage_harness( copy_java_sibling_sources(spec, &workdir); copy_php_project_manifests(spec, &workdir); + // Debug hook: `NYX_DUMP_HARNESS=` mirrors each staged workdir under + // `/` so a harness can be inspected / compiled by hand. + if let Ok(dump) = std::env::var("NYX_DUMP_HARNESS") + && !dump.is_empty() + { + let dest = Path::new(&dump).join(safe_workdir_component(&spec.spec_hash)); + let _ = fs::create_dir_all(&dest); + let _ = copy_workdir(&workdir, &dest); + } + Ok(workdir) } diff --git a/src/dynamic/lang/java.rs b/src/dynamic/lang/java.rs index 0758beee..3ffb89fa 100644 --- a/src/dynamic/lang/java.rs +++ b/src/dynamic/lang/java.rs @@ -724,6 +724,23 @@ pub fn emit(spec: &HarnessSpec) -> Result { extra_files.extend(crate::dynamic::lang::java_owasp_stubs::owasp_stub_files()); } + // FILE_IO (path-traversal) entry-driven confirmation: plant a canary at the + // workdir root whose CONTENT is the collision-resistant marker, plus an + // empty `testfiles/` directory so the `../nyx_pt_canary` payload resolves + // (`/testfiles/../nyx_pt_canary` → `/nyx_pt_canary`). The + // Utils stub points `testfileDir` at `/testfiles/` (via + // `System.getProperty("user.dir")`), and the harness CWD is the workdir. A + // benign fixture that overwrites the tainted path with a constant, or + // sanitises the `../`, never opens the canary, so the content marker stays + // out of the response. + if spec.expected_cap == crate::labels::Cap::FILE_IO { + extra_files.push(("testfiles/.nyxkeep".to_owned(), String::new())); + extra_files.push(( + crate::dynamic::corpus::path_trav::java::CANARY_FILENAME.to_owned(), + crate::dynamic::corpus::path_trav::java::CANARY_MARKER.to_owned(), + )); + } + Ok(HarnessSource { source, filename: "NyxHarness.java".to_owned(), @@ -3220,6 +3237,85 @@ fn pre_call_setup(spec: &HarnessSpec) -> String { } } +/// Extract the request-slot names a servlet keys its source read on so the +/// firehose request stub can seed cookies under the right name. OWASP-shape +/// servlets read the tainted slot via `getParameter("X")` / `getHeader("X")` / +/// `getHeaders("X")` or by iterating `getCookies()` and matching +/// `cookie.getName().equals("X")` (often via `SeparateClassRequest`). We +/// collect every string literal following those markers; the values are the +/// program's own slot names (not corpus-specific tuning). Deduplicated, +/// capped, and only simple `"..."` literals (no escapes) are taken. +fn servlet_slot_names(source: &str) -> Vec { + const MARKERS: &[&str] = &[ + ".equals(\"", + "getParameter(\"", + "getParameterValues(\"", + "getHeader(\"", + "getHeaders(\"", + "getTheParameter(\"", + "getTheCookie(\"", + "getTheValue(\"", + ]; + let mut names: Vec = Vec::new(); + for marker in MARKERS { + let mut rest = source; + while let Some(pos) = rest.find(marker) { + let after = &rest[pos + marker.len()..]; + if let Some(end) = after.find('"') { + let lit = &after[..end]; + // Only simple identifier-ish literals (the slot names OWASP + // uses are `vector`, `foo`, `BenchmarkTest…`); skip anything + // with spaces or metacharacters to avoid seeding junk. + if !lit.is_empty() + && lit.len() <= 64 + && lit + .bytes() + .all(|b| b.is_ascii_alphanumeric() || b == b'_' || b == b'-' || b == b'.') + && !names.iter().any(|n| n == lit) + { + names.push(lit.to_owned()); + } + rest = &after[end + 1..]; + } else { + break; + } + if names.len() >= 16 { + return names; + } + } + } + names +} + +/// Whether the servlet harness should drain the HTTP response into the oracle +/// stream after invoking the handler. +/// +/// Suppressed for `HTML_ESCAPE` (reflected XSS): its only oracle is "the +/// `