A security gate that measured nothing and still said OK

tests/security/mutation-gate.sh, issue #3583

The paired probe

Same command, same tree, minutes apart. The only difference is the Claude Code Bash sandbox.

mode	candidates	proven CAN FAIL	NOT provable	exit	verdict
sandboxed (default)	13	0	0	0	OK
sandbox disabled	13	11	2	0	OK

Both print a green OK. The first measured nothing at all.

Why

mutation-gate.sh:116   RESULT_DIR="$(mktemp -d -t ork-mutate-XXXXXX)"

mktemp -d -t PREFIX resolves to the darwin per-user temp directory and ignores $TMPDIR. When that is denied, mkdtemp fails, RESULT_DIR becomes the empty string, every parallel worker writes its result to /test-*.sh, and the reduce loop over ls "$RESULT_DIR" reads nothing back. Zero proven and zero non-provable then compare clean against a baseline of 2 and the gate exits 0.

It also gives harmful advice. The vacuous run prints: non-provable count fell to 0. Lower MUTATION_BASELINE to 0 to lock the gain in. Following that would zero a security ratchet on the strength of a measurement that never happened, and the ratchet is monotonic.

The fix: fail closed, twice

guard	refuses when
scratch dir	mktemp uses a TMPDIR-honouring template; refuse if the directory is missing or unwritable
accounting	every candidate must produce exactly one result file; a missing file is an un-measured test, not a passing one

Both follow the shape the script already used at line 101: No test-*.sh files found ... refusing to report success. MUTATION_BASELINE stays at 2. Raising it would suppress a correct signal, and the gate's complaint was true.

Evidence for this change

run	result	meaning
patched, sandbox disabled	10 proven, 2/2 baseline, exit 0	honest path unchanged, no regression
patched, sandboxed	non-provable rose to 5, exit 1	now measures and reports honestly instead of passing blind
patched, TMPDIR unwritable	named error, exit 1	early refusal with a diagnosis

An honest limit on the third row. The unpatched gate also exits 1 under an unwritable TMPDIR, for an unrelated reason: every test scores NO_ASSERTION. So that condition does not reproduce the fail-open. The real evidence is the sandboxed paired probe in the first table.

Blast radius

Same family as #3564 and the merged #3579, which gave every mktemp a template so it would honour TMPDIR. That sweep did not reach line 116. Until this lands, every green mutation-gate result obtained inside a CC sandbox is meaningless, including local pre-push runs.