We never tested the rules we shipped

Every rule in the operator permission payload, executed against a real trip case · 2026-08-31

Why this exists

The #3835 purge deleted 16 blocking hooks and handed their job to one JSON file that /ork:setup writes into the operator's own settings. The existing canary proves the mechanism: a synthetic deny rule blocks a Bash call under --dangerously-skip-permissions with no hooks loaded. That is a different claim from "the 53 rules we ship work". An untested rule in that file is a hole with a comment on it.

53 rules, four honest outcomes

27 green   PROVEN. Control arm executed, trip arm with that one rule did not.
10 blue    INERT AS SPELLED. No Write() spelling denies anything; the Edit() twin does.
 4 amber   REDUNDANT. Control carried NO rule and CC denied anyway. We did not need them.
12 grey    NOT PROVEN, by design. Running the trip case IS the harm the rule prevents.

None of the four is a silent pass. "Not proven" is printed as not proven, with a named live case of the same matcher shape next to it. A probe that reports a verdict on something it never examined is the defect this repo has hit nine times.

What it found on the first run

rulemeasured
Bash(find:*-delete*)matched nothing. Byte-identical outcome to running with no rule at all: 0 permission_denials, the file deleted. Bash(find *-delete*) denies
Bash(find:*-fprint*)same
CC's colon form is a PREFIX SEPARATOR. It works as cmd:*  ->  Bash(find:*) denies.
Text placed after the star kills the match     ->  Bash(find:*-delete*) denies nothing.

Where they came from: both were copied verbatim out of src/settings/ork.settings.json,
the plugin settings file CC never reads (it takes only agent + subagentStatusLine).
Nothing had ever exercised their syntax, so the typo survived for as long as the file
was inert. #3836 then promoted them into the file that IS read, still broken.

A rule that never ran was never tested. Moving it somewhere real is what exposed it.

The two that were mine, not the payload's

Both produced confident REGRESSED verdicts on rules that do in fact deny.
Both were checked against a standalone experiment BEFORE being believed.

1. tab is IFS *whitespace*, so `IFS=$'\t' read` collapses a run of tabs.
   Every bash case has an empty `target` column, so it vanished and every later
   field shifted one left: `effect_sh` arrived empty and `eval ""` returns 0,
   which scores the trip arm as "the command ran". Fix: US (0x1f) as delimiter.

2. `( sleep N; kill "$STUB_PID" ) &` expands STUB_PID AFTER the sleep, by which
   time it names a later arm's stub. The watchdog shot a live arm. It also
   inherited stdout, so its lingering sleep held the pipe open and every case
   cost exactly the timeout: 46s per case for work that takes 1.8s.

Guarding the next rule

payload-coverage.py compares the case map against the payload in BOTH directions:
  a rule with no case      -> exit 2, because it would ship untested
  a case with no rule      -> exit 2, a fixture outliving its rule reads as coverage
  a twin naming a rule the payload does not ship -> exit 2

That is the check the plugin's own settings.json never had, which is how it carried
26 rules that enforced nothing from v7.0.0 until this week.