EPIC B's probe verifies ork's allow-list against the shipped binary and fails loudly on
drift. But its check is missing-only: it flags events the binary names that the allow-list
lacks, and is structurally silent when the allow-list asserts an event the binary never corroborates.
Six of nine entries live in that silent half, and one of them (PostCompact) already has
independent evidence of being wrong (#3321).
before OK: every event the binary names is present in the generated allow-list. after allow-list: 9 asserted / 3 corroborated by the binary / 6 unverified WARN: no binary string names these events as additionalContext consumers; the missing-only check above cannot see them (#3418): PostCompact PostToolUse PostToolUseFailure PreToolUse SessionStart UserPromptSubmit OK: every event the binary names is present in the generated allow-list.
The nightly probe workflow tees this output into its run summary and keys only off exit codes, so the numbers reach the summary with no workflow change.
Absence from the binary's prose is absence of documentation, not proof of absence of support. #3386's lesson cuts both ways: a hand-typed list that drifted stripped valid output from six live hooks. Deleting the six uncorroborated entries on prose evidence alone could do exactly that again. So the gate now names them on every run, and each gets settled empirically — emit a marked additionalContext on that event and see whether the model receives it — before any is promoted to a failure or removed.
spec/cc-output-keys.spec.yml (header): "Every entry below carries the binary string that justifies it. An entry without evidence is not a contract, it is a guess." its additionalContext entry: 9 events its evidence block: 4 quoted strings 1 generic field-list line names NO event 3 event-naming lines Stop · SubagentStop · PostToolBatch corroborated by its own bar: 3 guesses: 6
Its header said Regenerate: node scripts/derive-cc-output-keys.mjs. That script only
ever reads the module (it imports it to compare against the binary); there is no write path
anywhere in it. A maintainer following the header's own instruction got a printed report and an
unchanged file, and could reasonably believe the file had been re-derived when nothing was rewritten.
The header now states the real workflow: edit the spec, mirror it here in the same commit, and let
--check arbitrate against the binary.
| gate | result |
|---|---|
--check on 2.1.227, new output | exit 0 · 9/3/6 printed · six named |
| the six match the independently derived set | byte-identical to the #3418 measurement |
| DRIFT path (missing-only, must stay fatal) | untouched, still exit 1 |
output-guard-cc-contract.test.ts | 18 passed |
| hooks + plugin build | exit 0, zero drift beyond the edited files |
What this sets up. The six now have names in every nightly summary instead of
hiding behind a green OK. Settling them one by one is the follow-up recorded on #3418, and
PostCompact goes first because #3321 already reports a live hook emitting
additionalContext into it that the model never sees.