Gate fault-arm audit: what remains

Lane gate-marker-audit on yonatangross/orchestkit. Snapshot generated 2026-09-06 12:43. Every number marked MEASURED was printed by a probe or a suite on this machine; CLAIMED means read from code or a screenshot, not executed.

1. The shape, before and after

The #3933 shape: a gate whose "nothing found" result is byte-identical to "checked and clean". Every consumer reads the empty result as pass. The fix is the same everywhere: assert the input exists and is non-empty before judging it.

flowchart LR
  A["input absent: empty dir, empty JSON file, failed fetch, wrong path"] --> B["loop never iterates, or a guard short-circuits"]
  B --> C["counter stays 0"]
  C --> D{"failures == 0 ?"}
  D -->|yes| E["PASSED: All 0 skills ... exit 0"]
  style E fill:#3a1d1d,stroke:#f85149,color:#fff
  style A fill:#1d2230,stroke:#58a6ff,color:#fff

flowchart LR
  A["input absent"] --> B["loop never iterates"]
  B --> C["counter stays 0"]
  C --> F{"scanned == 0 ?"}
  F -->|yes| G["FAIL: zero X found, nothing was checked, exit 1"]
  F -->|no| D{"failures == 0 ?"}
  D -->|yes| E["PASSED with N above 0"]
  style G fill:#1d3a24,stroke:#3fb950,color:#fff
  style E fill:#1d3a24,stroke:#3fb950,color:#fff
  style A fill:#1d2230,stroke:#58a6ff,color:#fff

Five shapes found, by count

pie showData title FAILS OPEN by mechanism (37)
  "loop never iterates, N=0 passes" : 22
  "presence guard turns absence into skip" : 7
  "swallowed failure read as empty result" : 5
  "jq | while subshell drops the fail counter" : 2
  "dead guard (nullglob, path that no longer exists)" : 1

2. Every gate

Columns: control = control-arm exit at HEAD 911cdd8bf; fault before = fault-arm exit at HEAD; fault after = re-run on this branch (MEASURED for every fixed row). Filter by state.

3. Verify latest

branch: feat/gate-marker-audit
main:   c492d3221 chore(main): release 10.0.0-alpha.81 (#3934)

9a6720fe1 docs(audits): gate fault-arm audit 2026-09-06, plus three last guards
d461c1764 fix(ci): fail 27 more gates on absent input, with paired fault arms
960a09e1c fix(ci): fail three gates that passed on absent input

tests/ci/test-run-tests-empty-roster.sh           3 passed, 0 failed
tests/ci/test-check-skill-structure.sh            6 passed, 0 failed
tests/ci/test-audit-skill-claims-blind-sweep.sh   7 passed, 0 failed
tests/manifests/test-fault-arms.sh                10 probes, 0 failed
tests/plugins/test-fault-arms.sh                  6 probes, 0 failed
tests/ci/test-fault-arms.sh                       9 probes, 0 failed
tests/ci/test-workflow-step-fault-arms.sh         26 passed, 0 failed
real-repo controls: validate-counts PASSED, sync-skills OK,
  lint.sh 6/6, check-workflow-gates OK, index-generation OK,
  stamp-counts --check OK, eval:static rc 0, actionlint clean

npm test --quick: IN PROGRESS at render time (48 passed, 2 failed so far)
CI on the PR: not opened yet at render time (waits on the suite line)
main moved to alpha.81 (c492d3221) while this ran; no file overlap, rebase pending

4. What remains: milestone board and the path through it

From the milestones page (CLAIMED from the screenshot, 10 open, 124 closed) plus gh issue list per milestone (MEASURED). The recommendation is to close the two near-complete milestones before touching the four opened on 2026-09-02.

flowchart TD
  PR["this PR: 37 gates fail closed, audit doc, paired arms"] --> E3306["issue 3306 EPIC A: DELETE what never ran"]
  PR --> I3937["issue 3937 validate-all.sh orphan: delete or fix"]
  I3937 --> E3306
  E3306 --> V10["v10.0.0 Convergence: 3 open, goes to 2"]
  I3448["issue 3448 brainstorm loses teammate research on timeout"] --> V10
  I3424["issue 3424 sandbox loose-first: a decision, not code"] --> V10
  I1848["issue 1848 wire claude agents cwd into ork dev"] --> CC["CC adoption: 1 open, closes"]
  V10 --> NEW["four 09-02 milestones: Sales, Glyph docs, page serve, Commerce. 0 of 9 done, hold until v10 closes"]
  H["Hook and Agent Quality: 8 open"] --> H1["issue 3493 git-validator remediation bug"]
  H --> H2["issue 3000 seed quality-index baseline"]
  H --> H3["6 enhancement items: park"]
  style PR fill:#1d3a24,stroke:#3fb950,color:#fff
  style V10 fill:#1d2230,stroke:#58a6ff,color:#fff
  style CC fill:#1d2230,stroke:#58a6ff,color:#fff
  style NEW fill:#3a2d1d,stroke:#d29922,color:#fff
  style H3 fill:#2a2f3a,stroke:#9aa3b2,color:#ddd

5. Follow-ups found by the audit, outside this PR

• filed #3937 tests/plugins/validate-all.sh: orphan, passes on absent input, fails on the real tree. Delete, or fix the ork-* glob and zero-count returns and rename to test-validate-all.sh.
• fails closed test-build-drift.sh section B: Apple diff -q - cannot read stdin from a pipe, so every agent is CHANGED on macOS (36 of 36 measured). Local-only red, CI green.
• inert at HEAD test-marketplace-structure.sh Tests 4 and 5 capture the literal git-subdir from object sources; MISSING_PATHS and MISSING_MANIFESTS are structurally 0. Test 2 was already moved to jq.
• wrong root test-cc-discovery-simulation.sh audits src/agents when plugin.json carries no agents key, while the plugin ships plugins/ork/agents.
• claimed readme-link-audit.yml: key the clean branch on the output being present, not on == 0.
• by design version-check.yml skips every conventional branch prefix (release-please owns versions); only the empty-diff hole was fixable.