# Phase-2 A/B — ledger-routed vs full verification (source/, 2026-06-12)

The first controlled test of the [REVIEW-LEDGER](../REVIEW-LEDGER.md) protocol:
one shared candidate set over `crates/pixtuoid-core/src/source/` at `a8aaae9`,
verified twice — a **control** arm (full adversarial pair — design-intent
skeptic + code-trace verifier — on ALL candidates) and a **treatment** arm
(ledger-routed: matched candidates get one cheap checker; everything else gets
the same full pair). Pipeline: 4 lens finders → dedup (10 distinct candidates)
→ ledger routing → both arms. Workflow `wf_04e5f98b-735`.

## Routing

9 of 10 candidates routed **unmatched** (genuinely new claims — mostly
comment-accuracy and missing-breadcrumb findings the prior reviews never
raised); 1 routed **regression-check** (C08, the #243–#256 subagent-arc
regression sweep, matched to ledger rows R0610-01 + SUB-01/04/05/06/07 +
GEO-01/02).
Zero demotions: no candidate re-derived a previously-refuted claim. The
routing agent applied the protocol's discipline unprompted — same-seam
candidates were sent to full verification with explicit "same seam ≠ same
claim" reasoning citing the socket-steal case.

## Results

| property | result |
|---|---|
| **false suppressions** (quality guard) | **0** of 1 routed candidate (the regression-check route; the demote route never fired, so its false-suppression rate is still unmeasured) — the cheap check agreed with control's full pair (conf 90 vs 85/88) |
| verdict parity, unrouted candidates | 8/9 same outcome across arms (3 identical clear verdicts + 2 agreeing modulo one residual↔confirmed slot + 3 identical 2-2 splits; strict pair-verdict identity 6/9); 1 cross-arm flip (C07, conf-35) **between two full pairs** — verifier variance, not a ledger effect |
| per-routed-candidate cost | cheap check 6,620 output tokens vs 17,064 full-pair average — **61% saving where routing fires** |
| arm totals (output tokens) | control 170,643 vs treatment 174,408 — **±0 at a 10% route rate** (single-candidate saving inside inter-run variance) |
| arm totals (cache-write) | control 5.31M vs treatment 4.76M |

## Honest read

1. **The safety property held in this run** — but only the regression-check
   route was exercised (n=1); the demote path, the one that can hide a real
   bug, fired zero times. Its false-suppression rate stays an open
   measurement for the next calibration.
2. **The economics scale with re-tread rate, and this corpus had almost
   none.** The module's three prior reviews ended in FIXES (#234–#237, #240)
   — fixed findings leave regression-check rows, not refutations to demote
   against — and fresh finder lenses surfaced fresh claims. The
   repeat-refutation pattern the ledger targets (June 9 → June 10
   whole-codebase runs re-refuting `/tmp`-socket, EMFILE, Storm>Rain) comes
   from same-shape reviews at near-identical HEADs; that is where the 61%
   per-candidate saving multiplies. A fresh-claim review sees no benefit and
   — by design — no harm.
3. **Full-pair verification has its own variance floor**: C07 (confidence 35)
   flipped confirmed↔refuted between two full pairs given identical prompts.
   Below ~conf-50, a single pair's verdict is not stable ground truth —
   which is an argument for the ledger's append-only flip-tracking, and for
   weighting adjudications by confidence when they become rows.

## Disposition

The experiment's clear adjudications are appended to the ledger as
R0612-01…05 (three 2-2 splits stay un-adjudicated, listed for re-derivation);
confirmed low findings (stale WHY comments, missing warn breadcrumbs, the
symlink-recursion guard — plus C07, filed with an explicit re-verify-premise
caveat since its conf-35 cross-arm flip sits below the stability floor of
point 3 above) are tracked in issue #262 rather than fixed here —
this branch is documentation scope. Next calibration: re-run the A/B on the
next whole-codebase review, where re-tread density is the realistic worst
case.
