# Cycle γ — retrieval arc CLOSURE: synth noise-robustness is the lever

**Date**: 2026-06-10 (closes the MuSiQue retrieval investigation)
**Status**: Cycle γ retrieval arc **CLOSED**. Five measurements ruled
out the entire retrieval side; the binding lever is **synth's handling
of noisy multi-hop context** — the model gets the supporting paragraphs
and still abstains.

---

## 1. The closing measurement (D2)

Pre-registered (`72dfa2a`, before run). Preserve both supporting
paragraphs into synth by disabling rerank and widening top_k, then ask:
does the model solve it, or still abstain on the distractors?

| | gold-in-answer | abstain | synth sees |
|---|---|---|---|
| R0 (rerank ON, top-5) | 8% (2/25) | 19/25 | rerank top-5 |
| **D2 (rerank OFF, top-16)** | **12% (3/25)** | 18/25 | all 16 retrieved docs |
| oracle (clean 2 paras) | 72% (18/25) | 6/25 | supporting only |

**§3 verdict: SYNTH noise-robustness is the binding lever** (D2 < 15%).
Giving the model the supporting paragraphs *plus distractors* lifts it
only 8%→12% (noise band). The same paragraphs *alone* (oracle) → 72%.
The 60pp gap is entirely distractor noise.

---

## 2. The full cycle γ retrieval chain — five exclusions

| Measurement | Result | Ruled out |
|---|---|---|
| breadth (top_k 8→16) | no change | search breadth |
| oracle (clean evidence) | 72% | model capability |
| D1 (static decompose) | 12% | query decomposition |
| D1b (iterative self-ask) | 12% | iterative bridging |
| **D2 (rerank OFF + wide)** | **12%** | **rerank / truncation** |
| **→ binding lever** | | **synth noisy-multihop evidence use** |

Plus the mid-cycle correction: retrieve(top-8) supporting recall = 0.76,
both 13/25 — retrieval was never the bottleneck (Phase C.2's
sources(top-3) attribution was a truncation artifact,
`feedback_oracle_phrase_artifacts`).

---

## 3. What's actually true now

```
retrieve(top-8)        → both supporting in candidates: 13/25 (52%)   ✅ enough
  rerank(top-5)        → both preserved to synth:        8/25         (drops 5)
    synth (R0)         → gold-in-answer:                 2/25 (8%)    (drops 6)
D2: rerank OFF, 16 docs → both preserved:                ~13/25
    synth (D2)         → gold-in-answer:                 3/25 (12%)   (still drops ~10)
oracle: supporting only → synth → gold-in-answer:        18/25 (72%)
```

The model can do the 2-hop reasoning (oracle 72%). It cannot pick the 2
supporting paragraphs out of 14 distractors (D2 12%). **The lever is
evidence cleanliness at the synth boundary, not retrieval, rerank, or
query rewriting.**

---

## 4. Next — two candidate directions (re-register before building)

### 4a. Multi-hop-aware evidence selection (retrieval-arc adjacent)
The oracle proves: feed clean supporting → 72%. The job is to *select*
the supporting paragraphs out of the retrieved set before synth.
- rerank ranks by the *original* query → misses hop-2 (query-dissimilar).
  The D1/D1b sub-questions (already built, default-OFF) could rank
  evidence **per sub-question** instead of feeding extra retrieval paths:
  take the top doc for each sub-q → small, clean synth context.
- This reuses the decomposer for *selection* rather than *retrieval* —
  the piece D1/D1b had backwards.

### 4b. Synth noise-robustness (connects to Phase E-min)
The deeper finding is that synth abstains on noisy multi-hop context.
Phase E-min already measured `rerank` / `cognitive_stages` effects on
RGB abstention. The same synth/abstention layer is the limiter here.
- Measure: does a synth-side evidence-compression / "answer using only
  the most relevant 2-3 passages" instruction lift gold-in-answer toward
  the oracle ceiling?
- Connect the multi-hop finding to the abstention-family axes (the LOSS
  half from Phase E-min) — they may be the same underlying synth
  behaviour.

Both keep the ⭐⭐ ceiling (external bench, prior-art mechanisms) and the
default-OFF / multi-axis-before-flip discipline.

---

## 5. Artifacts

| Purpose | Path |
|---|---|
| D2 pre-registration (LOCKED before run) | `docs/research/cycle-gamma-d2-rerank-vs-synth-lever-preregistration-2026-06-10.md` |
| D2 result JSON | `reports/cycle_gamma/phase-c2/musique-ans-mxtral-D2-rerankoff.json` |
| All cycle γ retrieval JSONs | `reports/cycle_gamma/phase-c2/musique-ans-mxtral-{R0,R0-topk16,ORACLE,D1,D1b,D2-rerankoff}.json` |
| D1b handover (correction) | `docs/handovers/v0.4-cycle-gamma-d1b-iterative-results-and-bottleneck-correction-2026-06-10.md` |
| Phase C.2 handover (corrected) | `docs/handovers/v0.4-cycle-gamma-phase-c2-musique-retrieval-bottleneck-2026-06-10.md` |

### Carry-forward
- **No code in D2** — measurement only (env-gates already existed).
- `feedback_oracle_phrase_artifacts` fired twice this cycle (sources
  top-3 misread). Always measure at the actual stage.
- `feedback_evidence_grounded_validity_check`: the oracle test cleanly
  separated capability from evidence; reuse for any RAG floor.
- Honest framing: every null reported as null; the Phase C.2 attribution
  was self-corrected from new evidence (9th-catch / measurement-driven).
- v0.5 retrieval-quality (D2 gate) evidence: JAMES retrieval is **not**
  the multi-hop weak point — synth evidence-selection is. This reframes
  any "improve retrieval" roadmap item toward the synth boundary.
