# Cycle γ D1b — iterative retrieval INSUFFICIENT + Phase C.2 bottleneck correction

**Date**: 2026-06-10 (same session as Phase C.2 + D1)
**Status**: D1b (iterative retrieval) **INSUFFICIENT**. More importantly,
chasing why it failed **corrected the Phase C.2 diagnosis**: retrieval
is NOT the dominant bottleneck — retrieve(top-8) already catches both
supporting paragraphs 52% of the time; the loss is downstream
(rerank/truncation + synth distractor-noise).

> This handover supersedes the "retrieval = dominant bottleneck"
> framing of `v0.4-cycle-gamma-phase-c2-musique-retrieval-bottleneck-2026-06-10.md`
> (PR #752). That conclusion was built on a **sources(top-3 shown)**
> measurement artifact. The mechanism finding (single-shot misses the
> 2nd hop *in the top-3*) stands; the *attribution* to retrieval was
> wrong.

---

## 1. D1b result (pre-registration §3 verdict)

D1b = iterative / self-ask: decompose → answer hop-1 from its own
retrieval → substitute the answer into hop-2 → 2nd retrieval round.
Pre-registered (`1b280b8`, before impl).

| Metric (mxtral n=25) | R0 | D1 static | **D1b iter** | oracle |
|---|---|---|---|---|
| gold-in-answer | 8% | 12% | **12%** | 72% |
| supporting recall (sources top-3) | 0.600 | 0.600 | **0.600** | — |
| both-supporting (top-3) | 5/25 | 5/25 | **5/25** | — |
| abstain | 19/25 | 20/25 | **18/25** | 6/25 |

**Verdict: INSUFFICIENT** (gold-in-answer < 15% AND supporting recall
flat). The iterative mechanism **fired correctly** — log shows real
substitutions: "Who is the founder of **Órion Pictures**?", "Which
company owns **Bombardier**?", "Where does **Ulrich Walter** work?" (the
extract→resolve step replaced the D1 anaphora). But it changed nothing,
because the bottleneck is not at the retrieve step the sub-questions
feed.

---

## 2. The correction — retrieval is not the bottleneck

While diagnosing D1b's null effect, a direct measurement of
**retrieve(top-8)** (not the sources(top-3) proxy Phase C.2 used):

| supporting-paragraph recall, n=25 | value | both-supporting |
|---|---|---|
| sources (top-3 shown) — Phase C.2 metric | 0.600 | 5/25 |
| **retrieve (top-8) — actual search** | **0.760** | **13/25** |

The original query's hybrid search **already retrieves both supporting
paragraphs 52% of the time** (13/25). Example: query "Who founded the
company that distributed UHF?" (support_idx [6,10]) — original retrieve
top-8 = [9,4,10,18,6,0,7,6], both 6 and 10 present. Phase C.2's
"retrieval misses the 2nd hop" was a **sources(top-3) truncation
artifact** — exactly the `feedback_oracle_phrase_artifacts` trap (a
display-side top-3 cut read as a retrieval failure).

### Where the answer is actually lost

retrieve catches supporting (0.76) but gold-in-answer is 8%. The loss is
between retrieve and answer:

1. **rerank + top-3/top-5 truncation** — the cross-encoder reranks by
   the *original* query, so the hop-2 supporting paragraph (query-
   dissimilar by construction) is down-ranked and cut before synth.
2. **synth distractor-noise** — even when more docs reach synth
   (top_k 16/10 = 10 docs to synth, supporting almost certainly
   included) the model still abstains 80%, vs the oracle (clean
   supporting-only) 72% solve. The model cannot pick the 2 supporting
   paragraphs out of the distractors.

oracle (clean, 2 paras) 72% vs noisy (supporting + distractors)
abstain 80% ⇒ **the dominant lever is synth's handling of noisy
multi-hop context**, not retrieval breadth, not decomposition, not
iterative bridging.

---

## 3. What this means for the cycle

Three retrieval-side interventions all produced null results, and now
we know **why** they were null:

| Intervention | Result | Why null (corrected) |
|---|---|---|
| breadth (top_k 8→16) | no change | retrieve already had supporting |
| D1 static decompose | INSUFFICIENT | hop-2 anaphora (+ bottleneck is downstream anyway) |
| D1b iterative | INSUFFICIENT | substitution worked, but bottleneck is downstream |

The lever is **downstream of retrieval**: rerank's multi-hop handling
and/or synth's noise robustness. This re-points the next work.

---

## 4. Next — measure the real lever (re-register)

Pre-register and measure, on the same 25 MuSiQue queries, primary axis
gold-in-answer + a **synth-context** supporting-recall (not sources
top-3):

1. **rerank OFF / rerank top_k up** — does preserving the hop-2
   supporting paragraph into synth lift gold-in-answer? (cheap, env-gate
   already exists: `JAMES_DISABLE_RERANK`, `JAMES_RERANK_TOP_K`.) This
   directly tests lever (1).
2. **synth context audit** — for each query, is the supporting paragraph
   actually in the synth prompt? If yes-but-abstains → lever (2), the
   synth/abstention layer. Instrument the synth context, not sources.
3. If lever (2): the question becomes synth noise-robustness on
   multi-hop — connect to Phase E-min (rerank/cog_stages effect on
   abstention) rather than retrieval.

The honest tier of any fix stays ⭐⭐ (external bench, prior-art
mechanisms). No default flip without multi-axis (MuSiQue gain vs RGB
abstention loss vs cost).

---

## 5. Artifacts

| Purpose | Path |
|---|---|
| D1b pre-registration (LOCKED before impl) | `docs/research/cycle-gamma-d1b-iterative-retrieval-preregistration-2026-06-10.md` |
| Iterative retriever | `core/retrieval/iterative_retriever.py` |
| Decomposer (+ `force`, `decomp_model`) | `core/retrieval/query_decomposer.py` |
| Wiring (D1b precedence) | `core/reasoning/pipeline_loops.py` |
| Result JSONs | `reports/cycle_gamma/phase-c2/musique-ans-mxtral-{R0,D1,D1b}.json` |
| Phase C.2 handover (corrected by this doc) | `docs/handovers/v0.4-cycle-gamma-phase-c2-musique-retrieval-bottleneck-2026-06-10.md` |

### Carry-forward
- **`feedback_oracle_phrase_artifacts` confirmed twice this cycle**: the
  sources(top-3) proxy misread retrieval coverage. Always measure at the
  actual stage (retrieve top-k / synth context), not the display field.
- All D1/D1b infra is **default-OFF byte-identical** (verified). Negative
  evidence preserved (D2 V2 / PR #746 pattern).
- Honest framing: D1b reported INSUFFICIENT; the Phase C.2 attribution
  self-corrected from new evidence (measurement-driven, 9th-catch
  pattern) rather than being defended.
