# Next Session Entry — α-6 Phase 3a Ready

> **Status**: cycle pause point — Phase 2 closed (M_S verdict
> REVERSES the tier-gated hypothesis).
> **Cycle PRs landed this session**: 65 → 75+ (see §6 PR ledger)
> **Project total**: 75+ PRs (#608 → #674+)
> **α-6 cycle PRs to date**: ~28
>
> **Resume entry point**: read this file → confirm Ollama healthy →
> execute §4 first command.

---

## 0. Where the project is

- **Version**: v0.4.1 (latest minted DOI). α-6 cycle is
  measurement / methodology; no version bump yet.
- **Active theme**: α-6 Phase 3a — **localize the S5+S6 capability
  floor** (gemma3 scale ladder).
- **Cumulative JAMES code change against α-5+α-6 measurement debt**:
  **0 lines**.
- **Cumulative wrong-fix-averted**: **9** (7 in α-5 + 1 in α-6
  Phase 2 rate-limit corruption + 1 in α-6 Phase 3a matrix runner
  tier→model silent substitution, PR #677). 9th sub-pattern variant:
  "wrong measurement averted" rather than "wrong code fix averted".

---

## 1. Headline finding this session (Phase 2 closure)

**The original tier-gated routing hypothesis (small models gain more from JAMES) is REVERSED.**

| | M_M (gemma4:e4b) | M_S (gemma3:4b) |
|---|---|---|
| pure LLM abst_f1 | 0.558 | 0.074 |
| + JAMES full stack abst_f1 | **0.591** ⬆️ | **0.000** ⬇️ |
| Δ from S5+S6 | +0.033 (modest gain) | **-0.074 (loss!)** |
| latency multiplier | 5.5× | 8.4× |

**Publishable framing**: *"JAMES's abstention + cognitive layers
are a capability amplifier, NOT a small-model crutch."* They
require sufficient model metacognitive ability to engage
meaningfully.

Additional finding (Phase 1 + Phase 2 combined): **RAG retrieval
is tier-conditional** — helps small (graded +0.043 at M_S), hurts
large (graded -0.020 at M_M). Lost-in-the-Middle reverses below
the capability threshold.

---

## 2. Routing policy preview (Phase 3a localizes the floor)

| Tier | Recommended |
|---|---|
| gemma4:e4b (M_M production) | **Adopt full JAMES stack** — S5+S6 measurably help |
| gemma3:4b (M_S small) | **S4 citation only** — S5+S6 produce zero recovery, skip the 8.4× latency tax |
| Capability floor | between 4b and e4b — **Phase 3a localizes** |

Phase 3a goal: identify *at which gemma3 scale (1b, 4b, 12b, 27b)
does S5+S6 recovery cross zero into positive territory*. Hypothesis:
between 4b and 12b.

---

## 3. What just landed this session (PRs #663 → #674+)

| PR | Title | What |
|---|---|---|
| #663 | matrix runner `--sector-cells` extension | wire 5 sector flags into cell envs |
| #664 | renderer sector-cells + progression table | --render-report picks up C_* cells |
| #665 | Phase 1 analysis TEMPLATE | 8-section operator-fill format |
| #666 | Phase 2 entry doc | tier-gated check at gemma3:4b — runbook |
| #667 | alpha6_fill_phase1.py | Phase 1 closure auto-fill |
| #668 | Phase 1 analysis filled | operator commentary + Phase 2 decision |
| #669 | 4 Phase 1 findings logged + memory drafts | promotion script run |
| #670 | α-6 cycle PR index | navigable index for 60 PRs |
| #671 | rate-limit corruption post-mortem (8th wrong-fix-averted) | discipline log + arithmetic step lesson |
| #672 | rate-limit env override + bench retry-on-429 + matrix runner bypass | fix |
| #673 | alpha6_fill_phase1.py tier-parametrized | Phase 2/3 reuse |
| #674 | Phase 2 closure — tier-gated REVERSED | publishable framing |
| (this PR) | session close — Phase 2 findings + handover | this doc |

---

## 4. First actions next session (in order)

### Action 0 — Ollama healthcheck (mandatory)

T1's timeout cascade (UUU #656) plus Phase 2's rate-limit corruption
(IIII #671) both showed measurement infrastructure can degrade
silently. Restart Ollama before launch:

```powershell
Restart-Service Ollama
ollama list  # gemma3:1b / 4b / 12b / 27b should respond snappy
```

### Action 1 — Phase 3a launch (gemma3 scale ladder)

Goal: localize the S5+S6 capability floor.

**Tier flag, NOT env var** — `qvt_ablation_matrix.py` overrides
`JAMES_LLM_MODEL` from `_TIER_MODELS[tier]` in `_build_env` (line 411),
so `$env:JAMES_LLM_MODEL = ...` was silently ignored. PR #677 added the
M_XS / M_XL tier mappings so `--tiers M_XS / M_L / M_XL` resolves to
gemma3:1b / 12b / 27b directly. Cell filenames also get distinct
`-M_XS` / `-M_L` / `-M_XL` suffixes — no collision with canonical
Phase 2 `M_S = gemma3:4b` cells.

```powershell
$env:JAMES_WORKSPACE = "./workspaces/hotpot_eval"
$env:PYTHONIOENCODING = "utf-8"
$env:PYTHONUNBUFFERED = "1"

# Step 1 — gemma3:1b via M_XS tier (extreme small / below-floor probe)
python -u scripts/qvt_ablation_matrix.py `
    --tiers M_XS --suite multihop_rag --n-runs 1 `
    --sector-cells C_minus,C_rag-full
# ~10-15 min compute (fast on 1b; watch for rate limit)

# Step 2 — gemma3:12b via M_L tier (ladder midpoint)
python -u scripts/qvt_ablation_matrix.py `
    --tiers M_L --suite multihop_rag --n-runs 1 `
    --sector-cells C_minus,C_rag-full
# ~30-90 min compute

# Step 3 — gemma3:27b via M_XL tier (saturation point probe)
python -u scripts/qvt_ablation_matrix.py `
    --tiers M_XL --suite multihop_rag --n-runs 1 `
    --sector-cells C_minus,C_rag-full
# ~30-90 min compute
```

**ETA total**: 2.5-3.5h (much less than the original 12-18h estimate
because we only run 2 cells per scale — C_minus + C_rag-full — not
the full sector ablation).

### Action 2 — Phase 3a analysis (post-completion)

```powershell
# Per scale point — match --tier to the tier you ran the cells under:
python scripts/alpha6_fill_phase1.py `
    --tier M_XS --phase-label "Phase 3a (gemma3:1b)" `
    --out reports/research-runs/alpha-6-phase-3a-gemma3-1b-analysis-20260602.md

python scripts/alpha6_fill_phase1.py `
    --tier M_L --phase-label "Phase 3a (gemma3:12b)" `
    --out reports/research-runs/alpha-6-phase-3a-gemma3-12b-analysis-20260602.md

python scripts/alpha6_fill_phase1.py `
    --tier M_XL --phase-label "Phase 3a (gemma3:27b)" `
    --out reports/research-runs/alpha-6-phase-3a-gemma3-27b-analysis-20260602.md

# Then a separate combined view doc (manual)
# showing the recovery curve across 1b/4b/12b/e4b/27b
```

Plot the **abst_f1 recovery** (= C_rag-full abst_f1 − C_minus abst_f1)
across scales — this is the publishable finding.

### Action 3 — Phase 3b (cross-family, optional)

Same {C_minus, C_rag-full} pair across qwen2.5:7b / llama3.1:8b /
deepseek-v2:16b. Tests whether capability floor is gemma-specific.

Compute: ~1.5-2h. Recommended only if Phase 3a confirms a clean
capability floor signal.

### Action 4 — Phase 3a/3b closure + α-6 cycle close

```
- ROADMAP §v0.4.x α-6 entry — replace "in flight" with closure
- Backlog handover §7.11 with α-6 closure verdict
- α-6 cycle PR index (#670) update to final
- Closure PR with cumulative publishable narrative
```

---

## 5. Critical gotchas (carry-over)

1. **4-step rule + arithmetic step** — when any axis looks saturated:
   - axis 0/extreme → trigger
   - read 3-5 raw samples
   - check JAMES response keys
   - reconcile design intent vs matcher
   - **NEW (IIII #671)**: check cell wall-clock vs latency × n_queries

2. **Per-layer intent verdict, not uniform 5-axis** — CLAUDE.md rule 2
   layer-intent extension (#652). Each layer judged on its
   design-intent axes.

3. **Rate-limit env now bypassed in matrix runner** (#672) — but if
   running bench.py standalone, set
   `JAMES_RATE_LIMIT_MAX=10000` manually.

4. **DON'T frame methodology as Replayable RAG validation** —
   position guard from `feedback_jameses_positioning_replayable_rag`
   §"정정 2026-05-31". Mother-platform framing primary.

5. **`fix` label exemption for measurement-side PRs** — CLAUDE.md
   rule 2 `fix` exemption. PRs correcting oracle / fixture / bench /
   matrix-runner wiring state `Quality delta: exempt (label: fix)`.

6. **Matrix runner tier→model is hardcoded** (#677, 9th wrong-fix-
   averted, before-launch variant) — `_TIER_MODELS[tier]` overwrites
   `JAMES_LLM_MODEL` in `_build_env`. Use `--tiers <tier>` only; env
   var override is inert. Cell filenames key on tier letter (not
   model tag), so same-tier-different-model runs collide — backup
   first. Caught here by latency arithmetic (1-2s expected on 1b →
   12-26s on C_rag-full meant 4b was running). See memory
   `feedback_matrix_runner_tier_model_override`.

6. **DOI hold** — no new JAMES Zenodo DOI until T3 ships AND
   mid-June joint piece framing confirmed (per SSS #655 4-question
   gate).

---

## 6. Phase 3a expected pattern (predictions to compare)

Hypothesis (to be tested):

| Model | Pure LLM abst_f1 | + JAMES full stack | S5+S6 recovery Δ | Conclusion |
|---|---|---|---|---|
| gemma3:1b | very low (~0.0-0.1) | 0.0 | 0 | below floor |
| gemma3:4b | 0.074 (observed) | 0.000 (observed) | -0.074 | below floor (Phase 2) |
| gemma4:e4b | 0.558 (observed) | 0.591 (observed) | +0.033 | above floor (Phase 1) |
| gemma3:12b | ~0.6-0.7 expected | ~0.7-0.8 expected | +0.05-0.10 expected | well above floor |
| gemma3:27b | ~0.8 expected | ~0.85 expected | +0.05 expected | saturated |

**Floor estimate**: between 4b and e4b (e4b is ~5b effective).
Phase 3a confirms by measuring the inversion point.

If reality differs from prediction → publishable surprise →
apply 4-step rule before drawing conclusions.

---

## 7. Memory entries to read on entry

- `feedback_oracle_phrase_artifacts` — 4-step rule (확장 1+2+3)
- `mechanism_layer_intent_axis_alignment` — per-layer intent
  matrix + backend-healthy prerequisite
- `feedback_jameses_positioning_replayable_rag` (정정 2026-05-31) —
  mother framing primary
- `feedback_intent_vehicle_gap_collective_archival` — Vehicle A/C
  separation (DOI hold)
- `project_state` — high-level current state

Drafts pending operator review in `reports/research-runs/promoted-findings/`:
- `finding_jameses_s5_s6_capability_floor_not_crutch.md` (NEW)
- `finding_tier_conditional_rag_helps_small_hurts_large.md` (NEW)
- `finding_s4_citation_tier_invariant.md` (NEW)
- `finding_jameses_rate_limit_corruption_arithmetic_step.md` (NEW)
- + 4 from morning Phase 1

---

## 8. Quick orientation queries

```powershell
# What's in flight?
git log --oneline -20

# What changed this session?
git diff HEAD~13..HEAD --stat

# Cycle index for navigation
cat reports/research-runs/alpha-6-cycle-pr-index.md

# Latest cycle verdict (Phase 2)
cat reports/research-runs/alpha-6-phase-2-analysis-20260601.md

# This handover (you're reading it)
cat docs/handovers/v0.4-next-session-entry-2026-06-01-PM.md
```

---

## 9. Done condition for next session

A clean stop point is when:

- [ ] Action 0 (Ollama healthy) confirmed
- [ ] Action 1 (Phase 3a 1b / 12b / 27b ladder) at least one new
      scale point measured
- [ ] Action 2 (analysis filled per scale)
- [ ] Phase 3a abst_f1 recovery curve visualised (or stated in prose)
- [ ] α-6 cycle closure PR drafted (optional — can defer)

If Action 1 finds the capability floor cleanly at 1 or 2 scale
points, Phase 3b (cross-family) becomes lower priority. If the
gemma3 ladder shows ambiguous transition, Phase 3b becomes
critical for disambiguation.

---

## 10. After-cycle follow-up tracks (still gated)

| Track | Trigger |
|---|---|
| Graph top-K filtering (finding #3) | post Phase 3 — informed by cross-tier data |
| AUTO_ROUTER multi-tier engineering | post α-6 closure |
| MultiHop-RAG paper baseline reproduction (#651) | independent ~5h, anytime |
| α-7 typed relations schema entry | after T3 / α-6 close |
| DOI bump | T3 ship + mid-June joint piece framing |
| Joint piece preprint | 6/6+ Ali resume |

---

## 11. References (single source of truth)

- α-6 design memo: `docs/design/v0.4-alpha-6-sector-llm-ablation-matrix.md`
- α-6 cycle PR index: `reports/research-runs/alpha-6-cycle-pr-index.md` (#670)
- Phase 1 analysis: `reports/research-runs/alpha-6-phase-1-analysis-20260601.md`
- Phase 2 analysis: `reports/research-runs/alpha-6-phase-2-analysis-20260601.md`
- Phase 2 entry: `docs/handovers/v0.4-alpha-6-phase-2-entry.md` (#666)
- Rate-limit fix: PR #672
- Rate-limit post-mortem: PR #671
- 4-step rule (+ arithmetic): `memory/feedback_oracle_phrase_artifacts.md`
- Layer-intent mechanism: `memory/mechanism_layer_intent_axis_alignment.md`
- Position guard: `memory/feedback_jameses_positioning_replayable_rag.md`
- CLAUDE.md rule 2 (PR gate): `CLAUDE.md` §"Critical rules for this session"
- Previous session entry (morning): `docs/handovers/v0.4-next-session-entry-2026-06-01.md` (#662 evolved)

---

## 12. End-of-session sign-off

Cumulative this session (afternoon):
- **12 PRs** (#663 → #674, plus this one)
- **2 phases closed** (Phase 1 ✅, Phase 2 ✅)
- **5 new publishable findings** locked
- **1 new wrong-fix-averted** (rate-limit; 8th overall)
- **0 JAMES code lines** against measurement debt

Phase 3a is the natural next launch, gated only on operator
choosing to spend ~2.5-3.5h compute. Closure of α-6 follows.

Good handoff. Next session starts cold by reading this file +
running Action 0.
