# v0.4.x — Consolidated Backlog Reconciliation (2026-05-30)

> **Purpose**: single source of truth for *all* open work — collaboration,
> engineering, research, and operator actions — in one ledger. Created
> because tracking had fragmented: the collaboration ledger
> (`v0.4.x-session-2026-05-29-collaboration-checkpoint.md` §5 M1–M11)
> tracked only the collaboration line, while the engineering/research
> backlog and operator actions lived scattered across ROADMAP.md, design
> docs, and project memories. This doc merges them.
>
> **Method**: 4-source reconciliation (git/PR history #489–#604, ROADMAP.md
> v0.4 sections + QVT design memo, all `docs/handovers/*`, project memories
> + open issues), 2026-05-30.
>
> **Maintenance**: when an item lands, mark it here AND in its origin doc.
> When a new backlog item appears, add it to the right lane below.

---

## 0. Audit verdict

Roadmap tracking was **not wrong, but incomplete in one place**: there was no
single ledger holding the engineering/research/operator backlog alongside the
collaboration line. Each item existed *somewhere*; none were lost; but several
were only discoverable by reading a specific memory or design doc. The items
most at risk of being lost: **QVT α-5 ablation matrix** (routing-decision
keystone, untracked as a "to-do" anywhere), the **gemma4:e4b think-mode**
items (new this session), **D1 stage expansion** (only in a memory as
"intentionally skipped"), and the **operator actions** (some stale in
`project_state` memory).

---

## 1. Stale markers — done but still shown open somewhere (sync these)

| Item | Actual status | Evidence | Stale where |
|---|---|---|---|
| M1 — dev.to `data_exfiltration` article | ✅ DONE | PR #335 + #585 | collab checkpoint §5 body ("overdue") |
| M3 — Direction 3 cross-family | ✅ CLOSED (measurement + mechanism) | #600 / #602 / #603 / #604 | some backlog rows say "dormant" |
| M4 — step7-bench variance | ✅ DONE | #586 / #590 | collab checkpoint §5 body |
| M8 — server-split 9/9 | ✅ DONE | #580 (server 6021→892 lines, -85%) | — (already 🟢) |
| F9 — q15 concept-anchor | ✅ CLOSED | #545 / #546 / #547 / #549 | memory `feedback_bl9_partial_*` ("spawned") |
| v0.4.1 DOI | ✅ MINTED `10.5281/zenodo.20426719` | #569 | memory `project_state` ("Zenodo mint 필요") |
| Issue #511 (LEO evidence-scope) | work landed, **issue still OPEN** | L.A–L.D #513/#514/#516/#517/#526 | GitHub → **close** (§5) |
| Issue #448 (Robin 26b cross-model) | answered by D3, **issue still OPEN** | #600 / #602–#604 | GitHub → **defer close to share timing** (§5) |

---

## 2. Lane A — solo-doable now (engineering / research)

These need no collaborator and no rendezvous. ★ = was not tracked as a to-do in
any ledger before this reconciliation.

| # | Item | Status | Notes |
|---|---|---|---|
| A1 ★ | **QVT α-5 ablation matrix** (T0 = 3 cells: L1/M_M + L5/M_M + sanity L1/M_M think=ON; T1/T2 deferred operator decision) | ✅ **T0 CLOSED 2026-05-31 15:52 KST** — Branch B verdict (routing inert at production tier, abst_f1 -0.091 regression on L5). Outputs: `reports/promo-assets/v0.4-qvt-ablation-matrix-20260531T065209.md` + `reports/research-runs/qvt-ablation-T0-smoke-result-20260531-1552.md` + `reports/research-runs/qvt-ablation-rescore-summary.md` | T1/T2 deferred to operator decision (see ROADMAP §v0.4.x α-5 cycle "Operator decision — T1/T2 sizing"). T0 closure itself shippable as production-tier evidence. Workspace: `./workspaces/hotpot_eval/`. |
| A2 ★ | **gemma4:e4b `think=false` operational fix** (§16.5 of D3 final) | ✅ LANDED #609 (2026-05-30 opt-in plumbing) | `core/reasoning/think_policy.py` + 5 stage call sites + `JAMES_GEMMA4_E4B_THINK_OFF` flag (default OFF). Default-flip is a separate follow-up gated on LLM-judge v2 + 1 week dogfood + Quality Delta Card on real-query baseline. |
| A3 ★ | **think-mode quality boundary experiment** | ✅ LANDED #608 (2026-05-30) | All 5 cognitive stages: think=OFF safe or wins on hard fixtures (gemma4:e4b, n=5, deterministic graders). Feeds A2. Follow-up = LLM-judge v2 (2-3 ambiguous cells, gemma3:12b paired judge) — reserved trigger only. |
| A4 | **pytest flaky structural root fix** | 🟡 current fix is stabilization | `test_native_done_reason` + `test_entity_name_markdown_strip` order/timeout pollution. Cluster stabilized (#582/#592/#595/#596) but not root-fixed. |
| A5 ★ | **D1 stage expansion** — wire planner/reflect/verify through `complete_with_retry` | 🟡 mechanism now known (2026-05-30) | The cap=4096 hold is now **explained**: planner/reflect/verify budgets are 82–98% thinking trace on e4b (verify 1164 tok → 23 visible). See `reports/research-runs/d1-7tier-thinking-decomposition-internal-2026-05-30.md`. D4 should route on the think-OFF/visible budget. Still gated on A3 (quality impact of think=false per stage). |
| A6 | **admin.html → 3-page split** (UI-IA Phase 3) | 🟡 open across two cycles | Inventory done (PR #385); split deferred. Operator decision. |
| A7 | **BL-8 wiki aliases** (proper-noun cross-lingual) | 🟡 deferred | Not a blocker — q15 routed through F9/query-expansion instead. |

## 3. Lane B — operator actions (not code)

| # | Item | Status |
|---|---|---|
| B1 | `migrate_v041_lifecycle.py` applied to production wiki | 🔴 **verify** (likely pending) |
| B2 | T2.D-4 — wiki seeding + `JAMES_T2D_INGEST_DISPATCH` OFF→ON + baseline re-capture | 🔴 pending |
| B3 | α-5 ablation run (the execution of A1) | 🔴 late-June, ~20h |

## 4. Lane C — collaboration-gated (Ali RESUMED 2026-06-01)

Per the 2026-06-02 session update, Ali's humanitarian pause ended
and Track 2c first deliverable arrived. Lane C status flips per
the table below; new rows added for Track 2c follow-up.

| # | Item | Gate / Status |
|---|---|---|
| C1 | M2 — ar_ecommerce.yaml 6/1 check + Ali light DM | ✅ **CLOSED 2026-06-02** — Ali delivered Track 2c 18 cases + REPORT.md + raw JSON; Jiwon ack DM v3 sent same day |
| C2 | M5 — Track 4 scope-lock note | 🟡 **active** — Ali engagement signal received; co-draft can begin |
| C3 | M6 — Vadym 3-way attribution (Phase 1 #589 done; Phase 2/3 pending) | 🟡 **next step** — Ali active + Robin DM sent 2026-06-02; Phase 2 (Ali ack of Vadym mention) ready |
| C4 | M9 — joint 3-author (or 4-way) Zenodo DOI | skeleton prep solo-doable; publish gated on attribution lock + T3 + mid-June joint framing per `feedback_intent_vehicle_gap_collective_archival` |
| C5 | M3 — D3 consolidated DM / share | ✅ **CLOSED 2026-06-02** — Robin DM sent with PR #600 summary + 7-model H1 confirmed / H2-H3 rejected + 5-point S4 ladder forward-look + Phase 3b plan |
| C6 | Track 3 — swap infra (`scripts/swap_eval_report.py` pre-work is solo-doable) | 🟡 pre-work solo-doable; execution gated on Ali Gemini backend land |
| C7 | Direction 6 — joint paper consolidation | ⏳ M5 후 활성화 |
| **C8 (NEW)** | **Track 2c Phase 4** — JAMES 18-case sweep at M_M | post α-7 5-tier (Ollama serialization). Runner ready (`scripts/adversarial_sweep.py`). ETA ~10-30 min Ollama |
| **C9 (NEW)** | **Track 2c Phase 5** — cross-stack comparison report | post C8. Compares JAMES gemma4:e4b vs Provia gpt-4o-mini on same 18 cases |
| **C10 (NEW)** | **Track 2c Phase 6** — α-8 acceptance test integration | α-8 implementation cycle. `poison_01` + `poison_04` are α-8 design memo §2.4 empirical acceptance criteria |
| **C11 (NEW)** | **Track 2c Phase 7** — Ali cross-stack results DM | post C9. Low-pressure, mirrors Ali's REPORT.md disclosure pattern |
| **C12 (NEW)** | **Bidi gate defence-in-depth** | iterative; ship `/query/` first (PR #682 draft), expand to chat / multimodal / wiki edit endpoints per subsequent PR |
| **C13 (NEW)** | **Future 100-case English↔Arabic Track 2c v2** | Ali commitment ("I'll loop you in when it's ready"). No JAMES-side action until Ali shares |
| M11 | Yang et al. 2605.09104 positioning | folds into joint piece |

## 5. Lane D — future (v0.4.2+ / v0.5)

| # | Item | Target |
|---|---|---|
| D1 | T3 Evidence Aging & Trust Decay (confidence decay by source) | v0.4.2 |
| D2 | T4 Reviewer Authority Hierarchy | v0.4.3 / v0.5-prep |
| D3 | T5 Replayable Audit Graph (partial: `reconstruct_view_at` exists) | v0.4.3 / v0.5-prep |
| D4 | Direction 2 — task-weight metric formalization | trigger: D1.B heuristic plateau |
| D5 | F3a — Claude provider native `done_reason` | 3-condition gated (claude backend + auto-router + observed retry rows) |
| D6 | QVT LLM-judge (paraphrase / partial-negation oracle) | after α stabilizes |
| D7 | External red-team pass (prompt injection) | re-deferred v0.2 → v0.4 → v0.5 |
| D8 | Repo rename "Replayable RAG" | mid-June, post-DOI + post-outline |
| D9 | Show HN 6/16 retry | karma sufficient |
| D10 | Fixture difficulty expansion (QVT, if recall ratio > 0.85) | v0.5 entry |

---

## 6. Cross-reference map (where each item's detail lives)

- Collaboration M1–M11: `v0.4.x-session-2026-05-29-collaboration-checkpoint.md` §5/§6/§11
- D3 / think-mode (A2/A3): `reports/research-runs/v3prime-cross-family-final-2026-05-29.md` §16/§16.5/§17; `scripts/research/v3prime_e4b_mechanism_probe.py`; memory `d3_e4b_floor_mechanism_thinking_trace`
- QVT α / α-5 (A1): `docs/design/v0.4-qvt-alpha-non-saturating-oracle.md`; memory `quality_verification_track`
- D1/D5 (A5/D4): memories `direction_1_adaptive_budget_closure`, `direction_5_auto_routing_closure`
- Lifecycle T3/T4/T5 (D1–D3): ROADMAP.md v0.4.2 / v0.5 sections
- v0.4.1 closure: memory `project_state` (note: DOI + F9 rows are stale — see §1)

---

## 7. Immediate next-step recommendation (solo window before 6/6)

The coherent solo thread right now is the **gemma4:e4b think-mode track**
(A3 → A2 → relates to A5), because it converts this session's D3 mechanism
finding into JAMES operational value and is foundation-hardening aligned.
**A1 (α-5)** is the higher-leverage but heavier (~20h operator) item and gates
the routing layer. Both are independent of the 6/6 collaboration rendezvous.

### 7.1 Update 2026-05-30 PM — A3 + A2 landed (#608 / #609)

The think-mode track moved end-to-end in one session:
- **A3 (#608)** measured the quality boundary: all 5 cognitive stages on
  gemma4:e4b run as well or better with `think=False` on hard fixtures.
  Most striking: synthesis hard-fixture conflict-detection 100% (OFF)
  vs 40% (ON); verify easy judgment 100% (OFF) vs 60% (ON) — deterministic
  graders caught overthinking actively hurting answer quality.
- **A2 (#609)** landed the operational plumbing as **opt-in** (default
  OFF, byte-identical to pre-A2; activate with
  `JAMES_GEMMA4_E4B_THINK_OFF=1`). E2E smoke: gemma4:e4b cap=400 default
  → empty/14.3s; think=False → 195 chars/0.6s (~24× speedup + clears
  §16.5 empty-output trap).

Next thread for the same solo window:
1. **LLM-judge v2** — paired gemma3:12b judge over the 2–3 most surprising
   A3 cells (verify EASY, synthesis HARD, planner HARD); ~1–2h. Reserved
   gate before the A2 default-flip PR.
2. **A2 default-flip follow-up PR** — gated on (LLM-judge v2 agreement) +
   (~1 week real-query dogfood with the flag ON) + (Quality Delta Card
   on real-query baseline).
3. **A1 (α-5 ablation)** — independent of the above; can start in parallel.

### 7.2 Update 2026-05-30 PM-2 — A3 v2 LLM-judge result reframes A2 default-flip

LLM-judge v2 landed (PR follow-up to #608/#609). Result on 3 surprising A3 cells:

| Cell | A3 (det. grader) | judge | agreement |
|---|---|---|---|
| verify EASY | OFF wins (0.6/1.0) | tie (4 tie / 1 biased) | WEAKER |
| synthesis HARD | OFF wins (0.4/1.0) | tie (5/5 position_biased) | WEAKER |
| planner HARD | tie | tie | AGREE |

Two findings reframe the A2 default-flip plan:

- **synthesis HARD was 5/5 position-biased**: gemma3:12b judge picked
  whichever answer was presented as A regardless of think mode. The
  A3 deterministic grader's "conflict_flag 1.0 vs 0.4" likely reflects
  keyword-matching variance, not a real quality gap between the two
  answers. A3's strongest "OFF wins" signal does not survive a
  paired-judge replay.
- **No cell produced an "ON wins" judge verdict** — judge never said
  thinking helps. The opt-in A2 plumbing remains safe to ship.

**Decision** — A2 default-flip is **not** scheduled. The new gate for
any future flip is a **real-query Quality Delta Card** (production
query distribution, not synthetic fixtures) against the α-3 step7
oracle, with the flag ON for ≥ 1 week of user dogfood. Synthetic
fixtures + deterministic grader + local LLM-judge are not strong
enough oracles to justify a global behavior change on their own.

**Methodological lesson** (added to memory): a fixture that lets the
deterministic grader read a clear signal does not guarantee that signal
is the actual quality dimension — keyword/JSON-shape matchers can pick
up artifacts that don't translate to "which answer is better." When
the grader-positive direction also fails LLM-judge confirmation, the
honest read is "no measurable difference," not "the grader was right
and the judge missed it." This applies generally to future
A3-style think-mode / cap / temperature studies.

A1 (α-5) is unaffected — it measures layer × model on QVT 3-axis,
which is a separate axis from think-mode.

A5 (D1 stage expansion) is partially unblocked: the cap=4096 hold is now
mechanistically explained (#606), but the per-stage cap policy update
should wait for the A2 default-flip so D4 (task-weight) measures on the
think-OFF/visible budget rather than the inflated default.

### 7.3 Update 2026-05-30 PM-5 — α-5 reset to MultiHop-RAG + 5-axis (post-#614)

Pre-α-5 audit of fixture v7 (#614) revealed **construct validity** and
**oracle reach** risks that would compromise the matrix run:

- v7 hard queries q18/q19/q20 were drafted by inspecting wiki entities
  first and writing queries to match — measuring "can JAMES find the
  entities I told it were there" rather than capability.
- No external benchmark in the repo (`eval/RESULTS.md` defers BEIR /
  KLUE-RC indefinitely). Α-5 routing decisions without external
  evidence would be hard to defend in review.

**Decision** — full reset of the α-5 evaluation surface:

1. **External benchmark**: MultiHop-RAG (Tang & Yang 2024, EMNLP) —
   2,556 multi-hop QA × 609 articles, CC-BY-4.0, 4 question types
   (comparison / inference / temporal / null).
2. **Workspace isolation**: `./workspaces/hotpot_eval/` via
   `JAMES_WORKSPACE` (existing config.py:74 abstraction — code change 0
   on production path).
3. **5-axis oracle**: quality 3 (path / graded / abstention) +
   **cost 2 (token, latency)** — user requirement #3 token / time
   efficiency integration. Pareto-aware verdict rule (strong-adopt /
   adopt / efficiency-adopt / tier-gated / reject / zero).
4. **Per-question-type cross-tab**: matrix runner now emits routing
   policy recommendations per query type — user requirement #2.
5. **think-mode handling**: matrix primary is think=OFF (workspace
   `.env` `JAMES_GEMMA4_E4B_THINK_OFF=1`), with a single sanity cell
   `L1/M_M think=ON` (`--sanity-think-on`) to keep A2 default-flip
   evidence inside the matrix.
6. **Incidental findings log**: `reports/research-runs/
   qvt-ablation-findings.md` running log; entries tagged
   `mechanism-candidate` / `universal-law` get memo drafts after the
   run (user requirement #5).

v7 PR #614 stays landed as **internal regression canary** — not the
load-bearing evidence for α-5 routing decisions.

Plan: `~/.claude/plans/quiet-hugging-iverson.md` (approved 2026-05-30).
Steps 1-6/8-10 ship as one PR; Steps 7/9-operator are user-side
(workspace ingest + baseline + matrix run, ~3-6h total compute).

### 7.4 Update 2026-05-31 — α-5 execution observations (post-#616)

End-to-end execution surfaced 3 issues + 1 methodological finding.
Recording so future cycles don't re-discover them.

**Fixes that landed (#616):**

- `scripts/hotpot/ingest_corpus.py` — plan referenced
  `scripts/ingest_pipeline.py` which doesn't exist. Wrapper spawns the
  workspace-aware server + mints admin JWT + posts each .txt to
  `/upload/`. Validated 0 failures on 183-article targeted ingest.
- `BENCH_SUBPROCESS_TIMEOUT_SEC` 2400→14400. The 40-min ceiling fit
  step7 (~8 min) but suffocated MultiHop-RAG balanced-100 at ~107 min.
  Caught when first baseline timed out at run 1.
- Baseline `baseline_f7762a3.json` aggregate captured (n_runs=1,
  balanced-100, 107 min).

**Workspace runtime observations:**

- Ingest: 183 articles, 264 min (~45-180s/article — latency grew
  monotonically with workspace size; first article 110s warmup, last
  was 197s). Final state: 931 entities + 46MB chroma.
- Baseline: 100 queries × ~64s = 107 min total. Heavy compared to
  step7 (~30s) due to entity_anchor expansion on 931 entities.
- T0 smoke estimate: 3 cells × 100 × 64s ≈ 5.3h.

**Per-question-type baseline signal** (the load-bearing finding for
user requirement #2 routing-policy evidence):

⚠️ **Corrected 2026-05-31 PM** — the original `path=0` and
`null abst_f1=0.387 (FN=19/25)` numbers below were **oracle-measurement
artifacts, not real JAMES behaviour**. Both were caused by oracle
matcher coverage gaps that PR #618 (sources field capture) and PR #619
(English refusal phrase coverage) fixed. The corrected re-score
appears below the original table.

**Original 1st-baseline (pre-#618/#619, methodologically misleading)**:

| type | path | graded | abst_f1 | token | latency |
|---|---|---|---|---|---|
| inference | 0.000 | **0.547** | 0.000* | 669 | 62s |
| comparison | 0.000 | 0.293 | 0.000* | 1389 | 67s |
| temporal | 0.000 | 0.320 | 0.000* | **1717** | 69s |
| null | 0.000 | 0.173 | **0.387** | 561 | 58s |

`*` abst_f1 is undefined for `present`-truth types (no positive
class samples) and reports as 0.

**Re-scored aggregate after #619 (abstention phrases, bucket: (d)
measurement artifact) on the same bench JSON**:

| Metric | Before #619 | After #619 |
|---|---|---|
| TP_abstain | 6 | 16 |
| FP_incorrect_abstention | 7 | 10 |
| FN_hallucination | **19** | **9** |
| TN_answer | 68 | 65 |
| **abstention_f1** | **0.316** | **0.627** |

Corrected hallucination rate is **36% (9/25), not 76%**. The 14 newly-
recovered TPs all use gemma4:e4b's grounding-trained English refusal
phrases ("impossible to determine", "cannot be answered", "insufficient
information") that the original Korean-dominant phrase list missed. The
JAMES grounding pipeline was already significantly better than the
original score implied. The remaining 9 FNs are real margin — routing
layers can still improve, but from a higher base.

The corrected baseline also gains a non-zero path axis once #618's
sources-field capture is applied (next baseline re-capture, ~107 min
operator-side). This entire correction chain — `axis 0/saturated →
sample answers manually → check JAMES response keys → reconcile design
vs matcher` — is recorded in memory `feedback_oracle_phrase_artifacts`
as a 4-step verification rule to run BEFORE interpreting matrix Δ
values in future cycles.

**Path-axis finding — RESOLVED** (#618, **bucket: (d) measurement artifact**):

The original "graph_paths surfaces concept/org/person entities but
expects article titles" framing was based on bench.py only reading
`graph_paths`. The actual `/query/` response also includes a `sources`
field (top-3 cited document filenames, per
`core/reasoning/pipeline.py:343`) — JAMES's citation feature has been
in place all along; bench was just dropping the field. PR #618:

1. `scripts/bench.py` now captures `response.sources` in each row.
2. `eval/qvt/oracle.py::score_path_coverage` now scores against BOTH
   `graph_paths` entities AND `sources` filenames after slug
   normalisation (lowercase, dash-separated, `multihop_<id>_` prefix
   stripped, 80-char cap).
3. Per-row tracks `via_graph` and `via_sources` separately so future
   matrix verdicts can distinguish "JAMES wins this query via concept
   traversal" from "JAMES wins via document citation."

Path axis is now active. The corrected baseline (re-capture in flight
at the time of this update) is expected to show non-zero `path_recall`.

**T0 smoke** running in background; full matrix render after.

### 7.5 Update 2026-05-31 PM — α-5 mid-cycle infra solidification (#624–#633)

After the #618/#619/#623 measurement-debt corrections, the cycle
continued with **10 more PRs** of infra / docs / methodology work
before T0 smoke returns. Cumulative cycle total is now **24 PRs**
(#608 → #633, the first 5 are A3/A2/A3-v2 think-mode follow-ups
that A1 inherits).

**Cycle PR ledger (full, in landing order)**:

| PR | Layer | What landed |
|---|---|---|
| #608 | feat | A3 think-mode quality boundary experiment (5-stage grader) |
| #609 | feat | A2 — `JAMES_GEMMA4_E4B_THINK_OFF` opt-in plumbing |
| #611 | feat | A3 v2 LLM-judge — 3 surprising cells re-checked with gemma3:12b |
| #614 | feat | step7 v7 fixture (now internal regression canary only — see §7.3) |
| #615 | reset | MultiHop-RAG external benchmark + 5-axis + per-question-type matrix |
| #616 | exec | Ingest wrapper + bench timeout fix + path-axis finding |
| #617 | docs | §7.4 first per-question-type baseline signal |
| #618 | fix (d) | source-recall — bench was dropping `response.sources` |
| #619 | fix (d) | abstention phrases — gemma4:e4b English refusals (10 → +7 narrow) |
| #620 | tool | re-score tool + §7.4 76% → 36% correction |
| #621 | docs | 4-bucket diagnostic taxonomy + dual purpose |
| #622 | fix | render_report defensive path + bucket retroactive |
| #623 | fix (d) | session_id suite-aware + 3 narrow abstention phrases (total 10) |
| #624 | docs | ROADMAP §v0.4.x α-5 cycle entry (initial form) |
| #625 | fix (a) | matrix runner — bench subprocess respects `--suite` |
| #626 | docs | T0 smoke result analysis TEMPLATE |
| #627 | docs | Pareto verdict walk-through + CLAUDE.md rule 2 `fix` exempt + post-mortem |
| #628 | feat | T0 analysis fill script |
| #629 | docs | oracle.py → package split design (deferred to post-matrix) |
| #630 | docs | α-5 cycle summary outline |
| #631 | docs | publishable narrative draft — "Don't Build a Layer for the Bug" |
| #632 | docs | bucket-(c) LLM-judge abstention detector design memo |
| #633 | feat | `qvt_promote_findings.py` — auto-draft memory entries from findings.md |

**Wrong-fix avoided count (cumulative)**: 3. Each landmark "JAMES
failure" turned out to be measurement-side; cumulative JAMES code
change attributable to α-5 measurement debt is **0 lines**.

**Deferred follow-up tracks** (each has a landed design doc; gated
on T0 closure):
- **Oracle package split** (#629) — gated on α-5 matrix closure PR.
- **Bucket-(c) LLM-judge** (#632) — gated on (α-5 closure) + (oracle
  split) + (phrase-rule stability audit).
- **Findings promotion loop** (#633) — operator runs after matrix
  yields mechanism-candidates, drafts land in
  `reports/research-runs/promoted-findings/`.
- **Publishable narrative** (#631) — fills with numbers after T0
  closes; co-author negotiations defer to v0.4-end + 1 week.

### 7.6 T0 smoke progress checkpoint (2026-05-31 11:21)

Background task `b1rk9h3xz` running since 10:17. Trace evidence (via
`reports/trace/2026-05-31/`):

- Cell 1 / 3 in flight (L1 baseline, M_M tier)
- Latest query `bench_multihop_rag_62` — 62/100, ~62 s/query
- Suite fix #625 confirmed working (session_id format proves
  multihop_rag fixture is loaded, not stale step7)
- Cell 1 ETA ~12:00; full T0 (3 cells + 1 sanity think=ON) ETA ~16:00

**Side observation (not affecting bench results)**: `core/observability.py:117-122::_trace_root()`
hardcodes `<repo>/reports/trace/` and ignores `JAMES_WORKSPACE`. Bench
JSONs (the source of truth for matrix verdicts) are unaffected — they
go through `_resolve_output_dir()` which DOES honour the env. Logged
as a v0.4.2+ hygiene item, not a blocker. New row D11 added to Lane D.

### 7.7 Stale-marker re-sweep (2026-05-31)

Re-verified §1 stale table; correcting in place:

| Issue | Actual state | Action |
|---|---|---|
| #511 (LEO evidence-scope) | **already CLOSED 2026-05-30 07:02 UTC** — §1 row was stale at write | mark §1 row resolved; no further action |
| #448 (Robin 26b cross-model) | OPEN — #600/#602/#603/#604 answered it but the share rendezvous is the right moment to close | **leave open** (defer close until D3 collab share, gate: 6/6+ Ali resume) |

The §1 row for `project_state` memory (DOI mint stale) was already
corrected in the 2026-05-30 PM session memory writes; no remaining
fix needed.

The §1 row for `project_state` memory (DOI mint stale) was already
corrected in the 2026-05-30 PM session memory writes; no remaining
fix needed.

### 7.8 Updates to Lane D (future)

| # | Item | Target | Status |
|---|---|---|---|
| D11 ★ | `core/observability.py::_trace_root()` workspace-aware (eliminate trace leak across workspaces) | v0.4.2 | observed during α-5 T0 — `JAMES_WORKSPACE` not honoured by trace writer. ✅ landed PR #635 (2026-05-31). |
| D12 ★ | A2 default-flip Quality Delta Card pre-template | post-α-5 closure | gate from §7.2 — real-query gate, paste-ready scaffold. ✅ landed PR #636 (2026-05-31). |

### 7.10 Update 2026-05-31 16:00 — T0 CLOSURE

Matrix completed at 15:49 KST. Background task `b1rk9h3xz` exit 0,
all 3 cells written (L1/M_M, L5/M_M, L1/M_M-thinkON). Closure
sequence executed per runbook (#641):

1. **Rescore** (step 1) — `qvt_rescore_all_cells.py` detected 2/3
   cells QQ-bugged (L1, sanity); L5 already rescored. All cells now
   reference correct multihop_rag bench JSONs.
2. **Render report** (step 2) — `qvt_ablation_matrix.py --render-report`
   produced `reports/promo-assets/v0.4-qvt-ablation-matrix-20260531T065209.md`.
3. **Fill T0 analysis** (step 3) — `qvt_fill_t0_analysis.py` populated
   `reports/research-runs/qvt-ablation-T0-smoke-result-20260531-1552.md`.
4. **Promote findings** (step 4) — idempotent skip (only path-axis
   finding, already drafted).

**Final 5-axis verdict** (rescored, vs `baseline_3a961a3_rescored`):

| Cell | Path Δ | Graded Δ | Abst F1 Δ | Token Δ | Latency Δ | Verdict |
|---|---|---|---|---|---|---|
| L1/M_M | +0.015 | -0.017 | -0.018 | +74 | +2.3 | reject (n_runs=1 noise floor) |
| L5/M_M | +0.008 | -0.027 | **-0.091** | +55 | +4.7 | **reject** (real regression) |
| Sanity think=ON | +0.000 | +0.027 | -0.058 | -109 | -1.7 | reject |

**Headline**: L5 (all routing layers ON) at M_M production tier =
**Branch B confirmed** in `alpha-5-publishable-narrative-DRAFT.md`
§6.2. Routing layers inert / mildly regressive. T0 closure shippable
as standalone evidence; T1/T2 are operator decisions.

**Via_graph vs via_sources** (path axis breakdown): **77/77 hits via
sources, 0 via graph_paths** on both L1 and L5. JAMES wins the path
axis entirely through its citation layer (`response.sources`), not
through concept-centric graph traversal. The original
"path_recall=0" panic before #618 had a single root: a measurement
gap in bench.py that masked an already-working citation mechanism.
Worth flagging for the publishable narrative: "JAMES wins paths
through citation, not graph traversal" on this benchmark.

**Per-cell numbers (rescored)**:

| Cell | path | graded | abst_f1 | token | latency |
|---|---|---|---|---|---|
| L1/M_M baseline | 0.4189 | 0.3267 | 0.5909 | 1224 | 65.99 |
| L5/M_M full stack | 0.4122 | 0.3167 | 0.5000 | 1205 | 68.37 |
| Sanity L1/M_M think=ON | 0.4044 | 0.3700 | 0.5333 | 1115 | 64.29 |
| Baseline reference | 0.4040 | 0.3433 | 0.6088 | 1150 | 64 |

**A2 default-flip note**: sanity (think=ON) shows graded **+0.043**
but abst_f1 -0.058 + path -0.014. No unambiguous "ON wins"; A2
default-flip gate (real-query QDC per template #636) unchanged.

**Cycle totals**: 36 PRs (#608 → #645+), 5 measurement-side fixes,
4 wrong-fix-averted, **0 lines of JAMES code changed against α-5
measurement debt**.

### 7.9 Update 2026-05-31 PM — matrix runner glob bug (Correction 4)

When cell 1 (L1/M_M) of the in-flight α-5 T0 smoke completed at
12:07 KST, the cell JSON aggregate showed `path_coverage = 0.000`,
`abstention_f1 = 0.000`, `graded_answer = 0.028` — three saturated
quality axes. Applying the 4-step rule from
`feedback_oracle_phrase_artifacts` as routine hygiene immediately
flagged the cell JSON's `runs[0].bench_output` field: it pointed at
`bench_nogit_step7_20260507_194228.json` (24 days old, step7 suite,
12 queries). The fresh multihop_rag bench file
`bench_87ed176_multihop_rag_20260531_120746.json` (100 queries,
6598 s of compute, 100/100 sources captured) was sitting
unreferenced beside it.

**Root cause** (`scripts/qvt_ablation_matrix.py:380`): the post-bench
detection glob was still hardcoded `bench_*_step7_*.json`. PR #625
correctly threaded `--suite={suite}` to the bench subprocess CALL
(line 355 uses `glob_pattern = f"bench_*_{suite}_*.json"` for the
pre-existing set), but the line 380 glob for the *after* set was
overlooked. The matrix subprocess wrote the correct multihop_rag
bench, but the runner could not see it; `new[-1]` returned the
lexicographically last stale step7 file.

**Bucket**: (a) architecture. Sibling oversight to #625. The plumbing
was 95% complete; this was the final 5%.

**Fix**: PR #638 — one-line change replacing `"bench_*_step7_*.json"`
with `glob_pattern` (the same variable already used for `pre_existing`).
Companion tool `scripts/qvt_rescore_ablation_cell.py` re-aggregates
existing cell JSONs against the actual multihop_rag bench files
(auto-resolve by mtime proximity to the cell file), writes a
`.before-rescore.json` side-copy for the diagnostic trail.

**Strategy chosen**: do NOT kill the running matrix task. The bench
JSONs are written correctly (the subprocess wiring is fine); only
the cell aggregate scoring is wrong. Cells 2/3 + sanity will finish
under the existing runner; post-hoc rescore at matrix end is
~seconds per cell.

**Real cell L1/M_M numbers** (after rescore against the actual bench
file):

| Axis              | Stale step7 (cell JSON) | Real multihop_rag |
|-------------------|-------------------------|-------------------|
| path_coverage     | 0.000                   | **0.419**         |
| graded_answer     | 0.028                   | **0.327**         |
| abstention_f1     | 0.000                   | **0.591**         |
| token_cost        | 2212 chars              | 1224 chars        |
| latency_cost      | 42.68 s                 | 65.99 s           |

These are mid-range baseline numbers — exactly what an L1 cell
should report. Layer-on cells need to *improve* on them, not
collapse from 0/0/0.028.

**Wrong-fix avoided** (Correction 4): had the saturated read been
taken as truth, the entire matrix would have re-interpreted as
"JAMES baseline is already dead; routing-layer cells can't help
what's broken." Multi-hour debug spiral averted by ~5 minutes of
4-step-rule application to the first cell JSON the matrix wrote.

**Cumulative score**: 4 wrong-fix-averted corrections in this cycle
(#618 / #619 / #623 / #638). Cumulative JAMES code change
attributable to α-5 measurement debt: **0 lines**.

**Discipline generalisation**: the 4-step rule was originally
codified in `feedback_oracle_phrase_artifacts` as a bucket-(d)
"matcher coverage" tool. Correction 4 demonstrates the same
procedure catches bucket-(a) plumbing bugs identically. Memory
update follow-up: broaden the rule's framing to "any measurement-
side wiring" and add Correction 4 as the worked bucket-(a) example.
