# v0.3.x — Ali Collaboration Track

> Status: Track 2 + 2b + 2c locked (Ali confirmed 2026-05-19); Tracks 1/3/4/5 open
> Created: 2026-05-18
> Trigger source: LinkedIn DM + dev.to comment exchanges with Ali Afana (Provia founder, dev.to Featured), 2026-05-16 → 2026-05-19, 6 turns
> Companion documents:
>   - `reports/promo-assets/gemma4-e4b-cognitive-stages-eval.md` (eval report — PR #307)
>   - `docs/handovers/v0.3.x-gemma4-feedback-track.md` (feedback routing — PR #307)
>   - `reports/promo-assets/devto-gemma4-write-track.md` (Write-track submission archive — PR #308)
>   - `reports/promo-assets/launch-tracker.md` (running log, 2026-05-16 → 2026-05-18 rows)
>   - `reports/promo-assets/injection-fixtures-schema-v0.md` (Track 2 deliverable — this PR)

## Purpose

A 4-turn exchange with Ali Afana across LinkedIn DM and dev.to comments produced **five written commitments on our side**. This handover converts those promises into concrete coding tracks so the next session can pick any one and start without re-deriving context.

Any session continuing this track should be able to read this single page and know:

- what we owe Ali by when
- the priority order
- the file paths the work lands at
- the cross-experiment decision matrix once results arrive

## Commits made in writing

Direct quotes from our outgoing LinkedIn DM (2026-05-18) and dev.to reply (2026-05-18):

| # | Track | Deliverable | Timeline committed | Channel |
|---|---|---|---|---|
| 1 | **Provider contract** | `core/plugins/base.py` Provider type + `JAMES_PLUGINS` loader + Ollama/Gemini API impls against same contract | "Realistic timing: 2–4 weeks" | LinkedIn DM |
| 2 | **Fixture JSON schema** | Schema v0 + Korean/English baseline fixtures pytest-compatible, ready for Arabic case additions | "Within 1–2 weeks" | LinkedIn DM |
| 3 | **STEP 7 swap infrastructure** | Same wiki corpus on E4B / 26B MoE / 31B Dense via Provider swap, two-layer test (typed-graph conditioning vs natural-language synthesis) | "Once Provider interface lands" | LinkedIn DM + dev.to comment |
| 4 | **Pre-experiment scope-lock note** | Short written note: data scope per side, final draft review both sides, joint byline conditions | "Once Provider contract is published" | LinkedIn DM |
| 5 | **Cross-experiment writeup** | Joint Part 2 (single article, joint byline) OR two linked articles depending on result | "After swap results come in" | LinkedIn DM |

All five are real-world commitments to a real-world collaborator. Not meeting them silently is worse than not committing in the first place.

---

## Track 1 — Provider contract (highest leverage)

**Why first**: tracks 3 / 4 / 5 are all gated on this. Until Ali can implement the Gemini API side against a stable contract, the cross-experiment cannot start.

**Scope**:

- `core/plugins/base.py` — abstract `Provider` type. Minimum surface: `complete(prompt: str, *, task_type: str, max_tokens: int, temperature: float, ...) → ProviderResponse`.
- `JAMES_PLUGINS` env var resolver — comma-separated list of plugin module paths, loaded at server start.
- Refactor `core/reasoning/backends/ollama_local.py` and `core/reasoning/backends/claude_code_cli.py` to be `Provider` implementations rather than ad-hoc backends. Existing `JAMES_REASONING_BACKEND` env switch (PR #305) becomes a special case of plugin selection.
- New file `docs/design/v0.3-llm-provider-contract.md` — the contract document we promised to publish *before* the implementation lands, so Ali can start his Gemini API side in parallel.

**Out of scope for this track**:

- Plugin discovery via `pyproject.toml` entry points (entry-points machinery can wait; explicit `JAMES_PLUGINS=foo.bar:Provider` env-var form is fine for v0.3.x).
- Per-stage backend selection (`JAMES_LLM_MODEL_META` etc.) — that is the option-A2 follow-up named in the cognitive-stages eval report, lands separately.

**Done when**:

- [x] `Provider` ABC compiles, has docstrings, has unit tests against a mock implementation. **(#324 PR-A — `Backend` Protocol in `core/reasoning/backends/__init__.py`, `resolve_backend_for_stage` per-stage override, 12 synth call sites swapped.)**
- [x] `OllamaProvider` and `ClaudeCodeCLIProvider` pass the same conformance test suite. **(#325 PR-B — `tests/test_backend_conformance.py` R1–R6 + `tests/test_no_sdk_leakage.py` for the R5 architectural rule; both reference backends pass.)**
- [x] `docs/design/v0.3-llm-provider-contract.md` is published and linked from `CHANGELOG.md`. **(#316, published 2026-05-18, before any wiring landed.)**
- [x] A 1-paragraph LinkedIn DM to Ali pointing at the contract URL with "you can start your Gemini Provider impl now if you like." **(Sent 2026-05-19 after #324/#325/#326 merged — see Closure section below for the exchange.)**

**Files affected**: `core/reasoning/backends/__init__.py` (updated), `core/reasoning/backends/ollama_local.py` (refactor), `core/reasoning/backends/claude_code_cli.py` (refactor), `core/reasoning/trace_helpers.py` (lambda → backend dispatch), `tests/test_backend_conformance.py` (new), `tests/test_no_sdk_leakage.py` (new), `tests/test_plugin_loader.py` (new), `docs/design/v0.3-llm-provider-contract.md` (new). `core/plugins/base.py` (the originally-named file) was reframed under the **pack-level** Plugin API track (Track C in `v0.3.0-platform-track.md`, design `docs/design/v0.3-plugin-api.md` #343, contract surface #344) since the backend layer naturally lives under `core/reasoning/backends/` rather than `core/plugins/`. The Provider contract Ali implements against is unchanged either way — same Protocol surface, same conformance suite.

---

### Track 1 — closure as of 2026-05-19 + Ali turn 7

**Status**: ✅ **Track 1 done, ~27 days ahead of the 2026-06-15 deadline we committed in writing.**

Three merged PRs:

- **#324** `feat(reasoning): synth call-site backend wiring (Track 1 PR-A)` — `resolve_backend_for_stage(stage)` per-stage override (`JAMES_BACKEND_SYNTH=…`) layered on `JAMES_REASONING_BACKEND` (PR #305). 12 synth call sites moved from direct `engine.llm.call_gemma` to `get_backend(name).complete(...)`. `trace_synth_call` signature reshaped (prompt + kwargs, no lambda) so R1 of the contract is honoured at the helper boundary.
- **#325** `test(reasoning): conformance suite + SDK leakage guard (Track 1 PR-B)` — R1–R6 conformance suite parameterized over both reference backends (hermetic, no live ollama / claude CLI in CI) + the R5 architectural test that AST-walks the middleware roots for forbidden SDK imports.
- **#326** `feat(reasoning): JAMES_PLUGINS loader + JAMES_LLM_TEMPERATURE env (Track 1 PR-C)` — the env-var-driven backend plugin loader (failures fatal, per the contract) + `temperature` reserved kwarg propagated end-to-end (the 3×3 experiment's swept variable).

**Outgoing DM to Ali (2026-05-19, post-merge)**:

> Hi Ali,
>
> Quick update — no action needed on your side.
>
> Track 1 (the Provider contract wiring we agreed on around 5/18) landed on `main` today, about a month ahead of the 2026-06-15 deadline we set. Three merged PRs: \[#324 / #325 / #326]. The contract document (`docs/design/v0.3-llm-provider-contract.md`, published earlier as #316) is unchanged — what landed is the wiring, plus tests pinning every clause. Both reference backends (`ollama_local`, `claude_code_cli`) pass the full conformance suite. A Gemini API backend that satisfies the same `Backend` Protocol should drop in with no core-side changes.
>
> No timeline pressure — your **mid-June response window** (after your Provia Arabic-localization sprint) still works perfectly on our end. When you do come back to Track 1, the contract is ready exactly as specified. Track 3 (the 3×3 cross-experiment swap infra) naturally follows once your backend lands; we'll start that work then, not before. Schema v1.1 (catalog_poisoning) from your Track 2c review is also locked on main (#322) — thanks again for that turn. Talk soon, Jiwon

**Ali's reply (2026-05-19, same day)**:

> Jiwon — received, and the velocity is appreciated. Track 1 landing a month early means the integration window is fully open by the time Arabic localization clears. Will surface on Track 1 mid-June with the Gemini backend, and `ar_ecommerce.yaml` is on track for 6/1. Talk soon.

**Decoded signals**:

1. **Velocity acknowledged** — month-early Track 1 lands as a credibility deposit rather than as pressure.
2. **mid-June Gemini backend commitment confirmed** — that PR will be **our first external PR**. The CLA workflow we shipped at #340 needs the four operator GUI steps (private signature repo, CLA Assistant install + Gist, `CLA_BOT_PAT` secret, dry-run) wrapped up *before* Ali's PR opens. Calendar pressure now exists on the operator side, not the code side.
3. **`ar_ecommerce.yaml` 6/1 deadline holds** — Track 2's calendar commitment is on track. The v1.1 validator in `tests/fixtures/injection/test_fixture_format.py` (PR #322) is already the landing pad. No further code action on our side until the fixtures arrive.
4. **Track 3 timing locks in** — once Ali's Gemini backend PR lands (mid-June), the 3×3 swap infrastructure track activates. We can do **pre-work** during the four-week window — `scripts/swap_eval_report.py` (the comparison harness from the 3×3 eval plan #315) can land before Ali's backend ships, since it's pack-agnostic.

**Outgoing ACK DM to Ali (2026-05-19, third turn)**:

> Got it. CLA workflow goes live this week so the Gemini backend PR has a clean signature flow. Landing pad for `ar_ecommerce.yaml` is the v1.1 validator that shipped in #322. Talk soon.

**Operator follow-ups (calendar-locked)**:

- ~~**This week (2026-05-19 to 2026-05-26)**: complete the four Track B GUI steps in `docs/handovers/session-2026-05-09-license-infrastructure.md` §3 (B-3 / B-4 / B-5 GUI / B-6 dry-run). The CLA workflow becomes operational at the end of step B-6.~~ **✅ DONE 2026-05-20 — 6 days ahead of the 2026-05-26 deadline.** B-3 (`james-rag-evol-cla` private repo) + B-4 (CLA Assistant install + Gist registration + repo link) + B-5 PAT (`CLA_BOT_PAT` repo secret) + B-6 (별도 GitHub 계정 fork → 봇 코멘트 → 서명 코멘트 → status check ✅) all verified end-to-end. Ali's mid-June PR will open against a fully-armed CLA workflow.
- **2026-06-01**: `ar_ecommerce.yaml` arrives from Ali. Test pipeline runs automatically — no action unless schema drift is reported.
- **Mid-June** (windowed): Gemini backend PR opens. CLA Assistant fires the signature request automatically (verified by 2026-05-20 dry-run). Conformance suite runs against the new backend.
- **mid-June to 2026-06-29**: Track 3 swap infrastructure (separate session work).

---

## Track 2 — Fixture JSON schema v0 (closest deadline)

**Why second**: only blocking thing on Ali's side is the schema. He committed to 15–20 Arabic e-commerce cases the moment we send the schema. We promised it "within 1–2 weeks" — the calendar pressure is here, not on the Provider work.

**Scope**:

- `reports/promo-assets/injection-fixtures-schema-v0.md` (this PR includes the v0 draft) — fields `id`, `category`, `prompt`, `expected_block`, `locale`, `notes` with examples.
- Spin out the relevant subset of `james_security_test.py` (the 83-item suite) into `tests/fixtures/injection/baseline_kr_en.yaml` (or `.jsonl`) following the schema.
- A pytest harness in `tests/fixtures/injection/test_fixture_format.py` that any fixture file (ours, Ali's Arabic, hypothetical future contributions) passes by structure.
- 1-paragraph LinkedIn DM to Ali with the schema URL the moment Track 2 is merged.

**Out of scope**:

- A separate GitHub repo (`james-injection-fixtures`) — keep it in-repo for v0.3.x; spin out to its own repo only if the file count or external contributors justify it. Move is one `git filter-repo` away later.
- Korean-key JSON schema experiment (hypothesis C from the eval report) — separate track, only relevant if Ali's swept data points at it.

**Done when**:

- Schema is documented + linked from `reports/promo-assets/launch-tracker.md`.
- 10+ baseline fixtures from `james_security_test.py` are in the new format, pass the pytest harness, and produce the same block-or-allow decisions the original suite does.
- Ali has the schema URL.

**Files affected**: `reports/promo-assets/injection-fixtures-schema-v0.md` (new, in this PR), `tests/fixtures/injection/` (new directory), `tests/fixtures/injection/baseline_kr_en.yaml` (new), `tests/fixtures/injection/test_fixture_format.py` (new), `james_security_test.py` (potentially: add a section noting which fixtures live in the new format and which still live inline).

### Track 2 + 2b + 2c — status as of 2026-05-19

| Sub-track | PR | What landed |
|---|---|---|
| 2 | #319 | 34-fixture baseline (24 prompt_injection + 6 benign + 4 data_exfiltration) + portable schema-validity harness + JAMES-specific decision harness |
| 2b | #320 | v1 schema enforcement — `expected_block_stage` enum + normalization invariant + `schema_version` field + 5 new pin tests |
| 2c | THIS PR | **v1.1 — `catalog_context: list[string]`**. Pre-emptive answer to Ali's flagged convention question ("how do I express the customer-query-that-triggers-retrieval alongside the poisoned content?") in his 2026-05-19 LinkedIn reply. The convention is now in the schema doc, the example is in the schema doc, and `test_fixture_format.py` has 2 new shape pins. Ali no longer has to freelance it during `ar_ecommerce.yaml` authoring. |

**Ali's 2026-05-19 reply, two named confirmations + one architectural insight + the convention question we just pre-empted**:

1. ✅ "The `byte_drift_expected` opt-in via notes is the right escape hatch" — v1 normalization invariant called clean by the collaborator.
2. ✅ "The `data_exfiltration` retrieval-stage insight is the architecturally correct framing" — v1 stage-mapping table validated; the framing carries forward as a teaching artifact opportunity (see "Open backlog" below).
3. 📅 Ali commits to **15–20 ar_ecommerce.yaml fixtures × 4 categories + ≥ 5 benign**, against **v1.1** as of this PR's merge. **June 1 deadline holds**.
4. 🚦 Convention question Ali flagged in advance: catalog_poisoning's customer-query-plus-poisoned-content shape. **Pre-empted by v1.1 in this PR** — Ali confirmed he'd surface it as a DM rather than freelance, so the v1.1 ship is the answer.
5. ⏳ Provider contract (Track 1) — Ali "reading the handover doc this week and will surface a separate response. The v0.3.x integration timing (~mid-June) lines up well with my Provia Arabic-localization sprint." → Track 1's 2026-06-15 deadline is calendar-aligned on both sides.

**What Ali should NOT be surprised by when reading this v1.1 update**:

- `catalog_context` is a list, not a single string — the rationale ("preserve the 1-of-N poisoned signal") is in the schema doc's new "Catalog context shape" section. If he disagrees, the conversation costs one DM.
- The field is opt-in for `catalog_poisoning` / `data_exfiltration` / `prompt_injection` only — other categories rejected by a shape pin so a fixture author can't accidentally smuggle unread mock data into the wrong category.

**Open backlog from Ali's reply** (not for this PR, just recorded so future sessions see it):

- "That's the kind of layered-security principle that turns the schema doc from a fixture spec into a teaching artifact. Worth keeping in mind if you ever publish it as a standalone reference." → candidate work: `docs/design/v0.3-injection-fixtures-schema.md` (in-repo design doc) or dev.to article. **Not blocking; not for this cycle.**

---

## Track 3 — STEP 7 swap infrastructure

**Why third**: gated on Track 1 (Provider). Once that lands, this is small.

**Scope**:

- `scripts/swap_eval.py` — takes a list of Provider names (e.g. `e4b,moe26b,dense31b,gemini-1.5-pro`) and runs the STEP 7 13-query suite against each, on the same wiki corpus, producing one `reports/swap_eval/<timestamp>-<provider>.json` per provider.
- Two-layer harness — per Ali's sharpened prediction, we measure both the typed-graph conditioning layer and the natural-language synthesis layer separately. The Provider call site in `core/reasoning/pipeline.py` is the typed-graph conditioning point; the call site in `core/reasoning/synth.py` is natural-language synthesis. Each is swapped independently.
- Aggregator: `scripts/swap_eval_report.py` that produces a single Markdown comparison table from the per-provider JSON files.

**Out of scope**:

- Latency budget enforcement during swap runs (one provider going slow shouldn't cancel the rest — log and continue).
- Cost accounting for cloud providers (Gemini API has a price tag; that is Ali's concern, not ours).

**Done when**:

- Swap script produces clean JSON for at least three local Ollama models.
- `reports/swap_eval/2026-XX-XX-baseline.md` snapshot exists.
- Two-layer measurement is verified by setting different providers at the two call sites and confirming the trace shows different model IDs at each.

---

## Track 4 — Pre-experiment scope-lock note template

**Why fourth**: it's a one-page document that the actual cross-experiment session will fill in. Drafting the template now means the experiment session can run without re-thinking the framing.

**Scope**:

- `reports/promo-assets/cross-experiment-scope-lock-template.md` — fields for: each author's data scope, each author's framing claim, success/failure criteria for each hypothesis, final draft review obligations, joint byline conditions, withdrawal clause if results are surprising.
- One-line note in this collaboration handover that the template is to be co-signed (or at minimum, mutually acknowledged in writing) before the experiment kicks off.

**Out of scope**:

- Legal language. This is a researcher-to-researcher artifact, not a contract. The tone should be the same tone our DM exchange has — clear, fair, falsifiable.

**Done when**:

- Template exists.
- LinkedIn DM to Ali links the template the moment Track 1 publishes the Provider contract.

---

## Track 5 — Cross-experiment writeup (decision matrix)

**Why last**: depends on Tracks 1 / 2 / 3 / 4 completing AND on actual swap results existing.

The decision matrix from our DM commitments:

| Swap result | Temperature sweep result | Writeup format | Authorship |
|---|---|---|---|
| Divergence collapses on typed-graph conditioning, holds on natural-language synthesis | Cap-monotonic on Dense, cap-curve on MoE | **Joint Part 2, single article, joint byline** — both predictions falsify cleanly, the story is one finding | Co-authored, Ali first author |
| Divergence collapses on both | Either | Asymmetric: Ali sharpens his original framing to "natural-language synthesis specifically"; we write a "Graph-RAG: where Ali's hypothesis lands" follow-up. Two linked articles. | Each their own |
| Divergence survives on Graph-RAG | Either | **Walk-back required** — Ali's original framing is broader than the data supports. Joint article framed as "where the original claim narrows" — Ali first author by request, since it is his framing being narrowed. | Co-authored, Ali first author, *if* he wants the joint form. Otherwise two linked articles. |
| Divergence survives on both, temperature shows no separation | Either | Hypothesis A from the eval report wins ("meta-reasoning floor at 4 B") — that becomes the cross-experiment's main finding, and the temperature thread is closed | Co-authored or each their own — author choice |

The matrix is committed to before results land. That is the entire point of Track 4's scope-lock note.

### Track 5 narrative evolution — 2026-05-24 update (V3'.e refined Pattern S)

Robin Converse's 2026-05-22~24 LinkedIn finding ("two operating modes,
one model" — substitution vs synthesis split, T-invariant on
substitution) was tested in JAMES on `gemma4:e4b` via the V3'.e driver
(PR #439) and the result PR shipping the data (#440). Result, N=20,
sovereign Ollama, T=0.2:

| Arm | Cap | Success | Latency |
|---|---|---|---|
| substitution (verbatim retrieval) | 400 | 20/20 (100%) | 0.8s |
| substitution | 4096 | 20/20 | 0.8s |
| synthesis (refund recommendation) | 400 | 14/20 (70%) | 4.0s |
| synthesis | 4096 | 20/20 | 4.5s |

**Three workload levels span three cap-behaviour signatures** when
composed with the earlier V3'.a~d 4-stage sweep:

| Workload | Driver | Success @ cap=400 |
|---|---|---|
| heavy synthesis (4-stage cognitive) | V3'.a~d | 0/10 |
| light synthesis (e-commerce refund) | V3'.e synthesis arm | 14/20 (70%) |
| no synthesis (verbatim retrieval) | V3'.e substitution arm | 20/20 (100%) |

#### Narrative shift

| Before V3'.e | After V3'.e |
|---|---|
| "3 contexts, 2 architectures, **1 mechanism**" | "3 contexts, 2 architectures, **2 mechanisms with task-weight gradient**" |
| Cap pathology = floor at some constant cost | Cap pathology = synthesis-mode entry cost that **scales with task weight** |
| JAMES = 4-stage replicator | JAMES = **gradient quantifier** (Robin's confirmation + new task-weight axis) |
| Single-layer finding | Two-layer finding (mode split + workload gradient) |

Headline phrase candidate the joint piece can stand on:
**"Substitution is free. Synthesis costs in proportion to what it
has to invent."**

#### What this changes about decision-matrix authorship

The decision matrix above was written for "1 mechanism" cap pathology
plus the original temperature thread. With V3'.e's refined Pattern S
in hand, the two-mechanism + gradient framing rotates which result
corresponds to which row:

- "Divergence collapses on typed-graph conditioning, holds on
  natural-language synthesis" (row 1) maps to the **substitution
  arm** of V3'.e — already confirmed for JAMES single-stack. Joint
  Part 2 single article is the live option contingent on Ali's
  Gemini-side cross-stack replication (Track 3, ~mid-June).
- The "task weight gradient" axis is a JAMES-led contribution that
  Ali's chatbot context can either corroborate (if his swap_eval on
  Gemini shows the same workload-scaling) or diverge from (new
  third-arm question).
- Robin's data (her LinkedIn series) is a *third* surface for the
  same mechanism, not a separate finding — she sees binary mode-
  split on a single model under temperature variation; JAMES sees
  workload gradient on a single model under cap variation. The two
  observations are orthogonal projections of the same mechanism.

The matrix's commitment to "Ali first author when his framing
narrows" still holds; what V3'.e changes is the **first author of
the gradient layer** — that lands on JAMES side. Joint piece becomes
co-authored with explicit role-split: Ali (chatbot-context
divergence), Robin (mode-internal split), JAMES (workload gradient
+ cognitive-middleware corroboration).

#### What's still pending before the joint piece runs

1. Track 3 swap infrastructure — Ali Gemini backend lands mid-June,
   then `scripts/swap_eval.py` (Track 3 deliverable, pre-work
   already designable) runs the same V3'.e arms on managed vs
   sovereign stacks.
2. ~~Robin's cross-temperature data from her 26B-MoE sweep — not yet
   published; if/when she publishes the raw matrix, joint piece
   gets a fourth row in the workload table~~ **CLOSED 2026-05-23
   12:39 UTC** — Robin shipped the 26b 2×2 matrix same-day. 80
   calls, zero failures, companion repo
   [triavalabs/gemma4-26b-mode-split](https://github.com/triavalabs/gemma4-26b-mode-split)
   + issue [#448](https://github.com/Hashevolution/James-RAG-Evol/issues/448)
   on this repo. Outcome documented in §Track 5 — 2026-05-23 Robin
   26b data arrival + 3-axis discovery (below).
3. Decision on whether the "very heavy synthesis" arm is worth a
   fourth V3'.e cell — would close the gradient measurement at the
   high end. Defer pending discussion.

### Track 5 — 2026-05-23 Ali + Robin LinkedIn comments (post-PR-#440 thread)

PR #440 (V3'.e result) shipped UTC 05:42 / KST 14:42 on 2026-05-23.
A LinkedIn share thread carrying the V3'.e narrative drew four
substantive comments the same day — one from Ali, three from Robin —
that lock the joint-piece headline, expand Robin's 26b experiment to
mirror our protocol, and elevate JAMES raw JSON from "data we publish"
to "schema another lab analyses against".

#### Comment 1 — Ali Afana (≈1h after share)

> "Substitution is free. Synthesis costs in proportion to what it
> has to invent — cost asymmetry in ten words. The three-workload-
> level signature turns Robin's qualitative split into something
> measurable across stacks."

The 10-word phrase appeared as a **headline candidate** in
`v3prime-e-substitution-synthesis-result.md §Implications` —
Ali re-derived it independently from the data and endorsed it before
Robin saw the thread. Cost-asymmetry framing now has two-author
attribution before any joint-piece draft.

#### Comment 2 — Robin Converse (≈sub-reply to JAMES comment, initial plan — later superseded)

> "V3'.e is the right next experiment. If only synthesis hits the
> floor, the conclusion sharpens to 'synthesis-mode entry has a
> token cost' — which is testable, falsifiable, and design-
> actionable. Operators who didn't know synthesis had a cost can
> now budget for it.
>
> The 26b MoE sweep this weekend stays as designed (single
> num_predict variable, fixed prompt type), but adding V3'.e-style
> prompt-type variation as a second pass is worth the extra hour.
> Two-dimensional sweep across cap × prompt-type on 26b would give
> the joint piece direct apples-to-apples comparison with your e4b
> V3'.e data. Sunday outputs: floor measurement (your protocol,
> n=10) + prompt-type variation. Both go in the same data dump."

**Scope expansion**: Robin's originally-1D sweep grows to 2D —
cap × prompt-type — on her 26b MoE side. This is the largest
collaborator-side scope expansion the track has produced.

**Note on timing**: this comment scheduled the matrix for the
*weekend* with *Sunday outputs*. Comments 3 and 4 (below) **pull
the schedule forward to "today" (2026-05-23)** with N escalated
from n=10 to N=20/cell. Treat this comment as the *scope* commit
(2D matrix accepted) and comments 3+4 as the *schedule + N*
commits.

#### Comment 3 — Robin Converse (≈second same-day entry)

> "Cost asymmetry in ten words — Ali's right, that's the line.
> Substitution is genuinely floor-immune; synthesis pays in
> proportion to invention. The three-workload-level signature is
> what makes the framing reproducible.
>
> 26b matrix building from your protocol today. If the gradient
> holds at higher parameter counts, 'across stacks' becomes
> 'across stacks and across model scales' — the framing's
> portability gets a second confirmation axis. PR #440 noted."

**Three-author consensus** on the 10-word framing before any
joint-piece draft. Schedule **pulled forward** — 26b matrix
construction starts 2026-05-23, not Sunday. **New portability
axis** introduced: "across stacks" (E4B Ollama vs 26b MoE) +
"across model scales" (4B vs 26B parameter counts). PR #440 noted
as the published-artifact anchor Robin runs against.

#### Comment 4 — Robin Converse (≈third same-day entry, protocol-lock)

> "Jiwon — reference baseline received. ~60-token flat
> substitution and 400-450 synthesis-with-recommendation are clean
> signatures. Running 26b 2×2 matrix today: cap × prompt-type at
> N=20/cell, mirroring your protocol exactly. If 26b substitution
> also flatlines around 60 tokens and synthesis scales
> proportionally, architecture-invariance is in hand. If either
> signature shifts, that's the next research thread. Pulling your
> raw JSONs as the analysis template. Data posted when it lands."

**Three concrete locks for the next experiment cycle**:

(a) **Robin's experiment shape is now exact-mirror of `scripts/
research/v3prime_e_mode_split.py`** — N=20/cell × 2 caps × 2
prompt types = 4 cells on 26b. Same N, same axis structure, same
arm semantics.

(b) **Architecture-invariance hypothesis is explicit** (reference
signatures: substitution `ollama_eval_count=62` true-flat on all 40
calls — not averaged; synthesis 400-450 tokens proportional):

| 26b 2×2 outcome | Interpretation | Joint-piece consequence |
|---|---|---|
| substitution flat @ ~62 tokens AND synthesis @ 400-450 proportional | Architecture-invariance confirmed — gradient is Gemma-4 family property, not e4b-specific | "across stacks and across model scales" portability claim lands in joint piece headline |
| Either signature shifts | Architecture sensitivity uncovered | Becomes "next research thread" — joint piece narrows to e4b-confirmed claim + open question for follow-up |

(c) **JAMES raw JSON (`reports/research-runs/v3prime-e-mode-
split-20260523T051{654,851}.json`) is adopted as cross-stack
analysis template** — Robin will pull our JSON shape and analyse
her 26b data against the same schema. First time JAMES research
artifact graduates from "data we publish" to "schema another lab
analyses against". Strongest collaboration-tier elevation on
the track to date.

#### Implications for the next experiment cycle (JAMES side)

- **Data-arrival watch**: Robin's 26b 2×2 matrix posts as a single
  artifact (per her comment 2 commitment). When it lands, the
  decision tree in (b) above runs immediately — no further design
  needed.
- **JSON schema stability**: do NOT alter the field shape of
  `reports/research-runs/v3prime-e-mode-split-*.json` until
  Robin's 26b data has been merged into the cross-stack
  comparison. Any schema migration after she has pulled the
  template breaks her downstream analysis.
- **"Very heavy synthesis" arm decision** (item 3 above) now has
  a natural trigger: if Robin's 26b data confirms architecture-
  invariance, the gradient measurement is *complete enough* to
  publish; if it shifts, the high-end arm becomes the obvious
  next driver to disambiguate.
- **Headline lock**: "Substitution is free. Synthesis costs in
  proportion to what it has to invent" is now the joint-piece
  candidate headline with three-author standing. No further
  framing debate needed; the data outcome decides whether the
  headline ships as-is or with a portability sub-clause.
- **Track 3 timing unaffected**: Ali's mid-June Gemini backend +
  `scripts/swap_eval.py` remain the managed-vs-sovereign axis.
  Robin's 26b data is the parameter-scale axis. The joint piece
  can now triangulate on three independent axes: deployment
  context (Ali), model scale (Robin), task weight (JAMES).

### Track 5 — 2026-05-23 Robin 26b data arrival + 3-axis discovery

Robin's 26b matrix landed **the same day she committed to it** —
companion repo + JAMES issue filed within hours of the original
4th LinkedIn comment. Operational discipline: 80 calls, 6 min
runtime, zero failures.

- **Companion repo**: [triavalabs/gemma4-26b-mode-split](https://github.com/triavalabs/gemma4-26b-mode-split) (created 2026-05-23 UTC 12:26; pushed 12:30; issue filed 12:39 — total 13 min repo creation to issue file)
- **Issue on this repo**: [#448](https://github.com/Hashevolution/James-RAG-Evol/issues/448) (Robin chose issue rather than comment on locked PR #440 — operationally precise)
- **Endpoint**: `https://api.triavalabs.com` (sovereign Ollama, Hetzner CCX33, public HTTPS via Caddy reverse proxy)
- **Model**: `gemma4:26b` MoE, 25.8B params, Q4_K_M, base, no system prompt
- **Protocol**: T=0.2, N=20/cell, `think: False`, exact V3'.e fixture structure

#### Pre-registered decision tree outcome — both arms partially true

The pre-registration (Robin's 4th LinkedIn comment, locked above):

| Pre-reg arm | Predicted signal | What actually arrived |
|---|---|---|
| Architecture-invariance confirmed | Substitution flat @ ~62 tokens AND synthesis @ 400-450 tokens proportional | ❌ Both signatures shifted (62→38; 400-450→49-54) |
| Either signature shifts → next research thread | Numerical signatures off | ✅ Both off — strict reading triggers this arm |

**Strict reading**: next research thread. **Robin's substantive
reading** (her [`analysis.md`](https://github.com/triavalabs/gemma4-26b-mode-split/blob/main/analysis.md)
+ issue #448 body): *"Framing held; the data sharpened it."* The
mode-split is preserved on 26b (substitution still deterministic;
synthesis still has a per-task cost). The signatures shifted in a
**systematic direction** — 26b is more efficient — which is *new
information*, not noise.

The honest synthesis: **both readings are simultaneously true**.
The pre-registered binary outcome was too narrow. Both signatures
shifted, but in a structured way that reveals a 3rd axis rather
than breaking the mode-split framing.

#### The 3rd axis — model-scale efficiency

Joint piece now has 3 independent axes, one per contributor:

| # | Axis | Owner | Evidence | Status |
|---|---|---|---|---|
| 1 | Mode split (substitution vs synthesis) | Robin (LinkedIn 2026-05-22) | T-sweep on 26b + bit-for-bit determinism on substitution arm (1 unique response across 40 calls) | Confirmed on both stacks |
| 2 | Workload gradient (heavy / light / no synthesis) | JAMES (V3'.e + V3'.a~d) | 4-stage 0/10 + e-commerce 14/20 + verbatim 20/20 @ cap=400 | Confirmed on e4b |
| 3 | Model-scale efficiency (synthesis cost ∝ 1/param-count; substitution invariant) | Robin (26b sweep, this section) | 26b synthesis ~9× more token-efficient than e4b on same fixture; substitution stays bit-for-bit deterministic | New — emerged from the architecture-invariance test itself |

Authorship rotation in the original decision matrix is now
**three-author with explicit role-split**:

- **Ali** — chatbot-context divergence (managed Gemini side, mid-June Track 3)
- **Robin** — mode split (LinkedIn 2026-05-22) + model-scale efficiency (26b sweep, 2026-05-23)
- **JAMES** — workload gradient (V3'.e + V3'.a~d 4-stage validation)

#### Headline status — 10-word phrase holds; sub-clause proposed

The 3-author-locked 10-word phrase still holds. The 3rd axis
becomes a sub-clause, not a re-write:

| Phrase | Status |
|---|---|
| *"Substitution is free. Synthesis costs in proportion to what it has to invent."* | 3-author locked (Ali + Robin + JAMES) — **unchanged** |
| *"…and inversely to parameter count."* (extension) | Draft proposal — Robin's data supports it; Ali has not yet weighed in; not 3-author locked |

#### Hand-off shape — Robin explicitly defers next-round framing to JAMES + Ali

Verbatim closing line of Robin's 4th comment (Image 5):

> *"Excited to see where you and Ali take this next."*

She delivered the data + the 3-axis read, then **explicitly handed
the next-round framing decision to JAMES + Ali**. Next moves on
our side:

1. **Decide whether to acknowledge issue #448** — strict like-only
   posture (per [[feedback_high_stakes_endorsement_posture]] in
   memory) applies to LinkedIn endorsement comments, NOT to a
   data-bearing GitHub issue on our own repo. Acknowledging the
   issue with a substantive technical reply is the "data-round
   response" pattern, not over-engagement. **Operator decision**.
2. **Decide whether to extend V3'.e to record unique-output count
   per cell** so we can verify Robin's "bit-for-bit deterministic
   substitution" claim on our e4b data (her Finding 1). If
   confirmed, this becomes a **publishable mechanism** in its own
   right ("the mode bypasses sampling") and strengthens axis 1.
3. **Decide whether to extend V3'.e with a "very heavy synthesis"
   4th arm** to close the gradient measurement at the high end
   (item 3 of "What's still pending" above, now the only
   remaining open V3'.e design question).
4. **Hand-off to Ali** — the 3rd axis (model-scale efficiency) is
   not in his Gemini API roadmap yet. The natural cross-link is:
   once his Track 3 swap landing produces a managed-Gemini result,
   we have **deployment-context × model-scale × task-weight = 3D
   parameter space** for the joint piece.

#### What this changes about the original Track 5 decision matrix

The matrix's "1 mechanism" assumption was already broken by V3'.e
(2 mechanisms + gradient). The 26b data breaks it further: the
gradient itself has a scaling axis. The matrix stays verbatim
above as the audit trail; the operative narrative is now the
**3-axis joint piece** described above, not the original
"divergence collapses on X vs Y" rows.

---

## What this handover deliberately does *not* cover

- The actual experiment runs. Those happen across two sessions on two physical machines.
- Ali's Gemini API side implementation. We owe him the contract, not the implementation.
- Any other Ali-direction collaboration not surfaced in the launch-tracker yet (e.g. if he proposes a third joint piece).
- The Part 3 / Part 4 follow-up arc — those are emergent from how Tracks 1–5 land, not pre-planned.

## How a future session uses this page

1. Pick **one** of Tracks 1 / 2 / 3 / 4 / 5.
2. Do *not* start with Track 5; it is the synthesis step.
3. Tracks 1 and 2 are independent — Track 2 can land first if calendar pressure makes it urgent (Ali's 1–2 week commitment expires 2026-06-01).
4. Track 3 depends on Track 1; do not start it before Track 1's `Provider` ABC is merged.
5. Once a Track is picked, mark its row in the **Commits made in writing** table above with `🟡 in progress` and a branch name. When merged, flip to `✅ done` with the PR number.

## Calendar

| Date | Milestone |
|---|---|
| 2026-05-18 | This handover lands |
| **2026-05-19** | **Track 1 closed ~27 days ahead of schedule (#324 / #325 / #326). Ali turn 7 confirms mid-June Gemini backend + `ar_ecommerce.yaml` 6/1 hold.** |
| 2026-05-19 to 2026-05-26 | **Operator window — Track B CLA GUI 4 steps** must complete before Ali's mid-June PR (it will be our first external PR). |
| **2026-06-01** | **Track 2 deadline — `ar_ecommerce.yaml` from Ali (15-20 fixtures × 4 categories + ≥ 5 benign).** Validator in #322 is the landing pad; no further code action our side. |
| **2026-06-15** | ~~Track 1 deadline (our 2–4 week commitment for the Provider contract)~~ **— met early (2026-05-19).** Stays as the calendar anchor for Ali's mid-June Gemini backend window. |
| 2026-06-15+ | Track 3 (swap infrastructure) activates once Ali's Gemini backend lands. Pre-work (`scripts/swap_eval_report.py`) can land in our queue before then. |
| 2026-06-15+ | Track 4 (scope-lock template) signed by both. |
| 2026-06-22+ | Cross-experiment runs (parallel: ours on JAMES, Ali's on Gemini API). |
| 2026-06-29+ | Track 5 (writeup) starts on actual data. |

~~Slip risk: Track 1 is closed early; the new tight item is the **operator-side CLA GUI window** (2026-05-19 → 2026-05-26). If those four GUI steps miss the calendar, Ali's mid-June PR opens against a half-armed CLA workflow.~~ **Resolved 2026-05-20** — CLA GUI window closed 6 days early, dry-run verified the bot fires the signature request on external PRs end-to-end. No remaining slip risk on the operator side; the next calendar-locked item is **Ali's 2026-06-01 `ar_ecommerce.yaml` arrival** (no action our side; validator in PR #322 is the landing pad).
