# Qwen3.5-122B-A10B — Full Evaluation & Decommission Record (2026-05-24)

**Outcome: the 122B escalation model was DECOMMISSIONED.** The everyday Qwen3.6-35B-A3B matches it on
quality, is 2–4× faster, uses half the memory, and runs full context effortlessly on crow. This document
preserves the methodology + numbers so the evaluation can be re-run quickly **if a new 122B-class model ships.**

## Hardware / setup
- crow: AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151), **124 GB UNIFIED memory** (vram=0, gtt=124 GiB).
- Everyday model: **Qwen3.6-35B-A3B** UD-Q6_K (28 GB) + mmproj-F16, **Vulkan** RADV, MTP draft-n2, vision, `:8003`.
- Escalation model (now dropped): **Qwen3.5-122B-A10B** (MoE 122B/10B-active, hybrid Gated-DeltaNet) UD quants,
  **ROCm** 7.2.3 (build 9189), MTP draft-mtp, vision. Bench harness: `scripts/bench/` (mtp-explore.sh,
  quality-eval.sh). Eval corpus = Gutenberg prose in `results/quality/wiki.test.raw` (misnamed; it's Gutenberg).

## 1. 122B quant quality — PPL + KLD (Gutenberg, CHUNKS=48 CTX=512; KLD vs Q5_K_M reference since BF16=249 GB)
| Quant | On-disk | PPL | Δ vs Q5 | Mean KLD vs Q5 | Same-top-token | RMS Δp |
|-------|---------|-----|---------|----------------|----------------|--------|
| Q5_K_M (reference) | 88 GB | 1.7743 | — | — | — | — |
| **Q4_K_XL** | 74 GB | 1.7802 | +0.34% | 0.0074 | 98.0% | 3.1% |
| **Q3_K_XL** | 55 GB | 1.8276 | +3.0% | 0.0305 (~4× Q4) | 96.0% | 7.0% |

**Verdict:** Q3 has a SMALL but REAL quality cost vs Q4 — +2.7% PPL, ~4× the KLD divergence, top-token
disagreement ~doubles (4% vs 2%). For scale, the 35B-Q6 was "near-lossless" at KLD 0.012 vs BF16; Q3-122B is
0.031 vs Q5. Q3 did NOT break any hard task (6/6 correct) but it is not "free." Q3's only benefit is ~20 GB
headroom; it is NOT faster than Q4 (Q4's MTP draft head accepts more: 85% vs 72%).

## 2. 122B speed (ctx 16K, ROCm, q8_0 KV, greedy, 5-run median)
| Config | code tok/s | prose tok/s | draft accept |
|--------|-----------|-------------|--------------|
| Q4 MTP-n3 | 31.3 | 28.3 | 85% / 75% |
| Q3 MTP-n3 | 29.9 | 29.2 | 72% / 70% |
| Q3 MTP-n2 | 30.6 | 30.0 | 82% / 80% |
(Prior committed Q4-n3 without KV-quant: 33.6 code. KV q8_0 costs ~7% gen speed.)

## 3. 35B vs 122B — quality
- **Hard tasks (our setup, 6 verifiable prompts: primes-trap, 100! trailing zeros, binary-search bug,
  kth-smallest-matrix, weber/SI units, 3-boxes logic):** BOTH **6/6 correct**, near-identical reasoning.
  No separation. (Raw outputs in `results/122b-eval-outputs/`.)
- **Published benchmarks (official model cards) — essentially a wash, split by category:**
  | Benchmark | 35B (Qwen3.6) | 122B (Qwen3.5) |
  |-----------|---------------|----------------|
  | SWE-bench Verified | **73.4** | 72.0 |
  | LiveCodeBench v6 | **80.4** | 78.9 |
  | Terminal-Bench 2.0 | **51.5** | 49.4 |
  | MMLU-Pro | 85.2 | **86.7** |
  | SuperGPQA | 64.7 | **67.1** |
  | HLE | 21.4 | **25.3** |
  | GPQA Diamond | 86.0 | 86.6 |
  | MMMU (vision) | 81.7 | **83.9** |
  → Newer **35B edges coding/agentic**; **122B edges knowledge/hardest-reasoning/vision** by 1–4 pts.
  Both: 262K native context, 1M extensible. (NOTE: PPL is NOT valid cross-model — different generations/
  tokenizers; the 35B's *lower* prose-PPL does not mean "better".)

## 4. 35B vs 122B — long context on this hardware
| Depth | 35B tg/s (Vulkan) | 122B-Q3 tg/s (ROCm) | 35B faster |
|-------|-------------------|---------------------|-----------|
| ~0 | 66.7 | 28.5 | 2.3× |
| 32K | 44.7 | 21.1 | 2.1× |
| 110K | **51.7** | **12.7** | **4.1×** |
- Needle-in-haystack @110K: **both PASS** retrieval. Equal comprehension.
- Memory free: 35B 55→49 GB (0→110K); 122B-Q3 30→23 GB. 35B uses ~half.
- The 122B degrades to a sluggish 12.7 tok/s deep; the gap widens with depth.

## 5. 35B full-context stress (PROD config: Vulkan, DEFAULT checkpoints, n2, vision)
- Filled to **244,000 tokens (93% of 262K)** + 5 multi-turn-at-depth + prompt-save spike → **SURVIVED**.
- Near-full specs: **31.4 tok/s**, used 81–87 GB / **34–43 GB free**, 31 checkpoints (capped at 32, no OOM).
- The 35B accumulates checkpoints (same mechanism that crashed the 122B) but its small footprint leaves a
  huge margin — full context on crow is a non-issue for the 35B.
- (Prompts >262K are REJECTED with HTTP 400 `exceed_context_size_error`, not truncated — size eval prompts under -c.)

## Decision & rationale
**Drop the 122B.** It is not a general upgrade: quality-equivalent on hard tasks, the newer 35B even edges it
on coding, while being 2–4× faster, half the memory, and trivially full-context. Its only edge (knowledge/
hardest-reasoning/vision, +1–4 pts) does not justify the cost (slow, memory-tight, 216 GB disk, ongoing
stabilization). The stabilization work (below) is preserved in case a future 122B is worth it.

## Stabilization lessons (preserved for a future large model on crow)
- **Unified memory:** treat RAM as ONE 124 GB GPU+CPU pool; Vulkan-vs-ROCm is irrelevant to OOM.
- **ctx-checkpoint accumulation** crashed the 122B at ~129K (default 32 checkpoints ≈ 13 GB accumulated across
  multi-turn) → kernel OOM (137) OR libamdhip64 HIP segfault (139). Fix: `--ctx-checkpoints 0
  --checkpoint-every-n-tokens -1`. The dormant bundle `bundles/llamacpp-rocm-qwen35-122b-mtp/docker-compose.yml`
  retains the full stabilized config (checkpoints-off, q8_0 KV, n-max2, oom_score_adj 800, mmproj) as a template.
- **mmap HANGS** on this box (>8 min load) — keep `--no-mmap`. CPU-MoE offload is moot on unified memory.
- **OS safety net:** sshd already oom_score_adj=-1000; large models should set oom_score_adj high (preferred
  victim). Consider `earlyoom` if OOM recurs.
- To re-evaluate a new 122B: re-run `scripts/bench/quality-eval.sh` (PPL/KLD) + `mtp-explore.sh measure` (speed)
  + the hard-task/needle/full-context scripts (archived approach in this doc), compare to the 35B baselines above.
