---
version: 0.21.1
date: 2026-06-18
headline: "Ship the Qwen2.5-3B Q8 GPU benchmark data — the kernel-core forward pass runs the 3B model on an 8 GB laptop 4070, argmax-exact."
themes:
  - gpu
  - bench
---

**TL;DR** — Captured benchmark data for the **kernel-core** (in-kernel CUDA backend) running
Qwen2.5-3B-Instruct Q8 on the RTX 4070 is now a shipped artifact, not a one-off run. The raw
`modelbench` JSON plus a self-contained README live under `fak/experiments/gpu/`. This is fak's
own pure-Go forward pass on the GPU (`-tags cuda`, `FAK_CUDA_Q8`), **not** a model fronted by
`fak serve`.

## GPU benchmark data (#gpu #bench)

- **What shipped** — `fak/experiments/gpu/qwen2.5-3b-q8-cuda-4070.json` (the `modelbench -out`
  report) + `fak/experiments/gpu/README.md` (conditions, witness, reproduce command).
  - *Why:* `GPU-QWEN-RESULTS.md` narrates the result; this lands the underlying data as a
    committed artifact so the number is auditable, per the house rule "every number is a real run."
  - *How:* `modelbench -hf <qwen2.5-3b> -lean -backend cuda -decode-steps 128 -decode-reps 5 -out …`.

- **Correctness — argmax-exact.** `gpucheck -lean -n 12` shows the CUDA Q8 device decode is
  token-for-token identical to the CPU-Q8 reference on the real 3B model; greedy decoding yields
  coherent English ("…the history of the internet. Certainly! The history of the").

- **Throughput (this run, v0.21.0 code, clean GPU clock).** Decode **25.1 tok/s** (39.8 ms/tok,
  128 steps × 5 reps), prefill ~25 tok/s, ~49 s load, ~3.5 GiB resident on the 8 GB 4070. Decode
  stays WSL-launch-bound vs `llama.cpp` Q8 (~32 tok/s) — the lever is a capture-safe CUDA graph,
  not the kernel (`GPU-QWEN-RESULTS.md` §4).

Data/docs only — no Go code, ABI, or behavior changed; the suite is green.
