---
title: PF8.1 BF16→FP8 e4m3 quant kernel — GPU runtime smoke PASS
date: 2026-05-10
type: research
status: pf8.1-runtime-verified
---

# PF8.1 BF16→FP8 e4m3 quant kernel — GPU runtime smoke PASS

> Per skill v1.11.0 #28+#31: build clean ≠ runtime correct. PF8.1
> kernel from `940f49e` was build-verified (cargo check 3m 40s) but
> NOT GPU-runtime-verified. This entry documents the standalone smoke
> test that confirms runtime correctness on sm_89 RTX 4070 Ti SUPER.

## §0 Direct evidence (raw nvcc + run output this tick, NOT memory recall)

### Build (g++-14 host compiler per ARLE convention)

```bash
$ NVCC_CCBIN=/usr/bin/g++-14 nvcc -arch=sm_89 -O3 -ccbin=/usr/bin/g++-14 \
    /tmp/test_pf8_quant.cu \
    /home/ckl/projects/arle/crates/cuda-kernels/csrc/gemm/w4_fp8_activation_quant.cu \
    -o /tmp/test_pf8_quant
[clean compile, no errors or warnings]
```

g++-16 (system default) breaks NVCC headers; ccbin=g++-14 required
(matches codex's pattern from prior cuda-kernels builds).

### Run

```bash
$ /tmp/test_pf8_quant
Testing PF8.1 BF16->FP8 e4m3 quant: rows=128 cols=256
Per-row scales (first 8): 0.0022 0.1138 0.2243 0.3371 0.4464 0.5603 0.6696 0.7812
Max abs err: 12.9521, max rel err: 0.0599
Outliers (rel_err > 0.20 with |orig| > 1e-3): 0 / 32768
PASS: PF8.1 quant runtime correct
```

### Test design

128 rows × 256 cols BF16 input. Per-row varying magnitudes (1, 51,
101, ..., 351) to test scale adaptation. Verifies dequant(quant(x)) ≈ x
within FP8 e4m3 precision (~3 mantissa bits, ~12.5% precision floor
per IEEE FP8 spec).

Smoke harness: `/tmp/test_pf8_quant.cu` (~110 LOC, standalone single-
binary, no Rust/cargo dependency).

## §1 Verdict

**PASS** on all gates:
- Per-row scales adapt linearly with input magnitude (0.0022 → 0.7812
  across 8 rows of increasing magnitude)
- Max relative error 5.99% < expected ~12.5% FP8 precision floor
  (kernel + scaling work as designed)
- Zero outliers (rel_err > 20% with non-trivial |orig|) out of 32768
  elements

Conservative reading: PF8.1 kernel is runtime-correct for the
small-shape test. Production-shape test (4096 cols typical) deferred
to PF8.5 A/B bench gate.

## §2 What this proves vs. doesn't

PROVES:
- Kernel compiles cleanly on sm_89 with g++-14 host compiler
- Kernel runs without segfault/CUDA error on real GPU
- Per-row absmax reduction works correctly (scales reflect input
  distribution)
- FP8 e4m3 conversion via `__nv_fp8_e4m3(qf)` constructor works
  (sm_89 native intrinsic)
- Dequant(quant(x)) precision matches FP8 e4m3 spec (~6%, well
  within ~12.5% floor)

DOES NOT prove:
- Production-scale shape (e.g. 4096 cols × 1024+ rows) behavior
- Performance vs INT8 quant kernel
- End-to-end correctness in W4+FP8 GEMM pipeline (PF8.3 GEMM kernel
  doesn't exist yet)
- PF8.2 weight preprocess (separate kernel, separate smoke needed)

## §3 Cooperative discipline applied (per skill v1.11.0 #28+#30+#31)

- Raw nvcc + run output quoted in §0 (per #28+#31: every claim has
  raw evidence in same response)
- Single-file commit (this research entry only); skill catalog edit
  earlier in tick was a separate commit (per #30 race-window discipline)
- File `/tmp/test_pf8_quant.cu` not committed (it's `/tmp/` smoke, not
  permanent — same pattern as `/tmp/cutlass_fp8_smoke.cu` and
  `/tmp/fp8_smoke.cu` from earlier in session)

## §4 What's NEXT for PF8 chain

After PF8.1 runtime PASS:
- **PF8.2 standalone smoke** (separate tick, ~80 LOC test for INT4
  weight preprocess kernel): verify subtraction-merge zero-point
  produces expected bit pattern
- **PF8.3 FP8 marlin GEMM** (~700 LOC): codex's natural pickup given
  P0.A spike experience. If codex idle for 2-3 more ticks, Claude
  self-implements mirroring marlin_w4a8_kernel.cu.
- **PF8.4 dispatch wiring** (~50 LOC, can land anytime): adds
  `MarlinW4FP8Prefill` enum variant + env var `INFER_MARLIN_W4_FP8_PREFILL`.
- **PF8.5 A/B bench**: end-to-end TTFT comparison W4+INT8 vs W4+FP8
  prefill on Qwen3-4B 4k workload. License gate per a66d99a §2:
  TTFT p50 Δ ≥ -8% with σ < 5% n=3.

## §5 Cross-references

- PF8.1 source: `crates/cuda-kernels/csrc/gemm/w4_fp8_activation_quant.cu` (940f49e)
- FFI binding: `crates/cuda-kernels/src/ffi/gemm.rs` `quantize_bf16_rows_to_fp8_e4m3_cuda`
- PF8.2 source: `crates/cuda-kernels/csrc/gemm/marlin_int4_fp8_preprocess.cu` (940f49e)
- Directive: `docs/research/2026-05-10-prefill-only-fp8-directive-draft.md` (a66d99a)
- Smoke harness: `/tmp/test_pf8_quant.cu` (this tick, /tmp/ scope per session pattern)
- Skill v1.11.0 anti-patterns #28+#31: `.claude/skills/kernel-optimization/SKILL.md` (b551bea)

## §6 Status

PF8.1 runtime SMOKE PASS confirmed empirically. Build-clean ≠
runtime-correct gate now satisfied. PF8.2 smoke is the natural next
empirical verification (small ~80 LOC test). Then PF8.3 GEMM kernel
implementation OR codex pickup of same.
