# P1.3 Quantized Fused MLP Killed — TTFT Regression

## Context

P1.3 tested `INFER_QUANT_FUSED_MLP=1` after P1.2 graph capture hoist (`ca0673b`) landed. The target was the hybrid W4A16/W4A8 prefill MLP path at the canonical longctx shape:

- model: `infer/models/Qwen3-4B-W4-hybrid-zpfix`
- server: `--num-slots 8 --max-seq-len 5120`
- env common: `INFER_HYBRID_W4A8_PREFILL=1`
- workload: 4096 input / 256 output / concurrency 4 / 120s / 10s warmup

The attempted implementation used a W4A8 fused gate+up sidecar for prefill and kept W4A16 Marlin gate/up for decode. Two decode/prefill boundary bugs were found and fixed during the experiment before benching:

- pure batched decode must not route through the prefill-only W4A8 gate_up sidecar, or CUDA graph capture asks for W4A8 scratch on a W4A16 decode path;
- pure decode `BatchDecodeBuffers` must size gate/up scratch from real decode-fused state, not from prefill-only fused state.

Those fixes made graph warmup capture B=1..8 cleanly, but the performance gate still failed.

## Results

Raw artifacts:

- env off: `bench-output/2026-05-09-p13-quant-fused-mlp-off-c4-run2/`
- env on: `bench-output/2026-05-09-p13-quant-fused-mlp-on-c4-run2/`

| Mode | TTFT mean | TTFT p50 | ITL p50 | out tok/s | total tok/s |
|---|---:|---:|---:|---:|---:|
| `INFER_QUANT_FUSED_MLP=0` | 1704.9 ms | 1639.0 ms | 11.45 ms | 224.42 | 3816.08 |
| `INFER_QUANT_FUSED_MLP=1` | 1819.6 ms | 1759.3 ms | 11.47 ms | 219.11 | 3725.67 |
| Delta | +6.7% | +7.3% | +0.2% | -2.4% | -2.4% |

License gate was `TTFT -5%..-15%` versus P1.0/P1.2 baseline. The env-on run regressed TTFT, so P1.3 is killed and the runtime diff was not shipped.

## Root Cause

The 3-launch fused gate+up route reduced launch count but made the W4A8 Marlin GEMM shape worse:

- separate path: two `N=9728` W4A8 GEMMs, then `silu_mul`, then down GEMM;
- attempted fused path: one `N=19456` W4A8 GEMM, then `silu_mul_split`, then down GEMM.

At this shape, the larger W4A8 output path likely loses more in Marlin scheduling, output conversion, or memory traffic than it saves in launch count. The unchanged ITL confirms this is prefill-only, not decode graph fallout.

## Fix

No runtime fix was landed. Keep P1.3 killed unless a lower-level profile proves a different fused shape wins.

Next viable experiments:

- profile separate `N=9728` vs fused `N=19456` W4A8 Marlin GEMM directly with CUDA events;
- test Option A instead: keep gate/up separate and only fuse `silu_mul` into down epilogue;
- skip P1.3 and continue on quant/KV axes with measured wall-clock gates.

## Rule

Launch-count reductions are not a license. For quantized Marlin, validate the exact fused GEMM shape before wiring model-level dispatch. Wall-clock TTFT remains the gate; env-gated substrate does not ship when the gate regresses.
