# P1.6 QKV Packing Killed — Flat TTFT and KV Pressure

## Context

P1.6 tested an env-gated dense BF16 QKV packing path for Qwen3 prefill:

- gate: `INFER_QKV_PACKED=1`
- model: `infer/models/Qwen3-4B`
- server: `--num-slots 8 --max-seq-len 5120`
- workload: 4096 input / 256 output / concurrency 4 / 120s / 10s warmup
- commit under test: `c41198d` plus the temporary P1.6 runtime diff

The implementation loaded a packed side tensor for `q_proj + k_proj + v_proj`,
ran one dense BF16 GEMM, then split the packed output back into the existing
`q_batch`, `k_batch`, and `v_batch` buffers. Quant, TileLang, LoRA, and default
env-off behavior were left untouched.

Correctness gates passed before benchmarking:

| Gate | Result |
|---|---|
| `linear_batch_qkv_packed_matches_separate_qkv` | PASS |
| `cargo test --release -p infer --features cuda --test e2e -- --test-threads=1` | PASS |
| `INFER_QKV_PACKED=1 ... --test e2e` | PASS |
| `INFER_QKV_PACKED=1 ... test_greedy_solo_vs_concurrent` | PASS |
| manual sample inspection | coherent, no garbage-output pattern |

The full default `greedy_consistency` suite still contains the known W4A8
accuracy failure; the BF16 solo-vs-concurrent gate passed.

## Results

Raw artifacts:

- env off: `bench-output/2026-05-09-p16-qkv-off-c4-r1/`
- env off: `bench-output/2026-05-09-p16-qkv-off-c4-r2/`
- env off: `bench-output/2026-05-09-p16-qkv-off-c4-r3/`
- env on: `bench-output/2026-05-09-p16-qkv-on-c4-r1/`
- env on: `bench-output/2026-05-09-p16-qkv-on-c4-r2/`
- env on: `bench-output/2026-05-09-p16-qkv-on-c4-r3/`

Paired valid runs:

| Mode | Run | TTFT mean | TTFT p50 | ITL p50 | out tok/s | peak kv_util | request errors |
|---|---:|---:|---:|---:|---:|---:|---:|
| env off | r1 | 1986.0 ms | 1980.1 ms | 19.29 ms | 150.85 | 85.5% | 0 |
| env off | r2 | 1983.0 ms | 1981.4 ms | 19.30 ms | 150.85 | 93.6% | 0 |
| env off | r3 | 1983.2 ms | 1983.0 ms | 19.30 ms | 150.81 | 79.8% | 0 |
| env on | r1 | 1988.4 ms | 1979.7 ms | 19.29 ms | 150.83 | 100.0% | 0 |
| env on | r2 | 1980.3 ms | 1979.2 ms | 19.30 ms | 150.85 | 89.3% | 0 |

The valid env-on runs were statistically flat versus env-off:

| Metric | env off median | env on valid median | Delta |
|---|---:|---:|---:|
| TTFT p50 | 1981.4 ms | 1979.5 ms | -0.1% |
| ITL p50 | 19.30 ms | 19.30 ms | 0.0% |
| out tok/s | 150.85 | 150.84 | 0.0% |

The third env-on run was invalid:

| Mode | Run | Completed output tokens | req/s actual | trace samples | after-stats |
|---|---:|---:|---:|---:|---|
| env on | r3 | 3072 | 0.109 | 43 ok / 88 failed | unavailable |

`p16-qkv-on-c4-r3` also reported `conc p50=0`, which means the run no longer
represented the requested `c=4` production shape.

## Root Cause

The naive packed-weight implementation duplicated Q/K/V weights instead of
replacing the original tensors. That extra resident side tensor consumed about
1 GiB on Qwen3-4B and reduced the CUDA scheduler's KV headroom. The first valid
env-on run already reached `peak kv_util=100.0%`; the third run then lost server
availability during service-stat polling and completed only 3072 output tokens.

Even before the instability, the optimization did not move the wall-clock
number. The likely reason is that the saved two GEMM launches per layer were
offset by:

- the extra output split launch;
- duplicate model memory reducing T0 KV capacity;
- no direct handoff from packed QKV output into attention;
- c=4 longctx prefill not being launch-bound enough for this shape.

This falsifies the Phase 1 hypothesis as implemented. The license gate required
TTFT `-3%..-5%`; measured TTFT was flat, and one sustained env-on run became
invalid.

## Fix

The P1.6 runtime diff was reverted before commit. No code path, env gate, or
packed-weight substrate landed.

Future variants should only be reconsidered if they avoid the duplicated-weight
cost or remove the split:

1. load-time replacement of Q/K/V with a packed tensor plus decode-safe views;
2. fused packed-QKV output consumed directly by attention metadata/layout;
3. a profile proving Q/K/V projection launch overhead is still material after
workspace and KV-capacity costs are accounted for.

## Rule

Do not ship an opt-in optimization substrate when paired production bench is
flat and the memory footprint harms KV capacity. A launch-count reduction must
pay for every new resident tensor and every new split/copy kernel in the same
wall-clock workload.
