# M3.9 — Mixed Policy Gate + Full Mixed Prefill Budget

## Context

Phase 1A v3 proved the async readback ring was correct substrate, but the
longctx 4k/c=4 owner-run fired kill criteria: default Mixed scheduling regressed
TTFT from `3403 ms` to `7905 ms` and output throughput from `122.5` to
`69.9 tok/s`.

Source survey confirmed this was not a kernel issue. Mixed scheduling capped
the whole mixed prefill token budget at `long_prefill_token_threshold`
(`4096`), while split/pure prefill could use the full `max_prefill_tokens`
budget (`16384`). That forced c=4 long prompts into one prefill row per mixed
tick.

## What Worked

- Added `--scheduler-mixed-policy split|mixed`.
- Defaulted production serving to `split`, restoring the pre-Phase-1A-v3
  production route while keeping the multi-slot readback ring available.
- Kept `mixed` as an explicit opt-in experiment path.
- Fixed `mixed_prefill_token_budget()` to return the full
  `max_prefill_tokens` whole-step budget. The per-request long-prefill cap is
  still applied when prefill candidates are created.
- Gated mixed workspace reserve behind the same policy, so default `split`
  serving does not shrink the KV pool for a disabled mixed path.

## Bench Status

Status: `landed`.

P0' default-Split validation ran on the current `main` line after
`5cacdcb` (`HEAD=adb2757`, docs-only commits after the scheduler fix) on
RTX 4070 Ti SUPER 16GiB, CUDA 13.2, Qwen3-4B. Server envelope:

```text
target/release/infer --model-path infer/models/Qwen3-4B --port 8000 \
  --max-seq-len 12288 --num-slots 8 --max-prefill-tokens 16384

Scheduling envelope (resolved | SGLang-equiv): max_num_batched_tokens=16384 | 16384, chunked_prefill_size=2048 | 2048, max_prefill_tokens=16384 | 16384, mem_fraction_static=0.85 | 0.85, max_slots=8 | (n/a - SGLang has no fixed cap)
```

The bare canonical sweep command requested by the handoff,
`scripts/bench_guidellm.sh p0prime-default-split`, was attempted first. It
entered the sweep saturation region (`waiting=512`) and produced invalid
0-token/connection-failed rows, so it was not used as the regression gate.
The direct gate for the stated target shape used the same wrapper at fixed
longctx 4k/c=4:

```bash
PATH=.venv/bin:$PATH scripts/bench_guidellm.sh p0prime-default-split-c4 \
  --concurrencies 4 --max-seconds 60 --warmup 10
```

Raw artifacts:
`bench-output/2026-05-07-p0prime-default-split-c4/`.

| Metric | F4-Small baseline (`bench-output/2026-05-07-m3.9-phase0-validation`) | P0' default Split | Δ vs F4-Small | Phase 1A v3 bad (`d083b2e`) |
|---|---:|---:|---:|---:|
| TTFT p50 | 3402.6 ms | **1976.4 ms** | **-41.9%** | 7905.4 ms |
| ITL p50 | 20.09 ms | **19.27 ms** | **-4.1%** | 29.43 ms |
| out tok/s | 122.45 | **153.83** | **+25.6%** | 69.94 |
| conc p50 | 4.0 | 4.0 | flat | 4.0 |

GuideLLM headline:

| rate | TTFT mean | TTFT std | TTFT p50 | TTFT p99 | TPOT mean | ITL mean | ITL std | ITL p50 | ITL p95 | ITL p99 | ITL max | E2E mean | E2E p99 | conc p50 | out tok/s | total tok/s | in tok/s | total in | total out | req/s actual |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| conc4 | 1989.2 | 19.7 | 1976.4 | 2029.2 | 26.97 | 19.27 | 0.03 | 19.27 | 19.39 | 19.39 | 19.39 | 6.90 | 6.93 | 4.0 | 153.83 | 2615.79 | 2712.46 | 131104 | 8192 | 0.56 |

Service trace:

- Samples: `78/78` OK; failed: `0`.
- Peak active: `4`; peak running_batch: `4`; peak waiting: `0`.
- Plan labels: `prefill=34`, `decode=2557`, `split=0`, `mixed=0`.
- Peak KV util: `84.5%`.
- Prefix hit rate: `0.0%`.

Conclusion: default production Split is back inside the F4-Small band and is
actually faster on this run. The d083b2e 132% TTFT regression is closed for
the default path. The fixed-c stream stayed in pure prefill/decode rather than
decode-active split, so the acceptance signal is restored c=4 latency plus
`mixed=0`; the Mixed opt-in path remains experimental and is tracked by the
separate Phase 1A validation entry.

## Verification

- `cargo check --release -p infer --no-default-features --features no-cuda`
- `cargo check --release -p infer --no-default-features --features metal,no-cuda`
- `cargo check --release -p infer --features cuda`
- `cargo clippy --release -p infer --features cuda -- -D warnings`
- `cargo test --release -p infer --features cuda mixed_prefill --lib -- --test-threads=1`
- `cargo test --release -p infer --features cuda scheduler_mixed_policy --lib -- --test-threads=1`
- `cargo test --release -p infer --features cuda runtime_defaults_match_documented_defaults --lib -- --test-threads=1`
- `cargo test --release -p infer --features cuda --test greedy_consistency -- --test-threads=1`
- `cargo test --release -p infer --features cuda --test e2e -- --test-threads=1`
- `codex review --uncommitted`: P2 mixed workspace reserve finding fixed.
- `codex review --uncommitted`: second round found no actionable issues.

## Rule

When decode-active prefill has both a per-request row cap and a whole-step cap,
do not reuse the per-request cap as the whole-step budget. That silently turns
cross-request packing off.
