# M3.7 B.1.2 — Async Paged-Prefill Completion

## Context

The 8k/c=4 vLLM control showed ARLE decode ahead on ITL but behind on TTFT.
The remaining scheduler-side prefill issue was the per-chunk
`cuStreamSynchronize` in Qwen3 paged prefill completion.

## What Worked

- Replaced Qwen3 prefill completion sync with a CUDA event recorded on the
  compute stream after the paged-prefill kernels and final logits projection.
- Changed the model prefill completion contract from blocking `Result<()>` to
  non-blocking `Result<bool>`, so the scheduler keeps the pending prefill batch
  alive when the event is not ready.
- Kept the pending prefill buffers, page-table uploads, forward metadata, and
  hidden/logit scratch owned by `Qwen3PrefillContext` until the event completes.
- Added per-row owner validation for async scheduler prefill batches so a
  pending completion cannot be applied to a different slot order.
- Kept launch-error paths in the same pending context. If a paged-prefill
  kernel launch fails after earlier work was enqueued, the scratch buffers stay
  alive until the completion event fires and only then does the scheduler see
  the error.
- Added a short scheduler backoff when the prefill event is not ready, avoiding
  a CPU-tight event-query spin while still avoiding compute-stream sync.
- Preserved synchronous helper behavior without `ctx.sync()` by polling the
  event until ready.

## Verification

- `cargo check --release -p infer --features cuda`
- `cargo check --release -p infer --no-default-features --features metal,no-cuda`
- `cargo clippy --release -p infer --features cuda -- -D warnings`
- `cargo fmt --all --check`
- `cargo test --release -p infer --features cuda --test e2e -- --test-threads=1`
- `cargo test --release -p infer --features cuda --test greedy_consistency -- --test-threads=1`
- Long-prompt HTTP smoke: `prompt_tokens=6500`, `max_tokens=8`,
  `chunked_prefill_size=2048`, chunks reached `2048/6500`, `4096/6500`,
  `6144/6500`, then completed successfully.

## Bench Status

Status: `pending-remote` / owner-run handoff.

The runtime diff is on the CUDA hot path, but the M3.7 acceptance flow assigns
the longctx 8k GuideLLM bench and nsys trace to the owner after this patch is
committed. Codex did not start the long benchmark in this turn so the GPU stays
available for that trace run.

## Bench Command

```bash
scripts/bench_guidellm.sh m3.7-b1.2-longctx-8k
```

Nsight target: Qwen3 `prefill.rs` hot path has zero `cuStreamSynchronize`
calls.

## Environment

- **Backend:** CUDA
- **Model:** Qwen3-4B BF16 (`infer/models/Qwen3-4B`)
- **Hardware:** RTX 4070 Ti SUPER 16GiB, CUDA 13.2, sm_89
- **Feature set:** `cargo build --release -p infer --features cuda`
- **Env vars:** `NVCC_CCBIN=/usr/bin/g++-14`,
  `INFER_TILELANG_PYTHON=/home/ckl/projects/arle/.venv/bin/python`,
  `TORCH_CUDA_ARCH_LIST=8.9`
- **Commit:** pending at entry creation; fill with the landed commit in the
  after-bench snapshot.

## Results

Pending owner-run GuideLLM and nsys results.

## Delta vs Baseline

- **Baseline:** ARLE F4-Small longctx 8k/c=4 TTFT p50 `4961 ms`; vLLM control
  TTFT p50 `2367 ms`; ARLE decode ITL p50 `23.9 ms` versus vLLM `26.7 ms`.
- **Target:** reduce ARLE TTFT by at least `800 ms` and remove prefill-path
  `cuStreamSynchronize` occurrences from the nsys trace.
- **After:** pending owner-run snapshot.

## Rule

Async prefill completion must keep the whole chunk-owned scratch set alive
until a compute-stream event says the chunk is complete. Do not advance
chunk progress, launch the next chunk, or release the owner rows while that
event is still pending.
