# Epic #4061 Benchmark Runbook

`nexus-bench` is the measurement harness for #4061 performance claims. The deterministic `noop` target is only a smoke test for trace parsing, reporting, diffing, and CI plumbing. Performance claims require `mount` or `http` suite artifacts from a real Nexus filesystem or server.

## Required Artifact Set

Every #4061 performance PR should attach or commit:

- `suite.json`
- `suite.md`
- all `runs/<workload>/run-NNN.json` files
- all `runs/<workload>/run-NNN.md` files
- the command line used to generate the suite
- runner details: OS, CPU, disk type, kernel version, Nexus commit, target type

Use at least one warmup and five measured iterations for a PR claim:

```bash
cargo run --manifest-path nexus-bench/Cargo.toml -- suite \
  --target http \
  --base-url http://localhost:2026 \
  --api-key "$NEXUS_API_KEY" \
  --trace-dir nexus-bench/traces \
  --out-dir target/nexus-bench/real-http \
  --warmups 1 \
  --iterations 5
```

For a mounted filesystem:

```bash
cargo run --manifest-path nexus-bench/Cargo.toml -- suite \
  --target mount \
  --mount-root /tmp/nexus-mount \
  --trace-dir nexus-bench/traces \
  --out-dir target/nexus-bench/real-mount \
  --warmups 1 \
  --iterations 5
```

## Baseline And Candidate Comparison

Run the suite on the baseline commit and the candidate commit with the same runner, target, seeded data, warmup count, and iteration count. Compare the relevant workload rows in `suite.md`, and use the per-run JSON files for deeper analysis.

The committed `noop` baseline at `nexus-bench/baselines/develop-5698d0026-noop.json` is not a performance baseline. It only proves that the harness can replay, report, and diff.

## Issue Matrix

The machine-readable matrix lives at `nexus-bench/matrices/epic-4061.json`.

| Issue | Required Workloads | Primary Metrics |
|---|---|---|
| #4053 foyer cache | `agent-cold-start`, `agent-warm-trace` | `cache_hit_rate`, p99/p95 latency |
| #4055 eager hydrate | `agent-cold-start`, `agent-warm-trace` | p50/p95 latency, `cache_hit_rate` |
| #4056 async HTTP | `concurrent-reads`, `agent-warm-trace` | throughput, p99 latency |
| #4057 prefetch | `seq-large-read`, `stride-read`, `agent-warm-trace` | throughput, egress, prefetch bytes |
| #4058 read batch | `metadata-storm`, `agent-cold-start` | ops/sec, p95 latency, read batch size |
| #4059 write coalescing | `bursty-write`, `read-after-write` | RPC count, coalescing flush/backend RPC metrics |
| #4060 passthrough | `seq-large-read` | throughput, p99 latency, passthrough count |

Metrics named `nexus_*` come from #4062 observability. Until #4062 lands, `suite.json` still captures the generic harness metrics, but claims that depend on cache, prefetch, read-batch, write-coalescing, or passthrough counters remain incomplete.

## Manual Workflow

Use `.github/workflows/nexus-bench-real.yml` for provisioned real-target runs. It is `workflow_dispatch` only because the runner must provide either:

- an already-mounted filesystem for `target=mount`, or
- a live Nexus HTTP endpoint plus `NEXUS_BENCH_API_KEY` secret for `target=http`.

The regular `.github/workflows/nexus-bench.yml` workflow remains the smoke gate and should not be interpreted as real performance evidence.
