---
import {
	benchColdStart,
	benchWorkloads,
	sandboxCostPerSec,
	SANDBOX_COLDSTART_PROVIDER,
	SANDBOX_COST_PROVIDER,
	SANDBOX_VCPU_COST_PER_HOUR,
	SANDBOX_GIB_COST_PER_HOUR,
	SANDBOX_MIN_VCPU,
	SANDBOX_MIN_MEMORY_GIB,
	SANDBOX_MIN_MEMORY_MB,
	UTILIZATION,
	HARDWARE,
} from "@/data/bench";

const agent = benchWorkloads.agent;
const shell = benchWorkloads.shell;
---

<h2>Cold start</h2>

<p>Time from requesting an execution to first code running. Measured using the sleep workload (minimal shell command). Sandbox baseline: <strong>{SANDBOX_COLDSTART_PROVIDER}</strong>, the fastest mainstream sandbox provider as of March 30, 2026. See <a href="https://www.computesdk.com/benchmarks/">ComputeSDK benchmarks</a> for independent sandbox comparison data.</p>

<table>
<thead><tr><th>Metric</th><th style="text-align:right">agentOS</th><th style="text-align:right">Fastest sandbox ({SANDBOX_COLDSTART_PROVIDER})</th></tr></thead>
<tbody>
{benchColdStart.map(row => (
<tr><td>Cold start {row.label}</td><td style="text-align:right">{row.agentOS} ms</td><td style="text-align:right">{row.sandbox.toLocaleString()} ms</td></tr>
))}
</tbody>
</table>

<h2>Memory per instance</h2>

<p>Measured via staircase benchmarking:</p>

<ol>
<li><strong>Warmup.</strong> A throwaway VM is created, started, and destroyed before measurement begins. This pays one-time costs (module cache, JIT compilation) that are amortized away in any real deployment where the host process is long-lived.</li>
<li><strong>Baseline.</strong> GC is forced twice (<code>--expose-gc</code>), then RSS is sampled across the entire process tree by reading <code>/proc/[pid]/statm</code> for the host process and all descendants. This captures child processes (e.g. V8 isolates running as separate processes) that <code>process.memoryUsage().rss</code> would miss.</li>
<li><strong>Staircase.</strong> VMs are added one at a time. After each VM starts and settles, GC is forced and RSS is sampled again. The delta from the previous sample is the incremental cost of that VM.</li>
<li><strong>Average.</strong> The per-VM cost is the mean of all step deltas.</li>
<li><strong>Teardown.</strong> All VMs are disposed and the reclaimed RSS is recorded.</li>
</ol>

<p>RSS is a process-wide metric that includes thread stacks and OS-mapped pages beyond the VM itself, so the reported figure is an upper bound on the true per-VM cost.</p>

<p>Sandbox baseline: <strong>{SANDBOX_COST_PROVIDER}</strong>, the cheapest mainstream sandbox provider as of March 30, 2026. Default sandbox: {SANDBOX_MIN_VCPU} vCPU + {SANDBOX_MIN_MEMORY_GIB} GiB RAM.</p>

<h3>Full coding agent</h3>
<p>{agent.description}.</p>
<table>
<thead><tr><th>Metric</th><th style="text-align:right">agentOS</th><th style="text-align:right">Cheapest sandbox ({SANDBOX_COST_PROVIDER})</th></tr></thead>
<tbody>
<tr><td>Memory per instance</td><td style="text-align:right">{agent.memory.agentOS}</td><td style="text-align:right">{agent.memory.sandbox}</td></tr>
</tbody>
</table>

<h3>Simple shell command</h3>
<p>{shell.description}.</p>
<table>
<thead><tr><th>Metric</th><th style="text-align:right">agentOS</th><th style="text-align:right">Cheapest sandbox ({SANDBOX_COST_PROVIDER})</th></tr></thead>
<tbody>
<tr><td>Memory per instance</td><td style="text-align:right">{shell.memory.agentOS}</td><td style="text-align:right">{shell.memory.sandbox}</td></tr>
</tbody>
</table>

<h2>Cost per execution-second</h2>

<p>Assumes one agent per sandbox (needed for isolation) and {Math.round(UTILIZATION * 100)}% host utilization for self-hosted hardware (the industry-standard HPA scaling threshold). Cost formula: <code>server cost per second / concurrent executions per server</code>, where concurrent executions = <code>floor(server RAM / agent memory) &times; {UTILIZATION}</code>.</p>

<p>Sandbox baseline: <strong>{SANDBOX_COST_PROVIDER}</strong> at ${SANDBOX_VCPU_COST_PER_HOUR}/vCPU-h + ${SANDBOX_GIB_COST_PER_HOUR}/GiB-h with a {SANDBOX_MIN_VCPU} vCPU + {SANDBOX_MIN_MEMORY_GIB} GiB minimum. Source: <a href="https://www.daytona.io/pricing">daytona.io/pricing</a>.</p>

<h3>Full coding agent</h3>
<table>
<thead><tr><th>Host tier</th><th style="text-align:right">agentOS</th><th style="text-align:right">Cheapest sandbox</th><th style="text-align:right">Difference</th></tr></thead>
<tbody>
{agent.cost.map(tier => (
<tr><td>{tier.label}</td><td style="text-align:right">{tier.value}</td><td style="text-align:right">{agent.sandboxCost}</td><td style="text-align:right">{tier.multiplier}</td></tr>
))}
</tbody>
</table>

<h3>Simple shell command</h3>
<table>
<thead><tr><th>Host tier</th><th style="text-align:right">agentOS</th><th style="text-align:right">Cheapest sandbox</th><th style="text-align:right">Difference</th></tr></thead>
<tbody>
{shell.cost.map(tier => (
<tr><td>{tier.label}</td><td style="text-align:right">{tier.value}</td><td style="text-align:right">{shell.sandboxCost}</td><td style="text-align:right">{tier.multiplier}</td></tr>
))}
</tbody>
</table>

<h2>Test environment</h2>

<table>
<thead><tr><th>Component</th><th>Details</th></tr></thead>
<tbody>
<tr><td>CPU</td><td>12th Gen Intel i7-12700KF, 12 cores / 20 threads @ 3.7 GHz, 25 MB cache</td></tr>
<tr><td>RAM</td><td>2&times; 32 GB DDR4 @ 2400 MT/s</td></tr>
<tr><td>Node.js</td><td>v24.13.0</td></tr>
<tr><td>OS</td><td>Linux 6.1.0 (Debian), x86_64</td></tr>
</tbody>
</table>

<h2>Sandbox baselines</h2>

<table>
<thead><tr><th>Comparison</th><th>Provider</th><th>Why this provider</th></tr></thead>
<tbody>
<tr><td>Cold start</td><td>{SANDBOX_COLDSTART_PROVIDER}</td><td>Fastest mainstream sandbox provider on <a href="https://www.computesdk.com/benchmarks/">ComputeSDK</a> as of March 30, 2026</td></tr>
<tr><td>Memory and cost</td><td>{SANDBOX_COST_PROVIDER}</td><td>Cheapest mainstream sandbox provider as of March 30, 2026 (${SANDBOX_VCPU_COST_PER_HOUR}/vCPU-h + ${SANDBOX_GIB_COST_PER_HOUR}/GiB-h)</td></tr>
</tbody>
</table>

<p>Self-hosted hardware tiers: AWS t4g.micro (ARM, $0.0084/h, 1 GiB), AWS t3.micro (x86, $0.0104/h, 1 GiB), Hetzner CAX11 (ARM, &euro;3.29/mo, 4 GiB), Hetzner CX22 (x86, &euro;5.39/mo, 4 GiB). All on-demand pricing.</p>

<h2>Reproducing</h2>

<p>agentOS benchmarks live in the <a href="https://github.com/rivet-dev/agent-os">agent-os repository</a> under <code>scripts/benchmarks/</code>.</p>

<pre><code>{`# Run memory benchmarks
npx tsx --expose-gc scripts/benchmarks/memory.bench.ts --workload=sleep
npx tsx --expose-gc scripts/benchmarks/memory.bench.ts --workload=pi-session

# Results are saved to scripts/benchmarks/results/`}</code></pre>

<p>Results will vary by hardware. The numbers above are from the test environment described above.</p>
