# OpenChrome v1.12.2

Release date: 2026-05-15

## Summary

v1.12.2 is the benchmark-readiness release for the Competitive Benchmark
Suite. It takes the v1.12.1 benchmark foundation and completes the next
layer of measurement infrastructure: expanded competitor adapters, larger
token-efficiency and WebVoyager corpora, per-axis runners, cross-platform
bridge smoke coverage, unified report generation, and the Epic #1254 close
gate that distinguishes mock-only evidence from measured benchmark
envelopes.

```
v1.12.1  →  v1.12.2   (this release)

  233 files changed
  +15,701 / -675 lines
  20 benchmark follow-up PR commits integrated (#1277–#1297)
  2  additional benchmark CI workflows
  43 WebVoyager tasks added (18 → 61)
  50 token-efficiency fixture pages added (5 categories × 10 cases)
  3  additional competitor adapters/bridges expanded or added
  6  benchmark report sections wired into the unified report pipeline
  0  new mandatory runtime dependencies
```

This remains an additive release. The runtime MCP contract is unchanged:
no existing tool is removed, renamed, or given a breaking input/output
shape. The new code is concentrated in `tests/benchmark/`, `benchmark/`,
`scripts/bench/`, and benchmark GitHub Actions workflows. Benchmark live
runs that need external credentials or browser-driver setup are still
opt-in; default CI continues to exercise deterministic and mock-safe paths.

---

## Upgrade

```bash
npm install -g openchrome-mcp@1.12.2
openchrome check
openchrome doctor
```

Fresh install:

```bash
npm install -g openchrome-mcp@latest
openchrome check
openchrome info
```

No configuration changes are required for existing v1.12.x users. If you
only consume the MCP server or CLI, this release should be a drop-in patch
upgrade. If you run the benchmark suite, review the new runners and report
sections below because several benchmark axes now expose richer measured
or mock-safe envelopes.

---

## Breaking changes

**None.** v1.12.2 does not introduce any source-level breaking changes for
MCP clients, CLI users, or existing benchmark consumers.

Observable notes:

- Benchmark report wording is stricter about mock-only, opt-in, and live
  evidence so release claims do not imply completed live measurements when
  only deterministic envelopes are available.
- `parallel-isolated.mjs` no longer presents the retired 15.3× compression
  estimate as a benchmark result. Historical estimated claims were removed
  in favor of explicit measured sections.
- New benchmark workflows and scripts may require optional local tools
  when you deliberately run live competitor lanes; default project install
  behavior is unchanged.

---

## Benchmark suite improvements

### Harness and competitor coverage

The harness now has broader competitor representation and bridge coverage:

- Added a Playwright MCP competitor adapter.
- Added a Crawlee competitor adapter.
- Added a Browser Use Python bridge adapter and a cross-platform bridge
  smoke workflow.
- Kept competitor metadata centralized so benchmark reports can attribute
  results by library, execution mode, and evidence type.

These additions make the suite ready to compare OpenChrome against
browser automation libraries without forcing every CI lane to run expensive
or credentialed live tasks.

### Token efficiency benchmark (#1256)

The token-efficiency axis was expanded from a small seed set into a larger
fixture corpus and competitor matrix:

- Added a 5×10 deterministic fixture corpus covering docs, ecommerce,
  news, search-results, and SPA-style pages.
- Added per-page ground-truth JSON fixtures.
- Added a competitor matrix runner for token-efficiency comparisons.
- Added a tokens-versus-retention scatter chart.
- Wired the token-efficiency results into the unified benchmark report as
  section **A**.

The suite is now suitable for repeatable local/CI measurement of extraction
retention versus token cost, while still separating deterministic fixture
coverage from any optional live extraction cells.

### Agent task success / WebVoyager benchmark (#1257)

The WebVoyager-style agent task axis now has a much larger corpus and more
explicit execution-mode modeling:

- Expanded the task corpus from 18 tasks to 61 tasks.
- Added native/passive execution-mode support.
- Added N-gate helper logic for dry-run and regression gating.
- Added a library matrix runner to compare benchmark libraries by mode.
- Added report section **B** and CI regression coverage for agent-success
  evidence.

The release does **not** claim that every real-LLM/live baseline has been
completed. Instead, the benchmark surface is now ready to run and record
those live baselines in a controlled, opt-in way.

### Speed and throughput benchmark (#1258)

The speed/throughput axis now has first-class runner and report support:

- Added a throughput runner with `--skip-live` support for CI-safe paths.
- Retired the old estimation path so generated reports prefer measured or
  explicitly labeled evidence.
- Added speed and throughput charts.
- Wired section **C** into the benchmark report.

This axis is now one of the most release-ready parts of the benchmark suite:
it can produce deterministic report artifacts while preserving optional
live lanes for fuller comparisons.

### Reliability and fault recovery benchmark (#1259)

Reliability coverage now includes both matrix-style recovery measurement
and long-run stability sampling:

- Added a reliability matrix runner.
- Added a long-run stability sampler.
- Added JSON result artifacts for reliability and long-run stability.
- Added cross-platform bridge-smoke coverage for Browser Use integration.

The generated artifacts make reliability work auditable, but long-running
live cells remain intentionally opt-in so normal CI is stable and cheap.

### Auth and real-world usability benchmark (#1260)

The auth/usability axis now has a runner and setup-cost script set:

- Added auth setup scripts for OpenChrome, Playwright, Puppeteer, and
  Browser Use.
- Added LOC/setup-cost measurement helpers.
- Added an auth runner and `auth-usability.json` result artifact.

This prepares the suite to measure logged-in workflow setup friction. Live
logged-in smoke coverage still depends on the runner environment and is not
silently enabled by default.

### Developer experience benchmark (#1261)

Developer-experience reporting now has dual chart support and a benchmark
report section:

- Added DX scripts for common navigation/form-fill tasks across OpenChrome,
  Playwright, and Puppeteer.
- Added schema-completeness and actionability rubrics.
- Added DX chart generation for framework and MCP-oriented views.
- Wired DX evidence into the unified report as section **F**.

The current matrix is ready for measurement and review, while the report
continues to identify which task/library cells are measured versus queued.

### Unified benchmark reporting and Epic #1254 close gate

The unified report pipeline now assembles the per-axis outputs into
`benchmark/results/BENCHMARK-REPORT.md`. The Epic status document also
records a stricter close gate: Epic #1254 should close only after each
benchmark axis has produced a measured, non-mock-only envelope or an
explicitly documented live-run exception.

That distinction matters for release quality. v1.12.2 makes the benchmark
suite runnable and auditable; it does not overstate pending live evidence as
completed production benchmark results.

---

## CI and automation

New or expanded benchmark automation includes:

- `.github/workflows/benchmark-agent.yml`
- `.github/workflows/benchmark-browser-use.yml`
- cross-platform Browser Use bridge smoke checks
- benchmark report generation scripts under `benchmark/`
- helper scripts under `scripts/bench/`

The latest `develop` checks used for this release were green for both the
main CI and benchmark workflows before the release branch was prepared.

---

## Runtime and dependency impact

- No new mandatory runtime dependency is introduced.
- Benchmark-only and developer-only dependencies remain isolated to test and
  measurement workflows.
- No MCP tool was removed or renamed.
- The default server/CLI upgrade path is unchanged from v1.12.1.

---

## Notable commits and PRs

- `#1277` — remove retired benchmark compression estimate.
- `#1278` — add Playwright MCP competitor adapter.
- `#1279` — add Crawlee competitor adapter.
- `#1280` — add Browser Use Python bridge adapter.
- `#1281` — expand token-efficiency corpus to 5×10 fixtures.
- `#1282` — add throughput runner with `--skip-live` mode.
- `#1283` — add token-efficiency competitor matrix runner.
- `#1284` — add token-retention scatter chart and report section A.
- `#1285` — retire the parallel isolated estimation path.
- `#1286` — add speed/throughput report section and charts.
- `#1287` — expand WebVoyager task corpus from 18 to 61 tasks.
- `#1288` — add WebVoyager library matrix runner and dry-run gate.
- `#1289` — add WebVoyager native/passive execution modes.
- `#1290` — add agent-success report section and CI regression gate.
- `#1291` — add cross-platform Browser Use bridge smoke CI.
- `#1292` — add reliability matrix runner.
- `#1293` — add long-run stability sampler.
- `#1294` — add auth runner and setup-cost scripts.
- `#1296` — add dual-chart DX renderer and report section F.
- `#1297` — add unified benchmark report and Epic #1254 close gate.

---

## Verification

Validated for release preparation:

- `npm version 1.12.2 --no-git-tag-version` updated `package.json` and
  `package-lock.json` consistently.
- Benchmark scripts are present for token, throughput, reliability, auth,
  DX, WebVoyager mock/real, and CI report generation lanes.
- The release branch was created by merging the latest `develop` benchmark
  work into the current `main` line before tagging.

Recommended post-upgrade smoke checks:

```bash
openchrome check
openchrome doctor
npm run benchmark:ci
```

Use live benchmark commands only in an environment that has the required
browser dependencies and credentials configured. Do not publish live-result
claims from mock-only lanes.
