# roboharness Ecosystem Research — 2026-05 Checkpoint

> A structured archive of `roboharness`'s investigation into the surrounding agent-harness, robot-simulator, multimodal-model, VLA, humanoid-hardware, and agent-tool-protocol stacks.
>
> **This checkpoint:** May 21, 2026
> **Next scheduled update:** end of June 2026 (monthly cadence, last week of month)
> **Maintenance mode:** monthly deep research + human review (see §0.2)
> **Type:** **first checkpoint — establishes baseline**

---

## 0. About this document

### 0.1 Why this checkpoint exists

`roboharness` is an approval/evidence harness for unattended robot-code changes — it compresses a long agent run into a proof pack a human can review quickly. The repo currently lives at v0.3, focused on the MuJoCo contract-first trust loop, with a maintained MuJoCo grasp wedge and a Unitree G1 + GR00T/SONIC humanoid path.

Every selection decision in this stack ages fast. The upstream "harness engineering" line was *named* in early 2026 and already has three rounds of refinement; Anthropic released Opus 4.7 with a 3× vision resolution upgrade on April 16, 2026; NVIDIA shipped Newton 1.0 at GTC 2026, refactoring the simulator-substrate landscape; OpenAI open-sourced Symphony on April 27, 2026; GR00T N1.7 went to early-access with commercial licensing on April 17, 2026. Any one of these would have triggered an off-schedule research pass; together they are the reason this baseline was overdue before the vectors card was even written.

Two jobs for this checkpoint:

1. **Freeze the current understanding** — what is true in mid-May 2026 about each of the 6 research vectors defined in `mithaq/vectors/roboharness.md`, and what is the state of each of the 7 hidden assumptions H1..H7.
2. **Establish a sustainable update mechanism** — by anchoring everything in the vectors card so subsequent checkpoints can note changes against this one rather than re-survey the field.

### 0.2 Update method

Each update follows this loop:

**Step 1 — incremental research.** Run a deep-research pass covering the 6 fixed vectors in [`mithaq/vectors/roboharness.md`](https://github.com/MiaoDX/mithaq/blob/main/vectors/roboharness.md):

1. Harness engineering upstream
2. Robot simulator substrate ecosystem
3. Agent visual review capabilities
4. VLA / robot-policy model ecosystem
5. Humanoid hardware + WBC stack
6. MCP / agent tool protocols

**Step 2 — incremental update.** Compare against this baseline. Produce three categories:

- **New** projects/concepts → add with `(new in YYYY-MM)`
- **Status changes** → annotate the original entry with `[updated YYYY-MM]`
- **Judgment revision** → keep prior recommendation visible, add `[revised YYYY-MM]` block explaining the change

**Step 3 — write the new checkpoint** at `docs/research-checkpoints/YYYY-MM.md` following this template.

**Step 4 — review.** Watch for: over-reliance on D-tier SEO content, slack ("nothing changed this month"), §6 recommendations disconnected from §3 evidence.

**Triggers for off-schedule passes** (full list in vectors card): flagship model release (Opus 5 / GPT-6); major VLA release (GR00T N2, π₁); substrate event (Newton 1.x, Isaac Lab 3.0 GA, MolmoSpaces multi-agent); key-dependency license/maintenance event; major harness-engineering release from Anthropic/OpenAI/Cursor.

### 0.3 Source-quality declaration

Four tiers; full definitions in template §0.3. This checkpoint draws primarily from A-tier (official Anthropic/OpenAI/NVIDIA/Cursor/Hugging Face blogs and official project repos) and arXiv. B-tier press (TechCrunch, MarkTechPost, helpnetsecurity, The Robot Report) is used when it adds context that primary sources omit; numeric claims from B-tier are cross-checked against the originating release. No D-tier "best X of 2026" sources are cited.

Repo-specific whitelist/blacklist: see `mithaq/vectors/roboharness.md` §Sources.

### 0.4 How to read this

- **Quick conclusions only**: §1 + §6
- **Ecosystem snapshot**: §3
- **Selection decisions**: §6 + §7
- **Prepare next checkpoint**: §0.2 + §8 + §7

---

## 1. Executive summary

**5 ecosystem-level judgments (baseline):**

1. **Harness engineering is a named discipline with three converging reference designs.** OpenAI Symphony (released April 27, 2026, 15K+ stars), Cursor's recursive Planner/Worker (Feb 2026), and Anthropic's CWC long-running agents primitives (open-sourced from Code with Claude 2026, based on the March 2026 harness-design paper) now define three distinct scaling axes — interaction, space, and time, respectively. Birgitta Böckeler's April 2026 article on martinfowler.com supersedes the earlier Martin Fowler memo and consolidates the user-side mental model. The term "harness engineering" itself is now attributed to Viv Trivedy (LangChain), not OpenAI.

2. **The MuJoCo / Isaac Lab divide is dissolving at the substrate layer.** NVIDIA Newton 1.0 (GTC 2026) integrates MuJoCo-Warp as a backend; Isaac Lab 3.0-beta ships an `isaaclab_newton` extension that runs Isaac Lab environments *without Isaac Sim* via MuJoCo-Warp; `mjlab` (Kevin Zakka et al., arXiv 2601.22074) exposes the Isaac Lab manager API directly on MuJoCo-Warp. Anyone choosing "MuJoCo vs Isaac Lab" in 2026 is increasingly choosing between API surfaces over the same physics core. **This strengthens H2 substantially.**

3. **Claude Opus 4.7 raised the same-agent visual review ceiling.** Released April 16, 2026: 3× vision resolution (up to 2,576 px / 3.75 MP, vs. 1,568 px / 1.15 MP previously), MMMU +13%, MathVista 69.8% → 79.3%, OSWorld-Verified 72.7% → 78.0%. One external tester reported visual-acuity benchmark 54.5% → 98.5%. Cognition reported it can work "for hours" through tool failures. **This supports H1**, but a counter-signal exists: MV-RoboBench (arXiv:2510.19400) finds SOTA VLMs still well below human performance on multi-view spatial reasoning in robotic scenes — which means roboharness's existing "view conflict → NEEDS_HUMAN" escalation policy is *empirically necessary*, not over-cautious.

4. **The humanoid demo stack (G1 + GR00T + SONIC) hardened, not shifted.** GR00T N1.7 (April 17, 2026) is now commercially licensed, with native UNITREE_G1_SONIC embodiment for whole-body manipulation+locomotion. Unitree G1 is officially supported in LeRobot (29 & 23 DoF). GR00T N2 was previewed at GTC March 2026 (DreamZero, "world action model"), expected end-2026. Figure 03 and Atlas remain closed: Atlas's full 2026 production is committed to Hyundai and Google DeepMind; no open SDK from Figure or Boston Dynamics. **H5 holds.**

5. **The agent-tool surface is consolidating around two open standards both relevant to roboharness.** Agent Skills (SKILL.md) became an open standard at agentskills.io around 2026-03-14, with 32 adopters by then; MCP gained robotics-specific servers — `isaacsim-mcp-server` (PyPI, April 6, 2026, 41 tools), `ros-mcp-server` (mature, Claude/Codex/Gemini compatible), and a `mujoco-mcp` community project. roboharness already plans SKILL.md as a distribution channel (see article 公众号文章001); the MCP question is whether to ship a roboharness MCP server vs. integrate with existing ones.

**Specific recommendations for `roboharness` (full list in §6):**

1. Run a controlled experiment: does Opus 4.7's 3.75 MP vision actually reduce roboharness's NEEDS_HUMAN rate on the maintained MuJoCo grasp wedge? Pair-tested at both 1.15 MP and 3.75 MP. Result feeds H1 status next checkpoint.
2. Add `mjlab` and Newton to the simulator wrapper compatibility roadmap as low-priority probes — not because we'll switch, but because the Isaac Lab manager API may be the highest-leverage entry point for Isaac Lab users to adopt roboharness without leaving their stack.
3. Keep the v0.3 milestone constraints intact (no new simulator backends, no SONIC scope expansion without real-model validation evidence). The ecosystem is moving but the *project's* discipline is what justified the recent test suite pass at 91% coverage.
4. Add the Anthropic CWC long-running-agents primitives repo (`anthropics/cwc-long-running-agents`) to design-review reading. The hooks + evaluator subagent pattern is closely analogous to roboharness's Two-Step Visual Review.
5. Defer the "ship a roboharness MCP server" decision to next checkpoint; first observe how isaacsim-mcp-server matures.

**Still uncertain, requires next checkpoint to verify:**

- Whether Opus 4.7's higher-resolution vision moves the needle on roboharness-specific NEEDS_HUMAN rate (experiment Q1 in §7)
- Whether `mjlab` and Newton converge or stay parallel (architectural Q2)
- π₁ release status (currently unconfirmed; openpi has π₀, π₀-FAST, π₀.₅ only as of May 2026)
- Whether Figure or Boston Dynamics will release any open SDK in 2026

---

## 2. Background: roboharness current state and constraints

**Project positioning:** Approval/evidence harness for unattended robot-code changes. Core wedge: *long unattended agent run → one proof pack → short human review*. Does not aim to be a general-purpose VLM judge service, a robot simulator, or a robot policy framework — those are upstream/sibling concerns.

**Current stack (May 2026, per `STATUS.md` 2026-05-20):**

- Checkpoint-oriented core harness with semantic task protocols
- Native MuJoCo + Meshcat backend for the maintained grasp wedge
- Gymnasium wrappers for zero-change integration
- LeRobot + Unitree G1 demo/evaluation paths (including SONIC planner and tracking)
- Metric evaluation, batch evaluation, trend history, approval evidence
- Bounded agent visual review: manifest preparation, record validation, summary aggregation
- Python-authored project harness contracts → drift-checked agent-skill artifacts under `agent-skill/<project-slug>-harness/`
- CLI for inspecting outputs, writing `report.json`, evaluating reports, tracking trends
- Optional MCP tools for checkpoint capture and evaluation

**Roadmap (per STATUS.md):**

- **v0.3** (current, in progress): MuJoCo contract-first trust loop. Keep evaluator corpus grounded; keep invalid contracts fail-closed; ambiguous evidence never self-promotes to pass; explicit human blessing for migration-mode baselines.
- **Phase next:** to be defined; constraints during v0.3 explicitly forbid adding new simulator backends or expanding SONIC scope without real-model validation evidence.

**Known technical constraints:**

- Primary language: Python (Python-Authored Contract is the source of truth; YAML/JSON contract files are normalized outputs only)
- License: Apache-2.0 (matches LeRobot, openpi, MuJoCo, GR00T)
- Latest local validation 2026-05-20: pytest 533 passed, 13 skipped, coverage 91.16%; ruff + mypy clean
- Headless rendering requires `MUJOCO_GL=osmesa` or `egl`
- Isaac Lab end-to-end validation requires NVIDIA RTX hardware; CPU test suite uses mocks

These constraints drive every judgment in §6 — in particular, the v0.3 "don't expand scope" discipline is the reason §6's recommendations are restrained even though the ecosystem moved significantly this quarter.

---

## 3. Ecosystem panorama

### 3.1 Vector 1 — Harness engineering upstream

Three reference designs now visible, each solving a different scaling axis:

| Project | Released | Scaling axis | Relevance to roboharness |
|---------|----------|---------------|---------------------------|
| Anthropic CWC long-running-agents | March 2026 paper + Code with Claude 2026 repo (`anthropics/cwc-long-running-agents`) | **Time** (one agent runs longer without drifting) | Generator/Evaluator separation is the architectural ancestor of roboharness's Two-Step Visual Review; the `/goal` command's evaluator-subagent pattern is directly applicable |
| Cursor recursive Planner-Worker (`cursor.com/blog/scaling-agents`, `cursor.com/blog/self-driving-codebases`) | Jan–Feb 2026 | **Space** (hundreds of agents in parallel) | Less directly relevant — roboharness is not orchestrating hundreds of robot-code agents — but the "judge agent at end of cycle" pattern reinforces independent-evaluator discipline |
| OpenAI Symphony (`github.com/openai/symphony`) | April 27, 2026 (15K+ GitHub stars by April 23 per OpenAI blog) | **Interaction** (humans don't supervise individual runs; they manage work) | Symphony's "Proof of Work" output (CI status + walkthrough + sometimes video) is conceptually identical to roboharness's Proof Pack. roboharness can position itself as "the Proof Pack standard for robotics workloads." |

Second-tier ecosystem developments worth noting:

- **HumanLayer's framing** ("Skill Issue: Harness Engineering for Coding Agents", March 2026): harness engineering ⊂ context engineering ⊂ improving agent reliability. Their key insight: sub-agents are useful as *context firewalls*, not as role-specialized "frontend engineer" / "backend engineer" personas. Relevant for how roboharness invokes its Visual Reviewer.
- **Birgitta Böckeler** (Thoughtworks, on martinfowler.com, April 2, 2026): superseded the earlier Martin Fowler harness-engineering memo. Frames harness as feedforward guides + feedback sensors that self-correct before output reaches human eyes. Matches roboharness's Hard Metric Gate / Visual Review Dimension split.
- **LangChain** (Viv Trivedy, who coined "harness engineering"): Terminal Bench 2.0 result of moving a coding agent from Top 30 → Top 5 by changing only the harness — direct evidence that harness >> model for reliability gains.
- **Addy Osmani** synthesis (April 19, 2026, `addyosmani.com/blog/agent-harness-engineering/`) consolidates the field; good entry-point reading for newcomers.

Pattern that crystallized this period: **"Agent = Model + Harness"** is no longer disputed and the term originator (Viv Trivedy) is now consistently credited. The vectors card should be updated to credit Viv Trivedy as the term's originator.

### 3.2 Vector 2 — Robot simulator substrate ecosystem

The big structural shift this period: **MuJoCo-Warp became the de facto GPU-physics core, sitting under multiple API surfaces.**

| Project | Maintainer | State | Relevance to roboharness |
|---------|-----------|-------|--------------------------|
| MuJoCo | google-deepmind | Stable. Wikipedia lists 3.2.7 from Jan 2025 but the GitHub repo continues active development | Direct dependency. roboharness's MuJoCo wedge is canonical |
| MuJoCo Warp (`google-deepmind/mujoco_warp`) | google-deepmind | Apache-2.0, active. GPU-accelerated MuJoCo on NVIDIA Warp; supports same features as MuJoCo (no differentiability yet, tracked in issue #500); includes a batch ray-tracing renderer for parallel worlds | High-throughput Warp backend can replace the headless osmesa/egl path for batch evaluation runs |
| NVIDIA Newton 1.0 (`newton-physics/newton`) | NVIDIA + Google DeepMind + Disney Research, Linux Foundation–managed | Released at GTC 2026 (~March). Multi-solver: MuJoCo-Warp + Kamino + Vertex Block Descent (deformables) + SDF collision + hydroelastic contact | Major. Newton subsumes MuJoCo-Warp as one of its rigid-body solvers; affects whether "MuJoCo" is still the unit of substrate selection |
| Isaac Lab 3.0-beta (`isaac-sim/IsaacLab`) | NVIDIA | Released April 2026. Factory-based multi-backend: `isaaclab_physx` (default), `isaaclab_newton` (powered by MuJoCo-Warp). New "kit-less" mode runs Isaac Lab API without Isaac Sim | Significant. The kit-less mode means Isaac Lab users no longer need an RTX-locked path to run a subset of environments — Isaac Lab compatibility for roboharness becomes more meaningful |
| mjlab (`mujocolab/mjlab`) | Kevin Zakka, Qiayuan Liao, Brent Yi, Louis Le Lay, Koushil Sreenath, Pieter Abbeel — arXiv:2601.22074 | Apache-2.0, beta, on PyPI. Isaac Lab manager-based API on top of MuJoCo-Warp. Includes example Unitree G1 velocity tracking | Direct interest. mjlab + G1 is roboharness's core demo combination on Isaac Lab API |
| MolmoSpaces (`allenai/molmospaces`) | Allen AI | Released Feb 11, 2026. 230K+ scenes, 130K+ object models, 42M+ grasps. Supports MuJoCo, Isaac, ManiSkill assets; **data generation and benchmarking only supported for MuJoCo** | Reinforces MuJoCo as the primary substrate for grasp evaluation work |
| Genesis (`Genesis-Embodied-AI/genesis-world`) | Genesis Embodied AI | v0.4.7, active. Claims 10–80× faster than MJX. Generative agent layer | Watchlist; the company behind it ("Genesis AI" with GENE-26.5, see Vector 4) raised $105M seed and is going full-stack robotics — substrate split risk |
| ManiSkill3 | HF/PKU | Active | Watchlist; one of the few MolmoSpaces-supported substrates |
| Isaac Sim 6.0 | NVIDIA | Tracking with Isaac Lab 3.0 | Heavyweight, RTX-locked |
| CoppeliaSim 4.10 | Coppelia Robotics | May 2025 | Out of scope for roboharness |

**Pattern: the question "MuJoCo vs Isaac Lab" is being replaced by "which API on top of MuJoCo-Warp."** Newton, isaaclab_newton, mjlab, and MolmoSpaces all converge on MuJoCo-Warp as the common kernel.

### 3.3 Vector 3 — Agent visual review capabilities

The dominant signal: **Claude Opus 4.7 (April 16, 2026) raised the same-agent visual review ceiling sharply.** Concrete deltas vs. Opus 4.6:

- **Vision resolution**: 1,568 px (~1.15 MP) → 2,576 px (~3.75 MP), 3.3×. This is a model-level change, not an API parameter.
- **MMMU** (multimodal college-level reasoning): +13%
- **MathVista** (visual-mathematical reasoning, charts, geometry): 69.8% → 79.3%
- **OSWorld-Verified** (computer use): 72.7% → 78.0%
- **Computer-use visual-acuity benchmark** (one tester): 54.5% → 98.5%
- **Long-horizon coherence** (Cognition/Devin report): "works coherently for hours" and pushes through tool failures that stopped Opus 4.6
- **/ultrareview** command added in Claude Code — a dedicated review session that flags bugs and design issues
- **Regression**: BrowseComp -4.4 points (agentic search regressed)

Other model versions in the multimodal landscape (less central to roboharness): GPT-5.2 (referenced in Cursor's long-running runs as outperforming Opus 4.5 on autonomous coding endurance, per their `scaling-agents` post); Gemini Pro retains lead on multilingual visual tasks per Opus 4.7 benchmark comparisons; Kimi K2.5 referenced in agent-swarm contexts.

**Counter-evidence to H1 ("same-agent visual review is enough, no separate VLM judge needed"):**

- **MV-RoboBench (arXiv:2510.19400)** — "Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes." 1.7K curated multi-view QA items across 8 subtasks. Finding: SOTA VLMs (open- and closed-source, including CoT variants) remain far below human performance. This is **not** counter-evidence to using same-agent review in general — it is direct evidence that multi-view robotic spatial reasoning is a known weakness for all VLMs, including dedicated judges. The implication for roboharness: keeping "view conflict" as a Human Escalation Reason is empirically justified.

- **StepEval (arXiv:2509.19524)** — proposes VLM-based subgoal evaluation framework as community-driven open-source. Frames VLMs as automated judges of subgoal outcomes from recorded images/videos. Notable because it is a design pattern that *competes* with roboharness's bet: a separate, model-agnostic VLM judge layer. However, it is currently a blueprint paper, not a production system, so it does not constitute deployment-level counter-evidence to H1.

### 3.4 Vector 4 — VLA / robot-policy model ecosystem

| Model | Maintainer | State (May 2026) | Relevance to roboharness |
|-------|-----------|------------------|--------------------------|
| GR00T N1.7 | NVIDIA Isaac team | Early Access, **commercially licensed**, April 17, 2026. 3B params. Action Cascade dual-system (Cosmos-Reason2-2B System 2 + 32-layer DiT System 1). Native UNITREE_G1_SONIC embodiment for whole-body control with GEAR-SONIC controller. Trained on 20K+ hours of human egocentric video. "First-ever dexterity scaling law" | **Directly central.** roboharness's G1 humanoid path uses GR00T + SONIC |
| GR00T N1.6 | NVIDIA | Released around CES 2026 (Jan); whole-body control unlock; Cosmos Reason brain | Superseded by N1.7 for new work |
| GR00T N2 (preview only) | NVIDIA | Previewed at GTC March 2026. Based on DreamZero research. New "World Action Model" architecture. Claims 2× success rate over leading VLAs on new tasks in new environments. Tops MolmoSpaces and RoboArena benchmarks. Expected by end of 2026 | Watchlist. World-action-model architecture may require new proof-pack dimensions (e.g., predicted-vs-actual world rollout comparison) |
| π₀, π₀-FAST, π₀.₅ | Physical Intelligence (`Physical-Intelligence/openpi`) | All three available as base checkpoints on openpi (Apache-2.0). π₀.₅ uses "knowledge insulation" for better open-world generalization. Both JAX and PyTorch frameworks supported. LeRobotDataset and RLDS data loading. ALOHA, DROID, LIBERO, UR5 platforms supported | Strong alternative VLA family. roboharness should remain platform-agnostic to allow π₀.₅ proof packs |
| π₁ | Physical Intelligence | **Not confirmed released as of May 2026.** Open issue #789 ("Any plans to release pi 0.6?") from late 2025 was the most recent public signal; no π₁ in current openpi releases | Speculation only. Vectors card listed it under "Current mainstream" — should be revised to "Watchlist" |
| SONIC | NVIDIA GEAR | Integrated with GR00T N1.7 as the whole-body controller decoder (latent action tokens → full-body joint commands) | Direct dependency |
| Helix / Helix 02 | Figure AI | Closed-source. Helix 02 announced as full-body autonomy enabler for Figure 03 (BotQ 12K+ units/year scaling) | Observable from press only; not actionable |
| Open VLA family (OpenVLA-OFT, CogACT, dVLA, etc.) | Various | Active arXiv development | Long-tail; no individual change forces vector revision |
| MEM (Physical Intelligence 2026-03) | PI | 15-minute kitchen tasks | Watchlist; demonstrates long-horizon execution as an emerging review dimension |

**Data format pattern: LeRobotDataset v3.0** rolled into `lerobot >= 0.4.0` (HF blog, August/September 2025 dated post). Multi-episode-per-file Parquet/MP4 packing, native streaming mode via `StreamingLeRobotDataset`. Reduces filesystem pressure at million-episode scale. Conversion utility provided from v2.1 datasets. **H4 strengthened.** NVIDIA has stated integration: "Isaac and GR00T technologies are now built into Hugging Face LeRobot library" (per Robot Report, CES 2026 coverage). Unitree G1 has native LeRobot support (`huggingface.co/docs/lerobot/unitree_g1`), with 29- and 23-DoF variants.

### 3.5 Vector 5 — Humanoid hardware + WBC stack

| Hardware | State (May 2026) | Open SDK? |
|----------|------------------|-----------|
| **Unitree G1 / H1** | 5,500+ humanoids shipped 2025; aggressive 2026 production targets. Native LeRobot integration with `lerobot[unitree_g1]` extras. `unitree_sdk2_python` is open | **Yes** (status quo) |
| **Atlas (Boston Dynamics)** | Production-ready CES 2026, 56 DoF, all 2026 fleet committed to Hyundai RMAC + Google DeepMind. 30K-units/year factory planned 2028. Powered by Google DeepMind embodied AI partnership | **No open SDK** |
| **Figure 02 / 03** | Figure 03 late 2025; Helix 02 for full-body autonomy; BotQ facility scaling to 12K+ units/year; BMW pilot with 30K+ vehicles, 1,250 hours, 90K+ parts handled | **No open SDK** |
| **Tesla Optimus Gen 3** | Production start targeting summer 2026 at Fremont; AI5 chip | **No** |
| **1X NEO** | Active | **No** (status quo) |
| **AGIBOT G2 / 智元** | Active in China market | Selective |
| **Apptronik Apollo, Sanctuary Phoenix, NEURA Robotics** | All active in NVIDIA Isaac partnership orbit | Various; mostly closed |
| **Reachy 2 (Pollen Robotics, now HF)** | Will run on NVIDIA Jetson Thor; integrated with LeRobot | **Yes** |
| **WBC stack** | Pinocchio active (`>=3.0.0`, with CasADi bindings); Pink IK active; MuJoCo MPC available; generative WBC papers (GMP, Trajectory Diffusion for WBC) continue | n/a |

**No open SDK shipped from Figure or Boston Dynamics in 2025 or so far in 2026.** The vectors card's question stays open. The "Chinese humanoid" line (Unitree, AGIBOT/智元) continues to dominate the open ecosystem.

### 3.6 Vector 6 — MCP / agent tool protocols (including robotics MCP servers)

**Two open standards now matter for roboharness:**

| Standard | State | Relevance to roboharness |
|----------|-------|--------------------------|
| **MCP** (Anthropic → Agentic AI Foundation/Linux Foundation, Dec 2025) | AAIF formed Dec 9, 2025 with Anthropic + OpenAI + Block as founders; 146 member orgs by Feb 2026 | roboharness already exposes optional MCP tools; the question is breadth of capability |
| **Agent Skills (SKILL.md)** at `agentskills.io` | Open standard since Dec 18, 2025; agentskills.io launched ~March 14, 2026 with 32 adopters (Google Gemini CLI, JetBrains Junie, AWS Kiro, Block Goose, Snowflake, Databricks, ByteDance, Mistral, Spring AI, Sourcegraph Amp, etc.); `anthropics/skills` repo ~75K stars | roboharness already generates SKILL.md from Python contracts (deterministic skill compilation). Cross-vendor SKILL.md adoption means a roboharness-generated skill works in Codex CLI, Gemini CLI, Cursor without modification |

**Robotics-specific MCP servers (status):**

- **`isaacsim-mcp-server`** (PyPI, released April 6, 2026, MIT, beta): 41 tools for scene management, robot control, sensors, simulation. Maintained by `omni-mcp` + `whats2000`. Requires `isaac.sim.mcp_extension` config.
- **`ros-mcp-server`** (`robotmcp/ros-mcp-server`, Apache-2.0): Mature. Supports ROS1 and ROS2 via rosbridge. Works with Claude Code, Codex CLI, Gemini CLI, Claude Desktop, ChatGPT, Cursor. Demonstrated controlling Unitree Go2 with natural language and debugging industrial robots.
- **`mujoco-mcp`** (community, e.g., `robotlearning123/mujoco-mcp` on LobeHub): Exists. Supports loading MuJoCo Menagerie robots, multi-robot coordination, walking robots (Unitree Go2 quadruped).
- **LeRobot official MCP server**: not yet announced as of May 2026.

The vectors card asked: "whether LeRobot ships an official MCP server" — still unanswered.

### 3.7 Stable patterns emerging this period

- **Proof-of-Work as the human-facing artifact.** Symphony's CI + walkthrough (sometimes video) output, Cursor's Cloud Agents' "merge-ready PR with artifacts proving the changes work," and Anthropic CWC's evaluator subagent output all converge on the same idea: the unit of human review is no longer code-diff-and-discussion but a bounded evidence package. This is **exactly** roboharness's Proof Pack — and the pattern's spread upstream is a strong tailwind for roboharness's framing.
- **Independent evaluator is the default.** Both Anthropic (CWC generator/evaluator split, no shared state) and Cursor (judge agent at end of cycle) treat evaluator independence as architectural, not optional. This validates roboharness's Two-Step Visual Review with manifest-bound input.
- **Substrate consolidation under MuJoCo-Warp.** See §3.2; the most under-discussed structural shift this period.
- **SKILL.md as cross-vendor distribution channel.** A SKILL.md file written for Claude Code now runs in Codex CLI, Gemini CLI, Cursor, Junie. For roboharness, this means the *generated project skill* (per Self-Bootstrapping Contract) reaches more agents without per-vendor work.
- **"Harness engineering ⊂ context engineering"** (HumanLayer's framing). Sub-agents are *context firewalls*, not role-specialized personas. Relevant to how the Visual Reviewer is invoked — already aligned in roboharness's Two-Step design.

---

## 4. Taxonomy / perspectives

### 4.1 Perspective I — By scaling axis (after Anthropic / Cursor / OpenAI)

- **Time scaling** (Anthropic CWC): one agent runs longer without drifting. roboharness's relevance: a long unattended robot-code edit produces a proof pack that survives one-agent → one-handoff.
- **Space scaling** (Cursor): hundreds of agents in parallel. roboharness's relevance: minimal direct application today, but the recursive Planner/Worker pattern *could* apply to large robot-policy sweeps if that ever becomes the workflow.
- **Interaction scaling** (OpenAI Symphony): humans manage work, not individual runs. roboharness's relevance: **central**. roboharness *is* the interaction layer between an unattended robot-code change and a 30-second human approval.

### 4.2 Perspective II — By layer of the stack

```
Layer 0 (model)        : Claude Opus 4.7, GPT-5.2, Gemini, Qwen-VL, Kimi
Layer 1 (substrate)    : MuJoCo-Warp (de facto kernel), Genesis, ManiSkill3, MolmoSpaces
Layer 2 (API surface)  : MuJoCo, Isaac Lab 3.x, mjlab, Newton, Habitat 3
Layer 3 (policy/VLA)   : GR00T N1.7/N2, π₀/π₀.₅, OpenVLA, Helix, SONIC
Layer 4 (data format)  : LeRobotDataset v3, Open X-Embodiment, RLDS
Layer 5 (harness)      : ROBOHARNESS, Anthropic CWC primitives, Symphony
Layer 6 (tool protocols): MCP, Agent Skills (SKILL.md)
```

roboharness sits at Layer 5, depends on Layer 2 (MuJoCo + Meshcat), Layer 3 (GR00T/SONIC), Layer 4 (LeRobotDataset), Layer 6 (SKILL.md + MCP).

### 4.3 When to use which perspective

- **Scaling-axis perspective** is useful when explaining roboharness to people who already understand harness engineering — quickly anchors what roboharness is and is not.
- **Layer perspective** is useful when arguing for or against a specific dependency adoption — clarifies whether a new project is a substitute (same layer) or complement (adjacent layer).

---

## 5. Existing academic and industry taxonomies

### 5.1 Academic surveys

- **MV-RoboBench** (arXiv:2510.19400) — Multi-view spatial reasoning of VLMs in robotic scenes. Direct counter-evidence consideration for H1; specifically supports keeping "view conflict" as Human Escalation Reason.
- **StepEval** (arXiv:2509.19524) — VLM-as-judge subgoal evaluation framework blueprint. Frames a *separate* VLM judge approach; competes with H1's "same agent" stance at a design-pattern level.
- **mjlab** (arXiv:2601.22074) — Reference for the converging MuJoCo-Warp + Isaac Lab manager API path.

### 5.2 Industry stack maps

- `ai-boost/awesome-harness-engineering` — current community-curated entry-point list. Useful for discovery; treat as C-tier (aggregator).
- Addy Osmani synthesis (April 19, 2026) on agent harness engineering — readable consolidation.
- Anthropic `cwc-long-running-agents` repo README — best public list of harness primitives and their Agent SDK equivalents.

---

## 6. Specific recommendations for `roboharness`

### 6.1 Trade-offs by phase

**During v0.3** (current milestone — MuJoCo contract-first trust loop): keep the discipline. The repo's own STATUS.md explicitly says "avoid adding new simulator backends, splitting new showcase repos, expanding SONIC scope without real-model validation evidence, or prioritizing outreach polish over trust-loop calibration." That discipline is correct and is what produced the 91% coverage. Recommendations below are scoped to **not violate** v0.3 constraints.

**Post-v0.3** (next milestone, not yet defined): the ecosystem shifts in this checkpoint (Newton/mjlab convergence, Opus 4.7 vision, GR00T N1.7 commercial) all argue for a particular next-milestone shape — probably a humanoid Proof Pack first-class path, exploiting the strengthened H5, and broader simulator-API coverage exploiting the strengthened H2 — but defining that is out of scope here.

### 6.2 Projects worth deep-reading first

1. **`anthropics/cwc-long-running-agents`** — repo and the underlying March 2026 Anthropic paper. Reason: the hooks + evaluator subagent pattern is the closest publicly available analogue to roboharness's Two-Step Visual Review. Worth a 1-hour read of each hook + the evaluator subagent script.
2. **`openai/symphony`** — SPEC.md + WORKFLOW.md. Reason: Symphony's Proof of Work output is conceptually identical to roboharness's Proof Pack. If roboharness wants to position itself as "the Proof Pack standard for robotics workloads," reading SPEC.md is the cheapest way to align vocabulary.
3. **`mujocolab/mjlab`** + arXiv:2601.22074. Reason: the Isaac Lab manager API on top of MuJoCo-Warp may be the lowest-friction surface to gain Isaac Lab users for roboharness without leaving Pinocchio/MuJoCo-native code paths.
4. **Birgitta Böckeler's `martinfowler.com/articles/harness-engineering.html`** (April 2, 2026). Reason: clearest user-facing mental model of harness as feedforward guides + feedback sensors. Worth reading to align roboharness's external docs voice.
5. **MV-RoboBench paper (arXiv:2510.19400)**. Reason: provides citable evidence for why "view conflict" is a Human Escalation Reason — not heuristic but empirical.

### 6.3 Concrete engineering actions

| # | Action | Status | Value |
|---|--------|--------|-------|
| 1 | Pair-test the maintained MuJoCo grasp wedge at 1.15 MP vs. 3.75 MP proof-pack image resolution under Opus 4.7. Measure: NEEDS_HUMAN rate, dimension-verdict confidence distribution, and any new failure modes from the larger image. | TODO | High — feeds H1 status next checkpoint; possibly the highest-ROI experiment this quarter |
| 2 | Add `mjlab` to the simulator-wrapper compatibility test matrix as a CPU-mocked smoke test. **Do not** add it as a supported backend during v0.3. | TODO | Medium — keeps the door open for the Isaac Lab manager API path without breaking v0.3 discipline |
| 3 | Cite MV-RoboBench (arXiv:2510.19400) in the "view conflict" Human Escalation Reason documentation. | TODO | Low effort, high credibility — turns a design choice into an evidence-backed policy |
| 4 | Cross-reference Anthropic CWC long-running-agents primitives in the Two-Step Visual Review design doc (`docs/designs/agent-visual-review-v1.md`). | TODO | Low effort — strengthens external positioning by anchoring in a recognized reference design |
| 5 | Add `isaacsim-mcp-server` (PyPI) to docs/community/ as a watched integration partner, not an integration target during v0.3. | TODO | Low effort — informs next-milestone decision in §7 Q5 |
| 6 | Update the vectors card next refresh: credit Viv Trivedy as the term originator of "harness engineering"; demote π₁ from "current mainstream" to "watchlist" until confirmed release; add Newton 1.0 as a Layer-1 substrate; add mjlab as a Layer-2 API surface. | TODO | Vectors-card hygiene; reduces drift risk in the next checkpoint |
| 7 | When v0.3 ships and a v0.4 milestone is opened, evaluate whether a humanoid (G1) first-class Proof Pack path or a broader simulator-API coverage move (Isaac Lab manager API via mjlab) gives more leverage. Both are *enabled* by this period's ecosystem changes. | deferred | High at v0.4 time; do not start during v0.3 |

---

## 7. Open questions and what next checkpoint should answer

### Experimental questions (require running code in this repo to answer)

- ☐ **Q1**: Does Opus 4.7's 3.75 MP vision reduce the maintained MuJoCo grasp wedge's NEEDS_HUMAN rate? Run paired baseline at 1.15 MP and 3.75 MP, same prompts, same evidence selection. (Action #1 in §6.3.)
- ☐ **Q2**: Can roboharness's Python-Authored Contract run on top of mjlab + MuJoCo-Warp with only a Gymnasium-wrapper change, or does the Newton/Warp data-type difference (wp.array vs torch.Tensor) require deeper refactor?

### Technical selection questions

- ☐ **Q3**: Should roboharness ship a first-party `roboharness-mcp` server, or integrate with `isaacsim-mcp-server` / `ros-mcp-server` / `mujoco-mcp`? Decision criterion: whether the bounded-evidence + manifest semantics survive translation through a third-party MCP layer. To be answered at v0.4 planning.
- ☐ **Q4**: At v0.4, choose between "humanoid Proof Pack first-class" vs. "broader simulator-API coverage" as the milestone theme.

### Ecosystem-tracking questions

- ☐ **Q5**: Will Figure or Boston Dynamics ship an open SDK in 2026? (Long-running; carries forward to next checkpoint as-is.)
- ☐ **Q6**: π₁ release status. As of May 2026, openpi has π₀, π₀-FAST, π₀.₅ only.
- ☐ **Q7**: Does GR00T N2 ship by end-2026 as previewed? If yes, do the World Action Model's predicted-vs-actual world rollouts constitute a new Visual Review Dimension (currently roboharness has none for predicted-state comparisons)?
- ☐ **Q8**: Will an official LeRobot MCP server be announced? (Vectors card listed this; still unanswered.)

### Long-horizon tracking questions

- ☐ **Q9**: Does the Newton 1.0 / mjlab / Isaac Lab 3.x convergence continue, or does Genesis AI's full-stack push (GENE-26.5, $105M seed) create a credible parallel substrate? Reassess at checkpoint 2026-08 (quarterly audit point).
- ☐ **Q10**: Does Symphony's Linear + Proof-of-Work pattern become a de facto standard for unattended development, and if so, is there a "Symphony for robotics" gap roboharness should fill?

### Source-quality improvement questions

- ☐ **Q11**: This checkpoint's claim about GR00T N1.7's "first-ever dexterity scaling law" rests on a Hugging Face blog post + NVIDIA press release. Verify next checkpoint against the actual GR00T technical report (arXiv 2503.14734 is GR00T N1, not N1.7).
- ☐ **Q12**: This checkpoint took Cognition's "Opus 4.7 works for hours" claim from secondary press coverage. Verify against Cognition's own writeup when available.

---

## 8. Changelog

### 2026-05-21 — Baseline established

**Architecture narrative (baseline):** roboharness is positioned at Layer 5 (harness) of the stack, with the MuJoCo grasp wedge as its anchor and a G1 humanoid path as the first-class second target. The repo is mid-v0.3, on a "MuJoCo contract-first trust loop" milestone, and the ecosystem this period validates the bets the repo made on MuJoCo (H2), G1+GR00T+SONIC (H5), and LeRobot (H4), while raising the same-agent visual review ceiling sharply (H1 strengthened by Opus 4.7, partially counterweighted by MV-RoboBench's multi-view findings).

**Initial assessment of hidden assumptions H1..H7:**

| # | Assumption | Status this period | Why |
|---|-----------|---------------------|-----|
| H1 | Same coding agent does visual review; independent VLM judge is the anti-pattern | **Strengthened, with one caveat** | Opus 4.7 vision resolution 3× and MMMU/MathVista jumps narrow the gap; MV-RoboBench paper shows the multi-view weakness is universal across VLMs (not specific to same-agent review) — which means roboharness's view-conflict escalation is empirically grounded |
| H2 | MuJoCo is the right primary substrate | **Strengthened** | Newton 1.0, Isaac Lab 3.0's `isaaclab_newton`, and mjlab all converge on MuJoCo-Warp as the GPU physics core; MolmoSpaces also defaults to MuJoCo for grasp generation/benchmarking |
| H3 | Contract-first Python authoring is right; YAML/DSL/prompt-only are not | **Supported, no direct evidence** | Symphony uses YAML+Liquid templated WORKFLOW.md; Anthropic CWC uses hooks+subagent scripts. Neither YAML-only nor Python-only has converged upstream. roboharness's Python contract + normalized YAML/JSON snapshot is consistent with the field |
| H4 | LeRobot is the right data format | **Strengthened** | LeRobotDataset v3 stable in `lerobot >= 0.4.0`; NVIDIA integrating GR00T into LeRobot; native G1 support |
| H5 | G1 + GR00T + SONIC is the right humanoid demo stack | **Strengthened** | GR00T N1.7 with UNITREE_G1_SONIC native embodiment and commercial license; Figure/BD still closed |
| H6 | Human review at end of agent run is the right loop | **Strengthened** | Symphony's "Proof of Work" + Cursor's "merge-ready PR" + Anthropic CWC's evaluator subagent all converge on bounded-evidence-at-end |
| H7 | Visual-plus-numeric dual-channel, metric-first with visual-fallback | **Supported** | No direct counter-evidence; aligns with Symphony's CI + walkthrough output and Birgitta Böckeler's "guides + sensors" mental model |

**New projects/concepts (this period — all new since this is the baseline checkpoint):**

- OpenAI Symphony (`openai/symphony`) — released April 27, 2026
- Anthropic CWC long-running-agents (`anthropics/cwc-long-running-agents`) — open-sourced from Code with Claude 2026
- NVIDIA Newton 1.0 (`newton-physics/newton`) — GTC 2026
- Isaac Lab 3.0-beta with `isaaclab_newton` backend
- mjlab (`mujocolab/mjlab`, arXiv:2601.22074) — Isaac Lab API on MuJoCo-Warp
- Claude Opus 4.7 (April 16, 2026) — 3× vision resolution
- GR00T N1.7 commercial early access (April 17, 2026) and N2 preview (GTC March 2026)
- MolmoSpaces v1 (Allen AI, February 11, 2026)
- `isaacsim-mcp-server` (April 6, 2026) — first robotics-specific MCP server on PyPI
- Agent Skills open standard at agentskills.io (~March 14, 2026)

**Status changes** (none — baseline): subsequent checkpoints will track changes against this baseline.

**Judgment revisions** (none — baseline).

**Vectors-card maintenance items to file** (will be applied in Mode B at next vectors-card audit, due 2026-08-21):

- Credit Viv Trivedy (LangChain) as originator of "harness engineering"
- Demote π₁ from "current mainstream" to "watchlist"
- Add Newton 1.0 as a Layer-1 substrate entity
- Add mjlab as a tracked entity under Vector 2
- Add MV-RoboBench and StepEval to Vector 3 academic line
- Note that the agentskills.io date in the vectors card (2026-03-14) is correct; the underlying Anthropic Skills launch was 2025-12-18

---

## Appendix A: full reference links

### A.1 Harness engineering upstream

- Anthropic, "Harness Design for Long-Running Application Development" (Prithvi Rajasekaran): https://www.anthropic.com/engineering/harness-design-long-running-apps
- Anthropic CWC long-running-agents repo: https://github.com/anthropics/cwc-long-running-agents
- OpenAI Symphony repo: https://github.com/openai/symphony
- OpenAI Symphony announcement: https://openai.com/index/open-source-codex-orchestration-symphony/
- Cursor, "Scaling long-running autonomous coding": https://cursor.com/blog/scaling-agents
- Cursor, "Towards self-driving codebases": https://cursor.com/blog/self-driving-codebases
- Cursor Cloud Agents (Feb 24, 2026 launch): tracked through their `cursor.com/blog/` channel
- HumanLayer, "Skill Issue: Harness Engineering for Coding Agents": https://www.humanlayer.dev/blog/skill-issue-harness-engineering-for-coding-agents
- Birgitta Böckeler, "Harness engineering for coding agent users": https://martinfowler.com/articles/harness-engineering.html
- Addy Osmani, "Agent Harness Engineering" (synthesis): https://addyosmani.com/blog/agent-harness-engineering/
- O'Reilly Radar, "Agent Harness Engineering": https://www.oreilly.com/radar/agent-harness-engineering/

### A.2 Robot simulator substrate

- MuJoCo: https://github.com/google-deepmind/mujoco
- MuJoCo Warp: https://github.com/google-deepmind/mujoco_warp
- NVIDIA Newton 1.0: https://github.com/newton-physics/newton
- Isaac Lab 3.0-beta release notes: https://github.com/isaac-sim/IsaacLab/releases/tag/v3.0.0-beta
- Isaac Lab development update discussion: https://github.com/isaac-sim/IsaacLab/discussions/4339
- mjlab: https://github.com/mujocolab/mjlab and arXiv:2601.22074
- mjlab on PyPI: https://pypi.org/project/mjlab/
- MolmoSpaces: https://github.com/allenai/molmospaces
- Allen AI MolmoBot blog: https://allenai.org/blog/molmobot-robot-manipulation
- Genesis: https://github.com/Genesis-Embodied-AI/genesis-world
- Genesis AI (the company): https://www.genesis.ai/blog/gene-26-5-advancing-robotic-manipulation-to-human-level

### A.3 Agent visual review capabilities

- Claude Opus 4.7 launch (April 16, 2026) — A-tier confirmations:
  - GitHub Changelog: https://github.blog/changelog/2026-04-16-claude-opus-4-7-is-generally-available/
  - AWS Bedrock launch announcement: https://aws.amazon.com/blogs/aws/introducing-anthropics-claude-opus-4-7-model-in-amazon-bedrock/
- B-tier press coverage (cross-verified against the above): CNBC, Axios, MarkTechPost (all April 16, 2026)
- MV-RoboBench: https://arxiv.org/pdf/2510.19400
- StepEval: https://arxiv.org/pdf/2509.19524

### A.4 VLA / robot-policy

- NVIDIA Isaac-GR00T: https://github.com/NVIDIA/Isaac-GR00T
- GR00T N1.7 Hugging Face blog: https://huggingface.co/blog/nvidia/gr00t-n1-7
- GR00T N1.7 developer forum announcement: https://forums.developer.nvidia.com/t/early-access-isaac-gr00t-n1-7-open-reasoning-vla-model-for-humanoid-robotics/366916
- NVIDIA GTC 2026 expansion announcement: https://nvidianews.nvidia.com/news/nvidia-expands-open-model-families-to-power-the-next-wave-of-agentic-physical-and-healthcare-ai
- openpi: https://github.com/Physical-Intelligence/openpi
- LeRobotDataset v3 docs: https://huggingface.co/docs/lerobot/lerobot-dataset-v3
- LeRobotDataset v3 blog: https://huggingface.co/blog/lerobot-datasets-v3
- LeRobot G1 docs: https://huggingface.co/docs/lerobot/unitree_g1
- unitree_lerobot: https://github.com/unitreerobotics/unitree_lerobot

### A.5 Humanoid hardware

- Boston Dynamics Atlas product announcement: https://bostondynamics.com/blog/boston-dynamics-unveils-new-atlas-robot-to-revolutionize-industry/
- Humanoid press tracker: https://humanoid.press/

### A.6 MCP / Agent Skills

- isaacsim-mcp-server: https://pypi.org/project/isaacsim-mcp-server/
- ros-mcp-server: https://github.com/robotmcp/ros-mcp-server
- mujoco-mcp (one community variant): https://lobehub.com/mcp/robotlearning123-mujoco-mcp
- The New Stack on Agent Skills: https://thenewstack.io/agent-skills-anthropics-next-bid-to-define-ai-standards/
- agentskills.io (open standard)

---

## Appendix B: glossary

| Term | Meaning |
|------|---------|
| Proof Pack | roboharness term for a bounded set of visual evidence + metrics + baseline comparisons prepared for agent visual review or human escalation. The structural analogue of OpenAI Symphony's "Proof of Work." |
| Visual Reviewer Invocation | A bounded agent visual review call against a manifest. Distinct from "an external VLM service." |
| Harness Contract | The typed Python source of truth for a downstream project's review loop; combines phases, hard metric gates, dimensions, evidence boundaries, approval policy. |
| Two-Step Visual Review | roboharness's bounded review pattern: harness prepares manifest + selected evidence; agent writes review record; harness ingests record. |
| Hard Metric Gate | Deterministic numeric/boolean condition. The safety floor below visual judgment. |
| Human Escalation Reason | Fixed taxonomy value explaining why a verdict requires human decision (e.g., view conflict, baseline blessing required, low-confidence high-risk judgment). |
| Newton 1.0 | NVIDIA + DeepMind + Disney + Linux Foundation open-source physics engine using MuJoCo-Warp + Kamino + VBD solvers under one umbrella. New this period. |
| mjlab | Kevin Zakka et al.'s Isaac Lab manager API on top of MuJoCo-Warp. arXiv:2601.22074. New this period as a tracked entity. |
| Symphony | OpenAI's open-source orchestration spec for Codex agents driven by Linear tickets. Released April 27, 2026. New this period. |
| Proof of Work | Symphony's evidence package output (CI status + walkthrough + sometimes video). Structurally identical to roboharness's Proof Pack. |
| CWC primitives | Anthropic's "Code with Claude 2026 long-running agents" example repo of hooks + evaluator subagent. Direct ancestor of roboharness's Two-Step Visual Review. |
| Same-agent visual review | The roboharness bet that the coding agent's own multimodal capability is sufficient for review; no separate VLM judge layer. H1. |
| World Action Model | GR00T N2's announced architecture (DreamZero-based). Currently preview only. Likely implies new proof-pack dimensions if it ships. |

---

**End of baseline checkpoint. Next checkpoint scheduled for end of June 2026.**

Run the incremental-research loop per §0.2, paying particular attention to the open questions in §7 — especially Q1 (the Opus 4.7 vision resolution experiment) and Q9 (whether the Newton/mjlab convergence continues at the 2026-08 quarterly audit point).
