Cube 工作流

Install dependencies

make install ```

Installation

```bash

Quickstart

Run Hello Example

The hello_miniwob recipe demonstrates running a ReAct agent on the MiniWob benchmark.

Start here — first 2 tasks, in-process (fast, no Ray required):

make debug          # → uv run recipes/hello_miniwob.py --limit 2

Full benchmark (parallel via Ray):

make hello          # → uv run recipes/hello_miniwob.py

Configuration

A recipe is a declarative config file: it imports canonical configs by name, tweaks a few attributes, builds one or more Experiment objects, and ends with run(...). Copy a recipe from recipes/ and edit it — recipes are documentation-by-example, not a CLI.

from cube_harness.agents.genny_configs import GENNY_CONFIGS  # "default", "swe"
from cube_harness.infra import INFRA_CONFIGS                  # ~/.cube/infra.py; "local" built in
from cube_harness.recipe import run

agent = GENNY_CONFIGS["swe"]          # every lookup is a fresh deep copy
agent.budget.cost_limit = 2.0         # validated at the assignment site

exp = Experiment(name="x", agent_config=agent, benchmark_config=..., infra=INFRA_CONFIGS["local"])
if __name__ == "__main__":
    run(exp)                          # or run(exp_a, exp_b)

run() is the only CLI, identical for every recipe and not extensible: --limit N (first N tasks, in-process), --ray N (worker count), --set dotted.path=value (ad-hoc override). For anything structural, clone the file. Config objects are typed Pydantic models, serialized with every experiment for reproducibility.

Infra is machine-local in ~/.cube/infra.py (a dict[str, InfraConfig], never committed; credentials come from env). "local" works with zero setup. To use a cluster/cloud, copy recipes/infra_template.py to ~/.cube/infra.py and edit it — it documents the process and shows LocalInfraConfig plus commented Toolkit/Azure examples.

See docs/configuration.md for the full philosophy, a comparison with Hydra/YAML/CLI approaches, and how to run sweeps.

API Keys

Set your OpenAI API key:

export OPENAI_API_KEY=your-key-here

Any LiteLLM-supported provider works — just change model_name in the recipe.

Core Components

• Agent — LLM-powered decision maker that receives observations and produces actions
• Environment — Executes actions, provides observations and rewards (tool + task composition)
• Tool — Modular action provider that exposes an action space, reusable across benchmarks
• ActionSpace — Defines the set of possible actions a tool can execute
• Task — Defines goals, validation logic, and action subsets
• Trajectory — Stores interaction history (observations, actions, rewards)
• Episode — Single agent-environment loop for one task; records a trajectory
• Benchmark — Collection of tasks; produces env configs for episodes
• Experiment — Coordinates execution of multiple episodes across a benchmark
• ExpRunner — Execution runtime (sequential or parallel via Ray)