# Interface BenchmarkAdapter<TInstance, TPrediction, TEvalResult>

Contract every benchmark implementation fulfills.

Type parameters:

- `TInstance`: one task / problem in the benchmark (e.g., a SWE-bench issue)
- `TPrediction`: the solver's output (e.g., a proposed patch)
- `TEvalResult`: the evaluator's verdict (e.g., patch applied + tests passed)

A correct implementation composes as: `loadInstances -> runInstance(each) -> evaluate(each) -> summarize`

#### Example

    class SweBenchAdapter implements BenchmarkAdapter<SweIssue, SwePatch, SweEval> {
      readonly name = 'swe-bench';
      readonly variant = 'lite';
      async loadInstances(config) { ... }
      async runInstance(inst, ctx) { ... }
      async evaluate(inst, pred) { ... }
      summarize(results) { ... }
    }

interface BenchmarkAdapter<TInstance, TPrediction, TEvalResult> {
name: string;
variant?: string;
loadInstances(
config: Record<string, unknown>,
): Promise<readonly TInstance[]>;
runInstance(
instance: TInstance,
ctx: BenchmarkRunContext,
): Promise<TPrediction>;
evaluate(
instance: TInstance,
prediction: TPrediction,
): Promise<TEvalResult>;
isPass(result: TEvalResult): boolean;
summarize(
results: readonly TEvalResult[],
runTimeMs: number,
): BenchmarkRunSummary;
}

#### Type Parameters

- TInstance
- TPrediction
- TEvalResult

- Defined in packages/nexus-agents/src/benchmarks/adapter.ts:72

##### Index

### Properties

name
variant?

### Methods

loadInstances
runInstance
evaluate
isPass
summarize

## Properties

### `Readonly` name

name: string

Stable identifier (e.g., 'swe-bench', 'humaneval'). Used in CLI routing and reporting.

- Defined in packages/nexus-agents/src/benchmarks/adapter.ts:74

### `Optional` `Readonly` variant

variant?: string

Optional variant within a benchmark family (e.g., 'lite' vs 'verified').

- Defined in packages/nexus-agents/src/benchmarks/adapter.ts:76

## Methods

### loadInstances

- loadInstances(config: Record<string, unknown>): Promise<readonly TInstance[]>

  Load the benchmark task set from disk/remote. Runs once per invocation.

  #### Parameters

  - config: Record<string, unknown>

  #### Returns Promise<readonly TInstance[]>

  - Defined in packages/nexus-agents/src/benchmarks/adapter.ts:79

### runInstance

- runInstance(instance: TInstance, ctx: BenchmarkRunContext): Promise<TPrediction>

  Execute the solver on one instance. No evaluation here — just generate the prediction.

  #### Parameters

  - instance: TInstance
  - ctx: BenchmarkRunContext

  #### Returns Promise<TPrediction>

  - Defined in packages/nexus-agents/src/benchmarks/adapter.ts:82

### evaluate

- evaluate(instance: TInstance, prediction: TPrediction): Promise<TEvalResult>

  Evaluate a prediction against ground truth. Returns a benchmark-specific verdict.

  #### Parameters

  - instance: TInstance
  - prediction: TPrediction

  #### Returns Promise<TEvalResult>

  - Defined in packages/nexus-agents/src/benchmarks/adapter.ts:85

### isPass

- isPass(result: TEvalResult): boolean

  Determine whether a verdict counts as pass. Keeps pass/fail semantics localized.

  #### Parameters

  - result: TEvalResult

  #### Returns boolean

  - Defined in packages/nexus-agents/src/benchmarks/adapter.ts:88

### summarize

- summarize(
  results: readonly TEvalResult[],
  runTimeMs: number,
  ): BenchmarkRunSummary

  Aggregate instance results into a summary. Should be pure + deterministic.

  #### Parameters

  - results: readonly TEvalResult[]
  - runTimeMs: number

  #### Returns BenchmarkRunSummary

  - Defined in packages/nexus-agents/src/benchmarks/adapter.ts:91