# Multi-SDK Benchmark Report

**Date:** 2026-03-22T12:03:41.445Z
**Duration:** 300.1s
**Runs:** 8 (7 success, 1 failed)
**Success Rate:** 87.5%

## SDK Comparison

| SDK    | Tier     | Avg Score | Avg Time | Avg Cost | Success |
| ------ | -------- | --------- | -------- | -------- | ------- |
| OpenAI | balanced | 66.9      | 1.7m     | $0.9255  | 88%     |

## Variant Rankings (by avg score)

| Rank | Variant          | Score | Time | Cost    | Success | Tier Eval |
| ---- | ---------------- | ----- | ---- | ------- | ------- | --------- |
| 1    | openai-default-0 | 66.9  | 1.7m | $0.9255 | 88%     | 0P/1W/6F  |

## Per-Commit Results

### Weather Card (simple)

- **Avg Score:** 73.0
- **Best:** openai-default-0
- **Worst:** openai-default-0
- **Tier Eval:** 0 pass, 0 warn, 1 fail

### Multi-Step Survey Form (medium)

- **Avg Score:** 80.4
- **Best:** openai-default-0
- **Worst:** openai-default-0
- **Tier Eval:** 0 pass, 0 warn, 1 fail

### Kanban Task Board (medium)

- **Avg Score:** 34.0
- **Best:** openai-default-0
- **Worst:** openai-default-0
- **Tier Eval:** 0 pass, 0 warn, 1 fail

### Interactive Periodic Table (complex)

- **Avg Score:** 72.0
- **Best:** openai-default-0
- **Worst:** openai-default-0
- **Tier Eval:** 0 pass, 1 warn, 0 fail

### E-Commerce Product Page (complex)

- **Avg Score:** 0.0
- **Best:** none
- **Worst:** none

### Chat Interface (medium)

- **Avg Score:** 68.8
- **Best:** openai-default-0
- **Worst:** openai-default-0
- **Tier Eval:** 0 pass, 0 warn, 1 fail

### Stock Ticker Dashboard (simple)

- **Avg Score:** 58.0
- **Best:** openai-default-0
- **Worst:** openai-default-0
- **Tier Eval:** 0 pass, 0 warn, 1 fail

### Onboarding Wizard (medium)

- **Avg Score:** 82.0
- **Best:** openai-default-0
- **Worst:** openai-default-0
- **Tier Eval:** 0 pass, 0 warn, 1 fail

## Failures

- **openai-default-0** × product-page: Timeout after 300000ms
