# Multi-SDK Benchmark Report

**Date:** 2026-03-18T03:06:44.071Z
**Duration:** 74.7s
**Runs:** 12 (11 success, 1 failed)
**Success Rate:** 91.7%

## SDK Comparison

| SDK    | Tier     | Avg Score | Avg Time | Avg Cost | Success |
| ------ | -------- | --------- | -------- | -------- | ------- |
| OpenAI | balanced | 81.9      | 26.1s    | $0.0436  | 92%     |

## Variant Rankings (by avg score)

| Rank | Variant      | Score | Time  | Cost    | Success |
| ---- | ------------ | ----- | ----- | ------- | ------- |
| 1    | openai-sdk-1 | 82.7  | 25.8s | $0.0000 | 100%    |
| 2    | openai-raw-0 | 81.0  | 26.4s | $0.0958 | 83%     |

## Per-Commit Results

### Weather Card (simple)

- **Avg Score:** 83.0
- **Best:** openai-raw-0
- **Worst:** openai-sdk-1

### Multi-Step Survey Form (medium)

- **Avg Score:** 76.4
- **Best:** openai-raw-0
- **Worst:** openai-sdk-1

### Interactive Periodic Table (complex)

- **Avg Score:** 80.8
- **Best:** openai-sdk-1
- **Worst:** openai-raw-0

### Chat Interface (medium)

- **Avg Score:** 89.2
- **Best:** openai-sdk-1
- **Worst:** openai-raw-0

### Stock Ticker Dashboard (simple)

- **Avg Score:** 85.0
- **Best:** openai-sdk-1
- **Worst:** openai-sdk-1

### Onboarding Wizard (medium)

- **Avg Score:** 78.5
- **Best:** openai-sdk-1
- **Worst:** openai-raw-0

## Failures

- **openai-raw-0** × stock-ticker: OpenAI raw iterative: no compiled code produced
