# Multi-SDK Benchmark Report

**Date:** 2026-03-18T01:25:21.641Z
**Duration:** 505.0s
**Runs:** 36 (34 success, 2 failed)
**Success Rate:** 94.4%

## SDK Comparison

| SDK              | Tier     | Avg Score | Avg Time | Avg Cost | Success |
| ---------------- | -------- | --------- | -------- | -------- | ------- |
| Anthropic Claude | balanced | 81.4      | 2.6m     | $0.6387  | 83%     |
| OpenAI           | balanced | 79.8      | 27.2s    | $0.0637  | 100%    |
| Google Gemini    | balanced | 76.2      | 3.7m     | $0.7035  | 100%    |

## Variant Rankings (by avg score)

| Rank | Variant      | Score | Time  | Cost    | Success |
| ---- | ------------ | ----- | ----- | ------- | ------- |
| 1    | claude-sdk-1 | 86.0  | 2.8m  | $0.3125 | 67%     |
| 2    | claude-raw-0 | 80.5  | 2.5m  | $0.8562 | 100%    |
| 3    | openai-raw-2 | 80.5  | 24.6s | $0.1274 | 100%    |
| 4    | openai-sdk-3 | 79.1  | 29.7s | $0.0000 | 100%    |
| 5    | google-raw-4 | 77.6  | 4.2m  | $0.9163 | 100%    |
| 6    | google-sdk-5 | 74.8  | 3.1m  | $0.4907 | 100%    |

## Per-Commit Results

### Weather Card (simple)

- **Avg Score:** 80.9
- **Best:** claude-sdk-1
- **Worst:** google-sdk-5

### Kanban Task Board (medium)

- **Avg Score:** 68.8
- **Best:** claude-raw-0
- **Worst:** openai-sdk-3

### Interactive Periodic Table (complex)

- **Avg Score:** 78.8
- **Best:** claude-raw-0
- **Worst:** google-sdk-5

### Chat Interface (medium)

- **Avg Score:** 77.8
- **Best:** openai-sdk-3
- **Worst:** google-sdk-5

### Stock Ticker Dashboard (simple)

- **Avg Score:** 84.0
- **Best:** openai-raw-2
- **Worst:** google-sdk-5

### Onboarding Wizard (medium)

- **Avg Score:** 81.7
- **Best:** openai-sdk-3
- **Worst:** claude-raw-0

## Failures

- **claude-sdk-1** × kanban-board: Claude SDK iterative: no compiled code produced
- **claude-sdk-1** × stock-ticker: Claude SDK iterative: no compiled code produced
