# Multi-SDK Benchmark Report

**Date:** 2026-03-18T08:54:47.476Z
**Duration:** 404.1s
**Runs:** 16 (12 success, 4 failed)
**Success Rate:** 75.0%

## SDK Comparison

| SDK           | Tier     | Avg Score | Avg Time | Avg Cost | Success |
| ------------- | -------- | --------- | -------- | -------- | ------- |
| Google Gemini | balanced | 74.0      | 1.6m     | $0.7659  | 75%     |

## Variant Rankings (by avg score)

| Rank | Variant      | Score | Time  | Cost    | Success |
| ---- | ------------ | ----- | ----- | ------- | ------- |
| 1    | google-raw-0 | 75.7  | 2.2m  | $1.0518 | 100%    |
| 2    | google-sdk-1 | 70.5  | 24.1s | $0.1941 | 50%     |

## Per-Commit Results

### Weather Card (simple)

- **Avg Score:** 69.0
- **Best:** google-raw-0
- **Worst:** google-sdk-1

### Multi-Step Survey Form (medium)

- **Avg Score:** 68.0
- **Best:** google-raw-0
- **Worst:** google-raw-0

### Kanban Task Board (medium)

- **Avg Score:** 66.0
- **Best:** google-raw-0
- **Worst:** google-sdk-1

### Interactive Periodic Table (complex)

- **Avg Score:** 79.0
- **Best:** google-raw-0
- **Worst:** google-raw-0

### E-Commerce Product Page (complex)

- **Avg Score:** 78.3
- **Best:** google-raw-0
- **Worst:** google-sdk-1

### Chat Interface (medium)

- **Avg Score:** 80.0
- **Best:** google-raw-0
- **Worst:** google-raw-0

### Stock Ticker Dashboard (simple)

- **Avg Score:** 80.5
- **Best:** google-raw-0
- **Worst:** google-sdk-1

### Onboarding Wizard (medium)

- **Avg Score:** 73.0
- **Best:** google-raw-0
- **Worst:** google-raw-0

## Failures

- **google-sdk-1** × survey-form: Google ADK iterative: no compiled code produced
- **google-sdk-1** × periodic-table: Google ADK iterative: no compiled code produced
- **google-sdk-1** × chat-interface: Google ADK iterative: no compiled code produced
- **google-sdk-1** × onboarding-wizard: Google ADK iterative: no compiled code produced
