# Multi-SDK Benchmark Report

**Date:** 2026-03-18T10:58:04.828Z
**Duration:** 185.2s
**Runs:** 16 (0 success, 16 failed)
**Success Rate:** 0.0%

## SDK Comparison

| SDK           | Tier     | Avg Score | Avg Time | Avg Cost | Success |
| ------------- | -------- | --------- | -------- | -------- | ------- |
| Google Gemini | balanced | n/a       | 0ms      | $0.0000  | 0%      |

## Variant Rankings (by avg score)

| Rank | Variant            | Score | Time | Cost    | Success |
| ---- | ------------------ | ----- | ---- | ------- | ------- |
| 1    | google-raw-c1-0    | 0.0   | 0ms  | $0.0000 | 0%      |
| 2    | google-raw-c8498-1 | 0.0   | 0ms  | $0.0000 | 0%      |

## Per-Commit Results

### Weather Card (simple)

- **Avg Score:** 0.0
- **Best:** none
- **Worst:** none

### Multi-Step Survey Form (medium)

- **Avg Score:** 0.0
- **Best:** none
- **Worst:** none

### Kanban Task Board (medium)

- **Avg Score:** 0.0
- **Best:** none
- **Worst:** none

### Interactive Periodic Table (complex)

- **Avg Score:** 0.0
- **Best:** none
- **Worst:** none

### E-Commerce Product Page (complex)

- **Avg Score:** 0.0
- **Best:** none
- **Worst:** none

### Chat Interface (medium)

- **Avg Score:** 0.0
- **Best:** none
- **Worst:** none

### Stock Ticker Dashboard (simple)

- **Avg Score:** 0.0
- **Best:** none
- **Worst:** none

### Onboarding Wizard (medium)

- **Avg Score:** 0.0
- **Best:** none
- **Worst:** none

## Failures

- **google-raw-c1-0** × weather-card: Claude Code returned an error result: There's an issue with the selected model (gemini-3-flash-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c1-0** × survey-form: Claude Code returned an error result: There's an issue with the selected model (gemini-3-flash-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c1-0** × kanban-board: Claude Code returned an error result: There's an issue with the selected model (gemini-3-flash-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c1-0** × periodic-table: Claude Code returned an error result: There's an issue with the selected model (gemini-3-flash-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c1-0** × product-page: Claude Code returned an error result: There's an issue with the selected model (gemini-3-flash-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c1-0** × chat-interface: Claude Code returned an error result: There's an issue with the selected model (gemini-3-flash-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c1-0** × stock-ticker: Claude Code returned an error result: There's an issue with the selected model (gemini-3-flash-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c1-0** × onboarding-wizard: Claude Code returned an error result: There's an issue with the selected model (gemini-3-flash-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c8498-1** × weather-card: Claude Code returned an error result: There's an issue with the selected model (gemini-3.1-flash-lite-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c8498-1** × survey-form: Claude Code returned an error result: There's an issue with the selected model (gemini-3.1-flash-lite-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c8498-1** × kanban-board: Claude Code returned an error result: There's an issue with the selected model (gemini-3.1-flash-lite-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c8498-1** × periodic-table: Claude Code returned an error result: There's an issue with the selected model (gemini-3.1-flash-lite-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c8498-1** × product-page: Claude Code returned an error result: There's an issue with the selected model (gemini-3.1-flash-lite-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c8498-1** × chat-interface: Claude Code returned an error result: There's an issue with the selected model (gemini-3.1-flash-lite-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c8498-1** × stock-ticker: Claude Code returned an error result: There's an issue with the selected model (gemini-3.1-flash-lite-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
- **google-raw-c8498-1** × onboarding-wizard: Claude Code returned an error result: There's an issue with the selected model (gemini-3.1-flash-lite-preview). It may not exist or you may not have access to it. Run --model to pick a different model.
