# Multi-SDK Benchmark Report

**Date:** 2026-03-19T13:29:44.793Z
**Duration:** 244.5s
**Runs:** 7 (6 success, 1 failed)
**Success Rate:** 85.7%

## SDK Comparison

| SDK           | Tier     | Avg Score | Avg Time | Avg Cost | Success |
| ------------- | -------- | --------- | -------- | -------- | ------- |
| Google Gemini | balanced | 78.1      | 2.5m     | $1.0610  | 86%     |

## Variant Rankings (by avg score)

| Rank | Variant                   | Score | Time | Cost    | Success |
| ---- | ------------------------- | ----- | ---- | ------- | ------- |
| 1    | google-single-component-0 | 78.1  | 2.5m | $1.0610 | 86%     |

## Per-Commit Results

### Weather Card (simple)

- **Avg Score:** 75.0
- **Best:** google-single-component-0
- **Worst:** google-single-component-0

### Multi-Step Survey Form (medium)

- **Avg Score:** 0.0
- **Best:** none
- **Worst:** none

### Kanban Task Board (medium)

- **Avg Score:** 75.0
- **Best:** google-single-component-0
- **Worst:** google-single-component-0

### Interactive Periodic Table (complex)

- **Avg Score:** 80.4
- **Best:** google-single-component-0
- **Worst:** google-single-component-0

### Chat Interface (medium)

- **Avg Score:** 75.0
- **Best:** google-single-component-0
- **Worst:** google-single-component-0

### Stock Ticker Dashboard (simple)

- **Avg Score:** 81.6
- **Best:** google-single-component-0
- **Worst:** google-single-component-0

### Onboarding Wizard (medium)

- **Avg Score:** 81.6
- **Best:** google-single-component-0
- **Worst:** google-single-component-0

## Failures

- **google-single-component-0** × survey-form: Enforced coding loop: all 30 attempts failed (2702409 tokens)
  Last error: ## self_check errors
  63 issue(s) found:

[ts2322] Line 59: Type '{ gap: any; var: true; }' is not assignable to type 'IntrinsicAttributes & StackProps'.
Property 'var' does not exist on type 'IntrinsicAttributes & StackProps'.
Fix: Check the prop type — see the component's TypeScript interface for allowed values

[ts2304] Line 59: Cannot find name 'ggui'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 59: Cannot find name 'spacing'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 59: Cannot find name 'px'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2322] Line 81: Type '{ gap: any; var: true; }' is not assignable to type 'IntrinsicAttributes & StackProps'.
Property 'var' does not exist on type 'IntrinsicAttributes & StackProps'.
Fix: Check the prop type — see the component's TypeScript interface for allowed values

[ts2304] Line 81: Cannot find name 'ggui'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 81: Cannot find name 'spacing'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 81: Cannot find name 'px'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 83: Cannot find name 'props'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 89: Cannot find name 'formData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 90: Cannot find name 'setFormData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 90: Cannot find name 'formData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 101: Cannot find name 'props'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 105: Cannot find name 'formData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 108: Cannot find name 'formData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 109: Cannot find name 'formData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 110: Cannot find name 'setFormData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 110: Cannot find name 'formData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 132: Cannot find name 'borderRadius'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2322] Line 133: Type '{ gap: any; var: true; }' is not assignable to type 'IntrinsicAttributes & StackProps'.
Property 'var' does not exist on type 'IntrinsicAttributes & StackProps'.
Fix: Check the prop type — see the component's TypeScript interface for allowed values

[ts2304] Line 133: Cannot find name 'ggui'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 133: Cannot find name 'spacing'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 133: Cannot find name 'px'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 135: Cannot find name 'formData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 136: Cannot find name 'formData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 137: Cannot find name 'formData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 138: Cannot find name 'formData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 139: Cannot find name 'formData'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2322] Line 149: Type '{ shadow: "md"; padding: any; var: true; }' is not assignable to type 'IntrinsicAttributes & CardProps'.
Property 'var' does not exist on type 'IntrinsicAttributes & CardProps'.
Fix: Check the prop type — see the component's TypeScript interface for allowed values

[ts2304] Line 149: Cannot find name 'ggui'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 149: Cannot find name 'spacing'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 149: Cannot find name 'px'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 149: Cannot find name 'radius'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 149: Cannot find name 'border'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2322] Line 150: Type '{ gap: any; var: true; }' is not assignable to type 'IntrinsicAttributes & StackProps'.
Property 'var' does not exist on type 'IntrinsicAttributes & StackProps'.
Fix: Check the prop type — see the component's TypeScript interface for allowed values

[ts2304] Line 150: Cannot find name 'ggui'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 150: Cannot find name 'spacing'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 150: Cannot find name 'px'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2322] Line 151: Type '{ gap: any; var: true; }' is not assignable to type 'IntrinsicAttributes & StackProps'.
Property 'var' does not exist on type 'IntrinsicAttributes & StackProps'.
Fix: Check the prop type — see the component's TypeScript interface for allowed values

[ts2304] Line 151: Cannot find name 'ggui'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 151: Cannot find name 'spacing'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 151: Cannot find name 'px'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 153: Cannot find name 'activeStep'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 155: Cannot find name 'activeStep'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 158: Cannot find name 'margin'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 158: Cannot find name 'ggui'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 158: Cannot find name 'spacing'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 158: Cannot find name 'px'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 159: Cannot find name 'direction'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 159: Cannot find name 'justify'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 161: Cannot find name 'activeStep'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 161: Cannot find name 'isSubmitting'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 162: Cannot find name 'setActiveStep'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 163: Cannot find name 'activeStep'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 168: Cannot find name 'isStepValid'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 168: Cannot find name 'isSubmitting'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 169: Cannot find name 'activeStep'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 169: Cannot find name 'handleSubmit'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 169: Cannot find name 'setActiveStep'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 170: Cannot find name 'isStepValid'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 170: Cannot find name 'isSubmitting'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 172: Cannot find name 'isSubmitting'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 172: Cannot find name 'activeStep'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

## compile_component errors

{
"success": false,
"phase": "self_check",
"error": "63 issue(s) found — fix these before recompiling:\n\n[ts2322] Line 59: Type '{ gap: any; var: true; }' is not assignable to type 'IntrinsicAttributes & StackProps'.\n Property 'var' does not exist on type 'IntrinsicAttributes & StackProps'.\n Fix: Check the prop type — see the component's TypeScript interface for allowed values\n\n[ts2304] Line 59: Cannot find name 'ggui'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 59: Cannot find name 'spacing'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 59: Cannot find name 'px'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2322] Line 81: Type '{ gap: any; var: true; }' is not assignable to type 'IntrinsicAttributes & StackProps'.\n Property 'var' does not exist on type 'IntrinsicAttributes & StackProps'.\n Fix: Check the prop type — see the component's TypeScript interface for allowed values\n\n[ts2304] Line 81: Cannot find name 'ggui'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 81: Cannot find name 'spacing'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 81: Cannot find name 'px'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 83: Cannot find name 'props'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 89: Cannot find name 'formData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 90: Cannot find name 'setFormData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 90: Cannot find name 'formData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 101: Cannot find name 'props'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 105: Cannot find name 'formData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 108: Cannot find name 'formData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 109: Cannot find name 'formData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 110: Cannot find name 'setFormData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 110: Cannot find name 'formData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 132: Cannot find name 'borderRadius'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2322] Line 133: Type '{ gap: any; var: true; }' is not assignable to type 'IntrinsicAttributes & StackProps'.\n Property 'var' does not exist on type 'IntrinsicAttributes & StackProps'.\n Fix: Check the prop type — see the component's TypeScript interface for allowed values\n\n[ts2304] Line 133: Cannot find name 'ggui'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 133: Cannot find name 'spacing'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 133: Cannot find name 'px'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 135: Cannot find name 'formData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 136: Cannot find name 'formData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 137: Cannot find name 'formData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 138: Cannot find name 'formData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 139: Cannot find name 'formData'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2322] Line 149: Type '{ shadow: \"md\"; padding: any; var: true; }' is not assignable to type 'IntrinsicAttributes & CardProps'.\n Property 'var' does not exist on type 'IntrinsicAttributes & CardProps'.\n Fix: Check the prop type — see the component's TypeScript interface for allowed values\n\n[ts2304] Line 149: Cannot find name 'ggui'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 149: Cannot find name 'spacing'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 149: Cannot find name 'px'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 149: Cannot find name 'radius'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 149: Cannot find name 'border'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2322] Line 150: Type '{ gap: any; var: true; }' is not assignable to type 'IntrinsicAttributes & StackProps'.\n Property 'var' does not exist on type 'IntrinsicAttributes & StackProps'.\n Fix: Check the prop type — see the component's TypeScript interface for allowed values\n\n[ts2304] Line 150: Cannot find name 'ggui'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 150: Cannot find name 'spacing'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 150: Cannot find name 'px'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2322] Line 151: Type '{ gap: any; var: true; }' is not assignable to type 'IntrinsicAttributes & StackProps'.\n Property 'var' does not exist on type 'IntrinsicAttributes & StackProps'.\n Fix: Check the prop type — see the component's TypeScript interface for allowed values\n\n[ts2304] Line 151: Cannot find name 'ggui'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 151: Cannot find name 'spacing'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 151: Cannot find name 'px'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 153: Cannot find name 'activeStep'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 155: Cannot find name 'activeStep'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 158: Cannot find name 'margin'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 158: Cannot find name 'ggui'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 158: Cannot find name 'spacing'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 158: Cannot find name 'px'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 159: Cannot find name 'direction'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 159: Cannot find name 'justify'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 161: Cannot find name 'activeStep'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 161: Cannot find name 'isSubmitting'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 162: Cannot find name 'setActiveStep'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 163: Cannot find name 'activeStep'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 168: Cannot find name 'isStepValid'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 168: Cannot find name 'isSubmitting'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 169: Cannot find name 'activeStep'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 169: Cannot find name 'handleSubmit'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 169: Cannot find name 'setActiveStep'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 170: Cannot find name 'isStepValid'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 170: Cannot find name 'isSubmitting'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 172: Cannot find name 'isSubmitting'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 172: Cannot find name 'activeStep'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components"
}
