# Multi-SDK Benchmark Report

**Date:** 2026-03-22T01:42:39.089Z
**Duration:** 1212.0s
**Runs:** 8 (7 success, 1 failed)
**Success Rate:** 87.5%

## SDK Comparison

| SDK              | Tier     | Avg Score | Avg Time | Avg Cost | Success |
| ---------------- | -------- | --------- | -------- | -------- | ------- |
| Anthropic Claude | balanced | 76.9      | 2.2m     | $0.6606  | 88%     |

## Variant Rankings (by avg score)

| Rank | Variant              | Score | Time | Cost    | Success | Tier Eval |
| ---- | -------------------- | ----- | ---- | ------- | ------- | --------- |
| 1    | claude-multi-agent-0 | 76.9  | 2.2m | $0.6606 | 88%     | n/a       |

## Per-Commit Results

### Weather Card (simple)

- **Avg Score:** 86.0
- **Best:** claude-multi-agent-0
- **Worst:** claude-multi-agent-0

### Multi-Step Survey Form (medium)

- **Avg Score:** 70.4
- **Best:** claude-multi-agent-0
- **Worst:** claude-multi-agent-0

### Kanban Task Board (medium)

- **Avg Score:** 0.0
- **Best:** none
- **Worst:** none

### Interactive Periodic Table (complex)

- **Avg Score:** 79.0
- **Best:** claude-multi-agent-0
- **Worst:** claude-multi-agent-0

### E-Commerce Product Page (complex)

- **Avg Score:** 73.0
- **Best:** claude-multi-agent-0
- **Worst:** claude-multi-agent-0

### Chat Interface (medium)

- **Avg Score:** 78.0
- **Best:** claude-multi-agent-0
- **Worst:** claude-multi-agent-0

### Stock Ticker Dashboard (simple)

- **Avg Score:** -1.0
- **Best:** claude-multi-agent-0
- **Worst:** claude-multi-agent-0

### Onboarding Wizard (medium)

- **Avg Score:** 75.0
- **Best:** claude-multi-agent-0
- **Worst:** claude-multi-agent-0

## Failures

- **claude-multi-agent-0** × kanban-board: Enforced coding loop: all 10 attempts failed (1038718 tokens)
  Last error: ## self_check errors
  2 issue(s) found:

[ts2304] Line 46: Cannot find name '**GGUI_STREAM_SPEC**'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

[ts2304] Line 85: Cannot find name '**GGUI_STREAM_SPEC_END**'.
Fix: This name is not exported. Check your imports — use get_primitives to see available components

## compile_component errors

{
"success": false,
"phase": "self_check",
"error": "2 issue(s) found — fix these before recompiling:\n\n[ts2304] Line 46: Cannot find name '**GGUI_STREAM_SPEC**'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components\n\n[ts2304] Line 85: Cannot find name '**GGUI_STREAM_SPEC_END**'.\n Fix: This name is not exported. Check your imports — use get_primitives to see available components"
}
