# HotpotQA End-to-End Benchmark

**Date:** 2026-03-22 15:48
**Questions:** 50
**Documents:** 500
**Total Cost:** $3.65

## Overall Results

| Metric | Value |
|--------|-------|
| Document Recall | **87.0%** (87/100 supporting docs found) |
| Full Recall (both docs found) | 39/50 (78.0%) |
| Partial Recall (≥1 doc found) | 48/50 (96.0%) |
| Avg Cost/Question | $0.073 |

## By Type

| Type | Questions | Recall |
|------|-----------|--------|
| bridge | 41 | 85.4% (70/82) |
| comparison | 9 | 94.4% (17/18) |

## By Level

| Level | Questions | Recall |
|-------|-----------|--------|
| hard | 50 | 87.0% (87/100) |

## Missed Documents (worst 10)

| Question | Type | Recall | Missed |
|----------|------|--------|--------|
| Who's fifth album and debut single are Startin' Fires and Au... | bridge | 0% | docs/Blake Shelton.md, docs/Startin' Fires.md |
| Who used a Barrack buster to shoot down a British Army Lynx ... | bridge | 0% | docs/1994 British Army Lynx shootdown.md, docs/Barrack buster.md |
| Scott Sandelin grew up in a town in what Minnesota county?... | bridge | 50% | docs/Hibbing, Minnesota.md |
| Who is the writer for an American teen sitcom created by Kim... | bridge | 50% | docs/King's Ransom (film).md |
| Where did the descendants of the group of black Indians asso... | bridge | 50% | docs/Mascogos.md |
| George Borba was part of the team that competed at what 1970... | bridge | 50% | docs/1970 FIFA World Cup.md |
| Which former mill town is near Rivington Hall Barn?... | bridge | 50% | docs/Bolton.md |
| What major city is John Bice Memorial Oval located within?... | bridge | 50% | docs/Christies Beach, South Australia.md |
| Which team based in South Wales participated in a final game... | bridge | 50% | docs/Newport County A.F.C..md |
| Which of the following is home to the 642nd Aviation Support... | comparison | 50% | docs/Valley International Airport.md |
