# HotpotQA End-to-End Benchmark

**Date:** 2026-03-23 11:37
**Questions:** 200
**Documents:** 1993
**Total Cost:** $12.48

## Overall Results

| Metric | Value |
|--------|-------|
| Document Recall | **87.5%** (350/400 supporting docs found) |
| Full Recall (both docs found) | 151/200 (75.5%) |
| Partial Recall (≥1 doc found) | 199/200 (99.5%) |
| Avg Cost/Question | $0.062 |

## By Type

| Type | Questions | Recall |
|------|-----------|--------|
| bridge | 165 | 85.5% (282/330) |
| comparison | 35 | 97.1% (68/70) |

## By Level

| Level | Questions | Recall |
|-------|-----------|--------|
| hard | 200 | 87.5% (350/400) |

## Missed Documents (worst 10)

| Question | Type | Recall | Missed |
|----------|------|--------|--------|
| When was the only long-term psychiatric hospital operated fo... | bridge | 0% | docs/Deer Lodge County, Montana.md, docs/Warm Springs, Montana.md |
| Who used a Barrack buster to shoot down a British Army Lynx ... | bridge | 50% | docs/Barrack buster.md |
| Scott Sandelin grew up in a town in what Minnesota county?... | bridge | 50% | docs/Hibbing, Minnesota.md |
| What continents are the Moneuptychia found in?... | bridge | 50% | docs/Neotropical realm.md |
| What British company was involved in recording the audiobook... | bridge | 50% | docs/Northern Lights (novel).md |
| Gleanntáin Ghlas' Ghaoth Dobhair is a song that can be heard... | bridge | 50% | docs/Irish pub.md |
| What city, located in Lincoln County, Montana, United States... | bridge | 50% | docs/Troy, Montana.md |
| Who is the writer for an American teen sitcom created by Kim... | bridge | 50% | docs/King's Ransom (film).md |
| George Borba was part of the team that competed at what 1970... | bridge | 50% | docs/1970 FIFA World Cup.md |
| Which former mill town is near Rivington Hall Barn?... | bridge | 50% | docs/Bolton.md |
