# HotpotQA End-to-End Benchmark

**Date:** 2026-03-25 16:45
**Questions:** 500
**Documents:** 4960
**Total Cost:** $31.65

## Overall Results

| Metric | Value |
|--------|-------|
| Document Recall | **93.0%** (930/1000 supporting docs found) |
| Full Recall (both docs found) | 435/500 (87.0%) |
| Partial Recall (≥1 doc found) | 495/500 (99.0%) |
| Avg Cost/Question | $0.063 |

## By Type

| Type | Questions | Recall |
|------|-----------|--------|
| bridge | 399 | 92.6% (739/798) |
| comparison | 101 | 94.6% (191/202) |

## By Level

| Level | Questions | Recall |
|-------|-----------|--------|
| hard | 500 | 93.0% (930/1000) |

## Missed Documents (worst 10)

| Question | Type | Recall | Missed |
|----------|------|--------|--------|
| Who is considered the more well known author, Roger Angell o... | comparison | 0% | docs/Harry Stephen Keeler.md, docs/Roger Angell.md |
| Do Chestnut and Arenga trees both grow in the northern hemis... | comparison | 0% | docs/Arenga.md, docs/Chestnut.md |
| Nekeshia Shiondrail Henderson is an American former professi... | bridge | 0% | docs/Nekeshia Henderson.md, docs/Texas Longhorns women's basketball.md |
| What film studio produced both National Treasure and The Com... | comparison | 0% | docs/National Treasure (film).md, docs/The Computer Wore Tennis Shoes.md |
| The University in Pullman, Washington is named after whom?... | bridge | 0% | docs/Edward R. Murrow College of Communication.md, docs/Washington State University.md |
| Scott Sandelin grew up in a town in what Minnesota county?... | bridge | 50% | docs/Hibbing, Minnesota.md |
| What is one risk to Norway's financial reserve?... | bridge | 50% | docs/Forsvarets Spesialkommando.md |
| What continents are the Moneuptychia found in?... | bridge | 50% | docs/Neotropical realm.md |
| Which market town in Lancashire is part of the BB postcode a... | bridge | 50% | docs/Burnley.md |
| Gleanntáin Ghlas' Ghaoth Dobhair is a song that can be heard... | bridge | 50% | docs/Irish pub.md |
