# HotpotQA End-to-End Benchmark

**Date:** 2026-03-23 14:25
**Questions:** 500
**Documents:** 4960
**Total Cost:** $28.64

## Overall Results

| Metric | Value |
|--------|-------|
| Document Recall | **89.6%** (896/1000 supporting docs found) |
| Full Recall (both docs found) | 400/500 (80.0%) |
| Partial Recall (≥1 doc found) | 496/500 (99.2%) |
| Avg Cost/Question | $0.057 |

## By Type

| Type | Questions | Recall |
|------|-----------|--------|
| bridge | 399 | 87.7% (700/798) |
| comparison | 101 | 97.0% (196/202) |

## By Level

| Level | Questions | Recall |
|-------|-----------|--------|
| hard | 500 | 89.6% (896/1000) |

## Missed Documents (worst 10)

| Question | Type | Recall | Missed |
|----------|------|--------|--------|
| Who used a Barrack buster to shoot down a British Army Lynx ... | bridge | 0% | docs/1994 British Army Lynx shootdown.md, docs/Barrack buster.md |
| What is one risk to Norway's financial reserve?... | bridge | 0% | docs/Economy of Norway.md, docs/Forsvarets Spesialkommando.md |
| Which market town in Lancashire is part of the BB postcode a... | bridge | 0% | docs/BB postcode area.md, docs/Burnley.md |
| What British company was involved in recording the audiobook... | bridge | 0% | docs/Northern Lights (novel).md, docs/Northern Lights Audio.md |
| Scott Sandelin grew up in a town in what Minnesota county?... | bridge | 50% | docs/Hibbing, Minnesota.md |
| The song "Medicine" by The 1975 appears in a film directed b... | bridge | 50% | docs/Drive (2011 film).md |
| What continents are the Moneuptychia found in?... | bridge | 50% | docs/Neotropical realm.md |
| Murray Seafield St George Head most recognised for which son... | bridge | 50% | docs/Murray Head.md |
| Which film was Shannyn Sossamon in that was directed by Mich... | bridge | 50% | docs/Shannyn Sossamon.md |
| What kind of act does Whaling in Australia and Right whale h... | bridge | 50% | docs/Right whale.md |
