# HotpotQA End-to-End Benchmark

**Date:** 2026-03-22 18:47
**Questions:** 200
**Documents:** 1993
**Total Cost:** $12.42

## Overall Results

| Metric | Value |
|--------|-------|
| Document Recall | **83.2%** (333/400 supporting docs found) |
| Full Recall (both docs found) | 138/200 (69.0%) |
| Partial Recall (≥1 doc found) | 195/200 (97.5%) |
| Avg Cost/Question | $0.062 |

## By Type

| Type | Questions | Recall |
|------|-----------|--------|
| bridge | 165 | 80.6% (266/330) |
| comparison | 35 | 95.7% (67/70) |

## By Level

| Level | Questions | Recall |
|-------|-----------|--------|
| hard | 200 | 83.2% (333/400) |

## Missed Documents (worst 10)

| Question | Type | Recall | Missed |
|----------|------|--------|--------|
| Who's fifth album and debut single are Startin' Fires and Au... | bridge | 0% | docs/Blake Shelton.md, docs/Startin' Fires.md |
| Who used a Barrack buster to shoot down a British Army Lynx ... | bridge | 0% | docs/1994 British Army Lynx shootdown.md, docs/Barrack buster.md |
| What is the year of the event that occured first, Making Tod... | bridge | 0% | docs/Frozen Fever.md, docs/Making Today a Perfect Day.md |
| The song "Medicine" by The 1975 appears in a film directed b... | bridge | 0% | docs/Drive (2011 film).md, docs/Medicine (The 1975 song).md |
| What is one risk to Norway's financial reserve?... | bridge | 0% | docs/Economy of Norway.md, docs/Forsvarets Spesialkommando.md |
| Scott Sandelin grew up in a town in what Minnesota county?... | bridge | 50% | docs/Hibbing, Minnesota.md |
| What continents are the Moneuptychia found in?... | bridge | 50% | docs/Neotropical realm.md |
| Which market town in Lancashire is part of the BB postcode a... | bridge | 50% | docs/Burnley.md |
| James Spedding was chiefly known as the editor of the works ... | bridge | 50% | docs/Francis Bacon.md |
| Murray Seafield St George Head most recognised for which son... | bridge | 50% | docs/One Night in Bangkok.md |
