# HotpotQA End-to-End Benchmark

**Date:** 2026-03-22 22:00
**Questions:** 200
**Documents:** 1993
**Total Cost:** $12.18

## Overall Results

| Metric | Value |
|--------|-------|
| Document Recall | **84.8%** (339/400 supporting docs found) |
| Full Recall (both docs found) | 140/200 (70.0%) |
| Partial Recall (≥1 doc found) | 199/200 (99.5%) |
| Avg Cost/Question | $0.061 |

## By Type

| Type | Questions | Recall |
|------|-----------|--------|
| bridge | 165 | 82.1% (271/330) |
| comparison | 35 | 97.1% (68/70) |

## By Level

| Level | Questions | Recall |
|-------|-----------|--------|
| hard | 200 | 84.8% (339/400) |

## Missed Documents (worst 10)

| Question | Type | Recall | Missed |
|----------|------|--------|--------|
| When was the only long-term psychiatric hospital operated fo... | bridge | 0% | docs/Deer Lodge County, Montana.md, docs/Warm Springs, Montana.md |
| Who used a Barrack buster to shoot down a British Army Lynx ... | bridge | 50% | docs/Barrack buster.md |
| Scott Sandelin grew up in a town in what Minnesota county?... | bridge | 50% | docs/Hibbing, Minnesota.md |
| What is one risk to Norway's financial reserve?... | bridge | 50% | docs/Forsvarets Spesialkommando.md |
| What continents are the Moneuptychia found in?... | bridge | 50% | docs/Neotropical realm.md |
| Which market town in Lancashire is part of the BB postcode a... | bridge | 50% | docs/Burnley.md |
| James Spedding was chiefly known as the editor of the works ... | bridge | 50% | docs/Francis Bacon.md |
| Murray Seafield St George Head most recognised for which son... | bridge | 50% | docs/One Night in Bangkok.md |
| What city, located in Lincoln County, Montana, United States... | bridge | 50% | docs/Troy, Montana.md |
| Who is the writer for an American teen sitcom created by Kim... | bridge | 50% | docs/King's Ransom (film).md |
