# HotpotQA End-to-End Benchmark

**Date:** 2026-03-22 13:44
**Questions:** 50
**Documents:** 500
**Total Cost:** $3.35

## Overall Results

| Metric | Value |
|--------|-------|
| Document Recall | **78.0%** (78/100 supporting docs found) |
| Full Recall (both docs found) | 30/50 (60.0%) |
| Partial Recall (≥1 doc found) | 48/50 (96.0%) |
| Avg Cost/Question | $0.067 |

## By Type

| Type | Questions | Recall |
|------|-----------|--------|
| bridge | 41 | 75.6% (62/82) |
| comparison | 9 | 88.9% (16/18) |

## By Level

| Level | Questions | Recall |
|-------|-----------|--------|
| hard | 50 | 78.0% (78/100) |

## Missed Documents (worst 10)

| Question | Type | Recall | Missed |
|----------|------|--------|--------|
| Who's fifth album and debut single are Startin' Fires and Au... | bridge | 0% | docs/Blake Shelton.md, docs/Startin' Fires.md |
| When was the only long-term psychiatric hospital operated fo... | bridge | 0% | docs/Deer Lodge County, Montana.md, docs/Warm Springs, Montana.md |
| Who used a Barrack buster to shoot down a British Army Lynx ... | bridge | 50% | docs/Barrack buster.md |
| Scott Sandelin grew up in a town in what Minnesota county?... | bridge | 50% | docs/Hibbing, Minnesota.md |
| What is one risk to Norway's financial reserve?... | bridge | 50% | docs/Forsvarets Spesialkommando.md |
| What continents are the Moneuptychia found in?... | bridge | 50% | docs/Neotropical realm.md |
| Which market town in Lancashire is part of the BB postcode a... | bridge | 50% | docs/Burnley.md |
| James Spedding was chiefly known as the editor of the works ... | bridge | 50% | docs/Francis Bacon.md |
| Murray Seafield St George Head most recognised for which son... | bridge | 50% | docs/One Night in Bangkok.md |
| What British company was involved in recording the audiobook... | bridge | 50% | docs/Northern Lights (novel).md |
