# HotpotQA End-to-End Benchmark

**Date:** 2026-03-26 21:13
**Questions:** 500
**Documents:** 4960
**Total Cost:** $50.02

## Overall Results

| Metric | Value |
|--------|-------|
| Document Recall | **87.7%** (877/1000 supporting docs found) |
| Full Recall (both docs found) | 380/500 (76.0%) |
| Partial Recall (≥1 doc found) | 497/500 (99.4%) |
| Avg Cost/Question | $0.100 |

## By Type

| Type | Questions | Recall |
|------|-----------|--------|
| bridge | 399 | 85.1% (679/798) |
| comparison | 101 | 98.0% (198/202) |

## By Level

| Level | Questions | Recall |
|-------|-----------|--------|
| hard | 500 | 87.7% (877/1000) |

## Missed Documents (worst 10)

| Question | Type | Recall | Missed |
|----------|------|--------|--------|
| When was the only long-term psychiatric hospital operated fo... | bridge | 0% | docs/Deer Lodge County, Montana.md, docs/Warm Springs, Montana.md |
| Bobby Bowden coached which former Toronto Blue Jays minor le... | bridge | 0% | docs/2000 Florida State Seminoles football team.md, docs/Chris Weinke.md |
| What was the nickname of the player who played his final sea... | bridge | 0% | docs/2016 Miami Marlins season.md, docs/José Fernández (pitcher).md |
| What is one risk to Norway's financial reserve?... | bridge | 50% | docs/Forsvarets Spesialkommando.md |
| What continents are the Moneuptychia found in?... | bridge | 50% | docs/Neotropical realm.md |
| Which market town in Lancashire is part of the BB postcode a... | bridge | 50% | docs/Burnley.md |
| What British company was involved in recording the audiobook... | bridge | 50% | docs/Northern Lights (novel).md |
| Gleanntáin Ghlas' Ghaoth Dobhair is a song that can be heard... | bridge | 50% | docs/Irish pub.md |
| The creator of "Wallace and Gromit" also created what animat... | bridge | 50% | docs/Creature Comforts.md |
| What city, located in Lincoln County, Montana, United States... | bridge | 50% | docs/Troy, Montana.md |
