# bench/corpora/conll2003_en

English Reuters newswire NER as a **precision probe** on real natural
text, with gold spans from the canonical CoNLL-2003 shared-task
annotations.

## Why

`ai4privacy_en` is the only other English corpus in the bench, and it
is synthetic — generated by an LLM from privacy-policy templates. That
answers "does anonde catch the PII we generated?" but tells us nothing
about whether anonde over-flags on capitalised common nouns, headlines,
or formal English prose. CoNLL-2003 is the canonical English NER
benchmark (Reuters newswire, 1996–1997) and every English NER paper
benchmarks against it, which makes the precision number here directly
comparable to published literature. We sample 300 sentences
(deterministic via `--seed`) from the `test` split — the standard
evaluation slice.

## Caveats — read before quoting numbers

- **BIO-precision-only gold.** CoNLL-2003 gold is PER / LOC / ORG / MISC.
  We map PER → PERSON, LOC → LOCATION, ORG → ORGANIZATION and **drop
  MISC** (no clean anonde canonical mapping; MISC mixes nationalities,
  events, products). Anonde emits 12+ entity types — any anonde finding
  outside PERSON / LOCATION / ORGANIZATION counts as a false positive
  in this scoring even when legitimate. Example: anonde tags "1996"
  as `DATE_TIME` → CoNLL gold has no DATE → FP. So **precision on this
  corpus is a LOWER BOUND**; true precision restricted to PER / LOC /
  ORG is higher.
- **Sentence-level, not paragraph-level.** Average length ~14 tokens.
  GLiNER's recall benefits from longer context; expect its scores here
  to underestimate full-document performance.
- **News-domain bias.** Reuters newswire from 1996–1997 skews heavily
  toward business, sports, and politics. Person and organisation
  density is much higher than general-purpose English text.
- **License.** CoNLL-2003 is research-only redistribution. The bench
  **does not commit the corpus data** — `loader.py` streams from HF
  at build time and emits the derived JSONL into `data/`, which is
  gitignored alongside the other corpora.

## Run

```bash
# fetch + emit corpus.jsonl
make -C bench/corpora/conll2003_en data

# include in the matrix (conll2003_en must be in bench/Makefile's
# EN_CORPORA list — parent agent will register it)
make -C bench corpus-conll2003_en
```

## Data provenance

- Source: `conll2003` on Hugging Face Datasets (mirror of the original
  Tjong Kim Sang & De Meulder 2003 shared-task corpus, Reuters newswire).
- Tag set: PER / LOC / ORG / MISC; we drop MISC and canonicalise the
  rest to PERSON / LOCATION / ORGANIZATION.
- Split: `test` (the standard evaluation slice, ~3.5k sentences).
- License: Research-only redistribution per the original Reuters
  license. The bench streams at build time; no copy is committed.

## Sample document

```json
{"id":"conll2003-en-0000",
 "text":"SOCCER - JAPAN GET LUCKY WIN , CHINA IN SURPRISE DEFEAT .",
 "entities":[
   {"start":10,"end":15,"type":"LOCATION"},
   {"start":31,"end":36,"type":"LOCATION"}
 ]}
```
