# PHASE 1 — Gold (ground truth) availability

## Reference-label investigation

Looked at `batch-runner/core/rubric_loader.py` and the local HF snapshot
`data/gdpval-local/data/train-00000-of-00001.parquet` (220 rows). The
GDPVal rubric schema exposes:

| column | content | per-item gold? |
|---|---|---|
| `rubric_json` | list of `{score, criterion}` | **no** verdict label |
| `rubric_pretty` | rendered text version of the rubric | no |
| `deliverable_files` / `deliverable_file_urls` / `deliverable_file_hf_uris` | empty arrays for every row inspected (no canonical answer files distributed) | no |
| `reference_files` | input materials for the task, **not** an answer | no |
| `RubricItem` dataclass | `rubric_item_id`, `criterion`, `score`, `required` | no `expected_verdict` |

There is **no per-rubric-item expected verdict** (pass/partial/fail) and
**no canonical deliverable answer** in the dataset. The judge's verdict
on each item therefore has nothing to be measured against, other than
another judge's verdict, which is the exact circularity the rev2 spec
calls out as the prior failure mode.

## Consequence

The thesis verdict in Phase 4 — "perception-on reduces critical
false-pass rate / raises gold agreement" — **cannot be computed from the
dataset alone**. It requires owner hand-grading of a small candidate
set.

## gold_candidates

Generated by `scripts/phase1_gold_candidates.py`. See
[gold_candidates.md](gold_candidates.md). Total: **19** rubric items
across the 10 shared exp003 tasks, selected as:

| reason | count |
|---|---:|
| visual modality (perception's direct target) | 12 |
| audio modality (perception's direct target)  | 1 |
| formatting modality (flip from standard, not perception-addressable) | 6 |

(`critical_regression` is also a selection rule, but Phase 0(a)'s 3
critical regressions are all formatting and are already in the formatting
bucket above.)

## STOP — owner-go required

Per rev2 spec ("STOP 지점: reference label이 없으면 여기서 owner-go
필요"), Phase 4 (gold-based thesis verdict) is **blocked** until the
owner hand-grades the 19 candidates and produces
`tasks/0531_sunday/gold_verdicts.json` of the form:

```json
[
  {"task": "<task_id>", "rid": "<rubric_item_id>", "gold_verdict": "pass|partial|fail|unsure"},
  ...
]
```

Phases 2 and 3 are gold-independent and proceed in parallel below.

## Honest framing

Without dataset gold and without owner hand-grades, any "thesis verdict"
the agent might produce in Phase 4 is just judge-vs-judge — i.e. the
same self-graded comparison that has produced misleading conclusions
before. The rev2 spec explicitly forbids that. Phase 4 below states
BLOCKED, not a verdict.
