# Exploit — Spike source-first article Markdown coverage

## Winning Approach

The explore winner is a pivot: do **not** add a new BioMCP fulltext rung from the sampled sources yet. The production-adjacent win is to surface provenance/quality/reuse flags over the existing source-first ladder, using metadata already available from Europe PMC core search and PMC OA manifests.

Why:

- Current Europe PMC PMCID `fullTextXML` already covered all sampled PMC/OA fulltext cases.
- NCBI BioC PMC matched that coverage and adds useful passage/license metadata, but did not improve fulltext coverage in this sample.
- PubTator3 BioC JSON covered all PMIDs but only as title/abstract + entity annotations, not full text.
- PMC OA manifests add license/retraction/package provenance, not a distinct renderer.
- S2ORC is a bulk PDF-derived dataset shape and belongs, if anywhere, in Vault/batch ingestion.

## Full-Scale Results

Ticket-defined full scale is the complete bounded candidate/source matrix, because this spike explicitly excludes production renderers/source rungs and requires no BioMCP runtime behavior changes.

Exploit added an offline analyzer:

- `architecture/experiments/381-spike-source-first-article-markdown-coverage/scripts/analyze_source_coverage.py`

It consumes persisted probe JSON and emits stable contract artifacts without re-hitting live sources. Full-scale exploit probe generated at `2026-05-24T11:16:14Z` over 4 article cases and 29 case/source rows.

Contract summary from `results/exploit_analysis_summary.json`:

| Metric | Exploit result |
|---|---:|
| Europe PMC core metadata coverage | 4/4 |
| Current Europe PMC PMCID `fullTextXML` coverage | 3/3 |
| Direct Europe PMC PMID `fullTextXML` coverage | 0/4 |
| NCBI BioC PMCID coverage | 3/3 |
| NCBI BioC PMID coverage | 3/4 |
| PubTator3 PMID coverage | 4/4 |
| PubTator3 PMCID coverage | 0/3 |
| PMC OA manifest coverage | 3/3 |
| Current XML total sections / paragraphs / tables / references | 91 / 432 / 3 / 485 |
| NCBI BioC PMCID total passages / text chars | 935 / 296,065 |
| PubTator3 PMID total annotations | 64 |

Full-scale decision remains unchanged: build provenance/quality flags first; defer BioC renderer/fallback until fixtures prove it wins on misses/degradations.

## Regression Control

Reproduced the explore benchmark with the same dataset, source endpoints, and methodology using:

- Baseline: `results/source_coverage_probe.json`, generated `2026-05-24T11:07:03Z`
- Exploit control: `results/exploit_source_coverage_probe.json`, generated `2026-05-24T11:16:14Z`
- Comparison: `results/exploit_regression_control.csv`

Coverage and quality contract numbers matched exactly:

| Contract number | Explore | Exploit |
|---|---:|---:|
| Europe PMC core metadata coverage | 4/4 | 4/4 |
| Current Europe PMC PMCID XML coverage | 3/3 | 3/3 |
| Direct Europe PMC PMID XML coverage | 0/4 | 0/4 |
| NCBI BioC PMCID coverage | 3/3 | 3/3 |
| NCBI BioC PMID coverage | 3/4 | 3/4 |
| PubTator3 PMID coverage | 4/4 | 4/4 |
| PubTator3 PMCID coverage | 0/3 | 0/3 |
| PMC OA manifest coverage | 3/3 | 3/3 |
| Current XML sections / paragraphs / tables / refs | 91 / 432 / 3 / 485 | 91 / 432 / 3 / 485 |
| BioC PMCID passages / text chars | 935 / 296,065 | 935 / 296,065 |
| PubTator PMID annotations | 64 | 64 |

Regression-rule carveouts:

- Metric waived: live-source elapsed latency for 4/29 rows.
- Rule applied: throughput/latency should match/beat explore or be within 3%; deviation is allowed only when evidence shows measurement noise rather than a real regression.
- Evidence: availability, HTTP status, parse state, and all coverage/quality counts were unchanged. The changed values are external live API elapsed times, and this exploit changed only offline analysis/reporting code, not fetch/runtime behavior.

Rows over 3% slower:

| Case | Source | Explore ms | Exploit ms | Ratio | Evidence |
|---|---|---:|---:|---:|---|
| `jats_verified_pmid` | Europe PMC core search | 408 | 517 | 1.267 | Same 200/found result and metadata coverage. |
| `jats_verified_pmid` | NCBI BioC by PMID | 202 | 234 | 1.158 | Same 200/parsed result and BioC counts. |
| `ncbi_bioc_sample` | Europe PMC PMID XML miss | 451 | 487 | 1.080 | Same 404 miss. |
| `non_pmc_control` | NCBI BioC PMID non-BioC response | 108 | 154 | 1.426 | Same 200 HTML/non-parse miss. |

No checksum/projection-output rule applies: the spike does not generate article Markdown output and makes no production renderer/runtime changes. The offline summary JSON changed only by adding exploit-derived reporting artifacts.

## Validation

Validated:

- `analyze_source_coverage.py` runs over explore JSON and exploit-control JSON.
- `exploit_analysis_summary.json` contract numbers equal `explore_analysis_summary.json`.
- `exploit_regression_control.csv` has no coverage regressions and no quality/count regressions.
- Ticket success criteria are satisfied:
  - coverage/quality/provenance matrix exists,
  - recommendation names BioMCP candidates, Vault handoff, and no-go/default-no sources,
  - follow-up sequence starts with fixture-backed provenance flags,
  - routine-gate strategy is fixture/request-contract based,
  - no BioMCP runtime behavior changed.
- `git diff --check` passed.

Remaining gap: the candidate set is bounded by design. Broader live checks should stay in operator/release smoke lanes, not routine tests.

## Produced Artifacts

Persistent locations:

- Offline analyzer: `architecture/experiments/381-spike-source-first-article-markdown-coverage/scripts/analyze_source_coverage.py`
- Explore analyzer output: `architecture/experiments/381-spike-source-first-article-markdown-coverage/results/explore_analysis_summary.json`
- Explore case/source rows: `architecture/experiments/381-spike-source-first-article-markdown-coverage/results/explore_case_source_rows.csv`
- Exploit live-control probe: `architecture/experiments/381-spike-source-first-article-markdown-coverage/results/exploit_source_coverage_probe.json`
- Exploit live-control compact matrix: `architecture/experiments/381-spike-source-first-article-markdown-coverage/results/exploit_source_coverage_matrix.csv`
- Exploit analyzer output: `architecture/experiments/381-spike-source-first-article-markdown-coverage/results/exploit_analysis_summary.json`
- Exploit case/source rows: `architecture/experiments/381-spike-source-first-article-markdown-coverage/results/exploit_case_source_rows.csv`
- Regression comparison: `architecture/experiments/381-spike-source-first-article-markdown-coverage/results/exploit_regression_control.csv`
- Report copy: `architecture/experiments/381-spike-source-first-article-markdown-coverage/exploit.md`

No `/tmp/` artifacts are required. No binary/index/large generated files were produced.

## Contract Numbers

Downstream spikes/builds can depend on exploit-measured values:

- Current XML ladder remains the fulltext coverage baseline for this candidate set: 3/3 PMCID/OA articles, 91 sections, 432 paragraphs, 3 tables, 485 references.
- Direct Europe PMC PMID `fullTextXML` remains a miss path: 0/4; keep ID conversion/PMCID bridge.
- NCBI BioC PMC is coverage-equivalent on PMCID/OA articles in this set: 3/3 by PMCID, 3/4 by PMID, 935 passages and 296,065 text chars, zero entity annotations.
- PubTator3 BioC JSON is annotation enrichment: 4/4 by PMID, 64 annotations total, but no fulltext signal and 0/3 by PMCID parameter.
- PMC OA manifest is provenance enrichment: 3/3 manifests with CC BY license, `retracted=no`, and tgz package links.
- S2ORC is not a BioMCP runtime rung: dataset metadata is available, but the source requires bulk dataset download and has ODC-BY dataset terms plus article-level rights caveats.

Recommended follow-up order:

1. Build fixture-backed article fulltext provenance/quality flags from existing Europe PMC core + PMC OA metadata.
2. Collect miss/degradation fixtures before considering an NCBI BioC renderer/fallback.
3. Treat PubTator as title/abstract entity annotation enrichment, not fulltext.
4. Hand S2ORC to Vault/batch architecture only if storage, license, and update policy are accepted.

## Conclusions

The exploit confirms the explore decision at full ticket scale. The source-first ladder should not grow a new live fulltext rung from this evidence. The smallest safe production slice is provenance/quality surfacing over existing source contracts, backed by fixtures/request contracts and truthful reuse warnings.

Main risk remaining: the bounded sample may miss articles where BioC succeeds and current XML/HTML misses. That risk should be retired by targeted fixture collection, not by adding BioC to the runtime ladder prematurely.
