# Exploit: 25-gene-all-latency

## Winning Approach

Explore promoted `parallel-top`: reuse the MyGene Ensembl ID for OpenTargets calls, then overlap the dominant independent post-MyGene sections (`clinical_context`, Enrichr, expression, HPA, druggability, ClinGen) before applying the smaller tail sections sequentially.

The exploit patch makes that winner the shipped default for section requests that benefit from the overlap, while keeping `BIOMCP_GENE_GET_STRATEGY=baseline|opentargets-ensembl|parallel-top` available for measurement and regression controls. Bare `get gene <symbol>` stays on the smaller sequential future because it does not request the heavy optional section set.

## Full-Scale Results

Full scale used 10 representative genes across cancer, rare-disease, and metabolic categories, both markdown and JSON, 5 isolated-cache runs per gene/mode/strategy. The originally planned `G6PC` input was ambiguous in the live resolver, so the final metabolic panel uses `G6PC1`.

| Gene | Mode | Baseline p50 | Exploit p50 | Delta | Baseline p95 | Exploit p95 |
|---|---|---:|---:|---:|---:|---:|
| BRAF | markdown | 8259.42 ms | 4984.34 ms | -39.65% | 8875.59 ms | 5008.89 ms |
| BRAF | JSON | 8703.06 ms | 5183.54 ms | -40.44% | 13988.35 ms | 11127.60 ms |
| TP53 | markdown | 8016.73 ms | 5206.61 ms | -35.05% | 8203.43 ms | 5509.23 ms |
| TP53 | JSON | 8216.26 ms | 5126.17 ms | -37.61% | 8315.01 ms | 5302.83 ms |
| EGFR | markdown | 8684.33 ms | 5069.72 ms | -41.62% | 9041.12 ms | 5125.12 ms |
| EGFR | JSON | 8001.71 ms | 5124.82 ms | -35.95% | 8015.72 ms | 5306.45 ms |
| KRAS | markdown | 7776.77 ms | 4942.14 ms | -36.45% | 14419.89 ms | 5181.59 ms |
| KRAS | JSON | 7569.82 ms | 5214.83 ms | -31.11% | 7767.31 ms | 18525.95 ms |
| CFTR | markdown | 7770.53 ms | 4969.99 ms | -36.04% | 7806.61 ms | 5231.30 ms |
| CFTR | JSON | 7696.04 ms | 5018.34 ms | -34.79% | 7843.94 ms | 5057.41 ms |
| OPA1 | markdown | 7725.92 ms | 4899.14 ms | -36.59% | 16638.50 ms | 5032.68 ms |
| OPA1 | JSON | 7570.99 ms | 4933.02 ms | -34.84% | 7846.66 ms | 5200.24 ms |
| MECP2 | markdown | 7756.29 ms | 4906.00 ms | -36.75% | 8264.56 ms | 5190.62 ms |
| MECP2 | JSON | 8383.89 ms | 4905.07 ms | -41.49% | 9263.90 ms | 5013.80 ms |
| LDLR | markdown | 8371.91 ms | 5748.97 ms | -31.33% | 11600.47 ms | 7255.72 ms |
| LDLR | JSON | 8046.41 ms | 5442.58 ms | -32.36% | 8161.76 ms | 5734.26 ms |
| PCSK9 | markdown | 7478.06 ms | 5278.72 ms | -29.41% | 8097.82 ms | 5399.23 ms |
| PCSK9 | JSON | 7432.79 ms | 5118.22 ms | -31.14% | 7654.50 ms | 5307.21 ms |
| G6PC1 | markdown | 9333.42 ms | 4900.63 ms | -47.49% | 17246.35 ms | 5286.62 ms |
| G6PC1 | JSON | 8480.77 ms | 4824.00 ms | -43.12% | 10324.16 ms | 5158.46 ms |

Summary:
- Median p50 improvement: 36.52% markdown, 35.40% JSON.
- Mean p50 improvement: 37.04% markdown, 36.29% JSON.
- All exploit p50 and p95 values stayed below the 30s target.
- The measured exploit p95 maximum was 18.53s (`KRAS` JSON), so the full-scale run also stayed under the 20s stretch target in this pass.

## Regression Control

Control reproduced the exact explore winner workload: `BRAF`, `TP53`, `CFTR`, both modes, 5 isolated-cache runs, release binary, shipped default exploit path.

| Gene | Mode | Explore `parallel-top` p50 | Exploit p50 | p50 rule | Explore `parallel-top` p95 | Exploit p95 | p95 rule |
|---|---|---:|---:|---|---:|---:|---|
| BRAF | markdown | 5751.61 ms | 5181.44 ms | pass | 11134.38 ms | 5593.79 ms | pass |
| BRAF | JSON | 5436.81 ms | 5049.38 ms | pass | 5549.99 ms | 10785.19 ms | noise carveout |
| TP53 | markdown | 5145.13 ms | 5193.15 ms | pass, +0.93% | 5273.99 ms | 9221.70 ms | noise carveout |
| TP53 | JSON | 5280.04 ms | 5367.60 ms | pass, +1.66% | 11812.43 ms | 6416.88 ms | pass |
| CFTR | markdown | 4935.72 ms | 4945.22 ms | pass, +0.19% | 5392.41 ms | 5156.49 ms | pass |
| CFTR | JSON | 5081.69 ms | 4912.80 ms | pass | 5183.53 ms | 5038.87 ms | pass |

Carveouts:
- `BRAF` JSON p95: literal p95 parity failed. The outlier was a single run with wall clock 12137.06 ms and Reactome `pathways` at 7244 ms; all other sections were in the normal band. A same-code rerun moved the JSON p95 to 6134.65 ms with no 7s pathways leg. This is upstream measurement noise, not a code-path regression.
- `TP53` markdown p95 and expression completion: literal p95 parity failed because one run hit the 8s expression cap (`expression` elapsed 8001 ms, outcome `empty`). A same-code rerun restored expression completion to 5/5 data runs and markdown p95 to 5692.17 ms. This is upstream measurement noise, not a code-path regression.

Correctness control:
- Final rebuilt binary output diff: `BRAF`, `TP53`, and `CFTR` markdown checksums were identical between `baseline` and exploit.
- Final rebuilt binary canonical JSON checksums were identical between `baseline` and exploit.
- Field-level JSON mismatch count was 0 for all three genes.

## Validation

Output parity:
- `architecture/experiments/25-gene-all-latency/results/pre_post_output_diff_final.json`
- Result: `BRAF`, `TP53`, `CFTR` all had `json_identical=true`, `markdown_identical=true`, `mismatch_count=0`.

Rust validation:
- `cargo fmt --check` passed.
- `cargo test --lib` passed: 1761 tests.
- `cargo clippy --lib --tests -- -D warnings` passed.

Spec validation:
- Consolidated the split `get gene CFTR all` markdown and JSON checks into one paired block in `spec/18-source-labels.md`.
- Ran `uv run --extra dev pytest spec/18-source-labels.md --mustmatch-lang bash --mustmatch-timeout 60 -v`.
- Result: 16 passed, 4 skipped.

Remaining validation gap:
- I did not run full `make spec-pr`; the ticket-specific touched spec file and the focused Rust gate are green.

## Produced Artifacts

Source and spec changes:
- `src/entities/gene.rs`: `parallel-top` is now the default strategy for heavy optional-section requests, with explicit env overrides preserved.
- `src/cli/tests/outcome.rs`: one deep alias-fallback test now uses the same large-stack thread pattern already used nearby; this prevents validation stack overflow without changing production behavior.
- `spec/18-source-labels.md`: gene markdown and JSON `all` checks are paired in one mustmatch block.
- `architecture/experiments/25-gene-all-latency/scripts/gene_all_latency_matrix.py`
- `architecture/experiments/25-gene-all-latency/scripts/gene_all_output_diff.py`

Checked-in result artifacts:
- `architecture/experiments/25-gene-all-latency/results/exploit_regression_control.json`
- `architecture/experiments/25-gene-all-latency/results/exploit_regression_control_matrix.json`
- `architecture/experiments/25-gene-all-latency/results/full_scale_before_after_final.json`
- `architecture/experiments/25-gene-all-latency/results/full_scale_before_after_final_matrix.json`
- `architecture/experiments/25-gene-all-latency/results/pre_post_output_diff_final.json`
- `architecture/experiments/25-gene-all-latency/results/braf_noise_check.json`
- `architecture/experiments/25-gene-all-latency/results/tp53_noise_check.json`
- `architecture/experiments/25-gene-all-latency/results/g6pc1_replacement.json`

Persistent local work artifacts, not committed:
- Raw timing/cache work directories under `architecture/experiments/25-gene-all-latency/work/`
- Final release binary at `target/release/biomcp`

Report copy:
- `.march/exploit.md`
- `architecture/experiments/25-gene-all-latency/exploit.md`

## Contract Numbers

Downstream spikes can depend on these exploit-measured values from the final full-scale panel:
- `BRAF all` p50: 4984.34 ms markdown, 5183.54 ms JSON.
- Full-scale median p50: approximately 4.98s markdown and 5.12s JSON across the 10-gene panel.
- Full-scale p50 range: 4899.14-5748.97 ms markdown, 4824.00-5442.58 ms JSON.
- Full-scale p95 maximum: 7255.72 ms markdown, 18525.95 ms JSON.
- Full-scale budget result: all measured exploit p50 and p95 values were below 30s; all were below 20s in this pass.
- Correctness contract: BRAF/TP53/CFTR output mismatch count is 0 between legacy baseline and exploit for canonical JSON and markdown.

## Conclusions

The exploit ships the explore winner and proves it at target scale. The original 44-45s failure still does not reproduce in this checkout, but the shipped path reduces the current live baseline by roughly 35-37% at p50 and gives enough headroom for paired markdown+JSON spec blocks.

The main remaining risk is upstream variance. The exploit removes serial local waiting, but p95 can still swing when individual providers stall, especially Reactome, GTEx/expression, or ClinGen. Section-level timeouts and empty-state fallbacks continue to bound those tails, and all measured exploit p95 values stayed below the 30s budget.
