# Exploit — Reset BioMCP test strategy around request contracts

## Winning Approach

Explore selected the **contract-first pyramid with an incremental request-plan seam**. Routine March gates should prove BioMCP-owned contracts deterministically — CLI intent normalization, request/plan construction, source request shape, fixture response/status mapping, entity orchestration, and renderer/envelope output — while moving broad live public-upstream checks to an explicit small release/live-smoke lane.

The request seam should be incremental:

- CLI/entity boundary: `RequestCommand`-style or entity-local request values after clap normalization and validation.
- Source boundary: source-local `SourceRequestPlan`-style values covering method, URL/path, query, headers, body, auth mode, and cache mode.
- Runtime clients can keep reqwest/existing execution paths; tests assert plans and use fixtures/wiremock before network execution.

## Full-Scale Results

Ticket 371 explicitly excludes rewriting the test suite or changing runtime product behavior. Full scale for exploit therefore means producing the durable accepted strategy artifact, backed by the full explore evidence and regression-controlled inventory.

Produced full-scale strategy artifact:

- `/home/ian/workspace/planning/biomcp/artifacts/371-reset-biomcp-test-strategy-around-request-contracts/request-contract-test-strategy.md`

The artifact includes:

- Current slow/flaky gate evidence and live dependencies.
- Routine test layers and what each proves.
- Request seam ownership boundaries.
- Proposed March profile behavior for kickoff/preflight, focused, spec-only, verify/full-blocking, and release/live-smoke.
- Keep/move/mock/delete matrix for OLS4/discover, disease crosswalk, article/source, and variant/source specs.
- Ordered follow-up ticket set.
- Rollout guardrails and contract numbers.

Exploit-measured target-scale contract values:

| Metric | Exploit value |
|---|---:|
| Representative spec sections classified | 39 |
| Live public CLI wrapper sections | 22 |
| Live non-fixture sections to move/mock/fixture or classify as smoke | 19 |
| Existing fixture-backed representative sections to preserve | 3 |
| OLS4/discover representative sections | 7 |
| Disease/crosswalk representative sections | 11 |
| Article-source representative sections | 15 |
| Variant-source representative sections | 10 |
| Existing source `Mock::given` contracts | 40 |
| Existing async source tests | 47 |
| Query-param assertions | 101 |
| Header assertions | 8 |
| Body assertions | 7 |
| Existing `env_base` seams | 6 |
| Existing explicit plan-function files to reuse | 3 |
| Ticket-370 preflight runs inspected | 3 |
| Ticket-370 preflight failures from unrelated live/spec partition | 2 |
| Known failed-section occurrences | 3 |

Additional exploit control runtime measurement:

- Inventory command: `./architecture/experiments/reset-biomcp-test-strategy-around-request-contracts/scripts/analyze_test_strategy.py`
- Elapsed: 0.06s
- Peak RSS: 25,872 KB

## Regression Control

Regression control reran the exact explore inventory script against the exploit state.

| Control metric | Explore baseline | Exploit result | Rule result |
|---|---:|---:|---|
| Inventory mismatch count | 0 expected | 0 | PASS |
| Result checksum | `b090cba04589f3f9a6e980d2314280698d369f698d9a952f1082ae8316198527` | `b090cba04589f3f9a6e980d2314280698d369f698d9a952f1082ae8316198527` | PASS, unchanged |
| Representative spec sections | 39 | 39 | PASS |
| Live non-fixture sections | 19 | 19 | PASS |
| Fixture-backed sections | 3 | 3 | PASS |
| Source `Mock::given` contracts | 40 | 40 | PASS |
| Query-param assertions | 101 | 101 | PASS |
| Header assertions | 8 | 8 | PASS |
| Body assertions | 7 | 7 | PASS |
| Existing explicit plan-function files | 3 | 3 | PASS |
| Ticket-370 preflight failures | 2 | 2 | PASS |

No carveouts were needed. The projection checksum did not change and mismatch count remained zero. No throughput, latency, RSS, or build/load regression was introduced; this spike changed documentation/results only, not product execution paths. Exploit recorded a new script runtime/RSS measurement for downstream comparison, but explore had not recorded RSS.

Control artifacts:

- `architecture/experiments/reset-biomcp-test-strategy-around-request-contracts/results/test_strategy_inventory.json`
- `architecture/experiments/reset-biomcp-test-strategy-around-request-contracts/results/exploit_regression_control.json`
- `architecture/experiments/reset-biomcp-test-strategy-around-request-contracts/results/exploit_inventory_time.txt`

## Validation

Validated the ticket success criteria against the full-scale strategy artifact:

- Durable artifact exists under `planning/biomcp/artifacts/`.
- Slow/flaky gates and live-source dependencies are named with repo/March evidence.
- Routine test layers and their proof obligations are defined.
- The request-plan seam decision is explicit and ownership boundaries are sketched.
- March validation-profile behavior is proposed for routine, verify, and release/live-smoke lanes.
- Keep/move/mock/delete matrix covers OLS4/discover, disease crosswalk, article/source, and variant/source specs.
- Ordered follow-up build/quickfix tickets are recommended.
- `git diff --name-only -- src Cargo.toml Cargo.lock Makefile spec tests tools docs README.md` reports zero runtime/product file changes.

Validation command also checked the strategy artifact and regression JSON exist and contain the required sections.

Remaining gap: implementation tickets still need to add the actual request-plan seams, deterministic replacements, and profile changes. This spike intentionally does not rewrite the suite.

## Produced Artifacts

Persistent artifacts:

- Strategy artifact: `/home/ian/workspace/planning/biomcp/artifacts/371-reset-biomcp-test-strategy-around-request-contracts/request-contract-test-strategy.md`
- Exploit report: `.march/exploit.md`
- Committed exploit report copy: `architecture/experiments/reset-biomcp-test-strategy-around-request-contracts/exploit.md`
- Regression summary: `architecture/experiments/reset-biomcp-test-strategy-around-request-contracts/results/exploit_regression_control.json`
- Inventory timing: `architecture/experiments/reset-biomcp-test-strategy-around-request-contracts/results/exploit_inventory_time.txt`
- Existing inventory result: `architecture/experiments/reset-biomcp-test-strategy-around-request-contracts/results/test_strategy_inventory.json`
- Existing inventory script: `architecture/experiments/reset-biomcp-test-strategy-around-request-contracts/scripts/analyze_test_strategy.py`

No artifacts were written to `/tmp/`.

## Contract Numbers

Downstream spikes/build tickets can depend on these exploit-measured values:

- Conversion target: 19 representative live non-fixture sections.
- Preserve existing fixture-backed pattern: 3 representative sections.
- First live canaries to isolate: disease `Synonym Rescue` (2 observed failures) and discover `MEF2 relational query` (1 observed failure).
- Existing deterministic source-test base: 40 `Mock::given` contracts, 101 query-param assertions, 8 header assertions, 7 body assertions, 47 async source tests.
- Existing plan seams to reuse: 3 files with plan functions: `src/cli/search_all/plan.rs`, `src/entities/article/planner.rs`, `src/cli/article/dispatch.rs`.
- Current ticket-370 green `spec-pr` evidence to beat by isolation: 127 main spec items in 179.32s plus 22 serialized items in 49.20s after setup.
- Inventory regression checksum: `b090cba04589f3f9a6e980d2314280698d369f698d9a952f1082ae8316198527`.

## Conclusions

The exploit confirms the explore decision at full planning scale: BioMCP already has enough deterministic source-test prior art to make request contracts the routine gate, but the executable spec corpus still overuses live public APIs for ordinary development proof.

This enables a safe incremental reset: add request-plan seams around the live-flaky hot spots, convert representative specs to fixtures/wiremock, split routine March profiles from release/live smoke, then prune brittle assertions only after deterministic replacements land.

Primary remaining risks are fixture drift and premature live-spec removal. The proposed live-smoke lane and staged follow-up tickets mitigate both without continuing to block ordinary tickets on unrelated upstream availability.
