How we prove our results — Precision Medicine MCP Platform

Explainability How each recommendation shows its work

5 mechanisms

Component What it proves Location

XAI Evidence Strength Summary

Every report cites evidence sources, confidence level, governing guideline, and counterfactual — per recommendation

All report types

key_drivers field

Identifies which specific tool outputs drove the recommendation

XAI block

confidence_counts

Platform calibrates uncertainty — validated at 82.1% moderate/low on MSS/low-TMB cohort (correct behavior)

XAI block · Table B

Evidence badges

Clinical claims typed and sourced: PATHOGENIC, ACTIONABLE, FDA-APPROVED, PLATFORM-ONLY

Clinical report

counterfactual field

States what would need to change for a different recommendation

XAI block per tool

Traceability How every action is recorded and auditable

6 mechanisms

Component What it proves Location

Tool-grounding

Every report requires a completed tool call chain — architectural guarantee, not model-dependent. 100% on MTBBench n=40

Pipeline · Table B

Guideline attribution

Each tool type mapped to governing standard (AMP/ASCO/CAP, NCCN, IEDB, HIPAA Safe Harbor)

eval harness · Table B

HITL gate

Human review triggered on actionable cases (TMB-High, MSI-H). 7.5% catch rate [0%, 17.5%] — fires selectively, not over-flagging

approve_patient_report

De-id integrity

validate_deidentification called on every report output. HIPAA Safe Harbor 45 CFR 164.514. Architectural guarantee

mcp-deidentify · Table B

HIPAA audit log

Timestamped record of every tool call, patient ID, and outcome. 45 CFR 164.312(b) Security Rule

Pending confirmation

Canonical fixtures + doc-audit

Version-controlled source of truth for all tool outputs; CI scan catches canonical value drift on every PR

tests/fixtures/ · audit.py

Evaluation How results are independently validated

5 mechanisms

Component What it proves Location

Case study accuracy (Table A)

100% with tool outputs vs. 33.3% without on target indications. p < 0.001 Fisher's exact test. 66.7pp improvement

eval/case_study/

MTBBench governance (Table B)

Governance metrics hold across cancer types. n=40 patients, 180 questions. Bootstrap 95% CIs reported

eval/mtbbench/

MTBBench methodological finding

MCP platforms require patient-specific tool outputs for fair accuracy evaluation — contribution to benchmark design

Table A MTBBench

CI regression suite

Case study results are deterministic from canonical fixtures — cannot regress silently between PRs

eval/test_*.py

Scheduled doc-audit

Automated canonical value consistency check — catches drift before it reaches the paper

audit.py · CI + cron