# v2.0 validation summary

The public evidence that each release in the v1.9.x bridge sequence
moved the engine forward on something measurable, packaged for the
v2.0 lock. Compiled from the per-release validation memos, the
v1.9.10 stratified-corpus aggregate, and the v1.9.11 trend table.
Numbers in this document come from the named source: this file is
summary, not new measurement.

## Reading guide

The engine got progressively more validated across v1.9.x, not
progressively more featureful. v1.7 added structural signal
extraction; v1.8 added the graph layer; v1.9.0 added probabilistic
inference; the v1.9.3 → v1.9.14 patches did calibration, stability,
and audit work without changing the inference model.

A v2.0 reviewer should be able to walk through the table below,
cross-check each row against its named source, and confirm the
work documented there matches the schema being locked.

This document does NOT claim recon's outputs got more accurate in
a ground-truth sense. recon is passive and zero-credential;
ground-truth comparison is not part of the operating model. The
claim is that the engine accumulated falsifiable validation
evidence at each release, and the v2.0 schema lock reflects what
survived the validation gates.

## Per-release validation milestones

| Release | Headline measurement | Source |
|---|---|---|
| v1.6 | Hardened-target signal recovery design baseline. Pre-Bayesian; deterministic rule-based fusion + chain-motif library. | `correlation.md` §3, §4.3 |
| v1.7.0 | Cert-side signal layer shipped: wildcard SAN sibling clusters, temporal CT issuance bursts, CNAME chain motifs, cross-source evidence conflicts. Always-emitted; no flag gating. | `CHANGELOG.md` v1.7.0 |
| v1.8.0 | Graph correlation layer shipped: Louvain community detection over CT co-occurrence graph (modularity score reported), per-batch hypergraph ecosystem view (`--include-ecosystem`), vertical-baseline anomaly rules on profiles. | `CHANGELOG.md` v1.8.0 |
| v1.9.0 | Probabilistic Bayesian network shipped behind `--fusion`. 9 nodes, exact variable-elimination inference, 80% credible intervals over high-level claims, n_eff-based interval width tied to evidence sparsity. | `validation/v1.9-validation-summary.md` |
| v1.9.3 | Topology surgery: `email_security_strong` split into `_modern_provider` + `_policy_enforcing` (52.6% spot-check agreement was a definitional gap, not miscalibration). Expanded `federated_identity` parents. | `validation/v1.9.3-calibration.md` |
| v1.9.4 | Hardened-adversarial validation across 50-domain stratified corpus (5 hardening postures). 100% spot-check agreement, 64.0% sparse-flag rate, asymmetric-likelihood design property validated. Failure-mode catalog in `correlation.md` §4.8.10. A/AAAA internal-DNS leak fixed in CNAME chain walker. | `validation/v1.9.4-calibration.md` |
| v1.9.5 | Per-node stability dispositions: criterion (a) evidence-response correctness + criterion (b) Brier/log-score/ECE + criterion (c) N≥10 independent firings. 8 of 9 nodes `stable`; `okta_idp` `not yet` (criterion (c), 7 firings on the v1.9.0 corpus). Parametric regression test pins criterion (a) for every node. | `validation/v1.9.5-stability.md` |
| v1.9.6 | CPT-change discipline (concept first, then parameters): `dkim_present` removed as binding for `email_security_policy_enforcing` after re-examining the definitional question rather than tuning the CPT. `okta_idp` carried over; v1.9.5 `not yet` disposition unchanged. | `validation/v1.9.6-stability-update.md` |
| v1.9.7 | Metadata-coverage gate flipped from advisory percentage to presence-enforcing. 298-detection backfill brought every detection in every category to a non-empty `description` field. CI / pre-commit gate runs the check on every PR. | `CHANGELOG.md` v1.9.7 |
| v1.9.8 | Catalog metadata richness pass. Every detection in every category satisfies the three-signal richness audit (`--report-richness`): description length ≥ 80 chars, scope-narrowing language, canonical vendor `reference` URL. Inline weight rationale on every non-default detection weight. | `validation/v1.9.8-metadata-audit.md` |
| v1.9.9 | Detection-gap UX surfaces: passive-DNS ceiling phrasing, apex-level multi-cloud rollup, wordlist breadth across data/AI/ops/security tiers. 182 new tests across 23 files (catalog-driven Hypothesis + boundary + mutation resistance + render-determinism + adversarial-input fuzz). MCP fallback shadow-load attack on Python 3.10 fully closed. | `validation/v1.9.9-detection-gap-ux.md`, `validation/v1.9.9-corpus-run.md` |
| v1.9.10 | Stratified-corpus pre-lock validation: 60 stratum-tagged synthetic fixtures across 6 cloud strata (GCP, Azure non-O365, Oracle, Alibaba, PaaS, SSE) + 19 base = 79 fixtures total. Per-stratum aggregator. Bayesian network re-validated against the v1.9.9 evidence-distribution shift; v1.9.6 disposition table holds. Cosmic-ray sweep slipped to v2.0 lock. | `validation/v1.9.10-pre-lock.md`, `validation/v1.9.10-bayesian-revalidation.md` |
| v1.9.10.1 | Docs-only patch: GitHub render errors in correlation.md (Mermaid + MathJax) fixed. | `CHANGELOG.md` v1.9.10.1 |
| v1.9.11 | Documentation polish dry-run + v2.0 prep: EXPERIMENTAL labels stripped from user-facing surfaces (CI gate enforces zero), migration guide populated, schema-disposition test green (ecosystem_hyperedges batch-wrapper added to schema), `okta_idp` keep-with-caveat decision, real-corpus runbook + result-shell, trend table source for this summary. | `CHANGELOG.md` v1.9.11, `validation/v1.9.11-trend-table.md` |
| v1.9.12 | Panel-display polish + doctor schema-fields verification + Mermaid evidence-DAG output. No model-level changes; v1.9.10 disposition table preserved. | `CHANGELOG.md` v1.9.12 |
| v1.9.13 | CNAME chain walker hardening pass: entry-point validation, terminus-only A/AAAA check, M365 redirect_domain suffix filter. Third bridge release after a 2026-05-17 scanner re-flag pinned to the v1.5.0 introducing commit. | `CHANGELOG.md` v1.9.13, `docs/security-audit-resolutions.md` |
| v1.9.14 | Revert the v1.9.13 terminus-only A/AAAA check after a follow-up scanner pass showed authoritative DNS can return type-dependent answers, defeating the v1.9.13 safety argument and reopening the v1.9.4 leak. Entry-point validation and M365 redirect_domain filter preserved. | `CHANGELOG.md` v1.9.14, `docs/security-audit-resolutions.md` |

## Post-v1.9.14 work toward the lock (v1.9.15 to v1.9.95)

The per-release table above ends at v1.9.14, where this summary was first
compiled. The bridge sequence continued; the validation-relevant changes since:

- **Calibration legitimacy (Track C-cal, CAL1 to CAL14).** A 2026-06 methodology
  review reframed the deterministic-versus-Bayesian number as *consistency /
  agreement* (near-tautological under the virtual-evidence construction), not
  *calibration*, and the credible interval as *evidence-responsive* rather than
  *calibrated* until frequentist coverage is tested (CAL1, CAL13; see
  `docs/correlation.md` section 4.8.4). Uncertainty is reported on the agreement
  and proxy-label metrics (CAL2), per-node coverage is split from calibration
  (CAL5), entropy reduction is reported as the information-recovery measure
  (CAL10, median about 0.85 nats on the first full-corpus pass), and a
  likelihood-perturbation sensitivity harness shows the calibration is robust to
  the hand-elicited CPTs (CAL8, worst-case ECE shift about 0.03 under +/-20%;
  `validation/cal8-likelihood-sensitivity.md`).
- **CAL14, the one model change since v1.9.6.** v1.9.90 made
  `email_security_policy_enforcing` a *declarative* node: the absence of a public
  DMARC / SPF / MTA-STS declaration is now disconfirming evidence (per-node MAR
  conditioning), with a corpus-grounded prior (0.25 to 0.62) and likelihoods
  (MTA-STS 0.70 to 0.06, a tenfold base-rate correction). The synthetic
  conditional ECE for that node fell from about 0.31 to about 0.03. This is the
  first CPT / topology change since the v1.9.6 binding removal, so the
  "disposition table carries through unchanged" framing below holds only through
  v1.9.14; the email-policy node's disposition is updated by CAL14. Design note:
  `validation/cal14-missingness-design.md`. v2.0 calibration claims gate on the
  maximum per-node conditional ECE, not the mean.
- **Catalog growth (Track C, C1 and C2).** 21 vendor-doc-sourced and 13
  corpus-mined `cname_target` vendors (808 to 841 entries), from local
  live-analysis batches and a full 5,241-domain gap-mining scan; aggregate counts
  and vetted patterns only, no real-company data committed
  (`validation/v1.9.87-c2-corpus-batch.md`).
- **External corroboration spot-check.** A small, optional, anonymized sanity
  check against public vendor case studies (7 of 7 positive detections
  corroborated, 0 false positives, the silences all scope-bounded or
  hidden-by-design; `validation/2026-06-04-case-study-spot-check.md`).
- **Schema hardening (SH1 to SH9, v1.9.91 to v1.9.95).** A four-lens pre-lock
  review reshaped the experimental and v2.0-new fields so the locked surface has
  no known regret (the SH track in `docs/roadmap.md`). The schema being locked is
  the post-SH surface, not the v1.9.14 one.

The latest full-corpus Bayesian pass on record (2026-05-26, pre-CAL14) had all
13,939 high-confidence posteriors agree with the deterministic pipeline and all
nine nodes inside the v1.9.5 stability gates. A CT-enabled full-corpus re-run that
also exercises `infrastructure_clusters` and the cert lexical surfaces, and that
reflects the CAL14 email-policy change on real input, is the remaining
corpus-driven step (Track C, C3). The nine Bayesian nodes are DNS / identity-fed,
so the existing passes calibrate the layer; CT only adds the separate cert
surfaces.

## Per-Bayesian-node trend (synthetic corpus, v1.9.0 → v1.9.10)

The numbers below come from the v1.9.5 stability table
(`validation/v1.9.5-stability.md`) plus the v1.9.10 synthetic
corpus aggregator output
(`validation/synthetic_corpus/aggregate.json`). The v1.9.0
spot-check rates are at-introduction; the v1.9.10 synthetic
counts are firing-on-corpus counts across the 79-fixture set.
The real-corpus run gated by v2.0 produces the truth-of-record
numbers; see `validation/v2.0-corpus-run.md`.

| Node | v1.9.0 spot-check | v1.9.5 stability | v1.9.10 synthetic firings | v2.0 disposition |
|---|---|---|---|---|
| `m365_tenant` | 100% | stable | per-fixture firings track input slugs | stable |
| `google_workspace_tenant` | 100% | stable | per-fixture firings track input slugs | stable |
| `federated_identity` | 100% | stable | parent posteriors flow correctly | stable |
| `okta_idp` | n/a (parent-only at v1.9.0) | not yet (7 firings, < N≥10) | ≥ 8 firings across GCP/Azure/Oracle/SSE strata | stable (with corpus-exposure caveat in YAML) |
| `email_security_modern_provider` | 100% (after v1.9.3 split) | stable | tracks M365 / GWS / gateway slugs | stable |
| `email_security_policy_enforcing` | 100% (after v1.9.3 split) | stable (v1.9.6 binding fix) | tracks DMARC / SPF / MTA-STS signals | stable |
| `email_gateway_present` | 100% | stable | tracks Proofpoint / Mimecast / Barracuda slugs | stable |
| `cdn_fronting` | 100% | stable | fires on Cloudflare / CloudFront / Fastly / Akamai | stable |
| `aws_hosting` | 100% | stable | fires on AWS-family slugs | stable |

## Security closure trail

Per `docs/security-audit-resolutions.md`, the May-2026 audit cycle
closed in the following sequence:

| Topic | Severity | Closed in | Closure note |
|---|---|---|---|
| MCP doctor/install can execute shadowed `recon_tool` package | High | v1.9.9 | Language-level `sys.path` strip in persisted launcher; works on Python 3.10 where `PYTHONSAFEPATH` is a no-op. |
| A/AAAA CNAME validation can trigger internal DNS lookups | Medium | v1.9.4 (initial), v1.9.14 (re-closure after v1.9.13 reintroduction via the terminus-only check on a type-dependent-answer path) | Walker is CNAME-only on every hop. |
| CNAME chain walking can query and leak internal DNS names | Medium | Mitigated: v1.9.3.5 + v1.9.4 + v1.9.13 (entry-point + redirect_domain filter) + v1.9.14 (terminus check reverted). Residual is documented, not closed. | |
| Splunk SIEM example uses unescaped regex slugs | Informational | v1.9.4 | `mvfilter(NOT in(...))` replaces unsafe regex alternation. |

Closure precedence: if a scanner re-flags any of the above against
its introducing commit, `docs/security-audit-resolutions.md` is
authoritative.

## Real-corpus gate

The real-corpus run gated by v2.0 produces the truth-of-record
numbers. Status:

- **`validation/v2.0-corpus-run.md`** shipped as a result-shell in
  v1.9.11 and was populated against the v1.9.14 corpus scan (commit
  `cca815c`, 4264 counted domains): multi-cloud rollup fires on 37.2%,
  the `okta_idp` node clears the criterion-(c) N>=10 threshold by an
  order of magnitude, and the synthetic-versus-real ceiling-firing
  divergence is documented as structural. No `<TO BE POPULATED>`
  strings remain.
- **`validation/v2.0-corpus-run-runbook.md`** is the maintainer recipe:
  scan, aggregate, populate the shell, commit.
- A CT-enabled full-corpus re-run (Track C, C3) that also reflects the
  CAL14 email-policy change on real input is the remaining
  corpus-driven refresh; it needs the maintainer's private corpus and
  live network access. The nine Bayesian nodes are DNS / identity-fed,
  so the populated no-CT and 2026-05-26 passes already calibrate the
  layer.

This summary covers the v1.9.x bridge work. The corpus shell
covers the firing-rate evidence that v1.9.10's synthetic numbers
are a useful proxy for real-world distribution. Both files
should be cross-readable before the v2.0 tag is cut.

## Honest framing

The v1.9.11 trend table observed: "The engine got progressively
more validated, not progressively more featureful." That holds
through v1.9.14. v1.9.12 was display polish, v1.9.13 was a
defensive hardening pass, v1.9.14 was a defensive reversion after
a follow-up scanner pass. None of v1.9.12 / v1.9.13 / v1.9.14
changed the Bayesian network topology, the CPT values, the
stability dispositions, or any user-facing schema field, so the
v1.9.10 disposition table carried through unchanged to v1.9.14.

Past v1.9.14 the discipline held but two deliberate, validated
changes did land, both documented in the post-v1.9.14 section above:
CAL14 (v1.9.90) updated the `email_security_policy_enforcing` node's
missingness handling, prior, and likelihoods, the first model change
since v1.9.6, taking its synthetic conditional ECE from about 0.31 to
about 0.03; and the SH1 to SH9 schema hardening (v1.9.91 to v1.9.95)
reshaped several experimental and v2.0-new fields before the lock. So
the surface v2.0 locks is the post-CAL14, post-SH surface, not the
v1.9.14 one, and every change that reached it carries its own
validation evidence.

A v2.0 review that wants to reconstruct the validation story from
sources should read, in order:

1. `validation/v1.9.4-calibration.md` for the hardened-adversarial
   baseline,
2. `validation/v1.9.5-stability.md` and `v1.9.6-stability-update.md`
   for the per-node stability dispositions,
3. `validation/v1.9.10-pre-lock.md` and
   `v1.9.10-bayesian-revalidation.md` for the stratified-corpus
   pre-lock check,
4. `validation/v1.9.11-trend-table.md` for the v1.6 → v1.9.11
   trajectory,
5. `validation/v2.0-corpus-run.md` for the real-corpus firing-rate
   numbers (populated against the v1.9.14 scan),
6. `docs/correlation.md` sections 4.8.3 and 4.8.4 plus
   `validation/cal14-missingness-design.md` for the CAL14
   node-dependent-missingness change and the CAL1 / CAL13 calibration
   reframing,
7. `validation/cal8-likelihood-sensitivity.md` for the CPT-robustness
   sensitivity check, and `validation/2026-06-04-case-study-spot-check.md`
   for the external corroboration spot-check,
8. the SH track in `docs/roadmap.md` for the pre-lock schema hardening,
9. `docs/security-audit-resolutions.md` for the security-audit closure
   trail.

This summary indexes those documents; it does not replace them.
