# v2.0 real-corpus run results

> **Status: populated.** Aggregator output for the v1.9.14 corpus
> run committed in this revision; see Reproducibility section for
> the source artefact.
>
> Future v2.x patches re-run the same workflow against an updated
> corpus snapshot and replace this content. Re-running is
> idempotent: the aggregator emits counts only (no apex names),
> and the result shell never references private corpus content.

## v2.0 lock-baseline fusion re-run (2026-06-05)

A full-corpus, fusion-enabled re-run on the v1.9.99 build (post-CAL14, post
schema-hardening SH1-SH9), the immediate pre-2.0-lock tip. This pass exercises
the Bayesian layer that v2.0 turns on by default (G2), which the v1.9.14
aggregator run below did not. Aggregate counts only; the run directory and
per-domain JSON stay gitignored under `validation/runs-private/`.

- **Build:** v1.9.99. **Corpus:** 5241 domains; 5238 classified, 3 load errors
  (0.06%); 6 domains refused by the SSRF guard (resolved to private IPs, working
  as designed). No-CT (the nine Bayesian nodes are DNS / identity-fed, so no-CT
  calibrates the layer).
- **Deterministic vs Bayesian consistency:** 100% (13560 / 13560 high-confidence,
  non-sparse firings agree, zero disagreements; 95% Rule-of-Three upper bound on
  disagreement 0.02%). This is consistency under shared evidence, not
  ground-truth calibration (CAL1 / CAL13).
- **CAL14 on real input:** `email_security_policy_enforcing` is the only
  0%-sparse node (declarative, by design). It splits into about 2054 confidently
  very-low (no public DMARC / SPF / MTA-STS, correctly disconfirmed by absence),
  about 2200 high (enforcing), and about 981 moderate. This is the declarative
  behavior the synthetic conditional-ECE improvement (about 0.31 to 0.03)
  predicted, confirmed at scale.
- **Cross-source conflicts:** 0 of 5238.
- **Multi-signal depth:** 74% of domains fire more than one evidence binding.
  Per-domain mean posterior median 0.563 (Q1 0.418, Q3 0.660). Overall
  sparse-flag rate 64.8% (the passive-observation ceiling).
- **Method:** `recon batch <corpus> --ndjson --fusion --no-ct` then
  `validation/analyze_v19_calibration.py`.

The v1.9.14 aggregator firing-rate run below remains the record for the
multi-cloud rollup, ceiling, and `okta_idp` corpus-exposure numbers.

## Run metadata

- **Date:** 2026-05-18 (scan started 00:15:52 UTC; aggregator
  output committed same UTC day)
- **Corpus snapshot:** `20260517-231201Z` under
  `validation/runs-private/` (gitignored). Scan stamp from
  `meta.json`; the timestamp is the moment the wrapper script
  started, scan kickoff is a few minutes later.
- **Build under test:** v1.9.14 (the v1.9.x bridge tip; v2.0
  release candidate per `docs/roadmap.md`)
- **Corpus size:** 4270 domains in the consolidated list;
  4269 reached the aggregator after one duplicate removed by
  the scan wrapper; 4264 counted after 5 load errors (entries
  recorded as `{"domain": ..., "error": ...}` rather than
  classified records). The aggregator's `counted` field is the
  authoritative denominator for the firing rates below.

## Aggregate firing rates (real corpus)

Numbers below are `rendered_fired` / `counted` from the
aggregator output, which corresponds to the end-to-end panel
render rather than the slug-only lower-bound estimator (the
estimator over-fires the ceiling; the renderer is the truth-of-
record per the aggregator's `_notes`).

- **Multi-cloud rollup fires on:** 1585 of 4264 (= 37.2%)
- **Passive-DNS ceiling fires on:** 0 of 4264 (= 0.0%)

## Per-stratum firing rates (real corpus)

The real corpus is consolidated from regional and vertical
files without per-stratum tagging. The aggregator therefore
emits a single `baseline` stratum covering every counted entry.
The synthetic 79-fixture corpus splits across seven strata by
construction (baseline / GCP / Azure / Oracle / Alibaba / PaaS /
SSE); that split is a property of the synthetic fixture
authoring, not the aggregator. The per-stratum comparison below
is read against the synthetic `baseline` row, which is the
nearest analogue.

| Stratum | counted | multi-cloud fires | rate | ceiling fires | rate |
|---|---|---|---|---|---|
| `baseline` | 4264 | 1585 | 37.2% | 0 | 0.0% |

## Comparison to synthetic-corpus shape

The synthetic 79-fixture corpus
(`validation/synthetic_corpus/aggregate.json`, v1.9.11
re-aggregation) reports:

- Multi-cloud 29.1% fired, ceiling 86.1% fired overall.
- Per-stratum (10 fixtures each except baseline=19): baseline 42% /
  GCP 30% / Azure 10% / Oracle 30% / Alibaba 10% / PaaS 70% /
  SSE 0% multi-cloud.

The real-corpus shape should be **within ±15 percentage points
per stratum** for the synthetic corpus to count as a useful
proxy. Outside that envelope, the runbook asks for the
divergence and the hypothesis for why.

**Multi-cloud — within envelope.** Real-corpus baseline 37.2%
vs synthetic baseline 42.1% is a divergence of -4.9 percentage
points, well within the ±15pp envelope. The synthetic baseline
slightly over-represents multi-cloud relative to the consolidated
real corpus, plausibly because synthetic baseline fixtures are
hand-authored to exercise cross-vendor evidence and the real
corpus is drawn from the long tail where single-vendor
deployments dominate. Same direction as the v1.9.12 run from
2026-05-15 (37.0% baseline multi-cloud), confirming the v1.9.13 /
v1.9.14 CNAME-walker changes did not perturb panel-level firing
rates.

**Ceiling — outside envelope, structural reason.** Real-corpus
0.0% vs synthetic 57.9% baseline (86.1% overall) is a -57.9pp
divergence and is structural rather than a calibration miss.
The passive-DNS ceiling fires only when *all four* trigger
conditions hold: `services` non-empty, `domain_count >= 3`,
`categorized_count < 5`, and `surface_attributions count < 5`
(see `validation/corpus_aggregator.py:_ceiling_fired`). The
synthetic fixtures are sparse by construction: most have only a
handful of slugs and a handful of surface attributions, so the
last two conditions hold trivially. Real apexes from a 4270-
domain corpus tend to have rich service signatures and many
surface attributions, so `categorized_count >= 5` or
`surface_attributions count >= 5` holds and the ceiling is
suppressed. The 0% real-corpus rate is the renderer correctly
suppressing the ceiling on rich apexes; it is not a missed
detection. This matches the v1.9.12 run shape (also 0%).

The synthetic corpus remains a useful proxy for multi-cloud
firing-rate calibration. It is not a useful proxy for ceiling
firing-rate calibration; the v1.9.10 stratified-corpus design
note already calls out that ceiling-firing is dominated by
fixture density, which is a synthetic-corpus authoring
parameter rather than a real-world distribution.

- **Per-stratum divergence vs synthetic:** multi-cloud
  baseline -4.9pp (within envelope, no action); ceiling
  baseline -57.9pp (outside envelope, structural — synthetic
  fixture sparsity differs from real-corpus richness, by
  construction).

## `okta_idp` corpus-exposure verdict

The v1.9.5 stability table marked `okta_idp` as `not yet` because
the v1.9.0 corpus firing count was 7 (below N≥10 threshold).
v1.9.10's synthetic stratified corpus brought the synthetic
firing count above 10, but only the real corpus produces the
authoritative criterion-(c) clearance number.

- **`okta_idp` firing count on real corpus:** 134 (counted from
  the `okta` slug, the IdP-tier signal that maps onto the
  `okta_idp` Bayesian-network node). 543 if the lower-tier
  `okta-sso-hub` signal is also counted; either number clears
  the N≥10 threshold by an order of magnitude.
- **Disposition confirmed for v2.0:** keep. The v1.9.11
  keep-with-caveat decision is supported by the real-corpus
  number (134 strict firings) without any caveat language being
  load-bearing for criterion (c). The corpus-exposure caveat in
  `recon_tool/data/bayesian_network.yaml` is retained as a
  reading aid for v2.x readers rather than as a stability
  qualifier.

## Reproducibility

The run shell above was populated from
`validation/runs-private/20260517-231201Z/aggregate.json`,
emitted by `validation/corpus_aggregator.py` against
`results.json` (converted from the wrapper's NDJSON output by
stripping the header line and joining the per-domain records
into a JSON array). The aggregator's behaviour is pinned by
`tests/test_corpus_aggregator.py`. The `okta_idp` firing count
was computed directly from `results.json` by counting entries
whose `slugs` list contains the `okta` slug (the IdP-tier
signal; lower-tier `okta-sso-hub` count reported separately
for transparency).

Per-domain JSON, gaps, candidates, and the run directory itself
stay gitignored under `/validation/runs-private/`. The
aggregator-emitted counts are the only artefact that lands in
this committed file.

Future v2.x patches re-run the same workflow against an updated
corpus snapshot.
