# 2026-05-20 — Plan: addressing Gemini's Monte Carlo methodology critique

## Context

Gemini reviewed two napkin_math outputs from snapshot 49:

- `20251114_paperclip_automation` — small industrial pilot. Surfaced
  five flaws in extraction, bounds, sampling, and rationale-citation
  fidelity.
- `20260516_datacenter_in_france` — 9 GW hyperscale infrastructure
  megaproject. Surfaced five further findings, two of which extend
  the first round. This plan scored **100% Robust** in the assessment
  despite being a €100B+ sovereign-infrastructure project with
  multiple existential regulatory dependencies.

This document maps each finding to the `.claude/skills` (and the
Python infrastructure) that has to change, and identifies the new
skill(s) the gap analysis surfaces. Findings are grouped by theme;
each is tagged R1 (paperclip) or R2 (datacenter).

### Findings inventory

**Round 1 — paperclip pilot:**

- R1.1 — *Disconnected aggregates.* Total cost drawn independently of
  sub-component costs; a single trial can pair sub-component p95s
  with a total p05.
- R1.2 — *Base = target fallacy.* `generate-bounds` centers `base` on
  the commitment value. With `low/base/high = 1.0/2.0/4.5`, the
  triangular CDF guarantees a fail-loaded distribution; the 29.3%
  gate-pass figure is a property of the bound shape, not the plan.
- R1.3 — *Independence.* Runner samples every variable independently;
  inflation can land at p95 in the same trial labor cost lands at p05.
- R1.4 — *Bounded vs. fat-tail.* Triangular `high` is a hard ceiling;
  0% of runs see a $700k overrun on a $500k project.
- R1.5 — *Hallucinated citation.* Rationale cited "Risk-5
  grading-protocol variance" in a plan whose Risk 5 is about
  supply-chain delays.

**Round 2 — 9 GW datacenter:**

- R2.1 — *Megaproject illusion.* Existential regulatory gates (RTE
  grid upgrade, DGSI security clearance, DREAL zoning) are correctly
  placed in `unmodelled_gates`, so the Monte Carlo evaluates only the
  benign budget variables. Headline reads 100% Robust, slideshow
  caveat invisible. The verdict is honest about its scope but framed
  in a way that lets the scope go unread.
- R2.2 — *Megaproject distributions.* Same root issue as R1.4, but
  the fix differs by `plan_type`. Hyperscale / geopolitical
  megaprojects must use lognormal CAPEX by default (Flyvbjerg's iron
  law); a $300k paperclip pilot does not.
- R2.3 — *Time-cost covariance.* Holding cost is named at €257.6M/yr
  for a 128.8 km² buffer; RTE delays could push the timeline 4 years
  out. Extract emitted `actual_holding_cost_eur` as a flat bounded
  variable rather than the deterministic relation
  `holding_cost = annual_burn × actual_delay_months / 12`.
- R2.4 — *Missed FX master equation.* Assumption Q5 explicitly states
  30% of CAPEX is USD-denominated and 70% hedged, with a 10% FX shift
  worth €4.32B. Extract emitted `phase1_capex_surplus` as a single
  bounded variable rather than decomposing into civil_works +
  remediation + hardware_eur + hardware_usd / fx_rate.
- R2.5 — *Threshold trivialization.* `buildable_area_km2` gate is
  `>= 0` (the model-defined default) when the plan states 32.2 km² is
  needed to physically support 9 GW. DREAL forcing the buffer to 20
  km² is the actual failure scenario; the `>= 0` gate never tests it.

A chunk of R2 (R2.3 holding-cost burn × delay, R2.4 FX × CAPEX) looks
like a correlation problem on first read but is actually a
decomposition problem: the plan names the deterministic equation
explicitly, and the extract should preserve it. The two themes are
separated below.

## Status as of 2026-05-21

### Landed on main

- **PR #737** (merged) — Phase 1 compress prompts + initial extract threshold-pairing rule + `OPTIMIZE_INSTRUCTIONS` discipline banner. Substantive content described below under "PR #737 detail" for continuity.
- **PR #739** (merged) — Proposal 141 ("Source-Preservation Audit for the Napkin Math Pipeline") landed as design only. No code or prompt change. Implementation deferred.
- **PR #740** (merged) — Phase 2 extract-prompt rules. Four commits landed in `extract-parameters-from-digest` and `extract-parameters-from-full`:
  - `4cda70ba` — Source-arithmetic preservation rule (Patterns 1/2/3: aggregate sum, burn rate × duration, explicit decomposition block) + threshold-pairing parity backfill into the full-extract skill.
  - `19f927b7` — Tightened aggregate-sum wording so independent caps/envelopes are NOT collapsed into derived sums; reconciled discipline-shared paragraph with the cap-pressure paragraph.
  - `8f94c8cd` — 20-word `source_text` cap reinforced with explicit truncation discipline (drop the consequence clause, end with ellipsis if mid-sentence).
  - `f9d90ebb` — Updated this plan-status section for PR #740's narrow scope and verification limits.
  All edits applied symmetrically to both extract skills. No corpus literals introduced.

### PR #737 detail (already on main)

- **Compress — `compress_report_section.py`**
  - `NUMERIC_VALUES_BUCKET_PROMPT` tightened so labels for rate-like
    quantities carry their per-period or per-unit denominator and
    source-stated rates land in `numeric_values`, not
    `missing_data` (R2.3, partial R1.1).
  - `MISSING_DATA_BUCKET_PROMPT` extended with the burn-rate /
    duration pairing rule (R2.3).
  - `GATES_AND_THRESHOLDS_BUCKET_PROMPT` extended so capacity
    requirements count as gates even when the source frames them as
    sizing math (R2.5).
  - New module-level `OPTIMIZE_INSTRUCTIONS` block: pipeline context,
    anti-overfitting categories, "regression probes are not acceptance
    criteria" discipline, and the corpus-leak ban that holds the
    banner itself to its own rule. Pattern modelled on
    `identify_potential_levers.py`.
- **Extract — `extract-parameters-from-digest/system-prompt.txt`**
  - Threshold-pairing rule added between "Coverage and capacity gate
    rule" and "Combined viability gate preservation": every extracted
    threshold (floor, cap, ceiling, target volume / share / deadline)
    must be paired with a realised-vs-threshold margin/surplus calc
    in `recommended_first_calculations` or `derived_questions`. Cap
    pressure resolves by dropping the least load-bearing key_value
    or moving a calc to `derived_questions`, never by skipping the
    pairing.

All five edits are corpus-agnostic. The baseline plans the prompts
were stress-tested against (a growing subset of the napkin_math
14-plan corpus) appear nowhere in the prompt text — they are
regression probes, not acceptance criteria.

### PR #740 verification posture

Compress + extract were re-run on a subset of the baseline including
paperclip, datacenter, crate_recovery, yellowstone, mars_gtld, and
euro_adoption digests. v51 outputs were produced fresh via the skill
under the updated prompts (no hand-patching). All six v51
`parameters.json` files validate with `valid=True, errors=0,
warnings=0` against `validate_parameters.py`.

**This is a regression check, not an improvement claim.** Same-LLM
same-session regeneration cannot prove behavioural improvement on
the rules being tested; the LLM that wrote the rules is also
applying them. Honest behavioural verification of these rules
requires a different LLM or fresh-context same-model extraction
against the same digests. That work is not in PR #740.

Behaviours that moved in the expected structural direction across
multiple probes (regression-check evidence only):

- Per-period denominator preservation in `numeric_values` labels and
  in `missing_data` rate / duration pairings.
- Datacenter's R2.5 capacity-as-gate (32.2 km² / 9 GW) lifted from
  numeric_values prose to a gates_and_thresholds entry in
  `compress_review_plan.md`.
- Threshold-pairing rule cleanly catches both the explicit failures
  Gemini flagged (`buildable_area_surplus`-style margin variables
  for the datacenter) and new ones discovered on other probes
  (crate volume target + donation floor; yellowstone hospital
  fuel-priority floor).
- Source-arithmetic preservation: mars pledge calc splits per
  Pattern-3 (`year1_revenue_usd → year1_revenue_after_pledge_usd →
  year1_viability_surplus_usd`) instead of collapsing into one
  combined surplus.
- Tightened aggregate-sum: no probe's independent budget cap was
  collapsed into a sum-of-allocations identity.

### Known limitations PR #737 and PR #740 do NOT address

Explicitly out of scope for both PRs. None should be bundled into
either; each gets its own follow-up:

- **Compress-LLM run-to-run variance.** Same prompts, same source,
  two compress passes can produce materially different bucket
  selections. The paperclip premortem tripwires (`$75k OPC UA bid`,
  `100ms` p99 latency, both in v49) drop at the compress stage in
  v50 and v51 and cannot be recovered at the extract layer because
  the digest itself does not surface them. This is the **clearest
  unresolved regression** across the probe set. The fix belongs in
  orchestration: deterministic retry/merge across N compress
  passes, or lower-temperature reruns for high-impact buckets.
- **Threshold-pairing rule × `missing_values_to_estimate` 5-cap.**
  When a plan names many independent thresholds, every-threshold
  pairing collides with the cap and forces a tradeoff. The
  "less load-bearing key_value to drop" rubric is judgement-based;
  the rule does not give a quantitative criterion. Concrete
  artifact: yellowstone's `public_compliance_*` trio was dropped
  under cap pressure in v50/v51 with the rationale recorded in
  chat but NOT in the parameters.json itself — which is exactly
  the gap proposal 141's `dropped_signals` schema is designed to
  close.
- **No formal source-preservation audit implementation.** Proposal
  141's design is on main; the implementation (Python audit script,
  `dropped_signals` schema in extract prompts) is not. Until that
  lands, v49 → v51 absences cannot be mechanically classified as
  "rename" vs "structural restructure" vs "acceptable drop" vs
  "silent regression". The audit script and schema field are the
  next-most-load-bearing piece of work.
- **Behavioural verification of the new rules on a different LLM.**
  PR #740's v51 regression check is same-LLM same-session; it
  proves the rules do not break valid extraction, not that they
  improve the behaviour of a fresh-context or different-model
  extraction. A separate validation run with the napkin_math
  pipeline driven by its default LLM (Gemini Flash Lite via
  Self-Improve) against these prompts would close that gap.
- **Prompt-hygiene cleanup of older domain-specific examples.** One
  domain-specific id (`european_prepper_active_buyers`) still
  appears in the extract-from-digest system prompt as a formula
  example; identified during PR #737 review, deferred. A
  scoped prompt-hygiene pass should replace it with an abstract
  placeholder.

### Process insights from the v50 work

These are written into the prompt-file `OPTIMIZE_INSTRUCTIONS`
blocks so they survive future prompt edits, but worth surfacing here
too:

- **Fix prompts, not outputs.** Hand-patching a `parameters.json` to
  add a missing margin reproduces the bug on the next run. Edit the
  prompt rule, re-invoke the skill, audit the natural output.
- **Regression probes ≠ acceptance criteria.** Picking a rule's
  shape from one plan's behaviour without testing across multiple
  probes is overfitting in a subtler form. Diagnose abstract
  patterns; verify the rule moves multiple probes; surface
  tradeoffs explicitly when a rule lifts one plan but flattens
  another.
- **Per-file `OPTIMIZE_INSTRUCTIONS` block.** Each prompt-engineering
  scoped file should carry its own discipline banner (modelled on
  `identify_potential_levers.py:27` and now
  `compress_report_section.py:52`). The banner lists categories of
  forbidden content abstractly — never concrete corpus literals,
  which would themselves be a leak.

### Phase status against the original plan

| Phase | Skill / module | Status |
|---|---|---|
| 1 | `compress_report_section.py` | **DONE on main via PR #737** (R2.3 numeric_values, R2.3 missing_data, R2.5 gates_and_thresholds, OPTIMIZE_INSTRUCTIONS banner) |
| 2 | `extract-parameters-from-{full,digest}` | **DONE for prompt-side directives on main via PR #740** — threshold-pairing on `from-digest` shipped in PR #737; source-arithmetic preservation (Patterns 1/2/3 for R1.1, R2.3, R2.4), threshold-pairing parity into `from-full`, aggregate-sum tightening, and source_text truncation discipline shipped in PR #740. Behavioural validation on a different LLM remains a follow-up, not additional prompt-scope work. |
| 3 | `validate-parameters` | not started for the no-dead-end / threshold-pair extensions in the plan. Note: `validate_parameters.py` itself exists and was used to validate v51. |
| 4 | `generate-bounds` | not started |
| 5 | `verify-bounds-citations` (new) | not started |
| 6 | `generate-calculations` | no change required per the original plan |
| 7 | `run-scenarios` | not started |
| 8 | `run_monte_carlo.py` | not started |
| 9 | `summarize-assessment` | not started |
| 10 | Documentation | active (this status section) |

### Next likely move

After PR #740, the next work should be ordered by what improves
napkin_math output quality most directly, not by what is easiest to
measure. Preferred order:

1. **Compress-LLM variance handling.** Deterministic retry/merge or
   lower-temperature reruns for high-impact compress buckets should
   come next. The clearest driver is the paperclip OPC UA / latency
   tripwires that v49 surfaced and v50/v51 drop at the compress
   layer. This is upstream of extraction: if the digest does not
   carry the tripwire, no extract prompt can recover it. Proposal
   141 would classify this loss, but variance handling is the piece
   that can restore the missing source signal.
2. **Implement proposal 141** (`dropped_signals` schema in extract
   prompts + `audit_source_preservation.py` deterministic script).
   This should follow close behind variance handling. It is the
   right guardrail for v49/v51 absences and cap-pressure tradeoffs,
   but it is primarily a measurement and accountability layer: it
   classifies preserved / replaced / dropped signals and records
   rationale in the artifact. It does not by itself make the
   compressor less lossy.
3. **Different-LLM behavioural validation** of the rules now on
   main. A Self-Improve run with the default napkin_math LLM
   (Gemini Flash Lite) against the same digests would close the
   same-LLM same-session confound. This should be treated as
   validation of prompt generality, not as the next quality fix.
4. **Prompt-hygiene pass** for the remaining domain-specific
   examples (e.g. `european_prepper_active_buyers`) in either
   extract prompt. This is worthwhile and small, but not
   load-bearing for the currently observed napkin_math failures.

These are separate PRs. The next PR should be compress variance only:
no corpus literals, no hand-patched outputs, rerun compress + extract
through the skills, validate regenerated `parameters.json`, and
compare against v49/v50/v51 honestly. Bundling the audit,
behavioural validation, or prompt hygiene into that PR would obscure
whether the upstream signal-loss fix actually worked.

## Per-theme mapping

### Theme A — Decomposed equations: aggregates, burn rates, FX

Covers R1.1, R2.3, R2.4. All three are instances of the same upstream
defect: the extract stage emitted a flat bounded variable when the
report itself names a deterministic relationship between
sub-components.

- R1.1 — `total = A + B + C + …`
- R2.3 — `derived_cost = burn_rate × duration`
- R2.4 — `cost_eur = hedged × historical_rate + unhedged ×
  simulated_rate` (or the inverse for USD-denominated hardware)

**Pipeline-stage routing:**

- R1.1 — primary fix in the extract stage. The total/constituent
  link cuts across sections; compress sees one section at a time and
  cannot reliably detect it.
- R2.3 — fix spans **compress** AND extract. The €257.6M/yr holding
  cost surfaces in the Review Plan section, the matching delay
  duration surfaces in Review Plan or Premortem. If `compress_report_section.py`
  flattens "€257.6M/yr" to "€257.6M holding cost", the rate × duration
  opportunity is lost before extract sees it.
- R2.4 — the FX master equation is in **Assumption Q5**, which is a
  **passthrough** section, NOT compressed. `compress_report_section.py`
  is not in the loop for R2.4; the fix lives entirely in the extract
  stage, which reads Q5 verbatim.

**Skills / modules to change:**

- `compress_report_section.py` (Stage 1, worker_plan) — two prompt
  edits to preserve burn-rate structure for R2.3:
  1. `NUMERIC_VALUES_BUCKET_PROMPT` — require the `label` to include
     the per-period unit (`/yr`, `/mo`, `/km²`, `/FTE`) when the
     source states a per-period rate, and require the `modelling
     role` to read like `burn rate scaling with <X>` so downstream
     code can detect the multiplication.
  2. `MISSING_DATA_BUCKET_PROMPT` — extend the denominator-pairing
     rule (lines 605–624) with: "a per-period burn rate needs the
     matching duration the rate will be multiplied by".
- `extract-parameters-from-full`, `extract-parameters-from-digest` —
  system prompts must enumerate the patterns to detect:
  - "total cost / total budget" alongside named constituents → declare
    total as `recommended_first_calculations` with
    `formula_hint = A + B + C + …`.
  - "per-year", "per-month", "annual burn", "monthly holding",
    "PMO overhead" alongside a variable duration → declare the
    dependent cost as `burn_rate * duration` and bound the duration,
    not the cost.
  - Explicit assumption blocks doing the math (e.g., Assumption Q5
    "30% USD, 10% shift, €4.32B impact") → preserve the decomposition
    as a calculation; the LLM MUST NOT collapse the explicit
    decomposition into a single flat variable.
- `validate-parameters` — extend the structural checks:
  - A variable that is the LHS of a sum-formula in another entry MUST
    NOT also appear in `missing_values_to_estimate`.
  - A variable whose id paraphrases "total", "sum", "aggregate",
    "holding_cost", "capex_eur" warns when bounded independently of
    any declared formula. Same backstop pattern as
    `strip_threshold_bounds`.
- `generate-bounds` — extend `strip_threshold_bounds` to also strip
  bounds for any variable declared as a calculation LHS (today it
  only strips threshold-named variables).
- `generate-calculations` — no code change; if `formula_hint` is
  declared correctly upstream, the existing code generator implements
  the sum / product correctly.

### Theme B — Base anchoring: realistic vs. committed

Covers R1.2. The actual-vs-commitment default rule in
`generate-bounds/system-prompt.txt` overcorrects: it forces `base` to
the commitment value regardless of whether the plan itself names a
likely shortfall.

**Skills to change:**

- `generate-bounds` (system-prompt.txt) — rewrite the
  actual-vs-commitment paragraph as a conditional rule:
  - If a Premortem / Expert Criticism / Risk-register passage
    forecasts a gap between commitment and reality, anchor `base` on
    the plan-internal gap estimate, cite it, tag `source: "data"`.
  - Otherwise fall back to the commitment value AND tag
    `source: "assumption"` (today the rule silently emits `data`).
  - Make the asymmetry visible in the rationale.
- `summarize-assessment` — surface in
  *Missing inputs ranked by impact* whether `base` was anchored on a
  report passage or on the commitment default. The reader currently
  cannot tell.

Most likely to swing gate verdicts on existing plans; run Self-Improve
against the 14-plan baseline before shipping.

### Theme C — Cross-variable correlation

Covers R1.3. Runner declares `correlation_groups: []` in
`DEFAULT_SETTINGS` (`run_monte_carlo.py:35`), reads it from user
settings (line 101), and never references it during sampling. The
monte-carlo SKILL.md acknowledges the gap at line 70.

**Skills to change:**

- `generate-bounds` — extend output schema with an optional
  `correlations` block (sibling of the per-variable entries). Declared
  by the LLM based on plan-internal logic ("if PLC is hostile, OPC UA
  bid is high AND manual-intervention hours are high").
- `monte-carlo` (runner code) — implement Gaussian-copula sampling.
  Rank-correlated uniforms for variables in a group fed through each
  variable's inverse-triangular (or inverse-lognormal). Inputs not in
  any group stay independent. ~80 lines.
- `summarize-assessment` — disclose the independence assumption in
  the slideshow when no correlations are declared; surface declared
  groups when they are.

**No new skill** for correlations. Folding them into `generate-bounds`
keeps the artifact count down.

Note: fix Theme A first. A chunk of what reads as correlation
(holding cost ↔ delay; CAPEX ↔ FX rate) collapses into deterministic
decomposition once the equations are restored. The remaining
correlation set is smaller and easier to declare honestly.

### Theme D — Distribution choice and fat tails

Covers R1.4 and R2.2. Triangular truncates at `high`, which is wrong
for R&D / integration cost (R1.4) and catastrophically wrong for
megaproject CAPEX (R2.2).

**Skills to change:**

- `generate-bounds` — add `sampling_discipline: "lognormal"` and
  `"pert"` to the enumeration. System prompt documents when each
  applies:
  - **Per-variable rule:** cost / duration variables tied to
    integration, R&D, legacy hardware, or "scope creep" passages →
    lognormal.
  - **`plan_type`-driven default:** if
    `parameters.plan_summary.plan_type` matches
    `hyperscale_infrastructure`, `geopolitical_megaproject`, or
    similar, ALL CAPEX and major OPEX variables default to lognormal
    unless the rationale explicitly justifies triangular. Anchor on
    Flyvbjerg's iron law of megaprojects. For
    `industrial_automation_pilot` and `small_scale_logistics`,
    triangular stays the default.
  - For lognormal, `low / high` mean P5 / P95, not hard cutoffs.
- `monte-carlo` (runner code) — implement lognormal and PERT
  samplers. Triangular stays default. Update `VALID_DISCIPLINES`.
- `run-scenarios` — policy decision. Scenarios evaluate at
  `low / base / high` deterministically; for a lognormal that means
  P5 / median / P95. Either run-scenarios learns the new disciplines
  or `generate-bounds` is constrained to triangular for variables
  that also have a scenario row. Simplest first cut: scenarios stay
  triangular-only; the LLM may not select lognormal for any variable
  with a scenario row.
- `validate-parameters` — extend the discipline-allowed list.
- `summarize-assessment` — disclose in the slideshow when any
  variable uses a fat-tail discipline; cite it as a strength, not a
  footnote. Also disclose `plan_type` and the discipline default it
  implies.

### Theme E — Rationale citation verification

Covers R1.5. The existing self-audit in `generate-bounds/SKILL.md:30`
says citations must "substantively support the claim". The LLM didn't
catch a clean context leak.

**Skills to change:**

- `generate-bounds` — strengthen the self-audit instruction with
  concrete failure examples (context-leak: citation refers to a Risk
  that exists by number but is about a different topic).

**New skill (or new step in `validate-parameters`):**

- **`verify-bounds-citations`** — deterministic Python, no LLM call.
  Parse Risk N / Issue N / Decision N tokens from each rationale
  string in `bounds.json`. For each citation, fetch the corresponding
  section from the source report. Compare topical keywords from the
  rationale against the cited section. Fail loud when topical overlap
  is below threshold. Same posture as `SchemaError`. Re-run
  `generate-bounds`. Start with substring / shared-stem matching;
  tighten if needed.

### Theme F — Threshold extraction quality

Covers R2.5. The `buildable_area_km2` gate landed as `>= 0`
(model-defined default) when the plan states 32.2 km² is required for
9 GW. DREAL forcing the buffer to 20 km² is the actual failure
scenario; the gate never tests it.

Two upstream failure modes possible:

1. Extract did not declare a margin variable
   `buildable_area_surplus = actual_buildable_area_km2 - 32.2`. The
   actual area was bounded; the requirement was lost.
2. Extract declared the variable but defaulted to `>= 0` because the
   formula had not subtracted the requirement.

A third upstream failure mode is possible too: compress's
`gates_and_thresholds` bucket may not have surfaced "32.2 km² required
for 9 GW" as a gate, because the source states it as a capacity
calculation rather than an explicit pass/fail condition. If extract
never saw it as a gate, the threshold defaulted to `>= 0`.

**Skills / modules to change:**

- `compress_report_section.py` (Stage 1, worker_plan) — extend
  `GATES_AND_THRESHOLDS_BUCKET_PROMPT` with: "Capacity requirements
  count as gates. When the source states a numeric minimum needed to
  physically support a declared capacity ('32.2 km² to support 9 GW',
  '11 GVA grid headroom for Phase 1'), the requirement IS the gate.
  Write it as `If <quantity> falls below <required value>, then
  <consequence>` even when the source does not frame it that way
  explicitly."
- `extract-parameters-from-full`, `extract-parameters-from-digest` —
  system prompts must require: when the plan states an explicit
  requirement for a capacity variable, the gate must be either
  `>= <requirement>` (explicit-value threshold) OR the variable must
  be re-expressed as a margin `actual - requirement` with a `>= 0`
  gate. The model-defined `>= 0` default may apply only when the
  underlying formula has already subtracted the requirement.
- `validate-parameters` — add a sanity check: if a `key_value`
  declares a numeric requirement (e.g., a `*_required` companion
  variable) and a margin / surplus variable is NOT declared, warn.
  The existing threshold-friendly naming rule is the hook to extend.
- `summarize-assessment` — surface `threshold_basis` more
  prominently. A `model_defined / >= 0` gate next to a value-bearing
  requirement in the same plan is a yellow flag, not a pass.

### Theme G — Overall verdict aggregation: the megaproject illusion

Covers R2.1, the most consequential new finding. The 9 GW datacenter
scored 100% Robust because every existential gate (RTE grid, DGSI,
DREAL) was correctly identified as binary regulatory and correctly
placed in `unmodelled_gates` — and then the headline verdict ignored
them entirely. The slideshow disclaimer
("unmodelled_gates qualify the modelled verdict but never enter the
simulation") is honest but easily missed, especially on a slide that
reads "100% pass rate" in large type.

**The fix is a verdict-aggregation change, not a sampling change.**
The Monte Carlo is producing the right number for the modelled-gates
question; the headline is asking the wrong question.

**Skills to change:**

- `summarize-assessment` — implement a composite risk band:
  - Today: `overall_risk_band` = band of the worst declared gate.
  - Proposed: `overall_risk_band` is capped by `unmodelled_gates`
    count. Suggested rule (calibratable):
    - 1–2 unmodelled gates → cap at Robust unchanged.
    - 3–4 unmodelled gates → cap at Marginal.
    - 5+ unmodelled gates → cap at Fragile.
  - The cap is monotonic: more unmodelled binary risks can only make
    the headline worse, never better.
  - Add a new manifest field `unmodelled_gate_penalty` naming exactly
    what was applied and why.
  - Slideshow text changes: "100% Robust (modelled gates), capped at
    Marginal due to 4 unmodelled existential gates" — verdict honest
    about its scope.
- `extract-parameters-from-full`, `extract-parameters-from-digest` —
  unchanged; they already populate `unmodelled_gates` correctly. The
  defect is downstream of extraction.
- `methology.md` and README — document the cap rule. Load-bearing
  change to the verdict semantics.

A structurally cleaner alternative is to lift existential gates into
the simulation as Bernoulli draws (probability of RTE approval ×
probability of DGSI clearance × …). That requires the LLM to assign
survival probabilities, which is a new source of error. **The cap
rule is the safer first cut**; Bernoulli-modelling existential gates
is a follow-up to pursue if the cap turns out to be too blunt to
distinguish plans that differ only in unmodelled-gate severity.

## Skill inventory summary

### Changed skills

| Skill / module | Findings | Type of change |
|---|---|---|
| `compress_report_section.py` (Stage 1, worker_plan) | R2.3, R2.5 | Prompt edits: per-period unit preservation in `numeric_values`; burn-rate / duration pairing in `missing_data_to_estimate`; capacity-requirement detection in `gates_and_thresholds` |
| `extract-parameters-from-full` | R1.1, R2.3, R2.4, R2.5 | System prompt: aggregates / burn rates / FX decomposition / explicit-requirement thresholds |
| `extract-parameters-from-digest` | R1.1, R2.3, R2.4, R2.5 | Same as above |
| `generate-bounds` | R1.1, R1.2, R1.3, R1.4, R1.5, R2.2 | System prompt + schema (correlations, lognormal / PERT, plan_type-driven defaults); self-audit examples; strip aggregates |
| `generate-calculations` | — | No change |
| `validate-parameters` | R1.1, R1.4, R2.2, R2.5 | New structural checks (aggregate-not-bounded, expanded discipline enum, requirement-margin pairing) |
| `monte-carlo` (runner) | R1.1, R1.3, R1.4, R2.2 | Strip aggregate bounds; copula sampler; lognormal / PERT samplers |
| `run-scenarios` | R1.4, R2.2 | Policy decision on lognormal / PERT semantics |
| `summarize-assessment` | R1.2, R1.3, R1.4, R2.1, R2.2, R2.5 | Disclose anchoring / correlations / discipline / plan_type / threshold-basis; implement composite-risk-band cap |
| `run-napkin-math-pipeline` | R1.5 | Orchestrate the new verifier between bounds and calculations |
| `test-napkin-math` | R1.1, R1.3, R1.4, R1.5, R2.1, R2.2, R2.3, R2.4, R2.5 | Smoke fixtures across every theme |
| `methology.md` / README | R2.1 | Document the composite-band cap rule |

### New skills

- **`verify-bounds-citations`** — deterministic post-processor that
  audits `bounds.json` rationale citations against the source report.
  Fail-loud, re-run-`generate-bounds` posture. Plugs into
  `run-napkin-math-pipeline` between Stage 5 (`generate-bounds`) and
  Stage 6 (`generate-calculations`).

I do **not** recommend separate skills for correlations, burn-rate
detection, FX decomposition, or composite-band aggregation. These all
sit inside existing stages with prompt and schema changes only.

## Implementation order

Upstream-first. Each phase ships a focused set of changes, the
napkin-math pipeline is re-run end-to-end against the 14-plan
baseline, and the next phase only starts once the previous phase's
output looks honest. Build order follows pipeline order (Stage 1 →
Stage 8) so each downstream stage sees inputs that already encode the
new structure.

The ROI argument that R2.1 (composite-band cap, the "100% Robust
datacenter" fix) is the loudest single failure is acknowledged but
deferred: shipping R2.1 first means calibrating the cap rule against
still-broken upstream data. Build from the bottom up; the cap rule
lands at Phase 9 against assessments whose modelled-gate verdicts are
already correct.

### Phase 1 — Compress stage (`compress_report_section.py`)

**Changes**

- `NUMERIC_VALUES_BUCKET_PROMPT`: preserve per-period units (`/yr`,
  `/mo`, `/km²`, `/FTE`) in the label; require the modelling role to
  read like "burn rate scaling with <X>". (R2.3)
- `MISSING_DATA_BUCKET_PROMPT`: extend the denominator-pairing rule
  with "a per-period burn rate needs the matching duration". (R2.3)
- `GATES_AND_THRESHOLDS_BUCKET_PROMPT`: capacity requirements count
  as gates even when the source frames them as capacity calculations.
  (R2.5)

**Verification gate**

- Re-run the 24 compress calls (4 sections × 6 buckets) against the
  paperclip and datacenter plans.
- Spot-check the digests: holding cost surfaces as `€257.6M/yr` (not
  `€257.6M`); `32.2 km² for 9 GW` surfaces in `gates_and_thresholds`,
  not buried in capacity prose.
- Regenerate snapshot fixtures; no compress smoke regressions.

### Phase 2 — Extract stages (`extract-parameters-from-{full,digest}`)

**Changes**

- Aggregates declared as `recommended_first_calculations`, not
  `missing_values_to_estimate`. (R1.1)
- Per-period rates trigger `derived = burn_rate * duration`
  calculations; bound the duration, not the dependent cost. (R2.3)
- Explicit decomposed assumption blocks (FX Q5 and similar) preserved
  as calculations rather than collapsed into a flat variable. (R2.4)
- Capacity-requirement gates use either `>= <requirement>` or a
  margin variable with `>= 0`. (R2.5)

**Verification gate**

- Re-run extract on paperclip and datacenter using the Phase-1
  digests.
- `parameters.json` checks: total cost in `recommended_first_calculations`;
  holding cost as a calculation, not a missing value; phase-1 CAPEX
  surplus decomposed into civil_works + remediation + hardware
  components; buildable-area gate either `>= 32.2` or a margin
  variable.
- Run the existing 14-plan baseline. Any plan whose `parameters.json`
  shifts entries between `missing_values_to_estimate` and
  `recommended_first_calculations` is expected and reviewed.

### Phase 3 — Validate stage (`validate-parameters`)

**Changes**

- New structural rule: LHS of a sum-formula MUST NOT also appear in
  `missing_values_to_estimate`. (R1.1)
- New structural rule: if a `*_required` key value exists, a
  margin / surplus variable must also exist. (R2.5)
- Expand `sampling_discipline` enum to include `lognormal` / `pert`
  (allow, do not require yet). (R1.4, R2.2)

**Verification gate**

- Validation runs cleanly on the Phase-2 `parameters.json` for all 14
  plans.
- Deliberately-malformed fixtures (total bounded AND computed;
  requirement without margin) fail validation with the new rules.

### Phase 4 — Generate-bounds (`generate-bounds`)

**Changes**

- System prompt: base-anchoring rewritten as a conditional rule —
  anchor on Premortem / Risk-register passages when they forecast a
  gap; fall back to commitment value with `source: "assumption"`.
  (R1.2)
- System prompt: self-audit examples for citation context-leak (Risk
  N referenced by number but with wrong topical content). (R1.5)
- System prompt: `plan_type`-driven lognormal default for
  hyperscale / geopolitical_megaproject. (R2.2)
- Output schema: optional `correlations` block. (R1.3)
- Output schema: `sampling_discipline` enum extended to lognormal /
  pert. (R1.4, R2.2)
- Code: extend `strip_threshold_bounds` to also strip bounds for any
  variable declared as a calculation LHS. (R1.1)

**Verification gate**

- Run Self-Improve against the 14-plan baseline. Flag any gate
  verdict that shifts more than one band for manual review.
- Paperclip: `actual_manual_intervention_hours_per_week` `base`
  should move off the 2.0-commitment if Premortem / Risk anchor it
  higher.
- Datacenter: CAPEX variables should be lognormal.

### Phase 5 — Verify-bounds-citations (new deterministic step)

**Changes**

- New deterministic Python: parse Risk / Issue / Decision N tokens
  from `bounds.json` rationale strings; fail loud when topical
  overlap with the cited section is below threshold. (R1.5)
- Wire into `run-napkin-math-pipeline` between Stage 5
  (`generate-bounds`) and Stage 6 (`generate-calculations`).

**Verification gate**

- Re-run on the original paperclip bounds (the one with the Risk-5
  grading-protocol hallucination). The verifier MUST fail loud.
- Re-run on the Phase-4 bounds: citation overlap is high enough that
  the verifier passes.

### Phase 6 — Generate-calculations (`generate-calculations`)

**Changes**

- None required; the existing code generator already handles new
  `formula_hint` entries from Phase 2.

**Verification gate**

- Regenerate `calculations.py` for paperclip and datacenter; module
  imports cleanly and includes the new aggregate / burn-rate / FX
  functions.
- `test-napkin-math` smoke tests pass.

### Phase 7 — Run-scenarios (`run-scenarios`)

**Changes**

- Policy: scenarios stay triangular-only; `generate-bounds` may NOT
  select lognormal / pert for any variable that also has a scenario
  row. Document the constraint in the bounds system prompt. (R1.4,
  R2.2)

**Verification gate**

- Three-point scenarios resolve for all 14 plans; no NaN /
  divide-by-zero. Bound widths remain readable.

### Phase 8 — Monte Carlo runner (`run_monte_carlo.py`)

**Changes**

- Strip bounds for calculation-LHS variables (deterministic backstop
  paralleling `strip_threshold_bounds`). (R1.1)
- Implement Gaussian-copula sampling; honour `correlations` block
  from `bounds.json` plus `correlation_groups` from
  `montecarlo_settings.json`. (R1.3)
- Implement lognormal and PERT samplers; extend `VALID_DISCIPLINES`.
  (R1.4, R2.2)
- Extend `test-napkin-math` smoke fixtures to exercise the new
  samplers and the new strip rule.

**Verification gate**

- Smoke tests pass.
- Re-run on the 14-plan baseline; pass-rate diffs from the
  pre-correlation, pre-lognormal runs are explainable per plan
  (correlation shrinks pass rate on plans with co-moving cost
  drivers; lognormal shrinks pass rate on plans with bounded-tail
  CAPEX bounds).

### Phase 9 — Summarize-assessment (`summarize-assessment`)

**Changes**

- Composite-band cap on `overall_risk_band` based on
  `unmodelled_gates.length`. Monotonic: more unmodelled gates can
  only worsen the headline. (R2.1)
- New manifest field `unmodelled_gate_penalty` naming exactly what
  was applied. (R2.1)
- Surface `threshold_basis` more prominently in the gate-verdicts
  table; flag `model_defined / >= 0` next to a value-bearing
  requirement. (R2.5)
- Disclose anchoring source, correlation groups, discipline (and
  `plan_type` default) in the slideshow. (R1.2, R1.3, R1.4, R2.2)

**Verification gate**

- Re-render all 14 plans' assessments. Calibrate the cap-rule
  thresholds (placeholder K = 3 / 5) against the actual spread of
  `unmodelled_gates.length` across the corpus.
- The datacenter plan no longer reads "100% Robust"; it reads
  something like "100% Robust (modelled gates), capped at Marginal
  due to 4 unmodelled existential gates".
- No benign plan is over-penalized into a worse band than its
  modelled verdict justifies.

### Phase 10 — Documentation

**Changes**

- Update `methology.md` and the README to describe the composite-band
  cap rule, the new disciplines, the new verifier step, and the
  declared correlation surface. (R2.1, R1.3, R1.4, R1.5, R2.2)
- Update the `run-napkin-math-pipeline` orchestrator skill so the
  new `verify-bounds-citations` step appears in the documented
  pipeline.

**Verification gate**

- Slideshow content slides (Slide A / Slide B) match actual pipeline
  behaviour; no drift between code and prose.

### Cross-phase notes

- Each phase's verification gate runs against the **14-plan
  baseline**, not just paperclip and datacenter. Single-plan checks
  exist to catch the specific failure that motivated the change;
  the 14-plan run catches over-correction.
- Phases 1, 2, 4, 8, 9 each change LLM-facing prompts or sampling
  behaviour; expect snapshot diffs in `output/v{N+1}/<plan>/`
  artifacts. Phases 3, 5, 6, 7, 10 should not move artifact contents
  except where Phase 1–2 changes propagated.
- If a phase's verification gate fails, the rollback is to the
  previous phase's tip (each phase is a small set of commits / a
  small PR).

## What does not change

- The runner's posture of "everything semantic comes from upstream
  stages; nothing is pattern-matched on id strings" stays. Every
  change above is either a new schema field declared by an LLM
  stage, a deterministic check on existing structure, or a
  verdict-aggregation rule.
- The slideshow's worst-gate framing for modelled gates stays; the
  composite cap adds a *separate* aggregation across
  modelled + unmodelled, not a rewrite of the modelled-gate logic.
- `summarize-assessment`'s `TRUST_NOT_VALIDATED` list shrinks as
  flaws land. "Independence or correlation assumptions across
  inputs" moves from "we don't model it" to "the user (or LLM)
  declared these groups" once R1.3 ships.

## Open questions

- Composite-band cap thresholds (1–2 / 3–4 / 5+) are placeholders.
  Retroactive run across the 14 existing plans is the right way to
  calibrate. Should the cap also weight by gate severity (a missing
  permit is heavier than a missing community-engagement plan)?
- Aggregate stripping — strip silently (like thresholds today) or
  warn loudly? Aggregates likely deserve a louder warning since the
  upstream defect is more recoverable.
- Lognormal in `run-scenarios` — accept the inconsistency (scenarios
  triangular, MC lognormal) or constrain `generate-bounds` to
  triangular for any variable with a scenario row?
- Citation verifier scope — Risk / Issue / Decision tokens only, or
  also Premortem / Expert-Criticism passages? Start narrow.
- Correlation-group ergonomics — full matrix, pairwise ρ, or named
  groups with single ρ? Named groups are lightest cognitive load.
- Burn-rate detection — should the extract emit a dedicated
  `is_burn` flag on `key_values` so `validate-parameters` can check
  the `burn × duration` multiplication is wired up downstream?
  Cleaner than relying on a formula parser.
- `plan_type` enumeration — current taxonomy is free-form. To make
  R2.2 work (megaproject lognormal default), the type list needs to
  be a controlled vocabulary. Where does that vocabulary live, and
  who maintains it?
- Bernoulli-modelling existential gates as a follow-up to R2.1 —
  what does the LLM need to know to assign survival probabilities
  responsibly? Anchor on the same Premortem / Risk-register passages
  used for cost bounds, or require a dedicated extraction step?
