---
name: skill-validate
description: Validate mode — check candidates against evidence and known results.
trigger: stage == "L4"
---

# Validate Mode

## MANDATORY: AskUserQuestion rule

When you need to ask the user ANY question (clarification, scope, direction, missing info), you MUST:
1. Call `ToolSearch(query="select:AskUserQuestion", max_results=1)` to load the tool.
2. Call `AskUserQuestion(questions=[{...}])` with your question and options.
NEVER type questions or options as plain text. ALWAYS use the popup tool.

---

You are in **validate** mode. Your role: **adversarial collaborator**. You are NOT
certifying your own work — you are trying to **falsify** the claim. A "pass" means
the claim survived your best attacks. The protocol BLOCKS submissions without evidence.

## What to Do

### Step 1: Read the candidate and set adversarial posture

Read `L3/candidates/<id>.md`. Then explicitly answer:
- What would falsify this claim? (a counterexample, a dimensional mismatch, a limit violation)
- What assumptions are load-bearing? (if any ONE assumption fails, the claim collapses)
- What regime boundaries are untested?

### Step 2: Check compute_target and L2 for contradictions

1. Call `aitp_get_execution_brief` → read `compute_target`. Confirm you are on the right machine.
   If the compute target needs to change:
   ```
   aitp_set_compute_target(topics_root, topic_slug, host="<hostname>", work_dir="<path>")
   ```
2. Call `aitp_query_l2` with the candidate's key concepts → search for conflicting claims.
3. Call `aitp_query_l2_graph` to check for contradictory edges.
4. If ANY contradiction found: record it, submit outcome="contradiction", do NOT pass.

### Step 3: Run independent verification

#### For formal_theory lane — SymPy verification (MANDATORY for pass)

You MUST run at least ONE of these before submitting pass:

```
aitp_verify_dimensions("H = hbar*omega*(N+1/2)",
    {"H":"energy","hbar":"action","omega":"frequency","N":"number"})
→ returns pass/fail with term-by-term breakdown

aitp_verify_algebra("a*a_dag - a_dag*a", "1")
→ returns pass/fail with simplified difference

aitp_verify_derivation_step("substitute", "x^2 + y", "x^2 + 2x + 1", "y=2x+1")
→ verify each derivation step against its inference rule

aitp_verify_limit("(n+1/2)*hbar*omega", "n", "oo", "n*hbar*omega")
→ check correspondence principle (quantum → classical limit)
```

Pass the results as `verification_evidence` to `aitp_submit_l4_review`.

For Lean formal verification (optional, highest assurance):
- `lean-lsp-mcp__lean_verify` — verify a theorem with axiom check

#### For toy_numeric / code_method lanes — executed scripts (MANDATORY for pass)

**Prerequisite — Binary regression check (BLOCKING for code_method):**
Before submitting any HPC validation job, verify the compiled binary against
its registered regression test. If the domain skill provides a smoke check
command, it MUST return pass before the L4 validation gate can clear.
- Fail → L4 outcome is `blocked`, do not proceed to HPC submission
- Evidence: record binary git commit + regression test result in review

1. Write validation scripts in `L4/scripts/validate_<check>.py`
2. Execute them on the declared compute_target:
   - For HPC/long-running jobs, use:
     ```
     aitp_l4_background_submit(topics_root, topic_slug, candidate_id="<id>",
         job_script="L4/scripts/validate_<check>.py",
         compute_host="<host>", work_dir="<path>")
     ```
    - Check results when the job completes:
      ```
      aitp_l4_check_results(topics_root, topic_slug, candidate_id="<id>")
      ```
 3. **Analyze HPC output** (MANDATORY for code_method before filing review):
    ```
    aitp_l4_analyze_run(topics_root, topic_slug, run_dir="<path>",
        run_id="<run_id>", literature_comparison=True)
    ```
    This automatically:
    - Parses LibRPA/ABACUS output files for key numerical values (band gap, VBM, CBM)
    - Compares against literature benchmarks (Si gap 1.17 eV etc.)
    - Checks QSGW convergence (delta_E < 1e-3 eV threshold)
    - Writes structured analysis to L4/outputs/<run_id>.md
    - Returns agreement_status: agrees | deviates | inconclusive | no_benchmark_available
 4. Save outputs to `L4/outputs/`
 5. For numerical results, record structured data:
    ```
    aitp_record_numerical_result(
        topics_root, topic_slug,
        observable="<name>",
        computed_value=<float>,
        expected_value=<float or null>,
        uncertainty=<float>,
        source_script="L4/scripts/validate_<check>.py",
    )
    ```
 6. Record data_provenance for every data point

### Step 4: Devil's advocate assessment (MANDATORY for pass)

Before submitting pass, answer ALL of the following. The MCP server blocks
pass without non-empty `devils_advocate`. Answer each dimension:

1. **Load-bearing assumption**: Which ONE assumption, if violated,
   would collapse the entire claim? Be specific (not "any assumption
   could break").

2. **Falsification scenario**: Describe a concrete experiment or
   calculation that would prove the claim wrong. What observable
   would differ? By how much?

3. **Untested regime boundary**: What parameter range or physical
   regime has NOT been checked? What could go wrong there?

4. **Alternative explanation**: Could a different physical mechanism
   produce the same result? What would distinguish them?

5. **Hidden approximation**: Is there an approximation that was made
   implicitly (e.g. thermodynamic limit, continuum limit, classical
   treatment of nuclei) that could hide a real effect?

Format as a single string with these 5 sections. The MCP server
checks that `devils_advocate` is non-empty — fill it substantively.

### Step 4.5: AI Physicist Check

Before filing the review, load `skill-physicist-check` and answer the four questions:

1. **What does L2 already know?** Query L2 for benchmark values, prior results, regime boundaries.
2. **Correspondence check**: Name a concrete physical limit. State expected vs actual behavior.
3. **Anomalies**: Any surprises? Deviations? Unexpected behavior?
4. **For the human to verify**: What's the single most critical thing to scrutinize?

Record findings under `## Physicist Check` in the review. The gate checks that
correspondence names a limit and anomalies are documented.

### Step 5: Submit L4 review

```
aitp_submit_l4_review(
    topics_root, topic_slug, candidate_id,
    outcome="pass",  # or "fail", "partial_pass", "contradiction", "stuck", "timeout"
    notes="Summary of what was checked and why it matters.",
    check_results={
        "dimensional_consistency": "pass: [H] = energy, [hbar*omega] = energy",
        "symmetry_compatibility": "pass: H commutes with parity",
        "limiting_case_check": "pass: classical HO recovered as n→∞",
        "conservation_check": "pass: energy conserved, Hamiltonian time-independent",
        "correspondence_check": "pass: matches Griffiths 2.61",
        "approximation_validity_check": "pass: harmonic approximation valid for small displacements",
        "unitarity_check": "pass: time-evolution operator e^{-iHt} is unitary",
        "causality_check": "pass: retarded Green's function vanishes for t<0",
    },
    devils_advocate="REQUIRED: state how this could still be wrong...",
    verification_evidence={
        "tool": "aitp_verify_dimensions",
        "result": {"pass": True, ...}
    },
    # For numeric lanes, also REQUIRED:
    evidence_scripts=["L4/scripts/validate_qho.py"],
    evidence_outputs=["L4/outputs/qho_check.log"],
    execution_environment="local (Windows 11, Python 3.12, SymPy 1.14)",
)
```

The MCP server will BLOCK submission if:
- outcome="pass" but devils_advocate is empty (ALL lanes)
- Lane is toy_numeric/code_method and evidence_scripts/outputs missing
- Lane is formal_theory and neither check_results nor verification_evidence provided
- check_results is missing any required physics check field (lane-dependent):
  formal_theory requires all 8 fields; other lanes require the first 5

### Step 6: After L4 decision

- pass → candidate status auto-set to "validated". Ask human about promotion.
- partial_pass → candidate status "partial_validated". Discuss what's useful.
- fail/contradiction/stuck/timeout → return to L3 analysis for revision:
  ```
  aitp_return_to_l3_from_l4(topics_root, topic_slug, reason="<specific findings>")
  ```
  This preserves all L4 review artifacts and sets L3 activity to `integrate`
  for post-validation analysis. The agent resumes with `skill-l3-analyze`.

## Rules

- **You are trying to falsify, not certify.** Approach every claim with skepticism.
- **No self-certification.** Every pass must be backed by SymPy, Lean, or executed scripts.
- **No anonymous data.** Every data point needs provenance (script, timestamp, method).
- **L4 pass does NOT mean "done".** Return to L3 for analysis and human check.
- **Record what was checked, how, and what passed/failed** — with file paths as evidence.

## Validation Outcomes

| Outcome | Meaning | Next Step |
|---------|---------|-----------|
| `pass` | Survived adversarial review with evidence | Route to promotion |
| `partial_pass` | Some criteria met, others inconclusive | Determine if partial result is useful |
| `fail` | Criteria not met | Return to L3 for revision |
| `contradiction` | Contradicts L2 or known results | Open gap, investigate, do NOT promote |
| `stuck` | Cannot complete with available resources | Record blocker, escalate |
| `timeout` | Verification taking too long | Reduce scope or switch approach |
