# Walkthrough: Real CAE Pipeline (STEP → mesh → solver → evidence)

This document walks through the complete real-binary CAE pipeline in
`aieng-ui`: generating a mesh with FreeCAD, running CalculiX, extracting
scalar results from the FRD output, and refreshing the CAE summary and
evidence index — **no mocks anywhere**.

For the mocked CI benchmark, see
[`quickstart-vertical-cae-demo.md`](quickstart-vertical-cae-demo.md).
For CalculiX install options, see
[`quickstart-real-ccx.md`](quickstart-real-ccx.md).

---

## Prerequisites

| Requirement | Notes |
|-------------|-------|
| Python 3.10+ with `aieng-ui` backend deps | `pip install -e ".[dev]"` from `backend/` |
| `aieng` repo present at `../aieng/` | semantic package engine |
| `aieng_freecad_mcp` repo present at `../aieng_freecad_mcp/` | FreeCAD bridge |
| **FreeCADCmd** on PATH or at `FREECAD_CMD` | or a standard Windows install path |
| **CalculiX (`ccx`)** on PATH | Windows binaries, WSL, or conda-forge — see `quickstart-real-ccx.md` |
| Env var `AIENG_TEST_REAL_FREECAD=1` | opt-in gate for tests that invoke real FreeCAD |

---

## Pipeline overview

```
geometry/source.step          (10×1×1 mm cube, generated by FreeCADCmd)
        │
        ▼  cae.generate_mesh (approval-gated)
simulation/mesh/mesh_2.0mm.inp           ← FreeCAD/Gmsh tetrahedral mesh
simulation/mesh/mesh_metadata.json
        │
        ▼  in-test deck completion helper
simulation/runs/run_001/solver_input.inp ← mesh + *MATERIAL/*BOUNDARY/*CLOAD/*NODE FILE/*EL FILE/*END STEP
        │
        ▼  cae.run_solver (approval-gated, real ccx subprocess)
simulation/runs/run_001/solver_run.json  ← execution metadata
simulation/runs/run_001/solver_log.txt   ← ccx stdout + stderr
simulation/runs/run_001/outputs/result.frd  ← raw CalculiX output
        │
        ▼  FRD extractor (extract_results=True inside cae.run_solver)
results/computed_metrics.json            ← max displacement + max von Mises
        │
        ▼  postprocess.refresh_cae_summary
results/result_summary.json             ← LLM-readable summary
results/evidence_index.json             ← auditable artifact catalog
results/postprocessing_summary.md       ← human-readable markdown
```

Each arrow is a real runtime tool call through the `POST /api/runtime/runs`
endpoint. The two approval-gated steps (`cae.generate_mesh` and
`cae.run_solver`) pause execution and require an explicit `POST
/api/runtime/runs/{id}/approve` before proceeding.

---

## Running the smoke tests

### Basic real-ccx smoke test (no FreeCAD needed)

Exercises `cae.run_solver` against a committed 1-element fixture deck:

```powershell
cd aieng-ui\backend
python -m pytest -c NUL tests/test_api.py::test_run_solver_real_ccx_skipped_if_unavailable -v
```

**Passes** when `ccx` is on PATH; **skips cleanly** otherwise.

### Full real-pipeline smoke test (FreeCAD + ccx)

Exercises the entire pipeline from STEP geometry to evidence index:

```powershell
$env:AIENG_TEST_REAL_FREECAD = "1"
python -m pytest -c NUL tests/test_api.py::test_full_real_pipeline_step_to_summary -v -s
```

**Passes** (≈10–30 s, dominated by FreeCAD meshing) when both binaries are
present and `AIENG_TEST_REAL_FREECAD=1` is set.
**Skips cleanly** when either binary is absent or the env var is unset.

### Full backend suite

```powershell
python -m pytest -c NUL tests/test_api.py -q
```

Expected: approximately `176 passed, 3 skipped` (the 3 skips are the
real-binary tests; this count reflects the state at commit `77a10e9`).

---

## Produced artifacts

After a successful full-pipeline run, the `.aieng` package contains:

| Path | Written by | Content |
|------|-----------|---------|
| `geometry/source.step` | test setup | 10×1×1 mm cube STEP |
| `simulation/mesh/mesh_2.0mm.inp` | `cae.generate_mesh` | FreeCAD/Gmsh mesh (nodes + elements only) |
| `simulation/mesh/mesh_metadata.json` | `cae.generate_mesh` | element count, mesh size, element type |
| `simulation/runs/run_001/solver_input.inp` | deck-completion helper | mesh + material, BC, load, output blocks |
| `simulation/runs/run_001/solver_run.json` | `cae.run_solver` | run ID, solver, state, return code, timestamps, duration |
| `simulation/runs/run_001/solver_log.txt` | `cae.run_solver` | ccx stdout + stderr |
| `simulation/runs/run_001/outputs/result.frd` | `cae.run_solver` | raw CalculiX FRD output |
| `results/computed_metrics.json` | FRD extractor (inside `cae.run_solver`) | max displacement (mm), max von Mises stress (MPa) |
| `results/result_summary.json` | `postprocess.refresh_cae_summary` | LLM-readable CAE summary |
| `results/evidence_index.json` | `postprocess.refresh_cae_summary` | auditable catalog of all CAE artifacts |
| `results/postprocessing_summary.md` | `postprocess.refresh_cae_summary` | human-readable markdown |

---

## Evidence index behavior

`results/evidence_index.json` is written by `postprocess.refresh_cae_summary`
via `aieng.cae_result_summary.generate_evidence_index`. It contains an
`entries` array — one object per catalogued artifact.

### Static catalog entries (always present, `exists` reflects reality)

Includes `results/computed_metrics.json`, `results/result_summary.json`,
`simulation/mesh/mesh_metadata.json`, `simulation/solver_settings.json`, and
the other 15 canonical `.aieng` artifact paths.

### Dynamic solver-run entries (added when files are present in the package)

`generate_evidence_index` scans the package ZIP for runtime solver output
and adds entries for each one it finds:

| Pattern matched | Entry `kind` | Entry `role` | `supports` |
|-----------------|-------------|-------------|-----------|
| `simulation/runs/*/solver_run.json` | `result` | `solver_run_metadata` | `["solver_execution_evidence", "audit"]` |
| `simulation/runs/*/outputs/*.frd` | `result` | `solver_raw_output` | `["numerical_result_source", "post-processing evidence"]` |

These paths cannot be in the static catalog because run IDs are generated at
runtime. The dynamic scan ensures they appear as first-class evidence entries
with auditable provenance.

### Entry shape

```json
{
  "id": "simulation_runs_run_001_solver_run_json",
  "path": "simulation/runs/run_001/solver_run.json",
  "kind": "result",
  "role": "solver_run_metadata",
  "exists": true,
  "supports": ["solver_execution_evidence", "audit"]
}
```

Every present entry (`exists: true`) provides `path`, `kind`, `role`, and a
non-empty `supports` list — enough to audit what produced the evidence and
what downstream reasoning it supports.

---

## Evidence vs. engineering claims

Running the pipeline and refreshing the summary **creates evidence artifacts**.
It does **not** automatically validate or advance engineering claims.

| Action | What it produces | What it does NOT do |
|--------|-----------------|---------------------|
| `cae.run_solver` | `solver_run.json`, `solver_log.txt`, `result.frd`, `computed_metrics.json` | does not write `ai/claim_map.json` or `results/claim_map.json` |
| `postprocess.refresh_cae_summary` | `result_summary.json`, `evidence_index.json`, `postprocessing_summary.md` | does not write any claim map or advance any claim |

`solver_run.json` carries `converged: null` because CalculiX exit codes alone
are not reliable convergence evidence. Engineering claims require an explicit
claim-update workflow — a separate, deliberate step that is not part of the
solver execution or summary refresh path.

This contract is tested by `test_evidence_claim_contract_after_cae_run` in
`backend/tests/test_api.py`.

---

## What this proves / what this does not prove

### Proves

- The workbench can execute FreeCAD mesh generation and CalculiX in a real
  subprocess, capture the outputs, and write them atomically into the `.aieng`
  package.
- The approval gate correctly pauses execution before both `cae.generate_mesh`
  and `cae.run_solver`, and resumes on explicit `approve`.
- The FRD extractor produces finite, non-zero displacement and stress values
  with correct units (`mm`, `MPa`) from real CalculiX output.
- The evidence index is extended with dynamic entries for the solver run
  artifacts, giving an auditable trace from raw FRD to summary metrics.
- `solver_run.json`, `result_summary.json`, and `evidence_index.json` together
  form a complete evidence trail for the run.

### Does not prove

- Physical correctness of the stress or displacement result. No experimental
  correlation, mesh convergence study, or independent validation is performed.
- Engineering design validity or certification readiness.
- General meshing robustness. The coordinate-based boundary condition synthesis
  in `_complete_solver_deck` is a test helper, not a production BC generator.
- Convergence. `converged: null` is the honest value because CalculiX exit
  codes are not reliable convergence evidence.
- Multi-solver or multi-CAD generality. Only CalculiX (`ccx`) and FreeCAD are
  exercised.

---

## Manual validation checklist

Use this after running `test_full_real_pipeline_step_to_summary` locally
with `-s` (show output) to verify the run looks correct.

```
[ ] Solver step PASSED (not SKIPPED) — both ccx and FreeCADCmd were found
[ ] cae.generate_mesh returned ok=True and mesh_artifact_path starts with simulation/mesh/
[ ] cae.run_solver returned return_code=0 and solver_execution_performed=True
[ ] simulation/runs/run_001/outputs/result.frd is present in the package
[ ] simulation/runs/run_001/solver_run.json is present and converged is null
[ ] results/computed_metrics.json is present in the package
[ ] GET /api/projects/{id}/cae-result-summary returns extrema_computed=True
[ ] max_displacement value is finite, > 0, < 5 mm
[ ] max_von_mises_stress value is finite, > 0, < 1e5 MPa
[ ] max_displacement unit is "mm"; max_von_mises_stress unit is "MPa"
[ ] results/evidence_index.json is present and evidence_type is "cae_artifacts"
[ ] evidence_index has entry for simulation/runs/run_001/solver_run.json (kind=result, role=solver_run_metadata)
[ ] evidence_index has entry for simulation/runs/run_001/outputs/result.frd (kind=result, role=solver_raw_output)
[ ] ai/claim_map.json is NOT in the package (no claim auto-advancement)
[ ] results/claim_map.json is NOT in the package
```

---

## Inspecting results via the API

After a real run against a running `aieng-ui` backend:

```bash
# Read the evidence index
curl http://localhost:8000/api/projects/{project_id}/artifact?path=results/evidence_index.json

# Read solver run metadata
curl "http://localhost:8000/api/projects/{project_id}/artifact?path=simulation/runs/run_001/solver_run.json"

# Read the CAE result summary
curl http://localhost:8000/api/projects/{project_id}/cae-result-summary

# Read computed metrics
curl "http://localhost:8000/api/projects/{project_id}/artifact?path=results/computed_metrics.json"
```

The `GET /api/projects/{project_id}/artifact?path=...` endpoint returns parsed
JSON for `.json` files (≤ 2 MB) and inline text for `.txt`/`.log` files (≤
256 KB). It is read-only and does not execute or advance anything.

---

## CAE result field summaries

Two read-only endpoints expose compact statistics for the `displacement` and
`stress` fields without serving full per-node arrays. Field summaries are also
**persisted as package artifacts** when `postprocess.refresh_cae_summary` runs
— making them auditable evidence entries in `evidence_index.json`.

### Persisted field summary artifacts

`postprocess.refresh_cae_summary` writes two compact JSON artifacts into the
`.aieng` package whenever `results/computed_metrics.json` has displacement and
stress values:

```text
results/fields/displacement.summary.json
results/fields/stress.summary.json
```

Each artifact contains the same compact information as the API response:
`schema_version`, `field_name`, `unit`, `source` (artifact path, source type,
`computed_metrics_path`), `stats` (`max_value` from computed_metrics;
`min_value` / `node_count` / `values_finite` are `null` since full arrays are
not stored), `evidence_role`, `claim_advancement: "none"`, and `generated_at`.

No full per-node arrays are stored. This is compact evidence, not a contour
field.

After the **second** call to `refresh_cae_summary`, the artifacts are present
in the package and `evidence_index.json` shows them with `exists: true` and
evidence roles `displacement_extrema` / `stress_extrema`.

```json
{
  "id": "results_fields_displacement_summary_json",
  "path": "results/fields/displacement.summary.json",
  "kind": "field",
  "role": "cae_field_summary",
  "exists": true,
  "supports": ["displacement_extrema", "field_evidence", "audit"]
}
```

### List available fields

```bash
curl http://localhost:8000/api/projects/{project_id}/cae-result-fields
```

Returns:

```json
{
  "schema_version": "0.1",
  "project_id": "...",
  "available_fields": [
    {"field_name": "displacement", "unit": "mm", "max_value": 0.42, "source_type": "computed_metrics", "source_artifact": "results/computed_metrics.json"},
    {"field_name": "stress",       "unit": "MPa","max_value": 18.5, "source_type": "computed_metrics", "source_artifact": "results/computed_metrics.json"}
  ],
  "frd_source": "simulation/runs/run_001/outputs/result.frd",
  "claim_advancement": "none",
  "warnings": []
}
```

If `results/computed_metrics.json` is absent but an FRD is present, `max_value`
is `null` and `source_type` is `"frd"`.

### Per-field compact summary

```bash
curl http://localhost:8000/api/projects/{project_id}/cae-result-fields/displacement
curl http://localhost:8000/api/projects/{project_id}/cae-result-fields/stress
```

Returns (when FRD is present and extraction succeeds):

```json
{
  "schema_version": "0.1",
  "field_name": "displacement",
  "unit": "mm",
  "source": {
    "frd_path": "simulation/runs/run_001/outputs/result.frd",
    "source_type": "frd",
    "computed_metrics_path": "results/computed_metrics.json"
  },
  "stats": {"min_value": 0.0, "max_value": 0.42, "node_count": 847, "values_finite": true},
  "evidence_role": "displacement_extrema",
  "claim_advancement": "none",
  "warnings": []
}
```

Falls back to `computed_metrics.json` max value when FRD extraction is
unavailable (`stats.node_count` and `stats.min_value` are `null` in that case).

Returns **404** for unsupported field names (`"temperature"`, etc.) or when no
result data exists in the package.

### Evidence and claim semantics

The persisted artifacts and the read-only endpoints are evidence records. They
do not:
- Store or serve full per-node field arrays
- Execute solvers
- Write or advance `ai/claim_map.json` or `results/claim_map.json`

`claim_advancement: "none"` appears in every artifact and API response as the
machine-readable guarantee. Engineering claims require an explicit
claim-update workflow — a separate, deliberate step.

---

## References

- [`quickstart-real-ccx.md`](quickstart-real-ccx.md) — CalculiX install options and basic smoke test
- [`quickstart-vertical-cae-demo.md`](quickstart-vertical-cae-demo.md) — mocked benchmark (no ccx or FreeCAD required)
- [`runtime_architecture.md`](runtime_architecture.md) — orchestration layer, approval gate, tool adapters
- [`../../docs/demo-vertical-cae-workflow.md`](../../docs/demo-vertical-cae-workflow.md) — agent prompt walkthrough
- [`../../docs/aieng-agent-workflow.md`](../../docs/aieng-agent-workflow.md) — reusable agent workflow pattern
