# Modeling Memory and Controlled Skill Evolution

Agentic SWMM is not only an automation workflow. It is a memory-informed, verification-first modeling system that can learn from audited modeling history through controlled skill refinement.

## Problem

Environmental modeling workflows create many hidden decisions, assumptions, failures, QA checks, and artifacts. If these are not remembered, agentic modeling becomes hard to audit and reproduce. A single successful SWMM execution is not enough to explain which inputs were trusted, which checks passed, which evidence was missing, or which failures repeated across attempts.

## Existing Audit Layer

`swmm-experiment-audit` records run-level evidence. It consolidates provenance, artifacts, QA checks, metrics, warnings, limitations, comparisons, and Obsidian-compatible experiment notes for individual runs.

The audit layer answers what happened in one run.

## New Modeling-Memory Layer

`swmm-modeling-memory` reads multiple audit records and turns them into reusable project memory. It scans historical `experiment_provenance.json`, `comparison.json`, `experiment_note.md`, and optional `model_diagnostics.json` files, tolerates partial runs, and summarizes repeated assumptions, QA issues, missing evidence, run-to-run differences, failure patterns, deterministic SWMM diagnostics, and successful practices.

The layer now has three granularities:

- Run-level memory cards: each audited run gets a `memory_summary.json` with success/failure, QA state, missing evidence, warnings, assumptions, suspect parameters, deterministic diagnostic IDs, and next-run cautions.
- Project/case-level memory: aggregate outputs under `memory/modeling-memory/projects/<project-key>/` keep Tod Creek, Tecnopolo, TUFLOW, Generate_SWMM_inp, acceptance, and other cases separate.
- Global modeling memory: the root index, lessons, and skill proposals still summarize cross-run patterns across the full `runs/` tree.

The modeling-memory skill does not automatically rewrite existing skills. It analyzes historical audit records and generates proposed refinements for relevant workflow skills, such as end-to-end orchestration, audit reporting, QA verification, model building, or result parsing.

The modeling-memory layer answers what keeps happening across runs.

## SWMM-Specific Diagnostics

SWMM-specific modeling intelligence should come from deterministic audit evidence rather than free-text inference. `swmm-experiment-audit` writes `model_diagnostics.json` when it can inspect model/report artifacts. The current screening diagnostics include high continuity error, node flooding reported by SWMM, invalid subcatchment area/width/imperviousness, missing rain gages, missing subcatchment outlets, suspicious conduit slopes, large routing steps, and disconnected outfalls.

`swmm-modeling-memory` only summarizes those diagnostics after they have been generated by the audit layer. It does not guess hydrologic model errors from notes.

## Controlled Skill Refinement

The intended controlled loop is:

1. SWMM run
2. experiment audit
3. deterministic model diagnostics
4. run-level memory card
5. Obsidian-compatible note
6. project/global modeling-memory summarization
7. failure-pattern extraction
8. skill update proposal
9. human review
10. benchmark verification
11. accepted skill update

The proposal step is intentionally separate from the accepted update step. Modeling memory may suggest where a workflow or skill appears weak, but it does not modify scientific rules or repository skills by itself. Proposed updates are accepted only after human review and benchmark verification.

## Curated vs Raw Memory (PRD M7.3 contract)

Memory is split into two layers, never mixed:

- **Curated**: `memory/modeling-memory/lessons_learned.md` and `memory/modeling-memory/INDEX.md`. LLM-summarised / LLM-curated. Subject to compaction (PRD M3) when the lessons file exceeds the configured size or pattern-count threshold. Recall tool: `recall_memory(pattern)`.
- **Raw**: every `runs/<case>/09_audit/experiment_note.md` and every `runs/<date>/<chat-session>/chat_note.md`. Never edited after audit; treated as the source of truth for evidence. Recall tool: `recall_memory_search(query, top_k)` returns raw + curated entries side-by-side, each tagged with `layer: "curated"` or `layer: "raw"` so the planner knows which side it is reading.

Recall results returned to the planner are wrapped in `<memory-context source="lessons|rag" stale="...">…</memory-context>` (PRD M7.1). A streaming scrubber on the final-output path strips that fence from any text the agent emits to the user, so historical memory can never be parsed as new user instructions.

## Safety Boundary

The agent does not autonomously rewrite scientific modeling rules. Skill update proposals are not evidence of correctness. A proposed refinement should only be accepted after human review, existing benchmark verification, and clear evidence that the change improves the workflow without hiding missing data, failed QA, or unsupported assumptions.

## Example CLI Usage

Main command:

```bash
python3 skills/swmm-modeling-memory/scripts/summarize_memory.py \
  --runs-dir runs \
  --out-dir memory/modeling-memory
```

This command writes aggregate memory under `memory/modeling-memory/` and writes `memory_summary.json` into each audited run directory.

Optional Obsidian export:

```bash
python3 skills/swmm-modeling-memory/scripts/summarize_memory.py \
  --runs-dir runs \
  --out-dir memory/modeling-memory \
  --obsidian-dir "/path/to/Obsidian/Agentic SWMM/05_Modeling_Memory"
```
