# v3.2.0 — Atomicity scope plan (§2.2)

`apply_engine.py` provides ACID-like guarantees by snapshotting
the workspace before any mutating apply, then either committing
the diff or restoring from the snapshot on failure. Two
directories are **deliberately** excluded from the snapshot scope:

- `maintenance/` — workspace housekeeping output (validation
  reports, compaction logs, scratch files).
- `intelligence/applied/` — the snapshot directory itself, plus
  the receipt log.

§2.2 of `AUDIT_FINDINGS_FOR_CLAUDE.md` flags this as a risk: if a
multi-stage apply fails partway through, files written under those
directories are not restored, which can leave untrackable state.

---

## 1. Why each exclusion exists today

### 1.1 `intelligence/applied/`

The snapshot directory itself is `intelligence/applied/<ts>/`.
Including it in the snapshot would create infinite recursion (the
snapshot containing yesterday's snapshot containing the day-before
snapshot, ad infinitum). This exclusion is **structurally
required** and must stay.

### 1.2 `maintenance/`

Originally excluded because:

1. Validation reports (`maintenance/validation-report.txt`) get
   regenerated on every run; preserving them across rollback is
   noise.
2. The compaction logs are append-only; rolling them back would
   discard real signal.
3. Snapshot size matters — `maintenance/` historically grew to
   tens of MB and bloated every snapshot.

These reasons are still valid for **append-only** files but not
for state files that affect the next apply's behaviour (e.g.,
compaction checkpoints, dedup-state hashes).

---

## 2. The actual risk

A partial apply that crashes between two stages:

1. Stage A — writes a new block to `memory/`.
2. Stage B — touches `maintenance/dedup-state.json` to record the
   block's hash.
3. Process crashes.
4. Restore runs — block from Stage A is rolled back, but
   `maintenance/dedup-state.json` still contains the hash.
5. Next apply attempt re-writes the block, sees the hash already
   present in dedup-state, **silently skips** the write.

This is the failure mode the audit warns about. The window is
narrow but real.

---

## 3. Three remediation options

### Option A — Subdivide `maintenance/`

Split the directory into two namespaces:

```
maintenance/
├── append-only/           # excluded from snapshot — reports, logs
└── tracked/               # included in snapshot — checkpoints, state
```

Move every file that affects future-apply behaviour into
`tracked/`. Update the snapshot scope:

```python
SNAPSHOT_DIRS = (
    *existing_dirs,
    "maintenance/tracked",
)
EXCLUDE_DIRS = (
    "maintenance/append-only",
    "intelligence/applied",
)
```

Migration: rename existing files in v3.2.0; ship a one-shot
migration helper that detects the old layout and moves files into
the new namespaces on first apply.

**Pro:** explicit intent per file; small snapshot bloat;
backward-compatible via the migration helper.

**Con:** churn — every file in `maintenance/` needs a categorisation
pass; the rename is visible to ops.

### Option B — Inclusion-by-default with size cap

Include `maintenance/` in the snapshot by default. Cap the snapshot
size at some sensible limit (say 100 MB); when a file would push
over the cap, replace it in the snapshot with a manifest entry
saying "skipped — too large". Restore reads the manifest and warns
on skipped files.

**Pro:** correct by default; new files don't need categorisation.

**Con:** silent skipping is a footgun if the cap fires on the wrong
file. Snapshots grow unbounded for normal workspaces.

### Option C — Idempotency contract on consumers

Don't change the snapshot scope. Instead, document a hard
**idempotency contract** for any file that lives outside the
snapshot scope: every reader must tolerate the file being out of
sync with the corpus, and every writer must be able to rebuild it
from the corpus.

Test the contract with a periodic CI job that:
1. Builds a workspace.
2. Drops every file under `maintenance/`.
3. Runs `mind-mem-validate` + every read-side path.
4. Asserts no errors and no behavioural change.

**Pro:** zero code change to apply_engine; pushes consistency
responsibility to the right layer (readers/writers, not the
snapshot mechanism).

**Con:** new tests required; existing consumers may already violate
the contract; the dedup-state-style failure mode in §2 above is
not caught until the consumer is fixed.

---

## 4. Recommendation

**Option A.** It's the only option that's **correct by
construction**, doesn't bloat snapshots, and survives audits
because the boundary is explicit per file.

Migration cost is moderate (one rename pass + a one-shot helper);
the audit findings are concrete and reviewers can verify the new
layout matches the contract.

---

## 5. Implementation plan (v3.2.0)

1. **PR 1 — categorise existing `maintenance/` content.** For each
   existing file, decide whether it's append-only or tracked.
   Document the rule in `docs/maintenance-namespaces.md`.
2. **PR 2 — add the `tracked/` snapshot path.** Update
   `apply_engine.SNAPSHOT_DIRS` and the matching restore code.
   Tests verify the new path is captured + restored.
3. **PR 3 — migration helper.** Detect the v3.1.x layout
   (`maintenance/<file>` directly), move into the v3.2.0 layout
   (`maintenance/tracked/<file>` for tracked files,
   `maintenance/append-only/<file>` otherwise) on first apply.
   Helper emits a one-line summary so users can audit.
4. **PR 4 — repro test.** Add the §2 multi-stage failure scenario
   as a pytest case. Pre-PR: test fails (proves the bug). Post-PR:
   test passes.
5. **PR 5 — docs.** Update `SPEC.md` to document the new namespaces
   and the migration semantics.

Estimated diff size: ~250 LOC across apply_engine, snapshot helpers,
and tests.

---

## 6. Risks + mitigations

| Risk                                 | Mitigation                          |
| ------------------------------------ | ----------------------------------- |
| User scripts hardcoding `maintenance/<file>` paths break | Migration helper writes both paths during the v3.2.x window; legacy paths removed in v4.0 |
| Snapshot size regression             | `tracked/` should stay small (configs + checkpoints); add a CI assertion that snapshots stay under ~10 MB on the reference workspaces |
| Cross-platform path normalisation    | Existing `os.sep`-vs-`/` handling already covered by `tests/test_snapshot_windows.py` — extend that test to the new namespaces |
| Incomplete categorisation pass       | PR 1 reviewer must touch every file; CI gate fails if any new file lands directly under `maintenance/` (must go in tracked/ or append-only/) |
