# Plan: Heuristic Case Folder Migration

## Overview

Move inbox cases and operation-record cases from single Markdown files to
per-case folders containing `ENTRY.md` (or `RECORD.md`), a redacted
`evidence/` subfolder, and an optional plan link. Add a redaction guardrail
to the helper so curated evidence can live in vcs without violating the
"runtime evidence is not automatically a repo artifact" rule. Migrate the
two existing flat-file artifacts in the same PR. The Tier 2 lifecycle
compression must land first.

## Read First

- Primary source: `docs/plans/heuristic-case-folder-migration/heuristic-case-folder-migration-discussion-source.md`
- Source type: discussion-to-implementation-doc
- Open questions carried into execution: plan-in-case-folder default, redact
  subcommand vs flag (default: flag).

## Execution

- Execution state: `docs/plans/heuristic-case-folder-migration/heuristic-case-folder-migration-execution-state.md`

## Scope

- In scope:
  - `HEURISTIC_SYSTEM.md`
  - `heuristic-system/README.md`
  - `heuristic-system/error-inbox/README.md`
  - `heuristic-system/error-inbox/archive/2026/deliver-gitlab-mr-skipped-pipeline-and-cleanup.md`
    (migration to folder)
  - `heuristic-system/operation-records/github-pr-required-check-gating.md`
    (migration to folder)
  - `skills/workflows/heuristic-system/heuristic-error-inbox/SKILL.md`
  - `skills/workflows/heuristic-system/heuristic-error-inbox/bin/heuristic_error_inbox.py`
  - `skills/workflows/heuristic-system/heuristic-error-inbox/scripts/heuristic-error-inbox.sh`
  - `skills/workflows/heuristic-system/heuristic-error-inbox/tests/test_workflows_heuristic_system_heuristic_error_inbox.py`
  - `tests/test_heuristic_system_docs.py`
  - `skills/workflows/plan/docs-plan-cleanup/scripts/docs-plan-cleanup.sh`
    (path matchers)
  - `skills/workflows/plan/durable-artifact-cleanup/SKILL.md`
    (path references)
- Out of scope:
  - nils-cli graduation (separate plan).
  - Lifecycle enum changes (covered by Tier 2 plan).
  - `skill-usage.record.v1` schema, `agent-out` paths, or new skills.
  - Allowing plans to live inside case folders.
  - Automatic evidence scraping or hook auto-writing.

## Assumptions

1. `heuristic-tier2-doc-compression` has shipped and the lifecycle enum is
   `open | promoted | wontfix`.
2. The helper can extend to folder ops without losing legacy-read paths from
   Tier 2.
3. Redaction is enforced in the helper, not by reviewer discipline.
4. Existing references in `docs-plan-cleanup.sh` and
   `durable-artifact-cleanup/SKILL.md` need a path-shape update but not a
   workflow rewrite.

## Sprint 1: Define Folder Schema And Redaction Guardrail

**Goal**: Land the folder schema, the redaction module, and helper plumbing
without touching existing artifacts.
**Demo/Validation**:

- Command(s):
  - `scripts/check.sh --tests -- -k heuristic_error_inbox`
  - `scripts/check.sh --markdown`
- Verify:
  - Folder schema documented and validated by helper.
  - Redaction module accepts curated input and rejects raw records.
  - No existing artifact moved yet.

**PR grouping intent**: per-sprint
**Execution Profile**: serial

### Task 1.1: Document the folder schema in HEURISTIC_SYSTEM and inbox READMEs

- **Location**:
  - `HEURISTIC_SYSTEM.md`
  - `heuristic-system/README.md`
  - `heuristic-system/error-inbox/README.md`
- **Description**: Add a `Case Layout` section that defines the per-case
  folder shape (`<slug>/ENTRY.md`, `<slug>/evidence/`) for inbox and
  (`<slug>/RECORD.md`, `<slug>/evidence/`) for operation records. Restate
  the redaction guardrail and the unchanged "runtime evidence is not
  automatically a repo artifact" rule.
- **Dependencies**:
  - none
- **Complexity**: 3
- **Acceptance criteria**:
  - HEURISTIC_SYSTEM.md states the folder layout for both subdirectories.
  - Both READMEs reference the layout without duplicating the redaction
    rules.
  - Tests assert the folder-layout statement and the redaction-rule presence.
- **Validation**:
  - `scripts/check.sh --tests -- -k heuristic_system`

### Task 1.2: Implement the redaction module in the helper

- **Location**:
  - `skills/workflows/heuristic-system/heuristic-error-inbox/bin/heuristic_error_inbox.py`
  - `skills/workflows/heuristic-system/heuristic-error-inbox/tests/test_workflows_heuristic_system_heuristic_error_inbox.py`
- **Description**: Add a `redact` helper module that enforces token-like
  pattern rejection, size limits, absolute-path normalization, and outright
  rejection of raw `skill-usage.record.json`. Unit-test acceptance and
  rejection paths.
- **Dependencies**:
  - Task 1.1
- **Complexity**: 5
- **Acceptance criteria**:
  - The redact module exposes `accept(path|content) -> NormalizedArtifact`
    and raises on rejection with a clear reason.
  - Tests cover at least: token-like content, oversize file, absolute path
    rewrite, raw record rejection.
- **Validation**:
  - `scripts/check.sh --tests -- -k heuristic_error_inbox`

### Task 1.3: Add folder-aware helper subcommands

- **Location**:
  - `skills/workflows/heuristic-system/heuristic-error-inbox/bin/heuristic_error_inbox.py`
  - `skills/workflows/heuristic-system/heuristic-error-inbox/scripts/heuristic-error-inbox.sh`
  - `skills/workflows/heuristic-system/heuristic-error-inbox/tests/test_workflows_heuristic_system_heuristic_error_inbox.py`
- **Description**: Extend `new`, `set-status`, `list`, `verify`, `archive`
  to operate on folder cases. Add `ingest-evidence` for redacted writes.
  Accept both folder path and `ENTRY.md` path inputs. Treat operation
  records as a parallel case type with `RECORD.md`.
- **Dependencies**:
  - Task 1.2
- **Complexity**: 6
- **Acceptance criteria**:
  - `new --slug X` creates `<root>/X/ENTRY.md` and an empty `evidence/`.
  - `ingest-evidence <case> --from <path>` runs through the redaction
    module.
  - `archive <case>` moves the whole folder.
  - `verify <case>` checks folder shape, ENTRY.md sections, and that all
    files in `evidence/` passed redaction.
- **Validation**:
  - `scripts/check.sh --tests -- -k heuristic_error_inbox`

## Sprint 2: Migrate Existing Artifacts And External References

**Goal**: Convert the two existing flat-file artifacts to folders, update
sibling tooling that references the old paths, and document the
manual-migration playbook for any future legacy entries.
**Demo/Validation**:

- Command(s):
  - `scripts/check.sh --tests -- -k "heuristic_system or heuristic_error_inbox"`
  - `scripts/check.sh --docs`
- Verify:
  - Migrated artifacts validate with the helper.
  - Path matchers in cleanup tooling resolve folder shapes.

**PR grouping intent**: per-sprint
**Execution Profile**: serial

### Task 2.1: Migrate the archived inbox entry to folder shape

- **Location**:
  - `heuristic-system/error-inbox/archive/2026/deliver-gitlab-mr-skipped-pipeline-and-cleanup.md`
    (becomes `heuristic-system/error-inbox/archive/2026/deliver-gitlab-mr-skipped-pipeline-and-cleanup/ENTRY.md`)
- **Description**: Move the archived entry into a folder. Preserve its
  historical lifecycle text verbatim. Do not add evidence files; leave
  `evidence/` empty (or omit it for legacy entries).
- **Dependencies**:
  - Task 1.3
- **Complexity**: 2
- **Acceptance criteria**:
  - The file moves via `git mv`; commit history of the original is
    preserved.
  - `heuristic-error-inbox.sh verify` succeeds on the new folder.
- **Validation**:
  - `heuristic-error-inbox.sh verify heuristic-system/error-inbox/archive/2026/deliver-gitlab-mr-skipped-pipeline-and-cleanup/`

### Task 2.2: Migrate the operation record to folder shape

- **Location**:
  - `heuristic-system/operation-records/github-pr-required-check-gating.md`
    (becomes `heuristic-system/operation-records/github-pr-required-check-gating/RECORD.md`)
- **Description**: Move the operation record into a folder. Add a sample
  `evidence/` file referencing the original validation commands, but only
  if its content already passes the redaction guardrail. Otherwise leave
  `evidence/` empty and link the validation commands inline.
- **Dependencies**:
  - Task 2.1
- **Complexity**: 2
- **Acceptance criteria**:
  - `verify` succeeds on the new folder.
  - `RECORD.md` keeps the original section list.
- **Validation**:
  - `heuristic-error-inbox.sh verify heuristic-system/operation-records/github-pr-required-check-gating/`

### Task 2.3: Update sibling tooling and docs references

- **Location**:
  - `skills/workflows/plan/docs-plan-cleanup/scripts/docs-plan-cleanup.sh`
  - `skills/workflows/plan/durable-artifact-cleanup/SKILL.md`
  - `skills/workflows/plan/durable-artifact-cleanup/tests/test_workflows_plan_durable_artifact_cleanup.py`
- **Description**: Update path matchers and doc references to handle the
  folder shape. Keep flat-file detection as a backward-compatible read.
- **Dependencies**:
  - Task 2.2
- **Complexity**: 3
- **Acceptance criteria**:
  - `docs-plan-cleanup` does not falsely classify case folders as plan
    artifacts.
  - `durable-artifact-cleanup` references the new layout.
  - Existing tests still pass; add at least one folder-shape regression
    test.
- **Validation**:
  - `scripts/check.sh --tests -- -k "docs_plan_cleanup or durable_artifact_cleanup"`

## Sprint 3: Validate And Land

**Goal**: Run the full validation gate, verify cross-document consistency,
and ensure no out-of-scope drift leaked in.
**Demo/Validation**:

- Command(s):
  - `plan-tooling validate --file docs/plans/heuristic-case-folder-migration/heuristic-case-folder-migration-plan.md`
  - `scripts/check.sh --all`
- Verify:
  - All targeted and full-repo gates pass.
  - No raw evidence committed.

**PR grouping intent**: per-sprint
**Execution Profile**: serial

### Task 3.1: Run focused gates and audit committed evidence

- **Location**:
  - `heuristic-system/`
- **Description**: Run the targeted heuristic + cleanup tests. Grep the
  diff for token-like content and oversized files before staging.
- **Dependencies**:
  - Task 2.3
- **Complexity**: 2
- **Acceptance criteria**:
  - Focused tests pass.
  - No committed file under `heuristic-system/` exceeds the configured size
    limit.
  - No token-like patterns in committed evidence.
- **Validation**:
  - `scripts/check.sh --tests -- -k "heuristic_system or heuristic_error_inbox"`

### Task 3.2: Run the full repo gate and prepare delivery

- **Location**:
  - `docs/plans/heuristic-case-folder-migration/heuristic-case-folder-migration-execution-state.md`
- **Description**: Run `scripts/check.sh --all`. Record validation
  evidence. Confirm no out-of-scope edits.
- **Dependencies**:
  - Task 3.1
- **Complexity**: 2
- **Acceptance criteria**:
  - `scripts/check.sh --all` passes.
  - Execution state lists final commands and outcomes.
- **Validation**:
  - `scripts/check.sh --all`

## Testing Strategy

- Unit: helper module tests (redaction, folder ops, archive move,
  ingest-evidence).
- Integration: doc tests on `HEURISTIC_SYSTEM.md`, the two READMEs, and the
  helper SKILL.md. Path-matcher regression tests for cleanup tooling.
- E2E/manual: a local smoke creating a case, ingesting an evidence excerpt,
  archiving it, and verifying the folder shape.

## Risks & gotchas

- A weak redaction guardrail is the single largest risk. Test acceptance
  and rejection paths exhaustively.
- Folder archive moves are heavier than file archive moves; the helper must
  not lose nested files mid-move on failure.
- Tooling that grepped for `*.md` directly under the inbox dir will need
  updating; audit before assuming nothing else references the old shape.
- Migrating the existing archived entry must preserve its historical
  lifecycle text and audit value.

## Rollback plan

- Revert helper changes (subcommands and redaction module) first.
- Revert artifact migrations using `git mv` back to flat files.
- Revert docs changes describing the folder layout.
- Revert sibling tooling path-matcher updates.
- Tier 2 lifecycle collapse remains unaffected by rollback of this plan.
