# Heuristic System Follow-Up Bundle Review

## Status

- Date: 2026-05-19
- Reviewer: discussion session captured against `main@29a7c2a`.
- Source PRs:
  - `feat/heuristic-tier2-doc-compression` → merge `250d177` (PR #262).
  - `feat/heuristic-case-folder-migration` → merge `bc66780` (PR #263).
  - `feat/heuristic-inbox-cli-graduation` → merge `b919843` (PR #264).
- Post-merge bookkeeping commit: `29a7c2a` (pin `heuristic-inbox 0.10.1`, add
  auto-log smoke test).
- Source plans:
  - `docs/plans/heuristic-tier2-doc-compression/`
  - `docs/plans/heuristic-case-folder-migration/`
  - `docs/plans/heuristic-inbox-cli-graduation/`

## Purpose

Record the intent-fidelity audit of the three merged heuristic-system follow-up
PRs so future maintainers can locate surviving risk, intentional drift, and
documentation residue without re-deriving them from `git log` plus three plan
trios. This document is the retrospective evidence for the bundle; it is not a
plan and does not drive new sprint execution by itself.

## Current Judgment

The bundle delivered the intent of all three plans. PR #262 and PR #263 match
their plan text without material drift. PR #264 realised intent with two
user-approved drift decisions during execution (`--kind` flag dropped, shim
deleted in-PR rather than carried over) that were captured in execution state
but not back-ported into the plan text. Boundary between judgment (`SKILL.md`)
and deterministic operations (`heuristic-inbox` primitive) is clean on both
sides, with two minor seams worth tracking. The redaction scope gap that
survived the original merge has since been fixed upstream in nils-cli
`0.10.2`: `heuristic-inbox verify` reports body-level warnings by default and
`verify --strict` fails absolute `$HOME` paths in `ENTRY.md` / `RECORD.md`
body text.

## Findings

| ID | Priority | Issue | Evidence | Fix Location | Acceptance |
| --- | --- | --- | --- | --- | --- |
| R1 | High | Redaction guardrail did not cover `ENTRY.md` / `RECORD.md` body. New entries built from `new --from-skill-usage` could inherit raw `$HOME` paths from the source record. | Original `heuristic-inbox 0.10.1` verification surfaced body paths as `raw_records` only. Current `heuristic-inbox 0.10.2` reports body warnings by default and fails them under `--strict`. | nils-cli `heuristic-inbox` primitive plus agent-kit docs/tests/retained-case alignment. | **Resolved 2026-05-19**: [sympoies/nils-cli#372](https://github.com/sympoies/nils-cli/issues/372) closed in `v0.10.2`; agent-kit follow-up tracked in `docs/plans/heuristic-inbox-strict-redaction/`. |
| R2 | Low-Medium | CLI-smoke tests `pytest.skip` when `heuristic-inbox` is absent. CI hosts without nils-cli silently pass boundary tests, mirroring `skill-usage` precedent. | `skills/workflows/heuristic-system/heuristic-error-inbox/tests/test_workflows_heuristic_system_heuristic_error_inbox.py:118-160` three `shutil.which("heuristic-inbox") is None` branches. | Either provision nils-cli in CI image, or surface skip counts in `scripts/check.sh` summary. | Skip count tracked or test marked `xfail` rather than silently skipped. **Resolved 2026-05-19**: three `pytest.skip(...)` calls converted to `pytest.xfail(...)`; pytest summary now reports `xfailed` (distinct from generic skips) and `-ra` surfaces each `XFAIL` reason. |
| R3 | Low | Plan-text drift: PR #264 plan + discussion-source still describe `--kind inbox` as the default selector and the shim carrying one minor. Both decisions were dropped during execution. | `docs/plans/heuristic-inbox-cli-graduation/heuristic-inbox-cli-graduation-plan.md:220` shows `--kind inbox` in the manual smoke; same plan's open questions list `--kind` as default. Execution state captured the change but plan text did not. | `docs/plans/heuristic-inbox-cli-graduation/heuristic-inbox-cli-graduation-plan.md`; `docs/plans/heuristic-inbox-cli-graduation/heuristic-inbox-cli-graduation-discussion-source.md`. | Plan + discussion-source carry "Resolved post-execution" notes pointing at execution state. |
| R4 | Low | Version-pin tightness: PR #264 merged at `0.10.0`; the agent-kit smoke that proves Task 1.2 (auto-log capture) was added in post-merge `29a7c2a` together with a later minimum-version pin. Collaborators on older nils-cli versions now fail the version assertion. | The post-merge pin started at `>=0.10.1`; strict body redaction later raised the current agent-kit minimum to `>=0.10.2`. | n/a (already enforced); call out as a collaborator hand-off note. | Mentioned in DEVELOPMENT.md prereqs; collaborators upgrade `brew upgrade nils-cli`. |
| R5 | Low | Deterministic create gap: `heuristic-inbox new` requires `--from-skill-usage`, so hand-authored ENTRY.md from scratch is off the primitive create path. `SKILL.md` workflow step 3 still invites manual `ENTRY.md` edits, so the gap is workable but documented imprecisely. | `heuristic-inbox new --help` shows `--from-skill-usage <PATH>` as required; `SKILL.md:140` invites manual ENTRY.md editing for existing cases. | Either nils-cli `heuristic-inbox new` gains a `--blank` mode, or SKILL.md gives an explicit hand-authoring template + post-write `verify`. | Hand-authoring path is either supported by the primitive or fully documented as judgment-only with a `verify` gate. **Resolved 2026-05-19**: SKILL.md step 3 gains an explicit hand-author subsection with canonical `ENTRY.md` template + mandatory `heuristic-inbox verify` gate; guard test asserts the template phrases stay present. |
| R6 | Bookkeeping | PR #264 Task 1.2 acceptance ("logs land under `agent-out` topic on every write") was proven by the test added in `29a7c2a`, not in PR #264 itself. Reverting PR #264 alone leaves `29a7c2a` stranded. | `git log --oneline 6c6b3a3..29a7c2a` shows the test landed after the merge commit `b919843`. | n/a (historical). | Treat `29a7c2a` as PR #264's fourth step when describing the bundle. |

## Boundary Cleanliness

| Side | Verdict | Notes |
| --- | --- | --- |
| `SKILL.md` (judgment) | Clean | Workflow steps 1–6 contain no detection / classification / policy logic. All deterministic ops route through `heuristic-inbox`. |
| `heuristic-inbox` CLI (deterministic) | Clean with one remaining seam | No policy verbs in subcommand surface. Seam 1: `new` enforces "must originate from skill-usage record" as a deterministic constraint, narrowing the judgment-create path. The former body-redaction seam is resolved by nils-cli `0.10.2` warning plus `--strict` enforcement. |

## Backlog (Recommended Next Fixes)

| Order | Action | Owner | Linked Finding |
| --- | --- | --- | --- |
| 1 | ~~Land body-level redaction in nils-cli (warn-not-fail; `--strict` opt-in). Tracked as [sympoies/nils-cli#372](https://github.com/sympoies/nils-cli/issues/372).~~ Resolved in nils-cli `v0.10.2`; agent-kit alignment tracked in `docs/plans/heuristic-inbox-strict-redaction/`. | nils-cli / agent-kit | R1 |
| 2 | ~~Edit `heuristic-inbox-cli-graduation-plan.md` + discussion-source to add "Resolved post-execution" notes for `--kind` removal and immediate shim deletion.~~ Resolved by PR #265. | agent-kit | R3 |
| 3 | ~~Decide whether to surface skip counts in `scripts/check.sh` summary so silent CLI-skip becomes visible.~~ Resolved 2026-05-19 by converting three `pytest.skip(...)` → `pytest.xfail(...)`; CI summary now reports `xfailed` count distinct from generic skips. | agent-kit | R2 |
| 4 | ~~Decide whether `heuristic-inbox new --blank` is worth adding, or formalise hand-authoring via SKILL.md template + `verify` gate.~~ Resolved 2026-05-19 by adding hand-author template + verify gate to SKILL.md; nils-cli `--blank` mode no longer needed in agent-kit's scope. | agent-kit | R5 |
| 5 | Treat `29a7c2a` as PR #264's fourth bookkeeping step in any future bundle reference. | agent-kit | R6 |

Backlog items 1–4 have landed or are in the current agent-kit alignment lane
(R1 via nils-cli `v0.10.2` plus
`docs/plans/heuristic-inbox-strict-redaction/`; R3 via PR #265 plan-trio backfill; R2 via
`pytest.xfail(...)` conversion; R5 via SKILL.md hand-author template + verify
gate, all on 2026-05-19). Item 5 (R6) remains a recommendation awaiting its
own decision and is explicitly out of scope for
`docs/plans/heuristic-inbox-strict-redaction/`.

## Retention Intent

Long-lived. This is the durable evidence that the heuristic-system follow-up
bundle reached merge with known intentional drift and later redaction follow-up.
Keep until a successor improvement record or cleanup plan supersedes it.

This record should not drive new implementation by itself. Follow-up execution
belongs in `docs/plans/<slug>/`; the current redaction alignment lane is
`docs/plans/heuristic-inbox-strict-redaction/`.

## Validation

| Command | Status | Notes |
| --- | --- | --- |
| `agent-docs --docs-home "$HOME/.claude" resolve --context project-dev --strict --format checklist` | pass | 4/4 required docs present. |
| `agent-docs --docs-home "$HOME/.claude" resolve --context startup --strict --format checklist` | warn | Pre-existing gap (`~/.claude/CODEX_AGENTS.md` and `~/.claude/docs/runbooks/agent-docs/MEMORY_USAGE.md` missing); orthogonal to this bundle, user opted to proceed. |
| `heuristic-inbox --version` | `0.10.0` at PR #264 merge; `0.10.1` at HEAD `29a7c2a` | Pin tightened post-merge. |
| `heuristic-inbox verify heuristic-system/error-inbox/archive/2026/deliver-gitlab-mr-skipped-pipeline-and-cleanup/ --format json` | `ok=true` | Migrated archived case validates. |
| `heuristic-inbox verify heuristic-system/operation-records/github-pr-required-check-gating/ --format json` | `ok=true` | Migrated operation record validates. |

`scripts/check.sh --markdown` is recommended after committing this document but
not required for the review record itself; the lint surface is identical to
sibling docs already in the repo.

## Do-Not-Do / Guardrails

- Do not re-implement R1 (body redaction) in agent-kit; the boundary contract
  puts deterministic redaction in `heuristic-inbox`. agent-kit should only pin
  the released primitive, document the behavior, and keep retained cases
  strict-clean.
- Do not turn this document into an inbox entry. R1 is a primitive feature
  gap, not an unresolved workflow gap; an inbox entry would dilute the curated
  queue.
- Do not rewrite broader historical context for style. Rewriting machine-local
  prefixes in retained case body paths to `<workspace>` is allowed when needed
  for `heuristic-inbox verify --strict`.
- Do not start a multi-sprint plan from this record. The two authorised
  actions (nils-cli issue + plan-text edit) are single-shot. Use `create-plan`
  or `create-dispatch-plan` only if a larger redaction-rework or
  hand-authoring redesign emerges.

## Open Questions

- Should `docs/improvements/` become a standardised top-level docs area, or
  should this record migrate into `docs/runbooks/` after the immediate
  follow-ups close? Defer until a second improvement record appears.
- Should a future cleanup standardize or archive `docs/improvements/`? Defer;
  current implementation follow-up should stay under `docs/plans/`.
