# Plan: Plan Source Artifact Audit

## Overview

Add deterministic support for two gaps in the plan workflow lifecycle:
plan-source bundle validation and repeated durable-artifact cleanup
classification. The nils-cli side owns the reusable parser and audit primitives
inside `plan-tooling`; the agent-kit side owns skill policy, fallback behavior,
and user-facing cleanup/reporting guidance. The helper remains audit-only:
classification does not delete, move, or archive files.

## Read First

- Primary source: `docs/plans/plan-source-artifact-audit/plan-source-artifact-audit-discussion-source.md`
- Source type: discussion-to-implementation-doc
- Open questions carried into execution: none

## Scope

- In scope:
  - `nils-cli` `plan-tooling validate` bundle checks for source doc, plan, and
    execution-state linkage.
  - `nils-cli` `plan-tooling artifact-audit` as an audit-only helper emitting
    `delete`, `keep`, `rehome`, and `manual-review`.
  - `agent-kit` skill docs and tests that adopt the helper while preserving
    manual fallback behavior.
  - Cross-repo validation, completion updates, and local adoption sequencing.
- Out of scope:
  - Deleting existing durable artifacts during implementation.
  - Adding an agent-kit shell script that duplicates nils-cli behavior.
  - Releasing nils-cli or updating Homebrew before implementation validation is
    green.
  - Forcing execution-state documents to exist before execution begins.

## Assumptions

1. `plan-tooling` is the preferred nils-cli home for the first implementation
   because it already owns plan parsing, `Read First` validation, repo-root
   resolution, and planning subcommands.
2. If artifact audit usage expands beyond planning and durable coordination
   docs, the shared scanning code can later move into `nils-common` or the
   command can be split into `agent-workflow-primitives`.
3. The audit helper is advisory and conservative. Ambiguous policy, retained
   evidence, incomplete execution, or unresolved references classify as
   `manual-review` or `keep`, not `delete`.
4. Existing `docs-plan-cleanup` behavior remains the broad batch executor after
   keep/delete intent is known.
5. Agent-kit can validate against a locally installed or locally built
   `plan-tooling` binary before a formal nils-cli release is cut.

## Sprint 1: Plan Bundle Validation In nils-cli

**Goal**: Extend `plan-tooling validate` so plan-created source docs, plans, and
execution-state documents can be checked as one bundle.
**Demo/Validation**:

- Command(s):
  - `cargo test -p nils-plan-tooling validate`
  - `plan-tooling validate --file docs/plans/plan-source-artifact-audit/plan-source-artifact-audit-plan.md`
- Verify:
  - Valid sibling bundles pass without requiring execution state before work
    starts.
  - Broken source, plan, execution-state, and direct-execution waiver links fail
    with actionable messages.

**PR grouping intent**: group
**Execution Profile**: parallel-x2

### Task 1.1: Define bundle validation contract and fixtures

- **Location**:
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/README.md`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/docs/README.md`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/docs/specs/plan-source-bundle-contract-v1.md`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/tests/fixtures/plan_bundle/valid-plan.md`
- **Description**: Document the first bundle contract for
  a `docs/plans/plan-slug/` folder: source doc recommends the plan, plan
  `Read First`
  points at the source doc, optional execution state points at the plan, and
  direct source-doc execution requires an explicit waiver line.
- **Dependencies**:
  - none
- **Complexity**: 3
- **Acceptance criteria**:
  - Contract names the accepted source doc patterns such as
    `plan-slug-discussion-source.md` and `plan-slug-review-source.md`.
  - Contract defines the recommended execution-state pattern such as
    `plan-slug-execution-state.md`.
  - Contract defines the direct source-doc execution waiver line.
  - Fixtures include at least one valid not-yet-started bundle.
- **Validation**:
  - `cargo test -p nils-plan-tooling validate_bundle_contract_docs`
  - `rg -n 'Direct source-doc execution waiver|Recommended execution state' crates/plan-tooling`

### Task 1.2: Add bundle extraction helpers

- **Location**:
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/src/bundle.rs`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/src/lib.rs`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/tests/integration/validate.rs`
- **Description**: Add helper functions that derive sibling bundle paths from a
  plan path, read source doc link fields, read execution-state `Source
  document`, and detect the direct source-doc execution waiver. Keep the parser
  small and line-oriented; do not introduce a general Markdown AST.
- **Dependencies**:
  - Task 1.1
- **Complexity**: 5
- **Acceptance criteria**:
  - Helpers return stable structs suitable for text and JSON validation errors.
  - Missing optional execution state is accepted for not-yet-started bundles.
  - Existing execution state is parsed only for the fields needed by validation.
  - Tests cover Unix path separators and repo-relative path normalization.
- **Validation**:
  - `cargo test -p nils-plan-tooling bundle`
  - `cargo test -p nils-plan-tooling validate`

### Task 1.3: Wire bundle checks into validate

- **Location**:
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/src/validate.rs`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/tests/integration/validate.rs`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/tests/fixtures/plan_bundle/missing-waiver-plan.md`
- **Description**: Run bundle validation after existing plan validation. Add
  actionable errors for source docs that point to another plan, execution states
  that point to the wrong plan, and execution states that point directly to a
  source doc without a waiver.
- **Dependencies**:
  - Task 1.2
- **Complexity**: 5
- **Acceptance criteria**:
  - `validate --format json` includes bundle errors in the existing `errors`
    array.
  - Text errors name the offending file and expected sibling path.
  - Direct source-doc execution without waiver fails.
  - Direct source-doc execution with waiver passes.
- **Validation**:
  - `cargo test -p nils-plan-tooling validate_fails_without_direct_source_doc_waiver -- --exact`
  - `cargo test -p nils-plan-tooling validate_accepts_direct_source_doc_waiver -- --exact`
  - `cargo test -p nils-plan-tooling validate -- --exact`

### Task 1.4: Update validate docs and examples

- **Location**:
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/README.md`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/docs/README.md`
  - `../../Project/sympoies/nils-cli/docs/specs/workspace-doc-retention-matrix-v1.md`
- **Description**: Add user-facing `validate` docs for bundle checks and
  explain that execution-state files are validated when present, not required
  before execution starts.
- **Dependencies**:
  - Task 1.3
- **Complexity**: 2
- **Acceptance criteria**:
  - README examples mention bundle validation in the `validate` section.
  - Docs distinguish plan-source bundle checks from cleanup audit.
  - Retention matrix stays consistent with plan-created source docs as temporary
    coordination artifacts.
- **Validation**:
  - `bash scripts/ci/nils-cli-checks-entrypoint.sh --docs-only`

## Sprint 2: Durable Artifact Audit Helper In nils-cli

**Goal**: Add a deterministic audit-only helper that classifies durable
coordination artifacts without deleting or moving them.
**Demo/Validation**:

- Command(s):
  - `cargo test -p nils-plan-tooling artifact_audit`
  - `plan-tooling artifact-audit --candidate docs/plans/example/example-plan.md --format json`
- Verify:
  - Output contains only `delete`, `keep`, `rehome`, and `manual-review`
    classifications with stable reasons.

**PR grouping intent**: group
**Execution Profile**: parallel-x2

### Task 2.1: Add artifact-audit command contract

- **Location**:
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/src/artifact_audit.rs`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/src/usage.rs`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/src/completion.rs`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/README.md`
- **Description**: Add the `plan-tooling artifact-audit` CLI surface with
  `--candidate`, `--candidate-file`, `--repo`, `--format text|json`, and
  `--explain`. Define a versioned JSON schema and text report that expose
  classification, reason, evidence, and blocking references.
- **Dependencies**:
  - Task 1.2
- **Complexity**: 4
- **Acceptance criteria**:
  - The command is audit-only and has no `--execute` option.
  - JSON output includes `schema_version`, `ok`, `root`, `items`, and
    per-item classification details.
  - Help text documents the four classifications exactly.
  - Completion definitions include the new subcommand and flags.
- **Validation**:
  - `cargo test -p nils-plan-tooling artifact_audit_usage`
  - `plan-tooling artifact-audit --help`

### Task 2.2: Implement conservative classification logic

- **Location**:
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/src/artifact_audit.rs`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/src/repo_root.rs`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/tests/integration/artifact_audit.rs`
- **Description**: Classify candidate Markdown coordination docs using status
  markers, sibling bundle state, execution completion, retained-evidence
  locations, and `rg`/git reference scans. Prefer `manual-review` when policy
  is unclear and `keep` when active references or incomplete execution remain.
- **Dependencies**:
  - Task 2.1
- **Complexity**: 6
- **Acceptance criteria**:
  - Completed unreferenced plan bundles can classify as `delete`.
  - Active, blocked, in-progress, or externally referenced artifacts classify as
    `keep`.
  - Artifacts with useful retained content but obsolete coordination location
    classify as `rehome`.
  - HEURISTIC_SYSTEM records, raw evidence, generated logs, and ambiguous
    status classify as `manual-review`.
- **Validation**:
  - `cargo test -p nils-plan-tooling artifact_audit_classifies_completed_bundle -- --exact`
  - `cargo test -p nils-plan-tooling artifact_audit_keeps_referenced_bundle -- --exact`
  - `cargo test -p nils-plan-tooling artifact_audit_manual_reviews_retained_evidence -- --exact`

### Task 2.3: Add fixtures, integration module, and generated completions

- **Location**:
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/tests/integration.rs`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/tests/fixtures/artifact_audit/completed-bundle.md`
  - `../../Project/sympoies/nils-cli/completions/zsh/_plan-tooling`
  - `../../Project/sympoies/nils-cli/completions/bash/plan-tooling`
- **Description**: Add fixture-backed integration coverage and regenerate shell
  completions after the new subcommand is wired into clap completion metadata.
- **Dependencies**:
  - Task 2.1
  - Task 2.2
- **Complexity**: 3
- **Acceptance criteria**:
  - Integration tests are registered in `tests/integration.rs`.
  - Completion assets include `artifact-audit` and its flags.
  - Completion syntax checks pass for both shells.
- **Validation**:
  - `cargo test -p nils-plan-tooling artifact_audit`
  - `zsh -n completions/zsh/_plan-tooling`
  - `bash -n completions/bash/plan-tooling`

## Sprint 3: Agent-Kit Skill Adoption

**Goal**: Update agent-kit policy skills to consume nils-cli audit output while
keeping manual fallback and deletion approval boundaries clear.
**Demo/Validation**:

- Command(s):
  - `scripts/check.sh --tests -- -k 'plan_workflow or durable_artifact_cleanup or docs_plan_cleanup'`
  - `scripts/check.sh --docs`
- Verify:
  - Skills describe the helper boundary and use the normalized classification
    classes.

**PR grouping intent**: group
**Execution Profile**: parallel-x2

### Task 3.1: Adopt normalized cleanup classes in durable-artifact-cleanup

- **Location**:
  - `skills/workflows/plan/durable-artifact-cleanup/SKILL.md`
  - `skills/workflows/plan/durable-artifact-cleanup/tests/test_workflows_plan_durable_artifact_cleanup.py`
  - `skills/workflows/plan/README.md`
- **Description**: Update cleanup output language to use `delete`, `keep`,
  `rehome`, and `manual-review`. Document that `plan-tooling artifact-audit`
  should be used when available for repeated candidate classification, while
  the skill remains responsible for policy interpretation and final deletion
  approval.
- **Dependencies**:
  - Task 2.3
- **Complexity**: 3
- **Acceptance criteria**:
  - `archive-or-rehome` is no longer the canonical report class.
  - The skill says helper output is advisory, not an auto-delete authority.
  - Manual audit fallback remains documented for environments without the new
    nils-cli command.
- **Validation**:
  - `scripts/check.sh --tests -- skills/workflows/plan/durable-artifact-cleanup/tests/test_workflows_plan_durable_artifact_cleanup.py`
  - `scripts/check.sh --docs`

### Task 3.2: Clarify docs-plan-cleanup executor boundary

- **Location**:
  - `skills/workflows/plan/docs-plan-cleanup/SKILL.md`
  - `skills/workflows/plan/docs-plan-cleanup/tests/test_workflows_plan_docs_plan_cleanup.py`
  - `tests/test_plan_workflow_lifecycle_docs.py`
- **Description**: Update docs-plan-cleanup guidance so broad cleanup can be
  driven by a prior `plan-tooling artifact-audit` report, but the script still
  only executes known keep/delete intent.
- **Dependencies**:
  - Task 3.1
- **Complexity**: 2
- **Acceptance criteria**:
  - The skill distinguishes deterministic audit helper output from deterministic
    batch deletion.
  - Tests assert that ambiguous classification still routes to
    `durable-artifact-cleanup`.
  - The existing docs-plan-cleanup report format remains unchanged.
- **Validation**:
  - `scripts/check.sh --tests -- skills/workflows/plan/docs-plan-cleanup/tests/test_workflows_plan_docs_plan_cleanup.py`
  - `scripts/check.sh --tests -- tests/test_plan_workflow_lifecycle_docs.py`

### Task 3.3: Make source, plan, and state links machine-checkable in skills

- **Location**:
  - `skills/workflows/plan/create-plan/SKILL.md`
  - `skills/workflows/plan/execute-from-plan/SKILL.md`
  - `skills/workflows/plan/_shared/references/PLAN_AUTHORING_BASELINE.md`
  - `skills/workflows/conversation/discussion-to-implementation-doc/SKILL.md`
  - `skills/workflows/conversation/review-to-improvement-doc/SKILL.md`
- **Description**: Add stable wording for recommended plan, recommended
  execution-state path, execution-state `Source document`, and direct
  source-doc execution waiver lines so future source docs and state ledgers are
  easy for `plan-tooling validate` to check.
- **Dependencies**:
  - Task 1.4
  - Task 3.1
- **Complexity**: 4
- **Acceptance criteria**:
  - Source-doc skills recommend sibling plan and execution-state paths using
    stable labels.
  - `execute-from-plan` execution-state template includes the waiver label with
    `not applicable` for normal plan-driven execution.
  - `create-plan` baseline explains that bundle validation expects the plan
    `Read First` primary source to point to the sibling source doc.
- **Validation**:
  - `scripts/check.sh --tests -- -k 'create_plan or execute_from_plan or discussion_to_implementation_doc or review_to_improvement_doc'`
  - `scripts/check.sh --docs`

## Sprint 4: Cross-Repo Validation And Adoption

**Goal**: Prove the new nils-cli primitives work from agent-kit workflows and
document the release/adoption sequence.
**Demo/Validation**:

- Command(s):
  - `NILS_CLI_TEST_RUNNER=nextest bash scripts/ci/nils-cli-checks-entrypoint.sh`
  - `scripts/check.sh --all`
- Verify:
  - nils-cli passes required checks, agent-kit passes maintenance checks, and
    agent-kit can invoke the new local `plan-tooling` behavior.

**PR grouping intent**: group
**Execution Profile**: parallel-x2

### Task 4.1: Run nils-cli required validation

- **Location**:
  - `../../Project/sympoies/nils-cli/DEVELOPMENT.md`
  - `../../Project/sympoies/nils-cli/scripts/ci/nils-cli-checks-entrypoint.sh`
  - `../../Project/sympoies/nils-cli/crates/plan-tooling/README.md`
- **Description**: Run the required nils-cli validation set for the
  `plan-tooling` changes, including completion syntax checks when generated
  completion assets changed.
- **Dependencies**:
  - Task 1.4
  - Task 2.3
- **Complexity**: 2
- **Acceptance criteria**:
  - Required nils-cli checks pass or failures are recorded with concrete
    follow-up.
  - Completion-specific checks are run after subcommand/flag changes.
  - `plan-tooling --version` and help output reflect the expected local binary.
- **Validation**:
  - `NILS_CLI_TEST_RUNNER=nextest bash scripts/ci/nils-cli-checks-entrypoint.sh`
  - `zsh -f tests/zsh/completion.test.zsh`
  - `zsh -n completions/zsh/_plan-tooling`
  - `bash -n completions/bash/plan-tooling`

### Task 4.2: Validate agent-kit against local plan-tooling

- **Location**:
  - `DEVELOPMENT.md`
  - `tests/test_plan_workflow_lifecycle_docs.py`
  - `docs/plans/plan-source-artifact-audit/plan-source-artifact-audit-plan.md`
- **Description**: Install or point PATH at the locally built nils-cli binary,
  then validate the new plan-source bundle and agent-kit skill docs with the
  updated `plan-tooling`.
- **Dependencies**:
  - Task 3.3
  - Task 4.1
- **Complexity**: 3
- **Acceptance criteria**:
  - `plan-tooling validate` passes for this plan bundle.
  - Agent-kit plan workflow tests pass.
  - `scripts/check.sh --all` passes before delivery.
- **Validation**:
  - `plan-tooling validate --file docs/plans/plan-source-artifact-audit/plan-source-artifact-audit-plan.md`
  - `scripts/check.sh --tests -- -k 'plan_workflow or durable_artifact_cleanup or docs_plan_cleanup'`
  - `scripts/check.sh --all`

### Task 4.3: Document release and fallback sequence

- **Location**:
  - `CHANGELOG.md`
  - `README.md`
  - `../../Project/sympoies/nils-cli/README.md`
  - `../../Project/sympoies/nils-cli/CHANGELOG.md`
- **Description**: Record the cross-repo adoption order: nils-cli primitive
  lands first, agent-kit consumes it with local PATH validation, then nils-cli
  release/Homebrew adoption can happen through the normal release workflow.
- **Dependencies**:
  - Task 4.1
  - Task 4.2
- **Complexity**: 2
- **Acceptance criteria**:
  - Agent-kit docs mention manual fallback when the new `plan-tooling`
    subcommand is unavailable.
  - nils-cli docs mention the new validation/audit functionality.
  - Release notes do not claim Homebrew availability until release is actually
    complete.
- **Validation**:
  - `scripts/check.sh --docs`
  - `bash scripts/ci/nils-cli-checks-entrypoint.sh --docs-only`

## Testing Strategy

- Unit: nils-cli helper-level tests for bundle path derivation, source-doc link
  extraction, waiver detection, and audit classification decisions.
- Integration: fixture-backed `plan-tooling validate` and `artifact-audit`
  tests with valid, broken, direct-execution, completed, referenced, and
  retained-evidence bundles.
- Docs/policy: agent-kit skill tests for cleanup class names, helper fallback,
  docs-plan-cleanup boundaries, and stable source/plan/state labels.
- E2E/manual: run `plan-tooling validate` and `plan-tooling artifact-audit`
  against a small agent-kit `docs/plans/` candidate set before using the report
  in a real cleanup session.

## Risks & gotchas

- Cross-repo sequencing matters: agent-kit should not require a nils-cli
  subcommand until the local binary has been built or released.
- The audit helper can produce false confidence if it treats missing evidence as
  safe deletion. Missing or ambiguous evidence must classify as
  `manual-review`.
- Adding the new `artifact-audit` subcommand changes completion assets. The
  nils-cli completion checks must be part of the implementation, not an
  afterthought.
- Existing historical plans may not follow the new bundle labels. The first
  implementation should validate sibling bundles conservatively and avoid
  forcing old plan-only waivers into new state conventions.
- Agent-kit docs should not duplicate nils-cli parsing rules in prose-heavy
  detail. The durable contract lives in nils-cli tests and plan-tooling docs;
  agent-kit should describe when to call it.

## Rollback plan

- If `plan-tooling validate` bundle checks are too noisy, keep the parser
  helpers and gate bundle checks behind a follow-up flag while agent-kit keeps
  manual source/plan/state verification.
- If `artifact-audit` proves too broad for `plan-tooling`, move the command
  surface to a dedicated nils-cli primitive and keep the shared path/reference
  logic reusable.
- If agent-kit adoption blocks on nils-cli release timing, keep the updated
  skill wording as "use when available" and retain the existing manual
  `durable-artifact-cleanup` workflow until the binary is available on PATH.
