---
name: boundary-audit
description: "Use when running the quarterly boundary audit of this repo's external assumptions — inventory the write/read/docs surfaces against the conformance ledger, track the verification-class trend, retro-check recent bugs for missing class-level ratchets, and spot-check docs freshness. Produces a dated report under docs/audits/ plus filed follow-up issues. Repo-maintenance skill, not a finance skill."
---

# Boundary Audit

Quarterly assumption audit (Epic D, issue #445). Gates catch anticipated drift;
this ritual catches the unanticipated. It reads the conformance ledger
(`src/conformance/ledger.ts`), the codebase, and recent GitHub history — never
the user's financial data.

## When to use

- A quarter (~90 days) has passed since the newest report in `docs/audits/`
- After a boundary incident (a bug caused by Copilot's server or cache drifting
  from our local model), to retro-check the whole boundary, not just the
  incident
- The user says "boundary audit" or asks how trustworthy our external
  assumptions currently are

## Do NOT use if

- The user wants to verify one specific enum/field against production → run the
  relevant `scripts/smoke/` script (`bun run smoke`) instead
- The user wants to audit their transactions or finances → `/finance-cleanup`
  or `/finance-pulse`; this skill never touches financial data

## Guardrails

1. **Read-only toward Copilot.** The audit reads source, the ledger, and GitHub.
   Never call MCP write tools, never run live probes against
   `app.copilot.money` — if an assumption needs a live probe, that is a filed
   follow-up issue, not an audit step.
2. **The report is committed — PII rules apply.** No real financial figures,
   account balances, account names, or transaction details anywhere in the
   report or in filed issues. The audit's subject matter is schemas and
   processes, so this should be natural; if an example needs a figure, use a
   placeholder ("$X", synthetic numbers).
3. **Findings become issues, not inline fixes.** The audit PR contains the
   report (and nothing else). Every actionable finding is filed as a GitHub
   issue; fixing them happens in their own PRs. Exception: none — even
   one-line doc staleness gets an issue, so the report's findings list and the
   issue tracker stay 1:1.
4. **Dedupe before filing.** Search open issues (`gh issue list --search`)
   before creating one; if an existing issue covers the finding, reference it
   in the report instead of filing a duplicate.
5. **No invented history.** Phase 3 reviews only PRs/issues that actually
   exist; quote numbers, not memories.

## Phase 0 — Baseline

1. Run from the repo root; `bun install` if `node_modules` is missing.
2. Find the previous report: newest `docs/audits/*-boundary-audit.md`. Note its
   date (`LAST_AUDIT`) and its recorded class distribution. If no report
   exists, this is the first run: set `LAST_AUDIT` to 90 days ago and treat
   every "vs last audit" comparison as "baseline established".
3. Confirm the ledger gate is green (it also runs in CI):

   ```bash
   bun test tests/conformance/
   ```

## Phase 1 — Inventory: assumptions vs ledger

Enumerate every class of external assumption and diff against the ledger.
Anything external, new, and unlisted is a finding.

1. **Dump the ledger's surfaces** for comparison:

   ```bash
   bun -e "const m = await import('./src/conformance/ledger.ts'); for (const e of m.CONFORMANCE_LEDGER) console.log(e.class.padEnd(13), e.kind.padEnd(14), e.surface);"
   ```

2. **Enumerate every GraphQL operation the code can send.** All operation
   documents live in `src/core/graphql/operations.generated.ts` (generated by
   `bun run generate:graphql` from `docs/graphql-capture/operations/`):

   ```bash
   grep -hoE "(query|mutation) [A-Za-z0-9_]+" src/core/graphql/operations.generated.ts | sort -u
   ```

   Note the operation *name* (e.g. `EditBudget`) can differ from the mutation
   *field* name the ledger uses (`Mutation.editCategoryBudget`) — resolve via
   the wrapper module in `src/core/graphql/` before declaring a mismatch.

3. **Write surface (mutations).** Bidirectional ledger coverage is enforced by
   `tests/conformance/ledger.test.ts` (Phase 0 confirmed it green), so this is
   automated — but check the escape hatch: GraphQL wrapper modules whose
   mutations have no MCP tool (no `toolParams`, e.g. `editAccount`) are NOT
   caught by the test. Confirm each mutation from step 2 appears in the
   ledger dump.

4. **Read surface (queries).** The ledger test does not reach these. Diff the
   `query` operations from step 2 against the ledger. Each query operation,
   its response shape (the hand-written TS interfaces in
   `src/core/graphql/queries/*.ts`), and any enum/string-union it sends (see
   `src/core/graphql/queries/_shared.ts`) is an external assumption. Unlisted
   → finding.

5. **Cache decoder surface.** `src/core/decoder.ts` assumes Copilot's local
   Firestore/LevelDB doc shapes — a different external boundary with its own
   controls (`warnUnreadFields`, `bun run scripts/decode-coverage.ts`). Note in
   the report whether those controls still run and whether any new collection
   processor was added since `LAST_AUDIT` without coverage. Unledgered cache
   assumptions are a finding only if no control covers them.

6. **Docs facts.** External-facing factual claims in CLAUDE.md / README.md /
   CONTRIBUTING.md (endpoints, paths, counts, flag behavior) are assumptions
   too; the freshness check is Phase 4 — here, only note any claim about
   Copilot's API that the ledger should own but doesn't.

## Phase 2 — Class distribution trend

The ledger is the source; the trend is the health metric.

```bash
bun -e "const m = await import('./src/conformance/ledger.ts'); console.log(m.formatClassDistribution());"
```

Record the table in the report next to `LAST_AUDIT`'s numbers. Findings:

- `unverified` count grew with no corresponding open follow-up issue
- `gated` share shrank (an oracle was removed or downgraded)
- Any `unverified` entry that has survived two consecutive audits with no
  filed issue (i.e. it appeared `unverified` in the previous report and
  remains `unverified` now) → file one (per-entry, with the surface name in
  the title)

## Phase 3 — Bug-class review (retroactive ratchet check)

Apply the Bug Response Ritual questions (CONTRIBUTING.md, from D1 #444) to
every incident since `LAST_AUDIT` — read them together, not one-by-one;
bug-class patterns only show up in the aggregate.

1. Collect incidents:

   ```bash
   gh pr list --search "merged:>=LAST_AUDIT" --limit 200 --json number,title,mergedAt
   gh issue list --state all --label bug --search "created:>=LAST_AUDIT" --json number,title,state
   ```

   Keep PRs whose titles/bodies are `fix:`-shaped or reference a bug issue.
   Don't trust the `bug` label alone — bug reports sometimes land unlabeled,
   so also scan the merged-PR list for incident-shaped titles (`fix`,
   `regression`, "wrong", "missing") regardless of label.

2. For each incident, answer in the report (one table row each):
   - **Class** — name the bug *class*, not the instance (e.g. "local enum
     transcribed wrong", "name→ID resolution skipped", "nullability stricter
     than reality")
   - **Detector?** — does a *class-level* detector now exist (smoke gate,
     ledger oracle, schema test, lint rule)? Cite it, or mark "instance-only
     regression test" / "none"
   - **Siblings?** — was the same class checked across sibling surfaces, or
     only where it bit?

3. Missing class-level detector or unchecked siblings → finding → issue,
   linking the "Bug Response Ritual" template in CONTRIBUTING.md (the audit
   questions above are that template, applied retroactively).

## Phase 4 — Docs freshness

Beyond what `tests/unit/doc-sync.test.ts` automates (tool-count phrases,
removed-path references), spot-check the rest of CLAUDE.md against the code:

- Every file/directory path mentioned exists (`ls` them)
- Every `bun run <script>` mentioned exists in `package.json` `scripts`, and
  prose descriptions of compound scripts (e.g. what `check` runs) match the
  actual script definition
- Headline behavior claims (which flags imply which, which tools swap in
  live mode, database location shape) match `src/server.ts` / `src/cli.ts`
- README.md and CONTRIBUTING.md: same treatment for their headline sections

Each stale fact is a finding. If a whole *category* of fact keeps rotting,
also consider a finding for "automate this in doc-sync" (that's the ratchet).

## Phase 5 — Report

Write `docs/audits/<YYYY-MM-DD>-boundary-audit.md` (today's date) using this
structure:

```markdown
# Boundary audit — <YYYY-MM-DD>

Previous audit: <date or "none (first run)">. Window reviewed: <range>.

## 1. Inventory diff (assumptions vs ledger)
<per-surface-class result: covered / uncovered, with counts>

## 2. Class distribution
| class | now | last audit | delta |
<table + one-line trend verdict>

## 3. Bug-class review
| incident | class | class-level detector | siblings checked |
<table + aggregate pattern observations>

## 4. Docs freshness
<stale facts found, or "clean">

## 5. Findings & filed issues
| # | finding | severity | issue |
<every finding, each with an issue number or an explicit "not filed because…">
```

PII check before commit: grep the report for `\$[0-9]`, 4-digit-plus integers
in financial context, and account/bank names.

## Phase 6 — File issues and open the PR

1. File one issue per actionable finding (after the Guardrail-4 dedupe
   search). Title prefix `boundary-audit:`; label `audit` plus `area:boundary`
   when it concerns the ledger/GraphQL surface, `documentation` for docs rot;
   body links the report path and the audit date.
2. Backfill the issue numbers into the report's findings table.
3. Branch, commit the report (only the report; commit-message conventions —
   trailers, PII rules — per CLAUDE.md), open a ready-for-review PR
   titled `audit: boundary audit <YYYY-MM-DD>`, and reference the filed issues
   in the body (plain references — do not write "Closes #N"; the issues stay
   open until their own PRs land).
