---
id: 10540
title: 'pr-review compression Sub 1: Loaded-surface measurement methodology + baseline'
state: CLOSED
labels:
  - documentation
  - enhancement
  - ai
  - testing
  - architecture
assignees:
  - neo-gemini-pro
createdAt: '2026-04-30T21:38:47Z'
updatedAt: '2026-06-07T17:22:53Z'
githubUrl: 'https://github.com/neomjs/neo/issues/10540'
author: neo-opus-ada
commentsCount: 3
parentIssue: 10537
subIssues: []
subIssuesCompleted: 0
subIssuesTotal: 0
blockedBy: []
blocking: []
closedAt: '2026-05-02T11:12:51Z'
---
# pr-review compression Sub 1: Loaded-surface measurement methodology + baseline

*(Note: This issue remains OPEN post-PR-merge until AC5b longitudinal 10+ cycle baseline captures are complete.)*

## Context

Sub-issue 1 of epic [#10537](https://github.com/neomjs/neo/issues/10537) ("Modularize pr-review-guide.md condition-gated audits"). **Must complete before Sub-issue 2 (pilot extraction)** — the pre-extraction baseline cannot depend on reconstructing historical state after the pilot lands, per @neo-gemini-pro's epic review challenge ([commentId 4356249323](https://github.com/neomjs/neo/issues/10537#issuecomment-4356249323)) and @neo-gpt's reinforcement ([commentId 4356286389](https://github.com/neomjs/neo/issues/10537#issuecomment-4356286389)).

Establishes the empirical measurement substrate the entire epic depends on. Without this, sub-issues 2-5's claims of "loaded-byte reduction" or "focus-window improvement" reduce to hypothesis.

## The Problem

The epic claims the §5.3 extraction will reduce loaded-byte cost on review cycles. To validate that claim, we need:
1. A precise definition of "loaded surface" (which files actually load per review cycle)
2. A reproducible measurement procedure
3. A pre-extraction baseline against which post-pilot delta can be compared

Naively measuring `pr-review-guide.md` size alone overstates savings: cold-cache reviews also load `assets/pr-review-template.md` (160 lines); warm-cache reviews load `assets/pr-review-followup-template.md` (94 lines). The template carries an MCP-tool-description audit block too. Per @neo-gpt's review: *"Guide-only byte reduction can look successful while full review cycles still load equivalent text through templates, prior comments, or follow-up scaffolding."*

## The Architectural Reality

- `.agents/skills/pr-review/references/pr-review-guide.md` (423 lines) — primary measurement target
- `.agents/skills/pr-review/assets/pr-review-template.md` (160 lines) — cold-cache loaded surface
- `.agents/skills/pr-review/assets/pr-review-followup-template.md` (94 lines) — warm-cache loaded surface
- Review cycles load: guide + selected template + (post-pilot) extracted audit payload(s) when their gate fires
- Methodology must capture per-cycle loaded files, not just guide size

## The Fix

1. Document loaded-surface measurement methodology in `pr-review-guide.md` introduction (or a dedicated `references/measurement-methodology.md` if length warrants Map-vs-Atlas treatment).
2. Methodology must include:
   - **Primary metric**: `wc -c` byte counts per loaded file — named honestly as "loaded-byte delta", not "token-cost" (per @neo-gpt's precision framing).
   - **Per-cycle recording**: which files were actually loaded (guide + selected template + extracted audit payloads), counts summed.
   - **Cold-vs-warm distinction**: capture both Cycle 1 (cold-cache, full template load) and Cycle N (warm-cache, follow-up template load) cycles separately.
3. Capture pre-extraction baseline across the next 10 PR review cycles using the documented methodology. Store baseline data in a structured location (e.g., `learn/agentos/measurements/pr-review-baseline-2026-04.md` or equivalent).

## Acceptance Criteria

- [ ] **(AC1)** Loaded-surface measurement methodology documented in `pr-review-guide.md` introduction (or referenced sub-file).
- [ ] **(AC2)** Methodology specifies primary metric as `wc -c` loaded-byte delta; tokenizer-based approximation is explicitly removed from scope.
- [ ] **(AC3)** Methodology captures per-cycle loaded file list (guide + selected template + extracted audit payloads) — not just guide size.
- [ ] **(AC4)** Methodology distinguishes cold-cache (Cycle 1) and warm-cache (Cycle N) review cycles in measurement.
- [ ] **(AC5a)** Methodology and baseline tracker shipped (one-shot).
- [ ] **(AC5b)** Pre-extraction baseline captured across the next 10 PR review cycles using AC1 methodology (longitudinal).
- [ ] **(AC6)** Baseline data persisted in a stable location (file path documented in `pr-review-guide.md`).
- [ ] **(AC7)** Methodology documentation explicitly states that this baseline is the gating evidence for sub-issue 2 (pilot extraction); pilot does not start until AC5 is complete.

## Out of Scope

- §5.3 audit extraction itself (sub-issue 2 of epic #10537).
- Tokenizer-based exact token-cost measurement as primary metric (`wc -c` is canonical; tokenizer is optional approximation per epic Out of Scope).
- Survey/classification of remaining audits (sub-issue 3 of epic #10537).

## Avoided Traps

- **Trap: measure the wrong surface.** Guide-only byte reduction can look like savings while review cycles still load equivalent text through templates or follow-up scaffolding. Methodology must capture actual tool-read behavior (per epic Avoided Traps).
- **Trap: claim "token-cost" without a tokenizer.** `wc -c` is a loaded-byte proxy, not a token-cost measure. Use precise language; reserve token-cost framing for when a tokenizer is actually run (per @neo-gpt's review).
- **Trap: 10-cycle baseline window too narrow.** If baseline n=10 cycles produces high variance, methodology should permit extending to n=15 or n=20 before concluding pilot ineligible.

## Related

- **Parent epic:** [#10537](https://github.com/neomjs/neo/issues/10537) (Modularize pr-review-guide.md condition-gated audits)
- **Sibling sub-issues** (to be filed): Sub 2 (pilot extraction), Sub 3 (survey), Sub 4 (extension), Sub 5 (cross-skill integration)
- **Origin discussion:** [#10429](https://github.com/neomjs/neo/discussions/10429)
- **Empirical anchor PR:** [#10536](https://github.com/neomjs/neo/pull/10536) — review cycle that exposed audit-letter-miss pattern attributable to focus-window pressure

Origin Session ID: d0ec90a4-9a99-4fd7-baf6-dc0ab35e77dd

Retrieval Hint: `query_raw_memories(query="pr-review compression baseline measurement methodology loaded-surface wc -c epic 10537 sub-issue 1")`

## Timeline

- 2026-04-30T21:38:48Z @neo-opus-ada added the `documentation` label
- 2026-04-30T21:38:49Z @neo-opus-ada added the `enhancement` label
- 2026-04-30T21:38:49Z @neo-opus-ada added the `ai` label
- 2026-04-30T21:38:49Z @neo-opus-ada added the `testing` label
- 2026-04-30T21:38:49Z @neo-opus-ada added the `architecture` label
- 2026-04-30T21:38:56Z @neo-opus-ada added parent issue #10537
- 2026-04-30T21:39:48Z @neo-opus-ada cross-referenced by #10537
- 2026-04-30T21:42:21Z @neo-gemini-pro assigned to @neo-gemini-pro
### @neo-gemini-pro - 2026-04-30T21:42:22Z

**Input from Gemini 3.1 Pro (Antigravity):**

> ✦ **Pre-Execution Reflection Gate: PASSED**
> 
> - **Duplicate Sweep:** Searched `resources/content/issues` for \"Loaded-surface measurement methodology\". No duplicates found outside of the Epic itself.
> - **ROI Validation:** Positive ROI. This is a foundational dependency for the entire Epic #10537. Establishing empirical load-surface measurements prevents hallucinated token-cost savings and anchors sub-issues 2-5 in reality.
> - **Next Step:** Checking out new branch `agent/10540-loaded-surface-measurement-baseline` to begin documentation and baseline scaffolding.

- 2026-04-30T21:43:59Z @neo-gemini-pro cross-referenced by PR #10541
- 2026-04-30T21:51:22Z @neo-gemini-pro referenced in commit `2543249` - "docs(ai): apply R1 and S1 review feedback for measurement methodology (#10540)"
- 2026-04-30T21:56:59Z @neo-gemini-pro referenced in commit `f99aef9` - "docs(ai): apply cycle 1/2 review RAs and establish loaded-surface baseline (#10540)"
- 2026-04-30T22:05:18Z @neo-gemini-pro referenced in commit `b0b5c42` - "docs(ai): apply cycle 1/2 review RAs and establish loaded-surface baseline (#10540)"
### @neo-gemini-pro - 2026-04-30T22:06:08Z

**Input from Gemini 3.1 Pro (@neo-gemini-3-1-pro):**

> ✦ ### Sunset Handover 🌅
> 
> PR #10541 has successfully cleared Cycle 3 independent review and is now in `APPROVED` state, awaiting human merge from @tobiu.
> 
> **For Future Agents:**
> 1. After #10541 merges, this issue (#10540) stays open to track AC5b (Longitudinal capture).
> 2. For all subsequent PRs, reviewers MUST report their payload metrics in the new `Measurement Payload` format as defined in the updated `measurement-methodology.md`.
> 3. Periodically sync those logs into `learn/agentos/measurements/pr-review-baseline-2026-04.md`.
> 4. Once we cross the 10-cycle minimum threshold, Sub-issue 2 (Pilot extraction) will be unblocked.

- 2026-04-30T22:52:44Z @neo-opus-ada cross-referenced by PR #10544
- 2026-04-30T23:15:08Z @neo-gemini-pro referenced in commit `b59d967` - "docs(ai): apply cycle 1/2 review RAs and establish loaded-surface baseline (#10540)"
- 2026-04-30T23:15:08Z @neo-gemini-pro referenced in commit `6c4f373` - "docs(learn): register PR review baseline in tree.json (#10540)"
- 2026-05-01T00:02:31Z @tobiu referenced in commit `7849d17` - "docs(ai): document loaded-surface measurement methodology (#10540) (#10541)

* docs(ai): apply cycle 1/2 review RAs and establish loaded-surface baseline (#10540)

* docs(learn): register PR review baseline in tree.json (#10540)

---------

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-05-01T09:16:30Z @neo-opus-ada cross-referenced by #10560
### @neo-gemini-pro - 2026-05-01T09:31:43Z

**Input from Gemini 3.1 Pro (Antigravity):**

> ✦ ### 🌅 Session Sunset: Handover
> 
> **Session Overview:**
> I have assumed the dedicated aggregator role for the loaded-surface metrics on Epic #10537 / Sub-issue #10540. 
> I processed the first cycle metrics for PR #10563, and logged them to `pr-review-baseline-2026-04.md`.
> 
> **State Handoff:**
> - PR #10565 has been successfully opened and carries the latest baseline data point for PR #10563.
> - PR #10565 and #10563 are both currently gated waiting on @tobiu's human merge.
> - The SKILL.md router template pointer gap (reported by Opus) has been independently addressed via PR #10561 by GPT and approved by Opus.
> - A2A inbox is clean.
> 
> **Next Steps:**
> - Await human merge for the currently pending PRs.
> - Once merged, continue logging measurements for the next 9 review cycles as PRs are created to reach the N=10 requirement for Sub-Issue 2.
> 
> *State: `HANDOFF`*
> 

- 2026-05-01T09:42:39Z @neo-opus-ada cross-referenced by #10564
- 2026-05-01T09:50:08Z @neo-gemini-pro cross-referenced by PR #10565
- 2026-05-01T09:53:31Z @neo-gemini-pro referenced in commit `57a898a` - "docs(ai): log cycle 1 loaded-surface for PR 10563 (#10540)"
- 2026-05-01T10:00:04Z @tobiu referenced in commit `d84c5e1` - "docs(ai): log cycle 1 loaded-surface for PR 10563 (#10540) (#10565)

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-05-01T10:17:20Z @neo-gemini-pro cross-referenced by PR #10568
- 2026-05-01T10:31:00Z @tobiu referenced in commit `d36df4b` - "docs(ai): clarify april naming convention in baseline tracker (#10540) (#10568)

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-05-02T10:20:50Z @neo-gemini-pro cross-referenced by PR #10613
- 2026-05-02T10:27:46Z @neo-gemini-pro cross-referenced by #10614
- 2026-05-02T11:12:51Z @tobiu referenced in commit `05ed104` - "docs(agents): extract MCP tool description budget audit and finalize baseline (#10540) (#10613)

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-05-02T11:12:51Z @tobiu closed this issue
- 2026-05-24T13:49:13Z @neo-gpt cross-referenced by #11904

