# Serena Self-Eval Autoresearch Loop

10 iterations of prompt-tuning driven by the agent's own tool-vs-built-ins
evaluation report (generated by `CliClaudeIntegrationTest.testSerenaSelfEvalPrompt`).

## Constraint

Only change MCP Steroid's **prompts/resources** (files under `prompts/src/main/prompts/`).
No tool code, no Kotlin, no tool-handler changes. The self-eval prompt itself
(`ij-plugin/src/integrationTest/resources/serena/self-eval-prompt.md`) is
frozen — it must stay byte-for-byte identical to Serena's upstream copy.

## Loop

Per iteration N ∈ {0..9}:

1. Build & run `./gradlew :ij-plugin:integrationTest --tests '*CliClaudeIntegrationTest.testSerenaSelfEvalPrompt' --rerun-tasks`.
2. Extract the agent's report (`EVAL_REPORT_START` → `EVAL_REPORT_END`) from the
   test stdout into `iter-NN/report.md`.
3. Read `iter-NN/report.md`. List:
   - **gaps**: category-(b) / negative-delta tasks where the agent said Serena/MCP-Steroid
     applies but offers no improvement, OR produced an error instead of an answer.
   - **confusion**: steps where the agent says it didn't know which tool to call, or
     called one tool when another would have been better.
   - **missing recipes**: refactoring items flagged "no suitable candidate" that are
     actually supported by MCP Steroid but the agent didn't know.
4. Write `iter-NN/analysis.md` — the 1–3 concrete prompt edits you are about to make
   to address the top gap. Karpathy-style: one narrow change per iteration, not a rewrite.
5. Apply the prompt edits under `prompts/src/main/prompts/`. Commit with message
   `autoresearch: iter NN — <short gap summary>`.
6. Advance to iteration N+1.

## Output files

- `iter-NN/report.md` — raw EVAL_REPORT from the run
- `iter-NN/analysis.md` — identified gap + planned edit
- `iter-NN/diff.patch` — `git diff` of the prompt edits the agent made
- `summary.md` — running table of report quality per iteration (# of category-(a)
  findings, # of category-(b), # of "no suitable candidate")

## Value signal

MCP Steroid's evaluation improves when:

- **More** (a)-category findings (clearly unique capability)
- **Fewer** (b)-category findings (apply-but-no-improvement)
- **Fewer** "no suitable candidate" where a recipe actually exists
- **Fewer** tool-misuse episodes in the "Detailed evidence" section
