---
title: eval-author
description: Turn acceptance criteria into eval fixtures (JSONL cases + rubric) wired to smithers eval.
---

`eval-author` turns plain-English acceptance criteria into a runnable Smithers
eval suite. It derives a set of cases (each an input plus an `expected` assertion
and a review rubric), then writes a `.jsonl` fixture under `.smithers/evals/` and
reports the exact `bunx smithers-orchestrator eval` command to run it. Use it when you have a goal
or acceptance criteria and want a repeatable, regression-safe check for a workflow.

```bash
bunx smithers-orchestrator workflow run eval-author \
  --input '{"prompt":"Release notes must list every breaking change","workflow":".smithers/workflows/release.tsx"}'
```

## Stages

1. **derive**: turn the criteria into a structured suite, with a kebab-case `suiteName` and a list of cases (`id`, `input`, `expected`, `rubric`).
2. **write**: write the JSONL fixture to `.smithers/evals/<suiteName>.jsonl` and return its `path`, `caseCount`, and the `runCommand` (`bunx smithers-orchestrator eval <workflow> --cases .smithers/evals/<suiteName>.jsonl --suite <suiteName>`).

## Inputs

| Input | Type | Default |
| --- | --- | --- |
| `prompt` | string | `"Describe the acceptance criteria / goal to turn into eval cases."` |
| `workflow` | string or null | `null` (leaves a `<workflow>` placeholder in the run command) |

The fixture's assertions support `status`, `output` (exact match), and
`outputContains` (partial match). See [Recipes](/recipes#eval-suites-for-regressions)
for the eval-suite format and how reports land in `.smithers/evals/<suite>.json`.
