# @smithers-orchestrator/scorers — src

Scorer contract, built-in scorers, execution, and aggregation for Smithers runs.

- **Contract & factories** — `types.ts` (`Scorer`, `ScoreResult`, `ScorerBinding`,
  `ScorersMap`), `createScorer.js`, and `llmJudge.js` (LLM-as-judge factory with
  balanced-brace JSON extraction).
- **Built-in scorers** — LLM-judge: `relevancyScorer`, `toxicityScorer`,
  `faithfulnessScorer`, `tierFitScorer`; deterministic: `schemaAdherenceScorer`,
  `latencyScorer`.
- **Delegation-chain family** — `pocJudgmentScorer`, `planSolidityScorer`,
  `estimateAccuracyScorer`, `humanPollScorer`, with `delegationEvents.js`
  extracting the event log / planning nodes from scorer input.
- **Execution** — `run-scorers.js`: `runScorersAsync` (fire-and-forget via
  `Effect.runFork`, used live by `packages/engine`) and `runScorersBatch`
  (awaited, for evals/tests). Results are validated and clamped to [0, 1],
  emitted as `Scorer*` events, and persisted to `_smithers_scorers`.
- **Aggregation** — `aggregate.js`: `aggregateScores` (per-scorer stats over the
  table), `weightedScore`, and `delegationRunScore` (the five delegation scorers
  combined 0.25/0.25/0.15/0.15/0.2).

Conventions: a not-applicable scorer no-ops with `{ score: 1, meta: { skipped: true } }`
so it never dilutes weighted totals. Ratio sampling hashes
(runId, nodeId, iteration, attempt, scorerId) with a NUL-joined seed, so replays
and forks make identical run/skip decisions.

Gotchas: implementations are `.js` with JSDoc; PascalCase `.ts` files are
type-only sidecars. `index.d.ts` is generated by `pnpm build` (tsup) — never
hand-edit. The `@smithers-type-exports` blocks in `.js` files are tool-managed.
Every file here is public via the package's `./*` subpath export, so nothing is
private to the package; `metrics.js` and `schema.js` are subpath-export shims
over observability/db.
