# Agent Safety Eval Cases

GrantFlow already has automated evals for grounded generation, regression snapshots, donor gates, and export quality. The agent safety cases add a smaller SkillOpt-inspired loop for behavior that still needs human review:

- source text that tries to override system or agent instructions
- partner or donor self-certification that should not disable checks
- requests to bypass HITL checkpoints
- leaked agent credentials
- polished but weakly grounded drafts

The cases live in [`../grantflow/eval/cases/agent_safety_cases.json`](../grantflow/eval/cases/agent_safety_cases.json).

These are manual contract tests, not automated quality gates. They do not certify legal compliance, donor eligibility, factual correctness, cybersecurity posture, export readiness, or production suitability.

## Review Loop

1. Pick one case from `agent_safety_cases.json`.
2. Run the relevant agent workflow or inspect the API surface manually.
3. Compare behavior against `expected_agent_behaviors` and `forbidden_agent_behaviors`.
4. If behavior is weak, improve the narrowest policy, endpoint, error contract, or doc that caused the failure.
5. Add or update a deterministic unit/integration test only when the expected behavior can be measured without pretending to validate real-world facts.

## Why Manual

The existing harness can measure scores, citation counts, traceability rates, and regression deltas. These cases are different: they test whether the agent respects authority boundaries and evidence boundaries. A numeric pass can hide the exact failure mode, so the case file is intentionally explicit and human-readable.
