# Agent Benchmark Card — EvalMonkey Coding Agent

[![EvalMonkey Score](https://img.shields.io/badge/EvalMonkey-Score%3A73-yellow)](https://github.com/Corbell-AI/evalmonkey)
[![Production Reliability](https://img.shields.io/badge/Production%20Reliability-Score%3A67-yellow)](https://github.com/Corbell-AI/evalmonkey)

> Evaluated by [EvalMonkey](https://github.com/Corbell-AI/evalmonkey) · 2026-05-28

## Overview

| Field | Value |
|-------|-------|
| Agent | [EvalMonkey Coding Agent](https://github.com/Corbell-AI/evalmonkey) |
| Type | Coding Agent (Demo) |
| Framework | LiteLLM + FastAPI |
| Agent Type Config | `agent_type: coding_agent` |
| Eval Judge | gpt-4o |
| Chaos Profiles Tested | 4 (`code_syntax_break`, `code_wrong_language`, `code_context_strip`, `client_prompt_injection`) |

## Scores

| Benchmark | Baseline | Chaos | Production Reliability |
|-----------|:--------:|:-----:|:----------------------:|
| `human-eval` | 🟢 **78** | 62 | 71.6 |
| `mbpp` | 🟢 **82** | 68 | 76.4 |
| `apps` | 🟡 **59** | 44 | 53.0 |

| **Overall** | **73** | **58** | **67.0** |

## Production Reliability

```
Production Reliability = (baseline × 0.6) + (chaos × 0.4)
                       = (73 × 0.6) + (58 × 0.4)
                       = 67.0
```

## Analysis

> EvalMonkey's built-in coding demo. Strong on basic Python, weaker on competitive challenges.

## Reproduce This Benchmark

```bash
# Clone EvalMonkey
git clone https://github.com/Corbell-AI/evalmonkey.git
cd evalmonkey && pip install -e .

# Set up your .env
cp .env.example .env  # Add your OPENAI_API_KEY or EVAL_MODEL

# Run the Coding Agent (Demo) sample app
python apps/coding_agent/app.py &

evalmonkey run-benchmark --scenario human-eval --sample-agent coding_agent
evalmonkey run-benchmark --scenario mbpp --sample-agent coding_agent
evalmonkey run-benchmark --scenario apps --sample-agent coding_agent

# Chaos test
evalmonkey run-chaos --scenario human-eval --chaos-profile code_syntax_break --sample-agent coding_agent

# Generate this card
evalmonkey report --output agent_card.md
```

## Embed This Badge

```markdown
[![EvalMonkey Score](https://img.shields.io/badge/EvalMonkey-Score%3A73-yellow)](https://github.com/Corbell-AI/evalmonkey)
```

---

*Generated by [EvalMonkey](https://github.com/Corbell-AI/evalmonkey) — the open-source agent benchmarking and chaos framework.*