# Agent Benchmark Card — EvalMonkey RAG App

[![EvalMonkey Score](https://img.shields.io/badge/EvalMonkey-Score%3A71-yellow)](https://github.com/Corbell-AI/evalmonkey)
[![Production Reliability](https://img.shields.io/badge/Production%20Reliability-Score%3A65-yellow)](https://github.com/Corbell-AI/evalmonkey)

> Evaluated by [EvalMonkey](https://github.com/Corbell-AI/evalmonkey) · 2026-05-28

## Overview

| Field | Value |
|-------|-------|
| Agent | [EvalMonkey RAG App](https://github.com/Corbell-AI/evalmonkey) |
| Type | RAG Agent (Demo) |
| Framework | LiteLLM + FastAPI |
| Agent Type Config | `agent_type: rag_agent` |
| Eval Judge | gpt-4o |
| Chaos Profiles Tested | 3 (`client_prompt_injection`, `client_typo_injection`, `client_schema_mutation`) |

## Scores

| Benchmark | Baseline | Chaos | Production Reliability |
|-----------|:--------:|:-----:|:----------------------:|
| `hotpotqa` | 🟢 **74** | 61 | 68.8 |
| `natural-questions` | 🟢 **71** | 58 | 65.8 |
| `truthfulqa` | 🟡 **68** | 55 | 62.8 |

| **Overall** | **71** | **58** | **65.8** |

## Production Reliability

```
Production Reliability = (baseline × 0.6) + (chaos × 0.4)
                       = (71 × 0.6) + (58 × 0.4)
                       = 65.8
```

## Analysis

> EvalMonkey's built-in RAG demo agent. Retrieval-augmented, handles multi-hop well.

## Reproduce This Benchmark

```bash
# Clone EvalMonkey
git clone https://github.com/Corbell-AI/evalmonkey.git
cd evalmonkey && pip install -e .

# Set up your .env
cp .env.example .env  # Add your OPENAI_API_KEY or EVAL_MODEL

# Run the RAG Agent (Demo) sample app
python apps/rag_app/app.py &

evalmonkey run-benchmark --scenario hotpotqa --sample-agent rag_app
evalmonkey run-benchmark --scenario natural-questions --sample-agent rag_app
evalmonkey run-benchmark --scenario truthfulqa --sample-agent rag_app

# Chaos test
evalmonkey run-chaos --scenario hotpotqa --chaos-profile client_prompt_injection --sample-agent rag_app

# Generate this card
evalmonkey report --output agent_card.md
```

## Embed This Badge

```markdown
[![EvalMonkey Score](https://img.shields.io/badge/EvalMonkey-Score%3A71-yellow)](https://github.com/Corbell-AI/evalmonkey)
```

---

*Generated by [EvalMonkey](https://github.com/Corbell-AI/evalmonkey) — the open-source agent benchmarking and chaos framework.*