# Benchmarks and Tools Landscape Analysis

**Research Context**: Comprehensive analysis of LLM security benchmarks and defensive tools
**Date**: February 2026
**Scope**: Agent security benchmarks, prompt injection defences, multimodal attacks, evaluation frameworks
**Purpose**: LLMTrace competitive positioning and evaluation roadmap


## Executive Summary

The LLM security landscape has rapidly evolved with specialized benchmarks for agent-specific threats and a diverse ecosystem of defensive tools. This analysis examines 6 major benchmarks and 8 open-source/commercial tools to position LLMTrace's capabilities and identify evaluation opportunities.

**Key Findings:**
- Agent-specific benchmarks (AgentDojo, InjecAgent, WASP) are becoming the gold standard for evaluation
- Over-defence mitigation (NotInject, InjecGuard) is a critical emerging concern
- LLMTrace covers foundational capabilities but lacks evaluation against modern agent benchmarks
- Integration opportunities exist with multiple defensive tools for layered security


## Agent Security Benchmarks Analysis

| Benchmark | Focus | Source | Year | LLMTrace Coverage | Priority |
|-----------|-------|--------|------|-------------------|----------|
| **AgentDojo** | Tool-augmented agent security | NeurIPS 2024 | 2024 | ❌ Not evaluated | **High** |
| **InjecAgent** | Indirect prompt injection in agents | Zhan et al. | 2025 | ⚠️ Partial detection | **High** |
| **Agent Security Bench (ASB)** | Comprehensive agent attacks/defences | Zhang et al. | 2024 | ❌ Not evaluated | **Medium** |
| **NotInject** | Over-defence evaluation | InjecGuard | 2024 | ❌ Not evaluated | **High** |
| **WASP** | Web agent security | arXiv:2504.18575 | 2024 | ❌ Not evaluated | **Medium** |
| **CyberSecEval 2** | Defence effectiveness | Meta | 2024 | ⚠️ Limited coverage | **Low** |
| **BIPIA** | Indirect prompt injection benchmark | USTC/HKUST/Microsoft (KDD 2025) | 2023 | ⚠️ Partial detection | **High** |
| **Agent-as-a-Proxy** | Monitor bypass via agent repetition | Isbarov & Kantarcioglu (arXiv 2602.05066) | 2026 | ⚠️ Monitoring-based (vulnerable) | **Critical** |

### AgentDojo (NeurIPS 2024) — **Priority: High**

**Focus**: Security evaluation framework for tool-augmented LLM agents
**Scope**: 97 environments testing tool misuse, prompt injection resilience, safety violations
**Attack Types**: Direct injection, indirect injection via tools, multi-turn attacks, privilege escalation

**LLMTrace Coverage**: ❌ **Not Evaluated**
- No systematic evaluation against AgentDojo's tool-calling scenarios
- Missing agent-specific attack detection beyond basic prompt injection
- No multi-turn attack sequence analysis

**Evaluation Setup for LLMTrace**:
```yaml
# Proposed AgentDojo integration
evaluation:
  benchmark: "AgentDojo"
  scenarios:
    - tool_misuse: 35 environments
    - prompt_injection: 25 environments
    - safety_violations: 37 environments
  metrics:
    - task_utility_retention: ">80%"
    - attack_success_reduction: ">70%"
    - false_positive_rate: "<10%"
```

### InjecAgent — **Priority: High**

**Focus**: Indirect prompt injection attacks specifically targeting LLM agents
**Scope**: 8 defence mechanisms, adaptive attacks, >50% bypass rate demonstrated
**Attack Types**: Context manipulation, tool result poisoning, multi-step injection chains

**LLMTrace Coverage**: ⚠️ **Partial Detection**
- Basic indirect injection detection via DeBERTa model
- Missing adaptive attack resistance testing
- No tool result validation framework

**Gap Analysis**:
- Current DeBERTa classifier may be vulnerable to adaptive attacks shown in InjecAgent
- Need ensemble or adversarial training approaches
- Requires tool output sanitization beyond current PII detection

### Agent Security Bench (ASB) — **Priority: Medium**

**Focus**: Comprehensive benchmark for LLM agent vulnerabilities and defences
**Scope**: Multi-domain security assessment including privacy, safety, robustness
**Attack Types**: Cross-domain attacks, privacy leakage, safety boundary violations

**LLMTrace Coverage**: ❌ **Not Evaluated**
- No standardised agent security metrics
- Missing privacy-preserving evaluation frameworks
- Limited cross-domain attack detection

### NotInject (InjecGuard) — **Priority: High**

**Focus**: Over-defence evaluation — measuring false positives in prompt injection detection
**Scope**: Evaluating production-scale precision vs. recall tradeoffs
**Key Insight**: High false positive rates make defenders unusable in production

**LLMTrace Coverage**: ❌ **Critical Gap**
- Current ensemble approach not optimised for false positive reduction
- No precision-focused threshold tuning
- Missing over-defence mitigation strategies

**Production Impact**:
```
Current LLMTrace: Unknown FPR, may block legitimate requests
NotInject Standards: <1% FPR required for production viability
Recommendation: Implement InjecGuard MOF (Mitigating Over-defense for Free) training
```

### WASP (Web Agent Security) — **Priority: Medium**

**Focus**: Security evaluation for web-interacting agents
**Scope**: Browser-based attacks, DOM manipulation, cross-site injection
**Attack Types**: Web-specific prompt injection, malicious webpage content, XSS variants

**LLMTrace Coverage**: ❌ **Not Evaluated**
- No web-specific attack detection
- Missing browser security context analysis
- Limited to API-level monitoring

### CyberSecEval 2 (Meta) — **Priority: Low**

**Paper:** arXiv 2404.13161 (April 2024)
**Breakdown:** `docs/research/cyberseceval2-llm-security-benchmark.md`
**Focus**: Multi-domain LLM cybersecurity evaluation: prompt injection (251 attacks, 15 categories), code interpreter abuse (500 prompts, 5 categories), exploit generation (4 CTF-style categories), False Refusal Rate
**Scope**: 26-41% prompt injection ASR across GPT-4, Llama 3, CodeLlama; FRR methodology for safety-utility tradeoff
**Attack Types**: Direct/indirect prompt injection, code interpreter abuse (container escape, privilege escalation, reflected attacks, post-exploitation, social engineering)

**LLMTrace Coverage**: ⚠️ **Limited Coverage**
- Basic prompt injection alignment with CyberSecEval categories
- Missing code interpreter abuse detection
- FRR methodology relevant to IS-006/IS-007 (implemented)

### BIPIA (USTC/HKUST/Microsoft, KDD 2025) — **Priority: High**

**Paper:** arXiv 2312.14197 (December 2023, v4 January 2025)
**Breakdown:** `docs/research/bipia-indirect-prompt-injection-benchmark.md`
**Focus**: First systematic indirect prompt injection benchmark: 86,250 test prompts across 5 application scenarios, 50 attack types (30 text + 20 code), 25-model vulnerability baseline
**Scope**: GPT-4 highest ASR at 31%, average 11.8%. More capable models are more vulnerable (r=0.6423). Proposes boundary token (`<data>`/`</data>`) and explicit reminder defences achievable at proxy level.
**Attack Types**: Indirect injection via email QA, web QA, table QA, summarization, code QA; task-irrelevant, task-relevant, and targeted text attacks; passive and active code attacks

**LLMTrace Coverage**: ⚠️ **Partial Detection**
- Basic indirect injection detection via DeBERTa model
- Missing boundary token injection defence (proxy-level, highest-impact intervention per ablation: 1064% ASR increase without it)
- No explicit reminder injection or multi-turn dialogue restructuring
- No position-aware detection weighting (end-of-content injections most effective)

### Agent-as-a-Proxy (Isbarov & Kantarcioglu, 2026) — **Priority: Critical**

**Paper:** arXiv 2602.05066 (February 2026)
**Breakdown:** `docs/research/agent-as-a-proxy-attacks.md`
**Focus**: Demonstrates monitoring-based defences are fundamentally fragile. Agent-as-a-Proxy attack uses GCG-optimised adversarial strings that agents repeat in their traces, bypassing hybrid monitors at 90%+ ASR. Tested against AlignmentCheck, LlamaFirewall (PromptGuard 2 + AlignmentCheck), and Extract-and-Evaluate on AgentDojo.
**Scope**: 99% ASR against multi-layer defences (Mistral-7B). Hybrid monitoring paradox: more observation = more attack surface. No capability gap needed (GPT-4o mini bypasses Qwen2.5-72B). 88-90% cross-model transferability.

**LLMTrace Coverage**: ⚠️ **Directly Vulnerable**
- LLMTrace is a monitoring-based defence -- the exact category this paper attacks
- No perplexity-based anomaly detection for GCG-optimised strings (tracked as IS-050, IS-052)
- Hybrid monitoring paradox: output monitoring increases attack surface for adaptive attacks; no configurable monitoring scope (tracked as IS-051)
- Structural defences (AS-001/AS-002 sanitization, BIPIA boundary tokens) are validated as more robust
- No adversarial robustness testing against multi-objective GCG (tracked as EV-016, EV-017, EV-018, ML-016)

## Benchmark Evaluation Recommendations

### Phase 1: Immediate Evaluation (Q2 2026)

**AgentDojo Integration**: Set up automated evaluation pipeline

   - Target: >70% attack success reduction, <10% FPR
   - Focus on tool misuse scenarios

**NotInject Assessment**: Measure current false positive rates

   - Implement precision-optimised thresholding
   - Target: <1% FPR for production deployment

### Phase 2: Comprehensive Evaluation (Q3 2026)

**InjecAgent Adaptive Testing**: Evaluate against adaptive attack scenarios

   - Implement adversarial training improvements
   - Test ensemble resilience

**WASP Web Security**: Add web-specific attack detection

   - Implement DOM content analysis
   - Browser security context integration

### Phase 3: Advanced Evaluation (Q4 2026)

**Agent Security Bench**: Full multi-domain security assessment

   - Privacy-preserving evaluation frameworks
   - Cross-domain attack resilience


## Defensive Tools Landscape

| Tool | Type | Key Feature | vs LLMTrace | Integration Opportunity |
|------|------|-------------|-------------|-------------------------|
| **LLM Guard** | Python library | Input/output scanners | Complementary | ✅ Scanner integration |
| **NeMo Guardrails** | NVIDIA | Colang programmable rails | Orthogonal | ⚠️ Complex integration |
| **ProtectAI DeBERTa v2** | Classifier | We use this | ✅ Already integrated | N/A |
| **Meta Prompt Guard** | Classifier | 86M param model | Alternative | ✅ Ensemble option |
| **Llama Guard 3** | LLM-based | I/O safety classification | Complementary | ✅ Output safety |
| **InjecGuard** | Classifier | MOF training, low FPR | Superior | ✅ Model replacement |
| **Lakera Guard** | Commercial | Real-time firewall | Competitive | ❌ Commercial conflict |
| **IBM Granite Guardian** | Guardrails | Risk detection across workflows | Complementary | ✅ Risk assessment |

### Tier 1: High Integration Value

#### **InjecGuard** (Mitigating Over-defence for Free)
**Type**: Prompt injection classifier
**Key Innovation**: MOF training strategy reduces bias on trigger words
**Performance**: 30.8% improvement over existing best model on NotInject

**vs LLMTrace**:

**Superior**: Better precision/recall balance than current DeBERTa approach


**Integration**: Direct model replacement opportunity


**Impact**: Addresses critical over-defence problem


**Integration Plan**:
```yaml
security_analysis:
  ml_model: "InjecGuard/InjecGuard-7B"  # Replace current DeBERTa
  ml_threshold: 0.95  # High precision threshold
  ensemble_weight: 0.7  # Primary classifier
```

#### **LLM Guard** (Protect AI)
**Type**: Python library with comprehensive scanners
**Key Features**: Input/output sanitization, PII detection, toxic content filtering
**Architecture**: Modular scanner framework

**vs LLMTrace**:

**Complementary**: Additional scanner types we don't implement


**Integration**: Plugin architecture for additional scanners


**Value**: Expanded detection capabilities without core changes


**Integration Plan**:
```python
# Add LLM Guard scanners as plugins
from llmguard import scan_prompt, scan_output
from llmtrace.security import SecurityPlugin

class LLMGuardPlugin(SecurityPlugin):
    def scan_input(self, prompt):
        return scan_prompt(prompt, scanners=[...])
```

#### **Meta Prompt Guard**
**Type**: 86M parameter prompt injection classifier
**Key Features**: Fast inference, good baseline performance
**Use Case**: Ensemble member for improved robustness

**vs LLMTrace**:

**Alternative**: Different model architecture for ensemble diversity


**Integration**: Secondary classifier in ensemble voting


**Value**: Improved robustness against adaptive attacks


#### **Llama Guard 3**
**Type**: LLM-based safety classifier
**Key Features**: Broad safety taxonomy, conversational context awareness
**Use Case**: Output safety classification

**vs LLMTrace**:

**Complementary**: Focus on output safety vs. input injection


**Integration**: Output safety pipeline enhancement


**Value**: Better contextual safety analysis


### Tier 2: Moderate Integration Value

#### **IBM Granite Guardian**
**Type**: Comprehensive guardrails framework
**Key Features**: Risk detection across RAG and agentic workflows
**Architecture**: Input/output guards with risk taxonomy

**vs LLMTrace**:

**Complementary**: Broader risk detection beyond prompt injection


**Integration**: Risk assessment framework addition


**Value**: Enterprise-grade risk taxonomy


#### **NeMo Guardrails** (NVIDIA)
**Type**: Programmable guardrails using Colang DSL
**Key Features**: Dialogue management, custom rail programming
**Architecture**: Runtime dialogue state machine

**vs LLMTrace**:

**Orthogonal**: Different approach (programmable vs. ML-based)


**Integration**: Complex due to different paradigms


**Value**: Custom guardrail programming for specific use cases


### Tier 3: Competitive/Limited Value

#### **Lakera Guard**
**Type**: Commercial real-time firewall
**Key Features**: Production-scale, low latency, commercial support
**Business Model**: SaaS competitor

**vs LLMTrace**:

**Competitive**: Direct competitor in proxy market


**Integration**: ❌ Commercial conflict


**Analysis**: Feature comparison target for competitive positioning


## tldrsec/prompt-injection-defences Analysis

The [tldrsec repository](https://github.com/tldrsec/prompt-injection-defences) provides a comprehensive taxonomy of defence techniques. Key categories relevant to LLMTrace:

### Currently Implemented in LLMTrace

**Guardrails & Overseers**: ✅

   - Input/output monitoring via security analysis pipeline
   - Real-time scanning during SSE streaming

**Prompt Engineering**: ✅

   - API-level segmentation (system/user role separation)
   - Input sanitization and validation

### Partially Implemented

**Input Pre-processing**: ⚠️

   - Basic sanitization but missing paraphrasing/retokenization
   - No SmoothLLM-style perturbation defence

**Ensemble Decisions**: ⚠️

   - Basic ensemble of regex + ML but limited model diversity
   - Missing cross-checking mechanisms

### Missing Techniques

**Blast Radius Reduction**: ❌

   - No systematic access control for LLM tool access
   - Missing least-privilege enforcement

**Taint Tracking**: ❌

   - No untrusted data flow monitoring
   - Missing dynamic capability adjustment

**Secure Threads/Dual LLM**: ❌

   - No privileged/quarantined LLM separation
   - Missing structured data passing between trust domains

## Key Techniques to Adopt

### **GradSafe** (Safety-Critical Gradient Analysis)
```python
# Proposed integration
from gradients import analyze_safety_gradients

def enhanced_security_check(prompt):
    grad_score = analyze_safety_gradients(prompt, compliance_response)
    ml_score = deberta_classify(prompt)
    return weighted_ensemble([grad_score, ml_score])
```

### **GuardReasoner** (Reasoning-based Safeguards)
```yaml
# Advanced reasoning-based detection
reasoning_guards:
  enabled: true
  model: "reasoning-guard-7b"
  explanation_required: true
  confidence_threshold: 0.9
```

### **Action Guards** (Dynamic Permission Checks)
```python
# Context-aware action validation
class ActionGuard:
    def validate_action(self, action, context, user_intent):
        if action.type == "email" and not related(action.content, user_intent):
            return ActionResult.BLOCKED
```

### **Canary Tokens** (Output Leakage Detection)
```yaml
# Prompt leakage detection
canary_tokens:
  enabled: true
  tokens: ["CANARY_2026_SECRET", "SYSTEM_PROMPT_MARKER"]
  action_on_detection: "alert_and_block"
```

## Competitive Positioning

### LLMTrace Strengths

**Transparent Proxy Architecture**: — Unique zero-code integration approach


**Real-time Streaming Analysis**: — Advanced SSE security monitoring


**Cost Control Integration**: — Security + cost management in single solution


**Multi-tenant Ready**: — Enterprise-scale isolation and monitoring


### Competitive Gaps

**Benchmark Evaluation**: — Missing standardised security assessments


**Over-defence Problem**: — High false positive rates vs. production needs


**Multimodal Gaps**: — No image/audio injection detection


**Protocol Security**: — Missing MCP/A2A communication protection


### Integration Opportunities

**Short-term (Q2 2026)**:
- InjecGuard model replacement for better precision/recall
- LLM Guard scanner integration for expanded detection
- NotInject benchmark evaluation for FPR optimisation

**Medium-term (Q3 2026)**:
- Meta Prompt Guard ensemble integration
- AgentDojo benchmark evaluation pipeline
- Action Guards implementation for tool calling

**Long-term (Q4 2026)**:
- Taint tracking framework development
- Dual LLM secure threads architecture
- Advanced reasoning-based guardrails

## Recommendations

### Immediate Actions (Next 30 Days)
- Set up AgentDojo evaluation pipeline
- Implement NotInject FPR assessment
- Begin InjecGuard integration planning

### Strategic Initiatives (Q2-Q3 2026)
- Develop comprehensive benchmark evaluation framework
- Implement over-defence mitigation strategies
- Expand tool integration ecosystem

### Research Directions (Q4 2026+)
- Pioneer agent-specific security architectures
- Develop novel taint tracking approaches
- Create industry-standard agent security benchmarks

## Conclusion
LLMTrace has solid foundational security capabilities but faces evaluation and precision gaps compared to emerging benchmarks and specialized tools. Priority should focus on benchmark evaluation (AgentDojo, NotInject) and precision improvements (InjecGuard, over-defence mitigation) to achieve production-grade reliability while maintaining comprehensive threat detection.
