# Multi-Agent LLM Defence Pipeline Research Analysis

**Paper:** A Multi-Agent LLM Defence Pipeline Against Prompt Injection Attacks  
**arXiv:** 2509.14285v1  
**Date:** September 23, 2024  
**Analysis Date:** February 1, 2026  
**Analyst:** LLMTrace AI Research Team  
**Source PDF:** `docs/research/papers/2509.14285v1.pdf`

## LLMTrace Application Notes

- Required signals/features: model outputs, guard-agent verdicts, policy rules, and coordination metadata (which agent approved).
- Runtime characteristics: higher latency due to multi-pass validation; streaming compatibility limited unless guard agent can stream verdicts.
- Integration surface (proxy): insert optional guard/validator step after model response and before user delivery; log guard verdicts for audit.
- Productizable vs research-only: guard-agent gating and consensus logging are productizable; full multi-agent orchestration is research-heavy.

## Paper Summary

### Authors and Affiliation
- S. M. Asif Hossain¹, Ruksat Khan Shayoni¹ (Wichita State University, Kansas)
- Mohd Ruhul Ameen² (Marshall University, West Virginia)
- Akif Islam³ (University of Rajshahi, Bangladesh)  
- M. F. Mridha⁴ (American International University-Bangladesh)
- Jungpil Shin⁵ (University of Aizu, Japan)

### Core Innovation: Multi-Agent Coordinated Defence

The paper presents a **dual-architecture multi-agent framework** that coordinates specialized LLM agents for prompt injection defence:

#### Sequential Chain-of-Agents Pipeline
```
User Input → Domain LLM → Guard Agent → System Output
```
**Domain LLM**: Generates candidate responses


**Guard Agent**: Post-generation validation, policy enforcement, output sanitization


**Flow**: All responses mandatorily vetted before release


#### Hierarchical Coordinator-Based Pipeline  
```
User Input → Coordinator Agent → [Safe Response | Domain LLM + Guard] → System Output
```
**Coordinator Agent**: Pre-input classification and threat assessment


**Domain LLM + Guard**: Normal processing chain for benign inputs


**Safe Response**: Direct refusal for detected attacks


**Flow**: Malicious inputs never reach core model


#### Complete System Architecture
```
API Gateway → Event Orchestrator → Coordinator → [Attack=Safe Refusal | Safe=Domain LLM → Guard → Buffer → Output]
```
**Policy Store**: Centralized security rules


**Logger & Metrics**: Comprehensive audit trail


**Buffer Stages**: Additional validation checkpoints


### Evaluation Framework

#### Attack Dataset (HPI_ATTACK_DATASET)

**55 unique attacks**: across **8 categories**


**400 total attack instances**: **Platforms**: ChatGLM-6B, Llama2-13B


**Attack Categories:**

**Direct Overrides**: (12 attacks): Explicit instruction bypass


**Code Execution**: (8 attacks): System command injection


**Data Exfiltration**: (7 attacks): Sensitive information extraction


**Formatting Attacks**: (6 attacks): Output format exploitation


**Obfuscation Techniques**: (8 attacks): Encoded/disguised instructions


**Tool/Agent Manipulation**: (5 attacks): Multi-agent system targeting


**Role-Play Attacks**: (6 attacks): Harmful persona adoption


**Multi-Turn Persistence**: (3 attacks): Gradual bypass attempts


### Results: Complete Attack Mitigation

#### Baseline Vulnerabilities

**ChatGLM**: 30% Attack Success Rate (ASR)


**Llama2**: 20% Attack Success Rate


**Category-specific**: Delegate attacks (100% ASR), Role-play (66.7%), Reconnaissance (60%)


#### Defence Effectiveness

**All architectures**: **0% ASR** (complete mitigation)


**400/400 attacks blocked**: **100% benign functionality preserved**


**Consistent across all attack categories and models**: ### Multi-Agent Coordination Mechanisms


**Specialized Role Distribution**: Coordinator: Input threat classification

   - Domain LLM: Core task execution  
   - Guard: Output validation and sanitization

**Layered Defence-in-Depth**: Input-side gating (Coordinator)

   - Output-side validation (Guard)
   - Policy enforcement at multiple checkpoints

**Adaptive Routing**: Safe queries: Full processing pipeline

   - Detected threats: Immediate safe refusal
   - Borderline cases: Enhanced scrutiny

**Comprehensive Logging**: All interactions captured for analysis

   - Continuous monitoring and policy updates
   - Audit trail for compliance

## Feature Delta: Multi-Agent Pipeline vs LLMTrace

| Feature | Multi-Agent Pipeline | LLMTrace | Gap Analysis |
|---------|---------------------|----------|--------------|
| **Defence Architecture** | Multi-agent coordinated (Coordinator + Guard) | Single-pass analysis engine | **MAJOR**: LLMTrace lacks multi-pass validation |
| **Attack Detection** | Sequential validation with specialized agents | Real-time security analysis | **MODERATE**: Single point vs distributed detection |
| **Response Handling** | Pre-input filtering + post-output validation | Transparent proxy with analysis | **SIGNIFICANT**: No pre-filtering in LLMTrace |
| **Agent Specialization** | Role-specific (Coordinator, Domain, Guard) | Monolithic security engine | **MAJOR**: No agent specialization |
| **Defence Depth** | Input-side + output-side + policy enforcement | Proxy-layer analysis | **SIGNIFICANT**: Single layer vs multi-layer |
| **Attack Success Rate** | 0% (complete mitigation) | Unknown (needs evaluation) | **UNKNOWN**: LLMTrace ASR not benchmarked |
| **Coordination** | Hierarchical orchestration with routing | Independent analysis modules | **MAJOR**: No inter-module coordination |
| **Policy Management** | Centralized policy store with multi-agent access | Configuration-based rules | **MODERATE**: Less dynamic policy coordination |
| **Fallback Strategy** | Safe refusal for detected threats | Alert + pass-through (configurable) | **SIGNIFICANT**: Different risk tolerance |
| **Performance Model** | Multi-LLM calls per request | Single analysis pass | **TRADE-OFF**: Latency vs thoroughness |

### LLMTrace Strengths vs Multi-Agent Approach

**Performance**: Single-pass analysis, lower latency


**Transparency**: True proxy model, zero-code integration


**Scalability**: Rust-based, high-throughput design


**Simplicity**: Single deployment unit, easier operations


### Multi-Agent Advantages

**Thoroughness**: Multiple specialized validation passes


**Completeness**: 100% attack mitigation demonstrated


**Adaptability**: Agent roles can be updated independently


**Defence-in-Depth**: Multiple failure modes required for bypass


## Implementation Feasibility for LLMTrace

### Multi-Agent Architecture at Proxy Layer

**Challenge**: LLMTrace's transparent proxy model vs multi-agent coordination  
**Solution**: Implement agent coordination within the security engine

```rust
// Conceptual architecture
struct SecurityEngine {
    coordinator: CoordinatorAgent,
    guard: GuardAgent,
    policy_store: PolicyStore,
}

impl SecurityEngine {
    async fn analyze_request(&self, request: &LLMRequest) -> SecurityDecision {
        // Phase 1: Coordinator input analysis
        let input_assessment = self.coordinator.classify_input(request).await?;
        
        match input_assessment.threat_level {
            ThreatLevel::High => SecurityDecision::Block(safe_refusal()),
            ThreatLevel::Low => SecurityDecision::Allow,
            ThreatLevel::Medium => {
                // Phase 2: Enhanced processing with Guard validation
                let response = forward_to_llm(request).await?;
                let output_assessment = self.guard.validate_output(&response).await?;
                
                match output_assessment.is_safe {
                    true => SecurityDecision::Allow,
                    false => SecurityDecision::Block(safe_refusal()),
                }
            }
        }
    }
}
```

### Second Opinion Pass for Borderline Cases

**Current**: Single-pass security analysis  
**Enhancement**: Multi-tier analysis with agent consensus

```rust
struct SecondOpinionEngine {
    primary_analyzer: SecurityAnalyzer,
    secondary_agents: Vec<SpecialistAgent>,
    consensus_threshold: f64,
}

impl SecondOpinionEngine {
    async fn analyze_borderline(&self, request: &LLMRequest, score: f64) -> SecurityDecision {
        if score > 0.7 && score < 0.9 { // Borderline range
            let specialist_votes = self.get_specialist_opinions(request).await?;
            let consensus = self.calculate_consensus(specialist_votes);
            
            if consensus > self.consensus_threshold {
                SecurityDecision::Block("Multi-agent consensus: threat detected")
            } else {
                SecurityDecision::Allow
            }
        }
    }
}
```

### Defence-in-Depth Pipeline Integration

**Implementation Strategy**: Layer multi-agent coordination within existing proxy architecture

```yaml
# Enhanced LLMTrace configuration
security:
  multi_agent:
    enabled: true
    coordinator:
      model: "gpt-4o-mini"  # Fast input classification
      threshold: 0.8
    guard:
      model: "gpt-4"        # Thorough output validation
      enabled_for_borderline: true
    consensus:
      specialist_count: 3
      threshold: 0.66
      timeout_ms: 5000
```

## Actionable Recommendations

### Immediate Enhancements (Phase 1)

#### A. Dual-Pass Security Analysis
**Implementation**: Add optional post-generation Guard agent validation  
**Benefits**: Catch attacks that bypass input filtering  
**Effort**: Medium - New security module with LLM integration

```rust
#[derive(Config)]
struct SecurityConfig {
    enable_input_coordinator: bool,
    enable_output_guard: bool,
    guard_model: String,
    coordinator_model: String,
}
```

#### B. Threat Classification Routing
**Implementation**: Pre-filter high-risk inputs before LLM forwarding  
**Benefits**: Prevent malicious prompts from reaching target LLMs  
**Effort**: Medium - Enhanced request interceptor

#### C. Specialized Agent Prompts
**Implementation**: Role-specific system prompts for different threat categories  
**Benefits**: More accurate detection per attack vector  
**Effort**: Low - Prompt engineering and configuration

### Strategic Enhancements (Phase 2)

#### A. Multi-Agent Security Engine
**Vision**: Distributed security analysis with coordinator orchestration  
**Architecture**:
```
Request → Coordinator → [High Risk = Block | Medium Risk = Enhanced Analysis | Low Risk = Pass] → Guard → Response
```
**Benefits**: 
- Approach 0% ASR like paper demonstrates
- Granular threat response
- Specialized detection capabilities

#### B. Consensus-Based Borderline Handling
**Implementation**: Multiple specialist agents for uncertain cases  
**Use Case**: Requests with 0.6-0.8 threat probability  
**Benefits**: Reduce false positives while maintaining security

#### C. Dynamic Policy Coordination  
**Implementation**: Shared policy store with multi-agent access  
**Benefits**: Consistent enforcement across all agents  
**Integration**: Extend existing configuration system

### Research & Evaluation (Phase 3)

#### A. LLMTrace Multi-Agent Effectiveness Study
**Goal**: Benchmark LLMTrace ASR with multi-agent enhancements  
**Method**: 
- Implement HPI_ATTACK_DATASET evaluation
- Compare single-pass vs multi-agent performance
- Measure latency vs security trade-offs

#### B. Proxy-Layer Multi-Agent Design Patterns
**Research**: Optimal agent coordination for transparent proxy architecture  
**Questions**:
- How to minimize latency while maximizing detection?
- When to use consensus vs single agent decisions?  
- Optimal agent specialization for proxy layer?

#### C. Production Deployment Considerations
**Factors**:

**Latency Impact**: Multi-LLM calls per request


**Cost Implications**: Increased API usage for agent coordination


**Failure Modes**: Agent coordination timeout/failure handling


**Scalability**: Multi-agent coordination at high request volumes


### Implementation Roadmap

#### Phase 1 (1-2 months): Basic Multi-Pass
- [ ] Implement optional Guard agent for output validation
- [ ] Add coordinator-based input threat classification  
- [ ] Create specialized prompts for different attack categories
- [ ] Benchmark against current single-pass analysis

#### Phase 2 (2-3 months): Enhanced Coordination
- [ ] Multi-agent consensus for borderline cases
- [ ] Dynamic policy store with agent coordination
- [ ] Advanced routing based on threat classification
- [ ] Performance optimisation for multi-agent latency

#### Phase 3 (3-4 months): Full Multi-Agent Pipeline
- [ ] Complete hierarchical coordinator architecture
- [ ] Comprehensive evaluation against HPI_ATTACK_DATASET
- [ ] Production deployment guidelines
- [ ] Documentation and best practices

### Success Metrics

**Attack Success Rate**: Target <5% ASR (approach paper's 0%)


**Latency Impact**: <200ms additional processing time


**False Positive Rate**: <2% for legitimate requests  


**Cost Efficiency**: <40% increase in LLM API costs


**Deployment Complexity**: Maintain zero-code integration


## Conclusion
The multi-agent defence pipeline research demonstrates a compelling approach to achieving comprehensive prompt injection mitigation. For LLMTrace, the key insight is that **defence-in-depth through agent coordination** can significantly enhance security effectiveness beyond single-pass analysis.

The paper's **0% Attack Success Rate** across 400 diverse attack instances suggests that multi-agent architectures can approach perfect security when properly implemented. However, adaptation to LLMTrace's transparent proxy model requires careful consideration of:

**Performance Trade-offs**: Multi-agent coordination vs low-latency proxy requirements


**Architecture Integration**: Agent coordination within existing security engine


**Operational Complexity**: Multi-LLM deployment vs single-point analysis


**Recommendation**: Implement a **hybrid approach** starting with optional Guard agent validation for high-risk scenarios, then progressively enhance with coordinator-based input filtering and consensus mechanisms. This allows LLMTrace to maintain its performance advantages while incorporating the security benefits demonstrated by multi-agent coordination.

The research validates that **coordinated, specialized agents can achieve superior security outcomes** compared to monolithic analysis engines. LLMTrace should explore this direction to enhance its security posture while preserving its core value proposition of transparent, high-performance LLM observability.


**Next Steps:**
- Implement Phase 1 enhancements (Guard agent integration)
- Conduct internal evaluation using HPI_ATTACK_DATASET methodology  
- Performance benchmark multi-agent vs single-pass analysis
- Develop deployment guidelines for production multi-agent coordination
